Skip to main content

Emotion-reactive brain canvas for EXEPERT Chat

· 3 min read
EXEPERT Team
Brain Research Visualizer

EXEPERT Chat now drives a synthetic affect layer that colors and pulses the brain canvas while a user chats with an AI model. The feature is designed for AI emotion research UX: it visualizes conversational affect signals, not clinical or literal emotion detection.

What changed​

  • src/affect/* adds the affect engine: shared types, the emotion palette, the deterministic local classifier, text hashing, affect-to-region mapping, and the browser client for saving affect annotations.
  • src/brain-ui.ts classifies user input immediately, classifies assistant output after streaming, renders message affect chips, updates the new Affective Field, and triggers emotion-colored neural pulses. Chat error states can also become assistant_error affect events when the failed span is available.
  • index.html adds the Affective Field readout inside the Chat tile and the canvas-stage affect overlay.
  • css/app.css styles the Affective Field, message chips, stage overlay, and responsive narrow-panel behavior.
  • rust/brain_sim/src/lib.rs extends the WASM signal pool with per-signal RGB color buffers. Stimuli can now pass a hex color, and signals propagate that color through neuron firing.
  • src/brain.ts binds the WASM particle color buffer to a Three.js color geometry attribute so simultaneous signals can render different emotion colors in the same canvas.
  • supabase/functions/chat-affect/index.ts adds an authenticated Edge Function that writes affective_state span annotations. It always stores the local CODE classifier output and can also store an LLM-refined annotation through 9Router.
  • supabase/functions/run-chat/index.ts now includes trace_id, span_id, otel_trace_id, and otel_span_id on streaming error payloads when the error span was persisted successfully.

Affect model​

The local classifier scores:

  • anger
  • fear
  • sadness
  • joy
  • disgust
  • surprise
  • curiosity
  • calm
  • distress
  • neutral

Profanity is context-aware. For example, "fuck yeah" routes toward high-arousal joy, while directed hostile profanity routes toward anger, distress, and higher toxicity. The UI can look dramatic, but docs and metadata call the output synthetic affect telemetry.

Persistence​

chat-affect writes to existing span_annotations rather than adding a new table:

  • name = "affective_state"
  • annotator_kind = "CODE" for local analysis
  • annotator_kind = "LLM" for optional refinement
  • identifier = chat-turn:<turn_id>:phase:<phase>:source:<source>

Metadata stores score vectors, region mix, color palette, classifier version, selected chat model, affect model, phase, and text hash. Raw message text is not duplicated in affect metadata.

Verification​

Checked locally with:

pnpm run typecheck
pnpm test
pnpm build
pnpm --dir docs-site build

A browser smoke check opened Chat at http://127.0.0.1:9001/, verified the Affective Field on desktop and narrow viewports, confirmed no Chat overflow, and confirmed the WebGL brain canvas screenshot was nonblank.