Replace the three-dot typing indicator with live activity feedback (Evaluating / Searching / Refining) #96

New Issue

2026-08-02T06:47:52-07:00

westfarn commented

2026-08-02 06:47:52 -07:00

Problem

When a prompt is sent, the only feedback is three bouncing dots. The user cannot tell whether the assistant is thinking, searching the web, reading their documents, or stuck. On a fast reply this is fine; on a slow one it looks frozen, and once the agentic work in chat_backend#63 lands — where a task can legitimately run for minutes — it becomes unusable.

We want the Gemini/ChatGPT pattern: a short, honest, changing status line describing what is actually happening right now.

Current state

The indicator is rendered inline in ConversationDetailCard, triggered purely by an empty message string:

// src/llm-fe/components/ConversationDetailCard/ConversationDetailCard.tsx:160-171
if (message.length === 0) {
  return (
    <MessageContainer $isUser={false}>
      <Bubble $isUser={false}>
        <LoadingContainer>
          <LoadingDot />
          <LoadingDot />
          <LoadingDot />
        </LoadingContainer>
      </Bubble>
    </MessageContainer>
  );
}

The placeholder is created optimistically on submit (AsyncDashboard2.tsx:387-390) as a ConversationPrompt with message: "". Dots show until the first token arrives, at which point stateMessage becomes non-empty and text replaces them.

MessageContext already tracks isGeneratingMessage / isGeneratingRef (MessageContext.tsx:47-54, 168-180) but AsyncDashboard2 never reads it. There is a phase counter, messageResponsePart, driven by sentinel strings (MessageContext.tsx:152-194):

Value	Set by
`1`	`CONVERSATION_ID`
`2`	`START_OF_THE_STREAM_ENDER_GAME_42`
`0`	`END_OF_THE_STREAM_ENDER_GAME_42`

So the frontend has no idea what the backend is doing between "connected" and "first token" — and the backend never tells it.

Dependency

This needs backend cooperation. The status frames must come from the server, not be faked on a timer — a fabricated "Searching the web…" shown while the backend skipped search would be worse than the dots.

Coordinate the frame protocol with chat_backend#62 (citation frames) and chat_backend#63 (agent step frames). All three add frames to the same WebSocket stream. Agree one versioned envelope rather than adding more bare sentinel strings:

{"v": 1, "type": "status", "data": {"stage": "searching", "label": "Searching the web", "detail": "Taylor Swift Travis Kelce wedding"}}

The frontend must ignore unknown type values and unknown stage values so backend and frontend can deploy independently.

Proposed implementation

1. Parse structured frames in `MessageContext`

In the onmessage handler (MessageContext.tsx:146-200), attempt a JSON parse before the existing sentinel-string checks. If the payload is a versioned envelope, route it by type; otherwise fall through to today's logic unchanged. Existing behaviour must be untouched when the backend sends no frames.

2. Add status state

Expose from MessageContext:

activityStage: string | null — current stage key
activityLabel: string | null — human-readable label
activityDetail: string | null — optional sub-text (the search query, document name, etc.)
activityHistory: {stage, label, startedAt, finishedAt}[] — completed stages for this turn

Clear all of it on END_OF_THE_STREAM_ENDER_GAME_42, on error, and when a new prompt is submitted.

3. Stage vocabulary

Start with a set that maps to what the backend actually does today and after #62:

Stage	Label	Emitted when
`queued`	Getting started	Message accepted
`moderating`	Checking your request	Moderation classifier running
`evaluating`	Evaluating the question	Grounding decision (#62)
`searching`	Searching the web	Search provider called; `detail` = the query
`reading_sources`	Reading sources	Ranking/extracting results
`retrieving_docs`	Searching your documents	Chroma retrieval
`analysing`	Analysing your file	Data-analysis path
`refining`	Refining the answer	Final generation begins
`writing`	(no label — tokens are streaming)	First token

The frontend must render an unknown stage's label verbatim rather than dropping it, so chat_backend#63 can add agent stages without a frontend release.

4. New `ActivityIndicator` component

Replace the three-dot block in ConversationDetailCard. Requirements:

Show current label with a subtle animated affordance (spinner or shimmer), plus detail as smaller secondary text when present.
Show completed stages for the turn collapsed above the current one, each with a check — this is the "rolling up" quality the user is after.
Animate label transitions; never flash. Enforce a minimum display time of ~400ms per stage so fast backends don't produce a strobe.
Fall back to today's three dots when activityStage is null — covers an older backend and the gap before the first frame.
Theme-aware (light/dark), matching the existing agent bubble styling.
Mobile-safe: truncate detail with ellipsis, never widen the bubble past the viewport.

5. Handle long-running and stalled turns

Show elapsed time once a turn exceeds ~10s.
If no frame and no token arrive for ~30s, show a soft "still working…" note rather than leaving a frozen label.
Keep working with the existing streamInterrupted banner (AsyncDashboard2.tsx:542-551); the activity indicator must clear when that fires.

6. Accessibility

role="status" with aria-live="polite" so screen readers announce stage changes.
Respect prefers-reduced-motion: no spinner or shimmer animation, just the text.

7. Analytics

Add events per ANALYTICS.md conventions for stage durations, so we can measure where slow turns actually spend their time.

Acceptance criteria

MessageContext parses versioned JSON frames and ignores unknown type and unknown stage values without breaking the stream.
Sentinel-string handling (CONVERSATION_ID, START_OF_THE_STREAM_ENDER_GAME_42, END_OF_THE_STREAM_ENDER_GAME_42) still works unchanged.
With a backend that emits no status frames, behaviour is identical to today — three dots, then text. Covered by test.
activityStage, activityLabel, activityDetail, and activityHistory are exposed and cleared on completion, on error, and on new submit.
ActivityIndicator replaces the inline three-dot block in ConversationDetailCard and shows the current label plus optional detail.
Completed stages for the current turn render collapsed above the active one with a completion mark.
An unknown stage renders its server-supplied label verbatim rather than being dropped. Covered by test.
Each stage is displayed for a minimum of ~400ms; rapid stage sequences do not strobe.
Elapsed time appears after ~10s; a "still working" note appears after ~30s of silence.
Indicator clears when streamInterrupted fires and does not linger.
role="status" / aria-live="polite" announce changes; prefers-reduced-motion disables animation.
Correct in both light and dark themes; no horizontal overflow at 320px width.
Verified on Android via Capacitor.
The frame envelope matches what chat_backend#62 and #63 emit; no conflicting type values across the three.
Unit tests cover the full frame sequence, the no-frame fallback, unknown stages, and cleanup on error.

Out of scope

Nested sub-agent step display — arrives with chat_backend#63; the unknown-stage passthrough is what keeps this issue forward-compatible with it.
Rendering citations — separate work, same frame protocol.

## Problem When a prompt is sent, the only feedback is three bouncing dots. The user cannot tell whether the assistant is thinking, searching the web, reading their documents, or stuck. On a fast reply this is fine; on a slow one it looks frozen, and once the agentic work in `chat_backend`#63 lands — where a task can legitimately run for minutes — it becomes unusable. We want the Gemini/ChatGPT pattern: a short, honest, changing status line describing what is actually happening right now. ## Current state The indicator is rendered inline in `ConversationDetailCard`, triggered purely by an empty message string: ```tsx // src/llm-fe/components/ConversationDetailCard/ConversationDetailCard.tsx:160-171 if (message.length === 0) { return ( <MessageContainer $isUser={false}> <Bubble $isUser={false}> <LoadingContainer> <LoadingDot /> <LoadingDot /> <LoadingDot /> </LoadingContainer> </Bubble> </MessageContainer> ); } ``` The placeholder is created optimistically on submit (`AsyncDashboard2.tsx:387-390`) as a `ConversationPrompt` with `message: ""`. Dots show until the first token arrives, at which point `stateMessage` becomes non-empty and text replaces them. `MessageContext` already tracks `isGeneratingMessage` / `isGeneratingRef` (`MessageContext.tsx:47-54`, `168-180`) but **`AsyncDashboard2` never reads it**. There is a phase counter, `messageResponsePart`, driven by sentinel strings (`MessageContext.tsx:152-194`): | Value | Set by | |---|---| | `1` | `CONVERSATION_ID` | | `2` | `START_OF_THE_STREAM_ENDER_GAME_42` | | `0` | `END_OF_THE_STREAM_ENDER_GAME_42` | So the frontend has no idea what the backend is doing between "connected" and "first token" — and the backend never tells it. ## Dependency This needs backend cooperation. The status frames must come from the server, not be faked on a timer — a fabricated "Searching the web…" shown while the backend skipped search would be worse than the dots. Coordinate the frame protocol with `chat_backend`#62 (citation frames) and `chat_backend`#63 (agent step frames). All three add frames to the same WebSocket stream. Agree **one** versioned envelope rather than adding more bare sentinel strings: ```json {"v": 1, "type": "status", "data": {"stage": "searching", "label": "Searching the web", "detail": "Taylor Swift Travis Kelce wedding"}} ``` The frontend must ignore unknown `type` values and unknown `stage` values so backend and frontend can deploy independently. ## Proposed implementation ### 1. Parse structured frames in `MessageContext` In the `onmessage` handler (`MessageContext.tsx:146-200`), attempt a JSON parse before the existing sentinel-string checks. If the payload is a versioned envelope, route it by `type`; otherwise fall through to today's logic unchanged. Existing behaviour must be untouched when the backend sends no frames. ### 2. Add status state Expose from `MessageContext`: - `activityStage: string | null` — current stage key - `activityLabel: string | null` — human-readable label - `activityDetail: string | null` — optional sub-text (the search query, document name, etc.) - `activityHistory: {stage, label, startedAt, finishedAt}[]` — completed stages for this turn Clear all of it on `END_OF_THE_STREAM_ENDER_GAME_42`, on error, and when a new prompt is submitted. ### 3. Stage vocabulary Start with a set that maps to what the backend actually does today and after #62: | Stage | Label | Emitted when | |---|---|---| | `queued` | Getting started | Message accepted | | `moderating` | Checking your request | Moderation classifier running | | `evaluating` | Evaluating the question | Grounding decision (#62) | | `searching` | Searching the web | Search provider called; `detail` = the query | | `reading_sources` | Reading sources | Ranking/extracting results | | `retrieving_docs` | Searching your documents | Chroma retrieval | | `analysing` | Analysing your file | Data-analysis path | | `refining` | Refining the answer | Final generation begins | | `writing` | *(no label — tokens are streaming)* | First token | The frontend must render an unknown stage's `label` verbatim rather than dropping it, so `chat_backend`#63 can add agent stages without a frontend release. ### 4. New `ActivityIndicator` component Replace the three-dot block in `ConversationDetailCard`. Requirements: - Show current label with a subtle animated affordance (spinner or shimmer), plus `detail` as smaller secondary text when present. - Show completed stages for the turn collapsed above the current one, each with a check — this is the "rolling up" quality the user is after. - Animate label transitions; never flash. Enforce a **minimum display time of ~400ms per stage** so fast backends don't produce a strobe. - Fall back to today's three dots when `activityStage` is `null` — covers an older backend and the gap before the first frame. - Theme-aware (light/dark), matching the existing agent bubble styling. - Mobile-safe: truncate `detail` with ellipsis, never widen the bubble past the viewport. ### 5. Handle long-running and stalled turns - Show elapsed time once a turn exceeds ~10s. - If no frame and no token arrive for ~30s, show a soft "still working…" note rather than leaving a frozen label. - Keep working with the existing `streamInterrupted` banner (`AsyncDashboard2.tsx:542-551`); the activity indicator must clear when that fires. ### 6. Accessibility - `role="status"` with `aria-live="polite"` so screen readers announce stage changes. - Respect `prefers-reduced-motion`: no spinner or shimmer animation, just the text. ### 7. Analytics Add events per `ANALYTICS.md` conventions for stage durations, so we can measure where slow turns actually spend their time. ## Acceptance criteria - [ ] `MessageContext` parses versioned JSON frames and ignores unknown `type` and unknown `stage` values without breaking the stream. - [ ] Sentinel-string handling (`CONVERSATION_ID`, `START_OF_THE_STREAM_ENDER_GAME_42`, `END_OF_THE_STREAM_ENDER_GAME_42`) still works unchanged. - [ ] With a backend that emits **no** status frames, behaviour is identical to today — three dots, then text. Covered by test. - [ ] `activityStage`, `activityLabel`, `activityDetail`, and `activityHistory` are exposed and cleared on completion, on error, and on new submit. - [ ] `ActivityIndicator` replaces the inline three-dot block in `ConversationDetailCard` and shows the current label plus optional detail. - [ ] Completed stages for the current turn render collapsed above the active one with a completion mark. - [ ] An unknown stage renders its server-supplied `label` verbatim rather than being dropped. Covered by test. - [ ] Each stage is displayed for a minimum of ~400ms; rapid stage sequences do not strobe. - [ ] Elapsed time appears after ~10s; a "still working" note appears after ~30s of silence. - [ ] Indicator clears when `streamInterrupted` fires and does not linger. - [ ] `role="status"` / `aria-live="polite"` announce changes; `prefers-reduced-motion` disables animation. - [ ] Correct in both light and dark themes; no horizontal overflow at 320px width. - [ ] Verified on Android via Capacitor. - [ ] The frame envelope matches what `chat_backend`#62 and #63 emit; no conflicting `type` values across the three. - [ ] Unit tests cover the full frame sequence, the no-frame fallback, unknown stages, and cleanup on error. ## Out of scope - Nested sub-agent step display — arrives with `chat_backend`#63; the unknown-stage passthrough is what keeps this issue forward-compatible with it. - Rendering citations — separate work, same frame protocol.

westfarn referenced this issue

2026-08-02 12:12:22 -07:00

Render grounded answer citations (Sources list + clickable [n]) #98

westfarn referenced this issue from ai_ml_operations/chat_backend

2026-08-02 12:12:25 -07:00

[TICKET] Answer accuracy overhaul: always-on grounded retrieval, better models, and citations #62

westfarn commented

2026-08-02 12:12:26 -07:00

Citations UI (explicitly out of scope here) tracked separately: #98.

Share the versioned WS frame parser with that work; backend citations from chat_backend#62.

Citations UI (explicitly out of scope here) tracked separately: [#98](https://git.aimloperations.com/ai_ml_operations/chat_web_app/issues/98). Share the versioned WS frame parser with that work; backend citations from [chat_backend#62](https://git.aimloperations.com/ai_ml_operations/chat_backend/issues/62).

westfarn referenced this issue from ai_ml_operations/chat_backend

2026-08-02 12:12:26 -07:00

[TICKET] Long-running agentic task execution: tool-calling loop, sub-agents, and durable runs #63

westfarn referenced this issue from a commit

2026-08-04 04:07:43 -07:00

Replace typing dots with live activity status frames (#96).

westfarn referenced a pull request that will close this issue

2026-08-04 04:07:54 -07:00

Live activity indicator replacing typing dots (#96) #104

westfarn referenced this issue from ai_ml_operations/chat_backend

2026-08-04 04:07:54 -07:00

Eval harness (#62 P4), status frames, and agentic runs (#63) #71