KofTwentyTwo/Jarvis
2
0 Fork 0
Code Issues 111 Pull Requests 16 Actions Packages Projects Releases 4 Wiki Activity

Agent: AgentHudIntent stream dormant; HUD shows no signal during thinking/speaking #36

New Issue
Closed
opened 2026-05-12 19:07:12 +00:00 by KofTwentyTwo · 1 comment
KofTwentyTwo commented 2026-05-12 19:07:12 +00:00 (Migrated from github.com)
Copy link
Copy source

Summary

AgentHudIntent stream is dormant — the HUD has no signal during thinking or speaking. AppDelegate constructs the stream and stores dormantAgentContinuation, but no production code ever yields into it. Combined with VoiceController.hudIntentFor mapping .thinking / .speaking / .idle all to .silent, the HUD ring sees .listening (wake-word fires), .reconfiguring (audio-graph rebuilds), and .silent everywhere else. Thinking and speaking look identical to idle.

Evidence

  • File: App/AppDelegate.swift:2487-2490 — constructs stream, stores dormantAgentContinuation
  • grep -rn dormantAgentContinuation App/ returns only declaration + assignment; no yield calls
  • File: packages/Voice/Sources/Voice/Control/VoiceController.swift:626-632 — hudIntentFor maps thinking/speaking → .silent

Impact

The HUD ring doesn't change shape during agent thinking or TTS playback. Cross-cutting with HUD audit — voice surface owns one end of it.

Recommended fix

Two options (pick one):

  1. VoiceController emits a distinguishing .speaking / .thinking intent (extends VoiceHudIntent enum). Pure voice-surface fix.

  2. Agent install path pumps AgentHudIntent values from the orchestrator events broadcaster:

    Task {
    for await event in orchestratorEvents {
        switch event {
        case .turnStarted: dormantAgentContinuation.yield(.thinking)
        case .toolResultFull: dormantAgentContinuation.yield(.thinking)
        case .turnEnded: dormantAgentContinuation.yield(.idle)
        default: break
        }
    }
}

Option 2 is the canonical agent-state-source path. Lands naturally as part of #7 (M-3 Diagnostics surface).

Test

  • Behavioral: trigger turn → assert HUD ring transitions idle → thinking → speaking → idle

Source

  • Audit: .planning/audit-2026-05-12/voice-audit.md (2026-05-12) — finding P1-5

Cross-references

  • Related: #7 (M-3 Diagnostics — HUD state moves under this surface)
  • Related: bus-hud audit S1 (MenuBarIconController.transition production-dead)
## Summary `AgentHudIntent` stream is dormant — the HUD has no signal during thinking or speaking. AppDelegate constructs the stream and stores `dormantAgentContinuation`, but no production code ever yields into it. Combined with `VoiceController.hudIntentFor` mapping `.thinking` / `.speaking` / `.idle` all to `.silent`, the HUD ring sees `.listening` (wake-word fires), `.reconfiguring` (audio-graph rebuilds), and `.silent` everywhere else. Thinking and speaking look identical to idle. ## Evidence - File: `App/AppDelegate.swift:2487-2490` — constructs stream, stores `dormantAgentContinuation` - `grep -rn dormantAgentContinuation App/` returns only declaration + assignment; no yield calls - File: `packages/Voice/Sources/Voice/Control/VoiceController.swift:626-632` — `hudIntentFor` maps thinking/speaking → `.silent` ## Impact The HUD ring doesn't change shape during agent thinking or TTS playback. Cross-cutting with HUD audit — voice surface owns one end of it. ## Recommended fix Two options (pick one): 1. `VoiceController` emits a distinguishing `.speaking` / `.thinking` intent (extends `VoiceHudIntent` enum). Pure voice-surface fix. 2. Agent install path pumps `AgentHudIntent` values from the orchestrator events broadcaster: ```swift Task { for await event in orchestratorEvents { switch event { case .turnStarted: dormantAgentContinuation.yield(.thinking) case .toolResultFull: dormantAgentContinuation.yield(.thinking) case .turnEnded: dormantAgentContinuation.yield(.idle) default: break } } } ``` Option 2 is the canonical agent-state-source path. Lands naturally as part of #7 (M-3 Diagnostics surface). ## Test - Behavioral: trigger turn → assert HUD ring transitions `idle → thinking → speaking → idle` ## Source - Audit: `.planning/audit-2026-05-12/voice-audit.md` (2026-05-12) — finding P1-5 ## Cross-references - Related: #7 (M-3 Diagnostics — HUD state moves under this surface) - Related: bus-hud audit S1 (MenuBarIconController.transition production-dead)
KofTwentyTwo commented 2026-05-13 11:52:07 +00:00 (Migrated from github.com)
Copy link
Copy source

Closed by v0.1 cluster merges on 2026-05-13. Develop now at a8c7f1a0.

Closed by v0.1 cluster merges on 2026-05-13. Develop now at `a8c7f1a0`.
Sign in to join this conversation.
No Branch/Tag Specified
Labels
Clear labels
area:agent
area:build
area:bus
area:devtool
area:hud
area:logging
area:mcp
area:memory
area:vision
area:voice
audit-finding
bug
Something isn't working
 carry-forward
comment-cleanup
Comment / docstring cleanup per .planning/code-commenting-guide.md
 dependencies
Dependency updates
 documentation
Improvements or additions to documentation
 duplicate
This issue or pull request already exists
 enhancement
New feature or request
 epic
good first issue
Good for newcomers
 help wanted
Extra attention is needed
 invalid
This doesn't seem right
 javascript
Pull requests that update javascript code
 migration
question
Further information is requested
 severity:critical
severity:high
severity:low
severity:medium
wontfix
This will not be worked on
No labels area:agent area:hud area:voice audit-finding bug severity:high
Milestone
No items
No Milestone v0.1 — It actually works (basic capabilities)
Projects
Clear projects
No projects
Assignees
Clear assignees
forge-recovery (Forge Recovery) james.maes (James Maes)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: KofTwentyTwo/Jarvis#36