`AppDelegate` constructed `dormantAgentContinuation` and stored it on
self, but no production code ever yielded into it. Combined with
`VoiceController.hudIntentFor` mapping `.thinking` / `.speaking` /
`.idle` all to `.silent`, the HUD ring only saw `.listening` (wake)
and `.reconfiguring` (audio-graph rebuilds). Thinking and speaking
looked identical to idle.
Wire the canonical fix from the audit (Option 2): pump from the
orchestrator events broadcaster into the dormant continuation.
Changes:
- New `OrchestratorEventBroadcaster.Priority.agentHud`. Protects
`.stateChange` / `.turnEnd` / `.error` — these drive ring
lifecycle transitions and can't be tail-dropped. All other event
classes (tokenDelta / thinkingDelta / toolCardUpdate / usage) are
drop-eligible (the HUD doesn't render them).
- `installAgent` registers an `agentHudSubscriberTask` that
subscribes under `.agentHud` and maps `.stateChange(TurnState)` to
`AgentHudIntent`:
- `.thinking` → `.thinking`
- `.speaking` → `.speaking`
- `.idle` → `.idle`
- `.turnEnd` / `.error` → `.idle` (terminal)
- `.booting`, `.listening`, `.awaitingConfirmation`,
`.reconfiguring` → no-op (HUD owns these from other sources)
- Task is cancelled in `applicationWillTerminate` alongside the
other subscriber tasks.
Extends the existing `testIsProtectedForPriorityMatrix` test in
`OrchestratorEventBroadcasterTests` to cover the new `.agentHud`
priority's protection rules.
M-6 / M-3 will replace this with a richer orchestrator state machine
(distinct `.speaking` emission tied to TTS playback), but the v0.1
patch lights up the ring TODAY.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`VoiceController.vadFactory` is the closure
`{ sileroVAD }` in production — it caches a single `SileroVAD`
instance for the process lifetime. `startSTTSession` invoked the
factory but never called `reset()`, so LSTM hidden state (and the
`wasSpeech` flag) leaked across sessions.
The audible failure mode: session N ends in `.speech` state →
`wasSpeech == true`. Session N+1's first sub-threshold window emits
`.speechEnd` (carrying over), then `.silence`. A soft-spoken first
syllable on session N+1 is misclassified — the opening word is
dropped from STT input.
Fix is one line: `vad.reset()` at the top of `startSTTSession`
after `vadFactory()`. `SileroVAD.reset()` already does the right
thing (zeroes LSTM `h, c` + `wasSpeech`), it just wasn't being
called.
Adds VAD-6 regression test: reuses a single `SileroVAD` across two
`VoiceController` sessions (production-shape factory pattern).
Session 1 runs three speech chunks, then PTT-up. Session 2 feeds a
single silence chunk and asserts no submit lands. Pre-fix, session
1's leaked `wasSpeech == true` would have caused session 2's first
silence chunk to emit `.speechEnd`, arming a stale finalize that
submits the empty STT result. Post-fix, `reset()` clears the state
and the session stays in pre-speech with no finalize armed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`VoiceBootHealthProbe` only checked `audioGraphOwner` non-nil +
`micStatus == .authorized` + `currentVariant != nil`. It reported
`state=ok` even after:
- the wake-word DAG died (P0-1 / #28 — pre-fix the DAG was
cancelled on every audio-graph rebuild and never re-armed)
- SpeechAnalyzer assets were missing (silent `assetUnavailable` on
first transcribe)
- TTS was never wired (pre-Track-B-3 `VoiceTTSAdapter(engine: nil)`
no-op'd every synth call)
The probe was useless for incident triage of the very bugs the v0.1
audit cluster targets. Three new watchdog probes fill the gap:
- `WakeWordBootHealthProbe` — asserts the DAG's feed Task is armed
via the new `WakeWordDAG.isFeedArmed` property. `.failed` loud when
the DAG is installed but not feeding; `.unknown` soft when never
installed (models absent, mic denied).
- `STTBootHealthProbe` — reads `SFSpeechRecognizer.authorizationStatus()`
as a proxy for SpeechAnalyzer asset availability. Synchronous read,
no TCC prompt.
- `TTSBootHealthProbe` — asserts `VoiceTTSAdapter.hasEngine` (engine
not nil). `.degraded` loud when the adapter is dormant.
AppDelegate gains two new stored handles (`wakeWordDAG`,
`voiceTTSAdapter`) so the probes can query the live subsystem on
each probe pass. The handles are nil-safe — probes report `.unknown`
when the install path short-circuited.
Adds W1-W3 / T1-T2 / S1 unit tests covering the three watchdog
shapes plus the canonical post-P0-1 (DAG dead) and pre-Track-B-3
(TTS dormant) regression scenarios.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`OpenWakeWordSession.runORT` appended to `melFrameBuffer` and
`embeddingBuffer` on every `feed()` call without ever trimming.
At one mel frame per 80 ms feed (production cadence), 24 h of
always-on listening accumulated:
- ~1.08M mel frames × 32 floats × 4 B ≈ 130 MB
- ~135k embeddings × 96 × 4 B ≈ 50 MB
The leak was masked by P0-1 (#28) because `WakeWordDAG` cancelled and
rebuilt on every audio-graph rebuild, dropping the session along with
the buffers. Closing #28 (stream survives rebuild, feed re-arms
against new ring) makes the session — and the buffers — live for the
process lifetime. Fix the bound now or memory grows unbounded.
Rolling-window trim inside `runORT` after the embedding-advance loop:
- Keep last `76 + embStride` mel frames (= 84) so the next embedding
window has full context after trim.
- Keep last 16 embeddings to match the classifier's `suffix(16)`
read.
- Slide `nextEmbStart` left by the drop count when mel frames are
trimmed so the index stays valid (it indexes into the trimmed
buffer). Clamp at 0.
Adds B1/B2 regression tests via two internal test seams
(`_testInjectFrame`, `_testBufferSizes`) that exercise the same
append + advance + trim logic without requiring real ORT models. B1
runs 10 000 simulated frames and asserts both buffers stay at the
documented bounds; B2 asserts the trim does NOT kick in for short
streams (no premature context loss).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`AppDelegate.installVoice` constructed the adapter with `engine:` only,
leaving `tierResolver` at the default `{ .tier1 }` constant. Setting
`tts.tier = "tier2"` in config did nothing for the real synthesis path
— only for the `get_self_state` tool which reads `snap.tts.tier` via
a separate closure.
Today this is decorative (Orpheus weights aren't shipped, so
`TTSEngineActor.synthesize` resolves tier-2 back to tier-1
internally). The moment weights arrive, however, the user would have
had no way to flip the tier from config — they'd have needed a code
change. The fix lands the wiring now so the config flag is live the
day Orpheus is.
Adds a regression test that asserts `tierResolver` is invoked once
per `synthesize` call on the adapter. Engine is a real tier-1
`TTSEngineActor`; the resolver is a counter-incrementing closure.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Before this fix, `TTSEngineActor.synthesize` yielded `.started →
.firstAudio → .finished` and never `.ttsStopped`. The only emit site
was `InterruptSequence.run` step 5, fired only on barge-in.
`TTSEvent.swift` and the `TTSEngineActor` header both documented the
canonical sequence as `... → .finished → .ttsStopped`, and the I4
ducking test in `TTSInterruptTests` validates ducking releases
specifically on `.ttsStopped` — not `.finished`.
The contract was structurally broken: any future ducking consumer
(none exists in production today, but M-6 will wire one) would have
left the input duck engaged forever after a normal spoken reply.
Two-emit-site model:
- Natural completion: `synthesize` now yields `.ttsStopped` after
`await task.value` returns. Cancellation flows through
`InterruptSequence.run` instead — no double-emit.
- Barge-in: `InterruptSequence.run` step 5 unchanged.
Loosens the InterruptSequence docstring from "ONLY site that emits
.ttsStopped" to describe the two-site cooperation, and updates the
TTSEngineActor header to match. The grep gate referenced in the prior
docstring was a comment-only contract — no CI gate file existed —
so no script changes.
Adds TET1-5 regression test: real AVSpeechSynthesizer + tier-1
synthesize, collector drains until `.ttsStopped`, asserts `.finished`
precedes `.ttsStopped` in the event stream.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two coupled bugs prevented WhisperKit fallback from being reachable
via config:
1. `installVoice` hard-coded the STT factory to
`STTBackendSelector.make(backend: "speech_analyzer")`, ignoring
`PerTurnSnapshot.stt.whisperKitFallback` entirely. Flipping the
config flag changed nothing for real synthesis.
2. The `get_self_state` tool returned camelCase strings
(`"speechAnalyzer"` / `"whisperKit"`) while
`STTBackendSelector.make` accepts only snake_case
(`"speech_analyzer"` / `"whisperkit"`). Any consumer routing the
tool output back through the selector landed in the unknown-backend
default branch — silent drift.
Fix:
- `VoiceController.sttFactory` is now `() async -> any STTProvider`
so the closure can `await configStore.perTurn()` per session. Test
sites using `{ stt }` syntax continue to compile because closures
with no awaits are implicitly async-compatible.
- AppDelegate's factory now reads
`snap.stt.whisperKitFallback` and routes through the canonical
`STTBackendSelector.backendSpeechAnalyzer` /
`STTBackendSelector.backendWhisperKit` constants.
- `get_self_state`'s `sttBackend` closure returns the same canonical
constants so the tool output round-trips through the selector
without falling into the default branch.
Adds B5 regression test: asserts the canonical constants round-trip
through `make(backend:)` without emitting an unknown-backend warning,
and that the pre-fix camelCase strings still produce a warning (locks
against silent drift if anyone re-introduces the mismatch).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`cancelInFlight` was wired to `WakeWordDAG.cancel()`, which calls
`streamCont.finish()` and permanently closes the public wake-word
stream. Every audio-graph rebuild (device change, mic re-grant, ring
overflow, AEC fallback) triggers `cancelInFlight` via
`AudioGraphOwner.teardown`, so the wake path died on first rebuild
event and never recovered.
`stopFeed()` already existed for this case — it cancels the feed Task
while preserving the public stream so consumers can iterate across the
rebuild boundary. The fix is two-part:
1. `setCancelInFlight` now invokes `stopFeed()` instead of `cancel()`.
2. The rebuild-completion consumer now re-arms the DAG against the new
ring (`wakeWordDAG.start(ring: newRing)`) before checking AEC
variant — without this, the DAG's feed Task is gone after a
rebuild and there's no producer for the surviving stream.
Adds WD-3 regression test: drives a real RingBuffer + DAG through
`stopFeed()` then `start(ring: newRing)` and asserts the consumer Task
stays alive AND the new ring's marker samples drive the session.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
james.maesandClaude Opus 4.7