KofTwentyTwo/Jarvis
2
0 Fork 0
Code Issues 111 Pull Requests 16 Actions Packages Projects Releases 4 Wiki Activity

Voice: STT backend hard-coded; whisperKitFallback config dead; camelCase/snake_case mismatch #29

New Issue
Closed
opened 2026-05-12 19:06:12 +00:00 by KofTwentyTwo · 1 comment
KofTwentyTwo commented 2026-05-12 19:06:12 +00:00 (Migrated from github.com)
Copy link
Copy source

Summary

STT backend is hard-coded to speech_analyzer; PerTurnSnapshot.stt is ignored. Compound bug: the get_self_state tool returns camelCase strings ("whisperKit" / "speechAnalyzer") which DON'T match STTBackendSelector.make's accepted snake_case values ("whisperkit" / "speech_analyzer"). Even fixing the hard-code would land in the default unknown backend fallthrough.

Evidence

  • File: App/AppDelegate.swift:1279 (confirmed at HEAD): 
    sttFactory: { STTBackendSelector.make(backend: "speech_analyzer") },
  • File: App/AppDelegate.swift:2025-2028: 
    return snap.stt.whisperKitFallback ? "whisperKit" : "speechAnalyzer"
  • STTBackendSelector constants are public static let backendSpeechAnalyzer = "speech_analyzer" / backendWhisperKit = "whisperkit" (snake_case)

Impact

If SpeechAnalyzer asset download fails on a fresh-install Release, the user cannot fall back to WhisperKit via config — they have to ship a new build.

Recommended fix

Two-part fix:

  1. Replace hard-code with config-aware closure:

    sttFactory: {
    let snap = await configStore.perTurn()
    let backend = snap.stt.whisperKitFallback ? STTBackendSelector.backendWhisperKit : STTBackendSelector.backendSpeechAnalyzer
    return STTBackendSelector.make(backend: backend)
},

  2. Unify string constants:

    • Use STTBackendSelector.backendSpeechAnalyzer / backendWhisperKit at both call sites (don't hard-code strings anywhere)
    • get_self_state tool returns the canonical snake_case ID
    • Document the values

Test

  • Set stt.whisperKitFallback = true in config → on next session, factory returns WhisperKitSTT
  • Unit: parameterized over both backend constants, STTBackendSelector.make returns non-default-fallthrough provider

Source

  • Audit: .planning/audit-2026-05-12/voice-audit.md (2026-05-12) — finding P0-2

Cross-references

  • Related: P2-1 (string mismatch documentation lie)
  • Related: P2-3 (whisperKitFallback feature flag is dead config)
  • Related: #11 (M-6 Voice) — STT backend selection should route through Settings surface
## Summary STT backend is hard-coded to `speech_analyzer`; `PerTurnSnapshot.stt` is ignored. Compound bug: the `get_self_state` tool returns camelCase strings (`"whisperKit"` / `"speechAnalyzer"`) which DON'T match `STTBackendSelector.make`'s accepted snake_case values (`"whisperkit"` / `"speech_analyzer"`). Even fixing the hard-code would land in the default `unknown backend` fallthrough. ## Evidence - File: `App/AppDelegate.swift:1279` (confirmed at HEAD): ```swift sttFactory: { STTBackendSelector.make(backend: "speech_analyzer") }, ``` - File: `App/AppDelegate.swift:2025-2028`: ```swift return snap.stt.whisperKitFallback ? "whisperKit" : "speechAnalyzer" ``` - `STTBackendSelector` constants are `public static let backendSpeechAnalyzer = "speech_analyzer"` / `backendWhisperKit = "whisperkit"` (snake_case) ## Impact If SpeechAnalyzer asset download fails on a fresh-install Release, the user cannot fall back to WhisperKit via config — they have to ship a new build. ## Recommended fix Two-part fix: 1. Replace hard-code with config-aware closure: ```swift sttFactory: { let snap = await configStore.perTurn() let backend = snap.stt.whisperKitFallback ? STTBackendSelector.backendWhisperKit : STTBackendSelector.backendSpeechAnalyzer return STTBackendSelector.make(backend: backend) }, ``` 2. Unify string constants: - Use `STTBackendSelector.backendSpeechAnalyzer` / `backendWhisperKit` at both call sites (don't hard-code strings anywhere) - `get_self_state` tool returns the canonical snake_case ID - Document the values ## Test - Set `stt.whisperKitFallback = true` in config → on next session, factory returns WhisperKitSTT - Unit: parameterized over both backend constants, `STTBackendSelector.make` returns non-default-fallthrough provider ## Source - Audit: `.planning/audit-2026-05-12/voice-audit.md` (2026-05-12) — finding P0-2 ## Cross-references - Related: P2-1 (string mismatch documentation lie) - Related: P2-3 (whisperKitFallback feature flag is dead config) - Related: #11 (M-6 Voice) — STT backend selection should route through Settings surface
KofTwentyTwo commented 2026-05-13 11:51:56 +00:00 (Migrated from github.com)
Copy link
Copy source

Closed by v0.1 cluster merges on 2026-05-13. Develop now at a8c7f1a0.

Closed by v0.1 cluster merges on 2026-05-13. Develop now at `a8c7f1a0`.
Sign in to join this conversation.
No Branch/Tag Specified
Labels
Clear labels
area:agent
area:build
area:bus
area:devtool
area:hud
area:logging
area:mcp
area:memory
area:vision
area:voice
audit-finding
bug
Something isn't working
 carry-forward
comment-cleanup
Comment / docstring cleanup per .planning/code-commenting-guide.md
 dependencies
Dependency updates
 documentation
Improvements or additions to documentation
 duplicate
This issue or pull request already exists
 enhancement
New feature or request
 epic
good first issue
Good for newcomers
 help wanted
Extra attention is needed
 invalid
This doesn't seem right
 javascript
Pull requests that update javascript code
 migration
question
Further information is requested
 severity:critical
severity:high
severity:low
severity:medium
wontfix
This will not be worked on
No labels area:voice audit-finding bug severity:critical
Milestone
No items
No Milestone v0.1 — It actually works (basic capabilities)
Projects
Clear projects
No projects
Assignees
Clear assignees
forge-recovery (Forge Recovery) james.maes (James Maes)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: KofTwentyTwo/Jarvis#29