Commit Graph
11 Commits
Author SHA1 Message Date
james.maes 5bb1e778b5 fix(hud): chat panel auto-scroll on streaming append — B-06 tactical (#51, #150)
Adds a useLayoutEffect on ChatPanel keyed to total content length (captures streaming token append where chatEvents.length is constant). 24px bottom-pin hysteresis; user scroll-up breaks the pin; auto-re-arms on scroll-back-to-bottom. scrollTo feature-detected with scrollTop fallback for JSDOM. 4 vitest cases assert decision logic.
2026-05-13 06:27:42 -05:00
james.maesandClaude Opus 4.7 4159d44f4e feat(track-c 3): HUD camera button emits frameAttachRequested
Closes vision-audit §4 "HUD button emitter has zero matches". The bus
protocol declared `BusInbound.frameAttachRequested` and the Swift
inbound handler at App/AppDelegate.swift:1696-1698 routed it into
`FrameAttachController.requestAttach(reason: .hudButton)`, but no React
component dispatched the message. Now the user has a clickable camera
affordance next to the chat input.

Changes:
- webview/packages/hud/src/chat/CameraButton.tsx (new): stateless button
  that posts `{ type: 'frameAttachRequested' }` via window.jarvisBus.send.
- ChatInput.tsx: imports CameraButton and renders it in the toolbar row
  between the input field and the Send button.
- chat-panel.css: `.camera-button` styling, matches `.chat-input__send`
  chrome so the toolbar reads as one row.

Tests: +3 (CameraButton.test.tsx)
- CB1: renders accessible button (aria-label "Attach camera frame").
- CB2: click posts the BusInbound.frameAttachRequested message.
- CB3: payload shape is `{ type: 'frameAttachRequested' }` only —
  matches the no-associated-value Swift case in BusInbound.swift.

JS-side path: webview/packages/hud/src/chat/CameraButton.tsx:14
  → window.jarvisBus.send({ type: 'frameAttachRequested' })
Swift-side path: App/AppDelegate.swift:1696-1698
  → frameAttachController.requestAttach(reason: .hudButton)

HUD vitest 94 → 97. Pre-existing SegmentedRing typecheck warnings are
unrelated (verified via git stash on a clean tree).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-04 13:06:47 -05:00
james.maesandClaude Opus 4.7 1f8e03edeb fix(track-a): demo-blocker class + audit reports + chat-turn e2e tests
User feedback: 50+ hours of agent work, nothing demonstrably working.
Fanned out 6 deep auditors (voice, HUD, vision, memory, LLM, tests);
all reports landed at .planning/audit-2026-05-03/ with brutal-honesty
mandates. Synthesis (SYNTHESIS.md) confirmed: every modality has at
least one fatal break between code-that-passes-tests and
code-that-runs. Track A is the smallest credible path to one working
modality (text turn + animated rings).

Closed under TDD discipline:

**Item 4 — HUD idle ring static rendering (F-A1-01).**
- webview/packages/hud/src/hud/stateUniforms.ts:30 — was
  `pulse: 0.0, rotate: 0.0` for idle, the all-static configuration
  that rendered STANDBY as dots instead of animated rings. Now
  pulse: 0.4, rotate: 0.05 — gentle ambient distinct from booting +
  thinking.
- webview/packages/hud/tests/stateUniforms.test.ts — table updated;
  new regression-guard `STATE_PARAMS[*] is not all-static` (7 cases,
  one per HudState) prevents anyone from re-zeroing a state.

**Item 1 — CacheHints eligibility gate (likely streamTruncated cause).**
- packages/AgentCore/Sources/AgentCore/CacheHints.swift —
  + static func eligibleForSystemPrompt(_:ttl:). Anthropic's prompt
  cache requires ≥1024 tokens at the breakpoint; below that, the
  server returns 200 OK + immediate-EOF for `extended-cache-ttl`
  hints. The orchestrator's default system prompt is ~10 tokens.
  Helper returns nil for sub-threshold prompts so no cache_control
  marker is emitted.
- packages/AgentCore/Sources/AgentOrchestrator/AgentOrchestrator.swift
  — call sites now pass `CacheHints.eligibleForSystemPrompt(finalSystem)`
  through runTurnLoop instead of unconditional `.extended1h`.
- Tests: CacheHintsEligibilityTests (6 cases), RequestBodyTests R5a/R5b
  (2 cases verifying small-prompt → no cache_control in encoded JSON,
  large-prompt → cache_control present).

**Item 2 — StreamOutcome diagnostic.**
- packages/AgentCore/Sources/AnthropicProvider/StreamOutcome.swift —
  new pure classifier that distinguishes:
    - non-2xx HTTP status
    - 200 OK + 0 bytes (cache_control / auth-header rejection class)
    - 200 OK + bytes + 0 frames (malformed body)
    - 200 OK + frames + EOF before message_stop (mid-stream drop)
    - 200 OK complete (healthy)
  AnthropicProvider's read loop counts frames + bytes and logs the
  summary once per stream. Production logs now distinguish failure
  modes that previously all surfaced as `streamTruncatedFinal`.
- Tests: StreamOutcomeTests (7 cases, all branches).

**Item 3 — KeychainError surfacing.**
- packages/AgentCore/Sources/AnthropicProvider/AnthropicAPIKeyProvider.swift
  — new builder `make(keychain:item:)` returns the
  `@Sendable () async throws -> String` closure with errors
  propagating verbatim. Closes the audit's hidden-401 finding:
  `(try? keychainStoreLocal.get(.anthropic)) ?? ""` swallowed the
  real KeychainError into an empty x-api-key, producing
  `401 x-api-key header is required` indistinguishable from a
  stale-key 401. Now the actual error reaches the chat panel via
  AnthropicProvider → orchestrator.error → BusForwarder →
  submitRejected.
- App/AppDelegate.swift — both Anthropic provider construction sites
  (installAgent, installVision T3 wiring) now use the propagating
  builder.
- Tests: AnthropicAPIKeyProviderTests (6 cases — itemNotFound,
  unexpectedStatus, duplicateItem propagation; happy path; explicit
  item; default item).

**Item 5 — F1 #2 chat-turn round-trip e2e tests.**
- packages/Bus/Tests/BusTests/RealWKWebViewIntegrationTests.swift —
  + 2 new tests using a real WKWebView:
    - test_chatTurnRoundTrip_busOutboundReachesPageHandler — Swift
      `bridge.send(.turnStarted/tokenDelta/turnEnded)` reaches the
      page's onOutbound handler. This is the test the v0.12.0 audit
      called out as catching BLOCKER-INT-1, INT-2, INT-3,
      F-E-RACE-1, F-E-FK-1, F-E-WIRE-1 in one shot.
    - test_chatTurnRoundTrip_busInboundFromJSReachesSwiftHandler —
      JS-side `window.jarvisBus.send({type:'chatSubmit',...})` —
      exactly what ChatInput.tsx does — reaches the bridge's
      onInbound handler with the right body.

**Verification:**
- AgentCore: 215/215 (was 178; +37 across BusForwarder, CacheHints,
  StreamOutcome, AnthropicAPIKeyProvider, RequestBody R5).
- Replay: 33/33.
- Bus: 57/58 (1 pre-existing failure: F-A2-01 stale `JarvisBusWorld`
  assertion documented in AUDIT-FINDINGS, separate scope).
- webview/packages/hud: 94/94 (was 87; +7 stateUniforms regression-
  guards + idle table updates).
- All 15 boundary grep gates: PASS.
- scripts/check-app-builds.sh: PASS.

**Audit reports retained at .planning/audit-2026-05-03/:**
- SYNTHESIS.md (consolidated, with triage plan)
- hud-audit.md, llm-audit.md, memory-audit.md, tests-audit.md,
  vision-audit.md, voice-audit.md, voice.md (per-modality detail)

**Carry-forward (NOT closed this commit):**
- F-A2-01 stale WebviewBridgeOutboundTests assertion (pre-existing).
- Voice (F): models never bundled, audio graph never instantiated,
  STT bridge body is `_ = chunk`, TTS engine nil. ~1 day.
- Vision (F): AVCapturePhotoOutput allocated but capturePhoto never
  called; HUD camera button doesn't exist. ~1 day.
- Memory (F): Apple's stripped libsqlite3 prevents sqlite-vec from
  loading; "future ops plan" has zero commits. ~3 days.
- LLM (D+): NoopBusGateway still drops tool-call events
  (BLOCKER-INT-1).
- Tests (C-): 5 XCTAssertTrue(true) tautologies + multiple
  rubber-stamp tests defending stubs.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-03 17:23:13 -05:00
james.maesandClaude Opus 4.7 621b454835 fix(hud,app): chat input UI + .bus turnStarted/turnEnded forwarding (closes BLOCKER-INT-3 + INT-2 tail)
The v0.12.0 milestone audit (.planning/v0.12.0-MILESTONE-AUDIT.md, status
gaps_found) flagged INT-3 as 🔴: webview ChatPanel.tsx was a read-only
renderer with zero JS-side producer of any inbound bus message. The Swift
inbound side (AppDelegate.swift handleChatSubmit / handleChatCancelAndSubmit)
was already wired by Plan 09-04, but with no UI to fire it the v0.12.0 demo
had no working text-modality path.

Pairs INT-3 with the remaining half of INT-2 because they are not
independently shippable: the JS bus dispatcher silently drops every
tokenDelta when currentTurnId === null, so without turnStarted forwarding
the ChatInput would post chatSubmit and the user would see exactly
nothing back. Commit 602a95b landed the .bus tokenDelta forwarder; this
commit extends that subscriber to also synthesize turnStarted on first
emission for a new TurnID and translate OrchestratorEvent.turnEnd into
BusOutbound.turnEnded with a StopReason → TurnTerminator mapping.

JS side:
- webview/packages/hud/src/chat/ChatInput.tsx — new component. Form
  with input + Send button. Empty/whitespace input is a no-op; trimmed
  text dispatches BusInbound.chatSubmit(text) when currentTurnId === null,
  chatCancelAndSubmit(text) (barge-in) otherwise. Pushes a kind:'text',
  role:'user' event to the chat-events store on submit so the user sees
  their own message in chronology immediately (optimistic local echo).
  Clears the input after dispatch.
- webview/packages/hud/src/App.tsx — mounts <ChatInput /> directly under
  <ChatPanel /> inside .jarvis-hud__chat. The parent flex-column keeps
  the input pinned to the bottom of the floating chat frame.
- webview/packages/hud/src/chat/chat-panel.css — .chat-input row + field
  + button styles using existing tokens.
- webview/packages/hud/src/bus/client.ts — added submitRejected (push
  inline error event) and sessionHistory (no-op until WARN-INT-2 hydration
  lands) arms to the dispatcher switch. Restores the _exhaustive: never
  compile-time check (was failing tsc on develop because both cases
  existed in BusOutbound but not in the switch).

Swift side:
- App/AppDelegate.swift installAgent — extended busSubscriberTask to
  pattern-match .tokenDelta and .turnEnd. tokenDelta synthesizes a
  turnStarted via flushAndSend on first emission for a new TurnID
  (OrchestratorEvent has no explicit start case — start is implicit
  from the first per-turn event). turnEnd flushes pending tokens and
  emits BusOutbound.turnEnded so lifecycle events never overtake their
  data. Uses local lastEmittedTurnId state in the subscriber task.
- AppDelegate.busTurnTerminator(for:) — StopReason → TurnTerminator
  mapping. endTurn / toolUse / maxTokens → .completed; refusal /
  streamTruncated → .errored. cancelled and superseded are reachable
  only via cancel-and-submit / barge-in flows the orchestrator does
  not yet emit; the mapping will extend when they do.

Tests:
- webview/packages/hud/tests/ChatInput.test.tsx — 6 cases (CI1-CI6)
  covering render, empty/whitespace no-op, trimmed-text submit + clear,
  Enter-key form submit, user-text echo to store, barge-in routing
  when currentTurnId !== null. All pass.
- webview/packages/hud/tests/bus-dispatch.test.ts — added B5:
  submitRejected outbound pushes an inline error event with the
  rejection reason. All pass.

Verification:
- webview/packages/hud tests: 87/87 pass.
- AgentCore swift test: 178/178 pass.
- scripts/check-app-builds.sh: PASS.
- All 15 boundary grep gates: PASS.
- Pre-existing: Bus 55/56 (F-A2-01 stale assertion), webview tsc still
  fails on src/hud/SegmentedRing.tsx (F-A1-01 territory). Neither
  introduced or addressed here.

End-to-end smoke (paired with INT-2 closure): user types → chatSubmit
reaches Swift → AgentOrchestrator.submit(.text) → first tokenDelta
synthesizes turnStarted → JS chat panel shows assistant text streaming
→ turnEnded clears the active turn for the next submit.

Closes BLOCKER-INT-3, closes BLOCKER-INT-2 fully (tokens + lifecycle).
The next-action gate of v0.12.0 transitioning from "passes tests" to
"is a working app for one modality" is now functional pending visual
launch confirmation.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-03 07:55:56 -05:00
james.maes 9e8e9ec5af fix(03-review): H-01 unify awaitingApproval sentinel on parsed-object helper
Lifts `isApprovalPlaceholder` out of `ToolCallCard.tsx` into a shared
`chat/approvalSentinel.ts` module and routes both the dispatcher (bus/client.ts
toolCallStart arm) and the card through it. Previously the dispatcher used
literal-byte JSON equality (`msg.argsPreview === '{"awaitingApproval":true}'`)
while the card used parsed-object semantics, so any Swift-side whitespace
drift (pretty-printed encoder, extra spaces, key-ordering) would silently
flip the status to 'running' while the card kept hiding args — a visible
UX desync and a latent drift between two defense-in-depth guards.

New test streaming.test.tsx D5b feeds `'{"awaitingApproval": true}'` (space
after colon) and asserts `status === 'awaiting-approval'`. Fails against the
old literal-string check; passes now.

Existing D5 (exact-string variant) continues to pass.
2026-04-24 16:44:04 -05:00
james.maes 4e2e05e565 feat(03-04): wire bus dispatcher → store; streaming + chronology tests
- store: add currentTurnId + activeTextPartId slices and beginTurn/endTurn actions
- bus/client.ts: extend dispatcher to route every Phase-2 BusOutbound variant
  - turnStarted → beginTurn; turnEnded → endTurn
  - tokenDelta → synthesize text-part on first-after-turnStarted with deterministic
    id turn:{id}:assistant[:n]; subsequent tokens append to same id via
    appendTokenToLastText
  - toolCallStart → upsertToolCall; status='awaiting-approval' when argsPreview is
    the Swift sentinel '{"awaitingApproval":true}', else 'running'. Also nulls
    activeTextPartId so next tokenDelta opens a fresh chronologically-correct part
  - toolCallEnd → upsertToolCall with status completed|failed; assigns result (ok)
    or error (fail)
  - audioLevel: intentional no-op (Phase 6 binds to mic RMS)
  - _exhaustive: never default branch preserves compile-time exhaustiveness
- tests/streaming.test.tsx: 12 tests (S1×3 store slices, D1-D8 dispatcher, I1 replay
  idempotency)
- tests/chronology.test.tsx: 2 tests (Ch1 interleave, Ch2 fixture replay with
  whitespace-normalized DOM content assertion)
- tests/fixtures/streaming-fixture.json: canonical 20-message replay

Tests: 79/79 green; typecheck + build clean
2026-04-24 09:19:12 -05:00
james.maes e2dcaed424 feat(03-04): ChatPanel + discriminator router + tool-call card with pre-approval guard
- ChatPanel renders flat chronological list of ChatEvents with role="log" + aria-live="polite"
- ChatEventRouter discriminates on kind with _exhaustive: never sentinel
- ToolCallCard: expandable card with 5 lifecycle status labels; hides args when sentinel
  {awaitingApproval: true} present OR status === 'awaiting-approval' (defense-in-depth
  for Phase 5 MCP-04; T-03-30 mitigation)
- TextPart renders event.text via JSX interpolation (React auto-escapes; T-03-31 mitigation)
- ErrorPart wraps error messages in role="alert"
- App.tsx composes <ChatPanel /> as sibling beneath LoadingFallbacks
- ChatPanel.test.tsx: 5 tests (C1-C3, TP1-TP2)
- ToolCallCard.test.tsx: 6 tests (T1, T2 spans 5 status labels, T3, T3b, T4, T5)

Tests: 65/65 green; typecheck + build clean
2026-04-24 09:14:58 -05:00
james.maes 4951bfdcd1 feat(03-03): animated shader-driven RingMesh + Reduce Motion DOM fallbacks
- Replace Wave 1's static RingMesh with a shader-driven animated version.
  Reads hudState/a11y/theme via useJarvisStore.getState() inside useFrame
  (Pitfall 1 guard) — no per-frame React rerenders.
- 150ms eased crossfade between states via pure easeStateBlend() helper
  (exported for unit tests).
- uOutwardWaveAmp set only when densityMod === 'wave' (speaking); always
  gated off by uReduceMotion.
- Geometry is NOT disposed in a cleanup — R3F reconciler owns its lifecycle
  (Pitfall 8 guard).
- LoadingFallbacks renders DOM affordances per HudState when reduceMotion
  is active (three-dots, pulse-dot, speaking-pulse, etc.). Persistent
  corner-dot for awaitingConfirmation regardless of reduceMotion.
- loading-fallback.css uses @media (prefers-reduced-motion: reduce) as
  defense-in-depth so WKWebView respects the OS setting even if the bus
  a11y message is delayed.
- Add RingMaterialImpl type export from RingMaterial.ts for typed matRef.
- App.tsx composes both ParticleRing and LoadingFallbacks.
- 15 tests: ParticleRing mounts without crash in jsdom, LoadingFallbacks
  produces correct data-hud-fallback slugs per state+reduceMotion combo,
  easeStateBlend edge cases (0/0.15=1, 0/0.075=0.5, clamp at 1, additive,
  no-op when delta=0).
2026-04-24 09:04:09 -05:00
james.maes d094b3b297 feat(03-03): add state uniforms, GLSL sources, RingMaterial + table-driven tests
- stateUniforms.ts exposes STATE_PARAMS + STATE_TO_IDX for all 7 HudStates
  per RESEARCH Particle Ring Visual Design table; parseHex helper for
  hex-to-RGB conversion with arc-reactor-glow fallback.
- ring.vert.glsl + ring.frag.glsl loaded via Vite ?raw imports.
- RingMaterial registers drei shaderMaterial under <ringMaterial> with R3F 9
  ThreeElements namespace augmentation (R3F 9 moved away from
  ReactThreeFiber.Object3DNode to ThreeElements['shaderMaterial']).
- 27 tests covering all 7 state entries, unique index mapping, color
  sentinel distribution, parseHex round-trip, and ?raw shader imports.
2026-04-24 08:54:04 -05:00
james.maesandClaude Opus 4.7 b9755b14aa test(03-02): vitest coverage for @jarvis/hud store + bus dispatcher (12 tests)
- store.test.ts (6 tests): defaults, actions present, setHudState isolation
  (chatEvents ref stable), subscribeWithSelector granularity (hudState selector
  doesn't fire on chatEvents change), appendTokenToLastText no-op on unknown id +
  happy-path append.
- bus-dispatch.test.ts (6 tests): hudState outbound routes to store.setHudState;
  state transitions (listening -> idle); decode errors surface via console.error
  for invalid JSON and missing-discriminator; main.tsx boot posts uiReady after
  React commit; exhaustiveness-sentinel compile-time documentation test.
- setup.ts: ResizeObserver polyfill for jsdom — R3F <Canvas> via react-use-measure
  needs it, jsdom doesn't ship it.

Probe documented in SUMMARY (no commit): locally appending a synthetic
BusOutbound variant produces TS2322 on `_exhaustive: never = msg`, confirming
the guard is live.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-23 18:58:23 -05:00
james.maesandClaude Opus 4.7 33e1afc1f1 feat(03-02): scaffold @jarvis/hud webview package (React 19 + R3F 9 + drei 10.7 + three r184 + Zustand 5 + Vite 8)
- New pnpm workspace package at webview/packages/hud/ with workspace dep on @jarvis/bus
- Vite 8 config uses base: './' for WKWebView file:// compat (Pitfall 3)
- CSP meta tag with connect-src 'none', script-src 'self' per RESEARCH §Security
- Zustand 5 store with subscribeWithSelector middleware; slices for hudState, chatEvents, a11y, theme, connection
- Exhaustive bus outbound dispatcher — const _exhaustive: never = msg (no `as never`); adding a BusOutbound case without a handler fails tsc --strict
- Minimal static 512-particle RingMesh (Plan 03-03 layers shader uniforms)
- queueMicrotask uiReady post after first React commit (avoids Pitfall 7 race)

Deviations: @testing-library/react pinned to ^16.3.0 (latest published) and
@testing-library/jest-dom to ^6.6.0 — RESEARCH called for 17/7 but npm only
publishes 16/6. vitest.config.ts removed from tsconfig include — vitest 3's
config re-exports vite 5 types alongside installed vite 8, tripping
exactOptionalPropertyTypes; vite build itself loads the config fine at runtime.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-23 18:54:11 -05:00