Commit Graph
12 Commits
Author SHA1 Message Date
james.maes a8c7f1a09b chore(deps): batch Dependabot bumps — fiber + plugin-react + jsdom + vite (#144, #145, #147, #148, #160)
Lands four SAFE-MERGE Dependabot bumps in one commit. Each was confirmed SAFE-MERGE by the H' triage on PR #155, then re-verified locally on the new lockfile baseline (vitest@3 + vite@8 + esbuild dropped) from PR #159.

- @react-three/fiber 9.6.0 → 9.6.1 (closes #148)
- @vitejs/plugin-react 5.x → 6.0.1 (closes #147)
- jsdom 25.x → 29.1.1 (closes #144)
- vite pinned to >=8.0.12 via pnpm.overrides (closes #145)

37/37 bus + 101/101 hud tests; both workspaces build; production webview bundle stamps cleanly. PR #146 (TypeScript 6) NOT included; deferred for focused evaluation.
2026-05-13 06:47:16 -05:00
james.maes 5bb1e778b5 fix(hud): chat panel auto-scroll on streaming append — B-06 tactical (#51, #150)
Adds a useLayoutEffect on ChatPanel keyed to total content length (captures streaming token append where chatEvents.length is constant). 24px bottom-pin hysteresis; user scroll-up breaks the pin; auto-re-arms on scroll-back-to-bottom. scrollTo feature-detected with scrollTop fallback for JSDOM. 4 vitest cases assert decision logic.
2026-05-13 06:27:42 -05:00
james.maesandClaude Opus 4.7 df6a4d27cc fix(track-c 4): explicit T2Provider missing-variant; remove silent t1 fallback
Closes vision-audit §3 "T2 is decorative". AppDelegate previously wired
the VisionRouter as `t2Provider: t1`, so `evaluatePostResponse →
.escalateToT2 → providerForTier(.t2LocalQuality) → t1Provider` silently
re-ran the same low-confidence response on the same model with no
operator signal that T2 was never built.

This commit takes the explicit-missing path. Real T2 wiring
(VllmMlxSidecar instantiation + binary bundling + codesigning +
feature-flag gating) is downstream work and out of scope for Track C.

Changes:
- packages/Vision/Sources/Vision/VisionError.swift: new
  `.t2ProviderUnavailable` case.
- packages/Vision/Sources/Vision/MissingT2Provider.swift (new):
  `LLMProvider` conformer whose text and multimodal `stream(...)`
  entry points finish with `VisionError.t2ProviderUnavailable`.
- App/AppDelegate.swift: replaces `t2Provider: t1` with
  `MissingT2Provider()`. Comment now documents that callers MUST gate
  T2 escalation on `evaluatePostResponse(..., t2Available: false)`,
  which routes to `.useT1Result` and never invokes T2 — so this
  provider is never actually streamed in the live flow.

Why explicit-missing over real T2: real T2 demands ~6 GB Qwen 3.5 VL
weights, vllm-mlx binary at a stable path, codesigning a nested .app
helper, feature-flag plumbing, and a new TCC surface. The bug being
fixed here is "silent T1-as-T2 fallback hides intent" — making the
fallback explicit (impossible to mistake for working T2) is enough.

Tests: +4 (MissingT2ProviderTests)
- testStreamFinishesWithExplicitError: text path throws
  t2ProviderUnavailable.
- testMultimodalStreamFinishesWithExplicitError: image path throws
  t2ProviderUnavailable.
- testRouterStillStaysOnT1WhenT2Unavailable: re-asserts router's
  no-auto-cloud invariant — production never reaches the missing
  provider's stream when wired correctly.
- testAppDelegateNoLongerCarriesSilentT1Fallback: structural grep
  enforces no regression to `t2Provider: t1`.

Vision 63 → 67. Six boundary gates + check-app-builds pass clean.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-04 13:13:38 -05:00
james.maesandClaude Opus 4.7 1f8e03edeb fix(track-a): demo-blocker class + audit reports + chat-turn e2e tests
User feedback: 50+ hours of agent work, nothing demonstrably working.
Fanned out 6 deep auditors (voice, HUD, vision, memory, LLM, tests);
all reports landed at .planning/audit-2026-05-03/ with brutal-honesty
mandates. Synthesis (SYNTHESIS.md) confirmed: every modality has at
least one fatal break between code-that-passes-tests and
code-that-runs. Track A is the smallest credible path to one working
modality (text turn + animated rings).

Closed under TDD discipline:

**Item 4 — HUD idle ring static rendering (F-A1-01).**
- webview/packages/hud/src/hud/stateUniforms.ts:30 — was
  `pulse: 0.0, rotate: 0.0` for idle, the all-static configuration
  that rendered STANDBY as dots instead of animated rings. Now
  pulse: 0.4, rotate: 0.05 — gentle ambient distinct from booting +
  thinking.
- webview/packages/hud/tests/stateUniforms.test.ts — table updated;
  new regression-guard `STATE_PARAMS[*] is not all-static` (7 cases,
  one per HudState) prevents anyone from re-zeroing a state.

**Item 1 — CacheHints eligibility gate (likely streamTruncated cause).**
- packages/AgentCore/Sources/AgentCore/CacheHints.swift —
  + static func eligibleForSystemPrompt(_:ttl:). Anthropic's prompt
  cache requires ≥1024 tokens at the breakpoint; below that, the
  server returns 200 OK + immediate-EOF for `extended-cache-ttl`
  hints. The orchestrator's default system prompt is ~10 tokens.
  Helper returns nil for sub-threshold prompts so no cache_control
  marker is emitted.
- packages/AgentCore/Sources/AgentOrchestrator/AgentOrchestrator.swift
  — call sites now pass `CacheHints.eligibleForSystemPrompt(finalSystem)`
  through runTurnLoop instead of unconditional `.extended1h`.
- Tests: CacheHintsEligibilityTests (6 cases), RequestBodyTests R5a/R5b
  (2 cases verifying small-prompt → no cache_control in encoded JSON,
  large-prompt → cache_control present).

**Item 2 — StreamOutcome diagnostic.**
- packages/AgentCore/Sources/AnthropicProvider/StreamOutcome.swift —
  new pure classifier that distinguishes:
    - non-2xx HTTP status
    - 200 OK + 0 bytes (cache_control / auth-header rejection class)
    - 200 OK + bytes + 0 frames (malformed body)
    - 200 OK + frames + EOF before message_stop (mid-stream drop)
    - 200 OK complete (healthy)
  AnthropicProvider's read loop counts frames + bytes and logs the
  summary once per stream. Production logs now distinguish failure
  modes that previously all surfaced as `streamTruncatedFinal`.
- Tests: StreamOutcomeTests (7 cases, all branches).

**Item 3 — KeychainError surfacing.**
- packages/AgentCore/Sources/AnthropicProvider/AnthropicAPIKeyProvider.swift
  — new builder `make(keychain:item:)` returns the
  `@Sendable () async throws -> String` closure with errors
  propagating verbatim. Closes the audit's hidden-401 finding:
  `(try? keychainStoreLocal.get(.anthropic)) ?? ""` swallowed the
  real KeychainError into an empty x-api-key, producing
  `401 x-api-key header is required` indistinguishable from a
  stale-key 401. Now the actual error reaches the chat panel via
  AnthropicProvider → orchestrator.error → BusForwarder →
  submitRejected.
- App/AppDelegate.swift — both Anthropic provider construction sites
  (installAgent, installVision T3 wiring) now use the propagating
  builder.
- Tests: AnthropicAPIKeyProviderTests (6 cases — itemNotFound,
  unexpectedStatus, duplicateItem propagation; happy path; explicit
  item; default item).

**Item 5 — F1 #2 chat-turn round-trip e2e tests.**
- packages/Bus/Tests/BusTests/RealWKWebViewIntegrationTests.swift —
  + 2 new tests using a real WKWebView:
    - test_chatTurnRoundTrip_busOutboundReachesPageHandler — Swift
      `bridge.send(.turnStarted/tokenDelta/turnEnded)` reaches the
      page's onOutbound handler. This is the test the v0.12.0 audit
      called out as catching BLOCKER-INT-1, INT-2, INT-3,
      F-E-RACE-1, F-E-FK-1, F-E-WIRE-1 in one shot.
    - test_chatTurnRoundTrip_busInboundFromJSReachesSwiftHandler —
      JS-side `window.jarvisBus.send({type:'chatSubmit',...})` —
      exactly what ChatInput.tsx does — reaches the bridge's
      onInbound handler with the right body.

**Verification:**
- AgentCore: 215/215 (was 178; +37 across BusForwarder, CacheHints,
  StreamOutcome, AnthropicAPIKeyProvider, RequestBody R5).
- Replay: 33/33.
- Bus: 57/58 (1 pre-existing failure: F-A2-01 stale `JarvisBusWorld`
  assertion documented in AUDIT-FINDINGS, separate scope).
- webview/packages/hud: 94/94 (was 87; +7 stateUniforms regression-
  guards + idle table updates).
- All 15 boundary grep gates: PASS.
- scripts/check-app-builds.sh: PASS.

**Audit reports retained at .planning/audit-2026-05-03/:**
- SYNTHESIS.md (consolidated, with triage plan)
- hud-audit.md, llm-audit.md, memory-audit.md, tests-audit.md,
  vision-audit.md, voice-audit.md, voice.md (per-modality detail)

**Carry-forward (NOT closed this commit):**
- F-A2-01 stale WebviewBridgeOutboundTests assertion (pre-existing).
- Voice (F): models never bundled, audio graph never instantiated,
  STT bridge body is `_ = chunk`, TTS engine nil. ~1 day.
- Vision (F): AVCapturePhotoOutput allocated but capturePhoto never
  called; HUD camera button doesn't exist. ~1 day.
- Memory (F): Apple's stripped libsqlite3 prevents sqlite-vec from
  loading; "future ops plan" has zero commits. ~3 days.
- LLM (D+): NoopBusGateway still drops tool-call events
  (BLOCKER-INT-1).
- Tests (C-): 5 XCTAssertTrue(true) tautologies + multiple
  rubber-stamp tests defending stubs.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-03 17:23:13 -05:00
james.maesandClaude Opus 4.7 621b454835 fix(hud,app): chat input UI + .bus turnStarted/turnEnded forwarding (closes BLOCKER-INT-3 + INT-2 tail)
The v0.12.0 milestone audit (.planning/v0.12.0-MILESTONE-AUDIT.md, status
gaps_found) flagged INT-3 as 🔴: webview ChatPanel.tsx was a read-only
renderer with zero JS-side producer of any inbound bus message. The Swift
inbound side (AppDelegate.swift handleChatSubmit / handleChatCancelAndSubmit)
was already wired by Plan 09-04, but with no UI to fire it the v0.12.0 demo
had no working text-modality path.

Pairs INT-3 with the remaining half of INT-2 because they are not
independently shippable: the JS bus dispatcher silently drops every
tokenDelta when currentTurnId === null, so without turnStarted forwarding
the ChatInput would post chatSubmit and the user would see exactly
nothing back. Commit 602a95b landed the .bus tokenDelta forwarder; this
commit extends that subscriber to also synthesize turnStarted on first
emission for a new TurnID and translate OrchestratorEvent.turnEnd into
BusOutbound.turnEnded with a StopReason → TurnTerminator mapping.

JS side:
- webview/packages/hud/src/chat/ChatInput.tsx — new component. Form
  with input + Send button. Empty/whitespace input is a no-op; trimmed
  text dispatches BusInbound.chatSubmit(text) when currentTurnId === null,
  chatCancelAndSubmit(text) (barge-in) otherwise. Pushes a kind:'text',
  role:'user' event to the chat-events store on submit so the user sees
  their own message in chronology immediately (optimistic local echo).
  Clears the input after dispatch.
- webview/packages/hud/src/App.tsx — mounts <ChatInput /> directly under
  <ChatPanel /> inside .jarvis-hud__chat. The parent flex-column keeps
  the input pinned to the bottom of the floating chat frame.
- webview/packages/hud/src/chat/chat-panel.css — .chat-input row + field
  + button styles using existing tokens.
- webview/packages/hud/src/bus/client.ts — added submitRejected (push
  inline error event) and sessionHistory (no-op until WARN-INT-2 hydration
  lands) arms to the dispatcher switch. Restores the _exhaustive: never
  compile-time check (was failing tsc on develop because both cases
  existed in BusOutbound but not in the switch).

Swift side:
- App/AppDelegate.swift installAgent — extended busSubscriberTask to
  pattern-match .tokenDelta and .turnEnd. tokenDelta synthesizes a
  turnStarted via flushAndSend on first emission for a new TurnID
  (OrchestratorEvent has no explicit start case — start is implicit
  from the first per-turn event). turnEnd flushes pending tokens and
  emits BusOutbound.turnEnded so lifecycle events never overtake their
  data. Uses local lastEmittedTurnId state in the subscriber task.
- AppDelegate.busTurnTerminator(for:) — StopReason → TurnTerminator
  mapping. endTurn / toolUse / maxTokens → .completed; refusal /
  streamTruncated → .errored. cancelled and superseded are reachable
  only via cancel-and-submit / barge-in flows the orchestrator does
  not yet emit; the mapping will extend when they do.

Tests:
- webview/packages/hud/tests/ChatInput.test.tsx — 6 cases (CI1-CI6)
  covering render, empty/whitespace no-op, trimmed-text submit + clear,
  Enter-key form submit, user-text echo to store, barge-in routing
  when currentTurnId !== null. All pass.
- webview/packages/hud/tests/bus-dispatch.test.ts — added B5:
  submitRejected outbound pushes an inline error event with the
  rejection reason. All pass.

Verification:
- webview/packages/hud tests: 87/87 pass.
- AgentCore swift test: 178/178 pass.
- scripts/check-app-builds.sh: PASS.
- All 15 boundary grep gates: PASS.
- Pre-existing: Bus 55/56 (F-A2-01 stale assertion), webview tsc still
  fails on src/hud/SegmentedRing.tsx (F-A1-01 territory). Neither
  introduced or addressed here.

End-to-end smoke (paired with INT-2 closure): user types → chatSubmit
reaches Swift → AgentOrchestrator.submit(.text) → first tokenDelta
synthesizes turnStarted → JS chat panel shows assistant text streaming
→ turnEnded clears the active turn for the next submit.

Closes BLOCKER-INT-3, closes BLOCKER-INT-2 fully (tokens + lifecycle).
The next-action gate of v0.12.0 transitioning from "passes tests" to
"is a working app for one modality" is now functional pending visual
launch confirmation.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-03 07:55:56 -05:00
james.maes 7ac36871cf build(phase-09): update webview entry to v2.3.0 bundle (index-PXUzusnJ.js) 2026-05-02 06:33:14 -05:00
james.maes dfc237db0c feat(07-06): installMemory + installVision + TS bus protocol 2.1.0 (Task 1)
- AppDelegate gains installMemory() and installVision() mirroring installVoice's
  six-step pattern. Memory installs BEFORE voice (memory ready before first
  turn); vision AFTER voice (camera lifecycle independent).
- MemoryStore + MemoryExtractionOrchestrator + MemoryExtractionCoordinator
  constructed; AppDelegateMemoryReplaySink bridges Memory.MemoryReplaySink ->
  Replay.ReplayLog.record(_:for:). Coordinator wiring deferred until
  AgentOrchestrator lands in AppDelegate (documented in SUMMARY).
- CameraCapture + PresenceMonitor instantiated; the SINGLE PresenceSignalBus
  reference (owned by PresenceMonitor) flows via AppDelegateContextBuilderAdapter
  to ContextBuilder.installPresence and via hudStateCoordinator?.attachPresence
  to the HUD coordinator. VISION-03 boundary preserved.
- VisionRouter constructed with T1=Ollama, T2=Ollama (sidecar deferred), T3=
  Anthropic (key from Keychain); the runTurn dispatcher branch is deferred
  until AgentOrchestrator wires up.
- DisablePresence menu-bar toggle wired alongside MuteWakeWord.
- BannerContent.cameraDenied + cameraRevoked presets; cameraDegradationTask
  watches CameraCapture.degradationStream and enqueues banners.
- HudStateCoordinator.attachPresence is a generic AsyncSequence subscriber
  preserving HUD-08 single-writer invariant (presence is record-only).
- ContextBuilder.installPresence(_:) is a static no-op drain (deferred wiring
  for the future per-turn system-prompt enricher).
- TS BUS_PROTOCOL_VERSION 2.0.0 -> 2.1.0; sessionHistory + TurnRow added to
  the BusOutbound discriminator union with full decode arm. Round-trip test
  bumped accordingly. scripts/check-bus-protocol-version.sh now passes.
2026-04-29 18:40:47 -05:00
james.maesandClaude Opus 4.7 c951f1262c fix(hud): add defer to entry script when stripping type=module
`type="module"` scripts are deferred-by-default — execution waits until
the body is parsed and `<div id="root">` exists. Plain `<script src=>`
runs immediately, mid-<head>, so React's `createRoot(document
.getElementById('root'))` resolves to `null` and silently mounts
nothing. No error fires, no console output, just a forever-empty #root.

Reproduced via an inline diagnostic that polled `#root.firstChild`
3 seconds after load — fired with "No errors captured but #root is
still empty after 3s — R3F never mounted." Adding `defer` to the
emitted script tag restores the parse-then-execute order modules had
built in, and the HUD now mounts and renders content (verified
visually: fallback particle ring paints on the transparent panel).

The Vite plugin's regex now both strips `type="module"` and
substitutes `defer` in its place, in a single pass per script tag.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-26 16:43:42 -05:00
james.maesandClaude Opus 4.7 06bb159197 fix(hud): Vite IIFE bundle + strip module/crossorigin attrs from index.html
The R3F bundle silently failed to load in the production HUD with
type=2/code=0 resource errors:

  resourceID=21 didFinishResourceLoad (length=649)   <- index.html
  resourceID=26 didFailResourceLoad                   <- 1.1 MB JS bundle
  resourceID=27 didFinishResourceLoad (length=4514)   <- CSS

Root cause: Vite's default ES-module output emits `<script type="module"
crossorigin>` for the entry script. The `type="module"` attribute forces
CORS-clean fetching even without `crossorigin`. From a `file://` origin
WKWebView treats every load as a "null origin" cross-origin request, and
the JS bundle subresource fails. The CSS loaded because `<link>` tags
don't carry the same module-fetch contract.

Two-part fix:

1. `format: 'iife'` in rollupOptions.output: produces a self-executing
   bundle that runs as a regular script with no import/export semantics.
   `inlineDynamicImports: true` is required because IIFE format forbids
   chunk splitting. Trade-off: no code splitting, but the HUD is a
   single-page R3F app where splitting wouldn't help first paint.

2. `stripModuleAttrs` Vite plugin: Vite hardcodes `type="module"
   crossorigin` in the emitted index.html script tag even when rollup
   format is IIFE — the HTML emitter doesn't track the format change.
   The plugin runs at `enforce: 'post'` and rewrites both attributes
   off via two regex passes against the transformed HTML.

R3F's drei pulls in modules that reference `import.meta`, which becomes
`{}` under IIFE format (build-time warning). The affected paths are
VideoTexture and FaceLandmarker — neither is on the particle-ring
critical path; verify on first-paint and accept as known-degraded for
now if those features ever need import.meta semantics, switch to a
custom WKURLSchemeHandler with `jarvis://` so `'self'` resolves to a
real origin and ES modules can load natively.

Empirically verified: after this fix, resourceID=26 reports
didFinishResourceLoad (length=1088575) instead of didFailResourceLoad —
the full 1.1 MB bundle reaches the WebContent process.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-26 16:21:42 -05:00
james.maes 94038cc1ec chore(03-review): rebuild webview bundle — hash drift from H-01/H-02 edits
Source changes in webview/packages/hud/src landed new hashed asset name
(index-BKfT8cyH.js replaces index-DYJVWSdi.js). The committed index.html
points at the new hash to keep the bundle in sync for fresh checkouts
that run xcodebuild without first running scripts/build-webview.sh.

assets/ is gitignored — the rsync populates it on each xcodebuild
pre-build phase. This commit only tracks the <script src> pointer.
2026-04-24 16:50:42 -05:00
james.maesandClaude Opus 4.7 a9a1b86ef9 feat(03-05): build-webview.sh + project.yml pre-build wiring + commit index.html
- scripts/build-webview.sh (exec): pnpm install (content-hash-cached) +
  @jarvis/bus build + @jarvis/hud build + rsync dist/ -> App/Resources/webview/
  with --exclude bus-harness.html (preserved for 02-04 parity script) and
  --exclude .gitignore (preserves the assets/ ignore pattern).
- project.yml: adds two preBuildScripts after check-no-evaluate-javascript:
    * Check single-writer HUD state (HUD-08)
    * Build webview bundle
  Ordering stays: existing parity/protocol/no-eval checks → single-writer
  lint → webview build → (postBuildScripts: pre-codesign verify → codesign
  → post-codesign verify → pbxproj lint).
- App/Resources/webview/.gitignore: ignore assets/ (regenerated every build).
- App/Resources/webview/index.html: committed entry point (R3F bundle).

Assets are NOT committed — they rebuild from webview/packages/hud/ on every
xcodebuild via the new pre-build script.

xcodebuild build -configuration Debug succeeds end-to-end; built app
has Contents/Resources/webview/index.html + assets/index-*.js with
relative (./assets/) paths.

packages/Bus swift test: 47/47 green.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-24 15:47:24 -05:00
james.maes 20e8647af4 feat(02-03): wire bus into AppDelegate + expose HUD webView + bundle harness
- JarvisHUDPanel.webView: private → public. Surgical one-line change per
  RESEARCH open question #4 — keeps the dep graph App → Bus, not inverted.
- AppDelegate.installBus(): new launch-chain step 4.5 (after installHUDPanel
  / installBannerPanel). Constructs WebviewBridge around panel.webView,
  wires alertPresenter → TCCAlertService.presentHardBlock + terminate,
  wires onHandshakeArmed → onBusArmed test hook, installs
  BridgeNavigationDelegate so WKNavigationDelegate.webView(_:didFinish:)
  triggers startHandshake, and loads bus-harness.html via
  Bundle.main.url(forResource:withExtension:subdirectory:). Missing
  harness is a hard-block (terminates) in production; silently skips
  under XCTest so unit tests assert bridge construction.
- AppDelegate.toggleHUD(): gates on handshakeState == .armed. Unarmed
  path enqueues BannerContent(id: "hud-not-ready", priority: 2) rather
  than popping an empty window.
- AppDelegate.onHandshakeMismatch + onBusArmed: @MainActor injectable
  closures for test hooks. Production default calls NSApp.terminate(nil)
  after the TCCAlertService modal returns.
- App/Resources/webview/bus-harness.html: bundle-vendored harness loaded
  from Jarvis.app/Contents/Resources/webview/bus-harness.html. Plan 04
  adds the parity check that keeps this in sync with webview/bus-harness.html.
- project.yml: adds Bus package dep, adds App/Resources/webview as a
  blue-folder reference (type: folder under sources:) so the subdirectory
  survives into the bundle. Excludes App/Resources/** from the flat
  source path to avoid double-including. xcodegen regenerates clean.
- AppDelegateBusWiringTests: 6 new test cases covering installBus
  construction, toggleHUD armed/unarmed gating, onHandshakeArmed hook
  firing, mismatch terminate hook, and Bundle.main harness lookup.

Verified:
- xcodegen generate + xcodebuild -scheme Jarvis build: BUILD SUCCEEDED.
- packages/Bus SPM tests: 47 passed / 0 failed.
- bus-harness.html + Bus_Bus.bundle/Injection.js both ship in the .app.
- grep -rn '.evaluateJavaScript(' App/ packages/Bus/Sources/: 0 matches.

Note: xcodebuild test -only-testing:JarvisAppTests fails at test-host
launch with dyld 'different Team IDs' on Xcode 26 + macOS 26 Debug — this
is a pre-existing repo-level issue documented in
.planning/debug/xctest-launch-runningboard-error-5.md and tracked in
01-HUMAN-UAT.md. Out of scope for Plan 02-03; SPM-level coverage is
complete via BatcherTests + WebviewBridgeOutboundTests.
2026-04-23 18:04:39 -05:00