M-5: Vision surface (typed requestFrameAttach; B-03 closure via WKWebView round-trip) #9

New Issue

2026-05-12T19:02:33Z

KofTwentyTwo commented

2026-05-12 19:02:33 +00:00

(Migrated from github.com)

Goal

Land Vision surface. requestFrameAttach returns typed Result<FrameAttachArmed, FrameAttachError> through a real WKWebView round-trip. Closes B-03 (HUD camera button click does nothing). First atomic webview+Swift commit choreography (see plan §4).

Scope — surfaces in this phase

JarvisAPI.Vision with command requestFrameAttach -> Result<FrameAttachArmed, FrameAttachError>
Guards frameAttachController != nil, never silently returns .success (R-003)
Events: framePending(captureId:), frameSent(turnId:captureId:), frameExpired(captureId:) — captures correlate via FrameCaptureID (UUID)
cancelFrameAttach command + getFrameAttachState query (for late-connect hydration)
Webview HUD camera button switches from frameAttachRequested inbound emit to typed Vision.requestFrameAttach call expecting Result
Deprecated frameAttachRequested inbound case kept for one release

Files (created/modified)

packages/JarvisAPI/Sources/JarvisAPI/Surfaces/Vision+Adapter.swift (new)
App/AppDelegate.swift:2115-2117 — keep deprecated inbound case during transition
webview/src/... — HUD camera button calls typed API (atomic with Swift)
webview/dist/ — rebuild via bash scripts/build-webview.sh and commit
packages/Harness/Tests/HarnessTests/VisionScenarioTests.swift (new)

Test contract

Scenarios S-VIS-001..S-VIS-006. BOTH transport modes mandatory; .jsonRoundTripWebView is NON-SKIPPABLE for S-VIS-001 — this is the gate that proves B-03 dissolved:

S-VIS-001 — requestFrameAttach returns FrameAttachArmed within 100 ms (happy path)
S-VIS-002 — nil controller → Result.failure(.cameraPermissionDenied) (R-003)
S-VIS-003 — 5s frameExpired deterministic via ManualClock (G-002)
S-VIS-004..006 — late-connect hydration, cancelFrameAttach, frameSent correlation

Acceptance criteria

bash scripts/check-vision-isolation.sh green
bash scripts/check-presence-vision-isolation.sh green
All Vision scenarios green via .inProcessActor runner
All Vision scenarios green via .jsonRoundTripWebView runner — S-VIS-001 non-skippable (B-03 dissolution gate)
requestFrameAttach with nil controller returns Result.failure(.cameraPermissionDenied), never .success
Webview dist/ parity with bash scripts/build-webview.sh output
5s frameExpired deterministic via ManualClock
PR merged to develop (atomic webview+Swift squash commit)

Carry-forward closures

Closes #48 (B-03: HUD camera button click does nothing)

Dependencies

Blocked by: #8 (M-4)
Blocks: #10 (M-6 Voice)

Sub-issues

#121 — M-5: define JarvisAPI.Vision.requestFrameAttach typed Result (R-003)
#122 — M-5: Vision events — framePending/frameSent/frameExpired (F-007/F-010)
#123 — M-5: Vision.cancelFrameAttach command + getFrameAttachState query
#124 — M-5: webview HUD camera button — typed Vision.requestFrameAttach call (atomic with Swift)
#125 — M-5: harness scenarios S-VIS-001..006 (B-03 dissolution via WKWebView round-trip)

Estimate

1.5–2.0 engineer-days

Source

.planning/architecture/JARVIS-API-MIGRATION-PLAN.md §2.6 + §4 (webview lockstep choreography)
.planning/architecture/JARVIS-API-DESIGN-v0.1.md — Vision surface
.planning/architecture/JARVIS-API-DESIGN-v0.2.md — F-007, F-010
.planning/architecture/REVIEW-MIGRATION-RISK.md R-003, R-015 partial
Parent epic: v1.0 API Migration — UI/Backend separation (M-0..M-7) (#76)

## Goal Land Vision surface. `requestFrameAttach` returns typed `Result<FrameAttachArmed, FrameAttachError>` through a real WKWebView round-trip. Closes B-03 (HUD camera button click does nothing). First atomic webview+Swift commit choreography (see plan §4). ## Scope — surfaces in this phase - **`JarvisAPI.Vision`** with command `requestFrameAttach -> Result<FrameAttachArmed, FrameAttachError>` - Guards `frameAttachController != nil`, never silently returns `.success` (R-003) - Events: `framePending(captureId:)`, `frameSent(turnId:captureId:)`, `frameExpired(captureId:)` — captures correlate via FrameCaptureID (UUID) - `cancelFrameAttach` command + `getFrameAttachState` query (for late-connect hydration) - Webview HUD camera button switches from `frameAttachRequested` inbound emit to typed `Vision.requestFrameAttach` call expecting `Result` - Deprecated `frameAttachRequested` inbound case kept for one release ## Files (created/modified) - `packages/JarvisAPI/Sources/JarvisAPI/Surfaces/Vision+Adapter.swift` (new) - `App/AppDelegate.swift:2115-2117` — keep deprecated inbound case during transition - `webview/src/...` — HUD camera button calls typed API (atomic with Swift) - `webview/dist/` — rebuild via `bash scripts/build-webview.sh` and commit - `packages/Harness/Tests/HarnessTests/VisionScenarioTests.swift` (new) ## Test contract Scenarios `S-VIS-001..S-VIS-006`. BOTH transport modes mandatory; `.jsonRoundTripWebView` is NON-SKIPPABLE for `S-VIS-001` — this is the gate that proves B-03 dissolved: - `S-VIS-001` — `requestFrameAttach` returns `FrameAttachArmed` within 100 ms (happy path) - `S-VIS-002` — nil controller → `Result.failure(.cameraPermissionDenied)` (R-003) - `S-VIS-003` — 5s `frameExpired` deterministic via `ManualClock` (G-002) - `S-VIS-004..006` — late-connect hydration, cancelFrameAttach, frameSent correlation ## Acceptance criteria - [ ] `bash scripts/check-vision-isolation.sh` green - [ ] `bash scripts/check-presence-vision-isolation.sh` green - [ ] All Vision scenarios green via `.inProcessActor` runner - [ ] All Vision scenarios green via `.jsonRoundTripWebView` runner — `S-VIS-001` non-skippable (B-03 dissolution gate) - [ ] `requestFrameAttach` with nil controller returns `Result.failure(.cameraPermissionDenied)`, never `.success` - [ ] Webview `dist/` parity with `bash scripts/build-webview.sh` output - [ ] 5s `frameExpired` deterministic via `ManualClock` - [ ] PR merged to develop (atomic webview+Swift squash commit) ## Carry-forward closures - Closes #48 (B-03: HUD camera button click does nothing) ## Dependencies - Blocked by: #8 (M-4) - Blocks: #10 (M-6 Voice) ## Sub-issues - #121 — M-5: define JarvisAPI.Vision.requestFrameAttach typed Result (R-003) - #122 — M-5: Vision events — framePending/frameSent/frameExpired (F-007/F-010) - #123 — M-5: Vision.cancelFrameAttach command + getFrameAttachState query - #124 — M-5: webview HUD camera button — typed Vision.requestFrameAttach call (atomic with Swift) - #125 — M-5: harness scenarios S-VIS-001..006 (B-03 dissolution via WKWebView round-trip) ## Estimate 1.5–2.0 engineer-days ## Source - `.planning/architecture/JARVIS-API-MIGRATION-PLAN.md` §2.6 + §4 (webview lockstep choreography) - `.planning/architecture/JARVIS-API-DESIGN-v0.1.md` — `Vision` surface - `.planning/architecture/JARVIS-API-DESIGN-v0.2.md` — F-007, F-010 - `.planning/architecture/REVIEW-MIGRATION-RISK.md` R-003, R-015 partial - Parent epic: #76

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: KofTwentyTwo/Jarvis#9