#1 (HIGH): new hasRealT2Provider init param on AgentOrchestrator; t2AvailableForThisTurn now requires both decision.tier != .t3Cloud AND hasRealT2Provider. Production passes false (MissingT2Provider only). Pre-existing escalation test still passes true with mock T2.
#2 (MED): new FrameAttachController.releaseAfterRejectedSubmit() routes through the sole discardFrame() emission site (D-15 single-emission grep gate still green). Both handleChatSubmit and handleChatCancelAndSubmit call it on .rejected.
#3 (MED): AVCaptureSessionWasInterruptedNotification observer yields .midSessionRevoked. macOS limitation surfaced: AVCaptureSessionInterruptionReasonKey is API_UNAVAILABLE(macos), so we cannot discriminate by reason — any interruption surfaces the banner. NSApplication.didBecomeActive observer calls becameAuthorized() on non-authorized → .authorized flip.
Vision 75 + AgentCore 243 tests pass; all 18 boundary gates green.
Adds README.md to each of the 13 SPM packages (AgentCore, Bus, Config,
DevOverlay, Harness, Keychain, Logging, MCP, Memory, Replay, Shell, Vision,
Voice). Each covers purpose, key public types, deps, consumers, invariants
(pulled from CLAUDE.md + grep-gates + audit findings), test stack, and
notable files. Memory README explicitly marks the package functionally OFF
until vec0.dylib + Ollama models land.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Closes vision-audit §7 critical gap: "zero tests exercise
CameraCapture.captureFrame()'s success path. No test asserts that a
non-trivial JPEG comes back. The 1×1 stub passes every test in this
package."
CameraCaptureRealHardwareTests:
- testRealCameraCaptureProducesNonTrivialJPEG: opens a real session,
invokes captureFrame() through the production photoOutput delegate
flow (Track-C 1), and asserts: jpeg byte count > 1000, SOI marker
prefix [0xFF, 0xD8], width > 1, height > 1. The historical 1×1 stub
was 125 bytes — the > 1000 threshold and width/height > 1 guards
make a regression to a trivial fixture trip the test.
- testRealCameraFrameStreamYieldsSamplesWithinTimeout: subscribes to
frameStream(forPresence: true) (Track-C 2 fan-out) and asserts a
PresenceFrameSample.sampleBuffer arrives within 2 seconds of opening
the session.
Gating (skip paths, never prompts):
1. JARVIS_REAL_CAMERA=1 must be set in the environment (matches the
JARVIS_REAL_MODELS convention used by Memory + Voice hardware
tests).
2. AVCaptureDevice.default(for: .video) must be non-nil.
3. AVCaptureDevice.authorizationStatus(for: .video) must be
.authorized — the test does NOT call requestAccess(for:), so a
headless / first-launch test runner never blocks on a TCC prompt.
Run interactively after granting Camera TCC to the test host:
JARVIS_REAL_CAMERA=1 swift test --package-path packages/Vision \
--filter CameraCaptureRealHardwareTests
Vision 70 → 72 (2 tests, both skip without env+TCC). Six boundary
gates pass clean.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Closes vision-audit §4 "confirmSend has zero non-test callers".
FrameAttachController.confirmSend(userText:) was structurally complete
but unreachable from production: requestAttach armed the slot, the
2-second default-cancel timeout fired, the frame was discarded, and the
agent ran text-only. Track-C 5 wires confirmSend into the chat-submit
flow so a user clicking the camera button (Track-C 3) and then sending
text within the confirm window actually delivers the captured frame to
the agent.
App/AppDelegate.swift changes:
- handleChatSubmit now calls a new private helper
`consumePendingFrameIfAny(userText:)` BEFORE submit. When the
controller has a pending frame, the helper pulls the resulting
`ImageBlock` via `confirmSend(userText: text)` and the submit routes
through `TurnInput.withImages(.text, text:, images: [imageBlock])`.
When nothing is pending the path collapses to the original
`submit(.text(text))` — text-only turns are unaffected.
- handleChatCancelAndSubmit gets the same wiring (mirrors the
WARNING-4 pattern that already applied to phrase triggers — without
it, barge-in after a camera-button arm silently drops the frame).
This completes the Track-C 3 click flow:
- Camera button click → BusInbound.frameAttachRequested
→ FrameAttachController.requestAttach(.hudButton) (already wired)
→ CameraCapture.captureFrame() returns real JPEG (Track-C 1 fix)
→ frame parked in pendingFrame slot
- User types + sends within 2s
→ handleChatSubmit consumes the pending frame via confirmSend
→ AgentOrchestrator.submit(.withImages(...)) routes through
VisionRouter.route(...) which selects T1 (or T3 with explicit
cloud opt-in)
Tests: +3 (FrameAttachConfirmSendWiringTests)
- testAppDelegateCallsConfirmSendInChatSubmitPath: structural grep
enforcing the wiring exists.
- testConfirmSendIsReachableFromBothChatHandlers: ensures both
chatSubmit AND chatCancelAndSubmit invoke the helper (WARNING-4
parity).
- testConfirmSendNilWhenNothingPending: confirms the contract the
AppDelegate wiring depends on (no surprise auto-confirm).
Vision 67 → 70. Six boundary gates + check-app-builds pass clean.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Closes vision-audit §3 "T2 is decorative". AppDelegate previously wired
the VisionRouter as `t2Provider: t1`, so `evaluatePostResponse →
.escalateToT2 → providerForTier(.t2LocalQuality) → t1Provider` silently
re-ran the same low-confidence response on the same model with no
operator signal that T2 was never built.
This commit takes the explicit-missing path. Real T2 wiring
(VllmMlxSidecar instantiation + binary bundling + codesigning +
feature-flag gating) is downstream work and out of scope for Track C.
Changes:
- packages/Vision/Sources/Vision/VisionError.swift: new
`.t2ProviderUnavailable` case.
- packages/Vision/Sources/Vision/MissingT2Provider.swift (new):
`LLMProvider` conformer whose text and multimodal `stream(...)`
entry points finish with `VisionError.t2ProviderUnavailable`.
- App/AppDelegate.swift: replaces `t2Provider: t1` with
`MissingT2Provider()`. Comment now documents that callers MUST gate
T2 escalation on `evaluatePostResponse(..., t2Available: false)`,
which routes to `.useT1Result` and never invokes T2 — so this
provider is never actually streamed in the live flow.
Why explicit-missing over real T2: real T2 demands ~6 GB Qwen 3.5 VL
weights, vllm-mlx binary at a stable path, codesigning a nested .app
helper, feature-flag plumbing, and a new TCC surface. The bug being
fixed here is "silent T1-as-T2 fallback hides intent" — making the
fallback explicit (impossible to mistake for working T2) is enough.
Tests: +4 (MissingT2ProviderTests)
- testStreamFinishesWithExplicitError: text path throws
t2ProviderUnavailable.
- testMultimodalStreamFinishesWithExplicitError: image path throws
t2ProviderUnavailable.
- testRouterStillStaysOnT1WhenT2Unavailable: re-asserts router's
no-auto-cloud invariant — production never reaches the missing
provider's stream when wired correctly.
- testAppDelegateNoLongerCarriesSilentT1Fallback: structural grep
enforces no regression to `t2Provider: t1`.
Vision 63 → 67. Six boundary gates + check-app-builds pass clean.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replaces the immediately-finished AsyncStream from 07-04..07-06 with a
real fan-out backed by `AVCaptureVideoDataOutputSampleBufferDelegate`.
CameraCapture changes:
- buildSession() now constructs a `VideoSampleDelegate` and binds it as
the AVCaptureVideoDataOutput's sampleBufferDelegate on a private
user-initiated DispatchQueue with `alwaysDiscardsLateVideoFrames`
enabled (RESEARCH §9 / D-11 throttle posture).
- frameStream(forPresence:) now generates a UUID per subscription, hops
into the actor to register the continuation against the live delegate,
and tears down via `cont.onTermination` on consumer cancellation.
When the session is not yet open, the stream finishes immediately
(consumers' for-await exits cleanly).
- shutdown() drains the delegate's subscription set via
`finishAllStreams()` so outstanding consumers terminate.
VideoSampleDelegate is a small NSObject (`@unchecked Sendable`) holding
an NSLock-guarded `[UUID: Continuation]` map. The fan-out path is:
captureOutput → snapshot under lock → yield outside lock. PresenceMonitor
is the only in-tree consumer today.
Tests: +4 (CameraCaptureFrameStreamTests)
- testFrameStreamYieldsInjectedSamplesAfterOpen: opens a real session
(XCTSkip if no camera), asserts an injected sample reaches the stream.
- testShutdownFinishesOutstandingStreams: shutdown() exits a draining
for-await within 200 ms.
- testFrameStreamFinishesImmediatelyWhenNotOpen: pre-open semantics.
- testFrameStreamSourceNoLongerImmediatelyFinishes: structural grep
gate against re-introducing the empty-stream stub.
Vision 59 → 63. Six boundary gates + check-app-builds pass clean.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replaces the 1×1 black-JPEG hardcode (vision-audit §1) with the real
AVCapturePhotoOutput delegate flow.
CameraCapture.captureFrame() now:
- Builds AVCapturePhotoSettings(format: jpeg) and invokes
photoOutput.capturePhoto(with:delegate:).
- Bridges the AVCapturePhotoCaptureDelegate callback into a
CheckedThrowingContinuation via a new internal PhotoCaptureProxy
(NSObject conformance — actors can't conform to the AVFoundation
delegate protocols). The proxy retains itself for the lifetime of
the capture (AVCapturePhotoOutput holds the delegate weakly), and
breaks the cycle on the first didFinishProcessingPhoto callback.
- Reads width/height from photo.resolvedSettings.photoDimensions
with a CGImageSource fallback for hosts that don't populate it.
Test seam: photoCaptureOverride lets unit tests exercise captureFrame()
end-to-end (including the session.isRunning precondition) without real
hardware. Mirrors the existing authStatusProbe pattern.
Tests: +3 (CameraCaptureDelegateTests)
- testCaptureFrameRoutesThroughOverrideSeam: opens a real session
(XCTSkip if no camera), asserts the override seam returns the
fixture frame.
- testCaptureFrameThrowsSessionNotRunningBeforeOpen: precondition
guard preserved.
- testCaptureFrameSourceDoesNotEmbedHardcodedJPEG: structural grep
gate against re-introducing the onePixelJPEG hardcode.
frameStream(forPresence:) is left as the empty-stream stub for now —
that's track-c 2.
Vision 56 → 59. Six boundary gates + check-app-builds pass clean.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per `.planning/audit-2026-05-03/tests-audit.md` section 5, five
`XCTAssertTrue(true)` placeholder tests were identified across the tree
that are mathematically incapable of failing. This commit fixes or
removes each, conservatively. Files touched and disposition:
- App/Tests/AppTests/MCPRuntimeWiringTests.swift:260
fixed: replaced trailing `XCTAssertTrue(true)` rubber-stamp at end of
CR-02 channel-drain test with a load-bearing assertion that the drain
task completed via `finish()`, not cancellation. The implicit `await
drainTask.value` is the real gate; the new assertion ensures regressions
that cancel-instead-of-finish surface.
- App/Tests/AppTests/WebviewBundleLoadTests.swift:60
deleted: `test_assetsDirectoryGitignored_documentedInSmokeScript`. Body
was `XCTAssertTrue(true)` with comment "intentionally has no assertion
— it exists to document the split of responsibility". Documentation,
not a test. Replaced with a top-level comment block documenting the
same split (smoke-test-hud.sh is the runtime gate).
- packages/AgentCore/Tests/AgentCoreTests/ToolChoiceTests.swift:38
fixed: removed trailing `XCTAssertTrue(true) // Test compiles → contract
holds.` from `testLLMProviderStreamRequiresToolChoice`. The closure
body itself is the contract — if the test compiles, the contract holds.
Comment updated to make the compile-time semantics explicit.
- packages/Shell/Tests/ShellTests/InputMonitoringDenialTests.swift:36
deleted: `test_localMonitorOnlyFallback`. Body was `XCTAssertTrue(true)`
documenting that the real fallback test lives in HotkeyBindingTests.
Replaced with a top-level comment pointing readers there.
- packages/Vision/Tests/VisionTests/PackageBoundaryTests.swift:14
deleted: `testVisionModuleCompilesStandalone`. Body was
`XCTAssertTrue(true)` with comment "compile is the assertion." VISION-03
isolation is enforced by `scripts/check-vision-isolation.sh` and the
Package.swift dependency graph, both of which are real gates. Folded
the documentation into the `testPublicAPISurface` doc-comment so the
explanation isn't lost.
HudStateEnumTests rubber-stamps (audit items #4–6 in section 5) were
left as-is. The voiceOverLabel literals are user-facing accessibility
text that VoiceOver actually speaks; they are load-bearing in the sense
the audit calls out as worth preserving ("replace with behavioral
assertions if the literal under test is load-bearing"). A regression
that breaks an a11y label IS a real regression. Conservative read; not
worth the blast radius of guessing wrong.
Pre-existing baseline issues unrelated to this batch:
- `packages/Shell/Tests/ShellTests/InputMonitoringDenialTests.swift`
fails to COMPILE on a clean tree (MockProbe missing
`isListenEventAccessGranted()`). Verified via `git stash` that the
failure is independent of this commit. Not addressed here per the
scope rules — flagged for follow-up.
Net test count: -3 tests (3 deleted, 0 added — the
MCPRuntimeWiringTests assertion replaces the tautology in-place).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- New PresenceStateSnapshot actor (.shared singleton + record + currentEnrichment); D-14 plain-sentence rendering with 5s present-suppression and 5min absent-threshold rules
- ContextBuilder.installPresence body now drains the AsyncStream into PresenceStateSnapshot.shared.record (replaces the 07-06 deferred-wiring no-op)
- VISION-03 boundary preserved: snapshot returns only String?; no Voice or AgentOrchestrator refs
- 10 PresenceStateSnapshotTests pass (initial nil, suppression, present, absent <5min/=7min, absentLongTerm, unknown, overwrite, type-only, ContextBuilder round-trip)
- AppDelegate gains installMemory() and installVision() mirroring installVoice's
six-step pattern. Memory installs BEFORE voice (memory ready before first
turn); vision AFTER voice (camera lifecycle independent).
- MemoryStore + MemoryExtractionOrchestrator + MemoryExtractionCoordinator
constructed; AppDelegateMemoryReplaySink bridges Memory.MemoryReplaySink ->
Replay.ReplayLog.record(_:for:). Coordinator wiring deferred until
AgentOrchestrator lands in AppDelegate (documented in SUMMARY).
- CameraCapture + PresenceMonitor instantiated; the SINGLE PresenceSignalBus
reference (owned by PresenceMonitor) flows via AppDelegateContextBuilderAdapter
to ContextBuilder.installPresence and via hudStateCoordinator?.attachPresence
to the HUD coordinator. VISION-03 boundary preserved.
- VisionRouter constructed with T1=Ollama, T2=Ollama (sidecar deferred), T3=
Anthropic (key from Keychain); the runTurn dispatcher branch is deferred
until AgentOrchestrator wires up.
- DisablePresence menu-bar toggle wired alongside MuteWakeWord.
- BannerContent.cameraDenied + cameraRevoked presets; cameraDegradationTask
watches CameraCapture.degradationStream and enqueues banners.
- HudStateCoordinator.attachPresence is a generic AsyncSequence subscriber
preserving HUD-08 single-writer invariant (presence is record-only).
- ContextBuilder.installPresence(_:) is a static no-op drain (deferred wiring
for the future per-turn system-prompt enricher).
- TS BUS_PROTOCOL_VERSION 2.0.0 -> 2.1.0; sessionHistory + TurnRow added to
the BusOutbound discriminator union with full decode arm. Round-trip test
bumped accordingly. scripts/check-bus-protocol-version.sh now passes.
- FrameConfirmationDecision: .send/.cancel/.timeout (default-cancel
semantic — anything other than .send discards)
- FrameAttachController actor:
* requestAttach(reason:) is the SINGLE entry point — phrase + HUD
button both reach it (D-13 dual-trigger)
* confirmSend(userText:) returns the ImageBlock the orchestrator
routes; replays the placeholder via FrameAttachReplaySink
* cancel() and timeout-fired both discard via the SOLE emission site
* onAssistantTurnComplete() releases bytes on LLMEvent.messageStop edge
* discardFrame() is the SOLE function that clears pendingFrame —
'pendingFrame = nil' appears EXACTLY ONCE in the file
* Cancellation-aware timeout task (do/catch around Task.sleep) so
confirmSend cancellation cleanly suppresses .timeout firing
- FrameAttachReplaySink: actor that records the placeholder JSON via
the injected ReplayLogProtocol; raw image bytes never enter the sink
- ReplayEvent.userInput doc comment: documents the D-15 placeholder
convention for image-bearing turns
- All 10 new tests + 37 existing tests pass; 4-leg grep gate enforces
single emission + raw-bytes egress fence
- VisionTier enum with t1Local/t2LocalQuality/t3Cloud + defaultModelID
mapping; VisionTier.swift is the SOLE source of the literal
'claude-opus-4-7' / 'gemma4:31b' / 'Qwen/Qwen3.5-35B-A3B-VL' strings
- VisionRouterConfig public struct with named planner-tuned fields
(substring set, length threshold, phrase sets, timeouts)
- VisionEscalationHeuristic.evaluateLowConfidence — pure function,
reads from passed config (no hidden defaults)
- EscalationPhraseDetector — word-bounded case-insensitive matchers
for cloud opt-in (D-18) and frame-attach (D-13) phrase sets
- ContextBuilder — sole non-test caller of EscalationPhraseDetector
- VisionRouter actor: route() never returns T2 directly; T3 only on
explicit opt-in; evaluatePostResponse() never returns escalateToT3
(D-18 no-auto-cloud invariant)
- All 16 new tests + 21 existing tests pass; grep gate enforces structural
no-orphan-cloud-reference invariant
Verbatim clone of packages/Voice/Sources/Voice/Control/MuteWakeWord.swift
adapted to PresenceMonitor:
- @MainActor final class
- UserDefaults key: features.vision.presenceDisabled
- NSMenuItem with title 'Disable presence'; state mirrors isDisabled bool
- handleToggle() flips state, persists, and dispatches pause()/resume() to
PresenceMonitor on a fire-and-forget Task
- toggleForTest() seam for synchronous test driving
D-12 invariant enforced: this class touches PresenceMonitor only —
CameraCapture is intentionally untouched so frame-attach (VISION-04)
remains usable while presence is paused. testDisablePresenceDoesNotTouchCameraCapture
opens DisablePresence.swift on disk, strips comment lines, and asserts
'CameraCapture' does not appear in non-comment lines.
Tests (4 cases, all PASS):
- testInitReadsPersistedDisabledState: defaults seeded true → isDisabled
true, menu item state .on
- testInitReadsPersistedEnabledState: defaults seeded false → isDisabled
false, menu item state .off
- testToggleFlipsStateAndPersists: toggle round-trips state, persistence,
and menu item state
- testDisablePresenceDoesNotTouchCameraCapture: static-grep invariant
All 14 Vision tests pass; both VISION-03 grep gates green.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Vision presence pipeline (signal-only, VISION-02 / VISION-03):
- PresenceEvent / Presence enums (Sendable, Equatable). Three published
states: .present, .absent(since:), .absentLongTerm(since:); .unknown is
the pre-first-frame sentinel and is never published.
- PresenceSignalBus: a bare AsyncStream-wrapper struct with public read
side and internal init — there is no subscriber registry, no broadcast
fan-out, no inbound API. Only PresenceMonitor constructs the bus.
- CameraCapture actor: AVCaptureSession lifecycle owner with TCC switch,
authStatusProbe test seam (mirrors AudioGraphOwner), open()/shutdown()
pair, becameAuthorized() entry for D-09 first-grant flow, single-frame
captureFrame() with sessionNotRunning guard (stub JPEG body until 07-05
wires AVCapturePhotoOutput delegate), nonisolated frameStream(forPresence:)
returning an empty stream until 07-06 wires the delegate.
- Mid-session AVCaptureSessionRuntimeError observer captures only
Sendable values (Continuation + label string) so the @Sendable closure
crosses the actor boundary cleanly under Swift 6 strict concurrency.
- CapturedFrame value type for VISION-04 frame-attach (consumed by 07-05).
- PresenceMonitor actor: WakeWordDAG-shaped (actor + nonisolated bus +
Task.detached feed loop + paused/resume + cancel). Edge-debounced 2.0s
state machine; 5-min absent secondary threshold; feedObservation +
evaluateLongTerm test seams (both honor paused for VOICE-12-style
parity). VNDetectFaceRectanglesRequest paths in for sampleBuffer +
cgImage; tests exercise the .syntheticDetection path.
- PresenceFrameSample: `@unchecked Sendable` because CMSampleBuffer +
CGImage are reference types with no Swift Sendable conformance; the
enum is single-producer / single-consumer in this codebase.
Tests (10 cases across 4 files, all PASS):
- PackageBoundaryTests: module compiles standalone; public API surface.
- PresenceSignalBusTests: PresenceEvent equality + bus delivery order.
- PresenceMonitorDebounceTests: sub-2s blips suppressed; 5-min long-term
fires; pause stops emissions.
- CameraCaptureTCCTests: TCC denied yields banner+throws; notDetermined
throws without banner; captureFrame throws when session not running.
Deviations from plan body (Rule 3 — blocking, deviation_rules):
1. Module renamed Vision -> JarvisVision. The plan's module name `Vision`
collides with Apple's Vision framework: when a file inside module
`Vision` writes `import Vision`, Swift silently treats it as a self-
import and the framework symbols (VNDetectFaceRectanglesRequest etc.)
never resolve. `JarvisVision` matches the existing JarvisLogging
convention. The package directory stays packages/Vision; the grep
gates key off the directory path, not the module name. Tests use
@testable import JarvisVision.
2. project.yml product entry: `product: JarvisVision` instead of `Vision`
— same root cause as #1.
3. testPauseStopsEmissions: feedObservation must honor the paused gate
identically to consume(); the plan's body skipped that guard inside
feedObservation, which would have made the paused test fail. Fix is a
one-line `guard !paused else { return }` at the top of feedObservation.
4. testEmitsAbsentLongTermAfter5Minutes: bumped the evaluator timestamp
from `t0 + 5*60 + 1` to `t0 + 3 + 5*60 + 1` because the second
feedObservation at `t0+3` is what publishes `.absent(since: t0+3)`,
so the 5-min threshold check uses t0+3 as the reference. The +3s
alignment matches the production semantics literally.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- New packages/Vision SPM target (macOS 14, swift-tools 6.0, strict-concurrency v6).
- Dependencies: Logging + AgentCore + swift-log only (NO Voice, NO AgentOrchestrator).
This is the compile-time half of VISION-03; the textual halves are the two
scripts below.
- VisionError (TCC + capture errors) and VisionDegradationReason (cameraDenied,
midSessionRevoked) — analogs of Voice's Error / DegradationReason.
- scripts/check-vision-isolation.sh: fails build on any `import Voice` /
`import AgentOrchestrator` under packages/Vision/Sources.
- scripts/check-presence-bus-no-tts-orchestrator.sh: fails build on any
PresenceSignalBus reference inside packages/Voice/Sources or
packages/AgentCore/Sources/AgentOrchestrator (non-comment lines only).
- Wired Vision into project.yml (XcodeGen) — workspace has no top-level
Package.swift; mirrors the Memory wiring from 07-01 (same deviation
pattern, same rationale).
- Updated NSCameraUsageDescription to the plan's wording (key was already
present from earlier scaffolding).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
james.maes
