Commit Graph
9 Commits
Author SHA1 Message Date
james.maes f59f5d12e3 fix(memory): v0.1 CRIT cluster — facts_vec / replay correlation / FTS5 sanitize / search_memory preamble (#12, #13, #14, #15, #16, #153)
#14 (CRIT): ContextBuilder.selfAwarePreamble names search_memory + forget_fact with explicit cue 'call search_memory FIRST. Never claim to remember without searching.'

#16 (HIGH): MemoryQueries.sanitizeFTS5Query strips non-alphanumeric, quotes tokens; HybridSearch.searchFacts calls it before binding to MATCH ?. Embedding still runs on raw query.

#12 (CRIT): MemoryStore.init accepts optional embedder; insertNewFact writes facts_vec after the facts commit. Vec failures log + leave FTS row (no rollback). installMemory shares one embedder between writer + reader.

#15 (HIGH): drop stableHash; pass sentinel 0 as sourceTurnId (option 1 per audit — smaller change surface than threading real rowid through 5 files). Schema comment updated.

#13 (CRIT): two-layer fix. Layer 1: AppDelegate owns a per-process sentinel TurnID; installAgent writes the matching turns row after beginSession; AppDelegateMemoryReplaySink attributes events to it. Layer 2: ReplayLog.flushPending retries failed batches event-by-event so one poisoned row only drops itself.

Memory 98 + Replay 34 + AgentCore 243 tests pass; all 18 boundary gates green.
2026-05-13 06:30:35 -05:00
james.maesandClaude Opus 4.7 048ab08ef6 fix(app,agent,replay): wire orchestrator install correctly + extract bus forwarder for testability
The 2026-05-03 INT-3 smoke test surfaced three latent runtime bugs the
unit-level GSD verifications all missed because the audits were source-
grep, not runtime-trace:

1. **F-E-RACE-1** — `agentInstallTask` awaited memory + vision installs
   but NOT the MCP runtime build task. MCPRuntime build is ~1s due to
   helper child-process spawn, so installAgent ALWAYS short-circuited on
   `mcpRuntime == nil` and the orchestrator + broadcaster + ALL six
   broadcaster subscribers stayed dormant for the entire process
   lifetime. Fix: capture `mcpInstallTask` Task handle and await it
   before calling installAgent. The v0.12.0 milestone audit's claim of
   "5 broadcaster subscribers wired" was accurate at source-grep level
   and false at runtime.

2. **F-E-FK-1** — installAgent constructed the orchestrator with a
   fresh SessionID but never called `replayLog.beginSession(...)`.
   Every `startTurn` violated the FK and returned
   `SubmitOutcome.rejected(reason: .configError)` — surfacing in the
   chat panel as "Config error — see ~/Library/Logs/Jarvis/system.log."
   Production code only ever called beginSession in tests
   (MCPRuntimeWiringTests). Fix: installAgent now reads
   CFBundleShortVersionString + CFBundleVersion from Info.plist, calls
   beginSession, and passes the returned SessionID to the orchestrator.

3. **F-E-WIRE-1** — `OutboundBatcher` was constructed only inside
   `installVoice`, AFTER OpenWakeWord/Silero models loaded. Voice DAG
   short-circuit on missing models in Debug builds left
   `self.outboundBatcher == nil`, and every `outboundBatcher?...`
   chained-optional in the .bus subscriber silently dropped tokenDelta
   / turnStarted / turnEnded / submitRejected. The chat panel got
   nothing back from the orchestrator. Fix: hoist batcher construction
   into installAgent (only depends on webviewBridge, not voice models);
   installVoice reuses self.outboundBatcher rather than overwriting.

In addition, the .bus subscriber's translation logic — including
StopReason → TurnTerminator mapping, turnStarted synthesis on first
emission for a fresh TurnID, error → submitRejected forwarding for
mid-stream failures — was inline in AppDelegate and therefore
impossible to unit-test (App target xctest harness is upstream-broken
on Xcode 26). Extracted to `BusForwarder` in AgentOrchestrator with a
sink protocol; App provides `AppBusForwarderSink` (~50 LOC) as the
glue. Every translation branch now has unit coverage.

Files:
- packages/AgentCore/Sources/AgentOrchestrator/BusForwarder.swift —
  Pure logic. `BusForwarderSink` protocol; `BusForwarder.terminator(for:)`
  StopReason mapping; `BusForwarder.drain(events:sink:)` async loop with
  per-turn lastEmittedTurnId state. No Bus package dep — keeps the
  layering clean.
- packages/AgentCore/Tests/AgentOrchestratorTests/BusForwarderTests.swift
  — 16 cases covering every branch: terminator mapping (5, exhaustive
  on StopReason), tokenDelta forwarding (3 — synthesize start, dedupe
  start across same-turn deltas, two-turn handoff), turnEnd forwarding
  (4 — mapped terminator, synthesize-start-on-bare-turnEnd,
  streamTruncated→errored, state reset), error forwarding (2), ignored
  events (1), malformed-id guard (1).
- packages/Replay/Tests/ReplayTests/SessionForeignKeyTests.swift —
  3 cases (FK1: startTurn-without-beginSession-throws; FK2: post-
  beginSession-succeeds; FK3: multiple-startTurns-share-one-session).
  Closes the bug class regardless of any future AppDelegate refactor.
- App/AppBusForwarderSink.swift — thin transport adapter. Translates
  `BusForwarder.Terminator` to wire-format `Bus.TurnTerminator` and
  dispatches across OutboundBatcher actor.
- App/AppDelegate.swift — race fix, beginSession call, batcher hoist,
  inline subscriber replaced with `BusForwarder.drain(events: sink:)`.
- Jarvis.xcodeproj/project.pbxproj — xcodegen regen picks up new App
  source file.
- .planning/AUDIT-FINDINGS.md — three new findings (F-E-RACE-1,
  F-E-FK-1, F-E-WIRE-1) documented and marked closed; refactor
  rationale recorded; carry-forward items called out
  (streamTruncatedFinal cause unverified without Keychain access; F1
  chat-turn end-to-end test is next).

Verification:
- AgentCore 194/194 (was 178; +16 BusForwarder).
- Replay 33/33 (was 30; +3 SessionFK).
- Bus 55/56 (F-A2-01 pre-existing).
- All 15 boundary grep gates PASS.
- scripts/check-app-builds.sh PASS.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-03 09:44:23 -05:00
james.maesandClaude Opus 4.7 13c8033028 feat(09-02): add VisionRouter dispatch + post-response escalation + voice-turn tracking (BLOCKER-2)
- ReplayEvent: new .escalationAttempt(turnId:kind:) case with .t1ToT2 EscalationKind (D-02 marker, distinct from AGENT-09 stream_truncated)
- ReplayEventKind: new .escalationAttempt = "escalation_attempt" rawvalue
- AgentOrchestrator: constructor gains optional visionRouter: VisionRouter? = nil
- runTurn pre-stream: route image-bearing turns through VisionRouter, swap provider, track imageBearingTurns (D-01, D-16)
- runTurnLoop: multimodal stream overload for image-bearing turns; assistantTextSoFar accumulator on .textDelta
- runTurnLoop .endTurn arm: post-response evaluatePostResponse hook, .escalateToT2 → discard T1 message, swap to T2, continue outer (WARNING-3 LOCKED labeled-loop strategy); records .escalationAttempt under SAME turnId (D-02)
- BLOCKER-2: voiceOriginatedTurns Set<TurnID> + recordVoiceTurn(_:) populator + turnSourceWasVoice(_:) public accessor (Plan 4 voice subscriber will gate emitTurnEnded on this)
- turnHadImage(_:) public accessor (Plan 2 frame-attach release subscriber consumer)
- AgentCore + Replay + MCP packages bumped from .v13 to .v14 (cascade — JarvisVision is .v14, project deployment target was already 14.0)
- MockLLMProvider extended with multimodal stream overload + RecordedCall.images
- 6 AgentOrchestratorVisionDispatchTests + 3 AgentOrchestratorVoiceTurnTrackingTests + 3 EscalationAttemptMarkerTests
- All 63 existing AgentOrchestratorTests still pass; all 30 Replay tests pass; orchestrator-events single-consumer grep gate exits 0

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-01 18:49:25 -05:00
james.maesandClaude Opus 4.7 0e5f860e89 fix(04-review): ME-06 inject ReplayLog batch-window for deterministic L3 test
L3 ("buffer below chunk threshold remains unflushed") relied on the
production 50ms batch-window timer not having fired by the time the
assertion ran. Under CI load the timer could race ahead of the
assertion and L3 would observe `count == 50` instead of `0`, failing
spuriously.

Add an injectable `batchWindowMs` parameter to `ReplayLog.init` (defaults
to `Self.batchWindowMs = 50` so production behavior is unchanged). The
init reads the override into a per-instance `batchWindowMsValue` that
gates the windowed-flush Task.sleep duration.

Update L3 to construct `ReplayLog(databaseURL:, batchWindowMs: 60_000)`
so the 60-second window cannot fire within the test's lifetime. The
only flush trigger that could plausibly fire is the 64-event chunk
threshold, which 50 events doesn't reach. L3 is now deterministic.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-24 19:18:11 -05:00
james.maesandClaude Opus 4.7 1a432fdf9b fix(04-review): ME-05 ReplayLog.flushPending re-buffers + escalates instead of silently dropping
Pre-fix, a transaction failure inside `flushPending` cleared `pending`
before the transaction began. On rollback the entire batch was lost
with no breadcrumb beyond `batch_size` + raw error. This violated
OBS-02's durability claim — silent truncation of the authoritative
byte record on disk-pressure / locked-WAL conditions.

Three changes:
 1. On rollback, re-buffer the dropped batch at the FRONT of `pending`
    so the next flush attempt can retry. Successful flushes reset the
    failure counter.
 2. Capture the affected `turn_ids` in the error metadata so a forensic
    reader can identify which turns lost which events. Redact the
    error message (defense-in-depth alongside HI-01).
 3. After 3 consecutive failures, escalate from `error` to `critical`
    log severity so chronic data-loss conditions surface to the
    DevOverlay / log scrapers. Don't `fatalError` — OBS-02 says replay
    is best-effort and must never propagate into orchestrator turn
    semantics.

Add ReplayLogTests.test_L9_flushFailureRebuffersEvents — happy-path
control verifies the re-buffer path doesn't drop events when flushes
succeed.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-24 19:16:11 -05:00
james.maesandClaude Opus 4.7 bd94699fd0 fix(04-review): ME-03 TokenDeltaDropOldestChannel.send loops instead of recursing
Pre-fix `send` for a non-dropTag element awaited a continuation, then
called `await send(element)` after resume. Swift does not guarantee
tail-call elimination across `await` suspension points, so each
backpressure cycle could add a stack frame. Under pathological consumer
stalls (slow drainer + many protected sends) the producer's stack would
grow proportional to retry depth.

Convert to a `while true` loop. Same shape as `BoundedAsyncChannel.send`
(no recursion). On resume the loop re-checks `finished`, re-tries the
fast paths, and re-attempts the enqueue at the same stack frame.

Add T7 stress test: 5000 protected (toolCall) sends through a capacity-1
channel with a serial drainer. Every send suspends until the previous
element is consumed. Pre-fix this would have recursed 5000 frames deep
in the actor's send invocation chain. Post-fix it stays O(1).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-24 19:14:11 -05:00
james.maes 3aabdd616e feat(04-03): TokenDeltaDropOldestChannel per-element drop policy + JarvisLogChannel .replay/.devoverlay (AGENT-10)
- TokenDeltaDropOldestChannel<Tag>: actor-backed AsyncSequence with per-element drop policy. Overflow drops oldest dropTag elements only; non-dropTag elements suspend the producer (backpressure). Saturated-with-non-dropTag fallback path also suspends rather than dropping a protected event.
- 6 tests: T1 below-cap, T2 oldest-drop on overflow, T3 toolCall NEVER dropped under tokenDelta firehose (AGENT-10 critical invariant), T4 load test 10000 tokenDelta + 1000 toolCall @ cap=2048 — exactly 1000 toolCalls delivered, T5 non-dropTag suspend path, T6 finish drains buffer.
- Logging ChannelTests: targeted assertions for the .replay + .devoverlay additions (LoggingTests.test_sevenChannelsAreDefined updated alongside the enum extension in commit 2614639).
2026-04-24 17:53:43 -05:00
james.maes a7fd068f70 feat(04-03): ReplayLog actor with batched writes + OrphanDetector for OBS-07 crash recovery
- ReplayLog actor: opens replay.db with WAL pragmas + DDL + meta seed; beginSession/startTurn/record/endTurn/close. Writes batched in 50ms window OR 64-event chunks. endTurn appends a turn_end event, flushes, UPDATEs ended_at + stop_reason, then PRAGMA wal_checkpoint(TRUNCATE) for the per-turn fsync. Failures logged via JarvisLogChannel.replay; never propagated.
- OrphanDetector actor: opens its own connection, increments meta.crash_count, runs the OBS-07 query (ended_at IS NULL AND turn_id NOT IN (turn_end events)), stamps each orphan with stop_reason='orphan_recovered' + recovery_marker='detected_at_launch:crash_count=N'.
- 12 new tests: L1-L8 (session/turn/buffering/window/chunk/large-blob/nothing-masked) + O1-O4 (orphan vs non-orphan, race window, crash_count increment, end-to-end SIGKILL parity).
- Adds JarvisLogChannel.replay + .devoverlay (additive — Logging test updated to assert 7 cases). The full Task 3 channel-tests file lands next commit.
2026-04-24 17:53:43 -05:00
james.maes 37b7053903 feat(04-03): scaffold Replay package with hand-rolled SQLite wrapper + schema DDL (OBS-02)
- Package.swift: new SPM library 'Replay' depending on AgentCore + JarvisLogging + Config (Swift 6).
- SQLiteConnection: import SQLite3 (libsqlite3.dylib) — no SQLite.swift dependency (Phase 7 needs sqlite-vec via load_extension). FULLMUTEX, parameterized bindings via SQLiteValue enum, SQLITE_TRANSIENT for text/blob.
- Schema: research §8 verbatim — meta/sessions/turns/events + events_by_turn + turns_by_session indices. Pragmas: WAL, synchronous=NORMAL, busy_timeout=3000, foreign_keys=ON, temp_store=MEMORY. seedMeta with INSERT OR IGNORE.
- ReplayEvent: ten cases with .encoded() returning (kind, payloadBytes); base64 envelope for tool_call_requested + tool_result_full so binary tool results round-trip through JSON.
- ReplayPaths: defaultDatabaseURL = ~/Library/Application Support/Jarvis/replay.db.
- 7 schema tests covering DDL, FK retry_of -> turns(turn_id), BLOB NOT NULL, indices, pragmas, seed, idempotent re-open.
2026-04-24 17:53:43 -05:00