The codebase moved to qwen3.6:latest (memory extractor) and
qwen2.5-coder:32b-instruct-q8_0 (agent fallback) in commit 8d0a959, but
CLAUDE.md and the MemoryRegressionCorpusTests doc-comments still cited
the vanilla qwen2.5-coder:32b string. Both updated to match what
ModelID.swift, MemoryExtractor.swift, and BootHealth.swift actually
use today.
No behavior change — documentation alignment only.
Closes#58
#14 (CRIT): ContextBuilder.selfAwarePreamble names search_memory + forget_fact with explicit cue 'call search_memory FIRST. Never claim to remember without searching.'
#16 (HIGH): MemoryQueries.sanitizeFTS5Query strips non-alphanumeric, quotes tokens; HybridSearch.searchFacts calls it before binding to MATCH ?. Embedding still runs on raw query.
#12 (CRIT): MemoryStore.init accepts optional embedder; insertNewFact writes facts_vec after the facts commit. Vec failures log + leave FTS row (no rollback). installMemory shares one embedder between writer + reader.
#15 (HIGH): drop stableHash; pass sentinel 0 as sourceTurnId (option 1 per audit — smaller change surface than threading real rowid through 5 files). Schema comment updated.
#13 (CRIT): two-layer fix. Layer 1: AppDelegate owns a per-process sentinel TurnID; installAgent writes the matching turns row after beginSession; AppDelegateMemoryReplaySink attributes events to it. Layer 2: ReplayLog.flushPending retries failed batches event-by-event so one poisoned row only drops itself.
Memory 98 + Replay 34 + AgentCore 243 tests pass; all 18 boundary gates green.
Matches commit 8d0a959 model flip — extractor default is now
ModelID.qwen36 ("qwen3.6:latest"). The previous test name + assertion
were stale and failed on `swift test --package-path packages/Memory`
with the "Optional(qwen3.6:latest)) is not equal to (qwen2.5-coder…)"
diff. No production code change.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Closes D-5 / D-6 (sqlite-vec dylib bundling) carry-forwards from the
2026-05-04 audit by adopting jkrukowski/SQLiteVec as a SPM dependency.
The package's CSQLiteVec target compiles sqlite3.c + sqlite-vec.c into
one statically-linked C target with `core_vec_init()` registering vec0
as an auto-extension. All sqlite3_* calls in Replay's SQLiteConnection
and Memory's MemoryStore resolve to this bundled SQLite (vec0 always
available) instead of Apple's stripped /usr/lib/libsqlite3.dylib.
The retired hand-rolled approach (dlsym pathway + JARVIS_VEC0_STUB_PATH
env-flag + Bundle.module placeholder text file) never worked on macOS
because Apple stripped sqlite3_load_extension from the system libsqlite3.
Memory init has been silently dormant since Phase 7 (2026-04 onward),
which is what made B-02 (history threading) appear permanently broken
to the user during dogfooding even after the 48ad8d6 tactical patch
landed.
Live verification 2026-05-11: fresh launch with empty jarvis.db creates
schema, persists turn rows across submits, B-02 multi-turn context
resolves correctly ("what breed is Toby?" → "Golden Retriever" after
prior turn established Toby is a Golden Retriever), and the new
get_memory_stats tool reports `turnsCount=6, vecVersion=v0.1.6,
sqliteVersion=3.51.0`.
Changes:
- packages/Replay/Package.swift
+ SQLiteVec SPM dependency (from: "0.0.14")
+ CSQLiteVec product wired into Replay target
- packages/Replay/Sources/Replay/SQLiteConnection.swift
- import SQLite3
+ @_exported import CSQLiteVec
All sqlite3_* call sites unchanged — linker now resolves to our
bundled sqlite3 with vec0 statically linked in.
- packages/Memory/Package.swift
+ SQLiteVec SPM dependency (named product so MemoryStore can import
CSQLiteVec directly for the core_vec_init() one-shot call)
- resources: PLACEHOLDER.txt copy (defunct vec0.dylib stub slot)
- packages/Memory/Sources/Memory/MemoryStore.swift
- import SQLite3 → import CSQLiteVec
+ _vecAutoExtensionRegistered static-let one-shot: calls
CSQLiteVec.core_vec_init() which registers vec0 via
sqlite3_auto_extension on the first MemoryStore construction.
Idempotent (static-let initializer fires exactly once).
- loadVecExtension(on:) dlsym/JARVIS_VEC0_STUB_PATH/Bundle.module
pathway (retired; ~70 lines removed)
Updated docstring to reflect the new hard-fail boot policy.
- packages/Memory/Sources/Memory/Resources/PLACEHOLDER.txt — deleted
- App/AppDelegate.swift installMemory()
Reverses Track-D D-1 (75a10be, 2026-05-04): MemoryStore.init failures
are now FATAL via Logger.critical + TCCAlertService.presentHardBlock
+ NSApp.terminate. The graceful-degradation path that kept the app
booting with memoryStore=nil hid the entire B-02/B-04/B-05 silent-
failure bug class. Aligns with the 2026-05-07 API-first pivot
thesis: "no silent failures".
`store` is now non-optional throughout installMemory; downstream
callers no longer need `if let store = store`.
- packages/MCP/Sources/MCP/InProcess/GetMemoryStatsTool.swift (new)
+ MemoryStatsDispatching protocol
+ MemoryStats Codable struct (turnsCount, factsTotal, factsActive,
sessionsCount, dbFileBytes, vecVersion, sqliteVersion,
lastTurnAtUnixMs, lastFactAtUnixMs)
+ GetMemoryStatsTool conforming to InProcessTool
- App/MCP/InProcessMemoryAdapters.swift
+ MemoryStatsStoreAdapter — bridges the protocol to MemoryStore's
public queryRowCount / querySingleString seams and FileManager
for the on-disk file size. Read-only, no confirmation.
- App/AppDelegate.swift installSelfKnowledgeTools()
+ Registers GetMemoryStatsTool with MemoryStatsStoreAdapter.
- packages/Memory/Tests/MemoryTests/*
Retired the JARVIS_VEC0_STUB_PATH XCTSkipUnless guards from 9 tests
(vec0 is always available now — no need to env-gate). Rewrote
MemoryStoreTests to drop the now-impossible "init throws when vec
missing" case and assert facts_vec virtual table creation +
vec_version() readback. Added seedTurnRow helper to
MemoryStoreApplyOpTests so the now-enforced
`facts.source_turn_id REFERENCES turns(id)` foreign key is honored
in tests that previously passed when FK enforcement was dormant.
Test summary: Memory full suite 91 tests, 10 skipped (env-gated real-
Ollama tests), 0 failures. App target builds clean. Substrate verified
end-to-end via live launch (vec_version: v0.1.6 logged at boot;
get_memory_stats returns actual schema row counts).
Pending D-7 carry-forward: ollama pull nomic-embed-text (~270MB) for
embedder; user already has qwen2.5-coder:32b-instruct-q8_0 which is
a higher-quality variant of the project's hardcoded
qwen2.5-coder:32b — small ModelID config delta to follow.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Carry-forward bug B-02 (live evidence 2026-05-07): conversation
continuity broken — "yes" / "why 4.5?" lose prior turn context because
AgentOrchestrator.process(_:) built [system, user(input.userText)]
with no history fetch.
Tactical patch outside the M-0..M-7 migration sequence, per
JARVIS-API-MIGRATION-PLAN.md §3 and UQ-2 (option B). Lands on develop
directly. M-4 (Memory surface) will reframe this path against the new
API contract but the user-visible bug closes today.
Substrate gap discovered + addressed in-scope. The `turns` table was
unwritten (schema present at MemorySchema.swift, FTS5 trigger ready,
read API at MemoryStore.recentTurnsForSession, but no INSERT site
anywhere in the codebase). The migration plan assumed turn writes
existed; they did not. Patch adds the missing writer plus the read
side wiring.
Changes:
- packages/Memory/Sources/Memory/MemoryQueries.swift
+ turnInsertSQL — single-row INSERT into turns
- packages/Memory/Sources/Memory/MemoryStore.swift
+ appendTurn(sessionId:role:content:source:createdAt:)
Single-row INSERT; FTS5 trigger mirrors content into turns_fts
automatically. T-06-05-03: function MUST NOT log content.
- packages/AgentCore/Sources/AgentOrchestrator/AgentOrchestrator.swift
+ PriorTurn struct + SessionHistoryLookup typealias
+ emptySessionHistoryLookup default (preserves pre-B-02 test setups)
+ sessionHistoryLookup ctor parameter
Replaces the [system, user] messages-array build with prior-history
hydration: system + chronological prior user/assistant rows + current
user. Lookup throws -> warning log + empty history (matches D-3
PriorFactsLookup resilience contract).
- App/AppDelegate.swift
+ sessionHistoryLookup closure: recentTurnsForSession(limit: 10),
reverse DESC -> chronological, filter to user/assistant only, return
[PriorTurn]
+ extends existing turnContent flush closure with appendTurn calls
(rides on memory-extraction drain path; assistant row offset +1 ms
so DESC + id-tiebreaker preserves user->assistant order)
Tests:
- packages/AgentCore/Tests/AgentOrchestratorTests/HistoryThreadingRegressionTests.swift
HT-1: third turn includes prior user/assistant pairs in chronological
order. 3/3 PASS.
HT-2: empty-history default produces [system, user] (pre-B-02
baseline preserved).
HT-3: three-turn happy path emits zero LLMProviderError
.streamTruncatedFinal events. Pins R-006 / Phase E gate behavior:
patch grows messages array, not finalSystem, so
CacheHints.eligibleForSystemPrompt(finalSystem) is unchanged.
- packages/Memory/Tests/MemoryTests/MemoryStoreAppendTurnTests.swift
AT-1: appendTurn round-trips via recentTurnsForSession.
AT-2: user/assistant pair reverses to chronological.
AT-3: session_id scoping isolates rows.
AT-4: turns_fts trigger fires on append.
All env-gated on JARVIS_VEC0_STUB_PATH per existing MemoryStore-test
convention; skip cleanly without vec0.dylib.
Verification:
- check-app-builds.sh PASS.
- Boundary gates sampled (check-no-leftover-stubs, applescript-confirm,
corpus-secrets, bus-protocol-version, single-memory-mutated-emit,
single-writer-hudstate) PASS.
- AgentCore affected-subset (Submit/Retry/TextEnd2End/CapRecovery/
HistoryThreading) 26/26 PASS, 0 failures.
- Memory full suite 92 run, 23 skipped (env-gated), 0 failures.
Degradation contract: when MemoryStore is nil (dormant memory — vec0
or Ollama unavailable), turn persistence is also dormant; the
appendTurn call is guarded by memoryStore != nil. When D-5/D-6 close
and memory comes up live, persistence activates without further code
changes.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per audit 2026-05-04 security MEDIUM-1: HybridSearch.searchFacts logged
the raw user query string at debug level. Voice transcripts are guarded
by T-06-05-03; memory queries are equivalent user-private content.
Replace `query='\(query)'` with `queryLen=\(query.count)` — log
metadata only, never content. Add regression test that grep-asserts the
source file at runtime so a future log-content regression is caught at
test time.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds `RememberBrutusEndToEndTests` proving the extraction -> store ->
search loop holds end-to-end. Without this test, Tracks D-1..D-3 each
move a single component but no test covers their composition; a future
regression that severs the loop at the extraction/search boundary
would slip past per-component tests.
Scenario:
1. User: "My dog Brutus is a Bernese Mountain Dog."
2. Stub LLM provider returns `apply_memory_ops(ADD subject=Brutus
predicate=breed object="Bernese Mountain Dog")`.
3. Real `MemoryExtractionOrchestrator` + `MemoryExtractor` drive the
ApplyOp callback into a `FakeStore` actor.
4. User: "What breed is Brutus?"
5. Real `HybridSearch` actor + stub embedder + `FakeStore`'s
`MemoryReadStore` conformance return one hit; assert
`summary == "Brutus breed Bernese Mountain Dog"` and the trigger
turnId propagated to retrieval recording.
Fakes used:
- `FakeStore` actor — captures applyOp via custom `applyAdd`, conforms
to `MemoryReadStore` so `HybridSearch.searchFacts` runs unmodified.
No real SQLite, no vec0.dylib needed.
- `StubEmbedder` — constant 768-dim vector (real path is
`OllamaEmbeddingClient` -> `nomic-embed-text`).
- `BrutusProvider` — single-shot stub `LLMProvider` emitting one
`toolUseRequested` then `messageStop`.
Boundary: stops at `HybridSearch.searchFacts`. The MCP-side dispatch
through `SearchMemoryTool` is covered by
`InProcessMemoryToolsTests.testSearchMemoryToolDispatch` in the MCP
package. Together those two tests cover the path the agent takes:
prompt -> extractor -> store -> search tool -> agent context.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Pre-D-3, `MemoryExtractionOrchestrator.process(_:)` hardcoded
`priorFacts: []` on every job. The mem0 extractor was therefore blind
to existing facts and could only emit ADD ops — UPDATE/supersede
never fired, so superseded values silently duplicated on every turn.
This commit:
- Adds `PriorFactsLookup` typealias and `priorFactsLookup` parameter on
`MemoryExtractionOrchestrator.init`. Default is `emptyPriorFactsLookup`
to preserve pre-D-3 behavior for tests that don't care about priors.
- Wires `process(_:)` to call `priorFactsLookup(job)`. Lookup throws
degrade to `priorFacts = []` with a warning log — the drain task
must never die because a single SQL read failed.
- Adds `MemoryStore.recentActiveFacts(limit:)` + matching SQL in
`MemoryQueries.recentActiveFactsSQL`. Active-only (`valid_to IS NULL
AND forgotten_at IS NULL`), most-recently-created first, server-side
cap. D-02 forgotten facts excluded.
- AppDelegate's `installMemory` passes a closure that returns up to 50
recent active facts. 50 is a deliberate cap — RESEARCH §5 caps the
rendered priorFacts block at ~4 KB (~50 facts at typical S/P/O
length). Closure returns [] when `memoryStore` is nil, preserving
the pre-D-3 no-store path.
Tests:
- `testD3_priorFactsLookupDrivesUpdateOp` — stub provider that emits
UPDATE when it sees a `[99]` prior fact id in the system prompt;
asserts `applyOp` receives `.update(supersedes: 99, …)`. Pre-D-3
this would have been `.add` because the orchestrator never told
the model about the prior fact.
- `testD3_priorFactsLookupErrorDegradesToEmpty` — lookup throws,
job still applies its ADD op (drain survives the failure).
T-06-05-03 invariant respected: warnings log turnId only, never
fact content.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- AppDelegate.installAgent() constructs AgentOrchestrator with the
existing 7-arg ctor (visionRouter / presenceSnapshot stay nil for
Plans 2 + 3), starts the OrchestratorEventBroadcaster, and creates
the TurnTranscriptStore. Memory + transcript + devOverlay subscribers
drain into MemoryExtractionCoordinator, the assistant-side
accumulator, and a no-op consumer respectively.
- ConfigStore is now built in applicationWillFinishLaunching from the
loaded launch + perTurn snapshots (was previously discarded).
- installMemory no longer calls coord.start; the wiring moved to
installAgent step 5a where the broadcaster's memory child stream is
available. Old agentOrchestratorEvents() nil-stub is gone.
- MemoryExtractionCoordinator gains a MemoryEnqueueing protocol seam
so the BLOCKER-1 end-to-end test can spy on enqueue without driving
the full extractor stack.
- New scripts/check-orchestrator-events-single-consumer.sh enforces
D-05/D-08 (exactly one production for-await over orchestrator.events;
WARNING-2 whitelist scoped to OrchestratorEventBroadcaster.swift).
Wired as a project.yml preBuildScript and exercised in-process by
PhaseNineGrepGateTests.
- MemoryWiringEndToEndTests proves enqueue receives non-nil userText +
assistantText through real broadcaster + transcript store + memory
coordinator wiring (BLOCKER-1 closure for INT-07-01).
- Test home note: MemoryWiringEndToEndTests lives in MemoryTests
(not AgentOrchestratorTests as the plan suggested) because Memory
depends on AgentOrchestrator via SPM and the cycle would block
test compilation. Recorded in the plan SUMMARY.
- start<S: AsyncSequence & Sendable> where S.Element == OrchestratorEvent
replaces the BoundedAsyncChannel<OrchestratorEvent>-only signature.
- AsyncSequence's throwing iterator is wrapped in do/catch with a new
warnDrainError helper; the start function stays non-throwing.
- BoundedAsyncChannel already conforms to AsyncSequence so existing
call sites compile unchanged (verified by 8 prior coordinator tests).
- New test testStartAcceptsAsyncStream drives the coordinator through
an AsyncStream — the type the OrchestratorEventBroadcaster's
subscribers emit — proving the broadcaster will plug in directly.
- scripts/check-single-memory-mutated-emit.sh — MEM-06 single-emission-site
for ReplayEvent.memoryMutation (allowlist: MemoryStore.swift,
Replay/ReplayEvent.swift, Replay/Schema.swift; tests + comments excluded).
- scripts/check-single-memory-used-emit.sh — D-05 single-emission-site for
ReplayEvent.memoryRetrieval (same allowlist shape, MemoryStore.recordRetrieval
is the sole emit site).
- scripts/check-presence-vision-isolation.sh — VISION-03 three-layer guard:
Layer 1 (Vision/Package.swift declares no Voice or AgentOrchestrator dep),
Layer 2 (no `import Voice` / `import AgentOrchestrator` and no TTSEngine /
AgentOrchestrator.runTurn / cancelAndSubmit token references in
packages/Vision/Sources/), Layer 3 (no App/ file co-locates a
PresenceSignalBus reference with a forbidden TTS / runTurn token).
- scripts/check-embedding-dim-literal.sh — MEM-02 invariant: no literal 768
in packages/Memory/Sources/Memory/ outside Constants.swift; comment-only
lines tolerated.
- All four scripts wired as Jarvis pre-build phases in project.yml; the
generated project.pbxproj reflects the updates.
- packages/Memory/Tests/MemoryTests/PhaseSevenGrepGateTests.swift mirrors
the four shell gates as XCTest assertions plus a fifth assertion
(testAppDelegateInstallOrder) verifying the call order in
applicationWillFinishLaunching: installMemory before installVoice
before installVision.
- App/Voice/NullVoiceAdapters.swift extracted from AppDelegate.swift to
honor the Layer 3 boundary cleanly. NullOrchestratorAdapter and
NullTTSAdapter are placeholder no-ops mirroring the prior structure;
Phase 7 future wiring replaces them with real bridges.
- Negative-case demonstration completed: introducing a stray
`.memoryMutation(...)` literal in App/AppDelegate.swift made
check-single-memory-mutated-emit.sh exit non-zero; the violation was
reverted before commit. Documented as evidence of gate sensitivity.
- MemoryRegressionCorpusTests with 10 scenarios mirroring the
OrpheusTTFATests env-gate pattern (JARVIS_REAL_MODELS=1):
1-3: ADD / NOOP / UPDATE extraction (D-03 / MEM-04)
4-5: supersede invariants — never DELETE (MEM-05)
6-8: FTS5+vec hybrid retrieval ranking (MEM-07)
9-10: forget_fact valid_to closure (D-02)
- All 10 tests skip cleanly when JARVIS_REAL_MODELS != 1; runs
interactively against real local Ollama (nomic-embed-text +
qwen2.5-coder:32b). Researcher Open Q #3 satisfied.
- MemoryStore gains four small read-only test seams used by the corpus:
searchFacts(query:k:embedder:) — facade over runHybridSearchSQL that
embeds via the injected EmbeddingProviding;
factById(_:) — fetch any-state row;
rawCountFacts(predicate:) — count rows regardless of valid_to /
forgotten_at (proves never-DELETE on supersede);
activeFacts(subject:predicate:) — uses idx_facts_active partial index.
- Memory test target gains OllamaProvider product dependency so the
corpus can construct a live LLMProvider for the env-gated runs.
- Adds MemoryExtractionOrchestratorTests with 8 cases covering MEM-06 non-blocking, serial drain, applyOp reach-through, error survival, coordinator turnEnd filtering, callback wiring, stop semantics
- Memory Package.swift now depends on AgentOrchestrator product (needed for OrchestratorEvent type used by coordinator)
- Tests fail to compile until orchestrator + coordinator land in Task 4 GREEN
- OllamaEmbeddingClientTests covers loopback guard, /api/embed routing, body shape, dim validation, HTTP/shape errors via URLProtocolStub
- EmbeddingNetworkSandboxTests is the dedicated MEM-04 file mandated by CONTEXT.md
- Tests fail to compile until OllamaEmbeddingClient lands in Task 2 GREEN
- SQLiteConnection.withHandle(_:): closure-based accessor so MemoryStore
can call sqlite3_load_extension without leaking the OpaquePointer.
- MemoryConstants.embeddingDim = 768 (MEM-02): single canonical symbol
for vector dimension. Hard-pinned to nomic-embed-text 768d output.
- EmbeddingDimSymbolTests:
testEmbeddingDimIsSeventySixtyEight — value sanity.
testEmbeddingDimIsSingleSymbol — workspace grep guard against any
parallel literal definition.
Deviation Rule 1: the plan-supplied repoRoot heuristic walked up looking
for 'a Package.swift WITHOUT a parent Package.swift', but this workspace
has no top-level Package.swift (it uses XcodeGen project.yml). Walk up
looking for project.yml or Jarvis.xcodeproj instead.
Deviation Rule 1: the plan-supplied symbol-grep flagged the test file's
own docstrings and the Constants.swift docstring as 'rogue literals'
(self-reference bug). Filtered #file and Constants.swift from the hit
list — the test still flags any other shadow definition anywhere in the
workspace, which is the intended invariant.
- New packages/Memory SPM target with Logging/AgentCore/Replay deps
- Resources/PLACEHOLDER.txt reserves bundle slot for future vec0.dylib
- Memory.swift + MemoryTestsScaffold.swift placeholders so the empty
target compiles; real symbols land in Tasks 2 and 3
- project.yml: add Memory to packages and Jarvis target deps (Voice analog)
- Jarvis.xcodeproj regenerated via xcodegen
Deviation Rule 3: plan referenced 'root Package.swift' but this workspace
uses XcodeGen project.yml (no root Package.swift exists). Wired Memory the
same way Voice was wired in Plan 06-05.
james.maes
