james.maesandClaude Opus 4.7
a5999e2dde
Root cause of "facts never persist" surfaced during 2026-05-12 dogfood:
ERROR [memory.orchestrator] memory applyOp failed: applyOpFailed(
underlying: Replay.SQLiteError.stepFailed(
code: 19, message: "FOREIGN KEY constraint failed"))
The schema declared:
source_turn_id INTEGER REFERENCES turns(id)
But the writer at MemoryExtractionOrchestrator.swift:125 inserts:
let turnIdInt: Int64 = stableHash(job.turnId.rawValue)
`TurnID` is a UUID-string wrapper; the writer FNV-1a-hashes it to an
Int64 for the integer column. The hash space and `turns.id`
(AUTOINCREMENT) are disjoint number systems by design, so every
INSERT into `facts` violated the FK and SQLite rolled back the row.
The BootHealth probe reports the symptom honestly ("26 turns, 0/0
facts") but couldn't flag this as a failure — 0 facts is the
legitimate empty-history case. The bug stayed invisible until the
user asked "is Toby a fact in the database?" and search returned
nothing despite asynchronous extraction running.
This bug was latent for the entire project history — the original
RESEARCH §1 schema specified the FK, but the writer was never wired
to use real `turns.id` rowids. Reintroducing the FK without first
fixing the writer (forward through MemoryStore.appendTurn returning
the rowid, then plumbed back to MemoryExtractionCoordinator) would
just re-break persistence.
Fix: drop the REFERENCES on `source_turn_id`. The column is now
documented as a "correlation tag", not a hard foreign key. The
`superseded_by REFERENCES facts(id)` constraint stays — that one
IS valid because supersede chains use `priorFact.id` (a real
AUTOINCREMENT rowid).
Existing DBs: `CREATE TABLE IF NOT EXISTS` won't ALTER an existing
table; the user must delete `~/Library/Application Support/Jarvis/
jarvis.db*` and relaunch. Their facts table has 0 rows so nothing
of value is lost. A proper migration framework (PRAGMA user_version)
is deferred to M-1+.
Follow-up (M-1+): plumb the real `turns.id` rowid through
`MemoryStore.appendTurn` → orchestrator state → coordinator →
applyOp so `source_turn_id` becomes a real FK again. Requires
changes across 4 files; not in scope for this hotfix.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Memory
Long-term fact store + extraction pipeline. SQLite (FTS5 + sqlite-vec) at ~/Library/Application Support/Jarvis/jarvis.db; mem0-style ADD/UPDATE/NOOP extraction; hybrid keyword + vector search via Reciprocal Rank Fusion.
Functional state. Memory is code-complete but functionally OFF until
vec0.dylibships (Track D-5/D-6 — user environment work) andollama pull nomic-embed-textruns (D-7).installMemorydegrades gracefully — extractor + orchestrator + coordinator construct even whenMemoryStoreinit throws.
Key public types
| Type | Purpose |
|---|---|
MemoryStore (actor) |
The only writer of jarvis.db. Opens with WAL, loads vec0.dylib, runs migrations |
MemoryExtractionOrchestrator (actor) |
Bounded-channel + serial-drain extraction Task |
MemoryExtractionCoordinator |
AppDelegate-side coordinator that listens to OrchestratorEvent.turnEnd |
MemoryExtractor |
Calls Qwen 2.5-Coder via Ollama; emits ADD / UPDATE(supersedes:) / NOOP |
MemoryPrompts |
Prompt templates for the extractor |
HybridSearch (actor) |
FTS5 + vec0 RRF SQL; emits ReplayEvent.memoryRetrieval per fact |
OllamaEmbeddingClient |
nomic-embed-text 768-dim embeddings |
Fact / FactRef / ExtractionJob |
Plain-data types |
MemoryEvent |
Stream events — factAdded, factUpdated, factForgotten, retrieval |
SessionHistory |
Session-scoped recent-turn browse (TEXT-03; separate from hybrid search) |
MemoryReplaySink (protocol) |
Test seam over Replay.ReplayLog |
PriorFactsLookup |
Closure type for D-3 — supplies recent active facts to the extractor |
Depends on
AgentCore, Replay, Logging. External: swift-log, system SQLite (with custom libsqlite3.dylib carrying SQLITE_ENABLE_LOAD_EXTENSION=1 once D-5 ships).
Used by
App/AppDelegate.installMemory, App/MCP/InProcessMemoryAdapters.swift (bridges to MCP SearchMemoryTool / ForgetFactTool).
Key invariants / contracts
MemoryStoreis the only writer. All writes go through actor methods.- Bounded extraction channel.
BoundedAsyncChannel<ExtractionJob>(capacity: 32, policy: .dropOldest). Producers (turnEnd) never block. - Single-consumer drain. One serial Task per orchestrator instance.
- Single-emission grep gates.
scripts/check-single-memory-mutated-emit.sh+check-single-memory-used-emit.sh.MemoryStore.recordRetrievalis the onlymemoryRetrievalemit site. - Embedding dim sourced from
Constants.swift.scripts/check-embedding-dim-literal.shforbids hand-typed768. - Temporal validity preserved. UPDATE supersedes by setting
valid_toon the prior fact + inserting a new one. No DELETE in the happy path. - D-3: priorFacts wiring.
MemoryExtractionOrchestrator.process(_:)callspriorFactsLookup(defaultemptyPriorFactsLookupfor back-compat); production wiresMemoryStore.recentActiveFacts(limit: 50). - T-06-05-03 analog.
HybridSearchdoes NOT log raw query text at debug level (P2-12 fix).
Tests
87 XCTest (19 skipped — vec0/Ollama gating). Includes the RememberBrutusEndToEndTests regression scenario (Track D-4) that proves extraction → store → search end-to-end with fakes.
Notable files
Sources/Memory/MemoryStore.swift— actor + SQLite + vec0 loaderSources/Memory/MemoryExtractionOrchestrator.swift— bounded-channel extractionSources/Memory/HybridSearch.swift— FTS5 + vec0 RRFSources/Memory/MemoryExtractor.swift— Qwen extractorSources/Memory/MemoryQueries.swift— SQL definitions