fix(memory): drop broken FK on facts.source_turn_id — closes Toby-fact persistence

Root cause of "facts never persist" surfaced during 2026-05-12 dogfood:

  ERROR [memory.orchestrator] memory applyOp failed: applyOpFailed(
    underlying: Replay.SQLiteError.stepFailed(
      code: 19, message: "FOREIGN KEY constraint failed"))

The schema declared:
  source_turn_id INTEGER REFERENCES turns(id)

But the writer at MemoryExtractionOrchestrator.swift:125 inserts:
  let turnIdInt: Int64 = stableHash(job.turnId.rawValue)

`TurnID` is a UUID-string wrapper; the writer FNV-1a-hashes it to an
Int64 for the integer column. The hash space and `turns.id`
(AUTOINCREMENT) are disjoint number systems by design, so every
INSERT into `facts` violated the FK and SQLite rolled back the row.

The BootHealth probe reports the symptom honestly ("26 turns, 0/0
facts") but couldn't flag this as a failure — 0 facts is the
legitimate empty-history case. The bug stayed invisible until the
user asked "is Toby a fact in the database?" and search returned
nothing despite asynchronous extraction running.

This bug was latent for the entire project history — the original
RESEARCH §1 schema specified the FK, but the writer was never wired
to use real `turns.id` rowids. Reintroducing the FK without first
fixing the writer (forward through MemoryStore.appendTurn returning
the rowid, then plumbed back to MemoryExtractionCoordinator) would
just re-break persistence.

Fix: drop the REFERENCES on `source_turn_id`. The column is now
documented as a "correlation tag", not a hard foreign key. The
`superseded_by REFERENCES facts(id)` constraint stays — that one
IS valid because supersede chains use `priorFact.id` (a real
AUTOINCREMENT rowid).

Existing DBs: `CREATE TABLE IF NOT EXISTS` won't ALTER an existing
table; the user must delete `~/Library/Application Support/Jarvis/
jarvis.db*` and relaunch. Their facts table has 0 rows so nothing
of value is lost. A proper migration framework (PRAGMA user_version)
is deferred to M-1+.

Follow-up (M-1+): plumb the real `turns.id` rowid through
`MemoryStore.appendTurn` → orchestrator state → coordinator →
applyOp so `source_turn_id` becomes a real FK again. Requires
changes across 4 files; not in scope for this hotfix.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
james.maesandClaude Opus 4.7 committed 2026-05-12 12:07:44 -05:00
1 parent b76327364d
commit a5999e2dde
1 file changed
+19 -2
@@ -30,14 +30,31 @@ public enum MemorySchema {
""",
"CREATE INDEX IF NOT EXISTS idx_turns_session ON turns(session_id, created_at);",
// RESEARCH section 1 + D-02: facts table with temporal validity + forgotten_at.
// Facts table — temporal validity + supersede chain + soft delete.
//
// `source_turn_id` is a CORRELATION TAG, NOT a foreign key. The
// memory-extraction writer (`MemoryExtractionOrchestrator.process`)
// FNV-1a-hashes the originating `TurnID` UUID to an Int64 because
// `TurnID` is a string-backed UUID and the writer needs an integer
// column. The hash space and `turns.id` (AUTOINCREMENT) never
// collide by design — they're disjoint number systems — so any
// `REFERENCES turns(id)` on this column would fail every insert
// with SQLITE_CONSTRAINT_FOREIGNKEY (code 19). The constraint was
// present in the original schema and silently broke fact persistence
// for the entire history of the project until the 2026-05-12 dogfood
// surfaced it. Reintroducing the FK without first wiring the writer
// to use real `turns.id` rowids would re-break persistence.
//
// `superseded_by REFERENCES facts(id)` IS a real FK — the supersede
// chain uses `priorFact.id` from a previous `applyOp` return value,
// which is the actual AUTOINCREMENT rowid.
"""
CREATE TABLE IF NOT EXISTS facts (
id INTEGER PRIMARY KEY,
subject TEXT NOT NULL,
predicate TEXT NOT NULL,
object TEXT NOT NULL,
source_turn_id INTEGER REFERENCES turns(id),
source_turn_id INTEGER,
valid_from INTEGER NOT NULL,
valid_to INTEGER,
superseded_by INTEGER REFERENCES facts(id),