Memory: facts.source_turn_id stores FNV hash, not turns.id rowid — correlation permanently broken #15

New Issue

2026-05-12T19:03:51Z

KofTwentyTwo commented

2026-05-12 19:03:51 +00:00

(Migrated from github.com)

Summary

MemoryExtractionOrchestrator.process still hashes TurnID and stores the FNV hash in facts.source_turn_id. After today's FK fix, fact INSERTs succeed, but source_turn_id holds the FNV-1a hash of TurnID.rawValue (UUID string), not a real turns.id rowid. Fact-to-turn correlation is permanently broken.

Evidence

File: packages/Memory/Sources/Memory/MemoryExtractionOrchestrator.swift:122-152 (process body + stableHash)
Live DB: SELECT source_turn_id FROM facts; → 7747877241524896942 (FNV hash)
No row in turns has id=7747877241524896942
Schema comment at MemorySchema.swift:35-50 openly acknowledges the disjoint number systems

Root cause / hypothesis

When the FK was dropped in a5999e2, the writer-side change to use a real turns.id rowid wasn't made — the FNV hash is a Phase-7 artifact that survived the fix. The orchestrator has job.turnId: TurnID (UUID string) and turns.id INTEGER PRIMARY KEY autoincrement, but no lookup translates one to the other. The B-02 patch (commit 48ad8d6) writes turn rows BEFORE memory extraction enqueues, but doesn't return the rowid, and there's no plumbing to feed it back.

Recommended fix

Two surgical options:

Drop source_turn_id from the schema entirely if no consumer reads it. Document that fact-to-turn correlation is via valid_from timestamp proximity. Cleaner.
Thread the real rowid: Make B-02's appendTurn return the new rowid; thread it through the flushPair callback (return triple (userText, assistantText, userTurnRowId)); have MemoryExtractionCoordinator.start's turnContent closure return it; pass as sourceTurnId through to applyOp. Drop stableHash entirely.

Option 1 is a one-day fix; option 2 is half a day and preserves correlation. The current state — silently storing nonsense — is the worst of both worlds.

Test

Assert: SELECT t.id FROM facts f JOIN turns t ON t.id = f.source_turn_id returns a non-empty result after writing a fact, OR source_turn_id column does not exist after schema migration
Schema test: no FNV-hash-shaped (Int64 max range) values in facts.source_turn_id

Source

Audit: .planning/audit-2026-05-12/memory-audit.md (2026-05-12) — finding M-4 HIGH

Cross-references

Related: M-2 (same FNV-hash shape in replay sink)
Related: B-02 tactical patch (commit 48ad8d6) — provides the substrate for option 2

## Summary `MemoryExtractionOrchestrator.process` still hashes `TurnID` and stores the FNV hash in `facts.source_turn_id`. After today's FK fix, fact INSERTs succeed, but `source_turn_id` holds the FNV-1a hash of `TurnID.rawValue` (UUID string), not a real `turns.id` rowid. Fact-to-turn correlation is permanently broken. ## Evidence - File: `packages/Memory/Sources/Memory/MemoryExtractionOrchestrator.swift:122-152` (`process` body + `stableHash`) - Live DB: `SELECT source_turn_id FROM facts;` → `7747877241524896942` (FNV hash) - No row in `turns` has `id=7747877241524896942` - Schema comment at `MemorySchema.swift:35-50` openly acknowledges the disjoint number systems ## Root cause / hypothesis When the FK was dropped in `a5999e2`, the writer-side change to use a real `turns.id` rowid wasn't made — the FNV hash is a Phase-7 artifact that survived the fix. The orchestrator has `job.turnId: TurnID` (UUID string) and `turns.id INTEGER PRIMARY KEY` autoincrement, but no lookup translates one to the other. The B-02 patch (commit `48ad8d6`) writes turn rows BEFORE memory extraction enqueues, but doesn't return the rowid, and there's no plumbing to feed it back. ## Recommended fix Two surgical options: 1. **Drop `source_turn_id` from the schema entirely** if no consumer reads it. Document that fact-to-turn correlation is via `valid_from` timestamp proximity. Cleaner. 2. **Thread the real rowid:** Make B-02's `appendTurn` return the new rowid; thread it through the `flushPair` callback (return triple `(userText, assistantText, userTurnRowId)`); have `MemoryExtractionCoordinator.start`'s `turnContent` closure return it; pass as `sourceTurnId` through to `applyOp`. Drop `stableHash` entirely. Option 1 is a one-day fix; option 2 is half a day and preserves correlation. The current state — silently storing nonsense — is the worst of both worlds. ## Test - Assert: `SELECT t.id FROM facts f JOIN turns t ON t.id = f.source_turn_id` returns a non-empty result after writing a fact, OR `source_turn_id` column does not exist after schema migration - Schema test: no FNV-hash-shaped (Int64 max range) values in `facts.source_turn_id` ## Source - Audit: `.planning/audit-2026-05-12/memory-audit.md` (2026-05-12) — finding M-4 HIGH ## Cross-references - Related: M-2 (same FNV-hash shape in replay sink) - Related: B-02 tactical patch (commit `48ad8d6`) — provides the substrate for option 2

KofTwentyTwo commented

2026-05-13 11:51:48 +00:00

(Migrated from github.com)

Closed by v0.1 cluster merges on 2026-05-13. Develop now at a8c7f1a0.

Closed by v0.1 cluster merges on 2026-05-13. Develop now at `a8c7f1a0`.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: KofTwentyTwo/Jarvis#15