Memory: BootHealthProbe evidence misleading post-FK-fix; no end-to-end writer probe #17

New Issue

2026-05-12T19:04:10Z

KofTwentyTwo commented

2026-05-12 19:04:10 +00:00

(Migrated from github.com)

Summary

MemoryBootHealthProbe evidence is misleading after today's FK fix. The probe still reports state=ok evidence="N turns, 0/0 facts, vec=v0.1.6" when memory has been written successfully — it just reports row counts at probe time. Today's user-burning bug was caught because the probe was happy with 0/0 while every insert failed; after the FK fix, 0/0 facts is genuinely a legit empty state and the probe doesn't know the difference between "extractor never fired", "extractor fired and got NOOP", and "extractor fired and FK-failed".

Evidence

File: App/Boot/BootHealthProbes.swift:32-55 (probe)
File: App/Boot/BootHealthProbes.swift:179-181 (stale comment about "the Toby case")
Live probe output subsystem=memory state=ok evidence="0 turns, 0/0 facts, vec=v0.1.6, sqlite=3.51.0" from 18:47:35; by 19:00 user had submitted turns but probe ran once at boot — evidence stale by construction

Root cause / hypothesis

Probe asserts schema queryability + vec0 loadability + DB openness — three real things — but doesn't probe the writer end-to-end. After the M-1 fix (no facts_vec writes), a stronger probe would assert facts_vec row count tracks facts row count within ±1. After M-2 is fixed, the probe could check replay.sqlite events table has recent rows.

Recommended fix

Two complementary improvements:

Update the stale comment at BootHealthProbes.swift:179-181. Current: "missing models = the Toby case. Embedding step fails, no row inserted in facts" — no longer accurate. Rewrite as: "missing models = embedder fails → search degrades to FTS-only; extractor fails → no facts inserted at all."
Add parity check to MemoryStats: factsTotal == factsVecCount ± tolerance, surface as the probe's degraded path. This is the structural defense against M-1-style impedance bugs.

Test

Probe unit: when facts_vec row count differs from facts row count by > tolerance, probe state is degraded not ok
Comment scrub: lint rule against "Toby case" comment

Source

Audit: .planning/audit-2026-05-12/memory-audit.md (2026-05-12) — finding M-6 MEDIUM

Cross-references

Related: #12 (M-1) — parity check is the structural defense

## Summary `MemoryBootHealthProbe` evidence is misleading after today's FK fix. The probe still reports `state=ok evidence="N turns, 0/0 facts, vec=v0.1.6"` when memory has been written successfully — it just reports row counts at probe time. Today's user-burning bug was caught because the probe was happy with `0/0` while every insert failed; after the FK fix, `0/0 facts` is genuinely a legit empty state and the probe doesn't know the difference between "extractor never fired", "extractor fired and got NOOP", and "extractor fired and FK-failed". ## Evidence - File: `App/Boot/BootHealthProbes.swift:32-55` (probe) - File: `App/Boot/BootHealthProbes.swift:179-181` (stale comment about "the Toby case") - Live probe output `subsystem=memory state=ok evidence="0 turns, 0/0 facts, vec=v0.1.6, sqlite=3.51.0"` from 18:47:35; by 19:00 user had submitted turns but probe ran once at boot — evidence stale by construction ## Root cause / hypothesis Probe asserts schema queryability + vec0 loadability + DB openness — three real things — but doesn't probe the writer end-to-end. After the M-1 fix (no `facts_vec` writes), a stronger probe would assert `facts_vec` row count tracks `facts` row count within ±1. After M-2 is fixed, the probe could check `replay.sqlite` events table has recent rows. ## Recommended fix Two complementary improvements: 1. Update the stale comment at `BootHealthProbes.swift:179-181`. Current: "missing models = the Toby case. Embedding step fails, no row inserted in `facts`" — no longer accurate. Rewrite as: "missing models = embedder fails → search degrades to FTS-only; extractor fails → no facts inserted at all." 2. Add parity check to `MemoryStats`: `factsTotal == factsVecCount ± tolerance`, surface as the probe's degraded path. This is the structural defense against M-1-style impedance bugs. ## Test - Probe unit: when `facts_vec` row count differs from `facts` row count by > tolerance, probe state is `degraded` not `ok` - Comment scrub: lint rule against "Toby case" comment ## Source - Audit: `.planning/audit-2026-05-12/memory-audit.md` (2026-05-12) — finding M-6 MEDIUM ## Cross-references - Related: #12 (M-1) — parity check is the structural defense

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: KofTwentyTwo/Jarvis#17