Files
Jarvis/packages/Memory/Tests/MemoryTests/MemoryStoreAppendTurnTests.swift
james.maesandClaude Opus 4.7 cb584a2ac8 feat(d5d6): real memory database — vec0 statically linked, hard-fail boot, stats tool
Closes D-5 / D-6 (sqlite-vec dylib bundling) carry-forwards from the
2026-05-04 audit by adopting jkrukowski/SQLiteVec as a SPM dependency.
The package's CSQLiteVec target compiles sqlite3.c + sqlite-vec.c into
one statically-linked C target with `core_vec_init()` registering vec0
as an auto-extension. All sqlite3_* calls in Replay's SQLiteConnection
and Memory's MemoryStore resolve to this bundled SQLite (vec0 always
available) instead of Apple's stripped /usr/lib/libsqlite3.dylib.

The retired hand-rolled approach (dlsym pathway + JARVIS_VEC0_STUB_PATH
env-flag + Bundle.module placeholder text file) never worked on macOS
because Apple stripped sqlite3_load_extension from the system libsqlite3.
Memory init has been silently dormant since Phase 7 (2026-04 onward),
which is what made B-02 (history threading) appear permanently broken
to the user during dogfooding even after the 48ad8d6 tactical patch
landed.

Live verification 2026-05-11: fresh launch with empty jarvis.db creates
schema, persists turn rows across submits, B-02 multi-turn context
resolves correctly ("what breed is Toby?" → "Golden Retriever" after
prior turn established Toby is a Golden Retriever), and the new
get_memory_stats tool reports `turnsCount=6, vecVersion=v0.1.6,
sqliteVersion=3.51.0`.

Changes:

- packages/Replay/Package.swift
    + SQLiteVec SPM dependency (from: "0.0.14")
    + CSQLiteVec product wired into Replay target
- packages/Replay/Sources/Replay/SQLiteConnection.swift
    - import SQLite3
    + @_exported import CSQLiteVec
    All sqlite3_* call sites unchanged — linker now resolves to our
    bundled sqlite3 with vec0 statically linked in.
- packages/Memory/Package.swift
    + SQLiteVec SPM dependency (named product so MemoryStore can import
      CSQLiteVec directly for the core_vec_init() one-shot call)
    - resources: PLACEHOLDER.txt copy (defunct vec0.dylib stub slot)
- packages/Memory/Sources/Memory/MemoryStore.swift
    - import SQLite3 → import CSQLiteVec
    + _vecAutoExtensionRegistered static-let one-shot: calls
      CSQLiteVec.core_vec_init() which registers vec0 via
      sqlite3_auto_extension on the first MemoryStore construction.
      Idempotent (static-let initializer fires exactly once).
    - loadVecExtension(on:) dlsym/JARVIS_VEC0_STUB_PATH/Bundle.module
      pathway (retired; ~70 lines removed)
    Updated docstring to reflect the new hard-fail boot policy.
- packages/Memory/Sources/Memory/Resources/PLACEHOLDER.txt — deleted
- App/AppDelegate.swift installMemory()
    Reverses Track-D D-1 (75a10be, 2026-05-04): MemoryStore.init failures
    are now FATAL via Logger.critical + TCCAlertService.presentHardBlock
    + NSApp.terminate. The graceful-degradation path that kept the app
    booting with memoryStore=nil hid the entire B-02/B-04/B-05 silent-
    failure bug class. Aligns with the 2026-05-07 API-first pivot
    thesis: "no silent failures".
    `store` is now non-optional throughout installMemory; downstream
    callers no longer need `if let store = store`.
- packages/MCP/Sources/MCP/InProcess/GetMemoryStatsTool.swift (new)
    + MemoryStatsDispatching protocol
    + MemoryStats Codable struct (turnsCount, factsTotal, factsActive,
      sessionsCount, dbFileBytes, vecVersion, sqliteVersion,
      lastTurnAtUnixMs, lastFactAtUnixMs)
    + GetMemoryStatsTool conforming to InProcessTool
- App/MCP/InProcessMemoryAdapters.swift
    + MemoryStatsStoreAdapter — bridges the protocol to MemoryStore's
      public queryRowCount / querySingleString seams and FileManager
      for the on-disk file size. Read-only, no confirmation.
- App/AppDelegate.swift installSelfKnowledgeTools()
    + Registers GetMemoryStatsTool with MemoryStatsStoreAdapter.
- packages/Memory/Tests/MemoryTests/*
    Retired the JARVIS_VEC0_STUB_PATH XCTSkipUnless guards from 9 tests
    (vec0 is always available now — no need to env-gate). Rewrote
    MemoryStoreTests to drop the now-impossible "init throws when vec
    missing" case and assert facts_vec virtual table creation +
    vec_version() readback. Added seedTurnRow helper to
    MemoryStoreApplyOpTests so the now-enforced
    `facts.source_turn_id REFERENCES turns(id)` foreign key is honored
    in tests that previously passed when FK enforcement was dormant.

Test summary: Memory full suite 91 tests, 10 skipped (env-gated real-
Ollama tests), 0 failures. App target builds clean. Substrate verified
end-to-end via live launch (vec_version: v0.1.6 logged at boot;
get_memory_stats returns actual schema row counts).

Pending D-7 carry-forward: ollama pull nomic-embed-text (~270MB) for
embedder; user already has qwen2.5-coder:32b-instruct-q8_0 which is
a higher-quality variant of the project's hardcoded
qwen2.5-coder:32b — small ModelID config delta to follow.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-11 10:25:56 -05:00

134 lines
5.3 KiB
Swift

import XCTest
import Foundation
@testable import Memory
/// **B-02 carry-forward bug — turn-row writer unit test.**
///
/// Exercises the new `MemoryStore.appendTurn` writer that backs the
/// orchestrator's `sessionHistoryLookup` path. Without a writer, the
/// `turns` table stays empty and B-02's `recentTurnsForSession` read
/// returns []. This suite proves the writer round-trips through the
/// schema's FTS5 trigger correctly.
///
/// Tests are env-gated (`JARVIS_VEC0_STUB_PATH`) because `MemoryStore.init`
/// requires vec0.dylib loaded; without the env var the suite skips —
/// matching the existing MemoryStore test corpus convention.
final class MemoryStoreAppendTurnTests: XCTestCase {
// MARK: - AT-1
/// AT-1: appendTurn writes one row that recentTurnsForSession surfaces.
/// Round-trips role/content/source verbatim and increments the row count.
func testAT1_appendTurnRoundTripsViaRecentTurnsForSession() async throws {
let tmp = try Self.tempDB()
let store = try MemoryStore(databaseURL: tmp)
let sessionId = "S-AT1"
try await store.appendTurn(
sessionId: sessionId,
role: "user",
content: "hello jarvis",
source: "userText",
createdAt: 1_000
)
let rows = try await store.recentTurnsForSession(sessionId: sessionId, limit: 10)
XCTAssertEqual(rows.count, 1)
XCTAssertEqual(rows[0].sessionId, sessionId)
XCTAssertEqual(rows[0].role, "user")
XCTAssertEqual(rows[0].content, "hello jarvis")
XCTAssertEqual(rows[0].source, "userText")
XCTAssertEqual(rows[0].createdAt, 1_000)
}
// MARK: - AT-2
/// AT-2: appendTurn for a (user, assistant) pair produces DESC ordering by
/// created_at + id-tiebreaker that, when reversed, reconstructs
/// chronological user→assistant order — the contract the AppDelegate
/// production wiring relies on.
func testAT2_userAssistantPairReversesToChronological() async throws {
let tmp = try Self.tempDB()
let store = try MemoryStore(databaseURL: tmp)
let sessionId = "S-AT2"
// Production callers offset assistant by +1 ms so the DESC ordering
// is unambiguous; pin that contract here too.
try await store.appendTurn(
sessionId: sessionId, role: "user",
content: "hi", source: "userText", createdAt: 100
)
try await store.appendTurn(
sessionId: sessionId, role: "assistant",
content: "hello back", source: "assistant", createdAt: 101
)
let descRows = try await store.recentTurnsForSession(sessionId: sessionId, limit: 10)
XCTAssertEqual(descRows.count, 2)
// DESC: assistant first, user second.
XCTAssertEqual(descRows[0].role, "assistant")
XCTAssertEqual(descRows[1].role, "user")
// Reverse to chronological — the orchestrator's sessionHistoryLookup
// contract.
let chrono = Array(descRows.reversed())
XCTAssertEqual(chrono.map(\.role), ["user", "assistant"])
XCTAssertEqual(chrono.map(\.content), ["hi", "hello back"])
}
// MARK: - AT-3
/// AT-3: appendTurn is scoped by session_id. Two sessions don't bleed.
func testAT3_sessionScopingIsolatesRows() async throws {
let tmp = try Self.tempDB()
let store = try MemoryStore(databaseURL: tmp)
try await store.appendTurn(
sessionId: "S-A", role: "user", content: "a-content",
source: "userText", createdAt: 1
)
try await store.appendTurn(
sessionId: "S-B", role: "user", content: "b-content",
source: "userText", createdAt: 2
)
let rowsA = try await store.recentTurnsForSession(sessionId: "S-A", limit: 10)
let rowsB = try await store.recentTurnsForSession(sessionId: "S-B", limit: 10)
XCTAssertEqual(rowsA.count, 1)
XCTAssertEqual(rowsB.count, 1)
XCTAssertEqual(rowsA[0].content, "a-content")
XCTAssertEqual(rowsB[0].content, "b-content")
}
// MARK: - AT-4 (FTS5 trigger sanity)
/// AT-4: the schema's `INSERT INTO turns_fts(rowid, content)` trigger
/// fires on appendTurn so the FTS5 index is populated. Verifies the
/// trigger isn't dormant — without this, future chat-search features
/// on `turns` would silently return empty results.
func testAT4_turnsFtsTriggerFiresOnAppend() async throws {
let tmp = try Self.tempDB()
let store = try MemoryStore(databaseURL: tmp)
try await store.appendTurn(
sessionId: "S-AT4", role: "user", content: "the rain in spain",
source: "userText", createdAt: 1
)
// FTS5 MATCH should find the row by content token.
let count = try await store.queryRowCount(
"SELECT 1 FROM turns_fts WHERE turns_fts MATCH 'spain'"
)
XCTAssertEqual(count, 1, "turns_fts trigger must mirror content from inserts")
}
// MARK: - Helpers
private static func tempDB() throws -> URL {
let dir = FileManager.default.temporaryDirectory
.appendingPathComponent("appendturn-tests-\(UUID().uuidString)", isDirectory: true)
try FileManager.default.createDirectory(at: dir, withIntermediateDirectories: true)
return dir.appendingPathComponent("jarvis.db")
}
}