Commit Graph
5 Commits
Author SHA1 Message Date
james.maesandClaude Opus 4.7 cb584a2ac8 feat(d5d6): real memory database — vec0 statically linked, hard-fail boot, stats tool
Closes D-5 / D-6 (sqlite-vec dylib bundling) carry-forwards from the
2026-05-04 audit by adopting jkrukowski/SQLiteVec as a SPM dependency.
The package's CSQLiteVec target compiles sqlite3.c + sqlite-vec.c into
one statically-linked C target with `core_vec_init()` registering vec0
as an auto-extension. All sqlite3_* calls in Replay's SQLiteConnection
and Memory's MemoryStore resolve to this bundled SQLite (vec0 always
available) instead of Apple's stripped /usr/lib/libsqlite3.dylib.

The retired hand-rolled approach (dlsym pathway + JARVIS_VEC0_STUB_PATH
env-flag + Bundle.module placeholder text file) never worked on macOS
because Apple stripped sqlite3_load_extension from the system libsqlite3.
Memory init has been silently dormant since Phase 7 (2026-04 onward),
which is what made B-02 (history threading) appear permanently broken
to the user during dogfooding even after the 48ad8d6 tactical patch
landed.

Live verification 2026-05-11: fresh launch with empty jarvis.db creates
schema, persists turn rows across submits, B-02 multi-turn context
resolves correctly ("what breed is Toby?" → "Golden Retriever" after
prior turn established Toby is a Golden Retriever), and the new
get_memory_stats tool reports `turnsCount=6, vecVersion=v0.1.6,
sqliteVersion=3.51.0`.

Changes:

- packages/Replay/Package.swift
    + SQLiteVec SPM dependency (from: "0.0.14")
    + CSQLiteVec product wired into Replay target
- packages/Replay/Sources/Replay/SQLiteConnection.swift
    - import SQLite3
    + @_exported import CSQLiteVec
    All sqlite3_* call sites unchanged — linker now resolves to our
    bundled sqlite3 with vec0 statically linked in.
- packages/Memory/Package.swift
    + SQLiteVec SPM dependency (named product so MemoryStore can import
      CSQLiteVec directly for the core_vec_init() one-shot call)
    - resources: PLACEHOLDER.txt copy (defunct vec0.dylib stub slot)
- packages/Memory/Sources/Memory/MemoryStore.swift
    - import SQLite3 → import CSQLiteVec
    + _vecAutoExtensionRegistered static-let one-shot: calls
      CSQLiteVec.core_vec_init() which registers vec0 via
      sqlite3_auto_extension on the first MemoryStore construction.
      Idempotent (static-let initializer fires exactly once).
    - loadVecExtension(on:) dlsym/JARVIS_VEC0_STUB_PATH/Bundle.module
      pathway (retired; ~70 lines removed)
    Updated docstring to reflect the new hard-fail boot policy.
- packages/Memory/Sources/Memory/Resources/PLACEHOLDER.txt — deleted
- App/AppDelegate.swift installMemory()
    Reverses Track-D D-1 (75a10be, 2026-05-04): MemoryStore.init failures
    are now FATAL via Logger.critical + TCCAlertService.presentHardBlock
    + NSApp.terminate. The graceful-degradation path that kept the app
    booting with memoryStore=nil hid the entire B-02/B-04/B-05 silent-
    failure bug class. Aligns with the 2026-05-07 API-first pivot
    thesis: "no silent failures".
    `store` is now non-optional throughout installMemory; downstream
    callers no longer need `if let store = store`.
- packages/MCP/Sources/MCP/InProcess/GetMemoryStatsTool.swift (new)
    + MemoryStatsDispatching protocol
    + MemoryStats Codable struct (turnsCount, factsTotal, factsActive,
      sessionsCount, dbFileBytes, vecVersion, sqliteVersion,
      lastTurnAtUnixMs, lastFactAtUnixMs)
    + GetMemoryStatsTool conforming to InProcessTool
- App/MCP/InProcessMemoryAdapters.swift
    + MemoryStatsStoreAdapter — bridges the protocol to MemoryStore's
      public queryRowCount / querySingleString seams and FileManager
      for the on-disk file size. Read-only, no confirmation.
- App/AppDelegate.swift installSelfKnowledgeTools()
    + Registers GetMemoryStatsTool with MemoryStatsStoreAdapter.
- packages/Memory/Tests/MemoryTests/*
    Retired the JARVIS_VEC0_STUB_PATH XCTSkipUnless guards from 9 tests
    (vec0 is always available now — no need to env-gate). Rewrote
    MemoryStoreTests to drop the now-impossible "init throws when vec
    missing" case and assert facts_vec virtual table creation +
    vec_version() readback. Added seedTurnRow helper to
    MemoryStoreApplyOpTests so the now-enforced
    `facts.source_turn_id REFERENCES turns(id)` foreign key is honored
    in tests that previously passed when FK enforcement was dormant.

Test summary: Memory full suite 91 tests, 10 skipped (env-gated real-
Ollama tests), 0 failures. App target builds clean. Substrate verified
end-to-end via live launch (vec_version: v0.1.6 logged at boot;
get_memory_stats returns actual schema row counts).

Pending D-7 carry-forward: ollama pull nomic-embed-text (~270MB) for
embedder; user already has qwen2.5-coder:32b-instruct-q8_0 which is
a higher-quality variant of the project's hardcoded
qwen2.5-coder:32b — small ModelID config delta to follow.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-11 10:25:56 -05:00
james.maesandClaude Opus 4.7 87aec282a0 feat(09-02): wire FrameAttachController + Bus frameAttachRequested + broadcaster release subscriber (D-15, D-16, INT-07-04)
- App/Vision/AppDelegateFrameAttachAdapters.swift (NEW): FrameAttachCaptureSourceAdapter wraps CameraCapture, FrameAttachReplaySinkAdapter wraps Replay.ReplayLog (placeholders log via JarvisLogChannel.replay — see file docstring for FK rationale), FrameAttachControllerReplaySinkBridge bridges Vision's FrameAttachReplaySink actor to FrameAttachController.ReplaySink protocol
- AppDelegate: new frameAttachController + frameAttachReleaseTask properties; installVision now constructs FrameAttachController (D-15); installAgent waits for visionInstallTask before constructing the orchestrator and passes visionRouter:; new step 5d frame-attach release subscriber subscribes broadcaster's .frameAttach priority and fires onAssistantTurnComplete() on .turnEnd for image-bearing turns (D-16)
- AppDelegate: tryPhraseAttachIfMatch(_:) helper exposed for Plan 4's chatSubmit + chatCancelAndSubmit handlers (WARNING-4: must call from BOTH paths)
- AppDelegate.installBus: bridge.onInbound now switches on BusInbound; .frameAttachRequested → frameAttachController.requestAttach(reason: .hudButton)
- BusInbound: additive .frameAttachRequested case + Discriminator + Codable arms (NO default — exhaustive-switch drift preventer); BUS_PROTOCOL_VERSION 2.1.0 → 2.2.0 in Swift + TS mirror
- TS protocol: matching frameAttachRequested case + decoder arm; round-trip.test.ts asserts version 2.2.0 + decodes new fixture
- New fixtures (parity): packages/Bus/Tests/BusTests/Fixtures/frameAttachRequested.json + webview/packages/bus/fixtures/frameAttachRequested.json (byte-identical, no trailing newline per check-bus-protocol-version.sh)
- Memory package bumped from .v13 to .v14 (cascade: Memory depends on AgentCore which now requires v14 due to JarvisVision dependency added in Task 1)
- xcodegen regen for the new App/Vision/AppDelegateFrameAttachAdapters.swift source file
- 2 new FrameAttachReleaseSubscriberTests prove broadcaster→onAssistantTurnComplete glue (image-bearing turns release once; text-only turns NEVER release); subscription uses devOverlay priority for the sync wait to avoid competing with broadcaster's drain on orch.events
- 1 new CodableRoundTripTests entry (test_roundTrip_frameAttachRequested)
- Phase 7 SOLE-emission-site invariant intact: FrameAttachDiscardSiteGrepTests passes; check-presence-vision-isolation.sh exits 0; orchestrator-events single-consumer grep gate exits 0; 65 AgentOrchestratorTests + 51 BusTests + 34 webview tests all green
- App target compiles cleanly (check-app-builds.sh exits 0)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-01 19:41:29 -05:00
james.maes 56aeaed171 test(07-06): env-gated 10-scenario memory regression corpus (Task 2)
- MemoryRegressionCorpusTests with 10 scenarios mirroring the
  OrpheusTTFATests env-gate pattern (JARVIS_REAL_MODELS=1):
    1-3: ADD / NOOP / UPDATE extraction (D-03 / MEM-04)
    4-5: supersede invariants — never DELETE (MEM-05)
    6-8: FTS5+vec hybrid retrieval ranking (MEM-07)
    9-10: forget_fact valid_to closure (D-02)
- All 10 tests skip cleanly when JARVIS_REAL_MODELS != 1; runs
  interactively against real local Ollama (nomic-embed-text +
  qwen2.5-coder:32b). Researcher Open Q #3 satisfied.
- MemoryStore gains four small read-only test seams used by the corpus:
  searchFacts(query:k:embedder:) — facade over runHybridSearchSQL that
    embeds via the injected EmbeddingProviding;
  factById(_:) — fetch any-state row;
  rawCountFacts(predicate:) — count rows regardless of valid_to /
    forgotten_at (proves never-DELETE on supersede);
  activeFacts(subject:predicate:) — uses idx_facts_active partial index.
- Memory test target gains OllamaProvider product dependency so the
  corpus can construct a live LLMProvider for the env-gated runs.
2026-04-29 18:44:56 -05:00
james.maes 040e578654 test(07-02): add failing orchestrator + coordinator tests; wire AgentOrchestrator dep (Task 4 RED)
- Adds MemoryExtractionOrchestratorTests with 8 cases covering MEM-06 non-blocking, serial drain, applyOp reach-through, error survival, coordinator turnEnd filtering, callback wiring, stop semantics
- Memory Package.swift now depends on AgentOrchestrator product (needed for OrchestratorEvent type used by coordinator)
- Tests fail to compile until orchestrator + coordinator land in Task 4 GREEN
2026-04-29 16:22:44 -05:00
james.maes 64e38ba15d feat(07-01): create packages/Memory SPM scaffold + wire into project.yml
- New packages/Memory SPM target with Logging/AgentCore/Replay deps
- Resources/PLACEHOLDER.txt reserves bundle slot for future vec0.dylib
- Memory.swift + MemoryTestsScaffold.swift placeholders so the empty
  target compiles; real symbols land in Tasks 2 and 3
- project.yml: add Memory to packages and Jarvis target deps (Voice analog)
- Jarvis.xcodeproj regenerated via xcodegen

Deviation Rule 3: plan referenced 'root Package.swift' but this workspace
uses XcodeGen project.yml (no root Package.swift exists). Wired Memory the
same way Voice was wired in Plan 06-05.
2026-04-29 15:45:35 -05:00