diff --git a/CLAUDE.md b/CLAUDE.md index 48a19cb..64b8130 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -64,7 +64,7 @@ The top-level shape is fixed and agreed. Push back with specific reasoning only - New tokenizer produces ~35% more tokens than Opus 3.x for the same text. Instrument token budgets accordingly; cap `tool_result` content at 8 KB. - Cache TTL default is 5 minutes ephemeral. For 1-hour TTL, pass `ttl: "1h"` explicitly on `cache_control` blocks **AND** include the request header `anthropic-beta: extended-cache-ttl-2025-04-11`. Without the beta header, 1h is silently ignored. Verify via DevOverlay watching `cache_creation_input_tokens` vs `cache_read_input_tokens` per turn. - Tool-use and extended thinking both stream; the SSE parser must handle `content_block_start` with `input_json_delta` for tool args. Close on `message_stop`, not `message_delta`; swallow `ping`; route `thinking_delta` to its own case; treat `stop_reason: "refusal"` as first-class; emit `partial_tool_use_at_disconnect` on mid-delta termination. -- **Ollama caveat:** as of April 2026, Qwen 3 / 3.5 / Gemma 4 tool-calling is broken in Ollama — confirmed via live issue tracker (ollama/ollama#14493 Qwen 3.5 27B non-functional; #14601 Qwen3 malformed tool defs via `/api/chat`; #14745 qwen3.5:9b print-not-execute; #15315 Gemma 4 tool parser). Root cause: Ollama's renderer/parser maps Qwen 3.5 through a Hermes-style JSON pipeline, but the model was trained on Qwen3-Coder XML format; unclosed `` tags corrupt multi-turn. **Known-good local tool-calling baseline: `qwen2.5-coder:32b`.** Qwen3 stays disabled as opt-in until upstream closes these issues. Llama 4 with the `llama4_pythonic` parser is worth testing but not yet trusted. Document the local model in use alongside every eval run. +- **Ollama caveat:** as of April 2026, Qwen 3 / 3.5 / Gemma 4 tool-calling is broken in Ollama — confirmed via live issue tracker (ollama/ollama#14493 Qwen 3.5 27B non-functional; #14601 Qwen3 malformed tool defs via `/api/chat`; #14745 qwen3.5:9b print-not-execute; #15315 Gemma 4 tool parser). Root cause: Ollama's renderer/parser maps Qwen 3.5 through a Hermes-style JSON pipeline, but the model was trained on Qwen3-Coder XML format; unclosed `` tags corrupt multi-turn. **Known-good local tool-calling baseline: `qwen2.5-coder:32b-instruct-q8_0`** (the variant pinned by `ModelID.qwen25coder32b`; vanilla `qwen2.5-coder:32b` is not used). Qwen3 stays disabled as opt-in until upstream closes these issues. Llama 4 with the `llama4_pythonic` parser is worth testing but not yet trusted. Document the local model in use alongside every eval run. - **Ollama transport gotcha:** `/api/chat` (native NDJSON) emits `tool_calls` on the chunk **preceding** the `done: true` terminator, not with it. Decoder must read `tool_calls` whenever seen and never gate on `done`. Separate decoders for `/api/chat` vs `/v1/chat/completions` (OpenAI-compat SSE with atomic `tool_calls`). - **Tool-choice discipline:** `LLMProvider.stream(..., toolChoice:)` is mandatory. Cap-recovery "one more call" sets `.none` (Anthropic: `{type: "none"}`; Ollama: drop `tools` array entirely). Eval asserts zero `.toolUseRequested` on recovery call. @@ -87,7 +87,7 @@ The top-level shape is fixed and agreed. Push back with specific reasoning only **Memory stack:** - **Store:** SQLite with **FTS5** (keyword search) + **sqlite-vec** extension (vector search), single file under `~/Library/Application Support/Jarvis/`. - **Embeddings:** `nomic-embed-text` via Ollama (local, free, 768-dim). -- **Extraction:** mem0's `ADD / UPDATE / NOOP` pattern — after each turn, a local extractor (Qwen 2.5-Coder 32B via Ollama) decides what facts to commit/update/ignore. **Fully local; no user data ever leaves the machine for memory.** +- **Extraction:** mem0's `ADD / UPDATE / NOOP` pattern — after each turn, a local extractor (`qwen3.6:latest` via Ollama, per commit `8d0a959`) decides what facts to commit/update/ignore. **Fully local; no user data ever leaves the machine for memory.** - **Temporal validity:** record `valid_from` / `valid_to` on facts (Zep/Graphiti style) so "Sarah works at Acme" can be superseded without deleting history. - **Scope tiers:** `UserDefaults` for trivial prefs, JSON for tool configs/feature flags, SQLite for everything conversational. - Secrets (API keys) → macOS **Keychain**, never plaintext, never checked in. @@ -227,7 +227,7 @@ Codesign + entitlement verification: `bash scripts/verify-entitlements.sh --pre- ```bash JARVIS_REAL_MODELS=1 swift test --package-path packages/Memory --filter MemoryRegressionCorpusTests -# requires: local Ollama with nomic-embed-text + qwen2.5-coder:32b + loadable vec0.dylib +# requires: local Ollama with nomic-embed-text + qwen3.6:latest (extractor) + qwen2.5-coder:32b-instruct-q8_0 (agent fallback) + loadable vec0.dylib JARVIS_REAL_CAMERA=1 swift test --package-path packages/Vision --filter CameraCaptureRealHardwareTests # requires: real AVCaptureDevice + .authorized TCC @@ -236,7 +236,7 @@ JARVIS_REAL_MODELS=1 swift test --package-path packages/Voice --filter OrpheusTT # requires: ~6 GB Orpheus weights downloaded; produces empirical TTFA measurement ``` -Eval harness (Anthropic + local-model eval matrix; corpus in `.planning/evals/`) — pinned to `qwen2.5-coder:32b` for the local-model lane. +Eval harness (Anthropic + local-model eval matrix; corpus in `.planning/evals/`) — pinned to `qwen2.5-coder:32b-instruct-q8_0` for the local-model lane (matches `ModelID.qwen25coder32b`). ### Self-hosted CI runner (Grogu) diff --git a/packages/Memory/Tests/MemoryTests/MemoryRegressionCorpusTests.swift b/packages/Memory/Tests/MemoryTests/MemoryRegressionCorpusTests.swift index 71a8981..c111248 100644 --- a/packages/Memory/Tests/MemoryTests/MemoryRegressionCorpusTests.swift +++ b/packages/Memory/Tests/MemoryTests/MemoryRegressionCorpusTests.swift @@ -12,7 +12,7 @@ import OllamaProvider // // Gated by JARVIS_REAL_MODELS=1 — skipped in CI. Runs interactively against // a real local Ollama with both `nomic-embed-text` (embedding) and -// `qwen2.5-coder:32b` (extractor) pulled. +// `qwen3.6:latest` (extractor, per commit 8d0a959) pulled. // // Run interactively on the developer's Apple Silicon Mac: // JARVIS_REAL_MODELS=1 swift test --package-path packages/Memory --filter MemoryRegressionCorpusTests @@ -37,7 +37,7 @@ final class MemoryRegressionCorpusTests: XCTestCase { private func skipIfNotRealModels() throws { try XCTSkipUnless( ProcessInfo.processInfo.environment["JARVIS_REAL_MODELS"] == "1", - "Set JARVIS_REAL_MODELS=1 to run memory regression corpus (requires local Ollama with nomic-embed-text + qwen2.5-coder:32b, plus a loadable vec0.dylib)" + "Set JARVIS_REAL_MODELS=1 to run memory regression corpus (requires local Ollama with nomic-embed-text + qwen3.6:latest, plus a loadable vec0.dylib)" ) }