feat(model): qwen3.6 for memory extractor; q8_0 for agent-loop fallback

Round 1 of the post-D-5/D-6 model strategy split. The user has both
`qwen3.6:latest` (~22GB MoE) and `qwen2.5-coder:32b-instruct-q8_0` (~32GB)
pulled. CLAUDE.md previously pinned the entire local-model strategy to
the base `qwen2.5-coder:32b` because of an April 2026 Ollama tool-call
bug affecting Qwen 3 / 3.5 / Gemma 4. That ban was correct then; today
(2026-05-11) it isn't.

**Empirical retest 2026-05-11:** POST /api/chat (non-streaming) against
qwen3.6:latest with a representative `apply_memory_ops` tool definition
returned a clean tool_calls array, well-formed JSON args, no leaked
<think> tags. Upstream Ollama appears to have closed the relevant
issues (#14493 / #14601 / #14745 / #15315) in the interim. Documented
in ModelID.qwen36 doc comment.

Two-model strategy:

- **MemoryExtractor default → qwen3.6:latest** (`ModelID.qwen36`)
  Single-turn, single-tool, no cap recovery — the lowest-risk place to
  validate the newer model. If extraction stays stable for a few days
  of dogfooding, we revisit the agent-loop fallback.
- **Agent-loop Ollama fallback → qwen2.5-coder:32b-instruct-q8_0**
  (`ModelID.qwen25coder32b`'s rawValue is now the Q8_0 variant the user
  already has — higher fidelity than the default Q4_K_M, same model
  family, same tool-call wire format). The agent loop has multi-turn
  tool chains, cap recovery, tool_choice=none paths, and streaming all
  in scope — kept conservative until Qwen3.6 is verified there too.

Changes:
- packages/AgentCore/Sources/AgentCore/ModelID.swift
    + `qwen36` constant for qwen3.6:latest with empirical retest doc.
    ! `qwen25coder32b` rawValue is now `qwen2.5-coder:32b-instruct-q8_0`
      (same constant name, the Q8_0 raw value). All ModelID.qwen25coder32b
      consumers pick up the change automatically.
- packages/Memory/Sources/Memory/MemoryExtractor.swift
    - `model: ModelID = .qwen25coder32b` → `.qwen36`.
    Docstring updated to explain the empirical retest + why the
    extractor is the safest place for the newer model.
- packages/Harness/Sources/Harness/Runners/LiveOllamaRunner.swift
    `requiredModelID` constant now sources from
    `ModelID.qwen25coder32b.rawValue` so a future single-edit on the
    canonical model name propagates here automatically. Doc updated to
    point at the q8_0 variant + ~32GB pull size.

User-facing note: no model pulls required for this commit; both
models are already on the user's machine.

Round 2 (boot-phase health check) and Round 3 (Status menu dialog)
deferred to a fresh session for context budget — handoff doc to follow.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
james.maesandClaude Opus 4.7 committed 2026-05-11 10:36:30 -05:00
1 parent cb584a2ac8
commit 8d0a959a19
3 files changed
+41 -12

No files matched your search

@@ -17,7 +17,28 @@ public struct ModelID: Sendable, RawRepresentable, Equatable, Hashable {
/// arbitrary string model IDs.
public static let opus47 = ModelID(rawValue: "claude-opus-4-7")
/// Local Ollama model — known-good tool-calling baseline per CLAUDE.md
/// (Qwen3 / Gemma 4 tool-calling broken in Ollama as of April 2026).
public static let qwen25coder32b = ModelID(rawValue: "qwen2.5-coder:32b")
/// Local Ollama agent-loop fallback. Q8_0 quantization of the project's
/// CLAUDE.md known-good tool-calling baseline. Higher fidelity than
/// the default Q4_K_M; same model family, same tool-call wire format.
/// Keep this conservative for the agent loop until multi-turn /
/// cap-recovery / streaming tool-calls are verified on newer models
/// (see `qwen36` doc).
public static let qwen25coder32b = ModelID(rawValue: "qwen2.5-coder:32b-instruct-q8_0")
/// Qwen3.6 (~36B MoE, Q4_K_M, runs comfortably on Apple Silicon Pro/Max).
///
/// **2026-05-11 empirical retest:** Ollama `/api/chat` non-streaming
/// tool-calling against this model returns a well-formed `tool_calls`
/// array with clean JSON args — no leaked `<think>` tags clobbering
/// the response, no malformed parser output. This contradicts the
/// April 2026 CLAUDE.md note that Qwen3 tool-calling was broken in
/// Ollama (#14493 / #14601 / #14745); upstream appears to have closed
/// those issues in the interim.
///
/// **Used as the memory extractor default** (`MemoryExtractor.init`
/// `model: .qwen36`). The extractor is single-turn, single tool, no
/// cap recovery — the safest place to test a newer model. If
/// extraction is stable for a few days of dogfooding, we revisit the
/// agent-loop fallback.
public static let qwen36 = ModelID(rawValue: "qwen3.6:latest")
}
@@ -13,10 +13,11 @@ import OllamaProvider
/// **D-06 preflight contract — never auto-pull:**
/// 1. GET `http://127.0.0.1:11434/api/tags` → must be 200. Fail loud
/// with operator instruction (`start ollama serve`).
/// 2. The returned `models` array must contain `qwen2.5-coder:32b`. Fail
/// loud with operator instruction (`run ollama pull qwen2.5-coder:32b`).
/// 2. The returned `models` array must contain `ModelID.qwen25coder32b
/// .rawValue` (today: `qwen2.5-coder:32b-instruct-q8_0`). Fail loud
/// with operator instruction (`run ollama pull <that-id>`).
/// 3. **NEVER** invoke `ollama pull` from inside this runner. The model
/// is ~20 GB; the operator must opt into the download.
/// is ~32 GB; the operator must opt into the download.
public actor LiveOllamaRunner {
public enum LiveError: Swift.Error, Equatable, Sendable {
@@ -33,7 +34,10 @@ public actor LiveOllamaRunner {
}
public static let baseURL = URL(string: "http://127.0.0.1:11434")!
public static let requiredModelID = "qwen2.5-coder:32b"
/// Sourced from `ModelID.qwen25coder32b` so a single edit to the
/// canonical agent-loop fallback (in AgentCore/ModelID.swift)
/// propagates here automatically.
public static let requiredModelID = ModelID.qwen25coder32b.rawValue
public init() {}
@@ -7,10 +7,14 @@ import Logging
/// `LLMProvider.stream` — collects exactly one `ToolUseRequest` for
/// `apply_memory_ops` and decodes it into `[MemoryOp]`.
///
/// Per RESEARCH §5 and D-03, the model is `qwen2.5-coder:32b` (Qwen3 is
/// banned per RESEARCH-DELTAS D3 — Ollama tool-calling broken). The
/// extractor asks for tool-only output via a system prompt directive and
/// uses `toolChoice .auto` so the model can also decide NOOP.
/// Model default updated 2026-05-11: `qwen3.6:latest` after an empirical
/// retest showed Qwen3.6 tool-calling works correctly in Ollama (see
/// `ModelID.qwen36` doc). The original April 2026 CLAUDE.md note banning
/// Qwen3 was correct at the time; upstream has since closed the relevant
/// issues. The extractor's single-turn, single-tool flow is the lowest-
/// risk place to validate the newer model — no cap recovery, no multi-
/// turn chains. The agent loop remains pinned to `qwen2.5-coder:32b-
/// instruct-q8_0` until streaming + cap-recovery on Qwen3.6 is verified.
///
/// **Single-turn only.** No outer loop, no cap recovery, no
/// UntrustedWrapper. If the model fails to call the tool, the extractor
@@ -23,7 +27,7 @@ public actor MemoryExtractor {
public init(
provider: any LLMProvider,
model: ModelID = .qwen25coder32b
model: ModelID = .qwen36
) {
self.provider = provider
self.model = model