feat(model): qwen3.6 for memory extractor; q8_0 for agent-loop fallback
Round 1 of the post-D-5/D-6 model strategy split. The user has both `qwen3.6:latest` (~22GB MoE) and `qwen2.5-coder:32b-instruct-q8_0` (~32GB) pulled. CLAUDE.md previously pinned the entire local-model strategy to the base `qwen2.5-coder:32b` because of an April 2026 Ollama tool-call bug affecting Qwen 3 / 3.5 / Gemma 4. That ban was correct then; today (2026-05-11) it isn't. **Empirical retest 2026-05-11:** POST /api/chat (non-streaming) against qwen3.6:latest with a representative `apply_memory_ops` tool definition returned a clean tool_calls array, well-formed JSON args, no leaked <think> tags. Upstream Ollama appears to have closed the relevant issues (#14493 / #14601 / #14745 / #15315) in the interim. Documented in ModelID.qwen36 doc comment. Two-model strategy: - **MemoryExtractor default → qwen3.6:latest** (`ModelID.qwen36`) Single-turn, single-tool, no cap recovery — the lowest-risk place to validate the newer model. If extraction stays stable for a few days of dogfooding, we revisit the agent-loop fallback. - **Agent-loop Ollama fallback → qwen2.5-coder:32b-instruct-q8_0** (`ModelID.qwen25coder32b`'s rawValue is now the Q8_0 variant the user already has — higher fidelity than the default Q4_K_M, same model family, same tool-call wire format). The agent loop has multi-turn tool chains, cap recovery, tool_choice=none paths, and streaming all in scope — kept conservative until Qwen3.6 is verified there too. Changes: - packages/AgentCore/Sources/AgentCore/ModelID.swift + `qwen36` constant for qwen3.6:latest with empirical retest doc. ! `qwen25coder32b` rawValue is now `qwen2.5-coder:32b-instruct-q8_0` (same constant name, the Q8_0 raw value). All ModelID.qwen25coder32b consumers pick up the change automatically. - packages/Memory/Sources/Memory/MemoryExtractor.swift - `model: ModelID = .qwen25coder32b` → `.qwen36`. Docstring updated to explain the empirical retest + why the extractor is the safest place for the newer model. - packages/Harness/Sources/Harness/Runners/LiveOllamaRunner.swift `requiredModelID` constant now sources from `ModelID.qwen25coder32b.rawValue` so a future single-edit on the canonical model name propagates here automatically. Doc updated to point at the q8_0 variant + ~32GB pull size. User-facing note: no model pulls required for this commit; both models are already on the user's machine. Round 2 (boot-phase health check) and Round 3 (Status menu dialog) deferred to a fresh session for context budget — handoff doc to follow. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
1 parent
cb584a2ac8
commit
8d0a959a19
3 files changed
+41
-12
No files matched your search
@@ -17,7 +17,28 @@ public struct ModelID: Sendable, RawRepresentable, Equatable, Hashable {
|
||||
/// arbitrary string model IDs.
|
||||
public static let opus47 = ModelID(rawValue: "claude-opus-4-7")
|
||||
|
||||
/// Local Ollama model — known-good tool-calling baseline per CLAUDE.md
|
||||
/// (Qwen3 / Gemma 4 tool-calling broken in Ollama as of April 2026).
|
||||
public static let qwen25coder32b = ModelID(rawValue: "qwen2.5-coder:32b")
|
||||
/// Local Ollama agent-loop fallback. Q8_0 quantization of the project's
|
||||
/// CLAUDE.md known-good tool-calling baseline. Higher fidelity than
|
||||
/// the default Q4_K_M; same model family, same tool-call wire format.
|
||||
/// Keep this conservative for the agent loop until multi-turn /
|
||||
/// cap-recovery / streaming tool-calls are verified on newer models
|
||||
/// (see `qwen36` doc).
|
||||
public static let qwen25coder32b = ModelID(rawValue: "qwen2.5-coder:32b-instruct-q8_0")
|
||||
|
||||
/// Qwen3.6 (~36B MoE, Q4_K_M, runs comfortably on Apple Silicon Pro/Max).
|
||||
///
|
||||
/// **2026-05-11 empirical retest:** Ollama `/api/chat` non-streaming
|
||||
/// tool-calling against this model returns a well-formed `tool_calls`
|
||||
/// array with clean JSON args — no leaked `<think>` tags clobbering
|
||||
/// the response, no malformed parser output. This contradicts the
|
||||
/// April 2026 CLAUDE.md note that Qwen3 tool-calling was broken in
|
||||
/// Ollama (#14493 / #14601 / #14745); upstream appears to have closed
|
||||
/// those issues in the interim.
|
||||
///
|
||||
/// **Used as the memory extractor default** (`MemoryExtractor.init`
|
||||
/// `model: .qwen36`). The extractor is single-turn, single tool, no
|
||||
/// cap recovery — the safest place to test a newer model. If
|
||||
/// extraction is stable for a few days of dogfooding, we revisit the
|
||||
/// agent-loop fallback.
|
||||
public static let qwen36 = ModelID(rawValue: "qwen3.6:latest")
|
||||
}
|
||||
@@ -13,10 +13,11 @@ import OllamaProvider
|
||||
/// **D-06 preflight contract — never auto-pull:**
|
||||
/// 1. GET `http://127.0.0.1:11434/api/tags` → must be 200. Fail loud
|
||||
/// with operator instruction (`start ollama serve`).
|
||||
/// 2. The returned `models` array must contain `qwen2.5-coder:32b`. Fail
|
||||
/// loud with operator instruction (`run ollama pull qwen2.5-coder:32b`).
|
||||
/// 2. The returned `models` array must contain `ModelID.qwen25coder32b
|
||||
/// .rawValue` (today: `qwen2.5-coder:32b-instruct-q8_0`). Fail loud
|
||||
/// with operator instruction (`run ollama pull <that-id>`).
|
||||
/// 3. **NEVER** invoke `ollama pull` from inside this runner. The model
|
||||
/// is ~20 GB; the operator must opt into the download.
|
||||
/// is ~32 GB; the operator must opt into the download.
|
||||
public actor LiveOllamaRunner {
|
||||
|
||||
public enum LiveError: Swift.Error, Equatable, Sendable {
|
||||
@@ -33,7 +34,10 @@ public actor LiveOllamaRunner {
|
||||
}
|
||||
|
||||
public static let baseURL = URL(string: "http://127.0.0.1:11434")!
|
||||
public static let requiredModelID = "qwen2.5-coder:32b"
|
||||
/// Sourced from `ModelID.qwen25coder32b` so a single edit to the
|
||||
/// canonical agent-loop fallback (in AgentCore/ModelID.swift)
|
||||
/// propagates here automatically.
|
||||
public static let requiredModelID = ModelID.qwen25coder32b.rawValue
|
||||
|
||||
public init() {}
|
||||
|
||||
|
||||
@@ -7,10 +7,14 @@ import Logging
|
||||
/// `LLMProvider.stream` — collects exactly one `ToolUseRequest` for
|
||||
/// `apply_memory_ops` and decodes it into `[MemoryOp]`.
|
||||
///
|
||||
/// Per RESEARCH §5 and D-03, the model is `qwen2.5-coder:32b` (Qwen3 is
|
||||
/// banned per RESEARCH-DELTAS D3 — Ollama tool-calling broken). The
|
||||
/// extractor asks for tool-only output via a system prompt directive and
|
||||
/// uses `toolChoice .auto` so the model can also decide NOOP.
|
||||
/// Model default updated 2026-05-11: `qwen3.6:latest` after an empirical
|
||||
/// retest showed Qwen3.6 tool-calling works correctly in Ollama (see
|
||||
/// `ModelID.qwen36` doc). The original April 2026 CLAUDE.md note banning
|
||||
/// Qwen3 was correct at the time; upstream has since closed the relevant
|
||||
/// issues. The extractor's single-turn, single-tool flow is the lowest-
|
||||
/// risk place to validate the newer model — no cap recovery, no multi-
|
||||
/// turn chains. The agent loop remains pinned to `qwen2.5-coder:32b-
|
||||
/// instruct-q8_0` until streaming + cap-recovery on Qwen3.6 is verified.
|
||||
///
|
||||
/// **Single-turn only.** No outer loop, no cap recovery, no
|
||||
/// UntrustedWrapper. If the model fails to call the tool, the extractor
|
||||
@@ -23,7 +27,7 @@ public actor MemoryExtractor {
|
||||
|
||||
public init(
|
||||
provider: any LLMProvider,
|
||||
model: ModelID = .qwen25coder32b
|
||||
model: ModelID = .qwen36
|
||||
) {
|
||||
self.provider = provider
|
||||
self.model = model
|
||||
|
||||
Reference in new issue
Block a user