docs(superpowers): local-first LLM routing design spec #197

2026-05-14T19:13:27Z

KofTwentyTwo commented

2026-05-14 19:13:27 +00:00

(Migrated from github.com)

Summary

Design spec for flipping the default chat-turn provider from Anthropic to local Ollama (`qwen2.5-coder:32b-instruct-q8_0`) with reactive escalation to Claude Opus 4.7 only on detectable Ollama failure modes (`streamTruncated`, `malformedToolCall`, `unknownTool`, `refusal`, `connectionFailure`, `emptyResponse`).

Approach: reactive fallback only — no predictive routing, no router-model classifier, no self-reported escalation tool. Extends the existing orchestrator escalation hook at `AgentOrchestrator.swift:793`.

Key design choices

Per-turn escalation budget — at most one Ollama→Anthropic switch per turn; second failure surfaces an error.
Per-turn (not sticky) — next user message starts fresh on Ollama.
Clean restart with visible badge — HUD shows a one-line "Escalated to Opus — " badge on the affected message.
Memory extraction unchanged — already Ollama-only via `MemoryExtractor` with injected provider.
Settings UX — `Local-first (recommended)` / `Anthropic only` / `Ollama only`.

Test plan

Reactive fallback fires on `malformedToolCall` fixture
Escalation budget honored (no infinite loop when both providers fail)
Pinned-Ollama mode disables escalation entirely
Existing eval harness lanes still green

Next: `superpowers:writing-plans` produces PLAN.md; implementation lands in a single follow-up PR (~3-4 hrs estimated).

🤖 Generated with Claude Code

## Summary Design spec for flipping the default chat-turn provider from Anthropic to local Ollama (\`qwen2.5-coder:32b-instruct-q8_0\`) with reactive escalation to Claude Opus 4.7 only on detectable Ollama failure modes (\`streamTruncated\`, \`malformedToolCall\`, \`unknownTool\`, \`refusal\`, \`connectionFailure\`, \`emptyResponse\`). Approach: reactive fallback only — no predictive routing, no router-model classifier, no self-reported escalation tool. Extends the existing orchestrator escalation hook at \`AgentOrchestrator.swift:793\`. ## Key design choices - **Per-turn escalation budget** — at most one Ollama→Anthropic switch per turn; second failure surfaces an error. - **Per-turn (not sticky)** — next user message starts fresh on Ollama. - **Clean restart with visible badge** — HUD shows a one-line "Escalated to Opus — <reason>" badge on the affected message. - **Memory extraction unchanged** — already Ollama-only via \`MemoryExtractor\` with injected provider. - **Settings UX** — \`Local-first (recommended)\` / \`Anthropic only\` / \`Ollama only\`. ## Test plan - [ ] Reactive fallback fires on \`malformedToolCall\` fixture - [ ] Escalation budget honored (no infinite loop when both providers fail) - [ ] Pinned-Ollama mode disables escalation entirely - [ ] Existing eval harness lanes still green Next: \`superpowers:writing-plans\` produces PLAN.md; implementation lands in a single follow-up PR (~3-4 hrs estimated). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: KofTwentyTwo/Jarvis#197