feat: simple/complex model routing and fallback chains (P-e) #27

2026-10-14T00:09:49Z

KofTwentyTwo commented

2026-10-14 00:09:49 +00:00

(Migrated from github.com)

P-e: simple vs complex routing, fallback chains across providers

Spec: docs/DESIGN.md part P, item 5. This is based on develop (after #26).

Routing (per profile)

assistant_profile gains routing_mode (OFF, HEURISTIC, CLASSIFIER) and the simple, complex and classifier models, all FKs to assistant_model. OFF keeps today's behaviour, including the AssistantModelRouting escalation config.
HEURISTIC marks a question COMPLEX when any of these hold:
- write context;
- more than 600 characters;
- at least 2 tables named;
- analysis words ("analyze", "compare", "why", "trend", ...);
- more than 8 turns;
- the previous turn hit the tool limit.
It is a small, documented, unit-tested rule set.
CLASSIFIER makes one call to the classifier model: the question text only, at most 4 output tokens, and a 3-second timeout. Any error or unexpected answer falls back to the heuristic. The call is recorded as ROUTE usage and counts toward budgets. It is skipped for PHI when the classifier's provider is not eligible.
Budgets (downgrade) and the PHI rule apply after routing.
The turn records route_decision and route_source.

Fallback chains

assistant_model.fallback_model_id chains models, across providers if needed. Cycles are rejected on save, and a chain has at most 4 hops.
When it falls back: only on a retryable failure (network, timeout, overloaded, server, rate-limited) that happens before any output reached the user.
Skipped candidates: disabled, down, over a STOP budget, or failing the PHI rule. A PHI turn never falls back to a non-eligible provider.
No explicit chain: a LOCAL model falls back only to other LOCAL models of the same adapter (for example lore to grogu), never to cloud.
Health: a circuit breaker keeps health_state and down_until per provider (60 s after a connection failure). Stale LOCAL providers get a cached 2 s health probe.
Mid-turn provider switch: history is rebuilt from the provider-neutral parts, and adapter state (Anthropic thinking signatures) is dropped. A test covers a tool round trip started on A and finished on B.
Every attempt is a usage row, and the turn records fallback_count. The user sees a notice: "Provider X is unavailable; answered with Y."
The home shows UP/DOWN health and counts fallbacks.

Tests

257 in total, all green in ./mvnw verify with the JaCoCo gates (core: 182).

Routing: the heuristic rule table, the classifier paths and the PHI skip.
Fallback:
- before output versus after partial text;
- retryable versus non-retryable errors;
- a 3-hop chain, and cycles rejected;
- disabled and down candidates skipped;
- PHI never reaches a non-eligible provider;
- local to local only, and STOP budgets skip candidates.
Health: the breaker opening and closing, and the probe.
Mid-turn switch: the provider-switch tool round trip.

🤖 Generated with Claude Code

## P-e: simple vs complex routing, fallback chains across providers Spec: `docs/DESIGN.md` part P, item 5. This is based on `develop` (after #26). ### Routing (per profile) - `assistant_profile` gains `routing_mode` (OFF, HEURISTIC, CLASSIFIER) and the simple, complex and classifier models, all FKs to `assistant_model`. OFF keeps today's behaviour, including the `AssistantModelRouting` escalation config. - **HEURISTIC** marks a question COMPLEX when any of these hold: - write context; - more than 600 characters; - at least 2 tables named; - analysis words ("analyze", "compare", "why", "trend", ...); - more than 8 turns; - the previous turn hit the tool limit. It is a small, documented, unit-tested rule set. - **CLASSIFIER** makes one call to the classifier model: the question text only, at most 4 output tokens, and a 3-second timeout. Any error or unexpected answer falls back to the heuristic. The call is recorded as `ROUTE` usage and counts toward budgets. It is skipped for PHI when the classifier's provider is not eligible. - Budgets (downgrade) and the PHI rule apply after routing. - The turn records `route_decision` and `route_source`. ### Fallback chains - `assistant_model.fallback_model_id` chains models, across providers if needed. Cycles are rejected on save, and a chain has at most 4 hops. - **When it falls back:** only on a retryable failure (network, timeout, overloaded, server, rate-limited) that happens **before any output** reached the user. - **Skipped candidates:** disabled, down, over a STOP budget, or failing the PHI rule. A PHI turn never falls back to a non-eligible provider. - **No explicit chain:** a LOCAL model falls back only to other LOCAL models of the same adapter (for example lore to grogu), never to cloud. - **Health:** a circuit breaker keeps `health_state` and `down_until` per provider (60 s after a connection failure). Stale LOCAL providers get a cached 2 s health probe. - **Mid-turn provider switch:** history is rebuilt from the provider-neutral parts, and adapter state (Anthropic thinking signatures) is dropped. A test covers a tool round trip started on A and finished on B. - Every attempt is a usage row, and the turn records `fallback_count`. The user sees a notice: "Provider X is unavailable; answered with Y." - The home shows UP/DOWN health and counts fallbacks. ### Tests 257 in total, all green in `./mvnw verify` with the JaCoCo gates (core: 182). - **Routing:** the heuristic rule table, the classifier paths and the PHI skip. - **Fallback:** - before output versus after partial text; - retryable versus non-retryable errors; - a 3-hop chain, and cycles rejected; - disabled and down candidates skipped; - PHI never reaches a non-eligible provider; - local to local only, and STOP budgets skip candidates. - **Health:** the breaker opening and closing, and the probe. - **Mid-turn switch:** the provider-switch tool round trip. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: QRun-IO/qbit-assistant#27