Agent: prompt cache hints never trigger in production — preamble below 4096-char threshold #23

New Issue

2026-05-12T19:05:10Z

KofTwentyTwo commented

2026-05-12 19:05:10 +00:00

(Migrated from github.com)

Summary

Cache hints effectively never trigger in production. MIN_CHARS_FOR_CACHE_BREAKPOINT = 4096 but the locked selfAwarePreamble is ~700 chars; .empty adds no presence/memory hydration; Round-4 degradation summary stays under threshold by design. On healthy boot, finalSystem.count is ~700-1200 chars, CacheHints.eligibleForSystemPrompt(finalSystem) returns nil on every turn. Not a correctness bug — but the prompt cache is effectively dead. Every turn pays full input-token cost. With Opus 4.7's ~1.35× token inflation vs 3.x, this matters more.

Evidence

File: packages/AgentCore/Sources/AgentCore/CacheHints.swift:50 — MIN_CHARS_FOR_CACHE_BREAKPOINT = 4096
File: packages/AgentCore/Sources/AgentCore/ContextBuilder.swift:64-74 — ~700-char preamble
File: App/AppDelegate.swift:1685 — orchestrator constructed with .empty context
File: AgentOrchestrator.swift:118 — Round-4 degradation summary bounded below 4096

Root cause / hypothesis

Phase E fix correctly returns nil to avoid the 200 OK + EOF cache-validation-failure cascade. But on a healthy boot the prompt never crosses the threshold, so cache eligibility is structurally false. Possible intentional cost-acceptance, possible oversight — not documented either way.

Recommended fix

Not a fix request — flagging for awareness. Possible mitigations:

Pad the preamble to ≥4096 chars (would need real prompt-engineering work)
Accept that cache is unused until memory hydration grows the prompt naturally (M-4 lands memory hydration → preamble grows organically)
Comment in CacheHints.swift documenting the trade-off explicitly

Decide whether this is intentional cost-acceptance or oversight; comment accordingly.

Test

Observation: instrument CacheHints.eligibleForSystemPrompt calls; assert in healthy boot the rate of non-nil returns matches the chosen policy

Source

Audit: .planning/audit-2026-05-12/mcp-agent-audit.md (2026-05-12) — finding HIGH-1

Cross-references

Related: #8 (M-4 Memory surface) — memory hydration will naturally grow the prompt past threshold

## Summary Cache hints effectively never trigger in production. `MIN_CHARS_FOR_CACHE_BREAKPOINT = 4096` but the locked `selfAwarePreamble` is ~700 chars; `.empty` adds no presence/memory hydration; Round-4 degradation summary stays under threshold by design. On healthy boot, `finalSystem.count` is ~700-1200 chars, `CacheHints.eligibleForSystemPrompt(finalSystem)` returns `nil` on every turn. Not a correctness bug — but the prompt cache is effectively dead. Every turn pays full input-token cost. With Opus 4.7's ~1.35× token inflation vs 3.x, this matters more. ## Evidence - File: `packages/AgentCore/Sources/AgentCore/CacheHints.swift:50` — `MIN_CHARS_FOR_CACHE_BREAKPOINT = 4096` - File: `packages/AgentCore/Sources/AgentCore/ContextBuilder.swift:64-74` — ~700-char preamble - File: `App/AppDelegate.swift:1685` — orchestrator constructed with `.empty` context - File: `AgentOrchestrator.swift:118` — Round-4 degradation summary bounded below 4096 ## Root cause / hypothesis Phase E fix correctly returns `nil` to avoid the 200 OK + EOF cache-validation-failure cascade. But on a healthy boot the prompt never crosses the threshold, so cache eligibility is structurally false. Possible intentional cost-acceptance, possible oversight — not documented either way. ## Recommended fix Not a fix request — flagging for awareness. Possible mitigations: - Pad the preamble to ≥4096 chars (would need real prompt-engineering work) - Accept that cache is unused until memory hydration grows the prompt naturally (M-4 lands memory hydration → preamble grows organically) - Comment in `CacheHints.swift` documenting the trade-off explicitly Decide whether this is intentional cost-acceptance or oversight; comment accordingly. ## Test - Observation: instrument `CacheHints.eligibleForSystemPrompt` calls; assert in healthy boot the rate of non-nil returns matches the chosen policy ## Source - Audit: `.planning/audit-2026-05-12/mcp-agent-audit.md` (2026-05-12) — finding HIGH-1 ## Cross-references - Related: #8 (M-4 Memory surface) — memory hydration will naturally grow the prompt past threshold

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: KofTwentyTwo/Jarvis#23