fix(alerts): page spike history instead of holding 7 days of query snapshots #17

2026-10-16T20:24:50Z

KofTwentyTwo commented

2026-10-16 20:24:50 +00:00

(Migrated from github.com)

Why

This is the follow-up promised in #16. After that PR fixed the attribution reads, the remaining large read on the kof22 worker is SnapshotEvaluator.spikes. On every snapshot evaluation it loaded RuleContext.window("query_snapshot", 7) as QRecords. kof22 writes about 33,000 query snapshots a day, so by day 7 each evaluation would hold about 230,000 records, which is hundreds of MB of worker heap.

What

SnapshotEvaluator.history(ctx, currentSnapshotId) reads the 7-day window in DbPerfStore.BATCH_SIZE keyset pages through DbPerfStore.forEachPage (added in #16). As before, it keeps only OK and PARTIAL snapshots.
For each statement, StatementHistory keeps only three things: the current interval's totalMs, the latest capture time, and earlier interval times in a growable double[].
The spike rule is unchanged: at least twelve prior samples, the median, the floor/factor settings, and absence after one hour.

Tests

SpikeHistoryBoundedTest counts every db_perf_query_snapshot record QQQ loads. With 2,500 samples, no page exceeds 1,000 rows. Samples from FAILED snapshots and from outside the window are excluded, and the current interval is found.
The existing SnapshotEvaluatorTest tests for baseline, median, recovery and absence still pass.
./mvnw verify is green, including the JaCoCo gate.

🤖 Generated with Claude Code

## Why This is the follow-up promised in #16. After that PR fixed the attribution reads, the remaining large read on the kof22 worker is `SnapshotEvaluator.spikes`. On every snapshot evaluation it loaded `RuleContext.window("query_snapshot", 7)` as `QRecord`s. kof22 writes about 33,000 query snapshots a day, so by day 7 each evaluation would hold about 230,000 records, which is hundreds of MB of worker heap. ## What - `SnapshotEvaluator.history(ctx, currentSnapshotId)` reads the 7-day window in `DbPerfStore.BATCH_SIZE` keyset pages through `DbPerfStore.forEachPage` (added in #16). As before, it keeps only OK and PARTIAL snapshots. - For each statement, `StatementHistory` keeps only three things: the current interval's `totalMs`, the latest capture time, and earlier interval times in a growable `double[]`. - The spike rule is unchanged: at least twelve prior samples, the median, the floor/factor settings, and absence after one hour. ## Tests - `SpikeHistoryBoundedTest` counts every `db_perf_query_snapshot` record QQQ loads. With 2,500 samples, no page exceeds 1,000 rows. Samples from FAILED snapshots and from outside the window are excluded, and the current interval is found. - The existing `SnapshotEvaluatorTest` tests for baseline, median, recovery and absence still pass. - `./mvnw verify` is green, including the JaCoCo gate. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: QRun-IO/qbit-db-performance#17