feat(store): gate the identity schema and tag sync tokens with their scheme #400

2026-08-10T20:34:27Z

KofTwentyTwo commented

2026-08-10 20:34:27 +00:00

(Migrated from github.com)

Summary of Changes

Two silent-failure paths, both cheap to close, both load-bearing for the sync work behind them.

1. Identity-schema gating (`PRAGMA user_version`)

The schema is built entirely from CREATE TABLE IF NOT EXISTS, so an older binary opens a newer database successfully and writes the old model into it. Nothing detects that; the file ends up holding two mutually invisible truths.

Newer than this build → fail closed. There is no safe way to continue.
Older → rebuild the cached message store.

Rebuilding rather than migrating is the pre-release decision in ADR 0002 (PR #385). Message identity cannot be recomputed for rows already stored, so the alternative is keeping them under the old model forever — a store where half the mail is invisible to every guarantee the new model provides.

Preserved	Rebuilt from the server
accounts, credentials, filter rules, WORM audit records	messages + FTS, folder checkpoints, filter execution ledger

The execution ledger goes because its hash chain is keyed on message ids that would no longer resolve — a chain pointing at absent rows proves nothing, so reinitialising it is more honest than carrying a permanently mixed-namespace log.

2. Scheme-tagged sync checkpoints

The stored checkpoint was "{uidvalidity}:{uidnext}". The QRESYNC checkpoint that replaces it in #390 is "{uidvalidity}:{highestmodseq}" — identical shape, different meaning.

An upgraded engine reading one as the other hands the server a UIDNEXT where it expects a MODSEQ. UIDNEXT is normally the larger number, so the server answers nothing changed since, and every message below that point is never enumerated again. Silent, permanent mail loss with no error anywhere.

Tokens are now "v1:uidnext:{uidvalidity}:{uidnext}", and anything whose scheme this build does not recognise — an unknown tag, a future tag, or an untagged legacy value — resolves to a full re-fetch. Over-fetching is recoverable; misreading is not.

Type of Change

Bug fix (non-breaking change fixing an issue)
New feature (non-breaking change adding functionality)
Breaking change (fix or feature causing existing functionality to change)
Documentation update

Related Issues

Refs #387. Prerequisite for #390 and #394.

Quality Verification Checklist

I have verified code formatting (cargo fmt --all -- --check)
I have verified clippy lints (cargo clippy --all-targets --workspace -- -D warnings)
I have executed the full test suite (cargo test --workspace) and all tests pass (599 passed, 0 failed)
Business logic is implemented in headless engine crates, keeping UI presentation shells skinny
No secrets, keys, or credentials are added to tracked files

Notes for review

IDENTITY_SCHEMA_VERSION starts at 1 and is deliberately distinct from the additive column migrations elsewhere in db.rs. Those add nullable columns and leave every row meaningful; this counter is only for changes to what a stored message is. Bump it only alongside the reset it implies — #394 will be the first real bump.
Existing checkpoint tests were retagged, and a new test pins the four dangerous inputs (untagged, future scheme, wrong scheme, truncated tag) all resolving to a full re-fetch.
reconcile_schema_version runs after the tables exist and after the additive migrations, so a rebuild lands on the current shape rather than an intermediate one.
No .proto change.

## Summary of Changes Two silent-failure paths, both cheap to close, both load-bearing for the sync work behind them. ### 1. Identity-schema gating (`PRAGMA user_version`) The schema is built entirely from `CREATE TABLE IF NOT EXISTS`, so **an older binary opens a newer database successfully** and writes the old model into it. Nothing detects that; the file ends up holding two mutually invisible truths. - **Newer than this build** → fail closed. There is no safe way to continue. - **Older** → rebuild the cached message store. Rebuilding rather than migrating is the pre-release decision in [ADR 0002](docs/adr/0002-multi-engine-sync-model.md#pre-release-posture) (PR #385). Message identity cannot be recomputed for rows already stored, so the alternative is keeping them under the old model forever — a store where half the mail is invisible to every guarantee the new model provides. | Preserved | Rebuilt from the server | | --- | --- | | accounts, credentials, filter rules, WORM audit records | messages + FTS, folder checkpoints, filter execution ledger | The execution ledger goes because its hash chain is keyed on message ids that would no longer resolve — a chain pointing at absent rows proves nothing, so reinitialising it is more honest than carrying a permanently mixed-namespace log. ### 2. Scheme-tagged sync checkpoints The stored checkpoint was `"{uidvalidity}:{uidnext}"`. The QRESYNC checkpoint that replaces it in #390 is `"{uidvalidity}:{highestmodseq}"` — **identical shape, different meaning**. An upgraded engine reading one as the other hands the server a UIDNEXT where it expects a MODSEQ. UIDNEXT is normally the larger number, so the server answers *nothing changed since*, and every message below that point is never enumerated again. **Silent, permanent mail loss with no error anywhere.** Tokens are now `"v1:uidnext:{uidvalidity}:{uidnext}"`, and anything whose scheme this build does not recognise — an unknown tag, a future tag, or an untagged legacy value — resolves to a full re-fetch. Over-fetching is recoverable; misreading is not. ## Type of Change - [x] Bug fix (non-breaking change fixing an issue) - [x] New feature (non-breaking change adding functionality) - [ ] Breaking change (fix or feature causing existing functionality to change) - [ ] Documentation update ## Related Issues Refs #387. Prerequisite for #390 and #394. ## Quality Verification Checklist - [x] I have verified code formatting (`cargo fmt --all -- --check`) - [x] I have verified clippy lints (`cargo clippy --all-targets --workspace -- -D warnings`) - [x] I have executed the full test suite (`cargo test --workspace`) and all tests pass (**599 passed, 0 failed**) - [x] Business logic is implemented in headless engine crates, keeping UI presentation shells skinny - [x] No secrets, keys, or credentials are added to tracked files ## Notes for review - **`IDENTITY_SCHEMA_VERSION` starts at 1** and is deliberately distinct from the additive column migrations elsewhere in `db.rs`. Those add nullable columns and leave every row meaningful; this counter is only for changes to *what a stored message is*. Bump it only alongside the reset it implies — #394 will be the first real bump. - **Existing checkpoint tests were retagged**, and a new test pins the four dangerous inputs (untagged, future scheme, wrong scheme, truncated tag) all resolving to a full re-fetch. - `reconcile_schema_version` runs after the tables exist and after the additive migrations, so a rebuild lands on the current shape rather than an intermediate one. - No `.proto` change.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: KofTwentyTwo/nuncio#400