docs(adr): record the convergent multi-engine sync model #385

2026-08-10T19:27:20Z

KofTwentyTwo commented

2026-08-10 19:27:20 +00:00

(Migrated from github.com)

Summary of Changes

Documentation only. No code, no contract changes.

Adds ADR 0002 — Convergent sync model for multiple concurrent engines and the review evidence behind it.

Why

The engine syncs as though it is the only client of an account: a forward-only {uidnext}:* high-water mark that never observes deletion, and fire-and-forget mutations recorded as completed without checking anything happened. A second client — another machine, a phone, or webmail — makes both assumptions false. A moved or deleted message becomes a permanent local ghost, and a mutation that lost a race reports success, because a UID command against a vanished UID returns OK having done nothing (RFC 3501 §6.4.8).

The decision

Build no coordination layer. IMAP and WebDAV already coordinate concurrent clients. If each engine is a correct client, N engines is O(N) server load and O(1) design complexity — so the work is finishing the sync model, not adding multi-engine support. No broker, no leader election, no CRDTs, no shared database.

One bounded exception: FORWARD and CALL WEBHOOK land on third parties with no shared resource to compare-and-set against, so ten engines page an on-call engineer ten times. Those get a single owner (filters_enabled, default off).

Four primitives: change enumeration replacing the high-water mark; conditional mutations with three outcomes (Applied/Conflict/Unknown); durable queryable conflicts; content-addressed, account-scoped identity split into messages and placements.

Why the review evidence is committed too

A draft was reviewed independently for protocol correctness, codebase fit, and adversarially. All three falsified load-bearing claims. The corrections are non-obvious and several cost real effort to establish:

Gmail advertises CONDSTORE but not QRESYNC; Exchange advertises neither. The draft's two-rung ladder put the largest provider on the most expensive path, against a published 2,500 MB/day per-account cap. Four rungs, selected per mailbox.
An absent COPYUID is conforming on success (RFC 6851 §4.3 makes it only SHOULD, and advises it untagged; RFC 4315 §3 permits omission for UIDNOTSTICKY). It cannot mean conflict — hence a third Unknown outcome.
Keying identity on hash(Message-ID) would have introduced a spoofing primitive. Message-IDs are public and echoed in References:; with INSERT OR REPLACE, a forged reuse silently overwrites the body of a trusted message and evades filters. This hole does not exist today.
The draft's own proposed convergence test could not detect its largest defect — ten forwards and one forward produce identical placement sets.
Sizing was ~3× optimistic: ~5,800–9,300 prod LOC + 4,000–6,600 test LOC, not 4–6 weeks.

Consequences worth reviewer attention

This gates the M7 API freeze. Mail has no Move/Delete/Flag RPCs; Conflict must be a typed error so old clients take the error path. Most additions are wire-additive, but redefining Message.id changes the value of every id while staying wire-compatible — invisible to the byte-exact golden, and that token appears at ten other wire positions.
Shared OAuth accounts across engines are declared out of scope — a token broker is exactly the coordination layer this ADR declines to build.
Migration is forward-only. Message-ID is never parsed or stored today, so existing rows cannot be re-keyed without re-fetching.

Type of Change

Bug fix (non-breaking change fixing an issue)
New feature (non-breaking change adding functionality)
Breaking change (fix or feature causing existing functionality to change)
Documentation update

Related Issues

Closes #

Issues for the work items will be filed against this ADR once the decision is agreed. Related open items it touches: #255 (CardDAV not representable), #257 (DAV substring XML parsing), #370 (ReDoS guard unwired), #251 (test theater), #143 (coverage gate).

Quality Verification Checklist

I have verified code formatting (cargo fmt --all -- --check)
I have verified clippy lints (cargo clippy --all-targets --workspace -- -D warnings)
I have executed the full test suite (cargo test --workspace) and all tests pass (100%)
Business logic is implemented in headless engine crates, keeping UI presentation shells skinny
No secrets, keys, or credentials are added to tracked files

Docs-only, so the gates are unaffected; verified green on the branch this was cut from.

Notes for review

docs/ROADMAP.md is deliberately not touched, so the dual-format roadmap rule does not apply. If you want this reflected in milestones, that is a follow-up that must update the rendered page in the same commit.
No .proto changes, so the one-proto-story-in-flight slot stays free.

## Summary of Changes Documentation only. No code, no contract changes. Adds **[ADR 0002 — Convergent sync model for multiple concurrent engines](docs/adr/0002-multi-engine-sync-model.md)** and the [review evidence](docs/reviews/2026-08-10-multi-engine-design-review.md) behind it. ### Why The engine syncs as though it is the only client of an account: a forward-only `{uidnext}:*` high-water mark that never observes deletion, and fire-and-forget mutations recorded as `completed` without checking anything happened. A second client — another machine, a phone, or webmail — makes both assumptions false. A moved or deleted message becomes a permanent local ghost, and a mutation that lost a race reports success, because a UID command against a vanished UID returns `OK` having done nothing (RFC 3501 §6.4.8). ### The decision **Build no coordination layer.** IMAP and WebDAV already coordinate concurrent clients. If each engine is a correct client, N engines is O(N) server load and O(1) design complexity — so the work is *finishing the sync model*, not *adding multi-engine support*. No broker, no leader election, no CRDTs, no shared database. One bounded exception: `FORWARD` and `CALL WEBHOOK` land on third parties with no shared resource to compare-and-set against, so ten engines page an on-call engineer ten times. Those get a single owner (`filters_enabled`, default off). Four primitives: change enumeration replacing the high-water mark; conditional mutations with three outcomes (`Applied`/`Conflict`/`Unknown`); durable queryable conflicts; content-addressed, account-scoped identity split into messages and placements. ### Why the review evidence is committed too A draft was reviewed independently for protocol correctness, codebase fit, and adversarially. All three falsified load-bearing claims. The corrections are non-obvious and several cost real effort to establish: - **Gmail advertises CONDSTORE but not QRESYNC; Exchange advertises neither.** The draft's two-rung ladder put the largest provider on the most expensive path, against a published 2,500 MB/day per-account cap. Four rungs, selected per *mailbox*. - **An absent `COPYUID` is conforming on success** (RFC 6851 §4.3 makes it only SHOULD, and advises it untagged; RFC 4315 §3 permits omission for UIDNOTSTICKY). It cannot mean conflict — hence a third `Unknown` outcome. - **Keying identity on `hash(Message-ID)` would have introduced a spoofing primitive.** Message-IDs are public and echoed in `References:`; with `INSERT OR REPLACE`, a forged reuse silently overwrites the body of a trusted message and evades filters. This hole does not exist today. - **The draft's own proposed convergence test could not detect its largest defect** — ten forwards and one forward produce identical placement sets. - **Sizing was ~3× optimistic**: ~5,800–9,300 prod LOC + 4,000–6,600 test LOC, not 4–6 weeks. ### Consequences worth reviewer attention - **This gates the M7 API freeze.** `Mail` has no Move/Delete/Flag RPCs; `Conflict` must be a typed error so old clients take the error path. Most additions are wire-additive, but redefining `Message.id` changes the *value* of every id while staying wire-compatible — invisible to the byte-exact golden, and that token appears at ten other wire positions. - **Shared OAuth accounts across engines are declared out of scope** — a token broker is exactly the coordination layer this ADR declines to build. - **Migration is forward-only.** Message-ID is never parsed or stored today, so existing rows cannot be re-keyed without re-fetching. ## Type of Change - [ ] Bug fix (non-breaking change fixing an issue) - [ ] New feature (non-breaking change adding functionality) - [ ] Breaking change (fix or feature causing existing functionality to change) - [x] Documentation update ## Related Issues Closes # Issues for the work items will be filed against this ADR once the decision is agreed. Related open items it touches: #255 (CardDAV not representable), #257 (DAV substring XML parsing), #370 (ReDoS guard unwired), #251 (test theater), #143 (coverage gate). ## Quality Verification Checklist - [x] I have verified code formatting (`cargo fmt --all -- --check`) - [x] I have verified clippy lints (`cargo clippy --all-targets --workspace -- -D warnings`) - [x] I have executed the full test suite (`cargo test --workspace`) and all tests pass (100%) - [x] Business logic is implemented in headless engine crates, keeping UI presentation shells skinny - [x] No secrets, keys, or credentials are added to tracked files Docs-only, so the gates are unaffected; verified green on the branch this was cut from. ## Notes for review - `docs/ROADMAP.md` is deliberately **not** touched, so the dual-format roadmap rule does not apply. If you want this reflected in milestones, that is a follow-up that must update the rendered page in the same commit. - No `.proto` changes, so the one-proto-story-in-flight slot stays free.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: KofTwentyTwo/nuncio#385