sync: IMAP change enumeration with a four-rung, per-mailbox capability ladder #390

New Issue

2026-08-10T19:28:11Z

KofTwentyTwo commented

2026-08-10 19:28:11 +00:00

(Migrated from github.com)

Highest value per unit of work. Ships alone and fixes ghost messages for single-engine users today.

Sync is a forward-only {uidnext}:* high-water mark and fetch_and_persist only ever calls save_email. Nothing observes deletion, so any message moved or removed by another client — including the user's own phone or webmail — becomes a permanent local ghost.

Replace it with CollectionSync::enumerate_changes(collection, since) -> {upserts, removals, next_token}. removals is the entire fix.

The ladder — selected per mailbox, not per account

Rung	Mechanism	Providers
1	QRESYNC `SELECT (QRESYNC …)` → `VANISHED`	Fastmail/Cyrus, Dovecot
2	CONDSTORE `UID FETCH 1:* (FLAGS) (CHANGEDSINCE n)` + periodic UID-set diff	Gmail
3	`STATUS`/`LIST-STATUS` gate, then ESEARCH UID-set diff	Exchange, Yahoo
4	Full resync	last resort

Gmail has CONDSTORE but no QRESYNC; Exchange has neither. NOMODSEQ is a mailbox property (RFC 7162 §3.1.2.2), so detection must be per mailbox.

Notes

QRESYNC never signals modseq expiry — RFC 7162 §3.2.6 makes a too-old modseq a large VANISHED, not an error. full_resync_required is always client-derived (UIDVALIDITY mismatch, NOMODSEQ, HIGHESTMODSEQ regression).
Sync state is structured: (uidvalidity, highestmodseq, known-uid-set, seq-match-sample).
Keep a periodic full UID-set reconciliation even on rung 1.
Generalize folder_sync_state to collection_sync_state(account_id, collection_id, token).

Depends on #387, #388. Testable against #384 (all three server profiles).

Design: ADR 0002 · review evidence (PR #385)

**Highest value per unit of work. Ships alone and fixes ghost messages for single-engine users today.** Sync is a forward-only `{uidnext}:*` high-water mark and `fetch_and_persist` only ever calls `save_email`. Nothing observes deletion, so any message moved or removed by another client — including the user's own phone or webmail — becomes a permanent local ghost. Replace it with `CollectionSync::enumerate_changes(collection, since) -> {upserts, removals, next_token}`. **`removals` is the entire fix.** ### The ladder — selected per *mailbox*, not per account | Rung | Mechanism | Providers | | --- | --- | --- | | 1 | QRESYNC `SELECT (QRESYNC …)` → `VANISHED` | Fastmail/Cyrus, Dovecot | | 2 | CONDSTORE `UID FETCH 1:* (FLAGS) (CHANGEDSINCE n)` + periodic UID-set diff | **Gmail** | | 3 | `STATUS`/`LIST-STATUS` gate, then ESEARCH UID-set diff | **Exchange**, Yahoo | | 4 | Full resync | last resort | **Gmail has CONDSTORE but no QRESYNC; Exchange has neither.** `NOMODSEQ` is a mailbox property (RFC 7162 §3.1.2.2), so detection must be per mailbox. ### Notes - QRESYNC **never signals modseq expiry** — RFC 7162 §3.2.6 makes a too-old modseq a large `VANISHED`, not an error. `full_resync_required` is always client-derived (UIDVALIDITY mismatch, `NOMODSEQ`, HIGHESTMODSEQ regression). - Sync state is structured: `(uidvalidity, highestmodseq, known-uid-set, seq-match-sample)`. - Keep a periodic full UID-set reconciliation even on rung 1. - Generalize `folder_sync_state` to `collection_sync_state(account_id, collection_id, token)`. Depends on #387, #388. Testable against #384 (all three server profiles). Design: [ADR 0002](https://github.com/KofTwentyTwo/nuncio/blob/dev/docs/adr/0002-multi-engine-sync-model.md) · [review evidence](https://github.com/KofTwentyTwo/nuncio/blob/dev/docs/reviews/2026-08-10-multi-engine-design-review.md) (PR #385)

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: KofTwentyTwo/nuncio#390