jmap: implement Email/changes so JMAP accounts stop accumulating ghost messages #408

New Issue

2026-08-12T16:04:00Z

KofTwentyTwo commented

2026-08-12 16:04:00 +00:00

(Migrated from github.com)

Carried forward from the ADR 0002 work; noted as unfiled in the 2026-08-11 handoff and still true after #394 (PR #405).

CollectionSync::enumerate_changes returns {upserts, removals, present, next_token}, and the removals/present pair is the entire fix for ghost messages: mail deleted or moved by any other client (another daemon, a phone, webmail) is reconciled away locally.

The IMAP side implements this across all four rungs. JMAP does not. It reports removals: [] and present: None unconditionally, so nothing is ever reconciled away and every JMAP account accumulates ghosts indefinitely — the exact single-engine bug the enumeration work fixed for IMAP.

present: None is the honest answer right now: it means "this pass was incremental and cannot speak to absence." Fabricating a present-set to make the types line up would be far worse — the sync path deletes any stored placement absent from a Some(...) list, so a partial list deletes live mail. The current code is correct; it is just incomplete.

Work

Implement JMAP Email/changes (RFC 8621 §4.2) and map it onto FolderChanges:

created / updated → upserts
destroyed → removals as PlacementKeys
the returned newState → the scheme-tagged sync token
honour hasMoreChanges by looping to convergence
on a cannotCalculateChanges error, fall back to a full enumeration and set present: Some(...) from it

Only emit present: Some(...) where the pass genuinely enumerated the whole mailbox.

Related: #339 (JMAP hardcodes folder_id = "inbox" in parse_email_get_response, mislabelling non-inbox mailboxes) will interact with this — placements make the mislabelling more visible, since the folder is now part of a placement's primary key.

Carried forward from the ADR 0002 work; noted as unfiled in the 2026-08-11 handoff and still true after #394 (PR #405). `CollectionSync::enumerate_changes` returns `{upserts, removals, present, next_token}`, and the `removals`/`present` pair is the entire fix for ghost messages: mail deleted or moved by any other client (another daemon, a phone, webmail) is reconciled away locally. The IMAP side implements this across all four rungs. **JMAP does not.** It reports `removals: []` and `present: None` unconditionally, so nothing is ever reconciled away and **every JMAP account accumulates ghosts indefinitely** — the exact single-engine bug the enumeration work fixed for IMAP. `present: None` is the honest answer right now: it means "this pass was incremental and cannot speak to absence." Fabricating a present-set to make the types line up would be far worse — the sync path deletes any stored placement absent from a `Some(...)` list, so a partial list deletes live mail. The current code is correct; it is just incomplete. ### Work Implement JMAP `Email/changes` (RFC 8621 §4.2) and map it onto `FolderChanges`: - `created` / `updated` → `upserts` - `destroyed` → `removals` as `PlacementKey`s - the returned `newState` → the scheme-tagged sync token - honour `hasMoreChanges` by looping to convergence - on a `cannotCalculateChanges` error, fall back to a full enumeration and set `present: Some(...)` from it Only emit `present: Some(...)` where the pass genuinely enumerated the whole mailbox. Related: #339 (JMAP hardcodes `folder_id = "inbox"` in `parse_email_get_response`, mislabelling non-inbox mailboxes) will interact with this — placements make the mislabelling more visible, since the folder is now part of a placement's primary key.

KofTwentyTwo commented

2026-08-14 22:57:29 +00:00

(Migrated from github.com)

Implemented in #414 (b4292b0), merged to dev. Email/changes (RFC 8621 §4.2) now maps onto FolderChanges: created/updated to upserts, destroyed to removals, newState to the scheme-tagged checkpoint, looping on hasMoreChanges, and falling back to a full enumeration on cannotCalculateChanges — the only path that reports present: Some(...). An ordinary incremental pass still reports present: None, because it cannot speak to absence. Paging, removals, incremental present and the fallback branch were each mutation-verified.

Implemented in #414 (`b4292b0`), merged to `dev`. `Email/changes` (RFC 8621 §4.2) now maps onto `FolderChanges`: `created`/`updated` to upserts, `destroyed` to removals, `newState` to the scheme-tagged checkpoint, looping on `hasMoreChanges`, and falling back to a full enumeration on `cannotCalculateChanges` — the only path that reports `present: Some(...)`. An ordinary incremental pass still reports `present: None`, because it cannot speak to absence. Paging, removals, incremental `present` and the fallback branch were each mutation-verified.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: KofTwentyTwo/nuncio#408