ImapEngine ignores since_state: every sync is a full historical re-fetch, not incremental #184

New Issue

2026-07-27T19:32:39Z

KofTwentyTwo commented

2026-07-27 19:32:39 +00:00

(Migrated from github.com)

What's wrong

MailBackend::sync_messages(folder_id, since_state: Option<&str>) is designed
for incremental sync: pass the checkpoint token from the previous sync,
get back only what changed since then. ImapEngine's implementation
(crates/nuncio-mail/src/imap.rs, sync_folder_messages/
sync_folder_messages_with_session) never uses since_state at all --
sync_messages takes _since_state: Option<&str> (explicitly discarded)
and always issues UID FETCH 1:*, fetching the full body of every message
in the folder regardless of what was already synced last time.

The returned "state" token is also not real: format!("imap-modseq-{}", emails.len()) -- a count of emails fetched, not an IMAP UID or MODSEQ. It
can't meaningfully be used as a checkpoint even if a caller tried.

Impact

Every single mail sync call re-downloads the entire historical mailbox
from scratch. Discovered while investigating #183 (a real account with a
59,771-message folder takes minutes per sync, every time, forever -- not
just on first sync).

Fix direction

Track the highest UID successfully synced per folder (persist it in the
store, keyed by account+folder) and pass it as since_state; have
sync_folder_messages_with_session use UID FETCH <highest_uid+1>:* when a
checkpoint is present, falling back to 1:* only for a genuinely first
sync. Longer term, IMAP CONDSTORE/QRESYNC (if the server supports it)
would give a real MODSEQ-based incremental sync including flag changes on
already-fetched messages, not just new mail.

## What's wrong `MailBackend::sync_messages(folder_id, since_state: Option<&str>)` is designed for incremental sync: pass the checkpoint token from the previous sync, get back only what changed since then. `ImapEngine`'s implementation (`crates/nuncio-mail/src/imap.rs`, `sync_folder_messages`/ `sync_folder_messages_with_session`) never uses `since_state` at all -- `sync_messages` takes `_since_state: Option<&str>` (explicitly discarded) and always issues `UID FETCH 1:*`, fetching the full body of every message in the folder regardless of what was already synced last time. The returned "state" token is also not real: `format!("imap-modseq-{}", emails.len())` -- a count of emails fetched, not an IMAP UID or MODSEQ. It can't meaningfully be used as a checkpoint even if a caller tried. ## Impact Every single `mail sync` call re-downloads the entire historical mailbox from scratch. Discovered while investigating #183 (a real account with a 59,771-message folder takes minutes per sync, every time, forever -- not just on first sync). ## Fix direction Track the highest UID successfully synced per folder (persist it in the store, keyed by account+folder) and pass it as `since_state`; have `sync_folder_messages_with_session` use `UID FETCH <highest_uid+1>:*` when a checkpoint is present, falling back to `1:*` only for a genuinely first sync. Longer term, IMAP `CONDSTORE`/`QRESYNC` (if the server supports it) would give a real MODSEQ-based incremental sync including flag changes on already-fetched messages, not just new mail.

KofTwentyTwo commented

2026-07-28 14:06:14 +00:00

(Migrated from github.com)

Superseded by #195 (M5) in the roadmap restructuring (milestones M1–M7). The replacement story carries the full, self-contained spec for a remote implementer. See docs/ROADMAP.md, docs/STORY-WORKFLOW.md, docs/HANDOFF.md.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: KofTwentyTwo/nuncio#184