Sync could only ever add. It resolved a UID boundary, fetched forward from it,
and stored what came back; nothing ever asked what stopped being there. A
message moved or deleted by any other client -- another daemon, a phone,
webmail -- stayed in the local store forever, and every account accumulated
ghosts. This is the defect, and it affects a single engine just as much as ten.
A folder pass now reports what changed, what vanished, and honestly whether it
was able to observe absence at all. The last part is why `present` is an
Option: None means the pass was incremental and cannot speak to absence, while
Some(empty) means the folder is genuinely empty. Conflating those deletes a
mailbox.
Four rungs, chosen per mailbox rather than per account, because NOMODSEQ is a
mailbox property (RFC 7162 3.1.2.2) and one folder can be unable to answer
mod-sequence queries on a server that otherwise can. QRESYNC has the server
state removals outright. CONDSTORE narrows the fetch but never volunteers them,
so absence comes from enumerating what remains -- this is Gmail's rung, which
advertises CONDSTORE and not QRESYNC, and a two-rung ladder would have pushed
the largest provider onto full sweeps against a published daily byte cap.
Neither extension leaves a UID boundary for arrivals plus an enumeration for
removals; that is Exchange. No usable checkpoint fetches everything.
The enumeration passes ask for UID only -- no bodies, no flags -- so detecting
removals costs one short line per message rather than a download. A mailbox
that can supply a mod-sequence gets a mod-sequence checkpoint even after a full
pass, so the next sync starts on a higher rung than this one did.
A UIDVALIDITY change is not a downgrade to a lesser rung, it is a full pass:
UIDs restart, so every stored boundary names a different message. The dangerous
reading would be to enumerate the new UID space, find none of the stored ids in
it, and conclude the folder was emptied.
The raw collector also gains a bounded read. A server that accepts a command
and then stops answering must surface as a stall rather than hanging the sync,
which is the same failure the typed fetch path already bounds.
Refs #390
Two silent-failure paths, both cheap to close and both load-bearing for the
sync work that follows.
The schema is built entirely from CREATE TABLE IF NOT EXISTS, so an older
binary opens a newer database successfully and writes the old model into it.
Nothing detects that; the file ends up holding two mutually invisible truths.
A user_version generation now gates opening: newer than this build is a
fail-closed error, and older rebuilds the cached message store.
Rebuilding rather than migrating is the pre-release decision recorded in ADR
0002. Message identity cannot be recomputed for rows already stored, so the
alternative is keeping them under the old model forever -- a store where half
the mail is invisible to every guarantee the new model provides. Accounts,
credentials and rules are kept; cached mail, checkpoints and the execution
ledger are rebuilt from the server, which is authoritative regardless. The
ledger goes with them because its hash chain is keyed on message ids that would
no longer resolve, and a chain pointing at absent rows proves nothing.
Separately, folder checkpoints now carry a scheme tag. The stored value was
"{uidvalidity}:{uidnext}" and the QRESYNC checkpoint that replaces it is
"{uidvalidity}:{highestmodseq}" -- identical shape, different meaning. An
upgraded engine reading one as the other hands the server a UIDNEXT where it
expects a MODSEQ; UIDNEXT is normally the larger number, so the server answers
"nothing changed since" and every message below that point is never enumerated
again. That is silent, permanent mail loss with no error anywhere. Any token
whose scheme this build does not recognise, including an untagged one, now
forces a full re-fetch: over-fetching is recoverable, misreading is not.
Refs #387
james.maes