feat: build the outbox remote-mutation executor (filters made live) #263

2026-07-31T15:33:41Z

KofTwentyTwo commented

2026-07-31 15:33:41 +00:00

(Migrated from github.com)

Closes the M6.5 headline completeness gap (#252): filter remote actions (MOVE/COPY/FLAG/UNFLAG/DELETE/FORWARD/CALL WEBHOOK) previously enqueued a PendingRemoteMutation to the outbox but never executed — the worker only bumped the retry counter and honestly refused to fake completion. This makes them genuinely execute, marking completed only on real success.

Non-proto: internal wiring + engine ops only. nuncio.proto untouched.

Layer 1 — engine remote-mutation ops (`nuncio-mail`)

Added one method to the MailBackend trait: apply_mutation(&self, spec: &RemoteMutationSpec), plus RemoteMutationKind (SetFlagged{value} / Move{to_folder} / Copy{to_folder} / Delete) and RemoteMutationSpec { message_id, folder_id, folder_checkpoint, kind }.

Why one apply_mutation(spec) over N discrete methods: every action shares the same addressing + UIDVALIDITY-guard concern; a single spec keeps that in one place and keeps the trait small. MARK READ/MARK UNREAD are applied to the local store inline and are deliberately not in the vocabulary; FORWARD/CALL WEBHOOK are not backend mutations, so they route through the sender/webhook seams instead.

IMAP (imap.rs): parses the UID from imap-uid-{uid}, SELECTs the folder, and verifies the mailbox's current UIDVALIDITY against the stored checkpoint's UIDVALIDITY ({uidvalidity}:{uidnext}) before issuing any mutating command — a mismatch (or a checkpoint that carries no UIDVALIDITY) returns an honest error and acts on nothing. Flags via UID STORE ±FLAGS (\Flagged); move prefers UID MOVE (RFC 6851) when the server advertises the MOVE capability, else falls back to UID COPY + UID STORE +FLAGS (\Deleted) + EXPUNGE; delete is UID STORE +FLAGS (\Deleted) + EXPUNGE; copy is UID COPY.
JMAP (jmap.rs): Email/set — keywords/$flagged patch for flags, mailboxIds replace for move / mailboxIds/{id} add for copy, destroy for delete — and confirms the id genuinely appears in updated/destroyed and not in notUpdated/notDestroyed.
MockMailBackend records every applied spec (applied_mutations()).
Tests: scripted tokio::io::duplex (IMAP) assert the exact protocol commands incl. the MOVE-vs-fallback split and the UIDVALIDITY-refusal; wiremock (JMAP) assert the Email/set bodies and that a server rejection surfaces as an error. Offline only.

Layer 2 — daemon outbox executor (`nunciod/src/outbox.rs`)

Replaced the no-op worker in main.rs. Per pending mutation: resolve the message via db.get_message (gone → fail permanently), parse MutationPayload, then dispatch by mutation_type:

MOVE/COPY/FLAG/UNFLAG/DELETE → Layer-1 apply_mutation on the account's backend, threading the source folder's stored folder_sync_state checkpoint for the UIDVALIDITY guard.
FORWARD → composes an OutboundMessage (From: = the account's configured address) and sends via the SMTP MessageSender.
CALL WEBHOOK → the SSRF-hardened WebhookDispatcher (signing key provisioned from the vault via a new WEBHOOK_SIGNING_KEY_ACCOUNT); 2xx/3xx = success, 4xx/5xx = retry, a blocked SSRF target = permanent fail.

Injectable RemoteExecutionEnv seam (mirrors the MessageSender/AccountConnectionTester override pattern): production ProductionExecutionEnv resolves real per-account engines from the store + keyring; the E2E injects mocks.

No fake completion: a mutation is completed only when the real op returns Ok. Transient failures keep it pending (retry_count bumped) until MAX_RETRIES, then failed; permanent failures fail immediately. The MARK READ/MARK UNREAD inline actions are never enqueued, so they are not double-handled.

How each action type is E2E-proven (`tests/outbox_executor_e2e_test.rs`, offline)

Each drives the real sync→filter→outbox→execute vertical (a matched rule enqueues via apply_filter_actions, then execute_pending_mutations drains it):

MOVE — backend records a Move spec carrying the source folder's UIDVALIDITY checkpoint; completed.
FLAG → SetFlagged{true}; DELETE → Delete; both recorded + completed.
FORWARD — MockMessageSender records the sent message (To: target, From: account address); completed.
CALL WEBHOOK — wiremock .expect(1) verifies the signed POST actually arrived; completed.
Failed op — a failing backend records nothing, is not completed, stays pending with a bumped retry count.
Vanished message — fails permanently, backend never touched.

Gate (all green, 0 ignored)

cargo fmt --all -- --check — clean
cargo check-all (clippy --all-targets --workspace -D warnings) — 0 warnings
cargo test-all — all pass, 0 failed, 0 ignored (nuncio-mail lib 65 + wiremock_jmap 7; nunciod outbox_executor_e2e 7; full workspace green)

No live network anywhere (scripted duplex / wiremock / mocks). Do not merge — for PO review.

Closes the M6.5 headline completeness gap (#252): filter remote actions (MOVE/COPY/FLAG/UNFLAG/DELETE/FORWARD/CALL WEBHOOK) previously enqueued a `PendingRemoteMutation` to the outbox but **never executed** — the worker only bumped the retry counter and honestly refused to fake completion. This makes them genuinely execute, marking `completed` only on real success. Non-proto: internal wiring + engine ops only. `nuncio.proto` untouched. ## Layer 1 — engine remote-mutation ops (`nuncio-mail`) Added one method to the `MailBackend` trait: `apply_mutation(&self, spec: &RemoteMutationSpec)`, plus `RemoteMutationKind` (`SetFlagged{value}` / `Move{to_folder}` / `Copy{to_folder}` / `Delete`) and `RemoteMutationSpec { message_id, folder_id, folder_checkpoint, kind }`. **Why one `apply_mutation(spec)` over N discrete methods:** every action shares the same addressing + UIDVALIDITY-guard concern; a single spec keeps that in one place and keeps the trait small. `MARK READ`/`MARK UNREAD` are applied to the local store inline and are deliberately *not* in the vocabulary; `FORWARD`/`CALL WEBHOOK` are not backend mutations, so they route through the sender/webhook seams instead. - **IMAP** (`imap.rs`): parses the UID from `imap-uid-{uid}`, `SELECT`s the folder, and **verifies the mailbox's current UIDVALIDITY against the stored checkpoint's UIDVALIDITY (`{uidvalidity}:{uidnext}`) before issuing any mutating command** — a mismatch (or a checkpoint that carries no UIDVALIDITY) returns an honest error and acts on nothing. Flags via `UID STORE ±FLAGS (\Flagged)`; **move prefers `UID MOVE` (RFC 6851) when the server advertises the `MOVE` capability, else falls back to `UID COPY` + `UID STORE +FLAGS (\Deleted)` + `EXPUNGE`**; delete is `UID STORE +FLAGS (\Deleted)` + `EXPUNGE`; copy is `UID COPY`. - **JMAP** (`jmap.rs`): `Email/set` — `keywords/$flagged` patch for flags, `mailboxIds` replace for move / `mailboxIds/{id}` add for copy, `destroy` for delete — and confirms the id genuinely appears in `updated`/`destroyed` and not in `notUpdated`/`notDestroyed`. - `MockMailBackend` records every applied spec (`applied_mutations()`). - Tests: scripted `tokio::io::duplex` (IMAP) assert the exact protocol commands incl. the MOVE-vs-fallback split and the UIDVALIDITY-refusal; wiremock (JMAP) assert the `Email/set` bodies and that a server rejection surfaces as an error. Offline only. ## Layer 2 — daemon outbox executor (`nunciod/src/outbox.rs`) Replaced the no-op worker in `main.rs`. Per pending mutation: resolve the message via `db.get_message` (gone → **fail permanently**), parse `MutationPayload`, then dispatch by `mutation_type`: - MOVE/COPY/FLAG/UNFLAG/DELETE → Layer-1 `apply_mutation` on the account's backend, threading the source folder's stored `folder_sync_state` checkpoint for the UIDVALIDITY guard. - FORWARD → composes an `OutboundMessage` (From: = the account's configured address) and sends via the SMTP `MessageSender`. - CALL WEBHOOK → the SSRF-hardened `WebhookDispatcher` (signing key provisioned from the vault via a new `WEBHOOK_SIGNING_KEY_ACCOUNT`); 2xx/3xx = success, 4xx/5xx = retry, a blocked SSRF target = permanent fail. Injectable `RemoteExecutionEnv` seam (mirrors the `MessageSender`/`AccountConnectionTester` override pattern): production `ProductionExecutionEnv` resolves real per-account engines from the store + keyring; the E2E injects mocks. **No fake completion:** a mutation is `completed` only when the real op returns `Ok`. Transient failures keep it `pending` (retry_count bumped) until `MAX_RETRIES`, then `failed`; permanent failures fail immediately. The `MARK READ`/`MARK UNREAD` inline actions are never enqueued, so they are not double-handled. ## How each action type is E2E-proven (`tests/outbox_executor_e2e_test.rs`, offline) Each drives the real `sync→filter→outbox→execute` vertical (a matched rule enqueues via `apply_filter_actions`, then `execute_pending_mutations` drains it): - **MOVE** — backend records a `Move` spec carrying the source folder's UIDVALIDITY checkpoint; completed. - **FLAG** → `SetFlagged{true}`; **DELETE** → `Delete`; both recorded + completed. - **FORWARD** — `MockMessageSender` records the sent message (To: target, From: account address); completed. - **CALL WEBHOOK** — wiremock `.expect(1)` verifies the signed POST actually arrived; completed. - **Failed op** — a failing backend records nothing, is **not** completed, stays `pending` with a bumped retry count. - **Vanished message** — fails permanently, backend never touched. ## Gate (all green, 0 ignored) - `cargo fmt --all -- --check` — clean - `cargo check-all` (clippy `--all-targets --workspace -D warnings`) — 0 warnings - `cargo test-all` — all pass, 0 failed, 0 ignored (nuncio-mail lib 65 + wiremock_jmap 7; nunciod outbox_executor_e2e 7; full workspace green) No live network anywhere (scripted duplex / wiremock / mocks). Do not merge — for PO review.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: KofTwentyTwo/nuncio#263

feat: build the outbox remote-mutation executor (filters made live) #263

Layer 1 — engine remote-mutation ops (nuncio-mail)

Layer 2 — daemon outbox executor (nunciod/src/outbox.rs)

How each action type is E2E-proven (tests/outbox_executor_e2e_test.rs, offline)

Gate (all green, 0 ignored)

Layer 1 — engine remote-mutation ops (`nuncio-mail`)

Layer 2 — daemon outbox executor (`nunciod/src/outbox.rs`)

How each action type is E2E-proven (`tests/outbox_executor_e2e_test.rs`, offline)