feat(nunciod): bounded-concurrency sync dispatcher with per-account timeout #366

2026-07-31T22:56:56Z

KofTwentyTwo commented

2026-07-31 22:56:56 +00:00

(Migrated from github.com)

Closes #292 (WS-C4). No proto change.

Problem

Account syncs were launched independently from two places (the Mail.Sync RPC handler and the daemon's CoreCommand consumer), each running an account's sync to completion before the next. One slow/hung account head-of-line-blocked the rest, and nothing bounded concurrency or per-account duration.

What this does

Introduces nunciod::sync_dispatcher::SyncDispatcher — the single place every production account sync now funnels through:

Bounded concurrency: a tokio::sync::Semaphore caps simultaneous account syncs. Env-tunable via NUNCIO_SYNC_CONCURRENCY (default 4); excess requests queue on the permit.
Per-account in-flight coalescing: a second request for an account already syncing joins the in-flight run (via a per-account watch channel) instead of double-fetching, and observes the same outcome.
Per-account timeout: exactly one place bounds each sync via tokio::time::timeout. Env-tunable via NUNCIO_SYNC_TIMEOUT seconds (default 300). A timed-out sync is cancelled cleanly and surfaces a typed SyncDispatchError::Timeout without poisoning the dispatcher.
Shutdown integration (WS-C1): the dispatcher holds the shared ShutdownSignal and races each in-flight sync against it (biased select!, mirroring WS-C2's outbox executor). Once shutdown is requested it admits no new work and abandons in-flight syncs for the next start, so a hung account can't delay graceful shutdown.

The actual per-account work is injected behind an AccountSyncer trait; production wires ProductionAccountSyncer (delegates to sync::run_account_sync). main.rs builds one dispatcher and shares it between the gRPC Mail.Sync handler and the CoreCommand sync consumer, so both origins share one cap/coalescing-map/timeout. An all-accounts sync now fans every configured mail account out through the same per-account path via sync_all_configured (replacing the old serial run_all_accounts_sync), which is the head-of-line fix. The injected-test-backend Sync seam is unchanged (a single in-memory double, deliberately bypassing the dispatcher).

Tests (all deterministic, offline, mock backends)

New sync_dispatcher unit tests:

bounds_simultaneous_syncs_to_the_configured_cap — counting mock proves peak concurrency == cap.
coalesces_a_duplicate_in_flight_sync_for_the_same_account — underlying sync runs exactly once for concurrent same-account requests.
a_hung_account_does_not_block_other_accounts — head-of-line regression: account A blocks forever, account B still completes.
a_hung_account_times_out_without_poisoning_the_dispatcher — typed Timeout, then a later healthy sync still works.
surfaces_the_underlying_sync_error_typed.
rejects_new_work_once_shutdown_is_requested / shutdown_unblocks_an_in_flight_sync_promptly.
sync_all_configured_dispatches_every_mail_account_and_skips_dav.

Existing Mail.Sync gRPC E2E and the spine E2E stay green (client Sync still works end to end).

Gate (three commands, all green, 0 ignored)

cargo fmt --all -- --check — clean.
cargo clippy --all-targets --workspace -- -D warnings — clean.
cargo test --workspace — 542 passed; 0 failed; 0 ignored.

No unsafe/unwrap/expect/panic/todo in non-test code. SSH-signed commit. No proto/golden change.

Closes #292 (WS-C4). No proto change. ## Problem Account syncs were launched independently from two places (the `Mail.Sync` RPC handler and the daemon's `CoreCommand` consumer), each running an account's sync to completion before the next. One slow/hung account head-of-line-blocked the rest, and nothing bounded concurrency or per-account duration. ## What this does Introduces `nunciod::sync_dispatcher::SyncDispatcher` — the single place every production account sync now funnels through: - **Bounded concurrency:** a `tokio::sync::Semaphore` caps simultaneous account syncs. Env-tunable via `NUNCIO_SYNC_CONCURRENCY` (default **4**); excess requests queue on the permit. - **Per-account in-flight coalescing:** a second request for an account already syncing **joins** the in-flight run (via a per-account `watch` channel) instead of double-fetching, and observes the same outcome. - **Per-account timeout:** exactly one place bounds each sync via `tokio::time::timeout`. Env-tunable via `NUNCIO_SYNC_TIMEOUT` seconds (default **300**). A timed-out sync is cancelled cleanly and surfaces a typed `SyncDispatchError::Timeout` without poisoning the dispatcher. - **Shutdown integration (WS-C1):** the dispatcher holds the shared `ShutdownSignal` and races each in-flight sync against it (biased `select!`, mirroring WS-C2's outbox executor). Once shutdown is requested it admits no new work and abandons in-flight syncs for the next start, so a hung account can't delay graceful shutdown. The actual per-account work is injected behind an `AccountSyncer` trait; production wires `ProductionAccountSyncer` (delegates to `sync::run_account_sync`). `main.rs` builds **one** dispatcher and shares it between the gRPC `Mail.Sync` handler and the `CoreCommand` sync consumer, so both origins share one cap/coalescing-map/timeout. An all-accounts sync now fans every configured mail account out through the same per-account path via `sync_all_configured` (replacing the old serial `run_all_accounts_sync`), which is the head-of-line fix. The injected-test-backend `Sync` seam is unchanged (a single in-memory double, deliberately bypassing the dispatcher). ## Tests (all deterministic, offline, mock backends) New `sync_dispatcher` unit tests: - `bounds_simultaneous_syncs_to_the_configured_cap` — counting mock proves peak concurrency == cap. - `coalesces_a_duplicate_in_flight_sync_for_the_same_account` — underlying sync runs exactly once for concurrent same-account requests. - `a_hung_account_does_not_block_other_accounts` — **head-of-line regression:** account A blocks forever, account B still completes. - `a_hung_account_times_out_without_poisoning_the_dispatcher` — typed `Timeout`, then a later healthy sync still works. - `surfaces_the_underlying_sync_error_typed`. - `rejects_new_work_once_shutdown_is_requested` / `shutdown_unblocks_an_in_flight_sync_promptly`. - `sync_all_configured_dispatches_every_mail_account_and_skips_dav`. Existing `Mail.Sync` gRPC E2E and the spine E2E stay green (client Sync still works end to end). ## Gate (three commands, all green, 0 ignored) - `cargo fmt --all -- --check` — clean. - `cargo clippy --all-targets --workspace -- -D warnings` — clean. - `cargo test --workspace` — **542 passed; 0 failed; 0 ignored**. No `unsafe`/`unwrap`/`expect`/`panic`/`todo` in non-test code. SSH-signed commit. No proto/golden change.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: KofTwentyTwo/nuncio#366