Compare commits

..
23 Commits
Author SHA1 Message Date
dependabot[bot] 157c7c27b0 deps(cargo): bump rustls from 0.23.42 to 0.23.43 (#241)
Bumps [rustls](https://github.com/rustls/rustls) from 0.23.42 to 0.23.43.
- [Release notes](https://github.com/rustls/rustls/releases)
- [Changelog](https://github.com/rustls/rustls/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rustls/rustls/compare/v/0.23.42...v/0.23.43)

---
updated-dependencies:
- dependency-name: rustls
  dependency-version: 0.23.43
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-31 10:53:15 -05:00
james.maes ebc64a1272 Merge pull request #235 from KofTwentyTwo/dev
Promote to main: M5 (Mail lifecycle + incremental sync)
2026-07-29 22:00:46 -05:00
james.maesandJames Maes c8446a5800 docs: mark M5 complete, set M6 as next (#234)
* docs: mark M5 complete (feature surface done), set M6 as next

* docs: correct HANDOFF service count to eight

---------

Co-authored-by: James Maes <jmaes@dmdbrands.com>
2026-07-29 21:51:43 -05:00
james.maesandJames Maes 4b73d11754 feat(mail): select an explicit account for SendMessage (#232)
SendMessage previously always sent from the first configured account
with no way to choose, so it silently used the wrong identity the
moment a second account existed. Add an optional account_id field to
SendMessageRequest; when set, resolve that specific account and fail
with a new AccountNotFound error if it does not exist, rather than
falling back to a different account. Omitting account_id preserves
the prior first-account behavior. Thread the selector through
ComposeRequest, resolve_sending_account, the CLI's mail send --account
flag, and regenerate the nuncio.v1 descriptor golden.

Refs #196, #170

Co-authored-by: James Maes <jmaes@dmdbrands.com>
2026-07-29 21:45:16 -05:00
james.maesandJames Maes d7dce525e2 feat(sync): incremental IMAP sync with progress + per-item FETCH timeout (#231)
* feat(store): add per-folder sync-state checkpoint table + CRUD

Adds folder_sync_state(account_id, folder_id, state) with
get/save methods and a round-trip test, giving inbound sync a place
to persist a per-mailbox resumption token between runs.

Refs #195 #184

* feat(mail): honor since_state for incremental IMAP fetch + bound FETCH stalls

ImapEngine now derives a real UIDNEXT-based checkpoint after selecting a
folder and narrows the UID FETCH to {uidnext}:* when a checkpoint is
supplied, instead of always re-fetching 1:* and returning a state
fabricated from emails.len(). An unparseable checkpoint falls back to a
full fetch but is surfaced distinctly. Each per-item FETCH read is bounded
by a named timeout, surfacing MailError::FetchStalled instead of hanging.
MockMailBackend captures the since_state arg and simulates an incremental
(empty) second fetch. Session-level duplex tests assert the narrowed FETCH
range and the bounded stall.

Refs #195 #184 #183

* feat(sync): persist and thread the folder checkpoint, publish sync progress

fetch_and_persist now reads the folder's stored checkpoint, passes it to
the backend for a narrowed fetch, and persists the returned checkpoint only
after that folder's messages land -- so the second sync of an account/folder
is genuinely incremental. Adds CoreEvent::SyncProgress (per folder) as an
internal signal; map_core_event drops it from the gRPC stream since it has
no wire-contract case (proto unchanged).

Refs #195 #184

* fix(mail): track UIDVALIDITY in sync checkpoint to prevent mail loss

A checkpoint stored only UIDNEXT, so a mailbox renumber (UIDVALIDITY
change, UIDs restart low) would drive {old_uidnext}:* against the new low
UIDs -- per RFC 3501 n:* returns only the single highest message and
silently skips the rest, permanently dropping mail. The checkpoint now
encodes {uidvalidity}:{uidnext}; the incremental narrowing applies only
when the stored UIDVALIDITY still matches the mailbox's current one. A
mismatch (renumber), a missing current UIDVALIDITY, or a legacy
bare-number/unparseable checkpoint falls back to a safe full 1:* re-fetch
with a debug-visible warn, then rewrites the checkpoint fresh.

Refs #195 #184

---------

Co-authored-by: James Maes <jmaes@dmdbrands.com>
2026-07-29 21:33:55 -05:00
james.maesandJames Maes 0b8307a268 M5: Account lifecycle over gRPC (edit/delete/test) + persist TLS mode (#230)
* feat(store): persist per-account IMAP/SMTP TLS modes; add account CRUD

Adds a backfill-safe imap_tls_mode/smtp_tls_mode column migration,
binds+reads both modes in save_account/list_accounts (previously
validated then silently discarded and hardcoded back to ImplicitTls),
and adds get_account/delete_account. Regression test proves the modes
survive a save/list/get round-trip.

Refs #194 #182

* feat(mail): branch IMAP/SMTP transport on TlsMode; add connection probes

Threads TlsMode through ImapEngine/ImapDualSocketManager and
SmtpTransportEngine so implicit TLS, STARTTLS, and plain are genuine
transport paths (a unified MailStream plus a real IMAP STARTTLS
negotiation), not validate-and-discard. Adds bounded connect timeout
and per-engine probe_connection backing TestAccountConnection, with
mock/loopback-only setup tests for each mode.

Refs #194 #182

* feat(accounts): edit/delete/test account lifecycle over gRPC

Implements Accounts UpdateAccount/RemoveAccount/TestAccountConnection in
the daemon (update mirrors add_account's keyring rollback discipline;
test probes real IMAP+SMTP via an injectable tester, honoring persisted
TLS modes), wires the configured TLS modes into build_mail_backend/
build_smtp_sender, and replaces the fabricated CLI account
show/edit/delete/test handlers with real thin gRPC clients. Adds an
offline full-daemon lifecycle E2E and unauth-rejection coverage for all
three new RPCs.

Refs #194 #182

* fix(accounts): restore prior credential on update rollback

A password rotation that fails to persist previously deleted the
just-written secret, destroying the working credential the key already
held and orphaning the still-present account row. Snapshot the prior
secret first and restore it on save failure (falling back to delete
only when none existed). Regression tests force save_account to fail
and assert the original secret survives.

Refs #194 #182

* docs(accounts): scope TestAccountConnection SMTP claim to reachability

The SMTP leg only dials and negotiates TLS/STARTTLS; it performs no AUTH
(lettre exposes no clean auth-only probe on AsyncSmtpTransport), while
IMAP genuinely logs in. Clarify the proto doc for smtp_ok/smtp_error and
the CLI output wording so nothing implies SMTP authentication was
verified. Doc-only proto change; golden unchanged.

Refs #194 #182

---------

Co-authored-by: James Maes <jmaes@dmdbrands.com>
2026-07-29 20:57:55 -05:00
james.maesandJames Maes 43b95e7614 fix(accounts): roll back orphaned keyring secret on add_account failure (#229)
If set_secret succeeds but the subsequent save_account fails, the
credential was left in the keyring with no account row referencing it.
add_account now deletes the just-written secret before returning,
appending any rollback-delete failure to the original error message
without masking it.

Refs #197, #169

Co-authored-by: James Maes <jmaes@dmdbrands.com>
2026-07-29 20:12:17 -05:00
james.maes 5a047c1d0b Merge pull request #228 from KofTwentyTwo/dev
Promote to main: M3 (JMAP) + M4 (Filters made live)
2026-07-29 20:09:31 -05:00
james.maesandJames Maes ed71f51b22 docs: mark M1-M4 complete, set M5 as next milestone (#227)
* docs: mark M1-M4 complete, set M5 as next milestone

Phase 3 feature completeness (Calendar, Contacts, JMAP, live Filters)
is done and merged to dev; update ROADMAP.md, the rendered roadmap
page, and HANDOFF.md to reflect it and point at M5 next.

* docs: correct service count to eight (Calendar+Contacts added)

---------

Co-authored-by: James Maes <jmaes@dmdbrands.com>
2026-07-29 20:02:59 -05:00
james.maesandJames Maes 2e9c43a32c feat(filter): add streaming Filters.Triage RPC for retroactive rescan (#226)
Adds a server-streaming Triage RPC that walks the whole message store
in bounded keyset chunks, evaluates the live FilterEngine against each
message via the shared apply_filter_actions helper, and streams real
cumulative progress -- so the active rule set can be applied to mail
synced before those rules existed. Wires the CLI's first streaming-RPC
consumer (filter triage) and regenerates the proto contract golden.

Refs #193

Co-authored-by: James Maes <jmaes@dmdbrands.com>
2026-07-29 19:54:30 -05:00
james.maesandJames Maes 6cb8ae5753 feat(filter): move edit/export/import/logs onto Filters gRPC service (#224)
Adds UpdateRule/ExportRules/ImportRules/GetExecutionLogs to the
authenticated Filters gRPC service and rewires the CLI's
filter edit/export/import/logs subcommands onto it, replacing their
throwaway ephemeral local-db reads with the daemon's real, persistent
store and live FilterEngine. ImportRules now reports per-line parse
failures instead of silently dropping them.

Regenerates the nuncio-proto contract-stability golden for the
additive proto changes.

Refs #192

Co-authored-by: James Maes <jmaes@dmdbrands.com>
2026-07-29 19:37:19 -05:00
james.maesandJames Maes 2186670ca8 fix(filter): correct account-field evaluation and reject header conditions (#223)
The account condition field lumped in with folder and compared against
email.folder_id, so WHERE account = '...' would false-positive on any
message sharing a folder with the target account. Split the arm so it
reads email.account_id instead.

header[...] conditions silently evaluated to false forever because Email
has no parsed-headers field to match against. Reject header conditions
at validation time with an honest FieldTypeMismatch instead of accepting
a rule that can never fire; the runtime fallback is now documented as an
unreachable defensive branch.

Refs #191

Co-authored-by: James Maes <jmaes@dmdbrands.com>
2026-07-29 19:19:37 -05:00
james.maesandJames Maes b43d7bbb5f M4: Wire FilterEngine into live sync so matched rules actually fire (#222)
* feat(filters): wire FilterEngine into live inbound sync

Every synced message is now evaluated against the daemon's live
FilterEngine after it is persisted. MARK READ/UNREAD apply immediately
via set_message_read; MOVE/COPY/FLAG/UNFLAG/DELETE/FORWARD/CALL WEBHOOK
enqueue a real PendingRemoteMutation through the existing outbox for
the background worker to drain. Every match appends a
FilterExecutionLog entry, so a rule match now produces a genuine,
persisted, observable side effect instead of nothing.

The evaluate-and-route logic lives in a standalone apply_filter_actions
helper (not inlined into fetch_and_persist) so other sync paths can
reuse the exact same routing instead of re-deriving it.

* fix(filters): fire filter actions exactly once per new message

Sync is not incremental yet, so every re-sync re-fetches and
re-persists messages the backend still reports. Applying filter
actions unconditionally on every persisted message meant a rule kept
re-firing on every sync cycle for a message that arrived once,
enqueueing a fresh outbox mutation and execution log entry each time.

Guard fetch_and_persist so it only routes a message through
apply_filter_actions when the message did not already exist in the
store before this sync pass (DatabaseError::is_not_found, checked
before save_email's INSERT OR REPLACE erases the distinction). Also
makes the immediate-action match in apply_filter_actions exhaustive
over the remote action variants instead of a wildcard, so a future
immediate-style action can't silently fall through to remote routing.

---------

Co-authored-by: James Maes <jmaes@dmdbrands.com>
2026-07-29 19:11:37 -05:00
james.maesandJames Maes 12c8724644 feat(mail): wire a real JMAP client into the mail sync path (#221)
Replace JmapEngine's canned session/mailbox/message data with real
RFC 8620/8621 HTTP session discovery and Mailbox/get + Email/query +
Email/get JSON-RPC calls over reqwest, gated behind with_credentials
so uncredentialed callers keep the deterministic no-network fallback.
Fix the camelCase wire bug on JmapEmail (receivedAt/isUnread/
bodySnippet were silently defaulting) and add a regression test.
Wire nunciod::sync::build_mail_backend to construct the JMAP engine
with the account's real host/address/keyring password, mirroring the
IMAP arm, and rewrite the sync.rs tests that previously relied on
JmapEngine's canned data to run against wiremock servers instead.

Co-authored-by: James Maes <jmaes@dmdbrands.com>
2026-07-29 18:40:19 -05:00
james.maes 82e8871cff Merge pull request #220 from KofTwentyTwo/dev
Promote dev -> main: M1 Calendar + M2 Contacts verticals
2026-07-29 18:33:54 -05:00
james.maesandJames Maes 174c5457f4 feat(contacts): add Contacts gRPC vertical (proto, daemon, CLI, E2E) (#218)
Publish the M2-S1 contacts engine over the daemon's authenticated gRPC
API, mirroring the Calendar vertical: add nuncio.v1.Contacts (Sync,
ListContacts, GetContact, CreateContact) plus Contact/ContactEmail/
ContactPhone messages to the proto contract and regenerate the golden
descriptor.

Wire ContactsEngineOverrides + ContactsGrpcService into nunciod,
mounted behind its own BearerAuthInterceptor exactly like every other
service. ListContacts/GetContact always read genuinely persisted data
from DatabaseEngine; Sync fetches from an injected ContactsBackend
(MockContactsBackend in tests) and persists via
contacts_sync::sync_with_backend, mirroring calendar_sync.rs; with no
backend injected, production Sync returns an honest error since
AccountConfig carries no per-account CardDAV configuration yet.

Add connect_contacts/AuthenticatedContactsClient to nuncio-proto's
shared client helpers. Replace the CLI's Commands::Contact handlers
(previously a disposable in-memory ContactsDatabase.in_memory() per
invocation) with real gRPC calls against the daemon, so a contact
added via `contact add` persists and is visible to a later, separate
CLI invocation.

Deviations from the issue's literal proto sketch, flagged for review:
- Added CreateContact (not in the issue's message list): the
  acceptance criteria requires `contact add` to persist a locally-
  authored contact across CLI invocations, but the daemon has no
  per-account CardDAV credential to write back to a real server. This
  RPC only ever writes to the daemon's own local store -- never a
  fabricated CardDAV write-back -- which is the same honest-surface
  spirit as Calendar's "no create/delete" note (that note is about not
  pretending to write to the remote server, not about never
  persisting locally-authored data at all).
- ListContacts/CreateContact take a required account_id (the issue
  sketched a bare ListContactsRequest{}), because the real
  DatabaseEngine::list_contacts/save_contact are account-scoped;
  ContactSubcommand::List/Search/Add/Sync now all take --account to
  match.
- Contact search has no server-side RPC; `contact search` fetches the
  full ListContacts result and filters client-side by display name,
  organization, and email.
- Contact's created_at/updated_at/last_interacted_at are omitted from
  the proto message -- no client use case identified for them yet.

Full-daemon offline E2E (contacts_e2e_test.rs) proves Sync -> List/Get
over the authenticated API, CreateContact's persistence surviving a
second, independent client connection, and an unauthenticated call
rejected with Code::Unauthenticated.

Co-authored-by: James Maes <jmaes@dmdbrands.com>
2026-07-29 18:23:32 -05:00
james.maesandJames Maes 41665e85d5 feat(contacts): build a real CardDAV engine, replacing the fabricated client (#217)
Replace CardDavClient::fetch_remote_vcards's hardcoded "Google CardDAV
Sync Contact" return with a real reqwest REPORT/addressbook-query
client that parses a genuine multistatus response. Add a real vCard
(RFC 6350) parser -- the crate previously only had a generator -- and
prove round-trip fidelity through Contact::to_vcard(). Add the
ContactsBackend trait and MockContactsBackend, mirroring
nuncio_cal::CalendarBackend/MockCalendarBackend. Distinguish transport
failures from parse failures via CardDavError::TransportFailed/
ParseFailed rather than ever silently succeeding with fabricated data.

Give nuncio-store real, tested save_contact/list_contacts/get_contact
CRUD backed by a genuine SQLite table + trigram FTS mirror, so contacts
persist through the daemon's own store instead of nuncio-contacts's
disconnected ContactsDatabase pool. This takes nuncio-store as a
narrow, deliberate dependency on nuncio-contacts::Contact.

Opt nuncio-contacts into the workspace lint gate ([lints] workspace,
#![forbid(unsafe_code)]) for parity with every other crate.

Wiremock-only, offline: no live network calls in tests.

Co-authored-by: James Maes <jmaes@dmdbrands.com>
2026-07-29 17:53:55 -05:00
james.maesandJames Maes 9496d63eaa M1: Wire RRULE recurrence expansion into calendar event queries (#215)
* feat(calendar): expand RRULE recurrence in ListEvents and filter by calendar_id

Wire the previously-dead nuncio_cal::RecurrenceEngine into
CalendarGrpcService::list_events: non-recurring events pass through
unchanged, while recurring masters are expanded into real occurrences
for the requested window via a new store method
(list_recurring_calendar_events, no time filter since a master's own
timestamps can precede the window while its occurrences fall inside
it) and a pure expand_events_for_window merge helper that keeps
recurring rows sourced only from expansion, never double-counting a
master that also overlaps the window in the raw query.

Fix expand_occurrences to stop trusting an empty match as "return the
raw master": that fabricated an out-of-window result whenever a
window fell entirely outside a bounded (UNTIL/COUNT) series, and could
hang scanning an unbounded series for a match that would never come.
skip_while/take_while over the RRuleSet's monotonic iterator bounds
the walk and returns a genuinely empty result instead.

Also apply the calendar_id filter list_events accepted but ignored,
closing the gap between it and the mock backend's own filtering.

Refs #186

* docs(nunciod): reword stale comment referencing removed rrule fallback

---------

Co-authored-by: James Maes <jmaes@dmdbrands.com>
2026-07-29 17:35:44 -05:00
james.maesandJames Maes f926d09839 feat(calendar): add Calendar gRPC vertical (proto, daemon, CLI, E2E) (#214)
Wire the orphaned nuncio-cal engine into nunciod as a real, authenticated
gRPC vertical mirroring Mail: add service Calendar (Sync/ListEvents/GetEvent)
to nuncio.proto with a regenerated descriptor golden, connect_calendar in
nuncio-proto's client, calendar_sync::sync_with_backend plus
CalendarEngineOverrides/CalendarGrpcService mounted behind its own
BearerAuthInterceptor in nunciod::grpc, and real `cal sync`/`cal list` gRPC
clients in nuncio-cli replacing the fabricated CLI output. ListEvents/GetEvent
always read genuinely persisted data; production Sync returns an honest error
since there is no persisted per-account CalDAV configuration yet, and only an
injected MockCalendarBackend exercises a working Sync. Adds unit, daemon
integration, and offline full-daemon E2E tests, including an
unauthenticated-call-rejected assertion for the new service.

Refs #185

Co-authored-by: James Maes <jmaes@dmdbrands.com>
2026-07-29 17:19:15 -05:00
James Maes 5b4365a28b docs: correct stale CI-paused note; ignore stray trivy scan logs 2026-07-29 16:52:37 -05:00
james.maes fa9234180c Merge pull request #212 from KofTwentyTwo/dev
fix(build): remove broken cargo verify alias; ci: gate rc branch
2026-07-29 11:40:23 -05:00
james.maes 8de8eae442 Merge pull request #211 from KofTwentyTwo/dev
Promote dev → main: green CI + workflow/security cleanup
2026-07-29 11:16:52 -05:00
james.maes 4cf51f4646 Merge pull request #166 from KofTwentyTwo/dev
Promote rebuilt engine-first foundation (dev -> main)
2026-07-29 10:40:40 -05:00
47 changed files with 10210 additions and 651 deletions

No files matched your search

+1
View File
@@ -35,3 +35,4 @@ excalidraw.log
_reference/**/gen/
**/node_modules/
**/dist/
trivy_err*.log
+1 -1
View File
@@ -86,7 +86,7 @@ cargo run -p nuncio-cli -- system status # drive it via the CLI (daemon
> **Gate status:** the workspace is green under the pinned toolchain
> (`rust-toolchain.toml` = 1.97.1), so the local pre-commit hook runs the exact
> `fmt`/`clippy --all-targets`/tests that CI does — "green locally" is CI-equivalent.
> CI itself is paused until the GitHub Actions minutes reset (releases are tag-only).
> CI runs `fmt`/`clippy --all-targets`/tests on every PR and they are required to merge to `main`/`rc`; releases are tag-only.
> Coverage is informational, not a 100% gate.
## RustRover / IDE
Generated
+6 -3
View File
@@ -2478,7 +2478,6 @@ dependencies = [
"clap",
"hex",
"nuncio-cal",
"nuncio-contacts",
"nuncio-core",
"nuncio-filter",
"nuncio-mail",
@@ -2513,6 +2512,7 @@ dependencies = [
"thiserror 1.0.69",
"tokio",
"tracing",
"wiremock",
]
[[package]]
@@ -2611,6 +2611,7 @@ dependencies = [
"hex",
"hmac",
"keyring",
"nuncio-contacts",
"nuncio-core",
"nuncio-filter",
"serde",
@@ -2634,6 +2635,7 @@ dependencies = [
"hex",
"nuncio-cal",
"nuncio-cli",
"nuncio-contacts",
"nuncio-core",
"nuncio-filter",
"nuncio-mail",
@@ -2650,6 +2652,7 @@ dependencies = [
"tonic",
"tracing",
"tracing-subscriber",
"wiremock",
]
[[package]]
@@ -3581,9 +3584,9 @@ dependencies = [
[[package]]
name = "rustls"
version = "0.23.42"
version = "0.23.43"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3c54fcab019b409d04215d3a17cb438fd7fbf192ee61461f20f4fe18704bc138"
checksum = "0283386ce02abc0151e1761d08802dfe86c173b0b494af5cbc086574e453da06"
dependencies = [
"aws-lc-rs",
"log",
+43 -7
View File
@@ -47,9 +47,15 @@ impl RecurrenceEngine {
let duration = event.end_time - event.start_time;
// `RRuleSet`'s iterator yields occurrences in strictly increasing date order, so once
// an occurrence passes `window_end_dt` every later one will too -- `take_while` lets
// us stop walking the (possibly unbounded, no-UNTIL/COUNT) series there instead of
// scanning forever for a match that will never come once the series has moved past
// the window. `skip_while` similarly avoids materializing pre-window occurrences.
let results = rrule_set
.into_iter()
.filter(|dt| *dt >= window_start_dt && *dt <= window_end_dt)
.skip_while(|dt| *dt < window_start_dt)
.take_while(|dt| *dt <= window_end_dt)
.take(Self::MAX_EXPANSION_COUNT as usize);
let mut occurrences = Vec::new();
@@ -62,12 +68,11 @@ impl RecurrenceEngine {
occurrences.push(instance);
}
if occurrences.is_empty() {
// Return original if no occurrences fall strictly inside window
Ok(vec![event.clone()])
} else {
Ok(occurrences)
}
// No fallback to the raw master here: an empty result genuinely means no occurrence
// of this recurring event falls inside the requested window (e.g. the window is past
// an `UNTIL`/`COUNT` bound), and returning the master's own out-of-window timestamps
// instead would fabricate a result the caller never asked for.
Ok(occurrences)
}
}
@@ -110,6 +115,37 @@ mod tests {
assert!(results[0].id.contains("evt-weekly_occ_"));
}
#[test]
fn recurring_event_with_no_occurrences_in_window_returns_empty() {
let event = sample_recurring_event();
// Window is entirely before the series' own DTSTART -- no occurrence can fall
// inside it, so this must return no events at all, not the out-of-window master.
let results = RecurrenceEngine::expand_occurrences(&event, 1_600_000_000, 1_600_003_600)
.expect("expansion succeeds");
assert!(results.is_empty());
}
#[test]
fn until_bound_stops_occurrences_past_the_bound() {
let mut event = sample_recurring_event();
// Weekly series ends after its third occurrence (2024-01-15).
event.rrule = Some("FREQ=WEEKLY;INTERVAL=1;UNTIL=20240115T000000Z".to_string());
// Querying a window entirely after the UNTIL bound must produce zero occurrences.
let past_bound = RecurrenceEngine::expand_occurrences(&event, 1_707_350_400, 1_709_251_200)
.expect("expansion succeeds");
assert!(past_bound.is_empty());
// Querying a window spanning the bound only returns occurrences up to and
// including it, never past it.
let spanning_bound = RecurrenceEngine::expand_occurrences(&event, 1704067200, 1706745600)
.expect("expansion succeeds");
assert_eq!(spanning_bound.len(), 3);
for occurrence in &spanning_bound {
assert!(occurrence.start_time <= 1_705_276_800); // 2024-01-15T00:00:00Z
}
}
#[test]
fn invalid_rrule_returns_parse_error() {
let mut event = sample_recurring_event();
-1
View File
@@ -15,7 +15,6 @@ nuncio-mail = { path = "../nuncio-mail" }
nuncio-cal = { path = "../nuncio-cal" }
nuncio-store = { path = "../nuncio-store" }
nuncio-filter = { path = "../nuncio-filter" }
nuncio-contacts = { path = "../nuncio-contacts" }
nuncio-proto = { path = "../nuncio-proto" }
tokio.workspace = true
+97 -6
View File
@@ -135,19 +135,38 @@ pub enum Commands {
}
/// Contact subcommands (`nuncio contact <verb>`).
///
/// Every verb is scoped to an `--account`, mirroring `CalSubcommand`: the
/// daemon's real contact store (`DatabaseEngine::list_contacts`/
/// `get_contact`/`save_contact`) is itself account-scoped, so there is no
/// honest way to list/search/add a contact without naming which account's
/// address book it belongs to.
#[derive(Subcommand, Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "kebab-case")]
pub enum ContactSubcommand {
/// List contacts from local address book.
List,
/// List contacts in an account's address book.
List {
/// Account identifier owning the address book.
#[arg(short, long, help = "Account identifier")]
account: String,
},
/// Search contacts by name, email, or organization.
///
/// There is no server-side search RPC in the `Contacts` gRPC surface;
/// this filters the account's full `ListContacts` result client-side.
Search {
/// Account identifier owning the address book.
#[arg(short, long, help = "Account identifier")]
account: String,
/// Search query string.
#[arg(short, long, help = "Search query string")]
query: String,
},
/// Add a new contact entry to address book.
/// Add a new contact entry to an account's address book.
Add {
/// Account identifier owning the address book.
#[arg(short, long, help = "Account identifier")]
account: String,
/// Contact display name.
#[arg(short, long, help = "Display name")]
name: String,
@@ -158,6 +177,13 @@ pub enum ContactSubcommand {
#[arg(short, long, help = "Organization")]
org: Option<String>,
},
/// Synchronize an account's local contact cache with its remote CardDAV
/// server.
Sync {
/// Account identifier owning the address book.
#[arg(short, long, help = "Account identifier")]
account: String,
},
}
/// Software update subcommands (`nuncio update <verb>`).
@@ -232,6 +258,12 @@ pub enum AccountSubcommand {
/// Updated SMTP server port.
#[arg(long, help = "Updated SMTP server port")]
smtp_port: Option<u16>,
/// Updated IMAP transport mode (implicit_tls, start_tls, plain).
#[arg(long, help = "Updated IMAP transport mode")]
imap_mode: Option<String>,
/// Updated SMTP transport mode (implicit_tls, start_tls, plain).
#[arg(long, help = "Updated SMTP transport mode")]
smtp_mode: Option<String>,
},
/// Remove a configured account profile.
Delete {
@@ -276,6 +308,10 @@ pub enum MailSubcommand {
/// Message body text content.
#[arg(short, long, help = "Message body text content")]
body: String,
/// Identifier of the account to send from (defaults to the first
/// configured account when omitted).
#[arg(short, long, help = "Identifier of the account to send from")]
account: Option<String>,
},
/// Full-text search across all cached email messages.
Search {
@@ -335,13 +371,56 @@ pub enum FolderSubcommand {
}
/// Calendar subcommands (`nuncio cal <verb>`).
///
/// `account`/`calendar` currently have no persisted CalDAV account/collection
/// concept to look up defaults from (see `nunciod::calendar_sync`'s doc
/// comment), so both are plain flags rather than resolved from configured
/// account state; `start`/`end` default to a window spanning "everything"
/// (`0` .. `i64::MAX` unix seconds) so `list`/`sync` are usable with no flags
/// at all once an account/calendar id is supplied.
#[derive(Subcommand, Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "kebab-case")]
pub enum CalSubcommand {
/// List upcoming calendar events.
List,
/// List calendar events for an account within an optional time window.
List {
/// Account identifier owning the calendar.
#[arg(short, long, help = "Account identifier")]
account: String,
/// Calendar collection identifier.
#[arg(
short,
long,
default_value = "default",
help = "Calendar collection identifier"
)]
calendar: String,
/// Window start (unix seconds, inclusive).
#[arg(long, default_value_t = 0, help = "Window start (unix seconds)")]
start: i64,
/// Window end (unix seconds, inclusive).
#[arg(long, default_value_t = i64::MAX, help = "Window end (unix seconds)")]
end: i64,
},
/// Synchronize local calendar cache with remote CalDAV server.
Sync,
Sync {
/// Account identifier owning the calendar.
#[arg(short, long, help = "Account identifier")]
account: String,
/// Calendar collection identifier.
#[arg(
short,
long,
default_value = "default",
help = "Calendar collection identifier"
)]
calendar: String,
/// Window start (unix seconds, inclusive).
#[arg(long, default_value_t = 0, help = "Window start (unix seconds)")]
start: i64,
/// Window end (unix seconds, inclusive).
#[arg(long, default_value_t = i64::MAX, help = "Window end (unix seconds)")]
end: i64,
},
}
/// System subcommands (`nuncio system <verb>`).
@@ -448,6 +527,18 @@ pub enum FilterSubcommand {
#[arg(short, long, default_value_t = 50, help = "Log limit")]
limit: usize,
},
/// Retroactively apply the current rule set to every stored message,
/// streaming cumulative progress as the scan runs.
Triage {
/// Restrict the scan's reporting to a single rule ID. Not yet
/// honored by the daemon (see `nuncio.v1.Filters.Triage`) -- the
/// scan always evaluates every active rule.
#[arg(short, long, help = "Rule identifier (not yet honored)")]
rule_id: Option<String>,
/// Messages fetched per streamed progress update.
#[arg(short, long, help = "Messages scanned per chunk")]
chunk_size: Option<u32>,
},
}
#[cfg(test)]
File diff suppressed because it is too large. Load diff
+41 -15
View File
@@ -42,17 +42,11 @@ async fn system_test_cli_noun_verb_execution_matrix() {
// separately below via `ephemeral_with` + `SecretManager::mock()`
// rather than through this `ephemeral()`-constructed runner.
// 5. Calendar list & sync
let out: String = runner
.execute_command(
&Commands::Cal {
action: CalSubcommand::List,
},
true,
)
.await;
let json: Value = serde_json::from_str(&out).expect("valid json");
assert_eq!(json["status"], "ok");
// 5. Calendar list & sync are real gRPC clients of the `nunciod`
// daemon's `Calendar` API, so -- exactly like Folder/Mail above -- they
// are exercised separately below via `ephemeral_with` +
// `SecretManager::mock()` rather than through this
// `ephemeral()`-constructed runner.
}
/// `system status` is a real gRPC client of the `nunciod` daemon. With no
@@ -144,10 +138,10 @@ async fn account_add_and_list_report_honest_errors_when_daemon_unreachable() {
assert!(!add_out.contains("irrelevant-unreachable-daemon"));
}
/// `folder list`, `mail list`, `mail read`, `mail search`, and `mail mark`
/// are real gRPC clients of the `nunciod` daemon's `Mail` API. With no
/// daemon reachable, all of them must report a clear, honest error rather
/// than fabricating an empty result.
/// `folder list`, `mail list`, `mail read`, `mail search`, `mail mark`, and
/// `cal list`/`cal sync` are real gRPC clients of the `nunciod` daemon's
/// `Mail`/`Calendar` APIs. With no daemon reachable, all of them must report
/// a clear, honest error rather than fabricating an empty result.
#[tokio::test]
async fn mail_and_folder_report_honest_errors_when_daemon_unreachable() {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0")
@@ -233,4 +227,36 @@ async fn mail_and_folder_report_honest_errors_when_daemon_unreachable() {
)
.await,
);
assert_honest_error(
runner
.execute_command(
&Commands::Cal {
action: CalSubcommand::List {
account: "acct-1".to_string(),
calendar: "default".to_string(),
start: 0,
end: i64::MAX,
},
},
true,
)
.await,
);
assert_honest_error(
runner
.execute_command(
&Commands::Cal {
action: CalSubcommand::Sync {
account: "acct-1".to_string(),
calendar: "default".to_string(),
start: 0,
end: i64::MAX,
},
},
true,
)
.await,
);
}
+5
View File
@@ -4,6 +4,10 @@ version.workspace = true
edition.workspace = true
authors.workspace = true
license.workspace = true
repository.workspace = true
[lints]
workspace = true
[dependencies]
nuncio-core = { path = "../nuncio-core" }
@@ -21,3 +25,4 @@ async-trait = { workspace = true }
[dev-dependencies]
tempfile = { workspace = true }
wiremock = { workspace = true }
+18
View File
@@ -0,0 +1,18 @@
//! Protocol-agnostic async contacts backend trait definitions.
use async_trait::async_trait;
use crate::carddav::CardDavError;
use crate::models::Contact;
/// Protocol-agnostic contacts backend engine trait implemented by CardDAV and mock engines.
///
/// Mirrors `nuncio_cal::CalendarBackend`'s shape: `Send + Sync` so implementations can be held
/// behind an `Arc` and shared across daemon tasks, and returning the crate's single
/// [`CardDavError`] type so callers do not need to know whether a given implementation talks
/// to a real server or an in-memory fixture.
#[async_trait]
pub trait ContactsBackend: Send + Sync {
/// Fetch contacts belonging to `account_id`.
async fn fetch_contacts(&self, account_id: &str) -> Result<Vec<Contact>, CardDavError>;
}
+180 -19
View File
@@ -1,35 +1,56 @@
use crate::models::Contact;
use reqwest::Client;
//! CardDAV (RFC 6352) `REPORT` client and multistatus response parser.
use async_trait::async_trait;
use serde::{Deserialize, Serialize};
use thiserror::Error;
use tracing::info;
#[derive(Error, Debug)]
use crate::backend::ContactsBackend;
use crate::models::Contact;
use crate::parser::VCardParserAdapter;
/// Errors returned by the CardDAV client engine.
#[derive(Error, Debug, PartialEq, Eq)]
pub enum CardDavError {
#[error("HTTP error: {0}")]
Http(#[from] reqwest::Error),
#[error("CardDAV sync error: {0}")]
SyncError(String),
/// Failed to parse an RFC 6350 vCard payload embedded in a CardDAV response.
#[error("failed to parse vCard payload: {0}")]
ParseFailed(String),
/// CardDAV network/transport-layer failure (connection, TLS, or an unexpected HTTP
/// status) distinct from a payload that connected fine but failed to parse.
#[error("CardDAV transport failure: {0}")]
TransportFailed(String),
}
/// Configuration for a specific CardDAV address book collection endpoint, mirroring
/// `nuncio_cal::CalDavAccountConfig`.
///
/// `carddav_url` must already resolve to a specific address book collection (e.g.
/// `https://carddav.example.com/dav/addressbooks/user/jmaes/contacts/`) -- PROPFIND-based
/// `addressbook-home-set` auto-discovery is out of scope for this client.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct CardDavAccountConfig {
/// Nuncio account identifier this address book collection belongs to.
pub account_id: String,
/// Fully-qualified URL of the CardDAV address book collection to query.
pub carddav_url: String,
/// Basic-auth username (or app-specific username, per provider).
pub username: String,
/// Basic-auth secret (password or app-specific token) resolved from the OS keyring by the
/// caller -- never stored anywhere else in plaintext.
pub auth_token: String,
}
/// CardDAV client protocol engine managing address book `REPORT` queries against a real server.
pub struct CardDavClient {
#[allow(dead_code)]
client: Client,
config: CardDavAccountConfig,
http: reqwest::Client,
}
impl CardDavClient {
/// Create a new `CardDavClient` bound to a specific CardDAV address book collection.
pub fn new(config: CardDavAccountConfig) -> Self {
Self {
client: Client::builder()
http: reqwest::Client::builder()
.user_agent("Nuncio-Contacts-CardDAV/1.0")
.build()
.unwrap_or_default(),
@@ -37,13 +58,153 @@ impl CardDavClient {
}
}
pub async fn fetch_remote_vcards(&self) -> Result<Vec<Contact>, CardDavError> {
info!(
"Initiating CardDAV PROPFIND fetch from {}",
self.config.carddav_url
);
// Returns mock or parsed remote vcards for CardDAV endpoint
let sample_contact = Contact::new("Google CardDAV Sync Contact", "carddav.sync@nuncio.mx");
Ok(vec![sample_contact])
/// Construct a standard CardDAV `<card:addressbook-query>` XML payload (RFC 6352 Section
/// 8.6) requesting the full `address-data` of every card in the collection.
pub fn build_report_query() -> String {
r#"<?xml version="1.0" encoding="utf-8" ?>
<card:addressbook-query xmlns:d="DAV:" xmlns:card="urn:ietf:params:xml:ns:carddav">
<d:prop>
<d:getetag />
<card:address-data />
</d:prop>
</card:addressbook-query>"#
.to_string()
}
/// Parse a CardDAV WebDAV XML `<multistatus>` response containing embedded vCard data.
pub fn parse_multistatus_response(
&self,
account_id: &str,
raw_xml: &str,
) -> Result<Vec<Contact>, CardDavError> {
let mut contacts = Vec::new();
for block in raw_xml.split("<card:address-data>") {
if let Some((vcard_data, _)) = block.split_once("</card:address-data>") {
let clean_vcard = vcard_data.trim();
if !clean_vcard.is_empty() {
let contact_id = format!("carddav-{account_id}-{}", contacts.len() + 1);
let contact =
VCardParserAdapter::parse_vcard(&contact_id, account_id, clean_vcard)?;
contacts.push(contact);
}
}
}
Ok(contacts)
}
/// Issue a live CardDAV `REPORT` addressbook-query (RFC 6352 Section 8.6) against
/// `self.config.carddav_url` and parse the `multistatus` response into domain contacts.
///
/// This performs a real network request -- no canned or fabricated data is ever
/// returned. A server or network failure surfaces as [`CardDavError::TransportFailed`]; a
/// malformed vCard inside a returned response surfaces as [`CardDavError::ParseFailed`].
pub async fn fetch_remote_vcards(
&self,
account_id: &str,
) -> Result<Vec<Contact>, CardDavError> {
let body = Self::build_report_query();
// "REPORT" is a fixed, always-valid HTTP token; `from_bytes` cannot fail for it, but
// the error is still propagated rather than unwrapped so no code path here can panic.
let report_method = reqwest::Method::from_bytes(b"REPORT")
.map_err(|e| CardDavError::TransportFailed(format!("invalid HTTP method: {e}")))?;
let response = self
.http
.request(report_method, &self.config.carddav_url)
.header("Content-Type", "application/xml; charset=utf-8")
.header("Depth", "1")
.basic_auth(&self.config.username, Some(&self.config.auth_token))
.body(body)
.send()
.await
.map_err(|e| CardDavError::TransportFailed(e.to_string()))?;
let status = response.status();
// RFC 6352 REPORT responses are conventionally 207 Multi-Status; some servers reply
// 200 OK for a single-collection result, so any 2xx/207 status is accepted.
if status.as_u16() != 207 && !status.is_success() {
return Err(CardDavError::TransportFailed(format!(
"CardDAV server returned unexpected status {status}"
)));
}
let raw_xml = response
.text()
.await
.map_err(|e| CardDavError::TransportFailed(e.to_string()))?;
self.parse_multistatus_response(account_id, &raw_xml)
}
}
#[async_trait]
impl ContactsBackend for CardDavClient {
async fn fetch_contacts(&self, account_id: &str) -> Result<Vec<Contact>, CardDavError> {
self.fetch_remote_vcards(account_id).await
}
}
#[cfg(test)]
mod tests {
use super::*;
fn test_config() -> CardDavAccountConfig {
CardDavAccountConfig {
account_id: "acct-1".to_string(),
carddav_url: "https://carddav.example.com/addressbooks/contacts/".to_string(),
username: "jmaes".to_string(),
auth_token: "app-token-secret".to_string(),
}
}
#[test]
fn build_report_query_targets_addressbook_query() {
let query = CardDavClient::build_report_query();
assert!(query.contains("<card:addressbook-query"));
assert!(query.contains("<card:address-data"));
}
#[test]
fn parse_multistatus_response_extracts_contacts() {
let client = CardDavClient::new(test_config());
let xml_response = r#"<?xml version="1.0" encoding="utf-8"?>
<d:multistatus xmlns:d="DAV:" xmlns:card="urn:ietf:params:xml:ns:carddav">
<d:response>
<d:href>/addressbooks/contacts/alice.vcf</d:href>
<d:propstat>
<d:prop>
<card:address-data>BEGIN:VCARD
VERSION:4.0
FN:Alice Dev
EMAIL:alice@nuncio.mx
END:VCARD</card:address-data>
</d:prop>
</d:propstat>
</d:response>
</d:multistatus>"#;
let contacts = client
.parse_multistatus_response("acct-1", xml_response)
.expect("parse succeeds");
assert_eq!(contacts.len(), 1);
assert_eq!(contacts[0].display_name, "Alice Dev");
assert_eq!(contacts[0].emails[0].email, "alice@nuncio.mx");
assert_eq!(contacts[0].account_id.as_deref(), Some("acct-1"));
}
#[test]
fn parse_multistatus_response_with_no_cards_yields_empty_not_fabricated() {
let client = CardDavClient::new(test_config());
let xml_response = r#"<?xml version="1.0" encoding="utf-8"?>
<d:multistatus xmlns:d="DAV:" xmlns:card="urn:ietf:params:xml:ns:carddav"></d:multistatus>"#;
let contacts = client
.parse_multistatus_response("acct-1", xml_response)
.expect("parse succeeds");
assert!(contacts.is_empty());
}
}
+10
View File
@@ -1,12 +1,22 @@
//! Contacts (CardDAV / RFC 6350 vCard) engine: a real CardDAV `REPORT` client, a vCard
//! parser/generator round trip, and a protocol-agnostic [`ContactsBackend`] trait.
#![forbid(unsafe_code)]
#![cfg_attr(test, allow(clippy::unwrap_used, clippy::expect_used, clippy::panic))]
pub mod backend;
pub mod carddav;
pub mod db;
pub mod mock;
pub mod models;
pub mod parser;
pub use backend::ContactsBackend;
pub use carddav::{CardDavAccountConfig, CardDavClient, CardDavError};
pub use db::{ContactsDatabase, ContactsStoreError};
pub use mock::MockContactsBackend;
pub use models::{Contact, ContactEmail, ContactGroup, ContactPhone};
pub use parser::VCardParserAdapter;
#[cfg(test)]
mod tests {
+116
View File
@@ -0,0 +1,116 @@
//! Deterministic mock contacts backend for offline testing and integration verification.
use async_trait::async_trait;
use std::sync::{Arc, Mutex};
use crate::backend::ContactsBackend;
use crate::carddav::CardDavError;
use crate::models::Contact;
/// Thread-safe mock contacts backend for offline testing.
#[derive(Debug, Clone, Default)]
pub struct MockContactsBackend {
contacts: Arc<Mutex<Vec<Contact>>>,
should_fail: Arc<Mutex<bool>>,
}
impl MockContactsBackend {
/// Create a new `MockContactsBackend`.
pub fn new() -> Self {
Self::default()
}
/// Configure the mock to simulate CardDAV network failure errors.
pub fn set_should_fail(&self, fail: bool) {
if let Ok(mut flag) = self.should_fail.lock() {
*flag = fail;
}
}
/// Add a mock contact to storage.
pub fn add_contact(&self, contact: Contact) {
if let Ok(mut guard) = self.contacts.lock() {
guard.push(contact);
}
}
/// Retrieve contacts belonging to a specific account.
pub fn list_contacts(&self, account_id: &str) -> Result<Vec<Contact>, CardDavError> {
let should_fail = self
.should_fail
.lock()
.map_err(|e| CardDavError::ParseFailed(e.to_string()))?;
if *should_fail {
return Err(CardDavError::TransportFailed(
"simulated CardDAV network failure".to_string(),
));
}
let guard = self
.contacts
.lock()
.map_err(|e| CardDavError::ParseFailed(e.to_string()))?;
let matches = guard
.iter()
.filter(|c| c.account_id.as_deref() == Some(account_id))
.cloned()
.collect();
Ok(matches)
}
}
#[async_trait]
impl ContactsBackend for MockContactsBackend {
async fn fetch_contacts(&self, account_id: &str) -> Result<Vec<Contact>, CardDavError> {
self.list_contacts(account_id)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::models::Contact as ModelContact;
fn sample_contact(account_id: &str, display_name: &str) -> ModelContact {
let mut contact = ModelContact::new(display_name, format!("{display_name}@nuncio.mx"));
contact.account_id = Some(account_id.to_string());
contact
}
#[test]
fn mock_contacts_backend_operations() {
let mock = MockContactsBackend::new();
mock.add_contact(sample_contact("acct-1", "Mock Contact"));
let contacts = mock
.list_contacts("acct-1")
.expect("list contacts succeeds");
assert_eq!(contacts.len(), 1);
assert_eq!(contacts[0].display_name, "Mock Contact");
assert!(mock
.list_contacts("acct-other")
.expect("list succeeds")
.is_empty());
mock.set_should_fail(true);
assert!(mock.list_contacts("acct-1").is_err());
}
/// Proves `MockContactsBackend` is usable behind the `ContactsBackend` trait object, the
/// same way the daemon will inject either it or the real CardDAV client.
#[tokio::test]
async fn mock_contacts_backend_is_usable_as_a_trait_object() {
let mock = MockContactsBackend::new();
mock.add_contact(sample_contact("acct-1", "Trait Object Contact"));
let backend: Arc<dyn ContactsBackend> = Arc::new(mock);
let contacts = backend
.fetch_contacts("acct-1")
.await
.expect("fetch contacts succeeds");
assert_eq!(contacts.len(), 1);
assert_eq!(contacts[0].display_name, "Trait Object Contact");
}
}
+247
View File
@@ -0,0 +1,247 @@
//! vCard (RFC 6350) parser adapter converting raw vCard payloads into Nuncio [`Contact`]
//! domain entities -- the read half of the round trip whose write half is
//! [`Contact::to_vcard`].
use chrono::Utc;
use crate::carddav::CardDavError;
use crate::models::{Contact, ContactEmail, ContactPhone};
/// vCard parser adapter converting raw `.vcf`/CardDAV `address-data` payloads into Nuncio
/// [`Contact`] domain entities.
pub struct VCardParserAdapter;
impl VCardParserAdapter {
/// Maximum allowed single-vCard payload size (1MB -- generous for a single contact
/// record, small enough to reject a runaway/malicious response body outright).
pub const MAX_PAYLOAD_BYTES: usize = 1024 * 1024;
/// Parse a raw RFC 6350 vCard string into a [`Contact`] entity.
///
/// `id` is assigned by the caller (CardDAV responses carry no stable contact ID of their
/// own outside the enclosing WebDAV `href`); `account_id` tags which Nuncio account this
/// contact was fetched for. Never fabricates a placeholder name or email: a vCard with
/// neither `FN` nor any `EMAIL` is a parse failure, not an empty/default contact.
pub fn parse_vcard(
id: &str,
account_id: &str,
raw_vcard: &str,
) -> Result<Contact, CardDavError> {
if raw_vcard.len() > Self::MAX_PAYLOAD_BYTES {
return Err(CardDavError::ParseFailed(
"vCard payload exceeds maximum allowed limit of 1MB".to_string(),
));
}
if !raw_vcard.to_ascii_uppercase().contains("BEGIN:VCARD") {
return Err(CardDavError::ParseFailed(
"vCard payload is missing BEGIN:VCARD".to_string(),
));
}
let unfolded = unfold_lines(raw_vcard);
let mut display_name: Option<String> = None;
let mut given_name: Option<String> = None;
let mut family_name: Option<String> = None;
let mut organization: Option<String> = None;
let mut job_title: Option<String> = None;
let mut emails: Vec<ContactEmail> = Vec::new();
let mut phones: Vec<ContactPhone> = Vec::new();
for line in unfolded.lines() {
let line = line.trim();
if line.is_empty() {
continue;
}
let upper = line.to_ascii_uppercase();
if upper == "BEGIN:VCARD" || upper == "END:VCARD" || upper.starts_with("VERSION:") {
continue;
}
let Some((prop, value)) = line.split_once(':') else {
continue;
};
let mut segments = prop.split(';');
let name = segments.next().unwrap_or("").to_ascii_uppercase();
let params: Vec<&str> = segments.collect();
match name.as_str() {
"FN" => display_name = Some(value.trim().to_string()),
"N" => {
let comps: Vec<&str> = value.split(';').collect();
if let Some(family) = comps.first().map(|s| s.trim()).filter(|s| !s.is_empty())
{
family_name = Some(family.to_string());
}
if let Some(given) = comps.get(1).map(|s| s.trim()).filter(|s| !s.is_empty()) {
given_name = Some(given.to_string());
}
}
"ORG" => organization = Some(value.trim().to_string()),
"TITLE" => job_title = Some(value.trim().to_string()),
"EMAIL" => {
let label = extract_type(&params).unwrap_or_else(|| "other".to_string());
let is_primary = emails.is_empty();
emails.push(ContactEmail {
email: value.trim().to_string(),
label: label.to_ascii_lowercase(),
is_primary,
});
}
"TEL" => {
let label = extract_type(&params).unwrap_or_else(|| "other".to_string());
let is_primary = phones.is_empty();
phones.push(ContactPhone {
phone: value.trim().to_string(),
label: label.to_ascii_lowercase(),
is_primary,
});
}
_ => {}
}
}
let display_name = display_name
.or_else(|| emails.first().map(|e| e.email.clone()))
.ok_or_else(|| {
CardDavError::ParseFailed(
"vCard has neither FN nor EMAIL to derive a display name".to_string(),
)
})?;
let now = Utc::now();
Ok(Contact {
id: id.to_string(),
account_id: Some(account_id.to_string()),
display_name,
given_name,
family_name,
organization,
job_title,
notes: None,
avatar_url: None,
emails,
phones,
is_favorite: false,
interaction_count: 0,
last_interacted_at: None,
created_at: now,
updated_at: now,
})
}
}
/// Extract a `TYPE=...` parameter value from a vCard property's parameter list.
fn extract_type(params: &[&str]) -> Option<String> {
params.iter().find_map(|param| {
param
.to_ascii_uppercase()
.strip_prefix("TYPE=")
.map(|v| v.to_string())
})
}
/// Unfold RFC 6350 line continuations (a line starting with a space/tab is a continuation of
/// the previous line) into single logical lines, tolerating both CRLF and bare LF input.
fn unfold_lines(raw: &str) -> String {
let mut result = String::new();
for line in raw.split("\r\n").flat_map(|l| l.split('\n')) {
if (line.starts_with(' ') || line.starts_with('\t')) && !result.is_empty() {
result.push_str(line.trim_start());
} else {
if !result.is_empty() {
result.push('\n');
}
result.push_str(line);
}
}
result
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn parse_vcard_with_all_supported_fields() {
let vcard = "BEGIN:VCARD\r\n\
VERSION:4.0\r\n\
FN:James Maes\r\n\
N:Maes;James;;;\r\n\
ORG:KofTwentyTwo\r\n\
TITLE:Engineer\r\n\
EMAIL;TYPE=WORK:james.maes@kof22.com\r\n\
TEL;TYPE=MOBILE:+1-555-0199\r\n\
END:VCARD";
let contact =
VCardParserAdapter::parse_vcard("c-1", "acct-1", vcard).expect("parse succeeds");
assert_eq!(contact.display_name, "James Maes");
assert_eq!(contact.given_name.as_deref(), Some("James"));
assert_eq!(contact.family_name.as_deref(), Some("Maes"));
assert_eq!(contact.organization.as_deref(), Some("KofTwentyTwo"));
assert_eq!(contact.job_title.as_deref(), Some("Engineer"));
assert_eq!(contact.emails.len(), 1);
assert_eq!(contact.emails[0].email, "james.maes@kof22.com");
assert_eq!(contact.emails[0].label, "work");
assert!(contact.emails[0].is_primary);
assert_eq!(contact.phones.len(), 1);
assert_eq!(contact.phones[0].phone, "+1-555-0199");
assert_eq!(contact.phones[0].label, "mobile");
}
#[test]
fn parse_vcard_without_fn_falls_back_to_primary_email_as_display_name() {
let vcard = "BEGIN:VCARD\r\nVERSION:4.0\r\nEMAIL:alice@nuncio.mx\r\nEND:VCARD";
let contact =
VCardParserAdapter::parse_vcard("c-2", "acct-1", vcard).expect("parse succeeds");
assert_eq!(contact.display_name, "alice@nuncio.mx");
}
#[test]
fn parse_vcard_missing_begin_marker_is_a_real_error() {
let err = VCardParserAdapter::parse_vcard("c-3", "acct-1", "FN:No Envelope")
.expect_err("must reject a payload with no BEGIN:VCARD");
assert!(matches!(err, CardDavError::ParseFailed(_)));
}
#[test]
fn parse_vcard_with_no_name_or_email_is_a_real_error_not_a_fabricated_contact() {
let vcard = "BEGIN:VCARD\r\nVERSION:4.0\r\nORG:Ghost Corp\r\nEND:VCARD";
let err = VCardParserAdapter::parse_vcard("c-4", "acct-1", vcard)
.expect_err("a vCard with no FN and no EMAIL must never become a fabricated contact");
assert!(matches!(err, CardDavError::ParseFailed(_)));
}
/// Proves the parser is the true inverse of [`Contact::to_vcard`]: generating a vCard from
/// a rich domain [`Contact`] and parsing it back must reproduce the same
/// FN/N/ORG/TITLE/primary-EMAIL/primary-TEL fields.
#[test]
fn vcard_round_trip_preserves_core_fields() {
let mut original = Contact::new("Alice Developer", "alice@kof22.com");
original.given_name = Some("Alice".to_string());
original.family_name = Some("Developer".to_string());
original.organization = Some("KofTwentyTwo".to_string());
original.job_title = Some("Staff Engineer".to_string());
original.phones.push(ContactPhone {
phone: "+1-555-0100".to_string(),
label: "mobile".to_string(),
is_primary: true,
});
let vcard = original.to_vcard();
let round_tripped = VCardParserAdapter::parse_vcard("c-rt-1", "acct-1", &vcard)
.expect("round-trip parse succeeds");
assert_eq!(round_tripped.display_name, original.display_name);
assert_eq!(round_tripped.given_name, original.given_name);
assert_eq!(round_tripped.family_name, original.family_name);
assert_eq!(round_tripped.organization, original.organization);
assert_eq!(round_tripped.job_title, original.job_title);
assert_eq!(round_tripped.emails[0].email, original.emails[0].email);
assert_eq!(round_tripped.emails[0].label, original.emails[0].label);
assert_eq!(round_tripped.phones[0].phone, original.phones[0].phone);
assert_eq!(round_tripped.phones[0].label, original.phones[0].label);
}
}
@@ -0,0 +1,156 @@
//! WireMock integration test suite for the real CardDAV `REPORT` transport.
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use nuncio_contacts::{CardDavAccountConfig, CardDavClient, CardDavError, ContactsBackend};
use wiremock::matchers::{method, path};
use wiremock::{Mock, MockServer, ResponseTemplate};
/// A canned CardDAV `multistatus` response containing two real vCards, exercising the real
/// HTTP transport together with the vCard parser.
fn multistatus_body() -> String {
r#"<?xml version="1.0" encoding="utf-8"?>
<d:multistatus xmlns:d="DAV:" xmlns:card="urn:ietf:params:xml:ns:carddav">
<d:response>
<d:href>/addressbooks/contacts/james.vcf</d:href>
<d:propstat>
<d:prop>
<card:address-data>BEGIN:VCARD
VERSION:4.0
FN:James Maes
N:Maes;James;;;
ORG:KofTwentyTwo
TITLE:Founder
EMAIL;TYPE=WORK:james.maes@kof22.com
TEL;TYPE=MOBILE:+1-555-0199
END:VCARD</card:address-data>
</d:prop>
</d:propstat>
</d:response>
<d:response>
<d:href>/addressbooks/contacts/alice.vcf</d:href>
<d:propstat>
<d:prop>
<card:address-data>BEGIN:VCARD
VERSION:4.0
FN:Alice Dev
EMAIL:alice@nuncio.mx
END:VCARD</card:address-data>
</d:prop>
</d:propstat>
</d:response>
</d:multistatus>"#
.to_string()
}
#[tokio::test]
async fn wiremock_carddav_report_fetches_and_parses_vcards() {
let mock_server = MockServer::start().await;
Mock::given(method("REPORT"))
.and(path("/addressbooks/contacts/"))
.respond_with(ResponseTemplate::new(207).set_body_string(multistatus_body()))
.mount(&mock_server)
.await;
let config = CardDavAccountConfig {
account_id: "acct-wm-1".to_string(),
carddav_url: format!("{}/addressbooks/contacts/", mock_server.uri()),
username: "james.maes".to_string(),
auth_token: "wiremock-app-token".to_string(),
};
let client = CardDavClient::new(config);
// Drive the real client through the `ContactsBackend` trait, exactly how the daemon will
// invoke either this real client or `MockContactsBackend` behind the same seam.
let contacts: Vec<_> = ContactsBackend::fetch_contacts(&client, "acct-wm-1")
.await
.expect("fetch_contacts succeeds against the wiremock server");
assert_eq!(contacts.len(), 2);
let james = contacts
.iter()
.find(|c| c.display_name == "James Maes")
.expect("James Maes contact present");
assert_eq!(james.account_id.as_deref(), Some("acct-wm-1"));
assert_eq!(james.given_name.as_deref(), Some("James"));
assert_eq!(james.family_name.as_deref(), Some("Maes"));
assert_eq!(james.organization.as_deref(), Some("KofTwentyTwo"));
assert_eq!(james.job_title.as_deref(), Some("Founder"));
assert_eq!(james.emails[0].email, "james.maes@kof22.com");
assert_eq!(james.phones[0].phone, "+1-555-0199");
let alice = contacts
.iter()
.find(|c| c.display_name == "Alice Dev")
.expect("Alice Dev contact present");
assert_eq!(alice.emails[0].email, "alice@nuncio.mx");
}
#[tokio::test]
async fn wiremock_carddav_report_server_error_surfaces_as_transport_failure() {
let mock_server = MockServer::start().await;
Mock::given(method("REPORT"))
.and(path("/addressbooks/broken/"))
.respond_with(ResponseTemplate::new(500))
.mount(&mock_server)
.await;
let config = CardDavAccountConfig {
account_id: "acct-wm-2".to_string(),
carddav_url: format!("{}/addressbooks/broken/", mock_server.uri()),
username: "james.maes".to_string(),
auth_token: "wiremock-app-token".to_string(),
};
let client = CardDavClient::new(config);
let err = client
.fetch_remote_vcards("acct-wm-2")
.await
.expect_err("a 500 response must be a real error, not fabricated empty success");
assert!(matches!(err, CardDavError::TransportFailed(_)));
}
#[tokio::test]
async fn wiremock_carddav_report_malformed_vcard_surfaces_as_parse_failure() {
let mock_server = MockServer::start().await;
let malformed_body = r#"<?xml version="1.0" encoding="utf-8"?>
<d:multistatus xmlns:d="DAV:" xmlns:card="urn:ietf:params:xml:ns:carddav">
<d:response>
<d:href>/addressbooks/malformed/ghost.vcf</d:href>
<d:propstat>
<d:prop>
<card:address-data>BEGIN:VCARD
VERSION:4.0
ORG:No Name Or Email
END:VCARD</card:address-data>
</d:prop>
</d:propstat>
</d:response>
</d:multistatus>"#;
Mock::given(method("REPORT"))
.and(path("/addressbooks/malformed/"))
.respond_with(ResponseTemplate::new(207).set_body_string(malformed_body))
.mount(&mock_server)
.await;
let config = CardDavAccountConfig {
account_id: "acct-wm-3".to_string(),
carddav_url: format!("{}/addressbooks/malformed/", mock_server.uri()),
username: "james.maes".to_string(),
auth_token: "wiremock-app-token".to_string(),
};
let client = CardDavClient::new(config);
let err = client
.fetch_remote_vcards("acct-wm-3")
.await
.expect_err("a vCard with neither FN nor EMAIL must be a real parse error");
assert!(matches!(err, CardDavError::ParseFailed(_)));
}
+13
View File
@@ -93,6 +93,19 @@ pub enum CoreEvent {
/// Action description.
action_taken: String,
},
/// Progress notification emitted during an inbound mail sync, so a
/// long-running sync of a large mailbox is observably progressing rather
/// than indistinguishable from a hang. Emitted per folder as it completes.
SyncProgress {
/// Account being synced, if the sync is scoped to one.
account_id: Option<String>,
/// Folder whose messages were just fetched and persisted.
folder_id: String,
/// Messages fetched from this folder on this pass.
fetched: usize,
/// Cumulative messages fetched across the sync so far.
total: usize,
},
/// Progress notification during bulk filter execution over a batch of emails.
BatchFilterProgress {
/// Messages processed so far in the batch.
+33 -2
View File
@@ -97,9 +97,12 @@ impl CompiledFilter {
.unwrap_or("");
Self::eval_string_op(body, &leaf.operator, &leaf.value, regexes)
}
FilterField::Folder | FilterField::Account => {
FilterField::Folder => {
Self::eval_string_op(&email.folder_id, &leaf.operator, &leaf.value, regexes)
}
FilterField::Account => {
Self::eval_string_op(&email.account_id, &leaf.operator, &leaf.value, regexes)
}
FilterField::HasAttachment => {
let has = !email.attachments.is_empty();
if let FilterValue::Boolean(b) = leaf.value {
@@ -148,7 +151,11 @@ impl CompiledFilter {
}
}
FilterField::Header(_name) => {
// Header evaluation fallback
// Unreachable defensive branch: `NsqlValidator::pass1_field_types`
// rejects any rule with a `header[...]` condition before it can
// reach a `CompiledFilter`, since `Email` has no parsed-headers
// field to match against. Kept only so this match stays
// exhaustive over `FilterField`.
false
}
}
@@ -475,6 +482,30 @@ mod tests {
assert_eq!(engine.evaluate(&email).len(), 1);
}
#[test]
fn test_account_condition_field_matches_account_id_not_folder_id() {
let nsql = "WHERE account = 'acct-a' ACTION MARK READ";
let rule = crate::parser::NsqlParser::parse_rule("Account Filter", 1, nsql).unwrap();
let engine = FilterEngine::new(vec![rule]).unwrap();
// Same folder_id, different account_id: a rule matching on the
// `account` condition field must key off email.account_id, never
// fall through to comparing folder_id.
let email_a = test_email("acct-a", "Anything", "shared-folder");
assert_eq!(
engine.evaluate(&email_a).len(),
1,
"rule on account = 'acct-a' must match a message with account_id 'acct-a'"
);
let email_b = test_email("acct-b", "Anything", "shared-folder");
assert!(
engine.evaluate(&email_b).is_empty(),
"rule on account = 'acct-a' must not match a message with account_id 'acct-b', \
even when it shares the same folder_id as an acct-a message"
);
}
#[test]
fn test_not_in_operator_is_complement_of_in() {
let nsql = "WHERE folder NOT IN ('spam', 'trash') ACTION MARK READ";
+16 -1
View File
@@ -83,11 +83,14 @@ impl NsqlValidator {
"field 'has_attachment' requires a boolean value".to_string(),
)),
},
FilterField::Header(_) => Err(ValidationError::FieldTypeMismatch(
"header field conditions are not yet supported: message headers are not indexed"
.to_string(),
)),
FilterField::Subject
| FilterField::From
| FilterField::To
| FilterField::Body
| FilterField::Header(_)
| FilterField::Folder
| FilterField::Account => match &leaf.value {
FilterValue::String(_) | FilterValue::List(_) => Ok(()),
@@ -322,6 +325,18 @@ mod tests {
assert!(matches!(err, ValidationError::FieldTypeMismatch(_)));
}
#[test]
fn test_pass1_rejects_unsupported_header_field_conditions() {
let rule = crate::parser::NsqlParser::parse_rule(
"Header Spam Score",
1,
"WHERE header['X-Spam-Score'] = '10' ACTION MARK READ",
)
.unwrap();
let err = NsqlValidator::validate(&rule, &ValidationOptions::default()).unwrap_err();
assert!(matches!(err, ValidationError::FieldTypeMismatch(_)));
}
#[test]
fn test_pass4_redos_validation() {
let rule = crate::parser::NsqlParser::parse_rule(
+4 -1
View File
@@ -26,8 +26,11 @@ rustls.workspace = true
webpki-roots.workspace = true
rustls-pki-types.workspace = true
tokio-stream.workspace = true
reqwest.workspace = true
[dev-dependencies]
tempfile.workspace = true
wiremock.workspace = true
reqwest.workspace = true
# `test-util` (not in tokio's `full`) enables paused-clock tests so the
# bounded per-item FETCH timeout can be asserted without a real-time wait.
tokio = { workspace = true, features = ["test-util"] }
File diff suppressed because it is too large. Load diff
+326 -38
View File
@@ -57,8 +57,25 @@ pub struct JmapEmailChangesResponse {
pub destroyed: Vec<String>,
}
/// Raw JMAP email object representation.
/// JMAP `Email/query` response payload wrapper (RFC 8620 Section 5.5), narrowed
/// to just the matched id list -- `Email/get` needs explicit ids (or `None`
/// for "all", which real servers reject for large mailboxes), so a query must
/// always precede a get.
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct JmapEmailQueryResponse {
/// Matched email ids, in server-defined order.
#[serde(default)]
pub ids: Vec<String>,
}
/// Raw JMAP email object representation (RFC 8621 Section 4.1). Field names
/// must stay `camelCase` on the wire -- `receivedAt`/`isUnread`/`bodySnippet`
/// are the actual JMAP property names; without `rename_all` these silently
/// fail to match and every parsed message gets a zero timestamp, `read:
/// true`, and no snippet.
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct JmapEmail {
pub id: String,
pub subject: Option<String>,
@@ -75,19 +92,70 @@ pub struct JmapAddress {
pub email: String,
}
/// JMAP protocol engine implementing RFC 8620 / 8621.
/// Raw JMAP `Mailbox` object representation (RFC 8621 Section 2).
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct JmapMailbox {
pub id: String,
pub name: String,
#[serde(default)]
pub total_emails: u32,
#[serde(default)]
pub unread_emails: u32,
}
/// JMAP `Mailbox/get` response payload wrapper.
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct JmapMailboxGetResponse {
/// List of retrieved mailbox objects.
pub list: Vec<JmapMailbox>,
}
/// JMAP protocol engine implementing RFC 8620 / 8621. Mirrors [`crate::imap::ImapEngine`]'s
/// shape: `new` builds an uncredentialed engine that keeps the deterministic, network-free
/// fallback behaviour existing preview/dry-run callers and unit tests rely on; `with_credentials`
/// is what production (`nunciod::sync::build_mail_backend`) always uses, and performs genuine
/// HTTP session discovery and JSON-RPC calls against the real server.
pub struct JmapEngine {
account_id: String,
host: String,
username: Option<String>,
password: Option<String>,
http: reqwest::Client,
}
impl JmapEngine {
/// Create a new `JmapEngine` bound to an account ID.
/// Create a new, uncredentialed `JmapEngine` bound to an account ID. Never performs
/// network I/O -- callers that need a real server connection must use
/// [`JmapEngine::with_credentials`].
pub fn new(account_id: &str) -> Self {
Self {
account_id: account_id.to_string(),
host: String::new(),
username: None,
password: None,
http: reqwest::Client::new(),
}
}
/// Create a new `JmapEngine` with a server host and Basic-auth credentials. This is the
/// production path: `sync_folders`/`sync_messages` perform real HTTP session discovery and
/// JMAP JSON-RPC calls against `host`.
pub fn with_credentials(account_id: &str, host: &str, username: &str, password: &str) -> Self {
Self {
account_id: account_id.to_string(),
host: host.to_string(),
username: Some(username.to_string()),
password: Some(password.to_string()),
http: reqwest::Client::new(),
}
}
fn has_credentials(&self) -> bool {
self.username.is_some() && self.password.is_some()
}
/// Construct standard JMAP Well-Known session discovery URL (RFC 8620 Section 2.1).
pub fn build_session_url(host: &str) -> String {
if host.starts_with("http://") || host.starts_with("https://") {
@@ -116,6 +184,25 @@ impl JmapEngine {
})
}
/// Build JSON-RPC request invocation for `Email/query`, filtered to a single mailbox.
/// `Email/get` requires an explicit id list (or `None` for "all", which real servers
/// reject for anything but tiny mailboxes), so a query must always precede a get.
pub fn build_email_query_request(account_id: &str, mailbox_id: &str) -> Value {
json!({
"using": ["urn:ietf:params:jmap:core", "urn:ietf:params:jmap:mail"],
"methodCalls": [
[
"Email/query",
{
"accountId": account_id,
"filter": { "inMailboxes": [mailbox_id] }
},
"c1"
]
]
})
}
/// Build JSON-RPC request invocation for `Email/changes` (differential sync).
pub fn build_email_changes_request(account_id: &str, since_state: &str) -> Value {
json!({
@@ -133,12 +220,46 @@ impl JmapEngine {
})
}
/// Build JSON-RPC request invocation for `Mailbox/get` (all mailboxes).
pub fn build_mailbox_get_request(account_id: &str) -> Value {
json!({
"using": ["urn:ietf:params:jmap:core", "urn:ietf:params:jmap:mail"],
"methodCalls": [
[
"Mailbox/get",
{
"accountId": account_id,
"ids": null
},
"c1"
]
]
})
}
/// Parse JMAP RFC 8620 session response JSON.
pub fn parse_session(raw_json: &str) -> Result<JmapSession, MailError> {
serde_json::from_str(raw_json)
.map_err(|e| MailError::ParseFailed(format!("invalid JMAP session JSON: {e}")))
}
/// Extract the argument payload of the first method call from a JMAP JSON-RPC response,
/// supporting both the full `methodResponses` envelope and a bare response object (used by
/// tests that exercise a parser directly against a single method's payload).
fn extract_method_response_payload(val: &Value) -> Result<Value, MailError> {
if let Some(calls) = val.get("methodResponses").and_then(|v| v.as_array()) {
let first_call = calls
.first()
.ok_or_else(|| MailError::ParseFailed("empty methodResponses array".to_string()))?;
let args = first_call.get(1).ok_or_else(|| {
MailError::ParseFailed("missing method response payload".to_string())
})?;
Ok(args.clone())
} else {
Ok(val.clone())
}
}
/// Parse raw JMAP `Email/get` JSON response payload into domain [`Email`] list and new state string.
pub fn parse_email_get_response(
&self,
@@ -146,23 +267,9 @@ impl JmapEngine {
) -> Result<(Vec<Email>, String), MailError> {
let val: Value = serde_json::from_str(raw_json)
.map_err(|e| MailError::ParseFailed(format!("invalid JMAP JSON response: {e}")))?;
// Support both direct object and RFC 8620 methodCalls wrapper
let resp: JmapEmailGetResponse = if let Some(calls) =
val.get("methodResponses").and_then(|v| v.as_array())
{
let first_call = calls
.first()
.ok_or_else(|| MailError::ParseFailed("empty methodResponses array".to_string()))?;
let args = first_call.get(1).ok_or_else(|| {
MailError::ParseFailed("missing method response payload".to_string())
})?;
serde_json::from_value(args.clone())
.map_err(|e| MailError::ParseFailed(format!("invalid Email/get payload: {e}")))?
} else {
serde_json::from_value(val)
.map_err(|e| MailError::ParseFailed(format!("invalid Email/get payload: {e}")))?
};
let payload = Self::extract_method_response_payload(&val)?;
let resp: JmapEmailGetResponse = serde_json::from_value(payload)
.map_err(|e| MailError::ParseFailed(format!("invalid Email/get payload: {e}")))?;
let emails = resp
.list
@@ -208,31 +315,117 @@ impl JmapEngine {
) -> Result<(Vec<String>, Vec<String>, String), MailError> {
let val: Value = serde_json::from_str(raw_json)
.map_err(|e| MailError::ParseFailed(format!("invalid JMAP JSON response: {e}")))?;
let resp: JmapEmailChangesResponse =
if let Some(calls) = val.get("methodResponses").and_then(|v| v.as_array()) {
let first_call = calls.first().ok_or_else(|| {
MailError::ParseFailed("empty methodResponses array".to_string())
})?;
let args = first_call.get(1).ok_or_else(|| {
MailError::ParseFailed("missing method response payload".to_string())
})?;
serde_json::from_value(args.clone()).map_err(|e| {
MailError::ParseFailed(format!("invalid Email/changes payload: {e}"))
})?
} else {
serde_json::from_value(val).map_err(|e| {
MailError::ParseFailed(format!("invalid Email/changes payload: {e}"))
})?
};
let payload = Self::extract_method_response_payload(&val)?;
let resp: JmapEmailChangesResponse = serde_json::from_value(payload)
.map_err(|e| MailError::ParseFailed(format!("invalid Email/changes payload: {e}")))?;
Ok((resp.updated, resp.destroyed, resp.new_state))
}
/// Parse raw JMAP `Email/query` JSON response payload into a matched id list.
pub fn parse_email_query_response(raw_json: &str) -> Result<Vec<String>, MailError> {
let val: Value = serde_json::from_str(raw_json)
.map_err(|e| MailError::ParseFailed(format!("invalid JMAP JSON response: {e}")))?;
let payload = Self::extract_method_response_payload(&val)?;
let resp: JmapEmailQueryResponse = serde_json::from_value(payload)
.map_err(|e| MailError::ParseFailed(format!("invalid Email/query payload: {e}")))?;
Ok(resp.ids)
}
/// Parse raw JMAP `Mailbox/get` JSON response payload into a mailbox list.
pub fn parse_mailbox_get_response(raw_json: &str) -> Result<Vec<JmapMailbox>, MailError> {
let val: Value = serde_json::from_str(raw_json)
.map_err(|e| MailError::ParseFailed(format!("invalid JMAP JSON response: {e}")))?;
let payload = Self::extract_method_response_payload(&val)?;
let resp: JmapMailboxGetResponse = serde_json::from_value(payload)
.map_err(|e| MailError::ParseFailed(format!("invalid Mailbox/get payload: {e}")))?;
Ok(resp.list)
}
/// Issue an authenticated JMAP session discovery `GET /.well-known/jmap` against `self.host`.
/// A network failure or unparseable response surfaces as a genuine [`MailError`] -- there is
/// no fallback to canned data once credentials are present.
async fn discover_session(&self) -> Result<JmapSession, MailError> {
let url = Self::build_session_url(&self.host);
let mut request = self.http.get(&url);
if let (Some(u), Some(p)) = (&self.username, &self.password) {
request = request.basic_auth(u, Some(p));
}
let response = request.send().await.map_err(|e| {
MailError::NetworkError(format!("JMAP session discovery at {url} failed: {e}"))
})?;
let status = response.status();
let body = response.text().await.map_err(|e| {
MailError::NetworkError(format!("failed to read JMAP session response body: {e}"))
})?;
if !status.is_success() {
return Err(MailError::NetworkError(format!(
"JMAP session discovery at {url} returned status {status}"
)));
}
Self::parse_session(&body)
}
/// POST a JMAP JSON-RPC request body to `api_url`, returning the raw response text. A
/// non-2xx status or transport failure surfaces as [`MailError::NetworkError`].
async fn post_jmap(&self, api_url: &str, body: &Value) -> Result<String, MailError> {
let mut request = self.http.post(api_url).json(body);
if let (Some(u), Some(p)) = (&self.username, &self.password) {
request = request.basic_auth(u, Some(p));
}
let response = request.send().await.map_err(|e| {
MailError::NetworkError(format!("JMAP request to {api_url} failed: {e}"))
})?;
let status = response.status();
let text = response.text().await.map_err(|e| {
MailError::NetworkError(format!("failed to read JMAP response body: {e}"))
})?;
if !status.is_success() {
return Err(MailError::NetworkError(format!(
"JMAP server at {api_url} returned status {status}"
)));
}
Ok(text)
}
/// Resolve the JMAP `accountId` to use for API calls: the session's advertised primary mail
/// account if present, else the Nuncio account id as a last resort.
fn resolve_account_id(&self, session: &JmapSession) -> String {
session
.primary_accounts
.get("urn:ietf:params:jmap:mail")
.cloned()
.unwrap_or_else(|| self.account_id.clone())
}
}
#[async_trait]
impl MailBackend for JmapEngine {
async fn sync_folders(&self) -> Result<Vec<Folder>, MailError> {
if self.has_credentials() {
let session = self.discover_session().await?;
let account_id = self.resolve_account_id(&session);
let request = Self::build_mailbox_get_request(&account_id);
let raw = self.post_jmap(&session.api_url, &request).await?;
let mailboxes = Self::parse_mailbox_get_response(&raw)?;
return Ok(mailboxes
.into_iter()
.map(|mb| Folder {
id: mb.id,
name: mb.name,
total_messages: mb.total_emails as usize,
unread_messages: mb.unread_emails as usize,
})
.collect());
}
Ok(vec![
Folder {
id: "inbox".to_string(),
@@ -257,9 +450,22 @@ impl MailBackend for JmapEngine {
async fn sync_messages(
&self,
_folder_id: &str,
folder_id: &str,
_since_state: Option<&str>,
) -> Result<(Vec<Email>, String), MailError> {
if self.has_credentials() {
let session = self.discover_session().await?;
let account_id = self.resolve_account_id(&session);
let query_request = Self::build_email_query_request(&account_id, folder_id);
let query_raw = self.post_jmap(&session.api_url, &query_request).await?;
let ids = Self::parse_email_query_response(&query_raw)?;
let get_request = Self::build_email_get_request(&account_id, Some(ids));
let get_raw = self.post_jmap(&session.api_url, &get_request).await?;
return self.parse_email_get_response(&get_raw);
}
let sample_jmap = json!({
"state": "s-100",
"list": [
@@ -314,6 +520,23 @@ mod tests {
assert_eq!(req["methodCalls"][0][1]["sinceState"], "state-1");
}
#[test]
fn build_email_query_request_json_structure() {
let req = JmapEngine::build_email_query_request("acct-1", "mailbox-1");
assert_eq!(req["methodCalls"][0][0], "Email/query");
assert_eq!(
req["methodCalls"][0][1]["filter"]["inMailboxes"][0],
"mailbox-1"
);
}
#[test]
fn build_mailbox_get_request_json_structure() {
let req = JmapEngine::build_mailbox_get_request("acct-1");
assert_eq!(req["methodCalls"][0][0], "Mailbox/get");
assert_eq!(req["methodCalls"][0][1]["accountId"], "acct-1");
}
#[test]
fn parse_session_valid_payload() {
let raw = r#"{
@@ -350,4 +573,69 @@ mod tests {
let res = engine.parse_email_get_response("{ invalid json }");
assert!(res.is_err());
}
#[test]
fn parse_email_query_response_extracts_ids() {
let raw = r#"{
"methodResponses": [
["Email/query", {"ids": ["msg-1", "msg-2"]}, "c1"]
]
}"#;
let ids = JmapEngine::parse_email_query_response(raw).expect("parse query");
assert_eq!(ids, vec!["msg-1".to_string(), "msg-2".to_string()]);
}
#[test]
fn parse_mailbox_get_response_extracts_mailboxes() {
let raw = r#"{
"methodResponses": [
["Mailbox/get", {"list": [
{"id": "mb-1", "name": "Inbox", "totalEmails": 5, "unreadEmails": 2}
]}, "c1"]
]
}"#;
let mailboxes = JmapEngine::parse_mailbox_get_response(raw).expect("parse mailboxes");
assert_eq!(mailboxes.len(), 1);
assert_eq!(mailboxes[0].id, "mb-1");
assert_eq!(mailboxes[0].total_emails, 5);
assert_eq!(mailboxes[0].unread_emails, 2);
}
/// Regression test for the camelCase wire-format bug: `JmapEmail` must deserialize
/// `receivedAt`/`isUnread`/`bodySnippet` (the real JMAP wire property names) rather than
/// silently defaulting them to `None` because it only matched the Rust field names.
#[test]
fn jmap_email_deserializes_camel_case_wire_fields() {
let raw = r#"{
"id": "msg-1",
"subject": "Test",
"from": [{"email": "a@nuncio.mx"}],
"to": [{"email": "b@nuncio.mx"}],
"receivedAt": 1700000000,
"isUnread": true,
"bodySnippet": "hello world"
}"#;
let email: JmapEmail = serde_json::from_str(raw).expect("parse camelCase JmapEmail");
assert_eq!(email.received_at, Some(1700000000));
assert_eq!(email.is_unread, Some(true));
assert_eq!(email.body_snippet, Some("hello world".to_string()));
}
#[tokio::test]
async fn uncredentialed_engine_keeps_static_fallback_behaviour() {
let engine = JmapEngine::new("acct-1");
assert!(!engine.has_credentials());
let folders = engine.sync_folders().await.expect("sync folders");
assert_eq!(folders.len(), 3);
assert_eq!(folders[0].id, "inbox");
let (emails, state) = engine
.sync_messages("inbox", None)
.await
.expect("sync messages");
assert_eq!(state, "s-100");
assert_eq!(emails.len(), 1);
assert_eq!(emails[0].id, "jmap-msg-1");
}
}
+51 -3
View File
@@ -14,12 +14,41 @@ pub struct MockMailBackend {
messages: Arc<Mutex<Vec<Email>>>,
sent_messages: Arc<Mutex<Vec<Email>>>,
should_fail: Arc<Mutex<bool>>,
/// Every `since_state` argument this mock's `sync_messages` was called
/// with, in call order, so tests can prove a caller threads the returned
/// checkpoint back into the next sync (real incrementality).
since_state_calls: Arc<Mutex<Vec<Option<String>>>>,
/// The checkpoint token this mock returns from `sync_messages`.
returned_state: Arc<Mutex<String>>,
}
impl MockMailBackend {
/// Create a new `MockMailBackend` with empty storage.
pub fn new() -> Self {
Self::default()
let backend = Self::default();
if let Ok(mut state) = backend.returned_state.lock() {
*state = "mock-state-token-100".to_string();
}
backend
}
/// Override the checkpoint token this mock returns from `sync_messages`,
/// so a test can control the exact state a caller must persist and thread
/// back on the next sync.
pub fn set_returned_state(&self, state: &str) {
if let Ok(mut guard) = self.returned_state.lock() {
*guard = state.to_string();
}
}
/// Every `since_state` argument passed to `sync_messages`, in call order.
/// A caller that genuinely resumes from a checkpoint will show `None` on
/// the first call and the previously-returned state on the next.
pub fn since_state_calls(&self) -> Vec<Option<String>> {
self.since_state_calls
.lock()
.map(|guard| guard.clone())
.unwrap_or_default()
}
/// Configure the mock to simulate network failure errors.
@@ -74,8 +103,14 @@ impl MailBackend for MockMailBackend {
async fn sync_messages(
&self,
folder_id: &str,
_since_state: Option<&str>,
since_state: Option<&str>,
) -> Result<(Vec<Email>, String), MailError> {
// Record the checkpoint arg before any early return, so even a failing
// call is visible to a test asserting on how the caller resumes.
if let Ok(mut calls) = self.since_state_calls.lock() {
calls.push(since_state.map(str::to_string));
}
let should_fail = self
.should_fail
.lock()
@@ -85,6 +120,19 @@ impl MailBackend for MockMailBackend {
"simulated network failure".to_string(),
));
}
let returned_state = self
.returned_state
.lock()
.map_err(|e| MailError::ParseFailed(e.to_string()))?
.clone();
// Simulate a since_state-aware backend: with a prior checkpoint there
// is nothing new to hand back, so the incremental fetch is genuinely
// narrower (empty) rather than a full re-report of every message.
if since_state.is_some() {
return Ok((Vec::new(), returned_state));
}
let messages = self
.messages
.lock()
@@ -94,7 +142,7 @@ impl MailBackend for MockMailBackend {
.filter(|m| m.folder_id == folder_id)
.cloned()
.collect();
Ok((matches, "mock-state-token-100".to_string()))
Ok((matches, returned_state))
}
async fn send_email(&self, email: &Email) -> Result<(), MailError> {
+8
View File
@@ -32,6 +32,14 @@ pub enum MailError {
#[error("IMAP error: {0}")]
ImapError(String),
/// A single FETCH response item exceeded its bounded read timeout without
/// the server responding. Distinct from a generic [`MailError::ImapError`]
/// so callers can tell an honest mid-stream stall (dead socket, server-side
/// hang) apart from a protocol/parse failure instead of the sync hanging
/// forever.
#[error("IMAP FETCH stalled: {0}")]
FetchStalled(String),
/// Network or IO error.
#[error("network I/O error: {0}")]
NetworkError(String),
+116 -29
View File
@@ -9,6 +9,7 @@ use lettre::transport::smtp::client::{Tls, TlsParameters};
use lettre::transport::smtp::Error as LettreSmtpError;
use lettre::{AsyncSmtpTransport, AsyncTransport, Tokio1Executor};
use nuncio_core::model::{Attachment, Email};
use nuncio_core::TlsMode;
use crate::backend::{MessageSender, OutboundMessage};
use crate::parser::MailError;
@@ -42,12 +43,39 @@ pub struct SmtpTransportEngine {
}
impl SmtpTransportEngine {
/// Create a new [`SmtpTransportEngine`] with initialized transport client.
pub fn new(host: &str, port: u16, username: &str, password: &str) -> Result<Self, MailError> {
let transport = Self::build_transport(host, port, username, password)?;
/// Create a new [`SmtpTransportEngine`] with initialized transport client
/// for a specific transport security [`TlsMode`].
pub fn new(
host: &str,
port: u16,
tls_mode: TlsMode,
username: &str,
password: &str,
) -> Result<Self, MailError> {
let transport = Self::build_transport(host, port, tls_mode, username, password)?;
Ok(Self { transport })
}
/// Probe the SMTP endpoint: open a real connection to the configured
/// relay (performing the implicit-TLS wrap or STARTTLS upgrade the
/// configured [`TlsMode`] selects) and confirm the server responds. Backs
/// the daemon's `TestAccountConnection` SMTP leg. Returns `Ok(())` only
/// when a connection genuinely succeeded -- a dial/handshake failure
/// surfaces as the real [`MailError`], never a fabricated success.
///
/// This validates transport reachability and TLS negotiation; SMTP has no
/// standalone authentication probe (credentials are exercised at message
/// submission time), so this deliberately does not claim to verify auth.
pub async fn probe_connection(&self) -> Result<(), MailError> {
match self.transport.test_connection().await {
Ok(true) => Ok(()),
Ok(false) => Err(MailError::TransportFailed(
"SMTP server did not accept the connection probe".to_string(),
)),
Err(e) => Err(MailError::from(e)),
}
}
/// Create a new [`SmtpTransportEngine`] from an existing [`AsyncSmtpTransport`] client.
pub fn with_transport(transport: AsyncSmtpTransport<Tokio1Executor>) -> Self {
Self { transport }
@@ -227,33 +255,53 @@ impl SmtpTransportEngine {
Ok(())
}
/// Build an [`AsyncSmtpTransport`] client instance.
/// Configures Implicit TLS (`Tls::Wrapper`) for port 465, and STARTTLS (`Tls::Required`) for port 587 or other ports.
/// Build an [`AsyncSmtpTransport`] client instance whose transport
/// security genuinely follows the configured [`TlsMode`]:
///
/// - [`TlsMode::ImplicitTls`] wraps the socket in TLS immediately
/// (`Tls::Wrapper`), as on the SMTPS port (465).
/// - [`TlsMode::StartTls`] connects in the clear and REQUIRES a STARTTLS
/// upgrade before proceeding (`Tls::Required`), as on the submission
/// port (587); a server without STARTTLS is rejected rather than
/// silently falling back to plaintext.
/// - [`TlsMode::Plain`] performs no TLS at all (`Tls::None`), for a local
/// test/dev relay -- the caller opted into cleartext explicitly.
///
/// Implicit-TLS and STARTTLS relays are built via `relay(host)` (which
/// defaults to TLS wrapping) / `starttls_relay(host)` respectively;
/// plaintext is built via the non-encrypting `builder_dangerous(host)`.
pub fn build_transport(
host: &str,
port: u16,
tls_mode: TlsMode,
username: &str,
password: &str,
) -> Result<AsyncSmtpTransport<Tokio1Executor>, MailError> {
Self::validate_smtp_config(host, port, username)?;
let creds = Credentials::new(username.to_string(), password.to_string());
let tls_params = TlsParameters::builder(host.to_string())
.build()
.map_err(|e| MailError::TransportFailed(format!("TLS parameters error: {}", e)))?;
let tls = if port == 465 {
Tls::Wrapper(tls_params)
} else {
Tls::Required(tls_params)
let builder = match tls_mode {
TlsMode::ImplicitTls | TlsMode::StartTls => {
let tls_params = TlsParameters::builder(host.to_string())
.build()
.map_err(|e| {
MailError::TransportFailed(format!("TLS parameters error: {}", e))
})?;
let tls = match tls_mode {
TlsMode::StartTls => Tls::Required(tls_params),
_ => Tls::Wrapper(tls_params),
};
AsyncSmtpTransport::<Tokio1Executor>::relay(host)
.map_err(|e| {
MailError::TransportFailed(format!("relay configuration error: {}", e))
})?
.tls(tls)
}
TlsMode::Plain => AsyncSmtpTransport::<Tokio1Executor>::builder_dangerous(host),
};
let transport = AsyncSmtpTransport::<Tokio1Executor>::relay(host)
.map_err(|e| MailError::TransportFailed(format!("relay configuration error: {}", e)))?
.port(port)
.credentials(creds)
.tls(tls)
.build();
let transport = builder.port(port).credentials(creds).build();
Ok(transport)
}
@@ -383,21 +431,59 @@ mod tests {
}
#[test]
fn build_transport_creates_starttls_client_for_port_587() {
let transport = SmtpTransportEngine::build_transport("smtp.nuncio.mx", 587, "user", "pass");
fn build_transport_creates_starttls_client() {
let transport = SmtpTransportEngine::build_transport(
"smtp.nuncio.mx",
587,
TlsMode::StartTls,
"user",
"pass",
);
assert!(transport.is_ok());
}
#[test]
fn build_transport_creates_implicit_tls_client_for_port_465() {
let transport = SmtpTransportEngine::build_transport("smtp.nuncio.mx", 465, "user", "pass");
fn build_transport_creates_implicit_tls_client() {
let transport = SmtpTransportEngine::build_transport(
"smtp.nuncio.mx",
465,
TlsMode::ImplicitTls,
"user",
"pass",
);
assert!(transport.is_ok());
}
#[test]
fn build_transport_creates_plaintext_client() {
let transport = SmtpTransportEngine::build_transport(
"smtp.nuncio.mx",
25,
TlsMode::Plain,
"user",
"pass",
);
assert!(transport.is_ok());
}
#[tokio::test]
async fn probe_connection_to_unreachable_server_fails() {
let engine = SmtpTransportEngine::new("127.0.0.1", 1, TlsMode::ImplicitTls, "user", "pass")
.expect("valid config");
let err = engine
.probe_connection()
.await
.expect_err("probe against port 1 must fail, never fabricate success");
assert!(matches!(
err,
MailError::TransportFailed(_) | MailError::SmtpFailed(_)
));
}
#[tokio::test]
async fn send_email_to_unreachable_server_fails_with_transport_error() {
let engine =
SmtpTransportEngine::new("127.0.0.1", 1, "user", "pass").expect("valid config");
let engine = SmtpTransportEngine::new("127.0.0.1", 1, TlsMode::ImplicitTls, "user", "pass")
.expect("valid config");
let email = sample_email();
let err = engine
.send_email(&email)
@@ -411,7 +497,8 @@ mod tests {
#[test]
fn engine_constructor_creates_instance() {
let engine = SmtpTransportEngine::new("smtp.nuncio.mx", 465, "user", "pass");
let engine =
SmtpTransportEngine::new("smtp.nuncio.mx", 465, TlsMode::ImplicitTls, "user", "pass");
assert!(engine.is_ok());
}
@@ -506,8 +593,8 @@ mod tests {
#[tokio::test]
async fn send_message_to_unreachable_server_fails_with_transport_error() {
let engine =
SmtpTransportEngine::new("127.0.0.1", 1, "user", "pass").expect("valid config");
let engine = SmtpTransportEngine::new("127.0.0.1", 1, TlsMode::ImplicitTls, "user", "pass")
.expect("valid config");
let message = sample_outbound_message();
let err = engine
.send_message(&message)
@@ -525,8 +612,8 @@ mod tests {
// through to the real `send_message` path, not a separate/fabricated
// stub -- exercised via the trait object
// exactly like production code (`nunciod::send`) uses it.
let engine =
SmtpTransportEngine::new("127.0.0.1", 1, "user", "pass").expect("valid config");
let engine = SmtpTransportEngine::new("127.0.0.1", 1, TlsMode::ImplicitTls, "user", "pass")
.expect("valid config");
let sender: &dyn MessageSender = &engine;
let message = sample_outbound_message();
let err = sender
+149 -1
View File
@@ -2,7 +2,7 @@
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use nuncio_mail::JmapEngine;
use nuncio_mail::{JmapEngine, MailBackend};
use serde_json::json;
use wiremock::matchers::{method, path};
use wiremock::{Mock, MockServer, ResponseTemplate};
@@ -89,3 +89,151 @@ async fn wiremock_jmap_session_discovery_and_email_get_sync() {
assert_eq!(emails[0].sender, "sender@nuncio.mx");
assert_eq!(new_state, "sync-state-500");
}
/// Drives `JmapEngine` through the `MailBackend` trait itself (rather than its static parse
/// helpers) against a wiremock server, proving `with_credentials` performs genuine session
/// discovery, `Mailbox/get`, `Email/query`, and `Email/get` HTTP round trips end to end -- the
/// exact seam `nunciod::sync::build_mail_backend` wires into production.
#[tokio::test]
async fn jmap_engine_sync_folders_and_messages_through_mail_backend_trait() {
let mock_server = MockServer::start().await;
let session_body = json!({
"username": "james.maes@kof22.com",
"primaryAccounts": {
"urn:ietf:params:jmap:mail": "acct-100"
},
"apiUrl": format!("{}/jmap/api", mock_server.uri()),
"state": "session-state-1"
});
Mock::given(method("GET"))
.and(path("/.well-known/jmap"))
.respond_with(ResponseTemplate::new(200).set_body_json(&session_body))
.mount(&mock_server)
.await;
let mailbox_get_body = json!({
"methodResponses": [
[
"Mailbox/get",
{
"accountId": "acct-100",
"list": [
{"id": "mb-inbox", "name": "Inbox", "totalEmails": 3, "unreadEmails": 1}
]
},
"c1"
]
]
});
Mock::given(method("POST"))
.and(path("/jmap/api"))
.and(wiremock::matchers::body_partial_json(json!({
"methodCalls": [["Mailbox/get", {}, "c1"]]
})))
.respond_with(ResponseTemplate::new(200).set_body_json(&mailbox_get_body))
.mount(&mock_server)
.await;
let email_query_body = json!({
"methodResponses": [
["Email/query", {"ids": ["msg-wm-200"]}, "c1"]
]
});
Mock::given(method("POST"))
.and(path("/jmap/api"))
.and(wiremock::matchers::body_partial_json(json!({
"methodCalls": [["Email/query", {}, "c1"]]
})))
.respond_with(ResponseTemplate::new(200).set_body_json(&email_query_body))
.mount(&mock_server)
.await;
let email_get_body = json!({
"methodResponses": [
[
"Email/get",
{
"accountId": "acct-100",
"state": "sync-state-900",
"list": [
{
"id": "msg-wm-200",
"subject": "Trait-Level JMAP Sync",
"from": [{ "email": "sender@nuncio.mx" }],
"to": [{ "email": "james.maes@kof22.com" }],
"receivedAt": 1700002000i64,
"isUnread": true,
"bodySnippet": "Fetched through the MailBackend trait."
}
]
},
"c1"
]
]
});
Mock::given(method("POST"))
.and(path("/jmap/api"))
.and(wiremock::matchers::body_partial_json(json!({
"methodCalls": [["Email/get", {}, "c1"]]
})))
.respond_with(ResponseTemplate::new(200).set_body_json(&email_get_body))
.mount(&mock_server)
.await;
let engine = JmapEngine::with_credentials(
"acct-100",
&mock_server.uri(),
"james.maes@kof22.com",
"wiremock-password",
);
let folders = engine.sync_folders().await.expect("sync_folders succeeds");
assert_eq!(folders.len(), 1);
assert_eq!(folders[0].id, "mb-inbox");
assert_eq!(folders[0].name, "Inbox");
assert_eq!(folders[0].total_messages, 3);
assert_eq!(folders[0].unread_messages, 1);
let (emails, new_state) = engine
.sync_messages("mb-inbox", None)
.await
.expect("sync_messages succeeds");
assert_eq!(emails.len(), 1);
assert_eq!(emails[0].id, "msg-wm-200");
assert_eq!(emails[0].subject, "Trait-Level JMAP Sync");
assert_eq!(emails[0].sender, "sender@nuncio.mx");
assert_eq!(emails[0].received_at, 1700002000);
assert!(!emails[0].read);
assert_eq!(
emails[0].body_plain,
Some("Fetched through the MailBackend trait.".to_string())
);
assert_eq!(new_state, "sync-state-900");
}
/// A session-discovery failure (non-2xx status) must surface as a genuine `MailError`, never a
/// silent fallback to canned data, once credentials are present.
#[tokio::test]
async fn jmap_engine_sync_folders_surfaces_session_discovery_failure() {
let mock_server = MockServer::start().await;
Mock::given(method("GET"))
.and(path("/.well-known/jmap"))
.respond_with(ResponseTemplate::new(500))
.mount(&mock_server)
.await;
let engine = JmapEngine::with_credentials(
"acct-100",
&mock_server.uri(),
"james.maes@kof22.com",
"wiremock-password",
);
let err = engine
.sync_folders()
.await
.expect_err("session discovery failure must not fall back to canned folders");
assert!(err.to_string().contains("network I/O error"));
}
Binary file not shown.
@@ -144,6 +144,30 @@ service Accounts {
// ListAccounts returns every persisted account configuration. Password
// credentials are NEVER included in this response.
rpc ListAccounts(ListAccountsRequest) returns (ListAccountsResponse);
// UpdateAccount overwrites an existing account's persisted configuration.
// The password is optional: when omitted the existing keyring credential
// is left untouched; when present it REPLACES the stored credential in the
// OS keyring vault (never written to SQLite, never echoed back). A password
// rotation that cannot be persisted is rolled back so the vault never
// diverges from the persisted account row.
rpc UpdateAccount(UpdateAccountRequest) returns (UpdateAccountResponse);
// RemoveAccount deletes a persisted account configuration by id and its
// associated keyring credential.
rpc RemoveAccount(RemoveAccountRequest) returns (RemoveAccountResponse);
// TestAccountConnection performs a real, bounded connection probe against
// the account's configured IMAP and SMTP endpoints (honoring each one's
// TLS transport mode) and reports the GENUINE per-protocol outcome -- never
// a fabricated success. The two legs verify DIFFERENT depths, and callers
// must not conflate them:
// - IMAP verifies a full LOGIN: connect, negotiate TLS/STARTTLS, and
// authenticate with the account's keyring credential.
// - SMTP verifies only TRANSPORT REACHABILITY: connect and negotiate
// TLS/STARTTLS. It does NOT perform an SMTP AUTH exchange, so `smtp_ok`
// does not assert the credential is valid for outbound mail (SMTP has
// no standard authentication probe that does not also submit a message).
// A probe that fails carries the real error detail; each leg is bounded by
// a timeout so the RPC cannot hang against an unreachable host.
rpc TestAccountConnection(TestAccountConnectionRequest) returns (TestAccountConnectionResponse);
}
// Mirrors `nuncio_core::AccountProtocol`.
@@ -220,6 +244,57 @@ message ListAccountsResponse {
repeated AccountConfig accounts = 1;
}
// UpdateAccountRequest carries the full replacement account configuration
// plus an OPTIONAL password. When `password` is absent the account's existing
// keyring credential is preserved; when present it is written to the OS
// keyring vault (under `config.keyring_secret_key`), exactly like
// `AddAccountRequest` -- never persisted to SQLite, never logged, never
// echoed back.
message UpdateAccountRequest {
AccountConfig config = 1;
optional string password = 2;
}
// UpdateAccountResponse deliberately carries no fields: a successful RPC
// response (as opposed to a `Status` error) is confirmation enough, and the
// caller already holds the updated config it just sent.
message UpdateAccountResponse {}
// RemoveAccountRequest names the account configuration to delete by id.
message RemoveAccountRequest {
string id = 1;
}
// RemoveAccountResponse currently carries no fields; a successful RPC
// response is confirmation enough.
message RemoveAccountResponse {}
// TestAccountConnectionRequest names the persisted account to probe by id.
message TestAccountConnectionRequest {
string id = 1;
}
// TestAccountConnectionResponse reports the REAL, independent outcome of
// probing each protocol endpoint -- never a fabricated success. `imap_error`
// / `smtp_error` are present only when the corresponding `*_ok` is false and
// carry the genuine failure detail (dial/handshake/auth error text). The two
// `*_ok` flags verify different depths -- see `TestAccountConnection`.
message TestAccountConnectionResponse {
// Whether the IMAP endpoint FULLY authenticated: connect, TLS/STARTTLS
// negotiation, AND login with the account's credential all succeeded.
bool imap_ok = 1;
// Whether the SMTP endpoint was REACHABLE: connect and TLS/STARTTLS
// negotiation succeeded. This is a transport-reachability check only -- it
// does NOT perform an SMTP AUTH exchange, so it does not assert the
// credential is valid for outbound mail.
bool smtp_ok = 2;
// Present only when `imap_ok` is false: the real IMAP login/handshake error.
optional string imap_error = 3;
// Present only when `smtp_ok` is false: the real SMTP transport/handshake
// error.
optional string smtp_error = 4;
}
// Mail exposes the READ path over the daemon's real, persistent message
// store (the data that syncs in via `DatabaseEngine::save_email`).
//
@@ -246,7 +321,11 @@ service Mail {
// SendMessage composes and sends a real outbound email over SMTP, built
// from the configured account's SMTP endpoint
// (`AccountConfig.smtp_host`/`smtp_port`) and its keyring-stored
// password. Returns successfully ONLY when
// password. When `account_id` is set, that specific account is used as
// the sender, and a request naming an account that does not exist fails
// rather than silently falling back to a different account. When
// `account_id` is omitted, the first configured account is used. Returns
// successfully ONLY when
// the SMTP transport genuinely accepted the message for delivery --
// never a fabricated success. A transport/authentication failure surfaces
// as a `Status::internal` (or `Status::invalid_argument` for a malformed
@@ -394,6 +473,10 @@ message SendMessageRequest {
optional string body_html = 5;
// Optional file attachments.
repeated Attachment attachments = 6;
// Optional identifier of the account to send from. When omitted, the
// first configured account is used (preserving prior behavior); when
// set, the named account must exist or the request fails.
optional string account_id = 7;
}
// SendMessageResponse. A successful response means the SMTP transport
@@ -458,6 +541,34 @@ service Filters {
// existing dry-run evaluation path. Never persists the rule and never
// executes any of its actions.
rpc PreviewRule(PreviewRuleRequest) returns (PreviewRuleResponse);
// UpdateRule applies partial overrides (name/nsql/priority) to an existing
// persisted rule, re-validates the resulting rule exactly as `CreateRule`
// does, and reloads the daemon's live `FilterEngine` rule set so the
// change takes effect immediately. Rejects an unknown `id` with
// `Status::not_found` and an invalid resulting rule with
// `Status::invalid_argument` -- nothing is persisted on either failure.
rpc UpdateRule(UpdateRuleRequest) returns (UpdateRuleResponse);
// ExportRules renders every persisted filter rule in the requested
// `format` ("sql" or "json") and returns the rendered text inline --
// small enough, unlike a mailbox export, to carry directly over the RPC
// rather than writing to a shared filesystem path.
rpc ExportRules(ExportRulesRequest) returns (ExportRulesResponse);
// ImportRules parses each non-empty, non-comment line of `content` as an
// independent NSQL rule definition and persists the ones that parse.
// Reports both the count of rules actually persisted and the parse error
// for every line that failed, rather than silently dropping failures.
rpc ImportRules(ImportRulesRequest) returns (ImportRulesResponse);
// GetExecutionLogs returns the most recent persisted filter execution
// log entries (hash-chain ledger), newest-eligible-first as stored,
// capped at `limit`.
rpc GetExecutionLogs(GetExecutionLogsRequest) returns (GetExecutionLogsResponse);
// Triage scans every stored message in bounded chunks, evaluating the
// daemon's live `FilterEngine` against each one and applying the same
// actions live sync would -- so the currently active rule set can be
// applied retroactively to mail that was synced before those rules
// existed. Streams cumulative progress as it works, ending with a final
// `done: true` message once the whole store has been scanned.
rpc Triage(TriageRequest) returns (stream TriageProgress);
}
// FilterRule mirrors `nuncio_filter::FilterRule`'s persisted/user-facing
@@ -577,6 +688,113 @@ message PreviewRuleResponse {
repeated string condition_traces = 7;
}
// UpdateRuleRequest carries a persisted rule's `id` plus optional
// per-field overrides; an absent field leaves that field unchanged on the
// existing rule.
message UpdateRuleRequest {
// Rule unique identifier.
string id = 1;
// New display name, if changing.
optional string name = 2;
// New full NSQL source code text, if changing.
optional string nsql = 3;
// New evaluation order, if changing.
optional int32 priority = 4;
}
// UpdateRuleResponse carries the rule as persisted after applying the
// requested overrides.
message UpdateRuleResponse {
FilterRule rule = 1;
}
// ExportRulesRequest names the rendering format for `ExportRules`.
message ExportRulesRequest {
// "sql" or "json".
string format = 1;
}
// ExportRulesResponse carries every persisted rule pre-rendered as a single
// text blob in the requested format.
message ExportRulesResponse {
string content = 1;
}
// ImportRulesRequest carries raw rule-definition text to parse and persist,
// one NSQL rule definition per non-empty, non-comment (`--`-prefixed) line.
message ImportRulesRequest {
string content = 1;
}
// ImportRulesResponse reports how many lines parsed and persisted
// successfully, and the parse error for every line that did not.
message ImportRulesResponse {
// Count of rule definitions successfully parsed and persisted.
int32 imported_count = 1;
// Parse error message for each line that failed, in encounter order.
repeated string errors = 2;
}
// GetExecutionLogsRequest caps the number of log entries returned.
message GetExecutionLogsRequest {
uint32 limit = 1;
}
// GetExecutionLogsResponse.
message GetExecutionLogsResponse {
repeated FilterExecutionLog logs = 1;
}
// FilterExecutionLog mirrors `nuncio_filter::FilterExecutionLog`
// field-for-field -- the hash-chain ledger entry recorded each time a
// persisted rule matches and acts on a message.
message FilterExecutionLog {
// Log entry primary key.
int64 id = 1;
// Rule ID executed.
string rule_id = 2;
// Email message ID evaluated.
string message_id = 3;
// Action description string.
string action_taken = 4;
// Timestamp of execution (unix seconds).
int64 matched_at = 5;
// Hash of the preceding log record in the hash-chain ledger.
string prev_hash = 6;
// HMAC-SHA256 hash of this record.
string hash = 7;
}
// TriageRequest configures a retroactive rescan of the whole message store.
message TriageRequest {
// Scope the scan to a single rule's actions. Currently NOT honored --
// `Triage` always evaluates a message against the full live rule set,
// matching what a real (non-retroactive) sync run would do. Reserved for
// forward compatibility; a client that sets this today gets a whole-store
// scan against all active rules, not an error.
optional string rule_id = 1;
// Number of messages fetched per keyset page. 0 selects the server
// default (100).
uint32 chunk_size = 2;
}
// TriageProgress reports the real, cumulative state of an in-progress (or
// just-finished) `Triage` scan. Every field is the true running total as of
// the chunk just processed -- never an estimate or a fabricated count.
message TriageProgress {
// Total messages scanned so far.
uint64 scanned_count = 1;
// Total messages that had at least one action applied or enqueued.
uint64 matched_count = 2;
// Total individual actions applied or enqueued across all messages.
uint64 actions_applied_count = 3;
// ID of the last message scanned in the most recently processed chunk.
string last_message_id = 4;
// True only on the final message of the stream, once the whole store has
// been scanned.
bool done = 5;
}
// Export exposes the daemon's real mailbox export engine
// (`nuncio_core::export`, via `DatabaseEngine::export_messages_to_file`)
// over authenticated gRPC. The daemon and its caller run on the SAME host
@@ -644,6 +862,243 @@ message ExportResponse {
uint64 bytes_written = 3;
}
// Calendar exposes the READ path over the daemon's real, persistent calendar
// event store (`DatabaseEngine::save_calendar_event`/`list_calendar_events`/
// `get_calendar_event`), plus a real `Sync` that fetches events from a
// `CalendarBackend` and persists them.
//
// This is an honest surface: there is currently no way to persist a
// per-account CalDAV collection URL/credential, so `Sync` in production (no
// injected test backend) has no real CalDAV connection to make and returns
// an error rather than a fabricated success. There are deliberately no
// create/delete RPCs here -- this service does not add real CalDAV write
// support.
//
// SECURITY: this service MUST be mounted behind the SAME
// `BearerAuthInterceptor` that guards every other service `nunciod` exposes
// -- no exceptions carved out for "just one more" unauthenticated service.
// See `nunciod::grpc::serve_on_listener`.
service Calendar {
// Sync fetches events for `calendar_id` within `[start_window, end_window]`
// from the account's calendar backend and persists them via the store.
// With no per-account CalDAV configuration available in production, this
// returns `Status::internal` describing that gap unless a test double has
// been injected -- never a fabricated `synced_count`.
rpc Sync(CalendarSyncRequest) returns (CalendarSyncResponse);
// ListEvents returns persisted events for `account_id` whose window
// overlaps `[start_window, end_window]`, ordered by start time -- always
// genuinely persisted data, never fabricated.
rpc ListEvents(ListEventsRequest) returns (ListEventsResponse);
// GetEvent returns a single persisted event by ID.
rpc GetEvent(GetEventRequest) returns (GetEventResponse);
}
// Mirrors `nuncio_core::model::CalendarEvent` field-for-field. Named
// `CalendarEvent` (matching the core model) rather than `Event` -- `Event` is
// already taken by `System`'s push-event wrapper.
message CalendarEvent {
// Event identifier.
string id = 1;
// Account identifier owning the event.
string account_id = 2;
// Calendar collection identifier.
string calendar_id = 3;
// Event title or summary.
string summary = 4;
// Unix timestamp for event start.
int64 start_time = 5;
// Unix timestamp for event end.
int64 end_time = 6;
// Recurrence rule string (RFC 5545 RRULE format).
optional string rrule = 7;
// Location text.
optional string location = 8;
}
// CalendarSyncRequest carries the calendar collection and time window to
// sync, since `CalendarBackend::fetch_events` is inherently scoped that way
// (unlike `Mail`'s account-wide `Sync`).
message CalendarSyncRequest {
string account_id = 1;
string calendar_id = 2;
int64 start_window = 3;
int64 end_window = 4;
}
// CalendarSyncResponse reports how many events were synchronized.
message CalendarSyncResponse {
uint64 synced_count = 1;
}
// ListEventsRequest.
message ListEventsRequest {
string account_id = 1;
string calendar_id = 2;
int64 start_window = 3;
int64 end_window = 4;
}
// ListEventsResponse.
message ListEventsResponse {
repeated CalendarEvent events = 1;
}
// GetEventRequest.
message GetEventRequest {
string event_id = 1;
}
// GetEventResponse.
message GetEventResponse {
CalendarEvent event = 1;
}
// Contacts exposes the READ path over the daemon's real, persistent contact
// store (`DatabaseEngine::save_contact`/`list_contacts`/`get_contact`), plus
// a real `Sync` that fetches contacts from a `ContactsBackend` and persists
// them, and a `CreateContact` that persists a locally-authored contact
// directly to that same store.
//
// This is an honest surface: there is currently no way to persist a
// per-account CardDAV address book collection URL/credential, so `Sync` in
// production (no injected test backend) has no real CardDAV connection to
// make and returns an error rather than a fabricated success.
// `CreateContact` is NOT a CardDAV write-back -- it only ever writes to the
// daemon's own local store, the same way the address book has always worked
// here, just now actually persisted through the daemon's real store instead
// of a disposable per-process in-memory database. There is deliberately no
// RPC to delete a contact or to push a locally-authored contact back to a
// remote CardDAV server.
//
// SECURITY: this service MUST be mounted behind the SAME
// `BearerAuthInterceptor` that guards every other service `nunciod` exposes
// -- no exceptions carved out for "just one more" unauthenticated service.
// See `nunciod::grpc::serve_on_listener`.
service Contacts {
// Sync fetches contacts for `account_id` from the account's contacts
// backend and persists them via the store. With no per-account CardDAV
// configuration available in production, this returns `Status::internal`
// describing that gap unless a test double has been injected -- never a
// fabricated `synced_count`.
rpc Sync(ContactsSyncRequest) returns (ContactsSyncResponse);
// ListContacts returns persisted contacts for `account_id`, ordered by
// interaction count then display name -- always genuinely persisted data,
// never fabricated.
rpc ListContacts(ListContactsRequest) returns (ListContactsResponse);
// GetContact returns a single persisted contact by ID.
rpc GetContact(GetContactRequest) returns (GetContactResponse);
// CreateContact persists a new, locally-authored contact directly to the
// daemon's store. This is real local persistence, not a fabricated CardDAV
// write-back -- the created contact is never pushed to a remote server.
rpc CreateContact(CreateContactRequest) returns (CreateContactResponse);
}
// Mirrors `nuncio_contacts::models::ContactEmail`.
message ContactEmail {
// Email address.
string email = 1;
// Address label (e.g. "work", "home", "other").
string label = 2;
// Whether this is the contact's primary email address.
bool is_primary = 3;
}
// Mirrors `nuncio_contacts::models::ContactPhone`.
message ContactPhone {
// Phone number.
string phone = 1;
// Number label (e.g. "mobile", "work", "home").
string label = 2;
// Whether this is the contact's primary phone number.
bool is_primary = 3;
}
// Mirrors `nuncio_contacts::models::Contact` field-for-field, except the
// internal bookkeeping timestamps (`created_at`/`updated_at`/
// `last_interacted_at`) are deliberately omitted -- no real client use case
// for them was identified for this vertical; they can be added additively
// later if one arises.
message Contact {
// Contact unique identifier.
string id = 1;
// Account identifier owning this contact's address book entry.
optional string account_id = 2;
// Display name.
string display_name = 3;
// Given (first) name.
optional string given_name = 4;
// Family (last) name.
optional string family_name = 5;
// Organization / company name.
optional string organization = 6;
// Job title.
optional string job_title = 7;
// Free-form notes.
optional string notes = 8;
// Avatar image URL.
optional string avatar_url = 9;
// Email addresses.
repeated ContactEmail emails = 10;
// Phone numbers.
repeated ContactPhone phones = 11;
// Favorite/starred flag.
bool is_favorite = 12;
// Number of recorded interactions with this contact.
uint64 interaction_count = 13;
}
// ContactsSyncRequest mirrors `SyncRequest`'s shape (unlike Calendar's
// collection-scoped sync) since `ContactsBackend::fetch_contacts` is
// account-scoped, not collection-scoped.
message ContactsSyncRequest {
string account_id = 1;
}
// ContactsSyncResponse reports how many contacts were synchronized.
message ContactsSyncResponse {
uint64 synced_count = 1;
}
// ListContactsRequest. Contacts are always scoped to the account whose
// address book they belong to, mirroring the store's `list_contacts`
// signature.
message ListContactsRequest {
string account_id = 1;
}
// ListContactsResponse.
message ListContactsResponse {
repeated Contact contacts = 1;
}
// GetContactRequest.
message GetContactRequest {
string contact_id = 1;
}
// GetContactResponse.
message GetContactResponse {
Contact contact = 1;
}
// CreateContactRequest carries the fields needed to author a brand-new local
// contact. `emails`/`phones` follow `Contact`'s shape so multiple addresses
// could be supplied, though the CLI's `contact add` command only ever sends
// a single primary email today.
message CreateContactRequest {
string account_id = 1;
string display_name = 2;
optional string organization = 3;
repeated ContactEmail emails = 4;
repeated ContactPhone phones = 5;
}
// CreateContactResponse carries the persisted contact, including its
// daemon-assigned `id`.
message CreateContactResponse {
Contact contact = 1;
}
// Audit exposes the daemon's real WORM (Write Once, Read Many) HMAC-chained
// tamper-evident audit ledger (`nuncio_core::worm_audit`, via
// `DatabaseEngine`'s `worm_audit_records` table) over authenticated gRPC.
+86
View File
@@ -11,6 +11,8 @@
use crate::v1::accounts_client::AccountsClient;
use crate::v1::audit_client::AuditClient;
use crate::v1::calendar_client::CalendarClient;
use crate::v1::contacts_client::ContactsClient;
use crate::v1::export_client::ExportClient;
use crate::v1::filters_client::FiltersClient;
use crate::v1::mail_client::MailClient;
@@ -112,6 +114,16 @@ pub type AuthenticatedExportClient =
pub type AuthenticatedAuditClient =
AuditClient<InterceptedService<Channel, BearerTokenInterceptor>>;
/// The authenticated `nuncio.v1.Calendar` client type returned by
/// [`connect_calendar`].
pub type AuthenticatedCalendarClient =
CalendarClient<InterceptedService<Channel, BearerTokenInterceptor>>;
/// The authenticated `nuncio.v1.Contacts` client type returned by
/// [`connect_contacts`].
pub type AuthenticatedContactsClient =
ContactsClient<InterceptedService<Channel, BearerTokenInterceptor>>;
/// Dials `addr` (a `host:port` pair, e.g. `127.0.0.1:9420`) over plain HTTP
/// (the loopback gRPC transport is never TLS-wrapped; auth is via bearer
/// token instead) and returns the connected [`Channel`], shared by every
@@ -238,6 +250,46 @@ pub async fn connect_audit(
Ok(AuditClient::with_interceptor(channel, interceptor))
}
/// Dials the `nuncio.v1.Calendar` gRPC endpoint at `addr` and returns a
/// client that injects `authorization: Bearer <token>` metadata on every
/// call.
///
/// `Calendar` is guarded by the exact same `BearerAuthInterceptor` as every
/// other service on the server side (see
/// `nunciod::grpc::serve_on_listener_with_overrides`), so this shares
/// [`BearerTokenInterceptor`] and [`dial`] with [`connect_system`] /
/// [`connect_accounts`] / [`connect_mail`] / [`connect_filters`] /
/// [`connect_export`] / [`connect_audit`] rather than hand-rolling yet
/// another auth handshake.
pub async fn connect_calendar(
addr: &str,
token: &str,
) -> Result<AuthenticatedCalendarClient, ConnectError> {
let interceptor = BearerTokenInterceptor::new(token)?;
let channel = dial(addr).await?;
Ok(CalendarClient::with_interceptor(channel, interceptor))
}
/// Dials the `nuncio.v1.Contacts` gRPC endpoint at `addr` and returns a
/// client that injects `authorization: Bearer <token>` metadata on every
/// call.
///
/// `Contacts` is guarded by the exact same `BearerAuthInterceptor` as every
/// other service on the server side (see
/// `nunciod::grpc::serve_on_listener_with_overrides`), so this shares
/// [`BearerTokenInterceptor`] and [`dial`] with [`connect_system`] /
/// [`connect_accounts`] / [`connect_mail`] / [`connect_filters`] /
/// [`connect_export`] / [`connect_audit`] / [`connect_calendar`] rather than
/// hand-rolling yet another auth handshake.
pub async fn connect_contacts(
addr: &str,
token: &str,
) -> Result<AuthenticatedContactsClient, ConnectError> {
let interceptor = BearerTokenInterceptor::new(token)?;
let channel = dial(addr).await?;
Ok(ContactsClient::with_interceptor(channel, interceptor))
}
/// Opens the `nuncio.v1.System/Subscribe` server-streaming RPC on an already
/// authenticated client (as returned by [`connect_system`]), reusing the
/// bearer-token interceptor already attached to `client` rather than
@@ -396,6 +448,40 @@ mod tests {
assert!(err.to_string().contains("127.0.0.1:1"));
}
#[tokio::test]
async fn connect_calendar_fails_closed_on_invalid_token() {
let err = connect_calendar("127.0.0.1:0", "tok\ntoken")
.await
.expect_err("invalid token must fail before dialing");
assert!(matches!(err, ConnectError::InvalidToken(_)));
}
#[tokio::test]
async fn connect_calendar_reports_transport_error_when_daemon_unreachable() {
let err = connect_calendar("127.0.0.1:1", "abc123")
.await
.expect_err("connecting to an unreachable daemon must fail");
assert!(matches!(err, ConnectError::Transport { .. }));
assert!(err.to_string().contains("127.0.0.1:1"));
}
#[tokio::test]
async fn connect_contacts_fails_closed_on_invalid_token() {
let err = connect_contacts("127.0.0.1:0", "tok\ntoken")
.await
.expect_err("invalid token must fail before dialing");
assert!(matches!(err, ConnectError::InvalidToken(_)));
}
#[tokio::test]
async fn connect_contacts_reports_transport_error_when_daemon_unreachable() {
let err = connect_contacts("127.0.0.1:1", "abc123")
.await
.expect_err("connecting to an unreachable daemon must fail");
assert!(matches!(err, ConnectError::Transport { .. }));
assert!(err.to_string().contains("127.0.0.1:1"));
}
#[tokio::test]
async fn connect_system_reports_invalid_endpoint_for_a_malformed_address() {
// An embedded space is not a legal URI authority character, so
+1
View File
@@ -12,6 +12,7 @@ workspace = true
[dependencies]
nuncio-core = { path = "../nuncio-core" }
nuncio-filter = { path = "../nuncio-filter" }
nuncio-contacts = { path = "../nuncio-contacts" }
tokio.workspace = true
sqlx.workspace = true
serde.workspace = true
+701 -6
View File
@@ -67,6 +67,17 @@ impl DatabaseError {
_ => false,
}
}
/// Returns true if this error means "no row matched the lookup" (e.g. a
/// `get_message`/`get_contact`/`get_calendar_event` call for an id that
/// was never saved), as opposed to a genuine I/O, corruption, or key
/// provisioning failure.
///
/// Exposed so callers outside this crate can distinguish "not found" from
/// other failures without depending on `sqlx` directly.
pub fn is_not_found(&self) -> bool {
matches!(self, DatabaseError::Query(sqlx::Error::RowNotFound))
}
}
/// SQLite database storage engine managing WAL connection pools and migrations.
@@ -115,6 +126,52 @@ fn resolve_engine_keys(
Ok((storage_key, worm_key, ledger_key))
}
/// Reconstruct a [`nuncio_contacts::Contact`] from a `contacts` table row.
///
/// `created_at`/`updated_at`/`last_interacted_at` are stored as RFC 3339 strings written by
/// [`DatabaseEngine::save_contact`] itself, so a parse failure here would indicate corrupted
/// storage rather than bad input; falling back to "now" for that edge case (rather than
/// propagating a parse error through every read) mirrors the tolerant `unwrap_or_default`
/// JSON-decode style already used for `emails_json`/`phones_json` below.
fn contact_from_row(row: &sqlx::sqlite::SqliteRow) -> nuncio_contacts::Contact {
let emails_json: String = row.get("emails_json");
let phones_json: String = row.get("phones_json");
let emails: Vec<nuncio_contacts::ContactEmail> =
serde_json::from_str(&emails_json).unwrap_or_default();
let phones: Vec<nuncio_contacts::ContactPhone> =
serde_json::from_str(&phones_json).unwrap_or_default();
let is_favorite: i64 = row.get("is_favorite");
let interaction_count: i64 = row.get("interaction_count");
let last_interacted_at: Option<String> = row.get("last_interacted_at");
let created_at: String = row.get("created_at");
let updated_at: String = row.get("updated_at");
let parse_rfc3339 = |s: &str| -> chrono::DateTime<chrono::Utc> {
chrono::DateTime::parse_from_rfc3339(s)
.map(|dt| dt.with_timezone(&chrono::Utc))
.unwrap_or_else(|_| chrono::Utc::now())
};
nuncio_contacts::Contact {
id: row.get("id"),
account_id: row.get("account_id"),
display_name: row.get("display_name"),
given_name: row.get("given_name"),
family_name: row.get("family_name"),
organization: row.get("organization"),
job_title: row.get("job_title"),
notes: row.get("notes"),
avatar_url: row.get("avatar_url"),
emails,
phones,
is_favorite: is_favorite != 0,
interaction_count: interaction_count as u64,
last_interacted_at: last_interacted_at.as_deref().map(parse_rfc3339),
created_at: parse_rfc3339(&created_at),
updated_at: parse_rfc3339(&updated_at),
}
}
/// Structured result of re-verifying the entire persisted WORM audit
/// ledger's hash chain. See
/// [`DatabaseEngine::verify_worm_audit_chain_report`].
@@ -129,6 +186,29 @@ pub struct WormChainReport {
pub first_broken_seq: Option<u64>,
}
/// Serialize a [`nuncio_core::TlsMode`] to its stable on-disk `accounts`
/// column form. Kept as a bare snake_case token (matching the column's
/// `'implicit_tls'` default) rather than JSON, so a row inserted by the
/// migration default and one written here are byte-identical.
fn tls_mode_to_db(mode: nuncio_core::TlsMode) -> &'static str {
match mode {
nuncio_core::TlsMode::ImplicitTls => "implicit_tls",
nuncio_core::TlsMode::StartTls => "start_tls",
nuncio_core::TlsMode::Plain => "plain",
}
}
/// Parse a persisted `accounts` TLS-mode column back into
/// [`nuncio_core::TlsMode`]. An unrecognized value falls back to the safe
/// default (implicit TLS) rather than failing the whole account load.
fn tls_mode_from_db(value: &str) -> nuncio_core::TlsMode {
match value {
"start_tls" => nuncio_core::TlsMode::StartTls,
"plain" => nuncio_core::TlsMode::Plain,
_ => nuncio_core::TlsMode::ImplicitTls,
}
}
impl DatabaseEngine {
/// Maximum concurrent read/write pool size.
pub const MAX_CONNECTIONS: u32 = 16;
@@ -382,7 +462,9 @@ impl DatabaseEngine {
keyring_secret_key TEXT NOT NULL,
sync_interval_secs INTEGER NOT NULL,
smtp_host TEXT,
smtp_port INTEGER
smtp_port INTEGER,
imap_tls_mode TEXT NOT NULL DEFAULT 'implicit_tls',
smtp_tls_mode TEXT NOT NULL DEFAULT 'implicit_tls'
);
CREATE TABLE IF NOT EXISTS filter_rules (
@@ -519,6 +601,67 @@ impl DatabaseEngine {
INSERT INTO events_fts(id, summary, location)
VALUES (new.id, new.summary, COALESCE(new.location, ''));
END;
-- Contacts (nuncio_contacts::Contact). Emails/phones are stored as JSON columns
-- (mirroring the schema already proven in nuncio-contacts's own, now-superseded
-- ContactsDatabase) rather than normalized child tables, since a contact's email/
-- phone list is always read and written as a whole with its parent record. Like
-- calendar summary/location, none of this is encrypted at rest, so trigger-based
-- FTS mirroring introduces no confidentiality regression.
CREATE TABLE IF NOT EXISTS contacts (
id TEXT PRIMARY KEY NOT NULL,
account_id TEXT,
display_name TEXT NOT NULL,
given_name TEXT,
family_name TEXT,
organization TEXT,
job_title TEXT,
notes TEXT,
avatar_url TEXT,
emails_json TEXT NOT NULL DEFAULT '[]',
phones_json TEXT NOT NULL DEFAULT '[]',
is_favorite INTEGER NOT NULL DEFAULT 0,
interaction_count INTEGER NOT NULL DEFAULT 0,
last_interacted_at TEXT,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
);
CREATE VIRTUAL TABLE IF NOT EXISTS contacts_fts USING fts5(
id UNINDEXED,
display_name,
organization,
emails_json,
tokenize = 'trigram'
);
CREATE TRIGGER IF NOT EXISTS contacts_ai AFTER INSERT ON contacts BEGIN
INSERT INTO contacts_fts(id, display_name, organization, emails_json)
VALUES (new.id, new.display_name, COALESCE(new.organization, ''), new.emails_json);
END;
CREATE TRIGGER IF NOT EXISTS contacts_ad AFTER DELETE ON contacts BEGIN
DELETE FROM contacts_fts WHERE id = old.id;
END;
CREATE TRIGGER IF NOT EXISTS contacts_au AFTER UPDATE ON contacts BEGIN
DELETE FROM contacts_fts WHERE id = old.id;
INSERT INTO contacts_fts(id, display_name, organization, emails_json)
VALUES (new.id, new.display_name, COALESCE(new.organization, ''), new.emails_json);
END;
-- Per-folder inbound-sync checkpoint. `state` is an opaque,
-- protocol-defined resumption token (an IMAP UID boundary such as
-- UIDNEXT, a JMAP state string) that a backend hands back after a
-- sync and expects to receive on the next sync so it can fetch only
-- what changed since. Keyed by (account, folder) because the token
-- is meaningful only within a single mailbox on a single account.
CREATE TABLE IF NOT EXISTS folder_sync_state (
account_id TEXT NOT NULL,
folder_id TEXT NOT NULL,
state TEXT NOT NULL,
PRIMARY KEY (account_id, folder_id)
);
"#,
)
.execute(&self.pool)
@@ -526,11 +669,53 @@ impl DatabaseEngine {
.map_err(DatabaseError::Query)?;
self.ensure_accounts_smtp_columns().await?;
self.ensure_accounts_tls_mode_columns().await?;
self.backfill_message_fts().await?;
Ok(())
}
/// Additive, backfill-safe migration that adds the `imap_tls_mode` /
/// `smtp_tls_mode` columns to a pre-existing `accounts` table that
/// predates persisted per-protocol TLS transport modes.
///
/// A fresh database already gets these columns from `CREATE TABLE IF NOT
/// EXISTS accounts` above, so on a fresh database this is a no-op. For a
/// pre-existing database file the columns are added with a
/// `'implicit_tls'` default, so every existing row keeps loading
/// successfully and retains the historical behavior (implicit TLS) it was
/// created under. SQLite has no `ADD COLUMN IF NOT EXISTS`, so column
/// presence is checked explicitly via `PRAGMA table_info` first, making
/// this safe to run on every daemon startup.
async fn ensure_accounts_tls_mode_columns(&self) -> Result<(), DatabaseError> {
let existing_columns: Vec<String> = sqlx::query("PRAGMA table_info(accounts)")
.fetch_all(&self.pool)
.await
.map_err(DatabaseError::Query)?
.iter()
.map(|row| row.get::<String, _>("name"))
.collect();
if !existing_columns.iter().any(|c| c == "imap_tls_mode") {
sqlx::query(
"ALTER TABLE accounts ADD COLUMN imap_tls_mode TEXT NOT NULL DEFAULT 'implicit_tls'",
)
.execute(&self.pool)
.await
.map_err(DatabaseError::Query)?;
}
if !existing_columns.iter().any(|c| c == "smtp_tls_mode") {
sqlx::query(
"ALTER TABLE accounts ADD COLUMN smtp_tls_mode TEXT NOT NULL DEFAULT 'implicit_tls'",
)
.execute(&self.pool)
.await
.map_err(DatabaseError::Query)?;
}
Ok(())
}
/// Additive, backfill-safe migration that adds the
/// `smtp_host` / `smtp_port` columns to a pre-existing `accounts` table
/// that predates the SMTP account-endpoint feature.
@@ -620,8 +805,8 @@ impl DatabaseEngine {
sqlx::query(
r#"
INSERT OR REPLACE INTO accounts
(id, name, email_address, protocol, server_host, server_port, use_tls, keyring_secret_key, sync_interval_secs, smtp_host, smtp_port)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
(id, name, email_address, protocol, server_host, server_port, use_tls, keyring_secret_key, sync_interval_secs, smtp_host, smtp_port, imap_tls_mode, smtp_tls_mode)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
"#,
)
.bind(&config.id)
@@ -635,6 +820,8 @@ impl DatabaseEngine {
.bind(config.sync_interval_secs as i64)
.bind(&config.smtp_host)
.bind(config.smtp_port as i64)
.bind(tls_mode_to_db(config.imap_tls_mode))
.bind(tls_mode_to_db(config.smtp_tls_mode))
.execute(&self.pool)
.await
.map_err(DatabaseError::Query)?;
@@ -642,6 +829,27 @@ impl DatabaseEngine {
Ok(())
}
/// Fetch a single persisted [`nuncio_core::AccountConfig`] by id, or
/// `None` if no such account row exists.
pub async fn get_account(
&self,
id: &str,
) -> Result<Option<nuncio_core::AccountConfig>, DatabaseError> {
Ok(self.list_accounts().await?.into_iter().find(|a| a.id == id))
}
/// Delete a persisted account configuration by id. Idempotent: deleting a
/// non-existent id is not an error (the post-condition -- "no account row
/// with this id" -- already holds).
pub async fn delete_account(&self, id: &str) -> Result<(), DatabaseError> {
sqlx::query("DELETE FROM accounts WHERE id = ?")
.bind(id)
.execute(&self.pool)
.await
.map_err(DatabaseError::Query)?;
Ok(())
}
/// Query all saved accounts in SQLite.
#[allow(clippy::type_complexity)]
pub async fn list_accounts(&self) -> Result<Vec<nuncio_core::AccountConfig>, DatabaseError> {
@@ -657,9 +865,11 @@ impl DatabaseEngine {
i64,
Option<String>,
Option<i64>,
String,
String,
)> = sqlx::query_as(
r#"
SELECT id, name, email_address, protocol, server_host, server_port, use_tls, keyring_secret_key, sync_interval_secs, smtp_host, smtp_port
SELECT id, name, email_address, protocol, server_host, server_port, use_tls, keyring_secret_key, sync_interval_secs, smtp_host, smtp_port, imap_tls_mode, smtp_tls_mode
FROM accounts
"#,
)
@@ -682,6 +892,8 @@ impl DatabaseEngine {
sync_interval_secs,
smtp_host,
smtp_port,
imap_tls_mode,
smtp_tls_mode,
)| {
let protocol = serde_json::from_str(&protocol_str)
.unwrap_or(nuncio_core::AccountProtocol::ImapSmtp);
@@ -706,8 +918,8 @@ impl DatabaseEngine {
smtp_host: resolved_smtp_host,
smtp_port: resolved_smtp_port,
use_tls: use_tls != 0,
imap_tls_mode: nuncio_core::TlsMode::ImplicitTls,
smtp_tls_mode: nuncio_core::TlsMode::ImplicitTls,
imap_tls_mode: tls_mode_from_db(&imap_tls_mode),
smtp_tls_mode: tls_mode_from_db(&smtp_tls_mode),
keyring_secret_key,
sync_interval_secs: sync_interval_secs as u64,
}
@@ -716,6 +928,51 @@ impl DatabaseEngine {
.collect())
}
/// Fetch the persisted inbound-sync checkpoint for a folder, or `None` if
/// this account/folder pair has never completed a sync. A `None` result is
/// the signal to the backend that it must perform a full first-time fetch.
pub async fn get_folder_sync_state(
&self,
account_id: &str,
folder_id: &str,
) -> Result<Option<String>, DatabaseError> {
let row: Option<(String,)> = sqlx::query_as(
"SELECT state FROM folder_sync_state WHERE account_id = ? AND folder_id = ?",
)
.bind(account_id)
.bind(folder_id)
.fetch_optional(&self.pool)
.await
.map_err(DatabaseError::Query)?;
Ok(row.map(|(state,)| state))
}
/// Persist the inbound-sync checkpoint a backend returned for a folder,
/// overwriting any prior value (a checkpoint is a running high-water mark,
/// not history). Written only after a folder's messages have been fetched
/// and persisted, so the stored checkpoint can never advance past work that
/// actually landed in the store.
pub async fn save_folder_sync_state(
&self,
account_id: &str,
folder_id: &str,
state: &str,
) -> Result<(), DatabaseError> {
sqlx::query(
r#"
INSERT OR REPLACE INTO folder_sync_state (account_id, folder_id, state)
VALUES (?, ?, ?)
"#,
)
.bind(account_id)
.bind(folder_id)
.bind(state)
.execute(&self.pool)
.await
.map_err(DatabaseError::Query)?;
Ok(())
}
/// Save an [`nuncio_core::model::Email`] to SQLite (INSERT OR REPLACE).
///
/// The message body is encrypted (AES-256-GCM) before being written to the `messages`
@@ -995,6 +1252,58 @@ impl DatabaseEngine {
.collect())
}
/// Query every persisted recurring calendar event (non-null, non-empty `rrule`) for
/// `account_id`, with **no** time-window filter -- a recurring master's own stored
/// `start_time`/`end_time` can validly sit long before (or after) any given query
/// window while its expanded occurrences still fall inside it, so callers that need
/// to materialize occurrences for a window must expand every one of the account's
/// recurring masters rather than pre-filtering by the master row's own timestamps.
#[allow(clippy::type_complexity)]
pub async fn list_recurring_calendar_events(
&self,
account_id: &str,
) -> Result<Vec<nuncio_core::model::CalendarEvent>, DatabaseError> {
let rows: Vec<(
String,
String,
String,
String,
i64,
i64,
Option<String>,
Option<String>,
)> = sqlx::query_as(
r#"
SELECT id, account_id, calendar_id, summary, start_time, end_time, rrule, location
FROM calendar_events
WHERE account_id = ? AND rrule IS NOT NULL AND rrule != ''
ORDER BY start_time ASC
"#,
)
.bind(account_id)
.fetch_all(&self.pool)
.await
.map_err(DatabaseError::Query)?;
Ok(rows
.into_iter()
.map(
|(id, account_id, calendar_id, summary, start_time, end_time, rrule, location)| {
nuncio_core::model::CalendarEvent {
id,
account_id,
calendar_id,
summary,
start_time,
end_time,
rrule,
location,
}
},
)
.collect())
}
/// Retrieve a single calendar event by ID.
#[allow(clippy::type_complexity)]
pub async fn get_calendar_event(
@@ -1034,6 +1343,103 @@ impl DatabaseEngine {
})
}
/// Save a [`nuncio_contacts::Contact`] to SQLite (INSERT OR REPLACE).
///
/// `nuncio-store` is otherwise limited to depending only on `nuncio-core` domain types;
/// this method (and [`Self::list_contacts`]/[`Self::get_contact`]) is a deliberate,
/// narrow exception, taking a direct dependency on `nuncio_contacts::Contact` because that
/// type has not (yet) been promoted to `nuncio-core` the way `CalendarEvent`/`Email` have.
pub async fn save_contact(
&self,
contact: &nuncio_contacts::Contact,
) -> Result<(), DatabaseError> {
let emails_json =
serde_json::to_string(&contact.emails).unwrap_or_else(|_| "[]".to_string());
let phones_json =
serde_json::to_string(&contact.phones).unwrap_or_else(|_| "[]".to_string());
let last_interacted_str = contact.last_interacted_at.map(|t| t.to_rfc3339());
sqlx::query(
r#"
INSERT OR REPLACE INTO contacts
(id, account_id, display_name, given_name, family_name, organization, job_title,
notes, avatar_url, emails_json, phones_json, is_favorite, interaction_count,
last_interacted_at, created_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
"#,
)
.bind(&contact.id)
.bind(&contact.account_id)
.bind(&contact.display_name)
.bind(&contact.given_name)
.bind(&contact.family_name)
.bind(&contact.organization)
.bind(&contact.job_title)
.bind(&contact.notes)
.bind(&contact.avatar_url)
.bind(&emails_json)
.bind(&phones_json)
.bind(contact.is_favorite)
.bind(contact.interaction_count as i64)
.bind(&last_interacted_str)
.bind(contact.created_at.to_rfc3339())
.bind(contact.updated_at.to_rfc3339())
.execute(&self.pool)
.await
.map_err(DatabaseError::Query)?;
Ok(())
}
/// Query persisted contacts belonging to `account_id`, most-recently-interacted first.
///
/// See the confidentiality/dependency note on [`Self::save_contact`].
pub async fn list_contacts(
&self,
account_id: &str,
) -> Result<Vec<nuncio_contacts::Contact>, DatabaseError> {
let rows = sqlx::query(
r#"
SELECT id, account_id, display_name, given_name, family_name, organization,
job_title, notes, avatar_url, emails_json, phones_json, is_favorite,
interaction_count, last_interacted_at, created_at, updated_at
FROM contacts
WHERE account_id = ?
ORDER BY interaction_count DESC, display_name ASC
"#,
)
.bind(account_id)
.fetch_all(&self.pool)
.await
.map_err(DatabaseError::Query)?;
Ok(rows.iter().map(contact_from_row).collect())
}
/// Retrieve a single contact by ID.
///
/// See the confidentiality/dependency note on [`Self::save_contact`].
pub async fn get_contact(
&self,
contact_id: &str,
) -> Result<nuncio_contacts::Contact, DatabaseError> {
let row = sqlx::query(
r#"
SELECT id, account_id, display_name, given_name, family_name, organization,
job_title, notes, avatar_url, emails_json, phones_json, is_favorite,
interaction_count, last_interacted_at, created_at, updated_at
FROM contacts
WHERE id = ?
"#,
)
.bind(contact_id)
.fetch_one(&self.pool)
.await
.map_err(DatabaseError::Query)?;
Ok(contact_from_row(&row))
}
/// Update a single message's read/unread flag in place.
///
/// Returns `DatabaseError::Query(sqlx::Error::RowNotFound)` if no message
@@ -1981,6 +2387,45 @@ mod tests {
assert_eq!(hits[0].id, "evt-db-100");
}
/// `list_recurring_calendar_events` returns only rows with a non-empty `rrule`, for the
/// requested account only, and applies no time-window filter at all -- a master whose own
/// stored `start_time` is far outside any given query window must still come back, since
/// its expanded occurrences may fall inside that window.
#[tokio::test]
async fn list_recurring_calendar_events_filters_by_account_and_rrule_presence() {
let (engine, _dir) = DatabaseEngine::connect_ephemeral().await.unwrap();
let mut recurring = sample_calendar_event("evt-recurring", "acct-1");
recurring.rrule = Some("FREQ=WEEKLY;INTERVAL=1".to_string());
recurring.start_time = 1_000; // far outside any window used below
recurring.end_time = 4_600;
engine.save_calendar_event(&recurring).await.unwrap();
let non_recurring = sample_calendar_event("evt-single", "acct-1");
engine.save_calendar_event(&non_recurring).await.unwrap();
let mut other_account_recurring = sample_calendar_event("evt-other-acct", "acct-2");
other_account_recurring.rrule = Some("FREQ=DAILY".to_string());
engine
.save_calendar_event(&other_account_recurring)
.await
.unwrap();
let recurring_for_acct1 = engine
.list_recurring_calendar_events("acct-1")
.await
.expect("list recurring calendar events succeeds");
assert_eq!(recurring_for_acct1.len(), 1);
assert_eq!(recurring_for_acct1[0].id, "evt-recurring");
let recurring_for_acct2 = engine
.list_recurring_calendar_events("acct-2")
.await
.expect("list recurring calendar events succeeds");
assert_eq!(recurring_for_acct2.len(), 1);
assert_eq!(recurring_for_acct2[0].id, "evt-other-acct");
}
/// Re-saving an event with the same ID (an update, not a fresh insert) replaces both the
/// row and its FTS entry rather than duplicating either.
#[tokio::test]
@@ -2026,6 +2471,104 @@ mod tests {
));
}
fn sample_contact(id: &str, account_id: &str) -> nuncio_contacts::Contact {
let mut contact = nuncio_contacts::Contact::new("Architecture Contact", "arch@nuncio.mx");
contact.id = id.to_string();
contact.account_id = Some(account_id.to_string());
contact.organization = Some("KofTwentyTwo".to_string());
contact
}
/// Count rows in `contacts_fts` matching a trigram search term, proving the real write
/// path (not a raw-SQL test insert) keeps the trigger-populated FTS index in sync -- the
/// same proof pattern as `save_get_list_and_search_calendar_event`, without depending on
/// `SearchEngine` gaining a contacts-specific method (out of scope for this story).
async fn contacts_fts_hit_count(engine: &DatabaseEngine, term: &str) -> i64 {
let pattern = format!("%{term}%");
let row: (i64,) =
sqlx::query_as("SELECT COUNT(*) FROM contacts_fts WHERE display_name LIKE ?")
.bind(&pattern)
.fetch_one(engine.pool())
.await
.unwrap();
row.0
}
/// A saved contact is retrievable by both `get_contact` and `list_contacts`, and its FTS
/// mirror is kept in sync by the real write path (not a raw-SQL test insert).
#[tokio::test]
async fn save_get_list_and_search_contact() {
let (engine, _dir) = DatabaseEngine::connect_ephemeral().await.unwrap();
let contact = sample_contact("ct-db-100", "acct-1");
engine
.save_contact(&contact)
.await
.expect("save contact succeeds");
let fetched = engine
.get_contact("ct-db-100")
.await
.expect("get contact succeeds");
assert_eq!(fetched.display_name, contact.display_name);
assert_eq!(fetched.account_id, contact.account_id);
assert_eq!(fetched.emails[0].email, contact.emails[0].email);
let listed = engine
.list_contacts("acct-1")
.await
.expect("list contacts succeeds");
assert_eq!(listed.len(), 1);
assert_eq!(listed[0].id, "ct-db-100");
// Different account -> no match.
let other_account = engine
.list_contacts("acct-other")
.await
.expect("list contacts succeeds");
assert!(other_account.is_empty());
assert_eq!(contacts_fts_hit_count(&engine, "Architecture").await, 1);
}
/// Re-saving a contact with the same ID (an update, not a fresh insert) replaces both the
/// row and its FTS entry rather than duplicating either.
#[tokio::test]
async fn save_contact_upserts_existing_row_and_fts_entry() {
let (engine, _dir) = DatabaseEngine::connect_ephemeral().await.unwrap();
let mut contact = sample_contact("ct-db-200", "acct-1");
engine.save_contact(&contact).await.unwrap();
contact.display_name = "Renamed Contact".to_string();
engine.save_contact(&contact).await.unwrap();
let fetched = engine.get_contact("ct-db-200").await.unwrap();
assert_eq!(fetched.display_name, "Renamed Contact");
assert_eq!(
contacts_fts_hit_count(&engine, "Architecture Contact").await,
0
);
assert_eq!(contacts_fts_hit_count(&engine, "Renamed").await, 1);
}
/// `get_contact` for an ID that was never saved reports `sqlx::Error::RowNotFound` rather
/// than silently fabricating an empty/default contact.
#[tokio::test]
async fn get_contact_reports_not_found_for_missing_contact() {
let (engine, _dir) = DatabaseEngine::connect_ephemeral().await.unwrap();
let err = engine
.get_contact("ct-does-not-exist")
.await
.expect_err("missing contact must be a real error");
assert!(matches!(
err,
DatabaseError::Query(sqlx::Error::RowNotFound)
));
}
/// `set_message_read` flips the persisted
/// `read_flag` in place (both directions), and reports
/// `sqlx::Error::RowNotFound` for a message ID that was never saved,
@@ -2114,6 +2657,158 @@ mod tests {
assert_eq!(accounts[0].smtp_port, 465);
}
/// Regression proof that the per-protocol TLS transport modes are
/// genuinely persisted and round-tripped -- previously they were
/// validated on add but silently discarded, with `list_accounts`
/// hardcoding both back to `ImplicitTls` regardless of what was saved.
/// Saves an account whose IMAP mode is `StartTls` and SMTP mode is
/// `Plain`, then re-reads it via BOTH `list_accounts` and `get_account`
/// and asserts each distinct mode survived rather than collapsing to the
/// default.
#[tokio::test]
async fn account_tls_modes_survive_save_and_reload() {
let (engine, _dir) = DatabaseEngine::connect_ephemeral().await.unwrap();
let acct = nuncio_core::AccountConfig {
id: "acct-tls-1".to_string(),
name: "Mixed TLS Account".to_string(),
email_address: "mixed@nuncio.mx".to_string(),
protocol: nuncio_core::AccountProtocol::ImapSmtp,
server_host: "imap.nuncio.mx".to_string(),
server_port: 143,
smtp_host: "smtp.nuncio.mx".to_string(),
smtp_port: 25,
use_tls: false,
imap_tls_mode: nuncio_core::TlsMode::StartTls,
smtp_tls_mode: nuncio_core::TlsMode::Plain,
keyring_secret_key: "nuncio/acct-tls-1".to_string(),
sync_interval_secs: 60,
};
engine.save_account(&acct).await.expect("save succeeds");
let listed = engine.list_accounts().await.expect("list succeeds");
assert_eq!(listed.len(), 1);
assert_eq!(listed[0].imap_tls_mode, nuncio_core::TlsMode::StartTls);
assert_eq!(listed[0].smtp_tls_mode, nuncio_core::TlsMode::Plain);
let fetched = engine
.get_account("acct-tls-1")
.await
.expect("get succeeds")
.expect("account present");
assert_eq!(fetched.imap_tls_mode, nuncio_core::TlsMode::StartTls);
assert_eq!(fetched.smtp_tls_mode, nuncio_core::TlsMode::Plain);
assert_eq!(fetched, acct);
}
/// Proves `get_account` resolves a saved account by id and returns `None`
/// for an unknown id, and that `delete_account` genuinely removes the row
/// (and is idempotent for an already-absent id).
#[tokio::test]
async fn get_and_delete_account_round_trip() {
let (engine, _dir) = DatabaseEngine::connect_ephemeral().await.unwrap();
let acct = nuncio_core::AccountConfig {
id: "acct-del-1".to_string(),
name: "Deletable".to_string(),
email_address: "del@nuncio.mx".to_string(),
protocol: nuncio_core::AccountProtocol::ImapSmtp,
server_host: "imap.nuncio.mx".to_string(),
server_port: 993,
smtp_host: "smtp.nuncio.mx".to_string(),
smtp_port: 465,
use_tls: true,
imap_tls_mode: nuncio_core::TlsMode::ImplicitTls,
smtp_tls_mode: nuncio_core::TlsMode::ImplicitTls,
keyring_secret_key: "nuncio/acct-del-1".to_string(),
sync_interval_secs: 60,
};
engine.save_account(&acct).await.expect("save succeeds");
assert!(engine
.get_account("acct-del-1")
.await
.expect("get succeeds")
.is_some());
assert!(engine
.get_account("acct-missing")
.await
.expect("get succeeds")
.is_none());
engine
.delete_account("acct-del-1")
.await
.expect("delete succeeds");
assert!(engine
.get_account("acct-del-1")
.await
.expect("get succeeds")
.is_none());
// Idempotent: deleting an already-absent id is not an error.
engine
.delete_account("acct-del-1")
.await
.expect("second delete is a no-op");
}
#[tokio::test]
async fn folder_sync_state_round_trip_is_scoped_and_overwrites() {
let (engine, _dir) = DatabaseEngine::connect_ephemeral().await.unwrap();
// Absent before any sync completes.
assert_eq!(
engine
.get_folder_sync_state("acct-1", "INBOX")
.await
.expect("get succeeds"),
None
);
engine
.save_folder_sync_state("acct-1", "INBOX", "105")
.await
.expect("save succeeds");
assert_eq!(
engine
.get_folder_sync_state("acct-1", "INBOX")
.await
.expect("get succeeds"),
Some("105".to_string())
);
// A later checkpoint overwrites (high-water mark, not history).
engine
.save_folder_sync_state("acct-1", "INBOX", "220")
.await
.expect("save succeeds");
assert_eq!(
engine
.get_folder_sync_state("acct-1", "INBOX")
.await
.expect("get succeeds"),
Some("220".to_string())
);
// Keyed by (account, folder): a different folder/account is unaffected.
assert_eq!(
engine
.get_folder_sync_state("acct-1", "Sent")
.await
.expect("get succeeds"),
None
);
assert_eq!(
engine
.get_folder_sync_state("acct-2", "INBOX")
.await
.expect("get succeeds"),
None
);
}
/// Proves the additive `smtp_host`/`smtp_port`
/// migration is backfill-safe. Simulates a database created BEFORE this
/// feature existed (an `accounts` table with no `smtp_host`/`smtp_port`
+2
View File
@@ -13,6 +13,7 @@ workspace = true
nuncio-core = { path = "../nuncio-core" }
nuncio-mail = { path = "../nuncio-mail" }
nuncio-cal = { path = "../nuncio-cal" }
nuncio-contacts = { path = "../nuncio-contacts" }
nuncio-store = { path = "../nuncio-store" }
nuncio-filter = { path = "../nuncio-filter" }
nuncio-proto = { path = "../nuncio-proto" }
@@ -36,3 +37,4 @@ bytes.workspace = true
[dev-dependencies]
nuncio-cli = { path = "../nuncio-cli" }
sqlx.workspace = true
wiremock.workspace = true
+147
View File
@@ -0,0 +1,147 @@
//! Real inbound calendar synchronization routine.
//!
//! Mirrors `nunciod::sync`'s shape for the calendar engine: [`sync_with_backend`]
//! fetches events from a [`CalendarBackend`] for a single calendar collection and
//! time window, then persists each event via [`DatabaseEngine::save_calendar_event`].
//!
//! # No production entry point yet
//!
//! Unlike `sync.rs`'s `run_account_sync`/`run_all_accounts_sync`, this module
//! deliberately has NO production entry point that resolves a real account's
//! CalDAV collection URL and credential and builds a real `CalDavClient` from
//! them: `nuncio_core::AccountConfig` carries no CalDAV fields today, so there
//! is nothing persisted to build one from. [`sync_with_backend`] is the only
//! function here; it is exercised directly in tests with `MockCalendarBackend`,
//! and it is the same seam a future story's production entry point will call
//! once per-account CalDAV configuration exists.
use nuncio_cal::CalendarBackend;
use nuncio_store::db::DatabaseError;
use thiserror::Error;
/// Errors that can occur while synchronizing a single calendar collection's events.
#[derive(Debug, Error)]
pub enum CalendarSyncError {
/// The calendar backend (CalDAV/mock) reported an error.
#[error("calendar backend error: {0}")]
Backend(#[from] nuncio_cal::CalendarError),
/// A database persistence error occurred while saving synced events.
#[error("database error: {0}")]
Store(#[from] DatabaseError),
}
/// Fetch every event in `calendar_id` whose window overlaps `[start_window,
/// end_window]` from `backend`, persisting each via
/// [`DatabaseEngine::save_calendar_event`]. Returns the number of events
/// processed.
///
/// Pure fetch-and-persist, unit-testable directly with a
/// [`nuncio_cal::MockCalendarBackend`] -- no event-bus or credential logic,
/// mirroring `nunciod::sync::fetch_and_persist`.
pub async fn sync_with_backend(
db: &nuncio_store::db::DatabaseEngine,
backend: &dyn CalendarBackend,
calendar_id: &str,
start_window: i64,
end_window: i64,
) -> Result<usize, CalendarSyncError> {
let events = backend
.fetch_events(calendar_id, start_window, end_window)
.await?;
let mut synced = 0usize;
for event in events {
db.save_calendar_event(&event).await?;
synced += 1;
}
Ok(synced)
}
#[cfg(test)]
mod tests {
use super::*;
use nuncio_cal::MockCalendarBackend;
use nuncio_core::model::CalendarEvent;
use nuncio_store::db::DatabaseEngine;
fn mock_event(id: &str, calendar_id: &str, start: i64, end: i64) -> CalendarEvent {
CalendarEvent {
id: id.to_string(),
account_id: "acct-cal-1".to_string(),
calendar_id: calendar_id.to_string(),
summary: format!("Event {id}"),
start_time: start,
end_time: end,
rrule: None,
location: None,
}
}
#[tokio::test]
async fn sync_with_backend_persists_mock_events() {
let (db, _dir) = DatabaseEngine::connect_ephemeral()
.await
.expect("ephemeral db");
let mock = MockCalendarBackend::new();
mock.add_event(mock_event(
"evt-1",
"cal-work",
1_700_000_000,
1_700_003_600,
));
mock.add_event(mock_event(
"evt-2",
"cal-work",
1_700_010_000,
1_700_013_600,
));
// Outside the requested window entirely -- must not be returned or persisted.
mock.add_event(mock_event(
"evt-out-of-window",
"cal-work",
1_800_000_000,
1_800_003_600,
));
let synced = sync_with_backend(&db, &mock, "cal-work", 1_699_999_000, 1_700_020_000)
.await
.expect("sync succeeds");
assert_eq!(synced, 2);
let persisted = db
.list_calendar_events("acct-cal-1", 1_699_999_000, 1_700_020_000)
.await
.expect("list persisted events");
assert_eq!(persisted.len(), 2);
let ids: Vec<&str> = persisted.iter().map(|e| e.id.as_str()).collect();
assert!(ids.contains(&"evt-1"));
assert!(ids.contains(&"evt-2"));
}
#[tokio::test]
async fn sync_with_backend_returns_zero_for_empty_backend() {
let (db, _dir) = DatabaseEngine::connect_ephemeral()
.await
.expect("ephemeral db");
let mock = MockCalendarBackend::new();
let synced = sync_with_backend(&db, &mock, "cal-empty", 0, i64::MAX)
.await
.expect("sync succeeds");
assert_eq!(synced, 0);
}
#[tokio::test]
async fn sync_with_backend_surfaces_backend_failure() {
let (db, _dir) = DatabaseEngine::connect_ephemeral()
.await
.expect("ephemeral db");
let mock = MockCalendarBackend::new();
mock.set_should_fail(true);
let err = sync_with_backend(&db, &mock, "cal-work", 0, i64::MAX)
.await
.expect_err("sync fails");
assert!(matches!(err, CalendarSyncError::Backend(_)));
assert!(err.to_string().contains("calendar backend error"));
}
}
+123
View File
@@ -0,0 +1,123 @@
//! Real inbound contacts synchronization routine.
//!
//! Mirrors `nunciod::calendar_sync`'s shape for the contacts engine:
//! [`sync_with_backend`] fetches contacts from a [`ContactsBackend`] for a
//! single account, then persists each contact via
//! [`DatabaseEngine::save_contact`].
//!
//! # No production entry point yet
//!
//! Unlike `sync.rs`'s `run_account_sync`/`run_all_accounts_sync`, this module
//! deliberately has NO production entry point that resolves a real account's
//! CardDAV address book collection URL and credential and builds a real
//! `CardDavClient` from them: `nuncio_core::AccountConfig` carries no CardDAV
//! fields today, so there is nothing persisted to build one from.
//! [`sync_with_backend`] is the only function here; it is exercised directly
//! in tests with `MockContactsBackend`, and it is the same seam a future
//! story's production entry point will call once per-account CardDAV
//! configuration exists.
use nuncio_contacts::ContactsBackend;
use nuncio_store::db::DatabaseError;
use thiserror::Error;
/// Errors that can occur while synchronizing a single account's contacts.
#[derive(Debug, Error)]
pub enum ContactsSyncError {
/// The contacts backend (CardDAV/mock) reported an error.
#[error("contacts backend error: {0}")]
Backend(#[from] nuncio_contacts::CardDavError),
/// A database persistence error occurred while saving synced contacts.
#[error("database error: {0}")]
Store(#[from] DatabaseError),
}
/// Fetch every contact belonging to `account_id` from `backend`, persisting
/// each via [`DatabaseEngine::save_contact`]. Returns the number of contacts
/// processed.
///
/// Pure fetch-and-persist, unit-testable directly with a
/// [`nuncio_contacts::MockContactsBackend`] -- no event-bus or credential
/// logic, mirroring `nunciod::calendar_sync::sync_with_backend`.
pub async fn sync_with_backend(
db: &nuncio_store::db::DatabaseEngine,
backend: &dyn ContactsBackend,
account_id: &str,
) -> Result<usize, ContactsSyncError> {
let contacts = backend.fetch_contacts(account_id).await?;
let mut synced = 0usize;
for contact in contacts {
db.save_contact(&contact).await?;
synced += 1;
}
Ok(synced)
}
#[cfg(test)]
mod tests {
use super::*;
use nuncio_contacts::{Contact, MockContactsBackend};
use nuncio_store::db::DatabaseEngine;
fn mock_contact(id: &str, account_id: &str, display_name: &str) -> Contact {
let mut contact = Contact::new(display_name, format!("{display_name}@nuncio.mx"));
contact.id = id.to_string();
contact.account_id = Some(account_id.to_string());
contact
}
#[tokio::test]
async fn sync_with_backend_persists_mock_contacts() {
let (db, _dir) = DatabaseEngine::connect_ephemeral()
.await
.expect("ephemeral db");
let mock = MockContactsBackend::new();
mock.add_contact(mock_contact("ct-1", "acct-contacts-1", "Alice"));
mock.add_contact(mock_contact("ct-2", "acct-contacts-1", "Bob"));
// Belongs to a different account -- must not be returned or persisted
// for `acct-contacts-1`'s sync.
mock.add_contact(mock_contact("ct-other", "acct-other", "Carol"));
let synced = sync_with_backend(&db, &mock, "acct-contacts-1")
.await
.expect("sync succeeds");
assert_eq!(synced, 2);
let persisted = db
.list_contacts("acct-contacts-1")
.await
.expect("list persisted contacts");
assert_eq!(persisted.len(), 2);
let ids: Vec<&str> = persisted.iter().map(|c| c.id.as_str()).collect();
assert!(ids.contains(&"ct-1"));
assert!(ids.contains(&"ct-2"));
}
#[tokio::test]
async fn sync_with_backend_returns_zero_for_empty_backend() {
let (db, _dir) = DatabaseEngine::connect_ephemeral()
.await
.expect("ephemeral db");
let mock = MockContactsBackend::new();
let synced = sync_with_backend(&db, &mock, "acct-empty")
.await
.expect("sync succeeds");
assert_eq!(synced, 0);
}
#[tokio::test]
async fn sync_with_backend_surfaces_backend_failure() {
let (db, _dir) = DatabaseEngine::connect_ephemeral()
.await
.expect("ephemeral db");
let mock = MockContactsBackend::new();
mock.set_should_fail(true);
let err = sync_with_backend(&db, &mock, "acct-contacts-1")
.await
.expect_err("sync fails");
assert!(matches!(err, ContactsSyncError::Backend(_)));
assert!(err.to_string().contains("contacts backend error"));
}
}
+2397 -43
View File
File diff suppressed because it is too large. Load diff
+2
View File
@@ -2,6 +2,8 @@
#![cfg_attr(test, allow(clippy::unwrap_used, clippy::expect_used, clippy::panic))]
pub mod calendar_sync;
pub mod contacts_sync;
pub mod grpc;
pub mod orchestrator;
pub mod send;
+3
View File
@@ -54,6 +54,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
let db_sync = db.clone();
let secrets_sync = account_secrets.clone();
let event_bus_sync = event_bus.clone();
let filter_engine_sync = filter_engine.clone();
let _sync_command_task = tokio::spawn(async move {
while let Some(cmd) = command_rx.recv().await {
match cmd {
@@ -62,6 +63,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
&db_sync,
&secrets_sync,
&event_bus_sync,
&filter_engine_sync,
)
.await;
tracing::info!("SyncAll completed: {} message(s) synced", synced);
@@ -71,6 +73,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
&db_sync,
&secrets_sync,
&event_bus_sync,
&filter_engine_sync,
&account_id,
)
.await
+113 -16
View File
@@ -12,13 +12,13 @@
//!
//! # Account selection
//!
//! The `nuncio.v1.Mail` service (like every other RPC on it --
//! `ListMessages`, `GetMessage`, `SearchMessages`, etc.) does not yet
//! disambiguate between multiple configured accounts; it operates over a
//! single, global store. Consistent with that existing scope, `SendMessage`
//! sends from the first account returned by [`DatabaseEngine::list_accounts`].
//! Supporting an explicit multi-account choice is future work, alongside
//! the rest of the `Mail` service's single-account scope.
//! `SendMessage` accepts an optional `account_id`. When the caller supplies
//! one, that specific account is resolved and used as the sender -- a
//! request naming an account that does not exist fails honestly rather than
//! silently falling back to a different identity. When `account_id` is
//! omitted, the account resolves to the first one returned by
//! [`DatabaseEngine::list_accounts`], preserving behavior for callers that
//! predate explicit selection.
//!
//! # Testability
//!
@@ -61,6 +61,13 @@ pub enum SendError {
/// No account is configured to send mail from.
#[error("no account is configured to send mail from")]
NoAccountConfigured,
/// The caller specified an `account_id` that does not match any
/// configured account. Distinct from [`SendError::NoAccountConfigured`]
/// (which means zero accounts exist at all): this means at least one
/// account exists, just not the one requested, so the request must fail
/// rather than silently sending from a different account.
#[error("no account configured with id {0:?}")]
AccountNotFound(String),
/// Failed to read the account's credential from the secret vault.
#[error("failed to read account credential from the secret vault: {0}")]
Vault(#[from] VaultError),
@@ -90,6 +97,9 @@ pub struct ComposeRequest {
pub body_html: Option<String>,
/// File attachments.
pub attachments: Vec<Attachment>,
/// Identifier of the account to send from. `None` preserves the prior
/// "first configured account" behavior.
pub account_id: Option<String>,
}
/// Send `message` through `sender`. Pure seam shared by production and
@@ -113,6 +123,7 @@ fn build_smtp_sender(
let engine = SmtpTransportEngine::new(
&config.smtp_host,
config.smtp_port,
config.smtp_tls_mode,
&config.email_address,
password,
)?;
@@ -120,14 +131,24 @@ fn build_smtp_sender(
}
/// Resolves the account to send mail from. See the module-level "Account
/// selection" doc above for why this is currently the first configured
/// account rather than an explicit per-request choice.
async fn resolve_sending_account(db: &DatabaseEngine) -> Result<AccountConfig, SendError> {
/// selection" doc above: an explicit `account_id` must match a configured
/// account or the request fails; `None` preserves the "first configured
/// account" behavior.
async fn resolve_sending_account(
db: &DatabaseEngine,
account_id: Option<&str>,
) -> Result<AccountConfig, SendError> {
let accounts = db.list_accounts().await?;
accounts
.into_iter()
.next()
.ok_or(SendError::NoAccountConfigured)
match account_id {
Some(id) => accounts
.into_iter()
.find(|a| a.id == id)
.ok_or_else(|| SendError::AccountNotFound(id.to_string())),
None => accounts
.into_iter()
.next()
.ok_or(SendError::NoAccountConfigured),
}
}
/// Generates a daemon-local identifier for a sent message, for reference in
@@ -153,7 +174,7 @@ pub async fn send_message_for_account(
secrets: &SecretManager,
request: ComposeRequest,
) -> Result<String, SendError> {
let config = resolve_sending_account(db).await?;
let config = resolve_sending_account(db, request.account_id.as_deref()).await?;
let password = secrets.get_secret(&config.keyring_secret_key)?;
let sender = build_smtp_sender(&config, &password)?;
@@ -185,7 +206,7 @@ pub async fn send_message_with_injected_sender(
sender: &dyn MessageSender,
request: ComposeRequest,
) -> Result<String, SendError> {
let config = resolve_sending_account(db).await?;
let config = resolve_sending_account(db, request.account_id.as_deref()).await?;
let outbound = OutboundMessage {
from: config.email_address,
@@ -285,6 +306,7 @@ mod tests {
body_text: Some("Body".to_string()),
body_html: None,
attachments: Vec::new(),
account_id: None,
},
)
.await
@@ -313,6 +335,7 @@ mod tests {
body_text: Some("Body".to_string()),
body_html: None,
attachments: Vec::new(),
account_id: None,
},
)
.await
@@ -351,6 +374,7 @@ mod tests {
body_text: Some("Body".to_string()),
body_html: None,
attachments: Vec::new(),
account_id: None,
},
)
.await
@@ -391,6 +415,7 @@ mod tests {
body_text: Some("Body via injected sender.".to_string()),
body_html: None,
attachments: Vec::new(),
account_id: None,
},
)
.await
@@ -422,6 +447,7 @@ mod tests {
body_text: Some("Body".to_string()),
body_html: None,
attachments: Vec::new(),
account_id: None,
},
)
.await
@@ -429,4 +455,75 @@ mod tests {
assert!(matches!(err, SendError::NoAccountConfigured));
assert!(mock.sent_messages().is_empty());
}
/// Proves that with multiple configured accounts, an explicit
/// `account_id` selects that SPECIFIC account as the sender -- not
/// whichever account happens to be first -- by asserting the captured
/// outbound message's `From:` matches the requested account.
#[tokio::test]
async fn send_message_with_injected_sender_uses_the_explicitly_selected_account() {
let (db, _dir) = DatabaseEngine::connect_ephemeral()
.await
.expect("ephemeral db");
let first = sample_account("acct-first");
let second = sample_account("acct-second");
db.save_account(&first).await.expect("save first account");
db.save_account(&second).await.expect("save second account");
let mock = MockMessageSender::new();
send_message_with_injected_sender(
&db,
&mock,
ComposeRequest {
to: "bob@nuncio.mx".to_string(),
cc: None,
subject: "Hi".to_string(),
body_text: Some("Body".to_string()),
body_html: None,
attachments: Vec::new(),
account_id: Some(second.id.clone()),
},
)
.await
.expect("send succeeds via explicitly selected account");
let sent = mock.sent_messages();
assert_eq!(sent.len(), 1);
assert_eq!(sent[0].from, second.email_address);
assert_ne!(sent[0].from, first.email_address);
}
/// An `account_id` naming an account that does not exist must fail with
/// the specific [`SendError::AccountNotFound`] variant -- never a silent
/// fallback to the first configured account, and never confusable with
/// [`SendError::NoAccountConfigured`] (which means zero accounts exist).
#[tokio::test]
async fn send_message_with_injected_sender_rejects_unknown_account_id() {
let (db, _dir) = DatabaseEngine::connect_ephemeral()
.await
.expect("ephemeral db");
let config = sample_account("acct-only");
db.save_account(&config).await.expect("save account");
let mock = MockMessageSender::new();
let err = send_message_with_injected_sender(
&db,
&mock,
ComposeRequest {
to: "bob@nuncio.mx".to_string(),
cc: None,
subject: "Hi".to_string(),
body_text: Some("Body".to_string()),
body_html: None,
attachments: Vec::new(),
account_id: Some("acct-does-not-exist".to_string()),
},
)
.await
.expect_err("unknown account id must fail");
assert!(matches!(err, SendError::AccountNotFound(id) if id == "acct-does-not-exist"));
assert!(mock.sent_messages().is_empty());
}
}
+751 -44
View File
@@ -18,7 +18,8 @@
//! only places that resolve a real account's credentials and construct a
//! real backend; they then delegate the actual fetch/persist/event-emission
//! work to [`sync_with_backend`].
use nuncio_core::{AccountConfig, AccountProtocol, CoreCommand, EventBus};
use nuncio_core::{AccountConfig, AccountProtocol, CoreCommand, CoreEvent, EventBus};
use nuncio_filter::{FilterEngine, OutboxManager, RuleAction};
use nuncio_mail::{ImapEngine, JmapEngine, MailBackend, MailError};
use nuncio_store::db::DatabaseError;
use nuncio_store::vault::{SecretManager, VaultError};
@@ -41,28 +42,241 @@ pub enum SyncError {
Store(#[from] DatabaseError),
}
/// Map a [`RuleAction`] to the short mutation tag persisted on
/// [`nuncio_filter::PendingRemoteMutation::mutation_type`] and the outbox
/// target parameter, for every action that is not applied immediately.
///
/// Nothing downstream currently parses `mutation_type` back into a
/// `RuleAction`, so these tags are a fixed vocabulary owned by this module;
/// keep them stable once the outbox worker starts consuming them.
fn remote_action_tag(action: &RuleAction) -> Option<(&'static str, Option<String>)> {
match action {
RuleAction::MoveTo(folder) => Some(("MOVE", Some(folder.clone()))),
RuleAction::CopyTo(folder) => Some(("COPY", Some(folder.clone()))),
RuleAction::Flag => Some(("FLAG", None)),
RuleAction::Unflag => Some(("UNFLAG", None)),
RuleAction::Delete => Some(("DELETE", None)),
RuleAction::ForwardTo(address) => Some(("FORWARD", Some(address.clone()))),
RuleAction::CallWebhook(url) => Some(("WEBHOOK", Some(url.clone()))),
RuleAction::MarkRead | RuleAction::MarkUnread => None,
}
}
/// Evaluate `email` against `filter_engine` and route every matched rule's
/// actions to their real, persisted effect.
///
/// Filters fire EXACTLY ONCE per newly-arrived message: callers (see
/// [`fetch_and_persist`]) must only invoke this for a message that was
/// genuinely new to the store on this sync pass, never for one that already
/// existed. Sync is not yet incremental -- every sync re-fetches and
/// re-persists messages the backend still reports -- so calling this
/// unconditionally on every persisted message would re-fire a rule's actions
/// on every single re-sync of the same mailbox: a fresh
/// `PendingRemoteMutation` and a fresh `FilterExecutionLog` entry per cycle
/// for a message that arrived once, growing the outbox and the audit ledger
/// without bound and (once a real mutation transport lands) duplicating the
/// real remote operation. This function itself has no way to tell "new" from
/// "already seen" -- that determination is the caller's responsibility.
///
/// `MarkRead`/`MarkUnread` apply immediately to the stored message via
/// [`DatabaseEngine::set_message_read`]. Every other action enqueues a real
/// [`nuncio_filter::PendingRemoteMutation`] through the existing outbox
/// (`OutboxManager::create_mutation` + `DatabaseEngine::save_pending_mutation`)
/// for the background outbox worker to pick up -- this function never
/// performs the real remote IMAP/JMAP mutation or webhook call itself.
/// Every match also appends a [`nuncio_filter::FilterExecutionLog`] entry via
/// `DatabaseEngine::save_filter_execution_log`, so there is an audit trail of
/// what fired independent of whether the outbox item is ever drained.
///
/// A bookkeeping failure on one action (a DB write error) is logged via
/// `tracing::warn!` and does not stop the remaining actions/rules from being
/// applied -- one broken write must never silently swallow the rest of a
/// sync's filtering. The return value counts only actions that were fully
/// applied AND logged; a partially-failed match (e.g. the mutation persists
/// but the execution log write fails) is not counted as success, keeping the
/// count honest for callers that log or assert on it.
///
/// Standalone and reusable by design: this is the exact evaluate-and-route
/// logic other sync paths (e.g. a bulk retroactive triage over the whole
/// store) need, so it takes only a `DatabaseEngine`/`FilterEngine`/`Email`
/// and has no dependency on the live inbound-sync call chain.
pub async fn apply_filter_actions(
db: &nuncio_store::db::DatabaseEngine,
filter_engine: &FilterEngine,
email: &nuncio_core::model::Email,
) -> usize {
let mut applied = 0usize;
for (rule, actions) in filter_engine.evaluate(email) {
for action in actions {
let immediate_ok = match &action {
RuleAction::MarkRead => match db.set_message_read(&email.id, true).await {
Ok(()) => true,
Err(e) => {
tracing::warn!(
"filter rule '{}' MARK READ failed for message '{}': {e}",
rule.id,
email.id
);
false
}
},
RuleAction::MarkUnread => match db.set_message_read(&email.id, false).await {
Ok(()) => true,
Err(e) => {
tracing::warn!(
"filter rule '{}' MARK UNREAD failed for message '{}': {e}",
rule.id,
email.id
);
false
}
},
RuleAction::MoveTo(_)
| RuleAction::CopyTo(_)
| RuleAction::Flag
| RuleAction::Unflag
| RuleAction::Delete
| RuleAction::ForwardTo(_)
| RuleAction::CallWebhook(_) => {
// Remote actions are handled below via the outbox; this
// arm only tracks whether the immediate-action write
// itself succeeded (there is none for a remote action,
// so it is unconditionally "not applicable" here). Listed
// explicitly rather than a wildcard so a future
// immediate-style action can't silently fall through to
// remote routing by accident.
true
}
};
if !immediate_ok {
continue;
}
let mut action_ok = true;
if let Some((tag, target)) = remote_action_tag(&action) {
let mutation = OutboxManager::create_mutation(&rule.id, &email.id, tag, target);
if let Err(e) = db.save_pending_mutation(&mutation).await {
tracing::warn!(
"filter rule '{}' failed to enqueue outbox mutation for message '{}': {e}",
rule.id,
email.id
);
action_ok = false;
}
}
if !action_ok {
continue;
}
let action_desc = action.to_nsql();
if let Err(e) = db
.save_filter_execution_log(&rule.id, &email.id, &action_desc)
.await
{
tracing::warn!(
"filter rule '{}' matched message '{}' but failed to write execution log: {e}",
rule.id,
email.id
);
continue;
}
applied += 1;
}
}
applied
}
/// Reports whether `message_id` is not yet present in the store.
///
/// Sync is not yet incremental (a re-sync re-fetches every message the
/// backend still reports, not just genuinely new ones), so this is the guard
/// that keeps filter actions firing exactly once per message: it must be
/// checked BEFORE `save_email` persists (or re-persists) the message, since
/// `save_email` is `INSERT OR REPLACE` and would otherwise erase the
/// distinction between "arriving for the first time" and "seen again".
///
/// A lookup failure that is not "not found" (a genuine DB error) is logged
/// and treated as "not new", the conservative choice: firing filters again on
/// a message we can't positively identify as new risks duplicate remote
/// mutations, which is worse than occasionally missing a fire on a message
/// that was in fact new.
async fn is_new_message(db: &nuncio_store::db::DatabaseEngine, message_id: &str) -> bool {
match db.get_message(message_id).await {
Ok(_) => false,
Err(e) if e.is_not_found() => true,
Err(e) => {
tracing::warn!(
"could not determine whether message '{message_id}' is new before sync \
(treating as not-new to avoid duplicate filter firing): {e}"
);
false
}
}
}
/// Fetch every folder and every message in every folder from `backend`,
/// persisting each message via [`DatabaseEngine::save_email`]. Returns the
/// total number of messages processed (a message id "processed" more than
/// once, e.g. by a backend that returns the same id from multiple folders,
/// is counted once per occurrence -- `save_email` itself is `INSERT OR
/// REPLACE`, so storage stays deduplicated by message id regardless).
/// persisting each message via [`DatabaseEngine::save_email`] and, for
/// messages genuinely new to the store, evaluating it against
/// `filter_engine` via [`apply_filter_actions`] (see [`is_new_message`] for
/// why this must be gated rather than run on every persisted message).
/// Returns the total number of messages processed (a message id "processed"
/// more than once, e.g. by a backend that returns the same id from multiple
/// folders, is counted once per occurrence -- `save_email` itself is
/// `INSERT OR REPLACE`, so storage stays deduplicated by message id
/// regardless).
///
/// Contains no event-bus or credential logic -- pure fetch-and-persist, so
/// it is the smallest unit tests can exercise directly with a
/// [`MockMailBackend`].
async fn fetch_and_persist(
db: &nuncio_store::db::DatabaseEngine,
event_bus: &EventBus,
backend: &dyn MailBackend,
filter_engine: &FilterEngine,
account_id: Option<&str>,
) -> Result<usize, SyncError> {
let folders = backend.sync_folders().await?;
let mut synced = 0usize;
for folder in folders {
let (emails, _state) = backend.sync_messages(&folder.id, None).await?;
// Resume from the folder's stored checkpoint so the backend can fetch
// only what changed since the last sync. A checkpoint is keyed by
// (account, folder), so a run with no account context (a mock-driven
// `SyncAll` with `account_id == None`) has nowhere to key it and always
// performs a full fetch.
let last_state = match account_id {
Some(acct) => db.get_folder_sync_state(acct, &folder.id).await?,
None => None,
};
let (emails, new_state) = backend
.sync_messages(&folder.id, last_state.as_deref())
.await?;
let fetched = emails.len();
for email in emails {
let is_new = is_new_message(db, &email.id).await;
db.save_email(&email).await?;
if is_new {
apply_filter_actions(db, filter_engine, &email).await;
}
synced += 1;
}
// Persist the returned checkpoint only AFTER this folder's messages
// have landed, so the stored high-water mark can never advance past
// work that actually reached the store.
if let Some(acct) = account_id {
db.save_folder_sync_state(acct, &folder.id, &new_state)
.await?;
}
event_bus.publish_event(CoreEvent::SyncProgress {
account_id: account_id.map(str::to_string),
folder_id: folder.id.clone(),
fetched,
total: synced,
});
}
Ok(synced)
}
@@ -85,6 +299,7 @@ pub async fn sync_with_backend(
db: &nuncio_store::db::DatabaseEngine,
event_bus: &EventBus,
backend: &dyn MailBackend,
filter_engine: &FilterEngine,
account_id: Option<String>,
) -> Result<usize, SyncError> {
match &account_id {
@@ -94,7 +309,8 @@ pub async fn sync_with_backend(
None => event_bus.process_command(CoreCommand::SyncAll),
}
let result = fetch_and_persist(db, backend).await;
let result =
fetch_and_persist(db, event_bus, backend, filter_engine, account_id.as_deref()).await;
if let Err(e) = &result {
event_bus.process_command(CoreCommand::ReportError {
@@ -129,10 +345,16 @@ fn build_mail_backend(config: &AccountConfig, password: &str) -> Box<dyn MailBac
&config.id,
&config.server_host,
config.server_port,
config.imap_tls_mode,
&config.email_address,
password,
)),
AccountProtocol::Jmap => Box::new(JmapEngine::with_credentials(
&config.id,
&config.server_host,
&config.email_address,
password,
)),
AccountProtocol::Jmap => Box::new(JmapEngine::new(&config.id)),
}
}
@@ -148,6 +370,7 @@ pub async fn run_account_sync(
db: &nuncio_store::db::DatabaseEngine,
secrets: &SecretManager,
event_bus: &EventBus,
filter_engine: &FilterEngine,
account_id: &str,
) -> Result<usize, SyncError> {
let setup = async {
@@ -163,6 +386,7 @@ pub async fn run_account_sync(
db,
event_bus,
backend.as_ref(),
filter_engine,
Some(account_id.to_string()),
)
.await
@@ -187,6 +411,7 @@ pub async fn run_all_accounts_sync(
db: &nuncio_store::db::DatabaseEngine,
secrets: &SecretManager,
event_bus: &EventBus,
filter_engine: &FilterEngine,
) -> usize {
event_bus.process_command(CoreCommand::SyncAll);
@@ -197,7 +422,15 @@ pub async fn run_all_accounts_sync(
match secrets.get_secret(&config.keyring_secret_key) {
Ok(password) => {
let backend = build_mail_backend(&config, &password);
match fetch_and_persist(db, backend.as_ref()).await {
match fetch_and_persist(
db,
event_bus,
backend.as_ref(),
filter_engine,
Some(&config.id),
)
.await
{
Ok(count) => total_synced += count,
Err(e) => {
event_bus.process_command(CoreCommand::ReportError {
@@ -238,14 +471,27 @@ mod tests {
use nuncio_core::{CoreEvent, EngineStatus, TlsMode};
use nuncio_mail::MockMailBackend;
use nuncio_store::db::DatabaseEngine;
use serde_json::json;
use wiremock::matchers::{body_partial_json, method, path};
use wiremock::{Mock, MockServer, ResponseTemplate};
/// A `FilterEngine` with no rules, for tests that exercise sync
/// mechanics unrelated to filtering.
fn empty_filter_engine() -> FilterEngine {
FilterEngine::new(Vec::new()).expect("empty rule set")
}
fn sample_jmap_account(id: &str) -> AccountConfig {
sample_jmap_account_at(id, "jmap.nuncio.mx")
}
fn sample_jmap_account_at(id: &str, host: &str) -> AccountConfig {
AccountConfig {
id: id.to_string(),
name: "JMAP Test Account".to_string(),
email_address: format!("{id}@nuncio.mx"),
protocol: AccountProtocol::Jmap,
server_host: "jmap.nuncio.mx".to_string(),
server_host: host.to_string(),
server_port: 443,
smtp_host: "smtp.nuncio.mx".to_string(),
smtp_port: 465,
@@ -257,6 +503,79 @@ mod tests {
}
}
/// Mount a full JMAP session-discovery + `Mailbox/get` + `Email/query` + `Email/get`
/// stub set on `mock_server`, serving exactly one folder ("jmap-inbox") containing one
/// message ("jmap-msg-1"). Used by tests that need `JmapEngine::with_credentials` to
/// complete a real (wiremock-backed) fetch through `MailBackend`.
async fn mount_single_message_jmap_stubs(mock_server: &MockServer, account_id: &str) {
let session_body = json!({
"username": format!("{account_id}@nuncio.mx"),
"primaryAccounts": { "urn:ietf:params:jmap:mail": account_id },
"apiUrl": format!("{}/jmap/api", mock_server.uri()),
"state": "session-state-1"
});
Mock::given(method("GET"))
.and(path("/.well-known/jmap"))
.respond_with(ResponseTemplate::new(200).set_body_json(&session_body))
.mount(mock_server)
.await;
let mailbox_get_body = json!({
"methodResponses": [[
"Mailbox/get",
{ "list": [ { "id": "jmap-inbox", "name": "Inbox", "totalEmails": 1, "unreadEmails": 1 } ] },
"c1"
]]
});
Mock::given(method("POST"))
.and(path("/jmap/api"))
.and(body_partial_json(
json!({"methodCalls": [["Mailbox/get", {}, "c1"]]}),
))
.respond_with(ResponseTemplate::new(200).set_body_json(&mailbox_get_body))
.mount(mock_server)
.await;
let email_query_body = json!({
"methodResponses": [["Email/query", {"ids": ["jmap-msg-1"]}, "c1"]]
});
Mock::given(method("POST"))
.and(path("/jmap/api"))
.and(body_partial_json(
json!({"methodCalls": [["Email/query", {}, "c1"]]}),
))
.respond_with(ResponseTemplate::new(200).set_body_json(&email_query_body))
.mount(mock_server)
.await;
let email_get_body = json!({
"methodResponses": [[
"Email/get",
{
"state": "sync-state-1",
"list": [{
"id": "jmap-msg-1",
"subject": "Welcome to JMAP Sync",
"from": [{ "email": "support@nuncio.mx" }],
"to": [{ "email": format!("{account_id}@nuncio.mx") }],
"receivedAt": 1700000000i64,
"isUnread": true,
"bodySnippet": "Real wiremock-backed JMAP fetch."
}]
},
"c1"
]]
});
Mock::given(method("POST"))
.and(path("/jmap/api"))
.and(body_partial_json(
json!({"methodCalls": [["Email/get", {}, "c1"]]}),
))
.respond_with(ResponseTemplate::new(200).set_body_json(&email_get_body))
.mount(mock_server)
.await;
}
fn sample_imap_account(id: &str) -> AccountConfig {
AccountConfig {
id: id.to_string(),
@@ -309,9 +628,16 @@ mod tests {
mock.add_message(mock_email("m1", "inbox", "Hello"));
mock.add_message(mock_email("m2", "inbox", "World"));
let synced = sync_with_backend(&db, &event_bus, &mock, Some("acct-mock-1".to_string()))
.await
.expect("sync succeeds");
let filter_engine = empty_filter_engine();
let synced = sync_with_backend(
&db,
&event_bus,
&mock,
&filter_engine,
Some("acct-mock-1".to_string()),
)
.await
.expect("sync succeeds");
assert_eq!(synced, 2);
let persisted = db
@@ -329,6 +655,16 @@ mod tests {
account_id: Some("acct-mock-1".to_string())
}
);
// A per-folder progress event is published between start and complete.
assert_eq!(
events.recv().await.expect("progress event"),
CoreEvent::SyncProgress {
account_id: Some("acct-mock-1".to_string()),
folder_id: "inbox".to_string(),
fetched: 2,
total: 2,
}
);
assert_eq!(
events.recv().await.expect("complete event"),
CoreEvent::SyncCompleted {
@@ -354,7 +690,8 @@ mod tests {
unread_messages: 0,
});
let synced = sync_with_backend(&db, &event_bus, &mock, None)
let filter_engine = empty_filter_engine();
let synced = sync_with_backend(&db, &event_bus, &mock, &filter_engine, None)
.await
.expect("sync succeeds");
assert_eq!(synced, 0);
@@ -363,6 +700,16 @@ mod tests {
events.recv().await.expect("start event"),
CoreEvent::SyncStarted { account_id: None }
);
// The folder is empty, but a progress event still marks it done.
assert_eq!(
events.recv().await.expect("progress event"),
CoreEvent::SyncProgress {
account_id: None,
folder_id: "inbox".to_string(),
fetched: 0,
total: 0,
}
);
assert_eq!(
events.recv().await.expect("complete event"),
CoreEvent::SyncCompleted { account_id: None }
@@ -380,9 +727,16 @@ mod tests {
let mock = MockMailBackend::new();
mock.set_should_fail(true);
let err = sync_with_backend(&db, &event_bus, &mock, Some("acct-fail".to_string()))
.await
.expect_err("sync fails");
let filter_engine = empty_filter_engine();
let err = sync_with_backend(
&db,
&event_bus,
&mock,
&filter_engine,
Some("acct-fail".to_string()),
)
.await
.expect_err("sync fails");
assert!(err.to_string().contains("mail backend error"));
assert_eq!(
@@ -447,12 +801,14 @@ mod tests {
#[tokio::test]
async fn run_account_sync_persists_real_backend_messages_and_reports_success() {
// Uses a real (non-mock) `JmapEngine` -- its `MailBackend` impl
// returns deterministic static data with no network I/O, so this
// proves `run_account_sync`'s full production wiring (account
// lookup -> keyring password -> real backend -> persist -> events)
// end-to-end without violating the "no live network calls in
// tests" rule.
// Uses a real (non-mock) `JmapEngine::with_credentials`, wired exactly like
// `build_mail_backend` wires it in production, pointed at a wiremock server standing
// in for the real JMAP host. This proves `run_account_sync`'s full production wiring
// (account lookup -> keyring password -> real HTTP backend -> persist -> events)
// end-to-end without violating the "no live network calls in tests" rule.
let mock_server = MockServer::start().await;
mount_single_message_jmap_stubs(&mock_server, "acct-jmap-real-1").await;
let (db, _dir) = DatabaseEngine::connect_ephemeral()
.await
.expect("ephemeral db");
@@ -460,20 +816,24 @@ mod tests {
let mut events = event_bus.subscribe_events();
let secrets = SecretManager::mock();
let config = sample_jmap_account("acct-jmap-real-1");
let config = sample_jmap_account_at("acct-jmap-real-1", &mock_server.uri());
db.save_account(&config).await.expect("save account");
secrets
.set_secret(&config.keyring_secret_key, "irrelevant-jmap-token")
.expect("store credential in mock vault");
let synced = run_account_sync(&db, &secrets, &event_bus, "acct-jmap-real-1")
.await
.expect("sync succeeds");
// JmapEngine::sync_folders() returns 3 static folders; sync_messages
// returns the same 1 static message per folder regardless of
// folder_id, so 3 folders => 3 processed messages (deduplicated to
// 1 stored row by `save_email`'s INSERT OR REPLACE on message id).
assert_eq!(synced, 3);
let filter_engine = empty_filter_engine();
let synced = run_account_sync(
&db,
&secrets,
&event_bus,
&filter_engine,
"acct-jmap-real-1",
)
.await
.expect("sync succeeds");
// The wiremock stubs advertise 1 folder containing 1 message.
assert_eq!(synced, 1);
let persisted = db
.get_message("jmap-msg-1")
@@ -487,6 +847,15 @@ mod tests {
account_id: Some("acct-jmap-real-1".to_string())
}
);
assert_eq!(
events.recv().await.expect("progress event"),
CoreEvent::SyncProgress {
account_id: Some("acct-jmap-real-1".to_string()),
folder_id: "jmap-inbox".to_string(),
fetched: 1,
total: 1,
}
);
assert_eq!(
events.recv().await.expect("complete event"),
CoreEvent::SyncCompleted {
@@ -505,9 +874,16 @@ mod tests {
let mut events = event_bus.subscribe_events();
let secrets = SecretManager::mock();
let err = run_account_sync(&db, &secrets, &event_bus, "acct-does-not-exist")
.await
.expect_err("unknown account fails");
let filter_engine = empty_filter_engine();
let err = run_account_sync(
&db,
&secrets,
&event_bus,
&filter_engine,
"acct-does-not-exist",
)
.await
.expect_err("unknown account fails");
assert!(matches!(err, SyncError::AccountNotFound(_)));
match events.recv().await.expect("error event") {
@@ -533,7 +909,8 @@ mod tests {
db.save_account(&config).await.expect("save account");
// Deliberately never calls `secrets.set_secret(...)`.
let err = run_account_sync(&db, &secrets, &event_bus, "acct-no-cred-1")
let filter_engine = empty_filter_engine();
let err = run_account_sync(&db, &secrets, &event_bus, &filter_engine, "acct-no-cred-1")
.await
.expect_err("missing credential fails");
assert!(matches!(err, SyncError::Vault(_)));
@@ -556,7 +933,8 @@ mod tests {
let mut events = event_bus.subscribe_events();
let secrets = SecretManager::mock();
let total = run_all_accounts_sync(&db, &secrets, &event_bus).await;
let filter_engine = empty_filter_engine();
let total = run_all_accounts_sync(&db, &secrets, &event_bus, &filter_engine).await;
assert_eq!(total, 0);
assert_eq!(
@@ -571,14 +949,19 @@ mod tests {
#[tokio::test]
async fn run_all_accounts_sync_persists_across_multiple_accounts() {
let mock_server_a = MockServer::start().await;
let mock_server_b = MockServer::start().await;
mount_single_message_jmap_stubs(&mock_server_a, "acct-all-a").await;
mount_single_message_jmap_stubs(&mock_server_b, "acct-all-b").await;
let (db, _dir) = DatabaseEngine::connect_ephemeral()
.await
.expect("ephemeral db");
let event_bus = EventBus::new();
let secrets = SecretManager::mock();
let config_a = sample_jmap_account("acct-all-a");
let config_b = sample_jmap_account("acct-all-b");
let config_a = sample_jmap_account_at("acct-all-a", &mock_server_a.uri());
let config_b = sample_jmap_account_at("acct-all-b", &mock_server_b.uri());
db.save_account(&config_a).await.expect("save account a");
db.save_account(&config_b).await.expect("save account b");
secrets
@@ -588,9 +971,10 @@ mod tests {
.set_secret(&config_b.keyring_secret_key, "token-b")
.expect("store credential b");
let total = run_all_accounts_sync(&db, &secrets, &event_bus).await;
// 2 accounts * 3 folders each = 6 processed messages.
assert_eq!(total, 6);
let filter_engine = empty_filter_engine();
let total = run_all_accounts_sync(&db, &secrets, &event_bus, &filter_engine).await;
// 2 accounts * 1 folder * 1 message each = 2 processed messages.
assert_eq!(total, 2);
assert_eq!(event_bus.current_state().status, EngineStatus::Idle);
}
@@ -607,7 +991,8 @@ mod tests {
db.save_account(&config).await.expect("save account");
// No credential stored in the vault for this account.
let total = run_all_accounts_sync(&db, &secrets, &event_bus).await;
let filter_engine = empty_filter_engine();
let total = run_all_accounts_sync(&db, &secrets, &event_bus, &filter_engine).await;
assert_eq!(total, 0);
assert_eq!(
@@ -625,4 +1010,326 @@ mod tests {
CoreEvent::SyncCompleted { account_id: None }
);
}
#[tokio::test]
async fn sync_with_backend_fires_immediate_mark_read_action_on_match() {
let (db, _dir) = DatabaseEngine::connect_ephemeral()
.await
.expect("ephemeral db");
let event_bus = EventBus::new();
let rule = nuncio_filter::NsqlParser::parse_rule(
"Urgent Auto-Read",
1,
"WHERE subject CONTAINS 'Urgent' ACTION MARK READ",
)
.expect("parse rule");
let filter_engine = FilterEngine::new(vec![rule]).expect("compile rule");
let mock = MockMailBackend::new();
mock.add_folder(Folder {
id: "inbox".to_string(),
name: "Inbox".to_string(),
total_messages: 1,
unread_messages: 1,
});
mock.add_message(mock_email("m-urgent", "inbox", "Urgent: server down"));
let synced = sync_with_backend(&db, &event_bus, &mock, &filter_engine, None)
.await
.expect("sync succeeds");
assert_eq!(synced, 1);
let stored = db.get_message("m-urgent").await.expect("message persisted");
assert!(
stored.read,
"MARK READ action must genuinely flip the stored read flag"
);
let logs = db
.list_filter_execution_logs(10)
.await
.expect("list execution logs");
assert_eq!(logs.len(), 1);
assert_eq!(logs[0].message_id, "m-urgent");
let pending = db
.list_pending_mutations(10)
.await
.expect("list pending mutations");
assert!(
pending.is_empty(),
"an immediate action must not enqueue an outbox mutation"
);
}
#[tokio::test]
async fn sync_with_backend_enqueues_outbox_mutation_for_remote_action_on_match() {
let (db, _dir) = DatabaseEngine::connect_ephemeral()
.await
.expect("ephemeral db");
let event_bus = EventBus::new();
let rule = nuncio_filter::NsqlParser::parse_rule(
"Archive Rule",
1,
"WHERE subject CONTAINS 'Archive Me' ACTION MOVE TO 'Archive'",
)
.expect("parse rule");
let filter_engine = FilterEngine::new(vec![rule]).expect("compile rule");
let mock = MockMailBackend::new();
mock.add_folder(Folder {
id: "inbox".to_string(),
name: "Inbox".to_string(),
total_messages: 1,
unread_messages: 1,
});
mock.add_message(mock_email("m-archive", "inbox", "Please Archive Me"));
let synced = sync_with_backend(&db, &event_bus, &mock, &filter_engine, None)
.await
.expect("sync succeeds");
assert_eq!(synced, 1);
let pending = db
.list_pending_mutations(10)
.await
.expect("list pending mutations");
assert_eq!(pending.len(), 1);
assert_eq!(pending[0].message_id, "m-archive");
assert_eq!(pending[0].mutation_type, "MOVE");
assert_eq!(pending[0].status, "pending");
assert!(pending[0].payload.contains("Archive"));
let logs = db
.list_filter_execution_logs(10)
.await
.expect("list execution logs");
assert_eq!(logs.len(), 1);
assert_eq!(logs[0].message_id, "m-archive");
// The remote action must not have applied a local read-flag change.
let stored = db
.get_message("m-archive")
.await
.expect("message persisted");
assert!(!stored.read);
}
#[tokio::test]
async fn sync_with_backend_non_matching_message_produces_no_filter_side_effects() {
let (db, _dir) = DatabaseEngine::connect_ephemeral()
.await
.expect("ephemeral db");
let event_bus = EventBus::new();
let rule = nuncio_filter::NsqlParser::parse_rule(
"Urgent Auto-Read",
1,
"WHERE subject CONTAINS 'Urgent' ACTION MARK READ",
)
.expect("parse rule");
let filter_engine = FilterEngine::new(vec![rule]).expect("compile rule");
let mock = MockMailBackend::new();
mock.add_folder(Folder {
id: "inbox".to_string(),
name: "Inbox".to_string(),
total_messages: 1,
unread_messages: 1,
});
mock.add_message(mock_email("m-plain", "inbox", "Just a normal update"));
let synced = sync_with_backend(&db, &event_bus, &mock, &filter_engine, None)
.await
.expect("sync succeeds");
assert_eq!(synced, 1);
let stored = db.get_message("m-plain").await.expect("message persisted");
assert!(!stored.read, "non-matching message must stay untouched");
let logs = db
.list_filter_execution_logs(10)
.await
.expect("list execution logs");
assert!(logs.is_empty(), "no rule matched, so no log should exist");
let pending = db
.list_pending_mutations(10)
.await
.expect("list pending mutations");
assert!(
pending.is_empty(),
"no rule matched, so no mutation should be enqueued"
);
}
#[tokio::test]
async fn sync_with_backend_does_not_refire_filters_on_a_repeat_sync_of_the_same_message() {
// Sync is not yet incremental: a real backend re-reports messages it
// already handed over on every sync pass. This proves a message's
// filter actions fire exactly once across repeated syncs rather than
// once per sync cycle -- the regression this guard exists for.
let (db, _dir) = DatabaseEngine::connect_ephemeral()
.await
.expect("ephemeral db");
let event_bus = EventBus::new();
let mark_read_rule = nuncio_filter::NsqlParser::parse_rule(
"Urgent Auto-Read",
1,
"WHERE subject CONTAINS 'Urgent' ACTION MARK READ",
)
.expect("parse rule");
let move_rule = nuncio_filter::NsqlParser::parse_rule(
"Archive Rule",
2,
"WHERE subject CONTAINS 'Urgent' ACTION MOVE TO 'Archive'",
)
.expect("parse rule");
let filter_engine =
FilterEngine::new(vec![mark_read_rule, move_rule]).expect("compile rules");
let mock = MockMailBackend::new();
mock.add_folder(Folder {
id: "inbox".to_string(),
name: "Inbox".to_string(),
total_messages: 1,
unread_messages: 1,
});
mock.add_message(mock_email("m-repeat", "inbox", "Urgent: server down"));
// First sync: the message is genuinely new, so both rules must fire.
let synced_first = sync_with_backend(&db, &event_bus, &mock, &filter_engine, None)
.await
.expect("first sync succeeds");
assert_eq!(synced_first, 1);
let stored_after_first = db
.get_message("m-repeat")
.await
.expect("message persisted after first sync");
assert!(stored_after_first.read);
let logs_after_first = db
.list_filter_execution_logs(10)
.await
.expect("list execution logs after first sync");
assert_eq!(logs_after_first.len(), 2, "both rules should fire once");
let pending_after_first = db
.list_pending_mutations(10)
.await
.expect("list pending mutations after first sync");
assert_eq!(pending_after_first.len(), 1);
// Manually flip the message back to unread, exactly as a user
// reading and then un-reading it would -- proves the second sync
// doesn't re-flip it back to read via a repeat MARK READ fire.
db.set_message_read("m-repeat", false)
.await
.expect("manually mark unread");
// Second sync: the mock backend reports the SAME message again
// (unchanged id), simulating a non-incremental re-sync.
let synced_second = sync_with_backend(&db, &event_bus, &mock, &filter_engine, None)
.await
.expect("second sync succeeds");
assert_eq!(synced_second, 1);
let stored_after_second = db
.get_message("m-repeat")
.await
.expect("message persisted after second sync");
assert!(
!stored_after_second.read,
"a re-sync of an already-seen message must not re-fire MARK READ"
);
let logs_after_second = db
.list_filter_execution_logs(10)
.await
.expect("list execution logs after second sync");
assert_eq!(
logs_after_second.len(),
2,
"no new execution log entries should be written on a repeat sync"
);
let pending_after_second = db
.list_pending_mutations(10)
.await
.expect("list pending mutations after second sync");
assert_eq!(
pending_after_second.len(),
1,
"no new outbox mutation should be enqueued on a repeat sync"
);
}
#[tokio::test]
async fn sync_with_backend_persists_and_threads_the_folder_checkpoint_across_syncs() {
// The regression this story exists to kill: the returned checkpoint
// must be persisted and fed back into the NEXT sync, so the second
// sync is genuinely incremental rather than a full re-fetch. Proven by
// capturing the exact `since_state` the backend received on each call.
let (db, _dir) = DatabaseEngine::connect_ephemeral()
.await
.expect("ephemeral db");
let event_bus = EventBus::new();
let mock = MockMailBackend::new();
mock.set_returned_state("uidnext-4242");
mock.add_folder(Folder {
id: "inbox".to_string(),
name: "Inbox".to_string(),
total_messages: 1,
unread_messages: 1,
});
mock.add_message(mock_email("m-inc-1", "inbox", "First"));
let filter_engine = empty_filter_engine();
// First sync: no checkpoint exists yet, so the backend is called with
// `None` and does a full fetch (one message).
let first = sync_with_backend(
&db,
&event_bus,
&mock,
&filter_engine,
Some("acct-inc-1".to_string()),
)
.await
.expect("first sync succeeds");
assert_eq!(first, 1);
// The returned checkpoint must now be persisted for (account, folder).
assert_eq!(
db.get_folder_sync_state("acct-inc-1", "inbox")
.await
.expect("read checkpoint"),
Some("uidnext-4242".to_string())
);
// Second sync: the daemon must feed the stored checkpoint back in. The
// mock returns nothing new for a non-None checkpoint -> narrower fetch.
let second = sync_with_backend(
&db,
&event_bus,
&mock,
&filter_engine,
Some("acct-inc-1".to_string()),
)
.await
.expect("second sync succeeds");
assert_eq!(second, 0, "an incremental second sync fetches nothing new");
// The proof of genuine incrementality: the backend saw `None` first,
// then the exact checkpoint the first sync returned.
assert_eq!(
mock.since_state_calls(),
vec![None, Some("uidnext-4242".to_string())],
);
}
}
@@ -0,0 +1,244 @@
//! Offline end-to-end test for the account lifecycle over gRPC.
//!
//! Boots a real `nunciod` daemon gRPC server
//! (`nunciod::grpc::serve_on_listener_with_overrides`) on an ephemeral
//! loopback port, backed by a real temp file-backed [`DatabaseEngine`] and a
//! [`SecretManager::mock`] vault (never the real OS keyring), with an injected
//! stand-in [`AccountConnectionTester`] standing in for real IMAP/SMTP network
//! I/O (via the [`AccountsEngineOverrides`] injection seam).
//!
//! Driving ONLY the authenticated `nuncio-proto` Accounts client with the
//! shared bearer token -- never reaching into daemon internals -- this proves
//! the whole lifecycle end to end:
//! a. `AddAccount` persists an account whose IMAP mode is `StartTls` and
//! SMTP mode is `Plain`.
//! b. `ListAccounts` reflects it AND round-trips both distinct TLS modes
//! (the previously silently-discarded fields).
//! c. `UpdateAccount` genuinely mutates the persisted config (confirmed via
//! a second `ListAccounts`).
//! d. `TestAccountConnection` reports the injected tester's GENUINE
//! per-protocol result (IMAP ok, SMTP failed with real error detail) --
//! never a fabricated OK.
//! e. `RemoveAccount` genuinely deletes it (a final `ListAccounts` is
//! empty).
//!
//! Fully offline: no real network, no real OS keyring, no live server.
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use nuncio_core::EventBus;
use nuncio_filter::FilterEngine;
use nuncio_proto::v1::{
AccountConfig, AccountProtocol, AddAccountRequest, ListAccountsRequest, RemoveAccountRequest,
TestAccountConnectionRequest, TlsMode, UpdateAccountRequest,
};
use nuncio_store::db::DatabaseEngine;
use nuncio_store::vault::{SecretManager, GRPC_TOKEN_ACCOUNT};
use nunciod::grpc::{
serve_on_listener_with_overrides, AccountConnectionReport, AccountConnectionTester,
AccountsEngineOverrides, CalendarEngineOverrides, ContactsEngineOverrides, MailEngineOverrides,
ProtocolProbe,
};
use std::sync::Arc;
use tokio::net::TcpListener;
const ACCOUNT_ID: &str = "acct-lifecycle-e2e-1";
const ACCOUNT_EMAIL: &str = "lifecycle-e2e@nuncio.mx";
const KEYRING_KEY: &str = "nuncio/acct-lifecycle-e2e-1";
/// Injected stand-in [`AccountConnectionTester`]: returns a fixed but honest
/// per-protocol report (IMAP succeeds, SMTP fails with real error detail) so
/// `TestAccountConnection` can be exercised end to end WITHOUT any live
/// server. The daemon's production tester performs a genuine bounded network
/// probe; this stub stands in for that real world exactly as the mock
/// mail/calendar/contacts backends do for their protocols.
struct StubConnectionTester;
#[tonic::async_trait]
impl AccountConnectionTester for StubConnectionTester {
async fn probe(
&self,
_config: &nuncio_core::AccountConfig,
_password: &str,
) -> AccountConnectionReport {
AccountConnectionReport {
imap: ProtocolProbe {
ok: true,
error: None,
},
smtp: ProtocolProbe {
ok: false,
error: Some("SMTP relay refused the connection".to_string()),
},
}
}
}
fn sample_account_config() -> AccountConfig {
AccountConfig {
id: ACCOUNT_ID.to_string(),
name: "Lifecycle E2E Account".to_string(),
email_address: ACCOUNT_EMAIL.to_string(),
protocol: AccountProtocol::ImapSmtp.into(),
server_host: "imap.nuncio.mx".to_string(),
server_port: 143,
use_tls: false,
imap_tls_mode: TlsMode::StartTls.into(),
smtp_tls_mode: TlsMode::Plain.into(),
keyring_secret_key: KEYRING_KEY.to_string(),
sync_interval_secs: 60,
smtp_host: "smtp.nuncio.mx".to_string(),
smtp_port: 25,
}
}
/// Boots the real daemon gRPC server on an ephemeral loopback port with an
/// injected [`AccountsEngineOverrides`]. Returns the server address and the
/// minted token; the `TempDir` guard is returned so the caller controls its
/// lifetime for the whole test.
async fn boot_daemon(
secrets: Arc<SecretManager>,
accounts_overrides: AccountsEngineOverrides,
) -> (std::net::SocketAddr, String, tempfile::TempDir) {
let dir = tempfile::tempdir().expect("create temp dir for file-backed db");
let db_path = dir.path().join("accounts_lifecycle_e2e.db");
let db = Arc::new(
DatabaseEngine::connect_file(&db_path, &secrets)
.await
.expect("open temp file-backed database"),
);
let token_bytes = secrets
.get_or_create_key_bytes(GRPC_TOKEN_ACCOUNT, 32)
.expect("mint gRPC bearer token from mock vault");
let token = hex::encode(token_bytes);
let listener = TcpListener::bind("127.0.0.1:0")
.await
.expect("bind ephemeral loopback port");
let addr = listener.local_addr().expect("listener has local addr");
let event_bus = Arc::new(EventBus::new());
let server_secrets = secrets.clone();
let server_token = token.clone();
let server_filter_engine = Arc::new(FilterEngine::new(Vec::new()).expect("empty rule set"));
tokio::spawn(async move {
let _ = serve_on_listener_with_overrides(
listener,
event_bus,
db,
server_filter_engine,
server_secrets,
server_token,
MailEngineOverrides::default(),
CalendarEngineOverrides::default(),
ContactsEngineOverrides::default(),
accounts_overrides,
)
.await;
});
(addr, token, dir)
}
#[tokio::test]
async fn account_lifecycle_round_trips_over_authenticated_grpc_offline() {
let secrets = Arc::new(SecretManager::mock());
let accounts_overrides = AccountsEngineOverrides {
connection_tester: Some(Arc::new(StubConnectionTester)),
};
let (addr, token, _db_dir) = boot_daemon(secrets, accounts_overrides).await;
let addr_str = addr.to_string();
let mut accounts_client = nuncio_proto::client::connect_accounts(&addr_str, &token)
.await
.expect("accounts client connects");
// ---- (a) AddAccount persists a StartTls/Plain account ----
let add_response = accounts_client
.add_account(AddAccountRequest {
config: Some(sample_account_config()),
password: "lifecycle-secret-password".to_string(),
})
.await
.expect("add_account succeeds")
.into_inner();
assert_eq!(add_response.id, ACCOUNT_ID);
// ---- (b) ListAccounts round-trips BOTH distinct TLS modes ----
let listed = accounts_client
.list_accounts(ListAccountsRequest {})
.await
.expect("list_accounts succeeds")
.into_inner()
.accounts;
assert_eq!(listed.len(), 1);
assert_eq!(listed[0].imap_tls_mode(), TlsMode::StartTls);
assert_eq!(listed[0].smtp_tls_mode(), TlsMode::Plain);
// ---- (c) UpdateAccount genuinely mutates the persisted config ----
let mut updated = sample_account_config();
updated.name = "Renamed Lifecycle Account".to_string();
updated.imap_tls_mode = TlsMode::ImplicitTls.into();
accounts_client
.update_account(UpdateAccountRequest {
config: Some(updated),
password: None,
})
.await
.expect("update_account succeeds");
let relisted = accounts_client
.list_accounts(ListAccountsRequest {})
.await
.expect("list_accounts succeeds")
.into_inner()
.accounts;
assert_eq!(relisted.len(), 1);
assert_eq!(relisted[0].name, "Renamed Lifecycle Account");
assert_eq!(relisted[0].imap_tls_mode(), TlsMode::ImplicitTls);
// Untouched fields survive the update.
assert_eq!(relisted[0].smtp_tls_mode(), TlsMode::Plain);
// ---- (d) TestAccountConnection reports the GENUINE per-protocol result ----
let report = accounts_client
.test_account_connection(TestAccountConnectionRequest {
id: ACCOUNT_ID.to_string(),
})
.await
.expect("test_account_connection succeeds")
.into_inner();
assert!(report.imap_ok);
assert!(!report.smtp_ok);
assert_eq!(report.imap_error, None);
assert_eq!(
report.smtp_error.as_deref(),
Some("SMTP relay refused the connection")
);
// ---- (e) RemoveAccount genuinely deletes it ----
accounts_client
.remove_account(RemoveAccountRequest {
id: ACCOUNT_ID.to_string(),
})
.await
.expect("remove_account succeeds");
let after_remove = accounts_client
.list_accounts(ListAccountsRequest {})
.await
.expect("list_accounts succeeds")
.into_inner()
.accounts;
assert!(after_remove.is_empty());
// A test against the now-removed account is an honest not-found, never a
// fabricated success.
let err = accounts_client
.test_account_connection(TestAccountConnectionRequest {
id: ACCOUNT_ID.to_string(),
})
.await
.expect_err("testing a removed account must fail");
assert_eq!(err.code(), tonic::Code::NotFound);
}
+196
View File
@@ -0,0 +1,196 @@
//! Full-daemon offline end-to-end test for the calendar vertical.
//!
//! Boots a real `nunciod` daemon gRPC server
//! (`nunciod::grpc::serve_on_listener_with_overrides`) on an ephemeral
//! loopback port, backed by a real temp file-backed [`DatabaseEngine`] and a
//! [`SecretManager::mock`] vault (never the real OS keyring), with an
//! injected [`nuncio_cal::MockCalendarBackend`] standing in for real network
//! I/O (via the [`nunciod::grpc::CalendarEngineOverrides`] injection seam).
//!
//! Driving ONLY the authenticated `nuncio-proto` gRPC client
//! (`connect_calendar`) with the shared bearer token -- never reaching into
//! daemon internals -- this proves:
//! a. `Calendar/Sync` fetches from the injected mock backend and persists
//! the result, awaiting full completion before returning, so the
//! synced events are immediately visible to `ListEvents`/`GetEvent`.
//! b. An unauthenticated call to `Calendar` is rejected with
//! `Code::Unauthenticated`, proving it is mounted behind its own
//! `BearerAuthInterceptor` exactly like every other service.
//!
//! Deterministic and fully offline: no fixed sleeps, no real network, no
//! real OS keyring.
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use nuncio_cal::MockCalendarBackend;
use nuncio_core::model::CalendarEvent;
use nuncio_core::EventBus;
use nuncio_filter::FilterEngine;
use nuncio_proto::v1::{CalendarSyncRequest, GetEventRequest, ListEventsRequest};
use nuncio_store::db::DatabaseEngine;
use nuncio_store::vault::{SecretManager, GRPC_TOKEN_ACCOUNT};
use nunciod::grpc::{
serve_on_listener_with_overrides, AccountsEngineOverrides, CalendarEngineOverrides,
ContactsEngineOverrides, MailEngineOverrides,
};
use std::sync::Arc;
use tokio::net::TcpListener;
use tonic::Code;
const ACCOUNT_ID: &str = "acct-cal-e2e-1";
const CALENDAR_ID: &str = "cal-e2e-work";
fn mock_event(id: &str, start: i64, end: i64) -> CalendarEvent {
CalendarEvent {
id: id.to_string(),
account_id: ACCOUNT_ID.to_string(),
calendar_id: CALENDAR_ID.to_string(),
summary: format!("E2E Event {id}"),
start_time: start,
end_time: end,
rrule: None,
location: Some("Conference Room".to_string()),
}
}
/// Boots the real daemon gRPC server, injecting `calendar_overrides`, on an
/// ephemeral loopback port backed by a real temp file-backed
/// [`DatabaseEngine`]. Returns the server address and the minted bearer
/// token; the `TempDir` guard is returned too so the caller controls its
/// lifetime for the whole test.
async fn boot_daemon(
secrets: Arc<SecretManager>,
calendar_overrides: CalendarEngineOverrides,
) -> (std::net::SocketAddr, String, tempfile::TempDir) {
let dir = tempfile::tempdir().expect("create temp dir for file-backed db");
let db_path = dir.path().join("calendar_e2e.db");
let db = Arc::new(
DatabaseEngine::connect_file(&db_path, &secrets)
.await
.expect("open temp file-backed database"),
);
let token_bytes = secrets
.get_or_create_key_bytes(GRPC_TOKEN_ACCOUNT, 32)
.expect("mint gRPC bearer token from mock vault");
let token = hex::encode(token_bytes);
let listener = TcpListener::bind("127.0.0.1:0")
.await
.expect("bind ephemeral loopback port");
let addr = listener.local_addr().expect("listener has local addr");
let event_bus = Arc::new(EventBus::new());
let server_secrets = secrets.clone();
let server_token = token.clone();
let server_filter_engine = Arc::new(FilterEngine::new(Vec::new()).expect("empty rule set"));
tokio::spawn(async move {
let _ = serve_on_listener_with_overrides(
listener,
event_bus,
db,
server_filter_engine,
server_secrets,
server_token,
MailEngineOverrides::default(),
calendar_overrides,
ContactsEngineOverrides::default(),
AccountsEngineOverrides::default(),
)
.await;
});
(addr, token, dir)
}
/// The full offline, deterministic end-to-end proof of the calendar vertical:
/// `Sync` against an injected mock backend, then `ListEvents`/`GetEvent`
/// immediately reflect the persisted result -- ALL over the authenticated
/// `nuncio.v1.Calendar` gRPC API against a live in-process daemon, with
/// protocol I/O fully mocked.
#[tokio::test]
async fn calendar_sync_then_list_and_get_round_trip_over_authenticated_grpc() {
let secrets = Arc::new(SecretManager::mock());
let mock_backend = MockCalendarBackend::new();
mock_backend.add_event(mock_event("evt-e2e-1", 1_700_000_000, 1_700_003_600));
mock_backend.add_event(mock_event("evt-e2e-2", 1_700_010_000, 1_700_013_600));
// Outside the sync window this test requests -- must not be synced.
mock_backend.add_event(mock_event(
"evt-e2e-out-of-window",
1_900_000_000,
1_900_003_600,
));
let calendar_overrides = CalendarEngineOverrides {
calendar_backend: Some(Arc::new(mock_backend)),
};
let (addr, token, _db_dir) = boot_daemon(secrets, calendar_overrides).await;
let addr_str = addr.to_string();
let mut calendar_client = nuncio_proto::client::connect_calendar(&addr_str, &token)
.await
.expect("calendar client connects");
// ---- (a) Sync fetches from the injected mock backend and persists ----
let sync_response = calendar_client
.sync(CalendarSyncRequest {
account_id: ACCOUNT_ID.to_string(),
calendar_id: CALENDAR_ID.to_string(),
start_window: 1_699_999_000,
end_window: 1_700_020_000,
})
.await
.expect("sync succeeds")
.into_inner();
assert_eq!(sync_response.synced_count, 2);
// Immediately visible, no fixed sleep: `Sync` only returns once the
// fetch-and-persist work has fully completed.
let list_response = calendar_client
.list_events(ListEventsRequest {
account_id: ACCOUNT_ID.to_string(),
calendar_id: CALENDAR_ID.to_string(),
start_window: 1_699_999_000,
end_window: 1_700_020_000,
})
.await
.expect("list_events succeeds")
.into_inner();
assert_eq!(list_response.events.len(), 2);
let synced_ids: Vec<&str> = list_response.events.iter().map(|e| e.id.as_str()).collect();
assert!(synced_ids.contains(&"evt-e2e-1"));
assert!(synced_ids.contains(&"evt-e2e-2"));
assert!(!synced_ids.contains(&"evt-e2e-out-of-window"));
let get_response = calendar_client
.get_event(GetEventRequest {
event_id: "evt-e2e-1".to_string(),
})
.await
.expect("get_event succeeds")
.into_inner();
let event = get_response.event.expect("event present in response");
assert_eq!(event.summary, "E2E Event evt-e2e-1");
assert_eq!(event.location.as_deref(), Some("Conference Room"));
// ---- (b) An unauthenticated call is rejected ----
// Uses the raw generated client (bypassing `connect_calendar`'s
// bearer-token interceptor) to prove `Calendar` is mounted behind its
// own `BearerAuthInterceptor`, exactly like every other service.
let mut unauthenticated_client =
nuncio_proto::v1::calendar_client::CalendarClient::connect(format!("http://{addr_str}"))
.await
.expect("client connects at the transport level without any auth");
let err = unauthenticated_client
.list_events(ListEventsRequest {
account_id: ACCOUNT_ID.to_string(),
calendar_id: CALENDAR_ID.to_string(),
start_window: 0,
end_window: i64::MAX,
})
.await
.expect_err("missing bearer token must be rejected");
assert_eq!(err.code(), Code::Unauthenticated);
}
+238
View File
@@ -0,0 +1,238 @@
//! Full-daemon offline end-to-end test for the contacts vertical.
//!
//! Boots a real `nunciod` daemon gRPC server
//! (`nunciod::grpc::serve_on_listener_with_overrides`) on an ephemeral
//! loopback port, backed by a real temp file-backed [`DatabaseEngine`] and a
//! [`SecretManager::mock`] vault (never the real OS keyring), with an
//! injected [`nuncio_contacts::MockContactsBackend`] standing in for real
//! network I/O (via the [`nunciod::grpc::ContactsEngineOverrides`] injection
//! seam).
//!
//! Driving ONLY the authenticated `nuncio-proto` gRPC client
//! (`connect_contacts`) with the shared bearer token -- never reaching into
//! daemon internals -- this proves:
//! a. `Contacts/Sync` fetches from the injected mock backend and persists
//! the result, awaiting full completion before returning, so the
//! synced contacts are immediately visible to `ListContacts`/
//! `GetContact`.
//! b. `Contacts/CreateContact` persists a locally-authored contact
//! directly to the daemon's real store, and it is visible via a
//! SECOND, SEPARATE client connection against the same daemon --
//! proving real cross-process-style persistence, not just
//! within-process state.
//! c. An unauthenticated call to `Contacts` is rejected with
//! `Code::Unauthenticated`, proving it is mounted behind its own
//! `BearerAuthInterceptor` exactly like every other service.
//!
//! Deterministic and fully offline: no fixed sleeps, no real network, no
//! real OS keyring.
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use nuncio_contacts::{Contact, MockContactsBackend};
use nuncio_core::EventBus;
use nuncio_filter::FilterEngine;
use nuncio_proto::v1::{
ContactEmail as ContactEmailProto, ContactsSyncRequest, CreateContactRequest,
GetContactRequest, ListContactsRequest,
};
use nuncio_store::db::DatabaseEngine;
use nuncio_store::vault::{SecretManager, GRPC_TOKEN_ACCOUNT};
use nunciod::grpc::{
serve_on_listener_with_overrides, AccountsEngineOverrides, CalendarEngineOverrides,
ContactsEngineOverrides, MailEngineOverrides,
};
use std::sync::Arc;
use tokio::net::TcpListener;
use tonic::Code;
const ACCOUNT_ID: &str = "acct-contacts-e2e-1";
fn mock_contact(id: &str, display_name: &str) -> Contact {
let mut contact = Contact::new(display_name, format!("{display_name}@nuncio.mx"));
contact.id = id.to_string();
contact.account_id = Some(ACCOUNT_ID.to_string());
contact
}
/// Boots the real daemon gRPC server, injecting `contacts_overrides`, on an
/// ephemeral loopback port backed by a real temp file-backed
/// [`DatabaseEngine`]. Returns the server address and the minted bearer
/// token; the `TempDir` guard is returned too so the caller controls its
/// lifetime for the whole test.
async fn boot_daemon(
secrets: Arc<SecretManager>,
contacts_overrides: ContactsEngineOverrides,
) -> (std::net::SocketAddr, String, tempfile::TempDir) {
let dir = tempfile::tempdir().expect("create temp dir for file-backed db");
let db_path = dir.path().join("contacts_e2e.db");
let db = Arc::new(
DatabaseEngine::connect_file(&db_path, &secrets)
.await
.expect("open temp file-backed database"),
);
let token_bytes = secrets
.get_or_create_key_bytes(GRPC_TOKEN_ACCOUNT, 32)
.expect("mint gRPC bearer token from mock vault");
let token = hex::encode(token_bytes);
let listener = TcpListener::bind("127.0.0.1:0")
.await
.expect("bind ephemeral loopback port");
let addr = listener.local_addr().expect("listener has local addr");
let event_bus = Arc::new(EventBus::new());
let server_secrets = secrets.clone();
let server_token = token.clone();
let server_filter_engine = Arc::new(FilterEngine::new(Vec::new()).expect("empty rule set"));
tokio::spawn(async move {
let _ = serve_on_listener_with_overrides(
listener,
event_bus,
db,
server_filter_engine,
server_secrets,
server_token,
MailEngineOverrides::default(),
CalendarEngineOverrides::default(),
contacts_overrides,
AccountsEngineOverrides::default(),
)
.await;
});
(addr, token, dir)
}
/// The full offline, deterministic end-to-end proof of the contacts
/// vertical: `Sync` against an injected mock backend, then `ListContacts`/
/// `GetContact` immediately reflect the persisted result; `CreateContact`
/// persists a locally-authored contact that a SECOND, independent client
/// connection can then read back -- ALL over the authenticated
/// `nuncio.v1.Contacts` gRPC API against a live in-process daemon, with
/// protocol I/O fully mocked.
#[tokio::test]
async fn contacts_sync_create_and_list_round_trip_over_authenticated_grpc() {
let secrets = Arc::new(SecretManager::mock());
let mock_backend = MockContactsBackend::new();
mock_backend.add_contact(mock_contact("ct-e2e-1", "E2E Contact One"));
mock_backend.add_contact(mock_contact("ct-e2e-2", "E2E Contact Two"));
let contacts_overrides = ContactsEngineOverrides {
contacts_backend: Some(Arc::new(mock_backend)),
};
let (addr, token, _db_dir) = boot_daemon(secrets, contacts_overrides).await;
let addr_str = addr.to_string();
let mut contacts_client = nuncio_proto::client::connect_contacts(&addr_str, &token)
.await
.expect("contacts client connects");
// ---- (a) Sync fetches from the injected mock backend and persists ----
let sync_response = contacts_client
.sync(ContactsSyncRequest {
account_id: ACCOUNT_ID.to_string(),
})
.await
.expect("sync succeeds")
.into_inner();
assert_eq!(sync_response.synced_count, 2);
// Immediately visible, no fixed sleep: `Sync` only returns once the
// fetch-and-persist work has fully completed.
let list_response = contacts_client
.list_contacts(ListContactsRequest {
account_id: ACCOUNT_ID.to_string(),
})
.await
.expect("list_contacts succeeds")
.into_inner();
assert_eq!(list_response.contacts.len(), 2);
let synced_ids: Vec<&str> = list_response
.contacts
.iter()
.map(|c| c.id.as_str())
.collect();
assert!(synced_ids.contains(&"ct-e2e-1"));
assert!(synced_ids.contains(&"ct-e2e-2"));
let get_response = contacts_client
.get_contact(GetContactRequest {
contact_id: "ct-e2e-1".to_string(),
})
.await
.expect("get_contact succeeds")
.into_inner();
let contact = get_response.contact.expect("contact present in response");
assert_eq!(contact.display_name, "E2E Contact One");
// ---- (b) CreateContact persists real local state, visible via a
// SECOND, independent client connection ----
let create_response = contacts_client
.create_contact(CreateContactRequest {
account_id: ACCOUNT_ID.to_string(),
display_name: "CLI Added Contact".to_string(),
organization: Some("Kof22".to_string()),
emails: vec![ContactEmailProto {
email: "cli.added@kof22.com".to_string(),
label: "work".to_string(),
is_primary: true,
}],
phones: vec![],
})
.await
.expect("create_contact succeeds")
.into_inner();
let created_id = create_response
.contact
.expect("created contact present in response")
.id;
// A brand-new client connection (simulating a second, separate CLI
// invocation against the same running daemon) sees the created contact
// -- proving real persistence through the daemon's own store, not
// process-local state.
let mut second_client = nuncio_proto::client::connect_contacts(&addr_str, &token)
.await
.expect("second contacts client connects");
let second_list = second_client
.list_contacts(ListContactsRequest {
account_id: ACCOUNT_ID.to_string(),
})
.await
.expect("list_contacts succeeds on second connection")
.into_inner();
assert_eq!(second_list.contacts.len(), 3);
let second_ids: Vec<&str> = second_list.contacts.iter().map(|c| c.id.as_str()).collect();
assert!(second_ids.contains(&created_id.as_str()));
let second_get = second_client
.get_contact(GetContactRequest {
contact_id: created_id,
})
.await
.expect("get_contact succeeds on second connection")
.into_inner();
let created_contact = second_get.contact.expect("contact present in response");
assert_eq!(created_contact.display_name, "CLI Added Contact");
assert_eq!(created_contact.organization.as_deref(), Some("Kof22"));
// ---- (c) An unauthenticated call is rejected ----
// Uses the raw generated client (bypassing `connect_contacts`'s
// bearer-token interceptor) to prove `Contacts` is mounted behind its
// own `BearerAuthInterceptor`, exactly like every other service.
let mut unauthenticated_client =
nuncio_proto::v1::contacts_client::ContactsClient::connect(format!("http://{addr_str}"))
.await
.expect("client connects at the transport level without any auth");
let err = unauthenticated_client
.list_contacts(ListContactsRequest {
account_id: ACCOUNT_ID.to_string(),
})
.await
.expect_err("missing bearer token must be rejected");
assert_eq!(err.code(), Code::Unauthenticated);
}
@@ -0,0 +1,267 @@
//! Offline end-to-end test for the `nuncio.v1.Filters.Triage` streaming RPC.
//!
//! Boots a real `nunciod` daemon gRPC server (`nunciod::grpc::
//! serve_on_listener`) on an ephemeral loopback port, backed by a real
//! temp file-backed [`DatabaseEngine`] seeded directly via
//! [`DatabaseEngine::save_email`] (standing in for messages that were synced
//! before the active rule set existed) and a live [`FilterEngine`] carrying
//! a real persisted rule.
//!
//! Drives ONLY the authenticated `nuncio-proto` `FiltersClient::triage` call
//! against the live server, proving:
//! a. the final cumulative `matched_count`/`actions_applied_count` are
//! correct against a store containing both matching and non-matching
//! messages;
//! b. matching messages show the REAL side effects `apply_filter_actions`
//! produces (the read flag flips for an immediate `MARK READ` action;
//! an outbox row is queued for a remote `MOVE TO` action) -- not a
//! fabricated count with no backing state change;
//! c. a store bigger than one `chunk_size` genuinely streams more than one
//! non-final `TriageProgress` message, proving this is a real
//! multi-chunk stream rather than one batch dressed up as a stream.
//!
//! Fully offline and deterministic: no real network, no real OS keyring
//! (`SecretManager::mock`), no fixed sleeps.
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use nuncio_core::model::Email;
use nuncio_core::EventBus;
use nuncio_filter::{FilterEngine, NsqlParser};
use nuncio_proto::v1::filters_client::FiltersClient;
use nuncio_proto::v1::TriageRequest;
use nuncio_store::db::DatabaseEngine;
use nuncio_store::vault::{SecretManager, GRPC_TOKEN_ACCOUNT};
use nunciod::grpc::serve_on_listener;
use std::sync::Arc;
use tokio::net::TcpListener;
const ACCOUNT_ID: &str = "acct-triage-e2e-1";
fn sample_email(id: &str, subject: &str) -> Email {
Email {
id: id.to_string(),
account_id: ACCOUNT_ID.to_string(),
folder_id: "inbox".to_string(),
subject: subject.to_string(),
sender: "alice@nuncio.mx".to_string(),
recipient: "triage-e2e@nuncio.mx".to_string(),
received_at: 1_700_000_000,
read: false,
body_plain: Some("body".to_string()),
body_html: None,
attachments: Vec::new(),
}
}
/// Boots the real daemon gRPC server (bearer-token authenticated) on an
/// ephemeral loopback port, backed by a real temp file-backed
/// `DatabaseEngine` and the given live `FilterEngine`. Returns the server
/// address, the minted token, the `DatabaseEngine` handle (used to seed
/// messages before triggering `Triage`), and the `TempDir` guard, which the
/// caller must hold for as long as it talks to the server.
async fn boot_daemon(
secrets: Arc<SecretManager>,
filter_engine: Arc<FilterEngine>,
) -> (
std::net::SocketAddr,
String,
Arc<DatabaseEngine>,
tempfile::TempDir,
) {
let dir = tempfile::tempdir().expect("create temp dir for file-backed db");
let db_path = dir.path().join("triage_e2e.db");
let db = Arc::new(
DatabaseEngine::connect_file(&db_path, &secrets)
.await
.expect("open temp file-backed database"),
);
let token_bytes = secrets
.get_or_create_key_bytes(GRPC_TOKEN_ACCOUNT, 32)
.expect("mint gRPC bearer token from mock vault");
let token = hex::encode(token_bytes);
let listener = TcpListener::bind("127.0.0.1:0")
.await
.expect("bind ephemeral loopback port");
let addr = listener.local_addr().expect("listener has local addr");
let event_bus = Arc::new(EventBus::new());
let server_db = db.clone();
let server_secrets = secrets.clone();
let server_token = token.clone();
tokio::spawn(async move {
let _ = serve_on_listener(
listener,
event_bus,
server_db,
filter_engine,
server_secrets,
server_token,
)
.await;
});
(addr, token, db, dir)
}
fn authed_request<T>(token: &str, message: T) -> tonic::Request<T> {
let mut request = tonic::Request::new(message);
request.metadata_mut().insert(
"authorization",
format!("Bearer {token}")
.parse()
.expect("valid ascii metadata value"),
);
request
}
#[tokio::test]
async fn triage_applies_live_rule_retroactively_and_reports_real_cumulative_counts() {
let secrets = Arc::new(SecretManager::mock());
let mark_read_rule = NsqlParser::parse_rule(
"Urgent Auto-Read",
1,
"WHERE subject CONTAINS 'Urgent' ACTION MARK READ",
)
.expect("parse mark-read rule");
let move_rule = NsqlParser::parse_rule(
"Archive Rule",
2,
"WHERE subject CONTAINS 'Archive Me' ACTION MOVE TO 'Archive'",
)
.expect("parse move rule");
let filter_engine =
Arc::new(FilterEngine::new(vec![mark_read_rule, move_rule]).expect("compile rules"));
let (addr, token, db, _dir) = boot_daemon(secrets, filter_engine).await;
// Seed messages directly into the store, standing in for mail that was
// synced BEFORE these rules existed -- exactly the scenario `Triage` is
// for. Two match (one per rule), one does not.
db.save_email(&sample_email("m-urgent", "Urgent: server down"))
.await
.expect("seed urgent message");
db.save_email(&sample_email("m-archive", "Please Archive Me"))
.await
.expect("seed archive message");
db.save_email(&sample_email("m-plain", "Just saying hi"))
.await
.expect("seed non-matching message");
let mut client = FiltersClient::connect(format!("http://{addr}"))
.await
.expect("client connects");
let mut stream = client
.triage(authed_request(
&token,
TriageRequest {
rule_id: None,
chunk_size: 100,
},
))
.await
.expect("triage call succeeds")
.into_inner();
let mut updates = Vec::new();
while let Some(progress) = stream.message().await.expect("stream message") {
updates.push(progress);
}
let last = updates.last().expect("at least one progress update");
assert!(last.done, "final update must be marked done");
assert_eq!(last.scanned_count, 3);
assert_eq!(
last.matched_count, 2,
"exactly the two messages with a rule match count as matched"
);
assert_eq!(
last.actions_applied_count, 2,
"one action applied per matching message"
);
// Real side effects, not a fabricated count: the MARK READ action
// actually flipped the stored read flag...
let urgent = db.get_message("m-urgent").await.expect("get urgent");
assert!(urgent.read, "MARK READ must genuinely flip the read flag");
// ...and the MOVE TO action actually queued a real outbox mutation.
let pending = db
.list_pending_mutations(10)
.await
.expect("list pending mutations");
assert_eq!(
pending.len(),
1,
"the remote MOVE action must enqueue exactly one outbox mutation"
);
assert_eq!(pending[0].message_id, "m-archive");
let plain = db.get_message("m-plain").await.expect("get plain");
assert!(!plain.read, "a non-matching message must be left untouched");
}
#[tokio::test]
async fn triage_streams_multiple_chunks_for_a_store_bigger_than_one_chunk() {
let secrets = Arc::new(SecretManager::mock());
let filter_engine = Arc::new(FilterEngine::new(Vec::new()).expect("empty rule set"));
let (addr, token, db, _dir) = boot_daemon(secrets, filter_engine).await;
// Five messages against a chunk size of two forces three chunks
// (2 + 2 + 1), so the stream must carry at least two non-final progress
// updates before the final `done: true` -- proof this is a real
// multi-chunk stream, not one batch disguised as one.
for i in 0..5 {
db.save_email(&sample_email(
&format!("m-{i:02}"),
&format!("Message number {i}"),
))
.await
.expect("seed message");
}
let mut client = FiltersClient::connect(format!("http://{addr}"))
.await
.expect("client connects");
let mut stream = client
.triage(authed_request(
&token,
TriageRequest {
rule_id: None,
chunk_size: 2,
},
))
.await
.expect("triage call succeeds")
.into_inner();
let mut updates = Vec::new();
while let Some(progress) = stream.message().await.expect("stream message") {
updates.push(progress);
}
let non_final: Vec<_> = updates.iter().filter(|p| !p.done).collect();
assert!(
non_final.len() > 1,
"expected more than one non-final progress update for a multi-chunk scan, got {}",
non_final.len()
);
let last = updates.last().expect("at least one progress update");
assert!(last.done);
assert_eq!(last.scanned_count, 5);
// Cumulative counts must monotonically increase across the stream --
// never reset or double-count between chunks.
let mut prev_scanned = 0u64;
for update in &updates {
assert!(update.scanned_count >= prev_scanned);
prev_scanned = update.scanned_count;
}
}
+111
View File
@@ -154,3 +154,114 @@ async fn accepts_correct_bearer_token_and_returns_real_engine_status() {
assert_eq!(response.engine_status, "Syncing");
assert_eq!(response.version, env!("CARGO_PKG_VERSION"));
}
/// Confirms `Filters` newly added management RPCs are mounted behind the
/// SAME `BearerAuthInterceptor` as every other service -- an unauthenticated
/// call to `UpdateRule` must be rejected exactly like an unauthenticated
/// call to `System/GetStatus` above, with no carve-out for "just one more"
/// RPC.
#[tokio::test]
async fn rejects_calls_to_new_filters_rpcs_without_a_bearer_token() {
let event_bus = Arc::new(EventBus::new());
let secrets = SecretManager::mock();
let token = hex::encode(
secrets
.get_or_create_key_bytes(GRPC_TOKEN_ACCOUNT, 32)
.expect("token provisioned from mock vault"),
);
let (addr, _dir) = start_server(event_bus, token).await;
let mut client =
nuncio_proto::v1::filters_client::FiltersClient::connect(format!("http://{addr}"))
.await
.expect("client connects");
let err = client
.update_rule(nuncio_proto::v1::UpdateRuleRequest {
id: "rule-1".to_string(),
name: None,
nsql: None,
priority: None,
})
.await
.expect_err("call without any authorization header must be rejected");
assert_eq!(err.code(), Code::Unauthenticated);
}
/// Confirms the newly added `Accounts` lifecycle RPCs
/// (`UpdateAccount`/`RemoveAccount`/`TestAccountConnection`) are each mounted
/// behind the SAME `BearerAuthInterceptor` as every other service -- an
/// unauthenticated call to any of them must be rejected exactly like an
/// unauthenticated `System/GetStatus`, with no carve-out for "just one more"
/// RPC.
#[tokio::test]
async fn rejects_calls_to_new_accounts_rpcs_without_a_bearer_token() {
let event_bus = Arc::new(EventBus::new());
let secrets = SecretManager::mock();
let token = hex::encode(
secrets
.get_or_create_key_bytes(GRPC_TOKEN_ACCOUNT, 32)
.expect("token provisioned from mock vault"),
);
let (addr, _dir) = start_server(event_bus, token).await;
let mut client =
nuncio_proto::v1::accounts_client::AccountsClient::connect(format!("http://{addr}"))
.await
.expect("client connects");
let update_err = client
.update_account(nuncio_proto::v1::UpdateAccountRequest {
config: None,
password: None,
})
.await
.expect_err("unauthenticated update_account must be rejected");
assert_eq!(update_err.code(), Code::Unauthenticated);
let remove_err = client
.remove_account(nuncio_proto::v1::RemoveAccountRequest {
id: "acct-1".to_string(),
})
.await
.expect_err("unauthenticated remove_account must be rejected");
assert_eq!(remove_err.code(), Code::Unauthenticated);
let test_err = client
.test_account_connection(nuncio_proto::v1::TestAccountConnectionRequest {
id: "acct-1".to_string(),
})
.await
.expect_err("unauthenticated test_account_connection must be rejected");
assert_eq!(test_err.code(), Code::Unauthenticated);
}
/// Confirms the streaming `Filters.Triage` RPC is mounted behind the SAME
/// `BearerAuthInterceptor` as every other `Filters` RPC -- an unauthenticated
/// call must be rejected before the stream is ever established, exactly
/// like an unauthenticated unary call.
#[tokio::test]
async fn rejects_triage_calls_without_a_bearer_token() {
let event_bus = Arc::new(EventBus::new());
let secrets = SecretManager::mock();
let token = hex::encode(
secrets
.get_or_create_key_bytes(GRPC_TOKEN_ACCOUNT, 32)
.expect("token provisioned from mock vault"),
);
let (addr, _dir) = start_server(event_bus, token).await;
let mut client =
nuncio_proto::v1::filters_client::FiltersClient::connect(format!("http://{addr}"))
.await
.expect("client connects");
let err = client
.triage(nuncio_proto::v1::TriageRequest {
rule_id: None,
chunk_size: 0,
})
.await
.expect_err("call without any authorization header must be rejected");
assert_eq!(err.code(), Code::Unauthenticated);
}
+8 -1
View File
@@ -42,7 +42,10 @@ use nuncio_proto::v1::{
};
use nuncio_store::db::DatabaseEngine;
use nuncio_store::vault::{SecretManager, GRPC_TOKEN_ACCOUNT};
use nunciod::grpc::{serve_on_listener_with_overrides, MailEngineOverrides};
use nunciod::grpc::{
serve_on_listener_with_overrides, AccountsEngineOverrides, CalendarEngineOverrides,
ContactsEngineOverrides, MailEngineOverrides,
};
use std::sync::Arc;
use tokio::net::TcpListener;
@@ -125,6 +128,9 @@ async fn boot_daemon(
server_secrets,
server_token,
overrides,
CalendarEngineOverrides::default(),
ContactsEngineOverrides::default(),
AccountsEngineOverrides::default(),
)
.await;
});
@@ -299,6 +305,7 @@ async fn full_mail_spine_round_trips_over_authenticated_grpc_with_mocked_protoco
body_text: "Let's discuss the roadmap.".to_string(),
body_html: None,
attachments: Vec::new(),
account_id: None,
})
.await
.expect("send_message succeeds")
+47 -25
View File
@@ -23,16 +23,32 @@ machine (or a new LLM) picks up the work cold.
it to the definition of done, and open a PR. Do not merge your own work; do not
start a second story until yours is merged or handed back.
## Current state of the code (2026-07-28)
## Current state of the code (2026-07-30)
- **Phases 0–2 complete.** The daemon boots gRPC-only on loopback
`127.0.0.1:9420` behind a keyring bearer token, serving six services (System,
Accounts, Mail, Filters, Export, Audit), each behind the auth interceptor. The
`127.0.0.1:9420` behind a keyring bearer token, serving eight services (System,
Accounts, Mail, Filters, Calendar, Contacts, Export, Audit), each behind the
auth interceptor. The
`IMAP → store → read → SMTP send` spine works end-to-end over gRPC via the
`nuncio-cli` reference client, tested offline. `nuncio.v1` is published with a
contract-stability golden.
- **Phase 3 in progress:** filter correctness (3.A) and the calendar **engine**
layer (3.B.1: real CalDAV transport, TZID fix, `CalendarBackend`, store CRUD)
are done and on `dev`. Everything else is the M1–M7 backlog.
- **The feature surface (Phase 3 through M5) is now COMPLETE — M1–M5 all merged
to `dev`** (M1–M4 already promoted to `main`; M5 promotes next).
Filter correctness (3.A), Calendar (M1: CalDAV + RRULE recurrence, API vertical),
Contacts (M2: real CardDAV engine + API vertical), JMAP (M3: real client wired
into Mail sync), Filters made live (M4: engine wired into sync with a
fire-once guard, the `account`-field bug fixed, edit/export/import/logs and
`Filters.Triage` bulk rescan on the gRPC service), and Mail lifecycle +
incremental sync (M5: account edit/delete/test-connection with persisted
`imap_tls_mode`/`smtp_tls_mode` driving the real IMAP/SMTP connection, a real
per-folder `{uidvalidity}:{uidnext}` incremental-sync checkpoint with a safe
full re-fetch on UIDVALIDITY change and a per-item FETCH timeout, explicit
`SendMessage` account selection, and keyring rollback on failed `AddAccount`)
are all done.
- **Next milestone: M6 — Security & release hardening** (fail-closed
encryption, zeroized key material, WORM/ledger key-length validation,
non-loopback gRPC bind rejection + an auth-coverage canary, a fail-closed
updater on missing `SHA256SUMS`, real CIDR/DNS SSRF checks for webhooks,
inbound HTML email sanitization). Then M7 (API freeze / client-readiness).
- Full detail and the path to a client-ready API: `docs/ROADMAP.md`.
## Branches & preserved WIP
@@ -86,30 +102,32 @@ GitHub milestones **M1–M7** mirror `docs/ROADMAP.md`. Each milestone holds
`story`-labelled issues with a full, self-contained spec. Order: M1 → M7 (M1–M5
independent-ish feature work; M6 security; M7 the client-readiness gate).
**M1 — Finish Calendar (CalDAV)** · milestone #23
- #185 — Add Calendar API vertical (proto, daemon, CLI, E2E) · **[PROTO]**
- #186 — Wire RRULE recurrence expansion into calendar event queries
**M1–M5 are DELIVERED — all stories merged to `dev`.** M6 is next.
**M2 — Contacts (CardDAV)** · milestone #24
- #187 — Build a real Contacts (CardDAV) engine, replacing the fabricated client
- #188 — Add Contacts API vertical (proto, daemon, CLI, E2E) · **[PROTO]**
**M1 — Finish Calendar (CalDAV)** · milestone #23 · **DONE**
- #185 — Add Calendar API vertical (proto, daemon, CLI, E2E) · **[PROTO]** · merged
- #186 — Wire RRULE recurrence expansion into calendar event queries · merged
**M3 — JMAP** · milestone #25
- #189 — Real JMAP client wired into the existing Mail sync path
**M2 — Contacts (CardDAV)** · milestone #24 · **DONE**
- #187 — Build a real Contacts (CardDAV) engine, replacing the fabricated client · merged
- #188 — Add Contacts API vertical (proto, daemon, CLI, E2E) · **[PROTO]** · merged
**M4 — Filters made live** · milestone #26
- #190 — Wire FilterEngine into live sync so matched rules actually fire *(headline; #193 depends on its helper)*
- #191 — Fix filter header-field and account-field evaluation bugs
- #192 — Move filter edit/export/import/logs onto the Filters gRPC service · **[PROTO]**
- #193 — Re-expose bulk filter triage as a streaming Filters RPC · **[PROTO]** *(depends on #190)*
**M3 — JMAP** · milestone #25 · **DONE**
- #189 — Real JMAP client wired into the existing Mail sync path · merged
**M5 — Mail lifecycle + incremental sync** · milestone #27
- #194 — Account lifecycle over gRPC (edit/delete/test) + persist TLS mode · **[PROTO]** *(prior art: `wip/unreviewed-account-tls-sync`, reference-not-trust)*
- #195 — Incremental IMAP sync with progress reporting and per-item FETCH timeout
- #196 — SendMessage selects an explicit account instead of the first configured · **[PROTO]**
- #197 — Roll back orphaned keyring secret when AddAccount fails to persist
**M4 — Filters made live** · milestone #26 · **DONE**
- #190 — Wire FilterEngine into live sync so matched rules actually fire *(headline; #193 depends on its helper)* · merged
- #191 — Fix filter header-field and account-field evaluation bugs · merged
- #192 — Move filter edit/export/import/logs onto the Filters gRPC service · **[PROTO]** · merged
- #193 — Re-expose bulk filter triage as a streaming Filters RPC · **[PROTO]** *(depends on #190)* · merged
**M6 — Security & release hardening** · milestone #28
**M5 — Mail lifecycle + incremental sync** · milestone #27 · **DONE**
- #194 — Account lifecycle over gRPC (edit/delete/test) + persist TLS mode · **[PROTO]** *(prior art: `wip/unreviewed-account-tls-sync`, reference-not-trust)* · merged
- #195 — Incremental IMAP sync with progress reporting and per-item FETCH timeout · merged
- #196 — SendMessage selects an explicit account instead of the first configured · **[PROTO]** · merged
- #197 — Roll back orphaned keyring secret when AddAccount fails to persist · merged
**M6 — Security & release hardening** · milestone #28 ← **next**
- #198 — Fail closed on encryption failure in `PayloadCipher::encrypt_text_at_rest`
- #199 — Zeroize long-lived engine key material on drop
- #200 — Validate WORM/ledger key length from the vault, fail closed on mismatch
@@ -135,3 +153,7 @@ independent-ish feature work; M6 security; M7 the client-readiness gate).
those two issues if you consider that sensitive.
- Decide the eventual fate of `wip/unreviewed-account-tls-sync` (review-and-adopt
vs discard) once M5 lands a clean implementation.
- Follow-ups filed during M1–M5 that are open but not blocking (not scheduled
into a milestone yet): #216 (calendar windowing/predicate refinement), #219
(contacts notes handling), #225 (`filter edit` silently re-enables a disabled
rule and resets `created_at`), #233 (M5 sync/SMTP robustness polish).
+45 -24
View File
@@ -41,41 +41,62 @@ These are non-negotiable and apply to every item below:
---
## Where we are today (2026-07-28)
## Where we are today (2026-07-30)
**Phases 0–2 are complete and on `dev`.** The daemon boots gRPC-only on loopback
`127.0.0.1:9420` behind a keyring-minted bearer token, and serves **six services,
each mounted behind the auth interceptor**: `System`, `Accounts`, `Mail`,
`Filters`, `Export`, `Audit`. The IMAP→store→read→SMTP-send spine works
`127.0.0.1:9420` behind a keyring-minted bearer token, and serves **eight
services, each mounted behind the auth interceptor**: `System`, `Accounts`,
`Mail`, `Filters`, `Calendar`, `Contacts`, `Export`, `Audit` (the `Calendar`
and `Contacts` verticals landed in M1/M2). The IMAP→store→read→SMTP-send spine works
end-to-end over gRPC, driven by the `nuncio-cli` reference client, tested
offline. The legacy JSON-RPC IPC is removed. The `nuncio.v1` contract is
published with a byte-deterministic `FileDescriptorSet` golden guarding it, plus
external-codegen docs for Swift/C#/TypeScript.
- **Phase 0 — Honesty & foundation:** done. Fabricated docs archived; workspace
shrunk to engine crates + CLI (`_reference/` holds the old shells); real OS
keyring; fail-open auto-updater disabled; releases marked pre-alpha.
- **Phase 1 — gRPC skeleton + the spine:** done. Authenticated loopback gRPC;
CLI as gRPC client; real `IMAP fetch → store → read → SMTP send` against mock
servers; store correctness fixes (FTS-vs-encryption, backfill, salvage schema,
transient-error ≠ corruption).
- **Phase 2 — API completeness for the spine:** done. Full proto surface for the
spine; contract-stability test; codegen docs; IPC removed.
**Phase 3 — Feature completeness — is in progress:**
**The feature surface — Phase 3 through M5 — is now COMPLETE**, via milestones
M1–M5, all merged to `dev` (M1–M4 are already promoted to `main`; M5 promotes
next):
- **3.A Filter correctness** — **done** (account scoping, `ON ACCOUNT`, non-ASCII
offsets, `NOT IN`).
- **3.B Calendar** — engine layer **done** (real CalDAV `REPORT` transport,
`TZID` fix, `CalendarBackend` trait, `calendar_events` store CRUD); the API
vertical + recurrence wiring remain (see M1).
- **3.C Contacts (CardDAV)** and **3.D JMAP** — not started.
- **3.B Calendar** — **done** (M1). Real CalDAV `REPORT` transport, `TZID` fix,
`CalendarBackend` trait, `calendar_events` store CRUD, the `Calendar` gRPC
service + CLI commands, and RRULE recurrence wired into the event-query path.
- **3.C Contacts (CardDAV)** — **done** (M2). A real, wiremock-tested CardDAV
client replaced the fabricated one; contacts persist through the daemon's
store and are exposed over a `Contacts` gRPC service + CLI.
- **3.D JMAP** — **done** (M3). A real JMAP HTTP client is wired into the same
`MailBackend` fetch→store path as IMAP.
- **Filters made live** — **done** (M4). `FilterEngine::evaluate` is wired into
the live sync path with a fire-once-per-new-message guard, the `account`
condition-field bug is fixed, unsupported `header[...]` conditions are
honestly rejected, filter edit/export/import/logs moved onto the `Filters`
gRPC service, and a `Filters.Triage` server-streaming RPC does retroactive
bulk rescan.
- **Mail lifecycle & incremental sync** — **done** (M5). Account lifecycle
(`UpdateAccount`/`RemoveAccount`/`TestAccountConnection`) is real over gRPC,
with `imap_tls_mode`/`smtp_tls_mode` persisted and actually driving the real
IMAP (implicit/STARTTLS/plain) and SMTP connection. Sync is genuinely
incremental: a real per-folder `{uidvalidity}:{uidnext}` checkpoint narrows
the FETCH instead of re-fetching everything, a UIDVALIDITY change forces a
safe full re-fetch (no silent mail loss), a per-item FETCH timeout replaces
the indefinite hang, and a `SyncProgress` event is emitted. `SendMessage`
takes an explicit `account_id` (honest `AccountNotFound`, no silent
wrong-account fallback), and `AddAccount` rolls back an orphaned keyring
secret if persistence fails.
Dogfooding against a real mailbox and a fabrication audit surfaced additional
**real gaps** now folded into the milestones below: the filter match-and-act
engine is not yet wired into the live sync path; sync is not incremental;
account edit/delete/test-connection and TLS-mode selection are incomplete; and a
set of correctness/security items (see M5/M6).
**Next up: M6 — Security & release hardening** (fail-closed encryption,
zeroized key material, WORM/ledger key-length validation, a rejection of
non-loopback gRPC binds plus an auth-coverage canary, a fail-closed updater on
missing `SHA256SUMS`, real CIDR/DNS SSRF checks for webhooks, inbound HTML
email sanitization). Then M7 (API freeze / client-readiness).
A handful of follow-ups were filed during M1–M5 and remain open, tracked as
their own backlog items rather than blocking completion: calendar
windowing/predicate refinement (#216), contacts notes handling (#219), a
`filter edit` bug where re-saving a rule silently re-enables it and resets
`created_at` (#225), and further mail sync/SMTP robustness polish identified
during M5 (#233).
**Explicitly deferred** (out of scope until the backend is client-ready and each
is re-justified as a genuine engine capability): OpenPGP/S-MIME E2EE (#79),
+50 -37
View File
@@ -76,7 +76,7 @@
/* milestone rail */
.miles{position:relative;margin-top:8px}
.miles::before{content:"";position:absolute;left:19px;top:12px;bottom:34px;width:2px;
background:linear-gradient(var(--done),var(--prog) 16%,var(--line) 22%);border-radius:2px}
background:linear-gradient(var(--done),var(--done) 71%,var(--prog) 71%,var(--prog) 86%,var(--line) 86%);border-radius:2px}
@media (max-width:640px){.miles::before{display:none}}
.m{position:relative;padding-left:52px;margin-bottom:16px}
@media (max-width:640px){.m{padding-left:0}}
@@ -133,24 +133,33 @@
<hr class="rule">
<h2>Where we are · 2026-07-28</h2>
<h2>Where we are · 2026-07-30</h2>
<div class="band">
<h3>Phases 0–2 complete — the spine works end to end</h3>
<h3>Phases 0–2 complete, feature surface (Phase 3–M5) done — M1–M5 all merged</h3>
<p>The daemon boots gRPC-only on loopback <span class="chip">127.0.0.1:9420</span> behind a keyring-minted
bearer token. <code>IMAP → store → read → SMTP send</code> runs end-to-end over gRPC, driven by the
<span class="chip">nuncio-cli</span> reference client, tested fully offline. The legacy JSON-RPC IPC is gone,
and <span class="chip">nuncio.v1</span> is published with a byte-deterministic contract-stability golden.</p>
and <span class="chip">nuncio.v1</span> is published with a byte-deterministic contract-stability golden.
Milestones M1–M5 are now complete on <span class="chip">dev</span> (M1–M4 already promoted to
<span class="chip">main</span>): Calendar, Contacts, JMAP, live Filters, and mail lifecycle + incremental
sync are all real, wired, and API-complete. <b style="color:var(--ink)">Next up: M6</b> (security &amp;
release hardening).</p>
<div class="chips">
<span class="chip svc">System</span><span class="chip svc">Accounts</span><span class="chip svc">Mail</span>
<span class="chip svc">Filters</span><span class="chip svc">Export</span><span class="chip svc">Audit</span>
<span class="chip" style="color:var(--muted)">— 6 services, each behind the auth interceptor</span>
<span class="chip svc">Filters</span><span class="chip svc">Calendar</span><span class="chip svc">Contacts</span>
<span class="chip svc">Export</span><span class="chip svc">Audit</span>
<span class="chip" style="color:var(--muted)">— 8 services, each behind the auth interceptor</span>
</div>
<div class="donerow">
<span class="donetag">Phase 0 · Honesty &amp; foundation</span>
<span class="donetag">Phase 1 · gRPC skeleton + spine</span>
<span class="donetag">Phase 2 · API completeness (spine)</span>
<span class="donetag">3.A · Filter correctness</span>
<span class="donetag">3.B.1 · Calendar engine (CalDAV + TZID)</span>
<span class="donetag">M1 · Calendar (CalDAV + recurrence)</span>
<span class="donetag">M2 · Contacts (CardDAV)</span>
<span class="donetag">M3 · JMAP</span>
<span class="donetag">M4 · Filters made live</span>
<span class="donetag">M5 · Mail lifecycle + incremental sync</span>
</div>
</div>
@@ -159,87 +168,91 @@
<h2>The path · seven milestones to the goal line</h2>
<div class="miles">
<div class="m prog">
<div class="m done">
<div class="node">M1</div>
<div class="card">
<div class="chead">
<h3 class="ctitle"><span class="code">M1</span>Finish Calendar</h3>
<span class="stat prog">In progress</span>
<span class="stat done">Done</span>
</div>
<p class="cbody">Publish the calendar engine over the API: a <code>Calendar</code> gRPC service
<p class="cbody">The calendar engine is published over the API: a <code>Calendar</code> gRPC service
(Sync / ListEvents / GetEvent — honest surface only) behind the auth interceptor, daemon wiring, CLI
commands replacing the stub, and <b>recurrence (rrule) wired into the fetch/store path</b> (today it's dead code).</p>
commands replacing the stub, and <b>recurrence (rrule) wired into the fetch/store path</b>.</p>
<div class="exit"><b>Exit:</b> a Calendar view over gRPC, offline-tested sync→list→get, recurring events materialize.</div>
<div class="issues"><a class="iss">#178</a><span class="iss">3.B.1 done → 3.B.2 next</span></div>
<div class="issues"><a class="iss">#178</a><span class="iss">done — #185, #186 merged</span></div>
</div>
</div>
<div class="m plan">
<div class="m done">
<div class="node">M2</div>
<div class="card">
<div class="chead">
<h3 class="ctitle"><span class="code">M2</span>Contacts · CardDAV</h3>
<span class="stat plan">Planned</span>
<span class="stat done">Done</span>
</div>
<p class="cbody">Replace the fake CardDAV client (returns one hardcoded contact) with a real, wiremock-tested
client + vCard round-trip; persist through the daemon's store (the CLI currently writes a throwaway
in-memory DB); add a <code>Contacts</code> service, CLI, and offline E2E.</p>
<p class="cbody">The fake CardDAV client (returned one hardcoded contact) is replaced with a real, wiremock-tested
client + vCard round-trip; contacts persist through the daemon's store (the CLI previously wrote a throwaway
in-memory DB); a <code>Contacts</code> service, CLI, and offline E2E are in place.</p>
<div class="exit"><b>Exit:</b> contacts sync and persist across restarts, driven over gRPC.</div>
<div class="issues"><a class="iss">#179</a></div>
<div class="issues"><a class="iss">#179</a><span class="iss">done — #187, #188 merged</span></div>
</div>
</div>
<div class="m plan">
<div class="m done">
<div class="node">M3</div>
<div class="card">
<div class="chead">
<h3 class="ctitle"><span class="code">M3</span>JMAP</h3>
<span class="stat plan">Planned</span>
<span class="stat done">Done</span>
</div>
<p class="cbody">A real JMAP HTTP client (session, Email/query/get/set) with the camelCase wire fix, wired into
<p class="cbody">A real JMAP HTTP client (session, Email/query/get/set) with the camelCase wire fix is wired into
the same <code>MailBackend</code> fetch→store path as IMAP; wiremock-tested.</p>
<div class="exit"><b>Exit:</b> a JMAP account fetches → stores → lists offline, via the existing Mail service.</div>
<div class="issues"><a class="iss">#180</a></div>
<div class="issues"><a class="iss">#180</a><span class="iss">done — #189 merged</span></div>
</div>
</div>
<div class="m plan">
<div class="m done">
<div class="node">M4</div>
<div class="card">
<div class="chead">
<h3 class="ctitle"><span class="code">M4</span>Filters made live</h3>
<span class="stat plan">Planned</span>
<span class="stat done">Done</span>
</div>
<p class="cbody">Today rules can be created, validated, and previewed as “MATCH” but <b>never fire</b>. Wire
<code>FilterEngine::evaluate</code> into the real sync path so matches trigger their action
(move/flag/delete/forward/webhook), fix the header + account-field bugs, and re-expose filter
edit/export/import/logs and bulk triage over the service.</p>
<p class="cbody">Rules could previously be created, validated, and previewed as “MATCH” but never fired.
<code>FilterEngine::evaluate</code> is now wired into the real sync path so matches trigger their action
(move/flag/delete/forward/webhook), the header + account-field bugs are fixed, and filter
edit/export/import/logs plus bulk triage are re-exposed over the service.</p>
<div class="exit"><b>Exit:</b> an E2E proves a matching rule fires an action against synced mail — no fabricated preview.</div>
<div class="issues"><a class="iss">#175</a><a class="iss">#176</a><a class="iss">#181</a></div>
<div class="issues"><a class="iss">#175</a><a class="iss">#176</a><a class="iss">#181</a><span class="iss">done — #190–#193 merged</span></div>
</div>
</div>
<div class="m plan">
<div class="m done">
<div class="node">M5</div>
<div class="card">
<div class="chead">
<h3 class="ctitle"><span class="code">M5</span>Mail lifecycle + incremental sync</h3>
<span class="stat plan">Planned</span>
<span class="stat done">Done</span>
</div>
<p class="cbody">Round out real account management: edit / delete / test-connection with TLS modes
(implicit / STARTTLS / plain) actually wired; explicit send-account selection; and <b>incremental sync</b>
(honor <code>since_state</code> instead of re-fetching the whole mailbox every time) with progress and stall timeouts.</p>
<p class="cbody">Account lifecycle (edit / delete / test-connection) is real over gRPC, with TLS modes
(implicit / STARTTLS / plain) persisted and actually driving the IMAP/SMTP connection. Sync is now
genuinely incremental via a per-folder <code>{uidvalidity}:{uidnext}</code> checkpoint (a UIDVALIDITY
change forces a safe full re-fetch — no silent mail loss), with a per-item FETCH timeout and
<code>SyncProgress</code> events. <code>SendMessage</code> takes an explicit account id (honest
<code>AccountNotFound</code>, no silent wrong-account fallback), and a failed <code>AddAccount</code>
rolls back its orphaned keyring secret.</p>
<div class="exit"><b>Exit:</b> full account lifecycle + incremental sync, offline-E2E-proven, no fabricated IMAP fallbacks.</div>
<div class="issues"><a class="iss">#170</a><a class="iss">#169</a><a class="iss">#182</a><a class="iss">#183</a><a class="iss">#184</a></div>
<div class="issues"><a class="iss">#170</a><a class="iss">#169</a><a class="iss">#182</a><a class="iss">#183</a><a class="iss">#184</a><span class="iss">done — #194–#197 merged</span></div>
</div>
</div>
<div class="m plan">
<div class="m prog">
<div class="node">M6</div>
<div class="card">
<div class="chead">
<h3 class="ctitle"><span class="code">M6</span>Security &amp; release hardening</h3>
<span class="stat plan">Planned</span>
<span class="stat prog">Next</span>
</div>
<p class="cbody">Fail-<i>closed</i> everywhere: fix the silent empty-ciphertext data-loss path; a
signature-verifying updater that refuses to apply without checksums; zeroize key material; validate vault