The CalDAV and CardDAV clients found their payloads by splitting the
response body on the literal strings `<c:calendar-data>` and
`<card:address-data>`. That is only correct for servers that happen to
pick the same prefixes Nuncio does. A compliant server that binds the
namespaces to `D:`/`CAL:`, or declares `DAV:` as the default namespace,
produced zero events or zero contacts and reported success -- the user
saw an empty calendar with nothing in the logs. An element whose name
merely contains the searched one (`<c:calendar-data-summary>`) was read
as if it were the payload, and a property returned under a `404`
propstat was treated as a value.
Add `nuncio_core::dav`, a namespace-aware `multistatus` walker built on
quick-xml (already a workspace dependency, previously declared but
unused) and shared by both clients, since RFC 4791 and RFC 6352 return
the same document shape. Elements are matched on their resolved
`{namespace}local-name`; `<propstat>` status is honoured; and a body
that cannot be traversed is now a `MalformedResponse` error rather than
an empty result indistinguishable from an empty collection. Responses
that carry no readable value are counted and logged at `warn` with the
href and reason, so an omission is visible instead of silent.
Refs #257
dev split message identity from placement, made filter evaluation per
placement, and replaced the schema-migration helpers with create-only
schemas. This branch's single-owner filter execution had to be re-seated
on top of all three.
The account gate stays where it was in intent -- a daemon that does not
own an account runs none of that account's rule actions -- but now sits
outside the per-placement fire-once claim rather than beside a per-message
one. On the sync path the gate is checked with the new-arrival test before
`apply_filter_actions` is reached, so a non-owner never takes a claim it
has already decided not to act on; leaving the claim unconsumed keeps it
available to the daemon that does own the account. Triage does the same
one level up: a message whose account this daemon does not own is counted
as scanned but its placements are never even read, and a daemon owning no
account still refuses the RPC outright instead of streaming an empty scan.
`filters_enabled` is now only a column in the created `accounts` schema;
the additive `ALTER TABLE` helper and its two migration tests are dropped,
since dev creates schemas and never upgrades them. The flag's persistence
is still covered -- the identity-rebuild test now asserts filter ownership
survives a rebuild, and the store round-trip assertions are unchanged.
The non-owner sync test keeps every assertion it had, translated to the
placement API: the mail is still stored, the occupancy's read flag is
still unflipped, and no execution log or outbox mutation is written.
The typed async-imap API cannot express the four responses that decide whether
a sync saw everything or whether a mutation actually happened. ENABLE does not
exist, and RFC 7162 3.2.3 makes a server answer BAD to a QRESYNC SELECT without
it. select()'s parser has no arm for VANISHED, so a deleted message is simply
never learned about. And run_command_and_check_ok discards the tagged response
code, which is exactly where RFC 4315 puts COPYUID and RFC 7162 puts MODIFIED.
Hand-rolled rather than forked. run_command and read_response are public and
imap-proto already parses Vanished and CopyUid, so what was missing is a
collector that reads to the tagged response and keeps what arrived on the way.
That is a hundred lines against maintaining a fork for the same result.
Two subtleties the collector exists to get right. COPYUID arrives in an
UNTAGGED OK for MOVE (RFC 6851 4.3 advises exactly that), so a collector
reading only the tagged response misses it on every conforming server and then
has no proof the move happened. And UID EXPUNGE answers with VANISHED rather
than EXPUNGE once QRESYNC is enabled (4.4), so counting EXPUNGE alone reports
failure on every successful delete; both are gathered.
MODIFIED is the one real gap. imap-proto 0.16.7 has no ResponseCode variant for
it, and ResponseData's raw bytes are behind a private module, so it cannot be
read from the parse tree at all. Its resp_text is opt(resp_text_code) followed
by free text, so an unrecognised code is not an error -- the whole
"[MODIFIED 7,9]" survives in the information string, and that is where it is
recovered from. Parsing a response code out of a human-readable field is not
how this should work forever; upstreaming the variant is the real fix. It is
deterministic, it is pinned by a test against a server emitting the real
syntax, and the alternative is no conflict detection at all.
Absent COPYUID is deliberately not treated as failure. A UID set matching
nothing succeeds having done nothing (RFC 3501 6.4.8), and a conforming server
may also omit the code for a UIDNOTSTICKY destination, so absence means unknown.
Also fixes the mock server reading the first parenthesised group of a STORE as
its flag list, which made a conditional store apply UNCHANGEDSINCE and the
mod-sequence as flag names. The flag list is always the final argument.
Refs #388
Wire -v/-vv/-vvv to a real stderr tracing subscriber (previously parsed
but never used), with NUNCIO_LOG/RUST_LOG override precedence matching
the daemon. Decode the WS-A1 ErrorInfo from a failed RPC's status
details and render the gRPC code plus the typed reason alongside the
message, in both text and --json output, instead of dropping the code
or dumping a raw Status. Document daemon log levels, the rotating log
file location, JSON log output, and request_id correlation in
docs/RUNNING.md.
Refs #365
Co-authored-by: James Maes <jmaes@dmdbrands.com>
Wrap IMAP/JMAP folder and message sync in tracing spans keyed by account
and folder, logging completion counts (fetched/oversized/skipped) so a
sync run's outcome is visible without a debugger. SMTP send attempts and
their outcome are logged with the recipient COUNT only, never addresses.
Every destructive IMAP mutation (MOVE, DELETE/UID-scoped EXPUNGE) is
logged with its UID set and the UIDVALIDITY guard's pass/fail result
BEFORE the command reaches the server, so an audit trail exists even
when the guard then refuses to act. JMAP mirrors the same pre-logging
for its Email/set move/destroy, keyed by the JMAP object id instead of
a UID (JMAP has no UIDVALIDITY-equivalent to guard).
Refs #361
Co-authored-by: James Maes <jmaes@dmdbrands.com>
An unparseable VEVENT inside a CalDAV multistatus response was silently
discarded, so a partial sync failure looked identical to a calendar with
fewer events. It now logs a WARN with the VEVENT's UID (if extractable)
and a short parse-error reason, never the raw calendar body.
CalDAV and CardDAV round trips are wrapped in spans and log completion
counts (fetched/parsed/dropped) keyed by account/calendar id only.
Additive SQLite column migrations and first-use vault key minting now
log at INFO, naming the table/column or key purpose without ever
exposing key material.
CalDavAccountConfig and CardDavAccountConfig held an auth_token behind a
derived Debug, which would have printed the raw token verbatim from any
future debug! log or panic message. Both now use a manual Debug impl
that redacts the token.
Refs #362
Co-authored-by: James Maes <jmaes@dmdbrands.com>
Add a tower Layer on the tonic server that wraps every RPC in a root span
carrying a generated request_id, the gRPC method, and the peer address. The
inner service (routing, the bearer-auth interceptor, and the handler) runs
inside that span, so all events emitted while handling a request inherit the
request_id. The span opens with a DEBUG event and closes, once the response
body drains, with an INFO event carrying the terminal gRPC status and elapsed
time.
Propagate the correlation span into the SyncDispatcher's detached sync tasks so
a sync triggered by Mail.Sync logs under the triggering request_id.
Log bearer-auth accept (DEBUG) and reject (WARN, with a fixed reason) in the
interceptor, carrying only the peer -- never the token, the authorization
header, or anything derived from them. The constant-time compare is unchanged.
Refs #358
Co-authored-by: James Maes <jmaes@dmdbrands.com>
Replace the stderr-only tracing_subscriber::fmt::init() bootstrap with a
layered Registry subscriber: a reloadable EnvFilter (NUNCIO_LOG > RUST_LOG >
info) exposed via a reload::Handle for future runtime level control, a
human-readable stderr console layer, and a non-blocking rotating daily file
sink under <data_dir>/logs/. NUNCIO_LOG_FORMAT=json switches the file sink to
JSON. Malformed filters and unwritable log dirs degrade gracefully instead of
crashing the daemon; the WorkerGuard is returned and bound in main for the
process lifetime.
Refs #357
Co-authored-by: James Maes <jmaes@dmdbrands.com>
Message.received_at, CalendarEvent.start_time/end_time,
CalendarSyncRequest/ListEventsRequest.start_window/end_window,
FilterRule.created_at/updated_at, FilterExecutionLog.matched_at,
AccountConfig.sync_interval_secs, PreviewRuleResponse.execution_time_us,
and AuditRecord.timestamp_ns all mixed unix-seconds and nanosecond int64
scalars, leaving the unit ambiguous at the wire level. Move every wall-clock
field to google.protobuf.Timestamp and every duration to
google.protobuf.Duration, add prost-types as the standard well-known-types
crate, and add nuncio_proto::time as the single conversion point between
the store's native integer instants and the wire types (including a
lossless unix-nanos split/join for the WORM audit ledger's sub-second
precision). Update the daemon's gRPC mappers and the CLI's request
construction/rendering accordingly, and regenerate the descriptor golden.
Refs #274
Co-authored-by: James Maes <jmaes@dmdbrands.com>
Acquire an exclusive OS advisory lock on a sidecar file beside the
database path (<db_path>.lock) before opening the store, so a second
nunciod pointed at the same database fails fast with an actionable
error instead of both instances writing to the same SQLite file.
Previously only an incidental gRPC port-bind collision offered any
protection against this.
Uses fs4 (flock on Unix, LockFileEx on Windows) since std has no
portable file-lock primitive. The guard is held for the process
lifetime and dropped at clean shutdown, releasing the lock; the OS
also releases it automatically if the process dies uncleanly, so
there is no PID-file staleness to reason about.
Refs #291
Co-authored-by: James Maes <jmaes@dmdbrands.com>
* feat(proto): add typed error taxonomy and Status helper
Introduce a frozen, machine-readable error taxonomy for the nuncio.v1
contract: an ErrorReason enum and an ErrorInfo message carrying a reason,
optional metadata, and a human message. A nuncio-proto errors helper builds
a tonic::Status that encodes ErrorInfo into the grpc-status-details-bin
trailer with the canonical gRPC code per reason, and decodes the reason back
on the client side. Regenerate the committed descriptor golden.
Refs #273
* feat(nunciod): return typed ErrorInfo details from error sites
Convert a representative set of daemon gRPC error returns to carry a typed
ErrorInfo in the status-details trailer via the nuncio-proto errors helper:
account/message/event/contact/rule not-found (with the offending id as
metadata), NSQL validation failures, and the calendar not-configured
precondition. Canonical status codes are unchanged, so existing error-path
assertions still hold. Add an offline over-the-wire E2E test that decodes the
reason and metadata from a returned Status against the real daemon server.
Refs #273
---------
Co-authored-by: James Maes <jmaes@dmdbrands.com>
list_filter_rules silently dropped stored rules that failed to re-parse,
making a rule disappear from enforcement with no trace. Now each
unparseable row is WARN-logged with its rule id and skipped, so the
failure is visible instead of indistinguishable from "never uploaded".
Also stop nunciod's boot-time filter load from collapsing a genuine
database error into an empty rule set: load_initial_filter_rules
propagates the error (ERROR-logged) instead of unwrap_or_default(),
so a DB failure fails startup visibly rather than quietly disabling
all filtering.
Refs #247
Co-authored-by: James Maes <jmaes@dmdbrands.com>
CreateRule derived a rule's id deterministically from
sha256(name + nsql), so two rules with identical name and NSQL text
produced the same id. save_filter_rule persists via INSERT OR REPLACE
with no existence check, so the second create silently overwrote the
first rule with no warning.
Generate each rule's id from a CSPRNG (the same OsRng nuncio-store
already uses to source key material) instead of deriving it from the
rule's content, so identical name+NSQL always yields distinct ids.
UpdateRule and list_filter_rules already reassign the id from the
persisted row after parsing, so they are unaffected.
Refs #249
Co-authored-by: James Maes <jmaes@dmdbrands.com>
Wrap DatabaseEngine's storage/WORM/ledger keys, WebhookDispatcher's
HMAC secret, and add_account's transient password in
zeroize::Zeroizing so their backing memory is overwritten once the
holding struct drops, instead of lingering as plain heap bytes for
the rest of the process lifetime.
Refs #199, #163
Co-authored-by: James Maes <jmaes@dmdbrands.com>
Replace JmapEngine's canned session/mailbox/message data with real
RFC 8620/8621 HTTP session discovery and Mailbox/get + Email/query +
Email/get JSON-RPC calls over reqwest, gated behind with_credentials
so uncredentialed callers keep the deterministic no-network fallback.
Fix the camelCase wire bug on JmapEmail (receivedAt/isUnread/
bodySnippet were silently defaulting) and add a regression test.
Wire nunciod::sync::build_mail_backend to construct the JMAP engine
with the account's real host/address/keyring password, mirroring the
IMAP arm, and rewrite the sync.rs tests that previously relied on
JmapEngine's canned data to run against wiremock servers instead.
Co-authored-by: James Maes <jmaes@dmdbrands.com>
Publish the M2-S1 contacts engine over the daemon's authenticated gRPC
API, mirroring the Calendar vertical: add nuncio.v1.Contacts (Sync,
ListContacts, GetContact, CreateContact) plus Contact/ContactEmail/
ContactPhone messages to the proto contract and regenerate the golden
descriptor.
Wire ContactsEngineOverrides + ContactsGrpcService into nunciod,
mounted behind its own BearerAuthInterceptor exactly like every other
service. ListContacts/GetContact always read genuinely persisted data
from DatabaseEngine; Sync fetches from an injected ContactsBackend
(MockContactsBackend in tests) and persists via
contacts_sync::sync_with_backend, mirroring calendar_sync.rs; with no
backend injected, production Sync returns an honest error since
AccountConfig carries no per-account CardDAV configuration yet.
Add connect_contacts/AuthenticatedContactsClient to nuncio-proto's
shared client helpers. Replace the CLI's Commands::Contact handlers
(previously a disposable in-memory ContactsDatabase.in_memory() per
invocation) with real gRPC calls against the daemon, so a contact
added via `contact add` persists and is visible to a later, separate
CLI invocation.
Deviations from the issue's literal proto sketch, flagged for review:
- Added CreateContact (not in the issue's message list): the
acceptance criteria requires `contact add` to persist a locally-
authored contact across CLI invocations, but the daemon has no
per-account CardDAV credential to write back to a real server. This
RPC only ever writes to the daemon's own local store -- never a
fabricated CardDAV write-back -- which is the same honest-surface
spirit as Calendar's "no create/delete" note (that note is about not
pretending to write to the remote server, not about never
persisting locally-authored data at all).
- ListContacts/CreateContact take a required account_id (the issue
sketched a bare ListContactsRequest{}), because the real
DatabaseEngine::list_contacts/save_contact are account-scoped;
ContactSubcommand::List/Search/Add/Sync now all take --account to
match.
- Contact search has no server-side RPC; `contact search` fetches the
full ListContacts result and filters client-side by display name,
organization, and email.
- Contact's created_at/updated_at/last_interacted_at are omitted from
the proto message -- no client use case identified for them yet.
Full-daemon offline E2E (contacts_e2e_test.rs) proves Sync -> List/Get
over the authenticated API, CreateContact's persistence surviving a
second, independent client connection, and an unauthenticated call
rejected with Code::Unauthenticated.
Co-authored-by: James Maes <jmaes@dmdbrands.com>
Replace CardDavClient::fetch_remote_vcards's hardcoded "Google CardDAV
Sync Contact" return with a real reqwest REPORT/addressbook-query
client that parses a genuine multistatus response. Add a real vCard
(RFC 6350) parser -- the crate previously only had a generator -- and
prove round-trip fidelity through Contact::to_vcard(). Add the
ContactsBackend trait and MockContactsBackend, mirroring
nuncio_cal::CalendarBackend/MockCalendarBackend. Distinguish transport
failures from parse failures via CardDavError::TransportFailed/
ParseFailed rather than ever silently succeeding with fabricated data.
Give nuncio-store real, tested save_contact/list_contacts/get_contact
CRUD backed by a genuine SQLite table + trigram FTS mirror, so contacts
persist through the daemon's own store instead of nuncio-contacts's
disconnected ContactsDatabase pool. This takes nuncio-store as a
narrow, deliberate dependency on nuncio-contacts::Contact.
Opt nuncio-contacts into the workspace lint gate ([lints] workspace,
#![forbid(unsafe_code)]) for parity with every other crate.
Wiremock-only, offline: no live network calls in tests.
Co-authored-by: James Maes <jmaes@dmdbrands.com>
Several gRPC test helpers created an ephemeral sqlite-backed
DatabaseEngine via connect_ephemeral() and then discarded the
returned TempDir guard before returning the still-live db/server to
the caller. Dropping the guard unlinks the directory backing the
sqlite file; on Windows the still-open file handle keeps the data
reachable (masking the bug locally), but on Linux the directory entry
disappears immediately, so any pool connection opened afterward fails
with "unable to open database file" -- exactly the flake CI hit in
nunciod::grpc::tests::preview_rule_dry_runs_against_a_stored_message_without_persisting.
Fixes the same pattern in nunciod::grpc's spawn_test_server and
spawn_filters_test_server helpers, nuncio-cli's HeadlessRunner
(ephemeral_with), and grpc_auth_test's start_server: each now returns
(or stores) the TempDir so it stays alive for as long as its caller
keeps using the database.
Earlier feature commits staged only crates/, leaving the root Cargo.lock behind
when nunciod gained bytes + a sqlx dev-dependency. Re-sync so a --locked build
matches the committed manifests.
Add System.Subscribe server-streaming RPC on the authenticated System service,
mapping every CoreEvent variant to a nuncio.v1.Event oneof. The daemon streams
live EventBus broadcasts (skip-on-lag with a warning; clean close when the bus
drops). Add a client subscribe helper in nuncio-proto. Regression test proves
20 concurrent unary GetStatus calls and 20 streamed events arrive uncorrupted
and in order -- the property the old hand-rolled JSON-RPC IPC demux lacked.
Refs #151
Move the gRPC address resolver and add an authenticated client helper
(connect_system + BearerTokenInterceptor) into the shared nuncio-proto crate so
client shells never depend on the nunciod binary. Wire nuncio-cli's system
status to query the live daemon over gRPC (token from an injectable OS keyring),
printing real status or an honest error when unreachable. Refactor nunciod to
reuse the shared addr items. Reconcile tests to assert the real gRPC round-trip
instead of local ephemeral state.
Refs #150
nunciod now serves the nuncio.v1.System gRPC service on loopback (default
127.0.0.1:9420, env NUNCIO_GRPC_ADDR), authenticated by a bearer token minted
into the OS keyring (SecretManager, CSPRNG, fail-closed). A tonic interceptor
requires 'authorization: Bearer <token>' and compares in constant time
(subtle::ConstantTimeEq); every missing/malformed/wrong-token path returns
Unauthenticated. GetStatus returns live engine status + version. Existing
JSON-RPC IPC (9422) unchanged. Security-reviewed, APPROVED.
Refs #149
New crate crates/nuncio-proto houses the versioned protobuf contract
(proto/nuncio/v1) and tonic-generated client+server stubs, starting with a
minimal System.GetStatus RPC. protoc is vendored via protoc-bin-vendored
(prebuilt binaries; no system protoc, cmake, or C++ toolchain) for reproducible
CI on Linux/macOS/Windows. Uses the tonic 0.14 split (tonic + tonic-prost
runtime, tonic-prost-build codegen). Generated code exempted from workspace
lints; hand-written code keeps them.
Refs #148
Move nuncio-tui, nuncio-gui (+src-tauri, ui), and nuncio-mcp out of the Cargo
workspace into _reference/ (git renames; history preserved), retained for the
Phase 5 native client-repo extraction. Remove the unused nuncio-mcp dev-dep
from nunciod and rework daemon_e2e_test to drive the real nuncio-cli
HeadlessRunner (more coverage, no archived-shell dependency). Workspace is now
engine + daemon + reference CLI (8 members); build/clippy/tests green.
Refs #145