M4 [PROTO]: Move filter edit/export/import/logs onto the Filters gRPC service #192

New Issue

2026-07-28T14:04:32Z

KofTwentyTwo commented

2026-07-28 14:04:32 +00:00

(Migrated from github.com)

Context

Story for milestone M4. Read docs/STORY-WORKFLOW.md first — it defines the process, gates, and patterns; this issue defines only the specific work. When the Filters gRPC service was introduced (crates/nunciod/src/grpc.rs's FiltersGrpcService, service Filters in crates/nuncio-proto/proto/nuncio/v1/nuncio.proto), only CreateRule/ListRules/DeleteRule/ValidateRule/PreviewRule were migrated. nuncio-cli's filter edit/filter export/filter import/filter logs subcommands (crates/nuncio-cli/src/args.rs FilterSubcommand::{Edit,Export,Import,Logs}) were deliberately left reading/writing the CLI process's own throwaway ephemeral local self.db (see the explicit comment at crates/nuncio-cli/src/runner.rs lines 453-464). This is a real, documented gap: a rule created via the real filter create (which DOES go through gRPC to the daemon's persistent store) will not show up in filter export/filter logs, because those still only see the CLI process's own private, throwaway database. This issue closes that gap by adding the missing RPCs to the existing Filters service and pointing the CLI's Edit/Export/Import/Logs subcommands at them, exactly like List/Create/Delete already are.

Goal

filter edit, filter export, filter import, and filter logs all become real thin gRPC clients of the daemon's persistent store and live FilterEngine, with the exact same honesty guarantees CreateRule/DeleteRule already have (persist-then-reload-engine, never fabricate a result). No local ephemeral-db fallback remains for these four subcommands. Nothing here changes matching/evaluation semantics (that is M4-S1/M4-S2's scope) — this is purely relocating existing, already-correct CLI-local logic onto the authenticated gRPC surface.

Files to touch / patterns to mirror

crates/nuncio-proto/proto/nuncio/v1/nuncio.proto — extend service Filters (line 437) with four new RPCs, following the exact doc-comment and message style of CreateRule/ValidateRule (lines 438-461):
- rpc UpdateRule(UpdateRuleRequest) returns (UpdateRuleResponse); — mirrors the CLI's current filter edit semantics (runner.rs lines 465-516: fetch existing rule by id, apply only the fields the caller actually supplied, re-parse via NsqlParser::parse_rule, preserve the original id, persist, reload the live FilterEngine). Message shape: UpdateRuleRequest { string id = 1; optional string name = 2; optional string nsql = 3; optional int32 priority = 4; }; UpdateRuleResponse { FilterRule rule = 1; }.
- rpc ExportRules(ExportRulesRequest) returns (ExportRulesResponse); — mirrors filter export's current logic (runner.rs lines 517-525: list all persisted rules, render as either NSQL text lines or JSON). ExportRulesRequest { string format = 1; } ("sql" or "json"); ExportRulesResponse { string content = 1; }.
- rpc ImportRules(ImportRulesRequest) returns (ImportRulesResponse); — mirrors filter import's current logic (runner.rs lines 526-557: parse each non-empty, non----comment line as NSQL via NsqlParser::parse_rule, persist the ones that parse). The CLI currently reads the file itself and only sends parsed rules nowhere (it wrote directly to its own local db) — now the CLI reads the file and sends the raw text content over the wire, and the DAEMON does the parsing/persisting/reloading. ImportRulesRequest { string content = 1; }; ImportRulesResponse { int32 imported_count = 1; repeated string errors = 2; } (the errors field is a deliberate, honest improvement over today's CLI behaviour, which silently drops lines that fail to parse — surface them instead of hiding the failure).
- rpc GetExecutionLogs(GetExecutionLogsRequest) returns (GetExecutionLogsResponse); — mirrors filter logs's current logic (runner.rs lines 558-579: db.list_filter_execution_logs(limit)). Add a new proto message FilterExecutionLog mirroring nuncio_filter::FilterExecutionLog (crates/nuncio-filter/src/ast.rs lines 302-318: id: i64, rule_id, message_id, action_taken, matched_at: i64, prev_hash, hash) field-for-field. GetExecutionLogsRequest { uint32 limit = 1; }; GetExecutionLogsResponse { repeated FilterExecutionLog logs = 1; }.
crates/nunciod/src/grpc.rs — FiltersGrpcService (struct at line 756, impl Filters for FiltersGrpcService at line 784): add update_rule/export_rules/import_rules/get_execution_logs methods.
- update_rule: fetch existing via self.db.list_filter_rules() + find by id (there is no single-rule getter today — list_filter_rules + find is fine, matching the CLI's own current approach), Status::not_found if missing, apply overrides, NsqlParser::parse_rule + NsqlValidator::validate (matching create_rule's validation, lines 791-795), self.db.save_filter_rule, self.reload_engine_from_store().await (existing helper, line 762), map result via map_filter_rule_to_proto (line 685).
- export_rules: self.db.list_filter_rules(), render per req.format (reuse the exact to_nsql()/JSON rendering the CLI used).
- import_rules: split req.content into lines the same way the CLI did, parse/persist each, count successes, collect parse-error strings for the ones that fail (do not silently swallow them like the CLI did), reload the engine once at the end.
- get_execution_logs: self.db.list_filter_execution_logs(req.limit as usize), map each nuncio_filter::FilterExecutionLog to the new proto message.
crates/nuncio-proto/src/client.rs — no NEW connect_* function needed (connect_filters/AuthenticatedFiltersClient, lines 102/194, already cover the whole Filters service); the new RPC methods appear automatically on AuthenticatedFiltersClient once the proto regenerates.
crates/nuncio-cli/src/runner.rs — replace the local-db bodies of FilterSubcommand::Edit/Export/Import/Logs (lines 465-579) with real gRPC calls via self.connect_filters_client() (line 1377), following the exact style of handle_filter_list/handle_filter_create/handle_filter_delete (lines 1172-1291). Import needs to read the file with std::fs::read_to_string (as today) but then send the content to ImportRules instead of parsing it locally.
crates/nunciod/tests/grpc_auth_test.rs — add (or extend an existing Filters-focused auth test) a case asserting an unauthenticated call to one of the new RPCs (e.g. update_rule) is rejected with Code::Unauthenticated, mirroring rejects_calls_without_a_bearer_token (lines 59-79). Filters is already mounted behind its own BearerAuthInterceptor (crates/nunciod/src/grpc.rs lines 1211-1219) — no new mounting code is needed, just a regression assertion that the new RPCs inherit it.

Implementation steps

Add the 4 new RPCs + their request/response messages + the FilterExecutionLog message to nuncio.proto, following §5 of STORY-WORKFLOW.md (regenerate nuncio-proto, copy the fresh nuncio_v1_descriptor.bin over the committed golden, re-test).
Implement the 4 new methods on FiltersGrpcService in grpc.rs, reusing reload_engine_from_store/map_filter_rule_to_proto exactly as create_rule/delete_rule do.
Add unit tests in grpc.rs's test module mirroring the existing create_rule_rejects_invalid_nsql_and_persists_nothing/delete_rule_rejects_empty_id style for: update-rule happy path + not-found + invalid-nsql rejection; export-rules sql vs json format; import-rules partial-failure (mix of valid and invalid lines) reporting both imported_count and errors; get-execution-logs returning entries in the right order/shape.
Replace the four FilterSubcommand arms in runner.rs with real connect_filters_client()-backed implementations.
Add the unauthenticated-rejection regression test.
Update/remove the now-stale comment block in runner.rs (lines 453-464) explaining why these four subcommands were exceptions — after this story they no longer are.
Run cargo verify; regenerate the contract golden if the descriptor test fails.

Acceptance criteria

New RPCs UpdateRule, ExportRules, ImportRules, GetExecutionLogs added to service Filters; new messages UpdateRuleRequest/UpdateRuleResponse/ExportRulesRequest/ExportRulesResponse/ImportRulesRequest/ImportRulesResponse/GetExecutionLogsRequest/GetExecutionLogsResponse/FilterExecutionLog added, all additive within nuncio.v1.
Contract golden regenerated; nuncio-proto's contract_stability::descriptor_matches_committed_golden test passes.
filter edit/filter export/filter import/filter logs in the CLI go through the daemon's real, persistent store and live FilterEngine — a rule created via filter create now visibly appears in filter export/filter logs output (prove this with an E2E: create via one RPC, read back via the corresponding new RPC).
ImportRules honestly reports per-line parse failures in errors rather than silently dropping them.
An unauthenticated call to at least one of the new RPCs is rejected with Code::Unauthenticated (regression proving the new RPCs inherited Filters's existing BearerAuthInterceptor; no new service mounting was needed since Filters already exists).
Offline E2E (mock-only, no live network) proves the full round trip for at least update + export + import + logs.
cargo verify green (fmt + clippy -D warnings all-targets + test-all), 0 ignored.

Definition of done & process

Follow docs/STORY-WORKFLOW.md: no fabricated data; comment convention (intent-only, no story/GH/phase refs in .rs/.proto); mock-only tests. Branch story/m4-filters-grpc-mgmt off dev; PR to dev with the gate output pasted as evidence; the PO reviews before the issue is closed.

Dependencies / notes

[PROTO]: confirm with the PO that no other proto-touching story is in flight before starting (M4-S4 is also [PROTO] and touches the same service Filters block — coordinate sequencing with the PO; landing this one first is recommended since it is the smaller, more self-contained of the two, but there is no hard code dependency between them). Independent of M4-S1/M4-S2 (pure management-surface relocation, no evaluation-semantics changes). Reconciles GH #175. Size: M.

## Context Story for milestone M4. **Read [`docs/STORY-WORKFLOW.md`](https://github.com/KofTwentyTwo/nuncio/blob/dev/docs/STORY-WORKFLOW.md) first** — it defines the process, gates, and patterns; this issue defines only the specific work. When the `Filters` gRPC service was introduced (`crates/nunciod/src/grpc.rs`'s `FiltersGrpcService`, `service Filters` in `crates/nuncio-proto/proto/nuncio/v1/nuncio.proto`), only `CreateRule`/`ListRules`/`DeleteRule`/`ValidateRule`/`PreviewRule` were migrated. `nuncio-cli`'s `filter edit`/`filter export`/`filter import`/`filter logs` subcommands (`crates/nuncio-cli/src/args.rs` `FilterSubcommand::{Edit,Export,Import,Logs}`) were deliberately left reading/writing the CLI process's own throwaway ephemeral local `self.db` (see the explicit comment at `crates/nuncio-cli/src/runner.rs` lines 453-464). This is a real, documented gap: a rule created via the real `filter create` (which DOES go through gRPC to the daemon's persistent store) will not show up in `filter export`/`filter logs`, because those still only see the CLI process's own private, throwaway database. This issue closes that gap by adding the missing RPCs to the existing `Filters` service and pointing the CLI's `Edit`/`Export`/`Import`/`Logs` subcommands at them, exactly like `List`/`Create`/`Delete` already are. ## Goal `filter edit`, `filter export`, `filter import`, and `filter logs` all become real thin gRPC clients of the daemon's persistent store and live `FilterEngine`, with the exact same honesty guarantees `CreateRule`/`DeleteRule` already have (persist-then-reload-engine, never fabricate a result). No local ephemeral-db fallback remains for these four subcommands. Nothing here changes matching/evaluation semantics (that is `M4-S1`/`M4-S2`'s scope) — this is purely relocating existing, already-correct CLI-local logic onto the authenticated gRPC surface. ## Files to touch / patterns to mirror - `crates/nuncio-proto/proto/nuncio/v1/nuncio.proto` — extend `service Filters` (line 437) with four new RPCs, following the exact doc-comment and message style of `CreateRule`/`ValidateRule` (lines 438-461): - `rpc UpdateRule(UpdateRuleRequest) returns (UpdateRuleResponse);` — mirrors the CLI's current `filter edit` semantics (`runner.rs` lines 465-516: fetch existing rule by id, apply only the fields the caller actually supplied, re-parse via `NsqlParser::parse_rule`, preserve the original `id`, persist, reload the live `FilterEngine`). Message shape: `UpdateRuleRequest { string id = 1; optional string name = 2; optional string nsql = 3; optional int32 priority = 4; }`; `UpdateRuleResponse { FilterRule rule = 1; }`. - `rpc ExportRules(ExportRulesRequest) returns (ExportRulesResponse);` — mirrors `filter export`'s current logic (`runner.rs` lines 517-525: list all persisted rules, render as either NSQL text lines or JSON). `ExportRulesRequest { string format = 1; }` (`"sql"` or `"json"`); `ExportRulesResponse { string content = 1; }`. - `rpc ImportRules(ImportRulesRequest) returns (ImportRulesResponse);` — mirrors `filter import`'s current logic (`runner.rs` lines 526-557: parse each non-empty, non-`--`-comment line as NSQL via `NsqlParser::parse_rule`, persist the ones that parse). The CLI currently reads the file itself and only sends parsed rules nowhere (it wrote directly to its own local db) — now the CLI reads the file and sends the raw text content over the wire, and the DAEMON does the parsing/persisting/reloading. `ImportRulesRequest { string content = 1; }`; `ImportRulesResponse { int32 imported_count = 1; repeated string errors = 2; }` (the `errors` field is a deliberate, honest improvement over today's CLI behaviour, which silently drops lines that fail to parse — surface them instead of hiding the failure). - `rpc GetExecutionLogs(GetExecutionLogsRequest) returns (GetExecutionLogsResponse);` — mirrors `filter logs`'s current logic (`runner.rs` lines 558-579: `db.list_filter_execution_logs(limit)`). Add a new proto message `FilterExecutionLog` mirroring `nuncio_filter::FilterExecutionLog` (`crates/nuncio-filter/src/ast.rs` lines 302-318: `id: i64`, `rule_id`, `message_id`, `action_taken`, `matched_at: i64`, `prev_hash`, `hash`) field-for-field. `GetExecutionLogsRequest { uint32 limit = 1; }`; `GetExecutionLogsResponse { repeated FilterExecutionLog logs = 1; }`. - `crates/nunciod/src/grpc.rs` — `FiltersGrpcService` (struct at line 756, `impl Filters for FiltersGrpcService` at line 784): add `update_rule`/`export_rules`/`import_rules`/`get_execution_logs` methods. - `update_rule`: fetch existing via `self.db.list_filter_rules()` + find by id (there is no single-rule getter today — `list_filter_rules` + find is fine, matching the CLI's own current approach), `Status::not_found` if missing, apply overrides, `NsqlParser::parse_rule` + `NsqlValidator::validate` (matching `create_rule`'s validation, lines 791-795), `self.db.save_filter_rule`, `self.reload_engine_from_store().await` (existing helper, line 762), map result via `map_filter_rule_to_proto` (line 685). - `export_rules`: `self.db.list_filter_rules()`, render per `req.format` (reuse the exact `to_nsql()`/JSON rendering the CLI used). - `import_rules`: split `req.content` into lines the same way the CLI did, parse/persist each, count successes, collect parse-error strings for the ones that fail (do not silently swallow them like the CLI did), reload the engine once at the end. - `get_execution_logs`: `self.db.list_filter_execution_logs(req.limit as usize)`, map each `nuncio_filter::FilterExecutionLog` to the new proto message. - `crates/nuncio-proto/src/client.rs` — no NEW `connect_*` function needed (`connect_filters`/`AuthenticatedFiltersClient`, lines 102/194, already cover the whole `Filters` service); the new RPC methods appear automatically on `AuthenticatedFiltersClient` once the proto regenerates. - `crates/nuncio-cli/src/runner.rs` — replace the local-db bodies of `FilterSubcommand::Edit`/`Export`/`Import`/`Logs` (lines 465-579) with real gRPC calls via `self.connect_filters_client()` (line 1377), following the exact style of `handle_filter_list`/`handle_filter_create`/`handle_filter_delete` (lines 1172-1291). `Import` needs to read the file with `std::fs::read_to_string` (as today) but then send the content to `ImportRules` instead of parsing it locally. - `crates/nunciod/tests/grpc_auth_test.rs` — add (or extend an existing `Filters`-focused auth test) a case asserting an unauthenticated call to one of the new RPCs (e.g. `update_rule`) is rejected with `Code::Unauthenticated`, mirroring `rejects_calls_without_a_bearer_token` (lines 59-79). `Filters` is already mounted behind its own `BearerAuthInterceptor` (`crates/nunciod/src/grpc.rs` lines 1211-1219) — no new mounting code is needed, just a regression assertion that the new RPCs inherit it. ## Implementation steps 1. Add the 4 new RPCs + their request/response messages + the `FilterExecutionLog` message to `nuncio.proto`, following §5 of `STORY-WORKFLOW.md` (regenerate `nuncio-proto`, copy the fresh `nuncio_v1_descriptor.bin` over the committed golden, re-test). 2. Implement the 4 new methods on `FiltersGrpcService` in `grpc.rs`, reusing `reload_engine_from_store`/`map_filter_rule_to_proto` exactly as `create_rule`/`delete_rule` do. 3. Add unit tests in `grpc.rs`'s test module mirroring the existing `create_rule_rejects_invalid_nsql_and_persists_nothing`/`delete_rule_rejects_empty_id` style for: update-rule happy path + not-found + invalid-nsql rejection; export-rules sql vs json format; import-rules partial-failure (mix of valid and invalid lines) reporting both `imported_count` and `errors`; get-execution-logs returning entries in the right order/shape. 4. Replace the four `FilterSubcommand` arms in `runner.rs` with real `connect_filters_client()`-backed implementations. 5. Add the unauthenticated-rejection regression test. 6. Update/remove the now-stale comment block in `runner.rs` (lines 453-464) explaining why these four subcommands were exceptions — after this story they no longer are. 7. Run `cargo verify`; regenerate the contract golden if the descriptor test fails. ## Acceptance criteria - [ ] New RPCs `UpdateRule`, `ExportRules`, `ImportRules`, `GetExecutionLogs` added to `service Filters`; new messages `UpdateRuleRequest`/`UpdateRuleResponse`/`ExportRulesRequest`/`ExportRulesResponse`/`ImportRulesRequest`/`ImportRulesResponse`/`GetExecutionLogsRequest`/`GetExecutionLogsResponse`/`FilterExecutionLog` added, all additive within `nuncio.v1`. - [ ] Contract golden regenerated; `nuncio-proto`'s `contract_stability::descriptor_matches_committed_golden` test passes. - [ ] `filter edit`/`filter export`/`filter import`/`filter logs` in the CLI go through the daemon's real, persistent store and live `FilterEngine` — a rule created via `filter create` now visibly appears in `filter export`/`filter logs` output (prove this with an E2E: create via one RPC, read back via the corresponding new RPC). - [ ] `ImportRules` honestly reports per-line parse failures in `errors` rather than silently dropping them. - [ ] An unauthenticated call to at least one of the new RPCs is rejected with `Code::Unauthenticated` (regression proving the new RPCs inherited `Filters`'s existing `BearerAuthInterceptor`; no new service mounting was needed since `Filters` already exists). - [ ] Offline E2E (mock-only, no live network) proves the full round trip for at least update + export + import + logs. - [ ] `cargo verify` green (fmt + clippy `-D warnings` all-targets + test-all), 0 ignored. ## Definition of done & process Follow [`docs/STORY-WORKFLOW.md`](https://github.com/KofTwentyTwo/nuncio/blob/dev/docs/STORY-WORKFLOW.md): no fabricated data; comment convention (intent-only, no story/GH/phase refs in `.rs`/`.proto`); mock-only tests. Branch `story/m4-filters-grpc-mgmt` off `dev`; PR to `dev` with the gate output pasted as evidence; the PO reviews before the issue is closed. ## Dependencies / notes **[PROTO]: confirm with the PO that no other proto-touching story is in flight before starting** (`M4-S4` is also `[PROTO]` and touches the same `service Filters` block — coordinate sequencing with the PO; landing this one first is recommended since it is the smaller, more self-contained of the two, but there is no hard code dependency between them). Independent of `M4-S1`/`M4-S2` (pure management-surface relocation, no evaluation-semantics changes). Reconciles GH #175. Size: M.

KofTwentyTwo commented

2026-07-30 00:37:21 +00:00

(Migrated from github.com)

Delivered via PR #224 (squash-merged to dev). UpdateRule/ExportRules/ImportRules/GetExecutionLogs added to the authenticated Filters service (additive within nuncio.v1; golden regenerated); CLI filter edit/export/import/logs now go through the daemon's persistent store + live FilterEngine instead of an ephemeral local DB; ImportRules surfaces per-line parse errors instead of dropping them. Adversarial review: spec ✅, quality approved, no blocking findings. Reconciles #175.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: KofTwentyTwo/nuncio#192