feat!: qbit-webhooks 1.0 (signed, SSRF-safe, replica-safe outbound webhooks) #8

2026-10-12T21:49:41Z

KofTwentyTwo commented

2026-10-12 21:49:41 +00:00

(Migrated from github.com)

Builds qbit-webhooks 1.0 per the assessment (QRun-IO/qqq#1105). Owner decisions D1–D8 were adopted as recommended. This release is breaking; see docs/migration-0.x-to-1.0.md.

What's in it

Tables (D3): webhooks_endpoint, webhooks_endpoint_secret, webhooks_subscription, webhooks_delivery, webhooks_delivery_attempt.
- Lowercase snake_case; the backend name equals the table name; columns are snake_case.
- DDL is in docs/ddl/webhooks-postgres.sql.
- Permissions are webhooks.view, webhooks.manage and webhooks.admin.
Signing: Standard Webhooks.
- Headers: webhook-id (the event id and idempotency key), webhook-timestamp, and webhook-signature: v1,<base64 HMAC-SHA256("<id>.<ts>.<body>")>.
- Per-endpoint secrets are either AES-256-GCM encrypted (key in WEBHOOKS_SECRET_KEY) or an environment reference.
- Rotation keeps old and new secrets signing during a 24 h overlap.
- A secret value never appears in results, logs or audits. Admins see it in an admin-only widget on the secret record.
- Receiver docs with Java, Node.js and Python verification code: docs/RECEIVERS.md.
SSRF-safe client (D2, D4). The following are blocked:
- private, loopback, link-local, CGNAT, multicast, reserved and documentation ranges, plus IPv6 ULA and link-local, with embedded IPv4 checked;
- cloud metadata addresses, even when allowlisted;
- every redirect (it is never followed).
How it works:
- DNS is resolved once, and the connection is pinned to the checked address.
- TLS 1.2 or 1.3; connect, read and total timeouts (httpTimeoutMillis is now honoured); a 64 KiB response cap.
- Internal targets need an endpoint allowlist that only webhooks.admin can set.
- shared/ is an internal copy of the core classes in QRun-IO/qqq#1109. It is replaced when QQQ 4.1.0-RC.5 publishes (DESIGN §11), because a qbit can only depend on a published core.
Event sources:
- Table insert, update and delete, through the QQQ 4.1 record change listener and the transactional outbox.
- Process completed and failed.
- ESB topics (WebhooksEsbSource): this instance's own events are skipped, so nothing goes out twice alongside the in-process path, and deliveries are deduplicated by CloudEvent id.
- Custom domain events (Webhooks.fire).
- A channel API for the Alerts qbit (Webhooks.deliver, idempotent).
Delivery (D8): at least once.
- Atomic claims: the unique (delivery, attempt number) insert, with leases and lease-expiry recovery.
- Retries with backoff and Retry-After; the runner never sleeps on a 429.
- Dead letters with Replay.
- Optional in-order delivery per endpoint and a per-endpoint rate limit.
- Health states: Healthy, Unhealthy, Probation, then auto-disabled.
- Alerts are raised through the Alerts producer SPI when present (by reflection, so it stays optional).
- Retention purge with long defaults; dead letters are kept forever.
Subscriptions:
- Record filters (QQQ filter JSON) and record-security matching; subscribing needs read permission on the table.
- Detail levels, plus "only these" and "never these" fields, and a code hook for the payload.
App:
- A real home dashboard: delivered and failed counts, dead letters, backlog, endpoints needing attention, latency p50/p95 and attempts per hour.
- The nav has only Endpoints, Subscriptions and Deliveries; secrets and attempts are relations on their parents.
- Six table and record actions with the standard review and result screens, none in the nav. Delivery, maintenance and retention run on the scheduler.
Integrations: WebhooksIntegrations provides ESB, Quick Search and API table hooks, with the same rules as the admin.
Repo standard:
- Default-active 4.1.0-RC.4 profile, JaCoCo gate, Checkstyle in verify, the Maven wrapper.
- Standard files: LICENSE, NOTICE, pointer files, dependabot, .gitattributes and .editorconfig.
- The README now matches the code. docs/DESIGN.md includes the phase 2 inbound design (D5).

Tests

./mvnw verify: 144 tests, 0 failures; Checkstyle clean; JaCoCo 91.7 % instructions, 82/82 classes. Highlights:

Area	Covered
Signing	Verify, tamper, rotation overlap, retirement, same id on retry, the Standard Webhooks reference vector
SSRF	Every blocked range; DNS rebinding blocked at send time; redirects not followed; internal URLs refused on save, including metadata despite an allowlist
Timeouts	Read, total and connect timeouts; response size cap
No double sends (two replicas)	H2: 2 instances × 2 runners, 150 deliveries, each received exactly once, work shared; plus expired-lease recovery
Dead letters	Dead letter, Replay, Send Now
Sources	ESB-sourced events (mapping, own-event skip, redelivery dedup); delete events; process events
Rules	Filters, field conditions, payload shaping; permissions (checker mapping, admin-only internal targets and secrets, read check on subscribe, record security)
Home	Empty state, real data, escaping, the narrow-screen layout
Persistence	A restart on an H2 file database; transactional outbox rollback

Notes for review

The 0.x sources are still in the tree. Removing them was not approved in this session. pom.xml excludes them from compile, test-compile and Checkstyle (marked TEMPORARY). Delete them and the excludes together once approved.
Alerts SPI: the test package com.kingsrook.qqq.backend.core.alerts is a stand-in for the Alerts SPI (#1090), which is not yet in core. Delete it when core ships the SPI.
Release: 1.0.0-RC.1 is cut from release/1.0 by the release captain.

Next UI check (added)

Checked with Next UI 1.0.0-RC.14 using the test-scoped WebhooksDemoServer (H2, 24 h of real traffic from the engine). The home, an endpoint record and a delivery record were each captured in light and dark at 390 and 1440 px. There was no page-level horizontal scroll and no inner scrollers in any of the 12 views. No qqq-frontend-next change is needed and the bundle budget is untouched.

That check led to these changes (commit f6ecc75):

The home uses inline styles only, because Next's sanitizer strips <style>. Its responsive grids need no media queries.
The relation widgets and the payload are full width.
Deliveries and attempts drop their insert, update and delete capabilities, so the UI shows no Edit or New.

kof22 adoption: KofTwentyTwo/Website-Backend#140 (draft until RC.1 is on Central).

🤖 Generated with Claude Code

Builds qbit-webhooks 1.0 per the assessment (QRun-IO/qqq#1105). Owner decisions D1–D8 were adopted as recommended. This release is breaking; see `docs/migration-0.x-to-1.0.md`. ## What's in it - **Tables** (D3): `webhooks_endpoint`, `webhooks_endpoint_secret`, `webhooks_subscription`, `webhooks_delivery`, `webhooks_delivery_attempt`. - Lowercase snake_case; the backend name equals the table name; columns are snake_case. - DDL is in `docs/ddl/webhooks-postgres.sql`. - Permissions are `webhooks.view`, `webhooks.manage` and `webhooks.admin`. - **Signing:** Standard Webhooks. - Headers: `webhook-id` (the event id and idempotency key), `webhook-timestamp`, and `webhook-signature: v1,<base64 HMAC-SHA256("<id>.<ts>.<body>")>`. - Per-endpoint secrets are either AES-256-GCM encrypted (key in `WEBHOOKS_SECRET_KEY`) or an environment reference. - Rotation keeps old and new secrets signing during a 24 h overlap. - A secret value never appears in results, logs or audits. Admins see it in an admin-only widget on the secret record. - Receiver docs with Java, Node.js and Python verification code: `docs/RECEIVERS.md`. - **SSRF-safe client** (D2, D4). The following are blocked: - private, loopback, link-local, CGNAT, multicast, reserved and documentation ranges, plus IPv6 ULA and link-local, with embedded IPv4 checked; - cloud metadata addresses, even when allowlisted; - every redirect (it is never followed). How it works: - DNS is resolved once, and the connection is pinned to the checked address. - TLS 1.2 or 1.3; connect, read and total timeouts (`httpTimeoutMillis` is now honoured); a 64 KiB response cap. - Internal targets need an endpoint allowlist that only `webhooks.admin` can set. - `shared/` is an **internal copy** of the core classes in QRun-IO/qqq#1109. It is replaced when QQQ 4.1.0-RC.5 publishes (DESIGN §11), because a qbit can only depend on a published core. - **Event sources:** - Table insert, update **and delete**, through the QQQ 4.1 record change listener and the transactional outbox. - **Process** completed and failed. - **ESB topics** (`WebhooksEsbSource`): this instance's own events are skipped, so nothing goes out twice alongside the in-process path, and deliveries are deduplicated by CloudEvent id. - **Custom** domain events (`Webhooks.fire`). - A **channel API** for the Alerts qbit (`Webhooks.deliver`, idempotent). - **Delivery** (D8): at least once. - Atomic claims: the unique (delivery, attempt number) insert, with leases and lease-expiry recovery. - Retries with backoff and `Retry-After`; the runner never sleeps on a 429. - Dead letters with Replay. - Optional in-order delivery per endpoint and a per-endpoint rate limit. - Health states: Healthy, Unhealthy, Probation, then auto-disabled. - Alerts are raised through the Alerts producer SPI when present (by reflection, so it stays optional). - Retention purge with long defaults; dead letters are kept forever. - **Subscriptions:** - Record filters (QQQ filter JSON) and record-security matching; subscribing needs read permission on the table. - Detail levels, plus "only these" and "never these" fields, and a code hook for the payload. - **App:** - A real home dashboard: delivered and failed counts, dead letters, backlog, endpoints needing attention, latency p50/p95 and attempts per hour. - The nav has only Endpoints, Subscriptions and Deliveries; secrets and attempts are relations on their parents. - Six table and record actions with the standard review and result screens, none in the nav. Delivery, maintenance and retention run on the scheduler. - **Integrations:** `WebhooksIntegrations` provides ESB, Quick Search and API table hooks, with the same rules as the admin. - **Repo standard:** - Default-active 4.1.0-RC.4 profile, JaCoCo gate, Checkstyle in `verify`, the Maven wrapper. - Standard files: LICENSE, NOTICE, pointer files, dependabot, .gitattributes and .editorconfig. - The README now matches the code. `docs/DESIGN.md` includes the phase 2 inbound design (D5). ## Tests `./mvnw verify`: **144 tests, 0 failures**; Checkstyle clean; JaCoCo **91.7 % instructions, 82/82 classes**. Highlights: | Area | Covered | |---|---| | Signing | Verify, tamper, rotation overlap, retirement, same id on retry, the Standard Webhooks reference vector | | SSRF | Every blocked range; DNS rebinding blocked at send time; redirects not followed; internal URLs refused on save, including metadata despite an allowlist | | Timeouts | Read, total and connect timeouts; response size cap | | No double sends (two replicas) | H2: 2 instances × 2 runners, 150 deliveries, each received exactly once, work shared; plus expired-lease recovery | | Dead letters | Dead letter, Replay, Send Now | | Sources | ESB-sourced events (mapping, own-event skip, redelivery dedup); delete events; process events | | Rules | Filters, field conditions, payload shaping; permissions (checker mapping, admin-only internal targets and secrets, read check on subscribe, record security) | | Home | Empty state, real data, escaping, the narrow-screen layout | | Persistence | A restart on an H2 file database; transactional outbox rollback | ## Notes for review - **The 0.x sources are still in the tree.** Removing them was not approved in this session. `pom.xml` excludes them from compile, test-compile and Checkstyle (marked TEMPORARY). Delete them and the excludes together once approved. - **Alerts SPI:** the test package `com.kingsrook.qqq.backend.core.alerts` is a stand-in for the Alerts SPI (#1090), which is not yet in core. Delete it when core ships the SPI. - **Release:** 1.0.0-RC.1 is cut from `release/1.0` by the release captain. ## Next UI check (added) Checked with Next UI 1.0.0-RC.14 using the test-scoped `WebhooksDemoServer` (H2, 24 h of real traffic from the engine). The home, an endpoint record and a delivery record were each captured in light and dark at 390 and 1440 px. There was no page-level horizontal scroll and no inner scrollers in any of the 12 views. No qqq-frontend-next change is needed and the bundle budget is untouched. That check led to these changes (commit f6ecc75): - The home uses inline styles only, because Next's sanitizer strips `<style>`. Its responsive grids need no media queries. - The relation widgets and the payload are full width. - Deliveries and attempts drop their insert, update and delete capabilities, so the UI shows no Edit or New. **kof22 adoption:** KofTwentyTwo/Website-Backend#140 (draft until RC.1 is on Central). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: QRun-IO/qbit-webhooks#8