diff --git a/AGENTS.md b/AGENTS.md index 3e4e33f..bedd90c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -31,7 +31,7 @@ Design: `docs/DESIGN.md` (from QRun-IO/qqq#1105). Inbound webhooks are phase 2 ( - `DeliveryEngine` (claim, send, outcome), `EndpointHealth` - `WebhooksAlerts` (reflective Alerts SPI) - `secrets` (AES-GCM, env references, rotation), `processes`, `app` (home, secret widget), `permissions`, `esb`, `support/SystemWrites`. - - `shared`: INTERNAL COPY of the core signer and safe HTTP client (QRun-IO/qqq#1109). Replace with `core.utils.webhooks` / `core.utils.http` at QQQ 4.1.0-RC.5 (DESIGN section 11). Keep it identical to core until then. + - Signing uses `core.utils.webhooks` directly at the QQQ 4.1.0-RC.5 baseline (QRun-IO/qqq#1160). `shared.http` remains an internal copy of the safe HTTP client pending a separate migration to `core.utils.http` (DESIGN section 11). ## Invariants diff --git a/CHANGELOG.md b/CHANGELOG.md index 7bc2003..7651621 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,10 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and ## [Unreleased] +### Changed +- Require QQQ 4.1.0-RC.5 and use its webhook signer, verifier, secret and verification exception (QRun-IO/qqq#1160, F11). Removed the internal signing package; Java consumers must import `com.kingsrook.qqq.backend.core.utils.webhooks` instead. +- Signing wire format, timestamp tolerance, rotation, endpoint management, encrypted secret storage and delivery behavior are unchanged. + ## [1.0.0] - not yet released A rebuild of the qbit on QQQ 4.1 (design: QRun-IO/qqq#1105, docs/DESIGN.md). Breaking: see docs/migration-0.x-to-1.0.md. diff --git a/CLAUDE.md b/CLAUDE.md index ad4e93f..f2659d8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -31,7 +31,7 @@ Design: `docs/DESIGN.md` (from QRun-IO/qqq#1105). Inbound webhooks are phase 2 ( - `DeliveryEngine` (claim, send, outcome), `EndpointHealth` - `WebhooksAlerts` (reflective Alerts SPI) - `secrets` (AES-GCM, env references, rotation), `processes`, `app` (home, secret widget), `permissions`, `esb`, `support/SystemWrites`. - - `shared`: INTERNAL COPY of the core signer and safe HTTP client (QRun-IO/qqq#1109). Replace with `core.utils.webhooks` / `core.utils.http` at QQQ 4.1.0-RC.5 (DESIGN section 11). Keep it identical to core until then. + - Signing uses `core.utils.webhooks` directly at the QQQ 4.1.0-RC.5 baseline (QRun-IO/qqq#1160). `shared.http` remains an internal copy of the safe HTTP client pending a separate migration to `core.utils.http` (DESIGN section 11). ## Invariants diff --git a/README.md b/README.md index b5a1469..8e17069 100644 --- a/README.md +++ b/README.md @@ -2,12 +2,12 @@ Signed, SSRF-safe outbound webhooks for QQQ applications. Admins add **endpoints** and **subscribe** them to event types: table inserts, updates and deletes (in-process or from the ESB), process completions and failures, and custom domain events. The qbit queues one **delivery** per event per endpoint in the same transaction as the change. It then signs each delivery in the [Standard Webhooks](https://www.standardwebhooks.com/) format and sends it with retries, backoff, dead letters and endpoint health. Every row is a normal QQQ table, and the app has a home dashboard. -Design: [QRun-IO/qqq#1105](https://github.com/QRun-IO/qqq/pull/1105) (`docs/specs/2026-10-12-qbit-webhooks-assessment.md`), and [docs/DESIGN.md](docs/DESIGN.md) for what 1.0 implements. The shared signer and safe HTTP client are in QQQ core from 4.1.0-RC.5 ([QRun-IO/qqq#1109](https://github.com/QRun-IO/qqq/pull/1109)). +Design: [QRun-IO/qqq#1105](https://github.com/QRun-IO/qqq/pull/1105) (`docs/specs/2026-10-12-qbit-webhooks-assessment.md`), and [docs/DESIGN.md](docs/DESIGN.md) for what 1.0 implements. Signing and secret generation use QQQ core directly; the local safe HTTP client remains pending a separate migration. Both primitives are available in QQQ core from 4.1.0-RC.5 ([QRun-IO/qqq#1109](https://github.com/QRun-IO/qqq/pull/1109)). ## Requirements - Java 21 -- QQQ 4.1.0-RC.4 or later (`qqq-backend-core`). Two modules are optional: +- QQQ 4.1.0-RC.5 or later (`qqq-backend-core`). Two modules are optional: - `qqq-middleware-api`, for payloads rendered through a QQQ API version. - `qqq-esb`, for ESB topics as an event source. - A SQL backend for the five tables (DDL in [docs/ddl/webhooks-postgres.sql](docs/ddl/webhooks-postgres.sql)), and a QQQ scheduler for delivery. diff --git a/docs/DESIGN.md b/docs/DESIGN.md index 9508877..36d0948 100644 --- a/docs/DESIGN.md +++ b/docs/DESIGN.md @@ -6,7 +6,7 @@ | # | Decision | |---|---| | D1 | Enhance the existing qbit into 1.0; breaking changes allowed | -| D2 | The signer and SSRF-safe client live in `qqq-backend-core` (QRun-IO/qqq#1109). This qbit carries an identical internal copy in `com.kingsrook.qbits.webhooks.shared` until QQQ 4.1.0-RC.5 is published (see section 11) | +| D2 | The signer and SSRF-safe client live in `qqq-backend-core` (QRun-IO/qqq#1109). Signing uses core directly at the QQQ 4.1.0-RC.5 baseline; the internal HTTP copy remains pending a separate migration (see section 11) | | D3 | Tables renamed to `webhooks_*` | | D4 | Internal and private targets blocked by default; admin-only allowlist per endpoint | | D5 | Inbound is phase 2, in this qbit | @@ -265,10 +265,16 @@ Goal: verify and record signed callbacks from third parties (Authentik, GitHub, ## 11. Moving to the core primitives (QQQ 4.1.0-RC.5) -QRun-IO/qqq#1109 adds the same classes to `qqq-backend-core` (`com.kingsrook.qqq.backend.core.utils.webhooks` and `...utils.http`). When RC.5 is on Maven Central: +QRun-IO/qqq#1160 (F11) migrates signing to the released `qqq-bom-pom` 4.1.0-RC.5 baseline. Delivery signing, secret generation/rotation and receiver verification use `com.kingsrook.qqq.backend.core.utils.webhooks` directly. The local `shared.signing` package is removed; endpoints, encrypted secret storage and delivery remain in this qbit. Inbound routes are still phase 2 (design only). -1. Bump `qqq.snapshot.version` to `4.1.0-RC.5`. -2. Delete `src/main/java/com/kingsrook/qbits/webhooks/shared/` and its tests. -3. Change the imports (`shared.signing` to `core.utils.webhooks`, `shared.http` to `core.utils.http`). +Comparison against core sources at tag `v4.1.0-RC.5` found no executable differences in `WebhookSigner`, `WebhookVerifier`, `WebhookSecret` or `WebhookVerificationException`; only package names, copyright years and migration comments differed. No compatibility adapter is needed: -The behaviour is identical, and the classes were kept the same on purpose. +- Headers remain `webhook-id`, `webhook-timestamp`, `webhook-signature` (case-insensitive when verifying). +- HMAC-SHA256 signs UTF-8 `id.timestamp.` followed by unchanged body bytes; signatures are padded Base64 with `v1,` and a single space between rotation signatures, in secret-list order. +- Secrets remain `whsec_` plus padded Base64 key bytes; unprefixed secrets use trimmed UTF-8 bytes. Generation uses 32 bytes, with a 16-byte minimum for imported keys. +- Default timestamp tolerance remains five minutes in either direction, including the boundary. +- Verification accepts any matching `v1` entry with any known secret and ignores unknown signature versions. + +Signing contract tests retain the reference vector and test independent local HMAC signing against core verification and core signing against local verification, including two-secret rotation and exact tolerance boundaries. Delivery integration tests verify actual requests with core. + +The `shared.http` copy is outside F11 and remains for a separate migration to `com.kingsrook.qqq.backend.core.utils.http`. diff --git a/docs/RECEIVERS.md b/docs/RECEIVERS.md index 4005fc1..659bc9a 100644 --- a/docs/RECEIVERS.md +++ b/docs/RECEIVERS.md @@ -28,6 +28,9 @@ webhook-attempt: 1 ## Java (QQQ core 4.1.0-RC.5 or later) ```java +import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret; +import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerifier; + WebhookVerifier verifier = new WebhookVerifier(List.of(WebhookSecret.parse(System.getenv("WEBHOOK_SECRET")))); verifier.verify(headersAsMap, rawBodyBytes); // throws WebhookVerificationException with a safe reason ``` diff --git a/pom.xml b/pom.xml index b625d5b..d7b100b 100644 --- a/pom.xml +++ b/pom.xml @@ -215,7 +215,7 @@ true - 4.1.0-RC.4 + 4.1.0-RC.5 diff --git a/src/main/java/com/kingsrook/qbits/webhooks/delivery/DeliveryEngine.java b/src/main/java/com/kingsrook/qbits/webhooks/delivery/DeliveryEngine.java index c93d9d8..7e8d73b 100644 --- a/src/main/java/com/kingsrook/qbits/webhooks/delivery/DeliveryEngine.java +++ b/src/main/java/com/kingsrook/qbits/webhooks/delivery/DeliveryEngine.java @@ -42,8 +42,6 @@ import com.kingsrook.qbits.webhooks.secrets.EndpointSecrets; import com.kingsrook.qbits.webhooks.shared.http.SafeHttpClient; import com.kingsrook.qbits.webhooks.shared.http.SafeHttpException; import com.kingsrook.qbits.webhooks.shared.http.SafeHttpResponse; -import com.kingsrook.qbits.webhooks.shared.signing.WebhookSecret; -import com.kingsrook.qbits.webhooks.shared.signing.WebhookSigner; import com.kingsrook.qbits.webhooks.support.SystemWrites; import com.kingsrook.qqq.backend.core.exceptions.QException; import com.kingsrook.qqq.backend.core.logging.QLogger; @@ -55,6 +53,8 @@ import com.kingsrook.qqq.backend.core.model.data.QRecord; import com.kingsrook.qqq.backend.core.utils.CollectionUtils; import com.kingsrook.qqq.backend.core.utils.StringUtils; import com.kingsrook.qqq.backend.core.utils.ValueUtils; +import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret; +import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSigner; import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair; diff --git a/src/main/java/com/kingsrook/qbits/webhooks/metadata/WebhooksTableCustomizers.java b/src/main/java/com/kingsrook/qbits/webhooks/metadata/WebhooksTableCustomizers.java index 9fe0688..2407302 100644 --- a/src/main/java/com/kingsrook/qbits/webhooks/metadata/WebhooksTableCustomizers.java +++ b/src/main/java/com/kingsrook/qbits/webhooks/metadata/WebhooksTableCustomizers.java @@ -38,7 +38,6 @@ import com.kingsrook.qbits.webhooks.registry.WebhooksRegistry; import com.kingsrook.qbits.webhooks.secrets.EnvironmentSecrets; import com.kingsrook.qbits.webhooks.shared.http.AddressRules; import com.kingsrook.qbits.webhooks.shared.http.SafeHttpException; -import com.kingsrook.qbits.webhooks.shared.signing.WebhookSecret; import com.kingsrook.qbits.webhooks.support.SystemWrites; import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface; import com.kingsrook.qqq.backend.core.actions.permissions.PermissionsHelper; @@ -50,6 +49,7 @@ import com.kingsrook.qqq.backend.core.model.data.QRecord; import com.kingsrook.qqq.backend.core.model.statusmessages.BadInputStatusMessage; import com.kingsrook.qqq.backend.core.utils.StringUtils; import com.kingsrook.qqq.backend.core.utils.ValueUtils; +import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret; /******************************************************************************* diff --git a/src/main/java/com/kingsrook/qbits/webhooks/secrets/EndpointSecrets.java b/src/main/java/com/kingsrook/qbits/webhooks/secrets/EndpointSecrets.java index 109eaf8..2459c8c 100644 --- a/src/main/java/com/kingsrook/qbits/webhooks/secrets/EndpointSecrets.java +++ b/src/main/java/com/kingsrook/qbits/webhooks/secrets/EndpointSecrets.java @@ -30,7 +30,6 @@ import com.kingsrook.qbits.webhooks.WebhooksRuntime; import com.kingsrook.qbits.webhooks.metadata.WebhooksEnums.SecretSource; import com.kingsrook.qbits.webhooks.metadata.WebhooksEnums.SecretStatus; import com.kingsrook.qbits.webhooks.metadata.WebhooksTables; -import com.kingsrook.qbits.webhooks.shared.signing.WebhookSecret; import com.kingsrook.qbits.webhooks.support.SystemWrites; import com.kingsrook.qqq.backend.core.exceptions.QException; import com.kingsrook.qqq.backend.core.logging.QLogger; @@ -41,6 +40,7 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter; import com.kingsrook.qqq.backend.core.model.data.QRecord; import com.kingsrook.qqq.backend.core.utils.CollectionUtils; import com.kingsrook.qqq.backend.core.utils.ValueUtils; +import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret; import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair; diff --git a/src/main/java/com/kingsrook/qbits/webhooks/shared/signing/WebhookSecret.java b/src/main/java/com/kingsrook/qbits/webhooks/shared/signing/WebhookSecret.java deleted file mode 100644 index 96ae352..0000000 --- a/src/main/java/com/kingsrook/qbits/webhooks/shared/signing/WebhookSecret.java +++ /dev/null @@ -1,183 +0,0 @@ -/* - * QQQ - Low-code Application Framework for Engineers. - * Copyright (C) 2021-2025. Kingsrook, LLC - * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States - * contact@kingsrook.com - * https://github.com/Kingsrook/ - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.kingsrook.qbits.webhooks.shared.signing; - - -import java.nio.charset.StandardCharsets; -import java.security.MessageDigest; -import java.security.NoSuchAlgorithmException; -import java.security.SecureRandom; -import java.util.Arrays; -import java.util.Base64; -import java.util.HexFormat; - - -/******************************************************************************* - ** A webhook signing secret (Standard Webhooks format). - ** - **

The text form is {@code whsec_} followed by the base64 of the key bytes. - ** The HMAC key is the decoded bytes. A secret without the prefix is used as - ** raw UTF-8 bytes, so a secret shared with an older receiver still works.

- ** - **

{@link #toString()} never reveals the key: it prints the fingerprint.

- ** - **

INTERNAL COPY: replaced by {@code com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret} - ** when the qbit moves to QQQ 4.1.0-RC.5 (QRun-IO/qqq, D2 of the webhooks assessment). - ** Keep the two identical.

- *******************************************************************************/ -public final class WebhookSecret -{ - public static final String PREFIX = "whsec_"; - - /** Key length for generated secrets, in bytes (256 bits). */ - public static final int GENERATED_KEY_BYTES = 32; - - /** Shortest key accepted, in bytes (128 bits). */ - public static final int MINIMUM_KEY_BYTES = 16; - - private static final SecureRandom RANDOM = new SecureRandom(); - - private final byte[] key; - - - - /******************************************************************************* - ** Constructor (keeps a copy of the key). - *******************************************************************************/ - private WebhookSecret(byte[] key) - { - if(key == null || key.length < MINIMUM_KEY_BYTES) - { - throw (new IllegalArgumentException("A webhook signing secret must be at least " + MINIMUM_KEY_BYTES + " bytes")); - } - this.key = Arrays.copyOf(key, key.length); - } - - - - /******************************************************************************* - ** A new random secret. - *******************************************************************************/ - public static WebhookSecret generate() - { - byte[] bytes = new byte[GENERATED_KEY_BYTES]; - RANDOM.nextBytes(bytes); - return (new WebhookSecret(bytes)); - } - - - - /******************************************************************************* - ** Parse the text form ({@code whsec_}, or raw text). - *******************************************************************************/ - public static WebhookSecret parse(String text) - { - if(text == null || text.isBlank()) - { - throw (new IllegalArgumentException("A webhook signing secret is required")); - } - String trimmed = text.trim(); - if(trimmed.startsWith(PREFIX)) - { - try - { - return (new WebhookSecret(Base64.getDecoder().decode(trimmed.substring(PREFIX.length())))); - } - catch(IllegalArgumentException e) - { - throw (new IllegalArgumentException("A " + PREFIX + " secret must be base64 after the prefix", e)); - } - } - return (new WebhookSecret(trimmed.getBytes(StandardCharsets.UTF_8))); - } - - - - /******************************************************************************* - ** The text form to give a receiver ({@code whsec_}). - *******************************************************************************/ - public String reveal() - { - return (PREFIX + Base64.getEncoder().encodeToString(key)); - } - - - - /******************************************************************************* - ** The HMAC key (a copy). - *******************************************************************************/ - public byte[] keyBytes() - { - return (Arrays.copyOf(key, key.length)); - } - - - - /******************************************************************************* - ** A short, non-reversible id for the secret (first 12 hex chars of its - ** SHA-256), safe to store and show, so people can tell secrets apart. - *******************************************************************************/ - public String fingerprint() - { - try - { - byte[] digest = MessageDigest.getInstance("SHA-256").digest(key); - return (HexFormat.of().formatHex(digest).substring(0, 12)); - } - catch(NoSuchAlgorithmException e) - { - throw (new IllegalStateException("SHA-256 is unavailable", e)); - } - } - - - - /******************************************************************************* - ** Never prints the key. - *******************************************************************************/ - @Override - public String toString() - { - return ("WebhookSecret[" + fingerprint() + "]"); - } - - - - /******************************************************************************* - ** Equal when the keys are equal (constant time). - *******************************************************************************/ - @Override - public boolean equals(Object o) - { - return (o instanceof WebhookSecret other && MessageDigest.isEqual(key, other.key)); - } - - - - /******************************************************************************* - ** Hash of the fingerprint. - *******************************************************************************/ - @Override - public int hashCode() - { - return (fingerprint().hashCode()); - } -} diff --git a/src/main/java/com/kingsrook/qbits/webhooks/shared/signing/WebhookSigner.java b/src/main/java/com/kingsrook/qbits/webhooks/shared/signing/WebhookSigner.java deleted file mode 100644 index 98c7f66..0000000 --- a/src/main/java/com/kingsrook/qbits/webhooks/shared/signing/WebhookSigner.java +++ /dev/null @@ -1,147 +0,0 @@ -/* - * QQQ - Low-code Application Framework for Engineers. - * Copyright (C) 2021-2025. Kingsrook, LLC - * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States - * contact@kingsrook.com - * https://github.com/Kingsrook/ - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.kingsrook.qbits.webhooks.shared.signing; - - -import javax.crypto.Mac; -import javax.crypto.spec.SecretKeySpec; -import java.nio.charset.StandardCharsets; -import java.security.GeneralSecurityException; -import java.time.Instant; -import java.util.ArrayList; -import java.util.Base64; -import java.util.LinkedHashMap; -import java.util.List; -import java.util.Map; - - -/******************************************************************************* - ** Signs webhook requests in the Standard Webhooks format - ** (https://www.standardwebhooks.com/). - ** - **

The signed content is {@code ..}: the message id, - ** a dot, the Unix timestamp in seconds, a dot, then the exact body bytes. The - ** signature is HMAC-SHA256 with the secret's key, base64 encoded, and sent as - ** {@code v1,}. During a secret rotation every active secret signs, and - ** the signatures are separated by spaces, so a receiver holding either the old - ** or the new secret accepts the request.

- ** - **

Headers: {@code webhook-id}, {@code webhook-timestamp}, - ** {@code webhook-signature}.

- ** - **

INTERNAL COPY: replaced by {@code com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSigner} - ** when the qbit moves to QQQ 4.1.0-RC.5. Keep the two identical.

- *******************************************************************************/ -public final class WebhookSigner -{ - public static final String HEADER_ID = "webhook-id"; - public static final String HEADER_TIMESTAMP = "webhook-timestamp"; - public static final String HEADER_SIGNATURE = "webhook-signature"; - - /** The only signature scheme today: HMAC-SHA256. */ - public static final String VERSION_V1 = "v1"; - - private static final String HMAC = "HmacSHA256"; - - - - /******************************************************************************* - ** Utility class. - *******************************************************************************/ - private WebhookSigner() - { - } - - - - /******************************************************************************* - ** The three signature headers for one request. - ** - ** @param messageId the event id (stable across retries: it is the idempotency key) - ** @param timestamp when this attempt is sent - ** @param body the exact bytes that are sent - ** @param secrets the active secrets (one, or two during a rotation) - *******************************************************************************/ - public static Map headers(String messageId, Instant timestamp, byte[] body, List secrets) - { - long seconds = timestamp.getEpochSecond(); - Map headers = new LinkedHashMap<>(); - headers.put(HEADER_ID, messageId); - headers.put(HEADER_TIMESTAMP, Long.toString(seconds)); - headers.put(HEADER_SIGNATURE, signatureHeader(messageId, seconds, body, secrets)); - return (headers); - } - - - - /******************************************************************************* - ** The {@code webhook-signature} value: {@code v1,} per secret, space - ** separated. - *******************************************************************************/ - public static String signatureHeader(String messageId, long timestampSeconds, byte[] body, List secrets) - { - if(secrets == null || secrets.isEmpty()) - { - throw (new IllegalArgumentException("At least one signing secret is required")); - } - List parts = new ArrayList<>(); - for(WebhookSecret secret : secrets) - { - parts.add(VERSION_V1 + "," + sign(messageId, timestampSeconds, body, secret)); - } - return (String.join(" ", parts)); - } - - - - /******************************************************************************* - ** One base64 HMAC-SHA256 signature over {@code ..}. - *******************************************************************************/ - public static String sign(String messageId, long timestampSeconds, byte[] body, WebhookSecret secret) - { - validateId(messageId); - try - { - Mac mac = Mac.getInstance(HMAC); - mac.init(new SecretKeySpec(secret.keyBytes(), HMAC)); - mac.update((messageId + "." + timestampSeconds + ".").getBytes(StandardCharsets.UTF_8)); - mac.update(body == null ? new byte[0] : body); - return (Base64.getEncoder().encodeToString(mac.doFinal())); - } - catch(GeneralSecurityException e) - { - throw (new IllegalStateException("HMAC-SHA256 is unavailable", e)); - } - } - - - - /******************************************************************************* - ** A message id may not contain the separator or whitespace. - *******************************************************************************/ - static void validateId(String messageId) - { - if(messageId == null || messageId.isEmpty() || messageId.contains(".") || messageId.chars().anyMatch(Character::isWhitespace)) - { - throw (new IllegalArgumentException("A webhook message id is required and may not contain '.' or whitespace")); - } - } -} diff --git a/src/main/java/com/kingsrook/qbits/webhooks/shared/signing/WebhookVerificationException.java b/src/main/java/com/kingsrook/qbits/webhooks/shared/signing/WebhookVerificationException.java deleted file mode 100644 index 4dcf41c..0000000 --- a/src/main/java/com/kingsrook/qbits/webhooks/shared/signing/WebhookVerificationException.java +++ /dev/null @@ -1,70 +0,0 @@ -/* - * QQQ - Low-code Application Framework for Engineers. - * Copyright (C) 2021-2025. Kingsrook, LLC - * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States - * contact@kingsrook.com - * https://github.com/Kingsrook/ - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.kingsrook.qbits.webhooks.shared.signing; - - -/******************************************************************************* - ** A webhook request failed verification. The message never contains a - ** secret or a signature, so it is safe to log and to return to the caller. - ** - **

INTERNAL COPY: replaced by the class of the same name in - ** {@code com.kingsrook.qqq.backend.core.utils.webhooks} at QQQ 4.1.0-RC.5.

- *******************************************************************************/ -public class WebhookVerificationException extends Exception -{ - private static final long serialVersionUID = 1L; - - /******************************************************************************* - ** Why verification failed. - *******************************************************************************/ - public enum Reason - { - MISSING_HEADERS, - BAD_TIMESTAMP, - TIMESTAMP_OUT_OF_TOLERANCE, - NO_MATCHING_SIGNATURE - } - - - - private final Reason reason; - - - - /******************************************************************************* - ** Constructor. - *******************************************************************************/ - public WebhookVerificationException(Reason reason, String message) - { - super(message); - this.reason = reason; - } - - - - /******************************************************************************* - ** Getter for reason. - *******************************************************************************/ - public Reason getReason() - { - return (reason); - } -} diff --git a/src/main/java/com/kingsrook/qbits/webhooks/shared/signing/WebhookVerifier.java b/src/main/java/com/kingsrook/qbits/webhooks/shared/signing/WebhookVerifier.java deleted file mode 100644 index 23133f7..0000000 --- a/src/main/java/com/kingsrook/qbits/webhooks/shared/signing/WebhookVerifier.java +++ /dev/null @@ -1,165 +0,0 @@ -/* - * QQQ - Low-code Application Framework for Engineers. - * Copyright (C) 2021-2025. Kingsrook, LLC - * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States - * contact@kingsrook.com - * https://github.com/Kingsrook/ - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.kingsrook.qbits.webhooks.shared.signing; - - -import java.nio.charset.StandardCharsets; -import java.security.MessageDigest; -import java.time.Clock; -import java.time.Duration; -import java.util.ArrayList; -import java.util.List; -import java.util.Locale; -import java.util.Map; - - -/******************************************************************************* - ** Verifies a signed webhook request (Standard Webhooks format, see - ** {@link WebhookSigner}). - ** - **

Checks, in order: the three headers are present, the timestamp is within - ** the tolerance of now (default five minutes, either direction, which stops - ** replays of old requests), and at least one {@code v1} signature matches one - ** of the receiver's secrets. Comparison is constant time. Unknown signature - ** versions are ignored, so a future scheme can be added alongside v1.

- ** - **

Receivers should also remember recent {@code webhook-id}s and ignore a - ** repeat: the same id is sent again on every retry of the same event.

- ** - **

INTERNAL COPY: replaced by {@code com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerifier} - ** when the qbit moves to QQQ 4.1.0-RC.5. Keep the two identical.

- *******************************************************************************/ -public final class WebhookVerifier -{ - public static final Duration DEFAULT_TOLERANCE = Duration.ofMinutes(5); - - private final List secrets; - private final Duration tolerance; - private final Clock clock; - - - - /******************************************************************************* - ** Verifier with the default tolerance and the system clock. - *******************************************************************************/ - public WebhookVerifier(List secrets) - { - this(secrets, DEFAULT_TOLERANCE, Clock.systemUTC()); - } - - - - /******************************************************************************* - ** Verifier. - *******************************************************************************/ - public WebhookVerifier(List secrets, Duration tolerance, Clock clock) - { - if(secrets == null || secrets.isEmpty()) - { - throw (new IllegalArgumentException("At least one secret is required to verify webhooks")); - } - this.secrets = new ArrayList<>(secrets); - this.tolerance = tolerance; - this.clock = clock; - } - - - - /******************************************************************************* - ** Verify a request. Header names are matched case-insensitively. - ** - ** @throws WebhookVerificationException with a reason safe to log - *******************************************************************************/ - public void verify(Map headers, byte[] body) throws WebhookVerificationException - { - String id = header(headers, WebhookSigner.HEADER_ID); - String timestamp = header(headers, WebhookSigner.HEADER_TIMESTAMP); - String signature = header(headers, WebhookSigner.HEADER_SIGNATURE); - if(id == null || timestamp == null || signature == null) - { - throw (new WebhookVerificationException(WebhookVerificationException.Reason.MISSING_HEADERS, "Missing webhook-id, webhook-timestamp or webhook-signature")); - } - - long seconds; - try - { - seconds = Long.parseLong(timestamp.trim()); - } - catch(NumberFormatException e) - { - throw (new WebhookVerificationException(WebhookVerificationException.Reason.BAD_TIMESTAMP, "webhook-timestamp is not a Unix timestamp")); - } - long now = clock.instant().getEpochSecond(); - if(Math.abs(now - seconds) > tolerance.toSeconds()) - { - throw (new WebhookVerificationException(WebhookVerificationException.Reason.TIMESTAMP_OUT_OF_TOLERANCE, "webhook-timestamp is outside the allowed window")); - } - - try - { - WebhookSigner.validateId(id); - } - catch(IllegalArgumentException e) - { - throw (new WebhookVerificationException(WebhookVerificationException.Reason.MISSING_HEADERS, e.getMessage())); - } - - for(String part : signature.trim().split(" +")) - { - int comma = part.indexOf(','); - if(comma < 0 || !WebhookSigner.VERSION_V1.equals(part.substring(0, comma))) - { - continue; - } - byte[] offered = part.substring(comma + 1).getBytes(StandardCharsets.US_ASCII); - for(WebhookSecret secret : secrets) - { - byte[] expected = WebhookSigner.sign(id, seconds, body, secret).getBytes(StandardCharsets.US_ASCII); - if(MessageDigest.isEqual(expected, offered)) - { - return; - } - } - } - throw (new WebhookVerificationException(WebhookVerificationException.Reason.NO_MATCHING_SIGNATURE, "No v1 signature matches a known secret")); - } - - - - /******************************************************************************* - ** Case-insensitive header lookup. - *******************************************************************************/ - private static String header(Map headers, String name) - { - if(headers == null) - { - return (null); - } - for(Map.Entry entry : headers.entrySet()) - { - if(entry.getKey() != null && entry.getKey().toLowerCase(Locale.ROOT).equals(name)) - { - return (entry.getValue()); - } - } - return (null); - } -} diff --git a/src/test/java/com/kingsrook/qbits/webhooks/delivery/RestartPersistenceTest.java b/src/test/java/com/kingsrook/qbits/webhooks/delivery/RestartPersistenceTest.java index e37b7f2..f0f45e9 100644 --- a/src/test/java/com/kingsrook/qbits/webhooks/delivery/RestartPersistenceTest.java +++ b/src/test/java/com/kingsrook/qbits/webhooks/delivery/RestartPersistenceTest.java @@ -27,15 +27,15 @@ import java.time.Duration; import java.util.List; import com.kingsrook.qbits.webhooks.H2Support; import com.kingsrook.qbits.webhooks.WebhooksRuntime; -import com.kingsrook.qbits.webhooks.WebhooksTestSupport; import com.kingsrook.qbits.webhooks.WebhooksTestSupport.Receiver; +import com.kingsrook.qbits.webhooks.WebhooksTestSupport; import com.kingsrook.qbits.webhooks.metadata.WebhooksTables; import com.kingsrook.qbits.webhooks.secrets.EndpointSecrets; -import com.kingsrook.qbits.webhooks.shared.signing.WebhookSecret; -import com.kingsrook.qbits.webhooks.shared.signing.WebhookVerifier; import com.kingsrook.qqq.backend.core.context.QContext; import com.kingsrook.qqq.backend.core.model.data.QRecord; import com.kingsrook.qqq.backend.core.model.metadata.QInstance; +import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret; +import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerifier; import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData; import org.junit.jupiter.api.Test; import static org.assertj.core.api.Assertions.assertThat; diff --git a/src/test/java/com/kingsrook/qbits/webhooks/delivery/SignedDeliveryTest.java b/src/test/java/com/kingsrook/qbits/webhooks/delivery/SignedDeliveryTest.java index dc8055a..9bbea51 100644 --- a/src/test/java/com/kingsrook/qbits/webhooks/delivery/SignedDeliveryTest.java +++ b/src/test/java/com/kingsrook/qbits/webhooks/delivery/SignedDeliveryTest.java @@ -23,16 +23,16 @@ package com.kingsrook.qbits.webhooks.delivery; import java.time.Duration; import java.util.List; -import com.kingsrook.qbits.webhooks.WebhooksTestSupport; import com.kingsrook.qbits.webhooks.WebhooksTestSupport.Receiver; +import com.kingsrook.qbits.webhooks.WebhooksTestSupport; import com.kingsrook.qbits.webhooks.metadata.WebhooksTables; import com.kingsrook.qbits.webhooks.processes.WebhooksProcesses; import com.kingsrook.qbits.webhooks.secrets.EndpointSecrets; -import com.kingsrook.qbits.webhooks.shared.signing.WebhookSecret; -import com.kingsrook.qbits.webhooks.shared.signing.WebhookVerificationException; -import com.kingsrook.qbits.webhooks.shared.signing.WebhookVerifier; import com.kingsrook.qqq.backend.core.model.data.QRecord; import com.kingsrook.qqq.backend.core.utils.JsonUtils; +import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret; +import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerificationException; +import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerifier; import org.json.JSONObject; import org.junit.jupiter.api.AfterEach; import org.junit.jupiter.api.BeforeEach; diff --git a/src/test/java/com/kingsrook/qbits/webhooks/shared/signing/WebhookSigningTest.java b/src/test/java/com/kingsrook/qbits/webhooks/signing/WebhookSigningTest.java similarity index 72% rename from src/test/java/com/kingsrook/qbits/webhooks/shared/signing/WebhookSigningTest.java rename to src/test/java/com/kingsrook/qbits/webhooks/signing/WebhookSigningTest.java index 49ddc8e..a312714 100644 --- a/src/test/java/com/kingsrook/qbits/webhooks/shared/signing/WebhookSigningTest.java +++ b/src/test/java/com/kingsrook/qbits/webhooks/signing/WebhookSigningTest.java @@ -18,17 +18,26 @@ * limitations under the License. */ -package com.kingsrook.qbits.webhooks.shared.signing; +package com.kingsrook.qbits.webhooks.signing; +import javax.crypto.Mac; +import javax.crypto.spec.SecretKeySpec; import java.nio.charset.StandardCharsets; +import java.security.MessageDigest; import java.time.Clock; import java.time.Duration; import java.time.Instant; import java.time.ZoneOffset; +import java.util.Arrays; +import java.util.Base64; import java.util.HashMap; import java.util.List; import java.util.Map; +import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret; +import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSigner; +import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerificationException; +import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerifier; import org.junit.jupiter.api.Test; import static org.assertj.core.api.Assertions.assertThat; import static org.assertj.core.api.Assertions.assertThatThrownBy; @@ -36,7 +45,7 @@ import static org.junit.jupiter.api.Assertions.assertDoesNotThrow; /******************************************************************************* - ** Unit test for WebhookSecret, WebhookSigner and WebhookVerifier. + ** Qbit wire contract and interoperability with the released core primitives. *******************************************************************************/ class WebhookSigningTest { @@ -45,6 +54,85 @@ class WebhookSigningTest + /******************************************************************************* + ** Independent local HMAC and core interoperate without re-encoding the body. + ** The reference uses literal wire names and prefixes to detect contract drift. + *******************************************************************************/ + @Test + void testLocalAndCoreInteropWithRotation() throws Exception + { + List texts = List.of("a-raw-shared-secret-of-enough-length", "whsec_MfKQ9r8GKYqrTwjUPD8ILPZIo2LaLaSw"); + List secrets = texts.stream().map(WebhookSecret::parse).toList(); + for(byte[] body : List.of(BODY, new byte[0], new byte[] { 0, (byte) 0xff, (byte) 0xc3, (byte) 0xa9, 10 })) + { + String first = localSignature("evt_interop", "1760000000", body, texts.get(0)); + String second = localSignature("evt_interop", "1760000000", body, texts.get(1)); + Map local = Map.of("webhook-id", "evt_interop", "webhook-timestamp", "1760000000", + "webhook-signature", "v1," + first + " v1," + second); + Map core = WebhookSigner.headers("evt_interop", NOW, body, secrets); + assertThat(core).isEqualTo(local); + for(int i = 0; i < secrets.size(); i++) + { + verifier(List.of(secrets.get(i))).verify(local, body); + assertThat(localVerify(core, body, texts.get(i))).isTrue(); + } + assertThat(localVerify(core, body, "an-unrelated-secret-of-enough-length")).isFalse(); + verifier(secrets).verify(WebhookSigner.headers("evt_interop", NOW, body, List.of(secrets.get(1))), body); + } + } + + + + /******************************************************************************* + ** The five-minute window includes both boundaries, but not the next second. + *******************************************************************************/ + @Test + void testExactTimestampTolerance() + { + assertThat(WebhookVerifier.DEFAULT_TOLERANCE).isEqualTo(Duration.ofSeconds(300)); + WebhookSecret secret = WebhookSecret.generate(); + for(long offset : new long[] { -300, 300 }) + { + assertDoesNotThrow(() -> verifier(List.of(secret)).verify(WebhookSigner.headers("evt_edge", NOW.plusSeconds(offset), BODY, List.of(secret)), BODY)); + } + for(long offset : new long[] { -301, 301 }) + { + assertThatThrownBy(() -> verifier(List.of(secret)).verify(WebhookSigner.headers("evt_edge", NOW.plusSeconds(offset), BODY, List.of(secret)), BODY)) + .isInstanceOf(WebhookVerificationException.class) + .extracting(e -> ((WebhookVerificationException) e).getReason()) + .isEqualTo(WebhookVerificationException.Reason.TIMESTAMP_OUT_OF_TOLERANCE); + } + } + + + + /******************************************************************************* + ** Test-only reference HMAC, independent of core signing and secret parsing. + *******************************************************************************/ + private static String localSignature(String id, String timestamp, byte[] body, String text) throws Exception + { + byte[] key = text.startsWith("whsec_") ? Base64.getDecoder().decode(text.substring(6)) : text.getBytes(StandardCharsets.UTF_8); + Mac mac = Mac.getInstance("HmacSHA256"); + mac.init(new SecretKeySpec(key, "HmacSHA256")); + mac.update((id + "." + timestamp + ".").getBytes(StandardCharsets.UTF_8)); + return (Base64.getEncoder().encodeToString(mac.doFinal(body))); + } + + + + /******************************************************************************* + ** Test-only receiver checks rotation signatures independently of core. + *******************************************************************************/ + private static boolean localVerify(Map headers, byte[] body, String secret) throws Exception + { + byte[] expected = localSignature(headers.get("webhook-id"), headers.get("webhook-timestamp"), body, secret).getBytes(StandardCharsets.US_ASCII); + return (Arrays.stream(headers.get("webhook-signature").split(" ")) + .filter(part -> part.startsWith("v1,")) + .anyMatch(part -> MessageDigest.isEqual(expected, part.substring(3).getBytes(StandardCharsets.US_ASCII)))); + } + + + /******************************************************************************* ** The Standard Webhooks reference vector (standard-webhooks/standard-webhooks, ** libraries/go/webhook_test.go, TestWebhookSign).