refactor(signing): core WebhookSigner/WebhookVerifier (qqq#1160) #13
No files matched your search
@@ -31,7 +31,7 @@ Design: `docs/DESIGN.md` (from QRun-IO/qqq#1105). Inbound webhooks are phase 2 (
|
||||
- `DeliveryEngine` (claim, send, outcome), `EndpointHealth`
|
||||
- `WebhooksAlerts` (reflective Alerts SPI)
|
||||
- `secrets` (AES-GCM, env references, rotation), `processes`, `app` (home, secret widget), `permissions`, `esb`, `support/SystemWrites`.
|
||||
- `shared`: INTERNAL COPY of the core signer and safe HTTP client (QRun-IO/qqq#1109). Replace with `core.utils.webhooks` / `core.utils.http` at QQQ 4.1.0-RC.5 (DESIGN section 11). Keep it identical to core until then.
|
||||
- Signing uses `core.utils.webhooks` directly at the QQQ 4.1.0-RC.5 baseline (QRun-IO/qqq#1160). `shared.http` remains an internal copy of the safe HTTP client pending a separate migration to `core.utils.http` (DESIGN section 11).
|
||||
|
||||
## Invariants
|
||||
|
||||
|
||||
@@ -5,6 +5,10 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Changed
|
||||
- Require QQQ 4.1.0-RC.5 and use its webhook signer, verifier, secret and verification exception (QRun-IO/qqq#1160, F11). Removed the internal signing package; Java consumers must import `com.kingsrook.qqq.backend.core.utils.webhooks` instead.
|
||||
- Signing wire format, timestamp tolerance, rotation, endpoint management, encrypted secret storage and delivery behavior are unchanged.
|
||||
|
||||
## [1.0.0] - not yet released
|
||||
|
||||
A rebuild of the qbit on QQQ 4.1 (design: QRun-IO/qqq#1105, docs/DESIGN.md). Breaking: see docs/migration-0.x-to-1.0.md.
|
||||
|
||||
@@ -31,7 +31,7 @@ Design: `docs/DESIGN.md` (from QRun-IO/qqq#1105). Inbound webhooks are phase 2 (
|
||||
- `DeliveryEngine` (claim, send, outcome), `EndpointHealth`
|
||||
- `WebhooksAlerts` (reflective Alerts SPI)
|
||||
- `secrets` (AES-GCM, env references, rotation), `processes`, `app` (home, secret widget), `permissions`, `esb`, `support/SystemWrites`.
|
||||
- `shared`: INTERNAL COPY of the core signer and safe HTTP client (QRun-IO/qqq#1109). Replace with `core.utils.webhooks` / `core.utils.http` at QQQ 4.1.0-RC.5 (DESIGN section 11). Keep it identical to core until then.
|
||||
- Signing uses `core.utils.webhooks` directly at the QQQ 4.1.0-RC.5 baseline (QRun-IO/qqq#1160). `shared.http` remains an internal copy of the safe HTTP client pending a separate migration to `core.utils.http` (DESIGN section 11).
|
||||
|
||||
## Invariants
|
||||
|
||||
|
||||
@@ -2,12 +2,12 @@
|
||||
|
||||
Signed, SSRF-safe outbound webhooks for QQQ applications. Admins add **endpoints** and **subscribe** them to event types: table inserts, updates and deletes (in-process or from the ESB), process completions and failures, and custom domain events. The qbit queues one **delivery** per event per endpoint in the same transaction as the change. It then signs each delivery in the [Standard Webhooks](https://www.standardwebhooks.com/) format and sends it with retries, backoff, dead letters and endpoint health. Every row is a normal QQQ table, and the app has a home dashboard.
|
||||
|
||||
Design: [QRun-IO/qqq#1105](https://github.com/QRun-IO/qqq/pull/1105) (`docs/specs/2026-10-12-qbit-webhooks-assessment.md`), and [docs/DESIGN.md](docs/DESIGN.md) for what 1.0 implements. The shared signer and safe HTTP client are in QQQ core from 4.1.0-RC.5 ([QRun-IO/qqq#1109](https://github.com/QRun-IO/qqq/pull/1109)).
|
||||
Design: [QRun-IO/qqq#1105](https://github.com/QRun-IO/qqq/pull/1105) (`docs/specs/2026-10-12-qbit-webhooks-assessment.md`), and [docs/DESIGN.md](docs/DESIGN.md) for what 1.0 implements. Signing and secret generation use QQQ core directly; the local safe HTTP client remains pending a separate migration. Both primitives are available in QQQ core from 4.1.0-RC.5 ([QRun-IO/qqq#1109](https://github.com/QRun-IO/qqq/pull/1109)).
|
||||
|
||||
## Requirements
|
||||
|
||||
- Java 21
|
||||
- QQQ 4.1.0-RC.4 or later (`qqq-backend-core`). Two modules are optional:
|
||||
- QQQ 4.1.0-RC.5 or later (`qqq-backend-core`). Two modules are optional:
|
||||
- `qqq-middleware-api`, for payloads rendered through a QQQ API version.
|
||||
- `qqq-esb`, for ESB topics as an event source.
|
||||
- A SQL backend for the five tables (DDL in [docs/ddl/webhooks-postgres.sql](docs/ddl/webhooks-postgres.sql)), and a QQQ scheduler for delivery.
|
||||
|
||||
+12
-6
@@ -6,7 +6,7 @@
|
||||
| # | Decision |
|
||||
|---|---|
|
||||
| D1 | Enhance the existing qbit into 1.0; breaking changes allowed |
|
||||
| D2 | The signer and SSRF-safe client live in `qqq-backend-core` (QRun-IO/qqq#1109). This qbit carries an identical internal copy in `com.kingsrook.qbits.webhooks.shared` until QQQ 4.1.0-RC.5 is published (see section 11) |
|
||||
| D2 | The signer and SSRF-safe client live in `qqq-backend-core` (QRun-IO/qqq#1109). Signing uses core directly at the QQQ 4.1.0-RC.5 baseline; the internal HTTP copy remains pending a separate migration (see section 11) |
|
||||
| D3 | Tables renamed to `webhooks_*` |
|
||||
| D4 | Internal and private targets blocked by default; admin-only allowlist per endpoint |
|
||||
| D5 | Inbound is phase 2, in this qbit |
|
||||
@@ -265,10 +265,16 @@ Goal: verify and record signed callbacks from third parties (Authentik, GitHub,
|
||||
|
||||
## 11. Moving to the core primitives (QQQ 4.1.0-RC.5)
|
||||
|
||||
QRun-IO/qqq#1109 adds the same classes to `qqq-backend-core` (`com.kingsrook.qqq.backend.core.utils.webhooks` and `...utils.http`). When RC.5 is on Maven Central:
|
||||
QRun-IO/qqq#1160 (F11) migrates signing to the released `qqq-bom-pom` 4.1.0-RC.5 baseline. Delivery signing, secret generation/rotation and receiver verification use `com.kingsrook.qqq.backend.core.utils.webhooks` directly. The local `shared.signing` package is removed; endpoints, encrypted secret storage and delivery remain in this qbit. Inbound routes are still phase 2 (design only).
|
||||
|
||||
1. Bump `qqq.snapshot.version` to `4.1.0-RC.5`.
|
||||
2. Delete `src/main/java/com/kingsrook/qbits/webhooks/shared/` and its tests.
|
||||
3. Change the imports (`shared.signing` to `core.utils.webhooks`, `shared.http` to `core.utils.http`).
|
||||
Comparison against core sources at tag `v4.1.0-RC.5` found no executable differences in `WebhookSigner`, `WebhookVerifier`, `WebhookSecret` or `WebhookVerificationException`; only package names, copyright years and migration comments differed. No compatibility adapter is needed:
|
||||
|
||||
The behaviour is identical, and the classes were kept the same on purpose.
|
||||
- Headers remain `webhook-id`, `webhook-timestamp`, `webhook-signature` (case-insensitive when verifying).
|
||||
- HMAC-SHA256 signs UTF-8 `id.timestamp.` followed by unchanged body bytes; signatures are padded Base64 with `v1,` and a single space between rotation signatures, in secret-list order.
|
||||
- Secrets remain `whsec_` plus padded Base64 key bytes; unprefixed secrets use trimmed UTF-8 bytes. Generation uses 32 bytes, with a 16-byte minimum for imported keys.
|
||||
- Default timestamp tolerance remains five minutes in either direction, including the boundary.
|
||||
- Verification accepts any matching `v1` entry with any known secret and ignores unknown signature versions.
|
||||
|
||||
Signing contract tests retain the reference vector and test independent local HMAC signing against core verification and core signing against local verification, including two-secret rotation and exact tolerance boundaries. Delivery integration tests verify actual requests with core.
|
||||
|
||||
The `shared.http` copy is outside F11 and remains for a separate migration to `com.kingsrook.qqq.backend.core.utils.http`.
|
||||
@@ -28,6 +28,9 @@ webhook-attempt: 1
|
||||
## Java (QQQ core 4.1.0-RC.5 or later)
|
||||
|
||||
```java
|
||||
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret;
|
||||
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerifier;
|
||||
|
||||
WebhookVerifier verifier = new WebhookVerifier(List.of(WebhookSecret.parse(System.getenv("WEBHOOK_SECRET"))));
|
||||
verifier.verify(headersAsMap, rawBodyBytes); // throws WebhookVerificationException with a safe reason
|
||||
```
|
||||
|
||||
@@ -215,7 +215,7 @@
|
||||
<activeByDefault>true</activeByDefault>
|
||||
</activation>
|
||||
<properties>
|
||||
<qqq.snapshot.version>4.1.0-RC.4</qqq.snapshot.version>
|
||||
<qqq.snapshot.version>4.1.0-RC.5</qqq.snapshot.version>
|
||||
</properties>
|
||||
<dependencyManagement>
|
||||
<dependencies>
|
||||
|
||||
@@ -42,8 +42,6 @@ import com.kingsrook.qbits.webhooks.secrets.EndpointSecrets;
|
||||
import com.kingsrook.qbits.webhooks.shared.http.SafeHttpClient;
|
||||
import com.kingsrook.qbits.webhooks.shared.http.SafeHttpException;
|
||||
import com.kingsrook.qbits.webhooks.shared.http.SafeHttpResponse;
|
||||
import com.kingsrook.qbits.webhooks.shared.signing.WebhookSecret;
|
||||
import com.kingsrook.qbits.webhooks.shared.signing.WebhookSigner;
|
||||
import com.kingsrook.qbits.webhooks.support.SystemWrites;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.logging.QLogger;
|
||||
@@ -55,6 +53,8 @@ import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
|
||||
import com.kingsrook.qqq.backend.core.utils.StringUtils;
|
||||
import com.kingsrook.qqq.backend.core.utils.ValueUtils;
|
||||
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret;
|
||||
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSigner;
|
||||
import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
|
||||
|
||||
|
||||
|
||||
@@ -38,7 +38,6 @@ import com.kingsrook.qbits.webhooks.registry.WebhooksRegistry;
|
||||
import com.kingsrook.qbits.webhooks.secrets.EnvironmentSecrets;
|
||||
import com.kingsrook.qbits.webhooks.shared.http.AddressRules;
|
||||
import com.kingsrook.qbits.webhooks.shared.http.SafeHttpException;
|
||||
import com.kingsrook.qbits.webhooks.shared.signing.WebhookSecret;
|
||||
import com.kingsrook.qbits.webhooks.support.SystemWrites;
|
||||
import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
|
||||
import com.kingsrook.qqq.backend.core.actions.permissions.PermissionsHelper;
|
||||
@@ -50,6 +49,7 @@ import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.statusmessages.BadInputStatusMessage;
|
||||
import com.kingsrook.qqq.backend.core.utils.StringUtils;
|
||||
import com.kingsrook.qqq.backend.core.utils.ValueUtils;
|
||||
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
|
||||
@@ -30,7 +30,6 @@ import com.kingsrook.qbits.webhooks.WebhooksRuntime;
|
||||
import com.kingsrook.qbits.webhooks.metadata.WebhooksEnums.SecretSource;
|
||||
import com.kingsrook.qbits.webhooks.metadata.WebhooksEnums.SecretStatus;
|
||||
import com.kingsrook.qbits.webhooks.metadata.WebhooksTables;
|
||||
import com.kingsrook.qbits.webhooks.shared.signing.WebhookSecret;
|
||||
import com.kingsrook.qbits.webhooks.support.SystemWrites;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.logging.QLogger;
|
||||
@@ -41,6 +40,7 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
|
||||
import com.kingsrook.qqq.backend.core.utils.ValueUtils;
|
||||
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret;
|
||||
import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
|
||||
|
||||
|
||||
|
||||
@@ -1,183 +0,0 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.webhooks.shared.signing;
|
||||
|
||||
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.security.MessageDigest;
|
||||
import java.security.NoSuchAlgorithmException;
|
||||
import java.security.SecureRandom;
|
||||
import java.util.Arrays;
|
||||
import java.util.Base64;
|
||||
import java.util.HexFormat;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A webhook signing secret (Standard Webhooks format).
|
||||
**
|
||||
** <p>The text form is {@code whsec_} followed by the base64 of the key bytes.
|
||||
** The HMAC key is the decoded bytes. A secret without the prefix is used as
|
||||
** raw UTF-8 bytes, so a secret shared with an older receiver still works.</p>
|
||||
**
|
||||
** <p>{@link #toString()} never reveals the key: it prints the fingerprint.</p>
|
||||
**
|
||||
** <p>INTERNAL COPY: replaced by {@code com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret}
|
||||
** when the qbit moves to QQQ 4.1.0-RC.5 (QRun-IO/qqq, D2 of the webhooks assessment).
|
||||
** Keep the two identical.</p>
|
||||
*******************************************************************************/
|
||||
public final class WebhookSecret
|
||||
{
|
||||
public static final String PREFIX = "whsec_";
|
||||
|
||||
/** Key length for generated secrets, in bytes (256 bits). */
|
||||
public static final int GENERATED_KEY_BYTES = 32;
|
||||
|
||||
/** Shortest key accepted, in bytes (128 bits). */
|
||||
public static final int MINIMUM_KEY_BYTES = 16;
|
||||
|
||||
private static final SecureRandom RANDOM = new SecureRandom();
|
||||
|
||||
private final byte[] key;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Constructor (keeps a copy of the key).
|
||||
*******************************************************************************/
|
||||
private WebhookSecret(byte[] key)
|
||||
{
|
||||
if(key == null || key.length < MINIMUM_KEY_BYTES)
|
||||
{
|
||||
throw (new IllegalArgumentException("A webhook signing secret must be at least " + MINIMUM_KEY_BYTES + " bytes"));
|
||||
}
|
||||
this.key = Arrays.copyOf(key, key.length);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A new random secret.
|
||||
*******************************************************************************/
|
||||
public static WebhookSecret generate()
|
||||
{
|
||||
byte[] bytes = new byte[GENERATED_KEY_BYTES];
|
||||
RANDOM.nextBytes(bytes);
|
||||
return (new WebhookSecret(bytes));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Parse the text form ({@code whsec_<base64>}, or raw text).
|
||||
*******************************************************************************/
|
||||
public static WebhookSecret parse(String text)
|
||||
{
|
||||
if(text == null || text.isBlank())
|
||||
{
|
||||
throw (new IllegalArgumentException("A webhook signing secret is required"));
|
||||
}
|
||||
String trimmed = text.trim();
|
||||
if(trimmed.startsWith(PREFIX))
|
||||
{
|
||||
try
|
||||
{
|
||||
return (new WebhookSecret(Base64.getDecoder().decode(trimmed.substring(PREFIX.length()))));
|
||||
}
|
||||
catch(IllegalArgumentException e)
|
||||
{
|
||||
throw (new IllegalArgumentException("A " + PREFIX + " secret must be base64 after the prefix", e));
|
||||
}
|
||||
}
|
||||
return (new WebhookSecret(trimmed.getBytes(StandardCharsets.UTF_8)));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The text form to give a receiver ({@code whsec_<base64>}).
|
||||
*******************************************************************************/
|
||||
public String reveal()
|
||||
{
|
||||
return (PREFIX + Base64.getEncoder().encodeToString(key));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The HMAC key (a copy).
|
||||
*******************************************************************************/
|
||||
public byte[] keyBytes()
|
||||
{
|
||||
return (Arrays.copyOf(key, key.length));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A short, non-reversible id for the secret (first 12 hex chars of its
|
||||
** SHA-256), safe to store and show, so people can tell secrets apart.
|
||||
*******************************************************************************/
|
||||
public String fingerprint()
|
||||
{
|
||||
try
|
||||
{
|
||||
byte[] digest = MessageDigest.getInstance("SHA-256").digest(key);
|
||||
return (HexFormat.of().formatHex(digest).substring(0, 12));
|
||||
}
|
||||
catch(NoSuchAlgorithmException e)
|
||||
{
|
||||
throw (new IllegalStateException("SHA-256 is unavailable", e));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Never prints the key.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String toString()
|
||||
{
|
||||
return ("WebhookSecret[" + fingerprint() + "]");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Equal when the keys are equal (constant time).
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public boolean equals(Object o)
|
||||
{
|
||||
return (o instanceof WebhookSecret other && MessageDigest.isEqual(key, other.key));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Hash of the fingerprint.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public int hashCode()
|
||||
{
|
||||
return (fingerprint().hashCode());
|
||||
}
|
||||
}
|
||||
@@ -1,147 +0,0 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.webhooks.shared.signing;
|
||||
|
||||
|
||||
import javax.crypto.Mac;
|
||||
import javax.crypto.spec.SecretKeySpec;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.security.GeneralSecurityException;
|
||||
import java.time.Instant;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Base64;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Signs webhook requests in the Standard Webhooks format
|
||||
** (https://www.standardwebhooks.com/).
|
||||
**
|
||||
** <p>The signed content is {@code <id>.<timestamp>.<body>}: the message id,
|
||||
** a dot, the Unix timestamp in seconds, a dot, then the exact body bytes. The
|
||||
** signature is HMAC-SHA256 with the secret's key, base64 encoded, and sent as
|
||||
** {@code v1,<base64>}. During a secret rotation every active secret signs, and
|
||||
** the signatures are separated by spaces, so a receiver holding either the old
|
||||
** or the new secret accepts the request.</p>
|
||||
**
|
||||
** <p>Headers: {@code webhook-id}, {@code webhook-timestamp},
|
||||
** {@code webhook-signature}.</p>
|
||||
**
|
||||
** <p>INTERNAL COPY: replaced by {@code com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSigner}
|
||||
** when the qbit moves to QQQ 4.1.0-RC.5. Keep the two identical.</p>
|
||||
*******************************************************************************/
|
||||
public final class WebhookSigner
|
||||
{
|
||||
public static final String HEADER_ID = "webhook-id";
|
||||
public static final String HEADER_TIMESTAMP = "webhook-timestamp";
|
||||
public static final String HEADER_SIGNATURE = "webhook-signature";
|
||||
|
||||
/** The only signature scheme today: HMAC-SHA256. */
|
||||
public static final String VERSION_V1 = "v1";
|
||||
|
||||
private static final String HMAC = "HmacSHA256";
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private WebhookSigner()
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The three signature headers for one request.
|
||||
**
|
||||
** @param messageId the event id (stable across retries: it is the idempotency key)
|
||||
** @param timestamp when this attempt is sent
|
||||
** @param body the exact bytes that are sent
|
||||
** @param secrets the active secrets (one, or two during a rotation)
|
||||
*******************************************************************************/
|
||||
public static Map<String, String> headers(String messageId, Instant timestamp, byte[] body, List<WebhookSecret> secrets)
|
||||
{
|
||||
long seconds = timestamp.getEpochSecond();
|
||||
Map<String, String> headers = new LinkedHashMap<>();
|
||||
headers.put(HEADER_ID, messageId);
|
||||
headers.put(HEADER_TIMESTAMP, Long.toString(seconds));
|
||||
headers.put(HEADER_SIGNATURE, signatureHeader(messageId, seconds, body, secrets));
|
||||
return (headers);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The {@code webhook-signature} value: {@code v1,<sig>} per secret, space
|
||||
** separated.
|
||||
*******************************************************************************/
|
||||
public static String signatureHeader(String messageId, long timestampSeconds, byte[] body, List<WebhookSecret> secrets)
|
||||
{
|
||||
if(secrets == null || secrets.isEmpty())
|
||||
{
|
||||
throw (new IllegalArgumentException("At least one signing secret is required"));
|
||||
}
|
||||
List<String> parts = new ArrayList<>();
|
||||
for(WebhookSecret secret : secrets)
|
||||
{
|
||||
parts.add(VERSION_V1 + "," + sign(messageId, timestampSeconds, body, secret));
|
||||
}
|
||||
return (String.join(" ", parts));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** One base64 HMAC-SHA256 signature over {@code <id>.<timestamp>.<body>}.
|
||||
*******************************************************************************/
|
||||
public static String sign(String messageId, long timestampSeconds, byte[] body, WebhookSecret secret)
|
||||
{
|
||||
validateId(messageId);
|
||||
try
|
||||
{
|
||||
Mac mac = Mac.getInstance(HMAC);
|
||||
mac.init(new SecretKeySpec(secret.keyBytes(), HMAC));
|
||||
mac.update((messageId + "." + timestampSeconds + ".").getBytes(StandardCharsets.UTF_8));
|
||||
mac.update(body == null ? new byte[0] : body);
|
||||
return (Base64.getEncoder().encodeToString(mac.doFinal()));
|
||||
}
|
||||
catch(GeneralSecurityException e)
|
||||
{
|
||||
throw (new IllegalStateException("HMAC-SHA256 is unavailable", e));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A message id may not contain the separator or whitespace.
|
||||
*******************************************************************************/
|
||||
static void validateId(String messageId)
|
||||
{
|
||||
if(messageId == null || messageId.isEmpty() || messageId.contains(".") || messageId.chars().anyMatch(Character::isWhitespace))
|
||||
{
|
||||
throw (new IllegalArgumentException("A webhook message id is required and may not contain '.' or whitespace"));
|
||||
}
|
||||
}
|
||||
}
|
||||
-70
@@ -1,70 +0,0 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.webhooks.shared.signing;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A webhook request failed verification. The message never contains a
|
||||
** secret or a signature, so it is safe to log and to return to the caller.
|
||||
**
|
||||
** <p>INTERNAL COPY: replaced by the class of the same name in
|
||||
** {@code com.kingsrook.qqq.backend.core.utils.webhooks} at QQQ 4.1.0-RC.5.</p>
|
||||
*******************************************************************************/
|
||||
public class WebhookVerificationException extends Exception
|
||||
{
|
||||
private static final long serialVersionUID = 1L;
|
||||
|
||||
/*******************************************************************************
|
||||
** Why verification failed.
|
||||
*******************************************************************************/
|
||||
public enum Reason
|
||||
{
|
||||
MISSING_HEADERS,
|
||||
BAD_TIMESTAMP,
|
||||
TIMESTAMP_OUT_OF_TOLERANCE,
|
||||
NO_MATCHING_SIGNATURE
|
||||
}
|
||||
|
||||
|
||||
|
||||
private final Reason reason;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Constructor.
|
||||
*******************************************************************************/
|
||||
public WebhookVerificationException(Reason reason, String message)
|
||||
{
|
||||
super(message);
|
||||
this.reason = reason;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for reason.
|
||||
*******************************************************************************/
|
||||
public Reason getReason()
|
||||
{
|
||||
return (reason);
|
||||
}
|
||||
}
|
||||
@@ -1,165 +0,0 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.webhooks.shared.signing;
|
||||
|
||||
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.security.MessageDigest;
|
||||
import java.time.Clock;
|
||||
import java.time.Duration;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Locale;
|
||||
import java.util.Map;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Verifies a signed webhook request (Standard Webhooks format, see
|
||||
** {@link WebhookSigner}).
|
||||
**
|
||||
** <p>Checks, in order: the three headers are present, the timestamp is within
|
||||
** the tolerance of now (default five minutes, either direction, which stops
|
||||
** replays of old requests), and at least one {@code v1} signature matches one
|
||||
** of the receiver's secrets. Comparison is constant time. Unknown signature
|
||||
** versions are ignored, so a future scheme can be added alongside v1.</p>
|
||||
**
|
||||
** <p>Receivers should also remember recent {@code webhook-id}s and ignore a
|
||||
** repeat: the same id is sent again on every retry of the same event.</p>
|
||||
**
|
||||
** <p>INTERNAL COPY: replaced by {@code com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerifier}
|
||||
** when the qbit moves to QQQ 4.1.0-RC.5. Keep the two identical.</p>
|
||||
*******************************************************************************/
|
||||
public final class WebhookVerifier
|
||||
{
|
||||
public static final Duration DEFAULT_TOLERANCE = Duration.ofMinutes(5);
|
||||
|
||||
private final List<WebhookSecret> secrets;
|
||||
private final Duration tolerance;
|
||||
private final Clock clock;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Verifier with the default tolerance and the system clock.
|
||||
*******************************************************************************/
|
||||
public WebhookVerifier(List<WebhookSecret> secrets)
|
||||
{
|
||||
this(secrets, DEFAULT_TOLERANCE, Clock.systemUTC());
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Verifier.
|
||||
*******************************************************************************/
|
||||
public WebhookVerifier(List<WebhookSecret> secrets, Duration tolerance, Clock clock)
|
||||
{
|
||||
if(secrets == null || secrets.isEmpty())
|
||||
{
|
||||
throw (new IllegalArgumentException("At least one secret is required to verify webhooks"));
|
||||
}
|
||||
this.secrets = new ArrayList<>(secrets);
|
||||
this.tolerance = tolerance;
|
||||
this.clock = clock;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Verify a request. Header names are matched case-insensitively.
|
||||
**
|
||||
** @throws WebhookVerificationException with a reason safe to log
|
||||
*******************************************************************************/
|
||||
public void verify(Map<String, String> headers, byte[] body) throws WebhookVerificationException
|
||||
{
|
||||
String id = header(headers, WebhookSigner.HEADER_ID);
|
||||
String timestamp = header(headers, WebhookSigner.HEADER_TIMESTAMP);
|
||||
String signature = header(headers, WebhookSigner.HEADER_SIGNATURE);
|
||||
if(id == null || timestamp == null || signature == null)
|
||||
{
|
||||
throw (new WebhookVerificationException(WebhookVerificationException.Reason.MISSING_HEADERS, "Missing webhook-id, webhook-timestamp or webhook-signature"));
|
||||
}
|
||||
|
||||
long seconds;
|
||||
try
|
||||
{
|
||||
seconds = Long.parseLong(timestamp.trim());
|
||||
}
|
||||
catch(NumberFormatException e)
|
||||
{
|
||||
throw (new WebhookVerificationException(WebhookVerificationException.Reason.BAD_TIMESTAMP, "webhook-timestamp is not a Unix timestamp"));
|
||||
}
|
||||
long now = clock.instant().getEpochSecond();
|
||||
if(Math.abs(now - seconds) > tolerance.toSeconds())
|
||||
{
|
||||
throw (new WebhookVerificationException(WebhookVerificationException.Reason.TIMESTAMP_OUT_OF_TOLERANCE, "webhook-timestamp is outside the allowed window"));
|
||||
}
|
||||
|
||||
try
|
||||
{
|
||||
WebhookSigner.validateId(id);
|
||||
}
|
||||
catch(IllegalArgumentException e)
|
||||
{
|
||||
throw (new WebhookVerificationException(WebhookVerificationException.Reason.MISSING_HEADERS, e.getMessage()));
|
||||
}
|
||||
|
||||
for(String part : signature.trim().split(" +"))
|
||||
{
|
||||
int comma = part.indexOf(',');
|
||||
if(comma < 0 || !WebhookSigner.VERSION_V1.equals(part.substring(0, comma)))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
byte[] offered = part.substring(comma + 1).getBytes(StandardCharsets.US_ASCII);
|
||||
for(WebhookSecret secret : secrets)
|
||||
{
|
||||
byte[] expected = WebhookSigner.sign(id, seconds, body, secret).getBytes(StandardCharsets.US_ASCII);
|
||||
if(MessageDigest.isEqual(expected, offered))
|
||||
{
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
throw (new WebhookVerificationException(WebhookVerificationException.Reason.NO_MATCHING_SIGNATURE, "No v1 signature matches a known secret"));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Case-insensitive header lookup.
|
||||
*******************************************************************************/
|
||||
private static String header(Map<String, String> headers, String name)
|
||||
{
|
||||
if(headers == null)
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
for(Map.Entry<String, String> entry : headers.entrySet())
|
||||
{
|
||||
if(entry.getKey() != null && entry.getKey().toLowerCase(Locale.ROOT).equals(name))
|
||||
{
|
||||
return (entry.getValue());
|
||||
}
|
||||
}
|
||||
return (null);
|
||||
}
|
||||
}
|
||||
@@ -27,15 +27,15 @@ import java.time.Duration;
|
||||
import java.util.List;
|
||||
import com.kingsrook.qbits.webhooks.H2Support;
|
||||
import com.kingsrook.qbits.webhooks.WebhooksRuntime;
|
||||
import com.kingsrook.qbits.webhooks.WebhooksTestSupport;
|
||||
import com.kingsrook.qbits.webhooks.WebhooksTestSupport.Receiver;
|
||||
import com.kingsrook.qbits.webhooks.WebhooksTestSupport;
|
||||
import com.kingsrook.qbits.webhooks.metadata.WebhooksTables;
|
||||
import com.kingsrook.qbits.webhooks.secrets.EndpointSecrets;
|
||||
import com.kingsrook.qbits.webhooks.shared.signing.WebhookSecret;
|
||||
import com.kingsrook.qbits.webhooks.shared.signing.WebhookVerifier;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
|
||||
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret;
|
||||
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerifier;
|
||||
import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
@@ -23,16 +23,16 @@ package com.kingsrook.qbits.webhooks.delivery;
|
||||
|
||||
import java.time.Duration;
|
||||
import java.util.List;
|
||||
import com.kingsrook.qbits.webhooks.WebhooksTestSupport;
|
||||
import com.kingsrook.qbits.webhooks.WebhooksTestSupport.Receiver;
|
||||
import com.kingsrook.qbits.webhooks.WebhooksTestSupport;
|
||||
import com.kingsrook.qbits.webhooks.metadata.WebhooksTables;
|
||||
import com.kingsrook.qbits.webhooks.processes.WebhooksProcesses;
|
||||
import com.kingsrook.qbits.webhooks.secrets.EndpointSecrets;
|
||||
import com.kingsrook.qbits.webhooks.shared.signing.WebhookSecret;
|
||||
import com.kingsrook.qbits.webhooks.shared.signing.WebhookVerificationException;
|
||||
import com.kingsrook.qbits.webhooks.shared.signing.WebhookVerifier;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.utils.JsonUtils;
|
||||
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret;
|
||||
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerificationException;
|
||||
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerifier;
|
||||
import org.json.JSONObject;
|
||||
import org.junit.jupiter.api.AfterEach;
|
||||
import org.junit.jupiter.api.BeforeEach;
|
||||
|
||||
+90
-2
@@ -18,17 +18,26 @@
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.webhooks.shared.signing;
|
||||
package com.kingsrook.qbits.webhooks.signing;
|
||||
|
||||
|
||||
import javax.crypto.Mac;
|
||||
import javax.crypto.spec.SecretKeySpec;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.security.MessageDigest;
|
||||
import java.time.Clock;
|
||||
import java.time.Duration;
|
||||
import java.time.Instant;
|
||||
import java.time.ZoneOffset;
|
||||
import java.util.Arrays;
|
||||
import java.util.Base64;
|
||||
import java.util.HashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret;
|
||||
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSigner;
|
||||
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerificationException;
|
||||
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerifier;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
@@ -36,7 +45,7 @@ import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Unit test for WebhookSecret, WebhookSigner and WebhookVerifier.
|
||||
** Qbit wire contract and interoperability with the released core primitives.
|
||||
*******************************************************************************/
|
||||
class WebhookSigningTest
|
||||
{
|
||||
@@ -45,6 +54,85 @@ class WebhookSigningTest
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Independent local HMAC and core interoperate without re-encoding the body.
|
||||
** The reference uses literal wire names and prefixes to detect contract drift.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void testLocalAndCoreInteropWithRotation() throws Exception
|
||||
{
|
||||
List<String> texts = List.of("a-raw-shared-secret-of-enough-length", "whsec_MfKQ9r8GKYqrTwjUPD8ILPZIo2LaLaSw");
|
||||
List<WebhookSecret> secrets = texts.stream().map(WebhookSecret::parse).toList();
|
||||
for(byte[] body : List.of(BODY, new byte[0], new byte[] { 0, (byte) 0xff, (byte) 0xc3, (byte) 0xa9, 10 }))
|
||||
{
|
||||
String first = localSignature("evt_interop", "1760000000", body, texts.get(0));
|
||||
String second = localSignature("evt_interop", "1760000000", body, texts.get(1));
|
||||
Map<String, String> local = Map.of("webhook-id", "evt_interop", "webhook-timestamp", "1760000000",
|
||||
"webhook-signature", "v1," + first + " v1," + second);
|
||||
Map<String, String> core = WebhookSigner.headers("evt_interop", NOW, body, secrets);
|
||||
assertThat(core).isEqualTo(local);
|
||||
for(int i = 0; i < secrets.size(); i++)
|
||||
{
|
||||
verifier(List.of(secrets.get(i))).verify(local, body);
|
||||
assertThat(localVerify(core, body, texts.get(i))).isTrue();
|
||||
}
|
||||
assertThat(localVerify(core, body, "an-unrelated-secret-of-enough-length")).isFalse();
|
||||
verifier(secrets).verify(WebhookSigner.headers("evt_interop", NOW, body, List.of(secrets.get(1))), body);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The five-minute window includes both boundaries, but not the next second.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void testExactTimestampTolerance()
|
||||
{
|
||||
assertThat(WebhookVerifier.DEFAULT_TOLERANCE).isEqualTo(Duration.ofSeconds(300));
|
||||
WebhookSecret secret = WebhookSecret.generate();
|
||||
for(long offset : new long[] { -300, 300 })
|
||||
{
|
||||
assertDoesNotThrow(() -> verifier(List.of(secret)).verify(WebhookSigner.headers("evt_edge", NOW.plusSeconds(offset), BODY, List.of(secret)), BODY));
|
||||
}
|
||||
for(long offset : new long[] { -301, 301 })
|
||||
{
|
||||
assertThatThrownBy(() -> verifier(List.of(secret)).verify(WebhookSigner.headers("evt_edge", NOW.plusSeconds(offset), BODY, List.of(secret)), BODY))
|
||||
.isInstanceOf(WebhookVerificationException.class)
|
||||
.extracting(e -> ((WebhookVerificationException) e).getReason())
|
||||
.isEqualTo(WebhookVerificationException.Reason.TIMESTAMP_OUT_OF_TOLERANCE);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Test-only reference HMAC, independent of core signing and secret parsing.
|
||||
*******************************************************************************/
|
||||
private static String localSignature(String id, String timestamp, byte[] body, String text) throws Exception
|
||||
{
|
||||
byte[] key = text.startsWith("whsec_") ? Base64.getDecoder().decode(text.substring(6)) : text.getBytes(StandardCharsets.UTF_8);
|
||||
Mac mac = Mac.getInstance("HmacSHA256");
|
||||
mac.init(new SecretKeySpec(key, "HmacSHA256"));
|
||||
mac.update((id + "." + timestamp + ".").getBytes(StandardCharsets.UTF_8));
|
||||
return (Base64.getEncoder().encodeToString(mac.doFinal(body)));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Test-only receiver checks rotation signatures independently of core.
|
||||
*******************************************************************************/
|
||||
private static boolean localVerify(Map<String, String> headers, byte[] body, String secret) throws Exception
|
||||
{
|
||||
byte[] expected = localSignature(headers.get("webhook-id"), headers.get("webhook-timestamp"), body, secret).getBytes(StandardCharsets.US_ASCII);
|
||||
return (Arrays.stream(headers.get("webhook-signature").split(" "))
|
||||
.filter(part -> part.startsWith("v1,"))
|
||||
.anyMatch(part -> MessageDigest.isEqual(expected, part.substring(3).getBytes(StandardCharsets.US_ASCII))));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The Standard Webhooks reference vector (standard-webhooks/standard-webhooks,
|
||||
** libraries/go/webhook_test.go, TestWebhookSign).
|
||||
Reference in new issue
Block a user