refactor(signing): core WebhookSigner/WebhookVerifier (qqq#1160) #13

Merged
KofTwentyTwo merged 1 commits from feature/arch-core-signer into develop 2026-10-14 00:36:50 +00:00
17 changed files with 125 additions and 589 deletions

No files matched your search

+1 -1
View File
@@ -31,7 +31,7 @@ Design: `docs/DESIGN.md` (from QRun-IO/qqq#1105). Inbound webhooks are phase 2 (
- `DeliveryEngine` (claim, send, outcome), `EndpointHealth`
- `WebhooksAlerts` (reflective Alerts SPI)
- `secrets` (AES-GCM, env references, rotation), `processes`, `app` (home, secret widget), `permissions`, `esb`, `support/SystemWrites`.
- `shared`: INTERNAL COPY of the core signer and safe HTTP client (QRun-IO/qqq#1109). Replace with `core.utils.webhooks` / `core.utils.http` at QQQ 4.1.0-RC.5 (DESIGN section 11). Keep it identical to core until then.
- Signing uses `core.utils.webhooks` directly at the QQQ 4.1.0-RC.5 baseline (QRun-IO/qqq#1160). `shared.http` remains an internal copy of the safe HTTP client pending a separate migration to `core.utils.http` (DESIGN section 11).
## Invariants
+4
View File
@@ -5,6 +5,10 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and
## [Unreleased]
### Changed
- Require QQQ 4.1.0-RC.5 and use its webhook signer, verifier, secret and verification exception (QRun-IO/qqq#1160, F11). Removed the internal signing package; Java consumers must import `com.kingsrook.qqq.backend.core.utils.webhooks` instead.
- Signing wire format, timestamp tolerance, rotation, endpoint management, encrypted secret storage and delivery behavior are unchanged.
## [1.0.0] - not yet released
A rebuild of the qbit on QQQ 4.1 (design: QRun-IO/qqq#1105, docs/DESIGN.md). Breaking: see docs/migration-0.x-to-1.0.md.
+1 -1
View File
@@ -31,7 +31,7 @@ Design: `docs/DESIGN.md` (from QRun-IO/qqq#1105). Inbound webhooks are phase 2 (
- `DeliveryEngine` (claim, send, outcome), `EndpointHealth`
- `WebhooksAlerts` (reflective Alerts SPI)
- `secrets` (AES-GCM, env references, rotation), `processes`, `app` (home, secret widget), `permissions`, `esb`, `support/SystemWrites`.
- `shared`: INTERNAL COPY of the core signer and safe HTTP client (QRun-IO/qqq#1109). Replace with `core.utils.webhooks` / `core.utils.http` at QQQ 4.1.0-RC.5 (DESIGN section 11). Keep it identical to core until then.
- Signing uses `core.utils.webhooks` directly at the QQQ 4.1.0-RC.5 baseline (QRun-IO/qqq#1160). `shared.http` remains an internal copy of the safe HTTP client pending a separate migration to `core.utils.http` (DESIGN section 11).
## Invariants
+2 -2
View File
@@ -2,12 +2,12 @@
Signed, SSRF-safe outbound webhooks for QQQ applications. Admins add **endpoints** and **subscribe** them to event types: table inserts, updates and deletes (in-process or from the ESB), process completions and failures, and custom domain events. The qbit queues one **delivery** per event per endpoint in the same transaction as the change. It then signs each delivery in the [Standard Webhooks](https://www.standardwebhooks.com/) format and sends it with retries, backoff, dead letters and endpoint health. Every row is a normal QQQ table, and the app has a home dashboard.
Design: [QRun-IO/qqq#1105](https://github.com/QRun-IO/qqq/pull/1105) (`docs/specs/2026-10-12-qbit-webhooks-assessment.md`), and [docs/DESIGN.md](docs/DESIGN.md) for what 1.0 implements. The shared signer and safe HTTP client are in QQQ core from 4.1.0-RC.5 ([QRun-IO/qqq#1109](https://github.com/QRun-IO/qqq/pull/1109)).
Design: [QRun-IO/qqq#1105](https://github.com/QRun-IO/qqq/pull/1105) (`docs/specs/2026-10-12-qbit-webhooks-assessment.md`), and [docs/DESIGN.md](docs/DESIGN.md) for what 1.0 implements. Signing and secret generation use QQQ core directly; the local safe HTTP client remains pending a separate migration. Both primitives are available in QQQ core from 4.1.0-RC.5 ([QRun-IO/qqq#1109](https://github.com/QRun-IO/qqq/pull/1109)).
## Requirements
- Java 21
- QQQ 4.1.0-RC.4 or later (`qqq-backend-core`). Two modules are optional:
- QQQ 4.1.0-RC.5 or later (`qqq-backend-core`). Two modules are optional:
- `qqq-middleware-api`, for payloads rendered through a QQQ API version.
- `qqq-esb`, for ESB topics as an event source.
- A SQL backend for the five tables (DDL in [docs/ddl/webhooks-postgres.sql](docs/ddl/webhooks-postgres.sql)), and a QQQ scheduler for delivery.
+12 -6
View File
@@ -6,7 +6,7 @@
| # | Decision |
|---|---|
| D1 | Enhance the existing qbit into 1.0; breaking changes allowed |
| D2 | The signer and SSRF-safe client live in `qqq-backend-core` (QRun-IO/qqq#1109). This qbit carries an identical internal copy in `com.kingsrook.qbits.webhooks.shared` until QQQ 4.1.0-RC.5 is published (see section 11) |
| D2 | The signer and SSRF-safe client live in `qqq-backend-core` (QRun-IO/qqq#1109). Signing uses core directly at the QQQ 4.1.0-RC.5 baseline; the internal HTTP copy remains pending a separate migration (see section 11) |
| D3 | Tables renamed to `webhooks_*` |
| D4 | Internal and private targets blocked by default; admin-only allowlist per endpoint |
| D5 | Inbound is phase 2, in this qbit |
@@ -265,10 +265,16 @@ Goal: verify and record signed callbacks from third parties (Authentik, GitHub,
## 11. Moving to the core primitives (QQQ 4.1.0-RC.5)
QRun-IO/qqq#1109 adds the same classes to `qqq-backend-core` (`com.kingsrook.qqq.backend.core.utils.webhooks` and `...utils.http`). When RC.5 is on Maven Central:
QRun-IO/qqq#1160 (F11) migrates signing to the released `qqq-bom-pom` 4.1.0-RC.5 baseline. Delivery signing, secret generation/rotation and receiver verification use `com.kingsrook.qqq.backend.core.utils.webhooks` directly. The local `shared.signing` package is removed; endpoints, encrypted secret storage and delivery remain in this qbit. Inbound routes are still phase 2 (design only).
1. Bump `qqq.snapshot.version` to `4.1.0-RC.5`.
2. Delete `src/main/java/com/kingsrook/qbits/webhooks/shared/` and its tests.
3. Change the imports (`shared.signing` to `core.utils.webhooks`, `shared.http` to `core.utils.http`).
Comparison against core sources at tag `v4.1.0-RC.5` found no executable differences in `WebhookSigner`, `WebhookVerifier`, `WebhookSecret` or `WebhookVerificationException`; only package names, copyright years and migration comments differed. No compatibility adapter is needed:
The behaviour is identical, and the classes were kept the same on purpose.
- Headers remain `webhook-id`, `webhook-timestamp`, `webhook-signature` (case-insensitive when verifying).
- HMAC-SHA256 signs UTF-8 `id.timestamp.` followed by unchanged body bytes; signatures are padded Base64 with `v1,` and a single space between rotation signatures, in secret-list order.
- Secrets remain `whsec_` plus padded Base64 key bytes; unprefixed secrets use trimmed UTF-8 bytes. Generation uses 32 bytes, with a 16-byte minimum for imported keys.
- Default timestamp tolerance remains five minutes in either direction, including the boundary.
- Verification accepts any matching `v1` entry with any known secret and ignores unknown signature versions.
Signing contract tests retain the reference vector and test independent local HMAC signing against core verification and core signing against local verification, including two-secret rotation and exact tolerance boundaries. Delivery integration tests verify actual requests with core.
The `shared.http` copy is outside F11 and remains for a separate migration to `com.kingsrook.qqq.backend.core.utils.http`.
+3
View File
@@ -28,6 +28,9 @@ webhook-attempt: 1
## Java (QQQ core 4.1.0-RC.5 or later)
```java
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerifier;
WebhookVerifier verifier = new WebhookVerifier(List.of(WebhookSecret.parse(System.getenv("WEBHOOK_SECRET"))));
verifier.verify(headersAsMap, rawBodyBytes); // throws WebhookVerificationException with a safe reason
```
+1 -1
View File
@@ -215,7 +215,7 @@
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<qqq.snapshot.version>4.1.0-RC.4</qqq.snapshot.version>
<qqq.snapshot.version>4.1.0-RC.5</qqq.snapshot.version>
</properties>
<dependencyManagement>
<dependencies>
@@ -42,8 +42,6 @@ import com.kingsrook.qbits.webhooks.secrets.EndpointSecrets;
import com.kingsrook.qbits.webhooks.shared.http.SafeHttpClient;
import com.kingsrook.qbits.webhooks.shared.http.SafeHttpException;
import com.kingsrook.qbits.webhooks.shared.http.SafeHttpResponse;
import com.kingsrook.qbits.webhooks.shared.signing.WebhookSecret;
import com.kingsrook.qbits.webhooks.shared.signing.WebhookSigner;
import com.kingsrook.qbits.webhooks.support.SystemWrites;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
@@ -55,6 +53,8 @@ import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
import com.kingsrook.qqq.backend.core.utils.StringUtils;
import com.kingsrook.qqq.backend.core.utils.ValueUtils;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSigner;
import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
@@ -38,7 +38,6 @@ import com.kingsrook.qbits.webhooks.registry.WebhooksRegistry;
import com.kingsrook.qbits.webhooks.secrets.EnvironmentSecrets;
import com.kingsrook.qbits.webhooks.shared.http.AddressRules;
import com.kingsrook.qbits.webhooks.shared.http.SafeHttpException;
import com.kingsrook.qbits.webhooks.shared.signing.WebhookSecret;
import com.kingsrook.qbits.webhooks.support.SystemWrites;
import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
import com.kingsrook.qqq.backend.core.actions.permissions.PermissionsHelper;
@@ -50,6 +49,7 @@ import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.statusmessages.BadInputStatusMessage;
import com.kingsrook.qqq.backend.core.utils.StringUtils;
import com.kingsrook.qqq.backend.core.utils.ValueUtils;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret;
/*******************************************************************************
@@ -30,7 +30,6 @@ import com.kingsrook.qbits.webhooks.WebhooksRuntime;
import com.kingsrook.qbits.webhooks.metadata.WebhooksEnums.SecretSource;
import com.kingsrook.qbits.webhooks.metadata.WebhooksEnums.SecretStatus;
import com.kingsrook.qbits.webhooks.metadata.WebhooksTables;
import com.kingsrook.qbits.webhooks.shared.signing.WebhookSecret;
import com.kingsrook.qbits.webhooks.support.SystemWrites;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
@@ -41,6 +40,7 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
import com.kingsrook.qqq.backend.core.utils.ValueUtils;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret;
import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
@@ -1,183 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.shared.signing;
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
import java.util.Arrays;
import java.util.Base64;
import java.util.HexFormat;
/*******************************************************************************
** A webhook signing secret (Standard Webhooks format).
**
** <p>The text form is {@code whsec_} followed by the base64 of the key bytes.
** The HMAC key is the decoded bytes. A secret without the prefix is used as
** raw UTF-8 bytes, so a secret shared with an older receiver still works.</p>
**
** <p>{@link #toString()} never reveals the key: it prints the fingerprint.</p>
**
** <p>INTERNAL COPY: replaced by {@code com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret}
** when the qbit moves to QQQ 4.1.0-RC.5 (QRun-IO/qqq, D2 of the webhooks assessment).
** Keep the two identical.</p>
*******************************************************************************/
public final class WebhookSecret
{
public static final String PREFIX = "whsec_";
/** Key length for generated secrets, in bytes (256 bits). */
public static final int GENERATED_KEY_BYTES = 32;
/** Shortest key accepted, in bytes (128 bits). */
public static final int MINIMUM_KEY_BYTES = 16;
private static final SecureRandom RANDOM = new SecureRandom();
private final byte[] key;
/*******************************************************************************
** Constructor (keeps a copy of the key).
*******************************************************************************/
private WebhookSecret(byte[] key)
{
if(key == null || key.length < MINIMUM_KEY_BYTES)
{
throw (new IllegalArgumentException("A webhook signing secret must be at least " + MINIMUM_KEY_BYTES + " bytes"));
}
this.key = Arrays.copyOf(key, key.length);
}
/*******************************************************************************
** A new random secret.
*******************************************************************************/
public static WebhookSecret generate()
{
byte[] bytes = new byte[GENERATED_KEY_BYTES];
RANDOM.nextBytes(bytes);
return (new WebhookSecret(bytes));
}
/*******************************************************************************
** Parse the text form ({@code whsec_<base64>}, or raw text).
*******************************************************************************/
public static WebhookSecret parse(String text)
{
if(text == null || text.isBlank())
{
throw (new IllegalArgumentException("A webhook signing secret is required"));
}
String trimmed = text.trim();
if(trimmed.startsWith(PREFIX))
{
try
{
return (new WebhookSecret(Base64.getDecoder().decode(trimmed.substring(PREFIX.length()))));
}
catch(IllegalArgumentException e)
{
throw (new IllegalArgumentException("A " + PREFIX + " secret must be base64 after the prefix", e));
}
}
return (new WebhookSecret(trimmed.getBytes(StandardCharsets.UTF_8)));
}
/*******************************************************************************
** The text form to give a receiver ({@code whsec_<base64>}).
*******************************************************************************/
public String reveal()
{
return (PREFIX + Base64.getEncoder().encodeToString(key));
}
/*******************************************************************************
** The HMAC key (a copy).
*******************************************************************************/
public byte[] keyBytes()
{
return (Arrays.copyOf(key, key.length));
}
/*******************************************************************************
** A short, non-reversible id for the secret (first 12 hex chars of its
** SHA-256), safe to store and show, so people can tell secrets apart.
*******************************************************************************/
public String fingerprint()
{
try
{
byte[] digest = MessageDigest.getInstance("SHA-256").digest(key);
return (HexFormat.of().formatHex(digest).substring(0, 12));
}
catch(NoSuchAlgorithmException e)
{
throw (new IllegalStateException("SHA-256 is unavailable", e));
}
}
/*******************************************************************************
** Never prints the key.
*******************************************************************************/
@Override
public String toString()
{
return ("WebhookSecret[" + fingerprint() + "]");
}
/*******************************************************************************
** Equal when the keys are equal (constant time).
*******************************************************************************/
@Override
public boolean equals(Object o)
{
return (o instanceof WebhookSecret other && MessageDigest.isEqual(key, other.key));
}
/*******************************************************************************
** Hash of the fingerprint.
*******************************************************************************/
@Override
public int hashCode()
{
return (fingerprint().hashCode());
}
}
@@ -1,147 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.shared.signing;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import java.nio.charset.StandardCharsets;
import java.security.GeneralSecurityException;
import java.time.Instant;
import java.util.ArrayList;
import java.util.Base64;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
/*******************************************************************************
** Signs webhook requests in the Standard Webhooks format
** (https://www.standardwebhooks.com/).
**
** <p>The signed content is {@code <id>.<timestamp>.<body>}: the message id,
** a dot, the Unix timestamp in seconds, a dot, then the exact body bytes. The
** signature is HMAC-SHA256 with the secret's key, base64 encoded, and sent as
** {@code v1,<base64>}. During a secret rotation every active secret signs, and
** the signatures are separated by spaces, so a receiver holding either the old
** or the new secret accepts the request.</p>
**
** <p>Headers: {@code webhook-id}, {@code webhook-timestamp},
** {@code webhook-signature}.</p>
**
** <p>INTERNAL COPY: replaced by {@code com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSigner}
** when the qbit moves to QQQ 4.1.0-RC.5. Keep the two identical.</p>
*******************************************************************************/
public final class WebhookSigner
{
public static final String HEADER_ID = "webhook-id";
public static final String HEADER_TIMESTAMP = "webhook-timestamp";
public static final String HEADER_SIGNATURE = "webhook-signature";
/** The only signature scheme today: HMAC-SHA256. */
public static final String VERSION_V1 = "v1";
private static final String HMAC = "HmacSHA256";
/*******************************************************************************
** Utility class.
*******************************************************************************/
private WebhookSigner()
{
}
/*******************************************************************************
** The three signature headers for one request.
**
** @param messageId the event id (stable across retries: it is the idempotency key)
** @param timestamp when this attempt is sent
** @param body the exact bytes that are sent
** @param secrets the active secrets (one, or two during a rotation)
*******************************************************************************/
public static Map<String, String> headers(String messageId, Instant timestamp, byte[] body, List<WebhookSecret> secrets)
{
long seconds = timestamp.getEpochSecond();
Map<String, String> headers = new LinkedHashMap<>();
headers.put(HEADER_ID, messageId);
headers.put(HEADER_TIMESTAMP, Long.toString(seconds));
headers.put(HEADER_SIGNATURE, signatureHeader(messageId, seconds, body, secrets));
return (headers);
}
/*******************************************************************************
** The {@code webhook-signature} value: {@code v1,<sig>} per secret, space
** separated.
*******************************************************************************/
public static String signatureHeader(String messageId, long timestampSeconds, byte[] body, List<WebhookSecret> secrets)
{
if(secrets == null || secrets.isEmpty())
{
throw (new IllegalArgumentException("At least one signing secret is required"));
}
List<String> parts = new ArrayList<>();
for(WebhookSecret secret : secrets)
{
parts.add(VERSION_V1 + "," + sign(messageId, timestampSeconds, body, secret));
}
return (String.join(" ", parts));
}
/*******************************************************************************
** One base64 HMAC-SHA256 signature over {@code <id>.<timestamp>.<body>}.
*******************************************************************************/
public static String sign(String messageId, long timestampSeconds, byte[] body, WebhookSecret secret)
{
validateId(messageId);
try
{
Mac mac = Mac.getInstance(HMAC);
mac.init(new SecretKeySpec(secret.keyBytes(), HMAC));
mac.update((messageId + "." + timestampSeconds + ".").getBytes(StandardCharsets.UTF_8));
mac.update(body == null ? new byte[0] : body);
return (Base64.getEncoder().encodeToString(mac.doFinal()));
}
catch(GeneralSecurityException e)
{
throw (new IllegalStateException("HMAC-SHA256 is unavailable", e));
}
}
/*******************************************************************************
** A message id may not contain the separator or whitespace.
*******************************************************************************/
static void validateId(String messageId)
{
if(messageId == null || messageId.isEmpty() || messageId.contains(".") || messageId.chars().anyMatch(Character::isWhitespace))
{
throw (new IllegalArgumentException("A webhook message id is required and may not contain '.' or whitespace"));
}
}
}
@@ -1,70 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.shared.signing;
/*******************************************************************************
** A webhook request failed verification. The message never contains a
** secret or a signature, so it is safe to log and to return to the caller.
**
** <p>INTERNAL COPY: replaced by the class of the same name in
** {@code com.kingsrook.qqq.backend.core.utils.webhooks} at QQQ 4.1.0-RC.5.</p>
*******************************************************************************/
public class WebhookVerificationException extends Exception
{
private static final long serialVersionUID = 1L;
/*******************************************************************************
** Why verification failed.
*******************************************************************************/
public enum Reason
{
MISSING_HEADERS,
BAD_TIMESTAMP,
TIMESTAMP_OUT_OF_TOLERANCE,
NO_MATCHING_SIGNATURE
}
private final Reason reason;
/*******************************************************************************
** Constructor.
*******************************************************************************/
public WebhookVerificationException(Reason reason, String message)
{
super(message);
this.reason = reason;
}
/*******************************************************************************
** Getter for reason.
*******************************************************************************/
public Reason getReason()
{
return (reason);
}
}
@@ -1,165 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.shared.signing;
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.time.Clock;
import java.time.Duration;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
import java.util.Map;
/*******************************************************************************
** Verifies a signed webhook request (Standard Webhooks format, see
** {@link WebhookSigner}).
**
** <p>Checks, in order: the three headers are present, the timestamp is within
** the tolerance of now (default five minutes, either direction, which stops
** replays of old requests), and at least one {@code v1} signature matches one
** of the receiver's secrets. Comparison is constant time. Unknown signature
** versions are ignored, so a future scheme can be added alongside v1.</p>
**
** <p>Receivers should also remember recent {@code webhook-id}s and ignore a
** repeat: the same id is sent again on every retry of the same event.</p>
**
** <p>INTERNAL COPY: replaced by {@code com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerifier}
** when the qbit moves to QQQ 4.1.0-RC.5. Keep the two identical.</p>
*******************************************************************************/
public final class WebhookVerifier
{
public static final Duration DEFAULT_TOLERANCE = Duration.ofMinutes(5);
private final List<WebhookSecret> secrets;
private final Duration tolerance;
private final Clock clock;
/*******************************************************************************
** Verifier with the default tolerance and the system clock.
*******************************************************************************/
public WebhookVerifier(List<WebhookSecret> secrets)
{
this(secrets, DEFAULT_TOLERANCE, Clock.systemUTC());
}
/*******************************************************************************
** Verifier.
*******************************************************************************/
public WebhookVerifier(List<WebhookSecret> secrets, Duration tolerance, Clock clock)
{
if(secrets == null || secrets.isEmpty())
{
throw (new IllegalArgumentException("At least one secret is required to verify webhooks"));
}
this.secrets = new ArrayList<>(secrets);
this.tolerance = tolerance;
this.clock = clock;
}
/*******************************************************************************
** Verify a request. Header names are matched case-insensitively.
**
** @throws WebhookVerificationException with a reason safe to log
*******************************************************************************/
public void verify(Map<String, String> headers, byte[] body) throws WebhookVerificationException
{
String id = header(headers, WebhookSigner.HEADER_ID);
String timestamp = header(headers, WebhookSigner.HEADER_TIMESTAMP);
String signature = header(headers, WebhookSigner.HEADER_SIGNATURE);
if(id == null || timestamp == null || signature == null)
{
throw (new WebhookVerificationException(WebhookVerificationException.Reason.MISSING_HEADERS, "Missing webhook-id, webhook-timestamp or webhook-signature"));
}
long seconds;
try
{
seconds = Long.parseLong(timestamp.trim());
}
catch(NumberFormatException e)
{
throw (new WebhookVerificationException(WebhookVerificationException.Reason.BAD_TIMESTAMP, "webhook-timestamp is not a Unix timestamp"));
}
long now = clock.instant().getEpochSecond();
if(Math.abs(now - seconds) > tolerance.toSeconds())
{
throw (new WebhookVerificationException(WebhookVerificationException.Reason.TIMESTAMP_OUT_OF_TOLERANCE, "webhook-timestamp is outside the allowed window"));
}
try
{
WebhookSigner.validateId(id);
}
catch(IllegalArgumentException e)
{
throw (new WebhookVerificationException(WebhookVerificationException.Reason.MISSING_HEADERS, e.getMessage()));
}
for(String part : signature.trim().split(" +"))
{
int comma = part.indexOf(',');
if(comma < 0 || !WebhookSigner.VERSION_V1.equals(part.substring(0, comma)))
{
continue;
}
byte[] offered = part.substring(comma + 1).getBytes(StandardCharsets.US_ASCII);
for(WebhookSecret secret : secrets)
{
byte[] expected = WebhookSigner.sign(id, seconds, body, secret).getBytes(StandardCharsets.US_ASCII);
if(MessageDigest.isEqual(expected, offered))
{
return;
}
}
}
throw (new WebhookVerificationException(WebhookVerificationException.Reason.NO_MATCHING_SIGNATURE, "No v1 signature matches a known secret"));
}
/*******************************************************************************
** Case-insensitive header lookup.
*******************************************************************************/
private static String header(Map<String, String> headers, String name)
{
if(headers == null)
{
return (null);
}
for(Map.Entry<String, String> entry : headers.entrySet())
{
if(entry.getKey() != null && entry.getKey().toLowerCase(Locale.ROOT).equals(name))
{
return (entry.getValue());
}
}
return (null);
}
}
@@ -27,15 +27,15 @@ import java.time.Duration;
import java.util.List;
import com.kingsrook.qbits.webhooks.H2Support;
import com.kingsrook.qbits.webhooks.WebhooksRuntime;
import com.kingsrook.qbits.webhooks.WebhooksTestSupport;
import com.kingsrook.qbits.webhooks.WebhooksTestSupport.Receiver;
import com.kingsrook.qbits.webhooks.WebhooksTestSupport;
import com.kingsrook.qbits.webhooks.metadata.WebhooksTables;
import com.kingsrook.qbits.webhooks.secrets.EndpointSecrets;
import com.kingsrook.qbits.webhooks.shared.signing.WebhookSecret;
import com.kingsrook.qbits.webhooks.shared.signing.WebhookVerifier;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerifier;
import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
@@ -23,16 +23,16 @@ package com.kingsrook.qbits.webhooks.delivery;
import java.time.Duration;
import java.util.List;
import com.kingsrook.qbits.webhooks.WebhooksTestSupport;
import com.kingsrook.qbits.webhooks.WebhooksTestSupport.Receiver;
import com.kingsrook.qbits.webhooks.WebhooksTestSupport;
import com.kingsrook.qbits.webhooks.metadata.WebhooksTables;
import com.kingsrook.qbits.webhooks.processes.WebhooksProcesses;
import com.kingsrook.qbits.webhooks.secrets.EndpointSecrets;
import com.kingsrook.qbits.webhooks.shared.signing.WebhookSecret;
import com.kingsrook.qbits.webhooks.shared.signing.WebhookVerificationException;
import com.kingsrook.qbits.webhooks.shared.signing.WebhookVerifier;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.utils.JsonUtils;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerificationException;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerifier;
import org.json.JSONObject;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
@@ -18,17 +18,26 @@
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.shared.signing;
package com.kingsrook.qbits.webhooks.signing;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.time.Clock;
import java.time.Duration;
import java.time.Instant;
import java.time.ZoneOffset;
import java.util.Arrays;
import java.util.Base64;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSigner;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerificationException;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerifier;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
@@ -36,7 +45,7 @@ import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
/*******************************************************************************
** Unit test for WebhookSecret, WebhookSigner and WebhookVerifier.
** Qbit wire contract and interoperability with the released core primitives.
*******************************************************************************/
class WebhookSigningTest
{
@@ -45,6 +54,85 @@ class WebhookSigningTest
/*******************************************************************************
** Independent local HMAC and core interoperate without re-encoding the body.
** The reference uses literal wire names and prefixes to detect contract drift.
*******************************************************************************/
@Test
void testLocalAndCoreInteropWithRotation() throws Exception
{
List<String> texts = List.of("a-raw-shared-secret-of-enough-length", "whsec_MfKQ9r8GKYqrTwjUPD8ILPZIo2LaLaSw");
List<WebhookSecret> secrets = texts.stream().map(WebhookSecret::parse).toList();
for(byte[] body : List.of(BODY, new byte[0], new byte[] { 0, (byte) 0xff, (byte) 0xc3, (byte) 0xa9, 10 }))
{
String first = localSignature("evt_interop", "1760000000", body, texts.get(0));
String second = localSignature("evt_interop", "1760000000", body, texts.get(1));
Map<String, String> local = Map.of("webhook-id", "evt_interop", "webhook-timestamp", "1760000000",
"webhook-signature", "v1," + first + " v1," + second);
Map<String, String> core = WebhookSigner.headers("evt_interop", NOW, body, secrets);
assertThat(core).isEqualTo(local);
for(int i = 0; i < secrets.size(); i++)
{
verifier(List.of(secrets.get(i))).verify(local, body);
assertThat(localVerify(core, body, texts.get(i))).isTrue();
}
assertThat(localVerify(core, body, "an-unrelated-secret-of-enough-length")).isFalse();
verifier(secrets).verify(WebhookSigner.headers("evt_interop", NOW, body, List.of(secrets.get(1))), body);
}
}
/*******************************************************************************
** The five-minute window includes both boundaries, but not the next second.
*******************************************************************************/
@Test
void testExactTimestampTolerance()
{
assertThat(WebhookVerifier.DEFAULT_TOLERANCE).isEqualTo(Duration.ofSeconds(300));
WebhookSecret secret = WebhookSecret.generate();
for(long offset : new long[] { -300, 300 })
{
assertDoesNotThrow(() -> verifier(List.of(secret)).verify(WebhookSigner.headers("evt_edge", NOW.plusSeconds(offset), BODY, List.of(secret)), BODY));
}
for(long offset : new long[] { -301, 301 })
{
assertThatThrownBy(() -> verifier(List.of(secret)).verify(WebhookSigner.headers("evt_edge", NOW.plusSeconds(offset), BODY, List.of(secret)), BODY))
.isInstanceOf(WebhookVerificationException.class)
.extracting(e -> ((WebhookVerificationException) e).getReason())
.isEqualTo(WebhookVerificationException.Reason.TIMESTAMP_OUT_OF_TOLERANCE);
}
}
/*******************************************************************************
** Test-only reference HMAC, independent of core signing and secret parsing.
*******************************************************************************/
private static String localSignature(String id, String timestamp, byte[] body, String text) throws Exception
{
byte[] key = text.startsWith("whsec_") ? Base64.getDecoder().decode(text.substring(6)) : text.getBytes(StandardCharsets.UTF_8);
Mac mac = Mac.getInstance("HmacSHA256");
mac.init(new SecretKeySpec(key, "HmacSHA256"));
mac.update((id + "." + timestamp + ".").getBytes(StandardCharsets.UTF_8));
return (Base64.getEncoder().encodeToString(mac.doFinal(body)));
}
/*******************************************************************************
** Test-only receiver checks rotation signatures independently of core.
*******************************************************************************/
private static boolean localVerify(Map<String, String> headers, byte[] body, String secret) throws Exception
{
byte[] expected = localSignature(headers.get("webhook-id"), headers.get("webhook-timestamp"), body, secret).getBytes(StandardCharsets.US_ASCII);
return (Arrays.stream(headers.get("webhook-signature").split(" "))
.filter(part -> part.startsWith("v1,"))
.anyMatch(part -> MessageDigest.isEqual(expected, part.substring(3).getBytes(StandardCharsets.US_ASCII))));
}
/*******************************************************************************
** The Standard Webhooks reference vector (standard-webhooks/standard-webhooks,
** libraries/go/webhook_test.go, TestWebhookSign).