Compare commits

..
Author SHA1 Message Date
james.maesandCodex 6ce155931e ci: centralize QBit source scanning
Co-Authored-By: Codex <noreply@openai.com>
2026-10-21 10:17:50 -05:00
james.maesandCodex 7b6eb40c18 ci: adopt central orb 0.7.12 and stable gates (#30)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 08:53:26 -05:00
james.maesandCodex af7c289349 ci: centralize rc release completion in qqq orb (#29)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 02:25:05 -05:00
james.maesandCodex cd35d575cb ci: adopt shared orb 0.7.11 (#28)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 00:56:25 -05:00
james.maes 20c17265e5 Merge pull request #27 from QRun-IO/feature/central-ci
ci: adopt shared QQQ orb standard
2026-10-20 19:20:12 -05:00
james.maesandCodex bde4a0a45d ci: adopt verified central QQQ orb 0.7.3
Co-Authored-By: Codex <noreply@openai.com>
2026-10-20 19:07:48 -05:00
james.maes 4b75120668 Merge pull request #25 from QRun-IO/fix/webhooks-health-count-url
fix(webhooks): filtered JSON drill links on health tile and home KPIs
2026-10-16 10:20:03 -05:00
james.maesandClaude Opus 5.5 63ee6c2909 fix(webhooks): filtered JSON drill links on the health tile and home KPIs
The webhooksHealth tile linked with a base64 filter, unlike every other
qbit (url-encoded JSON). Next 1.0.0-RC.35 deserializeFilter accepts both,
but use the common JSON form. The home KPI tiles all linked to the
unfiltered delivery table; they now open Deliveries (or Attempts) with the
matching filter (delivered in 24 h, failed attempts, dead letters, waiting).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 09:08:08 -05:00
james.maes a6bfc99e92 Merge pull request #23 from QRun-IO/ci/release-rc-secrets-inherit
ci(release-rc): pass secrets so the shared workflow can sign the RC tag
2026-10-16 08:04:11 -05:00
james.maesandClaude Opus 5.5 1d6ebc2c1f ci(release-rc): pass secrets to the shared workflow so it can sign the RC tag
QRun-IO/.github#2 adds a tag-and-release job that signs vX with the
QRUN_TAG_SIGNING_KEY secret after Maven Central confirms the artifact.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 16:20:54 -05:00
james.maes d83acbb0e4 Merge pull request #22 from QRun-IO/fix/producer-version-from-build
fix: qbit metadata version comes from the build
2026-10-15 01:42:56 -05:00
james.maesandClaude Opus 5.5 6b3be6d305 fix: the qbit metadata version comes from the build, not a hardcoded constant
The producer read a literal version, so released artifacts reported a stale
or -SNAPSHOT version. qbit-webhooks.properties (version=${project.version}) is
Maven-filtered and the producer loads it; tests assert producer == bundle ==
artifact version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 01:27:39 -05:00
james.maes fb37f1a662 Merge pull request #21 from QRun-IO/fix/help-bundle-version-test
test(help): bundle version test compares with the artifact version
2026-10-14 23:18:43 -05:00
james.maesandClaude Opus 5.5 d03c902f58 test(help): compare the help bundle version with the artifact version, not a hardcoded SNAPSHOT
The bundle version is now Maven-filtered, so a literal SNAPSHOT in the test
fails every release build. Surefire passes the artifact version in
qbit.artifactVersion.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 23:07:56 -05:00
james.maes 673e665aba Merge pull request #20 from QRun-IO/fix/help-bundle-version
fix(help): help bundle version comes from the build, not a hardcoded SNAPSHOT
2026-10-14 22:37:20 -05:00
james.maesandClaude Opus 5.5 9dfe565df0 fix(help): the help bundle carries the artifact version (Maven-filtered) instead of a hardcoded SNAPSHOT
On release branches the bundle's version and the producer's differed, so the
bundle test that asserts they match failed, and released bundles recorded
SNAPSHOT as their provenance. help/*.yaml is now filtered:
version: ${project.version}.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 22:04:41 -05:00
james.maes b67abf4ff4 Merge pull request #16 from QRun-IO/feature/health-checks
feat(health): register webhook endpoint and dispatcher checks with core central health
2026-10-14 21:27:00 -05:00
james.maesandClaude Opus 5.5 0a8ccb3442 Merge origin/develop into feature/health-checks; pin QQQ 4.1.0-RC.8
RC.8 ships the central health kernel (QRun-IO/qqq#1207).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 21:15:07 -05:00
james.maes b31e65f40d Merge pull request #19 from QRun-IO/feature/help-bundle
feat(help): qbit-webhooks help bundle via QQQ help bundles, with coverage gate
2026-10-14 20:29:38 -05:00
james.maesandClaude Opus 5.5 e84ec8b9e5 feat(help): ship the qbit-webhooks help bundle through QQQ help bundles, with coverage gate
Host-neutral help (deployment-specific facts removed), registered with
HelpBundles, gated by HelpBundleAssertions. QQQ 4.1.0-RC.7.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 20:17:15 -05:00
james.maes 35acb94756 Merge pull request #17 from QRun-IO/feature/foreign-keys
Foreign keys between the webhooks tables, with an orphan-safe upgrade
2026-10-14 18:00:28 -05:00
james.maesandClaude Opus 5.5 1259fa25fe Merge origin/develop into feature/foreign-keys
Keeps both changelog entries. H2 tests and the demo server load
src/test/resources/webhooks-h2.sql (FKs in H2 syntax), never the Postgres DDL
or the pg_temp upgrade script.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:52:26 -05:00
james.maes 3bd7cab084 Merge pull request #18 from QRun-IO/feature/tables-populated-defaults
feat: guided empty states, Setup card and tables guardrail
2026-10-14 17:11:31 -05:00
james.maesandClaude Opus 5.5 cf940eb3bb fix: setup card counts blocked endpoints as DeliveryEngine does; host help test; marker prose
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:07:08 -05:00
james.maesandClaude Opus 5.5 35af32dc1d feat: guided empty states, Setup card and tables guardrail
Help bundle for the app and all five tables (how rows get created, where
event types come from), applied by WebhooksHelp. Setup card on the home.
WebhooksTablesGuardrailTest. Test that an expired secret stops signing
before it is retired. Nothing is seeded (owner choices and logs).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 16:44:46 -05:00
james.maesandClaude Opus 5.5 3636aacc26 style(test): import order in the foreign key guardrail test
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 16:25:18 -05:00
james.maesandClaude Opus 5.5 a160ef216d feat(ddl): foreign keys between the webhooks tables, with an orphan-safe upgrade
Declare a named foreign key for every join and table possible-value field
between the five webhooks tables. Endpoint secrets and subscriptions are
deleted with their endpoint; deliveries and attempts keep it (RESTRICT);
a delivery outlives its subscription (SET NULL); attempts go with their
delivery. The upgrade script quarantines orphans before adding the keys and
replaces the NO ACTION endpoint keys. A Testcontainers guardrail test fails
when a new relation has no key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 16:15:19 -05:00
james.maesandClaude Opus 5.5 d650deb46f docs: webhooks health checks
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 14:55:12 -05:00
james.maesandClaude Opus 5.5 d988e00be6 refactor(webhooks): home links to central health instead of its own endpoint health tile
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 14:55:11 -05:00
james.maesandClaude Opus 5.5 3cf073e2d5 feat(health): register webhook endpoint and dispatcher checks with core central health
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 14:55:11 -05:00
james.maes 945ebe66ff Merge pull request #15 from QRun-IO/feature/orb-0.6.9
ci: qqq-orb 0.6.9 (refresh SNAPSHOT dependencies)
2026-10-14 10:46:45 -05:00
james.maesandClaude Opus 5.5 f7291b03b3 ci: qqq-orb 0.6.9 (refresh SNAPSHOT dependencies)
Moves qqq-orb@0.6.8 to qqq-orb@0.6.9, which passes --update-snapshots so cached SNAPSHOT dependencies are re-checked on every build (QRun-IO/qqq-orb#22).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 10:08:02 -05:00
james.maes 53a7844c5a Merge pull request #14 from QRun-IO/feature/guardrails-2.1.0
build: QQQ 4.1.0-RC.6 + guardrails (parent 2.1.0, enforced)
2026-10-14 07:22:11 -05:00
james.maesandClaude Opus 5.5 d62556b764 build: QQQ 4.1.0-RC.6 and qbit-build-parent 2.1.0 with guardrails enforced
Move the core baseline to the released 4.1.0-RC.6 (architecture review
finding F19, guardrail G7). Adopt qbit-build-parent 2.1.0 with the
architecture guardrails enforced (G1, G3, G4, G7) and tests in strict
raw-route mode (G2). Everything passes with no source changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 07:18:36 -05:00
james.maes f51b3ff75f Merge pull request #13 from QRun-IO/feature/arch-core-signer
refactor(signing): core WebhookSigner/WebhookVerifier (qqq#1160)
2026-10-13 19:36:49 -05:00
james.maesandClaude Opus 5.5 f33e592aef refactor(signing): use core WebhookSigner and WebhookVerifier
Delete the local shared/signing package (WebhookSigner, WebhookVerifier,
WebhookSecret, WebhookVerificationException) and use core
com.kingsrook.qqq.backend.core.utils.webhooks from QQQ 4.1.0-RC.5.
Delivery signing, inbound verification and secret generation and rotation
all go through core. The wire format is unchanged: same headers, v1,
prefix, whsec_ secrets and timestamp tolerance. Endpoints, secret storage
and delivery stay in this qbit.

Refs QRun-IO/qqq#1160 (this repo has issues disabled), architecture review
QRun-IO/qqq#1150 finding F11.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 19:33:16 -05:00
james.maes ad235f675f Merge pull request #12 from QRun-IO/feature/release-rc-automation
ci: automate RC releases; hotfix/* pushes no longer publish
2026-10-13 15:01:02 -05:00
james.maesandClaude Opus 5.5 1c73868acb ci: automate RC releases; hotfix/* pushes no longer publish
Removes publish_hotfix_release: a hotfix/* branch push published a GA
release to Maven Central with no tag or review (QRun-IO/qbit-cdn#12,
QRun-IO/qqq#1121). hotfix/* branches now run test_only; a hotfix
release publishes from a reviewed vX.Y.Z tag through publish_release.
Adds .github/workflows/release-rc.yml (reusable QRun-IO/.github
qbit-release-rc): a manual dispatch, or merging a develop PR labelled
release:rc, merges develop into the release line and the release/* push
runs the existing CircleCI RC publish.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 14:31:03 -05:00
james.maes 2b2a593ded Merge pull request #11 from QRun-IO/chore/remove-0x-sources
chore!: remove the 0.x webhooks sources and tests
2026-10-12 18:26:08 -05:00
james.maesandClaude Opus 5.5 09345924e4 chore!: remove the 0.x webhooks sources and tests
The owner approved removing the old code. Deletes the 0.x engine
(actions/, model/, tables/, the old processes and registry plugins) and
its tests, which 1.0 replaced, and drops the TEMPORARY compiler and
Checkstyle exclusions from pom.xml and the note in CLAUDE.md/AGENTS.md.

./mvnw clean verify: 144 tests, 0 failures, Checkstyle clean, JaCoCo
gate met.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 18:23:15 -05:00
james.maes 97a27195f8 Merge pull request #8 from QRun-IO/feature/webhooks-1.0
feat!: qbit-webhooks 1.0 (signed, SSRF-safe, replica-safe outbound webhooks)
2026-10-12 18:18:30 -05:00
97 changed files with 3033 additions and 10818 deletions

No files matched your search

+10 -73
View File
@@ -1,77 +1,14 @@
version: 2.1
setup: true
orbs:
qqq-orb: kingsrook/qqq-orb@0.6.8
commands:
configure-package-install:
steps:
- run:
name: Keep package installation unattended
command: |
# These build machines are discarded after the job.
sudo install -d /etc/needrestart/conf.d
printf '%s\n' '$nrconf{restart} = "l";' | sudo tee /etc/needrestart/conf.d/99-qqq-ci.conf >/dev/null
qqq-orb: kingsrook/qqq-orb@0.7.12
workflows:
test_only:
setup:
jobs:
- qqq-orb/mvn_test_only:
pre-steps:
- configure-package-install
context: [ qqq-maven-registry-credentials ]
filters:
branches:
ignore: /(develop|release/.*|hotfix/.*|integration.*)/
tags:
only: []
publish_snapshot:
jobs:
- qqq-orb/mvn_publish:
pre-steps:
- configure-package-install
context: [ qqq-maven-registry-credentials ]
branch_type: snapshot
filters:
branches:
only:
- develop
publish_release_candidate:
jobs:
- qqq-orb/mvn_publish:
pre-steps:
- configure-package-install
context: [ qqq-maven-registry-credentials ]
branch_type: release_candidate
filters:
branches:
only:
- /release\/.*/
publish_release:
jobs:
- qqq-orb/mvn_publish:
pre-steps:
- configure-package-install
context: [ qqq-maven-registry-credentials ]
branch_type: release
filters:
branches:
ignore: /.*/
tags:
only: [ /^v\d+\.\d+\.\d+$/ ]
publish_hotfix_release:
jobs:
- qqq-orb/mvn_publish:
pre-steps:
- configure-package-install
context: [ qqq-maven-registry-credentials ]
branch_type: hotfix
filters:
branches:
only:
- /hotfix\/.*/
- qqq-orb/generate_pipeline:
orb_version: 0.7.12
filters:
branches:
ignore: automation/qbit-rc-pending
tags:
only: /^v\d+\.\d+\.\d+$/
+23
View File
@@ -0,0 +1,23 @@
name: CodeQL
'on':
push:
branches:
- main
- develop
- release/**
- hotfix/**
pull_request: {}
schedule:
- cron: 41 16 * * 0
workflow_dispatch: {}
permissions:
contents: read
actions: read
security-events: write
packages: read
concurrency:
group: codeql-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs:
analyze:
uses: QRun-IO/qqq-orb/.github/workflows/qbit-codeql.yml@c288a8864fe9054f68410e1d78f5863aea4c4d22
+74
View File
@@ -0,0 +1,74 @@
name: Release candidate
'on':
workflow_dispatch:
inputs:
dry_run:
description: Plan a cut without changing release state
type: boolean
default: false
finalize:
description: Retry finalization of an existing cut; do not cut another RC
type: boolean
default: false
recover:
description: Recover the same published RC after a deployment timeout; never
reupload
type: boolean
default: false
circle_build:
description: Original Circle publisher build number (recovery only)
type: string
default: ''
merge_sha:
description: Existing cut merge_sha (finalize retry only)
type: string
default: ''
bump_sha:
description: Existing cut bump_sha (finalize retry only)
type: string
default: ''
branch:
description: Existing cut branch (finalize retry only)
type: string
default: ''
version:
description: Existing cut version (finalize retry only)
type: string
default: ''
pull_request:
types:
- closed
branches:
- develop
repository_dispatch:
types:
- qbit-rc-published
permissions:
contents: write
actions: read
checks: read
jobs:
release-rc:
if: github.event_name == 'workflow_dispatch' || github.event_name == 'repository_dispatch'
|| (github.event_name == 'pull_request' && github.event.pull_request.merged
== true && github.event.pull_request.base.ref == 'develop' && contains(github.event.pull_request.labels.*.name,
'release:rc'))
uses: QRun-IO/qqq-orb/.github/workflows/qbit-release-rc.yml@cc30f599e63c8e41629c99d23395bac1adb8bc44
with:
dry_run: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run || false
}}
finalize: ${{ github.event_name == 'repository_dispatch' || (github.event_name
== 'workflow_dispatch' && inputs.finalize) }}
recover: ${{ github.event_name == 'workflow_dispatch' && inputs.recover || false
}}
circle_build: ${{ github.event_name == 'workflow_dispatch' && inputs.circle_build
|| '' }}
merge_sha: ${{ github.event_name == 'repository_dispatch' && github.event.client_payload.merge_sha
|| inputs.merge_sha || '' }}
bump_sha: ${{ github.event_name == 'repository_dispatch' && github.event.client_payload.bump_sha
|| inputs.bump_sha || '' }}
branch: ${{ github.event_name == 'repository_dispatch' && github.event.client_payload.branch
|| inputs.branch || '' }}
version: ${{ github.event_name == 'repository_dispatch' && github.event.client_payload.version
|| inputs.version || '' }}
secrets: inherit
+16
View File
@@ -0,0 +1,16 @@
profile: qbit
orb_version: 0.7.12
default_branch: develop
java_version: '21'
browsers: true
api_checks: false
docs_only: false
release_on_main: false
release_tag_policy: stable
test_branch_policy: non-release-no-integration
publish:
snapshot: true
release_candidate: true
release: true
asciidoc: false
release_completion: true
+1 -7
View File
@@ -31,7 +31,7 @@ Design: `docs/DESIGN.md` (from QRun-IO/qqq#1105). Inbound webhooks are phase 2 (
- `DeliveryEngine` (claim, send, outcome), `EndpointHealth`
- `WebhooksAlerts` (reflective Alerts SPI)
- `secrets` (AES-GCM, env references, rotation), `processes`, `app` (home, secret widget), `permissions`, `esb`, `support/SystemWrites`.
- `shared`: INTERNAL COPY of the core signer and safe HTTP client (QRun-IO/qqq#1109). Replace with `core.utils.webhooks` / `core.utils.http` at QQQ 4.1.0-RC.5 (DESIGN section 11). Keep it identical to core until then.
- Signing uses `core.utils.webhooks` directly at the QQQ 4.1.0-RC.5 baseline (QRun-IO/qqq#1160). `shared.http` remains an internal copy of the safe HTTP client pending a separate migration to `core.utils.http` (DESIGN section 11).
## Invariants
@@ -52,12 +52,6 @@ Design: `docs/DESIGN.md` (from QRun-IO/qqq#1105). Inbound webhooks are phase 2 (
3-space indent, braces on the next line, `return (x);`, flower-box Javadoc on every class and method, fluent `withX()` setters.
## Temporary (until the owner approves removing them)
- The 0.x sources (`actions/`, `model/`, `tables/`, the old `processes/*`, `registry/*Plugin`) are still in the tree.
- `pom.xml` excludes them from compile, test-compile and Checkstyle.
- Delete them and the excludes together.
## Release
- CircleCI qqq-orb 0.6.8.
+24
View File
@@ -5,6 +5,30 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and
## [Unreleased]
### Fixed
- QBit metadata now reads the Maven artifact version from a filtered resource, keeping snapshot and release versions in sync with the help bundle.
### Added
- Central `webhooks.endpoint.<id>` and `webhooks.dispatcher` dependency checks, every 300 seconds, using persisted endpoint health and outbox age.
- A native home status line linking to the webhooks checks in System Health.
- Complete host-neutral in-app help bundle for Webhooks, with live metadata references and a `HelpBundleAssertions` coverage test including the optional ESB process.
- Foreign keys between all five tables in `docs/ddl/webhooks-postgres.sql`, named `fk_<table>_<column>`: an endpoint's secrets and subscriptions are deleted with it (CASCADE), deliveries and attempts keep their endpoint (RESTRICT: cancel and Delete Deliveries before deleting an endpoint with history), a delivery outlives its subscription (SET NULL) and attempts go with their delivery (CASCADE). New index `webhooks_delivery_subscription_ix`.
- `docs/ddl/webhooks-foreign-keys-upgrade.sql` for existing databases: records orphan rows in `qqq_fk_quarantine` (nulls nullable references, deletes rows with a missing required parent and keeps them as JSON), replaces the NO ACTION endpoint keys, and is idempotent.
- `ForeignKeyGuardrailTest` applies the DDL to PostgreSQL 17 (Testcontainers) and fails when a join or table possible-value field between webhooks tables has no foreign key.
- Every table explains itself: a help bundle (`help/qbit-webhooks.yaml`, QQQ help-bundle format) says what each `webhooks_*` table is for and how its rows get created, including where event types come from. It shows on the table screen and when the table is empty. Registered through QQQ `HelpBundles`, with inline empty-state help retained for hosts without `helpContent`; a host's own help is kept.
- A Setup card on the Webhooks home lists event types, encryption key, endpoints, signing secrets and subscriptions with their state and the next step.
- `WebhooksTablesGuardrailTest` (each table is used at runtime, seeded or explained, and has help) and a test that a previous secret stops signing at its Signs Until date even before the maintenance job retires it.
- Nothing is seeded, deliberately: the tables hold the owner's receivers and secrets, or logs.
### Removed
- The home's separate endpoint-health attention tile, list and diagnostic queries; endpoint state, alerts and Send Test Event remain unchanged.
### Changed
- Pin QQQ to 4.1.0-RC.8 for the released central health API.
- Pin QQQ to 4.1.0-RC.7 and use its help-bundle parser, registry and guarded startup synchronization. Hosts can override bundle text with a `HOST` bundle.
- Require QQQ 4.1.0-RC.5 and use its webhook signer, verifier, secret and verification exception (QRun-IO/qqq#1160, F11). Removed the internal signing package; Java consumers must import `com.kingsrook.qqq.backend.core.utils.webhooks` instead.
- Signing wire format, timestamp tolerance, rotation, endpoint management, encrypted secret storage and delivery behavior are unchanged.
## [1.0.0] - not yet released
A rebuild of the qbit on QQQ 4.1 (design: QRun-IO/qqq#1105, docs/DESIGN.md). Breaking: see docs/migration-0.x-to-1.0.md.
+1 -7
View File
@@ -31,7 +31,7 @@ Design: `docs/DESIGN.md` (from QRun-IO/qqq#1105). Inbound webhooks are phase 2 (
- `DeliveryEngine` (claim, send, outcome), `EndpointHealth`
- `WebhooksAlerts` (reflective Alerts SPI)
- `secrets` (AES-GCM, env references, rotation), `processes`, `app` (home, secret widget), `permissions`, `esb`, `support/SystemWrites`.
- `shared`: INTERNAL COPY of the core signer and safe HTTP client (QRun-IO/qqq#1109). Replace with `core.utils.webhooks` / `core.utils.http` at QQQ 4.1.0-RC.5 (DESIGN section 11). Keep it identical to core until then.
- Signing uses `core.utils.webhooks` directly at the QQQ 4.1.0-RC.5 baseline (QRun-IO/qqq#1160). `shared.http` remains an internal copy of the safe HTTP client pending a separate migration to `core.utils.http` (DESIGN section 11).
## Invariants
@@ -52,12 +52,6 @@ Design: `docs/DESIGN.md` (from QRun-IO/qqq#1105). Inbound webhooks are phase 2 (
3-space indent, braces on the next line, `return (x);`, flower-box Javadoc on every class and method, fluent `withX()` setters.
## Temporary (until the owner approves removing them)
- The 0.x sources (`actions/`, `model/`, `tables/`, the old `processes/*`, `registry/*Plugin`) are still in the tree.
- `pom.xml` excludes them from compile, test-compile and Checkstyle.
- Delete them and the excludes together.
## Release
- CircleCI qqq-orb 0.6.8.
+32 -2
View File
@@ -2,12 +2,12 @@
Signed, SSRF-safe outbound webhooks for QQQ applications. Admins add **endpoints** and **subscribe** them to event types: table inserts, updates and deletes (in-process or from the ESB), process completions and failures, and custom domain events. The qbit queues one **delivery** per event per endpoint in the same transaction as the change. It then signs each delivery in the [Standard Webhooks](https://www.standardwebhooks.com/) format and sends it with retries, backoff, dead letters and endpoint health. Every row is a normal QQQ table, and the app has a home dashboard.
Design: [QRun-IO/qqq#1105](https://github.com/QRun-IO/qqq/pull/1105) (`docs/specs/2026-10-12-qbit-webhooks-assessment.md`), and [docs/DESIGN.md](docs/DESIGN.md) for what 1.0 implements. The shared signer and safe HTTP client are in QQQ core from 4.1.0-RC.5 ([QRun-IO/qqq#1109](https://github.com/QRun-IO/qqq/pull/1109)).
Design: [QRun-IO/qqq#1105](https://github.com/QRun-IO/qqq/pull/1105) (`docs/specs/2026-10-12-qbit-webhooks-assessment.md`), and [docs/DESIGN.md](docs/DESIGN.md) for what 1.0 implements. Signing and secret generation use QQQ core directly; the local safe HTTP client remains pending a separate migration. Both primitives are available in QQQ core from 4.1.0-RC.5 ([QRun-IO/qqq#1109](https://github.com/QRun-IO/qqq/pull/1109)).
## Requirements
- Java 21
- QQQ 4.1.0-RC.4 or later (`qqq-backend-core`). Two modules are optional:
- QQQ 4.1.0-RC.8 or later (`qqq-backend-core`). Two modules are optional:
- `qqq-middleware-api`, for payloads rendered through a QQQ API version.
- `qqq-esb`, for ESB topics as an event source.
- A SQL backend for the five tables (DDL in [docs/ddl/webhooks-postgres.sql](docs/ddl/webhooks-postgres.sql)), and a QQQ scheduler for delivery.
@@ -50,6 +50,20 @@ Set `WEBHOOKS_SECRET_KEY` to the base64 of 32 random bytes, so the qbit can gene
Deliveries and attempts are written only by the qbit. People act on them through the actions below.
## Setup and empty states
Nothing in this qbit is seeded: endpoints, secrets and subscriptions are your choices, and deliveries and attempts are logs. Instead the app home has a **Setup** card (event types, encryption key, endpoints, signing secrets, subscriptions, each with its state and next step), and every table has help (`help/qbit-webhooks.yaml`) that explains how its rows get created and shows when the table is empty.
The event types you can subscribe to come from code, not from a table: the host's `tableEventSources`, `processEventSources` and ESB settings, custom events it fires with `Webhooks.fire`, and a built-in Test event, all held in the `WebhooksRegistry`.
## In-app help
The producer registers [help/qbit-webhooks.yaml](src/main/resources/help/qbit-webhooks.yaml), a `QBIT` bundle covering the app, tables, fields, actions, widgets and possible-value sources. QQQ synchronizes it on startup when the host has the `helpContent` table; registration is safe without that table, and inline app/table empty-state help remains available.
Hosts can register their own YAML bundle with `layer: HOST` through `HelpBundles.register(qInstance, "help/my-host.yaml")`. Use the same entry keys and roles to override selected text with deployment-specific guidance. Keep live `{{label:...}}`, `{{values:...}}` and `{{default:...}}` references where appropriate. If you customize the app name, supply its help under the customized app key in your host bundle.
`WebhooksHelpBundleTest` uses QQQ's `HelpBundleAssertions` against the produced instance, including the optional ESB process, to check coverage, summary length and live references. Run it with `./mvnw -q -Dtest=WebhooksHelpBundleTest test`; the full `./mvnw -q verify` also runs it. Keep the bundle's literal version aligned with the POM revision when releasing.
## Permissions
| Permission | Grants |
@@ -117,6 +131,22 @@ Upgrading from 0.x: see [docs/migration-0.x-to-1.0.md](docs/migration-0.x-to-1.0
./mvnw verify # tests, Checkstyle and the JaCoCo gate (always use the wrapper)
```
## Health checks
The producer registers passive checks with QQQ central health monitoring (requires QQQ 4.1.0-RC.8 or later):
| Name | Kind | Category | Interval |
|---|---|---|---|
| `webhooks.endpoint.<id>` | DEPENDENCY | INTEGRATION | 300 seconds |
| `webhooks.dispatcher` | DEPENDENCY | INTEGRATION | 300 seconds |
Both belong to component `webhooks`. Endpoint checks read the existing state machine: HEALTHY means UP; UNHEALTHY or PROBATION means DEGRADED; automatically disabled means DOWN. Active and automatically disabled endpoints are discovered dynamically; paused and manually disabled endpoints are excluded. Details contain only healthStatus, consecutiveFailures and unhealthySince when present.
The dispatcher reports `oldestPendingSeconds`, measured from the oldest queued delivery's createDate across PENDING, RETRY_WAIT and SENDING. More than 15 minutes is DEGRADED; more than 60 minutes is DOWN. An empty queue reports UP with zero age. Checks do not send requests or change endpoint state.
The home shows one native central status line linking to the webhooks checks in System Health. Disable checks centrally with `health_check.enabled`. The endpoint state machine, alerts and Send Test Event action remain unchanged.
## License
Apache-2.0. See [LICENSE](LICENSE) and [NOTICE](NOTICE).
+5
View File
@@ -0,0 +1,5 @@
# Upgrading
## Unreleased
No action is needed. The qbit adds help to its tables and a Setup card to its app home when it is produced; it adds no tables, columns or data, and a host's own table or app help is kept. Nothing is seeded.
+12 -6
View File
@@ -6,7 +6,7 @@
| # | Decision |
|---|---|
| D1 | Enhance the existing qbit into 1.0; breaking changes allowed |
| D2 | The signer and SSRF-safe client live in `qqq-backend-core` (QRun-IO/qqq#1109). This qbit carries an identical internal copy in `com.kingsrook.qbits.webhooks.shared` until QQQ 4.1.0-RC.5 is published (see section 11) |
| D2 | The signer and SSRF-safe client live in `qqq-backend-core` (QRun-IO/qqq#1109). Signing uses core directly at the QQQ 4.1.0-RC.5 baseline; the internal HTTP copy remains pending a separate migration (see section 11) |
| D3 | Tables renamed to `webhooks_*` |
| D4 | Internal and private targets blocked by default; admin-only allowlist per endpoint |
| D5 | Inbound is phase 2, in this qbit |
@@ -265,10 +265,16 @@ Goal: verify and record signed callbacks from third parties (Authentik, GitHub,
## 11. Moving to the core primitives (QQQ 4.1.0-RC.5)
QRun-IO/qqq#1109 adds the same classes to `qqq-backend-core` (`com.kingsrook.qqq.backend.core.utils.webhooks` and `...utils.http`). When RC.5 is on Maven Central:
QRun-IO/qqq#1160 (F11) migrates signing to the released `qqq-bom-pom` 4.1.0-RC.5 baseline. Delivery signing, secret generation/rotation and receiver verification use `com.kingsrook.qqq.backend.core.utils.webhooks` directly. The local `shared.signing` package is removed; endpoints, encrypted secret storage and delivery remain in this qbit. Inbound routes are still phase 2 (design only).
1. Bump `qqq.snapshot.version` to `4.1.0-RC.5`.
2. Delete `src/main/java/com/kingsrook/qbits/webhooks/shared/` and its tests.
3. Change the imports (`shared.signing` to `core.utils.webhooks`, `shared.http` to `core.utils.http`).
Comparison against core sources at tag `v4.1.0-RC.5` found no executable differences in `WebhookSigner`, `WebhookVerifier`, `WebhookSecret` or `WebhookVerificationException`; only package names, copyright years and migration comments differed. No compatibility adapter is needed:
The behaviour is identical, and the classes were kept the same on purpose.
- Headers remain `webhook-id`, `webhook-timestamp`, `webhook-signature` (case-insensitive when verifying).
- HMAC-SHA256 signs UTF-8 `id.timestamp.` followed by unchanged body bytes; signatures are padded Base64 with `v1,` and a single space between rotation signatures, in secret-list order.
- Secrets remain `whsec_` plus padded Base64 key bytes; unprefixed secrets use trimmed UTF-8 bytes. Generation uses 32 bytes, with a 16-byte minimum for imported keys.
- Default timestamp tolerance remains five minutes in either direction, including the boundary.
- Verification accepts any matching `v1` entry with any known secret and ignores unknown signature versions.
Signing contract tests retain the reference vector and test independent local HMAC signing against core verification and core signing against local verification, including two-secret rotation and exact tolerance boundaries. Delivery integration tests verify actual requests with core.
The `shared.http` copy is outside F11 and remains for a separate migration to `com.kingsrook.qqq.backend.core.utils.http`.
+3
View File
@@ -28,6 +28,9 @@ webhook-attempt: 1
## Java (QQQ core 4.1.0-RC.5 or later)
```java
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerifier;
WebhookVerifier verifier = new WebhookVerifier(List.of(WebhookSecret.parse(System.getenv("WEBHOOK_SECRET"))));
verifier.verify(headersAsMap, rawBodyBytes); // throws WebhookVerificationException with a safe reason
```
@@ -0,0 +1,68 @@
-- qbit-webhooks: add the foreign keys to an existing PostgreSQL schema
-- (one created from webhooks-postgres.sql before they were declared there).
--
-- Safe to run more than once: a foreign key that already exists with the
-- right ON DELETE (under any name) is left alone. Before each key is added,
-- rows that point at a missing parent are recorded in qqq_fk_quarantine and
-- then set to NULL (nullable column) or deleted (NOT NULL column; the whole
-- row is kept in row_data). An existing key on the column with a different
-- ON DELETE (endpoint_secret and subscription -> endpoint were NO ACTION) is
-- replaced by the named one.
--
-- Order matters: deliveries of a missing endpoint are removed before their
-- attempts are checked.
BEGIN;
CREATE TABLE IF NOT EXISTS qqq_fk_quarantine (id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, table_name VARCHAR(100) NOT NULL, column_name VARCHAR(100) NOT NULL, row_id VARCHAR(100), missing_parent_id VARCHAR(100), action VARCHAR(20) NOT NULL, row_data TEXT, changeset VARCHAR(200), quarantined_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP);
CREATE OR REPLACE FUNCTION pg_temp.qqq_add_fk(p_table TEXT, p_column TEXT, p_parent TEXT, p_on_delete TEXT) RETURNS VOID LANGUAGE plpgsql AS
$fn$
DECLARE
v_name TEXT := 'fk_' || p_table || '_' || p_column;
v_rule "char" := CASE p_on_delete WHEN 'CASCADE' THEN 'c' WHEN 'SET NULL' THEN 'n' WHEN 'RESTRICT' THEN 'r' ELSE 'a' END;
v_nullable BOOLEAN;
v_changeset TEXT := 'qbit-webhooks:webhooks-foreign-keys-upgrade.sql';
v_old RECORD;
BEGIN
IF EXISTS (SELECT 1 FROM pg_constraint c JOIN pg_attribute a ON a.attrelid = c.conrelid AND a.attnum = c.conkey[1]
WHERE c.contype = 'f' AND c.conrelid = p_table::regclass AND cardinality(c.conkey) = 1 AND a.attname = p_column
AND c.confrelid = p_parent::regclass AND c.confdeltype = v_rule) THEN
RETURN;
END IF;
FOR v_old IN SELECT c.conname FROM pg_constraint c JOIN pg_attribute a ON a.attrelid = c.conrelid AND a.attnum = c.conkey[1]
WHERE c.contype = 'f' AND c.conrelid = p_table::regclass AND cardinality(c.conkey) = 1 AND a.attname = p_column LOOP
EXECUTE format('ALTER TABLE %I DROP CONSTRAINT %I', p_table, v_old.conname);
END LOOP;
SELECT is_nullable = 'YES' INTO v_nullable FROM information_schema.columns
WHERE table_schema = current_schema() AND table_name = p_table AND column_name = p_column;
IF v_nullable THEN
EXECUTE format('INSERT INTO qqq_fk_quarantine (table_name, column_name, row_id, missing_parent_id, action, changeset) '
|| 'SELECT %L, %L, t.id::TEXT, t.%I::TEXT, ''NULLED'', %L FROM %I t WHERE t.%I IS NOT NULL AND NOT EXISTS (SELECT 1 FROM %I p WHERE p.id = t.%I)',
p_table, p_column, p_column, v_changeset, p_table, p_column, p_parent, p_column);
EXECUTE format('UPDATE %I t SET %I = NULL WHERE t.%I IS NOT NULL AND NOT EXISTS (SELECT 1 FROM %I p WHERE p.id = t.%I)',
p_table, p_column, p_column, p_parent, p_column);
ELSE
EXECUTE format('INSERT INTO qqq_fk_quarantine (table_name, column_name, row_id, missing_parent_id, action, row_data, changeset) '
|| 'SELECT %L, %L, t.id::TEXT, t.%I::TEXT, ''DELETED'', row_to_json(t)::TEXT, %L FROM %I t WHERE NOT EXISTS (SELECT 1 FROM %I p WHERE p.id = t.%I)',
p_table, p_column, p_column, v_changeset, p_table, p_parent, p_column);
EXECUTE format('DELETE FROM %I t WHERE NOT EXISTS (SELECT 1 FROM %I p WHERE p.id = t.%I)', p_table, p_parent, p_column);
END IF;
EXECUTE format('ALTER TABLE %I ADD CONSTRAINT %I FOREIGN KEY (%I) REFERENCES %I (id) ON DELETE %s', p_table, v_name, p_column, p_parent, p_on_delete);
END
$fn$;
SELECT pg_temp.qqq_add_fk('webhooks_endpoint_secret', 'endpoint_id', 'webhooks_endpoint', 'CASCADE');
SELECT pg_temp.qqq_add_fk('webhooks_subscription', 'endpoint_id', 'webhooks_endpoint', 'CASCADE');
SELECT pg_temp.qqq_add_fk('webhooks_delivery', 'endpoint_id', 'webhooks_endpoint', 'RESTRICT');
SELECT pg_temp.qqq_add_fk('webhooks_delivery', 'subscription_id', 'webhooks_subscription', 'SET NULL');
SELECT pg_temp.qqq_add_fk('webhooks_delivery_attempt', 'delivery_id', 'webhooks_delivery', 'CASCADE');
SELECT pg_temp.qqq_add_fk('webhooks_delivery_attempt', 'endpoint_id', 'webhooks_endpoint', 'RESTRICT');
CREATE INDEX IF NOT EXISTS webhooks_delivery_subscription_ix ON webhooks_delivery (subscription_id);
COMMIT;
+20 -6
View File
@@ -7,6 +7,18 @@
-- an event being queued twice, and (delivery_id, attempt_number) on
-- webhooks_delivery_attempt is the claim that stops two replicas sending the
-- same attempt.
--
-- Foreign keys (named fk_<table>_<column>):
-- endpoint_secret.endpoint_id, subscription.endpoint_id -> endpoint CASCADE:
-- an endpoint's secrets and subscriptions are its own configuration, so
-- deleting the endpoint (webhooks.admin) removes them.
-- delivery.endpoint_id, delivery_attempt.endpoint_id -> endpoint RESTRICT:
-- delivery history keeps its endpoint; cancel and Delete Deliveries first.
-- delivery.subscription_id -> subscription SET NULL:
-- a delivery outlives the subscription that queued it.
-- delivery_attempt.delivery_id -> delivery CASCADE:
-- attempts belong to their delivery (Retention deletes both).
-- Existing databases: run webhooks-foreign-keys-upgrade.sql.
CREATE TABLE webhooks_endpoint
(
@@ -40,7 +52,7 @@ CREATE TABLE webhooks_endpoint_secret
id SERIAL PRIMARY KEY,
create_date TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now(),
modify_date TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now(),
endpoint_id INTEGER NOT NULL REFERENCES webhooks_endpoint (id),
endpoint_id INTEGER NOT NULL CONSTRAINT fk_webhooks_endpoint_secret_endpoint_id REFERENCES webhooks_endpoint (id) ON DELETE CASCADE,
status VARCHAR(20) DEFAULT 'ACTIVE',
source VARCHAR(20) NOT NULL,
secret_ref VARCHAR(100),
@@ -59,7 +71,7 @@ CREATE TABLE webhooks_subscription
id SERIAL PRIMARY KEY,
create_date TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now(),
modify_date TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now(),
endpoint_id INTEGER NOT NULL REFERENCES webhooks_endpoint (id),
endpoint_id INTEGER NOT NULL CONSTRAINT fk_webhooks_subscription_endpoint_id REFERENCES webhooks_endpoint (id) ON DELETE CASCADE,
event_type_name VARCHAR(200) NOT NULL,
active_status VARCHAR(20) DEFAULT 'ACTIVE',
description VARCHAR(250),
@@ -83,8 +95,8 @@ CREATE TABLE webhooks_delivery
modify_date TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now(),
event_id VARCHAR(40) NOT NULL,
event_type_name VARCHAR(200) NOT NULL,
endpoint_id INTEGER NOT NULL,
subscription_id INTEGER,
endpoint_id INTEGER NOT NULL CONSTRAINT fk_webhooks_delivery_endpoint_id REFERENCES webhooks_endpoint (id) ON DELETE RESTRICT,
subscription_id INTEGER CONSTRAINT fk_webhooks_delivery_subscription_id REFERENCES webhooks_subscription (id) ON DELETE SET NULL,
source VARCHAR(20),
source_table VARCHAR(100),
source_record_id VARCHAR(100),
@@ -113,14 +125,16 @@ CREATE INDEX webhooks_delivery_status_ix ON webhooks_delivery (status, modify_da
CREATE INDEX webhooks_delivery_delivered_ix ON webhooks_delivery (delivered_date);
CREATE INDEX webhooks_delivery_subscription_ix ON webhooks_delivery (subscription_id);
CREATE TABLE webhooks_delivery_attempt
(
id SERIAL PRIMARY KEY,
create_date TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now(),
modify_date TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now(),
delivery_id INTEGER NOT NULL,
endpoint_id INTEGER NOT NULL,
delivery_id INTEGER NOT NULL CONSTRAINT fk_webhooks_delivery_attempt_delivery_id REFERENCES webhooks_delivery (id) ON DELETE CASCADE,
endpoint_id INTEGER NOT NULL CONSTRAINT fk_webhooks_delivery_attempt_endpoint_id REFERENCES webhooks_endpoint (id) ON DELETE RESTRICT,
attempt_number INTEGER NOT NULL,
outcome VARCHAR(20) NOT NULL,
claimed_by VARCHAR(100),
+41 -29
View File
@@ -22,6 +22,8 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<properties>
<qbit.guardrails.enforce>true</qbit.guardrails.enforce>
<qbit.guardrails.strictUiApiRoutes>true</qbit.guardrails.strictUiApiRoutes>
<revision>1.0.0-SNAPSHOT</revision>
<maven.compiler.source>21</maven.compiler.source>
<maven.compiler.target>21</maven.compiler.target>
@@ -32,7 +34,7 @@
<parent>
<groupId>com.kingsrook</groupId>
<artifactId>qbit-build-parent</artifactId>
<version>2.0.0</version>
<version>2.1.0</version>
</parent>
<groupId>com.kingsrook.qbits</groupId>
@@ -113,6 +115,19 @@
<version>${h2.version}</version>
<scope>test</scope>
</dependency>
<!-- the foreign key guardrail test: docs/ddl on a real PostgreSQL (src/test only) -->
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>42.7.9</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers-postgresql</artifactId>
<version>2.0.3</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
@@ -127,35 +142,34 @@
</dependencies>
<build>
<resources>
<!-- help bundles carry the artifact version (QQQ help bundle provenance) -->
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
<includes>
<include>help/*.yaml</include>
<include>qbit-webhooks.properties</include>
</includes>
</resource>
<resource>
<directory>src/main/resources</directory>
<filtering>false</filtering>
<excludes>
<exclude>help/*.yaml</exclude>
<exclude>qbit-webhooks.properties</exclude>
</excludes>
</resource>
</resources>
<plugins>
<!-- TEMPORARY (1.0 PR): the 0.x sources stay in the tree until their removal is approved; they are not compiled. -->
<!-- the artifact version, for the help bundle version test -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>com/kingsrook/qbits/webhooks/actions/**</exclude>
<exclude>com/kingsrook/qbits/webhooks/model/**</exclude>
<exclude>com/kingsrook/qbits/webhooks/tables/**</exclude>
<exclude>com/kingsrook/qbits/webhooks/registry/WebhooksRegistryEnricherPlugin.java</exclude>
<exclude>com/kingsrook/qbits/webhooks/registry/WebhooksRegistryValidatorPlugin.java</exclude>
<exclude>com/kingsrook/qbits/webhooks/processes/ManageWebhookHealth*.java</exclude>
<exclude>com/kingsrook/qbits/webhooks/processes/SendTestEventToWebhook*.java</exclude>
<exclude>com/kingsrook/qbits/webhooks/processes/SendWebhookEvent*.java</exclude>
<exclude>com/kingsrook/qbits/webhooks/processes/SyncWebhookScheduledJobProcess.java</exclude>
</excludes>
<testExcludes>
<testExclude>com/kingsrook/qbits/webhooks/BaseTest.java</testExclude>
<testExclude>com/kingsrook/qbits/webhooks/WebhooksQBitConfigTest.java</testExclude>
<testExclude>com/kingsrook/qbits/webhooks/WebhooksTestApplication.java</testExclude>
<testExclude>com/kingsrook/qbits/webhooks/WebhooksTestJavalinServer.java</testExclude>
<testExclude>com/kingsrook/qbits/webhooks/WebhooksTestReceiver*.java</testExclude>
<testExclude>com/kingsrook/qbits/webhooks/actions/**</testExclude>
<testExclude>com/kingsrook/qbits/webhooks/processes/ManageWebhookHealthTransformStepTest.java</testExclude>
<testExclude>com/kingsrook/qbits/webhooks/processes/SendTestEventToWebhookLoadStepTest.java</testExclude>
<testExclude>com/kingsrook/qbits/webhooks/processes/SendWebhookEvent*Test.java</testExclude>
<testExclude>com/kingsrook/qbits/webhooks/registry/WebhooksRegistryValidationAndEnrichmentTest.java</testExclude>
</testExcludes>
<systemPropertyVariables combine.children="append">
<qbit.artifactVersion>${project.version}</qbit.artifactVersion>
</systemPropertyVariables>
</configuration>
</plugin>
<plugin>
@@ -178,8 +192,6 @@
<failOnViolation>true</failOnViolation>
<violationSeverity>warning</violationSeverity>
<includeTestSourceDirectory>false</includeTestSourceDirectory>
<!-- TEMPORARY (1.0 PR): the uncompiled 0.x sources, until their removal is approved. -->
<excludes>com/kingsrook/qbits/webhooks/actions/**,com/kingsrook/qbits/webhooks/model/**,com/kingsrook/qbits/webhooks/tables/**,com/kingsrook/qbits/webhooks/registry/WebhooksRegistry*Plugin.java,com/kingsrook/qbits/webhooks/processes/ManageWebhookHealth*.java,com/kingsrook/qbits/webhooks/processes/SendTestEventToWebhook*.java,com/kingsrook/qbits/webhooks/processes/SendWebhookEvent*.java,com/kingsrook/qbits/webhooks/processes/SyncWebhookScheduledJobProcess.java</excludes>
</configuration>
<executions>
<execution>
@@ -247,7 +259,7 @@
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<qqq.snapshot.version>4.1.0-RC.4</qqq.snapshot.version>
<qqq.snapshot.version>4.1.0-RC.8</qqq.snapshot.version>
</properties>
<dependencyManagement>
<dependencies>
@@ -21,13 +21,18 @@
package com.kingsrook.qbits.webhooks;
import java.io.IOException;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Properties;
import com.kingsrook.qbits.webhooks.app.WebhooksApp;
import com.kingsrook.qbits.webhooks.events.WebhooksProcessListener;
import com.kingsrook.qbits.webhooks.events.WebhooksRecordChangeListener;
import com.kingsrook.qbits.webhooks.health.WebhooksHealthChecks;
import com.kingsrook.qbits.webhooks.metadata.WebhooksHelp;
import com.kingsrook.qbits.webhooks.metadata.WebhooksTableCustomizers;
import com.kingsrook.qbits.webhooks.metadata.WebhooksTables;
import com.kingsrook.qbits.webhooks.processes.WebhooksProcesses;
@@ -42,6 +47,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.help.bundles.HelpBundles;
import com.kingsrook.qqq.backend.core.model.metadata.joins.QJoinMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSource;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
@@ -72,13 +78,40 @@ public class WebhooksQBitProducer implements QBitMetaDataProducer<WebhooksQBitCo
{
public static final String GROUP_ID = "com.kingsrook.qbits";
public static final String ARTIFACT_ID = "qbit-webhooks";
public static final String VERSION = "1.0.0";
public static final String VERSION = loadVersion();
private WebhooksQBitConfig qBitConfig;
private String namespace;
/*******************************************************************************
** The artifact version, from the Maven-filtered qbit-webhooks.properties.
*******************************************************************************/
private static String loadVersion()
{
try(InputStream inputStream = WebhooksQBitProducer.class.getClassLoader().getResourceAsStream("qbit-webhooks.properties"))
{
if(inputStream == null)
{
throw (new IllegalStateException("Missing qbit-webhooks.properties; build with Maven to generate the artifact version."));
}
Properties properties = new Properties();
properties.load(inputStream);
String version = properties.getProperty("version");
if(version == null || version.isBlank() || version.contains("${"))
{
throw (new IllegalStateException("Missing or unfiltered version in qbit-webhooks.properties; build with Maven."));
}
return (version);
}
catch(IOException e)
{
throw (new IllegalStateException("Could not read artifact version from qbit-webhooks.properties.", e));
}
}
/*******************************************************************************
** Fluent setter for the config.
*******************************************************************************/
@@ -199,6 +232,8 @@ public class WebhooksQBitProducer implements QBitMetaDataProducer<WebhooksQBitCo
output.add(WebhooksApp.app(qBitConfig, tables));
output.addSelfToInstance(qInstance);
WebhooksHealthChecks.register(qInstance);
WebhooksRegistry registry = WebhooksRegistry.of(qInstance);
registry.registerFromConfig(qInstance, qBitConfig);
registry.register(new WebhookEventType().withName(WebhooksProcesses.TEST_EVENT_TYPE).withKind(WebhookEventKind.CUSTOM)
@@ -209,6 +244,9 @@ public class WebhooksQBitProducer implements QBitMetaDataProducer<WebhooksQBitCo
{
qInstance.withProcessLifecycleListener(new QCodeReference(WebhooksProcessListener.class));
}
// Registration is metadata-only; core guards startup sync when helpContent is absent.
HelpBundles.register(qInstance, "help/qbit-webhooks.yaml");
WebhooksHelp.apply(qInstance, qBitConfig);
return (new MetaDataProducerMultiOutput());
}
@@ -1,60 +0,0 @@
/*
- a
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.actions;
import java.util.List;
import java.util.Optional;
import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** table customizer that clears the memoizations in the WebhookSubscriptionsHelper
*******************************************************************************/
public class ClearWebhookSubscriptionHelperMemoizationsTableCustomizer implements TableCustomizerInterface
{
/***************************************************************************
**
***************************************************************************/
@Override
public List<QRecord> postInsertOrUpdate(AbstractActionInput input, List<QRecord> records, Optional<List<QRecord>> oldRecordList) throws QException
{
WebhookSubscriptionsHelper.clearMemoizations();;
return (records);
}
/***************************************************************************
**
***************************************************************************/
@Override
public List<QRecord> postDelete(DeleteInput deleteInput, List<QRecord> records) throws QException
{
WebhookSubscriptionsHelper.clearMemoizations();;
return (records);
}
}
@@ -1,31 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.actions;
/*******************************************************************************
** Default implementation of WebhookEventTypeCustomizerInterface - just lets
** all the default methods from the interface run.
*******************************************************************************/
public class DefaultWebhookEventTypeCustomizer implements WebhookEventTypeCustomizerInterface
{
}
@@ -1,89 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.actions;
import java.util.List;
import com.kingsrook.qbits.webhooks.model.WebhookEventCategory;
import com.kingsrook.qbits.webhooks.registry.WebhookEventType;
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
/*******************************************************************************
* fire an ad-hoc webhook event - e.g., one not tied to a basic table insert/
* update/delete
*******************************************************************************/
public class FireAdHocWebhookEvent
{
private static final QLogger LOG = QLogger.getLogger(FireAdHocWebhookEvent.class);
/***************************************************************************
* fire the event - creating webhookEvent records for any non-disabled
* subscriptions that match it.
*
* @param eventTypeName must be a registered event type name in the instance.
* @param tableName table that the key-record is from, which will be used as
* part of the subscription-matching
* @param keyRecord record that the event is related to - again may be used
* in subscription-matching (e.g., based on security fields
* @param transaction in case an update to the key record was made on a
* transaction, the events will be created in the same one.
***************************************************************************/
public void fire(String eventTypeName, String tableName, QRecord keyRecord, QBackendTransaction transaction) throws QException
{
List<WebhookEventType> webhookEventTypes = WebhookSubscriptionsHelper.getWebhookEventTypesToConsiderFiringEventsFor(WebhookEventCategory.Kind.AD_HOC, tableName);
if(CollectionUtils.nullSafeIsEmpty(webhookEventTypes))
{
return;
}
WebhookEventBuilder webhookEventBuilder = null;
for(WebhookEventType webhookEventType : webhookEventTypes)
{
//////////////////////////////
// filter by the event type //
//////////////////////////////
if(!webhookEventType.getName().equals(eventTypeName))
{
continue;
}
if(keyRecord.getTableName() == null)
{
keyRecord.setTableName(tableName);
}
webhookEventBuilder = FirePostInsertOrUpdateWebhookEventUtil.processSubscriptions(tableName, webhookEventType, keyRecord, webhookEventBuilder, transaction);
}
if(webhookEventBuilder != null)
{
webhookEventBuilder.storeWebhookEvents(transaction);
}
}
}
@@ -1,138 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.actions;
import java.io.Serializable;
import java.util.Collection;
import java.util.List;
import java.util.Objects;
import com.kingsrook.qbits.webhooks.model.WebhookSubscription;
import com.kingsrook.qbits.webhooks.registry.WebhookEventType;
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
import com.kingsrook.qqq.backend.core.utils.StringUtils;
import com.kingsrook.qqq.backend.core.utils.ValueUtils;
import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
/*******************************************************************************
** utility for the fire post insert/update events
*******************************************************************************/
public abstract class FirePostInsertOrUpdateWebhookEventUtil
{
private static final QLogger LOG = QLogger.getLogger(FirePostInsertOrUpdateWebhookEventUtil.class);
/***************************************************************************
**
***************************************************************************/
public static WebhookEventBuilder processSubscriptions(String tableName, WebhookEventType webhookEventType, QRecord record, WebhookEventBuilder webhookEventBuilder, QBackendTransaction transaction)
{
List<WebhookSubscription> webhookSubscriptions = CollectionUtils.nonNullList(WebhookSubscriptionsHelper.getSubscriptionsForEventType(webhookEventType));
for(WebhookSubscription webhookSubscription : webhookSubscriptions)
{
try
{
if(WebhookSubscriptionsHelper.doesRecordMatchSubscription(webhookEventType, webhookSubscription, record, transaction))
{
LOG.info("Building webhook event", logPair("tableName", tableName), logPair("recordId", record.getValue("id")), logPair("webhookEventType", () -> webhookEventType.getName()), logPair("webhookSubscriptionId", webhookSubscription.getId()));
webhookEventBuilder = Objects.requireNonNullElseGet(webhookEventBuilder, WebhookEventBuilder::new);
webhookEventBuilder.addWebhookEvent(webhookSubscription, record);
}
}
catch(QException e)
{
LOG.warn("Error processing webhookSubscription", e, logPair("tableName", tableName), logPair("recordId", record.getValue("id")), logPair("webhookEventType", () -> webhookEventType.getName()), logPair("webhookSubscriptionId", webhookSubscription.getId()));
}
}
return webhookEventBuilder;
}
/***************************************************************************
**
***************************************************************************/
public static QFieldMetaData getQFieldMetaData(WebhookEventType webhookEventType)
{
QFieldMetaData field = null;
if(StringUtils.hasContent(webhookEventType.getFieldName()))
{
field = QContext.getQInstance().getTable(webhookEventType.getTableName()).getFields().get(webhookEventType.getFieldName());
if(field == null)
{
LOG.warn("No field found for webhook event type", logPair("webhookEventType", webhookEventType), logPair("tableName", webhookEventType.getTableName()), logPair("fieldName", webhookEventType.getFieldName()));
}
}
return field;
}
/***************************************************************************
*
***************************************************************************/
public static Serializable getValue(QRecord record, QFieldMetaData field)
{
if(field == null)
{
return (null);
}
return ValueUtils.getValueAsFieldType(field.getType(), record.getValue(field.getName()));
}
/***************************************************************************
**
***************************************************************************/
public static boolean doesNewValueMatchEventValue(WebhookEventType webhookEventType, QFieldMetaData field, Serializable newValue)
{
Serializable eventValue = webhookEventType.getValue();
if(eventValue instanceof Collection<?> collection)
{
for(Object o : collection)
{
Serializable eventSubValueAsFieldType = ValueUtils.getValueAsFieldType(field.getType(), o);
if(Objects.equals(eventSubValueAsFieldType, newValue))
{
return true;
}
}
}
else
{
Serializable eventValueAsFieldType = ValueUtils.getValueAsFieldType(field.getType(), eventValue);
if(Objects.equals(eventValueAsFieldType, newValue))
{
return true;
}
}
return false;
}
}
@@ -1,143 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.actions;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.List;
import com.kingsrook.qbits.webhooks.model.WebhookEventCategory;
import com.kingsrook.qbits.webhooks.model.WebhooksActionFlags;
import com.kingsrook.qbits.webhooks.registry.WebhookEventType;
import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
import com.kingsrook.qqq.backend.core.utils.ValueUtils;
import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
/*******************************************************************************
** post-insert table customizer to fire webhook events
*******************************************************************************/
public class FirePostInsertWebhookEvent implements TableCustomizerInterface
{
private static final QLogger LOG = QLogger.getLogger(FirePostInsertWebhookEvent.class);
/***************************************************************************
**
***************************************************************************/
@Override
public List<QRecord> postInsert(InsertInput insertInput, List<QRecord> records) throws QException
{
if(insertInput.hasFlag(WebhooksActionFlags.OMIT_WEBHOOKS))
{
LOG.debug("Requested to omit webhooks after inert; returning early.", logPair("tableName", insertInput.getTableName()));
return (records);
}
List<WebhookEventType> insertWebhookEventTypes = WebhookSubscriptionsHelper.getWebhookEventTypesToConsiderFiringEventsFor(WebhookEventCategory.Kind.INSERT, insertInput.getTableName());
List<WebhookEventType> storeWebhookEventTypes = WebhookSubscriptionsHelper.getWebhookEventTypesToConsiderFiringEventsFor(WebhookEventCategory.Kind.STORE, insertInput.getTableName());
if(CollectionUtils.nullSafeIsEmpty(insertWebhookEventTypes) && CollectionUtils.nullSafeIsEmpty(storeWebhookEventTypes))
{
return (records);
}
List<WebhookEventType> webhookEventTypes = new ArrayList<>();
CollectionUtils.addAllIfNotNull(webhookEventTypes, insertWebhookEventTypes);
CollectionUtils.addAllIfNotNull(webhookEventTypes, storeWebhookEventTypes);
WebhookEventBuilder webhookEventBuilder = null;
for(WebhookEventType webhookEventType : webhookEventTypes)
{
QFieldMetaData field = FirePostInsertOrUpdateWebhookEventUtil.getQFieldMetaData(webhookEventType);
for(QRecord record : records)
{
if(CollectionUtils.nullSafeHasContents(record.getErrors()))
{
continue;
}
if(record.getTableName() == null)
{
record.setTableName(insertInput.getTableName());
}
if(doesRecordMatchWebhookEventType(record, webhookEventType, field))
{
webhookEventBuilder = FirePostInsertOrUpdateWebhookEventUtil.processSubscriptions(insertInput.getTableName(), webhookEventType, record, webhookEventBuilder, insertInput.getTransaction());
}
}
}
if(webhookEventBuilder != null)
{
webhookEventBuilder.storeWebhookEvents(insertInput.getTransaction());
}
return records;
}
/***************************************************************************
**
***************************************************************************/
boolean doesRecordMatchWebhookEventType(QRecord record, WebhookEventType webhookEventType, QFieldMetaData field)
{
switch(webhookEventType.getCategory())
{
case INSERT, STORE ->
{
return (true);
}
case INSERT_WITH_FIELD, STORE_WITH_FIELD ->
{
Serializable value = FirePostInsertOrUpdateWebhookEventUtil.getValue(record, field);
if(value != null && !"".equals(ValueUtils.getValueAsString(value)))
{
return (true);
}
}
case INSERT_WITH_VALUE, STORE_WITH_VALUE ->
{
Serializable value = FirePostInsertOrUpdateWebhookEventUtil.getValue(record, field);
if(FirePostInsertOrUpdateWebhookEventUtil.doesNewValueMatchEventValue(webhookEventType, field, value))
{
return (true);
}
}
default ->
{
return (false);
}
}
return (false);
}
}
@@ -1,177 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.actions;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import com.kingsrook.qbits.webhooks.model.WebhookEventCategory;
import com.kingsrook.qbits.webhooks.model.WebhooksActionFlags;
import com.kingsrook.qbits.webhooks.registry.WebhookEventType;
import com.kingsrook.qqq.backend.core.actions.customizers.OldRecordHelper;
import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
/*******************************************************************************
** post-update table customizer to fire webhook events
*******************************************************************************/
public class FirePostUpdateWebhookEvent implements TableCustomizerInterface
{
private static final QLogger LOG = QLogger.getLogger(FirePostUpdateWebhookEvent.class);
/***************************************************************************
**
***************************************************************************/
@Override
public List<QRecord> postUpdate(UpdateInput updateInput, List<QRecord> records, Optional<List<QRecord>> oldRecordList) throws QException
{
if(updateInput.hasFlag(WebhooksActionFlags.OMIT_WEBHOOKS))
{
LOG.debug("Requested to omit webhooks after update; returning early.", logPair("tableName", updateInput.getTableName()));
return (records);
}
List<WebhookEventType> updateWebhookEventTypes = WebhookSubscriptionsHelper.getWebhookEventTypesToConsiderFiringEventsFor(WebhookEventCategory.Kind.UPDATE, updateInput.getTableName());
List<WebhookEventType> storeWebhookEventTypes = WebhookSubscriptionsHelper.getWebhookEventTypesToConsiderFiringEventsFor(WebhookEventCategory.Kind.STORE, updateInput.getTableName());
if(CollectionUtils.nullSafeIsEmpty(updateWebhookEventTypes) && CollectionUtils.nullSafeIsEmpty(storeWebhookEventTypes))
{
return (records);
}
List<WebhookEventType> webhookEventTypes = new ArrayList<>();
CollectionUtils.addAllIfNotNull(webhookEventTypes, updateWebhookEventTypes);
CollectionUtils.addAllIfNotNull(webhookEventTypes, storeWebhookEventTypes);
OldRecordHelper oldRecordHelper = new OldRecordHelper(updateInput.getTableName(), oldRecordList);
WebhookEventBuilder webhookEventBuilder = null;
for(WebhookEventType webhookEventType : webhookEventTypes)
{
QFieldMetaData field = FirePostInsertOrUpdateWebhookEventUtil.getQFieldMetaData(webhookEventType);
for(QRecord record : records)
{
if(CollectionUtils.nullSafeHasContents(record.getErrors()))
{
continue;
}
if(record.getTableName() == null)
{
record.setTableName(updateInput.getTableName());
}
Optional<QRecord> oldRecord = oldRecordHelper.getOldRecord(record);
if(doesRecordMatchWebhookEventType(record, oldRecord, webhookEventType, field))
{
/////////////////////////////////////////////////////////////////////////////////////////
// in case the input record is "sparse" (e.g., missing some field values), and we have //
// the old record, then make a hybrid of the new + old for processing. //
/////////////////////////////////////////////////////////////////////////////////////////
QRecord recordToProcess = record;
if(oldRecord.isPresent())
{
recordToProcess = new QRecord(record);
for(Map.Entry<String, Serializable> entry : oldRecord.get().getValues().entrySet())
{
if(!recordToProcess.getValues().containsKey(entry.getKey()))
{
recordToProcess.getValues().put(entry.getKey(), entry.getValue());
}
}
}
webhookEventBuilder = FirePostInsertOrUpdateWebhookEventUtil.processSubscriptions(updateInput.getTableName(), webhookEventType, recordToProcess, webhookEventBuilder, updateInput.getTransaction());
}
}
}
if(webhookEventBuilder != null)
{
webhookEventBuilder.storeWebhookEvents(updateInput.getTransaction());
}
return records;
}
/***************************************************************************
**
***************************************************************************/
boolean doesRecordMatchWebhookEventType(QRecord record, Optional<QRecord> optionalOldRecord, WebhookEventType webhookEventType, QFieldMetaData field)
{
switch(webhookEventType.getCategory())
{
case UPDATE, STORE ->
{
return (true);
}
case UPDATE_WITH_FIELD, STORE_WITH_FIELD ->
{
if(optionalOldRecord.isEmpty())
{
return (false);
}
Serializable newValue = FirePostInsertOrUpdateWebhookEventUtil.getValue(record, field);
Serializable oldValue = FirePostInsertOrUpdateWebhookEventUtil.getValue(optionalOldRecord.get(), field);
if(!Objects.equals(newValue, oldValue))
{
return (true);
}
}
case UPDATE_WITH_VALUE, STORE_WITH_VALUE ->
{
if(optionalOldRecord.isEmpty())
{
return (false);
}
Serializable newValue = FirePostInsertOrUpdateWebhookEventUtil.getValue(record, field);
Serializable oldValue = FirePostInsertOrUpdateWebhookEventUtil.getValue(optionalOldRecord.get(), field);
if(!Objects.equals(newValue, oldValue) && FirePostInsertOrUpdateWebhookEventUtil.doesNewValueMatchEventValue(webhookEventType, field, newValue))
{
return (true);
}
}
default ->
{
return (false);
}
}
return (false);
}
}
@@ -1,152 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.actions;
import java.util.ArrayList;
import java.util.List;
import com.kingsrook.qbits.webhooks.WebhooksQBitConfig;
import com.kingsrook.qbits.webhooks.model.Webhook;
import com.kingsrook.qbits.webhooks.model.WebhookEvent;
import com.kingsrook.qbits.webhooks.model.WebhookEventContent;
import com.kingsrook.qbits.webhooks.model.WebhookEventStatus;
import com.kingsrook.qbits.webhooks.model.WebhookSubscription;
import com.kingsrook.qbits.webhooks.registry.WebhookEventType;
import com.kingsrook.qbits.webhooks.registry.WebhooksRegistry;
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
import com.kingsrook.qqq.backend.core.actions.customizers.QCodeLoader;
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitConfig;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import com.kingsrook.qqq.backend.core.model.tables.QQQTableTableManager;
import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
import com.kingsrook.qqq.backend.core.utils.ValueUtils;
import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
/*******************************************************************************
** class that build webhook events
*******************************************************************************/
public class WebhookEventBuilder
{
private static final QLogger LOG = QLogger.getLogger(WebhookEventBuilder.class);
private List<QRecord> webhookEvents = null;
/***************************************************************************
**
***************************************************************************/
public void addWebhookEvent(WebhookSubscription subscription, QRecord sourceRecord)
{
try
{
QTableMetaData table = QContext.getQInstance().getTable(sourceRecord.getTableName());
Integer sourceRecordId = ValueUtils.getValueAsInteger(sourceRecord.getValue(table.getPrimaryKeyField()));
Integer sourceTableId = QQQTableTableManager.getQQQTableId(QContext.getQInstance(), table.getName());
WebhookEvent webhookEvent = new WebhookEvent();
webhookEvent.setWebhookId(subscription.getWebhookId());
webhookEvent.setWebhookSubscriptionId(subscription.getId());
webhookEvent.setWebhookEventTypeName(subscription.getWebhookEventTypeName());
webhookEvent.setEventStatusId(WebhookEventStatus.NEW.getId());
webhookEvent.setEventSourceRecordId(sourceRecordId);
webhookEvent.setEventSourceRecordQqqTableId(sourceTableId);
webhookEvent.setContent(List.of(buildEventContent(sourceRecord, subscription.getWebhookEventTypeName(), subscription.getApiName(), subscription.getApiVersion())));
QRecord eventRecord = webhookEvent.toQRecord();
////////////////////////////////////////////////////////////////////////////
// copy security field values from the sourceRecord into the event record //
////////////////////////////////////////////////////////////////////////////
QTableMetaData webhookTable = QContext.getQInstance().getTable(Webhook.TABLE_NAME);
QBitConfig qbitConfig = webhookTable.getSourceQBitConfig();
if(qbitConfig instanceof WebhooksQBitConfig webhooksQBitConfig)
{
for(QFieldMetaData field : CollectionUtils.nonNullList(webhooksQBitConfig.getSecurityFields()))
{
//////////////////////////////////////////////////////////////////////////////////////
// todo - do we need to be more robust re: the field name within the source record? //
//////////////////////////////////////////////////////////////////////////////////////
eventRecord.setValue(field.getName(), sourceRecord.getValue(field.getName()));
}
}
if(webhookEvents == null)
{
webhookEvents = new ArrayList<>();
}
webhookEvents.add(eventRecord);
}
catch(Exception e)
{
LOG.warn("Error while creating webhook event", e, logPair("webhookSubscriptionId", () -> subscription.getId()), logPair("sourceRecord", sourceRecord));
}
}
/***************************************************************************
**
***************************************************************************/
private static WebhookEventContent buildEventContent(QRecord sourceRecord, String webhookEventTypeName, String apiName, String apiVersion) throws QException
{
WebhookEventType webhookEventType = WebhooksRegistry.ofOrWithNew(QContext.getQInstance()).getWebhookEventType(webhookEventTypeName);
WebhookEventTypeCustomizerInterface webhookEventTypeCustomizerInterface;
if(webhookEventType != null && webhookEventType.getCustomizer() != null)
{
webhookEventTypeCustomizerInterface = QCodeLoader.getAdHoc(WebhookEventTypeCustomizerInterface.class, webhookEventType.getCustomizer());
}
else
{
webhookEventTypeCustomizerInterface = new DefaultWebhookEventTypeCustomizer();
}
return webhookEventTypeCustomizerInterface.buildEventContent(sourceRecord, webhookEventTypeName, apiName, apiVersion);
}
/***************************************************************************
**
***************************************************************************/
public void storeWebhookEvents(QBackendTransaction transaction) throws QException
{
if(webhookEvents != null)
{
new InsertAction().execute(new InsertInput(WebhookEvent.TABLE_NAME)
.withRecords(webhookEvents)
.withTransaction(transaction));
}
}
}
@@ -1,417 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.actions;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.time.Instant;
import java.time.temporal.ChronoUnit;
import java.util.List;
import java.util.concurrent.TimeUnit;
import com.kingsrook.qbits.webhooks.WebhooksQBitConfig;
import com.kingsrook.qbits.webhooks.model.Webhook;
import com.kingsrook.qbits.webhooks.model.WebhookEvent;
import com.kingsrook.qbits.webhooks.model.WebhookEventContent;
import com.kingsrook.qbits.webhooks.model.WebhookEventSendLog;
import com.kingsrook.qbits.webhooks.model.WebhookEventStatus;
import com.kingsrook.qbits.webhooks.model.WebhookHealthStatus;
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
import com.kingsrook.qqq.backend.core.actions.audits.AuditAction;
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
import com.kingsrook.qqq.backend.core.utils.SleepUtils;
import com.kingsrook.qqq.backend.core.utils.StringUtils;
import org.apache.http.HttpStatus;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.util.EntityUtils;
import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
/*******************************************************************************
** class that send webhook events to a webhook
*******************************************************************************/
public class WebhookEventSender
{
private static final QLogger LOG = QLogger.getLogger(WebhookEventSender.class);
private static Integer maxAllowedAttempts = null;
private static Integer minutesToConsiderLeakedSendAttempt = null;
private static Integer[] backoffMinutes = null;
private static Integer maxAllowedRateLimitErrors = null;
private static Integer initialRateLimitBackoffMillis = null;
/***************************************************************************
**
***************************************************************************/
public boolean handleEvent(WebhookEvent webhookEvent, Webhook webhook, List<WebhookEventSendLog> sendLogs) throws QException
{
/////////////////////
// mark as sending //
/////////////////////
Instant nextAttemptTimestamp = Instant.now().plus(getMinutesToConsiderLeakedSendAttempt() * 60, ChronoUnit.SECONDS);
updateWebhookEvent(webhookEvent.getId(), WebhookEventStatus.SENDING, nextAttemptTimestamp, null);
/////////////////
// try to send //
/////////////////
WebhookEventSendLog sendLog = post(webhookEvent, webhook);
sendLog.setAttemptNo(sendLogs.size() + 1);
InsertInput insertSendLogInput = new InsertInput(WebhookEventSendLog.TABLE_NAME).withRecordEntity(sendLog);
try(QBackendTransaction transaction = QBackendTransaction.openFor(insertSendLogInput))
{
insertSendLogInput.setTransaction(transaction);
if(sendLog.getSuccessful())
{
///////////////////////
// mark as delivered //
///////////////////////
new InsertAction().execute(insertSendLogInput);
updateWebhookEvent(webhookEvent.getId(), WebhookEventStatus.DELIVERED, null, transaction);
transaction.commit();
return (true);
}
else
{
//////////////////////////////////////////////////////////////////////////////////////////
// consider if this event is now failed, or if it is retryable (based on # of failures) //
//////////////////////////////////////////////////////////////////////////////////////////
WebhookEventStatus eventStatus = sendLog.getAttemptNo() >= getMaxAllowedAttempts() ? WebhookEventStatus.FAILED : WebhookEventStatus.AWAITING_RETRY;
if(eventStatus.equals(WebhookEventStatus.FAILED) && WebhookHealthStatus.PROBATION.getId().equals(webhook.getHealthStatusId()))
{
String message = "This Webhook Event has had too many failures, but since its Webhook's health status is Probation, its status will be Awaiting Retry instead of Failure.";
AuditAction.execute(WebhookEvent.TABLE_NAME, webhookEvent.getId(), null, message);
eventStatus = WebhookEventStatus.AWAITING_RETRY;
}
///////////////////////////////////////////////////////////////////////
// set the next-send time based on how many attempts there have been //
///////////////////////////////////////////////////////////////////////
nextAttemptTimestamp = calculateNextAttemptBackoff(sendLog.getAttemptNo());
if(eventStatus.equals(WebhookEventStatus.FAILED))
{
nextAttemptTimestamp = null;
}
new InsertAction().execute(insertSendLogInput);
updateWebhookEvent(webhookEvent.getId(), eventStatus, nextAttemptTimestamp, transaction);
transaction.commit();
return (false);
}
}
}
/***************************************************************************
**
***************************************************************************/
public static Instant calculateNextAttemptBackoff(int attemptNumber)
{
Integer[] backoffMinutes = getBackoffMinutes();
int index = Math.min(attemptNumber - 1, backoffMinutes.length - 1);
return Instant.now().plusSeconds(backoffMinutes[index] * 60);
}
/***************************************************************************
**
***************************************************************************/
public WebhookEventSendLog post(WebhookEvent webhookEvent, Webhook webhook)
{
WebhookEventSendLog sendLog = new WebhookEventSendLog();
sendLog.setWebhookEventId(webhookEvent.getId());
sendLog.setWebhookId(webhook.getId());
sendLog.setStartTimestamp(Instant.now());
try
{
doPost(webhookEvent, webhook, sendLog);
//////////////////////////////////////////////////
// interpret no exception as meaning successful //
//////////////////////////////////////////////////
sendLog.setSuccessful(true);
}
catch(WebhookPostException e)
{
///////////////////////////////////////////////////////////////////////////
// any exception thrown means not successful - capture its error message //
///////////////////////////////////////////////////////////////////////////
sendLog.setSuccessful(false);
sendLog.setErrorMessage(e.getMessage());
}
sendLog.setEndTimestamp(Instant.now());
return (sendLog);
}
/***************************************************************************
**
***************************************************************************/
protected void doPost(WebhookEvent webhookEvent, Webhook webhook, WebhookEventSendLog sendLog) throws WebhookPostException
{
int rateLimitsCaught = 0;
int rateLimitSleepMillis = getInitialRateLimitBackoffMillis();
///////////////////////////////////
// loop in case of rate limiting //
///////////////////////////////////
while(true)
{
try(CloseableHttpClient httpClient = buildHttpClient())
{
HttpPost request = new HttpPost(webhook.getUrl());
request.setHeader("Content-Type", "application/json");
String postBody = getPostBody(webhookEvent);
request.setEntity(new StringEntity(postBody, StandardCharsets.UTF_8));
try(CloseableHttpResponse response = executeHttpRequest(httpClient, request))
{
int statusCode = response.getStatusLine().getStatusCode();
String responseString = EntityUtils.toString(response.getEntity());
if(statusCode == HttpStatus.SC_TOO_MANY_REQUESTS) // 429
{
throw (new WebhookRateLimitException(responseString));
}
sendLog.setHttpStatusCode(statusCode);
if(statusCode < 200 || statusCode >= 300)
{
if(!StringUtils.hasContent(responseString))
{
LOG.warn("Unsuccessful http status code, but no response body returned", logPair("statusCode", statusCode), logPair("webhookId", webhook.getId()));
responseString = "No response body returned";
}
throw new WebhookPostException(responseString);
}
////////////////////////////////////////////
// successful exit of the rate-limit loop //
////////////////////////////////////////////
break;
}
catch(WebhookRateLimitException rle)
{
rateLimitsCaught++;
if(rateLimitsCaught > getMaxAllowedRateLimitErrors())
{
sendLog.setHttpStatusCode(HttpStatus.SC_TOO_MANY_REQUESTS); // 429
LOG.warn("Giving up webhook post after too many rate limit errors", logPair("webhookId", webhook.getId()), logPair("rateLimitsCaught", rateLimitsCaught), logPair("response", rle.getMessage()));
throw (new WebhookPostException("Giving up after too many rate-limit errors (" + getMaxAllowedRateLimitErrors() + "). Latest response: " + rle.getMessage()));
}
LOG.info("Caught rate limit. Will sleep and re-try", logPair("rateLimitsCaught", rateLimitsCaught), logPair("webhookId", webhook.getId()), logPair("sleeping", rateLimitSleepMillis));
SleepUtils.sleep(rateLimitSleepMillis, TimeUnit.MILLISECONDS);
rateLimitSleepMillis *= 2;
}
catch(WebhookPostException wpe)
{
throw (wpe);
}
catch(Exception e)
{
LOG.warn("Exception executing http request", e, logPair("webhookId", webhook.getId()));
throw (new WebhookPostException(e.getMessage(), e));
}
}
catch(WebhookPostException wpe)
{
throw (wpe);
}
catch(Exception e)
{
LOG.warn("Exception building http client", e, logPair("webhookId", webhook.getId()));
throw (new WebhookPostException(e.getMessage(), e));
}
}
}
/***************************************************************************
**
***************************************************************************/
protected String getPostBody(WebhookEvent webhookEvent) throws QException
{
String postBody = null;
if(CollectionUtils.nullSafeIsEmpty(webhookEvent.getContent()))
{
List<QRecord> contentRecords = QueryAction.execute(WebhookEventContent.TABLE_NAME, new QQueryFilter(new QFilterCriteria("webhookEventId", QCriteriaOperator.EQUALS, webhookEvent.getId())));
if(CollectionUtils.nullSafeHasContents(contentRecords))
{
postBody = contentRecords.get(0).getValueString("postBody");
}
}
else
{
postBody = webhookEvent.getContent().get(0).getPostBody();
}
if(!StringUtils.hasContent(postBody))
{
throw (new QException("Missing content/postBody for webhook event: " + webhookEvent.getId()));
}
return postBody;
}
/***************************************************************************
**
***************************************************************************/
protected CloseableHttpResponse executeHttpRequest(CloseableHttpClient httpClient, HttpPost request) throws IOException
{
return httpClient.execute(request);
}
/*******************************************************************************
** Build the default HttpClient used by the makeRequest method
*******************************************************************************/
protected CloseableHttpClient buildHttpClient()
{
///////////////////////////////////////////////////////////////////////////////////////
// do we want this?? .setConnectionManager(new PoolingHttpClientConnectionManager()) //
// needs some good scrutiny. //
///////////////////////////////////////////////////////////////////////////////////////
return HttpClientBuilder.create()
.setDefaultRequestConfig(RequestConfig.custom()
.setConnectTimeout(5000)
.setConnectionRequestTimeout(5000)
.setSocketTimeout(5000).build())
.build();
}
/***************************************************************************
**
***************************************************************************/
private void updateWebhookEvent(Integer id, WebhookEventStatus status, Instant nextAttemptTimestamp, QBackendTransaction transaction) throws QException
{
new UpdateAction().execute(new UpdateInput(WebhookEvent.TABLE_NAME)
.withTransaction(transaction)
.withRecord(new QRecord()
.withValue("id", id)
.withValue("nextAttemptTimestamp", nextAttemptTimestamp)
.withValue("eventStatusId", status.getId())));
}
/*******************************************************************************
**
*******************************************************************************/
protected int getMaxAllowedRateLimitErrors()
{
if(maxAllowedRateLimitErrors == null)
{
maxAllowedRateLimitErrors = WebhooksQBitConfig.getConfigValue(config -> config.getMaxRateLimitRetries());
}
return (maxAllowedRateLimitErrors);
}
/*******************************************************************************
**
*******************************************************************************/
protected int getInitialRateLimitBackoffMillis()
{
if(initialRateLimitBackoffMillis == null)
{
initialRateLimitBackoffMillis = WebhooksQBitConfig.getConfigValue(config -> config.getInitialRateLimitBackoffMillis());
}
return (initialRateLimitBackoffMillis);
}
/***************************************************************************
*
***************************************************************************/
private static Integer getMaxAllowedAttempts()
{
if(maxAllowedAttempts == null)
{
maxAllowedAttempts = WebhooksQBitConfig.getConfigValue(config -> config.getMaxSendAttemptsBeforeFailure());
}
return (maxAllowedAttempts);
}
/***************************************************************************
*
***************************************************************************/
private static Integer getMinutesToConsiderLeakedSendAttempt()
{
if(minutesToConsiderLeakedSendAttempt == null)
{
minutesToConsiderLeakedSendAttempt = WebhooksQBitConfig.getConfigValue(config -> config.getMinutesToConsiderLeakedSendAttempt());
}
return (minutesToConsiderLeakedSendAttempt);
}
/***************************************************************************
*
***************************************************************************/
private static Integer[] getBackoffMinutes()
{
if(backoffMinutes == null)
{
List<Integer> configValue = WebhooksQBitConfig.getConfigValue(config -> config.getMinutesBetweenRetryAttempts());
backoffMinutes = configValue.toArray(new Integer[0]);
}
return (backoffMinutes);
}
}
@@ -1,92 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.actions;
import java.io.Serializable;
import java.time.Instant;
import java.util.LinkedHashMap;
import java.util.Map;
import com.fasterxml.jackson.annotation.JsonInclude;
import com.kingsrook.qbits.webhooks.model.WebhookEventContent;
import com.kingsrook.qbits.webhooks.registry.WebhookEventType;
import com.kingsrook.qbits.webhooks.registry.WebhooksRegistry;
import com.kingsrook.qqq.api.actions.QRecordApiAdapter;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.utils.JsonUtils;
import com.kingsrook.qqq.backend.core.utils.StringUtils;
/*******************************************************************************
** interface for doing work for a webhookEventType - where specific types can
** customize behavior.
*******************************************************************************/
public interface WebhookEventTypeCustomizerInterface
{
/***************************************************************************
**
***************************************************************************/
default WebhookEventContent buildEventContent(QRecord sourceRecord, String webhookEventTypeName, String apiName, String apiVersion) throws QException
{
WebhookEventType webhookEventType = WebhooksRegistry.of(QContext.getQInstance()).getWebhookEventType(webhookEventTypeName);
Map<String, Object> postBody = new LinkedHashMap<>();
Map<String, Object> webhookEventDetails = new LinkedHashMap<>();
postBody.put("webhookEventDetails", webhookEventDetails);
webhookEventDetails.put("webhookEventTypeName", webhookEventTypeName);
webhookEventDetails.put("eventTimestamp", Instant.now().toString());
if(webhookEventType != null && StringUtils.hasContent(webhookEventType.getTableName()))
{
webhookEventDetails.put("tableName", webhookEventType.getTableName());
}
webhookEventDetails.put("apiName", apiName);
webhookEventDetails.put("apiVersion", apiVersion);
Map<String, Serializable> apiRecord = QRecordApiAdapter.qRecordToApiMap(sourceRecord, sourceRecord.getTableName(), apiName, apiVersion);
postBody.put("record", apiRecord);
customizeEventContent(sourceRecord, webhookEventTypeName, apiName, apiVersion, postBody);
String json = JsonUtils.toJsonCustomized(postBody, builder ->
builder.serializationInclusion(JsonInclude.Include.ALWAYS));
return new WebhookEventContent().withPostBody(json);
}
/***************************************************************************
**
***************************************************************************/
default void customizeEventContent(QRecord sourceRecord, String webhookEventTypeName, String apiName, String apiVersion, Map<String, Object> postBody) throws QException
{
/////////////////////
// noop by default //
/////////////////////
}
}
@@ -1,114 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.actions;
import java.util.Stack;
import com.kingsrook.qbits.webhooks.WebhooksQBitConfig;
import com.kingsrook.qbits.webhooks.model.Webhook;
import com.kingsrook.qbits.webhooks.model.WebhookEventSendLog;
import com.kingsrook.qbits.webhooks.model.WebhookHealthStatus;
import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.scheduler.processes.AbstractRecordSyncToScheduledJobProcess;
import org.apache.commons.lang3.BooleanUtils;
import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
/*******************************************************************************
** class that manages webhook health status
*******************************************************************************/
public class WebhookHealthManager
{
private static final QLogger LOG = QLogger.getLogger(WebhookHealthManager.class);
/***************************************************************************
**
***************************************************************************/
public Webhook updateWebhookHealth(Stack<WebhookEventSendLog> recentSendLogs, Webhook webhook) throws QException
{
Integer repeatedFailsToGoUnhealthy = WebhooksQBitConfig.getConfigValue(config -> config.getRepeatedFailsToGoUnhealthy());
boolean lastSuccessful = !recentSendLogs.isEmpty() && BooleanUtils.isTrue(recentSendLogs.peek().getSuccessful());
if(lastSuccessful)
{
//////////////////////////////////////////////////////////////////
// if the last send was successful, and the webhook had been on //
// probation, then take it off probation and mark it as healthy //
//////////////////////////////////////////////////////////////////
if(WebhookHealthStatus.PROBATION.getId().equals(webhook.getHealthStatusId()))
{
LOG.info("Marking previously-probation webhook as healthy due to a success", logPair("id", webhook.getId()));
webhook = updateWebhookHealthStatus(webhook, WebhookHealthStatus.HEALTHY);
}
}
else
{
/////////////////////////////////////////
// else the last send was a failure... //
/////////////////////////////////////////
if(WebhookHealthStatus.PROBATION.getId().equals(webhook.getHealthStatusId()))
{
////////////////////////////////////////////////////////////////
// if the webhook was on probation, put it back on probation. //
////////////////////////////////////////////////////////////////
LOG.info("Marking previously-probation webhook as unhealthy due to a failure", logPair("id", webhook.getId()));
webhook = updateWebhookHealthStatus(webhook, WebhookHealthStatus.UNHEALTHY);
}
else if(WebhookHealthStatus.HEALTHY.getId().equals(webhook.getHealthStatusId()))
{
///////////////////////////////////////////////////////////////////////////////////////////////////
// if the webhook was healthy - mark it as unhealthy if there have been enough failures in-a-row //
///////////////////////////////////////////////////////////////////////////////////////////////////
if(repeatedFailsToGoUnhealthy != null && recentSendLogs.size() >= repeatedFailsToGoUnhealthy
&& recentSendLogs.stream().noneMatch(sendLog -> BooleanUtils.isTrue(sendLog.getSuccessful())))
{
LOG.info("Marking previously-healthy webhook as unhealthy due to repeated failures", logPair("id", webhook.getId()), logPair("repeatedFailLimit", repeatedFailsToGoUnhealthy));
webhook = updateWebhookHealthStatus(webhook, WebhookHealthStatus.UNHEALTHY);
}
}
}
return (webhook);
}
/***************************************************************************
**
***************************************************************************/
private Webhook updateWebhookHealthStatus(Webhook webhook, WebhookHealthStatus status) throws QException
{
new UpdateAction().execute(new UpdateInput(Webhook.TABLE_NAME)
.withFlag(AbstractRecordSyncToScheduledJobProcess.ActionFlags.DO_NOT_SYNC)
.withRecord(new QRecord()
.withValue("id", webhook.getId())
.withValue("healthStatusId", status.getId())));
webhook.setHealthStatusId(status.getId());
return (webhook);
}
}
@@ -1,50 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.actions;
import com.kingsrook.qqq.backend.core.exceptions.QException;
/*******************************************************************************
** exception thrown while posting to a webhook
*******************************************************************************/
public class WebhookPostException extends QException
{
/***************************************************************************
**
***************************************************************************/
public WebhookPostException(String message)
{
super(message);
}
/***************************************************************************
**
***************************************************************************/
public WebhookPostException(String message, Throwable cause)
{
super(message, cause);
}
}
@@ -1,37 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.actions;
/*******************************************************************************
**
*******************************************************************************/
public class WebhookRateLimitException extends Exception
{
/*******************************************************************************
** Constructor
**
*******************************************************************************/
public WebhookRateLimitException(String message)
{
super(message);
}
}
@@ -1,259 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.actions;
import java.io.Serializable;
import java.time.Duration;
import java.time.temporal.ChronoUnit;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import com.kingsrook.qbits.webhooks.WebhooksQBitConfig;
import com.kingsrook.qbits.webhooks.model.Webhook;
import com.kingsrook.qbits.webhooks.model.WebhookActiveStatus;
import com.kingsrook.qbits.webhooks.model.WebhookEventCategory;
import com.kingsrook.qbits.webhooks.model.WebhookSubscription;
import com.kingsrook.qbits.webhooks.registry.WebhookEventType;
import com.kingsrook.qbits.webhooks.registry.WebhooksRegistry;
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.actions.tables.helpers.ValidateRecordSecurityLockHelper;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryJoin;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitConfig;
import com.kingsrook.qqq.backend.core.model.metadata.security.QSecurityKeyType;
import com.kingsrook.qqq.backend.core.model.metadata.security.RecordSecurityLock;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import com.kingsrook.qqq.backend.core.model.session.QSession;
import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
import com.kingsrook.qqq.backend.core.utils.StringUtils;
import com.kingsrook.qqq.backend.core.utils.memoization.Memoization;
/*******************************************************************************
** helper class for working with webhook subscriptions
*******************************************************************************/
public class WebhookSubscriptionsHelper
{
private static final Memoization<String, List<WebhookSubscription>> subscriptionsByEventTypeMemoization = new Memoization<String, List<WebhookSubscription>>()
.withTimeout(Duration.of(WebhooksQBitConfig.getConfigValue(config -> config.getMemoizationTimeoutMinutes()).longValue(), ChronoUnit.MINUTES));
private static final Memoization<Integer, QRecord> webhookBySubscriptionIdMemoization = new Memoization<Integer, QRecord>()
.withTimeout(Duration.of(WebhooksQBitConfig.getConfigValue(config -> config.getMemoizationTimeoutMinutes()).longValue(), ChronoUnit.MINUTES));
private static List<String> allAccessSecurityKeysToUse = null;
/***************************************************************************
**
***************************************************************************/
public static void clearMemoizations()
{
subscriptionsByEventTypeMemoization.clear();
webhookBySubscriptionIdMemoization.clear();
}
/***************************************************************************
**
***************************************************************************/
public static List<WebhookEventType> getWebhookEventTypesToConsiderFiringEventsFor(WebhookEventCategory.Kind kind, String tableName)
{
List<WebhookEventType> eventTypes = null;
for(WebhookEventType webhookEventType : WebhooksRegistry.ofOrWithNew(QContext.getQInstance()).getAllWebhookEventTypes())
{
if(webhookEventType.getCategory().getKind().equals(kind))
{
if(!StringUtils.hasContent(webhookEventType.getTableName()) || Objects.equals(webhookEventType.getTableName(), tableName))
{
if(CollectionUtils.nullSafeHasContents(getSubscriptionsForEventType(webhookEventType)))
{
eventTypes = Objects.requireNonNullElseGet(eventTypes, ArrayList::new);
eventTypes.add(webhookEventType);
}
}
}
}
return (eventTypes);
}
/***************************************************************************
**
***************************************************************************/
public static List<WebhookSubscription> getSubscriptionsForEventType(WebhookEventType webhookEventType)
{
return subscriptionsByEventTypeMemoization.getResultThrowing(webhookEventType.getName(), name ->
{
try
{
return QueryAction.execute(WebhookSubscription.TABLE_NAME, WebhookSubscription.class, new QQueryFilter()
.withCriteria(new QFilterCriteria("webhookEventTypeName", QCriteriaOperator.EQUALS, name))
.withCriteria(new QFilterCriteria("activeStatusId", QCriteriaOperator.NOT_EQUALS, WebhookActiveStatus.DISABLED.getId()))
);
}
catch(QException e)
{
throw new RuntimeException(e);
}
}).orElse(Collections.emptyList());
}
/***************************************************************************
**
***************************************************************************/
public static QRecord getWebhookForSubscription(WebhookSubscription webhookSubscription)
{
return webhookBySubscriptionIdMemoization.getResultThrowing(webhookSubscription.getId(), subscriptionId ->
{
try
{
List<QRecord> webhooks = new QueryAction().execute(new QueryInput(Webhook.TABLE_NAME)
.withFilter(new QQueryFilter(new QFilterCriteria(WebhookSubscription.TABLE_NAME + ".id", QCriteriaOperator.EQUALS, subscriptionId)))
.withQueryJoin(new QueryJoin(WebhookSubscription.TABLE_NAME)))
.getRecords();
if(webhooks.isEmpty())
{
return (null);
}
else
{
return (webhooks.get(0));
}
}
catch(QException e)
{
throw new RuntimeException(e);
}
}).orElse(null);
}
/***************************************************************************
**
***************************************************************************/
public static List<String> getAllAccessSecurityKeysToUse()
{
if(allAccessSecurityKeysToUse == null)
{
List<String> list = new ArrayList<>();
Map<String, String> securityKeyTypeToAllAccessKeyNameMap = new HashMap<>();
for(QSecurityKeyType securityKeyType : QContext.getQInstance().getSecurityKeyTypes().values())
{
if(StringUtils.hasContent(securityKeyType.getAllAccessKeyName()))
{
securityKeyTypeToAllAccessKeyNameMap.put(securityKeyType.getName(), securityKeyType.getAllAccessKeyName());
}
}
QTableMetaData webhookTable = QContext.getQInstance().getTable(Webhook.TABLE_NAME);
QBitConfig qbitConfig = webhookTable.getSourceQBitConfig();
if(qbitConfig instanceof WebhooksQBitConfig webhooksQBitConfig)
{
for(RecordSecurityLock securityLock : CollectionUtils.nonNullList(webhooksQBitConfig.getRecordSecurityLocks()))
{
securityKeyTypeToAllAccessKeyNameMap.remove(securityLock.getSecurityKeyType());
}
list.addAll(securityKeyTypeToAllAccessKeyNameMap.values());
}
allAccessSecurityKeysToUse = list;
}
return (allAccessSecurityKeysToUse);
}
/***************************************************************************
**
***************************************************************************/
public static boolean doesRecordMatchSubscription(WebhookEventType webhookEventType, WebhookSubscription webhookSubscription, QRecord record, QBackendTransaction transaction) throws QException
{
String tableName = record.getTableName();
QTableMetaData table = QContext.getQInstance().getTable(tableName);
List<RecordSecurityLock> recordSecurityLocks = table.getRecordSecurityLocks();
if(CollectionUtils.nullSafeIsEmpty(recordSecurityLocks))
{
////////////////////////////////////////////
// if no locks on the table, always allow //
////////////////////////////////////////////
return (true);
}
QRecord webhookRecord = getWebhookForSubscription(webhookSubscription);
QTableMetaData webhookTable = QContext.getQInstance().getTable(Webhook.TABLE_NAME);
QBitConfig qbitConfig = webhookTable.getSourceQBitConfig();
QSession pseudoSession = new QSession();
////////////////////////////////////////////////////////////////////////////////////////////////////
// ask the qbit config what security locks are in place - e.g., values to copy from the webhook //
// record into a pseudo-session, to use for checking if the webhook is allowed to see the record. //
////////////////////////////////////////////////////////////////////////////////////////////////////
if(qbitConfig instanceof WebhooksQBitConfig webhooksQBitConfig)
{
for(RecordSecurityLock securityLock : CollectionUtils.nonNullList(webhooksQBitConfig.getRecordSecurityLocks()))
{
Serializable securityFieldValue = webhookRecord.getValue(securityLock.getFieldName());
pseudoSession.withSecurityKeyValue(securityLock.getSecurityKeyType(), securityFieldValue);
}
}
//////////////////////////////////////////////////////////////
// if there are no security locks on the qbit, always allow //
//////////////////////////////////////////////////////////////
if(pseudoSession.getSecurityKeyValues().isEmpty())
{
return (true);
}
//////////////////////////////////////////////////////////////////////////////////////////////////
// any keys in the instance that aren't used by the webhook qbit, if there's an all-access key, //
// then use it - to avoid records being blocked by a non-applicable lock //
//////////////////////////////////////////////////////////////////////////////////////////////////
for(String allAccessKey : getAllAccessSecurityKeysToUse())
{
pseudoSession.withSecurityKeyValue(allAccessKey, true);
}
return (ValidateRecordSecurityLockHelper.allowedToReadRecord(table, record, pseudoSession, transaction));
}
}
@@ -26,6 +26,7 @@ import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.webhooks.WebhooksQBitConfig;
import com.kingsrook.qbits.webhooks.WebhooksRuntime;
import com.kingsrook.qbits.webhooks.health.WebhooksHealthWidget;
import com.kingsrook.qbits.webhooks.metadata.WebhooksEnums.SecretSource;
import com.kingsrook.qbits.webhooks.metadata.WebhooksEnums.SecretStatus;
import com.kingsrook.qbits.webhooks.metadata.WebhooksTables;
@@ -34,7 +35,10 @@ import com.kingsrook.qbits.webhooks.permissions.WebhooksPermissions;
import com.kingsrook.qbits.webhooks.secrets.SecretCipher;
import com.kingsrook.qbits.webhooks.support.SystemWrites;
import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
import com.kingsrook.qqq.backend.core.actions.permissions.TablePermissionSubType;
import com.kingsrook.qqq.backend.core.actions.permissions.UseTablePermissionCustomPermissionChecker;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.health.HealthMetaDataProvider;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
@@ -47,6 +51,8 @@ import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.PermissionLevel;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRules;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
@@ -57,6 +63,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
*******************************************************************************/
public final class WebhooksApp
{
public static final String HEALTH_WIDGET = "webhooksHealth";
public static final String HOME_WIDGET = "webhooksHome";
public static final String SECRET_VALUE_WIDGET = "webhooksSecretValue";
@@ -77,6 +84,10 @@ public final class WebhooksApp
public static List<QWidgetMetaData> widgets()
{
return (List.of(
new QWidgetMetaData().withName(HEALTH_WIDGET).withLabel("System Health").withType(WidgetType.STATISTICS.getType())
.withCodeReference(new QCodeReference(WebhooksHealthWidget.class)).withGridColumns(12)
.withPermissionRules(new QPermissionRules().withLevel(PermissionLevel.HAS_ACCESS_PERMISSION)
.withCustomPermissionChecker(UseTablePermissionCustomPermissionChecker.build(HealthMetaDataProvider.CHECK_TABLE, TablePermissionSubType.READ))),
new QWidgetMetaData().withName(HOME_WIDGET).withLabel("Overview").withType(WidgetType.HTML.getType())
.withCodeReference(new QCodeReference(HomeRenderer.class)).withGridColumns(12).withIsCard(false)
.withPermissionRules(WebhooksTables.permissionRules()),
@@ -97,7 +108,7 @@ public final class WebhooksApp
.withLabel(config.getAppLabel())
.withIcon(new QIcon("webhook"))
.withPermissionRules(WebhooksTables.permissionRules())
.withWidgets(List.of(HOME_WIDGET));
.withWidgets(List.of(HEALTH_WIDGET, HOME_WIDGET));
for(String name : WebhooksTables.MAIN)
{
if(tables.containsKey(name))
@@ -26,15 +26,10 @@ import java.time.Instant;
import java.time.temporal.ChronoUnit;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import com.kingsrook.qbits.webhooks.WebhooksRuntime;
import com.kingsrook.qbits.webhooks.metadata.WebhooksEnums.ActiveStatus;
import com.kingsrook.qbits.webhooks.metadata.WebhooksEnums.AttemptOutcome;
import com.kingsrook.qbits.webhooks.metadata.WebhooksEnums.DeliveryStatus;
import com.kingsrook.qbits.webhooks.metadata.WebhooksEnums.HealthStatus;
import com.kingsrook.qbits.webhooks.metadata.WebhooksEnums.SecretStatus;
import com.kingsrook.qbits.webhooks.metadata.WebhooksTables;
import com.kingsrook.qqq.backend.core.actions.tables.CountAction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
@@ -59,7 +54,6 @@ public final class WebhooksHomeData
private static final int ATTEMPT_SAMPLE = 20000;
private Instant now;
private int endpointCount;
private int delivered24h;
private int failedAttempts24h;
private int rateLimited24h;
@@ -69,9 +63,8 @@ public final class WebhooksHomeData
private Long p95Millis;
private final int[] hourlyOk = new int[24];
private final int[] hourlyFailed = new int[24];
private final List<QRecord> attention = new ArrayList<>();
private final List<String> attentionWhy = new ArrayList<>();
private final List<QRecord> recentDeadLetters = new ArrayList<>();
private List<WebhooksSetupStatus.Step> setup = new ArrayList<>();
@@ -84,24 +77,6 @@ public final class WebhooksHomeData
data.now = WebhooksRuntime.clock().instant();
Instant since = data.now.minus(Duration.ofHours(24));
List<QRecord> endpoints = query(WebhooksTables.ENDPOINT, new QQueryFilter().withOrderBy(new QFilterOrderBy("name")).withLimit(500));
data.endpointCount = endpoints.size();
Set<Integer> withSecret = new HashSet<>();
for(QRecord secret : query(WebhooksTables.SECRET, new QQueryFilter(new QFilterCriteria("status", QCriteriaOperator.IN, SecretStatus.ACTIVE.name(), SecretStatus.PREVIOUS.name()))))
{
withSecret.add(secret.getValueInteger("endpointId"));
}
for(QRecord endpoint : endpoints)
{
String why = attentionReason(endpoint, withSecret.contains(endpoint.getValueInteger("id")));
if(why != null)
{
data.attention.add(endpoint);
data.attentionWhy.add(why);
}
}
data.delivered24h = count(WebhooksTables.DELIVERY, new QQueryFilter(new QFilterCriteria("deliveredDate", QCriteriaOperator.GREATER_THAN_OR_EQUALS, since)));
data.deadLetters = count(WebhooksTables.DELIVERY, new QQueryFilter(new QFilterCriteria("status", QCriteriaOperator.EQUALS, DeliveryStatus.DEAD_LETTER.name())));
data.backlog = count(WebhooksTables.DELIVERY, new QQueryFilter(new QFilterCriteria("status", QCriteriaOperator.IN,
@@ -150,44 +125,12 @@ public final class WebhooksHomeData
data.recentDeadLetters.addAll(query(WebhooksTables.DELIVERY, new QQueryFilter(new QFilterCriteria("status", QCriteriaOperator.EQUALS, DeliveryStatus.DEAD_LETTER.name()))
.withOrderBy(new QFilterOrderBy("id", false)).withLimit(5)));
data.setup = WebhooksSetupStatus.load();
return (data);
}
/*******************************************************************************
** Why an endpoint needs attention, or null.
*******************************************************************************/
static String attentionReason(QRecord endpoint, boolean hasSecret)
{
String health = endpoint.getValueString("healthStatus");
String active = endpoint.getValueString("activeStatus");
if(ActiveStatus.DISABLED.name().equals(active) && endpoint.getValue("autoDisabledDate") != null)
{
return ("Disabled after failing on probation");
}
if(HealthStatus.UNHEALTHY.name().equals(health))
{
return ("Unhealthy: deliveries paused");
}
if(HealthStatus.PROBATION.name().equals(health))
{
return ("On probation: one try per run");
}
if(ActiveStatus.ACTIVE.name().equals(active) && !hasSecret && !Boolean.TRUE.equals(endpoint.getValueBoolean("allowUnsigned")))
{
return ("No signing secret: deliveries are waiting");
}
Integer failures = endpoint.getValueInteger("consecutiveFailures");
if(failures != null && failures > 0)
{
return (failures + (failures == 1 ? " failure" : " failures") + " in a row");
}
return (null);
}
/*******************************************************************************
** Query as the viewer.
*******************************************************************************/
@@ -219,16 +162,6 @@ public final class WebhooksHomeData
/*******************************************************************************
** Getter for endpointCount.
*******************************************************************************/
public int getEndpointCount()
{
return (endpointCount);
}
/*******************************************************************************
** Getter for delivered24h.
*******************************************************************************/
@@ -319,26 +252,6 @@ public final class WebhooksHomeData
/*******************************************************************************
** Endpoints that need attention.
*******************************************************************************/
public List<QRecord> getAttention()
{
return (attention);
}
/*******************************************************************************
** The reason for each endpoint in {@link #getAttention()}.
*******************************************************************************/
public List<String> getAttentionWhy()
{
return (attentionWhy);
}
/*******************************************************************************
** The five newest dead letters.
*******************************************************************************/
@@ -346,4 +259,15 @@ public final class WebhooksHomeData
{
return (recentDeadLetters);
}
/*******************************************************************************
** The setup steps (state and next step of each thing an admin configures).
*******************************************************************************/
public List<WebhooksSetupStatus.Step> getSetup()
{
return (setup);
}
}
@@ -22,12 +22,18 @@ package com.kingsrook.qbits.webhooks.app;
import java.io.Serializable;
import java.time.Duration;
import java.time.Instant;
import java.time.ZoneOffset;
import java.time.format.DateTimeFormatter;
import java.util.List;
import java.util.Locale;
import com.kingsrook.qbits.webhooks.metadata.WebhooksEnums.AttemptOutcome;
import com.kingsrook.qbits.webhooks.metadata.WebhooksEnums.DeliveryStatus;
import com.kingsrook.qbits.webhooks.metadata.WebhooksTables;
import com.kingsrook.qbits.webhooks.support.FilterLinks;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.utils.ValueUtils;
@@ -85,26 +91,51 @@ public final class WebhooksHomeView
public static String render(WebhooksHomeData data, String tablePrefix)
{
StringBuilder html = new StringBuilder("<div style=\"" + HOME + "\">");
setup(html, data);
kpis(html, data, tablePrefix);
volume(html, data);
attention(html, data, tablePrefix);
deadLetters(html, data, tablePrefix);
return (html.append("</div>").toString());
}
/*******************************************************************************
** The Setup card: each thing to configure, its state and the next step.
*******************************************************************************/
private static void setup(StringBuilder html, WebhooksHomeData data)
{
html.append("<div style=\"").append(CARD).append("\"><h3 style=\"").append(TITLE).append("\">Setup</h3>");
for(WebhooksSetupStatus.Step step : data.getSetup())
{
String label = step.href() == null ? esc(step.label()) : "<a href=\"" + esc(step.href()) + "\">" + esc(step.label()) + "</a>";
String state = WebhooksSetupStatus.NEEDS_SETUP.equals(step.state()) ? "<span style=\"color:" + FAIL_COLOR + "\">" + esc(step.state()) + "</span>"
: WebhooksSetupStatus.SET_UP.equals(step.state()) ? "<span style=\"color:" + OK_COLOR + "\">" + esc(step.state()) + "</span>" : esc(step.state());
html.append("<div style=\"").append(ROW).append("\">").append(cell("Step", label)).append(cell("State", state)).append(cell("Next step", esc(step.next()))).append("</div>");
}
html.append("</div>");
}
/*******************************************************************************
** The KPI tiles.
*******************************************************************************/
private static void kpis(StringBuilder html, WebhooksHomeData data, String prefix)
{
html.append("<div style=\"").append(KPIS).append("\">");
kpi(html, String.valueOf(data.getDelivered24h()), "Delivered (24 h)", false, prefix + "/" + WebhooksTables.DELIVERY);
kpi(html, String.valueOf(data.getFailedAttempts24h()), "Failed attempts (24 h)", data.getFailedAttempts24h() > 0, prefix + "/" + WebhooksTables.DELIVERY);
kpi(html, String.valueOf(data.getDeadLetters()), "Dead letters", data.getDeadLetters() > 0, prefix + "/" + WebhooksTables.DELIVERY);
kpi(html, String.valueOf(data.getBacklog()), "Waiting to send", false, prefix + "/" + WebhooksTables.DELIVERY);
kpi(html, String.valueOf(data.getAttention().size()), "Endpoints needing attention", !data.getAttention().isEmpty(), prefix + "/" + WebhooksTables.ENDPOINT);
String delivery = prefix + "/" + WebhooksTables.DELIVERY;
String since = data.getNow().minus(Duration.ofHours(24)).toString();
kpi(html, String.valueOf(data.getDelivered24h()), "Delivered (24 h)", false,
FilterLinks.href(delivery, new QQueryFilter(new QFilterCriteria("deliveredDate", QCriteriaOperator.GREATER_THAN_OR_EQUALS, since))));
kpi(html, String.valueOf(data.getFailedAttempts24h()), "Failed attempts (24 h)", data.getFailedAttempts24h() > 0,
FilterLinks.href(prefix + "/" + WebhooksTables.ATTEMPT, new QQueryFilter(new QFilterCriteria("startDate", QCriteriaOperator.GREATER_THAN_OR_EQUALS, since),
new QFilterCriteria("outcome", QCriteriaOperator.NOT_IN, AttemptOutcome.SUCCEEDED.name(), AttemptOutcome.RATE_LIMITED.name(), AttemptOutcome.IN_FLIGHT.name()))));
kpi(html, String.valueOf(data.getDeadLetters()), "Dead letters", data.getDeadLetters() > 0,
FilterLinks.href(delivery, new QQueryFilter(new QFilterCriteria("status", QCriteriaOperator.EQUALS, DeliveryStatus.DEAD_LETTER.name()))));
kpi(html, String.valueOf(data.getBacklog()), "Waiting to send", false,
FilterLinks.href(delivery, new QQueryFilter(new QFilterCriteria("status", QCriteriaOperator.IN,
DeliveryStatus.PENDING.name(), DeliveryStatus.RETRY_WAIT.name(), DeliveryStatus.SENDING.name()))));
kpi(html, data.getP50Millis() == null ? "None yet" : data.getP50Millis() + " / " + data.getP95Millis() + " ms", "Latency, median / 95th (24 h)", false, null);
html.append("</div>");
}
@@ -177,41 +208,6 @@ public final class WebhooksHomeView
/*******************************************************************************
** Endpoints needing attention, one card row each.
*******************************************************************************/
private static void attention(StringBuilder html, WebhooksHomeData data, String prefix)
{
html.append("<div style=\"").append(CARD).append("\"><h3 style=\"").append(TITLE).append("\">Endpoints needing attention</h3>");
if(data.getEndpointCount() == 0)
{
html.append("<p style=\"").append(MUTED).append("\">No endpoints yet. <a href=\"").append(esc(prefix + "/" + WebhooksTables.ENDPOINT))
.append("\">Add an endpoint</a>, then subscribe it to events.</p></div>");
return;
}
if(data.getAttention().isEmpty())
{
html.append("<p style=\"").append(MUTED).append("\">All ").append(data.getEndpointCount()).append(" endpoints are healthy.</p></div>");
return;
}
List<QRecord> rows = data.getAttention();
for(int i = 0; i < rows.size(); i++)
{
QRecord endpoint = rows.get(i);
String message = endpoint.getValueString("lastFailureMessage");
html.append("<div style=\"").append(ROW).append("\">")
.append(cell("Endpoint", "<a href=\"" + esc(prefix + "/" + WebhooksTables.ENDPOINT + "/" + endpoint.getValue("id")) + "\">"
+ esc(endpoint.getValueString("name")) + "</a>"))
.append(cell("Why", esc(data.getAttentionWhy().get(i)) + (message == null ? "" : "<br><span style=\"opacity:.75\">" + esc(clamp(message, 140)) + "</span>")))
.append(cell("Last failure", when(endpoint.getValue("lastFailureDate"))))
.append(cell("Last success", when(endpoint.getValue("lastSuccessDate"))))
.append("</div>");
}
html.append("</div>");
}
/*******************************************************************************
** Newest dead letters.
*******************************************************************************/
@@ -0,0 +1,146 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.app;
import java.util.ArrayList;
import java.util.List;
import com.kingsrook.qbits.webhooks.WebhooksQBitConfig;
import com.kingsrook.qbits.webhooks.WebhooksRuntime;
import com.kingsrook.qbits.webhooks.metadata.WebhooksEnums.ActiveStatus;
import com.kingsrook.qbits.webhooks.metadata.WebhooksTables;
import com.kingsrook.qbits.webhooks.processes.WebhooksProcesses;
import com.kingsrook.qbits.webhooks.registry.WebhookEventType;
import com.kingsrook.qbits.webhooks.registry.WebhooksRegistry;
import com.kingsrook.qbits.webhooks.secrets.EndpointSecrets;
import com.kingsrook.qqq.backend.core.actions.tables.CountAction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.tables.count.CountInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** The Setup card on the Webhooks home: each thing an administrator configures
** (event types, encryption key, endpoints, secrets, subscriptions), its state
** and the next step. Nothing here is seeded: every step is the owner's choice.
*******************************************************************************/
public final class WebhooksSetupStatus
{
public static final String SET_UP = "Set up";
public static final String NEEDS_SETUP = "Needs setup";
public static final String OPTIONAL = "Optional";
/*******************************************************************************
** One line of the card.
*******************************************************************************/
public record Step(String label, String state, String next, String href)
{
}
/*******************************************************************************
** Utility class.
*******************************************************************************/
private WebhooksSetupStatus()
{
}
/*******************************************************************************
** Read the current state, as the viewer.
*******************************************************************************/
public static List<Step> load() throws QException
{
WebhooksQBitConfig config = WebhooksRuntime.config();
int eventTypes = 0;
for(WebhookEventType type : WebhooksRegistry.of(QContext.getQInstance()).getAll())
{
if(!WebhooksProcesses.TEST_EVENT_TYPE.equals(type.getName()))
{
eventTypes++;
}
}
int endpoints = count(WebhooksTables.ENDPOINT);
int subscriptions = count(WebhooksTables.SUBSCRIPTION);
int blocked = 0;
for(QRecord endpoint : new QueryAction().execute(new QueryInput(WebhooksTables.ENDPOINT)).getRecords())
{
///////////////////////////////////////////////////////////////////////////////////
// exactly DeliveryEngine's test: active, signing required, and no usable secret //
///////////////////////////////////////////////////////////////////////////////////
if(ActiveStatus.ACTIVE.name().equals(endpoint.getValueString("activeStatus")) && !Boolean.TRUE.equals(endpoint.getValueBoolean("allowUnsigned"))
&& EndpointSecrets.signingKeys(endpoint.getValueInteger("id")).isEmpty())
{
blocked++;
}
}
return (steps(config.getTablePathPrefix(), config.getEncryptionKeyEnvName(), eventTypes, EndpointSecrets.canGenerate(), endpoints, blocked, subscriptions));
}
/*******************************************************************************
** The steps, from the numbers (pure, for tests).
*******************************************************************************/
static List<Step> steps(String prefix, String keyEnvName, int eventTypes, boolean canGenerate, int endpoints, int blockedEndpoints, int subscriptions)
{
String endpointHref = prefix + "/" + WebhooksTables.ENDPOINT;
String subscriptionHref = prefix + "/" + WebhooksTables.SUBSCRIPTION;
List<Step> steps = new ArrayList<>();
steps.add(eventTypes > 0
? new Step("Event types", SET_UP, eventTypes + " available to subscribe to (from the code-side registry)", subscriptionHref)
: new Step("Event types", NEEDS_SETUP, "The host lists tables or processes that raise events in the qbit config (tableEventSources, processEventSources), or fires custom events", null));
steps.add(canGenerate
? new Step("Encryption key", SET_UP, "Generated signing secrets can be stored encrypted", null)
: new Step("Encryption key", OPTIONAL, "Set " + keyEnvName + " to generate secrets here, or use environment-variable secrets", null));
steps.add(endpoints > 0
? new Step("Endpoints", SET_UP, endpoints + (endpoints == 1 ? " endpoint" : " endpoints"), endpointHref)
: new Step("Endpoints", NEEDS_SETUP, "Add an endpoint: the URL that receives events", endpointHref));
steps.add(endpoints > 0 && blockedEndpoints == 0
? new Step("Signing secrets", SET_UP, "Every endpoint signs its deliveries", endpointHref)
: new Step("Signing secrets", NEEDS_SETUP, endpoints == 0 ? "Add an endpoint first, then give it a secret"
: blockedEndpoints + " active endpoint(s) have no usable signing secret, so their deliveries wait: open the endpoint and add one", endpointHref));
steps.add(subscriptions > 0
? new Step("Subscriptions", SET_UP, subscriptions + (subscriptions == 1 ? " subscription" : " subscriptions"), subscriptionHref)
: new Step("Subscriptions", NEEDS_SETUP, "Subscribe an endpoint to an event type, or nothing is ever sent", subscriptionHref));
return (steps);
}
/*******************************************************************************
** Count as the viewer.
*******************************************************************************/
private static int count(String table) throws QException
{
Integer count = new CountAction().execute(new CountInput(table)).getCount();
return (count == null ? 0 : count);
}
}
@@ -42,8 +42,6 @@ import com.kingsrook.qbits.webhooks.secrets.EndpointSecrets;
import com.kingsrook.qbits.webhooks.shared.http.SafeHttpClient;
import com.kingsrook.qbits.webhooks.shared.http.SafeHttpException;
import com.kingsrook.qbits.webhooks.shared.http.SafeHttpResponse;
import com.kingsrook.qbits.webhooks.shared.signing.WebhookSecret;
import com.kingsrook.qbits.webhooks.shared.signing.WebhookSigner;
import com.kingsrook.qbits.webhooks.support.SystemWrites;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
@@ -55,6 +53,8 @@ import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
import com.kingsrook.qqq.backend.core.utils.StringUtils;
import com.kingsrook.qqq.backend.core.utils.ValueUtils;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSigner;
import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
@@ -0,0 +1,230 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.health;
import java.time.Duration;
import java.time.Instant;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
import java.util.Set;
import com.kingsrook.qbits.webhooks.WebhooksRuntime;
import com.kingsrook.qbits.webhooks.metadata.WebhooksEnums.ActiveStatus;
import com.kingsrook.qbits.webhooks.metadata.WebhooksEnums.DeliveryStatus;
import com.kingsrook.qbits.webhooks.metadata.WebhooksEnums.HealthStatus;
import com.kingsrook.qbits.webhooks.metadata.WebhooksTables;
import com.kingsrook.qbits.webhooks.support.SystemWrites;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.health.HealthCategory;
import com.kingsrook.qqq.backend.core.health.HealthCheck;
import com.kingsrook.qqq.backend.core.health.HealthCheckInput;
import com.kingsrook.qqq.backend.core.health.HealthCheckKind;
import com.kingsrook.qqq.backend.core.health.HealthCheckMetaData;
import com.kingsrook.qqq.backend.core.health.HealthCheckOutcome;
import com.kingsrook.qqq.backend.core.health.HealthCheckSupplier;
import com.kingsrook.qqq.backend.core.health.HealthCheckSupplierMetaData;
import com.kingsrook.qqq.backend.core.health.HealthChecksMetaData;
import com.kingsrook.qqq.backend.core.health.HealthDetails;
import com.kingsrook.qqq.backend.core.health.HealthMetric;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
/*******************************************************************************
** Passive central checks over the existing endpoint state machine and outbox.
*******************************************************************************/
public class WebhooksHealthChecks implements HealthCheckSupplier
{
/*******************************************************************************
** Register the dispatcher and dynamic endpoint supplier from the producer.
*******************************************************************************/
public static void register(QInstance instance)
{
HealthChecksMetaData.ofOrWithNew(instance)
.withCheck(definition("webhooks.dispatcher", WebhooksHealthChecks::dispatcher))
.withSupplier(new HealthCheckSupplierMetaData().withNamePrefix("webhooks.endpoint").withComponent("webhooks")
.withSupplier(new QCodeReference(WebhooksHealthChecks.class)));
}
/*******************************************************************************
** Keep auto-disabled endpoints visible until an operator recovers them.
*******************************************************************************/
@Override
public List<HealthCheckMetaData> supply(QInstance qInstance) throws QException
{
List<HealthCheckMetaData> checks = new ArrayList<>();
for(QRecord endpoint : SystemWrites.call(() -> new QueryAction().execute(new QueryInput(WebhooksTables.ENDPOINT)
.withFilter(new QQueryFilter(new QFilterCriteria("activeStatus", QCriteriaOperator.IN, ActiveStatus.ACTIVE.name(), ActiveStatus.DISABLED.name())))
.withFieldNamesToInclude(Set.of("id", "activeStatus", "autoDisabledDate")).withTimeoutSeconds(5)).getRecords()))
{
if(monitored(endpoint))
{
Integer id = endpoint.getValueInteger("id");
checks.add(definition("webhooks.endpoint." + id.toString().toLowerCase(Locale.ROOT), input -> endpoint(input, id)));
}
}
return (checks);
}
/*******************************************************************************
** Common metadata and a boundary that never exposes backend error text.
*******************************************************************************/
private static HealthCheckMetaData definition(String name, HealthCheck check)
{
return (new HealthCheckMetaData().withName(name).withLabel(name).withComponent("webhooks")
.withCategory(HealthCategory.INTEGRATION).withKind(HealthCheckKind.DEPENDENCY).withIntervalSeconds(300).withTimeoutMs(5000)
.withCheckInstance(input ->
{
if(expired(input))
{
return (HealthCheckOutcome.down("Health check timed out"));
}
try
{
HealthCheckOutcome outcome = check.run(input);
return (expired(input) ? HealthCheckOutcome.down("Health check timed out") : outcome);
}
catch(Exception e)
{
return (HealthCheckOutcome.down("Unable to read webhook health"));
}
}));
}
/*******************************************************************************
** Read the latest EndpointHealth state, without probing or mutating it.
*******************************************************************************/
private static HealthCheckOutcome endpoint(HealthCheckInput input, Integer id) throws QException
{
List<QRecord> records = query(input, WebhooksTables.ENDPOINT,
new QQueryFilter(new QFilterCriteria("id", QCriteriaOperator.EQUALS, id)).withLimit(1),
Set.of("id", "activeStatus", "autoDisabledDate", "healthStatus", "consecutiveFailures", "unhealthySinceDate"));
if(records.isEmpty() || !monitored(records.getFirst()))
{
return (HealthCheckOutcome.unknown("Endpoint is absent or disabled"));
}
QRecord endpoint = records.getFirst();
String health = endpoint.getValueString("healthStatus");
HealthCheckOutcome outcome;
if(ActiveStatus.DISABLED.name().equals(endpoint.getValueString("activeStatus")))
{
outcome = HealthCheckOutcome.down("Endpoint automatically disabled");
}
else if(HealthStatus.HEALTHY.name().equals(health))
{
outcome = HealthCheckOutcome.up("Endpoint healthy");
}
else if(HealthStatus.UNHEALTHY.name().equals(health) || HealthStatus.PROBATION.name().equals(health))
{
outcome = HealthCheckOutcome.degraded("Endpoint unhealthy or on probation");
}
else
{
return (HealthCheckOutcome.unknown("Endpoint health is unavailable"));
}
// Only known state names may enter details, even if persisted data is corrupt.
String safeHealth = List.of("HEALTHY", "UNHEALTHY", "PROBATION").contains(health == null ? "" : health) ? health : "UNKNOWN";
Integer failures = endpoint.getValueInteger("consecutiveFailures");
HealthDetails details = new HealthDetails().withValue("healthStatus", safeHealth)
.withValue("consecutiveFailures", failures == null ? 0 : failures);
Instant unhealthySince = endpoint.getValueInstant("unhealthySinceDate");
if(unhealthySince != null)
{
details.withValue("unhealthySince", unhealthySince);
}
return (outcome.withDetails(details));
}
/*******************************************************************************
** Age of the oldest queued delivery, including retries and in-flight work.
*******************************************************************************/
private static HealthCheckOutcome dispatcher(HealthCheckInput input) throws QException
{
List<QRecord> records = query(input, WebhooksTables.DELIVERY,
new QQueryFilter(new QFilterCriteria("status", QCriteriaOperator.IN,
DeliveryStatus.PENDING.name(), DeliveryStatus.RETRY_WAIT.name(), DeliveryStatus.SENDING.name()))
.withOrderBy(new QFilterOrderBy("createDate", true)).withLimit(1), Set.of("createDate"));
long seconds = 0;
if(!records.isEmpty())
{
Instant queued = records.getFirst().getValueInstant("createDate");
if(queued == null)
{
return (HealthCheckOutcome.unknown("Pending delivery age is unavailable"));
}
seconds = Math.max(0, Duration.between(queued, WebhooksRuntime.clock().instant()).getSeconds());
}
HealthCheckOutcome outcome = seconds > 3600 ? HealthCheckOutcome.down("Delivery backlog exceeds one hour")
: seconds > 900 ? HealthCheckOutcome.degraded("Delivery backlog exceeds fifteen minutes") : HealthCheckOutcome.up("Delivery backlog within threshold");
return (outcome.withMetrics(List.of(new HealthMetric().withName("oldestPendingSeconds").withValue(seconds).withUnit("seconds"))));
}
/*******************************************************************************
** Only active or automatically disabled endpoints participate.
*******************************************************************************/
private static boolean monitored(QRecord endpoint)
{
String active = endpoint.getValueString("activeStatus");
return (ActiveStatus.ACTIVE.name().equals(active)
|| (ActiveStatus.DISABLED.name().equals(active) && endpoint.getValue("autoDisabledDate") != null));
}
/*******************************************************************************
** Query only allow-listed fields with the remaining database time budget.
** The runner enforces the precise deadline when the backend blocks.
*******************************************************************************/
private static List<QRecord> query(HealthCheckInput input, String table, QQueryFilter filter, Set<String> fields) throws QException
{
int seconds = (int) Math.max(1, Math.min(5, Duration.between(Instant.now(), input.getDeadline()).toSeconds()));
return (SystemWrites.call(() -> new QueryAction().execute(new QueryInput(table).withFilter(filter)
.withFieldNamesToInclude(fields).withTimeoutSeconds(seconds)).getRecords()));
}
/*******************************************************************************
** Use wall time for the runner deadline, independently of the domain clock.
*******************************************************************************/
private static boolean expired(HealthCheckInput input)
{
return (!Instant.now().isBefore(input.getDeadline()));
}
}
@@ -0,0 +1,70 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.health;
import java.util.List;
import java.util.Set;
import com.kingsrook.qbits.webhooks.support.FilterLinks;
import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.health.HealthMetaDataProvider;
import com.kingsrook.qqq.backend.core.health.HealthStatus;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.StatisticsData;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** One native status line linking to central health, using the viewer's access.
*******************************************************************************/
public class WebhooksHealthWidget extends AbstractWidgetRenderer
{
/*******************************************************************************
** Read the central component rollup; never run a probe from the home.
*******************************************************************************/
@Override
public RenderWidgetOutput render(RenderWidgetInput input) throws QException
{
String status = "No checks have run yet";
if(QContext.getQInstance().getTable(HealthMetaDataProvider.COMPONENT_TABLE) != null)
{
List<QRecord> records = new QueryAction().execute(new QueryInput(HealthMetaDataProvider.COMPONENT_TABLE)
.withFilter(new QQueryFilter(new QFilterCriteria("name", QCriteriaOperator.EQUALS, "webhooks")).withLimit(1))
.withFieldNamesToInclude(Set.of("status"))).getRecords();
if(!records.isEmpty())
{
String stored = records.getFirst().getValueString("status");
status = List.of("UP", "DEGRADED", "DOWN", "UNKNOWN").contains(stored == null ? "" : stored) ? stored : HealthStatus.UNKNOWN.name();
}
}
return (new RenderWidgetOutput(new StatisticsData().withCount(status).withCountContext("Webhooks - System Health")
.withCountURL(FilterLinks.href("/app/" + HealthMetaDataProvider.CHECK_TABLE,
new QQueryFilter(new QFilterCriteria("component", QCriteriaOperator.EQUALS, "webhooks"))))));
}
}
@@ -0,0 +1,108 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.metadata;
import java.util.LinkedHashMap;
import java.util.Map;
import com.kingsrook.qbits.webhooks.WebhooksQBitConfig;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.help.QHelpContent;
import com.kingsrook.qqq.backend.core.model.metadata.help.bundles.HelpBundleEntry;
import com.kingsrook.qqq.backend.core.model.metadata.help.bundles.HelpBundles;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
/*******************************************************************************
** Applies help/qbit-webhooks.yaml (the QQQ help-bundle format) to the
** Webhooks app and its webhooks_* tables: what each table is for and how its
** rows get created, shown on the table screen and when it is empty.
** Retains inline empty-state help for hosts without the helpContent table.
** Core owns bundle parsing, registration and persistent help synchronization.
*******************************************************************************/
public final class WebhooksHelp
{
public static final String RESOURCE = "help/qbit-webhooks.yaml";
public static final String SLOT = "help";
/*******************************************************************************
** Utility class.
*******************************************************************************/
private WebhooksHelp()
{
}
/*******************************************************************************
** The bundle's entries by key, validated (fails startup if malformed).
*******************************************************************************/
public static Map<String, HelpBundleEntry> entries() throws QException
{
Map<String, HelpBundleEntry> entries = new LinkedHashMap<>();
for(HelpBundleEntry entry : HelpBundles.load(RESOURCE, WebhooksHelp.class.getClassLoader()).getEntries())
{
entries.put(entry.getKey(), entry);
}
return (entries);
}
/*******************************************************************************
** Apply the help to the app and every webhooks_* table in the instance.
** A host's own help on a table or the app is kept (the bundle only fills
** the slot).
*******************************************************************************/
public static void apply(QInstance qInstance, WebhooksQBitConfig config) throws QException
{
Map<String, HelpBundleEntry> entries = entries();
for(QTableMetaData table : qInstance.getTables().values())
{
HelpBundleEntry entry = entries.get("table:" + table.getName() + ";slot:" + SLOT);
if(entry != null && (table.getHelpContent() == null || !table.getHelpContent().containsKey(SLOT)))
{
table.withHelpContent(SLOT, content(entry));
}
}
QAppMetaData app = qInstance.getApp(config.getAppName());
HelpBundleEntry appEntry = entries.get("app:webhooks;slot:" + SLOT);
if(app != null && appEntry != null && (app.getHelpContent() == null || !app.getHelpContent().containsKey(SLOT)))
{
app.withHelpContent(SLOT, content(appEntry));
}
}
/*******************************************************************************
** Summary as the first paragraph, then the content, as markdown.
*******************************************************************************/
static QHelpContent content(HelpBundleEntry entry)
{
return (new QHelpContent().withContentAsMarkdown(entry.getSummary().strip() + "\n\n" + entry.getContent().strip()));
}
}
@@ -38,7 +38,6 @@ import com.kingsrook.qbits.webhooks.registry.WebhooksRegistry;
import com.kingsrook.qbits.webhooks.secrets.EnvironmentSecrets;
import com.kingsrook.qbits.webhooks.shared.http.AddressRules;
import com.kingsrook.qbits.webhooks.shared.http.SafeHttpException;
import com.kingsrook.qbits.webhooks.shared.signing.WebhookSecret;
import com.kingsrook.qbits.webhooks.support.SystemWrites;
import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
import com.kingsrook.qqq.backend.core.actions.permissions.PermissionsHelper;
@@ -50,6 +49,7 @@ import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.statusmessages.BadInputStatusMessage;
import com.kingsrook.qqq.backend.core.utils.StringUtils;
import com.kingsrook.qqq.backend.core.utils.ValueUtils;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret;
/*******************************************************************************
@@ -1,538 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.model;
import java.time.Instant;
import java.util.ArrayList;
import java.util.List;
import com.kingsrook.qbits.webhooks.WebhooksQBitConfig;
import com.kingsrook.qbits.webhooks.actions.ClearWebhookSubscriptionHelperMemoizationsTableCustomizer;
import com.kingsrook.qbits.webhooks.processes.SyncWebhookScheduledJobProcess;
import com.kingsrook.qqq.backend.core.actions.customizers.MultiCustomizer;
import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizers;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
import com.kingsrook.qqq.backend.core.model.data.QAssociation;
import com.kingsrook.qqq.backend.core.model.data.QField;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.AdornmentType;
import com.kingsrook.qqq.backend.core.model.metadata.fields.FieldAdornment;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.ValueTooLongBehavior;
import com.kingsrook.qqq.backend.core.model.metadata.joins.QJoinMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
import com.kingsrook.qqq.backend.core.model.metadata.producers.MetaDataCustomizerInterface;
import com.kingsrook.qqq.backend.core.model.metadata.producers.annotations.ChildJoin;
import com.kingsrook.qqq.backend.core.model.metadata.producers.annotations.ChildRecordListWidget;
import com.kingsrook.qqq.backend.core.model.metadata.producers.annotations.ChildTable;
import com.kingsrook.qqq.backend.core.model.metadata.producers.annotations.QMetaDataProducingEntity;
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitConfig;
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitProductionContext;
import com.kingsrook.qqq.backend.core.model.metadata.security.RecordSecurityLock;
import com.kingsrook.qqq.backend.core.model.metadata.tables.Association;
import com.kingsrook.qqq.backend.core.model.metadata.tables.ExposedJoin;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QFieldSection;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.tables.SectionFactory;
import com.kingsrook.qqq.backend.core.scheduler.processes.BaseSyncToScheduledJobTableCustomizer;
import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
import com.kingsrook.qqq.backend.core.utils.collections.MutableList;
/*******************************************************************************
** QRecord Entity for Webhook table
*******************************************************************************/
@QMetaDataProducingEntity(
producePossibleValueSource = true,
produceTableMetaData = true,
tableMetaDataCustomizer = Webhook.TableMetaDataCustomizer.class,
childTables = {
@ChildTable(
childTableEntityClass = WebhookSubscription.class,
joinFieldName = "webhookId",
childJoin = @ChildJoin(enabled = true),
childRecordListWidget = @ChildRecordListWidget(label = "Subscriptions", enabled = true, maxRows = 250, canAddChildRecords = true, manageAssociationName = Webhook.SUBSCRIPTIONS_ASSOCIATION_NAME)),
@ChildTable(
childTableEntityClass = WebhookEvent.class,
joinFieldName = "webhookId",
childJoin = @ChildJoin(enabled = true),
childRecordListWidget = @ChildRecordListWidget(label = "Events", enabled = true, maxRows = 25, widgetMetaDataCustomizer = Webhook.EventAndEventSendLogChildRecordWidgetMetaDataCustomizer.class)),
@ChildTable(
childTableEntityClass = WebhookEventSendLog.class,
joinFieldName = "webhookId",
childJoin = @ChildJoin(enabled = true),
childRecordListWidget = @ChildRecordListWidget(label = "Event Send Log", enabled = true, maxRows = 25, widgetMetaDataCustomizer = Webhook.EventAndEventSendLogChildRecordWidgetMetaDataCustomizer.class))
}
)
public class Webhook extends QRecordEntity
{
public static final String TABLE_NAME = "webhook";
public static final String SUBSCRIPTIONS_ASSOCIATION_NAME = "subscriptions";
/***************************************************************************
**
***************************************************************************/
public static class TableMetaDataCustomizer implements MetaDataCustomizerInterface<QTableMetaData>
{
/***************************************************************************
**
***************************************************************************/
@Override
public QTableMetaData customizeMetaData(QInstance qInstance, QTableMetaData table) throws QException
{
String subscriptionChildJoinName = QJoinMetaData.makeInferredJoinName(Webhook.TABLE_NAME, WebhookSubscription.TABLE_NAME);
String eventChildJoinName = QJoinMetaData.makeInferredJoinName(Webhook.TABLE_NAME, WebhookEvent.TABLE_NAME);
String eventSendLogChildJoinName = QJoinMetaData.makeInferredJoinName(Webhook.TABLE_NAME, WebhookEventSendLog.TABLE_NAME);
QFieldSection t1section = SectionFactory.defaultT1("id", "name", "description");
table
.withIcon(new QIcon().withName("webhook"))
.withRecordLabelFormat("%s")
.withRecordLabelFields("name")
.withSection(t1section)
.withSection(SectionFactory.defaultT2("url", "activeStatusId", "healthStatusId"))
.withSection(SectionFactory.customT2("subscriptions", new QIcon("subscriptions")).withWidgetName(subscriptionChildJoinName))
.withSection(SectionFactory.customT2("events", new QIcon("notifications")).withWidgetName(eventChildJoinName))
.withSection(SectionFactory.customT2("sendLog", new QIcon("receipt_long")).withWidgetName(eventSendLogChildJoinName))
.withSection(SectionFactory.defaultT3("createDate", "modifyDate"))
.withExposedJoin(new ExposedJoin().withLabel("Subscriptions").withJoinPath(List.of(subscriptionChildJoinName)).withJoinTable(WebhookSubscription.TABLE_NAME))
.withAssociation(new Association().withName(SUBSCRIPTIONS_ASSOCIATION_NAME).withAssociatedTableName(WebhookSubscription.TABLE_NAME).withJoinName(subscriptionChildJoinName));
addActiveStatusChipAdornment(table.getField("activeStatusId"));
addHealthStatusChipAdornment(table.getField("healthStatusId"));
QBitConfig qBitConfig = QBitProductionContext.peekQBitConfig();
if(qBitConfig instanceof WebhooksQBitConfig webhooksQBitConfig)
{
t1section.setFieldNames(new MutableList<>(t1section.getFieldNames()));
for(QFieldMetaData fieldMetaData : CollectionUtils.nonNullList(webhooksQBitConfig.getSecurityFields()))
{
table.addField(fieldMetaData.clone());
t1section.getFieldNames().add(fieldMetaData.getName());
}
for(RecordSecurityLock recordSecurityLock : CollectionUtils.nonNullList(webhooksQBitConfig.getRecordSecurityLocks()))
{
table.withRecordSecurityLock(recordSecurityLock);
}
}
////////////////////////////////////////////////////////////////////////
// we need 2 customizers to run post insert, update, and delete: //
// first, to clear the webhookSubscriptionHelper class's memoizations //
// second, to sync scheduled jobs for this table //
////////////////////////////////////////////////////////////////////////
QCodeReference syncScheduledJobCustomizer = BaseSyncToScheduledJobTableCustomizer.makeCodeReference(table, new SyncWebhookScheduledJobProcess());
QCodeReference postEverythingMultiCustomizer = MultiCustomizer.of(new QCodeReference(ClearWebhookSubscriptionHelperMemoizationsTableCustomizer.class), syncScheduledJobCustomizer);
table.withCustomizer(TableCustomizers.POST_INSERT_RECORD, postEverythingMultiCustomizer);
table.withCustomizer(TableCustomizers.POST_UPDATE_RECORD, postEverythingMultiCustomizer);
table.withCustomizer(TableCustomizers.POST_DELETE_RECORD, postEverythingMultiCustomizer);
return (table);
}
/***************************************************************************
*
***************************************************************************/
public static QFieldMetaData addActiveStatusChipAdornment(QFieldMetaData field)
{
return field.withFieldAdornment(new FieldAdornment(AdornmentType.CHIP)
.withValues(AdornmentType.ChipValues.iconAndColorValues(WebhookActiveStatus.ACTIVE, "play_circle_outline", AdornmentType.ChipValues.COLOR_SUCCESS))
.withValues(AdornmentType.ChipValues.iconAndColorValues(WebhookActiveStatus.PAUSED, "pause_circle_outline", AdornmentType.ChipValues.COLOR_WARNING))
.withValues(AdornmentType.ChipValues.iconAndColorValues(WebhookActiveStatus.DISABLED, "not_interested", AdornmentType.ChipValues.COLOR_ERROR)));
}
/***************************************************************************
**
***************************************************************************/
public static QFieldMetaData addHealthStatusChipAdornment(QFieldMetaData field)
{
return field.withFieldAdornment(new FieldAdornment(AdornmentType.CHIP)
.withValues(AdornmentType.ChipValues.iconAndColorValues(WebhookHealthStatus.HEALTHY, "favorite_border", AdornmentType.ChipValues.COLOR_SUCCESS))
.withValues(AdornmentType.ChipValues.iconAndColorValues(WebhookHealthStatus.UNHEALTHY, "heart_broken", AdornmentType.ChipValues.COLOR_ERROR))
.withValues(AdornmentType.ChipValues.iconAndColorValues(WebhookHealthStatus.PROBATION, "healing", AdornmentType.ChipValues.COLOR_WARNING)));
}
}
/***************************************************************************
**
***************************************************************************/
public static class EventAndEventSendLogChildRecordWidgetMetaDataCustomizer implements MetaDataCustomizerInterface<QWidgetMetaData>
{
/***************************************************************************
**
***************************************************************************/
@Override
public QWidgetMetaData customizeMetaData(QInstance qInstance, QWidgetMetaData widget) throws QException
{
////////////////////////////
// sort events by id desc //
////////////////////////////
ArrayList<QFilterOrderBy> orderBys = new ArrayList<>(List.of(new QFilterOrderBy("id", false)));
return widget.withDefaultValue("orderBy", orderBys);
}
}
@QField(isEditable = false, isPrimaryKey = true)
private Integer id;
@QField(maxLength = 100, valueTooLongBehavior = ValueTooLongBehavior.ERROR, isRequired = true)
private String name;
@QField(maxLength = 250, valueTooLongBehavior = ValueTooLongBehavior.ERROR)
private String description;
@QField(maxLength = 250, valueTooLongBehavior = ValueTooLongBehavior.ERROR, isRequired = true, label = "URL")
private String url;
@QField(isRequired = true, possibleValueSourceName = WebhookActiveStatus.NAME, defaultValue = WebhookActiveStatus.DEFAULT_VALUE)
private Integer activeStatusId;
@QField(isRequired = true, possibleValueSourceName = WebhookHealthStatus.NAME, defaultValue = WebhookHealthStatus.DEFAULT_VALUE, isEditable = false)
private Integer healthStatusId;
@QField(isEditable = false)
private Instant createDate;
@QField(isEditable = false)
private Instant modifyDate;
@QAssociation(name = SUBSCRIPTIONS_ASSOCIATION_NAME)
private List<WebhookSubscription> subscriptions;
/*******************************************************************************
** Default constructor
*******************************************************************************/
public Webhook()
{
}
/*******************************************************************************
** Constructor that takes a QRecord
*******************************************************************************/
public Webhook(QRecord record)
{
populateFromQRecord(record);
}
/*******************************************************************************
** Getter for id
*******************************************************************************/
public Integer getId()
{
return (this.id);
}
/*******************************************************************************
** Setter for id
*******************************************************************************/
public void setId(Integer id)
{
this.id = id;
}
/*******************************************************************************
** Fluent setter for id
*******************************************************************************/
public Webhook withId(Integer id)
{
this.id = id;
return (this);
}
/*******************************************************************************
** Getter for name
*******************************************************************************/
public String getName()
{
return (this.name);
}
/*******************************************************************************
** Setter for name
*******************************************************************************/
public void setName(String name)
{
this.name = name;
}
/*******************************************************************************
** Fluent setter for name
*******************************************************************************/
public Webhook withName(String name)
{
this.name = name;
return (this);
}
/*******************************************************************************
** Getter for description
*******************************************************************************/
public String getDescription()
{
return (this.description);
}
/*******************************************************************************
** Setter for description
*******************************************************************************/
public void setDescription(String description)
{
this.description = description;
}
/*******************************************************************************
** Fluent setter for description
*******************************************************************************/
public Webhook withDescription(String description)
{
this.description = description;
return (this);
}
/*******************************************************************************
** Getter for url
*******************************************************************************/
public String getUrl()
{
return (this.url);
}
/*******************************************************************************
** Setter for url
*******************************************************************************/
public void setUrl(String url)
{
this.url = url;
}
/*******************************************************************************
** Fluent setter for url
*******************************************************************************/
public Webhook withUrl(String url)
{
this.url = url;
return (this);
}
/*******************************************************************************
** Getter for createDate
*******************************************************************************/
public Instant getCreateDate()
{
return (this.createDate);
}
/*******************************************************************************
** Setter for createDate
*******************************************************************************/
public void setCreateDate(Instant createDate)
{
this.createDate = createDate;
}
/*******************************************************************************
** Fluent setter for createDate
*******************************************************************************/
public Webhook withCreateDate(Instant createDate)
{
this.createDate = createDate;
return (this);
}
/*******************************************************************************
** Getter for modifyDate
*******************************************************************************/
public Instant getModifyDate()
{
return (this.modifyDate);
}
/*******************************************************************************
** Setter for modifyDate
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Fluent setter for modifyDate
*******************************************************************************/
public Webhook withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
/*******************************************************************************
** Getter for subscriptions
*******************************************************************************/
public List<WebhookSubscription> getSubscriptions()
{
return (this.subscriptions);
}
/*******************************************************************************
** Setter for subscriptions
*******************************************************************************/
public void setSubscriptions(List<WebhookSubscription> subscriptions)
{
this.subscriptions = subscriptions;
}
/*******************************************************************************
** Fluent setter for subscriptions
*******************************************************************************/
public Webhook withSubscriptions(List<WebhookSubscription> subscriptions)
{
this.subscriptions = subscriptions;
return (this);
}
/*******************************************************************************
** Getter for activeStatusId
*******************************************************************************/
public Integer getActiveStatusId()
{
return (this.activeStatusId);
}
/*******************************************************************************
** Setter for activeStatusId
*******************************************************************************/
public void setActiveStatusId(Integer activeStatusId)
{
this.activeStatusId = activeStatusId;
}
/*******************************************************************************
** Fluent setter for activeStatusId
*******************************************************************************/
public Webhook withActiveStatusId(Integer activeStatusId)
{
this.activeStatusId = activeStatusId;
return (this);
}
/*******************************************************************************
** Getter for healthStatusId
*******************************************************************************/
public Integer getHealthStatusId()
{
return (this.healthStatusId);
}
/*******************************************************************************
** Setter for healthStatusId
*******************************************************************************/
public void setHealthStatusId(Integer healthStatusId)
{
this.healthStatusId = healthStatusId;
}
/*******************************************************************************
** Fluent setter for healthStatusId
*******************************************************************************/
public Webhook withHealthStatusId(Integer healthStatusId)
{
this.healthStatusId = healthStatusId;
return (this);
}
}
@@ -1,127 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.model;
import java.util.Objects;
import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.PossibleValueEnum;
import com.kingsrook.qqq.backend.core.model.metadata.producers.annotations.QMetaDataProducingPossibleValueEnum;
/*******************************************************************************
**
*******************************************************************************/
@QMetaDataProducingPossibleValueEnum()
public enum WebhookActiveStatus implements PossibleValueEnum<Integer>
{
ACTIVE(1, "Active"),
PAUSED(2, "Paused"),
DISABLED(3, "Disabled");
private final Integer id;
private final String label;
public static final String NAME = "WebhookActiveStatus";
public static final String DEFAULT_VALUE = "1";
static
{
Objects.requireNonNull(getById(Integer.parseInt(DEFAULT_VALUE)), "Default value '" + DEFAULT_VALUE + " in WebhookActiveStatus is not a defined enum value");
}
/*******************************************************************************
**
*******************************************************************************/
WebhookActiveStatus(Integer id, String label)
{
this.id = id;
this.label = label;
}
/*******************************************************************************
** Get instance by id
**
*******************************************************************************/
public static WebhookActiveStatus getById(Integer id)
{
if(id == null)
{
return (null);
}
for(WebhookActiveStatus value : WebhookActiveStatus.values())
{
if(Objects.equals(value.id, id))
{
return (value);
}
}
return (null);
}
/*******************************************************************************
** Getter for id
**
*******************************************************************************/
public Integer getId()
{
return id;
}
/*******************************************************************************
** Getter for label
**
*******************************************************************************/
public String getLabel()
{
return label;
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public Integer getPossibleValueId()
{
return (getId());
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public String getPossibleValueLabel()
{
return (getLabel());
}
}
@@ -1,557 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.model;
import java.time.Instant;
import java.util.List;
import com.kingsrook.qbits.webhooks.WebhooksQBitConfig;
import com.kingsrook.qbits.webhooks.registry.WebhookEventTypePossibleValueSource;
import com.kingsrook.qbits.webhooks.tables.WebhookEventTableCustomizer;
import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizers;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.data.QAssociation;
import com.kingsrook.qqq.backend.core.model.data.QField;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.model.metadata.fields.AdornmentType;
import com.kingsrook.qqq.backend.core.model.metadata.fields.FieldAdornment;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.ValueTooLongBehavior;
import com.kingsrook.qqq.backend.core.model.metadata.joins.QJoinMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
import com.kingsrook.qqq.backend.core.model.metadata.producers.MetaDataCustomizerInterface;
import com.kingsrook.qqq.backend.core.model.metadata.producers.annotations.ChildJoin;
import com.kingsrook.qqq.backend.core.model.metadata.producers.annotations.ChildRecordListWidget;
import com.kingsrook.qqq.backend.core.model.metadata.producers.annotations.ChildTable;
import com.kingsrook.qqq.backend.core.model.metadata.producers.annotations.QMetaDataProducingEntity;
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitConfig;
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitProductionContext;
import com.kingsrook.qqq.backend.core.model.metadata.security.RecordSecurityLock;
import com.kingsrook.qqq.backend.core.model.metadata.tables.Association;
import com.kingsrook.qqq.backend.core.model.metadata.tables.ExposedJoin;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QFieldSection;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.tables.SectionFactory;
import com.kingsrook.qqq.backend.core.model.tables.QQQTable;
import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
import com.kingsrook.qqq.backend.core.utils.collections.MutableList;
/*******************************************************************************
** QRecord Entity for WebhookEvent table
*******************************************************************************/
@QMetaDataProducingEntity(
producePossibleValueSource = true,
produceTableMetaData = true,
tableMetaDataCustomizer = WebhookEvent.TableMetaDataCustomizer.class,
childTables = {
@ChildTable(
childTableEntityClass = WebhookEventSendLog.class,
joinFieldName = "webhookEventId",
childJoin = @ChildJoin(enabled = true),
childRecordListWidget = @ChildRecordListWidget(label = "Send Log", enabled = true)),
@ChildTable(
childTableEntityClass = WebhookEventContent.class,
joinFieldName = "webhookEventId",
childJoin = @ChildJoin(enabled = true, isOneToOne = true))
}
)
public class WebhookEvent extends QRecordEntity
{
public static final String TABLE_NAME = "webhookEvent";
public static final String CONTENT_ASSOCIATION_NAME = "content";
/***************************************************************************
**
***************************************************************************/
public static class TableMetaDataCustomizer implements MetaDataCustomizerInterface<QTableMetaData>
{
/***************************************************************************
**
***************************************************************************/
@Override
public QTableMetaData customizeMetaData(QInstance qInstance, QTableMetaData table) throws QException
{
String webhookParentJoinName = QJoinMetaData.makeInferredJoinName(Webhook.TABLE_NAME, WebhookEvent.TABLE_NAME);
String webhookSubscriptionParentJoinName = QJoinMetaData.makeInferredJoinName(WebhookSubscription.TABLE_NAME, WebhookEvent.TABLE_NAME);
String contentChildJoinName = QJoinMetaData.makeInferredJoinName(WebhookEvent.TABLE_NAME, WebhookEventContent.TABLE_NAME);
String sendLogChildJoinName = QJoinMetaData.makeInferredJoinName(WebhookEvent.TABLE_NAME, WebhookEventSendLog.TABLE_NAME);
QFieldSection t1section = SectionFactory.defaultT1("id", "webhookId", "webhookSubscriptionId", "webhookEventTypeName");
table
.withIcon(new QIcon().withName("notifications"))
.withRecordLabelFormat("%s")
.withRecordLabelFields("id")
.withSection(t1section)
.withSection(SectionFactory.defaultT2("eventStatusId", "eventSourceRecordQqqTableId", "eventSourceRecordId", "nextAttemptTimestamp"))
.withSection(SectionFactory.customT2("sendLogs", new QIcon("receipt_long")).withWidgetName(sendLogChildJoinName))
.withSection(SectionFactory.customT2("content", new QIcon("description"), WebhookEventContent.TABLE_NAME + ".postBody"))
.withSection(SectionFactory.defaultT3("createDate", "modifyDate"))
.withExposedJoin(new ExposedJoin().withLabel("Webhook").withJoinPath(List.of(webhookParentJoinName)).withJoinTable(Webhook.TABLE_NAME))
.withExposedJoin(new ExposedJoin().withLabel("Subscription").withJoinPath(List.of(webhookSubscriptionParentJoinName)).withJoinTable(WebhookSubscription.TABLE_NAME))
.withExposedJoin(new ExposedJoin().withLabel("Send Log").withJoinPath(List.of(sendLogChildJoinName)).withJoinTable(WebhookEventSendLog.TABLE_NAME))
.withExposedJoin(new ExposedJoin().withLabel("Content").withJoinPath(List.of(contentChildJoinName)).withJoinTable(WebhookEventContent.TABLE_NAME))
.withAssociation(new Association().withName(CONTENT_ASSOCIATION_NAME).withJoinName(contentChildJoinName).withAssociatedTableName(WebhookEventContent.TABLE_NAME));
QBitConfig qBitConfig = QBitProductionContext.peekQBitConfig();
if(qBitConfig instanceof WebhooksQBitConfig webhooksQBitConfig)
{
t1section.setFieldNames(new MutableList<>(t1section.getFieldNames()));
for(QFieldMetaData fieldMetaData : CollectionUtils.nonNullList(webhooksQBitConfig.getSecurityFields()))
{
table.addField(fieldMetaData.clone());
t1section.getFieldNames().add(fieldMetaData.getName());
}
for(RecordSecurityLock recordSecurityLock : CollectionUtils.nonNullList(webhooksQBitConfig.getRecordSecurityLocks()))
{
table.withRecordSecurityLock(recordSecurityLock);
}
}
table.withCustomizer(TableCustomizers.POST_QUERY_RECORD, new QCodeReference(WebhookEventTableCustomizer.class));
table.getField("eventSourceRecordId").withFieldAdornment(new FieldAdornment(AdornmentType.LINK).withValue(AdornmentType.LinkValues.TO_RECORD_FROM_TABLE_DYNAMIC, true));
QFieldMetaData eventStatusIdField = table.getField("eventStatusId");
addEventStatusChipAdornment(eventStatusIdField);
return (table);
}
/***************************************************************************
**
***************************************************************************/
public static QFieldMetaData addEventStatusChipAdornment(QFieldMetaData eventStatusIdField)
{
eventStatusIdField.withFieldAdornment(new FieldAdornment(AdornmentType.CHIP)
.withValues(AdornmentType.ChipValues.iconAndColorValues(WebhookEventStatus.NEW, "pending", AdornmentType.ChipValues.COLOR_DEFAULT))
.withValues(AdornmentType.ChipValues.iconAndColorValues(WebhookEventStatus.SENDING, "send", AdornmentType.ChipValues.COLOR_INFO))
.withValues(AdornmentType.ChipValues.iconAndColorValues(WebhookEventStatus.DELIVERED, "check", AdornmentType.ChipValues.COLOR_SUCCESS))
.withValues(AdornmentType.ChipValues.iconAndColorValues(WebhookEventStatus.AWAITING_RETRY, "hourglass_top", AdornmentType.ChipValues.COLOR_WARNING))
.withValues(AdornmentType.ChipValues.iconAndColorValues(WebhookEventStatus.FAILED, "error", AdornmentType.ChipValues.COLOR_ERROR)));
return eventStatusIdField;
}
}
@QField(isEditable = false, isPrimaryKey = true)
private Integer id;
@QField(possibleValueSourceName = Webhook.TABLE_NAME)
private Integer webhookId;
@QField(possibleValueSourceName = WebhookSubscription.TABLE_NAME)
private Integer webhookSubscriptionId;
@QField(maxLength = 100, valueTooLongBehavior = ValueTooLongBehavior.ERROR, isRequired = true, possibleValueSourceName = WebhookEventTypePossibleValueSource.NAME, label = "Event Type")
private String webhookEventTypeName;
@QField(label = "Event Source Record Table", possibleValueSourceName = QQQTable.TABLE_NAME)
private Integer eventSourceRecordQqqTableId;
@QField()
private Integer eventSourceRecordId;
@QField(possibleValueSourceName = WebhookEventStatus.NAME)
private Integer eventStatusId;
@QField(isEditable = false)
private Instant createDate;
@QField(isEditable = false)
private Instant modifyDate;
@QField(isEditable = false)
private Instant nextAttemptTimestamp;
@QAssociation(name = CONTENT_ASSOCIATION_NAME)
private List<WebhookEventContent> content;
/*******************************************************************************
** Default constructor
*******************************************************************************/
public WebhookEvent()
{
}
/*******************************************************************************
** Constructor that takes a QRecord
*******************************************************************************/
public WebhookEvent(QRecord record)
{
populateFromQRecord(record);
}
/*******************************************************************************
** Getter for id
*******************************************************************************/
public Integer getId()
{
return (this.id);
}
/*******************************************************************************
** Setter for id
*******************************************************************************/
public void setId(Integer id)
{
this.id = id;
}
/*******************************************************************************
** Fluent setter for id
*******************************************************************************/
public WebhookEvent withId(Integer id)
{
this.id = id;
return (this);
}
/*******************************************************************************
** Getter for webhookId
*******************************************************************************/
public Integer getWebhookId()
{
return (this.webhookId);
}
/*******************************************************************************
** Setter for webhookId
*******************************************************************************/
public void setWebhookId(Integer webhookId)
{
this.webhookId = webhookId;
}
/*******************************************************************************
** Fluent setter for webhookId
*******************************************************************************/
public WebhookEvent withWebhookId(Integer webhookId)
{
this.webhookId = webhookId;
return (this);
}
/*******************************************************************************
** Getter for webhookSubscriptionId
*******************************************************************************/
public Integer getWebhookSubscriptionId()
{
return (this.webhookSubscriptionId);
}
/*******************************************************************************
** Setter for webhookSubscriptionId
*******************************************************************************/
public void setWebhookSubscriptionId(Integer webhookSubscriptionId)
{
this.webhookSubscriptionId = webhookSubscriptionId;
}
/*******************************************************************************
** Fluent setter for webhookSubscriptionId
*******************************************************************************/
public WebhookEvent withWebhookSubscriptionId(Integer webhookSubscriptionId)
{
this.webhookSubscriptionId = webhookSubscriptionId;
return (this);
}
/*******************************************************************************
** Getter for webhookEventTypeName
*******************************************************************************/
public String getWebhookEventTypeName()
{
return (this.webhookEventTypeName);
}
/*******************************************************************************
** Setter for webhookEventTypeName
*******************************************************************************/
public void setWebhookEventTypeName(String webhookEventTypeName)
{
this.webhookEventTypeName = webhookEventTypeName;
}
/*******************************************************************************
** Fluent setter for webhookEventTypeName
*******************************************************************************/
public WebhookEvent withWebhookEventTypeName(String webhookEventTypeName)
{
this.webhookEventTypeName = webhookEventTypeName;
return (this);
}
/*******************************************************************************
** Getter for eventSourceRecordQqqTableId
*******************************************************************************/
public Integer getEventSourceRecordQqqTableId()
{
return (this.eventSourceRecordQqqTableId);
}
/*******************************************************************************
** Setter for eventSourceRecordQqqTableId
*******************************************************************************/
public void setEventSourceRecordQqqTableId(Integer eventSourceRecordQqqTableId)
{
this.eventSourceRecordQqqTableId = eventSourceRecordQqqTableId;
}
/*******************************************************************************
** Fluent setter for eventSourceRecordQqqTableId
*******************************************************************************/
public WebhookEvent withEventSourceRecordQqqTableId(Integer eventSourceRecordQqqTableId)
{
this.eventSourceRecordQqqTableId = eventSourceRecordQqqTableId;
return (this);
}
/*******************************************************************************
** Getter for eventSourceRecordId
*******************************************************************************/
public Integer getEventSourceRecordId()
{
return (this.eventSourceRecordId);
}
/*******************************************************************************
** Setter for eventSourceRecordId
*******************************************************************************/
public void setEventSourceRecordId(Integer eventSourceRecordId)
{
this.eventSourceRecordId = eventSourceRecordId;
}
/*******************************************************************************
** Fluent setter for eventSourceRecordId
*******************************************************************************/
public WebhookEvent withEventSourceRecordId(Integer eventSourceRecordId)
{
this.eventSourceRecordId = eventSourceRecordId;
return (this);
}
/*******************************************************************************
** Getter for eventStatusId
*******************************************************************************/
public Integer getEventStatusId()
{
return (this.eventStatusId);
}
/*******************************************************************************
** Setter for eventStatusId
*******************************************************************************/
public void setEventStatusId(Integer eventStatusId)
{
this.eventStatusId = eventStatusId;
}
/*******************************************************************************
** Fluent setter for eventStatusId
*******************************************************************************/
public WebhookEvent withEventStatusId(Integer eventStatusId)
{
this.eventStatusId = eventStatusId;
return (this);
}
/*******************************************************************************
** Getter for createDate
*******************************************************************************/
public Instant getCreateDate()
{
return (this.createDate);
}
/*******************************************************************************
** Setter for createDate
*******************************************************************************/
public void setCreateDate(Instant createDate)
{
this.createDate = createDate;
}
/*******************************************************************************
** Fluent setter for createDate
*******************************************************************************/
public WebhookEvent withCreateDate(Instant createDate)
{
this.createDate = createDate;
return (this);
}
/*******************************************************************************
** Getter for modifyDate
*******************************************************************************/
public Instant getModifyDate()
{
return (this.modifyDate);
}
/*******************************************************************************
** Setter for modifyDate
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Fluent setter for modifyDate
*******************************************************************************/
public WebhookEvent withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
/*******************************************************************************
** Getter for content
*******************************************************************************/
public List<WebhookEventContent> getContent()
{
return (this.content);
}
/*******************************************************************************
** Setter for content
*******************************************************************************/
public void setContent(List<WebhookEventContent> content)
{
this.content = content;
}
/*******************************************************************************
** Fluent setter for content
*******************************************************************************/
public WebhookEvent withContent(List<WebhookEventContent> content)
{
this.content = content;
return (this);
}
/*******************************************************************************
** Getter for nextAttemptTimestamp
*******************************************************************************/
public Instant getNextAttemptTimestamp()
{
return (this.nextAttemptTimestamp);
}
/*******************************************************************************
** Setter for nextAttemptTimestamp
*******************************************************************************/
public void setNextAttemptTimestamp(Instant nextAttemptTimestamp)
{
this.nextAttemptTimestamp = nextAttemptTimestamp;
}
/*******************************************************************************
** Fluent setter for nextAttemptTimestamp
*******************************************************************************/
public WebhookEvent withNextAttemptTimestamp(Instant nextAttemptTimestamp)
{
this.nextAttemptTimestamp = nextAttemptTimestamp;
return (this);
}
}
@@ -1,196 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.model;
import java.util.Objects;
import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.PossibleValueEnum;
import com.kingsrook.qqq.backend.core.model.metadata.producers.annotations.QMetaDataProducingPossibleValueEnum;
/*******************************************************************************
** WebhookEventCategory - possible value enum
*******************************************************************************/
@QMetaDataProducingPossibleValueEnum()
public enum WebhookEventCategory implements PossibleValueEnum<String>
{
INSERT("insert", "Record Inserted", Kind.INSERT, true, false, false),
INSERT_WITH_FIELD("insert-with-field", "Record Inserted with Field", Kind.INSERT, true, true, false),
INSERT_WITH_VALUE("insert-with-value", "Record Inserted with Specific Value", Kind.INSERT, true, true, true),
UPDATE("update", "Record Updated", Kind.UPDATE, true, false, false),
UPDATE_WITH_FIELD("update-with-field", "Record Updated with Field", Kind.UPDATE, true, true, false),
UPDATE_WITH_VALUE("update-with-value", "Record Updated with Specific Value", Kind.UPDATE, true, true, true),
STORE("store", "Record Stored", Kind.STORE, true, false, false),
STORE_WITH_FIELD("store-with-field", "Record Stored with Field", Kind.STORE, true, true, false),
STORE_WITH_VALUE("store-with-value", "Record Stored with Specific Value", Kind.STORE, true, true, true),
AD_HOC("ad-hoc", "Ad Hoc", Kind.AD_HOC, false, false, false);
private final String id;
private final String label;
private final Kind kind;
private final boolean requiresTable;
private final boolean requiresField;
private final boolean requiresValue;
public static final String NAME = "WebhookEventCategory";
/***************************************************************************
**
***************************************************************************/
public enum Kind
{
INSERT,
UPDATE,
STORE,
AD_HOC
}
/*******************************************************************************
**
*******************************************************************************/
WebhookEventCategory(String id, String label, Kind kind, boolean requiresTable, boolean requiresField, boolean requiresValue)
{
this.id = id;
this.label = label;
this.kind = kind;
this.requiresTable = requiresTable;
this.requiresField = requiresField;
this.requiresValue = requiresValue;
}
/*******************************************************************************
** Get instance by id
**
*******************************************************************************/
public static WebhookEventCategory getById(String id)
{
if(id == null)
{
return (null);
}
for(WebhookEventCategory value : WebhookEventCategory.values())
{
if(Objects.equals(value.id, id))
{
return (value);
}
}
return (null);
}
/*******************************************************************************
** Getter for id
**
*******************************************************************************/
public String getId()
{
return id;
}
/*******************************************************************************
** Getter for label
**
*******************************************************************************/
public String getLabel()
{
return label;
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public String getPossibleValueId()
{
return (getId());
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public String getPossibleValueLabel()
{
return (getLabel());
}
/*******************************************************************************
** Getter for requiresTable
**
*******************************************************************************/
public boolean getRequiresTable()
{
return requiresTable;
}
/*******************************************************************************
** Getter for requiresField
**
*******************************************************************************/
public boolean getRequiresField()
{
return requiresField;
}
/*******************************************************************************
** Getter for requiresValue
**
*******************************************************************************/
public boolean getRequiresValue()
{
return requiresValue;
}
/*******************************************************************************
** Getter for kind
**
*******************************************************************************/
public Kind getKind()
{
return kind;
}
}
@@ -1,235 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.model;
import java.util.List;
import com.kingsrook.qbits.webhooks.WebhooksQBitConfig;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.data.QField;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.fields.AdornmentType;
import com.kingsrook.qqq.backend.core.model.metadata.fields.FieldAdornment;
import com.kingsrook.qqq.backend.core.model.metadata.joins.QJoinMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
import com.kingsrook.qqq.backend.core.model.metadata.producers.MetaDataCustomizerInterface;
import com.kingsrook.qqq.backend.core.model.metadata.producers.annotations.QMetaDataProducingEntity;
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitConfig;
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitProductionContext;
import com.kingsrook.qqq.backend.core.model.metadata.security.RecordSecurityLock;
import com.kingsrook.qqq.backend.core.model.metadata.tables.ExposedJoin;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.tables.SectionFactory;
import com.kingsrook.qqq.backend.core.model.metadata.tables.UniqueKey;
import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
/*******************************************************************************
** QRecord Entity for WebhookEventContent table
*******************************************************************************/
@QMetaDataProducingEntity(
producePossibleValueSource = true,
produceTableMetaData = true,
tableMetaDataCustomizer = WebhookEventContent.TableMetaDataCustomizer.class
)
public class WebhookEventContent extends QRecordEntity
{
public static final String TABLE_NAME = "webhookEventContent";
/***************************************************************************
**
***************************************************************************/
public static class TableMetaDataCustomizer implements MetaDataCustomizerInterface<QTableMetaData>
{
/***************************************************************************
**
***************************************************************************/
@Override
public QTableMetaData customizeMetaData(QInstance qInstance, QTableMetaData table) throws QException
{
String parentJoinName = QJoinMetaData.makeInferredJoinName(WebhookEvent.TABLE_NAME, WebhookEventContent.TABLE_NAME);
table
.withIcon(new QIcon().withName("description"))
.withUniqueKey(new UniqueKey("webhookEventId"))
.withRecordLabelFormat("Event %s")
.withRecordLabelFields("webhookEventId")
.withSection(SectionFactory.defaultT1("id", "webhookEventId"))
.withSection(SectionFactory.defaultT2("postBody"))
.withExposedJoin(new ExposedJoin().withLabel("Event").withJoinPath(List.of(parentJoinName)).withJoinTable(WebhookEvent.TABLE_NAME));
table.getField("postBody").withFieldAdornment(new FieldAdornment(AdornmentType.CODE_EDITOR)
.withValue(AdornmentType.CodeEditorValues.languageMode("json")));
QBitConfig qBitConfig = QBitProductionContext.peekQBitConfig();
if(qBitConfig instanceof WebhooksQBitConfig webhooksQBitConfig)
{
for(RecordSecurityLock recordSecurityLock : CollectionUtils.nonNullList(webhooksQBitConfig.getRecordSecurityLocks()))
{
RecordSecurityLock lockClone = recordSecurityLock.clone();
lockClone.setFieldName(WebhookEvent.TABLE_NAME + "." + lockClone.getFieldName());
lockClone.setJoinNameChain(List.of(parentJoinName));
table.withRecordSecurityLock(lockClone);
}
}
return (table);
}
}
@QField(isEditable = false, isPrimaryKey = true)
private Integer id;
@QField(possibleValueSourceName = WebhookEvent.TABLE_NAME)
private Integer webhookEventId;
///////////////////////////
// text, so no maxLength //
///////////////////////////
@QField()
private String postBody;
// todo? ///////////////////////////
// todo? // text, so no maxLength //
// todo? ///////////////////////////
// todo? @QField()
// todo? private String headers;
/*******************************************************************************
** Default constructor
*******************************************************************************/
public WebhookEventContent()
{
}
/*******************************************************************************
** Constructor that takes a QRecord
*******************************************************************************/
public WebhookEventContent(QRecord record)
{
populateFromQRecord(record);
}
/*******************************************************************************
** Getter for id
*******************************************************************************/
public Integer getId()
{
return (this.id);
}
/*******************************************************************************
** Setter for id
*******************************************************************************/
public void setId(Integer id)
{
this.id = id;
}
/*******************************************************************************
** Fluent setter for id
*******************************************************************************/
public WebhookEventContent withId(Integer id)
{
this.id = id;
return (this);
}
/*******************************************************************************
** Getter for webhookEventId
*******************************************************************************/
public Integer getWebhookEventId()
{
return (this.webhookEventId);
}
/*******************************************************************************
** Setter for webhookEventId
*******************************************************************************/
public void setWebhookEventId(Integer webhookEventId)
{
this.webhookEventId = webhookEventId;
}
/*******************************************************************************
** Fluent setter for webhookEventId
*******************************************************************************/
public WebhookEventContent withWebhookEventId(Integer webhookEventId)
{
this.webhookEventId = webhookEventId;
return (this);
}
/*******************************************************************************
** Getter for postBody
*******************************************************************************/
public String getPostBody()
{
return (this.postBody);
}
/*******************************************************************************
** Setter for postBody
*******************************************************************************/
public void setPostBody(String postBody)
{
this.postBody = postBody;
}
/*******************************************************************************
** Fluent setter for postBody
*******************************************************************************/
public WebhookEventContent withPostBody(String postBody)
{
this.postBody = postBody;
return (this);
}
}
@@ -1,431 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.model;
import java.time.Instant;
import java.util.List;
import com.kingsrook.qbits.webhooks.WebhooksQBitConfig;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.data.QField;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.fields.AdornmentType;
import com.kingsrook.qqq.backend.core.model.metadata.fields.FieldAdornment;
import com.kingsrook.qqq.backend.core.model.metadata.fields.ValueTooLongBehavior;
import com.kingsrook.qqq.backend.core.model.metadata.joins.QJoinMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
import com.kingsrook.qqq.backend.core.model.metadata.producers.MetaDataCustomizerInterface;
import com.kingsrook.qqq.backend.core.model.metadata.producers.annotations.QMetaDataProducingEntity;
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitConfig;
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitProductionContext;
import com.kingsrook.qqq.backend.core.model.metadata.security.RecordSecurityLock;
import com.kingsrook.qqq.backend.core.model.metadata.tables.ExposedJoin;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.tables.SectionFactory;
import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
/*******************************************************************************
** QRecord Entity for WebhookEventSendLog table
*******************************************************************************/
@QMetaDataProducingEntity(
producePossibleValueSource = true,
produceTableMetaData = true,
tableMetaDataCustomizer = WebhookEventSendLog.TableMetaDataCustomizer.class
)
public class WebhookEventSendLog extends QRecordEntity
{
public static final String TABLE_NAME = "webhookEventSendLog";
/***************************************************************************
**
***************************************************************************/
public static class TableMetaDataCustomizer implements MetaDataCustomizerInterface<QTableMetaData>
{
/***************************************************************************
**
***************************************************************************/
@Override
public QTableMetaData customizeMetaData(QInstance qInstance, QTableMetaData table) throws QException
{
String parentJoinName = QJoinMetaData.makeInferredJoinName(WebhookEvent.TABLE_NAME, WebhookEventSendLog.TABLE_NAME);
table
.withIcon(new QIcon().withName("receipt_long"))
.withRecordLabelFormat("Event %s Attempt %s")
.withRecordLabelFields("webhookEventId", "attemptNo")
.withSection(SectionFactory.defaultT1("id", "webhookId", "webhookEventId", "attemptNo"))
.withSection(SectionFactory.defaultT2("successful", "httpStatusCode", "errorMessage"))
.withSection(SectionFactory.defaultT3("startTimestamp", "endTimestamp"))
.withExposedJoin(new ExposedJoin().withLabel("Event").withJoinPath(List.of(parentJoinName)).withJoinTable(WebhookEvent.TABLE_NAME));
table.getField("successful").withFieldAdornment(new FieldAdornment(AdornmentType.CHIP)
.withValues(AdornmentType.ChipValues.iconAndColorValues(true, "done", AdornmentType.ChipValues.COLOR_SUCCESS))
.withValues(AdornmentType.ChipValues.iconAndColorValues(false, "error", AdornmentType.ChipValues.COLOR_ERROR)));
QBitConfig qBitConfig = QBitProductionContext.peekQBitConfig();
if(qBitConfig instanceof WebhooksQBitConfig webhooksQBitConfig)
{
for(RecordSecurityLock recordSecurityLock : CollectionUtils.nonNullList(webhooksQBitConfig.getRecordSecurityLocks()))
{
RecordSecurityLock lockClone = recordSecurityLock.clone();
lockClone.setFieldName(WebhookEvent.TABLE_NAME + "." + lockClone.getFieldName());
lockClone.setJoinNameChain(List.of(parentJoinName));
table.withRecordSecurityLock(lockClone);
}
}
return (table);
}
}
@QField(isEditable = false, isPrimaryKey = true)
private Integer id;
@QField(possibleValueSourceName = Webhook.TABLE_NAME)
private Integer webhookId;
@QField(possibleValueSourceName = WebhookEvent.TABLE_NAME)
private Integer webhookEventId;
@QField(label = "HTTP Status Code")
private Integer httpStatusCode;
@QField(maxLength = 100, valueTooLongBehavior = ValueTooLongBehavior.TRUNCATE_ELLIPSIS)
private String errorMessage;
@QField()
private Boolean successful;
@QField()
private Integer attemptNo;
@QField(isEditable = false)
private Instant startTimestamp;
@QField(isEditable = false)
private Instant endTimestamp;
/*******************************************************************************
** Default constructor
*******************************************************************************/
public WebhookEventSendLog()
{
}
/*******************************************************************************
** Constructor that takes a QRecord
*******************************************************************************/
public WebhookEventSendLog(QRecord record)
{
populateFromQRecord(record);
}
/*******************************************************************************
** Getter for id
*******************************************************************************/
public Integer getId()
{
return (this.id);
}
/*******************************************************************************
** Setter for id
*******************************************************************************/
public void setId(Integer id)
{
this.id = id;
}
/*******************************************************************************
** Fluent setter for id
*******************************************************************************/
public WebhookEventSendLog withId(Integer id)
{
this.id = id;
return (this);
}
/*******************************************************************************
** Getter for webhookEventId
*******************************************************************************/
public Integer getWebhookEventId()
{
return (this.webhookEventId);
}
/*******************************************************************************
** Setter for webhookEventId
*******************************************************************************/
public void setWebhookEventId(Integer webhookEventId)
{
this.webhookEventId = webhookEventId;
}
/*******************************************************************************
** Fluent setter for webhookEventId
*******************************************************************************/
public WebhookEventSendLog withWebhookEventId(Integer webhookEventId)
{
this.webhookEventId = webhookEventId;
return (this);
}
/*******************************************************************************
** Getter for httpStatusCode
*******************************************************************************/
public Integer getHttpStatusCode()
{
return (this.httpStatusCode);
}
/*******************************************************************************
** Setter for httpStatusCode
*******************************************************************************/
public void setHttpStatusCode(Integer httpStatusCode)
{
this.httpStatusCode = httpStatusCode;
}
/*******************************************************************************
** Fluent setter for httpStatusCode
*******************************************************************************/
public WebhookEventSendLog withHttpStatusCode(Integer httpStatusCode)
{
this.httpStatusCode = httpStatusCode;
return (this);
}
/*******************************************************************************
** Getter for successful
*******************************************************************************/
public Boolean getSuccessful()
{
return (this.successful);
}
/*******************************************************************************
** Setter for successful
*******************************************************************************/
public void setSuccessful(Boolean successful)
{
this.successful = successful;
}
/*******************************************************************************
** Fluent setter for successful
*******************************************************************************/
public WebhookEventSendLog withSuccessful(Boolean successful)
{
this.successful = successful;
return (this);
}
/*******************************************************************************
** Getter for attemptNo
*******************************************************************************/
public Integer getAttemptNo()
{
return (this.attemptNo);
}
/*******************************************************************************
** Setter for attemptNo
*******************************************************************************/
public void setAttemptNo(Integer attemptNo)
{
this.attemptNo = attemptNo;
}
/*******************************************************************************
** Fluent setter for attemptNo
*******************************************************************************/
public WebhookEventSendLog withAttemptNo(Integer attemptNo)
{
this.attemptNo = attemptNo;
return (this);
}
/*******************************************************************************
** Getter for startTimestamp
*******************************************************************************/
public Instant getStartTimestamp()
{
return (this.startTimestamp);
}
/*******************************************************************************
** Setter for startTimestamp
*******************************************************************************/
public void setStartTimestamp(Instant startTimestamp)
{
this.startTimestamp = startTimestamp;
}
/*******************************************************************************
** Fluent setter for startTimestamp
*******************************************************************************/
public WebhookEventSendLog withStartTimestamp(Instant startTimestamp)
{
this.startTimestamp = startTimestamp;
return (this);
}
/*******************************************************************************
** Getter for endTimestamp
*******************************************************************************/
public Instant getEndTimestamp()
{
return (this.endTimestamp);
}
/*******************************************************************************
** Setter for endTimestamp
*******************************************************************************/
public void setEndTimestamp(Instant endTimestamp)
{
this.endTimestamp = endTimestamp;
}
/*******************************************************************************
** Fluent setter for endTimestamp
*******************************************************************************/
public WebhookEventSendLog withEndTimestamp(Instant endTimestamp)
{
this.endTimestamp = endTimestamp;
return (this);
}
/*******************************************************************************
** Getter for errorMessage
*******************************************************************************/
public String getErrorMessage()
{
return (this.errorMessage);
}
/*******************************************************************************
** Setter for errorMessage
*******************************************************************************/
public void setErrorMessage(String errorMessage)
{
this.errorMessage = errorMessage;
}
/*******************************************************************************
** Fluent setter for errorMessage
*******************************************************************************/
public WebhookEventSendLog withErrorMessage(String errorMessage)
{
this.errorMessage = errorMessage;
return (this);
}
/*******************************************************************************
** Getter for webhookId
*******************************************************************************/
public Integer getWebhookId()
{
return (this.webhookId);
}
/*******************************************************************************
** Setter for webhookId
*******************************************************************************/
public void setWebhookId(Integer webhookId)
{
this.webhookId = webhookId;
}
/*******************************************************************************
** Fluent setter for webhookId
*******************************************************************************/
public WebhookEventSendLog withWebhookId(Integer webhookId)
{
this.webhookId = webhookId;
return (this);
}
}
@@ -1,129 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.model;
import java.util.Objects;
import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.PossibleValueEnum;
import com.kingsrook.qqq.backend.core.model.metadata.producers.annotations.QMetaDataProducingPossibleValueEnum;
/*******************************************************************************
**
*******************************************************************************/
@QMetaDataProducingPossibleValueEnum()
public enum WebhookEventStatus implements PossibleValueEnum<Integer>
{
NEW(1, "New"),
SENDING(2, "Sending"),
DELIVERED(3, "Delivered"),
AWAITING_RETRY(4, "Awaiting Retry"),
FAILED(5, "Failed");
private final Integer id;
private final String label;
public static final String NAME = "WebhookEventStatus";
public static final String DEFAULT_VALUE = "1";
static
{
Objects.requireNonNull(getById(Integer.parseInt(DEFAULT_VALUE)), "Default value '" + DEFAULT_VALUE + " in WebhookEventStatus is not a defined enum value");
}
/*******************************************************************************
**
*******************************************************************************/
WebhookEventStatus(Integer id, String label)
{
this.id = id;
this.label = label;
}
/*******************************************************************************
** Get instance by id
**
*******************************************************************************/
public static WebhookEventStatus getById(Integer id)
{
if(id == null)
{
return (null);
}
for(WebhookEventStatus value : WebhookEventStatus.values())
{
if(Objects.equals(value.id, id))
{
return (value);
}
}
return (null);
}
/*******************************************************************************
** Getter for id
**
*******************************************************************************/
public Integer getId()
{
return id;
}
/*******************************************************************************
** Getter for label
**
*******************************************************************************/
public String getLabel()
{
return label;
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public Integer getPossibleValueId()
{
return (getId());
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public String getPossibleValueLabel()
{
return (getLabel());
}
}
@@ -1,127 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.model;
import java.util.Objects;
import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.PossibleValueEnum;
import com.kingsrook.qqq.backend.core.model.metadata.producers.annotations.QMetaDataProducingPossibleValueEnum;
/*******************************************************************************
**
*******************************************************************************/
@QMetaDataProducingPossibleValueEnum()
public enum WebhookHealthStatus implements PossibleValueEnum<Integer>
{
HEALTHY(1, "Healthy"),
UNHEALTHY(2, "Unhealthy"),
PROBATION(3, "Probation");
private final Integer id;
private final String label;
public static final String NAME = "WebhookHealthStatus";
public static final String DEFAULT_VALUE = "1";
static
{
Objects.requireNonNull(getById(Integer.parseInt(DEFAULT_VALUE)), "Default value '" + DEFAULT_VALUE + " in WebhookHealthStatus is not a defined enum value");
}
/*******************************************************************************
**
*******************************************************************************/
WebhookHealthStatus(Integer id, String label)
{
this.id = id;
this.label = label;
}
/*******************************************************************************
** Get instance by id
**
*******************************************************************************/
public static WebhookHealthStatus getById(Integer id)
{
if(id == null)
{
return (null);
}
for(WebhookHealthStatus value : WebhookHealthStatus.values())
{
if(Objects.equals(value.id, id))
{
return (value);
}
}
return (null);
}
/*******************************************************************************
** Getter for id
**
*******************************************************************************/
public Integer getId()
{
return id;
}
/*******************************************************************************
** Getter for label
**
*******************************************************************************/
public String getLabel()
{
return label;
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public Integer getPossibleValueId()
{
return (getId());
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public String getPossibleValueLabel()
{
return (getLabel());
}
}
@@ -1,471 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.model;
import java.time.Instant;
import java.util.ArrayList;
import java.util.List;
import com.kingsrook.qbits.webhooks.WebhooksQBitConfig;
import com.kingsrook.qbits.webhooks.actions.ClearWebhookSubscriptionHelperMemoizationsTableCustomizer;
import com.kingsrook.qbits.webhooks.registry.WebhookEventTypePossibleValueSource;
import com.kingsrook.qqq.api.model.metadata.ApiInstanceMetaDataProvider;
import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizers;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
import com.kingsrook.qqq.backend.core.model.data.QField;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.AdornmentType;
import com.kingsrook.qqq.backend.core.model.metadata.fields.FieldAdornment;
import com.kingsrook.qqq.backend.core.model.metadata.fields.ValueTooLongBehavior;
import com.kingsrook.qqq.backend.core.model.metadata.joins.QJoinMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
import com.kingsrook.qqq.backend.core.model.metadata.producers.MetaDataCustomizerInterface;
import com.kingsrook.qqq.backend.core.model.metadata.producers.annotations.ChildJoin;
import com.kingsrook.qqq.backend.core.model.metadata.producers.annotations.ChildRecordListWidget;
import com.kingsrook.qqq.backend.core.model.metadata.producers.annotations.ChildTable;
import com.kingsrook.qqq.backend.core.model.metadata.producers.annotations.QMetaDataProducingEntity;
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitConfig;
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitProductionContext;
import com.kingsrook.qqq.backend.core.model.metadata.security.RecordSecurityLock;
import com.kingsrook.qqq.backend.core.model.metadata.tables.ExposedJoin;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.tables.SectionFactory;
import com.kingsrook.qqq.backend.core.model.metadata.tables.UniqueKey;
import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
import com.kingsrook.qqq.backend.core.utils.StringUtils;
/*******************************************************************************
** QRecord Entity for WebhookSubscription table
*******************************************************************************/
@QMetaDataProducingEntity(
producePossibleValueSource = true,
produceTableMetaData = true,
tableMetaDataCustomizer = WebhookSubscription.TableMetaDataCustomizer.class,
childTables = {
@ChildTable(
childTableEntityClass = WebhookEvent.class,
joinFieldName = "webhookId",
childJoin = @ChildJoin(enabled = true),
childRecordListWidget = @ChildRecordListWidget(label = "Events", enabled = true, maxRows = 25, widgetMetaDataCustomizer = WebhookSubscription.EventChildRecordWidgetMetaDataCustomizer.class))
}
)
public class WebhookSubscription extends QRecordEntity
{
public static final String TABLE_NAME = "webhookSubscription";
/***************************************************************************
**
***************************************************************************/
public static class TableMetaDataCustomizer implements MetaDataCustomizerInterface<QTableMetaData>
{
/***************************************************************************
**
***************************************************************************/
@Override
public QTableMetaData customizeMetaData(QInstance qInstance, QTableMetaData table) throws QException
{
String parentJoinName = QJoinMetaData.makeInferredJoinName(Webhook.TABLE_NAME, WebhookSubscription.TABLE_NAME);
String childJoinName = QJoinMetaData.makeInferredJoinName(WebhookSubscription.TABLE_NAME, WebhookEvent.TABLE_NAME);
table
.withIcon(new QIcon().withName("subscriptions"))
.withRecordLabelFormat("%s - %s")
.withRecordLabelFields("webhookId", "webhookEventTypeName")
.withUniqueKey(new UniqueKey("webhookEventTypeName", "webhookId"))
.withSection(SectionFactory.defaultT1("id", "webhookId", "webhookEventTypeName"))
.withSection(SectionFactory.defaultT2("activeStatusId", "apiName", "apiVersion"))
// todo - if adding filters in future .withSection(SectionFactory.customT2("hidden", new QIcon().withName("hidden"), "queryFilterJson").withIsHidden(true))
// todo - if adding filters in future .withSection(SectionFactory.customT2("filter", new QIcon("filter_alt")).withWidgetName(WebhookSubscriptionQueryFilterWidgetMetaDataProducer.NAME))
.withSection(SectionFactory.customT2("events", new QIcon("notifications")).withWidgetName(childJoinName))
.withSection(SectionFactory.defaultT3("createDate", "modifyDate"))
.withExposedJoin(new ExposedJoin().withLabel("Webhook").withJoinPath(List.of(parentJoinName)).withJoinTable(Webhook.TABLE_NAME));
QBitConfig qBitConfig = QBitProductionContext.peekQBitConfig();
if(qBitConfig instanceof WebhooksQBitConfig webhooksQBitConfig)
{
if(StringUtils.hasContent(webhooksQBitConfig.getDefaultApiNameForNewSubscriptions()))
{
table.getField("apiName").withDefaultValue(webhooksQBitConfig.getDefaultApiNameForNewSubscriptions());
}
if(StringUtils.hasContent(webhooksQBitConfig.getDefaultApiVersionForNewSubscriptions()))
{
table.getField("apiVersion").withDefaultValue(webhooksQBitConfig.getDefaultApiVersionForNewSubscriptions());
}
for(RecordSecurityLock recordSecurityLock : CollectionUtils.nonNullList(webhooksQBitConfig.getRecordSecurityLocks()))
{
RecordSecurityLock lockClone = recordSecurityLock.clone();
lockClone.setFieldName(Webhook.TABLE_NAME + "." + lockClone.getFieldName());
lockClone.setJoinNameChain(List.of(parentJoinName));
table.withRecordSecurityLock(lockClone);
}
}
table.getField("activeStatusId").withFieldAdornment(new FieldAdornment(AdornmentType.CHIP)
.withValues(AdornmentType.ChipValues.iconAndColorValues(WebhookActiveStatus.ACTIVE, "play_circle_outline", AdornmentType.ChipValues.COLOR_SUCCESS))
.withValues(AdornmentType.ChipValues.iconAndColorValues(WebhookActiveStatus.PAUSED, "pause_circle_outline", AdornmentType.ChipValues.COLOR_WARNING))
.withValues(AdornmentType.ChipValues.iconAndColorValues(WebhookActiveStatus.DISABLED, "not_interested", AdornmentType.ChipValues.COLOR_ERROR)));
// todo - if adding filters in future table.getField("queryFilterJson").withBehavior(new FilterJsonFieldDisplayValueFormatter());
///////////////////////////////////////////////////////////////////////////////////////////
// to clear the webhookSubscriptionHelper class's memoizations post insert/update/delete //
///////////////////////////////////////////////////////////////////////////////////////////
QCodeReference clearWebhookSubscriptionHerlpMemoizations = new QCodeReference(ClearWebhookSubscriptionHelperMemoizationsTableCustomizer.class);
table.withCustomizer(TableCustomizers.POST_INSERT_RECORD, clearWebhookSubscriptionHerlpMemoizations);
table.withCustomizer(TableCustomizers.POST_UPDATE_RECORD, clearWebhookSubscriptionHerlpMemoizations);
table.withCustomizer(TableCustomizers.POST_DELETE_RECORD, clearWebhookSubscriptionHerlpMemoizations);
return (table);
}
}
/***************************************************************************
**
***************************************************************************/
public static class EventChildRecordWidgetMetaDataCustomizer implements MetaDataCustomizerInterface<QWidgetMetaData>
{
/***************************************************************************
**
***************************************************************************/
@Override
public QWidgetMetaData customizeMetaData(QInstance qInstance, QWidgetMetaData widget) throws QException
{
////////////////////////////
// sort events by id desc //
////////////////////////////
ArrayList<QFilterOrderBy> orderBys = new ArrayList<>(List.of(new QFilterOrderBy("id", false)));
return widget.withDefaultValue("orderBy", orderBys);
}
}
@QField(isEditable = false, isPrimaryKey = true)
private Integer id;
@QField(isRequired = true, possibleValueSourceName = Webhook.TABLE_NAME)
private Integer webhookId;
@QField(maxLength = 100, valueTooLongBehavior = ValueTooLongBehavior.ERROR, isRequired = true, possibleValueSourceName = WebhookEventTypePossibleValueSource.NAME, label = "Event Type")
private String webhookEventTypeName;
//////////////////////////////
// text, so omit max length //
//////////////////////////////
// todo - if adding filters in future @QField(label = "Filter")
// todo - if adding filters in future private String queryFilterJson;
@QField(isRequired = true, possibleValueSourceName = WebhookActiveStatus.NAME, defaultValue = WebhookActiveStatus.DEFAULT_VALUE)
private Integer activeStatusId;
@QField(isRequired = true, maxLength = 50, valueTooLongBehavior = ValueTooLongBehavior.ERROR, label = "API Name", possibleValueSourceName = ApiInstanceMetaDataProvider.API_NAME_PVS_NAME)
private String apiName;
@QField(isRequired = true, maxLength = 50, valueTooLongBehavior = ValueTooLongBehavior.ERROR, label = "API Version", possibleValueSourceName = ApiInstanceMetaDataProvider.API_VERSION_PVS_NAME)
private String apiVersion;
@QField(isRequired = true, isEditable = false)
private Instant createDate;
@QField(isEditable = false)
private Instant modifyDate;
/*******************************************************************************
** Default constructor
*******************************************************************************/
public WebhookSubscription()
{
}
/*******************************************************************************
** Constructor that takes a QRecord
*******************************************************************************/
public WebhookSubscription(QRecord record)
{
populateFromQRecord(record);
}
/*******************************************************************************
** Getter for id
*******************************************************************************/
public Integer getId()
{
return (this.id);
}
/*******************************************************************************
** Setter for id
*******************************************************************************/
public void setId(Integer id)
{
this.id = id;
}
/*******************************************************************************
** Fluent setter for id
*******************************************************************************/
public WebhookSubscription withId(Integer id)
{
this.id = id;
return (this);
}
/*******************************************************************************
** Getter for webhookId
*******************************************************************************/
public Integer getWebhookId()
{
return (this.webhookId);
}
/*******************************************************************************
** Setter for webhookId
*******************************************************************************/
public void setWebhookId(Integer webhookId)
{
this.webhookId = webhookId;
}
/*******************************************************************************
** Fluent setter for webhookId
*******************************************************************************/
public WebhookSubscription withWebhookId(Integer webhookId)
{
this.webhookId = webhookId;
return (this);
}
/*******************************************************************************
** Getter for webhookEventTypeName
*******************************************************************************/
public String getWebhookEventTypeName()
{
return (this.webhookEventTypeName);
}
/*******************************************************************************
** Setter for webhookEventTypeName
*******************************************************************************/
public void setWebhookEventTypeName(String webhookEventTypeName)
{
this.webhookEventTypeName = webhookEventTypeName;
}
/*******************************************************************************
** Fluent setter for webhookEventTypeName
*******************************************************************************/
public WebhookSubscription withWebhookEventTypeName(String webhookEventTypeName)
{
this.webhookEventTypeName = webhookEventTypeName;
return (this);
}
/*******************************************************************************
** Getter for apiName
*******************************************************************************/
public String getApiName()
{
return (this.apiName);
}
/*******************************************************************************
** Setter for apiName
*******************************************************************************/
public void setApiName(String apiName)
{
this.apiName = apiName;
}
/*******************************************************************************
** Fluent setter for apiName
*******************************************************************************/
public WebhookSubscription withApiName(String apiName)
{
this.apiName = apiName;
return (this);
}
/*******************************************************************************
** Getter for apiVersion
*******************************************************************************/
public String getApiVersion()
{
return (this.apiVersion);
}
/*******************************************************************************
** Setter for apiVersion
*******************************************************************************/
public void setApiVersion(String apiVersion)
{
this.apiVersion = apiVersion;
}
/*******************************************************************************
** Fluent setter for apiVersion
*******************************************************************************/
public WebhookSubscription withApiVersion(String apiVersion)
{
this.apiVersion = apiVersion;
return (this);
}
/*******************************************************************************
** Getter for createDate
*******************************************************************************/
public Instant getCreateDate()
{
return (this.createDate);
}
/*******************************************************************************
** Setter for createDate
*******************************************************************************/
public void setCreateDate(Instant createDate)
{
this.createDate = createDate;
}
/*******************************************************************************
** Fluent setter for createDate
*******************************************************************************/
public WebhookSubscription withCreateDate(Instant createDate)
{
this.createDate = createDate;
return (this);
}
/*******************************************************************************
** Getter for modifyDate
*******************************************************************************/
public Instant getModifyDate()
{
return (this.modifyDate);
}
/*******************************************************************************
** Setter for modifyDate
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Fluent setter for modifyDate
*******************************************************************************/
public WebhookSubscription withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
/*******************************************************************************
** Getter for activeStatusId
*******************************************************************************/
public Integer getActiveStatusId()
{
return (this.activeStatusId);
}
/*******************************************************************************
** Setter for activeStatusId
*******************************************************************************/
public void setActiveStatusId(Integer activeStatusId)
{
this.activeStatusId = activeStatusId;
}
/*******************************************************************************
** Fluent setter for activeStatusId
*******************************************************************************/
public WebhookSubscription withActiveStatusId(Integer activeStatusId)
{
this.activeStatusId = activeStatusId;
return (this);
}
}
@@ -1,37 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.model;
import com.kingsrook.qqq.backend.core.model.actions.tables.ActionFlag;
/*******************************************************************************
** flags that can be put in an insert/update/delete event, to control webhook
** events.
*******************************************************************************/
public enum WebhooksActionFlags implements ActionFlag
{
/***************************************************************************
** cause webhook events to not be fired for the action.
***************************************************************************/
OMIT_WEBHOOKS
}
@@ -1,74 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.processes;
import java.util.List;
import com.kingsrook.qbits.webhooks.model.Webhook;
import com.kingsrook.qbits.webhooks.model.WebhookHealthStatus;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducer;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ExtractViaQueryStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.LoadViaUpdateStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
/*******************************************************************************
** Meta Data Producer for ManageWebhookHealth
*******************************************************************************/
public class ManageWebhookHealthProcessMetaDataProducer extends MetaDataProducer<QProcessMetaData>
{
public static final String NAME = "ManageWebhookHealth";
/*******************************************************************************
**
*******************************************************************************/
@Override
public QProcessMetaData produce(QInstance qInstance) throws QException
{
QProcessMetaData processMetaData = StreamedETLWithFrontendProcess.processMetaDataBuilder()
.withName(NAME)
.withTableName(Webhook.TABLE_NAME)
.withIcon(new QIcon().withName("healing"))
.withExtractStepClass(ExtractViaQueryStep.class)
.withTransformStepClass(ManageWebhookHealthTransformStep.class)
.withLoadStepClass(LoadViaUpdateStep.class)
.withSourceTable(Webhook.TABLE_NAME)
.withDestinationTable(Webhook.TABLE_NAME)
.withPreviewMessage(StreamedETLWithFrontendProcess.DEFAULT_PREVIEW_MESSAGE_FOR_UPDATE)
.withReviewStepRecordFields(List.of(
new QFieldMetaData("id", QFieldType.INTEGER),
new QFieldMetaData("name", QFieldType.STRING),
Webhook.TableMetaDataCustomizer.addHealthStatusChipAdornment(new QFieldMetaData("healthStatusId", QFieldType.INTEGER).withPossibleValueSourceName(WebhookHealthStatus.NAME))
))
.getProcessMetaData();
return (processMetaData);
}
}
@@ -1,151 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.processes;
import java.time.Instant;
import java.time.temporal.ChronoUnit;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import com.kingsrook.qbits.webhooks.WebhooksQBitConfig;
import com.kingsrook.qbits.webhooks.model.Webhook;
import com.kingsrook.qbits.webhooks.model.WebhookEventSendLog;
import com.kingsrook.qbits.webhooks.model.WebhookHealthStatus;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractTransformStep;
import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
/*******************************************************************************
** check if any unhealthy webhooks are ready to go into Probation
*******************************************************************************/
public class ManageWebhookHealthTransformStep extends AbstractTransformStep
{
private static final QLogger LOG = QLogger.getLogger(ManageWebhookHealthTransformStep.class);
private ProcessSummaryLine willPutOnProbationLine = new ProcessSummaryLine(Status.OK)
.withMessageSuffix(" updated from Unhealthy to Probation")
.withSingularFutureMessage("will be")
.withPluralFutureMessage("will be")
.withSingularPastMessage("was")
.withPluralPastMessage("were");
private ProcessSummaryLine notUnhealthyLine = new ProcessSummaryLine(Status.INFO)
.withMessageSuffix(" not currently Unhealthy")
.withSingularFutureMessage("is")
.withPluralFutureMessage("are")
.withSingularPastMessage("was")
.withPluralPastMessage("were");
private ProcessSummaryLine notUnhealthyLongEnoughLine = new ProcessSummaryLine(Status.ERROR)
.withMessageSuffix(" Unhealthy long enough to be put on Probation")
.withSingularFutureMessage("has not been")
.withPluralFutureMessage("have not been")
.withSingularPastMessage("was not")
.withPluralPastMessage("were not");
/*******************************************************************************
**
*******************************************************************************/
@Override
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput runBackendStepOutput, boolean isForResultScreen)
{
ArrayList<ProcessSummaryLineInterface> rs = new ArrayList<>();
willPutOnProbationLine.addSelfToListIfAnyCount(rs);
notUnhealthyLine.addSelfToListIfAnyCount(rs);
notUnhealthyLongEnoughLine.addSelfToListIfAnyCount(rs);
return (rs);
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public void runOnePage(RunBackendStepInput runBackendStepInput, RunBackendStepOutput runBackendStepOutput) throws QException
{
Set<Webhook> unhealthy = new HashSet<>();
for(Webhook webhook : runBackendStepInput.getRecordsAsEntities(Webhook.class))
{
if(WebhookHealthStatus.UNHEALTHY.getId().equals(webhook.getHealthStatusId()))
{
unhealthy.add(webhook);
}
else
{
notUnhealthyLine.incrementCountAndAddPrimaryKey(webhook.getId());
}
}
if(CollectionUtils.nullSafeIsEmpty(unhealthy))
{
return;
}
Integer timeoutMinutes = WebhooksQBitConfig.getConfigValue(config -> config.getUnhealthyToProbationTimeoutMinutes());
Instant limitTime = Instant.now().minus(timeoutMinutes, ChronoUnit.MINUTES);
for(Webhook webhook : unhealthy)
{
List<WebhookEventSendLog> eventLogs = new QueryAction().execute(new QueryInput(WebhookEventSendLog.TABLE_NAME).withFilter(new QQueryFilter()
.withCriteria(new QFilterCriteria("webhookId", QCriteriaOperator.EQUALS, webhook.getId()))
.withOrderBy(new QFilterOrderBy("id", false))
.withLimit(1))).getRecordEntities(WebhookEventSendLog.class);
Instant lastEventTimestamp = Instant.now();
if(!eventLogs.isEmpty() && eventLogs.get(0).getStartTimestamp() != null)
{
lastEventTimestamp = eventLogs.get(0).getStartTimestamp();
}
if(lastEventTimestamp.isBefore(limitTime))
{
willPutOnProbationLine.incrementCountAndAddPrimaryKey(webhook.getId());
runBackendStepOutput.addRecord(new QRecord()
.withValue("id", webhook.getId())
.withValue("name", webhook.getName()) // just for preview screen
.withValue("healthStatusId", WebhookHealthStatus.PROBATION.getId()));
}
else
{
notUnhealthyLongEnoughLine.incrementCountAndAddPrimaryKey(webhook.getId());
}
}
}
}
@@ -1,170 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.processes;
import java.time.Instant;
import java.util.ArrayList;
import java.util.List;
import com.kingsrook.qbits.webhooks.actions.WebhookEventSender;
import com.kingsrook.qbits.webhooks.model.Webhook;
import com.kingsrook.qbits.webhooks.model.WebhookEvent;
import com.kingsrook.qbits.webhooks.model.WebhookEventContent;
import com.kingsrook.qbits.webhooks.model.WebhookEventSendLog;
import com.kingsrook.qbits.webhooks.model.WebhookHealthStatus;
import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractLoadStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ProcessSummaryProviderInterface;
import com.kingsrook.qqq.backend.core.processes.implementations.general.ProcessSummaryWarningsAndErrorsRollup;
import org.apache.commons.lang3.BooleanUtils;
/*******************************************************************************
** sends test events to webhooks.
*******************************************************************************/
public class SendTestEventToWebhookLoadStep extends AbstractLoadStep implements ProcessSummaryProviderInterface
{
private static final QLogger LOG = QLogger.getLogger(SendTestEventToWebhookLoadStep.class);
private ProcessSummaryLine okLine = new ProcessSummaryLine(Status.OK)
.withMessageSuffix(" delivered successfully")
.withSingularPastMessage("was")
.withPluralPastMessage("were");
private ProcessSummaryLine updatedToProbationLine = new ProcessSummaryLine(Status.INFO)
.withMessageSuffix(" updated from Unhealthy to Probation")
.withSingularPastMessage("was")
.withPluralPastMessage("were");
private ProcessSummaryWarningsAndErrorsRollup errorsRollup = new ProcessSummaryWarningsAndErrorsRollup()
.withDoReplaceSingletonCountLinesWithSuffixOnly(false)
.withErrorTemplate(new ProcessSummaryLine(Status.ERROR)
.withSingularPastMessage("failed to send with error message: ")
.withPluralPastMessage("failed to send with error message: "))
.withOtherErrorsSummary(new ProcessSummaryLine(Status.ERROR)
.withSingularPastMessage("had an other error.")
.withPluralPastMessage("had other errors."));
/***************************************************************************
**
***************************************************************************/
@Override
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput runBackendStepOutput, boolean isForResultScreen)
{
ArrayList<ProcessSummaryLineInterface> rs = new ArrayList<>();
okLine.addSelfToListIfAnyCount(rs);
updatedToProbationLine.addSelfToListIfAnyCount(rs);
errorsRollup.addToList(rs);
return rs;
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public void runOnePage(RunBackendStepInput runBackendStepInput, RunBackendStepOutput runBackendStepOutput) throws QException
{
boolean uponSuccessUpdateUnhealthyWebhooksToProbation = BooleanUtils.isTrue(runBackendStepInput.getValueBoolean("uponSuccessUpdateUnhealthyWebhooksToProbation"));
List<QRecord> webhooksToUpdate = new ArrayList<>();
for(Webhook webhook : runBackendStepInput.getRecordsAsEntities(Webhook.class))
{
WebhookEventSendLog sendLog;
try
{
WebhookEvent event = new WebhookEvent().withContent(List.of(new WebhookEventContent()
.withPostBody("""
{
"webhookEventDetails":
{
"webhookEventTypeName": "testEvent",
"eventTimestamp": "%s",
"tableName": null,
"apiName": null,
"apiVersion": null
},
"record":
{
}
}""".formatted(Instant.now().toString()))
));
sendLog = getWebhookEventSender().post(event, webhook);
}
catch(Exception e)
{
sendLog = new WebhookEventSendLog()
.withSuccessful(false)
.withErrorMessage(e.getMessage());
}
if(sendLog.getSuccessful())
{
if(WebhookHealthStatus.UNHEALTHY.getId().equals(webhook.getHealthStatusId()))
{
webhooksToUpdate.add(new QRecord().withValue("id", webhook.getId()).withValue("healthStatusId", WebhookHealthStatus.PROBATION.getId()));
updatedToProbationLine.incrementCountAndAddPrimaryKey(webhook.getId());
}
okLine.incrementCountAndAddPrimaryKey(webhook.getId());
}
else
{
if(sendLog.getHttpStatusCode() != null)
{
errorsRollup.addError("Status Code: " + sendLog.getHttpStatusCode() + "; " + sendLog.getErrorMessage(), webhook.getId());
}
else
{
errorsRollup.addError(sendLog.getErrorMessage(), webhook.getId());
}
}
}
if(uponSuccessUpdateUnhealthyWebhooksToProbation && !webhooksToUpdate.isEmpty())
{
new UpdateAction().execute(new UpdateInput(Webhook.TABLE_NAME).withRecords(webhooksToUpdate));
}
}
/***************************************************************************
**
***************************************************************************/
protected WebhookEventSender getWebhookEventSender()
{
return new WebhookEventSender();
}
}
@@ -1,86 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.processes;
import java.util.List;
import com.kingsrook.qbits.webhooks.model.Webhook;
import com.kingsrook.qbits.webhooks.model.WebhookHealthStatus;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducer;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QComponentType;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendComponentMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendStepMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ExtractViaQueryStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.NoopTransformStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
/*******************************************************************************
** Meta Data Producer for SendTestEventToWebhook
*******************************************************************************/
public class SendTestEventToWebhookProcessMetaDataProducer extends MetaDataProducer<QProcessMetaData>
{
public static final String NAME = "SendTestEventToWebhook";
/*******************************************************************************
**
*******************************************************************************/
@Override
public QProcessMetaData produce(QInstance qInstance) throws QException
{
QProcessMetaData processMetaData = StreamedETLWithFrontendProcess.processMetaDataBuilder()
.withName(NAME)
.withTableName(Webhook.TABLE_NAME)
.withIcon(new QIcon().withName("quiz"))
.withExtractStepClass(ExtractViaQueryStep.class)
.withTransformStepClass(NoopTransformStep.class)
.withLoadStepClass(SendTestEventToWebhookLoadStep.class)
.withSourceTable(Webhook.TABLE_NAME)
.withDestinationTable(Webhook.TABLE_NAME)
.withPreviewMessage(StreamedETLWithFrontendProcess.DEFAULT_PREVIEW_MESSAGE_PREFIX + " sent a test event")
.withReviewStepRecordFields(List.of(
new QFieldMetaData("id", QFieldType.INTEGER),
new QFieldMetaData("name", QFieldType.STRING),
new QFieldMetaData("url", QFieldType.STRING),
Webhook.TableMetaDataCustomizer.addHealthStatusChipAdornment(new QFieldMetaData("healthStatusId", QFieldType.INTEGER).withPossibleValueSourceName(WebhookHealthStatus.NAME))
))
.getProcessMetaData();
processMetaData.withStep(0, new QFrontendStepMetaData()
.withName("input")
.withComponent(new QFrontendComponentMetaData()
.withType(QComponentType.EDIT_FORM))
.withFormField(new QFieldMetaData("uponSuccessUpdateUnhealthyWebhooksToProbation", QFieldType.BOOLEAN)
.withDefaultValue("true")
.withLabel("Upon Success Update Unhealthy Webhooks To Probation?")));
return (processMetaData);
}
}
@@ -1,129 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.processes;
import java.time.Instant;
import com.kingsrook.qbits.webhooks.model.Webhook;
import com.kingsrook.qbits.webhooks.model.WebhookActiveStatus;
import com.kingsrook.qbits.webhooks.model.WebhookEventStatus;
import com.kingsrook.qbits.webhooks.model.WebhookHealthStatus;
import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ExtractViaQueryStep;
import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
/*******************************************************************************
**
*******************************************************************************/
public class SendWebhookEventExtractStep extends ExtractViaQueryStep
{
private static final QLogger LOG = QLogger.getLogger(SendWebhookEventExtractStep.class);
private static QQueryFilter falseFilter = new QQueryFilter(new QFilterCriteria("id", QCriteriaOperator.FALSE));
/***************************************************************************
**
***************************************************************************/
@Override
protected QQueryFilter getQueryFilter(RunBackendStepInput runBackendStepInput) throws QException
{
/////////////////////////////////////////////////////////////////////////////////////////////////////////
// if we've been asked to run for a specific webhook, then look up events that need sent for that hook //
/////////////////////////////////////////////////////////////////////////////////////////////////////////
Integer webhookId = runBackendStepInput.getValueInteger("webhookId");
if(webhookId != null)
{
return getQueryFilterForSingleWebhook(webhookId);
}
return super.getQueryFilter(runBackendStepInput);
}
/***************************************************************************
*
***************************************************************************/
private static QQueryFilter getQueryFilterForSingleWebhook(Integer webhookId) throws QException
{
QRecord webhookRecord = GetAction.execute(Webhook.TABLE_NAME, webhookId);
if(webhookRecord == null)
{
LOG.debug("Running for non-existing webhook - returning query to find no rows.", logPair("webhookId", webhookId));
return (falseFilter);
}
/////////////////////////////////////////////
// only return records for active webhooks //
/////////////////////////////////////////////
Webhook webhook = new Webhook(webhookRecord);
if(!WebhookActiveStatus.ACTIVE.getId().equals(webhook.getActiveStatusId()))
{
LOG.debug("Running for non-active webhook - returning query to find no rows.", logPair("webhookId", webhookId));
return falseFilter;
}
/////////////////////////////////////////////////
// don't return records for unhealthy webhooks //
/////////////////////////////////////////////////
if(WebhookHealthStatus.UNHEALTHY.getId().equals(webhook.getHealthStatusId()))
{
LOG.debug("Running for non-healthy webhook - returning query to find no rows.", logPair("webhookId", webhookId));
return falseFilter;
}
///////////////////////////////////////////////////////////////////////////////////
// webhook_id = ? //
// AND //
// ( //
// (status = NEW) //
// OR (status IN (SENDING, AWAITING_RETRY) AND next_attempt_timestamp < now()) //
// ) //
///////////////////////////////////////////////////////////////////////////////////
QQueryFilter filter = new QQueryFilter()
.withCriteria(new QFilterCriteria("webhookId", QCriteriaOperator.EQUALS, webhookId));
QQueryFilter orStatusFilters = new QQueryFilter().withBooleanOperator(QQueryFilter.BooleanOperator.OR);
filter.addSubFilter(orStatusFilters);
orStatusFilters.addSubFilter(new QQueryFilter()
.withCriteria(new QFilterCriteria("eventStatusId", QCriteriaOperator.EQUALS, WebhookEventStatus.NEW.getId())));
orStatusFilters.addSubFilter(new QQueryFilter()
.withCriteria(new QFilterCriteria("eventStatusId", QCriteriaOperator.IN, WebhookEventStatus.SENDING.getId(), WebhookEventStatus.AWAITING_RETRY.getId()))
.withCriteria(new QFilterCriteria("nextAttemptTimestamp", QCriteriaOperator.IS_NOT_BLANK))
.withCriteria(new QFilterCriteria("nextAttemptTimestamp", QCriteriaOperator.LESS_THAN, Instant.now())));
filter.addOrderBy(new QFilterOrderBy("id"));
return (filter);
}
}
@@ -1,259 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.processes;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Stack;
import com.kingsrook.qbits.webhooks.WebhooksQBitConfig;
import com.kingsrook.qbits.webhooks.actions.WebhookEventSender;
import com.kingsrook.qbits.webhooks.actions.WebhookHealthManager;
import com.kingsrook.qbits.webhooks.model.Webhook;
import com.kingsrook.qbits.webhooks.model.WebhookEvent;
import com.kingsrook.qbits.webhooks.model.WebhookEventSendLog;
import com.kingsrook.qbits.webhooks.model.WebhookHealthStatus;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractLoadStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ProcessSummaryProviderInterface;
import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
import com.kingsrook.qqq.backend.core.utils.ListingHash;
import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
import static com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator.IN;
/*******************************************************************************
**
*******************************************************************************/
public class SendWebhookEventLoadStep extends AbstractLoadStep implements ProcessSummaryProviderInterface
{
private static final QLogger LOG = QLogger.getLogger(SendWebhookEventLoadStep.class);
///////////////////////
// most recent first //
///////////////////////
private boolean doHealthChecks = false;
private Integer repeatedFailsToGoUnhealthy;
private Stack<WebhookEventSendLog> recentSendLogs;
private ProcessSummaryLine okLine = new ProcessSummaryLine(Status.OK)
.withMessageSuffix(" delivered")
.withSingularPastMessage("was")
.withPluralPastMessage("were");
private ProcessSummaryLine failLine = new ProcessSummaryLine(Status.ERROR, "failed to deliver");
private ProcessSummaryLine becameUnhealthyLine = new ProcessSummaryLine(Status.WARNING)
.withMessageSuffix(" associated webhook became unhealthy.")
.withSingularPastMessage("was not sent because its")
.withPluralPastMessage("were not sent because their");
/***************************************************************************
**
***************************************************************************/
@Override
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput runBackendStepOutput, boolean isForResultScreen)
{
ArrayList<ProcessSummaryLineInterface> rs = getTransformStep().getProcessSummary(runBackendStepOutput, isForResultScreen);
///////////////////////////////////////////////////////////////////////////
// replace the OK and WARNING lines from the transform step with our own //
///////////////////////////////////////////////////////////////////////////
rs.removeIf(p -> p instanceof ProcessSummaryLine psl && (psl.getStatus().equals(Status.OK) || psl.getStatus().equals(Status.WARNING)));
if(okLine.getCount() != null && okLine.getCount() > 0)
{
rs.add(0, okLine);
}
if(failLine.getCount() != null && failLine.getCount() > 0)
{
rs.add(0, failLine);
}
becameUnhealthyLine.addSelfToListIfAnyCount(rs);
return (rs);
}
/***************************************************************************
**
***************************************************************************/
@Override
public void preRun(RunBackendStepInput runBackendStepInput, RunBackendStepOutput runBackendStepOutput) throws QException
{
repeatedFailsToGoUnhealthy = WebhooksQBitConfig.getConfigValue(config -> config.getRepeatedFailsToGoUnhealthy());
if(repeatedFailsToGoUnhealthy != null)
{
recentSendLogs = newSendLogStack(repeatedFailsToGoUnhealthy);
doHealthChecks = true;
}
}
/***************************************************************************
**
***************************************************************************/
@Override
public void runOnePage(RunBackendStepInput runBackendStepInput, RunBackendStepOutput runBackendStepOutput) throws QException
{
if(runBackendStepInput.getRecords().isEmpty())
{
return;
}
Webhook webhook = ((SendWebhookEventTransformStep) getTransformStep()).getWebhook();
///////////////////////////////////////////////////////////////////////////////////////////
// look up the last several send logs for this webhook, to seed the recentSuccesses list //
///////////////////////////////////////////////////////////////////////////////////////////
if(doHealthChecks)
{
List<WebhookEventSendLog> recentSendLogs = QueryAction.execute(WebhookEventSendLog.TABLE_NAME, WebhookEventSendLog.class, new QQueryFilter()
.withCriteria(new QFilterCriteria("webhookId", QCriteriaOperator.EQUALS, webhook.getId()))
.withOrderBy(new QFilterOrderBy("id", false))
.withLimit(repeatedFailsToGoUnhealthy));
Collections.reverse(recentSendLogs);
this.recentSendLogs.addAll(recentSendLogs);
}
///////////////////////////////////////////////////
// look up send logs for all events in this page //
///////////////////////////////////////////////////
List<WebhookEvent> webhookEvents = runBackendStepInput.getRecordsAsEntities(WebhookEvent.class);
List<WebhookEventSendLog> eventLogs = QueryAction.execute(WebhookEventSendLog.TABLE_NAME, WebhookEventSendLog.class, new QQueryFilter(
new QFilterCriteria("webhookEventId", IN, webhookEvents.stream().map(e -> e.getId()).toList())));
ListingHash<Integer, WebhookEventSendLog> eventLogsByEventId = CollectionUtils.listToListingHash(eventLogs, l -> l.getWebhookEventId());
boolean becameUnhealthy = false;
for(WebhookEvent webhookEvent : webhookEvents)
{
if(becameUnhealthy)
{
becameUnhealthyLine.incrementCountAndAddPrimaryKey(webhookEvent.getId());
continue;
}
try
{
boolean lastSuccessful = newWebhookEventSender().handleEvent(webhookEvent, webhook, eventLogsByEventId.getOrDefault(webhookEvent.getId(), Collections.emptyList()));
if(lastSuccessful)
{
okLine.incrementCountAndAddPrimaryKey(webhookEvent.getId());
}
else
{
failLine.incrementCountAndAddPrimaryKey(webhookEvent.getId());
}
////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// if we're to update health, then add this last-successful boolean to the stack, and call the health manager //
////////////////////////////////////////////////////////////////////////////////////////////////////////////////
if(doHealthChecks)
{
recentSendLogs.push(new WebhookEventSendLog()
.withSuccessful(lastSuccessful));
webhook = new WebhookHealthManager().updateWebhookHealth(recentSendLogs, webhook);
}
if(WebhookHealthStatus.UNHEALTHY.getId().equals(webhook.getHealthStatusId()))
{
LOG.info("Webhook is now unhealthy - not attempting any more sends.", logPair("id", webhook.getId()));
becameUnhealthy = true;
}
}
catch(Exception e)
{
LOG.warn("Error handing webhook event", e, logPair("id", webhookEvent.getId()));
}
}
}
/***************************************************************************
** meant so class can be overridden to use a different sender
***************************************************************************/
protected WebhookEventSender newWebhookEventSender()
{
return new WebhookEventSender();
}
/***************************************************************************
* meant for tests - to make a stack of send log events
***************************************************************************/
public Stack<WebhookEventSendLog> newSendLogStack(int maxSize)
{
return (new FixedStack<>(maxSize));
}
/***************************************************************************
*
***************************************************************************/
static class FixedStack<T> extends Stack<T>
{
private final int maxSize;
/***************************************************************************
*
***************************************************************************/
public FixedStack(int maxSize)
{
this.maxSize = maxSize;
}
/***************************************************************************
*
***************************************************************************/
@Override
public T push(T item)
{
if(size() >= maxSize)
{
remove(0); // remove bottom element
}
return super.push(item); // then push on top
}
}
}
@@ -1,81 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.processes;
import java.util.List;
import com.kingsrook.qbits.webhooks.model.Webhook;
import com.kingsrook.qbits.webhooks.model.WebhookEvent;
import com.kingsrook.qbits.webhooks.model.WebhookEventStatus;
import com.kingsrook.qbits.webhooks.model.WebhookSubscription;
import com.kingsrook.qbits.webhooks.registry.WebhookEventTypePossibleValueSource;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducer;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
import com.kingsrook.qqq.backend.core.model.tables.QQQTable;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
/*******************************************************************************
** Meta Data Producer for SendWebhookEventProcess
*******************************************************************************/
public class SendWebhookEventProcessMetaDataProducer extends MetaDataProducer<QProcessMetaData>
{
public static final String NAME = "SendWebhookEvent";
/*******************************************************************************
**
*******************************************************************************/
@Override
public QProcessMetaData produce(QInstance qInstance) throws QException
{
QProcessMetaData processMetaData = StreamedETLWithFrontendProcess.processMetaDataBuilder()
.withName(NAME)
.withTableName(WebhookEvent.TABLE_NAME)
.withIcon(new QIcon().withName("send"))
.withExtractStepClass(SendWebhookEventExtractStep.class)
.withTransformStepClass(SendWebhookEventTransformStep.class)
.withLoadStepClass(SendWebhookEventLoadStep.class)
.withSourceTable(WebhookEvent.TABLE_NAME)
.withDestinationTable(WebhookEvent.TABLE_NAME)
.withPreviewMessage(StreamedETLWithFrontendProcess.DEFAULT_PREVIEW_MESSAGE_PREFIX + " considered for sending")
.withReviewStepRecordFields(List.of(
new QFieldMetaData("id", QFieldType.INTEGER),
new QFieldMetaData("webhookId", QFieldType.INTEGER).withPossibleValueSourceName(Webhook.TABLE_NAME),
new QFieldMetaData("webhookSubscriptionId", QFieldType.INTEGER).withPossibleValueSourceName(WebhookSubscription.TABLE_NAME),
new QFieldMetaData("webhookEventTypeName", QFieldType.STRING).withLabel("Event Type").withPossibleValueSourceName(WebhookEventTypePossibleValueSource.NAME),
new QFieldMetaData("eventSourceRecordQqqTableId", QFieldType.INTEGER).withLabel("Event Source Record Table").withPossibleValueSourceName(QQQTable.TABLE_NAME),
new QFieldMetaData("eventSourceRecordId", QFieldType.INTEGER),
WebhookEvent.TableMetaDataCustomizer.addEventStatusChipAdornment(new QFieldMetaData("eventStatusId", QFieldType.INTEGER).withPossibleValueSourceName(WebhookEventStatus.NAME))
))
.withInputFieldDefaultValue(StreamedETLWithFrontendProcess.FIELD_INCLUDE_ASSOCIATIONS, true)
.getProcessMetaData();
return (processMetaData);
}
}
@@ -1,312 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.processes;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Set;
import java.util.stream.Collectors;
import com.kingsrook.qbits.webhooks.model.Webhook;
import com.kingsrook.qbits.webhooks.model.WebhookActiveStatus;
import com.kingsrook.qbits.webhooks.model.WebhookEvent;
import com.kingsrook.qbits.webhooks.model.WebhookEventStatus;
import com.kingsrook.qbits.webhooks.model.WebhookHealthStatus;
import com.kingsrook.qbits.webhooks.model.WebhookSubscription;
import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractTransformStep;
/*******************************************************************************
**
*******************************************************************************/
public class SendWebhookEventTransformStep extends AbstractTransformStep
{
private Integer webhookId;
private Webhook webhook;
private Map<Integer, WebhookSubscription> subscriptionMap = new HashMap<>();
private ProcessSummaryLine okLine = new ProcessSummaryLine(Status.OK)
.withMessageSuffix(" sent")
.withSingularFutureMessage("will be")
.withPluralFutureMessage("will be")
.withSingularPastMessage("was")
.withPluralPastMessage("were");
private ProcessSummaryLine differentWebhookLine = new ProcessSummaryLine(Status.ERROR)
.withMessageSuffix(" sent because events were found for more than one webhook")
.withSingularFutureMessage("will not be")
.withPluralFutureMessage("will not be")
.withSingularPastMessage("was not")
.withPluralPastMessage("were not");
private ProcessSummaryLine missingWebhookLine = new ProcessSummaryLine(Status.ERROR)
.withMessageSuffix(" associated webhook was not found")
.withSingularFutureMessage("will not be sent because its")
.withPluralFutureMessage("will not be sent because their")
.withSingularPastMessage("was not sent because its")
.withPluralPastMessage("were not sent because their");
private ProcessSummaryLine disabledSubscriptionLine = new ProcessSummaryLine(Status.ERROR)
.withSingularFutureMessage("will not be sent because its associated subscription is disabled")
.withPluralFutureMessage("will not be sent because their associated subscriptions are disabled")
.withSingularPastMessage("was not sent because its associated subscription is disabled")
.withPluralPastMessage("were not sent because their associated subscriptions are disabled");
private ProcessSummaryLine disabledWebhookLine = new ProcessSummaryLine(Status.ERROR)
.withSingularFutureMessage("will not be sent because its associated webhook is disabled")
.withPluralFutureMessage("will not be sent because their associated webhooks disabled")
.withSingularPastMessage("was not sent because its associated webhook is disabled")
.withPluralPastMessage("were not sent because their associated webhooks are disabled");
private ProcessSummaryLine unhealthyWebhookLine = new ProcessSummaryLine(Status.ERROR)
.withSingularFutureMessage("will not be sent because its associated webhook is unhealthy")
.withPluralFutureMessage("will not be sent because their associated webhooks unhealthy")
.withSingularPastMessage("was not sent because its associated webhook is unhealthy")
.withPluralPastMessage("were not sent because their associated webhooks are unhealthy");
private ProcessSummaryLine missingSubscriptionLine = new ProcessSummaryLine(Status.ERROR)
.withMessageSuffix(" associated subscription was not found")
.withSingularFutureMessage("will not be sent because its")
.withPluralFutureMessage("will not be sent because their")
.withSingularPastMessage("was not sent because its")
.withPluralPastMessage("were not sent because their");
private ProcessSummaryLine alreadyDeliveredWarningLine = new ProcessSummaryLine(Status.WARNING)
.withSingularFutureMessage("was already successfully delivered, but will be sent again")
.withPluralFutureMessage("were already successfully delivered, but will be sent again")
.withSingularPastMessage("was already successfully delivered, but was sent again")
.withPluralPastMessage("were already successfully delivered, but were sent again");
/***************************************************************************
**
***************************************************************************/
@Override
public void preRun(RunBackendStepInput runBackendStepInput, RunBackendStepOutput runBackendStepOutput) throws QException
{
////////////////////////////////////////////////////////////////
// capture if we're running for a single specified webhook Id //
////////////////////////////////////////////////////////////////
webhookId = runBackendStepInput.getValueInteger("webhookId");
super.preRun(runBackendStepInput, runBackendStepOutput);
}
/***************************************************************************
**
***************************************************************************/
@Override
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput runBackendStepOutput, boolean isForResultScreen)
{
ArrayList<ProcessSummaryLineInterface> rs = new ArrayList<>();
okLine.addSelfToListIfAnyCount(rs);
alreadyDeliveredWarningLine.addSelfToListIfAnyCount(rs);
differentWebhookLine.addSelfToListIfAnyCount(rs);
missingWebhookLine.addSelfToListIfAnyCount(rs);
missingSubscriptionLine.addSelfToListIfAnyCount(rs);
disabledWebhookLine.addSelfToListIfAnyCount(rs);
unhealthyWebhookLine.addSelfToListIfAnyCount(rs);
disabledSubscriptionLine.addSelfToListIfAnyCount(rs);
return rs;
}
/***************************************************************************
**
***************************************************************************/
@Override
public void runOnePage(RunBackendStepInput runBackendStepInput, RunBackendStepOutput runBackendStepOutput) throws QException
{
////////////////////////////////////////////////////
// fetch the subscriptions for the webhook events //
////////////////////////////////////////////////////
List<WebhookEvent> webhookEvents = runBackendStepInput.getRecordsAsEntities(WebhookEvent.class);
Set<Integer> subscriptionIds = webhookEvents.stream().map(we -> we.getWebhookSubscriptionId()).collect(Collectors.toSet());
loadSubscriptions(subscriptionIds);
for(WebhookEvent webhookEvent : webhookEvents)
{
if(webhookEvent.getWebhookId() == null)
{
//////////////////////////////////////////////////
// a little bit of a misnomer, but close enough //
//////////////////////////////////////////////////
missingWebhookLine.incrementCountAndAddPrimaryKey(webhookEvent.getId());
continue;
}
if(webhookEvent.getWebhookSubscriptionId() == null || !subscriptionIds.contains(webhookEvent.getWebhookSubscriptionId()))
{
missingSubscriptionLine.incrementCountAndAddPrimaryKey(webhookEvent.getId());
continue;
}
/////////////////////////////////////////////////////////////////////////////////////////////
// if we aren't running for a specific webhook, then grab the id off the first one we find //
/////////////////////////////////////////////////////////////////////////////////////////////
if(webhookId == null)
{
webhookId = webhookEvent.getWebhookId();
}
//////////////////////////////////////////////////////////////////////////////////
// in case events from multiple webhooks were found, only allow them for one //
// (the first one we found). this will simplify sending, at least at this time. //
//////////////////////////////////////////////////////////////////////////////////
if(!Objects.equals(webhookId, webhookEvent.getWebhookId()))
{
differentWebhookLine.incrementCountAndAddPrimaryKey(webhookEvent.getId());
continue;
}
///////////////////////////////////////////////////////////////////////////////////////////////////////////////
// if we haven't loaded the webhook record yet, then do so now (should be on the first/only one we're using) //
///////////////////////////////////////////////////////////////////////////////////////////////////////////////
if(webhook == null && webhookId != null)
{
QRecord webhookRecord = GetAction.execute(Webhook.TABLE_NAME, webhookId);
if(webhookRecord == null)
{
/////////////////////////////////////////////////////////////////////////////////////////
// if that webhook is missing, skip this line. //
// note, if we had many records selected, all for a non-existing webhook, we'd do this //
// query a bunch, but, not too concerned about that scenario realistically happening. //
/////////////////////////////////////////////////////////////////////////////////////////
missingWebhookLine.incrementCountAndAddPrimaryKey(webhookEvent.getId());
continue;
}
webhook = new Webhook(webhookRecord);
}
if(webhook == null)
{
//////////////////////////////////////////////////////////////////////////////////////////////
// I think this is redundant, but it stops compiler warnings about webhook maybe being null //
//////////////////////////////////////////////////////////////////////////////////////////////
missingWebhookLine.incrementCountAndAddPrimaryKey(webhookEvent.getId());
continue;
}
if(Objects.equals(WebhookActiveStatus.DISABLED.getId(), webhook.getActiveStatusId()))
{
disabledWebhookLine.incrementCountAndAddPrimaryKey(webhookEvent.getId());
continue;
}
if(Objects.equals(WebhookHealthStatus.UNHEALTHY.getId(), webhook.getHealthStatusId()))
{
unhealthyWebhookLine.incrementCountAndAddPrimaryKey(webhookEvent.getId());
continue;
}
WebhookSubscription webhookSubscription = subscriptionMap.get(webhookEvent.getWebhookSubscriptionId());
if(webhookSubscription == null)
{
missingSubscriptionLine.incrementCountAndAddPrimaryKey(webhookEvent.getId());
continue;
}
if(Objects.equals(WebhookActiveStatus.DISABLED.getId(), webhookSubscription.getActiveStatusId()))
{
disabledSubscriptionLine.incrementCountAndAddPrimaryKey(webhookEvent.getId());
continue;
}
if(WebhookEventStatus.DELIVERED.getId().equals(webhookEvent.getEventStatusId()))
{
///////////////////////////////////////////////////////////////////////
// already delivered gets a warning, but still proceeds to load step //
///////////////////////////////////////////////////////////////////////
alreadyDeliveredWarningLine.incrementCountAndAddPrimaryKey(webhookEvent.getId());
runBackendStepOutput.addRecordEntity(webhookEvent);
}
else
{
///////////////////////////////////////////////////////
// finally all ok - move the record to the load step //
///////////////////////////////////////////////////////
okLine.incrementCountAndAddPrimaryKey(webhookEvent.getId());
runBackendStepOutput.addRecordEntity(webhookEvent);
}
}
}
/***************************************************************************
**
***************************************************************************/
private void loadSubscriptions(Set<Integer> subscriptionIds) throws QException
{
Set<Integer> subscriptionsToLookup = new HashSet<>(subscriptionIds);
subscriptionsToLookup.removeAll(subscriptionMap.keySet());
if(!subscriptionsToLookup.isEmpty())
{
List<WebhookSubscription> subscriptions = QueryAction.execute(WebhookSubscription.TABLE_NAME, WebhookSubscription.class, new QQueryFilter(new QFilterCriteria("id", QCriteriaOperator.IN, subscriptionsToLookup)));
for(WebhookSubscription subscription : subscriptions)
{
subscriptionMap.put(subscription.getId(), subscription);
}
}
}
/***************************************************************************
**
***************************************************************************/
Webhook getWebhook()
{
return webhook;
}
/***************************************************************************
** originally meant for tests
***************************************************************************/
SendWebhookEventTransformStep withWebhook(Webhook webhook)
{
this.webhook = webhook;
return this;
}
}
@@ -1,113 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.processes;
import com.kingsrook.qbits.webhooks.WebhooksQBitConfig;
import com.kingsrook.qbits.webhooks.model.Webhook;
import com.kingsrook.qbits.webhooks.model.WebhookActiveStatus;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.scheduledjobs.ScheduledJob;
import com.kingsrook.qqq.backend.core.scheduler.processes.AbstractRecordSyncToScheduledJobProcess;
import com.kingsrook.qqq.backend.core.utils.StringUtils;
/*******************************************************************************
**
*******************************************************************************/
public class SyncWebhookScheduledJobProcess extends AbstractRecordSyncToScheduledJobProcess
{
/***************************************************************************
**
***************************************************************************/
@Override
protected ScheduledJob customizeScheduledJob(ScheduledJob scheduledJob, QRecord sourceRecord) throws QException
{
scheduledJob.setRepeatSeconds(WebhooksQBitConfig.getConfigValue(config -> config.getSendWebhookEventProcessRepeatSeconds()));
boolean isActive = (WebhookActiveStatus.ACTIVE.getId().equals(sourceRecord.getValueInteger("activeStatusId")));
scheduledJob.setIsActive(isActive);
if(!StringUtils.hasContent(scheduledJob.getSchedulerName()))
{
scheduledJob.setSchedulerName(WebhooksQBitConfig.getConfigValue(config -> config.getSchedulerName()));
}
return (scheduledJob);
}
/***************************************************************************
**
***************************************************************************/
@Override
protected String getScheduledJobForeignKeyType()
{
return (Webhook.TABLE_NAME);
}
/***************************************************************************
**
***************************************************************************/
@Override
protected String getRecordForeignKeyFieldName()
{
return ("id");
}
/***************************************************************************
**
***************************************************************************/
@Override
protected String getRecordForeignKeyPossibleValueSourceName()
{
return (Webhook.TABLE_NAME);
}
/***************************************************************************
**
***************************************************************************/
@Override
protected String getSourceTableName()
{
return (Webhook.TABLE_NAME);
}
/***************************************************************************
**
***************************************************************************/
@Override
protected String getProcessNameScheduledJobParameter()
{
return (SendWebhookEventProcessMetaDataProducer.NAME);
}
}
@@ -1,40 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.registry;
import com.kingsrook.qqq.backend.core.instances.enrichment.plugins.QInstanceEnricherPluginInterface;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
/*******************************************************************************
**
*******************************************************************************/
public class WebhooksRegistryEnricherPlugin implements QInstanceEnricherPluginInterface<QInstance>
{
/***************************************************************************
**
***************************************************************************/
@Override
public void enrich(QInstance qInstance, QInstance qInstance2)
{
}
}
@@ -1,41 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.registry;
import com.kingsrook.qqq.backend.core.instances.QInstanceValidator;
import com.kingsrook.qqq.backend.core.instances.validation.plugins.QInstanceValidatorPluginInterface;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
/*******************************************************************************
**
*******************************************************************************/
public class WebhooksRegistryValidatorPlugin implements QInstanceValidatorPluginInterface<QInstance>
{
/***************************************************************************
**
***************************************************************************/
@Override
public void validate(QInstance qInstance, QInstance qInstance2, QInstanceValidator validator)
{
}
}
@@ -30,7 +30,6 @@ import com.kingsrook.qbits.webhooks.WebhooksRuntime;
import com.kingsrook.qbits.webhooks.metadata.WebhooksEnums.SecretSource;
import com.kingsrook.qbits.webhooks.metadata.WebhooksEnums.SecretStatus;
import com.kingsrook.qbits.webhooks.metadata.WebhooksTables;
import com.kingsrook.qbits.webhooks.shared.signing.WebhookSecret;
import com.kingsrook.qbits.webhooks.support.SystemWrites;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
@@ -41,6 +40,7 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
import com.kingsrook.qqq.backend.core.utils.ValueUtils;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret;
import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
@@ -1,183 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.shared.signing;
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
import java.util.Arrays;
import java.util.Base64;
import java.util.HexFormat;
/*******************************************************************************
** A webhook signing secret (Standard Webhooks format).
**
** <p>The text form is {@code whsec_} followed by the base64 of the key bytes.
** The HMAC key is the decoded bytes. A secret without the prefix is used as
** raw UTF-8 bytes, so a secret shared with an older receiver still works.</p>
**
** <p>{@link #toString()} never reveals the key: it prints the fingerprint.</p>
**
** <p>INTERNAL COPY: replaced by {@code com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret}
** when the qbit moves to QQQ 4.1.0-RC.5 (QRun-IO/qqq, D2 of the webhooks assessment).
** Keep the two identical.</p>
*******************************************************************************/
public final class WebhookSecret
{
public static final String PREFIX = "whsec_";
/** Key length for generated secrets, in bytes (256 bits). */
public static final int GENERATED_KEY_BYTES = 32;
/** Shortest key accepted, in bytes (128 bits). */
public static final int MINIMUM_KEY_BYTES = 16;
private static final SecureRandom RANDOM = new SecureRandom();
private final byte[] key;
/*******************************************************************************
** Constructor (keeps a copy of the key).
*******************************************************************************/
private WebhookSecret(byte[] key)
{
if(key == null || key.length < MINIMUM_KEY_BYTES)
{
throw (new IllegalArgumentException("A webhook signing secret must be at least " + MINIMUM_KEY_BYTES + " bytes"));
}
this.key = Arrays.copyOf(key, key.length);
}
/*******************************************************************************
** A new random secret.
*******************************************************************************/
public static WebhookSecret generate()
{
byte[] bytes = new byte[GENERATED_KEY_BYTES];
RANDOM.nextBytes(bytes);
return (new WebhookSecret(bytes));
}
/*******************************************************************************
** Parse the text form ({@code whsec_<base64>}, or raw text).
*******************************************************************************/
public static WebhookSecret parse(String text)
{
if(text == null || text.isBlank())
{
throw (new IllegalArgumentException("A webhook signing secret is required"));
}
String trimmed = text.trim();
if(trimmed.startsWith(PREFIX))
{
try
{
return (new WebhookSecret(Base64.getDecoder().decode(trimmed.substring(PREFIX.length()))));
}
catch(IllegalArgumentException e)
{
throw (new IllegalArgumentException("A " + PREFIX + " secret must be base64 after the prefix", e));
}
}
return (new WebhookSecret(trimmed.getBytes(StandardCharsets.UTF_8)));
}
/*******************************************************************************
** The text form to give a receiver ({@code whsec_<base64>}).
*******************************************************************************/
public String reveal()
{
return (PREFIX + Base64.getEncoder().encodeToString(key));
}
/*******************************************************************************
** The HMAC key (a copy).
*******************************************************************************/
public byte[] keyBytes()
{
return (Arrays.copyOf(key, key.length));
}
/*******************************************************************************
** A short, non-reversible id for the secret (first 12 hex chars of its
** SHA-256), safe to store and show, so people can tell secrets apart.
*******************************************************************************/
public String fingerprint()
{
try
{
byte[] digest = MessageDigest.getInstance("SHA-256").digest(key);
return (HexFormat.of().formatHex(digest).substring(0, 12));
}
catch(NoSuchAlgorithmException e)
{
throw (new IllegalStateException("SHA-256 is unavailable", e));
}
}
/*******************************************************************************
** Never prints the key.
*******************************************************************************/
@Override
public String toString()
{
return ("WebhookSecret[" + fingerprint() + "]");
}
/*******************************************************************************
** Equal when the keys are equal (constant time).
*******************************************************************************/
@Override
public boolean equals(Object o)
{
return (o instanceof WebhookSecret other && MessageDigest.isEqual(key, other.key));
}
/*******************************************************************************
** Hash of the fingerprint.
*******************************************************************************/
@Override
public int hashCode()
{
return (fingerprint().hashCode());
}
}
@@ -1,147 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.shared.signing;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import java.nio.charset.StandardCharsets;
import java.security.GeneralSecurityException;
import java.time.Instant;
import java.util.ArrayList;
import java.util.Base64;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
/*******************************************************************************
** Signs webhook requests in the Standard Webhooks format
** (https://www.standardwebhooks.com/).
**
** <p>The signed content is {@code <id>.<timestamp>.<body>}: the message id,
** a dot, the Unix timestamp in seconds, a dot, then the exact body bytes. The
** signature is HMAC-SHA256 with the secret's key, base64 encoded, and sent as
** {@code v1,<base64>}. During a secret rotation every active secret signs, and
** the signatures are separated by spaces, so a receiver holding either the old
** or the new secret accepts the request.</p>
**
** <p>Headers: {@code webhook-id}, {@code webhook-timestamp},
** {@code webhook-signature}.</p>
**
** <p>INTERNAL COPY: replaced by {@code com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSigner}
** when the qbit moves to QQQ 4.1.0-RC.5. Keep the two identical.</p>
*******************************************************************************/
public final class WebhookSigner
{
public static final String HEADER_ID = "webhook-id";
public static final String HEADER_TIMESTAMP = "webhook-timestamp";
public static final String HEADER_SIGNATURE = "webhook-signature";
/** The only signature scheme today: HMAC-SHA256. */
public static final String VERSION_V1 = "v1";
private static final String HMAC = "HmacSHA256";
/*******************************************************************************
** Utility class.
*******************************************************************************/
private WebhookSigner()
{
}
/*******************************************************************************
** The three signature headers for one request.
**
** @param messageId the event id (stable across retries: it is the idempotency key)
** @param timestamp when this attempt is sent
** @param body the exact bytes that are sent
** @param secrets the active secrets (one, or two during a rotation)
*******************************************************************************/
public static Map<String, String> headers(String messageId, Instant timestamp, byte[] body, List<WebhookSecret> secrets)
{
long seconds = timestamp.getEpochSecond();
Map<String, String> headers = new LinkedHashMap<>();
headers.put(HEADER_ID, messageId);
headers.put(HEADER_TIMESTAMP, Long.toString(seconds));
headers.put(HEADER_SIGNATURE, signatureHeader(messageId, seconds, body, secrets));
return (headers);
}
/*******************************************************************************
** The {@code webhook-signature} value: {@code v1,<sig>} per secret, space
** separated.
*******************************************************************************/
public static String signatureHeader(String messageId, long timestampSeconds, byte[] body, List<WebhookSecret> secrets)
{
if(secrets == null || secrets.isEmpty())
{
throw (new IllegalArgumentException("At least one signing secret is required"));
}
List<String> parts = new ArrayList<>();
for(WebhookSecret secret : secrets)
{
parts.add(VERSION_V1 + "," + sign(messageId, timestampSeconds, body, secret));
}
return (String.join(" ", parts));
}
/*******************************************************************************
** One base64 HMAC-SHA256 signature over {@code <id>.<timestamp>.<body>}.
*******************************************************************************/
public static String sign(String messageId, long timestampSeconds, byte[] body, WebhookSecret secret)
{
validateId(messageId);
try
{
Mac mac = Mac.getInstance(HMAC);
mac.init(new SecretKeySpec(secret.keyBytes(), HMAC));
mac.update((messageId + "." + timestampSeconds + ".").getBytes(StandardCharsets.UTF_8));
mac.update(body == null ? new byte[0] : body);
return (Base64.getEncoder().encodeToString(mac.doFinal()));
}
catch(GeneralSecurityException e)
{
throw (new IllegalStateException("HMAC-SHA256 is unavailable", e));
}
}
/*******************************************************************************
** A message id may not contain the separator or whitespace.
*******************************************************************************/
static void validateId(String messageId)
{
if(messageId == null || messageId.isEmpty() || messageId.contains(".") || messageId.chars().anyMatch(Character::isWhitespace))
{
throw (new IllegalArgumentException("A webhook message id is required and may not contain '.' or whitespace"));
}
}
}
@@ -1,165 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.shared.signing;
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.time.Clock;
import java.time.Duration;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
import java.util.Map;
/*******************************************************************************
** Verifies a signed webhook request (Standard Webhooks format, see
** {@link WebhookSigner}).
**
** <p>Checks, in order: the three headers are present, the timestamp is within
** the tolerance of now (default five minutes, either direction, which stops
** replays of old requests), and at least one {@code v1} signature matches one
** of the receiver's secrets. Comparison is constant time. Unknown signature
** versions are ignored, so a future scheme can be added alongside v1.</p>
**
** <p>Receivers should also remember recent {@code webhook-id}s and ignore a
** repeat: the same id is sent again on every retry of the same event.</p>
**
** <p>INTERNAL COPY: replaced by {@code com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerifier}
** when the qbit moves to QQQ 4.1.0-RC.5. Keep the two identical.</p>
*******************************************************************************/
public final class WebhookVerifier
{
public static final Duration DEFAULT_TOLERANCE = Duration.ofMinutes(5);
private final List<WebhookSecret> secrets;
private final Duration tolerance;
private final Clock clock;
/*******************************************************************************
** Verifier with the default tolerance and the system clock.
*******************************************************************************/
public WebhookVerifier(List<WebhookSecret> secrets)
{
this(secrets, DEFAULT_TOLERANCE, Clock.systemUTC());
}
/*******************************************************************************
** Verifier.
*******************************************************************************/
public WebhookVerifier(List<WebhookSecret> secrets, Duration tolerance, Clock clock)
{
if(secrets == null || secrets.isEmpty())
{
throw (new IllegalArgumentException("At least one secret is required to verify webhooks"));
}
this.secrets = new ArrayList<>(secrets);
this.tolerance = tolerance;
this.clock = clock;
}
/*******************************************************************************
** Verify a request. Header names are matched case-insensitively.
**
** @throws WebhookVerificationException with a reason safe to log
*******************************************************************************/
public void verify(Map<String, String> headers, byte[] body) throws WebhookVerificationException
{
String id = header(headers, WebhookSigner.HEADER_ID);
String timestamp = header(headers, WebhookSigner.HEADER_TIMESTAMP);
String signature = header(headers, WebhookSigner.HEADER_SIGNATURE);
if(id == null || timestamp == null || signature == null)
{
throw (new WebhookVerificationException(WebhookVerificationException.Reason.MISSING_HEADERS, "Missing webhook-id, webhook-timestamp or webhook-signature"));
}
long seconds;
try
{
seconds = Long.parseLong(timestamp.trim());
}
catch(NumberFormatException e)
{
throw (new WebhookVerificationException(WebhookVerificationException.Reason.BAD_TIMESTAMP, "webhook-timestamp is not a Unix timestamp"));
}
long now = clock.instant().getEpochSecond();
if(Math.abs(now - seconds) > tolerance.toSeconds())
{
throw (new WebhookVerificationException(WebhookVerificationException.Reason.TIMESTAMP_OUT_OF_TOLERANCE, "webhook-timestamp is outside the allowed window"));
}
try
{
WebhookSigner.validateId(id);
}
catch(IllegalArgumentException e)
{
throw (new WebhookVerificationException(WebhookVerificationException.Reason.MISSING_HEADERS, e.getMessage()));
}
for(String part : signature.trim().split(" +"))
{
int comma = part.indexOf(',');
if(comma < 0 || !WebhookSigner.VERSION_V1.equals(part.substring(0, comma)))
{
continue;
}
byte[] offered = part.substring(comma + 1).getBytes(StandardCharsets.US_ASCII);
for(WebhookSecret secret : secrets)
{
byte[] expected = WebhookSigner.sign(id, seconds, body, secret).getBytes(StandardCharsets.US_ASCII);
if(MessageDigest.isEqual(expected, offered))
{
return;
}
}
}
throw (new WebhookVerificationException(WebhookVerificationException.Reason.NO_MATCHING_SIGNATURE, "No v1 signature matches a known secret"));
}
/*******************************************************************************
** Case-insensitive header lookup.
*******************************************************************************/
private static String header(Map<String, String> headers, String name)
{
if(headers == null)
{
return (null);
}
for(Map.Entry<String, String> entry : headers.entrySet())
{
if(entry.getKey() != null && entry.getKey().toLowerCase(Locale.ROOT).equals(name))
{
return (entry.getValue());
}
}
return (null);
}
}
@@ -18,53 +18,42 @@
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.shared.signing;
package com.kingsrook.qbits.webhooks.support;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import java.util.LinkedHashMap;
import java.util.Map;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.utils.JsonUtils;
/*******************************************************************************
** A webhook request failed verification. The message never contains a
** secret or a signature, so it is safe to log and to return to the caller.
**
** <p>INTERNAL COPY: replaced by the class of the same name in
** {@code com.kingsrook.qqq.backend.core.utils.webhooks} at QQQ 4.1.0-RC.5.</p>
** Links to a filtered table, in the form every QQQ qbit uses and the Next UI
** reads: {@code <path>?filter=<url-encoded JSON QQueryFilter>}.
*******************************************************************************/
public class WebhookVerificationException extends Exception
public final class FilterLinks
{
private static final long serialVersionUID = 1L;
/*******************************************************************************
** Why verification failed.
** Utility class.
*******************************************************************************/
public enum Reason
private FilterLinks()
{
MISSING_HEADERS,
BAD_TIMESTAMP,
TIMESTAMP_OUT_OF_TOLERANCE,
NO_MATCHING_SIGNATURE
}
private final Reason reason;
/*******************************************************************************
** Constructor.
*******************************************************************************/
public WebhookVerificationException(Reason reason, String message)
{
super(message);
this.reason = reason;
}
/*******************************************************************************
** Getter for reason.
** The path (a table path, with any prefix), with the filter as URL-encoded JSON.
*******************************************************************************/
public Reason getReason()
public static String href(String path, QQueryFilter filter)
{
return (reason);
Map<String, Object> link = new LinkedHashMap<>();
link.put("criteria", filter.getCriteria());
link.put("orderBys", filter.getOrderBys());
link.put("subFilters", filter.getSubFilters());
link.put("booleanOperator", filter.getBooleanOperator());
return (path + "?filter=" + URLEncoder.encode(JsonUtils.toJson(link), StandardCharsets.UTF_8));
}
}
@@ -1,47 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.tables;
import java.util.List;
import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.tables.QueryOrGetInputInterface;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.tables.QQQTableTableManager;
/*******************************************************************************
**
*******************************************************************************/
public class WebhookEventTableCustomizer implements TableCustomizerInterface
{
/***************************************************************************
**
***************************************************************************/
@Override
public List<QRecord> postQuery(QueryOrGetInputInterface queryInput, List<QRecord> records) throws QException
{
return QQQTableTableManager.setRecordLinksToRecordsFromTableDynamicForPostQuery(queryInput, records, "eventSourceRecordQqqTableId", "eventSourceRecordId");
}
}
+576
View File
@@ -0,0 +1,576 @@
bundle: qbit-webhooks
version: ${project.version}
layer: QBIT
entries:
- key: app:webhooks;slot:help
summary: Send signed event messages to other systems from your application.
content: |
Webhooks tell outside services (a build, a chat room, another app) that something happened in your application. You add an **Endpoint** (the address to call), then add a **Subscription** that says which events it wants. Each matching event becomes one **Delivery**, which is sent with a signature, retried when it fails, and kept as a record you can inspect.
Your administrator configures which table changes, process results, message-bus events and custom events are available. Use the **Setup** card to check event types, endpoints, signing secrets and subscriptions. Nothing is seeded; you choose your receivers.
The home screen shows deliveries and failures for the last 24 hours, dead letters, how many are waiting, delivery speed, and central endpoint and dispatcher status with a link to System Health. Under it you can open **Endpoints**, **{{label:webhooks_subscription}}** and **Deliveries**. Signing secrets and attempts are shown inside the endpoint and the delivery.
Viewing needs the webhooks view permission, changing endpoints and re-sending needs manage, and secrets, internal-network targets and deleting need admin. Delivery is at least once, so receivers must tolerate a repeat.
- key: table:webhooks_endpoint;slot:help
summary: A web address that receives signed event messages, with its sending limits and health.
content: |
**How rows get created:** You add a receiver with **New**, then give it a signing secret and subscription.
An endpoint is one receiver. Give it a name, the https address, and a status, then add **{{label:webhooks_subscription}}** to choose what it hears about. Open the record to see its subscriptions, signing secrets and recent deliveries.
- By default, the address must be https, with no user name or password in it. It is checked when you save and again on every send.
- Addresses on private or internal networks are refused unless an administrator allows them under **Internal Network (Admin Only)**.
- Without a signing secret, deliveries wait and are not sent. Add one with **{{label:webhooks_rotate_secret}}** or an environment reference.
- Use **{{label:webhooks_send_test}}** to check that the receiver answers and verifies the signature.
Endpoints that keep failing are paused automatically and eventually disabled; see **{{label:webhooks_endpoint.healthStatus}}**. Deleting an endpoint needs the webhooks admin permission.
- key: table:webhooks_endpoint;section:identity
summary: The name, address and on/off state of the endpoint.
- key: table:webhooks_endpoint;section:delivery
summary: How fast, how patiently and how strictly messages are sent to this endpoint.
- key: table:webhooks_endpoint;section:health
summary: Failure counts and dates the system keeps up to date as it sends.
- key: table:webhooks_endpoint;section:network
summary: Lets this endpoint be an address on an internal network. Only webhook administrators can change it.
- key: table:webhooks_endpoint;section:subscriptions
summary: The events this endpoint is subscribed to.
- key: table:webhooks_endpoint;section:secrets
summary: The secrets used to sign messages to this endpoint.
- key: table:webhooks_endpoint;section:deliveries
summary: The newest deliveries sent to this endpoint.
- key: table:webhooks_endpoint;section:dates
summary: When the endpoint was created and last changed.
- key: table:webhooks_endpoint;field:name
summary: A short unique name for this receiver, used in lists and on the home screen.
- key: table:webhooks_endpoint;field:description
summary: Optional note on what this receiver is for.
- key: table:webhooks_endpoint;field:url
summary: 'Where messages are sent: a public https address, with no user name or password in it.'
content: |
Messages are sent as a POST with a JSON body. Redirects are not followed, so use the final address.
The address is checked when you save. A name that cannot be looked up yet is accepted, because DNS may not exist yet, but every send checks it again. Addresses that resolve to private, loopback or other internal networks are refused unless **{{label:webhooks_endpoint.allowInternalTargets}}** is on for this endpoint.
- key: table:webhooks_endpoint;field:activeStatus
summary: 'Whether this endpoint receives messages: {{values:webhooks_active_status}}.'
content: |
- **Active**: events are queued and sent.
- **Paused**: events are still queued, but nothing is sent until you set it back to Active.
- **Disabled**: nothing is queued or sent. The system sets this itself after repeated failed probation attempts; see **{{label:webhooks_endpoint.autoDisabledDate}}**.
- key: table:webhooks_endpoint;field:healthStatus
summary: How well sending to this endpoint is going, set automatically from the results.
content: |
- **Healthy**: sending normally.
- **Unhealthy**: after the configured number of failures in a row (10 by default), sending stops.
- **Probation**: after the configured unhealthy period (60 minutes by default), one attempt is made per run. A success makes it Healthy again; a failure makes it Unhealthy.
After the configured number of failed probations (3 by default) the endpoint is set to Disabled. Replies of "too many requests" (429) and policy blocks do not count as failures. **{{label:webhooks_send_now}}** and **{{label:webhooks_send_test}}** ignore health.
- key: table:webhooks_endpoint;field:serialDelivery
summary: Send one message at a time, oldest first, when the receiver needs events in order.
content: |
When on, the oldest unfinished delivery for this endpoint blocks the rest until it is delivered or becomes a dead letter. When off (default: {{default:webhooks_endpoint.serialDelivery}}), newer messages go ahead while a failing one waits to retry, so order is not guaranteed.
- key: table:webhooks_endpoint;field:maxPerMinute
summary: Caps how many send attempts this endpoint gets per minute. Leave empty for no limit.
content: |
Must be at least 1. The count is based on the attempts made in the last minute, so it is approximate: a few extra can slip through.
- key: table:webhooks_endpoint;field:timeoutSeconds
summary: How long to wait for the receiver to answer, in seconds, subject to the limits configured for your application.
content: |
Leave empty to use your application's timeout (10 seconds by default). A send that takes longer counts as a failed attempt.
- key: table:webhooks_endpoint;field:allowUnsigned
summary: Sends messages without a signature when no signing secret exists. Normally leave this off.
content: |
Unsigned messages still carry the id and timestamp headers, but the receiver cannot prove they came from your application. With this off, an endpoint with no usable secret does not get anything sent.
- key: table:webhooks_endpoint;field:allowInternalTargets
summary: Permits this endpoint to be an address on an internal or private network. Administrators only.
content: |
Only webhook administrators can turn this on or change it. Even when on, the cloud metadata addresses are always blocked, and when an allowlist is given only those entries are permitted.
- key: table:webhooks_endpoint;field:internalAllowlist
summary: The internal addresses this endpoint may call, separated by commas or spaces. Administrators only.
content: |
Each entry is a network range (for example 10.0.0.0/24), a single address, an exact host name, or a host name suffix starting with a dot. An entry that cannot be read is rejected when you save. Only used when **{{label:webhooks_endpoint.allowInternalTargets}}** is on.
- key: table:webhooks_endpoint;field:consecutiveFailures
summary: Failed attempts in a row since the last success, counted automatically.
- key: table:webhooks_endpoint;field:probationFailures
summary: How many times a probation attempt has failed, counted automatically. The configured limit controls when it is disabled.
- key: table:webhooks_endpoint;field:lastSuccessDate
summary: When the last delivery to this endpoint succeeded, filled in automatically.
- key: table:webhooks_endpoint;field:lastFailureDate
summary: When the last attempt to this endpoint failed, filled in automatically.
- key: table:webhooks_endpoint;field:lastFailureMessage
summary: The reason the last attempt failed, such as an HTTP status or a network error. Filled in automatically.
- key: table:webhooks_endpoint;field:unhealthySinceDate
summary: When the endpoint last became unhealthy, filled in automatically. Empty while healthy.
- key: table:webhooks_endpoint;field:autoDisabledDate
summary: When the system disabled this endpoint after repeated failed probation attempts. Set automatically.
content: |
To use the endpoint again, fix the receiver and set **{{label:webhooks_endpoint.activeStatus}}** back to Active.
- key: table:webhooks_endpoint_secret;slot:help
summary: A secret used to sign the messages sent to one endpoint, so the receiver can verify them.
content: |
**How rows get created:** You use **{{label:webhooks_rotate_secret}}** to generate a secret, or add an environment reference.
Every message carries a signature made with the endpoint's active secret. The receiver holds the same secret and checks it.
There are two kinds. A generated secret is created by **{{label:webhooks_rotate_secret}}** and stored encrypted. An environment reference only names a server setting that holds the secret, so the value never lives in the database.
The value of a generated secret is never shown in lists or results. Webhook administrators can open the secret record and reveal it behind "Show the secret". During a rotation the old secret stays valid for the configured overlap (24 hours by default) and both sign every message, so the receiver can switch over without missing anything. Adding or changing secrets needs the webhooks admin permission.
- key: table:webhooks_endpoint_secret;section:identity
summary: Which endpoint this secret belongs to, and whether it is in use.
- key: table:webhooks_endpoint_secret;section:data
summary: Where the secret comes from, when it signs until, and its fingerprint.
- key: table:webhooks_endpoint_secret;section:value
summary: Lets webhook administrators reveal a generated secret so they can copy it to the receiver.
- key: table:webhooks_endpoint_secret;section:dates
summary: When the secret was created and last changed.
- key: table:webhooks_endpoint_secret;field:endpointId
summary: The endpoint whose messages this secret signs.
- key: table:webhooks_endpoint_secret;field:status
summary: 'Where the secret is in its life: {{values:webhooks_secret_status}}.'
content: |
- **Active**: the secret in use.
- **Previous (still signing until it expires)**: replaced by a newer one but still signing for the overlap, 24 hours by default, so receivers can switch.
- **Retired**: expired and no longer used. The maintenance job retires expired secrets when scheduling is enabled.
- key: table:webhooks_endpoint_secret;field:source
summary: Whether the secret is stored encrypted here, or read from a server setting.
content: |
- **Generated (stored encrypted)**: created by **{{label:webhooks_rotate_secret}}**. You cannot add these by hand.
- **Environment reference**: you name a server setting under **{{label:webhooks_endpoint_secret.secretRef}}**; its value is read each time a message is sent.
- key: table:webhooks_endpoint_secret;field:secretRef
summary: The name of the server setting that holds the secret, for environment references only.
content: |
Enter only the variable name (letters, digits and underscores), never the secret itself. The value in that variable must be a whsec_ secret or at least 16 bytes long; a shorter one is rejected when you save.
- key: table:webhooks_endpoint_secret;field:encryptedSecret
summary: The encrypted form of a generated secret. Filled in automatically and not readable by people.
- key: table:webhooks_endpoint_secret;field:fingerprint
summary: A short code that identifies the secret without revealing it. Filled in automatically.
content: |
The same fingerprint appears under **Signed With (Fingerprints)** on each attempt, so you can tell which secret signed a message.
- key: table:webhooks_endpoint_secret;field:expiresDate
summary: When a Previous secret stops signing. Set automatically when a secret is replaced.
content: |
By default, it is 24 hours after the rotation; your application can configure another overlap. An active secret has no end date.
- key: table:webhooks_endpoint_secret;field:retiredDate
summary: When the secret was retired, filled in automatically.
- key: table:webhooks_endpoint_secret;field:note
summary: Optional note, such as who holds the matching copy of the secret.
- key: table:webhooks_subscription;slot:help
summary: Chooses which events go to an endpoint, optionally filtered, and how much of the record is sent.
content: |
**How rows get created:** You add a subscription and choose an endpoint and event type. Your administrator configures the available events.
A subscription links one **{{label:webhooks_subscription.endpointId}}** to one **{{label:webhooks_subscription.eventTypeName}}**. When a matching event happens, one delivery is queued for that endpoint.
An event matches when all of these hold:
- the subscription is Active and its endpoint is not Disabled;
- the event type is the same;
- the **{{label:webhooks_subscription.filterJson}}**, if any, matches the record;
- the record is one the subscription is allowed to read.
Event types look like a table name plus inserted, updated or deleted, or the completed or failed result of a configured process. You can only subscribe to a table's events if you can read that table, and an unknown event type is rejected.
Use **{{label:webhooks_subscription.detailLevel}}**, **{{label:webhooks_subscription.includeFields}}** and **{{label:webhooks_subscription.excludeFields}}** to control what the message contains. The message is built when the event happens, so retries send exactly the same content.
- key: table:webhooks_subscription;section:identity
summary: The endpoint, the event type and whether the subscription is on.
- key: table:webhooks_subscription;section:filter
summary: An optional note and a filter that limits which records trigger a message.
- key: table:webhooks_subscription;section:payload
summary: How much of the record is sent and how field names are shown.
- key: table:webhooks_subscription;section:deliveries
summary: The newest deliveries made for this subscription.
- key: table:webhooks_subscription;section:dates
summary: When the subscription was created and last changed.
- key: table:webhooks_subscription;field:endpointId
summary: The endpoint that receives the messages for this subscription.
- key: table:webhooks_subscription;field:eventTypeName
summary: The event this subscription listens for, for example a table being inserted, updated or deleted.
content: |
Pick from the list. Table events are named after the table plus inserted, updated or deleted. Configured process events report completed or failed. Custom and message-bus events may also be available. You need permission to read the table.
- key: table:webhooks_subscription;field:activeStatus
summary: 'Whether this subscription is used: {{values:webhooks_active_status}}.'
content: |
Only Active subscriptions queue new deliveries.
- key: table:webhooks_subscription;field:description
summary: Optional note on why this subscription exists.
- key: table:webhooks_subscription;field:filterJson
summary: Only send events for records that match this filter. Leave empty to send them all.
content: |
Enter a query filter as JSON, with criteria on the record's field names. It is checked against the record as it is after the change, with the old values filled in for anything not changed. If the JSON cannot be read, saving fails with a message.
- key: table:webhooks_subscription;field:detailLevel
summary: 'How much of the record each message carries: the full record, only what changed, or just the keys.'
content: |
- **Full record** (default: {{default:webhooks_subscription.detailLevel}}): the whole record, plus the old record for updates.
- **Changed fields only**: the fields that changed, and the record key.
- **Keys only**: just the record key, so the receiver fetches the rest itself.
- key: table:webhooks_subscription;field:includeFields
summary: If set, only these fields are sent. Separate names with commas or spaces. Leave empty for all.
content: |
Use the field names exactly as the record uses them. Combine with **{{label:webhooks_subscription.excludeFields}}** to remove a few.
- key: table:webhooks_subscription;field:excludeFields
summary: Fields that are always left out of the message. Separate names with commas or spaces.
- key: table:webhooks_subscription;field:apiName
summary: Send the record in the form of a published API, with that API's field names. Leave empty for the plain record.
content: |
Set together with **{{label:webhooks_subscription.apiVersion}}**. Both must be filled in for this to apply.
- key: table:webhooks_subscription;field:apiVersion
summary: The version of the named API used to shape the record.
- key: table:webhooks_delivery;slot:help
summary: One message queued for one endpoint, with its status, attempts and stored content.
content: |
**How rows get created:** A matching event queues a delivery automatically. Test sends and direct sends also create deliveries.
The system writes a delivery when an event matches a subscription, and updates it as it is sent. You cannot edit one. Use actions on selected deliveries instead.
A delivery moves from **Pending** to **Sending**, then **{{label:webhooks_delivery.deliveredDate}}**. If a send fails it follows your application's retry schedule. The default delays are 1, 5, 15, 60 and 240 minutes before the sixth attempt. After the configured attempt limit (6 by default) it becomes a **Dead letter**. A reply of "too many requests" (429) waits as the receiver asked and uses up no attempt.
- **{{label:webhooks_send_now}}** tries unfinished deliveries immediately.
- **{{label:webhooks_replay}}** queues finished or dead deliveries again with the same event id.
- **{{label:webhooks_cancel}}** stops pending ones.
- **{{label:webhooks_delete_deliveries}}** removes finished ones (administrators only).
Your application's retention settings determine when finished deliveries are purged. Open a delivery to see each attempt and the exact content that was sent.
- key: table:webhooks_delivery;section:identity
summary: The event type, the endpoint it went to, and the current status.
- key: table:webhooks_delivery;section:progress
summary: How many attempts were made, when the next one is due and how the last one ended.
- key: table:webhooks_delivery;section:source
summary: 'What caused this delivery: the event id and the record or run it came from.'
- key: table:webhooks_delivery;section:attempts
summary: Every send attempt for this delivery, newest included.
- key: table:webhooks_delivery;section:payloadSection
summary: The exact JSON content that is sent to the endpoint.
- key: table:webhooks_delivery;section:dates
summary: When the delivery was queued and last changed.
- key: table:webhooks_delivery;field:eventId
summary: The unique id of the event, sent to the receiver as the webhook-id header so it can ignore repeats.
content: |
It starts with evt_ and stays the same on every retry and replay. Receivers should remember the ids they have already processed.
- key: table:webhooks_delivery;field:eventTypeName
summary: The kind of event that was sent, for example a table insert or a publish completion.
- key: table:webhooks_delivery;field:endpointId
summary: The endpoint this message is sent to.
- key: table:webhooks_delivery;field:subscriptionId
summary: The subscription that caused this delivery. Empty for a test event.
- key: table:webhooks_delivery;field:source
summary: What kind of thing produced the event.
content: |
- **Table change**: a record was inserted, updated or deleted in your application.
- **ESB event**: the same kind of change, reported by another service.
- **Process**: a configured process completed or failed.
- **Custom event**: an event raised by your application.
- **Channel (another qbit)**: sent straight to one endpoint by another part of the system, such as alerts.
- **Test send**: made by **{{label:webhooks_send_test}}**.
- key: table:webhooks_delivery;field:sourceTable
summary: The table the changed record belongs to, when the event came from a record change.
- key: table:webhooks_delivery;field:sourceRecordId
summary: The id of the record that changed, when the event came from a record change.
- key: table:webhooks_delivery;field:sourceEventId
summary: The id of the original event or change this delivery came from, used to avoid queueing it twice.
- key: table:webhooks_delivery;field:dedupKey
summary: An internal key that makes sure the same event is queued only once per subscription.
- key: table:webhooks_delivery;field:status
summary: 'Where the delivery is: {{values:webhooks_delivery_status}}.'
content: |
- **Pending**: queued, not yet tried.
- **Sending**: an attempt is in progress.
- **Waiting to retry**: the last attempt failed; the next one is due at **{{label:webhooks_delivery.nextAttemptDate}}**.
- **{{label:webhooks_delivery.deliveredDate}}**: the receiver answered with success (2xx).
- **Dead letter**: reached the configured attempt limit since the last replay. You can replay or delete it; configured retention may also remove it.
- **Cancelled**: stopped with **{{label:webhooks_cancel}}**.
- key: table:webhooks_delivery;field:attemptCount
summary: Total send attempts made for this delivery, including before any replay.
- key: table:webhooks_delivery;field:cycleAttemptCount
summary: Attempts made since the delivery was queued or last replayed. The configured limit makes it a dead letter.
content: |
Replies of "too many requests" (429) do not count.
- key: table:webhooks_delivery;field:replayCount
summary: How many times this delivery has been queued again with Replay.
- key: table:webhooks_delivery;field:occurredDate
summary: When the event happened, which can be earlier than when the message was sent.
- key: table:webhooks_delivery;field:nextAttemptDate
summary: When the next attempt is due. Empty once the delivery is finished.
content: |
Retries follow your application's configured delays and attempt limit. A "too many requests" reply honors the receiver's delay up to the configured maximum (one hour by default).
- key: table:webhooks_delivery;field:leaseExpiresDate
summary: While Sending, the time after which another attempt may take over if the first one has stalled.
content: |
Normally 2 minutes after the attempt started.
- key: table:webhooks_delivery;field:lastHttpStatus
summary: The HTTP status code the receiver answered with on the last attempt. Empty if it did not answer.
- key: table:webhooks_delivery;field:lastError
summary: Why the last attempt failed, such as a network error, a blocked address or an HTTP status.
- key: table:webhooks_delivery;field:deliveredDate
summary: When the receiver accepted the message. Empty until delivered.
- key: table:webhooks_delivery;field:deadLetterDate
summary: When the delivery ran out of attempts and became a dead letter.
- key: table:webhooks_delivery;field:payloadBytes
summary: Size of the stored JSON content in bytes.
- key: table:webhooks_delivery;field:payload
summary: The exact JSON that is sent, built when the event happened, so every retry sends the same content.
content: |
It contains the event id, type, time, source and a data section with the record. Which fields appear is decided by the subscription's detail level and field lists.
- key: table:webhooks_delivery_attempt;slot:help
summary: One try at sending a delivery, with the result, timing and what the receiver said.
content: |
**How rows get created:** The system records each attempt to send a delivery. An empty list means no attempt has been recorded.
The system adds a row each time it sends a delivery, so you can see exactly what happened: the outcome, HTTP status, how long it took, a short excerpt of the receiver's reply, the error if any, and which secrets signed it. Attempts cannot be edited, and they are shown inside the delivery they belong to. Your application's retention settings determine when old attempts are removed.
- key: table:webhooks_delivery_attempt;section:identity
summary: The delivery and endpoint this attempt belongs to, and how it ended.
- key: table:webhooks_delivery_attempt;section:data
summary: The request, the receiver's response and the timing of this attempt.
- key: table:webhooks_delivery_attempt;section:dates
summary: When this attempt record was created and last changed.
- key: table:webhooks_delivery_attempt;field:deliveryId
summary: The delivery that was being sent.
- key: table:webhooks_delivery_attempt;field:endpointId
summary: The endpoint the request went to.
- key: table:webhooks_delivery_attempt;field:attemptNumber
summary: 'Which try this was for the delivery: 1, 2, 3 and so on.'
- key: table:webhooks_delivery_attempt;field:outcome
summary: How the attempt ended.
content: |
- **In flight**: still being sent.
- **Succeeded**: the receiver answered with success (2xx).
- **Failed**: an error status, a network problem or a timeout.
- **Rate limited (429)**: the receiver asked us to slow down; this uses no attempt.
- **Blocked by policy**: the address is not allowed, for example an internal network. It does not count against endpoint health.
- **Lease expired**: the sender stopped before recording a result, so a new attempt took over.
- key: table:webhooks_delivery_attempt;field:claimedBy
summary: Which server instance made this attempt, filled in automatically.
- key: table:webhooks_delivery_attempt;field:leaseExpiresDate
summary: The time this attempt was allowed to run until before another could take over.
- key: table:webhooks_delivery_attempt;field:startDate
summary: When the request started.
- key: table:webhooks_delivery_attempt;field:endDate
summary: When the request finished.
- key: table:webhooks_delivery_attempt;field:durationMillis
summary: How long the request took, in milliseconds.
- key: table:webhooks_delivery_attempt;field:httpStatus
summary: The HTTP status code in the receiver's reply. Empty if there was no reply.
- key: table:webhooks_delivery_attempt;field:responseExcerpt
summary: The start of the receiver's reply, up to 2,000 characters; longer replies are marked as cut off.
- key: table:webhooks_delivery_attempt;field:errorMessage
summary: What went wrong, if the attempt did not succeed.
- key: table:webhooks_delivery_attempt;field:remoteAddress
summary: The network address the request actually connected to.
- key: table:webhooks_delivery_attempt;field:requestUrl
summary: The address the request was sent to, as it was when this attempt ran.
- key: table:webhooks_delivery_attempt;field:signedWith
summary: Fingerprints of the secrets that signed this request, separated by spaces.
content: |
Two fingerprints mean a secret rotation is in progress. Match them against the fingerprints under **Signing Secrets**.
- key: process:webhooks_send_now;slot:help
summary: Tries to send the selected deliveries right now instead of waiting for their turn.
content: |
Select one or more deliveries and run it. Each unfinished delivery gets an attempt immediately, even if its endpoint is unhealthy or paused.
Deliveries that are already delivered, are dead letters (use **{{label:webhooks_replay}}** for those) or are cancelled are skipped. The result tells you how many were delivered and how many failed; failed ones go on retrying by the normal schedule. A delivery that another sender is working on at that moment is skipped too.
Needs the webhooks manage permission.
- key: process:webhooks_replay;slot:help
summary: Queues finished deliveries again, with the same event id and a fresh set of attempts.
content: |
Use it to resend something the receiver missed: a dead letter, or a delivery you want sent a second time. The delivery goes back to Pending, its attempt count since replay resets, and its **Replays** counter goes up. The event id stays the same, so a receiver that deduplicates will notice the repeat.
Deliveries that are still being delivered are skipped. It does not send straight away; the next scheduled run picks it up when scheduling is enabled (every 15 seconds by default), or use **{{label:webhooks_send_now}}**.
Needs the webhooks manage permission.
- key: process:webhooks_cancel;slot:help
summary: Stops the selected deliveries that have not been sent yet, so they will not be tried again.
content: |
Works on Pending and Waiting to retry deliveries. Deliveries being sent at that moment, and those already delivered, dead or cancelled, are skipped. A cancelled delivery can be queued again later with **{{label:webhooks_replay}}**.
Needs the webhooks manage permission.
- key: process:webhooks_delete_deliveries;slot:help
summary: Permanently deletes finished deliveries and all of their attempts.
content: |
Only Delivered, Dead letter and Cancelled deliveries are deleted; anything still pending or waiting is skipped, so cancel it first. This cannot be undone, and the history is lost with it. Replay dead letters before deleting if the receiver still needs them.
Needs the webhooks admin permission.
- key: process:webhooks_send_test;slot:help
summary: Sends a signed test message to the selected endpoint now and shows whether it was accepted.
content: |
Use it after adding an endpoint or changing its address or secret. The message has the type webhooks.test and a short note in its data. It ignores the endpoint's health and subscriptions, and it is recorded as a normal delivery with its attempt, so you can inspect it afterwards.
Disabled endpoints are skipped. The result shows delivered with the HTTP status, or the error. Needs the webhooks manage permission.
- key: process:webhooks_rotate_secret;slot:help
summary: Creates a new signing secret for the endpoint. The old one keeps signing during the configured overlap.
content: |
The current Active secret becomes Previous and signs alongside the new one during the configured overlap (24 hours by default), so you can update the receiver without losing any message. After that the old secret is retired automatically.
The new value is not shown in the result. Open the endpoint, go to **Signing Secrets**, open the new secret and reveal it, then copy it to the receiver. It is stored encrypted.
If the server has no encryption key set, the action is skipped; add an environment reference instead. Needs the webhooks admin permission.
- key: process:webhooks_deliver;slot:help
summary: Send due deliveries and retries when your application enables scheduling.
content: |
It sends pending deliveries and retries, honoring each endpoint's rate limit, order setting and health, and updates endpoint health from the results. With scheduling enabled, it runs every 15 seconds by default; use **{{label:webhooks_send_now}}** for a single delivery.
- key: process:webhooks_maintain;slot:help
summary: Maintain endpoint health and retire expired secrets, every five minutes when scheduling is enabled.
content: |
After an endpoint has been Unhealthy for the configured period (60 minutes by default) it goes to Probation so one attempt can be made. Secrets whose overlap has ended are set to Retired.
- key: process:webhooks_retention;slot:help
summary: Remove old finished deliveries and attempts according to your application's retention settings.
content: |
When scheduling is enabled, this job runs daily. By default, delivered and cancelled deliveries and completed attempts are kept for 400 days; dead letters are kept indefinitely. Your administrator can configure each retention period; zero disables that purge. Deleting a delivery also deletes its attempts. Export any history you need before it expires.
- key: widget:webhooksEndpointSubscriptions;slot:label
summary: The subscriptions that decide which events this endpoint receives.
- key: widget:webhooksEndpointSecrets;slot:label
summary: The signing secrets for this endpoint, with their status and fingerprints. Values are not shown here.
- key: widget:webhooksEndpointDeliveries;slot:label
summary: The most recent deliveries sent to this endpoint.
- key: widget:webhooksSubscriptionDeliveries;slot:label
summary: The most recent deliveries made for this subscription.
- key: widget:webhooksDeliveryAttempts;slot:label
summary: Every send attempt for this delivery, with result, status and timing.
- key: widget:webhooksHome;slot:label
summary: Setup steps, deliveries, failures, dead letters, backlog and speed over the last 24 hours.
content: |
The Setup card shows configuration steps and their next actions. The tiles show delivered and failed attempts for the last 24 hours, dead letters, deliveries waiting to send and median and 95th percentile response time. Below are an hourly chart and the newest dead letters. Endpoint and dispatcher health appear in the System Health status widget.
- key: widget:webhooksHealth;slot:label
summary: Central endpoint and dispatcher health, with a link to the webhooks checks in System Health.
- key: widget:webhooksSecretValue;slot:label
summary: Reveals the secret value behind a Show button. Webhook administrators only.
- key: possibleValueSource:webhooks_active_status;slot:help
summary: Whether an endpoint or subscription is in use.
content: |
- **Active**: used normally.
- **Paused**: events are still queued for a paused endpoint but not sent; a paused subscription queues nothing new.
- **Disabled**: not used. An endpoint can be disabled automatically after repeated failures.
- key: possibleValueSource:webhooks_health_status;slot:help
summary: How well sending to an endpoint is going.
content: |
- **Healthy**: sending normally.
- **Probation**: after an unhealthy spell, one attempt at a time is made to see if it works again.
- **Unhealthy**: reached the configured consecutive-failure limit (10 by default), so sending is stopped for a while.
- key: possibleValueSource:webhooks_delivery_status;slot:help
summary: The stage of a delivery.
content: |
- **Pending**: queued and not yet tried.
- **Sending**: an attempt is in progress.
- **Waiting to retry**: the last attempt failed and another is scheduled.
- **Delivered**: accepted by the receiver.
- **Dead letter**: failed too many times and was given up on.
- **Cancelled**: stopped by a person.
- key: possibleValueSource:webhooks_attempt_outcome;slot:help
summary: How one send attempt ended.
content: |
- **In flight**: still running.
- **Succeeded**: the receiver accepted it.
- **Failed**: error status, network problem or timeout.
- **Rate limited (429)**: the receiver asked to slow down.
- **Blocked by policy**: the address is not allowed.
- **Lease expired**: the sender stopped without recording a result.
- key: possibleValueSource:webhooks_event_source;slot:help
summary: What produced an event.
content: |
- **Table change**: a record insert, update or delete in your application.
- **ESB event**: a change reported by another service.
- **Process**: a configured process completing or failing.
- **Custom event**: an event raised by your application.
- **Channel (another qbit)**: sent directly to one endpoint, for example by alerts.
- **Test send**: from **{{label:webhooks_send_test}}**.
- key: possibleValueSource:webhooks_secret_status;slot:help
summary: Where a signing secret is in its life.
content: |
- **Active**: the secret in use.
- **Previous (still signing until it expires)**: replaced, but still signs for the configured overlap (24 hours by default).
- **Retired**: no longer used.
- key: possibleValueSource:webhooks_secret_source;slot:help
summary: How a signing secret is kept.
content: |
- **Generated (stored encrypted)**: created by the system and stored encrypted.
- **Environment reference**: only the name of a server setting is stored; the value is read when sending.
- key: possibleValueSource:webhooks_detail_level;slot:help
summary: How much of a record a message contains.
content: |
- **Full record**: the whole record, plus the old record for updates.
- **Changed fields only**: just the fields that changed and the key.
- **Keys only**: only the record key.
- key: possibleValueSource:webhooks_event_type;slot:help
summary: 'The events you can subscribe to: table inserts, updates and deletes, and process results and custom events.'
content: |
Table events are the table name followed by inserted, updated or deleted. You can only pick events for tables you are allowed to read.
- key: process:webhooks_esb_ingest;slot:help
summary: Turns events arriving on the message bus into webhook deliveries; it runs by itself and is not started by hand.
content: |
It listens on the ESB destinations configured for webhooks. A table event such as a record being inserted, updated or deleted becomes the matching webhook event, and a finished or failed process becomes a process event. Events of unregistered types are ignored. By default, events this server published itself are skipped so nothing is sent twice, and a message delivered again queues nothing new.
- key: process:webhooks_endpoint.bulkInsert;slot:help
summary: Add multiple endpoints from a file.
content: |
Upload your file, map its columns, and review the preview before running the import.
You need webhook manage permission to add or edit endpoints and admin permission to delete them. Configure signing secrets and subscriptions separately.
- key: process:webhooks_endpoint.bulkEdit;slot:help
summary: Change selected endpoints together.
content: |
Select the records you want to change, enter the new values, and review the preview before applying them.
You need webhook manage permission to add or edit endpoints and admin permission to delete them. Configure signing secrets and subscriptions separately.
- key: process:webhooks_endpoint.bulkEditWithFile;slot:help
summary: Update endpoints using values from a file.
content: |
Upload your file and map its columns to identify and update existing records. Review the preview before applying changes.
You need webhook manage permission to add or edit endpoints and admin permission to delete them. Configure signing secrets and subscriptions separately.
- key: process:webhooks_endpoint.bulkDelete;slot:help
summary: Delete selected endpoints.
content: |
Review the selected records before confirming deletion. This permanently removes records and cannot be undone.
You need webhook manage permission to add or edit endpoints and admin permission to delete them. Configure signing secrets and subscriptions separately.
- key: process:webhooks_endpoint_secret.bulkInsert;slot:help
summary: Add multiple signing secrets from a file.
content: |
Upload your file, map its columns, and review the preview before running the import.
You need webhook admin permission. Add only environment references here; use **{{label:webhooks_rotate_secret}}** to generate an encrypted secret. Never put a secret value in an import file.
- key: process:webhooks_endpoint_secret.bulkEdit;slot:help
summary: Change selected signing secrets together.
content: |
Select the records you want to change, enter the new values, and review the preview before applying them.
You need webhook admin permission. Add only environment references here; use **{{label:webhooks_rotate_secret}}** to generate an encrypted secret. Never put a secret value in an import file.
- key: process:webhooks_endpoint_secret.bulkEditWithFile;slot:help
summary: Update signing secrets using values from a file.
content: |
Upload your file and map its columns to identify and update existing records. Review the preview before applying changes.
You need webhook admin permission. Add only environment references here; use **{{label:webhooks_rotate_secret}}** to generate an encrypted secret. Never put a secret value in an import file.
- key: process:webhooks_endpoint_secret.bulkDelete;slot:help
summary: Delete selected signing secrets.
content: |
Review the selected records before confirming deletion. This permanently removes records and cannot be undone.
You need webhook admin permission. Add only environment references here; use **{{label:webhooks_rotate_secret}}** to generate an encrypted secret. Never put a secret value in an import file.
- key: process:webhooks_subscription.bulkInsert;slot:help
summary: Add multiple subscriptions from a file.
content: |
Upload your file, map its columns, and review the preview before running the import.
You need webhook manage permission. Choose valid endpoints and available event types; your access to source records still applies.
- key: process:webhooks_subscription.bulkEdit;slot:help
summary: Change selected subscriptions together.
content: |
Select the records you want to change, enter the new values, and review the preview before applying them.
You need webhook manage permission. Choose valid endpoints and available event types; your access to source records still applies.
- key: process:webhooks_subscription.bulkEditWithFile;slot:help
summary: Update subscriptions using values from a file.
content: |
Upload your file and map its columns to identify and update existing records. Review the preview before applying changes.
You need webhook manage permission. Choose valid endpoints and available event types; your access to source records still applies.
- key: process:webhooks_subscription.bulkDelete;slot:help
summary: Delete selected subscriptions.
content: |
Review the selected records before confirming deletion. This permanently removes records and cannot be undone.
You need webhook manage permission. Choose valid endpoints and available event types; your access to source records still applies.
- key: possibleValueSource:webhooks_endpoint;slot:help
summary: Choose the endpoint that receives your messages.
- key: possibleValueSource:webhooks_subscription;slot:help
summary: Choose the subscription that selects events for an endpoint.
- key: possibleValueSource:webhooks_delivery;slot:help
summary: Choose the delivery whose send attempts you want to inspect.
@@ -0,0 +1 @@
version=${project.version}
@@ -1,244 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks;
import java.io.Serializable;
import java.util.List;
import java.util.Random;
import java.util.stream.Collectors;
import com.kingsrook.qbits.webhooks.actions.WebhookSubscriptionsHelper;
import com.kingsrook.qbits.webhooks.model.Webhook;
import com.kingsrook.qbits.webhooks.model.WebhookActiveStatus;
import com.kingsrook.qbits.webhooks.model.WebhookEvent;
import com.kingsrook.qbits.webhooks.model.WebhookEventCategory;
import com.kingsrook.qbits.webhooks.model.WebhookEventStatus;
import com.kingsrook.qbits.webhooks.model.WebhookHealthStatus;
import com.kingsrook.qbits.webhooks.model.WebhookSubscription;
import com.kingsrook.qbits.webhooks.registry.WebhookEventType;
import com.kingsrook.qbits.webhooks.registry.WebhooksRegistry;
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.instances.QInstanceValidator;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.session.QSession;
import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryRecordStore;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import static org.junit.jupiter.api.Assertions.fail;
/*******************************************************************************
**
*******************************************************************************/
public class BaseTest
{
protected static Random random = new Random();
/*******************************************************************************
**
*******************************************************************************/
@BeforeEach
void baseBeforeEach() throws Exception
{
QInstance qInstance = defineQInstance();
new QInstanceValidator().validate(qInstance);
QContext.init(qInstance, new QSession().withSecurityKeyValue(WebhooksTestApplication.STORE_ID_ALL_ACCESS_KEY, true));
WebhookSubscriptionsHelper.clearMemoizations();
MemoryRecordStore.fullReset();
}
/*******************************************************************************
**
*******************************************************************************/
@AfterEach
void afterEach()
{
MemoryRecordStore.fullReset();
}
/***************************************************************************
**
***************************************************************************/
private QInstance defineQInstance() throws QException
{
return new WebhooksTestApplication().defineQInstance();
}
/***************************************************************************
**
***************************************************************************/
public static void registerAdHocEventType(String name) throws QException
{
WebhooksRegistry.ofOrWithNew(QContext.getQInstance()).registerWebhookEventType(new WebhookEventType()
.withName(name).withCategory(WebhookEventCategory.AD_HOC));
}
/***************************************************************************
**
***************************************************************************/
public static void registerEventType(String name, WebhookEventCategory category, String tableName) throws QException
{
WebhooksRegistry.ofOrWithNew(QContext.getQInstance()).registerWebhookEventType(new WebhookEventType()
.withName(name).withCategory(category).withTableName(tableName));
}
/***************************************************************************
**
***************************************************************************/
public static void registerEventType(String name, WebhookEventCategory category, String tableName, String fieldName) throws QException
{
WebhooksRegistry.ofOrWithNew(QContext.getQInstance()).registerWebhookEventType(new WebhookEventType()
.withName(name).withCategory(category).withTableName(tableName).withFieldName(fieldName));
}
/***************************************************************************
**
***************************************************************************/
public static void registerEventType(String name, WebhookEventCategory category, String tableName, String fieldName, Serializable value) throws QException
{
WebhooksRegistry.ofOrWithNew(QContext.getQInstance()).registerWebhookEventType(new WebhookEventType()
.withName(name).withCategory(category).withTableName(tableName).withFieldName(fieldName).withValue(value));
}
/***************************************************************************
**
***************************************************************************/
public static Integer insert(Webhook webhook) throws QException
{
Integer id = new InsertAction().execute(new InsertInput(Webhook.TABLE_NAME)
.withRecordEntity(webhook)).getRecords().get(0).getValueInteger("id");
if(id == null)
{
fail("Failed to insert a test webhook");
}
return (id);
}
/***************************************************************************
**
***************************************************************************/
public static Webhook newWebhook(String name)
{
return new Webhook()
.withId(random.nextInt(100000))
.withName(name)
.withUrl("http://localhost/")
.withActiveStatusId(WebhookActiveStatus.ACTIVE.getId())
.withHealthStatusId(WebhookHealthStatus.HEALTHY.getId());
}
/***************************************************************************
**
***************************************************************************/
public static Integer insert(WebhookSubscription subscription) throws QException
{
Integer id = new InsertAction().execute(new InsertInput(WebhookSubscription.TABLE_NAME)
.withRecordEntity(subscription)).getRecords().get(0).getValueInteger("id");
if(id == null)
{
fail("Failed to insert a test subscription");
}
return (id);
}
/***************************************************************************
**
***************************************************************************/
public static WebhookSubscription newWebhookSubscription(String eventTypeName)
{
return new WebhookSubscription()
.withId(random.nextInt(100000))
.withWebhookEventTypeName(eventTypeName)
.withApiName(WebhooksTestApplication.API_NAME)
.withApiVersion(WebhooksTestApplication.API_V1)
.withWebhookId(-1);
}
/***************************************************************************
**
***************************************************************************/
public static Integer insert(WebhookEvent event) throws QException
{
return insert(List.of(event)).get(0);
}
/***************************************************************************
**
***************************************************************************/
public static List<Integer> insert(List<WebhookEvent> events) throws QException
{
List<Integer> ids = new InsertAction().execute(new InsertInput(WebhookEvent.TABLE_NAME)
.withRecordEntities(events)).getRecords().stream().map(qr -> qr.getValueInteger("id")).collect(Collectors.toList());
if(ids.contains(null))
{
fail("Failed to insert a test event");
}
return (ids);
}
/***************************************************************************
**
***************************************************************************/
public static WebhookEvent newWebhookEvent(WebhookSubscription subscription, String eventTypeName) throws QException
{
WebhookEvent webhookEvent = new WebhookEvent();
webhookEvent.setWebhookId(subscription.getWebhookId());
webhookEvent.setWebhookSubscriptionId(subscription.getId());
webhookEvent.setWebhookEventTypeName(subscription.getWebhookEventTypeName());
webhookEvent.setEventStatusId(WebhookEventStatus.NEW.getId());
return (webhookEvent);
}
}
@@ -0,0 +1,431 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.TreeMap;
import java.util.TreeSet;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.joins.JoinOn;
import com.kingsrook.qqq.backend.core.model.metadata.joins.QJoinMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSource;
import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSourceType;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Assumptions;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.testcontainers.DockerClientFactory;
import org.testcontainers.postgresql.PostgreSQLContainer;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
/*******************************************************************************
** Referential integrity guardrail: docs/ddl/webhooks-postgres.sql, applied to
** a real PostgreSQL, has a foreign key for every join and every table
** possible-value field between the qbit's tables (or the relation is in
** {@link #EXEMPT} with a reason), each with the intended ON DELETE. A new
** table or join without its key fails here. The upgrade script is checked
** too: it quarantines orphans, adds the same keys, and is idempotent.
*******************************************************************************/
class ForeignKeyGuardrailTest
{
private static final Path DDL = Path.of("docs/ddl/webhooks-postgres.sql");
private static final Path UPGRADE = Path.of("docs/ddl/webhooks-foreign-keys-upgrade.sql");
/////////////////////////////////////////////////////////////////////////////
// child_table.child_column -> parent_table : ON DELETE (pg confdeltype) //
// c = CASCADE, n = SET NULL, r = RESTRICT //
/////////////////////////////////////////////////////////////////////////////
private static final Map<String, String> EXPECTED = new TreeMap<>(Map.of(
"webhooks_endpoint_secret.endpoint_id -> webhooks_endpoint", "c",
"webhooks_subscription.endpoint_id -> webhooks_endpoint", "c",
"webhooks_delivery.endpoint_id -> webhooks_endpoint", "r",
"webhooks_delivery.subscription_id -> webhooks_subscription", "n",
"webhooks_delivery_attempt.delivery_id -> webhooks_delivery", "c",
"webhooks_delivery_attempt.endpoint_id -> webhooks_endpoint", "r"));
/////////////////////////////////////////////////////////////////////////////
// relations that are deliberately not foreign keys, with the reason //
/////////////////////////////////////////////////////////////////////////////
private static final Map<String, String> EXEMPT = Map.of();
private static PostgreSQLContainer postgres;
/*******************************************************************************
** Start PostgreSQL once (a missing Docker fails in CI, skips locally).
*******************************************************************************/
@BeforeAll
static void startPostgres()
{
boolean docker;
try
{
docker = DockerClientFactory.instance().isDockerAvailable();
}
catch(RuntimeException e)
{
docker = false;
}
if(!docker && System.getenv("CI") != null)
{
Assertions.fail("Docker is required for the foreign key guardrail in CI");
}
Assumptions.assumeTrue(docker, "Docker is not available");
postgres = new PostgreSQLContainer("postgres:17-alpine").withDatabaseName("webhooks").withUsername("qqq").withPassword("qqq");
postgres.start();
}
/*******************************************************************************
** Stop PostgreSQL.
*******************************************************************************/
@AfterAll
static void stopPostgres()
{
if(postgres != null)
{
postgres.stop();
}
}
/*******************************************************************************
** A clean schema from the reference DDL.
*******************************************************************************/
@BeforeEach
void setUp() throws Exception
{
try(Connection c = connect(); Statement st = c.createStatement())
{
st.execute("DROP SCHEMA public CASCADE");
st.execute("CREATE SCHEMA public");
st.execute(Files.readString(DDL, StandardCharsets.UTF_8));
}
}
/*******************************************************************************
** Clear the QContext the metadata setup opened.
*******************************************************************************/
@AfterEach
void tearDown()
{
QContext.clear();
}
/*******************************************************************************
** Every declared relation is a foreign key (or exempt), with the intended
** ON DELETE, and every foreign key column leads an index.
*******************************************************************************/
@Test
void everyRelationHasAForeignKey() throws Exception
{
TreeSet<String> declared = declaredRelations(WebhooksTestSupport.setUp());
Map<String, String> keys = foreignKeys();
List<String> missing = new ArrayList<>();
for(String relation : declared)
{
if(!keys.containsKey(relation) && !EXEMPT.containsKey(relation))
{
missing.add(relation);
}
}
assertThat(missing).as("relations declared in metadata without a foreign key in " + DDL + " (add one, or exempt it with a reason)").isEmpty();
assertThat(declared).as("every expected relation is still declared").containsAll(EXPECTED.keySet());
assertThat(keys).as("foreign keys and their ON DELETE").isEqualTo(EXPECTED);
assertThat(unindexedForeignKeys()).as("foreign key columns that lead no index").isEmpty();
}
/*******************************************************************************
** The upgrade script adds the same keys to a schema without them, recording
** and removing orphans first, and is a no-op the second time.
*******************************************************************************/
@Test
void upgradeQuarantinesOrphansAndIsIdempotent() throws Exception
{
try(Connection c = connect(); Statement st = c.createStatement())
{
dropForeignKeys(st);
st.execute("""
INSERT INTO webhooks_endpoint (id, name, url) VALUES (1, 'one', 'https://one.example');
INSERT INTO webhooks_subscription (id, endpoint_id, event_type_name) VALUES (1, 1, 'person.inserted');
INSERT INTO webhooks_delivery (id, event_id, event_type_name, endpoint_id, subscription_id, dedup_key, status)
VALUES (1, 'e1', 'person.inserted', 1, 1, 'd1', 'PENDING'),
(2, 'e2', 'person.inserted', 1, 99, 'd2', 'PENDING'),
(3, 'e3', 'person.inserted', 98, NULL, 'd3', 'PENDING');
INSERT INTO webhooks_delivery_attempt (id, delivery_id, endpoint_id, attempt_number, outcome)
VALUES (1, 1, 1, 1, 'SUCCESS'), (2, 3, 1, 1, 'FAILURE');
""");
st.execute(Files.readString(UPGRADE, StandardCharsets.UTF_8));
assertThat(foreignKeys()).isEqualTo(EXPECTED);
assertThat(quarantine(st)).containsExactlyInAnyOrder(
"webhooks_delivery.subscription_id row 2 parent 99 NULLED",
"webhooks_delivery.endpoint_id row 3 parent 98 DELETED",
"webhooks_delivery_attempt.delivery_id row 2 parent 3 DELETED");
assertThat(count(st, "SELECT COUNT(*) FROM webhooks_delivery")).isEqualTo(2);
assertThat(count(st, "SELECT COUNT(*) FROM webhooks_delivery WHERE id = 2 AND subscription_id IS NULL")).isEqualTo(1);
st.execute(Files.readString(UPGRADE, StandardCharsets.UTF_8));
assertThat(foreignKeys()).isEqualTo(EXPECTED);
assertThat(quarantine(st)).hasSize(3);
}
}
/*******************************************************************************
** Deleting an endpoint takes its secrets and subscriptions with it, and is
** refused while it has delivery history.
*******************************************************************************/
@Test
void endpointDeleteCascadesConfigurationButNotHistory() throws Exception
{
try(Connection c = connect(); Statement st = c.createStatement())
{
st.execute("""
INSERT INTO webhooks_endpoint (id, name, url) VALUES (1, 'one', 'https://one.example'), (2, 'two', 'https://two.example');
INSERT INTO webhooks_endpoint_secret (endpoint_id, source) VALUES (1, 'GENERATED'), (2, 'GENERATED');
INSERT INTO webhooks_subscription (id, endpoint_id, event_type_name) VALUES (1, 1, 'person.inserted'), (2, 2, 'person.inserted');
INSERT INTO webhooks_delivery (id, event_id, event_type_name, endpoint_id, subscription_id, dedup_key, status)
VALUES (1, 'e1', 'person.inserted', 2, 2, 'd1', 'DELIVERED');
INSERT INTO webhooks_delivery_attempt (delivery_id, endpoint_id, attempt_number, outcome) VALUES (1, 2, 1, 'SUCCESS');
""");
st.execute("DELETE FROM webhooks_endpoint WHERE id = 1");
assertThat(count(st, "SELECT COUNT(*) FROM webhooks_endpoint_secret WHERE endpoint_id = 1")).isZero();
assertThat(count(st, "SELECT COUNT(*) FROM webhooks_subscription WHERE endpoint_id = 1")).isZero();
assertThatThrownBy(() -> st.execute("DELETE FROM webhooks_endpoint WHERE id = 2")).isInstanceOf(SQLException.class).hasMessageContaining("fk_webhooks_delivery");
st.execute("DELETE FROM webhooks_subscription WHERE id = 2");
assertThat(count(st, "SELECT COUNT(*) FROM webhooks_delivery WHERE id = 1 AND subscription_id IS NULL")).isEqualTo(1);
st.execute("DELETE FROM webhooks_delivery WHERE id = 1");
assertThat(count(st, "SELECT COUNT(*) FROM webhooks_delivery_attempt")).isZero();
st.execute("DELETE FROM webhooks_endpoint WHERE id = 2");
}
}
/*******************************************************************************
** child_table.child_column -> parent_table for every join and table PVS
** field between two of the qbit's tables.
*******************************************************************************/
static TreeSet<String> declaredRelations(QInstance instance)
{
TreeSet<String> relations = new TreeSet<>();
for(QTableMetaData table : instance.getTables().values())
{
if(!isQbitTable(table))
{
continue;
}
for(QFieldMetaData field : table.getFields().values())
{
QPossibleValueSource pvs = field.getPossibleValueSourceName() == null ? null : instance.getPossibleValueSource(field.getPossibleValueSourceName());
if(pvs != null && pvs.getType() == QPossibleValueSourceType.TABLE && isQbitTable(instance.getTable(pvs.getTableName())))
{
relations.add(relation(table, field, instance.getTable(pvs.getTableName())));
}
}
}
for(QJoinMetaData join : instance.getJoins().values())
{
QTableMetaData left = instance.getTable(join.getLeftTable());
QTableMetaData right = instance.getTable(join.getRightTable());
if(!isQbitTable(left) || !isQbitTable(right))
{
continue;
}
for(JoinOn on : join.getJoinOns())
{
if(on.getLeftField().equals(left.getPrimaryKeyField()))
{
relations.add(relation(right, right.getField(on.getRightField()), left));
}
else if(on.getRightField().equals(right.getPrimaryKeyField()))
{
relations.add(relation(left, left.getField(on.getLeftField()), right));
}
}
}
return (relations);
}
/*******************************************************************************
** Whether a table is one of the qbit's.
*******************************************************************************/
private static boolean isQbitTable(QTableMetaData table)
{
return (table != null && table.getName().startsWith("webhooks_"));
}
/*******************************************************************************
** A relation key.
*******************************************************************************/
private static String relation(QTableMetaData child, QFieldMetaData field, QTableMetaData parent)
{
String column = field.getBackendName() != null ? field.getBackendName() : field.getName();
return (child.getName() + "." + column + " -> " + parent.getName());
}
/*******************************************************************************
** The single-column foreign keys in the database, with their ON DELETE.
*******************************************************************************/
private static Map<String, String> foreignKeys() throws SQLException
{
Map<String, String> keys = new TreeMap<>();
try(Connection c = connect(); Statement st = c.createStatement(); ResultSet rs = st.executeQuery("""
SELECT c.conrelid::regclass::text AS child, a.attname AS col, c.confrelid::regclass::text AS parent, c.confdeltype::text AS rule
FROM pg_constraint c JOIN pg_attribute a ON a.attrelid = c.conrelid AND a.attnum = c.conkey[1]
WHERE c.contype = 'f' AND cardinality(c.conkey) = 1
"""))
{
while(rs.next())
{
keys.put(rs.getString("child") + "." + rs.getString("col") + " -> " + rs.getString("parent"), rs.getString("rule"));
}
}
return (keys);
}
/*******************************************************************************
** Foreign key columns that are not the first column of any index.
*******************************************************************************/
private static List<String> unindexedForeignKeys() throws SQLException
{
List<String> list = new ArrayList<>();
try(Connection c = connect(); Statement st = c.createStatement(); ResultSet rs = st.executeQuery("""
SELECT c.conrelid::regclass::text || '.' || a.attname AS col
FROM pg_constraint c JOIN pg_attribute a ON a.attrelid = c.conrelid AND a.attnum = c.conkey[1]
WHERE c.contype = 'f'
AND NOT EXISTS (SELECT 1 FROM pg_index i WHERE i.indrelid = c.conrelid AND i.indkey[0] = c.conkey[1])
"""))
{
while(rs.next())
{
list.add(rs.getString("col"));
}
}
return (list);
}
/*******************************************************************************
** Drop every foreign key (a schema as it was before they were declared).
*******************************************************************************/
private static void dropForeignKeys(Statement st) throws SQLException
{
List<String> drops = new ArrayList<>();
try(ResultSet rs = st.executeQuery("SELECT conrelid::regclass::text AS t, conname FROM pg_constraint WHERE contype = 'f'"))
{
while(rs.next())
{
drops.add("ALTER TABLE " + rs.getString("t") + " DROP CONSTRAINT " + rs.getString("conname"));
}
}
for(String drop : drops)
{
st.execute(drop);
}
}
/*******************************************************************************
** The quarantine rows, as text.
*******************************************************************************/
private static List<String> quarantine(Statement st) throws SQLException
{
List<String> rows = new ArrayList<>();
try(ResultSet rs = st.executeQuery("SELECT table_name, column_name, row_id, missing_parent_id, action FROM qqq_fk_quarantine ORDER BY id"))
{
while(rs.next())
{
rows.add(rs.getString(1) + "." + rs.getString(2) + " row " + rs.getString(3) + " parent " + rs.getString(4) + " " + rs.getString(5));
}
}
return (rows);
}
/*******************************************************************************
** A count.
*******************************************************************************/
private static int count(Statement st, String sql) throws SQLException
{
try(ResultSet rs = st.executeQuery(sql))
{
rs.next();
return (rs.getInt(1));
}
}
/*******************************************************************************
** A connection.
*******************************************************************************/
private static Connection connect() throws SQLException
{
return (DriverManager.getConnection(postgres.getJdbcUrl(), "qqq", "qqq"));
}
}
@@ -0,0 +1,81 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks;
import java.util.List;
import com.kingsrook.qbits.webhooks.esb.WebhooksEsbSource;
import com.kingsrook.qbits.webhooks.metadata.WebhooksHelp;
import com.kingsrook.qqq.backend.core.actions.help.HelpTargetCatalog;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.help.bundles.HelpBundles;
import com.kingsrook.qqq.backend.core.testing.HelpBundleAssertions;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** Shipped help covers the qbit, including its optional ESB integration.
*******************************************************************************/
class WebhooksHelpBundleTest
{
/*******************************************************************************
** Validate all entries and live references against produced metadata.
*******************************************************************************/
@Test
void bundleIsComplete() throws Exception
{
QInstance instance = WebhooksTestSupport.setUp(config -> config.withEsbDestinations(List.of("events")));
WebhooksEsbSource.configure(instance);
HelpBundleAssertions.forInstance(instance)
.bundles("help/qbit-webhooks.yaml")
.owning(target -> target.name() != null && target.name().startsWith("webhooks"))
.assertComplete();
// RC.7 requires app/table/process/field coverage; also enforce our widgets and value sources.
var entries = WebhooksHelp.entries();
for(var target : HelpTargetCatalog.collect(instance))
{
if(target.name() != null && target.name().startsWith("webhooks")
&& List.of("widget", "possibleValueSource").contains(target.kind()))
{
assertThat(entries).as(target.key()).containsKey(target.key());
}
}
}
/*******************************************************************************
** Hosts need no help table or ESB integration to register the bundle.
*******************************************************************************/
@Test
void registersWithoutHelpTable() throws Exception
{
QInstance instance = WebhooksTestSupport.setUp();
assertThat(instance.getTable("helpContent")).isNull();
HelpBundles.syncOnStartup(instance);
assertThat(HelpBundles.of(instance).getBundles()).singleElement().satisfies(bundle ->
{
assertThat(bundle.getId()).isEqualTo("qbit-webhooks");
assertThat(bundle.getVersion()).as("the help bundle carries the artifact version").isNotBlank().isEqualTo(System.getProperty("qbit.artifactVersion")).isEqualTo(WebhooksQBitProducer.VERSION);
});
}
}
@@ -1,190 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.function.Consumer;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
/*******************************************************************************
** Unit test for WebhooksQBitConfig
*******************************************************************************/
class WebhooksQBitConfigTest extends BaseTest
{
/*******************************************************************************
**
*******************************************************************************/
@Test
void testValidation()
{
/////////////////////////////////////////////
// see corresponding todo in the main code //
/////////////////////////////////////////////
/*
///////////////////////////////////////////////////////////////////////
// defaultBackendNameForTables is optional, but if given, must exist //
///////////////////////////////////////////////////////////////////////
assertSuccess(new WebhooksQBitConfig().withDefaultBackendNameForTables(null));
assertSuccess(new WebhooksQBitConfig().withDefaultBackendNameForTables("myBackend"), qInstance -> qInstance.addBackend(new QBackendMetaData().withName("myBackend")));
assertError("Unrecognized defaultBackendNameForTables: noSuchBackend", new WebhooksQBitConfig().withDefaultBackendNameForTables("noSuchBackend"));
//////////////////////////////////////////////////////////
// scheduler name is optional, but if given, must exist //
//////////////////////////////////////////////////////////
assertSuccess(new WebhooksQBitConfig().withSchedulerName(null));
assertSuccess(new WebhooksQBitConfig().withSchedulerName("myScheduler"), qInstance -> qInstance.addScheduler(new SimpleSchedulerMetaData().withName("myScheduler")));
assertError("Unrecognized schedulerName: noSuchScheduler", new WebhooksQBitConfig().withSchedulerName("noSuchScheduler"));
*/
////////////////////////////////////////////////////////////////////////
// sendWebhookEventProcessRepeatSeconds - cannot be null, must be > 0 //
////////////////////////////////////////////////////////////////////////
assertSuccess(new WebhooksQBitConfig().withSendWebhookEventProcessRepeatSeconds(1));
assertError("sendWebhookEventProcessRepeatSeconds must be non-null and > 0", new WebhooksQBitConfig().withSendWebhookEventProcessRepeatSeconds(null));
assertError("sendWebhookEventProcessRepeatSeconds must be non-null and > 0", new WebhooksQBitConfig().withSendWebhookEventProcessRepeatSeconds(-1));
assertError("sendWebhookEventProcessRepeatSeconds must be non-null and > 0", new WebhooksQBitConfig().withSendWebhookEventProcessRepeatSeconds(0));
////////////////////////////////////////////////////////
// repeatedFailsToGoUnhealthy - can be null, else > 0 //
////////////////////////////////////////////////////////
assertSuccess(new WebhooksQBitConfig().withRepeatedFailsToGoUnhealthy(null));
assertSuccess(new WebhooksQBitConfig().withRepeatedFailsToGoUnhealthy(5));
assertError("If repeatedFailsToGoUnhealthy is given, it must be > 0", new WebhooksQBitConfig().withRepeatedFailsToGoUnhealthy(-1));
assertError("If repeatedFailsToGoUnhealthy is given, it must be > 0", new WebhooksQBitConfig().withRepeatedFailsToGoUnhealthy(0));
////////////////////////////////////////////////////////////////
// unhealthyToProbationTimeoutMinutes - can be null, else > 0 //
////////////////////////////////////////////////////////////////
assertSuccess(new WebhooksQBitConfig().withUnhealthyToProbationTimeoutMinutes(null));
assertSuccess(new WebhooksQBitConfig().withUnhealthyToProbationTimeoutMinutes(5));
assertError("If unhealthyToProbationTimeoutMinutes is given, it must be > 0", new WebhooksQBitConfig().withUnhealthyToProbationTimeoutMinutes(-1));
assertError("If unhealthyToProbationTimeoutMinutes is given, it must be > 0", new WebhooksQBitConfig().withUnhealthyToProbationTimeoutMinutes(0));
/////////////////////////////////////////////////////////////
// memoizationTimeoutMinutes - cannot be null, must be > 0 //
/////////////////////////////////////////////////////////////
assertSuccess(new WebhooksQBitConfig().withMemoizationTimeoutMinutes(1));
assertSuccess(new WebhooksQBitConfig().withMemoizationTimeoutMinutes(0));
assertError("memoizationTimeoutMinutes must be non-null and >= 0", new WebhooksQBitConfig().withMemoizationTimeoutMinutes(null));
assertError("memoizationTimeoutMinutes must be non-null and >= 0", new WebhooksQBitConfig().withMemoizationTimeoutMinutes(-1));
/////////////////////////////////////////////////////////////////
// maxSendAttemptsBeforeFailure - cannot be null, must be >= 1 //
/////////////////////////////////////////////////////////////////
assertSuccess(new WebhooksQBitConfig().withMaxSendAttemptsBeforeFailure(1));
assertError("maxSendAttemptsBeforeFailure must be non-null and >= 1", new WebhooksQBitConfig().withMaxSendAttemptsBeforeFailure(null));
assertError("maxSendAttemptsBeforeFailure must be non-null and >= 1", new WebhooksQBitConfig().withMaxSendAttemptsBeforeFailure(0));
assertError("maxSendAttemptsBeforeFailure must be non-null and >= 1", new WebhooksQBitConfig().withMaxSendAttemptsBeforeFailure(-1));
//////////////////////////////////////////////////////////////////////
// minutesToConsiderLeakedSendAttempt - cannot be null, must be > 0 //
//////////////////////////////////////////////////////////////////////
assertSuccess(new WebhooksQBitConfig().withMinutesToConsiderLeakedSendAttempt(1));
assertError("minutesToConsiderLeakedSendAttempt must be non-null and >= 1", new WebhooksQBitConfig().withMinutesToConsiderLeakedSendAttempt(null));
assertError("minutesToConsiderLeakedSendAttempt must be non-null and >= 1", new WebhooksQBitConfig().withMinutesToConsiderLeakedSendAttempt(0));
assertError("minutesToConsiderLeakedSendAttempt must be non-null and >= 1", new WebhooksQBitConfig().withMinutesToConsiderLeakedSendAttempt(-1));
/////////////////////////////////////////////////////////////////////////////////////
// minutesBetweenRetryAttempts - cannot be null or empty; all entries must be >= 0 //
/////////////////////////////////////////////////////////////////////////////////////
assertSuccess(new WebhooksQBitConfig().withMinutesBetweenRetryAttempts(List.of(0)));
assertSuccess(new WebhooksQBitConfig().withMinutesBetweenRetryAttempts(List.of(1)));
assertSuccess(new WebhooksQBitConfig().withMinutesBetweenRetryAttempts(List.of(1, 5, 30)));
assertError("minutesBetweenRetryAttempts must be non-null and non-empty", new WebhooksQBitConfig().withMinutesBetweenRetryAttempts(null));
assertError("minutesBetweenRetryAttempts must be non-null and non-empty", new WebhooksQBitConfig().withMinutesBetweenRetryAttempts(Collections.emptyList()));
assertError("minutesBetweenRetryAttempts must have all values >= 0", new WebhooksQBitConfig().withMinutesBetweenRetryAttempts(List.of(-1)));
assertError("minutesBetweenRetryAttempts must have all values >= 0", new WebhooksQBitConfig().withMinutesBetweenRetryAttempts(List.of(1, 5, -1)));
////////////////////////////////////////////////////////
// maxRateLimitRetries - cannot be null, must be >= 0 //
////////////////////////////////////////////////////////
assertSuccess(new WebhooksQBitConfig().withMaxRateLimitRetries(1));
assertSuccess(new WebhooksQBitConfig().withMaxRateLimitRetries(0));
assertError("maxRateLimitRetries must be non-null and >= 0", new WebhooksQBitConfig().withMaxRateLimitRetries(null));
assertError("maxRateLimitRetries must be non-null and >= 0", new WebhooksQBitConfig().withMaxRateLimitRetries(-1));
/////////////////////////////////////////////////////////////////
// initialRateLimitBackoffMillis - cannot be null, must be > 0 //
/////////////////////////////////////////////////////////////////
assertSuccess(new WebhooksQBitConfig().withInitialRateLimitBackoffMillis(1));
assertError("initialRateLimitBackoffMillis must be non-null and > 0", new WebhooksQBitConfig().withInitialRateLimitBackoffMillis(null));
assertError("initialRateLimitBackoffMillis must be non-null and > 0", new WebhooksQBitConfig().withInitialRateLimitBackoffMillis(-1));
assertError("initialRateLimitBackoffMillis must be non-null and > 0", new WebhooksQBitConfig().withInitialRateLimitBackoffMillis(0));
/////////////////////////////////////////////////////
// httpTimeoutMillis - cannot be null, must be > 0 //
/////////////////////////////////////////////////////
assertSuccess(new WebhooksQBitConfig().withHttpTimeoutMillis(1));
assertError("httpTimeoutMillis must be non-null and > 0", new WebhooksQBitConfig().withHttpTimeoutMillis(null));
assertError("httpTimeoutMillis must be non-null and > 0", new WebhooksQBitConfig().withHttpTimeoutMillis(-1));
assertError("httpTimeoutMillis must be non-null and > 0", new WebhooksQBitConfig().withHttpTimeoutMillis(0));
}
/***************************************************************************
**
***************************************************************************/
private void assertError(String expectedErrorMessage, WebhooksQBitConfig config)
{
List<String> errors = new ArrayList<>();
config.validate(new QInstance(), errors);
assertEquals(1, errors.size());
assertEquals(expectedErrorMessage, errors.get(0));
}
/***************************************************************************
**
***************************************************************************/
private void assertSuccess(WebhooksQBitConfig config)
{
assertSuccess(config, null);
}
/***************************************************************************
**
***************************************************************************/
private void assertSuccess(WebhooksQBitConfig config, Consumer<QInstance> qInstanceConsumer)
{
List<String> errors = new ArrayList<>();
QInstance qInstance = new QInstance();
if(qInstanceConsumer != null)
{
qInstanceConsumer.accept(qInstance);
}
config.validate(qInstance, errors);
assertEquals(0, errors.size());
}
}
@@ -45,6 +45,19 @@ import static org.assertj.core.api.Assertions.assertThatThrownBy;
*******************************************************************************/
class WebhooksQBitProducerTest
{
/*******************************************************************************
** Producer metadata carries the artifact version supplied by Maven.
*******************************************************************************/
@Test
void versionMatchesArtifact()
{
String artifactVersion = System.getProperty("qbit.artifactVersion");
assertThat(artifactVersion).isNotBlank();
assertThat(WebhooksQBitProducer.VERSION).isEqualTo(artifactVersion);
assertThat(new WebhooksQBitProducer().getQBitMetaData().getVersion()).isEqualTo(artifactVersion);
}
/*******************************************************************************
** Table names: lowercase snake_case with the webhooks_ prefix; backend
** column names snake_case; a valid instance.
@@ -90,7 +103,7 @@ class WebhooksQBitProducerTest
{
assertThat(child).isNotInstanceOf(QProcessMetaData.class);
}
assertThat(app.getWidgets()).containsExactly("webhooksHome");
assertThat(app.getWidgets()).containsExactly("webhooksHealth", "webhooksHome");
Set<String> endpointWidgets = instance.getTable(WebhooksTables.ENDPOINT).getSections().stream().map(s -> s.getWidgetName()).filter(w -> w != null).collect(Collectors.toSet());
assertThat(endpointWidgets).contains(WebhooksTables.WIDGET_ENDPOINT_SUBSCRIPTIONS, WebhooksTables.WIDGET_ENDPOINT_SECRETS, WebhooksTables.WIDGET_ENDPOINT_DELIVERIES);
}
@@ -0,0 +1,170 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.List;
import java.util.Map;
import java.util.stream.Stream;
import com.kingsrook.qbits.webhooks.metadata.WebhooksHelp;
import com.kingsrook.qbits.webhooks.metadata.WebhooksTables;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.help.QHelpContent;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** Guardrail (owner requirement 2026-10-14: "all tables should be populated and
** the qbit should be fully using them"). For every webhooks_* table:
** - runtime code (not just metadata or the home screen) reads or writes it;
** - it is either seeded by shipped defaults, or listed here with why it starts
** empty (nothing in this qbit is seeded: endpoints, secrets and
** subscriptions are the owner's choices, deliveries and attempts are logs);
** - it has bundle help that says how its rows get created (shown on the table
** screen and when it is empty).
** A new table fails this test until it is wired, seeded or explained.
*******************************************************************************/
class WebhooksTablesGuardrailTest
{
private static final Path MAIN = Path.of("src", "main", "java", "com", "kingsrook", "qbits", "webhooks");
/*******************************************************************************
** Tables that legitimately start empty, and where their rows come from.
*******************************************************************************/
static final Map<String, String> STARTS_EMPTY = Map.of(
WebhooksTables.ENDPOINT, "host-specific: an administrator adds each receiver URL",
WebhooksTables.SECRET, "secrets are never seeded: an administrator generates one or names an environment variable",
WebhooksTables.SUBSCRIPTION, "an administrator picks the event types an endpoint receives",
WebhooksTables.DELIVERY, "log: written when an event matches a subscription",
WebhooksTables.ATTEMPT, "log: written on every HTTP attempt");
/*******************************************************************************
** Every table is read or written by runtime code (delivery, events, secrets,
** processes), not just declared in metadata or shown on the home.
*******************************************************************************/
@Test
void everyTableIsUsedAtRuntime() throws Exception
{
List<String> runtimeSource;
try(Stream<Path> files = Files.walk(MAIN))
{
runtimeSource = files.filter(p -> p.toString().endsWith(".java"))
.filter(p -> !p.startsWith(MAIN.resolve("metadata")) && !p.startsWith(MAIN.resolve("app")))
.map(WebhooksTablesGuardrailTest::read).toList();
}
for(String table : WebhooksTables.ALL)
{
String constant = "WebhooksTables." + constantFor(table);
assertThat(runtimeSource.stream().anyMatch(source -> source.contains(constant)))
.as(table + " must be read or written by runtime code, not only declared").isTrue();
}
}
/*******************************************************************************
** Every table is seeded (none are) or explained.
*******************************************************************************/
@Test
void everyTableIsSeededOrExplained()
{
assertThat(STARTS_EMPTY.keySet()).containsExactlyInAnyOrderElementsOf(WebhooksTables.ALL);
}
/*******************************************************************************
** Every table and the app have bundle help that says how rows get created.
*******************************************************************************/
@Test
void everyTableHasEmptyStateHelp() throws Exception
{
QInstance instance = WebhooksTestSupport.setUp();
assertThat(WebhooksHelp.entries()).allSatisfy((key, entry) -> assertThat(entry.getSummary().length()).as(key).isLessThanOrEqualTo(120));
for(String table : WebhooksTables.ALL)
{
List<QHelpContent> help = instance.getTable(table).getHelpContent() == null ? null : instance.getTable(table).getHelpContent().get(WebhooksHelp.SLOT);
assertThat(help).as(table + " table help").isNotEmpty();
assertThat(help.get(0).getContent()).as(table + " help says how rows get created").contains("How rows get created");
}
assertThat(instance.getApp("webhooks").getHelpContent()).containsKey(WebhooksHelp.SLOT);
assertThat(WebhooksHelp.entries().get("table:webhooks_subscription;slot:help").getContent()).contains("available events");
assertThat(WebhooksHelp.entries().keySet()).allMatch(key -> !key.startsWith("table:") || instance.getTable(key.substring(6, key.indexOf(';'))) != null,
"help names only real tables");
}
/*******************************************************************************
** The WebhooksTables constant name of a table.
*******************************************************************************/
private static String constantFor(String table)
{
return (switch(table)
{
case WebhooksTables.ENDPOINT -> "ENDPOINT";
case WebhooksTables.SECRET -> "SECRET";
case WebhooksTables.SUBSCRIPTION -> "SUBSCRIPTION";
case WebhooksTables.DELIVERY -> "DELIVERY";
case WebhooksTables.ATTEMPT -> "ATTEMPT";
default -> throw new IllegalStateException("no constant for " + table);
});
}
/*******************************************************************************
** Read a file.
*******************************************************************************/
private static String read(Path path)
{
try
{
return (Files.readString(path, StandardCharsets.UTF_8));
}
catch(Exception e)
{
throw (new IllegalStateException(e));
}
}
/*******************************************************************************
** A host's own help in the slot is kept; the bundle only fills empty slots.
*******************************************************************************/
@Test
void hostHelpIsKept() throws Exception
{
QInstance instance = WebhooksTestSupport.setUp();
instance.getTable(WebhooksTables.ENDPOINT).withHelpContent(WebhooksHelp.SLOT, new QHelpContent().withContentAsMarkdown("host text"));
instance.getApp("webhooks").withHelpContent(WebhooksHelp.SLOT, new QHelpContent().withContentAsMarkdown("host app text"));
WebhooksHelp.apply(instance, WebhooksRuntime.config(instance));
assertThat(instance.getTable(WebhooksTables.ENDPOINT).getHelpContent().get(WebhooksHelp.SLOT)).singleElement().extracting(QHelpContent::getContent).isEqualTo("host text");
assertThat(instance.getApp("webhooks").getHelpContent().get(WebhooksHelp.SLOT)).singleElement().extracting(QHelpContent::getContent).isEqualTo("host app text");
}
}
@@ -1,199 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks;
import java.util.List;
import com.kingsrook.qbits.webhooks.model.WebhookEventCategory;
import com.kingsrook.qbits.webhooks.registry.WebhookEventType;
import com.kingsrook.qbits.webhooks.registry.WebhooksRegistry;
import com.kingsrook.qqq.api.model.APIVersion;
import com.kingsrook.qqq.api.model.metadata.ApiInstanceMetaData;
import com.kingsrook.qqq.api.model.metadata.ApiInstanceMetaDataContainer;
import com.kingsrook.qqq.api.model.metadata.ApiInstanceMetaDataProvider;
import com.kingsrook.qqq.api.model.metadata.tables.ApiTableMetaData;
import com.kingsrook.qqq.api.model.metadata.tables.ApiTableMetaDataContainer;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.instances.AbstractQQQApplication;
import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerMultiOutput;
import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType;
import com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.audits.AuditLevel;
import com.kingsrook.qqq.backend.core.model.metadata.audits.QAuditRules;
import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppSection;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
import com.kingsrook.qqq.backend.core.model.metadata.security.QSecurityKeyType;
import com.kingsrook.qqq.backend.core.model.metadata.security.RecordSecurityLock;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import com.kingsrook.qqq.backend.core.model.tables.QQQTablesMetaDataProvider;
import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryBackendModule;
/*******************************************************************************
**
*******************************************************************************/
public class WebhooksTestApplication extends AbstractQQQApplication
{
public static final String MEMORY_BACKEND_NAME = "memory";
public static final String TABLE_NAME_PERSON = "person";
public static final String TABLE_NAME_ORDER = "order";
public static final String API_NAME = "test-api";
public static final String API_PATH = "/test-api/";
public static final String API_V1 = "v1";
public static final String PERSON_INSERTED_EVENT_TYPE_NAME = "person.inserted";
public static final String PERSON_STORED_EVENT_TYPE_NAME = "person.stored";
public static final String STORE_ID_ALL_ACCESS_KEY = "storeIdAllAccess";
public static final String STORE_ID_KEY = "storeId";
/***************************************************************************
**
***************************************************************************/
@Override
public QInstance defineQInstance() throws QException
{
QInstance qInstance = new QInstance();
qInstance.withInstanceDefaultAuthentication(new QAuthenticationMetaData().withType(QAuthenticationType.FULLY_ANONYMOUS));
qInstance.addBackend(new QBackendMetaData()
.withName(MEMORY_BACKEND_NAME)
.withBackendType(MemoryBackendModule.class));
//////////////////////////////
// add tables table and PVS //
//////////////////////////////
new QQQTablesMetaDataProvider().defineAll(qInstance, MEMORY_BACKEND_NAME, MEMORY_BACKEND_NAME, null);
//////////////////////
// produce our qbit //
//////////////////////
WebhooksQBitProducer producer = new WebhooksQBitProducer()
.withQBitConfig(new WebhooksQBitConfig()
.withSecurityFields(List.of(new QFieldMetaData("storeId", QFieldType.INTEGER)))
.withRecordSecurityLocks(List.of(new RecordSecurityLock().withFieldName("storeId").withSecurityKeyType(STORE_ID_KEY)))
.withDefaultBackendNameForTables(MEMORY_BACKEND_NAME));
MetaDataProducerMultiOutput allQBitMetaData = producer.produce(qInstance);
allQBitMetaData.addSelfToInstance(qInstance);
//////////////////////////////
// add a webhook event type //
//////////////////////////////
WebhooksRegistry.ofOrWithNew(qInstance).registerWebhookEventType(new WebhookEventType()
.withName(PERSON_INSERTED_EVENT_TYPE_NAME)
.withLabel("Person Inserted")
.withCategory(WebhookEventCategory.INSERT)
.withTableName(TABLE_NAME_PERSON));
WebhooksRegistry.ofOrWithNew(qInstance).registerWebhookEventType(new WebhookEventType()
.withName(PERSON_STORED_EVENT_TYPE_NAME)
.withLabel("Person Stored")
.withCategory(WebhookEventCategory.STORE)
.withTableName(TABLE_NAME_PERSON));
/////////////
// add api //
/////////////
defineApiMetaData(qInstance);
ApiInstanceMetaDataProvider.definePossibleValueSourcesForApiNameAndVersion(qInstance);
////////////////////////////////////////
// produce some test tables //
// note - nice to be a qbit this too! //
////////////////////////////////////////
qInstance.addTable(new QTableMetaData()
.withName(TABLE_NAME_PERSON)
.withBackendName(MEMORY_BACKEND_NAME)
.withPrimaryKeyField("id")
.withField(new QFieldMetaData("id", QFieldType.INTEGER).withIsEditable(false))
.withField(new QFieldMetaData("firstName", QFieldType.STRING).withIsEditable(true))
.withField(new QFieldMetaData("lastName", QFieldType.STRING).withIsEditable(true))
.withSupplementalMetaData(new ApiTableMetaDataContainer().withApiTableMetaData(API_NAME, new ApiTableMetaData()
.withInitialVersion(API_V1)))
);
qInstance.addTable(new QTableMetaData()
.withName(TABLE_NAME_ORDER)
.withBackendName(MEMORY_BACKEND_NAME)
.withPrimaryKeyField("id")
.withField(new QFieldMetaData("id", QFieldType.INTEGER).withIsEditable(false))
.withField(new QFieldMetaData("orderNo", QFieldType.STRING).withIsEditable(true))
.withField(new QFieldMetaData("storeId", QFieldType.INTEGER).withIsEditable(true))
.withRecordSecurityLock(new RecordSecurityLock().withFieldName("storeId").withSecurityKeyType(STORE_ID_KEY))
.withSupplementalMetaData(new ApiTableMetaDataContainer().withApiTableMetaData(API_NAME, new ApiTableMetaData()
.withInitialVersion(API_V1)))
);
qInstance.addSecurityKeyType(new QSecurityKeyType().withName(STORE_ID_KEY).withAllAccessKeyName(STORE_ID_ALL_ACCESS_KEY));
///////////////////////////////////////////
// turn off audits (why on by default??) //
///////////////////////////////////////////
qInstance.getTables().values().forEach(t -> t.setAuditRules(new QAuditRules().withAuditLevel(AuditLevel.NONE)));
/////////////////
// create apps //
/////////////////
qInstance.addApp(new QAppMetaData()
.withName("webhooks")
.withIcon(new QIcon("webhooks"))
.withSection(WebhooksQBitProducer.produceAppSection()));
qInstance.addApp(new QAppMetaData()
.withName("testData")
.withIcon(new QIcon("dataset"))
.withSection(new QAppSection()
.withName("people")
.withTables(List.of(TABLE_NAME_PERSON))));
return qInstance;
}
/***************************************************************************
**
***************************************************************************/
private void defineApiMetaData(QInstance qInstance)
{
qInstance.withSupplementalMetaData(new ApiInstanceMetaDataContainer()
.withApiInstanceMetaData(new ApiInstanceMetaData()
.withName(API_NAME)
.withPath(API_PATH)
.withLabel("Test API")
.withDescription("QQQ Test API")
.withContactEmail("contact@kingsrook.com")
.withCurrentVersion(new APIVersion(API_V1))
.withSupportedVersions(List.of(new APIVersion(API_V1)))
));
}
}
@@ -1,66 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks;
import java.util.List;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.middleware.javalin.QApplicationJavalinServer;
import com.kingsrook.qqq.middleware.javalin.specs.v1.MiddlewareVersionV1;
/*******************************************************************************
**
*******************************************************************************/
public class WebhooksTestJavalinServer
{
/***************************************************************************
**
***************************************************************************/
public static void main(String[] args)
{
startApplicationJavalinServer();
}
/***************************************************************************
**
***************************************************************************/
private static void startApplicationJavalinServer()
{
try
{
System.setProperty("qqq.javalin.hotSwapInstance", "true");
QApplicationJavalinServer javalinServer = new QApplicationJavalinServer(new WebhooksTestApplication())
.withMiddlewareVersionList(List.of(new MiddlewareVersionV1()))
.withServeFrontendMaterialDashboard(false);
javalinServer.start();
}
catch(QException e)
{
e.printStackTrace();
}
}
}
@@ -1,143 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks;
import java.time.Instant;
import java.util.List;
import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.instances.AbstractQQQApplication;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType;
import com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppSection;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryBackendModule;
import com.kingsrook.qqq.middleware.javalin.QJavalinMetaData;
import com.kingsrook.qqq.middleware.javalin.metadata.JavalinRouteProviderMetaData;
import com.kingsrook.qqq.middleware.javalin.routeproviders.ProcessBasedRouterPayload;
/*******************************************************************************
**
*******************************************************************************/
public class WebhooksTestReceiverApplication extends AbstractQQQApplication
{
public static final String MEMORY_BACKEND_NAME = "memory";
public static final String RECEIPT_TABLE_NAME = "receipts";
public static final String RECEIVE_PROCESS_NAME = "receiveWebhook";
/***************************************************************************
**
***************************************************************************/
@Override
public QInstance defineQInstance() throws QException
{
QInstance qInstance = new QInstance();
qInstance.withInstanceDefaultAuthentication(new QAuthenticationMetaData().withType(QAuthenticationType.FULLY_ANONYMOUS));
qInstance.addBackend(new QBackendMetaData()
.withName(MEMORY_BACKEND_NAME)
.withBackendType(MemoryBackendModule.class));
qInstance.addTable(new QTableMetaData()
.withName(RECEIPT_TABLE_NAME)
.withBackendName(MEMORY_BACKEND_NAME)
.withPrimaryKeyField("id")
.withField(new QFieldMetaData("id", QFieldType.INTEGER).withIsEditable(false))
.withField(new QFieldMetaData("timestamp", QFieldType.DATE_TIME).withIsEditable(true))
.withField(new QFieldMetaData("content", QFieldType.STRING).withIsEditable(true)));
qInstance.addProcess(new QProcessMetaData()
.withName(RECEIVE_PROCESS_NAME)
.withStep(new QBackendStepMetaData()
.withName("execute")
.withCode(new QCodeReference(ReceiveWebhookProcessStep.class))));
///////////////////////////////////////////
// turn off audits (why on by default??) //
///////////////////////////////////////////
//? qInstance.getTables().values().forEach(t -> t.setAuditRules(new QAuditRules().withAuditLevel(AuditLevel.NONE)));
/////////////////
// create apps //
/////////////////
qInstance.addApp(new QAppMetaData()
.withName("receiptsApp")
.withLabel("Receipts")
.withIcon(new QIcon("receipt_long"))
.withSection(new QAppSection()
.withName("data")
.withTables(List.of(RECEIPT_TABLE_NAME))));
QJavalinMetaData.ofOrWithNew(qInstance)
.withRouteProvider(new JavalinRouteProviderMetaData()
.withName("defaultReceiver")
.withProcessName(RECEIVE_PROCESS_NAME)
.withHostedPath("/" + RECEIVE_PROCESS_NAME)
.withMethods(List.of("POST", "GET")));
return qInstance;
}
/***************************************************************************
**
***************************************************************************/
public static class ReceiveWebhookProcessStep implements BackendStep
{
/***************************************************************************
**
***************************************************************************/
@Override
public void run(RunBackendStepInput runBackendStepInput, RunBackendStepOutput runBackendStepOutput) throws QException
{
ProcessBasedRouterPayload processPayload = runBackendStepInput.getProcessPayload(ProcessBasedRouterPayload.class);
String content = processPayload.getBodyString();
QRecord insertedRecord = new InsertAction().execute(new InsertInput(RECEIPT_TABLE_NAME).withRecord(new QRecord().withValue("timestamp", Instant.now()).withValue("content", content))).getRecords().get(0);
System.out.println("I received and inserted: " + insertedRecord.getValues());
processPayload.setResponseString("OK");
runBackendStepOutput.setProcessPayload(processPayload);
}
}
}
@@ -1,65 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.middleware.javalin.QApplicationJavalinServer;
/*******************************************************************************
**
*******************************************************************************/
public class WebhooksTestReceiverJavalinServer
{
/***************************************************************************
**
***************************************************************************/
public static void main(String[] args)
{
startApplicationJavalinServer();
}
/***************************************************************************
**
***************************************************************************/
private static void startApplicationJavalinServer()
{
try
{
System.setProperty("qqq.javalin.hotSwapInstance", "false");
Integer port = Integer.parseInt(System.getProperty("port", "7999"));
QApplicationJavalinServer javalinServer = new QApplicationJavalinServer(new WebhooksTestReceiverApplication())
.withPort(port)
.withServeFrontendMaterialDashboard(false);
javalinServer.start();
}
catch(QException e)
{
e.printStackTrace();
}
}
}
@@ -1,118 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.actions;
import java.util.List;
import com.kingsrook.qbits.webhooks.BaseTest;
import com.kingsrook.qbits.webhooks.WebhooksTestApplication;
import com.kingsrook.qbits.webhooks.model.WebhookEvent;
import com.kingsrook.qbits.webhooks.model.WebhookEventCategory;
import com.kingsrook.qbits.webhooks.registry.WebhookEventType;
import com.kingsrook.qbits.webhooks.registry.WebhooksRegistry;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QCollectingLogger;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import org.json.JSONObject;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
/*******************************************************************************
** Unit test for FireAdHocWebhookEvent
*******************************************************************************/
class FireAdHocWebhookEventTest extends BaseTest
{
/*******************************************************************************
**
*******************************************************************************/
@Test
void test() throws QException
{
QCollectingLogger collectingLogger = QLogger.activateCollectingLoggerForClass(FirePostInsertOrUpdateWebhookEventUtil.class);
WebhooksRegistry webhooksRegistry = WebhooksRegistry.of(QContext.getQInstance());
String typeA = "typeA";
webhooksRegistry.registerWebhookEventType(new WebhookEventType()
.withName(typeA)
.withCategory(WebhookEventCategory.AD_HOC));
String typeB = "typeB";
webhooksRegistry.registerWebhookEventType(new WebhookEventType()
.withName(typeB)
.withCategory(WebhookEventCategory.AD_HOC));
////////////////////////////////////////////////
// fire without any webhook - assert no log //
////////////////////////////////////////////////
FireAdHocWebhookEvent fireAdHocWebhookEvent = new FireAdHocWebhookEvent();
fireAdHocWebhookEvent.fire(typeA, WebhooksTestApplication.TABLE_NAME_PERSON, new QRecord().withValue("id", 1).withValue("firstName", "James T."), null);
assertThat(collectingLogger.getCollectedMessages())
.noneMatch(m -> m.getMessage().matches(".*Building webhook event.*"));
/////////////////////////////////////////////////////
// make a sub - then fire, and assert positive log //
/////////////////////////////////////////////////////
insert(newWebhookSubscription(typeA));
WebhookSubscriptionsHelper.clearMemoizations();
fireAdHocWebhookEvent.fire(typeA, WebhooksTestApplication.TABLE_NAME_PERSON, new QRecord().withValue("id", 2).withValue("firstName", "Jean Luc"), null);
assertThat(collectingLogger.getCollectedMessages())
.anyMatch(m -> m.getMessage().matches(".*Building webhook event.*"));
collectingLogger.clear();
///////////////////////////////////////////////
// query to make sure event was inserted too //
///////////////////////////////////////////////
List<QRecord> insertedEvents = new QueryAction().execute(new QueryInput(WebhookEvent.TABLE_NAME).withIncludeAssociations(true)).getRecords();
assertEquals(1, insertedEvents.size());
assertEquals(2, insertedEvents.get(0).getValueInteger("eventSourceRecordId"));
////////////////////////////////////
// assert about the contents too! //
////////////////////////////////////
List<QRecord> contents = insertedEvents.get(0).getAssociatedRecords().get(WebhookEvent.CONTENT_ASSOCIATION_NAME);
assertEquals(1, contents.size());
String postBody = contents.get(0).getValueString("postBody");
assertNotNull(postBody);
JSONObject postBodyJson = new JSONObject(postBody);
assertTrue(postBodyJson.has("record"));
assertEquals("Jean Luc", postBodyJson.getJSONObject("record").getString("firstName"));
assertEquals(2, postBodyJson.getJSONObject("record").getInt("id"));
/////////////////////////////////////////////////////////////////
// fire for the other ad-hoc with no subs - and should NOT log //
/////////////////////////////////////////////////////////////////
fireAdHocWebhookEvent.fire(typeB, WebhooksTestApplication.TABLE_NAME_PERSON, new QRecord().withValue("id", 1).withValue("firstName", "James T."), null);
assertThat(collectingLogger.getCollectedMessages())
.noneMatch(m -> m.getMessage().matches(".*Building webhook event.*"));
}
}
@@ -1,223 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.actions;
import java.util.ArrayList;
import java.util.List;
import com.kingsrook.qbits.webhooks.BaseTest;
import com.kingsrook.qbits.webhooks.WebhooksTestApplication;
import com.kingsrook.qbits.webhooks.model.WebhookEvent;
import com.kingsrook.qbits.webhooks.model.WebhookEventCategory;
import com.kingsrook.qbits.webhooks.model.WebhooksActionFlags;
import com.kingsrook.qbits.webhooks.registry.WebhookEventType;
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QCollectingLogger;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import org.json.JSONObject;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
/*******************************************************************************
** Unit test for FirePostInsertWebhookEvent
*******************************************************************************/
class FirePostInsertWebhookEventTest extends BaseTest
{
/*******************************************************************************
**
*******************************************************************************/
@Test
void test() throws QException
{
try
{
QCollectingLogger collectingLogger = QLogger.activateCollectingLoggerForClass(FirePostInsertOrUpdateWebhookEventUtil.class);
////////////////////////////////////////////////
// insert without any webhook - assert no log //
////////////////////////////////////////////////
new InsertAction().execute(new InsertInput(WebhooksTestApplication.TABLE_NAME_PERSON).withRecord(new QRecord()));
assertThat(collectingLogger.getCollectedMessages())
.noneMatch(m -> m.getMessage().matches(".*Building webhook event.*"));
///////////////////////////////////////////////////////////////////
// register event type and make a sub - then assert positive log //
///////////////////////////////////////////////////////////////////
insert(newWebhookSubscription(WebhooksTestApplication.PERSON_INSERTED_EVENT_TYPE_NAME));
WebhookSubscriptionsHelper.clearMemoizations();
Integer personId = new InsertAction().execute(new InsertInput(WebhooksTestApplication.TABLE_NAME_PERSON)
.withRecord(new QRecord().withValue("firstName", "Bubba").withValue("lastName", "Fit")))
.getRecords().get(0).getValueInteger("id");
assertThat(collectingLogger.getCollectedMessages())
.anyMatch(m -> m.getMessage().matches(".*Building webhook event.*"));
collectingLogger.clear();
///////////////////////////////////////////////
// query to make sure event was inserted too //
///////////////////////////////////////////////
List<QRecord> insertedEvents = new QueryAction().execute(new QueryInput(WebhookEvent.TABLE_NAME).withIncludeAssociations(true)).getRecords();
assertEquals(1, insertedEvents.size());
assertEquals(personId, insertedEvents.get(0).getValueInteger("eventSourceRecordId"));
///////////////////////////////////////
// run again, with omit flag present //
///////////////////////////////////////
new InsertAction().execute(new InsertInput(WebhooksTestApplication.TABLE_NAME_PERSON)
.withFlag(WebhooksActionFlags.OMIT_WEBHOOKS)
.withRecord(new QRecord().withValue("firstName", "Bubba").withValue("lastName", "Fit")))
.getRecords().get(0).getValueInteger("id");
assertThat(collectingLogger.getCollectedMessages())
.noneMatch(m -> m.getMessage().matches(".*Building webhook event.*"));
////////////////////////////////////
// assert about the contents too! //
////////////////////////////////////
List<QRecord> contents = insertedEvents.get(0).getAssociatedRecords().get(WebhookEvent.CONTENT_ASSOCIATION_NAME);
assertEquals(1, contents.size());
String postBody = contents.get(0).getValueString("postBody");
assertNotNull(postBody);
JSONObject postBodyJson = new JSONObject(postBody);
assertTrue(postBodyJson.has("record"));
assertEquals("Bubba", postBodyJson.getJSONObject("record").getString("firstName"));
assertEquals(personId, postBodyJson.getJSONObject("record").getInt("id"));
}
finally
{
QLogger.deactivateCollectingLoggerForClass(FirePostInsertOrUpdateWebhookEventUtil.class);
}
}
/*******************************************************************************
**
*******************************************************************************/
@Test
void testStoreEventKind() throws QException
{
try
{
QCollectingLogger collectingLogger = QLogger.activateCollectingLoggerForClass(FirePostInsertOrUpdateWebhookEventUtil.class);
/////////////////////////////////////////////////////////////////////////////
// register a 'store' event type and make a sub - then assert positive log //
/////////////////////////////////////////////////////////////////////////////
insert(newWebhookSubscription(WebhooksTestApplication.PERSON_STORED_EVENT_TYPE_NAME));
Integer personId = new InsertAction().execute(new InsertInput(WebhooksTestApplication.TABLE_NAME_PERSON)
.withRecord(new QRecord().withValue("firstName", "Bubba").withValue("lastName", "Fit")))
.getRecords().get(0).getValueInteger("id");
assertThat(collectingLogger.getCollectedMessages())
.anyMatch(m -> m.getMessage().matches(".*Building webhook event.*"));
collectingLogger.clear();
///////////////////////////////////////////////
// query to make sure event was inserted too //
///////////////////////////////////////////////
List<QRecord> insertedEvents = new QueryAction().execute(new QueryInput(WebhookEvent.TABLE_NAME).withIncludeAssociations(true)).getRecords();
assertEquals(1, insertedEvents.size());
assertEquals(personId, insertedEvents.get(0).getValueInteger("eventSourceRecordId"));
}
finally
{
QLogger.deactivateCollectingLoggerForClass(FirePostInsertOrUpdateWebhookEventUtil.class);
}
}
/*******************************************************************************
**
*******************************************************************************/
@Test
void testDoesRecordMatchWebhookEventType()
{
WebhookEventType insert = new WebhookEventType().withCategory(WebhookEventCategory.INSERT);
WebhookEventType insertFirstName = new WebhookEventType().withCategory(WebhookEventCategory.INSERT_WITH_FIELD).withFieldName("firstName");
WebhookEventType insertFirstNameJohn = new WebhookEventType().withCategory(WebhookEventCategory.INSERT_WITH_VALUE).withFieldName("firstName").withValue("John");
WebhookEventType insertFirstNameAnyBeatle = new WebhookEventType().withCategory(WebhookEventCategory.INSERT_WITH_VALUE).withFieldName("firstName").withValue(new ArrayList<>(List.of("John", "George", "Paul", "Ringo")));
WebhookEventType store = new WebhookEventType().withCategory(WebhookEventCategory.STORE);
WebhookEventType storeFirstName = new WebhookEventType().withCategory(WebhookEventCategory.STORE_WITH_FIELD).withFieldName("firstName");
WebhookEventType storeFirstNameJohn = new WebhookEventType().withCategory(WebhookEventCategory.STORE_WITH_VALUE).withFieldName("firstName").withValue("John");
QFieldMetaData field = QContext.getQInstance().getTable(WebhooksTestApplication.TABLE_NAME_PERSON).getField("firstName");
QRecord emptyRecord = new QRecord();
assertTrue(doesMatch(emptyRecord, insert, field));
assertFalse(doesMatch(emptyRecord, insertFirstName, field));
assertFalse(doesMatch(emptyRecord, insertFirstNameJohn, field));
assertFalse(doesMatch(emptyRecord, insertFirstNameAnyBeatle, field));
assertTrue(doesMatch(emptyRecord, store, field));
assertFalse(doesMatch(emptyRecord, storeFirstName, field));
assertFalse(doesMatch(emptyRecord, storeFirstNameJohn, field));
QRecord firstNameJoeRecord = new QRecord().withValue("firstName", "Joe");
assertTrue(doesMatch(firstNameJoeRecord, insert, field));
assertTrue(doesMatch(firstNameJoeRecord, insertFirstName, field));
assertFalse(doesMatch(firstNameJoeRecord, insertFirstNameJohn, field));
assertFalse(doesMatch(firstNameJoeRecord, insertFirstNameAnyBeatle, field));
assertTrue(doesMatch(firstNameJoeRecord, store, field));
assertTrue(doesMatch(firstNameJoeRecord, storeFirstName, field));
assertFalse(doesMatch(firstNameJoeRecord, storeFirstNameJohn, field));
QRecord firstNameJohnRecord = new QRecord().withValue("firstName", "John");
assertTrue(doesMatch(firstNameJohnRecord, insert, field));
assertTrue(doesMatch(firstNameJohnRecord, insertFirstName, field));
assertTrue(doesMatch(firstNameJohnRecord, insertFirstNameJohn, field));
assertTrue(doesMatch(firstNameJohnRecord, insertFirstNameAnyBeatle, field));
assertTrue(doesMatch(firstNameJohnRecord, store, field));
assertTrue(doesMatch(firstNameJohnRecord, storeFirstName, field));
assertTrue(doesMatch(firstNameJohnRecord, storeFirstNameJohn, field));
QRecord firstNameRingoRecord = new QRecord().withValue("firstName", "Ringo");
assertTrue(doesMatch(firstNameRingoRecord, insert, field));
assertTrue(doesMatch(firstNameRingoRecord, insertFirstName, field));
assertFalse(doesMatch(firstNameRingoRecord, insertFirstNameJohn, field));
assertTrue(doesMatch(firstNameRingoRecord, insertFirstNameAnyBeatle, field));
assertTrue(doesMatch(firstNameRingoRecord, store, field));
assertTrue(doesMatch(firstNameRingoRecord, storeFirstName, field));
assertFalse(doesMatch(firstNameRingoRecord, storeFirstNameJohn, field));
}
/***************************************************************************
**
***************************************************************************/
private boolean doesMatch(QRecord record, WebhookEventType webhookEventType, QFieldMetaData field)
{
return new FirePostInsertWebhookEvent().doesRecordMatchWebhookEventType(record, webhookEventType, field);
}
}
@@ -1,306 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.actions;
import java.util.ArrayList;
import java.util.List;
import java.util.Optional;
import com.kingsrook.qbits.webhooks.BaseTest;
import com.kingsrook.qbits.webhooks.WebhooksTestApplication;
import com.kingsrook.qbits.webhooks.model.Webhook;
import com.kingsrook.qbits.webhooks.model.WebhookEvent;
import com.kingsrook.qbits.webhooks.model.WebhookEventCategory;
import com.kingsrook.qbits.webhooks.model.WebhooksActionFlags;
import com.kingsrook.qbits.webhooks.registry.WebhookEventType;
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QCollectingLogger;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import org.json.JSONObject;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
/*******************************************************************************
** Unit test for FirePostInsertWebhookEvent
*******************************************************************************/
class FirePostUpdateWebhookEventTest extends BaseTest
{
/*******************************************************************************
**
*******************************************************************************/
@Test
void test() throws QException
{
try
{
Integer personId = new InsertAction().execute(new InsertInput(WebhooksTestApplication.TABLE_NAME_PERSON).withRecord(new QRecord())).getRecords().get(0).getValueInteger("id");
QCollectingLogger collectingLogger = QLogger.activateCollectingLoggerForClass(FirePostInsertOrUpdateWebhookEventUtil.class);
////////////////////////////////////////////////
// update without any webhook - assert no log //
////////////////////////////////////////////////
new UpdateAction().execute(new UpdateInput(WebhooksTestApplication.TABLE_NAME_PERSON).withRecord(new QRecord().withValue("id", personId).withValue("firstName", "Bubba")));
assertThat(collectingLogger.getCollectedMessages())
.noneMatch(m -> m.getMessage().matches(".*Building webhook event.*"));
///////////////////////////////////////////////////////////////////
// register event type and make a sub - then assert positive log //
///////////////////////////////////////////////////////////////////
String eventTypeName = "updatedPerson";
registerEventType(eventTypeName, WebhookEventCategory.UPDATE, WebhooksTestApplication.TABLE_NAME_PERSON);
Integer subscriptionId = insert(newWebhookSubscription(eventTypeName));
new UpdateAction().execute(new UpdateInput(WebhooksTestApplication.TABLE_NAME_PERSON)
.withRecord(new QRecord().withValue("id", personId).withValue("firstName", "Bobby")));
assertThat(collectingLogger.getCollectedMessages())
.anyMatch(m -> m.getMessage().matches(".*Building webhook event.*"));
collectingLogger.clear();
///////////////////////////////////////////////
// query to make sure event was inserted too //
///////////////////////////////////////////////
List<QRecord> insertedEvents = new QueryAction().execute(new QueryInput(WebhookEvent.TABLE_NAME)
.withFilter(new QQueryFilter(new QFilterCriteria("webhookSubscriptionId", QCriteriaOperator.EQUALS, subscriptionId)))
.withIncludeAssociations(true)).getRecords();
assertEquals(1, insertedEvents.size());
assertEquals(personId, insertedEvents.get(0).getValueInteger("eventSourceRecordId"));
///////////////////////////////////////
// run again, with omit flag present //
///////////////////////////////////////
new UpdateAction().execute(new UpdateInput(WebhooksTestApplication.TABLE_NAME_PERSON)
.withFlag(WebhooksActionFlags.OMIT_WEBHOOKS)
.withRecord(new QRecord().withValue("id", personId).withValue("firstName", "Bobby")));
assertThat(collectingLogger.getCollectedMessages())
.noneMatch(m -> m.getMessage().matches(".*Building webhook event.*"));
////////////////////////////////////
// assert about the contents too! //
////////////////////////////////////
List<QRecord> contents = insertedEvents.get(0).getAssociatedRecords().get(WebhookEvent.CONTENT_ASSOCIATION_NAME);
assertEquals(1, contents.size());
String postBody = contents.get(0).getValueString("postBody");
assertNotNull(postBody);
JSONObject postBodyJson = new JSONObject(postBody);
assertTrue(postBodyJson.has("record"));
assertEquals("Bobby", postBodyJson.getJSONObject("record").getString("firstName"));
assertEquals(personId, postBodyJson.getJSONObject("record").getInt("id"));
}
finally
{
QLogger.deactivateCollectingLoggerForClass(FirePostInsertOrUpdateWebhookEventUtil.class);
}
}
/*******************************************************************************
**
*******************************************************************************/
@Test
void testStoreEventType() throws QException
{
try
{
Integer personId = new InsertAction().execute(new InsertInput(WebhooksTestApplication.TABLE_NAME_PERSON).withRecord(new QRecord())).getRecords().get(0).getValueInteger("id");
QCollectingLogger collectingLogger = QLogger.activateCollectingLoggerForClass(FirePostInsertOrUpdateWebhookEventUtil.class);
///////////////////////////////////////////////////////////////////
// register event type and make a sub - then assert positive log //
///////////////////////////////////////////////////////////////////
Integer subscriptionId = insert(newWebhookSubscription(WebhooksTestApplication.PERSON_STORED_EVENT_TYPE_NAME));
new UpdateAction().execute(new UpdateInput(WebhooksTestApplication.TABLE_NAME_PERSON)
.withRecord(new QRecord().withValue("id", personId).withValue("firstName", "Bobby")));
assertThat(collectingLogger.getCollectedMessages())
.anyMatch(m -> m.getMessage().matches(".*Building webhook event.*"));
collectingLogger.clear();
///////////////////////////////////////////////
// query to make sure event was inserted too //
///////////////////////////////////////////////
List<QRecord> insertedEvents = new QueryAction().execute(new QueryInput(WebhookEvent.TABLE_NAME)
.withFilter(new QQueryFilter(new QFilterCriteria("webhookSubscriptionId", QCriteriaOperator.EQUALS, subscriptionId)))
.withIncludeAssociations(true)).getRecords();
assertEquals(1, insertedEvents.size());
assertEquals(personId, insertedEvents.get(0).getValueInteger("eventSourceRecordId"));
}
finally
{
QLogger.deactivateCollectingLoggerForClass(FirePostInsertOrUpdateWebhookEventUtil.class);
}
}
/*******************************************************************************
**
*******************************************************************************/
@Test
void testSecurityKeys() throws QException
{
Integer storeA = 17;
Integer storeB = 18;
Integer orderIdA = new InsertAction().execute(new InsertInput(WebhooksTestApplication.TABLE_NAME_ORDER).withRecord(new QRecord().withValue("storeId", storeA))).getRecords().get(0).getValueInteger("id");
Integer orderIdB = new InsertAction().execute(new InsertInput(WebhooksTestApplication.TABLE_NAME_ORDER).withRecord(new QRecord().withValue("storeId", storeB))).getRecords().get(0).getValueInteger("id");
/////////////////////////////////////////////////////////////////////
// register event type and make a webhook with a sub - for store A //
/////////////////////////////////////////////////////////////////////
String eventTypeName = "updatedOrder";
registerEventType(eventTypeName, WebhookEventCategory.UPDATE, WebhooksTestApplication.TABLE_NAME_ORDER);
Integer webhookId = new InsertAction().execute(new InsertInput(Webhook.TABLE_NAME).withRecord(newWebhook(eventTypeName).toQRecord().withValue("storeId", storeA))).getRecords().get(0).getValueInteger("id");
Integer subscriptionId = insert(newWebhookSubscription(eventTypeName).withWebhookId(webhookId));
///////////////////////////////////////////////////////////////////////////////////
// update the 2 orders - only an event for A should fire //
// of note - the record that's updated doesn't have its storeId field in it - //
// so originally this case failed to match the subscription - but, now it works. //
///////////////////////////////////////////////////////////////////////////////////
new UpdateAction().execute(new UpdateInput(WebhooksTestApplication.TABLE_NAME_ORDER).withRecord(new QRecord().withValue("id", orderIdA).withValue("orderNo", "A")));
new UpdateAction().execute(new UpdateInput(WebhooksTestApplication.TABLE_NAME_ORDER).withRecord(new QRecord().withValue("id", orderIdB).withValue("orderNo", "B")));
List<QRecord> insertedEvents = new QueryAction().execute(new QueryInput(WebhookEvent.TABLE_NAME)
.withFilter(new QQueryFilter(new QFilterCriteria("webhookSubscriptionId", QCriteriaOperator.EQUALS, subscriptionId)))
.withIncludeAssociations(true)).getRecords();
assertEquals(1, insertedEvents.size());
assertEquals(orderIdA, insertedEvents.get(0).getValueInteger("eventSourceRecordId"));
}
/*******************************************************************************
**
*******************************************************************************/
@Test
void testDoesRecordMatchWebhookEventType()
{
WebhookEventType update = new WebhookEventType().withTableName(WebhooksTestApplication.TABLE_NAME_PERSON).withCategory(WebhookEventCategory.UPDATE);
WebhookEventType updateFirstName = new WebhookEventType().withTableName(WebhooksTestApplication.TABLE_NAME_PERSON).withCategory(WebhookEventCategory.UPDATE_WITH_FIELD).withFieldName("firstName");
WebhookEventType updateFirstNameJohn = new WebhookEventType().withTableName(WebhooksTestApplication.TABLE_NAME_PERSON).withCategory(WebhookEventCategory.UPDATE_WITH_VALUE).withFieldName("firstName").withValue("John");
WebhookEventType updateFirstNameAnyBeatle = new WebhookEventType().withTableName(WebhooksTestApplication.TABLE_NAME_PERSON).withCategory(WebhookEventCategory.UPDATE_WITH_VALUE).withFieldName("firstName").withValue(new ArrayList<>(List.of("John", "George", "Paul", "Ringo")));
////////////////////////////////////////////////
// w/ no values, only the generic rule passes //
////////////////////////////////////////////////
QRecord blankRecord = new QRecord();
assertTrue(doesMatch(blankRecord, blankRecord, update));
assertFalse(doesMatch(blankRecord, blankRecord, updateFirstName));
assertFalse(doesMatch(blankRecord, blankRecord, updateFirstNameJohn));
assertFalse(doesMatch(blankRecord, blankRecord, updateFirstNameAnyBeatle));
/////////////////////////////////////////////////////////////////////////
// w/ firstname set to Joe, the generic and "any firstname" rules pass //
/////////////////////////////////////////////////////////////////////////
QRecord firstNameJoeRecord = new QRecord().withValue("firstName", "Joe");
assertTrue(doesMatch(firstNameJoeRecord, blankRecord, update));
assertTrue(doesMatch(firstNameJoeRecord, blankRecord, updateFirstName));
assertFalse(doesMatch(firstNameJoeRecord, blankRecord, updateFirstNameJohn));
assertFalse(doesMatch(firstNameJoeRecord, blankRecord, updateFirstNameAnyBeatle));
/////////////////////////////////////////////////////////////////////////////////
// an update that doesn't change firstName won't fire the "any firstname" rule //
/////////////////////////////////////////////////////////////////////////////////
QRecord firstNameJoeLastNameSmithRecord = new QRecord().withValue("firstName", "Joe").withValue("lastName", "Smith");
QRecord firstNameJoeLastNameJonesRecord = new QRecord().withValue("firstName", "Joe").withValue("lastName", "Jones");
assertTrue(doesMatch(firstNameJoeLastNameSmithRecord, firstNameJoeLastNameJonesRecord, update));
assertFalse(doesMatch(firstNameJoeLastNameSmithRecord, firstNameJoeLastNameJonesRecord, updateFirstName));
assertFalse(doesMatch(firstNameJoeLastNameSmithRecord, firstNameJoeLastNameJonesRecord, updateFirstNameJohn));
assertFalse(doesMatch(firstNameJoeLastNameSmithRecord, firstNameJoeLastNameJonesRecord, updateFirstNameAnyBeatle));
///////////////////////////////////////////////////////////////////////
// an update that only changes last name only fires the generic rule //
///////////////////////////////////////////////////////////////////////
QRecord lastNameJonesRecord = new QRecord().withValue("lastName", "Jones");
assertTrue(doesMatch(lastNameJonesRecord, blankRecord, update));
assertFalse(doesMatch(lastNameJonesRecord, blankRecord, updateFirstName));
assertFalse(doesMatch(lastNameJonesRecord, blankRecord, updateFirstNameJohn));
assertFalse(doesMatch(lastNameJonesRecord, blankRecord, updateFirstNameAnyBeatle));
////////////////////////////////////////////////////////////////
// changing first name to the target value passes all 4 rules //
////////////////////////////////////////////////////////////////
QRecord firstNameJohnRecord = new QRecord().withValue("firstName", "John");
assertTrue(doesMatch(firstNameJohnRecord, blankRecord, update));
assertTrue(doesMatch(firstNameJohnRecord, blankRecord, updateFirstName));
assertTrue(doesMatch(firstNameJohnRecord, blankRecord, updateFirstNameJohn));
assertTrue(doesMatch(firstNameJohnRecord, blankRecord, updateFirstNameAnyBeatle));
/////////////////////////////////////////////////////////////////////////////////////////
// changing first name to the target value from the list but not the single-value rule //
/////////////////////////////////////////////////////////////////////////////////////////
QRecord firstNameRingoRecord = new QRecord().withValue("firstName", "Ringo");
assertTrue(doesMatch(firstNameRingoRecord, blankRecord, update));
assertTrue(doesMatch(firstNameRingoRecord, blankRecord, updateFirstName));
assertFalse(doesMatch(firstNameRingoRecord, blankRecord, updateFirstNameJohn));
assertTrue(doesMatch(firstNameRingoRecord, blankRecord, updateFirstNameAnyBeatle));
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// an update that doesn't change firstName (even if it has the target value) won't fire the "firstname John" rule //
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
QRecord firstNameJohnRecordLastNameSmithRecord = new QRecord().withValue("firstName", "John").withValue("lastName", "Smith");
QRecord firstNameJohnRecordLastNameJonesRecord = new QRecord().withValue("firstName", "John").withValue("lastName", "Jones");
assertTrue(doesMatch(firstNameJohnRecordLastNameSmithRecord, firstNameJohnRecordLastNameJonesRecord, update));
assertFalse(doesMatch(firstNameJohnRecordLastNameSmithRecord, firstNameJohnRecordLastNameJonesRecord, updateFirstName));
assertFalse(doesMatch(firstNameJohnRecordLastNameSmithRecord, firstNameJohnRecordLastNameJonesRecord, updateFirstNameJohn));
assertFalse(doesMatch(firstNameJohnRecordLastNameSmithRecord, firstNameJohnRecordLastNameJonesRecord, updateFirstNameAnyBeatle));
//////////////////////////////////////////////////////////////
// without old-record, always fail the value-checking rules //
//////////////////////////////////////////////////////////////
assertTrue(doesMatch(firstNameJohnRecord, null, update));
assertFalse(doesMatch(firstNameJohnRecord, null, updateFirstName));
assertFalse(doesMatch(firstNameJohnRecord, null, updateFirstNameJohn));
assertFalse(doesMatch(firstNameJohnRecord, null, updateFirstNameAnyBeatle));
}
/***************************************************************************
**
***************************************************************************/
private boolean doesMatch(QRecord record, QRecord oldRecord, WebhookEventType webhookEventType)
{
FirePostUpdateWebhookEvent firePostUpdateWebhookEvent = new FirePostUpdateWebhookEvent();
QFieldMetaData field = QContext.getQInstance().getTable(webhookEventType.getTableName()).getFields().get(webhookEventType.getFieldName());
return firePostUpdateWebhookEvent.doesRecordMatchWebhookEventType(record, Optional.ofNullable(oldRecord), webhookEventType, field);
}
}
@@ -1,880 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.actions;
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.time.Instant;
import java.time.temporal.ChronoUnit;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
import java.util.Random;
import com.kingsrook.qbits.webhooks.BaseTest;
import com.kingsrook.qbits.webhooks.WebhooksTestApplication;
import com.kingsrook.qbits.webhooks.model.Webhook;
import com.kingsrook.qbits.webhooks.model.WebhookEvent;
import com.kingsrook.qbits.webhooks.model.WebhookEventContent;
import com.kingsrook.qbits.webhooks.model.WebhookEventSendLog;
import com.kingsrook.qbits.webhooks.model.WebhookEventStatus;
import com.kingsrook.qbits.webhooks.model.WebhookHealthStatus;
import com.kingsrook.qbits.webhooks.model.WebhookSubscription;
import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QCollectingLogger;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.utils.collections.ListBuilder;
import org.apache.http.Header;
import org.apache.http.HeaderIterator;
import org.apache.http.HttpEntity;
import org.apache.http.HttpHost;
import org.apache.http.HttpRequest;
import org.apache.http.ProtocolVersion;
import org.apache.http.StatusLine;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.conn.ClientConnectionManager;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.message.BasicStatusLine;
import org.apache.http.params.HttpParams;
import org.apache.http.protocol.HttpContext;
import org.assertj.core.data.TemporalUnitWithinOffset;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
/*******************************************************************************
** Unit test for WebhookEventSender
*******************************************************************************/
public class WebhookEventSenderTest extends BaseTest
{
private TemporalUnitWithinOffset WITHIN_5_SEC = new TemporalUnitWithinOffset(5, ChronoUnit.SECONDS);
QQueryFilter sendLogFilter = new QQueryFilter().withOrderBy(new QFilterOrderBy("id", false));
/*******************************************************************************
**
*******************************************************************************/
@Test
void testSimpleSuccess() throws QException
{
/////////////////////////////////////////////////////////////////
// use random ids to avoid everything being 1 and false-passes //
/////////////////////////////////////////////////////////////////
Integer webhookId = insert(newWebhook("Test"));
Integer subscriptionId = insert(newWebhookSubscription(WebhooksTestApplication.PERSON_INSERTED_EVENT_TYPE_NAME).withWebhookId(webhookId));
Integer eventId = insert(newWebhookEvent(new WebhookSubscription(GetAction.execute(WebhookSubscription.TABLE_NAME, subscriptionId)), WebhooksTestApplication.PERSON_INSERTED_EVENT_TYPE_NAME));
WebhookEvent event = new WebhookEvent(GetAction.execute(WebhookEvent.TABLE_NAME, eventId));
Webhook webhook = new Webhook(GetAction.execute(Webhook.TABLE_NAME, webhookId));
List<WebhookEventSendLog> sendLogs = new ArrayList<>();
/////////////////
// try to send //
/////////////////
WebhookEventSender sender = new WebhookEventSenderThatSucceeds();
boolean wasSuccessful = sender.handleEvent(event, webhook, sendLogs);
///////////////////////////////
// assert failure everywhere //
///////////////////////////////
assertTrue(wasSuccessful);
List<QRecord> insertedSendLogs = QueryAction.execute(WebhookEventSendLog.TABLE_NAME, sendLogFilter);
assertEquals(1, insertedSendLogs.size());
WebhookEventSendLog sendLog = new WebhookEventSendLog(insertedSendLogs.get(0));
assertNull(sendLog.getErrorMessage());
assertEquals(1, sendLog.getAttemptNo());
assertEquals(webhookId, sendLog.getWebhookId());
assertEquals(eventId, sendLog.getWebhookEventId());
assertTrue(sendLog.getSuccessful());
assertThat(sendLog.getStartTimestamp()).isCloseTo(Instant.now(), WITHIN_5_SEC);
assertThat(sendLog.getEndTimestamp()).isCloseTo(Instant.now(), WITHIN_5_SEC);
WebhookEvent updatedEvent = new WebhookEvent(GetAction.execute(WebhookEvent.TABLE_NAME, eventId));
assertEquals(WebhookEventStatus.DELIVERED.getId(), updatedEvent.getEventStatusId());
assertNull(updatedEvent.getNextAttemptTimestamp());
sendLogs.add(sendLog);
}
/*******************************************************************************
**
*******************************************************************************/
@Test
void testFiveFailures() throws QException
{
/////////////////////////////////////////////////////////////////
// use random ids to avoid everything being 1 and false-passes //
/////////////////////////////////////////////////////////////////
Random random = new Random();
Integer webhookId = insert(newWebhook("Test"));
Integer subscriptionId = insert(newWebhookSubscription(WebhooksTestApplication.PERSON_INSERTED_EVENT_TYPE_NAME).withWebhookId(webhookId));
Integer eventId = insert(newWebhookEvent(new WebhookSubscription(GetAction.execute(WebhookSubscription.TABLE_NAME, subscriptionId)), WebhooksTestApplication.PERSON_INSERTED_EVENT_TYPE_NAME));
WebhookEvent event = new WebhookEvent(GetAction.execute(WebhookEvent.TABLE_NAME, eventId));
Webhook webhook = new Webhook(GetAction.execute(Webhook.TABLE_NAME, webhookId));
List<WebhookEventSendLog> sendLogs = new ArrayList<>();
/////////////////
// try to send //
/////////////////
WebhookEventSender sender = new WebhookEventSenderThatFails();
boolean wasSuccessful = sender.handleEvent(event, webhook, sendLogs);
///////////////////////////////
// assert failure everywhere //
///////////////////////////////
assertFalse(wasSuccessful);
List<QRecord> insertedSendLogs = QueryAction.execute(WebhookEventSendLog.TABLE_NAME, sendLogFilter);
assertEquals(1, insertedSendLogs.size());
WebhookEventSendLog sendLog = new WebhookEventSendLog(insertedSendLogs.get(0));
assertEquals("Test failure", sendLog.getErrorMessage());
assertEquals(1, sendLog.getAttemptNo());
assertEquals(webhookId, sendLog.getWebhookId());
assertEquals(eventId, sendLog.getWebhookEventId());
assertFalse(sendLog.getSuccessful());
assertThat(sendLog.getStartTimestamp()).isCloseTo(Instant.now(), WITHIN_5_SEC);
assertThat(sendLog.getEndTimestamp()).isCloseTo(Instant.now(), WITHIN_5_SEC);
WebhookEvent updatedEvent = new WebhookEvent(GetAction.execute(WebhookEvent.TABLE_NAME, eventId));
assertEquals(WebhookEventStatus.AWAITING_RETRY.getId(), updatedEvent.getEventStatusId());
assertThat(updatedEvent.getNextAttemptTimestamp()).isCloseTo(Instant.now().plusSeconds(60), WITHIN_5_SEC);
sendLogs.add(sendLog);
////////////
// try #2 //
////////////
wasSuccessful = sender.handleEvent(updatedEvent, webhook, sendLogs);
assertFalse(wasSuccessful);
insertedSendLogs = QueryAction.execute(WebhookEventSendLog.TABLE_NAME, sendLogFilter);
assertEquals(2, insertedSendLogs.size());
sendLog = new WebhookEventSendLog(insertedSendLogs.get(0));
assertEquals(2, sendLog.getAttemptNo());
updatedEvent = new WebhookEvent(GetAction.execute(WebhookEvent.TABLE_NAME, eventId));
assertEquals(WebhookEventStatus.AWAITING_RETRY.getId(), updatedEvent.getEventStatusId());
assertThat(updatedEvent.getNextAttemptTimestamp()).isCloseTo(Instant.now().plusSeconds(5 * 60), WITHIN_5_SEC);
sendLogs.add(sendLog);
////////////
// try #3 //
////////////
wasSuccessful = sender.handleEvent(updatedEvent, webhook, sendLogs);
assertFalse(wasSuccessful);
insertedSendLogs = QueryAction.execute(WebhookEventSendLog.TABLE_NAME, sendLogFilter);
assertEquals(3, insertedSendLogs.size());
sendLog = new WebhookEventSendLog(insertedSendLogs.get(0));
assertEquals(3, sendLog.getAttemptNo());
updatedEvent = new WebhookEvent(GetAction.execute(WebhookEvent.TABLE_NAME, eventId));
assertEquals(WebhookEventStatus.AWAITING_RETRY.getId(), updatedEvent.getEventStatusId());
assertThat(updatedEvent.getNextAttemptTimestamp()).isCloseTo(Instant.now().plusSeconds(15 * 60), WITHIN_5_SEC);
sendLogs.add(sendLog);
////////////
// try #4 //
////////////
wasSuccessful = sender.handleEvent(updatedEvent, webhook, sendLogs);
assertFalse(wasSuccessful);
insertedSendLogs = QueryAction.execute(WebhookEventSendLog.TABLE_NAME, sendLogFilter);
assertEquals(4, insertedSendLogs.size());
sendLog = new WebhookEventSendLog(insertedSendLogs.get(0));
assertEquals(4, sendLog.getAttemptNo());
updatedEvent = new WebhookEvent(GetAction.execute(WebhookEvent.TABLE_NAME, eventId));
assertEquals(WebhookEventStatus.AWAITING_RETRY.getId(), updatedEvent.getEventStatusId());
assertThat(updatedEvent.getNextAttemptTimestamp()).isCloseTo(Instant.now().plusSeconds(60 * 60), WITHIN_5_SEC);
sendLogs.add(sendLog);
////////////
// try #5 //
////////////
wasSuccessful = sender.handleEvent(updatedEvent, webhook, sendLogs);
assertFalse(wasSuccessful);
insertedSendLogs = QueryAction.execute(WebhookEventSendLog.TABLE_NAME, sendLogFilter);
assertEquals(5, insertedSendLogs.size());
sendLog = new WebhookEventSendLog(insertedSendLogs.get(0));
assertEquals(5, sendLog.getAttemptNo());
updatedEvent = new WebhookEvent(GetAction.execute(WebhookEvent.TABLE_NAME, eventId));
assertEquals(WebhookEventStatus.FAILED.getId(), updatedEvent.getEventStatusId());
assertNull(updatedEvent.getNextAttemptTimestamp());
sendLogs.add(sendLog);
}
/*******************************************************************************
**
*******************************************************************************/
@Test
void testFifthFailureOnProbation() throws QException
{
/////////////////////////////////////////////////////////////////
// use random ids to avoid everything being 1 and false-passes //
/////////////////////////////////////////////////////////////////
Random random = new Random();
Integer webhookId = insert(newWebhook("Test").withHealthStatusId(WebhookHealthStatus.PROBATION.getId()));
Integer subscriptionId = insert(newWebhookSubscription(WebhooksTestApplication.PERSON_INSERTED_EVENT_TYPE_NAME).withWebhookId(webhookId));
Integer eventId = insert(newWebhookEvent(new WebhookSubscription(GetAction.execute(WebhookSubscription.TABLE_NAME, subscriptionId)), WebhooksTestApplication.PERSON_INSERTED_EVENT_TYPE_NAME));
WebhookEvent event = new WebhookEvent(GetAction.execute(WebhookEvent.TABLE_NAME, eventId));
Webhook webhook = new Webhook(GetAction.execute(Webhook.TABLE_NAME, webhookId));
List<WebhookEventSendLog> sendLogs = ListBuilder.of(
new WebhookEventSendLog().withAttemptNo(1),
new WebhookEventSendLog().withAttemptNo(2),
new WebhookEventSendLog().withAttemptNo(3),
new WebhookEventSendLog().withAttemptNo(4)
);
/////////////////
// try to send //
/////////////////
WebhookEventSender sender = new WebhookEventSenderThatFails();
boolean wasSuccessful = sender.handleEvent(event, webhook, sendLogs);
////////////////////
// assert failure //
////////////////////
assertFalse(wasSuccessful);
///////////////////////////////////////////////////////////////////////
// make sure we didn't go into failure, but stayed in awaiting retry //
///////////////////////////////////////////////////////////////////////
WebhookEvent updatedEvent = new WebhookEvent(GetAction.execute(WebhookEvent.TABLE_NAME, eventId));
assertEquals(WebhookEventStatus.AWAITING_RETRY.getId(), updatedEvent.getEventStatusId());
assertThat(updatedEvent.getNextAttemptTimestamp()).isCloseTo(Instant.now().plusSeconds(4 * 60 * 60), WITHIN_5_SEC);
List<QRecord> insertedSendLogs = QueryAction.execute(WebhookEventSendLog.TABLE_NAME, sendLogFilter);
WebhookEventSendLog sendLog = new WebhookEventSendLog(insertedSendLogs.get(0));
assertEquals(5, sendLog.getAttemptNo());
sendLogs.add(sendLog);
/////////////////////////////////////////////////////
// try once more - attemptNo should go up beyond 5 //
/////////////////////////////////////////////////////
sender.handleEvent(event, webhook, sendLogs);
updatedEvent = new WebhookEvent(GetAction.execute(WebhookEvent.TABLE_NAME, eventId));
assertEquals(WebhookEventStatus.AWAITING_RETRY.getId(), updatedEvent.getEventStatusId());
assertThat(updatedEvent.getNextAttemptTimestamp()).isCloseTo(Instant.now().plusSeconds(4 * 60 * 60), WITHIN_5_SEC);
insertedSendLogs = QueryAction.execute(WebhookEventSendLog.TABLE_NAME, sendLogFilter);
sendLog = new WebhookEventSendLog(insertedSendLogs.get(0));
assertEquals(6, sendLog.getAttemptNo());
sendLogs.add(sendLog);
}
/*******************************************************************************
**
*******************************************************************************/
@Test
void testHttpStatusCodes() throws QException
{
/////////////////////////////////////////////////////////////////
// use random ids to avoid everything being 1 and false-passes //
/////////////////////////////////////////////////////////////////
Random random = new Random();
Integer webhookId = insert(newWebhook("Test"));
Integer subscriptionId = insert(newWebhookSubscription(WebhooksTestApplication.PERSON_INSERTED_EVENT_TYPE_NAME).withWebhookId(webhookId));
Integer eventId = insert(newWebhookEvent(new WebhookSubscription(GetAction.execute(WebhookSubscription.TABLE_NAME, subscriptionId)), WebhooksTestApplication.PERSON_INSERTED_EVENT_TYPE_NAME));
WebhookEvent event = new WebhookEvent(GetAction.execute(WebhookEvent.TABLE_NAME, eventId));
event.setContent(List.of(new WebhookEventContent().withWebhookEventId(eventId).withPostBody("{}")));
Webhook webhook = new Webhook(GetAction.execute(Webhook.TABLE_NAME, webhookId));
List<WebhookEventSendLog> sendLogs = new ArrayList<>();
////////////////
// 200 - okay //
////////////////
{
WebhookEventSender sender = new WebhookEventSenderThatMocksHttp(200, "OK");
assertTrue(sender.handleEvent(event, webhook, sendLogs));
List<QRecord> insertedSendLogs = QueryAction.execute(WebhookEventSendLog.TABLE_NAME, sendLogFilter);
WebhookEventSendLog sendLog = new WebhookEventSendLog(insertedSendLogs.get(0));
assertNull(sendLog.getErrorMessage());
assertTrue(sendLog.getSuccessful());
assertEquals(200, sendLog.getHttpStatusCode());
}
////////////////
// 201 - okay //
////////////////
{
WebhookEventSender sender = new WebhookEventSenderThatMocksHttp(201, "OK");
assertTrue(sender.handleEvent(event, webhook, sendLogs));
List<QRecord> insertedSendLogs = QueryAction.execute(WebhookEventSendLog.TABLE_NAME, sendLogFilter);
WebhookEventSendLog sendLog = new WebhookEventSendLog(insertedSendLogs.get(0));
assertNull(sendLog.getErrorMessage());
assertTrue(sendLog.getSuccessful());
assertEquals(201, sendLog.getHttpStatusCode());
}
////////////////
// 301 - fail //
////////////////
{
WebhookEventSender sender = new WebhookEventSenderThatMocksHttp(301, "Go away");
assertFalse(sender.handleEvent(event, webhook, sendLogs));
List<QRecord> insertedSendLogs = QueryAction.execute(WebhookEventSendLog.TABLE_NAME, sendLogFilter);
WebhookEventSendLog sendLog = new WebhookEventSendLog(insertedSendLogs.get(0));
assertEquals("Go away", sendLog.getErrorMessage());
assertFalse(sendLog.getSuccessful());
assertEquals(301, sendLog.getHttpStatusCode());
}
/////////////////////////////
// 500 with no body - fail //
/////////////////////////////
{
WebhookEventSender sender = new WebhookEventSenderThatMocksHttp(500, "");
assertFalse(sender.handleEvent(event, webhook, sendLogs));
List<QRecord> insertedSendLogs = QueryAction.execute(WebhookEventSendLog.TABLE_NAME, sendLogFilter);
WebhookEventSendLog sendLog = new WebhookEventSendLog(insertedSendLogs.get(0));
assertEquals("No response body returned", sendLog.getErrorMessage());
assertFalse(sendLog.getSuccessful());
assertEquals(500, sendLog.getHttpStatusCode());
}
////////////////
// 199 - fail //
////////////////
{
WebhookEventSender sender = new WebhookEventSenderThatMocksHttp(199, "what even?");
assertFalse(sender.handleEvent(event, webhook, sendLogs));
List<QRecord> insertedSendLogs = QueryAction.execute(WebhookEventSendLog.TABLE_NAME, sendLogFilter);
WebhookEventSendLog sendLog = new WebhookEventSendLog(insertedSendLogs.get(0));
assertEquals("what even?", sendLog.getErrorMessage());
assertFalse(sendLog.getSuccessful());
assertEquals(199, sendLog.getHttpStatusCode());
}
////////////////////////////////////////
// IO Exception (e.g., can't connect) //
////////////////////////////////////////
{
WebhookEventSender sender = new WebhookEventSenderThatMocksHttp(new IOException("Connection Refused"));
assertFalse(sender.handleEvent(event, webhook, sendLogs));
List<QRecord> insertedSendLogs = QueryAction.execute(WebhookEventSendLog.TABLE_NAME, sendLogFilter);
WebhookEventSendLog sendLog = new WebhookEventSendLog(insertedSendLogs.get(0));
assertEquals("Connection Refused", sendLog.getErrorMessage());
assertFalse(sendLog.getSuccessful());
assertNull(sendLog.getHttpStatusCode());
}
}
/*******************************************************************************
**
*******************************************************************************/
@Test
void testRateLimit() throws QException
{
/////////////////////////////////////////////////////////////////
// use random ids to avoid everything being 1 and false-passes //
/////////////////////////////////////////////////////////////////
Random random = new Random();
Integer webhookId = insert(newWebhook("Test"));
Integer subscriptionId = insert(newWebhookSubscription(WebhooksTestApplication.PERSON_INSERTED_EVENT_TYPE_NAME).withWebhookId(webhookId));
Integer eventId = insert(newWebhookEvent(new WebhookSubscription(GetAction.execute(WebhookSubscription.TABLE_NAME, subscriptionId)), WebhooksTestApplication.PERSON_INSERTED_EVENT_TYPE_NAME));
WebhookEvent event = new WebhookEvent(GetAction.execute(WebhookEvent.TABLE_NAME, eventId));
event.setContent(List.of(new WebhookEventContent().withWebhookEventId(eventId).withPostBody("{}")));
Webhook webhook = new Webhook(GetAction.execute(Webhook.TABLE_NAME, webhookId));
List<WebhookEventSendLog> sendLogs = new ArrayList<>();
{
///////////////////////////////////////////////////////////
// use http mocker that will always fail with rate limit //
///////////////////////////////////////////////////////////
long start = System.currentTimeMillis();
WebhookEventSender sender = new WebhookEventSenderThatMocksHttp(429, "Too many requests");
assertFalse(sender.handleEvent(event, webhook, sendLogs));
List<QRecord> insertedSendLogs = QueryAction.execute(WebhookEventSendLog.TABLE_NAME, sendLogFilter);
WebhookEventSendLog sendLog = new WebhookEventSendLog(insertedSendLogs.get(0));
long end = System.currentTimeMillis();
assertEquals("Giving up after too many rate-limit errors (3). Latest response: Too many requests", sendLog.getErrorMessage());
assertFalse(sendLog.getSuccessful());
assertEquals(429, sendLog.getHttpStatusCode());
assertThat(end - start).isGreaterThan(WebhookEventSenderThatMocksHttp.BACKOFF_MILLIS * 3L);
}
{
///////////////////////////////////////////////////////////////
// use a sender that will rate-limit twice, but then succeed //
///////////////////////////////////////////////////////////////
QCollectingLogger collectingLogger = QLogger.activateCollectingLoggerForClass(WebhookEventSender.class);
long start = System.currentTimeMillis();
WebhookEventSender sender = new RateLimitPassEventuallySender("Too fast");
assertTrue(sender.handleEvent(event, webhook, sendLogs));
List<QRecord> insertedSendLogs = QueryAction.execute(WebhookEventSendLog.TABLE_NAME, sendLogFilter);
WebhookEventSendLog sendLog = new WebhookEventSendLog(insertedSendLogs.get(0));
QLogger.deactivateCollectingLoggerForClass(WebhookEventSender.class);
long end = System.currentTimeMillis();
assertNull(sendLog.getErrorMessage());
assertTrue(sendLog.getSuccessful());
assertEquals(200, sendLog.getHttpStatusCode());
assertThat(end - start).isGreaterThan(WebhookEventSenderThatMocksHttp.BACKOFF_MILLIS * 3L);
assertThat(collectingLogger.getCollectedMessages())
.anyMatch(m -> m.getMessage().contains("Caught rate limit. Will sleep and re-try"));
}
}
/***************************************************************************
**
***************************************************************************/
public static class WebhookEventSenderThatFails extends WebhookEventSender
{
/***************************************************************************
**
***************************************************************************/
@Override
protected void doPost(WebhookEvent webhookEvent, Webhook webhook, WebhookEventSendLog sendLog) throws WebhookPostException
{
throw new WebhookPostException("Test failure");
}
}
/***************************************************************************
**
***************************************************************************/
public static class WebhookEventSenderThatSucceeds extends WebhookEventSender
{
/***************************************************************************
**
***************************************************************************/
@Override
protected void doPost(WebhookEvent webhookEvent, Webhook webhook, WebhookEventSendLog sendLog) throws WebhookPostException
{
}
}
/***************************************************************************
**
***************************************************************************/
public static class WebhookEventSenderThatMocksHttp extends WebhookEventSender
{
protected Integer statusCode;
protected String responseBody;
private IOException e;
public static final Integer BACKOFF_MILLIS = 3;
/*******************************************************************************
** Constructor
**
*******************************************************************************/
public WebhookEventSenderThatMocksHttp(int statusCode, String responseBody)
{
this.statusCode = statusCode;
this.responseBody = responseBody;
}
/*******************************************************************************
** Constructor
**
*******************************************************************************/
public WebhookEventSenderThatMocksHttp(IOException e)
{
this.e = e;
}
/***************************************************************************
**
***************************************************************************/
@Override
protected int getInitialRateLimitBackoffMillis()
{
return (BACKOFF_MILLIS);
}
/***************************************************************************
**
***************************************************************************/
@Override
protected CloseableHttpClient buildHttpClient()
{
return new CloseableHttpClient()
{
@Override
protected CloseableHttpResponse doExecute(HttpHost httpHost, HttpRequest httpRequest, HttpContext httpContext) throws IOException, ClientProtocolException
{
return null;
}
@Override
public void close() throws IOException
{
}
@Override
public HttpParams getParams()
{
return null;
}
@Override
public ClientConnectionManager getConnectionManager()
{
return null;
}
};
}
/***************************************************************************
**
***************************************************************************/
@Override
protected CloseableHttpResponse executeHttpRequest(CloseableHttpClient httpClient, HttpPost request) throws IOException
{
if(e != null)
{
throw (e);
}
return new CloseableHttpResponse()
{
@Override
public void close() throws IOException
{
}
@Override
public StatusLine getStatusLine()
{
return new BasicStatusLine(new ProtocolVersion("HTTP", 1, 1), statusCode, "Yeah.");
}
@Override
public void setStatusLine(StatusLine statusLine)
{
}
@Override
public void setStatusLine(ProtocolVersion protocolVersion, int i)
{
}
@Override
public void setStatusLine(ProtocolVersion protocolVersion, int i, String s)
{
}
@Override
public void setStatusCode(int i) throws IllegalStateException
{
}
@Override
public void setReasonPhrase(String s) throws IllegalStateException
{
}
@Override
public HttpEntity getEntity()
{
try
{
return new StringEntity(responseBody);
}
catch(UnsupportedEncodingException e)
{
throw new RuntimeException(e);
}
}
@Override
public void setEntity(HttpEntity httpEntity)
{
}
@Override
public Locale getLocale()
{
return null;
}
@Override
public void setLocale(Locale locale)
{
}
@Override
public ProtocolVersion getProtocolVersion()
{
return null;
}
@Override
public boolean containsHeader(String s)
{
return false;
}
@Override
public Header[] getHeaders(String s)
{
return new Header[0];
}
@Override
public Header getFirstHeader(String s)
{
return null;
}
@Override
public Header getLastHeader(String s)
{
return null;
}
@Override
public Header[] getAllHeaders()
{
return new Header[0];
}
@Override
public void addHeader(Header header)
{
}
@Override
public void addHeader(String s, String s1)
{
}
@Override
public void setHeader(Header header)
{
}
@Override
public void setHeader(String s, String s1)
{
}
@Override
public void setHeaders(Header[] headers)
{
}
@Override
public void removeHeader(Header header)
{
}
@Override
public void removeHeaders(String s)
{
}
@Override
public HeaderIterator headerIterator()
{
return null;
}
@Override
public HeaderIterator headerIterator(String s)
{
return null;
}
@Override
public HttpParams getParams()
{
return null;
}
@Override
public void setParams(HttpParams httpParams)
{
}
};
}
}
/*******************************************************************************
**
*******************************************************************************/
public static class RateLimitPassEventuallySender extends WebhookEventSenderThatMocksHttp
{
int count = 0;
/*******************************************************************************
** Constructor
**
*******************************************************************************/
public RateLimitPassEventuallySender(String message)
{
super(429, message);
}
/***************************************************************************
**
***************************************************************************/
@Override
protected CloseableHttpResponse executeHttpRequest(CloseableHttpClient httpClient, HttpPost request) throws IOException
{
count++;
if(count >= 3)
{
statusCode = 200;
responseBody = "OK";
}
return super.executeHttpRequest(httpClient, request);
}
}
}
@@ -1,65 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.actions;
import com.kingsrook.qbits.webhooks.BaseTest;
import com.kingsrook.qbits.webhooks.WebhooksTestApplication;
import com.kingsrook.qbits.webhooks.model.WebhookEventContent;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import org.json.JSONObject;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;
/*******************************************************************************
** Unit test for WebhookEventTypeCustomizerInterface
*******************************************************************************/
class WebhookEventTypeCustomizerInterfaceTest extends BaseTest
{
/*******************************************************************************
** the post body sent to receivers must keep fields whose value is null, so a
** receiver can tell "field is null" from "field not in this api version".
*******************************************************************************/
@Test
void testBuildEventContentKeepsNullValues() throws QException
{
String eventTypeName = "personAdHoc";
registerAdHocEventType(eventTypeName);
QRecord person = new QRecord()
.withTableName(WebhooksTestApplication.TABLE_NAME_PERSON)
.withValue("id", 1)
.withValue("firstName", "Jean Luc");
WebhookEventContent content = new DefaultWebhookEventTypeCustomizer()
.buildEventContent(person, eventTypeName, WebhooksTestApplication.API_NAME, WebhooksTestApplication.API_V1);
JSONObject record = new JSONObject(content.getPostBody()).getJSONObject("record");
assertEquals("Jean Luc", record.getString("firstName"));
assertTrue(record.has("lastName"), "null-valued field should be present in the post body");
assertTrue(record.isNull("lastName"));
}
}
@@ -1,133 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.actions;
import java.util.Stack;
import com.kingsrook.qbits.webhooks.BaseTest;
import com.kingsrook.qbits.webhooks.model.Webhook;
import com.kingsrook.qbits.webhooks.model.WebhookEventSendLog;
import com.kingsrook.qbits.webhooks.model.WebhookHealthStatus;
import com.kingsrook.qbits.webhooks.processes.SendWebhookEventLoadStep;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
/*******************************************************************************
** Unit test for WebhookHealthManager
*******************************************************************************/
class WebhookHealthManagerTest extends BaseTest
{
/*******************************************************************************
**
*******************************************************************************/
@Test
void test() throws QException
{
Webhook webhook = newWebhook("test");
webhook.setId(insert(webhook));
assertHealthStatus(WebhookHealthStatus.HEALTHY, webhook, makeStack(1));
assertHealthStatus(WebhookHealthStatus.HEALTHY, webhook, makeStack(1, 1));
assertHealthStatus(WebhookHealthStatus.HEALTHY, webhook, makeStack(0, 1, 1));
assertHealthStatus(WebhookHealthStatus.HEALTHY, webhook, makeStack(0, 0, 0, 0, 0, 0, 0, 0, 0, 1));
Stack<WebhookEventSendLog> tenFails = makeStack(0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
assertHealthStatus(WebhookHealthStatus.UNHEALTHY, webhook, tenFails);
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// make sure that pushing another event onto a stack that was full of fails, that it then lets a success go out of probation //
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
tenFails.push(new WebhookEventSendLog().withSuccessful(true));
webhook.setHealthStatusId(WebhookHealthStatus.PROBATION.getId());
assertHealthStatus(WebhookHealthStatus.HEALTHY, webhook, tenFails);
webhook.setHealthStatusId(WebhookHealthStatus.PROBATION.getId());
assertHealthStatus(WebhookHealthStatus.HEALTHY, webhook, makeStack(0, 1, 1, 1));
///////////////////////////////////////////////////////////////////
// start webhook HEALTHy, and stack with 5 success, then 8 fails //
///////////////////////////////////////////////////////////////////
webhook.setHealthStatusId(WebhookHealthStatus.HEALTHY.getId());
Stack<WebhookEventSendLog> stack = makeStack(1);
for(int i = 0; i < 4; i++)
{
stack.push(new WebhookEventSendLog().withSuccessful(true));
}
for(int i = 0; i < 8; i++)
{
stack.push(new WebhookEventSendLog().withSuccessful(false));
}
//////////////////
// stay healthy //
//////////////////
assertHealthStatus(WebhookHealthStatus.HEALTHY, webhook, stack);
/////////////////////////////////////////
// another fail (now 9), stays healthy //
/////////////////////////////////////////
stack.push(new WebhookEventSendLog().withSuccessful(false));
assertHealthStatus(WebhookHealthStatus.HEALTHY, webhook, stack);
///////////////////////////////////////////
// another fail (now 10), goes unhealthy //
///////////////////////////////////////////
stack.push(new WebhookEventSendLog().withSuccessful(false));
assertHealthStatus(WebhookHealthStatus.UNHEALTHY, webhook, stack);
/////////////////////////////////////////////////////////////////////
// now if we got to probation, then a success, and we'd go healthy //
/////////////////////////////////////////////////////////////////////
webhook.setHealthStatusId(WebhookHealthStatus.PROBATION.getId());
stack.push(new WebhookEventSendLog().withSuccessful(true));
assertHealthStatus(WebhookHealthStatus.HEALTHY, webhook, stack);
}
/***************************************************************************
**
***************************************************************************/
private void assertHealthStatus(WebhookHealthStatus expected, Webhook webhook, Stack<WebhookEventSendLog> recentSendLogs) throws QException
{
Webhook updatedWebhook = new WebhookHealthManager().updateWebhookHealth(recentSendLogs, webhook);
assertEquals(expected.getId(), updatedWebhook.getHealthStatusId());
}
/***************************************************************************
**
***************************************************************************/
private static Stack<WebhookEventSendLog> makeStack(int... values)
{
Stack<WebhookEventSendLog> rs = new SendWebhookEventLoadStep().newSendLogStack(10);
for(int value : values)
{
rs.add(new WebhookEventSendLog().withId(rs.size() + 1).withSuccessful(value == 1));
}
return (rs);
}
}
@@ -1,98 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.actions;
import java.util.List;
import com.kingsrook.qbits.webhooks.BaseTest;
import com.kingsrook.qbits.webhooks.model.WebhookEventCategory;
import com.kingsrook.qbits.webhooks.model.WebhookEventCategory.Kind;
import com.kingsrook.qbits.webhooks.registry.WebhookEventType;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** Unit test for IdentifyWebhookSubscriptions
*******************************************************************************/
class WebhookSubscriptionsHelperTest extends BaseTest
{
/*******************************************************************************
**
*******************************************************************************/
@Test
void test() throws QException
{
String personTableName = "person";
String placeTableName = "place";
String insertedPersonEventName = "inserted-person";
String insertedPlaceEventName = "inserted-place";
String insertedPersonWithFirstNameEventName = "inserted-person-firstName";
//////////////////////////////////////////////
// nothing exists yet, so all null or empty //
//////////////////////////////////////////////
assertThat(getEventTypes(Kind.INSERT, "noSuchTable")).isNullOrEmpty();
assertThat(getEventTypes(Kind.INSERT, "noSuchTable")).isNullOrEmpty();
assertThat(getEventTypes(Kind.UPDATE, personTableName)).isNullOrEmpty();
assertThat(getEventTypes(Kind.AD_HOC, personTableName)).isNullOrEmpty();
///////////////////////////////////////////////////////////////////////////////////////
// register first event type, but no subscriptions to it, so still all empty results //
///////////////////////////////////////////////////////////////////////////////////////
registerEventType(insertedPersonEventName, WebhookEventCategory.INSERT, personTableName);
assertThat(getEventTypes(Kind.INSERT, personTableName)).isNullOrEmpty();
assertThat(getEventTypes(Kind.UPDATE, personTableName)).isNullOrEmpty();
assertThat(getEventTypes(Kind.AD_HOC, personTableName)).isNullOrEmpty();
//////////////////////////////////////////////////////////////
// add a second insert event with a sub on the person table //
//////////////////////////////////////////////////////////////
registerEventType(insertedPersonWithFirstNameEventName, WebhookEventCategory.INSERT, personTableName, "firstName");
insert(newWebhookSubscription(insertedPersonWithFirstNameEventName));
WebhookSubscriptionsHelper.clearMemoizations();
assertThat(getEventTypes(Kind.INSERT, personTableName)).hasSize(1);
assertThat(getEventTypes(Kind.UPDATE, personTableName)).isNullOrEmpty();
assertThat(getEventTypes(Kind.AD_HOC, personTableName)).isNullOrEmpty();
////////////////////////////////////////
// add insert event for another table //
////////////////////////////////////////
registerEventType(insertedPlaceEventName, WebhookEventCategory.INSERT, placeTableName);
insert(newWebhookSubscription(insertedPlaceEventName));
WebhookSubscriptionsHelper.clearMemoizations();
assertThat(getEventTypes(Kind.INSERT, personTableName)).hasSize(1);
assertThat(getEventTypes(Kind.INSERT, placeTableName)).hasSize(1);
}
/***************************************************************************
**
***************************************************************************/
private List<WebhookEventType> getEventTypes(Kind kind, String tableName)
{
return WebhookSubscriptionsHelper.getWebhookEventTypesToConsiderFiringEventsFor(kind, tableName);
}
}
@@ -21,7 +21,12 @@
package com.kingsrook.qbits.webhooks.app;
import java.net.URLDecoder;
import java.nio.charset.StandardCharsets;
import java.util.HashMap;
import java.util.Map;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import com.kingsrook.qbits.webhooks.WebhooksTestSupport;
import com.kingsrook.qbits.webhooks.WebhooksTestSupport.Receiver;
import com.kingsrook.qbits.webhooks.delivery.DeliveryEngine;
@@ -29,11 +34,14 @@ import com.kingsrook.qbits.webhooks.metadata.WebhooksTables;
import com.kingsrook.qbits.webhooks.permissions.WebhooksPermissions;
import com.kingsrook.qbits.webhooks.support.SystemWrites;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.RawHTML;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.session.QSession;
import com.kingsrook.qqq.backend.core.utils.JsonUtils;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
@@ -90,7 +98,7 @@ class WebhooksHomeTest
void emptyHome() throws Exception
{
String html = home();
assertThat(html).contains("No endpoints yet").contains("/admin/webhooks/webhooks_endpoint").contains("No delivery attempts in the last 24 hours")
assertThat(html).doesNotContain("Endpoints needing attention").contains("Setup").contains("Needs setup").contains("Add an endpoint: the URL that receives events").contains("/admin/webhooks/webhooks_endpoint").contains("No delivery attempts in the last 24 hours")
.contains("No dead letters").contains("None yet");
}
@@ -120,12 +128,10 @@ class WebhooksHomeTest
assertThat(data.getDelivered24h()).isEqualTo(3);
assertThat(data.getFailedAttempts24h()).isEqualTo(3);
assertThat(data.getDeadLetters()).isEqualTo(1);
assertThat(data.getAttention()).hasSize(1);
assertThat(data.getAttentionWhy().get(0)).contains("Unhealthy");
assertThat(data.getP50Millis()).isNotNull();
String html = home();
assertThat(html).contains("Delivered (24 h)").contains("attempts in the last 24 hours, by hour").contains("bad &lt;b&gt;").doesNotContain("bad <b>")
assertThat(html).contains("Delivered (24 h)").contains("attempts in the last 24 hours, by hour").doesNotContain("Endpoints needing attention").doesNotContain("bad <b>")
.contains("/admin/webhooks/webhooks_delivery/" + deadOne.getValue("id")).contains("Replay");
////////////////////////////////////////////////////////////////
// readability rules: no fixed widths that force a scroll bar //
@@ -135,6 +141,31 @@ class WebhooksHomeTest
/*******************************************************************************
** The KPI drill links open the delivery or attempt table with the matching
** filter, as URL-encoded JSON.
*******************************************************************************/
@Test
void kpiDrillLinksAreFilteredJson() throws Exception
{
String html = home();
Matcher matcher = Pattern.compile("href=\"(/admin/webhooks/([a-z_]+)\\?filter=([^\"]+))\"").matcher(html);
Map<String, QQueryFilter> byTable = new HashMap<>();
int links = 0;
while(matcher.find())
{
links++;
QQueryFilter filter = JsonUtils.toObject(URLDecoder.decode(matcher.group(3), StandardCharsets.UTF_8), QQueryFilter.class);
byTable.put(matcher.group(2) + ":" + filter.getCriteria().getFirst().getFieldName(), filter);
}
assertThat(links).isEqualTo(4);
assertThat(byTable).containsKeys(WebhooksTables.DELIVERY + ":deliveredDate", WebhooksTables.ATTEMPT + ":startDate", WebhooksTables.DELIVERY + ":status");
assertThat(byTable.get(WebhooksTables.DELIVERY + ":status").getCriteria()).hasSize(1);
assertThat(byTable.get(WebhooksTables.ATTEMPT + ":startDate").getCriteria().get(1).getOperator()).isEqualTo(QCriteriaOperator.NOT_IN);
}
/*******************************************************************************
** The secret value: admins only, behind a toggle; references not shown.
*******************************************************************************/
@@ -0,0 +1,132 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.app;
import java.time.Duration;
import java.util.List;
import com.kingsrook.qbits.webhooks.WebhooksTestSupport;
import com.kingsrook.qbits.webhooks.app.WebhooksSetupStatus.Step;
import com.kingsrook.qbits.webhooks.metadata.WebhooksTables;
import com.kingsrook.qbits.webhooks.secrets.EndpointSecrets;
import com.kingsrook.qbits.webhooks.support.SystemWrites;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** The Setup card: states follow what is configured, and it shows on the home.
*******************************************************************************/
class WebhooksSetupStatusTest
{
/*******************************************************************************
** A fresh install: everything the administrator chooses needs setup, each
** with a next step.
*******************************************************************************/
@Test
void freshInstallNeedsSetup()
{
List<Step> steps = WebhooksSetupStatus.steps("/admin/webhooks", "WEBHOOKS_SECRET_KEY", 0, false, 0, 0, 0);
assertThat(steps).extracting(Step::label).containsExactly("Event types", "Encryption key", "Endpoints", "Signing secrets", "Subscriptions");
assertThat(steps).extracting(Step::state).containsExactly("Needs setup", "Optional", "Needs setup", "Needs setup", "Needs setup");
assertThat(steps).allSatisfy(step -> assertThat(step.next()).isNotBlank());
assertThat(steps.get(1).next()).contains("WEBHOOKS_SECRET_KEY");
}
/*******************************************************************************
** A secretless endpoint is called out; a complete setup is all set up.
*******************************************************************************/
@Test
void progressIsReflected()
{
List<Step> partial = WebhooksSetupStatus.steps("/p", "K", 3, true, 2, 1, 0);
assertThat(partial).extracting(Step::state).containsExactly("Set up", "Set up", "Set up", "Needs setup", "Needs setup");
assertThat(partial.get(3).next()).contains("1 active endpoint(s) have no usable signing secret");
List<Step> done = WebhooksSetupStatus.steps("/p", "K", 3, true, 2, 0, 4);
assertThat(done).extracting(Step::state).containsOnly("Set up");
}
/*******************************************************************************
** The live card: an empty install needs endpoints, and an endpoint moves it
** forward.
*******************************************************************************/
@Test
void liveStatusFollowsTheTables() throws Exception
{
WebhooksTestSupport.setUp();
List<Step> empty = WebhooksSetupStatus.load();
assertThat(empty).filteredOn(step -> step.label().equals("Endpoints")).extracting(Step::state).containsExactly("Needs setup");
assertThat(empty).filteredOn(step -> step.label().equals("Event types")).extracting(Step::state).containsExactly("Set up");
WebhooksTestSupport.endpoint("crm", "https://example.com/hook");
assertThat(WebhooksSetupStatus.load()).filteredOn(step -> step.label().equals("Endpoints")).extracting(Step::state).containsExactly("Set up");
}
/*******************************************************************************
** The secrets step counts blocked endpoints exactly as DeliveryEngine does:
** active, signing required, no usable secret. An allowUnsigned endpoint, a
** paused endpoint and one whose only secret has expired are judged by the
** runtime's rules.
*******************************************************************************/
@Test
void secretsStepMatchesTheRuntime() throws Exception
{
WebhooksTestSupport.setUp();
Integer unsigned = WebhooksTestSupport.insert(WebhooksTables.ENDPOINT, new QRecord().withValue("name", "u").withValue("url", "https://example.com/u").withValue("allowUnsigned", true)).getValueInteger("id");
assertThat(secretsState()).as("allowUnsigned with no secret is not blocked").isEqualTo("Set up");
Integer paused = WebhooksTestSupport.insert(WebhooksTables.ENDPOINT, new QRecord().withValue("name", "p").withValue("url", "https://example.com/p").withValue("activeStatus", "PAUSED")).getValueInteger("id");
assertThat(paused).isNotNull();
assertThat(secretsState()).as("a paused endpoint with no secret is not blocked").isEqualTo("Set up");
Integer rotating = WebhooksTestSupport.endpoint("r", "https://example.com/r");
assertThat(secretsState()).as("a secret signs").isEqualTo("Set up");
WebhooksTestSupport.CLOCK.advance(Duration.ofHours(25));
EndpointSecrets.rotate(rotating, "again");
WebhooksTestSupport.CLOCK.advance(Duration.ofHours(25));
assertThat(unsigned).isNotNull();
assertThat(EndpointSecrets.signingKeys(rotating)).hasSize(1);
Integer onlyExpired = WebhooksTestSupport.insert(WebhooksTables.ENDPOINT, new QRecord().withValue("name", "e").withValue("url", "https://example.com/e")).getValueInteger("id");
SystemWrites.insertOne(WebhooksTables.SECRET, new QRecord().withValue("endpointId", onlyExpired).withValue("status", "PREVIOUS").withValue("source", "REFERENCE")
.withValue("secretRef", "WEBHOOKS_TEST_NOPE").withValue("expiresDate", WebhooksTestSupport.CLOCK.instant().minus(Duration.ofHours(1))));
assertThat(EndpointSecrets.signingKeys(onlyExpired)).isEmpty();
assertThat(secretsState()).as("an endpoint whose only secret expired is blocked").isEqualTo("Needs setup");
}
/*******************************************************************************
** The secrets step state.
*******************************************************************************/
private static String secretsState() throws Exception
{
return (WebhooksSetupStatus.load().stream().filter(step -> step.label().equals("Signing secrets")).findFirst().orElseThrow().state());
}
}
@@ -27,15 +27,15 @@ import java.time.Duration;
import java.util.List;
import com.kingsrook.qbits.webhooks.H2Support;
import com.kingsrook.qbits.webhooks.WebhooksRuntime;
import com.kingsrook.qbits.webhooks.WebhooksTestSupport;
import com.kingsrook.qbits.webhooks.WebhooksTestSupport.Receiver;
import com.kingsrook.qbits.webhooks.WebhooksTestSupport;
import com.kingsrook.qbits.webhooks.metadata.WebhooksTables;
import com.kingsrook.qbits.webhooks.secrets.EndpointSecrets;
import com.kingsrook.qbits.webhooks.shared.signing.WebhookSecret;
import com.kingsrook.qbits.webhooks.shared.signing.WebhookVerifier;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerifier;
import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
@@ -23,16 +23,16 @@ package com.kingsrook.qbits.webhooks.delivery;
import java.time.Duration;
import java.util.List;
import com.kingsrook.qbits.webhooks.WebhooksTestSupport;
import com.kingsrook.qbits.webhooks.WebhooksTestSupport.Receiver;
import com.kingsrook.qbits.webhooks.WebhooksTestSupport;
import com.kingsrook.qbits.webhooks.metadata.WebhooksTables;
import com.kingsrook.qbits.webhooks.processes.WebhooksProcesses;
import com.kingsrook.qbits.webhooks.secrets.EndpointSecrets;
import com.kingsrook.qbits.webhooks.shared.signing.WebhookSecret;
import com.kingsrook.qbits.webhooks.shared.signing.WebhookVerificationException;
import com.kingsrook.qbits.webhooks.shared.signing.WebhookVerifier;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.utils.JsonUtils;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerificationException;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerifier;
import org.json.JSONObject;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
@@ -175,6 +175,25 @@ class SignedDeliveryTest
/*******************************************************************************
** A previous secret stops signing when its Signs Until date passes, even
** before the maintenance job retires it (expiresDate is enforced when
** signing, not only when retiring).
*******************************************************************************/
@Test
void expiredSecretStopsSigningBeforeItIsRetired() throws Exception
{
Integer endpointId = WebhooksTestSupport.endpoint("crm", receiver.url("/hook"));
EndpointSecrets.rotate(endpointId, "expiry test");
assertThat(EndpointSecrets.signingKeys(endpointId)).hasSize(2);
WebhooksTestSupport.CLOCK.advance(Duration.ofHours(25));
assertThat(WebhooksTestSupport.all(WebhooksTables.SECRET)).extracting(r -> r.getValueString("status")).contains("PREVIOUS");
assertThat(EndpointSecrets.signingKeys(endpointId)).hasSize(1);
}
/*******************************************************************************
** A retry keeps the same webhook-id (the idempotency key) and payload.
*******************************************************************************/
@@ -0,0 +1,366 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.health;
import java.time.Instant;
import java.util.List;
import java.util.Map;
import java.util.UUID;
import com.kingsrook.qbits.webhooks.WebhooksRuntime;
import com.kingsrook.qbits.webhooks.WebhooksTestSupport;
import com.kingsrook.qbits.webhooks.delivery.EndpointHealth;
import com.kingsrook.qbits.webhooks.metadata.WebhooksTables;
import com.kingsrook.qbits.webhooks.support.SystemWrites;
import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizers;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.health.HealthCategory;
import com.kingsrook.qqq.backend.core.health.HealthCheckInput;
import com.kingsrook.qqq.backend.core.health.HealthCheckKind;
import com.kingsrook.qqq.backend.core.health.HealthCheckMetaData;
import com.kingsrook.qqq.backend.core.health.HealthCheckOutcome;
import com.kingsrook.qqq.backend.core.health.HealthChecksMetaData;
import com.kingsrook.qqq.backend.core.health.HealthStatus;
import com.kingsrook.qqq.backend.core.model.actions.tables.QueryOrGetInputInterface;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.model.session.QSession;
import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryRecordStore;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** Passive health checks use memory records and never contact a receiver.
*******************************************************************************/
class WebhooksHealthChecksTest
{
private static final String SENTINEL = "S3NT1NEL-secret";
private QInstance instance;
/*******************************************************************************
** Set up the producer and a sentinel credential reference.
*******************************************************************************/
@BeforeEach
void setUp() throws Exception
{
instance = WebhooksTestSupport.setUp();
System.setProperty("WEBHOOKS_HEALTH_SECRET", SENTINEL + "-long-enough");
}
/*******************************************************************************
** Remove the test credential.
*******************************************************************************/
@AfterEach
void tearDown()
{
System.clearProperty("WEBHOOKS_HEALTH_SECRET");
}
/*******************************************************************************
** Supplier refresh finds new endpoints and excludes manual disable/pause.
*******************************************************************************/
@Test
void registrationAndDynamicDiscovery() throws Exception
{
HealthChecksMetaData registry = HealthChecksMetaData.ofOrWithNew(instance);
assertThat(registry.getChecks()).extracting(HealthCheckMetaData::getName).containsExactly("webhooks.dispatcher");
assertThat(registry.getSuppliers()).hasSize(1);
assertThat(registry.getSuppliers().getFirst().getNamePrefix()).isEqualTo("webhooks.endpoint");
assertThat(registry.getSuppliers().getFirst().getComponent()).isEqualTo("webhooks");
assertThat(registry.resolveAll(instance)).hasSize(1);
Integer active = endpoint();
Integer disabled = endpoint();
update(disabled, "activeStatus", "DISABLED");
Integer paused = endpoint();
update(paused, "activeStatus", "PAUSED");
Integer automatic = endpoint();
update(automatic, "activeStatus", "DISABLED");
update(automatic, "autoDisabledDate", WebhooksTestSupport.CLOCK.instant());
QContext.init(instance, new QSession());
assertThat(registry.resolveAll(instance)).extracting(HealthCheckMetaData::getName)
.containsExactlyInAnyOrder("webhooks.dispatcher", "webhooks.endpoint." + active, "webhooks.endpoint." + automatic);
assertThat(registry.resolveAll(instance)).allSatisfy(check ->
{
assertThat(check.getKind()).isEqualTo(HealthCheckKind.DEPENDENCY);
assertThat(check.getCategory()).isEqualTo(HealthCategory.INTEGRATION);
assertThat(check.getComponent()).isEqualTo("webhooks");
assertThat(check.getIntervalSeconds()).isEqualTo(300);
assertThat(check.getTimeoutMs()).isLessThan(300000);
assertThat(check.getIncludeInReadiness()).isFalse();
});
}
/*******************************************************************************
** Read fresh persisted states and keep the existing state machine intact.
*******************************************************************************/
@Test
void endpointStatesAndRecovery() throws Exception
{
Integer id = endpoint();
HealthCheckMetaData check = check("webhooks.endpoint." + id);
assertOutcome(check, HealthStatus.UP);
for(String state : List.of("UNHEALTHY", "PROBATION"))
{
update(id, "healthStatus", state);
update(id, "consecutiveFailures", 3);
update(id, "unhealthySinceDate", WebhooksTestSupport.CLOCK.instant());
HealthCheckOutcome outcome = assertOutcome(check, HealthStatus.DEGRADED);
assertThat(outcome.getDetails().getValues()).containsExactlyInAnyOrderEntriesOf(Map.of(
"healthStatus", state, "consecutiveFailures", 3, "unhealthySince", WebhooksTestSupport.CLOCK.instant()));
}
update(id, "probationFailures", 1);
QRecord row = SystemWrites.query(WebhooksTables.ENDPOINT, new QQueryFilter()).getFirst();
EndpointHealth.recordFailure(WebhooksRuntime.config(), row, WebhooksTestSupport.CLOCK.instant(), SENTINEL);
assertOutcome(check, HealthStatus.DOWN);
assertThat(check("webhooks.endpoint." + id)).isNotNull();
update(id, "activeStatus", "ACTIVE");
EndpointHealth.recordSuccess(row, WebhooksTestSupport.CLOCK.instant());
assertOutcome(check, HealthStatus.UP);
update(id, "healthStatus", SENTINEL);
assertOutcome(check, HealthStatus.UNKNOWN);
update(id, "healthStatus", null);
assertOutcome(check, HealthStatus.UNKNOWN);
update(id, "activeStatus", "DISABLED");
assertOutcome(check, HealthStatus.DOWN);
update(id, "autoDisabledDate", null);
assertOutcome(check, HealthStatus.UNKNOWN);
SystemWrites.delete(WebhooksTables.ENDPOINT, new QQueryFilter());
assertOutcome(check, HealthStatus.UNKNOWN);
}
/*******************************************************************************
** Strict thresholds and terminal-status exclusion across every queue state.
*******************************************************************************/
@Test
void dispatcherThresholds() throws Exception
{
HealthCheckMetaData check = check("webhooks.dispatcher");
assertThat(assertOutcome(check, HealthStatus.UP).getMetrics().getFirst().getValue().longValue()).isZero();
for(String terminal : List.of("DELIVERED", "CANCELLED", "DEAD_LETTER"))
{
delivery(terminal, 10000);
}
assertOutcome(check, HealthStatus.UP);
for(String status : List.of("PENDING", "RETRY_WAIT", "SENDING"))
{
for(long age : new long[] {-10, 0, 900, 901, 3600, 3601})
{
QRecord record = delivery(status, age);
HealthCheckOutcome outcome = assertOutcome(check, age > 3600 ? HealthStatus.DOWN : age > 900 ? HealthStatus.DEGRADED : HealthStatus.UP);
assertThat(outcome.getMetrics().getFirst().getName()).isEqualTo("oldestPendingSeconds");
assertThat(outcome.getMetrics().getFirst().getValue().longValue()).isEqualTo(Math.max(0, age));
SystemWrites.updateOne(WebhooksTables.DELIVERY, new QRecord().withValue("id", record.getValue("id")).withValue("status", "DELIVERED"));
}
}
delivery("PENDING", 4000);
delivery("PENDING", 1);
assertThat(assertOutcome(check, HealthStatus.DOWN).getMetrics().getFirst().getValue().longValue()).isEqualTo(4000);
}
/*******************************************************************************
** A missing queue timestamp cannot be reported as a healthy backlog.
*******************************************************************************/
@Test
void missingQueueTimestamp() throws Exception
{
QRecord record = delivery("PENDING", 0);
MemoryRecordStore.getInstance().update(new UpdateInput(WebhooksTables.DELIVERY).withRecords(List.of(
new QRecord().withValue("id", record.getValue("id")).withValue("createDate", null))), false);
assertOutcome(check("webhooks.dispatcher"), HealthStatus.UNKNOWN);
}
/*******************************************************************************
** No reads start after the deadline; late responses are not accepted as UP.
*******************************************************************************/
@Test
void deadlines() throws Exception
{
endpoint();
delivery("PENDING", 0);
for(HealthCheckMetaData check : HealthChecksMetaData.ofOrWithNew(instance).resolveAll(instance))
{
assertThat(check.getCheckInstance().run(input().withDeadline(Instant.now().minusSeconds(1))).getStatus()).isEqualTo(HealthStatus.DOWN);
}
instance.getTable(WebhooksTables.DELIVERY).withCustomizer(TableCustomizers.POST_QUERY_RECORD, new QCodeReference(SlowQuery.class));
HealthCheckOutcome outcome = check("webhooks.dispatcher").getCheckInstance().run(input().withDeadline(Instant.now().plusMillis(50)));
assertThat(outcome.getStatus()).isEqualTo(HealthStatus.DOWN);
assertThat(outcome.getMessage()).contains("timed out");
}
/*******************************************************************************
** Exception messages, URLs, failure text and credentials never leave checks.
*******************************************************************************/
@Test
void secretBearingFailures() throws Exception
{
Integer id = endpoint();
HealthCheckMetaData endpoint = check("webhooks.endpoint." + id);
delivery("PENDING", 0);
assertOutcome(endpoint, HealthStatus.UP);
assertOutcome(check("webhooks.dispatcher"), HealthStatus.UP);
for(String table : List.of(WebhooksTables.ENDPOINT, WebhooksTables.DELIVERY))
{
instance.getTable(table).withCustomizer(TableCustomizers.POST_QUERY_RECORD, new QCodeReference(FailingQuery.class));
}
assertOutcome(endpoint, HealthStatus.DOWN);
assertOutcome(HealthChecksMetaData.ofOrWithNew(instance).getChecks().getFirst(), HealthStatus.DOWN);
}
/*******************************************************************************
** Persist a sentinel credential without ever resolving it for a health check.
*******************************************************************************/
private Integer endpoint() throws Exception
{
Integer id = WebhooksTestSupport.insert(WebhooksTables.ENDPOINT, new QRecord().withValue("name", SENTINEL + UUID.randomUUID())
.withValue("url", "https://example.invalid/" + SENTINEL)).getValueInteger("id");
WebhooksTestSupport.insert(WebhooksTables.SECRET, new QRecord().withValue("endpointId", id)
.withValue("source", "REFERENCE").withValue("secretRef", "WEBHOOKS_HEALTH_SECRET"));
update(id, "lastFailureMessage", SENTINEL);
return (id);
}
/*******************************************************************************
** Change an existing domain field as the qbit.
*******************************************************************************/
private void update(Integer id, String field, java.io.Serializable value) throws Exception
{
SystemWrites.updateOne(WebhooksTables.ENDPOINT, new QRecord().withValue("id", id).withValue(field, value));
}
/*******************************************************************************
** Insert a pending or terminal delivery with deterministic queue time.
*******************************************************************************/
private QRecord delivery(String status, long age) throws Exception
{
QRecord record = SystemWrites.insertOne(WebhooksTables.DELIVERY, new QRecord().withValue("status", status)
.withValue("payload", SENTINEL).withValue("lastError", SENTINEL));
MemoryRecordStore.getInstance().update(new UpdateInput(WebhooksTables.DELIVERY).withRecords(List.of(
new QRecord().withValue("id", record.getValue("id")).withValue("createDate", WebhooksTestSupport.CLOCK.instant().minusSeconds(age)))), false);
return (record);
}
/*******************************************************************************
** Find a producer-registered check, resolving the supplier afresh.
*******************************************************************************/
private HealthCheckMetaData check(String name) throws Exception
{
return (HealthChecksMetaData.ofOrWithNew(instance).resolveAll(instance).stream().filter(c -> name.equals(c.getName())).findFirst().orElseThrow());
}
/*******************************************************************************
** Bounded runner input.
*******************************************************************************/
private HealthCheckInput input()
{
return (new HealthCheckInput().withQInstance(instance).withDeadline(Instant.now().plusSeconds(5)));
}
/*******************************************************************************
** Assert raw outcomes are clean; HealthRunner.sanitize is package-private.
*******************************************************************************/
private HealthCheckOutcome assertOutcome(HealthCheckMetaData check, HealthStatus status) throws Exception
{
HealthCheckOutcome outcome = check.getCheckInstance().run(input());
assertThat(outcome.getStatus()).isEqualTo(status);
assertThat(outcome.getMessage()).doesNotContain(SENTINEL).doesNotContain("example.invalid").doesNotContain("\n");
assertThat(outcome.getDetails().toJson()).doesNotContain(SENTINEL).doesNotContain("example.invalid");
return (outcome);
}
/*******************************************************************************
** A backend failure containing sensitive text.
*******************************************************************************/
public static class FailingQuery implements TableCustomizerInterface
{
/***************************************************************************
** Throw after the read, before the check sees any records.
***************************************************************************/
@Override
public List<QRecord> postQuery(QueryOrGetInputInterface input, List<QRecord> records) throws QException
{
throw (new QException(SENTINEL));
}
}
/*******************************************************************************
** A database that responds after the check's deadline.
*******************************************************************************/
public static class SlowQuery implements TableCustomizerInterface
{
/***************************************************************************
** Simulate a slow database, with no network.
***************************************************************************/
@Override
public List<QRecord> postQuery(QueryOrGetInputInterface input, List<QRecord> records) throws QException
{
try
{
Thread.sleep(100);
}
catch(InterruptedException e)
{
Thread.currentThread().interrupt();
throw (new QException("Interrupted", e));
}
return (records);
}
}
}
@@ -0,0 +1,91 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.health;
import java.net.URLDecoder;
import java.nio.charset.StandardCharsets;
import java.util.List;
import com.kingsrook.qbits.webhooks.WebhooksTestSupport;
import com.kingsrook.qbits.webhooks.app.WebhooksApp;
import com.kingsrook.qbits.webhooks.support.SystemWrites;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.health.HealthMetaDataProvider;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.StatisticsData;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.session.QSession;
import com.kingsrook.qqq.backend.core.utils.JsonUtils;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** The native home line displays only the persisted central component state.
*******************************************************************************/
class WebhooksHealthWidgetTest
{
/*******************************************************************************
** No data means unknown, with a component-filtered link to System Health.
*******************************************************************************/
@Test
void emptyAndStoredComponent() throws Exception
{
var instance = WebhooksTestSupport.setUp();
assertThat(instance.getWidget(WebhooksApp.HEALTH_WIDGET).getType()).isEqualTo(WidgetType.STATISTICS.getType());
StatisticsData data = render();
assertThat(data.getCount()).isEqualTo("No checks have run yet");
assertThat(data.getCountURL()).startsWith("/app/health_check?filter=");
///////////////////////////////////////////////////////////////////////////////
// plain URL-encoded JSON (the form every qbit uses), not base64 //
///////////////////////////////////////////////////////////////////////////////
String filter = URLDecoder.decode(data.getCountURL().split("filter=")[1], StandardCharsets.UTF_8);
QQueryFilter decoded = JsonUtils.toObject(filter, QQueryFilter.class);
assertThat(decoded.getCriteria()).hasSize(1);
assertThat(decoded.getCriteria().getFirst().getFieldName()).isEqualTo("component");
assertThat(decoded.getCriteria().getFirst().getOperator()).isEqualTo(QCriteriaOperator.EQUALS);
assertThat(decoded.getCriteria().getFirst().getValues()).containsExactly("webhooks");
new HealthMetaDataProvider().defineAll(instance, WebhooksTestSupport.MEMORY, null);
QContext.init(instance, new QSession().withPermissions(List.of("health.read")));
assertThat(render().getCount()).isEqualTo("No checks have run yet");
QRecord component = SystemWrites.insertOne(HealthMetaDataProvider.COMPONENT_TABLE, new QRecord().withValue("name", "webhooks").withValue("status", "DOWN"));
for(String status : List.of("UP", "DEGRADED", "DOWN", "UNKNOWN", "S3NT1NEL-secret"))
{
SystemWrites.updateOne(HealthMetaDataProvider.COMPONENT_TABLE, new QRecord().withValue("id", component.getValue("id")).withValue("status", status));
assertThat(render().getCount()).isEqualTo(status.equals("S3NT1NEL-secret") ? "UNKNOWN" : status);
}
SystemWrites.updateOne(HealthMetaDataProvider.COMPONENT_TABLE, new QRecord().withValue("id", component.getValue("id")).withValue("status", null));
assertThat(render().getCount()).isEqualTo("UNKNOWN");
}
/*******************************************************************************
** Render the one-line native widget.
*******************************************************************************/
private StatisticsData render() throws Exception
{
return ((StatisticsData) new WebhooksHealthWidget().render(new RenderWidgetInput()).getWidgetData());
}
}
@@ -1,117 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.processes;
import java.time.Instant;
import java.time.temporal.ChronoUnit;
import java.util.ArrayList;
import java.util.List;
import com.kingsrook.qbits.webhooks.BaseTest;
import com.kingsrook.qbits.webhooks.model.Webhook;
import com.kingsrook.qbits.webhooks.model.WebhookEventSendLog;
import com.kingsrook.qbits.webhooks.model.WebhookHealthStatus;
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertEquals;
/*******************************************************************************
** Unit test for ManageWebhookHealthTransformStep
*******************************************************************************/
class ManageWebhookHealthTransformStepTest extends BaseTest
{
/*******************************************************************************
**
*******************************************************************************/
@Test
void test() throws QException
{
Integer healthyId = insert(newWebhook("healthy").withHealthStatusId(WebhookHealthStatus.HEALTHY.getId()));
Integer onProbationId = insert(newWebhook("probation").withHealthStatusId(WebhookHealthStatus.PROBATION.getId()));
Integer unhealthyJustNowId = insert(newWebhook("unhealthyJustNow").withHealthStatusId(WebhookHealthStatus.UNHEALTHY.getId()));
Integer unhealthyLongAgoId = insert(newWebhook("unhealthyLongAgo").withHealthStatusId(WebhookHealthStatus.UNHEALTHY.getId()));
Integer unhealthyNoLogsId = insert(newWebhook("unhealthyNoLogs").withHealthStatusId(WebhookHealthStatus.UNHEALTHY.getId()));
new InsertAction().execute(new InsertInput(WebhookEventSendLog.TABLE_NAME).withRecordEntities(List.of(
new WebhookEventSendLog().withWebhookId(unhealthyJustNowId).withStartTimestamp(Instant.now()),
new WebhookEventSendLog().withWebhookId(unhealthyLongAgoId).withStartTimestamp(Instant.now().minus(1, ChronoUnit.DAYS))
)));
RunBackendStepInput input = new RunBackendStepInput();
RunBackendStepOutput output = new RunBackendStepOutput();
input.withRecords(QueryAction.execute(Webhook.TABLE_NAME, new QQueryFilter()));
ManageWebhookHealthTransformStep manageWebhookHealthTransformStep = new ManageWebhookHealthTransformStep();
manageWebhookHealthTransformStep.runOnePage(input, output);
assertEquals(1, output.getRecords().size());
assertEquals(unhealthyLongAgoId, output.getRecords().get(0).getValueInteger("id"));
ArrayList<ProcessSummaryLineInterface> processSummaryLineInterfaces = manageWebhookHealthTransformStep.doGetProcessSummary(output, false);
assertEquals(3, processSummaryLineInterfaces.size());
assertEquals(Status.OK, processSummaryLineInterfaces.get(0).getStatus());
assertEquals(1, ((ProcessSummaryLine)processSummaryLineInterfaces.get(0)).getCount());
assertEquals(Status.INFO, processSummaryLineInterfaces.get(1).getStatus());
assertEquals(2, ((ProcessSummaryLine)processSummaryLineInterfaces.get(1)).getCount());
assertThat(processSummaryLineInterfaces.get(1).getMessage()).contains("not currently Unhealthy");
assertEquals(Status.ERROR, processSummaryLineInterfaces.get(2).getStatus());
assertEquals(2, ((ProcessSummaryLine)processSummaryLineInterfaces.get(2)).getCount());
assertThat(processSummaryLineInterfaces.get(2).getMessage()).contains("Unhealthy long enough to be put on Probation");
}
/*******************************************************************************
**
*******************************************************************************/
@Test
void testNoUnhealthyInputs() throws QException
{
Integer healthyId = insert(newWebhook("healthy").withHealthStatusId(WebhookHealthStatus.HEALTHY.getId()));
RunBackendStepInput input = new RunBackendStepInput();
RunBackendStepOutput output = new RunBackendStepOutput();
input.withRecords(QueryAction.execute(Webhook.TABLE_NAME, new QQueryFilter()));
ManageWebhookHealthTransformStep manageWebhookHealthTransformStep = new ManageWebhookHealthTransformStep();
manageWebhookHealthTransformStep.runOnePage(input, output);
assertEquals(0, output.getRecords().size());
}
}
@@ -1,149 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.processes;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import com.kingsrook.qbits.webhooks.BaseTest;
import com.kingsrook.qbits.webhooks.actions.WebhookEventSender;
import com.kingsrook.qbits.webhooks.actions.WebhookEventSenderTest;
import com.kingsrook.qbits.webhooks.model.Webhook;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
/*******************************************************************************
** Unit test for SendTestEventToWebhookLoadStep
*******************************************************************************/
class SendTestEventToWebhookLoadStepTest extends BaseTest
{
/*******************************************************************************
**
*******************************************************************************/
@Test
void testSuccess() throws QException
{
ProcessSummaryLine processSummaryLine = runProcessForSender(new WebhookEventSenderTest.WebhookEventSenderThatSucceeds(), 2);
assertEquals(Status.OK, processSummaryLine.getStatus());
assertEquals("were delivered successfully", processSummaryLine.getMessage());
assertEquals(2, processSummaryLine.getCount());
}
/*******************************************************************************
**
*******************************************************************************/
@Test
void testErrorWithoutStatusCode() throws QException
{
ProcessSummaryLine processSummaryLine = runProcessForSender(new WebhookEventSenderTest.WebhookEventSenderThatFails());
assertEquals(Status.ERROR, processSummaryLine.getStatus());
assertEquals("failed to send with error message: Test failure", processSummaryLine.getMessage());
}
/*******************************************************************************
**
*******************************************************************************/
@Test
void testErrorsRollup() throws QException
{
ProcessSummaryLine processSummaryLine = runProcessForSender(new WebhookEventSenderTest.WebhookEventSenderThatFails(), 3);
assertEquals(Status.ERROR, processSummaryLine.getStatus());
assertEquals("failed to send with error message: Test failure", processSummaryLine.getMessage());
assertEquals(3, processSummaryLine.getCount());
}
/*******************************************************************************
**
*******************************************************************************/
@Test
void testErrorWithStatusCode() throws QException
{
ProcessSummaryLine processSummaryLine = runProcessForSender(new WebhookEventSenderTest.WebhookEventSenderThatMocksHttp(500, "Server Error"));
assertEquals(Status.ERROR, processSummaryLine.getStatus());
assertEquals("failed to send with error message: Status Code: 500; Server Error", processSummaryLine.getMessage());
}
/***************************************************************************
**
***************************************************************************/
private static ProcessSummaryLine runProcessForSender(WebhookEventSender webhookEventSender) throws QException
{
return runProcessForSender(webhookEventSender, 1);
}
/***************************************************************************
**
***************************************************************************/
private static ProcessSummaryLine runProcessForSender(WebhookEventSender webhookEventSender, int nCopies) throws QException
{
SendTestEventToWebhookLoadStep step = getSendTestEventToWebhookLoadStep(webhookEventSender);
RunBackendStepOutput runBackendStepOutput = new RunBackendStepOutput();
List<QRecord> records = Collections.nCopies(nCopies, new Webhook().withUrl("http://localhost:8000/").toQRecord());
step.runOnePage(new RunBackendStepInput().withRecords(records), runBackendStepOutput);
ArrayList<ProcessSummaryLineInterface> processSummary = step.doGetProcessSummary(runBackendStepOutput, true);
ProcessSummaryLine processSummaryLine = (ProcessSummaryLine) processSummary.get(0);
return processSummaryLine;
}
/***************************************************************************
*
***************************************************************************/
private static SendTestEventToWebhookLoadStep getSendTestEventToWebhookLoadStep(WebhookEventSender webhookEventSender)
{
SendTestEventToWebhookLoadStep step = new SendTestEventToWebhookLoadStep()
{
/***************************************************************************
**
***************************************************************************/
@Override
protected WebhookEventSender getWebhookEventSender()
{
return webhookEventSender;
}
};
return step;
}
}
@@ -1,235 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.processes;
import java.time.Instant;
import java.util.List;
import java.util.Map;
import java.util.UUID;
import java.util.function.Consumer;
import com.kingsrook.qbits.webhooks.BaseTest;
import com.kingsrook.qbits.webhooks.WebhooksTestApplication;
import com.kingsrook.qbits.webhooks.model.Webhook;
import com.kingsrook.qbits.webhooks.model.WebhookActiveStatus;
import com.kingsrook.qbits.webhooks.model.WebhookEvent;
import com.kingsrook.qbits.webhooks.model.WebhookEventStatus;
import com.kingsrook.qbits.webhooks.model.WebhookHealthStatus;
import com.kingsrook.qbits.webhooks.model.WebhookSubscription;
import com.kingsrook.qqq.backend.core.actions.reporting.RecordPipe;
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ExtractViaQueryStep;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
/*******************************************************************************
** Unit test for SendWebhookEventExtractStep
*******************************************************************************/
class SendWebhookEventExtractStepTest extends BaseTest
{
/*******************************************************************************
**
*******************************************************************************/
@Test
void testEventsFilteredByWebhook() throws QException
{
/////////////////////////////////////////////////////////////////////////////////////////////////////////////
// make sure if there are valid events for multiple webhooks, that we only find the ones we're looking for //
/////////////////////////////////////////////////////////////////////////////////////////////////////////////
Integer webhookId1 = 1;
Integer webhookId2 = 2;
List<QRecord> webhooks = new InsertAction().execute(new InsertInput(Webhook.TABLE_NAME).withRecordEntities(List.of(
new Webhook().withId(webhookId1).withName("a").withUrl("a").withSubscriptions(List.of(newWebhookSubscription(WebhooksTestApplication.PERSON_INSERTED_EVENT_TYPE_NAME))),
new Webhook().withId(webhookId2).withName("b").withUrl("b").withSubscriptions(List.of(newWebhookSubscription(WebhooksTestApplication.PERSON_INSERTED_EVENT_TYPE_NAME)))
))).getRecords();
WebhookSubscription subscription1 = new WebhookSubscription(webhooks.get(0).getAssociatedRecords().get(Webhook.SUBSCRIPTIONS_ASSOCIATION_NAME).get(0));
WebhookSubscription subscription2 = new WebhookSubscription(webhooks.get(1).getAssociatedRecords().get(Webhook.SUBSCRIPTIONS_ASSOCIATION_NAME).get(0));
new InsertAction().execute(new InsertInput(WebhookEvent.TABLE_NAME).withRecordEntities(List.of(
////////////////////////////
// 3 events for webhook 1 //
////////////////////////////
newWebhookEvent(subscription1, WebhooksTestApplication.PERSON_INSERTED_EVENT_TYPE_NAME).withEventSourceRecordId(11),
newWebhookEvent(subscription1, WebhooksTestApplication.PERSON_INSERTED_EVENT_TYPE_NAME).withEventSourceRecordId(12),
newWebhookEvent(subscription1, WebhooksTestApplication.PERSON_INSERTED_EVENT_TYPE_NAME).withEventSourceRecordId(13),
////////////////////////////
// 2 events for webhook 2 //
////////////////////////////
newWebhookEvent(subscription2, WebhooksTestApplication.PERSON_INSERTED_EVENT_TYPE_NAME).withEventSourceRecordId(20),
newWebhookEvent(subscription2, WebhooksTestApplication.PERSON_INSERTED_EVENT_TYPE_NAME).withEventSourceRecordId(21)
)));
assertEquals(3, runStepForWebhookId(webhookId1).size());
assertEquals(2, runStepForWebhookId(webhookId2).size());
assertEquals(0, runStepForWebhookId(-1).size());
}
/*******************************************************************************
**
*******************************************************************************/
@Test
void testWebhookScenarios() throws QException
{
testWebhookScenario(true, new Webhook());
testWebhookScenario(false, new Webhook().withActiveStatusId(WebhookActiveStatus.PAUSED.getId()));
testWebhookScenario(false, new Webhook().withActiveStatusId(WebhookActiveStatus.DISABLED.getId()));
testWebhookScenario(false, new Webhook().withHealthStatusId(WebhookHealthStatus.UNHEALTHY.getId()));
testWebhookScenario(true, new Webhook().withHealthStatusId(WebhookHealthStatus.PROBATION.getId()));
}
/***************************************************************************
**
***************************************************************************/
private void testWebhookScenario(boolean expectToFindEvent, Webhook webhook) throws QException
{
QRecord insertedWebhook = new InsertAction().execute(new InsertInput(Webhook.TABLE_NAME).withRecordEntities(List.of(webhook
.withName(UUID.randomUUID().toString())
.withUrl("a")
.withSubscriptions(List.of(
newWebhookSubscription(WebhooksTestApplication.PERSON_INSERTED_EVENT_TYPE_NAME)
))))).getRecords().get(0);
WebhookSubscription subscription = new WebhookSubscription(insertedWebhook.getAssociatedRecords().get(Webhook.SUBSCRIPTIONS_ASSOCIATION_NAME).get(0));
new InsertAction().execute(new InsertInput(WebhookEvent.TABLE_NAME).withRecordEntities(List.of(
newWebhookEvent(subscription, WebhooksTestApplication.PERSON_INSERTED_EVENT_TYPE_NAME).withEventSourceRecordId(100)
)));
List<QRecord> events = runStepForWebhookId(insertedWebhook.getValueInteger("id"));
if(expectToFindEvent)
{
assertFalse(events.isEmpty());
}
else
{
assertTrue(events.isEmpty());
}
}
/*******************************************************************************
**
*******************************************************************************/
@Test
void testWebhookEventScenarios() throws QException
{
Instant past = Instant.now().minusSeconds(60);
Instant future = Instant.now().plusSeconds(60);
////////////////////////////////////////////////////////////////////////////////////////////
// new should always match //
// though we could imagine a change in the future to allow new events to be future-dated? //
////////////////////////////////////////////////////////////////////////////////////////////
testWebhookEventScenario(true, we -> we.withEventStatusId(WebhookEventStatus.NEW.getId()));
testWebhookEventScenario(true, we -> we.withEventStatusId(WebhookEventStatus.NEW.getId()).withNextAttemptTimestamp(future));
//////////////////////////////////////////////////////////////////////////////////////////////////
// sending or awaiting retry - they both use nextAttemptTimestamp, so only true if in the past. //
//////////////////////////////////////////////////////////////////////////////////////////////////
testWebhookEventScenario(false, we -> we.withEventStatusId(WebhookEventStatus.SENDING.getId()));
testWebhookEventScenario(true, we -> we.withEventStatusId(WebhookEventStatus.SENDING.getId()).withNextAttemptTimestamp(past));
testWebhookEventScenario(false, we -> we.withEventStatusId(WebhookEventStatus.SENDING.getId()).withNextAttemptTimestamp(future));
testWebhookEventScenario(false, we -> we.withEventStatusId(WebhookEventStatus.AWAITING_RETRY.getId()));
testWebhookEventScenario(true, we -> we.withEventStatusId(WebhookEventStatus.AWAITING_RETRY.getId()).withNextAttemptTimestamp(past));
testWebhookEventScenario(false, we -> we.withEventStatusId(WebhookEventStatus.AWAITING_RETRY.getId()).withNextAttemptTimestamp(future));
////////////////////////////////////////////////////////////////////////////////////
// delivered and failed - should never be found, even if timestamp is in the past //
////////////////////////////////////////////////////////////////////////////////////
testWebhookEventScenario(false, we -> we.withEventStatusId(WebhookEventStatus.DELIVERED.getId()));
testWebhookEventScenario(false, we -> we.withEventStatusId(WebhookEventStatus.DELIVERED.getId()).withNextAttemptTimestamp(past));
testWebhookEventScenario(false, we -> we.withEventStatusId(WebhookEventStatus.FAILED.getId()));
testWebhookEventScenario(false, we -> we.withEventStatusId(WebhookEventStatus.FAILED.getId()).withNextAttemptTimestamp(past));
}
/***************************************************************************
**
***************************************************************************/
private void testWebhookEventScenario(boolean expectToFindEvent, Consumer<WebhookEvent> webhookEventCustomizer) throws QException
{
QRecord insertedWebhook = new InsertAction().execute(new InsertInput(Webhook.TABLE_NAME).withRecordEntities(List.of(new Webhook()
.withName(UUID.randomUUID().toString())
.withUrl("a")
.withSubscriptions(List.of(
newWebhookSubscription(WebhooksTestApplication.PERSON_INSERTED_EVENT_TYPE_NAME)
))))).getRecords().get(0);
WebhookSubscription subscription = new WebhookSubscription(insertedWebhook.getAssociatedRecords().get(Webhook.SUBSCRIPTIONS_ASSOCIATION_NAME).get(0));
WebhookEvent webhookEvent = newWebhookEvent(subscription, WebhooksTestApplication.PERSON_INSERTED_EVENT_TYPE_NAME);
webhookEventCustomizer.accept(webhookEvent);
new InsertAction().execute(new InsertInput(WebhookEvent.TABLE_NAME).withRecordEntities(List.of(webhookEvent)));
List<QRecord> events = runStepForWebhookId(insertedWebhook.getValueInteger("id"));
if(expectToFindEvent)
{
assertFalse(events.isEmpty());
}
else
{
assertTrue(events.isEmpty());
}
}
/***************************************************************************
**
***************************************************************************/
private static List<QRecord> runStepForWebhookId(Integer webhookId1) throws QException
{
RunBackendStepInput input = new RunBackendStepInput().withValues(Map.of("webhookId", webhookId1, ExtractViaQueryStep.FIELD_SOURCE_TABLE, WebhookEvent.TABLE_NAME));
RunBackendStepOutput output = new RunBackendStepOutput();
SendWebhookEventExtractStep sendWebhookEventExtractStep = new SendWebhookEventExtractStep();
RecordPipe recordPipe = new RecordPipe();
sendWebhookEventExtractStep.setRecordPipe(recordPipe);
sendWebhookEventExtractStep.preRun(input, output);
sendWebhookEventExtractStep.run(input, output);
List<QRecord> records = recordPipe.consumeAvailableRecords();
return records;
}
}
@@ -1,280 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.processes;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import com.kingsrook.qbits.webhooks.BaseTest;
import com.kingsrook.qbits.webhooks.WebhooksTestApplication;
import com.kingsrook.qbits.webhooks.actions.WebhookEventSender;
import com.kingsrook.qbits.webhooks.actions.WebhookEventSenderTest;
import com.kingsrook.qbits.webhooks.model.Webhook;
import com.kingsrook.qbits.webhooks.model.WebhookEvent;
import com.kingsrook.qbits.webhooks.model.WebhookEventSendLog;
import com.kingsrook.qbits.webhooks.model.WebhookEventStatus;
import com.kingsrook.qbits.webhooks.model.WebhookHealthStatus;
import com.kingsrook.qbits.webhooks.model.WebhookSubscription;
import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertEquals;
/*******************************************************************************
** Unit test for SendWebhookEventLoadStep
*******************************************************************************/
class SendWebhookEventLoadStepTest extends BaseTest
{
/*******************************************************************************
* we had 12 fails on the books, so we'll have one more fail and go to unhealthy
*******************************************************************************/
@Test
void testLast12failedThenAnotherFailed() throws QException
{
String eventTypeName = WebhooksTestApplication.PERSON_INSERTED_EVENT_TYPE_NAME;
Integer webhookId0 = insert(newWebhook("test"));
Webhook webhook = new Webhook(GetAction.execute(Webhook.TABLE_NAME, webhookId0));
Integer subscriptionId = insert(newWebhookSubscription(eventTypeName).withWebhookId(webhookId0));
WebhookSubscription webhookSubscription = new WebhookSubscription(GetAction.execute(WebhookSubscription.TABLE_NAME, subscriptionId));
Integer webhookId = webhookSubscription.getWebhookId();
///////////////////////////////////////////
// insert 3 events, each with 4 failures //
///////////////////////////////////////////
Integer eventId0 = insertEventAwaitingRetry(webhookSubscription);
Integer eventId1 = insertEventAwaitingRetry(webhookSubscription);
Integer eventId2 = insertEventAwaitingRetry(webhookSubscription);
insertNFailLogsPerEvent(webhookId, 4, eventId0, eventId1, eventId2);
ArrayList<ProcessSummaryLineInterface> processSummaryLines = runProcess(webhook, new WebhookEventSenderTest.WebhookEventSenderThatFails(), eventId0, eventId1, eventId2);
////////////////////////////////////////
// make sure webhook became unhealthy //
////////////////////////////////////////
assertEquals(WebhookHealthStatus.UNHEALTHY.getId(), GetAction.execute(Webhook.TABLE_NAME, webhookId).getValue("healthStatusId"));
///////////////////////////////////////////////////////////////////////////////////////////////////////
// only the first event should have been processed, because the webhook became unhealthy after that. //
///////////////////////////////////////////////////////////////////////////////////////////////////////
assertWebhookEventStatus(WebhookEventStatus.FAILED, eventId0);
assertWebhookEventStatus(WebhookEventStatus.AWAITING_RETRY, eventId1);
assertWebhookEventStatus(WebhookEventStatus.AWAITING_RETRY, eventId2);
assertEquals(2, processSummaryLines.size());
assertEquals(Status.ERROR, processSummaryLines.get(0).getStatus());
assertEquals(1, ((ProcessSummaryLine) processSummaryLines.get(0)).getCount());
assertThat(processSummaryLines.get(0).getMessage()).contains("failed to deliver");
assertEquals(Status.WARNING, processSummaryLines.get(1).getStatus());
assertEquals(2, ((ProcessSummaryLine) processSummaryLines.get(1)).getCount());
assertThat(processSummaryLines.get(1).getMessage()).contains("were not sent because their associated webhook became unhealthy.");
}
/*******************************************************************************
* we had 12 fails on the books, but now a success and we'll be successful
*******************************************************************************/
@Test
void testLast12failedThenSuccess() throws QException
{
String eventTypeName = WebhooksTestApplication.PERSON_INSERTED_EVENT_TYPE_NAME;
Integer webhookId0 = insert(newWebhook("test").withHealthStatusId(WebhookHealthStatus.PROBATION.getId()));
Webhook webhook = new Webhook(GetAction.execute(Webhook.TABLE_NAME, webhookId0));
Integer subscriptionId = insert(newWebhookSubscription(eventTypeName).withWebhookId(webhookId0));
WebhookSubscription webhookSubscription = new WebhookSubscription(GetAction.execute(WebhookSubscription.TABLE_NAME, subscriptionId));
Integer webhookId = webhookSubscription.getWebhookId();
///////////////////////////////////////////
// insert 3 events, each with 4 failures //
///////////////////////////////////////////
Integer eventId0 = insertEventAwaitingRetry(webhookSubscription);
Integer eventId1 = insertEventAwaitingRetry(webhookSubscription);
Integer eventId2 = insertEventAwaitingRetry(webhookSubscription);
insertNFailLogsPerEvent(webhookId, 4, eventId0, eventId1, eventId2);
ArrayList<ProcessSummaryLineInterface> processSummaryLines = runProcess(webhook, new WebhookEventSenderTest.WebhookEventSenderThatSucceeds(), eventId0, eventId1, eventId2);
//////////////////////////////////////
// make sure webhook became healthy //
//////////////////////////////////////
assertEquals(WebhookHealthStatus.HEALTHY.getId(), GetAction.execute(Webhook.TABLE_NAME, webhookId).getValue("healthStatusId"));
///////////////////////////////////////////////////////////////////////////////////////////////////////
// only the first event should have been processed, because the webhook became unhealthy after that. //
///////////////////////////////////////////////////////////////////////////////////////////////////////
assertWebhookEventStatus(WebhookEventStatus.DELIVERED, eventId0);
assertWebhookEventStatus(WebhookEventStatus.DELIVERED, eventId1);
assertWebhookEventStatus(WebhookEventStatus.DELIVERED, eventId2);
assertEquals(1, processSummaryLines.size());
assertEquals(Status.OK, processSummaryLines.get(0).getStatus());
assertEquals(3, ((ProcessSummaryLine) processSummaryLines.get(0)).getCount());
assertThat(processSummaryLines.get(0).getMessage()).contains("were delivered");
}
/*******************************************************************************
* we have 6 fails on the books - now all 3 will fail, but not enough to go unhealthy
*******************************************************************************/
@Test
void testLast6failed() throws QException
{
String eventTypeName = WebhooksTestApplication.PERSON_INSERTED_EVENT_TYPE_NAME;
Integer webhookId0 = insert(newWebhook("test"));
Webhook webhook = new Webhook(GetAction.execute(Webhook.TABLE_NAME, webhookId0));
Integer subscriptionId = insert(newWebhookSubscription(eventTypeName).withWebhookId(webhookId0));
WebhookSubscription webhookSubscription = new WebhookSubscription(GetAction.execute(WebhookSubscription.TABLE_NAME, subscriptionId));
Integer webhookId = webhookSubscription.getWebhookId();
///////////////////////////////////////////
// insert 3 events, each with 2 failures //
///////////////////////////////////////////
Integer eventId0 = insertEventAwaitingRetry(webhookSubscription);
Integer eventId1 = insertEventAwaitingRetry(webhookSubscription);
Integer eventId2 = insertEventAwaitingRetry(webhookSubscription);
insertNFailLogsPerEvent(webhookId, 2, eventId0, eventId1, eventId2);
ArrayList<ProcessSummaryLineInterface> processSummaryLines = runProcess(webhook, new WebhookEventSenderTest.WebhookEventSenderThatFails(), eventId0, eventId1, eventId2);
////////////////////////////////////////
// make sure webhook is still healthy //
////////////////////////////////////////
assertEquals(WebhookHealthStatus.HEALTHY.getId(), GetAction.execute(Webhook.TABLE_NAME, webhookId).getValue("healthStatusId"));
//////////////////////////////////////////
// all 3 should still be awaiting retry //
//////////////////////////////////////////
assertWebhookEventStatus(WebhookEventStatus.AWAITING_RETRY, eventId0);
assertWebhookEventStatus(WebhookEventStatus.AWAITING_RETRY, eventId1);
assertWebhookEventStatus(WebhookEventStatus.AWAITING_RETRY, eventId2);
assertEquals(1, processSummaryLines.size());
assertEquals(Status.ERROR, processSummaryLines.get(0).getStatus());
assertEquals(3, ((ProcessSummaryLine) processSummaryLines.get(0)).getCount());
assertThat(processSummaryLines.get(0).getMessage()).contains("failed to deliver");
}
/***************************************************************************
*
***************************************************************************/
private void assertWebhookEventStatus(WebhookEventStatus webhookEventStatus, Integer eventId) throws QException
{
assertEquals(webhookEventStatus.getId(), GetAction.execute(WebhookEvent.TABLE_NAME, eventId).getValue("eventStatusId"));
}
/***************************************************************************
*
***************************************************************************/
private static ArrayList<ProcessSummaryLineInterface> runProcess(Webhook webhook, WebhookEventSender webhookEventSender, Integer... eventIds) throws QException
{
SendWebhookEventLoadStep loadStep = getSendWebhookEventLoadStep(webhookEventSender);
loadStep.setTransformStep(new SendWebhookEventTransformStep().withWebhook(webhook));
//////////////////////////////////
// run process for these events //
//////////////////////////////////
RunBackendStepInput input = new RunBackendStepInput();
input.withRecords(QueryAction.execute(WebhookEvent.TABLE_NAME, new QQueryFilter()
.withCriteria(new QFilterCriteria("id", QCriteriaOperator.IN, Arrays.stream(eventIds).toList()))
.withOrderBy(new QFilterOrderBy("id"))));
RunBackendStepOutput output = new RunBackendStepOutput();
loadStep.preRun(input, output);
loadStep.runOnePage(input, output);
ArrayList<ProcessSummaryLineInterface> processSummaryLines = loadStep.doGetProcessSummary(output, true);
return processSummaryLines;
}
/***************************************************************************
*
***************************************************************************/
private static void insertNFailLogsPerEvent(Integer webhookId, int n, Integer... eventIds) throws QException
{
for(Integer eventId : eventIds)
{
new InsertAction().execute(new InsertInput(WebhookEventSendLog.TABLE_NAME).withRecordEntities(Collections.nCopies(n,
new WebhookEventSendLog().withWebhookId(webhookId).withWebhookEventId(eventId).withSuccessful(false))));
}
}
/***************************************************************************
*
***************************************************************************/
private static Integer insertEventAwaitingRetry(WebhookSubscription webhookSubscription) throws QException
{
return insert(newWebhookEvent(webhookSubscription, webhookSubscription.getWebhookEventTypeName()).withEventStatusId(WebhookEventStatus.AWAITING_RETRY.getId()));
}
/***************************************************************************
*
***************************************************************************/
private static SendWebhookEventLoadStep getSendWebhookEventLoadStep(WebhookEventSender webhookEventSender)
{
SendWebhookEventLoadStep loadStep = new SendWebhookEventLoadStep()
{
/***************************************************************************
**
***************************************************************************/
@Override
protected WebhookEventSender newWebhookEventSender()
{
return webhookEventSender;
}
};
return loadStep;
}
}
@@ -1,149 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.processes;
import java.util.ArrayList;
import java.util.List;
import java.util.UUID;
import java.util.function.Consumer;
import com.kingsrook.qbits.webhooks.BaseTest;
import com.kingsrook.qbits.webhooks.WebhooksTestApplication;
import com.kingsrook.qbits.webhooks.model.Webhook;
import com.kingsrook.qbits.webhooks.model.WebhookActiveStatus;
import com.kingsrook.qbits.webhooks.model.WebhookEvent;
import com.kingsrook.qbits.webhooks.model.WebhookEventStatus;
import com.kingsrook.qbits.webhooks.model.WebhookHealthStatus;
import com.kingsrook.qbits.webhooks.model.WebhookSubscription;
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertEquals;
/*******************************************************************************
** Unit test for SendWebhookEventTransformStep
*******************************************************************************/
class SendWebhookEventTransformStepTest extends BaseTest
{
/*******************************************************************************
**
*******************************************************************************/
@Test
void test() throws QException
{
///////////////////////////////////////
// everything "normal", will be sent //
///////////////////////////////////////
testOneScenario("will be sent", null, null, null);
////////////////////////////
// missing webhook errors //
////////////////////////////
testOneScenario("webhook was not found", e -> e.setWebhookId(-1), null, null);
testOneScenario("webhook was not found", e -> e.setWebhookId(null), null, null);
/////////////////////////////////
// missing subscription errors //
/////////////////////////////////
testOneScenario("subscription was not found", e -> e.setWebhookSubscriptionId(-1), null, null);
testOneScenario("subscription was not found", e -> e.setWebhookSubscriptionId(null), null, null);
/////////////////////////////////
// disabled things do not send //
/////////////////////////////////
testOneScenario("webhook is disabled", null, w -> w.setActiveStatusId(WebhookActiveStatus.DISABLED.getId()), null);
testOneScenario("subscription is disabled", null, null, s -> s.setActiveStatusId(WebhookActiveStatus.DISABLED.getId()));
////////////////////////////////////////////////////////////////////////////////////////////////
// paused one do - if they were manually pulled into the process (extract step excludes them) //
////////////////////////////////////////////////////////////////////////////////////////////////
testOneScenario("will be sent", null, w -> w.setActiveStatusId(WebhookActiveStatus.PAUSED.getId()), null);
testOneScenario("will be sent", null, null, s -> s.setActiveStatusId(WebhookActiveStatus.PAUSED.getId()));
//////////////////////////////////////////////////////
// unhealthy webhook does not send - probation does //
//////////////////////////////////////////////////////
testOneScenario("webhook is unhealthy", null, w -> w.setHealthStatusId(WebhookHealthStatus.UNHEALTHY.getId()), null);
testOneScenario("will be sent", null, w -> w.setHealthStatusId(WebhookHealthStatus.PROBATION.getId()), null);
/////////////////////////////
// already-delivered warns //
/////////////////////////////
testOneScenario("will be sent again", e -> e.setEventStatusId(WebhookEventStatus.DELIVERED.getId()), null, null);
}
/***************************************************************************
**
***************************************************************************/
private ArrayList<ProcessSummaryLineInterface> testOneScenario(String expectedSummaryLineToContain, Consumer<WebhookEvent> eventCustomizer, Consumer<Webhook> webhookCustomizer, Consumer<WebhookSubscription> subscriptionCustomizer) throws QException
{
Webhook webhook = new Webhook()
.withName(UUID.randomUUID().toString())
.withUrl("a");
if(webhookCustomizer != null)
{
webhookCustomizer.accept(webhook);
}
QRecord insertedWebhook = new InsertAction().execute(new InsertInput(Webhook.TABLE_NAME).withRecordEntities(List.of(webhook))).getRecords().get(0);
WebhookSubscription subscription = newWebhookSubscription(WebhooksTestApplication.PERSON_INSERTED_EVENT_TYPE_NAME)
.withWebhookId(insertedWebhook.getValueInteger("id"));
if(subscriptionCustomizer != null)
{
subscriptionCustomizer.accept(subscription);
}
WebhookSubscription insertedSubscription = new WebhookSubscription(new InsertAction().execute(new InsertInput(WebhookSubscription.TABLE_NAME).withRecordEntities(List.of(subscription))).getRecords().get(0));
WebhookEvent webhookEvent = newWebhookEvent(insertedSubscription, WebhooksTestApplication.PERSON_INSERTED_EVENT_TYPE_NAME);
if(eventCustomizer != null)
{
eventCustomizer.accept(webhookEvent);
}
QRecord insertedEvent = new InsertAction().execute(new InsertInput(WebhookEvent.TABLE_NAME).withRecordEntities(List.of(webhookEvent))).getRecords().get(0);
RunBackendStepInput input = new RunBackendStepInput();
RunBackendStepOutput output = new RunBackendStepOutput();
input.setRecords(List.of(insertedEvent));
SendWebhookEventTransformStep sendWebhookEventTransformStep = new SendWebhookEventTransformStep();
sendWebhookEventTransformStep.preRun(input, output);
sendWebhookEventTransformStep.runOnePage(input, output);
ArrayList<ProcessSummaryLineInterface> processSummaryLineInterfaces = sendWebhookEventTransformStep.doGetProcessSummary(output, false);
assertEquals(1, processSummaryLineInterfaces.size());
assertThat(processSummaryLineInterfaces.get(0).getMessage()).contains(expectedSummaryLineToContain);
return processSummaryLineInterfaces;
}
}
@@ -1,233 +0,0 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.registry;
import java.util.List;
import java.util.function.Consumer;
import com.kingsrook.qbits.webhooks.BaseTest;
import com.kingsrook.qbits.webhooks.WebhooksTestApplication;
import com.kingsrook.qbits.webhooks.actions.DefaultWebhookEventTypeCustomizer;
import com.kingsrook.qbits.webhooks.model.WebhookEventCategory;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.exceptions.QInstanceValidationException;
import com.kingsrook.qqq.backend.core.instances.QInstanceValidator;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.fail;
/*******************************************************************************
** Unit test for validation in the webhooks registry
*******************************************************************************/
public class WebhooksRegistryValidationAndEnrichmentTest extends BaseTest
{
/*******************************************************************************
**
*******************************************************************************/
@Test
void testValidation() throws QException
{
assertValidationFailureReasons(
(qInstance -> WebhooksRegistry.ofOrWithNew(qInstance).registerWebhookEventType(new WebhookEventType())),
"Webhook Event Type: null is missing a name",
"Webhook Event Type: null is missing a category"
);
assertValidationFailureReasons(
(qInstance -> WebhooksRegistry.ofOrWithNew(qInstance).registerWebhookEventType(new WebhookEventType()
.withName("myEventType")
.withCategory(WebhookEventCategory.INSERT)
)),
"Webhook Event Type: myEventType requires a tableName, but does not have one"
);
assertValidationFailureReasons(
(qInstance -> WebhooksRegistry.ofOrWithNew(qInstance).registerWebhookEventType(new WebhookEventType()
.withName("myEventType")
.withCategory(WebhookEventCategory.INSERT)
.withTableName("nonTable")
)),
"Webhook Event Type: myEventType references a non-existing tableName: nonTable"
);
assertValidationFailureReasons(
(qInstance -> WebhooksRegistry.ofOrWithNew(qInstance).registerWebhookEventType(new WebhookEventType()
.withName("myEventType")
.withCategory(WebhookEventCategory.INSERT_WITH_FIELD)
.withTableName(WebhooksTestApplication.TABLE_NAME_PERSON)
)),
"Webhook Event Type: myEventType requires a fieldName, but does not have one"
);
assertValidationFailureReasons(
(qInstance -> WebhooksRegistry.ofOrWithNew(qInstance).registerWebhookEventType(new WebhookEventType()
.withName("myEventType")
.withCategory(WebhookEventCategory.INSERT_WITH_FIELD)
.withTableName(WebhooksTestApplication.TABLE_NAME_PERSON)
.withFieldName("nonField")
)),
"Webhook Event Type: myEventType references a non-existing fieldName: nonField for table person"
);
assertValidationFailureReasons(
(qInstance -> WebhooksRegistry.ofOrWithNew(qInstance).registerWebhookEventType(new WebhookEventType()
.withName("myEventType")
.withCategory(WebhookEventCategory.INSERT_WITH_VALUE)
.withTableName(WebhooksTestApplication.TABLE_NAME_PERSON)
.withFieldName("firstName")
)),
"Webhook Event Type: myEventType requires a value, but does not have one"
);
assertValidationSuccess(
(qInstance -> WebhooksRegistry.ofOrWithNew(qInstance).registerWebhookEventType(new WebhookEventType()
.withName("myEventType")
.withCategory(WebhookEventCategory.INSERT_WITH_VALUE)
.withTableName(WebhooksTestApplication.TABLE_NAME_PERSON)
.withFieldName("firstName")
.withValue("Jean-Luc")
)));
assertValidationFailureReasons(
(qInstance -> WebhooksRegistry.ofOrWithNew(qInstance).registerWebhookEventType(new WebhookEventType()
.withName("myEventType")
.withCategory(WebhookEventCategory.AD_HOC)
.withCustomizer(new QCodeReference(getClass()))
)),
"Webhook Event Type: myEventType customizer codeReference: CodeReference is not of the expected type: interface com.kingsrook.qbits.webhooks.actions.WebhookEventTypeCustomizerInterface"
);
assertValidationSuccess(
(qInstance -> WebhooksRegistry.ofOrWithNew(qInstance).registerWebhookEventType(new WebhookEventType()
.withName("myEventType")
.withCategory(WebhookEventCategory.AD_HOC)
.withCustomizer(new QCodeReference(DefaultWebhookEventTypeCustomizer.class))
)));
}
/*******************************************************************************
**
*******************************************************************************/
@Test
void testEnrichment() throws QException
{
QInstance enrichedInstance = assertValidationSuccess(
(qInstance -> WebhooksRegistry.ofOrWithNew(qInstance).registerWebhookEventType(new WebhookEventType()
.withName("myEventType")
.withCategory(WebhookEventCategory.AD_HOC)
)));
assertEquals("My Event Type", WebhooksRegistry.of(enrichedInstance).getWebhookEventType("myEventType").getLabel());
QInstance didntNeedEnrichedInstance = assertValidationSuccess(
(qInstance -> WebhooksRegistry.ofOrWithNew(qInstance).registerWebhookEventType(new WebhookEventType()
.withName("myEventType")
.withLabel("My Label")
.withCategory(WebhookEventCategory.AD_HOC)
)));
assertEquals("My Label", WebhooksRegistry.of(didntNeedEnrichedInstance).getWebhookEventType("myEventType").getLabel());
}
/*******************************************************************************
** Assert that an instance is valid!
*******************************************************************************/
public static QInstance assertValidationSuccess(Consumer<QInstance> setup) throws QException
{
try
{
QInstance qInstance = new WebhooksTestApplication().defineQInstance();
setup.accept(qInstance);
new QInstanceValidator().validate(qInstance);
return (qInstance);
}
catch(QInstanceValidationException e)
{
fail("Expected no validation errors, but received: " + e.getMessage());
return null;
}
}
/***************************************************************************
**
***************************************************************************/
private static void assertValidationFailureReasons(Consumer<QInstance> setup, String... expectedReasons) throws QException
{
try
{
QInstance qInstance = new WebhooksTestApplication().defineQInstance();
setup.accept(qInstance);
new QInstanceValidator().validate(qInstance);
fail("Should have thrown validationException");
}
catch(QInstanceValidationException e)
{
assertValidationFailureReasons(false, e.getReasons(), expectedReasons);
}
}
/*******************************************************************************
**
*******************************************************************************/
public static void assertValidationFailureReasons(boolean allowExtraReasons, List<String> actualReasons, String... expectedReasons)
{
if(!allowExtraReasons)
{
int noOfReasons = actualReasons == null ? 0 : actualReasons.size();
assertEquals(expectedReasons.length, noOfReasons, "Expected number of validation failure reasons.\nExpected reasons: " + String.join(",", expectedReasons)
+ "\nActual reasons: " + (noOfReasons > 0 ? String.join("\n", actualReasons) : "--"));
}
for(String reason : expectedReasons)
{
assertReason(reason, actualReasons);
}
}
/*******************************************************************************
** utility method for asserting that a specific reason string is found within
** the list of reasons in the QInstanceValidationException.
**
*******************************************************************************/
public static void assertReason(String reason, List<String> actualReasons)
{
assertNotNull(actualReasons, "Expected there to be a reason for the failure (but there was not)");
assertThat(actualReasons)
.withFailMessage("Expected any of:\n%s\nTo match: [%s]", actualReasons, reason)
.anyMatch(s -> s.contains(reason));
}
}
@@ -18,17 +18,26 @@
* limitations under the License.
*/
package com.kingsrook.qbits.webhooks.shared.signing;
package com.kingsrook.qbits.webhooks.signing;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.time.Clock;
import java.time.Duration;
import java.time.Instant;
import java.time.ZoneOffset;
import java.util.Arrays;
import java.util.Base64;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSecret;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookSigner;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerificationException;
import com.kingsrook.qqq.backend.core.utils.webhooks.WebhookVerifier;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
@@ -36,7 +45,7 @@ import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
/*******************************************************************************
** Unit test for WebhookSecret, WebhookSigner and WebhookVerifier.
** Qbit wire contract and interoperability with the released core primitives.
*******************************************************************************/
class WebhookSigningTest
{
@@ -45,6 +54,85 @@ class WebhookSigningTest
/*******************************************************************************
** Independent local HMAC and core interoperate without re-encoding the body.
** The reference uses literal wire names and prefixes to detect contract drift.
*******************************************************************************/
@Test
void testLocalAndCoreInteropWithRotation() throws Exception
{
List<String> texts = List.of("a-raw-shared-secret-of-enough-length", "whsec_MfKQ9r8GKYqrTwjUPD8ILPZIo2LaLaSw");
List<WebhookSecret> secrets = texts.stream().map(WebhookSecret::parse).toList();
for(byte[] body : List.of(BODY, new byte[0], new byte[] { 0, (byte) 0xff, (byte) 0xc3, (byte) 0xa9, 10 }))
{
String first = localSignature("evt_interop", "1760000000", body, texts.get(0));
String second = localSignature("evt_interop", "1760000000", body, texts.get(1));
Map<String, String> local = Map.of("webhook-id", "evt_interop", "webhook-timestamp", "1760000000",
"webhook-signature", "v1," + first + " v1," + second);
Map<String, String> core = WebhookSigner.headers("evt_interop", NOW, body, secrets);
assertThat(core).isEqualTo(local);
for(int i = 0; i < secrets.size(); i++)
{
verifier(List.of(secrets.get(i))).verify(local, body);
assertThat(localVerify(core, body, texts.get(i))).isTrue();
}
assertThat(localVerify(core, body, "an-unrelated-secret-of-enough-length")).isFalse();
verifier(secrets).verify(WebhookSigner.headers("evt_interop", NOW, body, List.of(secrets.get(1))), body);
}
}
/*******************************************************************************
** The five-minute window includes both boundaries, but not the next second.
*******************************************************************************/
@Test
void testExactTimestampTolerance()
{
assertThat(WebhookVerifier.DEFAULT_TOLERANCE).isEqualTo(Duration.ofSeconds(300));
WebhookSecret secret = WebhookSecret.generate();
for(long offset : new long[] { -300, 300 })
{
assertDoesNotThrow(() -> verifier(List.of(secret)).verify(WebhookSigner.headers("evt_edge", NOW.plusSeconds(offset), BODY, List.of(secret)), BODY));
}
for(long offset : new long[] { -301, 301 })
{
assertThatThrownBy(() -> verifier(List.of(secret)).verify(WebhookSigner.headers("evt_edge", NOW.plusSeconds(offset), BODY, List.of(secret)), BODY))
.isInstanceOf(WebhookVerificationException.class)
.extracting(e -> ((WebhookVerificationException) e).getReason())
.isEqualTo(WebhookVerificationException.Reason.TIMESTAMP_OUT_OF_TOLERANCE);
}
}
/*******************************************************************************
** Test-only reference HMAC, independent of core signing and secret parsing.
*******************************************************************************/
private static String localSignature(String id, String timestamp, byte[] body, String text) throws Exception
{
byte[] key = text.startsWith("whsec_") ? Base64.getDecoder().decode(text.substring(6)) : text.getBytes(StandardCharsets.UTF_8);
Mac mac = Mac.getInstance("HmacSHA256");
mac.init(new SecretKeySpec(key, "HmacSHA256"));
mac.update((id + "." + timestamp + ".").getBytes(StandardCharsets.UTF_8));
return (Base64.getEncoder().encodeToString(mac.doFinal(body)));
}
/*******************************************************************************
** Test-only receiver checks rotation signatures independently of core.
*******************************************************************************/
private static boolean localVerify(Map<String, String> headers, byte[] body, String secret) throws Exception
{
byte[] expected = localSignature(headers.get("webhook-id"), headers.get("webhook-timestamp"), body, secret).getBytes(StandardCharsets.US_ASCII);
return (Arrays.stream(headers.get("webhook-signature").split(" "))
.filter(part -> part.startsWith("v1,"))
.anyMatch(part -> MessageDigest.isEqual(expected, part.substring(3).getBytes(StandardCharsets.US_ASCII))));
}
/*******************************************************************************
** The Standard Webhooks reference vector (standard-webhooks/standard-webhooks,
** libraries/go/webhook_test.go, TestWebhookSign).
+8 -6
View File
@@ -49,7 +49,7 @@ CREATE TABLE webhooks_endpoint_secret
id INT AUTO_INCREMENT PRIMARY KEY,
create_date TIMESTAMP NOT NULL DEFAULT now(),
modify_date TIMESTAMP NOT NULL DEFAULT now(),
endpoint_id INTEGER NOT NULL REFERENCES webhooks_endpoint (id),
endpoint_id INTEGER NOT NULL CONSTRAINT fk_webhooks_endpoint_secret_endpoint_id REFERENCES webhooks_endpoint (id) ON DELETE CASCADE,
status VARCHAR(20) DEFAULT 'ACTIVE',
source VARCHAR(20) NOT NULL,
secret_ref VARCHAR(100),
@@ -68,7 +68,7 @@ CREATE TABLE webhooks_subscription
id INT AUTO_INCREMENT PRIMARY KEY,
create_date TIMESTAMP NOT NULL DEFAULT now(),
modify_date TIMESTAMP NOT NULL DEFAULT now(),
endpoint_id INTEGER NOT NULL REFERENCES webhooks_endpoint (id),
endpoint_id INTEGER NOT NULL CONSTRAINT fk_webhooks_subscription_endpoint_id REFERENCES webhooks_endpoint (id) ON DELETE CASCADE,
event_type_name VARCHAR(200) NOT NULL,
active_status VARCHAR(20) DEFAULT 'ACTIVE',
description VARCHAR(250),
@@ -92,8 +92,8 @@ CREATE TABLE webhooks_delivery
modify_date TIMESTAMP NOT NULL DEFAULT now(),
event_id VARCHAR(40) NOT NULL,
event_type_name VARCHAR(200) NOT NULL,
endpoint_id INTEGER NOT NULL,
subscription_id INTEGER,
endpoint_id INTEGER NOT NULL CONSTRAINT fk_webhooks_delivery_endpoint_id REFERENCES webhooks_endpoint (id) ON DELETE RESTRICT,
subscription_id INTEGER CONSTRAINT fk_webhooks_delivery_subscription_id REFERENCES webhooks_subscription (id) ON DELETE SET NULL,
source VARCHAR(20),
source_table VARCHAR(100),
source_record_id VARCHAR(100),
@@ -122,14 +122,16 @@ CREATE INDEX webhooks_delivery_status_ix ON webhooks_delivery (status, modify_da
CREATE INDEX webhooks_delivery_delivered_ix ON webhooks_delivery (delivered_date);
CREATE INDEX webhooks_delivery_subscription_ix ON webhooks_delivery (subscription_id);
CREATE TABLE webhooks_delivery_attempt
(
id INT AUTO_INCREMENT PRIMARY KEY,
create_date TIMESTAMP NOT NULL DEFAULT now(),
modify_date TIMESTAMP NOT NULL DEFAULT now(),
delivery_id INTEGER NOT NULL,
endpoint_id INTEGER NOT NULL,
delivery_id INTEGER NOT NULL CONSTRAINT fk_webhooks_delivery_attempt_delivery_id REFERENCES webhooks_delivery (id) ON DELETE CASCADE,
endpoint_id INTEGER NOT NULL CONSTRAINT fk_webhooks_delivery_attempt_endpoint_id REFERENCES webhooks_endpoint (id) ON DELETE RESTRICT,
attempt_number INTEGER NOT NULL,
outcome VARCHAR(20) NOT NULL,
claimed_by VARCHAR(100),