Compare commits

..
Author SHA1 Message Date
james.maesandCodex 18905b0067 ci: use the shared branch-aware CodeQL worker
Co-Authored-By: Codex <noreply@openai.com>
2026-10-21 10:02:17 -05:00
james.maesandCodex 9db6d4bb59 ci: adopt central orb 0.7.12 and stable gates (#31)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 08:58:18 -05:00
james.maesandCodex 9f581996d0 ci: centralize rc release completion in qqq orb (#30)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 02:30:33 -05:00
james.maesandCodex c9d324acd4 ci: adopt shared orb 0.7.11 (#29)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 01:01:51 -05:00
james.maes cd2128f78f Merge pull request #27 from QRun-IO/feature/central-ci
ci: adopt verified central QQQ orb 0.7.1
2026-10-20 18:53:02 -05:00
james.maesandCodex 3d336c3dfe ci: adopt verified central QQQ orb 0.7.2
Co-Authored-By: Codex <noreply@openai.com>
2026-10-20 18:42:43 -05:00
james.maes 73411423df Merge pull request #26 from QRun-IO/feature/edi-business-home
fix(app): business-only EDI home; system widgets for the Admin home
2026-10-16 10:19:39 -05:00
james.maesandClaude Opus 5.5 cccd0ac0ab fix(app): business-only EDI home; system widgets for the Admin home
Move System, Dead letters, Certificates expiring and Setup status off the
EDI home into EdiHome.ADMIN_WIDGET_NAMES, gated to edi.admin.hasAccess
(hidden when denied) and full width, for a host to place on its Admin
home. Every remaining home widget requires edi.hasAccess (hidden when
denied); none is NOT_PROTECTED.

Add "Inbound documents are delayed": an alert at the top of the home,
shown only when the oldest unprocessed inbound document is older than
inboundDelayedAfterMinutes (default 60), with its age and a link to the
filtered inbound documents.

Setup counts that fail show "Unavailable" and are logged, never -1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 09:28:46 -05:00
james.maes 2681447b07 Merge pull request #25 from QRun-IO/ci/release-rc-secrets-inherit
ci(release-rc): pass secrets so the shared workflow can sign the RC tag
2026-10-16 08:05:13 -05:00
james.maesandClaude Opus 5.5 55a28c7031 ci(release-rc): pass secrets to the shared workflow so it can sign the RC tag
QRun-IO/.github#2 adds a tag-and-release job that signs vX with the
QRUN_TAG_SIGNING_KEY secret after Maven Central confirms the artifact.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 16:19:12 -05:00
james.maes 2491669f23 Merge pull request #24 from QRun-IO/feature/ui-api-1.6
feat(ui-api): extension for QQQ UI API 1.6.0 (operationIds, BasicErrorResponseV1, QQQ 4.1.0-RC.15)
2026-10-15 16:13:04 -05:00
james.maesandClaude Opus 5.5 36f1b939b4 chore: target QQQ 4.1.0-RC.15 (UI API 1.6.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 12:27:06 -05:00
james.maesandClaude Opus 5.5 77d63135af feat(ui-api): extension for QQQ UI API 1.6.0 (operationIds, BasicErrorResponseV1, QQQ 4.1.0-RC.14)
Regenerated against UI API 1.6.0: every operation has an operationId and
the standard error responses (BasicErrorResponseV1: error, code, message,
fieldErrors, traceId); the extension's own error schema is kept but
deprecated; hand-written error bodies go through QJavalinUtils so they carry
code/message/traceId. Extension MINOR bumped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 11:43:38 -05:00
james.maes 603e518a34 Merge pull request #23 from QRun-IO/fix/outbound-enqueue-table-cache
fix(outbound): record-change listener only costs queries for configured source tables
2026-10-15 07:38:22 -05:00
james.maesandClaude Opus 5.5 d1eb78c1df fix(outbound): the record-change listener only costs queries for configured source tables
OutboundEnqueue applied to every non-edi_ table, and per record opened a
system session and queried edi_agreement_document, so every insert and
update of audit, auditDetail, worm_audit_chain, jobs, alerts and the
rest paid for it (and UPDATEs fetched old records for it).

- OutboundSourceTables (per EdiRuntime) caches the source tables that
  have an active RECORD_CHANGE outbound document on an active agreement,
  loaded from committed data. appliesTo answers from it, so other tables
  do no work at all.
- Changes to edi_agreement / edi_agreement_document invalidate it, and
  again after their transaction commits (a reload that raced the change
  is discarded); until then the changing transaction bypasses the cache
  and sees its own definitions. A 60s TTL covers other processes.
- onRecordsChanged resolves definitions (documents + active agreements)
  once per event, not once per record. MANUAL and ESB_EVENT enqueue keep
  their semantics.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 07:24:33 -05:00
james.maes 2810e00e9c Merge pull request #22 from QRun-IO/fix/producer-version-from-build
fix: qbit metadata version comes from the build
2026-10-15 01:42:56 -05:00
james.maesandClaude Opus 5.5 89a95737a5 fix: the qbit metadata version comes from the build, not a hardcoded constant
The producer read a literal version, so released artifacts reported a stale
or -SNAPSHOT version. qbit-edi.properties (version=${project.version}) is
Maven-filtered and the producer loads it; tests assert producer == bundle ==
artifact version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 01:29:54 -05:00
james.maes 86b8dbf13d Merge pull request #21 from QRun-IO/fix/help-bundle-version-test
test(help): bundle version test compares with the artifact version
2026-10-14 23:18:42 -05:00
james.maesandClaude Opus 5.5 4471a13e78 test(help): compare the help bundle version with the artifact version, not a hardcoded SNAPSHOT
The bundle version is now Maven-filtered, so a literal SNAPSHOT in the test
fails every release build. Surefire passes the artifact version in
qbit.artifactVersion.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 23:07:33 -05:00
james.maes c39d78901c Merge pull request #20 from QRun-IO/fix/help-bundle-version
fix(help): help bundle version comes from the build, not a hardcoded SNAPSHOT
2026-10-14 22:37:20 -05:00
james.maesandClaude Opus 5.5 1e25fe54d2 fix(help): the help bundle carries the artifact version (Maven-filtered) instead of a hardcoded SNAPSHOT
On release branches the bundle's version and the producer's differed, so the
bundle test that asserts they match failed, and released bundles recorded
SNAPSHOT as their provenance. help/*.yaml is now filtered:
version: ${project.version}.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 22:09:48 -05:00
james.maes 0fe99db48d Merge pull request #19 from QRun-IO/feature/help-bundle
feat(help): qbit-edi help bundle via QQQ help bundles, with coverage gate
2026-10-14 20:45:30 -05:00
james.maesandClaude Opus 5.5 b08057180b feat(help): ship the qbit-edi help bundle through QQQ help bundles, with coverage gate
Host-neutral help (deployment-specific facts removed), registered with
HelpBundles, gated by HelpBundleAssertions. QQQ 4.1.0-RC.7.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 20:27:22 -05:00
james.maes 79b85a163d Merge pull request #18 from QRun-IO/feature/tables-populated-defaults
feat: every edi table explains itself; Setup status; Install sample maps
2026-10-14 17:42:45 -05:00
james.maesandClaude Opus 5.5 4e32aebc00 fix: X12 routing honors identity isActive; tests for draft maps and host help
- X12 inbound routing and outbound building ignore inactive identities, as
  EDIFACT does (UPGRADE note).
- Test that a draft map bound to an outbound document is never used (it
  already was not) and that inactive identities stop sending.
- hostHelpIsKept test; drop the unfounded concurrency claim on Install
  sample maps; plain wording in the help bundle header.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:32:54 -05:00
james.maesandClaude Opus 5.5 7bafab62cc docs: upgrade note wording
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:17:54 -05:00
james.maesandClaude Opus 5.5 a1914f7553 feat: table help, Setup status, Install sample maps, tables guardrail
Every edi_* table and the app get help bundle text saying how rows get
created and the setup order. The home gets a Setup status section. A new
Install sample maps table action adds the bundled synthetic maps as
inactive, unbound drafts (insert-only by name).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:17:41 -05:00
james.maes 51a5eaef74 Merge pull request #17 from QRun-IO/fix/repeat-schedule-time-zone
fix: no cron time zone on repeat-seconds worker schedules
2026-10-14 15:10:02 -05:00
39 changed files with 4587 additions and 170 deletions

No files matched your search

+11 -57
View File
@@ -1,60 +1,14 @@
version: 2.1
setup: true
orbs:
qqq-orb: kingsrook/qqq-orb@0.6.9
commands:
configure-package-install:
steps:
- run:
name: Keep package installation unattended
command: |
# These build machines are discarded after the job.
sudo install -d /etc/needrestart/conf.d
printf '%s\n' '$nrconf{restart} = "l";' | sudo tee /etc/needrestart/conf.d/99-qqq-ci.conf >/dev/null
qqq-orb: kingsrook/qqq-orb@0.7.12
workflows:
test_only:
jobs:
- qqq-orb/mvn_test_only:
pre-steps:
- configure-package-install
context: [qqq-maven-registry-credentials]
filters:
branches:
ignore: /(develop|release\/.*)/
publish_snapshot:
jobs:
- qqq-orb/mvn_publish:
pre-steps:
- configure-package-install
context: [qqq-maven-registry-credentials]
branch_type: snapshot
filters:
branches:
only: [develop]
publish_release_candidate:
jobs:
- qqq-orb/mvn_publish:
pre-steps:
- configure-package-install
context: [qqq-maven-registry-credentials]
branch_type: release_candidate
filters:
branches:
only: [/release\/.*/]
publish_release:
jobs:
- qqq-orb/mvn_publish:
pre-steps:
- configure-package-install
context: [qqq-maven-registry-credentials]
branch_type: release
filters:
branches:
ignore: /.*/
tags:
only: [/^v\d+\.\d+\.\d+$/]
setup:
jobs:
- qqq-orb/generate_pipeline:
orb_version: 0.7.12
filters:
branches:
ignore: automation/qbit-rc-pending
tags:
only: /^v\d+\.\d+\.\d+$/
+23
View File
@@ -0,0 +1,23 @@
name: CodeQL
'on':
push:
branches:
- main
- develop
- release/**
- hotfix/**
pull_request: {}
schedule:
- cron: 41 16 * * 0
workflow_dispatch: {}
permissions:
contents: read
actions: read
security-events: write
packages: read
concurrency:
group: codeql-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs:
analyze:
uses: QRun-IO/qqq-orb/.github/workflows/qbit-codeql.yml@c288a8864fe9054f68410e1d78f5863aea4c4d22
+62 -13
View File
@@ -1,25 +1,74 @@
# Cut the next release candidate (QRun-IO/qqq docs/QRUN-QBIT-REPO-STANDARD.md, C6):
# run this workflow by hand, or merge a develop PR labelled `release:rc`. It merges develop into
# the release line and the release/* push runs CircleCI publish_release_candidate.
name: Release candidate
on:
'on':
workflow_dispatch:
inputs:
dry_run:
description: Print the plan without merging or creating branches
description: Plan a cut without changing release state
type: boolean
default: false
finalize:
description: Retry finalization of an existing cut; do not cut another RC
type: boolean
default: false
recover:
description: Recover the same published RC after a deployment timeout; never
reupload
type: boolean
default: false
circle_build:
description: Original Circle publisher build number (recovery only)
type: string
default: ''
merge_sha:
description: Existing cut merge_sha (finalize retry only)
type: string
default: ''
bump_sha:
description: Existing cut bump_sha (finalize retry only)
type: string
default: ''
branch:
description: Existing cut branch (finalize retry only)
type: string
default: ''
version:
description: Existing cut version (finalize retry only)
type: string
default: ''
pull_request:
types: [closed]
branches: [develop]
types:
- closed
branches:
- develop
repository_dispatch:
types:
- qbit-rc-published
permissions:
contents: write
actions: read
checks: read
jobs:
release-rc:
if: github.event_name == 'workflow_dispatch' || (github.event.pull_request.merged == true && contains(github.event.pull_request.labels.*.name, 'release:rc'))
uses: QRun-IO/.github/.github/workflows/qbit-release-rc.yml@main
if: github.event_name == 'workflow_dispatch' || github.event_name == 'repository_dispatch'
|| (github.event_name == 'pull_request' && github.event.pull_request.merged
== true && github.event.pull_request.base.ref == 'develop' && contains(github.event.pull_request.labels.*.name,
'release:rc'))
uses: QRun-IO/qqq-orb/.github/workflows/qbit-release-rc.yml@cc30f599e63c8e41629c99d23395bac1adb8bc44
with:
dry_run: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run }}
dry_run: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run || false
}}
finalize: ${{ github.event_name == 'repository_dispatch' || (github.event_name
== 'workflow_dispatch' && inputs.finalize) }}
recover: ${{ github.event_name == 'workflow_dispatch' && inputs.recover || false
}}
circle_build: ${{ github.event_name == 'workflow_dispatch' && inputs.circle_build
|| '' }}
merge_sha: ${{ github.event_name == 'repository_dispatch' && github.event.client_payload.merge_sha
|| inputs.merge_sha || '' }}
bump_sha: ${{ github.event_name == 'repository_dispatch' && github.event.client_payload.bump_sha
|| inputs.bump_sha || '' }}
branch: ${{ github.event_name == 'repository_dispatch' && github.event.client_payload.branch
|| inputs.branch || '' }}
version: ${{ github.event_name == 'repository_dispatch' && github.event.client_payload.version
|| inputs.version || '' }}
secrets: inherit
+16
View File
@@ -0,0 +1,16 @@
profile: qbit
orb_version: 0.7.12
default_branch: develop
java_version: '21'
browsers: true
api_checks: false
docs_only: false
release_on_main: false
release_tag_policy: stable
test_branch_policy: non-release
publish:
snapshot: true
release_candidate: true
release: true
asciidoc: false
release_completion: true
+17
View File
@@ -4,8 +4,25 @@ Following [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and [Semantic
## [Unreleased]
### Changed
- EDI home is business-only: System, Dead letters, Certificates expiring and Setup status move to `EdiHome.ADMIN_WIDGET_NAMES` for the host's Admin home (gated to `edi.admin.hasAccess`, full width); every home widget now requires `edi.hasAccess` and is hidden without it. A new "Inbound documents are delayed" alert links to the delayed inbound documents once the oldest is older than `inboundDelayedAfterMinutes` (default 60). Setup counts that fail show "Unavailable" and are logged instead of -1.
- UI API extension edi 1.1.0 (requires QQQ 4.1.0-RC.15, UI API 1.6.0): operationIds, standard error responses with BasicErrorResponseV1, extension tags, and retained deprecated EdiErrorV1 schema.
- Derive qbit metadata version from the Maven artifact version, keeping release metadata and help bundle versions in sync.
- Require QQQ 4.1.0-RC.7 for core help-bundle registration and coverage validation.
- X12 routing and sending honor identity `isActive`, matching EDIFACT.
### Added
- Complete, host-neutral `help/qbit-edi.yaml` bundle registered with QQQ core: app, tables, fields, actions, widgets and possible-value sources, with a coverage gate and HOST overrides. Table/app fallback remains available without `helpContent`.
- Setup status section on the EDI home (partner, identities, agreement, agreement documents, transport, certificates, guides, maps) with the next step; it counts only the active rows the runtime uses.
- "Install sample maps" action on Maps: standard streamed ETL, insert-only by name, installs the 32 bundled synthetic maps as draft, unbound maps bound to the built-in guide.
- `EdiTablesGuardrailTest`: every table is used at runtime, seeded or listed as starting empty, and has help.
- Phase f1 `edi` 1.0.0 spec'd UI API: guide tree, map target/transform metadata, encrypted base64 sample preview with unsaved rule overrides, and per-rule validation under `/qqq/v1/qbit/edi`.
- Versioned `Edi...V1` schemas and generated OpenAPI fragment with the RC.6 contract/compatibility checks; real HTTP strict-mode permission, PHI, upload, isolation and validation tests.
- Shared sandbox/record-action preview service, current/original guide selection, caller-session transforms, structured revision/expected diffs and value-free PHI diagnostics.
+57 -9
View File
@@ -34,7 +34,7 @@ ALTER TABLE edi_agreement ADD COLUMN syntax_identifier VARCHAR(255);
## Requirements and installation
Java 21, QQQ **4.1.0-RC.6+**, an RDBMS backend, and `com.kingsrook.qbits:qbit-edi:0.1.0-SNAPSHOT`.
Java 21, QQQ **4.1.0-RC.7+**, an RDBMS backend, and `com.kingsrook.qbits:qbit-edi:0.1.0-SNAPSHOT`.
Apply [the PostgreSQL schema](docs/ddl/edi-postgres.sql). Targets and their associations must
share the EDI backend so each transaction set can commit or roll back atomically.
@@ -60,6 +60,23 @@ The `edi` app has Activity/Setup sections, typed home widgets, record screens an
Review/Result actions. Main tables have RC.6 search-field metadata; `EdiIntegrations.mainTables()`
also exposes the list for host search registration.
## In-app help
The producer registers [help/qbit-edi.yaml](src/main/resources/help/qbit-edi.yaml), a `QBIT`
bundle versioned with this artifact. It covers the EDI app, tables, fields, actions, widgets,
and possible-value sources. Live `{{...}}` references follow the current metadata.
Hosts can register their own `layer: HOST` bundle with `HelpBundles.register(qInstance,
"help/my-host.yaml")`, using the same keys and roles to override selected entries. Enable
core's `helpContent` provider with provenance fields, then call
`QInstanceHelpContentManager.loadHelpContent(qInstance)` and `HelpBundles.syncOnStartup(qInstance)`
after defining metadata. Core preserves user edits during sync. If you configure a different
EDI app name, use that name in your host's app help key.
Registration is safe without `helpContent`; those hosts retain the table/app help fallback.
`EdiHelpBundleTest` runs the core `HelpBundleAssertions` coverage gate and checks widget and
possible-value-source coverage during `./mvnw verify`, without host-specific exclusions.
## Tables
There are **26 persisted tables**, whose SQL names equal their QQQ names.
@@ -78,8 +95,8 @@ transports/guides and local certificates/identities.
## UI API
With optional Javalin present, the producer registers the `edi` extension, version `1.0.0`,
under `/qqq/v1/qbit/edi`. All routes are endpoint specs and pass strict UI-route auditing.
With optional Javalin present, the producer registers the `edi` extension, version `1.1.0`,
under `/qqq/v1/qbit/edi`, requiring QQQ 4.1.0-RC.15 (UI API 1.6.0). All routes are endpoint specs and pass strict UI-route auditing.
| Method | Path after `/qqq/v1/qbit/edi` | Result |
|---|---|---|
@@ -116,15 +133,15 @@ The mapper saves `edi_map_rule` through the **core table API**, using its normal
`edi.map_rule.insert|edit|delete` permissions and record locks. There are no custom rule-write routes.
The Next mapper page is separate phase-f work.
The generated contract is `src/main/resources/openapi/qqq-v1/edi.yaml`; all component names are
`Edi...V1`. Regenerate intentionally with:
The generated contract is `src/main/resources/openapi/qqq-v1/edi.yaml`; extension component names are
`Edi...V1`, alongside core `BasicErrorResponseV1`. `EdiErrorV1` remains deprecated for compatibility. Regenerate with the locally installed migration build:
```powershell
./mvnw.cmd -B '-Dtest=EdiApiContractTest' '-Dedi.openapi.update=true' test
./mvnw.cmd -o -B '-Dqqq.snapshot.version=4.1.99-SNAPSHOT' '-Dtest=EdiApiContractTest' '-Dedi.openapi.update=true' test
```
RC.6 provides `UiApiContract.check`, used to compare generated and checked-in YAML and run
`OpenApiCompatibilityChecker` against this initial fragment baseline. The extension also appears
Core provides `UiApiContract.check`, used to compare generated and checked-in YAML, validate
operation IDs and references, and run `OpenApiCompatibilityChecker` against the committed fragment baseline. The extension also appears
in the host's `/qqq/v1/openapi.yaml` and `/qqq/v1/version`.
## Inbound operation
@@ -167,6 +184,30 @@ synthetic EDI on the standard Review step. Browser uploads require the host's RC
staging. The alias encrypts streams and persists samples; this process accepts no filesystem archive.
Samples must contain synthetic data only.
## Setup and starter data
No Liquibase changelog or host include is needed: the qbit seeds its own reference data. When the
host calls `EdiRuntime.initialize()` at startup (after the schema exists) the 41 built-in guides
load, with their segments, elements, rules, code lists and code values. It only changes a guide when
the bundled content changes, so edits to your own rows are kept.
Everything else is host or runtime data, and every table explains itself when empty (help bundle
`help/qbit-edi.yaml`). Set up in this order:
1. Partner.
2. Identities: one of yours (no partner) and one per partner.
3. Agreement: pairs your identity with the partner's, for a standard and test or production.
4. Agreement documents: one per transaction set and direction, with the guide and optionally a map.
5. Transport (AS2, SFTP, HTTPS), linked on the agreement. Certificates only for AS2 signing and encryption.
The **Setup status** widget (`edi_home_setup`, for the host's Admin home) shows each step, whether
it is done (counting only the active rows the runtime uses) and the next step. A count that fails
shows "Unavailable" and is logged.
Maps start empty. The **Install sample maps** action on Maps adds the 32 bundled synthetic examples
as draft maps with no partner and no agreement document, by name and never over an existing map. Point
a map at your own table, preview it, activate it, then select it on an agreement document.
## Permissions
Full table permissions include `edi.partner.read|insert|edit|delete`, `edi.map.*`,
@@ -189,7 +230,14 @@ Full table permissions include `edi.partner.read|insert|edit|delete`, `edi.map.*
| Scheduled workers / ESB enqueue | `edi.admin` |
Partner locks use `edi_partner` with `edi.admin` all-access. Home queries explicitly enforce
the viewing user's permissions. Processes are table/record actions, never navigation entries.
the viewing user's permissions.
The EDI home shows business work only. Its widgets (`EdiHome.HOME_WIDGET_NAMES`) need
`edi.hasAccess` and are hidden without it. "Inbound documents delayed" appears only when the oldest
unprocessed inbound document is older than `inboundDelayedAfterMinutes` (default 60), with a link to
those documents. System widgets (`EdiHome.ADMIN_WIDGET_NAMES`: System, Dead letters, Certificates
expiring, Setup status) are not on the EDI home; they need `edi.admin.hasAccess`, are hidden without
it, and a host places them on its Admin home. Processes are table/record actions, never navigation entries.
## Configuration
+11
View File
@@ -0,0 +1,11 @@
# Upgrade notes
## Unreleased
- No schema change and no Liquibase include.
- Behavior change: an inactive identity (`isActive` false) now stops X12 routing and sending, as it already did for EDIFACT. Inbound documents from or to it are rejected as an unknown partner, and outbound requests for its agreements wait until it is active. Identities default to active, so only identities someone switched off are affected.
- Hosts that already call `EdiRuntime.initialize()` at startup keep getting the 41 built-in guides and code lists. A host that never called it will now see "Needs setup" for Guides and code lists on the home.
- New widget "Setup status" (`edi_home_setup`).
- Behavior change: the EDI home shows business widgets only (`EdiHome.HOME_WIDGET_NAMES`), each now requiring `edi.hasAccess` (hidden without it). `edi_home_system`, `edi_home_dead`, `edi_home_certificates` and `edi_home_setup` left the EDI home: they are listed in `EdiHome.ADMIN_WIDGET_NAMES`, require `edi.admin.hasAccess` (hidden without it), and the host places them on its Admin home.
- New business widget `edi_home_delayed` ("Inbound documents are delayed"), shown only when the oldest unprocessed inbound document is older than the new config value `inboundDelayedAfterMinutes` (default 60).
- New table action "Install sample maps" on Maps, permission `edi.admin`. It is never run automatically and only adds draft maps whose name does not exist yet.
+30 -1
View File
@@ -126,7 +126,36 @@
</dependencies>
<build>
<resources>
<!-- help bundles carry the artifact version (QQQ help bundle provenance) -->
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
<includes>
<include>help/*.yaml</include>
<include>qbit-edi.properties</include>
</includes>
</resource>
<resource>
<directory>src/main/resources</directory>
<filtering>false</filtering>
<excludes>
<exclude>help/*.yaml</exclude>
<exclude>qbit-edi.properties</exclude>
</excludes>
</resource>
</resources>
<plugins>
<!-- the artifact version, for the help bundle version test -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<systemPropertyVariables combine.children="append">
<qbit.artifactVersion>${project.version}</qbit.artifactVersion>
</systemPropertyVariables>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-checkstyle-plugin</artifactId>
@@ -213,7 +242,7 @@
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<qqq.snapshot.version>4.1.0-RC.6</qqq.snapshot.version>
<qqq.snapshot.version>4.1.0-RC.15</qqq.snapshot.version>
</properties>
<dependencyManagement>
<dependencies>
@@ -89,6 +89,7 @@ public class EdiQBitConfig implements QBitConfig
private String appLabel = "EDI";
private String parentAppName;
private String publicBaseUrl;
private Integer inboundDelayedAfterMinutes = 60;
@@ -114,6 +115,10 @@ public class EdiQBitConfig implements QBitConfig
{
errors.add("secretsResolver and partnerLocks are required");
}
if(inboundDelayedAfterMinutes == null || inboundDelayedAfterMinutes < 1)
{
errors.add("inboundDelayedAfterMinutes must be at least 1");
}
if(appName == null || appName.isBlank())
{
errors.add("appName is required");
@@ -390,4 +395,22 @@ public class EdiQBitConfig implements QBitConfig
publicBaseUrl = value;
return (this);
}
/*******************************************************************************
** How old (in minutes) the oldest unprocessed inbound document may get before
** the EDI home says "Inbound documents are delayed". Default 60.
*******************************************************************************/
public Integer getInboundDelayedAfterMinutes()
{
return (inboundDelayedAfterMinutes);
}
/*******************************************************************************
** Set the inbound delay threshold in minutes.
*******************************************************************************/
public EdiQBitConfig withInboundDelayedAfterMinutes(Integer value)
{
inboundDelayedAfterMinutes = value;
return (this);
}
}
@@ -21,9 +21,13 @@
package com.kingsrook.qbits.edi;
import java.io.IOException;
import java.io.InputStream;
import java.util.Properties;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerMultiOutput;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.help.bundles.HelpBundles;
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitMetaDataProducer;
@@ -36,13 +40,41 @@ public class EdiQBitProducer implements QBitMetaDataProducer<EdiQBitConfig>
{
public static final String GROUP_ID = "com.kingsrook.qbits";
public static final String ARTIFACT_ID = "qbit-edi";
public static final String VERSION = "0.1.0";
public static final String VERSION = loadVersion();
private EdiQBitConfig qBitConfig;
private String namespace;
/*******************************************************************************
** Read the artifact version from the Maven-filtered resource.
*******************************************************************************/
private static String loadVersion()
{
try(InputStream inputStream = EdiQBitProducer.class.getClassLoader().getResourceAsStream("qbit-edi.properties"))
{
if(inputStream == null)
{
throw new IllegalStateException("Missing qbit-edi.properties; build with Maven to supply the artifact version");
}
Properties properties = new Properties();
properties.load(inputStream);
String version = properties.getProperty("version");
if(version == null || version.isBlank() || version.contains("${"))
{
throw new IllegalStateException("Missing or unfiltered version in qbit-edi.properties; build with Maven");
}
return (version);
}
catch(IOException e)
{
throw new IllegalStateException("Could not read the artifact version from qbit-edi.properties", e);
}
}
/*******************************************************************************
** Fluent configuration setter.
*******************************************************************************/
@@ -130,6 +162,16 @@ public class EdiQBitProducer implements QBitMetaDataProducer<EdiQBitConfig>
MetaDataProducerMultiOutput output = QBitMetaDataProducer.super.produce(instance);
output.addSelfToInstance(instance);
EdiRuntime.install(instance, qBitConfig);
// Registration is metadata-only; core startup sync skips hosts without helpContent.
HelpBundles.register(instance, "help/qbit-edi.yaml");
// The app name is configurable; its bundle key must follow the installed app.
HelpBundles.of(instance).getBundles().getLast().getEntries().stream()
.filter(entry -> "app:edi;slot:help".equals(entry.getKey()))
.forEach(entry -> entry.setKey("app:" + qBitConfig.getAppName() + ";slot:help"));
if(instance.getTable("helpContent") == null)
{
com.kingsrook.qbits.edi.metadata.EdiHelp.apply(instance, qBitConfig.getAppName());
}
instance.withSupplementalMetaData(new com.kingsrook.qbits.edi.metadata.EdiRecordLabels());
instance.withRecordChangeListener(new com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference(com.kingsrook.qbits.edi.flow.OutboundEnqueue.class));
instance.withSupplementalMetaData(new com.kingsrook.qbits.edi.transport.EdiTransportPluginsMetaData()
@@ -64,6 +64,7 @@ public final class EdiRuntime
private final Ack999Builder ack999 = new Ack999Builder();
private final TA1Builder ta1 = new TA1Builder();
private final InboundAckReader acknowledgements = new InboundAckReader();
private final com.kingsrook.qbits.edi.flow.OutboundSourceTables outboundSources = new com.kingsrook.qbits.edi.flow.OutboundSourceTables();
/*******************************************************************************
** Construct services without resolving keys until content is stored or opened.
@@ -95,6 +96,22 @@ public final class EdiRuntime
return (Objects.requireNonNull(INSTANCES.get(QContext.getQInstance()), "EDI runtime has not been installed"));
}
/*******************************************************************************
** Services for the current QQQ context, or null when none are installed.
*******************************************************************************/
public static EdiRuntime find()
{
return (QContext.getQInstance() == null ? null : INSTANCES.get(QContext.getQInstance()));
}
/*******************************************************************************
** Cached source tables that have active RECORD_CHANGE outbound definitions.
*******************************************************************************/
public com.kingsrook.qbits.edi.flow.OutboundSourceTables outboundSources()
{
return (outboundSources);
}
/*******************************************************************************
** Call at host startup after DDL, with the service session established.
*******************************************************************************/
@@ -36,6 +36,7 @@ import com.kingsrook.qqq.middleware.javalin.specs.BasicResponse;
import com.kingsrook.qqq.middleware.javalin.specs.extensions.AbstractJsonEndpointSpec;
import com.kingsrook.qqq.middleware.javalin.specs.extensions.MiddlewareSpecExtension;
import com.kingsrook.qqq.middleware.javalin.specs.extensions.MiddlewareSpecExtensionsMetaData;
import com.kingsrook.qqq.middleware.javalin.specs.v1.responses.BasicErrorResponseV1;
import com.kingsrook.qqq.openapi.model.HttpMethod;
import com.kingsrook.qqq.openapi.model.In;
import com.kingsrook.qqq.openapi.model.Parameter;
@@ -44,7 +45,7 @@ import com.kingsrook.qqq.openapi.model.Type;
import io.javalin.http.HttpStatus;
/*******************************************************************************
** The EDI 1.0.0 UI API: four spec'd operations in the default qbit namespace.
** The EDI 1.1.0 UI API: four spec'd operations in the default qbit namespace.
*******************************************************************************/
public final class EdiApi
{
@@ -68,7 +69,7 @@ public final class EdiApi
*******************************************************************************/
public static MiddlewareSpecExtension extension()
{
return (new MiddlewareSpecExtension().withName("edi").withSemanticVersion("1.0.0")
return (new MiddlewareSpecExtension().withName("edi").withSemanticVersion("1.1.0")
.withDescription("EDI sandbox and mapper, using the signed-in user's permissions.")
.withSpec(GuideTreeSpec.class).withSpec(MapTargetSpec.class).withSpec(PreviewSpec.class).withSpec(ValidateSpec.class));
}
@@ -148,11 +149,11 @@ public final class EdiApi
@Override
public List<BasicResponse> defineAdditionalBasicResponses()
{
return (List.of(new BasicResponse(HttpStatus.BAD_REQUEST, "Invalid request or sample exceeds 5 MiB", "EdiErrorV1"),
new BasicResponse(HttpStatus.UNAUTHORIZED, "Sign in required", "EdiErrorV1"),
new BasicResponse(HttpStatus.FORBIDDEN, "Permission denied", "EdiErrorV1"),
new BasicResponse(HttpStatus.NOT_FOUND, "Missing or invisible record", "EdiErrorV1"),
new BasicResponse(HttpStatus.CONTENT_TOO_LARGE, "Host HTTP request size limit exceeded", "EdiErrorV1")));
return (List.of(new BasicResponse(HttpStatus.BAD_REQUEST, "Invalid request or sample exceeds 5 MiB", BasicErrorResponseV1.class.getSimpleName()),
new BasicResponse(HttpStatus.UNAUTHORIZED, "Sign in required", BasicErrorResponseV1.class.getSimpleName()),
new BasicResponse(HttpStatus.FORBIDDEN, "Permission denied", BasicErrorResponseV1.class.getSimpleName()),
new BasicResponse(HttpStatus.NOT_FOUND, "Missing or invisible record", BasicErrorResponseV1.class.getSimpleName()),
new BasicResponse(HttpStatus.CONTENT_TOO_LARGE, "Host HTTP request size limit exceeded", BasicErrorResponseV1.class.getSimpleName())));
}
/*******************************************************************************
@@ -489,7 +489,7 @@ public final class EdiSchemas
@Override
public Schema toSchema()
{
return (new Schema().withType(Type.OBJECT).withDescription("Core middleware error body.")
return (new Schema().withType(Type.OBJECT).withDescription("Deprecated since 1.1.0: errors use BasicErrorResponseV1.")
.withProperties(new TreeMap<>(Map.ofEntries(
Map.entry("error", new Schema().withType(Type.STRING).withNullable(true))))));
}
@@ -40,6 +40,8 @@ import com.kingsrook.qqq.backend.core.actions.permissions.PermissionEnforcement;
import com.kingsrook.qqq.backend.core.actions.tables.CountAction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.tables.count.CountInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
@@ -47,6 +49,7 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.AlertData;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.StatisticsData;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.TableData;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
@@ -56,9 +59,11 @@ import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppSection;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.DenyBehavior;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.PermissionLevel;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRules;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
/*******************************************************************************
** Typed EDI home. Every data query explicitly enforces the viewing user's access.
@@ -67,9 +72,31 @@ public class EdiHome extends AbstractWidgetRenderer
{
public static final List<String> ACTIVITY = List.of("edi_transaction", "edi_interchange", "edi_exception");
public static final List<String> SETUP = List.of("edi_partner", "edi_agreement", "edi_transport", "edi_certificate", "edi_guide", "edi_code_list", "edi_map");
private static final Map<String, String> LABELS = new LinkedHashMap<>();
/*******************************************************************************
** System-audience widgets (component and queue health, dead letters,
** certificate expiry, setup and configuration). They are not on the EDI
** business home: a host places them on its Admin home. Each is gated to
** ADMIN_PERMISSION_BASE_NAME and hidden from everyone else.
*******************************************************************************/
public static final List<String> ADMIN_WIDGET_NAMES = List.of("edi_home_system", "edi_home_dead", "edi_home_certificates", "edi_home_setup");
/*******************************************************************************
** Business widgets on the EDI home, in order. Each is gated to the EDI app
** permission (edi.hasAccess) and hidden from everyone else.
*******************************************************************************/
public static final List<String> HOME_WIDGET_NAMES = List.of("edi_home_delayed", "edi_home_in_today", "edi_home_out_today", "edi_home_exceptions",
"edi_home_overdue", "edi_home_queue", "edi_home_acks", "edi_home_errors", "edi_home_volume");
/*******************************************************************************
** The EDI administrator permission (edi.admin.hasAccess), as on the admin processes.
*******************************************************************************/
public static final String ADMIN_PERMISSION_BASE_NAME = EdiQBitConfig.PERMISSION_BASE_NAME + ".admin";
private static final QLogger LOG = QLogger.getLogger(EdiHome.class);
private static final Map<String, String> LABELS = new LinkedHashMap<>();
private static final Map<String, String> DOCUMENT_NAMES = new HashMap<>();
private static final List<String> TABLES = List.of("queue", "acks", "errors", "volume", "system");
private static final List<String> TABLES = List.of("setup", "queue", "acks", "errors", "volume", "system");
static
{
@@ -79,12 +106,14 @@ public class EdiHome extends AbstractWidgetRenderer
Map.entry("835", "Remittance"), Map.entry("270", "Eligibility Inquiry"), Map.entry("271", "Eligibility Response"), Map.entry("276", "Claim Status Inquiry"),
Map.entry("277", "Claim Status Response"), Map.entry("834", "Enrollment"), Map.entry("997", "Functional Ack"), Map.entry("999", "Implementation Ack"),
Map.entry("TA1", "Interchange Ack")));
LABELS.put("delayed", "Inbound documents delayed");
LABELS.put("in_today", "Inbound today");
LABELS.put("out_today", "Outbound today");
LABELS.put("exceptions", "Open exceptions");
LABELS.put("overdue", "Overdue acks");
LABELS.put("dead", "Dead letters");
LABELS.put("certificates", "Certificates expiring (30 days)");
LABELS.put("setup", "Setup status");
LABELS.put("queue", "Exception queue");
LABELS.put("acks", "Overdue acknowledgements");
LABELS.put("errors", "Recent errors (24 hours)");
@@ -98,7 +127,7 @@ public class EdiHome extends AbstractWidgetRenderer
public static QAppMetaData app(EdiQBitConfig config, List<QTableMetaData> tables)
{
QAppMetaData app = new QAppMetaData().withName(config.getAppName()).withLabel(config.getAppLabel()).withIcon(new QIcon("swap_horiz"))
.withWidgets(LABELS.keySet().stream().map(n -> "edi_home_" + n).toList())
.withWidgets(HOME_WIDGET_NAMES)
.withPermissionRules(new QPermissionRules().withLevel(PermissionLevel.NOT_PROTECTED))
.withSection(new QAppSection().withName("activity").withLabel("Activity").withTables(ACTIVITY))
.withSection(new QAppSection().withName("setup").withLabel("Setup").withTables(SETUP));
@@ -108,15 +137,23 @@ public class EdiHome extends AbstractWidgetRenderer
}
/*******************************************************************************
** RC.5 typed statistics and tables; no raw markup in metadata or widget data.
** RC.5 typed statistics, tables and one alert. Business widgets need the EDI
** app permission; the ADMIN_WIDGET_NAMES need the EDI administrator
** permission. Denied widgets are hidden, never shown as locked shells.
*******************************************************************************/
public static List<QWidgetMetaData> widgets()
{
List<QWidgetMetaData> widgets = new ArrayList<>();
LABELS.forEach((name, label) -> widgets.add(new QWidgetMetaData().withName("edi_home_" + name).withLabel(label)
.withType((TABLES.contains(name) ? WidgetType.TABLE : WidgetType.STATISTICS).getType())
.withGridColumns(TABLES.contains(name) ? 12 : 2).withCodeReference(new QCodeReference(EdiHome.class))
.withPermissionRules(new QPermissionRules().withLevel(PermissionLevel.NOT_PROTECTED))));
LABELS.forEach((name, label) ->
{
String widgetName = "edi_home_" + name;
boolean admin = ADMIN_WIDGET_NAMES.contains(widgetName);
String type = name.equals("delayed") ? WidgetType.ALERT.getType() : (TABLES.contains(name) ? WidgetType.TABLE : WidgetType.STATISTICS).getType();
widgets.add(new QWidgetMetaData().withName(widgetName).withLabel(label).withType(type)
.withGridColumns(TABLES.contains(name) || admin || name.equals("delayed") ? 12 : 2).withCodeReference(new QCodeReference(EdiHome.class))
.withPermissionRules(new QPermissionRules().withPermissionBaseName(admin ? ADMIN_PERMISSION_BASE_NAME : EdiQBitConfig.PERMISSION_BASE_NAME)
.withLevel(PermissionLevel.HAS_ACCESS_PERMISSION).withDenyBehavior(DenyBehavior.HIDDEN)));
});
return (widgets);
}
@@ -136,8 +173,10 @@ public class EdiHome extends AbstractWidgetRenderer
case "exceptions" -> count("edi_exception", openExceptions());
case "overdue" -> count("edi_functional_group", overdueAcks(now));
case "dead" -> count("edi_exception", openExceptions().withCriteria("type", QCriteriaOperator.IN, "DEAD_LETTER", "TRANSPORT"));
case "delayed" -> delayed(now);
case "certificates" -> count("edi_certificate", new QQueryFilter().withCriteria("notAfter", QCriteriaOperator.LESS_THAN_OR_EQUALS, now.plus(30, ChronoUnit.DAYS))
.withCriteria("status", QCriteriaOperator.EQUALS, "ACTIVE"));
case "setup" -> setup();
case "queue" -> queue(now);
case "acks" -> acks(now);
case "errors" -> errors(now);
@@ -191,6 +230,95 @@ public class EdiHome extends AbstractWidgetRenderer
/*******************************************************************************
** The setup steps in order, with their status and the next step. Counts are
** taken as the viewer; a table the viewer cannot read shows as not visible.
*******************************************************************************/
private static RenderWidgetOutput setup() throws QException
{
QQueryFilter active = new QQueryFilter().withCriteria("isActive", QCriteriaOperator.EQUALS, true);
EdiSetupStatus.Counts counts = new EdiSetupStatus.Counts(safeCount("edi_partner", new QQueryFilter()),
safeCount("edi_identity", new QQueryFilter().withCriteria("isActive", QCriteriaOperator.EQUALS, true).withCriteria("partnerId", QCriteriaOperator.IS_BLANK)),
safeCount("edi_identity", new QQueryFilter().withCriteria("isActive", QCriteriaOperator.EQUALS, true).withCriteria("partnerId", QCriteriaOperator.IS_NOT_BLANK)),
safeCount("edi_agreement", new QQueryFilter()), safeCount("edi_agreement", active),
safeCount("edi_agreement_document", new QQueryFilter()), safeCount("edi_agreement_document", active),
safeCount("edi_transport", new QQueryFilter()), safeCount("edi_transport", active),
safeCount("edi_certificate", new QQueryFilter()), safeCount("edi_guide", new QQueryFilter()), safeCount("edi_code_list", new QQueryFilter()),
safeCount("edi_map", new QQueryFilter()), safeCount("edi_map", new QQueryFilter().withCriteria("status", QCriteriaOperator.EQUALS, "ACTIVE")));
List<Map<String, Object>> rows = new ArrayList<>();
for(EdiSetupStatus.Item item : EdiSetupStatus.items(counts))
{
rows.add(row("Step", item.step(), "Status", item.state().label() + (item.detail().isEmpty() ? "" : ": " + item.detail()), "Next step", item.nextStep().isEmpty() ? "Nothing to do." : item.nextStep()));
}
return (table(List.of("Step", "Status", "Next step"), rows, "edi_partner", new QQueryFilter(), "Open partners"));
}
/*******************************************************************************
** A count that may fail (for example a table the viewer cannot read): null,
** shown as "Unavailable", and logged; never a made-up number.
*******************************************************************************/
static Integer safeCount(String table, QQueryFilter filter)
{
try
{
return (countOf(table, filter));
}
catch(QException | RuntimeException e)
{
LOG.warn("EDI setup count unavailable", e, logPair("table", table));
return (null);
}
}
/*******************************************************************************
** The business face of a stalled inbound flow: when the oldest unprocessed
** inbound document is older than the configured threshold, a warning with its
** age that links to those documents. Hidden when nothing is delayed, or when
** the viewer cannot read the inbound documents.
*******************************************************************************/
private static RenderWidgetOutput delayed(Instant now) throws QException
{
Integer minutes = EdiRuntime.current().config().getInboundDelayedAfterMinutes();
Instant cutoff = now.minus(minutes, ChronoUnit.MINUTES);
List<QRecord> oldest = List.of();
try
{
oldest = query("edi_inbox", delayedInbound(cutoff).withOrderBy(new QFilterOrderBy("receivedAt", true)).withLimit(1));
}
catch(QPermissionDeniedException e)
{
///////////////////////////////////////////////////////////
// omitted, not locked: the viewer cannot open the inbox //
///////////////////////////////////////////////////////////
}
if(oldest.isEmpty())
{
return (new RenderWidgetOutput(new AlertData().withHideWidget(true)));
}
int count = countOf("edi_inbox", delayedInbound(cutoff));
String age = humanize(Duration.between(oldest.getFirst().getValueInstant("receivedAt"), now));
String html = "<strong>Inbound documents are delayed.</strong> " + count + (count == 1 ? " document has" : " documents have")
+ " waited more than " + humanize(Duration.ofMinutes(minutes)) + " to be processed; the oldest has waited " + age + ". "
+ "<a href=\"" + link("edi_inbox", delayedInbound(cutoff)) + "\">View delayed documents</a>";
return (new RenderWidgetOutput(new AlertData(AlertData.AlertType.WARNING, html)));
}
/*******************************************************************************
** Unprocessed inbound documents received before the cutoff.
*******************************************************************************/
static QQueryFilter delayedInbound(Instant cutoff)
{
return (inboxBacklog().withCriteria("receivedAt", QCriteriaOperator.LESS_THAN, cutoff));
}
/*******************************************************************************
** Open exceptions, oldest first, in four narrow columns.
*******************************************************************************/
@@ -0,0 +1,159 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi.app;
import java.util.ArrayList;
import java.util.List;
/*******************************************************************************
** The Setup status on the EDI home: each setup step in the order it has to be
** done (partner, identity, agreement, document, transport), whether it is
** done, and the next step. The counts use the same conditions the runtime
** uses to route documents (active rows), so the card never reports "set up"
** for rows the runtime ignores.
*******************************************************************************/
public final class EdiSetupStatus
{
/*******************************************************************************
** DONE: working. TODO: the exchange needs it. BUILT_IN: provided by the
** qbit. OPTIONAL: only some setups need it. UNKNOWN: its count failed (for
** example the viewer cannot read the table), shown as "Unavailable".
*******************************************************************************/
public enum State
{
DONE("Set up"), TODO("Needs setup"), BUILT_IN("Built in"), OPTIONAL("Optional"), UNKNOWN("Unavailable");
private final String label;
/*******************************************************************************
**
*******************************************************************************/
State(String label)
{
this.label = label;
}
/*******************************************************************************
** The label shown on the card.
*******************************************************************************/
public String label()
{
return (label);
}
}
/*******************************************************************************
** One line of the card.
*******************************************************************************/
public record Item(String step, String tableName, State state, String detail, String nextStep)
{
}
/*******************************************************************************
** The row counts the card is built from; null means the count failed and the
** step shows as unavailable. Active counts are rows with isActive true (what
** the runtime uses).
*******************************************************************************/
public record Counts(Integer partners, Integer ownIdentities, Integer partnerIdentities, Integer agreements, Integer activeAgreements, Integer agreementDocuments,
Integer activeAgreementDocuments, Integer transports, Integer activeTransports, Integer certificates, Integer guides, Integer codeLists,
Integer maps, Integer activeMaps)
{
}
/*******************************************************************************
** Utility class.
*******************************************************************************/
private EdiSetupStatus()
{
}
/*******************************************************************************
** The items, in setup order.
*******************************************************************************/
public static List<Item> items(Counts c)
{
List<Item> items = new ArrayList<>();
items.add(item("1. Partner", "edi_partner", c.partners(), c.partners(), "partner", "Add the trading partner with New on Partners."));
items.add(identities(c));
items.add(item("3. Agreement", "edi_agreement", c.agreements(), c.activeAgreements(), "agreement",
"Add an agreement on Agreements: your identity, the partner's identity, the standard and test or production. Make sure Active is on."));
items.add(item("4. Agreement documents", "edi_agreement_document", c.agreementDocuments(), c.activeAgreementDocuments(), "agreement document",
"Add one per transaction set and direction on Agreement Documents: pick the guide, and a map if it should become records. A document type with no row is not processed."));
items.add(item("5. Transport", "edi_transport", c.transports(), c.activeTransports(), "transport",
"Add the AS2, SFTP or HTTPS connection on Transports (secrets as variable names), link it on the agreement, then run Test Connection."));
items.add(c.certificates() == null ? new Item("Certificates", "edi_certificate", State.UNKNOWN, "", "")
: c.certificates() == 0
? new Item("Certificates", "edi_certificate", State.OPTIONAL, "None.", "Needed only for AS2 signing and encryption: add yours and the partner's on Certificates.")
: new Item("Certificates", "edi_certificate", State.DONE, c.certificates() + (c.certificates() == 1 ? " certificate." : " certificates."), ""));
items.add(c.guides() == null || c.codeLists() == null ? new Item("Guides and code lists", "edi_guide", State.UNKNOWN, "", "")
: c.guides() == 0 || c.codeLists() == 0
? new Item("Guides and code lists", "edi_guide", State.TODO, "The built-in guides are not loaded.", "The host must call EdiRuntime.initialize() at startup, after the schema exists.")
: new Item("Guides and code lists", "edi_guide", State.BUILT_IN, c.guides() + " guides and " + c.codeLists() + " code lists.", ""));
items.add(c.maps() == null || c.activeMaps() == null ? new Item("Maps", "edi_map", State.UNKNOWN, "", "")
: c.maps() == 0
? new Item("Maps", "edi_map", State.OPTIONAL, "None.", "Needed to turn documents into records. Run Install sample maps on Maps to start from an example.")
: c.activeMaps() == 0
? new Item("Maps", "edi_map", State.TODO, c.maps() + " maps, none active.", "Point a map at your table, preview it, then run Activate Map.")
: new Item("Maps", "edi_map", State.DONE, c.activeMaps() + " active of " + c.maps() + ".", ""));
return (items);
}
/*******************************************************************************
** Step 2: your own identity (no partner) and the partner's.
*******************************************************************************/
private static Item identities(Counts c)
{
String step = "2. Identities";
if(c.ownIdentities() == null || c.partnerIdentities() == null)
{
return (new Item(step, "edi_identity", State.UNKNOWN, "", ""));
}
if(c.ownIdentities() == 0)
{
return (new Item(step, "edi_identity", State.TODO, "None of yours.", "Add your own ISA or UNB qualifier and ID on Identities, with no partner."));
}
if(c.partnerIdentities() == 0)
{
return (new Item(step, "edi_identity", State.TODO, "Yours is set; no partner has one.", "Add the partner's qualifier and ID on Identities, with the partner selected."));
}
return (new Item(step, "edi_identity", State.DONE, c.ownIdentities() + " of yours, " + c.partnerIdentities() + " for partners.", ""));
}
/*******************************************************************************
** A step that needs at least one active row.
*******************************************************************************/
private static Item item(String step, String table, Integer total, Integer active, String noun, String next)
{
if(total == null || active == null)
{
return (new Item(step, table, State.UNKNOWN, "", ""));
}
if(total == 0)
{
return (new Item(step, table, State.TODO, "None.", next));
}
if(active == 0)
{
return (new Item(step, table, State.TODO, total + " " + noun + (total == 1 ? "" : "s") + ", none active.", "Turn Active on for the one to use."));
}
return (new Item(step, table, State.DONE, active + " active.", ""));
}
}
@@ -220,7 +220,8 @@ public class InboundProcessor
{
QRecord sender = row("edi_identity", agreement.getValueInteger("partnerIdentityId"));
QRecord receiver = row("edi_identity", agreement.getValueInteger("localIdentityId"));
if(sender != null && receiver != null && Objects.equals(sender.getValueInteger("partnerId"), agreement.getValueInteger("partnerId"))
if(sender != null && receiver != null && !Boolean.FALSE.equals(sender.getValueBoolean("isActive")) && !Boolean.FALSE.equals(receiver.getValueBoolean("isActive"))
&& Objects.equals(sender.getValueInteger("partnerId"), agreement.getValueInteger("partnerId"))
&& receiver.getValue("partnerId") == null && x12.isa().value(5).equals(sender.getValueString("interchangeQualifier"))
&& x12.isa().value(6).trim().equals(sender.getValueString("interchangeId")) && x12.isa().value(7).equals(receiver.getValueString("interchangeQualifier"))
&& x12.isa().value(8).trim().equals(receiver.getValueString("interchangeId")))
@@ -20,7 +20,9 @@
package com.kingsrook.qbits.edi.flow;
import java.util.ArrayList;
import java.util.List;
import java.util.Set;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.kingsrook.qbits.edi.EdiRuntime;
import com.kingsrook.qbits.edi.storage.EdiRecords;
@@ -30,36 +32,80 @@ import com.kingsrook.qqq.backend.core.actions.tables.listeners.RecordChangeListe
import com.kingsrook.qqq.backend.core.actions.tables.listeners.RecordChangeType;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
/*******************************************************************************
** Transactional source listener and explicit enqueue boundary for MANUAL/ESB.
** The listener applies only to tables with an active RECORD_CHANGE definition
** (cached per runtime, see {@link OutboundSourceTables}), and to the definition
** tables themselves, whose changes invalidate that cache.
*******************************************************************************/
public class OutboundEnqueue implements RecordChangeListenerInterface
{
private static final QLogger LOG = QLogger.getLogger(OutboundEnqueue.class);
/*******************************************************************************
** Ignore internal tables and deletes; definitions are read at event time.
** Definition changes (any type), and inserts/updates of configured sources.
*******************************************************************************/
public boolean appliesTo(String table, RecordChangeType type)
{
return (!table.startsWith("edi_") && type != RecordChangeType.DELETE);
EdiRuntime runtime = EdiRuntime.find();
if(runtime == null)
{
return (false);
}
if(OutboundSourceTables.DEFINITION_TABLES.contains(table))
{
return (true);
}
if(table.startsWith("edi_") || type == RecordChangeType.DELETE)
{
return (false);
}
try
{
return (runtime.outboundSources().contains(table, QContext.getQBackendTransaction()));
}
catch(Exception e)
{
//////////////////////////////////////////////////////////////////////
// if the definitions can't be read, apply - so the event runs its //
// own queries and reports the failure, rather than silently //
// dropping outbound work. //
//////////////////////////////////////////////////////////////////////
LOG.warn("Unable to read EDI outbound source tables", e, logPair("table", table));
return (true);
}
}
/*******************************************************************************
** Use the event transaction, even when the caller did not put it in QContext.
** Definitions are read once per event, not once per record.
*******************************************************************************/
public void onRecordsChanged(RecordChangeEvent event) throws QException
{
if(OutboundSourceTables.DEFINITION_TABLES.contains(event.getTableName()))
{
EdiRuntime.current().outboundSources().invalidate(event.getTransaction());
return;
}
var previous = QContext.getQBackendTransaction();
QContext.setTransaction(event.getTransaction());
try
{
for(QRecord record : event.getRecords())
SystemWrites.call(() ->
{
enqueue(event.getTableName(), record, "RECORD_CHANGE", null);
}
List<QRecord> definitions = definitions(event.getTableName(), OutboundSourceTables.TRIGGER);
for(QRecord record : event.getRecords())
{
enqueue(event.getTableName(), record, null, definitions);
}
return (null);
});
}
finally
{
@@ -72,45 +118,60 @@ public class OutboundEnqueue implements RecordChangeListenerInterface
*******************************************************************************/
public static Integer enqueue(String table, QRecord source, String trigger, String eventId) throws QException
{
return (SystemWrites.call(() ->
return (SystemWrites.call(() -> enqueue(table, source, eventId, definitions(table, trigger))));
}
/*******************************************************************************
** Active outbound documents for the table whose trigger matches (MANUAL
** matches every trigger) and whose agreement is active. Called inside a
** {@link SystemWrites#call}.
*******************************************************************************/
static List<QRecord> definitions(String table, String trigger) throws QException
{
List<QRecord> matching = new ArrayList<>();
for(QRecord document : EdiRecords.query("edi_agreement_document", EdiRecords.equal("sourceTableName", table)
.withCriteria("direction", QCriteriaOperator.EQUALS, "OUTBOUND").withCriteria("isActive", QCriteriaOperator.EQUALS, true)))
{
int count = 0;
var definitions = EdiRecords.query("edi_agreement_document", EdiRecords.equal("sourceTableName", table)
.withCriteria("direction", QCriteriaOperator.EQUALS, "OUTBOUND").withCriteria("isActive", QCriteriaOperator.EQUALS, true));
for(QRecord document : definitions)
if("MANUAL".equals(trigger) || trigger.equals(document.getValueString("outboundTrigger")))
{
if(!"MANUAL".equals(trigger) && !trigger.equals(document.getValueString("outboundTrigger")))
{
continue;
}
QRecord agreement = OutboundWorker.row("edi_agreement", document.getValueInteger("agreementId"));
if(!Boolean.TRUE.equals(agreement.getValueBoolean("isActive")))
{
continue;
}
var metadata = QContext.getQInstance().getTable(table);
if(!metadata.getBackendName().equals(EdiRuntime.current().config().getBackendName()))
{
throw new QException("Outbound source and EDI must share the transactional backend");
}
String filter = document.getValueString("triggerFilter");
if(filter != null && !filter.isBlank() && EdiRecords.query(table, EdiRecords.equal(metadata.getPrimaryKeyField(), source.getValue(metadata.getPrimaryKeyField()))
.withSubFilter(new ObjectMapper().readValue(filter, QQueryFilter.class))).isEmpty())
{
continue;
}
if(eventId != null && !EdiRecords.query("edi_outbound_request", EdiRecords.equal("eventId", eventId)
.withCriteria("agreementDocumentId", QCriteriaOperator.EQUALS, document.getValue("id"))).isEmpty())
{
continue;
}
SystemWrites.insert("edi_outbound_request", new QRecord().withValue("agreementDocumentId", document.getValue("id"))
.withValue("sourceTableName", table).withValue("sourceRecordId", source.getValueString(metadata.getPrimaryKeyField()))
.withValue("eventId", eventId).withValue("status", "PENDING").withValue("attempts", 0)
.withValue("nextAttemptAt", EdiRuntime.current().config().getClock().instant()));
count++;
matching.add(document);
}
return (count);
}));
}
Set<Integer> activeAgreements = OutboundSourceTables.activeAgreementIds(matching);
matching.removeIf(document -> !activeAgreements.contains(document.getValueInteger("agreementId")));
return (matching);
}
/*******************************************************************************
** Enqueue one source record against already-resolved definitions.
*******************************************************************************/
private static int enqueue(String table, QRecord source, String eventId, List<QRecord> definitions) throws Exception
{
int count = 0;
for(QRecord document : definitions)
{
var metadata = QContext.getQInstance().getTable(table);
if(!metadata.getBackendName().equals(EdiRuntime.current().config().getBackendName()))
{
throw new QException("Outbound source and EDI must share the transactional backend");
}
String filter = document.getValueString("triggerFilter");
if(filter != null && !filter.isBlank() && EdiRecords.query(table, EdiRecords.equal(metadata.getPrimaryKeyField(), source.getValue(metadata.getPrimaryKeyField()))
.withSubFilter(new ObjectMapper().readValue(filter, QQueryFilter.class))).isEmpty())
{
continue;
}
if(eventId != null && !EdiRecords.query("edi_outbound_request", EdiRecords.equal("eventId", eventId)
.withCriteria("agreementDocumentId", QCriteriaOperator.EQUALS, document.getValue("id"))).isEmpty())
{
continue;
}
SystemWrites.insert("edi_outbound_request", new QRecord().withValue("agreementDocumentId", document.getValue("id"))
.withValue("sourceTableName", table).withValue("sourceRecordId", source.getValueString(metadata.getPrimaryKeyField()))
.withValue("eventId", eventId).withValue("status", "PENDING").withValue("attempts", 0)
.withValue("nextAttemptAt", EdiRuntime.current().config().getClock().instant()));
count++;
}
return (count);
}
}
@@ -0,0 +1,183 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi.flow;
import java.time.Duration;
import java.util.Collections;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import java.util.WeakHashMap;
import java.util.concurrent.atomic.AtomicLong;
import com.kingsrook.qbits.edi.storage.EdiRecords;
import com.kingsrook.qbits.edi.support.SystemWrites;
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** Per-runtime cache of the source tables that have an active RECORD_CHANGE
** outbound definition on an active agreement. The record-change listener
** consults it so writes to every other table (audits, jobs, alerts...) cost
** nothing. It is loaded from committed data only (never in a writer's
** transaction). A change to an agreement or document invalidates it, and
** again once that change commits (discarding any reload that raced it); until
** then the changing transaction bypasses the cache, so it sees its own
** definitions. A short TTL covers changes made by another process.
*******************************************************************************/
public final class OutboundSourceTables
{
static final String TRIGGER = "RECORD_CHANGE";
static final Set<String> DEFINITION_TABLES = Set.of("edi_agreement", "edi_agreement_document");
private static final Duration TTL = Duration.ofSeconds(60);
private final AtomicLong generation = new AtomicLong();
private final AtomicLong loads = new AtomicLong();
private final Set<QBackendTransaction> changingTransactions = Collections.synchronizedSet(Collections.newSetFromMap(new WeakHashMap<>()));
private volatile Snapshot snapshot;
/*******************************************************************************
** Loaded table names, with the generation and time they were loaded at.
*******************************************************************************/
private record Snapshot(Set<String> tables, long generation, long loadedAtNanos)
{
}
/*******************************************************************************
** Whether record changes to this table, written in this transaction (may be
** null), can enqueue outbound work.
*******************************************************************************/
public boolean contains(String table, QBackendTransaction transaction) throws QException
{
if(transaction != null && changingTransactions.contains(transaction))
{
return (true);
}
Snapshot current = snapshot;
if(current == null || current.generation() != generation.get() || System.nanoTime() - current.loadedAtNanos() > TTL.toNanos())
{
current = load();
}
return (current.tables().contains(table));
}
/*******************************************************************************
** Drop the cache now, and again once the changing transaction commits.
*******************************************************************************/
public void invalidate(QBackendTransaction transaction)
{
generation.incrementAndGet();
if(transaction != null)
{
changingTransactions.add(transaction);
transaction.addAfterCommitCallback(() ->
{
changingTransactions.remove(transaction);
generation.incrementAndGet();
});
}
}
/*******************************************************************************
** How many times the definitions have been loaded (for tests and diagnostics).
*******************************************************************************/
public long loadCount()
{
return (loads.get());
}
/*******************************************************************************
** Read committed active definitions once. A load that raced an invalidation
** is not kept as current, because it carries the older generation.
*******************************************************************************/
private Snapshot load() throws QException
{
long loadGeneration = generation.get();
Set<String> tables = SystemWrites.call(() ->
{
List<QRecord> documents = committed("edi_agreement_document", new QQueryFilter()
.withCriteria("direction", QCriteriaOperator.EQUALS, "OUTBOUND").withCriteria("isActive", QCriteriaOperator.EQUALS, true)
.withCriteria("outboundTrigger", QCriteriaOperator.EQUALS, TRIGGER));
Set<Integer> agreementIds = agreementIds(documents);
Set<Integer> activeAgreements = new HashSet<>();
if(!agreementIds.isEmpty())
{
committed("edi_agreement", new QQueryFilter().withCriteria("isActive", QCriteriaOperator.EQUALS, true)
.withCriteria("id", QCriteriaOperator.IN, agreementIds)).forEach(agreement -> activeAgreements.add(agreement.getValueInteger("id")));
}
Set<String> result = new HashSet<>();
for(QRecord document : documents)
{
if(document.getValueString("sourceTableName") != null && activeAgreements.contains(document.getValueInteger("agreementId")))
{
result.add(document.getValueString("sourceTableName"));
}
}
return (Set.copyOf(result));
});
loads.incrementAndGet();
Snapshot loaded = new Snapshot(tables, loadGeneration, System.nanoTime());
snapshot = loaded;
return (loaded);
}
/*******************************************************************************
** Query committed rows - deliberately without the caller's transaction.
*******************************************************************************/
private static List<QRecord> committed(String table, QQueryFilter filter) throws QException
{
return (new QueryAction().execute(new QueryInput(table).withFilter(filter)).getRecords());
}
/*******************************************************************************
** The distinct non-null agreement ids the documents reference.
*******************************************************************************/
private static Set<Integer> agreementIds(List<QRecord> documents)
{
Set<Integer> ids = new HashSet<>();
documents.forEach(document -> ids.add(document.getValueInteger("agreementId")));
ids.remove(null);
return (ids);
}
/*******************************************************************************
** Active agreement ids among those the documents reference, in one query in
** the current transaction.
*******************************************************************************/
static Set<Integer> activeAgreementIds(List<QRecord> documents) throws QException
{
Set<Integer> ids = agreementIds(documents);
Set<Integer> active = new HashSet<>();
if(!ids.isEmpty())
{
for(QRecord agreement : EdiRecords.query("edi_agreement", EdiRecords.equal("isActive", true).withCriteria("id", QCriteriaOperator.IN, ids)))
{
active.add(agreement.getValueInteger("id"));
}
}
return (active);
}
}
@@ -87,6 +87,15 @@ public class OutboundWorker
{
return (0);
}
QRecord localIdentity = row("edi_identity", agreement.getValueInteger("localIdentityId"));
QRecord partnerIdentity = row("edi_identity", agreement.getValueInteger("partnerIdentityId"));
if(localIdentity == null || partnerIdentity == null || Boolean.FALSE.equals(localIdentity.getValueBoolean("isActive")) || Boolean.FALSE.equals(partnerIdentity.getValueBoolean("isActive")))
{
///////////////////////////////////////////////////////////////////////////
// an inactive identity stops sending, as it stops routing; requests wait //
///////////////////////////////////////////////////////////////////////////
return (0);
}
var requests = EdiRecords.query("edi_outbound_request", EdiRecords.equal("agreementDocumentId", id).withCriteria("status", QCriteriaOperator.EQUALS, "PENDING").withLimit(100));
if(requests.isEmpty())
{
@@ -0,0 +1,157 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi.map;
import java.io.IOException;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.kingsrook.qbits.edi.storage.EdiRecords;
import com.kingsrook.qbits.edi.support.SystemWrites;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** The bundled sample maps (synthetic, one per built-in document and version)
** and their insert-only installation. Installed maps are drafts, bound to no
** partner and to no agreement document, so installing one never changes how
** any document is processed; the target table in each is an example name the
** host replaces before activating.
*******************************************************************************/
public final class SampleMaps
{
public static final List<String> RESOURCES = List.of("204-004010", "204-005010", "204-out-004010", "204-out-005010", "214-004010", "214-005010",
"810-004010", "810-005010", "846-004010", "846-005010", "850-004010", "850-005010", "855-004010", "855-005010", "856-004010", "856-005010",
"860-004010", "860-005010", "940-004010", "940-005010", "945-004010", "945-005010", "945-in-004010", "945-in-005010",
"DESADV-D01B", "DESADV-D96A", "INVOIC-D01B", "INVOIC-D96A", "ORDERS-D01B", "ORDERS-D96A", "ORDRSP-D01B", "ORDRSP-D96A");
/*******************************************************************************
** Utility class.
*******************************************************************************/
private SampleMaps()
{
}
/*******************************************************************************
** One bundled sample: the map's values and its rules.
*******************************************************************************/
public static JsonNode read(String resource) throws IOException
{
try(InputStream input = SampleMaps.class.getResourceAsStream("/edi/maps/" + resource + "-sample.json"))
{
if(input == null)
{
throw (new IOException("Unknown sample map " + resource));
}
return (new ObjectMapper().readTree(input));
}
}
/*******************************************************************************
** The bundled samples whose map name does not exist yet (any revision, any
** partner), in bundle order. Reads the map table as the processing user.
*******************************************************************************/
public static List<String> missing() throws QException, IOException
{
Set<String> existing = new HashSet<>();
for(QRecord map : SystemWrites.query("edi_map", new QQueryFilter()))
{
existing.add(map.getValueString("name"));
}
List<String> missing = new ArrayList<>();
for(String resource : RESOURCES)
{
if(!existing.contains(read(resource).path("map").path("name").asText()))
{
missing.add(resource);
}
}
return (missing);
}
/*******************************************************************************
** The map record a sample installs as: a draft, revision 0, no partner,
** bound to the built-in guide for its document and version when present.
*******************************************************************************/
public static QRecord mapRecord(String resource) throws IOException, QException
{
JsonNode map = read(resource).path("map");
QRecord record = new QRecord();
map.fields().forEachRemaining(e ->
{
if(!e.getValue().isNull() && !List.of("status", "revision", "partnerId", "guideId", "parentMapId").contains(e.getKey()))
{
record.withValue(e.getKey(), e.getValue().asText());
}
});
record.withValue("status", "DRAFT").withValue("revision", 0);
Integer guideId = null;
int revision = -1;
for(QRecord guide : SystemWrites.query("edi_guide", new QQueryFilter().withCriteria("transactionSet", QCriteriaOperator.EQUALS, map.path("transactionSet").asText())
.withCriteria("version", QCriteriaOperator.EQUALS, map.path("version").asText()).withCriteria("source", QCriteriaOperator.EQUALS, "BUILT_IN")
.withCriteria("status", QCriteriaOperator.EQUALS, "ACTIVE")))
{
int candidate = guide.getValueInteger("revision") == null ? 0 : guide.getValueInteger("revision");
if(candidate > revision)
{
revision = candidate;
guideId = guide.getValueInteger("id");
}
}
return (record.withValue("guideId", guideId));
}
/*******************************************************************************
** Insert one sample and its ordered rules. Skips (returns null) a sample
** whose map name now exists, so a repeated run adds nothing. It is an
** administrator action; two people running it at the same moment is not
** guarded against (edi_map has no unique name, since revisions share one).
*******************************************************************************/
public static Integer install(String resource) throws Exception
{
JsonNode sample = read(resource);
String name = sample.path("map").path("name").asText();
if(!EdiRecords.query("edi_map", EdiRecords.equal("name", name)).isEmpty())
{
return (null);
}
QRecord map = SystemWrites.insert("edi_map", mapRecord(resource));
for(JsonNode rule : sample.path("rules"))
{
QRecord row = new QRecord();
rule.fields().forEachRemaining(e ->
{
if(!e.getValue().isNull())
{
row.withValue(e.getKey(), e.getValue().isInt() ? (java.io.Serializable) e.getValue().intValue() : e.getValue().isBoolean() ? (java.io.Serializable) e.getValue().booleanValue() : e.getValue().asText());
}
});
SystemWrites.insert("edi_map_rule", row.withValue("mapId", map.getValue("id")));
}
return (map.getValueInteger("id"));
}
}
@@ -0,0 +1,109 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi.metadata;
import java.util.LinkedHashMap;
import java.util.Map;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.help.QHelpContent;
import com.kingsrook.qqq.backend.core.model.metadata.help.bundles.HelpBundles;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
/*******************************************************************************
** Applies help/qbit-edi.yaml (the QQQ help-bundle format) to the Edi app
** and its edi_* tables: what each table is for and how its rows get created,
** shown on the table screen and when it is empty.
** Fallback for hosts without the helpContent table. Hosts with that table use
** the core bundle registry and startup sync, including HOST and USER overrides.
*******************************************************************************/
public final class EdiHelp
{
public static final String RESOURCE = "/help/qbit-edi.yaml";
public static final String SLOT = "help";
public static final String APP_KEY = "edi";
/*******************************************************************************
** Utility class.
*******************************************************************************/
private EdiHelp()
{
}
/*******************************************************************************
** The bundle's entries by key, validated (fails startup if malformed).
*******************************************************************************/
public static Map<String, JsonNode> entries() throws QException
{
Map<String, JsonNode> entries = new LinkedHashMap<>();
ObjectMapper mapper = new ObjectMapper();
for(var entry : HelpBundles.load("help/qbit-edi.yaml", EdiHelp.class.getClassLoader()).getEntries())
{
entries.put(entry.getKey(), mapper.valueToTree(entry));
}
return (entries);
}
/*******************************************************************************
** Apply the help to the app (by its configured name) and every edi_* table. A
** host's own help on a table is kept (the bundle only fills the slot).
*******************************************************************************/
public static void apply(QInstance qInstance, String appName) throws QException
{
Map<String, JsonNode> entries = entries();
for(QTableMetaData table : qInstance.getTables().values())
{
JsonNode entry = entries.get("table:" + table.getName() + ";slot:" + SLOT);
if(entry != null && (table.getHelpContent() == null || !table.getHelpContent().containsKey(SLOT)))
{
table.withHelpContent(SLOT, content(entry));
}
}
QAppMetaData app = qInstance.getApp(appName);
JsonNode appEntry = entries.get("app:" + APP_KEY + ";slot:" + SLOT);
if(app != null && appEntry != null && (app.getHelpContent() == null || !app.getHelpContent().containsKey(SLOT)))
{
app.withHelpContent(SLOT, content(appEntry));
}
}
/*******************************************************************************
** Preserve the tooltip summary separately from the Markdown long help.
*******************************************************************************/
static QHelpContent content(JsonNode entry)
{
return (new QHelpContent().withSummary(entry.path("summary").asText().strip()).withContentAsMarkdown(entry.path("content").asText().strip()));
}
}
@@ -88,6 +88,7 @@ public final class EdiProcesses
result.add(action("edi_guide", "override", "Create Partner Override", "edi.guide.override"));
result.add(action("edi_guide", "activate", "Activate Guide", "edi.guide.activate"));
result.add(action("edi_map", "activate", "Activate Map", "edi.map.activate"));
result.add(InstallSampleMapsProcess.process());
QProcessMetaData scheduled = new QProcessMetaData().withName("edi_inbox_worker").withLabel("Process Inbox").withIsHidden(true)
.withPermissionRules(permission("edi.admin")).withStep(new QBackendStepMetaData().withName("run").withCode(new QCodeReference(WorkerStep.class)));
if(config.getSchedulerName() != null)
@@ -0,0 +1,210 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi.processes;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.edi.map.SampleMaps;
import com.kingsrook.qbits.edi.support.SystemWrites;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.PermissionLevel;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRules;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractExtractStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractLoadStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractTransformStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ProcessSummaryProviderInterface;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
/*******************************************************************************
** "Install sample maps": a table action on edi_map that adds the bundled
** synthetic sample maps whose name does not exist yet. Standard streamed ETL:
** Review lists what will be added, Result what was added. Insert-only by map
** name, so existing maps and edits are never touched. Installed maps are
** drafts with no partner and no agreement document, so nothing changes how a
** document is processed until an administrator activates and binds one.
*******************************************************************************/
public final class InstallSampleMapsProcess
{
public static final String NAME = "edi_map_installSamples";
/*******************************************************************************
** Utility class.
*******************************************************************************/
private InstallSampleMapsProcess()
{
}
/*******************************************************************************
** The process: a table-level action (no records need to be selected).
*******************************************************************************/
public static QProcessMetaData process()
{
Map<String, Serializable> values = new LinkedHashMap<>();
values.put(StreamedETLWithFrontendProcess.FIELD_SOURCE_TABLE, "edi_map");
values.put(StreamedETLWithFrontendProcess.FIELD_DESTINATION_TABLE, "edi_map");
values.put(StreamedETLWithFrontendProcess.FIELD_SUPPORTS_FULL_VALIDATION, false);
values.put(StreamedETLWithFrontendProcess.FIELD_PREVIEW_MESSAGE,
"These sample maps will be added as drafts, with no partner and no agreement document. Maps that already exist are not changed.");
QProcessMetaData process = StreamedETLWithFrontendProcess.defineProcessMetaData(Extract.class, Transform.class, Load.class, values);
process.setName(NAME);
process.setLabel("Install sample maps");
process.setIcon(new QIcon("playlist_add"));
process.setTableName("edi_map");
process.setIsHidden(true);
process.setMinInputRecords(0);
process.setPermissionRules(new QPermissionRules().withPermissionBaseName("edi.admin").withLevel(PermissionLevel.HAS_ACCESS_PERMISSION));
return (process);
}
/*******************************************************************************
** The samples still to install, as the processing user.
*******************************************************************************/
static List<String> missing() throws QException
{
return (SystemWrites.call(SampleMaps::missing));
}
/*******************************************************************************
** A summary line with the standard four wordings.
*******************************************************************************/
private static ProcessSummaryLine line(Status status, String future, String futurePlural, String past, String pastPlural)
{
ProcessSummaryLine line = new ProcessSummaryLine(status, 0, null);
line.setSingularFutureMessage(future);
line.setPluralFutureMessage(futurePlural);
line.setSingularPastMessage(past);
line.setPluralPastMessage(pastPlural);
return (line);
}
/*******************************************************************************
** Extract: the missing samples, computed (no query of selected records).
*******************************************************************************/
public static class Extract extends AbstractExtractStep
{
/*******************************************************************************
** How many samples will be added.
*******************************************************************************/
@Override
public Integer doCount(RunBackendStepInput input) throws QException
{
return (missing().size());
}
/*******************************************************************************
** One record per missing sample.
*******************************************************************************/
@Override
public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
for(String resource : missing())
{
QRecord map = SystemWrites.call(() -> SampleMaps.mapRecord(resource));
getRecordPipe().addRecord(map.withValue("sample", resource));
}
}
}
/*******************************************************************************
** Transform: forward the records and count them by standard and direction.
*******************************************************************************/
public static class Transform extends AbstractTransformStep implements ProcessSummaryProviderInterface
{
private final Map<String, ProcessSummaryLine> byKind = new LinkedHashMap<>();
/*******************************************************************************
** Count by standard and direction.
*******************************************************************************/
@Override
public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
for(QRecord record : input.getRecords())
{
String kind = record.getValueString("standard") + " " + record.getValueString("direction").toLowerCase();
byKind.computeIfAbsent(kind, k -> line(Status.OK, k + " sample map will be added as a draft", k + " sample maps will be added as drafts",
k + " sample map was added as a draft", k + " sample maps were added as drafts")).incrementCount();
output.getRecords().add(record);
}
}
/*******************************************************************************
** Standard Review summary.
*******************************************************************************/
@Override
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean isForResultScreen)
{
ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>();
byKind.values().forEach(line -> line.addSelfToListIfAnyCount(lines));
if(lines.isEmpty())
{
lines.add(new ProcessSummaryLine(Status.INFO, null, "Every sample map is already installed."));
}
return (lines);
}
}
/*******************************************************************************
** Load: insert each map and its rules as the processing user.
*******************************************************************************/
public static class Load extends AbstractLoadStep implements ProcessSummaryProviderInterface
{
private final ProcessSummaryLine added = line(Status.OK, "map will be added", "maps will be added", "map was added as a draft", "maps were added as drafts");
/*******************************************************************************
** One transaction per map, so a map never lands without its rules.
*******************************************************************************/
@Override
public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
for(QRecord record : input.getRecords())
{
Integer id = SystemWrites.call(() -> SystemWrites.transaction(() -> SampleMaps.install(record.getValueString("sample"))));
if(id != null)
{
added.incrementCountAndAddPrimaryKey(id);
output.getRecords().add(new QRecord().withValue("id", id));
}
}
}
/*******************************************************************************
** Standard Result summary.
*******************************************************************************/
@Override
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean isForResultScreen)
{
ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>();
added.addSelfToListIfAnyCount(lines);
return (lines);
}
}
}
File diff suppressed because it is too large. Load diff
+188 -22
View File
@@ -1,5 +1,44 @@
components:
schemas:
BasicErrorResponseV1:
description: "The error body of every UI API operation (core and extensions)."
properties:
error:
description: "The error message (the same as message; kept for clients of\
\ UI API 1.0 to 1.5)."
type: "string"
code:
description: "A stable, machine-readable code. Well-known values: BAD_REQUEST\
\ (400), UNAUTHENTICATED (401),\nPERMISSION_DENIED (403), NOT_FOUND (404),\
\ CONFLICT (409), RATE_LIMITED (429), INTERNAL_ERROR (500),\nERROR (other).\
\ Extensions may add codes; treat an unknown code by the HTTP status.\
\ Since 1.6.0."
type: "string"
message:
description: "A message to show the user. Since 1.6.0."
type: "string"
fieldErrors:
description: "Errors about particular request fields, when known. Since\
\ 1.6.0."
items:
properties:
field:
description: "The field (or parameter) name."
type: "string"
message:
description: "The error about that field."
type: "string"
code:
description: "An optional machine-readable code, e.g. REQUIRED, TOO_LONG,\
\ PATTERN."
type: "string"
type: "object"
type: "array"
traceId:
description: "Identifies this request in the server logs (also the X-QQQ-Trace-Id\
\ response header); quote it when reporting a problem. Since 1.6.0."
type: "string"
type: "object"
EdiAckV1:
description: "Concatenated acknowledgement previews; type is 997, 999, TA1,\
\ CONTRL, NONE or MIXED. text is null for PHI."
@@ -93,7 +132,7 @@ components:
type: "string"
type: "object"
EdiErrorV1:
description: "Core middleware error body."
description: "Deprecated since 1.1.0: errors use BasicErrorResponseV1."
properties:
error:
nullable: true
@@ -397,21 +436,55 @@ components:
type: "object"
securitySchemes:
sessionUuidCookie:
description: "The session cookie set by POST /qqq/v1/manageSession (web clients)."
in: "cookie"
name: "sessionUUID"
type: "apiKey"
bearerAuth:
bearerFormat: "JWT or session UUID"
description: "`Authorization: Bearer <token>`, where the token is either\n-\
\ an OAuth2 access token (a signed JWT) issued by the app's identity provider\
\ to the app's client,\n or to its native client (`values.nativeClientId`\
\ in GET /qqq/v1/metaData/authentication), for\n example from the device-authorization\
\ grant or auth-code + PKCE. It is verified against the\n provider's keys\
\ (issuer, audience, expiry, signature) and gives the same user and permissions\n\
\ as signing in on the web. When it expires (401), refresh it at the provider\
\ and retry; or\n- a session UUID returned by POST /qqq/v1/manageSession,\
\ which the server keeps refreshed."
scheme: "bearer"
type: "http"
oauth2:
description: "The app's OAuth2 / OpenID Connect provider. The real endpoints\
\ come from the provider's discovery\ndocument: `{values.baseUrl}.well-known/openid-configuration`,\
\ with values from\nGET /qqq/v1/metaData/authentication. Native clients use\
\ `values.nativeClientId` (a public client)\nwith auth-code + PKCE (loopback\
\ or custom-scheme redirect URI registered at the provider) or the\ndevice-authorization\
\ grant (`device_authorization_endpoint`, RFC 8628), then send the access\
\ token\nas `Authorization: Bearer`."
flows:
authorizationCode:
authorizationUrl: "/authorize"
scopes:
email: "Email address"
offline_access: "A refresh token, to renew the access token without signing\
\ in again"
openid: "OpenID Connect sign-in"
profile: "Name, username and groups (permissions come from the groups)"
tokenUrl: "/token"
type: "oauth2"
info:
contact:
email: "contact@kingsrook.com"
description: "EDI sandbox and mapper, using the signed-in user's permissions."
title: "QQQ Middleware API extension: edi"
version: "1.0.0"
version: "1.1.0"
openapi: "3.0.3"
paths:
/qqq/v1/qbit/edi/guides/{guideId}/tree:
get:
description: "Read the guide tree. Uses the caller session and table read permissions.\
\ Preview also requires edi.map.preview. Rule writes use core table CRUD."
operationId: "getQbitEdiGuidesByGuideIdTree"
parameters:
- description: "Positive record identifier."
in: "path"
@@ -424,25 +497,25 @@ paths:
content:
application/json:
schema:
$ref: "#/components/schemas/EdiErrorV1"
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Invalid request or sample exceeds 5 MiB"
401:
content:
application/json:
schema:
$ref: "#/components/schemas/EdiErrorV1"
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Sign in required"
403:
content:
application/json:
schema:
$ref: "#/components/schemas/EdiErrorV1"
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Permission denied"
404:
content:
application/json:
schema:
$ref: "#/components/schemas/EdiErrorV1"
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Missing or invisible record"
200:
content:
@@ -454,17 +527,39 @@ paths:
content:
application/json:
schema:
$ref: "#/components/schemas/EdiErrorV1"
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Host HTTP request size limit exceeded"
409:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Conflict with the current state, e.g. an edit conflict or\
\ an invalid transition (code CONFLICT)."
429:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Too many requests (code RATE_LIMITED); retry later."
security:
- sessionUuidCookie:
- "N/A"
- bearerAuth:
- "N/A"
- oauth2:
- "openid"
- "profile"
- "email"
summary: "Read the guide tree"
tags:
- "edi"
/qqq/v1/qbit/edi/maps/{mapId}/target:
get:
description: "Read target metadata and map rules. Uses the caller session and\
\ table read permissions. Preview also requires edi.map.preview. Rule writes\
\ use core table CRUD."
operationId: "getQbitEdiMapsByMapIdTarget"
parameters:
- description: "Positive record identifier."
in: "path"
@@ -477,25 +572,25 @@ paths:
content:
application/json:
schema:
$ref: "#/components/schemas/EdiErrorV1"
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Invalid request or sample exceeds 5 MiB"
401:
content:
application/json:
schema:
$ref: "#/components/schemas/EdiErrorV1"
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Sign in required"
403:
content:
application/json:
schema:
$ref: "#/components/schemas/EdiErrorV1"
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Permission denied"
404:
content:
application/json:
schema:
$ref: "#/components/schemas/EdiErrorV1"
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Missing or invisible record"
200:
content:
@@ -507,17 +602,39 @@ paths:
content:
application/json:
schema:
$ref: "#/components/schemas/EdiErrorV1"
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Host HTTP request size limit exceeded"
409:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Conflict with the current state, e.g. an edit conflict or\
\ an invalid transition (code CONFLICT)."
429:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Too many requests (code RATE_LIMITED); retry later."
security:
- sessionUuidCookie:
- "N/A"
- bearerAuth:
- "N/A"
- oauth2:
- "openid"
- "profile"
- "email"
summary: "Read target metadata and map rules"
tags:
- "edi"
/qqq/v1/qbit/edi/sandbox/preview:
post:
description: "Preview an encrypted sample with optional unsaved rules. Uses\
\ the caller session and table read permissions. Preview also requires edi.map.preview.\
\ Rule writes use core table CRUD."
operationId: "postQbitEdiSandboxPreview"
requestBody:
content:
application/json:
@@ -529,25 +646,25 @@ paths:
content:
application/json:
schema:
$ref: "#/components/schemas/EdiErrorV1"
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Invalid request or sample exceeds 5 MiB"
401:
content:
application/json:
schema:
$ref: "#/components/schemas/EdiErrorV1"
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Sign in required"
403:
content:
application/json:
schema:
$ref: "#/components/schemas/EdiErrorV1"
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Permission denied"
404:
content:
application/json:
schema:
$ref: "#/components/schemas/EdiErrorV1"
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Missing or invisible record"
200:
content:
@@ -559,17 +676,39 @@ paths:
content:
application/json:
schema:
$ref: "#/components/schemas/EdiErrorV1"
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Host HTTP request size limit exceeded"
409:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Conflict with the current state, e.g. an edit conflict or\
\ an invalid transition (code CONFLICT)."
429:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Too many requests (code RATE_LIMITED); retry later."
security:
- sessionUuidCookie:
- "N/A"
- bearerAuth:
- "N/A"
- oauth2:
- "openid"
- "profile"
- "email"
summary: "Preview an encrypted sample with optional unsaved rules"
tags:
- "edi"
/qqq/v1/qbit/edi/maps/{mapId}/validate:
post:
description: "Validate unsaved map rules. Uses the caller session and table\
\ read permissions. Preview also requires edi.map.preview. Rule writes use\
\ core table CRUD."
operationId: "postQbitEdiMapsByMapIdValidate"
parameters:
- description: "Positive record identifier."
in: "path"
@@ -588,25 +727,25 @@ paths:
content:
application/json:
schema:
$ref: "#/components/schemas/EdiErrorV1"
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Invalid request or sample exceeds 5 MiB"
401:
content:
application/json:
schema:
$ref: "#/components/schemas/EdiErrorV1"
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Sign in required"
403:
content:
application/json:
schema:
$ref: "#/components/schemas/EdiErrorV1"
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Permission denied"
404:
content:
application/json:
schema:
$ref: "#/components/schemas/EdiErrorV1"
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Missing or invisible record"
200:
content:
@@ -618,9 +757,36 @@ paths:
content:
application/json:
schema:
$ref: "#/components/schemas/EdiErrorV1"
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Host HTTP request size limit exceeded"
409:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Conflict with the current state, e.g. an edit conflict or\
\ an invalid transition (code CONFLICT)."
429:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Too many requests (code RATE_LIMITED); retry later."
security:
- sessionUuidCookie:
- "N/A"
- bearerAuth:
- "N/A"
- oauth2:
- "openid"
- "profile"
- "email"
summary: "Validate unsaved map rules"
tags:
- "edi"
servers:
- description: "The QQQ server that serves this document (paths include /qqq/v1)."
url: "/"
tags:
- description: "Operations of the edi extension."
name: "edi"
+1
View File
@@ -0,0 +1 @@
version=${project.version}
@@ -35,7 +35,7 @@ import static org.assertj.core.api.Assertions.assertThat;
class EdiApiContractTest
{
/*******************************************************************************
** Use RC.6's real helper, including compatibility comparison against this first baseline.
** Check the generated contract and compatibility against the committed baseline.
*******************************************************************************/
@Test
void contract() throws Exception
@@ -53,15 +53,15 @@ class EdiApiContractTest
assertThat(UiApiContract.check(generated, committed, committed, 1,
"mvnw.cmd -B -Dtest=EdiApiContractTest -Dedi.openapi.update=true test")).isEmpty();
var model = version.generateExtensionOpenAPIModel("qqq", extension);
assertThat(model.getComponents().getSchemas().keySet()).allMatch(n -> n.startsWith("Edi") && n.endsWith("V1"));
assertThat(model.getComponents().getSchemas().keySet()).allMatch(n -> (n.startsWith("Edi") && n.endsWith("V1")) || n.equals("BasicErrorResponseV1"));
assertThat(model.getPaths()).hasSize(4);
assertThat(model.getPaths().keySet()).allMatch(n -> n.startsWith("/qqq/v1/qbit/edi/"));
assertThat(extension.getEffectivePathPrefix()).isEqualTo("/qbit/edi");
assertThat(extension.getSemanticVersion()).isEqualTo("1.0.0");
assertThat(extension.getSemanticVersion()).isEqualTo("1.1.0");
}
/*******************************************************************************
** Producer registers the extension and all metadata validates with RC.6.
** Producer registers the extension and all metadata validates with core middleware.
*******************************************************************************/
@Test
void registration() throws Exception
@@ -169,8 +169,18 @@ class EdiApiHttpTest
private HttpResponse<String> request(String method, String path, Object body, String role) throws Exception
{
String text = body == null ? "" : body instanceof String s ? s : JSON.writeValueAsString(body);
return (client.send(HttpRequest.newBuilder(URI.create(baseUrl + path)).header("Cookie", "sessionUUID=" + role + suffix)
.header("Content-Type", "application/json").method(method, HttpRequest.BodyPublishers.ofString(text)).build(), HttpResponse.BodyHandlers.ofString()));
var response = client.send(HttpRequest.newBuilder(URI.create(baseUrl + path)).header("Cookie", "sessionUUID=" + role + suffix)
.header("Content-Type", "application/json").method(method, HttpRequest.BodyPublishers.ofString(text)).build(), HttpResponse.BodyHandlers.ofString());
if(response.statusCode() >= 400)
{
JsonNode error = JSON.readTree(response.body());
for(String field : List.of("error", "code", "message", "traceId"))
{
assertThat(error.path(field).isTextual()).as(field + " in " + response.body()).isTrue();
assertThat(error.path(field).asText()).isNotBlank();
}
}
return (response);
}
/*******************************************************************************
@@ -0,0 +1,88 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi;
import com.kingsrook.qqq.backend.core.actions.help.HelpTargetCatalog;
import com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.help.bundles.HelpBundles;
import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryBackendModule;
import com.kingsrook.qqq.backend.core.testing.HelpBundleAssertions;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** Coverage of the qbit's own help, independent of host integrations.
*******************************************************************************/
class EdiHelpBundleTest
{
/*******************************************************************************
** Every owned target resolves and has help, even without a help table.
*******************************************************************************/
@Test
void bundleIsComplete() throws Exception
{
QInstance instance = new QInstance();
instance.addBackend(new QBackendMetaData().withName("memory").withBackendType(MemoryBackendModule.class));
new EdiQBitProducer().withQBitConfig(new EdiQBitConfig().withBackendName("memory")).produce(instance);
assertThat(instance.getTable("helpContent")).isNull();
assertThat(HelpBundles.of(instance).getBundles()).singleElement().satisfies(bundle ->
{
assertThat(bundle.getId()).isEqualTo("qbit-edi");
assertThat(bundle.getVersion()).as("the help bundle carries the artifact version").isEqualTo(System.getProperty("qbit.artifactVersion")).isEqualTo(EdiQBitProducer.VERSION);
});
HelpBundles.syncOnStartup(instance);
HelpBundleAssertions.forInstance(instance)
.bundles("help/qbit-edi.yaml")
.owning(target -> target.name().equals("edi") || target.name().startsWith("edi_"))
.assertComplete();
var keys = HelpBundles.of(instance).getBundles().getFirst().getEntries().stream().map(entry -> entry.getKey()).toList();
// Core's gate currently requires apps, tables, processes and fields; also cover our widgets and value sources.
for(var target : HelpTargetCatalog.collect(instance))
{
if(target.name().startsWith("edi_") && java.util.List.of("widget", "possibleValueSource").contains(target.kind()))
{
assertThat(keys).as(target.key()).contains(target.key());
}
}
}
/*******************************************************************************
** Help-enabled hosts use core sync, and a configured app name is respected.
*******************************************************************************/
@Test
void registersForHelpEnabledHost() throws Exception
{
QInstance instance = new QInstance();
instance.addBackend(new QBackendMetaData().withName("memory").withBackendType(MemoryBackendModule.class));
instance.addTable(new com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData().withName("helpContent"));
new EdiQBitProducer().withQBitConfig(new EdiQBitConfig().withBackendName("memory").withAppName("trading")).produce(instance);
var entries = HelpBundles.of(instance).getBundles().getFirst().getEntries();
assertThat(entries).anyMatch(entry -> "app:trading;slot:help".equals(entry.getKey()))
.noneMatch(entry -> "app:edi;slot:help".equals(entry.getKey()));
assertThat(instance.getTable("edi_partner").getHelpContent()).isNullOrEmpty();
assertThat(instance.getApp("trading").getHelpContent()).isNullOrEmpty();
}
}
@@ -0,0 +1,217 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi;
import java.net.URLDecoder;
import java.nio.charset.StandardCharsets;
import java.time.Clock;
import java.time.Instant;
import java.time.ZoneOffset;
import java.time.temporal.ChronoUnit;
import java.util.List;
import com.kingsrook.qbits.edi.app.EdiHome;
import com.kingsrook.qbits.edi.support.SystemWrites;
import com.kingsrook.qqq.backend.core.actions.dashboard.RenderWidgetAction;
import com.kingsrook.qqq.backend.core.actions.metadata.MetaDataAction;
import com.kingsrook.qqq.backend.core.actions.permissions.PermissionEnforcement;
import com.kingsrook.qqq.backend.core.actions.permissions.PermissionsHelper;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
import com.kingsrook.qqq.backend.core.model.actions.metadata.MetaDataInput;
import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.AlertData;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.TableData;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.DenyBehavior;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.PermissionLevel;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import com.kingsrook.qqq.backend.core.model.session.QSession;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
/*******************************************************************************
** The EDI home is for business users: system widgets live in a separate list
** for the host's Admin home, gated to the EDI administrator; a stalled inbound
** flow shows as one business item; every widget is permission-protected.
*******************************************************************************/
class EdiHomeAudienceTest
{
private static final Instant RECEIVED = Instant.parse("2026-10-16T12:00:00Z");
/*******************************************************************************
**
*******************************************************************************/
@AfterEach
void clear()
{
QContext.clear();
}
/*******************************************************************************
** The app's widget list is the business list; system widgets stay defined,
** are gated to edi.admin and hidden, and nothing is NOT_PROTECTED.
*******************************************************************************/
@Test
void homeWidgetListExcludesSystemWidgets() throws Exception
{
QInstance instance = H2Support.create();
List<String> appWidgets = instance.getApp(EdiRuntime.current().config().getAppName()).getWidgets();
assertThat(appWidgets).containsExactlyElementsOf(EdiHome.HOME_WIDGET_NAMES).doesNotContainAnyElementsOf(EdiHome.ADMIN_WIDGET_NAMES);
assertThat(EdiHome.ADMIN_WIDGET_NAMES).containsExactlyInAnyOrder("edi_home_system", "edi_home_dead", "edi_home_certificates", "edi_home_setup");
assertThat(appWidgets.getFirst()).as("attention comes first").isEqualTo("edi_home_delayed");
for(String name : EdiHome.ADMIN_WIDGET_NAMES)
{
QWidgetMetaData widget = (QWidgetMetaData) instance.getWidget(name);
assertThat(widget).as(name).isNotNull();
assertThat(widget.getPermissionRules().getPermissionBaseName()).as(name).isEqualTo("edi.admin");
assertThat(widget.getGridColumns()).as(name + " is full width in its Admin placement").isEqualTo(12);
}
for(String name : EdiHome.HOME_WIDGET_NAMES)
{
assertThat(((QWidgetMetaData) instance.getWidget(name)).getPermissionRules().getPermissionBaseName()).as(name).isEqualTo("edi");
}
instance.getWidgets().values().stream().filter(w -> w.getName().startsWith("edi_home_")).forEach(widget ->
{
assertThat(widget.getPermissionRules().getLevel()).as(widget.getName()).isEqualTo(PermissionLevel.HAS_ACCESS_PERMISSION);
assertThat(widget.getPermissionRules().getDenyBehavior()).as(widget.getName()).isEqualTo(DenyBehavior.HIDDEN);
});
}
/*******************************************************************************
** A business EDI user (edi.hasAccess, no edi.admin) gets no system widget, in
** metadata or by rendering; an EDI administrator gets them; a user without
** edi.hasAccess gets no home widget at all.
*******************************************************************************/
@Test
void businessUserCannotRenderSystemWidgets() throws Exception
{
QInstance instance = H2Support.create();
QContext.setQSession(readers(new QSession().withPermissions("edi.hasAccess")));
assertThat(new MetaDataAction().execute(new MetaDataInput()).getWidgets().keySet())
.containsAll(EdiHome.HOME_WIDGET_NAMES).doesNotContainAnyElementsOf(EdiHome.ADMIN_WIDGET_NAMES);
for(String name : EdiHome.ADMIN_WIDGET_NAMES)
{
assertThatThrownBy(() -> new RenderWidgetAction().execute(render(instance, name)))
.as(name).isInstanceOf(QPermissionDeniedException.class);
}
assertThat(new RenderWidgetAction().execute(render(instance, "edi_home_queue")).getWidgetData()).isInstanceOf(TableData.class);
QContext.setQSession(readers(new QSession().withPermissions("edi.hasAccess", "edi.admin.hasAccess")));
assertThat(new MetaDataAction().execute(new MetaDataInput()).getWidgets().keySet()).containsAll(EdiHome.ADMIN_WIDGET_NAMES);
for(String name : EdiHome.ADMIN_WIDGET_NAMES)
{
assertThat(new RenderWidgetAction().execute(render(instance, name)).getWidgetData()).as(name).isNotNull();
}
QContext.setQSession(readers(new QSession()));
assertThat(new MetaDataAction().execute(new MetaDataInput()).getWidgets().keySet()).noneMatch(name -> name.startsWith("edi_home_"));
}
/*******************************************************************************
** "Inbound documents are delayed" appears only past the threshold, with the
** age and a link to the filtered inbound documents; hidden otherwise.
*******************************************************************************/
@Test
void delayedInboundOnlyPastThreshold() throws Exception
{
QInstance instance = H2Support.create();
assertThat(EdiRuntime.current().config().getInboundDelayedAfterMinutes()).as("single-node default").isEqualTo(60);
QContext.setQSession(readers(new QSession().withPermissions("edi.hasAccess")));
assertThat(delayed(instance, RECEIVED).getHideWidget()).as("nothing waiting").isTrue();
SystemWrites.call(() -> SystemWrites.insert("edi_inbox", new QRecord().withValue("transportType", "SFTP").withValue("externalId", "synthetic-delayed")
.withValue("fileName", "synthetic.edi").withValue("payloadRef", "synthetic").withValue("payloadSha256", "0").withValue("sizeBytes", 1)
.withValue("receivedAt", RECEIVED).withValue("status", "PENDING").withValue("attempts", 0).withValue("nextAttemptAt", RECEIVED)));
assertThat(delayed(instance, RECEIVED.plus(30, ChronoUnit.MINUTES)).getHideWidget()).as("within the threshold").isTrue();
AlertData alert = delayed(instance, RECEIVED.plus(95, ChronoUnit.MINUTES));
assertThat(alert.getHideWidget()).isFalse();
assertThat(alert.getAlertType()).isEqualTo(AlertData.AlertType.WARNING);
assertThat(alert.getHtml()).contains("Inbound documents are delayed", "1 document has waited more than 1 h", "the oldest has waited 1 h 35 m")
.contains("href=\"/app/edi_inbox?filter=").doesNotContain("-1");
String filter = URLDecoder.decode(alert.getHtml().replaceAll(".*\\?filter=([^\"]*)\".*", "$1"), StandardCharsets.UTF_8);
assertThat(filter).contains("receivedAt", "LESS_THAN", "PENDING");
EdiRuntime.current().config().withInboundDelayedAfterMinutes(120);
assertThat(delayed(instance, RECEIVED.plus(95, ChronoUnit.MINUTES)).getHideWidget()).as("configurable threshold").isTrue();
EdiRuntime.current().config().withInboundDelayedAfterMinutes(60);
QContext.setQSession(new QSession().withPermissions("edi.hasAccess").withSecurityKeyValue("edi.admin", true));
assertThat(delayed(instance, RECEIVED.plus(95, ChronoUnit.MINUTES)).getHideWidget()).as("viewer cannot read inbound documents").isTrue();
}
/*******************************************************************************
** A setup count that fails shows "Unavailable", never -1.
*******************************************************************************/
@Test
void setupCountFailureIsUnavailable() throws Exception
{
QInstance instance = H2Support.create();
QSession session = readers(new QSession().withPermissions("edi.hasAccess", "edi.admin.hasAccess"));
session.getPermissions().remove("edi.transport.read");
QContext.setQSession(session);
var data = (TableData) new RenderWidgetAction().execute(render(instance, "edi_home_setup")).getWidgetData();
assertThat(data.getRows()).anySatisfy(row -> assertThat(row).containsEntry("Step", "5. Transport").containsEntry("Status", "Unavailable"));
assertThat(data.getRows()).allSatisfy(row -> assertThat(row.values()).noneMatch(value -> String.valueOf(value).contains("-1")));
}
/*******************************************************************************
** Render the delayed-inbound alert at a given time.
*******************************************************************************/
private static AlertData delayed(QInstance instance, Instant now) throws Exception
{
EdiRuntime.current().config().withClock(Clock.fixed(now, ZoneOffset.UTC));
return ((AlertData) new RenderWidgetAction().execute(render(instance, "edi_home_delayed")).getWidgetData());
}
/*******************************************************************************
** A render input that enforces the viewer's permissions, as the UI API does.
*******************************************************************************/
private static RenderWidgetInput render(QInstance instance, String widgetName)
{
RenderWidgetInput input = new RenderWidgetInput().withWidgetMetaData(instance.getWidget(widgetName));
input.setPermissionEnforcement(PermissionEnforcement.ENFORCE);
return (input);
}
/*******************************************************************************
** Add read on every EDI table and all-partner row access.
*******************************************************************************/
private static QSession readers(QSession session)
{
QInstance instance = QContext.getQInstance();
for(QTableMetaData table : instance.getTables().values())
{
if(table.getName().startsWith("edi_"))
{
session.withPermissions(PermissionsHelper.getEffectivePermissionBaseName(PermissionsHelper.getEffectivePermissionRules(table, instance), table.getName()) + ".read");
}
}
return (session.withSecurityKeyValue("edi.admin", true));
}
}
@@ -65,6 +65,8 @@ class EdiQBitProducerTest
assertThat(producer.getQBitConfig()).isSameAs(config);
assertThat(producer.getNamespace()).isEqualTo("ns");
assertThat(producer.getQBitMetaData().getArtifactId()).isEqualTo("qbit-edi");
assertThat(System.getProperty("qbit.artifactVersion")).isNotBlank();
assertThat(producer.getQBitMetaData().getVersion()).isEqualTo(System.getProperty("qbit.artifactVersion")).isEqualTo(EdiQBitProducer.VERSION);
assertThat(producer.getPackageNameForFindingMetaDataProducers()).isEqualTo("com.kingsrook.qbits.edi.metadata");
assertThat(config.getAppName()).isEqualTo("edi");
assertThat(config.getAppLabel()).isEqualTo("EDI");
@@ -0,0 +1,129 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.edi.app.EdiSetupStatus;
import com.kingsrook.qbits.edi.support.SystemWrites;
import com.kingsrook.qqq.backend.core.actions.dashboard.RenderWidgetAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.TableData;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** The Setup status on the EDI home: the steps in setup order, each with its
** state and next step, counting only the rows the runtime would use.
*******************************************************************************/
class EdiSetupStatusTest
{
/*******************************************************************************
**
*******************************************************************************/
@AfterEach
void clear()
{
QContext.clear();
}
/*******************************************************************************
** The pure logic: order, states and next steps.
*******************************************************************************/
@Test
void stepsAreInSetupOrder()
{
List<EdiSetupStatus.Item> items = EdiSetupStatus.items(new EdiSetupStatus.Counts(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 41, 20, 0, 0));
assertThat(items).extracting(EdiSetupStatus.Item::step).containsExactly("1. Partner", "2. Identities", "3. Agreement", "4. Agreement documents", "5. Transport",
"Certificates", "Guides and code lists", "Maps");
assertThat(items.subList(0, 5)).allSatisfy(item -> assertThat(item.state()).isEqualTo(EdiSetupStatus.State.TODO));
assertThat(items.get(5).state()).isEqualTo(EdiSetupStatus.State.OPTIONAL);
assertThat(items.get(6).state()).isEqualTo(EdiSetupStatus.State.BUILT_IN);
assertThat(items.get(7).state()).isEqualTo(EdiSetupStatus.State.OPTIONAL);
assertThat(items.get(7).nextStep()).contains("Install sample maps");
items = EdiSetupStatus.items(new EdiSetupStatus.Counts(1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0));
assertThat(items.get(1).nextStep()).contains("partner's");
assertThat(items.get(6).state()).isEqualTo(EdiSetupStatus.State.TODO);
assertThat(items.get(6).nextStep()).contains("EdiRuntime.initialize()");
assertThat(items.get(7).state()).as("maps exist but none active").isEqualTo(EdiSetupStatus.State.TODO);
items = EdiSetupStatus.items(new EdiSetupStatus.Counts(2, 1, 2, 2, 1, 3, 3, 1, 1, 1, 41, 20, 5, 2));
assertThat(items).allSatisfy(item -> assertThat(item.state()).isIn(EdiSetupStatus.State.DONE, EdiSetupStatus.State.BUILT_IN));
items = EdiSetupStatus.items(new EdiSetupStatus.Counts(1, 1, 1, 2, 0, 1, 1, 1, 1, 0, 41, 20, 0, 0));
assertThat(items.get(2).state()).as("agreements exist but none is active, so the runtime ignores them").isEqualTo(EdiSetupStatus.State.TODO);
items = EdiSetupStatus.items(new EdiSetupStatus.Counts(null, null, null, null, null, null, null, null, null, null, null, null, null, null));
assertThat(items).allSatisfy(item -> assertThat(item.state()).isEqualTo(EdiSetupStatus.State.UNKNOWN));
assertThat(EdiSetupStatus.State.UNKNOWN.label()).as("a failed count is never shown as a number").isEqualTo("Unavailable");
}
/*******************************************************************************
** The home widget moves from "needs setup" to "set up" as the rows are added.
*******************************************************************************/
@Test
void homeWidgetFollowsTheData() throws Exception
{
QInstance instance = H2Support.create();
EdiRuntime.current().initialize();
assertThat(instance.getWidget("edi_home_setup")).isNotNull();
assertThat(instance.getApp(EdiRuntime.current().config().getAppName()).getWidgets()).as("setup is for the Admin home").doesNotContain("edi_home_setup");
assertThat(com.kingsrook.qbits.edi.app.EdiHome.ADMIN_WIDGET_NAMES).contains("edi_home_setup");
Map<String, String> status = status(instance);
assertThat(status).containsKeys("1. Partner", "5. Transport");
assertThat(status.get("1. Partner")).startsWith("Needs setup");
assertThat(status.get("2. Identities")).startsWith("Needs setup");
assertThat(status.get("Guides and code lists")).startsWith("Built in: 41 guides");
assertThat(status.get("Maps")).startsWith("Optional");
SystemWrites.call(() ->
{
QRecord partner = SystemWrites.insert("edi_partner", new QRecord().withValue("name", "Synthetic Retailer").withValue("code", "SYNTH"));
SystemWrites.insert("edi_identity", new QRecord().withValue("standard", "X12").withValue("interchangeQualifier", "ZZ").withValue("interchangeId", "QQQTEST")
.withValue("applicationCode", "QQQTEST").withValue("isActive", true));
SystemWrites.insert("edi_identity", new QRecord().withValue("partnerId", partner.getValue("id")).withValue("standard", "X12").withValue("interchangeQualifier", "ZZ")
.withValue("interchangeId", "SYNTHETIC").withValue("applicationCode", "SYNTHETIC").withValue("isActive", false));
return (null);
});
status = status(instance);
assertThat(status.get("1. Partner")).startsWith("Set up");
assertThat(status.get("2. Identities")).as("the partner's identity is inactive, so the runtime would ignore it").startsWith("Needs setup: Yours is set; no partner has one");
assertThat(status.get("3. Agreement")).startsWith("Needs setup");
}
/*******************************************************************************
** Render the widget and return step to "state: detail".
*******************************************************************************/
private Map<String, String> status(QInstance instance) throws Exception
{
var data = (TableData) new RenderWidgetAction().execute(new RenderWidgetInput().withWidgetMetaData(instance.getWidget("edi_home_setup"))).getWidgetData();
Map<String, String> status = new LinkedHashMap<>();
data.getRows().forEach(row -> status.put(String.valueOf(row.get("Step")), String.valueOf(row.get("Status"))));
return (status);
}
}
@@ -0,0 +1,247 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.regex.Pattern;
import java.util.stream.Stream;
import com.kingsrook.qbits.edi.guide.BuiltInGuideLoader;
import com.kingsrook.qbits.edi.metadata.EdiHelp;
import com.kingsrook.qbits.edi.storage.EdiRecords;
import com.kingsrook.qbits.edi.support.SystemWrites;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.help.QHelpContent;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** Guardrail (owner requirement 2026-10-14: "all tables should be populated and
** the qbit should be fully using them"). For every edi_* table:
** - runtime code (not just its model, metadata or home screen) reads or
** writes it;
** - it is either seeded by the qbit at startup, or listed here with why it
** starts empty;
** - it has bundle help that says what it is for and how its rows get created
** (shown on the table screen and when it is empty).
** A new table fails this test until it is wired, seeded or explained.
*******************************************************************************/
class EdiTablesGuardrailTest
{
private static final Path MAIN = Path.of("src", "main", "java", "com", "kingsrook", "qbits", "edi");
/*******************************************************************************
** Tables the qbit seeds itself (EdiRuntime.initialize, idempotent).
*******************************************************************************/
static final Map<String, String> SEEDED = Map.of(
"edi_guide", "BuiltInGuideLoader: 41 built-in guides",
"edi_guide_segment", "with each built-in guide",
"edi_guide_element", "with each built-in guide",
"edi_guide_rule", "with each built-in guide",
"edi_code_list", "with the built-in guides",
"edi_code_list_value", "with the built-in guides");
/*******************************************************************************
** Tables that legitimately start empty, and where their rows come from.
*******************************************************************************/
static final Map<String, String> STARTS_EMPTY = Map.ofEntries(
Map.entry("edi_partner", "host data: the trading partners"),
Map.entry("edi_identity", "host data: your ISA/UNB identity and each partner's"),
Map.entry("edi_agreement", "host data: the agreement per partner"),
Map.entry("edi_agreement_document", "host data: transaction sets per agreement"),
Map.entry("edi_transport", "host connection config: addresses and secret variable names"),
Map.entry("edi_certificate", "host data: AS2 certificates (optional)"),
Map.entry("edi_map", "host data; the 'Install sample maps' action adds bundled examples as inactive, unbound drafts"),
Map.entry("edi_map_rule", "with a map; 'Install sample maps' adds the rules of each sample"),
Map.entry("edi_map_sample", "user content: synthetic documents uploaded with Preview Map"),
Map.entry("edi_control_number", "runtime: allocated on the first send for a scope"),
Map.entry("edi_inbox", "runtime: every received document"),
Map.entry("edi_payload", "runtime: raw bytes, only when payload storage is DATABASE"),
Map.entry("edi_interchange", "runtime: every envelope sent or received"),
Map.entry("edi_functional_group", "runtime: every group"),
Map.entry("edi_transaction", "runtime: every document"),
Map.entry("edi_transaction_error", "runtime: validation findings"),
Map.entry("edi_exception", "runtime: failures that need a person"),
Map.entry("edi_event", "audit log: written at each processing step (EdiJournal), shown on the interchange record"),
Map.entry("edi_as2_mdn", "runtime: AS2 receipts sent and received; read to ignore duplicates and replay a sent receipt"),
Map.entry("edi_outbound_request", "runtime: a host record queued to be sent"),
Map.entry("edi_outbound_message", "runtime: the outbound queue"));
private QInstance instance;
/*******************************************************************************
**
*******************************************************************************/
@AfterEach
void clear()
{
QContext.clear();
}
/*******************************************************************************
** Every table is used by runtime code.
*******************************************************************************/
@Test
void everyTableIsUsedAtRuntime() throws Exception
{
List<String> runtimeSource = new ArrayList<>();
try(Stream<Path> files = Files.walk(MAIN))
{
files.filter(p -> p.toString().endsWith(".java") && !p.startsWith(MAIN.resolve("model")) && !p.startsWith(MAIN.resolve("metadata")) && !p.startsWith(MAIN.resolve("app")))
.forEach(p -> runtimeSource.add(read(p)));
}
for(String table : tables())
{
Pattern use = Pattern.compile("\\b" + table + "\\b");
assertThat(runtimeSource.stream().anyMatch(source -> use.matcher(source).find()))
.as(table + " must be read or written by runtime code, not only declared").isTrue();
}
}
/*******************************************************************************
** Every table is seeded or explained.
*******************************************************************************/
@Test
void everyTableIsSeededOrExplained() throws Exception
{
for(String table : tables())
{
assertThat(SEEDED.containsKey(table) || STARTS_EMPTY.containsKey(table)).as(table + " must be seeded or listed in STARTS_EMPTY").isTrue();
assertThat(SEEDED.containsKey(table) && STARTS_EMPTY.containsKey(table)).as(table + " cannot be both").isFalse();
}
assertThat(tables()).containsAll(STARTS_EMPTY.keySet()).containsAll(SEEDED.keySet());
}
/*******************************************************************************
** The built-in guides fill the seeded tables, a second start adds nothing, and
** an administrator's edit survives.
*******************************************************************************/
@Test
void seedIsInsertOnly() throws Exception
{
H2Support.create();
EdiRuntime.current().initialize();
Map<String, Integer> first = counts();
assertThat(first.get("edi_guide")).isEqualTo(BuiltInGuideLoader.RESOURCES.size()).isEqualTo(41);
for(String table : SEEDED.keySet())
{
assertThat(first.get(table)).as(table + " is seeded").isGreaterThan(0);
}
QRecord guide = SystemWrites.query("edi_guide", new QQueryFilter()).getFirst();
SystemWrites.call(() ->
{
SystemWrites.update("edi_guide", new QRecord().withValue("id", guide.getValue("id")).withValue("description", "Edited by an administrator"));
return (null);
});
EdiRuntime.current().initialize();
assertThat(counts()).isEqualTo(first);
assertThat(EdiRecords.query("edi_guide", EdiRecords.equal("id", guide.getValue("id"))).getFirst().getValueString("description")).isEqualTo("Edited by an administrator");
}
/*******************************************************************************
** Every table and the app have bundle help that says how rows get created.
*******************************************************************************/
@Test
void everyTableHasEmptyStateHelp() throws Exception
{
QInstance instance = H2Support.create();
assertThat(EdiHelp.entries()).allSatisfy((key, entry) -> assertThat(entry.path("summary").asText().length()).as(key).isLessThanOrEqualTo(120));
for(String table : tables())
{
List<QHelpContent> help = instance.getTable(table).getHelpContent() == null ? null : instance.getTable(table).getHelpContent().get(EdiHelp.SLOT);
assertThat(help).as(table + " table help").isNotEmpty();
assertThat(help.get(0).getContent()).as(table + " help says how rows get created").contains("How rows get created");
}
assertThat(instance.getApp(EdiRuntime.current().config().getAppName()).getHelpContent()).containsKey(EdiHelp.SLOT);
assertThat(EdiHelp.entries().get("table:edi_map;slot:help").path("content").asText()).contains("Install sample maps");
assertThat(EdiHelp.entries().get("app:edi;slot:help").path("content").asText().indexOf("Partner")).isLessThan(EdiHelp.entries().get("app:edi;slot:help").path("content").asText().indexOf("Transport"));
}
/*******************************************************************************
** Help a host put in the slot first is never replaced by the bundle.
*******************************************************************************/
@Test
void hostHelpIsKept() throws Exception
{
QInstance instance = H2Support.create();
String appName = EdiRuntime.current().config().getAppName();
QHelpContent host = new QHelpContent().withContentAsMarkdown("Host help for partners");
instance.getTable("edi_partner").getHelpContent().put(EdiHelp.SLOT, new ArrayList<>(List.of(host)));
instance.getApp(appName).getHelpContent().put(EdiHelp.SLOT, new ArrayList<>(List.of(host)));
EdiHelp.apply(instance, appName);
assertThat(instance.getTable("edi_partner").getHelpContent().get(EdiHelp.SLOT)).containsExactly(host);
assertThat(instance.getApp(appName).getHelpContent().get(EdiHelp.SLOT)).containsExactly(host);
assertThat(instance.getTable("edi_identity").getHelpContent().get(EdiHelp.SLOT).get(0).getContent()).contains("How rows get created");
}
/*******************************************************************************
** Row counts of the seeded tables.
*******************************************************************************/
private Map<String, Integer> counts() throws Exception
{
Map<String, Integer> counts = new java.util.TreeMap<>();
for(String table : SEEDED.keySet())
{
counts.put(table, SystemWrites.count(table, new QQueryFilter()));
}
return (counts);
}
/*******************************************************************************
** The qbit's tables (the preview download backend table is not data).
*******************************************************************************/
private List<String> tables() throws Exception
{
if(instance == null)
{
instance = H2Support.create();
}
List<String> tables = instance.getTables().values().stream().map(QTableMetaData::getName)
.filter(name -> name.startsWith("edi_") && !name.equals("edi_preview_download")).sorted().toList();
assertThat(tables).hasSize(27);
return (tables);
}
/*******************************************************************************
**
*******************************************************************************/
private static String read(Path path)
{
try
{
return (Files.readString(path, StandardCharsets.UTF_8));
}
catch(Exception e)
{
throw (new IllegalStateException(e));
}
}
}
@@ -744,4 +744,32 @@ class InboundTest
assertThat(rows("sample_order_line")).isEmpty();
assertThat(rows("edi_exception")).extracting(r -> r.getValueString("type")).contains("APPLY");
}
/*******************************************************************************
** An inactive identity stops X12 routing, as it does for EDIFACT.
*******************************************************************************/
@Test
void inactiveIdentityStopsX12Routing() throws Exception
{
setup("004010");
QRecord remote = rows("edi_identity").stream().filter(r -> r.getValue("partnerId") != null).findFirst().orElseThrow();
SystemWrites.call(() ->
{
SystemWrites.update("edi_identity", new QRecord().withValue("id", remote.getValue("id")).withValue("isActive", false));
return (null);
});
receive(raw(), "inactive");
new InboxWorker().runOnce();
assertThat(rows("sample_order")).isEmpty();
assertThat(rows("edi_exception")).extracting(r -> r.getValueString("type")).contains("UNKNOWN_PARTNER");
SystemWrites.call(() ->
{
SystemWrites.update("edi_identity", new QRecord().withValue("id", remote.getValue("id")).withValue("isActive", true));
return (null);
});
receive(Synthetic850.generate(18, "004010", X12Separators.DEFAULT), "active");
new InboxWorker().runOnce();
assertThat(rows("sample_order")).hasSize(1);
}
}
@@ -0,0 +1,189 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi;
import java.io.File;
import java.util.Arrays;
import java.util.List;
import com.kingsrook.qbits.edi.map.SampleMaps;
import com.kingsrook.qbits.edi.processes.InstallSampleMapsProcess;
import com.kingsrook.qbits.edi.storage.EdiRecords;
import com.kingsrook.qbits.edi.support.SystemWrites;
import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** The "Install sample maps" table action on edi_map: standard streamed ETL,
** insert-only by name, installing inactive and unbound drafts.
*******************************************************************************/
class InstallSampleMapsTest
{
/*******************************************************************************
**
*******************************************************************************/
@AfterEach
void clear()
{
QContext.clear();
}
/*******************************************************************************
** Every bundled sample is listed, and every listed sample exists.
*******************************************************************************/
@Test
void theListMatchesTheBundledFiles()
{
String[] files = new File("src/main/resources/edi/maps").list((dir, name) -> name.endsWith("-sample.json"));
assertThat(Arrays.stream(files).map(name -> name.replace("-sample.json", "")).toList()).containsExactlyInAnyOrderElementsOf(SampleMaps.RESOURCES);
}
/*******************************************************************************
** A standard streamed-ETL table action that needs no selected records.
*******************************************************************************/
@Test
void isAStandardTableAction() throws Exception
{
QInstance instance = H2Support.create();
QProcessMetaData process = instance.getProcess(InstallSampleMapsProcess.NAME);
assertThat(process).isNotNull();
assertThat(process.getTableName()).isEqualTo("edi_map");
assertThat(process.getMinInputRecords()).isEqualTo(0);
assertThat(process.getPermissionRules().getPermissionBaseName()).isEqualTo("edi.admin");
assertThat(process.getStepList()).extracting(step -> step.getName()).contains("review", "result");
}
/*******************************************************************************
** Installs every sample as an inactive, unbound draft with its rules; a
** second run adds nothing; an edited map is kept; a deleted one comes back.
*******************************************************************************/
@Test
void installsDraftsInsertOnly() throws Exception
{
H2Support.create();
EdiRuntime.current().initialize();
assertThat(count("edi_map")).isZero();
run();
assertThat(count("edi_map")).isEqualTo(SampleMaps.RESOURCES.size());
List<QRecord> maps = SystemWrites.query("edi_map", new QQueryFilter());
assertThat(maps).allSatisfy(map ->
{
assertThat(map.getValueString("status")).as(map.getValueString("name")).isEqualTo("DRAFT");
assertThat(map.getValueInteger("revision")).isZero();
assertThat(map.getValue("partnerId")).as("unbound to a partner").isNull();
assertThat(map.getValue("guideId")).as("bound to the built-in guide for its document").isNotNull();
});
assertThat(count("edi_agreement_document")).as("bound to no agreement document").isZero();
QRecord sample850 = maps.stream().filter(map -> "Sample 850 004010".equals(map.getValueString("name"))).findFirst().orElseThrow();
int rules = count("edi_map_rule");
assertThat(EdiRecords.query("edi_map_rule", EdiRecords.equal("mapId", sample850.getValue("id")))).hasSize(5);
assertThat(rules).isGreaterThan(SampleMaps.RESOURCES.size());
SystemWrites.call(() ->
{
SystemWrites.update("edi_map", new QRecord().withValue("id", sample850.getValue("id")).withValue("tableName", "my_orders").withValue("status", "ACTIVE"));
return (null);
});
run();
assertThat(count("edi_map")).isEqualTo(SampleMaps.RESOURCES.size());
assertThat(count("edi_map_rule")).isEqualTo(rules);
QRecord edited = EdiRecords.query("edi_map", EdiRecords.equal("id", sample850.getValue("id"))).getFirst();
assertThat(edited.getValueString("tableName")).isEqualTo("my_orders");
assertThat(edited.getValueString("status")).isEqualTo("ACTIVE");
SystemWrites.call(() ->
{
new com.kingsrook.qqq.backend.core.actions.tables.DeleteAction().execute(new com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput("edi_map_rule")
.withQueryFilter(new QQueryFilter().withCriteria("mapId", com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator.EQUALS, sample850.getValue("id"))));
new com.kingsrook.qqq.backend.core.actions.tables.DeleteAction().execute(new com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput("edi_map")
.withPrimaryKey(sample850.getValue("id")));
return (null);
});
assertThat(SampleMaps.RESOURCES).hasSize(32);
assertThat(SystemWrites.call(SampleMaps::missing)).containsExactly("850-004010");
run();
assertThat(count("edi_map")).isEqualTo(SampleMaps.RESOURCES.size());
assertThat(EdiRecords.query("edi_map_rule", EdiRecords.equal("mapId", EdiRecords.query("edi_map", EdiRecords.equal("name", "Sample 850 004010")).getFirst().getValue("id")))).hasSize(5);
}
/*******************************************************************************
** A host that already has a map of the same name keeps it, and gets no copy.
*******************************************************************************/
@Test
void neverTouchesAMapTheHostAlreadyHas() throws Exception
{
H2Support.create();
EdiRuntime.current().initialize();
SystemWrites.call(() -> SystemWrites.insert("edi_map", new QRecord().withValue("name", "Sample 850 004010").withValue("direction", "INBOUND")
.withValue("transactionSet", "850").withValue("version", "004010").withValue("status", "ACTIVE").withValue("revision", 3).withValue("tableName", "host_orders")));
run();
List<QRecord> same = EdiRecords.query("edi_map", EdiRecords.equal("name", "Sample 850 004010"));
assertThat(same).hasSize(1);
assertThat(same.getFirst().getValueString("tableName")).isEqualTo("host_orders");
assertThat(same.getFirst().getValueString("status")).isEqualTo("ACTIVE");
assertThat(count("edi_map")).isEqualTo(SampleMaps.RESOURCES.size());
assertThat(count("edi_map_rule")).as("no rules were added to the host's map").isEqualTo(
SampleMaps.RESOURCES.stream().filter(r -> !r.equals("850-004010")).mapToInt(InstallSampleMapsTest::ruleCount).sum());
}
/*******************************************************************************
** Run the real process through the standard runner (frontend steps skipped).
*******************************************************************************/
private void run() throws Exception
{
RunProcessInput input = new RunProcessInput();
input.setProcessName(InstallSampleMapsProcess.NAME);
input.setFrontendStepBehavior(RunProcessInput.FrontendStepBehavior.SKIP);
new RunProcessAction().execute(input);
}
/*******************************************************************************
** The number of rules a bundled sample carries.
*******************************************************************************/
private static int ruleCount(String resource)
{
try
{
return (SampleMaps.read(resource).path("rules").size());
}
catch(java.io.IOException e)
{
throw (new IllegalStateException(e));
}
}
/*******************************************************************************
**
*******************************************************************************/
private int count(String table) throws Exception
{
return (SystemWrites.count(table, new QQueryFilter()));
}
}
@@ -0,0 +1,213 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi;
import java.util.List;
import com.kingsrook.qbits.edi.flow.OutboundEnqueue;
import com.kingsrook.qbits.edi.flow.OutboundSourceTables;
import com.kingsrook.qbits.edi.storage.EdiRecords;
import com.kingsrook.qbits.edi.support.SystemWrites;
import com.kingsrook.qqq.backend.core.actions.tables.listeners.RecordChangeType;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** The record-change listener only costs queries for configured source tables,
** and its cache follows agreement and document changes.
*******************************************************************************/
class OutboundSourceTablesTest
{
private QRecord document;
private QRecord agreement;
/*******************************************************************************
** One active RECORD_CHANGE agreement document, for sample_order.
*******************************************************************************/
@BeforeEach
void setup() throws Exception
{
H2Support.create();
RetailSupport.tables();
EdiRuntime.current().initialize();
SystemWrites.call(() ->
{
QRecord partner = SystemWrites.insert("edi_partner", new QRecord().withValue("name", "Retail test").withValue("code", "RETAIL").withValue("status", "ACTIVE"));
QRecord local = SystemWrites.insert("edi_identity", new QRecord().withValue("standard", "X12").withValue("interchangeQualifier", "ZZ").withValue("interchangeId", "QQQTEST").withValue("applicationCode", "QQQTEST"));
QRecord remote = SystemWrites.insert("edi_identity", new QRecord().withValue("partnerId", partner.getValue("id")).withValue("standard", "X12").withValue("interchangeQualifier", "ZZ").withValue("interchangeId", "SYNTHETIC").withValue("applicationCode", "SYNTHETIC"));
QRecord transport = SystemWrites.insert("edi_transport", new QRecord().withValue("name", "retail-out").withValue("type", "GATEWAY").withValue("direction", "BOTH").withValue("gatewayEndpointName", "retail-outbox").withValue("isActive", true).withValue("maxAttempts", 3));
agreement = SystemWrites.insert("edi_agreement", new QRecord().withValue("partnerId", partner.getValue("id")).withValue("localIdentityId", local.getValue("id")).withValue("partnerIdentityId", remote.getValue("id"))
.withValue("standard", "X12").withValue("usageIndicator", "T").withValue("isActive", true).withValue("outboundTransportId", transport.getValue("id")));
QRecord map = RetailSupport.installMap("855", "004010");
document = SystemWrites.insert("edi_agreement_document", outboundDocument(map, "sample_order"));
return (null);
});
}
/*******************************************************************************
** Reset context.
*******************************************************************************/
@AfterEach
void clear()
{
QContext.clear();
}
/*******************************************************************************
** Writes to a table without a definition neither apply nor reload definitions.
*******************************************************************************/
@Test
void unconfiguredTableDoesNoWork() throws Exception
{
OutboundEnqueue listener = new OutboundEnqueue();
assertThat(listener.appliesTo("sample_order", RecordChangeType.INSERT)).isTrue();
long loads = sources().loadCount();
assertThat(listener.appliesTo("sample_invoice", RecordChangeType.INSERT)).isFalse();
assertThat(listener.appliesTo("sample_invoice", RecordChangeType.UPDATE)).isFalse();
assertThat(listener.appliesTo("sample_order", RecordChangeType.DELETE)).isFalse();
SystemWrites.call(() -> SystemWrites.transaction(() ->
{
for(int i = 0; i < 5; i++)
{
RetailSupport.saveSource("sample_invoice", new QRecord().withValue("invoiceNumber", "INV-" + i));
}
return (null);
}));
assertThat(sources().loadCount()).isEqualTo(loads);
assertThat(rows("edi_outbound_request")).isEmpty();
}
/*******************************************************************************
** Configured tables still enqueue, once per record, from one cached load.
*******************************************************************************/
@Test
void configuredTableEnqueues() throws Exception
{
SystemWrites.call(() -> SystemWrites.transaction(() -> RetailSupport.saveSource("sample_order", RetailSupport.source())));
long loads = sources().loadCount();
SystemWrites.call(() -> SystemWrites.transaction(() -> RetailSupport.saveSource("sample_order", RetailSupport.source().withValue("poNumber", "PO-2"))));
assertThat(sources().loadCount()).isEqualTo(loads);
assertThat(rows("edi_outbound_request")).hasSize(2);
}
/*******************************************************************************
** Document and agreement changes invalidate the cached tables.
*******************************************************************************/
@Test
void definitionChangesInvalidate() throws Exception
{
OutboundEnqueue listener = new OutboundEnqueue();
assertThat(listener.appliesTo("sample_order", RecordChangeType.INSERT)).isTrue();
assertThat(listener.appliesTo("edi_agreement_document", RecordChangeType.DELETE)).isTrue();
update("edi_agreement_document", document.getValue("id"), "isActive", false);
assertThat(listener.appliesTo("sample_order", RecordChangeType.INSERT)).isFalse();
update("edi_agreement_document", document.getValue("id"), "isActive", true);
assertThat(listener.appliesTo("sample_order", RecordChangeType.INSERT)).isTrue();
update("edi_agreement", agreement.getValue("id"), "isActive", false);
assertThat(listener.appliesTo("sample_order", RecordChangeType.INSERT)).isFalse();
SystemWrites.call(() -> SystemWrites.transaction(() -> RetailSupport.saveSource("sample_order", RetailSupport.source())));
assertThat(rows("edi_outbound_request")).isEmpty();
update("edi_agreement", agreement.getValue("id"), "isActive", true);
assertThat(listener.appliesTo("sample_order", RecordChangeType.INSERT)).isTrue();
}
/*******************************************************************************
** A transaction that adds a definition sees it before commit; a rolled-back
** one leaves the cache on committed data.
*******************************************************************************/
@Test
void changingTransactionSeesItsOwnDefinitions() throws Exception
{
QRecord map = SystemWrites.call(() -> EdiRecords.query("edi_map", new QQueryFilter()).getFirst());
SystemWrites.call(() -> SystemWrites.transaction(() ->
{
SystemWrites.insert("edi_agreement_document", outboundDocument(map, "sample_invoice").withValue("transactionSet", "810"));
RetailSupport.saveSource("sample_invoice", new QRecord().withValue("invoiceNumber", "INV-1"));
return (null);
}));
assertThat(rows("edi_outbound_request")).hasSize(1);
assertThat(new OutboundEnqueue().appliesTo("sample_invoice", RecordChangeType.INSERT)).isTrue();
try
{
SystemWrites.call(() -> SystemWrites.transaction(() ->
{
SystemWrites.insert("edi_agreement_document", outboundDocument(map, "sample_inventory").withValue("transactionSet", "846"));
throw new IllegalStateException("rollback synthetic write");
}));
}
catch(Exception e)
{
////////////////////////////
// expected - rolled back //
////////////////////////////
}
assertThat(new OutboundEnqueue().appliesTo("sample_inventory", RecordChangeType.INSERT)).isFalse();
}
/*******************************************************************************
** An active outbound RECORD_CHANGE document for the source table.
*******************************************************************************/
private QRecord outboundDocument(QRecord map, String sourceTable)
{
return (new QRecord().withValue("agreementId", agreement.getValue("id")).withValue("direction", "OUTBOUND").withValue("transactionSet", "855")
.withValue("version", "004010").withValue("functionalIdCode", "PR").withValue("guideId", map.getValue("guideId")).withValue("mapId", map.getValue("id"))
.withValue("sourceTableName", sourceTable).withValue("outboundTrigger", "RECORD_CHANGE").withValue("batchMode", "IMMEDIATE").withValue("ackExpectation", "997").withValue("ackDueMinutes", 1).withValue("isActive", true));
}
/*******************************************************************************
** Update one field through QQQ, outside any transaction.
*******************************************************************************/
private static void update(String table, Object id, String field, Object value) throws Exception
{
SystemWrites.call(() ->
{
SystemWrites.update(table, new QRecord().withValue("id", (java.io.Serializable) id).withValue(field, (java.io.Serializable) value));
return (null);
});
}
/*******************************************************************************
** The runtime cache under test.
*******************************************************************************/
private static OutboundSourceTables sources()
{
return (EdiRuntime.current().outboundSources());
}
/*******************************************************************************
** All rows.
*******************************************************************************/
private static List<QRecord> rows(String table) throws Exception
{
return (SystemWrites.call(() -> EdiRecords.query(table, new QQueryFilter())));
}
}
@@ -345,4 +345,44 @@ class OutboundTest
var cron = com.kingsrook.qbits.edi.processes.OutboundProcesses.process("edi_test_cron", config, null, "0 0 * * * ?").getSchedule();
assertThat(cron.getCronTimeZoneId()).isEqualTo("UTC");
}
/*******************************************************************************
** A draft map bound to an outbound document is never used: the request fails
** as a mapping error and nothing is queued. An inactive identity stops the
** document from being built at all; its requests wait.
*******************************************************************************/
@Test
void draftMapAndInactiveIdentityStopSending() throws Exception
{
setup();
SystemWrites.call(() ->
{
SystemWrites.update("edi_map", new QRecord().withValue("id", document.getValue("mapId")).withValue("status", "DRAFT"));
return (null);
});
SystemWrites.call(() -> SystemWrites.transaction(() -> RetailSupport.saveSource("sample_order", RetailSupport.source())));
new OutboundWorker().runOnce();
assertThat(rows("edi_outbound_message")).isEmpty();
assertThat(rows("edi_outbound_request")).extracting(r -> r.getValueString("status")).containsExactly("FAILED");
assertThat(rows("edi_exception")).extracting(r -> r.getValueString("type")).contains("MAPPING");
SystemWrites.call(() ->
{
SystemWrites.update("edi_map", new QRecord().withValue("id", document.getValue("mapId")).withValue("status", "ACTIVE"));
SystemWrites.update("edi_identity", new QRecord().withValue("id", agreement.getValue("partnerIdentityId")).withValue("isActive", false));
return (null);
});
SystemWrites.call(() -> SystemWrites.transaction(() -> RetailSupport.saveSource("sample_order", RetailSupport.source().withValue("poNumber", "DEMO-1002"))));
assertThat(new OutboundWorker().runOnce()).isZero();
assertThat(rows("edi_outbound_message")).isEmpty();
assertThat(rows("edi_outbound_request")).extracting(r -> r.getValueString("status")).contains("PENDING");
SystemWrites.call(() ->
{
SystemWrites.update("edi_identity", new QRecord().withValue("id", agreement.getValue("partnerIdentityId")).withValue("isActive", true));
return (null);
});
new OutboundWorker().runOnce();
assertThat(rows("edi_outbound_message")).hasSize(1);
}
}
@@ -359,7 +359,7 @@ class PhiCanaryTest
var enforcement = instance.getEnforcePermissionsInCoreActions();
instance.setEnforcePermissionsInCoreActions(true);
var fullSession = QContext.getQSession();
var viewer = new com.kingsrook.qqq.backend.core.model.session.QSession().withPermissions("edi.admin").withSecurityKeyValue("edi.admin", true).withSecurityKeyValue("edi.guide.loader", true);
var viewer = new com.kingsrook.qqq.backend.core.model.session.QSession().withPermissions("edi.admin", "edi.hasAccess", "edi.admin.hasAccess").withSecurityKeyValue("edi.admin", true).withSecurityKeyValue("edi.guide.loader", true);
for(var table : instance.getTables().values())
{
if(!table.getName().startsWith("edi_") || table.getName().equals(PreviewDownloadBackend.TABLE))
@@ -68,7 +68,7 @@ class EdiDemoServerTest
Set<String> properties = new HashSet<>();
new com.fasterxml.jackson.databind.ObjectMapper().readTree(json).fieldNames().forEachRemaining(properties::add);
assertThat(properties).isSubsetOf("sendAsEdiTables", "defaultBackendNameForTables", "backendName", "appName", "appLabel", "parentAppName",
"schedulerName", "maxSegmentsPerTransaction", "partnerLocks", "payloadBucketAlias", "esbDestinationName");
"schedulerName", "maxSegmentsPerTransaction", "partnerLocks", "payloadBucketAlias", "esbDestinationName", "inboundDelayedAfterMinutes");
assertThat(calls.get()).isZero();
}
}