Compare commits

..
Author SHA1 Message Date
james.maesandClaude Opus 5.5 d1eb78c1df fix(outbound): the record-change listener only costs queries for configured source tables
OutboundEnqueue applied to every non-edi_ table, and per record opened a
system session and queried edi_agreement_document, so every insert and
update of audit, auditDetail, worm_audit_chain, jobs, alerts and the
rest paid for it (and UPDATEs fetched old records for it).

- OutboundSourceTables (per EdiRuntime) caches the source tables that
  have an active RECORD_CHANGE outbound document on an active agreement,
  loaded from committed data. appliesTo answers from it, so other tables
  do no work at all.
- Changes to edi_agreement / edi_agreement_document invalidate it, and
  again after their transaction commits (a reload that raced the change
  is discarded); until then the changing transaction bypasses the cache
  and sees its own definitions. A 60s TTL covers other processes.
- onRecordsChanged resolves definitions (documents + active agreements)
  once per event, not once per record. MANUAL and ESB_EVENT enqueue keep
  their semantics.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 07:24:33 -05:00
james.maes 2810e00e9c Merge pull request #22 from QRun-IO/fix/producer-version-from-build
fix: qbit metadata version comes from the build
2026-10-15 01:42:56 -05:00
james.maesandClaude Opus 5.5 89a95737a5 fix: the qbit metadata version comes from the build, not a hardcoded constant
The producer read a literal version, so released artifacts reported a stale
or -SNAPSHOT version. qbit-edi.properties (version=${project.version}) is
Maven-filtered and the producer loads it; tests assert producer == bundle ==
artifact version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 01:29:54 -05:00
james.maes 86b8dbf13d Merge pull request #21 from QRun-IO/fix/help-bundle-version-test
test(help): bundle version test compares with the artifact version
2026-10-14 23:18:42 -05:00
james.maesandClaude Opus 5.5 4471a13e78 test(help): compare the help bundle version with the artifact version, not a hardcoded SNAPSHOT
The bundle version is now Maven-filtered, so a literal SNAPSHOT in the test
fails every release build. Surefire passes the artifact version in
qbit.artifactVersion.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 23:07:33 -05:00
james.maes c39d78901c Merge pull request #20 from QRun-IO/fix/help-bundle-version
fix(help): help bundle version comes from the build, not a hardcoded SNAPSHOT
2026-10-14 22:37:20 -05:00
james.maesandClaude Opus 5.5 1e25fe54d2 fix(help): the help bundle carries the artifact version (Maven-filtered) instead of a hardcoded SNAPSHOT
On release branches the bundle's version and the producer's differed, so the
bundle test that asserts they match failed, and released bundles recorded
SNAPSHOT as their provenance. help/*.yaml is now filtered:
version: ${project.version}.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 22:09:48 -05:00
james.maes 0fe99db48d Merge pull request #19 from QRun-IO/feature/help-bundle
feat(help): qbit-edi help bundle via QQQ help bundles, with coverage gate
2026-10-14 20:45:30 -05:00
james.maesandClaude Opus 5.5 b08057180b feat(help): ship the qbit-edi help bundle through QQQ help bundles, with coverage gate
Host-neutral help (deployment-specific facts removed), registered with
HelpBundles, gated by HelpBundleAssertions. QQQ 4.1.0-RC.7.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 20:27:22 -05:00
james.maes 79b85a163d Merge pull request #18 from QRun-IO/feature/tables-populated-defaults
feat: every edi table explains itself; Setup status; Install sample maps
2026-10-14 17:42:45 -05:00
james.maesandClaude Opus 5.5 4e32aebc00 fix: X12 routing honors identity isActive; tests for draft maps and host help
- X12 inbound routing and outbound building ignore inactive identities, as
  EDIFACT does (UPGRADE note).
- Test that a draft map bound to an outbound document is never used (it
  already was not) and that inactive identities stop sending.
- hostHelpIsKept test; drop the unfounded concurrency claim on Install
  sample maps; plain wording in the help bundle header.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:32:54 -05:00
james.maesandClaude Opus 5.5 7bafab62cc docs: upgrade note wording
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:17:54 -05:00
james.maesandClaude Opus 5.5 a1914f7553 feat: table help, Setup status, Install sample maps, tables guardrail
Every edi_* table and the app get help bundle text saying how rows get
created and the setup order. The home gets a Setup status section. A new
Install sample maps table action adds the bundled synthetic maps as
inactive, unbound drafts (insert-only by name).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:17:41 -05:00
james.maes 51a5eaef74 Merge pull request #17 from QRun-IO/fix/repeat-schedule-time-zone
fix: no cron time zone on repeat-seconds worker schedules
2026-10-14 15:10:02 -05:00
james.maesandClaude Opus 5.5 47c3aac1b2 fix: no cron time zone on repeat-seconds worker schedules
OutboundProcesses.process set cronTimeZoneId("UTC") on every schedule.
Core instance validation refuses a time zone on a non-cron schedule, so a
host that configures a scheduler (kof22) failed validation at start. Only
cron schedules get UTC now; OutboundTest covers both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 14:59:12 -05:00
james.maes bf8f40959a Merge pull request #16 from QRun-IO/feature/orb-0.6.9
ci: qqq-orb 0.6.9 (refresh SNAPSHOT dependencies)
2026-10-14 10:57:01 -05:00
james.maesandClaude Opus 5.5 f7c031b793 ci: qqq-orb 0.6.9 (refresh SNAPSHOT dependencies)
Moves qqq-orb@0.6.8 to qqq-orb@0.6.9, which passes --update-snapshots so cached SNAPSHOT dependencies are re-checked on every build (QRun-IO/qqq-orb#22).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 10:06:51 -05:00
james.maes cd9423e966 Merge pull request #15 from QRun-IO/feature/phase-f-mapper-api
feat: phase f1 - spec'd UI API extension for the EDI mapper and sandbox
2026-10-14 09:58:48 -05:00
28 changed files with 3922 additions and 49 deletions

No files matched your search

+1 -1
View File
@@ -1,7 +1,7 @@
version: 2.1
orbs:
qqq-orb: kingsrook/qqq-orb@0.6.8
qqq-orb: kingsrook/qqq-orb@0.6.9
commands:
configure-package-install:
+13
View File
@@ -4,8 +4,21 @@ Following [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and [Semantic
## [Unreleased]
### Changed
- Derive qbit metadata version from the Maven artifact version, keeping release metadata and help bundle versions in sync.
- Require QQQ 4.1.0-RC.7 for core help-bundle registration and coverage validation.
- X12 routing and sending honor identity `isActive`, matching EDIFACT.
### Added
- Complete, host-neutral `help/qbit-edi.yaml` bundle registered with QQQ core: app, tables, fields, actions, widgets and possible-value sources, with a coverage gate and HOST overrides. Table/app fallback remains available without `helpContent`.
- Setup status section on the EDI home (partner, identities, agreement, agreement documents, transport, certificates, guides, maps) with the next step; it counts only the active rows the runtime uses.
- "Install sample maps" action on Maps: standard streamed ETL, insert-only by name, installs the 32 bundled synthetic maps as draft, unbound maps bound to the built-in guide.
- `EdiTablesGuardrailTest`: every table is used at runtime, seeded or listed as starting empty, and has help.
- Phase f1 `edi` 1.0.0 spec'd UI API: guide tree, map target/transform metadata, encrypted base64 sample preview with unsaved rule overrides, and per-rule validation under `/qqq/v1/qbit/edi`.
- Versioned `Edi...V1` schemas and generated OpenAPI fragment with the RC.6 contract/compatibility checks; real HTTP strict-mode permission, PHI, upload, isolation and validation tests.
- Shared sandbox/record-action preview service, current/original guide selection, caller-session transforms, structured revision/expected diffs and value-free PHI diagnostics.
+41 -1
View File
@@ -34,7 +34,7 @@ ALTER TABLE edi_agreement ADD COLUMN syntax_identifier VARCHAR(255);
## Requirements and installation
Java 21, QQQ **4.1.0-RC.6+**, an RDBMS backend, and `com.kingsrook.qbits:qbit-edi:0.1.0-SNAPSHOT`.
Java 21, QQQ **4.1.0-RC.7+**, an RDBMS backend, and `com.kingsrook.qbits:qbit-edi:0.1.0-SNAPSHOT`.
Apply [the PostgreSQL schema](docs/ddl/edi-postgres.sql). Targets and their associations must
share the EDI backend so each transaction set can commit or roll back atomically.
@@ -60,6 +60,23 @@ The `edi` app has Activity/Setup sections, typed home widgets, record screens an
Review/Result actions. Main tables have RC.6 search-field metadata; `EdiIntegrations.mainTables()`
also exposes the list for host search registration.
## In-app help
The producer registers [help/qbit-edi.yaml](src/main/resources/help/qbit-edi.yaml), a `QBIT`
bundle versioned with this artifact. It covers the EDI app, tables, fields, actions, widgets,
and possible-value sources. Live `{{...}}` references follow the current metadata.
Hosts can register their own `layer: HOST` bundle with `HelpBundles.register(qInstance,
"help/my-host.yaml")`, using the same keys and roles to override selected entries. Enable
core's `helpContent` provider with provenance fields, then call
`QInstanceHelpContentManager.loadHelpContent(qInstance)` and `HelpBundles.syncOnStartup(qInstance)`
after defining metadata. Core preserves user edits during sync. If you configure a different
EDI app name, use that name in your host's app help key.
Registration is safe without `helpContent`; those hosts retain the table/app help fallback.
`EdiHelpBundleTest` runs the core `HelpBundleAssertions` coverage gate and checks widget and
possible-value-source coverage during `./mvnw verify`, without host-specific exclusions.
## Tables
There are **26 persisted tables**, whose SQL names equal their QQQ names.
@@ -167,6 +184,29 @@ synthetic EDI on the standard Review step. Browser uploads require the host's RC
staging. The alias encrypts streams and persists samples; this process accepts no filesystem archive.
Samples must contain synthetic data only.
## Setup and starter data
No Liquibase changelog or host include is needed: the qbit seeds its own reference data. When the
host calls `EdiRuntime.initialize()` at startup (after the schema exists) the 41 built-in guides
load, with their segments, elements, rules, code lists and code values. It only changes a guide when
the bundled content changes, so edits to your own rows are kept.
Everything else is host or runtime data, and every table explains itself when empty (help bundle
`help/qbit-edi.yaml`). Set up in this order:
1. Partner.
2. Identities: one of yours (no partner) and one per partner.
3. Agreement: pairs your identity with the partner's, for a standard and test or production.
4. Agreement documents: one per transaction set and direction, with the guide and optionally a map.
5. Transport (AS2, SFTP, HTTPS), linked on the agreement. Certificates only for AS2 signing and encryption.
The **Setup status** section on the home shows each step, whether it is done (counting only the
active rows the runtime uses) and the next step.
Maps start empty. The **Install sample maps** action on Maps adds the 32 bundled synthetic examples
as draft maps with no partner and no agreement document, by name and never over an existing map. Point
a map at your own table, preview it, activate it, then select it on an agreement document.
## Permissions
Full table permissions include `edi.partner.read|insert|edit|delete`, `edi.map.*`,
+9
View File
@@ -0,0 +1,9 @@
# Upgrade notes
## Unreleased
- No schema change and no Liquibase include.
- Behavior change: an inactive identity (`isActive` false) now stops X12 routing and sending, as it already did for EDIFACT. Inbound documents from or to it are rejected as an unknown partner, and outbound requests for its agreements wait until it is active. Identities default to active, so only identities someone switched off are affected.
- Hosts that already call `EdiRuntime.initialize()` at startup keep getting the 41 built-in guides and code lists. A host that never called it will now see "Needs setup" for Guides and code lists on the home.
- New home section "Setup status" (widget `edi_home_setup`).
- New table action "Install sample maps" on Maps, permission `edi.admin`. It is never run automatically and only adds draft maps whose name does not exist yet.
+30 -1
View File
@@ -126,7 +126,36 @@
</dependencies>
<build>
<resources>
<!-- help bundles carry the artifact version (QQQ help bundle provenance) -->
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
<includes>
<include>help/*.yaml</include>
<include>qbit-edi.properties</include>
</includes>
</resource>
<resource>
<directory>src/main/resources</directory>
<filtering>false</filtering>
<excludes>
<exclude>help/*.yaml</exclude>
<exclude>qbit-edi.properties</exclude>
</excludes>
</resource>
</resources>
<plugins>
<!-- the artifact version, for the help bundle version test -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<systemPropertyVariables combine.children="append">
<qbit.artifactVersion>${project.version}</qbit.artifactVersion>
</systemPropertyVariables>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-checkstyle-plugin</artifactId>
@@ -213,7 +242,7 @@
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<qqq.snapshot.version>4.1.0-RC.6</qqq.snapshot.version>
<qqq.snapshot.version>4.1.0-RC.7</qqq.snapshot.version>
</properties>
<dependencyManagement>
<dependencies>
@@ -21,9 +21,13 @@
package com.kingsrook.qbits.edi;
import java.io.IOException;
import java.io.InputStream;
import java.util.Properties;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerMultiOutput;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.help.bundles.HelpBundles;
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitMetaDataProducer;
@@ -36,13 +40,41 @@ public class EdiQBitProducer implements QBitMetaDataProducer<EdiQBitConfig>
{
public static final String GROUP_ID = "com.kingsrook.qbits";
public static final String ARTIFACT_ID = "qbit-edi";
public static final String VERSION = "0.1.0";
public static final String VERSION = loadVersion();
private EdiQBitConfig qBitConfig;
private String namespace;
/*******************************************************************************
** Read the artifact version from the Maven-filtered resource.
*******************************************************************************/
private static String loadVersion()
{
try(InputStream inputStream = EdiQBitProducer.class.getClassLoader().getResourceAsStream("qbit-edi.properties"))
{
if(inputStream == null)
{
throw new IllegalStateException("Missing qbit-edi.properties; build with Maven to supply the artifact version");
}
Properties properties = new Properties();
properties.load(inputStream);
String version = properties.getProperty("version");
if(version == null || version.isBlank() || version.contains("${"))
{
throw new IllegalStateException("Missing or unfiltered version in qbit-edi.properties; build with Maven");
}
return (version);
}
catch(IOException e)
{
throw new IllegalStateException("Could not read the artifact version from qbit-edi.properties", e);
}
}
/*******************************************************************************
** Fluent configuration setter.
*******************************************************************************/
@@ -130,6 +162,16 @@ public class EdiQBitProducer implements QBitMetaDataProducer<EdiQBitConfig>
MetaDataProducerMultiOutput output = QBitMetaDataProducer.super.produce(instance);
output.addSelfToInstance(instance);
EdiRuntime.install(instance, qBitConfig);
// Registration is metadata-only; core startup sync skips hosts without helpContent.
HelpBundles.register(instance, "help/qbit-edi.yaml");
// The app name is configurable; its bundle key must follow the installed app.
HelpBundles.of(instance).getBundles().getLast().getEntries().stream()
.filter(entry -> "app:edi;slot:help".equals(entry.getKey()))
.forEach(entry -> entry.setKey("app:" + qBitConfig.getAppName() + ";slot:help"));
if(instance.getTable("helpContent") == null)
{
com.kingsrook.qbits.edi.metadata.EdiHelp.apply(instance, qBitConfig.getAppName());
}
instance.withSupplementalMetaData(new com.kingsrook.qbits.edi.metadata.EdiRecordLabels());
instance.withRecordChangeListener(new com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference(com.kingsrook.qbits.edi.flow.OutboundEnqueue.class));
instance.withSupplementalMetaData(new com.kingsrook.qbits.edi.transport.EdiTransportPluginsMetaData()
@@ -64,6 +64,7 @@ public final class EdiRuntime
private final Ack999Builder ack999 = new Ack999Builder();
private final TA1Builder ta1 = new TA1Builder();
private final InboundAckReader acknowledgements = new InboundAckReader();
private final com.kingsrook.qbits.edi.flow.OutboundSourceTables outboundSources = new com.kingsrook.qbits.edi.flow.OutboundSourceTables();
/*******************************************************************************
** Construct services without resolving keys until content is stored or opened.
@@ -95,6 +96,22 @@ public final class EdiRuntime
return (Objects.requireNonNull(INSTANCES.get(QContext.getQInstance()), "EDI runtime has not been installed"));
}
/*******************************************************************************
** Services for the current QQQ context, or null when none are installed.
*******************************************************************************/
public static EdiRuntime find()
{
return (QContext.getQInstance() == null ? null : INSTANCES.get(QContext.getQInstance()));
}
/*******************************************************************************
** Cached source tables that have active RECORD_CHANGE outbound definitions.
*******************************************************************************/
public com.kingsrook.qbits.edi.flow.OutboundSourceTables outboundSources()
{
return (outboundSources);
}
/*******************************************************************************
** Call at host startup after DDL, with the service session established.
*******************************************************************************/
@@ -69,7 +69,7 @@ public class EdiHome extends AbstractWidgetRenderer
public static final List<String> SETUP = List.of("edi_partner", "edi_agreement", "edi_transport", "edi_certificate", "edi_guide", "edi_code_list", "edi_map");
private static final Map<String, String> LABELS = new LinkedHashMap<>();
private static final Map<String, String> DOCUMENT_NAMES = new HashMap<>();
private static final List<String> TABLES = List.of("queue", "acks", "errors", "volume", "system");
private static final List<String> TABLES = List.of("setup", "queue", "acks", "errors", "volume", "system");
static
{
@@ -85,6 +85,7 @@ public class EdiHome extends AbstractWidgetRenderer
LABELS.put("overdue", "Overdue acks");
LABELS.put("dead", "Dead letters");
LABELS.put("certificates", "Certificates expiring (30 days)");
LABELS.put("setup", "Setup status");
LABELS.put("queue", "Exception queue");
LABELS.put("acks", "Overdue acknowledgements");
LABELS.put("errors", "Recent errors (24 hours)");
@@ -138,6 +139,7 @@ public class EdiHome extends AbstractWidgetRenderer
case "dead" -> count("edi_exception", openExceptions().withCriteria("type", QCriteriaOperator.IN, "DEAD_LETTER", "TRANSPORT"));
case "certificates" -> count("edi_certificate", new QQueryFilter().withCriteria("notAfter", QCriteriaOperator.LESS_THAN_OR_EQUALS, now.plus(30, ChronoUnit.DAYS))
.withCriteria("status", QCriteriaOperator.EQUALS, "ACTIVE"));
case "setup" -> setup();
case "queue" -> queue(now);
case "acks" -> acks(now);
case "errors" -> errors(now);
@@ -191,6 +193,48 @@ public class EdiHome extends AbstractWidgetRenderer
/*******************************************************************************
** The setup steps in order, with their status and the next step. Counts are
** taken as the viewer; a table the viewer cannot read shows as not visible.
*******************************************************************************/
private static RenderWidgetOutput setup() throws QException
{
QQueryFilter active = new QQueryFilter().withCriteria("isActive", QCriteriaOperator.EQUALS, true);
EdiSetupStatus.Counts counts = new EdiSetupStatus.Counts(safeCount("edi_partner", new QQueryFilter()),
safeCount("edi_identity", new QQueryFilter().withCriteria("isActive", QCriteriaOperator.EQUALS, true).withCriteria("partnerId", QCriteriaOperator.IS_BLANK)),
safeCount("edi_identity", new QQueryFilter().withCriteria("isActive", QCriteriaOperator.EQUALS, true).withCriteria("partnerId", QCriteriaOperator.IS_NOT_BLANK)),
safeCount("edi_agreement", new QQueryFilter()), safeCount("edi_agreement", active),
safeCount("edi_agreement_document", new QQueryFilter()), safeCount("edi_agreement_document", active),
safeCount("edi_transport", new QQueryFilter()), safeCount("edi_transport", active),
safeCount("edi_certificate", new QQueryFilter()), safeCount("edi_guide", new QQueryFilter()), safeCount("edi_code_list", new QQueryFilter()),
safeCount("edi_map", new QQueryFilter()), safeCount("edi_map", new QQueryFilter().withCriteria("status", QCriteriaOperator.EQUALS, "ACTIVE")));
List<Map<String, Object>> rows = new ArrayList<>();
for(EdiSetupStatus.Item item : EdiSetupStatus.items(counts))
{
rows.add(row("Step", item.step(), "Status", item.state().label() + (item.detail().isEmpty() ? "" : ": " + item.detail()), "Next step", item.nextStep().isEmpty() ? "Nothing to do." : item.nextStep()));
}
return (table(List.of("Step", "Status", "Next step"), rows, "edi_partner", new QQueryFilter(), "Open partners"));
}
/*******************************************************************************
** A count the viewer may not be allowed to read: -1 instead of an error.
*******************************************************************************/
private static int safeCount(String table, QQueryFilter filter)
{
try
{
return (countOf(table, filter));
}
catch(QException | RuntimeException e)
{
return (-1);
}
}
/*******************************************************************************
** Open exceptions, oldest first, in four narrow columns.
*******************************************************************************/
@@ -0,0 +1,156 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi.app;
import java.util.ArrayList;
import java.util.List;
/*******************************************************************************
** The Setup status on the EDI home: each setup step in the order it has to be
** done (partner, identity, agreement, document, transport), whether it is
** done, and the next step. The counts use the same conditions the runtime
** uses to route documents (active rows), so the card never reports "set up"
** for rows the runtime ignores.
*******************************************************************************/
public final class EdiSetupStatus
{
/*******************************************************************************
** DONE: working. TODO: the exchange needs it. BUILT_IN: provided by the
** qbit. OPTIONAL: only some setups need it. UNKNOWN: the viewer cannot see it.
*******************************************************************************/
public enum State
{
DONE("Set up"), TODO("Needs setup"), BUILT_IN("Built in"), OPTIONAL("Optional"), UNKNOWN("Not visible to you");
private final String label;
/*******************************************************************************
**
*******************************************************************************/
State(String label)
{
this.label = label;
}
/*******************************************************************************
** The label shown on the card.
*******************************************************************************/
public String label()
{
return (label);
}
}
/*******************************************************************************
** One line of the card.
*******************************************************************************/
public record Item(String step, String tableName, State state, String detail, String nextStep)
{
}
/*******************************************************************************
** The row counts the card is built from; -1 means the viewer cannot read it.
** Active counts are rows with isActive true (what the runtime uses).
*******************************************************************************/
public record Counts(int partners, int ownIdentities, int partnerIdentities, int agreements, int activeAgreements, int agreementDocuments,
int activeAgreementDocuments, int transports, int activeTransports, int certificates, int guides, int codeLists, int maps, int activeMaps)
{
}
/*******************************************************************************
** Utility class.
*******************************************************************************/
private EdiSetupStatus()
{
}
/*******************************************************************************
** The items, in setup order.
*******************************************************************************/
public static List<Item> items(Counts c)
{
List<Item> items = new ArrayList<>();
items.add(item("1. Partner", "edi_partner", c.partners(), c.partners(), "partner", "Add the trading partner with New on Partners."));
items.add(identities(c));
items.add(item("3. Agreement", "edi_agreement", c.agreements(), c.activeAgreements(), "agreement",
"Add an agreement on Agreements: your identity, the partner's identity, the standard and test or production. Make sure Active is on."));
items.add(item("4. Agreement documents", "edi_agreement_document", c.agreementDocuments(), c.activeAgreementDocuments(), "agreement document",
"Add one per transaction set and direction on Agreement Documents: pick the guide, and a map if it should become records. A document type with no row is not processed."));
items.add(item("5. Transport", "edi_transport", c.transports(), c.activeTransports(), "transport",
"Add the AS2, SFTP or HTTPS connection on Transports (secrets as variable names), link it on the agreement, then run Test Connection."));
items.add(c.certificates() < 0 ? new Item("Certificates", "edi_certificate", State.UNKNOWN, "", "")
: c.certificates() == 0
? new Item("Certificates", "edi_certificate", State.OPTIONAL, "None.", "Needed only for AS2 signing and encryption: add yours and the partner's on Certificates.")
: new Item("Certificates", "edi_certificate", State.DONE, c.certificates() + (c.certificates() == 1 ? " certificate." : " certificates."), ""));
items.add(c.guides() < 0 || c.codeLists() < 0 ? new Item("Guides and code lists", "edi_guide", State.UNKNOWN, "", "")
: c.guides() == 0 || c.codeLists() == 0
? new Item("Guides and code lists", "edi_guide", State.TODO, "The built-in guides are not loaded.", "The host must call EdiRuntime.initialize() at startup, after the schema exists.")
: new Item("Guides and code lists", "edi_guide", State.BUILT_IN, c.guides() + " guides and " + c.codeLists() + " code lists.", ""));
items.add(c.maps() < 0 ? new Item("Maps", "edi_map", State.UNKNOWN, "", "")
: c.maps() == 0
? new Item("Maps", "edi_map", State.OPTIONAL, "None.", "Needed to turn documents into records. Run Install sample maps on Maps to start from an example.")
: c.activeMaps() == 0
? new Item("Maps", "edi_map", State.TODO, c.maps() + " maps, none active.", "Point a map at your table, preview it, then run Activate Map.")
: new Item("Maps", "edi_map", State.DONE, c.activeMaps() + " active of " + c.maps() + ".", ""));
return (items);
}
/*******************************************************************************
** Step 2: your own identity (no partner) and the partner's.
*******************************************************************************/
private static Item identities(Counts c)
{
String step = "2. Identities";
if(c.ownIdentities() < 0 || c.partnerIdentities() < 0)
{
return (new Item(step, "edi_identity", State.UNKNOWN, "", ""));
}
if(c.ownIdentities() == 0)
{
return (new Item(step, "edi_identity", State.TODO, "None of yours.", "Add your own ISA or UNB qualifier and ID on Identities, with no partner."));
}
if(c.partnerIdentities() == 0)
{
return (new Item(step, "edi_identity", State.TODO, "Yours is set; no partner has one.", "Add the partner's qualifier and ID on Identities, with the partner selected."));
}
return (new Item(step, "edi_identity", State.DONE, c.ownIdentities() + " of yours, " + c.partnerIdentities() + " for partners.", ""));
}
/*******************************************************************************
** A step that needs at least one active row.
*******************************************************************************/
private static Item item(String step, String table, int total, int active, String noun, String next)
{
if(total < 0 || active < 0)
{
return (new Item(step, table, State.UNKNOWN, "", ""));
}
if(total == 0)
{
return (new Item(step, table, State.TODO, "None.", next));
}
if(active == 0)
{
return (new Item(step, table, State.TODO, total + " " + noun + (total == 1 ? "" : "s") + ", none active.", "Turn Active on for the one to use."));
}
return (new Item(step, table, State.DONE, active + " active.", ""));
}
}
@@ -220,7 +220,8 @@ public class InboundProcessor
{
QRecord sender = row("edi_identity", agreement.getValueInteger("partnerIdentityId"));
QRecord receiver = row("edi_identity", agreement.getValueInteger("localIdentityId"));
if(sender != null && receiver != null && Objects.equals(sender.getValueInteger("partnerId"), agreement.getValueInteger("partnerId"))
if(sender != null && receiver != null && !Boolean.FALSE.equals(sender.getValueBoolean("isActive")) && !Boolean.FALSE.equals(receiver.getValueBoolean("isActive"))
&& Objects.equals(sender.getValueInteger("partnerId"), agreement.getValueInteger("partnerId"))
&& receiver.getValue("partnerId") == null && x12.isa().value(5).equals(sender.getValueString("interchangeQualifier"))
&& x12.isa().value(6).trim().equals(sender.getValueString("interchangeId")) && x12.isa().value(7).equals(receiver.getValueString("interchangeQualifier"))
&& x12.isa().value(8).trim().equals(receiver.getValueString("interchangeId")))
@@ -20,7 +20,9 @@
package com.kingsrook.qbits.edi.flow;
import java.util.ArrayList;
import java.util.List;
import java.util.Set;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.kingsrook.qbits.edi.EdiRuntime;
import com.kingsrook.qbits.edi.storage.EdiRecords;
@@ -30,36 +32,80 @@ import com.kingsrook.qqq.backend.core.actions.tables.listeners.RecordChangeListe
import com.kingsrook.qqq.backend.core.actions.tables.listeners.RecordChangeType;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
/*******************************************************************************
** Transactional source listener and explicit enqueue boundary for MANUAL/ESB.
** The listener applies only to tables with an active RECORD_CHANGE definition
** (cached per runtime, see {@link OutboundSourceTables}), and to the definition
** tables themselves, whose changes invalidate that cache.
*******************************************************************************/
public class OutboundEnqueue implements RecordChangeListenerInterface
{
private static final QLogger LOG = QLogger.getLogger(OutboundEnqueue.class);
/*******************************************************************************
** Ignore internal tables and deletes; definitions are read at event time.
** Definition changes (any type), and inserts/updates of configured sources.
*******************************************************************************/
public boolean appliesTo(String table, RecordChangeType type)
{
return (!table.startsWith("edi_") && type != RecordChangeType.DELETE);
EdiRuntime runtime = EdiRuntime.find();
if(runtime == null)
{
return (false);
}
if(OutboundSourceTables.DEFINITION_TABLES.contains(table))
{
return (true);
}
if(table.startsWith("edi_") || type == RecordChangeType.DELETE)
{
return (false);
}
try
{
return (runtime.outboundSources().contains(table, QContext.getQBackendTransaction()));
}
catch(Exception e)
{
//////////////////////////////////////////////////////////////////////
// if the definitions can't be read, apply - so the event runs its //
// own queries and reports the failure, rather than silently //
// dropping outbound work. //
//////////////////////////////////////////////////////////////////////
LOG.warn("Unable to read EDI outbound source tables", e, logPair("table", table));
return (true);
}
}
/*******************************************************************************
** Use the event transaction, even when the caller did not put it in QContext.
** Definitions are read once per event, not once per record.
*******************************************************************************/
public void onRecordsChanged(RecordChangeEvent event) throws QException
{
if(OutboundSourceTables.DEFINITION_TABLES.contains(event.getTableName()))
{
EdiRuntime.current().outboundSources().invalidate(event.getTransaction());
return;
}
var previous = QContext.getQBackendTransaction();
QContext.setTransaction(event.getTransaction());
try
{
for(QRecord record : event.getRecords())
SystemWrites.call(() ->
{
enqueue(event.getTableName(), record, "RECORD_CHANGE", null);
}
List<QRecord> definitions = definitions(event.getTableName(), OutboundSourceTables.TRIGGER);
for(QRecord record : event.getRecords())
{
enqueue(event.getTableName(), record, null, definitions);
}
return (null);
});
}
finally
{
@@ -72,45 +118,60 @@ public class OutboundEnqueue implements RecordChangeListenerInterface
*******************************************************************************/
public static Integer enqueue(String table, QRecord source, String trigger, String eventId) throws QException
{
return (SystemWrites.call(() ->
return (SystemWrites.call(() -> enqueue(table, source, eventId, definitions(table, trigger))));
}
/*******************************************************************************
** Active outbound documents for the table whose trigger matches (MANUAL
** matches every trigger) and whose agreement is active. Called inside a
** {@link SystemWrites#call}.
*******************************************************************************/
static List<QRecord> definitions(String table, String trigger) throws QException
{
List<QRecord> matching = new ArrayList<>();
for(QRecord document : EdiRecords.query("edi_agreement_document", EdiRecords.equal("sourceTableName", table)
.withCriteria("direction", QCriteriaOperator.EQUALS, "OUTBOUND").withCriteria("isActive", QCriteriaOperator.EQUALS, true)))
{
int count = 0;
var definitions = EdiRecords.query("edi_agreement_document", EdiRecords.equal("sourceTableName", table)
.withCriteria("direction", QCriteriaOperator.EQUALS, "OUTBOUND").withCriteria("isActive", QCriteriaOperator.EQUALS, true));
for(QRecord document : definitions)
if("MANUAL".equals(trigger) || trigger.equals(document.getValueString("outboundTrigger")))
{
if(!"MANUAL".equals(trigger) && !trigger.equals(document.getValueString("outboundTrigger")))
{
continue;
}
QRecord agreement = OutboundWorker.row("edi_agreement", document.getValueInteger("agreementId"));
if(!Boolean.TRUE.equals(agreement.getValueBoolean("isActive")))
{
continue;
}
var metadata = QContext.getQInstance().getTable(table);
if(!metadata.getBackendName().equals(EdiRuntime.current().config().getBackendName()))
{
throw new QException("Outbound source and EDI must share the transactional backend");
}
String filter = document.getValueString("triggerFilter");
if(filter != null && !filter.isBlank() && EdiRecords.query(table, EdiRecords.equal(metadata.getPrimaryKeyField(), source.getValue(metadata.getPrimaryKeyField()))
.withSubFilter(new ObjectMapper().readValue(filter, QQueryFilter.class))).isEmpty())
{
continue;
}
if(eventId != null && !EdiRecords.query("edi_outbound_request", EdiRecords.equal("eventId", eventId)
.withCriteria("agreementDocumentId", QCriteriaOperator.EQUALS, document.getValue("id"))).isEmpty())
{
continue;
}
SystemWrites.insert("edi_outbound_request", new QRecord().withValue("agreementDocumentId", document.getValue("id"))
.withValue("sourceTableName", table).withValue("sourceRecordId", source.getValueString(metadata.getPrimaryKeyField()))
.withValue("eventId", eventId).withValue("status", "PENDING").withValue("attempts", 0)
.withValue("nextAttemptAt", EdiRuntime.current().config().getClock().instant()));
count++;
matching.add(document);
}
return (count);
}));
}
Set<Integer> activeAgreements = OutboundSourceTables.activeAgreementIds(matching);
matching.removeIf(document -> !activeAgreements.contains(document.getValueInteger("agreementId")));
return (matching);
}
/*******************************************************************************
** Enqueue one source record against already-resolved definitions.
*******************************************************************************/
private static int enqueue(String table, QRecord source, String eventId, List<QRecord> definitions) throws Exception
{
int count = 0;
for(QRecord document : definitions)
{
var metadata = QContext.getQInstance().getTable(table);
if(!metadata.getBackendName().equals(EdiRuntime.current().config().getBackendName()))
{
throw new QException("Outbound source and EDI must share the transactional backend");
}
String filter = document.getValueString("triggerFilter");
if(filter != null && !filter.isBlank() && EdiRecords.query(table, EdiRecords.equal(metadata.getPrimaryKeyField(), source.getValue(metadata.getPrimaryKeyField()))
.withSubFilter(new ObjectMapper().readValue(filter, QQueryFilter.class))).isEmpty())
{
continue;
}
if(eventId != null && !EdiRecords.query("edi_outbound_request", EdiRecords.equal("eventId", eventId)
.withCriteria("agreementDocumentId", QCriteriaOperator.EQUALS, document.getValue("id"))).isEmpty())
{
continue;
}
SystemWrites.insert("edi_outbound_request", new QRecord().withValue("agreementDocumentId", document.getValue("id"))
.withValue("sourceTableName", table).withValue("sourceRecordId", source.getValueString(metadata.getPrimaryKeyField()))
.withValue("eventId", eventId).withValue("status", "PENDING").withValue("attempts", 0)
.withValue("nextAttemptAt", EdiRuntime.current().config().getClock().instant()));
count++;
}
return (count);
}
}
@@ -0,0 +1,183 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi.flow;
import java.time.Duration;
import java.util.Collections;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import java.util.WeakHashMap;
import java.util.concurrent.atomic.AtomicLong;
import com.kingsrook.qbits.edi.storage.EdiRecords;
import com.kingsrook.qbits.edi.support.SystemWrites;
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** Per-runtime cache of the source tables that have an active RECORD_CHANGE
** outbound definition on an active agreement. The record-change listener
** consults it so writes to every other table (audits, jobs, alerts...) cost
** nothing. It is loaded from committed data only (never in a writer's
** transaction). A change to an agreement or document invalidates it, and
** again once that change commits (discarding any reload that raced it); until
** then the changing transaction bypasses the cache, so it sees its own
** definitions. A short TTL covers changes made by another process.
*******************************************************************************/
public final class OutboundSourceTables
{
static final String TRIGGER = "RECORD_CHANGE";
static final Set<String> DEFINITION_TABLES = Set.of("edi_agreement", "edi_agreement_document");
private static final Duration TTL = Duration.ofSeconds(60);
private final AtomicLong generation = new AtomicLong();
private final AtomicLong loads = new AtomicLong();
private final Set<QBackendTransaction> changingTransactions = Collections.synchronizedSet(Collections.newSetFromMap(new WeakHashMap<>()));
private volatile Snapshot snapshot;
/*******************************************************************************
** Loaded table names, with the generation and time they were loaded at.
*******************************************************************************/
private record Snapshot(Set<String> tables, long generation, long loadedAtNanos)
{
}
/*******************************************************************************
** Whether record changes to this table, written in this transaction (may be
** null), can enqueue outbound work.
*******************************************************************************/
public boolean contains(String table, QBackendTransaction transaction) throws QException
{
if(transaction != null && changingTransactions.contains(transaction))
{
return (true);
}
Snapshot current = snapshot;
if(current == null || current.generation() != generation.get() || System.nanoTime() - current.loadedAtNanos() > TTL.toNanos())
{
current = load();
}
return (current.tables().contains(table));
}
/*******************************************************************************
** Drop the cache now, and again once the changing transaction commits.
*******************************************************************************/
public void invalidate(QBackendTransaction transaction)
{
generation.incrementAndGet();
if(transaction != null)
{
changingTransactions.add(transaction);
transaction.addAfterCommitCallback(() ->
{
changingTransactions.remove(transaction);
generation.incrementAndGet();
});
}
}
/*******************************************************************************
** How many times the definitions have been loaded (for tests and diagnostics).
*******************************************************************************/
public long loadCount()
{
return (loads.get());
}
/*******************************************************************************
** Read committed active definitions once. A load that raced an invalidation
** is not kept as current, because it carries the older generation.
*******************************************************************************/
private Snapshot load() throws QException
{
long loadGeneration = generation.get();
Set<String> tables = SystemWrites.call(() ->
{
List<QRecord> documents = committed("edi_agreement_document", new QQueryFilter()
.withCriteria("direction", QCriteriaOperator.EQUALS, "OUTBOUND").withCriteria("isActive", QCriteriaOperator.EQUALS, true)
.withCriteria("outboundTrigger", QCriteriaOperator.EQUALS, TRIGGER));
Set<Integer> agreementIds = agreementIds(documents);
Set<Integer> activeAgreements = new HashSet<>();
if(!agreementIds.isEmpty())
{
committed("edi_agreement", new QQueryFilter().withCriteria("isActive", QCriteriaOperator.EQUALS, true)
.withCriteria("id", QCriteriaOperator.IN, agreementIds)).forEach(agreement -> activeAgreements.add(agreement.getValueInteger("id")));
}
Set<String> result = new HashSet<>();
for(QRecord document : documents)
{
if(document.getValueString("sourceTableName") != null && activeAgreements.contains(document.getValueInteger("agreementId")))
{
result.add(document.getValueString("sourceTableName"));
}
}
return (Set.copyOf(result));
});
loads.incrementAndGet();
Snapshot loaded = new Snapshot(tables, loadGeneration, System.nanoTime());
snapshot = loaded;
return (loaded);
}
/*******************************************************************************
** Query committed rows - deliberately without the caller's transaction.
*******************************************************************************/
private static List<QRecord> committed(String table, QQueryFilter filter) throws QException
{
return (new QueryAction().execute(new QueryInput(table).withFilter(filter)).getRecords());
}
/*******************************************************************************
** The distinct non-null agreement ids the documents reference.
*******************************************************************************/
private static Set<Integer> agreementIds(List<QRecord> documents)
{
Set<Integer> ids = new HashSet<>();
documents.forEach(document -> ids.add(document.getValueInteger("agreementId")));
ids.remove(null);
return (ids);
}
/*******************************************************************************
** Active agreement ids among those the documents reference, in one query in
** the current transaction.
*******************************************************************************/
static Set<Integer> activeAgreementIds(List<QRecord> documents) throws QException
{
Set<Integer> ids = agreementIds(documents);
Set<Integer> active = new HashSet<>();
if(!ids.isEmpty())
{
for(QRecord agreement : EdiRecords.query("edi_agreement", EdiRecords.equal("isActive", true).withCriteria("id", QCriteriaOperator.IN, ids)))
{
active.add(agreement.getValueInteger("id"));
}
}
return (active);
}
}
@@ -87,6 +87,15 @@ public class OutboundWorker
{
return (0);
}
QRecord localIdentity = row("edi_identity", agreement.getValueInteger("localIdentityId"));
QRecord partnerIdentity = row("edi_identity", agreement.getValueInteger("partnerIdentityId"));
if(localIdentity == null || partnerIdentity == null || Boolean.FALSE.equals(localIdentity.getValueBoolean("isActive")) || Boolean.FALSE.equals(partnerIdentity.getValueBoolean("isActive")))
{
///////////////////////////////////////////////////////////////////////////
// an inactive identity stops sending, as it stops routing; requests wait //
///////////////////////////////////////////////////////////////////////////
return (0);
}
var requests = EdiRecords.query("edi_outbound_request", EdiRecords.equal("agreementDocumentId", id).withCriteria("status", QCriteriaOperator.EQUALS, "PENDING").withLimit(100));
if(requests.isEmpty())
{
@@ -0,0 +1,157 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi.map;
import java.io.IOException;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.kingsrook.qbits.edi.storage.EdiRecords;
import com.kingsrook.qbits.edi.support.SystemWrites;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** The bundled sample maps (synthetic, one per built-in document and version)
** and their insert-only installation. Installed maps are drafts, bound to no
** partner and to no agreement document, so installing one never changes how
** any document is processed; the target table in each is an example name the
** host replaces before activating.
*******************************************************************************/
public final class SampleMaps
{
public static final List<String> RESOURCES = List.of("204-004010", "204-005010", "204-out-004010", "204-out-005010", "214-004010", "214-005010",
"810-004010", "810-005010", "846-004010", "846-005010", "850-004010", "850-005010", "855-004010", "855-005010", "856-004010", "856-005010",
"860-004010", "860-005010", "940-004010", "940-005010", "945-004010", "945-005010", "945-in-004010", "945-in-005010",
"DESADV-D01B", "DESADV-D96A", "INVOIC-D01B", "INVOIC-D96A", "ORDERS-D01B", "ORDERS-D96A", "ORDRSP-D01B", "ORDRSP-D96A");
/*******************************************************************************
** Utility class.
*******************************************************************************/
private SampleMaps()
{
}
/*******************************************************************************
** One bundled sample: the map's values and its rules.
*******************************************************************************/
public static JsonNode read(String resource) throws IOException
{
try(InputStream input = SampleMaps.class.getResourceAsStream("/edi/maps/" + resource + "-sample.json"))
{
if(input == null)
{
throw (new IOException("Unknown sample map " + resource));
}
return (new ObjectMapper().readTree(input));
}
}
/*******************************************************************************
** The bundled samples whose map name does not exist yet (any revision, any
** partner), in bundle order. Reads the map table as the processing user.
*******************************************************************************/
public static List<String> missing() throws QException, IOException
{
Set<String> existing = new HashSet<>();
for(QRecord map : SystemWrites.query("edi_map", new QQueryFilter()))
{
existing.add(map.getValueString("name"));
}
List<String> missing = new ArrayList<>();
for(String resource : RESOURCES)
{
if(!existing.contains(read(resource).path("map").path("name").asText()))
{
missing.add(resource);
}
}
return (missing);
}
/*******************************************************************************
** The map record a sample installs as: a draft, revision 0, no partner,
** bound to the built-in guide for its document and version when present.
*******************************************************************************/
public static QRecord mapRecord(String resource) throws IOException, QException
{
JsonNode map = read(resource).path("map");
QRecord record = new QRecord();
map.fields().forEachRemaining(e ->
{
if(!e.getValue().isNull() && !List.of("status", "revision", "partnerId", "guideId", "parentMapId").contains(e.getKey()))
{
record.withValue(e.getKey(), e.getValue().asText());
}
});
record.withValue("status", "DRAFT").withValue("revision", 0);
Integer guideId = null;
int revision = -1;
for(QRecord guide : SystemWrites.query("edi_guide", new QQueryFilter().withCriteria("transactionSet", QCriteriaOperator.EQUALS, map.path("transactionSet").asText())
.withCriteria("version", QCriteriaOperator.EQUALS, map.path("version").asText()).withCriteria("source", QCriteriaOperator.EQUALS, "BUILT_IN")
.withCriteria("status", QCriteriaOperator.EQUALS, "ACTIVE")))
{
int candidate = guide.getValueInteger("revision") == null ? 0 : guide.getValueInteger("revision");
if(candidate > revision)
{
revision = candidate;
guideId = guide.getValueInteger("id");
}
}
return (record.withValue("guideId", guideId));
}
/*******************************************************************************
** Insert one sample and its ordered rules. Skips (returns null) a sample
** whose map name now exists, so a repeated run adds nothing. It is an
** administrator action; two people running it at the same moment is not
** guarded against (edi_map has no unique name, since revisions share one).
*******************************************************************************/
public static Integer install(String resource) throws Exception
{
JsonNode sample = read(resource);
String name = sample.path("map").path("name").asText();
if(!EdiRecords.query("edi_map", EdiRecords.equal("name", name)).isEmpty())
{
return (null);
}
QRecord map = SystemWrites.insert("edi_map", mapRecord(resource));
for(JsonNode rule : sample.path("rules"))
{
QRecord row = new QRecord();
rule.fields().forEachRemaining(e ->
{
if(!e.getValue().isNull())
{
row.withValue(e.getKey(), e.getValue().isInt() ? (java.io.Serializable) e.getValue().intValue() : e.getValue().isBoolean() ? (java.io.Serializable) e.getValue().booleanValue() : e.getValue().asText());
}
});
SystemWrites.insert("edi_map_rule", row.withValue("mapId", map.getValue("id")));
}
return (map.getValueInteger("id"));
}
}
@@ -0,0 +1,109 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi.metadata;
import java.util.LinkedHashMap;
import java.util.Map;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.help.QHelpContent;
import com.kingsrook.qqq.backend.core.model.metadata.help.bundles.HelpBundles;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
/*******************************************************************************
** Applies help/qbit-edi.yaml (the QQQ help-bundle format) to the Edi app
** and its edi_* tables: what each table is for and how its rows get created,
** shown on the table screen and when it is empty.
** Fallback for hosts without the helpContent table. Hosts with that table use
** the core bundle registry and startup sync, including HOST and USER overrides.
*******************************************************************************/
public final class EdiHelp
{
public static final String RESOURCE = "/help/qbit-edi.yaml";
public static final String SLOT = "help";
public static final String APP_KEY = "edi";
/*******************************************************************************
** Utility class.
*******************************************************************************/
private EdiHelp()
{
}
/*******************************************************************************
** The bundle's entries by key, validated (fails startup if malformed).
*******************************************************************************/
public static Map<String, JsonNode> entries() throws QException
{
Map<String, JsonNode> entries = new LinkedHashMap<>();
ObjectMapper mapper = new ObjectMapper();
for(var entry : HelpBundles.load("help/qbit-edi.yaml", EdiHelp.class.getClassLoader()).getEntries())
{
entries.put(entry.getKey(), mapper.valueToTree(entry));
}
return (entries);
}
/*******************************************************************************
** Apply the help to the app (by its configured name) and every edi_* table. A
** host's own help on a table is kept (the bundle only fills the slot).
*******************************************************************************/
public static void apply(QInstance qInstance, String appName) throws QException
{
Map<String, JsonNode> entries = entries();
for(QTableMetaData table : qInstance.getTables().values())
{
JsonNode entry = entries.get("table:" + table.getName() + ";slot:" + SLOT);
if(entry != null && (table.getHelpContent() == null || !table.getHelpContent().containsKey(SLOT)))
{
table.withHelpContent(SLOT, content(entry));
}
}
QAppMetaData app = qInstance.getApp(appName);
JsonNode appEntry = entries.get("app:" + APP_KEY + ";slot:" + SLOT);
if(app != null && appEntry != null && (app.getHelpContent() == null || !app.getHelpContent().containsKey(SLOT)))
{
app.withHelpContent(SLOT, content(appEntry));
}
}
/*******************************************************************************
** Preserve the tooltip summary separately from the Markdown long help.
*******************************************************************************/
static QHelpContent content(JsonNode entry)
{
return (new QHelpContent().withSummary(entry.path("summary").asText().strip()).withContentAsMarkdown(entry.path("content").asText().strip()));
}
}
@@ -88,6 +88,7 @@ public final class EdiProcesses
result.add(action("edi_guide", "override", "Create Partner Override", "edi.guide.override"));
result.add(action("edi_guide", "activate", "Activate Guide", "edi.guide.activate"));
result.add(action("edi_map", "activate", "Activate Map", "edi.map.activate"));
result.add(InstallSampleMapsProcess.process());
QProcessMetaData scheduled = new QProcessMetaData().withName("edi_inbox_worker").withLabel("Process Inbox").withIsHidden(true)
.withPermissionRules(permission("edi.admin")).withStep(new QBackendStepMetaData().withName("run").withCode(new QCodeReference(WorkerStep.class)));
if(config.getSchedulerName() != null)
@@ -0,0 +1,210 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi.processes;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.edi.map.SampleMaps;
import com.kingsrook.qbits.edi.support.SystemWrites;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.PermissionLevel;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRules;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractExtractStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractLoadStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractTransformStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ProcessSummaryProviderInterface;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
/*******************************************************************************
** "Install sample maps": a table action on edi_map that adds the bundled
** synthetic sample maps whose name does not exist yet. Standard streamed ETL:
** Review lists what will be added, Result what was added. Insert-only by map
** name, so existing maps and edits are never touched. Installed maps are
** drafts with no partner and no agreement document, so nothing changes how a
** document is processed until an administrator activates and binds one.
*******************************************************************************/
public final class InstallSampleMapsProcess
{
public static final String NAME = "edi_map_installSamples";
/*******************************************************************************
** Utility class.
*******************************************************************************/
private InstallSampleMapsProcess()
{
}
/*******************************************************************************
** The process: a table-level action (no records need to be selected).
*******************************************************************************/
public static QProcessMetaData process()
{
Map<String, Serializable> values = new LinkedHashMap<>();
values.put(StreamedETLWithFrontendProcess.FIELD_SOURCE_TABLE, "edi_map");
values.put(StreamedETLWithFrontendProcess.FIELD_DESTINATION_TABLE, "edi_map");
values.put(StreamedETLWithFrontendProcess.FIELD_SUPPORTS_FULL_VALIDATION, false);
values.put(StreamedETLWithFrontendProcess.FIELD_PREVIEW_MESSAGE,
"These sample maps will be added as drafts, with no partner and no agreement document. Maps that already exist are not changed.");
QProcessMetaData process = StreamedETLWithFrontendProcess.defineProcessMetaData(Extract.class, Transform.class, Load.class, values);
process.setName(NAME);
process.setLabel("Install sample maps");
process.setIcon(new QIcon("playlist_add"));
process.setTableName("edi_map");
process.setIsHidden(true);
process.setMinInputRecords(0);
process.setPermissionRules(new QPermissionRules().withPermissionBaseName("edi.admin").withLevel(PermissionLevel.HAS_ACCESS_PERMISSION));
return (process);
}
/*******************************************************************************
** The samples still to install, as the processing user.
*******************************************************************************/
static List<String> missing() throws QException
{
return (SystemWrites.call(SampleMaps::missing));
}
/*******************************************************************************
** A summary line with the standard four wordings.
*******************************************************************************/
private static ProcessSummaryLine line(Status status, String future, String futurePlural, String past, String pastPlural)
{
ProcessSummaryLine line = new ProcessSummaryLine(status, 0, null);
line.setSingularFutureMessage(future);
line.setPluralFutureMessage(futurePlural);
line.setSingularPastMessage(past);
line.setPluralPastMessage(pastPlural);
return (line);
}
/*******************************************************************************
** Extract: the missing samples, computed (no query of selected records).
*******************************************************************************/
public static class Extract extends AbstractExtractStep
{
/*******************************************************************************
** How many samples will be added.
*******************************************************************************/
@Override
public Integer doCount(RunBackendStepInput input) throws QException
{
return (missing().size());
}
/*******************************************************************************
** One record per missing sample.
*******************************************************************************/
@Override
public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
for(String resource : missing())
{
QRecord map = SystemWrites.call(() -> SampleMaps.mapRecord(resource));
getRecordPipe().addRecord(map.withValue("sample", resource));
}
}
}
/*******************************************************************************
** Transform: forward the records and count them by standard and direction.
*******************************************************************************/
public static class Transform extends AbstractTransformStep implements ProcessSummaryProviderInterface
{
private final Map<String, ProcessSummaryLine> byKind = new LinkedHashMap<>();
/*******************************************************************************
** Count by standard and direction.
*******************************************************************************/
@Override
public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
for(QRecord record : input.getRecords())
{
String kind = record.getValueString("standard") + " " + record.getValueString("direction").toLowerCase();
byKind.computeIfAbsent(kind, k -> line(Status.OK, k + " sample map will be added as a draft", k + " sample maps will be added as drafts",
k + " sample map was added as a draft", k + " sample maps were added as drafts")).incrementCount();
output.getRecords().add(record);
}
}
/*******************************************************************************
** Standard Review summary.
*******************************************************************************/
@Override
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean isForResultScreen)
{
ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>();
byKind.values().forEach(line -> line.addSelfToListIfAnyCount(lines));
if(lines.isEmpty())
{
lines.add(new ProcessSummaryLine(Status.INFO, null, "Every sample map is already installed."));
}
return (lines);
}
}
/*******************************************************************************
** Load: insert each map and its rules as the processing user.
*******************************************************************************/
public static class Load extends AbstractLoadStep implements ProcessSummaryProviderInterface
{
private final ProcessSummaryLine added = line(Status.OK, "map will be added", "maps will be added", "map was added as a draft", "maps were added as drafts");
/*******************************************************************************
** One transaction per map, so a map never lands without its rules.
*******************************************************************************/
@Override
public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
for(QRecord record : input.getRecords())
{
Integer id = SystemWrites.call(() -> SystemWrites.transaction(() -> SampleMaps.install(record.getValueString("sample"))));
if(id != null)
{
added.incrementCountAndAddPrimaryKey(id);
output.getRecords().add(new QRecord().withValue("id", id));
}
}
}
/*******************************************************************************
** Standard Result summary.
*******************************************************************************/
@Override
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean isForResultScreen)
{
ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>();
added.addSelfToListIfAnyCount(lines);
return (lines);
}
}
}
@@ -61,7 +61,11 @@ public class OutboundProcesses implements BackendStep
.withStep(new QBackendStepMetaData().withName("execute").withCode(new QCodeReference(OutboundProcesses.class)));
if(config.getSchedulerName() != null && (seconds != null || cron != null))
{
process.withSchedule(new QScheduleMetaData().withSchedulerName(config.getSchedulerName()).withRepeatSeconds(seconds).withCronExpression(cron).withCronTimeZoneId("UTC"));
//////////////////////////////////////////////////////////////////////////
// core validation refuses a cron time zone on a repeat schedule //
//////////////////////////////////////////////////////////////////////////
process.withSchedule(new QScheduleMetaData().withSchedulerName(config.getSchedulerName()).withRepeatSeconds(seconds).withCronExpression(cron)
.withCronTimeZoneId(cron == null ? null : "UTC"));
}
return (process);
}
File diff suppressed because it is too large. Load diff
+1
View File
@@ -0,0 +1 @@
version=${project.version}
@@ -0,0 +1,88 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi;
import com.kingsrook.qqq.backend.core.actions.help.HelpTargetCatalog;
import com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.help.bundles.HelpBundles;
import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryBackendModule;
import com.kingsrook.qqq.backend.core.testing.HelpBundleAssertions;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** Coverage of the qbit's own help, independent of host integrations.
*******************************************************************************/
class EdiHelpBundleTest
{
/*******************************************************************************
** Every owned target resolves and has help, even without a help table.
*******************************************************************************/
@Test
void bundleIsComplete() throws Exception
{
QInstance instance = new QInstance();
instance.addBackend(new QBackendMetaData().withName("memory").withBackendType(MemoryBackendModule.class));
new EdiQBitProducer().withQBitConfig(new EdiQBitConfig().withBackendName("memory")).produce(instance);
assertThat(instance.getTable("helpContent")).isNull();
assertThat(HelpBundles.of(instance).getBundles()).singleElement().satisfies(bundle ->
{
assertThat(bundle.getId()).isEqualTo("qbit-edi");
assertThat(bundle.getVersion()).as("the help bundle carries the artifact version").isEqualTo(System.getProperty("qbit.artifactVersion")).isEqualTo(EdiQBitProducer.VERSION);
});
HelpBundles.syncOnStartup(instance);
HelpBundleAssertions.forInstance(instance)
.bundles("help/qbit-edi.yaml")
.owning(target -> target.name().equals("edi") || target.name().startsWith("edi_"))
.assertComplete();
var keys = HelpBundles.of(instance).getBundles().getFirst().getEntries().stream().map(entry -> entry.getKey()).toList();
// Core's gate currently requires apps, tables, processes and fields; also cover our widgets and value sources.
for(var target : HelpTargetCatalog.collect(instance))
{
if(target.name().startsWith("edi_") && java.util.List.of("widget", "possibleValueSource").contains(target.kind()))
{
assertThat(keys).as(target.key()).contains(target.key());
}
}
}
/*******************************************************************************
** Help-enabled hosts use core sync, and a configured app name is respected.
*******************************************************************************/
@Test
void registersForHelpEnabledHost() throws Exception
{
QInstance instance = new QInstance();
instance.addBackend(new QBackendMetaData().withName("memory").withBackendType(MemoryBackendModule.class));
instance.addTable(new com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData().withName("helpContent"));
new EdiQBitProducer().withQBitConfig(new EdiQBitConfig().withBackendName("memory").withAppName("trading")).produce(instance);
var entries = HelpBundles.of(instance).getBundles().getFirst().getEntries();
assertThat(entries).anyMatch(entry -> "app:trading;slot:help".equals(entry.getKey()))
.noneMatch(entry -> "app:edi;slot:help".equals(entry.getKey()));
assertThat(instance.getTable("edi_partner").getHelpContent()).isNullOrEmpty();
assertThat(instance.getApp("trading").getHelpContent()).isNullOrEmpty();
}
}
@@ -65,6 +65,8 @@ class EdiQBitProducerTest
assertThat(producer.getQBitConfig()).isSameAs(config);
assertThat(producer.getNamespace()).isEqualTo("ns");
assertThat(producer.getQBitMetaData().getArtifactId()).isEqualTo("qbit-edi");
assertThat(System.getProperty("qbit.artifactVersion")).isNotBlank();
assertThat(producer.getQBitMetaData().getVersion()).isEqualTo(System.getProperty("qbit.artifactVersion")).isEqualTo(EdiQBitProducer.VERSION);
assertThat(producer.getPackageNameForFindingMetaDataProducers()).isEqualTo("com.kingsrook.qbits.edi.metadata");
assertThat(config.getAppName()).isEqualTo("edi");
assertThat(config.getAppLabel()).isEqualTo("EDI");
@@ -0,0 +1,127 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.edi.app.EdiSetupStatus;
import com.kingsrook.qbits.edi.support.SystemWrites;
import com.kingsrook.qqq.backend.core.actions.dashboard.RenderWidgetAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.TableData;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** The Setup status on the EDI home: the steps in setup order, each with its
** state and next step, counting only the rows the runtime would use.
*******************************************************************************/
class EdiSetupStatusTest
{
/*******************************************************************************
**
*******************************************************************************/
@AfterEach
void clear()
{
QContext.clear();
}
/*******************************************************************************
** The pure logic: order, states and next steps.
*******************************************************************************/
@Test
void stepsAreInSetupOrder()
{
List<EdiSetupStatus.Item> items = EdiSetupStatus.items(new EdiSetupStatus.Counts(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 41, 20, 0, 0));
assertThat(items).extracting(EdiSetupStatus.Item::step).containsExactly("1. Partner", "2. Identities", "3. Agreement", "4. Agreement documents", "5. Transport",
"Certificates", "Guides and code lists", "Maps");
assertThat(items.subList(0, 5)).allSatisfy(item -> assertThat(item.state()).isEqualTo(EdiSetupStatus.State.TODO));
assertThat(items.get(5).state()).isEqualTo(EdiSetupStatus.State.OPTIONAL);
assertThat(items.get(6).state()).isEqualTo(EdiSetupStatus.State.BUILT_IN);
assertThat(items.get(7).state()).isEqualTo(EdiSetupStatus.State.OPTIONAL);
assertThat(items.get(7).nextStep()).contains("Install sample maps");
items = EdiSetupStatus.items(new EdiSetupStatus.Counts(1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0));
assertThat(items.get(1).nextStep()).contains("partner's");
assertThat(items.get(6).state()).isEqualTo(EdiSetupStatus.State.TODO);
assertThat(items.get(6).nextStep()).contains("EdiRuntime.initialize()");
assertThat(items.get(7).state()).as("maps exist but none active").isEqualTo(EdiSetupStatus.State.TODO);
items = EdiSetupStatus.items(new EdiSetupStatus.Counts(2, 1, 2, 2, 1, 3, 3, 1, 1, 1, 41, 20, 5, 2));
assertThat(items).allSatisfy(item -> assertThat(item.state()).isIn(EdiSetupStatus.State.DONE, EdiSetupStatus.State.BUILT_IN));
items = EdiSetupStatus.items(new EdiSetupStatus.Counts(1, 1, 1, 2, 0, 1, 1, 1, 1, 0, 41, 20, 0, 0));
assertThat(items.get(2).state()).as("agreements exist but none is active, so the runtime ignores them").isEqualTo(EdiSetupStatus.State.TODO);
items = EdiSetupStatus.items(new EdiSetupStatus.Counts(-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1));
assertThat(items).allSatisfy(item -> assertThat(item.state()).isEqualTo(EdiSetupStatus.State.UNKNOWN));
}
/*******************************************************************************
** The home widget moves from "needs setup" to "set up" as the rows are added.
*******************************************************************************/
@Test
void homeWidgetFollowsTheData() throws Exception
{
QInstance instance = H2Support.create();
EdiRuntime.current().initialize();
assertThat(instance.getWidget("edi_home_setup")).isNotNull();
assertThat(instance.getApp(EdiRuntime.current().config().getAppName()).getWidgets()).contains("edi_home_setup");
Map<String, String> status = status(instance);
assertThat(status).containsKeys("1. Partner", "5. Transport");
assertThat(status.get("1. Partner")).startsWith("Needs setup");
assertThat(status.get("2. Identities")).startsWith("Needs setup");
assertThat(status.get("Guides and code lists")).startsWith("Built in: 41 guides");
assertThat(status.get("Maps")).startsWith("Optional");
SystemWrites.call(() ->
{
QRecord partner = SystemWrites.insert("edi_partner", new QRecord().withValue("name", "Synthetic Retailer").withValue("code", "SYNTH"));
SystemWrites.insert("edi_identity", new QRecord().withValue("standard", "X12").withValue("interchangeQualifier", "ZZ").withValue("interchangeId", "QQQTEST")
.withValue("applicationCode", "QQQTEST").withValue("isActive", true));
SystemWrites.insert("edi_identity", new QRecord().withValue("partnerId", partner.getValue("id")).withValue("standard", "X12").withValue("interchangeQualifier", "ZZ")
.withValue("interchangeId", "SYNTHETIC").withValue("applicationCode", "SYNTHETIC").withValue("isActive", false));
return (null);
});
status = status(instance);
assertThat(status.get("1. Partner")).startsWith("Set up");
assertThat(status.get("2. Identities")).as("the partner's identity is inactive, so the runtime would ignore it").startsWith("Needs setup: Yours is set; no partner has one");
assertThat(status.get("3. Agreement")).startsWith("Needs setup");
}
/*******************************************************************************
** Render the widget and return step to "state: detail".
*******************************************************************************/
private Map<String, String> status(QInstance instance) throws Exception
{
var data = (TableData) new RenderWidgetAction().execute(new RenderWidgetInput().withWidgetMetaData(instance.getWidget("edi_home_setup"))).getWidgetData();
Map<String, String> status = new LinkedHashMap<>();
data.getRows().forEach(row -> status.put(String.valueOf(row.get("Step")), String.valueOf(row.get("Status"))));
return (status);
}
}
@@ -0,0 +1,247 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.regex.Pattern;
import java.util.stream.Stream;
import com.kingsrook.qbits.edi.guide.BuiltInGuideLoader;
import com.kingsrook.qbits.edi.metadata.EdiHelp;
import com.kingsrook.qbits.edi.storage.EdiRecords;
import com.kingsrook.qbits.edi.support.SystemWrites;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.help.QHelpContent;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** Guardrail (owner requirement 2026-10-14: "all tables should be populated and
** the qbit should be fully using them"). For every edi_* table:
** - runtime code (not just its model, metadata or home screen) reads or
** writes it;
** - it is either seeded by the qbit at startup, or listed here with why it
** starts empty;
** - it has bundle help that says what it is for and how its rows get created
** (shown on the table screen and when it is empty).
** A new table fails this test until it is wired, seeded or explained.
*******************************************************************************/
class EdiTablesGuardrailTest
{
private static final Path MAIN = Path.of("src", "main", "java", "com", "kingsrook", "qbits", "edi");
/*******************************************************************************
** Tables the qbit seeds itself (EdiRuntime.initialize, idempotent).
*******************************************************************************/
static final Map<String, String> SEEDED = Map.of(
"edi_guide", "BuiltInGuideLoader: 41 built-in guides",
"edi_guide_segment", "with each built-in guide",
"edi_guide_element", "with each built-in guide",
"edi_guide_rule", "with each built-in guide",
"edi_code_list", "with the built-in guides",
"edi_code_list_value", "with the built-in guides");
/*******************************************************************************
** Tables that legitimately start empty, and where their rows come from.
*******************************************************************************/
static final Map<String, String> STARTS_EMPTY = Map.ofEntries(
Map.entry("edi_partner", "host data: the trading partners"),
Map.entry("edi_identity", "host data: your ISA/UNB identity and each partner's"),
Map.entry("edi_agreement", "host data: the agreement per partner"),
Map.entry("edi_agreement_document", "host data: transaction sets per agreement"),
Map.entry("edi_transport", "host connection config: addresses and secret variable names"),
Map.entry("edi_certificate", "host data: AS2 certificates (optional)"),
Map.entry("edi_map", "host data; the 'Install sample maps' action adds bundled examples as inactive, unbound drafts"),
Map.entry("edi_map_rule", "with a map; 'Install sample maps' adds the rules of each sample"),
Map.entry("edi_map_sample", "user content: synthetic documents uploaded with Preview Map"),
Map.entry("edi_control_number", "runtime: allocated on the first send for a scope"),
Map.entry("edi_inbox", "runtime: every received document"),
Map.entry("edi_payload", "runtime: raw bytes, only when payload storage is DATABASE"),
Map.entry("edi_interchange", "runtime: every envelope sent or received"),
Map.entry("edi_functional_group", "runtime: every group"),
Map.entry("edi_transaction", "runtime: every document"),
Map.entry("edi_transaction_error", "runtime: validation findings"),
Map.entry("edi_exception", "runtime: failures that need a person"),
Map.entry("edi_event", "audit log: written at each processing step (EdiJournal), shown on the interchange record"),
Map.entry("edi_as2_mdn", "runtime: AS2 receipts sent and received; read to ignore duplicates and replay a sent receipt"),
Map.entry("edi_outbound_request", "runtime: a host record queued to be sent"),
Map.entry("edi_outbound_message", "runtime: the outbound queue"));
private QInstance instance;
/*******************************************************************************
**
*******************************************************************************/
@AfterEach
void clear()
{
QContext.clear();
}
/*******************************************************************************
** Every table is used by runtime code.
*******************************************************************************/
@Test
void everyTableIsUsedAtRuntime() throws Exception
{
List<String> runtimeSource = new ArrayList<>();
try(Stream<Path> files = Files.walk(MAIN))
{
files.filter(p -> p.toString().endsWith(".java") && !p.startsWith(MAIN.resolve("model")) && !p.startsWith(MAIN.resolve("metadata")) && !p.startsWith(MAIN.resolve("app")))
.forEach(p -> runtimeSource.add(read(p)));
}
for(String table : tables())
{
Pattern use = Pattern.compile("\\b" + table + "\\b");
assertThat(runtimeSource.stream().anyMatch(source -> use.matcher(source).find()))
.as(table + " must be read or written by runtime code, not only declared").isTrue();
}
}
/*******************************************************************************
** Every table is seeded or explained.
*******************************************************************************/
@Test
void everyTableIsSeededOrExplained() throws Exception
{
for(String table : tables())
{
assertThat(SEEDED.containsKey(table) || STARTS_EMPTY.containsKey(table)).as(table + " must be seeded or listed in STARTS_EMPTY").isTrue();
assertThat(SEEDED.containsKey(table) && STARTS_EMPTY.containsKey(table)).as(table + " cannot be both").isFalse();
}
assertThat(tables()).containsAll(STARTS_EMPTY.keySet()).containsAll(SEEDED.keySet());
}
/*******************************************************************************
** The built-in guides fill the seeded tables, a second start adds nothing, and
** an administrator's edit survives.
*******************************************************************************/
@Test
void seedIsInsertOnly() throws Exception
{
H2Support.create();
EdiRuntime.current().initialize();
Map<String, Integer> first = counts();
assertThat(first.get("edi_guide")).isEqualTo(BuiltInGuideLoader.RESOURCES.size()).isEqualTo(41);
for(String table : SEEDED.keySet())
{
assertThat(first.get(table)).as(table + " is seeded").isGreaterThan(0);
}
QRecord guide = SystemWrites.query("edi_guide", new QQueryFilter()).getFirst();
SystemWrites.call(() ->
{
SystemWrites.update("edi_guide", new QRecord().withValue("id", guide.getValue("id")).withValue("description", "Edited by an administrator"));
return (null);
});
EdiRuntime.current().initialize();
assertThat(counts()).isEqualTo(first);
assertThat(EdiRecords.query("edi_guide", EdiRecords.equal("id", guide.getValue("id"))).getFirst().getValueString("description")).isEqualTo("Edited by an administrator");
}
/*******************************************************************************
** Every table and the app have bundle help that says how rows get created.
*******************************************************************************/
@Test
void everyTableHasEmptyStateHelp() throws Exception
{
QInstance instance = H2Support.create();
assertThat(EdiHelp.entries()).allSatisfy((key, entry) -> assertThat(entry.path("summary").asText().length()).as(key).isLessThanOrEqualTo(120));
for(String table : tables())
{
List<QHelpContent> help = instance.getTable(table).getHelpContent() == null ? null : instance.getTable(table).getHelpContent().get(EdiHelp.SLOT);
assertThat(help).as(table + " table help").isNotEmpty();
assertThat(help.get(0).getContent()).as(table + " help says how rows get created").contains("How rows get created");
}
assertThat(instance.getApp(EdiRuntime.current().config().getAppName()).getHelpContent()).containsKey(EdiHelp.SLOT);
assertThat(EdiHelp.entries().get("table:edi_map;slot:help").path("content").asText()).contains("Install sample maps");
assertThat(EdiHelp.entries().get("app:edi;slot:help").path("content").asText().indexOf("Partner")).isLessThan(EdiHelp.entries().get("app:edi;slot:help").path("content").asText().indexOf("Transport"));
}
/*******************************************************************************
** Help a host put in the slot first is never replaced by the bundle.
*******************************************************************************/
@Test
void hostHelpIsKept() throws Exception
{
QInstance instance = H2Support.create();
String appName = EdiRuntime.current().config().getAppName();
QHelpContent host = new QHelpContent().withContentAsMarkdown("Host help for partners");
instance.getTable("edi_partner").getHelpContent().put(EdiHelp.SLOT, new ArrayList<>(List.of(host)));
instance.getApp(appName).getHelpContent().put(EdiHelp.SLOT, new ArrayList<>(List.of(host)));
EdiHelp.apply(instance, appName);
assertThat(instance.getTable("edi_partner").getHelpContent().get(EdiHelp.SLOT)).containsExactly(host);
assertThat(instance.getApp(appName).getHelpContent().get(EdiHelp.SLOT)).containsExactly(host);
assertThat(instance.getTable("edi_identity").getHelpContent().get(EdiHelp.SLOT).get(0).getContent()).contains("How rows get created");
}
/*******************************************************************************
** Row counts of the seeded tables.
*******************************************************************************/
private Map<String, Integer> counts() throws Exception
{
Map<String, Integer> counts = new java.util.TreeMap<>();
for(String table : SEEDED.keySet())
{
counts.put(table, SystemWrites.count(table, new QQueryFilter()));
}
return (counts);
}
/*******************************************************************************
** The qbit's tables (the preview download backend table is not data).
*******************************************************************************/
private List<String> tables() throws Exception
{
if(instance == null)
{
instance = H2Support.create();
}
List<String> tables = instance.getTables().values().stream().map(QTableMetaData::getName)
.filter(name -> name.startsWith("edi_") && !name.equals("edi_preview_download")).sorted().toList();
assertThat(tables).hasSize(27);
return (tables);
}
/*******************************************************************************
**
*******************************************************************************/
private static String read(Path path)
{
try
{
return (Files.readString(path, StandardCharsets.UTF_8));
}
catch(Exception e)
{
throw (new IllegalStateException(e));
}
}
}
@@ -744,4 +744,32 @@ class InboundTest
assertThat(rows("sample_order_line")).isEmpty();
assertThat(rows("edi_exception")).extracting(r -> r.getValueString("type")).contains("APPLY");
}
/*******************************************************************************
** An inactive identity stops X12 routing, as it does for EDIFACT.
*******************************************************************************/
@Test
void inactiveIdentityStopsX12Routing() throws Exception
{
setup("004010");
QRecord remote = rows("edi_identity").stream().filter(r -> r.getValue("partnerId") != null).findFirst().orElseThrow();
SystemWrites.call(() ->
{
SystemWrites.update("edi_identity", new QRecord().withValue("id", remote.getValue("id")).withValue("isActive", false));
return (null);
});
receive(raw(), "inactive");
new InboxWorker().runOnce();
assertThat(rows("sample_order")).isEmpty();
assertThat(rows("edi_exception")).extracting(r -> r.getValueString("type")).contains("UNKNOWN_PARTNER");
SystemWrites.call(() ->
{
SystemWrites.update("edi_identity", new QRecord().withValue("id", remote.getValue("id")).withValue("isActive", true));
return (null);
});
receive(Synthetic850.generate(18, "004010", X12Separators.DEFAULT), "active");
new InboxWorker().runOnce();
assertThat(rows("sample_order")).hasSize(1);
}
}
@@ -0,0 +1,189 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi;
import java.io.File;
import java.util.Arrays;
import java.util.List;
import com.kingsrook.qbits.edi.map.SampleMaps;
import com.kingsrook.qbits.edi.processes.InstallSampleMapsProcess;
import com.kingsrook.qbits.edi.storage.EdiRecords;
import com.kingsrook.qbits.edi.support.SystemWrites;
import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** The "Install sample maps" table action on edi_map: standard streamed ETL,
** insert-only by name, installing inactive and unbound drafts.
*******************************************************************************/
class InstallSampleMapsTest
{
/*******************************************************************************
**
*******************************************************************************/
@AfterEach
void clear()
{
QContext.clear();
}
/*******************************************************************************
** Every bundled sample is listed, and every listed sample exists.
*******************************************************************************/
@Test
void theListMatchesTheBundledFiles()
{
String[] files = new File("src/main/resources/edi/maps").list((dir, name) -> name.endsWith("-sample.json"));
assertThat(Arrays.stream(files).map(name -> name.replace("-sample.json", "")).toList()).containsExactlyInAnyOrderElementsOf(SampleMaps.RESOURCES);
}
/*******************************************************************************
** A standard streamed-ETL table action that needs no selected records.
*******************************************************************************/
@Test
void isAStandardTableAction() throws Exception
{
QInstance instance = H2Support.create();
QProcessMetaData process = instance.getProcess(InstallSampleMapsProcess.NAME);
assertThat(process).isNotNull();
assertThat(process.getTableName()).isEqualTo("edi_map");
assertThat(process.getMinInputRecords()).isEqualTo(0);
assertThat(process.getPermissionRules().getPermissionBaseName()).isEqualTo("edi.admin");
assertThat(process.getStepList()).extracting(step -> step.getName()).contains("review", "result");
}
/*******************************************************************************
** Installs every sample as an inactive, unbound draft with its rules; a
** second run adds nothing; an edited map is kept; a deleted one comes back.
*******************************************************************************/
@Test
void installsDraftsInsertOnly() throws Exception
{
H2Support.create();
EdiRuntime.current().initialize();
assertThat(count("edi_map")).isZero();
run();
assertThat(count("edi_map")).isEqualTo(SampleMaps.RESOURCES.size());
List<QRecord> maps = SystemWrites.query("edi_map", new QQueryFilter());
assertThat(maps).allSatisfy(map ->
{
assertThat(map.getValueString("status")).as(map.getValueString("name")).isEqualTo("DRAFT");
assertThat(map.getValueInteger("revision")).isZero();
assertThat(map.getValue("partnerId")).as("unbound to a partner").isNull();
assertThat(map.getValue("guideId")).as("bound to the built-in guide for its document").isNotNull();
});
assertThat(count("edi_agreement_document")).as("bound to no agreement document").isZero();
QRecord sample850 = maps.stream().filter(map -> "Sample 850 004010".equals(map.getValueString("name"))).findFirst().orElseThrow();
int rules = count("edi_map_rule");
assertThat(EdiRecords.query("edi_map_rule", EdiRecords.equal("mapId", sample850.getValue("id")))).hasSize(5);
assertThat(rules).isGreaterThan(SampleMaps.RESOURCES.size());
SystemWrites.call(() ->
{
SystemWrites.update("edi_map", new QRecord().withValue("id", sample850.getValue("id")).withValue("tableName", "my_orders").withValue("status", "ACTIVE"));
return (null);
});
run();
assertThat(count("edi_map")).isEqualTo(SampleMaps.RESOURCES.size());
assertThat(count("edi_map_rule")).isEqualTo(rules);
QRecord edited = EdiRecords.query("edi_map", EdiRecords.equal("id", sample850.getValue("id"))).getFirst();
assertThat(edited.getValueString("tableName")).isEqualTo("my_orders");
assertThat(edited.getValueString("status")).isEqualTo("ACTIVE");
SystemWrites.call(() ->
{
new com.kingsrook.qqq.backend.core.actions.tables.DeleteAction().execute(new com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput("edi_map_rule")
.withQueryFilter(new QQueryFilter().withCriteria("mapId", com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator.EQUALS, sample850.getValue("id"))));
new com.kingsrook.qqq.backend.core.actions.tables.DeleteAction().execute(new com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput("edi_map")
.withPrimaryKey(sample850.getValue("id")));
return (null);
});
assertThat(SampleMaps.RESOURCES).hasSize(32);
assertThat(SystemWrites.call(SampleMaps::missing)).containsExactly("850-004010");
run();
assertThat(count("edi_map")).isEqualTo(SampleMaps.RESOURCES.size());
assertThat(EdiRecords.query("edi_map_rule", EdiRecords.equal("mapId", EdiRecords.query("edi_map", EdiRecords.equal("name", "Sample 850 004010")).getFirst().getValue("id")))).hasSize(5);
}
/*******************************************************************************
** A host that already has a map of the same name keeps it, and gets no copy.
*******************************************************************************/
@Test
void neverTouchesAMapTheHostAlreadyHas() throws Exception
{
H2Support.create();
EdiRuntime.current().initialize();
SystemWrites.call(() -> SystemWrites.insert("edi_map", new QRecord().withValue("name", "Sample 850 004010").withValue("direction", "INBOUND")
.withValue("transactionSet", "850").withValue("version", "004010").withValue("status", "ACTIVE").withValue("revision", 3).withValue("tableName", "host_orders")));
run();
List<QRecord> same = EdiRecords.query("edi_map", EdiRecords.equal("name", "Sample 850 004010"));
assertThat(same).hasSize(1);
assertThat(same.getFirst().getValueString("tableName")).isEqualTo("host_orders");
assertThat(same.getFirst().getValueString("status")).isEqualTo("ACTIVE");
assertThat(count("edi_map")).isEqualTo(SampleMaps.RESOURCES.size());
assertThat(count("edi_map_rule")).as("no rules were added to the host's map").isEqualTo(
SampleMaps.RESOURCES.stream().filter(r -> !r.equals("850-004010")).mapToInt(InstallSampleMapsTest::ruleCount).sum());
}
/*******************************************************************************
** Run the real process through the standard runner (frontend steps skipped).
*******************************************************************************/
private void run() throws Exception
{
RunProcessInput input = new RunProcessInput();
input.setProcessName(InstallSampleMapsProcess.NAME);
input.setFrontendStepBehavior(RunProcessInput.FrontendStepBehavior.SKIP);
new RunProcessAction().execute(input);
}
/*******************************************************************************
** The number of rules a bundled sample carries.
*******************************************************************************/
private static int ruleCount(String resource)
{
try
{
return (SampleMaps.read(resource).path("rules").size());
}
catch(java.io.IOException e)
{
throw (new IllegalStateException(e));
}
}
/*******************************************************************************
**
*******************************************************************************/
private int count(String table) throws Exception
{
return (SystemWrites.count(table, new QQueryFilter()));
}
}
@@ -0,0 +1,213 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi;
import java.util.List;
import com.kingsrook.qbits.edi.flow.OutboundEnqueue;
import com.kingsrook.qbits.edi.flow.OutboundSourceTables;
import com.kingsrook.qbits.edi.storage.EdiRecords;
import com.kingsrook.qbits.edi.support.SystemWrites;
import com.kingsrook.qqq.backend.core.actions.tables.listeners.RecordChangeType;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** The record-change listener only costs queries for configured source tables,
** and its cache follows agreement and document changes.
*******************************************************************************/
class OutboundSourceTablesTest
{
private QRecord document;
private QRecord agreement;
/*******************************************************************************
** One active RECORD_CHANGE agreement document, for sample_order.
*******************************************************************************/
@BeforeEach
void setup() throws Exception
{
H2Support.create();
RetailSupport.tables();
EdiRuntime.current().initialize();
SystemWrites.call(() ->
{
QRecord partner = SystemWrites.insert("edi_partner", new QRecord().withValue("name", "Retail test").withValue("code", "RETAIL").withValue("status", "ACTIVE"));
QRecord local = SystemWrites.insert("edi_identity", new QRecord().withValue("standard", "X12").withValue("interchangeQualifier", "ZZ").withValue("interchangeId", "QQQTEST").withValue("applicationCode", "QQQTEST"));
QRecord remote = SystemWrites.insert("edi_identity", new QRecord().withValue("partnerId", partner.getValue("id")).withValue("standard", "X12").withValue("interchangeQualifier", "ZZ").withValue("interchangeId", "SYNTHETIC").withValue("applicationCode", "SYNTHETIC"));
QRecord transport = SystemWrites.insert("edi_transport", new QRecord().withValue("name", "retail-out").withValue("type", "GATEWAY").withValue("direction", "BOTH").withValue("gatewayEndpointName", "retail-outbox").withValue("isActive", true).withValue("maxAttempts", 3));
agreement = SystemWrites.insert("edi_agreement", new QRecord().withValue("partnerId", partner.getValue("id")).withValue("localIdentityId", local.getValue("id")).withValue("partnerIdentityId", remote.getValue("id"))
.withValue("standard", "X12").withValue("usageIndicator", "T").withValue("isActive", true).withValue("outboundTransportId", transport.getValue("id")));
QRecord map = RetailSupport.installMap("855", "004010");
document = SystemWrites.insert("edi_agreement_document", outboundDocument(map, "sample_order"));
return (null);
});
}
/*******************************************************************************
** Reset context.
*******************************************************************************/
@AfterEach
void clear()
{
QContext.clear();
}
/*******************************************************************************
** Writes to a table without a definition neither apply nor reload definitions.
*******************************************************************************/
@Test
void unconfiguredTableDoesNoWork() throws Exception
{
OutboundEnqueue listener = new OutboundEnqueue();
assertThat(listener.appliesTo("sample_order", RecordChangeType.INSERT)).isTrue();
long loads = sources().loadCount();
assertThat(listener.appliesTo("sample_invoice", RecordChangeType.INSERT)).isFalse();
assertThat(listener.appliesTo("sample_invoice", RecordChangeType.UPDATE)).isFalse();
assertThat(listener.appliesTo("sample_order", RecordChangeType.DELETE)).isFalse();
SystemWrites.call(() -> SystemWrites.transaction(() ->
{
for(int i = 0; i < 5; i++)
{
RetailSupport.saveSource("sample_invoice", new QRecord().withValue("invoiceNumber", "INV-" + i));
}
return (null);
}));
assertThat(sources().loadCount()).isEqualTo(loads);
assertThat(rows("edi_outbound_request")).isEmpty();
}
/*******************************************************************************
** Configured tables still enqueue, once per record, from one cached load.
*******************************************************************************/
@Test
void configuredTableEnqueues() throws Exception
{
SystemWrites.call(() -> SystemWrites.transaction(() -> RetailSupport.saveSource("sample_order", RetailSupport.source())));
long loads = sources().loadCount();
SystemWrites.call(() -> SystemWrites.transaction(() -> RetailSupport.saveSource("sample_order", RetailSupport.source().withValue("poNumber", "PO-2"))));
assertThat(sources().loadCount()).isEqualTo(loads);
assertThat(rows("edi_outbound_request")).hasSize(2);
}
/*******************************************************************************
** Document and agreement changes invalidate the cached tables.
*******************************************************************************/
@Test
void definitionChangesInvalidate() throws Exception
{
OutboundEnqueue listener = new OutboundEnqueue();
assertThat(listener.appliesTo("sample_order", RecordChangeType.INSERT)).isTrue();
assertThat(listener.appliesTo("edi_agreement_document", RecordChangeType.DELETE)).isTrue();
update("edi_agreement_document", document.getValue("id"), "isActive", false);
assertThat(listener.appliesTo("sample_order", RecordChangeType.INSERT)).isFalse();
update("edi_agreement_document", document.getValue("id"), "isActive", true);
assertThat(listener.appliesTo("sample_order", RecordChangeType.INSERT)).isTrue();
update("edi_agreement", agreement.getValue("id"), "isActive", false);
assertThat(listener.appliesTo("sample_order", RecordChangeType.INSERT)).isFalse();
SystemWrites.call(() -> SystemWrites.transaction(() -> RetailSupport.saveSource("sample_order", RetailSupport.source())));
assertThat(rows("edi_outbound_request")).isEmpty();
update("edi_agreement", agreement.getValue("id"), "isActive", true);
assertThat(listener.appliesTo("sample_order", RecordChangeType.INSERT)).isTrue();
}
/*******************************************************************************
** A transaction that adds a definition sees it before commit; a rolled-back
** one leaves the cache on committed data.
*******************************************************************************/
@Test
void changingTransactionSeesItsOwnDefinitions() throws Exception
{
QRecord map = SystemWrites.call(() -> EdiRecords.query("edi_map", new QQueryFilter()).getFirst());
SystemWrites.call(() -> SystemWrites.transaction(() ->
{
SystemWrites.insert("edi_agreement_document", outboundDocument(map, "sample_invoice").withValue("transactionSet", "810"));
RetailSupport.saveSource("sample_invoice", new QRecord().withValue("invoiceNumber", "INV-1"));
return (null);
}));
assertThat(rows("edi_outbound_request")).hasSize(1);
assertThat(new OutboundEnqueue().appliesTo("sample_invoice", RecordChangeType.INSERT)).isTrue();
try
{
SystemWrites.call(() -> SystemWrites.transaction(() ->
{
SystemWrites.insert("edi_agreement_document", outboundDocument(map, "sample_inventory").withValue("transactionSet", "846"));
throw new IllegalStateException("rollback synthetic write");
}));
}
catch(Exception e)
{
////////////////////////////
// expected - rolled back //
////////////////////////////
}
assertThat(new OutboundEnqueue().appliesTo("sample_inventory", RecordChangeType.INSERT)).isFalse();
}
/*******************************************************************************
** An active outbound RECORD_CHANGE document for the source table.
*******************************************************************************/
private QRecord outboundDocument(QRecord map, String sourceTable)
{
return (new QRecord().withValue("agreementId", agreement.getValue("id")).withValue("direction", "OUTBOUND").withValue("transactionSet", "855")
.withValue("version", "004010").withValue("functionalIdCode", "PR").withValue("guideId", map.getValue("guideId")).withValue("mapId", map.getValue("id"))
.withValue("sourceTableName", sourceTable).withValue("outboundTrigger", "RECORD_CHANGE").withValue("batchMode", "IMMEDIATE").withValue("ackExpectation", "997").withValue("ackDueMinutes", 1).withValue("isActive", true));
}
/*******************************************************************************
** Update one field through QQQ, outside any transaction.
*******************************************************************************/
private static void update(String table, Object id, String field, Object value) throws Exception
{
SystemWrites.call(() ->
{
SystemWrites.update(table, new QRecord().withValue("id", (java.io.Serializable) id).withValue(field, (java.io.Serializable) value));
return (null);
});
}
/*******************************************************************************
** The runtime cache under test.
*******************************************************************************/
private static OutboundSourceTables sources()
{
return (EdiRuntime.current().outboundSources());
}
/*******************************************************************************
** All rows.
*******************************************************************************/
private static List<QRecord> rows(String table) throws Exception
{
return (SystemWrites.call(() -> EdiRecords.query(table, new QQueryFilter())));
}
}
@@ -329,4 +329,60 @@ class OutboundTest
{
return (SystemWrites.call(() -> EdiRecords.query(table, new QQueryFilter())));
}
/*******************************************************************************
** Repeat-seconds worker schedules carry no cron time zone (core instance
** validation refuses one on a non-cron schedule); cron schedules use UTC.
*******************************************************************************/
@Test
void repeatSchedulesHaveNoCronTimeZone()
{
com.kingsrook.qbits.edi.EdiQBitConfig config = new com.kingsrook.qbits.edi.EdiQBitConfig().withSchedulerName("scheduler");
var repeat = com.kingsrook.qbits.edi.processes.OutboundProcesses.process("edi_test_repeat", config, 15, null).getSchedule();
assertThat(repeat.getRepeatSeconds()).isEqualTo(15);
assertThat(repeat.getCronTimeZoneId()).isNull();
var cron = com.kingsrook.qbits.edi.processes.OutboundProcesses.process("edi_test_cron", config, null, "0 0 * * * ?").getSchedule();
assertThat(cron.getCronTimeZoneId()).isEqualTo("UTC");
}
/*******************************************************************************
** A draft map bound to an outbound document is never used: the request fails
** as a mapping error and nothing is queued. An inactive identity stops the
** document from being built at all; its requests wait.
*******************************************************************************/
@Test
void draftMapAndInactiveIdentityStopSending() throws Exception
{
setup();
SystemWrites.call(() ->
{
SystemWrites.update("edi_map", new QRecord().withValue("id", document.getValue("mapId")).withValue("status", "DRAFT"));
return (null);
});
SystemWrites.call(() -> SystemWrites.transaction(() -> RetailSupport.saveSource("sample_order", RetailSupport.source())));
new OutboundWorker().runOnce();
assertThat(rows("edi_outbound_message")).isEmpty();
assertThat(rows("edi_outbound_request")).extracting(r -> r.getValueString("status")).containsExactly("FAILED");
assertThat(rows("edi_exception")).extracting(r -> r.getValueString("type")).contains("MAPPING");
SystemWrites.call(() ->
{
SystemWrites.update("edi_map", new QRecord().withValue("id", document.getValue("mapId")).withValue("status", "ACTIVE"));
SystemWrites.update("edi_identity", new QRecord().withValue("id", agreement.getValue("partnerIdentityId")).withValue("isActive", false));
return (null);
});
SystemWrites.call(() -> SystemWrites.transaction(() -> RetailSupport.saveSource("sample_order", RetailSupport.source().withValue("poNumber", "DEMO-1002"))));
assertThat(new OutboundWorker().runOnce()).isZero();
assertThat(rows("edi_outbound_message")).isEmpty();
assertThat(rows("edi_outbound_request")).extracting(r -> r.getValueString("status")).contains("PENDING");
SystemWrites.call(() ->
{
SystemWrites.update("edi_identity", new QRecord().withValue("id", agreement.getValue("partnerIdentityId")).withValue("isActive", true));
return (null);
});
new OutboundWorker().runOnce();
assertThat(rows("edi_outbound_message")).hasSize(1);
}
}