From c15872a3731f3dde2e26b1e05b3756751ab68e20 Mon Sep 17 00:00:00 2001 From: James Maes Date: Wed, 14 Oct 2026 09:48:59 -0500 Subject: [PATCH] feat: phase f1 - spec'd UI API extension for the EDI mapper - MiddlewareSpecExtension "edi" 1.0.0 under /qqq/v1/qbit/edi: GET guides/{guideId}/tree, GET maps/{mapId}/target, POST sandbox/preview (unsaved rule overrides, base64 uploads up to 5 MB, PHI suppression) and POST maps/{mapId}/validate. - MapPreviewService shared by the Preview Map action and the endpoint; MapRuleValidator for paths, transforms and arguments. - Committed fragment src/main/resources/openapi/qqq-v1/edi.yaml with a contract test; HTTP tests in strict raw-route mode. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 5 + README.md | 51 ++ docs/DESIGN.md | 33 +- .../kingsrook/qbits/edi/EdiQBitProducer.java | 1 + .../com/kingsrook/qbits/edi/api/EdiApi.java | 293 ++++++++ .../qbits/edi/api/EdiApiService.java | 321 +++++++++ .../kingsrook/qbits/edi/api/EdiSchemas.java | 528 +++++++++++++++ .../qbits/edi/guide/GuideRepository.java | 29 +- .../qbits/edi/map/InboundMapper.java | 4 + .../kingsrook/qbits/edi/map/MapPreview.java | 174 +---- .../qbits/edi/map/MapPreviewService.java | 321 +++++++++ .../qbits/edi/map/MapRuleValidator.java | 204 ++++++ .../qbits/edi/map/TransformRegistry.java | 49 ++ .../qbits/edi/processes/EdiProcesses.java | 2 + .../qbits/edi/storage/EdiRecords.java | 11 + src/main/resources/openapi/qqq-v1/edi.yaml | 626 ++++++++++++++++++ .../qbits/edi/EdiApiContractTest.java | 79 +++ .../kingsrook/qbits/edi/EdiApiHttpTest.java | 564 ++++++++++++++++ 18 files changed, 3128 insertions(+), 167 deletions(-) create mode 100644 src/main/java/com/kingsrook/qbits/edi/api/EdiApi.java create mode 100644 src/main/java/com/kingsrook/qbits/edi/api/EdiApiService.java create mode 100644 src/main/java/com/kingsrook/qbits/edi/api/EdiSchemas.java create mode 100644 src/main/java/com/kingsrook/qbits/edi/map/MapPreviewService.java create mode 100644 src/main/java/com/kingsrook/qbits/edi/map/MapRuleValidator.java create mode 100644 src/main/resources/openapi/qqq-v1/edi.yaml create mode 100644 src/test/java/com/kingsrook/qbits/edi/EdiApiContractTest.java create mode 100644 src/test/java/com/kingsrook/qbits/edi/EdiApiHttpTest.java diff --git a/CHANGELOG.md b/CHANGELOG.md index 0c503a7..23ff67c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,11 @@ Following [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and [Semantic ### Added +- Phase f1 `edi` 1.0.0 spec'd UI API: guide tree, map target/transform metadata, encrypted base64 sample preview with unsaved rule overrides, and per-rule validation under `/qqq/v1/qbit/edi`. +- Versioned `Edi...V1` schemas and generated OpenAPI fragment with the RC.6 contract/compatibility checks; real HTTP strict-mode permission, PHI, upload, isolation and validation tests. +- Shared sandbox/record-action preview service, current/original guide selection, caller-session transforms, structured revision/expected diffs and value-free PHI diagnostics. +- Transform registry argument schemas, including host-supplied schemas. Preview Map record actions now enforce the documented exact `edi.map.preview` permission through core's full-name permission checker. + - Phase g EDIFACT syntax 3/4 envelopes (UNA/UNB, optional UNG/UNE, UNH/UNT, UNZ), release escaping, native envelope findings, UNB identity routing, 14-digit UN control allocation and duplicate detection. - Original D96A/D01B ORDERS, ORDRSP, DESADV, INVOIC and CONTRL guides; ORDERS inbound and retail response outbound sample maps with composite fields and segment groups. - Standard-neutral document validation/mapping, mixed X12/EDIFACT inbox framing, CONTRL generation and reconciliation with overdue resolution and rejection alerts, native outbound envelopes and archived resend-as-new. diff --git a/README.md b/README.md index b83e44b..c8f5074 100644 --- a/README.md +++ b/README.md @@ -76,6 +76,57 @@ There are **26 persisted tables**, whose SQL names equal their QQQ names. appear as child lists. Partner-scoped form choices use RC.6 contingent filters, including shared transports/guides and local certificates/identities. +## UI API + +With optional Javalin present, the producer registers the `edi` extension, version `1.0.0`, +under `/qqq/v1/qbit/edi`. All routes are endpoint specs and pass strict UI-route auditing. + +| Method | Path after `/qqq/v1/qbit/edi` | Result | +|---|---|---| +| GET | `/guides/{guideId}/tree` | Nested loops, ordered segments, elements and up to 200 active codes per element | +| GET | `/maps/{mapId}/target` | Target fields, direct associations, current rules and registered transform argument schemas | +| POST | `/sandbox/preview` | Parse counts, positional findings, acknowledgement, mapped records and revision/expected diffs | +| POST | `/maps/{mapId}/validate` | Per-rule path, transform and required-argument errors | + +Preview accepts `{mapId, sampleId?, sampleBase64?, sampleName?, ruleOverrides?, guideRevision?}`. +Supply exactly one sample source. Uploads are base64 JSON, at most **5 MiB decoded**, archived as +encrypted `edi_map_sample` records through the same storage as Preview Map. No runtime files are +created. Configure the host's Javalin `http.maxRequestSize` to at least 8 MiB to allow base64 +overhead and request metadata. Oversized decoded samples return 400; a smaller host limit returns 413. + +`ruleOverrides` replaces the entire rule list in memory, including an empty list; it never saves +rules. `CURRENT` (default) selects the newest active revision of the pinned guide's name, standard, +version, transaction set and partner; `ORIGINAL` uses the map's pinned guide. Validate accepts +`{rules: [...]}` and checks the pinned guide. Rule fields and `transformArgs` have the same shape +as core table CRUD (`transformArgs` is a JSON object encoded as a string). + +Every read enforces table permissions and partner locks with the caller's session, regardless of +the host's default core-enforcement setting. Preview additionally requires **`edi.map.preview`**, +shared with both Preview Map record actions. Reading target metadata requires target/association +table read permissions; guide definitions, code lists, map rules and stored samples require their +own table reads. LOOKUP and host transforms run as the caller, without service-account elevation. +Only the internal PHI classification check and encrypted sample storage use the EDI system writer. + +For PHI, `result` and `ack.text` are null and both diff lists are empty; findings retain positions +and codes with generic messages. Mapping failures are value-free `MAP` findings. Preview never +writes target records. Diffs contain differing document records as `{documentIndex, before, after}`; +acknowledgements for multiple groups are concatenated, with `type: MIXED` for different ack types. + +The mapper saves `edi_map_rule` through the **core table API**, using its normal +`edi.map_rule.insert|edit|delete` permissions and record locks. There are no custom rule-write routes. +The Next mapper page is separate phase-f work. + +The generated contract is `src/main/resources/openapi/qqq-v1/edi.yaml`; all component names are +`Edi...V1`. Regenerate intentionally with: + +```powershell +./mvnw.cmd -B '-Dtest=EdiApiContractTest' '-Dedi.openapi.update=true' test +``` + +RC.6 provides `UiApiContract.check`, used to compare generated and checked-in YAML and run +`OpenApiCompatibilityChecker` against this initial fragment baseline. The extension also appears +in the host's `/qqq/v1/openapi.yaml` and `/qqq/v1/version`. + ## Inbound operation `InboundSink.receive(InboundPayload)` stores encrypted bytes and SHA-256 before returning a durable diff --git a/docs/DESIGN.md b/docs/DESIGN.md index 84cf838..5fe57ab 100644 --- a/docs/DESIGN.md +++ b/docs/DESIGN.md @@ -576,16 +576,41 @@ upload is stored as a sample), and the standard Review/Result lines show: 5. the diff against the previous map revision's result and against `expectedResultJson`. Nothing is written to target tables: the preview runs the whole pipeline with a no-op applier. -Phase f adds spec'd endpoints (`POST /qqq/v1/qbit/edi/sandbox/preview` and related) for the mapper -UI. +Phase f1 provides the `edi` 1.0.0 `MiddlewareSpecExtension`, with component schemas named +`Edi...V1` and a generated, contract-tested fragment at `openapi/qqq-v1/edi.yaml`: -### 8.4 No-code mapper UI (phase f; specified now, built later) +- `GET /qqq/v1/qbit/edi/guides/{guideId}/tree` +- `GET /qqq/v1/qbit/edi/maps/{mapId}/target` +- `POST /qqq/v1/qbit/edi/sandbox/preview` +- `POST /qqq/v1/qbit/edi/maps/{mapId}/validate` + +Preview Map and the HTTP preview share `MapPreviewService`. HTTP preview takes `mapId`, exactly +one of `sampleId`/`sampleBase64`, optional `sampleName`, a complete in-memory `ruleOverrides` +replacement, and `guideRevision`. Uploads are limited to 5 MiB decoded and stored encrypted as +samples; the host must allow at least 8 MiB HTTP bodies for base64 overhead. `CURRENT` (HTTP default) +selects the newest active revision in the pinned guide family and partner; `ORIGINAL` uses the +map's pinned guide, as the record action does. + +The response contains parse counts, positional findings, acknowledgement type/text, mapped records +and associations, structured per-document diffs, and `phiSuppressed`. PHI returns null result/ack +text, empty diffs and generic finding messages. Mapping errors are value-free `MAP` findings. +The preview action permission is exactly `edi.map.preview`, with no `.hasAccess` suffix. +Guide, sample and rule reads explicitly enforce permissions and record locks; transforms run in +the caller session. Only classification and encrypted sample storage use the internal system writer. + +### 8.4 No-code mapper UI (phase f1 API built; Next page follows) A custom Next UI page (custom UIs may be nav entries): - left, the guide tree (loops, segments, elements with usage and code lists); right, the target table's fields and associations; drag to create a rule, then choose a transform; - a live preview pane on a chosen sample (the sandbox endpoint), with errors inline; - revisions: edit a DRAFT, activate, compare revisions; -- reads and writes only through `/qqq/v1` (CRUD on `edi_map_rule` plus the extension endpoints). +- reads and writes only through `/qqq/v1`: the guide-tree and map-target GETs above, the sandbox + preview POST and the map validate POST; writes remain core CRUD on `edi_map_rule`, enforcing + `edi.map_rule.insert|edit|delete` and record locks. +- target metadata includes direct associations and transform argument JSON Schemas from the + registry. Validate accepts `{rules: [...]}`, returns indexed per-rule errors against the pinned + guide/target and never executes transforms or saves rules. Guide code lists return at most + 200 active codes per element. ## 9. Flows diff --git a/src/main/java/com/kingsrook/qbits/edi/EdiQBitProducer.java b/src/main/java/com/kingsrook/qbits/edi/EdiQBitProducer.java index 381a045..62dbbdf 100644 --- a/src/main/java/com/kingsrook/qbits/edi/EdiQBitProducer.java +++ b/src/main/java/com/kingsrook/qbits/edi/EdiQBitProducer.java @@ -138,6 +138,7 @@ public class EdiQBitProducer implements QBitMetaDataProducer .withPlugin(new com.kingsrook.qbits.edi.as2.As2Transport()).withPlugin(new com.kingsrook.qbits.edi.transport.UnavailableTransport("EMAIL"))); try { + com.kingsrook.qbits.edi.api.EdiApi.register(instance); com.kingsrook.qqq.middleware.javalin.QJavalinMetaData.ofOrWithNew(instance).withAdditionalRouteProviderReference( new com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference(com.kingsrook.qbits.edi.transport.HttpsRouteProvider.class)) .withAdditionalRouteProviderReference(new com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference(com.kingsrook.qbits.edi.as2.As2RouteProvider.class)); diff --git a/src/main/java/com/kingsrook/qbits/edi/api/EdiApi.java b/src/main/java/com/kingsrook/qbits/edi/api/EdiApi.java new file mode 100644 index 0000000..fc75eee --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/edi/api/EdiApi.java @@ -0,0 +1,293 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.edi.api; + +import java.util.List; +import java.util.Map; +import com.fasterxml.jackson.databind.DeserializationFeature; +import com.fasterxml.jackson.databind.MapperFeature; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.databind.cfg.CoercionAction; +import com.fasterxml.jackson.databind.cfg.CoercionInputShape; +import com.fasterxml.jackson.databind.json.JsonMapper; +import com.fasterxml.jackson.databind.type.LogicalType; +import com.kingsrook.qqq.backend.core.exceptions.QBadRequestException; +import com.kingsrook.qqq.backend.core.model.metadata.QInstance; +import com.kingsrook.qqq.middleware.javalin.specs.BasicOperation; +import com.kingsrook.qqq.middleware.javalin.specs.BasicResponse; +import com.kingsrook.qqq.middleware.javalin.specs.extensions.AbstractJsonEndpointSpec; +import com.kingsrook.qqq.middleware.javalin.specs.extensions.MiddlewareSpecExtension; +import com.kingsrook.qqq.middleware.javalin.specs.extensions.MiddlewareSpecExtensionsMetaData; +import com.kingsrook.qqq.openapi.model.HttpMethod; +import com.kingsrook.qqq.openapi.model.In; +import com.kingsrook.qqq.openapi.model.Parameter; +import com.kingsrook.qqq.openapi.model.Schema; +import com.kingsrook.qqq.openapi.model.Type; +import io.javalin.http.HttpStatus; + +/******************************************************************************* + ** The EDI 1.0.0 UI API: four spec'd operations in the default qbit namespace. + *******************************************************************************/ +public final class EdiApi +{ + /******************************************************************************* + ** Static registration only. + *******************************************************************************/ + private EdiApi() + { + } + + /******************************************************************************* + ** Install with optional Javalin from the qbit producer. + *******************************************************************************/ + public static void register(QInstance instance) + { + MiddlewareSpecExtensionsMetaData.ofOrWithNew(instance).withExtension(extension()); + } + + /******************************************************************************* + ** Shared descriptor for route binding and the committed contract. + *******************************************************************************/ + public static MiddlewareSpecExtension extension() + { + return (new MiddlewareSpecExtension().withName("edi").withSemanticVersion("1.0.0") + .withDescription("EDI sandbox and mapper, using the signed-in user's permissions.") + .withSpec(GuideTreeSpec.class).withSpec(MapTargetSpec.class).withSpec(PreviewSpec.class).withSpec(ValidateSpec.class)); + } + + /******************************************************************************* + ** Common typed JSON contract; core owns authentication and error mapping. + *******************************************************************************/ + public abstract static class EdiSpec extends AbstractJsonEndpointSpec + { + private final String path; + private final String response; + private final String request; + private final String summary; + + /******************************************************************************* + ** Immutable operation metadata. + *******************************************************************************/ + protected EdiSpec(String path, String response, String request, String summary) + { + this.path = path; + this.response = response; + this.request = request; + this.summary = summary; + } + + /******************************************************************************* + ** Publish the method, path and permission expectations. + *******************************************************************************/ + @Override + public BasicOperation defineBasicOperation() + { + return (new BasicOperation().withPath("/qbit/edi" + path).withHttpMethod(request == null ? HttpMethod.GET : HttpMethod.POST) + .withShortSummary(summary).withLongDescription(summary + ". Uses the caller session and table read permissions. Preview also requires edi.map.preview. Rule writes use core table CRUD.")); + } + + /******************************************************************************* + ** Declare positive integer record identifiers. + *******************************************************************************/ + @Override + public List defineRequestParameters() + { + String name = path.contains("{guideId}") ? "guideId" : path.contains("{mapId}") ? "mapId" : null; + return (name == null ? List.of() : List.of(new Parameter().withName(name).withIn(In.PATH).withRequired(true) + .withDescription("Positive record identifier.").withSchema(new Schema().withType(Type.INTEGER)))); + } + + /******************************************************************************* + ** All schemas use the Edi prefix and V1 suffix. + *******************************************************************************/ + @Override + public Map defineComponentSchemas() + { + return (path.startsWith("/guides/") ? EdiSchemas.all() : Map.of()); + } + + /******************************************************************************* + ** Response component name. + *******************************************************************************/ + @Override + protected String getResponseSchemaName() + { + return (response); + } + + /******************************************************************************* + ** Optional request body contract. + *******************************************************************************/ + @Override + protected Schema defineRequestBodySchema() + { + return (request == null ? null : EdiSchemas.ref(request)); + } + + /******************************************************************************* + ** Core middleware errors are explicitly documented. + *******************************************************************************/ + @Override + public List defineAdditionalBasicResponses() + { + return (List.of(new BasicResponse(HttpStatus.BAD_REQUEST, "Invalid request or sample exceeds 5 MiB", "EdiErrorV1"), + new BasicResponse(HttpStatus.UNAUTHORIZED, "Sign in required", "EdiErrorV1"), + new BasicResponse(HttpStatus.FORBIDDEN, "Permission denied", "EdiErrorV1"), + new BasicResponse(HttpStatus.NOT_FOUND, "Missing or invisible record", "EdiErrorV1"), + new BasicResponse(HttpStatus.CONTENT_TOO_LARGE, "Host HTTP request size limit exceeded", "EdiErrorV1"))); + } + + /******************************************************************************* + ** Reject malformed or overflowing identifiers without echoing request content. + *******************************************************************************/ + protected Integer id(JsonInput input, String name) throws QBadRequestException + { + try + { + int value = Integer.parseInt(input.getPathParam(name)); + if(value < 1) + { + throw (new NumberFormatException()); + } + return (value); + } + catch(NumberFormatException e) + { + throw (new QBadRequestException("A positive integer identifier is required")); + } + } + + /******************************************************************************* + ** Strict scalar typing prevents coercion of unsaved edits and sample identifiers. + *******************************************************************************/ + protected T body(JsonInput input, Class type) throws QBadRequestException + { + ObjectMapper mapper = JsonMapper.builder().disable(MapperFeature.ALLOW_COERCION_OF_SCALARS) + .disable(DeserializationFeature.ACCEPT_FLOAT_AS_INT).build(); + mapper.coercionConfigFor(LogicalType.Textual).setCoercion(CoercionInputShape.Integer, CoercionAction.Fail) + .setCoercion(CoercionInputShape.Float, CoercionAction.Fail).setCoercion(CoercionInputShape.Boolean, CoercionAction.Fail); + try + { + if(input.getBody() == null) + { + throw (new IllegalArgumentException("Object body required")); + } + return (mapper.convertValue(input.getBody(), type)); + } + catch(IllegalArgumentException e) + { + throw (new QBadRequestException("Invalid body; check property names and types")); + } + } + } + + /******************************************************************************* + ** Read the guide tree. + *******************************************************************************/ + public static class GuideTreeSpec extends EdiSpec + { + /******************************************************************************* + ** Public constructor for core spec discovery. + *******************************************************************************/ + public GuideTreeSpec() + { + super("/guides/{guideId}/tree", "EdiGuideTreeV1", null, "Read the guide tree"); + } + + /******************************************************************************* + ** Execute with the authenticated caller. + *******************************************************************************/ + @Override + protected Object handle(JsonInput input) throws Exception + { + return (new EdiApiService().tree(id(input, "guideId"))); + } + } + + /******************************************************************************* + ** Read target metadata and map rules. + *******************************************************************************/ + public static class MapTargetSpec extends EdiSpec + { + /******************************************************************************* + ** Public constructor for core spec discovery. + *******************************************************************************/ + public MapTargetSpec() + { + super("/maps/{mapId}/target", "EdiTargetV1", null, "Read target metadata and map rules"); + } + + /******************************************************************************* + ** Execute with the authenticated caller. + *******************************************************************************/ + @Override + protected Object handle(JsonInput input) throws Exception + { + return (new EdiApiService().target(id(input, "mapId"))); + } + } + + /******************************************************************************* + ** Preview an encrypted sample with optional unsaved rules. + *******************************************************************************/ + public static class PreviewSpec extends EdiSpec + { + /******************************************************************************* + ** Public constructor for core spec discovery. + *******************************************************************************/ + public PreviewSpec() + { + super("/sandbox/preview", "EdiPreviewV1", "EdiPreviewRequestV1", "Preview an encrypted sample with optional unsaved rules"); + } + + /******************************************************************************* + ** Execute with the authenticated caller. + *******************************************************************************/ + @Override + protected Object handle(JsonInput input) throws Exception + { + return (new EdiApiService().preview(body(input, EdiSchemas.EdiPreviewRequestV1.class))); + } + } + + /******************************************************************************* + ** Validate unsaved map rules. + *******************************************************************************/ + public static class ValidateSpec extends EdiSpec + { + /******************************************************************************* + ** Public constructor for core spec discovery. + *******************************************************************************/ + public ValidateSpec() + { + super("/maps/{mapId}/validate", "EdiValidationV1", "EdiValidateRequestV1", "Validate unsaved map rules"); + } + + /******************************************************************************* + ** Execute with the authenticated caller. + *******************************************************************************/ + @Override + protected Object handle(JsonInput input) throws Exception + { + return (new EdiApiService().validate(id(input, "mapId"), body(input, EdiSchemas.EdiValidateRequestV1.class))); + } + } +} diff --git a/src/main/java/com/kingsrook/qbits/edi/api/EdiApiService.java b/src/main/java/com/kingsrook/qbits/edi/api/EdiApiService.java new file mode 100644 index 0000000..41c5236 --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/edi/api/EdiApiService.java @@ -0,0 +1,321 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.edi.api; + +import java.io.ByteArrayInputStream; +import java.util.ArrayList; +import java.util.Base64; +import java.util.Comparator; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.Objects; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.kingsrook.qbits.edi.EdiRuntime; +import com.kingsrook.qbits.edi.api.EdiSchemas.EdiAssociationV1; +import com.kingsrook.qbits.edi.api.EdiSchemas.EdiElementV1; +import com.kingsrook.qbits.edi.api.EdiSchemas.EdiFieldV1; +import com.kingsrook.qbits.edi.api.EdiSchemas.EdiGuideTreeV1; +import com.kingsrook.qbits.edi.api.EdiSchemas.EdiLoopV1; +import com.kingsrook.qbits.edi.api.EdiSchemas.EdiPreviewRequestV1; +import com.kingsrook.qbits.edi.api.EdiSchemas.EdiPreviewV1; +import com.kingsrook.qbits.edi.api.EdiSchemas.EdiRuleErrorsV1; +import com.kingsrook.qbits.edi.api.EdiSchemas.EdiRuleV1; +import com.kingsrook.qbits.edi.api.EdiSchemas.EdiSegmentV1; +import com.kingsrook.qbits.edi.api.EdiSchemas.EdiTargetV1; +import com.kingsrook.qbits.edi.api.EdiSchemas.EdiTransformV1; +import com.kingsrook.qbits.edi.api.EdiSchemas.EdiValidateRequestV1; +import com.kingsrook.qbits.edi.api.EdiSchemas.EdiValidationV1; +import com.kingsrook.qbits.edi.map.MapPreviewService; +import com.kingsrook.qbits.edi.map.MapRuleValidator; +import com.kingsrook.qbits.edi.storage.EdiRecords; +import com.kingsrook.qbits.edi.storage.PayloadKind; +import com.kingsrook.qbits.edi.support.SystemWrites; +import com.kingsrook.qqq.backend.core.actions.permissions.PermissionEnforcement; +import com.kingsrook.qqq.backend.core.actions.permissions.PermissionsHelper; +import com.kingsrook.qqq.backend.core.actions.permissions.TablePermissionSubType; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.exceptions.QBadRequestException; +import com.kingsrook.qqq.backend.core.exceptions.QNotFoundException; +import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData; + +/******************************************************************************* + ** Caller-authorized projections and sandbox orchestration, using normal QQQ actions. + *******************************************************************************/ +public class EdiApiService +{ + public static final int MAX_SAMPLE_BYTES = 5 * 1024 * 1024; + private static final ObjectMapper JSON = new ObjectMapper().findAndRegisterModules(); + + /******************************************************************************* + ** Enforced reads preserve table permissions, field locks and partner record locks. + *******************************************************************************/ + private static List query(String table, QQueryFilter filter) throws Exception + { + return (EdiRecords.queryEnforced(table, filter)); + } + + /******************************************************************************* + ** A missing or invisible record has the same response. + *******************************************************************************/ + private static QRecord row(String table, Integer id) throws Exception + { + if(id == null || id < 1) + { + throw (new QBadRequestException("A positive identifier is required")); + } + var records = query(table, EdiRecords.equal("id", id)); + if(records.isEmpty()) + { + throw (new QNotFoundException("Not found")); + } + return (records.getFirst()); + } + + /******************************************************************************* + ** Read a guide as a nested tree, preserving duplicate loop ids in separate parents. + *******************************************************************************/ + public EdiGuideTreeV1 tree(Integer id) throws Exception + { + row("edi_guide", id); + List roots = new ArrayList<>(); + List rootSegments = new ArrayList<>(); + Map loops = new LinkedHashMap<>(); + Map latestPaths = new LinkedHashMap<>(); + var segments = query("edi_guide_segment", EdiRecords.equal("guideId", id)); + segments.sort(Comparator.comparing(r -> r.getValueInteger("sequence"))); + for(var segment : segments) + { + List elements = new ArrayList<>(); + var rows = query("edi_guide_element", EdiRecords.equal("guideSegmentId", segment.getValue("id"))); + rows.sort(Comparator.comparing((QRecord r) -> r.getValueInteger("position")).thenComparing(r -> Objects.requireNonNullElse(r.getValueInteger("componentPosition"), 0))); + for(var element : rows) + { + List codes = List.of(); + Integer codeList = element.getValueInteger("codeListId"); + if(codeList != null) + { + row("edi_code_list", codeList); + codes = query("edi_code_list_value", EdiRecords.equal("codeListId", codeList) + .withCriteria("isActive", com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator.EQUALS, true)) + .stream().map(r -> r.getValueString("code")).sorted().limit(200).toList(); + } + elements.add(new EdiElementV1(element.getValueInteger("position"), element.getValueInteger("componentPosition"), + element.getValueString("elementId"), element.getValueString("name"), element.getValueString("dataType"), + element.getValueInteger("minLength"), element.getValueInteger("maxLength"), element.getValueString("usage"), codeList, codes)); + } + EdiSegmentV1 node = new EdiSegmentV1(segment.getValueString("segmentId"), segment.getValueString("position"), segment.getValueString("usage"), + segment.getValueInteger("minUse"), segment.getValueInteger("maxUse"), segment.getValueString("name"), elements); + String loopId = segment.getValueString("loopId"); + if(loopId == null || loopId.isBlank()) + { + rootSegments.add(node); + } + else + { + String parent = latestPaths.get(segment.getValueString("parentLoopId")); + String path = (parent == null ? "" : parent + "/") + loopId; + EdiLoopV1 loop = loops.get(path); + if(loop == null) + { + loop = new EdiLoopV1(loopId, segment.getValueInteger("loopMaxRepeat"), new ArrayList<>(), new ArrayList<>()); + loops.put(path, loop); + (parent == null ? roots : loops.get(parent).loops()).add(loop); + } + latestPaths.put(loopId, path); + loop.segments().add(node); + } + } + return (new EdiGuideTreeV1(roots, rootSegments)); + } + + /******************************************************************************* + ** Return visible target metadata, rules and the same registry used by mapping. + *******************************************************************************/ + public EdiTargetV1 target(Integer id) throws Exception + { + QRecord map = row("edi_map", id); + QTableMetaData table = targetTable(map); + List associations = new ArrayList<>(); + if(table.getAssociations() != null) + { + for(var association : table.getAssociations()) + { + QTableMetaData child = QContext.getQInstance().getTable(association.getAssociatedTableName()); + associations.add(new EdiAssociationV1(association.getName(), child.getName(), Objects.requireNonNullElse(child.getLabel(), association.getName()), fields(child))); + } + } + List rules = new ArrayList<>(); + for(QRecord rule : query("edi_map_rule", EdiRecords.equal("mapId", id))) + { + rules.add(rule(rule)); + } + rules.sort(Comparator.comparing(r -> Objects.requireNonNullElse(r.sequence(), 0))); + return (new EdiTargetV1(table.getName(), fields(table), associations, rules, + EdiRuntime.current().transforms().argumentSchemas().entrySet().stream().map(e -> new EdiTransformV1(e.getKey(), e.getValue())).toList())); + } + + /******************************************************************************* + ** Resolve the configured target without accepting a client-supplied table. + *******************************************************************************/ + private QTableMetaData targetTable(QRecord map) throws Exception + { + QTableMetaData table = QContext.getQInstance().getTable(map.getValueString("tableName")); + if(table == null) + { + throw (new QBadRequestException("The map target table is unavailable")); + } + return (table); + } + + /******************************************************************************* + ** Table metadata is only exposed with the table's read permission. + *******************************************************************************/ + private List fields(QTableMetaData table) throws Exception + { + QueryInput input = new QueryInput(table.getName()); + input.setPermissionEnforcement(PermissionEnforcement.ENFORCE); + PermissionsHelper.checkTablePermissionThrowing(input, TablePermissionSubType.READ); + return (table.getFields().values().stream().map(f -> new EdiFieldV1(f.getName(), f.getLabel(), f.getType().name(), f.getIsRequired(), f.getPossibleValueSourceName())).toList()); + } + + /******************************************************************************* + ** Copy only documented rule fields; do not leak persistence or diagnostic fields. + *******************************************************************************/ + private EdiRuleV1 rule(QRecord r) + { + return (new EdiRuleV1(r.getValueInteger("id"), r.getValueInteger("sequence"), r.getValueString("ediPath"), r.getValueString("qqqPath"), + r.getValueString("transform"), r.getValueString("transformArgs"), r.getValueBoolean("required"), r.getValueString("condition"), r.getValueString("defaultValue"))); + } + + /******************************************************************************* + ** Convert unsaved edits to detached records; mapId is never accepted from a rule. + *******************************************************************************/ + private List records(List rules) throws Exception + { + if(rules == null || rules.size() > 10000 || rules.stream().anyMatch(Objects::isNull)) + { + throw (new QBadRequestException("Supply an array of at most 10000 rules")); + } + List result = new ArrayList<>(); + for(EdiRuleV1 rule : rules) + { + result.add(new QRecord().withValue("id", rule.id()).withValue("sequence", rule.sequence()).withValue("ediPath", rule.ediPath()) + .withValue("qqqPath", rule.qqqPath()).withValue("transform", rule.transform()).withValue("transformArgs", rule.transformArgs()) + .withValue("required", rule.required()).withValue("condition", rule.condition()).withValue("defaultValue", rule.defaultValue())); + } + return (result); + } + + /******************************************************************************* + ** Validate all submitted rules, with stable index correlation and no persistence. + *******************************************************************************/ + public EdiValidationV1 validate(Integer id, EdiValidateRequestV1 request) throws Exception + { + QRecord map = row("edi_map", id); + targetTable(map); + var guide = MapPreviewService.guide(map, "ORIGINAL"); + List rules = records(request.rules()); + List results = new ArrayList<>(); + for(int i = 0; i < rules.size(); i++) + { + QRecord rule = rules.get(i); + results.add(new EdiRuleErrorsV1(i, rule.getValueInteger("id"), rule.getValueInteger("sequence"), + new MapRuleValidator().validate(rule, map, guide, EdiRuntime.current().transforms()))); + } + return (new EdiValidationV1(results)); + } + + /******************************************************************************* + ** Authorize before decoding or saving an upload; previews never change map rules. + *******************************************************************************/ + public EdiPreviewV1 preview(EdiPreviewRequestV1 request) throws Exception + { + QRecord map = row("edi_map", request.mapId()); + AbstractActionInput permission = new AbstractActionInput(); + permission.setPermissionEnforcement(PermissionEnforcement.ENFORCE); + PermissionsHelper.checkProcessPermissionThrowing(permission, "edi_map_preview"); + String revision = Objects.requireNonNullElse(request.guideRevision(), "CURRENT"); + if(!List.of("CURRENT", "ORIGINAL").contains(revision)) + { + throw (new QBadRequestException("guideRevision must be CURRENT or ORIGINAL")); + } + if((request.sampleId() == null) == (request.sampleBase64() == null)) + { + throw (new QBadRequestException("Supply exactly one of sampleId and sampleBase64")); + } + List overrides = request.ruleOverrides() == null ? null : records(request.ruleOverrides()); + QRecord sample; + if(request.sampleId() != null) + { + sample = row("edi_map_sample", request.sampleId()); + if(!Objects.equals(sample.getValueInteger("mapId"), request.mapId())) + { + throw (new QNotFoundException("Not found")); + } + } + else + { + String encoded = request.sampleBase64(); + if(encoded.length() > ((MAX_SAMPLE_BYTES + 2) / 3) * 4) + { + throw (new QBadRequestException("Sample exceeds the 5 MiB limit")); + } + byte[] bytes; + try + { + bytes = Base64.getDecoder().decode(encoded); + } + catch(IllegalArgumentException e) + { + throw (new QBadRequestException("sampleBase64 must be valid base64")); + } + if(bytes.length == 0 || bytes.length > MAX_SAMPLE_BYTES) + { + throw (new QBadRequestException("Sample must contain 1 byte through 5 MiB")); + } + if(request.sampleName() != null && request.sampleName().length() > 255) + { + throw (new QBadRequestException("sampleName must be at most 255 characters")); + } + // As in the record action, only encrypted sample archival uses the EDI writer. + sample = SystemWrites.call(() -> + { + var ref = EdiRuntime.current().payloads().put(PayloadKind.SAMPLE, new ByteArrayInputStream(bytes)); + try + { + return (SystemWrites.insert("edi_map_sample", new QRecord().withValue("mapId", request.mapId()) + .withValue("name", Objects.requireNonNullElse(request.sampleName(), "Synthetic upload")).withValue("payloadRef", ref.value()))); + } + catch(Exception e) + { + EdiRuntime.current().payloads().delete(ref); + throw (e); + } + }); + } + return (JSON.convertValue(new MapPreviewService().preview(map, sample, overrides, revision), EdiPreviewV1.class)); + } +} diff --git a/src/main/java/com/kingsrook/qbits/edi/api/EdiSchemas.java b/src/main/java/com/kingsrook/qbits/edi/api/EdiSchemas.java new file mode 100644 index 0000000..f66daa0 --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/edi/api/EdiSchemas.java @@ -0,0 +1,528 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.edi.api; + +import java.util.List; +import java.util.Map; +import java.util.TreeMap; +import com.kingsrook.qqq.middleware.javalin.schemabuilder.ToSchema; +import com.kingsrook.qqq.openapi.model.Schema; +import com.kingsrook.qqq.openapi.model.Type; + +/******************************************************************************* + ** Versioned wire types; schemas describe values without exposing QRecord internals. + *******************************************************************************/ +public final class EdiSchemas +{ + /******************************************************************************* + ** Static schema catalog. + *******************************************************************************/ + private EdiSchemas() + { + } + + /******************************************************************************* + ** A reference to another versioned wire type. + *******************************************************************************/ + public static Schema ref(String name) + { + return (new Schema().withRefToSchema(name)); + } + + /******************************************************************************* + ** Array schema with an explicit item shape. + *******************************************************************************/ + private static Schema array(Schema item) + { + return (new Schema().withType(Type.ARRAY).withItems(item)); + } + + /******************************************************************************* + ** Ordered guide tree. Root segments are outside loops; nested loops retain parent scope. + *******************************************************************************/ + public record EdiGuideTreeV1(List loops, List segments) implements ToSchema + { + /******************************************************************************* + ** Describe this wire object for generated clients. + *******************************************************************************/ + @Override + public Schema toSchema() + { + return (new Schema().withType(Type.OBJECT).withDescription("Ordered guide tree. Root segments are outside loops; nested loops retain parent scope.") + .withProperties(new TreeMap<>(Map.ofEntries( + Map.entry("loops", array(ref("EdiLoopV1"))), + Map.entry("segments", array(ref("EdiSegmentV1"))))))); + } + } + + /******************************************************************************* + ** Guide loop, nested within its parent; repeat is the maximum occurrence count. + *******************************************************************************/ + public record EdiLoopV1(String id, Integer repeat, List loops, List segments) implements ToSchema + { + /******************************************************************************* + ** Describe this wire object for generated clients. + *******************************************************************************/ + @Override + public Schema toSchema() + { + return (new Schema().withType(Type.OBJECT).withDescription("Guide loop, nested within its parent; repeat is the maximum occurrence count.") + .withProperties(new TreeMap<>(Map.ofEntries( + Map.entry("id", new Schema().withType(Type.STRING).withNullable(true)), + Map.entry("repeat", new Schema().withType(Type.INTEGER).withNullable(true)), + Map.entry("loops", array(ref("EdiLoopV1"))), + Map.entry("segments", array(ref("EdiSegmentV1"))))))); + } + } + + /******************************************************************************* + ** Segment definition in guide sequence order. + *******************************************************************************/ + public record EdiSegmentV1(String id, String position, String usage, Integer min, Integer max, String name, List elements) implements ToSchema + { + /******************************************************************************* + ** Describe this wire object for generated clients. + *******************************************************************************/ + @Override + public Schema toSchema() + { + return (new Schema().withType(Type.OBJECT).withDescription("Segment definition in guide sequence order.") + .withProperties(new TreeMap<>(Map.ofEntries( + Map.entry("id", new Schema().withType(Type.STRING).withNullable(true)), + Map.entry("position", new Schema().withType(Type.STRING).withNullable(true)), + Map.entry("usage", new Schema().withType(Type.STRING).withNullable(true)), + Map.entry("min", new Schema().withType(Type.INTEGER).withNullable(true)), + Map.entry("max", new Schema().withType(Type.INTEGER).withNullable(true)), + Map.entry("name", new Schema().withType(Type.STRING).withNullable(true)), + Map.entry("elements", array(ref("EdiElementV1"))))))); + } + } + + /******************************************************************************* + ** Element or composite component; codes contain at most 200 active codes in lexical order. + *******************************************************************************/ + public record EdiElementV1(Integer position, Integer component, String id, String name, String type, Integer minLength, Integer maxLength, String usage, Integer codeListId, List codes) implements ToSchema + { + /******************************************************************************* + ** Describe this wire object for generated clients. + *******************************************************************************/ + @Override + public Schema toSchema() + { + return (new Schema().withType(Type.OBJECT).withDescription("Element or composite component; codes contain at most 200 active codes in lexical order.") + .withProperties(new TreeMap<>(Map.ofEntries( + Map.entry("position", new Schema().withType(Type.INTEGER).withNullable(true)), + Map.entry("component", new Schema().withType(Type.INTEGER).withNullable(true)), + Map.entry("id", new Schema().withType(Type.STRING).withNullable(true)), + Map.entry("name", new Schema().withType(Type.STRING).withNullable(true)), + Map.entry("type", new Schema().withType(Type.STRING).withNullable(true)), + Map.entry("minLength", new Schema().withType(Type.INTEGER).withNullable(true)), + Map.entry("maxLength", new Schema().withType(Type.INTEGER).withNullable(true)), + Map.entry("usage", new Schema().withType(Type.STRING).withNullable(true)), + Map.entry("codeListId", new Schema().withType(Type.INTEGER).withNullable(true)), + Map.entry("codes", array(new Schema().withType(Type.STRING).withNullable(true))))))); + } + } + + /******************************************************************************* + ** Target metadata visible to the caller, current rules and registered transforms. + *******************************************************************************/ + public record EdiTargetV1(String table, List fields, List associations, List rules, List transforms) implements ToSchema + { + /******************************************************************************* + ** Describe this wire object for generated clients. + *******************************************************************************/ + @Override + public Schema toSchema() + { + return (new Schema().withType(Type.OBJECT).withDescription("Target metadata visible to the caller, current rules and registered transforms.") + .withProperties(new TreeMap<>(Map.ofEntries( + Map.entry("table", new Schema().withType(Type.STRING).withNullable(true)), + Map.entry("fields", array(ref("EdiFieldV1"))), + Map.entry("associations", array(ref("EdiAssociationV1"))), + Map.entry("rules", array(ref("EdiRuleV1"))), + Map.entry("transforms", array(ref("EdiTransformV1"))))))); + } + } + + /******************************************************************************* + ** QQQ target field metadata. + *******************************************************************************/ + public record EdiFieldV1(String name, String label, String type, Boolean required, String possibleValueSourceName) implements ToSchema + { + /******************************************************************************* + ** Describe this wire object for generated clients. + *******************************************************************************/ + @Override + public Schema toSchema() + { + return (new Schema().withType(Type.OBJECT).withDescription("QQQ target field metadata.") + .withProperties(new TreeMap<>(Map.ofEntries( + Map.entry("name", new Schema().withType(Type.STRING).withNullable(true)), + Map.entry("label", new Schema().withType(Type.STRING).withNullable(true)), + Map.entry("type", new Schema().withType(Type.STRING).withNullable(true)), + Map.entry("required", new Schema().withType(Type.BOOLEAN)), + Map.entry("possibleValueSourceName", new Schema().withType(Type.STRING).withNullable(true)))))); + } + } + + /******************************************************************************* + ** Direct QQQ association and its target fields. + *******************************************************************************/ + public record EdiAssociationV1(String name, String table, String label, List fields) implements ToSchema + { + /******************************************************************************* + ** Describe this wire object for generated clients. + *******************************************************************************/ + @Override + public Schema toSchema() + { + return (new Schema().withType(Type.OBJECT).withDescription("Direct QQQ association and its target fields.") + .withProperties(new TreeMap<>(Map.ofEntries( + Map.entry("name", new Schema().withType(Type.STRING).withNullable(true)), + Map.entry("table", new Schema().withType(Type.STRING).withNullable(true)), + Map.entry("label", new Schema().withType(Type.STRING).withNullable(true)), + Map.entry("fields", array(ref("EdiFieldV1"))))))); + } + } + + /******************************************************************************* + ** Map rule. id is optional for unsaved rules. transformArgs is a JSON object encoded as a string, as in core table CRUD. + *******************************************************************************/ + public record EdiRuleV1(Integer id, Integer sequence, String ediPath, String qqqPath, String transform, String transformArgs, Boolean required, String condition, String defaultValue) implements ToSchema + { + /******************************************************************************* + ** Describe this wire object for generated clients. + *******************************************************************************/ + @Override + public Schema toSchema() + { + return (new Schema().withType(Type.OBJECT).withDescription("Map rule. id is optional for unsaved rules. transformArgs is a JSON object encoded as a string, as in core table CRUD.") + .withProperties(new TreeMap<>(Map.ofEntries( + Map.entry("id", new Schema().withType(Type.INTEGER).withNullable(true)), + Map.entry("sequence", new Schema().withType(Type.INTEGER).withNullable(true)), + Map.entry("ediPath", new Schema().withType(Type.STRING).withNullable(true)), + Map.entry("qqqPath", new Schema().withType(Type.STRING).withNullable(true)), + Map.entry("transform", new Schema().withType(Type.STRING).withNullable(true)), + Map.entry("transformArgs", new Schema().withType(Type.STRING).withNullable(true)), + Map.entry("required", new Schema().withType(Type.BOOLEAN).withNullable(true)), + Map.entry("condition", new Schema().withType(Type.STRING).withNullable(true)), + Map.entry("defaultValue", new Schema().withType(Type.STRING).withNullable(true)))))); + } + } + + /******************************************************************************* + ** Registered transform and JSON Schema describing its string-valued arguments. + *******************************************************************************/ + public record EdiTransformV1(String name, Map argumentSchema) implements ToSchema + { + /******************************************************************************* + ** Describe this wire object for generated clients. + *******************************************************************************/ + @Override + public Schema toSchema() + { + return (new Schema().withType(Type.OBJECT).withDescription("Registered transform and JSON Schema describing its string-valued arguments.") + .withProperties(new TreeMap<>(Map.ofEntries( + Map.entry("name", new Schema().withType(Type.STRING).withNullable(true)), + Map.entry("argumentSchema", new Schema().withType(Type.OBJECT).withAdditionalProperties(true).withDescription("JSON Schema: type, properties, required and additionalProperties. All transform arguments are strings.")))))); + } + } + + /******************************************************************************* + ** mapId is required. Supply exactly one of sampleId or sampleBase64. Uploads are at most 5 MiB decoded, stored encrypted. ruleOverrides replaces the whole rule set in memory. CURRENT (default) uses the newest active guide in the pinned guide family; ORIGINAL uses the pinned guide. + *******************************************************************************/ + public record EdiPreviewRequestV1(Integer mapId, Integer sampleId, String sampleBase64, String sampleName, List ruleOverrides, String guideRevision) implements ToSchema + { + /******************************************************************************* + ** Describe this wire object for generated clients. + *******************************************************************************/ + @Override + public Schema toSchema() + { + return (new Schema().withType(Type.OBJECT).withDescription("mapId is required. Supply exactly one of sampleId or sampleBase64. Uploads are at most 5 MiB decoded, stored encrypted. ruleOverrides replaces the whole rule set in memory. CURRENT (default) uses the newest active guide in the pinned guide family; ORIGINAL uses the pinned guide.") + .withProperties(new TreeMap<>(Map.ofEntries( + Map.entry("mapId", new Schema().withType(Type.INTEGER).withNullable(true)), + Map.entry("sampleId", new Schema().withType(Type.INTEGER).withNullable(true)), + Map.entry("sampleBase64", new Schema().withType(Type.STRING).withNullable(true).withFormat("byte").withMaxLength(6990508)), + Map.entry("sampleName", new Schema().withType(Type.STRING).withNullable(true).withMaxLength(255)), + Map.entry("ruleOverrides", array(ref("EdiRuleV1")).withNullable(true)), + Map.entry("guideRevision", new Schema().withType(Type.STRING).withNullable(true).withEnumValues(List.of("CURRENT", "ORIGINAL"))))))); + } + } + + /******************************************************************************* + ** Parse, validate, acknowledge and map without target writes. PHI suppresses result, ack text, and both diffs. + *******************************************************************************/ + public record EdiPreviewV1(EdiParseV1 parse, List findings, EdiAckV1 ack, EdiResultV1 result, EdiDiffV1 diff, Boolean phiSuppressed) implements ToSchema + { + /******************************************************************************* + ** Describe this wire object for generated clients. + *******************************************************************************/ + @Override + public Schema toSchema() + { + return (new Schema().withType(Type.OBJECT).withDescription("Parse, validate, acknowledge and map without target writes. PHI suppresses result, ack text, and both diffs.") + .withProperties(new TreeMap<>(Map.ofEntries( + Map.entry("parse", ref("EdiParseV1")), + Map.entry("findings", array(ref("EdiFindingV1"))), + Map.entry("ack", ref("EdiAckV1")), + Map.entry("result", ref("EdiResultV1").withNullable(true)), + Map.entry("diff", ref("EdiDiffV1")), + Map.entry("phiSuppressed", new Schema().withType(Type.BOOLEAN)))))); + } + } + + /******************************************************************************* + ** Counts of wire interchanges, groups and documents. + *******************************************************************************/ + public record EdiParseV1(Integer interchanges, Integer groups, Integer documents) implements ToSchema + { + /******************************************************************************* + ** Describe this wire object for generated clients. + *******************************************************************************/ + @Override + public Schema toSchema() + { + return (new Schema().withType(Type.OBJECT).withDescription("Counts of wire interchanges, groups and documents.") + .withProperties(new TreeMap<>(Map.ofEntries( + Map.entry("interchanges", new Schema().withType(Type.INTEGER).withNullable(true)), + Map.entry("groups", new Schema().withType(Type.INTEGER).withNullable(true)), + Map.entry("documents", new Schema().withType(Type.INTEGER).withNullable(true)))))); + } + } + + /******************************************************************************* + ** Value-free diagnostic. PHI messages are generic; only positions and codes are retained. + *******************************************************************************/ + public record EdiFindingV1(Integer snipLevel, String severity, Integer segmentPosition, String segmentId, String loopId, Integer elementPosition, String code, String message) implements ToSchema + { + /******************************************************************************* + ** Describe this wire object for generated clients. + *******************************************************************************/ + @Override + public Schema toSchema() + { + return (new Schema().withType(Type.OBJECT).withDescription("Value-free diagnostic. PHI messages are generic; only positions and codes are retained.") + .withProperties(new TreeMap<>(Map.ofEntries( + Map.entry("snipLevel", new Schema().withType(Type.INTEGER).withNullable(true)), + Map.entry("severity", new Schema().withType(Type.STRING).withNullable(true)), + Map.entry("segmentPosition", new Schema().withType(Type.INTEGER).withNullable(true)), + Map.entry("segmentId", new Schema().withType(Type.STRING).withNullable(true)), + Map.entry("loopId", new Schema().withType(Type.STRING).withNullable(true)), + Map.entry("elementPosition", new Schema().withType(Type.INTEGER).withNullable(true)), + Map.entry("code", new Schema().withType(Type.STRING).withNullable(true)), + Map.entry("message", new Schema().withType(Type.STRING).withNullable(true)))))); + } + } + + /******************************************************************************* + ** Concatenated acknowledgement previews; type is 997, 999, TA1, CONTRL, NONE or MIXED. text is null for PHI. + *******************************************************************************/ + public record EdiAckV1(String type, String text) implements ToSchema + { + /******************************************************************************* + ** Describe this wire object for generated clients. + *******************************************************************************/ + @Override + public Schema toSchema() + { + return (new Schema().withType(Type.OBJECT).withDescription("Concatenated acknowledgement previews; type is 997, 999, TA1, CONTRL, NONE or MIXED. text is null for PHI.") + .withProperties(new TreeMap<>(Map.ofEntries( + Map.entry("type", new Schema().withType(Type.STRING).withNullable(true)), + Map.entry("text", new Schema().withType(Type.STRING).withNullable(true)))))); + } + } + + /******************************************************************************* + ** Mapped records in document order; no target writes. + *******************************************************************************/ + public record EdiResultV1(List records) implements ToSchema + { + /******************************************************************************* + ** Describe this wire object for generated clients. + *******************************************************************************/ + @Override + public Schema toSchema() + { + return (new Schema().withType(Type.OBJECT).withDescription("Mapped records in document order; no target writes.") + .withProperties(new TreeMap<>(Map.ofEntries( + Map.entry("records", array(ref("EdiRecordV1"))))))); + } + } + + /******************************************************************************* + ** Field values and recursively associated records. Values use the target table field types. + *******************************************************************************/ + public record EdiRecordV1(Map values, Map> associations) implements ToSchema + { + /******************************************************************************* + ** Describe this wire object for generated clients. + *******************************************************************************/ + @Override + public Schema toSchema() + { + return (new Schema().withType(Type.OBJECT).withDescription("Field values and recursively associated records. Values use the target table field types.") + .withProperties(new TreeMap<>(Map.ofEntries( + Map.entry("values", new Schema().withType(Type.OBJECT).withAdditionalProperties(true)), + Map.entry("associations", new Schema().withType(Type.OBJECT).withAdditionalProperties(array(ref("EdiRecordV1")))))))); + } + } + + /******************************************************************************* + ** Only differing records are returned. Both lists are empty for PHI. + *******************************************************************************/ + public record EdiDiffV1(List againstPreviousRevision, List againstExpected) implements ToSchema + { + /******************************************************************************* + ** Describe this wire object for generated clients. + *******************************************************************************/ + @Override + public Schema toSchema() + { + return (new Schema().withType(Type.OBJECT).withDescription("Only differing records are returned. Both lists are empty for PHI.") + .withProperties(new TreeMap<>(Map.ofEntries( + Map.entry("againstPreviousRevision", array(ref("EdiDifferenceV1"))), + Map.entry("againstExpected", array(ref("EdiDifferenceV1"))))))); + } + } + + /******************************************************************************* + ** One document comparison; documentIndex is zero based. + *******************************************************************************/ + public record EdiDifferenceV1(Integer documentIndex, Object before, Object after) implements ToSchema + { + /******************************************************************************* + ** Describe this wire object for generated clients. + *******************************************************************************/ + @Override + public Schema toSchema() + { + return (new Schema().withType(Type.OBJECT).withDescription("One document comparison; documentIndex is zero based.") + .withProperties(new TreeMap<>(Map.ofEntries( + Map.entry("documentIndex", new Schema().withType(Type.INTEGER).withNullable(true)), + Map.entry("before", new Schema().withNullable(true).withDescription("Expected JSON or a previous mapped record.")), + Map.entry("after", new Schema().withType(Type.OBJECT).withAdditionalProperties(true).withNullable(true)))))); + } + } + + /******************************************************************************* + ** rules is required and may be empty; rules are never persisted. + *******************************************************************************/ + public record EdiValidateRequestV1(List rules) implements ToSchema + { + /******************************************************************************* + ** Describe this wire object for generated clients. + *******************************************************************************/ + @Override + public Schema toSchema() + { + return (new Schema().withType(Type.OBJECT).withDescription("rules is required and may be empty; rules are never persisted.") + .withProperties(new TreeMap<>(Map.ofEntries( + Map.entry("rules", array(ref("EdiRuleV1"))))))); + } + } + + /******************************************************************************* + ** One result per submitted rule, including rules with no errors. + *******************************************************************************/ + public record EdiValidationV1(List rules) implements ToSchema + { + /******************************************************************************* + ** Describe this wire object for generated clients. + *******************************************************************************/ + @Override + public Schema toSchema() + { + return (new Schema().withType(Type.OBJECT).withDescription("One result per submitted rule, including rules with no errors.") + .withProperties(new TreeMap<>(Map.ofEntries( + Map.entry("rules", array(ref("EdiRuleErrorsV1"))))))); + } + } + + /******************************************************************************* + ** Correlate errors using zero-based index, optional id and sequence. + *******************************************************************************/ + public record EdiRuleErrorsV1(Integer index, Integer id, Integer sequence, List errors) implements ToSchema + { + /******************************************************************************* + ** Describe this wire object for generated clients. + *******************************************************************************/ + @Override + public Schema toSchema() + { + return (new Schema().withType(Type.OBJECT).withDescription("Correlate errors using zero-based index, optional id and sequence.") + .withProperties(new TreeMap<>(Map.ofEntries( + Map.entry("index", new Schema().withType(Type.INTEGER).withNullable(true)), + Map.entry("id", new Schema().withType(Type.INTEGER).withNullable(true)), + Map.entry("sequence", new Schema().withType(Type.INTEGER).withNullable(true)), + Map.entry("errors", array(new Schema().withType(Type.STRING).withNullable(true))))))); + } + } + + /******************************************************************************* + ** Core middleware error body. + *******************************************************************************/ + public record EdiErrorV1(String error) implements ToSchema + { + /******************************************************************************* + ** Describe this wire object for generated clients. + *******************************************************************************/ + @Override + public Schema toSchema() + { + return (new Schema().withType(Type.OBJECT).withDescription("Core middleware error body.") + .withProperties(new TreeMap<>(Map.ofEntries( + Map.entry("error", new Schema().withType(Type.STRING).withNullable(true)))))); + } + } + + /******************************************************************************* + ** Complete deterministic schema catalog for all four operations. + *******************************************************************************/ + public static Map all() + { + Map result = new TreeMap<>(); + result.put("EdiGuideTreeV1", new EdiGuideTreeV1(null, null).toSchema()); + result.put("EdiLoopV1", new EdiLoopV1(null, null, null, null).toSchema()); + result.put("EdiSegmentV1", new EdiSegmentV1(null, null, null, null, null, null, null).toSchema()); + result.put("EdiElementV1", new EdiElementV1(null, null, null, null, null, null, null, null, null, null).toSchema()); + result.put("EdiTargetV1", new EdiTargetV1(null, null, null, null, null).toSchema()); + result.put("EdiFieldV1", new EdiFieldV1(null, null, null, null, null).toSchema()); + result.put("EdiAssociationV1", new EdiAssociationV1(null, null, null, null).toSchema()); + result.put("EdiRuleV1", new EdiRuleV1(null, null, null, null, null, null, null, null, null).toSchema()); + result.put("EdiTransformV1", new EdiTransformV1(null, null).toSchema()); + result.put("EdiPreviewRequestV1", new EdiPreviewRequestV1(null, null, null, null, null, null).toSchema()); + result.put("EdiPreviewV1", new EdiPreviewV1(null, null, null, null, null, null).toSchema()); + result.put("EdiParseV1", new EdiParseV1(null, null, null).toSchema()); + result.put("EdiFindingV1", new EdiFindingV1(null, null, null, null, null, null, null, null).toSchema()); + result.put("EdiAckV1", new EdiAckV1(null, null).toSchema()); + result.put("EdiResultV1", new EdiResultV1(null).toSchema()); + result.put("EdiRecordV1", new EdiRecordV1(null, null).toSchema()); + result.put("EdiDiffV1", new EdiDiffV1(null, null).toSchema()); + result.put("EdiDifferenceV1", new EdiDifferenceV1(null, null, null).toSchema()); + result.put("EdiValidateRequestV1", new EdiValidateRequestV1(null).toSchema()); + result.put("EdiValidationV1", new EdiValidationV1(null).toSchema()); + result.put("EdiRuleErrorsV1", new EdiRuleErrorsV1(null, null, null, null).toSchema()); + result.put("EdiErrorV1", new EdiErrorV1(null).toSchema()); + return (result); + } +} diff --git a/src/main/java/com/kingsrook/qbits/edi/guide/GuideRepository.java b/src/main/java/com/kingsrook/qbits/edi/guide/GuideRepository.java index 0e386ad..91f9be1 100644 --- a/src/main/java/com/kingsrook/qbits/edi/guide/GuideRepository.java +++ b/src/main/java/com/kingsrook/qbits/edi/guide/GuideRepository.java @@ -47,24 +47,37 @@ public class GuideRepository *******************************************************************************/ public Guide load(Integer id) throws Exception { - QRecord guide = rows("edi_guide", "id", id).getFirst(); - List definitions = new ArrayList<>(rows("edi_guide_segment", "guideId", id)); + return (load(id, false)); + } + + /******************************************************************************* + ** UI preview explicitly enforces every guide read, independently of host defaults. + *******************************************************************************/ + public Guide load(Integer id, boolean enforcePermissions) throws Exception + { + var guideRows = rows("edi_guide", "id", id, enforcePermissions); + if(guideRows.isEmpty()) + { + throw (new com.kingsrook.qqq.backend.core.exceptions.QNotFoundException("Not found")); + } + QRecord guide = guideRows.getFirst(); + List definitions = new ArrayList<>(rows("edi_guide_segment", "guideId", id, enforcePermissions)); definitions.sort(Comparator.comparing(r -> r.getValueInteger("sequence"))); List segments = new ArrayList<>(); - List rules = rows("edi_guide_rule", "guideId", id); + List rules = rows("edi_guide_rule", "guideId", id, enforcePermissions); for(QRecord definition : definitions) { Integer segmentId = definition.getValueInteger("id"); List elements = new ArrayList<>(); - for(QRecord element : rows("edi_guide_element", "guideSegmentId", segmentId)) + for(QRecord element : rows("edi_guide_element", "guideSegmentId", segmentId, enforcePermissions)) { List codes = null; QRecord codeList = null; if(element.getValueInteger("codeListId") != null) { - codeList = rows("edi_code_list", "id", element.getValueInteger("codeListId")).getFirst(); + codeList = rows("edi_code_list", "id", element.getValueInteger("codeListId"), enforcePermissions).getFirst(); codes = new ArrayList<>(); - for(QRecord code : rows("edi_code_list_value", "codeListId", element.getValueInteger("codeListId"))) + for(QRecord code : rows("edi_code_list_value", "codeListId", element.getValueInteger("codeListId"), enforcePermissions)) { if(Boolean.TRUE.equals(code.getValueBoolean("isActive"))) { @@ -206,9 +219,9 @@ public class GuideRepository /******************************************************************************* ** Query normalized child records. *******************************************************************************/ - private static List rows(String table, String field, Integer id) throws Exception + private static List rows(String table, String field, Integer id, boolean enforcePermissions) throws Exception { - return (EdiRecords.query(table, EdiRecords.equal(field, id))); + return (enforcePermissions ? EdiRecords.queryEnforced(table, EdiRecords.equal(field, id)) : EdiRecords.query(table, EdiRecords.equal(field, id))); } /******************************************************************************* diff --git a/src/main/java/com/kingsrook/qbits/edi/map/InboundMapper.java b/src/main/java/com/kingsrook/qbits/edi/map/InboundMapper.java index e6f6baf..3c69048 100644 --- a/src/main/java/com/kingsrook/qbits/edi/map/InboundMapper.java +++ b/src/main/java/com/kingsrook/qbits/edi/map/InboundMapper.java @@ -121,6 +121,10 @@ public class InboundMapper } } } + catch(com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException e) + { + throw (e); + } catch(Exception e) { throw new IllegalArgumentException("Mapping failed at rule " + rule.getValueInteger("sequence") + " (" + rule.getValueString("ediPath") + " -> " + rule.getValueString("qqqPath") + ")"); diff --git a/src/main/java/com/kingsrook/qbits/edi/map/MapPreview.java b/src/main/java/com/kingsrook/qbits/edi/map/MapPreview.java index 36830ba..df4fec0 100644 --- a/src/main/java/com/kingsrook/qbits/edi/map/MapPreview.java +++ b/src/main/java/com/kingsrook/qbits/edi/map/MapPreview.java @@ -20,183 +20,47 @@ package com.kingsrook.qbits.edi.map; -import java.io.InputStream; import java.util.ArrayList; import java.util.List; import java.util.Map; import com.fasterxml.jackson.databind.ObjectMapper; -import com.kingsrook.qbits.edi.EdiRuntime; -import com.kingsrook.qbits.edi.ack.AckLevel; -import com.kingsrook.qbits.edi.guide.GuideRepository; -import com.kingsrook.qbits.edi.storage.EdiRecords; -import com.kingsrook.qbits.edi.storage.PayloadRef; import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine; import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface; import com.kingsrook.qqq.backend.core.model.actions.processes.Status; import com.kingsrook.qqq.backend.core.model.data.QRecord; /******************************************************************************* - ** No-op apply preview with ordinary Result lines; PHI never enters process output. + ** Standard record-action presentation of the shared sandbox preview. *******************************************************************************/ public class MapPreview { /******************************************************************************* - ** Parse, validate, acknowledge and map a stored synthetic sample without target writes. + ** Render the same pipeline as HTTP using ordinary Review/Result lines. *******************************************************************************/ public ArrayList preview(QRecord map, QRecord sample) throws Exception { - EdiRuntime runtime = EdiRuntime.current(); - boolean phi = List.of("837", "835", "270", "271", "276", "277", "834").contains(map.getValueString("transactionSet")); - phi = phi || com.kingsrook.qbits.edi.support.SystemWrites.call(() -> !EdiRecords.query("edi_agreement_document", EdiRecords.equal("mapId", map.getValue("id")) - .withCriteria("containsPhi", com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator.EQUALS, true)).isEmpty()); - var guide = new GuideRepository().load(map.getValueInteger("guideId")); + Map preview = new MapPreviewService().preview(map, sample, null, "ORIGINAL"); + var json = new ObjectMapper().findAndRegisterModules(); ArrayList lines = new ArrayList<>(); - try(var logging = com.kingsrook.qbits.edi.security.PhiLogScope.open(phi); InputStream stream = runtime.payloads().open(new PayloadRef(sample.getValueString("payloadRef"), null, 0))) + lines.add(new ProcessSummaryLine(Status.INFO, "Parsed: " + json.writeValueAsString(preview.get("parse")))); + lines.add(new ProcessSummaryLine(Status.INFO, "SNIP findings: " + json.writeValueAsString(preview.get("findings")))); + if(Boolean.TRUE.equals(preview.get("phiSuppressed"))) { - if("EDIFACT".equals(map.getValueString("standard"))) - { - return (edifact(map, sample, stream, phi)); - } - var interchanges = runtime.reader().read(stream); - lines.add(new ProcessSummaryLine(Status.INFO, interchanges.size(), "interchanges parsed")); - for(var interchange : interchanges) - { - lines.add(new ProcessSummaryLine(Status.INFO, interchange.groups().size(), "functional groups parsed")); - for(var group : interchange.groups()) - { - lines.add(new ProcessSummaryLine(Status.INFO, group.transactions().size(), "transaction sets parsed")); - Map> all = new java.util.LinkedHashMap<>(); - for(var transaction : group.transactions()) - { - var findings = runtime.validator().validate(transaction, guide, phi); - all.put(transaction.st().value(2), findings); - for(int snip = 1; snip <= 7; snip++) - { - int level = snip; - lines.add(new ProcessSummaryLine(Status.INFO, (int) findings.stream().filter(f -> f.snipLevel() == level).count(), "findings at SNIP " + snip)); - } - List rules = EdiRecords.query("edi_map_rule", EdiRecords.equal("mapId", map.getValue("id"))); - rules.sort(java.util.Comparator.comparing(r -> r.getValueInteger("sequence"))); - var mapped = InboundApplier.asService(runtime.config().getServiceSessionSupplier(), () -> new InboundMapper(runtime.transforms()).map(transaction, guide, map, rules)); - if(phi) - { - lines.add(new ProcessSummaryLine(Status.INFO, "PHI map result suppressed; no target records written")); - } - else - { - ObjectMapper json = new ObjectMapper().findAndRegisterModules(); - String actual = resultJson(mapped.record()); - lines.add(new ProcessSummaryLine(Status.INFO, actual)); - String expected = sample.getValueString("expectedResultJson"); - if(expected != null) - { - diff(lines, "Expected result", json.readTree(expected), json.readTree(actual)); - } - Integer revision = map.getValueInteger("revision"); - var previous = EdiRecords.query("edi_map", EdiRecords.equal("name", map.getValue("name"))); - previous.removeIf(r -> !java.util.Objects.equals(r.getValue("partnerId"), map.getValue("partnerId")) - || revision == null || r.getValueInteger("revision") == null || r.getValueInteger("revision") >= revision); - previous.sort(java.util.Comparator.comparing((QRecord r) -> r.getValueInteger("revision")).reversed()); - if(!previous.isEmpty()) - { - QRecord prior = previous.getFirst(); - var priorGuide = new GuideRepository().load(prior.getValueInteger("guideId")); - List priorRules = EdiRecords.query("edi_map_rule", EdiRecords.equal("mapId", prior.getValue("id"))); - priorRules.sort(java.util.Comparator.comparing(r -> r.getValueInteger("sequence"))); - var priorResult = InboundApplier.asService(runtime.config().getServiceSessionSupplier(), - () -> new InboundMapper(runtime.transforms()).map(transaction, priorGuide, prior, priorRules)); - diff(lines, "Previous revision " + prior.getValueInteger("revision"), json.readTree(resultJson(priorResult.record())), json.readTree(actual)); - } - } - } - var ack = phi && group.gs().value(8).startsWith("005010X") ? runtime.ack999().build(group, all, AckLevel.ALL_TRANSACTIONS, "0001") - : runtime.ack997().build(group, all, AckLevel.ALL_TRANSACTIONS, "0001"); - lines.add(new ProcessSummaryLine(Status.INFO, phi ? "Acknowledgement preview suppressed for PHI document" - : runtime.writer().write(com.kingsrook.qbits.edi.ack.AckEnvelope.wrap(interchange, group, ack, "1", "1", java.time.LocalDateTime.now(runtime.config().getClock()))))); - } - } - } - return (lines); - } - - /******************************************************************************* - ** Preview native EDIFACT messages and CONTRL without applying target records. - *******************************************************************************/ - private ArrayList edifact(QRecord map, QRecord sample, InputStream stream, boolean phi) throws Exception - { - var runtime = EdiRuntime.current(); - var guide = new GuideRepository().load(map.getValueInteger("guideId")); - var rules = EdiRecords.query("edi_map_rule", EdiRecords.equal("mapId", map.getValue("id"))); - rules.sort(java.util.Comparator.comparing(r -> r.getValueInteger("sequence"))); - ArrayList lines = new ArrayList<>(); - for(var interchange : new com.kingsrook.qbits.edi.edifact.EdifactEnvelopeReader(runtime.config().getMaxSegmentsPerTransaction()).read(stream)) - { - Map> findings = new java.util.LinkedHashMap<>(); - lines.add(new ProcessSummaryLine(Status.INFO, interchange.findings().size(), "envelope findings")); - for(var message : interchange.allMessages()) - { - var result = runtime.validator().validate(message, guide, phi); - findings.put(message.reference(), result); - lines.add(new ProcessSummaryLine(Status.INFO, result.size(), "validation findings")); - var mapped = InboundApplier.asService(runtime.config().getServiceSessionSupplier(), () -> new InboundMapper(runtime.transforms()).map(message, guide, map, rules)); - if(phi) - { - lines.add(new ProcessSummaryLine(Status.INFO, "PHI map result suppressed; no target records written")); - continue; - } - String actual = resultJson(mapped.record()); - lines.add(new ProcessSummaryLine(Status.INFO, actual)); - if(sample.getValueString("expectedResultJson") != null) - { - diff(lines, "Expected result", new ObjectMapper().readTree(sample.getValueString("expectedResultJson")), new ObjectMapper().readTree(actual)); - } - } - if(interchange.unb() != null && !phi) - { - var ack = new com.kingsrook.qbits.edi.edifact.ContrlBuilder().build(interchange, findings, AckLevel.ALL_TRANSACTIONS, "1", map.getValueString("version")); - lines.add(new ProcessSummaryLine(Status.INFO, new com.kingsrook.qbits.edi.edifact.EdifactWriter().write(ack.segments(), interchange.separators(), ""))); - } - } - return (lines); - } - - /******************************************************************************* - ** Stable record JSON excludes QRecord diagnostics and implementation metadata. - *******************************************************************************/ - private static String resultJson(QRecord record) throws Exception - { - Map result = new java.util.LinkedHashMap<>(); - result.put("values", record.getValues()); - Map associations = new java.util.LinkedHashMap<>(); - if(record.getAssociatedRecords() != null) - { - for(var entry : record.getAssociatedRecords().entrySet()) - { - List children = new ArrayList<>(); - for(QRecord child : entry.getValue()) - { - children.add(new ObjectMapper().readTree(resultJson(child))); - } - associations.put(entry.getKey(), children); - } - } - result.put("associations", associations); - return (new ObjectMapper().findAndRegisterModules().writeValueAsString(result)); - } - - /******************************************************************************* - ** Show a structured before/after diff for synthetic samples only. - *******************************************************************************/ - private static void diff(ArrayList lines, String label, com.fasterxml.jackson.databind.JsonNode before, com.fasterxml.jackson.databind.JsonNode after) throws Exception - { - if(before.equals(after)) - { - lines.add(new ProcessSummaryLine(Status.INFO, label + " matches")); + lines.add(new ProcessSummaryLine(Status.INFO, "PHI map result suppressed; no target records written")); + lines.add(new ProcessSummaryLine(Status.INFO, "Acknowledgement preview suppressed for PHI document")); } else { - Map difference = Map.of("comparison", label, "before", before, "after", after); - lines.add(new ProcessSummaryLine(Status.INFO, label + " differs: " + new ObjectMapper().writeValueAsString(difference))); + lines.add(new ProcessSummaryLine(Status.INFO, json.writeValueAsString(preview.get("result")))); + lines.add(new ProcessSummaryLine(Status.INFO, json.writeValueAsString(preview.get("ack")))); + Map diff = (Map) preview.get("diff"); + for(String key : List.of("againstExpected", "againstPreviousRevision")) + { + String label = key.equals("againstExpected") ? "Expected result" : "Previous revision"; + List differences = (List) diff.get(key); + lines.add(new ProcessSummaryLine(Status.INFO, label + (differences.isEmpty() ? " matches" : " differs: " + json.writeValueAsString(differences)))); + } } + return (lines); } } diff --git a/src/main/java/com/kingsrook/qbits/edi/map/MapPreviewService.java b/src/main/java/com/kingsrook/qbits/edi/map/MapPreviewService.java new file mode 100644 index 0000000..c3025bf --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/edi/map/MapPreviewService.java @@ -0,0 +1,321 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.edi.map; + +import java.io.InputStream; +import java.util.ArrayList; +import java.util.Comparator; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.Objects; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.kingsrook.qbits.edi.EdiRuntime; +import com.kingsrook.qbits.edi.ack.AckEnvelope; +import com.kingsrook.qbits.edi.ack.AckLevel; +import com.kingsrook.qbits.edi.document.EdiDocument; +import com.kingsrook.qbits.edi.edifact.ContrlBuilder; +import com.kingsrook.qbits.edi.edifact.EdifactEnvelopeReader; +import com.kingsrook.qbits.edi.edifact.EdifactWriter; +import com.kingsrook.qbits.edi.guide.Guide; +import com.kingsrook.qbits.edi.guide.GuideRepository; +import com.kingsrook.qbits.edi.guide.ValidationFinding; +import com.kingsrook.qbits.edi.security.PhiLogScope; +import com.kingsrook.qbits.edi.storage.EdiRecords; +import com.kingsrook.qbits.edi.storage.PayloadRef; +import com.kingsrook.qbits.edi.support.SystemWrites; +import com.kingsrook.qbits.edi.x12.EnvelopeFinding; +import com.kingsrook.qqq.backend.core.exceptions.QBadRequestException; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator; +import com.kingsrook.qqq.backend.core.model.data.QRecord; + +/******************************************************************************* + ** Shared no-apply preview pipeline. All mapping, including LOOKUP, uses the caller. + ** This service has no optional middleware dependency. + *******************************************************************************/ +public class MapPreviewService +{ + private static final ObjectMapper JSON = new ObjectMapper().findAndRegisterModules(); + + /******************************************************************************* + ** Read ordered persisted rules, or order a copy of the unsaved replacement set. + *******************************************************************************/ + public static List rules(QRecord map, List overrides) throws Exception + { + List result = new ArrayList<>(overrides == null ? EdiRecords.queryEnforced("edi_map_rule", EdiRecords.equal("mapId", map.getValue("id"))) : overrides); + result.sort(Comparator.comparing(r -> Objects.requireNonNullElse(r.getValueInteger("sequence"), 0))); + return (result); + } + + /******************************************************************************* + ** PHI classification cannot be weakened by the caller or by hidden agreements. + *******************************************************************************/ + public static boolean containsPhi(QRecord map) throws Exception + { + return (healthcare(map.getValueString("transactionSet")) || SystemWrites.call(() -> !EdiRecords.query("edi_agreement_document", + EdiRecords.equal("mapId", map.getValue("id")).withCriteria("containsPhi", QCriteriaOperator.EQUALS, true)).isEmpty())); + } + + /******************************************************************************* + ** Healthcare document types always suppress values, even with a mismatched map. + *******************************************************************************/ + private static boolean healthcare(String type) + { + return (type != null && List.of("837", "835", "270", "271", "276", "277", "834").contains(type)); + } + + /******************************************************************************* + ** ORIGINAL is the pinned row; CURRENT follows only the same guide family. + *******************************************************************************/ + public static Guide guide(QRecord map, String revision) throws Exception + { + Integer id = map.getValueInteger("guideId"); + if("CURRENT".equals(revision)) + { + var pinnedRows = EdiRecords.queryEnforced("edi_guide", EdiRecords.equal("id", id)); + if(pinnedRows.isEmpty()) + { + throw (new com.kingsrook.qqq.backend.core.exceptions.QNotFoundException("Not found")); + } + QRecord pinned = pinnedRows.getFirst(); + var candidates = EdiRecords.queryEnforced("edi_guide", EdiRecords.equal("name", pinned.getValue("name")).withCriteria("status", QCriteriaOperator.EQUALS, "ACTIVE")); + candidates.removeIf(r -> !Objects.equals(r.getValue("partnerId"), pinned.getValue("partnerId")) + || !Objects.equals(r.getValue("standard"), pinned.getValue("standard")) + || !Objects.equals(r.getValue("version"), pinned.getValue("version")) + || !Objects.equals(r.getValue("transactionSet"), pinned.getValue("transactionSet"))); + candidates.sort(Comparator.comparing((QRecord r) -> Objects.requireNonNullElse(r.getValueInteger("revision"), 0)).reversed()); + if(!candidates.isEmpty()) + { + id = candidates.getFirst().getValueInteger("id"); + } + } + return (new GuideRepository().load(id, true)); + } + + /******************************************************************************* + ** Parse and inspect the complete sample before exposing any values. + *******************************************************************************/ + public Map preview(QRecord map, QRecord sample, List overrides, String revision) throws Exception + { + EdiRuntime runtime = EdiRuntime.current(); + boolean phi = containsPhi(map); + Guide guide = guide(map, revision); + List rules = rules(map, overrides); + State state = new State(); + try(var logging = PhiLogScope.open(true); InputStream stream = SystemWrites.call(() -> runtime.payloads().open(new PayloadRef(sample.getValueString("payloadRef"), null, 0)))) + { + if("EDIFACT".equals(map.getValueString("standard"))) + { + var interchanges = new EdifactEnvelopeReader(runtime.config().getMaxSegmentsPerTransaction()).read(stream); + state.interchanges = interchanges.size(); + for(var interchange : interchanges) + { + state.groups += interchange.groups().size(); + state.envelope(interchange.findings()); + interchange.groups().forEach(group -> state.envelope(group.findings())); + Map> all = new LinkedHashMap<>(); + for(var message : interchange.allMessages()) + { + all.put(message.reference(), document(state, map, sample, rules, guide, message, phi)); + } + if(interchange.unb() != null) + { + var ack = new ContrlBuilder().build(interchange, all, AckLevel.ALL_TRANSACTIONS, "1", map.getValueString("version")); + state.ack("CONTRL", phi ? null : new EdifactWriter().write(ack.segments(), interchange.separators(), "")); + } + } + } + else + { + var interchanges = runtime.reader().read(stream); + phi = phi || interchanges.stream().flatMap(i -> i.groups().stream()).flatMap(g -> g.transactions().stream()).anyMatch(t -> healthcare(t.st().value(1))); + state.interchanges = interchanges.size(); + for(var interchange : interchanges) + { + state.groups += interchange.groups().size(); + state.envelope(interchange.findings()); + if(!interchange.findings().isEmpty()) + { + var ta1 = runtime.ta1().build(interchange.isa(), interchange.findings()); + var isa = AckEnvelope.replyIsa(interchange.isa(), "1", java.time.LocalDateTime.now(runtime.config().getClock())); + var reply = new com.kingsrook.qbits.edi.x12.X12Interchange(interchange.separators(), isa, + com.kingsrook.qbits.edi.x12.X12Segment.of("IEA", 3, "0", isa.value(13)), List.of(), List.of(ta1), List.of()); + state.ack("TA1", phi ? null : runtime.writer().write(reply)); + } + for(var group : interchange.groups()) + { + state.envelope(group.findings()); + Map> all = new LinkedHashMap<>(); + for(var transaction : group.transactions()) + { + all.put(transaction.st().value(2), document(state, map, sample, rules, guide, transaction, phi)); + } + boolean use999 = phi && group.gs().value(8).startsWith("005010X"); + var ack = use999 ? runtime.ack999().build(group, all, AckLevel.ALL_TRANSACTIONS, "0001") : runtime.ack997().build(group, all, AckLevel.ALL_TRANSACTIONS, "0001"); + state.ack(use999 ? "999" : "997", phi ? null : runtime.writer().write(AckEnvelope.wrap(interchange, group, ack, "1", "1", java.time.LocalDateTime.now(runtime.config().getClock())))); + } + } + } + } + catch(com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException e) + { + throw (e); + } + catch(Exception e) + { + // Parser, custom transform and expected-JSON diagnostics may contain payload values. + throw (new QBadRequestException("Sample could not be previewed; check the document, guide and rules.")); + } + return (object("parse", Map.of("interchanges", state.interchanges, "groups", state.groups, "documents", state.documents), + "findings", state.findings, "ack", object("type", state.ackType, "text", phi ? null : state.ackText.toString()), + "result", phi ? null : Map.of("records", state.records), "diff", Map.of("againstPreviousRevision", state.previous, "againstExpected", state.expected), "phiSuppressed", phi)); + } + + /******************************************************************************* + ** Validate and map one document, collecting only value-free findings for PHI. + *******************************************************************************/ + private List document(State state, QRecord map, QRecord sample, List rules, Guide guide, EdiDocument document, boolean phi) throws Exception + { + int index = state.documents++; + state.envelope(document.findings()); + var findings = EdiRuntime.current().validator().validate(document, guide, phi); + for(var finding : findings) + { + state.findings.add(object("snipLevel", finding.snipLevel(), "severity", finding.severity(), "segmentPosition", finding.segmentPosition(), + "segmentId", finding.segmentId(), "loopId", finding.loopId(), "elementPosition", finding.elementPosition(), + "code", finding.elementPosition() == null ? finding.ak3Code() : finding.ak4Code(), "message", phi ? "Validation finding" : finding.message())); + } + QRecord mapped; + try + { + mapped = new InboundMapper(EdiRuntime.current().transforms()).map(document, guide, map, rules).record(); + } + catch(com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException e) + { + throw (e); + } + catch(Exception e) + { + state.findings.add(object("snipLevel", 0, "severity", "ERROR", "segmentPosition", null, "segmentId", null, "loopId", null, + "elementPosition", null, "code", "MAP", "message", "Mapping failed; check the rule paths and transform arguments")); + return (findings); + } + if(!phi) + { + Map actual = record(mapped); + state.records.add(actual); + String expected = sample.getValueString("expectedResultJson"); + if(expected != null) + { + difference(state.expected, index, JSON.readValue(expected, Object.class), actual); + } + Integer revision = map.getValueInteger("revision"); + var previous = EdiRecords.queryEnforced("edi_map", EdiRecords.equal("name", map.getValue("name"))); + previous.removeIf(r -> !Objects.equals(r.getValue("partnerId"), map.getValue("partnerId")) + || !Objects.equals(r.getValue("standard"), map.getValue("standard")) || !Objects.equals(r.getValue("transactionSet"), map.getValue("transactionSet")) + || !Objects.equals(r.getValue("direction"), map.getValue("direction")) || !Objects.equals(r.getValue("version"), map.getValue("version")) + || !Objects.equals(r.getValue("tableName"), map.getValue("tableName")) + || revision == null || r.getValueInteger("revision") == null || r.getValueInteger("revision") >= revision); + previous.sort(Comparator.comparing((QRecord r) -> r.getValueInteger("revision")).reversed()); + if(!previous.isEmpty() && !containsPhi(previous.getFirst())) + { + QRecord prior = previous.getFirst(); + QRecord priorResult = new InboundMapper(EdiRuntime.current().transforms()).map(document, guide(prior, "ORIGINAL"), prior, rules(prior, null)).record(); + difference(state.previous, index, record(priorResult), actual); + } + } + return (findings); + } + + /******************************************************************************* + ** Stable record values and recursive associations, without QQQ diagnostics. + *******************************************************************************/ + private static Map record(QRecord record) + { + Map associations = new LinkedHashMap<>(); + if(record.getAssociatedRecords() != null) + { + record.getAssociatedRecords().forEach((name, children) -> associations.put(name, children.stream().map(MapPreviewService::record).toList())); + } + return (object("values", record.getValues(), "associations", associations)); + } + + /******************************************************************************* + ** Compare normalized JSON so temporal values and numeric types match wire output. + *******************************************************************************/ + private static void difference(List> result, int index, Object before, Object after) + { + if(!JSON.valueToTree(before).equals(JSON.valueToTree(after))) + { + result.add(object("documentIndex", index, "before", before, "after", after)); + } + } + + /******************************************************************************* + ** Ordered objects allowing explicit nulls on the wire. + *******************************************************************************/ + private static Map object(Object... pairs) + { + Map result = new LinkedHashMap<>(); + for(int i = 0; i < pairs.length; i += 2) + { + result.put((String) pairs[i], pairs[i + 1]); + } + return (result); + } + + /******************************************************************************* + ** One invocation's accumulation; never shared across requests. + *******************************************************************************/ + private static class State + { + private int interchanges; + private int groups; + private int documents; + private final List> findings = new ArrayList<>(); + private final List> records = new ArrayList<>(); + private final List> previous = new ArrayList<>(); + private final List> expected = new ArrayList<>(); + private String ackType = "NONE"; + private final StringBuilder ackText = new StringBuilder(); + + /******************************************************************************* + ** Envelope diagnostics carry no raw envelope identifiers. + *******************************************************************************/ + private void envelope(List values) + { + values.forEach(f -> findings.add(object("snipLevel", 1, "severity", "ERROR", "segmentPosition", f.segmentPosition(), + "segmentId", null, "loopId", null, "elementPosition", null, "code", f.code(), "message", "Envelope validation finding"))); + } + + /******************************************************************************* + ** Aggregate acknowledgements in wire order. + *******************************************************************************/ + private void ack(String type, String text) + { + ackType = "NONE".equals(ackType) || type.equals(ackType) ? type : "MIXED"; + if(text != null) + { + ackText.append(text); + } + } + } +} diff --git a/src/main/java/com/kingsrook/qbits/edi/map/MapRuleValidator.java b/src/main/java/com/kingsrook/qbits/edi/map/MapRuleValidator.java new file mode 100644 index 0000000..c9332f3 --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/edi/map/MapRuleValidator.java @@ -0,0 +1,204 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.edi.map; + +import java.util.ArrayList; +import java.util.List; +import java.util.Locale; +import java.util.Map; +import java.util.Objects; +import java.util.regex.Pattern; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.kingsrook.qbits.edi.guide.Guide; +import com.kingsrook.qbits.edi.guide.GuideSegment; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData; + +/******************************************************************************* + ** Static validation of unsaved rules; never runs transforms or writes records. + *******************************************************************************/ +public class MapRuleValidator +{ + private static final Pattern ELEMENT = Pattern.compile("([A-Z][A-Z0-9]{1,2}?)([0-9]{2})(?:-([0-9]+))?"); + private static final Pattern STEP = Pattern.compile("([^\\[]+)(?:\\[(\\*|[1-9][0-9]*|[^\\[\\]]+=[^\\[\\]]+)])?"); + + /******************************************************************************* + ** Report independent errors without echoing rule literals or default values. + *******************************************************************************/ + public List validate(QRecord rule, QRecord map, Guide guide, TransformRegistry registry) + { + List errors = new ArrayList<>(); + if(rule.getValueInteger("sequence") == null || rule.getValueInteger("sequence") < 1) + { + errors.add("sequence must be a positive integer"); + } + boolean outbound = "OUTBOUND".equals(map.getValueString("direction")); + String ediPath = rule.getValueString("ediPath"); + String transform = Objects.requireNonNullElse(rule.getValueString("transform"), "").toUpperCase(Locale.ROOT); + if(ediPath == null || ediPath.isBlank()) + { + if(outbound || !transform.equals("CONSTANT") && rule.getValueString("defaultValue") == null) + { + errors.add("ediPath is required"); + } + } + else if(!resolves(ediPath, guide)) + { + errors.add("ediPath does not resolve in the guide"); + } + String target = rule.getValueString("qqqPath"); + QTableMetaData table = QContext.getQInstance().getTable(map.getValueString("tableName")); + String[] fields = target == null ? new String[0] : target.split("\\.", -1); + for(int i = 0; table != null && i < fields.length - 1; i++) + { + var associations = table.getAssociations(); + String name = fields[i]; + var association = associations == null ? null : associations.stream().filter(a -> a.getName().equals(name)).findFirst().orElse(null); + table = association == null ? null : QContext.getQInstance().getTable(association.getAssociatedTableName()); + } + boolean generated = outbound && (target == null || target.isBlank()) && List.of("CONSTANT", "SEQUENCE", "HL").contains(transform); + if(!generated && (table == null || fields.length == 0 || !table.getFields().containsKey(fields[fields.length - 1]) + || !outbound && fields.length > 2)) + { + errors.add("qqqPath does not resolve in the target table"); + } + if(fields.length > 1 && !"OUTBOUND".equals(map.getValueString("direction")) && (ediPath == null || !ediPath.contains("[*]/"))) + { + errors.add("An inbound association requires a wildcard loop"); + } + Map schema = registry.argumentSchemas().get(transform); + if(!transform.isEmpty() && schema == null) + { + errors.add("Unknown transform"); + } + try + { + String raw = rule.getValueString("transformArgs"); + var args = new ObjectMapper().readTree(raw == null || raw.isBlank() ? "{}" : raw); + if(!args.isObject() || java.util.stream.StreamSupport.stream(java.util.Spliterators.spliteratorUnknownSize(args.elements(), 0), false).anyMatch(v -> !v.isTextual())) + { + errors.add("transformArgs must be a JSON object of strings"); + } + else if(schema != null && schema.get("required") instanceof List required) + { + for(Object key : required) + { + if(!args.hasNonNull(key.toString())) + { + errors.add("Missing required transform argument: " + key); + } + } + } + } + catch(Exception e) + { + errors.add("transformArgs must be a JSON object of strings"); + } + return (errors); + } + + /******************************************************************************* + ** Resolve every joined element path, including selectors and composite positions. + *******************************************************************************/ + private boolean resolves(String path, Guide guide) + { + for(String part : path.split("\\+(?![^\\[]*])", -1)) + { + List scope = guide.segments(); + String[] steps = part.trim().split("/", -1); + for(int index = 0; index < steps.length; index++) + { + var step = STEP.matcher(steps[index]); + if(!step.matches()) + { + return (false); + } + String name = step.group(1); + String qualifier = step.group(2); + var element = ELEMENT.matcher(name); + if(index == steps.length - 1) + { + if(!element.matches() || !element(scope, name)) + { + return (false); + } + scope = scope.stream().filter(s -> s.segmentId().equals(element.group(1))).toList(); + } + else + { + List current = scope; + scope = current.stream().filter(s -> s.segmentId().equals(name) || inLoop(s, name, guide, 0)).toList(); + if(scope.isEmpty()) + { + return (false); + } + } + if(qualifier != null && qualifier.contains("=") && !element(scope, qualifier.substring(0, qualifier.indexOf('=')))) + { + return (false); + } + } + } + return (true); + } + + /******************************************************************************* + ** Ancestor scope matters when different loops contain the same segment. + *******************************************************************************/ + private boolean inLoop(GuideSegment segment, String loop, Guide guide, int depth) + { + if(loop.equals(segment.loopId())) + { + return (true); + } + if(segment.parentLoopId() == null || depth > guide.segments().size()) + { + return (false); + } + return (loop.equals(segment.parentLoopId()) || guide.segments().stream().filter(s -> segment.parentLoopId().equals(s.loopId())) + .anyMatch(parent -> inLoop(parent, loop, guide, depth + 1))); + } + + /******************************************************************************* + ** Match an element and its optional composite component. + *******************************************************************************/ + private boolean element(List segments, String path) + { + var match = ELEMENT.matcher(path); + if(!match.matches()) + { + return (false); + } + int position = Integer.parseInt(match.group(2)); + Integer component; + try + { + component = match.group(3) == null ? null : Integer.valueOf(match.group(3)); + } + catch(NumberFormatException e) + { + return (false); + } + return (segments.stream().filter(s -> s.segmentId().equals(match.group(1))).flatMap(s -> s.elements().stream()) + .anyMatch(e -> e.position() == position && (component == null || component.equals(e.componentPosition())))); + } +} diff --git a/src/main/java/com/kingsrook/qbits/edi/map/TransformRegistry.java b/src/main/java/com/kingsrook/qbits/edi/map/TransformRegistry.java index b568668..c5eb536 100644 --- a/src/main/java/com/kingsrook/qbits/edi/map/TransformRegistry.java +++ b/src/main/java/com/kingsrook/qbits/edi/map/TransformRegistry.java @@ -51,6 +51,7 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput; public class TransformRegistry { private final Map transforms = new ConcurrentHashMap<>(); + private final Map> argumentSchemas = new ConcurrentHashMap<>(); /******************************************************************************* ** Transform a selected path's values in the service session. @@ -118,11 +119,59 @@ public class TransformRegistry ** Register or replace a named host transform. *******************************************************************************/ public TransformRegistry register(String name, Transform transform) + { + return (register(name, transform, schema(name))); + } + + /******************************************************************************* + ** Host transforms publish their argument contract alongside executable behavior. + *******************************************************************************/ + public TransformRegistry register(String name, Transform transform, Map argumentSchema) { transforms.put(name.toUpperCase(Locale.ROOT), transform); + argumentSchemas.put(name.toUpperCase(Locale.ROOT), Map.copyOf(argumentSchema)); return (this); } + /******************************************************************************* + ** Sorted snapshot for mapper clients and static validation. + *******************************************************************************/ + public Map> argumentSchemas() + { + return (java.util.Collections.unmodifiableMap(new java.util.TreeMap<>(argumentSchemas))); + } + + /******************************************************************************* + ** Built-ins accept strings, matching the persisted transformArgs representation. + *******************************************************************************/ + private static Map schema(String name) + { + List keys = switch(name.toUpperCase(Locale.ROOT)) + { + case "CONSTANT" -> List.of("value"); + case "LOOKUP" -> List.of("table", "key", "result"); + case "DATETIME" -> List.of("time", "timezone"); + case "DATEPART", "TIMEPART" -> List.of("timezone"); + case "DECIMAL" -> List.of("scale"); + case "CONCAT" -> List.of("separator"); + case "SPLIT" -> List.of("separator", "index"); + case "SUBSTRING" -> List.of("start", "end"); + case "UNIT" -> List.of("factor", "divisor", "scale"); + case "SEQUENCE" -> List.of("start"); + case "HL" -> List.of("code"); + default -> List.of(); + }; + Map properties = new java.util.TreeMap<>(); + keys.forEach(key -> properties.put(key, Map.of("type", "string"))); + List required = switch(name.toUpperCase(Locale.ROOT)) + { + case "CONSTANT" -> List.of("value"); + case "LOOKUP" -> List.of("table", "key"); + default -> List.of(); + }; + return (Map.of("type", "object", "properties", properties, "required", required, "additionalProperties", Map.of("type", "string"))); + } + /******************************************************************************* ** Execute a closed registry entry; unknown names fail explicitly. *******************************************************************************/ diff --git a/src/main/java/com/kingsrook/qbits/edi/processes/EdiProcesses.java b/src/main/java/com/kingsrook/qbits/edi/processes/EdiProcesses.java index 1a69597..1a12176 100644 --- a/src/main/java/com/kingsrook/qbits/edi/processes/EdiProcesses.java +++ b/src/main/java/com/kingsrook/qbits/edi/processes/EdiProcesses.java @@ -137,6 +137,8 @@ public final class EdiProcesses .withName(table + "_" + operation).withLabel(label).withTableName(table).withIsHidden(true).withMinInputRecords(1).withPermissionRules(permission(permission)); if(operation.equals("preview")) { + process.getPermissionRules().withCustomPermissionChecker( + com.kingsrook.qqq.backend.core.actions.permissions.UseOtherPermissionNameCustomPermissionChecker.build("edi.map.preview")); if(table.equals("edi_map")) { process.getFrontendStep("review").withFormField(new QFieldMetaData("sampleId", QFieldType.INTEGER) diff --git a/src/main/java/com/kingsrook/qbits/edi/storage/EdiRecords.java b/src/main/java/com/kingsrook/qbits/edi/storage/EdiRecords.java index eea0caf..418bfb8 100644 --- a/src/main/java/com/kingsrook/qbits/edi/storage/EdiRecords.java +++ b/src/main/java/com/kingsrook/qbits/edi/storage/EdiRecords.java @@ -60,6 +60,17 @@ public final class EdiRecords return (new QueryAction().execute(new QueryInput(table).withFilter(filter).withShouldFetchHeavyFields(true).withTransaction(com.kingsrook.qqq.backend.core.context.QContext.getQBackendTransaction())).getRecords()); } + /******************************************************************************* + ** UI-facing reads enforce permissions even when the host disables default enforcement. + *******************************************************************************/ + public static List queryEnforced(String table, QQueryFilter filter) throws QException + { + QueryInput input = new QueryInput(table).withFilter(filter).withShouldFetchHeavyFields(true) + .withTransaction(com.kingsrook.qqq.backend.core.context.QContext.getQBackendTransaction()); + input.setPermissionEnforcement(com.kingsrook.qqq.backend.core.actions.permissions.PermissionEnforcement.ENFORCE); + return (new QueryAction().execute(input).getRecords()); + } + /******************************************************************************* ** Match one field by exact value. *******************************************************************************/ diff --git a/src/main/resources/openapi/qqq-v1/edi.yaml b/src/main/resources/openapi/qqq-v1/edi.yaml new file mode 100644 index 0000000..0c8cc1c --- /dev/null +++ b/src/main/resources/openapi/qqq-v1/edi.yaml @@ -0,0 +1,626 @@ +components: + schemas: + EdiAckV1: + description: "Concatenated acknowledgement previews; type is 997, 999, TA1,\ + \ CONTRL, NONE or MIXED. text is null for PHI." + properties: + text: + nullable: true + type: "string" + type: + nullable: true + type: "string" + type: "object" + EdiAssociationV1: + description: "Direct QQQ association and its target fields." + properties: + fields: + items: + $ref: "#/components/schemas/EdiFieldV1" + type: "array" + label: + nullable: true + type: "string" + name: + nullable: true + type: "string" + table: + nullable: true + type: "string" + type: "object" + EdiDiffV1: + description: "Only differing records are returned. Both lists are empty for\ + \ PHI." + properties: + againstExpected: + items: + $ref: "#/components/schemas/EdiDifferenceV1" + type: "array" + againstPreviousRevision: + items: + $ref: "#/components/schemas/EdiDifferenceV1" + type: "array" + type: "object" + EdiDifferenceV1: + description: "One document comparison; documentIndex is zero based." + properties: + after: + additionalProperties: true + nullable: true + type: "object" + before: + description: "Expected JSON or a previous mapped record." + nullable: true + documentIndex: + nullable: true + type: "integer" + type: "object" + EdiElementV1: + description: "Element or composite component; codes contain at most 200 active\ + \ codes in lexical order." + properties: + codeListId: + nullable: true + type: "integer" + codes: + items: + nullable: true + type: "string" + type: "array" + component: + nullable: true + type: "integer" + id: + nullable: true + type: "string" + maxLength: + nullable: true + type: "integer" + minLength: + nullable: true + type: "integer" + name: + nullable: true + type: "string" + position: + nullable: true + type: "integer" + type: + nullable: true + type: "string" + usage: + nullable: true + type: "string" + type: "object" + EdiErrorV1: + description: "Core middleware error body." + properties: + error: + nullable: true + type: "string" + type: "object" + EdiFieldV1: + description: "QQQ target field metadata." + properties: + label: + nullable: true + type: "string" + name: + nullable: true + type: "string" + possibleValueSourceName: + nullable: true + type: "string" + required: + type: "boolean" + type: + nullable: true + type: "string" + type: "object" + EdiFindingV1: + description: "Value-free diagnostic. PHI messages are generic; only positions\ + \ and codes are retained." + properties: + code: + nullable: true + type: "string" + elementPosition: + nullable: true + type: "integer" + loopId: + nullable: true + type: "string" + message: + nullable: true + type: "string" + segmentId: + nullable: true + type: "string" + segmentPosition: + nullable: true + type: "integer" + severity: + nullable: true + type: "string" + snipLevel: + nullable: true + type: "integer" + type: "object" + EdiGuideTreeV1: + description: "Ordered guide tree. Root segments are outside loops; nested loops\ + \ retain parent scope." + properties: + loops: + items: + $ref: "#/components/schemas/EdiLoopV1" + type: "array" + segments: + items: + $ref: "#/components/schemas/EdiSegmentV1" + type: "array" + type: "object" + EdiLoopV1: + description: "Guide loop, nested within its parent; repeat is the maximum occurrence\ + \ count." + properties: + id: + nullable: true + type: "string" + loops: + items: + $ref: "#/components/schemas/EdiLoopV1" + type: "array" + repeat: + nullable: true + type: "integer" + segments: + items: + $ref: "#/components/schemas/EdiSegmentV1" + type: "array" + type: "object" + EdiParseV1: + description: "Counts of wire interchanges, groups and documents." + properties: + documents: + nullable: true + type: "integer" + groups: + nullable: true + type: "integer" + interchanges: + nullable: true + type: "integer" + type: "object" + EdiPreviewRequestV1: + description: "mapId is required. Supply exactly one of sampleId or sampleBase64.\ + \ Uploads are at most 5 MiB decoded, stored encrypted. ruleOverrides replaces\ + \ the whole rule set in memory. CURRENT (default) uses the newest active guide\ + \ in the pinned guide family; ORIGINAL uses the pinned guide." + properties: + guideRevision: + enum: + - "CURRENT" + - "ORIGINAL" + nullable: true + type: "string" + mapId: + nullable: true + type: "integer" + ruleOverrides: + items: + $ref: "#/components/schemas/EdiRuleV1" + nullable: true + type: "array" + sampleBase64: + format: "byte" + maxLength: 6990508 + nullable: true + type: "string" + sampleId: + nullable: true + type: "integer" + sampleName: + maxLength: 255 + nullable: true + type: "string" + type: "object" + EdiPreviewV1: + description: "Parse, validate, acknowledge and map without target writes. PHI\ + \ suppresses result, ack text, and both diffs." + properties: + ack: + $ref: "#/components/schemas/EdiAckV1" + diff: + $ref: "#/components/schemas/EdiDiffV1" + findings: + items: + $ref: "#/components/schemas/EdiFindingV1" + type: "array" + parse: + $ref: "#/components/schemas/EdiParseV1" + phiSuppressed: + type: "boolean" + result: + $ref: "#/components/schemas/EdiResultV1" + nullable: true + type: "object" + EdiRecordV1: + description: "Field values and recursively associated records. Values use the\ + \ target table field types." + properties: + associations: + additionalProperties: + items: + $ref: "#/components/schemas/EdiRecordV1" + type: "array" + type: "object" + values: + additionalProperties: true + type: "object" + type: "object" + EdiResultV1: + description: "Mapped records in document order; no target writes." + properties: + records: + items: + $ref: "#/components/schemas/EdiRecordV1" + type: "array" + type: "object" + EdiRuleErrorsV1: + description: "Correlate errors using zero-based index, optional id and sequence." + properties: + errors: + items: + nullable: true + type: "string" + type: "array" + id: + nullable: true + type: "integer" + index: + nullable: true + type: "integer" + sequence: + nullable: true + type: "integer" + type: "object" + EdiRuleV1: + description: "Map rule. id is optional for unsaved rules. transformArgs is a\ + \ JSON object encoded as a string, as in core table CRUD." + properties: + condition: + nullable: true + type: "string" + defaultValue: + nullable: true + type: "string" + ediPath: + nullable: true + type: "string" + id: + nullable: true + type: "integer" + qqqPath: + nullable: true + type: "string" + required: + nullable: true + type: "boolean" + sequence: + nullable: true + type: "integer" + transform: + nullable: true + type: "string" + transformArgs: + nullable: true + type: "string" + type: "object" + EdiSegmentV1: + description: "Segment definition in guide sequence order." + properties: + elements: + items: + $ref: "#/components/schemas/EdiElementV1" + type: "array" + id: + nullable: true + type: "string" + max: + nullable: true + type: "integer" + min: + nullable: true + type: "integer" + name: + nullable: true + type: "string" + position: + nullable: true + type: "string" + usage: + nullable: true + type: "string" + type: "object" + EdiTargetV1: + description: "Target metadata visible to the caller, current rules and registered\ + \ transforms." + properties: + associations: + items: + $ref: "#/components/schemas/EdiAssociationV1" + type: "array" + fields: + items: + $ref: "#/components/schemas/EdiFieldV1" + type: "array" + rules: + items: + $ref: "#/components/schemas/EdiRuleV1" + type: "array" + table: + nullable: true + type: "string" + transforms: + items: + $ref: "#/components/schemas/EdiTransformV1" + type: "array" + type: "object" + EdiTransformV1: + description: "Registered transform and JSON Schema describing its string-valued\ + \ arguments." + properties: + argumentSchema: + additionalProperties: true + description: "JSON Schema: type, properties, required and additionalProperties.\ + \ All transform arguments are strings." + type: "object" + name: + nullable: true + type: "string" + type: "object" + EdiValidateRequestV1: + description: "rules is required and may be empty; rules are never persisted." + properties: + rules: + items: + $ref: "#/components/schemas/EdiRuleV1" + type: "array" + type: "object" + EdiValidationV1: + description: "One result per submitted rule, including rules with no errors." + properties: + rules: + items: + $ref: "#/components/schemas/EdiRuleErrorsV1" + type: "array" + type: "object" + securitySchemes: + sessionUuidCookie: + in: "cookie" + name: "sessionUUID" + type: "apiKey" +info: + contact: + email: "contact@kingsrook.com" + description: "EDI sandbox and mapper, using the signed-in user's permissions." + title: "QQQ Middleware API extension: edi" + version: "1.0.0" +openapi: "3.0.3" +paths: + /qqq/v1/qbit/edi/guides/{guideId}/tree: + get: + description: "Read the guide tree. Uses the caller session and table read permissions.\ + \ Preview also requires edi.map.preview. Rule writes use core table CRUD." + parameters: + - description: "Positive record identifier." + in: "path" + name: "guideId" + required: true + schema: + type: "integer" + responses: + 400: + content: + application/json: + schema: + $ref: "#/components/schemas/EdiErrorV1" + description: "Invalid request or sample exceeds 5 MiB" + 401: + content: + application/json: + schema: + $ref: "#/components/schemas/EdiErrorV1" + description: "Sign in required" + 403: + content: + application/json: + schema: + $ref: "#/components/schemas/EdiErrorV1" + description: "Permission denied" + 404: + content: + application/json: + schema: + $ref: "#/components/schemas/EdiErrorV1" + description: "Missing or invisible record" + 200: + content: + application/json: + schema: + $ref: "#/components/schemas/EdiGuideTreeV1" + description: "Successful Response" + 413: + content: + application/json: + schema: + $ref: "#/components/schemas/EdiErrorV1" + description: "Host HTTP request size limit exceeded" + security: + - sessionUuidCookie: + - "N/A" + summary: "Read the guide tree" + /qqq/v1/qbit/edi/maps/{mapId}/target: + get: + description: "Read target metadata and map rules. Uses the caller session and\ + \ table read permissions. Preview also requires edi.map.preview. Rule writes\ + \ use core table CRUD." + parameters: + - description: "Positive record identifier." + in: "path" + name: "mapId" + required: true + schema: + type: "integer" + responses: + 400: + content: + application/json: + schema: + $ref: "#/components/schemas/EdiErrorV1" + description: "Invalid request or sample exceeds 5 MiB" + 401: + content: + application/json: + schema: + $ref: "#/components/schemas/EdiErrorV1" + description: "Sign in required" + 403: + content: + application/json: + schema: + $ref: "#/components/schemas/EdiErrorV1" + description: "Permission denied" + 404: + content: + application/json: + schema: + $ref: "#/components/schemas/EdiErrorV1" + description: "Missing or invisible record" + 200: + content: + application/json: + schema: + $ref: "#/components/schemas/EdiTargetV1" + description: "Successful Response" + 413: + content: + application/json: + schema: + $ref: "#/components/schemas/EdiErrorV1" + description: "Host HTTP request size limit exceeded" + security: + - sessionUuidCookie: + - "N/A" + summary: "Read target metadata and map rules" + /qqq/v1/qbit/edi/sandbox/preview: + post: + description: "Preview an encrypted sample with optional unsaved rules. Uses\ + \ the caller session and table read permissions. Preview also requires edi.map.preview.\ + \ Rule writes use core table CRUD." + requestBody: + content: + application/json: + schema: + $ref: "#/components/schemas/EdiPreviewRequestV1" + required: true + responses: + 400: + content: + application/json: + schema: + $ref: "#/components/schemas/EdiErrorV1" + description: "Invalid request or sample exceeds 5 MiB" + 401: + content: + application/json: + schema: + $ref: "#/components/schemas/EdiErrorV1" + description: "Sign in required" + 403: + content: + application/json: + schema: + $ref: "#/components/schemas/EdiErrorV1" + description: "Permission denied" + 404: + content: + application/json: + schema: + $ref: "#/components/schemas/EdiErrorV1" + description: "Missing or invisible record" + 200: + content: + application/json: + schema: + $ref: "#/components/schemas/EdiPreviewV1" + description: "Successful Response" + 413: + content: + application/json: + schema: + $ref: "#/components/schemas/EdiErrorV1" + description: "Host HTTP request size limit exceeded" + security: + - sessionUuidCookie: + - "N/A" + summary: "Preview an encrypted sample with optional unsaved rules" + /qqq/v1/qbit/edi/maps/{mapId}/validate: + post: + description: "Validate unsaved map rules. Uses the caller session and table\ + \ read permissions. Preview also requires edi.map.preview. Rule writes use\ + \ core table CRUD." + parameters: + - description: "Positive record identifier." + in: "path" + name: "mapId" + required: true + schema: + type: "integer" + requestBody: + content: + application/json: + schema: + $ref: "#/components/schemas/EdiValidateRequestV1" + required: true + responses: + 400: + content: + application/json: + schema: + $ref: "#/components/schemas/EdiErrorV1" + description: "Invalid request or sample exceeds 5 MiB" + 401: + content: + application/json: + schema: + $ref: "#/components/schemas/EdiErrorV1" + description: "Sign in required" + 403: + content: + application/json: + schema: + $ref: "#/components/schemas/EdiErrorV1" + description: "Permission denied" + 404: + content: + application/json: + schema: + $ref: "#/components/schemas/EdiErrorV1" + description: "Missing or invisible record" + 200: + content: + application/json: + schema: + $ref: "#/components/schemas/EdiValidationV1" + description: "Successful Response" + 413: + content: + application/json: + schema: + $ref: "#/components/schemas/EdiErrorV1" + description: "Host HTTP request size limit exceeded" + security: + - sessionUuidCookie: + - "N/A" + summary: "Validate unsaved map rules" diff --git a/src/test/java/com/kingsrook/qbits/edi/EdiApiContractTest.java b/src/test/java/com/kingsrook/qbits/edi/EdiApiContractTest.java new file mode 100644 index 0000000..27b339e --- /dev/null +++ b/src/test/java/com/kingsrook/qbits/edi/EdiApiContractTest.java @@ -0,0 +1,79 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.edi; + +import java.nio.file.Files; +import java.nio.file.Path; +import com.kingsrook.qbits.edi.api.EdiApi; +import com.kingsrook.qqq.middleware.javalin.specs.extensions.MiddlewareSpecExtensionsMetaData; +import com.kingsrook.qqq.middleware.javalin.specs.v1.MiddlewareVersionV1; +import com.kingsrook.qqq.middleware.javalin.tools.UiApiContract; +import org.junit.jupiter.api.Test; +import static org.assertj.core.api.Assertions.assertThat; + +/******************************************************************************* + ** Generated extension fragment is the exact versioned contract shipped to clients. + *******************************************************************************/ +class EdiApiContractTest +{ + /******************************************************************************* + ** Use RC.6's real helper, including compatibility comparison against this first baseline. + *******************************************************************************/ + @Test + void contract() throws Exception + { + var version = new MiddlewareVersionV1(); + var extension = EdiApi.extension(); + String generated = UiApiContract.generateExtensionYaml(version, extension); + Path path = Path.of("src/main/resources/openapi/qqq-v1/edi.yaml").toAbsolutePath(); + if(Boolean.getBoolean("edi.openapi.update")) + { + Files.createDirectories(path.getParent()); + Files.writeString(path, generated); + } + String committed = Files.readString(path); + assertThat(UiApiContract.check(generated, committed, committed, 1, + "mvnw.cmd -B -Dtest=EdiApiContractTest -Dedi.openapi.update=true test")).isEmpty(); + var model = version.generateExtensionOpenAPIModel("qqq", extension); + assertThat(model.getComponents().getSchemas().keySet()).allMatch(n -> n.startsWith("Edi") && n.endsWith("V1")); + assertThat(model.getPaths()).hasSize(4); + assertThat(model.getPaths().keySet()).allMatch(n -> n.startsWith("/qqq/v1/qbit/edi/")); + assertThat(extension.getEffectivePathPrefix()).isEqualTo("/qbit/edi"); + assertThat(extension.getSemanticVersion()).isEqualTo("1.0.0"); + } + + /******************************************************************************* + ** Producer registers the extension and all metadata validates with RC.6. + *******************************************************************************/ + @Test + void registration() throws Exception + { + try + { + var instance = H2Support.create(); + assertThat(MiddlewareSpecExtensionsMetaData.ofOrWithNew(instance).getExtensions()).hasSize(1); + } + finally + { + com.kingsrook.qqq.backend.core.context.QContext.clear(); + } + } +} diff --git a/src/test/java/com/kingsrook/qbits/edi/EdiApiHttpTest.java b/src/test/java/com/kingsrook/qbits/edi/EdiApiHttpTest.java new file mode 100644 index 0000000..412466c --- /dev/null +++ b/src/test/java/com/kingsrook/qbits/edi/EdiApiHttpTest.java @@ -0,0 +1,564 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.edi; + +import java.net.ServerSocket; +import java.net.URI; +import java.net.http.HttpClient; +import java.net.http.HttpRequest; +import java.net.http.HttpResponse; +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Base64; +import java.util.List; +import java.util.Map; +import java.util.UUID; +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.kingsrook.qbits.edi.api.EdiApiService; +import com.kingsrook.qbits.edi.storage.EdiRecords; +import com.kingsrook.qbits.edi.storage.PayloadKind; +import com.kingsrook.qbits.edi.storage.PayloadRef; +import com.kingsrook.qbits.edi.support.SystemWrites; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.instances.AbstractQQQApplication; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType; +import com.kingsrook.qqq.backend.core.model.metadata.QInstance; +import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference; +import com.kingsrook.qqq.backend.core.model.session.QSession; +import com.kingsrook.qqq.backend.core.model.session.QSystemUserSession; +import com.kingsrook.qqq.backend.core.model.session.QUser; +import com.kingsrook.qqq.backend.core.modules.authentication.QAuthenticationModuleCustomizerInterface; +import com.kingsrook.qqq.middleware.javalin.QApplicationJavalinServer; +import com.kingsrook.qqq.middleware.javalin.QJavalinImplementation; +import org.junit.jupiter.api.AfterEach; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import static org.assertj.core.api.Assertions.assertThat; + +/******************************************************************************* + ** Real HTTP against strict-mode QQQ, real SQL, real encrypted samples and caller sessions. + *******************************************************************************/ +class EdiApiHttpTest +{ + private static final ObjectMapper JSON = new ObjectMapper(); + private QApplicationJavalinServer server; + private HttpClient client; + private QInstance instance; + private QRecord map; + private QRecord sample; + private String baseUrl; + private String suffix; + + /******************************************************************************* + ** The existing synthetic vertical-slice fixture supplies tables, guide and map. + *******************************************************************************/ + @BeforeEach + void start() throws Exception + { + new InboundTest().setup("004010"); + instance = QContext.getQInstance(); + map = EdiRecords.query("edi_map", new QQueryFilter()).getFirst(); + SystemWrites.call(() -> + { + SystemWrites.update("edi_map", new QRecord().withValue("id", map.getValue("id")).withValue("partnerId", 1)); + SystemWrites.update("edi_guide", new QRecord().withValue("id", map.getValue("guideId")).withValue("partnerId", 1)); + return (null); + }); + map.withValue("partnerId", 1); + sample = saveSample(Synthetic850.generate(17, "004010", com.kingsrook.qbits.edi.x12.X12Separators.DEFAULT), map.getValueInteger("id")); + instance.withInstanceDefaultAuthentication(new QAuthenticationMetaData().withName("mock").withType(QAuthenticationType.MOCK) + .withCustomizer(new QCodeReference(Sessions.class))); + int port; + try(ServerSocket socket = new ServerSocket(0)) + { + port = socket.getLocalPort(); + } + baseUrl = "http://127.0.0.1:" + port + "/qqq/v1/qbit/edi"; + suffix = "-" + UUID.randomUUID(); + client = HttpClient.newHttpClient(); + server = new QApplicationJavalinServer(new AbstractQQQApplication() + { + @Override + public QInstance defineQInstance() + { + return (instance); + } + }).withPort(port).withServeFrontendNext(false).withServeFrontendMaterialDashboard(false) + .withServeLegacyUnversionedMiddlewareAPI(false).withStrictUiApiRoutes(true) + .withJavalinConfigCustomizer(config -> config.http.maxRequestSize = 8 * 1024 * 1024); + server.start(); + QContext.init(instance, new QSystemUserSession(instance)); + } + + /******************************************************************************* + ** Release server globals and thread context. + *******************************************************************************/ + @AfterEach + void stop() + { + if(server != null) + { + server.stop(); + } + if(client != null) + { + client.close(); + } + QJavalinImplementation.setSessionCookieHttpOnly(false); + QContext.clear(); + } + + /******************************************************************************* + ** Ordinary users receive selected reads; none receives raw-payload access. + *******************************************************************************/ + public static class Sessions implements QAuthenticationModuleCustomizerInterface + { + @Override + public void customizeSession(QInstance instance, QSession session, Map context) + { + String role = session.getUuid().split("-")[0]; + session.withUser(new QUser().withIdReference(role)); + session.withSecurityKeyValue("edi_partner", 1); + if(!role.equals("denied")) + { + for(String table : instance.getTables().keySet()) + { + if(!table.equals("edi_payload") && !table.equals("edi_preview_download") + && !(role.equals("noguide") && table.equals("edi_guide")) + && !(role.equals("norules") && table.equals("edi_map_rule")) + && !(role.equals("notarget") && table.equals("sample_order"))) + { + String permission = table.startsWith("edi_") ? "edi." + table.substring(4) + ".read" + : table.equals("sample_order") ? "sample.order.read" : "sample.line.read"; + session.withPermissions(permission); + } + } + if(!role.equals("reader")) + { + session.withPermissions("edi.map.preview"); + } + } + } + } + + /******************************************************************************* + ** Transport helper retains the exact response for egress scans. + *******************************************************************************/ + private HttpResponse request(String method, String path, Object body, String role) throws Exception + { + String text = body == null ? "" : body instanceof String s ? s : JSON.writeValueAsString(body); + return (client.send(HttpRequest.newBuilder(URI.create(baseUrl + path)).header("Cookie", "sessionUUID=" + role + suffix) + .header("Content-Type", "application/json").method(method, HttpRequest.BodyPublishers.ofString(text)).build(), HttpResponse.BodyHandlers.ofString())); + } + + /******************************************************************************* + ** Assert success before parsing the wire JSON. + *******************************************************************************/ + private JsonNode ok(String method, String path, Object body) throws Exception + { + var response = request(method, path, body, "user"); + assertThat(response.statusCode()).as(response.body()).isEqualTo(200); + return (JSON.readTree(response.body())); + } + + /******************************************************************************* + ** An authorized fixture writer archives synthetic bytes only. + *******************************************************************************/ + private QRecord saveSample(String text, Integer mapId) throws Exception + { + return (SystemWrites.call(() -> + { + var ref = EdiRuntime.current().payloads().put(PayloadKind.SAMPLE, InboundTest.bytes(text)); + return (EdiRecords.insert("edi_map_sample", new QRecord().withValue("name", "Synthetic HTTP sample").withValue("mapId", mapId).withValue("payloadRef", ref.value()))); + })); + } + + /******************************************************************************* + ** Shared stored-sample request. + *******************************************************************************/ + private Map preview() + { + return (Map.of("mapId", map.getValueInteger("id"), "sampleId", sample.getValueInteger("id"))); + } + + /******************************************************************************* + ** Guide structure and target schema include associations, rules and argument contracts. + *******************************************************************************/ + @Test + void guideAndTarget() throws Exception + { + var tree = ok("GET", "/guides/" + map.getValue("guideId") + "/tree", null); + assertThat(tree.get("loops")).isNotEmpty(); + assertThat(tree.toString()).contains("\"id\":\"BEG\"", "\"position\":\"0030\"", "codeListId", "minLength"); + var target = ok("GET", "/maps/" + map.getValue("id") + "/target", null); + assertThat(target.get("table").asText()).isEqualTo("sample_order"); + assertThat(target.get("fields").toString()).contains("poNumber", "possibleValueSourceName"); + assertThat(target.get("associations").toString()).contains("lines", "sample_order_line", "quantity"); + assertThat(target.get("rules")).hasSize(5); + assertThat(target.get("transforms").toString()).contains("LOOKUP", "argumentSchema", "required", "table", "key"); + } + + /******************************************************************************* + ** Each operation returns real 403 responses for denied table or action permissions. + *******************************************************************************/ + @Test + void permissions() throws Exception + { + String target = "/maps/" + map.getValue("id") + "/target"; + String validate = "/maps/" + map.getValue("id") + "/validate"; + assertThat(request("GET", "/guides/" + map.getValue("guideId") + "/tree", null, "denied").statusCode()).isEqualTo(403); + assertThat(request("GET", target, null, "denied").statusCode()).isEqualTo(403); + assertThat(request("POST", validate, Map.of("rules", List.of()), "denied").statusCode()).isEqualTo(403); + assertThat(request("POST", "/sandbox/preview", preview(), "denied").statusCode()).isEqualTo(403); + assertThat(request("POST", "/sandbox/preview", preview(), "reader").statusCode()).isEqualTo(403); + assertThat(request("GET", target, null, "notarget").statusCode()).isEqualTo(403); + assertThat(request("GET", target, null, "norules").statusCode()).isEqualTo(403); + assertThat(request("GET", "/guides/" + map.getValue("guideId") + "/tree", null, "noguide").statusCode()).isEqualTo(403); + } + + /******************************************************************************* + ** Preview returns the mapped structure and ack while leaving target tables untouched. + *******************************************************************************/ + @Test + void storedPreview() throws Exception + { + var result = ok("POST", "/sandbox/preview", preview()); + assertThat(result.at("/parse/interchanges").asInt()).isEqualTo(1); + assertThat(result.at("/parse/groups").asInt()).isEqualTo(1); + assertThat(result.at("/parse/documents").asInt()).isEqualTo(1); + assertThat(result.at("/result/records/0/values/poNumber").asText()).isNotBlank(); + assertThat(result.at("/result/records/0/associations/lines")).hasSize(1); + assertThat(result.at("/ack/type").asText()).isEqualTo("997"); + assertThat(result.at("/ack/text").asText()).contains("AK9"); + assertThat(result.get("phiSuppressed").asBoolean()).isFalse(); + assertThat(EdiRecords.query("sample_order", new QQueryFilter())).isEmpty(); + } + + /******************************************************************************* + ** Unsaved overrides replace rules in memory and custom transforms see the caller. + *******************************************************************************/ + @Test + void unsavedOverridesAndCallerSession() throws Exception + { + EdiRuntime.current().transforms().register("SESSION", (values, args) -> QContext.getQSession().getUser().getIdReference(), + Map.of("type", "object", "properties", Map.of(), "required", List.of())); + var rule = Map.of("sequence", 1, "ediPath", "BEG03", "qqqPath", "poNumber", "transform", "SESSION"); + var body = new java.util.LinkedHashMap<>(preview()); + body.put("ruleOverrides", List.of(rule)); + var result = ok("POST", "/sandbox/preview", body); + assertThat(result.at("/result/records/0/values/poNumber").asText()).isEqualTo("user"); + assertThat(EdiRecords.query("edi_map_rule", EdiRecords.equal("mapId", map.getValue("id")))).hasSize(5); + assertThat(ok("POST", "/sandbox/preview", preview()).at("/result/records/0/values/poNumber").asText()).isNotEqualTo("user"); + body.put("ruleOverrides", List.of()); + assertThat(ok("POST", "/sandbox/preview", body).at("/result/records/0/values").size()).isZero(); + } + + /******************************************************************************* + ** Base64 uploads become encrypted samples, with no plaintext in the payload table. + *******************************************************************************/ + @Test + void upload() throws Exception + { + String raw = Synthetic850.generate(18, "004010", com.kingsrook.qbits.edi.x12.X12Separators.DEFAULT); + int count = EdiRecords.query("edi_map_sample", new QQueryFilter()).size(); + var result = ok("POST", "/sandbox/preview", Map.of("mapId", map.getValueInteger("id"), "sampleBase64", + Base64.getEncoder().encodeToString(raw.getBytes(StandardCharsets.US_ASCII)), "sampleName", "Uploaded sample")); + assertThat(result.at("/parse/documents").asInt()).isEqualTo(1); + var samples = EdiRecords.query("edi_map_sample", new QQueryFilter()); + assertThat(samples).hasSize(count + 1); + var uploaded = samples.stream().filter(r -> "Uploaded sample".equals(r.getValueString("name"))).findFirst().orElseThrow(); + try(var stream = EdiRuntime.current().payloads().open(new PayloadRef(uploaded.getValueString("payloadRef"), null, 0))) + { + assertThat(new String(stream.readAllBytes(), StandardCharsets.US_ASCII)).isEqualTo(raw); + } + for(var payload : EdiRecords.query("edi_payload", new QQueryFilter())) + { + assertThat(new String((byte[]) payload.getValue("ciphertext"), StandardCharsets.ISO_8859_1)).doesNotContain("SYNTH-1", raw); + } + } + + /******************************************************************************* + ** Reject oversized decoded bytes as well as encoded lengths before persistence. + *******************************************************************************/ + @Test + void uploadLimitAndBadRequests() throws Exception + { + int count = EdiRecords.query("edi_map_sample", new QQueryFilter()).size(); + for(int size : List.of(EdiApiService.MAX_SAMPLE_BYTES + 1, EdiApiService.MAX_SAMPLE_BYTES + 4)) + { + var response = request("POST", "/sandbox/preview", Map.of("mapId", map.getValueInteger("id"), "sampleBase64", + Base64.getEncoder().encodeToString(new byte[size])), "user"); + assertThat(response.statusCode()).as(response.body()).isEqualTo(400); + } + for(Object body : List.of(Map.of("mapId", map.getValueInteger("id"), "sampleBase64", "!invalid!"), + Map.of("mapId", map.getValueInteger("id")), Map.of("mapId", map.getValueInteger("id"), "sampleId", sample.getValueInteger("id"), "sampleBase64", "AAAA"), + Map.of("mapId", map.getValueInteger("id"), "sampleId", sample.getValueInteger("id"), "guideRevision", "BAD"), + "{\"mapId\":1.2}", "{\"mapId\":\"1\"}", "{\"mapId\":1,\"ruleOverrides\":[null]}", "[]", "null", "invalid")) + { + assertThat(request("POST", "/sandbox/preview", body, "user").statusCode()).isEqualTo(400); + } + assertThat(EdiRecords.query("edi_map_sample", new QQueryFilter())).hasSize(count); + assertThat(request("GET", "/maps/invalid/target", null, "user").statusCode()).isEqualTo(400); + assertThat(request("GET", "/guides/999999/tree", null, "user").statusCode()).isEqualTo(404); + } + + /******************************************************************************* + ** PHI is detected through hidden agreement data, and canaries never reach responses. + *******************************************************************************/ + @Test + void phiSuppression() throws Exception + { + String canary = "CANARY_PRIVATE_VALUE"; + SystemWrites.call(() -> + { + var document = EdiRecords.query("edi_agreement_document", EdiRecords.equal("mapId", map.getValue("id"))).getFirst(); + SystemWrites.update("edi_agreement_document", new QRecord().withValue("id", document.getValue("id")).withValue("containsPhi", true)); + return (null); + }); + var body = new java.util.LinkedHashMap<>(preview()); + body.put("ruleOverrides", List.of(Map.of("sequence", 1, "qqqPath", "poNumber", "transform", "CONSTANT", "transformArgs", "{\"value\":\"" + canary + "\"}"))); + var response = request("POST", "/sandbox/preview", body, "user"); + assertThat(response.statusCode()).as(response.body()).isEqualTo(200); + assertThat(response.body()).doesNotContain(canary, "SYNTH-1", "poNumber"); + var json = JSON.readTree(response.body()); + assertThat(json.get("phiSuppressed").asBoolean()).isTrue(); + assertThat(json.get("result").isNull()).isTrue(); + assertThat(json.at("/ack/text").isNull()).isTrue(); + assertThat(json.at("/diff/againstExpected")).isEmpty(); + assertThat(json.at("/diff/againstPreviousRevision")).isEmpty(); + } + + /******************************************************************************* + ** Rule errors are independently correlated; valid paths and transforms succeed. + *******************************************************************************/ + @Test + void validateRules() throws Exception + { + var rules = ok("GET", "/maps/" + map.getValue("id") + "/target", null).get("rules"); + var valid = ok("POST", "/maps/" + map.getValue("id") + "/validate", Map.of("rules", rules)); + assertThat(valid.get("rules")).hasSize(5); + valid.get("rules").forEach(rule -> assertThat(rule.get("errors")).as(rule.toString()).isEmpty()); + var invalid = List.of( + Map.of("sequence", 1, "ediPath", "BAD99", "qqqPath", "unknown", "transform", "NO_SUCH_TRANSFORM"), + Map.of("sequence", 2, "ediPath", "BEG03", "qqqPath", "poNumber", "transform", "LOOKUP", "transformArgs", "{}"), + Map.of("sequence", 3, "ediPath", "PO1[*]/PO199", "qqqPath", "lines.sku", "transformArgs", "[]"), + Map.of("sequence", 4, "ediPath", "BEG03", "qqqPath", "lines.sku", "transformArgs", "{\"index\":1}"), + Map.of("sequence", 0, "qqqPath", "poNumber")); + var errors = ok("POST", "/maps/" + map.getValue("id") + "/validate", Map.of("rules", invalid)); + assertThat(errors.at("/rules/0/errors")).hasSize(3); + assertThat(errors.at("/rules/1/errors").toString()).contains("table", "key"); + assertThat(errors.at("/rules/2/errors").toString()).contains("ediPath", "transformArgs"); + assertThat(errors.at("/rules/3/errors").toString()).contains("wildcard", "transformArgs"); + assertThat(errors.at("/rules/4/errors").toString()).contains("sequence", "ediPath"); + assertThat(request("POST", "/maps/" + map.getValue("id") + "/validate", Map.of(), "user").statusCode()).isEqualTo(400); + } + + /******************************************************************************* + ** Existing samples must belong to the selected map, and partner locks apply first. + *******************************************************************************/ + @Test + void sampleAndPartnerIsolation() throws Exception + { + QRecord another = SystemWrites.call(() -> EdiRecords.insert("edi_map", new QRecord().withValue("name", "Other map").withValue("partnerId", 1) + .withValue("guideId", map.getValue("guideId")).withValue("tableName", "sample_order").withValue("direction", "INBOUND").withValue("transactionSet", "850"))); + assertThat(request("POST", "/sandbox/preview", Map.of("mapId", another.getValueInteger("id"), "sampleId", sample.getValueInteger("id")), "user").statusCode()).isEqualTo(404); + SystemWrites.call(() -> + { + SystemWrites.update("edi_map", new QRecord().withValue("id", map.getValue("id")).withValue("partnerId", 2)); + return (null); + }); + assertThat(request("GET", "/maps/" + map.getValue("id") + "/target", null, "user").statusCode()).isEqualTo(404); + assertThat(request("POST", "/sandbox/preview", preview(), "user").statusCode()).isEqualTo(404); + } + + /******************************************************************************* + ** A runtime transform failure returns a safe diagnostic instead of an exception value. + *******************************************************************************/ + @Test + void mappingFailureIsValueFree() throws Exception + { + EdiRuntime.current().transforms().register("FAIL", (values, args) -> + { + throw new IllegalArgumentException("SECRET_CANARY"); + }); + var body = new java.util.LinkedHashMap<>(preview()); + body.put("ruleOverrides", List.of(Map.of("sequence", 1, "ediPath", "BEG03", "qqqPath", "poNumber", "transform", "FAIL"))); + var response = request("POST", "/sandbox/preview", body, "user"); + assertThat(response.statusCode()).as(response.body()).isEqualTo(200); + assertThat(response.body()).doesNotContain("SECRET_CANARY").contains("\"code\":\"MAP\""); + } + + /******************************************************************************* + ** The exact decoded limit is accepted, including base64 expansion. + *******************************************************************************/ + @Test + void exactUploadLimit() throws Exception + { + String raw = Synthetic850.generate(19, "004010", com.kingsrook.qbits.edi.x12.X12Separators.DEFAULT); + String padded = raw + " ".repeat(EdiApiService.MAX_SAMPLE_BYTES - raw.length()); + var result = ok("POST", "/sandbox/preview", Map.of("mapId", map.getValueInteger("id"), "sampleBase64", + Base64.getEncoder().encodeToString(padded.getBytes(StandardCharsets.US_ASCII)))); + assertThat(result.at("/parse/documents").asInt()).isEqualTo(1); + assertThat(EdiRecords.query("edi_payload", new QQueryFilter())).anyMatch(r -> r.getValueLong("sizeBytes") == EdiApiService.MAX_SAMPLE_BYTES); + } + + /******************************************************************************* + ** Required LOOKUP permissions are not borrowed from the configured service session. + *******************************************************************************/ + @Test + void lookupCannotElevate() throws Exception + { + var body = new java.util.LinkedHashMap<>(preview()); + body.put("ruleOverrides", List.of(Map.of("sequence", 1, "ediPath", "BEG03", "qqqPath", "poNumber", "transform", "LOOKUP", + "transformArgs", "{\"table\":\"sample_order\",\"key\":\"poNumber\"}"))); + var response = request("POST", "/sandbox/preview", body, "notarget"); + assertThat(response.statusCode()).as(response.body()).isEqualTo(403); + assertThat(request("POST", "/sandbox/preview", preview(), "norules").statusCode()).isEqualTo(403); + assertThat(request("POST", "/maps/" + map.getValue("id") + "/validate", Map.of("rules", List.of()), "noguide").statusCode()).isEqualTo(403); + } + + /******************************************************************************* + ** A later active guide changes CURRENT findings while ORIGINAL keeps the pinned guide. + *******************************************************************************/ + @Test + void guideRevisionSelection() throws Exception + { + var repository = new com.kingsrook.qbits.edi.guide.GuideRepository(); + var guide = repository.load(map.getValueInteger("guideId")); + var segments = new ArrayList<>(guide.segments()); + segments.add(segments.size() - 1, new com.kingsrook.qbits.edi.guide.GuideSegment("XYZ", null, null, 1, "REQUIRED", 1, 1, List.of(), List.of(), null)); + var changed = new com.kingsrook.qbits.edi.guide.Guide(guide.name(), guide.standard(), guide.version(), guide.transactionSet(), guide.functionalIdCode(), segments, guide.rules()); + SystemWrites.call(() -> + { + Integer id = repository.save(changed, "synthetic-http-revision", 2); + SystemWrites.update("edi_guide", new QRecord().withValue("id", id).withValue("partnerId", 1)); + return (null); + }); + var body = new java.util.LinkedHashMap<>(preview()); + body.put("guideRevision", "ORIGINAL"); + var original = ok("POST", "/sandbox/preview", body); + body.put("guideRevision", "CURRENT"); + var current = ok("POST", "/sandbox/preview", body); + assertThat(current.get("findings").size()).isGreaterThan(original.get("findings").size()); + assertThat(current.get("findings").toString()).contains("XYZ"); + } + + /******************************************************************************* + ** Previous-revision and expected diffs identify the changed document and values. + *******************************************************************************/ + @Test + void revisionAndExpectedDiffs() throws Exception + { + QRecord revised = SystemWrites.call(() -> + { + QRecord next = new QRecord().withValue("revision", 2).withValue("partnerId", 1).withValue("guideId", map.getValue("guideId")); + for(String field : List.of("name", "direction", "standard", "version", "transactionSet", "tableName", "status")) + { + next.withValue(field, map.getValue(field)); + } + return (EdiRecords.insert("edi_map", next)); + }); + var nextSample = saveSample(Synthetic850.generate(17, "004010", com.kingsrook.qbits.edi.x12.X12Separators.DEFAULT), revised.getValueInteger("id")); + SystemWrites.call(() -> + { + SystemWrites.update("edi_map_sample", new QRecord().withValue("id", nextSample.getValue("id")).withValue("expectedResultJson", "{\"values\":{\"poNumber\":\"EXPECTED\"},\"associations\":{}}")); + return (null); + }); + var result = ok("POST", "/sandbox/preview", Map.of("mapId", revised.getValueInteger("id"), "sampleId", nextSample.getValueInteger("id"), + "ruleOverrides", List.of(Map.of("sequence", 1, "qqqPath", "poNumber", "transform", "CONSTANT", "transformArgs", "{\"value\":\"EDITED\"}")))); + assertThat(result.at("/diff/againstPreviousRevision")).hasSize(1); + assertThat(result.at("/diff/againstExpected")).hasSize(1); + assertThat(result.at("/diff/againstPreviousRevision/0/after/values/poNumber").asText()).isEqualTo("EDITED"); + assertThat(result.at("/diff/againstExpected/0/before/values/poNumber").asText()).isEqualTo("EXPECTED"); + } + + /******************************************************************************* + ** Large code lists are deterministically capped in the guide-tree projection. + *******************************************************************************/ + @Test + void codeListLimit() throws Exception + { + var definitions = EdiRecords.query("edi_guide_segment", EdiRecords.equal("guideId", map.getValue("guideId"))); + Integer codeListId = null; + for(var definition : definitions) + { + for(var element : EdiRecords.query("edi_guide_element", EdiRecords.equal("guideSegmentId", definition.getValue("id")))) + { + if(element.getValueInteger("codeListId") != null) + { + codeListId = element.getValueInteger("codeListId"); + break; + } + } + if(codeListId != null) + { + break; + } + } + final Integer listId = codeListId; + SystemWrites.call(() -> + { + for(int i = 0; i < 205; i++) + { + EdiRecords.insert("edi_code_list_value", new QRecord().withValue("codeListId", listId).withValue("code", "SYNTH" + i).withValue("isActive", true)); + } + return (null); + }); + var tree = ok("GET", "/guides/" + map.getValue("guideId") + "/tree", null); + assertThat(tree.findValues("codes")).allSatisfy(codes -> assertThat(codes.size()).isLessThanOrEqualTo(200)); + assertThat(tree.findValues("codes")).anyMatch(codes -> codes.size() == 200); + } + + /******************************************************************************* + ** Transaction and interchange envelope errors are represented in findings and acknowledgements. + *******************************************************************************/ + @Test + void envelopeFindings() throws Exception + { + String raw = Synthetic850.generate(20, "004010", com.kingsrook.qbits.edi.x12.X12Separators.DEFAULT); + var invalid = saveSample(raw.replace("SE*9*", "SE*99*").replace("IEA*1*", "IEA*2*"), map.getValueInteger("id")); + var response = ok("POST", "/sandbox/preview", Map.of("mapId", map.getValueInteger("id"), "sampleId", invalid.getValueInteger("id"))); + assertThat(response.get("findings")).hasSizeGreaterThanOrEqualTo(2); + assertThat(response.get("findings").toString()).contains("\"snipLevel\":1"); + assertThat(response.at("/ack/type").asText()).isEqualTo("MIXED"); + assertThat(response.at("/ack/text").asText()).contains("TA1*", "ISA*", "AK9*"); + } + + /******************************************************************************* + ** A caller cannot inspect another partner's guide via a visible map. + *******************************************************************************/ + @Test + void hiddenGuide() throws Exception + { + SystemWrites.call(() -> + { + SystemWrites.update("edi_guide", new QRecord().withValue("id", map.getValue("guideId")).withValue("partnerId", 2)); + return (null); + }); + assertThat(request("GET", "/guides/" + map.getValue("guideId") + "/tree", null, "user").statusCode()).isEqualTo(404); + assertThat(request("POST", "/sandbox/preview", preview(), "user").statusCode()).isEqualTo(404); + assertThat(request("POST", "/maps/" + map.getValue("id") + "/validate", Map.of("rules", List.of()), "user").statusCode()).isEqualTo(404); + } +} -- 2.54.0