QRun-IO/qbit-edi
3
0 Fork 0
Code Issues Pull Requests 4 Actions Packages Projects Releases 10 Wiki Activity

feat: phase c - AS2 transport #10

Merged
KofTwentyTwo merged 1 commits from feature/phase-c-as2 into develop 2026-10-14 13:27:25 +00:00
Conversation 0 Commits 1 Files Changed 44
+6259 -61
44 changed files with 6259 additions and 61 deletions
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

AGENTS.md
+1 -1
View File
@@ -53,7 +53,7 @@ The package layout per phase is in `docs/DESIGN.md`.
Java 21; parent `com.kingsrook:qbit-build-parent:2.0.0`. Default-active `qqq-snapshot` imports QQQ
`4.1.0-RC.5` ahead of the parent's 4.0.0 BOM (ADR-0007); move to each new RC within a week (G7).
Middleware Javalin is optional. Planned: StAEDI and as2-lib (both Apache-2.0).
Middleware Javalin is optional. Also StAEDI (X12) and as2-lib with BouncyCastle (AS2), all Apache-2.0/MIT-style licensed.
## Release
CHANGELOG.md
+7
View File
@@ -7,6 +7,13 @@ Following [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and [Semantic
### Added
- Phase d warehouse and logistics baseline guides for 940, 945, 214 and 204 in 004010 and 005010 (syntax notes, situational control-total and stop rules, AT7 pairing), inbound/outbound sample maps (940 in, 945 out/in, 214 in, 204 in/out), DATEPART/TIMEPART outbound transforms and `+`-joined inbound paths for date/time pairs, and Harbor Logistics and Ridgeline Freight demo traffic.
||||||| parent of bfb63ce (feat: phase c - AS2 transport)
- Phase c AS2: `As2Transport` send and receive (RFC 4130) with S/MIME signing (SHA-256/384/512; SHA1 flagged), encryption (AES-128/256 CBC or GCM; 3DES flagged), optional compression, synchronous and asynchronous signed MDNs with MIC verification, Test Connection, optional TLS pinning and VAN transports with no partner. Built on as2-lib 5.1.8 and BouncyCastle, entirely in memory (no keystore or temporary files).
- `POST /qqq/hooks/edi/as2/{transport}` and `/{transport}/mdn` route provider: identity, decryption (ACTIVE or NEXT keys), signature and MIC checks, Message-ID dedupe returning the original MDN, error MDNs with no inbox row, and a queued, retried asynchronous MDN sender.
- Read-only `edi_as2_mdn` table with "AS2 MDNs" child lists on outbound and inbound messages; raw MDNs in the encrypted payload store; `edi_outbound_message.transport_mic`; `edi_transport.weak_crypto`; `EdiQBitConfig.publicBaseUrl`.
- Certificates: parsed on save (private keys refused), keys from environment PEM PKCS#8 or base64 PKCS#12, ACTIVE/NEXT rollover, record actions Promote Certificate and Export Public Certificate, expiry alerts for ACTIVE and NEXT, and the `edi.mdn.overdue` alert.
- Demo: Pacific Wholesale over AS2 against an in-process BouncyCastle peer with in-memory certificates; AS2 acceptance tests in both directions against our own route and an independent peer.
- Phase b retail baseline guides for 855/856/810/846/860 in 004010 and 005010; inbound/outbound sample maps and nested outbound associations with SEQUENCE/HL transforms.
- Transactional RECORD_CHANGE and manual enqueue, host-wired ESB process, batch schedules, validated encrypted envelopes, send workers, retries and dead-letter alerts.
- Instance transport registry, MINA SFTP streaming push/poll with pinned host keys, gateway OUTBOX stand-in/download delivery, and authenticated HTTPS ingress/egress using core Standard Webhooks.
CLAUDE.md
+1 -1
View File
@@ -53,7 +53,7 @@ The package layout per phase is in `docs/DESIGN.md`.
Java 21; parent `com.kingsrook:qbit-build-parent:2.0.0`. Default-active `qqq-snapshot` imports QQQ
`4.1.0-RC.5` ahead of the parent's 4.0.0 BOM (ADR-0007); move to each new RC within a week (G7).
Middleware Javalin is optional. Planned: StAEDI and as2-lib (both Apache-2.0).
Middleware Javalin is optional. Also StAEDI (X12) and as2-lib with BouncyCastle (AS2), all Apache-2.0/MIT-style licensed.
## Release
README.md
+60 -4
View File
@@ -5,10 +5,11 @@ and application, and TA1/997/999 acknowledgements. See [the design](docs/DESIGN.
## Status
Phases a1, a2 and b implement inbound/outbound retail EDI. Original baseline guides cover
Phases a1, a2, b and c implement inbound/outbound retail EDI over gateway, SFTP, HTTPS and AS2. Original baseline guides cover
850, 855, 856, 810, 846 and 860 in 004010/005010, plus 997/999. Source records and associations
produce validated, encrypted outbound interchanges, with control-number tracking, retries,
acknowledgement monitoring and operator resend/cancel actions. AS2 and email sending remain disabled.
acknowledgement monitoring and operator resend/cancel actions. AS2 (signed, encrypted, compressed, synchronous and
asynchronous MDN, certificate rollover) is built in; email sending remains disabled until the qbit-email API ships.
A runnable synthetic [demo](docs/DEMO.md) includes Next UI and needs no external services.
## Requirements and installation
@@ -112,6 +113,8 @@ Full table permissions include `edi.partner.read|insert|edit|delete`, `edi.map.*
| Resend / Resend As New | `edi.resend` |
| Cancel | `edi.cancel` |
| Test Connection | `edi.transport.test` |
| Promote Certificate | `edi.certificate.promote` |
| Export Public Certificate | `edi.certificate.read` |
| Scheduled workers / ESB enqueue | `edi.admin` |
Partner locks use `edi_partner` with `edi.admin` all-access. Home queries explicitly enforce
@@ -131,6 +134,7 @@ the viewing user's permissions. Processes are table/record actions, never naviga
| `payloadStore` | `DATABASE` or `OBJECT_STORAGE` |
| `payloadBucketAlias` | `edi` |
| `secretsResolver` | Environment-variable lookup |
| `publicBaseUrl` | Null; the deployment base URL (for example `https://edi.example.com`), required for ASYNC AS2 receipts |
| `clock` | UTC system clock |
| `esbDestinationName` | Null; optional core ESB destination |
@@ -146,7 +150,7 @@ alias and select `OBJECT_STORAGE`. EDI uses only its public API and system acces
Optional `qqq-esb` publishes setup/activity events after commit. Document events contain only ID
and status because RC.5's standard factory otherwise copies PHI fields. Alert names are
`edi.inbox.dead-letter`, `edi.ack.overdue`, `edi.ack.rejected`, `edi.send.failed`,
`edi.certificate.expiring` and `edi.certificate.expired` (RC.5 requires dotted names).
`edi.certificate.expiring`, `edi.certificate.expired` and `edi.mdn.overdue` (RC.5 requires dotted names).
RC.5 lacks the planned core `DataClassification` API. `EdiClassifications` marks PHI/PII/confidential
fields and PHI fields deny AI exposure. Hosts must supply secured-data protection, database
@@ -180,7 +184,8 @@ TRANSPORT exception and raise `edi.send.failed`.
| GATEWAY | `gatewayEndpointName`; marked `FileGatewayOutbox` stand-in with a provider registration hook. Put is SENT; `EdiFileGatewayConsumer.onDownloaded` is DELIVERED. |
| SFTP | MINA client from core's filesystem module; required `sftpHostKeyFingerprint` in MINA SHA256 format, username and password/private-key env name. Streams upload to `.tmp`, then rename. `sftpPollCron` schedules remote pickup; `sftpFilePattern` is a Java regex (default `.*\.x12`). DELETE/MOVE occurs only after durable inbox commit. |
| HTTPS | POST with NONE/BASIC/BEARER/HMAC; TLS required except loopback. BASIC's secret is `user:password`; BEARER's is the token; HMAC uses core Standard Webhooks `whsec_` format. `httpsHeadersJson` holds nonsecret headers only. Redirects are refused; 2xx is DELIVERED, 429/5xx retry. |
| AS2 / EMAIL | Settings validation only. Sending returns a permanent unavailable result until phase c / the email API. |
| AS2 | See [AS2](#as2). |
| EMAIL | Settings validation only. Sending returns a permanent unavailable result until the qbit-email API ships. |
Optional Javalin registers `POST /qqq/hooks/edi/https/{transport}`. `httpsInboundAuth` must be
BASIC, BEARER or HMAC with `httpsSecretEnvVar`. It authenticates exact bytes, caps intake at
@@ -200,6 +205,57 @@ without re-running its map and links `originalInterchangeId`. Cancel accepts QUE
Existing a2 databases need `edi_interchange.original_interchange_id INTEGER` and a unique index
on `edi_outbound_request(agreement_document_id, event_id)` in addition to their prior schema.
## AS2
`As2Transport` (type `AS2`) sends and receives AS2 (RFC 4130) with S/MIME signing and encryption
(SHA-256/384/512, AES-128/256 CBC or GCM; SHA1 and 3DES only when set explicitly and flagged by
the read-only `weakCrypto` field), optional zlib compression, and synchronous or asynchronous
signed MDNs. Everything is in memory: no keystore, key or temporary file is ever written.
**Transport settings** (`edi_transport`, type AS2): `as2LocalId`, `as2PartnerId`, `as2Url` (HTTPS;
plain HTTP only on loopback), `as2Subject`, `signAlgorithm` (blank = unsigned), `encryptAlgorithm`
(blank = unencrypted), `compress`, `mdnMode` (`SYNC` default, `ASYNC`, `NONE`), `mdnSigned`,
`localCertificateId` (ours), `partnerCertificateId`, optional `partnerTlsCertificateId` (pins the
partner TLS certificate), and optional `httpAuthUserEnvVar` / `httpAuthPasswordEnvVar` for HTTP
Basic. For inbound, a set `signAlgorithm` / `encryptAlgorithm` means signed / encrypted messages are
required (otherwise `insufficient-message-security`). A VAN is an AS2 transport with `isVan` true
and no `partnerId`; its partner certificate belongs to the VAN operator's partner record, and
routing to trading partners stays by ISA identity.
**Certificates** (`edi_certificate`): paste the public PEM only; subject, issuer, serial,
SHA-256 fingerprint and validity are parsed on save, and a pasted private key is refused. Our private
key is never stored: `privateKeyEnvVar` names an environment variable holding a PEM PKCS#8 key or a
base64 PKCS#12 store (with `privateKeyPasswordEnvVar` for its password). A missing or mismatched
variable fails plainly, naming only the variable. Status `ACTIVE` signs and encrypts outbound;
inbound accepts `ACTIVE` and `NEXT` (ours for decryption, the partner's for verification), so a
rollover is: add the new certificate as `NEXT`, tell the partner, then run **Promote Certificate**
(`NEXT` becomes `ACTIVE`, the previous `ACTIVE` of the same partner and usage becomes `RETIRED`).
**Export Public Certificate** downloads the PEM to send to a partner. The monitor alerts at 30, 14
and 7 days before expiry and at expiry for `ACTIVE` and `NEXT` certificates.
**Hook URLs** (outside `/qqq/v1`, authenticated per transport by AS2 identity, signature and
certificate; give your partner the first, and the second is sent in `Receipt-Delivery-Option`):
| URL | Purpose |
| --- | --- |
| `POST {base}/qqq/hooks/edi/as2/{transport}` | Receive messages; answers with the MDN, or 200 and a queued asynchronous MDN |
| `POST {base}/qqq/hooks/edi/as2/{transport}/mdn` | Receive asynchronous MDNs for messages we sent |
`{transport}` is the transport `name`. Set `EdiQBitConfig.withPublicBaseUrl(...)` so `ASYNC`
transports can announce the second URL. Bodies are limited by the largest `maxInterchangeBytes` of
the agreements on the transport (50 MB default). A retransmitted Message-ID is answered with the
original MDN and is not reprocessed; a bad signature, decryption or MIC returns an MDN with
`processed/error: authentication-failed`, `decryption-failed` or `integrity-check-failed` and creates
no inbox row; a storage failure returns `unexpected-processing-error` so the partner retries. Every MDN
sent or received is kept in the read-only `edi_as2_mdn` table (AS2 MDNs child lists on the
outbound message and the inbound message) with the raw MDN in the encrypted payload store.
`ASYNC` messages stay SENT until the MDN arrives; the `edi_as2_mdn_sender` step delivers the MDNs we
owe, retrying with the send backoff, and `edi.mdn.overdue` warns when an `ASYNC` MDN is later than
the agreement document `ackDueMinutes` (60 when unset).
Existing databases need the new `edi_as2_mdn` table, `edi_outbound_message.transport_mic` and
`edi_transport.weak_crypto` (see `docs/ddl/edi-postgres.sql`).
## Build and verification
```bash
docs/DEMO.md
+14 -1
View File
@@ -19,7 +19,20 @@ invoices and inventory. Forty synthetic inbound documents go through intake and
850s, 860s, validation errors, an unknown sender and a byte duplicate. Northwind's 855/856/810/846
pass the outbound worker and an in-memory gateway; incoming 997s reconcile some sent groups.
An overdue group, an exhausted send and several open exceptions provide operational examples.
Two public self-signed certificates are generated in memory; private keys are never persisted.
Public self-signed certificates are generated in memory (two for expiry alerts, three for AS2); private keys are never persisted.
Pacific Wholesale (code PACIFIC) is the AS2 partner. An in-process BouncyCastle peer listens on a
random loopback port and the seed generates synthetic RSA certificates in memory (our ACTIVE and NEXT
certificates and the partner's; the private keys live only in the demo secrets map and are never
written to disk). The seed sends three 850s to the application's own AS2 hook (signed, encrypted;
two with base64 transfer encoding), which answers with signed MDNs (`edi_as2_mdn` rows, shown as
"AS2 MDNs" on the inbound message); the 997s go back over AS2 as well. An 855 built from a source record is sent
over AS2 to the peer, which verifies the signature and MIC and returns a signed synchronous MDN
(the outbound message becomes DELIVERED with its `transport_mic`). One more message goes to the
asynchronous transport `pacific-as2-async`: it stays SENT until the peer posts the MDN to
`/qqq/hooks/edi/as2/pacific-as2-async/mdn`. The running demo serves
`POST /qqq/hooks/edi/as2/pacific-as2` (and `.../mdn`) on the same port; Test Connection,
Export Public Certificate and Promote Certificate (on `demo-as2-next`) work on the AS2 records.
Phase d adds warehouse and logistics traffic, seeded last so the retail scenarios above are
unchanged. Harbor Logistics (the 3PL) gets a gateway OUTBOX transport (the partner pulls over SFTP)
docs/DESIGN.md
+77 -24
View File
@@ -349,30 +349,61 @@ For partners and VANs that host their own SFTP, EDI connects out with core's SFT
not depend on it (G1) but uses the same core client, so behaviour and host-key handling match.
### 5.5 AS2 (phase c)
Built on as2-lib (Apache-2.0):
- **Send:** `AS2Client` with partnership settings built from `edi_transport` and `edi_certificate`
(a custom partnership factory and certificate factory backed by our tables, keys from env vars, no
keystore file). Sign, then encrypt, optional compression; the `Message-ID` is recorded. Sync MDN:
verify the signature and the MIC against the sent content; a mismatch or a failed disposition
fails the send. Async MDN: `Receipt-Delivery-Option` is
`https://<host>/qqq/hooks/edi/as2/{transport}/mdn`; the message stays SENT until the MDN arrives,
then becomes DELIVERED or FAILED. An MDN not received within `ackDueMinutes` raises
`edi_mdn_overdue`.
- **Receive:** `POST /qqq/hooks/edi/as2/{transport}` (a route provider outside `/qqq/v1`). The
handler reads the body up to the agreement's `maxInterchangeBytes` (in memory up to 8 MB, then
streamed to object storage, never to a temp file), uses as2-lib's receive helpers to decrypt,
verify the signature against the partner's certificate, decompress and compute the MIC, then calls
`InboundSink.receive` and returns the signed sync MDN, or sends the async MDN after commit. A
message that cannot be stored gets a failure MDN (`processed/error:
unexpected-processing-error`) so the partner retries.
- The AS2 `Message-ID` is the inbox dedupe key: a retransmitted message gets the original MDN again
and is not reprocessed.
- MDNs, sent and received, are stored in **`edi_as2_mdn`** [AS2 MDN] (child of the outbound message
or the inbox row): `direction`, `messageId`, `originalMessageId`, `disposition`, `micAlgorithm`,
`micValue`, `signed`, `signatureValid`, `payloadRef` (the raw MDN in object storage),
`receivedAt`.
- TLS: outbound uses the JVM trust store plus optional pinning (`partnerTlsCertificateId`). Inbound
TLS terminates at the host's ingress.
Implemented by `As2Transport` (send, Test Connection) and `As2RouteProvider` (receive). All of it is
in memory; no keystore, key or temporary file is written, and as2-lib's file-backed pieces are not used.
- **Libraries.** as2-lib 5.1.8 supplies the algorithm tables (`ECryptoAlgorithmSign/Crypt`), the
`MIC` and `DispositionOptions` types and its BouncyCastle S/MIME signer. as2-lib's decrypt and
verify stage content through `AS2ResourceHelper` temporary files and its receive side needs
servlet classes, so decryption, signature verification, compression, encryption and the MIC
(RFC 4130, RFC 5751, RFC 5402) use BouncyCastle (`bcpkix`, `bcjmail`) directly in `As2Crypto`,
and MDNs are built and parsed in `As2Mdn`. The MIC is the digest of the exact bytes of the signed
MIME entity (headers included), or of the transmitted entity when unsigned, so any partner
transfer encoding (binary or base64) agrees.
- **Send:** the message is compressed (optional), signed, then encrypted; `Message-ID` is stable per
outbound message (`edi-<id>-<digest>@<as2LocalId>`) and recorded as `transportMessageId`, and the MIC
we computed is kept in `edi_outbound_message.transportMic`. Configuration problems (a missing key
variable, an expired certificate) are RETRY results that name only the problem. Sync MDN: verify the
signature (required when `mdnSigned`), compare Received-Content-MIC, and require disposition
`processed`; a mismatch or failed disposition is FAILED with the (sanitized) disposition text.
`NONE` leaves the message SENT. Async MDN: `Receipt-Delivery-Option` is
`{publicBaseUrl}/qqq/hooks/edi/as2/{transport}/mdn`; the message stays SENT until the MDN arrives,
then becomes DELIVERED or FAILED; an MDN not received within the agreement document `ackDueMinutes`
(60 when unset) raises the warning `edi.mdn.overdue`, resolved by the MDN.
- **Receive:** `POST /qqq/hooks/edi/as2/{transport}` (outside `/qqq/v1`). The handler checks
`AS2-From`/`AS2-To` against the transport (403 otherwise), reads the body up to the largest
`maxInterchangeBytes` of the agreements on the transport (50 MB default, in memory), decrypts with any
of our ACTIVE or NEXT keys, verifies the signature against the partner's ACTIVE and NEXT certificates
(signed is required when `signAlgorithm` is set, encrypted when `encryptAlgorithm` is set), unwraps
compression, computes the MIC, calls `InboundSink.receive` (`transportType` AS2, `externalId` =
Message-ID, scoped by transport id if another partner already used the same id), and answers with the
MDN: synchronous in the response (signed with our ACTIVE key when requested or `mdnSigned`), or, when
`Receipt-Delivery-Option` is present, 200 at once and a queued `edi_as2_mdn` row that
`As2MdnSender` (scheduled `edi_as2_mdn_sender`, every 15 seconds) posts with the send backoff and
`maxAttempts`, failing with `edi.send.failed`. An asynchronous receipt address must be HTTPS
(loopback excepted) and, unless signatures are required, on the partner's own host; failures before
the sender is authenticated are always answered synchronously.
- A retransmitted Message-ID returns the original MDN again, byte for byte, and is not reprocessed
(an asynchronous one is queued again). Failures return MDNs with `processed/error:
authentication-failed` (bad or unverifiable signature), `integrity-check-failed` (digest mismatch),
`decryption-failed`, `insufficient-message-security` or `unexpected-processing-error` (storage, or a
key we cannot load, so the partner retries); no inbox row exists for a rejected message.
- **Certificates:** `CertificateService` resolves our key from the `privateKeyEnvVar` variable (PEM
PKCS#8 or base64 PKCS#12 with `privateKeyPasswordEnvVar`), proves it matches the certificate, and
never caches or logs it. Outbound signs and encrypts with the ACTIVE certificate (the referenced one
when ACTIVE, otherwise the ACTIVE one of the same partner and usage), inbound accepts ACTIVE and NEXT.
"Promote Certificate" makes NEXT ACTIVE and retires the previous ACTIVE one of the same partner and
usage. A VAN transport (`isVan`, no `partnerId`) has its partner certificate on the VAN operator's
partner record, or with no partner for a shared certificate (no rollover candidates).
- MDNs, sent and received, are stored in **`edi_as2_mdn`** [AS2 MDN] (read-only; child list "AS2 MDNs"
of the outbound message and the inbox row; record label `<direction> MDN <originalMessageId>`):
`direction` (INBOUND = received, OUTBOUND = we sent it), `messageId`, `originalMessageId`,
`disposition`, `micAlgorithm`, `micValue`, `signed`, `signatureValid`, `payloadRef` (the raw MDN,
encrypted), `receivedAt`, `transportId`, `partnerId`, `outboundMessageId`, `inboxId`, and the
queue columns `deliveryUrl`, `deliveryStatus` (NONE, PENDING, SENT, FAILED), `deliveryAttempts`,
`nextAttemptAt`, `deliveredAt`.
- TLS: outbound uses the JVM trust store, or only the pinned certificate when `partnerTlsCertificateId`
is set. Inbound TLS terminates at the host's ingress. SHA1 and 3DES are accepted only when set
explicitly and set the read-only `weakCrypto` flag on the transport.
### 5.6 HTTPS and email
- **HTTPS out:** POST the interchange (`application/edi-x12` or `application/EDIFACT`) with the
@@ -924,3 +955,25 @@ RC.5 adaptations implemented in this worktree:
- `InboundApplier` assumed every target table has associations; a flat table (the 214 status sample) hit a null pointer. It is now null-safe.
- The 214 sample map records the first LX event per transaction into a flat status table and always inserts (no match key), so repeated statuses form a history; a host that wants every LX event maps an association.
- Demo: Harbor Logistics gets a gateway OUTBOX transport for 945s (the partner pulls over SFTP, per 5.3) after the retail seed has run, so its earlier acknowledgements keep their visible retries; the fourth partner Ridgeline Freight is added, so `EdiDemoServerTest` now expects four partners.
||||||| parent of bfb63ce (feat: phase c - AS2 transport)
## Phase c implementation notes (2026-10-14)
- as2-lib's decrypt and verify write temporary files and its receive helpers need servlet classes, so only
its enum tables, `MIC`, `DispositionOptions` and signer are used; the rest is BouncyCastle in memory
(section 5.5). Its dependency tree pulls `bcjmail` 1.85 (Jakarta Mail), so `bcpkix` is pinned to 1.85 and
promoted from test scope; no other QQQ dependency changed.
- The MIC is computed over the exact signed bytes (headers included) instead of as2-lib's
decode-then-re-encode, which differs for base64 content; verified against an independent
BouncyCastle peer with base64 and binary transfer encodings.
- Encryption wraps the exact bytes that were hashed (CMS enveloped data over the serialized signed
entity), so the MIC cannot drift between signing and encryption.
- `edi_as2_mdn` doubles as the asynchronous MDN queue (`deliveryStatus`, `deliveryAttempts`,
`nextAttemptAt`) instead of a separate table; the scheduled step is `edi_as2_mdn_sender`.
- New columns: `edi_outbound_message.transport_mic` (the MIC we sent, for asynchronous comparison) and
`edi_transport.weak_crypto`. New config: `EdiQBitConfig.publicBaseUrl`.
- The alert is `edi.mdn.overdue` (RC.5 rejects underscores); `SendWorker` now records the transport
failure reason (value-free by contract: an HTTP status, an AS2 disposition or a variable name) in
`lastError` instead of a generic text.
- Not implemented: EC and RSA-PSS signing keys (RSA only), MIC comparison under a different algorithm than
the one requested, CAQH/Peppol profiles, and AS2 over a host-mounted client certificate (`MTLS`).
docs/ddl/edi-postgres.sql
+28 -1
View File
@@ -55,6 +55,31 @@ CREATE TABLE edi_agreement_document (
UNIQUE (agreement_id, direction, transaction_set, version)
);
CREATE TABLE edi_as2_mdn (
create_date TIMESTAMP,
delivered_at TIMESTAMP,
delivery_attempts INTEGER,
delivery_status VARCHAR(20),
delivery_url VARCHAR(1000),
direction VARCHAR(20),
disposition VARCHAR(255),
id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
inbox_id INTEGER,
message_id VARCHAR(255),
mic_algorithm VARCHAR(50),
mic_value VARCHAR(255),
modify_date TIMESTAMP,
next_attempt_at TIMESTAMP,
original_message_id VARCHAR(255),
outbound_message_id INTEGER,
partner_id INTEGER,
payload_ref VARCHAR(255),
received_at TIMESTAMP,
signature_valid BOOLEAN,
signed BOOLEAN,
transport_id INTEGER
);
CREATE TABLE edi_certificate (
alias VARCHAR(255),
certificate_pem VARCHAR(10000),
@@ -388,7 +413,8 @@ CREATE TABLE edi_outbound_message (
size_bytes BIGINT,
status VARCHAR(255),
transport_id INTEGER,
transport_message_id VARCHAR(255)
transport_message_id VARCHAR(255),
transport_mic VARCHAR(255)
);
CREATE TABLE edi_outbound_request (
@@ -536,5 +562,6 @@ CREATE TABLE edi_transport (
sign_algorithm VARCHAR(255),
type VARCHAR(255),
van_name VARCHAR(255),
weak_crypto BOOLEAN,
UNIQUE (name)
);
pom.xml
+2 -1
View File
@@ -69,7 +69,8 @@
<dependencies>
<dependency><groupId>com.kingsrook.qqq</groupId><artifactId>qqq-frontend-next</artifactId><version>1.0.0-RC.14</version><scope>test</scope></dependency>
<dependency><groupId>org.bouncycastle</groupId><artifactId>bcpkix-jdk18on</artifactId><version>1.81</version><scope>test</scope></dependency>
<dependency><groupId>org.bouncycastle</groupId><artifactId>bcpkix-jdk18on</artifactId><version>1.85</version></dependency>
<dependency><groupId>com.helger.as2</groupId><artifactId>as2-lib</artifactId><version>5.1.8</version></dependency>
<dependency><groupId>com.kingsrook.qqq</groupId><artifactId>qqq-backend-module-filesystem</artifactId></dependency>
<dependency>
<groupId>com.kingsrook.qqq</groupId>
src/main/java/com/kingsrook/qbits/edi/EdiQBitConfig.java
+20
View File
@@ -68,6 +68,7 @@ public class EdiQBitConfig implements QBitConfig
private String appName = "edi";
private String appLabel = "EDI";
private String parentAppName;
private String publicBaseUrl;
@@ -350,4 +351,23 @@ public class EdiQBitConfig implements QBitConfig
esbDestinationName = value;
return (this);
}
/*******************************************************************************
** Public base URL of this deployment (for example https://edi.example.com),
** used to build the asynchronous AS2 receipt address
** {base}/qqq/hooks/edi/as2/{transport}/mdn.
*******************************************************************************/
public String getPublicBaseUrl()
{
return (publicBaseUrl);
}
/*******************************************************************************
** Set the public base URL.
*******************************************************************************/
public EdiQBitConfig withPublicBaseUrl(String value)
{
publicBaseUrl = value;
return (this);
}
}
src/main/java/com/kingsrook/qbits/edi/EdiQBitProducer.java
+3 -2
View File
@@ -135,11 +135,12 @@ public class EdiQBitProducer implements QBitMetaDataProducer<EdiQBitConfig>
instance.withSupplementalMetaData(new com.kingsrook.qbits.edi.transport.EdiTransportPluginsMetaData()
.withPlugin(new com.kingsrook.qbits.edi.transport.gateway.GatewayTransport())
.withPlugin(new com.kingsrook.qbits.edi.transport.HttpsTransport()).withPlugin(new com.kingsrook.qbits.edi.transport.SftpTransport())
.withPlugin(new com.kingsrook.qbits.edi.transport.UnavailableTransport("AS2")).withPlugin(new com.kingsrook.qbits.edi.transport.UnavailableTransport("EMAIL")));
.withPlugin(new com.kingsrook.qbits.edi.as2.As2Transport()).withPlugin(new com.kingsrook.qbits.edi.transport.UnavailableTransport("EMAIL")));
try
{
com.kingsrook.qqq.middleware.javalin.QJavalinMetaData.ofOrWithNew(instance).withAdditionalRouteProviderReference(
new com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference(com.kingsrook.qbits.edi.transport.HttpsRouteProvider.class));
new com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference(com.kingsrook.qbits.edi.transport.HttpsRouteProvider.class))
.withAdditionalRouteProviderReference(new com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference(com.kingsrook.qbits.edi.as2.As2RouteProvider.class));
}
catch(NoClassDefFoundError optionalMiddlewareAbsent)
{
src/main/java/com/kingsrook/qbits/edi/as2/As2Crypto.java
+646
View File
@@ -0,0 +1,646 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi.as2;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.security.cert.X509Certificate;
import java.util.Arrays;
import java.util.Date;
import java.util.List;
import com.helger.as2lib.crypto.BCCryptoHelper;
import com.helger.as2lib.crypto.ECryptoAlgorithmCrypt;
import com.helger.as2lib.crypto.ECryptoAlgorithmSign;
import com.helger.as2lib.crypto.MIC;
import com.helger.mail.cte.EContentTransferEncoding;
import com.kingsrook.qbits.edi.as2.CertificateService.LocalKey;
import jakarta.activation.DataHandler;
import jakarta.mail.internet.ContentType;
import jakarta.mail.internet.MimeBodyPart;
import jakarta.mail.internet.MimeMultipart;
import jakarta.mail.internet.MimeUtility;
import jakarta.mail.util.ByteArrayDataSource;
import org.bouncycastle.cms.CMSEnvelopedData;
import org.bouncycastle.cms.CMSEnvelopedDataGenerator;
import org.bouncycastle.cms.CMSProcessableByteArray;
import org.bouncycastle.cms.CMSSignerDigestMismatchException;
import org.bouncycastle.cms.RecipientInformation;
import org.bouncycastle.cms.SignerInformation;
import org.bouncycastle.cms.jcajce.JcaSimpleSignerInfoVerifierBuilder;
import org.bouncycastle.cms.jcajce.JceCMSContentEncryptorBuilder;
import org.bouncycastle.cms.jcajce.JceKeyTransEnvelopedRecipient;
import org.bouncycastle.cms.jcajce.JceKeyTransRecipientId;
import org.bouncycastle.cms.jcajce.JceKeyTransRecipientInfoGenerator;
import org.bouncycastle.cms.jcajce.ZlibCompressor;
import org.bouncycastle.cms.jcajce.ZlibExpanderProvider;
import org.bouncycastle.mail.smime.SMIMECompressed;
import org.bouncycastle.mail.smime.SMIMECompressedGenerator;
import org.bouncycastle.mail.smime.SMIMEEnveloped;
import org.bouncycastle.mail.smime.SMIMESigned;
/*******************************************************************************
** The AS2 cryptographic layer, entirely in memory. S/MIME signing and the
** algorithm tables use as2-lib's BouncyCastle helper; encryption, decryption,
** signature verification, compression and the MIC use BouncyCastle directly,
** because as2-lib's decrypt and verify stage content in temporary files and this
** qbit never writes local files. The MIC is computed over the exact bytes of the
** signed (or, when unsigned, the transmitted) MIME entity, headers included, so
** both ends agree whatever transfer encoding was used. Messages follow RFC 4130
** (AS2), RFC 5751 (S/MIME) and RFC 5402 (compression).
*******************************************************************************/
public final class As2Crypto
{
private static final BCCryptoHelper HELPER = new BCCryptoHelper();
private static final String PROVIDER = BCCryptoHelper.DEFAULT_SECURITY_PROVIDER_NAME;
private static final byte[] HEADER_END = "\r\n\r\n".getBytes(StandardCharsets.US_ASCII);
private static final byte[] LINE_END = "\r\n".getBytes(StandardCharsets.US_ASCII);
public static final long MAX_EXPANDED_BYTES = 256L * 1024 * 1024;
/*******************************************************************************
** The result of unwrapping an inbound message: the EDI payload, what protection
** was applied and the MIC of the signed (or received) content.
*******************************************************************************/
public record Decoded(byte[] payload, String fileName, Boolean encrypted, Boolean signed, Boolean compressed, MIC mic)
{
}
/*******************************************************************************
** A MIME entity split for HTTP: the Content-Type, transfer encoding and
** disposition go in HTTP headers, the body is the HTTP body.
*******************************************************************************/
public record Wire(String contentType, String transferEncoding, String contentDisposition, byte[] body)
{
}
/*******************************************************************************
** Utility class.
*******************************************************************************/
private As2Crypto()
{
}
/*******************************************************************************
** Map the signAlgorithm setting; null means unsigned.
*******************************************************************************/
public static ECryptoAlgorithmSign signAlgorithm(String setting)
{
if(setting == null || setting.isBlank())
{
return (null);
}
return (switch(setting)
{
case "SHA256" -> ECryptoAlgorithmSign.DIGEST_SHA_256;
case "SHA384" -> ECryptoAlgorithmSign.DIGEST_SHA_384;
case "SHA512" -> ECryptoAlgorithmSign.DIGEST_SHA_512;
case "SHA1" -> ECryptoAlgorithmSign.DIGEST_SHA_1;
default -> throw new IllegalArgumentException("Unsupported signAlgorithm");
});
}
/*******************************************************************************
** Map the encryptAlgorithm setting; null means unencrypted.
*******************************************************************************/
public static ECryptoAlgorithmCrypt encryptAlgorithm(String setting)
{
if(setting == null || setting.isBlank())
{
return (null);
}
return (switch(setting)
{
case "AES128_CBC" -> ECryptoAlgorithmCrypt.CRYPT_AES128_CBC;
case "AES256_CBC" -> ECryptoAlgorithmCrypt.CRYPT_AES256_CBC;
case "AES128_GCM" -> ECryptoAlgorithmCrypt.CRYPT_AES128_GCM;
case "AES256_GCM" -> ECryptoAlgorithmCrypt.CRYPT_AES256_GCM;
case "3DES" -> ECryptoAlgorithmCrypt.CRYPT_3DES;
default -> throw new IllegalArgumentException("Unsupported encryptAlgorithm");
});
}
/*******************************************************************************
** SHA1 signatures and 3DES encryption are accepted only when set explicitly,
** and are flagged as weak.
*******************************************************************************/
public static boolean isWeak(String sign, String encrypt)
{
return ("SHA1".equals(sign) || "3DES".equals(encrypt));
}
/*******************************************************************************
** Parse a complete MIME entity (headers, blank line, body) held in memory.
*******************************************************************************/
public static MimeBodyPart parse(byte[] entity) throws As2Exception
{
try
{
return (new MimeBodyPart(new ByteArrayInputStream(entity)));
}
catch(Exception e)
{
throw (new As2Exception("The AS2 message is not a valid MIME entity", As2Exception.UNEXPECTED_PROCESSING));
}
}
/*******************************************************************************
** Rebuild the MIME entity of an HTTP request or response from its Content-Type,
** Content-Transfer-Encoding and Content-Disposition headers and its body.
*******************************************************************************/
public static MimeBodyPart fromHttp(String contentType, String transferEncoding, String contentDisposition, byte[] body) throws As2Exception
{
return (parse(entity(contentType, transferEncoding, contentDisposition, body)));
}
/*******************************************************************************
** The MIME entity bytes (headers, blank line, body) for HTTP-level parts. Header
** values with line breaks are refused.
*******************************************************************************/
public static byte[] entity(String contentType, String transferEncoding, String contentDisposition, byte[] body) throws As2Exception
{
for(String value : Arrays.asList(contentType, transferEncoding, contentDisposition))
{
if(value != null && (value.contains("\r") || value.contains("\n")))
{
throw (new As2Exception("The AS2 message has an invalid header", As2Exception.UNEXPECTED_PROCESSING));
}
}
if(contentType == null || contentType.isBlank())
{
throw (new As2Exception("The AS2 message has no valid Content-Type", As2Exception.UNEXPECTED_PROCESSING));
}
ByteArrayOutputStream entity = new ByteArrayOutputStream();
String headers = "Content-Type: " + contentType + "\r\n" + (transferEncoding == null || transferEncoding.isBlank() ? "" : "Content-Transfer-Encoding: " + transferEncoding + "\r\n")
+ (contentDisposition == null || contentDisposition.isBlank() ? "" : "Content-Disposition: " + contentDisposition + "\r\n") + "\r\n";
entity.writeBytes(headers.getBytes(StandardCharsets.ISO_8859_1));
entity.writeBytes(body);
return (entity.toByteArray());
}
/*******************************************************************************
** Serialize a MIME entity (headers and encoded body).
*******************************************************************************/
public static byte[] toBytes(MimeBodyPart part) throws As2Exception
{
try
{
ByteArrayOutputStream out = new ByteArrayOutputStream();
part.writeTo(out);
return (out.toByteArray());
}
catch(Exception e)
{
throw (new As2Exception("The AS2 message could not be serialized", As2Exception.UNEXPECTED_PROCESSING));
}
}
/*******************************************************************************
** Split a MIME entity into the HTTP headers that describe it and its body.
*******************************************************************************/
public static Wire toWire(MimeBodyPart part) throws As2Exception
{
return (splitRaw(toBytes(part)));
}
/*******************************************************************************
** Split already serialized MIME entity bytes into the Content-Type,
** Content-Transfer-Encoding and Content-Disposition values and the body,
** without re-serializing (which could invalidate a signature).
*******************************************************************************/
public static Wire splitRaw(byte[] bytes) throws As2Exception
{
int split = indexOf(bytes, HEADER_END, 0);
if(split < 0)
{
throw (new As2Exception("The AS2 message has no body", As2Exception.UNEXPECTED_PROCESSING));
}
String contentType = null;
String encoding = null;
String disposition = null;
String current = null;
StringBuilder value = new StringBuilder();
for(String line : (new String(bytes, 0, split, StandardCharsets.ISO_8859_1) + "\r\nX-End: end").split("\r\n"))
{
if((line.startsWith(" ") || line.startsWith("\t")) && current != null)
{
value.append(' ').append(line.trim());
continue;
}
if(current != null)
{
contentType = "content-type".equals(current) ? value.toString() : contentType;
encoding = "content-transfer-encoding".equals(current) ? value.toString() : encoding;
disposition = "content-disposition".equals(current) ? value.toString() : disposition;
}
int colon = line.indexOf(':');
current = colon < 0 ? null : line.substring(0, colon).trim().toLowerCase(java.util.Locale.ROOT);
value = new StringBuilder(colon < 0 ? "" : line.substring(colon + 1).trim());
}
byte[] body = new byte[bytes.length - split - HEADER_END.length];
System.arraycopy(bytes, split + HEADER_END.length, body, 0, body.length);
return (new Wire(contentType, encoding, disposition, body));
}
/*******************************************************************************
** The EDI document as a binary MIME part.
*******************************************************************************/
public static MimeBodyPart payloadPart(byte[] payload, String contentType, String fileName) throws As2Exception
{
try
{
MimeBodyPart part = new MimeBodyPart();
part.setDataHandler(new DataHandler(new ByteArrayDataSource(payload, contentType)));
part.setHeader("Content-Type", contentType);
part.setHeader("Content-Transfer-Encoding", "binary");
part.setHeader("Content-Disposition", "attachment; filename=\"" + fileName + "\"");
return (part);
}
catch(Exception e)
{
throw (new As2Exception("The EDI document could not be wrapped for AS2", As2Exception.UNEXPECTED_PROCESSING));
}
}
/*******************************************************************************
** Compress a part (RFC 5402, zlib).
*******************************************************************************/
public static MimeBodyPart compress(MimeBodyPart part) throws As2Exception
{
try
{
SMIMECompressedGenerator generator = new SMIMECompressedGenerator();
generator.setContentTransferEncoding("binary");
return (generator.generate(part, new ZlibCompressor()));
}
catch(Exception e)
{
throw (new As2Exception("The AS2 message could not be compressed", As2Exception.UNEXPECTED_PROCESSING));
}
}
/*******************************************************************************
** The MIC of a MIME entity: the digest of its exact bytes, headers included
** (RFC 4130 section 7.3.1).
*******************************************************************************/
public static MIC mic(byte[] entity, ECryptoAlgorithmSign algorithm) throws As2Exception
{
try
{
return (new MIC(MessageDigest.getInstance(algorithm.getID().toUpperCase(java.util.Locale.ROOT)).digest(entity), algorithm));
}
catch(Exception e)
{
throw (new As2Exception("The message integrity check could not be computed", As2Exception.UNEXPECTED_PROCESSING));
}
}
/*******************************************************************************
** The exact bytes of the first body part of a multipart entity: the signed
** content of multipart/signed, without the delimiter lines around it.
*******************************************************************************/
public static byte[] firstPart(String contentType, byte[] body) throws As2Exception
{
try
{
String boundary = new ContentType(contentType).getParameter("boundary");
int open = indexOf(body, ("--" + boundary).getBytes(StandardCharsets.ISO_8859_1), 0);
int start = open < 0 ? -1 : indexOf(body, LINE_END, open) + LINE_END.length;
int end = start < 0 ? -1 : indexOf(body, ("\r\n--" + boundary).getBytes(StandardCharsets.ISO_8859_1), start - LINE_END.length);
if(end < 0)
{
throw (new IllegalArgumentException("no signed content"));
}
return (Arrays.copyOfRange(body, start, end));
}
catch(Exception e)
{
throw (new As2Exception("The signed AS2 message is malformed", As2Exception.AUTHENTICATION_FAILED));
}
}
/*******************************************************************************
** Sign a part into multipart/signed with the local key.
*******************************************************************************/
public static MimeBodyPart sign(MimeBodyPart part, LocalKey key, ECryptoAlgorithmSign algorithm) throws As2Exception
{
try
{
return (HELPER.sign(part, key.certificate(), key.key(), algorithm, true, algorithm == ECryptoAlgorithmSign.DIGEST_SHA_1, false, EContentTransferEncoding.BINARY));
}
catch(Exception e)
{
throw (new As2Exception("The AS2 message could not be signed with certificate " + key.row().getValueString("alias") + ": " + e.getClass().getSimpleName(), As2Exception.UNEXPECTED_PROCESSING));
}
}
/*******************************************************************************
** Encrypt the exact bytes of a MIME entity to the partner certificate (CMS
** enveloped data), returning the application/pkcs7-mime wire message.
*******************************************************************************/
public static Wire encrypt(byte[] entity, X509Certificate certificate, ECryptoAlgorithmCrypt algorithm) throws As2Exception
{
try
{
CMSEnvelopedDataGenerator generator = new CMSEnvelopedDataGenerator();
generator.addRecipientInfoGenerator(new JceKeyTransRecipientInfoGenerator(certificate).setProvider(PROVIDER));
CMSEnvelopedData enveloped = generator.generate(new CMSProcessableByteArray(entity), new JceCMSContentEncryptorBuilder(algorithm.getOID()).setProvider(PROVIDER).build());
return (new Wire("application/pkcs7-mime; smime-type=enveloped-data; name=smime.p7m", "binary", "attachment; filename=\"smime.p7m\"", enveloped.getEncoded()));
}
catch(Exception e)
{
throw (new As2Exception("The AS2 message could not be encrypted: " + e.getClass().getSimpleName(), As2Exception.UNEXPECTED_PROCESSING));
}
}
/*******************************************************************************
** Unwrap an inbound entity: decrypt with any of our keys, verify the signature
** against the partner certificates, decompress, and compute the MIC over the
** exact signed bytes (or, when unsigned, the entity after decryption). Layers
** may nest in any order (RFC 5402 allows compression before or after signing).
*******************************************************************************/
public static Decoded decode(byte[] entity, List<LocalKey> keys, List<X509Certificate> partnerCertificates, ECryptoAlgorithmSign micAlgorithm, Date now) throws As2Exception
{
byte[] raw = entity;
byte[] micBytes = raw;
MimeBodyPart part = parse(raw);
boolean encrypted = false;
boolean signed = false;
boolean compressed = false;
for(int depth = 0; depth < 6; depth++)
{
if(isEncrypted(part))
{
raw = decrypt(part, keys);
part = parse(raw);
micBytes = signed ? micBytes : raw;
encrypted = true;
}
else if(isSigned(part))
{
verify(part, partnerCertificates, now);
Wire wire = splitRaw(raw);
raw = firstPart(wire.contentType(), wire.body());
micBytes = raw;
part = parse(raw);
signed = true;
}
else if(isCompressed(part))
{
raw = expand(part);
part = parse(raw);
compressed = true;
}
else
{
break;
}
}
try
{
MIC mic = micAlgorithm == null ? null : mic(micBytes, micAlgorithm);
return (new Decoded(part.getInputStream().readAllBytes(), part.getFileName(), encrypted, signed, compressed, mic));
}
catch(As2Exception e)
{
throw (e);
}
catch(Exception e)
{
throw (new As2Exception("The AS2 payload could not be read", As2Exception.UNEXPECTED_PROCESSING));
}
}
/*******************************************************************************
** Verify a multipart/signed entity against the candidate certificates and
** return the signed content. A digest that does not match the content is an
** integrity failure; a signature no candidate certificate verifies is an
** authentication failure. Expired certificates are not accepted.
*******************************************************************************/
public static MimeBodyPart verify(MimeBodyPart part, List<X509Certificate> certificates, Date now) throws As2Exception
{
SMIMESigned signed;
try
{
signed = new SMIMESigned(new MimeMultipart(new ByteArrayDataSource(part.getRawInputStream().readAllBytes(), MimeUtility.unfold(part.getContentType()))));
}
catch(Exception e)
{
throw (new As2Exception("The signed AS2 message is malformed", As2Exception.AUTHENTICATION_FAILED));
}
for(SignerInformation signer : signed.getSignerInfos().getSigners())
{
for(X509Certificate certificate : certificates)
{
if(certificate.getNotAfter().before(now))
{
continue;
}
try
{
if(signer.verify(new JcaSimpleSignerInfoVerifierBuilder().setProvider(PROVIDER).build(certificate)))
{
return (signed.getContent());
}
}
catch(CMSSignerDigestMismatchException e)
{
throw (new As2Exception("The message digest does not match the signed content", As2Exception.INTEGRITY_CHECK_FAILED));
}
catch(Exception e)
{
// This certificate cannot verify the signature; try the next.
continue;
}
}
}
throw (new As2Exception("The signature could not be verified with the partner certificates", As2Exception.AUTHENTICATION_FAILED));
}
/*******************************************************************************
** Decrypt with the first of our keys the message was encrypted to, returning
** the clear MIME entity bytes.
*******************************************************************************/
private static byte[] decrypt(MimeBodyPart part, List<LocalKey> keys) throws As2Exception
{
SMIMEEnveloped enveloped;
try
{
enveloped = new SMIMEEnveloped(part);
}
catch(Exception e)
{
throw (new As2Exception("The encrypted AS2 message is malformed", As2Exception.DECRYPTION_FAILED));
}
for(LocalKey key : keys)
{
try
{
RecipientInformation recipient = enveloped.getRecipientInfos().get(new JceKeyTransRecipientId(key.certificate()));
if(recipient != null)
{
return (recipient.getContent(new JceKeyTransEnvelopedRecipient(key.key()).setProvider(PROVIDER)));
}
}
catch(Exception e)
{
// The key matched the recipient but could not decrypt; try the next key.
continue;
}
}
throw (new As2Exception("The message was not encrypted to an accepted certificate", As2Exception.DECRYPTION_FAILED));
}
/*******************************************************************************
** Inflate a compressed-data entity, bounded to prevent decompression bombs.
*******************************************************************************/
private static byte[] expand(MimeBodyPart part) throws As2Exception
{
try
{
return (new SMIMECompressed(part).getContent(new ZlibExpanderProvider(MAX_EXPANDED_BYTES)));
}
catch(Exception e)
{
throw (new As2Exception("The compressed AS2 message could not be expanded", As2Exception.UNEXPECTED_PROCESSING));
}
}
/*******************************************************************************
** True for application/pkcs7-mime enveloped-data.
*******************************************************************************/
private static boolean isEncrypted(MimeBodyPart part) throws As2Exception
{
try
{
return (HELPER.isEncrypted(part));
}
catch(Exception e)
{
throw (new As2Exception("The AS2 Content-Type is invalid", As2Exception.UNEXPECTED_PROCESSING));
}
}
/*******************************************************************************
** True for multipart/signed.
*******************************************************************************/
private static boolean isSigned(MimeBodyPart part) throws As2Exception
{
try
{
return (HELPER.isSigned(part));
}
catch(Exception e)
{
throw (new As2Exception("The AS2 Content-Type is invalid", As2Exception.UNEXPECTED_PROCESSING));
}
}
/*******************************************************************************
** True for application/pkcs7-mime compressed-data.
*******************************************************************************/
private static boolean isCompressed(MimeBodyPart part) throws As2Exception
{
try
{
return (HELPER.isCompressed(part.getContentType()));
}
catch(Exception e)
{
throw (new As2Exception("The AS2 Content-Type is invalid", As2Exception.UNEXPECTED_PROCESSING));
}
}
/*******************************************************************************
** Index of a byte sequence at or after a position, or -1.
*******************************************************************************/
private static int indexOf(byte[] data, byte[] pattern, int from)
{
for(int i = Math.max(from, 0); i <= data.length - pattern.length; i++)
{
int j = 0;
while(j < pattern.length && data[i + j] == pattern[j])
{
j++;
}
if(j == pattern.length)
{
return (i);
}
}
return (-1);
}
}
src/main/java/com/kingsrook/qbits/edi/as2/As2Exception.java
+69
View File
@@ -0,0 +1,69 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi.as2;
/*******************************************************************************
** A plain, value-free AS2 failure. The message never carries key material,
** secrets or payload content; the optional code is the RFC 4130 disposition
** modifier a receiver reports to the sender (for example authentication-failed).
*******************************************************************************/
public class As2Exception extends Exception
{
public static final String DECRYPTION_FAILED = "decryption-failed";
public static final String AUTHENTICATION_FAILED = "authentication-failed";
public static final String INTEGRITY_CHECK_FAILED = "integrity-check-failed";
public static final String INSUFFICIENT_SECURITY = "insufficient-message-security";
public static final String UNEXPECTED_PROCESSING = "unexpected-processing-error";
private final String code;
/*******************************************************************************
** Failure without a disposition code (configuration and local problems).
*******************************************************************************/
public As2Exception(String message)
{
this(message, null);
}
/*******************************************************************************
** Failure carrying the RFC 4130 disposition modifier.
*******************************************************************************/
public As2Exception(String message, String code)
{
super(message);
this.code = code;
}
/*******************************************************************************
** The RFC 4130 disposition modifier, or null when the failure is local.
*******************************************************************************/
public String getCode()
{
return (code);
}
}
src/main/java/com/kingsrook/qbits/edi/as2/As2Http.java
+234
View File
@@ -0,0 +1,234 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi.as2;
import javax.net.ssl.SSLContext;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509TrustManager;
import java.io.InputStream;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.security.SecureRandom;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;
import java.time.Duration;
import java.util.Base64;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.TreeMap;
/*******************************************************************************
** HTTP client helpers for AS2: bounded timeouts, no redirects, optional pinning
** of the partner TLS certificate (otherwise the JVM trust store applies), and
** bounded response bodies.
*******************************************************************************/
public final class As2Http
{
public static final int MAX_RESPONSE_BYTES = 10 * 1024 * 1024;
/*******************************************************************************
** Status, lower-case header names and the (bounded) body of a response.
*******************************************************************************/
public record Response(int status, Map<String, String> headers, byte[] body)
{
}
/*******************************************************************************
** Accepts only the pinned certificate as the server leaf; the hostname is
** still checked by the TLS stack.
*******************************************************************************/
private static final class PinnedTrustManager implements X509TrustManager
{
private final byte[] pinned;
/*******************************************************************************
** Pin the SHA-256 of the encoded certificate.
*******************************************************************************/
private PinnedTrustManager(X509Certificate certificate) throws Exception
{
pinned = MessageDigest.getInstance("SHA-256").digest(certificate.getEncoded());
}
/*******************************************************************************
** This client never authenticates with a client certificate.
*******************************************************************************/
@Override
public void checkClientTrusted(X509Certificate[] chain, String authType) throws CertificateException
{
throw (new CertificateException("Client certificates are not accepted"));
}
/*******************************************************************************
** Require the leaf to be exactly the pinned certificate.
*******************************************************************************/
@Override
public void checkServerTrusted(X509Certificate[] chain, String authType) throws CertificateException
{
try
{
if(chain == null || chain.length == 0 || !MessageDigest.isEqual(pinned, MessageDigest.getInstance("SHA-256").digest(chain[0].getEncoded())))
{
throw (new CertificateException("The server certificate does not match the pinned partner TLS certificate"));
}
}
catch(CertificateException e)
{
throw (e);
}
catch(Exception e)
{
throw (new CertificateException("The server certificate could not be checked against the pin"));
}
}
/*******************************************************************************
** No trust anchors beyond the pin.
*******************************************************************************/
@Override
public X509Certificate[] getAcceptedIssuers()
{
return (new X509Certificate[0]);
}
}
/*******************************************************************************
** Utility class.
*******************************************************************************/
private As2Http()
{
}
/*******************************************************************************
** A client using the JVM trust store, or only the pinned certificate.
*******************************************************************************/
public static HttpClient client(X509Certificate pinned) throws Exception
{
HttpClient.Builder builder = HttpClient.newBuilder().connectTimeout(Duration.ofSeconds(15)).followRedirects(HttpClient.Redirect.NEVER);
if(pinned != null)
{
SSLContext context = SSLContext.getInstance("TLS");
context.init(null, new TrustManager[] {new PinnedTrustManager(pinned)}, new SecureRandom());
builder.sslContext(context);
}
return (builder.build());
}
/*******************************************************************************
** Trust manager for pin checks in tests and diagnostics.
*******************************************************************************/
public static X509TrustManager pinnedTrustManager(X509Certificate pinned) throws Exception
{
return (new PinnedTrustManager(pinned));
}
/*******************************************************************************
** POST bytes with the given headers.
*******************************************************************************/
public static Response post(HttpClient client, URI uri, Map<String, String> headers, byte[] body, Duration timeout) throws Exception
{
HttpRequest.Builder request = HttpRequest.newBuilder(uri).timeout(timeout).POST(HttpRequest.BodyPublishers.ofByteArray(body));
headers.forEach(request::header);
return (read(client.send(request.build(), HttpResponse.BodyHandlers.ofInputStream())));
}
/*******************************************************************************
** OPTIONS probe: no body is sent.
*******************************************************************************/
public static Response options(HttpClient client, URI uri, Map<String, String> headers, Duration timeout) throws Exception
{
HttpRequest.Builder request = HttpRequest.newBuilder(uri).timeout(timeout).method("OPTIONS", HttpRequest.BodyPublishers.noBody());
headers.forEach(request::header);
return (read(client.send(request.build(), HttpResponse.BodyHandlers.ofInputStream())));
}
/*******************************************************************************
** Read the bounded body and lower-case the header names.
*******************************************************************************/
private static Response read(HttpResponse<InputStream> response) throws Exception
{
try(InputStream stream = response.body())
{
Map<String, String> headers = new TreeMap<>();
response.headers().map().forEach((name, values) -> headers.put(name.toLowerCase(Locale.ROOT), values.getFirst()));
return (new Response(response.statusCode(), headers, stream.readNBytes(MAX_RESPONSE_BYTES)));
}
}
/*******************************************************************************
** An Authorization header value for HTTP Basic authentication.
*******************************************************************************/
public static String basic(String user, String password)
{
return ("Basic " + Base64.getEncoder().encodeToString((user + ":" + password).getBytes(StandardCharsets.UTF_8)));
}
/*******************************************************************************
** TLS is required except on loopback, which the tests and local peers use.
*******************************************************************************/
public static boolean isAllowedUrl(String url)
{
try
{
URI uri = URI.create(url);
return (uri.getHost() != null && uri.getUserInfo() == null
&& ("https".equals(uri.getScheme()) || "http".equals(uri.getScheme()) && List.of("127.0.0.1", "localhost", "[::1]").contains(uri.getHost())));
}
catch(Exception e)
{
return (false);
}
}
}
src/main/java/com/kingsrook/qbits/edi/as2/As2Mdn.java
+261
View File
@@ -0,0 +1,261 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi.as2;
import java.io.ByteArrayInputStream;
import java.nio.charset.StandardCharsets;
import java.security.cert.X509Certificate;
import java.util.Date;
import java.util.List;
import com.helger.as2lib.crypto.ECryptoAlgorithmSign;
import com.helger.as2lib.crypto.MIC;
import com.kingsrook.qbits.edi.as2.CertificateService.LocalKey;
import jakarta.mail.internet.InternetHeaders;
import jakarta.mail.internet.MimeBodyPart;
import jakarta.mail.internet.MimeMultipart;
import jakarta.mail.internet.MimeUtility;
import jakarta.mail.util.ByteArrayDataSource;
/*******************************************************************************
** Builds and parses AS2 message disposition notifications (RFC 4130 section 7,
** RFC 8098): multipart/report with a message/disposition-notification part,
** optionally wrapped in multipart/signed. Memory only.
*******************************************************************************/
public final class As2Mdn
{
public static final String PROCESSED = "automatic-action/MDN-sent-automatically; processed";
/*******************************************************************************
** A receipt ready to send or store: HTTP-level parts and the complete MIME
** entity (headers and body) kept as the stored raw MDN.
*******************************************************************************/
public record Built(String messageId, As2Crypto.Wire wire, byte[] entity, Boolean signed)
{
}
/*******************************************************************************
** A receipt read from a partner. Fields come from the disposition notification;
** signed and signatureValid describe the signature, when there was one.
*******************************************************************************/
public record Parsed(String originalMessageId, String disposition, String micText, MIC mic, Boolean signed, Boolean signatureValid)
{
/*******************************************************************************
** Success means the partner processed the message (a warning still counts).
*******************************************************************************/
public boolean processed()
{
if(disposition == null)
{
return (false);
}
int separator = disposition.indexOf(';');
String status = (separator < 0 ? disposition : disposition.substring(separator + 1)).trim().toLowerCase(java.util.Locale.ROOT);
return (status.equals("processed") || status.startsWith("processed/warning"));
}
}
/*******************************************************************************
** Utility class.
*******************************************************************************/
private As2Mdn()
{
}
/*******************************************************************************
** Disposition text for a failure, e.g. processed/error: authentication-failed.
*******************************************************************************/
public static String failure(String code)
{
return ("automatic-action/MDN-sent-automatically; processed/error: " + code);
}
/*******************************************************************************
** Normalize an AS2 identifier or Message-ID: trim, drop angle brackets and
** quotes.
*******************************************************************************/
public static String normalizeId(String value)
{
if(value == null)
{
return (null);
}
String id = value.trim();
while(id.length() >= 2 && ((id.startsWith("<") && id.endsWith(">")) || (id.startsWith("\"") && id.endsWith("\""))))
{
id = id.substring(1, id.length() - 1).trim();
}
return (id);
}
/*******************************************************************************
** Keep only characters that are safe to show and store from a remote string.
*******************************************************************************/
public static String sanitize(String value)
{
if(value == null)
{
return (null);
}
String clean = value.replaceAll("[^A-Za-z0-9 /;:._,=+@<>-]", "");
return (clean.length() <= 200 ? clean : clean.substring(0, 200));
}
/*******************************************************************************
** Build a receipt. The MIC is included for processed messages; a signing key
** and algorithm produce a signed receipt.
*******************************************************************************/
public static Built build(String messageId, String ourId, String originalMessageId, String disposition, MIC mic, LocalKey key, ECryptoAlgorithmSign algorithm) throws As2Exception
{
try
{
String fields = "Reporting-UA: qbit-edi\r\nOriginal-Recipient: rfc822; " + ourId + "\r\nFinal-Recipient: rfc822; " + ourId
+ "\r\nOriginal-Message-ID: <" + originalMessageId + ">\r\nDisposition: " + disposition + "\r\n"
+ (mic == null ? "" : "Received-Content-MIC: " + mic.getAsAS2String() + "\r\n");
MimeMultipart report = new MimeMultipart("report; report-type=disposition-notification");
MimeBodyPart text = new MimeBodyPart();
text.setText("The AS2 message " + originalMessageId + " was processed with disposition: " + disposition + "\r\n", "us-ascii");
text.setHeader("Content-Transfer-Encoding", "7bit");
report.addBodyPart(text);
InternetHeaders headers = new InternetHeaders();
headers.addHeader("Content-Type", "message/disposition-notification");
headers.addHeader("Content-Transfer-Encoding", "7bit");
report.addBodyPart(new MimeBodyPart(headers, fields.getBytes(StandardCharsets.US_ASCII)));
MimeBodyPart outer = new MimeBodyPart();
outer.setContent(report);
outer.setHeader("Content-Type", report.getContentType());
MimeBodyPart result = key == null ? outer : As2Crypto.sign(outer, key, algorithm);
byte[] entity = As2Crypto.toBytes(result);
return (new Built(messageId, As2Crypto.splitRaw(entity), entity, key != null));
}
catch(As2Exception e)
{
throw (e);
}
catch(Exception e)
{
throw (new As2Exception("The MDN could not be built", As2Exception.UNEXPECTED_PROCESSING));
}
}
/*******************************************************************************
** Parse a receipt. A signature is verified against the candidate certificates
** and reported in signatureValid; an invalid one still yields the readable
** content so the failure can be recorded, but callers must not trust it.
*******************************************************************************/
public static Parsed parse(String contentType, String transferEncoding, byte[] body, List<X509Certificate> certificates, Date now) throws As2Exception
{
MimeBodyPart part = As2Crypto.fromHttp(contentType, transferEncoding, null, body);
boolean signed = false;
Boolean valid = null;
try
{
MimeBodyPart content = part;
if(part.isMimeType("multipart/signed"))
{
signed = true;
try
{
content = As2Crypto.verify(part, certificates, now);
valid = true;
}
catch(As2Exception e)
{
valid = false;
content = (MimeBodyPart) new MimeMultipart(new ByteArrayDataSource(part.getRawInputStream().readAllBytes(), MimeUtility.unfold(part.getContentType()))).getBodyPart(0);
}
}
if(!content.isMimeType("multipart/report"))
{
throw (new As2Exception("The response is not a message disposition notification"));
}
MimeMultipart report = new MimeMultipart(new ByteArrayDataSource(content.getRawInputStream().readAllBytes(), MimeUtility.unfold(content.getContentType())));
for(int i = 0; i < report.getCount(); i++)
{
if(report.getBodyPart(i).isMimeType("message/disposition-notification"))
{
InternetHeaders fields = new InternetHeaders(new ByteArrayInputStream(report.getBodyPart(i).getInputStream().readAllBytes()));
return (fromFields(fields, signed, valid));
}
}
throw (new As2Exception("The response has no disposition notification"));
}
catch(As2Exception e)
{
throw (e);
}
catch(Exception e)
{
throw (new As2Exception("The response is not a readable message disposition notification"));
}
}
/*******************************************************************************
** Read the notification fields.
*******************************************************************************/
private static Parsed fromFields(InternetHeaders fields, boolean signed, Boolean valid)
{
String original = normalizeId(first(fields, "Original-Message-ID"));
String disposition = sanitize(first(fields, "Disposition"));
String micText = sanitize(first(fields, "Received-Content-MIC"));
MIC mic = null;
try
{
mic = MIC.parse(micText);
}
catch(Exception e)
{
// An unreadable MIC is reported as a mismatch by the caller.
mic = null;
}
return (new Parsed(original == null ? null : sanitize(original), disposition, micText, mic, signed, valid));
}
/*******************************************************************************
** First value of a notification field, unfolded, or null.
*******************************************************************************/
private static String first(InternetHeaders fields, String name)
{
String[] values = fields.getHeader(name);
return (values == null || values.length == 0 ? null : MimeUtility.unfold(values[0]).trim());
}
}
src/main/java/com/kingsrook/qbits/edi/as2/As2MdnReceiver.java
+163
View File
@@ -0,0 +1,163 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi.as2;
import java.security.cert.X509Certificate;
import java.util.Date;
import java.util.List;
import java.util.Map;
import com.helger.as2lib.crypto.MIC;
import com.kingsrook.qbits.edi.EdiRuntime;
import com.kingsrook.qbits.edi.flow.EdiJournal;
import com.kingsrook.qbits.edi.model.EdiTransport;
import com.kingsrook.qbits.edi.storage.EdiRecords;
import com.kingsrook.qbits.edi.storage.PayloadRef;
import com.kingsrook.qbits.edi.support.SystemWrites;
import com.kingsrook.qqq.backend.core.alerts.AlertSeverity;
import com.kingsrook.qqq.backend.core.alerts.AlertSignal;
import com.kingsrook.qqq.backend.core.alerts.QAlerts;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** Receives asynchronous MDNs for messages we sent: parses the receipt, verifies
** its signature against the partner certificates, matches Original-Message-ID to
** an outbound message, compares the MIC, then marks the message DELIVERED or
** FAILED, stores the raw MDN, writes a timeline event and resolves the overdue
** alert. Callers run inside the system context.
*******************************************************************************/
public class As2MdnReceiver
{
private static final QLogger LOG = QLogger.getLogger(As2MdnReceiver.class);
/*******************************************************************************
** Handle one inbound MDN; header names are lower case. A receipt that cannot be
** authenticated changes nothing.
*******************************************************************************/
public As2Receiver.Result receive(EdiTransport transport, Map<String, String> headers, byte[] body) throws Exception
{
String from = As2Mdn.normalizeId(headers.get("as2-from"));
String to = As2Mdn.normalizeId(headers.get("as2-to"));
if(from == null || to == null || !from.equalsIgnoreCase(transport.getAs2PartnerId()) || !to.equalsIgnoreCase(transport.getAs2LocalId()))
{
LOG.warn("AS2 MDN refused for transport " + transport.getName() + ": unknown AS2 identity");
return (As2Receiver.Result.text(403, "Unknown AS2 identity"));
}
As2Mdn.Parsed mdn;
try
{
List<X509Certificate> certificates = transport.getPartnerCertificateId() == null ? List.of() : new CertificateService().partnerVerificationCertificates(transport);
mdn = As2Mdn.parse(headers.get("content-type"), headers.get("content-transfer-encoding"), body, certificates, Date.from(EdiRuntime.current().config().getClock().instant()));
}
catch(As2Exception e)
{
return (As2Receiver.Result.text(400, "The MDN could not be read"));
}
if((Boolean.TRUE.equals(mdn.signed()) && !Boolean.TRUE.equals(mdn.signatureValid())) || (Boolean.TRUE.equals(transport.getMdnSigned()) && !Boolean.TRUE.equals(mdn.signed())))
{
LOG.warn("AS2 MDN refused for transport " + transport.getName() + ": signature check failed");
return (As2Receiver.Result.text(401, "The MDN signature could not be verified"));
}
List<QRecord> messages = mdn.originalMessageId() == null ? List.of() : EdiRecords.query("edi_outbound_message", EdiRecords.equal("transportMessageId", mdn.originalMessageId())
.withCriteria("transportId", QCriteriaOperator.EQUALS, transport.getId()));
if(messages.isEmpty())
{
return (As2Receiver.Result.text(404, "No message was sent with that Original-Message-ID"));
}
String mdnId = As2Mdn.sanitize(As2Mdn.normalizeId(headers.get("message-id")));
if(mdnId != null && !EdiRecords.query(As2Store.TABLE, EdiRecords.equal("direction", "INBOUND").withCriteria("messageId", QCriteriaOperator.EQUALS, mdnId)
.withCriteria("originalMessageId", QCriteriaOperator.EQUALS, mdn.originalMessageId())).isEmpty())
{
return (new As2Receiver.Result(200, Map.of(), new byte[0]));
}
settle(transport, messages.getFirst(), mdn, mdnId, headers, body);
return (new As2Receiver.Result(200, Map.of(), new byte[0]));
}
/*******************************************************************************
** Record the receipt and settle the message it answers.
*******************************************************************************/
private void settle(EdiTransport transport, QRecord message, As2Mdn.Parsed mdn, String mdnId, Map<String, String> headers, byte[] body) throws Exception
{
MIC ours = null;
try
{
ours = MIC.parse(message.getValueString("transportMic"));
}
catch(Exception e)
{
LOG.warn("The MIC stored for outbound message " + message.getValue("id") + " could not be read");
}
String failure = As2Transport.judge(transport, mdn, ours);
byte[] raw = As2Crypto.entity(headers.get("content-type"), headers.get("content-transfer-encoding"), null, body);
SystemWrites.transaction(() ->
{
PayloadRef ref = As2Store.storeRaw(raw);
As2Store.insert(new QRecord().withValue("direction", "INBOUND").withValue("messageId", mdnId).withValue("originalMessageId", mdn.originalMessageId())
.withValue("disposition", mdn.disposition()).withValue("micAlgorithm", mdn.mic() == null ? null : mdn.mic().getDigestAlgorithm().getID())
.withValue("micValue", As2Store.micValue(mdn.mic())).withValue("signed", mdn.signed()).withValue("signatureValid", mdn.signatureValid()).withValue("payloadRef", ref.value())
.withValue("transportId", transport.getId()).withValue("partnerId", message.getValue("partnerId")).withValue("outboundMessageId", message.getValue("id")).withValue("deliveryStatus", "NONE"));
Integer interchangeId = message.getValueInteger("interchangeId");
EdiJournal.event(null, interchangeId, null, "MDN");
if(SystemWrites.guard("UPDATE edi_outbound_message SET id=id WHERE id=? AND status='SENT'", message.getValue("id")))
{
var now = EdiRuntime.current().config().getClock().instant();
String status = failure == null ? "DELIVERED" : "FAILED";
QRecord update = new QRecord().withValue("id", message.getValue("id")).withValue("status", status).withValue("lastError", failure == null ? null : truncate(failure));
if(failure == null)
{
update.withValue("deliveredAt", now);
}
SystemWrites.update("edi_outbound_message", update);
if(interchangeId != null)
{
SystemWrites.update("edi_interchange", new QRecord().withValue("id", interchangeId).withValue("status", status));
}
EdiJournal.event(null, interchangeId, null, status);
if(failure != null)
{
EdiJournal.exception(null, interchangeId, null, message.getValueInteger("partnerId"), "TRANSPORT", "Outbound delivery failed");
QAlerts.raise(new AlertSignal("edi.send.failed").withSeverity(AlertSeverity.ERROR).withDedupKey("edi_outbound_message:" + message.getValue("id"))
.withTitle("EDI delivery failed").withSummary("Review the outbound message"));
}
}
return (null);
});
QAlerts.resolve("edi.mdn.overdue", "edi_outbound_message:" + message.getValue("id"), "MDN received");
}
/*******************************************************************************
** Keep an error within its column.
*******************************************************************************/
private static String truncate(String text)
{
return (text.length() <= 255 ? text : text.substring(0, 255));
}
}
src/main/java/com/kingsrook/qbits/edi/as2/As2MdnSender.java
+158
View File
@@ -0,0 +1,158 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi.as2;
import java.io.InputStream;
import java.net.URI;
import java.net.http.HttpClient;
import java.time.Duration;
import java.time.Instant;
import java.util.LinkedHashMap;
import java.util.Map;
import java.util.Objects;
import com.kingsrook.qbits.edi.EdiRuntime;
import com.kingsrook.qbits.edi.flow.OutboundWorker;
import com.kingsrook.qbits.edi.flow.SendWorker;
import com.kingsrook.qbits.edi.model.EdiTransport;
import com.kingsrook.qbits.edi.storage.EdiRecords;
import com.kingsrook.qbits.edi.storage.PayloadRef;
import com.kingsrook.qbits.edi.support.SystemWrites;
import com.kingsrook.qqq.backend.core.alerts.AlertSeverity;
import com.kingsrook.qqq.backend.core.alerts.AlertSignal;
import com.kingsrook.qqq.backend.core.alerts.QAlerts;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** Delivers the asynchronous MDNs we owe partners. The edi_as2_mdn row is the
** queue entry (deliveryStatus PENDING): a scheduled step posts the stored MDN to
** the address the partner gave in Receipt-Delivery-Option, retries with the send
** backoff, and fails (with an alert) after the transport maxAttempts. Row locks
** fence concurrent senders.
*******************************************************************************/
public class As2MdnSender
{
private static final QLogger LOG = QLogger.getLogger(As2MdnSender.class);
/*******************************************************************************
** Deliver pending MDNs that are due; returns how many were attempted.
*******************************************************************************/
public Integer runOnce() throws QException
{
return (SystemWrites.call(() ->
{
int count = 0;
var due = new QQueryFilter().withBooleanOperator(QQueryFilter.BooleanOperator.OR).withCriteria("nextAttemptAt", QCriteriaOperator.IS_BLANK)
.withCriteria("nextAttemptAt", QCriteriaOperator.LESS_THAN_OR_EQUALS, EdiRuntime.current().config().getClock().instant());
for(QRecord row : EdiRecords.query(As2Store.TABLE, new QQueryFilter().withCriteria("deliveryStatus", QCriteriaOperator.EQUALS, "PENDING").withSubFilter(due).withLimit(20)))
{
count += deliver(row);
}
return (count);
}));
}
/*******************************************************************************
** One delivery attempt, fenced by a row lock on the selected state.
*******************************************************************************/
private Integer deliver(QRecord selected) throws QException
{
return (SystemWrites.transaction(() ->
{
if(!SystemWrites.guard("UPDATE edi_as2_mdn SET id=id WHERE id=? AND delivery_status='PENDING' AND COALESCE(delivery_attempts,0)=?", selected.getValue("id"),
Objects.requireNonNullElse(selected.getValueInteger("deliveryAttempts"), 0)))
{
return (0);
}
QRecord row = OutboundWorker.row(As2Store.TABLE, selected.getValue("id"));
EdiTransport transport = OutboundWorker.row("edi_transport", row.getValue("transportId")).toEntity(EdiTransport.class);
int attempts = Objects.requireNonNullElse(row.getValueInteger("deliveryAttempts"), 0) + 1;
boolean delivered = post(row, transport);
int maximum = Objects.requireNonNullElse(transport.getMaxAttempts(), 8);
Instant now = EdiRuntime.current().config().getClock().instant();
String status = delivered ? "SENT" : attempts >= maximum ? "FAILED" : "PENDING";
QRecord update = new QRecord().withValue("id", row.getValue("id")).withValue("deliveryStatus", status).withValue("deliveryAttempts", attempts)
.withValue("nextAttemptAt", "PENDING".equals(status) ? now.plusSeconds(SendWorker.BACKOFF.get(Math.min(attempts - 1, SendWorker.BACKOFF.size() - 1))) : null);
if(delivered)
{
update.withValue("deliveredAt", now);
}
SystemWrites.update(As2Store.TABLE, update);
if("FAILED".equals(status))
{
QAlerts.raise(new AlertSignal("edi.send.failed").withSeverity(AlertSeverity.ERROR).withDedupKey("edi_as2_mdn:" + row.getValue("id"))
.withTitle("AS2 receipt delivery failed").withSummary("Review the AS2 receipt"));
}
return (1);
}));
}
/*******************************************************************************
** POST the stored MDN to the partner receipt address; true on any 2xx.
*******************************************************************************/
private boolean post(QRecord row, EdiTransport transport)
{
try
{
byte[] raw;
try(InputStream stream = EdiRuntime.current().payloads().open(new PayloadRef(row.getValueString("payloadRef"), null, 0)))
{
raw = stream.readAllBytes();
}
As2Crypto.Wire wire = As2Crypto.splitRaw(raw);
Map<String, String> headers = new LinkedHashMap<>();
headers.put("AS2-Version", "1.2");
headers.put("AS2-From", As2Transport.asId(transport.getAs2LocalId()));
headers.put("AS2-To", As2Transport.asId(transport.getAs2PartnerId()));
headers.put("Message-ID", "<" + row.getValueString("messageId") + ">");
headers.put("MIME-Version", "1.0");
headers.put("Content-Type", wire.contentType());
if(wire.transferEncoding() != null)
{
headers.put("Content-Transfer-Encoding", wire.transferEncoding());
}
if(transport.getHttpAuthUserEnvVar() != null)
{
var secrets = EdiRuntime.current().config().getSecretsResolver();
headers.put("Authorization", As2Http.basic(secrets.resolve(transport.getHttpAuthUserEnvVar()), secrets.resolve(transport.getHttpAuthPasswordEnvVar())));
}
HttpClient client = As2Http.client(null);
int status = As2Http.post(client, URI.create(row.getValueString("deliveryUrl")), headers, wire.body(), Duration.ofSeconds(60)).status();
return (status >= 200 && status < 300);
}
catch(Exception e)
{
LOG.warn("AS2 receipt delivery attempt failed for receipt " + row.getValue("id"));
return (false);
}
}
}
src/main/java/com/kingsrook/qbits/edi/as2/As2Receiver.java
+356
View File
@@ -0,0 +1,356 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi.as2;
import java.io.ByteArrayInputStream;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.security.cert.X509Certificate;
import java.time.Instant;
import java.util.Date;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.UUID;
import com.helger.as2lib.crypto.ECryptoAlgorithmSign;
import com.helger.as2lib.crypto.MIC;
import com.helger.as2lib.disposition.DispositionOptions;
import com.kingsrook.qbits.edi.EdiRuntime;
import com.kingsrook.qbits.edi.as2.CertificateService.LocalKey;
import com.kingsrook.qbits.edi.flow.EdiJournal;
import com.kingsrook.qbits.edi.flow.InboundPayload;
import com.kingsrook.qbits.edi.flow.InboundSink;
import com.kingsrook.qbits.edi.model.EdiTransport;
import com.kingsrook.qbits.edi.storage.EdiRecords;
import com.kingsrook.qbits.edi.storage.PayloadRef;
import com.kingsrook.qbits.edi.support.SystemWrites;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import jakarta.mail.internet.MimeBodyPart;
/*******************************************************************************
** Receives AS2 messages for one transport: checks the AS2 identities, decrypts
** with our ACTIVE or NEXT keys, verifies the signature against the partner
** certificates, computes the MIC, hands the document to the inbox, and answers
** with a synchronous MDN or queues an asynchronous one. A retransmitted
** Message-ID gets the original MDN again and is not reprocessed. Callers run
** inside the system context; failures are reported as RFC 4130 dispositions.
*******************************************************************************/
public class As2Receiver
{
private static final QLogger LOG = QLogger.getLogger(As2Receiver.class);
/*******************************************************************************
** An HTTP answer: status, headers and body.
*******************************************************************************/
public record Result(int status, Map<String, String> headers, byte[] body)
{
/*******************************************************************************
** A plain-text answer.
*******************************************************************************/
static Result text(int status, String text)
{
return (new Result(status, Map.of("Content-Type", "text/plain; charset=utf-8"), text.getBytes(StandardCharsets.UTF_8)));
}
}
/*******************************************************************************
** What was asked of us for this message.
*******************************************************************************/
private record Request(String messageId, boolean mdnRequested, String deliveryUrl, boolean signMdn, ECryptoAlgorithmSign micAlgorithm)
{
/*******************************************************************************
** The same request, answered in the HTTP response only. Used before the sender
** is authenticated, so an unauthenticated sender cannot aim a receipt at an
** address of its choosing.
*******************************************************************************/
Request synchronous()
{
return (new Request(messageId, mdnRequested, null, signMdn, micAlgorithm));
}
}
/*******************************************************************************
** Handle one inbound message; header names are lower case.
*******************************************************************************/
public Result receive(EdiTransport transport, Map<String, String> headers, byte[] body, String remoteAddress) throws Exception
{
String messageId = As2Mdn.normalizeId(headers.get("message-id"));
String from = As2Mdn.normalizeId(headers.get("as2-from"));
String to = As2Mdn.normalizeId(headers.get("as2-to"));
if(messageId == null || messageId.isBlank() || messageId.length() > 250 || !messageId.matches("[\\x21-\\x7E]+") || from == null || to == null)
{
return (Result.text(400, "The AS2-From, AS2-To and Message-ID headers are required"));
}
if(!from.equalsIgnoreCase(transport.getAs2PartnerId()) || !to.equalsIgnoreCase(transport.getAs2LocalId()))
{
LOG.warn("AS2 message refused for transport " + transport.getName() + ": unknown AS2 identity");
return (Result.text(403, "Unknown AS2 identity"));
}
Request request = request(transport, headers, messageId);
if(request.deliveryUrl() != null && !deliveryUrlAllowed(transport, request.deliveryUrl()))
{
return (Result.text(400, "The Receipt-Delivery-Option is not acceptable"));
}
CertificateService certificates = new CertificateService();
CertificateService.KeySet keys = certificates.localDecryptionKeys();
As2Crypto.Decoded decoded;
try
{
List<X509Certificate> partnerCertificates = transport.getPartnerCertificateId() == null ? List.of() : certificates.partnerVerificationCertificates(transport);
byte[] top = As2Crypto.entity(headers.get("content-type"), headers.get("content-transfer-encoding"), headers.get("content-disposition"), body);
decoded = As2Crypto.decode(top, keys.keys(), partnerCertificates, request.micAlgorithm(), Date.from(EdiRuntime.current().config().getClock().instant()));
if((transport.getSignAlgorithm() != null && !decoded.signed()) || (transport.getEncryptAlgorithm() != null && !decoded.encrypted()))
{
throw (new As2Exception("The message does not meet the transport security requirements", As2Exception.INSUFFICIENT_SECURITY));
}
}
catch(As2Exception e)
{
boolean ourFault = e.getCode() == null || (As2Exception.DECRYPTION_FAILED.equals(e.getCode()) && !keys.problems().isEmpty());
return (reject(transport, request.synchronous(), ourFault ? As2Exception.UNEXPECTED_PROCESSING : e.getCode(), certificates));
}
String externalId = externalId(transport, messageId);
QRecord replay = alreadyAnswered(transport, request, externalId);
if(replay != null)
{
return (replay(transport, request, replay));
}
QRecord inbox;
try
{
inbox = new InboundSink(EdiRuntime.current().payloads()).receive(new InboundPayload("AS2", externalId, decoded.fileName(), remoteAddress, new ByteArrayInputStream(decoded.payload())));
if(inbox.getValue("transportId") == null)
{
SystemWrites.update("edi_inbox", new QRecord().withValue("id", inbox.getValue("id")).withValue("transportId", transport.getId()));
}
}
catch(Exception e)
{
LOG.warn("AS2 intake failed for transport " + transport.getName());
return (reject(transport, request, As2Exception.UNEXPECTED_PROCESSING, certificates));
}
if(!request.mdnRequested())
{
return (new Result(200, Map.of(), new byte[0]));
}
try
{
return (answer(transport, request, As2Mdn.PROCESSED, decoded.mic(), inbox.getValueInteger("id"), certificates));
}
catch(As2Exception e)
{
LOG.warn("AS2 receipt could not be produced for transport " + transport.getName() + ": " + e.getMessage());
return (Result.text(500, "The receipt could not be produced"));
}
}
/*******************************************************************************
** Read what the message asks for: whether a receipt is wanted, where an
** asynchronous one goes, and whether and how to sign it.
*******************************************************************************/
private Request request(EdiTransport transport, Map<String, String> headers, String messageId)
{
DispositionOptions options = new DispositionOptions();
try
{
options = DispositionOptions.createFromString(headers.get("disposition-notification-options"));
}
catch(Exception e)
{
LOG.warn("AS2 disposition options ignored for transport " + transport.getName());
}
ECryptoAlgorithmSign algorithm = options.getFirstMICAlg();
if(algorithm == null)
{
algorithm = As2Crypto.signAlgorithm(transport.getSignAlgorithm()) == null ? ECryptoAlgorithmSign.DIGEST_SHA_256 : As2Crypto.signAlgorithm(transport.getSignAlgorithm());
}
String delivery = headers.get("receipt-delivery-option");
return (new Request(messageId, headers.get("disposition-notification-to") != null, delivery == null || delivery.isBlank() ? null : delivery.trim(),
options.getProtocol() != null || Boolean.TRUE.equals(transport.getMdnSigned()), algorithm));
}
/*******************************************************************************
** An asynchronous receipt address must use TLS (loopback excepted) and, unless
** the transport requires signed messages, must be on the partner's own host,
** so an unauthenticated sender cannot aim our receipts at third parties.
*******************************************************************************/
static boolean deliveryUrlAllowed(EdiTransport transport, String url)
{
if(!As2Http.isAllowedUrl(url))
{
return (false);
}
if(transport.getSignAlgorithm() != null)
{
return (true);
}
try
{
return (transport.getAs2Url() != null && java.net.URI.create(url).getHost().equalsIgnoreCase(java.net.URI.create(transport.getAs2Url()).getHost()));
}
catch(Exception e)
{
return (false);
}
}
/*******************************************************************************
** The inbox dedupe key: the Message-ID. Message-IDs are unique per sender, so if
** the same Message-ID was already received on a different transport (another
** partner), the key is scoped by the transport id so neither message is lost.
*******************************************************************************/
private static String externalId(EdiTransport transport, String messageId) throws Exception
{
List<QRecord> rows = EdiRecords.query("edi_inbox", EdiRecords.equal("transportType", "AS2").withCriteria("externalId", QCriteriaOperator.EQUALS, messageId));
boolean foreign = !rows.isEmpty() && rows.getFirst().getValue("transportId") != null && !rows.getFirst().getValue("transportId").equals(transport.getId());
return (foreign ? transport.getId() + ":" + messageId : messageId);
}
/*******************************************************************************
** The receipt row we already sent for this Message-ID (a retransmission), or
** null when the original was never answered.
*******************************************************************************/
private QRecord alreadyAnswered(EdiTransport transport, Request request, String externalId) throws Exception
{
if(EdiRecords.query("edi_inbox", EdiRecords.equal("transportType", "AS2").withCriteria("externalId", QCriteriaOperator.EQUALS, externalId)).isEmpty())
{
return (null);
}
List<QRecord> rows = EdiRecords.query(As2Store.TABLE, EdiRecords.equal("direction", "OUTBOUND").withCriteria("originalMessageId", QCriteriaOperator.EQUALS, request.messageId())
.withCriteria("transportId", QCriteriaOperator.EQUALS, transport.getId()));
return (rows.isEmpty() ? null : rows.getFirst());
}
/*******************************************************************************
** Answer a retransmission with the original receipt, unchanged. An
** asynchronous receipt is queued to be delivered again.
*******************************************************************************/
private Result replay(EdiTransport transport, Request request, QRecord original) throws Exception
{
if(!request.mdnRequested())
{
return (new Result(200, Map.of(), new byte[0]));
}
if(request.deliveryUrl() != null)
{
SystemWrites.update(As2Store.TABLE, new QRecord().withValue("id", original.getValue("id")).withValue("deliveryUrl", request.deliveryUrl()).withValue("deliveryStatus", "PENDING")
.withValue("deliveryAttempts", 0).withValue("nextAttemptAt", null));
return (new Result(200, Map.of(), new byte[0]));
}
byte[] raw;
try(InputStream stream = EdiRuntime.current().payloads().open(new PayloadRef(original.getValueString("payloadRef"), null, 0)))
{
raw = stream.readAllBytes();
}
return (syncResult(transport, original.getValueString("messageId"), As2Crypto.splitRaw(raw)));
}
/*******************************************************************************
** Reject a message that could not be accepted. With a receipt requested the
** failure is reported in an MDN so the partner can act or retry; otherwise an
** HTTP error answers. No inbox row exists for a rejected message.
*******************************************************************************/
private Result reject(EdiTransport transport, Request request, String code, CertificateService certificates) throws Exception
{
LOG.warn("AS2 message rejected for transport " + transport.getName() + ": " + code);
if(!request.mdnRequested())
{
return (Result.text(As2Exception.UNEXPECTED_PROCESSING.equals(code) ? 500 : 400, "AS2 message rejected: " + code));
}
try
{
return (answer(transport, request, As2Mdn.failure(code), null, null, certificates));
}
catch(As2Exception e)
{
LOG.warn("AS2 receipt could not be produced for transport " + transport.getName() + ": " + e.getMessage());
return (Result.text(500, "The receipt could not be produced"));
}
}
/*******************************************************************************
** Build, store and answer or queue the MDN.
*******************************************************************************/
private Result answer(EdiTransport transport, Request request, String disposition, MIC mic, Integer inboxId, CertificateService certificates) throws Exception
{
LocalKey key = request.signMdn() ? certificates.activeLocal(transport) : null;
String mdnId = "edi-mdn-" + UUID.randomUUID() + "@" + transport.getAs2LocalId().replaceAll("[^A-Za-z0-9.-]", "-");
As2Mdn.Built built = As2Mdn.build(mdnId, transport.getAs2LocalId(), request.messageId(), disposition, mic, key, request.micAlgorithm());
PayloadRef raw = As2Store.storeRaw(built.entity());
Instant now = EdiRuntime.current().config().getClock().instant();
QRecord row = As2Store.insert(new QRecord().withValue("direction", "OUTBOUND").withValue("messageId", mdnId).withValue("originalMessageId", request.messageId())
.withValue("disposition", disposition).withValue("micAlgorithm", mic == null ? null : mic.getDigestAlgorithm().getID()).withValue("micValue", As2Store.micValue(mic))
.withValue("signed", built.signed()).withValue("payloadRef", raw.value()).withValue("receivedAt", now).withValue("transportId", transport.getId())
.withValue("partnerId", transport.getPartnerId()).withValue("inboxId", inboxId).withValue("deliveryStatus", request.deliveryUrl() == null ? "NONE" : "PENDING")
.withValue("deliveryUrl", request.deliveryUrl()).withValue("deliveryAttempts", 0).withValue("nextAttemptAt", null));
if(inboxId != null)
{
EdiJournal.event(inboxId, null, null, "MDN");
}
LOG.info("AS2 receipt " + row.getValue("id") + " " + (request.deliveryUrl() == null ? "returned" : "queued") + " for transport " + transport.getName());
return (request.deliveryUrl() != null ? new Result(200, Map.of(), new byte[0]) : syncResult(transport, mdnId, built.wire()));
}
/*******************************************************************************
** The synchronous MDN response with its AS2 headers.
*******************************************************************************/
private Result syncResult(EdiTransport transport, String mdnId, As2Crypto.Wire wire)
{
Map<String, String> headers = new LinkedHashMap<>();
headers.put("AS2-Version", "1.2");
headers.put("AS2-From", As2Transport.asId(transport.getAs2LocalId()));
headers.put("AS2-To", As2Transport.asId(transport.getAs2PartnerId()));
headers.put("Message-ID", "<" + mdnId + ">");
headers.put("MIME-Version", "1.0");
headers.put("Content-Type", wire.contentType());
if(wire.transferEncoding() != null)
{
headers.put("Content-Transfer-Encoding", wire.transferEncoding());
}
return (new Result(200, headers, wire.body()));
}
}
src/main/java/com/kingsrook/qbits/edi/as2/As2RouteProvider.java
+148
View File
@@ -0,0 +1,148 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi.as2;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.TreeMap;
import com.kingsrook.qbits.edi.model.EdiTransport;
import com.kingsrook.qbits.edi.storage.EdiRecords;
import com.kingsrook.qbits.edi.support.SystemWrites;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.middleware.javalin.QJavalinRouteProviderInterface;
import io.javalin.config.JavalinConfig;
import io.javalin.http.Context;
/*******************************************************************************
** AS2 machine endpoints outside the UI API, authenticated per transport by AS2
** identity, S/MIME signature and certificates (never a user session):
** POST /qqq/hooks/edi/as2/{transport} receives messages and answers with the
** MDN, and POST /qqq/hooks/edi/as2/{transport}/mdn receives asynchronous MDNs.
*******************************************************************************/
public class As2RouteProvider implements QJavalinRouteProviderInterface
{
private static final QLogger LOG = QLogger.getLogger(As2RouteProvider.class);
public static final long DEFAULT_MAX_BODY_BYTES = 50L * 1024 * 1024;
public static final long ABSOLUTE_MAX_BODY_BYTES = 200L * 1024 * 1024;
private QInstance instance;
/*******************************************************************************
** Host instance supplied by Javalin.
*******************************************************************************/
public void setQInstance(QInstance instance)
{
this.instance = instance;
}
/*******************************************************************************
** Register the non-UI hooks.
*******************************************************************************/
public void acceptJavalinConfig(JavalinConfig config)
{
config.routes.post("/qqq/hooks/edi/as2/{transport}", context -> handle(context, false));
config.routes.post("/qqq/hooks/edi/as2/{transport}/mdn", context -> handle(context, true));
}
/*******************************************************************************
** Find the active AS2 transport, read the bounded body and delegate.
*******************************************************************************/
private void handle(Context context, boolean mdn)
{
var previous = QContext.capture();
try
{
QContext.init(instance, null);
As2Receiver.Result result = SystemWrites.call(() -> process(context, mdn));
context.status(result.status());
result.headers().forEach(context::header);
context.result(result.body());
}
catch(Exception e)
{
LOG.warn("AS2 hook failed: " + e.getClass().getSimpleName());
context.status(503);
context.result("The AS2 request could not be processed".getBytes(java.nio.charset.StandardCharsets.UTF_8));
}
finally
{
QContext.init(previous);
}
}
/*******************************************************************************
** The hook body, inside the system context.
*******************************************************************************/
private As2Receiver.Result process(Context context, boolean mdn) throws Exception
{
List<QRecord> rows = EdiRecords.query("edi_transport", EdiRecords.equal("name", context.pathParam("transport")).withCriteria("type", QCriteriaOperator.EQUALS, "AS2")
.withCriteria("isActive", QCriteriaOperator.EQUALS, true));
if(rows.size() != 1 || (mdn ? "INBOUND" : "OUTBOUND").equals(rows.getFirst().getValueString("direction")))
{
return (As2Receiver.Result.text(404, "Unknown AS2 transport"));
}
EdiTransport transport = rows.getFirst().toEntity(EdiTransport.class);
long limit = mdn ? As2Http.MAX_RESPONSE_BYTES : maxBodyBytes(transport);
if(context.req().getContentLengthLong() > limit)
{
return (As2Receiver.Result.text(413, "The message is too large"));
}
byte[] body = context.req().getInputStream().readNBytes((int) limit + 1);
if(body.length > limit)
{
return (As2Receiver.Result.text(413, "The message is too large"));
}
Map<String, String> headers = new TreeMap<>();
context.headerMap().forEach((name, value) -> headers.put(name.toLowerCase(Locale.ROOT), value));
return (mdn ? new As2MdnReceiver().receive(transport, headers, body) : new As2Receiver().receive(transport, headers, body, context.ip()));
}
/*******************************************************************************
** The largest interchange any agreement on this transport allows, or 50 MB.
*******************************************************************************/
private static long maxBodyBytes(EdiTransport transport) throws Exception
{
long limit = 0;
for(QRecord agreement : EdiRecords.query("edi_agreement", EdiRecords.equal("inboundTransportId", transport.getId())))
{
limit = Math.max(limit, agreement.getValueLong("maxInterchangeBytes") == null ? 0 : agreement.getValueLong("maxInterchangeBytes"));
}
return (limit <= 0 ? DEFAULT_MAX_BODY_BYTES : Math.min(limit + limit / 2 + 65536, ABSOLUTE_MAX_BODY_BYTES));
}
}
src/main/java/com/kingsrook/qbits/edi/as2/As2Store.java
+86
View File
@@ -0,0 +1,86 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi.as2;
import java.io.ByteArrayInputStream;
import java.time.Instant;
import com.helger.as2lib.crypto.MIC;
import com.kingsrook.qbits.edi.EdiRuntime;
import com.kingsrook.qbits.edi.storage.PayloadKind;
import com.kingsrook.qbits.edi.storage.PayloadRef;
import com.kingsrook.qbits.edi.support.SystemWrites;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** Persistence helpers for AS2 receipts: the raw MDN goes to the encrypted payload
** store and a read-only edi_as2_mdn row describes it. Callers run inside the
** system context.
*******************************************************************************/
public final class As2Store
{
public static final String TABLE = "edi_as2_mdn";
/*******************************************************************************
** Utility class.
*******************************************************************************/
private As2Store()
{
}
/*******************************************************************************
** Store the raw MDN entity (headers and body) encrypted.
*******************************************************************************/
public static PayloadRef storeRaw(byte[] entity) throws Exception
{
return (EdiRuntime.current().payloads().put(PayloadKind.MDN, new ByteArrayInputStream(entity)));
}
/*******************************************************************************
** The base64 value of a MIC without its algorithm.
*******************************************************************************/
public static String micValue(MIC mic)
{
return (mic == null ? null : java.util.Base64.getEncoder().encodeToString(mic.micBytes()));
}
/*******************************************************************************
** Insert a receipt row and return it.
*******************************************************************************/
public static QRecord insert(QRecord row) throws QException
{
if(row.getValue("receivedAt") == null)
{
row.setValue("receivedAt", Instant.now(EdiRuntime.current().config().getClock()));
}
return (SystemWrites.insert(TABLE, row));
}
}
src/main/java/com/kingsrook/qbits/edi/as2/As2Transport.java
+584
View File
@@ -0,0 +1,584 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi.as2;
import java.io.InputStream;
import java.net.URI;
import java.net.http.HttpClient;
import java.security.MessageDigest;
import java.security.cert.X509Certificate;
import java.time.Duration;
import java.time.ZoneOffset;
import java.time.ZonedDateTime;
import java.time.format.DateTimeFormatter;
import java.util.Date;
import java.util.HexFormat;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import com.helger.as2lib.crypto.ECryptoAlgorithmCrypt;
import com.helger.as2lib.crypto.ECryptoAlgorithmSign;
import com.helger.as2lib.crypto.MIC;
import com.helger.as2lib.disposition.DispositionOptions;
import com.kingsrook.qbits.edi.EdiRuntime;
import com.kingsrook.qbits.edi.as2.CertificateService.LocalKey;
import com.kingsrook.qbits.edi.flow.EdiJournal;
import com.kingsrook.qbits.edi.model.EdiOutboundMessage;
import com.kingsrook.qbits.edi.model.EdiTransport;
import com.kingsrook.qbits.edi.storage.EdiRecords;
import com.kingsrook.qbits.edi.storage.PayloadRef;
import com.kingsrook.qbits.edi.support.SystemWrites;
import com.kingsrook.qbits.edi.transport.EdiSendResult;
import com.kingsrook.qbits.edi.transport.EdiTestResult;
import com.kingsrook.qbits.edi.transport.EdiTransportPlugin;
import com.kingsrook.qqq.backend.core.alerts.QAlerts;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** AS2 (RFC 4130) transport. Sending builds the message (compress, sign, then
** encrypt), POSTs it, and for a synchronous receipt verifies the MDN signature,
** the received-content MIC and the disposition. An asynchronous receipt leaves
** the message SENT until the MDN arrives at the receipt hook. No local files are
** used; keys come from environment variables through the secrets resolver.
*******************************************************************************/
public class As2Transport implements EdiTransportPlugin
{
public static final String TYPE = "AS2";
private static final String ID_PATTERN = "[A-Za-z0-9 ._@:/-]{1,128}";
private static final Duration TIMEOUT = Duration.ofSeconds(60);
/*******************************************************************************
** The prepared wire message and the MIC computed for it.
*******************************************************************************/
private record Prepared(As2Crypto.Wire wire, MIC mic)
{
}
/*******************************************************************************
** Registry key.
*******************************************************************************/
@Override
public String getType()
{
return (TYPE);
}
/*******************************************************************************
** Check identifiers, URL, algorithms, receipt mode and certificate ownership
** without resolving any secret. SHA1 and 3DES are accepted only when set
** explicitly; they set the weakCrypto flag on the record.
*******************************************************************************/
@Override
public void validate(EdiTransport transport, List<String> errors)
{
boolean outbound = !"INBOUND".equals(transport.getDirection());
if(transport.getAs2LocalId() == null || !transport.getAs2LocalId().matches(ID_PATTERN))
{
errors.add("as2LocalId is required (letters, digits, spaces and . _ @ : / - only)");
}
if(transport.getAs2PartnerId() == null || !transport.getAs2PartnerId().matches(ID_PATTERN))
{
errors.add("as2PartnerId is required (letters, digits, spaces and . _ @ : / - only)");
}
if(outbound && (transport.getAs2Url() == null || !As2Http.isAllowedUrl(transport.getAs2Url())))
{
errors.add("as2Url must use TLS (HTTP is allowed on loopback only) and carry no credentials");
}
try
{
As2Crypto.signAlgorithm(transport.getSignAlgorithm());
}
catch(IllegalArgumentException e)
{
errors.add("Unsupported signAlgorithm");
}
try
{
As2Crypto.encryptAlgorithm(transport.getEncryptAlgorithm());
}
catch(IllegalArgumentException e)
{
errors.add("Unsupported encryptAlgorithm");
}
if(transport.getMdnMode() != null && !List.of("SYNC", "ASYNC", "NONE").contains(transport.getMdnMode()))
{
errors.add("Unsupported mdnMode");
}
if(Boolean.TRUE.equals(transport.getCompress()) && transport.getSignAlgorithm() == null && transport.getEncryptAlgorithm() == null)
{
errors.add("compress requires signing or encryption");
}
if(transport.getPartnerId() == null && !Boolean.TRUE.equals(transport.getIsVan()))
{
errors.add("partnerId is required unless the transport is a VAN (isVan)");
}
if((transport.getHttpAuthUserEnvVar() == null) != (transport.getHttpAuthPasswordEnvVar() == null))
{
errors.add("httpAuthUserEnvVar and httpAuthPasswordEnvVar must be set together");
}
if(outbound && "ASYNC".equals(transport.getMdnMode()) && baseUrl() == null)
{
errors.add("ASYNC receipts need the qbit publicBaseUrl configuration");
}
try
{
validateCertificates(transport, errors);
}
catch(Exception e)
{
errors.add("Certificates could not be checked");
}
}
/*******************************************************************************
** Certificate references must exist and have the right owner: ours (no
** partner, with a key variable) as the local certificate; the partner's own, or
** for a VAN a certificate that is not ours, as the partner certificate; a TLS
** certificate for pinning. Retired certificates are refused.
*******************************************************************************/
private static void validateCertificates(EdiTransport transport, List<String> errors) throws Exception
{
boolean secured = transport.getSignAlgorithm() != null || transport.getEncryptAlgorithm() != null || Boolean.TRUE.equals(transport.getMdnSigned());
QRecord local = certificate(transport.getLocalCertificateId());
QRecord partner = certificate(transport.getPartnerCertificateId());
QRecord tls = certificate(transport.getPartnerTlsCertificateId());
if(transport.getLocalCertificateId() == null ? secured : local == null)
{
errors.add(transport.getLocalCertificateId() == null ? "localCertificateId is required when signing or encrypting" : "localCertificateId does not exist");
}
else if(local != null && (local.getValue("partnerId") != null || local.getValueString("privateKeyEnvVar") == null || "RETIRED".equals(local.getValueString("status"))))
{
errors.add("localCertificateId must be one of our own certificates (no partner, with a private key variable) that is not retired");
}
if(transport.getPartnerCertificateId() == null ? secured : partner == null)
{
errors.add(transport.getPartnerCertificateId() == null ? "partnerCertificateId is required when signing or encrypting" : "partnerCertificateId does not exist");
}
else if(partner != null)
{
boolean ownedByPartner = transport.getPartnerId() == null || transport.getPartnerId().equals(partner.getValueInteger("partnerId"));
if(!ownedByPartner || partner.getValueString("privateKeyEnvVar") != null || "RETIRED".equals(partner.getValueString("status")))
{
errors.add("partnerCertificateId must be a certificate of the partner (or, for a VAN, of the VAN) that is not retired and holds no private key");
}
}
if(transport.getPartnerTlsCertificateId() != null && (tls == null || !"TLS".equals(tls.getValueString("usage"))))
{
errors.add("partnerTlsCertificateId must be a certificate with usage TLS");
}
}
/*******************************************************************************
** Certificate row by id, or null.
*******************************************************************************/
private static QRecord certificate(Integer id) throws Exception
{
if(id == null)
{
return (null);
}
return (SystemWrites.call(() -> EdiRecords.query(CertificateService.TABLE, EdiRecords.equal("id", id)).stream().findFirst().orElse(null)));
}
/*******************************************************************************
** The deployment public base URL without a trailing slash, or null.
*******************************************************************************/
static String baseUrl()
{
try
{
String url = EdiRuntime.current().config().getPublicBaseUrl();
return (url == null || url.isBlank() ? null : url.replaceAll("/+$", ""));
}
catch(Exception e)
{
return (null);
}
}
/*******************************************************************************
** Build, sign, encrypt and POST the message; settle it from the receipt.
*******************************************************************************/
@Override
public EdiSendResult send(EdiOutboundMessage message, PayloadRef payload, EdiTransport transport) throws Exception
{
return (SystemWrites.call(() -> deliver(message, payload, transport)));
}
/*******************************************************************************
** The send itself, inside the system context. Configuration problems (a
** missing variable, an expired or missing certificate) are RETRY results that
** name only the problem, so the retry policy and alerts apply once fixed.
*******************************************************************************/
private EdiSendResult deliver(EdiOutboundMessage message, PayloadRef payload, EdiTransport transport) throws Exception
{
byte[] edi;
try(InputStream stream = EdiRuntime.current().payloads().open(payload))
{
edi = stream.readAllBytes();
}
String messageId = messageId(message.getId(), transport, edi);
CertificateService certificates = new CertificateService();
String mode = mode(transport);
As2Http.Response response;
Prepared prepared;
try
{
prepared = prepare(message, transport, edi, messageId, certificates);
HttpClient client = As2Http.client(certificates.tlsPin(transport));
response = As2Http.post(client, URI.create(transport.getAs2Url()), headers(transport, messageId, prepared, mode), prepared.wire().body(), TIMEOUT);
}
catch(As2Exception e)
{
return (new EdiSendResult("RETRY", null, e.getMessage()));
}
catch(java.io.IOException e)
{
return (new EdiSendResult("RETRY", null, "The AS2 endpoint could not be reached"));
}
if(response.status() >= 500 || response.status() == 429)
{
return (new EdiSendResult("RETRY", null, "HTTP " + response.status()));
}
if(response.status() < 200 || response.status() >= 300)
{
return (new EdiSendResult("FAILED", null, "HTTP " + response.status()));
}
SystemWrites.update("edi_outbound_message", new QRecord().withValue("id", message.getId()).withValue("transportMic", prepared.mic().getAsAS2String()));
if(!"SYNC".equals(mode))
{
return (new EdiSendResult("SENT", messageId, "ASYNC".equals(mode) ? "Awaiting the asynchronous MDN" : "No receipt requested"));
}
return (settleSync(message, transport, messageId, prepared, response, certificates));
}
/*******************************************************************************
** Compress, sign and encrypt as configured, computing the MIC over the content
** that is signed.
*******************************************************************************/
private Prepared prepare(EdiOutboundMessage message, EdiTransport transport, byte[] edi, String messageId, CertificateService certificates) throws Exception
{
ECryptoAlgorithmSign signAlgorithm = As2Crypto.signAlgorithm(transport.getSignAlgorithm());
ECryptoAlgorithmCrypt encryptAlgorithm = As2Crypto.encryptAlgorithm(transport.getEncryptAlgorithm());
LocalKey key = signAlgorithm == null ? null : certificates.activeLocal(transport);
X509Certificate partner = encryptAlgorithm == null ? null : certificates.activePartnerCertificate(transport);
checkValid("Local certificate", key == null ? null : key.certificate());
checkValid("Partner certificate", partner);
String contentType = edi.length >= 3 && edi[0] == 'U' && edi[1] == 'N' ? "application/edifact" : "application/edi-x12";
var part = As2Crypto.payloadPart(edi, contentType, "edi-" + message.getId() + ".edi");
if(Boolean.TRUE.equals(transport.getCompress()))
{
part = As2Crypto.compress(part);
}
ECryptoAlgorithmSign micAlgorithm = signAlgorithm == null ? ECryptoAlgorithmSign.DIGEST_SHA_256 : signAlgorithm;
byte[] clear;
MIC mic;
if(signAlgorithm == null)
{
clear = As2Crypto.toBytes(part);
mic = As2Crypto.mic(clear, micAlgorithm);
}
else
{
clear = As2Crypto.toBytes(As2Crypto.sign(part, key, signAlgorithm));
As2Crypto.Wire signed = As2Crypto.splitRaw(clear);
mic = As2Crypto.mic(As2Crypto.firstPart(signed.contentType(), signed.body()), micAlgorithm);
}
return (new Prepared(encryptAlgorithm == null ? As2Crypto.splitRaw(clear) : As2Crypto.encrypt(clear, partner, encryptAlgorithm), mic));
}
/*******************************************************************************
** Certificates outside their validity period are not used.
*******************************************************************************/
private static void checkValid(String what, X509Certificate certificate) throws As2Exception
{
if(certificate != null)
{
try
{
certificate.checkValidity();
}
catch(Exception e)
{
throw (new As2Exception(what + " " + certificate.getSerialNumber().toString(16) + " is expired or not yet valid"));
}
}
}
/*******************************************************************************
** The AS2 request headers.
*******************************************************************************/
private Map<String, String> headers(EdiTransport transport, String messageId, Prepared prepared, String mode) throws Exception
{
Map<String, String> headers = new LinkedHashMap<>();
headers.put("AS2-Version", "1.2");
headers.put("AS2-From", asId(transport.getAs2LocalId()));
headers.put("AS2-To", asId(transport.getAs2PartnerId()));
headers.put("Message-ID", "<" + messageId + ">");
headers.put("Subject", subject(transport));
headers.put("Date", DateTimeFormatter.RFC_1123_DATE_TIME.format(ZonedDateTime.now(EdiRuntime.current().config().getClock().withZone(ZoneOffset.UTC))));
headers.put("MIME-Version", "1.0");
headers.put("User-Agent", "qbit-edi");
headers.put("Content-Type", prepared.wire().contentType());
if(prepared.wire().transferEncoding() != null)
{
headers.put("Content-Transfer-Encoding", prepared.wire().transferEncoding());
}
if(prepared.wire().contentDisposition() != null)
{
headers.put("Content-Disposition", prepared.wire().contentDisposition());
}
if(!"NONE".equals(mode))
{
headers.put("Disposition-Notification-To", transport.getAs2LocalId());
if(Boolean.TRUE.equals(transport.getMdnSigned()))
{
ECryptoAlgorithmSign alg = As2Crypto.signAlgorithm(transport.getSignAlgorithm()) == null ? ECryptoAlgorithmSign.DIGEST_SHA_256 : As2Crypto.signAlgorithm(transport.getSignAlgorithm());
headers.put("Disposition-Notification-Options", new DispositionOptions().setProtocolImportance(DispositionOptions.IMPORTANCE_REQUIRED)
.setProtocol(DispositionOptions.PROTOCOL_PKCS7_SIGNATURE).setMICAlgImportance(DispositionOptions.IMPORTANCE_REQUIRED).setMICAlg(alg).getAsString());
}
}
if("ASYNC".equals(mode))
{
headers.put("Receipt-Delivery-Option", baseUrl() + "/qqq/hooks/edi/as2/" + transport.getName() + "/mdn");
}
if(transport.getHttpAuthUserEnvVar() != null)
{
headers.put("Authorization", As2Http.basic(secret(transport.getHttpAuthUserEnvVar()), secret(transport.getHttpAuthPasswordEnvVar())));
}
return (headers);
}
/*******************************************************************************
** Evaluate a synchronous receipt: signature, disposition and MIC.
*******************************************************************************/
private EdiSendResult settleSync(EdiOutboundMessage message, EdiTransport transport, String messageId, Prepared prepared, As2Http.Response response, CertificateService certificates) throws Exception
{
String contentType = response.headers().get("content-type");
if(response.body().length == 0 || contentType == null)
{
return (new EdiSendResult("RETRY", messageId, "The partner returned no MDN"));
}
As2Mdn.Parsed mdn;
try
{
mdn = As2Mdn.parse(contentType, response.headers().get("content-transfer-encoding"), response.body(), certificates.partnerVerificationCertificates(transport),
Date.from(EdiRuntime.current().config().getClock().instant()));
}
catch(As2Exception e)
{
return (new EdiSendResult("RETRY", messageId, e.getMessage()));
}
String failure = judge(transport, mdn, prepared.mic());
PayloadRef raw = As2Store.storeRaw(As2Crypto.entity(contentType, response.headers().get("content-transfer-encoding"), null, response.body()));
As2Store.insert(new QRecord().withValue("direction", "INBOUND").withValue("messageId", As2Mdn.sanitize(As2Mdn.normalizeId(response.headers().get("message-id"))))
.withValue("originalMessageId", messageId).withValue("disposition", mdn.disposition()).withValue("micAlgorithm", mdn.mic() == null ? null : mdn.mic().getDigestAlgorithm().getID())
.withValue("micValue", As2Store.micValue(mdn.mic())).withValue("signed", mdn.signed()).withValue("signatureValid", mdn.signatureValid()).withValue("payloadRef", raw.value())
.withValue("transportId", transport.getId()).withValue("partnerId", message.getPartnerId()).withValue("outboundMessageId", message.getId()).withValue("deliveryStatus", "NONE"));
EdiJournal.event(null, message.getInterchangeId(), null, "MDN");
if(failure != null)
{
return (new EdiSendResult("FAILED", messageId, failure));
}
QAlerts.resolve("edi.mdn.overdue", "edi_outbound_message:" + message.getId(), "MDN received");
return (new EdiSendResult("DELIVERED", messageId, "MDN processed"));
}
/*******************************************************************************
** The reason a receipt does not confirm delivery, or null when it does.
*******************************************************************************/
static String judge(EdiTransport transport, As2Mdn.Parsed mdn, MIC ours)
{
if(Boolean.TRUE.equals(mdn.signed()) && !Boolean.TRUE.equals(mdn.signatureValid()))
{
return ("The MDN signature could not be verified");
}
if(Boolean.TRUE.equals(transport.getMdnSigned()) && !Boolean.TRUE.equals(mdn.signed()))
{
return ("The MDN is not signed");
}
if(!mdn.processed())
{
return ("Partner disposition: " + mdn.disposition());
}
if(ours != null && (mdn.mic() == null ? Boolean.TRUE.equals(transport.getMdnSigned()) || mdn.micText() != null : !mdn.mic().equals(ours)))
{
return ("The MDN Received-Content-MIC does not match the message sent");
}
return (null);
}
/*******************************************************************************
** Probe configuration without sending any document: resolve keys and
** certificates, then OPTIONS the endpoint.
*******************************************************************************/
@Override
public EdiTestResult test(EdiTransport transport)
{
try
{
return (SystemWrites.call(() -> probe(transport)));
}
catch(Exception e)
{
return (new EdiTestResult(false, "AS2 test could not run; check the database and configuration"));
}
}
/*******************************************************************************
** Resolve keys and certificates and reach the endpoint.
*******************************************************************************/
private EdiTestResult probe(EdiTransport transport) throws Exception
{
CertificateService certificates = new CertificateService();
List<String> notes = new java.util.ArrayList<>();
try
{
boolean secured = transport.getSignAlgorithm() != null || transport.getEncryptAlgorithm() != null || Boolean.TRUE.equals(transport.getMdnSigned());
if(secured || transport.getLocalCertificateId() != null)
{
LocalKey key = certificates.activeLocal(transport);
notes.add("Local key " + key.row().getValueString("alias") + " resolved (valid until " + key.certificate().getNotAfter().toInstant() + ")");
}
if(secured || transport.getPartnerCertificateId() != null)
{
certificates.activePartnerCertificate(transport);
notes.add(certificates.partnerVerificationCertificates(transport).size() + " partner certificate(s) accepted for verification");
}
HttpClient client = As2Http.client(certificates.tlsPin(transport));
if(!"INBOUND".equals(transport.getDirection()) && transport.getAs2Url() != null)
{
int status = As2Http.options(client, URI.create(transport.getAs2Url()), Map.of(), Duration.ofSeconds(15)).status();
notes.add("Endpoint responded with HTTP " + status);
if(status >= 500)
{
return (new EdiTestResult(false, String.join("; ", notes)));
}
}
}
catch(As2Exception e)
{
notes.add(e.getMessage());
return (new EdiTestResult(false, String.join("; ", notes)));
}
catch(java.io.IOException e)
{
notes.add("The AS2 endpoint could not be reached");
return (new EdiTestResult(false, String.join("; ", notes)));
}
if(As2Crypto.isWeak(transport.getSignAlgorithm(), transport.getEncryptAlgorithm()))
{
notes.add("Warning: weak crypto (SHA1 or 3DES) is configured");
}
return (new EdiTestResult(true, String.join("; ", notes)));
}
/*******************************************************************************
** Receipt mode, defaulting to SYNC.
*******************************************************************************/
static String mode(EdiTransport transport)
{
return (transport.getMdnMode() == null ? "SYNC" : transport.getMdnMode());
}
/*******************************************************************************
** A stable Message-ID: retries of the same outbound message reuse it, so the
** partner can recognize them.
*******************************************************************************/
static String messageId(Integer messageId, EdiTransport transport, byte[] edi) throws Exception
{
String digest = HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(edi)).substring(0, 12);
return ("edi-" + messageId + "-" + digest + "@" + transport.getAs2LocalId().replaceAll("[^A-Za-z0-9.-]", "-"));
}
/*******************************************************************************
** An AS2 identifier as a header value; identifiers with spaces are quoted.
*******************************************************************************/
static String asId(String id)
{
return (id.contains(" ") ? "\"" + id + "\"" : id);
}
/*******************************************************************************
** Subject header without line breaks.
*******************************************************************************/
private static String subject(EdiTransport transport)
{
String subject = transport.getAs2Subject() == null || transport.getAs2Subject().isBlank() ? "EDI interchange" : transport.getAs2Subject();
return (subject.replaceAll("[\\r\\n]+", " "));
}
/*******************************************************************************
** Resolve a secret by environment variable name; failure names only the name.
*******************************************************************************/
private static String secret(String variable) throws As2Exception
{
String value = EdiRuntime.current().config().getSecretsResolver().resolve(variable);
if(value == null)
{
throw (new As2Exception("environment variable " + variable + " is not set"));
}
return (value);
}
}
src/main/java/com/kingsrook/qbits/edi/as2/CertificateCustomizer.java
+102
View File
@@ -0,0 +1,102 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi.as2;
import java.security.cert.X509Certificate;
import java.time.Instant;
import java.util.List;
import java.util.Objects;
import java.util.Optional;
import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.statusmessages.BadInputStatusMessage;
/*******************************************************************************
** Pre-insert and pre-update customizer for edi_certificate. It parses the pasted
** public certificate and fills subject, issuer, serial number, fingerprint and
** validity dates, and refuses a private key pasted into certificatePem (the
** private key belongs in the environment variable named by privateKeyEnvVar).
*******************************************************************************/
public class CertificateCustomizer implements TableCustomizerInterface
{
/*******************************************************************************
** Parse the certificate on save; reject private keys and key variables on
** partner certificates without echoing any value.
*******************************************************************************/
@Override
public List<QRecord> preInsertOrUpdate(AbstractActionInput input, List<QRecord> records, boolean preview, Optional<List<QRecord>> oldRecords) throws QException
{
for(QRecord record : records)
{
QRecord merged = new QRecord();
oldRecords.orElse(List.of()).stream().filter(old -> Objects.equals(old.getValue("id"), record.getValue("id")))
.findFirst().ifPresent(old -> merged.getValues().putAll(old.getValues()));
merged.getValues().putAll(record.getValues());
boolean insert = oldRecords.isEmpty() || record.getValue("id") == null;
if(insert || record.getValues().containsKey("certificatePem"))
{
try
{
X509Certificate certificate = CertificateService.parse(merged.getValueString("certificatePem"));
record.setValue("subject", abbreviate(certificate.getSubjectX500Principal().getName()));
record.setValue("issuer", abbreviate(certificate.getIssuerX500Principal().getName()));
record.setValue("serialNumber", certificate.getSerialNumber().toString(16).toUpperCase());
record.setValue("fingerprintSha256", CertificateService.fingerprint(certificate));
record.setValue("notBefore", certificate.getNotBefore().toInstant());
record.setValue("notAfter", certificate.getNotAfter().toInstant());
merged.setValue("notAfter", certificate.getNotAfter().toInstant());
}
catch(As2Exception e)
{
record.addError(new BadInputStatusMessage(e.getMessage()));
}
}
if(insert && merged.getValueString("status") == null)
{
record.setValue("status", "ACTIVE");
}
if(merged.getValueInteger("partnerId") != null && merged.getValueString("privateKeyEnvVar") != null && !merged.getValueString("privateKeyEnvVar").isBlank())
{
record.addError(new BadInputStatusMessage("privateKeyEnvVar is only for our own certificates; leave it blank on a partner certificate"));
}
Instant notAfter = merged.getValueInstant("notAfter");
if(notAfter != null && merged.getValueInstant("notBefore") != null && !notAfter.isAfter(merged.getValueInstant("notBefore")))
{
record.addError(new BadInputStatusMessage("The certificate validity period is empty"));
}
}
return (records);
}
/*******************************************************************************
** Keep parsed names within the column length.
*******************************************************************************/
private static String abbreviate(String value)
{
return (value.length() <= 255 ? value : value.substring(0, 255));
}
}
src/main/java/com/kingsrook/qbits/edi/as2/CertificateService.java
+474
View File
@@ -0,0 +1,474 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi.as2;
import java.io.ByteArrayInputStream;
import java.io.StringReader;
import java.nio.charset.StandardCharsets;
import java.security.KeyStore;
import java.security.MessageDigest;
import java.security.PrivateKey;
import java.security.Signature;
import java.security.cert.CertificateFactory;
import java.security.cert.X509Certificate;
import java.util.ArrayList;
import java.util.Base64;
import java.util.Collection;
import java.util.Collections;
import java.util.HexFormat;
import java.util.List;
import java.util.Objects;
import com.kingsrook.qbits.edi.EdiRuntime;
import com.kingsrook.qbits.edi.model.EdiTransport;
import com.kingsrook.qbits.edi.storage.EdiRecords;
import com.kingsrook.qbits.edi.support.SystemWrites;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import org.bouncycastle.asn1.pkcs.PrivateKeyInfo;
import org.bouncycastle.openssl.PEMKeyPair;
import org.bouncycastle.openssl.PEMParser;
import org.bouncycastle.openssl.jcajce.JcaPEMKeyConverter;
import org.bouncycastle.openssl.jcajce.JceOpenSSLPKCS8DecryptorProviderBuilder;
import org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo;
/*******************************************************************************
** Resolves the certificates and keys an AS2 transport uses. Public certificates
** come from edi_certificate rows; our private keys come only from environment
** variables (a PEM PKCS#8 key or a base64 PKCS#12 store) through the replaceable
** secrets resolver, are matched to the certificate, and are never cached,
** logged, stored or returned. Callers run inside the system context.
*******************************************************************************/
public final class CertificateService
{
private static final QLogger LOG = QLogger.getLogger(CertificateService.class);
public static final String TABLE = "edi_certificate";
/*******************************************************************************
** Our certificate with its matched private key; the key is excluded from any
** string form of this object.
*******************************************************************************/
public record LocalKey(QRecord row, X509Certificate certificate, PrivateKey key)
{
/*******************************************************************************
** Never render key material.
*******************************************************************************/
@Override
public String toString()
{
return ("LocalKey[" + row.getValueString("alias") + "]");
}
}
/*******************************************************************************
** Local keys usable for decryption, plus the problems (naming environment
** variables only) for keys that could not be loaded.
*******************************************************************************/
public record KeySet(List<LocalKey> keys, List<String> problems)
{
}
/*******************************************************************************
** True when the text contains any private key block.
*******************************************************************************/
public static boolean containsPrivateKey(String text)
{
return (text != null && text.toUpperCase().contains("PRIVATE KEY"));
}
/*******************************************************************************
** Parse the first X.509 certificate of a PEM text; private keys are refused
** without echoing any content.
*******************************************************************************/
public static X509Certificate parse(String pem) throws As2Exception
{
if(pem == null || pem.isBlank())
{
throw (new As2Exception("certificatePem is required"));
}
if(containsPrivateKey(pem))
{
throw (new As2Exception("certificatePem must hold only the public certificate; keep the private key in the environment variable named by privateKeyEnvVar"));
}
if(!pem.contains("-----BEGIN CERTIFICATE-----"))
{
throw (new As2Exception("certificatePem must be a PEM encoded X.509 certificate"));
}
Collection<?> certificates;
try
{
certificates = CertificateFactory.getInstance("X.509").generateCertificates(new ByteArrayInputStream(pem.getBytes(StandardCharsets.UTF_8)));
}
catch(Exception e)
{
throw (new As2Exception("certificatePem could not be parsed as an X.509 certificate"));
}
if(certificates.isEmpty() || !(certificates.iterator().next() instanceof X509Certificate certificate))
{
throw (new As2Exception("certificatePem does not contain an X.509 certificate"));
}
return (certificate);
}
/*******************************************************************************
** Upper-case colon separated SHA-256 fingerprint of the certificate.
*******************************************************************************/
public static String fingerprint(X509Certificate certificate) throws As2Exception
{
try
{
return (HexFormat.ofDelimiter(":").withUpperCase().formatHex(MessageDigest.getInstance("SHA-256").digest(certificate.getEncoded())));
}
catch(Exception e)
{
throw (new As2Exception("certificate fingerprint could not be computed"));
}
}
/*******************************************************************************
** Resolve the key used to sign messages and receipts for the transport: the
** referenced certificate when it is ACTIVE, otherwise the ACTIVE certificate
** of ours with the same usage (so a promoted rollover takes effect at once).
*******************************************************************************/
public LocalKey activeLocal(EdiTransport transport) throws As2Exception, QException
{
QRecord referenced = byId(transport.getLocalCertificateId());
if(referenced == null)
{
throw (new As2Exception("The local certificate of transport " + transport.getName() + " is not configured"));
}
QRecord active = "ACTIVE".equals(referenced.getValueString("status")) ? referenced : findActive(null, referenced.getValueString("usage"), true);
if(active == null)
{
throw (new As2Exception("No ACTIVE local certificate is available for transport " + transport.getName()));
}
return (load(active));
}
/*******************************************************************************
** Every local key inbound decryption accepts: our ACTIVE and NEXT certificates
** that can decrypt. A key that cannot be loaded is skipped and reported by
** environment variable name only.
*******************************************************************************/
public KeySet localDecryptionKeys() throws QException
{
List<LocalKey> keys = new ArrayList<>();
List<String> problems = new ArrayList<>();
QQueryFilter filter = new QQueryFilter().withCriteria("status", QCriteriaOperator.IN, "ACTIVE", "NEXT")
.withCriteria("partnerId", QCriteriaOperator.IS_BLANK).withCriteria("privateKeyEnvVar", QCriteriaOperator.IS_NOT_BLANK);
for(QRecord row : EdiRecords.query(TABLE, filter))
{
if(!List.of("ENCRYPTION", "SIGNING_AND_ENCRYPTION").contains(row.getValueString("usage")))
{
continue;
}
try
{
keys.add(load(row));
}
catch(As2Exception e)
{
problems.add(e.getMessage());
LOG.warn("Local AS2 key skipped: " + e.getMessage());
}
}
return (new KeySet(keys, problems));
}
/*******************************************************************************
** The partner certificate outbound messages are encrypted to: the referenced
** certificate when ACTIVE, otherwise the partner ACTIVE certificate of the
** same usage.
*******************************************************************************/
public X509Certificate activePartnerCertificate(EdiTransport transport) throws As2Exception, QException
{
QRecord referenced = byId(transport.getPartnerCertificateId());
if(referenced == null)
{
throw (new As2Exception("The partner certificate of transport " + transport.getName() + " is not configured"));
}
QRecord active = "ACTIVE".equals(referenced.getValueString("status")) ? referenced : findActive(referenced.getValueInteger("partnerId"), referenced.getValueString("usage"), false);
if(active == null)
{
throw (new As2Exception("No ACTIVE partner certificate is available for transport " + transport.getName()));
}
return (parse(active.getValueString("certificatePem")));
}
/*******************************************************************************
** Partner certificates a signature may verify against: the referenced one and
** the same partner other ACTIVE or NEXT signing certificates. A partner
** certificate with no partner (a shared VAN) has no rollover candidates.
*******************************************************************************/
public List<X509Certificate> partnerVerificationCertificates(EdiTransport transport) throws As2Exception, QException
{
QRecord referenced = byId(transport.getPartnerCertificateId());
if(referenced == null)
{
throw (new As2Exception("The partner certificate of transport " + transport.getName() + " is not configured"));
}
List<QRecord> rows = new ArrayList<>();
if(List.of("ACTIVE", "NEXT").contains(referenced.getValueString("status")))
{
rows.add(referenced);
}
if(referenced.getValueInteger("partnerId") != null)
{
QQueryFilter filter = EdiRecords.equal("partnerId", referenced.getValueInteger("partnerId")).withCriteria("status", QCriteriaOperator.IN, "ACTIVE", "NEXT")
.withCriteria("usage", QCriteriaOperator.IN, "SIGNING", "SIGNING_AND_ENCRYPTION").withCriteria("privateKeyEnvVar", QCriteriaOperator.IS_BLANK);
for(QRecord row : EdiRecords.query(TABLE, filter))
{
if(rows.stream().noneMatch(existing -> Objects.equals(existing.getValue("id"), row.getValue("id"))))
{
rows.add(row);
}
}
}
List<X509Certificate> certificates = new ArrayList<>();
for(QRecord row : rows)
{
certificates.add(parse(row.getValueString("certificatePem")));
}
return (certificates);
}
/*******************************************************************************
** The pinned TLS certificate of the partner endpoint, or null when no pin is set.
*******************************************************************************/
public X509Certificate tlsPin(EdiTransport transport) throws As2Exception, QException
{
QRecord row = byId(transport.getPartnerTlsCertificateId());
if(transport.getPartnerTlsCertificateId() != null && row == null)
{
throw (new As2Exception("The partner TLS certificate of transport " + transport.getName() + " does not exist"));
}
return (row == null ? null : parse(row.getValueString("certificatePem")));
}
/*******************************************************************************
** Promote a NEXT certificate to ACTIVE and retire the previous ACTIVE one of
** the same partner and usage. Returns false when the certificate is not NEXT.
*******************************************************************************/
public boolean promote(QRecord row) throws QException
{
if(!"NEXT".equals(row.getValueString("status")))
{
return (false);
}
boolean local = row.getValueInteger("partnerId") == null && row.getValueString("privateKeyEnvVar") != null;
QRecord previous = findActive(row.getValueInteger("partnerId"), row.getValueString("usage"), local);
while(previous != null)
{
SystemWrites.update(TABLE, new QRecord().withValue("id", previous.getValue("id")).withValue("status", "RETIRED"));
previous = findActive(row.getValueInteger("partnerId"), row.getValueString("usage"), local);
}
SystemWrites.update(TABLE, new QRecord().withValue("id", row.getValue("id")).withValue("status", "ACTIVE"));
return (true);
}
/*******************************************************************************
** Certificate row by id, or null.
*******************************************************************************/
private QRecord byId(Integer id) throws QException
{
if(id == null)
{
return (null);
}
List<QRecord> rows = EdiRecords.query(TABLE, EdiRecords.equal("id", id));
return (rows.isEmpty() ? null : rows.getFirst());
}
/*******************************************************************************
** The ACTIVE certificate of a partner (null partner means ours or a shared
** one) and usage, local ones being those with a private key variable.
*******************************************************************************/
private QRecord findActive(Integer partnerId, String usage, boolean local) throws QException
{
QQueryFilter filter = EdiRecords.equal("status", "ACTIVE").withCriteria("partnerId", partnerId == null ? QCriteriaOperator.IS_BLANK : QCriteriaOperator.EQUALS, partnerId)
.withCriteria("usage", QCriteriaOperator.EQUALS, usage).withCriteria("privateKeyEnvVar", local ? QCriteriaOperator.IS_NOT_BLANK : QCriteriaOperator.IS_BLANK);
List<QRecord> rows = EdiRecords.query(TABLE, filter);
return (rows.stream().min((a, b) -> Integer.compare(a.getValueInteger("id"), b.getValueInteger("id"))).orElse(null));
}
/*******************************************************************************
** Parse the certificate, read its private key from the environment and prove
** the two belong together by signing and verifying a probe.
*******************************************************************************/
private LocalKey load(QRecord row) throws As2Exception
{
X509Certificate certificate = parse(row.getValueString("certificatePem"));
String variable = row.getValueString("privateKeyEnvVar");
if(variable == null || variable.isBlank())
{
throw (new As2Exception("Certificate " + row.getValueString("alias") + " has no privateKeyEnvVar"));
}
PrivateKey key = readKey(variable, row.getValueString("privateKeyPasswordEnvVar"));
if(!matches(key, certificate))
{
throw (new As2Exception("The private key in environment variable " + variable + " does not match certificate " + row.getValueString("alias")));
}
return (new LocalKey(row, certificate, key));
}
/*******************************************************************************
** Read the key from the named environment variable; failures name only the
** variable.
*******************************************************************************/
private static PrivateKey readKey(String variable, String passwordVariable) throws As2Exception
{
var secrets = EdiRuntime.current().config().getSecretsResolver();
String value = secrets.resolve(variable);
if(value == null || value.isBlank())
{
throw (new As2Exception("environment variable " + variable + " is not set"));
}
char[] password = new char[0];
if(passwordVariable != null && !passwordVariable.isBlank())
{
String resolved = secrets.resolve(passwordVariable);
if(resolved == null)
{
throw (new As2Exception("environment variable " + passwordVariable + " is not set"));
}
password = resolved.toCharArray();
}
try
{
return (parseKey(value, password));
}
catch(Exception e)
{
throw (new As2Exception("The private key in environment variable " + variable + " could not be read as a PEM PKCS#8 key or a base64 PKCS#12 store"));
}
}
/*******************************************************************************
** Parse a PEM key (PKCS#8, encrypted PKCS#8 or traditional) or a base64 PKCS#12
** store, entirely in memory.
*******************************************************************************/
static PrivateKey parseKey(String raw, char[] password) throws Exception
{
String text = raw.trim();
if(text.indexOf('\n') < 0 && text.contains("\\n"))
{
text = text.replace("\\n", "\n");
}
if(text.contains("-----BEGIN"))
{
JcaPEMKeyConverter converter = new JcaPEMKeyConverter();
try(PEMParser parser = new PEMParser(new StringReader(text)))
{
Object object = parser.readObject();
while(object != null)
{
if(object instanceof PrivateKeyInfo info)
{
return (converter.getPrivateKey(info));
}
if(object instanceof PEMKeyPair pair)
{
return (converter.getKeyPair(pair).getPrivate());
}
if(object instanceof PKCS8EncryptedPrivateKeyInfo encrypted)
{
return (converter.getPrivateKey(encrypted.decryptPrivateKeyInfo(new JceOpenSSLPKCS8DecryptorProviderBuilder().build(password))));
}
object = parser.readObject();
}
}
throw (new IllegalArgumentException("no private key block"));
}
KeyStore store = KeyStore.getInstance("PKCS12");
store.load(new ByteArrayInputStream(Base64.getMimeDecoder().decode(text)), password);
for(String alias : Collections.list(store.aliases()))
{
if(store.isKeyEntry(alias))
{
return ((PrivateKey) store.getKey(alias, password));
}
}
throw (new IllegalArgumentException("no key entry"));
}
/*******************************************************************************
** Sign a probe with the key and verify it with the certificate public key.
*******************************************************************************/
private static boolean matches(PrivateKey key, X509Certificate certificate)
{
try
{
String algorithm = "EC".equals(key.getAlgorithm()) ? "SHA256withECDSA" : "SHA256with" + key.getAlgorithm();
byte[] probe = "qbit-edi-key-match".getBytes(StandardCharsets.UTF_8);
Signature signer = Signature.getInstance(algorithm);
signer.initSign(key);
signer.update(probe);
byte[] signature = signer.sign();
Signature verifier = Signature.getInstance(algorithm);
verifier.initVerify(certificate.getPublicKey());
verifier.update(probe);
return (verifier.verify(signature));
}
catch(Exception e)
{
return (false);
}
}
}
src/main/java/com/kingsrook/qbits/edi/flow/AckMonitor.java
+44 -2
View File
@@ -30,6 +30,7 @@ import com.kingsrook.qqq.backend.core.alerts.AlertSignal;
import com.kingsrook.qqq.backend.core.alerts.QAlerts;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
@@ -65,7 +66,7 @@ public class AckMonitor
}
count++;
}
for(QRecord certificate : EdiRecords.query("edi_certificate", EdiRecords.equal("status", "ACTIVE")))
for(QRecord certificate : EdiRecords.query("edi_certificate", new QQueryFilter().withCriteria("status", QCriteriaOperator.IN, "ACTIVE", "NEXT")))
{
if(certificate.getValueInstant("notAfter") == null)
{
@@ -87,10 +88,51 @@ public class AckMonitor
}
}
}
return (count);
return (count + overdueMdns(now));
}));
}
/*******************************************************************************
** SENT messages on an asynchronous-MDN AS2 transport whose MDN has not arrived
** within the agreement document ackDueMinutes (60 when none is set) raise a
** warning, deduplicated per message and resolved when the MDN arrives.
*******************************************************************************/
private static int overdueMdns(java.time.Instant now) throws QException
{
List<java.io.Serializable> transports = new java.util.ArrayList<>();
for(QRecord transport : EdiRecords.query("edi_transport", EdiRecords.equal("type", "AS2").withCriteria("mdnMode", QCriteriaOperator.EQUALS, "ASYNC")))
{
transports.add(transport.getValue("id"));
}
if(transports.isEmpty())
{
return (0);
}
int count = 0;
for(QRecord message : EdiRecords.query("edi_outbound_message", EdiRecords.equal("status", "SENT").withCriteria("transportId", QCriteriaOperator.IN, transports)
.withCriteria("sentAt", QCriteriaOperator.IS_NOT_BLANK)))
{
int minutes = 60;
Integer smallest = null;
for(QRecord document : message.getValue("agreementId") == null ? List.<QRecord>of()
: EdiRecords.query("edi_agreement_document", EdiRecords.equal("agreementId", message.getValue("agreementId")).withCriteria("direction", QCriteriaOperator.EQUALS, "OUTBOUND")))
{
Integer due = document.getValueInteger("ackDueMinutes");
smallest = due == null ? smallest : smallest == null ? due : Math.min(smallest, due);
}
minutes = smallest == null ? minutes : smallest;
if(message.getValueInstant("sentAt").plus(minutes, ChronoUnit.MINUTES).isAfter(now))
{
continue;
}
QAlerts.raise(new AlertSignal("edi.mdn.overdue").withSeverity(AlertSeverity.WARNING).withDedupKey("edi_outbound_message:" + message.getValue("id"))
.withRecordRef(new com.kingsrook.qqq.backend.core.alerts.AlertRecordRef("edi_outbound_message", message.getValue("id")))
.withTitle("AS2 receipt overdue").withSummary("The asynchronous MDN has not arrived; review the outbound message"));
count++;
}
return (count);
}
/*******************************************************************************
** Each certificate threshold has its own stable deduplication key.
*******************************************************************************/
src/main/java/com/kingsrook/qbits/edi/flow/SendWorker.java
+12 -1
View File
@@ -101,7 +101,7 @@ public class SendWorker
boolean sent = List.of("SENT", "DELIVERED").contains(result.status());
String status = sent ? result.status() : "FAILED".equals(result.status()) ? "FAILED" : attempts >= maximum ? "DEAD_LETTER" : "RETRY_WAIT";
QRecord update = new QRecord().withValue("id", message.getValue("id")).withValue("status", status).withValue("attempts", attempts)
.withValue("claimedAt", null).withValue("lastError", sent ? null : "Transport delivery failed")
.withValue("claimedAt", null).withValue("lastError", sent ? null : failureText(result))
.withValue("nextAttemptAt", "RETRY_WAIT".equals(status) ? now().plusSeconds(BACKOFF.get(Math.min(attempts - 1, BACKOFF.size() - 1))) : null);
if(sent)
{
@@ -145,6 +145,17 @@ public class SendWorker
}));
}
/*******************************************************************************
** The recorded failure: the transport reason (value-free by contract, for
** example an HTTP status, an AS2 disposition or a missing variable name), or a
** generic text.
*******************************************************************************/
private static String failureText(EdiSendResult result)
{
String reason = result.reason();
return (reason == null || reason.isBlank() ? "Transport delivery failed" : reason.length() <= 255 ? reason : reason.substring(0, 255));
}
/*******************************************************************************
** Clock controlled by the host and deterministic tests.
*******************************************************************************/
src/main/java/com/kingsrook/qbits/edi/metadata/EdiMetaDataProducer.java
+1 -1
View File
@@ -69,7 +69,7 @@ public class EdiMetaDataProducer extends QBitComponentMetaDataProducer<MetaDataP
}
output.add(com.kingsrook.qbits.edi.app.EdiHome.app(getQBitConfig(), tables));
com.kingsrook.qbits.edi.app.EdiHome.widgets().forEach(output::add);
for(String name : java.util.List.of("edi.inbox.dead-letter", "edi.ack.overdue", "edi.ack.rejected", "edi.send.failed", "edi.certificate.expiring", "edi.certificate.expired"))
for(String name : java.util.List.of("edi.inbox.dead-letter", "edi.ack.overdue", "edi.ack.rejected", "edi.send.failed", "edi.certificate.expiring", "edi.certificate.expired", "edi.mdn.overdue"))
{
output.add(new com.kingsrook.qqq.backend.core.alerts.QAlertTypeMetaData().withName(name).withLabel(name)
.withSourceName("edi").withDefaultSeverity(com.kingsrook.qqq.backend.core.alerts.AlertSeverity.ERROR)
src/main/java/com/kingsrook/qbits/edi/metadata/EdiRecordLabels.java
+1
View File
@@ -80,6 +80,7 @@ public class EdiRecordLabels implements TableCustomizerInterface, QSupplementalI
case "edi_exception" -> table.withRecordLabelFormatAndFields("%s: %s", "type", "summary");
case "edi_inbox" -> table.withRecordLabelFormatAndFields("%s %s", "transportType", "externalId");
case "edi_outbound_message" -> table.withRecordLabelFormatAndFields("%s %s", "transportId", "status");
case "edi_as2_mdn" -> table.withRecordLabelFormatAndFields("%s MDN %s", "direction", "originalMessageId");
case "edi_outbound_request" -> table.withRecordLabelFormatAndFields("%s %s %s", "sourceTableName", "sourceRecordId", "status");
case "edi_transaction_error" -> table.withRecordLabelFormatAndFields("%s: %s", "severity", "message");
case "edi_event" -> table.withRecordLabelFormatAndFields("%s %s", "type", "at");
src/main/java/com/kingsrook/qbits/edi/metadata/EdiTablePresentation.java
+3 -1
View File
@@ -40,7 +40,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.tables.Tier;
*******************************************************************************/
public final class EdiTablePresentation
{
private static final Set<String> ACRONYMS = Set.of("ID", "PHI", "PII", "URL", "ISA", "GS", "ST", "AS2", "SFTP", "HTTPS", "HTTP", "MDN", "TLS", "TA1", "EDI", "UPC", "NPI", "QQQ", "JSON", "PEM", "SHA256", "SNIP", "VAN", "AK3", "AK4", "IK3", "IK4");
private static final Set<String> ACRONYMS = Set.of("ID", "PHI", "PII", "URL", "ISA", "GS", "ST", "AS2", "SFTP", "HTTPS", "HTTP", "MDN", "TLS", "TA1", "EDI", "UPC", "NPI", "QQQ", "JSON", "PEM", "SHA256", "SNIP", "VAN", "AK3", "AK4", "IK3", "IK4", "MIC");
@@ -76,6 +76,7 @@ public final class EdiTablePresentation
case "edi_inbox" -> "Inbound Message";
case "edi_outbound_request" -> "Outbound Request";
case "edi_outbound_message" -> "Outbound Message";
case "edi_as2_mdn" -> "AS2 MDN";
case "edi_payload" -> "Payload";
default -> throw new IllegalArgumentException("Missing EDI table label: " + table);
});
@@ -198,6 +199,7 @@ public final class EdiTablePresentation
case "edi_inbox" -> List.of("transportType", "externalId", "status", "partnerId", "transportId", "receivedAt", "attempts");
case "edi_outbound_request" -> List.of("status", "agreementDocumentId", "sourceTableName", "sourceRecordId", "attempts", "nextAttemptAt");
case "edi_outbound_message" -> List.of("transportId", "status", "partnerId", "agreementId", "attempts", "sentAt", "deliveredAt");
case "edi_as2_mdn" -> List.of("direction", "originalMessageId", "disposition", "signed", "signatureValid", "partnerId", "transportId", "receivedAt");
case "edi_payload" -> List.of("kind", "sizeBytes", "sha256");
default -> throw new IllegalArgumentException("Missing EDI field order: " + table);
});
src/main/java/com/kingsrook/qbits/edi/metadata/EdiTables.java
+9 -1
View File
@@ -26,6 +26,7 @@ import java.util.List;
import com.kingsrook.qbits.edi.EdiQBitConfig;
import com.kingsrook.qbits.edi.model.EdiAgreement;
import com.kingsrook.qbits.edi.model.EdiAgreementDocument;
import com.kingsrook.qbits.edi.model.EdiAs2Mdn;
import com.kingsrook.qbits.edi.model.EdiCertificate;
import com.kingsrook.qbits.edi.model.EdiCodeList;
import com.kingsrook.qbits.edi.model.EdiCodeListValue;
@@ -118,6 +119,8 @@ public final class EdiTables
.withCustomizer(TableCustomizers.PRE_UPDATE_RECORD, new QCodeReference(TransportValidation.class));
tables.add(transport);
QTableMetaData certificate = base(config, EdiCertificate.TABLE_NAME, EdiCertificate.class, "alias", false);
certificate.withCustomizer(TableCustomizers.PRE_INSERT_RECORD, new QCodeReference(com.kingsrook.qbits.edi.as2.CertificateCustomizer.class))
.withCustomizer(TableCustomizers.PRE_UPDATE_RECORD, new QCodeReference(com.kingsrook.qbits.edi.as2.CertificateCustomizer.class));
tables.add(certificate);
QTableMetaData guide = base(config, EdiGuide.TABLE_NAME, EdiGuide.class, "name", false);
guide.withUniqueKey(new UniqueKey("name", "revision"));
@@ -156,6 +159,7 @@ public final class EdiTables
tables.add(base(config, "edi_map_sample", EdiMapSample.class, "name", false));
tables.add(base(config, "edi_outbound_request", EdiOutboundRequest.class, "id", true).withUniqueKey(new UniqueKey("agreementDocumentId", "eventId")));
tables.add(base(config, "edi_outbound_message", EdiOutboundMessage.class, "id", true));
tables.add(base(config, EdiAs2Mdn.TABLE_NAME, EdiAs2Mdn.class, "originalMessageId", true));
for(QTableMetaData table : tables)
{
List<String> guidePath = switch(table.getName())
@@ -305,6 +309,7 @@ public final class EdiTables
sources.add(enumSource("edi_exception_type", "UNKNOWN_PARTNER", "PARTNER_MISMATCH", "NOT_CONFIGURED", "DUPLICATE", "ENVELOPE", "VALIDATION", "MAPPING", "APPLY", "TRANSPORT", "REJECTED_ACK", "OVERDUE_ACK", "DEAD_LETTER"));
sources.add(enumSource("edi_exception_status", "OPEN", "IN_PROGRESS", "RESOLVED", "IGNORED"));
sources.add(enumSource("edi_inbox_status", "PENDING", "PROCESSING", "PROCESSED", "RETRY_WAIT", "DEAD_LETTER", "DUPLICATE", "IGNORED"));
sources.add(enumSource("edi_as2_mdn_delivery_status", "NONE", "PENDING", "SENT", "FAILED"));
sources.add(enumSource("edi_payload_kind", "INBOUND", "OUTBOUND", "AS2", "MDN", "SAMPLE"));
sources.add(QPossibleValueSource.newForTable("edi_partner"));
sources.add(QPossibleValueSource.newForTable("edi_identity"));
@@ -333,6 +338,7 @@ public final class EdiTables
sources.add(QPossibleValueSource.newForTable("edi_map_sample"));
sources.add(QPossibleValueSource.newForTable("edi_outbound_request"));
sources.add(QPossibleValueSource.newForTable("edi_outbound_message"));
sources.add(QPossibleValueSource.newForTable("edi_as2_mdn"));
return (sources);
}
@@ -403,6 +409,8 @@ public final class EdiTables
joins.add(new QJoinMetaData().withName("edi_agreement_document_agreement_document_id_edi_outbound_request").withType(JoinType.ONE_TO_MANY).withLeftTable("edi_agreement_document").withRightTable("edi_outbound_request").withJoinOn(new JoinOn("id", "agreementDocumentId")));
joins.add(new QJoinMetaData().withName("edi_agreement_agreement_id_edi_outbound_message").withType(JoinType.ONE_TO_MANY).withLeftTable("edi_agreement").withRightTable("edi_outbound_message").withJoinOn(new JoinOn("id", "agreementId")));
joins.add(new QJoinMetaData().withName("edi_outbound_message_outbound_message_id_edi_interchange").withType(JoinType.ONE_TO_MANY).withLeftTable("edi_outbound_message").withRightTable("edi_interchange").withJoinOn(new JoinOn("id", "outboundMessageId")));
joins.add(new QJoinMetaData().withName("edi_outbound_message_outbound_message_id_edi_as2_mdn").withType(JoinType.ONE_TO_MANY).withLeftTable("edi_outbound_message").withRightTable("edi_as2_mdn").withJoinOn(new JoinOn("id", "outboundMessageId")));
joins.add(new QJoinMetaData().withName("edi_inbox_inbox_id_edi_as2_mdn").withType(JoinType.ONE_TO_MANY).withLeftTable("edi_inbox").withRightTable("edi_as2_mdn").withJoinOn(new JoinOn("id", "inboxId")));
joins.add(new QJoinMetaData().withName("edi_inbox_inbox_id_edi_event").withType(JoinType.ONE_TO_MANY).withLeftTable("edi_inbox").withRightTable("edi_event").withJoinOn(new JoinOn("id", "inboxId")));
return (joins);
}
@@ -424,7 +432,7 @@ public final class EdiTables
{
return (switch(table)
{
case "edi_transaction", "edi_interchange", "edi_exception", "edi_inbox", "edi_event", "edi_outbound_request", "edi_outbound_message" ->
case "edi_transaction", "edi_interchange", "edi_exception", "edi_inbox", "edi_event", "edi_outbound_request", "edi_outbound_message", "edi_as2_mdn" ->
List.of(new QFilterOrderBy("createDate", false), new QFilterOrderBy("id", false));
case "edi_guide_segment", "edi_map_rule" -> List.of(new QFilterOrderBy("sequence"), new QFilterOrderBy("id"));
case "edi_guide_element" -> List.of(new QFilterOrderBy("position"), new QFilterOrderBy("componentPosition"), new QFilterOrderBy("id"));
src/main/java/com/kingsrook/qbits/edi/metadata/TransportValidation.java
+1
View File
@@ -74,6 +74,7 @@ public class TransportValidation implements TableCustomizerInterface
errors.add("maxAttempts must be positive");
}
errors.forEach(error -> record.addError(new BadInputStatusMessage(error)));
record.setValue("weakCrypto", "AS2".equals(merged.getValueString("type")) && com.kingsrook.qbits.edi.as2.As2Crypto.isWeak(merged.getValueString("signAlgorithm"), merged.getValueString("encryptAlgorithm")));
}
catch(Exception e)
{
src/main/java/com/kingsrook/qbits/edi/model/EdiAs2Mdn.java
+718
View File
@@ -0,0 +1,718 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi.model;
import java.time.Instant;
import com.kingsrook.qqq.backend.core.model.data.QField;
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
/*******************************************************************************
** Persistent edi_as2_mdn record: an AS2 receipt we sent or received. Rows are
** written by the qbit only; the delivery columns queue asynchronous receipts.
*******************************************************************************/
public class EdiAs2Mdn extends QRecordEntity
{
public static final String TABLE_NAME = "edi_as2_mdn";
@QField(backendName = "id", isPrimaryKey = true, isEditable = false)
private Integer id;
@QField(backendName = "direction", maxLength = 20, possibleValueSourceName = "edi_direction")
private String direction;
@QField(backendName = "message_id", maxLength = 255)
private String messageId;
@QField(backendName = "original_message_id", maxLength = 255)
private String originalMessageId;
@QField(backendName = "disposition", maxLength = 255)
private String disposition;
@QField(backendName = "mic_algorithm", maxLength = 50)
private String micAlgorithm;
@QField(backendName = "mic_value", maxLength = 255)
private String micValue;
@QField(backendName = "signed")
private Boolean signed;
@QField(backendName = "signature_valid")
private Boolean signatureValid;
@QField(backendName = "payload_ref", maxLength = 255)
private String payloadRef;
@QField(backendName = "received_at")
private Instant receivedAt;
@QField(backendName = "transport_id", possibleValueSourceName = "edi_transport")
private Integer transportId;
@QField(backendName = "partner_id", possibleValueSourceName = "edi_partner")
private Integer partnerId;
@QField(backendName = "outbound_message_id", possibleValueSourceName = "edi_outbound_message")
private Integer outboundMessageId;
@QField(backendName = "inbox_id", possibleValueSourceName = "edi_inbox")
private Integer inboxId;
@QField(backendName = "delivery_url", maxLength = 1000)
private String deliveryUrl;
@QField(backendName = "delivery_status", maxLength = 20, possibleValueSourceName = "edi_as2_mdn_delivery_status")
private String deliveryStatus;
@QField(backendName = "delivery_attempts")
private Integer deliveryAttempts;
@QField(backendName = "next_attempt_at")
private Instant nextAttemptAt;
@QField(backendName = "delivered_at")
private Instant deliveredAt;
@QField(backendName = "create_date", isEditable = false)
private Instant createDate;
@QField(backendName = "modify_date", isEditable = false)
private Instant modifyDate;
/*******************************************************************************
** Get id.
*******************************************************************************/
public Integer getId()
{
return (id);
}
/*******************************************************************************
** Set id.
*******************************************************************************/
public void setId(Integer id)
{
this.id = id;
}
/*******************************************************************************
** Set id fluently.
*******************************************************************************/
public EdiAs2Mdn withId(Integer id)
{
this.id = id;
return (this);
}
/*******************************************************************************
** Get direction.
*******************************************************************************/
public String getDirection()
{
return (direction);
}
/*******************************************************************************
** Set direction.
*******************************************************************************/
public void setDirection(String direction)
{
this.direction = direction;
}
/*******************************************************************************
** Set direction fluently.
*******************************************************************************/
public EdiAs2Mdn withDirection(String direction)
{
this.direction = direction;
return (this);
}
/*******************************************************************************
** Get messageId.
*******************************************************************************/
public String getMessageId()
{
return (messageId);
}
/*******************************************************************************
** Set messageId.
*******************************************************************************/
public void setMessageId(String messageId)
{
this.messageId = messageId;
}
/*******************************************************************************
** Set messageId fluently.
*******************************************************************************/
public EdiAs2Mdn withMessageId(String messageId)
{
this.messageId = messageId;
return (this);
}
/*******************************************************************************
** Get originalMessageId.
*******************************************************************************/
public String getOriginalMessageId()
{
return (originalMessageId);
}
/*******************************************************************************
** Set originalMessageId.
*******************************************************************************/
public void setOriginalMessageId(String originalMessageId)
{
this.originalMessageId = originalMessageId;
}
/*******************************************************************************
** Set originalMessageId fluently.
*******************************************************************************/
public EdiAs2Mdn withOriginalMessageId(String originalMessageId)
{
this.originalMessageId = originalMessageId;
return (this);
}
/*******************************************************************************
** Get disposition.
*******************************************************************************/
public String getDisposition()
{
return (disposition);
}
/*******************************************************************************
** Set disposition.
*******************************************************************************/
public void setDisposition(String disposition)
{
this.disposition = disposition;
}
/*******************************************************************************
** Set disposition fluently.
*******************************************************************************/
public EdiAs2Mdn withDisposition(String disposition)
{
this.disposition = disposition;
return (this);
}
/*******************************************************************************
** Get micAlgorithm.
*******************************************************************************/
public String getMicAlgorithm()
{
return (micAlgorithm);
}
/*******************************************************************************
** Set micAlgorithm.
*******************************************************************************/
public void setMicAlgorithm(String micAlgorithm)
{
this.micAlgorithm = micAlgorithm;
}
/*******************************************************************************
** Set micAlgorithm fluently.
*******************************************************************************/
public EdiAs2Mdn withMicAlgorithm(String micAlgorithm)
{
this.micAlgorithm = micAlgorithm;
return (this);
}
/*******************************************************************************
** Get micValue.
*******************************************************************************/
public String getMicValue()
{
return (micValue);
}
/*******************************************************************************
** Set micValue.
*******************************************************************************/
public void setMicValue(String micValue)
{
this.micValue = micValue;
}
/*******************************************************************************
** Set micValue fluently.
*******************************************************************************/
public EdiAs2Mdn withMicValue(String micValue)
{
this.micValue = micValue;
return (this);
}
/*******************************************************************************
** Get signed.
*******************************************************************************/
public Boolean getSigned()
{
return (signed);
}
/*******************************************************************************
** Set signed.
*******************************************************************************/
public void setSigned(Boolean signed)
{
this.signed = signed;
}
/*******************************************************************************
** Set signed fluently.
*******************************************************************************/
public EdiAs2Mdn withSigned(Boolean signed)
{
this.signed = signed;
return (this);
}
/*******************************************************************************
** Get signatureValid.
*******************************************************************************/
public Boolean getSignatureValid()
{
return (signatureValid);
}
/*******************************************************************************
** Set signatureValid.
*******************************************************************************/
public void setSignatureValid(Boolean signatureValid)
{
this.signatureValid = signatureValid;
}
/*******************************************************************************
** Set signatureValid fluently.
*******************************************************************************/
public EdiAs2Mdn withSignatureValid(Boolean signatureValid)
{
this.signatureValid = signatureValid;
return (this);
}
/*******************************************************************************
** Get payloadRef.
*******************************************************************************/
public String getPayloadRef()
{
return (payloadRef);
}
/*******************************************************************************
** Set payloadRef.
*******************************************************************************/
public void setPayloadRef(String payloadRef)
{
this.payloadRef = payloadRef;
}
/*******************************************************************************
** Set payloadRef fluently.
*******************************************************************************/
public EdiAs2Mdn withPayloadRef(String payloadRef)
{
this.payloadRef = payloadRef;
return (this);
}
/*******************************************************************************
** Get receivedAt.
*******************************************************************************/
public Instant getReceivedAt()
{
return (receivedAt);
}
/*******************************************************************************
** Set receivedAt.
*******************************************************************************/
public void setReceivedAt(Instant receivedAt)
{
this.receivedAt = receivedAt;
}
/*******************************************************************************
** Set receivedAt fluently.
*******************************************************************************/
public EdiAs2Mdn withReceivedAt(Instant receivedAt)
{
this.receivedAt = receivedAt;
return (this);
}
/*******************************************************************************
** Get transportId.
*******************************************************************************/
public Integer getTransportId()
{
return (transportId);
}
/*******************************************************************************
** Set transportId.
*******************************************************************************/
public void setTransportId(Integer transportId)
{
this.transportId = transportId;
}
/*******************************************************************************
** Set transportId fluently.
*******************************************************************************/
public EdiAs2Mdn withTransportId(Integer transportId)
{
this.transportId = transportId;
return (this);
}
/*******************************************************************************
** Get partnerId.
*******************************************************************************/
public Integer getPartnerId()
{
return (partnerId);
}
/*******************************************************************************
** Set partnerId.
*******************************************************************************/
public void setPartnerId(Integer partnerId)
{
this.partnerId = partnerId;
}
/*******************************************************************************
** Set partnerId fluently.
*******************************************************************************/
public EdiAs2Mdn withPartnerId(Integer partnerId)
{
this.partnerId = partnerId;
return (this);
}
/*******************************************************************************
** Get outboundMessageId.
*******************************************************************************/
public Integer getOutboundMessageId()
{
return (outboundMessageId);
}
/*******************************************************************************
** Set outboundMessageId.
*******************************************************************************/
public void setOutboundMessageId(Integer outboundMessageId)
{
this.outboundMessageId = outboundMessageId;
}
/*******************************************************************************
** Set outboundMessageId fluently.
*******************************************************************************/
public EdiAs2Mdn withOutboundMessageId(Integer outboundMessageId)
{
this.outboundMessageId = outboundMessageId;
return (this);
}
/*******************************************************************************
** Get inboxId.
*******************************************************************************/
public Integer getInboxId()
{
return (inboxId);
}
/*******************************************************************************
** Set inboxId.
*******************************************************************************/
public void setInboxId(Integer inboxId)
{
this.inboxId = inboxId;
}
/*******************************************************************************
** Set inboxId fluently.
*******************************************************************************/
public EdiAs2Mdn withInboxId(Integer inboxId)
{
this.inboxId = inboxId;
return (this);
}
/*******************************************************************************
** Get deliveryUrl.
*******************************************************************************/
public String getDeliveryUrl()
{
return (deliveryUrl);
}
/*******************************************************************************
** Set deliveryUrl.
*******************************************************************************/
public void setDeliveryUrl(String deliveryUrl)
{
this.deliveryUrl = deliveryUrl;
}
/*******************************************************************************
** Set deliveryUrl fluently.
*******************************************************************************/
public EdiAs2Mdn withDeliveryUrl(String deliveryUrl)
{
this.deliveryUrl = deliveryUrl;
return (this);
}
/*******************************************************************************
** Get deliveryStatus.
*******************************************************************************/
public String getDeliveryStatus()
{
return (deliveryStatus);
}
/*******************************************************************************
** Set deliveryStatus.
*******************************************************************************/
public void setDeliveryStatus(String deliveryStatus)
{
this.deliveryStatus = deliveryStatus;
}
/*******************************************************************************
** Set deliveryStatus fluently.
*******************************************************************************/
public EdiAs2Mdn withDeliveryStatus(String deliveryStatus)
{
this.deliveryStatus = deliveryStatus;
return (this);
}
/*******************************************************************************
** Get deliveryAttempts.
*******************************************************************************/
public Integer getDeliveryAttempts()
{
return (deliveryAttempts);
}
/*******************************************************************************
** Set deliveryAttempts.
*******************************************************************************/
public void setDeliveryAttempts(Integer deliveryAttempts)
{
this.deliveryAttempts = deliveryAttempts;
}
/*******************************************************************************
** Set deliveryAttempts fluently.
*******************************************************************************/
public EdiAs2Mdn withDeliveryAttempts(Integer deliveryAttempts)
{
this.deliveryAttempts = deliveryAttempts;
return (this);
}
/*******************************************************************************
** Get nextAttemptAt.
*******************************************************************************/
public Instant getNextAttemptAt()
{
return (nextAttemptAt);
}
/*******************************************************************************
** Set nextAttemptAt.
*******************************************************************************/
public void setNextAttemptAt(Instant nextAttemptAt)
{
this.nextAttemptAt = nextAttemptAt;
}
/*******************************************************************************
** Set nextAttemptAt fluently.
*******************************************************************************/
public EdiAs2Mdn withNextAttemptAt(Instant nextAttemptAt)
{
this.nextAttemptAt = nextAttemptAt;
return (this);
}
/*******************************************************************************
** Get deliveredAt.
*******************************************************************************/
public Instant getDeliveredAt()
{
return (deliveredAt);
}
/*******************************************************************************
** Set deliveredAt.
*******************************************************************************/
public void setDeliveredAt(Instant deliveredAt)
{
this.deliveredAt = deliveredAt;
}
/*******************************************************************************
** Set deliveredAt fluently.
*******************************************************************************/
public EdiAs2Mdn withDeliveredAt(Instant deliveredAt)
{
this.deliveredAt = deliveredAt;
return (this);
}
/*******************************************************************************
** Get createDate.
*******************************************************************************/
public Instant getCreateDate()
{
return (createDate);
}
/*******************************************************************************
** Set createDate.
*******************************************************************************/
public void setCreateDate(Instant createDate)
{
this.createDate = createDate;
}
/*******************************************************************************
** Set createDate fluently.
*******************************************************************************/
public EdiAs2Mdn withCreateDate(Instant createDate)
{
this.createDate = createDate;
return (this);
}
/*******************************************************************************
** Get modifyDate.
*******************************************************************************/
public Instant getModifyDate()
{
return (modifyDate);
}
/*******************************************************************************
** Set modifyDate.
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Set modifyDate fluently.
*******************************************************************************/
public EdiAs2Mdn withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
}
src/main/java/com/kingsrook/qbits/edi/model/EdiOutboundMessage.java
+28
View File
@@ -39,6 +39,9 @@ public class EdiOutboundMessage extends QRecordEntity
@QField(backendName = "transport_message_id", maxLength = 255)
private String transportMessageId;
@QField(backendName = "transport_mic", maxLength = 255)
private String transportMic;
@QField(backendName = "id", isPrimaryKey = true, isEditable = false)
private Integer id;
@@ -97,6 +100,31 @@ public class EdiOutboundMessage extends QRecordEntity
private Instant modifyDate;
/*******************************************************************************
** Get transportMic.
*******************************************************************************/
public String getTransportMic()
{
return (transportMic);
}
/*******************************************************************************
** Set transportMic.
*******************************************************************************/
public void setTransportMic(String value)
{
transportMic = value;
}
/*******************************************************************************
** Configure transportMic.
*******************************************************************************/
public EdiOutboundMessage withTransportMic(String value)
{
transportMic = value;
return (this);
}
/*******************************************************************************
** Get id.
*******************************************************************************/
src/main/java/com/kingsrook/qbits/edi/model/EdiTransport.java
+28
View File
@@ -81,6 +81,9 @@ public class EdiTransport extends QRecordEntity
@QField(backendName = "mdn_signed")
private Boolean mdnSigned;
@QField(backendName = "weak_crypto", isEditable = false)
private Boolean weakCrypto;
@QField(backendName = "local_certificate_id", possibleValueSourceName = "edi_certificate")
private Integer localCertificateId;
@@ -1583,4 +1586,29 @@ public class EdiTransport extends QRecordEntity
return (this);
}
/*******************************************************************************
** Get weakCrypto: true when SHA1 or 3DES is configured. Set by the qbit on save.
*******************************************************************************/
public Boolean getWeakCrypto()
{
return (weakCrypto);
}
/*******************************************************************************
** Set weakCrypto.
*******************************************************************************/
public void setWeakCrypto(Boolean weakCrypto)
{
this.weakCrypto = weakCrypto;
}
/*******************************************************************************
** Set weakCrypto fluently.
*******************************************************************************/
public EdiTransport withWeakCrypto(Boolean weakCrypto)
{
this.weakCrypto = weakCrypto;
return (this);
}
}
src/main/java/com/kingsrook/qbits/edi/processes/EdiProcesses.java
+25 -6
View File
@@ -95,11 +95,13 @@ public final class EdiProcesses
result.add(action(table, "cancel", "Cancel", "edi.cancel"));
}
result.add(action("edi_transport", "test", "Test Connection", "edi.transport.test"));
result.add(action("edi_certificate", "promote", "Promote Certificate", "edi.certificate.promote"));
result.add(action("edi_certificate", "export", "Export Public Certificate", "edi.certificate.read"));
for(String table : config.getSendAsEdiTables())
{
result.add(action(table, "send", "Send As EDI", "edi.send"));
}
for(String name : List.of("edi_outbound_worker", "edi_send_worker", "edi_ack_monitor", "edi_outbound_enqueue"))
for(String name : List.of("edi_outbound_worker", "edi_send_worker", "edi_as2_mdn_sender", "edi_ack_monitor", "edi_outbound_enqueue"))
{
result.add(OutboundProcesses.process(name, config, name.equals("edi_outbound_enqueue") ? null : name.equals("edi_ack_monitor") ? 300 : 15, null));
}
@@ -135,6 +137,9 @@ public final class EdiProcesses
}
process.getFrontendStep("review").withFormField(new QFieldMetaData("sampleUpload", QFieldType.BLOB).withLabel("Upload synthetic sample (optional)")
.withFieldAdornment(com.kingsrook.qqq.backend.core.model.metadata.fields.AdornmentType.FileUploadAdornment.newFieldAdornment()));
}
if(operation.equals("preview") || operation.equals("export"))
{
process.getFrontendStep("result").withComponent(new com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendComponentMetaData()
.withType(com.kingsrook.qqq.backend.core.model.metadata.processes.QComponentType.DOWNLOAD_FORM));
for(String field : List.of("downloadFileName", "storageTableName", "storageReference"))
@@ -204,7 +209,16 @@ public final class EdiProcesses
var query = new com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput(table).withFilter(EdiRecords.equal("id", selected.getValue("id"))).withShouldFetchHeavyFields(true);
query.setPermissionEnforcement(com.kingsrook.qqq.backend.core.actions.permissions.PermissionEnforcement.ENFORCE);
QRecord row = new com.kingsrook.qqq.backend.core.actions.tables.QueryAction().execute(query).getRecords().getFirst();
if(operation.equals("preview"))
if(operation.equals("promote") && !"NEXT".equals(row.getValueString("status")))
{
lines.add(new ProcessSummaryLine(Status.WARNING, 1, "only a NEXT certificate can be promoted"));
continue;
}
if(operation.equals("export"))
{
lines.add(new ProcessSummaryLine(Status.OK, 1, "public certificate exported"));
}
else if(operation.equals("preview"))
{
QRecord map = table.equals("edi_map") ? row : EdiRecords.query("edi_map", EdiRecords.equal("id", row.getValue("mapId"))).getFirst();
List<QRecord> samples = table.equals("edi_map_sample") ? List.of(row) : EdiRecords.query("edi_map_sample", EdiRecords.equal("mapId", row.getValue("id")));
@@ -265,6 +279,10 @@ public final class EdiProcesses
.test(row.toEntity(com.kingsrook.qbits.edi.model.EdiTransport.class));
lines.add(new ProcessSummaryLine(Boolean.TRUE.equals(test.success()) ? Status.OK : Status.ERROR, 1, test.summary()));
}
else if(operation.equals("promote"))
{
new com.kingsrook.qbits.edi.as2.CertificateService().promote(row);
}
else if(List.of("assign", "ignore", "resolve").contains(operation))
{
QRecord update = new QRecord().withValue("id", row.getValue("id")).withValue("status", operation.equals("assign") ? "IN_PROGRESS" : operation.equals("ignore") ? "IGNORED" : "RESOLVED");
@@ -286,14 +304,15 @@ public final class EdiProcesses
}));
lines.add(new ProcessSummaryLine(Status.OK, 1, "record processed"));
}
if(operation.equals("preview"))
if(operation.equals("preview") || operation.equals("export"))
{
String json = new com.fasterxml.jackson.databind.ObjectMapper().findAndRegisterModules().writeValueAsString(lines);
boolean export = operation.equals("export");
String content = export ? row.getValueString("certificatePem") : new com.fasterxml.jackson.databind.ObjectMapper().findAndRegisterModules().writeValueAsString(lines);
var ref = SystemWrites.call(() -> EdiRuntime.current().payloads().put(com.kingsrook.qbits.edi.storage.PayloadKind.SAMPLE,
new java.io.ByteArrayInputStream(json.getBytes(java.nio.charset.StandardCharsets.UTF_8))));
new java.io.ByteArrayInputStream(content.getBytes(java.nio.charset.StandardCharsets.UTF_8))));
var storage = new com.kingsrook.qqq.backend.core.model.actions.tables.storage.StorageInput(com.kingsrook.qbits.edi.storage.PreviewDownloadBackend.TABLE).withReference(ref.value());
com.kingsrook.qqq.backend.core.actions.processes.ProcessFileDownload.registerStorage(storage);
output.addValue("downloadFileName", "edi-map-preview.json");
output.addValue("downloadFileName", export ? row.getValueString("alias").replaceAll("[^A-Za-z0-9._-]", "-") + ".pem" : "edi-map-preview.json");
output.addValue("storageTableName", storage.getTableName());
output.addValue("storageReference", ref.value());
}
src/main/java/com/kingsrook/qbits/edi/processes/OutboundProcesses.java
+4
View File
@@ -106,6 +106,10 @@ public class OutboundProcesses implements BackendStep
{
new SendWorker().runOnce();
}
else if(name.equals("edi_as2_mdn_sender"))
{
new com.kingsrook.qbits.edi.as2.As2MdnSender().runOnce();
}
else if(name.equals("edi_ack_monitor"))
{
new AckMonitor().runOnce();
src/main/java/com/kingsrook/qbits/edi/transport/UnavailableTransport.java
+4 -3
View File
@@ -26,7 +26,8 @@ import com.kingsrook.qbits.edi.model.EdiTransport;
import com.kingsrook.qbits.edi.storage.PayloadRef;
/*******************************************************************************
** Explicit phase boundaries for AS2 and the unpublished email API.
** Explicit boundary for the unpublished qbit-email API; AS2 is implemented by
** As2Transport.
*******************************************************************************/
public record UnavailableTransport(String type) implements EdiTransportPlugin
{
@@ -43,7 +44,7 @@ public record UnavailableTransport(String type) implements EdiTransportPlugin
*******************************************************************************/
public void validate(EdiTransport transport, List<String> errors)
{
if("AS2".equals(type) ? transport.getAs2Url() == null : transport.getEmailTo() == null)
if(transport.getEmailTo() == null)
{
errors.add("Transport destination is required");
}
@@ -54,6 +55,6 @@ public record UnavailableTransport(String type) implements EdiTransportPlugin
*******************************************************************************/
public EdiSendResult send(EdiOutboundMessage message, PayloadRef payload, EdiTransport transport)
{
return (new EdiSendResult("FAILED", null, "Not available until phase c / qbit-email API"));
return (new EdiSendResult("FAILED", null, "Not available until the qbit-email API ships"));
}
}
src/test/java/com/kingsrook/qbits/edi/As2Support.java
+596
View File
@@ -0,0 +1,596 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.edi;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.math.BigInteger;
import java.net.InetSocketAddress;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.charset.StandardCharsets;
import java.security.KeyPair;
import java.security.KeyPairGenerator;
import java.security.MessageDigest;
import java.security.PrivateKey;
import java.security.Security;
import java.security.cert.X509Certificate;
import java.time.Instant;
import java.util.ArrayList;
import java.util.Base64;
import java.util.Date;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.concurrent.atomic.AtomicReference;
import com.kingsrook.qbits.edi.security.EdiSecrets;
import com.sun.net.httpserver.HttpServer;
import jakarta.activation.DataHandler;
import jakarta.mail.internet.InternetHeaders;
import jakarta.mail.internet.MimeBodyPart;
import jakarta.mail.internet.MimeMultipart;
import jakarta.mail.internet.MimeUtility;
import jakarta.mail.util.ByteArrayDataSource;
import org.bouncycastle.asn1.x500.X500Name;
import org.bouncycastle.cert.X509v3CertificateBuilder;
import org.bouncycastle.cert.jcajce.JcaCertStore;
import org.bouncycastle.cert.jcajce.JcaX509CertificateConverter;
import org.bouncycastle.cert.jcajce.JcaX509v3CertificateBuilder;
import org.bouncycastle.cms.CMSAlgorithm;
import org.bouncycastle.cms.SignerInformation;
import org.bouncycastle.cms.jcajce.JcaSimpleSignerInfoGeneratorBuilder;
import org.bouncycastle.cms.jcajce.JcaSimpleSignerInfoVerifierBuilder;
import org.bouncycastle.cms.jcajce.JceCMSContentEncryptorBuilder;
import org.bouncycastle.cms.jcajce.JceKeyTransEnvelopedRecipient;
import org.bouncycastle.cms.jcajce.JceKeyTransRecipientId;
import org.bouncycastle.cms.jcajce.JceKeyTransRecipientInfoGenerator;
import org.bouncycastle.jce.provider.BouncyCastleProvider;
import org.bouncycastle.mail.smime.SMIMEEnveloped;
import org.bouncycastle.mail.smime.SMIMEEnvelopedGenerator;
import org.bouncycastle.mail.smime.SMIMESigned;
import org.bouncycastle.mail.smime.SMIMESignedGenerator;
import org.bouncycastle.operator.jcajce.JcaContentSignerBuilder;
/*******************************************************************************
** Synthetic in-memory certificates and a BouncyCastle-only AS2 peer, written
** independently of the production classes so that both directions are tested
** against a second implementation. Nothing here touches the filesystem.
*******************************************************************************/
public final class As2Support
{
public static final Map<String, String> ENV = new ConcurrentHashMap<>();
private static final Map<String, Party> PARTIES = new ConcurrentHashMap<>();
/*******************************************************************************
** A synthetic self-signed certificate and its key pair.
*******************************************************************************/
public record Party(KeyPair pair, X509Certificate certificate)
{
/*******************************************************************************
** PEM of the public certificate.
*******************************************************************************/
public String certificatePem()
{
try
{
return ("-----BEGIN CERTIFICATE-----\n" + Base64.getMimeEncoder(64, "\n".getBytes(StandardCharsets.US_ASCII)).encodeToString(certificate.getEncoded()) + "\n-----END CERTIFICATE-----\n");
}
catch(Exception e)
{
throw new IllegalStateException(e);
}
}
/*******************************************************************************
** PEM PKCS#8 of the private key (synthetic, memory only).
*******************************************************************************/
public String keyPem()
{
return ("-----BEGIN PRIVATE KEY-----\n" + Base64.getMimeEncoder(64, "\n".getBytes(StandardCharsets.US_ASCII)).encodeToString(pair.getPrivate().getEncoded()) + "\n-----END PRIVATE KEY-----\n");
}
/*******************************************************************************
** The private key.
*******************************************************************************/
public PrivateKey key()
{
return (pair.getPrivate());
}
}
/*******************************************************************************
** A signed or unsigned reply from the peer or from the system under test.
*******************************************************************************/
public record Reply(int status, Map<String, String> headers, byte[] body)
{
/*******************************************************************************
** Body as text.
*******************************************************************************/
public String text()
{
return (new String(body, StandardCharsets.ISO_8859_1));
}
}
/*******************************************************************************
** What a peer learned from an MDN.
*******************************************************************************/
public record PeerMdn(String disposition, String originalMessageId, String receivedMic, Boolean signatureValid, String raw)
{
}
/*******************************************************************************
** A message the peer server received and unwrapped.
*******************************************************************************/
public record PeerReceived(String messageId, byte[] payload, String mic, boolean signed, boolean encrypted)
{
}
/*******************************************************************************
** Utility class.
*******************************************************************************/
private As2Support()
{
}
/*******************************************************************************
** A named synthetic party, created once per test run. Validity runs from one
** day ago for the given number of days.
*******************************************************************************/
public static Party party(String name, long validDays)
{
return (PARTIES.computeIfAbsent(name + ":" + validDays, k -> create(name, Instant.now().minusSeconds(86400), Instant.now().plusSeconds(validDays * 86400))));
}
/*******************************************************************************
** Generate an RSA 2048 key pair and a self-signed certificate.
*******************************************************************************/
public static Party create(String name, Instant notBefore, Instant notAfter)
{
try
{
Security.addProvider(new BouncyCastleProvider());
KeyPairGenerator generator = KeyPairGenerator.getInstance("RSA");
generator.initialize(2048);
KeyPair pair = generator.generateKeyPair();
X500Name subject = new X500Name("CN=" + name + ",O=Synthetic Test");
X509v3CertificateBuilder builder = new JcaX509v3CertificateBuilder(subject, BigInteger.valueOf(System.nanoTime()), Date.from(notBefore), Date.from(notAfter), subject, pair.getPublic());
X509Certificate certificate = new JcaX509CertificateConverter().getCertificate(builder.build(new JcaContentSignerBuilder("SHA256withRSA").build(pair.getPrivate())));
return (new Party(pair, certificate));
}
catch(Exception e)
{
throw new IllegalStateException(e);
}
}
/*******************************************************************************
** Resolver for environment variables held only in memory; the payload key is
** always available.
*******************************************************************************/
public static EdiSecrets secrets()
{
return (name -> name.startsWith("QQQ_EDI_PAYLOAD") ? Base64.getEncoder().encodeToString(new byte[32]) : ENV.get(name));
}
/*******************************************************************************
** Base64 SHA-256 of bytes.
*******************************************************************************/
public static String sha256(byte[] bytes) throws Exception
{
return (Base64.getEncoder().encodeToString(MessageDigest.getInstance("SHA-256").digest(bytes)));
}
/*******************************************************************************
** Send an AS2 message built with BouncyCastle only.
*******************************************************************************/
public static Reply peerSend(String url, Map<String, String> ids, String messageId, byte[] payload, Party signer, X509Certificate encryptTo, boolean base64, boolean requestMdn, boolean signedMdn,
String asyncUrl) throws Exception
{
MimeBodyPart part = new MimeBodyPart();
part.setDataHandler(new DataHandler(new ByteArrayDataSource(payload, "application/edi-x12")));
part.setHeader("Content-Type", "application/edi-x12");
part.setHeader("Content-Transfer-Encoding", base64 ? "base64" : "binary");
part.setHeader("Content-Disposition", "attachment; filename=\"peer.edi\"");
String mic = Base64.getEncoder().encodeToString(MessageDigest.getInstance("SHA-256").digest(serialize(part))) + ", sha-256";
MimeBodyPart outer = part;
if(signer != null)
{
SMIMESignedGenerator generator = new SMIMESignedGenerator();
generator.addSignerInfoGenerator(new JcaSimpleSignerInfoGeneratorBuilder().setProvider("BC").build("SHA256withRSA", signer.key(), signer.certificate()));
generator.addCertificates(new JcaCertStore(List.of(signer.certificate())));
MimeMultipart multipart = generator.generate(part);
outer = new MimeBodyPart();
outer.setContent(multipart);
outer.setHeader("Content-Type", multipart.getContentType());
}
if(encryptTo != null)
{
SMIMEEnvelopedGenerator generator = new SMIMEEnvelopedGenerator();
generator.addRecipientInfoGenerator(new JceKeyTransRecipientInfoGenerator(encryptTo).setProvider("BC"));
outer = generator.generate(outer, new JceCMSContentEncryptorBuilder(CMSAlgorithm.AES256_CBC).setProvider("BC").build());
}
byte[] wire = serialize(outer);
int split = indexOf(wire, "\r\n\r\n".getBytes(StandardCharsets.US_ASCII));
HttpRequest.Builder request = HttpRequest.newBuilder(URI.create(url)).POST(HttpRequest.BodyPublishers.ofByteArray(java.util.Arrays.copyOfRange(wire, split + 4, wire.length)))
.header("Content-Type", MimeUtility.unfold(outer.getHeader("Content-Type", null)));
if(outer.getHeader("Content-Transfer-Encoding", null) != null)
{
request.header("Content-Transfer-Encoding", outer.getHeader("Content-Transfer-Encoding", null));
}
ids.forEach(request::header);
request.header("AS2-Version", "1.2").header("Message-ID", "<" + messageId + ">");
if(requestMdn)
{
request.header("Disposition-Notification-To", "peer@example.invalid");
if(signedMdn)
{
request.header("Disposition-Notification-Options", "signed-receipt-protocol=required, pkcs7-signature; signed-receipt-micalg=required, sha-256");
}
if(asyncUrl != null)
{
request.header("Receipt-Delivery-Option", asyncUrl);
}
}
try(HttpClient client = HttpClient.newHttpClient())
{
HttpResponse<byte[]> response = client.send(request.build(), HttpResponse.BodyHandlers.ofByteArray());
Map<String, String> headers = new LinkedHashMap<>();
response.headers().map().forEach((k, v) -> headers.put(k.toLowerCase(java.util.Locale.ROOT), v.getFirst()));
headers.put("x-sent-mic", mic);
return (new Reply(response.statusCode(), headers, response.body()));
}
}
/*******************************************************************************
** Read an MDN with BouncyCastle, verifying a signature with the given
** certificate when the MDN is signed.
*******************************************************************************/
public static PeerMdn peerReadMdn(Reply reply, X509Certificate signer) throws Exception
{
String contentType = reply.headers().get("content-type");
byte[] body = reply.body();
Boolean valid = null;
MimeMultipart report;
if(contentType.toLowerCase(java.util.Locale.ROOT).startsWith("multipart/signed"))
{
SMIMESigned signed = new SMIMESigned(new MimeMultipart(new ByteArrayDataSource(body, contentType)));
valid = false;
for(SignerInformation info : signed.getSignerInfos().getSigners())
{
valid = info.verify(new JcaSimpleSignerInfoVerifierBuilder().setProvider("BC").build(signer));
}
MimeBodyPart content = signed.getContent();
report = new MimeMultipart(new ByteArrayDataSource(content.getRawInputStream().readAllBytes(), MimeUtility.unfold(content.getContentType())));
}
else
{
report = new MimeMultipart(new ByteArrayDataSource(body, contentType));
}
for(int i = 0; i < report.getCount(); i++)
{
if(report.getBodyPart(i).getContentType().toLowerCase(java.util.Locale.ROOT).startsWith("message/disposition-notification"))
{
InternetHeaders fields = new InternetHeaders(new ByteArrayInputStream(report.getBodyPart(i).getInputStream().readAllBytes()));
String[] mic = fields.getHeader("Received-Content-MIC");
return (new PeerMdn(fields.getHeader("Disposition")[0].trim(), fields.getHeader("Original-Message-ID")[0].trim(), mic == null ? null : mic[0].trim(), valid, new String(body, StandardCharsets.ISO_8859_1)));
}
}
throw new IllegalStateException("no disposition notification");
}
/*******************************************************************************
** A BouncyCastle AS2 receiver on a random loopback port. It records what it
** decrypts and verifies, and answers with an MDN it builds itself.
*******************************************************************************/
public static final class PeerServer implements AutoCloseable
{
public final HttpServer server;
public final List<PeerReceived> received = new CopyOnWriteArrayList<>();
public final List<Map<String, String>> requests = new CopyOnWriteArrayList<>();
public final AtomicReference<String> disposition = new AtomicReference<>("automatic-action/MDN-sent-automatically; processed");
public final AtomicReference<Boolean> corruptMic = new AtomicReference<>(false);
public final AtomicReference<Boolean> signMdn = new AtomicReference<>(true);
public final AtomicReference<Boolean> async = new AtomicReference<>(false);
public final AtomicReference<Boolean> silent = new AtomicReference<>(false);
public final AtomicReference<Reply> asyncResult = new AtomicReference<>();
public final List<Party> decryptionKeys = new CopyOnWriteArrayList<>();
public final List<X509Certificate> senderCertificates = new CopyOnWriteArrayList<>();
public volatile Party self;
/*******************************************************************************
** Start listening; self signs MDNs and decrypts messages.
*******************************************************************************/
public PeerServer(Party self, X509Certificate sender) throws Exception
{
this.self = self;
decryptionKeys.add(self);
senderCertificates.add(sender);
server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
server.createContext("/as2", exchange ->
{
try
{
if("OPTIONS".equals(exchange.getRequestMethod()))
{
exchange.sendResponseHeaders(200, -1);
return;
}
byte[] body = exchange.getRequestBody().readAllBytes();
Map<String, String> headers = new LinkedHashMap<>();
exchange.getRequestHeaders().forEach((k, v) -> headers.put(k.toLowerCase(java.util.Locale.ROOT), v.getFirst()));
requests.add(headers);
byte[] response = handle(headers, body);
if(response == null)
{
exchange.sendResponseHeaders(200, -1);
}
else
{
int split = indexOf(response, "\r\n\r\n".getBytes(StandardCharsets.US_ASCII));
String head = new String(response, 0, split, StandardCharsets.ISO_8859_1);
for(String line : head.split("\r\n"))
{
if(line.toLowerCase(java.util.Locale.ROOT).startsWith("content-type:"))
{
exchange.getResponseHeaders().add("Content-Type", line.substring(13).trim());
}
}
exchange.getResponseHeaders().add("AS2-Version", "1.2");
exchange.getResponseHeaders().add("Message-ID", "<peer-mdn-" + System.nanoTime() + "@peer>");
byte[] out = java.util.Arrays.copyOfRange(response, split + 4, response.length);
exchange.sendResponseHeaders(200, out.length);
exchange.getResponseBody().write(out);
}
}
catch(Exception e)
{
exchange.sendResponseHeaders(500, -1);
}
finally
{
exchange.close();
}
});
server.start();
}
/*******************************************************************************
** URL of the AS2 endpoint.
*******************************************************************************/
public String url()
{
return ("http://127.0.0.1:" + server.getAddress().getPort() + "/as2");
}
/*******************************************************************************
** Unwrap, verify and answer one message.
*******************************************************************************/
private byte[] handle(Map<String, String> headers, byte[] body) throws Exception
{
String contentType = headers.get("content-type");
String encoding = headers.get("content-transfer-encoding");
MimeBodyPart part = new MimeBodyPart(new ByteArrayInputStream(concat(("Content-Type: " + contentType + "\r\n" + (encoding == null ? "" : "Content-Transfer-Encoding: " + encoding + "\r\n") + "\r\n").getBytes(StandardCharsets.ISO_8859_1), body)));
boolean encrypted = false;
boolean signed = false;
if(part.getContentType().toLowerCase(java.util.Locale.ROOT).startsWith("application/pkcs7-mime"))
{
SMIMEEnveloped enveloped = new SMIMEEnveloped(part);
byte[] decrypted = null;
for(Party key : decryptionKeys)
{
var recipient = enveloped.getRecipientInfos().get(new JceKeyTransRecipientId(key.certificate()));
if(recipient != null)
{
decrypted = recipient.getContent(new JceKeyTransEnvelopedRecipient(key.key()).setProvider("BC"));
}
}
if(decrypted == null)
{
throw new IllegalStateException("not encrypted to this peer");
}
part = new MimeBodyPart(new ByteArrayInputStream(decrypted));
encrypted = true;
}
MimeBodyPart content = part;
if(part.getContentType().toLowerCase(java.util.Locale.ROOT).startsWith("multipart/signed"))
{
SMIMESigned smime = new SMIMESigned(new MimeMultipart(new ByteArrayDataSource(part.getRawInputStream().readAllBytes(), MimeUtility.unfold(part.getContentType()))));
boolean verified = false;
for(SignerInformation info : smime.getSignerInfos().getSigners())
{
for(X509Certificate certificate : senderCertificates)
{
verified |= info.verify(new JcaSimpleSignerInfoVerifierBuilder().setProvider("BC").build(certificate));
}
}
if(!verified)
{
throw new IllegalStateException("bad signature");
}
content = smime.getContent();
signed = true;
}
String mic = Base64.getEncoder().encodeToString(MessageDigest.getInstance("SHA-256").digest(serialize(content))) + ", sha-256";
String messageId = headers.get("message-id").replaceAll("[<>]", "");
received.add(new PeerReceived(messageId, content.getInputStream().readAllBytes(), mic, signed, encrypted));
String reportedMic = corruptMic.get() ? Base64.getEncoder().encodeToString(new byte[32]) + ", sha-256" : mic;
byte[] mdn = mdn(messageId, disposition.get(), reportedMic, signMdn.get() && headers.containsKey("disposition-notification-options"));
if(silent.get())
{
return (null);
}
if(async.get())
{
String target = headers.get("receipt-delivery-option");
Thread.startVirtualThread(() ->
{
try
{
asyncResult.set(postMdn(target, mdn, headers));
}
catch(Exception e)
{
asyncResult.set(new Reply(-1, Map.of(), e.toString().getBytes(StandardCharsets.UTF_8)));
}
});
return (null);
}
return (mdn);
}
/*******************************************************************************
** Post an asynchronous MDN to the address from the request.
*******************************************************************************/
private Reply postMdn(String target, byte[] mdn, Map<String, String> request) throws Exception
{
int split = indexOf(mdn, "\r\n\r\n".getBytes(StandardCharsets.US_ASCII));
String head = new String(mdn, 0, split, StandardCharsets.ISO_8859_1);
String contentType = head.lines().filter(l -> l.toLowerCase(java.util.Locale.ROOT).startsWith("content-type:")).findFirst().orElseThrow().substring(13).trim();
HttpRequest.Builder builder = HttpRequest.newBuilder(URI.create(target)).header("Content-Type", contentType).header("AS2-From", request.get("as2-to")).header("AS2-To", request.get("as2-from"))
.header("AS2-Version", "1.2").header("Message-ID", "<peer-async-" + System.nanoTime() + "@peer>")
.POST(HttpRequest.BodyPublishers.ofByteArray(java.util.Arrays.copyOfRange(mdn, split + 4, mdn.length)));
try(HttpClient client = HttpClient.newHttpClient())
{
HttpResponse<byte[]> response = client.send(builder.build(), HttpResponse.BodyHandlers.ofByteArray());
return (new Reply(response.statusCode(), Map.of(), response.body()));
}
}
/*******************************************************************************
** An MDN entity (headers and body), built with BouncyCastle.
*******************************************************************************/
public byte[] mdn(String originalMessageId, String disposition, String mic, boolean signed) throws Exception
{
MimeMultipart report = new MimeMultipart("report; report-type=disposition-notification");
MimeBodyPart text = new MimeBodyPart();
text.setText("Received by the synthetic peer", "us-ascii");
report.addBodyPart(text);
InternetHeaders headers = new InternetHeaders();
headers.addHeader("Content-Type", "message/disposition-notification");
headers.addHeader("Content-Transfer-Encoding", "7bit");
String fields = "Reporting-UA: synthetic-peer\r\nOriginal-Recipient: rfc822; peer\r\nFinal-Recipient: rfc822; peer\r\nOriginal-Message-ID: <" + originalMessageId + ">\r\nDisposition: " + disposition + "\r\n"
+ (disposition.contains("error") ? "" : "Received-Content-MIC: " + mic + "\r\n");
report.addBodyPart(new MimeBodyPart(headers, fields.getBytes(StandardCharsets.US_ASCII)));
MimeBodyPart outer = new MimeBodyPart();
outer.setContent(report);
outer.setHeader("Content-Type", report.getContentType());
if(signed)
{
SMIMESignedGenerator generator = new SMIMESignedGenerator();
generator.addSignerInfoGenerator(new JcaSimpleSignerInfoGeneratorBuilder().setProvider("BC").build("SHA256withRSA", self.key(), self.certificate()));
MimeMultipart multipart = generator.generate(outer);
outer = new MimeBodyPart();
outer.setContent(multipart);
outer.setHeader("Content-Type", multipart.getContentType());
}
return (serialize(outer));
}
/*******************************************************************************
** Stop the server.
*******************************************************************************/
@Override
public void close()
{
server.stop(0);
}
}
/*******************************************************************************
** POST a MIME entity (headers and body) the way an AS2 partner does: the entity
** Content-Type becomes the HTTP Content-Type and the body the HTTP body.
*******************************************************************************/
public static Reply post(String url, byte[] entity, Map<String, String> headers) throws Exception
{
int split = indexOf(entity, "\r\n\r\n".getBytes(StandardCharsets.US_ASCII));
String head = new String(entity, 0, split, StandardCharsets.ISO_8859_1);
HttpRequest.Builder request = HttpRequest.newBuilder(URI.create(url)).POST(HttpRequest.BodyPublishers.ofByteArray(java.util.Arrays.copyOfRange(entity, split + 4, entity.length)));
for(String line : head.split("\r\n"))
{
if(line.toLowerCase(java.util.Locale.ROOT).startsWith("content-type:"))
{
request.header("Content-Type", line.substring(13).trim());
}
}
headers.forEach(request::header);
try(HttpClient client = HttpClient.newHttpClient())
{
HttpResponse<byte[]> response = client.send(request.build(), HttpResponse.BodyHandlers.ofByteArray());
Map<String, String> responseHeaders = new LinkedHashMap<>();
response.headers().map().forEach((k, v) -> responseHeaders.put(k.toLowerCase(java.util.Locale.ROOT), v.getFirst()));
return (new Reply(response.statusCode(), responseHeaders, response.body()));
}
}
/*******************************************************************************
** Serialize a MIME entity to bytes.
*******************************************************************************/
public static byte[] serialize(MimeBodyPart part) throws Exception
{
ByteArrayOutputStream out = new ByteArrayOutputStream();
part.writeTo(out);
return (out.toByteArray());
}
/*******************************************************************************
** Concatenate byte arrays.
*******************************************************************************/
public static byte[] concat(byte[] first, byte[] second)
{
byte[] all = new byte[first.length + second.length];
System.arraycopy(first, 0, all, 0, first.length);
System.arraycopy(second, 0, all, first.length, second.length);
return (all);
}
/*******************************************************************************
** Index of a sequence.
*******************************************************************************/
public static int indexOf(byte[] data, byte[] pattern)
{
for(int i = 0; i <= data.length - pattern.length; i++)
{
if(java.util.Arrays.equals(data, i, i + pattern.length, pattern, 0, pattern.length))
{
return (i);
}
}
return (-1);
}
/*******************************************************************************
** Names of everything directly under the JVM temp directory.
*******************************************************************************/
public static List<String> tempNames() throws Exception
{
List<String> names = new ArrayList<>();
try(var stream = java.nio.file.Files.list(java.nio.file.Path.of(System.getProperty("java.io.tmpdir"))))
{
stream.forEach(p -> names.add(p.getFileName().toString()));
}
return (names);
}
}
src/test/java/com/kingsrook/qbits/edi/As2Test.java
+1143
View File
File diff suppressed because it is too large. Load diff
src/test/java/com/kingsrook/qbits/edi/ContingenciesTest.java
+2 -1
View File
@@ -114,7 +114,8 @@ class ContingenciesTest
{
EdiRecords.insert("edi_identity", new QRecord().withValue("partnerId", partner).withValue("standard", "X12").withValue("interchangeQualifier", "ZZ")
.withValue("interchangeId", "SYNTH" + partner).withValue("applicationCode", "APP" + partner));
EdiRecords.insert("edi_certificate", new QRecord().withValue("partnerId", partner).withValue("alias", "Synthetic " + partner));
EdiRecords.insert("edi_certificate", new QRecord().withValue("partnerId", partner).withValue("alias", "Synthetic " + partner)
.withValue("certificatePem", As2Support.party("contingency-" + partner, 365).certificatePem()));
}
return (null);
});
src/test/java/com/kingsrook/qbits/edi/DatabaseTest.java
+1 -1
View File
@@ -110,7 +110,7 @@ class DatabaseTest
void metadata() throws Exception
{
QInstance instance = H2Support.create();
assertThat(instance.getTables()).hasSize(27);
assertThat(instance.getTables()).hasSize(28);
for(QTableMetaData table : instance.getTables().values())
{
if(table.getName().equals("edi_preview_download"))
src/test/java/com/kingsrook/qbits/edi/EdiPresentationTest.java
+8 -3
View File
@@ -60,14 +60,14 @@ class EdiPresentationTest
{
var instance = H2Support.create();
Set<String> externalIds = Set.of("interchangeId", "senderId", "receiverId", "elementId", "segmentId", "loopId", "parentLoopId",
"as2LocalId", "as2PartnerId", "externalId", "transportMessageId", "eventId", "sourceRecordId", "targetRecordId", "keyId", "gatewayUserId", "assignedToUserId");
"as2LocalId", "as2PartnerId", "messageId", "originalMessageId", "externalId", "transportMessageId", "eventId", "sourceRecordId", "targetRecordId", "keyId", "gatewayUserId", "assignedToUserId");
for(var table : instance.getTables().values())
{
assertThat(table.getLabel()).as(table.getName()).matches("[A-Z][A-Za-z0-9]*( [A-Z][A-Za-z0-9]*)*");
for(var field : table.getFields().values())
{
assertThat(field.getLabel()).as(table.getName() + "." + field.getName())
.doesNotContainPattern("\\b(Id|Phi|Pii|Url|Isa|Gs|St|As2|Sftp|Https|Http|Mdn|Tls|Ta1|Edi|Upc|Npi|Qqq|Json|Pem|Snip|Van)\\b");
.doesNotContainPattern("\\b(Id|Phi|Pii|Url|Isa|Gs|St|As2|Sftp|Https|Http|Mdn|Mic|Tls|Ta1|Edi|Upc|Npi|Qqq|Json|Pem|Snip|Van)\\b");
if(field.getName().endsWith("Id") && !(field.getType() == QFieldType.STRING && externalIds.contains(field.getName())))
{
assertThat(field.getPossibleValueSourceName()).as(table.getName() + "." + field.getName()).isNotBlank();
@@ -85,6 +85,9 @@ class EdiPresentationTest
}
assertThat(instance.getTable("edi_exception").getField("assignedToUserId").getLabel()).isEqualTo("Assigned To");
assertThat(instance.getTable("edi_exception").getField("outboundMessageId").getPossibleValueSourceName()).isEqualTo("edi_outbound_message");
assertThat(EdiTables.childWidgets()).extracting(widget -> widget.getLabel()).contains("AS2 MDNs");
assertThat(instance.getTable("edi_as2_mdn").getLabel()).isEqualTo("AS2 MDN");
assertThat(instance.getTable("edi_as2_mdn").getField("micValue").getLabel()).isEqualTo("MIC Value");
assertThat(EdiTables.childWidgets()).allSatisfy(widget -> assertThat(widget.getLabel()).doesNotContain("(", "_", "transaction", "identity"));
}
@@ -215,13 +218,15 @@ class EdiPresentationTest
case "edi_inbox" -> "SFTP demo-message";
case "edi_outbound_message" -> "Demo SFTP SENT";
case "edi_map", "edi_guide" -> "Retail r2";
case "edi_as2_mdn" -> "INBOUND MDN <synthetic-message@peer>";
default -> null;
};
if(expected != null)
{
QRecord row = new QRecord().withValue("direction", "INBOUND").withValue("controlNumber", "000001").withValue("type", "VALIDATION")
.withValue("summary", "Synthetic summary").withValue("transportType", "SFTP").withValue("externalId", "demo-message")
.withValue("fileName", "SYNTHETIC-PRIVATE-FILENAME").withValue("transportId", 1).withValue("status", "SENT").withValue("name", "Retail").withValue("revision", 2);
.withValue("fileName", "SYNTHETIC-PRIVATE-FILENAME").withValue("transportId", 1).withValue("status", "SENT").withValue("name", "Retail").withValue("revision", 2)
.withValue("originalMessageId", "<synthetic-message@peer>");
row.setDisplayValue("transportId", "Demo SFTP");
assertThat(QValueFormatter.formatRecordLabel(table, row)).as(table.getName()).isEqualTo(expected);
}
src/test/java/com/kingsrook/qbits/edi/EntityContractTest.java
+1 -1
View File
@@ -64,7 +64,7 @@ class EntityContractTest
*******************************************************************************/
static Stream<Class<? extends QRecordEntity>> entities()
{
return (Stream.of(EdiPartner.class, EdiIdentity.class, EdiAgreement.class, EdiAgreementDocument.class, EdiControlNumber.class, EdiTransport.class, EdiCertificate.class, EdiGuide.class, EdiGuideSegment.class, EdiGuideElement.class, EdiGuideRule.class, EdiCodeList.class, EdiCodeListValue.class, EdiInterchange.class, EdiFunctionalGroup.class, EdiTransaction.class, EdiTransactionError.class, EdiEvent.class, EdiException.class, EdiInbox.class, EdiPayload.class, com.kingsrook.qbits.edi.model.EdiMap.class, com.kingsrook.qbits.edi.model.EdiMapRule.class, com.kingsrook.qbits.edi.model.EdiMapSample.class, com.kingsrook.qbits.edi.model.EdiOutboundRequest.class, com.kingsrook.qbits.edi.model.EdiOutboundMessage.class));
return (Stream.of(EdiPartner.class, EdiIdentity.class, EdiAgreement.class, EdiAgreementDocument.class, EdiControlNumber.class, EdiTransport.class, EdiCertificate.class, EdiGuide.class, EdiGuideSegment.class, EdiGuideElement.class, EdiGuideRule.class, EdiCodeList.class, EdiCodeListValue.class, EdiInterchange.class, EdiFunctionalGroup.class, EdiTransaction.class, EdiTransactionError.class, EdiEvent.class, EdiException.class, EdiInbox.class, EdiPayload.class, com.kingsrook.qbits.edi.model.EdiMap.class, com.kingsrook.qbits.edi.model.EdiMapRule.class, com.kingsrook.qbits.edi.model.EdiMapSample.class, com.kingsrook.qbits.edi.model.EdiOutboundRequest.class, com.kingsrook.qbits.edi.model.EdiOutboundMessage.class, com.kingsrook.qbits.edi.model.EdiAs2Mdn.class));
}
/*******************************************************************************
src/test/java/com/kingsrook/qbits/edi/OutboundTest.java
+3 -2
View File
@@ -273,8 +273,9 @@ class OutboundTest
assertThat(rows("edi_exception")).filteredOn(r -> "OVERDUE_ACK".equals(r.getValueString("type"))).allMatch(r -> "RESOLVED".equals(r.getValueString("status")));
SystemWrites.call(() ->
{
SystemWrites.insert("edi_certificate", new QRecord().withValue("alias", "synthetic-expiring").withValue("status", "ACTIVE").withValue("notAfter", Instant.now().plusSeconds(6 * 86400)));
SystemWrites.insert("edi_certificate", new QRecord().withValue("alias", "synthetic-expired").withValue("status", "ACTIVE").withValue("notAfter", Instant.now().minusSeconds(1)));
SystemWrites.insert("edi_certificate", new QRecord().withValue("alias", "synthetic-expiring").withValue("status", "ACTIVE").withValue("certificatePem", As2Support.party("synthetic-expiring", 6).certificatePem()));
SystemWrites.insert("edi_certificate", new QRecord().withValue("alias", "synthetic-expired").withValue("status", "ACTIVE")
.withValue("certificatePem", As2Support.create("synthetic-expired", Instant.now().minusSeconds(2 * 86400), Instant.now().minusSeconds(1)).certificatePem()));
return (null);
});
new AckMonitor().runOnce();
src/test/java/com/kingsrook/qbits/edi/demo/EdiDemoServer.java
+120
View File
@@ -29,6 +29,7 @@ import java.util.ArrayList;
import java.util.Base64;
import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.edi.As2Support;
import com.kingsrook.qbits.edi.EdiRuntime;
import com.kingsrook.qbits.edi.H2Support;
import com.kingsrook.qbits.edi.LogisticsSupport;
@@ -73,6 +74,9 @@ public class EdiDemoServer extends AbstractQQQApplication implements AutoCloseab
private Map<String, String> authenticationModules;
private String previousAuthentication;
private final Map<String, byte[]> outbox = new java.util.concurrent.ConcurrentHashMap<>();
private final Map<String, String> secrets = new java.util.concurrent.ConcurrentHashMap<>();
private As2Support.PeerServer peer;
private Javalin hooks;
/*******************************************************************************
** Allocate a fresh database; payload keys and certificate keys stay in memory.
@@ -80,6 +84,7 @@ public class EdiDemoServer extends AbstractQQQApplication implements AutoCloseab
public EdiDemoServer() throws Exception
{
instance = H2Support.create();
EdiRuntime.current().config().withSecretsResolver(name -> secrets.getOrDefault(name, Base64.getEncoder().encodeToString(new byte[32])));
RetailSupport.tables();
LogisticsSupport.tables();
instance.setHomeAppName("edi");
@@ -239,6 +244,7 @@ public class EdiDemoServer extends AbstractQQQApplication implements AutoCloseab
installOutbox();
}
logistics(local, agreements.get(1));
seedAs2(local);
return (null);
});
}
@@ -319,6 +325,111 @@ public class EdiDemoServer extends AbstractQQQApplication implements AutoCloseab
SystemWrites.insert("edi_agreement_document", document);
}
/*******************************************************************************
** Pacific Wholesale over AS2: an in-process BouncyCastle peer on a random port,
** synthetic certificates generated in memory (keys only in the demo secrets map,
** never written anywhere), three 850s received over AS2, an 855 sent with a
** synchronous MDN and one asynchronous MDN example.
*******************************************************************************/
private void seedAs2(QRecord local) throws Exception
{
var ours = As2Support.party("demo-qqq", 365);
var oursNext = As2Support.party("demo-qqq-next", 400);
var pacific = As2Support.party("demo-pacific", 365);
secrets.put("DEMO_AS2_KEY", ours.keyPem());
secrets.put("DEMO_AS2_KEY_NEXT", oursNext.keyPem());
peer = new As2Support.PeerServer(pacific, ours.certificate());
var provider = new com.kingsrook.qbits.edi.as2.As2RouteProvider();
provider.setQInstance(instance);
hooks = Javalin.create(provider::acceptJavalinConfig).start(0);
EdiRuntime.current().config().withPublicBaseUrl("http://127.0.0.1:" + hooks.port());
QRecord partner = SystemWrites.insert("edi_partner", new QRecord().withValue("name", "Pacific Wholesale").withValue("code", "PACIFIC").withValue("contactName", "Pacific Wholesale Demo Contact")
.withValue("contactEmail", "pacific@example.invalid").withValue("contactPhone", "202-555-0150").withValue("status", "ACTIVE").withValue("defaultTimezone", "America/Los_Angeles"));
QRecord remote = SystemWrites.insert("edi_identity", new QRecord().withValue("partnerId", partner.getValue("id")).withValue("standard", "X12").withValue("interchangeQualifier", "ZZ")
.withValue("interchangeId", "PACIFICWS").withValue("applicationCode", "PACIFICWS").withValue("isActive", true));
QRecord oursActive = as2Certificate("demo-as2-active", ours, null, "ACTIVE", "DEMO_AS2_KEY");
as2Certificate("demo-as2-next", oursNext, null, "NEXT", "DEMO_AS2_KEY_NEXT");
QRecord theirs = as2Certificate("demo-pacific", pacific, partner.getValueInteger("id"), "ACTIVE", null);
QRecord sync = as2Transport("pacific-as2", "BOTH", partner, oursActive, theirs, "SYNC");
QRecord async = as2Transport("pacific-as2-async", "OUTBOUND", partner, oursActive, theirs, "ASYNC");
QRecord agreement = SystemWrites.insert("edi_agreement", new QRecord().withValue("partnerId", partner.getValue("id")).withValue("localIdentityId", local.getValue("id"))
.withValue("partnerIdentityId", remote.getValue("id")).withValue("standard", "X12").withValue("usageIndicator", "T").withValue("interchangeVersion", "00401")
.withValue("isActive", true).withValue("applyTestDocuments", true).withValue("inboundTransportId", sync.getValue("id")).withValue("outboundTransportId", sync.getValue("id")));
for(String document : List.of("850", "997"))
{
QRecord map = document.equals("997") ? null : RetailSupport.installMap(document, "004010");
QRecord guide = EdiRecords.query("edi_guide", EdiRecords.equal("transactionSet", document).withCriteria("version", com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator.EQUALS, "004010")).getFirst();
SystemWrites.insert("edi_agreement_document", new QRecord().withValue("agreementId", agreement.getValue("id")).withValue("direction", "INBOUND")
.withValue("transactionSet", document).withValue("version", "004010").withValue("guideId", guide.getValue("id")).withValue("mapId", map == null ? null : map.getValue("id"))
.withValue("ackExpectation", document.equals("997") ? "NONE" : "997").withValue("ackLevel", "ALL_TRANSACTIONS").withValue("isActive", true));
}
QRecord map855 = RetailSupport.installMap("855", "004010");
SystemWrites.insert("edi_agreement_document", new QRecord().withValue("agreementId", agreement.getValue("id")).withValue("direction", "OUTBOUND").withValue("transactionSet", "855")
.withValue("functionalIdCode", EdiRuntime.current().guides().resource("855-004010").functionalIdCode()).withValue("version", "004010").withValue("guideId", map855.getValue("guideId"))
.withValue("mapId", map855.getValue("id")).withValue("sourceTableName", map855.getValueString("tableName")).withValue("outboundTrigger", "MANUAL").withValue("batchMode", "IMMEDIATE")
.withValue("ackExpectation", "997").withValue("ackDueMinutes", 1440).withValue("isActive", true));
String hook = "http://127.0.0.1:" + hooks.port() + "/qqq/hooks/edi/as2/pacific-as2";
for(int i = 0; i < 3; i++)
{
String text = Synthetic850.generate(50 + i, "004010", X12Separators.DEFAULT).replace("SYNTHETIC", "PACIFICWS")
.replace("000000001", String.format("%09d", 300 + i)).replace("*1200*1*X*", "*1200*" + (300 + i) + "*X*").replace("GE*1*1~", "GE*1*" + (300 + i) + "~");
As2Support.peerSend(hook, Map.of("AS2-From", "PACIFIC-WS", "AS2-To", "QQQ-DEMO"), "demo-as2-" + i + "@pacific", text.getBytes(StandardCharsets.US_ASCII), pacific, ours.certificate(), i != 1, true, true, null);
}
while(new InboxWorker().runOnce() > 0)
{
// Drain the bounded inbox in the demo setup only.
}
QRecord order = SystemWrites.transaction(() -> RetailSupport.saveSource("sample_order", RetailSupport.source().withValue("poNumber", "PACIFIC-2001")));
OutboundEnqueue.enqueue("sample_order", order, "MANUAL", null);
new OutboundWorker().runOnce();
while(new SendWorker().runOnce() > 0)
{
// Deliver the 855 over AS2; its MDN is verified and recorded.
}
QRecord delivered = EdiRecords.query("edi_outbound_message", EdiRecords.equal("transportId", sync.getValue("id"))).getFirst();
peer.async.set(true);
PayloadRef copy = EdiRuntime.current().payloads().put(com.kingsrook.qbits.edi.storage.PayloadKind.OUTBOUND,
EdiRuntime.current().payloads().open(new PayloadRef(delivered.getValueString("payloadRef"), null, 0)));
QRecord example = SystemWrites.insert("edi_outbound_message", new QRecord().withValue("agreementId", agreement.getValue("id")).withValue("partnerId", partner.getValue("id"))
.withValue("transportId", async.getValue("id")).withValue("payloadRef", copy.value()).withValue("payloadSha256", copy.sha256()).withValue("sizeBytes", copy.sizeBytes())
.withValue("status", "QUEUED").withValue("attempts", 0));
new SendWorker().runOnce();
for(int i = 0; i < 100 && peer.asyncResult.get() == null; i++)
{
Thread.sleep(50);
}
peer.async.set(false);
hooks.stop();
hooks = null;
EdiRuntime.current().config().withPublicBaseUrl(null);
if(!"DELIVERED".equals(EdiRecords.query("edi_outbound_message", EdiRecords.equal("id", example.getValue("id"))).getFirst().getValueString("status")))
{
throw new IllegalStateException("The asynchronous AS2 example was not delivered");
}
}
/*******************************************************************************
** An AS2 certificate row; the qbit parses the public certificate on save.
*******************************************************************************/
private static QRecord as2Certificate(String alias, As2Support.Party party, Integer partnerId, String status, String keyVariable) throws Exception
{
return (SystemWrites.insert("edi_certificate", new QRecord().withValue("alias", alias).withValue("partnerId", partnerId).withValue("usage", "SIGNING_AND_ENCRYPTION")
.withValue("certificatePem", party.certificatePem()).withValue("privateKeyEnvVar", keyVariable).withValue("status", status)));
}
/*******************************************************************************
** An AS2 transport to the in-process peer: signed, encrypted, signed MDN.
*******************************************************************************/
private QRecord as2Transport(String name, String direction, QRecord partner, QRecord local, QRecord theirs, String mdnMode) throws Exception
{
return (SystemWrites.insert("edi_transport", new QRecord().withValue("partnerId", partner.getValue("id")).withValue("name", name).withValue("type", "AS2").withValue("direction", direction)
.withValue("isActive", true).withValue("as2LocalId", "QQQ-DEMO").withValue("as2PartnerId", "PACIFIC-WS").withValue("as2Url", peer.url()).withValue("signAlgorithm", "SHA256")
.withValue("encryptAlgorithm", "AES256_CBC").withValue("mdnMode", mdnMode).withValue("mdnSigned", true).withValue("localCertificateId", local.getValue("id"))
.withValue("partnerCertificateId", theirs.getValue("id")).withValue("maxAttempts", 3)));
}
/*******************************************************************************
** Synthetic change order retains a valid envelope and guide-compatible details.
*******************************************************************************/
@@ -391,6 +502,7 @@ public class EdiDemoServer extends AbstractQQQApplication implements AutoCloseab
new QInstanceValidator().validate(instance);
server = new QApplicationJavalinServer(this).withPort(port).withServeFrontendNext(true).withJavalinConfigurationCustomizer(app -> javalin = app);
server.start();
EdiRuntime.current().config().withPublicBaseUrl("http://localhost:" + javalin.port());
return (javalin.port());
}
@@ -409,6 +521,14 @@ public class EdiDemoServer extends AbstractQQQApplication implements AutoCloseab
authenticationModules.put(QAuthenticationType.FULLY_ANONYMOUS.getName(), previousAuthentication);
}
FileGatewayOutbox.register(null);
if(hooks != null)
{
hooks.stop();
}
if(peer != null)
{
peer.close();
}
QContext.clear();
}
src/test/java/com/kingsrook/qbits/edi/demo/EdiDemoServerTest.java
+13 -3
View File
@@ -86,15 +86,15 @@ class EdiDemoServerTest
demo.seed();
demo.seed();
assertThat(demo.defineQInstance().getHomeAppName()).isEqualTo("edi");
assertThat(SystemWrites.call(() -> EdiRecords.query("edi_partner", new QQueryFilter()))).hasSize(4);
assertThat(SystemWrites.call(() -> EdiRecords.query("edi_certificate", new QQueryFilter()))).hasSize(2);
assertThat(SystemWrites.call(() -> EdiRecords.query("edi_partner", new QQueryFilter()))).hasSize(5);
assertThat(SystemWrites.call(() -> EdiRecords.query("edi_certificate", new QQueryFilter()))).hasSize(5);
assertThat(SystemWrites.call(() -> EdiRecords.query("edi_inbox", new QQueryFilter()))).hasSizeGreaterThanOrEqualTo(40);
assertThat(SystemWrites.call(() -> EdiRecords.query("edi_transaction", new QQueryFilter()))).anyMatch(row -> "OUTBOUND".equals(row.getValueString("direction")));
assertThat(SystemWrites.call(() -> EdiRecords.query("edi_outbound_message", new QQueryFilter()))).anyMatch(row -> "DEAD_LETTER".equals(row.getValueString("status")));
assertThat(SystemWrites.call(() -> EdiRecords.query("edi_functional_group", new QQueryFilter()))).anyMatch(row -> row.getValueInstant("ackReceivedAt") != null);
assertThat(SystemWrites.call(() -> EdiRecords.query("edi_exception", new QQueryFilter()))).anyMatch(row -> "OVERDUE_ACK".equals(row.getValueString("type")));
var partners = SystemWrites.call(() -> EdiRecords.query("edi_partner", new QQueryFilter()));
assertThat(partners).extracting(row -> row.getValueString("code")).containsExactlyInAnyOrder("NORTHWIND", "HARBOR", "SUMMIT", "RIDGELINE");
assertThat(partners).extracting(row -> row.getValueString("code")).containsExactlyInAnyOrder("NORTHWIND", "HARBOR", "SUMMIT", "RIDGELINE", "PACIFIC");
assertThat(partners).allSatisfy(row -> assertThat(row.getValueString("contactEmail")).endsWith("@example.invalid"));
assertThat(SystemWrites.call(() -> EdiRecords.query("sample_wms_order", new QQueryFilter()))).hasSize(3);
assertThat(SystemWrites.call(() -> EdiRecords.query("sample_shipment_status", new QQueryFilter()))).hasSize(3);
@@ -103,7 +103,17 @@ class EdiDemoServerTest
assertThat(transactions).anyMatch(row -> "204".equals(row.getValueString("transactionSet")) && "SENT".equals(row.getValueString("status")));
assertThat(transactions).anyMatch(row -> "940".equals(row.getValueString("transactionSet")) && "INVALID".equals(row.getValueString("status")));
assertThat(transactions).anyMatch(row -> "214".equals(row.getValueString("transactionSet")) && "INVALID".equals(row.getValueString("status")));
var as2Inbox = SystemWrites.call(() -> EdiRecords.query("edi_inbox", EdiRecords.equal("transportType", "AS2")));
assertThat(as2Inbox).hasSize(3).allMatch(row -> row.getValueString("externalId").endsWith("@pacific"));
var receipts = SystemWrites.call(() -> EdiRecords.query("edi_as2_mdn", new QQueryFilter()));
assertThat(receipts).anyMatch(row -> "INBOUND".equals(row.getValueString("direction")) && Boolean.TRUE.equals(row.getValueBoolean("signatureValid")));
assertThat(receipts).filteredOn(row -> "OUTBOUND".equals(row.getValueString("direction"))).hasSize(3);
var as2Transports = SystemWrites.call(() -> EdiRecords.query("edi_transport", EdiRecords.equal("type", "AS2"))).stream().map(row -> row.getValue("id")).toList();
var as2Messages = SystemWrites.call(() -> EdiRecords.query("edi_outbound_message", new QQueryFilter())).stream().filter(row -> as2Transports.contains(row.getValue("transportId"))).toList();
assertThat(as2Messages).isNotEmpty().allMatch(row -> "DELIVERED".equals(row.getValueString("status")));
assertThat(as2Messages).anyMatch(row -> row.getValueString("transportMic") != null);
int port = demo.start(0);
assertThat(client.send(HttpRequest.newBuilder(URI.create("http://localhost:" + port + "/qqq/hooks/edi/as2/nobody")).POST(HttpRequest.BodyPublishers.ofString("x")).build(), HttpResponse.BodyHandlers.ofString()).statusCode()).isEqualTo(404);
assertThat(client.send(HttpRequest.newBuilder(URI.create("http://localhost:" + port + "/app/edi")).GET().build(), HttpResponse.BodyHandlers.ofString()).statusCode()).isEqualTo(200);
for(String table : java.util.List.of("edi_partner", "edi_transaction", "edi_exception"))
{