Compare commits
17
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
8c0da3dfad
|
||
|
|
aa37cfb53c | ||
|
|
ee2345682b | ||
|
|
ab1e7c0fec
|
||
|
|
9f7aed6e1c
|
||
|
|
10e96115f9
|
||
|
|
d8d3d1cabd
|
||
|
|
4a49515fc4
|
||
|
|
f318ff79a9
|
||
|
|
1752ba5f7f
|
||
|
|
ddcb3d1d71
|
||
|
|
35b91aba98
|
||
|
|
35fa331884 | ||
|
|
9a3461fdf3
|
||
|
|
31ecdae873 | ||
|
|
5415cdb280
|
||
|
|
660f68c42a |
No files matched your search
@@ -19,7 +19,20 @@ Design: `docs/DESIGN.md`.
|
||||
## Architecture
|
||||
|
||||
- **Entry point:** `EmailQBitProducer implements QBitMetaDataProducer<EmailQBitConfig>`, package `com.kingsrook.qbits.email`.
|
||||
- Packages are added per phase as `docs/DESIGN.md` describes.
|
||||
- `metadata`: seven tables, enum sources, relations and provider/suppression validation.
|
||||
- `outbox`: EmailService, queue-time template rendering, protected resend and branded ad-hoc rendering.
|
||||
- `delivery`: durable dispatcher, attempt leases, rate limits, health and optional alerts.
|
||||
- `transport`: shared MIME, SMTP/SES transports, settings and registry.
|
||||
- `bridge`: QBIT_EMAIL adapter for core SendMessageAction.
|
||||
- `processes`: Review/Result record actions, backend send API and scheduled jobs.
|
||||
- `permissions`, `app`, `support`: custom permission checker, app/navigation and authorized database operations.
|
||||
- `EmailRuntime` resolves config from the current instance; `EmailIntegrations` declares ESB, API and Quick Search tables.
|
||||
- `template`: parsed variable language/schema, locale resolution, immutable versions, version-bound PHI approval, MJML compilation, lazy secure-message built-in and inline preview.
|
||||
- `render`: CommonMark extensions, table-based light/dark layout, contrast validation, text conversion and unsubscribe-link signing (handler is phase c).
|
||||
- `phi`: RC.5 classification/token stand-ins and AES-GCM encryption; only dispatch decrypts bodies.
|
||||
- `app/EmailHome`: viewer-scoped counts and inline cards, with standard hourly/suppression chart widgets.
|
||||
- Version operations serialize on the template row; active/approved version selectors are parent-scoped.
|
||||
- Preview UI API endpoint awaits RC.6; webhook/unsubscribe handlers remain phase c.
|
||||
|
||||
## Invariants
|
||||
|
||||
|
||||
@@ -6,4 +6,20 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
- Phase (a), part 2: versioned Markdown/HTML/MJML templates, typed variable schemas, escaped blocks/loops/fallbacks, branded light/dark layout and contrast validation.
|
||||
- New Version, Activate Version and version-bound Approve For PHI actions, edit permissions, immutable history and parent-scoped version selectors.
|
||||
- Sender-scoped record variables, classification stand-in, HMAC snapshot tokens, AES-GCM bodies, encrypted query displays and protected resend.
|
||||
- Lazy secure-message built-in, typed templated core bridge, process JSON variables, inline preview and viewer-scoped dashboard charts/cards.
|
||||
- PHI canary egress, masking, lifecycle, parent-filter, rendering, branding and GreenMail template tests.
|
||||
|
||||
- Phase (b) Postmark, SendGrid and Mailgun HTTP transports with fixed HTTPS endpoints, bounded requests, tracking options, safe error classification and Retry-After support; no new dependencies or filesystem storage.
|
||||
- Strict Gmail SMTP credential/STARTTLS validation, provider-specific credential and webhook environment defaults, and the [provider setup guide](docs/PROVIDERS.md).
|
||||
- HTTP request-contract tests and local-server end-to-end outbox tests covering success, provider failures, permanent rejection, retries and secret-safe stored errors.
|
||||
- Phase (a), part 1 email tables, PostgreSQL DDL, enum sources, child relations, permissions and Email app navigation.
|
||||
- Provider validation with environment-name allowlists and presence-only credential displays.
|
||||
- Transactional ad-hoc EmailService, idempotency, suppression and a replaceable Markdown/HTML/text renderer.
|
||||
- Durable dispatcher with unique attempt claims, leases, retry/dead-letter handling, shared provider rate limits, health probes and optional alerts.
|
||||
- SMTP with authentication/TLS/timeouts, SES v2 raw MIME, and a Gmail SMTP preset.
|
||||
- QBIT_EMAIL core messaging bridge, Review/Result actions, backend send process, dispatch/maintenance/retention schedules and host integration declarations.
|
||||
- H2 concurrency and transaction tests, GreenMail end-to-end tests, fake SES tests, metadata/permission checks and process-runner coverage.
|
||||
- Repository scaffold to the QRun qbit repository standard (Apache-2.0, CircleCI qqq-orb 0.6.8 with the `configure-package-install` pre-step, GA-only `publish_release`, the `release-rc` caller workflow).
|
||||
@@ -19,7 +19,20 @@ Design: `docs/DESIGN.md`.
|
||||
## Architecture
|
||||
|
||||
- **Entry point:** `EmailQBitProducer implements QBitMetaDataProducer<EmailQBitConfig>`, package `com.kingsrook.qbits.email`.
|
||||
- Packages are added per phase as `docs/DESIGN.md` describes.
|
||||
- `metadata`: seven tables, enum sources, relations and provider/suppression validation.
|
||||
- `outbox`: EmailService, queue-time template rendering, protected resend and branded ad-hoc rendering.
|
||||
- `delivery`: durable dispatcher, attempt leases, rate limits, health and optional alerts.
|
||||
- `transport`: shared MIME, SMTP/SES transports, settings and registry.
|
||||
- `bridge`: QBIT_EMAIL adapter for core SendMessageAction.
|
||||
- `processes`: Review/Result record actions, backend send API and scheduled jobs.
|
||||
- `permissions`, `app`, `support`: custom permission checker, app/navigation and authorized database operations.
|
||||
- `EmailRuntime` resolves config from the current instance; `EmailIntegrations` declares ESB, API and Quick Search tables.
|
||||
- `template`: parsed variable language/schema, locale resolution, immutable versions, version-bound PHI approval, MJML compilation, lazy secure-message built-in and inline preview.
|
||||
- `render`: CommonMark extensions, table-based light/dark layout, contrast validation, text conversion and unsubscribe-link signing (handler is phase c).
|
||||
- `phi`: RC.5 classification/token stand-ins and AES-GCM encryption; only dispatch decrypts bodies.
|
||||
- `app/EmailHome`: viewer-scoped counts and inline cards, with standard hourly/suppression chart widgets.
|
||||
- Version operations serialize on the template row; active/approved version selectors are parent-scoped.
|
||||
- Preview UI API endpoint awaits RC.6; webhook/unsubscribe handlers remain phase c.
|
||||
|
||||
## Invariants
|
||||
|
||||
|
||||
@@ -1,21 +1,23 @@
|
||||
# QBit Email
|
||||
|
||||
One email service for a QQQ system: an outbox with async sending, retries, dead letters and
|
||||
idempotency keys; providers as data rows (SMTP relay, Amazon SES, Postmark, SendGrid, Mailgun,
|
||||
Gmail); Markdown and MJML/HTML templates with versions and branding; delivery tracking from
|
||||
provider webhooks; a suppression list; RFC 8058 one-click unsubscribe; and PHI-safe sending.
|
||||
Inbound mail is phase 2.
|
||||
One email service for a QQQ system: a durable outbox (`email_message`) with retries, dead
|
||||
letters, idempotency keys, rate limits and provider health; SMTP, Amazon SES, Postmark,
|
||||
SendGrid, Mailgun and Gmail SMTP transports; versioned Markdown, HTML and MJML templates with
|
||||
dark-mode-safe branding; classified variables and PHI protection; preview widgets; a
|
||||
real-data home; and the core messaging bridge. Every send goes through the outbox.
|
||||
|
||||
Design: [`docs/DESIGN.md`](docs/DESIGN.md).
|
||||
|
||||
## Requirements
|
||||
|
||||
- Java 21
|
||||
- QQQ 4.1 (`qqq-bom-pom` 4.1.0-RC.4 or later)
|
||||
- An RDBMS backend for the `email_` tables
|
||||
The [design](docs/DESIGN.md) also covers delivery tracking, unsubscribe, Preferences, the
|
||||
consumer migrations and inbound mail. Preview uses the standard widget endpoint; the
|
||||
dedicated preview API arrives with QQQ RC.6. Attachments are refused until
|
||||
qbit-object-storage is wired in.
|
||||
|
||||
## Install
|
||||
|
||||
Requires Java 21, QQQ **4.1.0-RC.5**, and a transactional RDBMS with the supplied unique
|
||||
constraints. Apply [the PostgreSQL DDL](docs/ddl/email-postgres.sql) before producing the qbit. SQL timestamps store UTC values without a timezone, matching QQQ RC.5.
|
||||
The physical table names equal their metadata names; `backendName` selects the host's
|
||||
QQQ database backend, as it does in qbit-webhooks.
|
||||
|
||||
```xml
|
||||
<dependency>
|
||||
<groupId>com.kingsrook.qbits</groupId>
|
||||
@@ -26,27 +28,383 @@ Design: [`docs/DESIGN.md`](docs/DESIGN.md).
|
||||
|
||||
```java
|
||||
new EmailQBitProducer()
|
||||
.withQBitConfig(new EmailQBitConfig().withBackendName("rdbms"))
|
||||
.withQBitConfig(new EmailQBitConfig()
|
||||
.withBackendName("rdbms")
|
||||
.withSchedulerName("scheduler"))
|
||||
.produce(qInstance);
|
||||
```
|
||||
|
||||
## Tables, permissions and configuration
|
||||
The scheduler must already exist. Without a scheduler, callers can enqueue and run
|
||||
`EmailDispatcher` or Send Now manually; automatic dispatch is not installed.
|
||||
Configuration is looked up from the current `QInstance` on every call, never cached
|
||||
in a static variable.
|
||||
|
||||
See [`docs/DESIGN.md`](docs/DESIGN.md). Tables are prefixed `email_`; permissions use the full
|
||||
qbit name (`email.view`, `email.send`, `email.manage`, `email.admin`).
|
||||
## Tables and permissions
|
||||
|
||||
## Adopting in another app
|
||||
| Table | Navigation | Writes |
|
||||
|---|---|---|
|
||||
| `email_message` | Messages | Qbit only; people use record actions |
|
||||
| `email_message_attempt` | Attempts on a message | Qbit only |
|
||||
| `email_template` | Templates | `email.manage`; template edit grants also apply |
|
||||
| `email_template_version` | Versions on a template | Draft edits only; lifecycle uses record actions |
|
||||
| `email_brand` | Brands | `email.manage`; template edit grants also apply |
|
||||
| `email_suppression` | Suppressions | `email.manage`; changing/removing a complaint needs `email.admin` |
|
||||
| `email_provider` | Providers, with child Messages | `email.admin`, including reads and test sends |
|
||||
|
||||
Add the dependency, produce the qbit against your RDBMS backend, add a provider row, and send
|
||||
through `EmailService` (or the `email_send_templated` process). Code that already calls core's
|
||||
`SendEmailAction` can route through the outbox with the messaging bridge (DESIGN section 2).
|
||||
`email.view` reads messages, templates, versions, brands and suppressions.
|
||||
`email.send` permits the Java service, bridge and backend send process. Reading
|
||||
messages requires `email.view`, `email.manage` or `email.admin`; send alone grants no read access.
|
||||
`email.manage` also allows sending, Resend, Cancel and Send Now. `email.admin`
|
||||
implies all permissions. Grant `email.hasAccess` for the app's derived base access.
|
||||
Enable the host's `QInstance.enforcePermissionsInCoreActions` for direct Java action
|
||||
permission enforcement; the tables and processes use the shared custom checker.
|
||||
Trusted system callers use `SystemWrites.call(...)`, which opens core's authorized
|
||||
scope and still preserves record security locks.
|
||||
|
||||
All tables and columns follow the `email_`/snake_case database convention. Java field
|
||||
names are camelCase, matching the webhooks reference. System tables disable direct
|
||||
insert/update/delete capabilities. Labels contain no status decorations.
|
||||
|
||||
## Providers and credentials
|
||||
|
||||
See the [provider setup guide](docs/PROVIDERS.md) for row fields, DNS/domain setup,
|
||||
limits, Gmail app passwords and the webhook configuration planned for phase (c).
|
||||
|
||||
| Kind | Transport | Status |
|
||||
|---|---|---|
|
||||
| SMTP | Relay or authenticated SMTP, STARTTLS / TLS | Available |
|
||||
| SES | AWS SES v2, raw MIME | Available |
|
||||
| POSTMARK | HTTPS JSON, server token and message stream | Available |
|
||||
| SENDGRID | HTTPS JSON, bearer API key | Available |
|
||||
| MAILGUN | HTTPS MIME multipart, US / EU | Available |
|
||||
| GMAIL_SMTP | smtp.gmail.com:587, STARTTLS and app password | Available |
|
||||
| GMAIL_API | OAuth Gmail API | Deferred; use GMAIL_SMTP |
|
||||
|
||||
Create the first enabled provider with `isDefault=true`. When any providers are
|
||||
enabled, exactly one must be the enabled default. Switch defaults in a batch update,
|
||||
clearing the old default first. A nullable unique `default_slot` backs the validation.
|
||||
Providers with message history cannot be deleted; disable them instead.
|
||||
|
||||
For an unauthenticated relay, use kind `SMTP`, host/port, security `NONE`, and a
|
||||
`fromAddress`. Leave credential names empty. Authenticated SMTP uses `usernameEnvName`
|
||||
and `secretEnvName`; security is `NONE`, `STARTTLS` (required) or `TLS`. Timeouts are
|
||||
10 seconds to connect and 15 seconds each to read and write. The Gmail SMTP preset
|
||||
uses `smtp.gmail.com:587` with required STARTTLS.
|
||||
|
||||
SES needs a `region`, optionally `configurationSet`. With no credential references
|
||||
it uses the AWS default chain. Otherwise both username and secret references are
|
||||
required and resolve to the access key and secret key. SES sends raw MIME with explicit
|
||||
envelope destinations, keeping BCC recipients out of the raw message headers. SDK
|
||||
retries are disabled so outbox attempts own retry accounting; an API call is bounded
|
||||
at 40 seconds.
|
||||
|
||||
Credential fields contain **environment variable names**, matching
|
||||
`^[A-Z_][A-Z0-9_]{0,99}$`. The effective allowlist is `allowedEnvNames` plus these defaults
|
||||
for the selected kind:
|
||||
|
||||
| Kind | Default allowed names |
|
||||
|---|---|
|
||||
| SMTP / GMAIL_SMTP | `EMAIL_SMTP_USERNAME`, `EMAIL_SMTP_PASSWORD`, `EMAIL_GMAIL_USERNAME`, `EMAIL_GMAIL_APP_PASSWORD`, `GMAIL_USERNAME`, `GMAIL_APP_PASSWORD` |
|
||||
| SES | `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN` |
|
||||
| POSTMARK | `POSTMARK_SERVER_TOKEN`, `POSTMARK_WEBHOOK_PASSWORD`; legacy `EMAIL_POSTMARK_SERVER_TOKEN`, `EMAIL_POSTMARK_WEBHOOK_PASSWORD` |
|
||||
| SENDGRID | `SENDGRID_API_KEY`, `SENDGRID_WEBHOOK_PUBLIC_KEY`; legacy `EMAIL_SENDGRID_API_KEY`, `EMAIL_SENDGRID_WEBHOOK_PUBLIC_KEY` |
|
||||
| MAILGUN | `MAILGUN_API_KEY`, `MAILGUN_WEBHOOK_SIGNING_KEY`; legacy `EMAIL_MAILGUN_API_KEY`, `EMAIL_MAILGUN_SIGNING_KEY` |
|
||||
| GMAIL_API | `EMAIL_GMAIL_CLIENT_ID`, `EMAIL_GMAIL_CLIENT_SECRET`, `EMAIL_GMAIL_REFRESH_TOKEN` |
|
||||
|
||||
Allowlisting does not automatically select a name. Empty SMTP references mean no auth;
|
||||
empty SES references mean the default chain. Explicit SES references use a static
|
||||
access/secret pair; temporary session credentials belong in the default AWS chain.
|
||||
`withSecretResolver(Function<String,String>)` defaults to `System::getenv`. Values are
|
||||
resolved during an attempt and never retained, audited, returned or logged. The
|
||||
provider record's display-only fields return only `Set` / `Not set`; checking presence
|
||||
necessarily invokes the resolver too. Enqueue and internal queries do not resolve it.
|
||||
|
||||
POSTMARK, SENDGRID and MAILGUN require a secret environment name. Mailgun also
|
||||
requires a sending domain and US/EU region. Gmail SMTP requires both credential
|
||||
names and STARTTLS. GMAIL_API is refused with
|
||||
`Gmail API sending is specified but not built; use GMAIL_SMTP`.
|
||||
|
||||
HTTP transports use fixed HTTPS endpoints, 10-second connect and 30-second request
|
||||
timeouts, and no redirects. HTTP 429/5xx retry with optional Retry-After; account
|
||||
and credential failures pause the provider. Tracking is explicitly controlled by
|
||||
`trackOpens` / `trackClicks`, off by default. Outbound tracking settings do not
|
||||
install the event webhook receiver; that arrives in phase (c).
|
||||
|
||||
## Sending
|
||||
|
||||
```java
|
||||
EmailSendResult result = EmailService.send(new EmailRequest()
|
||||
.withTo("Pat <pat@example.com>")
|
||||
.withCc("copy@example.com")
|
||||
.withSubject("Your export is ready")
|
||||
.withMarkdownBody("[Open your export](https://app.example.com/exports/42)")
|
||||
.withCategory(EmailCategory.NOTIFICATION)
|
||||
.withIdempotencyKey("export:42")
|
||||
.withLinkedRecord("export", 42)
|
||||
.withSender("exports", "complete"));
|
||||
```
|
||||
|
||||
Use `withHtmlBody` or `withTextBody` instead of Markdown, with optional explicit text
|
||||
alongside HTML. `withFrom`, `withReplyTo`, `withBcc`, `withProviderName`, `withSendAfter`,
|
||||
and `withLocale` are supported. `sendingDomain`, when present, must match the sender's
|
||||
domain. Addresses are parsed strictly; header newlines are refused. Markdown raw HTML
|
||||
and unsafe URL schemes are disabled. Templates always use the validated template renderer. The ad-hoc renderer extension
|
||||
remains compatible. Both alternatives are stored at enqueue time and reused on retries.
|
||||
|
||||
Results expose `messageId()`, `status()` (`QUEUED`, `DUPLICATE`, `SUPPRESSED`, `REFUSED`)
|
||||
and `reasons()`. Idempotency is global within the outbox. A repeated key returns the
|
||||
original id; unique constraints protect races. Idempotent inserts briefly lock the
|
||||
oldest provider row so concurrent service calls do not abort a caller-owned SQL
|
||||
transaction with a duplicate-key error.
|
||||
|
||||
`withTransaction(transaction, backendName)` joins a matching backend transaction.
|
||||
The one-argument overload assumes the email backend. A different backend registers
|
||||
an after-commit insert, discarded on rollback. Its immediate result has a null id
|
||||
and a reason explaining that enqueue is deferred: an auto-generated id does not
|
||||
exist before that callback runs. Cross-backend callbacks have the same crash window
|
||||
as qbit-webhooks' fallback; use the same backend when atomic enqueue is required.
|
||||
Without a caller transaction, the service owns and commits its transaction.
|
||||
|
||||
For multi-tenant hosts, configure `securityFields` and `recordSecurityLocks`, add the
|
||||
matching columns to the DDL, and provide the request's `securityValues`. Security
|
||||
values propagate to attempts and resends.
|
||||
|
||||
Suppression compares canonical lowercase mailboxes. ALL applies across categories,
|
||||
except UNSUBSCRIBE cannot block TRANSACTIONAL or SECURITY. Category suppressions apply
|
||||
to NOTIFICATION and MARKETING. Expired rows are ignored. Partial suppression drops
|
||||
only matching recipients and records them in `suppressedAddresses`; all-suppressed
|
||||
messages remain visible with SUPPRESSED status and never consume an attempt.
|
||||
|
||||
## Templates, variables and branding
|
||||
|
||||
Create an `email_template`, then use **New Version** to copy the active or latest
|
||||
version into a DRAFT. Edit its source, schema and synthetic sample values. **Activate
|
||||
Version** validates syntax, samples, classifications, brand contrast and optional
|
||||
MJML compilation, then retires the previous active version for that locale in one
|
||||
transaction. ACTIVE and RETIRED versions are immutable. Template-specific
|
||||
`editPermissionName` applies to CRUD and actions, in addition to the email permission;
|
||||
even email administrators need that explicit grant. Trusted system writes may bypass it.
|
||||
|
||||
```java
|
||||
EmailService.send(new EmailRequest()
|
||||
.withTemplateName("order_ready")
|
||||
.withTo("pat@example.com")
|
||||
.withRecordVariable("order", "order", 123)
|
||||
.withVariable("count", 3)
|
||||
.withLocale("en"));
|
||||
```
|
||||
|
||||
A schema is a JSON list, for example:
|
||||
|
||||
```json
|
||||
[
|
||||
{"name":"order","label":"Order","type":"RECORD","required":true},
|
||||
{"name":"count","label":"Items","type":"NUMBER","required":true},
|
||||
{"name":"link","type":"URL","classification":"PUBLIC"}
|
||||
]
|
||||
```
|
||||
|
||||
Types are STRING, NUMBER, DATE, DATE_TIME, BOOLEAN, URL, LIST and RECORD. Optional
|
||||
`table`/`field` bind a variable's classification to metadata. URL values must be
|
||||
absolute HTTP(S) URLs. Use `{{name}}`, `{{order.number}}`, `{{name|fallback text}}`,
|
||||
`{{#if count}}...{{/if}}`, and `{{#each items}}{{@index}}: {{this.name}}{{/each}}`.
|
||||
`this` also supports scalar list elements. Empty/null/false/zero values are false.
|
||||
Unknown variables, unsupported helpers, unclosed expressions and mismatched blocks
|
||||
produce line-numbered errors. Records and lists allow dynamic child paths; declare
|
||||
specific dotted entries when child types/requiredness matter. There is no raw output.
|
||||
HTML values are escaped; Markdown values cannot inject Markdown syntax. Subject and
|
||||
text contexts use plain values, with subject newlines refused.
|
||||
|
||||
Built-ins are `brand.*` (snake_case names such as `label`, `site_url`, `primary_color`),
|
||||
`message.id` (the actual outbox primary key), `now` (UTC), and `unsubscribe_url`.
|
||||
The latter is empty for TRANSACTIONAL/SECURITY and when no notification opt-out key
|
||||
is configured. The requested locale resolves to an ACTIVE version, falling back to
|
||||
`en`. The template supplies the category unless the request overrides it.
|
||||
|
||||
Brand precedence is request name, template brand, the configured app's brand,
|
||||
default brand, then the built-in neutral brand. Markdown uses CommonMark tables and
|
||||
autolinks; raw HTML is escaped. `[Label](https://example.com){.button}` creates a table
|
||||
button. The 600px shell includes preheader, logo variants, footer, social links and
|
||||
light/dark colors; marketing requires a postal address. Brand saves and activation
|
||||
require 4.5:1 text and button contrast in both modes. Optional `darkPrimaryColor` and
|
||||
`darkButtonTextColor` fall back to their light counterparts. Colors use `#RRGGBB`.
|
||||
|
||||
HTML versions retain authored HTML, with escaped variables and an opt-out footer
|
||||
before `</body>`. For MJML configure `withMjmlCompiler(new CliMjmlCompiler(binaryPath))`
|
||||
or another compiler implementation. Compilation uses stdin/stdout only, with a
|
||||
20-second timeout; the stored `compiledHtml` is reused for sends. No runtime files,
|
||||
source files or compiler temp files are written by this qbit.
|
||||
|
||||
The record Preview widget uses synthetic `sampleVariables`, sanitized inline light
|
||||
(390px maximum) and dark (600px maximum) samples plus wrapping plain text. Next's
|
||||
sanitizer removes iframes and style blocks, so this preview is not an exact mail-client
|
||||
emulator. Draft MJML is compiled in memory for preview when a compiler is configured;
|
||||
preview never stores the result or changes the version.
|
||||
|
||||
## PHI and PII
|
||||
|
||||
`withRecordVariable` reads through `QueryAction` in the sender's QContext; it does not
|
||||
open an authorized scope. Core password masking is preserved. RC.5 does not enforce
|
||||
ordinary `FieldSecurityLock` during QueryAction, so the email record loader also
|
||||
applies the field lock against the sender's security keys before copying any values.
|
||||
Classification precedence is reflective core `AiDataClassifications.ofField`, config
|
||||
`classificationOverrides`, schema declaration, then PUBLIC.
|
||||
|
||||
PII/PHI variables require **Approve For PHI** (`email.admin`) on the exact version.
|
||||
Approval records actor/date and `approvedVersionId`; it freezes an approved draft.
|
||||
A new sensitive version must be approved before activation. Activating a version
|
||||
without its own approval clears the template's old approval. Approval of one locale
|
||||
version does not approve another locale. PHI-approved templates cannot be MARKETING,
|
||||
use provider open/click tracking, or use a provider without `phiAllowed`. Provider
|
||||
attestation and tracking are checked again immediately before dispatch.
|
||||
|
||||
`email_secure_message` is created transactionally on its first send. It needs only
|
||||
a required URL variable `link` and brand built-ins; use it to send a sign-in link
|
||||
without putting patient data into email.
|
||||
|
||||
PII/PHI snapshot values become `hmac:v1:<keyId>:<base64url>`, or `changed:v1` without
|
||||
a valid base64 `QQQ_AUDIT_HMAC_KEY`. Record tokens bind table, field and primary key;
|
||||
context tokens bind `email`, template name, variable name and message id. Context
|
||||
booleans and values with fewer than four distinct characters always use `changed:v1`.
|
||||
`shred:v1` is reserved for future core wipe-rule record fields and is never produced
|
||||
for context values. The classification and token helpers are explicit RC.7 stand-ins.
|
||||
|
||||
PHI-approved subjects and HTML/text bodies are AES-256-GCM encrypted as
|
||||
`enc:v1:<base64url(iv|ciphertext|tag)>`, using SHA-256 of `QQQ_EMAIL_BODY_KEY`.
|
||||
Missing keys refuse the send before an outbox insert. Only dispatch decrypts; query,
|
||||
record and API displays show `[encrypted]`, while home shows `Secure message`.
|
||||
Resend preserves ciphertext and policy; its token snapshot stays bound to the original
|
||||
message, which the new message links to. Errors and logs omit provider diagnostics
|
||||
and content. Keys resolve through `secretResolver`; no secret values are stored.
|
||||
|
||||
NOTIFICATION/MARKETING templates prepare footer links and headers using
|
||||
`QQQ_EMAIL_UNSUBSCRIBE_KEY` and `publicBaseUrl`; marketing refuses without them.
|
||||
Tokens bind the message, recipient-envelope hash and category. The phase-(c)
|
||||
unsubscribe route and Preferences are still pending: signed links are prepared here,
|
||||
but this qbit alone does not yet serve the unsubscribe destination. Do not deploy
|
||||
opt-out mail without that receiving handler.
|
||||
|
||||
The home uses viewer-scoped counts and queries, UTC hour/day charts, provider health
|
||||
(admins), and 20 recent messages with masked recipients. No sample data or sampled
|
||||
counts are used. Bounce/complaint totals use dedicated event timestamps, populated
|
||||
by the future webhook phase or host integrations.
|
||||
|
||||
## Delivery and actions
|
||||
|
||||
Each claim inserts a unique `(message_id, attempt_number)` row and commits SENDING
|
||||
with a lease before network I/O. Provider-row database locks serialize rate checks
|
||||
and claims across replicas; they are released before contacting the provider. Limits
|
||||
count attempts in rolling minute and 24-hour windows. Waiting consumes no attempt.
|
||||
Per-message SMTP 5xx and SES MessageRejected/InvalidParameterValue failures become
|
||||
DEAD_LETTER, as do invalid MIME addresses or headers; transient failures
|
||||
use the configured backoff. Explicit retry-after values are capped at one hour;
|
||||
the configured backoff itself is not capped.
|
||||
|
||||
Expired attempts become LEASE_EXPIRED, then retry or dead-letter at the attempt limit.
|
||||
Late completion from an expired runner cannot overwrite the newer attempt's outcome.
|
||||
Delivery is **at least once**: provider acceptance followed by a runner crash can
|
||||
produce a repeat. Credential, authentication and SES account failures immediately mark
|
||||
the provider UNHEALTHY and stop its remaining deliveries for that dispatch pass.
|
||||
These attempts are recorded as PROVIDER_FAILURE and keep messages in RETRY_WAIT with
|
||||
backoff. They increment the historical attempt count but do not consume the message's
|
||||
maxAttempts budget (including during lease recovery). Ten consecutive other transient failures mark a provider UNHEALTHY;
|
||||
one queued message probes it every five minutes, and success restores HEALTHY.
|
||||
Optional core alerts raise `email.provider.unhealthy` and `email.message.dead-letter`,
|
||||
with dead-letter deduplication per provider/hour. No alert sink is required.
|
||||
Alert failures cannot fail an already committed delivery.
|
||||
|
||||
SMTP sign-in requires STARTTLS or TLS. Plaintext authentication is refused at provider
|
||||
validation and again before connecting, except for loopback (`localhost`, `127.0.0.1`,
|
||||
`::1`) or an explicit `withAllowPlaintextSmtpAuth(true)` host configuration (default false).
|
||||
Provider sender/reply-to fields and MIME subjects/addresses cannot contain CR or LF.
|
||||
|
||||
Messages offer standard Review/Result actions: Resend (DEAD_LETTER, BOUNCED,
|
||||
COMPLAINED, with retained content), Cancel (QUEUED/RETRY_WAIT), and Send Now
|
||||
(QUEUED/RETRY_WAIT). Resend creates a new linked message and rechecks suppression.
|
||||
Send Now overrides deferral but preserves rate limits, health and active leases.
|
||||
Providers offer Send Test Email with a recipient input and a reported outcome.
|
||||
|
||||
`email_send_templated` is a hidden backend API process. Inputs include `templateName`,
|
||||
`to`, `cc`, `bcc`, `replyTo`, `subject`, `markdownBody`, `htmlBody`, `textBody`,
|
||||
`providerName`, `brandName`, `category`, `locale`, `variablesJson`, `recordVariablesJson`,
|
||||
`linkedTableName`, `linkedRecordId`, and `idempotencyKey`. `variablesJson` is an object;
|
||||
record bindings use `{"patient":{"table":"patient","id":123}}` in
|
||||
`recordVariablesJson`. Output includes `messageId`, `status`, and `reasons`.
|
||||
|
||||
Hidden schedules are `email_dispatch` (15 s by default), `email_maintain` (300 s, lease
|
||||
recovery including disabled providers, and health probes), and `email_retention`
|
||||
(daily). Retention clears final-message bodies after 30 days, preserving `bodySha256`;
|
||||
message deletion is disabled by default. `finishedDate` records finalization independently
|
||||
of enqueue and later retention updates.
|
||||
|
||||
## Core messaging bridge
|
||||
|
||||
The producer adds messaging provider `qbitEmail` (configurable), type `QBIT_EMAIL`.
|
||||
Existing callers switch only `messagingProviderName`:
|
||||
|
||||
```java
|
||||
new SendMessageAction().execute(new SendMessageInput()
|
||||
.withMessagingProviderName("qbitEmail")
|
||||
.withTo(new Party().withAddress("pat@example.com"))
|
||||
.withSubject("Ready")
|
||||
.withContent(new Content().withContentRole(EmailContentRole.TEXT).withBody("Your export is ready.")));
|
||||
```
|
||||
|
||||
The bridge recursively maps Party/MultiParty TO, CC, BCC, FROM and REPLY_TO roles, plus
|
||||
TEXT/HTML alternatives. REFUSED throws `QException`; successful calls acknowledge
|
||||
queueing, not delivery. The built-in EMAIL and SES types are unchanged.
|
||||
|
||||
For schema-aware core bridge calls, use the typed extension:
|
||||
|
||||
```java
|
||||
new SendMessageAction().execute(new EmailTemplatedMessageInput()
|
||||
.withEmailRequest(new EmailRequest().withTemplateName("email_secure_message")
|
||||
.withTo("pat@example.com").withVariable("link", "https://app.example.com/inbox"))
|
||||
.withMessagingProviderName("qbitEmail"));
|
||||
```
|
||||
|
||||
Ordinary core `SendMessageInput` remains ad-hoc and cannot infer classifications
|
||||
from arbitrary pre-rendered text. Use the typed extension for classified variables.
|
||||
|
||||
## Configuration
|
||||
|
||||
All settings have fluent `withX` setters and getters.
|
||||
|
||||
| Setting | Default / purpose |
|
||||
|---|---|
|
||||
| `backendName` | Required existing backend |
|
||||
| `schedulerName` | Optional existing scheduler; required for automatic dispatch |
|
||||
| `publicBaseUrl` | Public app URL for signed unsubscribe links |
|
||||
| `mjmlCompiler` | Optional `MjmlCompilerInterface`; required for MJML activation |
|
||||
| `classificationOverrides` | Map of `table.field` to `EmailClassifications.Level` for RC.5 |
|
||||
| `defaultProviderName` | Optional named provider override; otherwise enabled default row |
|
||||
| `bridgeProviderName` | `qbitEmail` |
|
||||
| `secretResolver`, `allowedEnvNames` | `System::getenv`, empty additional allowlist |
|
||||
| `dispatchIntervalSeconds`, `batchSize` | 15, 100 |
|
||||
| `maxAttempts`, `retryMinutes` | 6; 1, 5, 15, 60, 240, 720 |
|
||||
| `leaseSeconds` | 120; minimum 60 |
|
||||
| `bodyRetentionDays`, `messageRetentionDays` | 30, 0 (keep message rows forever) |
|
||||
| `appName`, `appLabel` | `email`, `Email` |
|
||||
| `securityFields`, `recordSecurityLocks` | Empty host additions |
|
||||
| `tableMetaDataCustomizer` | Optional final metadata customization |
|
||||
|
||||
`EmailIntegrations.esbTableNames()`, `apiTableNames()` and `quickSearchTables()` expose
|
||||
host integration declarations for messages, templates and suppressions. Quick Search
|
||||
weights exclude subjects, bodies and variable snapshots. Hosts wire their ESB/API/search
|
||||
modules to these declarations.
|
||||
|
||||
## Build
|
||||
|
||||
```bash
|
||||
./mvnw verify # tests, Checkstyle and the JaCoCo gate
|
||||
./mvnw -o -q verify
|
||||
```
|
||||
|
||||
Windows can use `./mvnw.cmd -o -q verify`. No new dependencies are required. Tests use
|
||||
H2, GreenMail, a fake SES client and local HTTP servers, including transaction rollback, concurrent claims,
|
||||
concurrent idempotency, shared rate budgets, health recovery and process screens.
|
||||
The tests also exercise template lifecycle, parent-scoped possible values, rendering,
|
||||
PHI canary egress, encryption, preview widgets and home data.
|
||||
|
||||
## License
|
||||
|
||||
Apache License 2.0. See [LICENSE](LICENSE) and [NOTICE](NOTICE).
|
||||
+489
-2
@@ -1,3 +1,490 @@
|
||||
# QBit Email design
|
||||
# qbit-email design
|
||||
|
||||
The design spec arrives in the spec PR (`feature/design-spec`).
|
||||
**Status:** phase (a) implemented; later phases remain specification. Phases (a) to (d) build outbound; phase (e) is inbound (section 12, design only).
|
||||
**Brief:** owner decisions of 2026-10-13 (providers, outbound first, both template kinds).
|
||||
**Owner decisions (2026-10-12): all recommendations accepted.** Every "Recommended" choice and
|
||||
every decision below stands; it is adjusted once it can be seen working.
|
||||
|
||||
One email service for a whole QQQ system. Every qbit and app that sends mail queues it through
|
||||
this qbit's outbox and gets the same providers, templates, branding, tracking, suppression,
|
||||
unsubscribe and PHI rules.
|
||||
|
||||
| # | Decision |
|
||||
|---|---|
|
||||
| D1 | Build on core's messaging SPI (`MessagingProviderInterface`, `SendMessageAction`). The qbit registers one provider type, `QBIT_EMAIL`, that writes to the outbox, so existing `SendMessageAction` callers move by pointing their `messagingProviderName` at it (section 2.4). Core's built-in `EMAIL` and `SES` types are not overridden. |
|
||||
| D2 | Delivery transports live in the qbit (section 3): core's SMTP provider has no auth, STARTTLS or timeouts, and its SES provider uses AWS SDK v1 with static keys in metadata. No core PR is required; the transports can move to core later without changing the tables. |
|
||||
| D3 | Providers are data rows (`email_provider`). Credentials are environment-variable NAMES, never values (the qbit-assistant rule). |
|
||||
| D4 | Every send goes through the outbox (`email_message`); nothing sends inline. Async, at least once, retries with backoff, dead letters, idempotency keys. |
|
||||
| D5 | Two template kinds: Markdown with `{{variables}}` inside the branded responsive layout (the default), and raw MJML or HTML (marketing). Templates are versioned. |
|
||||
| D6 | Tracking events (SES via SNS, Postmark, SendGrid, Mailgun) arrive at `/qqq/hooks/email/...` (not the UI API), are signature-verified, and land in a durable inbox before processing. |
|
||||
| D7 | Hard bounces and complaints suppress the address. One-click unsubscribe (RFC 8058) records a suppression for the message's category, and users see it as a Preferences section (core #1162). |
|
||||
| D8 | PHI-safe by default: PII/PHI-classified variables are refused unless the template is `phi_approved`. The recommended PHI pattern is a "secure message" (a sign-in link, no PHI in the body). |
|
||||
| D9 | Gmail: SMTP with an app password in phase (b), as a preset of the SMTP transport. The Gmail API with OAuth is specified (section 3.6) and built only on request. |
|
||||
| D10 | MJML is compiled when a version is activated, by a pluggable compiler (default: the `mjml` CLI when configured, through stdin and stdout). Sends never need the compiler; HTML templates need nothing. |
|
||||
| D11 | Owner rule (2026-10-13): no files or filesystem, only the database and S3. Attachments, rendered bodies and (phase e) inbound mail live in the database or in object storage (qbit-object-storage), never on local disk; the services are stateless, so any replica can send, receive a webhook or render a preview. |
|
||||
|
||||
## 1. Flow
|
||||
|
||||
```
|
||||
EmailService.send(EmailRequest) ─┐
|
||||
email_send_templated (process) ─┤
|
||||
SendMessageAction (type QBIT_EMAIL) ─┼─► EmailOutbox.enqueue ─► email_message (QUEUED; rendered, redacted)
|
||||
record action "Send Test Email" ─┘ idempotency key unique; suppression and PHI checks first
|
||||
│
|
||||
scheduler (every 15 s) ─► EmailDispatcher.runOnce
|
||||
claim attempt n (unique insert into email_message_attempt)
|
||||
─► provider rate limit ─► EmailTransport.send (SMTP | SES | Postmark | SendGrid | Mailgun | Gmail)
|
||||
─► SENT (provider message id) │ RETRY_WAIT (backoff) │ DEAD_LETTER
|
||||
│
|
||||
provider webhook ─► /qqq/hooks/email/{kind}/{providerName} ─► verify ─► email_webhook_inbox (durable)
|
||||
─► inbox processor ─► email_event (DELIVERED, BOUNCED, COMPLAINED, OPENED, CLICKED, ...)
|
||||
─► message status ─► email_suppression (hard bounce, complaint)
|
||||
List-Unsubscribe one-click POST ─► /qqq/hooks/email/unsubscribe/{token} ─► email_suppression (category)
|
||||
```
|
||||
|
||||
## 2. Sending API
|
||||
|
||||
### 2.1 `EmailService` (Java, for every consumer)
|
||||
|
||||
```java
|
||||
EmailSendResult result = EmailService.send(new EmailRequest()
|
||||
.withTemplateName("alerts_notification") // or withSubject + withMarkdownBody / withHtmlBody for ad-hoc mail
|
||||
.withTo("pat@example.com") // withCc, withBcc, withReplyTo; "addr" or "Name <addr>"
|
||||
.withVariable("count", 3) // plain values, checked against the template's schema
|
||||
.withRecordVariable("order", "order", 1234) // a record, read as the sender; classification known
|
||||
.withCategory(EmailCategory.NOTIFICATION) // default: the template's
|
||||
.withProviderName("pmg") // optional; else the default provider
|
||||
.withBrandName("kof22") // optional; else the template's, else the default brand
|
||||
.withIdempotencyKey("alerts:notif:991") // optional; a repeat returns the existing message
|
||||
.withLinkedRecord("alerts_alert", 991) // shown as a link on the message record
|
||||
.withSender("qbit-alerts", "alerts_notify") // sender qbit and process
|
||||
.withSendAfter(instant)); // optional deferral
|
||||
// EmailSendResult { messageId, status (QUEUED | DUPLICATE | SUPPRESSED | REFUSED), reasons }
|
||||
```
|
||||
|
||||
- Runs inside the caller's transaction when the backends match (the row commits with the caller's
|
||||
write), else after commit, as qbit-webhooks does.
|
||||
- Validation is synchronous: unknown template, missing required variables, bad addresses, a
|
||||
PII/PHI variable on a template that is not `phi_approved` (section 7), and a disabled or missing
|
||||
provider all return `REFUSED` with plain reasons, and nothing is queued.
|
||||
- All recipients suppressed for the category: `SUPPRESSED`; the message is stored with that status
|
||||
(visible in the outbox) and not sent. Partly suppressed: those addresses are dropped and listed
|
||||
on the message.
|
||||
|
||||
### 2.2 Process `email_send_templated`
|
||||
The same request as a backend process (template, to, cc, bcc, variables JSON, linked table and
|
||||
id, idempotency key, provider, brand) for scripts, the API and other qbits. Not in the nav;
|
||||
callable through the API and `/qqq/v1/processes`.
|
||||
|
||||
### 2.3 Permissions
|
||||
`email.view` (read every `email_` table), `email.send` (send through the API or the process;
|
||||
system callers use the authorized scope), `email.manage` (templates, brands, suppressions, resend,
|
||||
cancel), `email.admin` (providers, test send, PHI approval, removing complaint suppressions). A
|
||||
template may set `edit_permission_name` to restrict who edits it (e.g. marketing templates to
|
||||
`marketing.manage`).
|
||||
|
||||
### 2.4 Core messaging bridge
|
||||
- `EmailOutboxMessagingProviderMetaData` (type `QBIT_EMAIL`, a `QMessagingProviderMetaData`)
|
||||
registers `EmailOutboxMessagingProvider` with `QMessagingProviderDispatcher`. The producer adds
|
||||
one instance named `qbitEmail` (configurable).
|
||||
- `sendMessage(SendMessageInput)` maps the parties (`EmailPartyRole` TO/CC/BCC/FROM), subject and
|
||||
contents (`EmailContentRole` TEXT/HTML) to an ad-hoc `EmailRequest` (no template, category
|
||||
TRANSACTIONAL unless configured) and enqueues it. It throws on `REFUSED`, so callers still see
|
||||
failures; delivery is then asynchronous.
|
||||
- Attachments are stored through qbit-object-storage when it is installed, else refused with a
|
||||
plain message (no blobs in the outbox, nothing on local disk: D11).
|
||||
- Callers (qbit-alerts `EmailChannel` and `NotificationEmailer`, qbit-marketing
|
||||
`QqqMessagingEmailTransport`) can switch by configuration at once, then move to `EmailService`
|
||||
in phase (d) for templates, categories, idempotency and linked records.
|
||||
|
||||
## 3. Providers and transports
|
||||
|
||||
### 3.1 `email_provider`
|
||||
| Field | Notes |
|
||||
|---|---|
|
||||
| name, label | unique name, used by `withProviderName` |
|
||||
| kind | SMTP, SES, POSTMARK, SENDGRID, MAILGUN, GMAIL_SMTP, GMAIL_API |
|
||||
| enabled, is_default | exactly one enabled default (validated) |
|
||||
| from_address, from_name, reply_to | defaults for messages |
|
||||
| sending_domain | the domain the provider is verified for; other from domains are refused |
|
||||
| host, port, security | SMTP: NONE, STARTTLS, TLS; timeouts fixed (10 s connect, 15 s read and write) |
|
||||
| region | SES region; Mailgun region (US or EU) |
|
||||
| username_env_name, secret_env_name | env-var NAMES only (`^[A-Z_][A-Z0-9_]{0,99}$`, and in the config allowlist) |
|
||||
| webhook_secret_env_name | env-var NAME for webhook verification (Mailgun signing key, SendGrid public key, Postmark basic-auth password) |
|
||||
| sns_topic_arn, configuration_set, message_stream | SES events topic and configuration set; Postmark message stream |
|
||||
| max_per_minute, max_per_day | counted from `email_message_attempt` in the database, so they hold across replicas |
|
||||
| track_opens, track_clicks | off by default; refused for `phi_approved` templates |
|
||||
| phi_allowed | admin attestation that the provider may carry PHI (BAA in place, or an internal relay) |
|
||||
| health_status, last_success_date, last_failure_date, last_failure_message | kept by the dispatcher |
|
||||
|
||||
Secrets resolve through `EmailQBitConfig.secretResolver` (default `System::getenv`) at send time.
|
||||
They are never stored, logged, audited, put in process output or returned. The record screen shows
|
||||
whether each named variable is set (yes or no), never the value.
|
||||
|
||||
**Record action "Send Test Email"** (`email.admin`): queues a fixed test message to a given
|
||||
address through that provider and reports the outcome on the standard result screen.
|
||||
|
||||
### 3.2 `EmailTransport` SPI (in the qbit)
|
||||
```java
|
||||
public interface EmailTransport
|
||||
{
|
||||
EmailTransportResult send(EmailEnvelope envelope, EmailProviderSettings settings) throws EmailTransportException;
|
||||
}
|
||||
```
|
||||
`EmailEnvelope` holds the rendered MIME parts: from, to, cc, bcc, reply-to, subject, HTML, text,
|
||||
and headers (`List-Unsubscribe`, `Auto-Submitted`, `X-QQQ-Email-Id`). The result is the
|
||||
provider's message id. The exception says whether the failure is **permanent** (rejected address,
|
||||
sender refused, 4xx validation) or **transient** (connection, 5xx, throttling with an optional
|
||||
retry-after).
|
||||
|
||||
### 3.3 SMTP (phase a)
|
||||
`jakarta.mail` (already in core): auth when a username env name is set, STARTTLS or implicit TLS,
|
||||
connect/read/write timeouts (the kof22 `Mailer` lesson: a stuck relay must not pin a thread).
|
||||
Covers PMG (`smtp.galaxy.lan:26`, no auth), any relay, and Gmail SMTP.
|
||||
|
||||
### 3.4 SES (phase a)
|
||||
AWS SDK v2 `sesv2` `SendEmail` with raw content, so our MIME and headers are kept. Credentials:
|
||||
the default AWS chain when no env names are set (IRSA, instance role, `AWS_*`), else the named
|
||||
access-key and secret env vars. The configuration set routes events to SNS (section 6).
|
||||
|
||||
### 3.5 Postmark, SendGrid, Mailgun (phase b)
|
||||
HTTP JSON APIs through `java.net.http`, no vendor SDKs: Postmark `POST /email` (server-token
|
||||
header, message stream), SendGrid `POST /v3/mail/send` (bearer key; `X-Message-Id`), Mailgun
|
||||
`POST /v3/{domain}/messages.mime` (basic auth `api:key`, US or EU base). 429 and 5xx are
|
||||
transient; other 4xx are permanent.
|
||||
|
||||
### 3.6 Gmail
|
||||
- **Phase (b): `GMAIL_SMTP`**, a preset of SMTP (`smtp.gmail.com:587`, STARTTLS) with the account
|
||||
as username and an app password env var. `docs/PROVIDERS.md` documents it: it needs 2-step
|
||||
verification and an app password; Google's daily limits (about 500 for a personal account, 2000
|
||||
for Workspace) go in `max_per_day`.
|
||||
- **`GMAIL_API` (specified, built on request):** OAuth client id, secret and refresh token (env
|
||||
names), `users.messages.send` with the raw MIME. It needs a Google Cloud project and consent
|
||||
screen and gives system mail nothing the app password does not, so it is deferred.
|
||||
|
||||
## 4. Outbox and delivery
|
||||
|
||||
### 4.1 `email_message`
|
||||
| Field | Notes |
|
||||
|---|---|
|
||||
| status | QUEUED, SENDING, SENT, DELIVERED, RETRY_WAIT, DEAD_LETTER, BOUNCED, COMPLAINED, SUPPRESSED, CANCELLED |
|
||||
| to_addresses, cc_addresses, bcc_addresses, from_address, reply_to | addresses are PII; the table is classified PII (section 7) |
|
||||
| subject | rendered; encrypted for `phi_approved` templates (section 7.3) |
|
||||
| template_id, template_version_id, brand_id, category, locale | |
|
||||
| variables_snapshot | JSON, redacted per section 7.3 |
|
||||
| html_body, text_body | rendered at queue time so retries send the same bytes; encrypted for `phi_approved`; cleared after `bodyRetentionDays` (default 30) once final, keeping `body_sha256` |
|
||||
| idempotency_key | unique when present |
|
||||
| provider_id, provider_message_id | |
|
||||
| attempt_count, next_attempt_date, lease_expires_date, last_error, sent_date, delivered_date, dead_letter_date | |
|
||||
| linked_table_name, linked_record_id | rendered as a link on the record screen |
|
||||
| sender_qbit, sender_process | |
|
||||
| suppressed_addresses | addresses dropped at queue time |
|
||||
|
||||
Sub-tables, shown as relations on the message: `email_message_attempt` (message_id and
|
||||
attempt_number unique as a pair, outcome, provider_id, claimed_by, start and end dates, duration,
|
||||
error) and `email_event` (section 6).
|
||||
|
||||
### 4.2 Semantics (as qbit-webhooks, proven across replicas)
|
||||
- **Claim:** to send attempt *n* a runner inserts (`message_id`, *n*) into
|
||||
`email_message_attempt`; the pair is unique, so exactly one runner sends.
|
||||
- **Lease:** the winner marks the message SENDING with a 120 s lease; an expired lease is retried
|
||||
as attempt *n+1*.
|
||||
- **At least once:** a runner that dies after the provider accepted, but before recording it,
|
||||
causes at most one repeat. None of the providers offers a send-idempotency key, so the spec says
|
||||
so plainly.
|
||||
- **Outcomes:** success → SENT. Transient → RETRY_WAIT after `retryMinutes` (1, 5, 15, 60, 240,
|
||||
720; retry-after honoured, capped at 1 h). Permanent → DEAD_LETTER at once. After `maxAttempts`
|
||||
(6) → DEAD_LETTER.
|
||||
- **Rate limits** per provider per minute and per day, from the database. Over a limit the message
|
||||
waits; that is not an attempt.
|
||||
- **Provider health:** HEALTHY → UNHEALTHY after 10 consecutive transient failures; while
|
||||
unhealthy one message is tried every 5 minutes; a success → HEALTHY. Alerts (optional, section
|
||||
9): `email.provider.unhealthy` (ERROR, resolved on recovery) and `email.message.dead_letter`
|
||||
(WARNING, deduplicated per provider per hour).
|
||||
- **Record actions:** Resend (dead letters and bounces: a new linked message with the same
|
||||
content), Cancel (queued or waiting), Send Now (waiting). Standard streamed-ETL Review and Result
|
||||
screens.
|
||||
- **Retention:** a daily job clears bodies after `bodyRetentionDays` and deletes final messages
|
||||
older than `messageRetentionDays` (0 = keep forever; kof22 keeps).
|
||||
|
||||
## 5. Templates and branding
|
||||
|
||||
### 5.1 Tables
|
||||
- `email_template`: name (unique, snake_case), label, category (TRANSACTIONAL, NOTIFICATION,
|
||||
MARKETING, SECURITY), kind (MARKDOWN, MJML, HTML), brand_id, active_version_id, phi_approved,
|
||||
phi_approved_by, phi_approved_date, edit_permission_name, owner_qbit, description.
|
||||
- `email_template_version` (child list on the template): template_id, version_number, status
|
||||
(DRAFT, ACTIVE, RETIRED), locale (default `en`), subject, preheader, body_source, text_source
|
||||
(optional), compiled_html (MJML), variables_schema (JSON), sample_variables (JSON), created_by,
|
||||
activated_date.
|
||||
- `email_brand`: name, label, logo_url, logo_dark_url, logo_alt, primary_color, text_color,
|
||||
background_color, dark_background_color, dark_text_color, button_text_color, footer_markdown,
|
||||
postal_address (shown on marketing mail, for CAN-SPAM), social_links (JSON list of label and
|
||||
url), site_url, is_default, app_name (optional: the brand for that app's sends).
|
||||
|
||||
An ACTIVE version cannot be edited. **Record action "New Version"** copies it to a DRAFT;
|
||||
**"Activate Version"** validates it (schema, a render with the samples, the PHI check, MJML
|
||||
compile, brand contrast) and makes it active, retiring the previous one. A message always records
|
||||
the exact version it used.
|
||||
|
||||
### 5.2 Rendering
|
||||
- **Variables:** `{{name}}` and dotted paths (`{{order.number}}`), always HTML-escaped;
|
||||
`{{#if name}}…{{/if}}` and `{{#each items}}…{{/each}}`; `{{name|fallback text}}`. No unescaped
|
||||
output and no helpers that run code. Unknown variables fail activation, not sends.
|
||||
- **Schema:** each variable has name, label, type (STRING, NUMBER, DATE, DATE_TIME, BOOLEAN, URL,
|
||||
LIST, RECORD), required, an optional classification, and an optional bound `table` and `field`
|
||||
(the classification then comes from the field, section 7).
|
||||
- **Markdown** (commonmark-java with GFM tables and autolinks; raw HTML in Markdown disabled)
|
||||
renders into the branded layout: a fixed, tested, table-based responsive shell (600 px max,
|
||||
single column, inline styles plus one `<style>` block for media queries) with the logo, a hidden
|
||||
preheader, the body, a button style for links written `[Label](url){.button}`, and the footer
|
||||
(brand footer, postal address for MARKETING, the unsubscribe link when it applies).
|
||||
- **Dark mode:** the layout declares `color-scheme: light dark` and `supported-color-schemes`,
|
||||
switches to the brand's dark colours under `@media (prefers-color-scheme: dark)` (and the
|
||||
`[data-ogsc]` hooks Outlook uses), uses the dark logo when set, never puts text over a
|
||||
background image, and keeps text and button contrast at WCAG AA in both schemes. Activation
|
||||
checks the brand colours and refuses a brand that fails, quoting the ratio.
|
||||
- **MJML:** compiled at activation by `MjmlCompilerInterface` (default `CliMjmlCompiler`, which
|
||||
runs the configured `mjml` binary with a timeout). With no compiler configured, MJML versions
|
||||
cannot be activated, and say so. The compiled HTML is stored on the version.
|
||||
- **HTML:** used as given; variables are still substituted and escaped. The qbit adds the
|
||||
unsubscribe footer and headers for MARKETING and NOTIFICATION.
|
||||
- **Plain text** is generated from the rendered HTML (links as `text (url)`, headings and lists
|
||||
kept) unless the version has its own text.
|
||||
- **i18n-ready:** versions carry a locale; `EmailRequest.withLocale` picks that locale's active
|
||||
version, else the default locale's. One locale (`en`) ships.
|
||||
|
||||
### 5.3 Preview
|
||||
|
||||
**Part 2 status:** standard record widget implemented. Next's sanitizer removes
|
||||
iframes and style blocks, so it uses sanitized inline light (390px maximum) and dark
|
||||
(600px maximum) samples, with wrapping plain text. Draft MJML previews compile in memory
|
||||
with the configured compiler; stored versions are unchanged. TODO: endpoint arrives with RC.6.
|
||||
|
||||
Each version record shows a **Preview** widget: the rendered email with the sample variables in a
|
||||
sandboxed iframe (`srcdoc`, scripts disabled), Light/Dark and 390/600 px toggles, and the plain
|
||||
text. It is served through the UI API (section 10): `POST /qqq/v1/qbit/email/preview` (a version
|
||||
id, or unsaved source, plus variables) returns subject, html, text and warnings.
|
||||
|
||||
### 5.4 Part 2 implementation choices
|
||||
- Approval records `approved_version_id` on the template. An approved DRAFT freezes;
|
||||
approve a new sensitive version before activation. Activating a different,
|
||||
unapproved version clears approval. Approval is specific to one locale version.
|
||||
- `EmailBuiltIns.ensure()` seeds `email_secure_message` on its first send, using the
|
||||
database unique name and a transaction, with no static bootstrap cache.
|
||||
- `dark_primary_color` and `dark_button_text_color` optionally override their light
|
||||
counterparts; both button pairs are contrast-checked. Brand colors use #RRGGBB.
|
||||
- `EmailTemplatedMessageInput` extends core SendMessageInput for schema variables and
|
||||
record references. Ordinary core input remains compatible ad-hoc content.
|
||||
- RC.5 QueryAction masks passwords but does not apply ordinary FieldSecurityLock;
|
||||
record-variable loading additionally applies that lock to sender session keys.
|
||||
- The real outbox id is bound during the enqueue transaction, before commit. Failed
|
||||
validation persists no message. Resends preserve encrypted content and its original
|
||||
token snapshot, linking to the original message for the token binding.
|
||||
- The home follows the newer Alerts pattern: standard QQQ chart widgets plus inline,
|
||||
theme-neutral cards. All queries/counts use the viewer's session. Bounce/complaint
|
||||
dates have dedicated columns for phase-(c) events; no dates or metrics are invented.
|
||||
- Part 2 signs opt-out links and prepares headers; the receiving unsubscribe handler
|
||||
remains phase (c). Without its key/base URL, notification omits opt-out links and
|
||||
headers, while marketing refuses as specified in 8.2. Do not use opt-out delivery
|
||||
without a receiving handler. The token's address hash covers the recipient envelope.
|
||||
|
||||
## 6. Delivery tracking
|
||||
|
||||
### 6.1 Inbound route
|
||||
`EmailWebhookRouteProvider` (a `QJavalinRouteProviderInterface`, registered idempotently as in
|
||||
qbit-easypost-tracking) serves `POST /qqq/hooks/email/{kind}/{providerName}`: size cap (413), find
|
||||
the enabled provider of that kind (404), verify (401 on failure, 503 when its secret is not set),
|
||||
dedup-insert the raw payload into `email_webhook_inbox` (provider_id and external_event_id unique,
|
||||
status PENDING, attempts, payload_json, payload_sha256, received_date) and answer 200. A scheduled
|
||||
process drains the inbox into `email_event`.
|
||||
|
||||
| Kind | Verification |
|
||||
|---|---|
|
||||
| SES (SNS) | SNS message signature (SignatureVersion 1 and 2) with the certificate from `SigningCertURL`, which must be `https://sns.<region>.amazonaws.com/...pem`; the topic ARN must be the provider's. `SubscriptionConfirmation` is confirmed by fetching `SubscribeURL` (same host rule). |
|
||||
| Postmark | HTTP basic auth on the webhook URL (Postmark does not sign); password from the webhook secret env name, compared in constant time. |
|
||||
| SendGrid | Signed Event Webhook: ECDSA P-256 over timestamp + body with the public key from the env name; timestamp within 5 minutes. |
|
||||
| Mailgun | HMAC-SHA256 of timestamp + token with the signing key; timestamp within 5 minutes; a token seen before is refused. |
|
||||
|
||||
### 6.2 `email_event`
|
||||
message_id (matched by provider message id or our header), provider_id, event_type (DELIVERED,
|
||||
DEFERRED, BOUNCED, COMPLAINED, OPENED, CLICKED, UNSUBSCRIBED, REJECTED), bounce_type (HARD, SOFT,
|
||||
UNDETERMINED), recipient, occurred_date, detail (the provider's reason, truncated; never body
|
||||
content), url (clicks only). Opens and clicks are stored only when the provider row enables them.
|
||||
|
||||
## 7. PHI and PII
|
||||
|
||||
### 7.1 Classification
|
||||
Variables bound to a field (schema `table` + `field`, or `withRecordVariable`) take that field's
|
||||
classification from core `AiDataClassifications.ofField` (`DataClassification` PUBLIC,
|
||||
CONFIDENTIAL, PII, PHI; core #1142, first in 4.1.0-RC.6). Until RC.6 the qbit uses a stand-in,
|
||||
`EmailClassifications` (reflective lookup of the core class, else config-declared
|
||||
classifications), marked for replacement. Unbound variables may be declared PII or PHI in the
|
||||
schema.
|
||||
|
||||
### 7.2 Refusal
|
||||
- Activation and sending refuse a template whose variables include PII or PHI unless it is
|
||||
`phi_approved`. Only `email.admin` sets that, through the record action "Approve For PHI", which
|
||||
records who and when; a new version needs approval again.
|
||||
- A `phi_approved` template may not use open or click tracking, may not be MARKETING, and must go
|
||||
through a provider marked `phi_allowed`.
|
||||
- **The default PHI pattern** is the built-in template `email_secure_message`: "You have a new
|
||||
secure message from {{brand.label}}. Sign in to read it." with a button to the link the caller
|
||||
passes. No PHI in the body, so it needs no approval. Consumers that hold PHI use it.
|
||||
|
||||
### 7.3 Redaction and storage
|
||||
Following core `docs/SECURED-DATA.md` (section 15, QRun-IO/qqq#1161):
|
||||
- `variables_snapshot`: PII/PHI values become tokens in the format of core
|
||||
`ClassifiedValueTokens` (package `...core.actions.permissions.fieldaccess`, SECURED-DATA §14.5):
|
||||
`hmac:v1:<keyId>:<b64url(HMAC-SHA256(key, m))>`, with `m` joined by `` and the canonical
|
||||
value last, and the instance key from `QQQ_AUDIT_HMAC_KEY` (base64).
|
||||
- **Record-bound variables** use the record binding, `m = table␟field␟pk␟canonical(value)`
|
||||
(`forRecordField`), so the token matches the audit's and a field under a wipe rule gets
|
||||
`shred:v1` with the same per-record-field key: a wipe also shreds the outbox snapshot.
|
||||
- **Other values** use a domain-prefixed binding,
|
||||
`m = "email"␟template␟variable␟messageId␟canonical(value)` (`forContext`), so they never
|
||||
collide with a record binding. They get `hmac:v1` only; a low-entropy value (booleans, values
|
||||
with under 4 distinct characters) gets `changed:v1`, because there is no record key to shred.
|
||||
- No key configured: `changed:v1`.
|
||||
- Until core ships the helper, the qbit carries `EmailValueTokens`, marked
|
||||
`STAND-IN(core=#1161-impl, removeBy=RC.7)`, with the same names and output.
|
||||
- The subject and bodies of `phi_approved` messages are stored AES-256-GCM encrypted with a key
|
||||
from `QQQ_EMAIL_BODY_KEY` (no key: `phi_approved` sends are refused) and cleared after
|
||||
`bodyRetentionDays` once final.
|
||||
- Logs carry the message id and status only, never addresses, subjects or bodies.
|
||||
- Record variables are read as the sending user, through core masking, so a value the sender may
|
||||
not see arrives masked.
|
||||
- **Canary-PHI egress test:** plants `CANARY-PHI-<random>` in a PHI field, sends through every
|
||||
path (EmailService, the process, the core bridge, retry, dead letter, webhook events, an export
|
||||
of `email_message`, the home widget, captured logs) and fails on any substring of the canary
|
||||
longer than 4 characters outside the encrypted columns. It follows the core procedure so it can
|
||||
join the shared canary suite when that exists.
|
||||
|
||||
## 8. Suppression and unsubscribe
|
||||
|
||||
### 8.1 `email_suppression`
|
||||
address (lowercased), scope (ALL or a category), reason (HARD_BOUNCE, COMPLAINT, UNSUBSCRIBE,
|
||||
MANUAL), source_message_id, source_event_id, note, created_date, expires_date (optional). Unique on
|
||||
(address, scope). `email.manage` adds and removes rows; removing a COMPLAINT needs `email.admin`.
|
||||
- Hard bounce → ALL. Complaint → ALL. Unsubscribe → the message's category.
|
||||
- TRANSACTIONAL and SECURITY mail ignores UNSUBSCRIBE rows (they cannot be opted out of) but
|
||||
respects HARD_BOUNCE and COMPLAINT.
|
||||
|
||||
### 8.2 One-click unsubscribe (RFC 8058, RFC 2369)
|
||||
- MARKETING and NOTIFICATION messages carry
|
||||
`List-Unsubscribe: <https://{publicBaseUrl}/qqq/hooks/email/unsubscribe/{token}>` and
|
||||
`List-Unsubscribe-Post: List-Unsubscribe=One-Click`, and the footer links to the same URL.
|
||||
- The token is `v1.<b64url(payload)>.<b64url(HMAC-SHA256)>` over the message id, an address hash
|
||||
and the category, keyed by `QQQ_EMAIL_UNSUBSCRIBE_KEY`. With no key there is no header and
|
||||
MARKETING sends are refused. Tokens do not expire (RFC 8058 expects the link to keep working).
|
||||
- `POST` with `List-Unsubscribe=One-Click` records the suppression and returns 200. `GET` returns a
|
||||
minimal page with one "Unsubscribe" button that POSTs, because mail scanners prefetch GETs; then
|
||||
"You are unsubscribed from {category}", with a link to Preferences.
|
||||
- An `UNSUBSCRIBED` event is recorded on the message.
|
||||
|
||||
### 8.3 Preferences (core #1162)
|
||||
The qbit registers a Preferences section `email` ("Email") whose provider
|
||||
(`EmailPreferenceProvider implements UserPreferenceProviderInterface`) maps the signed-in user's
|
||||
address (`UserPreferenceRequestContext.getUserEmailAddress()`, never an address from the request)
|
||||
to `email_suppression` rows: one toggle per opt-out-able category (NOTIFICATION,
|
||||
MARKETING) and read-only rows saying that account and security mail is always sent. Qbits with
|
||||
finer settings keep their own sections: qbit-alerts' "Notifications" decides which alerts are
|
||||
emailed; this section decides whether NOTIFICATION mail reaches the address at all. It is built
|
||||
once #1162 is in an RC; one-click unsubscribe does not depend on it. With no address on the
|
||||
account, the rows are read-only ("No email address on your account") and writes are refused. The
|
||||
section's description explains the split with qbit-alerts' Notifications section.
|
||||
|
||||
## 9. Integrations
|
||||
- **Alerts (core `QAlerts`, #1108):** called reflectively when present (section 4.2).
|
||||
- **ESB and Quick Search:** `email_message`, `email_template` and `email_suppression` publish table
|
||||
events and are searchable (`EmailIntegrations`); bodies and snapshots are excluded.
|
||||
- **API:** the same tables, permissions and customizers as the admin (QQQ API middleware).
|
||||
- **Object storage (#1089):** attachments (section 2.4) and, in phase (e), inbound attachments.
|
||||
|
||||
## 10. App and UI
|
||||
- App **Email**, with one home dashboard (one HTML widget, real data only, inline styles, no inner
|
||||
or horizontal scroll, stacked cards under 640 px, light and dark):
|
||||
- sent, delivered, dead letters, bounced and complained in the last 24 h and 7 days;
|
||||
- sends per hour for 24 h (sent and failed);
|
||||
- suppression growth (new rows per day for 30 days, by reason);
|
||||
- provider health (status, last success, last failure reason, rate-limit headroom);
|
||||
- recent messages (newest 20: when, recipient masked as `p***@example.com`, subject unless
|
||||
`phi_approved`, status), each linking to its record.
|
||||
- Nav: Messages, Templates, Brands, Suppressions, Providers. Attempts, events, versions and inbox
|
||||
rows are relations on their parents.
|
||||
- UI API extension `email` (1.0.0, `/qqq/v1/qbit/email/...`): `POST /preview` (section 5.3), with
|
||||
its OpenAPI fragment committed and contract-tested. Everything else is standard tables,
|
||||
processes and widgets. The webhook and unsubscribe routes are non-UI (`/qqq/hooks/email/...`).
|
||||
|
||||
## 11. Consumers (phase d, one PR per repo)
|
||||
| Consumer | Change |
|
||||
|---|---|
|
||||
| qbit-alerts | `EmailChannel` and `NotificationEmailer` use `EmailService` with templates `alerts_notification` and `alerts_digest` (NOTIFICATION), idempotency `alerts:<id>`, linked alert. `messagingProviderName` stays as the fallback when qbit-email is absent. |
|
||||
| qbit-collaboration | an `EmailCollaborationNotifier` for mentions, shares and task reminders (templates `collaboration_mention`, `collaboration_share`, `collaboration_task_reminder`); record labels obey `isRecordLabelFullAccess`. |
|
||||
| qbit-assistant | scheduled-task results by email through `EmailService` (template `assistant_scheduled_result`), after its EXTERNAL approval rule. |
|
||||
| qbit-easypost-tracking | delivery notices through a notifier that sends `easypost_delivery_notice` (NOTIFICATION). |
|
||||
| qbit-marketing | a `MarketingTransport` over `EmailService` (MARKETING, MJML/HTML templates, unsubscribe headers, suppression honoured). |
|
||||
| kof22 (Website-Backend) | the contact form's `Mailer` is replaced by `EmailService` with template `kof22_contact`, through an SMTP provider row for PMG (`smtp.galaxy.lan:26`, no auth, no new secrets). A draft PR with a Liquibase changeset for the `email_` tables and seed rows. |
|
||||
| Authentik | **keeps its own SMTP to PMG.** It is a separate product (Python) with its own flows and templates (recovery, enrollment, MFA); it must send when the QQQ app is down, since it guards sign-in to that app; and it has no hook to call a Java service. Its mail is account and security mail, to which suppression-by-unsubscribe would not apply anyway. |
|
||||
|
||||
## 12. Phase 2: inbound (design only)
|
||||
- Sources: SES inbound (receipt rule → S3 → SNS → `/qqq/hooks/email/inbound/ses/{name}`), Mailgun
|
||||
routes, Postmark inbound, SendGrid inbound parse, and an IMAP or Gmail mailbox poller
|
||||
(scheduled).
|
||||
- `email_inbound`: provider, message id, in_reply_to, references, from, to, subject, text and html
|
||||
(sanitized), received_date, status (NEW, ROUTED, UNMATCHED, REJECTED), routed_table, routed_id,
|
||||
spam verdicts. Attachments go to qbit-object-storage (`email_inbound_attachment` rows).
|
||||
- **Reply-to-record threading:** outbound NOTIFICATION mail sets
|
||||
`Reply-To: reply+<token>@<inbound domain>`, where the token (HMAC) names the message and its
|
||||
linked record. A reply is matched by token; the sender must be the original recipient; quoted
|
||||
text is stripped; an `InboundEmailHandler` registered by the linked table's qbit acts (e.g.
|
||||
qbit-collaboration adds a comment as that user). Unmatched mail stays UNMATCHED for review.
|
||||
- PHI: inbound bodies are classified PHI by default (unknown content) and stored encrypted.
|
||||
|
||||
## 13. Data model summary
|
||||
| Table | In the nav | Written by |
|
||||
|---|---|---|
|
||||
| `email_message` | yes | the qbit (people: record actions only) |
|
||||
| `email_message_attempt` | relation | the qbit |
|
||||
| `email_event` | relation | the qbit |
|
||||
| `email_template` | yes | people (`email.manage`, per-template permission) |
|
||||
| `email_template_version` | relation | people (drafts), actions (activation) |
|
||||
| `email_brand` | yes | people |
|
||||
| `email_suppression` | yes | people and the qbit |
|
||||
| `email_provider` | yes | admins |
|
||||
| `email_webhook_inbox` | relation (on the provider) | the qbit |
|
||||
| `email_inbound`, `email_inbound_attachment` | phase (e) | the qbit |
|
||||
|
||||
Postgres DDL in `docs/ddl/email-postgres.sql` (phase a).
|
||||
|
||||
## 14. Configuration (`EmailQBitConfig`)
|
||||
backendName (required), schedulerName (required to send), publicBaseUrl (unsubscribe links),
|
||||
defaultProviderName, bridgeProviderName (`qbitEmail`), secretResolver, allowedEnvNames,
|
||||
dispatchIntervalSeconds (15), batchSize (100), maxAttempts (6), retryMinutes, leaseSeconds (120),
|
||||
bodyRetentionDays (30), messageRetentionDays (0), mjmlCompiler, classificationOverrides, appName
|
||||
and appLabel, securityFields and recordSecurityLocks (multi-tenant hosts).
|
||||
|
||||
## 15. Phases and acceptance
|
||||
| Phase | Scope | Acceptance (tests) |
|
||||
|---|---|---|
|
||||
| (a) | tables, outbox and dispatcher, SMTP and SES, Markdown/HTML/MJML templates and versions, brands and the layout, preview, EmailService, the process, the core bridge, PHI refusal and redaction, the app home | GreenMail SMTP sink end to end; SES through a mocked `SesV2Client`; the claim race (two runners, each message sent once); retry, dead letter, idempotency; rendering (variables, escaping, `#if`/`#each`, plain text, dark-mode contrast); PHI refusal; snapshot redaction; the canary test; producer standards (names, nav, permissions) |
|
||||
| (b) | Postmark, SendGrid, Mailgun, Gmail SMTP | HTTP mocks per provider (success, 4xx permanent, 429/5xx transient); the Gmail preset. HTML/MJML moved into part (a), part 2 at owner request. |
|
||||
| (c) | webhook route and inbox, events, suppression, unsubscribe, Preferences section | signature verification per provider (valid, tampered, stale, missing secret); the SNS certificate host rule; suppression on hard bounce and complaint; one-click POST and the GET page; category rules |
|
||||
| (d) | consumer migrations | each consumer's own tests; a contract test of `EmailService` |
|
||||
| (e) | inbound | design only for now |
|
||||
|
||||
Dependencies: phase (a) builds on QQQ 4.1.0-RC.5 with the classification stand-in, and moves to
|
||||
RC.6 for `DataClassification` and Preferences.
|
||||
|
||||
## 16. Owner steps (outside code)
|
||||
- **PMG (kof22):** none; the relay already accepts the cluster.
|
||||
- **SES (when a consumer adopts it):** create the SES identity for each sending domain in the
|
||||
`terraform/dns-route53` root (owner-applied), request production access, and create a
|
||||
configuration set with an SNS destination to `https://<app>/qqq/hooks/email/ses/<provider>`.
|
||||
The DNS records (DKIM CNAMEs, a custom MAIL FROM subdomain with its MX and SPF; DMARC stays at
|
||||
p=reject) go in as Praetor octoDNS PRs once the identity exists, because SES issues the DKIM
|
||||
tokens.
|
||||
- **Postmark, SendGrid, Mailgun, Gmail:** accounts and keys only if used; keys go into Infisical
|
||||
under the env names the provider rows reference.
|
||||
- **Keys:** `QQQ_EMAIL_UNSUBSCRIBE_KEY` and `QQQ_EMAIL_BODY_KEY` per deployment (Infisical, seeded
|
||||
write-only like other app keys).
|
||||
@@ -0,0 +1,215 @@
|
||||
# Email providers
|
||||
|
||||
Create an enabled `email_provider` row and choose exactly one enabled default.
|
||||
Provider edits and Send Test Email require `email.admin`. Test sends queue through
|
||||
the outbox and obey the same rate limits as application messages.
|
||||
|
||||
Field names here are database columns. QQQ record/API names are camelCase:
|
||||
`secret_env_name` is `secretEnvName`, for example. All kinds use `name`, `label`,
|
||||
`kind`, `enabled`, `is_default`, `from_address`, optional `from_name` / `reply_to`,
|
||||
and optional `max_per_minute` / `max_per_day`. When set, `sending_domain` must
|
||||
match the From address domain exactly.
|
||||
|
||||
## Credentials and delivery
|
||||
|
||||
Credential fields contain environment variable **names**, never values. Names
|
||||
must match `^[A-Z_][A-Z0-9_]{0,99}$` and the effective allowlist. The defaults
|
||||
below supplement `EmailQBitConfig.allowedEnvNames`; explicitly allow custom names.
|
||||
Allowlisting does not automatically select a name for a row.
|
||||
|
||||
| Kind | `username_env_name` | `secret_env_name` | `webhook_secret_env_name` |
|
||||
|---|---|---|---|
|
||||
| SMTP | `EMAIL_SMTP_USERNAME` (optional) | `EMAIL_SMTP_PASSWORD` | Not used |
|
||||
| SES | `AWS_ACCESS_KEY_ID` (optional) | `AWS_SECRET_ACCESS_KEY` (optional pair) | Not used; `sns_topic_arn` is a plain ARN |
|
||||
| POSTMARK | Not used | `POSTMARK_SERVER_TOKEN` | `POSTMARK_WEBHOOK_PASSWORD` |
|
||||
| SENDGRID | Not used | `SENDGRID_API_KEY` | `SENDGRID_WEBHOOK_PUBLIC_KEY` |
|
||||
| MAILGUN | Not used | `MAILGUN_API_KEY` | `MAILGUN_WEBHOOK_SIGNING_KEY` |
|
||||
| GMAIL_SMTP | `GMAIL_USERNAME` | `GMAIL_APP_PASSWORD` | Not used |
|
||||
|
||||
Legacy `EMAIL_POSTMARK_SERVER_TOKEN`, `EMAIL_POSTMARK_WEBHOOK_PASSWORD`,
|
||||
`EMAIL_SENDGRID_API_KEY`, `EMAIL_SENDGRID_WEBHOOK_PUBLIC_KEY`, `EMAIL_MAILGUN_API_KEY`,
|
||||
`EMAIL_MAILGUN_SIGNING_KEY`, `EMAIL_GMAIL_USERNAME` and `EMAIL_GMAIL_APP_PASSWORD`
|
||||
remain allowlisted for their kinds. SES also allowlists `AWS_SESSION_TOKEN` for
|
||||
the default credential chain; explicit row credentials are an access/secret pair.
|
||||
|
||||
Values resolve through `secretResolver` (default `System::getenv`) for each send
|
||||
attempt. Rotation needs no service restart. Provider record presence indicators
|
||||
also resolve values, returning only Set / Not set; enqueue and internal queries
|
||||
do not resolve credentials. Values and provider response diagnostics are never
|
||||
logged, stored in errors, returned in exceptions or cached by these transports.
|
||||
|
||||
HTTP uses fixed HTTPS endpoints, a 10-second connect timeout, a 30-second request
|
||||
timeout, no redirects and no client retries. The outbox owns retries. MIME and
|
||||
multipart bodies are built entirely in memory; services use no filesystem storage.
|
||||
|
||||
HTTP 429 and 5xx are transient. Retry-After accepts seconds or an HTTP date;
|
||||
the dispatcher caps explicit delays at one hour. Authentication/account failures
|
||||
mark the provider UNHEALTHY, stop its remaining deliveries for the dispatch pass,
|
||||
and retain messages for retry without consuming their failure budget. Permanent
|
||||
message errors become DEAD_LETTER. Exceptions use fixed text and numeric codes;
|
||||
stored errors use fixed messages, never response bodies.
|
||||
|
||||
Acceptance means SENT, not delivered. Delivery is at least once: a crash after
|
||||
acceptance can cause a duplicate. Limits count attempts across replicas in rolling
|
||||
minute and 24-hour windows, not recipients or successful deliveries. Leave room
|
||||
for retries, recipient quotas and other applications using the same account.
|
||||
|
||||
## SMTP
|
||||
|
||||
Set `kind=SMTP`, `host`, `port`, and `security=NONE`, `STARTTLS`, or `TLS`. For
|
||||
authenticated SMTP, set both credential names; for an unauthenticated relay,
|
||||
leave both empty. Authentication requires encryption except for loopback or an
|
||||
explicit host `allowPlaintextSmtpAuth` configuration.
|
||||
|
||||
Configure the relay's allowed sender domains and route. For public delivery,
|
||||
verify the domain with the operator, publish its DKIM and SPF records, and align
|
||||
the From domain with DMARC. Update the existing SPF policy rather than adding a
|
||||
second SPF record. Set limits below the relay's quota. SMTP has 10-second connect
|
||||
and 15-second read/write timeouts; there is no provider event webhook here.
|
||||
|
||||
## Amazon SES
|
||||
|
||||
Set `kind=SES`, `region`, and a verified `from_address`. Optionally set
|
||||
`sending_domain`, `configuration_set` and `sns_topic_arn`. Leave credential names
|
||||
empty for the AWS default chain (workload/instance role or AWS environment);
|
||||
otherwise supply both access-key and secret-key environment names.
|
||||
|
||||
Verify the sender identity in the sending region and publish SES's DKIM records.
|
||||
Configure SPF for a custom MAIL FROM domain if used. Request production access
|
||||
when leaving the sandbox. Configure the configuration set's SNS event destination
|
||||
and put the topic ARN in `sns_topic_arn`, not a secret field.
|
||||
|
||||
Sending quotas are regional and account-specific, with sandbox, recipient and
|
||||
rate restrictions. Set both rate fields to fit those quotas. SES sends raw MIME
|
||||
with explicit envelope destinations and strips Bcc from its MIME headers.
|
||||
|
||||
## Postmark
|
||||
|
||||
Set `kind=POSTMARK`, `secret_env_name=POSTMARK_SERVER_TOKEN`, a verified
|
||||
`from_address`, and optionally `sending_domain`. Use a server token, not an
|
||||
account token. `message_stream` defaults to `outbound`; another transactional or
|
||||
broadcast stream must exist on that server. `track_opens` and `track_clicks`
|
||||
default to false; click tracking maps to HtmlAndText or None.
|
||||
|
||||
Verify the domain or sender signature, publish Postmark's DKIM and recommended
|
||||
Return-Path/SPF DNS records, obtain account approval and create the desired
|
||||
message stream. The endpoint is `https://api.postmarkapp.com/email`;
|
||||
the returned `MessageID` is retained.
|
||||
|
||||
| Signal | Outcome |
|
||||
|---|---|
|
||||
| ErrorCode 10 | Provider failure: bad token |
|
||||
| ErrorCode 300 / 406 | Permanent: invalid request / inactive recipient |
|
||||
| ErrorCode 400/401, 405, 412, 413, 422 | Provider failure: signature, sending approval or server/account configuration |
|
||||
| ErrorCode 1235/1236 / 1480 | Provider failure: stream / IP configuration |
|
||||
| HTTP 401/403 | Provider failure |
|
||||
| HTTP 429 / 5xx | Transient; takes precedence over a body code |
|
||||
| Other request errors | Permanent; malformed successful responses are transient |
|
||||
|
||||
API ErrorCode 413 is an account problem; HTTP 413 is a payload-size error.
|
||||
Check plan, recipient and payload limits before setting the rate fields. The
|
||||
single-message API documents a 10 MB message limit. See
|
||||
[Postmark API errors and limits](https://postmarkapp.com/developer/api/overview).
|
||||
|
||||
## SendGrid
|
||||
|
||||
Set `kind=SENDGRID`, `secret_env_name=SENDGRID_API_KEY`, a verified `from_address`
|
||||
and optionally `sending_domain`. Create a key with Mail Send permission.
|
||||
Authenticate the domain and publish SendGrid's DKIM/SPF DNS records; single-sender
|
||||
verification can be used for initial tests. Confirm the account can send.
|
||||
|
||||
`track_opens` controls open tracking; `track_clicks` controls HTML and plain-text
|
||||
click tracking. Explicit false values override account tracking defaults.
|
||||
The global endpoint is `https://api.sendgrid.com/v3/mail/send`; only HTTP 202
|
||||
counts as success and `X-Message-Id` becomes the provider message ID.
|
||||
|
||||
One personalization contains To/Cc/Bcc with display names. `custom_args` carries
|
||||
`qqq_email_id`. Reserved headers are filtered case-insensitively: `x-sg-id`,
|
||||
`x-sg-eid`, `received`, `dkim-signature`, `Content-Type`, `Content-Transfer-Encoding`,
|
||||
`To`, `From`, `Subject`, `Reply-To`, `CC`, `BCC`. `List-Unsubscribe` and
|
||||
`List-Unsubscribe-Post` are retained.
|
||||
|
||||
HTTP 401/403 are provider failures, 400/413 permanent, and 429/5xx transient.
|
||||
Set rate limits for the account's plan/API quota. Payload and recipient limits
|
||||
apply separately from outbox attempt limits. See
|
||||
[SendGrid Mail Send](https://www.twilio.com/docs/sendgrid/api-reference/mail-send/mail-send).
|
||||
|
||||
## Mailgun
|
||||
|
||||
Set `kind=MAILGUN`, `secret_env_name=MAILGUN_API_KEY`, `sending_domain` (required,
|
||||
a DNS name without a scheme/path), and `region=US` or `EU` (required).
|
||||
The From address domain must match the sending domain.
|
||||
|
||||
Add and verify the domain in that region. Publish Mailgun's SPF/DKIM records and
|
||||
its requested tracking DNS records if enabling tracking. Use an API key allowed
|
||||
to send for that domain. Sandbox domains restrict recipients; check production
|
||||
status and plan/rate quotas before choosing `max_per_minute` / `max_per_day`.
|
||||
Those fields do not enforce MIME-size or recipient limits.
|
||||
|
||||
The endpoint is `https://api.mailgun.net/v3/{sending_domain}/messages.mime`, or
|
||||
`https://api.eu.mailgun.net/v3/{sending_domain}/messages.mime` for EU. Basic auth
|
||||
uses `api:<resolved key>`. Multipart includes all To/Cc/Bcc mailboxes as envelope
|
||||
destinations, MIME `message` with Bcc stripped, `o:tracking-opens`,
|
||||
`o:tracking-clicks` (explicit yes/no from the row), and `v:qqq_email_id`.
|
||||
The response `id` is stored without surrounding angle brackets.
|
||||
|
||||
HTTP 401/403 are provider failures, 400 permanent, and 429/5xx transient.
|
||||
|
||||
## Gmail SMTP
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| `kind` | `GMAIL_SMTP` |
|
||||
| `username_env_name` | `GMAIL_USERNAME`: value is the full account email address |
|
||||
| `secret_env_name` | `GMAIL_APP_PASSWORD`: value is the generated app password |
|
||||
| `security` | `STARTTLS` |
|
||||
| `from_address` | Account address or a permitted configured alias |
|
||||
| `max_per_day` | At most about 500 personal / 2000 paid Workspace; lower with headroom |
|
||||
|
||||
The preset always uses `smtp.gmail.com:587` with required STARTTLS, ignoring
|
||||
custom host/port values. Both env names are required; other security values are
|
||||
refused even if plaintext auth is enabled for other relays. An omitted security
|
||||
value uses the preset at the settings layer, but provider forms default to NONE,
|
||||
so select STARTTLS when creating the row.
|
||||
|
||||
Enable 2-step verification and generate an app password for the service; do not
|
||||
use the normal account password. Account/admin security policy may disable app
|
||||
passwords. A Google password change revokes existing app passwords, requiring a
|
||||
new environment value. See
|
||||
[Google app password setup](https://support.google.com/accounts/answer/185833).
|
||||
|
||||
Google manages domain authentication for personal gmail.com senders. For
|
||||
Workspace, verify the domain and configure Google's SPF/DKIM records and DMARC.
|
||||
Trials, new accounts, SMTP recipient quotas and other account users can reduce
|
||||
available capacity. The 500/2000 limit is an upper planning bound, not guaranteed
|
||||
throughput. See [Workspace limits](https://support.google.com/a/answer/166852)
|
||||
and [personal Gmail limits](https://support.google.com/mail/answer/22839).
|
||||
|
||||
`GMAIL_API` is refused with:
|
||||
`Gmail API sending is specified but not built; use GMAIL_SMTP`.
|
||||
It would require a Google Cloud project, OAuth consent, client credentials and
|
||||
refresh-token management. It adds no needed system-mail capability over the app
|
||||
password preset in this phase and is built only on request (DESIGN 3.6).
|
||||
|
||||
## Event webhooks: phase (c)
|
||||
|
||||
Event ingestion is **not installed by phase (b)**. After phase (c) is deployed,
|
||||
configure provider event delivery at:
|
||||
|
||||
`https://<app>/qqq/hooks/email/{kind}/{providerName}`
|
||||
|
||||
Use the configured provider name and provider-kind route defined by phase (c).
|
||||
Do not expect delivery/bounce/complaint/open/click updates until the receiver
|
||||
exists. Tracking options can operate on the provider side now.
|
||||
|
||||
| Provider | Event setup to prepare |
|
||||
|---|---|
|
||||
| Postmark | Server/stream event webhooks with Basic auth; `POSTMARK_WEBHOOK_PASSWORD` |
|
||||
| SendGrid | Signed Event Webhook; verification key in `SENDGRID_WEBHOOK_PUBLIC_KEY` |
|
||||
| Mailgun | Regional domain event webhooks; `MAILGUN_WEBHOOK_SIGNING_KEY`, separate from the sending key |
|
||||
| SES | Configuration-set events to SNS, HTTP subscription and `sns_topic_arn`; signature/topic verification arrives in phase (c) |
|
||||
| SMTP / Gmail SMTP | No native provider event webhook in this design |
|
||||
|
||||
Webhook environment names are allowlisted now so rows can be prepared; outbound
|
||||
sends never resolve them. Verification, durable event processing and unsubscribe
|
||||
handling arrive in later phases.
|
||||
@@ -0,0 +1,186 @@
|
||||
-- Phase (a), parts 1 and 2. Apply before producing the qbit. Host security columns must be added by the host.
|
||||
|
||||
CREATE TABLE email_provider (
|
||||
id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
|
||||
create_date TIMESTAMP,
|
||||
modify_date TIMESTAMP,
|
||||
name VARCHAR(500),
|
||||
label VARCHAR(500),
|
||||
kind VARCHAR(500),
|
||||
enabled BOOLEAN,
|
||||
is_default BOOLEAN,
|
||||
from_address VARCHAR(500),
|
||||
from_name VARCHAR(500),
|
||||
reply_to VARCHAR(500),
|
||||
sending_domain VARCHAR(500),
|
||||
host VARCHAR(500),
|
||||
port INTEGER,
|
||||
security VARCHAR(500),
|
||||
region VARCHAR(500),
|
||||
username_env_name VARCHAR(500),
|
||||
secret_env_name VARCHAR(500),
|
||||
webhook_secret_env_name VARCHAR(500),
|
||||
sns_topic_arn VARCHAR(500),
|
||||
configuration_set VARCHAR(500),
|
||||
message_stream VARCHAR(500),
|
||||
max_per_minute INTEGER,
|
||||
max_per_day INTEGER,
|
||||
track_opens BOOLEAN,
|
||||
track_clicks BOOLEAN,
|
||||
phi_allowed BOOLEAN,
|
||||
health_status VARCHAR(500),
|
||||
last_success_date TIMESTAMP,
|
||||
last_failure_date TIMESTAMP,
|
||||
last_failure_message VARCHAR(500),
|
||||
dispatch_lock INTEGER,
|
||||
default_slot INTEGER,
|
||||
UNIQUE (name),
|
||||
UNIQUE (default_slot)
|
||||
);
|
||||
|
||||
CREATE TABLE email_message (
|
||||
id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
|
||||
create_date TIMESTAMP,
|
||||
modify_date TIMESTAMP,
|
||||
status VARCHAR(500),
|
||||
to_addresses TEXT,
|
||||
cc_addresses TEXT,
|
||||
bcc_addresses TEXT,
|
||||
from_address VARCHAR(500),
|
||||
reply_to VARCHAR(500),
|
||||
subject TEXT,
|
||||
template_id INTEGER,
|
||||
template_version_id INTEGER,
|
||||
brand_id INTEGER,
|
||||
category VARCHAR(500),
|
||||
locale VARCHAR(500),
|
||||
variables_snapshot TEXT,
|
||||
html_body TEXT,
|
||||
text_body TEXT,
|
||||
body_sha256 VARCHAR(500),
|
||||
phi_approved BOOLEAN,
|
||||
unsubscribe_url TEXT,
|
||||
idempotency_key VARCHAR(500),
|
||||
provider_id INTEGER,
|
||||
provider_message_id VARCHAR(500),
|
||||
attempt_count INTEGER,
|
||||
next_attempt_date TIMESTAMP,
|
||||
lease_expires_date TIMESTAMP,
|
||||
last_error VARCHAR(500),
|
||||
finished_date TIMESTAMP,
|
||||
sent_date TIMESTAMP,
|
||||
delivered_date TIMESTAMP,
|
||||
bounced_date TIMESTAMP,
|
||||
complained_date TIMESTAMP,
|
||||
dead_letter_date TIMESTAMP,
|
||||
linked_table_name VARCHAR(500),
|
||||
linked_record_id VARCHAR(500),
|
||||
sender_qbit VARCHAR(500),
|
||||
sender_process VARCHAR(500),
|
||||
suppressed_addresses TEXT,
|
||||
send_after TIMESTAMP,
|
||||
UNIQUE (idempotency_key)
|
||||
);
|
||||
|
||||
CREATE TABLE email_message_attempt (
|
||||
id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
|
||||
create_date TIMESTAMP,
|
||||
modify_date TIMESTAMP,
|
||||
message_id INTEGER NOT NULL,
|
||||
attempt_number INTEGER NOT NULL,
|
||||
outcome VARCHAR(500),
|
||||
provider_id INTEGER,
|
||||
claimed_by VARCHAR(500),
|
||||
start_date TIMESTAMP,
|
||||
end_date TIMESTAMP,
|
||||
lease_expires_date TIMESTAMP,
|
||||
duration_millis INTEGER,
|
||||
error VARCHAR(500),
|
||||
UNIQUE (message_id, attempt_number)
|
||||
);
|
||||
|
||||
CREATE TABLE email_template (
|
||||
id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
|
||||
create_date TIMESTAMP,
|
||||
modify_date TIMESTAMP,
|
||||
name VARCHAR(500),
|
||||
label VARCHAR(500),
|
||||
category VARCHAR(500),
|
||||
kind VARCHAR(500),
|
||||
brand_id INTEGER,
|
||||
active_version_id INTEGER,
|
||||
approved_version_id INTEGER,
|
||||
version_lock INTEGER,
|
||||
phi_approved BOOLEAN,
|
||||
phi_approved_by VARCHAR(500),
|
||||
phi_approved_date TIMESTAMP,
|
||||
edit_permission_name VARCHAR(500),
|
||||
owner_qbit VARCHAR(500),
|
||||
description TEXT,
|
||||
UNIQUE (name)
|
||||
);
|
||||
|
||||
CREATE TABLE email_template_version (
|
||||
id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
|
||||
create_date TIMESTAMP,
|
||||
modify_date TIMESTAMP,
|
||||
template_id INTEGER,
|
||||
version_number INTEGER,
|
||||
status VARCHAR(500),
|
||||
locale VARCHAR(500),
|
||||
subject TEXT,
|
||||
preheader TEXT,
|
||||
body_source TEXT,
|
||||
text_source TEXT,
|
||||
compiled_html TEXT,
|
||||
variables_schema TEXT,
|
||||
sample_variables TEXT,
|
||||
created_by VARCHAR(500),
|
||||
activated_date TIMESTAMP,
|
||||
UNIQUE (template_id, version_number, locale)
|
||||
);
|
||||
|
||||
CREATE TABLE email_brand (
|
||||
id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
|
||||
create_date TIMESTAMP,
|
||||
modify_date TIMESTAMP,
|
||||
name VARCHAR(500),
|
||||
label VARCHAR(500),
|
||||
logo_url VARCHAR(500),
|
||||
logo_dark_url VARCHAR(500),
|
||||
logo_alt VARCHAR(500),
|
||||
primary_color VARCHAR(500),
|
||||
text_color VARCHAR(500),
|
||||
background_color VARCHAR(500),
|
||||
dark_background_color VARCHAR(500),
|
||||
dark_text_color VARCHAR(500),
|
||||
dark_primary_color VARCHAR(500),
|
||||
dark_button_text_color VARCHAR(500),
|
||||
button_text_color VARCHAR(500),
|
||||
footer_markdown TEXT,
|
||||
postal_address TEXT,
|
||||
social_links TEXT,
|
||||
site_url VARCHAR(500),
|
||||
is_default BOOLEAN,
|
||||
app_name VARCHAR(500),
|
||||
UNIQUE (name)
|
||||
);
|
||||
|
||||
CREATE TABLE email_suppression (
|
||||
id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
|
||||
create_date TIMESTAMP,
|
||||
modify_date TIMESTAMP,
|
||||
address VARCHAR(500),
|
||||
scope VARCHAR(500),
|
||||
reason VARCHAR(500),
|
||||
source_message_id INTEGER,
|
||||
source_event_id INTEGER,
|
||||
note TEXT,
|
||||
created_date TIMESTAMP,
|
||||
expires_date TIMESTAMP,
|
||||
UNIQUE (address, scope)
|
||||
);
|
||||
|
||||
CREATE INDEX email_message_due ON email_message (status, next_attempt_date);
|
||||
CREATE INDEX email_attempt_provider_start ON email_message_attempt (provider_id, start_date);
|
||||
CREATE INDEX email_attempt_provider_end ON email_message_attempt (provider_id, end_date, id);
|
||||
@@ -27,6 +27,9 @@
|
||||
<maven.compiler.target>21</maven.compiler.target>
|
||||
<jacoco.version>0.8.15</jacoco.version>
|
||||
<h2.version>2.2.224</h2.version>
|
||||
<commonmark.version>0.27.1</commonmark.version>
|
||||
<awssdk.version>2.41.34</awssdk.version>
|
||||
<greenmail.version>2.0.1</greenmail.version>
|
||||
</properties>
|
||||
|
||||
<parent>
|
||||
@@ -77,6 +80,37 @@
|
||||
<artifactId>qqq-backend-core</artifactId>
|
||||
</dependency>
|
||||
|
||||
<!-- Markdown templates (DESIGN 5.2). -->
|
||||
<dependency>
|
||||
<groupId>org.commonmark</groupId>
|
||||
<artifactId>commonmark</artifactId>
|
||||
<version>${commonmark.version}</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.commonmark</groupId>
|
||||
<artifactId>commonmark-ext-gfm-tables</artifactId>
|
||||
<version>${commonmark.version}</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.commonmark</groupId>
|
||||
<artifactId>commonmark-ext-autolink</artifactId>
|
||||
<version>${commonmark.version}</version>
|
||||
</dependency>
|
||||
|
||||
<!-- SES transport (DESIGN 3.4): AWS SDK v2 sesv2. -->
|
||||
<dependency>
|
||||
<groupId>software.amazon.awssdk</groupId>
|
||||
<artifactId>sesv2</artifactId>
|
||||
<version>${awssdk.version}</version>
|
||||
</dependency>
|
||||
|
||||
<!-- Webhook and unsubscribe routes (DESIGN 6, 8); the host app provides Javalin. -->
|
||||
<dependency>
|
||||
<groupId>com.kingsrook.qqq</groupId>
|
||||
<artifactId>qqq-middleware-javalin</artifactId>
|
||||
<scope>provided</scope>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>com.kingsrook.qqq</groupId>
|
||||
<artifactId>qqq-backend-module-rdbms</artifactId>
|
||||
@@ -84,7 +118,14 @@
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>com.kingsrook.qqq</groupId>
|
||||
<artifactId>qqq-middleware-javalin</artifactId>
|
||||
<artifactId>qqq-frontend-next</artifactId>
|
||||
<version>1.0.0-RC.19</version>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>com.icegreen</groupId>
|
||||
<artifactId>greenmail-junit5</artifactId>
|
||||
<version>${greenmail.version}</version>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
<dependency>
|
||||
@@ -143,6 +184,11 @@
|
||||
<groupId>org.jacoco</groupId>
|
||||
<artifactId>jacoco-maven-plugin</artifactId>
|
||||
<version>${jacoco.version}</version>
|
||||
<configuration>
|
||||
<excludes>
|
||||
<exclude>com/kingsrook/qbits/email/demo/**</exclude>
|
||||
</excludes>
|
||||
</configuration>
|
||||
<executions>
|
||||
<execution>
|
||||
<id>prepare-agent</id>
|
||||
@@ -195,7 +241,7 @@
|
||||
<activeByDefault>true</activeByDefault>
|
||||
</activation>
|
||||
<properties>
|
||||
<qqq.snapshot.version>4.1.0-RC.4</qqq.snapshot.version>
|
||||
<qqq.snapshot.version>4.1.0-RC.5</qqq.snapshot.version>
|
||||
</properties>
|
||||
<dependencyManagement>
|
||||
<dependencies>
|
||||
|
||||
@@ -0,0 +1,94 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email;
|
||||
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTables;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** What a host wires into its ESB, Quick Search and API, without this qbit
|
||||
** depending on those modules (the same opt-in hooks as qbit-sql-workbench).
|
||||
**
|
||||
** <ul>
|
||||
** <li>{@link #esbTableNames()}: publish these tables' inserts, updates and
|
||||
** deletes to an ESB topic (the main tables).</li>
|
||||
** <li>{@link #quickSearchTables()}: index these in Quick Search, with field
|
||||
** weights.</li>
|
||||
** <li>{@link #apiTableNames()}: expose these in the host's QQQ API. The API
|
||||
** gets exactly the admin's rules: the same permissions, customizers and
|
||||
** read-only tables.</li>
|
||||
** </ul>
|
||||
*******************************************************************************/
|
||||
public final class EmailIntegrations
|
||||
{
|
||||
/*******************************************************************************
|
||||
** A table for Quick Search: field weights, the timestamp for incremental
|
||||
** indexing, and the record label.
|
||||
*******************************************************************************/
|
||||
public record SearchableTable(String tableName, Map<String, Integer> fieldWeights, String basepullTimestamp, String recordLabelFormat, List<String> recordLabelFields)
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private EmailIntegrations()
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Tables to publish on the ESB.
|
||||
*******************************************************************************/
|
||||
public static List<String> esbTableNames()
|
||||
{
|
||||
return (List.of(EmailTables.MESSAGE, EmailTables.TEMPLATE, EmailTables.SUPPRESSION));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Tables for the host's QQQ API.
|
||||
*******************************************************************************/
|
||||
public static List<String> apiTableNames()
|
||||
{
|
||||
return (List.of(EmailTables.MESSAGE, EmailTables.TEMPLATE, EmailTables.SUPPRESSION));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Tables for Quick Search.
|
||||
*******************************************************************************/
|
||||
public static List<SearchableTable> quickSearchTables()
|
||||
{
|
||||
return (List.of(
|
||||
new SearchableTable(EmailTables.MESSAGE, Map.of("id", 10, "providerMessageId", 8, "status", 4, "linkedRecordId", 3), "modifyDate", "%s", List.of("id")),
|
||||
new SearchableTable(EmailTables.TEMPLATE, Map.of("name", 10, "label", 8, "description", 3), "modifyDate", "%s", List.of("name")),
|
||||
new SearchableTable(EmailTables.SUPPRESSION, Map.of("address", 10, "scope", 4, "reason", 3), "modifyDate", "%s", List.of("address"))));
|
||||
}
|
||||
}
|
||||
@@ -21,41 +21,142 @@
|
||||
package com.kingsrook.qbits.email;
|
||||
|
||||
|
||||
import java.util.HashSet;
|
||||
import java.util.List;
|
||||
import java.util.Set;
|
||||
import java.util.function.Function;
|
||||
import java.util.regex.Pattern;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.producers.MetaDataCustomizerInterface;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitConfig;
|
||||
import com.kingsrook.qqq.backend.core.utils.StringUtils;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.security.RecordSecurityLock;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Configuration for the email qbit. See docs/DESIGN.md.
|
||||
** Configuration for phase (a) email delivery.
|
||||
*******************************************************************************/
|
||||
public class EmailQBitConfig implements QBitConfig
|
||||
{
|
||||
private com.kingsrook.qbits.email.template.MjmlCompilerInterface mjmlCompiler;
|
||||
private java.util.Map<String, com.kingsrook.qbits.email.phi.EmailClassifications.Level> classificationOverrides = java.util.Map.of();
|
||||
private java.util.function.Supplier<com.kingsrook.qbits.email.transport.EmailTransportRegistry> transportRegistryFactory = com.kingsrook.qbits.email.transport.EmailTransportRegistry::new;
|
||||
private Set<String> allowedImageHosts = Set.of();
|
||||
private String backendName;
|
||||
private String schedulerName;
|
||||
private String publicBaseUrl;
|
||||
private String defaultProviderName;
|
||||
private String bridgeProviderName = "qbitEmail";
|
||||
private Function<String, String> secretResolver = System::getenv;
|
||||
private boolean allowPlaintextSmtpAuth = false;
|
||||
private Set<String> allowedEnvNames = Set.of();
|
||||
private Integer dispatchIntervalSeconds = 15;
|
||||
private Integer batchSize = 100;
|
||||
private Integer maxAttempts = 6;
|
||||
private List<Integer> retryMinutes = List.of(1, 5, 15, 60, 240, 720);
|
||||
private Integer leaseSeconds = 120;
|
||||
private Integer bodyRetentionDays = 30;
|
||||
private Integer messageRetentionDays = 0;
|
||||
private String appName = "email";
|
||||
private String appLabel = "Email";
|
||||
private List<QFieldMetaData> securityFields = List.of();
|
||||
private List<RecordSecurityLock> recordSecurityLocks = List.of();
|
||||
private MetaDataCustomizerInterface<QTableMetaData> tableMetaDataCustomizer;
|
||||
|
||||
|
||||
public static final Pattern ENV_NAME = Pattern.compile("^[A-Z_][A-Z0-9_]{0,99}$");
|
||||
|
||||
/*******************************************************************************
|
||||
** Validate the configuration against the instance.
|
||||
** Whether the host explicitly permits SMTP credentials without TLS.
|
||||
*******************************************************************************/
|
||||
public boolean getAllowPlaintextSmtpAuth()
|
||||
{
|
||||
return (allowPlaintextSmtpAuth);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Opt in to plaintext SMTP authentication outside loopback.
|
||||
*******************************************************************************/
|
||||
public EmailQBitConfig withAllowPlaintextSmtpAuth(boolean allowPlaintextSmtpAuth)
|
||||
{
|
||||
this.allowPlaintextSmtpAuth = allowPlaintextSmtpAuth;
|
||||
return (this);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Validate against the host instance.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void validate(QInstance qInstance, List<String> errors)
|
||||
public void validate(QInstance instance, List<String> errors)
|
||||
{
|
||||
if(!StringUtils.hasContent(backendName))
|
||||
if(backendName == null || backendName.isBlank())
|
||||
{
|
||||
errors.add("EmailQBitConfig: backendName is required");
|
||||
errors.add("backendName is required");
|
||||
}
|
||||
else if(qInstance.getBackend(backendName) == null)
|
||||
else if(instance.getBackend(backendName) == null)
|
||||
{
|
||||
errors.add("EmailQBitConfig: backend not found: " + backendName);
|
||||
errors.add("backend not found: " + backendName);
|
||||
}
|
||||
if(schedulerName != null && instance.getScheduler(schedulerName) == null)
|
||||
{
|
||||
errors.add("schedulerName is not a scheduler in the instance");
|
||||
}
|
||||
if(maxAttempts == null || maxAttempts < 1 || batchSize == null || batchSize < 1 || dispatchIntervalSeconds == null || dispatchIntervalSeconds < 1)
|
||||
{
|
||||
errors.add("Attempts, batch size and dispatch interval must be positive");
|
||||
}
|
||||
if(leaseSeconds == null || leaseSeconds < 60)
|
||||
{
|
||||
errors.add("leaseSeconds must be at least 60, longer than transport timeouts");
|
||||
}
|
||||
if(retryMinutes == null || retryMinutes.isEmpty() || retryMinutes.stream().anyMatch(n -> n == null || n < 1))
|
||||
{
|
||||
errors.add("retryMinutes must contain positive delays");
|
||||
}
|
||||
if(bodyRetentionDays == null || bodyRetentionDays < 0 || messageRetentionDays == null || messageRetentionDays < 0)
|
||||
{
|
||||
errors.add("Retention days must not be negative");
|
||||
}
|
||||
if(secretResolver == null || allowedEnvNames == null || allowedEnvNames.stream().anyMatch(n -> n == null || !ENV_NAME.matcher(n).matches()))
|
||||
{
|
||||
errors.add("A secret resolver and valid allowed environment names are required");
|
||||
}
|
||||
if(bridgeProviderName == null || bridgeProviderName.isBlank() || appName == null || appName.isBlank())
|
||||
{
|
||||
errors.add("Bridge provider and app names are required");
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Retry minutes after a one-based attempt, repeating the final delay.
|
||||
*******************************************************************************/
|
||||
public int retryMinutesAfterAttempt(int number)
|
||||
{
|
||||
return (retryMinutes.get(Math.max(0, Math.min(number - 1, retryMinutes.size() - 1))));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Allowed names include the documented defaults for this provider kind.
|
||||
*******************************************************************************/
|
||||
public Set<String> effectiveAllowedEnvNames(String kind)
|
||||
{
|
||||
Set<String> names = new HashSet<>(allowedEnvNames);
|
||||
names.addAll(switch(kind == null ? "" : kind)
|
||||
{
|
||||
case "SMTP", "GMAIL_SMTP" -> Set.of("EMAIL_SMTP_USERNAME", "EMAIL_SMTP_PASSWORD", "EMAIL_GMAIL_USERNAME", "EMAIL_GMAIL_APP_PASSWORD", "GMAIL_USERNAME", "GMAIL_APP_PASSWORD");
|
||||
case "SES" -> Set.of("AWS_ACCESS_KEY_ID", "AWS_SECRET_ACCESS_KEY", "AWS_SESSION_TOKEN");
|
||||
case "POSTMARK" -> Set.of("EMAIL_POSTMARK_SERVER_TOKEN", "EMAIL_POSTMARK_WEBHOOK_PASSWORD", "POSTMARK_SERVER_TOKEN", "POSTMARK_WEBHOOK_PASSWORD");
|
||||
case "SENDGRID" -> Set.of("EMAIL_SENDGRID_API_KEY", "EMAIL_SENDGRID_WEBHOOK_PUBLIC_KEY", "SENDGRID_API_KEY", "SENDGRID_WEBHOOK_PUBLIC_KEY");
|
||||
case "MAILGUN" -> Set.of("EMAIL_MAILGUN_API_KEY", "EMAIL_MAILGUN_SIGNING_KEY", "MAILGUN_API_KEY", "MAILGUN_WEBHOOK_SIGNING_KEY");
|
||||
case "GMAIL_API" -> Set.of("EMAIL_GMAIL_CLIENT_ID", "EMAIL_GMAIL_CLIENT_SECRET", "EMAIL_GMAIL_REFRESH_TOKEN");
|
||||
default -> Set.of();
|
||||
});
|
||||
return (names);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for backendName
|
||||
** Getter for backendName.
|
||||
*******************************************************************************/
|
||||
public String getBackendName()
|
||||
{
|
||||
@@ -63,13 +164,423 @@ public class EmailQBitConfig implements QBitConfig
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for backendName
|
||||
** Fluent setter for backendName.
|
||||
*******************************************************************************/
|
||||
public EmailQBitConfig withBackendName(String backendName)
|
||||
{
|
||||
this.backendName = backendName;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for schedulerName.
|
||||
*******************************************************************************/
|
||||
public String getSchedulerName()
|
||||
{
|
||||
return (schedulerName);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for schedulerName.
|
||||
*******************************************************************************/
|
||||
public EmailQBitConfig withSchedulerName(String schedulerName)
|
||||
{
|
||||
this.schedulerName = schedulerName;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for publicBaseUrl.
|
||||
*******************************************************************************/
|
||||
public String getPublicBaseUrl()
|
||||
{
|
||||
return (publicBaseUrl);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for publicBaseUrl.
|
||||
*******************************************************************************/
|
||||
public EmailQBitConfig withPublicBaseUrl(String publicBaseUrl)
|
||||
{
|
||||
this.publicBaseUrl = publicBaseUrl;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for defaultProviderName.
|
||||
*******************************************************************************/
|
||||
public String getDefaultProviderName()
|
||||
{
|
||||
return (defaultProviderName);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for defaultProviderName.
|
||||
*******************************************************************************/
|
||||
public EmailQBitConfig withDefaultProviderName(String defaultProviderName)
|
||||
{
|
||||
this.defaultProviderName = defaultProviderName;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for bridgeProviderName.
|
||||
*******************************************************************************/
|
||||
public String getBridgeProviderName()
|
||||
{
|
||||
return (bridgeProviderName);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for bridgeProviderName.
|
||||
*******************************************************************************/
|
||||
public EmailQBitConfig withBridgeProviderName(String bridgeProviderName)
|
||||
{
|
||||
this.bridgeProviderName = bridgeProviderName;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for secretResolver.
|
||||
*******************************************************************************/
|
||||
public Function<String, String> getSecretResolver()
|
||||
{
|
||||
return (secretResolver);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for secretResolver.
|
||||
*******************************************************************************/
|
||||
public EmailQBitConfig withSecretResolver(Function<String, String> secretResolver)
|
||||
{
|
||||
this.secretResolver = secretResolver;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for allowedEnvNames.
|
||||
*******************************************************************************/
|
||||
public Set<String> getAllowedEnvNames()
|
||||
{
|
||||
return (allowedEnvNames);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for allowedEnvNames.
|
||||
*******************************************************************************/
|
||||
public EmailQBitConfig withAllowedEnvNames(Set<String> allowedEnvNames)
|
||||
{
|
||||
this.allowedEnvNames = allowedEnvNames;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for dispatchIntervalSeconds.
|
||||
*******************************************************************************/
|
||||
public Integer getDispatchIntervalSeconds()
|
||||
{
|
||||
return (dispatchIntervalSeconds);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for dispatchIntervalSeconds.
|
||||
*******************************************************************************/
|
||||
public EmailQBitConfig withDispatchIntervalSeconds(Integer dispatchIntervalSeconds)
|
||||
{
|
||||
this.dispatchIntervalSeconds = dispatchIntervalSeconds;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for batchSize.
|
||||
*******************************************************************************/
|
||||
public Integer getBatchSize()
|
||||
{
|
||||
return (batchSize);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for batchSize.
|
||||
*******************************************************************************/
|
||||
public EmailQBitConfig withBatchSize(Integer batchSize)
|
||||
{
|
||||
this.batchSize = batchSize;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for maxAttempts.
|
||||
*******************************************************************************/
|
||||
public Integer getMaxAttempts()
|
||||
{
|
||||
return (maxAttempts);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for maxAttempts.
|
||||
*******************************************************************************/
|
||||
public EmailQBitConfig withMaxAttempts(Integer maxAttempts)
|
||||
{
|
||||
this.maxAttempts = maxAttempts;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for retryMinutes.
|
||||
*******************************************************************************/
|
||||
public List<Integer> getRetryMinutes()
|
||||
{
|
||||
return (retryMinutes);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for retryMinutes.
|
||||
*******************************************************************************/
|
||||
public EmailQBitConfig withRetryMinutes(List<Integer> retryMinutes)
|
||||
{
|
||||
this.retryMinutes = retryMinutes;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for leaseSeconds.
|
||||
*******************************************************************************/
|
||||
public Integer getLeaseSeconds()
|
||||
{
|
||||
return (leaseSeconds);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for leaseSeconds.
|
||||
*******************************************************************************/
|
||||
public EmailQBitConfig withLeaseSeconds(Integer leaseSeconds)
|
||||
{
|
||||
this.leaseSeconds = leaseSeconds;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for bodyRetentionDays.
|
||||
*******************************************************************************/
|
||||
public Integer getBodyRetentionDays()
|
||||
{
|
||||
return (bodyRetentionDays);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for bodyRetentionDays.
|
||||
*******************************************************************************/
|
||||
public EmailQBitConfig withBodyRetentionDays(Integer bodyRetentionDays)
|
||||
{
|
||||
this.bodyRetentionDays = bodyRetentionDays;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for messageRetentionDays.
|
||||
*******************************************************************************/
|
||||
public Integer getMessageRetentionDays()
|
||||
{
|
||||
return (messageRetentionDays);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for messageRetentionDays.
|
||||
*******************************************************************************/
|
||||
public EmailQBitConfig withMessageRetentionDays(Integer messageRetentionDays)
|
||||
{
|
||||
this.messageRetentionDays = messageRetentionDays;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for appName.
|
||||
*******************************************************************************/
|
||||
public String getAppName()
|
||||
{
|
||||
return (appName);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for appName.
|
||||
*******************************************************************************/
|
||||
public EmailQBitConfig withAppName(String appName)
|
||||
{
|
||||
this.appName = appName;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for appLabel.
|
||||
*******************************************************************************/
|
||||
public String getAppLabel()
|
||||
{
|
||||
return (appLabel);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for appLabel.
|
||||
*******************************************************************************/
|
||||
public EmailQBitConfig withAppLabel(String appLabel)
|
||||
{
|
||||
this.appLabel = appLabel;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for securityFields.
|
||||
*******************************************************************************/
|
||||
public List<QFieldMetaData> getSecurityFields()
|
||||
{
|
||||
return (securityFields);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for securityFields.
|
||||
*******************************************************************************/
|
||||
public EmailQBitConfig withSecurityFields(List<QFieldMetaData> securityFields)
|
||||
{
|
||||
this.securityFields = securityFields;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for recordSecurityLocks.
|
||||
*******************************************************************************/
|
||||
public List<RecordSecurityLock> getRecordSecurityLocks()
|
||||
{
|
||||
return (recordSecurityLocks);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for recordSecurityLocks.
|
||||
*******************************************************************************/
|
||||
public EmailQBitConfig withRecordSecurityLocks(List<RecordSecurityLock> recordSecurityLocks)
|
||||
{
|
||||
this.recordSecurityLocks = recordSecurityLocks;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for tableMetaDataCustomizer.
|
||||
*******************************************************************************/
|
||||
public MetaDataCustomizerInterface<QTableMetaData> getTableMetaDataCustomizer()
|
||||
{
|
||||
return (tableMetaDataCustomizer);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for tableMetaDataCustomizer.
|
||||
*******************************************************************************/
|
||||
public EmailQBitConfig withTableMetaDataCustomizer(MetaDataCustomizerInterface<QTableMetaData> tableMetaDataCustomizer)
|
||||
{
|
||||
this.tableMetaDataCustomizer = tableMetaDataCustomizer;
|
||||
return (this);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Activation-time compiler, optional unless MJML templates are used.
|
||||
*******************************************************************************/
|
||||
public com.kingsrook.qbits.email.template.MjmlCompilerInterface getMjmlCompiler()
|
||||
{
|
||||
return (mjmlCompiler);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Configure a compiler without introducing a runtime dependency.
|
||||
*******************************************************************************/
|
||||
public EmailQBitConfig withMjmlCompiler(com.kingsrook.qbits.email.template.MjmlCompilerInterface compiler)
|
||||
{
|
||||
mjmlCompiler = compiler;
|
||||
return (this);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Field classifications used until core supplies classification metadata.
|
||||
*******************************************************************************/
|
||||
public java.util.Map<String, com.kingsrook.qbits.email.phi.EmailClassifications.Level> getClassificationOverrides()
|
||||
{
|
||||
return (classificationOverrides);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Configure explicit table.field classifications.
|
||||
*******************************************************************************/
|
||||
public EmailQBitConfig withClassificationOverrides(java.util.Map<String, com.kingsrook.qbits.email.phi.EmailClassifications.Level> overrides)
|
||||
{
|
||||
classificationOverrides = java.util.Map.copyOf(overrides);
|
||||
return (this);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Additional HTTPS image hosts permitted for approved email content.
|
||||
*******************************************************************************/
|
||||
public Set<String> getAllowedImageHosts()
|
||||
{
|
||||
return (allowedImageHosts);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Configure exact host names, without wildcards or URL paths.
|
||||
*******************************************************************************/
|
||||
public EmailQBitConfig withAllowedImageHosts(Set<String> hosts)
|
||||
{
|
||||
allowedImageHosts = hosts.stream().map(host -> host.toLowerCase(java.util.Locale.ROOT)).collect(java.util.stream.Collectors.toUnmodifiableSet());
|
||||
return (this);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Fresh transports for every dispatcher, including process actions.
|
||||
*******************************************************************************/
|
||||
public com.kingsrook.qbits.email.transport.EmailTransportRegistry newTransportRegistry()
|
||||
{
|
||||
return (transportRegistryFactory.get());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Host-scoped transport injection for embedded applications and local demos.
|
||||
*******************************************************************************/
|
||||
public EmailQBitConfig withTransportRegistryFactory(java.util.function.Supplier<com.kingsrook.qbits.email.transport.EmailTransportRegistry> factory)
|
||||
{
|
||||
transportRegistryFactory = java.util.Objects.requireNonNull(factory);
|
||||
return (this);
|
||||
}
|
||||
|
||||
}
|
||||
@@ -22,17 +22,30 @@ package com.kingsrook.qbits.email;
|
||||
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import com.kingsrook.qbits.email.app.EmailApp;
|
||||
import com.kingsrook.qbits.email.bridge.EmailOutboxMessagingProviderMetaData;
|
||||
import com.kingsrook.qbits.email.delivery.EmailAlerts;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTableCustomizers;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTables;
|
||||
import com.kingsrook.qbits.email.processes.EmailProcesses;
|
||||
import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizers;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerMultiOutput;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.fields.QVirtualFieldMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitMetaDataProducer;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.tables.QFieldSectionAlternativeType;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Entry point of the email qbit (scaffold: the tables, providers and app
|
||||
** arrive in phase (a), see docs/DESIGN.md).
|
||||
** Entry point of the email outbox qbit.
|
||||
*******************************************************************************/
|
||||
public class EmailQBitProducer implements QBitMetaDataProducer<EmailQBitConfig>
|
||||
{
|
||||
@@ -112,8 +125,7 @@ public class EmailQBitProducer implements QBitMetaDataProducer<EmailQBitConfig>
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Validate the config and register the qbit (scaffold: the email_ tables,
|
||||
** providers, processes and app arrive in phase (a), docs/DESIGN.md).
|
||||
** Validate and register the tables, bridge, processes, alerts and app.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public MetaDataProducerMultiOutput produce(QInstance qInstance) throws QException
|
||||
@@ -125,6 +137,64 @@ public class EmailQBitProducer implements QBitMetaDataProducer<EmailQBitConfig>
|
||||
throw (new QException("Email qbit configuration is invalid: " + String.join("; ", errors)));
|
||||
}
|
||||
qInstance.addQBit(getQBitMetaData());
|
||||
MetaDataProducerMultiOutput output = new MetaDataProducerMultiOutput();
|
||||
EmailTables.possibleValueSources().forEach(output::add);
|
||||
Map<String, QTableMetaData> tables = new LinkedHashMap<>();
|
||||
for(QTableMetaData table : EmailTables.tables(qBitConfig))
|
||||
{
|
||||
Class<?> customizer = EmailTables.PROVIDER.equals(table.getName()) ? EmailTableCustomizers.ProviderCustomizer.class
|
||||
: EmailTables.SUPPRESSION.equals(table.getName()) ? EmailTableCustomizers.SuppressionCustomizer.class
|
||||
: List.of(EmailTables.TEMPLATE, EmailTables.VERSION, EmailTables.BRAND).contains(table.getName()) ? com.kingsrook.qbits.email.template.EmailTemplateCustomizers.class : null;
|
||||
if(customizer != null)
|
||||
{
|
||||
table.withCustomizer(TableCustomizers.PRE_INSERT_RECORD, new QCodeReference(customizer))
|
||||
.withCustomizer(TableCustomizers.PRE_UPDATE_RECORD, new QCodeReference(customizer))
|
||||
.withCustomizer(TableCustomizers.PRE_DELETE_RECORD, new QCodeReference(customizer));
|
||||
}
|
||||
if(EmailTables.MESSAGE.equals(table.getName()))
|
||||
{
|
||||
table.withCustomizer(TableCustomizers.POST_QUERY_RECORD, new QCodeReference(EmailTableCustomizers.MessageLinksCustomizer.class));
|
||||
}
|
||||
if(EmailTables.VERSION.equals(table.getName()))
|
||||
{
|
||||
table.withCustomizer(TableCustomizers.POST_QUERY_RECORD, new QCodeReference(com.kingsrook.qbits.email.template.EmailVersionLabels.class));
|
||||
}
|
||||
if(EmailTables.PROVIDER.equals(table.getName()))
|
||||
{
|
||||
table.withCustomizer(TableCustomizers.POST_QUERY_RECORD, new QCodeReference(EmailTableCustomizers.ProviderCustomizer.class));
|
||||
for(String field : List.of("usernameEnvName", "secretEnvName", "webhookSecretEnvName"))
|
||||
{
|
||||
table.withVirtualField(new QVirtualFieldMetaData(field + "Status", QFieldType.STRING)
|
||||
.withLabel(field.equals("usernameEnvName") ? "Username set" : field.equals("secretEnvName") ? "Secret set" : "Webhook secret set")
|
||||
.withIsEditable(false).withIsQuerySelectable(true));
|
||||
}
|
||||
table.getSections().get(1).withAlternative(QFieldSectionAlternativeType.RECORD_VIEW, section ->
|
||||
{
|
||||
List<String> fields = new ArrayList<>(section.getFieldNames());
|
||||
fields.addAll(List.of("usernameEnvNameStatus", "secretEnvNameStatus", "webhookSecretEnvNameStatus"));
|
||||
section.setFieldNames(fields);
|
||||
});
|
||||
}
|
||||
if(qBitConfig.getTableMetaDataCustomizer() != null)
|
||||
{
|
||||
table = qBitConfig.getTableMetaDataCustomizer().customizeMetaData(qInstance, table);
|
||||
}
|
||||
table.setSourceQBitName(getQBitMetaData().getName());
|
||||
tables.put(table.getName(), table);
|
||||
output.add(table);
|
||||
}
|
||||
EmailTables.joins().forEach(output::add);
|
||||
EmailTables.childWidgets().forEach(output::add);
|
||||
EmailApp.widgets().forEach(output::add);
|
||||
EmailProcesses.processes(qBitConfig).forEach(process ->
|
||||
{
|
||||
process.setSourceQBitName(getQBitMetaData().getName());
|
||||
output.add(process);
|
||||
});
|
||||
EmailAlerts.types().forEach(output::add);
|
||||
output.add(EmailApp.app(qBitConfig, tables));
|
||||
output.add(new EmailOutboxMessagingProviderMetaData().withName(qBitConfig.getBridgeProviderName()));
|
||||
output.addSelfToInstance(qInstance);
|
||||
return (new MetaDataProducerMultiOutput());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,126 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email;
|
||||
|
||||
|
||||
import java.lang.management.ManagementFactory;
|
||||
import java.time.Clock;
|
||||
import java.util.UUID;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitMetaData;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Runtime lookups for the qbit.
|
||||
**
|
||||
** <p>The config is read from the current QInstance on every call (never held
|
||||
** in a static), so reconfiguring, or two instances in one JVM, never see
|
||||
** stale settings. The only JVM-wide state is the runner identity
|
||||
** and the clock, which tests replace.</p>
|
||||
*******************************************************************************/
|
||||
public final class EmailRuntime
|
||||
{
|
||||
|
||||
private static final String NODE_ID = ManagementFactory.getRuntimeMXBean().getName() + "/" + UUID.randomUUID().toString().substring(0, 8);
|
||||
|
||||
private static volatile Clock clock = Clock.systemUTC();
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private EmailRuntime()
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The config in the current instance, or null when the qbit is not installed.
|
||||
*******************************************************************************/
|
||||
public static EmailQBitConfig config()
|
||||
{
|
||||
return (config(QContext.getQInstance()));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The config in an instance, or null when the qbit is not installed.
|
||||
*******************************************************************************/
|
||||
public static EmailQBitConfig config(QInstance qInstance)
|
||||
{
|
||||
if(qInstance == null || qInstance.getQBits() == null)
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
for(QBitMetaData qBit : qInstance.getQBits().values())
|
||||
{
|
||||
if(EmailQBitProducer.ARTIFACT_ID.equals(qBit.getArtifactId()) && qBit.getConfig() instanceof EmailQBitConfig config)
|
||||
{
|
||||
return (config);
|
||||
}
|
||||
}
|
||||
return (null);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** True when the qbit is installed in the current instance.
|
||||
*******************************************************************************/
|
||||
public static boolean isInstalled()
|
||||
{
|
||||
return (config() != null);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The clock (tests replace it).
|
||||
*******************************************************************************/
|
||||
public static Clock clock()
|
||||
{
|
||||
return (clock);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Replace the clock (tests).
|
||||
*******************************************************************************/
|
||||
public static void setClock(Clock newClock)
|
||||
{
|
||||
clock = newClock == null ? Clock.systemUTC() : newClock;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Runner identity stored on claimed attempts.
|
||||
*******************************************************************************/
|
||||
public static String nodeId()
|
||||
{
|
||||
return (NODE_ID);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,121 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.app;
|
||||
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import com.kingsrook.qbits.email.EmailQBitConfig;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTables;
|
||||
import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
|
||||
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Email app with viewer-scoped dashboard widgets and the five main tables in navigation.
|
||||
** Versions and attempts are child lists, and processes are record actions.
|
||||
*******************************************************************************/
|
||||
public final class EmailApp
|
||||
{
|
||||
public static final String HOME_WIDGET = "emailHome";
|
||||
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private EmailApp()
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Standard home and record preview widgets.
|
||||
*******************************************************************************/
|
||||
public static List<QWidgetMetaData> widgets()
|
||||
{
|
||||
return (List.of(
|
||||
widget(HOME_WIDGET, "Overview", WidgetType.HTML.getType(), HomeRenderer.class),
|
||||
widget("emailTemplatePreview", "Preview", WidgetType.HTML.getType(), com.kingsrook.qbits.email.template.EmailPreviewWidget.class)));
|
||||
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The app, with the main tables as children.
|
||||
*******************************************************************************/
|
||||
public static QAppMetaData app(EmailQBitConfig config, Map<String, QTableMetaData> tables)
|
||||
{
|
||||
QAppMetaData app = new QAppMetaData()
|
||||
.withName(config.getAppName())
|
||||
.withLabel(config.getAppLabel())
|
||||
.withIcon(new QIcon("mail"))
|
||||
.withPermissionRules(EmailTables.permissionRules())
|
||||
.withWidgets(List.of(HOME_WIDGET));
|
||||
for(String name : EmailTables.MAIN)
|
||||
{
|
||||
if(tables.containsKey(name))
|
||||
{
|
||||
app.withChild(tables.get(name));
|
||||
}
|
||||
}
|
||||
return (app);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Home renderer.
|
||||
*******************************************************************************/
|
||||
public static class HomeRenderer extends AbstractWidgetRenderer
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Render.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public RenderWidgetOutput render(RenderWidgetInput input) throws QException
|
||||
{
|
||||
return (new RenderWidgetOutput(EmailHome.overview()));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Full-width widgets follow the Alerts home pattern.
|
||||
*******************************************************************************/
|
||||
private static QWidgetMetaData widget(String name, String label, String type, Class<?> renderer)
|
||||
{
|
||||
return (new QWidgetMetaData().withName(name).withLabel(label).withType(type).withGridColumns(12).withIsCard(false)
|
||||
.withShowReloadButton(true).withCodeReference(new QCodeReference(renderer)).withPermissionRules(EmailTables.permissionRules()));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,201 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.app;
|
||||
|
||||
|
||||
import java.time.Instant;
|
||||
import java.time.ZoneOffset;
|
||||
import java.time.format.DateTimeFormatter;
|
||||
import java.time.temporal.ChronoUnit;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Locale;
|
||||
import com.kingsrook.qbits.email.EmailRuntime;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTables;
|
||||
import com.kingsrook.qbits.email.permissions.EmailPermissionChecker;
|
||||
import com.kingsrook.qbits.email.permissions.EmailPermissions;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.CountAction;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.count.CountInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
|
||||
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.ChartData;
|
||||
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.RawHTML;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Real viewer-scoped data. Counts preserve record security without sampling;
|
||||
** inline bar charts and rows use the same theme-neutral styles as Webhooks.
|
||||
*******************************************************************************/
|
||||
public final class EmailHome
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private EmailHome()
|
||||
{
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Counts and recent messages, without loading bodies or variable snapshots.
|
||||
*******************************************************************************/
|
||||
public static RawHTML overview() throws QException
|
||||
{
|
||||
Instant now = EmailRuntime.clock().instant();
|
||||
List<Integer> totals = new ArrayList<>();
|
||||
String[] fields = {"sentDate", "deliveredDate", "deadLetterDate", "bouncedDate", "complainedDate"};
|
||||
for(int i = 0; i < fields.length; i++)
|
||||
{
|
||||
totals.add(count(EmailTables.MESSAGE, since(fields[i], now.minus(i < 3 ? 1 : 7, ChronoUnit.DAYS))
|
||||
.withCriteria(fields[i], QCriteriaOperator.LESS_THAN_OR_EQUALS, now)));
|
||||
}
|
||||
totals.add(count(EmailTables.SUPPRESSION, new QQueryFilter()));
|
||||
List<QRecord> messages = query(EmailTables.MESSAGE, new QQueryFilter().withOrderBy(new QFilterOrderBy("createDate", false))
|
||||
.withOrderBy(new QFilterOrderBy("id", false)).withLimit(20), List.of("id", "createDate", "toAddresses", "subject", "phiApproved", "status"));
|
||||
boolean admin = EmailPermissionChecker.has(EmailPermissions.ADMIN);
|
||||
List<QRecord> providers = admin ? query(EmailTables.PROVIDER, new QQueryFilter().withOrderBy(new QFilterOrderBy("id", true)),
|
||||
List.of("id", "label", "name", "enabled", "healthStatus", "lastSuccessDate", "lastFailureMessage", "maxPerMinute")) : List.of();
|
||||
for(QRecord provider : providers)
|
||||
{
|
||||
Integer limit = provider.getValueInteger("maxPerMinute");
|
||||
if(limit != null)
|
||||
{
|
||||
int used = count(EmailTables.ATTEMPT, since("startDate", now.minusSeconds(60))
|
||||
.withCriteria("startDate", QCriteriaOperator.LESS_THAN_OR_EQUALS, now)
|
||||
.withCriteria("providerId", QCriteriaOperator.EQUALS, provider.getValue("id")));
|
||||
provider.setValue("headroom", Math.max(0, limit - used));
|
||||
}
|
||||
}
|
||||
return (new RawHTML(null, EmailHomeView.render(totals, hourly(), providers, admin, messages, suppressions())));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Twenty-four UTC hour buckets, counting successful and failed attempts.
|
||||
*******************************************************************************/
|
||||
public static ChartData hourly() throws QException
|
||||
{
|
||||
Instant start = EmailRuntime.clock().instant().truncatedTo(ChronoUnit.HOURS).minus(23, ChronoUnit.HOURS);
|
||||
List<String> labels = new ArrayList<>();
|
||||
List<Number> sent = new ArrayList<>();
|
||||
List<Number> failed = new ArrayList<>();
|
||||
for(int i = 0; i < 24; i++)
|
||||
{
|
||||
Instant hour = start.plus(i, ChronoUnit.HOURS);
|
||||
labels.add(DateTimeFormatter.ofPattern("HH:mm", Locale.ENGLISH).withZone(ZoneOffset.UTC).format(hour));
|
||||
QQueryFilter filter = since("endDate", hour).withCriteria("endDate", QCriteriaOperator.LESS_THAN, hour.plus(1, ChronoUnit.HOURS));
|
||||
sent.add(count(EmailTables.ATTEMPT, filter.clone().withCriteria("outcome", QCriteriaOperator.EQUALS, "SUCCEEDED")));
|
||||
failed.add(count(EmailTables.ATTEMPT, filter.clone().withCriteria("outcome", QCriteriaOperator.IN, List.of("TRANSIENT_FAILURE", "PROVIDER_FAILURE", "PERMANENT_FAILURE", "LEASE_EXPIRED"))));
|
||||
}
|
||||
return (chart(labels, List.of(dataset("Sent", sent, "#16a34a"), dataset("Failed", failed, "#dc2626")), "No completed attempts in the last 24 hours. Counts use UTC hours."));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Suppression rows created per UTC day and reason, for thirty days.
|
||||
*******************************************************************************/
|
||||
public static ChartData suppressions() throws QException
|
||||
{
|
||||
Instant start = EmailRuntime.clock().instant().truncatedTo(ChronoUnit.DAYS).minus(29, ChronoUnit.DAYS);
|
||||
List<String> labels = new ArrayList<>();
|
||||
List<ChartData.Data.Dataset> datasets = new ArrayList<>();
|
||||
List<String> reasons = List.of("HARD_BOUNCE", "COMPLAINT", "UNSUBSCRIBE", "MANUAL");
|
||||
List<String> colors = List.of("#dc2626", "#d97706", "#2563eb", "#6b7280");
|
||||
for(int r = 0; r < reasons.size(); r++)
|
||||
{
|
||||
List<Number> counts = new ArrayList<>();
|
||||
for(int i = 0; i < 30; i++)
|
||||
{
|
||||
Instant day = start.plus(i, ChronoUnit.DAYS);
|
||||
if(r == 0)
|
||||
{
|
||||
labels.add(DateTimeFormatter.ofPattern("MMM d", Locale.ENGLISH).withZone(ZoneOffset.UTC).format(day));
|
||||
}
|
||||
counts.add(count(EmailTables.SUPPRESSION, since("createdDate", day).withCriteria("createdDate", QCriteriaOperator.LESS_THAN, day.plus(1, ChronoUnit.DAYS))
|
||||
.withCriteria("reason", QCriteriaOperator.EQUALS, reasons.get(r))));
|
||||
}
|
||||
datasets.add(dataset(com.kingsrook.qbits.email.metadata.EmailEnums.SuppressionReason.valueOf(reasons.get(r)).getPossibleValueLabel(), counts, colors.get(r)));
|
||||
}
|
||||
return (chart(labels, datasets, "No new suppressions in the last 30 days. Counts use UTC dates."));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Standard chart with an honest empty-state sublabel.
|
||||
*******************************************************************************/
|
||||
private static ChartData chart(List<String> labels, List<ChartData.Data.Dataset> datasets, String empty)
|
||||
{
|
||||
ChartData chart = new ChartData().withChartData(new ChartData.Data().withLabels(labels).withDatasets(datasets));
|
||||
boolean any = datasets.stream().anyMatch(d -> d.getData().stream().anyMatch(n -> n.doubleValue() > 0));
|
||||
chart.setSublabel(any ? "Real data visible to your session; UTC buckets." : empty);
|
||||
return (chart);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** A standard chart series.
|
||||
*******************************************************************************/
|
||||
private static ChartData.Data.Dataset dataset(String label, List<Number> values, String color)
|
||||
{
|
||||
return (new ChartData.Data.Dataset().withLabel(label).withData(values).withColor(color));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Mask display names and all but the first local-part character.
|
||||
*******************************************************************************/
|
||||
public static String mask(String recipients)
|
||||
{
|
||||
try
|
||||
{
|
||||
String address = jakarta.mail.internet.InternetAddress.parse(recipients)[0].getAddress();
|
||||
int at = address.lastIndexOf('@');
|
||||
return (at > 0 ? address.charAt(0) + "***" + address.substring(at) : "[masked]");
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
return ("[masked]");
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Date lower bound, always used with viewer-scoped actions.
|
||||
*******************************************************************************/
|
||||
private static QQueryFilter since(String field, Instant since)
|
||||
{
|
||||
return (new QQueryFilter().withCriteria(field, QCriteriaOperator.GREATER_THAN_OR_EQUALS, since));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Exact count as the viewer, without authorized scopes.
|
||||
*******************************************************************************/
|
||||
private static int count(String table, QQueryFilter filter) throws QException
|
||||
{
|
||||
return (new CountAction().execute(new CountInput(table).withFilter(filter)).getCount());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Select only home fields; row and field security remain active.
|
||||
*******************************************************************************/
|
||||
private static List<QRecord> query(String table, QQueryFilter filter, List<String> fields) throws QException
|
||||
{
|
||||
return (new QueryAction().execute(new QueryInput(table).withFilter(filter).withFieldNamesToInclude(new java.util.HashSet<>(fields)).withShouldFetchHeavyFields(true)).getRecords());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,243 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.app;
|
||||
|
||||
|
||||
import java.time.Instant;
|
||||
import java.time.ZoneOffset;
|
||||
import java.time.format.DateTimeFormatter;
|
||||
import java.util.List;
|
||||
import java.util.Locale;
|
||||
import com.kingsrook.qbits.email.metadata.EmailEnums;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTables;
|
||||
import com.kingsrook.qbits.email.template.EmailTemplateEngine;
|
||||
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.ChartData;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Inline styles survive the UI sanitizer. Grids collapse into labelled cards;
|
||||
** bars shrink with their container and all ordinary text inherits the theme.
|
||||
*******************************************************************************/
|
||||
public final class EmailHomeView
|
||||
{
|
||||
private static final String WRAP = "min-width:0;max-width:100%;overflow-wrap:anywhere;word-break:break-word;box-sizing:border-box;";
|
||||
private static final String CARD = WRAP + "padding:16px;border:1px solid rgba(127,127,127,.3);border-radius:8px;";
|
||||
private static final String ROW = WRAP + "display:grid;grid-template-columns:repeat(auto-fit,minmax(min(100%,12rem),1fr));gap:6px 16px;padding:10px 0;border-top:1px solid rgba(127,127,127,.2);";
|
||||
private static final String FAIL = "#d64545";
|
||||
private static final String OK = "#2e9e5b";
|
||||
private static final DateTimeFormatter WHEN = DateTimeFormatter.ofPattern("MMM d, HH:mm 'UTC'", Locale.ENGLISH).withZone(ZoneOffset.UTC);
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private EmailHomeView()
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** One ordered home with real, viewer-scoped data supplied by EmailHome.
|
||||
*******************************************************************************/
|
||||
public static String render(List<Integer> totals, ChartData hourly, List<QRecord> providers, boolean admin, List<QRecord> messages, ChartData suppressions)
|
||||
{
|
||||
StringBuilder html = new StringBuilder("<div style=\"" + WRAP + "display:flex;flex-direction:column;gap:16px;color:inherit;line-height:1.4\">");
|
||||
html.append("<div style=\"" + WRAP + "display:grid;grid-template-columns:repeat(auto-fit,minmax(min(100%,9rem),1fr));gap:12px\">");
|
||||
List<String> labels = List.of("Sent (24 h)", "Delivered (24 h)", "Dead letters (24 h)", "Bounced (7 d)", "Complaints (7 d)", "Suppressed addresses");
|
||||
for(int i = 0; i < labels.size(); i++)
|
||||
{
|
||||
html.append("<div style=\"").append(CARD).append("padding:12px\"><div style=\"font-size:1.6rem;font-weight:600;line-height:1.2;")
|
||||
.append(i >= 2 && i <= 4 && totals.get(i) > 0 ? "color:" + FAIL : "color:inherit").append("\">")
|
||||
.append(String.format(Locale.ENGLISH, "%,d", totals.get(i))).append("</div><div style=\"font-size:.85rem;opacity:.75\">")
|
||||
.append(labels.get(i)).append("</div></div>");
|
||||
}
|
||||
html.append("</div>");
|
||||
bars(html, "Sends per hour, last 24 hours", hourly);
|
||||
providers(html, providers, admin);
|
||||
messages(html, messages);
|
||||
bars(html, "Suppressions added, last 30 days", suppressions);
|
||||
return (html.append("</div>").toString());
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A bounded stacked bar chart, including exact values in each bar's title.
|
||||
*******************************************************************************/
|
||||
private static void bars(StringBuilder html, String title, ChartData chart)
|
||||
{
|
||||
section(html, title);
|
||||
var data = chart.getChartData();
|
||||
int max = 0;
|
||||
for(int i = 0; i < data.getLabels().size(); i++)
|
||||
{
|
||||
int total = 0;
|
||||
for(var dataset : data.getDatasets())
|
||||
{
|
||||
total += dataset.getData().get(i).intValue();
|
||||
}
|
||||
max = Math.max(max, total);
|
||||
}
|
||||
if(max == 0)
|
||||
{
|
||||
html.append("<p style=\"margin:0;opacity:.75\">").append(esc(chart.getSublabel())).append("</p></section>");
|
||||
return;
|
||||
}
|
||||
html.append("<div style=\"font-size:.75rem;opacity:.75\">Peak ").append(max).append("</div>")
|
||||
.append("<div role=\"img\" aria-label=\"").append(esc(title)).append("; stacked counts, UTC\" style=\"")
|
||||
.append(WRAP).append("display:flex;align-items:flex-end;gap:2px;height:140px;width:100%\">");
|
||||
for(int i = 0; i < data.getLabels().size(); i++)
|
||||
{
|
||||
StringBuilder description = new StringBuilder(data.getLabels().get(i));
|
||||
for(var dataset : data.getDatasets())
|
||||
{
|
||||
description.append("; ").append(dataset.getLabel()).append(": ").append(dataset.getData().get(i));
|
||||
}
|
||||
html.append("<div title=\"").append(esc(description.toString())).append("\" style=\"flex:1 1 0;min-width:0;height:100%;display:flex;flex-direction:column-reverse\">");
|
||||
for(var dataset : data.getDatasets())
|
||||
{
|
||||
html.append("<div style=\"background:").append(dataset.getColor()).append(";height:")
|
||||
.append(String.format(Locale.ROOT, "%.3f", 100.0 * dataset.getData().get(i).doubleValue() / max)).append("%\"></div>");
|
||||
}
|
||||
html.append("</div>");
|
||||
}
|
||||
html.append("</div><div style=\"display:flex;justify-content:space-between;gap:8px;min-width:0;font-size:.75rem;opacity:.75;margin-top:4px\">");
|
||||
for(int i : new int[] {0, data.getLabels().size() / 2, data.getLabels().size() - 1})
|
||||
{
|
||||
html.append("<span>").append(esc(data.getLabels().get(i))).append("</span>");
|
||||
}
|
||||
html.append("</div><div style=\"display:flex;flex-wrap:wrap;gap:8px 16px;min-width:0;font-size:.8rem;margin-top:8px\">");
|
||||
for(var dataset : data.getDatasets())
|
||||
{
|
||||
html.append("<span><span style=\"display:inline-block;width:.7rem;height:.7rem;margin-right:5px;background:")
|
||||
.append(dataset.getColor()).append("\"></span>").append(esc(dataset.getLabel())).append("</span>");
|
||||
}
|
||||
html.append("<span style=\"opacity:.75\">UTC</span></div></section>");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Provider health is admin-only; disabled providers never appear healthy.
|
||||
*******************************************************************************/
|
||||
private static void providers(StringBuilder html, List<QRecord> providers, boolean admin)
|
||||
{
|
||||
section(html, "Provider health");
|
||||
if(!admin || providers.isEmpty())
|
||||
{
|
||||
html.append("<p style=\"margin:0;opacity:.75\">").append(admin ? "No providers configured." : "Provider health needs email.admin.").append("</p>");
|
||||
}
|
||||
for(QRecord provider : providers)
|
||||
{
|
||||
String status = Boolean.FALSE.equals(provider.getValueBoolean("enabled")) ? "Disabled"
|
||||
: "HEALTHY".equals(provider.getValueString("healthStatus")) ? "Healthy" : "UNHEALTHY".equals(provider.getValueString("healthStatus")) ? "Unhealthy" : "Unknown";
|
||||
String color = "Healthy".equals(status) ? OK : "Unhealthy".equals(status) ? FAIL : "#808080";
|
||||
html.append("<div style=\"").append(ROW).append("\">")
|
||||
.append(cell("Provider", "<strong>" + esc(value(provider, "label", value(provider, "name", "Provider"))) + "</strong><br><span style=\"color:" + color + "\">" + status + "</span>"))
|
||||
.append(cell("Last success", esc(when(provider.getValueInstant("lastSuccessDate")))))
|
||||
.append(cell("Last failure", esc(value(provider, "lastFailureMessage", "None"))))
|
||||
.append(cell("Per-minute headroom", esc(value(provider, "headroom", "No limit")))).append("</div>");
|
||||
}
|
||||
html.append("</section>");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The whole row is a link. Labelled columns stack as cards on narrow screens.
|
||||
*******************************************************************************/
|
||||
private static void messages(StringBuilder html, List<QRecord> messages)
|
||||
{
|
||||
section(html, "Recent messages");
|
||||
if(messages.isEmpty())
|
||||
{
|
||||
html.append("<p style=\"margin:0;opacity:.75\">No messages yet.</p>");
|
||||
}
|
||||
for(QRecord message : messages)
|
||||
{
|
||||
String subject = Boolean.TRUE.equals(message.getValueBoolean("phiApproved")) || "[encrypted]".equals(message.getValueString("subject"))
|
||||
? "Secure message" : value(message, "subject", "Message " + message.getValueInteger("id"));
|
||||
String status = message.getValueString("status");
|
||||
html.append("<a href=\"/app/").append(EmailTables.MESSAGE).append('/').append(message.getValueInteger("id"))
|
||||
.append("\" style=\"").append(ROW).append("color:inherit;text-decoration:none\">")
|
||||
.append(cell("When", esc(when(message.getValueInstant("createDate")))))
|
||||
.append(cell("To", esc(EmailHome.mask(message.getValueString("toAddresses")))))
|
||||
.append(cell("Subject", esc(subject)))
|
||||
.append(cell("Status", esc(status == null ? "Unknown" : EmailEnums.MessageStatus.valueOf(status).getPossibleValueLabel())))
|
||||
.append("</a>");
|
||||
}
|
||||
html.append("</section>");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** One bordered section, using the surrounding theme's text colour.
|
||||
*******************************************************************************/
|
||||
private static void section(StringBuilder html, String title)
|
||||
{
|
||||
html.append("<section style=\"").append(CARD).append("\"><h3 style=\"margin:0 0 12px;font-size:1rem;font-weight:600\">").append(esc(title)).append("</h3>");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A labelled, shrinkable cell that remains understandable when stacked.
|
||||
*******************************************************************************/
|
||||
private static String cell(String label, String content)
|
||||
{
|
||||
return ("<div style=\"" + WRAP + "\"><span style=\"display:block;font-size:.75rem;opacity:.65\">" + esc(label) + "</span> " + content + "</div>");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Compact UTC dates, with an honest missing value.
|
||||
*******************************************************************************/
|
||||
private static String when(Instant instant)
|
||||
{
|
||||
return (instant == null ? "None" : WHEN.format(instant));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Blank fields use a human-readable fallback.
|
||||
*******************************************************************************/
|
||||
private static String value(QRecord row, String field, String fallback)
|
||||
{
|
||||
String value = row.getValueString(field);
|
||||
return (value == null || value.isBlank() ? fallback : value);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Escape every record-sourced value before inserting HTML.
|
||||
*******************************************************************************/
|
||||
private static String esc(String value)
|
||||
{
|
||||
return (EmailTemplateEngine.escape(value));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,135 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.bridge;
|
||||
|
||||
|
||||
import com.kingsrook.qbits.email.outbox.EmailRequest;
|
||||
import com.kingsrook.qbits.email.outbox.EmailSendResult;
|
||||
import com.kingsrook.qbits.email.outbox.EmailService;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.messaging.MultiParty;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.messaging.Party;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.messaging.SendMessageInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.messaging.SendMessageOutput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.messaging.email.EmailContentRole;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.messaging.email.EmailPartyRole;
|
||||
import com.kingsrook.qqq.backend.core.modules.messaging.MessagingProviderInterface;
|
||||
import jakarta.mail.internet.InternetAddress;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Adapts core messaging calls to durable asynchronous outbox inserts.
|
||||
*******************************************************************************/
|
||||
public class EmailOutboxMessagingProvider implements MessagingProviderInterface
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Distinct provider type.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getType()
|
||||
{
|
||||
return (EmailOutboxMessagingProviderMetaData.TYPE);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Map core parties and alternatives; refuse unsupported attachments.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public SendMessageOutput sendMessage(SendMessageInput input) throws QException
|
||||
{
|
||||
if(input.getAttachmentList() != null && !input.getAttachmentList().isEmpty())
|
||||
{
|
||||
throw (new QException("Email attachments are not supported in part 1"));
|
||||
}
|
||||
EmailRequest request = input instanceof EmailTemplatedMessageInput templated && templated.getEmailRequest() != null
|
||||
? templated.getEmailRequest() : new EmailRequest().withSubject(input.getSubject());
|
||||
parties(request, input.getTo(), EmailPartyRole.TO);
|
||||
parties(request, input.getFrom(), EmailPartyRole.FROM);
|
||||
if(input.getContentList() != null)
|
||||
{
|
||||
for(var content : input.getContentList())
|
||||
{
|
||||
if(content.getContentRole() == EmailContentRole.HTML)
|
||||
{
|
||||
request.withHtmlBody(content.getBody());
|
||||
}
|
||||
else if(content.getContentRole() == EmailContentRole.TEXT || content.getContentRole() == null)
|
||||
{
|
||||
request.withTextBody(content.getBody());
|
||||
}
|
||||
else
|
||||
{
|
||||
throw (new QException("Unsupported email content role"));
|
||||
}
|
||||
}
|
||||
}
|
||||
EmailSendResult result = EmailService.send(request);
|
||||
if(result.status() == EmailSendResult.Status.REFUSED)
|
||||
{
|
||||
throw (new QException(String.join("; ", result.reasons())));
|
||||
}
|
||||
return (new SendMessageOutput());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Walk nested MultiParty inputs, inheriting a parent role when omitted.
|
||||
*******************************************************************************/
|
||||
private static void parties(EmailRequest request, Party party, EmailPartyRole inherited) throws QException
|
||||
{
|
||||
if(party == null)
|
||||
{
|
||||
return;
|
||||
}
|
||||
EmailPartyRole role = party.getRole() instanceof EmailPartyRole explicit ? explicit : inherited;
|
||||
if(party instanceof MultiParty multi)
|
||||
{
|
||||
if(multi.getPartyList() != null)
|
||||
{
|
||||
for(Party child : multi.getPartyList())
|
||||
{
|
||||
parties(request, child, role);
|
||||
}
|
||||
}
|
||||
return;
|
||||
}
|
||||
String address = party.getAddress();
|
||||
try
|
||||
{
|
||||
if(party.getLabel() != null)
|
||||
{
|
||||
address = new InternetAddress(address, party.getLabel(), "UTF-8").toString();
|
||||
}
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
throw (new QException("Invalid email party"));
|
||||
}
|
||||
switch(role)
|
||||
{
|
||||
case TO -> request.withTo(address);
|
||||
case CC -> request.withCc(address);
|
||||
case BCC -> request.withBcc(address);
|
||||
case FROM -> request.withFrom(address);
|
||||
case REPLY_TO -> request.withReplyTo(address);
|
||||
default -> throw (new QException("Unsupported email party role"));
|
||||
}
|
||||
}
|
||||
}
|
||||
+47
@@ -0,0 +1,47 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.bridge;
|
||||
|
||||
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.messaging.QMessagingProviderMetaData;
|
||||
import com.kingsrook.qqq.backend.core.modules.messaging.QMessagingProviderDispatcher;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Core messaging provider type registered independently of EMAIL and SES.
|
||||
*******************************************************************************/
|
||||
public class EmailOutboxMessagingProviderMetaData extends QMessagingProviderMetaData
|
||||
{
|
||||
public static final String TYPE = "QBIT_EMAIL";
|
||||
|
||||
static
|
||||
{
|
||||
QMessagingProviderDispatcher.registerMessagingProvider(new EmailOutboxMessagingProvider());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** The bridge type.
|
||||
*******************************************************************************/
|
||||
public EmailOutboxMessagingProviderMetaData()
|
||||
{
|
||||
setType(TYPE);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,52 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.bridge;
|
||||
|
||||
|
||||
import com.kingsrook.qbits.email.outbox.EmailRequest;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.messaging.SendMessageInput;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Optional typed template payload for core SendMessageAction bridge callers.
|
||||
** Plain SendMessageInput remains an ad-hoc message for backwards compatibility.
|
||||
*******************************************************************************/
|
||||
public class EmailTemplatedMessageInput extends SendMessageInput
|
||||
{
|
||||
private EmailRequest emailRequest;
|
||||
|
||||
/*******************************************************************************
|
||||
** Set schema variables and record references for the normal validated API.
|
||||
*******************************************************************************/
|
||||
public EmailTemplatedMessageInput withEmailRequest(EmailRequest request)
|
||||
{
|
||||
emailRequest = request;
|
||||
return (this);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** The caller's template request; party fields can still come from core input.
|
||||
*******************************************************************************/
|
||||
public EmailRequest getEmailRequest()
|
||||
{
|
||||
return (emailRequest);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,76 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.delivery;
|
||||
|
||||
|
||||
import java.util.List;
|
||||
import com.kingsrook.qbits.email.EmailRuntime;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTables;
|
||||
import com.kingsrook.qqq.backend.core.alerts.AlertSeverity;
|
||||
import com.kingsrook.qqq.backend.core.alerts.AlertSignal;
|
||||
import com.kingsrook.qqq.backend.core.alerts.QAlertTypeMetaData;
|
||||
import com.kingsrook.qqq.backend.core.alerts.QAlerts;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Optional alert sink integration with content-free signals.
|
||||
*******************************************************************************/
|
||||
public final class EmailAlerts
|
||||
{
|
||||
public static final String UNHEALTHY = "email.provider.unhealthy";
|
||||
public static final String DEAD_LETTER = "email.message.dead-letter";
|
||||
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private EmailAlerts()
|
||||
{
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Declare alert types using RC.5 dotted-name syntax.
|
||||
*******************************************************************************/
|
||||
public static List<QAlertTypeMetaData> types()
|
||||
{
|
||||
return (List.of(new QAlertTypeMetaData().withName(UNHEALTHY).withLabel("Email provider unhealthy").withDefaultSeverity(AlertSeverity.ERROR)
|
||||
.withSubjectTableName(EmailTables.PROVIDER).withVisibilityPermission("email.admin").withSourceName("email"),
|
||||
new QAlertTypeMetaData().withName(DEAD_LETTER).withLabel("Email dead letters").withDefaultSeverity(AlertSeverity.WARNING)
|
||||
.withSubjectTableName(EmailTables.MESSAGE).withVisibilityPermission("email.view").withSourceName("email")));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Raise without addresses, provider diagnostics or content.
|
||||
*******************************************************************************/
|
||||
public static void raise(String type, Integer providerId)
|
||||
{
|
||||
String key = "provider:" + providerId + (DEAD_LETTER.equals(type) ? ":" + EmailRuntime.clock().instant().getEpochSecond() / 3600 : "");
|
||||
QAlerts.raise(new AlertSignal(type).withDedupKey(key).withTitle(UNHEALTHY.equals(type) ? "Email provider unhealthy" : "Email message dead-lettered")
|
||||
.withSummary("Review the Email tables for status and retry actions."));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Resolve on recovery; core guards a missing or failing sink.
|
||||
*******************************************************************************/
|
||||
public static void recovered(Integer providerId)
|
||||
{
|
||||
QAlerts.resolve(UNHEALTHY, "provider:" + providerId, "An email attempt succeeded");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,520 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.delivery;
|
||||
|
||||
|
||||
import java.time.Duration;
|
||||
import java.time.Instant;
|
||||
import java.util.ArrayList;
|
||||
import java.util.HashSet;
|
||||
import java.util.List;
|
||||
import java.util.Set;
|
||||
import java.util.UUID;
|
||||
import com.kingsrook.qbits.email.EmailQBitConfig;
|
||||
import com.kingsrook.qbits.email.EmailRuntime;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTables;
|
||||
import com.kingsrook.qbits.email.support.EmailStore;
|
||||
import com.kingsrook.qbits.email.support.SystemWrites;
|
||||
import com.kingsrook.qbits.email.transport.EmailEnvelope;
|
||||
import com.kingsrook.qbits.email.transport.EmailProviderSettings;
|
||||
import com.kingsrook.qbits.email.transport.EmailTransportException;
|
||||
import com.kingsrook.qbits.email.transport.EmailTransportRegistry;
|
||||
import com.kingsrook.qbits.email.transport.EmailTransportResult;
|
||||
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.logging.QLogger;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Durable at-least-once delivery with unique attempt claims and database leases.
|
||||
*******************************************************************************/
|
||||
public class EmailDispatcher
|
||||
{
|
||||
private static final QLogger LOG = QLogger.getLogger(EmailDispatcher.class);
|
||||
|
||||
public static final List<String> DUE = List.of("QUEUED", "RETRY_WAIT");
|
||||
public static final List<String> FINAL = List.of("SENT", "DELIVERED", "DEAD_LETTER", "BOUNCED", "COMPLAINED", "SUPPRESSED", "CANCELLED");
|
||||
private final EmailTransportRegistry transports;
|
||||
private final String runner = EmailRuntime.nodeId() + "/" + UUID.randomUUID();
|
||||
|
||||
/*******************************************************************************
|
||||
** Default delivery transports.
|
||||
*******************************************************************************/
|
||||
public EmailDispatcher()
|
||||
{
|
||||
this(EmailRuntime.config() == null ? new EmailTransportRegistry() : EmailRuntime.config().newTransportRegistry());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Inject transport registry for host extensions and tests.
|
||||
*******************************************************************************/
|
||||
public EmailDispatcher(EmailTransportRegistry transports)
|
||||
{
|
||||
this.transports = transports;
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** One message delivery result.
|
||||
*******************************************************************************/
|
||||
public enum SendResult
|
||||
{
|
||||
SENT,
|
||||
RETRY_WAIT,
|
||||
DEAD_LETTER,
|
||||
PROVIDER_FAILURE,
|
||||
NOT_SENT
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Counts from a dispatch pass.
|
||||
*******************************************************************************/
|
||||
public record RunSummary(int sent, int failed, int skipped)
|
||||
{
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** A database claim, fully committed before network I/O.
|
||||
*******************************************************************************/
|
||||
private record Claim(QRecord message, QRecord provider, QRecord attempt)
|
||||
{
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Dispatch the configured batch.
|
||||
*******************************************************************************/
|
||||
public RunSummary runOnce() throws QException
|
||||
{
|
||||
return (run(null));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Dispatch due message ids, or all due messages when null.
|
||||
*******************************************************************************/
|
||||
public RunSummary run(List<Integer> ids) throws QException
|
||||
{
|
||||
if(EmailRuntime.config() == null || (ids != null && ids.isEmpty()))
|
||||
{
|
||||
return (new RunSummary(0, 0, 0));
|
||||
}
|
||||
QQueryFilter filter = new QQueryFilter(new QFilterCriteria("status", QCriteriaOperator.IN, "QUEUED", "RETRY_WAIT", "SENDING"))
|
||||
.withCriteria(new QFilterCriteria("nextAttemptDate", QCriteriaOperator.LESS_THAN_OR_EQUALS, EmailRuntime.clock().instant()))
|
||||
.withOrderBy(new QFilterOrderBy("id")).withLimit(EmailRuntime.config().getBatchSize());
|
||||
if(ids != null)
|
||||
{
|
||||
filter.withCriteria(new QFilterCriteria("id", QCriteriaOperator.IN, new ArrayList<>(ids)));
|
||||
}
|
||||
int sent = 0;
|
||||
int failed = 0;
|
||||
int skipped = 0;
|
||||
Integer afterId = 0;
|
||||
Set<Integer> failedProviders = new HashSet<>();
|
||||
while(sent + failed < EmailRuntime.config().getBatchSize())
|
||||
{
|
||||
QQueryFilter pageFilter = filter.clone().withCriteria(new QFilterCriteria("id", QCriteriaOperator.GREATER_THAN, afterId));
|
||||
List<QRecord> page = SystemWrites.query(EmailTables.MESSAGE, pageFilter);
|
||||
if(page.isEmpty())
|
||||
{
|
||||
break;
|
||||
}
|
||||
for(QRecord row : page)
|
||||
{
|
||||
afterId = row.getValueInteger("id");
|
||||
if(failedProviders.contains(row.getValueInteger("providerId")))
|
||||
{
|
||||
skipped++;
|
||||
continue;
|
||||
}
|
||||
try
|
||||
{
|
||||
SendResult result = attempt(afterId, false);
|
||||
switch(result)
|
||||
{
|
||||
case SENT -> sent++;
|
||||
case RETRY_WAIT, DEAD_LETTER -> failed++;
|
||||
case PROVIDER_FAILURE ->
|
||||
{
|
||||
failed++;
|
||||
failedProviders.add(row.getValueInteger("providerId"));
|
||||
}
|
||||
default -> skipped++;
|
||||
}
|
||||
}
|
||||
catch(QException | RuntimeException e)
|
||||
{
|
||||
failed++;
|
||||
LOG.warn("Email attempt failed", logPair("messageId", afterId));
|
||||
}
|
||||
if(sent + failed >= EmailRuntime.config().getBatchSize())
|
||||
{
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
return (new RunSummary(sent, failed, skipped));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Manual send bypasses due time but preserves rate limits, health and leases.
|
||||
*******************************************************************************/
|
||||
public SendResult sendNow(Integer id) throws QException
|
||||
{
|
||||
SendResult result = attempt(id, true);
|
||||
return (result == SendResult.PROVIDER_FAILURE ? SendResult.RETRY_WAIT : result);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Claim, send outside the transaction, then persist the fenced result.
|
||||
*******************************************************************************/
|
||||
protected SendResult attempt(Integer id, boolean now) throws QException
|
||||
{
|
||||
Claim claim = claim(id, now);
|
||||
if(claim == null)
|
||||
{
|
||||
return (SendResult.NOT_SENT);
|
||||
}
|
||||
EmailTransportResult result = null;
|
||||
EmailTransportException error = null;
|
||||
try
|
||||
{
|
||||
if(Boolean.TRUE.equals(claim.message().getValueBoolean("phiApproved")))
|
||||
{
|
||||
com.kingsrook.qbits.email.template.EmailTemplateRenderer.checkProvider(claim.provider());
|
||||
}
|
||||
result = transports.get(claim.provider().getValueString("kind")).send(envelope(claim.message()), new EmailProviderSettings(claim.provider(), EmailRuntime.config()));
|
||||
}
|
||||
catch(EmailTransportException e)
|
||||
{
|
||||
error = e;
|
||||
}
|
||||
catch(RuntimeException e)
|
||||
{
|
||||
error = new EmailTransportException("Transport temporarily unavailable", false);
|
||||
}
|
||||
return (finish(claim, result, error));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Unique attempt insertion under a short provider lock, including rate checks.
|
||||
*******************************************************************************/
|
||||
private Claim claim(Integer id, boolean immediate) throws QException
|
||||
{
|
||||
EmailQBitConfig config = EmailRuntime.config();
|
||||
QRecord initial = EmailStore.get(EmailTables.MESSAGE, id, null);
|
||||
if(initial == null || config == null)
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
try(QBackendTransaction tx = QBackendTransaction.openFor(new UpdateInput(EmailTables.PROVIDER)))
|
||||
{
|
||||
QRecord provider = EmailStore.lockProvider(initial.getValueInteger("providerId"), tx);
|
||||
QRecord message = EmailStore.get(EmailTables.MESSAGE, id, tx);
|
||||
Instant time = EmailRuntime.clock().instant();
|
||||
if(provider == null || message == null || !Boolean.TRUE.equals(provider.getValueBoolean("enabled")))
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
String status = message.getValueString("status");
|
||||
int previous = message.getValueInteger("attemptCount") == null ? 0 : message.getValueInteger("attemptCount");
|
||||
if("SENDING".equals(status))
|
||||
{
|
||||
Instant lease = message.getValueInstant("leaseExpiresDate");
|
||||
if(lease != null && lease.isAfter(time))
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
expire(message, tx, time);
|
||||
if(budgetAttempts(id, previous, tx) >= config.getMaxAttempts())
|
||||
{
|
||||
EmailStore.update(EmailTables.MESSAGE, new QRecord().withValue("id", id).withValue("status", "DEAD_LETTER")
|
||||
.withValue("deadLetterDate", time).withValue("finishedDate", time).withValue("leaseExpiresDate", null).withValue("nextAttemptDate", null).withValue("lastError", "Attempt lease expired"), tx);
|
||||
tx.commit();
|
||||
alert(() -> EmailAlerts.raise(EmailAlerts.DEAD_LETTER, provider.getValueInteger("id")));
|
||||
return (null);
|
||||
}
|
||||
}
|
||||
else if(!DUE.contains(status))
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
if(!immediate && ((message.getValueInstant("sendAfter") != null && message.getValueInstant("sendAfter").isAfter(time))
|
||||
|| (message.getValueInstant("nextAttemptDate") != null && message.getValueInstant("nextAttemptDate").isAfter(time))))
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
if(limited(provider, tx, time))
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
Instant lease = time.plusSeconds(config.getLeaseSeconds());
|
||||
QRecord attempt = new QRecord().withValue("messageId", id).withValue("providerId", provider.getValue("id"))
|
||||
.withValue("attemptNumber", previous + 1).withValue("outcome", "IN_FLIGHT").withValue("claimedBy", runner)
|
||||
.withValue("startDate", time).withValue("leaseExpiresDate", lease);
|
||||
for(var field : config.getSecurityFields())
|
||||
{
|
||||
attempt.setValue(field.getName(), message.getValue(field.getName()));
|
||||
}
|
||||
QRecord inserted = SystemWrites.insert(EmailTables.ATTEMPT, List.of(attempt), tx).getRecords().get(0);
|
||||
if(inserted.getValueInteger("id") == null || (inserted.getErrors() != null && !inserted.getErrors().isEmpty()))
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
EmailStore.update(EmailTables.MESSAGE, new QRecord().withValue("id", id).withValue("status", "SENDING").withValue("attemptCount", previous + 1)
|
||||
.withValue("leaseExpiresDate", lease).withValue("nextAttemptDate", lease), tx);
|
||||
tx.commit();
|
||||
return (new Claim(message, provider, inserted));
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Shared rate and unhealthy-provider probe budgets, counted from attempts.
|
||||
*******************************************************************************/
|
||||
private static boolean limited(QRecord provider, QBackendTransaction transaction, Instant now) throws QException
|
||||
{
|
||||
for(String field : List.of("maxPerMinute", "maxPerDay"))
|
||||
{
|
||||
Integer limit = provider.getValueInteger(field);
|
||||
Instant after = now.minusSeconds("maxPerMinute".equals(field) ? 60 : 86400);
|
||||
if(limit != null && SystemWrites.count(EmailTables.ATTEMPT, EmailStore.equal("providerId", provider.getValueInteger("id"))
|
||||
.withCriteria(new QFilterCriteria("startDate", QCriteriaOperator.GREATER_THAN, after)), transaction) >= limit)
|
||||
{
|
||||
return (true);
|
||||
}
|
||||
}
|
||||
if("UNHEALTHY".equals(provider.getValueString("healthStatus")))
|
||||
{
|
||||
return (SystemWrites.count(EmailTables.ATTEMPT, EmailStore.equal("providerId", provider.getValueInteger("id"))
|
||||
.withCriteria(new QFilterCriteria("startDate", QCriteriaOperator.GREATER_THAN, now.minusSeconds(300))), transaction) > 0);
|
||||
}
|
||||
return (false);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Mark the abandoned attempt before retrying.
|
||||
*******************************************************************************/
|
||||
private static void expire(QRecord message, QBackendTransaction tx, Instant now) throws QException
|
||||
{
|
||||
for(QRecord attempt : SystemWrites.query(EmailTables.ATTEMPT, EmailStore.equal("messageId", message.getValueInteger("id"))
|
||||
.withCriteria(new QFilterCriteria("outcome", QCriteriaOperator.EQUALS, "IN_FLIGHT")), tx))
|
||||
{
|
||||
EmailStore.update(EmailTables.ATTEMPT, new QRecord().withValue("id", attempt.getValue("id")).withValue("outcome", "LEASE_EXPIRED")
|
||||
.withValue("endDate", now).withValue("error", "Attempt lease expired"), tx);
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Fence late completions by attempt number and persist outcome atomically.
|
||||
*******************************************************************************/
|
||||
private SendResult finish(Claim claim, EmailTransportResult result, EmailTransportException error) throws QException
|
||||
{
|
||||
try(QBackendTransaction tx = QBackendTransaction.openFor(new UpdateInput(EmailTables.PROVIDER)))
|
||||
{
|
||||
Integer providerId = claim.provider().getValueInteger("id");
|
||||
QRecord provider = EmailStore.lockProvider(providerId, tx);
|
||||
QRecord message = EmailStore.get(EmailTables.MESSAGE, claim.message().getValueInteger("id"), tx);
|
||||
int number = claim.attempt().getValueInteger("attemptNumber");
|
||||
if(message == null || !"SENDING".equals(message.getValueString("status")) || !Integer.valueOf(number).equals(message.getValueInteger("attemptCount")))
|
||||
{
|
||||
return (SendResult.NOT_SENT);
|
||||
}
|
||||
Instant now = EmailRuntime.clock().instant();
|
||||
boolean success = error == null && result != null;
|
||||
boolean permanent = error != null && error.getPermanent();
|
||||
boolean providerLevel = error != null && error.getProviderLevel();
|
||||
String safeError = error == null ? null : permanent ? "Provider rejected the message" : "Provider temporarily unavailable";
|
||||
if(providerLevel)
|
||||
{
|
||||
safeError = "Credentials missing or refused";
|
||||
}
|
||||
if(error != null && "Gmail API sending is specified but not built; use GMAIL_SMTP".equals(error.getMessage()))
|
||||
{
|
||||
safeError = "Gmail API sending is specified but not built; use GMAIL_SMTP";
|
||||
}
|
||||
EmailStore.update(EmailTables.ATTEMPT, new QRecord().withValue("id", claim.attempt().getValue("id"))
|
||||
.withValue("outcome", success ? "SUCCEEDED" : providerLevel ? "PROVIDER_FAILURE" : permanent ? "PERMANENT_FAILURE" : "TRANSIENT_FAILURE")
|
||||
.withValue("endDate", now).withValue("durationMillis", (int) Math.max(0, Duration.between(claim.attempt().getValueInstant("startDate"), now).toMillis()))
|
||||
.withValue("error", safeError), tx);
|
||||
QRecord update = new QRecord().withValue("id", message.getValue("id")).withValue("leaseExpiresDate", null).withValue("lastError", safeError);
|
||||
SendResult outcome;
|
||||
if(success)
|
||||
{
|
||||
outcome = SendResult.SENT;
|
||||
update.withValue("status", "SENT").withValue("sentDate", now).withValue("finishedDate", now).withValue("providerMessageId", result.providerMessageId()).withValue("nextAttemptDate", null);
|
||||
EmailStore.update(EmailTables.PROVIDER, new QRecord().withValue("id", providerId).withValue("healthStatus", "HEALTHY").withValue("lastSuccessDate", now), tx);
|
||||
}
|
||||
else
|
||||
{
|
||||
boolean dead = permanent || (!providerLevel && budgetAttempts(message.getValueInteger("id"), number, tx) >= EmailRuntime.config().getMaxAttempts());
|
||||
outcome = dead ? SendResult.DEAD_LETTER : SendResult.RETRY_WAIT;
|
||||
long seconds = 60L * EmailRuntime.config().retryMinutesAfterAttempt(number);
|
||||
if(error != null && error.getRetryAfter() != null)
|
||||
{
|
||||
seconds = Math.max(1, Math.min(3600, error.getRetryAfter().getSeconds()));
|
||||
}
|
||||
update.withValue("status", outcome.name()).withValue("deadLetterDate", dead ? now : null).withValue("finishedDate", dead ? now : null).withValue("nextAttemptDate", dead ? null : now.plusSeconds(seconds));
|
||||
QRecord health = new QRecord().withValue("id", providerId).withValue("lastFailureDate", now).withValue("lastFailureMessage", safeError);
|
||||
if(providerLevel || (!permanent && consecutiveFailures(providerId, tx) >= 10))
|
||||
{
|
||||
health.withValue("healthStatus", "UNHEALTHY");
|
||||
}
|
||||
EmailStore.update(EmailTables.PROVIDER, health, tx);
|
||||
}
|
||||
EmailStore.update(EmailTables.MESSAGE, update, tx);
|
||||
QRecord updatedProvider = EmailStore.get(EmailTables.PROVIDER, providerId, tx);
|
||||
tx.commit();
|
||||
if(success && "UNHEALTHY".equals(provider.getValueString("healthStatus")))
|
||||
{
|
||||
alert(() -> EmailAlerts.recovered(providerId));
|
||||
}
|
||||
if("UNHEALTHY".equals(updatedProvider.getValueString("healthStatus")) && !"UNHEALTHY".equals(provider.getValueString("healthStatus")))
|
||||
{
|
||||
alert(() -> EmailAlerts.raise(EmailAlerts.UNHEALTHY, providerId));
|
||||
}
|
||||
if(outcome == SendResult.DEAD_LETTER)
|
||||
{
|
||||
alert(() -> EmailAlerts.raise(EmailAlerts.DEAD_LETTER, providerId));
|
||||
}
|
||||
LOG.info("Email attempt completed", logPair("messageId", message.getValue("id")), logPair("status", outcome.name()));
|
||||
return (providerLevel ? SendResult.PROVIDER_FAILURE : outcome);
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Keep claim numbering monotonic while excluding provider failures from limits.
|
||||
*******************************************************************************/
|
||||
private static long budgetAttempts(Integer messageId, int attempts, QBackendTransaction tx) throws QException
|
||||
{
|
||||
return (attempts - SystemWrites.count(EmailTables.ATTEMPT, EmailStore.equal("messageId", messageId)
|
||||
.withCriteria(new QFilterCriteria("outcome", QCriteriaOperator.EQUALS, "PROVIDER_FAILURE")), tx));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Optional alerting must never change the result of a committed delivery.
|
||||
*******************************************************************************/
|
||||
private static void alert(Runnable notification)
|
||||
{
|
||||
try
|
||||
{
|
||||
notification.run();
|
||||
}
|
||||
catch(RuntimeException e)
|
||||
{
|
||||
LOG.warn("Email alert unavailable");
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Consecutive completed transient failures; a success or permanent rejection breaks the streak.
|
||||
*******************************************************************************/
|
||||
private static int consecutiveFailures(Integer providerId, QBackendTransaction tx) throws QException
|
||||
{
|
||||
int failures = 0;
|
||||
for(QRecord attempt : SystemWrites.query(EmailTables.ATTEMPT, EmailStore.equal("providerId", providerId)
|
||||
.withCriteria(new QFilterCriteria("outcome", QCriteriaOperator.NOT_EQUALS, "IN_FLIGHT"))
|
||||
.withOrderBy(new QFilterOrderBy("endDate", false)).withOrderBy(new QFilterOrderBy("id", false)).withLimit(10), tx))
|
||||
{
|
||||
if(!"TRANSIENT_FAILURE".equals(attempt.getValueString("outcome")))
|
||||
{
|
||||
break;
|
||||
}
|
||||
failures++;
|
||||
}
|
||||
return (failures);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** The persisted alternatives are reused verbatim on each attempt.
|
||||
*******************************************************************************/
|
||||
private static EmailEnvelope envelope(QRecord message)
|
||||
{
|
||||
return (new EmailEnvelope().withMessageId(message.getValueString("id")).withFrom(message.getValueString("fromAddress"))
|
||||
.withTo(split(message.getValueString("toAddresses"))).withCc(split(message.getValueString("ccAddresses"))).withBcc(split(message.getValueString("bccAddresses")))
|
||||
.withReplyTo(message.getValueString("replyTo")).withSubject(content(message, "subject"))
|
||||
.withHtmlBody(content(message, "htmlBody"))
|
||||
.withTextBody(content(message, "textBody"))
|
||||
.withHeaders(message.getValueString("unsubscribeUrl") == null ? java.util.Map.of() : java.util.Map.of("List-Unsubscribe", "<" + message.getValueString("unsubscribeUrl") + ">", "List-Unsubscribe-Post", "List-Unsubscribe=One-Click")));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Only policy-marked messages may invoke body decryption. A caller-supplied
|
||||
** enc:v1 string in ad-hoc content is text, never a decryption capability.
|
||||
*******************************************************************************/
|
||||
private static String content(QRecord message, String field)
|
||||
{
|
||||
String value = message.getValueString(field);
|
||||
if(!Boolean.TRUE.equals(message.getValueBoolean("phiApproved")))
|
||||
{
|
||||
return (value);
|
||||
}
|
||||
if(value != null && !value.startsWith("enc:v1:"))
|
||||
{
|
||||
throw (new IllegalArgumentException("Protected email content must be encrypted"));
|
||||
}
|
||||
return (com.kingsrook.qbits.email.phi.EmailBodyEncryption.decrypt(value, field, message.getValueString("id")));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Preserve quoted display names by treating the stored list as one RFC address list.
|
||||
*******************************************************************************/
|
||||
private static List<String> split(String value)
|
||||
{
|
||||
return (value == null || value.isBlank() ? List.of() : List.of(value));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Lease recovery and five-minute provider probes share the normal claim path.
|
||||
*******************************************************************************/
|
||||
public RunSummary maintain() throws QException
|
||||
{
|
||||
Instant now = EmailRuntime.clock().instant();
|
||||
for(QRecord candidate : SystemWrites.query(EmailTables.MESSAGE, EmailStore.equal("status", "SENDING")
|
||||
.withCriteria(new QFilterCriteria("leaseExpiresDate", QCriteriaOperator.LESS_THAN_OR_EQUALS, now))))
|
||||
{
|
||||
try(QBackendTransaction tx = QBackendTransaction.openFor(new UpdateInput(EmailTables.PROVIDER)))
|
||||
{
|
||||
EmailStore.lockProvider(candidate.getValueInteger("providerId"), tx);
|
||||
QRecord current = EmailStore.get(EmailTables.MESSAGE, candidate.getValueInteger("id"), tx);
|
||||
if(current == null || !"SENDING".equals(current.getValueString("status")) || current.getValueInstant("leaseExpiresDate").isAfter(now))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
expire(current, tx, now);
|
||||
boolean dead = budgetAttempts(current.getValueInteger("id"), current.getValueInteger("attemptCount"), tx) >= EmailRuntime.config().getMaxAttempts();
|
||||
EmailStore.update(EmailTables.MESSAGE, new QRecord().withValue("id", current.getValue("id"))
|
||||
.withValue("status", dead ? "DEAD_LETTER" : "RETRY_WAIT").withValue("nextAttemptDate", dead ? null : now)
|
||||
.withValue("leaseExpiresDate", null).withValue("lastError", "Attempt lease expired")
|
||||
.withValue("deadLetterDate", dead ? now : null).withValue("finishedDate", dead ? now : null), tx);
|
||||
tx.commit();
|
||||
if(dead)
|
||||
{
|
||||
alert(() -> EmailAlerts.raise(EmailAlerts.DEAD_LETTER, candidate.getValueInteger("providerId")));
|
||||
}
|
||||
}
|
||||
}
|
||||
return (runOnce());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,315 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.metadata;
|
||||
|
||||
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.PossibleValueEnum;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Enumerated database values and their display labels.
|
||||
*******************************************************************************/
|
||||
public final class EmailEnums
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private EmailEnums()
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** ProviderKind values.
|
||||
*******************************************************************************/
|
||||
public enum ProviderKind implements PossibleValueEnum<String>
|
||||
{
|
||||
SMTP,
|
||||
SES,
|
||||
POSTMARK,
|
||||
SENDGRID,
|
||||
MAILGUN,
|
||||
GMAIL_SMTP,
|
||||
GMAIL_API;
|
||||
|
||||
/*******************************************************************************
|
||||
** Stable database value.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getPossibleValueId()
|
||||
{
|
||||
return (name());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Plain display label.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getPossibleValueLabel()
|
||||
{
|
||||
return (name().charAt(0) + name().substring(1).toLowerCase(java.util.Locale.ROOT).replace('_', ' '));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Security values.
|
||||
*******************************************************************************/
|
||||
public enum Security implements PossibleValueEnum<String>
|
||||
{
|
||||
NONE,
|
||||
STARTTLS,
|
||||
TLS;
|
||||
|
||||
/*******************************************************************************
|
||||
** Stable database value.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getPossibleValueId()
|
||||
{
|
||||
return (name());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Plain display label.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getPossibleValueLabel()
|
||||
{
|
||||
return (name().charAt(0) + name().substring(1).toLowerCase(java.util.Locale.ROOT).replace('_', ' '));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** MessageStatus values.
|
||||
*******************************************************************************/
|
||||
public enum MessageStatus implements PossibleValueEnum<String>
|
||||
{
|
||||
QUEUED,
|
||||
SENDING,
|
||||
SENT,
|
||||
DELIVERED,
|
||||
RETRY_WAIT,
|
||||
DEAD_LETTER,
|
||||
BOUNCED,
|
||||
COMPLAINED,
|
||||
SUPPRESSED,
|
||||
CANCELLED;
|
||||
|
||||
/*******************************************************************************
|
||||
** Stable database value.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getPossibleValueId()
|
||||
{
|
||||
return (name());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Plain display label.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getPossibleValueLabel()
|
||||
{
|
||||
return (this == RETRY_WAIT ? "Waiting to retry" : name().charAt(0) + name().substring(1).toLowerCase(java.util.Locale.ROOT).replace('_', ' '));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** AttemptOutcome values.
|
||||
*******************************************************************************/
|
||||
public enum AttemptOutcome implements PossibleValueEnum<String>
|
||||
{
|
||||
IN_FLIGHT,
|
||||
SUCCEEDED,
|
||||
TRANSIENT_FAILURE,
|
||||
PROVIDER_FAILURE,
|
||||
PERMANENT_FAILURE,
|
||||
LEASE_EXPIRED;
|
||||
|
||||
/*******************************************************************************
|
||||
** Stable database value.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getPossibleValueId()
|
||||
{
|
||||
return (name());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Plain display label.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getPossibleValueLabel()
|
||||
{
|
||||
return (name().charAt(0) + name().substring(1).toLowerCase(java.util.Locale.ROOT).replace('_', ' '));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** HealthStatus values.
|
||||
*******************************************************************************/
|
||||
public enum HealthStatus implements PossibleValueEnum<String>
|
||||
{
|
||||
HEALTHY,
|
||||
UNHEALTHY;
|
||||
|
||||
/*******************************************************************************
|
||||
** Stable database value.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getPossibleValueId()
|
||||
{
|
||||
return (name());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Plain display label.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getPossibleValueLabel()
|
||||
{
|
||||
return (name().charAt(0) + name().substring(1).toLowerCase(java.util.Locale.ROOT).replace('_', ' '));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** TemplateKind values.
|
||||
*******************************************************************************/
|
||||
public enum TemplateKind implements PossibleValueEnum<String>
|
||||
{
|
||||
MARKDOWN,
|
||||
MJML,
|
||||
HTML;
|
||||
|
||||
/*******************************************************************************
|
||||
** Stable database value.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getPossibleValueId()
|
||||
{
|
||||
return (name());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Plain display label.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getPossibleValueLabel()
|
||||
{
|
||||
return (name().charAt(0) + name().substring(1).toLowerCase(java.util.Locale.ROOT).replace('_', ' '));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** VersionStatus values.
|
||||
*******************************************************************************/
|
||||
public enum VersionStatus implements PossibleValueEnum<String>
|
||||
{
|
||||
DRAFT,
|
||||
ACTIVE,
|
||||
RETIRED;
|
||||
|
||||
/*******************************************************************************
|
||||
** Stable database value.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getPossibleValueId()
|
||||
{
|
||||
return (name());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Plain display label.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getPossibleValueLabel()
|
||||
{
|
||||
return (name().charAt(0) + name().substring(1).toLowerCase(java.util.Locale.ROOT).replace('_', ' '));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** SuppressionScope values.
|
||||
*******************************************************************************/
|
||||
public enum SuppressionScope implements PossibleValueEnum<String>
|
||||
{
|
||||
ALL,
|
||||
TRANSACTIONAL,
|
||||
NOTIFICATION,
|
||||
MARKETING,
|
||||
SECURITY;
|
||||
|
||||
/*******************************************************************************
|
||||
** Stable database value.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getPossibleValueId()
|
||||
{
|
||||
return (name());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Plain display label.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getPossibleValueLabel()
|
||||
{
|
||||
return (name().charAt(0) + name().substring(1).toLowerCase(java.util.Locale.ROOT).replace('_', ' '));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** SuppressionReason values.
|
||||
*******************************************************************************/
|
||||
public enum SuppressionReason implements PossibleValueEnum<String>
|
||||
{
|
||||
HARD_BOUNCE,
|
||||
COMPLAINT,
|
||||
UNSUBSCRIBE,
|
||||
MANUAL;
|
||||
|
||||
/*******************************************************************************
|
||||
** Stable database value.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getPossibleValueId()
|
||||
{
|
||||
return (name());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Plain display label.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getPossibleValueLabel()
|
||||
{
|
||||
return (name().charAt(0) + name().substring(1).toLowerCase(java.util.Locale.ROOT).replace('_', ' '));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,384 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.metadata;
|
||||
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Locale;
|
||||
import java.util.Objects;
|
||||
import java.util.Optional;
|
||||
import com.kingsrook.qbits.email.EmailQBitConfig;
|
||||
import com.kingsrook.qbits.email.EmailRuntime;
|
||||
import com.kingsrook.qbits.email.permissions.EmailPermissionChecker;
|
||||
import com.kingsrook.qbits.email.permissions.EmailPermissions;
|
||||
import com.kingsrook.qbits.email.support.SystemWrites;
|
||||
import com.kingsrook.qbits.email.transport.EmailProviderSettings;
|
||||
import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.QueryOrGetInputInterface;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.statusmessages.BadInputStatusMessage;
|
||||
import jakarta.mail.internet.InternetAddress;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Provider and suppression validation shared by UI, API and code.
|
||||
*******************************************************************************/
|
||||
public final class EmailTableCustomizers
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private EmailTableCustomizers()
|
||||
{
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Merged values for sparse updates.
|
||||
*******************************************************************************/
|
||||
private static QRecord effective(QRecord record, Optional<List<QRecord>> oldRecords)
|
||||
{
|
||||
QRecord merged = new QRecord();
|
||||
if(oldRecords != null && oldRecords.isPresent())
|
||||
{
|
||||
oldRecords.get().stream().filter(r -> Objects.equals(r.getValue("id"), record.getValue("id"))).findFirst()
|
||||
.ifPresent(r -> merged.getValues().putAll(r.getValues()));
|
||||
}
|
||||
merged.getValues().putAll(record.getValues());
|
||||
return (merged);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Validate a provider without resolving any credentials.
|
||||
*******************************************************************************/
|
||||
public static List<String> validateProvider(QRecord row, EmailQBitConfig config)
|
||||
{
|
||||
List<String> errors = new ArrayList<>();
|
||||
String kind = row.getValueString("kind");
|
||||
if("GMAIL_API".equals(kind))
|
||||
{
|
||||
errors.add("Gmail API sending is specified but not built; use GMAIL_SMTP");
|
||||
}
|
||||
if(List.of("POSTMARK", "SENDGRID", "MAILGUN", "GMAIL_SMTP").contains(kind == null ? "" : kind)
|
||||
&& (row.getValueString("secretEnvName") == null || row.getValueString("secretEnvName").isBlank()))
|
||||
{
|
||||
errors.add("Provider needs a secret environment name");
|
||||
}
|
||||
if("GMAIL_SMTP".equals(kind))
|
||||
{
|
||||
if(row.getValueString("usernameEnvName") == null || row.getValueString("usernameEnvName").isBlank())
|
||||
{
|
||||
errors.add("Gmail SMTP needs an account address environment name");
|
||||
}
|
||||
if(row.getValueString("security") != null && !"STARTTLS".equals(row.getValueString("security")))
|
||||
{
|
||||
errors.add("Gmail SMTP requires STARTTLS");
|
||||
}
|
||||
}
|
||||
if("MAILGUN".equals(kind))
|
||||
{
|
||||
if(!EmailProviderSettings.validSendingDomain(row.getValueString("sendingDomain")))
|
||||
{
|
||||
errors.add("Mailgun needs a valid sending domain");
|
||||
}
|
||||
if(!List.of("US", "EU").contains(row.getValueString("region") == null ? "" : row.getValueString("region")))
|
||||
{
|
||||
errors.add("Mailgun region must be US or EU");
|
||||
}
|
||||
}
|
||||
if(("SMTP".equals(kind) || "GMAIL_SMTP".equals(kind))
|
||||
&& !new EmailProviderSettings(row, config).allowsSmtpAuth())
|
||||
{
|
||||
errors.add("SMTP sign-in needs STARTTLS or TLS");
|
||||
}
|
||||
for(String field : List.of("fromName", "fromAddress", "replyTo"))
|
||||
{
|
||||
String value = row.getValueString(field);
|
||||
if(value != null && (value.contains("\r") || value.contains("\n")))
|
||||
{
|
||||
errors.add("Provider sender and reply-to fields must not contain line breaks");
|
||||
}
|
||||
}
|
||||
for(String field : List.of("usernameEnvName", "secretEnvName", "webhookSecretEnvName"))
|
||||
{
|
||||
String name = row.getValueString(field);
|
||||
if(name != null && (!EmailQBitConfig.ENV_NAME.matcher(name).matches() || !config.effectiveAllowedEnvNames(kind).contains(name)))
|
||||
{
|
||||
errors.add("Credential references must be allowed environment variable names");
|
||||
}
|
||||
}
|
||||
if("SMTP".equals(kind) && (row.getValueString("host") == null || row.getValueString("host").isBlank()
|
||||
|| row.getValueInteger("port") == null || row.getValueInteger("port") < 1 || row.getValueInteger("port") > 65535))
|
||||
{
|
||||
errors.add("SMTP needs a host and a port between 1 and 65535");
|
||||
}
|
||||
if("SES".equals(kind) && (row.getValueString("region") == null || row.getValueString("region").isBlank()))
|
||||
{
|
||||
errors.add("SES needs a region");
|
||||
}
|
||||
if("SES".equals(kind) && (row.getValueString("usernameEnvName") == null) != (row.getValueString("secretEnvName") == null))
|
||||
{
|
||||
errors.add("SES needs both credential environment names or neither for the default chain");
|
||||
}
|
||||
for(String field : List.of("maxPerMinute", "maxPerDay"))
|
||||
{
|
||||
if(row.getValueInteger(field) != null && row.getValueInteger(field) < 1)
|
||||
{
|
||||
errors.add("Rate limits must be positive or empty");
|
||||
}
|
||||
}
|
||||
try
|
||||
{
|
||||
InternetAddress from = new InternetAddress(row.getValueString("fromAddress"), true);
|
||||
from.validate();
|
||||
String domain = row.getValueString("sendingDomain");
|
||||
if(domain != null && !domain.equalsIgnoreCase(from.getAddress().substring(from.getAddress().lastIndexOf('@') + 1)))
|
||||
{
|
||||
errors.add("From address domain must match sending domain");
|
||||
}
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
errors.add("A valid from address is required");
|
||||
}
|
||||
return (errors);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Provider validation and secret-presence display.
|
||||
*******************************************************************************/
|
||||
public static class ProviderCustomizer implements TableCustomizerInterface
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Validate the resulting provider set, including sparse and batch writes.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public List<QRecord> preInsertOrUpdate(AbstractActionInput input, List<QRecord> records, boolean preview, Optional<List<QRecord>> oldRecords) throws QException
|
||||
{
|
||||
if(SystemWrites.active())
|
||||
{
|
||||
return (records);
|
||||
}
|
||||
List<QRecord> resulting = new ArrayList<>(SystemWrites.query(EmailTables.PROVIDER, new QQueryFilter(),
|
||||
input instanceof com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput insert ? insert.getTransaction()
|
||||
: input instanceof com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput update ? update.getTransaction() : null));
|
||||
for(QRecord record : records)
|
||||
{
|
||||
QRecord merged = effective(record, oldRecords);
|
||||
validateProvider(merged, EmailRuntime.config()).forEach(error -> record.addError(new BadInputStatusMessage(error)));
|
||||
for(String field : List.of("usernameEnvName", "secretEnvName", "webhookSecretEnvName"))
|
||||
{
|
||||
String name = record.getValueString(field);
|
||||
if(name != null && !EmailQBitConfig.ENV_NAME.matcher(name).matches())
|
||||
{
|
||||
// An accidentally pasted credential must not be echoed in the failed write.
|
||||
record.setValue(field, null);
|
||||
}
|
||||
}
|
||||
resulting.removeIf(r -> Objects.equals(r.getValue("id"), record.getValue("id")));
|
||||
resulting.add(merged);
|
||||
record.setValue("defaultSlot", Boolean.TRUE.equals(merged.getValueBoolean("enabled")) && Boolean.TRUE.equals(merged.getValueBoolean("isDefault")) ? 1 : null);
|
||||
}
|
||||
long enabled = resulting.stream().filter(r -> Boolean.TRUE.equals(r.getValueBoolean("enabled"))).count();
|
||||
long defaults = resulting.stream().filter(r -> Boolean.TRUE.equals(r.getValueBoolean("enabled")) && Boolean.TRUE.equals(r.getValueBoolean("isDefault"))).count();
|
||||
if(enabled > 0 && defaults != 1)
|
||||
{
|
||||
records.forEach(r -> r.addError(new BadInputStatusMessage("Exactly one enabled provider must be the default")));
|
||||
}
|
||||
return (records);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Prevent deleting the default while other enabled providers remain.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public List<QRecord> preDelete(DeleteInput input, List<QRecord> records, boolean preview) throws QException
|
||||
{
|
||||
for(QRecord row : records)
|
||||
{
|
||||
if(!SystemWrites.query(EmailTables.MESSAGE, com.kingsrook.qbits.email.support.EmailStore.equal("providerId", row.getValueInteger("id")).withLimit(1), input.getTransaction()).isEmpty())
|
||||
{
|
||||
row.addError(new BadInputStatusMessage("Providers with message history cannot be deleted; disable the provider instead"));
|
||||
}
|
||||
}
|
||||
List<QRecord> remaining = new ArrayList<>(SystemWrites.query(EmailTables.PROVIDER, new QQueryFilter(), input.getTransaction()));
|
||||
remaining.removeIf(r -> records.stream().anyMatch(d -> Objects.equals(d.getValue("id"), r.getValue("id"))));
|
||||
if(remaining.stream().anyMatch(r -> Boolean.TRUE.equals(r.getValueBoolean("enabled")))
|
||||
&& remaining.stream().noneMatch(r -> Boolean.TRUE.equals(r.getValueBoolean("enabled")) && Boolean.TRUE.equals(r.getValueBoolean("isDefault"))))
|
||||
{
|
||||
records.forEach(r -> r.addError(new BadInputStatusMessage("Choose another enabled default before deleting this provider")));
|
||||
}
|
||||
return (records);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Presence only; internal queries never resolve credentials.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public List<QRecord> postQuery(QueryOrGetInputInterface input, List<QRecord> records)
|
||||
{
|
||||
for(QRecord row : records)
|
||||
{
|
||||
String label = row.getValueString("label");
|
||||
row.setRecordLabel(label == null || label.isBlank() ? row.getValueString("name") : label);
|
||||
}
|
||||
if(!SystemWrites.active())
|
||||
{
|
||||
for(QRecord row : records)
|
||||
{
|
||||
for(String field : List.of("usernameEnvName", "secretEnvName", "webhookSecretEnvName"))
|
||||
{
|
||||
String value = null;
|
||||
String name = row.getValueString(field);
|
||||
EmailQBitConfig config = EmailRuntime.config();
|
||||
if(name != null && EmailQBitConfig.ENV_NAME.matcher(name).matches() && config.effectiveAllowedEnvNames(row.getValueString("kind")).contains(name))
|
||||
{
|
||||
try
|
||||
{
|
||||
value = config.getSecretResolver().apply(name);
|
||||
}
|
||||
catch(RuntimeException e)
|
||||
{
|
||||
// A failing resolver must never leak exception text or values.
|
||||
}
|
||||
}
|
||||
row.setValue(field + "Status", value == null || value.isBlank() ? "Not set" : "Set");
|
||||
}
|
||||
}
|
||||
}
|
||||
return (records);
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Canonical addresses and administrator protection for complaint removal.
|
||||
*******************************************************************************/
|
||||
public static class SuppressionCustomizer implements TableCustomizerInterface
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Normalize and protect existing complaint rows from being weakened.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public List<QRecord> preInsertOrUpdate(AbstractActionInput input, List<QRecord> records, boolean preview, Optional<List<QRecord>> oldRecords)
|
||||
{
|
||||
for(QRecord record : records)
|
||||
{
|
||||
QRecord merged = effective(record, oldRecords);
|
||||
if(merged.getValue("createdDate") == null)
|
||||
{
|
||||
record.setValue("createdDate", EmailRuntime.clock().instant());
|
||||
}
|
||||
if(record.getValueString("address") != null)
|
||||
{
|
||||
try
|
||||
{
|
||||
InternetAddress address = new InternetAddress(record.getValueString("address"), true);
|
||||
address.validate();
|
||||
record.setValue("address", address.getAddress().toLowerCase(Locale.ROOT));
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
record.addError(new BadInputStatusMessage("A valid suppression address is required"));
|
||||
}
|
||||
}
|
||||
if(oldRecords != null && oldRecords.isPresent() && oldRecords.get().stream().anyMatch(r -> Objects.equals(r.getValue("id"), merged.getValue("id")) && "COMPLAINT".equals(r.getValueString("reason"))))
|
||||
{
|
||||
protect(record);
|
||||
}
|
||||
}
|
||||
return (records);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Only administrators remove complaint suppressions.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public List<QRecord> preDelete(DeleteInput input, List<QRecord> records, boolean preview)
|
||||
{
|
||||
records.stream().filter(r -> "COMPLAINT".equals(r.getValueString("reason"))).forEach(SuppressionCustomizer::protect);
|
||||
return (records);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Apply the complaint permission even if core enforcement is disabled.
|
||||
*******************************************************************************/
|
||||
private static void protect(QRecord record)
|
||||
{
|
||||
if(!SystemWrites.active() && !EmailPermissionChecker.has(EmailPermissions.ADMIN))
|
||||
{
|
||||
record.addError(new BadInputStatusMessage("Changing a complaint suppression needs email.admin"));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Link application records through the standard dynamic record-link adornment.
|
||||
*******************************************************************************/
|
||||
public static class MessageLinksCustomizer implements TableCustomizerInterface
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Supply only the target table name; no linked record content is loaded.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public List<QRecord> postQuery(QueryOrGetInputInterface input, List<QRecord> records)
|
||||
{
|
||||
if(!SystemWrites.active())
|
||||
{
|
||||
for(QRecord row : records)
|
||||
{
|
||||
for(String field : List.of("subject", "htmlBody", "textBody"))
|
||||
{
|
||||
String value = row.getValueString(field);
|
||||
if(value != null && value.startsWith("enc:v1:"))
|
||||
{
|
||||
row.setValue(field, "[encrypted]");
|
||||
row.setDisplayValue(field, "[encrypted]");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
for(QRecord row : records)
|
||||
{
|
||||
String subject = row.getValueString("subject");
|
||||
boolean secure = Boolean.TRUE.equals(row.getValueBoolean("phiApproved")) || "[encrypted]".equals(subject) || (subject != null && subject.startsWith("enc:v1:"));
|
||||
row.setRecordLabel(secure ? "Secure message" : subject == null || subject.isBlank() ? "Message " + row.getValue("id") : subject);
|
||||
}
|
||||
if(input.getShouldGenerateDisplayValues())
|
||||
{
|
||||
for(QRecord row : records)
|
||||
{
|
||||
String table = row.getValueString("linkedTableName");
|
||||
if(table != null && row.getValue("linkedRecordId") != null
|
||||
&& com.kingsrook.qqq.backend.core.context.QContext.getQInstance().getTable(table) != null)
|
||||
{
|
||||
row.setDisplayValue("linkedRecordId:toRecordFromTableDynamic", table);
|
||||
}
|
||||
}
|
||||
}
|
||||
return (records);
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
@@ -0,0 +1,435 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.metadata;
|
||||
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import com.kingsrook.qbits.email.EmailQBitConfig;
|
||||
import com.kingsrook.qbits.email.outbox.EmailCategory;
|
||||
import com.kingsrook.qbits.email.permissions.EmailPermissionChecker;
|
||||
import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.ChildRecordListRenderer;
|
||||
import com.kingsrook.qqq.backend.core.instances.QInstanceEnricher;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.fields.AdornmentType;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.fields.FieldAdornment;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.joins.JoinOn;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.joins.JoinType;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.joins.QJoinMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.permissions.PermissionLevel;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRules;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSource;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.tables.Capability;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.tables.SectionFactory;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.tables.UniqueKey;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Standard visible tables, with system-owned messages and attempt claims.
|
||||
*******************************************************************************/
|
||||
public final class EmailTables
|
||||
{
|
||||
public static final String PROVIDER = "email_provider";
|
||||
public static final String MESSAGE = "email_message";
|
||||
public static final String ATTEMPT = "email_message_attempt";
|
||||
public static final String TEMPLATE = "email_template";
|
||||
public static final String VERSION = "email_template_version";
|
||||
public static final String BRAND = "email_brand";
|
||||
public static final String SUPPRESSION = "email_suppression";
|
||||
public static final List<String> ALL = List.of(PROVIDER, MESSAGE, ATTEMPT, TEMPLATE, VERSION, BRAND, SUPPRESSION);
|
||||
public static final List<String> MAIN = List.of(MESSAGE, TEMPLATE, BRAND, SUPPRESSION, PROVIDER);
|
||||
public static final List<String> SYSTEM_WRITTEN = List.of(MESSAGE, ATTEMPT);
|
||||
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private EmailTables()
|
||||
{
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Shared permission base and custom action checker.
|
||||
*******************************************************************************/
|
||||
public static QPermissionRules permissionRules()
|
||||
{
|
||||
return (new QPermissionRules().withLevel(PermissionLevel.HAS_ACCESS_PERMISSION).withPermissionBaseName("email")
|
||||
.withCustomPermissionChecker(new QCodeReference(EmailPermissionChecker.class)));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** All enum and table possible-value sources.
|
||||
*******************************************************************************/
|
||||
public static List<QPossibleValueSource> possibleValueSources()
|
||||
{
|
||||
List<QPossibleValueSource> result = new ArrayList<>();
|
||||
result.add(QPossibleValueSource.newForEnum("email_provider_kind", EmailEnums.ProviderKind.values()));
|
||||
result.add(QPossibleValueSource.newForEnum("email_security", EmailEnums.Security.values()));
|
||||
result.add(QPossibleValueSource.newForEnum("email_health_status", EmailEnums.HealthStatus.values()));
|
||||
result.add(QPossibleValueSource.newForEnum("email_message_status", EmailEnums.MessageStatus.values()));
|
||||
result.add(QPossibleValueSource.newForEnum("email_attempt_outcome", EmailEnums.AttemptOutcome.values()));
|
||||
result.add(QPossibleValueSource.newForEnum("email_template_kind", EmailEnums.TemplateKind.values()));
|
||||
result.add(QPossibleValueSource.newForEnum("email_version_status", EmailEnums.VersionStatus.values()));
|
||||
result.add(QPossibleValueSource.newForEnum("email_suppression_scope", EmailEnums.SuppressionScope.values()));
|
||||
result.add(QPossibleValueSource.newForEnum("email_suppression_reason", EmailEnums.SuppressionReason.values()));
|
||||
result.add(QPossibleValueSource.newForEnum("email_category", EmailCategory.values()));
|
||||
ALL.forEach(name -> result.add(QPossibleValueSource.newForTable(name)));
|
||||
for(QPossibleValueSource source : result)
|
||||
{
|
||||
if(List.of(PROVIDER, MESSAGE, VERSION).contains(source.getName()))
|
||||
{
|
||||
source.withSearchFields(PROVIDER.equals(source.getName()) ? List.of("label", "name") : MESSAGE.equals(source.getName()) ? List.of("subject") : List.of("versionNumber", "locale"))
|
||||
.withOrderByField("id");
|
||||
}
|
||||
}
|
||||
return (result);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** The seven phase (a) tables.
|
||||
*******************************************************************************/
|
||||
public static List<QTableMetaData> tables(EmailQBitConfig config)
|
||||
{
|
||||
List<QTableMetaData> tables = new ArrayList<>();
|
||||
tables.add(provider(config));
|
||||
tables.add(message(config));
|
||||
tables.add(messageattempt(config));
|
||||
tables.add(template(config));
|
||||
tables.add(templateversion(config));
|
||||
tables.add(brand(config));
|
||||
tables.add(suppression(config));
|
||||
for(QTableMetaData table : tables)
|
||||
{
|
||||
for(QFieldMetaData field : config.getSecurityFields())
|
||||
{
|
||||
table.addField(field.clone());
|
||||
table.getSections().get(0).getFieldNames().add(field.getName());
|
||||
}
|
||||
config.getRecordSecurityLocks().forEach(table::withRecordSecurityLock);
|
||||
if(SYSTEM_WRITTEN.contains(table.getName()))
|
||||
{
|
||||
table.withoutCapabilities(Capability.TABLE_INSERT, Capability.TABLE_UPDATE, Capability.TABLE_DELETE);
|
||||
table.getFields().values().forEach(f -> f.setIsEditable(false));
|
||||
}
|
||||
table.getFields().values().forEach(f -> f.setBackendName(QInstanceEnricher.inferBackendName(f.getName())));
|
||||
}
|
||||
return (tables);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Common fields and record label.
|
||||
*******************************************************************************/
|
||||
private static QTableMetaData base(EmailQBitConfig config, String name, String label)
|
||||
{
|
||||
return (new QTableMetaData().withName(name).withLabel(label).withBackendName(config.getBackendName())
|
||||
.withPrimaryKeyField("id").withIcon(new QIcon("mail")).withPermissionRules(permissionRules())
|
||||
.withRecordLabelFormatAndFields("%s", "id")
|
||||
.withField(field("id", QFieldType.INTEGER).withIsEditable(false))
|
||||
.withField(field("createDate", QFieldType.DATE_TIME).withIsEditable(false))
|
||||
.withField(field("modifyDate", QFieldType.DATE_TIME).withIsEditable(false)));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Field with a plain English label.
|
||||
*******************************************************************************/
|
||||
private static QFieldMetaData field(String name, QFieldType type)
|
||||
{
|
||||
String words = name.replaceAll("([a-z])([A-Z])", "$1 $2");
|
||||
return (new QFieldMetaData(name, type).withLabel(Character.toUpperCase(words.charAt(0)) + words.substring(1)));
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Metadata for email_provider.
|
||||
*******************************************************************************/
|
||||
private static QTableMetaData provider(EmailQBitConfig config)
|
||||
{
|
||||
QTableMetaData table = base(config, PROVIDER, "Provider")
|
||||
.withField(field("name", QFieldType.STRING).withIsRequired(true).withMaxLength(500))
|
||||
.withField(field("label", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("kind", QFieldType.STRING).withPossibleValueSourceName("email_provider_kind").withIsRequired(true).withMaxLength(500))
|
||||
.withField(field("enabled", QFieldType.BOOLEAN).withDefaultValue(true))
|
||||
.withField(field("isDefault", QFieldType.BOOLEAN).withDefaultValue(false))
|
||||
.withField(field("fromAddress", QFieldType.STRING).withIsSensitive(true).withMaxLength(500))
|
||||
.withField(field("fromName", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("replyTo", QFieldType.STRING).withIsSensitive(true).withMaxLength(500))
|
||||
.withField(field("sendingDomain", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("host", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("port", QFieldType.INTEGER))
|
||||
.withField(field("security", QFieldType.STRING).withPossibleValueSourceName("email_security").withDefaultValue("NONE").withMaxLength(500))
|
||||
.withField(field("region", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("usernameEnvName", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("secretEnvName", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("webhookSecretEnvName", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("snsTopicArn", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("configurationSet", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("messageStream", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("maxPerMinute", QFieldType.INTEGER))
|
||||
.withField(field("maxPerDay", QFieldType.INTEGER))
|
||||
.withField(field("trackOpens", QFieldType.BOOLEAN).withDefaultValue(false))
|
||||
.withField(field("trackClicks", QFieldType.BOOLEAN).withDefaultValue(false))
|
||||
.withField(field("phiAllowed", QFieldType.BOOLEAN).withDefaultValue(false))
|
||||
.withField(field("healthStatus", QFieldType.STRING).withPossibleValueSourceName("email_health_status").withDefaultValue("HEALTHY").withMaxLength(500))
|
||||
.withField(field("lastSuccessDate", QFieldType.DATE_TIME))
|
||||
.withField(field("lastFailureDate", QFieldType.DATE_TIME))
|
||||
.withField(field("lastFailureMessage", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("dispatchLock", QFieldType.INTEGER).withDefaultValue(0).withIsHidden(true).withIsEditable(false))
|
||||
.withField(field("defaultSlot", QFieldType.INTEGER).withIsHidden(true).withIsEditable(false));
|
||||
table.withRecordLabelFormat(null).withRecordLabelFields(List.of());
|
||||
table.withUniqueKey(new UniqueKey("name"));
|
||||
table.withUniqueKey(new UniqueKey("defaultSlot"));
|
||||
table.withSection(SectionFactory.defaultT1("id", "name", "label", "kind", "enabled", "isDefault"));
|
||||
table.withSection(SectionFactory.defaultT2("fromAddress", "fromName", "replyTo", "sendingDomain", "host", "port", "security", "region", "usernameEnvName", "secretEnvName", "webhookSecretEnvName", "snsTopicArn", "configurationSet", "messageStream", "maxPerMinute", "maxPerDay", "trackOpens", "trackClicks", "phiAllowed", "healthStatus", "lastSuccessDate", "lastFailureDate", "lastFailureMessage"));
|
||||
table.withSection(SectionFactory.defaultT3("createDate", "modifyDate"));
|
||||
table.withSection(SectionFactory.customT2("children", new QIcon("list")).withLabel("Messages").withWidgetName(PROVIDER + "_children"));
|
||||
return (table);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Metadata for email_message.
|
||||
*******************************************************************************/
|
||||
private static QTableMetaData message(EmailQBitConfig config)
|
||||
{
|
||||
QTableMetaData table = base(config, MESSAGE, "Message")
|
||||
.withField(field("status", QFieldType.STRING).withPossibleValueSourceName("email_message_status").withMaxLength(500))
|
||||
.withField(field("toAddresses", QFieldType.TEXT).withIsSensitive(true))
|
||||
.withField(field("ccAddresses", QFieldType.TEXT).withIsSensitive(true))
|
||||
.withField(field("bccAddresses", QFieldType.TEXT).withIsSensitive(true))
|
||||
.withField(field("fromAddress", QFieldType.STRING).withIsSensitive(true).withMaxLength(500))
|
||||
.withField(field("replyTo", QFieldType.STRING).withIsSensitive(true).withMaxLength(500))
|
||||
.withField(field("subject", QFieldType.TEXT).withIsSensitive(true))
|
||||
.withField(field("templateId", QFieldType.INTEGER).withPossibleValueSourceName(TEMPLATE))
|
||||
.withField(field("templateVersionId", QFieldType.INTEGER).withPossibleValueSourceName(VERSION).withPossibleValueSourceFilter(com.kingsrook.qbits.email.support.EmailStore.equal("templateId", "${input.templateId}")))
|
||||
.withField(field("brandId", QFieldType.INTEGER).withPossibleValueSourceName(BRAND))
|
||||
.withField(field("category", QFieldType.STRING).withPossibleValueSourceName("email_category").withMaxLength(500))
|
||||
.withField(field("locale", QFieldType.STRING).withDefaultValue("en").withMaxLength(500))
|
||||
.withField(field("variablesSnapshot", QFieldType.TEXT).withIsSensitive(true))
|
||||
.withField(field("htmlBody", QFieldType.TEXT).withIsSensitive(true))
|
||||
.withField(field("textBody", QFieldType.TEXT).withIsSensitive(true))
|
||||
.withField(field("phiApproved", QFieldType.BOOLEAN).withDefaultValue(false).withIsEditable(false))
|
||||
.withField(field("unsubscribeUrl", QFieldType.TEXT))
|
||||
.withField(field("bodySha256", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("idempotencyKey", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("providerId", QFieldType.INTEGER).withPossibleValueSourceName(PROVIDER))
|
||||
.withField(field("providerMessageId", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("attemptCount", QFieldType.INTEGER).withDefaultValue(0))
|
||||
.withField(field("nextAttemptDate", QFieldType.DATE_TIME))
|
||||
.withField(field("leaseExpiresDate", QFieldType.DATE_TIME))
|
||||
.withField(field("lastError", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("finishedDate", QFieldType.DATE_TIME))
|
||||
.withField(field("sentDate", QFieldType.DATE_TIME))
|
||||
.withField(field("deliveredDate", QFieldType.DATE_TIME))
|
||||
.withField(field("bouncedDate", QFieldType.DATE_TIME))
|
||||
.withField(field("complainedDate", QFieldType.DATE_TIME))
|
||||
.withField(field("deadLetterDate", QFieldType.DATE_TIME))
|
||||
.withField(field("linkedTableName", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("linkedRecordId", QFieldType.STRING).withMaxLength(500)
|
||||
.withFieldAdornment(new FieldAdornment(AdornmentType.LINK).withValue(AdornmentType.LinkValues.TO_RECORD_FROM_TABLE_DYNAMIC, true)))
|
||||
.withField(field("senderQbit", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("senderProcess", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("suppressedAddresses", QFieldType.TEXT).withIsSensitive(true))
|
||||
.withField(field("sendAfter", QFieldType.DATE_TIME));
|
||||
table.withRecordLabelFormat(null).withRecordLabelFields(List.of());
|
||||
table.getField("subject").setIsHeavy(false);
|
||||
table.withUniqueKey(new UniqueKey("idempotencyKey"));
|
||||
table.withSection(SectionFactory.defaultT1("id", "status", "toAddresses", "ccAddresses", "bccAddresses", "fromAddress"));
|
||||
table.withSection(SectionFactory.defaultT2("replyTo", "subject", "templateId", "templateVersionId", "brandId", "category", "locale", "variablesSnapshot", "htmlBody", "textBody", "phiApproved", "unsubscribeUrl", "bodySha256", "idempotencyKey", "providerId", "providerMessageId", "attemptCount", "nextAttemptDate", "leaseExpiresDate", "lastError", "finishedDate", "sentDate", "deliveredDate", "bouncedDate", "complainedDate", "deadLetterDate", "linkedTableName", "linkedRecordId", "senderQbit", "senderProcess", "suppressedAddresses", "sendAfter"));
|
||||
table.withSection(SectionFactory.defaultT3("createDate", "modifyDate"));
|
||||
table.withSection(SectionFactory.customT2("children", new QIcon("list")).withLabel("Attempts").withWidgetName(MESSAGE + "_children"));
|
||||
return (table);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Metadata for email_message_attempt.
|
||||
*******************************************************************************/
|
||||
private static QTableMetaData messageattempt(EmailQBitConfig config)
|
||||
{
|
||||
QTableMetaData table = base(config, ATTEMPT, "Message Attempt")
|
||||
.withField(field("messageId", QFieldType.INTEGER).withPossibleValueSourceName(MESSAGE).withIsRequired(true))
|
||||
.withField(field("attemptNumber", QFieldType.INTEGER).withIsRequired(true))
|
||||
.withField(field("outcome", QFieldType.STRING).withPossibleValueSourceName("email_attempt_outcome").withMaxLength(500))
|
||||
.withField(field("providerId", QFieldType.INTEGER).withPossibleValueSourceName(PROVIDER))
|
||||
.withField(field("claimedBy", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("startDate", QFieldType.DATE_TIME))
|
||||
.withField(field("endDate", QFieldType.DATE_TIME))
|
||||
.withField(field("leaseExpiresDate", QFieldType.DATE_TIME))
|
||||
.withField(field("durationMillis", QFieldType.INTEGER))
|
||||
.withField(field("error", QFieldType.STRING).withMaxLength(500));
|
||||
table.withUniqueKey(new UniqueKey("messageId", "attemptNumber"));
|
||||
table.withSection(SectionFactory.defaultT1("id", "messageId", "attemptNumber", "outcome", "providerId", "claimedBy"));
|
||||
table.withSection(SectionFactory.defaultT2("startDate", "endDate", "leaseExpiresDate", "durationMillis", "error"));
|
||||
table.withSection(SectionFactory.defaultT3("createDate", "modifyDate"));
|
||||
return (table);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Metadata for email_template.
|
||||
*******************************************************************************/
|
||||
private static QTableMetaData template(EmailQBitConfig config)
|
||||
{
|
||||
QTableMetaData table = base(config, TEMPLATE, "Template")
|
||||
.withField(field("name", QFieldType.STRING).withIsRequired(true).withMaxLength(500))
|
||||
.withField(field("label", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("category", QFieldType.STRING).withPossibleValueSourceName("email_category").withMaxLength(500))
|
||||
.withField(field("kind", QFieldType.STRING).withPossibleValueSourceName("email_template_kind").withIsRequired(true).withMaxLength(500))
|
||||
.withField(field("brandId", QFieldType.INTEGER).withPossibleValueSourceName(BRAND))
|
||||
.withField(field("activeVersionId", QFieldType.INTEGER).withPossibleValueSourceName(VERSION).withIsEditable(false).withPossibleValueSourceFilter(com.kingsrook.qbits.email.support.EmailStore.equal("templateId", "${input.id}")))
|
||||
.withField(field("approvedVersionId", QFieldType.INTEGER).withIsEditable(false).withPossibleValueSourceName(VERSION).withPossibleValueSourceFilter(com.kingsrook.qbits.email.support.EmailStore.equal("templateId", "${input.id}")))
|
||||
.withField(field("versionLock", QFieldType.INTEGER).withIsHidden(true).withIsEditable(false))
|
||||
.withField(field("phiApproved", QFieldType.BOOLEAN).withDefaultValue(false).withIsEditable(false))
|
||||
.withField(field("phiApprovedBy", QFieldType.STRING).withMaxLength(500).withIsEditable(false))
|
||||
.withField(field("phiApprovedDate", QFieldType.DATE_TIME).withIsEditable(false))
|
||||
.withField(field("editPermissionName", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("ownerQbit", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("description", QFieldType.TEXT));
|
||||
table.withRecordLabelFormatAndFields("%s", "name");
|
||||
table.withUniqueKey(new UniqueKey("name"));
|
||||
table.withSection(SectionFactory.defaultT1("id", "name", "label", "category", "kind", "brandId"));
|
||||
table.withSection(SectionFactory.defaultT2("activeVersionId", "approvedVersionId", "phiApproved", "phiApprovedBy", "phiApprovedDate", "editPermissionName", "ownerQbit", "description"));
|
||||
table.withSection(SectionFactory.defaultT3("createDate", "modifyDate"));
|
||||
table.withSection(SectionFactory.customT2("children", new QIcon("list")).withLabel("Versions").withWidgetName(TEMPLATE + "_children"));
|
||||
return (table);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Metadata for email_template_version.
|
||||
*******************************************************************************/
|
||||
private static QTableMetaData templateversion(EmailQBitConfig config)
|
||||
{
|
||||
QTableMetaData table = base(config, VERSION, "Template Version")
|
||||
.withField(field("templateId", QFieldType.INTEGER).withPossibleValueSourceName(TEMPLATE).withIsRequired(true))
|
||||
.withField(field("versionNumber", QFieldType.INTEGER).withIsRequired(true))
|
||||
.withField(field("status", QFieldType.STRING).withPossibleValueSourceName("email_version_status").withDefaultValue("DRAFT").withIsEditable(false).withMaxLength(500))
|
||||
.withField(field("locale", QFieldType.STRING).withDefaultValue("en").withMaxLength(500).withIsRequired(true))
|
||||
.withField(field("subject", QFieldType.TEXT).withIsSensitive(true))
|
||||
.withField(field("preheader", QFieldType.TEXT))
|
||||
.withField(field("bodySource", QFieldType.TEXT))
|
||||
.withField(field("textSource", QFieldType.TEXT))
|
||||
.withField(field("compiledHtml", QFieldType.TEXT).withIsEditable(false))
|
||||
.withField(field("variablesSchema", QFieldType.TEXT))
|
||||
.withField(field("sampleVariables", QFieldType.TEXT))
|
||||
.withField(field("createdBy", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("activatedDate", QFieldType.DATE_TIME));
|
||||
table.withRecordLabelFormat(null).withRecordLabelFields(List.of());
|
||||
table.withUniqueKey(new UniqueKey("templateId", "versionNumber", "locale"));
|
||||
table.withSection(SectionFactory.customT2("preview", new QIcon("preview")).withLabel("Preview").withWidgetName("emailTemplatePreview"));
|
||||
table.withSection(SectionFactory.defaultT1("id", "templateId", "versionNumber", "status", "locale", "subject"));
|
||||
table.withSection(SectionFactory.defaultT2("preheader", "bodySource", "textSource", "compiledHtml", "variablesSchema", "sampleVariables", "createdBy", "activatedDate"));
|
||||
table.withSection(SectionFactory.defaultT3("createDate", "modifyDate"));
|
||||
return (table);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Metadata for email_brand.
|
||||
*******************************************************************************/
|
||||
private static QTableMetaData brand(EmailQBitConfig config)
|
||||
{
|
||||
QTableMetaData table = base(config, BRAND, "Brand")
|
||||
.withField(field("name", QFieldType.STRING).withIsRequired(true).withMaxLength(500))
|
||||
.withField(field("label", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("logoUrl", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("logoDarkUrl", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("logoAlt", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("primaryColor", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("textColor", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("backgroundColor", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("darkBackgroundColor", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("darkPrimaryColor", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("darkButtonTextColor", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("darkTextColor", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("buttonTextColor", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("footerMarkdown", QFieldType.TEXT))
|
||||
.withField(field("postalAddress", QFieldType.TEXT))
|
||||
.withField(field("socialLinks", QFieldType.TEXT))
|
||||
.withField(field("siteUrl", QFieldType.STRING).withMaxLength(500))
|
||||
.withField(field("isDefault", QFieldType.BOOLEAN).withDefaultValue(false))
|
||||
.withField(field("appName", QFieldType.STRING).withMaxLength(500));
|
||||
table.withRecordLabelFormatAndFields("%s", "name");
|
||||
table.withUniqueKey(new UniqueKey("name"));
|
||||
table.withSection(SectionFactory.defaultT1("id", "name", "label", "logoUrl", "logoDarkUrl", "logoAlt"));
|
||||
table.withSection(SectionFactory.defaultT2("primaryColor", "textColor", "backgroundColor", "darkBackgroundColor", "darkTextColor", "darkPrimaryColor", "darkButtonTextColor", "buttonTextColor", "footerMarkdown", "postalAddress", "socialLinks", "siteUrl", "isDefault", "appName"));
|
||||
table.withSection(SectionFactory.defaultT3("createDate", "modifyDate"));
|
||||
return (table);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Metadata for email_suppression.
|
||||
*******************************************************************************/
|
||||
private static QTableMetaData suppression(EmailQBitConfig config)
|
||||
{
|
||||
QTableMetaData table = base(config, SUPPRESSION, "Suppression")
|
||||
.withField(field("address", QFieldType.STRING).withIsRequired(true).withMaxLength(500))
|
||||
.withField(field("scope", QFieldType.STRING).withPossibleValueSourceName("email_suppression_scope").withIsRequired(true).withMaxLength(500))
|
||||
.withField(field("reason", QFieldType.STRING).withPossibleValueSourceName("email_suppression_reason").withIsRequired(true).withMaxLength(500))
|
||||
.withField(field("sourceMessageId", QFieldType.INTEGER))
|
||||
.withField(field("sourceEventId", QFieldType.INTEGER))
|
||||
.withField(field("note", QFieldType.TEXT))
|
||||
.withField(field("createdDate", QFieldType.DATE_TIME))
|
||||
.withField(field("expiresDate", QFieldType.DATE_TIME));
|
||||
table.withUniqueKey(new UniqueKey("address", "scope"));
|
||||
table.withSection(SectionFactory.defaultT1("id", "address", "scope", "reason", "sourceMessageId", "sourceEventId"));
|
||||
table.withSection(SectionFactory.defaultT2("note", "createdDate", "expiresDate"));
|
||||
table.withSection(SectionFactory.defaultT3("createDate", "modifyDate"));
|
||||
return (table);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Relations used by child lists.
|
||||
*******************************************************************************/
|
||||
public static List<QJoinMetaData> joins()
|
||||
{
|
||||
return (List.of(join(PROVIDER, MESSAGE, "providerId"), join(MESSAGE, ATTEMPT, "messageId"), join(TEMPLATE, VERSION, "templateId")));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Parent to child relation.
|
||||
*******************************************************************************/
|
||||
private static QJoinMetaData join(String parent, String child, String field)
|
||||
{
|
||||
return (new QJoinMetaData().withName(parent + "_join_" + child).withType(JoinType.ONE_TO_MANY)
|
||||
.withLeftTable(parent).withRightTable(child).withJoinOn(new JoinOn("id", field)));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Child lists live on the parent record, never in navigation.
|
||||
*******************************************************************************/
|
||||
public static List<QWidgetMetaData> childWidgets()
|
||||
{
|
||||
List<QWidgetMetaData> widgets = new ArrayList<>();
|
||||
for(QJoinMetaData join : joins())
|
||||
{
|
||||
widgets.add(ChildRecordListRenderer.widgetMetaDataBuilder(join).withName(join.getLeftTable() + "_children")
|
||||
.withLabel(join.getRightTable().equals(ATTEMPT) ? "Attempts" : join.getRightTable().equals(VERSION) ? "Versions" : "Messages")
|
||||
.withMaxRows(50).getWidgetMetaData().withGridColumns(12).withPermissionRules(permissionRules()));
|
||||
}
|
||||
return (widgets);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.outbox;
|
||||
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Branded ad-hoc rendering. Markdown raw HTML and unsafe URL schemes are disabled.
|
||||
*******************************************************************************/
|
||||
public class AdHocEmailRenderer implements EmailRenderer
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Render markdown, HTML, or plain text into both alternatives.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public Rendered render(EmailRequest request)
|
||||
{
|
||||
try
|
||||
{
|
||||
var brand = com.kingsrook.qbits.email.render.EmailBrands.resolve(request.getBrandName(), null);
|
||||
String html = request.getHtmlBody();
|
||||
if(html == null)
|
||||
{
|
||||
String body = request.getMarkdownBody() == null ? "<pre>" + com.kingsrook.qbits.email.template.EmailTemplateEngine.escape(request.getTextBody()) + "</pre>"
|
||||
: com.kingsrook.qbits.email.render.EmailMarkdown.render(request.getMarkdownBody(), brand);
|
||||
html = com.kingsrook.qbits.email.render.EmailLayout.render(body, "", brand,
|
||||
request.getCategory() == null ? EmailCategory.TRANSACTIONAL : request.getCategory(), "");
|
||||
}
|
||||
return (new Rendered(html, request.getTextBody() == null ? com.kingsrook.qbits.email.render.HtmlToText.convert(html) : request.getTextBody()));
|
||||
}
|
||||
catch(com.kingsrook.qqq.backend.core.exceptions.QException e)
|
||||
{
|
||||
throw (new IllegalArgumentException("Email brand is unavailable"));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.outbox;
|
||||
|
||||
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.PossibleValueEnum;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Purpose of a message; transactional and security messages cannot be opted out.
|
||||
*******************************************************************************/
|
||||
public enum EmailCategory implements PossibleValueEnum<String>
|
||||
{
|
||||
TRANSACTIONAL,
|
||||
NOTIFICATION,
|
||||
MARKETING,
|
||||
SECURITY;
|
||||
|
||||
/*******************************************************************************
|
||||
** Stable database value.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getPossibleValueId()
|
||||
{
|
||||
return (name());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Display label.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getPossibleValueLabel()
|
||||
{
|
||||
return (name().charAt(0) + name().substring(1).toLowerCase(java.util.Locale.ROOT));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.outbox;
|
||||
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Replaceable rendering seam for the versioned template and brand renderer.
|
||||
*******************************************************************************/
|
||||
@FunctionalInterface
|
||||
public interface EmailRenderer
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Render already validated ad-hoc content.
|
||||
*******************************************************************************/
|
||||
Rendered render(EmailRequest request);
|
||||
|
||||
/*******************************************************************************
|
||||
** Rendered alternatives persisted once for all attempts.
|
||||
*******************************************************************************/
|
||||
record Rendered(String html, String text)
|
||||
{
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,609 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.outbox;
|
||||
|
||||
|
||||
import java.io.Serializable;
|
||||
import java.time.Instant;
|
||||
import java.util.ArrayList;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Synchronous enqueue request with explicit record provenance.
|
||||
*******************************************************************************/
|
||||
public class EmailRequest
|
||||
{
|
||||
private List<String> to = new ArrayList<>();
|
||||
private List<String> cc = new ArrayList<>();
|
||||
private List<String> bcc = new ArrayList<>();
|
||||
private String from;
|
||||
private String replyTo;
|
||||
private String subject;
|
||||
private String markdownBody;
|
||||
private String htmlBody;
|
||||
private String textBody;
|
||||
private String providerName;
|
||||
private String templateName;
|
||||
private String brandName;
|
||||
private String locale = "en";
|
||||
private EmailCategory category;
|
||||
private String idempotencyKey;
|
||||
private Instant sendAfter;
|
||||
private String linkedTableName;
|
||||
private String linkedRecordId;
|
||||
private String senderQbit;
|
||||
private String senderProcess;
|
||||
private QBackendTransaction transaction;
|
||||
private String transactionBackendName;
|
||||
private Map<String, RecordVariable> recordVariables = new LinkedHashMap<>();
|
||||
private Map<String, Serializable> variables = new LinkedHashMap<>();
|
||||
private Map<String, Serializable> securityValues = new LinkedHashMap<>();
|
||||
|
||||
/*******************************************************************************
|
||||
** Append to recipients.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withTo(String... addresses)
|
||||
{
|
||||
to.addAll(List.of(addresses));
|
||||
return (this);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Append cc recipients.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withCc(String... addresses)
|
||||
{
|
||||
cc.addAll(List.of(addresses));
|
||||
return (this);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Append bcc recipients.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withBcc(String... addresses)
|
||||
{
|
||||
bcc.addAll(List.of(addresses));
|
||||
return (this);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Link the message to an application record.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withLinkedRecord(String table, Object id)
|
||||
{
|
||||
linkedTableName = table;
|
||||
linkedRecordId = id == null ? null : String.valueOf(id);
|
||||
return (this);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Identify the producing qbit and process.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withSender(String qbit, String process)
|
||||
{
|
||||
senderQbit = qbit;
|
||||
senderProcess = process;
|
||||
return (this);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Declare the backend of the caller transaction; QBackendTransaction does not expose it.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withTransaction(QBackendTransaction transaction, String backendName)
|
||||
{
|
||||
this.transaction = transaction;
|
||||
this.transactionBackendName = backendName;
|
||||
return (this);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Bind one schema variable.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withVariable(String name, Serializable value)
|
||||
{
|
||||
variables.put(name, value);
|
||||
return (this);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Accept a record reference without reading or snapshotting the record.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withRecordVariable(String name, String table, Serializable id)
|
||||
{
|
||||
recordVariables.put(name, new RecordVariable(table, id));
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for to.
|
||||
*******************************************************************************/
|
||||
public List<String> getTo()
|
||||
{
|
||||
return (to);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for to.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withTo(List<String> to)
|
||||
{
|
||||
this.to = to;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for cc.
|
||||
*******************************************************************************/
|
||||
public List<String> getCc()
|
||||
{
|
||||
return (cc);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for cc.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withCc(List<String> cc)
|
||||
{
|
||||
this.cc = cc;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for bcc.
|
||||
*******************************************************************************/
|
||||
public List<String> getBcc()
|
||||
{
|
||||
return (bcc);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for bcc.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withBcc(List<String> bcc)
|
||||
{
|
||||
this.bcc = bcc;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for from.
|
||||
*******************************************************************************/
|
||||
public String getFrom()
|
||||
{
|
||||
return (from);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for from.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withFrom(String from)
|
||||
{
|
||||
this.from = from;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for replyTo.
|
||||
*******************************************************************************/
|
||||
public String getReplyTo()
|
||||
{
|
||||
return (replyTo);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for replyTo.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withReplyTo(String replyTo)
|
||||
{
|
||||
this.replyTo = replyTo;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for subject.
|
||||
*******************************************************************************/
|
||||
public String getSubject()
|
||||
{
|
||||
return (subject);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for subject.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withSubject(String subject)
|
||||
{
|
||||
this.subject = subject;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for markdownBody.
|
||||
*******************************************************************************/
|
||||
public String getMarkdownBody()
|
||||
{
|
||||
return (markdownBody);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for markdownBody.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withMarkdownBody(String markdownBody)
|
||||
{
|
||||
this.markdownBody = markdownBody;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for htmlBody.
|
||||
*******************************************************************************/
|
||||
public String getHtmlBody()
|
||||
{
|
||||
return (htmlBody);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for htmlBody.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withHtmlBody(String htmlBody)
|
||||
{
|
||||
this.htmlBody = htmlBody;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for textBody.
|
||||
*******************************************************************************/
|
||||
public String getTextBody()
|
||||
{
|
||||
return (textBody);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for textBody.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withTextBody(String textBody)
|
||||
{
|
||||
this.textBody = textBody;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for providerName.
|
||||
*******************************************************************************/
|
||||
public String getProviderName()
|
||||
{
|
||||
return (providerName);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for providerName.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withProviderName(String providerName)
|
||||
{
|
||||
this.providerName = providerName;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for templateName.
|
||||
*******************************************************************************/
|
||||
public String getTemplateName()
|
||||
{
|
||||
return (templateName);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for templateName.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withTemplateName(String templateName)
|
||||
{
|
||||
this.templateName = templateName;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for brandName.
|
||||
*******************************************************************************/
|
||||
public String getBrandName()
|
||||
{
|
||||
return (brandName);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for brandName.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withBrandName(String brandName)
|
||||
{
|
||||
this.brandName = brandName;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for locale.
|
||||
*******************************************************************************/
|
||||
public String getLocale()
|
||||
{
|
||||
return (locale);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for locale.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withLocale(String locale)
|
||||
{
|
||||
this.locale = locale;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for category.
|
||||
*******************************************************************************/
|
||||
public EmailCategory getCategory()
|
||||
{
|
||||
return (category);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for category.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withCategory(EmailCategory category)
|
||||
{
|
||||
this.category = category;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for idempotencyKey.
|
||||
*******************************************************************************/
|
||||
public String getIdempotencyKey()
|
||||
{
|
||||
return (idempotencyKey);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for idempotencyKey.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withIdempotencyKey(String idempotencyKey)
|
||||
{
|
||||
this.idempotencyKey = idempotencyKey;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for sendAfter.
|
||||
*******************************************************************************/
|
||||
public Instant getSendAfter()
|
||||
{
|
||||
return (sendAfter);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for sendAfter.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withSendAfter(Instant sendAfter)
|
||||
{
|
||||
this.sendAfter = sendAfter;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for linkedTableName.
|
||||
*******************************************************************************/
|
||||
public String getLinkedTableName()
|
||||
{
|
||||
return (linkedTableName);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for linkedTableName.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withLinkedTableName(String linkedTableName)
|
||||
{
|
||||
this.linkedTableName = linkedTableName;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for linkedRecordId.
|
||||
*******************************************************************************/
|
||||
public String getLinkedRecordId()
|
||||
{
|
||||
return (linkedRecordId);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for linkedRecordId.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withLinkedRecordId(String linkedRecordId)
|
||||
{
|
||||
this.linkedRecordId = linkedRecordId;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for senderQbit.
|
||||
*******************************************************************************/
|
||||
public String getSenderQbit()
|
||||
{
|
||||
return (senderQbit);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for senderQbit.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withSenderQbit(String senderQbit)
|
||||
{
|
||||
this.senderQbit = senderQbit;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for senderProcess.
|
||||
*******************************************************************************/
|
||||
public String getSenderProcess()
|
||||
{
|
||||
return (senderProcess);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for senderProcess.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withSenderProcess(String senderProcess)
|
||||
{
|
||||
this.senderProcess = senderProcess;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for transaction.
|
||||
*******************************************************************************/
|
||||
public QBackendTransaction getTransaction()
|
||||
{
|
||||
return (transaction);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for transaction.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withTransaction(QBackendTransaction transaction)
|
||||
{
|
||||
this.transaction = transaction;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for transactionBackendName.
|
||||
*******************************************************************************/
|
||||
public String getTransactionBackendName()
|
||||
{
|
||||
return (transactionBackendName);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for transactionBackendName.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withTransactionBackendName(String transactionBackendName)
|
||||
{
|
||||
this.transactionBackendName = transactionBackendName;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for variables.
|
||||
*******************************************************************************/
|
||||
public Map<String, Serializable> getVariables()
|
||||
{
|
||||
return (variables);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for variables.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withVariables(Map<String, Serializable> variables)
|
||||
{
|
||||
this.variables = variables;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for securityValues.
|
||||
*******************************************************************************/
|
||||
public Map<String, Serializable> getSecurityValues()
|
||||
{
|
||||
return (securityValues);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for securityValues.
|
||||
*******************************************************************************/
|
||||
public EmailRequest withSecurityValues(Map<String, Serializable> securityValues)
|
||||
{
|
||||
this.securityValues = securityValues;
|
||||
return (this);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** A record reference is distinct from an ordinary JSON object.
|
||||
*******************************************************************************/
|
||||
public record RecordVariable(String table, Serializable id)
|
||||
{
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Record bindings, read under the sender session by the service.
|
||||
*******************************************************************************/
|
||||
public Map<String, RecordVariable> getRecordVariables()
|
||||
{
|
||||
return (recordVariables);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.outbox;
|
||||
|
||||
|
||||
import java.util.List;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Queue result. Cross-backend deferred inserts initially have no message id.
|
||||
*******************************************************************************/
|
||||
public record EmailSendResult(Integer messageId, Status status, List<String> reasons)
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Enqueue outcomes.
|
||||
*******************************************************************************/
|
||||
public enum Status
|
||||
{
|
||||
QUEUED,
|
||||
DUPLICATE,
|
||||
SUPPRESSED,
|
||||
REFUSED
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,531 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.outbox;
|
||||
|
||||
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.security.MessageDigest;
|
||||
import java.time.Instant;
|
||||
import java.util.ArrayList;
|
||||
import java.util.HexFormat;
|
||||
import java.util.LinkedHashSet;
|
||||
import java.util.List;
|
||||
import java.util.Locale;
|
||||
import java.util.Set;
|
||||
import com.kingsrook.qbits.email.EmailQBitConfig;
|
||||
import com.kingsrook.qbits.email.EmailRuntime;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTableCustomizers;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTables;
|
||||
import com.kingsrook.qbits.email.permissions.EmailPermissionChecker;
|
||||
import com.kingsrook.qbits.email.permissions.EmailPermissions;
|
||||
import com.kingsrook.qbits.email.support.EmailStore;
|
||||
import com.kingsrook.qbits.email.support.SystemWrites;
|
||||
import com.kingsrook.qbits.email.template.EmailTemplateRenderer;
|
||||
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
|
||||
import com.kingsrook.qqq.backend.core.actions.permissions.CorePermissionEnforcer;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.messaging.SendMessageInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import jakarta.mail.internet.InternetAddress;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Validated transactional outbox API. This class never performs network delivery.
|
||||
*******************************************************************************/
|
||||
public final class EmailService
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private EmailService()
|
||||
{
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Queue validated, rendered alternatives with optional versioned templates.
|
||||
*******************************************************************************/
|
||||
public static EmailSendResult send(EmailRequest request) throws QException
|
||||
{
|
||||
return (send(request, new AdHocEmailRenderer()));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Queue with a host-supplied ad-hoc renderer; templates always use the validated engine.
|
||||
*******************************************************************************/
|
||||
public static EmailSendResult send(EmailRequest request, EmailRenderer renderer) throws QException
|
||||
{
|
||||
return (send(request, renderer, null));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Shared enqueue path, including already protected retained content.
|
||||
*******************************************************************************/
|
||||
private static EmailSendResult send(EmailRequest request, EmailRenderer renderer, QRecord retained) throws QException
|
||||
{
|
||||
EmailQBitConfig config = EmailRuntime.config();
|
||||
if(config == null)
|
||||
{
|
||||
return (refused("Email qbit is not installed"));
|
||||
}
|
||||
if(CorePermissionEnforcer.shouldEnforce(new SendMessageInput()) && !EmailPermissionChecker.has(EmailPermissions.SEND))
|
||||
{
|
||||
return (refused("Sending email needs email.send"));
|
||||
}
|
||||
if(request == null)
|
||||
{
|
||||
return (refused("An email request is required"));
|
||||
}
|
||||
EmailTemplateRenderer templateRenderer;
|
||||
QRecord templateContent;
|
||||
try
|
||||
{
|
||||
templateRenderer = request.getTemplateName() == null ? null : EmailTemplateRenderer.prepare(request);
|
||||
templateContent = templateRenderer == null ? retained : templateRenderer.render("pending", "https://example.invalid/unsubscribe/pending", true);
|
||||
}
|
||||
catch(IllegalArgumentException e)
|
||||
{
|
||||
return (refused(e.getMessage()));
|
||||
}
|
||||
if(templateRenderer == null && (!request.getVariables().isEmpty() || !request.getRecordVariables().isEmpty()))
|
||||
{
|
||||
return (refused("Variables require a template with a schema"));
|
||||
}
|
||||
if(templateRenderer == null && (request.getSubject() == null || request.getSubject().contains("\r") || request.getSubject().contains("\n")
|
||||
|| (request.getMarkdownBody() == null && request.getHtmlBody() == null && request.getTextBody() == null)))
|
||||
{
|
||||
return (refused("A subject and message body are required; subject must be one line"));
|
||||
}
|
||||
EmailCategory category = templateContent == null ? (request.getCategory() == null ? EmailCategory.TRANSACTIONAL : request.getCategory())
|
||||
: EmailCategory.valueOf(templateContent.getValueString("category"));
|
||||
QBackendTransaction transaction = request.getTransaction();
|
||||
boolean sameBackend = transaction == null || request.getTransactionBackendName() == null || config.getBackendName().equals(request.getTransactionBackendName());
|
||||
QBackendTransaction queryTransaction = sameBackend ? transaction : null;
|
||||
QRecord duplicate = duplicate(request.getIdempotencyKey(), queryTransaction);
|
||||
if(duplicate != null)
|
||||
{
|
||||
return (new EmailSendResult(duplicate.getValueInteger("id"), EmailSendResult.Status.DUPLICATE, List.of()));
|
||||
}
|
||||
String providerName = request.getProviderName() == null ? config.getDefaultProviderName() : request.getProviderName();
|
||||
QQueryFilter providerFilter = new QQueryFilter(new QFilterCriteria("enabled", QCriteriaOperator.EQUALS, true));
|
||||
providerFilter.withCriteria(new QFilterCriteria(providerName == null ? "isDefault" : "name", QCriteriaOperator.EQUALS, providerName == null ? true : providerName));
|
||||
List<QRecord> providers = SystemWrites.query(EmailTables.PROVIDER, providerFilter, queryTransaction);
|
||||
if(providers.size() != 1)
|
||||
{
|
||||
return (refused("An enabled named or default provider is required"));
|
||||
}
|
||||
QRecord provider = providers.get(0);
|
||||
List<String> providerErrors = EmailTableCustomizers.validateProvider(provider, config);
|
||||
if(!providerErrors.isEmpty())
|
||||
{
|
||||
return (new EmailSendResult(null, EmailSendResult.Status.REFUSED, providerErrors));
|
||||
}
|
||||
try
|
||||
{
|
||||
if(templateRenderer != null)
|
||||
{
|
||||
templateRenderer.validateProvider(provider);
|
||||
}
|
||||
else if(retained != null && Boolean.TRUE.equals(retained.getValueBoolean("phiApproved")))
|
||||
{
|
||||
EmailTemplateRenderer.checkProvider(provider);
|
||||
com.kingsrook.qbits.email.phi.EmailBodyEncryption.encrypt(null, "subject", "pending");
|
||||
}
|
||||
}
|
||||
catch(IllegalArgumentException e)
|
||||
{
|
||||
return (refused(e.getMessage()));
|
||||
}
|
||||
List<String> to;
|
||||
List<String> cc;
|
||||
List<String> bcc;
|
||||
String from;
|
||||
String replyTo;
|
||||
try
|
||||
{
|
||||
to = addresses(request.getTo());
|
||||
cc = addresses(request.getCc());
|
||||
bcc = addresses(request.getBcc());
|
||||
String sender = request.getFrom() == null ? provider.getValueString("fromAddress") : request.getFrom();
|
||||
from = single(sender);
|
||||
if(request.getFrom() == null && provider.getValueString("fromName") != null)
|
||||
{
|
||||
from = new InternetAddress(new InternetAddress(from).getAddress(), provider.getValueString("fromName"), "UTF-8").toString();
|
||||
}
|
||||
String reply = request.getReplyTo() == null ? provider.getValueString("replyTo") : request.getReplyTo();
|
||||
replyTo = reply == null ? null : single(reply);
|
||||
String domain = provider.getValueString("sendingDomain");
|
||||
String address = new InternetAddress(from).getAddress();
|
||||
if(domain != null && !domain.equalsIgnoreCase(address.substring(address.lastIndexOf('@') + 1)))
|
||||
{
|
||||
return (refused("From address domain must match sending domain"));
|
||||
}
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
return (refused("One or more email addresses are invalid"));
|
||||
}
|
||||
if(to.isEmpty() && cc.isEmpty() && bcc.isEmpty())
|
||||
{
|
||||
return (refused("At least one recipient is required"));
|
||||
}
|
||||
Set<String> blocked = suppressed(category, queryTransaction);
|
||||
List<String> dropped = new ArrayList<>();
|
||||
for(List<String> addresses : List.of(to, cc, bcc))
|
||||
{
|
||||
addresses.removeIf(address ->
|
||||
{
|
||||
try
|
||||
{
|
||||
if(blocked.contains(new InternetAddress(address).getAddress().toLowerCase(Locale.ROOT)))
|
||||
{
|
||||
dropped.add(address);
|
||||
return (true);
|
||||
}
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
// Addresses were validated above.
|
||||
}
|
||||
return (false);
|
||||
});
|
||||
}
|
||||
boolean allSuppressed = to.isEmpty() && cc.isEmpty() && bcc.isEmpty();
|
||||
EmailRenderer.Rendered rendered;
|
||||
try
|
||||
{
|
||||
rendered = templateContent == null ? renderer.render(request) : new EmailRenderer.Rendered(templateContent.getValueString("htmlBody"), templateContent.getValueString("textBody"));
|
||||
}
|
||||
catch(IllegalArgumentException e)
|
||||
{
|
||||
return (refused(e.getMessage()));
|
||||
}
|
||||
Instant now = EmailRuntime.clock().instant();
|
||||
QRecord row = new QRecord().withValue("providerId", provider.getValue("id"))
|
||||
.withValue("status", allSuppressed ? "SUPPRESSED" : "QUEUED").withValue("finishedDate", allSuppressed ? now : null).withValue("attemptCount", 0)
|
||||
.withValue("toAddresses", String.join(",", to)).withValue("ccAddresses", String.join(",", cc)).withValue("bccAddresses", String.join(",", bcc))
|
||||
.withValue("fromAddress", from).withValue("replyTo", replyTo).withValue("subject", request.getSubject())
|
||||
.withValue("htmlBody", rendered.html()).withValue("textBody", rendered.text()).withValue("bodySha256", digest(rendered))
|
||||
.withValue("category", category.name()).withValue("locale", request.getLocale()).withValue("idempotencyKey", request.getIdempotencyKey())
|
||||
.withValue("linkedTableName", request.getLinkedTableName()).withValue("linkedRecordId", request.getLinkedRecordId())
|
||||
.withValue("senderQbit", request.getSenderQbit()).withValue("senderProcess", request.getSenderProcess())
|
||||
.withValue("sendAfter", request.getSendAfter()).withValue("nextAttemptDate", request.getSendAfter() == null ? now : request.getSendAfter())
|
||||
.withValue("suppressedAddresses", String.join(",", dropped));
|
||||
if(templateContent != null)
|
||||
{
|
||||
row.getValues().putAll(templateContent.getValues());
|
||||
}
|
||||
try
|
||||
{
|
||||
com.kingsrook.qbits.email.render.EmailUnsubscribeLinks.create("pending", String.join(",", to), category.name());
|
||||
}
|
||||
catch(IllegalArgumentException e)
|
||||
{
|
||||
return (refused(e.getMessage()));
|
||||
}
|
||||
for(var field : config.getSecurityFields())
|
||||
{
|
||||
row.setValue(field.getName(), request.getSecurityValues().get(field.getName()));
|
||||
}
|
||||
if(transaction != null && !sameBackend)
|
||||
{
|
||||
transaction.addAfterCommitCallback(() ->
|
||||
{
|
||||
try
|
||||
{
|
||||
if(insert(row, null, templateRenderer).status() == EmailSendResult.Status.REFUSED)
|
||||
{
|
||||
throw (new QException("Email after-commit enqueue failed"));
|
||||
}
|
||||
}
|
||||
catch(QException e)
|
||||
{
|
||||
// Core logs callback failures; propagate only a sanitized message.
|
||||
throw (new IllegalStateException("Email after-commit enqueue failed"));
|
||||
}
|
||||
});
|
||||
return (new EmailSendResult(null, allSuppressed ? EmailSendResult.Status.SUPPRESSED : EmailSendResult.Status.QUEUED,
|
||||
List.of("Enqueue deferred until caller transaction commits")));
|
||||
}
|
||||
return (insert(row, transaction, templateRenderer));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Resend retained content without decrypting it or downgrading PHI policy.
|
||||
** Snapshot tokens keep the original binding; the new row links to that row.
|
||||
*******************************************************************************/
|
||||
public static EmailSendResult resend(Integer id) throws QException
|
||||
{
|
||||
if(!SystemWrites.active() && !EmailPermissionChecker.has(EmailPermissions.MANAGE))
|
||||
{
|
||||
return (refused("Resending email needs email.manage"));
|
||||
}
|
||||
QRecord row = EmailStore.get(EmailTables.MESSAGE, id, null);
|
||||
if(row == null || row.getValueString("htmlBody") == null && row.getValueString("textBody") == null)
|
||||
{
|
||||
return (refused("Retained message content is unavailable"));
|
||||
}
|
||||
QRecord provider = EmailStore.get(EmailTables.PROVIDER, row.getValueInteger("providerId"), null);
|
||||
if(provider == null)
|
||||
{
|
||||
return (refused("Provider is unavailable"));
|
||||
}
|
||||
EmailRequest request = new EmailRequest().withSubject(row.getValueString("subject")).withHtmlBody(row.getValueString("htmlBody"))
|
||||
.withTextBody(row.getValueString("textBody")).withFrom(row.getValueString("fromAddress")).withReplyTo(row.getValueString("replyTo"))
|
||||
.withProviderName(provider.getValueString("name")).withCategory(EmailCategory.valueOf(row.getValueString("category")))
|
||||
.withLinkedRecord(EmailTables.MESSAGE, id).withSender("email", "email_resend");
|
||||
if(row.getValueString("toAddresses") != null && !row.getValueString("toAddresses").isBlank())
|
||||
{
|
||||
request.withTo(row.getValueString("toAddresses"));
|
||||
}
|
||||
if(row.getValueString("ccAddresses") != null && !row.getValueString("ccAddresses").isBlank())
|
||||
{
|
||||
request.withCc(row.getValueString("ccAddresses"));
|
||||
}
|
||||
if(row.getValueString("bccAddresses") != null && !row.getValueString("bccAddresses").isBlank())
|
||||
{
|
||||
request.withBcc(row.getValueString("bccAddresses"));
|
||||
}
|
||||
for(var field : EmailRuntime.config().getSecurityFields())
|
||||
{
|
||||
request.getSecurityValues().put(field.getName(), row.getValue(field.getName()));
|
||||
}
|
||||
QRecord content = new QRecord();
|
||||
for(String field : List.of("subject", "htmlBody", "textBody", "variablesSnapshot", "phiApproved", "templateId", "templateVersionId", "brandId", "locale", "category", "unsubscribeUrl"))
|
||||
{
|
||||
content.setValue(field, row.getValue(field));
|
||||
}
|
||||
content.withValue("encryptionSourceId", row.getValueString("id"));
|
||||
return (send(request, ignored -> new EmailRenderer.Rendered(row.getValueString("htmlBody"), row.getValueString("textBody")), content));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Insert once; the database unique key arbitrates concurrent requests.
|
||||
*******************************************************************************/
|
||||
private static EmailSendResult insert(QRecord row, QBackendTransaction caller, EmailTemplateRenderer templateRenderer) throws QException
|
||||
{
|
||||
if(caller != null)
|
||||
{
|
||||
return (insertInTransaction(row, caller, templateRenderer));
|
||||
}
|
||||
try(QBackendTransaction transaction = QBackendTransaction.openFor(new InsertInput(EmailTables.MESSAGE)))
|
||||
{
|
||||
EmailSendResult result = insertInTransaction(row, transaction, templateRenderer);
|
||||
if(result.status() == EmailSendResult.Status.REFUSED)
|
||||
{
|
||||
transaction.rollback();
|
||||
}
|
||||
else
|
||||
{
|
||||
transaction.commit();
|
||||
return (result);
|
||||
}
|
||||
}
|
||||
QRecord existing = duplicate(row.getValueString("idempotencyKey"), null);
|
||||
return (existing == null ? refused("Could not queue the email message") : new EmailSendResult(existing.getValueInteger("id"), EmailSendResult.Status.DUPLICATE, List.of()));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Insert using the caller transaction, returning only safe diagnostics.
|
||||
*******************************************************************************/
|
||||
private static EmailSendResult insertInTransaction(QRecord row, QBackendTransaction transaction, EmailTemplateRenderer templateRenderer) throws QException
|
||||
{
|
||||
// A shared provider-row gate prevents a duplicate-key failure from aborting
|
||||
// a caller-owned SQL transaction. The unique index remains the final arbiter.
|
||||
if(row.getValueString("idempotencyKey") != null)
|
||||
{
|
||||
List<QRecord> providers = SystemWrites.query(EmailTables.PROVIDER, new QQueryFilter().withOrderBy(new QFilterOrderBy("id")).withLimit(1), transaction);
|
||||
if(!providers.isEmpty())
|
||||
{
|
||||
EmailStore.lockProvider(providers.get(0).getValueInteger("id"), transaction);
|
||||
}
|
||||
}
|
||||
QRecord existing = duplicate(row.getValueString("idempotencyKey"), transaction);
|
||||
if(existing != null)
|
||||
{
|
||||
return (new EmailSendResult(existing.getValueInteger("id"), EmailSendResult.Status.DUPLICATE, List.of()));
|
||||
}
|
||||
QRecord inserted;
|
||||
try
|
||||
{
|
||||
inserted = SystemWrites.insert(EmailTables.MESSAGE, List.of(row), transaction).getRecords().get(0);
|
||||
}
|
||||
catch(QException e)
|
||||
{
|
||||
return (refused("Could not queue the email message"));
|
||||
}
|
||||
if(inserted.getErrors() != null && !inserted.getErrors().isEmpty())
|
||||
{
|
||||
return (refused("Could not queue the email message"));
|
||||
}
|
||||
try
|
||||
{
|
||||
String unsubscribe = row.getValueString("unsubscribeUrl");
|
||||
if(unsubscribe == null && (templateRenderer != null || !Boolean.TRUE.equals(row.getValueBoolean("phiApproved"))))
|
||||
{
|
||||
unsubscribe = com.kingsrook.qbits.email.render.EmailUnsubscribeLinks.create(inserted.getValueString("id"),
|
||||
row.getValueString("toAddresses") + "," + row.getValueString("ccAddresses") + "," + row.getValueString("bccAddresses"), row.getValueString("category"));
|
||||
}
|
||||
QRecord content;
|
||||
if(templateRenderer != null)
|
||||
{
|
||||
content = templateRenderer.render(inserted.getValueString("id"), unsubscribe, true);
|
||||
}
|
||||
else
|
||||
{
|
||||
content = new QRecord();
|
||||
if(Boolean.TRUE.equals(row.getValueBoolean("phiApproved")))
|
||||
{
|
||||
for(String field : List.of("subject", "htmlBody", "textBody"))
|
||||
{
|
||||
content.setValue(field, com.kingsrook.qbits.email.phi.EmailBodyEncryption.rebind(row.getValueString(field), field,
|
||||
row.getValueString("encryptionSourceId"), inserted.getValueString("id")));
|
||||
}
|
||||
}
|
||||
if(unsubscribe != null && row.getValueString("unsubscribeUrl") == null)
|
||||
{
|
||||
String link = "<p><a href=\"" + com.kingsrook.qbits.email.template.EmailTemplateEngine.escape(unsubscribe) + "\">Unsubscribe</a></p>";
|
||||
String html = row.getValueString("htmlBody");
|
||||
java.util.regex.Matcher end = java.util.regex.Pattern.compile("</footer>|</body>", java.util.regex.Pattern.CASE_INSENSITIVE).matcher(html);
|
||||
content.withValue("htmlBody", end.find() ? html.substring(0, end.start()) + link + html.substring(end.start()) : html + link)
|
||||
.withValue("textBody", row.getValueString("textBody") + "\nUnsubscribe (" + unsubscribe + ")");
|
||||
}
|
||||
}
|
||||
content.withValue("id", inserted.getValue("id")).withValue("unsubscribeUrl", unsubscribe);
|
||||
if(content.getValueString("htmlBody") != null)
|
||||
{
|
||||
content.withValue("bodySha256", digest(new EmailRenderer.Rendered(content.getValueString("htmlBody"), content.getValueString("textBody"))));
|
||||
}
|
||||
EmailStore.update(EmailTables.MESSAGE, content, transaction);
|
||||
}
|
||||
catch(IllegalArgumentException e)
|
||||
{
|
||||
SystemWrites.delete(EmailTables.MESSAGE, EmailStore.equal("id", inserted.getValueInteger("id")), transaction);
|
||||
return (refused(e.getMessage()));
|
||||
}
|
||||
return (new EmailSendResult(inserted.getValueInteger("id"), "SUPPRESSED".equals(row.getValueString("status"))
|
||||
? EmailSendResult.Status.SUPPRESSED : EmailSendResult.Status.QUEUED, templateRenderer == null ? List.of() : templateRenderer.warnings()));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Look up an idempotency key in the appropriate transaction.
|
||||
*******************************************************************************/
|
||||
private static QRecord duplicate(String key, QBackendTransaction transaction) throws QException
|
||||
{
|
||||
if(key == null)
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
List<QRecord> rows = SystemWrites.query(EmailTables.MESSAGE, new QQueryFilter(new QFilterCriteria("idempotencyKey", QCriteriaOperator.EQUALS, key)), transaction);
|
||||
return (rows.isEmpty() ? null : rows.get(0));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Suppression matching; opt-outs never suppress transactional or security mail.
|
||||
*******************************************************************************/
|
||||
private static Set<String> suppressed(EmailCategory category, QBackendTransaction transaction) throws QException
|
||||
{
|
||||
Set<String> result = new LinkedHashSet<>();
|
||||
boolean optOut = category == EmailCategory.NOTIFICATION || category == EmailCategory.MARKETING;
|
||||
for(QRecord row : SystemWrites.query(EmailTables.SUPPRESSION, new QQueryFilter(), transaction))
|
||||
{
|
||||
Instant expires = row.getValueInstant("expiresDate");
|
||||
if(expires != null && !expires.isAfter(EmailRuntime.clock().instant()))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
if(!optOut && "UNSUBSCRIBE".equals(row.getValueString("reason")))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
if("ALL".equals(row.getValueString("scope")) || (optOut && category.name().equals(row.getValueString("scope"))))
|
||||
{
|
||||
result.add(row.getValueString("address").toLowerCase(Locale.ROOT));
|
||||
}
|
||||
}
|
||||
return (result);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Strict address parsing, rejecting header injection and local-only addresses.
|
||||
*******************************************************************************/
|
||||
public static List<String> addresses(List<String> input) throws Exception
|
||||
{
|
||||
List<String> result = new ArrayList<>();
|
||||
for(String value : input)
|
||||
{
|
||||
if(value == null || value.contains("\r") || value.contains("\n"))
|
||||
{
|
||||
throw (new IllegalArgumentException("Invalid address"));
|
||||
}
|
||||
for(InternetAddress address : InternetAddress.parse(value, true))
|
||||
{
|
||||
address.validate();
|
||||
if(address.isGroup() || !address.getAddress().contains("@"))
|
||||
{
|
||||
throw (new IllegalArgumentException("Invalid address"));
|
||||
}
|
||||
result.add(address.toString());
|
||||
}
|
||||
}
|
||||
return (result);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** One sender or reply-to mailbox.
|
||||
*******************************************************************************/
|
||||
private static String single(String value) throws Exception
|
||||
{
|
||||
List<String> addresses = addresses(List.of(value));
|
||||
if(addresses.size() != 1)
|
||||
{
|
||||
throw (new IllegalArgumentException("One address required"));
|
||||
}
|
||||
return (addresses.get(0));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Keep a digest after body retention clears both alternatives.
|
||||
*******************************************************************************/
|
||||
private static String digest(EmailRenderer.Rendered rendered)
|
||||
{
|
||||
try
|
||||
{
|
||||
return (HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest((rendered.html() + "\u0000" + rendered.text()).getBytes(StandardCharsets.UTF_8))));
|
||||
}
|
||||
catch(java.security.NoSuchAlgorithmException e)
|
||||
{
|
||||
throw (new IllegalStateException("SHA-256 unavailable"));
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Refusal with no stored row.
|
||||
*******************************************************************************/
|
||||
private static EmailSendResult refused(String reason)
|
||||
{
|
||||
return (new EmailSendResult(null, EmailSendResult.Status.REFUSED, List.of(reason)));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,128 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.permissions;
|
||||
|
||||
|
||||
import com.kingsrook.qbits.email.metadata.EmailTables;
|
||||
import com.kingsrook.qbits.email.processes.EmailProcesses;
|
||||
import com.kingsrook.qqq.backend.core.actions.permissions.CustomPermissionChecker;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.permissions.MetaDataWithPermissionRules;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.session.QSession;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Maps every action on the qbit's tables, processes, widgets and app to one
|
||||
** of {@code email.view}, {@code email.manage} or {@code email.admin}
|
||||
** (admin implies manage, manage implies view).
|
||||
**
|
||||
** <p>Deliveries and attempts are written only by the qbit (its own writes run
|
||||
** in an authorized scope); people and the API can read them and act on them
|
||||
** through the processes.</p>
|
||||
*******************************************************************************/
|
||||
public class EmailPermissionChecker implements CustomPermissionChecker
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Check, or throw.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void checkPermissionsThrowing(AbstractActionInput actionInput, MetaDataWithPermissionRules metaData) throws QPermissionDeniedException
|
||||
{
|
||||
String needed = required(actionInput, metaData);
|
||||
if(needed == null)
|
||||
{
|
||||
throw (new QPermissionDeniedException("Email messages and attempts are written only by the email qbit. Use Send Now, Resend or Cancel."));
|
||||
}
|
||||
if(!has(needed))
|
||||
{
|
||||
throw (new QPermissionDeniedException("This needs the " + needed + " permission."));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The permission an action needs (null: nobody may do it directly).
|
||||
*******************************************************************************/
|
||||
static String required(AbstractActionInput actionInput, MetaDataWithPermissionRules metaData)
|
||||
{
|
||||
boolean write = actionInput instanceof InsertInput || actionInput instanceof UpdateInput || actionInput instanceof DeleteInput;
|
||||
if(metaData instanceof QTableMetaData table)
|
||||
{
|
||||
if(EmailTables.PROVIDER.equals(table.getName()))
|
||||
{
|
||||
return (EmailPermissions.ADMIN);
|
||||
}
|
||||
if(!write)
|
||||
{
|
||||
return (EmailPermissions.VIEW);
|
||||
}
|
||||
if(EmailTables.SYSTEM_WRITTEN.contains(table.getName()))
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
if(EmailTables.PROVIDER.equals(table.getName()))
|
||||
{
|
||||
return (EmailPermissions.ADMIN);
|
||||
}
|
||||
return (EmailPermissions.MANAGE);
|
||||
}
|
||||
if(metaData instanceof QProcessMetaData process)
|
||||
{
|
||||
return (EmailProcesses.permissionFor(process.getName()));
|
||||
}
|
||||
return (EmailPermissions.VIEW);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** True when the current session holds the permission (or a higher one).
|
||||
*******************************************************************************/
|
||||
public static boolean has(String permission)
|
||||
{
|
||||
QSession session = QContext.getQSession();
|
||||
if(session == null)
|
||||
{
|
||||
return (false);
|
||||
}
|
||||
if(session.hasPermission(EmailPermissions.ADMIN))
|
||||
{
|
||||
return (true);
|
||||
}
|
||||
if(EmailPermissions.ADMIN.equals(permission))
|
||||
{
|
||||
return (false);
|
||||
}
|
||||
if(session.hasPermission(EmailPermissions.MANAGE))
|
||||
{
|
||||
return (true);
|
||||
}
|
||||
return (session.hasPermission(permission));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.permissions;
|
||||
|
||||
|
||||
import java.util.List;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Email permissions: view, send, manage and admin, plus the derived app access.
|
||||
** Administrators imply all permissions; managers may also send and view.
|
||||
*******************************************************************************/
|
||||
public final class EmailPermissions
|
||||
{
|
||||
public static final String SEND = "email.send";
|
||||
public static final String VIEW = "email.view";
|
||||
public static final String MANAGE = "email.manage";
|
||||
public static final String ADMIN = "email.admin";
|
||||
public static final String HAS_ACCESS = "email.hasAccess";
|
||||
|
||||
public static final List<String> ALL = List.of(VIEW, SEND, MANAGE, ADMIN, HAS_ACCESS);
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private EmailPermissions()
|
||||
{
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,137 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.phi;
|
||||
|
||||
|
||||
import javax.crypto.Cipher;
|
||||
import javax.crypto.spec.GCMParameterSpec;
|
||||
import javax.crypto.spec.SecretKeySpec;
|
||||
import java.nio.ByteBuffer;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.security.MessageDigest;
|
||||
import java.security.SecureRandom;
|
||||
import java.util.Base64;
|
||||
import com.kingsrook.qbits.email.EmailRuntime;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Stateless AES-256-GCM body protection. Only the dispatcher decrypts content.
|
||||
*******************************************************************************/
|
||||
public final class EmailBodyEncryption
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private EmailBodyEncryption()
|
||||
{
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Resolve the deployment key, never retaining it between operations.
|
||||
*******************************************************************************/
|
||||
private static SecretKeySpec key() throws Exception
|
||||
{
|
||||
String secret = EmailRuntime.config().getSecretResolver().apply("QQQ_EMAIL_BODY_KEY");
|
||||
if(secret == null || secret.isBlank())
|
||||
{
|
||||
throw (new IllegalArgumentException("PHI email needs QQQ_EMAIL_BODY_KEY"));
|
||||
}
|
||||
return (new SecretKeySpec(MessageDigest.getInstance("SHA-256").digest(secret.getBytes(StandardCharsets.UTF_8)), "AES"));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Encrypt with a fresh 96-bit nonce and 128-bit authentication tag.
|
||||
*******************************************************************************/
|
||||
public static String encrypt(String value, String column, String messageId)
|
||||
{
|
||||
try
|
||||
{
|
||||
SecretKeySpec key = key();
|
||||
if(value == null)
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
byte[] iv = new byte[12];
|
||||
new SecureRandom().nextBytes(iv);
|
||||
Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
|
||||
cipher.init(Cipher.ENCRYPT_MODE, key, new GCMParameterSpec(128, iv));
|
||||
cipher.updateAAD(aad(column, messageId));
|
||||
byte[] encrypted = cipher.doFinal(value.getBytes(StandardCharsets.UTF_8));
|
||||
return ("enc:v1:" + Base64.getUrlEncoder().withoutPadding().encodeToString(ByteBuffer.allocate(iv.length + encrypted.length).put(iv).put(encrypted).array()));
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
throw (new IllegalArgumentException("PHI email needs QQQ_EMAIL_BODY_KEY"));
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Authenticate before exposing plaintext. Failures never echo ciphertext.
|
||||
*******************************************************************************/
|
||||
public static String decrypt(String value, String column, String messageId)
|
||||
{
|
||||
if(value == null || !value.startsWith("enc:v1:"))
|
||||
{
|
||||
return (value);
|
||||
}
|
||||
try
|
||||
{
|
||||
ByteBuffer bytes = ByteBuffer.wrap(Base64.getUrlDecoder().decode(value.substring(7)));
|
||||
byte[] iv = new byte[12];
|
||||
bytes.get(iv);
|
||||
byte[] encrypted = new byte[bytes.remaining()];
|
||||
bytes.get(encrypted);
|
||||
Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
|
||||
cipher.init(Cipher.DECRYPT_MODE, key(), new GCMParameterSpec(128, iv));
|
||||
cipher.updateAAD(aad(column, messageId));
|
||||
return (new String(cipher.doFinal(encrypted), StandardCharsets.UTF_8));
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
throw (new IllegalArgumentException("Encrypted email body could not be opened"));
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Bind ciphertext to its storage table, column and message identity.
|
||||
*******************************************************************************/
|
||||
private static byte[] aad(String column, String messageId)
|
||||
{
|
||||
String storageColumn = switch(column)
|
||||
{
|
||||
case "subject" -> "subject";
|
||||
case "htmlBody" -> "html_body";
|
||||
case "textBody" -> "text_body";
|
||||
default -> throw (new IllegalArgumentException("Unknown protected email column"));
|
||||
};
|
||||
return (("email_message\n" + storageColumn + "\n" + java.util.Objects.requireNonNull(messageId)).getBytes(StandardCharsets.UTF_8));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Authenticate and rebind retained ciphertext without exposing plaintext to
|
||||
** the resend caller. Only delivery callers receive decrypted content.
|
||||
*******************************************************************************/
|
||||
public static String rebind(String value, String column, String oldMessageId, String newMessageId)
|
||||
{
|
||||
return (encrypt(decrypt(value, column, oldMessageId), column, newMessageId));
|
||||
}
|
||||
|
||||
}
|
||||
@@ -0,0 +1,109 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.phi;
|
||||
|
||||
|
||||
import com.kingsrook.qbits.email.EmailQBitConfig;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** STAND-IN(core=#1142, removeBy=RC.7). Core classification wins when available;
|
||||
** overrides then schema declarations provide the RC.5 compatibility contract.
|
||||
*******************************************************************************/
|
||||
public final class EmailClassifications
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Ordered data sensitivity.
|
||||
*******************************************************************************/
|
||||
public enum Level
|
||||
{
|
||||
PUBLIC, CONFIDENTIAL, PII, PHI
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private EmailClassifications()
|
||||
{
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Resolve a field without a compile-time dependency on unreleased core.
|
||||
*******************************************************************************/
|
||||
public static Level ofField(QTableMetaData table, String field, String declared, EmailQBitConfig config)
|
||||
{
|
||||
if(table != null && field != null)
|
||||
{
|
||||
try
|
||||
{
|
||||
Class<?> core = Class.forName("com.kingsrook.qqq.backend.core.actions.ai.AiDataClassifications");
|
||||
Object value = core.getMethod("ofField", QTableMetaData.class, String.class).invoke(null, table, field);
|
||||
if(value instanceof Enum<?> level)
|
||||
{
|
||||
return (Level.valueOf(level.name()));
|
||||
}
|
||||
}
|
||||
catch(ClassNotFoundException e)
|
||||
{
|
||||
// Expected on RC.5; use the explicit stand-in configuration.
|
||||
}
|
||||
catch(ReflectiveOperationException | IllegalArgumentException e)
|
||||
{
|
||||
throw (new IllegalArgumentException("Field classification is unavailable"));
|
||||
}
|
||||
Level override = config.getClassificationOverrides().get(table.getName() + "." + field);
|
||||
if(override != null)
|
||||
{
|
||||
return (override);
|
||||
}
|
||||
}
|
||||
return (declared == null ? Level.PUBLIC : Level.valueOf(declared));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Schema declarations alone cannot attest a record's classification source.
|
||||
*******************************************************************************/
|
||||
public static boolean hasSource(QTableMetaData table, String field, EmailQBitConfig config)
|
||||
{
|
||||
if(config.getClassificationOverrides().containsKey(table.getName() + "." + field))
|
||||
{
|
||||
return (true);
|
||||
}
|
||||
try
|
||||
{
|
||||
Class<?> core = Class.forName("com.kingsrook.qqq.backend.core.actions.ai.AiDataClassifications");
|
||||
return (core.getMethod("ofField", QTableMetaData.class, String.class).invoke(null, table, field) instanceof Enum<?>);
|
||||
}
|
||||
catch(ReflectiveOperationException e)
|
||||
{
|
||||
return (false);
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** PII has the same email approval requirement as PHI.
|
||||
*******************************************************************************/
|
||||
public static boolean sensitive(Level level)
|
||||
{
|
||||
return (level == Level.PII || level == Level.PHI);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,149 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.phi;
|
||||
|
||||
|
||||
import javax.crypto.Mac;
|
||||
import javax.crypto.spec.SecretKeySpec;
|
||||
import java.io.Serializable;
|
||||
import java.math.BigDecimal;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.security.MessageDigest;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Base64;
|
||||
import java.util.List;
|
||||
import com.kingsrook.qbits.email.EmailRuntime;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
|
||||
import com.kingsrook.qqq.backend.core.utils.ValueUtils;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** STAND-IN(core=#1161-impl, removeBy=RC.7). Mirrors ClassifiedValueTokens.
|
||||
** Tokens are comparisons, never recoverable snapshots. No secret is retained.
|
||||
*******************************************************************************/
|
||||
public final class EmailValueTokens
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private EmailValueTokens()
|
||||
{
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Bind to table, field and record identity. TODO core: wipe-rule record
|
||||
** fields will produce shred:v1; context values must never use shred:v1.
|
||||
*******************************************************************************/
|
||||
public static String forRecordField(QTableMetaData table, QFieldMetaData field, Serializable pk, Serializable value)
|
||||
{
|
||||
return (token(List.of(table.getName(), field.getName(), String.valueOf(pk)), value));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Non-record low-entropy inputs cannot yield dictionary-testable tokens.
|
||||
*******************************************************************************/
|
||||
public static String forContext(List<String> bindingParts, Serializable value)
|
||||
{
|
||||
if(value instanceof Boolean || canonical(value).codePoints().distinct().count() < 4)
|
||||
{
|
||||
return (changed());
|
||||
}
|
||||
return (token(bindingParts, value));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Deliberately non-verifiable marker.
|
||||
*******************************************************************************/
|
||||
public static String changed()
|
||||
{
|
||||
return ("changed:v1");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Verify only a genuine keyed token, with constant-time comparison.
|
||||
*******************************************************************************/
|
||||
public static boolean verifyRecordField(QTableMetaData table, QFieldMetaData field, Serializable pk, Serializable value, String token)
|
||||
{
|
||||
return (matches(token, forRecordField(table, field, pk, value)));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Verify a context token, never changed or shred markers.
|
||||
*******************************************************************************/
|
||||
public static boolean verifyContext(List<String> bindingParts, Serializable value, String token)
|
||||
{
|
||||
return (matches(token, forContext(bindingParts, value)));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Stable canonical representation shared with future core.
|
||||
*******************************************************************************/
|
||||
public static String canonical(Serializable value)
|
||||
{
|
||||
if(value instanceof BigDecimal decimal)
|
||||
{
|
||||
return (decimal.stripTrailingZeros().toPlainString());
|
||||
}
|
||||
return (ValueUtils.getValueAsString(value));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Construct the versioned HMAC without logging key or input.
|
||||
*******************************************************************************/
|
||||
private static String token(List<String> parts, Serializable value)
|
||||
{
|
||||
try
|
||||
{
|
||||
String secret = EmailRuntime.config().getSecretResolver().apply("QQQ_AUDIT_HMAC_KEY");
|
||||
if(secret == null || secret.isBlank())
|
||||
{
|
||||
return (changed());
|
||||
}
|
||||
byte[] key = Base64.getDecoder().decode(secret);
|
||||
String keyId = EmailRuntime.config().getSecretResolver().apply("QQQ_AUDIT_HMAC_KEY_ID");
|
||||
if(keyId == null || keyId.isBlank())
|
||||
{
|
||||
keyId = "env1";
|
||||
}
|
||||
Mac mac = Mac.getInstance("HmacSHA256");
|
||||
mac.init(new SecretKeySpec(key, "HmacSHA256"));
|
||||
List<String> binding = new ArrayList<>(parts);
|
||||
binding.add(canonical(value));
|
||||
return ("hmac:v1:" + keyId + ":" + Base64.getUrlEncoder().withoutPadding()
|
||||
.encodeToString(mac.doFinal(String.join("\u001f", binding).getBytes(StandardCharsets.UTF_8))));
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
// An unavailable or malformed audit key must not reveal the value.
|
||||
return (changed());
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Compare only HMAC tokens, never sentinel strings.
|
||||
*******************************************************************************/
|
||||
private static boolean matches(String supplied, String computed)
|
||||
{
|
||||
return (supplied != null && supplied.startsWith("hmac:v1:") && computed.startsWith("hmac:v1:")
|
||||
&& MessageDigest.isEqual(supplied.getBytes(StandardCharsets.UTF_8), computed.getBytes(StandardCharsets.UTF_8)));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,482 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.processes;
|
||||
|
||||
|
||||
import java.io.Serializable;
|
||||
import java.time.Duration;
|
||||
import java.time.Instant;
|
||||
import java.util.ArrayList;
|
||||
import java.util.HashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import com.kingsrook.qbits.email.EmailQBitConfig;
|
||||
import com.kingsrook.qbits.email.EmailRuntime;
|
||||
import com.kingsrook.qbits.email.delivery.EmailDispatcher;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTables;
|
||||
import com.kingsrook.qbits.email.outbox.EmailCategory;
|
||||
import com.kingsrook.qbits.email.outbox.EmailRequest;
|
||||
import com.kingsrook.qbits.email.outbox.EmailSendResult;
|
||||
import com.kingsrook.qbits.email.outbox.EmailService;
|
||||
import com.kingsrook.qbits.email.permissions.EmailPermissions;
|
||||
import com.kingsrook.qbits.email.support.EmailStore;
|
||||
import com.kingsrook.qbits.email.support.SystemWrites;
|
||||
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
|
||||
import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.processes.QComponentType;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendComponentMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendStepMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.scheduleing.QScheduleMetaData;
|
||||
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractLoadStep;
|
||||
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractTransformStep;
|
||||
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ExtractViaQueryStep;
|
||||
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ProcessSummaryProviderInterface;
|
||||
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Standard Review/Result record actions and hidden scheduled maintenance.
|
||||
*******************************************************************************/
|
||||
public final class EmailProcesses
|
||||
{
|
||||
public static final String NEW_VERSION = "email_new_version";
|
||||
public static final String ACTIVATE_VERSION = "email_activate_version";
|
||||
public static final String APPROVE_PHI = "email_approve_phi";
|
||||
public static final String RESEND = "email_resend";
|
||||
public static final String CANCEL = "email_cancel";
|
||||
public static final String SEND_NOW = "email_send_now";
|
||||
public static final String SEND_TEST = "email_send_test";
|
||||
public static final String SEND_AD_HOC = "email_send_ad_hoc";
|
||||
public static final String SEND_TEMPLATED = "email_send_templated";
|
||||
public static final String DISPATCH = "email_dispatch";
|
||||
public static final String MAINTAIN = "email_maintain";
|
||||
public static final String RETENTION = "email_retention";
|
||||
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private EmailProcesses()
|
||||
{
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Required action permission.
|
||||
*******************************************************************************/
|
||||
public static String permissionFor(String name)
|
||||
{
|
||||
return (switch(name)
|
||||
{
|
||||
case RESEND, CANCEL, SEND_NOW, NEW_VERSION, ACTIVATE_VERSION -> EmailPermissions.MANAGE;
|
||||
case SEND_TEMPLATED, SEND_AD_HOC -> EmailPermissions.SEND;
|
||||
default -> EmailPermissions.ADMIN;
|
||||
});
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** All actions and schedules; none is an app child.
|
||||
*******************************************************************************/
|
||||
public static List<QProcessMetaData> processes(EmailQBitConfig config)
|
||||
{
|
||||
QProcessMetaData test = action(SEND_TEST, "Send Test Email", EmailTables.PROVIDER);
|
||||
test.withStep(0, new QFrontendStepMetaData().withName("input").withLabel("Recipient")
|
||||
.withComponent(new QFrontendComponentMetaData().withType(QComponentType.EDIT_FORM))
|
||||
.withFormField(new QFieldMetaData("toAddress", QFieldType.STRING).withLabel("To address").withIsRequired(true)));
|
||||
QFunctionInputMetaData inputs = new QFunctionInputMetaData();
|
||||
QFunctionInputMetaData adHocInputs = new QFunctionInputMetaData();
|
||||
for(String field : List.of("templateName", "to", "cc", "bcc", "replyTo", "subject", "markdownBody", "htmlBody", "textBody", "providerName", "brandName",
|
||||
"locale", "variablesJson", "recordVariablesJson", "linkedTableName", "linkedRecordId", "idempotencyKey"))
|
||||
{
|
||||
inputs.withField(new QFieldMetaData(field, QFieldType.STRING));
|
||||
adHocInputs.withField(new QFieldMetaData(field, QFieldType.STRING));
|
||||
}
|
||||
adHocInputs.withField(new QFieldMetaData("category", QFieldType.STRING));
|
||||
QProcessMetaData adHoc = new QProcessMetaData().withName(SEND_AD_HOC).withLabel("Send Ad Hoc Email").withIsHidden(true)
|
||||
.withPermissionRules(EmailTables.permissionRules())
|
||||
.withStep(new QBackendStepMetaData().withName("send").withCode(new QCodeReference(SendStep.class)).withInputData(adHocInputs));
|
||||
QProcessMetaData send = new QProcessMetaData().withName(SEND_TEMPLATED).withLabel("Send Email").withIsHidden(true)
|
||||
.withPermissionRules(EmailTables.permissionRules())
|
||||
.withStep(new QBackendStepMetaData().withName("send").withCode(new QCodeReference(SendStep.class)).withInputData(inputs));
|
||||
return (List.of(action(RESEND, "Resend", EmailTables.MESSAGE), action(CANCEL, "Cancel", EmailTables.MESSAGE),
|
||||
action(SEND_NOW, "Send Now", EmailTables.MESSAGE), test, send, adHoc,
|
||||
action(NEW_VERSION, "New Version", EmailTables.TEMPLATE), action(ACTIVATE_VERSION, "Activate Version", EmailTables.VERSION),
|
||||
action(APPROVE_PHI, "Approve For PHI", EmailTables.VERSION),
|
||||
scheduled(config, DISPATCH, "Dispatch Email", config.getDispatchIntervalSeconds()),
|
||||
scheduled(config, MAINTAIN, "Maintain Email", 300), scheduled(config, RETENTION, "Email Retention", 86400)));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** A standard streamed-ETL action.
|
||||
*******************************************************************************/
|
||||
private static QProcessMetaData action(String name, String label, String table)
|
||||
{
|
||||
Map<String, Serializable> values = new HashMap<>();
|
||||
values.put(StreamedETLWithFrontendProcess.FIELD_SOURCE_TABLE, table);
|
||||
values.put(StreamedETLWithFrontendProcess.FIELD_DESTINATION_TABLE, table);
|
||||
values.put(StreamedETLWithFrontendProcess.FIELD_SUPPORTS_FULL_VALIDATION, true);
|
||||
values.put(StreamedETLWithFrontendProcess.FIELD_DO_FULL_VALIDATION, true);
|
||||
values.put(StreamedETLWithFrontendProcess.FIELD_FETCH_HEAVY_FIELDS, true);
|
||||
values.put(StreamedETLWithFrontendProcess.FIELD_PREVIEW_MESSAGE, label + " for the selected records.");
|
||||
return (StreamedETLWithFrontendProcess.defineProcessMetaData(ExtractViaQueryStep.class, ActionTransform.class, ActionLoad.class, values)
|
||||
.withName(name).withLabel(label).withIcon(new QIcon("mail")).withTableName(table).withPermissionRules(EmailTables.permissionRules()));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Hidden backend schedule; without a scheduler manual dispatch remains possible.
|
||||
*******************************************************************************/
|
||||
private static QProcessMetaData scheduled(EmailQBitConfig config, String name, String label, int seconds)
|
||||
{
|
||||
QProcessMetaData process = new QProcessMetaData().withName(name).withLabel(label).withIsHidden(true)
|
||||
.withPermissionRules(EmailTables.permissionRules()).withStep(new QBackendStepMetaData().withName("run").withCode(new QCodeReference(ScheduledStep.class)));
|
||||
if(config.getSchedulerName() != null)
|
||||
{
|
||||
process.withSchedule(new QScheduleMetaData().withSchedulerName(config.getSchedulerName()).withRepeatSeconds(seconds).withInitialDelaySeconds(Math.min(seconds, 30)));
|
||||
}
|
||||
return (process);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Whether a selected row can participate in the action.
|
||||
*******************************************************************************/
|
||||
private static boolean eligible(String process, QRecord row)
|
||||
{
|
||||
if(NEW_VERSION.equals(process))
|
||||
{
|
||||
return (true);
|
||||
}
|
||||
if(ACTIVATE_VERSION.equals(process) || APPROVE_PHI.equals(process))
|
||||
{
|
||||
return ("DRAFT".equals(row.getValueString("status")) || APPROVE_PHI.equals(process) && "ACTIVE".equals(row.getValueString("status")));
|
||||
}
|
||||
if(SEND_TEST.equals(process))
|
||||
{
|
||||
return (Boolean.TRUE.equals(row.getValueBoolean("enabled")));
|
||||
}
|
||||
String status = row.getValueString("status");
|
||||
if(RESEND.equals(process))
|
||||
{
|
||||
return (List.of("DEAD_LETTER", "BOUNCED", "COMPLAINED").contains(status)
|
||||
&& (row.getValueString("htmlBody") != null || row.getValueString("textBody") != null));
|
||||
}
|
||||
return (EmailDispatcher.DUE.contains(status));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Review screen validation without side effects.
|
||||
*******************************************************************************/
|
||||
public static class ActionTransform extends AbstractTransformStep implements ProcessSummaryProviderInterface
|
||||
{
|
||||
private final ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>();
|
||||
|
||||
/*******************************************************************************
|
||||
** Filter rows and explain skipped records.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
|
||||
{
|
||||
for(QRecord row : input.getRecords())
|
||||
{
|
||||
boolean valid = eligible(input.getProcessName(), row);
|
||||
String diagnostic = valid ? "record will be processed" : "record skipped: status or retained content does not permit this action";
|
||||
if(valid && List.of(NEW_VERSION, ACTIVATE_VERSION, APPROVE_PHI).contains(input.getProcessName()))
|
||||
{
|
||||
try
|
||||
{
|
||||
QRecord template = NEW_VERSION.equals(input.getProcessName()) ? row : EmailStore.get(EmailTables.TEMPLATE, row.getValueInteger("templateId"), null);
|
||||
com.kingsrook.qbits.email.template.EmailVersions.requireEdit(template, APPROVE_PHI.equals(input.getProcessName()));
|
||||
if(ACTIVATE_VERSION.equals(input.getProcessName()))
|
||||
{
|
||||
com.kingsrook.qbits.email.template.EmailVersions.validate(template, row);
|
||||
}
|
||||
}
|
||||
catch(IllegalArgumentException e)
|
||||
{
|
||||
valid = false;
|
||||
diagnostic = e.getMessage();
|
||||
}
|
||||
}
|
||||
lines.add(new ProcessSummaryLine(valid ? Status.OK : Status.INFO, 1, diagnostic));
|
||||
if(valid)
|
||||
{
|
||||
output.getRecords().add(row);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Review lines.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean result)
|
||||
{
|
||||
return (lines);
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Execute only after review; re-read rows to avoid stale actions.
|
||||
*******************************************************************************/
|
||||
public static class ActionLoad extends AbstractLoadStep implements ProcessSummaryProviderInterface
|
||||
{
|
||||
private final ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>();
|
||||
|
||||
/*******************************************************************************
|
||||
** Perform and report one selected row at a time.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
|
||||
{
|
||||
String process = input.getProcessName();
|
||||
for(QRecord row : input.getRecords())
|
||||
{
|
||||
Integer id = row.getValueInteger("id");
|
||||
QRecord current = EmailStore.get(NEW_VERSION.equals(process) ? EmailTables.TEMPLATE : List.of(ACTIVATE_VERSION, APPROVE_PHI).contains(process) ? EmailTables.VERSION
|
||||
: SEND_TEST.equals(process) ? EmailTables.PROVIDER : EmailTables.MESSAGE, id, null);
|
||||
String result = "Skipped";
|
||||
if(current != null && eligible(process, current))
|
||||
{
|
||||
result = switch(process)
|
||||
{
|
||||
case NEW_VERSION -> "Created DRAFT " + com.kingsrook.qbits.email.template.EmailVersions.newVersion(id).getValueInteger("versionNumber");
|
||||
case ACTIVATE_VERSION ->
|
||||
{
|
||||
com.kingsrook.qbits.email.template.EmailVersions.activate(id);
|
||||
yield "ACTIVE";
|
||||
}
|
||||
case APPROVE_PHI ->
|
||||
{
|
||||
com.kingsrook.qbits.email.template.EmailVersions.approve(id);
|
||||
yield "Approved for PHI";
|
||||
}
|
||||
case SEND_NOW -> new EmailDispatcher().sendNow(id).name();
|
||||
case CANCEL -> cancel(id) ? "CANCELLED" : "Skipped";
|
||||
case RESEND -> resend(current).status().name();
|
||||
case SEND_TEST -> sendTest(current, input.getValueString("toAddress"));
|
||||
default -> "Skipped";
|
||||
};
|
||||
}
|
||||
lines.add(new ProcessSummaryLine(Status.INFO, 1, "record: " + result));
|
||||
output.getRecords().add(row);
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Result screen lines.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean result)
|
||||
{
|
||||
return (lines);
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Cancel under the same database gate as claims.
|
||||
*******************************************************************************/
|
||||
public static boolean cancel(Integer id) throws QException
|
||||
{
|
||||
QRecord row = EmailStore.get(EmailTables.MESSAGE, id, null);
|
||||
if(row == null)
|
||||
{
|
||||
return (false);
|
||||
}
|
||||
try(QBackendTransaction tx = QBackendTransaction.openFor(new UpdateInput(EmailTables.PROVIDER)))
|
||||
{
|
||||
EmailStore.lockProvider(row.getValueInteger("providerId"), tx);
|
||||
row = EmailStore.get(EmailTables.MESSAGE, id, tx);
|
||||
if(!EmailDispatcher.DUE.contains(row.getValueString("status")))
|
||||
{
|
||||
return (false);
|
||||
}
|
||||
EmailStore.update(EmailTables.MESSAGE, new QRecord().withValue("id", id).withValue("status", "CANCELLED").withValue("finishedDate", EmailRuntime.clock().instant()).withValue("nextAttemptDate", null), tx);
|
||||
tx.commit();
|
||||
return (true);
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** A fresh, linked message through suppression and provider validation again.
|
||||
*******************************************************************************/
|
||||
private static EmailSendResult resend(QRecord row) throws QException
|
||||
{
|
||||
return (EmailService.resend(row.getValueInteger("id")));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Queue and dispatch a fixed provider test message.
|
||||
*******************************************************************************/
|
||||
private static String sendTest(QRecord provider, String to) throws QException
|
||||
{
|
||||
if(to == null)
|
||||
{
|
||||
return ("REFUSED: a to address is required");
|
||||
}
|
||||
EmailSendResult result = EmailService.send(new EmailRequest().withProviderName(provider.getValueString("name")).withTo(to)
|
||||
.withSubject("Email delivery test").withMarkdownBody("This is a test of your email provider.").withSender("email", SEND_TEST));
|
||||
return (result.status() == EmailSendResult.Status.QUEUED ? new EmailDispatcher().sendNow(result.messageId()).name() : result.status().name());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Backend sending API with inputs declared as QQQ fields.
|
||||
*******************************************************************************/
|
||||
public static class SendStep implements BackendStep
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Map process inputs to the same service API.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
|
||||
{
|
||||
EmailRequest request = new EmailRequest().withTemplateName(input.getValueString("templateName"))
|
||||
.withSubject(input.getValueString("subject")).withMarkdownBody(input.getValueString("markdownBody"))
|
||||
.withHtmlBody(input.getValueString("htmlBody")).withTextBody(input.getValueString("textBody"))
|
||||
.withReplyTo(input.getValueString("replyTo")).withProviderName(input.getValueString("providerName"))
|
||||
.withBrandName(input.getValueString("brandName")).withIdempotencyKey(input.getValueString("idempotencyKey"))
|
||||
.withLinkedRecord(input.getValueString("linkedTableName"), input.getValueString("linkedRecordId"));
|
||||
if(input.getValueString("to") != null)
|
||||
{
|
||||
request.withTo(input.getValueString("to"));
|
||||
}
|
||||
if(input.getValueString("cc") != null)
|
||||
{
|
||||
request.withCc(input.getValueString("cc"));
|
||||
}
|
||||
if(input.getValueString("bcc") != null)
|
||||
{
|
||||
request.withBcc(input.getValueString("bcc"));
|
||||
}
|
||||
if(input.getValueString("variablesJson") != null)
|
||||
{
|
||||
request.withVariables(com.kingsrook.qbits.email.template.VariableSchema.values(input.getValueString("variablesJson")));
|
||||
}
|
||||
if(input.getValueString("locale") != null)
|
||||
{
|
||||
request.withLocale(input.getValueString("locale"));
|
||||
}
|
||||
if(input.getValueString("recordVariablesJson") != null)
|
||||
{
|
||||
var bindings = com.kingsrook.qbits.email.template.VariableSchema.values(input.getValueString("recordVariablesJson"));
|
||||
for(var binding : bindings.entrySet())
|
||||
{
|
||||
if(!(binding.getValue() instanceof Map<?, ?> ref) || !(ref.get("table") instanceof String table) || !(ref.get("id") instanceof Serializable id))
|
||||
{
|
||||
throw (new QException("Record variables need table and id"));
|
||||
}
|
||||
request.withRecordVariable(binding.getKey(), table, id);
|
||||
}
|
||||
}
|
||||
if(request.getTemplateName() == null && input.getValueString("category") != null)
|
||||
{
|
||||
try
|
||||
{
|
||||
request.withCategory(EmailCategory.valueOf(input.getValueString("category")));
|
||||
}
|
||||
catch(IllegalArgumentException e)
|
||||
{
|
||||
throw (new QException("Unknown email category"));
|
||||
}
|
||||
}
|
||||
EmailSendResult result = EmailService.send(request);
|
||||
output.addValue("messageId", result.messageId());
|
||||
output.addValue("status", result.status().name());
|
||||
output.addValue("reasons", String.join("; ", result.reasons()));
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Scheduler entry point.
|
||||
*******************************************************************************/
|
||||
public static class ScheduledStep implements BackendStep
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Dispatch, recover leases and probe health, or retain final messages.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
|
||||
{
|
||||
switch(input.getProcessName())
|
||||
{
|
||||
case RETENTION -> retain();
|
||||
case MAINTAIN -> new EmailDispatcher().maintain();
|
||||
default -> new EmailDispatcher().runOnce();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Clear retained bodies and optionally delete final messages and their attempts.
|
||||
*******************************************************************************/
|
||||
public static void retain() throws QException
|
||||
{
|
||||
EmailQBitConfig config = EmailRuntime.config();
|
||||
Instant now = EmailRuntime.clock().instant();
|
||||
for(QRecord row : SystemWrites.query(EmailTables.MESSAGE, new QQueryFilter(new QFilterCriteria("status", QCriteriaOperator.IN, new ArrayList<>(EmailDispatcher.FINAL)))))
|
||||
{
|
||||
Instant finalDate = row.getValueInstant("finishedDate");
|
||||
if(finalDate == null)
|
||||
{
|
||||
finalDate = row.getValueInstant("sentDate");
|
||||
}
|
||||
if(finalDate == null)
|
||||
{
|
||||
finalDate = row.getValueInstant("deadLetterDate");
|
||||
}
|
||||
if(finalDate == null)
|
||||
{
|
||||
finalDate = row.getValueInstant("createDate");
|
||||
}
|
||||
if(finalDate == null)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
if(config.getMessageRetentionDays() > 0 && finalDate.isBefore(now.minus(Duration.ofDays(config.getMessageRetentionDays()))))
|
||||
{
|
||||
try(QBackendTransaction transaction = QBackendTransaction.openFor(new UpdateInput(EmailTables.MESSAGE)))
|
||||
{
|
||||
SystemWrites.delete(EmailTables.ATTEMPT, EmailStore.equal("messageId", row.getValueInteger("id")), transaction);
|
||||
SystemWrites.delete(EmailTables.MESSAGE, EmailStore.equal("id", row.getValueInteger("id")), transaction);
|
||||
transaction.commit();
|
||||
}
|
||||
}
|
||||
else if(finalDate.isBefore(now.minus(Duration.ofDays(config.getBodyRetentionDays()))) && (row.getValue("htmlBody") != null || row.getValue("textBody") != null))
|
||||
{
|
||||
SystemWrites.updateOne(EmailTables.MESSAGE, new QRecord().withValue("id", row.getValue("id")).withValue("htmlBody", null).withValue("textBody", null));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,112 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.render;
|
||||
|
||||
|
||||
import java.util.Locale;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** WCAG relative luminance and AA contrast for both email color schemes.
|
||||
*******************************************************************************/
|
||||
public final class ContrastCheck
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private ContrastCheck()
|
||||
{
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Check every actual foreground/background pair in the branded shell.
|
||||
*******************************************************************************/
|
||||
public static void validate(QRecord brand)
|
||||
{
|
||||
check("Text on background", color(brand, "textColor"), color(brand, "backgroundColor"), "light");
|
||||
check("Text on background", color(brand, "darkTextColor"), color(brand, "darkBackgroundColor"), "dark");
|
||||
check("Button text on primary", color(brand, "buttonTextColor"), color(brand, "primaryColor"), "light");
|
||||
check("Button text on primary", color(brand, "darkButtonTextColor"), color(brand, "darkPrimaryColor"), "dark");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Defaults are neutral and accessible; reject anything except hex colors.
|
||||
*******************************************************************************/
|
||||
public static String color(QRecord brand, String field)
|
||||
{
|
||||
String value = brand.getValueString(field);
|
||||
if(value == null || value.isBlank())
|
||||
{
|
||||
value = switch(field)
|
||||
{
|
||||
case "backgroundColor", "buttonTextColor", "darkTextColor" -> "#ffffff";
|
||||
case "primaryColor" -> "#2459a6";
|
||||
case "darkPrimaryColor" -> color(brand, "primaryColor");
|
||||
case "darkButtonTextColor" -> ratio(color(brand, "buttonTextColor"), color(brand, "darkPrimaryColor")) >= 4.5
|
||||
? color(brand, "buttonTextColor") : ratio("#000000", color(brand, "darkPrimaryColor")) >= 4.5 ? "#000000" : "#ffffff";
|
||||
default -> "#171717";
|
||||
};
|
||||
}
|
||||
if(!value.matches("#[0-9a-fA-F]{6}"))
|
||||
{
|
||||
throw (new IllegalArgumentException("Brand colors must use six-digit hex values"));
|
||||
}
|
||||
return (value);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Contrast is independent of which color is lighter.
|
||||
*******************************************************************************/
|
||||
public static double ratio(String first, String second)
|
||||
{
|
||||
double a = luminance(first);
|
||||
double b = luminance(second);
|
||||
return ((Math.max(a, b) + 0.05) / (Math.min(a, b) + 0.05));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** sRGB linearization and weighted luminance.
|
||||
*******************************************************************************/
|
||||
private static double luminance(String color)
|
||||
{
|
||||
double[] weights = {0.2126, 0.7152, 0.0722};
|
||||
double result = 0;
|
||||
for(int i = 0; i < 3; i++)
|
||||
{
|
||||
double channel = Integer.parseInt(color.substring(1 + i * 2, 3 + i * 2), 16) / 255.0;
|
||||
result += weights[i] * (channel <= 0.04045 ? channel / 12.92 : Math.pow((channel + 0.055) / 1.055, 2.4));
|
||||
}
|
||||
return (result);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Author-facing refusal explains the failing scheme and threshold.
|
||||
*******************************************************************************/
|
||||
private static void check(String label, String foreground, String background, String scheme)
|
||||
{
|
||||
double ratio = ratio(foreground, background);
|
||||
if(ratio < 4.5)
|
||||
{
|
||||
throw (new IllegalArgumentException(String.format(Locale.ROOT, "%s is %.1f:1 in %s mode; it needs 4.5:1", label, ratio, scheme)));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,156 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.render;
|
||||
|
||||
|
||||
import java.util.List;
|
||||
import com.kingsrook.qbits.email.EmailRuntime;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTables;
|
||||
import com.kingsrook.qbits.email.support.EmailStore;
|
||||
import com.kingsrook.qbits.email.support.SystemWrites;
|
||||
import com.kingsrook.qbits.email.template.VariableSchema;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Brand selection shared by sends, activation and previews.
|
||||
*******************************************************************************/
|
||||
public final class EmailBrands
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private EmailBrands()
|
||||
{
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Explicit request, template, app brand, default brand, then neutral.
|
||||
*******************************************************************************/
|
||||
public static QRecord resolve(String requested, QRecord template) throws QException
|
||||
{
|
||||
if(requested != null)
|
||||
{
|
||||
List<QRecord> rows = SystemWrites.query(EmailTables.BRAND, EmailStore.equal("name", requested));
|
||||
if(rows.isEmpty())
|
||||
{
|
||||
throw (new IllegalArgumentException("Unknown email brand"));
|
||||
}
|
||||
return (rows.getFirst());
|
||||
}
|
||||
if(template != null && template.getValueInteger("brandId") != null)
|
||||
{
|
||||
QRecord brand = EmailStore.get(EmailTables.BRAND, template.getValueInteger("brandId"), null);
|
||||
if(brand == null)
|
||||
{
|
||||
throw (new IllegalArgumentException("Template brand is missing"));
|
||||
}
|
||||
return (brand);
|
||||
}
|
||||
List<QRecord> brands = SystemWrites.query(EmailTables.BRAND, EmailStore.equal("appName", EmailRuntime.config().getAppName()));
|
||||
if(brands.isEmpty())
|
||||
{
|
||||
brands = SystemWrites.query(EmailTables.BRAND, EmailStore.equal("isDefault", true));
|
||||
}
|
||||
if(brands.size() > 1)
|
||||
{
|
||||
throw (new IllegalArgumentException("More than one default email brand matches"));
|
||||
}
|
||||
return (brands.isEmpty() ? new QRecord().withValue("label", EmailRuntime.config().getAppLabel()).withValue("name", "neutral") : brands.getFirst());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Refuse unsafe URLs and inaccessible colors before a brand is saved.
|
||||
*******************************************************************************/
|
||||
public static void validate(QRecord brand)
|
||||
{
|
||||
ContrastCheck.validate(brand);
|
||||
for(String field : List.of("logoUrl", "logoDarkUrl", "siteUrl"))
|
||||
{
|
||||
String value = brand.getValueString(field);
|
||||
if(value != null && !value.isBlank())
|
||||
{
|
||||
VariableSchema.url(value);
|
||||
if(!"siteUrl".equals(field))
|
||||
{
|
||||
https(value);
|
||||
}
|
||||
}
|
||||
}
|
||||
social(brand);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Parse the small social-link list with strict web URLs.
|
||||
*******************************************************************************/
|
||||
public static List<java.util.Map<String, String>> social(QRecord brand)
|
||||
{
|
||||
try
|
||||
{
|
||||
String json = brand.getValueString("socialLinks");
|
||||
if(json == null || json.isBlank())
|
||||
{
|
||||
return (List.of());
|
||||
}
|
||||
var links = new com.fasterxml.jackson.databind.ObjectMapper().readValue(json,
|
||||
new com.fasterxml.jackson.core.type.TypeReference<List<java.util.Map<String, String>>>() {});
|
||||
links.forEach(link -> https(link.get("url")));
|
||||
return (links);
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
throw (new IllegalArgumentException("Social links need labels and https URLs"));
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Logos and social destinations require HTTPS at the save boundary.
|
||||
*******************************************************************************/
|
||||
private static void https(String value)
|
||||
{
|
||||
VariableSchema.url(value);
|
||||
if(!"https".equalsIgnoreCase(java.net.URI.create(value).getScheme()))
|
||||
{
|
||||
throw (new IllegalArgumentException("Brand logo and social URLs must use https"));
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Approved messages restrict every image to the brand site or an explicit host.
|
||||
*******************************************************************************/
|
||||
public static boolean allowedImage(String value, QRecord brand)
|
||||
{
|
||||
try
|
||||
{
|
||||
https(value);
|
||||
String host = java.net.URI.create(value).getHost();
|
||||
String site = brand.getValueString("siteUrl");
|
||||
return (EmailRuntime.config().getAllowedImageHosts().contains(host.toLowerCase(java.util.Locale.ROOT))
|
||||
|| site != null && host.equalsIgnoreCase(java.net.URI.create(site).getHost()));
|
||||
}
|
||||
catch(IllegalArgumentException e)
|
||||
{
|
||||
return (false);
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
@@ -0,0 +1,135 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.render;
|
||||
|
||||
|
||||
import com.kingsrook.qbits.email.outbox.EmailCategory;
|
||||
import com.kingsrook.qbits.email.template.EmailTemplateEngine;
|
||||
import com.kingsrook.qbits.email.template.VariableSchema;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Single-column, 600px email shell with inline fallbacks and dark-mode hooks.
|
||||
*******************************************************************************/
|
||||
public final class EmailLayout
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private EmailLayout()
|
||||
{
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Render both schemes from the same validated brand.
|
||||
*******************************************************************************/
|
||||
public static String render(String body, String preheader, QRecord brand, EmailCategory category, String unsubscribe)
|
||||
{
|
||||
EmailBrands.validate(brand);
|
||||
String dark = ".email-shell,.email-content{background-color:" + ContrastCheck.color(brand, "darkBackgroundColor")
|
||||
+ "!important;color:" + ContrastCheck.color(brand, "darkTextColor") + "!important}.email-button{background-color:"
|
||||
+ ContrastCheck.color(brand, "darkPrimaryColor") + "!important}.email-button a{color:" + ContrastCheck.color(brand, "darkButtonTextColor")
|
||||
+ "!important}.email-logo-light{display:none!important}.email-logo-dark{display:block!important}";
|
||||
String outlook = "[data-ogsc] .email-content,[data-ogsc] .email-shell,[data-ogsb] .email-content,[data-ogsb] .email-shell{background-color:" + ContrastCheck.color(brand, "darkBackgroundColor")
|
||||
+ "!important;color:" + ContrastCheck.color(brand, "darkTextColor") + "!important}[data-ogsc] .email-button,[data-ogsb] .email-button{background-color:"
|
||||
+ ContrastCheck.color(brand, "darkPrimaryColor") + "!important}[data-ogsc] .email-button a,[data-ogsb] .email-button a{color:" + ContrastCheck.color(brand, "darkButtonTextColor")
|
||||
+ "!important}[data-ogsc] .email-logo-light,[data-ogsb] .email-logo-light{display:none!important}[data-ogsc] .email-logo-dark,[data-ogsb] .email-logo-dark{display:block!important}";
|
||||
return ("<!doctype html><html><head><meta charset=\"utf-8\"><meta name=\"viewport\" content=\"width=device-width,initial-scale=1\">"
|
||||
+ "<meta name=\"color-scheme\" content=\"light dark\"><meta name=\"supported-color-schemes\" content=\"light dark\">"
|
||||
+ "<style>:root{color-scheme:light dark;supported-color-schemes:light dark}@media(max-width:640px){.email-content{padding:16px!important;width:100%!important}}"
|
||||
+ "@media(prefers-color-scheme:dark){" + dark + "}" + outlook + "</style></head><body class=\"email-shell\" style=\"margin:0;padding:0;background-color:"
|
||||
+ ContrastCheck.color(brand, "backgroundColor") + ";color:" + ContrastCheck.color(brand, "textColor") + "\">"
|
||||
+ "<div style=\"display:none;max-height:0;overflow:hidden;opacity:0;mso-hide:all\">" + EmailTemplateEngine.escape(preheader) + "</div>"
|
||||
+ "<table role=\"presentation\" width=\"100%\" cellspacing=\"0\" cellpadding=\"0\"><tr><td align=\"center\">"
|
||||
+ "<!--[if mso]><table role=\"presentation\" width=\"600\"><tr><td><![endif]-->"
|
||||
+ "<table role=\"presentation\" width=\"100%\" cellspacing=\"0\" cellpadding=\"0\" style=\"max-width:600px;table-layout:fixed\"><tr>"
|
||||
+ "<td class=\"email-content\" style=\"padding:24px;font-family:Arial,sans-serif;line-height:1.6;overflow-wrap:anywhere;word-break:break-word;background-color:"
|
||||
+ ContrastCheck.color(brand, "backgroundColor") + ";color:" + ContrastCheck.color(brand, "textColor") + "\">" + logos(brand)
|
||||
+ body + footer(brand, category, unsubscribe) + "</td></tr></table><!--[if mso]></td></tr></table><![endif]--></td></tr></table></body></html>");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Logo alternatives, with dimensions that never force horizontal overflow.
|
||||
*******************************************************************************/
|
||||
private static String logos(QRecord brand)
|
||||
{
|
||||
String light = brand.getValueString("logoUrl");
|
||||
if(light == null || light.isBlank())
|
||||
{
|
||||
return ("<p><strong>" + EmailTemplateEngine.escape(brand.getValueString("label")) + "</strong></p>");
|
||||
}
|
||||
String dark = brand.getValueString("logoDarkUrl");
|
||||
return (logo(light, brand.getValueString("logoAlt"), dark == null ? "" : "email-logo-light", "block")
|
||||
+ (dark == null ? "" : logo(dark, brand.getValueString("logoAlt"), "email-logo-dark", "none")));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Escaped logo element.
|
||||
*******************************************************************************/
|
||||
private static String logo(String url, String alt, String css, String display)
|
||||
{
|
||||
return ("<img class=\"" + css + "\" src=\"" + EmailTemplateEngine.escape(VariableSchema.url(url)) + "\" alt=\""
|
||||
+ EmailTemplateEngine.escape(alt) + "\" width=\"180\" style=\"display:" + display + ";max-width:100%;height:auto\">");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Category-specific footer, including postal address for marketing.
|
||||
*******************************************************************************/
|
||||
public static String footer(QRecord brand, EmailCategory category, String unsubscribe)
|
||||
{
|
||||
StringBuilder footer = new StringBuilder("<footer style=\"margin-top:24px;font-size:12px;overflow-wrap:anywhere\">");
|
||||
footer.append(EmailMarkdown.render(brand.getValueString("footerMarkdown"), brand));
|
||||
if(category == EmailCategory.MARKETING)
|
||||
{
|
||||
String address = brand.getValueString("postalAddress");
|
||||
if(address == null || address.isBlank())
|
||||
{
|
||||
throw (new IllegalArgumentException("Marketing email needs a brand postal address"));
|
||||
}
|
||||
footer.append("<p>").append(EmailTemplateEngine.escape(address)).append("</p>");
|
||||
}
|
||||
for(var link : EmailBrands.social(brand))
|
||||
{
|
||||
footer.append("<a style=\"color:inherit\" href=\"").append(EmailTemplateEngine.escape(link.get("url"))).append("\">")
|
||||
.append(EmailTemplateEngine.escape(link.get("label"))).append("</a> ");
|
||||
}
|
||||
if((category == EmailCategory.NOTIFICATION || category == EmailCategory.MARKETING) && unsubscribe != null && !unsubscribe.isBlank())
|
||||
{
|
||||
footer.append("<p><a style=\"color:inherit\" href=\"").append(EmailTemplateEngine.escape(unsubscribe)).append("\">Unsubscribe</a></p>");
|
||||
}
|
||||
return (footer.append("</footer>").toString());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** HTML and compiled MJML retain their shell; append opt-out before body end.
|
||||
*******************************************************************************/
|
||||
public static String appendFooter(String html, QRecord brand, EmailCategory category, String unsubscribe)
|
||||
{
|
||||
if(category != EmailCategory.NOTIFICATION && category != EmailCategory.MARKETING)
|
||||
{
|
||||
return (html);
|
||||
}
|
||||
String footer = footer(brand, category, unsubscribe);
|
||||
java.util.regex.Matcher end = java.util.regex.Pattern.compile("</body\\s*>", java.util.regex.Pattern.CASE_INSENSITIVE).matcher(html);
|
||||
return (end.find() ? html.substring(0, end.start()) + footer + html.substring(end.start()) : html + footer);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,71 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.render;
|
||||
|
||||
|
||||
import java.util.List;
|
||||
import java.util.regex.Matcher;
|
||||
import java.util.regex.Pattern;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import org.commonmark.ext.autolink.AutolinkExtension;
|
||||
import org.commonmark.ext.gfm.tables.TablesExtension;
|
||||
import org.commonmark.parser.Parser;
|
||||
import org.commonmark.renderer.html.HtmlRenderer;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** CommonMark with GFM tables, autolinks and a constrained button attribute.
|
||||
*******************************************************************************/
|
||||
public final class EmailMarkdown
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private EmailMarkdown()
|
||||
{
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Raw HTML is escaped and URLs sanitized before adding table buttons.
|
||||
*******************************************************************************/
|
||||
public static String render(String markdown, QRecord brand)
|
||||
{
|
||||
var extensions = List.of(TablesExtension.create(), AutolinkExtension.create());
|
||||
String html = HtmlRenderer.builder().escapeHtml(true).sanitizeUrls(true).extensions(extensions).build()
|
||||
.render(Parser.builder().extensions(extensions).build().parse(markdown == null ? "" : markdown))
|
||||
.replace("<a ", "<a style=\"color:inherit;text-decoration:underline\" ");
|
||||
Matcher links = Pattern.compile("<a[^>]* href=\"([^\"]*)\"[^>]*>((?:(?!</?a\\b)[^\\r\\n])*)</a>\\{\\.button}").matcher(html);
|
||||
StringBuilder result = new StringBuilder();
|
||||
while(links.find())
|
||||
{
|
||||
String button = "<table role=\"presentation\" cellspacing=\"0\" cellpadding=\"0\"><tr><td class=\"email-button\" bgcolor=\""
|
||||
+ ContrastCheck.color(brand, "primaryColor") + "\" style=\"border-radius:4px;padding:12px 20px;background-color:"
|
||||
+ ContrastCheck.color(brand, "primaryColor") + "\"><a href=\"" + links.group(1) + "\" style=\"display:inline-block;text-decoration:none;color:"
|
||||
+ ContrastCheck.color(brand, "buttonTextColor") + ";font-weight:bold\">" + links.group(2) + "</a></td></tr></table>";
|
||||
links.appendReplacement(result, Matcher.quoteReplacement(button));
|
||||
}
|
||||
links.appendTail(result);
|
||||
return (result.toString().replace("<p><table", "<table").replace("</table></p>", "</table>").replace("<table>", "<table style=\"width:100%;table-layout:fixed;border-collapse:collapse\">")
|
||||
.replace("<td>", "<td style=\"padding:6px;border:1px solid #888;overflow-wrap:anywhere\">")
|
||||
.replace("<th>", "<th style=\"padding:6px;border:1px solid #888;overflow-wrap:anywhere\">")
|
||||
.replace("<img ", "<img style=\"max-width:100%;height:auto\" ").replace("<pre>", "<pre style=\"white-space:pre-wrap;overflow-wrap:anywhere\">"));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,83 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.render;
|
||||
|
||||
|
||||
import javax.crypto.Mac;
|
||||
import javax.crypto.spec.SecretKeySpec;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.security.MessageDigest;
|
||||
import java.util.Base64;
|
||||
import java.util.HexFormat;
|
||||
import java.util.Map;
|
||||
import com.kingsrook.qbits.email.EmailRuntime;
|
||||
import com.kingsrook.qbits.email.template.VariableSchema;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Queue-time link signing from DESIGN 8.2. The receiving route is phase (c).
|
||||
*******************************************************************************/
|
||||
public final class EmailUnsubscribeLinks
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private EmailUnsubscribeLinks()
|
||||
{
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Bind the message, recipient-envelope hash and category without exposing PII.
|
||||
** No key means no link/header; marketing must refuse rather than omit opt-out.
|
||||
*******************************************************************************/
|
||||
public static String create(String messageId, String recipients, String category)
|
||||
{
|
||||
if(!"NOTIFICATION".equals(category) && !"MARKETING".equals(category))
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
try
|
||||
{
|
||||
String key = EmailRuntime.config().getSecretResolver().apply("QQQ_EMAIL_UNSUBSCRIBE_KEY");
|
||||
String base = EmailRuntime.config().getPublicBaseUrl();
|
||||
if(key == null || key.isBlank() || base == null || base.isBlank())
|
||||
{
|
||||
if("MARKETING".equals(category))
|
||||
{
|
||||
throw (new IllegalArgumentException());
|
||||
}
|
||||
return (null);
|
||||
}
|
||||
VariableSchema.url(base);
|
||||
String hash = HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(recipients.getBytes(StandardCharsets.UTF_8)));
|
||||
String payload = Base64.getUrlEncoder().withoutPadding().encodeToString(VariableSchema.json(Map.of("messageId", messageId, "addressHash", hash, "category", category)).getBytes(StandardCharsets.UTF_8));
|
||||
Mac mac = Mac.getInstance("HmacSHA256");
|
||||
mac.init(new SecretKeySpec(key.getBytes(StandardCharsets.UTF_8), "HmacSHA256"));
|
||||
String signed = "v1." + payload;
|
||||
String signature = Base64.getUrlEncoder().withoutPadding().encodeToString(mac.doFinal(signed.getBytes(StandardCharsets.UTF_8)));
|
||||
return (base.replaceAll("/+$", "") + "/qqq/hooks/email/unsubscribe/" + signed + "." + signature);
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
throw (new IllegalArgumentException("Opt-out email needs publicBaseUrl and QQQ_EMAIL_UNSUBSCRIBE_KEY"));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,132 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.render;
|
||||
|
||||
|
||||
import javax.swing.text.MutableAttributeSet;
|
||||
import javax.swing.text.html.HTML;
|
||||
import javax.swing.text.html.HTMLEditorKit;
|
||||
import javax.swing.text.html.parser.ParserDelegator;
|
||||
import java.io.StringReader;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** HTML to readable text using the JDK parser (including entity decoding).
|
||||
*******************************************************************************/
|
||||
public final class HtmlToText
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private HtmlToText()
|
||||
{
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Keep links, headings, lists and table rows; omit scripts and styles.
|
||||
*******************************************************************************/
|
||||
public static String convert(String html)
|
||||
{
|
||||
StringBuilder text = new StringBuilder();
|
||||
try
|
||||
{
|
||||
new ParserDelegator().parse(new StringReader(html == null ? "" : html), new HTMLEditorKit.ParserCallback()
|
||||
{
|
||||
private int hidden;
|
||||
private String href;
|
||||
|
||||
/*******************************************************************************
|
||||
** Start readable block boundaries and remember link destinations.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void handleStartTag(HTML.Tag tag, MutableAttributeSet attributes, int position)
|
||||
{
|
||||
if(tag == HTML.Tag.STYLE || tag == HTML.Tag.SCRIPT || tag == HTML.Tag.HEAD)
|
||||
{
|
||||
hidden++;
|
||||
}
|
||||
if(tag == HTML.Tag.A)
|
||||
{
|
||||
href = (String) attributes.getAttribute(HTML.Attribute.HREF);
|
||||
}
|
||||
if(tag == HTML.Tag.LI)
|
||||
{
|
||||
text.append("\n- ");
|
||||
}
|
||||
if(tag == HTML.Tag.TD || tag == HTML.Tag.TH)
|
||||
{
|
||||
text.append(" ");
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Add link destinations and block separators.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void handleEndTag(HTML.Tag tag, int position)
|
||||
{
|
||||
if(tag == HTML.Tag.STYLE || tag == HTML.Tag.SCRIPT || tag == HTML.Tag.HEAD)
|
||||
{
|
||||
hidden = Math.max(0, hidden - 1);
|
||||
}
|
||||
if(tag == HTML.Tag.A && href != null)
|
||||
{
|
||||
text.append(" (").append(href).append(')');
|
||||
href = null;
|
||||
}
|
||||
if(tag.isBlock())
|
||||
{
|
||||
text.append('\n');
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Line breaks remain line breaks.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void handleSimpleTag(HTML.Tag tag, MutableAttributeSet attributes, int position)
|
||||
{
|
||||
if(tag == HTML.Tag.BR)
|
||||
{
|
||||
text.append('\n');
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Parser-decoded text, excluding executable and metadata content.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void handleText(char[] data, int position)
|
||||
{
|
||||
if(hidden == 0)
|
||||
{
|
||||
text.append(data);
|
||||
}
|
||||
}
|
||||
}, true);
|
||||
}
|
||||
catch(java.io.IOException e)
|
||||
{
|
||||
throw (new IllegalArgumentException("Could not convert email HTML"));
|
||||
}
|
||||
return (text.toString().replaceAll("[ \\t]+", " ").replaceAll(" *\\n *", "\n").replaceAll("\\n{3,}", "\n\n").strip());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,88 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.support;
|
||||
|
||||
|
||||
import java.io.Serializable;
|
||||
import java.util.List;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTables;
|
||||
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Small database helpers shared by dispatcher and actions.
|
||||
*******************************************************************************/
|
||||
public final class EmailStore
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private EmailStore()
|
||||
{
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Equality filter.
|
||||
*******************************************************************************/
|
||||
public static QQueryFilter equal(String field, Serializable value)
|
||||
{
|
||||
return (new QQueryFilter(new QFilterCriteria(field, QCriteriaOperator.EQUALS, value)));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Read a row in a transaction.
|
||||
*******************************************************************************/
|
||||
public static QRecord get(String table, Integer id, QBackendTransaction transaction) throws QException
|
||||
{
|
||||
List<QRecord> rows = SystemWrites.query(table, equal("id", id), transaction);
|
||||
return (rows.isEmpty() ? null : rows.get(0));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Check write errors without exposing record contents.
|
||||
*******************************************************************************/
|
||||
public static void update(String table, QRecord row, QBackendTransaction transaction) throws QException
|
||||
{
|
||||
List<QRecord> rows = SystemWrites.update(table, List.of(row), transaction);
|
||||
if(rows.isEmpty() || (rows.get(0).getErrors() != null && !rows.get(0).getErrors().isEmpty()))
|
||||
{
|
||||
throw (new QException("Email database update failed"));
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Serialize provider rate checks and claims using the database row lock.
|
||||
*******************************************************************************/
|
||||
public static QRecord lockProvider(Integer id, QBackendTransaction transaction) throws QException
|
||||
{
|
||||
if(get(EmailTables.PROVIDER, id, transaction) == null)
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
update(EmailTables.PROVIDER, new QRecord().withValue("id", id).withValue("dispatchLock", 0), transaction);
|
||||
return (get(EmailTables.PROVIDER, id, transaction));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,207 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.support;
|
||||
|
||||
|
||||
import java.util.List;
|
||||
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
|
||||
import com.kingsrook.qqq.backend.core.actions.permissions.CorePermissionEnforcer;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.CountAction;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.DeleteAction;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.count.CountInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertOutput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The qbit's own reads and writes (the delivery engine, the outbox, health
|
||||
** and rotation). They run in core's authorized scope, so they work with core
|
||||
** permission enforcement on, and they mark the thread so the tables'
|
||||
** customizers can tell a qbit write from a person's or the API's.
|
||||
*******************************************************************************/
|
||||
public final class SystemWrites
|
||||
{
|
||||
private static final ThreadLocal<Integer> DEPTH = ThreadLocal.withInitial(() -> 0);
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Work that may throw.
|
||||
*******************************************************************************/
|
||||
@FunctionalInterface
|
||||
public interface Work<T>
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Do it.
|
||||
*******************************************************************************/
|
||||
T run() throws QException;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private SystemWrites()
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Run work as the qbit.
|
||||
*******************************************************************************/
|
||||
public static <T> T call(Work<T> work) throws QException
|
||||
{
|
||||
DEPTH.set(DEPTH.get() + 1);
|
||||
try(CorePermissionEnforcer.AuthorizedScope ignored = CorePermissionEnforcer.openAuthorizedScope())
|
||||
{
|
||||
return (work.run());
|
||||
}
|
||||
finally
|
||||
{
|
||||
DEPTH.set(DEPTH.get() - 1);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** True while the qbit itself is writing on this thread.
|
||||
*******************************************************************************/
|
||||
public static boolean active()
|
||||
{
|
||||
return (DEPTH.get() > 0);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Insert, returning the output (records may carry errors).
|
||||
*******************************************************************************/
|
||||
public static InsertOutput insert(String table, List<QRecord> records, QBackendTransaction transaction) throws QException
|
||||
{
|
||||
return (call(() -> new InsertAction().execute(new InsertInput(table).withRecords(records).withTransaction(transaction))));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Insert one record, returning it (with errors if it failed).
|
||||
*******************************************************************************/
|
||||
public static QRecord insertOne(String table, QRecord record) throws QException
|
||||
{
|
||||
return (insert(table, List.of(record), null).getRecords().get(0));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Update.
|
||||
*******************************************************************************/
|
||||
public static List<QRecord> update(String table, List<QRecord> records, QBackendTransaction transaction) throws QException
|
||||
{
|
||||
if(records.isEmpty())
|
||||
{
|
||||
return (records);
|
||||
}
|
||||
return (call(() -> new UpdateAction().execute(new UpdateInput(table).withRecords(records).withTransaction(transaction)).getRecords()));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Update one record.
|
||||
*******************************************************************************/
|
||||
public static void updateOne(String table, QRecord record) throws QException
|
||||
{
|
||||
update(table, List.of(record), null);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Query.
|
||||
*******************************************************************************/
|
||||
public static List<QRecord> query(String table, QQueryFilter filter) throws QException
|
||||
{
|
||||
return (call(() -> new QueryAction().execute(new QueryInput(table).withShouldFetchHeavyFields(true).withShouldOmitHiddenFields(false).withFilter(filter)).getRecords()));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Query in a transaction.
|
||||
*******************************************************************************/
|
||||
public static List<QRecord> query(String table, QQueryFilter filter, QBackendTransaction transaction) throws QException
|
||||
{
|
||||
return (call(() -> new QueryAction().execute(new QueryInput(table).withShouldFetchHeavyFields(true).withShouldOmitHiddenFields(false).withFilter(filter).withTransaction(transaction)).getRecords()));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Count.
|
||||
*******************************************************************************/
|
||||
public static int count(String table, QQueryFilter filter) throws QException
|
||||
{
|
||||
return (count(table, filter, null));
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Count within the claim transaction so rate budgets remain shared.
|
||||
*******************************************************************************/
|
||||
public static int count(String table, QQueryFilter filter, QBackendTransaction transaction) throws QException
|
||||
{
|
||||
Integer count = call(() -> new CountAction().execute(new CountInput(table).withFilter(filter).withTransaction(transaction)).getCount());
|
||||
return (count == null ? 0 : count);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Delete by filter, returning the number deleted.
|
||||
*******************************************************************************/
|
||||
public static int delete(String table, QQueryFilter filter) throws QException
|
||||
{
|
||||
return (delete(table, filter, null));
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Delete within a transaction, including retention's parent and child rows.
|
||||
*******************************************************************************/
|
||||
public static int delete(String table, QQueryFilter filter, QBackendTransaction transaction) throws QException
|
||||
{
|
||||
Integer deleted = call(() -> new DeleteAction().execute(new DeleteInput(table).withQueryFilter(filter).withTransaction(transaction)).getDeletedRecordCount());
|
||||
return (deleted == null ? 0 : deleted);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,133 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.template;
|
||||
|
||||
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.concurrent.Executors;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** MJML stdin/stdout compiler with a total 20-second deadline and no files.
|
||||
*******************************************************************************/
|
||||
public class CliMjmlCompiler implements MjmlCompilerInterface
|
||||
{
|
||||
private final String binary;
|
||||
|
||||
/*******************************************************************************
|
||||
** Explicit executable path; no shell interpolation or PATH guessing.
|
||||
*******************************************************************************/
|
||||
public CliMjmlCompiler(String binary)
|
||||
{
|
||||
this.binary = binary;
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Drain stdout concurrently with stdin, discarding potentially private stderr.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String compile(String source)
|
||||
{
|
||||
rejectIncludes(source);
|
||||
Process process = null;
|
||||
try(var executor = Executors.newVirtualThreadPerTaskExecutor())
|
||||
{
|
||||
process = processBuilder().start();
|
||||
Process running = process;
|
||||
var task = executor.submit(() ->
|
||||
{
|
||||
var writer = executor.submit(() ->
|
||||
{
|
||||
try(var stdin = running.getOutputStream())
|
||||
{
|
||||
stdin.write(source.getBytes(StandardCharsets.UTF_8));
|
||||
}
|
||||
return (null);
|
||||
});
|
||||
byte[] output = running.getInputStream().readNBytes(10_000_001);
|
||||
if(output.length > 10_000_000)
|
||||
{
|
||||
throw (new IllegalArgumentException("MJML output too large"));
|
||||
}
|
||||
writer.get();
|
||||
if(running.waitFor() != 0)
|
||||
{
|
||||
throw (new IllegalArgumentException("MJML compilation failed"));
|
||||
}
|
||||
return (new String(output, StandardCharsets.UTF_8));
|
||||
});
|
||||
try
|
||||
{
|
||||
return (task.get(20, TimeUnit.SECONDS));
|
||||
}
|
||||
finally
|
||||
{
|
||||
running.descendants().forEach(ProcessHandle::destroyForcibly);
|
||||
running.destroyForcibly();
|
||||
task.cancel(true);
|
||||
}
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
if(e instanceof InterruptedException)
|
||||
{
|
||||
Thread.currentThread().interrupt();
|
||||
}
|
||||
throw (new IllegalArgumentException("MJML compilation failed or exceeded 20 seconds"));
|
||||
}
|
||||
finally
|
||||
{
|
||||
if(process != null)
|
||||
{
|
||||
process.destroyForcibly();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** MJML 4.x/5 supports allowIncludes=false. Refusing the source tag as well
|
||||
** removes include resolution and its dependence on the inherited directory.
|
||||
*******************************************************************************/
|
||||
public static void rejectIncludes(String source)
|
||||
{
|
||||
if(source != null && java.util.regex.Pattern.compile("(?i)<\\s*mj-include\\b").matcher(source).find())
|
||||
{
|
||||
throw (new IllegalArgumentException("MJML includes are not allowed"));
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Package seam for checking child isolation without executing a compiler.
|
||||
*******************************************************************************/
|
||||
ProcessBuilder processBuilder()
|
||||
{
|
||||
ProcessBuilder builder = new ProcessBuilder(binary, "-i", "-s", "--config.allowIncludes=false");
|
||||
String path = builder.environment().get("PATH");
|
||||
builder.environment().clear();
|
||||
if(path != null)
|
||||
{
|
||||
builder.environment().put("PATH", path);
|
||||
}
|
||||
return (builder.redirectError(ProcessBuilder.Redirect.DISCARD));
|
||||
}
|
||||
|
||||
}
|
||||
@@ -0,0 +1,77 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.template;
|
||||
|
||||
|
||||
import java.util.List;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTables;
|
||||
import com.kingsrook.qbits.email.support.EmailStore;
|
||||
import com.kingsrook.qbits.email.support.SystemWrites;
|
||||
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Built-ins are created on first use, transactionally and without JVM state.
|
||||
*******************************************************************************/
|
||||
public final class EmailBuiltIns
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private EmailBuiltIns()
|
||||
{
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Seed the secure-link pattern. The unique template name arbitrates replicas.
|
||||
*******************************************************************************/
|
||||
public static void ensure() throws QException
|
||||
{
|
||||
if(!SystemWrites.query(EmailTables.TEMPLATE, EmailStore.equal("name", "email_secure_message")).isEmpty())
|
||||
{
|
||||
return;
|
||||
}
|
||||
try(QBackendTransaction tx = QBackendTransaction.openFor(new InsertInput(EmailTables.TEMPLATE)))
|
||||
{
|
||||
QRecord template = SystemWrites.insert(EmailTables.TEMPLATE, List.of(new QRecord().withValue("name", "email_secure_message")
|
||||
.withValue("label", "Secure message").withValue("kind", "MARKDOWN").withValue("category", "SECURITY").withValue("phiApproved", false)), tx).getRecords().getFirst();
|
||||
if(template.getValue("id") == null || template.getErrors() != null && !template.getErrors().isEmpty())
|
||||
{
|
||||
tx.rollback();
|
||||
return;
|
||||
}
|
||||
QRecord version = SystemWrites.insert(EmailTables.VERSION, List.of(new QRecord().withValue("templateId", template.getValue("id"))
|
||||
.withValue("versionNumber", 1).withValue("status", "ACTIVE").withValue("locale", "en").withValue("subject", "New secure message")
|
||||
.withValue("bodySource", "You have a new secure message from {{brand.label}}. Sign in to read it.\n\n[Read secure message]({{link}}){.button}")
|
||||
.withValue("variablesSchema", "[{\"name\":\"link\",\"type\":\"URL\",\"required\":true}]")
|
||||
.withValue("sampleVariables", "{\"link\":\"https://example.invalid/sign-in\"}")), tx).getRecords().getFirst();
|
||||
if(version.getValue("id") == null || version.getErrors() != null && !version.getErrors().isEmpty())
|
||||
{
|
||||
throw (new QException("Could not install secure-message template"));
|
||||
}
|
||||
EmailStore.update(EmailTables.TEMPLATE, new QRecord().withValue("id", template.getValue("id")).withValue("activeVersionId", version.getValue("id")), tx);
|
||||
tx.commit();
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,130 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.template;
|
||||
|
||||
|
||||
import java.util.List;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTables;
|
||||
import com.kingsrook.qbits.email.outbox.EmailRequest;
|
||||
import com.kingsrook.qbits.email.render.ContrastCheck;
|
||||
import com.kingsrook.qbits.email.render.EmailBrands;
|
||||
import com.kingsrook.qbits.email.support.EmailStore;
|
||||
import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
|
||||
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.RawHTML;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import org.jsoup.Jsoup;
|
||||
import org.jsoup.safety.Safelist;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Inline preview fallback: Next strips iframes and style blocks. Samples are
|
||||
** sanitized server-side too; no live record references are read for previews.
|
||||
*******************************************************************************/
|
||||
public class EmailPreviewWidget extends AbstractWidgetRenderer
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Fetch the version and parent as the viewer, then render sample data.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public RenderWidgetOutput render(RenderWidgetInput input) throws QException
|
||||
{
|
||||
try
|
||||
{
|
||||
int id = Integer.parseInt(input.getQueryParams().get("id"));
|
||||
List<QRecord> versions = new QueryAction().execute(new QueryInput(EmailTables.VERSION).withShouldFetchHeavyFields(true)
|
||||
.withFilter(EmailStore.equal("id", id))).getRecords();
|
||||
if(versions.isEmpty())
|
||||
{
|
||||
return (output("<p>Version unavailable.</p>"));
|
||||
}
|
||||
QRecord version = versions.getFirst();
|
||||
List<QRecord> templates = new QueryAction().execute(new QueryInput(EmailTables.TEMPLATE)
|
||||
.withFilter(EmailStore.equal("id", version.getValueInteger("templateId")))).getRecords();
|
||||
if(templates.isEmpty())
|
||||
{
|
||||
return (output("<p>Template unavailable.</p>"));
|
||||
}
|
||||
QRecord template = templates.getFirst();
|
||||
if("MJML".equals(template.getValueString("kind")) && version.getValueString("compiledHtml") == null)
|
||||
{
|
||||
if(!com.kingsrook.qbits.email.permissions.EmailPermissionChecker.has(com.kingsrook.qbits.email.permissions.EmailPermissions.MANAGE))
|
||||
{
|
||||
return (output("<p>Not compiled yet. Activate the version or use Preview as a manager.</p>"));
|
||||
}
|
||||
EmailVersions.validate(template, version);
|
||||
}
|
||||
EmailTemplateRenderer renderer = new EmailTemplateRenderer(template, version,
|
||||
new EmailRequest().withVariables(VariableSchema.values(version.getValueString("sampleVariables"))));
|
||||
QRecord rendered = renderer.render("preview", "https://example.invalid/unsubscribe/preview", false);
|
||||
QRecord brand = EmailBrands.resolve(null, template);
|
||||
String html = "<div style=\"min-width:0;overflow-wrap:anywhere\"><h3>" + EmailTemplateEngine.escape(rendered.getValueString("subject")) + "</h3>"
|
||||
+ "<p>Sample preview. Inline rendering; mail clients may display it differently.</p>"
|
||||
+ sample(rendered.getValueString("htmlBody"), brand, false, 390) + sample(rendered.getValueString("htmlBody"), brand, true, 600)
|
||||
+ "<h4>Plain text</h4><pre style=\"white-space:pre-wrap;overflow-wrap:anywhere;max-width:100%\">"
|
||||
+ EmailTemplateEngine.escape(rendered.getValueString("textBody")) + "</pre></div>";
|
||||
return (output(html + "<p>" + EmailTemplateEngine.escape(String.join("; ", renderer.warnings())) + "</p>"));
|
||||
}
|
||||
catch(IllegalArgumentException e)
|
||||
{
|
||||
return (output("<p>Preview unavailable: " + EmailTemplateEngine.escape(e.getMessage()) + "</p>"));
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Safe inline sample with bounded width and no executable source or CSS.
|
||||
*******************************************************************************/
|
||||
private static String sample(String html, QRecord brand, boolean dark, int width)
|
||||
{
|
||||
var document = Jsoup.parse(html);
|
||||
document.select("head,script,style,iframe,form").remove();
|
||||
document.select(".email-logo-" + (dark ? "light" : "dark")).remove();
|
||||
String clean = Jsoup.clean(document.body().html(), Safelist.relaxed().addAttributes("td", "class"));
|
||||
document = Jsoup.parseBodyFragment(clean);
|
||||
document.select("table").forEach(e -> e.attr("style", "width:100%;table-layout:fixed;border-collapse:collapse"));
|
||||
document.select("td,th").forEach(e -> e.attr("style", "overflow-wrap:anywhere;padding:6px"));
|
||||
document.select("img").forEach(e -> e.attr("style", "max-width:100%;height:auto"));
|
||||
document.select("pre").forEach(e -> e.attr("style", "white-space:pre-wrap;overflow-wrap:anywhere"));
|
||||
document.select("a").forEach(e -> e.attr("style", "color:inherit;overflow-wrap:anywhere"));
|
||||
document.select(".email-content").forEach(e -> e.attr("style", e.attr("style") + ";background-color:"
|
||||
+ ContrastCheck.color(brand, dark ? "darkBackgroundColor" : "backgroundColor") + ";color:"
|
||||
+ ContrastCheck.color(brand, dark ? "darkTextColor" : "textColor")));
|
||||
document.select(".email-button").forEach(e -> e.attr("style", "padding:12px 20px;background-color:" + ContrastCheck.color(brand, dark ? "darkPrimaryColor" : "primaryColor")
|
||||
+ ";color:" + ContrastCheck.color(brand, dark ? "darkButtonTextColor" : "buttonTextColor") + "!important"));
|
||||
document.select(".email-button a").forEach(e -> e.attr("style", "color:" + ContrastCheck.color(brand, dark ? "darkButtonTextColor" : "buttonTextColor") + "!important;display:block;overflow-wrap:anywhere"));
|
||||
document.select("[class]").forEach(e -> e.removeAttr("class"));
|
||||
return ("<h4>" + (dark ? "Dark" : "Light") + " · " + width + "px maximum</h4><div style=\"box-sizing:border-box;width:100%;max-width:"
|
||||
+ width + "px;padding:12px;overflow-wrap:anywhere;background-color:" + ContrastCheck.color(brand, dark ? "darkBackgroundColor" : "backgroundColor")
|
||||
+ ";color:" + ContrastCheck.color(brand, dark ? "darkTextColor" : "textColor") + "\">" + document.body().html() + "</div>");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Standard widget response.
|
||||
*******************************************************************************/
|
||||
private static RenderWidgetOutput output(String html)
|
||||
{
|
||||
return (new RenderWidgetOutput(new RawHTML(null, html)));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,172 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.template;
|
||||
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Objects;
|
||||
import java.util.Optional;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTables;
|
||||
import com.kingsrook.qbits.email.render.EmailBrands;
|
||||
import com.kingsrook.qbits.email.support.EmailStore;
|
||||
import com.kingsrook.qbits.email.support.SystemWrites;
|
||||
import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.statusmessages.BadInputStatusMessage;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Lifecycle and template edit permissions apply equally to forms and APIs.
|
||||
*******************************************************************************/
|
||||
public class EmailTemplateCustomizers implements TableCustomizerInterface
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Merge sparse updates and protect stored lifecycle state, not caller claims.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public List<QRecord> preInsertOrUpdate(AbstractActionInput input, List<QRecord> rows, boolean preview, Optional<List<QRecord>> oldRecords) throws QException
|
||||
{
|
||||
if(SystemWrites.active())
|
||||
{
|
||||
return (rows);
|
||||
}
|
||||
for(QRecord row : rows)
|
||||
{
|
||||
QRecord old = oldRecords.orElse(List.of()).stream().filter(v -> Objects.equals(v.getValue("id"), row.getValue("id"))).findFirst().orElse(new QRecord());
|
||||
QRecord merged = new QRecord();
|
||||
merged.getValues().putAll(old.getValues());
|
||||
merged.getValues().putAll(row.getValues());
|
||||
try
|
||||
{
|
||||
String table = input instanceof com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput insert ? insert.getTableName()
|
||||
: ((com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput) input).getTableName();
|
||||
if(EmailTables.BRAND.equals(table))
|
||||
{
|
||||
EmailBrands.validate(merged);
|
||||
}
|
||||
else if(EmailTables.TEMPLATE.equals(table))
|
||||
{
|
||||
EmailVersions.requireEdit(old.getValue("id") == null ? merged : old, false);
|
||||
protectedFields(row, old, List.of("phiApproved", "approvedVersionId", "phiApprovedBy", "phiApprovedDate", "activeVersionId"));
|
||||
if(Boolean.TRUE.equals(old.getValueBoolean("phiApproved")))
|
||||
{
|
||||
protectedFields(row, old, List.of("kind", "category", "brandId", "name"));
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
QRecord parent = EmailStore.get(EmailTables.TEMPLATE, merged.getValueInteger("templateId"), null);
|
||||
if(parent == null)
|
||||
{
|
||||
throw (new IllegalArgumentException("Template is required"));
|
||||
}
|
||||
EmailVersions.requireEdit(parent, false);
|
||||
if(old.getValue("id") != null)
|
||||
{
|
||||
protectedFields(row, old, List.of("templateId", "versionNumber", "status", "compiledHtml", "activatedDate"));
|
||||
immutable(old, parent);
|
||||
}
|
||||
else
|
||||
{
|
||||
if(merged.getValueString("status") != null && !"DRAFT".equals(merged.getValueString("status")))
|
||||
{
|
||||
throw (new IllegalArgumentException("New versions must be DRAFT"));
|
||||
}
|
||||
row.setValue("status", "DRAFT");
|
||||
row.setValue("compiledHtml", null);
|
||||
}
|
||||
new VariableSchema(merged.getValueString("variablesSchema"));
|
||||
}
|
||||
}
|
||||
catch(IllegalArgumentException e)
|
||||
{
|
||||
row.addError(new BadInputStatusMessage(e.getMessage()));
|
||||
}
|
||||
}
|
||||
return (rows);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Lifecycle-owned fields cannot be changed through ordinary CRUD.
|
||||
*******************************************************************************/
|
||||
private static void protectedFields(QRecord row, QRecord old, List<String> fields)
|
||||
{
|
||||
for(String field : fields)
|
||||
{
|
||||
Object value = row.getValue(field);
|
||||
if(row.getValues().containsKey(field) && !Objects.equals(value, old.getValue(field)) && !(old.getValue("id") == null && Boolean.FALSE.equals(value)))
|
||||
{
|
||||
throw (new IllegalArgumentException("Use the version actions to change " + field));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Approving a draft freezes its content, preventing approval substitution.
|
||||
*******************************************************************************/
|
||||
private static void immutable(QRecord version, QRecord template)
|
||||
{
|
||||
if(!"DRAFT".equals(version.getValueString("status")) || Objects.equals(version.getValue("id"), template.getValue("approvedVersionId")))
|
||||
{
|
||||
throw (new IllegalArgumentException("ACTIVE, RETIRED and PHI-approved versions cannot be edited; use New Version"));
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Preserve active/history rows and the same edit grants on deletion.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public List<QRecord> preDelete(DeleteInput input, List<QRecord> rows, boolean preview) throws QException
|
||||
{
|
||||
if(SystemWrites.active())
|
||||
{
|
||||
return (rows);
|
||||
}
|
||||
for(QRecord row : rows)
|
||||
{
|
||||
try
|
||||
{
|
||||
if(EmailTables.VERSION.equals(input.getTableName()))
|
||||
{
|
||||
QRecord parent = EmailStore.get(EmailTables.TEMPLATE, row.getValueInteger("templateId"), input.getTransaction());
|
||||
EmailVersions.requireEdit(parent, false);
|
||||
immutable(row, parent);
|
||||
}
|
||||
else if(EmailTables.TEMPLATE.equals(input.getTableName()))
|
||||
{
|
||||
EmailVersions.requireEdit(row, false);
|
||||
if(!SystemWrites.query(EmailTables.VERSION, EmailStore.equal("templateId", row.getValueInteger("id")), input.getTransaction()).isEmpty())
|
||||
{
|
||||
throw (new IllegalArgumentException("Templates with versions cannot be deleted"));
|
||||
}
|
||||
}
|
||||
}
|
||||
catch(IllegalArgumentException e)
|
||||
{
|
||||
row.addError(new BadInputStatusMessage(e.getMessage()));
|
||||
}
|
||||
}
|
||||
return (rows);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,418 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.template;
|
||||
|
||||
|
||||
import java.math.BigDecimal;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.ArrayDeque;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Collection;
|
||||
import java.util.Deque;
|
||||
import java.util.LinkedHashSet;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import java.util.regex.Pattern;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Small parsed template language: values, fallback, if and each, without code.
|
||||
*******************************************************************************/
|
||||
public class EmailTemplateEngine
|
||||
{
|
||||
private static final Pattern URL_ATTRIBUTE = Pattern.compile("(?i)(?:^|[\\s<])(href|src|action|background)\\s*=\\s*[\"']$");
|
||||
private static final Pattern RENDERED_URL = Pattern.compile("(?is)\\b(href|src|action|background)(\\s*=\\s*)([\"'])(.*?)\\3");
|
||||
|
||||
/*******************************************************************************
|
||||
** One budget shared by all parts and nested blocks of a render.
|
||||
*******************************************************************************/
|
||||
public static class Budget
|
||||
{
|
||||
private int iterations;
|
||||
private int bytes;
|
||||
|
||||
/*******************************************************************************
|
||||
** Count work before entering a loop body.
|
||||
*******************************************************************************/
|
||||
private void iterate()
|
||||
{
|
||||
if(++iterations > 10_000)
|
||||
{
|
||||
throw (new IllegalArgumentException("Template output is too large"));
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Bound UTF-8 output before appending it.
|
||||
*******************************************************************************/
|
||||
private void append(StringBuilder out, String value)
|
||||
{
|
||||
if(value.length() > 1_048_576 || (bytes += value.getBytes(StandardCharsets.UTF_8).length) > 1_048_576)
|
||||
{
|
||||
throw (new IllegalArgumentException("Template output is too large"));
|
||||
}
|
||||
out.append(value);
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Rendering context. Markdown uses entities for all punctuation so values
|
||||
** cannot inject Markdown links, images or block syntax.
|
||||
*******************************************************************************/
|
||||
public enum Context
|
||||
{
|
||||
HTML, TEXT, MARKDOWN
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** A parsed literal, expression or block.
|
||||
*******************************************************************************/
|
||||
private record Node(String kind, String value, String fallback, int line, List<Node> children, String attribute)
|
||||
{
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Validated immutable parse result.
|
||||
*******************************************************************************/
|
||||
public static class Parsed
|
||||
{
|
||||
private final List<Node> nodes;
|
||||
private final Set<String> references;
|
||||
|
||||
/*******************************************************************************
|
||||
** Store parsed nodes.
|
||||
*******************************************************************************/
|
||||
private Parsed(List<Node> nodes, Set<String> references)
|
||||
{
|
||||
this.nodes = nodes;
|
||||
this.references = Set.copyOf(references);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Render the already validated syntax tree.
|
||||
*******************************************************************************/
|
||||
public String render(Map<String, ?> values, Context context)
|
||||
{
|
||||
return (render(values, context, new Budget(), new ArrayList<>()));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Bindings referenced by values and block conditions, including nested blocks.
|
||||
*******************************************************************************/
|
||||
public Set<String> references()
|
||||
{
|
||||
return (references);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Render using the message-wide budget and safe author-facing warnings.
|
||||
*******************************************************************************/
|
||||
public String render(Map<String, ?> values, Context context, Budget budget, List<String> warnings)
|
||||
{
|
||||
StringBuilder out = new StringBuilder();
|
||||
renderNodes(nodes, values, null, -1, context, budget, warnings, out);
|
||||
return (context == Context.HTML ? validateRenderedUrls(out.toString(), warnings) : out.toString());
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Parse once, checking balanced blocks and every variable, with line numbers.
|
||||
*******************************************************************************/
|
||||
public Parsed parse(String source, VariableSchema schema)
|
||||
{
|
||||
source = source == null ? "" : source;
|
||||
List<Node> root = new ArrayList<>();
|
||||
Deque<Node> stack = new ArrayDeque<>();
|
||||
Set<String> references = new LinkedHashSet<>();
|
||||
int position = 0;
|
||||
int line = 1;
|
||||
while(position < source.length())
|
||||
{
|
||||
List<Node> current = stack.isEmpty() ? root : stack.peek().children();
|
||||
int start = source.indexOf("{{", position);
|
||||
if(start < 0)
|
||||
{
|
||||
current.add(new Node("literal", source.substring(position), null, 0, List.of(), null));
|
||||
break;
|
||||
}
|
||||
current.add(new Node("literal", source.substring(position, start), null, 0, List.of(), null));
|
||||
for(int i = position; i < start; i++)
|
||||
{
|
||||
if(source.charAt(i) == '\n')
|
||||
{
|
||||
line++;
|
||||
}
|
||||
}
|
||||
int end = source.indexOf("}}", start + 2);
|
||||
if(end < 0)
|
||||
{
|
||||
throw (error(line, "Unclosed expression"));
|
||||
}
|
||||
String expression = source.substring(start + 2, end).trim();
|
||||
if(expression.startsWith("/"))
|
||||
{
|
||||
if(stack.isEmpty() || !stack.peek().kind().equals(expression.substring(1)))
|
||||
{
|
||||
throw (error(line, "Block nesting error"));
|
||||
}
|
||||
stack.pop();
|
||||
}
|
||||
else
|
||||
{
|
||||
String kind = expression.startsWith("#if ") ? "if" : expression.startsWith("#each ") ? "each" : "value";
|
||||
String binding = kind.equals("value") ? expression : expression.substring(kind.length() + 2).trim();
|
||||
String[] parts = binding.split("\\|", 2);
|
||||
String name = parts[0].trim();
|
||||
boolean inEach = stack.stream().anyMatch(n -> n.kind().equals("each"));
|
||||
boolean local = name.equals("this") || name.startsWith("this.") || name.equals("@index");
|
||||
if(!name.matches("(?:[A-Za-z_][A-Za-z0-9_]*(?:\\.[A-Za-z_][A-Za-z0-9_]*)*|@index)")
|
||||
|| (local ? !inEach : !schema.knows(name)) || !kind.equals("value") && parts.length > 1)
|
||||
{
|
||||
throw (error(line, "Unknown or unsupported variable " + name));
|
||||
}
|
||||
if(!local)
|
||||
{
|
||||
references.add(name);
|
||||
}
|
||||
var attribute = URL_ATTRIBUTE.matcher(source.substring(position, start));
|
||||
Node node = new Node(kind, name, parts.length == 2 ? parts[1] : "", line, new ArrayList<>(), attribute.find() ? attribute.group(1).toLowerCase(java.util.Locale.ROOT) : null);
|
||||
current.add(node);
|
||||
if(!kind.equals("value"))
|
||||
{
|
||||
if(stack.size() >= 8)
|
||||
{
|
||||
throw (error(line, "Blocks exceed 8 levels"));
|
||||
}
|
||||
stack.push(node);
|
||||
}
|
||||
}
|
||||
for(int i = start; i < end + 2; i++)
|
||||
{
|
||||
if(source.charAt(i) == '\n')
|
||||
{
|
||||
line++;
|
||||
}
|
||||
}
|
||||
position = end + 2;
|
||||
}
|
||||
if(!stack.isEmpty())
|
||||
{
|
||||
throw (error(stack.peek().line(), "Unclosed " + stack.peek().kind() + " block"));
|
||||
}
|
||||
return (new Parsed(root, references));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Render syntax nodes; loop scope is restored on every recursive return.
|
||||
*******************************************************************************/
|
||||
private static void renderNodes(List<Node> nodes, Map<String, ?> values, Object item, int index, Context context, Budget budget, List<String> warnings, StringBuilder out)
|
||||
{
|
||||
for(Node node : nodes)
|
||||
{
|
||||
Object value = node.value().equals("this") ? item : node.value().startsWith("this.") ? lookup(item, node.value().substring(5))
|
||||
: node.value().equals("@index") ? index : lookup(values, node.value());
|
||||
switch(node.kind())
|
||||
{
|
||||
case "literal" -> budget.append(out, node.value());
|
||||
case "value" ->
|
||||
{
|
||||
String text = value == null || value.toString().isEmpty() ? node.fallback() : value.toString();
|
||||
if(context == Context.HTML && node.attribute() != null && !text.isEmpty())
|
||||
{
|
||||
try
|
||||
{
|
||||
VariableSchema.url(text, "href".equals(node.attribute()));
|
||||
}
|
||||
catch(IllegalArgumentException e)
|
||||
{
|
||||
text = "";
|
||||
String warning = "Unsafe URL omitted for " + node.value();
|
||||
if(!warnings.contains(warning))
|
||||
{
|
||||
warnings.add(warning);
|
||||
}
|
||||
}
|
||||
}
|
||||
if(text.length() > 1_048_576)
|
||||
{
|
||||
throw (new IllegalArgumentException("Template output is too large"));
|
||||
}
|
||||
budget.append(out, encode(text, context));
|
||||
}
|
||||
case "if" ->
|
||||
{
|
||||
if(truthy(value))
|
||||
{
|
||||
renderNodes(node.children(), values, item, index, context, budget, warnings, out);
|
||||
}
|
||||
}
|
||||
case "each" ->
|
||||
{
|
||||
if(value instanceof List<?> list)
|
||||
{
|
||||
for(int i = 0; i < list.size(); i++)
|
||||
{
|
||||
budget.iterate();
|
||||
renderNodes(node.children(), values, list.get(i), i, context, budget, warnings, out);
|
||||
}
|
||||
}
|
||||
}
|
||||
default -> throw (new IllegalStateException("Invalid template node"));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Expressions in attributes require quotes, including after literal prefixes.
|
||||
*******************************************************************************/
|
||||
public static void validateHtml(String source)
|
||||
{
|
||||
boolean tag = false;
|
||||
char quote = 0;
|
||||
for(int i = 0; source != null && i < source.length(); i++)
|
||||
{
|
||||
char c = source.charAt(i);
|
||||
if(tag && quote == 0 && source.startsWith("{{", i))
|
||||
{
|
||||
throw (new IllegalArgumentException("Template expressions in attributes must be quoted"));
|
||||
}
|
||||
if(quote != 0)
|
||||
{
|
||||
if(c == quote)
|
||||
{
|
||||
quote = 0;
|
||||
}
|
||||
}
|
||||
else if(c == '<')
|
||||
{
|
||||
tag = true;
|
||||
}
|
||||
else if(c == '>')
|
||||
{
|
||||
tag = false;
|
||||
}
|
||||
else if(tag && (c == '\"' || c == '\''))
|
||||
{
|
||||
quote = c;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Recheck complete attributes after block expansion, so whitespace, split
|
||||
** values and conditional expressions cannot hide the URL context.
|
||||
*******************************************************************************/
|
||||
private static String validateRenderedUrls(String html, List<String> warnings)
|
||||
{
|
||||
var matcher = RENDERED_URL.matcher(html);
|
||||
StringBuilder result = new StringBuilder();
|
||||
while(matcher.find())
|
||||
{
|
||||
String value = org.jsoup.parser.Parser.unescapeEntities(matcher.group(4), true);
|
||||
try
|
||||
{
|
||||
if(!value.isEmpty())
|
||||
{
|
||||
VariableSchema.url(value, "href".equalsIgnoreCase(matcher.group(1)));
|
||||
}
|
||||
matcher.appendReplacement(result, java.util.regex.Matcher.quoteReplacement(matcher.group()));
|
||||
}
|
||||
catch(IllegalArgumentException e)
|
||||
{
|
||||
String warning = "Unsafe " + matcher.group(1).toLowerCase(java.util.Locale.ROOT) + " URL omitted";
|
||||
if(!warnings.contains(warning))
|
||||
{
|
||||
warnings.add(warning);
|
||||
}
|
||||
matcher.appendReplacement(result, java.util.regex.Matcher.quoteReplacement(matcher.group(1) + matcher.group(2) + matcher.group(3) + matcher.group(3)));
|
||||
}
|
||||
}
|
||||
matcher.appendTail(result);
|
||||
return (result.toString());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Dotted paths traverse maps only; no reflection, methods or arbitrary code.
|
||||
*******************************************************************************/
|
||||
public static Object lookup(Object values, String path)
|
||||
{
|
||||
if(values instanceof Map<?, ?> map)
|
||||
{
|
||||
if(map.containsKey(path))
|
||||
{
|
||||
return (map.get(path));
|
||||
}
|
||||
int dot = path.indexOf('.');
|
||||
return (dot < 0 ? map.get(path) : lookup(map.get(path.substring(0, dot)), path.substring(dot + 1)));
|
||||
}
|
||||
return (null);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Explicit language truthiness.
|
||||
*******************************************************************************/
|
||||
public static boolean truthy(Object value)
|
||||
{
|
||||
if(value == null || Boolean.FALSE.equals(value) || value instanceof CharSequence s && s.isEmpty()
|
||||
|| value instanceof Collection<?> c && c.isEmpty() || value instanceof Map<?, ?> m && m.isEmpty())
|
||||
{
|
||||
return (false);
|
||||
}
|
||||
return (!(value instanceof Number n) || new BigDecimal(n.toString()).compareTo(BigDecimal.ZERO) != 0);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Escape values exactly once for the consuming grammar.
|
||||
*******************************************************************************/
|
||||
public static String encode(String value, Context context)
|
||||
{
|
||||
if(context == Context.TEXT)
|
||||
{
|
||||
return (value);
|
||||
}
|
||||
if(context == Context.MARKDOWN)
|
||||
{
|
||||
StringBuilder encoded = new StringBuilder();
|
||||
value.codePoints().forEach(c -> encoded.append(Character.isLetterOrDigit(c) || c == ' ' ? new String(Character.toChars(c)) : "&#" + c + ";"));
|
||||
return (encoded.toString());
|
||||
}
|
||||
return (escape(value));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Attribute and text escaping, including both quote types.
|
||||
*******************************************************************************/
|
||||
public static String escape(Object value)
|
||||
{
|
||||
return (value == null ? "" : value.toString().replace("&", "&").replace("<", "<").replace(">", ">")
|
||||
.replace("\"", """).replace("'", "'"));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Stable author-facing parse diagnostic.
|
||||
*******************************************************************************/
|
||||
private static IllegalArgumentException error(int line, String message)
|
||||
{
|
||||
return (new IllegalArgumentException("Line " + line + ": " + message));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,453 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.template;
|
||||
|
||||
|
||||
import java.io.Serializable;
|
||||
import java.util.ArrayList;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Objects;
|
||||
import com.kingsrook.qbits.email.EmailRuntime;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTables;
|
||||
import com.kingsrook.qbits.email.outbox.EmailCategory;
|
||||
import com.kingsrook.qbits.email.outbox.EmailRequest;
|
||||
import com.kingsrook.qbits.email.phi.EmailBodyEncryption;
|
||||
import com.kingsrook.qbits.email.phi.EmailClassifications;
|
||||
import com.kingsrook.qbits.email.phi.EmailClassifications.Level;
|
||||
import com.kingsrook.qbits.email.phi.EmailValueTokens;
|
||||
import com.kingsrook.qbits.email.render.EmailBrands;
|
||||
import com.kingsrook.qbits.email.render.EmailLayout;
|
||||
import com.kingsrook.qbits.email.render.EmailMarkdown;
|
||||
import com.kingsrook.qbits.email.render.HtmlToText;
|
||||
import com.kingsrook.qbits.email.support.EmailStore;
|
||||
import com.kingsrook.qbits.email.support.SystemWrites;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Queue-time template resolution and rendering with sender-visible record data.
|
||||
*******************************************************************************/
|
||||
public final class EmailTemplateRenderer
|
||||
{
|
||||
private final QRecord template;
|
||||
private final QRecord version;
|
||||
private final QRecord brand;
|
||||
private final VariableSchema schema;
|
||||
private final Map<String, Serializable> values;
|
||||
private final Map<String, EmailRequest.RecordVariable> records;
|
||||
private final EmailCategory category;
|
||||
private final boolean approved;
|
||||
private final List<String> warnings = new ArrayList<>();
|
||||
|
||||
/*******************************************************************************
|
||||
** Resolve a template's active locale, falling back only to English.
|
||||
*******************************************************************************/
|
||||
public static EmailTemplateRenderer prepare(EmailRequest request) throws QException
|
||||
{
|
||||
if("email_secure_message".equals(request.getTemplateName()))
|
||||
{
|
||||
EmailBuiltIns.ensure();
|
||||
}
|
||||
List<QRecord> templates = SystemWrites.query(EmailTables.TEMPLATE, EmailStore.equal("name", request.getTemplateName()));
|
||||
if(templates.size() != 1)
|
||||
{
|
||||
throw (new IllegalArgumentException("Unknown email template"));
|
||||
}
|
||||
QRecord template = templates.getFirst();
|
||||
List<QRecord> versions = SystemWrites.query(EmailTables.VERSION, EmailStore.equal("templateId", template.getValueInteger("id"))
|
||||
.withCriteria(new QFilterCriteria("status", QCriteriaOperator.EQUALS, "ACTIVE")));
|
||||
QRecord version = versions.stream().filter(v -> Objects.equals(request.getLocale(), v.getValueString("locale"))).findFirst()
|
||||
.orElseGet(() -> versions.stream().filter(v -> "en".equals(v.getValueString("locale"))).findFirst().orElse(null));
|
||||
if(version == null)
|
||||
{
|
||||
throw (new IllegalArgumentException("Template has no active version for this locale or en"));
|
||||
}
|
||||
return (new EmailTemplateRenderer(template, version, request));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Validate schema, provenance and all template syntax before any outbox write.
|
||||
*******************************************************************************/
|
||||
public EmailTemplateRenderer(QRecord template, QRecord version, EmailRequest request) throws QException
|
||||
{
|
||||
this.template = template;
|
||||
this.version = version;
|
||||
approved = Boolean.TRUE.equals(template.getValueBoolean("phiApproved")) && version.getValue("id") != null
|
||||
&& Objects.equals(template.getValue("approvedVersionId"), version.getValue("id"));
|
||||
brand = EmailBrands.resolve(approved ? null : request.getBrandName(), template);
|
||||
if(approved)
|
||||
{
|
||||
for(String field : List.of("logoUrl", "logoDarkUrl"))
|
||||
{
|
||||
if(brand.getValueString(field) != null && !EmailBrands.allowedImage(brand.getValueString(field), brand))
|
||||
{
|
||||
brand.setValue(field, null);
|
||||
warnings.add("Brand logo omitted: image host is not allowed");
|
||||
}
|
||||
}
|
||||
}
|
||||
EmailBrands.validate(brand);
|
||||
schema = new VariableSchema(version.getValueString("variablesSchema"));
|
||||
records = new LinkedHashMap<>(request.getRecordVariables());
|
||||
java.util.Set<String> references = new java.util.LinkedHashSet<>();
|
||||
for(String field : List.of("subject", "preheader", "bodySource", "textSource", "compiledHtml"))
|
||||
{
|
||||
references.addAll(new EmailTemplateEngine().parse(version.getValueString(field), schema).references());
|
||||
}
|
||||
if(!"MARKDOWN".equals(template.getValueString("kind")))
|
||||
{
|
||||
EmailTemplateEngine.validateHtml(version.getValueString("bodySource"));
|
||||
EmailTemplateEngine.validateHtml(version.getValueString("compiledHtml"));
|
||||
}
|
||||
Map<String, Serializable> supplied = new LinkedHashMap<>(request.getVariables());
|
||||
for(var binding : records.entrySet())
|
||||
{
|
||||
supplied.keySet().removeIf(key -> key.startsWith(binding.getKey() + "."));
|
||||
QTableMetaData table = QContext.getQInstance().getTable(binding.getValue().table());
|
||||
if(table == null)
|
||||
{
|
||||
throw (new IllegalArgumentException("Record variable table is unavailable"));
|
||||
}
|
||||
java.util.Set<String> fields = references.stream().filter(path -> path.startsWith(binding.getKey() + "."))
|
||||
.map(path -> path.substring(binding.getKey().length() + 1).split("\\.")[0]).collect(java.util.stream.Collectors.toSet());
|
||||
if(fields.isEmpty())
|
||||
{
|
||||
supplied.put(binding.getKey(), new LinkedHashMap<>());
|
||||
continue;
|
||||
}
|
||||
// Never use SystemWrites here: masking and record locks belong to the sender.
|
||||
List<QRecord> rows = new QueryAction().execute(new QueryInput(table.getName()).withShouldFetchHeavyFields(true).withFieldNamesToInclude(fields)
|
||||
.withFilter(EmailStore.equal(table.getPrimaryKeyField(), binding.getValue().id()).withLimit(1))).getRecords();
|
||||
if(rows.isEmpty())
|
||||
{
|
||||
throw (new IllegalArgumentException("Record variable is unavailable to the sender"));
|
||||
}
|
||||
Map<String, Serializable> visible = new LinkedHashMap<>();
|
||||
fields.forEach(field -> visible.put(field, rows.getFirst().getValue(field)));
|
||||
for(var field : table.getFields().values())
|
||||
{
|
||||
var lock = field.getFieldSecurityLock();
|
||||
if(lock != null && fields.contains(field.getName()))
|
||||
{
|
||||
var session = QContext.getQSession();
|
||||
List<Serializable> keys = session == null ? List.of() : session.getSecurityKeyValues(lock.getSecurityKeyType());
|
||||
boolean override = keys != null && lock.getOverrideValues() != null && keys.stream().anyMatch(lock.getOverrideValues()::contains);
|
||||
boolean allowed = lock.getDefaultBehavior() == com.kingsrook.qqq.backend.core.model.metadata.security.FieldSecurityLock.Behavior.ALLOW;
|
||||
if(override ? allowed : !allowed)
|
||||
{
|
||||
visible.put(field.getName(), "************");
|
||||
}
|
||||
}
|
||||
}
|
||||
supplied.put(binding.getKey(), new LinkedHashMap<>(visible));
|
||||
}
|
||||
for(String name : supplied.keySet())
|
||||
{
|
||||
if(!schema.knows(name) || name.startsWith("brand.") || name.equals("message.id") || name.equals("now") || name.equals("unsubscribe_url"))
|
||||
{
|
||||
throw (new IllegalArgumentException("Unknown or reserved input variable " + name));
|
||||
}
|
||||
}
|
||||
values = schema.validate(supplied);
|
||||
category = EmailCategory.valueOf(Objects.requireNonNullElse(template.getValueString("category"), "TRANSACTIONAL"));
|
||||
if(request.getCategory() != null && request.getCategory() != category)
|
||||
{
|
||||
throw (new IllegalArgumentException("Template category cannot be overridden"));
|
||||
}
|
||||
if(approved && category == EmailCategory.MARKETING)
|
||||
{
|
||||
throw (new IllegalArgumentException("PHI-approved templates cannot send MARKETING email"));
|
||||
}
|
||||
for(VariableSchema.Entry entry : schema.entries().values())
|
||||
{
|
||||
requireApproval(entry.name(), classification(entry.name()));
|
||||
}
|
||||
inspect(values, "");
|
||||
for(String field : List.of("subject", "preheader", "bodySource", "textSource"))
|
||||
{
|
||||
new EmailTemplateEngine().parse(version.getValueString(field), schema);
|
||||
}
|
||||
if(version.getValueString("subject") == null || version.getValueString("bodySource") == null)
|
||||
{
|
||||
throw (new IllegalArgumentException("A template subject and body are required"));
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Check actual nested fields as well as declared but absent bindings.
|
||||
*******************************************************************************/
|
||||
private void inspect(Object object, String path)
|
||||
{
|
||||
if(!path.isEmpty())
|
||||
{
|
||||
requireApproval(path, classification(path));
|
||||
}
|
||||
if(object instanceof Map<?, ?> map)
|
||||
{
|
||||
map.forEach((key, value) -> inspect(value, path.isEmpty() ? key.toString() : path + "." + key));
|
||||
}
|
||||
else if(object instanceof List<?> list)
|
||||
{
|
||||
list.forEach(value -> inspect(value, path));
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** A classification failure names only the binding, never its value.
|
||||
*******************************************************************************/
|
||||
private void requireApproval(String name, Level level)
|
||||
{
|
||||
if(EmailClassifications.sensitive(level) && !approved)
|
||||
{
|
||||
throw (new IllegalArgumentException("The variable " + name + " is " + level + "; this template is not approved for PHI."));
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Resolve record provenance before schema declarations; inherit parent levels.
|
||||
*******************************************************************************/
|
||||
private Level classification(String path)
|
||||
{
|
||||
VariableSchema.Entry declaration = schema.entries().get(path);
|
||||
if(declaration == null)
|
||||
{
|
||||
declaration = schema.entries().values().stream().filter(e -> path.startsWith(e.name() + "."))
|
||||
.max(java.util.Comparator.comparingInt(e -> e.name().length())).orElse(null);
|
||||
}
|
||||
String declared = declaration == null ? null : declaration.classification();
|
||||
for(var record : records.entrySet())
|
||||
{
|
||||
if(path.startsWith(record.getKey() + "."))
|
||||
{
|
||||
String field = path.substring(record.getKey().length() + 1).split("\\.")[0];
|
||||
return (EmailClassifications.ofField(QContext.getQInstance().getTable(record.getValue().table()), field, declared, EmailRuntime.config()));
|
||||
}
|
||||
}
|
||||
QTableMetaData table = declaration == null || declaration.table() == null ? null : QContext.getQInstance().getTable(declaration.table());
|
||||
if(declaration != null && declaration.table() != null && (table == null || table.getField(declaration.field()) == null))
|
||||
{
|
||||
throw (new IllegalArgumentException("Schema field binding is unavailable"));
|
||||
}
|
||||
return (EmailClassifications.ofField(table, declaration == null ? null : declaration.field(), declared, EmailRuntime.config()));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Validate the actual provider at queue and delivery time.
|
||||
*******************************************************************************/
|
||||
public void validateProvider(QRecord provider)
|
||||
{
|
||||
if(approved)
|
||||
{
|
||||
checkProvider(provider);
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Recheck mutable provider attestations before encrypted content leaves storage.
|
||||
*******************************************************************************/
|
||||
public static void checkProvider(QRecord provider)
|
||||
{
|
||||
if(!Boolean.TRUE.equals(provider.getValueBoolean("phiAllowed")) || Boolean.TRUE.equals(provider.getValueBoolean("trackOpens"))
|
||||
|| Boolean.TRUE.equals(provider.getValueBoolean("trackClicks")))
|
||||
{
|
||||
throw (new IllegalArgumentException("PHI email needs a phi_allowed provider with open and click tracking disabled"));
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Render with the real message identity; no compiled source is rebuilt at send.
|
||||
*******************************************************************************/
|
||||
public QRecord render(String messageId, String unsubscribe, boolean encrypt)
|
||||
{
|
||||
Map<String, Serializable> context = new LinkedHashMap<>(values);
|
||||
for(String field : List.of("label", "name", "siteUrl", "logoUrl", "logoDarkUrl", "logoAlt", "postalAddress", "primaryColor", "textColor", "backgroundColor", "darkBackgroundColor", "darkTextColor", "buttonTextColor", "darkPrimaryColor", "darkButtonTextColor", "footerMarkdown"))
|
||||
{
|
||||
context.put("brand." + field.replaceAll("([a-z])([A-Z])", "$1_$2").toLowerCase(java.util.Locale.ROOT), brand.getValueString(field));
|
||||
}
|
||||
context.put("message.id", messageId);
|
||||
context.put("now", EmailRuntime.clock().instant().toString());
|
||||
context.put("unsubscribe_url", category == EmailCategory.NOTIFICATION || category == EmailCategory.MARKETING ? unsubscribe : "");
|
||||
EmailTemplateEngine engine = new EmailTemplateEngine();
|
||||
EmailTemplateEngine.Budget budget = new EmailTemplateEngine.Budget();
|
||||
String subject = engine.parse(version.getValueString("subject"), schema).render(context, EmailTemplateEngine.Context.TEXT, budget, warnings);
|
||||
if(subject.contains("\r") || subject.contains("\n"))
|
||||
{
|
||||
throw (new IllegalArgumentException("Subject must be one line"));
|
||||
}
|
||||
String kind = template.getValueString("kind");
|
||||
String source = "MJML".equals(kind) ? version.getValueString("compiledHtml") : version.getValueString("bodySource");
|
||||
if(source == null)
|
||||
{
|
||||
throw (new IllegalArgumentException("MJML version must be compiled before sending"));
|
||||
}
|
||||
String body = engine.parse(source, schema).render(context, "MARKDOWN".equals(kind) ? EmailTemplateEngine.Context.MARKDOWN : EmailTemplateEngine.Context.HTML, budget, warnings);
|
||||
String preheader = engine.parse(version.getValueString("preheader"), schema).render(context, EmailTemplateEngine.Context.TEXT, budget, warnings);
|
||||
String html = "MARKDOWN".equals(kind) ? EmailLayout.render(EmailMarkdown.render(body, brand), preheader, brand, category, unsubscribe)
|
||||
: EmailLayout.appendFooter(body, brand, category, unsubscribe);
|
||||
if(approved)
|
||||
{
|
||||
html = restrictImages(html);
|
||||
}
|
||||
String text = version.getValueString("textSource") == null ? HtmlToText.convert(html)
|
||||
: engine.parse(version.getValueString("textSource"), schema).render(context, EmailTemplateEngine.Context.TEXT, budget, warnings);
|
||||
if((long) subject.getBytes(java.nio.charset.StandardCharsets.UTF_8).length + html.getBytes(java.nio.charset.StandardCharsets.UTF_8).length
|
||||
+ text.getBytes(java.nio.charset.StandardCharsets.UTF_8).length > 1_048_576)
|
||||
{
|
||||
throw (new IllegalArgumentException("Template output is too large"));
|
||||
}
|
||||
return (new QRecord().withValue("subject", approved && encrypt ? EmailBodyEncryption.encrypt(subject, "subject", messageId) : subject)
|
||||
.withValue("htmlBody", approved && encrypt ? EmailBodyEncryption.encrypt(html, "htmlBody", messageId) : html)
|
||||
.withValue("textBody", approved && encrypt ? EmailBodyEncryption.encrypt(text, "textBody", messageId) : text)
|
||||
.withValue("variablesSnapshot", VariableSchema.json(snapshot(values, "", messageId)))
|
||||
.withValue("templateId", template.getValue("id")).withValue("templateVersionId", version.getValue("id"))
|
||||
.withValue("brandId", brand.getValue("id")).withValue("locale", version.getValue("locale"))
|
||||
.withValue("category", category.name()).withValue("phiApproved", approved));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Recursive redaction prevents nested records or lists bypassing tokenization.
|
||||
*******************************************************************************/
|
||||
private Object snapshot(Object value, String path, String messageId)
|
||||
{
|
||||
if(!path.isEmpty() && (EmailClassifications.sensitive(classification(path)) || unclassifiedRecordField(path)))
|
||||
{
|
||||
for(var record : records.entrySet())
|
||||
{
|
||||
if(path.startsWith(record.getKey() + ".") && !path.substring(record.getKey().length() + 1).contains("."))
|
||||
{
|
||||
QTableMetaData table = QContext.getQInstance().getTable(record.getValue().table());
|
||||
var field = table.getField(path.substring(record.getKey().length() + 1));
|
||||
if(field != null)
|
||||
{
|
||||
return (EmailValueTokens.forRecordField(table, field, record.getValue().id(), (Serializable) value));
|
||||
}
|
||||
}
|
||||
}
|
||||
return (EmailValueTokens.forContext(List.of("email", template.getValueString("name"), path, messageId), (Serializable) value));
|
||||
}
|
||||
if(value instanceof Map<?, ?> map)
|
||||
{
|
||||
Map<String, Object> result = new LinkedHashMap<>();
|
||||
map.forEach((key, item) -> result.put(key.toString(), snapshot(item, path.isEmpty() ? key.toString() : path + "." + key, messageId)));
|
||||
return (result);
|
||||
}
|
||||
if(value instanceof List<?> list)
|
||||
{
|
||||
List<Object> result = new ArrayList<>();
|
||||
list.forEach(item -> result.add(snapshot(item, path, messageId)));
|
||||
return (result);
|
||||
}
|
||||
return (value == null || value instanceof Number || value instanceof Boolean ? value : value.toString());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Missing record classification fails closed only in the retained snapshot.
|
||||
*******************************************************************************/
|
||||
private boolean unclassifiedRecordField(String path)
|
||||
{
|
||||
for(var record : records.entrySet())
|
||||
{
|
||||
if(path.startsWith(record.getKey() + "."))
|
||||
{
|
||||
String field = path.substring(record.getKey().length() + 1).split("\\.")[0];
|
||||
return (!EmailClassifications.hasSource(QContext.getQInstance().getTable(record.getValue().table()), field, EmailRuntime.config()));
|
||||
}
|
||||
}
|
||||
return (false);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Warnings contain binding names and policy reasons, never input values.
|
||||
*******************************************************************************/
|
||||
public List<String> warnings()
|
||||
{
|
||||
return (List.copyOf(new java.util.LinkedHashSet<>(warnings)));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Apply the image policy after Markdown, layout and brand footer expansion.
|
||||
*******************************************************************************/
|
||||
private String restrictImages(String html)
|
||||
{
|
||||
var document = org.jsoup.Jsoup.parse(html);
|
||||
document.outputSettings().prettyPrint(false);
|
||||
document.select("[src],[background],[srcset],image").forEach(element ->
|
||||
{
|
||||
element.removeAttr("srcset");
|
||||
if(element.hasAttr("src") && !EmailBrands.allowedImage(element.attr("src"), brand))
|
||||
{
|
||||
element.remove();
|
||||
warnings.add("Image omitted: image host is not allowed");
|
||||
}
|
||||
if(element.hasAttr("background") && !EmailBrands.allowedImage(element.attr("background"), brand))
|
||||
{
|
||||
element.removeAttr("background");
|
||||
warnings.add("Image omitted: image host is not allowed");
|
||||
}
|
||||
if("image".equals(element.normalName()))
|
||||
{
|
||||
element.remove();
|
||||
warnings.add("SVG image omitted from approved email");
|
||||
}
|
||||
});
|
||||
org.jsoup.select.NodeTraversor.filter((node, depth) ->
|
||||
{
|
||||
if(node instanceof org.jsoup.nodes.Comment comment && comment.getData().toLowerCase(java.util.Locale.ROOT).contains("[if"))
|
||||
{
|
||||
warnings.add("Conditional markup omitted from approved email");
|
||||
return (org.jsoup.select.NodeFilter.FilterResult.REMOVE);
|
||||
}
|
||||
return (org.jsoup.select.NodeFilter.FilterResult.CONTINUE);
|
||||
}, document);
|
||||
// CSS image expressions can use escapes and image-set, so approved content
|
||||
// uses img elements instead of CSS images. Layout colors remain intact.
|
||||
document.select("[style]").forEach(element ->
|
||||
{
|
||||
String style = element.attr("style");
|
||||
if(style.toLowerCase(java.util.Locale.ROOT).contains("url") || style.contains("\\") || style.toLowerCase(java.util.Locale.ROOT).contains("image-set"))
|
||||
{
|
||||
element.removeAttr("style");
|
||||
warnings.add("CSS image omitted from approved email");
|
||||
}
|
||||
});
|
||||
document.select("style").forEach(element ->
|
||||
{
|
||||
String style = element.data();
|
||||
if(style.toLowerCase(java.util.Locale.ROOT).contains("url") || style.contains("\\") || style.toLowerCase(java.util.Locale.ROOT).contains("@import") || style.toLowerCase(java.util.Locale.ROOT).contains("image-set"))
|
||||
{
|
||||
element.remove();
|
||||
warnings.add("CSS image omitted from approved email");
|
||||
}
|
||||
});
|
||||
return (document.outerHtml());
|
||||
}
|
||||
|
||||
}
|
||||
@@ -0,0 +1,68 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.template;
|
||||
|
||||
|
||||
import java.util.HashMap;
|
||||
import java.util.HashSet;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Objects;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTables;
|
||||
import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.QueryOrGetInputInterface;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Human-readable version labels, resolving parent names as the current viewer.
|
||||
*******************************************************************************/
|
||||
public class EmailVersionLabels implements TableCustomizerInterface
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Batch parents to avoid one query per version, without bypassing security.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public List<QRecord> postQuery(QueryOrGetInputInterface input, List<QRecord> records) throws QException
|
||||
{
|
||||
var ids = records.stream().map(r -> r.getValueInteger("templateId")).filter(Objects::nonNull).distinct().toList();
|
||||
Map<Integer, String> names = new HashMap<>();
|
||||
if(!ids.isEmpty())
|
||||
{
|
||||
List<QRecord> templates = new QueryAction().execute(new QueryInput(EmailTables.TEMPLATE)
|
||||
.withFilter(new QQueryFilter().withCriteria("id", QCriteriaOperator.IN, ids))
|
||||
.withFieldNamesToInclude(new HashSet<>(List.of("id", "name")))).getRecords();
|
||||
templates.forEach(t -> names.put(t.getValueInteger("id"), t.getValueString("name")));
|
||||
}
|
||||
for(QRecord row : records)
|
||||
{
|
||||
String locale = row.getValueString("locale");
|
||||
row.setRecordLabel(names.getOrDefault(row.getValueInteger("templateId"), "Template") + " v" + row.getValueInteger("versionNumber")
|
||||
+ (locale == null || locale.isBlank() || "en".equals(locale) ? "" : " (" + locale + ")"));
|
||||
}
|
||||
return (records);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,215 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.template;
|
||||
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Objects;
|
||||
import com.kingsrook.qbits.email.EmailRuntime;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTables;
|
||||
import com.kingsrook.qbits.email.outbox.EmailRequest;
|
||||
import com.kingsrook.qbits.email.permissions.EmailPermissionChecker;
|
||||
import com.kingsrook.qbits.email.permissions.EmailPermissions;
|
||||
import com.kingsrook.qbits.email.support.EmailStore;
|
||||
import com.kingsrook.qbits.email.support.SystemWrites;
|
||||
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Atomic version lifecycle. Approval binds to exactly one immutable version.
|
||||
*******************************************************************************/
|
||||
public final class EmailVersions
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private EmailVersions()
|
||||
{
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Enforce both the qbit permission and the template's independent edit grant.
|
||||
*******************************************************************************/
|
||||
public static void requireEdit(QRecord template, boolean approval)
|
||||
{
|
||||
if(SystemWrites.active())
|
||||
{
|
||||
return;
|
||||
}
|
||||
if(!EmailPermissionChecker.has(approval ? EmailPermissions.ADMIN : EmailPermissions.MANAGE))
|
||||
{
|
||||
throw (new IllegalArgumentException(approval ? "PHI approval needs email.admin" : "Editing templates needs email.manage"));
|
||||
}
|
||||
String permission = template.getValueString("editPermissionName");
|
||||
if(permission != null && !permission.isBlank() && (QContext.getQSession() == null || !QContext.getQSession().hasPermission(permission)))
|
||||
{
|
||||
throw (new IllegalArgumentException("Editing this template needs " + permission));
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Serialize lifecycle operations across replicas using the template row.
|
||||
*******************************************************************************/
|
||||
private static QRecord lock(Integer templateId, QBackendTransaction tx) throws QException
|
||||
{
|
||||
QRecord template = EmailStore.get(EmailTables.TEMPLATE, templateId, tx);
|
||||
if(template == null)
|
||||
{
|
||||
throw (new IllegalArgumentException("Template not found"));
|
||||
}
|
||||
requireEdit(template, false);
|
||||
EmailStore.update(EmailTables.TEMPLATE, new QRecord().withValue("id", templateId).withValue("versionLock", 0), tx);
|
||||
return (EmailStore.get(EmailTables.TEMPLATE, templateId, tx));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Copy active (or latest) into a new draft, with a globally increasing number.
|
||||
*******************************************************************************/
|
||||
public static QRecord newVersion(Integer templateId) throws QException
|
||||
{
|
||||
try(QBackendTransaction tx = QBackendTransaction.openFor(new UpdateInput(EmailTables.TEMPLATE)))
|
||||
{
|
||||
QRecord template = lock(templateId, tx);
|
||||
List<QRecord> versions = SystemWrites.query(EmailTables.VERSION, EmailStore.equal("templateId", templateId)
|
||||
.withOrderBy(new QFilterOrderBy("versionNumber", false)), tx);
|
||||
QRecord source = versions.stream().filter(v -> Objects.equals(template.getValue("activeVersionId"), v.getValue("id"))).findFirst()
|
||||
.orElse(versions.isEmpty() ? new QRecord().withValue("locale", "en").withValue("variablesSchema", "[]").withValue("sampleVariables", "{}") : versions.getFirst());
|
||||
QRecord draft = new QRecord();
|
||||
for(String field : List.of("locale", "subject", "preheader", "bodySource", "textSource", "variablesSchema", "sampleVariables"))
|
||||
{
|
||||
draft.setValue(field, source.getValue(field));
|
||||
}
|
||||
draft.withValue("templateId", templateId).withValue("versionNumber", versions.isEmpty() ? 1 : versions.getFirst().getValueInteger("versionNumber") + 1)
|
||||
.withValue("status", "DRAFT").withValue("createdBy", actor());
|
||||
QRecord result = SystemWrites.insert(EmailTables.VERSION, List.of(draft), tx).getRecords().getFirst();
|
||||
if(result.getValue("id") == null || result.getErrors() != null && !result.getErrors().isEmpty())
|
||||
{
|
||||
throw (new QException("Could not create template version"));
|
||||
}
|
||||
tx.commit();
|
||||
return (result);
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Approval is granted to the selected draft or active version, never retired.
|
||||
*******************************************************************************/
|
||||
public static void approve(Integer versionId) throws QException
|
||||
{
|
||||
QRecord version = requiredVersion(versionId, null);
|
||||
try(QBackendTransaction tx = QBackendTransaction.openFor(new UpdateInput(EmailTables.TEMPLATE)))
|
||||
{
|
||||
QRecord template = lock(version.getValueInteger("templateId"), tx);
|
||||
requireEdit(template, true);
|
||||
version = requiredVersion(versionId, tx);
|
||||
if("RETIRED".equals(version.getValueString("status")) || "MARKETING".equals(template.getValueString("category")))
|
||||
{
|
||||
throw (new IllegalArgumentException("Retired versions and MARKETING templates cannot be approved for PHI"));
|
||||
}
|
||||
EmailStore.update(EmailTables.TEMPLATE, new QRecord().withValue("id", template.getValue("id"))
|
||||
.withValue("phiApproved", true).withValue("approvedVersionId", versionId).withValue("phiApprovedBy", actor())
|
||||
.withValue("phiApprovedDate", EmailRuntime.clock().instant()), tx);
|
||||
tx.commit();
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Validate samples and syntax, compile MJML, then atomically retire and activate.
|
||||
*******************************************************************************/
|
||||
public static void activate(Integer versionId) throws QException
|
||||
{
|
||||
QRecord initial = requiredVersion(versionId, null);
|
||||
try(QBackendTransaction tx = QBackendTransaction.openFor(new UpdateInput(EmailTables.TEMPLATE)))
|
||||
{
|
||||
QRecord template = lock(initial.getValueInteger("templateId"), tx);
|
||||
QRecord version = requiredVersion(versionId, tx);
|
||||
if(!"DRAFT".equals(version.getValueString("status")))
|
||||
{
|
||||
throw (new IllegalArgumentException("Only a DRAFT version can be activated"));
|
||||
}
|
||||
validate(template, version);
|
||||
for(QRecord previous : SystemWrites.query(EmailTables.VERSION, EmailStore.equal("templateId", template.getValueInteger("id")), tx))
|
||||
{
|
||||
if("ACTIVE".equals(previous.getValueString("status")) && Objects.equals(version.getValue("locale"), previous.getValue("locale")))
|
||||
{
|
||||
EmailStore.update(EmailTables.VERSION, new QRecord().withValue("id", previous.getValue("id")).withValue("status", "RETIRED"), tx);
|
||||
}
|
||||
}
|
||||
EmailStore.update(EmailTables.VERSION, new QRecord().withValue("id", versionId).withValue("status", "ACTIVE")
|
||||
.withValue("compiledHtml", version.getValue("compiledHtml")).withValue("activatedDate", EmailRuntime.clock().instant()), tx);
|
||||
QRecord change = new QRecord().withValue("id", template.getValue("id")).withValue("activeVersionId", versionId);
|
||||
if(!Objects.equals(template.getValue("approvedVersionId"), versionId))
|
||||
{
|
||||
change.withValue("phiApproved", false).withValue("approvedVersionId", null).withValue("phiApprovedBy", null).withValue("phiApprovedDate", null);
|
||||
}
|
||||
EmailStore.update(EmailTables.TEMPLATE, change, tx);
|
||||
tx.commit();
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Review validation is side-effect-free; compiled output stays in memory.
|
||||
*******************************************************************************/
|
||||
public static void validate(QRecord template, QRecord version) throws QException
|
||||
{
|
||||
requireEdit(template, false);
|
||||
if("MJML".equals(template.getValueString("kind")))
|
||||
{
|
||||
CliMjmlCompiler.rejectIncludes(version.getValueString("bodySource"));
|
||||
EmailTemplateEngine.validateHtml(version.getValueString("bodySource"));
|
||||
if(EmailRuntime.config().getMjmlCompiler() == null)
|
||||
{
|
||||
throw (new IllegalArgumentException("MJML activation needs a configured compiler"));
|
||||
}
|
||||
new EmailTemplateEngine().parse(version.getValueString("bodySource"), new VariableSchema(version.getValueString("variablesSchema")));
|
||||
version.setValue("compiledHtml", EmailRuntime.config().getMjmlCompiler().compile(version.getValueString("bodySource")));
|
||||
}
|
||||
EmailTemplateRenderer renderer = new EmailTemplateRenderer(template, version,
|
||||
new EmailRequest().withVariables(VariableSchema.values(version.getValueString("sampleVariables"))));
|
||||
renderer.render("preview", "https://example.invalid/unsubscribe/preview", false);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Load a required version without exposing database diagnostics.
|
||||
*******************************************************************************/
|
||||
public static QRecord requiredVersion(Integer id, QBackendTransaction tx) throws QException
|
||||
{
|
||||
QRecord version = EmailStore.get(EmailTables.VERSION, id, tx);
|
||||
if(version == null)
|
||||
{
|
||||
throw (new IllegalArgumentException("Template version not found"));
|
||||
}
|
||||
return (version);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Stable session actor identity, with an explicit system fallback.
|
||||
*******************************************************************************/
|
||||
private static String actor()
|
||||
{
|
||||
return (QContext.getQSession() == null || QContext.getQSession().getUser() == null ? "system" : QContext.getQSession().getUser().getIdReference());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.template;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Activation-time compilation only. Implementations must not persist sources.
|
||||
*******************************************************************************/
|
||||
@FunctionalInterface
|
||||
public interface MjmlCompilerInterface
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Return HTML or a content-free failure.
|
||||
*******************************************************************************/
|
||||
String compile(String source);
|
||||
}
|
||||
@@ -0,0 +1,212 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.template;
|
||||
|
||||
|
||||
import java.io.Serializable;
|
||||
import java.math.BigDecimal;
|
||||
import java.net.URI;
|
||||
import java.time.Instant;
|
||||
import java.time.LocalDate;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import com.fasterxml.jackson.core.type.TypeReference;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Declarative variable contract. Validation errors never include input values.
|
||||
*******************************************************************************/
|
||||
public class VariableSchema
|
||||
{
|
||||
private final Map<String, Entry> entries = new LinkedHashMap<>();
|
||||
private static final ObjectMapper JSON = new ObjectMapper();
|
||||
|
||||
/*******************************************************************************
|
||||
** One schema binding, including optional record provenance.
|
||||
*******************************************************************************/
|
||||
public record Entry(String name, String label, String type, boolean required, String classification, String table, String field)
|
||||
{
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Parse the JSON list and reject ambiguous or unsupported declarations.
|
||||
*******************************************************************************/
|
||||
public VariableSchema(String json)
|
||||
{
|
||||
try
|
||||
{
|
||||
List<Entry> parsed = JSON.readValue(json == null || json.isBlank() ? "[]" : json, new TypeReference<List<Entry>>() {});
|
||||
for(Entry entry : parsed)
|
||||
{
|
||||
if(entry.name() == null || !entry.name().matches("[A-Za-z_][A-Za-z0-9_]*(\\.[A-Za-z_][A-Za-z0-9_]*)*")
|
||||
|| !List.of("STRING", "NUMBER", "DATE", "DATE_TIME", "BOOLEAN", "URL", "LIST", "RECORD").contains(entry.type())
|
||||
|| entries.putIfAbsent(entry.name(), entry) != null
|
||||
|| entry.classification() != null && !List.of("PUBLIC", "CONFIDENTIAL", "PII", "PHI").contains(entry.classification())
|
||||
|| (entry.table() == null) != (entry.field() == null))
|
||||
{
|
||||
throw (new IllegalArgumentException());
|
||||
}
|
||||
}
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
throw (new IllegalArgumentException("Invalid variables schema"));
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Declared bindings in stable order.
|
||||
*******************************************************************************/
|
||||
public Map<String, Entry> entries()
|
||||
{
|
||||
return (Map.copyOf(entries));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Built-ins and paths under declared records are known to the parser.
|
||||
*******************************************************************************/
|
||||
public boolean knows(String name)
|
||||
{
|
||||
if(name.equals("now") || name.equals("message.id") || name.equals("unsubscribe_url")
|
||||
|| name.matches("brand\\.(label|name|site_url|logo_url|logo_dark_url|logo_alt|postal_address|primary_color|text_color|background_color|dark_background_color|dark_text_color|button_text_color|dark_primary_color|dark_button_text_color|footer_markdown)"))
|
||||
{
|
||||
return (true);
|
||||
}
|
||||
return (entries.containsKey(name) || entries.values().stream().anyMatch(e ->
|
||||
("RECORD".equals(e.type()) || "LIST".equals(e.type())) && name.startsWith(e.name() + ".")));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Validate and coerce declared values without discarding record structure.
|
||||
*******************************************************************************/
|
||||
public Map<String, Serializable> validate(Map<String, Serializable> values)
|
||||
{
|
||||
Map<String, Serializable> result = new LinkedHashMap<>(values);
|
||||
for(Entry entry : entries.values())
|
||||
{
|
||||
Object value = EmailTemplateEngine.lookup(values, entry.name());
|
||||
if(value == null || value instanceof String s && s.isEmpty())
|
||||
{
|
||||
if(entry.required())
|
||||
{
|
||||
throw (new IllegalArgumentException("Required variable " + entry.name() + " is missing"));
|
||||
}
|
||||
continue;
|
||||
}
|
||||
try
|
||||
{
|
||||
Serializable coerced = switch(entry.type())
|
||||
{
|
||||
case "STRING" -> value.toString();
|
||||
case "NUMBER" -> new BigDecimal(value.toString());
|
||||
case "DATE" -> LocalDate.parse(value.toString());
|
||||
case "DATE_TIME" -> Instant.parse(value.toString());
|
||||
case "BOOLEAN" -> booleanValue(value);
|
||||
case "URL" -> url(value.toString());
|
||||
case "LIST" -> (Serializable) (List<?>) value;
|
||||
case "RECORD" -> (Serializable) (Map<?, ?>) value;
|
||||
default -> throw (new IllegalArgumentException());
|
||||
};
|
||||
result.put(entry.name(), coerced);
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
throw (new IllegalArgumentException("Variable " + entry.name() + " must be " + entry.type()));
|
||||
}
|
||||
}
|
||||
return (result);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Strict boolean coercion avoids silently turning misspellings into false.
|
||||
*******************************************************************************/
|
||||
private static Boolean booleanValue(Object value)
|
||||
{
|
||||
if(!List.of("true", "false").contains(value.toString().toLowerCase(java.util.Locale.ROOT)))
|
||||
{
|
||||
throw (new IllegalArgumentException());
|
||||
}
|
||||
return (Boolean.valueOf(value.toString()));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Only absolute web URLs are allowed in variable and brand URL fields.
|
||||
*******************************************************************************/
|
||||
public static String url(String value)
|
||||
{
|
||||
return (url(value, false));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** The same URL policy, optionally allowing mailto only for link destinations.
|
||||
*******************************************************************************/
|
||||
public static String url(String value, boolean allowMailto)
|
||||
{
|
||||
URI uri = URI.create(value);
|
||||
if(allowMailto && "mailto".equalsIgnoreCase(uri.getScheme()) && !uri.getRawSchemeSpecificPart().isBlank()
|
||||
&& !value.matches("(?is).*%(?:0a|0d).*"))
|
||||
{
|
||||
return (value);
|
||||
}
|
||||
if(uri.getScheme() == null || !List.of("http", "https").contains(uri.getScheme().toLowerCase(java.util.Locale.ROOT)) || uri.getHost() == null || uri.getUserInfo() != null)
|
||||
{
|
||||
throw (new IllegalArgumentException("URL must use http or https"));
|
||||
}
|
||||
return (value);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Parse process inputs and sample values; never echo malformed JSON.
|
||||
*******************************************************************************/
|
||||
public static Map<String, Serializable> values(String json)
|
||||
{
|
||||
try
|
||||
{
|
||||
Map<String, Serializable> result = JSON.readValue(json == null || json.isBlank() ? "{}" : json, new TypeReference<LinkedHashMap<String, Serializable>>() {});
|
||||
if(result == null)
|
||||
{
|
||||
throw (new IllegalArgumentException());
|
||||
}
|
||||
return (result);
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
throw (new IllegalArgumentException("Variables must be a JSON object"));
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Serialize safe snapshots using the existing Jackson dependency.
|
||||
*******************************************************************************/
|
||||
public static String json(Object value)
|
||||
{
|
||||
try
|
||||
{
|
||||
return (JSON.writeValueAsString(value));
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
throw (new IllegalArgumentException("Could not serialize email variables"));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,234 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.transport;
|
||||
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Rendered, immutable-by-convention content passed to a transport.
|
||||
*******************************************************************************/
|
||||
public class EmailEnvelope
|
||||
{
|
||||
private String messageId;
|
||||
private String from;
|
||||
private List<String> to = List.of();
|
||||
private List<String> cc = List.of();
|
||||
private List<String> bcc = List.of();
|
||||
private String replyTo;
|
||||
private String subject;
|
||||
private String htmlBody;
|
||||
private String textBody;
|
||||
private Map<String, String> headers = Map.of();
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for messageId.
|
||||
*******************************************************************************/
|
||||
public String getMessageId()
|
||||
{
|
||||
return (messageId);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for messageId.
|
||||
*******************************************************************************/
|
||||
public EmailEnvelope withMessageId(String messageId)
|
||||
{
|
||||
this.messageId = messageId;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for from.
|
||||
*******************************************************************************/
|
||||
public String getFrom()
|
||||
{
|
||||
return (from);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for from.
|
||||
*******************************************************************************/
|
||||
public EmailEnvelope withFrom(String from)
|
||||
{
|
||||
this.from = from;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for to.
|
||||
*******************************************************************************/
|
||||
public List<String> getTo()
|
||||
{
|
||||
return (to);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for to.
|
||||
*******************************************************************************/
|
||||
public EmailEnvelope withTo(List<String> to)
|
||||
{
|
||||
this.to = to;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for cc.
|
||||
*******************************************************************************/
|
||||
public List<String> getCc()
|
||||
{
|
||||
return (cc);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for cc.
|
||||
*******************************************************************************/
|
||||
public EmailEnvelope withCc(List<String> cc)
|
||||
{
|
||||
this.cc = cc;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for bcc.
|
||||
*******************************************************************************/
|
||||
public List<String> getBcc()
|
||||
{
|
||||
return (bcc);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for bcc.
|
||||
*******************************************************************************/
|
||||
public EmailEnvelope withBcc(List<String> bcc)
|
||||
{
|
||||
this.bcc = bcc;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for replyTo.
|
||||
*******************************************************************************/
|
||||
public String getReplyTo()
|
||||
{
|
||||
return (replyTo);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for replyTo.
|
||||
*******************************************************************************/
|
||||
public EmailEnvelope withReplyTo(String replyTo)
|
||||
{
|
||||
this.replyTo = replyTo;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for subject.
|
||||
*******************************************************************************/
|
||||
public String getSubject()
|
||||
{
|
||||
return (subject);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for subject.
|
||||
*******************************************************************************/
|
||||
public EmailEnvelope withSubject(String subject)
|
||||
{
|
||||
this.subject = subject;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for htmlBody.
|
||||
*******************************************************************************/
|
||||
public String getHtmlBody()
|
||||
{
|
||||
return (htmlBody);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for htmlBody.
|
||||
*******************************************************************************/
|
||||
public EmailEnvelope withHtmlBody(String htmlBody)
|
||||
{
|
||||
this.htmlBody = htmlBody;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for textBody.
|
||||
*******************************************************************************/
|
||||
public String getTextBody()
|
||||
{
|
||||
return (textBody);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for textBody.
|
||||
*******************************************************************************/
|
||||
public EmailEnvelope withTextBody(String textBody)
|
||||
{
|
||||
this.textBody = textBody;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for headers.
|
||||
*******************************************************************************/
|
||||
public Map<String, String> getHeaders()
|
||||
{
|
||||
return (headers);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for headers.
|
||||
*******************************************************************************/
|
||||
public EmailEnvelope withHeaders(Map<String, String> headers)
|
||||
{
|
||||
this.headers = headers;
|
||||
return (this);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.transport;
|
||||
|
||||
|
||||
import java.io.IOException;
|
||||
import java.net.http.HttpClient;
|
||||
import java.net.http.HttpRequest;
|
||||
import java.net.http.HttpResponse;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.time.Duration;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Package-visible HTTP seam; no provider SDK, credential cache or filesystem.
|
||||
*******************************************************************************/
|
||||
@FunctionalInterface
|
||||
interface EmailHttp
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Exchange one request without retries.
|
||||
*******************************************************************************/
|
||||
HttpResponse<String> send(HttpRequest request) throws IOException, InterruptedException;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Production requests use HTTPS and never forward credentials on redirects.
|
||||
*******************************************************************************/
|
||||
static HttpResponse<String> sendJdk(HttpRequest request) throws IOException, InterruptedException
|
||||
{
|
||||
if(!"https".equalsIgnoreCase(request.uri().getScheme()))
|
||||
{
|
||||
throw (new IOException("Email HTTP requires HTTPS"));
|
||||
}
|
||||
try(HttpClient client = newClient())
|
||||
{
|
||||
return (client.send(request, HttpResponse.BodyHandlers.ofString(StandardCharsets.UTF_8)));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A bounded, stateless JDK client; the caller closes it after each exchange.
|
||||
*******************************************************************************/
|
||||
static HttpClient newClient()
|
||||
{
|
||||
return (HttpClient.newBuilder().connectTimeout(Duration.ofSeconds(10)).followRedirects(HttpClient.Redirect.NEVER).build());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,230 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.transport;
|
||||
|
||||
|
||||
import java.io.IOException;
|
||||
import java.net.URI;
|
||||
import java.net.http.HttpRequest;
|
||||
import java.net.http.HttpResponse;
|
||||
import java.time.Duration;
|
||||
import java.time.Instant;
|
||||
import java.time.ZonedDateTime;
|
||||
import java.time.format.DateTimeFormatter;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.Map;
|
||||
import java.util.Properties;
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import jakarta.mail.MessagingException;
|
||||
import jakarta.mail.Session;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Shared serialization, timeouts and sanitized HTTP failures.
|
||||
*******************************************************************************/
|
||||
final class EmailHttpSupport
|
||||
{
|
||||
static final ObjectMapper JSON = new ObjectMapper();
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private EmailHttpSupport()
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Use the shared envelope validation even for JSON APIs.
|
||||
*******************************************************************************/
|
||||
static void validate(EmailEnvelope envelope) throws EmailTransportException
|
||||
{
|
||||
try
|
||||
{
|
||||
EmailMime.build(Session.getInstance(new Properties()), envelope);
|
||||
}
|
||||
catch(MessagingException | RuntimeException e)
|
||||
{
|
||||
throw (new EmailTransportException("Invalid email message", true));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Preserve application headers and the stable correlation identifier.
|
||||
*******************************************************************************/
|
||||
static Map<String, String> headers(EmailEnvelope envelope)
|
||||
{
|
||||
Map<String, String> headers = new LinkedHashMap<>();
|
||||
envelope.getHeaders().forEach((name, value) ->
|
||||
{
|
||||
if(!name.equalsIgnoreCase("Bcc") && !name.equalsIgnoreCase("Auto-Submitted") && !name.equalsIgnoreCase("X-QQQ-Email-Id"))
|
||||
{
|
||||
headers.put(name, value);
|
||||
}
|
||||
});
|
||||
headers.put("Auto-Submitted", "auto-generated");
|
||||
headers.put("X-QQQ-Email-Id", envelope.getMessageId());
|
||||
return (headers);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Serialize only in memory using Jackson already supplied by core.
|
||||
*******************************************************************************/
|
||||
static byte[] json(Object value) throws EmailTransportException
|
||||
{
|
||||
try
|
||||
{
|
||||
return (JSON.writeValueAsBytes(value));
|
||||
}
|
||||
catch(IOException | RuntimeException e)
|
||||
{
|
||||
throw (new EmailTransportException("Invalid email message", true));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Resolve auth before calling; never attach raw exceptions to a safe failure.
|
||||
*******************************************************************************/
|
||||
static HttpResponse<String> post(EmailHttp http, String url, String contentType, String authHeader, String auth, byte[] body) throws EmailTransportException
|
||||
{
|
||||
try
|
||||
{
|
||||
HttpRequest request = HttpRequest.newBuilder(URI.create(url)).timeout(Duration.ofSeconds(30))
|
||||
.header("Content-Type", contentType).header("Accept", "application/json").header(authHeader, auth)
|
||||
.POST(HttpRequest.BodyPublishers.ofByteArray(body)).build();
|
||||
return (http.send(request));
|
||||
}
|
||||
catch(InterruptedException e)
|
||||
{
|
||||
Thread.currentThread().interrupt();
|
||||
throw (new EmailTransportException("Email HTTP request interrupted", false));
|
||||
}
|
||||
catch(IOException e)
|
||||
{
|
||||
throw (new EmailTransportException("Email HTTP temporarily unavailable", false));
|
||||
}
|
||||
catch(RuntimeException e)
|
||||
{
|
||||
throw (new EmailTransportException("Email HTTP configuration invalid", false).withProviderLevel(true));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Unparseable replies must not expose bodies or cause permanent message loss.
|
||||
*******************************************************************************/
|
||||
static JsonNode responseJson(HttpResponse<String> response) throws EmailTransportException
|
||||
{
|
||||
try
|
||||
{
|
||||
JsonNode json = JSON.readTree(response.body());
|
||||
if(json != null && json.isObject())
|
||||
{
|
||||
return (json);
|
||||
}
|
||||
}
|
||||
catch(IOException | RuntimeException e)
|
||||
{
|
||||
// Bodies and parser diagnostics may contain recipient addresses.
|
||||
}
|
||||
throw (new EmailTransportException("Email provider returned an invalid response", false));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Parse either Retry-After syntax; the dispatcher applies the one-hour cap.
|
||||
*******************************************************************************/
|
||||
static Duration retryAfter(HttpResponse<String> response)
|
||||
{
|
||||
String value = response.headers().firstValue("Retry-After").orElse("").trim();
|
||||
try
|
||||
{
|
||||
long seconds = Long.parseLong(value);
|
||||
return (seconds < 0 ? null : Duration.ofSeconds(seconds));
|
||||
}
|
||||
catch(RuntimeException e)
|
||||
{
|
||||
try
|
||||
{
|
||||
Duration delay = Duration.between(Instant.now(), ZonedDateTime.parse(value, DateTimeFormatter.RFC_1123_DATE_TIME).toInstant());
|
||||
return (delay.isNegative() ? Duration.ZERO : delay);
|
||||
}
|
||||
catch(RuntimeException ignored)
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Only numeric status/code and fixed text escape the transport.
|
||||
*******************************************************************************/
|
||||
static EmailTransportException failure(HttpResponse<String> response, String provider, Integer code, boolean permanent, boolean providerLevel)
|
||||
{
|
||||
return (new EmailTransportException(provider + " request failed (HTTP " + response.statusCode()
|
||||
+ (code == null ? "" : ", code " + code) + ")", permanent).withProviderLevel(providerLevel).withRetryAfter(retryAfter(response)));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Common HTTP precedence: throttling and server errors always remain retryable.
|
||||
*******************************************************************************/
|
||||
static void checkStatus(HttpResponse<String> response, String provider, int success) throws EmailTransportException
|
||||
{
|
||||
int status = response.statusCode();
|
||||
if(status != success)
|
||||
{
|
||||
boolean transientFailure = status == 408 || status == 429 || status >= 500;
|
||||
boolean providerLevel = status == 401 || status == 403 || (status >= 300 && status < 400);
|
||||
throw (failure(response, provider, null, !transientFailure && !providerLevel && status >= 400, providerLevel));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A success must contain a usable id; never mistake a malformed body for success.
|
||||
*******************************************************************************/
|
||||
static EmailTransportResult result(String id) throws EmailTransportException
|
||||
{
|
||||
if(id == null || id.isBlank() || id.length() > 500 || id.contains("\r") || id.contains("\n"))
|
||||
{
|
||||
throw (new EmailTransportException("Email provider returned an invalid message id", false));
|
||||
}
|
||||
return (new EmailTransportResult(id));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,114 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.transport;
|
||||
|
||||
|
||||
import java.util.Date;
|
||||
import java.util.List;
|
||||
import jakarta.mail.Message;
|
||||
import jakarta.mail.MessagingException;
|
||||
import jakarta.mail.Session;
|
||||
import jakarta.mail.internet.InternetAddress;
|
||||
import jakarta.mail.internet.MimeBodyPart;
|
||||
import jakarta.mail.internet.MimeMessage;
|
||||
import jakarta.mail.internet.MimeMultipart;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Shared UTF-8 multipart MIME construction for SMTP and SES.
|
||||
*******************************************************************************/
|
||||
final class EmailMime
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private EmailMime()
|
||||
{
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Construct both alternative parts and stable application headers.
|
||||
*******************************************************************************/
|
||||
static MimeMessage build(Session session, EmailEnvelope envelope) throws MessagingException
|
||||
{
|
||||
noLineBreaks(envelope.getSubject());
|
||||
noLineBreaks(envelope.getFrom());
|
||||
noLineBreaks(envelope.getReplyTo());
|
||||
noLineBreaks(envelope.getMessageId());
|
||||
MimeMessage message = new MimeMessage(session);
|
||||
message.setFrom(new InternetAddress(envelope.getFrom(), true));
|
||||
recipients(message, Message.RecipientType.TO, envelope.getTo());
|
||||
recipients(message, Message.RecipientType.CC, envelope.getCc());
|
||||
recipients(message, Message.RecipientType.BCC, envelope.getBcc());
|
||||
if(envelope.getReplyTo() != null)
|
||||
{
|
||||
message.setReplyTo(InternetAddress.parse(envelope.getReplyTo(), true));
|
||||
}
|
||||
message.setSubject(envelope.getSubject(), "UTF-8");
|
||||
message.setSentDate(new Date());
|
||||
MimeMultipart alternative = new MimeMultipart("alternative");
|
||||
MimeBodyPart text = new MimeBodyPart();
|
||||
text.setText(envelope.getTextBody() == null ? "" : envelope.getTextBody(), "UTF-8");
|
||||
alternative.addBodyPart(text);
|
||||
MimeBodyPart html = new MimeBodyPart();
|
||||
html.setContent(envelope.getHtmlBody() == null ? "" : envelope.getHtmlBody(), "text/html; charset=UTF-8");
|
||||
alternative.addBodyPart(html);
|
||||
message.setContent(alternative);
|
||||
for(var header : envelope.getHeaders().entrySet())
|
||||
{
|
||||
if(!header.getKey().matches("[A-Za-z0-9-]+") || header.getValue().contains("\r") || header.getValue().contains("\n"))
|
||||
{
|
||||
throw (new MessagingException("Invalid envelope header"));
|
||||
}
|
||||
message.setHeader(header.getKey(), header.getValue());
|
||||
}
|
||||
message.setHeader("Auto-Submitted", "auto-generated");
|
||||
message.setHeader("X-QQQ-Email-Id", envelope.getMessageId());
|
||||
message.saveChanges();
|
||||
return (message);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Add one recipient role.
|
||||
*******************************************************************************/
|
||||
private static void recipients(MimeMessage message, Message.RecipientType type, List<String> addresses) throws MessagingException
|
||||
{
|
||||
for(String address : addresses)
|
||||
{
|
||||
noLineBreaks(address);
|
||||
}
|
||||
if(!addresses.isEmpty())
|
||||
{
|
||||
message.setRecipients(type, InternetAddress.parse(String.join(",", addresses), true));
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Reject header injection even for envelopes that bypass queue validation.
|
||||
*******************************************************************************/
|
||||
private static void noLineBreaks(String value) throws MessagingException
|
||||
{
|
||||
if(value != null && (value.contains("\r") || value.contains("\n")))
|
||||
{
|
||||
throw (new MessagingException("Invalid envelope header"));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,147 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.transport;
|
||||
|
||||
|
||||
import com.kingsrook.qbits.email.EmailQBitConfig;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Provider row and resolver. Resolved secrets are never retained.
|
||||
*******************************************************************************/
|
||||
public final class EmailProviderSettings
|
||||
{
|
||||
private final QRecord row;
|
||||
private final EmailQBitConfig config;
|
||||
|
||||
/*******************************************************************************
|
||||
** Settings for one attempt, without resolving secrets.
|
||||
*******************************************************************************/
|
||||
public EmailProviderSettings(QRecord row, EmailQBitConfig config)
|
||||
{
|
||||
this.row = row;
|
||||
this.config = config;
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Nonsecret provider setting.
|
||||
*******************************************************************************/
|
||||
public String value(String name)
|
||||
{
|
||||
return (row.getValueString(name));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** SMTP port, including the optional Gmail preset.
|
||||
*******************************************************************************/
|
||||
public int port()
|
||||
{
|
||||
return ("GMAIL_SMTP".equals(value("kind")) ? 587 : row.getValueInteger("port"));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** SMTP host.
|
||||
*******************************************************************************/
|
||||
public String host()
|
||||
{
|
||||
return ("GMAIL_SMTP".equals(value("kind")) ? "smtp.gmail.com" : value("host"));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** SMTP security.
|
||||
*******************************************************************************/
|
||||
public String security()
|
||||
{
|
||||
return ("GMAIL_SMTP".equals(value("kind")) ? "STARTTLS" : value("security"));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Shared validation and send-time guard, without resolving credentials or DNS.
|
||||
*******************************************************************************/
|
||||
public boolean allowsSmtpAuth()
|
||||
{
|
||||
boolean credentials = value("usernameEnvName") != null || value("secretEnvName") != null;
|
||||
String security = value("security") == null ? security() : value("security");
|
||||
return (!credentials || "STARTTLS".equals(security) || "TLS".equals(security)
|
||||
|| config.getAllowPlaintextSmtpAuth() || "localhost".equalsIgnoreCase(host())
|
||||
|| "127.0.0.1".equals(host()) || "::1".equals(host()));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** A domain is a single DNS name, never a URL or a path component escape.
|
||||
*******************************************************************************/
|
||||
public static boolean validSendingDomain(String domain)
|
||||
{
|
||||
return (domain != null && domain.length() <= 253
|
||||
&& domain.matches("(?i)[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?(?:\\.[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?)+"));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Tracking is opt-in; null database values mean disabled.
|
||||
*******************************************************************************/
|
||||
boolean enabled(String field)
|
||||
{
|
||||
return (Boolean.TRUE.equals(row.getValueBoolean(field)));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** HTTP providers must never send a missing token as the literal string null.
|
||||
*******************************************************************************/
|
||||
String requiredSecret() throws EmailTransportException
|
||||
{
|
||||
String secret = resolve("secretEnvName");
|
||||
if(secret == null)
|
||||
{
|
||||
throw (new EmailTransportException("Provider needs a secret environment name", false).withProviderLevel(true));
|
||||
}
|
||||
return (secret);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Resolve an explicitly named secret immediately before sending.
|
||||
*******************************************************************************/
|
||||
String resolve(String field) throws EmailTransportException
|
||||
{
|
||||
String name = value(field);
|
||||
if(name == null)
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
if(!EmailQBitConfig.ENV_NAME.matcher(name).matches() || !config.effectiveAllowedEnvNames(value("kind")).contains(name))
|
||||
{
|
||||
throw (new EmailTransportException("Credential environment name is not allowed", false).withProviderLevel(true));
|
||||
}
|
||||
try
|
||||
{
|
||||
String value = config.getSecretResolver().apply(name);
|
||||
if(value != null && !value.isBlank())
|
||||
{
|
||||
return (value);
|
||||
}
|
||||
}
|
||||
catch(RuntimeException e)
|
||||
{
|
||||
// Resolver exception text may contain a secret.
|
||||
}
|
||||
throw (new EmailTransportException("Credential environment variable is not set", false).withProviderLevel(true));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.transport;
|
||||
|
||||
|
||||
import com.kingsrook.qbits.email.transport.EmailTransportException;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Delivery transport; called only for a claimed outbox attempt.
|
||||
*******************************************************************************/
|
||||
@FunctionalInterface
|
||||
public interface EmailTransport
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Send a previously rendered envelope.
|
||||
*******************************************************************************/
|
||||
EmailTransportResult send(EmailEnvelope envelope, EmailProviderSettings settings) throws EmailTransportException;
|
||||
}
|
||||
@@ -0,0 +1,99 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.transport;
|
||||
|
||||
|
||||
import java.time.Duration;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Sanitized transport failure. Never retain a provider exception or credential.
|
||||
*******************************************************************************/
|
||||
public class EmailTransportException extends Exception
|
||||
{
|
||||
private boolean permanent;
|
||||
private boolean providerLevel;
|
||||
private Duration retryAfter;
|
||||
|
||||
/*******************************************************************************
|
||||
** Create a safe failure with a permanent or transient classification.
|
||||
*******************************************************************************/
|
||||
public EmailTransportException(String message, boolean permanent)
|
||||
{
|
||||
super(message);
|
||||
this.permanent = permanent;
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for permanent.
|
||||
*******************************************************************************/
|
||||
public boolean getPermanent()
|
||||
{
|
||||
return (permanent && !providerLevel);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Provider failures are retryable without consuming the message retry budget.
|
||||
*******************************************************************************/
|
||||
public boolean getProviderLevel()
|
||||
{
|
||||
return (providerLevel);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Classify a failure affecting the provider rather than this message.
|
||||
*******************************************************************************/
|
||||
public EmailTransportException withProviderLevel(boolean providerLevel)
|
||||
{
|
||||
this.providerLevel = providerLevel;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for permanent.
|
||||
*******************************************************************************/
|
||||
public EmailTransportException withPermanent(boolean permanent)
|
||||
{
|
||||
this.permanent = permanent;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for retryAfter.
|
||||
*******************************************************************************/
|
||||
public Duration getRetryAfter()
|
||||
{
|
||||
return (retryAfter);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for retryAfter.
|
||||
*******************************************************************************/
|
||||
public EmailTransportException withRetryAfter(Duration retryAfter)
|
||||
{
|
||||
this.retryAfter = retryAfter;
|
||||
return (this);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.transport;
|
||||
|
||||
|
||||
import java.util.HashMap;
|
||||
import java.util.Map;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Per-dispatcher registry; no configuration or credential caches.
|
||||
*******************************************************************************/
|
||||
public class EmailTransportRegistry
|
||||
{
|
||||
private final Map<String, EmailTransport> transports = new HashMap<>();
|
||||
|
||||
/*******************************************************************************
|
||||
** Install the outbound transports.
|
||||
*******************************************************************************/
|
||||
public EmailTransportRegistry()
|
||||
{
|
||||
transports.put("SMTP", new SmtpEmailTransport());
|
||||
transports.put("GMAIL_SMTP", new SmtpEmailTransport());
|
||||
transports.put("SES", new SesEmailTransport());
|
||||
transports.put("POSTMARK", new PostmarkEmailTransport());
|
||||
transports.put("SENDGRID", new SendGridEmailTransport());
|
||||
transports.put("MAILGUN", new MailgunEmailTransport());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Host or test transport override.
|
||||
*******************************************************************************/
|
||||
public EmailTransportRegistry withTransport(String kind, EmailTransport transport)
|
||||
{
|
||||
transports.put(kind, transport);
|
||||
return (this);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Resolve a kind, explicitly refusing future transports.
|
||||
*******************************************************************************/
|
||||
public EmailTransport get(String kind) throws EmailTransportException
|
||||
{
|
||||
EmailTransport transport = transports.get(kind);
|
||||
if(transport == null)
|
||||
{
|
||||
throw (new EmailTransportException("GMAIL_API".equals(kind)
|
||||
? "Gmail API sending is specified but not built; use GMAIL_SMTP" : "Unknown email provider kind", true));
|
||||
}
|
||||
return (transport);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,31 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.transport;
|
||||
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Provider acknowledgment, never a delivery receipt.
|
||||
*******************************************************************************/
|
||||
public record EmailTransportResult(String providerMessageId)
|
||||
{
|
||||
}
|
||||
@@ -0,0 +1,132 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.transport;
|
||||
|
||||
|
||||
import java.io.ByteArrayOutputStream;
|
||||
import java.io.IOException;
|
||||
import java.net.http.HttpResponse;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Base64;
|
||||
import java.util.List;
|
||||
import java.util.Properties;
|
||||
import java.util.UUID;
|
||||
import jakarta.mail.MessagingException;
|
||||
import jakarta.mail.Session;
|
||||
import jakarta.mail.internet.InternetAddress;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Mailgun MIME upload, entirely in memory, with explicit blind destinations.
|
||||
*******************************************************************************/
|
||||
public class MailgunEmailTransport implements EmailTransport
|
||||
{
|
||||
private final EmailHttp http;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fixed regional production endpoints and JDK HTTP implementation.
|
||||
*******************************************************************************/
|
||||
public MailgunEmailTransport()
|
||||
{
|
||||
this(EmailHttp::sendJdk);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Package-visible request/response test seam.
|
||||
*******************************************************************************/
|
||||
MailgunEmailTransport(EmailHttp http)
|
||||
{
|
||||
this.http = http;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Envelope destinations include Bcc; serialized MIME never includes its header.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public EmailTransportResult send(EmailEnvelope envelope, EmailProviderSettings settings) throws EmailTransportException
|
||||
{
|
||||
String domain = settings.value("sendingDomain");
|
||||
String region = settings.value("region");
|
||||
if(!EmailProviderSettings.validSendingDomain(domain) || (!"US".equals(region) && !"EU".equals(region)))
|
||||
{
|
||||
throw (new EmailTransportException("Mailgun needs a sending domain and US or EU region", false).withProviderLevel(true));
|
||||
}
|
||||
String boundary = "qqq-email-" + UUID.randomUUID();
|
||||
ByteArrayOutputStream body = new ByteArrayOutputStream();
|
||||
try
|
||||
{
|
||||
List<String> destinations = new ArrayList<>();
|
||||
for(List<String> role : List.of(envelope.getTo(), envelope.getCc(), envelope.getBcc()))
|
||||
{
|
||||
for(String value : role)
|
||||
{
|
||||
for(InternetAddress address : InternetAddress.parse(value, true))
|
||||
{
|
||||
address.validate();
|
||||
destinations.add(address.getAddress());
|
||||
}
|
||||
}
|
||||
}
|
||||
field(body, boundary, "to", String.join(",", destinations));
|
||||
field(body, boundary, "o:tracking-opens", settings.enabled("trackOpens") ? "yes" : "no");
|
||||
field(body, boundary, "o:tracking-clicks", settings.enabled("trackClicks") ? "yes" : "no");
|
||||
field(body, boundary, "v:qqq_email_id", envelope.getMessageId());
|
||||
body.writeBytes(("--" + boundary + "\r\nContent-Disposition: form-data; name=\"message\"; filename=\"message.eml\"\r\n"
|
||||
+ "Content-Type: message/rfc822\r\n\r\n").getBytes(StandardCharsets.UTF_8));
|
||||
EmailMime.build(Session.getInstance(new Properties()), envelope).writeTo(body, new String[] {"Bcc"});
|
||||
body.writeBytes(("\r\n--" + boundary + "--\r\n").getBytes(StandardCharsets.UTF_8));
|
||||
}
|
||||
catch(MessagingException | IOException | RuntimeException e)
|
||||
{
|
||||
throw (new EmailTransportException("Invalid email message", true));
|
||||
}
|
||||
String auth = Base64.getEncoder().encodeToString(("api:" + settings.requiredSecret()).getBytes(StandardCharsets.UTF_8));
|
||||
String host = "EU".equals(region) ? "https://api.eu.mailgun.net" : "https://api.mailgun.net";
|
||||
HttpResponse<String> response = EmailHttpSupport.post(http, host + "/v3/" + domain + "/messages.mime",
|
||||
"multipart/form-data; boundary=" + boundary, "Authorization", "Basic " + auth, body.toByteArray());
|
||||
EmailHttpSupport.checkStatus(response, "Mailgun", 200);
|
||||
String id = EmailHttpSupport.responseJson(response).path("id").textValue();
|
||||
if(id != null && id.startsWith("<") && id.endsWith(">"))
|
||||
{
|
||||
id = id.substring(1, id.length() - 1);
|
||||
}
|
||||
return (EmailHttpSupport.result(id));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Encode fixed-name UTF-8 multipart fields without touching the filesystem.
|
||||
*******************************************************************************/
|
||||
private static void field(ByteArrayOutputStream body, String boundary, String name, String value)
|
||||
{
|
||||
body.writeBytes(("--" + boundary + "\r\nContent-Disposition: form-data; name=\"" + name + "\"\r\n\r\n"
|
||||
+ value + "\r\n").getBytes(StandardCharsets.UTF_8));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,129 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.transport;
|
||||
|
||||
|
||||
import java.net.http.HttpResponse;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Postmark's single-message API with per-attempt server token resolution.
|
||||
*******************************************************************************/
|
||||
public class PostmarkEmailTransport implements EmailTransport
|
||||
{
|
||||
private final EmailHttp http;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fixed production endpoint and JDK HTTP implementation.
|
||||
*******************************************************************************/
|
||||
public PostmarkEmailTransport()
|
||||
{
|
||||
this(EmailHttp::sendJdk);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Package-visible request/response test seam.
|
||||
*******************************************************************************/
|
||||
PostmarkEmailTransport(EmailHttp http)
|
||||
{
|
||||
this.http = http;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Send a rendered message; provider response diagnostics never escape.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public EmailTransportResult send(EmailEnvelope envelope, EmailProviderSettings settings) throws EmailTransportException
|
||||
{
|
||||
EmailHttpSupport.validate(envelope);
|
||||
Map<String, Object> body = new LinkedHashMap<>();
|
||||
body.put("From", envelope.getFrom());
|
||||
body.put("To", String.join(",", envelope.getTo()));
|
||||
body.put("Cc", String.join(",", envelope.getCc()));
|
||||
body.put("Bcc", String.join(",", envelope.getBcc()));
|
||||
if(envelope.getReplyTo() != null)
|
||||
{
|
||||
body.put("ReplyTo", envelope.getReplyTo());
|
||||
}
|
||||
body.put("Subject", envelope.getSubject());
|
||||
if(envelope.getHtmlBody() != null)
|
||||
{
|
||||
body.put("HtmlBody", envelope.getHtmlBody());
|
||||
}
|
||||
if(envelope.getTextBody() != null)
|
||||
{
|
||||
body.put("TextBody", envelope.getTextBody());
|
||||
}
|
||||
body.put("Headers", EmailHttpSupport.headers(envelope).entrySet().stream()
|
||||
.map(h -> Map.of("Name", h.getKey(), "Value", h.getValue())).toList());
|
||||
String stream = settings.value("messageStream");
|
||||
body.put("MessageStream", stream == null || stream.isBlank() ? "outbound" : stream);
|
||||
body.put("TrackOpens", settings.enabled("trackOpens"));
|
||||
body.put("TrackLinks", settings.enabled("trackClicks") ? "HtmlAndText" : "None");
|
||||
HttpResponse<String> response = EmailHttpSupport.post(http, "https://api.postmarkapp.com/email", "application/json",
|
||||
"X-Postmark-Server-Token", settings.requiredSecret(), EmailHttpSupport.json(body));
|
||||
int status = response.statusCode();
|
||||
if(status == 408 || status == 429 || status >= 500 || status == 401 || status == 403 || (status >= 300 && status < 400))
|
||||
{
|
||||
EmailHttpSupport.checkStatus(response, "Postmark", 200);
|
||||
}
|
||||
JsonNode json;
|
||||
try
|
||||
{
|
||||
json = EmailHttpSupport.responseJson(response);
|
||||
}
|
||||
catch(EmailTransportException e)
|
||||
{
|
||||
EmailHttpSupport.checkStatus(response, "Postmark", 200);
|
||||
throw (e);
|
||||
}
|
||||
Integer code = json.path("ErrorCode").canConvertToInt() ? json.path("ErrorCode").intValue() : null;
|
||||
if(status == 200 && code == null)
|
||||
{
|
||||
throw (new EmailTransportException("Email provider returned an invalid response", false));
|
||||
}
|
||||
if(status != 200 || code == null || code != 0)
|
||||
{
|
||||
// Postmark ErrorCode (not HTTP status) | Classification
|
||||
// 10 bad token | provider-level
|
||||
// 300 invalid request, 406 inactive | permanent message failure
|
||||
// 400/401 sender signature, 405 send disabled, 412 pending approval,
|
||||
// 413 account unapproved, 422 invalid server, 1235/1236 stream,
|
||||
// 1480 IP restriction | provider-level
|
||||
// HTTP 429 / 5xx | transient, takes precedence
|
||||
// Other request codes | permanent (unknown 200 is transient)
|
||||
boolean providerLevel = code != null && Set.of(10, 400, 401, 405, 412, 413, 422, 1235, 1236, 1480).contains(code);
|
||||
throw (EmailHttpSupport.failure(response, "Postmark", code, !providerLevel && (status >= 400 || code != null && code != 0), providerLevel));
|
||||
}
|
||||
return (EmailHttpSupport.result(json.path("MessageID").textValue()));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,145 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.transport;
|
||||
|
||||
|
||||
import java.net.http.HttpResponse;
|
||||
import java.util.ArrayList;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Locale;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import jakarta.mail.internet.AddressException;
|
||||
import jakarta.mail.internet.InternetAddress;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** SendGrid v3 mail send with explicit per-message tracking settings.
|
||||
*******************************************************************************/
|
||||
public class SendGridEmailTransport implements EmailTransport
|
||||
{
|
||||
private static final Set<String> RESERVED_HEADERS = Set.of("x-sg-id", "x-sg-eid", "received", "dkim-signature",
|
||||
"content-type", "content-transfer-encoding", "to", "from", "subject", "reply-to", "cc", "bcc");
|
||||
private final EmailHttp http;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fixed production endpoint and JDK HTTP implementation.
|
||||
*******************************************************************************/
|
||||
public SendGridEmailTransport()
|
||||
{
|
||||
this(EmailHttp::sendJdk);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Package-visible request/response test seam.
|
||||
*******************************************************************************/
|
||||
SendGridEmailTransport(EmailHttp http)
|
||||
{
|
||||
this.http = http;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Build one personalization, keeping blind recipients out of custom headers.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public EmailTransportResult send(EmailEnvelope envelope, EmailProviderSettings settings) throws EmailTransportException
|
||||
{
|
||||
EmailHttpSupport.validate(envelope);
|
||||
Map<String, Object> body = new LinkedHashMap<>();
|
||||
Map<String, Object> personalization = new LinkedHashMap<>();
|
||||
personalization.put("to", addresses(envelope.getTo()));
|
||||
if(!envelope.getCc().isEmpty())
|
||||
{
|
||||
personalization.put("cc", addresses(envelope.getCc()));
|
||||
}
|
||||
if(!envelope.getBcc().isEmpty())
|
||||
{
|
||||
personalization.put("bcc", addresses(envelope.getBcc()));
|
||||
}
|
||||
body.put("personalizations", List.of(personalization));
|
||||
body.put("from", addresses(List.of(envelope.getFrom())).getFirst());
|
||||
if(envelope.getReplyTo() != null)
|
||||
{
|
||||
body.put("reply_to", addresses(List.of(envelope.getReplyTo())).getFirst());
|
||||
}
|
||||
body.put("subject", envelope.getSubject());
|
||||
List<Map<String, String>> content = new ArrayList<>();
|
||||
if(envelope.getTextBody() != null && !envelope.getTextBody().isEmpty())
|
||||
{
|
||||
content.add(Map.of("type", "text/plain", "value", envelope.getTextBody()));
|
||||
}
|
||||
if(envelope.getHtmlBody() != null && !envelope.getHtmlBody().isEmpty())
|
||||
{
|
||||
content.add(Map.of("type", "text/html", "value", envelope.getHtmlBody()));
|
||||
}
|
||||
body.put("content", content);
|
||||
Map<String, String> headers = EmailHttpSupport.headers(envelope);
|
||||
headers.keySet().removeIf(name -> RESERVED_HEADERS.contains(name.toLowerCase(Locale.ROOT)));
|
||||
body.put("headers", headers);
|
||||
body.put("custom_args", Map.of("qqq_email_id", envelope.getMessageId()));
|
||||
body.put("tracking_settings", Map.of("open_tracking", Map.of("enable", settings.enabled("trackOpens")),
|
||||
"click_tracking", Map.of("enable", settings.enabled("trackClicks"), "enable_text", settings.enabled("trackClicks"))));
|
||||
HttpResponse<String> response = EmailHttpSupport.post(http, "https://api.sendgrid.com/v3/mail/send", "application/json",
|
||||
"Authorization", "Bearer " + settings.requiredSecret(), EmailHttpSupport.json(body));
|
||||
EmailHttpSupport.checkStatus(response, "SendGrid", 202);
|
||||
return (EmailHttpSupport.result(response.headers().firstValue("X-Message-Id").orElse(null)));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Convert mailboxes without losing their decoded display names.
|
||||
*******************************************************************************/
|
||||
private static List<Map<String, String>> addresses(List<String> values) throws EmailTransportException
|
||||
{
|
||||
List<Map<String, String>> result = new ArrayList<>();
|
||||
try
|
||||
{
|
||||
for(String value : values)
|
||||
{
|
||||
for(InternetAddress address : InternetAddress.parse(value, true))
|
||||
{
|
||||
address.validate();
|
||||
Map<String, String> entry = new LinkedHashMap<>();
|
||||
entry.put("email", address.getAddress());
|
||||
if(address.getPersonal() != null)
|
||||
{
|
||||
entry.put("name", address.getPersonal());
|
||||
}
|
||||
result.add(entry);
|
||||
}
|
||||
}
|
||||
}
|
||||
catch(AddressException e)
|
||||
{
|
||||
throw (new EmailTransportException("Invalid email message", true));
|
||||
}
|
||||
return (result);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,163 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.transport;
|
||||
|
||||
|
||||
import java.io.ByteArrayOutputStream;
|
||||
import java.io.IOException;
|
||||
import java.time.Duration;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Properties;
|
||||
import java.util.concurrent.atomic.AtomicBoolean;
|
||||
import java.util.function.Function;
|
||||
import jakarta.mail.MessagingException;
|
||||
import jakarta.mail.Session;
|
||||
import jakarta.mail.internet.AddressException;
|
||||
import jakarta.mail.internet.InternetAddress;
|
||||
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
|
||||
import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
|
||||
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
|
||||
import software.amazon.awssdk.core.SdkBytes;
|
||||
import software.amazon.awssdk.regions.Region;
|
||||
import software.amazon.awssdk.services.sesv2.SesV2Client;
|
||||
import software.amazon.awssdk.services.sesv2.SesV2ClientBuilder;
|
||||
import software.amazon.awssdk.services.sesv2.model.Destination;
|
||||
import software.amazon.awssdk.services.sesv2.model.EmailContent;
|
||||
import software.amazon.awssdk.services.sesv2.model.MessageRejectedException;
|
||||
import software.amazon.awssdk.services.sesv2.model.RawMessage;
|
||||
import software.amazon.awssdk.services.sesv2.model.SendEmailRequest;
|
||||
import software.amazon.awssdk.services.sesv2.model.SesV2Exception;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** SES v2 raw MIME transport; credentials are resolved only for an attempt.
|
||||
*******************************************************************************/
|
||||
public class SesEmailTransport implements EmailTransport
|
||||
{
|
||||
private final Function<SesV2ClientBuilder, SesV2Client> clientFactory;
|
||||
|
||||
/*******************************************************************************
|
||||
** Production client factory.
|
||||
*******************************************************************************/
|
||||
public SesEmailTransport()
|
||||
{
|
||||
this(SesV2ClientBuilder::build);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Injectable client factory for embedded hosts and local demonstrations.
|
||||
*******************************************************************************/
|
||||
public SesEmailTransport(Function<SesV2ClientBuilder, SesV2Client> clientFactory)
|
||||
{
|
||||
this.clientFactory = clientFactory;
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Send the same MIME parts and headers as SMTP.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public EmailTransportResult send(EmailEnvelope envelope, EmailProviderSettings settings) throws EmailTransportException
|
||||
{
|
||||
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
|
||||
try
|
||||
{
|
||||
EmailMime.build(Session.getInstance(new Properties()), envelope).writeTo(bytes, new String[] {"Bcc"});
|
||||
}
|
||||
catch(MessagingException | IOException | RuntimeException e)
|
||||
{
|
||||
throw (new EmailTransportException("Invalid email message", true));
|
||||
}
|
||||
AtomicBoolean credentialsFailed = new AtomicBoolean();
|
||||
try(DefaultCredentialsProvider defaultCredentials = DefaultCredentialsProvider.builder().build())
|
||||
{
|
||||
SesV2ClientBuilder builder = SesV2Client.builder().region(Region.of(settings.value("region")))
|
||||
.overrideConfiguration(c -> c.apiCallTimeout(Duration.ofSeconds(40)).apiCallAttemptTimeout(Duration.ofSeconds(35))
|
||||
.retryPolicy(software.amazon.awssdk.core.retry.RetryPolicy.none()));
|
||||
if(settings.value("usernameEnvName") != null)
|
||||
{
|
||||
builder.credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create(settings.resolve("usernameEnvName"), settings.resolve("secretEnvName"))));
|
||||
}
|
||||
else
|
||||
{
|
||||
builder.credentialsProvider(() ->
|
||||
{
|
||||
try
|
||||
{
|
||||
return (defaultCredentials.resolveCredentials());
|
||||
}
|
||||
catch(RuntimeException e)
|
||||
{
|
||||
credentialsFailed.set(true);
|
||||
throw (e);
|
||||
}
|
||||
});
|
||||
}
|
||||
SendEmailRequest request = SendEmailRequest.builder()
|
||||
.fromEmailAddress(envelope.getFrom())
|
||||
.destination(Destination.builder().toAddresses(destinations(envelope.getTo())).ccAddresses(destinations(envelope.getCc())).bccAddresses(destinations(envelope.getBcc())).build())
|
||||
.configurationSetName(settings.value("configurationSet"))
|
||||
.content(EmailContent.builder().raw(RawMessage.builder().data(SdkBytes.fromByteArray(bytes.toByteArray())).build()).build()).build();
|
||||
try(SesV2Client client = clientFactory.apply(builder))
|
||||
{
|
||||
return (new EmailTransportResult(client.sendEmail(request).messageId()));
|
||||
}
|
||||
}
|
||||
catch(EmailTransportException e)
|
||||
{
|
||||
throw (e);
|
||||
}
|
||||
catch(SesV2Exception e)
|
||||
{
|
||||
String code = e.awsErrorDetails() == null ? null : e.awsErrorDetails().errorCode();
|
||||
boolean clientFailure = e.statusCode() >= 400 && e.statusCode() < 500 && e.statusCode() != 429 && !e.isThrottlingException();
|
||||
boolean permanent = clientFailure && e.statusCode() != 401 && e.statusCode() != 403
|
||||
&& (e instanceof MessageRejectedException || "MessageRejected".equals(code) || "InvalidParameterValue".equals(code));
|
||||
throw (new EmailTransportException(permanent ? "SES rejected the message" : "SES temporarily unavailable", permanent)
|
||||
.withProviderLevel(clientFailure && !permanent));
|
||||
}
|
||||
catch(AddressException e)
|
||||
{
|
||||
throw (new EmailTransportException("Invalid email message", true));
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
throw (new EmailTransportException("SES temporarily unavailable", false).withProviderLevel(credentialsFailed.get()));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** SES envelope destinations are individual mailboxes, without display names.
|
||||
*******************************************************************************/
|
||||
private static List<String> destinations(List<String> values) throws AddressException
|
||||
{
|
||||
List<String> addresses = new ArrayList<>();
|
||||
for(String value : values)
|
||||
{
|
||||
for(InternetAddress address : InternetAddress.parse(value, true))
|
||||
{
|
||||
addresses.add(address.getAddress());
|
||||
}
|
||||
}
|
||||
return (addresses);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,135 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.transport;
|
||||
|
||||
|
||||
import java.util.HashSet;
|
||||
import java.util.Properties;
|
||||
import java.util.Set;
|
||||
import com.sun.mail.smtp.SMTPAddressFailedException;
|
||||
import com.sun.mail.smtp.SMTPSendFailedException;
|
||||
import jakarta.mail.AuthenticationFailedException;
|
||||
import jakarta.mail.MessagingException;
|
||||
import jakarta.mail.Session;
|
||||
import jakarta.mail.Transport;
|
||||
import jakarta.mail.internet.MimeMessage;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Authenticated or relay SMTP with bounded connect, read and write timeouts.
|
||||
*******************************************************************************/
|
||||
public class SmtpEmailTransport implements EmailTransport
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Submit one claimed message. Provider diagnostics are never retained.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public EmailTransportResult send(EmailEnvelope envelope, EmailProviderSettings settings) throws EmailTransportException
|
||||
{
|
||||
if(!settings.allowsSmtpAuth())
|
||||
{
|
||||
throw (new EmailTransportException("SMTP sign-in needs STARTTLS or TLS", false).withProviderLevel(true));
|
||||
}
|
||||
if("GMAIL_SMTP".equals(settings.value("kind"))
|
||||
&& (settings.value("usernameEnvName") == null || settings.value("secretEnvName") == null
|
||||
|| (settings.value("security") != null && !"STARTTLS".equals(settings.value("security")))))
|
||||
{
|
||||
throw (new EmailTransportException("Gmail SMTP needs both credential names and STARTTLS", false).withProviderLevel(true));
|
||||
}
|
||||
Properties properties = new Properties();
|
||||
properties.setProperty("mail.smtp.connectiontimeout", "10000");
|
||||
properties.setProperty("mail.smtp.timeout", "15000");
|
||||
properties.setProperty("mail.smtp.writetimeout", "15000");
|
||||
properties.setProperty("mail.smtp.auth", String.valueOf(settings.value("usernameEnvName") != null));
|
||||
properties.setProperty("mail.smtp.starttls.enable", String.valueOf("STARTTLS".equals(settings.security())));
|
||||
properties.setProperty("mail.smtp.starttls.required", String.valueOf("STARTTLS".equals(settings.security())));
|
||||
properties.setProperty("mail.smtp.ssl.enable", String.valueOf("TLS".equals(settings.security())));
|
||||
properties.setProperty("mail.smtp.ssl.checkserveridentity", "true");
|
||||
Session session = Session.getInstance(properties);
|
||||
MimeMessage message;
|
||||
try
|
||||
{
|
||||
message = EmailMime.build(session, envelope);
|
||||
}
|
||||
catch(MessagingException | RuntimeException e)
|
||||
{
|
||||
throw (new EmailTransportException("Invalid email message", true));
|
||||
}
|
||||
try(Transport transport = session.getTransport("smtp"))
|
||||
{
|
||||
String username = settings.resolve("usernameEnvName");
|
||||
String password = username == null ? null : settings.resolve("secretEnvName");
|
||||
transport.connect(settings.host(), settings.port(), username, password);
|
||||
transport.sendMessage(message, message.getAllRecipients());
|
||||
return (new EmailTransportResult(message.getMessageID()));
|
||||
}
|
||||
catch(MessagingException e)
|
||||
{
|
||||
boolean permanent = permanent(e);
|
||||
throw (new EmailTransportException(permanent ? "SMTP rejected the message" : "SMTP temporarily unavailable", permanent).withProviderLevel(providerLevel(e)));
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Inspect chained SMTP reply codes without exposing addresses or diagnostics.
|
||||
*******************************************************************************/
|
||||
static boolean permanent(Throwable error)
|
||||
{
|
||||
if(providerLevel(error))
|
||||
{
|
||||
return (false);
|
||||
}
|
||||
Set<Throwable> seen = new HashSet<>();
|
||||
while(error != null && seen.add(error))
|
||||
{
|
||||
if(error instanceof SMTPAddressFailedException address && address.getReturnCode() >= 500)
|
||||
{
|
||||
return (true);
|
||||
}
|
||||
if(error instanceof SMTPSendFailedException send && send.getReturnCode() >= 500)
|
||||
{
|
||||
return (true);
|
||||
}
|
||||
error = error instanceof MessagingException messaging && messaging.getNextException() != null ? messaging.getNextException() : error.getCause();
|
||||
}
|
||||
return (false);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Authentication failures affect every message, including temporary refusals.
|
||||
*******************************************************************************/
|
||||
static boolean providerLevel(Throwable error)
|
||||
{
|
||||
Set<Throwable> seen = new HashSet<>();
|
||||
while(error != null && seen.add(error))
|
||||
{
|
||||
if(error instanceof AuthenticationFailedException
|
||||
|| error instanceof SMTPSendFailedException send && (send.getReturnCode() == 530 || send.getReturnCode() == 535)
|
||||
|| error instanceof SMTPAddressFailedException address && (address.getReturnCode() == 530 || address.getReturnCode() == 535))
|
||||
{
|
||||
return (true);
|
||||
}
|
||||
error = error instanceof MessagingException messaging && messaging.getNextException() != null ? messaging.getNextException() : error.getCause();
|
||||
}
|
||||
return (false);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,929 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email;
|
||||
|
||||
import java.net.ServerSocket;
|
||||
import java.time.*;
|
||||
import java.util.*;
|
||||
import java.util.concurrent.*;
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
import com.icegreen.greenmail.util.GreenMail;
|
||||
import com.icegreen.greenmail.util.ServerSetup;
|
||||
import com.kingsrook.qbits.email.delivery.*;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTables;
|
||||
import com.kingsrook.qbits.email.outbox.*;
|
||||
import com.kingsrook.qbits.email.processes.EmailProcesses;
|
||||
import com.kingsrook.qbits.email.support.*;
|
||||
import com.kingsrook.qbits.email.transport.*;
|
||||
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
|
||||
import com.kingsrook.qqq.backend.core.actions.messaging.SendMessageAction;
|
||||
import com.kingsrook.qqq.backend.core.alerts.AlertResolution;
|
||||
import com.kingsrook.qqq.backend.core.alerts.AlertSignal;
|
||||
import com.kingsrook.qqq.backend.core.alerts.AlertSinkInterface;
|
||||
import com.kingsrook.qqq.backend.core.alerts.AlertsInstanceMetaData;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.messaging.*;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.messaging.email.*;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
|
||||
import jakarta.mail.Multipart;
|
||||
import org.junit.jupiter.api.*;
|
||||
import org.junit.jupiter.params.ParameterizedTest;
|
||||
import org.junit.jupiter.params.provider.ValueSource;
|
||||
import static org.assertj.core.api.Assertions.*;
|
||||
|
||||
/*******************************************************************************
|
||||
** Email Delivery Test.
|
||||
*******************************************************************************/
|
||||
class EmailDeliveryTest
|
||||
{
|
||||
GreenMail mail;
|
||||
Instant now = Instant.parse("2026-10-13T12:00:00Z");
|
||||
|
||||
/*******************************************************************************
|
||||
** Setup.
|
||||
*******************************************************************************/
|
||||
@BeforeEach
|
||||
void setup() throws Exception
|
||||
{
|
||||
EmailRuntime.setClock(Clock.fixed(now, ZoneOffset.UTC));
|
||||
H2Support.setup(c ->
|
||||
{
|
||||
});
|
||||
mail = new GreenMail(new ServerSetup(0, "127.0.0.1", "smtp"));
|
||||
mail.start();
|
||||
H2Support.provider(mail.getSmtp().getPort());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Cleanup.
|
||||
*******************************************************************************/
|
||||
@AfterEach
|
||||
void cleanup()
|
||||
{
|
||||
if(mail != null)
|
||||
{
|
||||
mail.stop();
|
||||
}
|
||||
QContext.clear();
|
||||
EmailRuntime.setClock(null);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Request.
|
||||
*******************************************************************************/
|
||||
EmailRequest request()
|
||||
{
|
||||
return (new EmailRequest()
|
||||
.withTo("Person <person@example.com>")
|
||||
.withSubject("Hello")
|
||||
.withMarkdownBody("# Hello\n\nA **message**."));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Message.
|
||||
*******************************************************************************/
|
||||
QRecord message(Integer id) throws Exception
|
||||
{
|
||||
return (EmailStore.get(EmailTables.MESSAGE, id, null));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Fake.
|
||||
*******************************************************************************/
|
||||
EmailDispatcher fake(EmailTransport transport)
|
||||
{
|
||||
return (new EmailDispatcher(new EmailTransportRegistry().withTransport("SMTP", transport)));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Missing credentials and SMTP 535 pause a provider until credentials recover.
|
||||
*******************************************************************************/
|
||||
@ParameterizedTest
|
||||
@ValueSource(booleans = {false, true})
|
||||
void providerCredentialsRecoverWithoutExhaustingMessageBudget(boolean refused) throws Exception
|
||||
{
|
||||
mail.setUser("person@example.com", "login", "password");
|
||||
EmailRuntime.config().withMaxAttempts(1).withSecretResolver(name -> refused ? "wrong" : null);
|
||||
SystemWrites.updateOne(EmailTables.PROVIDER, new QRecord().withValue("id", 1)
|
||||
.withValue("usernameEnvName", "EMAIL_SMTP_USERNAME").withValue("secretEnvName", "EMAIL_SMTP_PASSWORD"));
|
||||
var first = EmailService.send(request());
|
||||
var second = EmailService.send(request());
|
||||
var dispatcher = new EmailDispatcher();
|
||||
for(int i = 0; i < 2; i++)
|
||||
{
|
||||
EmailRuntime.setClock(Clock.fixed(now.plusSeconds(i * 301L), ZoneOffset.UTC));
|
||||
assertThat(dispatcher.runOnce()).isEqualTo(new EmailDispatcher.RunSummary(0, 1, 1));
|
||||
assertThat(message(first.messageId()).getValueString("status")).isEqualTo("RETRY_WAIT");
|
||||
assertThat(message(first.messageId()).getValueInteger("attemptCount")).isEqualTo(i + 1);
|
||||
assertThat(message(first.messageId()).getValueInstant("nextAttemptDate"))
|
||||
.isEqualTo(now.plusSeconds(i * 301L + (i == 0 ? 60 : 300)));
|
||||
assertThat(message(second.messageId()).getValueInteger("attemptCount")).isZero();
|
||||
var provider = H2Support.all(EmailTables.PROVIDER).getFirst();
|
||||
assertThat(provider.getValueString("healthStatus")).isEqualTo("UNHEALTHY");
|
||||
assertThat(provider.getValueString("lastFailureMessage")).isEqualTo("Credentials missing or refused");
|
||||
}
|
||||
EmailRuntime.config().withSecretResolver(name -> name.endsWith("USERNAME") ? "login" : "password");
|
||||
EmailRuntime.setClock(Clock.fixed(now.plusSeconds(602), ZoneOffset.UTC));
|
||||
assertThat(dispatcher.runOnce().sent()).isEqualTo(2);
|
||||
assertThat(message(first.messageId()).getValueString("status")).isEqualTo("SENT");
|
||||
assertThat(H2Support.all(EmailTables.PROVIDER).getFirst().getValueString("healthStatus")).isEqualTo("HEALTHY");
|
||||
assertThat(H2Support.all(EmailTables.ATTEMPT)).extracting(row -> row.getValueString("outcome"))
|
||||
.containsExactly("PROVIDER_FAILURE", "PROVIDER_FAILURE", "SUCCEEDED", "SUCCEEDED");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Ordinary transient failures still have their full budget after provider repair.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void providerFailuresDoNotConsumeLaterTransientBudget() throws Exception
|
||||
{
|
||||
EmailRuntime.config().withMaxAttempts(2);
|
||||
var queued = EmailService.send(request());
|
||||
var broken = fake((e, settings) ->
|
||||
{
|
||||
throw new EmailTransportException("credentials", false).withProviderLevel(true);
|
||||
});
|
||||
for(int i = 0; i < 3; i++)
|
||||
{
|
||||
EmailRuntime.setClock(Clock.fixed(now.plusSeconds(i * 901L), ZoneOffset.UTC));
|
||||
assertThat(broken.sendNow(queued.messageId())).isEqualTo(EmailDispatcher.SendResult.RETRY_WAIT);
|
||||
}
|
||||
var temporary = fake((e, settings) ->
|
||||
{
|
||||
throw new EmailTransportException("temporary", false);
|
||||
});
|
||||
EmailRuntime.setClock(Clock.fixed(now.plusSeconds(2703), ZoneOffset.UTC));
|
||||
assertThat(temporary.sendNow(queued.messageId())).isEqualTo(EmailDispatcher.SendResult.RETRY_WAIT);
|
||||
EmailRuntime.setClock(Clock.fixed(now.plusSeconds(3004), ZoneOffset.UTC));
|
||||
assertThat(temporary.sendNow(queued.messageId())).isEqualTo(EmailDispatcher.SendResult.DEAD_LETTER);
|
||||
assertThat(message(queued.messageId()).getValueInteger("attemptCount")).isEqualTo(5);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Both lease recovery paths exclude earlier provider failures from the budget.
|
||||
*******************************************************************************/
|
||||
@ParameterizedTest
|
||||
@ValueSource(booleans = {false, true})
|
||||
void leaseRecoveryExcludesProviderFailures(boolean maintenance) throws Exception
|
||||
{
|
||||
EmailRuntime.config().withMaxAttempts(2);
|
||||
var queued = EmailService.send(request());
|
||||
for(int i = 0; i < 2; i++)
|
||||
{
|
||||
EmailRuntime.setClock(Clock.fixed(now.plusSeconds(i * 301L), ZoneOffset.UTC));
|
||||
assertThat(fake((e, settings) ->
|
||||
{
|
||||
throw new EmailTransportException("credentials", false).withProviderLevel(true);
|
||||
}).sendNow(queued.messageId())).isEqualTo(EmailDispatcher.SendResult.RETRY_WAIT);
|
||||
}
|
||||
EmailRuntime.setClock(Clock.fixed(now.plusSeconds(602), ZoneOffset.UTC));
|
||||
var slow = fake((e, settings) ->
|
||||
{
|
||||
EmailRuntime.setClock(Clock.fixed(now.plusSeconds(903), ZoneOffset.UTC));
|
||||
var recovery = fake((envelope, provider) -> new EmailTransportResult("recovered"));
|
||||
try
|
||||
{
|
||||
assertThat((maintenance ? recovery.maintain() : recovery.runOnce()).sent()).isEqualTo(1);
|
||||
}
|
||||
catch(Exception ex)
|
||||
{
|
||||
throw new RuntimeException(ex);
|
||||
}
|
||||
return (new EmailTransportResult("late"));
|
||||
});
|
||||
assertThat(slow.sendNow(queued.messageId())).isEqualTo(EmailDispatcher.SendResult.NOT_SENT);
|
||||
assertThat(message(queued.messageId()).getValueString("providerMessageId")).isEqualTo("recovered");
|
||||
assertThat(H2Support.all(EmailTables.ATTEMPT)).extracting(row -> row.getValueString("outcome"))
|
||||
.containsExactly("PROVIDER_FAILURE", "PROVIDER_FAILURE", "LEASE_EXPIRED", "SUCCEEDED");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Even a slow tick stops the failed provider while processing another provider.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void providerFailureStopsTheTickAfterProbeWindowPasses() throws Exception
|
||||
{
|
||||
EmailService.send(request());
|
||||
EmailService.send(request());
|
||||
H2Support.insert(EmailTables.PROVIDER, new QRecord().withValue("name", "other").withValue("kind", "SMTP")
|
||||
.withValue("host", "localhost").withValue("port", 25).withValue("fromAddress", "sender@example.com")
|
||||
.withValue("enabled", true).withValue("isDefault", false));
|
||||
var other = EmailService.send(request().withProviderName("other"));
|
||||
AtomicInteger calls = new AtomicInteger();
|
||||
var dispatcher = new EmailDispatcher(new EmailTransportRegistry().withTransport("SMTP", (e, settings) ->
|
||||
{
|
||||
calls.incrementAndGet();
|
||||
if("relay".equals(settings.value("name")))
|
||||
{
|
||||
throw new EmailTransportException("credentials", false).withProviderLevel(true);
|
||||
}
|
||||
return (new EmailTransportResult("ok"));
|
||||
}))
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Advance time only after the failed attempt has committed.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
protected SendResult attempt(Integer id, boolean immediate) throws QException
|
||||
{
|
||||
SendResult result = super.attempt(id, immediate);
|
||||
EmailRuntime.setClock(Clock.fixed(now.plusSeconds(601), ZoneOffset.UTC));
|
||||
return (result);
|
||||
}
|
||||
};
|
||||
assertThat(dispatcher.runOnce()).isEqualTo(new EmailDispatcher.RunSummary(1, 1, 1));
|
||||
assertThat(calls.get()).isEqualTo(2);
|
||||
assertThat(message(other.messageId()).getValueString("status")).isEqualTo("SENT");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Corrupt queued MIME is dead-lettered and does not prevent the next delivery.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void messageBuildFailureIsPermanent() throws Exception
|
||||
{
|
||||
var invalidAddress = EmailService.send(request());
|
||||
var invalidSubject = EmailService.send(request());
|
||||
var valid = EmailService.send(request());
|
||||
SystemWrites.updateOne(EmailTables.MESSAGE, new QRecord().withValue("id", invalidAddress.messageId()).withValue("fromAddress", "broken <"));
|
||||
SystemWrites.updateOne(EmailTables.MESSAGE, new QRecord().withValue("id", invalidSubject.messageId()).withValue("subject", "bad\r\nheader"));
|
||||
assertThat(new EmailDispatcher().runOnce()).isEqualTo(new EmailDispatcher.RunSummary(1, 2, 0));
|
||||
assertThat(message(invalidAddress.messageId()).getValueString("status")).isEqualTo("DEAD_LETTER");
|
||||
assertThat(message(invalidSubject.messageId()).getValueString("status")).isEqualTo("DEAD_LETTER");
|
||||
assertThat(message(valid.messageId()).getValueString("status")).isEqualTo("SENT");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** A database or runtime failure on one message cannot abort a batch.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void isolatesCheckedAndRuntimeAttemptFailures() throws Exception
|
||||
{
|
||||
var checked = EmailService.send(request());
|
||||
var unchecked = EmailService.send(request());
|
||||
var valid = EmailService.send(request());
|
||||
var dispatcher = new EmailDispatcher()
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Inject failures at the same boundary as database claim/completion errors.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
protected SendResult attempt(Integer id, boolean immediate) throws QException
|
||||
{
|
||||
if(id.equals(checked.messageId()))
|
||||
{
|
||||
throw new QException("test database failure");
|
||||
}
|
||||
if(id.equals(unchecked.messageId()))
|
||||
{
|
||||
throw new IllegalStateException("test runtime failure");
|
||||
}
|
||||
return (super.attempt(id, immediate));
|
||||
}
|
||||
};
|
||||
assertThat(dispatcher.runOnce()).isEqualTo(new EmailDispatcher.RunSummary(1, 2, 0));
|
||||
assertThat(message(valid.messageId()).getValueString("status")).isEqualTo("SENT");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Both unhealthy raises and recovery resolutions tolerate a throwing sink.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void failingAlertSinkCannotFailCommittedOutcomes() throws Exception
|
||||
{
|
||||
ThrowingAlertSink.raised = 0;
|
||||
ThrowingAlertSink.resolved = 0;
|
||||
AlertsInstanceMetaData.ofOrWithNew(QContext.getQInstance())
|
||||
.withSink(new QCodeReference(ThrowingAlertSink.class));
|
||||
var queued = EmailService.send(request());
|
||||
assertThat(fake((e, settings) ->
|
||||
{
|
||||
throw new EmailTransportException("credentials", false).withProviderLevel(true);
|
||||
}).sendNow(queued.messageId())).isEqualTo(EmailDispatcher.SendResult.RETRY_WAIT);
|
||||
assertThat(ThrowingAlertSink.raised).isEqualTo(1);
|
||||
EmailRuntime.setClock(Clock.fixed(now.plusSeconds(301), ZoneOffset.UTC));
|
||||
assertThat(new EmailDispatcher().sendNow(queued.messageId())).isEqualTo(EmailDispatcher.SendResult.SENT);
|
||||
assertThat(ThrowingAlertSink.resolved).isEqualTo(1);
|
||||
assertThat(message(queued.messageId()).getValueString("status")).isEqualTo("SENT");
|
||||
var rejected = EmailService.send(request());
|
||||
assertThat(fake((e, settings) ->
|
||||
{
|
||||
throw new EmailTransportException("rejected", true);
|
||||
}).sendNow(rejected.messageId())).isEqualTo(EmailDispatcher.SendResult.DEAD_LETTER);
|
||||
assertThat(ThrowingAlertSink.raised).isEqualTo(2);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Loadable sink that records invocation before simulating an alert outage.
|
||||
*******************************************************************************/
|
||||
public static class ThrowingAlertSink implements AlertSinkInterface
|
||||
{
|
||||
static int raised;
|
||||
static int resolved;
|
||||
|
||||
/*******************************************************************************
|
||||
** Simulate a failed alert raise.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void raise(AlertSignal signal)
|
||||
{
|
||||
raised++;
|
||||
throw new IllegalStateException("test alert outage");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Simulate a failed recovery resolution.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void resolve(AlertResolution resolution)
|
||||
{
|
||||
resolved++;
|
||||
throw new IllegalStateException("test alert outage");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Heartbeats are not used by email delivery.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void heartbeat(String source, Instant timestamp)
|
||||
{
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Smtp End To End And Idempotency.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void smtpEndToEndAndIdempotency() throws Exception
|
||||
{
|
||||
var queued = EmailService.send(request().withIdempotencyKey("one"));
|
||||
assertThat(queued.status()).isEqualTo(EmailSendResult.Status.QUEUED);
|
||||
assertThat(mail.getReceivedMessages()).isEmpty();
|
||||
var duplicate = EmailService.send(request().withIdempotencyKey("one"));
|
||||
assertThat(duplicate.status()).isEqualTo(EmailSendResult.Status.DUPLICATE);
|
||||
assertThat(duplicate.messageId()).isEqualTo(queued.messageId());
|
||||
assertThat(new EmailDispatcher().runOnce().sent()).isEqualTo(1);
|
||||
assertThat(mail.waitForIncomingEmail(2000, 1)).isTrue();
|
||||
var received = mail.getReceivedMessages()[0];
|
||||
Multipart parts = (Multipart) received.getContent();
|
||||
assertThat(parts.getCount()).isEqualTo(2);
|
||||
assertThat(parts.getBodyPart(0).getContent().toString()).contains("Hello", "message");
|
||||
assertThat(parts.getBodyPart(1).getContent().toString()).contains("<strong>message</strong>");
|
||||
assertThat(received.getHeader("Auto-Submitted")).containsExactly("auto-generated");
|
||||
assertThat(received.getHeader("X-QQQ-Email-Id")).containsExactly(queued.messageId().toString());
|
||||
assertThat(message(queued.messageId()).getValueString("providerMessageId"))
|
||||
.isEqualTo(received.getMessageID());
|
||||
assertThat(message(queued.messageId()).getValueString("status")).isEqualTo("SENT");
|
||||
assertThat(H2Support.all(EmailTables.ATTEMPT)).hasSize(1);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Smtp Authentication And Secret Resolution Only At Send.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void smtpAuthenticationAndSecretResolutionOnlyAtSend() throws Exception
|
||||
{
|
||||
mail.setUser("person@example.com", "login", "password");
|
||||
AtomicInteger resolutions = new AtomicInteger();
|
||||
EmailRuntime.config()
|
||||
.withSecretResolver(
|
||||
name ->
|
||||
{
|
||||
resolutions.incrementAndGet();
|
||||
return (name.endsWith("USERNAME") ? "login" : "password");
|
||||
});
|
||||
SystemWrites.updateOne(
|
||||
EmailTables.PROVIDER,
|
||||
new QRecord()
|
||||
.withValue("id", 1)
|
||||
.withValue("usernameEnvName", "EMAIL_SMTP_USERNAME")
|
||||
.withValue("secretEnvName", "EMAIL_SMTP_PASSWORD"));
|
||||
var queued = EmailService.send(request());
|
||||
assertThat(resolutions.get()).isZero();
|
||||
assertThat(new EmailDispatcher().sendNow(queued.messageId()))
|
||||
.isEqualTo(EmailDispatcher.SendResult.SENT);
|
||||
assertThat(resolutions.get()).isEqualTo(2);
|
||||
assertThat(mail.waitForIncomingEmail(2000, 1)).isTrue();
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Unreachable Smtp Retries.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void unreachableSmtpRetries() throws Exception
|
||||
{
|
||||
int port;
|
||||
try(ServerSocket socket = new ServerSocket(0))
|
||||
{
|
||||
port = socket.getLocalPort();
|
||||
}
|
||||
SystemWrites.updateOne(
|
||||
EmailTables.PROVIDER, new QRecord().withValue("id", 1).withValue("port", port));
|
||||
var queued = EmailService.send(request());
|
||||
assertThat(new EmailDispatcher().sendNow(queued.messageId()))
|
||||
.isEqualTo(EmailDispatcher.SendResult.RETRY_WAIT);
|
||||
assertThat(message(queued.messageId()).getValueInstant("nextAttemptDate"))
|
||||
.isEqualTo(now.plusSeconds(60));
|
||||
assertThat(message(queued.messageId()).getValueString("lastError"))
|
||||
.doesNotContain("person", "Hello");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Retries Backoff Max Attempts And Retry After.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void retriesBackoffMaxAttemptsAndRetryAfter() throws Exception
|
||||
{
|
||||
EmailRuntime.config().withMaxAttempts(3);
|
||||
var queued = EmailService.send(request());
|
||||
var dispatcher =
|
||||
fake(
|
||||
(e, s) ->
|
||||
{
|
||||
throw new EmailTransportException("secret diagnostic", false);
|
||||
});
|
||||
assertThat(dispatcher.runOnce().failed()).isEqualTo(1);
|
||||
assertThat(message(queued.messageId()).getValueInstant("nextAttemptDate"))
|
||||
.isEqualTo(now.plusSeconds(60));
|
||||
assertThat(dispatcher.runOnce().failed()).isZero();
|
||||
EmailRuntime.setClock(Clock.fixed(now.plusSeconds(60), ZoneOffset.UTC));
|
||||
dispatcher.runOnce();
|
||||
assertThat(message(queued.messageId()).getValueInstant("nextAttemptDate"))
|
||||
.isEqualTo(now.plusSeconds(360));
|
||||
EmailRuntime.setClock(Clock.fixed(now.plusSeconds(360), ZoneOffset.UTC));
|
||||
assertThat(dispatcher.runOnce().failed()).isEqualTo(1);
|
||||
assertThat(message(queued.messageId()).getValueString("status")).isEqualTo("DEAD_LETTER");
|
||||
var second = EmailService.send(request());
|
||||
fake((e, s) ->
|
||||
{
|
||||
throw new EmailTransportException("throttle", false).withRetryAfter(Duration.ofHours(2));
|
||||
})
|
||||
.runOnce();
|
||||
assertThat(message(second.messageId()).getValueInstant("nextAttemptDate"))
|
||||
.isEqualTo(now.plusSeconds(3960));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Permanent Failure And Future Transport Are Dead Letters.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void permanentFailureAndFutureTransportAreDeadLetters() throws Exception
|
||||
{
|
||||
var queued = EmailService.send(request());
|
||||
assertThat(
|
||||
fake((e, s) ->
|
||||
{
|
||||
throw new EmailTransportException("rejected", true);
|
||||
})
|
||||
.sendNow(queued.messageId()))
|
||||
.isEqualTo(EmailDispatcher.SendResult.DEAD_LETTER);
|
||||
var future = EmailService.send(request());
|
||||
SystemWrites.updateOne(
|
||||
EmailTables.PROVIDER, new QRecord().withValue("id", 1).withValue("kind", "GMAIL_API"));
|
||||
assertThat(new EmailDispatcher().sendNow(future.messageId()))
|
||||
.isEqualTo(EmailDispatcher.SendResult.DEAD_LETTER);
|
||||
assertThatThrownBy(() -> new EmailTransportRegistry().get("GMAIL_API"))
|
||||
.hasMessage("Gmail API sending is specified but not built; use GMAIL_SMTP");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Expired Lease Retries And Stale Completion Cannot Overwrite.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void expiredLeaseRetriesAndStaleCompletionCannotOverwrite() throws Exception
|
||||
{
|
||||
var queued = EmailService.send(request());
|
||||
var recovery = fake((e, s) -> new EmailTransportResult("recovered"));
|
||||
var slow =
|
||||
fake(
|
||||
(e, s) ->
|
||||
{
|
||||
EmailRuntime.setClock(Clock.fixed(now.plusSeconds(121), ZoneOffset.UTC));
|
||||
try
|
||||
{
|
||||
assertThat(recovery.runOnce().sent()).isEqualTo(1);
|
||||
}
|
||||
catch(Exception ex)
|
||||
{
|
||||
throw new RuntimeException(ex);
|
||||
}
|
||||
return (new EmailTransportResult("late"));
|
||||
});
|
||||
assertThat(slow.sendNow(queued.messageId())).isEqualTo(EmailDispatcher.SendResult.NOT_SENT);
|
||||
assertThat(message(queued.messageId()).getValueString("providerMessageId"))
|
||||
.isEqualTo("recovered");
|
||||
assertThat(H2Support.all(EmailTables.ATTEMPT))
|
||||
.extracting(r -> r.getValueString("outcome"))
|
||||
.containsExactly("LEASE_EXPIRED", "SUCCEEDED");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Suppression Scope Category Expiry And Partial.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void suppressionScopeCategoryExpiryAndPartial() throws Exception
|
||||
{
|
||||
EmailRuntime.config().withPublicBaseUrl("https://app.example.com").withSecretResolver(name -> "test-unsubscribe-key");
|
||||
H2Support.insert(EmailTables.BRAND, new QRecord().withValue("name", "postal").withValue("isDefault", true).withValue("postalAddress", "123 Test Street"));
|
||||
H2Support.insert(
|
||||
EmailTables.SUPPRESSION,
|
||||
new QRecord()
|
||||
.withValue("address", "PERSON@example.com")
|
||||
.withValue("scope", "ALL")
|
||||
.withValue("reason", "HARD_BOUNCE"));
|
||||
var suppressed = EmailService.send(request());
|
||||
assertThat(suppressed.status()).isEqualTo(EmailSendResult.Status.SUPPRESSED);
|
||||
assertThat(new EmailDispatcher().runOnce().sent()).isZero();
|
||||
var partial = EmailService.send(request().withCc("other@example.com"));
|
||||
assertThat(partial.status()).isEqualTo(EmailSendResult.Status.QUEUED);
|
||||
assertThat(message(partial.messageId()).getValueString("toAddresses")).isEmpty();
|
||||
assertThat(message(partial.messageId()).getValueString("suppressedAddresses"))
|
||||
.contains("person@example.com");
|
||||
SystemWrites.delete(
|
||||
EmailTables.SUPPRESSION,
|
||||
new com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter());
|
||||
H2Support.insert(
|
||||
EmailTables.SUPPRESSION,
|
||||
new QRecord()
|
||||
.withValue("address", "person@example.com")
|
||||
.withValue("scope", "NOTIFICATION")
|
||||
.withValue("reason", "UNSUBSCRIBE"));
|
||||
assertThat(EmailService.send(request()).status()).isEqualTo(EmailSendResult.Status.QUEUED);
|
||||
assertThat(EmailService.send(request().withCategory(EmailCategory.NOTIFICATION)).status())
|
||||
.isEqualTo(EmailSendResult.Status.SUPPRESSED);
|
||||
SystemWrites.updateOne(
|
||||
EmailTables.SUPPRESSION, new QRecord().withValue("id", 2).withValue("scope", "ALL"));
|
||||
assertThat(EmailService.send(request().withCategory(EmailCategory.SECURITY)).status())
|
||||
.isEqualTo(EmailSendResult.Status.QUEUED);
|
||||
SystemWrites.updateOne(
|
||||
EmailTables.SUPPRESSION,
|
||||
new QRecord().withValue("id", 2).withValue("expiresDate", now.minusSeconds(1)));
|
||||
assertThat(EmailService.send(request().withCategory(EmailCategory.MARKETING)).status())
|
||||
.isEqualTo(EmailSendResult.Status.QUEUED);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Rate Limits Do Not Consume Attempts And Send After Is Honored.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void rateLimitsDoNotConsumeAttemptsAndSendAfterIsHonored() throws Exception
|
||||
{
|
||||
SystemWrites.updateOne(
|
||||
EmailTables.PROVIDER,
|
||||
new QRecord().withValue("id", 1).withValue("maxPerMinute", 1).withValue("maxPerDay", 2));
|
||||
EmailService.send(request());
|
||||
EmailService.send(request());
|
||||
var deferred = EmailService.send(request().withSendAfter(now.plusSeconds(1000)));
|
||||
var dispatcher = fake((e, s) -> new EmailTransportResult("ok"));
|
||||
assertThat(dispatcher.runOnce().sent()).isEqualTo(1);
|
||||
assertThat(H2Support.all(EmailTables.ATTEMPT)).hasSize(1);
|
||||
EmailRuntime.setClock(Clock.fixed(now.plusSeconds(61), ZoneOffset.UTC));
|
||||
assertThat(dispatcher.runOnce().sent()).isEqualTo(1);
|
||||
assertThat(dispatcher.sendNow(deferred.messageId()))
|
||||
.isEqualTo(EmailDispatcher.SendResult.NOT_SENT);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Transaction Commit Rollback And Cross Backend Callback.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void transactionCommitRollbackAndCrossBackendCallback() throws Exception
|
||||
{
|
||||
try(var tx = QBackendTransaction.openFor(new InsertInput(EmailTables.MESSAGE)))
|
||||
{
|
||||
assertThat(EmailService.send(request().withTransaction(tx, "rdbms")).status())
|
||||
.isEqualTo(EmailSendResult.Status.QUEUED);
|
||||
tx.rollback();
|
||||
}
|
||||
assertThat(H2Support.all(EmailTables.MESSAGE)).isEmpty();
|
||||
try(var tx = QBackendTransaction.openFor(new InsertInput(EmailTables.MESSAGE)))
|
||||
{
|
||||
EmailService.send(request().withTransaction(tx, "rdbms"));
|
||||
tx.commit();
|
||||
}
|
||||
assertThat(H2Support.all(EmailTables.MESSAGE)).hasSize(1);
|
||||
var other = new QBackendTransaction();
|
||||
var result = EmailService.send(request().withTransaction(other, "elsewhere"));
|
||||
assertThat(result.messageId()).isNull();
|
||||
assertThat(H2Support.all(EmailTables.MESSAGE)).hasSize(1);
|
||||
other.commit();
|
||||
assertThat(H2Support.all(EmailTables.MESSAGE)).hasSize(2);
|
||||
var rolledBack = new QBackendTransaction();
|
||||
EmailService.send(request().withTransaction(rolledBack, "elsewhere"));
|
||||
rolledBack.rollback();
|
||||
rolledBack.commit();
|
||||
assertThat(H2Support.all(EmailTables.MESSAGE)).hasSize(2);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Bridge Core Action Queues And Delivers.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void bridgeCoreActionQueuesAndDelivers() throws Exception
|
||||
{
|
||||
new SendMessageAction()
|
||||
.execute(
|
||||
new SendMessageInput()
|
||||
.withMessagingProviderName("qbitEmail")
|
||||
.withSubject("Bridge")
|
||||
.withTo(
|
||||
new MultiParty()
|
||||
.withParty(
|
||||
new Party()
|
||||
.withAddress("person@example.com")
|
||||
.withRole(EmailPartyRole.TO))
|
||||
.withParty(
|
||||
new Party()
|
||||
.withAddress("copy@example.com")
|
||||
.withRole(EmailPartyRole.CC)))
|
||||
.withFrom(new Party().withAddress("sender@example.com").withLabel("Sender"))
|
||||
.withContent(
|
||||
new Content().withContentRole(EmailContentRole.HTML).withBody("<p>Bridge</p>"))
|
||||
.withContent(
|
||||
new Content().withContentRole(EmailContentRole.TEXT).withBody("Bridge")));
|
||||
assertThat(H2Support.all(EmailTables.MESSAGE))
|
||||
.hasSize(1)
|
||||
.allMatch(r -> "QUEUED".equals(r.getValueString("status")));
|
||||
assertThat(mail.getReceivedMessages()).isEmpty();
|
||||
assertThat(new EmailDispatcher().runOnce().sent()).isEqualTo(1);
|
||||
assertThat(mail.waitForIncomingEmail(2000, 2)).isTrue();
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Provider Health Ten Failures Probe And Recovery.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void providerHealthTenFailuresProbeAndRecovery() throws Exception
|
||||
{
|
||||
EmailRuntime.config().withMaxAttempts(20);
|
||||
for(int i = 0; i < 11; i++)
|
||||
{
|
||||
EmailService.send(request());
|
||||
}
|
||||
var dispatcher =
|
||||
fake(
|
||||
(e, s) ->
|
||||
{
|
||||
throw new EmailTransportException("temporary", false);
|
||||
});
|
||||
assertThat(dispatcher.runOnce().failed()).isEqualTo(10);
|
||||
assertThat(H2Support.all(EmailTables.PROVIDER).getFirst().getValueString("healthStatus"))
|
||||
.isEqualTo("UNHEALTHY");
|
||||
assertThat(dispatcher.runOnce().failed()).isZero();
|
||||
EmailRuntime.setClock(Clock.fixed(now.plusSeconds(301), ZoneOffset.UTC));
|
||||
assertThat(dispatcher.maintain().failed()).isEqualTo(1);
|
||||
EmailRuntime.setClock(Clock.fixed(now.plusSeconds(602), ZoneOffset.UTC));
|
||||
assertThat(fake((e, s) -> new EmailTransportResult("ok")).runOnce().sent()).isGreaterThan(0);
|
||||
assertThat(H2Support.all(EmailTables.PROVIDER).getFirst().getValueString("healthStatus"))
|
||||
.isEqualTo("HEALTHY");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Retention Keeps Digest And Unfinished Messages.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void retentionKeepsDigestAndUnfinishedMessages() throws Exception
|
||||
{
|
||||
var sent = EmailService.send(request());
|
||||
fake((e, s) -> new EmailTransportResult("ok")).sendNow(sent.messageId());
|
||||
var waiting = EmailService.send(request());
|
||||
EmailRuntime.setClock(Clock.fixed(now.plus(Duration.ofDays(31)), ZoneOffset.UTC));
|
||||
EmailProcesses.retain();
|
||||
assertThat(message(sent.messageId()).getValueString("htmlBody")).isNull();
|
||||
assertThat(message(sent.messageId()).getValueString("bodySha256")).hasSize(64);
|
||||
assertThat(message(waiting.messageId()).getValueString("htmlBody")).isNotNull();
|
||||
EmailRuntime.config().withMessageRetentionDays(30);
|
||||
EmailProcesses.retain();
|
||||
assertThat(message(sent.messageId())).isNull();
|
||||
assertThat(H2Support.all(EmailTables.ATTEMPT)).isEmpty();
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Two Replicas Claim Each Message Once And Share Rate Budget.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void twoReplicasClaimEachMessageOnceAndShareRateBudget() throws Exception
|
||||
{
|
||||
var instanceA = QContext.getQInstance();
|
||||
var backend =
|
||||
(com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData)
|
||||
instanceA.getBackend("rdbms");
|
||||
var instanceB = H2Support.instance(backend, c ->
|
||||
{
|
||||
});
|
||||
for(int i = 0; i < 25; i++)
|
||||
{
|
||||
EmailService.send(request());
|
||||
}
|
||||
var received = new ConcurrentHashMap<String, AtomicInteger>();
|
||||
var ready = new CyclicBarrier(2);
|
||||
try(var pool = Executors.newFixedThreadPool(2))
|
||||
{
|
||||
List<Future<?>> futures = new ArrayList<>();
|
||||
for(var instance : List.of(instanceA, instanceB))
|
||||
{
|
||||
futures.add(
|
||||
pool.submit(
|
||||
() ->
|
||||
{
|
||||
QContext.init(instance, H2Support.admin());
|
||||
try
|
||||
{
|
||||
ready.await();
|
||||
var worker =
|
||||
fake(
|
||||
(e, s) ->
|
||||
{
|
||||
received
|
||||
.computeIfAbsent(e.getMessageId(), k -> new AtomicInteger())
|
||||
.incrementAndGet();
|
||||
return (new EmailTransportResult("ok"));
|
||||
});
|
||||
for(int i = 0; i < 3; i++)
|
||||
{
|
||||
worker.runOnce();
|
||||
}
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
throw new RuntimeException(e);
|
||||
}
|
||||
finally
|
||||
{
|
||||
QContext.clear();
|
||||
}
|
||||
}));
|
||||
}
|
||||
for(var future : futures)
|
||||
{
|
||||
future.get(30, TimeUnit.SECONDS);
|
||||
}
|
||||
}
|
||||
assertThat(received).hasSize(25);
|
||||
assertThat(received.values()).allMatch(n -> n.get() == 1);
|
||||
assertThat(H2Support.all(EmailTables.ATTEMPT)).hasSize(25);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Concurrent Idempotency Joins Caller Transactions.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void concurrentIdempotencyJoinsCallerTransactions() throws Exception
|
||||
{
|
||||
var instance = QContext.getQInstance();
|
||||
try(var pool = Executors.newFixedThreadPool(4))
|
||||
{
|
||||
var barrier = new CyclicBarrier(4);
|
||||
List<Future<EmailSendResult>> results = new ArrayList<>();
|
||||
for(int i = 0; i < 4; i++)
|
||||
{
|
||||
results.add(
|
||||
pool.submit(
|
||||
() ->
|
||||
{
|
||||
QContext.init(instance, H2Support.admin());
|
||||
try(var tx = QBackendTransaction.openFor(new InsertInput(EmailTables.MESSAGE)))
|
||||
{
|
||||
barrier.await();
|
||||
var result =
|
||||
EmailService.send(
|
||||
request()
|
||||
.withIdempotencyKey("concurrent")
|
||||
.withTransaction(tx, "rdbms"));
|
||||
tx.commit();
|
||||
return (result);
|
||||
}
|
||||
finally
|
||||
{
|
||||
QContext.clear();
|
||||
}
|
||||
}));
|
||||
}
|
||||
List<EmailSendResult> outcomes = new ArrayList<>();
|
||||
for(var f : results)
|
||||
{
|
||||
outcomes.add(f.get(20, TimeUnit.SECONDS));
|
||||
}
|
||||
assertThat(outcomes).filteredOn(r -> r.status() == EmailSendResult.Status.QUEUED).hasSize(1);
|
||||
assertThat(outcomes)
|
||||
.filteredOn(r -> r.status() == EmailSendResult.Status.DUPLICATE)
|
||||
.hasSize(3);
|
||||
assertThat(outcomes.stream().map(EmailSendResult::messageId).distinct()).hasSize(1);
|
||||
}
|
||||
assertThat(H2Support.all(EmailTables.MESSAGE)).hasSize(1);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Replicas Share Rate Limit And Limited Provider Does Not Starve Others.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void replicasShareRateLimitAndLimitedProviderDoesNotStarveOthers() throws Exception
|
||||
{
|
||||
EmailRuntime.config().withBatchSize(1);
|
||||
SystemWrites.updateOne(
|
||||
EmailTables.PROVIDER, new QRecord().withValue("id", 1).withValue("maxPerMinute", 1));
|
||||
var other =
|
||||
H2Support.insert(
|
||||
EmailTables.PROVIDER,
|
||||
new QRecord()
|
||||
.withValue("name", "other")
|
||||
.withValue("kind", "SMTP")
|
||||
.withValue("host", "127.0.0.1")
|
||||
.withValue("port", 25)
|
||||
.withValue("fromAddress", "sender@example.com")
|
||||
.withValue("isDefault", false)
|
||||
.withValue("enabled", true));
|
||||
for(int i = 0; i < 3; i++)
|
||||
{
|
||||
EmailService.send(request());
|
||||
}
|
||||
var otherMessage = EmailService.send(request().withProviderName("other"));
|
||||
var instance = QContext.getQInstance();
|
||||
try(var pool = Executors.newFixedThreadPool(2))
|
||||
{
|
||||
List<Future<?>> futures = new ArrayList<>();
|
||||
for(int i = 0; i < 2; i++)
|
||||
{
|
||||
futures.add(
|
||||
pool.submit(
|
||||
() ->
|
||||
{
|
||||
QContext.init(instance, H2Support.admin());
|
||||
try
|
||||
{
|
||||
fake((e, settings) -> new EmailTransportResult("ok")).runOnce();
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
throw new RuntimeException(e);
|
||||
}
|
||||
finally
|
||||
{
|
||||
QContext.clear();
|
||||
}
|
||||
}));
|
||||
}
|
||||
for(var f : futures)
|
||||
{
|
||||
f.get(20, TimeUnit.SECONDS);
|
||||
}
|
||||
}
|
||||
assertThat(H2Support.all(EmailTables.ATTEMPT)).hasSize(2);
|
||||
assertThat(message(otherMessage.messageId()).getValueString("status")).isEqualTo("SENT");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Maintenance Recovers Disabled Provider Lease And Exhausted Attempts.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void maintenanceRecoversDisabledProviderLeaseAndExhaustedAttempts() throws Exception
|
||||
{
|
||||
EmailRuntime.config().withMaxAttempts(1);
|
||||
var queued = EmailService.send(request());
|
||||
var inFlight =
|
||||
fake(
|
||||
(e, settings) ->
|
||||
{
|
||||
try
|
||||
{
|
||||
SystemWrites.updateOne(
|
||||
EmailTables.PROVIDER,
|
||||
new QRecord().withValue("id", 1).withValue("enabled", false));
|
||||
EmailRuntime.setClock(Clock.fixed(now.plusSeconds(121), ZoneOffset.UTC));
|
||||
new EmailDispatcher().maintain();
|
||||
}
|
||||
catch(Exception ex)
|
||||
{
|
||||
throw new RuntimeException(ex);
|
||||
}
|
||||
return (new EmailTransportResult("too-late"));
|
||||
});
|
||||
assertThat(inFlight.sendNow(queued.messageId())).isEqualTo(EmailDispatcher.SendResult.NOT_SENT);
|
||||
assertThat(message(queued.messageId()).getValueString("status")).isEqualTo("DEAD_LETTER");
|
||||
assertThat(message(queued.messageId()).getValueInstant("finishedDate"))
|
||||
.isEqualTo(now.plusSeconds(121));
|
||||
assertThat(H2Support.all(EmailTables.ATTEMPT).getFirst().getValueString("outcome"))
|
||||
.isEqualTo("LEASE_EXPIRED");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,256 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email;
|
||||
|
||||
|
||||
import java.time.Clock;
|
||||
import java.time.Instant;
|
||||
import java.time.ZoneOffset;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import com.kingsrook.qbits.email.app.EmailHome;
|
||||
import com.kingsrook.qbits.email.metadata.EmailEnums;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTables;
|
||||
import com.kingsrook.qbits.email.render.ContrastCheck;
|
||||
import com.kingsrook.qbits.email.render.EmailLayout;
|
||||
import com.kingsrook.qbits.email.outbox.EmailCategory;
|
||||
import com.kingsrook.qbits.email.support.SystemWrites;
|
||||
import com.kingsrook.qqq.backend.core.actions.dashboard.RenderWidgetAction;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
|
||||
import com.kingsrook.qqq.backend.core.actions.values.SearchPossibleValueSourceAction;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.values.SearchPossibleValueSourceInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
|
||||
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.RawHTML;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import org.jsoup.Jsoup;
|
||||
import org.jsoup.nodes.Element;
|
||||
import org.junit.jupiter.api.AfterEach;
|
||||
import org.junit.jupiter.api.BeforeEach;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Home data and rendered output, real record labels, and branded preview contrast.
|
||||
*******************************************************************************/
|
||||
class EmailHomePolishTest
|
||||
{
|
||||
private static final Instant NOW = Instant.parse("2026-10-14T02:12:00Z");
|
||||
|
||||
/*******************************************************************************
|
||||
** Use a fixed UTC clock and isolated database for exact assertions.
|
||||
*******************************************************************************/
|
||||
@BeforeEach
|
||||
void setup() throws Exception
|
||||
{
|
||||
H2Support.setup(c -> { });
|
||||
EmailRuntime.setClock(Clock.fixed(NOW, ZoneOffset.UTC));
|
||||
H2Support.provider(25);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Clear session and clock.
|
||||
*******************************************************************************/
|
||||
@AfterEach
|
||||
void cleanup()
|
||||
{
|
||||
QContext.clear();
|
||||
EmailRuntime.setClock(null);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Exact counts, plain labels, large number first, and failures alone are red.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void tilesAndOrderedSections() throws Exception
|
||||
{
|
||||
SystemWrites.insertOne(EmailTables.MESSAGE, new QRecord().withValue("status", "DEAD_LETTER")
|
||||
.withValue("sentDate", NOW.minusSeconds(86400)).withValue("deliveredDate", NOW.plusSeconds(1))
|
||||
.withValue("deadLetterDate", NOW).withValue("bouncedDate", NOW.minusSeconds(7 * 86400))
|
||||
.withValue("complainedDate", NOW.minusSeconds(7 * 86400 + 1)));
|
||||
H2Support.insert(EmailTables.SUPPRESSION, new QRecord().withValue("address", "a@example.test").withValue("scope", "ALL").withValue("reason", "MANUAL"));
|
||||
var document = Jsoup.parse(EmailHome.overview().getHtml());
|
||||
var tiles = document.select("body > div > div > div");
|
||||
assertThat(tiles).extracting(Element::text).containsExactly("1 Sent (24 h)", "0 Delivered (24 h)", "1 Dead letters (24 h)", "1 Bounced (7 d)", "0 Complaints (7 d)", "1 Suppressed addresses");
|
||||
for(int i = 0; i < tiles.size(); i++)
|
||||
{
|
||||
assertThat(tiles.get(i).child(0).attr("style")).contains("font-size:1.6rem");
|
||||
assertThat(tiles.get(i).child(0).attr("style").contains("#d64545")).isEqualTo(i == 2 || i == 3);
|
||||
}
|
||||
assertThat(document.select("h3")).extracting(Element::text).containsExactly("Sends per hour, last 24 hours", "Provider health", "Recent messages", "Suppressions added, last 30 days");
|
||||
assertThat(QContext.getQInstance().getApp("email").getWidgets()).containsExactly("emailHome");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Every attempt outcome and suppression reason feeds its actual UTC bucket.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void stackedBarsAndCompactAxes() throws Exception
|
||||
{
|
||||
int number = 0;
|
||||
for(String outcome : List.of("SUCCEEDED", "TRANSIENT_FAILURE", "PROVIDER_FAILURE", "PERMANENT_FAILURE", "LEASE_EXPIRED", "IN_FLIGHT"))
|
||||
{
|
||||
SystemWrites.insertOne(EmailTables.ATTEMPT, new QRecord().withValue("messageId", 1).withValue("attemptNumber", ++number)
|
||||
.withValue("outcome", outcome).withValue("endDate", NOW));
|
||||
}
|
||||
for(var reason : EmailEnums.SuppressionReason.values())
|
||||
{
|
||||
H2Support.insert(EmailTables.SUPPRESSION, new QRecord().withValue("address", reason.name() + "@example.test").withValue("scope", "ALL").withValue("reason", reason.name()));
|
||||
}
|
||||
var hourly = EmailHome.hourly().getChartData();
|
||||
assertThat(hourly.getLabels()).hasSize(24).allMatch(s -> s.matches("[0-2][0-9]:00"));
|
||||
assertThat(hourly.getLabels().getLast()).isEqualTo("02:00");
|
||||
assertThat(hourly.getDatasets().get(0).getData().getLast()).isEqualTo(1);
|
||||
assertThat(hourly.getDatasets().get(1).getData().getLast()).isEqualTo(4);
|
||||
var suppressions = EmailHome.suppressions().getChartData();
|
||||
assertThat(suppressions.getLabels()).hasSize(30).endsWith("Oct 14");
|
||||
assertThat(suppressions.getDatasets()).extracting(d -> d.getLabel()).containsExactly("Hard bounce", "Complaint", "Unsubscribe", "Manual");
|
||||
assertThat(suppressions.getDatasets()).allSatisfy(d -> assertThat(d.getData().getLast()).isEqualTo(1));
|
||||
String html = EmailHome.overview().getHtml();
|
||||
assertThat(html).contains("flex-direction:column-reverse", "02:00; Sent: 1; Failed: 4", "Oct 14; Hard bounce: 1; Complaint: 1; Unsubscribe: 1; Manual: 1")
|
||||
.doesNotContain("<canvas", "<svg", "<style", "overflow:auto", "HARD_BOUNCE", "2026-10-");
|
||||
assertThat(Jsoup.parse(html).select("div[role=img] > div")).hasSize(54).allSatisfy(e -> assertThat(e.attr("style")).contains("flex:1 1 0;min-width:0"));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Twenty newest linked rows wrap long text, mask addresses, and hide PHI.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void recentRowsAndProviderHealth() throws Exception
|
||||
{
|
||||
for(int i = 0; i < 21; i++)
|
||||
{
|
||||
SystemWrites.insertOne(EmailTables.MESSAGE, new QRecord().withValue("status", i == 20 ? "RETRY_WAIT" : "SENT")
|
||||
.withValue("subject", i == 19 ? "private-canary" : "Subject <" + i + ">" + "x".repeat(150))
|
||||
.withValue("phiApproved", i == 19).withValue("toAddresses", "Person <person@" + "x".repeat(100) + ".test>"));
|
||||
}
|
||||
SystemWrites.updateOne(EmailTables.PROVIDER, new QRecord().withValue("id", 1).withValue("lastSuccessDate", NOW)
|
||||
.withValue("healthStatus", "UNHEALTHY").withValue("lastFailureMessage", "<broken>").withValue("maxPerMinute", 2));
|
||||
SystemWrites.insertOne(EmailTables.ATTEMPT, new QRecord().withValue("messageId", 1).withValue("attemptNumber", 1).withValue("providerId", 1).withValue("startDate", NOW));
|
||||
String html = EmailHome.overview().getHtml();
|
||||
var document = Jsoup.parse(html);
|
||||
var rows = document.select("a[href^=/app/email_message/]");
|
||||
assertThat(rows).hasSize(20);
|
||||
assertThat(rows.first().attr("href")).isEqualTo("/app/email_message/21");
|
||||
assertThat(rows.last().attr("href")).isEqualTo("/app/email_message/2");
|
||||
assertThat(rows.first().text()).contains("When", "To", "Subject", "Status", "Waiting to retry");
|
||||
assertThat(html).contains("Secure message", "p***@", "Oct 14, 02:12 UTC", "Unhealthy", "<broken>", "min-width:0", "overflow-wrap:anywhere")
|
||||
.doesNotContain("private-canary", "Person <", "RETRY_WAIT", "2026-10-14T");
|
||||
assertThat(document.select("section").get(1).text()).contains("Last success Oct 14, 02:12 UTC", "Per-minute headroom 1");
|
||||
SystemWrites.updateOne(EmailTables.PROVIDER, new QRecord().withValue("id", 1).withValue("enabled", false).withValue("maxPerMinute", null));
|
||||
assertThat(EmailHome.overview().getHtml()).contains("Disabled", "No limit");
|
||||
QContext.getQSession().setPermissions(Set.of("email.view"));
|
||||
assertThat(EmailHome.overview().getHtml()).contains("Provider health needs email.admin").doesNotContain("<broken>");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Query, get and possible-value labels use the same human-readable titles.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void recordLabels() throws Exception
|
||||
{
|
||||
QRecord version = EmailTemplatesTest.draft("welcome", "MARKDOWN", "[]", "{}", "Hello");
|
||||
assertThat(get(EmailTables.VERSION, version.getValueInteger("id")).getRecordLabel()).isEqualTo("welcome v1");
|
||||
SystemWrites.updateOne(EmailTables.VERSION, new QRecord().withValue("id", version.getValue("id")).withValue("locale", "fr"));
|
||||
assertThat(get(EmailTables.VERSION, version.getValueInteger("id")).getRecordLabel()).isEqualTo("welcome v1 (fr)");
|
||||
assertThat(get(EmailTables.PROVIDER, 1).getRecordLabel()).isEqualTo("Relay");
|
||||
SystemWrites.updateOne(EmailTables.PROVIDER, new QRecord().withValue("id", 1).withValue("label", ""));
|
||||
assertThat(get(EmailTables.PROVIDER, 1).getRecordLabel()).isEqualTo("relay");
|
||||
for(String subject : List.of("A useful subject", "", "enc:v1:private"))
|
||||
{
|
||||
SystemWrites.insertOne(EmailTables.MESSAGE, new QRecord().withValue("subject", subject));
|
||||
}
|
||||
assertThat(get(EmailTables.MESSAGE, 1).getRecordLabel()).isEqualTo("A useful subject");
|
||||
assertThat(get(EmailTables.MESSAGE, 2).getRecordLabel()).isEqualTo("Message 2");
|
||||
assertThat(get(EmailTables.MESSAGE, 3).getRecordLabel()).isEqualTo("Secure message");
|
||||
var queried = new QueryAction().execute(new QueryInput(EmailTables.MESSAGE).withShouldGenerateDisplayValues(true)).getRecords();
|
||||
assertThat(queried).extracting(QRecord::getRecordLabel).containsExactly("A useful subject", "Message 2", "Secure message");
|
||||
var possible = new SearchPossibleValueSourceAction().execute(new SearchPossibleValueSourceInput().withPossibleValueSourceName(EmailTables.VERSION).withSearchTerm("fr"));
|
||||
assertThat(possible.getResults()).extracting(v -> v.getLabel()).contains("welcome v1 (fr)");
|
||||
assertThat(EmailEnums.MessageStatus.RETRY_WAIT.getPossibleValueLabel()).isEqualTo("Waiting to retry");
|
||||
assertThat(EmailEnums.MessageStatus.DEAD_LETTER.getPossibleValueLabel()).isEqualTo("Dead letter");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** The demo brand's actual inline preview matches the email dark media query.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void demoPreviewButtonContrast() throws Exception
|
||||
{
|
||||
QRecord brand = H2Support.insert(EmailTables.BRAND, new QRecord().withValue("name", "Kof22").withValue("label", "Kof22")
|
||||
.withValue("primaryColor", "#2459a6").withValue("buttonTextColor", "#ffffff")
|
||||
.withValue("textColor", "#172033").withValue("backgroundColor", "#ffffff")
|
||||
.withValue("darkPrimaryColor", "#93c5fd").withValue("darkButtonTextColor", "#172033")
|
||||
.withValue("darkBackgroundColor", "#111827").withValue("darkTextColor", "#f9fafb").withValue("isDefault", true));
|
||||
QRecord version = EmailTemplatesTest.draft("preview", "MARKDOWN", "[]", "{}", "[Visit Kof22](https://example.test){.button}");
|
||||
String html = ((RawHTML) new RenderWidgetAction().execute(new RenderWidgetInput().withWidgetMetaData(QContext.getQInstance().getWidget("emailTemplatePreview"))
|
||||
.withUrlParams(Map.of("id", version.getValueString("id")))).getWidgetData()).getHtml();
|
||||
var document = Jsoup.parse(html);
|
||||
Element dark = document.select("h4").stream().filter(e -> e.text().startsWith("Dark")).findFirst().orElseThrow().nextElementSibling();
|
||||
Element button = dark.select("a[href=https://example.test]").first();
|
||||
assertThat(button).isNotNull();
|
||||
String foreground = css(button, "color");
|
||||
String background = css(button.parent(), "background-color");
|
||||
assertThat(foreground).isEqualTo("#172033");
|
||||
assertThat(background).isEqualTo("#93c5fd");
|
||||
assertThat(ContrastCheck.ratio(foreground, background)).isGreaterThanOrEqualTo(4.5);
|
||||
assertThat(dark.attr("style")).contains("background-color:#111827", "color:#f9fafb");
|
||||
assertThat(EmailLayout.render("Body", "", brand, EmailCategory.TRANSACTIONAL, null))
|
||||
.contains(".email-button{background-color:" + background + "!important}", ".email-button a{color:" + foreground + "!important}");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Missing dark button text chooses an accessible colour for either primary.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void accessibleDarkButtonFallback()
|
||||
{
|
||||
for(String primary : List.of("#93c5fd", "#172033", "#777777"))
|
||||
{
|
||||
QRecord brand = new QRecord().withValue("darkPrimaryColor", primary);
|
||||
assertThat(ContrastCheck.ratio(ContrastCheck.color(brand, "darkButtonTextColor"), primary)).isGreaterThanOrEqualTo(4.5);
|
||||
ContrastCheck.validate(brand);
|
||||
}
|
||||
assertThat(ContrastCheck.color(new QRecord().withValue("darkPrimaryColor", "#93c5fd").withValue("darkButtonTextColor", "#172033"), "darkButtonTextColor")).isEqualTo("#172033");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Exercise real get/display-value actions, not just metadata declarations.
|
||||
*******************************************************************************/
|
||||
private static QRecord get(String table, int id) throws Exception
|
||||
{
|
||||
return (new GetAction().execute(new GetInput(table).withPrimaryKey(id).withShouldGenerateDisplayValues(true)).getRecord());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Read the rendered inline pair; remove CSS priority before computing ratio.
|
||||
*******************************************************************************/
|
||||
private static String css(Element element, String property)
|
||||
{
|
||||
return (List.of(element.attr("style").split(";")).stream().map(String::trim).filter(s -> s.startsWith(property + ":"))
|
||||
.findFirst().orElseThrow().substring(property.length() + 1).replace("!important", "").trim());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,502 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email;
|
||||
|
||||
import java.util.*;
|
||||
import com.kingsrook.qbits.email.app.EmailApp;
|
||||
import com.kingsrook.qbits.email.metadata.*;
|
||||
import com.kingsrook.qbits.email.outbox.*;
|
||||
import com.kingsrook.qbits.email.permissions.*;
|
||||
import com.kingsrook.qbits.email.processes.*;
|
||||
import com.kingsrook.qbits.email.support.*;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.*;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.fields.*;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.tables.Capability;
|
||||
import org.junit.jupiter.api.*;
|
||||
import static org.assertj.core.api.Assertions.*;
|
||||
|
||||
/*******************************************************************************
|
||||
** Email Metadata Test.
|
||||
*******************************************************************************/
|
||||
class EmailMetadataTest
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Setup.
|
||||
*******************************************************************************/
|
||||
@BeforeEach
|
||||
void setup() throws Exception
|
||||
{
|
||||
H2Support.setup(c ->
|
||||
{
|
||||
});
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Teardown.
|
||||
*******************************************************************************/
|
||||
@AfterEach
|
||||
void teardown()
|
||||
{
|
||||
QContext.clear();
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Producer Standards And Integration Fields.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void producerStandardsAndIntegrationFields() throws Exception
|
||||
{
|
||||
var instance = QContext.getQInstance();
|
||||
assertThat(instance.getTables()).hasSize(7);
|
||||
for(var table : instance.getTables().values())
|
||||
{
|
||||
assertThat(table.getName()).matches("email_[a-z_]+");
|
||||
assertThat(table.getPermissionRules().getPermissionBaseName()).isEqualTo("email");
|
||||
assertThat(table.getBackendName()).isEqualTo("rdbms");
|
||||
assertThat(table.getIsHidden()).isFalse();
|
||||
for(var field : table.getFields().values())
|
||||
{
|
||||
assertThat(field.getBackendName()).matches("[a-z][a-z0-9_]*");
|
||||
}
|
||||
}
|
||||
var children = instance.getApp("email").getChildren();
|
||||
assertThat(children).extracting(c -> c.getName()).containsExactlyElementsOf(EmailTables.MAIN);
|
||||
assertThat(EmailTables.SYSTEM_WRITTEN)
|
||||
.allSatisfy(
|
||||
t ->
|
||||
assertThat(instance.getTable(t).getDisabledCapabilities())
|
||||
.contains(
|
||||
Capability.TABLE_INSERT, Capability.TABLE_UPDATE, Capability.TABLE_DELETE));
|
||||
assertThat(instance.getMessagingProvider("qbitEmail").getType()).isEqualTo("QBIT_EMAIL");
|
||||
assertThat(EmailIntegrations.esbTableNames())
|
||||
.containsExactly(EmailTables.MESSAGE, EmailTables.TEMPLATE, EmailTables.SUPPRESSION);
|
||||
assertThat(EmailIntegrations.apiTableNames()).isEqualTo(EmailIntegrations.esbTableNames());
|
||||
assertThat(EmailIntegrations.quickSearchTables())
|
||||
.allSatisfy(
|
||||
t ->
|
||||
assertThat(t.fieldWeights())
|
||||
.doesNotContainKeys("subject", "htmlBody", "textBody", "variablesSnapshot"));
|
||||
assertThat(
|
||||
new EmailApp.HomeRenderer()
|
||||
.render(
|
||||
new com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput().withWidgetMetaData(instance.getWidget(EmailApp.HOME_WIDGET))))
|
||||
.isNotNull();
|
||||
assertThat(EmailRuntime.isInstalled()).isTrue();
|
||||
assertThat(EmailRuntime.config(new QInstance())).isNull();
|
||||
assertThat(EmailRuntime.config(null)).isNull();
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Provider.
|
||||
*******************************************************************************/
|
||||
QRecord provider(String name)
|
||||
{
|
||||
return (new QRecord()
|
||||
.withValue("name", name)
|
||||
.withValue("kind", "SMTP")
|
||||
.withValue("host", "localhost")
|
||||
.withValue("port", 25)
|
||||
.withValue("fromAddress", "sender@example.com")
|
||||
.withValue("enabled", true)
|
||||
.withValue("isDefault", true));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Insert.
|
||||
*******************************************************************************/
|
||||
QRecord insert(QRecord row) throws Exception
|
||||
{
|
||||
return (new InsertAction()
|
||||
.execute(new InsertInput(EmailTables.PROVIDER).withRecords(List.of(row)))
|
||||
.getRecords()
|
||||
.get(0));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Validates Env Name Allowlist And Never Returns Values.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void validatesEnvNameAllowlistAndNeverReturnsValues() throws Exception
|
||||
{
|
||||
EmailRuntime.config().withSecretResolver(n -> "top-secret-do-not-expose");
|
||||
assertThat(insert(provider("bad").withValue("secretEnvName", "literal-password")).getErrors())
|
||||
.isNotEmpty();
|
||||
assertThat(insert(provider("bad2").withValue("secretEnvName", "OTHER_SECRET")).getErrors())
|
||||
.isNotEmpty();
|
||||
assertThat(
|
||||
insert(provider("domain").withValue("sendingDomain", "different.example")).getErrors())
|
||||
.isNotEmpty();
|
||||
assertThat(insert(provider("port").withValue("port", 0)).getErrors()).isNotEmpty();
|
||||
assertThat(insert(provider("no-default").withValue("isDefault", false)).getErrors())
|
||||
.isNotEmpty();
|
||||
var good = insert(provider("relay").withValue("secretEnvName", "EMAIL_SMTP_PASSWORD"));
|
||||
assertThat(good.getErrors()).isNullOrEmpty();
|
||||
var queried =
|
||||
new QueryAction().execute(new QueryInput(EmailTables.PROVIDER)).getRecords().getFirst();
|
||||
assertThat(queried.getValueString("secretEnvNameStatus")).isEqualTo("Set");
|
||||
assertThat(queried.getValueString("usernameEnvNameStatus")).isEqualTo("Not set");
|
||||
assertThat(queried.getValues().values().toString()).doesNotContain("top-secret");
|
||||
EmailRuntime.config()
|
||||
.withSecretResolver(
|
||||
n ->
|
||||
{
|
||||
throw new RuntimeException("secret in exception");
|
||||
});
|
||||
queried =
|
||||
new QueryAction().execute(new QueryInput(EmailTables.PROVIDER)).getRecords().getFirst();
|
||||
assertThat(queried.getValueString("secretEnvNameStatus")).isEqualTo("Not set");
|
||||
assertThat(insert(provider("duplicate-default")).getErrors()).isNotEmpty();
|
||||
assertThat(insert(provider("secondary").withValue("isDefault", false)).getErrors())
|
||||
.isNullOrEmpty();
|
||||
var updated =
|
||||
new UpdateAction()
|
||||
.execute(
|
||||
new UpdateInput(EmailTables.PROVIDER)
|
||||
.withRecords(
|
||||
List.of(
|
||||
new QRecord()
|
||||
.withValue("id", good.getValue("id"))
|
||||
.withValue("label", "New label"))));
|
||||
assertThat(updated.getRecords().getFirst().getErrors()).isNullOrEmpty();
|
||||
var invalid =
|
||||
new UpdateAction()
|
||||
.execute(
|
||||
new UpdateInput(EmailTables.PROVIDER)
|
||||
.withRecords(
|
||||
List.of(
|
||||
new QRecord()
|
||||
.withValue("id", good.getValue("id"))
|
||||
.withValue("enabled", false))));
|
||||
assertThat(invalid.getRecords().getFirst().getErrors()).isNotEmpty();
|
||||
var delete =
|
||||
new DeleteAction()
|
||||
.execute(
|
||||
new DeleteInput(EmailTables.PROVIDER)
|
||||
.withQueryFilter(EmailStore.equal("id", good.getValueInteger("id"))));
|
||||
assertThat(H2Support.all(EmailTables.PROVIDER)).hasSize(2);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Service Refuses Invalid Inputs And Rendering Escapes Raw Html.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void serviceRefusesInvalidInputsAndRenderingEscapesRawHtml() throws Exception
|
||||
{
|
||||
H2Support.provider(25);
|
||||
var good =
|
||||
new EmailRequest()
|
||||
.withTo("a@example.com")
|
||||
.withSubject("s")
|
||||
.withMarkdownBody("<script>alert(1)</script>\n\n**Hello**");
|
||||
assertThat(EmailService.send(new EmailRequest().withTemplateName("later")).reasons())
|
||||
.contains("Unknown email template");
|
||||
assertThat(EmailService.send(null).status()).isEqualTo(EmailSendResult.Status.REFUSED);
|
||||
assertThat(EmailService.send(new EmailRequest()).status())
|
||||
.isEqualTo(EmailSendResult.Status.REFUSED);
|
||||
assertThat(EmailService.send(new EmailRequest().withSubject("s").withTextBody("body")).status())
|
||||
.isEqualTo(EmailSendResult.Status.REFUSED);
|
||||
assertThat(
|
||||
EmailService.send(
|
||||
new EmailRequest().withTo("not-address").withSubject("s").withTextBody("body"))
|
||||
.status())
|
||||
.isEqualTo(EmailSendResult.Status.REFUSED);
|
||||
assertThat(
|
||||
EmailService.send(
|
||||
new EmailRequest()
|
||||
.withTo("a@example.com\r\nBcc:x@example.com")
|
||||
.withSubject("s")
|
||||
.withTextBody("body"))
|
||||
.status())
|
||||
.isEqualTo(EmailSendResult.Status.REFUSED);
|
||||
assertThat(
|
||||
EmailService.send(
|
||||
new EmailRequest()
|
||||
.withTo("a@example.com")
|
||||
.withSubject("s")
|
||||
.withTextBody("body")
|
||||
.withVariable("phi", "no"))
|
||||
.status())
|
||||
.isEqualTo(EmailSendResult.Status.REFUSED);
|
||||
var rendered = new AdHocEmailRenderer().render(good);
|
||||
assertThat(rendered.html()).contains("<script>").doesNotContain("<script>");
|
||||
assertThat(new AdHocEmailRenderer().render(new EmailRequest().withTextBody("<safe>")).html())
|
||||
.contains("<safe>");
|
||||
assertThat(
|
||||
new AdHocEmailRenderer()
|
||||
.render(new EmailRequest().withHtmlBody("<p>Hello & bye</p>"))
|
||||
.text())
|
||||
.isEqualTo("Hello & bye");
|
||||
assertThat(EmailService.send(good.withProviderName("missing")).status())
|
||||
.isEqualTo(EmailSendResult.Status.REFUSED);
|
||||
assertThat(H2Support.all(EmailTables.MESSAGE)).isEmpty();
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Permission Checker Protects System Tables Providers And Actions.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void permissionCheckerProtectsSystemTablesProvidersAndActions() throws Exception
|
||||
{
|
||||
var checker = new EmailPermissionChecker();
|
||||
QContext.getQSession().setPermissions(Set.of("email.view"));
|
||||
var instance = QContext.getQInstance();
|
||||
checker.checkPermissionsThrowing(
|
||||
new QueryInput(EmailTables.MESSAGE), instance.getTable(EmailTables.MESSAGE));
|
||||
assertThatThrownBy(
|
||||
() ->
|
||||
checker.checkPermissionsThrowing(
|
||||
new InsertInput(EmailTables.MESSAGE), instance.getTable(EmailTables.MESSAGE)))
|
||||
.hasMessageContaining("only");
|
||||
assertThatThrownBy(
|
||||
() ->
|
||||
checker.checkPermissionsThrowing(
|
||||
new QueryInput(EmailTables.PROVIDER), instance.getTable(EmailTables.PROVIDER)))
|
||||
.hasMessageContaining("email.admin");
|
||||
assertThatThrownBy(
|
||||
() ->
|
||||
checker.checkPermissionsThrowing(
|
||||
new com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput(),
|
||||
instance.getProcess(EmailProcesses.SEND_TEMPLATED)))
|
||||
.hasMessageContaining("email.send");
|
||||
QContext.getQSession().setPermissions(Set.of("email.send"));
|
||||
assertThat(EmailPermissionChecker.has("email.send")).isTrue();
|
||||
assertThat(EmailPermissionChecker.has("email.view")).isFalse();
|
||||
instance.setEnforcePermissionsInCoreActions(true);
|
||||
assertThatThrownBy(() -> new QueryAction().execute(new QueryInput(EmailTables.MESSAGE)))
|
||||
.hasMessageContaining("email.view");
|
||||
checker.checkPermissionsThrowing(new com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput(),
|
||||
instance.getProcess(EmailProcesses.SEND_TEMPLATED));
|
||||
assertThat(EmailPermissionChecker.has("email.admin")).isFalse();
|
||||
QContext.getQSession().setPermissions(Set.of("email.manage"));
|
||||
assertThat(EmailPermissionChecker.has("email.send")).isTrue();
|
||||
assertThat(EmailPermissionChecker.has("email.view")).isTrue();
|
||||
assertThat(EmailProcesses.permissionFor(EmailProcesses.SEND_TEST)).isEqualTo("email.admin");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Credentialed SMTP needs encryption unless loopback or explicitly allowed.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void plaintextSmtpAuthenticationValidation()
|
||||
{
|
||||
var config = EmailRuntime.config();
|
||||
assertThat(config.getAllowPlaintextSmtpAuth()).isFalse();
|
||||
for(String kind : List.of("SMTP"))
|
||||
{
|
||||
for(String credential : List.of("usernameEnvName", "secretEnvName"))
|
||||
{
|
||||
var row = provider("auth").withValue("kind", kind).withValue("host", "relay.example.com")
|
||||
.withValue("security", "NONE").withValue(credential, "EMAIL_SMTP_USERNAME");
|
||||
assertThat(EmailTableCustomizers.validateProvider(row, config)).contains("SMTP sign-in needs STARTTLS or TLS");
|
||||
for(String security : List.of("STARTTLS", "TLS"))
|
||||
{
|
||||
assertThat(EmailTableCustomizers.validateProvider(row.withValue("security", security), config)).isEmpty();
|
||||
}
|
||||
row.withValue("security", "NONE");
|
||||
assertThat(EmailTableCustomizers.validateProvider(row, config.withAllowPlaintextSmtpAuth(true))).isEmpty();
|
||||
config.withAllowPlaintextSmtpAuth(false);
|
||||
}
|
||||
}
|
||||
for(String host : List.of("localhost", "LOCALHOST", "127.0.0.1", "::1"))
|
||||
{
|
||||
assertThat(EmailTableCustomizers.validateProvider(provider("local").withValue("host", host)
|
||||
.withValue("security", "NONE").withValue("usernameEnvName", "EMAIL_SMTP_USERNAME"), config)).isEmpty();
|
||||
}
|
||||
assertThat(EmailTableCustomizers.validateProvider(provider("relay").withValue("host", "relay.example.com")
|
||||
.withValue("security", "NONE"), config)).isEmpty();
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Sender headers cannot inject new headers, including sparse UI updates.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void providerHeadersRefuseLineBreaks() throws Exception
|
||||
{
|
||||
var stored = insert(provider("headers"));
|
||||
for(String field : List.of("fromName", "fromAddress", "replyTo"))
|
||||
{
|
||||
for(String newline : List.of("\r", "\n"))
|
||||
{
|
||||
var row = provider("bad").withValue(field, "sender@example.com" + newline + "Bcc: hidden@example.com");
|
||||
assertThat(EmailTableCustomizers.validateProvider(row, EmailRuntime.config())).isNotEmpty();
|
||||
var updated = new UpdateAction().execute(new UpdateInput(EmailTables.PROVIDER)
|
||||
.withRecords(List.of(new QRecord().withValue("id", stored.getValue("id")).withValue(field, row.getValue(field)))));
|
||||
assertThat(updated.getRecords().getFirst().getErrors()).isNotEmpty();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Complaint Suppression Is Admin Protected On Update And Delete.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void complaintSuppressionIsAdminProtectedOnUpdateAndDelete() throws Exception
|
||||
{
|
||||
var row =
|
||||
H2Support.insert(
|
||||
EmailTables.SUPPRESSION,
|
||||
new QRecord()
|
||||
.withValue("address", "a@example.com")
|
||||
.withValue("scope", "ALL")
|
||||
.withValue("reason", "COMPLAINT"));
|
||||
QContext.getQSession().setPermissions(Set.of("email.manage", "email.hasAccess"));
|
||||
var update =
|
||||
new UpdateAction()
|
||||
.execute(
|
||||
new UpdateInput(EmailTables.SUPPRESSION)
|
||||
.withRecords(
|
||||
List.of(
|
||||
new QRecord()
|
||||
.withValue("id", row.getValue("id"))
|
||||
.withValue("reason", "MANUAL"))));
|
||||
assertThat(update.getRecords().getFirst().getErrors()).isNotEmpty();
|
||||
new DeleteAction()
|
||||
.execute(new DeleteInput(EmailTables.SUPPRESSION).withQueryFilter(new QQueryFilter()));
|
||||
assertThat(H2Support.all(EmailTables.SUPPRESSION)).hasSize(1);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Configuration Validation And Customizers.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void configurationValidationAndCustomizers() throws Exception
|
||||
{
|
||||
var config =
|
||||
new EmailQBitConfig()
|
||||
.withBackendName("rdbms")
|
||||
.withSchedulerName("missing")
|
||||
.withMaxAttempts(0)
|
||||
.withBatchSize(0)
|
||||
.withRetryMinutes(List.of())
|
||||
.withLeaseSeconds(1)
|
||||
.withBodyRetentionDays(-1)
|
||||
.withAllowedEnvNames(Set.of("bad-name"))
|
||||
.withAppName("");
|
||||
List<String> errors = new ArrayList<>();
|
||||
config.validate(QContext.getQInstance(), errors);
|
||||
assertThat(errors).hasSizeGreaterThanOrEqualTo(6);
|
||||
assertThat(new EmailQBitConfig().retryMinutesAfterAttempt(20)).isEqualTo(720);
|
||||
for(String kind :
|
||||
List.of(
|
||||
"SMTP",
|
||||
"SES",
|
||||
"GMAIL_SMTP",
|
||||
"POSTMARK",
|
||||
"SENDGRID",
|
||||
"MAILGUN",
|
||||
"GMAIL_API",
|
||||
"future"))
|
||||
{
|
||||
assertThat(
|
||||
new EmailQBitConfig()
|
||||
.withAllowedEnvNames(Set.of("CUSTOM"))
|
||||
.effectiveAllowedEnvNames(kind))
|
||||
.contains("CUSTOM");
|
||||
}
|
||||
H2Support.setup(
|
||||
c ->
|
||||
c.withTableMetaDataCustomizer(
|
||||
(instance, table) -> table.withLabel("Custom " + table.getLabel()))
|
||||
.withSecurityFields(List.of(new QFieldMetaData("tenantId", QFieldType.INTEGER))));
|
||||
assertThat(
|
||||
QContext.getQInstance()
|
||||
.getTable(EmailTables.MESSAGE)
|
||||
.getField("tenantId")
|
||||
.getBackendName())
|
||||
.isEqualTo("tenant_id");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Core Enforcement Rejects Direct Writes But Permits Authorized Enqueue.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void coreEnforcementRejectsDirectWritesButPermitsAuthorizedEnqueue() throws Exception
|
||||
{
|
||||
H2Support.provider(25);
|
||||
QContext.getQInstance().setEnforcePermissionsInCoreActions(true);
|
||||
QContext.getQSession().setPermissions(Set.of("email.view", "email.hasAccess"));
|
||||
var request = new EmailRequest().withTo("a@example.com").withSubject("s").withTextBody("body");
|
||||
assertThat(EmailService.send(request).status()).isEqualTo(EmailSendResult.Status.REFUSED);
|
||||
assertThatThrownBy(
|
||||
() ->
|
||||
new InsertAction()
|
||||
.execute(
|
||||
new InsertInput(EmailTables.MESSAGE).withRecords(List.of(new QRecord()))))
|
||||
.isInstanceOf(com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException.class);
|
||||
assertThatThrownBy(() -> new QueryAction().execute(new QueryInput(EmailTables.PROVIDER)))
|
||||
.isInstanceOf(com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException.class);
|
||||
assertThat(SystemWrites.call(() -> EmailService.send(request)).status())
|
||||
.isEqualTo(EmailSendResult.Status.QUEUED);
|
||||
QContext.getQSession().setPermissions(Set.of("email.send", "email.hasAccess"));
|
||||
assertThat(EmailService.send(request).status()).isEqualTo(EmailSendResult.Status.QUEUED);
|
||||
assertThatThrownBy(() -> new QueryAction().execute(new QueryInput(EmailTables.MESSAGE)))
|
||||
.isInstanceOf(com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException.class);
|
||||
assertThat(SystemWrites.active()).isFalse();
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Bridge Refuses Attachments And Invalid Requests.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void bridgeRefusesAttachmentsAndInvalidRequests() throws Exception
|
||||
{
|
||||
H2Support.provider(25);
|
||||
var bridge = new com.kingsrook.qbits.email.bridge.EmailOutboxMessagingProvider();
|
||||
assertThatThrownBy(
|
||||
() ->
|
||||
bridge.sendMessage(
|
||||
new com.kingsrook.qqq.backend.core.model.actions.messaging.SendMessageInput()
|
||||
.withAttachment(
|
||||
new com.kingsrook.qqq.backend.core.model.actions.messaging
|
||||
.Attachment())))
|
||||
.hasMessageContaining("attachments");
|
||||
assertThatThrownBy(
|
||||
() ->
|
||||
bridge.sendMessage(
|
||||
new com.kingsrook.qqq.backend.core.model.actions.messaging.SendMessageInput()))
|
||||
.hasMessageContaining("subject");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Scheduled metadata uses the configured scheduler and exact intervals.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void scheduledMetadataUsesConfiguredIntervals() throws Exception
|
||||
{
|
||||
QContext.getQInstance().addScheduler(new com.kingsrook.qqq.backend.core.model.metadata.scheduleing.simple.SimpleSchedulerMetaData().withName("scheduler"));
|
||||
EmailQBitConfig config = new EmailQBitConfig().withBackendName("rdbms").withSchedulerName("scheduler");
|
||||
List<String> errors = new ArrayList<>();
|
||||
config.validate(QContext.getQInstance(), errors);
|
||||
assertThat(errors).isEmpty();
|
||||
var scheduled = EmailProcesses.processes(config).stream().filter(p -> p.getSchedule() != null).toList();
|
||||
assertThat(scheduled).hasSize(3).allMatch(p -> p.getIsHidden());
|
||||
assertThat(scheduled).extracting(p -> p.getSchedule().getRepeatSeconds()).containsExactly(15, 300, 86400);
|
||||
}
|
||||
|
||||
}
|
||||
@@ -0,0 +1,174 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email;
|
||||
|
||||
import java.time.*;
|
||||
import java.util.*;
|
||||
import com.icegreen.greenmail.util.*;
|
||||
import com.kingsrook.qbits.email.delivery.*;
|
||||
import com.kingsrook.qbits.email.metadata.*;
|
||||
import com.kingsrook.qbits.email.outbox.*;
|
||||
import com.kingsrook.qbits.email.processes.*;
|
||||
import com.kingsrook.qbits.email.support.*;
|
||||
import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.*;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import org.junit.jupiter.api.*;
|
||||
import static org.assertj.core.api.Assertions.*;
|
||||
|
||||
/*******************************************************************************
|
||||
** Email Processes Test.
|
||||
*******************************************************************************/
|
||||
class EmailProcessesTest
|
||||
{
|
||||
GreenMail mail;
|
||||
|
||||
/*******************************************************************************
|
||||
** Setup.
|
||||
*******************************************************************************/
|
||||
@BeforeEach
|
||||
void setup() throws Exception
|
||||
{
|
||||
H2Support.setup(c ->
|
||||
{
|
||||
});
|
||||
mail = new GreenMail(new ServerSetup(0, "127.0.0.1", "smtp"));
|
||||
mail.start();
|
||||
H2Support.provider(mail.getSmtp().getPort());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Cleanup.
|
||||
*******************************************************************************/
|
||||
@AfterEach
|
||||
void cleanup()
|
||||
{
|
||||
if(mail != null)
|
||||
{
|
||||
mail.stop();
|
||||
}
|
||||
QContext.clear();
|
||||
EmailRuntime.setClock(null);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Request.
|
||||
*******************************************************************************/
|
||||
EmailRequest request()
|
||||
{
|
||||
return (new EmailRequest()
|
||||
.withTo("a@example.com")
|
||||
.withSubject("Test")
|
||||
.withMarkdownBody("Hello"));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Cancel Send Now Resend And Send Test Use Review And Result.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void cancelSendNowResendAndSendTestUseReviewAndResult() throws Exception
|
||||
{
|
||||
var cancelled = EmailService.send(request());
|
||||
var cancel = ProcessRunner.run(EmailProcesses.CANCEL, cancelled.messageId());
|
||||
assertThat(cancel.review()).isNotEmpty();
|
||||
assertThat(cancel.result()).anyMatch(s -> s.contains("CANCELLED"));
|
||||
assertThat(
|
||||
EmailStore.get(EmailTables.MESSAGE, cancelled.messageId(), null)
|
||||
.getValueString("status"))
|
||||
.isEqualTo("CANCELLED");
|
||||
var queued = EmailService.send(request().withSendAfter(Instant.now().plusSeconds(3600)));
|
||||
var sent = ProcessRunner.run(EmailProcesses.SEND_NOW, queued.messageId());
|
||||
assertThat(sent.review()).isNotEmpty();
|
||||
assertThat(sent.result()).anyMatch(s -> s.contains("SENT"));
|
||||
assertThat(mail.waitForIncomingEmail(2000, 1)).isTrue();
|
||||
var dead = EmailService.send(request());
|
||||
SystemWrites.updateOne(
|
||||
EmailTables.MESSAGE,
|
||||
new QRecord().withValue("id", dead.messageId()).withValue("status", "DEAD_LETTER"));
|
||||
var replay = ProcessRunner.run(EmailProcesses.RESEND, dead.messageId());
|
||||
assertThat(replay.result()).anyMatch(s -> s.contains("QUEUED"));
|
||||
var messages = H2Support.all(EmailTables.MESSAGE);
|
||||
assertThat(messages).hasSize(4);
|
||||
assertThat(messages.getLast().getValueString("linkedTableName")).isEqualTo(EmailTables.MESSAGE);
|
||||
assertThat(messages.getLast().getValueString("htmlBody"))
|
||||
.isEqualTo(messages.get(2).getValueString("htmlBody"));
|
||||
var linked = new com.kingsrook.qqq.backend.core.actions.tables.QueryAction().execute(
|
||||
new com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput(EmailTables.MESSAGE)
|
||||
.withFilter(EmailStore.equal("id", messages.getLast().getValueInteger("id"))).withShouldGenerateDisplayValues(true)).getRecords().getFirst();
|
||||
assertThat(linked.getDisplayValue("linkedRecordId:toRecordFromTableDynamic")).isEqualTo(EmailTables.MESSAGE);
|
||||
var test =
|
||||
ProcessRunner.runWithValues(
|
||||
EmailProcesses.SEND_TEST, Map.of("toAddress", "test@example.com"), 1);
|
||||
assertThat(test.result()).anyMatch(s -> s.contains("SENT"));
|
||||
var skipped = ProcessRunner.run(EmailProcesses.CANCEL, queued.messageId());
|
||||
assertThat(skipped.review()).anyMatch(s -> s.contains("skipped"));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Run.
|
||||
*******************************************************************************/
|
||||
RunProcessOutput run(String name, Map<String, java.io.Serializable> values) throws Exception
|
||||
{
|
||||
var input = new RunProcessInput();
|
||||
input.setProcessName(name);
|
||||
values.forEach(input::addValue);
|
||||
input.setFrontendStepBehavior(RunProcessInput.FrontendStepBehavior.SKIP);
|
||||
var result = new RunProcessAction().execute(input);
|
||||
if(result.getException().isPresent())
|
||||
{
|
||||
throw result.getException().get();
|
||||
}
|
||||
return (result);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Backend Send And Scheduled Jobs.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void backendSendAndScheduledJobs() throws Exception
|
||||
{
|
||||
assertThat(QContext.getQInstance().getProcess(EmailProcesses.SEND_TEMPLATED).getBackendStep("send").getInputMetaData().getField("category")).isEmpty();
|
||||
assertThat(QContext.getQInstance().getProcess(EmailProcesses.SEND_AD_HOC).getBackendStep("send").getInputMetaData().getField("category")).isPresent();
|
||||
var queued =
|
||||
run(
|
||||
EmailProcesses.SEND_AD_HOC,
|
||||
Map.of(
|
||||
"to",
|
||||
"a@example.com",
|
||||
"subject",
|
||||
"Backend",
|
||||
"markdownBody",
|
||||
"Hello",
|
||||
"category",
|
||||
"NOTIFICATION"));
|
||||
assertThat(queued.getValues().get("status")).isEqualTo("QUEUED");
|
||||
assertThat(H2Support.all(EmailTables.MESSAGE).getFirst().getValueString("category")).isEqualTo("NOTIFICATION");
|
||||
run(EmailProcesses.DISPATCH, Map.of());
|
||||
assertThat(mail.waitForIncomingEmail(2000, 1)).isTrue();
|
||||
run(EmailProcesses.MAINTAIN, Map.of());
|
||||
EmailRuntime.setClock(Clock.fixed(Instant.now().plus(Duration.ofDays(31)), ZoneOffset.UTC));
|
||||
run(EmailProcesses.RETENTION, Map.of());
|
||||
assertThat(H2Support.all(EmailTables.MESSAGE).getFirst().getValueString("htmlBody")).isNull();
|
||||
var refused = run(EmailProcesses.SEND_TEMPLATED, Map.of("templateName", "later"));
|
||||
assertThat(refused.getValues().get("status")).isEqualTo("REFUSED");
|
||||
}
|
||||
}
|
||||
@@ -20,15 +20,13 @@
|
||||
|
||||
package com.kingsrook.qbits.email;
|
||||
|
||||
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Scaffold checks for the producer and config.
|
||||
*******************************************************************************/
|
||||
@@ -42,9 +40,10 @@ class EmailQBitProducerTest
|
||||
{
|
||||
QInstance instance = new QInstance();
|
||||
instance.addBackend(new QBackendMetaData().withName("memory"));
|
||||
EmailQBitProducer producer = new EmailQBitProducer()
|
||||
.withNamespace(null)
|
||||
.withQBitConfig(new EmailQBitConfig().withBackendName("memory"));
|
||||
EmailQBitProducer producer =
|
||||
new EmailQBitProducer()
|
||||
.withNamespace(null)
|
||||
.withQBitConfig(new EmailQBitConfig().withBackendName("memory"));
|
||||
assertThat(producer.isEnabled()).isTrue();
|
||||
assertThat(producer.getNamespace()).isNull();
|
||||
assertThat(producer.getQBitConfig().getBackendName()).isEqualTo("memory");
|
||||
@@ -52,8 +51,6 @@ class EmailQBitProducerTest
|
||||
assertThat(instance.getQBits()).isNotEmpty();
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A missing or unknown backend is refused; no config means disabled.
|
||||
*******************************************************************************/
|
||||
@@ -61,9 +58,19 @@ class EmailQBitProducerTest
|
||||
void refusesInvalidConfig()
|
||||
{
|
||||
assertThat(new EmailQBitProducer().isEnabled()).isFalse();
|
||||
assertThatThrownBy(() -> new EmailQBitProducer().withQBitConfig(new EmailQBitConfig()).produce(new QInstance()))
|
||||
.isInstanceOf(QException.class).hasMessageContaining("backendName is required");
|
||||
assertThatThrownBy(() -> new EmailQBitProducer().withQBitConfig(new EmailQBitConfig().withBackendName("nope")).produce(new QInstance()))
|
||||
.isInstanceOf(QException.class).hasMessageContaining("backend not found");
|
||||
assertThatThrownBy(
|
||||
() ->
|
||||
new EmailQBitProducer()
|
||||
.withQBitConfig(new EmailQBitConfig())
|
||||
.produce(new QInstance()))
|
||||
.isInstanceOf(QException.class)
|
||||
.hasMessageContaining("backendName is required");
|
||||
assertThatThrownBy(
|
||||
() ->
|
||||
new EmailQBitProducer()
|
||||
.withQBitConfig(new EmailQBitConfig().withBackendName("nope"))
|
||||
.produce(new QInstance()))
|
||||
.isInstanceOf(QException.class)
|
||||
.hasMessageContaining("backend not found");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,175 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email;
|
||||
|
||||
import java.math.BigDecimal;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import com.kingsrook.qbits.email.outbox.EmailCategory;
|
||||
import com.kingsrook.qbits.email.render.ContrastCheck;
|
||||
import com.kingsrook.qbits.email.render.EmailBrands;
|
||||
import com.kingsrook.qbits.email.render.EmailLayout;
|
||||
import com.kingsrook.qbits.email.render.EmailMarkdown;
|
||||
import com.kingsrook.qbits.email.render.HtmlToText;
|
||||
import com.kingsrook.qbits.email.template.CliMjmlCompiler;
|
||||
import com.kingsrook.qbits.email.template.EmailTemplateEngine;
|
||||
import com.kingsrook.qbits.email.template.VariableSchema;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
/*******************************************************************************
|
||||
** Template grammar, content boundaries, accessible layout and validation.
|
||||
*******************************************************************************/
|
||||
class EmailRenderingTest
|
||||
{
|
||||
private final EmailTemplateEngine engine = new EmailTemplateEngine();
|
||||
private final VariableSchema schema = new VariableSchema("[{\"name\":\"name\",\"type\":\"STRING\"},{\"name\":\"items\",\"type\":\"LIST\"},{\"name\":\"record\",\"type\":\"RECORD\"}]");
|
||||
|
||||
/*******************************************************************************
|
||||
** Escaping belongs to the output context, and fallbacks are escaped too.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void escapingAndFallbacks()
|
||||
{
|
||||
String attack = "<img src=x onerror='bad()'> & \"test\"";
|
||||
var parsed = engine.parse("{{name}} {{record.name|<fallback>}}", schema);
|
||||
assertThat(parsed.render(Map.of("name", attack), EmailTemplateEngine.Context.HTML)).contains("<img", "'", """, "<fallback>").doesNotContain("<img");
|
||||
assertThat(parsed.render(Map.of("name", attack), EmailTemplateEngine.Context.TEXT)).contains(attack);
|
||||
String markdown = engine.parse("{{name}}", schema).render(Map.of("name", "[x](javascript:alert(1))\n# heading <script>x</script>"), EmailTemplateEngine.Context.MARKDOWN);
|
||||
String html = EmailMarkdown.render(markdown, new QRecord());
|
||||
assertThat(html).doesNotContain("<a", "<h1", "<script").contains("javascript:alert(1)");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Nested conditionals, loops, dotted values, indices and restored scope.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void blocksAndTruthiness()
|
||||
{
|
||||
var parsed = engine.parse("{{#if name}}Hello {{name}}{{/if}}{{#each items}}[{{@index}}:{{this.field|empty}}{{#if this.ok}}!{{/if}}]{{/each}} {{record.name}}", schema);
|
||||
assertThat(parsed.render(Map.of("name", "Pat", "items", List.of(Map.of("field", "A", "ok", true), Map.of()), "record", Map.of("name", "R")), EmailTemplateEngine.Context.TEXT))
|
||||
.isEqualTo("Hello Pat[0:A!][1:empty] R");
|
||||
assertThat(engine.parse("{{#each items}}{{this}}{{/each}}", schema).render(Map.of("items", List.of("a", "b")), EmailTemplateEngine.Context.HTML)).isEqualTo("ab");
|
||||
for(Object falsy : List.of(false, "", List.of(), Map.of(), 0, new BigDecimal("0.00")))
|
||||
{
|
||||
assertThat(EmailTemplateEngine.truthy(falsy)).isFalse();
|
||||
}
|
||||
assertThat(EmailTemplateEngine.truthy(null)).isFalse();
|
||||
assertThat(EmailTemplateEngine.truthy(-1)).isTrue();
|
||||
assertThat(EmailTemplateEngine.truthy("false")).isTrue();
|
||||
assertThat(engine.parse("literal", schema).render(Map.of(), EmailTemplateEngine.Context.HTML)).isEqualTo("literal");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Authors get precise source-line diagnostics for unsupported syntax.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void parseErrors()
|
||||
{
|
||||
assertThatThrownBy(() -> engine.parse("line\n{{missing}}", schema)).hasMessageContaining("Line 2").hasMessageContaining("missing");
|
||||
assertThatThrownBy(() -> engine.parse("\n{{#if name}}", schema)).hasMessageContaining("Line 2: Unclosed");
|
||||
for(String source : List.of("{{#if name}}{{/each}}", "{{/if}}", "{{this}}", "{{@index}}", "{{{name}}}", "{{> partial}}", "{{name", "{{#if name|fallback}}"))
|
||||
{
|
||||
assertThatThrownBy(() -> engine.parse(source, schema)).isInstanceOf(IllegalArgumentException.class);
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Schema coercion is explicit, including strict URLs and required values.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void schemaValidation()
|
||||
{
|
||||
String json = "[{\"name\":\"n\",\"type\":\"NUMBER\",\"required\":true},{\"name\":\"d\",\"type\":\"DATE\"},{\"name\":\"t\",\"type\":\"DATE_TIME\"},{\"name\":\"b\",\"type\":\"BOOLEAN\"},{\"name\":\"url\",\"type\":\"URL\"}]";
|
||||
VariableSchema schema = new VariableSchema(json);
|
||||
var values = schema.validate(VariableSchema.values("{\"n\":\"1.20\",\"d\":\"2026-10-13\",\"t\":\"2026-10-13T12:00:00Z\",\"b\":\"true\",\"url\":\"https://example.com/path\"}"));
|
||||
assertThat(values.get("n")).isEqualTo(new BigDecimal("1.20"));
|
||||
assertThat(values.get("b")).isEqualTo(true);
|
||||
assertThatThrownBy(() -> schema.validate(Map.of())).hasMessageContaining("Required variable n");
|
||||
assertThatThrownBy(() -> schema.validate(Map.of("n", 1, "b", "yes"))).hasMessageContaining("BOOLEAN");
|
||||
for(String url : List.of("javascript:alert(1)", "data:text/html,x", "/relative", "https://user:pass@example.com", "https://"))
|
||||
{
|
||||
assertThatThrownBy(() -> VariableSchema.url(url)).isInstanceOf(IllegalArgumentException.class);
|
||||
}
|
||||
for(String invalid : List.of("{}", "[{}]", "[{\"name\":\"x\",\"type\":\"CODE\"}]", "[{\"name\":\"x\",\"type\":\"STRING\",\"table\":\"t\"}]"))
|
||||
{
|
||||
assertThatThrownBy(() -> new VariableSchema(invalid)).hasMessage("Invalid variables schema");
|
||||
}
|
||||
assertThatThrownBy(() -> VariableSchema.values("[]")).hasMessage("Variables must be a JSON object");
|
||||
assertThat(schema.knows("brand.site_url")).isTrue();
|
||||
assertThat(schema.knows("brand.missing")).isFalse();
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** CommonMark extensions and table buttons cannot enable raw HTML.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void markdownAndText()
|
||||
{
|
||||
String html = EmailMarkdown.render("# Heading\n\n<script>alert(1)</script>\n\n| A | B |\n|---|---|\n| One | Two |\n\n[Open](https://example.com){.button}\n\nwww.example.com\n\n- First\n- Second", new QRecord());
|
||||
assertThat(html).contains("<h1>", "<script>", "table-layout:fixed", "email-button", "href=\"https://example.com\"", "http://www.example.com").doesNotContain("<script>", "{.button}");
|
||||
String text = HtmlToText.convert(html);
|
||||
assertThat(text).contains("Heading", "One", "Two", "Open (https://example.com)", "- First", "- Second");
|
||||
assertThat(HtmlToText.convert("<html><head><style>bad-css</style></head><body>A & B<br>C<script>bad-code</script></body></html>"))
|
||||
.contains("A & B", "C").doesNotContain("bad-css", "bad-code");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Both schemes, logo variants and category-specific footers are present.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void layoutAndContrast()
|
||||
{
|
||||
QRecord brand = new QRecord().withValue("label", "Brand").withValue("logoUrl", "https://example.com/logo.png")
|
||||
.withValue("logoDarkUrl", "https://example.com/dark.png").withValue("logoAlt", "Brand logo")
|
||||
.withValue("footerMarkdown", "Thanks **friend**").withValue("postalAddress", "123 Main Street")
|
||||
.withValue("socialLinks", "[{\"label\":\"Social\",\"url\":\"https://example.com/social\"}]");
|
||||
ContrastCheck.validate(brand);
|
||||
assertThat(ContrastCheck.ratio("#000000", "#ffffff")).isEqualTo(21);
|
||||
for(EmailCategory category : EmailCategory.values())
|
||||
{
|
||||
String html = EmailLayout.render("<p>Content</p>", "Preheader", brand, category, "https://example.com/unsubscribe");
|
||||
assertThat(html).contains("prefers-color-scheme:dark", "color-scheme", "supported-color-schemes", "[data-ogsc]", "[data-ogsb]", "max-width:600px", "email-logo-dark", "Preheader", "Social").doesNotContain("background-image");
|
||||
assertThat(html.contains("Unsubscribe")).isEqualTo(category == EmailCategory.NOTIFICATION || category == EmailCategory.MARKETING);
|
||||
assertThat(html.contains("123 Main Street")).isEqualTo(category == EmailCategory.MARKETING);
|
||||
}
|
||||
String appended = EmailLayout.appendFooter("<html><body>content</body></html>", brand, EmailCategory.NOTIFICATION, "https://example.com/u");
|
||||
assertThat(appended.indexOf("Unsubscribe")).isLessThan(appended.indexOf("</body>"));
|
||||
assertThat(EmailLayout.appendFooter("fragment", brand, EmailCategory.TRANSACTIONAL, "")).isEqualTo("fragment");
|
||||
assertThatThrownBy(() -> ContrastCheck.validate(new QRecord().withValue("darkTextColor", "#555555"))).hasMessageContaining("dark mode; it needs 4.5:1");
|
||||
assertThatThrownBy(() -> ContrastCheck.validate(new QRecord().withValue("primaryColor", "#ffffff"))).hasMessageContaining("Button text");
|
||||
assertThatThrownBy(() -> EmailBrands.validate(new QRecord().withValue("logoUrl", "javascript:x"))).hasMessageContaining("http or https");
|
||||
assertThatThrownBy(() -> EmailBrands.validate(new QRecord().withValue("textColor", "red;display:none"))).hasMessageContaining("six-digit");
|
||||
assertThatThrownBy(() -> EmailLayout.footer(new QRecord(), EmailCategory.MARKETING, "")).hasMessageContaining("postal address");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Missing or failing CLI tools return only a safe diagnostic.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void cliFailure()
|
||||
{
|
||||
assertThatThrownBy(() -> new CliMjmlCompiler("missing-mjml-binary-for-test").compile("private-source"))
|
||||
.hasMessage("MJML compilation failed or exceeded 20 seconds");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,285 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email;
|
||||
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.Collections;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTables;
|
||||
import com.kingsrook.qbits.email.outbox.EmailCategory;
|
||||
import com.kingsrook.qbits.email.outbox.EmailRequest;
|
||||
import com.kingsrook.qbits.email.outbox.EmailSendResult;
|
||||
import com.kingsrook.qbits.email.outbox.EmailService;
|
||||
import com.kingsrook.qbits.email.phi.EmailBodyEncryption;
|
||||
import com.kingsrook.qbits.email.render.EmailBrands;
|
||||
import com.kingsrook.qbits.email.render.EmailMarkdown;
|
||||
import com.kingsrook.qbits.email.support.EmailStore;
|
||||
import com.kingsrook.qbits.email.support.SystemWrites;
|
||||
import com.kingsrook.qbits.email.template.EmailTemplateEngine;
|
||||
import com.kingsrook.qbits.email.template.EmailTemplateRenderer;
|
||||
import com.kingsrook.qbits.email.template.EmailVersions;
|
||||
import com.kingsrook.qbits.email.template.VariableSchema;
|
||||
import com.kingsrook.qqq.backend.core.actions.dashboard.RenderWidgetAction;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
|
||||
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.RawHTML;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import org.junit.jupiter.api.AfterEach;
|
||||
import org.junit.jupiter.api.BeforeEach;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Regression checks for the PR review's trust and resource boundaries.
|
||||
*******************************************************************************/
|
||||
class EmailReviewFixesTest
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Isolated records and local test keys.
|
||||
*******************************************************************************/
|
||||
@BeforeEach
|
||||
void setup() throws Exception
|
||||
{
|
||||
H2Support.setup(c -> c.withSecretResolver(name -> "test-key").withPublicBaseUrl("https://example.com"));
|
||||
H2Support.provider(25);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Release request state.
|
||||
*******************************************************************************/
|
||||
@AfterEach
|
||||
void cleanup()
|
||||
{
|
||||
QContext.clear();
|
||||
EmailRuntime.setClock(null);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Template categories cannot be downgraded, including notification suppression.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void categoryOverrideRefused() throws Exception
|
||||
{
|
||||
for(String category : List.of("MARKETING", "NOTIFICATION"))
|
||||
{
|
||||
QRecord version = EmailTemplatesTest.draft(category, "HTML", "[]", "{}", "<p>Hello</p>");
|
||||
SystemWrites.updateOne(EmailTables.TEMPLATE, new QRecord().withValue("id", version.getValue("templateId")).withValue("category", category));
|
||||
if("MARKETING".equals(category))
|
||||
{
|
||||
H2Support.insert(EmailTables.BRAND, new QRecord().withValue("name", "brand").withValue("postalAddress", "123 Main St").withValue("isDefault", true));
|
||||
}
|
||||
EmailVersions.activate(version.getValueInteger("id"));
|
||||
var refused = EmailService.send(new EmailRequest().withTemplateName(category).withTo("p@example.com").withCategory(EmailCategory.TRANSACTIONAL));
|
||||
assertThat(refused.status()).isEqualTo(EmailSendResult.Status.REFUSED);
|
||||
assertThat(refused.reasons()).contains("Template category cannot be overridden");
|
||||
assertThat(EmailService.send(new EmailRequest().withTemplateName(category).withTo("p@example.com")).status()).isEqualTo(EmailSendResult.Status.QUEUED);
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Nested loops share work and byte limits, and nesting stops at eight.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void sharedBudgetAndNesting()
|
||||
{
|
||||
var engine = new EmailTemplateEngine();
|
||||
var schema = new VariableSchema("[{\"name\":\"items\",\"type\":\"LIST\"},{\"name\":\"value\",\"type\":\"STRING\"}]");
|
||||
var loop = engine.parse("{{#each items}}{{#each items}}{{/each}}{{/each}}", schema);
|
||||
assertThatThrownBy(() -> loop.render(Map.of("items", Collections.nCopies(100, "")), EmailTemplateEngine.Context.TEXT)).hasMessage("Template output is too large");
|
||||
assertThatThrownBy(() -> engine.parse("{{value}}", schema).render(Map.of("value", "\u00e9".repeat(600_000)), EmailTemplateEngine.Context.TEXT))
|
||||
.hasMessage("Template output is too large");
|
||||
assertThat(engine.parse("{{#if value}}".repeat(8) + "ok" + "{{/if}}".repeat(8), schema).render(Map.of("value", true), EmailTemplateEngine.Context.TEXT)).isEqualTo("ok");
|
||||
assertThatThrownBy(() -> engine.parse("{{#if value}}".repeat(9) + "{{/if}}".repeat(9), schema)).hasMessageContaining("8 levels");
|
||||
var budget = new EmailTemplateEngine.Budget();
|
||||
var parsed = engine.parse("{{value}}", schema);
|
||||
parsed.render(Map.of("value", "x".repeat(600_000)), EmailTemplateEngine.Context.TEXT, budget, new ArrayList<>());
|
||||
assertThatThrownBy(() -> parsed.render(Map.of("value", "x".repeat(600_000)), EmailTemplateEngine.Context.TEXT, budget, new ArrayList<>())).hasMessage("Template output is too large");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Activation reports budgets and sends refuse rather than storing excess output.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void budgetAtLifecycleBoundaries() throws Exception
|
||||
{
|
||||
QRecord version = EmailTemplatesTest.draft("budget", "HTML", "[{\"name\":\"items\",\"type\":\"LIST\"}]", "{}", "{{#each items}}x{{/each}}");
|
||||
EmailVersions.activate(version.getValueInteger("id"));
|
||||
var result = EmailService.send(new EmailRequest().withTemplateName("budget").withTo("p@example.com").withVariable("items", new ArrayList<>(Collections.nCopies(10_001, "x"))));
|
||||
assertThat(result.status()).isEqualTo(EmailSendResult.Status.REFUSED);
|
||||
assertThat(result.reasons()).contains("Template output is too large");
|
||||
QRecord oversized = EmailTemplatesTest.draft("oversized", "HTML", "[]", "{}", "x".repeat(1_048_577));
|
||||
assertThatThrownBy(() -> EmailVersions.activate(oversized.getValueInteger("id"))).hasMessage("Template output is too large");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Unsafe URL values are omitted with warnings, while normal links still work.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void urlContexts()
|
||||
{
|
||||
var engine = new EmailTemplateEngine();
|
||||
var schema = new VariableSchema("[{\"name\":\"value\",\"type\":\"STRING\"}]");
|
||||
for(String attribute : List.of("href", "src", "action", "background"))
|
||||
{
|
||||
for(String quote : List.of("\"", "'"))
|
||||
{
|
||||
var parsed = engine.parse("<a " + attribute + "=" + quote + "{{value}}" + quote + ">x</a>", schema);
|
||||
for(String value : List.of("javascript:alert(1)", "data:text/html,evil"))
|
||||
{
|
||||
List<String> warnings = new ArrayList<>();
|
||||
assertThat(parsed.render(Map.of("value", value), EmailTemplateEngine.Context.HTML, new EmailTemplateEngine.Budget(), warnings)).doesNotContain(value).contains(attribute + "=" + quote + quote);
|
||||
assertThat(warnings).containsExactly("Unsafe URL omitted for value");
|
||||
}
|
||||
assertThat(parsed.render(Map.of("value", "https://example.com"), EmailTemplateEngine.Context.HTML)).contains("https://example.com");
|
||||
assertThat(parsed.render(Map.of("value", "mailto:p@example.com"), EmailTemplateEngine.Context.HTML).contains("mailto:")).isEqualTo(attribute.equals("href"));
|
||||
}
|
||||
}
|
||||
for(String source : List.of("<a href=\" {{value}}\">x</a>", "<a href=\"{{#if value}}{{value}}{{/if}}\">x</a>", "<a href=\"java{{value}}\">x</a>"))
|
||||
{
|
||||
List<String> warnings = new ArrayList<>();
|
||||
String html = engine.parse(source, schema).render(Map.of("value", source.contains("java{{") ? "script:alert(1)" : "javascript:alert(1)"),
|
||||
EmailTemplateEngine.Context.HTML, new EmailTemplateEngine.Budget(), warnings);
|
||||
assertThat(html).contains("href=\"\"").doesNotContain("javascript:");
|
||||
assertThat(warnings).isNotEmpty();
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Refuse unquoted values and includes before a configured compiler executes.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void activationRejectsUnsafeSource() throws Exception
|
||||
{
|
||||
QRecord version = EmailTemplatesTest.draft("unquoted", "HTML", "[]", "{}", "<a href={{brand.site_url}}>Go</a>");
|
||||
assertThatThrownBy(() -> EmailVersions.activate(version.getValueInteger("id"))).hasMessageContaining("must be quoted");
|
||||
for(String source : List.of("<a {{brand.label}}>x</a>", "<{{brand.label}}>"))
|
||||
{
|
||||
assertThatThrownBy(() -> EmailTemplateEngine.validateHtml(source)).hasMessageContaining("must be quoted");
|
||||
}
|
||||
EmailRuntime.config().withMjmlCompiler(source -> { throw new AssertionError("Unsafe MJML reached compiler"); });
|
||||
QRecord mjml = EmailTemplatesTest.draft("includes", "MJML", "[]", "{}", "<mjml><mj-include path='secret'/></mjml>");
|
||||
assertThatThrownBy(() -> EmailVersions.activate(mjml.getValueInteger("id"))).hasMessage("MJML includes are not allowed");
|
||||
assertThat(preview(mjml)).contains("MJML includes are not allowed");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** A viewer can inspect stored output but cannot start a compiler.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void viewerNeverCompiles() throws Exception
|
||||
{
|
||||
QRecord version = EmailTemplatesTest.draft("viewer", "MJML", "[]", "{}", "<mjml/>");
|
||||
EmailRuntime.config().withMjmlCompiler(source -> { throw new AssertionError("Viewer compiled MJML"); });
|
||||
QContext.getQSession().setPermissions(Set.of("email.view"));
|
||||
assertThat(preview(version)).contains("Not compiled yet. Activate the version or use Preview as a manager.");
|
||||
QContext.getQSession().setPermissions(Set.of("email.manage"));
|
||||
EmailRuntime.config().withMjmlCompiler(source -> "<p>Compiled sample</p>");
|
||||
assertThat(preview(version)).contains("Compiled sample");
|
||||
assertThat(EmailStore.get(EmailTables.VERSION, version.getValueInteger("id"), null).getValue("compiledHtml")).isNull();
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Approved sends retain their template brand and restrict final image hosts.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void approvedBrandAndImages() throws Exception
|
||||
{
|
||||
QRecord brand = H2Support.insert(EmailTables.BRAND, new QRecord().withValue("name", "trusted").withValue("label", "Trusted")
|
||||
.withValue("siteUrl", "https://brand.example.com").withValue("logoUrl", "https://other.example.com/logo.png"));
|
||||
H2Support.insert(EmailTables.BRAND, new QRecord().withValue("name", "override").withValue("label", "Override"));
|
||||
QRecord version = EmailTemplatesTest.draft("approved", "HTML", "[]", "{}", "<p>{{brand.label}}</p><img src='https://brand.example.com/a.png'><img src='https://cdn.example.com/b.png'><img src='https://evil.example.com/c.png'>"
|
||||
+ "<!--[if mso]><img src='https://evil.example.com/outlook.png'><![endif]--><style>@IMPORT 'https://evil.example.com/pixel.css';</style>");
|
||||
SystemWrites.updateOne(EmailTables.TEMPLATE, new QRecord().withValue("id", version.getValue("templateId")).withValue("brandId", brand.getValue("id")));
|
||||
EmailVersions.approve(version.getValueInteger("id"));
|
||||
EmailVersions.activate(version.getValueInteger("id"));
|
||||
SystemWrites.updateOne(EmailTables.PROVIDER, new QRecord().withValue("id", 1).withValue("phiAllowed", true));
|
||||
EmailRuntime.config().withAllowedImageHosts(Set.of("cdn.example.com"));
|
||||
var renderer = EmailTemplateRenderer.prepare(new EmailRequest().withTemplateName("approved").withBrandName("override"));
|
||||
QRecord rendered = renderer.render("preview", null, false);
|
||||
assertThat(rendered.getValue("brandId")).isEqualTo(brand.getValue("id"));
|
||||
assertThat(rendered.getValueString("htmlBody")).contains("Trusted", "brand.example.com/a.png", "cdn.example.com/b.png").doesNotContain("Override", "evil.example.com", "other.example.com");
|
||||
assertThat(renderer.warnings()).contains("Brand logo omitted: image host is not allowed", "Image omitted: image host is not allowed");
|
||||
assertThat(EmailBrands.allowedImage("http://brand.example.com/a.png", brand)).isFalse();
|
||||
assertThat(EmailBrands.allowedImage("https://brand.example.com.evil.com/a.png", brand)).isFalse();
|
||||
assertThatThrownBy(() -> H2Support.insert(EmailTables.BRAND, new QRecord().withValue("name", "badlogo").withValue("logoUrl", "http://example.com/logo"))).hasMessageContaining("https");
|
||||
assertThatThrownBy(() -> H2Support.insert(EmailTables.BRAND, new QRecord().withValue("name", "badsocial").withValue("socialLinks", "[{\"label\":\"X\",\"url\":\"http://example.com\"}]"))).hasMessageContaining("https");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Button conversion cannot consume an earlier ordinary link.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void buttonMatchesOneLink()
|
||||
{
|
||||
String html = EmailMarkdown.render("[First](https://first.example) and [Second](https://second.example){.button}", new QRecord());
|
||||
var document = org.jsoup.Jsoup.parse(html);
|
||||
assertThat(document.select(".email-button a").eachAttr("href")).containsExactly("https://second.example");
|
||||
assertThat(document.select("a")).hasSize(2);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Authentication binds both row and column without changing the storage prefix.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void ciphertextSwapFails()
|
||||
{
|
||||
String first = EmailBodyEncryption.encrypt("first", "htmlBody", "1");
|
||||
String second = EmailBodyEncryption.encrypt("second", "htmlBody", "2");
|
||||
assertThat(first).startsWith("enc:v1:");
|
||||
assertThat(EmailBodyEncryption.decrypt(first, "htmlBody", "1")).isEqualTo("first");
|
||||
assertThatThrownBy(() -> EmailBodyEncryption.decrypt(first, "htmlBody", "2")).hasMessageContaining("could not be opened");
|
||||
assertThatThrownBy(() -> EmailBodyEncryption.decrypt(second, "htmlBody", "1")).hasMessageContaining("could not be opened");
|
||||
assertThatThrownBy(() -> EmailBodyEncryption.decrypt(first, "textBody", "1")).hasMessageContaining("could not be opened");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Swapped stored ciphertext never reaches a transport during delivery.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void swappedRowsCannotBeDispatched() throws Exception
|
||||
{
|
||||
QRecord version = EmailTemplatesTest.draft("protected", "HTML", "[]", "{}", "<p>Private message</p>");
|
||||
EmailVersions.approve(version.getValueInteger("id"));
|
||||
EmailVersions.activate(version.getValueInteger("id"));
|
||||
SystemWrites.updateOne(EmailTables.PROVIDER, new QRecord().withValue("id", 1).withValue("phiAllowed", true));
|
||||
var first = EmailService.send(new EmailRequest().withTemplateName("protected").withTo("a@example.com"));
|
||||
var second = EmailService.send(new EmailRequest().withTemplateName("protected").withTo("b@example.com"));
|
||||
QRecord original = EmailStore.get(EmailTables.MESSAGE, first.messageId(), null);
|
||||
SystemWrites.updateOne(EmailTables.MESSAGE, new QRecord().withValue("id", second.messageId()).withValue("htmlBody", original.getValue("htmlBody")));
|
||||
var dispatcher = new com.kingsrook.qbits.email.delivery.EmailDispatcher(new com.kingsrook.qbits.email.transport.EmailTransportRegistry()
|
||||
.withTransport("SMTP", (envelope, settings) -> { throw new AssertionError("Swapped ciphertext reached transport"); }));
|
||||
assertThat(dispatcher.sendNow(second.messageId())).isEqualTo(com.kingsrook.qbits.email.delivery.EmailDispatcher.SendResult.RETRY_WAIT);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Exercise the actual inline widget with current user permissions.
|
||||
*******************************************************************************/
|
||||
private String preview(QRecord version) throws Exception
|
||||
{
|
||||
return (((RawHTML) new RenderWidgetAction().execute(new RenderWidgetInput().withWidgetMetaData(QContext.getQInstance().getWidget("emailTemplatePreview"))
|
||||
.withUrlParams(Map.of("id", version.getValueString("id")))).getWidgetData()).getHtml());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,308 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import com.icegreen.greenmail.util.GreenMail;
|
||||
import com.icegreen.greenmail.util.ServerSetup;
|
||||
import com.kingsrook.qbits.email.app.EmailHome;
|
||||
import com.kingsrook.qbits.email.delivery.EmailDispatcher;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTables;
|
||||
import com.kingsrook.qbits.email.outbox.EmailRequest;
|
||||
import com.kingsrook.qbits.email.outbox.EmailSendResult;
|
||||
import com.kingsrook.qbits.email.outbox.EmailService;
|
||||
import com.kingsrook.qbits.email.processes.EmailProcesses;
|
||||
import com.kingsrook.qbits.email.support.EmailStore;
|
||||
import com.kingsrook.qbits.email.support.SystemWrites;
|
||||
import com.kingsrook.qbits.email.template.EmailVersions;
|
||||
import com.kingsrook.qqq.backend.core.actions.dashboard.RenderWidgetAction;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
|
||||
import com.kingsrook.qqq.backend.core.actions.values.SearchPossibleValueSourceAction;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.values.SearchPossibleValueSourceInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
|
||||
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.RawHTML;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import jakarta.mail.Multipart;
|
||||
import org.junit.jupiter.api.AfterEach;
|
||||
import org.junit.jupiter.api.BeforeEach;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
/*******************************************************************************
|
||||
** Version lifecycle, permissions, metadata filters, previews and real delivery.
|
||||
*******************************************************************************/
|
||||
class EmailTemplatesTest
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Isolated database and sender session.
|
||||
*******************************************************************************/
|
||||
@BeforeEach
|
||||
void setup() throws Exception
|
||||
{
|
||||
H2Support.setup(c -> c.withPublicBaseUrl("https://app.example.com"));
|
||||
H2Support.provider(25);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Clear thread and clock state.
|
||||
*******************************************************************************/
|
||||
@AfterEach
|
||||
void cleanup()
|
||||
{
|
||||
QContext.clear();
|
||||
EmailRuntime.setClock(null);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Reusable real template/version fixture.
|
||||
*******************************************************************************/
|
||||
static QRecord draft(String name, String kind, String schema, String samples, String body) throws Exception
|
||||
{
|
||||
QRecord template = H2Support.insert(EmailTables.TEMPLATE, new QRecord().withValue("name", name).withValue("kind", kind).withValue("category", "TRANSACTIONAL"));
|
||||
QRecord version = EmailVersions.newVersion(template.getValueInteger("id"));
|
||||
QRecord update = new QRecord().withValue("id", version.getValue("id")).withValue("subject", "Hello {{brand.label}} {{message.id}}")
|
||||
.withValue("bodySource", body).withValue("variablesSchema", schema).withValue("sampleVariables", samples);
|
||||
var output = new UpdateAction().execute(new UpdateInput(EmailTables.VERSION).withRecords(List.of(update)));
|
||||
assertThat(output.getRecords().getFirst().getErrors()).isNullOrEmpty();
|
||||
return (EmailStore.get(EmailTables.VERSION, version.getValueInteger("id"), null));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Review/Result creates drafts and atomically retires the old active locale.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void lifecycleActionsAndImmutability() throws Exception
|
||||
{
|
||||
QRecord first = draft("welcome", "MARKDOWN", "[]", "{}", "Hello");
|
||||
assertThat(ProcessRunner.run(EmailProcesses.ACTIVATE_VERSION, first.getValue("id")).result().toString()).contains("ACTIVE");
|
||||
var attempted = new UpdateAction().execute(new UpdateInput(EmailTables.VERSION).withRecords(List.of(new QRecord().withValue("id", first.getValue("id")).withValue("bodySource", "changed"))));
|
||||
assertThat(attempted.getRecords().getFirst().getErrors()).isNotEmpty();
|
||||
assertThat(ProcessRunner.run(EmailProcesses.NEW_VERSION, first.getValue("templateId")).result().toString()).contains("DRAFT 2");
|
||||
QRecord next = H2Support.all(EmailTables.VERSION).stream().filter(v -> v.getValueInteger("versionNumber") == 2).findFirst().orElseThrow();
|
||||
assertThat(next.getValueString("bodySource")).isEqualTo("Hello");
|
||||
EmailVersions.activate(next.getValueInteger("id"));
|
||||
assertThat(EmailStore.get(EmailTables.VERSION, first.getValueInteger("id"), null).getValueString("status")).isEqualTo("RETIRED");
|
||||
assertThat(EmailStore.get(EmailTables.TEMPLATE, first.getValueInteger("templateId"), null).getValue("activeVersionId")).isEqualTo(next.getValue("id"));
|
||||
assertThatThrownBy(() -> EmailVersions.activate(first.getValueInteger("id"))).hasMessageContaining("DRAFT");
|
||||
attempted = new UpdateAction().execute(new UpdateInput(EmailTables.VERSION).withRecords(List.of(new QRecord().withValue("id", first.getValue("id")).withValue("status", "DRAFT"))));
|
||||
assertThat(attempted.getRecords().getFirst().getErrors()).isNotEmpty();
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** New PHI versions need their own approval and drafts freeze on approval.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void phiApprovalAndEditPermission() throws Exception
|
||||
{
|
||||
QRecord version = draft("private_notice", "HTML", "[{\"name\":\"patient\",\"type\":\"STRING\",\"classification\":\"PHI\"}]", "{\"patient\":\"sample\"}", "<p>{{patient}}</p>");
|
||||
assertThatThrownBy(() -> EmailVersions.activate(version.getValueInteger("id"))).hasMessageContaining("patient is PHI");
|
||||
QContext.getQSession().setPermissions(Set.of("email.manage"));
|
||||
assertThatThrownBy(() -> EmailVersions.approve(version.getValueInteger("id"))).hasMessageContaining("email.admin");
|
||||
QContext.getQSession().setPermissions(Set.of("email.admin"));
|
||||
assertThat(ProcessRunner.run(EmailProcesses.APPROVE_PHI, version.getValue("id")).result().toString()).contains("Approved");
|
||||
var edit = new UpdateAction().execute(new UpdateInput(EmailTables.VERSION).withRecords(List.of(new QRecord().withValue("id", version.getValue("id")).withValue("bodySource", "replaced"))));
|
||||
assertThat(edit.getRecords().getFirst().getErrors()).isNotEmpty();
|
||||
EmailVersions.activate(version.getValueInteger("id"));
|
||||
QRecord next = EmailVersions.newVersion(version.getValueInteger("templateId"));
|
||||
assertThatThrownBy(() -> EmailVersions.activate(next.getValueInteger("id"))).hasMessageContaining("not approved");
|
||||
SystemWrites.updateOne(EmailTables.TEMPLATE, new QRecord().withValue("id", version.getValue("templateId")).withValue("editPermissionName", "clinical.edit"));
|
||||
assertThatThrownBy(() -> EmailVersions.newVersion(version.getValueInteger("templateId"))).hasMessageContaining("clinical.edit");
|
||||
QContext.getQSession().setPermissions(Set.of("email.admin", "clinical.edit"));
|
||||
assertThat(EmailVersions.newVersion(version.getValueInteger("templateId")).getValueInteger("versionNumber")).isEqualTo(3);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** MJML requires activation compilation, and sending never runs it again.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void mjmlCompilerAndLocaleFallback() throws Exception
|
||||
{
|
||||
QRecord version = draft("mjml", "MJML", "[]", "{}", "<mjml><mj-body>{{brand.label}}</mj-body></mjml>");
|
||||
assertThatThrownBy(() -> EmailVersions.activate(version.getValueInteger("id"))).hasMessageContaining("configured compiler");
|
||||
EmailRuntime.config().withMjmlCompiler(source -> "<html><body><p>{{brand.label}}</p></body></html>");
|
||||
var preview = new RenderWidgetAction().execute(new RenderWidgetInput().withWidgetMetaData(QContext.getQInstance().getWidget("emailTemplatePreview"))
|
||||
.withUrlParams(Map.of("id", version.getValueString("id"))));
|
||||
assertThat(((RawHTML) preview.getWidgetData()).getHtml()).contains("<p>Email</p>");
|
||||
assertThat(EmailStore.get(EmailTables.VERSION, version.getValueInteger("id"), null).getValue("compiledHtml")).isNull();
|
||||
EmailVersions.activate(version.getValueInteger("id"));
|
||||
EmailRuntime.config().withMjmlCompiler(source -> { throw new AssertionError("Compiler ran at send time"); });
|
||||
var result = EmailService.send(new EmailRequest().withTemplateName("mjml").withLocale("fr").withTo("p@example.com"));
|
||||
assertThat(result.status()).isEqualTo(EmailSendResult.Status.QUEUED);
|
||||
QRecord row = EmailStore.get(EmailTables.MESSAGE, result.messageId(), null);
|
||||
assertThat(row.getValueString("locale")).isEqualTo("en");
|
||||
assertThat(row.getValueString("htmlBody")).contains("<p>Email</p>");
|
||||
assertThat(row.getValueString("subject")).endsWith(result.messageId().toString());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Branded rendered templates reach a real SMTP sink with both alternatives.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void greenMailAndBuiltIn() throws Exception
|
||||
{
|
||||
GreenMail mail = new GreenMail(new ServerSetup(0, "127.0.0.1", "smtp"));
|
||||
try
|
||||
{
|
||||
mail.start();
|
||||
SystemWrites.updateOne(EmailTables.PROVIDER, new QRecord().withValue("id", 1).withValue("port", mail.getSmtp().getPort()));
|
||||
H2Support.insert(EmailTables.BRAND, new QRecord().withValue("name", "blue").withValue("label", "Blue Brand").withValue("primaryColor", "#123456").withValue("isDefault", true));
|
||||
var result = EmailService.send(new EmailRequest().withTemplateName("email_secure_message").withTo("p@example.com").withVariable("link", "https://example.com/secure"));
|
||||
assertThat(result.status()).isEqualTo(EmailSendResult.Status.QUEUED);
|
||||
assertThat(new EmailDispatcher().runOnce().sent()).isEqualTo(1);
|
||||
assertThat(mail.waitForIncomingEmail(2000, 1)).isTrue();
|
||||
Multipart parts = (Multipart) mail.getReceivedMessages()[0].getContent();
|
||||
assertThat(parts.getBodyPart(0).getContent().toString()).contains("Blue Brand", "https://example.com/secure");
|
||||
assertThat(parts.getBodyPart(1).getContent().toString()).contains("#123456", "email-button", "Blue Brand");
|
||||
EmailService.send(new EmailRequest().withTemplateName("email_secure_message").withTo("p@example.com").withVariable("link", "https://example.com/secure"));
|
||||
assertThat(H2Support.all(EmailTables.TEMPLATE)).hasSize(1);
|
||||
}
|
||||
finally
|
||||
{
|
||||
mail.stop();
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Preview and home render through the standard widget endpoint, with empties.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void previewAndHome() throws Exception
|
||||
{
|
||||
assertThat(EmailHome.overview().getHtml()).contains("No messages yet", "Relay").doesNotContain("<style", "overflow:auto");
|
||||
assertThat(EmailHome.hourly().getSublabel()).contains("No completed attempts");
|
||||
assertThat(EmailHome.suppressions().getSublabel()).contains("No new suppressions");
|
||||
QRecord version = draft("preview", "HTML", "[]", "{}", "<html><body><p>Preview text</p><script>evil()</script><img src=x onerror=evil()></body></html>");
|
||||
var widget = QContext.getQInstance().getWidget("emailTemplatePreview");
|
||||
var output = new RenderWidgetAction().execute(new RenderWidgetInput().withWidgetMetaData(widget).withUrlParams(Map.of("id", version.getValueString("id"))));
|
||||
String html = ((RawHTML) output.getWidgetData()).getHtml();
|
||||
assertThat(html).contains("Light", "Dark", "390px", "600px", "Plain text", "Preview text").doesNotContain("<iframe", "<script", "onerror", "<style");
|
||||
EmailVersions.activate(version.getValueInteger("id"));
|
||||
EmailService.send(new EmailRequest().withTemplateName("preview").withTo("Person <person@example.com>"));
|
||||
assertThat(EmailHome.overview().getHtml()).contains("p***@example.com", "/app/email_message/");
|
||||
QContext.getQSession().setPermissions(Set.of("email.view"));
|
||||
assertThat(EmailHome.overview().getHtml()).contains("needs email.admin");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Exact metrics use event dates and viewer permissions, with no sample limits.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void homeCountsAndBrandPrecedence() throws Exception
|
||||
{
|
||||
java.time.Instant now = java.time.Instant.parse("2026-10-13T12:00:00Z");
|
||||
EmailRuntime.setClock(java.time.Clock.fixed(now, java.time.ZoneOffset.UTC));
|
||||
SystemWrites.insertOne(EmailTables.MESSAGE, new QRecord().withValue("status", "DELIVERED").withValue("subject", "Visible")
|
||||
.withValue("toAddresses", "person@example.com").withValue("sentDate", now.minusSeconds(60)).withValue("deliveredDate", now)
|
||||
.withValue("bouncedDate", now.minusSeconds(2 * 86400)).withValue("complainedDate", now.minusSeconds(8 * 86400)));
|
||||
SystemWrites.insertOne(EmailTables.ATTEMPT, new QRecord().withValue("messageId", 1).withValue("providerId", 1).withValue("attemptNumber", 1)
|
||||
.withValue("outcome", "SUCCEEDED").withValue("startDate", now.minusSeconds(10)).withValue("endDate", now));
|
||||
H2Support.insert(EmailTables.SUPPRESSION, new QRecord().withValue("address", "x@example.com").withValue("scope", "ALL").withValue("reason", "MANUAL"));
|
||||
var tiles = org.jsoup.Jsoup.parse(EmailHome.overview().getHtml()).select("body > div > div > div");
|
||||
assertThat(tiles).extracting(org.jsoup.nodes.Element::text).containsExactly("1 Sent (24 h)", "1 Delivered (24 h)", "0 Dead letters (24 h)", "1 Bounced (7 d)", "0 Complaints (7 d)", "1 Suppressed addresses");
|
||||
assertThat(EmailHome.hourly().getChartData().getDatasets().getFirst().getData()).contains(1);
|
||||
assertThat(EmailHome.suppressions().getChartData().getDatasets().get(3).getData()).contains(1);
|
||||
QRecord defaultBrand = H2Support.insert(EmailTables.BRAND, new QRecord().withValue("name", "default").withValue("label", "Default").withValue("isDefault", true));
|
||||
QRecord appBrand = H2Support.insert(EmailTables.BRAND, new QRecord().withValue("name", "app").withValue("label", "App").withValue("appName", "email"));
|
||||
assertThat(com.kingsrook.qbits.email.render.EmailBrands.resolve(null, null).getValue("id")).isEqualTo(appBrand.getValue("id"));
|
||||
assertThat(com.kingsrook.qbits.email.render.EmailBrands.resolve(null, new QRecord().withValue("brandId", defaultBrand.getValue("id"))).getValue("id")).isEqualTo(defaultBrand.getValue("id"));
|
||||
assertThat(com.kingsrook.qbits.email.render.EmailBrands.resolve("app", new QRecord().withValue("brandId", defaultBrand.getValue("id"))).getValue("id")).isEqualTo(appBrand.getValue("id"));
|
||||
assertThatThrownBy(() -> H2Support.insert(EmailTables.BRAND, new QRecord().withValue("name", "bad").withValue("textColor", "#ffffff")))
|
||||
.hasMessageContaining("needs 4.5:1");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Signed links bind the real database id, and HTML footers precede body end.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void signedNotificationFooter() throws Exception
|
||||
{
|
||||
EmailRuntime.config().withSecretResolver(name -> "QQQ_EMAIL_UNSUBSCRIBE_KEY".equals(name) ? "test-unsubscribe-key" : null);
|
||||
QRecord version = draft("notice", "HTML", "[]", "{}", "<html><body><p>Notice</p></body></html>");
|
||||
SystemWrites.updateOne(EmailTables.TEMPLATE, new QRecord().withValue("id", version.getValue("templateId")).withValue("category", "NOTIFICATION"));
|
||||
EmailVersions.activate(version.getValueInteger("id"));
|
||||
var result = EmailService.send(new EmailRequest().withTemplateName("notice").withTo("person@example.com"));
|
||||
assertThat(result.status()).isEqualTo(EmailSendResult.Status.QUEUED);
|
||||
QRecord message = EmailStore.get(EmailTables.MESSAGE, result.messageId(), null);
|
||||
String link = message.getValueString("unsubscribeUrl");
|
||||
assertThat(link).startsWith("https://app.example.com/qqq/hooks/email/unsubscribe/v1.").doesNotContain("person@example.com");
|
||||
String token = link.substring(link.lastIndexOf('/') + 1);
|
||||
String payload = new String(java.util.Base64.getUrlDecoder().decode(token.split("\\.")[1]), java.nio.charset.StandardCharsets.UTF_8);
|
||||
assertThat(payload).contains("\"messageId\":\"" + result.messageId() + "\"", "NOTIFICATION", "addressHash");
|
||||
assertThat(message.getValueString("htmlBody").indexOf("Unsubscribe")).isLessThan(message.getValueString("htmlBody").indexOf("</body>"));
|
||||
var dispatcher = new EmailDispatcher(new com.kingsrook.qbits.email.transport.EmailTransportRegistry().withTransport("SMTP", (envelope, settings) ->
|
||||
{
|
||||
assertThat(envelope.getHeaders()).containsEntry("List-Unsubscribe", "<" + link + ">");
|
||||
return new com.kingsrook.qbits.email.transport.EmailTransportResult("signed");
|
||||
}));
|
||||
assertThat(dispatcher.sendNow(result.messageId())).isEqualTo(EmailDispatcher.SendResult.SENT);
|
||||
EmailRuntime.config().withSecretResolver(name -> null);
|
||||
assertThat(com.kingsrook.qbits.email.render.EmailUnsubscribeLinks.create("1", "address", "NOTIFICATION")).isNull();
|
||||
assertThatThrownBy(() -> com.kingsrook.qbits.email.render.EmailUnsubscribeLinks.create("1", "address", "MARKETING")).hasMessageContaining("QQQ_EMAIL_UNSUBSCRIBE_KEY");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Template active-version possible values are scoped to the template record.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void templateVersionPossibleValues() throws Exception
|
||||
{
|
||||
possibleValues(EmailTables.TEMPLATE, "activeVersionId", "id");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Message version possible values follow the selected template input.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void messageVersionPossibleValues() throws Exception
|
||||
{
|
||||
possibleValues(EmailTables.MESSAGE, "templateVersionId", "templateId");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Approval pointers use the same parent-scoped possible values.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void approvedVersionPossibleValues() throws Exception
|
||||
{
|
||||
possibleValues(EmailTables.TEMPLATE, "approvedVersionId", "id");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Execute core's possible-value search with the interpreted form binding.
|
||||
*******************************************************************************/
|
||||
private void possibleValues(String table, String field, String inputName) throws Exception
|
||||
{
|
||||
QRecord a = draft("a", "HTML", "[]", "{}", "A");
|
||||
QRecord b = draft("b", "HTML", "[]", "{}", "B");
|
||||
var metadata = QContext.getQInstance().getTable(table).getField(field);
|
||||
assertThat(metadata.getPossibleValueSourceFilter()).isNotNull();
|
||||
var filter = metadata.getPossibleValueSourceFilter().clone();
|
||||
assertThat(filter.toString()).contains("${input." + inputName + "}");
|
||||
filter.interpretValues(Map.of(inputName, a.getValue("templateId")));
|
||||
var found = new SearchPossibleValueSourceAction().execute(new SearchPossibleValueSourceInput().withPossibleValueSourceName(metadata.getPossibleValueSourceName()).withDefaultQueryFilter(filter));
|
||||
assertThat(found.getResults().stream().map(v -> v.getId().toString()).toList()).contains(a.getValueString("id")).doesNotContain(b.getValueString("id"));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,166 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email;
|
||||
|
||||
import java.sql.Connection;
|
||||
import java.util.List;
|
||||
import java.util.UUID;
|
||||
import java.util.function.Consumer;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTables;
|
||||
import com.kingsrook.qbits.email.permissions.EmailPermissions;
|
||||
import com.kingsrook.qbits.email.support.SystemWrites;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.instances.QInstanceValidator;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.session.QSession;
|
||||
import com.kingsrook.qqq.backend.module.rdbms.jdbc.ConnectionManager;
|
||||
import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData;
|
||||
|
||||
/*******************************************************************************
|
||||
** H2Support.
|
||||
*******************************************************************************/
|
||||
public final class H2Support
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Backend.
|
||||
*******************************************************************************/
|
||||
public static RDBMSBackendMetaData backend()
|
||||
{
|
||||
var backend =
|
||||
new RDBMSBackendMetaData()
|
||||
.withName("rdbms")
|
||||
.withVendor("h2")
|
||||
.withHostName("mem")
|
||||
.withDatabaseName("email_" + UUID.randomUUID())
|
||||
.withUsername("sa");
|
||||
backend.setJdbcUrl(
|
||||
ConnectionManager.getJdbcUrl(backend)
|
||||
+ ";DATABASE_TO_UPPER=FALSE;DB_CLOSE_DELAY=-1;LOCK_TIMEOUT=10000");
|
||||
return (backend);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Prime.
|
||||
*******************************************************************************/
|
||||
public static void prime(RDBMSBackendMetaData backend) throws Exception
|
||||
{
|
||||
try(Connection connection = new ConnectionManager().getConnection(backend))
|
||||
{
|
||||
for(String sql : new String(H2Support.class.getResourceAsStream("/email-h2.sql").readAllBytes(), java.nio.charset.StandardCharsets.UTF_8).split(";"))
|
||||
{
|
||||
if(!sql.isBlank())
|
||||
{
|
||||
connection.createStatement().execute(sql);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Instance.
|
||||
*******************************************************************************/
|
||||
public static QInstance instance(RDBMSBackendMetaData backend, Consumer<EmailQBitConfig> tweak)
|
||||
throws Exception
|
||||
{
|
||||
QInstance instance = new QInstance();
|
||||
instance.withInstanceDefaultAuthentication(
|
||||
new QAuthenticationMetaData()
|
||||
.withName("anonymous")
|
||||
.withType(QAuthenticationType.FULLY_ANONYMOUS));
|
||||
instance.addBackend(backend);
|
||||
EmailQBitConfig config = new EmailQBitConfig().withBackendName("rdbms");
|
||||
tweak.accept(config);
|
||||
new EmailQBitProducer().withQBitConfig(config).produce(instance);
|
||||
new QInstanceValidator().validate(instance);
|
||||
return (instance);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Setup.
|
||||
*******************************************************************************/
|
||||
public static QInstance setup(Consumer<EmailQBitConfig> tweak) throws Exception
|
||||
{
|
||||
ConnectionManager.resetConnectionProviders();
|
||||
var backend = backend();
|
||||
prime(backend);
|
||||
var instance = instance(backend, tweak);
|
||||
QContext.init(instance, admin());
|
||||
return (instance);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Admin.
|
||||
*******************************************************************************/
|
||||
public static QSession admin()
|
||||
{
|
||||
return (new QSession().withPermissions(EmailPermissions.ALL));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Provider.
|
||||
*******************************************************************************/
|
||||
public static QRecord provider(int port) throws Exception
|
||||
{
|
||||
return (insert(
|
||||
EmailTables.PROVIDER,
|
||||
new QRecord()
|
||||
.withValue("name", "relay")
|
||||
.withValue("label", "Relay")
|
||||
.withValue("kind", "SMTP")
|
||||
.withValue("host", "127.0.0.1")
|
||||
.withValue("port", port)
|
||||
.withValue("security", "NONE")
|
||||
.withValue("fromAddress", "sender@example.com")
|
||||
.withValue("enabled", true)
|
||||
.withValue("isDefault", true)));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Insert.
|
||||
*******************************************************************************/
|
||||
public static QRecord insert(String table, QRecord record) throws Exception
|
||||
{
|
||||
QRecord result =
|
||||
new InsertAction()
|
||||
.execute(new InsertInput(table).withRecords(List.of(record)))
|
||||
.getRecords()
|
||||
.get(0);
|
||||
if(result.getErrors() != null && !result.getErrors().isEmpty())
|
||||
{
|
||||
throw new IllegalStateException("Insert failed: " + result.getErrors());
|
||||
}
|
||||
return (result);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** All.
|
||||
*******************************************************************************/
|
||||
public static List<QRecord> all(String table) throws Exception
|
||||
{
|
||||
return (SystemWrites.query(table, new QQueryFilter()));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,419 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email;
|
||||
|
||||
import java.io.Serializable;
|
||||
import java.math.BigDecimal;
|
||||
import java.sql.Connection;
|
||||
import java.time.Instant;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Base64;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.UUID;
|
||||
import com.kingsrook.qbits.email.app.EmailHome;
|
||||
import com.kingsrook.qbits.email.bridge.EmailTemplatedMessageInput;
|
||||
import com.kingsrook.qbits.email.delivery.EmailDispatcher;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTables;
|
||||
import com.kingsrook.qbits.email.outbox.EmailRequest;
|
||||
import com.kingsrook.qbits.email.outbox.EmailSendResult;
|
||||
import com.kingsrook.qbits.email.outbox.EmailService;
|
||||
import com.kingsrook.qbits.email.phi.EmailBodyEncryption;
|
||||
import com.kingsrook.qbits.email.phi.EmailClassifications;
|
||||
import com.kingsrook.qbits.email.phi.EmailValueTokens;
|
||||
import com.kingsrook.qbits.email.processes.EmailProcesses;
|
||||
import com.kingsrook.qbits.email.support.EmailStore;
|
||||
import com.kingsrook.qbits.email.support.SystemWrites;
|
||||
import com.kingsrook.qbits.email.template.EmailVersions;
|
||||
import com.kingsrook.qqq.backend.core.actions.messaging.SendMessageAction;
|
||||
import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
|
||||
import com.kingsrook.qqq.backend.module.rdbms.jdbc.ConnectionManager;
|
||||
import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData;
|
||||
import org.apache.logging.log4j.LogManager;
|
||||
import org.apache.logging.log4j.core.LogEvent;
|
||||
import org.apache.logging.log4j.core.LoggerContext;
|
||||
import org.apache.logging.log4j.core.appender.AbstractAppender;
|
||||
import org.junit.jupiter.api.AfterEach;
|
||||
import org.junit.jupiter.api.BeforeEach;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
/*******************************************************************************
|
||||
** Canary egress across every templated entry path, retry, dead letter, export,
|
||||
** home and Log4j. Transport receives plaintext; no other inspected surface does.
|
||||
*******************************************************************************/
|
||||
class PhiCanaryEgressTest
|
||||
{
|
||||
private String canary;
|
||||
private QRecord version;
|
||||
private QTableMetaData patient;
|
||||
|
||||
/*******************************************************************************
|
||||
** Plant PHI in an application table and configure only classification overrides.
|
||||
*******************************************************************************/
|
||||
@BeforeEach
|
||||
void setup() throws Exception
|
||||
{
|
||||
canary = "CANARY-PHI-" + UUID.randomUUID();
|
||||
H2Support.setup(c -> c.withMaxAttempts(2).withSecretResolver(PhiCanaryEgressTest::secret)
|
||||
.withClassificationOverrides(Map.of("test_patient.name", EmailClassifications.Level.PHI)));
|
||||
H2Support.provider(25);
|
||||
SystemWrites.updateOne(EmailTables.PROVIDER, new QRecord().withValue("id", 1).withValue("phiAllowed", true));
|
||||
patient = new QTableMetaData().withName("test_patient").withBackendName("rdbms").withPrimaryKeyField("id")
|
||||
.withField(new QFieldMetaData("id", QFieldType.INTEGER).withBackendName("id"))
|
||||
.withField(new QFieldMetaData("name", QFieldType.STRING).withBackendName("name"));
|
||||
QContext.getQInstance().addTable(patient);
|
||||
try(Connection connection = new ConnectionManager().getConnection((RDBMSBackendMetaData) QContext.getQInstance().getBackend("rdbms")))
|
||||
{
|
||||
connection.createStatement().execute("CREATE TABLE test_patient (id INTEGER PRIMARY KEY, name VARCHAR(200))");
|
||||
try(var statement = connection.prepareStatement("INSERT INTO test_patient (id, name) VALUES (1, ?)"))
|
||||
{
|
||||
statement.setString(1, canary);
|
||||
statement.executeUpdate();
|
||||
}
|
||||
}
|
||||
version = EmailTemplatesTest.draft("phi_notice", "MARKDOWN", "[{\"name\":\"patient\",\"type\":\"RECORD\",\"required\":true}]",
|
||||
"{\"patient\":{\"name\":\"Example\"}}", "# {{patient.name}}\n\nPrivate message");
|
||||
SystemWrites.updateOne(EmailTables.VERSION, new QRecord().withValue("id", version.getValue("id")).withValue("subject", "{{patient.name}}"));
|
||||
}
|
||||
|
||||
private static boolean coreVectorKey = false;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The stand-in matches core ClassifiedValueTokens (QRun-IO/qqq#1182) on its
|
||||
** published context vector: key bytes 00..1f, key id env1.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void standInMatchesCoreVector()
|
||||
{
|
||||
coreVectorKey = true;
|
||||
try
|
||||
{
|
||||
assertThat(EmailValueTokens.forContext(List.of("email", "welcome", "recipient", "message-7"), "Ada@example.com"))
|
||||
.isEqualTo("hmac:v1:env1:p1BggBbb8ytFgS5YeM6eBiriGgC1j3x0YGjics5qVeM");
|
||||
}
|
||||
finally
|
||||
{
|
||||
coreVectorKey = false;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fixed non-production keys, resolved in memory only.
|
||||
*******************************************************************************/
|
||||
private static String secret(String name)
|
||||
{
|
||||
if(coreVectorKey && "QQQ_AUDIT_HMAC_KEY".equals(name))
|
||||
{
|
||||
byte[] key = new byte[32];
|
||||
for(int i = 0; i < key.length; i++)
|
||||
{
|
||||
key[i] = (byte) i;
|
||||
}
|
||||
return (Base64.getEncoder().encodeToString(key));
|
||||
}
|
||||
return (switch(name)
|
||||
{
|
||||
case "QQQ_EMAIL_BODY_KEY" -> "test-only-body-key";
|
||||
case "QQQ_AUDIT_HMAC_KEY" -> Base64.getEncoder().encodeToString("test-only-audit-key-32-bytes-long!".getBytes(java.nio.charset.StandardCharsets.UTF_8));
|
||||
default -> null;
|
||||
});
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Clear request-local state.
|
||||
*******************************************************************************/
|
||||
@AfterEach
|
||||
void cleanup()
|
||||
{
|
||||
QContext.clear();
|
||||
EmailRuntime.setClock(null);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** The record is fetched under the existing sender session at send time.
|
||||
*******************************************************************************/
|
||||
private EmailRequest request()
|
||||
{
|
||||
return (new EmailRequest().withTemplateName("phi_notice").withTo("person@example.com").withRecordVariable("patient", "test_patient", 1));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** No unapproved PHI request can leave any outbox or attempt row.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void unapprovedRefusedWithoutStorage() throws Exception
|
||||
{
|
||||
EmailVersions.activate(version.getValueInteger("id"));
|
||||
var refused = EmailService.send(request());
|
||||
assertThat(refused.status()).isEqualTo(EmailSendResult.Status.REFUSED);
|
||||
assertThat(refused.reasons()).contains("The variable patient.name is PHI; this template is not approved for PHI.");
|
||||
assertThat(H2Support.all(EmailTables.MESSAGE)).isEmpty();
|
||||
assertThat(H2Support.all(EmailTables.ATTEMPT)).isEmpty();
|
||||
noCanary(refused.toString());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Every path encrypts, redacts, retries and dead-letters without egress.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void canaryAcrossServiceProcessBridgeAndOperationalSurfaces() throws Exception
|
||||
{
|
||||
EmailVersions.approve(version.getValueInteger("id"));
|
||||
EmailVersions.activate(version.getValueInteger("id"));
|
||||
List<String> logs = new ArrayList<>();
|
||||
AbstractAppender capture = new AbstractAppender("phi-canary", null, null, true, null)
|
||||
{
|
||||
@Override
|
||||
public void append(LogEvent event)
|
||||
{
|
||||
logs.add(event.getMessage().getFormattedMessage());
|
||||
if(event.getThrown() != null)
|
||||
{
|
||||
logs.add(event.getThrown().toString());
|
||||
}
|
||||
}
|
||||
};
|
||||
capture.start();
|
||||
LoggerContext context = (LoggerContext) LogManager.getContext(false);
|
||||
var root = context.getConfiguration().getRootLogger();
|
||||
var priorLevel = root.getLevel();
|
||||
root.addAppender(capture, org.apache.logging.log4j.Level.ALL, null);
|
||||
root.setLevel(org.apache.logging.log4j.Level.INFO);
|
||||
context.updateLoggers();
|
||||
try
|
||||
{
|
||||
var sent = EmailService.send(request());
|
||||
assertThat(sent.status()).isEqualTo(EmailSendResult.Status.QUEUED);
|
||||
var input = new RunProcessInput();
|
||||
input.setProcessName(EmailProcesses.SEND_TEMPLATED);
|
||||
input.setFrontendStepBehavior(RunProcessInput.FrontendStepBehavior.SKIP);
|
||||
input.addValue("templateName", "phi_notice");
|
||||
input.addValue("to", "person@example.com");
|
||||
input.addValue("recordVariablesJson", "{\"patient\":{\"table\":\"test_patient\",\"id\":1}}");
|
||||
var process = new RunProcessAction().execute(input);
|
||||
assertThat(process.getException()).isEmpty();
|
||||
assertThat(process.getValues().get("status")).isEqualTo("QUEUED");
|
||||
noCanary(process.getValues().toString());
|
||||
new SendMessageAction().execute(new EmailTemplatedMessageInput().withEmailRequest(request()).withMessagingProviderName("qbitEmail"));
|
||||
assertThat(H2Support.all(EmailTables.MESSAGE)).hasSize(3);
|
||||
var dispatcher = new EmailDispatcher(new com.kingsrook.qbits.email.transport.EmailTransportRegistry().withTransport("SMTP", (envelope, settings) ->
|
||||
{
|
||||
assertThat(envelope.getSubject()).isEqualTo(canary);
|
||||
assertThat(envelope.getHtmlBody()).contains(canary);
|
||||
throw new com.kingsrook.qbits.email.transport.EmailTransportException("Provider echoed " + canary, false);
|
||||
}));
|
||||
for(QRecord message : H2Support.all(EmailTables.MESSAGE))
|
||||
{
|
||||
Integer id = message.getValueInteger("id");
|
||||
assertThat(dispatcher.sendNow(id)).isEqualTo(EmailDispatcher.SendResult.RETRY_WAIT);
|
||||
assertThat(dispatcher.sendNow(id)).isEqualTo(EmailDispatcher.SendResult.DEAD_LETTER);
|
||||
QRecord raw = EmailStore.get(EmailTables.MESSAGE, id, null);
|
||||
for(String field : List.of("subject", "htmlBody", "textBody"))
|
||||
{
|
||||
assertThat(raw.getValueString(field)).startsWith("enc:v1:").doesNotContain(canary);
|
||||
assertThat(EmailBodyEncryption.decrypt(raw.getValueString(field), field, raw.getValueString("id"))).contains(canary);
|
||||
}
|
||||
assertThat(raw.getValueString("variablesSnapshot")).contains("hmac:v1:");
|
||||
for(var value : raw.getValues().entrySet())
|
||||
{
|
||||
if(!List.of("subject", "htmlBody", "textBody").contains(value.getKey()))
|
||||
{
|
||||
noCanary(String.valueOf(value.getValue()));
|
||||
}
|
||||
}
|
||||
}
|
||||
for(String table : List.of(EmailTables.MESSAGE, EmailTables.ATTEMPT))
|
||||
{
|
||||
var exported = new QueryAction().execute(new QueryInput(table).withShouldFetchHeavyFields(true).withShouldGenerateDisplayValues(true));
|
||||
noCanary(exported.getRecords().toString());
|
||||
if(table.equals(EmailTables.MESSAGE))
|
||||
{
|
||||
assertThat(exported.getRecords()).allSatisfy(row ->
|
||||
{
|
||||
assertThat(row.getValueString("subject")).isEqualTo("[encrypted]");
|
||||
assertThat(row.getValueString("htmlBody")).isEqualTo("[encrypted]");
|
||||
assertThat(row.getValueString("textBody")).isEqualTo("[encrypted]");
|
||||
});
|
||||
}
|
||||
}
|
||||
noCanary(EmailHome.overview().getHtml());
|
||||
assertThat(EmailHome.overview().getHtml()).contains("Secure message");
|
||||
noCanary(EmailHome.hourly().getSublabel());
|
||||
assertThat(logs).isNotEmpty();
|
||||
logs.forEach(this::noCanary);
|
||||
}
|
||||
finally
|
||||
{
|
||||
root.removeAppender(capture.getName());
|
||||
root.setLevel(priorLevel);
|
||||
context.updateLoggers();
|
||||
capture.stop();
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Checking every five-character window detects every longer substring too.
|
||||
*******************************************************************************/
|
||||
private void noCanary(String output)
|
||||
{
|
||||
for(int i = 0; i <= canary.length() - 5; i++)
|
||||
{
|
||||
assertThat(output).doesNotContain(canary.substring(i, i + 5));
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Provider attestation, tracking, encryption key and category are mandatory.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void phiPolicyAndKeyRefusals() throws Exception
|
||||
{
|
||||
EmailVersions.approve(version.getValueInteger("id"));
|
||||
EmailVersions.activate(version.getValueInteger("id"));
|
||||
EmailRuntime.config().withSecretResolver(name -> null);
|
||||
assertThat(EmailService.send(request()).reasons()).contains("PHI email needs QQQ_EMAIL_BODY_KEY");
|
||||
EmailRuntime.config().withSecretResolver(PhiCanaryEgressTest::secret);
|
||||
SystemWrites.updateOne(EmailTables.PROVIDER, new QRecord().withValue("id", 1).withValue("trackOpens", true));
|
||||
assertThat(EmailService.send(request()).status()).isEqualTo(EmailSendResult.Status.REFUSED);
|
||||
SystemWrites.updateOne(EmailTables.PROVIDER, new QRecord().withValue("id", 1).withValue("trackOpens", false).withValue("phiAllowed", false));
|
||||
assertThat(EmailService.send(request()).status()).isEqualTo(EmailSendResult.Status.REFUSED);
|
||||
assertThat(EmailService.send(request().withCategory(com.kingsrook.qbits.email.outbox.EmailCategory.MARKETING)).reasons().toString()).contains("Template category cannot be overridden");
|
||||
assertThat(H2Support.all(EmailTables.MESSAGE)).isEmpty();
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Masked fields remain masked when rendered, even for an approved template.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void recordMaskingAndProviderChangesAreRespected() throws Exception
|
||||
{
|
||||
EmailVersions.approve(version.getValueInteger("id"));
|
||||
EmailVersions.activate(version.getValueInteger("id"));
|
||||
patient.getField("name").withFieldSecurityLock(new com.kingsrook.qqq.backend.core.model.metadata.security.FieldSecurityLock()
|
||||
.withSecurityKeyType("clinical").withDefaultBehavior(com.kingsrook.qqq.backend.core.model.metadata.security.FieldSecurityLock.Behavior.DENY)
|
||||
.withOverrideValues(List.of("allowed")));
|
||||
var result = EmailService.send(request());
|
||||
assertThat(result.status()).isEqualTo(EmailSendResult.Status.QUEUED);
|
||||
var row = EmailStore.get(EmailTables.MESSAGE, result.messageId(), null);
|
||||
assertThat(EmailBodyEncryption.decrypt(row.getValueString("subject"), "subject", row.getValueString("id"))).isEqualTo("************");
|
||||
SystemWrites.updateOne(EmailTables.PROVIDER, new QRecord().withValue("id", 1).withValue("trackClicks", true));
|
||||
var dispatcher = new EmailDispatcher(new com.kingsrook.qbits.email.transport.EmailTransportRegistry().withTransport("SMTP", (envelope, settings) ->
|
||||
{
|
||||
throw new AssertionError("Unsafe provider received PHI");
|
||||
}));
|
||||
assertThat(dispatcher.sendNow(result.messageId())).isEqualTo(EmailDispatcher.SendResult.RETRY_WAIT);
|
||||
assertThat(EmailService.resend(result.messageId()).status()).isEqualTo(EmailSendResult.Status.REFUSED);
|
||||
SystemWrites.updateOne(EmailTables.PROVIDER, new QRecord().withValue("id", 1).withValue("trackClicks", false));
|
||||
var resent = EmailService.resend(result.messageId());
|
||||
assertThat(resent.status()).isEqualTo(EmailSendResult.Status.QUEUED);
|
||||
QRecord copy = EmailStore.get(EmailTables.MESSAGE, resent.messageId(), null);
|
||||
assertThat(copy.getValueBoolean("phiApproved")).isTrue();
|
||||
assertThat(copy.getValueString("subject")).isNotEqualTo(row.getValueString("subject"));
|
||||
assertThat(EmailBodyEncryption.decrypt(copy.getValueString("subject"), "subject", copy.getValueString("id"))).isEqualTo("************");
|
||||
assertThatThrownBy(() -> EmailBodyEncryption.decrypt(copy.getValueString("subject"), "subject", row.getValueString("id"))).hasMessageContaining("could not be opened");
|
||||
assertThat(copy.getValue("templateVersionId")).isEqualTo(row.getValue("templateVersionId"));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Ad-hoc callers cannot turn a ciphertext string into a decryption request.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void ciphertextPrefixIsNotAuthority() throws Exception
|
||||
{
|
||||
String encrypted = EmailBodyEncryption.encrypt(canary, "subject", "test");
|
||||
var result = EmailService.send(new EmailRequest().withTo("p@example.com").withSubject(encrypted).withTextBody(encrypted));
|
||||
assertThat(result.status()).isEqualTo(EmailSendResult.Status.QUEUED);
|
||||
var dispatcher = new EmailDispatcher(new com.kingsrook.qbits.email.transport.EmailTransportRegistry().withTransport("SMTP", (envelope, settings) ->
|
||||
{
|
||||
assertThat(envelope.getSubject()).isEqualTo(encrypted);
|
||||
assertThat(envelope.getTextBody()).isEqualTo(encrypted);
|
||||
return new com.kingsrook.qbits.email.transport.EmailTransportResult("safe");
|
||||
}));
|
||||
assertThat(dispatcher.sendNow(result.messageId())).isEqualTo(EmailDispatcher.SendResult.SENT);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Token bindings, canonicalization, sentinels and authenticated encryption.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void tokensAndEncryption()
|
||||
{
|
||||
List<String> binding = List.of("email", "template", "variable", "123");
|
||||
String token = EmailValueTokens.forContext(binding, canary);
|
||||
assertThat(token).startsWith("hmac:v1:");
|
||||
assertThat(EmailValueTokens.forContext(binding, canary)).isEqualTo(token);
|
||||
assertThat(EmailValueTokens.verifyContext(binding, canary, token)).isTrue();
|
||||
assertThat(EmailValueTokens.verifyContext(List.of("different"), canary, token)).isFalse();
|
||||
assertThat(EmailValueTokens.forContext(binding, true)).isEqualTo("changed:v1");
|
||||
assertThat(EmailValueTokens.forContext(binding, "aaaaaa")).isEqualTo("changed:v1");
|
||||
assertThat(EmailValueTokens.verifyContext(binding, true, "shred:v1")).isFalse();
|
||||
assertThat(EmailValueTokens.verifyContext(binding, true, "changed:v1")).isFalse();
|
||||
String record = EmailValueTokens.forRecordField(patient, patient.getField("name"), 1, canary);
|
||||
assertThat(EmailValueTokens.verifyRecordField(patient, patient.getField("name"), 1, canary, record)).isTrue();
|
||||
assertThat(EmailValueTokens.verifyRecordField(patient, patient.getField("name"), 2, canary, record)).isFalse();
|
||||
assertThat(EmailValueTokens.canonical(new BigDecimal("1.2000"))).isEqualTo("1.2");
|
||||
assertThat(EmailValueTokens.canonical(Instant.parse("2026-10-13T12:00:00Z"))).isEqualTo("2026-10-13T12:00:00Z");
|
||||
String encrypted = EmailBodyEncryption.encrypt(canary, "subject", "test");
|
||||
assertThat(EmailBodyEncryption.encrypt(canary, "subject", "test")).isNotEqualTo(encrypted);
|
||||
assertThat(EmailBodyEncryption.decrypt(encrypted, "subject", "test")).isEqualTo(canary);
|
||||
assertThatThrownBy(() -> EmailBodyEncryption.decrypt(encrypted.substring(0, encrypted.length() - 4) + "AAAA", "subject", "test")).hasMessageContaining("could not be opened");
|
||||
EmailRuntime.config().withSecretResolver(name -> null);
|
||||
assertThat(EmailValueTokens.forContext(binding, canary)).isEqualTo("changed:v1");
|
||||
assertThatThrownBy(() -> EmailBodyEncryption.encrypt(canary, "subject", "test")).hasMessageContaining("QQQ_EMAIL_BODY_KEY");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Record projection follows parsed references and snapshots fail closed.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void referencedFieldsOnlyAndUnclassifiedTokens() throws Exception
|
||||
{
|
||||
EmailRuntime.config().withClassificationOverrides(Map.of());
|
||||
patient.withField(new QFieldMetaData("unused", QFieldType.STRING).withBackendName("unused"));
|
||||
try(Connection connection = new ConnectionManager().getConnection((RDBMSBackendMetaData) QContext.getQInstance().getBackend("rdbms")))
|
||||
{
|
||||
connection.createStatement().execute("ALTER TABLE test_patient ADD unused VARCHAR(200) DEFAULT 'never-load-this'");
|
||||
}
|
||||
SystemWrites.updateOne(EmailTables.VERSION, new QRecord().withValue("id", version.getValue("id"))
|
||||
.withValue("subject", "Notice").withValue("bodySource", "{{#if patient.name}}Hello {{patient.name}}{{/if}}"));
|
||||
EmailVersions.activate(version.getValueInteger("id"));
|
||||
var result = EmailService.send(request());
|
||||
assertThat(result.status()).isEqualTo(EmailSendResult.Status.QUEUED);
|
||||
QRecord row = EmailStore.get(EmailTables.MESSAGE, result.messageId(), null);
|
||||
assertThat(row.getValueString("htmlBody")).contains(canary);
|
||||
assertThat(row.getValueString("variablesSnapshot")).contains("name", "hmac:v1:").doesNotContain(canary, "unused", "never-load-this", "\"id\"");
|
||||
}
|
||||
|
||||
}
|
||||
@@ -0,0 +1,133 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email;
|
||||
|
||||
import java.io.Serializable;
|
||||
import java.util.Arrays;
|
||||
import java.util.List;
|
||||
import java.util.stream.Collectors;
|
||||
import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessOutput;
|
||||
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
|
||||
|
||||
/*******************************************************************************
|
||||
** Runs a streamed-ETL table or record action the way the UI does: the
|
||||
** review screen, then the result.
|
||||
*******************************************************************************/
|
||||
public final class ProcessRunner
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Review and result lines, as shown.
|
||||
*******************************************************************************/
|
||||
public record Lines(List<String> review, List<String> result)
|
||||
{
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private ProcessRunner()
|
||||
{
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Run on some record ids.
|
||||
*******************************************************************************/
|
||||
@SuppressWarnings("unchecked")
|
||||
public static Lines run(String process, Serializable... ids) throws Exception
|
||||
{
|
||||
return (runWithValues(process, java.util.Map.of(), ids));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Run With Values.
|
||||
*******************************************************************************/
|
||||
public static Lines runWithValues(
|
||||
String process, java.util.Map<String, Serializable> values, Serializable... ids)
|
||||
throws Exception
|
||||
{
|
||||
RunProcessInput input = new RunProcessInput();
|
||||
input.setProcessName(process);
|
||||
input.addValue("recordsParam", "recordIds");
|
||||
input.addValue(
|
||||
"recordIds", Arrays.stream(ids).map(String::valueOf).collect(Collectors.joining(",")));
|
||||
values.forEach(input::addValue);
|
||||
input.setFrontendStepBehavior(RunProcessInput.FrontendStepBehavior.BREAK);
|
||||
RunProcessOutput review = new RunProcessAction().execute(input);
|
||||
if(process.equals(com.kingsrook.qbits.email.processes.EmailProcesses.SEND_TEST))
|
||||
{
|
||||
RunProcessInput supplied = new RunProcessInput();
|
||||
supplied.setProcessName(process);
|
||||
supplied.setProcessUUID(review.getProcessUUID());
|
||||
supplied.setStartAfterStep("input");
|
||||
supplied.setFrontendStepBehavior(RunProcessInput.FrontendStepBehavior.BREAK);
|
||||
values.forEach(supplied::addValue);
|
||||
review = new RunProcessAction().execute(supplied);
|
||||
}
|
||||
if(review.getException().isPresent())
|
||||
{
|
||||
throw (review.getException().get());
|
||||
}
|
||||
List<ProcessSummaryLineInterface> reviewLines =
|
||||
(List<ProcessSummaryLineInterface>)
|
||||
review.getValues().get(StreamedETLWithFrontendProcess.FIELD_VALIDATION_SUMMARY);
|
||||
|
||||
RunProcessInput go = new RunProcessInput();
|
||||
go.setProcessName(process);
|
||||
go.setProcessUUID(review.getProcessUUID());
|
||||
go.setStartAfterStep(StreamedETLWithFrontendProcess.STEP_NAME_REVIEW);
|
||||
go.setFrontendStepBehavior(RunProcessInput.FrontendStepBehavior.SKIP);
|
||||
RunProcessOutput result = new RunProcessAction().execute(go);
|
||||
if(result.getException().isPresent())
|
||||
{
|
||||
throw (result.getException().get());
|
||||
}
|
||||
List<ProcessSummaryLineInterface> resultLines =
|
||||
(List<ProcessSummaryLineInterface>)
|
||||
result.getValues().get(StreamedETLWithFrontendProcess.FIELD_PROCESS_SUMMARY);
|
||||
return (new Lines(text(reviewLines, false), text(resultLines, true)));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Lines as text.
|
||||
*******************************************************************************/
|
||||
private static List<String> text(List<ProcessSummaryLineInterface> lines, boolean past)
|
||||
{
|
||||
return (lines == null
|
||||
? List.of()
|
||||
: lines.stream()
|
||||
.map(
|
||||
line ->
|
||||
{
|
||||
if(line instanceof ProcessSummaryLine summary)
|
||||
{
|
||||
summary.pickMessage(past);
|
||||
return ((summary.getCount() == null ? "" : summary.getCount() + " ")
|
||||
+ summary.getMessage());
|
||||
}
|
||||
return (String.valueOf(line));
|
||||
})
|
||||
.toList());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,293 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.demo;
|
||||
|
||||
|
||||
import java.time.Clock;
|
||||
import java.time.Duration;
|
||||
import java.time.Instant;
|
||||
import java.time.ZoneOffset;
|
||||
import java.time.temporal.ChronoUnit;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.UUID;
|
||||
import com.icegreen.greenmail.util.GreenMail;
|
||||
import com.icegreen.greenmail.util.ServerSetup;
|
||||
import com.kingsrook.qbits.email.EmailQBitConfig;
|
||||
import com.kingsrook.qbits.email.EmailQBitProducer;
|
||||
import com.kingsrook.qbits.email.EmailRuntime;
|
||||
import com.kingsrook.qbits.email.H2Support;
|
||||
import com.kingsrook.qbits.email.delivery.EmailDispatcher;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTables;
|
||||
import com.kingsrook.qbits.email.outbox.EmailRequest;
|
||||
import com.kingsrook.qbits.email.outbox.EmailSendResult;
|
||||
import com.kingsrook.qbits.email.outbox.EmailService;
|
||||
import com.kingsrook.qbits.email.permissions.EmailPermissions;
|
||||
import com.kingsrook.qbits.email.support.EmailStore;
|
||||
import com.kingsrook.qbits.email.support.SystemWrites;
|
||||
import com.kingsrook.qbits.email.template.EmailBuiltIns;
|
||||
import com.kingsrook.qbits.email.template.EmailVersions;
|
||||
import com.kingsrook.qbits.email.transport.EmailTransportRegistry;
|
||||
import com.kingsrook.qbits.email.transport.SesEmailTransport;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.instances.AbstractQQQApplication;
|
||||
import com.kingsrook.qqq.backend.core.instances.QInstanceValidator;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.session.QSession;
|
||||
import com.kingsrook.qqq.backend.core.modules.authentication.QAuthenticationModuleDispatcher;
|
||||
import com.kingsrook.qqq.backend.core.modules.authentication.implementations.FullyAnonymousAuthenticationModule;
|
||||
import com.kingsrook.qqq.backend.module.rdbms.jdbc.ConnectionManager;
|
||||
import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData;
|
||||
import com.kingsrook.qqq.middleware.javalin.QApplicationJavalinServer;
|
||||
import software.amazon.awssdk.services.sesv2.SesV2Client;
|
||||
import software.amazon.awssdk.services.sesv2.model.SendEmailResponse;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Local screenshot application: H2, GreenMail and fake SES, all in memory.
|
||||
** Home links already use /app/table/id, equivalent to webhooks' table prefix.
|
||||
** Nothing in this test package is shipped or included in coverage.
|
||||
**
|
||||
** ./mvnw -o test-compile exec:java -Dexec.classpathScope=test
|
||||
** -Dexec.mainClass=com.kingsrook.qbits.email.demo.EmailDemoServer -Dexec.args=8098
|
||||
*******************************************************************************/
|
||||
public class EmailDemoServer extends AbstractQQQApplication
|
||||
{
|
||||
private static final RDBMSBackendMetaData DATABASE = backend();
|
||||
|
||||
/*******************************************************************************
|
||||
** Anonymous sessions with all email permissions, exclusively for this demo.
|
||||
*******************************************************************************/
|
||||
public static class DemoAuthenticationModule extends FullyAnonymousAuthenticationModule
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Local demo session.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public QSession createSession(QInstance instance, Map<String, String> context)
|
||||
{
|
||||
return (super.createSession(instance, context).withPermissions(EmailPermissions.ALL));
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** H2 survives connections for this JVM only, without filesystem storage.
|
||||
*******************************************************************************/
|
||||
private static RDBMSBackendMetaData backend()
|
||||
{
|
||||
var backend = new RDBMSBackendMetaData().withName("rdbms").withVendor("h2").withHostName("mem").withDatabaseName("email_demo").withUsername("sa");
|
||||
backend.setJdbcUrl(ConnectionManager.getJdbcUrl(backend) + ";DATABASE_TO_UPPER=FALSE;DB_CLOSE_DELAY=-1");
|
||||
return (backend);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Exercise SES MIME and request construction without making AWS calls.
|
||||
*******************************************************************************/
|
||||
private static SesV2Client fakeSes()
|
||||
{
|
||||
return ((SesV2Client) java.lang.reflect.Proxy.newProxyInstance(SesV2Client.class.getClassLoader(), new Class<?>[] {SesV2Client.class}, (proxy, method, args) ->
|
||||
{
|
||||
return (switch(method.getName())
|
||||
{
|
||||
case "sendEmail" -> SendEmailResponse.builder().messageId("demo-ses-" + UUID.randomUUID()).build();
|
||||
case "serviceName" -> "sesv2";
|
||||
case "close" -> null;
|
||||
default -> throw (new UnsupportedOperationException("Unsupported demo SES call"));
|
||||
});
|
||||
}));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** UI actions also use the fake SES client, after startup seeding completes.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public QInstance defineQInstance() throws QException
|
||||
{
|
||||
QInstance instance = new QInstance();
|
||||
instance.withInstanceDefaultAuthentication(new QAuthenticationMetaData().withName("anonymous").withType(QAuthenticationType.FULLY_ANONYMOUS));
|
||||
instance.addBackend(DATABASE);
|
||||
new EmailQBitProducer().withQBitConfig(new EmailQBitConfig().withBackendName("rdbms")
|
||||
.withPublicBaseUrl("https://kof22.example").withMaxAttempts(3).withRetryMinutes(List.of(1, 5)).withSecretResolver(System::getProperty)
|
||||
.withTransportRegistryFactory(() -> new EmailTransportRegistry().withTransport("SES", new SesEmailTransport(builder -> fakeSes()))))
|
||||
.produce(instance);
|
||||
instance.setHomeAppName("email");
|
||||
return (instance);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Provider rows pass through actual save validation.
|
||||
*******************************************************************************/
|
||||
private static QRecord provider(String name, String label, String kind, int port, boolean defaultProvider) throws Exception
|
||||
{
|
||||
return (H2Support.insert(EmailTables.PROVIDER, new QRecord().withValue("name", name).withValue("label", label).withValue("kind", kind)
|
||||
.withValue("host", "127.0.0.1").withValue("port", port).withValue("security", "NONE").withValue("region", "us-east-1")
|
||||
.withValue("fromAddress", "hello@kof22.example").withValue("fromName", "Kof22").withValue("enabled", true)
|
||||
.withValue("isDefault", defaultProvider).withValue("phiAllowed", true).withValue("trackOpens", false).withValue("trackClicks", false)));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Real template/version lifecycle with sample data for the inline preview.
|
||||
*******************************************************************************/
|
||||
private static QRecord template(String name, String label, String kind, String category, Integer brand, String body, boolean active) throws Exception
|
||||
{
|
||||
QRecord template = H2Support.insert(EmailTables.TEMPLATE, new QRecord().withValue("name", name).withValue("label", label)
|
||||
.withValue("kind", kind).withValue("category", category).withValue("brandId", brand));
|
||||
QRecord version = EmailVersions.newVersion(template.getValueInteger("id"));
|
||||
SystemWrites.updateOne(EmailTables.VERSION, new QRecord().withValue("id", version.getValue("id")).withValue("subject", label + " - {{topic}}")
|
||||
.withValue("preheader", "An update from Kof22").withValue("bodySource", body)
|
||||
.withValue("variablesSchema", "[{\"name\":\"name\",\"type\":\"STRING\"},{\"name\":\"topic\",\"type\":\"STRING\"}]")
|
||||
.withValue("sampleVariables", "{\"name\":\"Alex Morgan\",\"topic\":\"Your community project\"}"));
|
||||
if(active)
|
||||
{
|
||||
EmailVersions.activate(version.getValueInteger("id"));
|
||||
}
|
||||
return (version);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Messages and attempts are created only by the real service and dispatcher.
|
||||
*******************************************************************************/
|
||||
static Map<String, Integer> seed(QInstance instance, GreenMail mail, int closedPort) throws Exception
|
||||
{
|
||||
H2Support.prime(DATABASE);
|
||||
QContext.init(instance, H2Support.admin());
|
||||
QRecord relay = provider("galaxy", "Galaxy relay (PMG)", "SMTP", mail.getSmtp().getPort(), true);
|
||||
provider("ses", "Amazon SES", "SES", 25, false);
|
||||
provider("backup", "Backup relay", "SMTP", closedPort, false);
|
||||
QRecord brand = H2Support.insert(EmailTables.BRAND, new QRecord().withValue("name", "Kof22").withValue("label", "Kof22")
|
||||
.withValue("isDefault", true).withValue("siteUrl", "https://kof22.com").withValue("postalAddress", "100 Market Street, Chester, IL 62233")
|
||||
.withValue("primaryColor", "#2459a6").withValue("textColor", "#172033").withValue("backgroundColor", "#ffffff")
|
||||
.withValue("buttonTextColor", "#ffffff").withValue("darkPrimaryColor", "#93c5fd").withValue("darkButtonTextColor", "#172033")
|
||||
.withValue("darkBackgroundColor", "#111827").withValue("darkTextColor", "#f9fafb").withValue("footerMarkdown", "**Kof22** - Ideas, projects, and community."));
|
||||
Integer brandId = brand.getValueInteger("id");
|
||||
QRecord contact = template("contact_form_received", "Contact form received", "MARKDOWN", "NOTIFICATION", brandId,
|
||||
"# Thanks for reaching out, {{name}}\n\nWe received your message about **{{topic}}**. We will follow up shortly.\n\n[Visit Kof22](https://kof22.com){.button}", true);
|
||||
QRecord v2 = EmailVersions.newVersion(contact.getValueInteger("templateId"));
|
||||
EmailVersions.activate(v2.getValueInteger("id"));
|
||||
template("alerts_notification", "Project alert", "MARKDOWN", "NOTIFICATION", brandId,
|
||||
"# A project needs your attention\n\nHello {{name}}, there is an update for **{{topic}}**.\n\n| Item | Status |\n|---|---|\n| Community project | Ready for review |\n\n[Review update](https://kof22.com){.button}", true);
|
||||
template("newsletter_october", "October at Kof22", "HTML", "MARKETING", brandId,
|
||||
"<html><body><h1>October at Kof22</h1><p>Hello {{name}},</p><p>This month: {{topic}}.</p><h2>Building something useful</h2><p>Stories from our projects and community.</p><a href='https://kof22.com'>Explore the latest</a></body></html>", false);
|
||||
EmailBuiltIns.ensure();
|
||||
QRecord secure = SystemWrites.query(EmailTables.TEMPLATE, EmailStore.equal("name", "email_secure_message")).getFirst();
|
||||
SystemWrites.updateOne(EmailTables.TEMPLATE, new QRecord().withValue("id", secure.getValue("id")).withValue("brandId", brandId));
|
||||
EmailVersions.approve(secure.getValueInteger("activeVersionId"));
|
||||
|
||||
Instant now = Instant.now().truncatedTo(ChronoUnit.MINUTES);
|
||||
String[] reasons = {"HARD_BOUNCE", "COMPLAINT", "UNSUBSCRIBE", "MANUAL"};
|
||||
for(int i = 0; i < 20; i++)
|
||||
{
|
||||
EmailRuntime.setClock(Clock.fixed(now.minus(Duration.ofDays(i * 29L / 19)), ZoneOffset.UTC));
|
||||
H2Support.insert(EmailTables.SUPPRESSION, new QRecord().withValue("address", "suppressed" + i + "@example.test")
|
||||
.withValue("scope", "ALL").withValue("reason", reasons[i % reasons.length]).withValue("note", "Demo suppression history"));
|
||||
}
|
||||
Integer messageId = null;
|
||||
EmailDispatcher dispatcher = new EmailDispatcher();
|
||||
for(int hour = 23; hour >= 0; hour--)
|
||||
{
|
||||
Instant start = now.minus(Duration.ofHours(hour)).minus(Duration.ofMinutes(25));
|
||||
EmailRuntime.setClock(Clock.fixed(start, ZoneOffset.UTC));
|
||||
for(int i = 0; i < 5; i++)
|
||||
{
|
||||
int sequence = (23 - hour) * 5 + i;
|
||||
boolean phi = sequence == 37 || sequence == 88;
|
||||
EmailRequest request = new EmailRequest().withTemplateName(phi ? "email_secure_message" : sequence % 3 == 0 ? "alerts_notification" : "contact_form_received")
|
||||
.withProviderName(sequence % 11 == 0 ? "backup" : sequence % 5 == 0 ? "ses" : "galaxy")
|
||||
.withTo(sequence % 19 == 0 ? "suppressed" + (sequence % 20) + "@example.test" : "reader" + sequence + "@example.test");
|
||||
if(phi)
|
||||
{
|
||||
request.withVariable("link", "https://kof22.com/secure/demo-" + sequence);
|
||||
}
|
||||
else
|
||||
{
|
||||
request.withVariable("name", List.of("Alex Morgan", "Jordan Lee", "Sam Taylor", "Casey Rivera", "Riley Chen").get(i))
|
||||
.withVariable("topic", List.of("Community center", "October newsletter", "Website feedback", "Volunteer program", "Project update").get(i));
|
||||
}
|
||||
EmailSendResult result = EmailService.send(request);
|
||||
if(result.status() == EmailSendResult.Status.REFUSED)
|
||||
{
|
||||
throw (new IllegalStateException("Demo send refused: " + result.reasons()));
|
||||
}
|
||||
if(messageId == null && result.status() == EmailSendResult.Status.QUEUED)
|
||||
{
|
||||
messageId = result.messageId();
|
||||
}
|
||||
SystemWrites.updateOne(EmailTables.MESSAGE, new QRecord().withValue("id", result.messageId()).withValue("createDate", start));
|
||||
}
|
||||
for(int pass = 0; pass < 3; pass++)
|
||||
{
|
||||
EmailRuntime.setClock(Clock.fixed(start.plus(Duration.ofMinutes(pass * 6L)), ZoneOffset.UTC));
|
||||
dispatcher.runOnce();
|
||||
}
|
||||
}
|
||||
EmailRuntime.setClock(null);
|
||||
System.out.println("Seeded " + H2Support.all(EmailTables.MESSAGE).size() + " messages, " + H2Support.all(EmailTables.ATTEMPT).size() + " real attempts; " + mail.getReceivedMessages().length + " captured SMTP deliveries.");
|
||||
QContext.clear();
|
||||
return (Map.of("message", messageId, "version", v2.getValueInteger("id"), "provider", relay.getValueInteger("id")));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Start SMTP and Next UI; shutdown releases in-memory mail.
|
||||
*******************************************************************************/
|
||||
public static void main(String[] args) throws Exception
|
||||
{
|
||||
int port = args.length == 0 ? 8098 : Integer.parseInt(args[0]);
|
||||
System.setProperty("QQQ_EMAIL_BODY_KEY", "local-demo-only-body-key");
|
||||
System.setProperty("QQQ_EMAIL_UNSUBSCRIBE_KEY", "local-demo-only-unsubscribe-key");
|
||||
var field = QAuthenticationModuleDispatcher.class.getDeclaredField("authenticationTypeToModuleClassNameMap");
|
||||
field.setAccessible(true);
|
||||
@SuppressWarnings("unchecked")
|
||||
Map<String, String> modules = (Map<String, String>) field.get(null);
|
||||
modules.put(QAuthenticationType.FULLY_ANONYMOUS.getName(), DemoAuthenticationModule.class.getName());
|
||||
GreenMail mail = new GreenMail(new ServerSetup(0, "127.0.0.1", "smtp"));
|
||||
mail.start();
|
||||
Runtime.getRuntime().addShutdownHook(new Thread(mail::stop));
|
||||
int closedPort;
|
||||
try(var socket = new java.net.ServerSocket(0, 1, java.net.InetAddress.getLoopbackAddress()))
|
||||
{
|
||||
closedPort = socket.getLocalPort();
|
||||
}
|
||||
try
|
||||
{
|
||||
EmailDemoServer application = new EmailDemoServer();
|
||||
QInstance instance = application.defineQInstance();
|
||||
new QInstanceValidator().validate(instance);
|
||||
Map<String, Integer> ids = seed(instance, mail, closedPort);
|
||||
new QApplicationJavalinServer(application).withPort(port).withServeFrontendNext(true).start();
|
||||
String base = "http://localhost:" + port + "/app/";
|
||||
System.out.println("Home: " + base + "email");
|
||||
System.out.println("Message: " + base + EmailTables.MESSAGE + "/" + ids.get("message"));
|
||||
System.out.println("Template preview: " + base + EmailTables.VERSION + "/" + ids.get("version"));
|
||||
System.out.println("Provider: " + base + EmailTables.PROVIDER + "/" + ids.get("provider"));
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
mail.stop();
|
||||
EmailRuntime.setClock(null);
|
||||
QContext.clear();
|
||||
throw (e);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.template;
|
||||
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Compiler isolation without spawning a child or writing temporary files.
|
||||
*******************************************************************************/
|
||||
class CliMjmlCompilerTest
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Only PATH survives; includes are disabled even for an include-free input.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void isolatedProcessBuilder()
|
||||
{
|
||||
var builder = new CliMjmlCompiler("mjml").processBuilder();
|
||||
assertThat(builder.environment().keySet()).allMatch("PATH"::equals);
|
||||
assertThat(builder.command()).containsExactly("mjml", "-i", "-s", "--config.allowIncludes=false");
|
||||
assertThat(builder.redirectError()).isEqualTo(ProcessBuilder.Redirect.DISCARD);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Refuse before attempting to launch even a nonexistent executable.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void includesRefusedBeforeLaunch()
|
||||
{
|
||||
assertThatThrownBy(() -> new CliMjmlCompiler("nonexistent").compile("<MJ-INCLUDE path='private'/>"))
|
||||
.hasMessage("MJML includes are not allowed");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,756 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.transport;
|
||||
|
||||
|
||||
import java.io.ByteArrayInputStream;
|
||||
import java.io.IOException;
|
||||
import java.net.InetSocketAddress;
|
||||
import java.net.URI;
|
||||
import java.net.http.HttpClient;
|
||||
import java.net.http.HttpHeaders;
|
||||
import java.net.http.HttpRequest;
|
||||
import java.net.http.HttpResponse;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.time.Clock;
|
||||
import java.time.Duration;
|
||||
import java.time.Instant;
|
||||
import java.time.ZoneOffset;
|
||||
import java.time.ZonedDateTime;
|
||||
import java.time.format.DateTimeFormatter;
|
||||
import java.util.Base64;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Optional;
|
||||
import java.util.Properties;
|
||||
import java.util.Set;
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
import java.util.concurrent.atomic.AtomicReference;
|
||||
import javax.net.ssl.SSLSession;
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.kingsrook.qbits.email.EmailQBitConfig;
|
||||
import com.kingsrook.qbits.email.EmailRuntime;
|
||||
import com.kingsrook.qbits.email.H2Support;
|
||||
import com.kingsrook.qbits.email.delivery.EmailDispatcher;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTableCustomizers;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTables;
|
||||
import com.kingsrook.qbits.email.outbox.EmailRequest;
|
||||
import com.kingsrook.qbits.email.outbox.EmailSendResult;
|
||||
import com.kingsrook.qbits.email.outbox.EmailService;
|
||||
import com.kingsrook.qbits.email.support.EmailStore;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.sun.net.httpserver.HttpServer;
|
||||
import jakarta.mail.Session;
|
||||
import jakarta.mail.internet.MimeMessage;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.params.ParameterizedTest;
|
||||
import org.junit.jupiter.params.provider.CsvSource;
|
||||
import org.junit.jupiter.params.provider.ValueSource;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Provider contracts, safe failures and database-to-HTTP delivery without vendors.
|
||||
*******************************************************************************/
|
||||
class EmailHttpTransportsTest
|
||||
{
|
||||
private static final String SECRET = "fake-sensitive-token";
|
||||
private static final String ECHO = "private-recipient@example.com " + SECRET;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A deterministic response without any network or third-party test dependency.
|
||||
*******************************************************************************/
|
||||
private record Response(int statusCode, HttpHeaders headers, String body) implements HttpResponse<String>
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Unused request metadata.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public HttpRequest request()
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Redirects are never followed.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public Optional<HttpResponse<String>> previousResponse()
|
||||
{
|
||||
return (Optional.empty());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** No TLS session is needed for the fake.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public Optional<SSLSession> sslSession()
|
||||
{
|
||||
return (Optional.empty());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** No response URL is consumed by a transport.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public URI uri()
|
||||
{
|
||||
return (URI.create("https://example.com"));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Test protocol version.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public HttpClient.Version version()
|
||||
{
|
||||
return (HttpClient.Version.HTTP_1_1);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Build an HTTP response with case-insensitive headers.
|
||||
*******************************************************************************/
|
||||
private static Response response(int status, String body, Map<String, List<String>> headers)
|
||||
{
|
||||
return (new Response(status, HttpHeaders.of(headers, (name, value) -> true), body));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The same envelope exercises alternatives, display names and blind recipients.
|
||||
*******************************************************************************/
|
||||
private static EmailEnvelope envelope()
|
||||
{
|
||||
return (new EmailEnvelope().withMessageId("42").withFrom("Sender <sender@example.com>")
|
||||
.withTo(List.of("Person <to@example.com>")).withCc(List.of("cc@example.com")).withBcc(List.of("bcc@example.com"))
|
||||
.withReplyTo("Replies <reply@example.com>").withSubject("Hello \u00e9").withTextBody("Plain \u00e9").withHtmlBody("<b>Hello \u00e9</b>")
|
||||
.withHeaders(Map.of("List-Unsubscribe", "<https://example.com/unsubscribe>",
|
||||
"List-Unsubscribe-Post", "List-Unsubscribe=One-Click", "X-Custom", "kept")));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Required row fields, using the new default allowlisted credential names.
|
||||
*******************************************************************************/
|
||||
private static QRecord row(String kind)
|
||||
{
|
||||
String env = switch(kind)
|
||||
{
|
||||
case "POSTMARK" -> "POSTMARK_SERVER_TOKEN";
|
||||
case "SENDGRID" -> "SENDGRID_API_KEY";
|
||||
case "MAILGUN" -> "MAILGUN_API_KEY";
|
||||
default -> "GMAIL_APP_PASSWORD";
|
||||
};
|
||||
return (new QRecord().withValue("kind", kind).withValue("secretEnvName", env)
|
||||
.withValue("fromAddress", "sender@example.com").withValue("sendingDomain", "example.com").withValue("region", "US"));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Secrets are supplied only when the transport starts its attempt.
|
||||
*******************************************************************************/
|
||||
private static EmailProviderSettings settings(QRecord row)
|
||||
{
|
||||
return (new EmailProviderSettings(row, new EmailQBitConfig().withSecretResolver(name -> SECRET)));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Select the real transport behind the fake exchange seam.
|
||||
*******************************************************************************/
|
||||
private static EmailTransport transport(String kind, EmailHttp http)
|
||||
{
|
||||
return (switch(kind)
|
||||
{
|
||||
case "POSTMARK" -> new PostmarkEmailTransport(http);
|
||||
case "SENDGRID" -> new SendGridEmailTransport(http);
|
||||
case "MAILGUN" -> new MailgunEmailTransport(http);
|
||||
default -> throw new IllegalArgumentException(kind);
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Read an in-memory request publisher just as a client would.
|
||||
*******************************************************************************/
|
||||
private static byte[] body(HttpRequest request) throws Exception
|
||||
{
|
||||
var subscriber = HttpResponse.BodySubscribers.ofByteArray();
|
||||
request.bodyPublisher().orElseThrow().subscribe(new java.util.concurrent.Flow.Subscriber<java.nio.ByteBuffer>()
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Pass demand through to the in-memory publisher.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void onSubscribe(java.util.concurrent.Flow.Subscription subscription)
|
||||
{
|
||||
subscriber.onSubscribe(subscription);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Request publishers emit buffers; response subscribers consume buffer lists.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void onNext(java.nio.ByteBuffer buffer)
|
||||
{
|
||||
subscriber.onNext(List.of(buffer));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Preserve test failures through the completion stage.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void onError(Throwable error)
|
||||
{
|
||||
subscriber.onError(error);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Complete the byte array.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void onComplete()
|
||||
{
|
||||
subscriber.onComplete();
|
||||
}
|
||||
});
|
||||
return (subscriber.getBody().toCompletableFuture().get());
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Postmark includes all rendered fields, opt-in tracking and the chosen stream.
|
||||
*******************************************************************************/
|
||||
@ParameterizedTest
|
||||
@ValueSource(booleans = {false, true})
|
||||
void postmarkRequestAndSuccess(boolean tracking) throws Exception
|
||||
{
|
||||
AtomicReference<HttpRequest> captured = new AtomicReference<>();
|
||||
var transport = new PostmarkEmailTransport(request ->
|
||||
{
|
||||
captured.set(request);
|
||||
return (response(200, "{\"ErrorCode\":0,\"MessageID\":\"pm-id\",\"Message\":\"" + ECHO + "\"}", Map.of()));
|
||||
});
|
||||
var provider = row("POSTMARK").withValue("trackOpens", tracking).withValue("trackClicks", tracking)
|
||||
.withValue("messageStream", tracking ? "broadcasts" : null);
|
||||
assertThat(transport.send(envelope(), settings(provider)).providerMessageId()).isEqualTo("pm-id");
|
||||
HttpRequest request = captured.get();
|
||||
assertThat(request.uri().toString()).isEqualTo("https://api.postmarkapp.com/email");
|
||||
assertThat(request.method()).isEqualTo("POST");
|
||||
assertThat(request.timeout()).contains(Duration.ofSeconds(30));
|
||||
assertThat(request.headers().firstValue("X-Postmark-Server-Token")).contains(SECRET);
|
||||
assertThat(request.headers().firstValue("Content-Type")).contains("application/json");
|
||||
JsonNode json = EmailHttpSupport.JSON.readTree(body(request));
|
||||
assertThat(json.path("From").asText()).isEqualTo(envelope().getFrom());
|
||||
assertThat(json.path("To").asText()).isEqualTo("Person <to@example.com>");
|
||||
assertThat(json.path("Cc").asText()).isEqualTo("cc@example.com");
|
||||
assertThat(json.path("Bcc").asText()).isEqualTo("bcc@example.com");
|
||||
assertThat(json.path("ReplyTo").asText()).isEqualTo(envelope().getReplyTo());
|
||||
assertThat(json.path("Subject").asText()).isEqualTo(envelope().getSubject());
|
||||
assertThat(json.path("TextBody").asText()).isEqualTo(envelope().getTextBody());
|
||||
assertThat(json.path("HtmlBody").asText()).isEqualTo(envelope().getHtmlBody());
|
||||
assertThat(json.path("MessageStream").asText()).isEqualTo(tracking ? "broadcasts" : "outbound");
|
||||
assertThat(json.path("TrackOpens").asBoolean()).isEqualTo(tracking);
|
||||
assertThat(json.path("TrackLinks").asText()).isEqualTo(tracking ? "HtmlAndText" : "None");
|
||||
assertThat(json.path("Headers").toString()).contains("List-Unsubscribe", "List-Unsubscribe-Post", "X-QQQ-Email-Id", "42", "Auto-Submitted");
|
||||
assertThat(json.toString()).doesNotContain(SECRET);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** SendGrid filters forbidden headers case-insensitively, retaining unsubscribe.
|
||||
*******************************************************************************/
|
||||
@ParameterizedTest
|
||||
@ValueSource(booleans = {false, true})
|
||||
void sendGridRequestAndSuccess(boolean tracking) throws Exception
|
||||
{
|
||||
AtomicReference<HttpRequest> captured = new AtomicReference<>();
|
||||
var transport = new SendGridEmailTransport(request ->
|
||||
{
|
||||
captured.set(request);
|
||||
return (response(202, "", Map.of("x-message-id", List.of("sg-id"))));
|
||||
});
|
||||
Map<String, String> headers = new java.util.HashMap<>(envelope().getHeaders());
|
||||
for(String reserved : List.of("X-SG-ID", "x-sg-eid", "Received", "DKIM-Signature", "Content-Type",
|
||||
"Content-Transfer-Encoding", "To", "From", "Subject", "Reply-To", "CC", "BcC"))
|
||||
{
|
||||
headers.put(reserved, "must-be-removed");
|
||||
}
|
||||
assertThat(transport.send(envelope().withHeaders(headers), settings(row("SENDGRID")
|
||||
.withValue("trackOpens", tracking).withValue("trackClicks", tracking))).providerMessageId()).isEqualTo("sg-id");
|
||||
HttpRequest request = captured.get();
|
||||
assertThat(request.uri().toString()).isEqualTo("https://api.sendgrid.com/v3/mail/send");
|
||||
assertThat(request.method()).isEqualTo("POST");
|
||||
assertThat(request.timeout()).contains(Duration.ofSeconds(30));
|
||||
assertThat(request.headers().firstValue("Authorization")).contains("Bearer " + SECRET);
|
||||
JsonNode json = EmailHttpSupport.JSON.readTree(body(request));
|
||||
assertThat(json.at("/personalizations/0/to/0/email").asText()).isEqualTo("to@example.com");
|
||||
assertThat(json.at("/personalizations/0/to/0/name").asText()).isEqualTo("Person");
|
||||
assertThat(json.at("/personalizations/0/cc/0/email").asText()).isEqualTo("cc@example.com");
|
||||
assertThat(json.at("/personalizations/0/bcc/0/email").asText()).isEqualTo("bcc@example.com");
|
||||
assertThat(json.at("/from/email").asText()).isEqualTo("sender@example.com");
|
||||
assertThat(json.at("/from/name").asText()).isEqualTo("Sender");
|
||||
assertThat(json.at("/reply_to/email").asText()).isEqualTo("reply@example.com");
|
||||
assertThat(json.path("subject").asText()).isEqualTo(envelope().getSubject());
|
||||
assertThat(json.at("/content/0/type").asText()).isEqualTo("text/plain");
|
||||
assertThat(json.at("/content/0/value").asText()).isEqualTo(envelope().getTextBody());
|
||||
assertThat(json.at("/content/1/type").asText()).isEqualTo("text/html");
|
||||
assertThat(json.at("/content/1/value").asText()).isEqualTo(envelope().getHtmlBody());
|
||||
assertThat(json.at("/custom_args/qqq_email_id").asText()).isEqualTo("42");
|
||||
assertThat(json.path("headers").toString()).doesNotContain("must-be-removed").contains("List-Unsubscribe", "List-Unsubscribe-Post");
|
||||
assertThat(json.at("/tracking_settings/open_tracking/enable").asBoolean()).isEqualTo(tracking);
|
||||
assertThat(json.at("/tracking_settings/click_tracking/enable").asBoolean()).isEqualTo(tracking);
|
||||
assertThat(json.at("/tracking_settings/click_tracking/enable_text").asBoolean()).isEqualTo(tracking);
|
||||
assertThat(json.toString()).doesNotContain(SECRET);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Multipart preserves the MIME bytes and explicit Bcc envelope destinations.
|
||||
*******************************************************************************/
|
||||
@ParameterizedTest
|
||||
@CsvSource({"US,false", "EU,true"})
|
||||
void mailgunRequestAndSuccess(String region, boolean tracking) throws Exception
|
||||
{
|
||||
AtomicReference<HttpRequest> captured = new AtomicReference<>();
|
||||
var transport = new MailgunEmailTransport(request ->
|
||||
{
|
||||
captured.set(request);
|
||||
return (response(200, "{\"id\":\"<mg-id@example.com>\",\"message\":\"" + ECHO + "\"}", Map.of()));
|
||||
});
|
||||
assertThat(transport.send(envelope(), settings(row("MAILGUN").withValue("region", region)
|
||||
.withValue("trackOpens", tracking).withValue("trackClicks", tracking))).providerMessageId()).isEqualTo("mg-id@example.com");
|
||||
HttpRequest request = captured.get();
|
||||
assertThat(request.uri().toString()).isEqualTo("https://api." + ("EU".equals(region) ? "eu." : "") + "mailgun.net/v3/example.com/messages.mime");
|
||||
assertThat(request.method()).isEqualTo("POST");
|
||||
assertThat(request.timeout()).contains(Duration.ofSeconds(30));
|
||||
assertThat(request.headers().firstValue("Authorization")).contains("Basic " + Base64.getEncoder().encodeToString(("api:" + SECRET).getBytes(StandardCharsets.UTF_8)));
|
||||
String contentType = request.headers().firstValue("Content-Type").orElseThrow();
|
||||
String boundary = contentType.substring(contentType.indexOf("boundary=") + 9);
|
||||
String multipart = new String(body(request), StandardCharsets.UTF_8);
|
||||
assertThat(multipart).startsWith("--" + boundary + "\r\n").endsWith("\r\n--" + boundary + "--\r\n");
|
||||
assertThat(multipart).contains("name=\"to\"\r\n\r\nto@example.com,cc@example.com,bcc@example.com\r\n");
|
||||
assertThat(multipart).contains("name=\"o:tracking-opens\"\r\n\r\n" + (tracking ? "yes" : "no") + "\r\n");
|
||||
assertThat(multipart).contains("name=\"o:tracking-clicks\"\r\n\r\n" + (tracking ? "yes" : "no") + "\r\n");
|
||||
assertThat(multipart).contains("name=\"v:qqq_email_id\"\r\n\r\n42\r\n", "name=\"message\"; filename=\"message.eml\"");
|
||||
String mime = multipart.substring(multipart.indexOf("Content-Type: message/rfc822\r\n\r\n") + "Content-Type: message/rfc822\r\n\r\n".length());
|
||||
mime = mime.substring(0, mime.lastIndexOf("\r\n--" + boundary));
|
||||
assertThat(mime).doesNotContain("bcc@example.com", "Bcc:", SECRET);
|
||||
MimeMessage message = new MimeMessage(Session.getInstance(new Properties()), new ByteArrayInputStream(mime.getBytes(StandardCharsets.UTF_8)));
|
||||
assertThat(message.getSubject()).isEqualTo(envelope().getSubject());
|
||||
assertThat(message.getHeader("X-QQQ-Email-Id", null)).isEqualTo("42");
|
||||
assertThat(message.getHeader("List-Unsubscribe-Post", null)).isEqualTo("List-Unsubscribe=One-Click");
|
||||
assertThat(message.getRecipients(jakarta.mail.Message.RecipientType.CC)[0].toString()).isEqualTo("cc@example.com");
|
||||
var alternatives = (jakarta.mail.Multipart) message.getContent();
|
||||
assertThat(alternatives.getBodyPart(0).getContent().toString()).isEqualTo(envelope().getTextBody());
|
||||
assertThat(alternatives.getBodyPart(1).getContent().toString()).isEqualTo(envelope().getHtmlBody());
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Provider codes are numeric and never include echoed diagnostics.
|
||||
*******************************************************************************/
|
||||
@ParameterizedTest
|
||||
@CsvSource({
|
||||
"10,false,true", "300,true,false", "406,true,false", "400,false,true", "401,false,true",
|
||||
"405,false,true", "412,false,true", "413,false,true", "422,false,true",
|
||||
"1235,false,true", "1236,false,true", "1480,false,true", "403,true,false"
|
||||
})
|
||||
void postmarkErrorCodes(int code, boolean permanent, boolean providerLevel)
|
||||
{
|
||||
assertFailure(new PostmarkEmailTransport(request -> response(422,
|
||||
"{\"ErrorCode\":" + code + ",\"Message\":\"" + ECHO + "\"}", Map.of())), "POSTMARK", permanent, providerLevel, null, "code " + code);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** HTTP status classification takes precedence over Postmark's JSON error code.
|
||||
*******************************************************************************/
|
||||
@ParameterizedTest
|
||||
@CsvSource({
|
||||
"POSTMARK,401,false,true", "POSTMARK,403,false,true", "POSTMARK,400,true,false", "POSTMARK,413,true,false",
|
||||
"POSTMARK,429,false,false", "POSTMARK,500,false,false", "POSTMARK,503,false,false", "POSTMARK,302,false,true",
|
||||
"SENDGRID,401,false,true", "SENDGRID,403,false,true", "SENDGRID,400,true,false", "SENDGRID,413,true,false",
|
||||
"SENDGRID,429,false,false", "SENDGRID,500,false,false", "SENDGRID,503,false,false", "SENDGRID,302,false,true",
|
||||
"MAILGUN,401,false,true", "MAILGUN,403,false,true", "MAILGUN,400,true,false", "MAILGUN,413,true,false",
|
||||
"MAILGUN,429,false,false", "MAILGUN,500,false,false", "MAILGUN,503,false,false", "MAILGUN,302,false,true"
|
||||
})
|
||||
void httpFailures(String kind, int status, boolean permanent, boolean providerLevel)
|
||||
{
|
||||
String body = status >= 429 ? "{\"ErrorCode\":10,\"Message\":\"" + ECHO + "\"}" : ECHO;
|
||||
assertFailure(transport(kind, request -> response(status, body, Map.of("Retry-After", List.of("123")))),
|
||||
kind, permanent, providerLevel, Duration.ofSeconds(123), "HTTP " + status);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Assert flags and the entire exposed exception remain free of diagnostics.
|
||||
*******************************************************************************/
|
||||
private static void assertFailure(EmailTransport transport, String kind, boolean permanent, boolean providerLevel, Duration retryAfter, String message)
|
||||
{
|
||||
assertThatThrownBy(() -> transport.send(envelope(), settings(row(kind))))
|
||||
.isInstanceOfSatisfying(EmailTransportException.class, error ->
|
||||
{
|
||||
assertThat(error.getPermanent()).isEqualTo(permanent);
|
||||
assertThat(error.getProviderLevel()).isEqualTo(providerLevel);
|
||||
assertThat(error.getRetryAfter()).isEqualTo(retryAfter);
|
||||
assertThat(error).hasMessageContaining(message).hasNoCause();
|
||||
assertThat(error.toString()).doesNotContain(SECRET, ECHO, "private-recipient");
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Support delta seconds, HTTP dates, stale dates and invalid retry headers.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void retryAfterAndHttpPolicy() throws Exception
|
||||
{
|
||||
String future = ZonedDateTime.now(ZoneOffset.UTC).plusMinutes(5).format(DateTimeFormatter.RFC_1123_DATE_TIME);
|
||||
Duration delay = EmailHttpSupport.retryAfter(response(429, "", Map.of("retry-after", List.of(future))));
|
||||
assertThat(delay.getSeconds()).isBetween(295L, 300L);
|
||||
assertThat(EmailHttpSupport.retryAfter(response(429, "", Map.of("retry-after", List.of("0"))))).isEqualTo(Duration.ZERO);
|
||||
assertThat(EmailHttpSupport.retryAfter(response(429, "", Map.of("retry-after", List.of("Sun, 06 Nov 1994 08:49:37 GMT"))))).isEqualTo(Duration.ZERO);
|
||||
for(String value : List.of("-1", "99999999999999999999999", "invalid", ""))
|
||||
{
|
||||
assertThat(EmailHttpSupport.retryAfter(response(429, "", Map.of("retry-after", List.of(value))))).isNull();
|
||||
}
|
||||
try(HttpClient client = EmailHttp.newClient())
|
||||
{
|
||||
assertThat(client.followRedirects()).isEqualTo(HttpClient.Redirect.NEVER);
|
||||
assertThat(client.connectTimeout()).contains(Duration.ofSeconds(10));
|
||||
}
|
||||
assertThatThrownBy(() -> EmailHttp.sendJdk(HttpRequest.newBuilder(URI.create("http://127.0.0.1/")).build()))
|
||||
.isInstanceOf(IOException.class).hasMessage("Email HTTP requires HTTPS");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Unknown or malformed successes must not be silently recorded as sent.
|
||||
*******************************************************************************/
|
||||
@ParameterizedTest
|
||||
@ValueSource(strings = {"POSTMARK", "SENDGRID", "MAILGUN"})
|
||||
void invalidResponsesAndNetworkFailures(String kind) throws Exception
|
||||
{
|
||||
int success = "SENDGRID".equals(kind) ? 202 : 200;
|
||||
for(String body : List.of(ECHO, "", "[]", "null", "{}", "{\"ErrorCode\":0}", "{\"id\":123}", "{\"ErrorCode\":\"" + SECRET + "\"}"))
|
||||
{
|
||||
assertFailure(transport(kind, request -> response(success, body, Map.of())), kind, false, false, null, "provider");
|
||||
}
|
||||
assertFailure(transport(kind, request ->
|
||||
{
|
||||
throw new IOException(ECHO);
|
||||
}), kind, false, false, null, "temporarily unavailable");
|
||||
assertFailure(transport(kind, request ->
|
||||
{
|
||||
throw new IllegalArgumentException(ECHO);
|
||||
}), kind, false, true, null, "configuration invalid");
|
||||
try
|
||||
{
|
||||
assertFailure(transport(kind, request ->
|
||||
{
|
||||
throw new InterruptedException(ECHO);
|
||||
}), kind, false, false, null, "interrupted");
|
||||
assertThat(Thread.currentThread().isInterrupted()).isTrue();
|
||||
}
|
||||
finally
|
||||
{
|
||||
Thread.interrupted();
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Required secrets resolve per attempt, never on construction or failed validation.
|
||||
*******************************************************************************/
|
||||
@ParameterizedTest
|
||||
@ValueSource(strings = {"POSTMARK", "SENDGRID", "MAILGUN"})
|
||||
void secretResolutionAndInvalidEnvelopes(String kind) throws Exception
|
||||
{
|
||||
EmailHttp mustNotSend = request ->
|
||||
{
|
||||
throw new AssertionError("No HTTP request should have been sent");
|
||||
};
|
||||
var transport = transport(kind, mustNotSend);
|
||||
var config = new EmailQBitConfig().withSecretResolver(name -> null);
|
||||
assertThatThrownBy(() -> transport.send(envelope(), new EmailProviderSettings(row(kind), config)))
|
||||
.hasNoCause().hasMessage("Credential environment variable is not set");
|
||||
config.withSecretResolver(name ->
|
||||
{
|
||||
throw new IllegalStateException(ECHO);
|
||||
});
|
||||
assertThatThrownBy(() -> transport.send(envelope(), new EmailProviderSettings(row(kind), config)))
|
||||
.hasNoCause().hasMessage("Credential environment variable is not set");
|
||||
assertThatThrownBy(() -> transport.send(envelope(), settings(row(kind).withValue("secretEnvName", null))))
|
||||
.hasNoCause().hasMessage("Provider needs a secret environment name");
|
||||
assertThatThrownBy(() -> transport.send(envelope(), settings(row(kind).withValue("secretEnvName", "UNAPPROVED"))))
|
||||
.hasNoCause().hasMessage("Credential environment name is not allowed");
|
||||
for(EmailEnvelope invalid : List.of(envelope().withSubject("Injected\r\nBcc: " + ECHO),
|
||||
envelope().withTo(List.of("bad\r\naddress")), envelope().withHeaders(Map.of("X-Test", "bad\n" + ECHO))))
|
||||
{
|
||||
assertThatThrownBy(() -> transport.send(invalid, settings(row(kind))))
|
||||
.hasNoCause().hasMessage("Invalid email message");
|
||||
}
|
||||
|
||||
AtomicInteger resolutions = new AtomicInteger();
|
||||
AtomicInteger sends = new AtomicInteger();
|
||||
var rotating = new EmailProviderSettings(row(kind), new EmailQBitConfig().withSecretResolver(name -> "token-" + resolutions.incrementAndGet()));
|
||||
var real = transport(kind, request ->
|
||||
{
|
||||
String token = "token-" + sends.incrementAndGet();
|
||||
String auth = "POSTMARK".equals(kind) ? request.headers().firstValue("X-Postmark-Server-Token").orElseThrow()
|
||||
: request.headers().firstValue("Authorization").orElseThrow();
|
||||
String expected = "MAILGUN".equals(kind) ? "Basic " + Base64.getEncoder().encodeToString(("api:" + token).getBytes(StandardCharsets.UTF_8))
|
||||
: "SENDGRID".equals(kind) ? "Bearer " + token : token;
|
||||
assertThat(auth).isEqualTo(expected);
|
||||
return (response("SENDGRID".equals(kind) ? 202 : 200,
|
||||
"{\"ErrorCode\":0,\"MessageID\":\"id\",\"id\":\"id\"}", Map.of("X-Message-Id", List.of("id"))));
|
||||
});
|
||||
assertThat(resolutions.get()).isZero();
|
||||
real.send(envelope(), rotating);
|
||||
real.send(envelope(), rotating);
|
||||
assertThat(resolutions.get()).isEqualTo(2);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Optional fields are omitted, null tracking is disabled and streams default.
|
||||
*******************************************************************************/
|
||||
@ParameterizedTest
|
||||
@ValueSource(strings = {"POSTMARK", "SENDGRID"})
|
||||
void minimalJsonMessages(String kind) throws Exception
|
||||
{
|
||||
AtomicReference<HttpRequest> captured = new AtomicReference<>();
|
||||
var transport = transport(kind, request ->
|
||||
{
|
||||
captured.set(request);
|
||||
return (response("SENDGRID".equals(kind) ? 202 : 200, "{\"ErrorCode\":0,\"MessageID\":\"id\"}",
|
||||
Map.of("X-Message-Id", List.of("id"))));
|
||||
});
|
||||
for(boolean htmlOnly : List.of(true, false))
|
||||
{
|
||||
var minimal = envelope().withFrom("sender@example.com").withCc(List.of()).withBcc(List.of()).withReplyTo(null)
|
||||
.withTextBody(htmlOnly ? null : "plain").withHtmlBody(htmlOnly ? "<b>html</b>" : null);
|
||||
transport.send(minimal, settings(row(kind).withValue("messageStream", "")));
|
||||
JsonNode json = EmailHttpSupport.JSON.readTree(body(captured.get()));
|
||||
if("SENDGRID".equals(kind))
|
||||
{
|
||||
assertThat(json.at("/content").size()).isEqualTo(1);
|
||||
assertThat(json.at("/personalizations/0").has("cc")).isFalse();
|
||||
assertThat(json.at("/personalizations/0").has("bcc")).isFalse();
|
||||
assertThat(json.has("reply_to")).isFalse();
|
||||
assertThat(json.at("/from").has("name")).isFalse();
|
||||
}
|
||||
else
|
||||
{
|
||||
assertThat(json.has("ReplyTo")).isFalse();
|
||||
assertThat(json.has(htmlOnly ? "TextBody" : "HtmlBody")).isFalse();
|
||||
assertThat(json.path("MessageStream").asText()).isEqualTo("outbound");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Authentication and regional requirements are validated without resolving values.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void providerValidationAndGmailPreset() throws Exception
|
||||
{
|
||||
var config = new EmailQBitConfig().withSecretResolver(name ->
|
||||
{
|
||||
throw new AssertionError("Validation must not resolve a secret");
|
||||
});
|
||||
for(String kind : List.of("POSTMARK", "SENDGRID", "MAILGUN"))
|
||||
{
|
||||
config.withAllowedEnvNames(Set.of());
|
||||
QRecord provider = row(kind);
|
||||
assertThat(EmailTableCustomizers.validateProvider(provider, config)).isEmpty();
|
||||
assertThat(EmailTableCustomizers.validateProvider(provider.withValue("secretEnvName", null), config))
|
||||
.contains("Provider needs a secret environment name");
|
||||
provider.withValue("secretEnvName", "CUSTOM_TOKEN");
|
||||
assertThat(EmailTableCustomizers.validateProvider(provider, config)).isNotEmpty();
|
||||
assertThat(EmailTableCustomizers.validateProvider(provider, config.withAllowedEnvNames(Set.of("CUSTOM_TOKEN")))).isEmpty();
|
||||
}
|
||||
for(String domain : List.of("", "https://example.com", "../example.com", "example.com/path", "example.com?x=y", "-bad.com", "bad-.com"))
|
||||
{
|
||||
assertThat(EmailTableCustomizers.validateProvider(row("MAILGUN").withValue("sendingDomain", domain), config)).isNotEmpty();
|
||||
assertThatThrownBy(() -> new MailgunEmailTransport(request ->
|
||||
{
|
||||
throw new AssertionError("Invalid domain must not send");
|
||||
}).send(envelope(), settings(row("MAILGUN").withValue("sendingDomain", domain))))
|
||||
.isInstanceOfSatisfying(EmailTransportException.class, error -> assertThat(error.getProviderLevel()).isTrue());
|
||||
}
|
||||
assertThat(EmailTableCustomizers.validateProvider(row("MAILGUN").withValue("region", "EU"), config)).isEmpty();
|
||||
assertThat(EmailTableCustomizers.validateProvider(row("MAILGUN").withValue("region", null), config)).contains("Mailgun region must be US or EU");
|
||||
assertThat(EmailTableCustomizers.validateProvider(row("MAILGUN").withValue("region", "APAC"), config)).contains("Mailgun region must be US or EU");
|
||||
Map<String, String> webhookNames = Map.of("POSTMARK", "POSTMARK_WEBHOOK_PASSWORD", "SENDGRID", "SENDGRID_WEBHOOK_PUBLIC_KEY", "MAILGUN", "MAILGUN_WEBHOOK_SIGNING_KEY");
|
||||
webhookNames.forEach((kind, env) -> assertThat(EmailTableCustomizers.validateProvider(row(kind).withValue("webhookSecretEnvName", env), config)).isEmpty());
|
||||
assertThat(config.effectiveAllowedEnvNames("SES")).doesNotContain("SES_SNS_TOPIC_ARN");
|
||||
|
||||
QRecord gmail = row("GMAIL_SMTP").withValue("usernameEnvName", "GMAIL_USERNAME");
|
||||
assertThat(EmailTableCustomizers.validateProvider(gmail, config)).isEmpty();
|
||||
var settings = new EmailProviderSettings(gmail.withValue("host", "ignored.example.com").withValue("port", 25), config);
|
||||
assertThat(settings.host()).isEqualTo("smtp.gmail.com");
|
||||
assertThat(settings.port()).isEqualTo(587);
|
||||
assertThat(settings.security()).isEqualTo("STARTTLS");
|
||||
assertThat(new EmailTransportRegistry().get("GMAIL_SMTP")).isInstanceOf(SmtpEmailTransport.class);
|
||||
assertThat(EmailTableCustomizers.validateProvider(gmail.withValue("security", "STARTTLS"), config)).isEmpty();
|
||||
for(String security : List.of("NONE", "TLS", "invalid"))
|
||||
{
|
||||
assertThat(EmailTableCustomizers.validateProvider(gmail.withValue("security", security), config.withAllowPlaintextSmtpAuth(true)))
|
||||
.contains("Gmail SMTP requires STARTTLS");
|
||||
assertThatThrownBy(() -> new SmtpEmailTransport().send(envelope(), settings))
|
||||
.isInstanceOfSatisfying(EmailTransportException.class, error -> assertThat(error.getProviderLevel()).isTrue());
|
||||
}
|
||||
gmail.withValue("security", "STARTTLS").withValue("usernameEnvName", null);
|
||||
assertThat(EmailTableCustomizers.validateProvider(gmail, config)).contains("Gmail SMTP needs an account address environment name");
|
||||
assertThatThrownBy(() -> new SmtpEmailTransport().send(envelope(), settings)).hasMessageContaining("both credential names");
|
||||
gmail.withValue("usernameEnvName", "GMAIL_USERNAME").withValue("secretEnvName", null);
|
||||
assertThat(EmailTableCustomizers.validateProvider(gmail, config)).contains("Provider needs a secret environment name");
|
||||
assertThat(EmailTableCustomizers.validateProvider(row("GMAIL_API"), config))
|
||||
.contains("Gmail API sending is specified but not built; use GMAIL_SMTP");
|
||||
assertThatThrownBy(() -> new EmailTransportRegistry().get("GMAIL_API"))
|
||||
.hasMessage("Gmail API sending is specified but not built; use GMAIL_SMTP");
|
||||
assertThatThrownBy(() -> new EmailTransportRegistry().get("future")).hasMessage("Unknown email provider kind");
|
||||
assertThat(new EmailTransportRegistry().get("POSTMARK")).isInstanceOf(PostmarkEmailTransport.class);
|
||||
assertThat(new EmailTransportRegistry().get("SENDGRID")).isInstanceOf(SendGridEmailTransport.class);
|
||||
assertThat(new EmailTransportRegistry().get("MAILGUN")).isInstanceOf(MailgunEmailTransport.class);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A real socket exchange from enqueue through dispatcher, including stored errors.
|
||||
*******************************************************************************/
|
||||
@ParameterizedTest
|
||||
@CsvSource({
|
||||
"POSTMARK,200,SENT", "POSTMARK,401,RETRY_WAIT", "POSTMARK,400,DEAD_LETTER", "POSTMARK,429,RETRY_WAIT",
|
||||
"SENDGRID,202,SENT", "SENDGRID,403,RETRY_WAIT", "SENDGRID,413,DEAD_LETTER", "SENDGRID,503,RETRY_WAIT",
|
||||
"MAILGUN,200,SENT", "MAILGUN,401,RETRY_WAIT", "MAILGUN,400,DEAD_LETTER", "MAILGUN,429,RETRY_WAIT"
|
||||
})
|
||||
void dispatchThroughLocalHttp(String kind, int status, String expectedStatus) throws Exception
|
||||
{
|
||||
Instant now = Instant.parse("2026-10-13T12:00:00Z");
|
||||
AtomicInteger resolutions = new AtomicInteger();
|
||||
AtomicReference<String> received = new AtomicReference<>();
|
||||
HttpServer server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
|
||||
server.createContext("/", exchange ->
|
||||
{
|
||||
received.set(new String(exchange.getRequestBody().readAllBytes(), StandardCharsets.UTF_8));
|
||||
exchange.getResponseHeaders().set("X-Message-Id", "http-id");
|
||||
exchange.getResponseHeaders().set("Retry-After", "123");
|
||||
byte[] reply = ("{\"ErrorCode\":" + (status == 200 ? 0 : 300)
|
||||
+ ",\"MessageID\":\"http-id\",\"id\":\"<http-id>\",\"Message\":\"" + ECHO + "\"}").getBytes(StandardCharsets.UTF_8);
|
||||
exchange.sendResponseHeaders(status, reply.length);
|
||||
exchange.getResponseBody().write(reply);
|
||||
exchange.close();
|
||||
});
|
||||
server.start();
|
||||
try(HttpClient client = EmailHttp.newClient())
|
||||
{
|
||||
EmailRuntime.setClock(Clock.fixed(now, ZoneOffset.UTC));
|
||||
H2Support.setup(config -> config.withSecretResolver(name ->
|
||||
{
|
||||
resolutions.incrementAndGet();
|
||||
return (SECRET);
|
||||
}));
|
||||
H2Support.insert(EmailTables.PROVIDER, row(kind).withValue("name", "http").withValue("label", "HTTP")
|
||||
.withValue("enabled", true).withValue("isDefault", true));
|
||||
var queued = EmailService.send(new EmailRequest().withTo("to@example.com").withCc("cc@example.com").withBcc("bcc@example.com")
|
||||
.withSubject("HTTP integration").withTextBody("body"));
|
||||
assertThat(queued.status()).isEqualTo(EmailSendResult.Status.QUEUED);
|
||||
assertThat(resolutions.get()).isZero();
|
||||
assertThat(received.get()).isNull();
|
||||
EmailHttp local = request ->
|
||||
{
|
||||
// Only the test seam rewrites the fixed production URL to a local server.
|
||||
HttpRequest.Builder localRequest = HttpRequest.newBuilder(URI.create("http://127.0.0.1:" + server.getAddress().getPort() + request.uri().getPath()))
|
||||
.timeout(request.timeout().orElseThrow()).POST(request.bodyPublisher().orElseThrow());
|
||||
request.headers().map().forEach((name, values) -> values.forEach(value -> localRequest.header(name, value)));
|
||||
return (client.send(localRequest.build(), HttpResponse.BodyHandlers.ofString()));
|
||||
};
|
||||
var dispatcher = new EmailDispatcher(new EmailTransportRegistry().withTransport(kind, transport(kind, local)));
|
||||
dispatcher.runOnce();
|
||||
assertThat(resolutions.get()).isEqualTo(1);
|
||||
assertThat(received.get()).contains("to@example.com", "bcc@example.com").doesNotContain(SECRET);
|
||||
QRecord message = EmailStore.get(EmailTables.MESSAGE, queued.messageId(), null);
|
||||
QRecord attempt = H2Support.all(EmailTables.ATTEMPT).getFirst();
|
||||
QRecord provider = H2Support.all(EmailTables.PROVIDER).getFirst();
|
||||
assertThat(message.getValueString("status")).isEqualTo(expectedStatus);
|
||||
if("SENT".equals(expectedStatus))
|
||||
{
|
||||
assertThat(message.getValueString("providerMessageId")).isEqualTo("http-id");
|
||||
assertThat(attempt.getValueString("outcome")).isEqualTo("SUCCEEDED");
|
||||
assertThat(message.getValueString("lastError")).isNull();
|
||||
}
|
||||
else
|
||||
{
|
||||
assertThat(message.getValueString("lastError")).doesNotContain(SECRET, "private-recipient", ECHO);
|
||||
assertThat(attempt.getValueString("error")).isEqualTo(message.getValueString("lastError"));
|
||||
assertThat(provider.getValueString("lastFailureMessage")).isEqualTo(message.getValueString("lastError"));
|
||||
if(status == 401 || status == 403)
|
||||
{
|
||||
assertThat(provider.getValueString("healthStatus")).isEqualTo("UNHEALTHY");
|
||||
assertThat(attempt.getValueString("outcome")).isEqualTo("PROVIDER_FAILURE");
|
||||
}
|
||||
else
|
||||
{
|
||||
assertThat(attempt.getValueString("outcome")).isEqualTo("DEAD_LETTER".equals(expectedStatus) ? "PERMANENT_FAILURE" : "TRANSIENT_FAILURE");
|
||||
}
|
||||
if("RETRY_WAIT".equals(expectedStatus))
|
||||
{
|
||||
assertThat(message.getValueInstant("nextAttemptDate")).isEqualTo(now.plusSeconds(123));
|
||||
}
|
||||
}
|
||||
}
|
||||
finally
|
||||
{
|
||||
server.stop(0);
|
||||
QContext.clear();
|
||||
EmailRuntime.setClock(null);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,411 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.email.transport;
|
||||
|
||||
import java.io.ByteArrayInputStream;
|
||||
import java.lang.reflect.Proxy;
|
||||
import java.time.Clock;
|
||||
import java.time.Instant;
|
||||
import java.time.ZoneOffset;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Properties;
|
||||
import java.util.concurrent.atomic.AtomicReference;
|
||||
import com.kingsrook.qbits.email.EmailQBitConfig;
|
||||
import com.kingsrook.qbits.email.EmailRuntime;
|
||||
import com.kingsrook.qbits.email.H2Support;
|
||||
import com.kingsrook.qbits.email.delivery.EmailDispatcher;
|
||||
import com.kingsrook.qbits.email.metadata.EmailTables;
|
||||
import com.kingsrook.qbits.email.outbox.EmailRequest;
|
||||
import com.kingsrook.qbits.email.outbox.EmailService;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.sun.mail.smtp.SMTPAddressFailedException;
|
||||
import com.sun.mail.smtp.SMTPSendFailedException;
|
||||
import jakarta.mail.MessagingException;
|
||||
import jakarta.mail.SendFailedException;
|
||||
import jakarta.mail.Session;
|
||||
import jakarta.mail.internet.InternetAddress;
|
||||
import jakarta.mail.internet.MimeMessage;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import software.amazon.awssdk.services.sesv2.SesV2Client;
|
||||
import software.amazon.awssdk.services.sesv2.model.*;
|
||||
import static org.assertj.core.api.Assertions.*;
|
||||
|
||||
/*******************************************************************************
|
||||
** Email Transports Test.
|
||||
*******************************************************************************/
|
||||
class EmailTransportsTest
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Envelope.
|
||||
*******************************************************************************/
|
||||
EmailEnvelope envelope()
|
||||
{
|
||||
return (new EmailEnvelope()
|
||||
.withMessageId("123")
|
||||
.withFrom("sender@example.com")
|
||||
.withTo(List.of("a@example.com"))
|
||||
.withBcc(List.of("hidden@example.com"))
|
||||
.withReplyTo("reply@example.com")
|
||||
.withSubject("Subject")
|
||||
.withHtmlBody("<p>HTML</p>")
|
||||
.withTextBody("Text")
|
||||
.withHeaders(Map.of("X-Test", "yes")));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Settings.
|
||||
*******************************************************************************/
|
||||
EmailProviderSettings settings()
|
||||
{
|
||||
return (new EmailProviderSettings(
|
||||
new QRecord()
|
||||
.withValue("kind", "SES")
|
||||
.withValue("region", "us-east-1")
|
||||
.withValue("configurationSet", "events")
|
||||
.withValue("usernameEnvName", "AWS_ACCESS_KEY_ID")
|
||||
.withValue("secretEnvName", "AWS_SECRET_ACCESS_KEY"),
|
||||
new EmailQBitConfig().withSecretResolver(n -> "test-only")));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Client.
|
||||
*******************************************************************************/
|
||||
SesV2Client client(java.util.function.Function<SendEmailRequest, SendEmailResponse> send)
|
||||
{
|
||||
return ((SesV2Client)
|
||||
Proxy.newProxyInstance(
|
||||
getClass().getClassLoader(),
|
||||
new Class<?>[] {SesV2Client.class},
|
||||
(proxy, method, args) ->
|
||||
{
|
||||
if(method.getName().equals("sendEmail"))
|
||||
{
|
||||
return (send.apply((SendEmailRequest) args[0]));
|
||||
}
|
||||
if(method.getName().equals("serviceName"))
|
||||
{
|
||||
return ("sesv2");
|
||||
}
|
||||
return (null);
|
||||
}));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Ses Success Preserves Mime And Does Not Expose Bcc.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void sesSuccessPreservesMimeAndDoesNotExposeBcc() throws Exception
|
||||
{
|
||||
AtomicReference<SendEmailRequest> captured = new AtomicReference<>();
|
||||
var transport =
|
||||
new SesEmailTransport(
|
||||
builder ->
|
||||
client(
|
||||
request ->
|
||||
{
|
||||
captured.set(request);
|
||||
return (SendEmailResponse.builder().messageId("ses-id").build());
|
||||
}));
|
||||
assertThat(transport.send(envelope().withTo(List.of("Pat <a@example.com>, b@example.com")), settings()).providerMessageId()).isEqualTo("ses-id");
|
||||
assertThat(captured.get().destination().toAddresses()).containsExactly("a@example.com", "b@example.com");
|
||||
assertThat(captured.get().configurationSetName()).isEqualTo("events");
|
||||
assertThat(captured.get().destination().bccAddresses()).containsExactly("hidden@example.com");
|
||||
var mime =
|
||||
new MimeMessage(
|
||||
Session.getInstance(new Properties()),
|
||||
new ByteArrayInputStream(captured.get().content().raw().data().asByteArray()));
|
||||
assertThat(mime.getHeader("Bcc")).isNull();
|
||||
assertThat(mime.getHeader("X-Test")).containsExactly("yes");
|
||||
assertThat(mime.getHeader("Auto-Submitted")).containsExactly("auto-generated");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Ses Classifies Throttling Rejection And Server Failure.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void sesClassifiesThrottlingRejectionAndServerFailure() throws Exception
|
||||
{
|
||||
for(var error :
|
||||
List.of(
|
||||
TooManyRequestsException.builder().statusCode(429).message("secret").build(),
|
||||
MessageRejectedException.builder().statusCode(400).message("secret").build(),
|
||||
SesV2Exception.builder().statusCode(503).message("secret").build()))
|
||||
{
|
||||
var transport =
|
||||
new SesEmailTransport(
|
||||
builder ->
|
||||
client(
|
||||
request ->
|
||||
{
|
||||
throw error;
|
||||
}));
|
||||
try
|
||||
{
|
||||
transport.send(envelope(), settings());
|
||||
fail("Expected failure");
|
||||
}
|
||||
catch(EmailTransportException actual)
|
||||
{
|
||||
assertThat(actual.getPermanent()).isEqualTo(error.statusCode() == 400);
|
||||
assertThat(actual.getMessage()).doesNotContain("secret");
|
||||
assertThat(actual.getCause()).isNull();
|
||||
}
|
||||
}
|
||||
var defaultChain =
|
||||
new EmailProviderSettings(
|
||||
new QRecord().withValue("kind", "SES").withValue("region", "us-east-1"),
|
||||
new EmailQBitConfig());
|
||||
assertThat(
|
||||
new SesEmailTransport(
|
||||
builder ->
|
||||
client(r -> SendEmailResponse.builder().messageId("default").build()))
|
||||
.send(envelope(), defaultChain)
|
||||
.providerMessageId())
|
||||
.isEqualTo("default");
|
||||
assertThatThrownBy(
|
||||
() ->
|
||||
new SesEmailTransport(
|
||||
b ->
|
||||
{
|
||||
throw new IllegalArgumentException("secret");
|
||||
})
|
||||
.send(envelope(), settings()))
|
||||
.hasMessage("SES temporarily unavailable");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Smtp Classifies Nested Replies.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void smtpClassifiesNestedReplies() throws Exception
|
||||
{
|
||||
var permanent =
|
||||
new SMTPAddressFailedException(
|
||||
new InternetAddress("a@example.com"), "RCPT TO", 550, "secret recipient");
|
||||
var transientError =
|
||||
new SMTPAddressFailedException(
|
||||
new InternetAddress("a@example.com"), "RCPT TO", 450, "temporary");
|
||||
var wrapper = new SendFailedException("failed");
|
||||
wrapper.setNextException(permanent);
|
||||
assertThat(SmtpEmailTransport.permanent(wrapper)).isTrue();
|
||||
assertThat(SmtpEmailTransport.permanent(transientError)).isFalse();
|
||||
assertThat(
|
||||
SmtpEmailTransport.permanent(
|
||||
new MessagingException("timeout", new java.net.SocketTimeoutException())))
|
||||
.isFalse();
|
||||
assertThat(SmtpEmailTransport.permanent(new jakarta.mail.AuthenticationFailedException("454 Temporary authentication failure"))).isFalse();
|
||||
assertThat(SmtpEmailTransport.permanent(new jakarta.mail.AuthenticationFailedException()))
|
||||
.isFalse();
|
||||
assertThat(SmtpEmailTransport.providerLevel(new jakarta.mail.AuthenticationFailedException("535 Authentication refused"))).isTrue();
|
||||
assertThat(
|
||||
SmtpEmailTransport.permanent(
|
||||
new SMTPSendFailedException("DATA", 550, "failed", null, null, null, null)))
|
||||
.isTrue();
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Account and credential errors are provider failures, not message rejections.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void sesAccountFailuresAreRetryable() throws Exception
|
||||
{
|
||||
for(String code : List.of("AccountSuspended", "SendingPaused", "MailFromDomainNotVerified", "InvalidClientTokenId", "SignatureDoesNotMatch"))
|
||||
{
|
||||
assertSesFailure(400, code, true, false);
|
||||
}
|
||||
assertSesFailure(401, "Unauthorized", true, false);
|
||||
assertSesFailure(403, "AccessDenied", true, false);
|
||||
assertSesFailure(400, "InvalidParameterValue", false, true);
|
||||
assertSesFailure(400, "MessageRejected", false, true);
|
||||
assertSesFailure(429, "TooManyRequests", false, false);
|
||||
assertSesFailure(503, "ServiceUnavailable", false, false);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** SES 403 is persisted as an unhealthy provider and recovers without a dead letter.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void sesForbiddenPausesProviderAndRecovers() throws Exception
|
||||
{
|
||||
var now = Instant.parse("2026-10-13T12:00:00Z");
|
||||
try
|
||||
{
|
||||
H2Support.setup(config -> config.withMaxAttempts(1));
|
||||
EmailRuntime.setClock(Clock.fixed(now, ZoneOffset.UTC));
|
||||
H2Support.insert(EmailTables.PROVIDER,
|
||||
new QRecord().withValue("kind", "SES").withValue("name", "ses").withValue("region", "us-east-1")
|
||||
.withValue("fromAddress", "sender@example.com").withValue("enabled", true).withValue("isDefault", true));
|
||||
var queued = EmailService.send(new EmailRequest()
|
||||
.withTo("a@example.com").withSubject("subject").withTextBody("body"));
|
||||
AtomicReference<Boolean> forbidden = new AtomicReference<>(true);
|
||||
var transport = new SesEmailTransport(builder -> client(request ->
|
||||
{
|
||||
if(forbidden.get())
|
||||
{
|
||||
throw SesV2Exception.builder().statusCode(403).message("private diagnostic").build();
|
||||
}
|
||||
return (SendEmailResponse.builder().messageId("ses-ok").build());
|
||||
}));
|
||||
var dispatcher = new EmailDispatcher(new EmailTransportRegistry().withTransport("SES", transport));
|
||||
assertThat(dispatcher.runOnce().failed()).isEqualTo(1);
|
||||
assertThat(H2Support.all(EmailTables.PROVIDER).getFirst().getValueString("healthStatus"))
|
||||
.isEqualTo("UNHEALTHY");
|
||||
assertThat(H2Support.all(EmailTables.MESSAGE).getFirst().getValueString("status"))
|
||||
.isEqualTo("RETRY_WAIT");
|
||||
forbidden.set(false);
|
||||
EmailRuntime.setClock(Clock.fixed(now.plusSeconds(301), ZoneOffset.UTC));
|
||||
assertThat(dispatcher.sendNow(queued.messageId())).isEqualTo(EmailDispatcher.SendResult.SENT);
|
||||
assertThat(H2Support.all(EmailTables.PROVIDER).getFirst().getValueString("healthStatus"))
|
||||
.isEqualTo("HEALTHY");
|
||||
}
|
||||
finally
|
||||
{
|
||||
QContext.clear();
|
||||
EmailRuntime.setClock(null);
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Exercise classification through the real SES send path with a fake client.
|
||||
*******************************************************************************/
|
||||
private void assertSesFailure(int status, String code, boolean provider, boolean permanent)
|
||||
{
|
||||
var transport = new SesEmailTransport(builder -> client(request ->
|
||||
{
|
||||
throw SesV2Exception.builder().statusCode(status).message("private diagnostic")
|
||||
.awsErrorDetails(software.amazon.awssdk.awscore.exception.AwsErrorDetails.builder().errorCode(code).build()).build();
|
||||
}));
|
||||
assertThatThrownBy(() -> transport.send(envelope(), settings())).isInstanceOfSatisfying(EmailTransportException.class, error ->
|
||||
{
|
||||
assertThat(error.getProviderLevel()).isEqualTo(provider);
|
||||
assertThat(error.getPermanent()).isEqualTo(permanent);
|
||||
assertThat(error.getMessage()).doesNotContain("private diagnostic");
|
||||
assertThat(error.getCause()).isNull();
|
||||
});
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** MIME construction refuses CR and LF in every address role and the subject.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void mimeRejectsHeaderInjectionAndBuildFailuresArePermanent()
|
||||
{
|
||||
for(String newline : List.of("\r", "\n"))
|
||||
{
|
||||
String bad = "a@example.com" + newline + "Bcc: hidden@example.com";
|
||||
for(EmailEnvelope envelope : List.of(envelope().withSubject(bad), envelope().withFrom(bad), envelope().withReplyTo(bad),
|
||||
envelope().withTo(List.of(bad)), envelope().withCc(List.of(bad)), envelope().withBcc(List.of(bad)),
|
||||
envelope().withHeaders(Map.of("X-Test", bad)), envelope().withFrom("broken <")))
|
||||
{
|
||||
assertThatThrownBy(() -> EmailMime.build(Session.getInstance(new Properties()), envelope)).isInstanceOf(MessagingException.class);
|
||||
var smtp = new EmailProviderSettings(new QRecord().withValue("kind", "SMTP").withValue("host", "127.0.0.1")
|
||||
.withValue("port", 1).withValue("security", "NONE"), new EmailQBitConfig());
|
||||
for(EmailTransport transport : List.of(new SmtpEmailTransport(), new SesEmailTransport(builder ->
|
||||
{
|
||||
throw new AssertionError("Invalid MIME must not reach the client");
|
||||
})))
|
||||
{
|
||||
assertThatThrownBy(() -> transport.send(envelope, transport instanceof SmtpEmailTransport ? smtp : settings()))
|
||||
.isInstanceOfSatisfying(EmailTransportException.class, error ->
|
||||
{
|
||||
assertThat(error.getPermanent()).isTrue();
|
||||
assertThat(error.getProviderLevel()).isFalse();
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Guard legacy rows before resolving any secret or opening a connection.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void smtpBlocksPlaintextAuthenticationBeforeConnecting()
|
||||
{
|
||||
for(String kind : List.of("SMTP", "GMAIL_SMTP"))
|
||||
{
|
||||
var row = new QRecord().withValue("kind", kind).withValue("host", "relay.example.com").withValue("port", 25)
|
||||
.withValue("security", "NONE").withValue("usernameEnvName", "EMAIL_SMTP_USERNAME");
|
||||
var config = new EmailQBitConfig().withSecretResolver(name ->
|
||||
{
|
||||
throw new AssertionError("Must reject before resolving credentials");
|
||||
});
|
||||
assertThatThrownBy(() -> new SmtpEmailTransport().send(envelope(), new EmailProviderSettings(row, config)))
|
||||
.hasMessage("SMTP sign-in needs STARTTLS or TLS").isInstanceOfSatisfying(EmailTransportException.class, error ->
|
||||
{
|
||||
assertThat(error.getProviderLevel()).isTrue();
|
||||
assertThat(error.getPermanent()).isFalse();
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Settings Validate References And Missing Secrets.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void settingsValidateReferencesAndMissingSecrets() throws Exception
|
||||
{
|
||||
var gmail =
|
||||
new EmailProviderSettings(
|
||||
new QRecord().withValue("kind", "GMAIL_SMTP"), new EmailQBitConfig());
|
||||
assertThat(gmail.host()).isEqualTo("smtp.gmail.com");
|
||||
assertThat(gmail.port()).isEqualTo(587);
|
||||
assertThat(gmail.security()).isEqualTo("STARTTLS");
|
||||
var row = new QRecord().withValue("kind", "SMTP").withValue("secretEnvName", "INVALID");
|
||||
assertThatThrownBy(
|
||||
() -> new EmailProviderSettings(row, new EmailQBitConfig()).resolve("secretEnvName"))
|
||||
.hasMessageContaining("not allowed").isInstanceOfSatisfying(EmailTransportException.class, error ->
|
||||
{
|
||||
assertThat(error.getProviderLevel()).isTrue();
|
||||
assertThat(error.getPermanent()).isFalse();
|
||||
});
|
||||
row.withValue("secretEnvName", "EMAIL_SMTP_PASSWORD");
|
||||
assertThatThrownBy(
|
||||
() ->
|
||||
new EmailProviderSettings(row, new EmailQBitConfig().withSecretResolver(n -> null))
|
||||
.resolve("secretEnvName"))
|
||||
.hasMessageContaining("not set").isInstanceOfSatisfying(EmailTransportException.class, error ->
|
||||
{
|
||||
assertThat(error.getProviderLevel()).isTrue();
|
||||
assertThat(error.getPermanent()).isFalse();
|
||||
});
|
||||
assertThatThrownBy(
|
||||
() ->
|
||||
new EmailProviderSettings(
|
||||
row,
|
||||
new EmailQBitConfig()
|
||||
.withSecretResolver(
|
||||
n ->
|
||||
{
|
||||
throw new RuntimeException("secret");
|
||||
}))
|
||||
.resolve("secretEnvName"))
|
||||
.hasMessageContaining("not set");
|
||||
assertThatThrownBy(
|
||||
() ->
|
||||
EmailMime.build(
|
||||
Session.getInstance(new Properties()),
|
||||
envelope().withHeaders(Map.of("X-Header", "bad\r\nvalue"))))
|
||||
.isInstanceOf(MessagingException.class);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,186 @@
|
||||
-- Phase (a), parts 1 and 2. Apply before producing the qbit. Host security columns must be added by the host.
|
||||
|
||||
CREATE TABLE email_provider (
|
||||
id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
|
||||
create_date TIMESTAMP,
|
||||
modify_date TIMESTAMP,
|
||||
name VARCHAR(500),
|
||||
label VARCHAR(500),
|
||||
kind VARCHAR(500),
|
||||
enabled BOOLEAN,
|
||||
is_default BOOLEAN,
|
||||
from_address VARCHAR(500),
|
||||
from_name VARCHAR(500),
|
||||
reply_to VARCHAR(500),
|
||||
sending_domain VARCHAR(500),
|
||||
host VARCHAR(500),
|
||||
port INTEGER,
|
||||
security VARCHAR(500),
|
||||
region VARCHAR(500),
|
||||
username_env_name VARCHAR(500),
|
||||
secret_env_name VARCHAR(500),
|
||||
webhook_secret_env_name VARCHAR(500),
|
||||
sns_topic_arn VARCHAR(500),
|
||||
configuration_set VARCHAR(500),
|
||||
message_stream VARCHAR(500),
|
||||
max_per_minute INTEGER,
|
||||
max_per_day INTEGER,
|
||||
track_opens BOOLEAN,
|
||||
track_clicks BOOLEAN,
|
||||
phi_allowed BOOLEAN,
|
||||
health_status VARCHAR(500),
|
||||
last_success_date TIMESTAMP,
|
||||
last_failure_date TIMESTAMP,
|
||||
last_failure_message VARCHAR(500),
|
||||
dispatch_lock INTEGER,
|
||||
default_slot INTEGER,
|
||||
UNIQUE (name),
|
||||
UNIQUE (default_slot)
|
||||
);
|
||||
|
||||
CREATE TABLE email_message (
|
||||
id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
|
||||
create_date TIMESTAMP,
|
||||
modify_date TIMESTAMP,
|
||||
status VARCHAR(500),
|
||||
to_addresses TEXT,
|
||||
cc_addresses TEXT,
|
||||
bcc_addresses TEXT,
|
||||
from_address VARCHAR(500),
|
||||
reply_to VARCHAR(500),
|
||||
subject TEXT,
|
||||
template_id INTEGER,
|
||||
template_version_id INTEGER,
|
||||
brand_id INTEGER,
|
||||
category VARCHAR(500),
|
||||
locale VARCHAR(500),
|
||||
variables_snapshot TEXT,
|
||||
html_body TEXT,
|
||||
text_body TEXT,
|
||||
body_sha256 VARCHAR(500),
|
||||
phi_approved BOOLEAN,
|
||||
unsubscribe_url TEXT,
|
||||
idempotency_key VARCHAR(500),
|
||||
provider_id INTEGER,
|
||||
provider_message_id VARCHAR(500),
|
||||
attempt_count INTEGER,
|
||||
next_attempt_date TIMESTAMP,
|
||||
lease_expires_date TIMESTAMP,
|
||||
last_error VARCHAR(500),
|
||||
finished_date TIMESTAMP,
|
||||
sent_date TIMESTAMP,
|
||||
delivered_date TIMESTAMP,
|
||||
bounced_date TIMESTAMP,
|
||||
complained_date TIMESTAMP,
|
||||
dead_letter_date TIMESTAMP,
|
||||
linked_table_name VARCHAR(500),
|
||||
linked_record_id VARCHAR(500),
|
||||
sender_qbit VARCHAR(500),
|
||||
sender_process VARCHAR(500),
|
||||
suppressed_addresses TEXT,
|
||||
send_after TIMESTAMP,
|
||||
UNIQUE (idempotency_key)
|
||||
);
|
||||
|
||||
CREATE TABLE email_message_attempt (
|
||||
id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
|
||||
create_date TIMESTAMP,
|
||||
modify_date TIMESTAMP,
|
||||
message_id INTEGER NOT NULL,
|
||||
attempt_number INTEGER NOT NULL,
|
||||
outcome VARCHAR(500),
|
||||
provider_id INTEGER,
|
||||
claimed_by VARCHAR(500),
|
||||
start_date TIMESTAMP,
|
||||
end_date TIMESTAMP,
|
||||
lease_expires_date TIMESTAMP,
|
||||
duration_millis INTEGER,
|
||||
error VARCHAR(500),
|
||||
UNIQUE (message_id, attempt_number)
|
||||
);
|
||||
|
||||
CREATE TABLE email_template (
|
||||
id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
|
||||
create_date TIMESTAMP,
|
||||
modify_date TIMESTAMP,
|
||||
name VARCHAR(500),
|
||||
label VARCHAR(500),
|
||||
category VARCHAR(500),
|
||||
kind VARCHAR(500),
|
||||
brand_id INTEGER,
|
||||
active_version_id INTEGER,
|
||||
approved_version_id INTEGER,
|
||||
version_lock INTEGER,
|
||||
phi_approved BOOLEAN,
|
||||
phi_approved_by VARCHAR(500),
|
||||
phi_approved_date TIMESTAMP,
|
||||
edit_permission_name VARCHAR(500),
|
||||
owner_qbit VARCHAR(500),
|
||||
description TEXT,
|
||||
UNIQUE (name)
|
||||
);
|
||||
|
||||
CREATE TABLE email_template_version (
|
||||
id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
|
||||
create_date TIMESTAMP,
|
||||
modify_date TIMESTAMP,
|
||||
template_id INTEGER,
|
||||
version_number INTEGER,
|
||||
status VARCHAR(500),
|
||||
locale VARCHAR(500),
|
||||
subject TEXT,
|
||||
preheader TEXT,
|
||||
body_source TEXT,
|
||||
text_source TEXT,
|
||||
compiled_html TEXT,
|
||||
variables_schema TEXT,
|
||||
sample_variables TEXT,
|
||||
created_by VARCHAR(500),
|
||||
activated_date TIMESTAMP,
|
||||
UNIQUE (template_id, version_number, locale)
|
||||
);
|
||||
|
||||
CREATE TABLE email_brand (
|
||||
id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
|
||||
create_date TIMESTAMP,
|
||||
modify_date TIMESTAMP,
|
||||
name VARCHAR(500),
|
||||
label VARCHAR(500),
|
||||
logo_url VARCHAR(500),
|
||||
logo_dark_url VARCHAR(500),
|
||||
logo_alt VARCHAR(500),
|
||||
primary_color VARCHAR(500),
|
||||
text_color VARCHAR(500),
|
||||
background_color VARCHAR(500),
|
||||
dark_background_color VARCHAR(500),
|
||||
dark_text_color VARCHAR(500),
|
||||
dark_primary_color VARCHAR(500),
|
||||
dark_button_text_color VARCHAR(500),
|
||||
button_text_color VARCHAR(500),
|
||||
footer_markdown TEXT,
|
||||
postal_address TEXT,
|
||||
social_links TEXT,
|
||||
site_url VARCHAR(500),
|
||||
is_default BOOLEAN,
|
||||
app_name VARCHAR(500),
|
||||
UNIQUE (name)
|
||||
);
|
||||
|
||||
CREATE TABLE email_suppression (
|
||||
id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
|
||||
create_date TIMESTAMP,
|
||||
modify_date TIMESTAMP,
|
||||
address VARCHAR(500),
|
||||
scope VARCHAR(500),
|
||||
reason VARCHAR(500),
|
||||
source_message_id INTEGER,
|
||||
source_event_id INTEGER,
|
||||
note TEXT,
|
||||
created_date TIMESTAMP,
|
||||
expires_date TIMESTAMP,
|
||||
UNIQUE (address, scope)
|
||||
);
|
||||
|
||||
CREATE INDEX email_message_due ON email_message (status, next_attempt_date);
|
||||
CREATE INDEX email_attempt_provider_start ON email_message_attempt (provider_id, start_date);
|
||||
CREATE INDEX email_attempt_provider_end ON email_message_attempt (provider_id, end_date, id);
|
||||
@@ -0,0 +1,13 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<Configuration status="WARN">
|
||||
<Appenders>
|
||||
<Console name="Console" target="SYSTEM_ERR">
|
||||
<LevelMatchFilter level="OFF" onMatch="DENY" onMismatch="NEUTRAL"/>
|
||||
<PatternLayout pattern="%level %logger{1}: %message%n"/>
|
||||
</Console>
|
||||
</Appenders>
|
||||
<Loggers>
|
||||
<Logger name="com.icegreen.greenmail" level="OFF"/>
|
||||
<Root level="WARN"><AppenderRef ref="Console"/></Root>
|
||||
</Loggers>
|
||||
</Configuration>
|
||||
Reference in new issue
Block a user