diff --git a/AGENTS.md b/AGENTS.md index 26e29c6..f7798f3 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -20,14 +20,19 @@ Design: `docs/DESIGN.md`. - **Entry point:** `EmailQBitProducer implements QBitMetaDataProducer`, package `com.kingsrook.qbits.email`. - `metadata`: seven tables, enum sources, relations and provider/suppression validation. -- `outbox`: EmailService, requests/results and the replaceable ad-hoc renderer. +- `outbox`: EmailService, queue-time template rendering, protected resend and branded ad-hoc rendering. - `delivery`: durable dispatcher, attempt leases, rate limits, health and optional alerts. - `transport`: shared MIME, SMTP/SES transports, settings and registry. - `bridge`: QBIT_EMAIL adapter for core SendMessageAction. - `processes`: Review/Result record actions, backend send API and scheduled jobs. - `permissions`, `app`, `support`: custom permission checker, app/navigation and authorized database operations. - `EmailRuntime` resolves config from the current instance; `EmailIntegrations` declares ESB, API and Quick Search tables. -- Templates/branding/PHI/full home rendering are part 2; their tables exist now. +- `template`: parsed variable language/schema, locale resolution, immutable versions, version-bound PHI approval, MJML compilation, lazy secure-message built-in and inline preview. +- `render`: CommonMark extensions, table-based light/dark layout, contrast validation, text conversion and unsubscribe-link signing (handler is phase c). +- `phi`: RC.5 classification/token stand-ins and AES-GCM encryption; only dispatch decrypts bodies. +- `app/EmailHome`: viewer-scoped counts and inline cards, with standard hourly/suppression chart widgets. +- Version operations serialize on the template row; active/approved version selectors are parent-scoped. +- Preview UI API endpoint awaits RC.6; webhook/unsubscribe handlers remain phase c. ## Invariants diff --git a/CHANGELOG.md b/CHANGELOG.md index 40b8461..d9ad5e0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,12 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and ## [Unreleased] ### Added +- Phase (a), part 2: versioned Markdown/HTML/MJML templates, typed variable schemas, escaped blocks/loops/fallbacks, branded light/dark layout and contrast validation. +- New Version, Activate Version and version-bound Approve For PHI actions, edit permissions, immutable history and parent-scoped version selectors. +- Sender-scoped record variables, classification stand-in, HMAC snapshot tokens, AES-GCM bodies, encrypted query displays and protected resend. +- Lazy secure-message built-in, typed templated core bridge, process JSON variables, inline preview and viewer-scoped dashboard charts/cards. +- PHI canary egress, masking, lifecycle, parent-filter, rendering, branding and GreenMail template tests. + - Phase (b) Postmark, SendGrid and Mailgun HTTP transports with fixed HTTPS endpoints, bounded requests, tracking options, safe error classification and Retry-After support; no new dependencies or filesystem storage. - Strict Gmail SMTP credential/STARTTLS validation, provider-specific credential and webhook environment defaults, and the [provider setup guide](docs/PROVIDERS.md). - HTTP request-contract tests and local-server end-to-end outbox tests covering success, provider failures, permanent rejection, retries and secret-safe stored errors. diff --git a/CLAUDE.md b/CLAUDE.md index 116f242..382fc51 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -20,14 +20,19 @@ Design: `docs/DESIGN.md`. - **Entry point:** `EmailQBitProducer implements QBitMetaDataProducer`, package `com.kingsrook.qbits.email`. - `metadata`: seven tables, enum sources, relations and provider/suppression validation. -- `outbox`: EmailService, requests/results and the replaceable ad-hoc renderer. +- `outbox`: EmailService, queue-time template rendering, protected resend and branded ad-hoc rendering. - `delivery`: durable dispatcher, attempt leases, rate limits, health and optional alerts. - `transport`: shared MIME, SMTP/SES transports, settings and registry. - `bridge`: QBIT_EMAIL adapter for core SendMessageAction. - `processes`: Review/Result record actions, backend send API and scheduled jobs. - `permissions`, `app`, `support`: custom permission checker, app/navigation and authorized database operations. - `EmailRuntime` resolves config from the current instance; `EmailIntegrations` declares ESB, API and Quick Search tables. -- Templates/branding/PHI/full home rendering are part 2; their tables exist now. +- `template`: parsed variable language/schema, locale resolution, immutable versions, version-bound PHI approval, MJML compilation, lazy secure-message built-in and inline preview. +- `render`: CommonMark extensions, table-based light/dark layout, contrast validation, text conversion and unsubscribe-link signing (handler is phase c). +- `phi`: RC.5 classification/token stand-ins and AES-GCM encryption; only dispatch decrypts bodies. +- `app/EmailHome`: viewer-scoped counts and inline cards, with standard hourly/suppression chart widgets. +- Version operations serialize on the template row; active/approved version selectors are parent-scoped. +- Preview UI API endpoint awaits RC.6; webhook/unsubscribe handlers remain phase c. ## Invariants diff --git a/README.md b/README.md index 5d72005..d8afdf5 100644 --- a/README.md +++ b/README.md @@ -1,21 +1,20 @@ # QBit Email -Phases (a) and (b): a durable email outbox for QQQ, with SMTP, Amazon SES, Postmark, -SendGrid, Mailgun and Gmail SMTP transports, -suppression, retries, provider health, record actions, and the core messaging bridge. -Every send is queued in `email_message` before a dispatcher contacts a provider. +One email service for a QQQ system: a durable outbox (`email_message`) with retries, dead +letters, idempotency keys, rate limits and provider health; SMTP, Amazon SES, Postmark, +SendGrid, Mailgun and Gmail SMTP transports; versioned Markdown, HTML and MJML templates with +dark-mode-safe branding; classified variables and PHI protection; preview widgets; a +real-data home; and the core messaging bridge. Every send goes through the outbox. -The approved [design](docs/DESIGN.md) covers later phases too. This implementation -accepts ad-hoc Markdown, HTML and text. Template rendering, branding, PHI handling and -the full home dashboard belong to part 2. Template and variable-bearing requests are -refused in part 1; attachments are refused. Tracking webhooks and unsubscribe remain -later phases. +The [design](docs/DESIGN.md) also covers delivery tracking, unsubscribe, Preferences, the +consumer migrations and inbound mail. Preview uses the standard widget endpoint; the +dedicated preview API arrives with QQQ RC.6. Attachments are refused until +qbit-object-storage is wired in. ## Install Requires Java 21, QQQ **4.1.0-RC.5**, and a transactional RDBMS with the supplied unique -constraints. Apply [the PostgreSQL DDL](docs/ddl/email-postgres.sql) before producing -the qbit. SQL timestamps store UTC values without a timezone, matching QQQ RC.5. +constraints. Apply [the PostgreSQL DDL](docs/ddl/email-postgres.sql) before producing the qbit. SQL timestamps store UTC values without a timezone, matching QQQ RC.5. The physical table names equal their metadata names; `backendName` selects the host's QQQ database backend, as it does in qbit-webhooks. @@ -46,9 +45,9 @@ in a static variable. |---|---|---| | `email_message` | Messages | Qbit only; people use record actions | | `email_message_attempt` | Attempts on a message | Qbit only | -| `email_template` | Templates | `email.manage`; rendering arrives in part 2 | -| `email_template_version` | Versions on a template | `email.manage`; activation arrives in part 2 | -| `email_brand` | Brands | `email.manage`; rendering arrives in part 2 | +| `email_template` | Templates | `email.manage`; template edit grants also apply | +| `email_template_version` | Versions on a template | Draft edits only; lifecycle uses record actions | +| `email_brand` | Brands | `email.manage`; template edit grants also apply | | `email_suppression` | Suppressions | `email.manage`; changing/removing a complaint needs `email.admin` | | `email_provider` | Providers, with child Messages | `email.admin`, including reads and test sends | @@ -149,8 +148,8 @@ Use `withHtmlBody` or `withTextBody` instead of Markdown, with optional explicit alongside HTML. `withFrom`, `withReplyTo`, `withBcc`, `withProviderName`, `withSendAfter`, and `withLocale` are supported. `sendingDomain`, when present, must match the sender's domain. Addresses are parsed strictly; header newlines are refused. Markdown raw HTML -and unsafe URL schemes are disabled. `EmailRenderer` is the replacement seam for -part 2. Both alternatives are stored at enqueue time and reused on retries. +and unsafe URL schemes are disabled. Templates always use the validated template renderer. The ad-hoc renderer extension +remains compatible. Both alternatives are stored at enqueue time and reused on retries. Results expose `messageId()`, `status()` (`QUEUED`, `DUPLICATE`, `SUPPRESSED`, `REFUSED`) and `reasons()`. Idempotency is global within the outbox. A repeated key returns the @@ -176,6 +175,120 @@ to NOTIFICATION and MARKETING. Expired rows are ignored. Partial suppression dro only matching recipients and records them in `suppressedAddresses`; all-suppressed messages remain visible with SUPPRESSED status and never consume an attempt. +## Templates, variables and branding + +Create an `email_template`, then use **New Version** to copy the active or latest +version into a DRAFT. Edit its source, schema and synthetic sample values. **Activate +Version** validates syntax, samples, classifications, brand contrast and optional +MJML compilation, then retires the previous active version for that locale in one +transaction. ACTIVE and RETIRED versions are immutable. Template-specific +`editPermissionName` applies to CRUD and actions, in addition to the email permission; +even email administrators need that explicit grant. Trusted system writes may bypass it. + +```java +EmailService.send(new EmailRequest() + .withTemplateName("order_ready") + .withTo("pat@example.com") + .withRecordVariable("order", "order", 123) + .withVariable("count", 3) + .withLocale("en")); +``` + +A schema is a JSON list, for example: + +```json +[ + {"name":"order","label":"Order","type":"RECORD","required":true}, + {"name":"count","label":"Items","type":"NUMBER","required":true}, + {"name":"link","type":"URL","classification":"PUBLIC"} +] +``` + +Types are STRING, NUMBER, DATE, DATE_TIME, BOOLEAN, URL, LIST and RECORD. Optional +`table`/`field` bind a variable's classification to metadata. URL values must be +absolute HTTP(S) URLs. Use `{{name}}`, `{{order.number}}`, `{{name|fallback text}}`, +`{{#if count}}...{{/if}}`, and `{{#each items}}{{@index}}: {{this.name}}{{/each}}`. +`this` also supports scalar list elements. Empty/null/false/zero values are false. +Unknown variables, unsupported helpers, unclosed expressions and mismatched blocks +produce line-numbered errors. Records and lists allow dynamic child paths; declare +specific dotted entries when child types/requiredness matter. There is no raw output. +HTML values are escaped; Markdown values cannot inject Markdown syntax. Subject and +text contexts use plain values, with subject newlines refused. + +Built-ins are `brand.*` (snake_case names such as `label`, `site_url`, `primary_color`), +`message.id` (the actual outbox primary key), `now` (UTC), and `unsubscribe_url`. +The latter is empty for TRANSACTIONAL/SECURITY and when no notification opt-out key +is configured. The requested locale resolves to an ACTIVE version, falling back to +`en`. The template supplies the category unless the request overrides it. + +Brand precedence is request name, template brand, the configured app's brand, +default brand, then the built-in neutral brand. Markdown uses CommonMark tables and +autolinks; raw HTML is escaped. `[Label](https://example.com){.button}` creates a table +button. The 600px shell includes preheader, logo variants, footer, social links and +light/dark colors; marketing requires a postal address. Brand saves and activation +require 4.5:1 text and button contrast in both modes. Optional `darkPrimaryColor` and +`darkButtonTextColor` fall back to their light counterparts. Colors use `#RRGGBB`. + +HTML versions retain authored HTML, with escaped variables and an opt-out footer +before ``. For MJML configure `withMjmlCompiler(new CliMjmlCompiler(binaryPath))` +or another compiler implementation. Compilation uses stdin/stdout only, with a +20-second timeout; the stored `compiledHtml` is reused for sends. No runtime files, +source files or compiler temp files are written by this qbit. + +The record Preview widget uses synthetic `sampleVariables`, sanitized inline light +(390px maximum) and dark (600px maximum) samples plus wrapping plain text. Next's +sanitizer removes iframes and style blocks, so this preview is not an exact mail-client +emulator. Draft MJML is compiled in memory for preview when a compiler is configured; +preview never stores the result or changes the version. + +## PHI and PII + +`withRecordVariable` reads through `QueryAction` in the sender's QContext; it does not +open an authorized scope. Core password masking is preserved. RC.5 does not enforce +ordinary `FieldSecurityLock` during QueryAction, so the email record loader also +applies the field lock against the sender's security keys before copying any values. +Classification precedence is reflective core `AiDataClassifications.ofField`, config +`classificationOverrides`, schema declaration, then PUBLIC. + +PII/PHI variables require **Approve For PHI** (`email.admin`) on the exact version. +Approval records actor/date and `approvedVersionId`; it freezes an approved draft. +A new sensitive version must be approved before activation. Activating a version +without its own approval clears the template's old approval. Approval of one locale +version does not approve another locale. PHI-approved templates cannot be MARKETING, +use provider open/click tracking, or use a provider without `phiAllowed`. Provider +attestation and tracking are checked again immediately before dispatch. + +`email_secure_message` is created transactionally on its first send. It needs only +a required URL variable `link` and brand built-ins; use it to send a sign-in link +without putting patient data into email. + +PII/PHI snapshot values become `hmac:v1::`, or `changed:v1` without +a valid base64 `QQQ_AUDIT_HMAC_KEY`. Record tokens bind table, field and primary key; +context tokens bind `email`, template name, variable name and message id. Context +booleans and values with fewer than four distinct characters always use `changed:v1`. +`shred:v1` is reserved for future core wipe-rule record fields and is never produced +for context values. The classification and token helpers are explicit RC.7 stand-ins. + +PHI-approved subjects and HTML/text bodies are AES-256-GCM encrypted as +`enc:v1:`, using SHA-256 of `QQQ_EMAIL_BODY_KEY`. +Missing keys refuse the send before an outbox insert. Only dispatch decrypts; query, +record and API displays show `[encrypted]`, while home shows `Secure message`. +Resend preserves ciphertext and policy; its token snapshot stays bound to the original +message, which the new message links to. Errors and logs omit provider diagnostics +and content. Keys resolve through `secretResolver`; no secret values are stored. + +NOTIFICATION/MARKETING templates prepare footer links and headers using +`QQQ_EMAIL_UNSUBSCRIBE_KEY` and `publicBaseUrl`; marketing refuses without them. +Tokens bind the message, recipient-envelope hash and category. The phase-(c) +unsubscribe route and Preferences are still pending: signed links are prepared here, +but this qbit alone does not yet serve the unsubscribe destination. Do not deploy +opt-out mail without that receiving handler. + +The home uses viewer-scoped counts and queries, UTC hour/day charts, provider health +(admins), and 20 recent messages with masked recipients. No sample data or sampled +counts are used. Bounce/complaint totals use dedicated event timestamps, populated +by the future webhook phase or host integrations. + ## Delivery and actions Each claim inserts a unique `(message_id, attempt_number)` row and commits SENDING @@ -213,9 +326,10 @@ Providers offer Send Test Email with a recipient input and a reported outcome. `email_send_templated` is a hidden backend API process. Inputs include `templateName`, `to`, `cc`, `bcc`, `replyTo`, `subject`, `markdownBody`, `htmlBody`, `textBody`, -`providerName`, `brandName`, `category`, `variablesJson`, `linkedTableName`, -`linkedRecordId`, and `idempotencyKey`. Ad-hoc content works now; template/brand/variable -inputs return REFUSED until part 2. Output includes `messageId`, `status`, and `reasons`. +`providerName`, `brandName`, `category`, `locale`, `variablesJson`, `recordVariablesJson`, +`linkedTableName`, `linkedRecordId`, and `idempotencyKey`. `variablesJson` is an object; +record bindings use `{"patient":{"table":"patient","id":123}}` in +`recordVariablesJson`. Output includes `messageId`, `status`, and `reasons`. Hidden schedules are `email_dispatch` (15 s by default), `email_maintain` (300 s, lease recovery including disabled providers, and health probes), and `email_retention` @@ -240,6 +354,18 @@ The bridge recursively maps Party/MultiParty TO, CC, BCC, FROM and REPLY_TO role TEXT/HTML alternatives. REFUSED throws `QException`; successful calls acknowledge queueing, not delivery. The built-in EMAIL and SES types are unchanged. +For schema-aware core bridge calls, use the typed extension: + +```java +new SendMessageAction().execute(new EmailTemplatedMessageInput() + .withEmailRequest(new EmailRequest().withTemplateName("email_secure_message") + .withTo("pat@example.com").withVariable("link", "https://app.example.com/inbox")) + .withMessagingProviderName("qbitEmail")); +``` + +Ordinary core `SendMessageInput` remains ad-hoc and cannot infer classifications +from arbitrary pre-rendered text. Use the typed extension for classified variables. + ## Configuration All settings have fluent `withX` setters and getters. @@ -248,7 +374,9 @@ All settings have fluent `withX` setters and getters. |---|---| | `backendName` | Required existing backend | | `schedulerName` | Optional existing scheduler; required for automatic dispatch | -| `publicBaseUrl` | Reserved for later unsubscribe links | +| `publicBaseUrl` | Public app URL for signed unsubscribe links | +| `mjmlCompiler` | Optional `MjmlCompilerInterface`; required for MJML activation | +| `classificationOverrides` | Map of `table.field` to `EmailClassifications.Level` for RC.5 | | `defaultProviderName` | Optional named provider override; otherwise enabled default row | | `bridgeProviderName` | `qbitEmail` | | `secretResolver`, `allowedEnvNames` | `System::getenv`, empty additional allowlist | @@ -274,7 +402,8 @@ modules to these declarations. Windows can use `./mvnw.cmd -o -q verify`. No new dependencies are required. Tests use H2, GreenMail, a fake SES client and local HTTP servers, including transaction rollback, concurrent claims, concurrent idempotency, shared rate budgets, health recovery and process screens. -See [implementation notes](docs/PART1-NOTES.md) for design/API accommodations. +The tests also exercise template lifecycle, parent-scoped possible values, rendering, +PHI canary egress, encryption, preview widgets and home data. ## License diff --git a/docs/DESIGN.md b/docs/DESIGN.md index d104b18..b89658e 100644 --- a/docs/DESIGN.md +++ b/docs/DESIGN.md @@ -1,6 +1,6 @@ # qbit-email design -**Status:** spec. Phases (a) to (d) build outbound; phase (e) is inbound (section 12, design only). +**Status:** phase (a) implemented; later phases remain specification. Phases (a) to (d) build outbound; phase (e) is inbound (section 12, design only). **Brief:** owner decisions of 2026-10-13 (providers, outbound first, both template kinds). **Owner decisions (2026-10-12): all recommendations accepted.** Every "Recommended" choice and every decision below stands; it is adjusted once it can be seen working. @@ -255,11 +255,40 @@ the exact version it used. version, else the default locale's. One locale (`en`) ships. ### 5.3 Preview + +**Part 2 status:** standard record widget implemented. Next's sanitizer removes +iframes and style blocks, so it uses sanitized inline light (390px maximum) and dark +(600px maximum) samples, with wrapping plain text. Draft MJML previews compile in memory +with the configured compiler; stored versions are unchanged. TODO: endpoint arrives with RC.6. + Each version record shows a **Preview** widget: the rendered email with the sample variables in a sandboxed iframe (`srcdoc`, scripts disabled), Light/Dark and 390/600 px toggles, and the plain text. It is served through the UI API (section 10): `POST /qqq/v1/qbit/email/preview` (a version id, or unsaved source, plus variables) returns subject, html, text and warnings. +### 5.4 Part 2 implementation choices +- Approval records `approved_version_id` on the template. An approved DRAFT freezes; + approve a new sensitive version before activation. Activating a different, + unapproved version clears approval. Approval is specific to one locale version. +- `EmailBuiltIns.ensure()` seeds `email_secure_message` on its first send, using the + database unique name and a transaction, with no static bootstrap cache. +- `dark_primary_color` and `dark_button_text_color` optionally override their light + counterparts; both button pairs are contrast-checked. Brand colors use #RRGGBB. +- `EmailTemplatedMessageInput` extends core SendMessageInput for schema variables and + record references. Ordinary core input remains compatible ad-hoc content. +- RC.5 QueryAction masks passwords but does not apply ordinary FieldSecurityLock; + record-variable loading additionally applies that lock to sender session keys. +- The real outbox id is bound during the enqueue transaction, before commit. Failed + validation persists no message. Resends preserve encrypted content and its original + token snapshot, linking to the original message for the token binding. +- The home follows the newer Alerts pattern: standard QQQ chart widgets plus inline, + theme-neutral cards. All queries/counts use the viewer's session. Bounce/complaint + dates have dedicated columns for phase-(c) events; no dates or metrics are invented. +- Part 2 signs opt-out links and prepares headers; the receiving unsubscribe handler + remains phase (c). Without its key/base URL, notification omits opt-out links and + headers, while marketing refuses as specified in 8.2. Do not use opt-out delivery + without a receiving handler. The token's address hash covers the recipient envelope. + ## 6. Delivery tracking ### 6.1 Inbound route @@ -438,8 +467,8 @@ and appLabel, securityFields and recordSecurityLocks (multi-tenant hosts). ## 15. Phases and acceptance | Phase | Scope | Acceptance (tests) | |---|---|---| -| (a) | tables, outbox and dispatcher, SMTP and SES, Markdown templates and versions, brands and the layout, preview, EmailService, the process, the core bridge, PHI refusal and redaction, the app home | GreenMail SMTP sink end to end; SES through a mocked `SesV2Client`; the claim race (two runners, each message sent once); retry, dead letter, idempotency; rendering (variables, escaping, `#if`/`#each`, plain text, dark-mode contrast); PHI refusal; snapshot redaction; the canary test; producer standards (names, nav, permissions) | -| (b) | Postmark, SendGrid, Mailgun, Gmail SMTP; MJML and HTML templates | HTTP mocks per provider (success, 4xx permanent, 429/5xx transient); MJML through a fake compiler; the Gmail preset | +| (a) | tables, outbox and dispatcher, SMTP and SES, Markdown/HTML/MJML templates and versions, brands and the layout, preview, EmailService, the process, the core bridge, PHI refusal and redaction, the app home | GreenMail SMTP sink end to end; SES through a mocked `SesV2Client`; the claim race (two runners, each message sent once); retry, dead letter, idempotency; rendering (variables, escaping, `#if`/`#each`, plain text, dark-mode contrast); PHI refusal; snapshot redaction; the canary test; producer standards (names, nav, permissions) | +| (b) | Postmark, SendGrid, Mailgun, Gmail SMTP | HTTP mocks per provider (success, 4xx permanent, 429/5xx transient); the Gmail preset. HTML/MJML moved into part (a), part 2 at owner request. | | (c) | webhook route and inbox, events, suppression, unsubscribe, Preferences section | signature verification per provider (valid, tampered, stale, missing secret); the SNS certificate host rule; suppression on hard bounce and complaint; one-click POST and the GET page; category rules | | (d) | consumer migrations | each consumer's own tests; a contract test of `EmailService` | | (e) | inbound | design only for now | diff --git a/docs/ddl/email-postgres.sql b/docs/ddl/email-postgres.sql index 12e7909..3bafb55 100644 --- a/docs/ddl/email-postgres.sql +++ b/docs/ddl/email-postgres.sql @@ -1,4 +1,4 @@ --- Phase (a), part 1. Apply before producing the qbit. Host security columns must be added by the host. +-- Phase (a), parts 1 and 2. Apply before producing the qbit. Host security columns must be added by the host. CREATE TABLE email_provider ( id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, @@ -58,6 +58,8 @@ CREATE TABLE email_message ( html_body TEXT, text_body TEXT, body_sha256 VARCHAR(500), + phi_approved BOOLEAN, + unsubscribe_url TEXT, idempotency_key VARCHAR(500), provider_id INTEGER, provider_message_id VARCHAR(500), @@ -68,6 +70,8 @@ CREATE TABLE email_message ( finished_date TIMESTAMP, sent_date TIMESTAMP, delivered_date TIMESTAMP, + bounced_date TIMESTAMP, + complained_date TIMESTAMP, dead_letter_date TIMESTAMP, linked_table_name VARCHAR(500), linked_record_id VARCHAR(500), @@ -105,6 +109,8 @@ CREATE TABLE email_template ( kind VARCHAR(500), brand_id INTEGER, active_version_id INTEGER, + approved_version_id INTEGER, + version_lock INTEGER, phi_approved BOOLEAN, phi_approved_by VARCHAR(500), phi_approved_date TIMESTAMP, @@ -148,6 +154,8 @@ CREATE TABLE email_brand ( background_color VARCHAR(500), dark_background_color VARCHAR(500), dark_text_color VARCHAR(500), + dark_primary_color VARCHAR(500), + dark_button_text_color VARCHAR(500), button_text_color VARCHAR(500), footer_markdown TEXT, postal_address TEXT, diff --git a/pom.xml b/pom.xml index 2f79dee..0243ef0 100644 --- a/pom.xml +++ b/pom.xml @@ -116,6 +116,12 @@ qqq-backend-module-rdbms test + + com.kingsrook.qqq + qqq-frontend-next + 1.0.0-RC.19 + test + com.icegreen greenmail-junit5 @@ -178,6 +184,11 @@ org.jacoco jacoco-maven-plugin ${jacoco.version} + + + com/kingsrook/qbits/email/demo/** + + prepare-agent diff --git a/src/main/java/com/kingsrook/qbits/email/EmailQBitConfig.java b/src/main/java/com/kingsrook/qbits/email/EmailQBitConfig.java index 0ef8c05..a967493 100644 --- a/src/main/java/com/kingsrook/qbits/email/EmailQBitConfig.java +++ b/src/main/java/com/kingsrook/qbits/email/EmailQBitConfig.java @@ -39,6 +39,10 @@ import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData; *******************************************************************************/ public class EmailQBitConfig implements QBitConfig { + private com.kingsrook.qbits.email.template.MjmlCompilerInterface mjmlCompiler; + private java.util.Map classificationOverrides = java.util.Map.of(); + private java.util.function.Supplier transportRegistryFactory = com.kingsrook.qbits.email.transport.EmailTransportRegistry::new; + private Set allowedImageHosts = Set.of(); private String backendName; private String schedulerName; private String publicBaseUrl; @@ -510,4 +514,73 @@ public class EmailQBitConfig implements QBitConfig this.tableMetaDataCustomizer = tableMetaDataCustomizer; return (this); } + + /******************************************************************************* + ** Activation-time compiler, optional unless MJML templates are used. + *******************************************************************************/ + public com.kingsrook.qbits.email.template.MjmlCompilerInterface getMjmlCompiler() + { + return (mjmlCompiler); + } + + /******************************************************************************* + ** Configure a compiler without introducing a runtime dependency. + *******************************************************************************/ + public EmailQBitConfig withMjmlCompiler(com.kingsrook.qbits.email.template.MjmlCompilerInterface compiler) + { + mjmlCompiler = compiler; + return (this); + } + + /******************************************************************************* + ** Field classifications used until core supplies classification metadata. + *******************************************************************************/ + public java.util.Map getClassificationOverrides() + { + return (classificationOverrides); + } + + /******************************************************************************* + ** Configure explicit table.field classifications. + *******************************************************************************/ + public EmailQBitConfig withClassificationOverrides(java.util.Map overrides) + { + classificationOverrides = java.util.Map.copyOf(overrides); + return (this); + } + + /******************************************************************************* + ** Additional HTTPS image hosts permitted for approved email content. + *******************************************************************************/ + public Set getAllowedImageHosts() + { + return (allowedImageHosts); + } + + /******************************************************************************* + ** Configure exact host names, without wildcards or URL paths. + *******************************************************************************/ + public EmailQBitConfig withAllowedImageHosts(Set hosts) + { + allowedImageHosts = hosts.stream().map(host -> host.toLowerCase(java.util.Locale.ROOT)).collect(java.util.stream.Collectors.toUnmodifiableSet()); + return (this); + } + + /******************************************************************************* + ** Fresh transports for every dispatcher, including process actions. + *******************************************************************************/ + public com.kingsrook.qbits.email.transport.EmailTransportRegistry newTransportRegistry() + { + return (transportRegistryFactory.get()); + } + + /******************************************************************************* + ** Host-scoped transport injection for embedded applications and local demos. + *******************************************************************************/ + public EmailQBitConfig withTransportRegistryFactory(java.util.function.Supplier factory) + { + transportRegistryFactory = java.util.Objects.requireNonNull(factory); + return (this); + } + } diff --git a/src/main/java/com/kingsrook/qbits/email/EmailQBitProducer.java b/src/main/java/com/kingsrook/qbits/email/EmailQBitProducer.java index bdd787b..79ea470 100644 --- a/src/main/java/com/kingsrook/qbits/email/EmailQBitProducer.java +++ b/src/main/java/com/kingsrook/qbits/email/EmailQBitProducer.java @@ -143,7 +143,8 @@ public class EmailQBitProducer implements QBitMetaDataProducer for(QTableMetaData table : EmailTables.tables(qBitConfig)) { Class customizer = EmailTables.PROVIDER.equals(table.getName()) ? EmailTableCustomizers.ProviderCustomizer.class - : EmailTables.SUPPRESSION.equals(table.getName()) ? EmailTableCustomizers.SuppressionCustomizer.class : null; + : EmailTables.SUPPRESSION.equals(table.getName()) ? EmailTableCustomizers.SuppressionCustomizer.class + : List.of(EmailTables.TEMPLATE, EmailTables.VERSION, EmailTables.BRAND).contains(table.getName()) ? com.kingsrook.qbits.email.template.EmailTemplateCustomizers.class : null; if(customizer != null) { table.withCustomizer(TableCustomizers.PRE_INSERT_RECORD, new QCodeReference(customizer)) @@ -154,6 +155,10 @@ public class EmailQBitProducer implements QBitMetaDataProducer { table.withCustomizer(TableCustomizers.POST_QUERY_RECORD, new QCodeReference(EmailTableCustomizers.MessageLinksCustomizer.class)); } + if(EmailTables.VERSION.equals(table.getName())) + { + table.withCustomizer(TableCustomizers.POST_QUERY_RECORD, new QCodeReference(com.kingsrook.qbits.email.template.EmailVersionLabels.class)); + } if(EmailTables.PROVIDER.equals(table.getName())) { table.withCustomizer(TableCustomizers.POST_QUERY_RECORD, new QCodeReference(EmailTableCustomizers.ProviderCustomizer.class)); diff --git a/src/main/java/com/kingsrook/qbits/email/app/EmailApp.java b/src/main/java/com/kingsrook/qbits/email/app/EmailApp.java index 2b0e03b..43c4225 100644 --- a/src/main/java/com/kingsrook/qbits/email/app/EmailApp.java +++ b/src/main/java/com/kingsrook/qbits/email/app/EmailApp.java @@ -29,7 +29,6 @@ import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRe import com.kingsrook.qqq.backend.core.exceptions.QException; import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput; import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput; -import com.kingsrook.qqq.backend.core.model.dashboard.widgets.RawHTML; import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType; import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference; import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData; @@ -39,7 +38,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData; /******************************************************************************* - ** Email app with a placeholder overview and the five main tables in navigation. + ** Email app with viewer-scoped dashboard widgets and the five main tables in navigation. ** Versions and attempts are child lists, and processes are record actions. *******************************************************************************/ public final class EmailApp @@ -59,14 +58,14 @@ public final class EmailApp /******************************************************************************* - ** Placeholder home widget. + ** Standard home and record preview widgets. *******************************************************************************/ public static List widgets() { return (List.of( - new QWidgetMetaData().withName(HOME_WIDGET).withLabel("Overview").withType(WidgetType.HTML.getType()) - .withCodeReference(new QCodeReference(HomeRenderer.class)).withGridColumns(12).withIsCard(false) - .withPermissionRules(EmailTables.permissionRules()))); + widget(HOME_WIDGET, "Overview", WidgetType.HTML.getType(), HomeRenderer.class), + widget("emailTemplatePreview", "Preview", WidgetType.HTML.getType(), com.kingsrook.qbits.email.template.EmailPreviewWidget.class))); + } @@ -105,10 +104,18 @@ public final class EmailApp @Override public RenderWidgetOutput render(RenderWidgetInput input) throws QException { - return (new RenderWidgetOutput(new RawHTML(null, "

Use Messages to follow queued email and Providers to manage delivery.

"))); + return (new RenderWidgetOutput(EmailHome.overview())); } } + /******************************************************************************* + ** Full-width widgets follow the Alerts home pattern. + *******************************************************************************/ + private static QWidgetMetaData widget(String name, String label, String type, Class renderer) + { + return (new QWidgetMetaData().withName(name).withLabel(label).withType(type).withGridColumns(12).withIsCard(false) + .withShowReloadButton(true).withCodeReference(new QCodeReference(renderer)).withPermissionRules(EmailTables.permissionRules())); + } } diff --git a/src/main/java/com/kingsrook/qbits/email/app/EmailHome.java b/src/main/java/com/kingsrook/qbits/email/app/EmailHome.java new file mode 100644 index 0000000..490bd89 --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/email/app/EmailHome.java @@ -0,0 +1,201 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.email.app; + + +import java.time.Instant; +import java.time.ZoneOffset; +import java.time.format.DateTimeFormatter; +import java.time.temporal.ChronoUnit; +import java.util.ArrayList; +import java.util.List; +import java.util.Locale; +import com.kingsrook.qbits.email.EmailRuntime; +import com.kingsrook.qbits.email.metadata.EmailTables; +import com.kingsrook.qbits.email.permissions.EmailPermissionChecker; +import com.kingsrook.qbits.email.permissions.EmailPermissions; +import com.kingsrook.qqq.backend.core.actions.tables.CountAction; +import com.kingsrook.qqq.backend.core.actions.tables.QueryAction; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.model.actions.tables.count.CountInput; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput; +import com.kingsrook.qqq.backend.core.model.dashboard.widgets.ChartData; +import com.kingsrook.qqq.backend.core.model.dashboard.widgets.RawHTML; +import com.kingsrook.qqq.backend.core.model.data.QRecord; + + +/******************************************************************************* + ** Real viewer-scoped data. Counts preserve record security without sampling; + ** inline bar charts and rows use the same theme-neutral styles as Webhooks. + *******************************************************************************/ +public final class EmailHome +{ + /******************************************************************************* + ** Utility class. + *******************************************************************************/ + private EmailHome() + { + } + + /******************************************************************************* + ** Counts and recent messages, without loading bodies or variable snapshots. + *******************************************************************************/ + public static RawHTML overview() throws QException + { + Instant now = EmailRuntime.clock().instant(); + List totals = new ArrayList<>(); + String[] fields = {"sentDate", "deliveredDate", "deadLetterDate", "bouncedDate", "complainedDate"}; + for(int i = 0; i < fields.length; i++) + { + totals.add(count(EmailTables.MESSAGE, since(fields[i], now.minus(i < 3 ? 1 : 7, ChronoUnit.DAYS)) + .withCriteria(fields[i], QCriteriaOperator.LESS_THAN_OR_EQUALS, now))); + } + totals.add(count(EmailTables.SUPPRESSION, new QQueryFilter())); + List messages = query(EmailTables.MESSAGE, new QQueryFilter().withOrderBy(new QFilterOrderBy("createDate", false)) + .withOrderBy(new QFilterOrderBy("id", false)).withLimit(20), List.of("id", "createDate", "toAddresses", "subject", "phiApproved", "status")); + boolean admin = EmailPermissionChecker.has(EmailPermissions.ADMIN); + List providers = admin ? query(EmailTables.PROVIDER, new QQueryFilter().withOrderBy(new QFilterOrderBy("id", true)), + List.of("id", "label", "name", "enabled", "healthStatus", "lastSuccessDate", "lastFailureMessage", "maxPerMinute")) : List.of(); + for(QRecord provider : providers) + { + Integer limit = provider.getValueInteger("maxPerMinute"); + if(limit != null) + { + int used = count(EmailTables.ATTEMPT, since("startDate", now.minusSeconds(60)) + .withCriteria("startDate", QCriteriaOperator.LESS_THAN_OR_EQUALS, now) + .withCriteria("providerId", QCriteriaOperator.EQUALS, provider.getValue("id"))); + provider.setValue("headroom", Math.max(0, limit - used)); + } + } + return (new RawHTML(null, EmailHomeView.render(totals, hourly(), providers, admin, messages, suppressions()))); + } + + /******************************************************************************* + ** Twenty-four UTC hour buckets, counting successful and failed attempts. + *******************************************************************************/ + public static ChartData hourly() throws QException + { + Instant start = EmailRuntime.clock().instant().truncatedTo(ChronoUnit.HOURS).minus(23, ChronoUnit.HOURS); + List labels = new ArrayList<>(); + List sent = new ArrayList<>(); + List failed = new ArrayList<>(); + for(int i = 0; i < 24; i++) + { + Instant hour = start.plus(i, ChronoUnit.HOURS); + labels.add(DateTimeFormatter.ofPattern("HH:mm", Locale.ENGLISH).withZone(ZoneOffset.UTC).format(hour)); + QQueryFilter filter = since("endDate", hour).withCriteria("endDate", QCriteriaOperator.LESS_THAN, hour.plus(1, ChronoUnit.HOURS)); + sent.add(count(EmailTables.ATTEMPT, filter.clone().withCriteria("outcome", QCriteriaOperator.EQUALS, "SUCCEEDED"))); + failed.add(count(EmailTables.ATTEMPT, filter.clone().withCriteria("outcome", QCriteriaOperator.IN, List.of("TRANSIENT_FAILURE", "PROVIDER_FAILURE", "PERMANENT_FAILURE", "LEASE_EXPIRED")))); + } + return (chart(labels, List.of(dataset("Sent", sent, "#16a34a"), dataset("Failed", failed, "#dc2626")), "No completed attempts in the last 24 hours. Counts use UTC hours.")); + } + + /******************************************************************************* + ** Suppression rows created per UTC day and reason, for thirty days. + *******************************************************************************/ + public static ChartData suppressions() throws QException + { + Instant start = EmailRuntime.clock().instant().truncatedTo(ChronoUnit.DAYS).minus(29, ChronoUnit.DAYS); + List labels = new ArrayList<>(); + List datasets = new ArrayList<>(); + List reasons = List.of("HARD_BOUNCE", "COMPLAINT", "UNSUBSCRIBE", "MANUAL"); + List colors = List.of("#dc2626", "#d97706", "#2563eb", "#6b7280"); + for(int r = 0; r < reasons.size(); r++) + { + List counts = new ArrayList<>(); + for(int i = 0; i < 30; i++) + { + Instant day = start.plus(i, ChronoUnit.DAYS); + if(r == 0) + { + labels.add(DateTimeFormatter.ofPattern("MMM d", Locale.ENGLISH).withZone(ZoneOffset.UTC).format(day)); + } + counts.add(count(EmailTables.SUPPRESSION, since("createdDate", day).withCriteria("createdDate", QCriteriaOperator.LESS_THAN, day.plus(1, ChronoUnit.DAYS)) + .withCriteria("reason", QCriteriaOperator.EQUALS, reasons.get(r)))); + } + datasets.add(dataset(com.kingsrook.qbits.email.metadata.EmailEnums.SuppressionReason.valueOf(reasons.get(r)).getPossibleValueLabel(), counts, colors.get(r))); + } + return (chart(labels, datasets, "No new suppressions in the last 30 days. Counts use UTC dates.")); + } + + /******************************************************************************* + ** Standard chart with an honest empty-state sublabel. + *******************************************************************************/ + private static ChartData chart(List labels, List datasets, String empty) + { + ChartData chart = new ChartData().withChartData(new ChartData.Data().withLabels(labels).withDatasets(datasets)); + boolean any = datasets.stream().anyMatch(d -> d.getData().stream().anyMatch(n -> n.doubleValue() > 0)); + chart.setSublabel(any ? "Real data visible to your session; UTC buckets." : empty); + return (chart); + } + + /******************************************************************************* + ** A standard chart series. + *******************************************************************************/ + private static ChartData.Data.Dataset dataset(String label, List values, String color) + { + return (new ChartData.Data.Dataset().withLabel(label).withData(values).withColor(color)); + } + + /******************************************************************************* + ** Mask display names and all but the first local-part character. + *******************************************************************************/ + public static String mask(String recipients) + { + try + { + String address = jakarta.mail.internet.InternetAddress.parse(recipients)[0].getAddress(); + int at = address.lastIndexOf('@'); + return (at > 0 ? address.charAt(0) + "***" + address.substring(at) : "[masked]"); + } + catch(Exception e) + { + return ("[masked]"); + } + } + + /******************************************************************************* + ** Date lower bound, always used with viewer-scoped actions. + *******************************************************************************/ + private static QQueryFilter since(String field, Instant since) + { + return (new QQueryFilter().withCriteria(field, QCriteriaOperator.GREATER_THAN_OR_EQUALS, since)); + } + + /******************************************************************************* + ** Exact count as the viewer, without authorized scopes. + *******************************************************************************/ + private static int count(String table, QQueryFilter filter) throws QException + { + return (new CountAction().execute(new CountInput(table).withFilter(filter)).getCount()); + } + + /******************************************************************************* + ** Select only home fields; row and field security remain active. + *******************************************************************************/ + private static List query(String table, QQueryFilter filter, List fields) throws QException + { + return (new QueryAction().execute(new QueryInput(table).withFilter(filter).withFieldNamesToInclude(new java.util.HashSet<>(fields)).withShouldFetchHeavyFields(true)).getRecords()); + } +} diff --git a/src/main/java/com/kingsrook/qbits/email/app/EmailHomeView.java b/src/main/java/com/kingsrook/qbits/email/app/EmailHomeView.java new file mode 100644 index 0000000..806bff3 --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/email/app/EmailHomeView.java @@ -0,0 +1,243 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.email.app; + + +import java.time.Instant; +import java.time.ZoneOffset; +import java.time.format.DateTimeFormatter; +import java.util.List; +import java.util.Locale; +import com.kingsrook.qbits.email.metadata.EmailEnums; +import com.kingsrook.qbits.email.metadata.EmailTables; +import com.kingsrook.qbits.email.template.EmailTemplateEngine; +import com.kingsrook.qqq.backend.core.model.dashboard.widgets.ChartData; +import com.kingsrook.qqq.backend.core.model.data.QRecord; + + +/******************************************************************************* + ** Inline styles survive the UI sanitizer. Grids collapse into labelled cards; + ** bars shrink with their container and all ordinary text inherits the theme. + *******************************************************************************/ +public final class EmailHomeView +{ + private static final String WRAP = "min-width:0;max-width:100%;overflow-wrap:anywhere;word-break:break-word;box-sizing:border-box;"; + private static final String CARD = WRAP + "padding:16px;border:1px solid rgba(127,127,127,.3);border-radius:8px;"; + private static final String ROW = WRAP + "display:grid;grid-template-columns:repeat(auto-fit,minmax(min(100%,12rem),1fr));gap:6px 16px;padding:10px 0;border-top:1px solid rgba(127,127,127,.2);"; + private static final String FAIL = "#d64545"; + private static final String OK = "#2e9e5b"; + private static final DateTimeFormatter WHEN = DateTimeFormatter.ofPattern("MMM d, HH:mm 'UTC'", Locale.ENGLISH).withZone(ZoneOffset.UTC); + + + + /******************************************************************************* + ** Utility class. + *******************************************************************************/ + private EmailHomeView() + { + } + + + + /******************************************************************************* + ** One ordered home with real, viewer-scoped data supplied by EmailHome. + *******************************************************************************/ + public static String render(List totals, ChartData hourly, List providers, boolean admin, List messages, ChartData suppressions) + { + StringBuilder html = new StringBuilder("
"); + html.append("
"); + List labels = List.of("Sent (24 h)", "Delivered (24 h)", "Dead letters (24 h)", "Bounced (7 d)", "Complaints (7 d)", "Suppressed addresses"); + for(int i = 0; i < labels.size(); i++) + { + html.append("
= 2 && i <= 4 && totals.get(i) > 0 ? "color:" + FAIL : "color:inherit").append("\">") + .append(String.format(Locale.ENGLISH, "%,d", totals.get(i))).append("
") + .append(labels.get(i)).append("
"); + } + html.append("
"); + bars(html, "Sends per hour, last 24 hours", hourly); + providers(html, providers, admin); + messages(html, messages); + bars(html, "Suppressions added, last 30 days", suppressions); + return (html.append("
").toString()); + } + + + + /******************************************************************************* + ** A bounded stacked bar chart, including exact values in each bar's title. + *******************************************************************************/ + private static void bars(StringBuilder html, String title, ChartData chart) + { + section(html, title); + var data = chart.getChartData(); + int max = 0; + for(int i = 0; i < data.getLabels().size(); i++) + { + int total = 0; + for(var dataset : data.getDatasets()) + { + total += dataset.getData().get(i).intValue(); + } + max = Math.max(max, total); + } + if(max == 0) + { + html.append("

").append(esc(chart.getSublabel())).append("

"); + return; + } + html.append("
Peak ").append(max).append("
") + .append("
"); + for(int i = 0; i < data.getLabels().size(); i++) + { + StringBuilder description = new StringBuilder(data.getLabels().get(i)); + for(var dataset : data.getDatasets()) + { + description.append("; ").append(dataset.getLabel()).append(": ").append(dataset.getData().get(i)); + } + html.append("
"); + for(var dataset : data.getDatasets()) + { + html.append("
"); + } + html.append("
"); + } + html.append("
"); + for(int i : new int[] {0, data.getLabels().size() / 2, data.getLabels().size() - 1}) + { + html.append("").append(esc(data.getLabels().get(i))).append(""); + } + html.append("
"); + for(var dataset : data.getDatasets()) + { + html.append("").append(esc(dataset.getLabel())).append(""); + } + html.append("UTC
"); + } + + + + /******************************************************************************* + ** Provider health is admin-only; disabled providers never appear healthy. + *******************************************************************************/ + private static void providers(StringBuilder html, List providers, boolean admin) + { + section(html, "Provider health"); + if(!admin || providers.isEmpty()) + { + html.append("

").append(admin ? "No providers configured." : "Provider health needs email.admin.").append("

"); + } + for(QRecord provider : providers) + { + String status = Boolean.FALSE.equals(provider.getValueBoolean("enabled")) ? "Disabled" + : "HEALTHY".equals(provider.getValueString("healthStatus")) ? "Healthy" : "UNHEALTHY".equals(provider.getValueString("healthStatus")) ? "Unhealthy" : "Unknown"; + String color = "Healthy".equals(status) ? OK : "Unhealthy".equals(status) ? FAIL : "#808080"; + html.append("
") + .append(cell("Provider", "" + esc(value(provider, "label", value(provider, "name", "Provider"))) + "
" + status + "")) + .append(cell("Last success", esc(when(provider.getValueInstant("lastSuccessDate"))))) + .append(cell("Last failure", esc(value(provider, "lastFailureMessage", "None")))) + .append(cell("Per-minute headroom", esc(value(provider, "headroom", "No limit")))).append("
"); + } + html.append(""); + } + + + + /******************************************************************************* + ** The whole row is a link. Labelled columns stack as cards on narrow screens. + *******************************************************************************/ + private static void messages(StringBuilder html, List messages) + { + section(html, "Recent messages"); + if(messages.isEmpty()) + { + html.append("

No messages yet.

"); + } + for(QRecord message : messages) + { + String subject = Boolean.TRUE.equals(message.getValueBoolean("phiApproved")) || "[encrypted]".equals(message.getValueString("subject")) + ? "Secure message" : value(message, "subject", "Message " + message.getValueInteger("id")); + String status = message.getValueString("status"); + html.append("") + .append(cell("When", esc(when(message.getValueInstant("createDate"))))) + .append(cell("To", esc(EmailHome.mask(message.getValueString("toAddresses"))))) + .append(cell("Subject", esc(subject))) + .append(cell("Status", esc(status == null ? "Unknown" : EmailEnums.MessageStatus.valueOf(status).getPossibleValueLabel()))) + .append(""); + } + html.append(""); + } + + + + /******************************************************************************* + ** One bordered section, using the surrounding theme's text colour. + *******************************************************************************/ + private static void section(StringBuilder html, String title) + { + html.append("

").append(esc(title)).append("

"); + } + + + + /******************************************************************************* + ** A labelled, shrinkable cell that remains understandable when stacked. + *******************************************************************************/ + private static String cell(String label, String content) + { + return ("
" + esc(label) + " " + content + "
"); + } + + + + /******************************************************************************* + ** Compact UTC dates, with an honest missing value. + *******************************************************************************/ + private static String when(Instant instant) + { + return (instant == null ? "None" : WHEN.format(instant)); + } + + + + /******************************************************************************* + ** Blank fields use a human-readable fallback. + *******************************************************************************/ + private static String value(QRecord row, String field, String fallback) + { + String value = row.getValueString(field); + return (value == null || value.isBlank() ? fallback : value); + } + + + + /******************************************************************************* + ** Escape every record-sourced value before inserting HTML. + *******************************************************************************/ + private static String esc(String value) + { + return (EmailTemplateEngine.escape(value)); + } +} diff --git a/src/main/java/com/kingsrook/qbits/email/bridge/EmailOutboxMessagingProvider.java b/src/main/java/com/kingsrook/qbits/email/bridge/EmailOutboxMessagingProvider.java index a657a9f..811e37b 100644 --- a/src/main/java/com/kingsrook/qbits/email/bridge/EmailOutboxMessagingProvider.java +++ b/src/main/java/com/kingsrook/qbits/email/bridge/EmailOutboxMessagingProvider.java @@ -59,7 +59,8 @@ public class EmailOutboxMessagingProvider implements MessagingProviderInterface { throw (new QException("Email attachments are not supported in part 1")); } - EmailRequest request = new EmailRequest().withSubject(input.getSubject()); + EmailRequest request = input instanceof EmailTemplatedMessageInput templated && templated.getEmailRequest() != null + ? templated.getEmailRequest() : new EmailRequest().withSubject(input.getSubject()); parties(request, input.getTo(), EmailPartyRole.TO); parties(request, input.getFrom(), EmailPartyRole.FROM); if(input.getContentList() != null) diff --git a/src/main/java/com/kingsrook/qbits/email/bridge/EmailTemplatedMessageInput.java b/src/main/java/com/kingsrook/qbits/email/bridge/EmailTemplatedMessageInput.java new file mode 100644 index 0000000..907a3d0 --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/email/bridge/EmailTemplatedMessageInput.java @@ -0,0 +1,52 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.email.bridge; + + +import com.kingsrook.qbits.email.outbox.EmailRequest; +import com.kingsrook.qqq.backend.core.model.actions.messaging.SendMessageInput; + + +/******************************************************************************* + ** Optional typed template payload for core SendMessageAction bridge callers. + ** Plain SendMessageInput remains an ad-hoc message for backwards compatibility. + *******************************************************************************/ +public class EmailTemplatedMessageInput extends SendMessageInput +{ + private EmailRequest emailRequest; + + /******************************************************************************* + ** Set schema variables and record references for the normal validated API. + *******************************************************************************/ + public EmailTemplatedMessageInput withEmailRequest(EmailRequest request) + { + emailRequest = request; + return (this); + } + + /******************************************************************************* + ** The caller's template request; party fields can still come from core input. + *******************************************************************************/ + public EmailRequest getEmailRequest() + { + return (emailRequest); + } +} diff --git a/src/main/java/com/kingsrook/qbits/email/delivery/EmailDispatcher.java b/src/main/java/com/kingsrook/qbits/email/delivery/EmailDispatcher.java index 7d95f4c..f5a8be4 100644 --- a/src/main/java/com/kingsrook/qbits/email/delivery/EmailDispatcher.java +++ b/src/main/java/com/kingsrook/qbits/email/delivery/EmailDispatcher.java @@ -67,7 +67,7 @@ public class EmailDispatcher *******************************************************************************/ public EmailDispatcher() { - this(new EmailTransportRegistry()); + this(EmailRuntime.config() == null ? new EmailTransportRegistry() : EmailRuntime.config().newTransportRegistry()); } /******************************************************************************* @@ -201,6 +201,10 @@ public class EmailDispatcher EmailTransportException error = null; try { + if(Boolean.TRUE.equals(claim.message().getValueBoolean("phiApproved"))) + { + com.kingsrook.qbits.email.template.EmailTemplateRenderer.checkProvider(claim.provider()); + } result = transports.get(claim.provider().getValueString("kind")).send(envelope(claim.message()), new EmailProviderSettings(claim.provider(), EmailRuntime.config())); } catch(EmailTransportException e) @@ -449,8 +453,28 @@ public class EmailDispatcher { return (new EmailEnvelope().withMessageId(message.getValueString("id")).withFrom(message.getValueString("fromAddress")) .withTo(split(message.getValueString("toAddresses"))).withCc(split(message.getValueString("ccAddresses"))).withBcc(split(message.getValueString("bccAddresses"))) - .withReplyTo(message.getValueString("replyTo")).withSubject(message.getValueString("subject")) - .withHtmlBody(message.getValueString("htmlBody")).withTextBody(message.getValueString("textBody"))); + .withReplyTo(message.getValueString("replyTo")).withSubject(content(message, "subject")) + .withHtmlBody(content(message, "htmlBody")) + .withTextBody(content(message, "textBody")) + .withHeaders(message.getValueString("unsubscribeUrl") == null ? java.util.Map.of() : java.util.Map.of("List-Unsubscribe", "<" + message.getValueString("unsubscribeUrl") + ">", "List-Unsubscribe-Post", "List-Unsubscribe=One-Click"))); + } + + /******************************************************************************* + ** Only policy-marked messages may invoke body decryption. A caller-supplied + ** enc:v1 string in ad-hoc content is text, never a decryption capability. + *******************************************************************************/ + private static String content(QRecord message, String field) + { + String value = message.getValueString(field); + if(!Boolean.TRUE.equals(message.getValueBoolean("phiApproved"))) + { + return (value); + } + if(value != null && !value.startsWith("enc:v1:")) + { + throw (new IllegalArgumentException("Protected email content must be encrypted")); + } + return (com.kingsrook.qbits.email.phi.EmailBodyEncryption.decrypt(value, field, message.getValueString("id"))); } /******************************************************************************* diff --git a/src/main/java/com/kingsrook/qbits/email/metadata/EmailEnums.java b/src/main/java/com/kingsrook/qbits/email/metadata/EmailEnums.java index 9d35729..6dc9424 100644 --- a/src/main/java/com/kingsrook/qbits/email/metadata/EmailEnums.java +++ b/src/main/java/com/kingsrook/qbits/email/metadata/EmailEnums.java @@ -130,7 +130,7 @@ public final class EmailEnums @Override public String getPossibleValueLabel() { - return (name().charAt(0) + name().substring(1).toLowerCase(java.util.Locale.ROOT).replace('_', ' ')); + return (this == RETRY_WAIT ? "Waiting to retry" : name().charAt(0) + name().substring(1).toLowerCase(java.util.Locale.ROOT).replace('_', ' ')); } } diff --git a/src/main/java/com/kingsrook/qbits/email/metadata/EmailTableCustomizers.java b/src/main/java/com/kingsrook/qbits/email/metadata/EmailTableCustomizers.java index c083699..ab1a579 100644 --- a/src/main/java/com/kingsrook/qbits/email/metadata/EmailTableCustomizers.java +++ b/src/main/java/com/kingsrook/qbits/email/metadata/EmailTableCustomizers.java @@ -239,6 +239,11 @@ public final class EmailTableCustomizers @Override public List postQuery(QueryOrGetInputInterface input, List records) { + for(QRecord row : records) + { + String label = row.getValueString("label"); + row.setRecordLabel(label == null || label.isBlank() ? row.getValueString("name") : label); + } if(!SystemWrites.active()) { for(QRecord row : records) @@ -339,6 +344,27 @@ public final class EmailTableCustomizers @Override public List postQuery(QueryOrGetInputInterface input, List records) { + if(!SystemWrites.active()) + { + for(QRecord row : records) + { + for(String field : List.of("subject", "htmlBody", "textBody")) + { + String value = row.getValueString(field); + if(value != null && value.startsWith("enc:v1:")) + { + row.setValue(field, "[encrypted]"); + row.setDisplayValue(field, "[encrypted]"); + } + } + } + } + for(QRecord row : records) + { + String subject = row.getValueString("subject"); + boolean secure = Boolean.TRUE.equals(row.getValueBoolean("phiApproved")) || "[encrypted]".equals(subject) || (subject != null && subject.startsWith("enc:v1:")); + row.setRecordLabel(secure ? "Secure message" : subject == null || subject.isBlank() ? "Message " + row.getValue("id") : subject); + } if(input.getShouldGenerateDisplayValues()) { for(QRecord row : records) diff --git a/src/main/java/com/kingsrook/qbits/email/metadata/EmailTables.java b/src/main/java/com/kingsrook/qbits/email/metadata/EmailTables.java index dce42c5..8948716 100644 --- a/src/main/java/com/kingsrook/qbits/email/metadata/EmailTables.java +++ b/src/main/java/com/kingsrook/qbits/email/metadata/EmailTables.java @@ -96,6 +96,14 @@ public final class EmailTables result.add(QPossibleValueSource.newForEnum("email_suppression_reason", EmailEnums.SuppressionReason.values())); result.add(QPossibleValueSource.newForEnum("email_category", EmailCategory.values())); ALL.forEach(name -> result.add(QPossibleValueSource.newForTable(name))); + for(QPossibleValueSource source : result) + { + if(List.of(PROVIDER, MESSAGE, VERSION).contains(source.getName())) + { + source.withSearchFields(PROVIDER.equals(source.getName()) ? List.of("label", "name") : MESSAGE.equals(source.getName()) ? List.of("subject") : List.of("versionNumber", "locale")) + .withOrderByField("id"); + } + } return (result); } @@ -189,7 +197,7 @@ public final class EmailTables .withField(field("lastFailureMessage", QFieldType.STRING).withMaxLength(500)) .withField(field("dispatchLock", QFieldType.INTEGER).withDefaultValue(0).withIsHidden(true).withIsEditable(false)) .withField(field("defaultSlot", QFieldType.INTEGER).withIsHidden(true).withIsEditable(false)); - table.withRecordLabelFormatAndFields("%s", "name"); + table.withRecordLabelFormat(null).withRecordLabelFields(List.of()); table.withUniqueKey(new UniqueKey("name")); table.withUniqueKey(new UniqueKey("defaultSlot")); table.withSection(SectionFactory.defaultT1("id", "name", "label", "kind", "enabled", "isDefault")); @@ -214,13 +222,15 @@ public final class EmailTables .withField(field("replyTo", QFieldType.STRING).withIsSensitive(true).withMaxLength(500)) .withField(field("subject", QFieldType.TEXT).withIsSensitive(true)) .withField(field("templateId", QFieldType.INTEGER).withPossibleValueSourceName(TEMPLATE)) - .withField(field("templateVersionId", QFieldType.INTEGER).withPossibleValueSourceName(VERSION)) + .withField(field("templateVersionId", QFieldType.INTEGER).withPossibleValueSourceName(VERSION).withPossibleValueSourceFilter(com.kingsrook.qbits.email.support.EmailStore.equal("templateId", "${input.templateId}"))) .withField(field("brandId", QFieldType.INTEGER).withPossibleValueSourceName(BRAND)) .withField(field("category", QFieldType.STRING).withPossibleValueSourceName("email_category").withMaxLength(500)) .withField(field("locale", QFieldType.STRING).withDefaultValue("en").withMaxLength(500)) .withField(field("variablesSnapshot", QFieldType.TEXT).withIsSensitive(true)) .withField(field("htmlBody", QFieldType.TEXT).withIsSensitive(true)) .withField(field("textBody", QFieldType.TEXT).withIsSensitive(true)) + .withField(field("phiApproved", QFieldType.BOOLEAN).withDefaultValue(false).withIsEditable(false)) + .withField(field("unsubscribeUrl", QFieldType.TEXT)) .withField(field("bodySha256", QFieldType.STRING).withMaxLength(500)) .withField(field("idempotencyKey", QFieldType.STRING).withMaxLength(500)) .withField(field("providerId", QFieldType.INTEGER).withPossibleValueSourceName(PROVIDER)) @@ -232,6 +242,8 @@ public final class EmailTables .withField(field("finishedDate", QFieldType.DATE_TIME)) .withField(field("sentDate", QFieldType.DATE_TIME)) .withField(field("deliveredDate", QFieldType.DATE_TIME)) + .withField(field("bouncedDate", QFieldType.DATE_TIME)) + .withField(field("complainedDate", QFieldType.DATE_TIME)) .withField(field("deadLetterDate", QFieldType.DATE_TIME)) .withField(field("linkedTableName", QFieldType.STRING).withMaxLength(500)) .withField(field("linkedRecordId", QFieldType.STRING).withMaxLength(500) @@ -240,9 +252,11 @@ public final class EmailTables .withField(field("senderProcess", QFieldType.STRING).withMaxLength(500)) .withField(field("suppressedAddresses", QFieldType.TEXT).withIsSensitive(true)) .withField(field("sendAfter", QFieldType.DATE_TIME)); + table.withRecordLabelFormat(null).withRecordLabelFields(List.of()); + table.getField("subject").setIsHeavy(false); table.withUniqueKey(new UniqueKey("idempotencyKey")); table.withSection(SectionFactory.defaultT1("id", "status", "toAddresses", "ccAddresses", "bccAddresses", "fromAddress")); - table.withSection(SectionFactory.defaultT2("replyTo", "subject", "templateId", "templateVersionId", "brandId", "category", "locale", "variablesSnapshot", "htmlBody", "textBody", "bodySha256", "idempotencyKey", "providerId", "providerMessageId", "attemptCount", "nextAttemptDate", "leaseExpiresDate", "lastError", "finishedDate", "sentDate", "deliveredDate", "deadLetterDate", "linkedTableName", "linkedRecordId", "senderQbit", "senderProcess", "suppressedAddresses", "sendAfter")); + table.withSection(SectionFactory.defaultT2("replyTo", "subject", "templateId", "templateVersionId", "brandId", "category", "locale", "variablesSnapshot", "htmlBody", "textBody", "phiApproved", "unsubscribeUrl", "bodySha256", "idempotencyKey", "providerId", "providerMessageId", "attemptCount", "nextAttemptDate", "leaseExpiresDate", "lastError", "finishedDate", "sentDate", "deliveredDate", "bouncedDate", "complainedDate", "deadLetterDate", "linkedTableName", "linkedRecordId", "senderQbit", "senderProcess", "suppressedAddresses", "sendAfter")); table.withSection(SectionFactory.defaultT3("createDate", "modifyDate")); table.withSection(SectionFactory.customT2("children", new QIcon("list")).withLabel("Attempts").withWidgetName(MESSAGE + "_children")); return (table); @@ -284,17 +298,19 @@ public final class EmailTables .withField(field("category", QFieldType.STRING).withPossibleValueSourceName("email_category").withMaxLength(500)) .withField(field("kind", QFieldType.STRING).withPossibleValueSourceName("email_template_kind").withIsRequired(true).withMaxLength(500)) .withField(field("brandId", QFieldType.INTEGER).withPossibleValueSourceName(BRAND)) - .withField(field("activeVersionId", QFieldType.INTEGER).withPossibleValueSourceName(VERSION)) - .withField(field("phiApproved", QFieldType.BOOLEAN).withDefaultValue(false)) - .withField(field("phiApprovedBy", QFieldType.STRING).withMaxLength(500)) - .withField(field("phiApprovedDate", QFieldType.DATE_TIME)) + .withField(field("activeVersionId", QFieldType.INTEGER).withPossibleValueSourceName(VERSION).withIsEditable(false).withPossibleValueSourceFilter(com.kingsrook.qbits.email.support.EmailStore.equal("templateId", "${input.id}"))) + .withField(field("approvedVersionId", QFieldType.INTEGER).withIsEditable(false).withPossibleValueSourceName(VERSION).withPossibleValueSourceFilter(com.kingsrook.qbits.email.support.EmailStore.equal("templateId", "${input.id}"))) + .withField(field("versionLock", QFieldType.INTEGER).withIsHidden(true).withIsEditable(false)) + .withField(field("phiApproved", QFieldType.BOOLEAN).withDefaultValue(false).withIsEditable(false)) + .withField(field("phiApprovedBy", QFieldType.STRING).withMaxLength(500).withIsEditable(false)) + .withField(field("phiApprovedDate", QFieldType.DATE_TIME).withIsEditable(false)) .withField(field("editPermissionName", QFieldType.STRING).withMaxLength(500)) .withField(field("ownerQbit", QFieldType.STRING).withMaxLength(500)) .withField(field("description", QFieldType.TEXT)); table.withRecordLabelFormatAndFields("%s", "name"); table.withUniqueKey(new UniqueKey("name")); table.withSection(SectionFactory.defaultT1("id", "name", "label", "category", "kind", "brandId")); - table.withSection(SectionFactory.defaultT2("activeVersionId", "phiApproved", "phiApprovedBy", "phiApprovedDate", "editPermissionName", "ownerQbit", "description")); + table.withSection(SectionFactory.defaultT2("activeVersionId", "approvedVersionId", "phiApproved", "phiApprovedBy", "phiApprovedDate", "editPermissionName", "ownerQbit", "description")); table.withSection(SectionFactory.defaultT3("createDate", "modifyDate")); table.withSection(SectionFactory.customT2("children", new QIcon("list")).withLabel("Versions").withWidgetName(TEMPLATE + "_children")); return (table); @@ -307,20 +323,22 @@ public final class EmailTables private static QTableMetaData templateversion(EmailQBitConfig config) { QTableMetaData table = base(config, VERSION, "Template Version") - .withField(field("templateId", QFieldType.INTEGER).withPossibleValueSourceName(TEMPLATE)) - .withField(field("versionNumber", QFieldType.INTEGER)) - .withField(field("status", QFieldType.STRING).withPossibleValueSourceName("email_version_status").withMaxLength(500)) - .withField(field("locale", QFieldType.STRING).withDefaultValue("en").withMaxLength(500)) + .withField(field("templateId", QFieldType.INTEGER).withPossibleValueSourceName(TEMPLATE).withIsRequired(true)) + .withField(field("versionNumber", QFieldType.INTEGER).withIsRequired(true)) + .withField(field("status", QFieldType.STRING).withPossibleValueSourceName("email_version_status").withDefaultValue("DRAFT").withIsEditable(false).withMaxLength(500)) + .withField(field("locale", QFieldType.STRING).withDefaultValue("en").withMaxLength(500).withIsRequired(true)) .withField(field("subject", QFieldType.TEXT).withIsSensitive(true)) .withField(field("preheader", QFieldType.TEXT)) .withField(field("bodySource", QFieldType.TEXT)) .withField(field("textSource", QFieldType.TEXT)) - .withField(field("compiledHtml", QFieldType.TEXT)) + .withField(field("compiledHtml", QFieldType.TEXT).withIsEditable(false)) .withField(field("variablesSchema", QFieldType.TEXT)) .withField(field("sampleVariables", QFieldType.TEXT)) .withField(field("createdBy", QFieldType.STRING).withMaxLength(500)) .withField(field("activatedDate", QFieldType.DATE_TIME)); + table.withRecordLabelFormat(null).withRecordLabelFields(List.of()); table.withUniqueKey(new UniqueKey("templateId", "versionNumber", "locale")); + table.withSection(SectionFactory.customT2("preview", new QIcon("preview")).withLabel("Preview").withWidgetName("emailTemplatePreview")); table.withSection(SectionFactory.defaultT1("id", "templateId", "versionNumber", "status", "locale", "subject")); table.withSection(SectionFactory.defaultT2("preheader", "bodySource", "textSource", "compiledHtml", "variablesSchema", "sampleVariables", "createdBy", "activatedDate")); table.withSection(SectionFactory.defaultT3("createDate", "modifyDate")); @@ -343,6 +361,8 @@ public final class EmailTables .withField(field("textColor", QFieldType.STRING).withMaxLength(500)) .withField(field("backgroundColor", QFieldType.STRING).withMaxLength(500)) .withField(field("darkBackgroundColor", QFieldType.STRING).withMaxLength(500)) + .withField(field("darkPrimaryColor", QFieldType.STRING).withMaxLength(500)) + .withField(field("darkButtonTextColor", QFieldType.STRING).withMaxLength(500)) .withField(field("darkTextColor", QFieldType.STRING).withMaxLength(500)) .withField(field("buttonTextColor", QFieldType.STRING).withMaxLength(500)) .withField(field("footerMarkdown", QFieldType.TEXT)) @@ -354,7 +374,7 @@ public final class EmailTables table.withRecordLabelFormatAndFields("%s", "name"); table.withUniqueKey(new UniqueKey("name")); table.withSection(SectionFactory.defaultT1("id", "name", "label", "logoUrl", "logoDarkUrl", "logoAlt")); - table.withSection(SectionFactory.defaultT2("primaryColor", "textColor", "backgroundColor", "darkBackgroundColor", "darkTextColor", "buttonTextColor", "footerMarkdown", "postalAddress", "socialLinks", "siteUrl", "isDefault", "appName")); + table.withSection(SectionFactory.defaultT2("primaryColor", "textColor", "backgroundColor", "darkBackgroundColor", "darkTextColor", "darkPrimaryColor", "darkButtonTextColor", "buttonTextColor", "footerMarkdown", "postalAddress", "socialLinks", "siteUrl", "isDefault", "appName")); table.withSection(SectionFactory.defaultT3("createDate", "modifyDate")); return (table); } diff --git a/src/main/java/com/kingsrook/qbits/email/outbox/AdHocEmailRenderer.java b/src/main/java/com/kingsrook/qbits/email/outbox/AdHocEmailRenderer.java index cdb7ae6..93c1ed4 100644 --- a/src/main/java/com/kingsrook/qbits/email/outbox/AdHocEmailRenderer.java +++ b/src/main/java/com/kingsrook/qbits/email/outbox/AdHocEmailRenderer.java @@ -21,15 +21,10 @@ package com.kingsrook.qbits.email.outbox; -import java.util.List; -import org.commonmark.ext.autolink.AutolinkExtension; -import org.commonmark.ext.gfm.tables.TablesExtension; -import org.commonmark.parser.Parser; -import org.commonmark.renderer.html.HtmlRenderer; /******************************************************************************* - ** Minimal part 1 layout. Markdown raw HTML and unsafe URL schemes are disabled. + ** Branded ad-hoc rendering. Markdown raw HTML and unsafe URL schemes are disabled. *******************************************************************************/ public class AdHocEmailRenderer implements EmailRenderer { @@ -39,36 +34,22 @@ public class AdHocEmailRenderer implements EmailRenderer @Override public Rendered render(EmailRequest request) { - String html = request.getHtmlBody(); - if(html == null) + try { - if(request.getMarkdownBody() != null) + var brand = com.kingsrook.qbits.email.render.EmailBrands.resolve(request.getBrandName(), null); + String html = request.getHtmlBody(); + if(html == null) { - var extensions = List.of(TablesExtension.create(), AutolinkExtension.create()); - html = HtmlRenderer.builder().escapeHtml(true).sanitizeUrls(true).extensions(extensions).build() - .render(Parser.builder().extensions(extensions).build().parse(request.getMarkdownBody())); - } - else - { - html = "
" + escape(request.getTextBody()) + "
"; + String body = request.getMarkdownBody() == null ? "
" + com.kingsrook.qbits.email.template.EmailTemplateEngine.escape(request.getTextBody()) + "
" + : com.kingsrook.qbits.email.render.EmailMarkdown.render(request.getMarkdownBody(), brand); + html = com.kingsrook.qbits.email.render.EmailLayout.render(body, "", brand, + request.getCategory() == null ? EmailCategory.TRANSACTIONAL : request.getCategory(), ""); } + return (new Rendered(html, request.getTextBody() == null ? com.kingsrook.qbits.email.render.HtmlToText.convert(html) : request.getTextBody())); } - String text = request.getTextBody(); - if(text == null) + catch(com.kingsrook.qqq.backend.core.exceptions.QException e) { - text = html.replaceAll("(?is)<(script|style)[^>]*>.*?", "") - .replaceAll("(?i)|", "\n") - .replaceAll("<[^>]+>", "").replace("<", "<").replace(">", ">").replace(""", "\"") - .replace("'", "'").replace(" ", " ").replace("&", "&").strip(); + throw (new IllegalArgumentException("Email brand is unavailable")); } - return (new Rendered("
" + html + "
", text)); - } - - /******************************************************************************* - ** Escape plain text for the fallback HTML part. - *******************************************************************************/ - private static String escape(String text) - { - return (text == null ? "" : text.replace("&", "&").replace("<", "<").replace(">", ">")); } } diff --git a/src/main/java/com/kingsrook/qbits/email/outbox/EmailRequest.java b/src/main/java/com/kingsrook/qbits/email/outbox/EmailRequest.java index f4b5e49..6b0b7e9 100644 --- a/src/main/java/com/kingsrook/qbits/email/outbox/EmailRequest.java +++ b/src/main/java/com/kingsrook/qbits/email/outbox/EmailRequest.java @@ -31,7 +31,7 @@ import com.kingsrook.qqq.backend.core.actions.QBackendTransaction; /******************************************************************************* - ** Synchronous enqueue request. Templates and classified variables arrive in part 2. + ** Synchronous enqueue request with explicit record provenance. *******************************************************************************/ public class EmailRequest { @@ -48,7 +48,7 @@ public class EmailRequest private String templateName; private String brandName; private String locale = "en"; - private EmailCategory category = EmailCategory.TRANSACTIONAL; + private EmailCategory category; private String idempotencyKey; private Instant sendAfter; private String linkedTableName; @@ -57,6 +57,7 @@ public class EmailRequest private String senderProcess; private QBackendTransaction transaction; private String transactionBackendName; + private Map recordVariables = new LinkedHashMap<>(); private Map variables = new LinkedHashMap<>(); private Map securityValues = new LinkedHashMap<>(); @@ -118,7 +119,7 @@ public class EmailRequest } /******************************************************************************* - ** Accept template variables as a seam; part 1 refuses variable-bearing requests. + ** Bind one schema variable. *******************************************************************************/ public EmailRequest withVariable(String name, Serializable value) { @@ -131,7 +132,7 @@ public class EmailRequest *******************************************************************************/ public EmailRequest withRecordVariable(String name, String table, Serializable id) { - variables.put(name, new LinkedHashMap<>(Map.of("table", table, "id", id))); + recordVariables.put(name, new RecordVariable(table, id)); return (this); } @@ -590,4 +591,19 @@ public class EmailRequest this.securityValues = securityValues; return (this); } + + /******************************************************************************* + ** A record reference is distinct from an ordinary JSON object. + *******************************************************************************/ + public record RecordVariable(String table, Serializable id) + { + } + + /******************************************************************************* + ** Record bindings, read under the sender session by the service. + *******************************************************************************/ + public Map getRecordVariables() + { + return (recordVariables); + } } diff --git a/src/main/java/com/kingsrook/qbits/email/outbox/EmailService.java b/src/main/java/com/kingsrook/qbits/email/outbox/EmailService.java index 620b923..47a6332 100644 --- a/src/main/java/com/kingsrook/qbits/email/outbox/EmailService.java +++ b/src/main/java/com/kingsrook/qbits/email/outbox/EmailService.java @@ -38,6 +38,7 @@ import com.kingsrook.qbits.email.permissions.EmailPermissionChecker; import com.kingsrook.qbits.email.permissions.EmailPermissions; import com.kingsrook.qbits.email.support.EmailStore; import com.kingsrook.qbits.email.support.SystemWrites; +import com.kingsrook.qbits.email.template.EmailTemplateRenderer; import com.kingsrook.qqq.backend.core.actions.QBackendTransaction; import com.kingsrook.qqq.backend.core.actions.permissions.CorePermissionEnforcer; import com.kingsrook.qqq.backend.core.exceptions.QException; @@ -64,7 +65,7 @@ public final class EmailService } /******************************************************************************* - ** Queue with the part 1 renderer. + ** Queue validated, rendered alternatives with optional versioned templates. *******************************************************************************/ public static EmailSendResult send(EmailRequest request) throws QException { @@ -72,9 +73,17 @@ public final class EmailService } /******************************************************************************* - ** Queue with a renderer supplied by the host or a later phase. + ** Queue with a host-supplied ad-hoc renderer; templates always use the validated engine. *******************************************************************************/ public static EmailSendResult send(EmailRequest request, EmailRenderer renderer) throws QException + { + return (send(request, renderer, null)); + } + + /******************************************************************************* + ** Shared enqueue path, including already protected retained content. + *******************************************************************************/ + private static EmailSendResult send(EmailRequest request, EmailRenderer renderer, QRecord retained) throws QException { EmailQBitConfig config = EmailRuntime.config(); if(config == null) @@ -89,20 +98,28 @@ public final class EmailService { return (refused("An email request is required")); } - if(request.getTemplateName() != null) + EmailTemplateRenderer templateRenderer; + QRecord templateContent; + try { - return (refused("templates arrive in part 2")); + templateRenderer = request.getTemplateName() == null ? null : EmailTemplateRenderer.prepare(request); + templateContent = templateRenderer == null ? retained : templateRenderer.render("pending", "https://example.invalid/unsubscribe/pending", true); } - if(!request.getVariables().isEmpty() || request.getBrandName() != null) + catch(IllegalArgumentException e) { - return (refused("Template variables and branding arrive in part 2")); + return (refused(e.getMessage())); } - if(request.getSubject() == null || request.getSubject().contains("\r") || request.getSubject().contains("\n") - || (request.getMarkdownBody() == null && request.getHtmlBody() == null && request.getTextBody() == null)) + if(templateRenderer == null && (!request.getVariables().isEmpty() || !request.getRecordVariables().isEmpty())) + { + return (refused("Variables require a template with a schema")); + } + if(templateRenderer == null && (request.getSubject() == null || request.getSubject().contains("\r") || request.getSubject().contains("\n") + || (request.getMarkdownBody() == null && request.getHtmlBody() == null && request.getTextBody() == null))) { return (refused("A subject and message body are required; subject must be one line")); } - EmailCategory category = request.getCategory() == null ? EmailCategory.TRANSACTIONAL : request.getCategory(); + EmailCategory category = templateContent == null ? (request.getCategory() == null ? EmailCategory.TRANSACTIONAL : request.getCategory()) + : EmailCategory.valueOf(templateContent.getValueString("category")); QBackendTransaction transaction = request.getTransaction(); boolean sameBackend = transaction == null || request.getTransactionBackendName() == null || config.getBackendName().equals(request.getTransactionBackendName()); QBackendTransaction queryTransaction = sameBackend ? transaction : null; @@ -125,6 +142,22 @@ public final class EmailService { return (new EmailSendResult(null, EmailSendResult.Status.REFUSED, providerErrors)); } + try + { + if(templateRenderer != null) + { + templateRenderer.validateProvider(provider); + } + else if(retained != null && Boolean.TRUE.equals(retained.getValueBoolean("phiApproved"))) + { + EmailTemplateRenderer.checkProvider(provider); + com.kingsrook.qbits.email.phi.EmailBodyEncryption.encrypt(null, "subject", "pending"); + } + } + catch(IllegalArgumentException e) + { + return (refused(e.getMessage())); + } List to; List cc; List bcc; @@ -180,7 +213,15 @@ public final class EmailService }); } boolean allSuppressed = to.isEmpty() && cc.isEmpty() && bcc.isEmpty(); - EmailRenderer.Rendered rendered = renderer.render(request); + EmailRenderer.Rendered rendered; + try + { + rendered = templateContent == null ? renderer.render(request) : new EmailRenderer.Rendered(templateContent.getValueString("htmlBody"), templateContent.getValueString("textBody")); + } + catch(IllegalArgumentException e) + { + return (refused(e.getMessage())); + } Instant now = EmailRuntime.clock().instant(); QRecord row = new QRecord().withValue("providerId", provider.getValue("id")) .withValue("status", allSuppressed ? "SUPPRESSED" : "QUEUED").withValue("finishedDate", allSuppressed ? now : null).withValue("attemptCount", 0) @@ -192,6 +233,18 @@ public final class EmailService .withValue("senderQbit", request.getSenderQbit()).withValue("senderProcess", request.getSenderProcess()) .withValue("sendAfter", request.getSendAfter()).withValue("nextAttemptDate", request.getSendAfter() == null ? now : request.getSendAfter()) .withValue("suppressedAddresses", String.join(",", dropped)); + if(templateContent != null) + { + row.getValues().putAll(templateContent.getValues()); + } + try + { + com.kingsrook.qbits.email.render.EmailUnsubscribeLinks.create("pending", String.join(",", to), category.name()); + } + catch(IllegalArgumentException e) + { + return (refused(e.getMessage())); + } for(var field : config.getSecurityFields()) { row.setValue(field.getName(), request.getSecurityValues().get(field.getName())); @@ -202,7 +255,7 @@ public final class EmailService { try { - if(insert(row, null).status() == EmailSendResult.Status.REFUSED) + if(insert(row, null, templateRenderer).status() == EmailSendResult.Status.REFUSED) { throw (new QException("Email after-commit enqueue failed")); } @@ -216,21 +269,70 @@ public final class EmailService return (new EmailSendResult(null, allSuppressed ? EmailSendResult.Status.SUPPRESSED : EmailSendResult.Status.QUEUED, List.of("Enqueue deferred until caller transaction commits"))); } - return (insert(row, transaction)); + return (insert(row, transaction, templateRenderer)); + } + + /******************************************************************************* + ** Resend retained content without decrypting it or downgrading PHI policy. + ** Snapshot tokens keep the original binding; the new row links to that row. + *******************************************************************************/ + public static EmailSendResult resend(Integer id) throws QException + { + if(!SystemWrites.active() && !EmailPermissionChecker.has(EmailPermissions.MANAGE)) + { + return (refused("Resending email needs email.manage")); + } + QRecord row = EmailStore.get(EmailTables.MESSAGE, id, null); + if(row == null || row.getValueString("htmlBody") == null && row.getValueString("textBody") == null) + { + return (refused("Retained message content is unavailable")); + } + QRecord provider = EmailStore.get(EmailTables.PROVIDER, row.getValueInteger("providerId"), null); + if(provider == null) + { + return (refused("Provider is unavailable")); + } + EmailRequest request = new EmailRequest().withSubject(row.getValueString("subject")).withHtmlBody(row.getValueString("htmlBody")) + .withTextBody(row.getValueString("textBody")).withFrom(row.getValueString("fromAddress")).withReplyTo(row.getValueString("replyTo")) + .withProviderName(provider.getValueString("name")).withCategory(EmailCategory.valueOf(row.getValueString("category"))) + .withLinkedRecord(EmailTables.MESSAGE, id).withSender("email", "email_resend"); + if(row.getValueString("toAddresses") != null && !row.getValueString("toAddresses").isBlank()) + { + request.withTo(row.getValueString("toAddresses")); + } + if(row.getValueString("ccAddresses") != null && !row.getValueString("ccAddresses").isBlank()) + { + request.withCc(row.getValueString("ccAddresses")); + } + if(row.getValueString("bccAddresses") != null && !row.getValueString("bccAddresses").isBlank()) + { + request.withBcc(row.getValueString("bccAddresses")); + } + for(var field : EmailRuntime.config().getSecurityFields()) + { + request.getSecurityValues().put(field.getName(), row.getValue(field.getName())); + } + QRecord content = new QRecord(); + for(String field : List.of("subject", "htmlBody", "textBody", "variablesSnapshot", "phiApproved", "templateId", "templateVersionId", "brandId", "locale", "category", "unsubscribeUrl")) + { + content.setValue(field, row.getValue(field)); + } + content.withValue("encryptionSourceId", row.getValueString("id")); + return (send(request, ignored -> new EmailRenderer.Rendered(row.getValueString("htmlBody"), row.getValueString("textBody")), content)); } /******************************************************************************* ** Insert once; the database unique key arbitrates concurrent requests. *******************************************************************************/ - private static EmailSendResult insert(QRecord row, QBackendTransaction caller) throws QException + private static EmailSendResult insert(QRecord row, QBackendTransaction caller, EmailTemplateRenderer templateRenderer) throws QException { if(caller != null) { - return (insertInTransaction(row, caller)); + return (insertInTransaction(row, caller, templateRenderer)); } try(QBackendTransaction transaction = QBackendTransaction.openFor(new InsertInput(EmailTables.MESSAGE))) { - EmailSendResult result = insertInTransaction(row, transaction); + EmailSendResult result = insertInTransaction(row, transaction, templateRenderer); if(result.status() == EmailSendResult.Status.REFUSED) { transaction.rollback(); @@ -248,7 +350,7 @@ public final class EmailService /******************************************************************************* ** Insert using the caller transaction, returning only safe diagnostics. *******************************************************************************/ - private static EmailSendResult insertInTransaction(QRecord row, QBackendTransaction transaction) throws QException + private static EmailSendResult insertInTransaction(QRecord row, QBackendTransaction transaction, EmailTemplateRenderer templateRenderer) throws QException { // A shared provider-row gate prevents a duplicate-key failure from aborting // a caller-owned SQL transaction. The unique index remains the final arbiter. @@ -278,8 +380,53 @@ public final class EmailService { return (refused("Could not queue the email message")); } + try + { + String unsubscribe = row.getValueString("unsubscribeUrl"); + if(unsubscribe == null && (templateRenderer != null || !Boolean.TRUE.equals(row.getValueBoolean("phiApproved")))) + { + unsubscribe = com.kingsrook.qbits.email.render.EmailUnsubscribeLinks.create(inserted.getValueString("id"), + row.getValueString("toAddresses") + "," + row.getValueString("ccAddresses") + "," + row.getValueString("bccAddresses"), row.getValueString("category")); + } + QRecord content; + if(templateRenderer != null) + { + content = templateRenderer.render(inserted.getValueString("id"), unsubscribe, true); + } + else + { + content = new QRecord(); + if(Boolean.TRUE.equals(row.getValueBoolean("phiApproved"))) + { + for(String field : List.of("subject", "htmlBody", "textBody")) + { + content.setValue(field, com.kingsrook.qbits.email.phi.EmailBodyEncryption.rebind(row.getValueString(field), field, + row.getValueString("encryptionSourceId"), inserted.getValueString("id"))); + } + } + if(unsubscribe != null && row.getValueString("unsubscribeUrl") == null) + { + String link = "

Unsubscribe

"; + String html = row.getValueString("htmlBody"); + java.util.regex.Matcher end = java.util.regex.Pattern.compile("|", java.util.regex.Pattern.CASE_INSENSITIVE).matcher(html); + content.withValue("htmlBody", end.find() ? html.substring(0, end.start()) + link + html.substring(end.start()) : html + link) + .withValue("textBody", row.getValueString("textBody") + "\nUnsubscribe (" + unsubscribe + ")"); + } + } + content.withValue("id", inserted.getValue("id")).withValue("unsubscribeUrl", unsubscribe); + if(content.getValueString("htmlBody") != null) + { + content.withValue("bodySha256", digest(new EmailRenderer.Rendered(content.getValueString("htmlBody"), content.getValueString("textBody")))); + } + EmailStore.update(EmailTables.MESSAGE, content, transaction); + } + catch(IllegalArgumentException e) + { + SystemWrites.delete(EmailTables.MESSAGE, EmailStore.equal("id", inserted.getValueInteger("id")), transaction); + return (refused(e.getMessage())); + } return (new EmailSendResult(inserted.getValueInteger("id"), "SUPPRESSED".equals(row.getValueString("status")) - ? EmailSendResult.Status.SUPPRESSED : EmailSendResult.Status.QUEUED, List.of())); + ? EmailSendResult.Status.SUPPRESSED : EmailSendResult.Status.QUEUED, templateRenderer == null ? List.of() : templateRenderer.warnings())); } /******************************************************************************* diff --git a/src/main/java/com/kingsrook/qbits/email/phi/EmailBodyEncryption.java b/src/main/java/com/kingsrook/qbits/email/phi/EmailBodyEncryption.java new file mode 100644 index 0000000..d6c0ed3 --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/email/phi/EmailBodyEncryption.java @@ -0,0 +1,137 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.email.phi; + + +import javax.crypto.Cipher; +import javax.crypto.spec.GCMParameterSpec; +import javax.crypto.spec.SecretKeySpec; +import java.nio.ByteBuffer; +import java.nio.charset.StandardCharsets; +import java.security.MessageDigest; +import java.security.SecureRandom; +import java.util.Base64; +import com.kingsrook.qbits.email.EmailRuntime; + + +/******************************************************************************* + ** Stateless AES-256-GCM body protection. Only the dispatcher decrypts content. + *******************************************************************************/ +public final class EmailBodyEncryption +{ + /******************************************************************************* + ** Utility class. + *******************************************************************************/ + private EmailBodyEncryption() + { + } + + /******************************************************************************* + ** Resolve the deployment key, never retaining it between operations. + *******************************************************************************/ + private static SecretKeySpec key() throws Exception + { + String secret = EmailRuntime.config().getSecretResolver().apply("QQQ_EMAIL_BODY_KEY"); + if(secret == null || secret.isBlank()) + { + throw (new IllegalArgumentException("PHI email needs QQQ_EMAIL_BODY_KEY")); + } + return (new SecretKeySpec(MessageDigest.getInstance("SHA-256").digest(secret.getBytes(StandardCharsets.UTF_8)), "AES")); + } + + /******************************************************************************* + ** Encrypt with a fresh 96-bit nonce and 128-bit authentication tag. + *******************************************************************************/ + public static String encrypt(String value, String column, String messageId) + { + try + { + SecretKeySpec key = key(); + if(value == null) + { + return (null); + } + byte[] iv = new byte[12]; + new SecureRandom().nextBytes(iv); + Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding"); + cipher.init(Cipher.ENCRYPT_MODE, key, new GCMParameterSpec(128, iv)); + cipher.updateAAD(aad(column, messageId)); + byte[] encrypted = cipher.doFinal(value.getBytes(StandardCharsets.UTF_8)); + return ("enc:v1:" + Base64.getUrlEncoder().withoutPadding().encodeToString(ByteBuffer.allocate(iv.length + encrypted.length).put(iv).put(encrypted).array())); + } + catch(Exception e) + { + throw (new IllegalArgumentException("PHI email needs QQQ_EMAIL_BODY_KEY")); + } + } + + /******************************************************************************* + ** Authenticate before exposing plaintext. Failures never echo ciphertext. + *******************************************************************************/ + public static String decrypt(String value, String column, String messageId) + { + if(value == null || !value.startsWith("enc:v1:")) + { + return (value); + } + try + { + ByteBuffer bytes = ByteBuffer.wrap(Base64.getUrlDecoder().decode(value.substring(7))); + byte[] iv = new byte[12]; + bytes.get(iv); + byte[] encrypted = new byte[bytes.remaining()]; + bytes.get(encrypted); + Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding"); + cipher.init(Cipher.DECRYPT_MODE, key(), new GCMParameterSpec(128, iv)); + cipher.updateAAD(aad(column, messageId)); + return (new String(cipher.doFinal(encrypted), StandardCharsets.UTF_8)); + } + catch(Exception e) + { + throw (new IllegalArgumentException("Encrypted email body could not be opened")); + } + } + + /******************************************************************************* + ** Bind ciphertext to its storage table, column and message identity. + *******************************************************************************/ + private static byte[] aad(String column, String messageId) + { + String storageColumn = switch(column) + { + case "subject" -> "subject"; + case "htmlBody" -> "html_body"; + case "textBody" -> "text_body"; + default -> throw (new IllegalArgumentException("Unknown protected email column")); + }; + return (("email_message\n" + storageColumn + "\n" + java.util.Objects.requireNonNull(messageId)).getBytes(StandardCharsets.UTF_8)); + } + + /******************************************************************************* + ** Authenticate and rebind retained ciphertext without exposing plaintext to + ** the resend caller. Only delivery callers receive decrypted content. + *******************************************************************************/ + public static String rebind(String value, String column, String oldMessageId, String newMessageId) + { + return (encrypt(decrypt(value, column, oldMessageId), column, newMessageId)); + } + +} diff --git a/src/main/java/com/kingsrook/qbits/email/phi/EmailClassifications.java b/src/main/java/com/kingsrook/qbits/email/phi/EmailClassifications.java new file mode 100644 index 0000000..3fe2c34 --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/email/phi/EmailClassifications.java @@ -0,0 +1,109 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.email.phi; + + +import com.kingsrook.qbits.email.EmailQBitConfig; +import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData; + + +/******************************************************************************* + ** STAND-IN(core=#1142, removeBy=RC.7). Core classification wins when available; + ** overrides then schema declarations provide the RC.5 compatibility contract. + *******************************************************************************/ +public final class EmailClassifications +{ + /******************************************************************************* + ** Ordered data sensitivity. + *******************************************************************************/ + public enum Level + { + PUBLIC, CONFIDENTIAL, PII, PHI + } + + /******************************************************************************* + ** Utility class. + *******************************************************************************/ + private EmailClassifications() + { + } + + /******************************************************************************* + ** Resolve a field without a compile-time dependency on unreleased core. + *******************************************************************************/ + public static Level ofField(QTableMetaData table, String field, String declared, EmailQBitConfig config) + { + if(table != null && field != null) + { + try + { + Class core = Class.forName("com.kingsrook.qqq.backend.core.actions.ai.AiDataClassifications"); + Object value = core.getMethod("ofField", QTableMetaData.class, String.class).invoke(null, table, field); + if(value instanceof Enum level) + { + return (Level.valueOf(level.name())); + } + } + catch(ClassNotFoundException e) + { + // Expected on RC.5; use the explicit stand-in configuration. + } + catch(ReflectiveOperationException | IllegalArgumentException e) + { + throw (new IllegalArgumentException("Field classification is unavailable")); + } + Level override = config.getClassificationOverrides().get(table.getName() + "." + field); + if(override != null) + { + return (override); + } + } + return (declared == null ? Level.PUBLIC : Level.valueOf(declared)); + } + + /******************************************************************************* + ** Schema declarations alone cannot attest a record's classification source. + *******************************************************************************/ + public static boolean hasSource(QTableMetaData table, String field, EmailQBitConfig config) + { + if(config.getClassificationOverrides().containsKey(table.getName() + "." + field)) + { + return (true); + } + try + { + Class core = Class.forName("com.kingsrook.qqq.backend.core.actions.ai.AiDataClassifications"); + return (core.getMethod("ofField", QTableMetaData.class, String.class).invoke(null, table, field) instanceof Enum); + } + catch(ReflectiveOperationException e) + { + return (false); + } + } + + /******************************************************************************* + ** PII has the same email approval requirement as PHI. + *******************************************************************************/ + public static boolean sensitive(Level level) + { + return (level == Level.PII || level == Level.PHI); + } +} diff --git a/src/main/java/com/kingsrook/qbits/email/phi/EmailValueTokens.java b/src/main/java/com/kingsrook/qbits/email/phi/EmailValueTokens.java new file mode 100644 index 0000000..ed806b8 --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/email/phi/EmailValueTokens.java @@ -0,0 +1,149 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.email.phi; + + +import javax.crypto.Mac; +import javax.crypto.spec.SecretKeySpec; +import java.io.Serializable; +import java.math.BigDecimal; +import java.nio.charset.StandardCharsets; +import java.security.MessageDigest; +import java.util.ArrayList; +import java.util.Base64; +import java.util.List; +import com.kingsrook.qbits.email.EmailRuntime; +import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData; +import com.kingsrook.qqq.backend.core.utils.ValueUtils; + + +/******************************************************************************* + ** STAND-IN(core=#1161-impl, removeBy=RC.7). Mirrors ClassifiedValueTokens. + ** Tokens are comparisons, never recoverable snapshots. No secret is retained. + *******************************************************************************/ +public final class EmailValueTokens +{ + /******************************************************************************* + ** Utility class. + *******************************************************************************/ + private EmailValueTokens() + { + } + + /******************************************************************************* + ** Bind to table, field and record identity. TODO core: wipe-rule record + ** fields will produce shred:v1; context values must never use shred:v1. + *******************************************************************************/ + public static String forRecordField(QTableMetaData table, QFieldMetaData field, Serializable pk, Serializable value) + { + return (token(List.of(table.getName(), field.getName(), String.valueOf(pk)), value)); + } + + /******************************************************************************* + ** Non-record low-entropy inputs cannot yield dictionary-testable tokens. + *******************************************************************************/ + public static String forContext(List bindingParts, Serializable value) + { + if(value instanceof Boolean || canonical(value).codePoints().distinct().count() < 4) + { + return (changed()); + } + return (token(bindingParts, value)); + } + + /******************************************************************************* + ** Deliberately non-verifiable marker. + *******************************************************************************/ + public static String changed() + { + return ("changed:v1"); + } + + /******************************************************************************* + ** Verify only a genuine keyed token, with constant-time comparison. + *******************************************************************************/ + public static boolean verifyRecordField(QTableMetaData table, QFieldMetaData field, Serializable pk, Serializable value, String token) + { + return (matches(token, forRecordField(table, field, pk, value))); + } + + /******************************************************************************* + ** Verify a context token, never changed or shred markers. + *******************************************************************************/ + public static boolean verifyContext(List bindingParts, Serializable value, String token) + { + return (matches(token, forContext(bindingParts, value))); + } + + /******************************************************************************* + ** Stable canonical representation shared with future core. + *******************************************************************************/ + public static String canonical(Serializable value) + { + if(value instanceof BigDecimal decimal) + { + return (decimal.stripTrailingZeros().toPlainString()); + } + return (ValueUtils.getValueAsString(value)); + } + + /******************************************************************************* + ** Construct the versioned HMAC without logging key or input. + *******************************************************************************/ + private static String token(List parts, Serializable value) + { + try + { + String secret = EmailRuntime.config().getSecretResolver().apply("QQQ_AUDIT_HMAC_KEY"); + if(secret == null || secret.isBlank()) + { + return (changed()); + } + byte[] key = Base64.getDecoder().decode(secret); + String keyId = EmailRuntime.config().getSecretResolver().apply("QQQ_AUDIT_HMAC_KEY_ID"); + if(keyId == null || keyId.isBlank()) + { + keyId = "env1"; + } + Mac mac = Mac.getInstance("HmacSHA256"); + mac.init(new SecretKeySpec(key, "HmacSHA256")); + List binding = new ArrayList<>(parts); + binding.add(canonical(value)); + return ("hmac:v1:" + keyId + ":" + Base64.getUrlEncoder().withoutPadding() + .encodeToString(mac.doFinal(String.join("\u001f", binding).getBytes(StandardCharsets.UTF_8)))); + } + catch(Exception e) + { + // An unavailable or malformed audit key must not reveal the value. + return (changed()); + } + } + + /******************************************************************************* + ** Compare only HMAC tokens, never sentinel strings. + *******************************************************************************/ + private static boolean matches(String supplied, String computed) + { + return (supplied != null && supplied.startsWith("hmac:v1:") && computed.startsWith("hmac:v1:") + && MessageDigest.isEqual(supplied.getBytes(StandardCharsets.UTF_8), computed.getBytes(StandardCharsets.UTF_8))); + } +} diff --git a/src/main/java/com/kingsrook/qbits/email/processes/EmailProcesses.java b/src/main/java/com/kingsrook/qbits/email/processes/EmailProcesses.java index 0d628fb..2c27e98 100644 --- a/src/main/java/com/kingsrook/qbits/email/processes/EmailProcesses.java +++ b/src/main/java/com/kingsrook/qbits/email/processes/EmailProcesses.java @@ -33,7 +33,6 @@ import com.kingsrook.qbits.email.EmailRuntime; import com.kingsrook.qbits.email.delivery.EmailDispatcher; import com.kingsrook.qbits.email.metadata.EmailTables; import com.kingsrook.qbits.email.outbox.EmailCategory; -import com.kingsrook.qbits.email.outbox.EmailRenderer; import com.kingsrook.qbits.email.outbox.EmailRequest; import com.kingsrook.qbits.email.outbox.EmailSendResult; import com.kingsrook.qbits.email.outbox.EmailService; @@ -76,10 +75,14 @@ import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwith *******************************************************************************/ public final class EmailProcesses { + public static final String NEW_VERSION = "email_new_version"; + public static final String ACTIVATE_VERSION = "email_activate_version"; + public static final String APPROVE_PHI = "email_approve_phi"; public static final String RESEND = "email_resend"; public static final String CANCEL = "email_cancel"; public static final String SEND_NOW = "email_send_now"; public static final String SEND_TEST = "email_send_test"; + public static final String SEND_AD_HOC = "email_send_ad_hoc"; public static final String SEND_TEMPLATED = "email_send_templated"; public static final String DISPATCH = "email_dispatch"; public static final String MAINTAIN = "email_maintain"; @@ -99,8 +102,8 @@ public final class EmailProcesses { return (switch(name) { - case RESEND, CANCEL, SEND_NOW -> EmailPermissions.MANAGE; - case SEND_TEMPLATED -> EmailPermissions.SEND; + case RESEND, CANCEL, SEND_NOW, NEW_VERSION, ACTIVATE_VERSION -> EmailPermissions.MANAGE; + case SEND_TEMPLATED, SEND_AD_HOC -> EmailPermissions.SEND; default -> EmailPermissions.ADMIN; }); } @@ -115,16 +118,24 @@ public final class EmailProcesses .withComponent(new QFrontendComponentMetaData().withType(QComponentType.EDIT_FORM)) .withFormField(new QFieldMetaData("toAddress", QFieldType.STRING).withLabel("To address").withIsRequired(true))); QFunctionInputMetaData inputs = new QFunctionInputMetaData(); + QFunctionInputMetaData adHocInputs = new QFunctionInputMetaData(); for(String field : List.of("templateName", "to", "cc", "bcc", "replyTo", "subject", "markdownBody", "htmlBody", "textBody", "providerName", "brandName", - "category", "variablesJson", "linkedTableName", "linkedRecordId", "idempotencyKey")) + "locale", "variablesJson", "recordVariablesJson", "linkedTableName", "linkedRecordId", "idempotencyKey")) { inputs.withField(new QFieldMetaData(field, QFieldType.STRING)); + adHocInputs.withField(new QFieldMetaData(field, QFieldType.STRING)); } + adHocInputs.withField(new QFieldMetaData("category", QFieldType.STRING)); + QProcessMetaData adHoc = new QProcessMetaData().withName(SEND_AD_HOC).withLabel("Send Ad Hoc Email").withIsHidden(true) + .withPermissionRules(EmailTables.permissionRules()) + .withStep(new QBackendStepMetaData().withName("send").withCode(new QCodeReference(SendStep.class)).withInputData(adHocInputs)); QProcessMetaData send = new QProcessMetaData().withName(SEND_TEMPLATED).withLabel("Send Email").withIsHidden(true) .withPermissionRules(EmailTables.permissionRules()) .withStep(new QBackendStepMetaData().withName("send").withCode(new QCodeReference(SendStep.class)).withInputData(inputs)); return (List.of(action(RESEND, "Resend", EmailTables.MESSAGE), action(CANCEL, "Cancel", EmailTables.MESSAGE), - action(SEND_NOW, "Send Now", EmailTables.MESSAGE), test, send, + action(SEND_NOW, "Send Now", EmailTables.MESSAGE), test, send, adHoc, + action(NEW_VERSION, "New Version", EmailTables.TEMPLATE), action(ACTIVATE_VERSION, "Activate Version", EmailTables.VERSION), + action(APPROVE_PHI, "Approve For PHI", EmailTables.VERSION), scheduled(config, DISPATCH, "Dispatch Email", config.getDispatchIntervalSeconds()), scheduled(config, MAINTAIN, "Maintain Email", 300), scheduled(config, RETENTION, "Email Retention", 86400))); } @@ -164,6 +175,14 @@ public final class EmailProcesses *******************************************************************************/ private static boolean eligible(String process, QRecord row) { + if(NEW_VERSION.equals(process)) + { + return (true); + } + if(ACTIVATE_VERSION.equals(process) || APPROVE_PHI.equals(process)) + { + return ("DRAFT".equals(row.getValueString("status")) || APPROVE_PHI.equals(process) && "ACTIVE".equals(row.getValueString("status"))); + } if(SEND_TEST.equals(process)) { return (Boolean.TRUE.equals(row.getValueBoolean("enabled"))); @@ -188,12 +207,30 @@ public final class EmailProcesses ** Filter rows and explain skipped records. *******************************************************************************/ @Override - public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) + public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException { for(QRecord row : input.getRecords()) { boolean valid = eligible(input.getProcessName(), row); - lines.add(new ProcessSummaryLine(valid ? Status.OK : Status.INFO, 1, valid ? "record will be processed" : "record skipped: status or retained content does not permit this action")); + String diagnostic = valid ? "record will be processed" : "record skipped: status or retained content does not permit this action"; + if(valid && List.of(NEW_VERSION, ACTIVATE_VERSION, APPROVE_PHI).contains(input.getProcessName())) + { + try + { + QRecord template = NEW_VERSION.equals(input.getProcessName()) ? row : EmailStore.get(EmailTables.TEMPLATE, row.getValueInteger("templateId"), null); + com.kingsrook.qbits.email.template.EmailVersions.requireEdit(template, APPROVE_PHI.equals(input.getProcessName())); + if(ACTIVATE_VERSION.equals(input.getProcessName())) + { + com.kingsrook.qbits.email.template.EmailVersions.validate(template, row); + } + } + catch(IllegalArgumentException e) + { + valid = false; + diagnostic = e.getMessage(); + } + } + lines.add(new ProcessSummaryLine(valid ? Status.OK : Status.INFO, 1, diagnostic)); if(valid) { output.getRecords().add(row); @@ -228,12 +265,24 @@ public final class EmailProcesses for(QRecord row : input.getRecords()) { Integer id = row.getValueInteger("id"); - QRecord current = EmailStore.get(SEND_TEST.equals(process) ? EmailTables.PROVIDER : EmailTables.MESSAGE, id, null); + QRecord current = EmailStore.get(NEW_VERSION.equals(process) ? EmailTables.TEMPLATE : List.of(ACTIVATE_VERSION, APPROVE_PHI).contains(process) ? EmailTables.VERSION + : SEND_TEST.equals(process) ? EmailTables.PROVIDER : EmailTables.MESSAGE, id, null); String result = "Skipped"; if(current != null && eligible(process, current)) { result = switch(process) { + case NEW_VERSION -> "Created DRAFT " + com.kingsrook.qbits.email.template.EmailVersions.newVersion(id).getValueInteger("versionNumber"); + case ACTIVATE_VERSION -> + { + com.kingsrook.qbits.email.template.EmailVersions.activate(id); + yield "ACTIVE"; + } + case APPROVE_PHI -> + { + com.kingsrook.qbits.email.template.EmailVersions.approve(id); + yield "Approved for PHI"; + } case SEND_NOW -> new EmailDispatcher().sendNow(id).name(); case CANCEL -> cancel(id) ? "CANCELLED" : "Skipped"; case RESEND -> resend(current).status().name(); @@ -285,29 +334,7 @@ public final class EmailProcesses *******************************************************************************/ private static EmailSendResult resend(QRecord row) throws QException { - QRecord provider = EmailStore.get(EmailTables.PROVIDER, row.getValueInteger("providerId"), null); - EmailRequest request = new EmailRequest().withSubject(row.getValueString("subject")).withHtmlBody(row.getValueString("htmlBody")) - .withTextBody(row.getValueString("textBody")).withFrom(row.getValueString("fromAddress")).withReplyTo(row.getValueString("replyTo")) - .withProviderName(provider.getValueString("name")).withCategory(EmailCategory.valueOf(row.getValueString("category"))) - .withLinkedRecord(EmailTables.MESSAGE, row.getValue("id")).withSender("email", RESEND); - for(String field : List.of("toAddresses", "ccAddresses", "bccAddresses")) - { - String value = row.getValueString(field); - if(value != null && !value.isBlank()) - { - switch(field) - { - case "toAddresses" -> request.withTo(value); - case "ccAddresses" -> request.withCc(value); - default -> request.withBcc(value); - } - } - } - for(var field : EmailRuntime.config().getSecurityFields()) - { - request.getSecurityValues().put(field.getName(), row.getValue(field.getName())); - } - return (EmailService.send(request, ignored -> new EmailRenderer.Rendered(row.getValueString("htmlBody"), row.getValueString("textBody")))); + return (EmailService.resend(row.getValueInteger("id"))); } /******************************************************************************* @@ -355,9 +382,25 @@ public final class EmailProcesses } if(input.getValueString("variablesJson") != null) { - request.withVariable("variablesJson", input.getValueString("variablesJson")); + request.withVariables(com.kingsrook.qbits.email.template.VariableSchema.values(input.getValueString("variablesJson"))); } - if(input.getValueString("category") != null) + if(input.getValueString("locale") != null) + { + request.withLocale(input.getValueString("locale")); + } + if(input.getValueString("recordVariablesJson") != null) + { + var bindings = com.kingsrook.qbits.email.template.VariableSchema.values(input.getValueString("recordVariablesJson")); + for(var binding : bindings.entrySet()) + { + if(!(binding.getValue() instanceof Map ref) || !(ref.get("table") instanceof String table) || !(ref.get("id") instanceof Serializable id)) + { + throw (new QException("Record variables need table and id")); + } + request.withRecordVariable(binding.getKey(), table, id); + } + } + if(request.getTemplateName() == null && input.getValueString("category") != null) { try { diff --git a/src/main/java/com/kingsrook/qbits/email/render/ContrastCheck.java b/src/main/java/com/kingsrook/qbits/email/render/ContrastCheck.java new file mode 100644 index 0000000..a6a3e19 --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/email/render/ContrastCheck.java @@ -0,0 +1,112 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.email.render; + + +import java.util.Locale; +import com.kingsrook.qqq.backend.core.model.data.QRecord; + + +/******************************************************************************* + ** WCAG relative luminance and AA contrast for both email color schemes. + *******************************************************************************/ +public final class ContrastCheck +{ + /******************************************************************************* + ** Utility class. + *******************************************************************************/ + private ContrastCheck() + { + } + + /******************************************************************************* + ** Check every actual foreground/background pair in the branded shell. + *******************************************************************************/ + public static void validate(QRecord brand) + { + check("Text on background", color(brand, "textColor"), color(brand, "backgroundColor"), "light"); + check("Text on background", color(brand, "darkTextColor"), color(brand, "darkBackgroundColor"), "dark"); + check("Button text on primary", color(brand, "buttonTextColor"), color(brand, "primaryColor"), "light"); + check("Button text on primary", color(brand, "darkButtonTextColor"), color(brand, "darkPrimaryColor"), "dark"); + } + + /******************************************************************************* + ** Defaults are neutral and accessible; reject anything except hex colors. + *******************************************************************************/ + public static String color(QRecord brand, String field) + { + String value = brand.getValueString(field); + if(value == null || value.isBlank()) + { + value = switch(field) + { + case "backgroundColor", "buttonTextColor", "darkTextColor" -> "#ffffff"; + case "primaryColor" -> "#2459a6"; + case "darkPrimaryColor" -> color(brand, "primaryColor"); + case "darkButtonTextColor" -> ratio(color(brand, "buttonTextColor"), color(brand, "darkPrimaryColor")) >= 4.5 + ? color(brand, "buttonTextColor") : ratio("#000000", color(brand, "darkPrimaryColor")) >= 4.5 ? "#000000" : "#ffffff"; + default -> "#171717"; + }; + } + if(!value.matches("#[0-9a-fA-F]{6}")) + { + throw (new IllegalArgumentException("Brand colors must use six-digit hex values")); + } + return (value); + } + + /******************************************************************************* + ** Contrast is independent of which color is lighter. + *******************************************************************************/ + public static double ratio(String first, String second) + { + double a = luminance(first); + double b = luminance(second); + return ((Math.max(a, b) + 0.05) / (Math.min(a, b) + 0.05)); + } + + /******************************************************************************* + ** sRGB linearization and weighted luminance. + *******************************************************************************/ + private static double luminance(String color) + { + double[] weights = {0.2126, 0.7152, 0.0722}; + double result = 0; + for(int i = 0; i < 3; i++) + { + double channel = Integer.parseInt(color.substring(1 + i * 2, 3 + i * 2), 16) / 255.0; + result += weights[i] * (channel <= 0.04045 ? channel / 12.92 : Math.pow((channel + 0.055) / 1.055, 2.4)); + } + return (result); + } + + /******************************************************************************* + ** Author-facing refusal explains the failing scheme and threshold. + *******************************************************************************/ + private static void check(String label, String foreground, String background, String scheme) + { + double ratio = ratio(foreground, background); + if(ratio < 4.5) + { + throw (new IllegalArgumentException(String.format(Locale.ROOT, "%s is %.1f:1 in %s mode; it needs 4.5:1", label, ratio, scheme))); + } + } +} diff --git a/src/main/java/com/kingsrook/qbits/email/render/EmailBrands.java b/src/main/java/com/kingsrook/qbits/email/render/EmailBrands.java new file mode 100644 index 0000000..e634983 --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/email/render/EmailBrands.java @@ -0,0 +1,156 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.email.render; + + +import java.util.List; +import com.kingsrook.qbits.email.EmailRuntime; +import com.kingsrook.qbits.email.metadata.EmailTables; +import com.kingsrook.qbits.email.support.EmailStore; +import com.kingsrook.qbits.email.support.SystemWrites; +import com.kingsrook.qbits.email.template.VariableSchema; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.model.data.QRecord; + + +/******************************************************************************* + ** Brand selection shared by sends, activation and previews. + *******************************************************************************/ +public final class EmailBrands +{ + /******************************************************************************* + ** Utility class. + *******************************************************************************/ + private EmailBrands() + { + } + + /******************************************************************************* + ** Explicit request, template, app brand, default brand, then neutral. + *******************************************************************************/ + public static QRecord resolve(String requested, QRecord template) throws QException + { + if(requested != null) + { + List rows = SystemWrites.query(EmailTables.BRAND, EmailStore.equal("name", requested)); + if(rows.isEmpty()) + { + throw (new IllegalArgumentException("Unknown email brand")); + } + return (rows.getFirst()); + } + if(template != null && template.getValueInteger("brandId") != null) + { + QRecord brand = EmailStore.get(EmailTables.BRAND, template.getValueInteger("brandId"), null); + if(brand == null) + { + throw (new IllegalArgumentException("Template brand is missing")); + } + return (brand); + } + List brands = SystemWrites.query(EmailTables.BRAND, EmailStore.equal("appName", EmailRuntime.config().getAppName())); + if(brands.isEmpty()) + { + brands = SystemWrites.query(EmailTables.BRAND, EmailStore.equal("isDefault", true)); + } + if(brands.size() > 1) + { + throw (new IllegalArgumentException("More than one default email brand matches")); + } + return (brands.isEmpty() ? new QRecord().withValue("label", EmailRuntime.config().getAppLabel()).withValue("name", "neutral") : brands.getFirst()); + } + + /******************************************************************************* + ** Refuse unsafe URLs and inaccessible colors before a brand is saved. + *******************************************************************************/ + public static void validate(QRecord brand) + { + ContrastCheck.validate(brand); + for(String field : List.of("logoUrl", "logoDarkUrl", "siteUrl")) + { + String value = brand.getValueString(field); + if(value != null && !value.isBlank()) + { + VariableSchema.url(value); + if(!"siteUrl".equals(field)) + { + https(value); + } + } + } + social(brand); + } + + /******************************************************************************* + ** Parse the small social-link list with strict web URLs. + *******************************************************************************/ + public static List> social(QRecord brand) + { + try + { + String json = brand.getValueString("socialLinks"); + if(json == null || json.isBlank()) + { + return (List.of()); + } + var links = new com.fasterxml.jackson.databind.ObjectMapper().readValue(json, + new com.fasterxml.jackson.core.type.TypeReference>>() {}); + links.forEach(link -> https(link.get("url"))); + return (links); + } + catch(Exception e) + { + throw (new IllegalArgumentException("Social links need labels and https URLs")); + } + } + + /******************************************************************************* + ** Logos and social destinations require HTTPS at the save boundary. + *******************************************************************************/ + private static void https(String value) + { + VariableSchema.url(value); + if(!"https".equalsIgnoreCase(java.net.URI.create(value).getScheme())) + { + throw (new IllegalArgumentException("Brand logo and social URLs must use https")); + } + } + + /******************************************************************************* + ** Approved messages restrict every image to the brand site or an explicit host. + *******************************************************************************/ + public static boolean allowedImage(String value, QRecord brand) + { + try + { + https(value); + String host = java.net.URI.create(value).getHost(); + String site = brand.getValueString("siteUrl"); + return (EmailRuntime.config().getAllowedImageHosts().contains(host.toLowerCase(java.util.Locale.ROOT)) + || site != null && host.equalsIgnoreCase(java.net.URI.create(site).getHost())); + } + catch(IllegalArgumentException e) + { + return (false); + } + } + +} diff --git a/src/main/java/com/kingsrook/qbits/email/render/EmailLayout.java b/src/main/java/com/kingsrook/qbits/email/render/EmailLayout.java new file mode 100644 index 0000000..82b2a73 --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/email/render/EmailLayout.java @@ -0,0 +1,135 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.email.render; + + +import com.kingsrook.qbits.email.outbox.EmailCategory; +import com.kingsrook.qbits.email.template.EmailTemplateEngine; +import com.kingsrook.qbits.email.template.VariableSchema; +import com.kingsrook.qqq.backend.core.model.data.QRecord; + + +/******************************************************************************* + ** Single-column, 600px email shell with inline fallbacks and dark-mode hooks. + *******************************************************************************/ +public final class EmailLayout +{ + /******************************************************************************* + ** Utility class. + *******************************************************************************/ + private EmailLayout() + { + } + + /******************************************************************************* + ** Render both schemes from the same validated brand. + *******************************************************************************/ + public static String render(String body, String preheader, QRecord brand, EmailCategory category, String unsubscribe) + { + EmailBrands.validate(brand); + String dark = ".email-shell,.email-content{background-color:" + ContrastCheck.color(brand, "darkBackgroundColor") + + "!important;color:" + ContrastCheck.color(brand, "darkTextColor") + "!important}.email-button{background-color:" + + ContrastCheck.color(brand, "darkPrimaryColor") + "!important}.email-button a{color:" + ContrastCheck.color(brand, "darkButtonTextColor") + + "!important}.email-logo-light{display:none!important}.email-logo-dark{display:block!important}"; + String outlook = "[data-ogsc] .email-content,[data-ogsc] .email-shell,[data-ogsb] .email-content,[data-ogsb] .email-shell{background-color:" + ContrastCheck.color(brand, "darkBackgroundColor") + + "!important;color:" + ContrastCheck.color(brand, "darkTextColor") + "!important}[data-ogsc] .email-button,[data-ogsb] .email-button{background-color:" + + ContrastCheck.color(brand, "darkPrimaryColor") + "!important}[data-ogsc] .email-button a,[data-ogsb] .email-button a{color:" + ContrastCheck.color(brand, "darkButtonTextColor") + + "!important}[data-ogsc] .email-logo-light,[data-ogsb] .email-logo-light{display:none!important}[data-ogsc] .email-logo-dark,[data-ogsb] .email-logo-dark{display:block!important}"; + return ("" + + "" + + "" + + "
" + EmailTemplateEngine.escape(preheader) + "
" + + "
" + + "" + + "" + + "
" + logos(brand) + + body + footer(brand, category, unsubscribe) + "
"); + } + + /******************************************************************************* + ** Logo alternatives, with dimensions that never force horizontal overflow. + *******************************************************************************/ + private static String logos(QRecord brand) + { + String light = brand.getValueString("logoUrl"); + if(light == null || light.isBlank()) + { + return ("

" + EmailTemplateEngine.escape(brand.getValueString("label")) + "

"); + } + String dark = brand.getValueString("logoDarkUrl"); + return (logo(light, brand.getValueString("logoAlt"), dark == null ? "" : "email-logo-light", "block") + + (dark == null ? "" : logo(dark, brand.getValueString("logoAlt"), "email-logo-dark", "none"))); + } + + /******************************************************************************* + ** Escaped logo element. + *******************************************************************************/ + private static String logo(String url, String alt, String css, String display) + { + return ("\"""); + } + + /******************************************************************************* + ** Category-specific footer, including postal address for marketing. + *******************************************************************************/ + public static String footer(QRecord brand, EmailCategory category, String unsubscribe) + { + StringBuilder footer = new StringBuilder("
"); + footer.append(EmailMarkdown.render(brand.getValueString("footerMarkdown"), brand)); + if(category == EmailCategory.MARKETING) + { + String address = brand.getValueString("postalAddress"); + if(address == null || address.isBlank()) + { + throw (new IllegalArgumentException("Marketing email needs a brand postal address")); + } + footer.append("

").append(EmailTemplateEngine.escape(address)).append("

"); + } + for(var link : EmailBrands.social(brand)) + { + footer.append("") + .append(EmailTemplateEngine.escape(link.get("label"))).append(" "); + } + if((category == EmailCategory.NOTIFICATION || category == EmailCategory.MARKETING) && unsubscribe != null && !unsubscribe.isBlank()) + { + footer.append("

Unsubscribe

"); + } + return (footer.append("
").toString()); + } + + /******************************************************************************* + ** HTML and compiled MJML retain their shell; append opt-out before body end. + *******************************************************************************/ + public static String appendFooter(String html, QRecord brand, EmailCategory category, String unsubscribe) + { + if(category != EmailCategory.NOTIFICATION && category != EmailCategory.MARKETING) + { + return (html); + } + String footer = footer(brand, category, unsubscribe); + java.util.regex.Matcher end = java.util.regex.Pattern.compile("", java.util.regex.Pattern.CASE_INSENSITIVE).matcher(html); + return (end.find() ? html.substring(0, end.start()) + footer + html.substring(end.start()) : html + footer); + } +} diff --git a/src/main/java/com/kingsrook/qbits/email/render/EmailMarkdown.java b/src/main/java/com/kingsrook/qbits/email/render/EmailMarkdown.java new file mode 100644 index 0000000..8e708bb --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/email/render/EmailMarkdown.java @@ -0,0 +1,71 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.email.render; + + +import java.util.List; +import java.util.regex.Matcher; +import java.util.regex.Pattern; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import org.commonmark.ext.autolink.AutolinkExtension; +import org.commonmark.ext.gfm.tables.TablesExtension; +import org.commonmark.parser.Parser; +import org.commonmark.renderer.html.HtmlRenderer; + + +/******************************************************************************* + ** CommonMark with GFM tables, autolinks and a constrained button attribute. + *******************************************************************************/ +public final class EmailMarkdown +{ + /******************************************************************************* + ** Utility class. + *******************************************************************************/ + private EmailMarkdown() + { + } + + /******************************************************************************* + ** Raw HTML is escaped and URLs sanitized before adding table buttons. + *******************************************************************************/ + public static String render(String markdown, QRecord brand) + { + var extensions = List.of(TablesExtension.create(), AutolinkExtension.create()); + String html = HtmlRenderer.builder().escapeHtml(true).sanitizeUrls(true).extensions(extensions).build() + .render(Parser.builder().extensions(extensions).build().parse(markdown == null ? "" : markdown)) + .replace("]* href=\"([^\"]*)\"[^>]*>((?:(?!\\{\\.button}").matcher(html); + StringBuilder result = new StringBuilder(); + while(links.find()) + { + String button = "
" + links.group(2) + "
"; + links.appendReplacement(result, Matcher.quoteReplacement(button)); + } + links.appendTail(result); + return (result.toString().replace("

", "").replace("", "
") + .replace("
", "") + .replace("", "") + .replace("", "
"));
+   }
+}
diff --git a/src/main/java/com/kingsrook/qbits/email/render/EmailUnsubscribeLinks.java b/src/main/java/com/kingsrook/qbits/email/render/EmailUnsubscribeLinks.java
new file mode 100644
index 0000000..983b704
--- /dev/null
+++ b/src/main/java/com/kingsrook/qbits/email/render/EmailUnsubscribeLinks.java
@@ -0,0 +1,83 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.email.render;
+
+
+import javax.crypto.Mac;
+import javax.crypto.spec.SecretKeySpec;
+import java.nio.charset.StandardCharsets;
+import java.security.MessageDigest;
+import java.util.Base64;
+import java.util.HexFormat;
+import java.util.Map;
+import com.kingsrook.qbits.email.EmailRuntime;
+import com.kingsrook.qbits.email.template.VariableSchema;
+
+
+/*******************************************************************************
+ ** Queue-time link signing from DESIGN 8.2. The receiving route is phase (c).
+ *******************************************************************************/
+public final class EmailUnsubscribeLinks
+{
+   /*******************************************************************************
+    ** Utility class.
+    *******************************************************************************/
+   private EmailUnsubscribeLinks()
+   {
+   }
+
+   /*******************************************************************************
+    ** Bind the message, recipient-envelope hash and category without exposing PII.
+    ** No key means no link/header; marketing must refuse rather than omit opt-out.
+    *******************************************************************************/
+   public static String create(String messageId, String recipients, String category)
+   {
+      if(!"NOTIFICATION".equals(category) && !"MARKETING".equals(category))
+      {
+         return (null);
+      }
+      try
+      {
+         String key = EmailRuntime.config().getSecretResolver().apply("QQQ_EMAIL_UNSUBSCRIBE_KEY");
+         String base = EmailRuntime.config().getPublicBaseUrl();
+         if(key == null || key.isBlank() || base == null || base.isBlank())
+         {
+            if("MARKETING".equals(category))
+            {
+               throw (new IllegalArgumentException());
+            }
+            return (null);
+         }
+         VariableSchema.url(base);
+         String hash = HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(recipients.getBytes(StandardCharsets.UTF_8)));
+         String payload = Base64.getUrlEncoder().withoutPadding().encodeToString(VariableSchema.json(Map.of("messageId", messageId, "addressHash", hash, "category", category)).getBytes(StandardCharsets.UTF_8));
+         Mac mac = Mac.getInstance("HmacSHA256");
+         mac.init(new SecretKeySpec(key.getBytes(StandardCharsets.UTF_8), "HmacSHA256"));
+         String signed = "v1." + payload;
+         String signature = Base64.getUrlEncoder().withoutPadding().encodeToString(mac.doFinal(signed.getBytes(StandardCharsets.UTF_8)));
+         return (base.replaceAll("/+$", "") + "/qqq/hooks/email/unsubscribe/" + signed + "." + signature);
+      }
+      catch(Exception e)
+      {
+         throw (new IllegalArgumentException("Opt-out email needs publicBaseUrl and QQQ_EMAIL_UNSUBSCRIBE_KEY"));
+      }
+   }
+}
diff --git a/src/main/java/com/kingsrook/qbits/email/render/HtmlToText.java b/src/main/java/com/kingsrook/qbits/email/render/HtmlToText.java
new file mode 100644
index 0000000..d0c9a34
--- /dev/null
+++ b/src/main/java/com/kingsrook/qbits/email/render/HtmlToText.java
@@ -0,0 +1,132 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.email.render;
+
+
+import javax.swing.text.MutableAttributeSet;
+import javax.swing.text.html.HTML;
+import javax.swing.text.html.HTMLEditorKit;
+import javax.swing.text.html.parser.ParserDelegator;
+import java.io.StringReader;
+
+
+/*******************************************************************************
+ ** HTML to readable text using the JDK parser (including entity decoding).
+ *******************************************************************************/
+public final class HtmlToText
+{
+   /*******************************************************************************
+    ** Utility class.
+    *******************************************************************************/
+   private HtmlToText()
+   {
+   }
+
+   /*******************************************************************************
+    ** Keep links, headings, lists and table rows; omit scripts and styles.
+    *******************************************************************************/
+   public static String convert(String html)
+   {
+      StringBuilder text = new StringBuilder();
+      try
+      {
+         new ParserDelegator().parse(new StringReader(html == null ? "" : html), new HTMLEditorKit.ParserCallback()
+         {
+            private int hidden;
+            private String href;
+
+            /*******************************************************************************
+             ** Start readable block boundaries and remember link destinations.
+             *******************************************************************************/
+            @Override
+            public void handleStartTag(HTML.Tag tag, MutableAttributeSet attributes, int position)
+            {
+               if(tag == HTML.Tag.STYLE || tag == HTML.Tag.SCRIPT || tag == HTML.Tag.HEAD)
+               {
+                  hidden++;
+               }
+               if(tag == HTML.Tag.A)
+               {
+                  href = (String) attributes.getAttribute(HTML.Attribute.HREF);
+               }
+               if(tag == HTML.Tag.LI)
+               {
+                  text.append("\n- ");
+               }
+               if(tag == HTML.Tag.TD || tag == HTML.Tag.TH)
+               {
+                  text.append(" ");
+               }
+            }
+
+            /*******************************************************************************
+             ** Add link destinations and block separators.
+             *******************************************************************************/
+            @Override
+            public void handleEndTag(HTML.Tag tag, int position)
+            {
+               if(tag == HTML.Tag.STYLE || tag == HTML.Tag.SCRIPT || tag == HTML.Tag.HEAD)
+               {
+                  hidden = Math.max(0, hidden - 1);
+               }
+               if(tag == HTML.Tag.A && href != null)
+               {
+                  text.append(" (").append(href).append(')');
+                  href = null;
+               }
+               if(tag.isBlock())
+               {
+                  text.append('\n');
+               }
+            }
+
+            /*******************************************************************************
+             ** Line breaks remain line breaks.
+             *******************************************************************************/
+            @Override
+            public void handleSimpleTag(HTML.Tag tag, MutableAttributeSet attributes, int position)
+            {
+               if(tag == HTML.Tag.BR)
+               {
+                  text.append('\n');
+               }
+            }
+
+            /*******************************************************************************
+             ** Parser-decoded text, excluding executable and metadata content.
+             *******************************************************************************/
+            @Override
+            public void handleText(char[] data, int position)
+            {
+               if(hidden == 0)
+               {
+                  text.append(data);
+               }
+            }
+         }, true);
+      }
+      catch(java.io.IOException e)
+      {
+         throw (new IllegalArgumentException("Could not convert email HTML"));
+      }
+      return (text.toString().replaceAll("[ \\t]+", " ").replaceAll(" *\\n *", "\n").replaceAll("\\n{3,}", "\n\n").strip());
+   }
+}
diff --git a/src/main/java/com/kingsrook/qbits/email/support/SystemWrites.java b/src/main/java/com/kingsrook/qbits/email/support/SystemWrites.java
index 1744b85..0fbc02d 100644
--- a/src/main/java/com/kingsrook/qbits/email/support/SystemWrites.java
+++ b/src/main/java/com/kingsrook/qbits/email/support/SystemWrites.java
@@ -152,7 +152,7 @@ public final class SystemWrites
     *******************************************************************************/
    public static List query(String table, QQueryFilter filter) throws QException
    {
-      return (call(() -> new QueryAction().execute(new QueryInput(table).withFilter(filter)).getRecords()));
+      return (call(() -> new QueryAction().execute(new QueryInput(table).withShouldFetchHeavyFields(true).withShouldOmitHiddenFields(false).withFilter(filter)).getRecords()));
    }
 
 
@@ -162,7 +162,7 @@ public final class SystemWrites
     *******************************************************************************/
    public static List query(String table, QQueryFilter filter, QBackendTransaction transaction) throws QException
    {
-      return (call(() -> new QueryAction().execute(new QueryInput(table).withFilter(filter).withTransaction(transaction)).getRecords()));
+      return (call(() -> new QueryAction().execute(new QueryInput(table).withShouldFetchHeavyFields(true).withShouldOmitHiddenFields(false).withFilter(filter).withTransaction(transaction)).getRecords()));
    }
 
 
diff --git a/src/main/java/com/kingsrook/qbits/email/template/CliMjmlCompiler.java b/src/main/java/com/kingsrook/qbits/email/template/CliMjmlCompiler.java
new file mode 100644
index 0000000..d133a5b
--- /dev/null
+++ b/src/main/java/com/kingsrook/qbits/email/template/CliMjmlCompiler.java
@@ -0,0 +1,133 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.email.template;
+
+
+import java.nio.charset.StandardCharsets;
+import java.util.concurrent.Executors;
+import java.util.concurrent.TimeUnit;
+
+
+/*******************************************************************************
+ ** MJML stdin/stdout compiler with a total 20-second deadline and no files.
+ *******************************************************************************/
+public class CliMjmlCompiler implements MjmlCompilerInterface
+{
+   private final String binary;
+
+   /*******************************************************************************
+    ** Explicit executable path; no shell interpolation or PATH guessing.
+    *******************************************************************************/
+   public CliMjmlCompiler(String binary)
+   {
+      this.binary = binary;
+   }
+
+   /*******************************************************************************
+    ** Drain stdout concurrently with stdin, discarding potentially private stderr.
+    *******************************************************************************/
+   @Override
+   public String compile(String source)
+   {
+      rejectIncludes(source);
+      Process process = null;
+      try(var executor = Executors.newVirtualThreadPerTaskExecutor())
+      {
+         process = processBuilder().start();
+         Process running = process;
+         var task = executor.submit(() ->
+         {
+            var writer = executor.submit(() ->
+            {
+               try(var stdin = running.getOutputStream())
+               {
+                  stdin.write(source.getBytes(StandardCharsets.UTF_8));
+               }
+               return (null);
+            });
+            byte[] output = running.getInputStream().readNBytes(10_000_001);
+            if(output.length > 10_000_000)
+            {
+               throw (new IllegalArgumentException("MJML output too large"));
+            }
+            writer.get();
+            if(running.waitFor() != 0)
+            {
+               throw (new IllegalArgumentException("MJML compilation failed"));
+            }
+            return (new String(output, StandardCharsets.UTF_8));
+         });
+         try
+         {
+            return (task.get(20, TimeUnit.SECONDS));
+         }
+         finally
+         {
+            running.descendants().forEach(ProcessHandle::destroyForcibly);
+            running.destroyForcibly();
+            task.cancel(true);
+         }
+      }
+      catch(Exception e)
+      {
+         if(e instanceof InterruptedException)
+         {
+            Thread.currentThread().interrupt();
+         }
+         throw (new IllegalArgumentException("MJML compilation failed or exceeded 20 seconds"));
+      }
+      finally
+      {
+         if(process != null)
+         {
+            process.destroyForcibly();
+         }
+      }
+   }
+
+   /*******************************************************************************
+    ** MJML 4.x/5 supports allowIncludes=false. Refusing the source tag as well
+    ** removes include resolution and its dependence on the inherited directory.
+    *******************************************************************************/
+   public static void rejectIncludes(String source)
+   {
+      if(source != null && java.util.regex.Pattern.compile("(?i)<\\s*mj-include\\b").matcher(source).find())
+      {
+         throw (new IllegalArgumentException("MJML includes are not allowed"));
+      }
+   }
+
+   /*******************************************************************************
+    ** Package seam for checking child isolation without executing a compiler.
+    *******************************************************************************/
+   ProcessBuilder processBuilder()
+   {
+      ProcessBuilder builder = new ProcessBuilder(binary, "-i", "-s", "--config.allowIncludes=false");
+      String path = builder.environment().get("PATH");
+      builder.environment().clear();
+      if(path != null)
+      {
+         builder.environment().put("PATH", path);
+      }
+      return (builder.redirectError(ProcessBuilder.Redirect.DISCARD));
+   }
+
+}
diff --git a/src/main/java/com/kingsrook/qbits/email/template/EmailBuiltIns.java b/src/main/java/com/kingsrook/qbits/email/template/EmailBuiltIns.java
new file mode 100644
index 0000000..49cc67e
--- /dev/null
+++ b/src/main/java/com/kingsrook/qbits/email/template/EmailBuiltIns.java
@@ -0,0 +1,77 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.email.template;
+
+
+import java.util.List;
+import com.kingsrook.qbits.email.metadata.EmailTables;
+import com.kingsrook.qbits.email.support.EmailStore;
+import com.kingsrook.qbits.email.support.SystemWrites;
+import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+
+
+/*******************************************************************************
+ ** Built-ins are created on first use, transactionally and without JVM state.
+ *******************************************************************************/
+public final class EmailBuiltIns
+{
+   /*******************************************************************************
+    ** Utility class.
+    *******************************************************************************/
+   private EmailBuiltIns()
+   {
+   }
+
+   /*******************************************************************************
+    ** Seed the secure-link pattern. The unique template name arbitrates replicas.
+    *******************************************************************************/
+   public static void ensure() throws QException
+   {
+      if(!SystemWrites.query(EmailTables.TEMPLATE, EmailStore.equal("name", "email_secure_message")).isEmpty())
+      {
+         return;
+      }
+      try(QBackendTransaction tx = QBackendTransaction.openFor(new InsertInput(EmailTables.TEMPLATE)))
+      {
+         QRecord template = SystemWrites.insert(EmailTables.TEMPLATE, List.of(new QRecord().withValue("name", "email_secure_message")
+            .withValue("label", "Secure message").withValue("kind", "MARKDOWN").withValue("category", "SECURITY").withValue("phiApproved", false)), tx).getRecords().getFirst();
+         if(template.getValue("id") == null || template.getErrors() != null && !template.getErrors().isEmpty())
+         {
+            tx.rollback();
+            return;
+         }
+         QRecord version = SystemWrites.insert(EmailTables.VERSION, List.of(new QRecord().withValue("templateId", template.getValue("id"))
+            .withValue("versionNumber", 1).withValue("status", "ACTIVE").withValue("locale", "en").withValue("subject", "New secure message")
+            .withValue("bodySource", "You have a new secure message from {{brand.label}}. Sign in to read it.\n\n[Read secure message]({{link}}){.button}")
+            .withValue("variablesSchema", "[{\"name\":\"link\",\"type\":\"URL\",\"required\":true}]")
+            .withValue("sampleVariables", "{\"link\":\"https://example.invalid/sign-in\"}")), tx).getRecords().getFirst();
+         if(version.getValue("id") == null || version.getErrors() != null && !version.getErrors().isEmpty())
+         {
+            throw (new QException("Could not install secure-message template"));
+         }
+         EmailStore.update(EmailTables.TEMPLATE, new QRecord().withValue("id", template.getValue("id")).withValue("activeVersionId", version.getValue("id")), tx);
+         tx.commit();
+      }
+   }
+}
diff --git a/src/main/java/com/kingsrook/qbits/email/template/EmailPreviewWidget.java b/src/main/java/com/kingsrook/qbits/email/template/EmailPreviewWidget.java
new file mode 100644
index 0000000..ba41fe2
--- /dev/null
+++ b/src/main/java/com/kingsrook/qbits/email/template/EmailPreviewWidget.java
@@ -0,0 +1,130 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.email.template;
+
+
+import java.util.List;
+import com.kingsrook.qbits.email.metadata.EmailTables;
+import com.kingsrook.qbits.email.outbox.EmailRequest;
+import com.kingsrook.qbits.email.render.ContrastCheck;
+import com.kingsrook.qbits.email.render.EmailBrands;
+import com.kingsrook.qbits.email.support.EmailStore;
+import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
+import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
+import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
+import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.RawHTML;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import org.jsoup.Jsoup;
+import org.jsoup.safety.Safelist;
+
+
+/*******************************************************************************
+ ** Inline preview fallback: Next strips iframes and style blocks. Samples are
+ ** sanitized server-side too; no live record references are read for previews.
+ *******************************************************************************/
+public class EmailPreviewWidget extends AbstractWidgetRenderer
+{
+   /*******************************************************************************
+    ** Fetch the version and parent as the viewer, then render sample data.
+    *******************************************************************************/
+   @Override
+   public RenderWidgetOutput render(RenderWidgetInput input) throws QException
+   {
+      try
+      {
+         int id = Integer.parseInt(input.getQueryParams().get("id"));
+         List versions = new QueryAction().execute(new QueryInput(EmailTables.VERSION).withShouldFetchHeavyFields(true)
+            .withFilter(EmailStore.equal("id", id))).getRecords();
+         if(versions.isEmpty())
+         {
+            return (output("
Version unavailable.
"));
+         }
+         QRecord version = versions.getFirst();
+         List templates = new QueryAction().execute(new QueryInput(EmailTables.TEMPLATE)
+            .withFilter(EmailStore.equal("id", version.getValueInteger("templateId")))).getRecords();
+         if(templates.isEmpty())
+         {
+            return (output("
Template unavailable.
"));
+         }
+         QRecord template = templates.getFirst();
+         if("MJML".equals(template.getValueString("kind")) && version.getValueString("compiledHtml") == null)
+         {
+            if(!com.kingsrook.qbits.email.permissions.EmailPermissionChecker.has(com.kingsrook.qbits.email.permissions.EmailPermissions.MANAGE))
+            {
+               return (output("
Not compiled yet. Activate the version or use Preview as a manager.
"));
+            }
+            EmailVersions.validate(template, version);
+         }
+         EmailTemplateRenderer renderer = new EmailTemplateRenderer(template, version,
+            new EmailRequest().withVariables(VariableSchema.values(version.getValueString("sampleVariables"))));
+         QRecord rendered = renderer.render("preview", "https://example.invalid/unsubscribe/preview", false);
+         QRecord brand = EmailBrands.resolve(null, template);
+         String html = "
" + EmailTemplateEngine.escape(rendered.getValueString("subject")) + "
"
+            + "
Sample preview. Inline rendering; mail clients may display it differently.
"
+            + sample(rendered.getValueString("htmlBody"), brand, false, 390) + sample(rendered.getValueString("htmlBody"), brand, true, 600)
+            + "
Plain text
"
+            + EmailTemplateEngine.escape(rendered.getValueString("textBody")) + "
";
+         return (output(html + "
" + EmailTemplateEngine.escape(String.join("; ", renderer.warnings())) + "
"));
+      }
+      catch(IllegalArgumentException e)
+      {
+         return (output("
Preview unavailable: " + EmailTemplateEngine.escape(e.getMessage()) + "
"));
+      }
+   }
+
+   /*******************************************************************************
+    ** Safe inline sample with bounded width and no executable source or CSS.
+    *******************************************************************************/
+   private static String sample(String html, QRecord brand, boolean dark, int width)
+   {
+      var document = Jsoup.parse(html);
+      document.select("head,script,style,iframe,form").remove();
+      document.select(".email-logo-" + (dark ? "light" : "dark")).remove();
+      String clean = Jsoup.clean(document.body().html(), Safelist.relaxed().addAttributes("td", "class"));
+      document = Jsoup.parseBodyFragment(clean);
+      document.select("table").forEach(e -> e.attr("style", "width:100%;table-layout:fixed;border-collapse:collapse"));
+      document.select("td,th").forEach(e -> e.attr("style", "overflow-wrap:anywhere;padding:6px"));
+      document.select("img").forEach(e -> e.attr("style", "max-width:100%;height:auto"));
+      document.select("pre").forEach(e -> e.attr("style", "white-space:pre-wrap;overflow-wrap:anywhere"));
+      document.select("a").forEach(e -> e.attr("style", "color:inherit;overflow-wrap:anywhere"));
+      document.select(".email-content").forEach(e -> e.attr("style", e.attr("style") + ";background-color:"
+         + ContrastCheck.color(brand, dark ? "darkBackgroundColor" : "backgroundColor") + ";color:"
+         + ContrastCheck.color(brand, dark ? "darkTextColor" : "textColor")));
+      document.select(".email-button").forEach(e -> e.attr("style", "padding:12px 20px;background-color:" + ContrastCheck.color(brand, dark ? "darkPrimaryColor" : "primaryColor")
+         + ";color:" + ContrastCheck.color(brand, dark ? "darkButtonTextColor" : "buttonTextColor") + "!important"));
+      document.select(".email-button a").forEach(e -> e.attr("style", "color:" + ContrastCheck.color(brand, dark ? "darkButtonTextColor" : "buttonTextColor") + "!important;display:block;overflow-wrap:anywhere"));
+      document.select("[class]").forEach(e -> e.removeAttr("class"));
+      return ("
" + (dark ? "Dark" : "Light") + " · " + width + "px maximum
" + document.body().html() + "
");
+   }
+
+   /*******************************************************************************
+    ** Standard widget response.
+    *******************************************************************************/
+   private static RenderWidgetOutput output(String html)
+   {
+      return (new RenderWidgetOutput(new RawHTML(null, html)));
+   }
+}
diff --git a/src/main/java/com/kingsrook/qbits/email/template/EmailTemplateCustomizers.java b/src/main/java/com/kingsrook/qbits/email/template/EmailTemplateCustomizers.java
new file mode 100644
index 0000000..6498466
--- /dev/null
+++ b/src/main/java/com/kingsrook/qbits/email/template/EmailTemplateCustomizers.java
@@ -0,0 +1,172 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.email.template;
+
+
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import com.kingsrook.qbits.email.metadata.EmailTables;
+import com.kingsrook.qbits.email.render.EmailBrands;
+import com.kingsrook.qbits.email.support.EmailStore;
+import com.kingsrook.qbits.email.support.SystemWrites;
+import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.statusmessages.BadInputStatusMessage;
+
+
+/*******************************************************************************
+ ** Lifecycle and template edit permissions apply equally to forms and APIs.
+ *******************************************************************************/
+public class EmailTemplateCustomizers implements TableCustomizerInterface
+{
+   /*******************************************************************************
+    ** Merge sparse updates and protect stored lifecycle state, not caller claims.
+    *******************************************************************************/
+   @Override
+   public List preInsertOrUpdate(AbstractActionInput input, List rows, boolean preview, Optional> oldRecords) throws QException
+   {
+      if(SystemWrites.active())
+      {
+         return (rows);
+      }
+      for(QRecord row : rows)
+      {
+         QRecord old = oldRecords.orElse(List.of()).stream().filter(v -> Objects.equals(v.getValue("id"), row.getValue("id"))).findFirst().orElse(new QRecord());
+         QRecord merged = new QRecord();
+         merged.getValues().putAll(old.getValues());
+         merged.getValues().putAll(row.getValues());
+         try
+         {
+            String table = input instanceof com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput insert ? insert.getTableName()
+               : ((com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput) input).getTableName();
+            if(EmailTables.BRAND.equals(table))
+            {
+               EmailBrands.validate(merged);
+            }
+            else if(EmailTables.TEMPLATE.equals(table))
+            {
+               EmailVersions.requireEdit(old.getValue("id") == null ? merged : old, false);
+               protectedFields(row, old, List.of("phiApproved", "approvedVersionId", "phiApprovedBy", "phiApprovedDate", "activeVersionId"));
+               if(Boolean.TRUE.equals(old.getValueBoolean("phiApproved")))
+               {
+                  protectedFields(row, old, List.of("kind", "category", "brandId", "name"));
+               }
+            }
+            else
+            {
+               QRecord parent = EmailStore.get(EmailTables.TEMPLATE, merged.getValueInteger("templateId"), null);
+               if(parent == null)
+               {
+                  throw (new IllegalArgumentException("Template is required"));
+               }
+               EmailVersions.requireEdit(parent, false);
+               if(old.getValue("id") != null)
+               {
+                  protectedFields(row, old, List.of("templateId", "versionNumber", "status", "compiledHtml", "activatedDate"));
+                  immutable(old, parent);
+               }
+               else
+               {
+                  if(merged.getValueString("status") != null && !"DRAFT".equals(merged.getValueString("status")))
+                  {
+                     throw (new IllegalArgumentException("New versions must be DRAFT"));
+                  }
+                  row.setValue("status", "DRAFT");
+                  row.setValue("compiledHtml", null);
+               }
+               new VariableSchema(merged.getValueString("variablesSchema"));
+            }
+         }
+         catch(IllegalArgumentException e)
+         {
+            row.addError(new BadInputStatusMessage(e.getMessage()));
+         }
+      }
+      return (rows);
+   }
+
+   /*******************************************************************************
+    ** Lifecycle-owned fields cannot be changed through ordinary CRUD.
+    *******************************************************************************/
+   private static void protectedFields(QRecord row, QRecord old, List fields)
+   {
+      for(String field : fields)
+      {
+         Object value = row.getValue(field);
+         if(row.getValues().containsKey(field) && !Objects.equals(value, old.getValue(field)) && !(old.getValue("id") == null && Boolean.FALSE.equals(value)))
+         {
+            throw (new IllegalArgumentException("Use the version actions to change " + field));
+         }
+      }
+   }
+
+   /*******************************************************************************
+    ** Approving a draft freezes its content, preventing approval substitution.
+    *******************************************************************************/
+   private static void immutable(QRecord version, QRecord template)
+   {
+      if(!"DRAFT".equals(version.getValueString("status")) || Objects.equals(version.getValue("id"), template.getValue("approvedVersionId")))
+      {
+         throw (new IllegalArgumentException("ACTIVE, RETIRED and PHI-approved versions cannot be edited; use New Version"));
+      }
+   }
+
+   /*******************************************************************************
+    ** Preserve active/history rows and the same edit grants on deletion.
+    *******************************************************************************/
+   @Override
+   public List preDelete(DeleteInput input, List rows, boolean preview) throws QException
+   {
+      if(SystemWrites.active())
+      {
+         return (rows);
+      }
+      for(QRecord row : rows)
+      {
+         try
+         {
+            if(EmailTables.VERSION.equals(input.getTableName()))
+            {
+               QRecord parent = EmailStore.get(EmailTables.TEMPLATE, row.getValueInteger("templateId"), input.getTransaction());
+               EmailVersions.requireEdit(parent, false);
+               immutable(row, parent);
+            }
+            else if(EmailTables.TEMPLATE.equals(input.getTableName()))
+            {
+               EmailVersions.requireEdit(row, false);
+               if(!SystemWrites.query(EmailTables.VERSION, EmailStore.equal("templateId", row.getValueInteger("id")), input.getTransaction()).isEmpty())
+               {
+                  throw (new IllegalArgumentException("Templates with versions cannot be deleted"));
+               }
+            }
+         }
+         catch(IllegalArgumentException e)
+         {
+            row.addError(new BadInputStatusMessage(e.getMessage()));
+         }
+      }
+      return (rows);
+   }
+}
diff --git a/src/main/java/com/kingsrook/qbits/email/template/EmailTemplateEngine.java b/src/main/java/com/kingsrook/qbits/email/template/EmailTemplateEngine.java
new file mode 100644
index 0000000..de19adf
--- /dev/null
+++ b/src/main/java/com/kingsrook/qbits/email/template/EmailTemplateEngine.java
@@ -0,0 +1,418 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.email.template;
+
+
+import java.math.BigDecimal;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayDeque;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Deque;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.regex.Pattern;
+
+
+/*******************************************************************************
+ ** Small parsed template language: values, fallback, if and each, without code.
+ *******************************************************************************/
+public class EmailTemplateEngine
+{
+   private static final Pattern URL_ATTRIBUTE = Pattern.compile("(?i)(?:^|[\\s<])(href|src|action|background)\\s*=\\s*[\"']$");
+   private static final Pattern RENDERED_URL = Pattern.compile("(?is)\\b(href|src|action|background)(\\s*=\\s*)([\"'])(.*?)\\3");
+
+   /*******************************************************************************
+    ** One budget shared by all parts and nested blocks of a render.
+    *******************************************************************************/
+   public static class Budget
+   {
+      private int iterations;
+      private int bytes;
+
+      /*******************************************************************************
+       ** Count work before entering a loop body.
+       *******************************************************************************/
+      private void iterate()
+      {
+         if(++iterations > 10_000)
+         {
+            throw (new IllegalArgumentException("Template output is too large"));
+         }
+      }
+
+      /*******************************************************************************
+       ** Bound UTF-8 output before appending it.
+       *******************************************************************************/
+      private void append(StringBuilder out, String value)
+      {
+         if(value.length() > 1_048_576 || (bytes += value.getBytes(StandardCharsets.UTF_8).length) > 1_048_576)
+         {
+            throw (new IllegalArgumentException("Template output is too large"));
+         }
+         out.append(value);
+      }
+   }
+
+   /*******************************************************************************
+    ** Rendering context. Markdown uses entities for all punctuation so values
+    ** cannot inject Markdown links, images or block syntax.
+    *******************************************************************************/
+   public enum Context
+   {
+      HTML, TEXT, MARKDOWN
+   }
+
+   /*******************************************************************************
+    ** A parsed literal, expression or block.
+    *******************************************************************************/
+   private record Node(String kind, String value, String fallback, int line, List children, String attribute)
+   {
+   }
+
+   /*******************************************************************************
+    ** Validated immutable parse result.
+    *******************************************************************************/
+   public static class Parsed
+   {
+      private final List nodes;
+      private final Set references;
+
+      /*******************************************************************************
+       ** Store parsed nodes.
+       *******************************************************************************/
+      private Parsed(List nodes, Set references)
+      {
+         this.nodes = nodes;
+         this.references = Set.copyOf(references);
+      }
+
+      /*******************************************************************************
+       ** Render the already validated syntax tree.
+       *******************************************************************************/
+      public String render(Map values, Context context)
+      {
+         return (render(values, context, new Budget(), new ArrayList<>()));
+      }
+
+      /*******************************************************************************
+       ** Bindings referenced by values and block conditions, including nested blocks.
+       *******************************************************************************/
+      public Set references()
+      {
+         return (references);
+      }
+
+      /*******************************************************************************
+       ** Render using the message-wide budget and safe author-facing warnings.
+       *******************************************************************************/
+      public String render(Map values, Context context, Budget budget, List warnings)
+      {
+         StringBuilder out = new StringBuilder();
+         renderNodes(nodes, values, null, -1, context, budget, warnings, out);
+         return (context == Context.HTML ? validateRenderedUrls(out.toString(), warnings) : out.toString());
+      }
+
+   }
+
+   /*******************************************************************************
+    ** Parse once, checking balanced blocks and every variable, with line numbers.
+    *******************************************************************************/
+   public Parsed parse(String source, VariableSchema schema)
+   {
+      source = source == null ? "" : source;
+      List root = new ArrayList<>();
+      Deque stack = new ArrayDeque<>();
+      Set references = new LinkedHashSet<>();
+      int position = 0;
+      int line = 1;
+      while(position < source.length())
+      {
+         List current = stack.isEmpty() ? root : stack.peek().children();
+         int start = source.indexOf("{{", position);
+         if(start < 0)
+         {
+            current.add(new Node("literal", source.substring(position), null, 0, List.of(), null));
+            break;
+         }
+         current.add(new Node("literal", source.substring(position, start), null, 0, List.of(), null));
+         for(int i = position; i < start; i++)
+         {
+            if(source.charAt(i) == '\n')
+            {
+               line++;
+            }
+         }
+         int end = source.indexOf("}}", start + 2);
+         if(end < 0)
+         {
+            throw (error(line, "Unclosed expression"));
+         }
+         String expression = source.substring(start + 2, end).trim();
+         if(expression.startsWith("/"))
+         {
+            if(stack.isEmpty() || !stack.peek().kind().equals(expression.substring(1)))
+            {
+               throw (error(line, "Block nesting error"));
+            }
+            stack.pop();
+         }
+         else
+         {
+            String kind = expression.startsWith("#if ") ? "if" : expression.startsWith("#each ") ? "each" : "value";
+            String binding = kind.equals("value") ? expression : expression.substring(kind.length() + 2).trim();
+            String[] parts = binding.split("\\|", 2);
+            String name = parts[0].trim();
+            boolean inEach = stack.stream().anyMatch(n -> n.kind().equals("each"));
+            boolean local = name.equals("this") || name.startsWith("this.") || name.equals("@index");
+            if(!name.matches("(?:[A-Za-z_][A-Za-z0-9_]*(?:\\.[A-Za-z_][A-Za-z0-9_]*)*|@index)")
+               || (local ? !inEach : !schema.knows(name)) || !kind.equals("value") && parts.length > 1)
+            {
+               throw (error(line, "Unknown or unsupported variable " + name));
+            }
+            if(!local)
+            {
+               references.add(name);
+            }
+            var attribute = URL_ATTRIBUTE.matcher(source.substring(position, start));
+            Node node = new Node(kind, name, parts.length == 2 ? parts[1] : "", line, new ArrayList<>(), attribute.find() ? attribute.group(1).toLowerCase(java.util.Locale.ROOT) : null);
+            current.add(node);
+            if(!kind.equals("value"))
+            {
+               if(stack.size() >= 8)
+               {
+                  throw (error(line, "Blocks exceed 8 levels"));
+               }
+               stack.push(node);
+            }
+         }
+         for(int i = start; i < end + 2; i++)
+         {
+            if(source.charAt(i) == '\n')
+            {
+               line++;
+            }
+         }
+         position = end + 2;
+      }
+      if(!stack.isEmpty())
+      {
+         throw (error(stack.peek().line(), "Unclosed " + stack.peek().kind() + " block"));
+      }
+      return (new Parsed(root, references));
+   }
+
+   /*******************************************************************************
+    ** Render syntax nodes; loop scope is restored on every recursive return.
+    *******************************************************************************/
+   private static void renderNodes(List nodes, Map values, Object item, int index, Context context, Budget budget, List warnings, StringBuilder out)
+   {
+      for(Node node : nodes)
+      {
+         Object value = node.value().equals("this") ? item : node.value().startsWith("this.") ? lookup(item, node.value().substring(5))
+            : node.value().equals("@index") ? index : lookup(values, node.value());
+         switch(node.kind())
+         {
+            case "literal" -> budget.append(out, node.value());
+            case "value" ->
+            {
+               String text = value == null || value.toString().isEmpty() ? node.fallback() : value.toString();
+               if(context == Context.HTML && node.attribute() != null && !text.isEmpty())
+               {
+                  try
+                  {
+                     VariableSchema.url(text, "href".equals(node.attribute()));
+                  }
+                  catch(IllegalArgumentException e)
+                  {
+                     text = "";
+                     String warning = "Unsafe URL omitted for " + node.value();
+                     if(!warnings.contains(warning))
+                     {
+                        warnings.add(warning);
+                     }
+                  }
+               }
+               if(text.length() > 1_048_576)
+               {
+                  throw (new IllegalArgumentException("Template output is too large"));
+               }
+               budget.append(out, encode(text, context));
+            }
+            case "if" ->
+            {
+               if(truthy(value))
+               {
+                  renderNodes(node.children(), values, item, index, context, budget, warnings, out);
+               }
+            }
+            case "each" ->
+            {
+               if(value instanceof List list)
+               {
+                  for(int i = 0; i < list.size(); i++)
+                  {
+                     budget.iterate();
+                     renderNodes(node.children(), values, list.get(i), i, context, budget, warnings, out);
+                  }
+               }
+            }
+            default -> throw (new IllegalStateException("Invalid template node"));
+         }
+      }
+   }
+
+   /*******************************************************************************
+    ** Expressions in attributes require quotes, including after literal prefixes.
+    *******************************************************************************/
+   public static void validateHtml(String source)
+   {
+      boolean tag = false;
+      char quote = 0;
+      for(int i = 0; source != null && i < source.length(); i++)
+      {
+         char c = source.charAt(i);
+         if(tag && quote == 0 && source.startsWith("{{", i))
+         {
+            throw (new IllegalArgumentException("Template expressions in attributes must be quoted"));
+         }
+         if(quote != 0)
+         {
+            if(c == quote)
+            {
+               quote = 0;
+            }
+         }
+         else if(c == '<')
+         {
+            tag = true;
+         }
+         else if(c == '>')
+         {
+            tag = false;
+         }
+         else if(tag && (c == '\"' || c == '\''))
+         {
+            quote = c;
+         }
+      }
+   }
+
+   /*******************************************************************************
+    ** Recheck complete attributes after block expansion, so whitespace, split
+    ** values and conditional expressions cannot hide the URL context.
+    *******************************************************************************/
+   private static String validateRenderedUrls(String html, List warnings)
+   {
+      var matcher = RENDERED_URL.matcher(html);
+      StringBuilder result = new StringBuilder();
+      while(matcher.find())
+      {
+         String value = org.jsoup.parser.Parser.unescapeEntities(matcher.group(4), true);
+         try
+         {
+            if(!value.isEmpty())
+            {
+               VariableSchema.url(value, "href".equalsIgnoreCase(matcher.group(1)));
+            }
+            matcher.appendReplacement(result, java.util.regex.Matcher.quoteReplacement(matcher.group()));
+         }
+         catch(IllegalArgumentException e)
+         {
+            String warning = "Unsafe " + matcher.group(1).toLowerCase(java.util.Locale.ROOT) + " URL omitted";
+            if(!warnings.contains(warning))
+            {
+               warnings.add(warning);
+            }
+            matcher.appendReplacement(result, java.util.regex.Matcher.quoteReplacement(matcher.group(1) + matcher.group(2) + matcher.group(3) + matcher.group(3)));
+         }
+      }
+      matcher.appendTail(result);
+      return (result.toString());
+   }
+
+   /*******************************************************************************
+    ** Dotted paths traverse maps only; no reflection, methods or arbitrary code.
+    *******************************************************************************/
+   public static Object lookup(Object values, String path)
+   {
+      if(values instanceof Map map)
+      {
+         if(map.containsKey(path))
+         {
+            return (map.get(path));
+         }
+         int dot = path.indexOf('.');
+         return (dot < 0 ? map.get(path) : lookup(map.get(path.substring(0, dot)), path.substring(dot + 1)));
+      }
+      return (null);
+   }
+
+   /*******************************************************************************
+    ** Explicit language truthiness.
+    *******************************************************************************/
+   public static boolean truthy(Object value)
+   {
+      if(value == null || Boolean.FALSE.equals(value) || value instanceof CharSequence s && s.isEmpty()
+         || value instanceof Collection c && c.isEmpty() || value instanceof Map m && m.isEmpty())
+      {
+         return (false);
+      }
+      return (!(value instanceof Number n) || new BigDecimal(n.toString()).compareTo(BigDecimal.ZERO) != 0);
+   }
+
+   /*******************************************************************************
+    ** Escape values exactly once for the consuming grammar.
+    *******************************************************************************/
+   public static String encode(String value, Context context)
+   {
+      if(context == Context.TEXT)
+      {
+         return (value);
+      }
+      if(context == Context.MARKDOWN)
+      {
+         StringBuilder encoded = new StringBuilder();
+         value.codePoints().forEach(c -> encoded.append(Character.isLetterOrDigit(c) || c == ' ' ? new String(Character.toChars(c)) : "&#" + c + ";"));
+         return (encoded.toString());
+      }
+      return (escape(value));
+   }
+
+   /*******************************************************************************
+    ** Attribute and text escaping, including both quote types.
+    *******************************************************************************/
+   public static String escape(Object value)
+   {
+      return (value == null ? "" : value.toString().replace("&", "&").replace("<", "<").replace(">", ">")
+         .replace("\"", """).replace("'", "'"));
+   }
+
+   /*******************************************************************************
+    ** Stable author-facing parse diagnostic.
+    *******************************************************************************/
+   private static IllegalArgumentException error(int line, String message)
+   {
+      return (new IllegalArgumentException("Line " + line + ": " + message));
+   }
+}
diff --git a/src/main/java/com/kingsrook/qbits/email/template/EmailTemplateRenderer.java b/src/main/java/com/kingsrook/qbits/email/template/EmailTemplateRenderer.java
new file mode 100644
index 0000000..4de944b
--- /dev/null
+++ b/src/main/java/com/kingsrook/qbits/email/template/EmailTemplateRenderer.java
@@ -0,0 +1,453 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.email.template;
+
+
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import com.kingsrook.qbits.email.EmailRuntime;
+import com.kingsrook.qbits.email.metadata.EmailTables;
+import com.kingsrook.qbits.email.outbox.EmailCategory;
+import com.kingsrook.qbits.email.outbox.EmailRequest;
+import com.kingsrook.qbits.email.phi.EmailBodyEncryption;
+import com.kingsrook.qbits.email.phi.EmailClassifications;
+import com.kingsrook.qbits.email.phi.EmailClassifications.Level;
+import com.kingsrook.qbits.email.phi.EmailValueTokens;
+import com.kingsrook.qbits.email.render.EmailBrands;
+import com.kingsrook.qbits.email.render.EmailLayout;
+import com.kingsrook.qbits.email.render.EmailMarkdown;
+import com.kingsrook.qbits.email.render.HtmlToText;
+import com.kingsrook.qbits.email.support.EmailStore;
+import com.kingsrook.qbits.email.support.SystemWrites;
+import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+
+
+/*******************************************************************************
+ ** Queue-time template resolution and rendering with sender-visible record data.
+ *******************************************************************************/
+public final class EmailTemplateRenderer
+{
+   private final QRecord template;
+   private final QRecord version;
+   private final QRecord brand;
+   private final VariableSchema schema;
+   private final Map values;
+   private final Map records;
+   private final EmailCategory category;
+   private final boolean approved;
+   private final List warnings = new ArrayList<>();
+
+   /*******************************************************************************
+    ** Resolve a template's active locale, falling back only to English.
+    *******************************************************************************/
+   public static EmailTemplateRenderer prepare(EmailRequest request) throws QException
+   {
+      if("email_secure_message".equals(request.getTemplateName()))
+      {
+         EmailBuiltIns.ensure();
+      }
+      List templates = SystemWrites.query(EmailTables.TEMPLATE, EmailStore.equal("name", request.getTemplateName()));
+      if(templates.size() != 1)
+      {
+         throw (new IllegalArgumentException("Unknown email template"));
+      }
+      QRecord template = templates.getFirst();
+      List versions = SystemWrites.query(EmailTables.VERSION, EmailStore.equal("templateId", template.getValueInteger("id"))
+         .withCriteria(new QFilterCriteria("status", QCriteriaOperator.EQUALS, "ACTIVE")));
+      QRecord version = versions.stream().filter(v -> Objects.equals(request.getLocale(), v.getValueString("locale"))).findFirst()
+         .orElseGet(() -> versions.stream().filter(v -> "en".equals(v.getValueString("locale"))).findFirst().orElse(null));
+      if(version == null)
+      {
+         throw (new IllegalArgumentException("Template has no active version for this locale or en"));
+      }
+      return (new EmailTemplateRenderer(template, version, request));
+   }
+
+   /*******************************************************************************
+    ** Validate schema, provenance and all template syntax before any outbox write.
+    *******************************************************************************/
+   public EmailTemplateRenderer(QRecord template, QRecord version, EmailRequest request) throws QException
+   {
+      this.template = template;
+      this.version = version;
+      approved = Boolean.TRUE.equals(template.getValueBoolean("phiApproved")) && version.getValue("id") != null
+         && Objects.equals(template.getValue("approvedVersionId"), version.getValue("id"));
+      brand = EmailBrands.resolve(approved ? null : request.getBrandName(), template);
+      if(approved)
+      {
+         for(String field : List.of("logoUrl", "logoDarkUrl"))
+         {
+            if(brand.getValueString(field) != null && !EmailBrands.allowedImage(brand.getValueString(field), brand))
+            {
+               brand.setValue(field, null);
+               warnings.add("Brand logo omitted: image host is not allowed");
+            }
+         }
+      }
+      EmailBrands.validate(brand);
+      schema = new VariableSchema(version.getValueString("variablesSchema"));
+      records = new LinkedHashMap<>(request.getRecordVariables());
+      java.util.Set references = new java.util.LinkedHashSet<>();
+      for(String field : List.of("subject", "preheader", "bodySource", "textSource", "compiledHtml"))
+      {
+         references.addAll(new EmailTemplateEngine().parse(version.getValueString(field), schema).references());
+      }
+      if(!"MARKDOWN".equals(template.getValueString("kind")))
+      {
+         EmailTemplateEngine.validateHtml(version.getValueString("bodySource"));
+         EmailTemplateEngine.validateHtml(version.getValueString("compiledHtml"));
+      }
+      Map supplied = new LinkedHashMap<>(request.getVariables());
+      for(var binding : records.entrySet())
+      {
+         supplied.keySet().removeIf(key -> key.startsWith(binding.getKey() + "."));
+         QTableMetaData table = QContext.getQInstance().getTable(binding.getValue().table());
+         if(table == null)
+         {
+            throw (new IllegalArgumentException("Record variable table is unavailable"));
+         }
+         java.util.Set fields = references.stream().filter(path -> path.startsWith(binding.getKey() + "."))
+            .map(path -> path.substring(binding.getKey().length() + 1).split("\\.")[0]).collect(java.util.stream.Collectors.toSet());
+         if(fields.isEmpty())
+         {
+            supplied.put(binding.getKey(), new LinkedHashMap<>());
+            continue;
+         }
+         // Never use SystemWrites here: masking and record locks belong to the sender.
+         List rows = new QueryAction().execute(new QueryInput(table.getName()).withShouldFetchHeavyFields(true).withFieldNamesToInclude(fields)
+            .withFilter(EmailStore.equal(table.getPrimaryKeyField(), binding.getValue().id()).withLimit(1))).getRecords();
+         if(rows.isEmpty())
+         {
+            throw (new IllegalArgumentException("Record variable is unavailable to the sender"));
+         }
+         Map visible = new LinkedHashMap<>();
+         fields.forEach(field -> visible.put(field, rows.getFirst().getValue(field)));
+         for(var field : table.getFields().values())
+         {
+            var lock = field.getFieldSecurityLock();
+            if(lock != null && fields.contains(field.getName()))
+            {
+               var session = QContext.getQSession();
+               List keys = session == null ? List.of() : session.getSecurityKeyValues(lock.getSecurityKeyType());
+               boolean override = keys != null && lock.getOverrideValues() != null && keys.stream().anyMatch(lock.getOverrideValues()::contains);
+               boolean allowed = lock.getDefaultBehavior() == com.kingsrook.qqq.backend.core.model.metadata.security.FieldSecurityLock.Behavior.ALLOW;
+               if(override ? allowed : !allowed)
+               {
+                  visible.put(field.getName(), "************");
+               }
+            }
+         }
+         supplied.put(binding.getKey(), new LinkedHashMap<>(visible));
+      }
+      for(String name : supplied.keySet())
+      {
+         if(!schema.knows(name) || name.startsWith("brand.") || name.equals("message.id") || name.equals("now") || name.equals("unsubscribe_url"))
+         {
+            throw (new IllegalArgumentException("Unknown or reserved input variable " + name));
+         }
+      }
+      values = schema.validate(supplied);
+      category = EmailCategory.valueOf(Objects.requireNonNullElse(template.getValueString("category"), "TRANSACTIONAL"));
+      if(request.getCategory() != null && request.getCategory() != category)
+      {
+         throw (new IllegalArgumentException("Template category cannot be overridden"));
+      }
+      if(approved && category == EmailCategory.MARKETING)
+      {
+         throw (new IllegalArgumentException("PHI-approved templates cannot send MARKETING email"));
+      }
+      for(VariableSchema.Entry entry : schema.entries().values())
+      {
+         requireApproval(entry.name(), classification(entry.name()));
+      }
+      inspect(values, "");
+      for(String field : List.of("subject", "preheader", "bodySource", "textSource"))
+      {
+         new EmailTemplateEngine().parse(version.getValueString(field), schema);
+      }
+      if(version.getValueString("subject") == null || version.getValueString("bodySource") == null)
+      {
+         throw (new IllegalArgumentException("A template subject and body are required"));
+      }
+   }
+
+   /*******************************************************************************
+    ** Check actual nested fields as well as declared but absent bindings.
+    *******************************************************************************/
+   private void inspect(Object object, String path)
+   {
+      if(!path.isEmpty())
+      {
+         requireApproval(path, classification(path));
+      }
+      if(object instanceof Map map)
+      {
+         map.forEach((key, value) -> inspect(value, path.isEmpty() ? key.toString() : path + "." + key));
+      }
+      else if(object instanceof List list)
+      {
+         list.forEach(value -> inspect(value, path));
+      }
+   }
+
+   /*******************************************************************************
+    ** A classification failure names only the binding, never its value.
+    *******************************************************************************/
+   private void requireApproval(String name, Level level)
+   {
+      if(EmailClassifications.sensitive(level) && !approved)
+      {
+         throw (new IllegalArgumentException("The variable " + name + " is " + level + "; this template is not approved for PHI."));
+      }
+   }
+
+   /*******************************************************************************
+    ** Resolve record provenance before schema declarations; inherit parent levels.
+    *******************************************************************************/
+   private Level classification(String path)
+   {
+      VariableSchema.Entry declaration = schema.entries().get(path);
+      if(declaration == null)
+      {
+         declaration = schema.entries().values().stream().filter(e -> path.startsWith(e.name() + "."))
+            .max(java.util.Comparator.comparingInt(e -> e.name().length())).orElse(null);
+      }
+      String declared = declaration == null ? null : declaration.classification();
+      for(var record : records.entrySet())
+      {
+         if(path.startsWith(record.getKey() + "."))
+         {
+            String field = path.substring(record.getKey().length() + 1).split("\\.")[0];
+            return (EmailClassifications.ofField(QContext.getQInstance().getTable(record.getValue().table()), field, declared, EmailRuntime.config()));
+         }
+      }
+      QTableMetaData table = declaration == null || declaration.table() == null ? null : QContext.getQInstance().getTable(declaration.table());
+      if(declaration != null && declaration.table() != null && (table == null || table.getField(declaration.field()) == null))
+      {
+         throw (new IllegalArgumentException("Schema field binding is unavailable"));
+      }
+      return (EmailClassifications.ofField(table, declaration == null ? null : declaration.field(), declared, EmailRuntime.config()));
+   }
+
+   /*******************************************************************************
+    ** Validate the actual provider at queue and delivery time.
+    *******************************************************************************/
+   public void validateProvider(QRecord provider)
+   {
+      if(approved)
+      {
+         checkProvider(provider);
+      }
+   }
+
+   /*******************************************************************************
+    ** Recheck mutable provider attestations before encrypted content leaves storage.
+    *******************************************************************************/
+   public static void checkProvider(QRecord provider)
+   {
+      if(!Boolean.TRUE.equals(provider.getValueBoolean("phiAllowed")) || Boolean.TRUE.equals(provider.getValueBoolean("trackOpens"))
+         || Boolean.TRUE.equals(provider.getValueBoolean("trackClicks")))
+      {
+         throw (new IllegalArgumentException("PHI email needs a phi_allowed provider with open and click tracking disabled"));
+      }
+   }
+
+   /*******************************************************************************
+    ** Render with the real message identity; no compiled source is rebuilt at send.
+    *******************************************************************************/
+   public QRecord render(String messageId, String unsubscribe, boolean encrypt)
+   {
+      Map context = new LinkedHashMap<>(values);
+      for(String field : List.of("label", "name", "siteUrl", "logoUrl", "logoDarkUrl", "logoAlt", "postalAddress", "primaryColor", "textColor", "backgroundColor", "darkBackgroundColor", "darkTextColor", "buttonTextColor", "darkPrimaryColor", "darkButtonTextColor", "footerMarkdown"))
+      {
+         context.put("brand." + field.replaceAll("([a-z])([A-Z])", "$1_$2").toLowerCase(java.util.Locale.ROOT), brand.getValueString(field));
+      }
+      context.put("message.id", messageId);
+      context.put("now", EmailRuntime.clock().instant().toString());
+      context.put("unsubscribe_url", category == EmailCategory.NOTIFICATION || category == EmailCategory.MARKETING ? unsubscribe : "");
+      EmailTemplateEngine engine = new EmailTemplateEngine();
+      EmailTemplateEngine.Budget budget = new EmailTemplateEngine.Budget();
+      String subject = engine.parse(version.getValueString("subject"), schema).render(context, EmailTemplateEngine.Context.TEXT, budget, warnings);
+      if(subject.contains("\r") || subject.contains("\n"))
+      {
+         throw (new IllegalArgumentException("Subject must be one line"));
+      }
+      String kind = template.getValueString("kind");
+      String source = "MJML".equals(kind) ? version.getValueString("compiledHtml") : version.getValueString("bodySource");
+      if(source == null)
+      {
+         throw (new IllegalArgumentException("MJML version must be compiled before sending"));
+      }
+      String body = engine.parse(source, schema).render(context, "MARKDOWN".equals(kind) ? EmailTemplateEngine.Context.MARKDOWN : EmailTemplateEngine.Context.HTML, budget, warnings);
+      String preheader = engine.parse(version.getValueString("preheader"), schema).render(context, EmailTemplateEngine.Context.TEXT, budget, warnings);
+      String html = "MARKDOWN".equals(kind) ? EmailLayout.render(EmailMarkdown.render(body, brand), preheader, brand, category, unsubscribe)
+         : EmailLayout.appendFooter(body, brand, category, unsubscribe);
+      if(approved)
+      {
+         html = restrictImages(html);
+      }
+      String text = version.getValueString("textSource") == null ? HtmlToText.convert(html)
+         : engine.parse(version.getValueString("textSource"), schema).render(context, EmailTemplateEngine.Context.TEXT, budget, warnings);
+      if((long) subject.getBytes(java.nio.charset.StandardCharsets.UTF_8).length + html.getBytes(java.nio.charset.StandardCharsets.UTF_8).length
+         + text.getBytes(java.nio.charset.StandardCharsets.UTF_8).length > 1_048_576)
+      {
+         throw (new IllegalArgumentException("Template output is too large"));
+      }
+      return (new QRecord().withValue("subject", approved && encrypt ? EmailBodyEncryption.encrypt(subject, "subject", messageId) : subject)
+         .withValue("htmlBody", approved && encrypt ? EmailBodyEncryption.encrypt(html, "htmlBody", messageId) : html)
+         .withValue("textBody", approved && encrypt ? EmailBodyEncryption.encrypt(text, "textBody", messageId) : text)
+         .withValue("variablesSnapshot", VariableSchema.json(snapshot(values, "", messageId)))
+         .withValue("templateId", template.getValue("id")).withValue("templateVersionId", version.getValue("id"))
+         .withValue("brandId", brand.getValue("id")).withValue("locale", version.getValue("locale"))
+         .withValue("category", category.name()).withValue("phiApproved", approved));
+   }
+
+   /*******************************************************************************
+    ** Recursive redaction prevents nested records or lists bypassing tokenization.
+    *******************************************************************************/
+   private Object snapshot(Object value, String path, String messageId)
+   {
+      if(!path.isEmpty() && (EmailClassifications.sensitive(classification(path)) || unclassifiedRecordField(path)))
+      {
+         for(var record : records.entrySet())
+         {
+            if(path.startsWith(record.getKey() + ".") && !path.substring(record.getKey().length() + 1).contains("."))
+            {
+               QTableMetaData table = QContext.getQInstance().getTable(record.getValue().table());
+               var field = table.getField(path.substring(record.getKey().length() + 1));
+               if(field != null)
+               {
+                  return (EmailValueTokens.forRecordField(table, field, record.getValue().id(), (Serializable) value));
+               }
+            }
+         }
+         return (EmailValueTokens.forContext(List.of("email", template.getValueString("name"), path, messageId), (Serializable) value));
+      }
+      if(value instanceof Map map)
+      {
+         Map result = new LinkedHashMap<>();
+         map.forEach((key, item) -> result.put(key.toString(), snapshot(item, path.isEmpty() ? key.toString() : path + "." + key, messageId)));
+         return (result);
+      }
+      if(value instanceof List list)
+      {
+         List result = new ArrayList<>();
+         list.forEach(item -> result.add(snapshot(item, path, messageId)));
+         return (result);
+      }
+      return (value == null || value instanceof Number || value instanceof Boolean ? value : value.toString());
+   }
+
+   /*******************************************************************************
+    ** Missing record classification fails closed only in the retained snapshot.
+    *******************************************************************************/
+   private boolean unclassifiedRecordField(String path)
+   {
+      for(var record : records.entrySet())
+      {
+         if(path.startsWith(record.getKey() + "."))
+         {
+            String field = path.substring(record.getKey().length() + 1).split("\\.")[0];
+            return (!EmailClassifications.hasSource(QContext.getQInstance().getTable(record.getValue().table()), field, EmailRuntime.config()));
+         }
+      }
+      return (false);
+   }
+
+   /*******************************************************************************
+    ** Warnings contain binding names and policy reasons, never input values.
+    *******************************************************************************/
+   public List warnings()
+   {
+      return (List.copyOf(new java.util.LinkedHashSet<>(warnings)));
+   }
+
+   /*******************************************************************************
+    ** Apply the image policy after Markdown, layout and brand footer expansion.
+    *******************************************************************************/
+   private String restrictImages(String html)
+   {
+      var document = org.jsoup.Jsoup.parse(html);
+      document.outputSettings().prettyPrint(false);
+      document.select("[src],[background],[srcset],image").forEach(element ->
+      {
+         element.removeAttr("srcset");
+         if(element.hasAttr("src") && !EmailBrands.allowedImage(element.attr("src"), brand))
+         {
+            element.remove();
+            warnings.add("Image omitted: image host is not allowed");
+         }
+         if(element.hasAttr("background") && !EmailBrands.allowedImage(element.attr("background"), brand))
+         {
+            element.removeAttr("background");
+            warnings.add("Image omitted: image host is not allowed");
+         }
+         if("image".equals(element.normalName()))
+         {
+            element.remove();
+            warnings.add("SVG image omitted from approved email");
+         }
+      });
+      org.jsoup.select.NodeTraversor.filter((node, depth) ->
+      {
+         if(node instanceof org.jsoup.nodes.Comment comment && comment.getData().toLowerCase(java.util.Locale.ROOT).contains("[if"))
+         {
+            warnings.add("Conditional markup omitted from approved email");
+            return (org.jsoup.select.NodeFilter.FilterResult.REMOVE);
+         }
+         return (org.jsoup.select.NodeFilter.FilterResult.CONTINUE);
+      }, document);
+      // CSS image expressions can use escapes and image-set, so approved content
+      // uses img elements instead of CSS images. Layout colors remain intact.
+      document.select("[style]").forEach(element ->
+      {
+         String style = element.attr("style");
+         if(style.toLowerCase(java.util.Locale.ROOT).contains("url") || style.contains("\\") || style.toLowerCase(java.util.Locale.ROOT).contains("image-set"))
+         {
+            element.removeAttr("style");
+            warnings.add("CSS image omitted from approved email");
+         }
+      });
+      document.select("style").forEach(element ->
+      {
+         String style = element.data();
+         if(style.toLowerCase(java.util.Locale.ROOT).contains("url") || style.contains("\\") || style.toLowerCase(java.util.Locale.ROOT).contains("@import") || style.toLowerCase(java.util.Locale.ROOT).contains("image-set"))
+         {
+            element.remove();
+            warnings.add("CSS image omitted from approved email");
+         }
+      });
+      return (document.outerHtml());
+   }
+
+}
diff --git a/src/main/java/com/kingsrook/qbits/email/template/EmailVersionLabels.java b/src/main/java/com/kingsrook/qbits/email/template/EmailVersionLabels.java
new file mode 100644
index 0000000..0244cc2
--- /dev/null
+++ b/src/main/java/com/kingsrook/qbits/email/template/EmailVersionLabels.java
@@ -0,0 +1,68 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.email.template;
+
+
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import com.kingsrook.qbits.email.metadata.EmailTables;
+import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
+import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.tables.QueryOrGetInputInterface;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+
+
+/*******************************************************************************
+ ** Human-readable version labels, resolving parent names as the current viewer.
+ *******************************************************************************/
+public class EmailVersionLabels implements TableCustomizerInterface
+{
+   /*******************************************************************************
+    ** Batch parents to avoid one query per version, without bypassing security.
+    *******************************************************************************/
+   @Override
+   public List postQuery(QueryOrGetInputInterface input, List records) throws QException
+   {
+      var ids = records.stream().map(r -> r.getValueInteger("templateId")).filter(Objects::nonNull).distinct().toList();
+      Map names = new HashMap<>();
+      if(!ids.isEmpty())
+      {
+         List templates = new QueryAction().execute(new QueryInput(EmailTables.TEMPLATE)
+            .withFilter(new QQueryFilter().withCriteria("id", QCriteriaOperator.IN, ids))
+            .withFieldNamesToInclude(new HashSet<>(List.of("id", "name")))).getRecords();
+         templates.forEach(t -> names.put(t.getValueInteger("id"), t.getValueString("name")));
+      }
+      for(QRecord row : records)
+      {
+         String locale = row.getValueString("locale");
+         row.setRecordLabel(names.getOrDefault(row.getValueInteger("templateId"), "Template") + " v" + row.getValueInteger("versionNumber")
+            + (locale == null || locale.isBlank() || "en".equals(locale) ? "" : " (" + locale + ")"));
+      }
+      return (records);
+   }
+}
diff --git a/src/main/java/com/kingsrook/qbits/email/template/EmailVersions.java b/src/main/java/com/kingsrook/qbits/email/template/EmailVersions.java
new file mode 100644
index 0000000..da96213
--- /dev/null
+++ b/src/main/java/com/kingsrook/qbits/email/template/EmailVersions.java
@@ -0,0 +1,215 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.email.template;
+
+
+import java.util.List;
+import java.util.Objects;
+import com.kingsrook.qbits.email.EmailRuntime;
+import com.kingsrook.qbits.email.metadata.EmailTables;
+import com.kingsrook.qbits.email.outbox.EmailRequest;
+import com.kingsrook.qbits.email.permissions.EmailPermissionChecker;
+import com.kingsrook.qbits.email.permissions.EmailPermissions;
+import com.kingsrook.qbits.email.support.EmailStore;
+import com.kingsrook.qbits.email.support.SystemWrites;
+import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
+import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+
+
+/*******************************************************************************
+ ** Atomic version lifecycle. Approval binds to exactly one immutable version.
+ *******************************************************************************/
+public final class EmailVersions
+{
+   /*******************************************************************************
+    ** Utility class.
+    *******************************************************************************/
+   private EmailVersions()
+   {
+   }
+
+   /*******************************************************************************
+    ** Enforce both the qbit permission and the template's independent edit grant.
+    *******************************************************************************/
+   public static void requireEdit(QRecord template, boolean approval)
+   {
+      if(SystemWrites.active())
+      {
+         return;
+      }
+      if(!EmailPermissionChecker.has(approval ? EmailPermissions.ADMIN : EmailPermissions.MANAGE))
+      {
+         throw (new IllegalArgumentException(approval ? "PHI approval needs email.admin" : "Editing templates needs email.manage"));
+      }
+      String permission = template.getValueString("editPermissionName");
+      if(permission != null && !permission.isBlank() && (QContext.getQSession() == null || !QContext.getQSession().hasPermission(permission)))
+      {
+         throw (new IllegalArgumentException("Editing this template needs " + permission));
+      }
+   }
+
+   /*******************************************************************************
+    ** Serialize lifecycle operations across replicas using the template row.
+    *******************************************************************************/
+   private static QRecord lock(Integer templateId, QBackendTransaction tx) throws QException
+   {
+      QRecord template = EmailStore.get(EmailTables.TEMPLATE, templateId, tx);
+      if(template == null)
+      {
+         throw (new IllegalArgumentException("Template not found"));
+      }
+      requireEdit(template, false);
+      EmailStore.update(EmailTables.TEMPLATE, new QRecord().withValue("id", templateId).withValue("versionLock", 0), tx);
+      return (EmailStore.get(EmailTables.TEMPLATE, templateId, tx));
+   }
+
+   /*******************************************************************************
+    ** Copy active (or latest) into a new draft, with a globally increasing number.
+    *******************************************************************************/
+   public static QRecord newVersion(Integer templateId) throws QException
+   {
+      try(QBackendTransaction tx = QBackendTransaction.openFor(new UpdateInput(EmailTables.TEMPLATE)))
+      {
+         QRecord template = lock(templateId, tx);
+         List versions = SystemWrites.query(EmailTables.VERSION, EmailStore.equal("templateId", templateId)
+            .withOrderBy(new QFilterOrderBy("versionNumber", false)), tx);
+         QRecord source = versions.stream().filter(v -> Objects.equals(template.getValue("activeVersionId"), v.getValue("id"))).findFirst()
+            .orElse(versions.isEmpty() ? new QRecord().withValue("locale", "en").withValue("variablesSchema", "[]").withValue("sampleVariables", "{}") : versions.getFirst());
+         QRecord draft = new QRecord();
+         for(String field : List.of("locale", "subject", "preheader", "bodySource", "textSource", "variablesSchema", "sampleVariables"))
+         {
+            draft.setValue(field, source.getValue(field));
+         }
+         draft.withValue("templateId", templateId).withValue("versionNumber", versions.isEmpty() ? 1 : versions.getFirst().getValueInteger("versionNumber") + 1)
+            .withValue("status", "DRAFT").withValue("createdBy", actor());
+         QRecord result = SystemWrites.insert(EmailTables.VERSION, List.of(draft), tx).getRecords().getFirst();
+         if(result.getValue("id") == null || result.getErrors() != null && !result.getErrors().isEmpty())
+         {
+            throw (new QException("Could not create template version"));
+         }
+         tx.commit();
+         return (result);
+      }
+   }
+
+   /*******************************************************************************
+    ** Approval is granted to the selected draft or active version, never retired.
+    *******************************************************************************/
+   public static void approve(Integer versionId) throws QException
+   {
+      QRecord version = requiredVersion(versionId, null);
+      try(QBackendTransaction tx = QBackendTransaction.openFor(new UpdateInput(EmailTables.TEMPLATE)))
+      {
+         QRecord template = lock(version.getValueInteger("templateId"), tx);
+         requireEdit(template, true);
+         version = requiredVersion(versionId, tx);
+         if("RETIRED".equals(version.getValueString("status")) || "MARKETING".equals(template.getValueString("category")))
+         {
+            throw (new IllegalArgumentException("Retired versions and MARKETING templates cannot be approved for PHI"));
+         }
+         EmailStore.update(EmailTables.TEMPLATE, new QRecord().withValue("id", template.getValue("id"))
+            .withValue("phiApproved", true).withValue("approvedVersionId", versionId).withValue("phiApprovedBy", actor())
+            .withValue("phiApprovedDate", EmailRuntime.clock().instant()), tx);
+         tx.commit();
+      }
+   }
+
+   /*******************************************************************************
+    ** Validate samples and syntax, compile MJML, then atomically retire and activate.
+    *******************************************************************************/
+   public static void activate(Integer versionId) throws QException
+   {
+      QRecord initial = requiredVersion(versionId, null);
+      try(QBackendTransaction tx = QBackendTransaction.openFor(new UpdateInput(EmailTables.TEMPLATE)))
+      {
+         QRecord template = lock(initial.getValueInteger("templateId"), tx);
+         QRecord version = requiredVersion(versionId, tx);
+         if(!"DRAFT".equals(version.getValueString("status")))
+         {
+            throw (new IllegalArgumentException("Only a DRAFT version can be activated"));
+         }
+         validate(template, version);
+         for(QRecord previous : SystemWrites.query(EmailTables.VERSION, EmailStore.equal("templateId", template.getValueInteger("id")), tx))
+         {
+            if("ACTIVE".equals(previous.getValueString("status")) && Objects.equals(version.getValue("locale"), previous.getValue("locale")))
+            {
+               EmailStore.update(EmailTables.VERSION, new QRecord().withValue("id", previous.getValue("id")).withValue("status", "RETIRED"), tx);
+            }
+         }
+         EmailStore.update(EmailTables.VERSION, new QRecord().withValue("id", versionId).withValue("status", "ACTIVE")
+            .withValue("compiledHtml", version.getValue("compiledHtml")).withValue("activatedDate", EmailRuntime.clock().instant()), tx);
+         QRecord change = new QRecord().withValue("id", template.getValue("id")).withValue("activeVersionId", versionId);
+         if(!Objects.equals(template.getValue("approvedVersionId"), versionId))
+         {
+            change.withValue("phiApproved", false).withValue("approvedVersionId", null).withValue("phiApprovedBy", null).withValue("phiApprovedDate", null);
+         }
+         EmailStore.update(EmailTables.TEMPLATE, change, tx);
+         tx.commit();
+      }
+   }
+
+   /*******************************************************************************
+    ** Review validation is side-effect-free; compiled output stays in memory.
+    *******************************************************************************/
+   public static void validate(QRecord template, QRecord version) throws QException
+   {
+      requireEdit(template, false);
+      if("MJML".equals(template.getValueString("kind")))
+      {
+         CliMjmlCompiler.rejectIncludes(version.getValueString("bodySource"));
+         EmailTemplateEngine.validateHtml(version.getValueString("bodySource"));
+         if(EmailRuntime.config().getMjmlCompiler() == null)
+         {
+            throw (new IllegalArgumentException("MJML activation needs a configured compiler"));
+         }
+         new EmailTemplateEngine().parse(version.getValueString("bodySource"), new VariableSchema(version.getValueString("variablesSchema")));
+         version.setValue("compiledHtml", EmailRuntime.config().getMjmlCompiler().compile(version.getValueString("bodySource")));
+      }
+      EmailTemplateRenderer renderer = new EmailTemplateRenderer(template, version,
+         new EmailRequest().withVariables(VariableSchema.values(version.getValueString("sampleVariables"))));
+      renderer.render("preview", "https://example.invalid/unsubscribe/preview", false);
+   }
+
+   /*******************************************************************************
+    ** Load a required version without exposing database diagnostics.
+    *******************************************************************************/
+   public static QRecord requiredVersion(Integer id, QBackendTransaction tx) throws QException
+   {
+      QRecord version = EmailStore.get(EmailTables.VERSION, id, tx);
+      if(version == null)
+      {
+         throw (new IllegalArgumentException("Template version not found"));
+      }
+      return (version);
+   }
+
+   /*******************************************************************************
+    ** Stable session actor identity, with an explicit system fallback.
+    *******************************************************************************/
+   private static String actor()
+   {
+      return (QContext.getQSession() == null || QContext.getQSession().getUser() == null ? "system" : QContext.getQSession().getUser().getIdReference());
+   }
+}
diff --git a/src/main/java/com/kingsrook/qbits/email/template/MjmlCompilerInterface.java b/src/main/java/com/kingsrook/qbits/email/template/MjmlCompilerInterface.java
new file mode 100644
index 0000000..6199d7b
--- /dev/null
+++ b/src/main/java/com/kingsrook/qbits/email/template/MjmlCompilerInterface.java
@@ -0,0 +1,34 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.email.template;
+
+
+/*******************************************************************************
+ ** Activation-time compilation only. Implementations must not persist sources.
+ *******************************************************************************/
+@FunctionalInterface
+public interface MjmlCompilerInterface
+{
+   /*******************************************************************************
+    ** Return HTML or a content-free failure.
+    *******************************************************************************/
+   String compile(String source);
+}
diff --git a/src/main/java/com/kingsrook/qbits/email/template/VariableSchema.java b/src/main/java/com/kingsrook/qbits/email/template/VariableSchema.java
new file mode 100644
index 0000000..d710633
--- /dev/null
+++ b/src/main/java/com/kingsrook/qbits/email/template/VariableSchema.java
@@ -0,0 +1,212 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.email.template;
+
+
+import java.io.Serializable;
+import java.math.BigDecimal;
+import java.net.URI;
+import java.time.Instant;
+import java.time.LocalDate;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import com.fasterxml.jackson.core.type.TypeReference;
+import com.fasterxml.jackson.databind.ObjectMapper;
+
+
+/*******************************************************************************
+ ** Declarative variable contract. Validation errors never include input values.
+ *******************************************************************************/
+public class VariableSchema
+{
+   private final Map entries = new LinkedHashMap<>();
+   private static final ObjectMapper JSON = new ObjectMapper();
+
+   /*******************************************************************************
+    ** One schema binding, including optional record provenance.
+    *******************************************************************************/
+   public record Entry(String name, String label, String type, boolean required, String classification, String table, String field)
+   {
+   }
+
+   /*******************************************************************************
+    ** Parse the JSON list and reject ambiguous or unsupported declarations.
+    *******************************************************************************/
+   public VariableSchema(String json)
+   {
+      try
+      {
+         List parsed = JSON.readValue(json == null || json.isBlank() ? "[]" : json, new TypeReference>() {});
+         for(Entry entry : parsed)
+         {
+            if(entry.name() == null || !entry.name().matches("[A-Za-z_][A-Za-z0-9_]*(\\.[A-Za-z_][A-Za-z0-9_]*)*")
+               || !List.of("STRING", "NUMBER", "DATE", "DATE_TIME", "BOOLEAN", "URL", "LIST", "RECORD").contains(entry.type())
+               || entries.putIfAbsent(entry.name(), entry) != null
+               || entry.classification() != null && !List.of("PUBLIC", "CONFIDENTIAL", "PII", "PHI").contains(entry.classification())
+               || (entry.table() == null) != (entry.field() == null))
+            {
+               throw (new IllegalArgumentException());
+            }
+         }
+      }
+      catch(Exception e)
+      {
+         throw (new IllegalArgumentException("Invalid variables schema"));
+      }
+   }
+
+   /*******************************************************************************
+    ** Declared bindings in stable order.
+    *******************************************************************************/
+   public Map entries()
+   {
+      return (Map.copyOf(entries));
+   }
+
+   /*******************************************************************************
+    ** Built-ins and paths under declared records are known to the parser.
+    *******************************************************************************/
+   public boolean knows(String name)
+   {
+      if(name.equals("now") || name.equals("message.id") || name.equals("unsubscribe_url")
+         || name.matches("brand\\.(label|name|site_url|logo_url|logo_dark_url|logo_alt|postal_address|primary_color|text_color|background_color|dark_background_color|dark_text_color|button_text_color|dark_primary_color|dark_button_text_color|footer_markdown)"))
+      {
+         return (true);
+      }
+      return (entries.containsKey(name) || entries.values().stream().anyMatch(e ->
+         ("RECORD".equals(e.type()) || "LIST".equals(e.type())) && name.startsWith(e.name() + ".")));
+   }
+
+   /*******************************************************************************
+    ** Validate and coerce declared values without discarding record structure.
+    *******************************************************************************/
+   public Map validate(Map values)
+   {
+      Map result = new LinkedHashMap<>(values);
+      for(Entry entry : entries.values())
+      {
+         Object value = EmailTemplateEngine.lookup(values, entry.name());
+         if(value == null || value instanceof String s && s.isEmpty())
+         {
+            if(entry.required())
+            {
+               throw (new IllegalArgumentException("Required variable " + entry.name() + " is missing"));
+            }
+            continue;
+         }
+         try
+         {
+            Serializable coerced = switch(entry.type())
+            {
+               case "STRING" -> value.toString();
+               case "NUMBER" -> new BigDecimal(value.toString());
+               case "DATE" -> LocalDate.parse(value.toString());
+               case "DATE_TIME" -> Instant.parse(value.toString());
+               case "BOOLEAN" -> booleanValue(value);
+               case "URL" -> url(value.toString());
+               case "LIST" -> (Serializable) (List) value;
+               case "RECORD" -> (Serializable) (Map) value;
+               default -> throw (new IllegalArgumentException());
+            };
+            result.put(entry.name(), coerced);
+         }
+         catch(Exception e)
+         {
+            throw (new IllegalArgumentException("Variable " + entry.name() + " must be " + entry.type()));
+         }
+      }
+      return (result);
+   }
+
+   /*******************************************************************************
+    ** Strict boolean coercion avoids silently turning misspellings into false.
+    *******************************************************************************/
+   private static Boolean booleanValue(Object value)
+   {
+      if(!List.of("true", "false").contains(value.toString().toLowerCase(java.util.Locale.ROOT)))
+      {
+         throw (new IllegalArgumentException());
+      }
+      return (Boolean.valueOf(value.toString()));
+   }
+
+   /*******************************************************************************
+    ** Only absolute web URLs are allowed in variable and brand URL fields.
+    *******************************************************************************/
+   public static String url(String value)
+   {
+      return (url(value, false));
+   }
+
+   /*******************************************************************************
+    ** The same URL policy, optionally allowing mailto only for link destinations.
+    *******************************************************************************/
+   public static String url(String value, boolean allowMailto)
+   {
+      URI uri = URI.create(value);
+      if(allowMailto && "mailto".equalsIgnoreCase(uri.getScheme()) && !uri.getRawSchemeSpecificPart().isBlank()
+         && !value.matches("(?is).*%(?:0a|0d).*"))
+      {
+         return (value);
+      }
+      if(uri.getScheme() == null || !List.of("http", "https").contains(uri.getScheme().toLowerCase(java.util.Locale.ROOT)) || uri.getHost() == null || uri.getUserInfo() != null)
+      {
+         throw (new IllegalArgumentException("URL must use http or https"));
+      }
+      return (value);
+   }
+
+   /*******************************************************************************
+    ** Parse process inputs and sample values; never echo malformed JSON.
+    *******************************************************************************/
+   public static Map values(String json)
+   {
+      try
+      {
+         Map result = JSON.readValue(json == null || json.isBlank() ? "{}" : json, new TypeReference>() {});
+         if(result == null)
+         {
+            throw (new IllegalArgumentException());
+         }
+         return (result);
+      }
+      catch(Exception e)
+      {
+         throw (new IllegalArgumentException("Variables must be a JSON object"));
+      }
+   }
+
+   /*******************************************************************************
+    ** Serialize safe snapshots using the existing Jackson dependency.
+    *******************************************************************************/
+   public static String json(Object value)
+   {
+      try
+      {
+         return (JSON.writeValueAsString(value));
+      }
+      catch(Exception e)
+      {
+         throw (new IllegalArgumentException("Could not serialize email variables"));
+      }
+   }
+}
diff --git a/src/main/java/com/kingsrook/qbits/email/transport/SesEmailTransport.java b/src/main/java/com/kingsrook/qbits/email/transport/SesEmailTransport.java
index 671c5aa..e499533 100644
--- a/src/main/java/com/kingsrook/qbits/email/transport/SesEmailTransport.java
+++ b/src/main/java/com/kingsrook/qbits/email/transport/SesEmailTransport.java
@@ -64,9 +64,9 @@ public class SesEmailTransport implements EmailTransport
    }
 
    /*******************************************************************************
-    ** Package-visible fake-client seam.
+    ** Injectable client factory for embedded hosts and local demonstrations.
     *******************************************************************************/
-   SesEmailTransport(Function clientFactory)
+   public SesEmailTransport(Function clientFactory)
    {
       this.clientFactory = clientFactory;
    }
diff --git a/src/test/java/com/kingsrook/qbits/email/EmailDeliveryTest.java b/src/test/java/com/kingsrook/qbits/email/EmailDeliveryTest.java
index 1492106..bf1e83f 100644
--- a/src/test/java/com/kingsrook/qbits/email/EmailDeliveryTest.java
+++ b/src/test/java/com/kingsrook/qbits/email/EmailDeliveryTest.java
@@ -544,6 +544,8 @@ class EmailDeliveryTest
    @Test
    void suppressionScopeCategoryExpiryAndPartial() throws Exception
    {
+      EmailRuntime.config().withPublicBaseUrl("https://app.example.com").withSecretResolver(name -> "test-unsubscribe-key");
+      H2Support.insert(EmailTables.BRAND, new QRecord().withValue("name", "postal").withValue("isDefault", true).withValue("postalAddress", "123 Test Street"));
       H2Support.insert(
             EmailTables.SUPPRESSION,
             new QRecord()
diff --git a/src/test/java/com/kingsrook/qbits/email/EmailHomePolishTest.java b/src/test/java/com/kingsrook/qbits/email/EmailHomePolishTest.java
new file mode 100644
index 0000000..9ca402a
--- /dev/null
+++ b/src/test/java/com/kingsrook/qbits/email/EmailHomePolishTest.java
@@ -0,0 +1,256 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.email;
+
+
+import java.time.Clock;
+import java.time.Instant;
+import java.time.ZoneOffset;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import com.kingsrook.qbits.email.app.EmailHome;
+import com.kingsrook.qbits.email.metadata.EmailEnums;
+import com.kingsrook.qbits.email.metadata.EmailTables;
+import com.kingsrook.qbits.email.render.ContrastCheck;
+import com.kingsrook.qbits.email.render.EmailLayout;
+import com.kingsrook.qbits.email.outbox.EmailCategory;
+import com.kingsrook.qbits.email.support.SystemWrites;
+import com.kingsrook.qqq.backend.core.actions.dashboard.RenderWidgetAction;
+import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
+import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
+import com.kingsrook.qqq.backend.core.actions.values.SearchPossibleValueSourceAction;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
+import com.kingsrook.qqq.backend.core.model.actions.values.SearchPossibleValueSourceInput;
+import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.RawHTML;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import org.jsoup.Jsoup;
+import org.jsoup.nodes.Element;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import static org.assertj.core.api.Assertions.assertThat;
+
+
+/*******************************************************************************
+ ** Home data and rendered output, real record labels, and branded preview contrast.
+ *******************************************************************************/
+class EmailHomePolishTest
+{
+   private static final Instant NOW = Instant.parse("2026-10-14T02:12:00Z");
+
+   /*******************************************************************************
+    ** Use a fixed UTC clock and isolated database for exact assertions.
+    *******************************************************************************/
+   @BeforeEach
+   void setup() throws Exception
+   {
+      H2Support.setup(c -> { });
+      EmailRuntime.setClock(Clock.fixed(NOW, ZoneOffset.UTC));
+      H2Support.provider(25);
+   }
+
+   /*******************************************************************************
+    ** Clear session and clock.
+    *******************************************************************************/
+   @AfterEach
+   void cleanup()
+   {
+      QContext.clear();
+      EmailRuntime.setClock(null);
+   }
+
+   /*******************************************************************************
+    ** Exact counts, plain labels, large number first, and failures alone are red.
+    *******************************************************************************/
+   @Test
+   void tilesAndOrderedSections() throws Exception
+   {
+      SystemWrites.insertOne(EmailTables.MESSAGE, new QRecord().withValue("status", "DEAD_LETTER")
+         .withValue("sentDate", NOW.minusSeconds(86400)).withValue("deliveredDate", NOW.plusSeconds(1))
+         .withValue("deadLetterDate", NOW).withValue("bouncedDate", NOW.minusSeconds(7 * 86400))
+         .withValue("complainedDate", NOW.minusSeconds(7 * 86400 + 1)));
+      H2Support.insert(EmailTables.SUPPRESSION, new QRecord().withValue("address", "a@example.test").withValue("scope", "ALL").withValue("reason", "MANUAL"));
+      var document = Jsoup.parse(EmailHome.overview().getHtml());
+      var tiles = document.select("body > div > div > div");
+      assertThat(tiles).extracting(Element::text).containsExactly("1 Sent (24 h)", "0 Delivered (24 h)", "1 Dead letters (24 h)", "1 Bounced (7 d)", "0 Complaints (7 d)", "1 Suppressed addresses");
+      for(int i = 0; i < tiles.size(); i++)
+      {
+         assertThat(tiles.get(i).child(0).attr("style")).contains("font-size:1.6rem");
+         assertThat(tiles.get(i).child(0).attr("style").contains("#d64545")).isEqualTo(i == 2 || i == 3);
+      }
+      assertThat(document.select("h3")).extracting(Element::text).containsExactly("Sends per hour, last 24 hours", "Provider health", "Recent messages", "Suppressions added, last 30 days");
+      assertThat(QContext.getQInstance().getApp("email").getWidgets()).containsExactly("emailHome");
+   }
+
+   /*******************************************************************************
+    ** Every attempt outcome and suppression reason feeds its actual UTC bucket.
+    *******************************************************************************/
+   @Test
+   void stackedBarsAndCompactAxes() throws Exception
+   {
+      int number = 0;
+      for(String outcome : List.of("SUCCEEDED", "TRANSIENT_FAILURE", "PROVIDER_FAILURE", "PERMANENT_FAILURE", "LEASE_EXPIRED", "IN_FLIGHT"))
+      {
+         SystemWrites.insertOne(EmailTables.ATTEMPT, new QRecord().withValue("messageId", 1).withValue("attemptNumber", ++number)
+            .withValue("outcome", outcome).withValue("endDate", NOW));
+      }
+      for(var reason : EmailEnums.SuppressionReason.values())
+      {
+         H2Support.insert(EmailTables.SUPPRESSION, new QRecord().withValue("address", reason.name() + "@example.test").withValue("scope", "ALL").withValue("reason", reason.name()));
+      }
+      var hourly = EmailHome.hourly().getChartData();
+      assertThat(hourly.getLabels()).hasSize(24).allMatch(s -> s.matches("[0-2][0-9]:00"));
+      assertThat(hourly.getLabels().getLast()).isEqualTo("02:00");
+      assertThat(hourly.getDatasets().get(0).getData().getLast()).isEqualTo(1);
+      assertThat(hourly.getDatasets().get(1).getData().getLast()).isEqualTo(4);
+      var suppressions = EmailHome.suppressions().getChartData();
+      assertThat(suppressions.getLabels()).hasSize(30).endsWith("Oct 14");
+      assertThat(suppressions.getDatasets()).extracting(d -> d.getLabel()).containsExactly("Hard bounce", "Complaint", "Unsubscribe", "Manual");
+      assertThat(suppressions.getDatasets()).allSatisfy(d -> assertThat(d.getData().getLast()).isEqualTo(1));
+      String html = EmailHome.overview().getHtml();
+      assertThat(html).contains("flex-direction:column-reverse", "02:00; Sent: 1; Failed: 4", "Oct 14; Hard bounce: 1; Complaint: 1; Unsubscribe: 1; Manual: 1")
+         .doesNotContain(" div")).hasSize(54).allSatisfy(e -> assertThat(e.attr("style")).contains("flex:1 1 0;min-width:0"));
+   }
+
+   /*******************************************************************************
+    ** Twenty newest linked rows wrap long text, mask addresses, and hide PHI.
+    *******************************************************************************/
+   @Test
+   void recentRowsAndProviderHealth() throws Exception
+   {
+      for(int i = 0; i < 21; i++)
+      {
+         SystemWrites.insertOne(EmailTables.MESSAGE, new QRecord().withValue("status", i == 20 ? "RETRY_WAIT" : "SENT")
+            .withValue("subject", i == 19 ? "private-canary" : "Subject <" + i + ">" + "x".repeat(150))
+            .withValue("phiApproved", i == 19).withValue("toAddresses", "Person "));
+      }
+      SystemWrites.updateOne(EmailTables.PROVIDER, new QRecord().withValue("id", 1).withValue("lastSuccessDate", NOW)
+         .withValue("healthStatus", "UNHEALTHY").withValue("lastFailureMessage", "").withValue("maxPerMinute", 2));
+      SystemWrites.insertOne(EmailTables.ATTEMPT, new QRecord().withValue("messageId", 1).withValue("attemptNumber", 1).withValue("providerId", 1).withValue("startDate", NOW));
+      String html = EmailHome.overview().getHtml();
+      var document = Jsoup.parse(html);
+      var rows = document.select("a[href^=/app/email_message/]");
+      assertThat(rows).hasSize(20);
+      assertThat(rows.first().attr("href")).isEqualTo("/app/email_message/21");
+      assertThat(rows.last().attr("href")).isEqualTo("/app/email_message/2");
+      assertThat(rows.first().text()).contains("When", "To", "Subject", "Status", "Waiting to retry");
+      assertThat(html).contains("Secure message", "p***@", "Oct 14, 02:12 UTC", "Unhealthy", "<broken>", "min-width:0", "overflow-wrap:anywhere")
+         .doesNotContain("private-canary", "Person <", "RETRY_WAIT", "2026-10-14T");
+      assertThat(document.select("section").get(1).text()).contains("Last success Oct 14, 02:12 UTC", "Per-minute headroom 1");
+      SystemWrites.updateOne(EmailTables.PROVIDER, new QRecord().withValue("id", 1).withValue("enabled", false).withValue("maxPerMinute", null));
+      assertThat(EmailHome.overview().getHtml()).contains("Disabled", "No limit");
+      QContext.getQSession().setPermissions(Set.of("email.view"));
+      assertThat(EmailHome.overview().getHtml()).contains("Provider health needs email.admin").doesNotContain("<broken>");
+   }
+
+   /*******************************************************************************
+    ** Query, get and possible-value labels use the same human-readable titles.
+    *******************************************************************************/
+   @Test
+   void recordLabels() throws Exception
+   {
+      QRecord version = EmailTemplatesTest.draft("welcome", "MARKDOWN", "[]", "{}", "Hello");
+      assertThat(get(EmailTables.VERSION, version.getValueInteger("id")).getRecordLabel()).isEqualTo("welcome v1");
+      SystemWrites.updateOne(EmailTables.VERSION, new QRecord().withValue("id", version.getValue("id")).withValue("locale", "fr"));
+      assertThat(get(EmailTables.VERSION, version.getValueInteger("id")).getRecordLabel()).isEqualTo("welcome v1 (fr)");
+      assertThat(get(EmailTables.PROVIDER, 1).getRecordLabel()).isEqualTo("Relay");
+      SystemWrites.updateOne(EmailTables.PROVIDER, new QRecord().withValue("id", 1).withValue("label", ""));
+      assertThat(get(EmailTables.PROVIDER, 1).getRecordLabel()).isEqualTo("relay");
+      for(String subject : List.of("A useful subject", "", "enc:v1:private"))
+      {
+         SystemWrites.insertOne(EmailTables.MESSAGE, new QRecord().withValue("subject", subject));
+      }
+      assertThat(get(EmailTables.MESSAGE, 1).getRecordLabel()).isEqualTo("A useful subject");
+      assertThat(get(EmailTables.MESSAGE, 2).getRecordLabel()).isEqualTo("Message 2");
+      assertThat(get(EmailTables.MESSAGE, 3).getRecordLabel()).isEqualTo("Secure message");
+      var queried = new QueryAction().execute(new QueryInput(EmailTables.MESSAGE).withShouldGenerateDisplayValues(true)).getRecords();
+      assertThat(queried).extracting(QRecord::getRecordLabel).containsExactly("A useful subject", "Message 2", "Secure message");
+      var possible = new SearchPossibleValueSourceAction().execute(new SearchPossibleValueSourceInput().withPossibleValueSourceName(EmailTables.VERSION).withSearchTerm("fr"));
+      assertThat(possible.getResults()).extracting(v -> v.getLabel()).contains("welcome v1 (fr)");
+      assertThat(EmailEnums.MessageStatus.RETRY_WAIT.getPossibleValueLabel()).isEqualTo("Waiting to retry");
+      assertThat(EmailEnums.MessageStatus.DEAD_LETTER.getPossibleValueLabel()).isEqualTo("Dead letter");
+   }
+
+   /*******************************************************************************
+    ** The demo brand's actual inline preview matches the email dark media query.
+    *******************************************************************************/
+   @Test
+   void demoPreviewButtonContrast() throws Exception
+   {
+      QRecord brand = H2Support.insert(EmailTables.BRAND, new QRecord().withValue("name", "Kof22").withValue("label", "Kof22")
+         .withValue("primaryColor", "#2459a6").withValue("buttonTextColor", "#ffffff")
+         .withValue("textColor", "#172033").withValue("backgroundColor", "#ffffff")
+         .withValue("darkPrimaryColor", "#93c5fd").withValue("darkButtonTextColor", "#172033")
+         .withValue("darkBackgroundColor", "#111827").withValue("darkTextColor", "#f9fafb").withValue("isDefault", true));
+      QRecord version = EmailTemplatesTest.draft("preview", "MARKDOWN", "[]", "{}", "[Visit Kof22](https://example.test){.button}");
+      String html = ((RawHTML) new RenderWidgetAction().execute(new RenderWidgetInput().withWidgetMetaData(QContext.getQInstance().getWidget("emailTemplatePreview"))
+         .withUrlParams(Map.of("id", version.getValueString("id")))).getWidgetData()).getHtml();
+      var document = Jsoup.parse(html);
+      Element dark = document.select("h4").stream().filter(e -> e.text().startsWith("Dark")).findFirst().orElseThrow().nextElementSibling();
+      Element button = dark.select("a[href=https://example.test]").first();
+      assertThat(button).isNotNull();
+      String foreground = css(button, "color");
+      String background = css(button.parent(), "background-color");
+      assertThat(foreground).isEqualTo("#172033");
+      assertThat(background).isEqualTo("#93c5fd");
+      assertThat(ContrastCheck.ratio(foreground, background)).isGreaterThanOrEqualTo(4.5);
+      assertThat(dark.attr("style")).contains("background-color:#111827", "color:#f9fafb");
+      assertThat(EmailLayout.render("Body", "", brand, EmailCategory.TRANSACTIONAL, null))
+         .contains(".email-button{background-color:" + background + "!important}", ".email-button a{color:" + foreground + "!important}");
+   }
+
+   /*******************************************************************************
+    ** Missing dark button text chooses an accessible colour for either primary.
+    *******************************************************************************/
+   @Test
+   void accessibleDarkButtonFallback()
+   {
+      for(String primary : List.of("#93c5fd", "#172033", "#777777"))
+      {
+         QRecord brand = new QRecord().withValue("darkPrimaryColor", primary);
+         assertThat(ContrastCheck.ratio(ContrastCheck.color(brand, "darkButtonTextColor"), primary)).isGreaterThanOrEqualTo(4.5);
+         ContrastCheck.validate(brand);
+      }
+      assertThat(ContrastCheck.color(new QRecord().withValue("darkPrimaryColor", "#93c5fd").withValue("darkButtonTextColor", "#172033"), "darkButtonTextColor")).isEqualTo("#172033");
+   }
+
+   /*******************************************************************************
+    ** Exercise real get/display-value actions, not just metadata declarations.
+    *******************************************************************************/
+   private static QRecord get(String table, int id) throws Exception
+   {
+      return (new GetAction().execute(new GetInput(table).withPrimaryKey(id).withShouldGenerateDisplayValues(true)).getRecord());
+   }
+
+   /*******************************************************************************
+    ** Read the rendered inline pair; remove CSS priority before computing ratio.
+    *******************************************************************************/
+   private static String css(Element element, String property)
+   {
+      return (List.of(element.attr("style").split(";")).stream().map(String::trim).filter(s -> s.startsWith(property + ":"))
+         .findFirst().orElseThrow().substring(property.length() + 1).replace("!important", "").trim());
+   }
+}
diff --git a/src/test/java/com/kingsrook/qbits/email/EmailMetadataTest.java b/src/test/java/com/kingsrook/qbits/email/EmailMetadataTest.java
index 80588b9..bf5166b 100644
--- a/src/test/java/com/kingsrook/qbits/email/EmailMetadataTest.java
+++ b/src/test/java/com/kingsrook/qbits/email/EmailMetadataTest.java
@@ -105,7 +105,7 @@ class EmailMetadataTest
       assertThat(
                   new EmailApp.HomeRenderer()
                         .render(
-                              new com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput()))
+                              new com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput().withWidgetMetaData(instance.getWidget(EmailApp.HOME_WIDGET))))
             .isNotNull();
       assertThat(EmailRuntime.isInstalled()).isTrue();
       assertThat(EmailRuntime.config(new QInstance())).isNull();
@@ -215,7 +215,7 @@ class EmailMetadataTest
                   .withSubject("s")
                   .withMarkdownBody("\n\n**Hello**");
       assertThat(EmailService.send(new EmailRequest().withTemplateName("later")).reasons())
-            .contains("templates arrive in part 2");
+            .contains("Unknown email template");
       assertThat(EmailService.send(null).status()).isEqualTo(EmailSendResult.Status.REFUSED);
       assertThat(EmailService.send(new EmailRequest()).status())
             .isEqualTo(EmailSendResult.Status.REFUSED);
diff --git a/src/test/java/com/kingsrook/qbits/email/EmailProcessesTest.java b/src/test/java/com/kingsrook/qbits/email/EmailProcessesTest.java
index 0fa7fcc..6f92573 100644
--- a/src/test/java/com/kingsrook/qbits/email/EmailProcessesTest.java
+++ b/src/test/java/com/kingsrook/qbits/email/EmailProcessesTest.java
@@ -146,9 +146,11 @@ class EmailProcessesTest
    @Test
    void backendSendAndScheduledJobs() throws Exception
    {
+      assertThat(QContext.getQInstance().getProcess(EmailProcesses.SEND_TEMPLATED).getBackendStep("send").getInputMetaData().getField("category")).isEmpty();
+      assertThat(QContext.getQInstance().getProcess(EmailProcesses.SEND_AD_HOC).getBackendStep("send").getInputMetaData().getField("category")).isPresent();
       var queued =
             run(
-                  EmailProcesses.SEND_TEMPLATED,
+                  EmailProcesses.SEND_AD_HOC,
                   Map.of(
                         "to",
                         "a@example.com",
@@ -159,6 +161,7 @@ class EmailProcessesTest
                         "category",
                         "NOTIFICATION"));
       assertThat(queued.getValues().get("status")).isEqualTo("QUEUED");
+      assertThat(H2Support.all(EmailTables.MESSAGE).getFirst().getValueString("category")).isEqualTo("NOTIFICATION");
       run(EmailProcesses.DISPATCH, Map.of());
       assertThat(mail.waitForIncomingEmail(2000, 1)).isTrue();
       run(EmailProcesses.MAINTAIN, Map.of());
diff --git a/src/test/java/com/kingsrook/qbits/email/EmailRenderingTest.java b/src/test/java/com/kingsrook/qbits/email/EmailRenderingTest.java
new file mode 100644
index 0000000..7297418
--- /dev/null
+++ b/src/test/java/com/kingsrook/qbits/email/EmailRenderingTest.java
@@ -0,0 +1,175 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.email;
+
+import java.math.BigDecimal;
+import java.util.List;
+import java.util.Map;
+import com.kingsrook.qbits.email.outbox.EmailCategory;
+import com.kingsrook.qbits.email.render.ContrastCheck;
+import com.kingsrook.qbits.email.render.EmailBrands;
+import com.kingsrook.qbits.email.render.EmailLayout;
+import com.kingsrook.qbits.email.render.EmailMarkdown;
+import com.kingsrook.qbits.email.render.HtmlToText;
+import com.kingsrook.qbits.email.template.CliMjmlCompiler;
+import com.kingsrook.qbits.email.template.EmailTemplateEngine;
+import com.kingsrook.qbits.email.template.VariableSchema;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import org.junit.jupiter.api.Test;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+/*******************************************************************************
+ ** Template grammar, content boundaries, accessible layout and validation.
+ *******************************************************************************/
+class EmailRenderingTest
+{
+   private final EmailTemplateEngine engine = new EmailTemplateEngine();
+   private final VariableSchema schema = new VariableSchema("[{\"name\":\"name\",\"type\":\"STRING\"},{\"name\":\"items\",\"type\":\"LIST\"},{\"name\":\"record\",\"type\":\"RECORD\"}]");
+
+   /*******************************************************************************
+    ** Escaping belongs to the output context, and fallbacks are escaped too.
+    *******************************************************************************/
+   @Test
+   void escapingAndFallbacks()
+   {
+      String attack = " & \"test\"";
+      var parsed = engine.parse("{{name}} {{record.name|}}", schema);
+      assertThat(parsed.render(Map.of("name", attack), EmailTemplateEngine.Context.HTML)).contains("<img", "'", """, "<fallback>").doesNotContain("x"), EmailTemplateEngine.Context.MARKDOWN);
+      String html = EmailMarkdown.render(markdown, new QRecord());
+      assertThat(html).doesNotContain(" engine.parse("line\n{{missing}}", schema)).hasMessageContaining("Line 2").hasMessageContaining("missing");
+      assertThatThrownBy(() -> engine.parse("\n{{#if name}}", schema)).hasMessageContaining("Line 2: Unclosed");
+      for(String source : List.of("{{#if name}}{{/each}}", "{{/if}}", "{{this}}", "{{@index}}", "{{{name}}}", "{{> partial}}", "{{name", "{{#if name|fallback}}"))
+      {
+         assertThatThrownBy(() -> engine.parse(source, schema)).isInstanceOf(IllegalArgumentException.class);
+      }
+   }
+
+   /*******************************************************************************
+    ** Schema coercion is explicit, including strict URLs and required values.
+    *******************************************************************************/
+   @Test
+   void schemaValidation()
+   {
+      String json = "[{\"name\":\"n\",\"type\":\"NUMBER\",\"required\":true},{\"name\":\"d\",\"type\":\"DATE\"},{\"name\":\"t\",\"type\":\"DATE_TIME\"},{\"name\":\"b\",\"type\":\"BOOLEAN\"},{\"name\":\"url\",\"type\":\"URL\"}]";
+      VariableSchema schema = new VariableSchema(json);
+      var values = schema.validate(VariableSchema.values("{\"n\":\"1.20\",\"d\":\"2026-10-13\",\"t\":\"2026-10-13T12:00:00Z\",\"b\":\"true\",\"url\":\"https://example.com/path\"}"));
+      assertThat(values.get("n")).isEqualTo(new BigDecimal("1.20"));
+      assertThat(values.get("b")).isEqualTo(true);
+      assertThatThrownBy(() -> schema.validate(Map.of())).hasMessageContaining("Required variable n");
+      assertThatThrownBy(() -> schema.validate(Map.of("n", 1, "b", "yes"))).hasMessageContaining("BOOLEAN");
+      for(String url : List.of("javascript:alert(1)", "data:text/html,x", "/relative", "https://user:pass@example.com", "https://"))
+      {
+         assertThatThrownBy(() -> VariableSchema.url(url)).isInstanceOf(IllegalArgumentException.class);
+      }
+      for(String invalid : List.of("{}", "[{}]", "[{\"name\":\"x\",\"type\":\"CODE\"}]", "[{\"name\":\"x\",\"type\":\"STRING\",\"table\":\"t\"}]"))
+      {
+         assertThatThrownBy(() -> new VariableSchema(invalid)).hasMessage("Invalid variables schema");
+      }
+      assertThatThrownBy(() -> VariableSchema.values("[]")).hasMessage("Variables must be a JSON object");
+      assertThat(schema.knows("brand.site_url")).isTrue();
+      assertThat(schema.knows("brand.missing")).isFalse();
+   }
+
+   /*******************************************************************************
+    ** CommonMark extensions and table buttons cannot enable raw HTML.
+    *******************************************************************************/
+   @Test
+   void markdownAndText()
+   {
+      String html = EmailMarkdown.render("# Heading\n\n\n\n| A | B |\n|---|---|\n| One | Two |\n\n[Open](https://example.com){.button}\n\nwww.example.com\n\n- First\n- Second", new QRecord());
+      assertThat(html).contains("
", "<script>", "table-layout:fixed", "email-button", "href=\"https://example.com\"", "http://www.example.com").doesNotContain(""))
+         .contains("A & B", "C").doesNotContain("bad-css", "bad-code");
+   }
+
+   /*******************************************************************************
+    ** Both schemes, logo variants and category-specific footers are present.
+    *******************************************************************************/
+   @Test
+   void layoutAndContrast()
+   {
+      QRecord brand = new QRecord().withValue("label", "Brand").withValue("logoUrl", "https://example.com/logo.png")
+         .withValue("logoDarkUrl", "https://example.com/dark.png").withValue("logoAlt", "Brand logo")
+         .withValue("footerMarkdown", "Thanks **friend**").withValue("postalAddress", "123 Main Street")
+         .withValue("socialLinks", "[{\"label\":\"Social\",\"url\":\"https://example.com/social\"}]");
+      ContrastCheck.validate(brand);
+      assertThat(ContrastCheck.ratio("#000000", "#ffffff")).isEqualTo(21);
+      for(EmailCategory category : EmailCategory.values())
+      {
+         String html = EmailLayout.render("
Content
", "Preheader", brand, category, "https://example.com/unsubscribe");
+         assertThat(html).contains("prefers-color-scheme:dark", "color-scheme", "supported-color-schemes", "[data-ogsc]", "[data-ogsb]", "max-width:600px", "email-logo-dark", "Preheader", "Social").doesNotContain("background-image");
+         assertThat(html.contains("Unsubscribe")).isEqualTo(category == EmailCategory.NOTIFICATION || category == EmailCategory.MARKETING);
+         assertThat(html.contains("123 Main Street")).isEqualTo(category == EmailCategory.MARKETING);
+      }
+      String appended = EmailLayout.appendFooter("content", brand, EmailCategory.NOTIFICATION, "https://example.com/u");
+      assertThat(appended.indexOf("Unsubscribe")).isLessThan(appended.indexOf(""));
+      assertThat(EmailLayout.appendFooter("fragment", brand, EmailCategory.TRANSACTIONAL, "")).isEqualTo("fragment");
+      assertThatThrownBy(() -> ContrastCheck.validate(new QRecord().withValue("darkTextColor", "#555555"))).hasMessageContaining("dark mode; it needs 4.5:1");
+      assertThatThrownBy(() -> ContrastCheck.validate(new QRecord().withValue("primaryColor", "#ffffff"))).hasMessageContaining("Button text");
+      assertThatThrownBy(() -> EmailBrands.validate(new QRecord().withValue("logoUrl", "javascript:x"))).hasMessageContaining("http or https");
+      assertThatThrownBy(() -> EmailBrands.validate(new QRecord().withValue("textColor", "red;display:none"))).hasMessageContaining("six-digit");
+      assertThatThrownBy(() -> EmailLayout.footer(new QRecord(), EmailCategory.MARKETING, "")).hasMessageContaining("postal address");
+   }
+
+   /*******************************************************************************
+    ** Missing or failing CLI tools return only a safe diagnostic.
+    *******************************************************************************/
+   @Test
+   void cliFailure()
+   {
+      assertThatThrownBy(() -> new CliMjmlCompiler("missing-mjml-binary-for-test").compile("private-source"))
+         .hasMessage("MJML compilation failed or exceeded 20 seconds");
+   }
+}
diff --git a/src/test/java/com/kingsrook/qbits/email/EmailReviewFixesTest.java b/src/test/java/com/kingsrook/qbits/email/EmailReviewFixesTest.java
new file mode 100644
index 0000000..58f4c0d
--- /dev/null
+++ b/src/test/java/com/kingsrook/qbits/email/EmailReviewFixesTest.java
@@ -0,0 +1,285 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.email;
+
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import com.kingsrook.qbits.email.metadata.EmailTables;
+import com.kingsrook.qbits.email.outbox.EmailCategory;
+import com.kingsrook.qbits.email.outbox.EmailRequest;
+import com.kingsrook.qbits.email.outbox.EmailSendResult;
+import com.kingsrook.qbits.email.outbox.EmailService;
+import com.kingsrook.qbits.email.phi.EmailBodyEncryption;
+import com.kingsrook.qbits.email.render.EmailBrands;
+import com.kingsrook.qbits.email.render.EmailMarkdown;
+import com.kingsrook.qbits.email.support.EmailStore;
+import com.kingsrook.qbits.email.support.SystemWrites;
+import com.kingsrook.qbits.email.template.EmailTemplateEngine;
+import com.kingsrook.qbits.email.template.EmailTemplateRenderer;
+import com.kingsrook.qbits.email.template.EmailVersions;
+import com.kingsrook.qbits.email.template.VariableSchema;
+import com.kingsrook.qqq.backend.core.actions.dashboard.RenderWidgetAction;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.RawHTML;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+
+/*******************************************************************************
+ ** Regression checks for the PR review's trust and resource boundaries.
+ *******************************************************************************/
+class EmailReviewFixesTest
+{
+   /*******************************************************************************
+    ** Isolated records and local test keys.
+    *******************************************************************************/
+   @BeforeEach
+   void setup() throws Exception
+   {
+      H2Support.setup(c -> c.withSecretResolver(name -> "test-key").withPublicBaseUrl("https://example.com"));
+      H2Support.provider(25);
+   }
+
+   /*******************************************************************************
+    ** Release request state.
+    *******************************************************************************/
+   @AfterEach
+   void cleanup()
+   {
+      QContext.clear();
+      EmailRuntime.setClock(null);
+   }
+
+   /*******************************************************************************
+    ** Template categories cannot be downgraded, including notification suppression.
+    *******************************************************************************/
+   @Test
+   void categoryOverrideRefused() throws Exception
+   {
+      for(String category : List.of("MARKETING", "NOTIFICATION"))
+      {
+         QRecord version = EmailTemplatesTest.draft(category, "HTML", "[]", "{}", "
Hello
");
+         SystemWrites.updateOne(EmailTables.TEMPLATE, new QRecord().withValue("id", version.getValue("templateId")).withValue("category", category));
+         if("MARKETING".equals(category))
+         {
+            H2Support.insert(EmailTables.BRAND, new QRecord().withValue("name", "brand").withValue("postalAddress", "123 Main St").withValue("isDefault", true));
+         }
+         EmailVersions.activate(version.getValueInteger("id"));
+         var refused = EmailService.send(new EmailRequest().withTemplateName(category).withTo("p@example.com").withCategory(EmailCategory.TRANSACTIONAL));
+         assertThat(refused.status()).isEqualTo(EmailSendResult.Status.REFUSED);
+         assertThat(refused.reasons()).contains("Template category cannot be overridden");
+         assertThat(EmailService.send(new EmailRequest().withTemplateName(category).withTo("p@example.com")).status()).isEqualTo(EmailSendResult.Status.QUEUED);
+      }
+   }
+
+   /*******************************************************************************
+    ** Nested loops share work and byte limits, and nesting stops at eight.
+    *******************************************************************************/
+   @Test
+   void sharedBudgetAndNesting()
+   {
+      var engine = new EmailTemplateEngine();
+      var schema = new VariableSchema("[{\"name\":\"items\",\"type\":\"LIST\"},{\"name\":\"value\",\"type\":\"STRING\"}]");
+      var loop = engine.parse("{{#each items}}{{#each items}}{{/each}}{{/each}}", schema);
+      assertThatThrownBy(() -> loop.render(Map.of("items", Collections.nCopies(100, "")), EmailTemplateEngine.Context.TEXT)).hasMessage("Template output is too large");
+      assertThatThrownBy(() -> engine.parse("{{value}}", schema).render(Map.of("value", "\u00e9".repeat(600_000)), EmailTemplateEngine.Context.TEXT))
+         .hasMessage("Template output is too large");
+      assertThat(engine.parse("{{#if value}}".repeat(8) + "ok" + "{{/if}}".repeat(8), schema).render(Map.of("value", true), EmailTemplateEngine.Context.TEXT)).isEqualTo("ok");
+      assertThatThrownBy(() -> engine.parse("{{#if value}}".repeat(9) + "{{/if}}".repeat(9), schema)).hasMessageContaining("8 levels");
+      var budget = new EmailTemplateEngine.Budget();
+      var parsed = engine.parse("{{value}}", schema);
+      parsed.render(Map.of("value", "x".repeat(600_000)), EmailTemplateEngine.Context.TEXT, budget, new ArrayList<>());
+      assertThatThrownBy(() -> parsed.render(Map.of("value", "x".repeat(600_000)), EmailTemplateEngine.Context.TEXT, budget, new ArrayList<>())).hasMessage("Template output is too large");
+   }
+
+   /*******************************************************************************
+    ** Activation reports budgets and sends refuse rather than storing excess output.
+    *******************************************************************************/
+   @Test
+   void budgetAtLifecycleBoundaries() throws Exception
+   {
+      QRecord version = EmailTemplatesTest.draft("budget", "HTML", "[{\"name\":\"items\",\"type\":\"LIST\"}]", "{}", "{{#each items}}x{{/each}}");
+      EmailVersions.activate(version.getValueInteger("id"));
+      var result = EmailService.send(new EmailRequest().withTemplateName("budget").withTo("p@example.com").withVariable("items", new ArrayList<>(Collections.nCopies(10_001, "x"))));
+      assertThat(result.status()).isEqualTo(EmailSendResult.Status.REFUSED);
+      assertThat(result.reasons()).contains("Template output is too large");
+      QRecord oversized = EmailTemplatesTest.draft("oversized", "HTML", "[]", "{}", "x".repeat(1_048_577));
+      assertThatThrownBy(() -> EmailVersions.activate(oversized.getValueInteger("id"))).hasMessage("Template output is too large");
+   }
+
+   /*******************************************************************************
+    ** Unsafe URL values are omitted with warnings, while normal links still work.
+    *******************************************************************************/
+   @Test
+   void urlContexts()
+   {
+      var engine = new EmailTemplateEngine();
+      var schema = new VariableSchema("[{\"name\":\"value\",\"type\":\"STRING\"}]");
+      for(String attribute : List.of("href", "src", "action", "background"))
+      {
+         for(String quote : List.of("\"", "'"))
+         {
+            var parsed = engine.parse("x", schema);
+            for(String value : List.of("javascript:alert(1)", "data:text/html,evil"))
+            {
+               List warnings = new ArrayList<>();
+               assertThat(parsed.render(Map.of("value", value), EmailTemplateEngine.Context.HTML, new EmailTemplateEngine.Budget(), warnings)).doesNotContain(value).contains(attribute + "=" + quote + quote);
+               assertThat(warnings).containsExactly("Unsafe URL omitted for value");
+            }
+            assertThat(parsed.render(Map.of("value", "https://example.com"), EmailTemplateEngine.Context.HTML)).contains("https://example.com");
+            assertThat(parsed.render(Map.of("value", "mailto:p@example.com"), EmailTemplateEngine.Context.HTML).contains("mailto:")).isEqualTo(attribute.equals("href"));
+         }
+      }
+      for(String source : List.of("x", "x", "x"))
+      {
+         List warnings = new ArrayList<>();
+         String html = engine.parse(source, schema).render(Map.of("value", source.contains("java{{") ? "script:alert(1)" : "javascript:alert(1)"),
+            EmailTemplateEngine.Context.HTML, new EmailTemplateEngine.Budget(), warnings);
+         assertThat(html).contains("href=\"\"").doesNotContain("javascript:");
+         assertThat(warnings).isNotEmpty();
+      }
+   }
+
+   /*******************************************************************************
+    ** Refuse unquoted values and includes before a configured compiler executes.
+    *******************************************************************************/
+   @Test
+   void activationRejectsUnsafeSource() throws Exception
+   {
+      QRecord version = EmailTemplatesTest.draft("unquoted", "HTML", "[]", "{}", "Go");
+      assertThatThrownBy(() -> EmailVersions.activate(version.getValueInteger("id"))).hasMessageContaining("must be quoted");
+      for(String source : List.of("x", "<{{brand.label}}>"))
+      {
+         assertThatThrownBy(() -> EmailTemplateEngine.validateHtml(source)).hasMessageContaining("must be quoted");
+      }
+      EmailRuntime.config().withMjmlCompiler(source -> { throw new AssertionError("Unsafe MJML reached compiler"); });
+      QRecord mjml = EmailTemplatesTest.draft("includes", "MJML", "[]", "{}", "");
+      assertThatThrownBy(() -> EmailVersions.activate(mjml.getValueInteger("id"))).hasMessage("MJML includes are not allowed");
+      assertThat(preview(mjml)).contains("MJML includes are not allowed");
+   }
+
+   /*******************************************************************************
+    ** A viewer can inspect stored output but cannot start a compiler.
+    *******************************************************************************/
+   @Test
+   void viewerNeverCompiles() throws Exception
+   {
+      QRecord version = EmailTemplatesTest.draft("viewer", "MJML", "[]", "{}", "");
+      EmailRuntime.config().withMjmlCompiler(source -> { throw new AssertionError("Viewer compiled MJML"); });
+      QContext.getQSession().setPermissions(Set.of("email.view"));
+      assertThat(preview(version)).contains("Not compiled yet. Activate the version or use Preview as a manager.");
+      QContext.getQSession().setPermissions(Set.of("email.manage"));
+      EmailRuntime.config().withMjmlCompiler(source -> "
Compiled sample
");
+      assertThat(preview(version)).contains("Compiled sample");
+      assertThat(EmailStore.get(EmailTables.VERSION, version.getValueInteger("id"), null).getValue("compiledHtml")).isNull();
+   }
+
+   /*******************************************************************************
+    ** Approved sends retain their template brand and restrict final image hosts.
+    *******************************************************************************/
+   @Test
+   void approvedBrandAndImages() throws Exception
+   {
+      QRecord brand = H2Support.insert(EmailTables.BRAND, new QRecord().withValue("name", "trusted").withValue("label", "Trusted")
+         .withValue("siteUrl", "https://brand.example.com").withValue("logoUrl", "https://other.example.com/logo.png"));
+      H2Support.insert(EmailTables.BRAND, new QRecord().withValue("name", "override").withValue("label", "Override"));
+      QRecord version = EmailTemplatesTest.draft("approved", "HTML", "[]", "{}", "
{{brand.label}}
"
+         + "");
+      SystemWrites.updateOne(EmailTables.TEMPLATE, new QRecord().withValue("id", version.getValue("templateId")).withValue("brandId", brand.getValue("id")));
+      EmailVersions.approve(version.getValueInteger("id"));
+      EmailVersions.activate(version.getValueInteger("id"));
+      SystemWrites.updateOne(EmailTables.PROVIDER, new QRecord().withValue("id", 1).withValue("phiAllowed", true));
+      EmailRuntime.config().withAllowedImageHosts(Set.of("cdn.example.com"));
+      var renderer = EmailTemplateRenderer.prepare(new EmailRequest().withTemplateName("approved").withBrandName("override"));
+      QRecord rendered = renderer.render("preview", null, false);
+      assertThat(rendered.getValue("brandId")).isEqualTo(brand.getValue("id"));
+      assertThat(rendered.getValueString("htmlBody")).contains("Trusted", "brand.example.com/a.png", "cdn.example.com/b.png").doesNotContain("Override", "evil.example.com", "other.example.com");
+      assertThat(renderer.warnings()).contains("Brand logo omitted: image host is not allowed", "Image omitted: image host is not allowed");
+      assertThat(EmailBrands.allowedImage("http://brand.example.com/a.png", brand)).isFalse();
+      assertThat(EmailBrands.allowedImage("https://brand.example.com.evil.com/a.png", brand)).isFalse();
+      assertThatThrownBy(() -> H2Support.insert(EmailTables.BRAND, new QRecord().withValue("name", "badlogo").withValue("logoUrl", "http://example.com/logo"))).hasMessageContaining("https");
+      assertThatThrownBy(() -> H2Support.insert(EmailTables.BRAND, new QRecord().withValue("name", "badsocial").withValue("socialLinks", "[{\"label\":\"X\",\"url\":\"http://example.com\"}]"))).hasMessageContaining("https");
+   }
+
+   /*******************************************************************************
+    ** Button conversion cannot consume an earlier ordinary link.
+    *******************************************************************************/
+   @Test
+   void buttonMatchesOneLink()
+   {
+      String html = EmailMarkdown.render("[First](https://first.example) and [Second](https://second.example){.button}", new QRecord());
+      var document = org.jsoup.Jsoup.parse(html);
+      assertThat(document.select(".email-button a").eachAttr("href")).containsExactly("https://second.example");
+      assertThat(document.select("a")).hasSize(2);
+   }
+
+   /*******************************************************************************
+    ** Authentication binds both row and column without changing the storage prefix.
+    *******************************************************************************/
+   @Test
+   void ciphertextSwapFails()
+   {
+      String first = EmailBodyEncryption.encrypt("first", "htmlBody", "1");
+      String second = EmailBodyEncryption.encrypt("second", "htmlBody", "2");
+      assertThat(first).startsWith("enc:v1:");
+      assertThat(EmailBodyEncryption.decrypt(first, "htmlBody", "1")).isEqualTo("first");
+      assertThatThrownBy(() -> EmailBodyEncryption.decrypt(first, "htmlBody", "2")).hasMessageContaining("could not be opened");
+      assertThatThrownBy(() -> EmailBodyEncryption.decrypt(second, "htmlBody", "1")).hasMessageContaining("could not be opened");
+      assertThatThrownBy(() -> EmailBodyEncryption.decrypt(first, "textBody", "1")).hasMessageContaining("could not be opened");
+   }
+
+   /*******************************************************************************
+    ** Swapped stored ciphertext never reaches a transport during delivery.
+    *******************************************************************************/
+   @Test
+   void swappedRowsCannotBeDispatched() throws Exception
+   {
+      QRecord version = EmailTemplatesTest.draft("protected", "HTML", "[]", "{}", "
Private message
");
+      EmailVersions.approve(version.getValueInteger("id"));
+      EmailVersions.activate(version.getValueInteger("id"));
+      SystemWrites.updateOne(EmailTables.PROVIDER, new QRecord().withValue("id", 1).withValue("phiAllowed", true));
+      var first = EmailService.send(new EmailRequest().withTemplateName("protected").withTo("a@example.com"));
+      var second = EmailService.send(new EmailRequest().withTemplateName("protected").withTo("b@example.com"));
+      QRecord original = EmailStore.get(EmailTables.MESSAGE, first.messageId(), null);
+      SystemWrites.updateOne(EmailTables.MESSAGE, new QRecord().withValue("id", second.messageId()).withValue("htmlBody", original.getValue("htmlBody")));
+      var dispatcher = new com.kingsrook.qbits.email.delivery.EmailDispatcher(new com.kingsrook.qbits.email.transport.EmailTransportRegistry()
+         .withTransport("SMTP", (envelope, settings) -> { throw new AssertionError("Swapped ciphertext reached transport"); }));
+      assertThat(dispatcher.sendNow(second.messageId())).isEqualTo(com.kingsrook.qbits.email.delivery.EmailDispatcher.SendResult.RETRY_WAIT);
+   }
+
+   /*******************************************************************************
+    ** Exercise the actual inline widget with current user permissions.
+    *******************************************************************************/
+   private String preview(QRecord version) throws Exception
+   {
+      return (((RawHTML) new RenderWidgetAction().execute(new RenderWidgetInput().withWidgetMetaData(QContext.getQInstance().getWidget("emailTemplatePreview"))
+         .withUrlParams(Map.of("id", version.getValueString("id")))).getWidgetData()).getHtml());
+   }
+}
diff --git a/src/test/java/com/kingsrook/qbits/email/EmailTemplatesTest.java b/src/test/java/com/kingsrook/qbits/email/EmailTemplatesTest.java
new file mode 100644
index 0000000..007e436
--- /dev/null
+++ b/src/test/java/com/kingsrook/qbits/email/EmailTemplatesTest.java
@@ -0,0 +1,308 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.email;
+
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import com.icegreen.greenmail.util.GreenMail;
+import com.icegreen.greenmail.util.ServerSetup;
+import com.kingsrook.qbits.email.app.EmailHome;
+import com.kingsrook.qbits.email.delivery.EmailDispatcher;
+import com.kingsrook.qbits.email.metadata.EmailTables;
+import com.kingsrook.qbits.email.outbox.EmailRequest;
+import com.kingsrook.qbits.email.outbox.EmailSendResult;
+import com.kingsrook.qbits.email.outbox.EmailService;
+import com.kingsrook.qbits.email.processes.EmailProcesses;
+import com.kingsrook.qbits.email.support.EmailStore;
+import com.kingsrook.qbits.email.support.SystemWrites;
+import com.kingsrook.qbits.email.template.EmailVersions;
+import com.kingsrook.qqq.backend.core.actions.dashboard.RenderWidgetAction;
+import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
+import com.kingsrook.qqq.backend.core.actions.values.SearchPossibleValueSourceAction;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
+import com.kingsrook.qqq.backend.core.model.actions.values.SearchPossibleValueSourceInput;
+import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.RawHTML;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import jakarta.mail.Multipart;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+/*******************************************************************************
+ ** Version lifecycle, permissions, metadata filters, previews and real delivery.
+ *******************************************************************************/
+class EmailTemplatesTest
+{
+   /*******************************************************************************
+    ** Isolated database and sender session.
+    *******************************************************************************/
+   @BeforeEach
+   void setup() throws Exception
+   {
+      H2Support.setup(c -> c.withPublicBaseUrl("https://app.example.com"));
+      H2Support.provider(25);
+   }
+
+   /*******************************************************************************
+    ** Clear thread and clock state.
+    *******************************************************************************/
+   @AfterEach
+   void cleanup()
+   {
+      QContext.clear();
+      EmailRuntime.setClock(null);
+   }
+
+   /*******************************************************************************
+    ** Reusable real template/version fixture.
+    *******************************************************************************/
+   static QRecord draft(String name, String kind, String schema, String samples, String body) throws Exception
+   {
+      QRecord template = H2Support.insert(EmailTables.TEMPLATE, new QRecord().withValue("name", name).withValue("kind", kind).withValue("category", "TRANSACTIONAL"));
+      QRecord version = EmailVersions.newVersion(template.getValueInteger("id"));
+      QRecord update = new QRecord().withValue("id", version.getValue("id")).withValue("subject", "Hello {{brand.label}} {{message.id}}")
+         .withValue("bodySource", body).withValue("variablesSchema", schema).withValue("sampleVariables", samples);
+      var output = new UpdateAction().execute(new UpdateInput(EmailTables.VERSION).withRecords(List.of(update)));
+      assertThat(output.getRecords().getFirst().getErrors()).isNullOrEmpty();
+      return (EmailStore.get(EmailTables.VERSION, version.getValueInteger("id"), null));
+   }
+
+   /*******************************************************************************
+    ** Review/Result creates drafts and atomically retires the old active locale.
+    *******************************************************************************/
+   @Test
+   void lifecycleActionsAndImmutability() throws Exception
+   {
+      QRecord first = draft("welcome", "MARKDOWN", "[]", "{}", "Hello");
+      assertThat(ProcessRunner.run(EmailProcesses.ACTIVATE_VERSION, first.getValue("id")).result().toString()).contains("ACTIVE");
+      var attempted = new UpdateAction().execute(new UpdateInput(EmailTables.VERSION).withRecords(List.of(new QRecord().withValue("id", first.getValue("id")).withValue("bodySource", "changed"))));
+      assertThat(attempted.getRecords().getFirst().getErrors()).isNotEmpty();
+      assertThat(ProcessRunner.run(EmailProcesses.NEW_VERSION, first.getValue("templateId")).result().toString()).contains("DRAFT 2");
+      QRecord next = H2Support.all(EmailTables.VERSION).stream().filter(v -> v.getValueInteger("versionNumber") == 2).findFirst().orElseThrow();
+      assertThat(next.getValueString("bodySource")).isEqualTo("Hello");
+      EmailVersions.activate(next.getValueInteger("id"));
+      assertThat(EmailStore.get(EmailTables.VERSION, first.getValueInteger("id"), null).getValueString("status")).isEqualTo("RETIRED");
+      assertThat(EmailStore.get(EmailTables.TEMPLATE, first.getValueInteger("templateId"), null).getValue("activeVersionId")).isEqualTo(next.getValue("id"));
+      assertThatThrownBy(() -> EmailVersions.activate(first.getValueInteger("id"))).hasMessageContaining("DRAFT");
+      attempted = new UpdateAction().execute(new UpdateInput(EmailTables.VERSION).withRecords(List.of(new QRecord().withValue("id", first.getValue("id")).withValue("status", "DRAFT"))));
+      assertThat(attempted.getRecords().getFirst().getErrors()).isNotEmpty();
+   }
+
+   /*******************************************************************************
+    ** New PHI versions need their own approval and drafts freeze on approval.
+    *******************************************************************************/
+   @Test
+   void phiApprovalAndEditPermission() throws Exception
+   {
+      QRecord version = draft("private_notice", "HTML", "[{\"name\":\"patient\",\"type\":\"STRING\",\"classification\":\"PHI\"}]", "{\"patient\":\"sample\"}", "
{{patient}}
");
+      assertThatThrownBy(() -> EmailVersions.activate(version.getValueInteger("id"))).hasMessageContaining("patient is PHI");
+      QContext.getQSession().setPermissions(Set.of("email.manage"));
+      assertThatThrownBy(() -> EmailVersions.approve(version.getValueInteger("id"))).hasMessageContaining("email.admin");
+      QContext.getQSession().setPermissions(Set.of("email.admin"));
+      assertThat(ProcessRunner.run(EmailProcesses.APPROVE_PHI, version.getValue("id")).result().toString()).contains("Approved");
+      var edit = new UpdateAction().execute(new UpdateInput(EmailTables.VERSION).withRecords(List.of(new QRecord().withValue("id", version.getValue("id")).withValue("bodySource", "replaced"))));
+      assertThat(edit.getRecords().getFirst().getErrors()).isNotEmpty();
+      EmailVersions.activate(version.getValueInteger("id"));
+      QRecord next = EmailVersions.newVersion(version.getValueInteger("templateId"));
+      assertThatThrownBy(() -> EmailVersions.activate(next.getValueInteger("id"))).hasMessageContaining("not approved");
+      SystemWrites.updateOne(EmailTables.TEMPLATE, new QRecord().withValue("id", version.getValue("templateId")).withValue("editPermissionName", "clinical.edit"));
+      assertThatThrownBy(() -> EmailVersions.newVersion(version.getValueInteger("templateId"))).hasMessageContaining("clinical.edit");
+      QContext.getQSession().setPermissions(Set.of("email.admin", "clinical.edit"));
+      assertThat(EmailVersions.newVersion(version.getValueInteger("templateId")).getValueInteger("versionNumber")).isEqualTo(3);
+   }
+
+   /*******************************************************************************
+    ** MJML requires activation compilation, and sending never runs it again.
+    *******************************************************************************/
+   @Test
+   void mjmlCompilerAndLocaleFallback() throws Exception
+   {
+      QRecord version = draft("mjml", "MJML", "[]", "{}", "{{brand.label}}");
+      assertThatThrownBy(() -> EmailVersions.activate(version.getValueInteger("id"))).hasMessageContaining("configured compiler");
+      EmailRuntime.config().withMjmlCompiler(source -> "
{{brand.label}}
");
+      var preview = new RenderWidgetAction().execute(new RenderWidgetInput().withWidgetMetaData(QContext.getQInstance().getWidget("emailTemplatePreview"))
+         .withUrlParams(Map.of("id", version.getValueString("id"))));
+      assertThat(((RawHTML) preview.getWidgetData()).getHtml()).contains("
Email
");
+      assertThat(EmailStore.get(EmailTables.VERSION, version.getValueInteger("id"), null).getValue("compiledHtml")).isNull();
+      EmailVersions.activate(version.getValueInteger("id"));
+      EmailRuntime.config().withMjmlCompiler(source -> { throw new AssertionError("Compiler ran at send time"); });
+      var result = EmailService.send(new EmailRequest().withTemplateName("mjml").withLocale("fr").withTo("p@example.com"));
+      assertThat(result.status()).isEqualTo(EmailSendResult.Status.QUEUED);
+      QRecord row = EmailStore.get(EmailTables.MESSAGE, result.messageId(), null);
+      assertThat(row.getValueString("locale")).isEqualTo("en");
+      assertThat(row.getValueString("htmlBody")).contains("
Email
");
+      assertThat(row.getValueString("subject")).endsWith(result.messageId().toString());
+   }
+
+   /*******************************************************************************
+    ** Branded rendered templates reach a real SMTP sink with both alternatives.
+    *******************************************************************************/
+   @Test
+   void greenMailAndBuiltIn() throws Exception
+   {
+      GreenMail mail = new GreenMail(new ServerSetup(0, "127.0.0.1", "smtp"));
+      try
+      {
+         mail.start();
+         SystemWrites.updateOne(EmailTables.PROVIDER, new QRecord().withValue("id", 1).withValue("port", mail.getSmtp().getPort()));
+         H2Support.insert(EmailTables.BRAND, new QRecord().withValue("name", "blue").withValue("label", "Blue Brand").withValue("primaryColor", "#123456").withValue("isDefault", true));
+         var result = EmailService.send(new EmailRequest().withTemplateName("email_secure_message").withTo("p@example.com").withVariable("link", "https://example.com/secure"));
+         assertThat(result.status()).isEqualTo(EmailSendResult.Status.QUEUED);
+         assertThat(new EmailDispatcher().runOnce().sent()).isEqualTo(1);
+         assertThat(mail.waitForIncomingEmail(2000, 1)).isTrue();
+         Multipart parts = (Multipart) mail.getReceivedMessages()[0].getContent();
+         assertThat(parts.getBodyPart(0).getContent().toString()).contains("Blue Brand", "https://example.com/secure");
+         assertThat(parts.getBodyPart(1).getContent().toString()).contains("#123456", "email-button", "Blue Brand");
+         EmailService.send(new EmailRequest().withTemplateName("email_secure_message").withTo("p@example.com").withVariable("link", "https://example.com/secure"));
+         assertThat(H2Support.all(EmailTables.TEMPLATE)).hasSize(1);
+      }
+      finally
+      {
+         mail.stop();
+      }
+   }
+
+   /*******************************************************************************
+    ** Preview and home render through the standard widget endpoint, with empties.
+    *******************************************************************************/
+   @Test
+   void previewAndHome() throws Exception
+   {
+      assertThat(EmailHome.overview().getHtml()).contains("No messages yet", "Relay").doesNotContain("
Preview text
");
+      var widget = QContext.getQInstance().getWidget("emailTemplatePreview");
+      var output = new RenderWidgetAction().execute(new RenderWidgetInput().withWidgetMetaData(widget).withUrlParams(Map.of("id", version.getValueString("id"))));
+      String html = ((RawHTML) output.getWidgetData()).getHtml();
+      assertThat(html).contains("Light", "Dark", "390px", "600px", "Plain text", "Preview text").doesNotContain(""));
+      assertThat(EmailHome.overview().getHtml()).contains("p***@example.com", "/app/email_message/");
+      QContext.getQSession().setPermissions(Set.of("email.view"));
+      assertThat(EmailHome.overview().getHtml()).contains("needs email.admin");
+   }
+
+   /*******************************************************************************
+    ** Exact metrics use event dates and viewer permissions, with no sample limits.
+    *******************************************************************************/
+   @Test
+   void homeCountsAndBrandPrecedence() throws Exception
+   {
+      java.time.Instant now = java.time.Instant.parse("2026-10-13T12:00:00Z");
+      EmailRuntime.setClock(java.time.Clock.fixed(now, java.time.ZoneOffset.UTC));
+      SystemWrites.insertOne(EmailTables.MESSAGE, new QRecord().withValue("status", "DELIVERED").withValue("subject", "Visible")
+         .withValue("toAddresses", "person@example.com").withValue("sentDate", now.minusSeconds(60)).withValue("deliveredDate", now)
+         .withValue("bouncedDate", now.minusSeconds(2 * 86400)).withValue("complainedDate", now.minusSeconds(8 * 86400)));
+      SystemWrites.insertOne(EmailTables.ATTEMPT, new QRecord().withValue("messageId", 1).withValue("providerId", 1).withValue("attemptNumber", 1)
+         .withValue("outcome", "SUCCEEDED").withValue("startDate", now.minusSeconds(10)).withValue("endDate", now));
+      H2Support.insert(EmailTables.SUPPRESSION, new QRecord().withValue("address", "x@example.com").withValue("scope", "ALL").withValue("reason", "MANUAL"));
+      var tiles = org.jsoup.Jsoup.parse(EmailHome.overview().getHtml()).select("body > div > div > div");
+      assertThat(tiles).extracting(org.jsoup.nodes.Element::text).containsExactly("1 Sent (24 h)", "1 Delivered (24 h)", "0 Dead letters (24 h)", "1 Bounced (7 d)", "0 Complaints (7 d)", "1 Suppressed addresses");
+      assertThat(EmailHome.hourly().getChartData().getDatasets().getFirst().getData()).contains(1);
+      assertThat(EmailHome.suppressions().getChartData().getDatasets().get(3).getData()).contains(1);
+      QRecord defaultBrand = H2Support.insert(EmailTables.BRAND, new QRecord().withValue("name", "default").withValue("label", "Default").withValue("isDefault", true));
+      QRecord appBrand = H2Support.insert(EmailTables.BRAND, new QRecord().withValue("name", "app").withValue("label", "App").withValue("appName", "email"));
+      assertThat(com.kingsrook.qbits.email.render.EmailBrands.resolve(null, null).getValue("id")).isEqualTo(appBrand.getValue("id"));
+      assertThat(com.kingsrook.qbits.email.render.EmailBrands.resolve(null, new QRecord().withValue("brandId", defaultBrand.getValue("id"))).getValue("id")).isEqualTo(defaultBrand.getValue("id"));
+      assertThat(com.kingsrook.qbits.email.render.EmailBrands.resolve("app", new QRecord().withValue("brandId", defaultBrand.getValue("id"))).getValue("id")).isEqualTo(appBrand.getValue("id"));
+      assertThatThrownBy(() -> H2Support.insert(EmailTables.BRAND, new QRecord().withValue("name", "bad").withValue("textColor", "#ffffff")))
+         .hasMessageContaining("needs 4.5:1");
+   }
+
+   /*******************************************************************************
+    ** Signed links bind the real database id, and HTML footers precede body end.
+    *******************************************************************************/
+   @Test
+   void signedNotificationFooter() throws Exception
+   {
+      EmailRuntime.config().withSecretResolver(name -> "QQQ_EMAIL_UNSUBSCRIBE_KEY".equals(name) ? "test-unsubscribe-key" : null);
+      QRecord version = draft("notice", "HTML", "[]", "{}", "
Notice
");
+      SystemWrites.updateOne(EmailTables.TEMPLATE, new QRecord().withValue("id", version.getValue("templateId")).withValue("category", "NOTIFICATION"));
+      EmailVersions.activate(version.getValueInteger("id"));
+      var result = EmailService.send(new EmailRequest().withTemplateName("notice").withTo("person@example.com"));
+      assertThat(result.status()).isEqualTo(EmailSendResult.Status.QUEUED);
+      QRecord message = EmailStore.get(EmailTables.MESSAGE, result.messageId(), null);
+      String link = message.getValueString("unsubscribeUrl");
+      assertThat(link).startsWith("https://app.example.com/qqq/hooks/email/unsubscribe/v1.").doesNotContain("person@example.com");
+      String token = link.substring(link.lastIndexOf('/') + 1);
+      String payload = new String(java.util.Base64.getUrlDecoder().decode(token.split("\\.")[1]), java.nio.charset.StandardCharsets.UTF_8);
+      assertThat(payload).contains("\"messageId\":\"" + result.messageId() + "\"", "NOTIFICATION", "addressHash");
+      assertThat(message.getValueString("htmlBody").indexOf("Unsubscribe")).isLessThan(message.getValueString("htmlBody").indexOf(""));
+      var dispatcher = new EmailDispatcher(new com.kingsrook.qbits.email.transport.EmailTransportRegistry().withTransport("SMTP", (envelope, settings) ->
+      {
+         assertThat(envelope.getHeaders()).containsEntry("List-Unsubscribe", "<" + link + ">");
+         return new com.kingsrook.qbits.email.transport.EmailTransportResult("signed");
+      }));
+      assertThat(dispatcher.sendNow(result.messageId())).isEqualTo(EmailDispatcher.SendResult.SENT);
+      EmailRuntime.config().withSecretResolver(name -> null);
+      assertThat(com.kingsrook.qbits.email.render.EmailUnsubscribeLinks.create("1", "address", "NOTIFICATION")).isNull();
+      assertThatThrownBy(() -> com.kingsrook.qbits.email.render.EmailUnsubscribeLinks.create("1", "address", "MARKETING")).hasMessageContaining("QQQ_EMAIL_UNSUBSCRIBE_KEY");
+   }
+
+   /*******************************************************************************
+    ** Template active-version possible values are scoped to the template record.
+    *******************************************************************************/
+   @Test
+   void templateVersionPossibleValues() throws Exception
+   {
+      possibleValues(EmailTables.TEMPLATE, "activeVersionId", "id");
+   }
+
+   /*******************************************************************************
+    ** Message version possible values follow the selected template input.
+    *******************************************************************************/
+   @Test
+   void messageVersionPossibleValues() throws Exception
+   {
+      possibleValues(EmailTables.MESSAGE, "templateVersionId", "templateId");
+   }
+
+   /*******************************************************************************
+    ** Approval pointers use the same parent-scoped possible values.
+    *******************************************************************************/
+   @Test
+   void approvedVersionPossibleValues() throws Exception
+   {
+      possibleValues(EmailTables.TEMPLATE, "approvedVersionId", "id");
+   }
+
+   /*******************************************************************************
+    ** Execute core's possible-value search with the interpreted form binding.
+    *******************************************************************************/
+   private void possibleValues(String table, String field, String inputName) throws Exception
+   {
+      QRecord a = draft("a", "HTML", "[]", "{}", "A");
+      QRecord b = draft("b", "HTML", "[]", "{}", "B");
+      var metadata = QContext.getQInstance().getTable(table).getField(field);
+      assertThat(metadata.getPossibleValueSourceFilter()).isNotNull();
+      var filter = metadata.getPossibleValueSourceFilter().clone();
+      assertThat(filter.toString()).contains("${input." + inputName + "}");
+      filter.interpretValues(Map.of(inputName, a.getValue("templateId")));
+      var found = new SearchPossibleValueSourceAction().execute(new SearchPossibleValueSourceInput().withPossibleValueSourceName(metadata.getPossibleValueSourceName()).withDefaultQueryFilter(filter));
+      assertThat(found.getResults().stream().map(v -> v.getId().toString()).toList()).contains(a.getValueString("id")).doesNotContain(b.getValueString("id"));
+   }
+}
diff --git a/src/test/java/com/kingsrook/qbits/email/H2Support.java b/src/test/java/com/kingsrook/qbits/email/H2Support.java
index 02eb0af..e1083b9 100644
--- a/src/test/java/com/kingsrook/qbits/email/H2Support.java
+++ b/src/test/java/com/kingsrook/qbits/email/H2Support.java
@@ -20,8 +20,6 @@
 
 package com.kingsrook.qbits.email;
 
-import java.nio.file.Files;
-import java.nio.file.Path;
 import java.sql.Connection;
 import java.util.List;
 import java.util.UUID;
@@ -72,7 +70,7 @@ public final class H2Support
    {
       try(Connection connection = new ConnectionManager().getConnection(backend))
       {
-         for(String sql : Files.readString(Path.of("src/test/resources/email-h2.sql")).split(";"))
+         for(String sql : new String(H2Support.class.getResourceAsStream("/email-h2.sql").readAllBytes(), java.nio.charset.StandardCharsets.UTF_8).split(";"))
          {
             if(!sql.isBlank())
             {
diff --git a/src/test/java/com/kingsrook/qbits/email/PhiCanaryEgressTest.java b/src/test/java/com/kingsrook/qbits/email/PhiCanaryEgressTest.java
new file mode 100644
index 0000000..e744078
--- /dev/null
+++ b/src/test/java/com/kingsrook/qbits/email/PhiCanaryEgressTest.java
@@ -0,0 +1,419 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.email;
+
+import java.io.Serializable;
+import java.math.BigDecimal;
+import java.sql.Connection;
+import java.time.Instant;
+import java.util.ArrayList;
+import java.util.Base64;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import com.kingsrook.qbits.email.app.EmailHome;
+import com.kingsrook.qbits.email.bridge.EmailTemplatedMessageInput;
+import com.kingsrook.qbits.email.delivery.EmailDispatcher;
+import com.kingsrook.qbits.email.metadata.EmailTables;
+import com.kingsrook.qbits.email.outbox.EmailRequest;
+import com.kingsrook.qbits.email.outbox.EmailSendResult;
+import com.kingsrook.qbits.email.outbox.EmailService;
+import com.kingsrook.qbits.email.phi.EmailBodyEncryption;
+import com.kingsrook.qbits.email.phi.EmailClassifications;
+import com.kingsrook.qbits.email.phi.EmailValueTokens;
+import com.kingsrook.qbits.email.processes.EmailProcesses;
+import com.kingsrook.qbits.email.support.EmailStore;
+import com.kingsrook.qbits.email.support.SystemWrites;
+import com.kingsrook.qbits.email.template.EmailVersions;
+import com.kingsrook.qqq.backend.core.actions.messaging.SendMessageAction;
+import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
+import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+import com.kingsrook.qqq.backend.module.rdbms.jdbc.ConnectionManager;
+import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.core.LogEvent;
+import org.apache.logging.log4j.core.LoggerContext;
+import org.apache.logging.log4j.core.appender.AbstractAppender;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+/*******************************************************************************
+ ** Canary egress across every templated entry path, retry, dead letter, export,
+ ** home and Log4j. Transport receives plaintext; no other inspected surface does.
+ *******************************************************************************/
+class PhiCanaryEgressTest
+{
+   private String canary;
+   private QRecord version;
+   private QTableMetaData patient;
+
+   /*******************************************************************************
+    ** Plant PHI in an application table and configure only classification overrides.
+    *******************************************************************************/
+   @BeforeEach
+   void setup() throws Exception
+   {
+      canary = "CANARY-PHI-" + UUID.randomUUID();
+      H2Support.setup(c -> c.withMaxAttempts(2).withSecretResolver(PhiCanaryEgressTest::secret)
+         .withClassificationOverrides(Map.of("test_patient.name", EmailClassifications.Level.PHI)));
+      H2Support.provider(25);
+      SystemWrites.updateOne(EmailTables.PROVIDER, new QRecord().withValue("id", 1).withValue("phiAllowed", true));
+      patient = new QTableMetaData().withName("test_patient").withBackendName("rdbms").withPrimaryKeyField("id")
+         .withField(new QFieldMetaData("id", QFieldType.INTEGER).withBackendName("id"))
+         .withField(new QFieldMetaData("name", QFieldType.STRING).withBackendName("name"));
+      QContext.getQInstance().addTable(patient);
+      try(Connection connection = new ConnectionManager().getConnection((RDBMSBackendMetaData) QContext.getQInstance().getBackend("rdbms")))
+      {
+         connection.createStatement().execute("CREATE TABLE test_patient (id INTEGER PRIMARY KEY, name VARCHAR(200))");
+         try(var statement = connection.prepareStatement("INSERT INTO test_patient (id, name) VALUES (1, ?)"))
+         {
+            statement.setString(1, canary);
+            statement.executeUpdate();
+         }
+      }
+      version = EmailTemplatesTest.draft("phi_notice", "MARKDOWN", "[{\"name\":\"patient\",\"type\":\"RECORD\",\"required\":true}]",
+         "{\"patient\":{\"name\":\"Example\"}}", "# {{patient.name}}\n\nPrivate message");
+      SystemWrites.updateOne(EmailTables.VERSION, new QRecord().withValue("id", version.getValue("id")).withValue("subject", "{{patient.name}}"));
+   }
+
+   private static boolean coreVectorKey = false;
+
+
+
+   /*******************************************************************************
+    ** The stand-in matches core ClassifiedValueTokens (QRun-IO/qqq#1182) on its
+    ** published context vector: key bytes 00..1f, key id env1.
+    *******************************************************************************/
+   @Test
+   void standInMatchesCoreVector()
+   {
+      coreVectorKey = true;
+      try
+      {
+         assertThat(EmailValueTokens.forContext(List.of("email", "welcome", "recipient", "message-7"), "Ada@example.com"))
+            .isEqualTo("hmac:v1:env1:p1BggBbb8ytFgS5YeM6eBiriGgC1j3x0YGjics5qVeM");
+      }
+      finally
+      {
+         coreVectorKey = false;
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Fixed non-production keys, resolved in memory only.
+    *******************************************************************************/
+   private static String secret(String name)
+   {
+      if(coreVectorKey && "QQQ_AUDIT_HMAC_KEY".equals(name))
+      {
+         byte[] key = new byte[32];
+         for(int i = 0; i < key.length; i++)
+         {
+            key[i] = (byte) i;
+         }
+         return (Base64.getEncoder().encodeToString(key));
+      }
+      return (switch(name)
+      {
+         case "QQQ_EMAIL_BODY_KEY" -> "test-only-body-key";
+         case "QQQ_AUDIT_HMAC_KEY" -> Base64.getEncoder().encodeToString("test-only-audit-key-32-bytes-long!".getBytes(java.nio.charset.StandardCharsets.UTF_8));
+         default -> null;
+      });
+   }
+
+   /*******************************************************************************
+    ** Clear request-local state.
+    *******************************************************************************/
+   @AfterEach
+   void cleanup()
+   {
+      QContext.clear();
+      EmailRuntime.setClock(null);
+   }
+
+   /*******************************************************************************
+    ** The record is fetched under the existing sender session at send time.
+    *******************************************************************************/
+   private EmailRequest request()
+   {
+      return (new EmailRequest().withTemplateName("phi_notice").withTo("person@example.com").withRecordVariable("patient", "test_patient", 1));
+   }
+
+   /*******************************************************************************
+    ** No unapproved PHI request can leave any outbox or attempt row.
+    *******************************************************************************/
+   @Test
+   void unapprovedRefusedWithoutStorage() throws Exception
+   {
+      EmailVersions.activate(version.getValueInteger("id"));
+      var refused = EmailService.send(request());
+      assertThat(refused.status()).isEqualTo(EmailSendResult.Status.REFUSED);
+      assertThat(refused.reasons()).contains("The variable patient.name is PHI; this template is not approved for PHI.");
+      assertThat(H2Support.all(EmailTables.MESSAGE)).isEmpty();
+      assertThat(H2Support.all(EmailTables.ATTEMPT)).isEmpty();
+      noCanary(refused.toString());
+   }
+
+   /*******************************************************************************
+    ** Every path encrypts, redacts, retries and dead-letters without egress.
+    *******************************************************************************/
+   @Test
+   void canaryAcrossServiceProcessBridgeAndOperationalSurfaces() throws Exception
+   {
+      EmailVersions.approve(version.getValueInteger("id"));
+      EmailVersions.activate(version.getValueInteger("id"));
+      List logs = new ArrayList<>();
+      AbstractAppender capture = new AbstractAppender("phi-canary", null, null, true, null)
+      {
+         @Override
+         public void append(LogEvent event)
+         {
+            logs.add(event.getMessage().getFormattedMessage());
+            if(event.getThrown() != null)
+            {
+               logs.add(event.getThrown().toString());
+            }
+         }
+      };
+      capture.start();
+      LoggerContext context = (LoggerContext) LogManager.getContext(false);
+      var root = context.getConfiguration().getRootLogger();
+      var priorLevel = root.getLevel();
+      root.addAppender(capture, org.apache.logging.log4j.Level.ALL, null);
+      root.setLevel(org.apache.logging.log4j.Level.INFO);
+      context.updateLoggers();
+      try
+      {
+         var sent = EmailService.send(request());
+         assertThat(sent.status()).isEqualTo(EmailSendResult.Status.QUEUED);
+         var input = new RunProcessInput();
+         input.setProcessName(EmailProcesses.SEND_TEMPLATED);
+         input.setFrontendStepBehavior(RunProcessInput.FrontendStepBehavior.SKIP);
+         input.addValue("templateName", "phi_notice");
+         input.addValue("to", "person@example.com");
+         input.addValue("recordVariablesJson", "{\"patient\":{\"table\":\"test_patient\",\"id\":1}}");
+         var process = new RunProcessAction().execute(input);
+         assertThat(process.getException()).isEmpty();
+         assertThat(process.getValues().get("status")).isEqualTo("QUEUED");
+         noCanary(process.getValues().toString());
+         new SendMessageAction().execute(new EmailTemplatedMessageInput().withEmailRequest(request()).withMessagingProviderName("qbitEmail"));
+         assertThat(H2Support.all(EmailTables.MESSAGE)).hasSize(3);
+         var dispatcher = new EmailDispatcher(new com.kingsrook.qbits.email.transport.EmailTransportRegistry().withTransport("SMTP", (envelope, settings) ->
+         {
+            assertThat(envelope.getSubject()).isEqualTo(canary);
+            assertThat(envelope.getHtmlBody()).contains(canary);
+            throw new com.kingsrook.qbits.email.transport.EmailTransportException("Provider echoed " + canary, false);
+         }));
+         for(QRecord message : H2Support.all(EmailTables.MESSAGE))
+         {
+            Integer id = message.getValueInteger("id");
+            assertThat(dispatcher.sendNow(id)).isEqualTo(EmailDispatcher.SendResult.RETRY_WAIT);
+            assertThat(dispatcher.sendNow(id)).isEqualTo(EmailDispatcher.SendResult.DEAD_LETTER);
+            QRecord raw = EmailStore.get(EmailTables.MESSAGE, id, null);
+            for(String field : List.of("subject", "htmlBody", "textBody"))
+            {
+               assertThat(raw.getValueString(field)).startsWith("enc:v1:").doesNotContain(canary);
+               assertThat(EmailBodyEncryption.decrypt(raw.getValueString(field), field, raw.getValueString("id"))).contains(canary);
+            }
+            assertThat(raw.getValueString("variablesSnapshot")).contains("hmac:v1:");
+            for(var value : raw.getValues().entrySet())
+            {
+               if(!List.of("subject", "htmlBody", "textBody").contains(value.getKey()))
+               {
+                  noCanary(String.valueOf(value.getValue()));
+               }
+            }
+         }
+         for(String table : List.of(EmailTables.MESSAGE, EmailTables.ATTEMPT))
+         {
+            var exported = new QueryAction().execute(new QueryInput(table).withShouldFetchHeavyFields(true).withShouldGenerateDisplayValues(true));
+            noCanary(exported.getRecords().toString());
+            if(table.equals(EmailTables.MESSAGE))
+            {
+               assertThat(exported.getRecords()).allSatisfy(row ->
+               {
+                  assertThat(row.getValueString("subject")).isEqualTo("[encrypted]");
+                  assertThat(row.getValueString("htmlBody")).isEqualTo("[encrypted]");
+                  assertThat(row.getValueString("textBody")).isEqualTo("[encrypted]");
+               });
+            }
+         }
+         noCanary(EmailHome.overview().getHtml());
+         assertThat(EmailHome.overview().getHtml()).contains("Secure message");
+         noCanary(EmailHome.hourly().getSublabel());
+         assertThat(logs).isNotEmpty();
+         logs.forEach(this::noCanary);
+      }
+      finally
+      {
+         root.removeAppender(capture.getName());
+         root.setLevel(priorLevel);
+         context.updateLoggers();
+         capture.stop();
+      }
+   }
+
+   /*******************************************************************************
+    ** Checking every five-character window detects every longer substring too.
+    *******************************************************************************/
+   private void noCanary(String output)
+   {
+      for(int i = 0; i <= canary.length() - 5; i++)
+      {
+         assertThat(output).doesNotContain(canary.substring(i, i + 5));
+      }
+   }
+
+   /*******************************************************************************
+    ** Provider attestation, tracking, encryption key and category are mandatory.
+    *******************************************************************************/
+   @Test
+   void phiPolicyAndKeyRefusals() throws Exception
+   {
+      EmailVersions.approve(version.getValueInteger("id"));
+      EmailVersions.activate(version.getValueInteger("id"));
+      EmailRuntime.config().withSecretResolver(name -> null);
+      assertThat(EmailService.send(request()).reasons()).contains("PHI email needs QQQ_EMAIL_BODY_KEY");
+      EmailRuntime.config().withSecretResolver(PhiCanaryEgressTest::secret);
+      SystemWrites.updateOne(EmailTables.PROVIDER, new QRecord().withValue("id", 1).withValue("trackOpens", true));
+      assertThat(EmailService.send(request()).status()).isEqualTo(EmailSendResult.Status.REFUSED);
+      SystemWrites.updateOne(EmailTables.PROVIDER, new QRecord().withValue("id", 1).withValue("trackOpens", false).withValue("phiAllowed", false));
+      assertThat(EmailService.send(request()).status()).isEqualTo(EmailSendResult.Status.REFUSED);
+      assertThat(EmailService.send(request().withCategory(com.kingsrook.qbits.email.outbox.EmailCategory.MARKETING)).reasons().toString()).contains("Template category cannot be overridden");
+      assertThat(H2Support.all(EmailTables.MESSAGE)).isEmpty();
+   }
+
+   /*******************************************************************************
+    ** Masked fields remain masked when rendered, even for an approved template.
+    *******************************************************************************/
+   @Test
+   void recordMaskingAndProviderChangesAreRespected() throws Exception
+   {
+      EmailVersions.approve(version.getValueInteger("id"));
+      EmailVersions.activate(version.getValueInteger("id"));
+      patient.getField("name").withFieldSecurityLock(new com.kingsrook.qqq.backend.core.model.metadata.security.FieldSecurityLock()
+         .withSecurityKeyType("clinical").withDefaultBehavior(com.kingsrook.qqq.backend.core.model.metadata.security.FieldSecurityLock.Behavior.DENY)
+         .withOverrideValues(List.of("allowed")));
+      var result = EmailService.send(request());
+      assertThat(result.status()).isEqualTo(EmailSendResult.Status.QUEUED);
+      var row = EmailStore.get(EmailTables.MESSAGE, result.messageId(), null);
+      assertThat(EmailBodyEncryption.decrypt(row.getValueString("subject"), "subject", row.getValueString("id"))).isEqualTo("************");
+      SystemWrites.updateOne(EmailTables.PROVIDER, new QRecord().withValue("id", 1).withValue("trackClicks", true));
+      var dispatcher = new EmailDispatcher(new com.kingsrook.qbits.email.transport.EmailTransportRegistry().withTransport("SMTP", (envelope, settings) ->
+      {
+         throw new AssertionError("Unsafe provider received PHI");
+      }));
+      assertThat(dispatcher.sendNow(result.messageId())).isEqualTo(EmailDispatcher.SendResult.RETRY_WAIT);
+      assertThat(EmailService.resend(result.messageId()).status()).isEqualTo(EmailSendResult.Status.REFUSED);
+      SystemWrites.updateOne(EmailTables.PROVIDER, new QRecord().withValue("id", 1).withValue("trackClicks", false));
+      var resent = EmailService.resend(result.messageId());
+      assertThat(resent.status()).isEqualTo(EmailSendResult.Status.QUEUED);
+      QRecord copy = EmailStore.get(EmailTables.MESSAGE, resent.messageId(), null);
+      assertThat(copy.getValueBoolean("phiApproved")).isTrue();
+      assertThat(copy.getValueString("subject")).isNotEqualTo(row.getValueString("subject"));
+      assertThat(EmailBodyEncryption.decrypt(copy.getValueString("subject"), "subject", copy.getValueString("id"))).isEqualTo("************");
+      assertThatThrownBy(() -> EmailBodyEncryption.decrypt(copy.getValueString("subject"), "subject", row.getValueString("id"))).hasMessageContaining("could not be opened");
+      assertThat(copy.getValue("templateVersionId")).isEqualTo(row.getValue("templateVersionId"));
+   }
+
+   /*******************************************************************************
+    ** Ad-hoc callers cannot turn a ciphertext string into a decryption request.
+    *******************************************************************************/
+   @Test
+   void ciphertextPrefixIsNotAuthority() throws Exception
+   {
+      String encrypted = EmailBodyEncryption.encrypt(canary, "subject", "test");
+      var result = EmailService.send(new EmailRequest().withTo("p@example.com").withSubject(encrypted).withTextBody(encrypted));
+      assertThat(result.status()).isEqualTo(EmailSendResult.Status.QUEUED);
+      var dispatcher = new EmailDispatcher(new com.kingsrook.qbits.email.transport.EmailTransportRegistry().withTransport("SMTP", (envelope, settings) ->
+      {
+         assertThat(envelope.getSubject()).isEqualTo(encrypted);
+         assertThat(envelope.getTextBody()).isEqualTo(encrypted);
+         return new com.kingsrook.qbits.email.transport.EmailTransportResult("safe");
+      }));
+      assertThat(dispatcher.sendNow(result.messageId())).isEqualTo(EmailDispatcher.SendResult.SENT);
+   }
+
+   /*******************************************************************************
+    ** Token bindings, canonicalization, sentinels and authenticated encryption.
+    *******************************************************************************/
+   @Test
+   void tokensAndEncryption()
+   {
+      List binding = List.of("email", "template", "variable", "123");
+      String token = EmailValueTokens.forContext(binding, canary);
+      assertThat(token).startsWith("hmac:v1:");
+      assertThat(EmailValueTokens.forContext(binding, canary)).isEqualTo(token);
+      assertThat(EmailValueTokens.verifyContext(binding, canary, token)).isTrue();
+      assertThat(EmailValueTokens.verifyContext(List.of("different"), canary, token)).isFalse();
+      assertThat(EmailValueTokens.forContext(binding, true)).isEqualTo("changed:v1");
+      assertThat(EmailValueTokens.forContext(binding, "aaaaaa")).isEqualTo("changed:v1");
+      assertThat(EmailValueTokens.verifyContext(binding, true, "shred:v1")).isFalse();
+      assertThat(EmailValueTokens.verifyContext(binding, true, "changed:v1")).isFalse();
+      String record = EmailValueTokens.forRecordField(patient, patient.getField("name"), 1, canary);
+      assertThat(EmailValueTokens.verifyRecordField(patient, patient.getField("name"), 1, canary, record)).isTrue();
+      assertThat(EmailValueTokens.verifyRecordField(patient, patient.getField("name"), 2, canary, record)).isFalse();
+      assertThat(EmailValueTokens.canonical(new BigDecimal("1.2000"))).isEqualTo("1.2");
+      assertThat(EmailValueTokens.canonical(Instant.parse("2026-10-13T12:00:00Z"))).isEqualTo("2026-10-13T12:00:00Z");
+      String encrypted = EmailBodyEncryption.encrypt(canary, "subject", "test");
+      assertThat(EmailBodyEncryption.encrypt(canary, "subject", "test")).isNotEqualTo(encrypted);
+      assertThat(EmailBodyEncryption.decrypt(encrypted, "subject", "test")).isEqualTo(canary);
+      assertThatThrownBy(() -> EmailBodyEncryption.decrypt(encrypted.substring(0, encrypted.length() - 4) + "AAAA", "subject", "test")).hasMessageContaining("could not be opened");
+      EmailRuntime.config().withSecretResolver(name -> null);
+      assertThat(EmailValueTokens.forContext(binding, canary)).isEqualTo("changed:v1");
+      assertThatThrownBy(() -> EmailBodyEncryption.encrypt(canary, "subject", "test")).hasMessageContaining("QQQ_EMAIL_BODY_KEY");
+   }
+
+   /*******************************************************************************
+    ** Record projection follows parsed references and snapshots fail closed.
+    *******************************************************************************/
+   @Test
+   void referencedFieldsOnlyAndUnclassifiedTokens() throws Exception
+   {
+      EmailRuntime.config().withClassificationOverrides(Map.of());
+      patient.withField(new QFieldMetaData("unused", QFieldType.STRING).withBackendName("unused"));
+      try(Connection connection = new ConnectionManager().getConnection((RDBMSBackendMetaData) QContext.getQInstance().getBackend("rdbms")))
+      {
+         connection.createStatement().execute("ALTER TABLE test_patient ADD unused VARCHAR(200) DEFAULT 'never-load-this'");
+      }
+      SystemWrites.updateOne(EmailTables.VERSION, new QRecord().withValue("id", version.getValue("id"))
+         .withValue("subject", "Notice").withValue("bodySource", "{{#if patient.name}}Hello {{patient.name}}{{/if}}"));
+      EmailVersions.activate(version.getValueInteger("id"));
+      var result = EmailService.send(request());
+      assertThat(result.status()).isEqualTo(EmailSendResult.Status.QUEUED);
+      QRecord row = EmailStore.get(EmailTables.MESSAGE, result.messageId(), null);
+      assertThat(row.getValueString("htmlBody")).contains(canary);
+      assertThat(row.getValueString("variablesSnapshot")).contains("name", "hmac:v1:").doesNotContain(canary, "unused", "never-load-this", "\"id\"");
+   }
+
+}
diff --git a/src/test/java/com/kingsrook/qbits/email/demo/EmailDemoServer.java b/src/test/java/com/kingsrook/qbits/email/demo/EmailDemoServer.java
new file mode 100644
index 0000000..b7369bc
--- /dev/null
+++ b/src/test/java/com/kingsrook/qbits/email/demo/EmailDemoServer.java
@@ -0,0 +1,293 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.email.demo;
+
+
+import java.time.Clock;
+import java.time.Duration;
+import java.time.Instant;
+import java.time.ZoneOffset;
+import java.time.temporal.ChronoUnit;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import com.icegreen.greenmail.util.GreenMail;
+import com.icegreen.greenmail.util.ServerSetup;
+import com.kingsrook.qbits.email.EmailQBitConfig;
+import com.kingsrook.qbits.email.EmailQBitProducer;
+import com.kingsrook.qbits.email.EmailRuntime;
+import com.kingsrook.qbits.email.H2Support;
+import com.kingsrook.qbits.email.delivery.EmailDispatcher;
+import com.kingsrook.qbits.email.metadata.EmailTables;
+import com.kingsrook.qbits.email.outbox.EmailRequest;
+import com.kingsrook.qbits.email.outbox.EmailSendResult;
+import com.kingsrook.qbits.email.outbox.EmailService;
+import com.kingsrook.qbits.email.permissions.EmailPermissions;
+import com.kingsrook.qbits.email.support.EmailStore;
+import com.kingsrook.qbits.email.support.SystemWrites;
+import com.kingsrook.qbits.email.template.EmailBuiltIns;
+import com.kingsrook.qbits.email.template.EmailVersions;
+import com.kingsrook.qbits.email.transport.EmailTransportRegistry;
+import com.kingsrook.qbits.email.transport.SesEmailTransport;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.instances.AbstractQQQApplication;
+import com.kingsrook.qqq.backend.core.instances.QInstanceValidator;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData;
+import com.kingsrook.qqq.backend.core.model.session.QSession;
+import com.kingsrook.qqq.backend.core.modules.authentication.QAuthenticationModuleDispatcher;
+import com.kingsrook.qqq.backend.core.modules.authentication.implementations.FullyAnonymousAuthenticationModule;
+import com.kingsrook.qqq.backend.module.rdbms.jdbc.ConnectionManager;
+import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData;
+import com.kingsrook.qqq.middleware.javalin.QApplicationJavalinServer;
+import software.amazon.awssdk.services.sesv2.SesV2Client;
+import software.amazon.awssdk.services.sesv2.model.SendEmailResponse;
+
+
+/*******************************************************************************
+ ** Local screenshot application: H2, GreenMail and fake SES, all in memory.
+ ** Home links already use /app/table/id, equivalent to webhooks' table prefix.
+ ** Nothing in this test package is shipped or included in coverage.
+ **
+ ** ./mvnw -o test-compile exec:java -Dexec.classpathScope=test
+ **    -Dexec.mainClass=com.kingsrook.qbits.email.demo.EmailDemoServer -Dexec.args=8098
+ *******************************************************************************/
+public class EmailDemoServer extends AbstractQQQApplication
+{
+   private static final RDBMSBackendMetaData DATABASE = backend();
+
+   /*******************************************************************************
+    ** Anonymous sessions with all email permissions, exclusively for this demo.
+    *******************************************************************************/
+   public static class DemoAuthenticationModule extends FullyAnonymousAuthenticationModule
+   {
+      /*******************************************************************************
+       ** Local demo session.
+       *******************************************************************************/
+      @Override
+      public QSession createSession(QInstance instance, Map context)
+      {
+         return (super.createSession(instance, context).withPermissions(EmailPermissions.ALL));
+      }
+   }
+
+   /*******************************************************************************
+    ** H2 survives connections for this JVM only, without filesystem storage.
+    *******************************************************************************/
+   private static RDBMSBackendMetaData backend()
+   {
+      var backend = new RDBMSBackendMetaData().withName("rdbms").withVendor("h2").withHostName("mem").withDatabaseName("email_demo").withUsername("sa");
+      backend.setJdbcUrl(ConnectionManager.getJdbcUrl(backend) + ";DATABASE_TO_UPPER=FALSE;DB_CLOSE_DELAY=-1");
+      return (backend);
+   }
+
+   /*******************************************************************************
+    ** Exercise SES MIME and request construction without making AWS calls.
+    *******************************************************************************/
+   private static SesV2Client fakeSes()
+   {
+      return ((SesV2Client) java.lang.reflect.Proxy.newProxyInstance(SesV2Client.class.getClassLoader(), new Class[] {SesV2Client.class}, (proxy, method, args) ->
+      {
+         return (switch(method.getName())
+         {
+            case "sendEmail" -> SendEmailResponse.builder().messageId("demo-ses-" + UUID.randomUUID()).build();
+            case "serviceName" -> "sesv2";
+            case "close" -> null;
+            default -> throw (new UnsupportedOperationException("Unsupported demo SES call"));
+         });
+      }));
+   }
+
+   /*******************************************************************************
+    ** UI actions also use the fake SES client, after startup seeding completes.
+    *******************************************************************************/
+   @Override
+   public QInstance defineQInstance() throws QException
+   {
+      QInstance instance = new QInstance();
+      instance.withInstanceDefaultAuthentication(new QAuthenticationMetaData().withName("anonymous").withType(QAuthenticationType.FULLY_ANONYMOUS));
+      instance.addBackend(DATABASE);
+      new EmailQBitProducer().withQBitConfig(new EmailQBitConfig().withBackendName("rdbms")
+         .withPublicBaseUrl("https://kof22.example").withMaxAttempts(3).withRetryMinutes(List.of(1, 5)).withSecretResolver(System::getProperty)
+         .withTransportRegistryFactory(() -> new EmailTransportRegistry().withTransport("SES", new SesEmailTransport(builder -> fakeSes()))))
+         .produce(instance);
+      instance.setHomeAppName("email");
+      return (instance);
+   }
+
+   /*******************************************************************************
+    ** Provider rows pass through actual save validation.
+    *******************************************************************************/
+   private static QRecord provider(String name, String label, String kind, int port, boolean defaultProvider) throws Exception
+   {
+      return (H2Support.insert(EmailTables.PROVIDER, new QRecord().withValue("name", name).withValue("label", label).withValue("kind", kind)
+         .withValue("host", "127.0.0.1").withValue("port", port).withValue("security", "NONE").withValue("region", "us-east-1")
+         .withValue("fromAddress", "hello@kof22.example").withValue("fromName", "Kof22").withValue("enabled", true)
+         .withValue("isDefault", defaultProvider).withValue("phiAllowed", true).withValue("trackOpens", false).withValue("trackClicks", false)));
+   }
+
+   /*******************************************************************************
+    ** Real template/version lifecycle with sample data for the inline preview.
+    *******************************************************************************/
+   private static QRecord template(String name, String label, String kind, String category, Integer brand, String body, boolean active) throws Exception
+   {
+      QRecord template = H2Support.insert(EmailTables.TEMPLATE, new QRecord().withValue("name", name).withValue("label", label)
+         .withValue("kind", kind).withValue("category", category).withValue("brandId", brand));
+      QRecord version = EmailVersions.newVersion(template.getValueInteger("id"));
+      SystemWrites.updateOne(EmailTables.VERSION, new QRecord().withValue("id", version.getValue("id")).withValue("subject", label + " - {{topic}}")
+         .withValue("preheader", "An update from Kof22").withValue("bodySource", body)
+         .withValue("variablesSchema", "[{\"name\":\"name\",\"type\":\"STRING\"},{\"name\":\"topic\",\"type\":\"STRING\"}]")
+         .withValue("sampleVariables", "{\"name\":\"Alex Morgan\",\"topic\":\"Your community project\"}"));
+      if(active)
+      {
+         EmailVersions.activate(version.getValueInteger("id"));
+      }
+      return (version);
+   }
+
+   /*******************************************************************************
+    ** Messages and attempts are created only by the real service and dispatcher.
+    *******************************************************************************/
+   static Map seed(QInstance instance, GreenMail mail, int closedPort) throws Exception
+   {
+      H2Support.prime(DATABASE);
+      QContext.init(instance, H2Support.admin());
+      QRecord relay = provider("galaxy", "Galaxy relay (PMG)", "SMTP", mail.getSmtp().getPort(), true);
+      provider("ses", "Amazon SES", "SES", 25, false);
+      provider("backup", "Backup relay", "SMTP", closedPort, false);
+      QRecord brand = H2Support.insert(EmailTables.BRAND, new QRecord().withValue("name", "Kof22").withValue("label", "Kof22")
+         .withValue("isDefault", true).withValue("siteUrl", "https://kof22.com").withValue("postalAddress", "100 Market Street, Chester, IL 62233")
+         .withValue("primaryColor", "#2459a6").withValue("textColor", "#172033").withValue("backgroundColor", "#ffffff")
+         .withValue("buttonTextColor", "#ffffff").withValue("darkPrimaryColor", "#93c5fd").withValue("darkButtonTextColor", "#172033")
+         .withValue("darkBackgroundColor", "#111827").withValue("darkTextColor", "#f9fafb").withValue("footerMarkdown", "**Kof22** - Ideas, projects, and community."));
+      Integer brandId = brand.getValueInteger("id");
+      QRecord contact = template("contact_form_received", "Contact form received", "MARKDOWN", "NOTIFICATION", brandId,
+         "# Thanks for reaching out, {{name}}\n\nWe received your message about **{{topic}}**. We will follow up shortly.\n\n[Visit Kof22](https://kof22.com){.button}", true);
+      QRecord v2 = EmailVersions.newVersion(contact.getValueInteger("templateId"));
+      EmailVersions.activate(v2.getValueInteger("id"));
+      template("alerts_notification", "Project alert", "MARKDOWN", "NOTIFICATION", brandId,
+         "# A project needs your attention\n\nHello {{name}}, there is an update for **{{topic}}**.\n\n| Item | Status |\n|---|---|\n| Community project | Ready for review |\n\n[Review update](https://kof22.com){.button}", true);
+      template("newsletter_october", "October at Kof22", "HTML", "MARKETING", brandId,
+         "
October at Kof22
Hello {{name}},
This month: {{topic}}.
Building something useful
Stories from our projects and community.
Explore the latest", false);
+      EmailBuiltIns.ensure();
+      QRecord secure = SystemWrites.query(EmailTables.TEMPLATE, EmailStore.equal("name", "email_secure_message")).getFirst();
+      SystemWrites.updateOne(EmailTables.TEMPLATE, new QRecord().withValue("id", secure.getValue("id")).withValue("brandId", brandId));
+      EmailVersions.approve(secure.getValueInteger("activeVersionId"));
+
+      Instant now = Instant.now().truncatedTo(ChronoUnit.MINUTES);
+      String[] reasons = {"HARD_BOUNCE", "COMPLAINT", "UNSUBSCRIBE", "MANUAL"};
+      for(int i = 0; i < 20; i++)
+      {
+         EmailRuntime.setClock(Clock.fixed(now.minus(Duration.ofDays(i * 29L / 19)), ZoneOffset.UTC));
+         H2Support.insert(EmailTables.SUPPRESSION, new QRecord().withValue("address", "suppressed" + i + "@example.test")
+            .withValue("scope", "ALL").withValue("reason", reasons[i % reasons.length]).withValue("note", "Demo suppression history"));
+      }
+      Integer messageId = null;
+      EmailDispatcher dispatcher = new EmailDispatcher();
+      for(int hour = 23; hour >= 0; hour--)
+      {
+         Instant start = now.minus(Duration.ofHours(hour)).minus(Duration.ofMinutes(25));
+         EmailRuntime.setClock(Clock.fixed(start, ZoneOffset.UTC));
+         for(int i = 0; i < 5; i++)
+         {
+            int sequence = (23 - hour) * 5 + i;
+            boolean phi = sequence == 37 || sequence == 88;
+            EmailRequest request = new EmailRequest().withTemplateName(phi ? "email_secure_message" : sequence % 3 == 0 ? "alerts_notification" : "contact_form_received")
+               .withProviderName(sequence % 11 == 0 ? "backup" : sequence % 5 == 0 ? "ses" : "galaxy")
+               .withTo(sequence % 19 == 0 ? "suppressed" + (sequence % 20) + "@example.test" : "reader" + sequence + "@example.test");
+            if(phi)
+            {
+               request.withVariable("link", "https://kof22.com/secure/demo-" + sequence);
+            }
+            else
+            {
+               request.withVariable("name", List.of("Alex Morgan", "Jordan Lee", "Sam Taylor", "Casey Rivera", "Riley Chen").get(i))
+                  .withVariable("topic", List.of("Community center", "October newsletter", "Website feedback", "Volunteer program", "Project update").get(i));
+            }
+            EmailSendResult result = EmailService.send(request);
+            if(result.status() == EmailSendResult.Status.REFUSED)
+            {
+               throw (new IllegalStateException("Demo send refused: " + result.reasons()));
+            }
+            if(messageId == null && result.status() == EmailSendResult.Status.QUEUED)
+            {
+               messageId = result.messageId();
+            }
+            SystemWrites.updateOne(EmailTables.MESSAGE, new QRecord().withValue("id", result.messageId()).withValue("createDate", start));
+         }
+         for(int pass = 0; pass < 3; pass++)
+         {
+            EmailRuntime.setClock(Clock.fixed(start.plus(Duration.ofMinutes(pass * 6L)), ZoneOffset.UTC));
+            dispatcher.runOnce();
+         }
+      }
+      EmailRuntime.setClock(null);
+      System.out.println("Seeded " + H2Support.all(EmailTables.MESSAGE).size() + " messages, " + H2Support.all(EmailTables.ATTEMPT).size() + " real attempts; " + mail.getReceivedMessages().length + " captured SMTP deliveries.");
+      QContext.clear();
+      return (Map.of("message", messageId, "version", v2.getValueInteger("id"), "provider", relay.getValueInteger("id")));
+   }
+
+   /*******************************************************************************
+    ** Start SMTP and Next UI; shutdown releases in-memory mail.
+    *******************************************************************************/
+   public static void main(String[] args) throws Exception
+   {
+      int port = args.length == 0 ? 8098 : Integer.parseInt(args[0]);
+      System.setProperty("QQQ_EMAIL_BODY_KEY", "local-demo-only-body-key");
+      System.setProperty("QQQ_EMAIL_UNSUBSCRIBE_KEY", "local-demo-only-unsubscribe-key");
+      var field = QAuthenticationModuleDispatcher.class.getDeclaredField("authenticationTypeToModuleClassNameMap");
+      field.setAccessible(true);
+      @SuppressWarnings("unchecked")
+      Map modules = (Map) field.get(null);
+      modules.put(QAuthenticationType.FULLY_ANONYMOUS.getName(), DemoAuthenticationModule.class.getName());
+      GreenMail mail = new GreenMail(new ServerSetup(0, "127.0.0.1", "smtp"));
+      mail.start();
+      Runtime.getRuntime().addShutdownHook(new Thread(mail::stop));
+      int closedPort;
+      try(var socket = new java.net.ServerSocket(0, 1, java.net.InetAddress.getLoopbackAddress()))
+      {
+         closedPort = socket.getLocalPort();
+      }
+      try
+      {
+         EmailDemoServer application = new EmailDemoServer();
+         QInstance instance = application.defineQInstance();
+         new QInstanceValidator().validate(instance);
+         Map ids = seed(instance, mail, closedPort);
+         new QApplicationJavalinServer(application).withPort(port).withServeFrontendNext(true).start();
+         String base = "http://localhost:" + port + "/app/";
+         System.out.println("Home: " + base + "email");
+         System.out.println("Message: " + base + EmailTables.MESSAGE + "/" + ids.get("message"));
+         System.out.println("Template preview: " + base + EmailTables.VERSION + "/" + ids.get("version"));
+         System.out.println("Provider: " + base + EmailTables.PROVIDER + "/" + ids.get("provider"));
+      }
+      catch(Exception e)
+      {
+         mail.stop();
+         EmailRuntime.setClock(null);
+         QContext.clear();
+         throw (e);
+      }
+   }
+}
diff --git a/src/test/java/com/kingsrook/qbits/email/template/CliMjmlCompilerTest.java b/src/test/java/com/kingsrook/qbits/email/template/CliMjmlCompilerTest.java
new file mode 100644
index 0000000..35eecae
--- /dev/null
+++ b/src/test/java/com/kingsrook/qbits/email/template/CliMjmlCompilerTest.java
@@ -0,0 +1,55 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.email.template;
+
+
+import org.junit.jupiter.api.Test;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+
+/*******************************************************************************
+ ** Compiler isolation without spawning a child or writing temporary files.
+ *******************************************************************************/
+class CliMjmlCompilerTest
+{
+   /*******************************************************************************
+    ** Only PATH survives; includes are disabled even for an include-free input.
+    *******************************************************************************/
+   @Test
+   void isolatedProcessBuilder()
+   {
+      var builder = new CliMjmlCompiler("mjml").processBuilder();
+      assertThat(builder.environment().keySet()).allMatch("PATH"::equals);
+      assertThat(builder.command()).containsExactly("mjml", "-i", "-s", "--config.allowIncludes=false");
+      assertThat(builder.redirectError()).isEqualTo(ProcessBuilder.Redirect.DISCARD);
+   }
+
+   /*******************************************************************************
+    ** Refuse before attempting to launch even a nonexistent executable.
+    *******************************************************************************/
+   @Test
+   void includesRefusedBeforeLaunch()
+   {
+      assertThatThrownBy(() -> new CliMjmlCompiler("nonexistent").compile(""))
+         .hasMessage("MJML includes are not allowed");
+   }
+}
diff --git a/src/test/resources/email-h2.sql b/src/test/resources/email-h2.sql
index 12e7909..3bafb55 100644
--- a/src/test/resources/email-h2.sql
+++ b/src/test/resources/email-h2.sql
@@ -1,4 +1,4 @@
--- Phase (a), part 1. Apply before producing the qbit. Host security columns must be added by the host.
+-- Phase (a), parts 1 and 2. Apply before producing the qbit. Host security columns must be added by the host.
 
 CREATE TABLE email_provider (
    id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
@@ -58,6 +58,8 @@ CREATE TABLE email_message (
    html_body TEXT,
    text_body TEXT,
    body_sha256 VARCHAR(500),
+   phi_approved BOOLEAN,
+   unsubscribe_url TEXT,
    idempotency_key VARCHAR(500),
    provider_id INTEGER,
    provider_message_id VARCHAR(500),
@@ -68,6 +70,8 @@ CREATE TABLE email_message (
    finished_date TIMESTAMP,
    sent_date TIMESTAMP,
    delivered_date TIMESTAMP,
+   bounced_date TIMESTAMP,
+   complained_date TIMESTAMP,
    dead_letter_date TIMESTAMP,
    linked_table_name VARCHAR(500),
    linked_record_id VARCHAR(500),
@@ -105,6 +109,8 @@ CREATE TABLE email_template (
    kind VARCHAR(500),
    brand_id INTEGER,
    active_version_id INTEGER,
+   approved_version_id INTEGER,
+   version_lock INTEGER,
    phi_approved BOOLEAN,
    phi_approved_by VARCHAR(500),
    phi_approved_date TIMESTAMP,
@@ -148,6 +154,8 @@ CREATE TABLE email_brand (
    background_color VARCHAR(500),
    dark_background_color VARCHAR(500),
    dark_text_color VARCHAR(500),
+   dark_primary_color VARCHAR(500),
+   dark_button_text_color VARCHAR(500),
    button_text_color VARCHAR(500),
    footer_markdown TEXT,
    postal_address TEXT,