CHANGELOG.md

@@ -6,6 +6,9 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and
 ## [Unreleased]
 
 ### Added
+- Persisted unique RFC Message-IDs, stable across delivery attempts, with SMTP/Gmail SMTP, SES raw MIME and Postmark/SendGrid/Mailgun request propagation (SES replaces the header downstream).
+- Core-independent `EmailEffects` handoff, reference lookups by idempotency key or RFC Message-ID, refusal mapping and an RC.7 binding design sketch; offline retry and provider-contract tests.
+- Checkstyle now checks test sources as well as main sources; existing test imports and formatting corrected.
 - Provider sender addresses: child table `email_provider_sender_address` (validated lowercase address, display name, one default, enabled, note; unique per provider; domain bound to the provider's sending domain; admin-only), shown on the provider record. `EmailRequest.withFromAddressId`, `from_address_id` on messages, refusal of a from address that is not an enabled row of the resolved provider, and the provider `from_address` / `from_name` kept as the fallback.
 - Contingent possible-value filters on the new from-address pickers (send processes by `providerId`, Send Test Email by the acted-on provider, message `fromAddressId` by `providerId`) and on the attempt and event message pickers (by `providerId`) and the suppression source event (by `sourceMessageId`); the send processes also take `providerId`.
 - Phase (c): authenticated SNS, Postmark, SendGrid and Mailgun webhooks, durable inbox deduplication and Mailgun nonce replay protection; configurable Javalin routes.

README.md

@@ -501,6 +501,26 @@ concurrent idempotency, shared rate budgets, health recovery and process screens
 The tests also exercise template lifecycle, parent-scoped possible values, rendering,
 PHI canary egress, encryption, preview widgets and home data.
 
+## Durable jobs
+
+`EmailEffects.send(request, effectIdempotencyKey)` hands a message to the committed
+email outbox exactly once per key and returns an `EmailEffects.Result` containing
+`EmailEffectOutcome` and an `EmailMessageRef`. `QUEUED_OR_DUPLICATE` means the outbox
+accepted responsibility; `REFUSED` means a definite refusal, including suppression.
+Use `findByIdempotencyKey` or `findByRfcMessageId` to reconcile a handoff. These are
+trusted system reads returning only references. Database failures propagate.
+Requests must not carry a caller transaction; the effect dispatcher calls after its
+own chunk commit. Existing `EmailService` transaction support is unchanged.
+
+Each new message stores a unique `rfcMessageId` once. SMTP (including the Gmail preset),
+Postmark, SendGrid and Mailgun carry it on every attempt. SES receives it in the raw
+request but replaces it downstream; every transport retains `X-QQQ-Email-Id`.
+Message-ID enables correlation and receiver deduplication where supported; it does
+not promise exactly-once inbox delivery.
+
+See [the handoff contract, provider references, schema upgrade and RC.7 binding sketch](docs/DURABLE-JOBS.md).
+The build remains on released QQQ **4.1.0-RC.5**, without durable-jobs dependencies.
+
 ## License
 
 Apache License 2.0. See [LICENSE](LICENSE) and [NOTICE](NOTICE).

docs/DESIGN.md

@@ -185,6 +185,7 @@ transient; other 4xx are permanent.
 | variables_snapshot | JSON, redacted per section 7.3 |
 | html_body, text_body | rendered at queue time so retries send the same bytes; encrypted for `phi_approved`; cleared after `bodyRetentionDays` (default 30) once final, keeping `body_sha256` |
 | idempotency_key | unique when present |
+| rfc_message_id | STRING(500), unique; stored once at enqueue and reused on every attempt; see [durable job effects](DURABLE-JOBS.md) |
 | provider_id, provider_message_id | |
 | attempt_count, next_attempt_date, lease_expires_date, last_error, sent_date, delivered_date, dead_letter_date | |
 | linked_table_name, linked_record_id | rendered as a link on the record screen |

docs/DURABLE-JOBS.md

@@ -0,0 +1,123 @@
+# Durable job email effects
+
+This implements the email-side contract for QRun-IO/qqq#1191, spec section 2.13,
+without importing core jobs types. QQQ stays at released 4.1.0-RC.5. The
+`EmailJobEffectTarget implements EffectTarget` binding is deferred until RC.7 ships.
+
+## Stored identity and schema upgrade
+
+`email_message.rfc_message_id` (`rfcMessageId` in metadata) is a unique STRING(500)
+shown in the message T2 details. Before insertion, the service stores
+`<token@domain>`, where `token` is the first 32 lowercase hex characters of SHA-256
+of the UTF-8 idempotency key. Without a key, the seed is `qbit-email:` plus a fresh
+UUID, since the row id is not yet allocated. The domain is the normalized From
+mailbox domain, excluding display names; unavailable or unsuitable domains use
+`qbit-email.local`. The key itself is never included in the header.
+
+The dispatcher reads this stored value on every attempt. JavaMail's
+`updateMessageID()` is overridden so repeated `saveChanges()` calls cannot replace
+it. A manual **Resend** creates a new outbox row and a new identity; a delivery retry
+keeps its identity. `providerMessageId` remains the provider's response identifier,
+distinct from `rfcMessageId` and the numeric outbox `id`.
+
+New installations use [email-postgres.sql](ddl/email-postgres.sql). Existing
+PostgreSQL installations must pause enqueue and dispatch, then apply
+[email-message-id-upgrade.sql](ddl/email-message-id-upgrade.sql) before starting
+this version. The migration backfills old rows using the existing key, or
+`qbit-email:` plus the already allocated row id, and the stored From domain.
+It cannot recover previously generated random transport Message-IDs. Review host
+schema adaptations first; host security columns are unaffected.
+
+## Provider behavior
+
+| Transport | Request mechanism | Identity preservation |
+|---|---|---|
+| SMTP / Gmail SMTP preset | MIME `Message-ID` | Sent unchanged by this transport; downstream relay policy still applies |
+| SES v2 | Raw MIME `Message-ID` | **SES replaces Message-ID downstream** |
+| Postmark | `Headers: [{Name: "Message-ID", Value: "<...>"}]` | Custom Message-ID supported |
+| SendGrid | `headers: {"Message-ID": "<...>"}` | Sent as a custom header |
+| Mailgun | MIME Message-ID plus multipart `h:Message-Id` | Sent in the `/messages.mime` request |
+
+All transports retain `X-QQQ-Email-Id` with the numeric outbox id. Postmark's
+`MessageID`, SendGrid's response `X-Message-Id`, and SES's returned id are provider
+references, not our RFC header. A fixed header enables recognition and receiver
+deduplication where supported, but is not an exactly-once inbox guarantee.
+
+Provider references: [Postmark custom Message-ID](https://postmarkapp.com/developer/user-guide/send-email-with-api/send-a-single-email),
+[SendGrid custom headers](https://www.twilio.com/docs/sendgrid/api-reference/mail-send/mail-send),
+[Mailgun sending parameters](https://documentation.mailgun.com/docs/mailgun/api-reference/send/mailgun/messages/post-v3--domain-name--messages),
+and [SES header replacement](https://docs.aws.amazon.com/ses/latest/APIReference/API_SendRawEmail.html).
+
+## Handoff and reconcile
+
+`EmailEffects.send(request, effectIdempotencyKey)` sets the request's key, refusing
+a different pre-existing key, a missing/blank/overlength effect key, a null request
+or a caller transaction. The service performs its usual validation and send
+permission checks. Success is returned only after the outbox transaction commits;
+there is no transport call here. Use ordinary `EmailService.send` for caller-owned
+or cross-backend deferred transactions.
+
+`EmailEffects.Result(outcome, message)` contains an `EmailEffectOutcome` and a
+nullable `EmailMessageRef(Integer id, String status, String rfcMessageId,
+String providerMessageId)`:
+
+- `QUEUED_OR_DUPLICATE`: one committed row accepted the handoff, including replays.
+- `REFUSED`: invalid or suppressed request. A suppressed audit row has a reference;
+  refusals without a stored row have a null reference. Replaying a suppressed row
+  stays refused even though `EmailService` returns `DUPLICATE`.
+- Database/query failures throw `QException`; callers must retry or reconcile,
+  never turn an uncertain outcome into a permanent refusal.
+
+`findByIdempotencyKey(key)` and `findByRfcMessageId(messageId)` return
+`Optional<EmailMessageRef>` from system reads, like other internal outbox reads.
+They return empty for null/blank/missing values. These are trusted backend APIs,
+not viewer-scoped public endpoints. Lookups reflect current status and provider
+reference and do not expose bodies, recipients or variables.
+
+The effect's `SENT` means successful durable handoff, not mailbox delivery. The
+email dispatcher owns later delivery/retry failures. Reconcile acknowledges an
+existing outbox row (including an audit row); it must never create another email
+merely because that row is terminal. Retain message rows and idempotency keys for
+at least the maximum effect retry/reconciliation horizon.
+
+## RC.7 binding design note (not compiled)
+
+Mapping: `deliver -> QUEUED_OR_DUPLICATE => SENT(ref.id)`, `REFUSED => FAILED`;
+`reconcile -> findByIdempotencyKey present => FOUND(ref.id), absent => NOT_FOUND`.
+Exceptions escape to core's retry/unknown handling. Payload decoding must preserve
+the existing PHI/variable policies and must not attach a caller transaction.
+
+This 25-line sketch is documentation-only. Match core names and signatures to the
+published RC.7 contract when it ships; `decode` represents payload conversion.
+
+```java
+// Future RC.7 binding; no core jobs imports in the RC.5 implementation.
+final class EmailJobEffectTarget implements EffectTarget
+{
+   // SENT acknowledges the committed qbit outbox row.
+   @Override
+   public DeliveryResult deliver(Effect effect) throws Exception
+   {
+      EmailRequest request = decode(effect.payload());
+      EmailEffects.Result result = EmailEffects.send(request, effect.idempotencyKey());
+      if(result.outcome() == EmailEffectOutcome.REFUSED)
+      {
+         return (DeliveryResult.failed("Email refused"));
+      }
+      return (DeliveryResult.sent(result.message().id().toString()));
+   }
+
+   // Existence reconciles handoff independently of mailbox delivery.
+   @Override
+   public ReconcileResult reconcile(Effect effect) throws Exception
+   {
+      return (EmailEffects.findByIdempotencyKey(effect.idempotencyKey())
+         .map(ref -> ReconcileResult.found(ref.id().toString()))
+         .orElseGet(ReconcileResult::notFound));
+   }
+}
+```
+
+Tests use H2, the existing GreenMail SMTP sink, fake SES and HTTP request seams;
+no vendor endpoints are called. The PostgreSQL migration is supplied for host
+deployment and is not executed by the offline Java test gate.

docs/ddl/email-message-id-upgrade.sql

@@ -0,0 +1,11 @@
+-- Existing PostgreSQL installations: pause enqueue and dispatch before applying.
+-- SHA-256 is PostgreSQL's built-in bytea function; no extension is required.
+BEGIN;
+ALTER TABLE email_message ADD COLUMN rfc_message_id VARCHAR(500);
+UPDATE email_message
+SET rfc_message_id = '<' || left(encode(sha256(convert_to(
+   coalesce(idempotency_key, 'qbit-email:' || id::text), 'UTF8')), 'hex'), 32)
+   || '@' || coalesce(lower(substring(from_address from
+      '@([A-Za-z0-9]+(?:[.-][A-Za-z0-9]+)*)(?:>|$)')), 'qbit-email.local') || '>';
+ALTER TABLE email_message ADD CONSTRAINT email_message_rfc_message_id_unique UNIQUE (rfc_message_id);
+COMMIT;

docs/ddl/email-postgres.sql

@@ -75,6 +75,7 @@ CREATE TABLE email_message (
    phi_approved BOOLEAN,
    unsubscribe_url TEXT,
    idempotency_key VARCHAR(500),
+   rfc_message_id VARCHAR(500),
    provider_id INTEGER,
    provider_message_id VARCHAR(500),
    attempt_count INTEGER,
@@ -93,7 +94,8 @@ CREATE TABLE email_message (
    sender_process VARCHAR(500),
    suppressed_addresses TEXT,
    send_after TIMESTAMP,
-   UNIQUE (idempotency_key)
+   UNIQUE (idempotency_key),
+   UNIQUE (rfc_message_id)
 );
 
 CREATE TABLE email_message_attempt (

pom.xml

@@ -168,7 +168,7 @@
                <failsOnError>false</failsOnError>
                <failOnViolation>true</failOnViolation>
                <violationSeverity>warning</violationSeverity>
-               <includeTestSourceDirectory>false</includeTestSourceDirectory>
+               <includeTestSourceDirectory>true</includeTestSourceDirectory>
             </configuration>
             <executions>
                <execution>

src/main/java/com/kingsrook/qbits/email/delivery/EmailDispatcher.java

@@ -451,7 +451,7 @@ public class EmailDispatcher
     *******************************************************************************/
    private static EmailEnvelope envelope(QRecord message)
    {
-      return (new EmailEnvelope().withMessageId(message.getValueString("id")).withFrom(message.getValueString("fromAddress"))
+      return (new EmailEnvelope().withMessageId(message.getValueString("id")).withRfcMessageId(message.getValueString("rfcMessageId")).withFrom(message.getValueString("fromAddress"))
          .withTo(split(message.getValueString("toAddresses"))).withCc(split(message.getValueString("ccAddresses"))).withBcc(split(message.getValueString("bccAddresses")))
          .withReplyTo(message.getValueString("replyTo")).withSubject(content(message, "subject"))
          .withHtmlBody(content(message, "htmlBody"))

src/main/java/com/kingsrook/qbits/email/metadata/EmailTables.java

@@ -269,6 +269,7 @@ public final class EmailTables
          .withField(field("unsubscribeUrl", QFieldType.TEXT))
          .withField(field("bodySha256", QFieldType.STRING).withMaxLength(500))
          .withField(field("idempotencyKey", QFieldType.STRING).withMaxLength(500))
+         .withField(field("rfcMessageId", QFieldType.STRING).withMaxLength(500))
          .withField(field("providerId", QFieldType.INTEGER).withPossibleValueSourceName(PROVIDER))
          .withField(field("providerMessageId", QFieldType.STRING).withMaxLength(500))
          .withField(field("attemptCount", QFieldType.INTEGER).withDefaultValue(0))
@@ -291,8 +292,9 @@ public final class EmailTables
       table.withRecordLabelFormat(null).withRecordLabelFields(List.of());
       table.getField("subject").setIsHeavy(false);
       table.withUniqueKey(new UniqueKey("idempotencyKey"));
+      table.withUniqueKey(new UniqueKey("rfcMessageId"));
       table.withSection(SectionFactory.defaultT1("id", "status", "toAddresses", "ccAddresses", "bccAddresses", "fromAddress"));
-      table.withSection(SectionFactory.defaultT2("fromAddressId", "replyTo", "subject", "templateId", "templateVersionId", "brandId", "category", "locale", "variablesSnapshot", "htmlBody", "textBody", "phiApproved", "unsubscribeUrl", "bodySha256", "idempotencyKey", "providerId", "providerMessageId", "attemptCount", "nextAttemptDate", "leaseExpiresDate", "lastError", "finishedDate", "sentDate", "deliveredDate", "bouncedDate", "complainedDate", "deadLetterDate", "linkedTableName", "linkedRecordId", "senderQbit", "senderProcess", "suppressedAddresses", "sendAfter"));
+      table.withSection(SectionFactory.defaultT2("fromAddressId", "replyTo", "subject", "templateId", "templateVersionId", "brandId", "category", "locale", "variablesSnapshot", "htmlBody", "textBody", "phiApproved", "unsubscribeUrl", "bodySha256", "idempotencyKey", "rfcMessageId", "providerId", "providerMessageId", "attemptCount", "nextAttemptDate", "leaseExpiresDate", "lastError", "finishedDate", "sentDate", "deliveredDate", "bouncedDate", "complainedDate", "deadLetterDate", "linkedTableName", "linkedRecordId", "senderQbit", "senderProcess", "suppressedAddresses", "sendAfter"));
       table.withSection(SectionFactory.defaultT3("createDate", "modifyDate"));
       table.withSection(SectionFactory.customT2("children", new QIcon("list")).withLabel("Attempts").withWidgetName(MESSAGE + "_children"));
       table.withSection(SectionFactory.customT2("events", new QIcon("list")).withLabel("Events").withWidgetName(EVENT + "_children"));

src/main/java/com/kingsrook/qbits/email/outbox/EmailEffectOutcome.java

@@ -0,0 +1,31 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.email.outbox;
+
+
+/*******************************************************************************
+ ** Durable handoff outcomes. Operational failures propagate to the caller.
+ *******************************************************************************/
+public enum EmailEffectOutcome
+{
+   QUEUED_OR_DUPLICATE,
+   REFUSED
+}

src/main/java/com/kingsrook/qbits/email/outbox/EmailEffects.java

@@ -0,0 +1,114 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.email.outbox;
+
+
+import java.util.List;
+import java.util.Optional;
+import com.kingsrook.qbits.email.metadata.EmailTables;
+import com.kingsrook.qbits.email.support.EmailStore;
+import com.kingsrook.qbits.email.support.SystemWrites;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+
+
+/*******************************************************************************
+ ** Outbox handoff and system-read reconciliation for trusted effect dispatchers.
+ ** This API has no dependency on core durable-job types. Sending retains the
+ ** EmailService permission checks; callers must supply a request without a transaction.
+ *******************************************************************************/
+public final class EmailEffects
+{
+   /*******************************************************************************
+    ** A refusal can have a reference when suppression retained an audit row.
+    *******************************************************************************/
+   public record Result(EmailEffectOutcome outcome, EmailMessageRef message)
+   {
+   }
+
+
+
+   /*******************************************************************************
+    ** Utility class.
+    *******************************************************************************/
+   private EmailEffects()
+   {
+   }
+
+
+
+   /*******************************************************************************
+    ** Hand off once with the effect's identity. Refuse invalid or conflicting keys
+    ** and caller transactions: success must mean the outbox has committed already.
+    ** Database failures throw QException and must not become a permanent refusal.
+    *******************************************************************************/
+   public static Result send(EmailRequest request, String effectIdempotencyKey) throws QException
+   {
+      if(request == null || effectIdempotencyKey == null || effectIdempotencyKey.isBlank() || effectIdempotencyKey.length() > 500
+         || request.getTransaction() != null
+         || request.getIdempotencyKey() != null && !request.getIdempotencyKey().equals(effectIdempotencyKey))
+      {
+         return (new Result(EmailEffectOutcome.REFUSED, null));
+      }
+      request.withIdempotencyKey(effectIdempotencyKey);
+      EmailSendResult result = EmailService.send(request);
+      EmailMessageRef message = result.messageId() == null ? null : find("id", result.messageId()).orElseThrow(() -> new QException("Queued email reference is unavailable"));
+      boolean refused = result.status() == EmailSendResult.Status.REFUSED || result.status() == EmailSendResult.Status.SUPPRESSED
+         || message != null && "SUPPRESSED".equals(message.status());
+      return (new Result(refused ? EmailEffectOutcome.REFUSED : EmailEffectOutcome.QUEUED_OR_DUPLICATE, message));
+   }
+
+
+
+   /*******************************************************************************
+    ** Look up a committed outbox row by its exact effect or application key.
+    *******************************************************************************/
+   public static Optional<EmailMessageRef> findByIdempotencyKey(String key) throws QException
+   {
+      return (key == null || key.isBlank() ? Optional.empty() : find("idempotencyKey", key));
+   }
+
+
+
+   /*******************************************************************************
+    ** Look up the exact bracketed RFC Message-ID stored at enqueue time.
+    *******************************************************************************/
+   public static Optional<EmailMessageRef> findByRfcMessageId(String messageId) throws QException
+   {
+      return (messageId == null || messageId.isBlank() ? Optional.empty() : find("rfcMessageId", messageId));
+   }
+
+
+
+   /*******************************************************************************
+    ** Use the same authorized read scope as the dispatcher; expose only references.
+    *******************************************************************************/
+   private static Optional<EmailMessageRef> find(String field, java.io.Serializable value) throws QException
+   {
+      List<QRecord> rows = SystemWrites.query(EmailTables.MESSAGE, EmailStore.equal(field, value).withLimit(1));
+      if(rows.isEmpty())
+      {
+         return (Optional.empty());
+      }
+      QRecord row = rows.getFirst();
+      return (Optional.of(new EmailMessageRef(row.getValueInteger("id"), row.getValueString("status"), row.getValueString("rfcMessageId"), row.getValueString("providerMessageId"))));
+   }
+}

src/main/java/com/kingsrook/qbits/email/outbox/EmailMessageRef.java

@@ -0,0 +1,29 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.email.outbox;
+
+
+/*******************************************************************************
+ ** Reconciliation reference without message bodies, recipients or credentials.
+ *******************************************************************************/
+public record EmailMessageRef(Integer id, String status, String rfcMessageId, String providerMessageId)
+{
+}

src/main/java/com/kingsrook/qbits/email/outbox/EmailService.java

@@ -30,6 +30,7 @@ import java.util.LinkedHashSet;
 import java.util.List;
 import java.util.Locale;
 import java.util.Set;
+import java.util.UUID;
 import com.kingsrook.qbits.email.EmailQBitConfig;
 import com.kingsrook.qbits.email.EmailRuntime;
 import com.kingsrook.qbits.email.metadata.EmailTableCustomizers;
@@ -261,6 +262,7 @@ public final class EmailService
          .withValue("htmlBody", rendered.html()).withValue("textBody", rendered.text()).withValue("bodySha256", digest(rendered))
          .withValue("brandId", adHocBrand == null ? null : adHocBrand.getValue("id"))
          .withValue("category", category.name()).withValue("locale", request.getLocale()).withValue("idempotencyKey", request.getIdempotencyKey())
+         .withValue("rfcMessageId", messageId(request.getIdempotencyKey(), from))
          .withValue("linkedTableName", request.getLinkedTableName()).withValue("linkedRecordId", request.getLinkedRecordId())
          .withValue("senderQbit", request.getSenderQbit()).withValue("senderProcess", request.getSenderProcess())
          .withValue("sendAfter", request.getSendAfter()).withValue("nextAttemptDate", request.getSendAfter() == null ? now : request.getSendAfter())
@@ -391,18 +393,10 @@ public final class EmailService
       {
          return (new EmailSendResult(existing.getValueInteger("id"), EmailSendResult.Status.DUPLICATE, List.of()));
       }
-      QRecord inserted;
-      try
-      {
-         inserted = SystemWrites.insert(EmailTables.MESSAGE, List.of(row), transaction).getRecords().get(0);
-      }
-      catch(QException e)
-      {
-         return (refused("Could not queue the email message"));
-      }
+      QRecord inserted = SystemWrites.insert(EmailTables.MESSAGE, List.of(row), transaction).getRecords().get(0);
       if(inserted.getErrors() != null && !inserted.getErrors().isEmpty())
       {
-         return (refused("Could not queue the email message"));
+         throw (new QException("Could not queue the email message"));
       }
       try
       {
@@ -596,6 +590,42 @@ public final class EmailService
       }
    }
 
+
+
+   /*******************************************************************************
+    ** Create once before insertion. A UUID seeds messages without an idempotency key.
+    ** The mailbox parser ignores display names; malformed domains use a local fallback.
+    *******************************************************************************/
+   static String messageId(String key, String from)
+   {
+      String domain = "qbit-email.local";
+      try
+      {
+         String address = new InternetAddress(from).getAddress();
+         String candidate = address.substring(address.lastIndexOf('@') + 1);
+         if(address.contains("@") && candidate.matches("[A-Za-z0-9]+(?:[.-][A-Za-z0-9]+)*"))
+         {
+            domain = candidate.toLowerCase(Locale.ROOT);
+         }
+      }
+      catch(Exception e)
+      {
+         // A fallback also supports callers preparing legacy message rows.
+      }
+      try
+      {
+         String seed = key == null ? "qbit-email:" + UUID.randomUUID() : key;
+         String token = HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(seed.getBytes(StandardCharsets.UTF_8))).substring(0, 32);
+         return ("<" + token + "@" + domain + ">");
+      }
+      catch(java.security.NoSuchAlgorithmException e)
+      {
+         throw (new IllegalStateException("SHA-256 unavailable"));
+      }
+   }
+
+
+
    /*******************************************************************************
     ** Refusal with no stored row.
     *******************************************************************************/

src/main/java/com/kingsrook/qbits/email/transport/EmailEnvelope.java

@@ -31,6 +31,7 @@ import java.util.Map;
 public class EmailEnvelope
 {
    private String messageId;
+   private String rfcMessageId;
    private String from;
    private List<String> to = List.of();
    private List<String> cc = List.of();
@@ -62,6 +63,27 @@ public class EmailEnvelope
    }
 
 
+
+   /*******************************************************************************
+    ** Getter for the persisted RFC 5322 Message-ID, distinct from the outbox row id.
+    *******************************************************************************/
+   public String getRfcMessageId()
+   {
+      return (rfcMessageId);
+   }
+
+
+
+   /*******************************************************************************
+    ** Fluent setter for the persisted RFC 5322 Message-ID.
+    *******************************************************************************/
+   public EmailEnvelope withRfcMessageId(String rfcMessageId)
+   {
+      this.rfcMessageId = rfcMessageId;
+      return (this);
+   }
+
+
    /*******************************************************************************
     ** Getter for from.
     *******************************************************************************/

src/main/java/com/kingsrook/qbits/email/transport/EmailHttpSupport.java

@@ -81,13 +81,17 @@ final class EmailHttpSupport
       Map<String, String> headers = new LinkedHashMap<>();
       envelope.getHeaders().forEach((name, value) ->
       {
-         if(!name.equalsIgnoreCase("Bcc") && !name.equalsIgnoreCase("Auto-Submitted") && !name.equalsIgnoreCase("X-QQQ-Email-Id"))
+         if(!name.equalsIgnoreCase("Bcc") && !name.equalsIgnoreCase("Auto-Submitted") && !name.equalsIgnoreCase("X-QQQ-Email-Id") && !name.equalsIgnoreCase("Message-ID"))
          {
             headers.put(name, value);
          }
       });
       headers.put("Auto-Submitted", "auto-generated");
       headers.put("X-QQQ-Email-Id", envelope.getMessageId());
+      if(envelope.getRfcMessageId() != null)
+      {
+         headers.put("Message-ID", envelope.getRfcMessageId());
+      }
       return (headers);
    }
 

src/main/java/com/kingsrook/qbits/email/transport/EmailMime.java

@@ -53,7 +53,26 @@ final class EmailMime
       noLineBreaks(envelope.getFrom());
       noLineBreaks(envelope.getReplyTo());
       noLineBreaks(envelope.getMessageId());
-      MimeMessage message = new MimeMessage(session);
+      noLineBreaks(envelope.getRfcMessageId());
+      String rfcMessageId = envelope.getRfcMessageId();
+      MimeMessage message = new MimeMessage(session)
+      {
+         /*******************************************************************************
+          ** JavaMail may save again when sending; keep the persisted identity each time.
+          *******************************************************************************/
+         @Override
+         protected void updateMessageID() throws MessagingException
+         {
+            if(rfcMessageId == null)
+            {
+               super.updateMessageID();
+            }
+            else
+            {
+               setHeader("Message-ID", rfcMessageId);
+            }
+         }
+      };
       message.setFrom(new InternetAddress(envelope.getFrom(), true));
       recipients(message, Message.RecipientType.TO, envelope.getTo());
       recipients(message, Message.RecipientType.CC, envelope.getCc());

src/main/java/com/kingsrook/qbits/email/transport/MailgunEmailTransport.java

@@ -96,6 +96,10 @@ public class MailgunEmailTransport implements EmailTransport
          field(body, boundary, "o:tracking-opens", settings.enabled("trackOpens") ? "yes" : "no");
          field(body, boundary, "o:tracking-clicks", settings.enabled("trackClicks") ? "yes" : "no");
          field(body, boundary, "v:qqq_email_id", envelope.getMessageId());
+         if(envelope.getRfcMessageId() != null)
+         {
+            field(body, boundary, "h:Message-Id", envelope.getRfcMessageId());
+         }
          body.writeBytes(("--" + boundary + "\r\nContent-Disposition: form-data; name=\"message\"; filename=\"message.eml\"\r\n"
             + "Content-Type: message/rfc822\r\n\r\n").getBytes(StandardCharsets.UTF_8));
          EmailMime.build(Session.getInstance(new Properties()), envelope).writeTo(body, new String[] {"Bcc"});

src/main/java/com/kingsrook/qbits/email/transport/SesEmailTransport.java

@@ -50,6 +50,9 @@ import software.amazon.awssdk.services.sesv2.model.SesV2Exception;
 
 /*******************************************************************************
  ** SES v2 raw MIME transport; credentials are resolved only for an attempt.
+ ** The raw request carries the stored Message-ID, but SES replaces it downstream.
+ ** Receiver deduplication by that header is unavailable with SES; X-QQQ-Email-Id
+ ** remains the application correlation header. See docs/DURABLE-JOBS.md.
  *******************************************************************************/
 public class SesEmailTransport implements EmailTransport
 {

src/test/java/com/kingsrook/qbits/email/EmailDeliveryTest.java

@@ -21,18 +21,34 @@
 package com.kingsrook.qbits.email;
 
 import java.net.ServerSocket;
-import java.time.*;
-import java.util.*;
-import java.util.concurrent.*;
+import java.time.Clock;
+import java.time.Duration;
+import java.time.Instant;
+import java.time.ZoneOffset;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.CyclicBarrier;
+import java.util.concurrent.Executors;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
 import java.util.concurrent.atomic.AtomicInteger;
 import com.icegreen.greenmail.util.GreenMail;
 import com.icegreen.greenmail.util.ServerSetup;
-import com.kingsrook.qbits.email.delivery.*;
+import com.kingsrook.qbits.email.delivery.EmailDispatcher;
 import com.kingsrook.qbits.email.metadata.EmailTables;
-import com.kingsrook.qbits.email.outbox.*;
+import com.kingsrook.qbits.email.outbox.EmailCategory;
+import com.kingsrook.qbits.email.outbox.EmailRequest;
+import com.kingsrook.qbits.email.outbox.EmailSendResult;
+import com.kingsrook.qbits.email.outbox.EmailService;
 import com.kingsrook.qbits.email.processes.EmailProcesses;
-import com.kingsrook.qbits.email.support.*;
-import com.kingsrook.qbits.email.transport.*;
+import com.kingsrook.qbits.email.support.EmailStore;
+import com.kingsrook.qbits.email.support.SystemWrites;
+import com.kingsrook.qbits.email.transport.EmailTransport;
+import com.kingsrook.qbits.email.transport.EmailTransportException;
+import com.kingsrook.qbits.email.transport.EmailTransportRegistry;
+import com.kingsrook.qbits.email.transport.EmailTransportResult;
+import com.kingsrook.qbits.email.transport.SmtpEmailTransport;
 import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
 import com.kingsrook.qqq.backend.core.actions.messaging.SendMessageAction;
 import com.kingsrook.qqq.backend.core.alerts.AlertResolution;
@@ -41,16 +57,23 @@ import com.kingsrook.qqq.backend.core.alerts.AlertSinkInterface;
 import com.kingsrook.qqq.backend.core.alerts.AlertsInstanceMetaData;
 import com.kingsrook.qqq.backend.core.context.QContext;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.messaging.*;
-import com.kingsrook.qqq.backend.core.model.actions.messaging.email.*;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.Content;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.MultiParty;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.Party;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.SendMessageInput;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.email.EmailContentRole;
+import com.kingsrook.qqq.backend.core.model.actions.messaging.email.EmailPartyRole;
 import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
 import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
 import jakarta.mail.Multipart;
-import org.junit.jupiter.api.*;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
 import org.junit.jupiter.params.ParameterizedTest;
 import org.junit.jupiter.params.provider.ValueSource;
-import static org.assertj.core.api.Assertions.*;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
 
 /*******************************************************************************
  ** Email Delivery Test.
@@ -398,6 +421,39 @@ class EmailDeliveryTest
       assertThat(H2Support.all(EmailTables.ATTEMPT)).hasSize(1);
    }
 
+   /*******************************************************************************
+    ** A lost acknowledgement retries the stored identity through the real SMTP sink.
+    *******************************************************************************/
+   @Test
+   void smtpRetryPreservesStoredMessageId() throws Exception
+   {
+      var queued = EmailService.send(request().withIdempotencyKey("job:1:row:mail"));
+      String stored = message(queued.messageId()).getValueString("rfcMessageId");
+      AtomicInteger calls = new AtomicInteger();
+      var dispatcher = fake((envelope, settings) ->
+      {
+         assertThat(envelope.getRfcMessageId()).isEqualTo(stored);
+         var sent = new SmtpEmailTransport().send(envelope, settings);
+         if(calls.incrementAndGet() == 1)
+         {
+            throw new EmailTransportException("Lost acknowledgement", false);
+         }
+         return (sent);
+      });
+      assertThat(dispatcher.sendNow(queued.messageId())).isEqualTo(EmailDispatcher.SendResult.RETRY_WAIT);
+      EmailRuntime.setClock(Clock.fixed(now.plusSeconds(301), ZoneOffset.UTC));
+      assertThat(dispatcher.sendNow(queued.messageId())).isEqualTo(EmailDispatcher.SendResult.SENT);
+      assertThat(mail.waitForIncomingEmail(2000, 2)).isTrue();
+      assertThat(mail.getReceivedMessages()).hasSize(2);
+      for(var received : mail.getReceivedMessages())
+      {
+         assertThat(received.getMessageID()).isEqualTo(stored);
+         assertThat(received.getHeader("X-QQQ-Email-Id")).containsExactly(queued.messageId().toString());
+      }
+      assertThat(message(queued.messageId()).getValueString("rfcMessageId")).isEqualTo(stored);
+      assertThat(H2Support.all(EmailTables.ATTEMPT)).hasSize(2);
+   }
+
    /*******************************************************************************
     ** Smtp Authentication And Secret Resolution Only At Send.
     *******************************************************************************/
@@ -477,8 +533,8 @@ class EmailDeliveryTest
       var second = EmailService.send(request());
       fake((e, s) ->
       {
-               throw new EmailTransportException("throttle", false).withRetryAfter(Duration.ofHours(2));
-            })
+         throw new EmailTransportException("throttle", false).withRetryAfter(Duration.ofHours(2));
+      })
             .runOnce();
       assertThat(message(second.messageId()).getValueInstant("nextAttemptDate"))
             .isEqualTo(now.plusSeconds(3960));
@@ -494,8 +550,8 @@ class EmailDeliveryTest
       assertThat(
                   fake((e, s) ->
                   {
-                           throw new EmailTransportException("rejected", true);
-                        })
+                     throw new EmailTransportException("rejected", true);
+                  })
                         .sendNow(queued.messageId()))
             .isEqualTo(EmailDispatcher.SendResult.DEAD_LETTER);
       var future = EmailService.send(request());

src/test/java/com/kingsrook/qbits/email/EmailHomePolishTest.java

@@ -30,9 +30,9 @@ import java.util.Set;
 import com.kingsrook.qbits.email.app.EmailHome;
 import com.kingsrook.qbits.email.metadata.EmailEnums;
 import com.kingsrook.qbits.email.metadata.EmailTables;
+import com.kingsrook.qbits.email.outbox.EmailCategory;
 import com.kingsrook.qbits.email.render.ContrastCheck;
 import com.kingsrook.qbits.email.render.EmailLayout;
-import com.kingsrook.qbits.email.outbox.EmailCategory;
 import com.kingsrook.qbits.email.support.SystemWrites;
 import com.kingsrook.qqq.backend.core.actions.dashboard.RenderWidgetAction;
 import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
@@ -66,7 +66,9 @@ class EmailHomePolishTest
    @BeforeEach
    void setup() throws Exception
    {
-      H2Support.setup(c -> { });
+      H2Support.setup(c ->
+      {
+      });
       EmailRuntime.setClock(Clock.fixed(NOW, ZoneOffset.UTC));
       H2Support.provider(25);
    }

src/test/java/com/kingsrook/qbits/email/EmailMetadataTest.java

@@ -20,14 +20,24 @@
 
 package com.kingsrook.qbits.email;
 
-import java.util.*;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Set;
 import com.kingsrook.qbits.email.app.EmailApp;
-import com.kingsrook.qbits.email.metadata.*;
-import com.kingsrook.qbits.email.outbox.*;
-import com.kingsrook.qbits.email.permissions.*;
-import com.kingsrook.qbits.email.processes.*;
-import com.kingsrook.qbits.email.support.*;
-import com.kingsrook.qqq.backend.core.actions.tables.*;
+import com.kingsrook.qbits.email.metadata.EmailTableCustomizers;
+import com.kingsrook.qbits.email.metadata.EmailTables;
+import com.kingsrook.qbits.email.outbox.AdHocEmailRenderer;
+import com.kingsrook.qbits.email.outbox.EmailRequest;
+import com.kingsrook.qbits.email.outbox.EmailSendResult;
+import com.kingsrook.qbits.email.outbox.EmailService;
+import com.kingsrook.qbits.email.permissions.EmailPermissionChecker;
+import com.kingsrook.qbits.email.processes.EmailProcesses;
+import com.kingsrook.qbits.email.support.EmailStore;
+import com.kingsrook.qbits.email.support.SystemWrites;
+import com.kingsrook.qqq.backend.core.actions.tables.DeleteAction;
+import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
+import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
+import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
 import com.kingsrook.qqq.backend.core.context.QContext;
 import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
@@ -36,10 +46,14 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.*;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.Capability;
-import org.junit.jupiter.api.*;
-import static org.assertj.core.api.Assertions.*;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
 
 /*******************************************************************************
  ** Email Metadata Test.
@@ -412,7 +426,7 @@ class EmailMetadataTest
                   "MAILGUN",
                   "GMAIL_API",
                   "future"))
-                  {
+      {
          assertThat(
                      new EmailQBitConfig()
                            .withAllowedEnvNames(Set.of("CUSTOM"))

src/test/java/com/kingsrook/qbits/email/EmailProcessesTest.java

@@ -20,20 +20,28 @@
 
 package com.kingsrook.qbits.email;
 
-import java.time.*;
-import java.util.*;
-import com.icegreen.greenmail.util.*;
-import com.kingsrook.qbits.email.delivery.*;
-import com.kingsrook.qbits.email.metadata.*;
-import com.kingsrook.qbits.email.outbox.*;
-import com.kingsrook.qbits.email.processes.*;
-import com.kingsrook.qbits.email.support.*;
+import java.time.Clock;
+import java.time.Duration;
+import java.time.Instant;
+import java.time.ZoneOffset;
+import java.util.Map;
+import com.icegreen.greenmail.util.GreenMail;
+import com.icegreen.greenmail.util.ServerSetup;
+import com.kingsrook.qbits.email.metadata.EmailTables;
+import com.kingsrook.qbits.email.outbox.EmailRequest;
+import com.kingsrook.qbits.email.outbox.EmailService;
+import com.kingsrook.qbits.email.processes.EmailProcesses;
+import com.kingsrook.qbits.email.support.EmailStore;
+import com.kingsrook.qbits.email.support.SystemWrites;
 import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
 import com.kingsrook.qqq.backend.core.context.QContext;
-import com.kingsrook.qqq.backend.core.model.actions.processes.*;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessOutput;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import org.junit.jupiter.api.*;
-import static org.assertj.core.api.Assertions.*;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import static org.assertj.core.api.Assertions.assertThat;
 
 /*******************************************************************************
  ** Email Processes Test.

src/test/java/com/kingsrook/qbits/email/EmailReviewFixesTest.java

@@ -109,7 +109,7 @@ class EmailReviewFixesTest
       var schema = new VariableSchema("[{\"name\":\"items\",\"type\":\"LIST\"},{\"name\":\"value\",\"type\":\"STRING\"}]");
       var loop = engine.parse("{{#each items}}{{#each items}}{{/each}}{{/each}}", schema);
       assertThatThrownBy(() -> loop.render(Map.of("items", Collections.nCopies(100, "")), EmailTemplateEngine.Context.TEXT)).hasMessage("Template output is too large");
-      assertThatThrownBy(() -> engine.parse("{{value}}", schema).render(Map.of("value", "\u00e9".repeat(600_000)), EmailTemplateEngine.Context.TEXT))
+      assertThatThrownBy(() -> engine.parse("{{value}}", schema).render(Map.of("value", "é".repeat(600_000)), EmailTemplateEngine.Context.TEXT))
          .hasMessage("Template output is too large");
       assertThat(engine.parse("{{#if value}}".repeat(8) + "ok" + "{{/if}}".repeat(8), schema).render(Map.of("value", true), EmailTemplateEngine.Context.TEXT)).isEqualTo("ok");
       assertThatThrownBy(() -> engine.parse("{{#if value}}".repeat(9) + "{{/if}}".repeat(9), schema)).hasMessageContaining("8 levels");
@@ -179,7 +179,10 @@ class EmailReviewFixesTest
       {
          assertThatThrownBy(() -> EmailTemplateEngine.validateHtml(source)).hasMessageContaining("must be quoted");
       }
-      EmailRuntime.config().withMjmlCompiler(source -> { throw new AssertionError("Unsafe MJML reached compiler"); });
+      EmailRuntime.config().withMjmlCompiler(source ->
+      {
+         throw new AssertionError("Unsafe MJML reached compiler");
+      });
       QRecord mjml = EmailTemplatesTest.draft("includes", "MJML", "[]", "{}", "<mjml><mj-include path='secret'/></mjml>");
       assertThatThrownBy(() -> EmailVersions.activate(mjml.getValueInteger("id"))).hasMessage("MJML includes are not allowed");
       assertThat(preview(mjml)).contains("MJML includes are not allowed");
@@ -192,7 +195,10 @@ class EmailReviewFixesTest
    void viewerNeverCompiles() throws Exception
    {
       QRecord version = EmailTemplatesTest.draft("viewer", "MJML", "[]", "{}", "<mjml/>");
-      EmailRuntime.config().withMjmlCompiler(source -> { throw new AssertionError("Viewer compiled MJML"); });
+      EmailRuntime.config().withMjmlCompiler(source ->
+      {
+         throw new AssertionError("Viewer compiled MJML");
+      });
       QContext.getQSession().setPermissions(Set.of("email.view"));
       assertThat(preview(version)).contains("Not compiled yet. Activate the version or use Preview as a manager.");
       QContext.getQSession().setPermissions(Set.of("email.manage"));
@@ -270,7 +276,10 @@ class EmailReviewFixesTest
       QRecord original = EmailStore.get(EmailTables.MESSAGE, first.messageId(), null);
       SystemWrites.updateOne(EmailTables.MESSAGE, new QRecord().withValue("id", second.messageId()).withValue("htmlBody", original.getValue("htmlBody")));
       var dispatcher = new com.kingsrook.qbits.email.delivery.EmailDispatcher(new com.kingsrook.qbits.email.transport.EmailTransportRegistry()
-         .withTransport("SMTP", (envelope, settings) -> { throw new AssertionError("Swapped ciphertext reached transport"); }));
+         .withTransport("SMTP", (envelope, settings) ->
+         {
+            throw new AssertionError("Swapped ciphertext reached transport");
+         }));
       assertThat(dispatcher.sendNow(second.messageId())).isEqualTo(com.kingsrook.qbits.email.delivery.EmailDispatcher.SendResult.RETRY_WAIT);
    }
 

src/test/java/com/kingsrook/qbits/email/EmailTemplatesTest.java

@@ -148,7 +148,10 @@ class EmailTemplatesTest
       assertThat(((RawHTML) preview.getWidgetData()).getHtml()).contains("<p>Email</p>");
       assertThat(EmailStore.get(EmailTables.VERSION, version.getValueInteger("id"), null).getValue("compiledHtml")).isNull();
       EmailVersions.activate(version.getValueInteger("id"));
-      EmailRuntime.config().withMjmlCompiler(source -> { throw new AssertionError("Compiler ran at send time"); });
+      EmailRuntime.config().withMjmlCompiler(source ->
+      {
+         throw new AssertionError("Compiler ran at send time");
+      });
       var result = EmailService.send(new EmailRequest().withTemplateName("mjml").withLocale("fr").withTo("p@example.com"));
       assertThat(result.status()).isEqualTo(EmailSendResult.Status.QUEUED);
       QRecord row = EmailStore.get(EmailTables.MESSAGE, result.messageId(), null);
@@ -295,7 +298,10 @@ class EmailTemplatesTest
       EmailRuntime.config().withSecretResolver(name -> null);
       assertThat(com.kingsrook.qbits.email.render.EmailUnsubscribeLinks.create("1", "address", "NOTIFICATION")).isNull();
       assertThatThrownBy(() -> com.kingsrook.qbits.email.render.EmailUnsubscribeLinks.create("1", "address", "MARKETING")).hasMessageContaining("QQQ_EMAIL_UNSUBSCRIBE_KEY");
-      EmailRuntime.config().withSecretResolver(name -> { throw new IllegalArgumentException("resolver detail"); });
+      EmailRuntime.config().withSecretResolver(name ->
+      {
+         throw new IllegalArgumentException("resolver detail");
+      });
       assertThatThrownBy(() -> com.kingsrook.qbits.email.render.EmailUnsubscribeLinks.create("1", "person@example.com", "NOTIFICATION"))
          .hasMessage("Opt-out email needs publicBaseUrl and QQQ_EMAIL_UNSUBSCRIBE_KEY");
    }

src/test/java/com/kingsrook/qbits/email/H2Support.java

@@ -85,7 +85,7 @@ public final class H2Support
     *******************************************************************************/
    public static QInstance instance(RDBMSBackendMetaData backend, Consumer<EmailQBitConfig> tweak)
          throws Exception
-         {
+   {
       QInstance instance = new QInstance();
       instance.withInstanceDefaultAuthentication(
             new QAuthenticationMetaData()

src/test/java/com/kingsrook/qbits/email/PhiCanaryEgressTest.java

@@ -81,7 +81,17 @@ class PhiCanaryEgressTest
    @BeforeEach
    void setup() throws Exception
    {
-      canary = "CANARY-PHI-" + UUID.randomUUID();
+      ////////////////////////////////////////////////////////////////////////
+      // the random part is mapped from hex to the letters g-v: a five-char //
+      // window of a hex canary could match any hex digest (body hashes,    //
+      // Message-IDs) by chance; these windows can only match real leakage  //
+      ////////////////////////////////////////////////////////////////////////
+      StringBuilder random = new StringBuilder();
+      for(char c : UUID.randomUUID().toString().replace("-", "").toCharArray())
+      {
+         random.append((char) ('g' + Character.digit(c, 16)));
+      }
+      canary = "CANARY-PHI-" + random;
       H2Support.setup(c -> c.withMaxAttempts(2).withSecretResolver(PhiCanaryEgressTest::secret)
          .withClassificationOverrides(Map.of("test_patient.name", EmailClassifications.Level.PHI)));
       H2Support.provider(25);

src/test/java/com/kingsrook/qbits/email/ProcessRunner.java

@@ -66,7 +66,7 @@ public final class ProcessRunner
    public static Lines runWithValues(
          String process, java.util.Map<String, Serializable> values, Serializable... ids)
          throws Exception
-         {
+   {
       RunProcessInput input = new RunProcessInput();
       input.setProcessName(process);
       input.addValue("recordsParam", "recordIds");

src/test/java/com/kingsrook/qbits/email/outbox/EmailEffectsTest.java

@@ -0,0 +1,212 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.email.outbox;
+
+
+import java.nio.charset.StandardCharsets;
+import java.security.MessageDigest;
+import java.util.HexFormat;
+import java.util.List;
+import com.kingsrook.qbits.email.H2Support;
+import com.kingsrook.qbits.email.metadata.EmailTables;
+import com.kingsrook.qbits.email.support.SystemWrites;
+import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.session.QSession;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+
+/*******************************************************************************
+ ** Durable handoff semantics and stored identity using the real H2 outbox.
+ *******************************************************************************/
+class EmailEffectsTest
+{
+   /*******************************************************************************
+    ** Prepare a provider without opening a transport connection.
+    *******************************************************************************/
+   @BeforeEach
+   void setup() throws Exception
+   {
+      H2Support.setup(config ->
+      {
+      });
+      H2Support.provider(25);
+   }
+
+
+
+   /*******************************************************************************
+    ** Release the per-thread context.
+    *******************************************************************************/
+   @AfterEach
+   void cleanup()
+   {
+      QContext.clear();
+   }
+
+
+
+   /*******************************************************************************
+    ** A plain message uses the configured sender.
+    *******************************************************************************/
+   private EmailRequest request()
+   {
+      return (new EmailRequest().withTo("person@example.com").withSubject("Effect").withTextBody("Body"));
+   }
+
+
+
+   /*******************************************************************************
+    ** Replays return the same persisted reference without another row.
+    *******************************************************************************/
+   @Test
+   void handoffDeduplicatesAndLookupsReturnCurrentReferences() throws Exception
+   {
+      String key = "job:2:row:email";
+      EmailRequest request = request();
+      var first = EmailEffects.send(request, key);
+      var second = EmailEffects.send(request(), key);
+      assertThat(request.getIdempotencyKey()).isEqualTo(key);
+      assertThat(first.outcome()).isEqualTo(EmailEffectOutcome.QUEUED_OR_DUPLICATE);
+      assertThat(second).isEqualTo(first);
+      assertThat(H2Support.all(EmailTables.MESSAGE)).hasSize(1);
+      String token = HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(key.getBytes(StandardCharsets.UTF_8))).substring(0, 32);
+      assertThat(first.message().rfcMessageId()).isEqualTo("<" + token + "@example.com>");
+      assertThat(first.message().status()).isEqualTo("QUEUED");
+      assertThat(first.message().providerMessageId()).isNull();
+      SystemWrites.updateOne(EmailTables.MESSAGE, new QRecord().withValue("id", first.message().id())
+         .withValue("status", "SENT").withValue("providerMessageId", "provider-reference"));
+      QContext.init(QContext.getQInstance(), new QSession().withPermissions(List.of()));
+      var found = EmailEffects.findByIdempotencyKey(key).orElseThrow();
+      assertThat(found.id()).isEqualTo(first.message().id());
+      assertThat(found.status()).isEqualTo("SENT");
+      assertThat(found.providerMessageId()).isEqualTo("provider-reference");
+      assertThat(EmailEffects.findByRfcMessageId(first.message().rfcMessageId())).contains(found);
+      assertThat(EmailEffects.findByIdempotencyKey("missing")).isEmpty();
+      assertThat(EmailEffects.findByRfcMessageId("<missing@example.com>")).isEmpty();
+      assertThat(EmailEffects.findByIdempotencyKey(null)).isEmpty();
+      assertThat(EmailEffects.findByRfcMessageId(null)).isEmpty();
+      assertThat(EmailEffects.findByIdempotencyKey(" ")).isEmpty();
+      assertThat(EmailEffects.findByRfcMessageId(" ")).isEmpty();
+   }
+
+
+
+   /*******************************************************************************
+    ** Conflicting or unusable requests never enqueue or change an existing key.
+    *******************************************************************************/
+   @Test
+   void refusesConflictingKeysAndInvalidRequests() throws Exception
+   {
+      EmailRequest conflict = request().withIdempotencyKey("other");
+      assertThat(EmailEffects.send(conflict, "effect").outcome()).isEqualTo(EmailEffectOutcome.REFUSED);
+      assertThat(conflict.getIdempotencyKey()).isEqualTo("other");
+      assertThat(EmailEffects.send(request(), null).outcome()).isEqualTo(EmailEffectOutcome.REFUSED);
+      assertThat(EmailEffects.send(request(), " ").outcome()).isEqualTo(EmailEffectOutcome.REFUSED);
+      assertThat(EmailEffects.send(request(), "a".repeat(501)).outcome()).isEqualTo(EmailEffectOutcome.REFUSED);
+      assertThat(EmailEffects.send(null, "effect").outcome()).isEqualTo(EmailEffectOutcome.REFUSED);
+      var invalid = EmailEffects.send(request().withSubject(null), "invalid");
+      assertThat(invalid.outcome()).isEqualTo(EmailEffectOutcome.REFUSED);
+      assertThat(invalid.message()).isNull();
+      assertThat(H2Support.all(EmailTables.MESSAGE)).isEmpty();
+      assertThat(EmailEffects.send(request().withIdempotencyKey("same"), "same").outcome()).isEqualTo(EmailEffectOutcome.QUEUED_OR_DUPLICATE);
+   }
+
+
+
+   /*******************************************************************************
+    ** A suppressed audit row stays refused on replay, including duplicate results.
+    *******************************************************************************/
+   @Test
+   void suppressedRecipientsAreRefusedOnEveryHandoff() throws Exception
+   {
+      H2Support.insert(EmailTables.SUPPRESSION, new QRecord().withValue("address", "person@example.com")
+         .withValue("scope", "ALL").withValue("reason", "HARD_BOUNCE"));
+      var first = EmailEffects.send(request(), "suppressed");
+      var second = EmailEffects.send(request(), "suppressed");
+      assertThat(first.outcome()).isEqualTo(EmailEffectOutcome.REFUSED);
+      assertThat(first.message().status()).isEqualTo("SUPPRESSED");
+      assertThat(second).isEqualTo(first);
+      assertThat(H2Support.all(EmailTables.MESSAGE)).hasSize(1);
+   }
+
+
+
+   /*******************************************************************************
+    ** Never acknowledge an uncommitted or deferred handoff to the jobs dispatcher.
+    *******************************************************************************/
+   @Test
+   void callerTransactionsAreRefused() throws Exception
+   {
+      try(QBackendTransaction transaction = QBackendTransaction.openFor(new InsertInput(EmailTables.MESSAGE)))
+      {
+         assertThat(EmailEffects.send(request().withTransaction(transaction, "rdbms"), "same-backend").outcome()).isEqualTo(EmailEffectOutcome.REFUSED);
+         assertThat(EmailEffects.send(request().withTransaction(transaction, "other"), "other-backend").outcome()).isEqualTo(EmailEffectOutcome.REFUSED);
+         transaction.commit();
+      }
+      assertThat(H2Support.all(EmailTables.MESSAGE)).isEmpty();
+   }
+
+
+
+   /*******************************************************************************
+    ** Database uniqueness failures must escape, rather than become terminal refusals.
+    *******************************************************************************/
+   @Test
+   void storageFailurePropagatesAndRfcIdIsUnique() throws Exception
+   {
+      var first = EmailEffects.send(request(), "collision");
+      SystemWrites.updateOne(EmailTables.MESSAGE, new QRecord().withValue("id", first.message().id()).withValue("idempotencyKey", "changed"));
+      assertThatThrownBy(() -> EmailEffects.send(request(), "collision")).isInstanceOf(QException.class);
+      assertThat(H2Support.all(EmailTables.MESSAGE)).hasSize(1);
+      assertThat(EmailEffects.findByIdempotencyKey("collision")).isEmpty();
+   }
+
+
+
+   /*******************************************************************************
+    ** Identity hashing is deterministic, ignores display names and has a safe fallback.
+    *******************************************************************************/
+   @Test
+   void generationAndUnkeyedMessages() throws Exception
+   {
+      assertThat(EmailService.messageId("key", "Display <sender@EXAMPLE.com>"))
+         .isEqualTo(EmailService.messageId("key", "sender@example.com"));
+      for(String from : new String[] {null, "local", "broken <", "sender@[]"})
+      {
+         assertThat(EmailService.messageId("key", from)).matches("<[0-9a-f]{32}@qbit-email\\.local>");
+      }
+      var first = EmailService.send(request());
+      var second = EmailService.send(request());
+      var rows = H2Support.all(EmailTables.MESSAGE);
+      assertThat(rows).hasSize(2);
+      assertThat(rows.get(0).getValueString("rfcMessageId")).matches("<[0-9a-f]{32}@example\\.com>")
+         .isNotEqualTo(rows.get(1).getValueString("rfcMessageId"));
+      assertThat(first.messageId()).isNotEqualTo(second.messageId());
+   }
+}

src/test/java/com/kingsrook/qbits/email/tracking/EmailEventParserTest.java

@@ -26,8 +26,12 @@ import org.json.JSONObject;
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.params.ParameterizedTest;
 import org.junit.jupiter.params.provider.CsvSource;
-import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.*;
-import static org.assertj.core.api.Assertions.*;
+import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.NOW;
+import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.message;
+import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.provider;
+import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.resource;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
 
 /*******************************************************************************
  ** Provider-specific normalization contracts use realistic fixture envelopes.

src/test/java/com/kingsrook/qbits/email/tracking/EmailInboxProcessorTest.java

@@ -47,8 +47,12 @@ import org.junit.jupiter.api.Test;
 import org.junit.jupiter.params.ParameterizedTest;
 import org.junit.jupiter.params.provider.CsvSource;
 import org.junit.jupiter.params.provider.ValueSource;
-import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.*;
-import static org.assertj.core.api.Assertions.*;
+import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.NOW;
+import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.SECRET;
+import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.message;
+import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.provider;
+import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.resource;
+import static org.assertj.core.api.Assertions.assertThat;
 
 /*******************************************************************************
  ** Durable processing, rollback, suppression and race acceptance on real H2.
@@ -422,6 +426,7 @@ class EmailInboxProcessorTest
       /*******************************************************************************
        ** Test adapter implementation.
        *******************************************************************************/
+
       @Override
       public void raise(com.kingsrook.qqq.backend.core.alerts.AlertSignal signal)
       {

src/test/java/com/kingsrook/qbits/email/tracking/EmailWebhookRoutesTest.java

@@ -32,7 +32,6 @@ import java.time.ZoneOffset;
 import java.util.ArrayList;
 import java.util.Base64;
 import java.util.List;
-import java.util.Map;
 import java.util.concurrent.Executors;
 import java.util.concurrent.TimeUnit;
 import com.kingsrook.qbits.email.EmailRuntime;
@@ -52,8 +51,18 @@ import org.junit.jupiter.api.BeforeEach;
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.params.ParameterizedTest;
 import org.junit.jupiter.params.provider.ValueSource;
-import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.*;
-import static org.assertj.core.api.Assertions.*;
+import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.NOW;
+import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.SECRET;
+import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.TOPIC;
+import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.certificateTime;
+import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.message;
+import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.provider;
+import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.resource;
+import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.response;
+import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.signMailgun;
+import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.signSns;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
 
 /*******************************************************************************
  ** Real random-port HTTP exercises the exact route provider and database boundary.

src/test/java/com/kingsrook/qbits/email/tracking/TrackingTestSupport.java

@@ -20,6 +20,9 @@
 
 package com.kingsrook.qbits.email.tracking;
 
+import javax.crypto.Mac;
+import javax.crypto.spec.SecretKeySpec;
+import javax.net.ssl.SSLSession;
 import java.net.URI;
 import java.net.http.HttpHeaders;
 import java.net.http.HttpRequest;
@@ -36,9 +39,6 @@ import java.util.Base64;
 import java.util.List;
 import java.util.Map;
 import java.util.Optional;
-import javax.crypto.Mac;
-import javax.crypto.spec.SecretKeySpec;
-import javax.net.ssl.SSLSession;
 import com.kingsrook.qbits.email.H2Support;
 import com.kingsrook.qbits.email.metadata.EmailTables;
 import com.kingsrook.qbits.email.support.SystemWrites;

src/test/java/com/kingsrook/qbits/email/tracking/WebhookVerifiersTest.java

@@ -31,8 +31,16 @@ import org.json.JSONObject;
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.params.ParameterizedTest;
 import org.junit.jupiter.params.provider.ValueSource;
-import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.*;
-import static org.assertj.core.api.Assertions.*;
+import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.NOW;
+import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.SECRET;
+import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.TOPIC;
+import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.certificateTime;
+import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.resource;
+import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.response;
+import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.signMailgun;
+import static com.kingsrook.qbits.email.tracking.TrackingTestSupport.signSns;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
 
 /*******************************************************************************
  ** Crypto boundaries use genuine signatures and a public test-only RSA fixture.

src/test/java/com/kingsrook/qbits/email/transport/EmailHttpTransportsTest.java

@@ -21,6 +21,7 @@
 package com.kingsrook.qbits.email.transport;
 
 
+import javax.net.ssl.SSLSession;
 import java.io.ByteArrayInputStream;
 import java.io.IOException;
 import java.net.InetSocketAddress;
@@ -44,7 +45,6 @@ import java.util.Properties;
 import java.util.Set;
 import java.util.concurrent.atomic.AtomicInteger;
 import java.util.concurrent.atomic.AtomicReference;
-import javax.net.ssl.SSLSession;
 import com.fasterxml.jackson.databind.JsonNode;
 import com.kingsrook.qbits.email.EmailQBitConfig;
 import com.kingsrook.qbits.email.EmailRuntime;
@@ -147,9 +147,9 @@ class EmailHttpTransportsTest
     *******************************************************************************/
    private static EmailEnvelope envelope()
    {
-      return (new EmailEnvelope().withMessageId("42").withFrom("Sender <sender@example.com>")
+      return (new EmailEnvelope().withMessageId("42").withRfcMessageId("<0123456789abcdef0123456789abcdef@example.com>").withFrom("Sender <sender@example.com>")
          .withTo(List.of("Person <to@example.com>")).withCc(List.of("cc@example.com")).withBcc(List.of("bcc@example.com"))
-         .withReplyTo("Replies <reply@example.com>").withSubject("Hello \u00e9").withTextBody("Plain \u00e9").withHtmlBody("<b>Hello \u00e9</b>")
+         .withReplyTo("Replies <reply@example.com>").withSubject("Hello é").withTextBody("Plain é").withHtmlBody("<b>Hello é</b>")
          .withHeaders(Map.of("List-Unsubscribe", "<https://example.com/unsubscribe>",
             "List-Unsubscribe-Post", "List-Unsubscribe=One-Click", "X-Custom", "kept")));
    }
@@ -283,6 +283,11 @@ class EmailHttpTransportsTest
       assertThat(json.path("MessageStream").asText()).isEqualTo(tracking ? "broadcasts" : "outbound");
       assertThat(json.path("TrackOpens").asBoolean()).isEqualTo(tracking);
       assertThat(json.path("TrackLinks").asText()).isEqualTo(tracking ? "HtmlAndText" : "None");
+      assertThat(json.path("Headers")).anySatisfy(header ->
+      {
+         assertThat(header.path("Name").asText()).isEqualTo("Message-ID");
+         assertThat(header.path("Value").asText()).isEqualTo(envelope().getRfcMessageId());
+      });
       assertThat(json.path("Headers").toString()).contains("List-Unsubscribe", "List-Unsubscribe-Post", "X-QQQ-Email-Id", "42", "Auto-Submitted");
       assertThat(json.toString()).doesNotContain(SECRET);
    }
@@ -303,6 +308,7 @@ class EmailHttpTransportsTest
          return (response(202, "", Map.of("x-message-id", List.of("sg-id"))));
       });
       Map<String, String> headers = new java.util.HashMap<>(envelope().getHeaders());
+      headers.put("message-id", "<override@example.com>");
       for(String reserved : List.of("X-SG-ID", "x-sg-eid", "Received", "DKIM-Signature", "Content-Type",
          "Content-Transfer-Encoding", "To", "From", "Subject", "Reply-To", "CC", "BcC"))
       {
@@ -329,6 +335,8 @@ class EmailHttpTransportsTest
       assertThat(json.at("/content/1/type").asText()).isEqualTo("text/html");
       assertThat(json.at("/content/1/value").asText()).isEqualTo(envelope().getHtmlBody());
       assertThat(json.at("/custom_args/qqq_email_id").asText()).isEqualTo("42");
+      assertThat(json.at("/headers/Message-ID").asText()).isEqualTo(envelope().getRfcMessageId());
+      assertThat(json.path("headers").has("message-id")).isFalse();
       assertThat(json.path("headers").toString()).doesNotContain("must-be-removed").contains("List-Unsubscribe", "List-Unsubscribe-Post");
       assertThat(json.at("/tracking_settings/open_tracking/enable").asBoolean()).isEqualTo(tracking);
       assertThat(json.at("/tracking_settings/click_tracking/enable").asBoolean()).isEqualTo(tracking);
@@ -371,6 +379,8 @@ class EmailHttpTransportsTest
       assertThat(mime).doesNotContain("bcc@example.com", "Bcc:", SECRET);
       MimeMessage message = new MimeMessage(Session.getInstance(new Properties()), new ByteArrayInputStream(mime.getBytes(StandardCharsets.UTF_8)));
       assertThat(message.getSubject()).isEqualTo(envelope().getSubject());
+      assertThat(message.getMessageID()).isEqualTo(envelope().getRfcMessageId());
+      assertThat(multipart).contains("name=\"h:Message-Id\"\r\n\r\n" + envelope().getRfcMessageId() + "\r\n");
       assertThat(message.getHeader("X-QQQ-Email-Id", null)).isEqualTo("42");
       assertThat(message.getHeader("List-Unsubscribe-Post", null)).isEqualTo("List-Unsubscribe=One-Click");
       assertThat(message.getRecipients(jakarta.mail.Message.RecipientType.CC)[0].toString()).isEqualTo("cc@example.com");

src/test/java/com/kingsrook/qbits/email/transport/EmailTransportsTest.java

@@ -47,8 +47,15 @@ import jakarta.mail.internet.InternetAddress;
 import jakarta.mail.internet.MimeMessage;
 import org.junit.jupiter.api.Test;
 import software.amazon.awssdk.services.sesv2.SesV2Client;
-import software.amazon.awssdk.services.sesv2.model.*;
-import static org.assertj.core.api.Assertions.*;
+import software.amazon.awssdk.services.sesv2.model.Message;
+import software.amazon.awssdk.services.sesv2.model.MessageRejectedException;
+import software.amazon.awssdk.services.sesv2.model.SendEmailRequest;
+import software.amazon.awssdk.services.sesv2.model.SendEmailResponse;
+import software.amazon.awssdk.services.sesv2.model.SesV2Exception;
+import software.amazon.awssdk.services.sesv2.model.TooManyRequestsException;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+import static org.assertj.core.api.Assertions.fail;
 
 /*******************************************************************************
  ** Email Transports Test.
@@ -61,7 +68,7 @@ class EmailTransportsTest
    EmailEnvelope envelope()
    {
       return (new EmailEnvelope()
-            .withMessageId("123")
+            .withMessageId("123").withRfcMessageId("<0123456789abcdef0123456789abcdef@example.com>")
             .withFrom("sender@example.com")
             .withTo(List.of("a@example.com"))
             .withBcc(List.of("hidden@example.com"))
@@ -135,10 +142,26 @@ class EmailTransportsTest
                   Session.getInstance(new Properties()),
                   new ByteArrayInputStream(captured.get().content().raw().data().asByteArray()));
       assertThat(mime.getHeader("Bcc")).isNull();
+      assertThat(mime.getMessageID()).isEqualTo(envelope().getRfcMessageId());
+      assertThat(mime.getHeader("X-QQQ-Email-Id")).containsExactly("123");
       assertThat(mime.getHeader("X-Test")).containsExactly("yes");
       assertThat(mime.getHeader("Auto-Submitted")).containsExactly("auto-generated");
    }
 
+   /*******************************************************************************
+    ** Repeated JavaMail saves cannot replace the identity or accept a custom override.
+    *******************************************************************************/
+   @Test
+   void mimeIdentitySurvivesSaveChanges() throws Exception
+   {
+      var envelope = envelope().withHeaders(Map.of("message-id", "<override@example.com>"));
+      var message = EmailMime.build(Session.getInstance(new Properties()), envelope);
+      message.saveChanges();
+      message.saveChanges();
+      assertThat(message.getHeader("Message-ID")).containsExactly(envelope.getRfcMessageId());
+      assertThat(EmailMime.build(Session.getInstance(new Properties()), envelope().withRfcMessageId(null)).getMessageID()).isNotBlank();
+   }
+
    /*******************************************************************************
     ** Ses Classifies Throttling Rejection And Server Failure.
     *******************************************************************************/
@@ -150,7 +173,7 @@ class EmailTransportsTest
                   TooManyRequestsException.builder().statusCode(429).message("secret").build(),
                   MessageRejectedException.builder().statusCode(400).message("secret").build(),
                   SesV2Exception.builder().statusCode(503).message("secret").build()))
-                  {
+      {
          var transport =
                new SesEmailTransport(
                      builder ->
@@ -313,7 +336,7 @@ class EmailTransportsTest
       for(String newline : List.of("\r", "\n"))
       {
          String bad = "a@example.com" + newline + "Bcc: hidden@example.com";
-         for(EmailEnvelope envelope : List.of(envelope().withSubject(bad), envelope().withFrom(bad), envelope().withReplyTo(bad),
+         for(EmailEnvelope envelope : List.of(envelope().withRfcMessageId(bad), envelope().withSubject(bad), envelope().withFrom(bad), envelope().withReplyTo(bad),
             envelope().withTo(List.of(bad)), envelope().withCc(List.of(bad)), envelope().withBcc(List.of(bad)),
             envelope().withHeaders(Map.of("X-Test", bad)), envelope().withFrom("broken <")))
          {

src/test/resources/email-h2.sql

@@ -75,6 +75,7 @@ CREATE TABLE email_message (
    phi_approved BOOLEAN,
    unsubscribe_url TEXT,
    idempotency_key VARCHAR(500),
+   rfc_message_id VARCHAR(500),
    provider_id INTEGER,
    provider_message_id VARCHAR(500),
    attempt_count INTEGER,
@@ -93,7 +94,8 @@ CREATE TABLE email_message (
    sender_process VARCHAR(500),
    suppressed_addresses TEXT,
    send_after TIMESTAMP,
-   UNIQUE (idempotency_key)
+   UNIQUE (idempotency_key),
+   UNIQUE (rfc_message_id)
 );
 
 CREATE TABLE email_message_attempt (