Merge remote-tracking branch 'origin/develop' into feature/platform-reuse

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

README.md

@@ -1,6 +1,6 @@
 # QBit CRM
 
-**0.2.0-SNAPSHOT — unreleased, P1 Platform.** CRM tables, processes and widgets for QQQ applications. This phase upgrades the build, names, permissions, audit integration and database schema. It does not complete the later 0.2.0 product phases.
+**0.2.0-SNAPSHOT — unreleased, P1 Platform + P2 App, home and nav.** CRM tables, processes and widgets for QQQ applications. This work upgrades the build, names, permissions, audit integration and database schema, and supplies the CRM app and record navigation. It does not complete the later 0.2.0 product phases.
 
 ## Build and verification
 
@@ -50,7 +50,7 @@ new CrmQBitProducer()
 qInstance.setEnforcePermissionsInCoreActions(true);
 ```
 
-4. Map the host's authenticated roles to the permissions below. Configure the host's core audit storage. Add tables, processes and widgets to the host UI as needed: `produceAppSection()` currently returns an empty CRM section; P1 does not provide an assembled app or home dashboard.
+4. Map the host's authenticated roles to the permissions below. Configure the host's core audit storage. The `crm` app is enabled by default, with main and Settings navigation and a real-data home. Set `withAppEnabled(false)` to embed the tables in the host's own app. `produceAppSection()` returns the same populated main section; processes are record/table actions, never navigation entries.
 
 ## Configuration and access
 
@@ -68,7 +68,7 @@ qInstance.setEnforcePermissionsInCoreActions(true);
 | Merge, bulk assignment, list and sequence management processes | `crm.manage.hasAccess` |
 | GDPR anonymization | `crm.admin.hasAccess` |
 
-Permissions are independent grants. Every CRM-produced table, process and widget has explicit `QPermissionRules` with `DenyBehavior.HIDDEN` and QBit provenance. `security.CrmPermissions.ALL` lists the grants for host administration and tests. The widget provenance adapter supplies core's `SourceQBitAware` interface because RC.6's widget class lacks it.
+Permissions are independent grants. Every CRM-produced table, process and widget has explicit `QPermissionRules` with `DenyBehavior.HIDDEN` and QBit provenance. `security.CrmPermissions.ALL` lists the grants for host administration and tests. The app and widget provenance adapters supply core's `SourceQBitAware` interface because RC.6's app and widget classes lack it.
 
 ## Tables
 
@@ -90,52 +90,76 @@ Contact, company, deal and activity declare core FIELD-level audit rules. The be
 
 ## Processes
 
-The QBit produces 25 processes with these names and labels. QQQ may additionally generate standard table actions such as `<table>.bulkInsert`; those retain core's naming conventions. CRM processes are available to host wiring and are not registered as scheduled jobs.
+The QBit produces 28 table-bound record actions. Every action uses the standard
+streamed ETL Review and Result screens with linked summary lines. Required inputs
+are collected before Review. Processes never appear in app navigation. Find
+Duplicates scans the selected records (select all matching records to scan the
+table); each duplicate group links to its members. Merges require exactly two
+selected records and a choice of which record to keep.
 
-| Process name | Label |
-|---|---|
-| `crm_auto_assign` | Auto Assign |
-| `crm_bulk_assign_owner` | Bulk Assign Owner |
-| `crm_calculate_lead_scores` | Calculate Lead Scores |
-| `crm_clone_deal` | Clone Deal |
-| `crm_convert_form_to_contact` | Convert Form to Contact |
-| `crm_convert_lead` | Convert Lead |
-| `crm_detect_duplicates` | Detect Duplicates |
-| `crm_enroll_in_sequence` | Enroll in Sequence |
-| `crm_fire_reminders` | Fire Reminders |
-| `crm_gdpr_anonymize_contact` | GDPR Anonymize Contact |
-| `crm_import_contacts` | Import Contacts |
-| `crm_log_activity` | Log Activity |
-| `crm_manage_contact_on_deal` | Manage Contact on Deal |
-| `crm_merge_companies` | Merge Companies |
-| `crm_merge_contacts` | Merge Contacts |
-| `crm_move_deal_stage` | Move Deal Stage |
-| `crm_process_overdue_tasks` | Process Overdue Tasks |
-| `crm_process_sequence_steps` | Process Sequence Steps |
-| `crm_reopen_deal` | Reopen Deal |
-| `crm_send_email` | Send Email |
-| `crm_subscribe_to_list` | Subscribe to List |
-| `crm_track_email_event` | Track Email Event |
-| `crm_transfer_ownership` | Transfer Ownership |
-| `crm_unenroll_from_sequence` | Unenroll from Sequence |
-| `crm_unsubscribe_from_list` | Unsubscribe from List |
+| Table | Process name | Label |
+|---|---|---|
+| `crm_contact` | `crm_bulk_assign_owner` | Assign Owner |
+| `crm_contact` | `crm_auto_assign` | Auto-Assign |
+| `crm_contact` | `crm_subscribe_to_list` | Add to List |
+| `crm_contact` | `crm_unsubscribe_from_list` | Remove from List |
+| `crm_contact` | `crm_enroll_in_sequence` | Enroll in Sequence |
+| `crm_contact` | `crm_unenroll_from_sequence` | Unenroll from Sequence |
+| `crm_contact` | `crm_log_activity` | Log Activity |
+| `crm_contact` | `crm_recalculate_score` | Recalculate Score |
+| `crm_contact` | `crm_merge_contacts` | Merge Contacts |
+| `crm_contact` | `crm_detect_duplicates` | Find Duplicates |
+| `crm_contact` | `crm_convert_lead` | Create Deal |
+| `crm_contact` | `crm_gdpr_anonymize_contact` | Erase Personal Data |
+| `crm_contact` | `crm_send_email` | Send Email |
+| `crm_company` | `crm_assign_company_owner` | Assign Owner |
+| `crm_company` | `crm_merge_companies` | Merge Companies |
+| `crm_company` | `crm_find_duplicate_companies` | Find Duplicates |
+| `crm_company` | `crm_log_company_activity` | Log Activity |
+| `crm_deal` | `crm_move_deal_stage` | Move Stage |
+| `crm_deal` | `crm_mark_deal_won` | Mark Won |
+| `crm_deal` | `crm_mark_deal_lost` | Mark Lost |
+| `crm_deal` | `crm_reopen_deal` | Reopen |
+| `crm_deal` | `crm_clone_deal` | Clone |
+| `crm_deal` | `crm_transfer_ownership` | Assign Owner |
+| `crm_deal` | `crm_manage_contact_on_deal` | Add Contact |
+| `crm_deal` | `crm_log_deal_activity` | Log Activity |
+| `crm_form_submission` | `crm_convert_form_to_contact` | Convert to Contact |
+| `crm_email_sequence` | `crm_activate_sequence` | Activate |
+| `crm_email_sequence` | `crm_pause_sequence` | Pause |
+
+Move Stage and Reopen choose a stage from the selected deals' pipeline. Select
+one pipeline at a time for those actions. Mark Won and Mark Lost require a
+win/loss reason and resolve the terminal stage within each deal's pipeline.
+
+Two hidden background processes have no table binding: `crm_process_sequence_steps`
+and `crm_calculate_lead_scores`. Both use id keyset pages of 200. They have no
+shipped schedules yet; the host can invoke them. QQQ also generates standard
+bulk-load and bulk-edit actions such as `<table>.bulkInsert`.
+
+Fire Reminders and Process Overdue Tasks were removed in favor of collaboration
+tasks and alerts. Import Contacts was removed in favor of core bulk load. Track
+Email Event was removed in favor of the later delivery sync.
 
 ## Current behavior and limits
 
-The existing pipeline/stage logic, stage history, product totals, currency conversion, contact/company merge, activity logging, form auto-linking, list subscribe/unsubscribe, scoring and assignment processes remain. Fourteen custom dashboard widget producers remain available for host placement, together with generated child-record widgets.
+The existing pipeline/stage logic, stage history, product totals, currency conversion, contact/company merge, activity logging, form auto-linking, list subscribe/unsubscribe, scoring and assignment processes remain. The CRM home contains Today (five standard statistics), Pipeline by stage, Forecast (current month plus five), Needs attention (ten oldest rotting deals), Recent web leads (latest ten), Goals (up to fifty current goals, hidden when none are visible), and Leaderboard (quarter-to-date won revenue by owner). Reporting boundaries use UTC. Monetary summaries use base currency, with weighted amounts grouped by currency before conversion. Aggregate rows link to their underlying records; stage name and deal count share a column to keep tables to three columns. The pipeline picker appears when multiple active pipelines are visible.
 
-Some process names describe unfinished behavior inherited from 0.1.0:
+The contact Timeline shows the latest fifty recorded events from activities, email messages, form submissions and consent, newest first by creation date. Native child lists connect all specified parent screens. Deal primary-contact and email-message contact references are included in the fresh-install schema. The trend widgets, redundant recent-deals/activity-count widgets and activity feed have been retired.
 
-- Send Email and sequence email steps record CRM activity/message state; they do not deliver email. Template merging and live delivery tracking are not implemented here.
-- Sequence execution, lead scoring and overdue-task work require host invocation. No schedules are shipped. Assignment is a process, not an automatic insert hook; reminders do not deliver notifications.
-- Scoring's date-decay operators remain unimplemented. Import Contacts expects pre-parsed input and does not provide the planned streamed bulk-load workflow.
+Later phases complete the following behavior:
+
+- Send Email is a non-sending action shell. It checks for `CrmEmailTransport` with `Class.forName` and reports **Email sending arrives with the email transport** when absent. It never writes a fake sent message or activity. If the class is present, the shell reports that action integration is still pending. Sequence email steps retain the existing activity-only behavior until transport integration.
+- Sequence execution and lead scoring require host invocation. No schedules are shipped. Assignment remains an action; automatic insert hooks, collaboration and alert integration arrive later.
+- Activate/Pause update the sequence active flag. The existing sequence runner's business logic is preserved in P2B and does not yet consult that flag; enforcing it during execution is a follow-up.
+- Scoring's date-decay operators remain unimplemented. Core bulk load is the import path; saved CRM import profiles arrive later.
 - GDPR anonymization scrubs the contact, unenrolls sequences and records withdrawn consent. It does not yet erase all linked PII or core audit details. The P1 schema retains existing PII fields; classification, complete erasure and redesigned tenancy are later work.
 
-The later intake APIs, public subscription routes, collaboration integration, real email transport, durable jobs and full app navigation are not part of this phase.
+The later intake APIs, public subscription routes, collaboration integration, real email transport, durable jobs and the remaining P2 security/integration work are not part of this phase.
 
 ## Source layout
 
-`CrmQBitProducer` discovers entity, process and widget producers. `core/` holds contacts, companies, forms, lists and consent; `deals/`, `activities/`, `email/` and `scoring/` hold their respective metadata and behavior. `security/` defines permissions, `sync/` holds external mappings and currency utilities, and `widgets/` contains widget producers/renderers. SQL is shipped under `src/main/resources/db/liquibase/qbit-crm/`.
+`CrmQBitProducer` discovers entity, process and widget producers. `core/` holds contacts, companies, forms, lists and consent; `deals/`, `activities/`, `email/` and `scoring/` hold their respective metadata and behavior. `actions/` defines the streamed record actions and their inputs/summaries. `security/` defines permissions, `sync/` holds external mappings and currency utilities, and `widgets/` contains widget producers/renderers. SQL is shipped under `src/main/resources/db/liquibase/qbit-crm/`.
 
 Licensed under [Apache License 2.0](LICENSE).
 

docs/ddl/postgres.sql

@@ -1,6 +1,6 @@
 -- Copyright 2026 Kingsrook, LLC. Licensed under the Apache License, Version 2.0.
 -- https://www.apache.org/licenses/LICENSE-2.0
--- QBit CRM 0.2.0 P1 PostgreSQL reference schema (33 tables).
+-- QBit CRM 0.2.0 P1 PostgreSQL reference schema (34 tables).
 -- QQQ table names equal SQL names; Java field names map to snake_case columns.
 -- Unique constraints match QQQ metadata. Relationship columns are indexed;
 -- foreign-key constraints are deliberately not imposed (the QQQ/host layer owns
@@ -189,7 +189,7 @@ CREATE TABLE IF NOT EXISTS crm_contact (
    company_id INTEGER,
    lifecycle_stage_id INTEGER,
    lead_source_id INTEGER,
-   owner_user_id TEXT NOT NULL,
+   owner_user_id TEXT,
    address_line1 VARCHAR(250),
    address_line2 VARCHAR(250),
    city VARCHAR(100),
@@ -251,7 +251,8 @@ CREATE TABLE IF NOT EXISTS crm_deal (
    probability_override_pct INTEGER,
    weighted_amount DECIMAL(20,6),
    company_id INTEGER,
-   owner_user_id TEXT NOT NULL,
+   primary_contact_id INTEGER,
+   owner_user_id TEXT,
    lead_source_id INTEGER,
    expected_close_date DATE,
    actual_close_date DATE,
@@ -266,6 +267,7 @@ CREATE TABLE IF NOT EXISTS crm_deal (
 
 CREATE INDEX IF NOT EXISTS ix_crm_deal_pipeline_id ON crm_deal (pipeline_id);
 CREATE INDEX IF NOT EXISTS ix_crm_deal_pipeline_stage_id ON crm_deal (pipeline_stage_id);
+CREATE INDEX IF NOT EXISTS ix_crm_deal_primary_contact_id ON crm_deal (primary_contact_id);
 CREATE INDEX IF NOT EXISTS ix_crm_deal_company_id ON crm_deal (company_id);
 CREATE INDEX IF NOT EXISTS ix_crm_deal_owner_user_id ON crm_deal (owner_user_id);
 CREATE INDEX IF NOT EXISTS ix_crm_deal_lead_source_id ON crm_deal (lead_source_id);
@@ -362,8 +364,13 @@ CREATE INDEX IF NOT EXISTS ix_crm_email_list_member_form_submission_id ON crm_em
 
 CREATE TABLE IF NOT EXISTS crm_email_message (
    id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
+   idempotency_key VARCHAR(250) UNIQUE,
+   email_message_id VARCHAR(100),
+   delivered_date TIMESTAMP WITHOUT TIME ZONE,
+   bounced_date TIMESTAMP WITHOUT TIME ZONE,
    activity_id INTEGER,
-   from_address VARCHAR(250) NOT NULL,
+   contact_id INTEGER,
+   from_address VARCHAR(250),
    from_name VARCHAR(100),
    to_addresses TEXT NOT NULL,
    cc_addresses TEXT,
@@ -389,6 +396,8 @@ CREATE TABLE IF NOT EXISTS crm_email_message (
    modify_date TIMESTAMP WITHOUT TIME ZONE
 );
 
+CREATE INDEX IF NOT EXISTS ix_crm_email_message_email_message_id ON crm_email_message (email_message_id);
+CREATE INDEX IF NOT EXISTS ix_crm_email_message_contact_id ON crm_email_message (contact_id);
 CREATE INDEX IF NOT EXISTS ix_crm_email_message_activity_id ON crm_email_message (activity_id);
 CREATE INDEX IF NOT EXISTS ix_crm_email_message_thread_id ON crm_email_message (thread_id);
 CREATE INDEX IF NOT EXISTS ix_crm_email_message_email_template_id ON crm_email_message (email_template_id);
@@ -636,3 +645,10 @@ CREATE TABLE IF NOT EXISTS crm_win_loss_reason (
 );
 
 CREATE INDEX IF NOT EXISTS ix_crm_consent_record_evidence_form_submission_id ON crm_consent_record (evidence_form_submission_id);
+
+CREATE TABLE IF NOT EXISTS crm_job_state (
+   id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
+   name VARCHAR(250) NOT NULL UNIQUE,
+   last_success TIMESTAMP WITHOUT TIME ZONE,
+   last_full_pass TIMESTAMP WITHOUT TIME ZONE
+);

src/main/java/com/kingsrook/qbits/crm/CrmAppMetaData.java

@@ -0,0 +1,117 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2026.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+package com.kingsrook.qbits.crm;
+
+
+import java.util.ArrayList;
+import java.util.List;
+import com.kingsrook.qbits.crm.security.CrmPermissions;
+import com.kingsrook.qbits.crm.widgets.CrmHomeWidgets;
+import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerMultiOutput;
+import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppSection;
+import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
+import com.kingsrook.qqq.backend.core.model.metadata.qbits.SourceQBitAware;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+
+
+/*******************************************************************************
+ ** The CRM home and the two navigation sections. Child tables stay on records.
+ *******************************************************************************/
+public final class CrmAppMetaData extends QAppMetaData implements SourceQBitAware
+{
+   private String sourceQBitName;
+
+   public static final List<String> MAIN_TABLES = List.of("crm_contact", "crm_company", "crm_deal", "crm_activity", "crm_form_submission", "crm_email_list");
+   public static final List<String> SETTINGS_TABLES = List.of("crm_pipeline", "crm_product", "crm_email_template", "crm_email_sequence",
+      "crm_lead_score_rule", "crm_assignment_rule", "crm_sales_goal", "crm_currency", "crm_lifecycle_stage", "crm_lead_source",
+      "crm_industry", "crm_activity_type", "crm_activity_outcome", "crm_contact_role", "crm_win_loss_reason");
+
+
+   /*******************************************************************************
+    ** Metadata factory only.
+    *******************************************************************************/
+   private CrmAppMetaData()
+   {
+   }
+
+
+   /*******************************************************************************
+    ** Main navigation, also available to hosts embedding these tables.
+    *******************************************************************************/
+   public static QAppSection mainSection()
+   {
+      return (new QAppSection().withName("crm").withLabel("CRM").withIcon(new QIcon("contacts"))
+         .withTables(new ArrayList<>(MAIN_TABLES)));
+   }
+
+
+   /*******************************************************************************
+    ** Construct the app after package discovery has produced the tables.
+    *******************************************************************************/
+   public static QAppMetaData build(MetaDataProducerMultiOutput output, String source)
+   {
+      QAppMetaData app = new CrmAppMetaData().withSourceQBitName(source).withName("crm").withLabel("CRM").withIcon(new QIcon("contacts"))
+         .withPermissionRules(CrmPermissions.hasAccessRules(CrmPermissions.BASE))
+         .withWidgets(new ArrayList<>(CrmHomeWidgets.HOME))
+         .withSection(mainSection())
+         .withSection(new QAppSection().withName("crm_settings").withLabel("Settings").withIcon(new QIcon("settings"))
+            .withTables(new ArrayList<>(SETTINGS_TABLES)));
+      List<String> tables = new ArrayList<>(MAIN_TABLES);
+      tables.addAll(SETTINGS_TABLES);
+      for(String name : tables)
+      {
+         app.withChild(output.get(QTableMetaData.class, name));
+      }
+      return (app);
+   }
+
+   /*******************************************************************************
+    ** QBit provenance, until core apps implement SourceQBitAware.
+    *******************************************************************************/
+   @Override
+   public String getSourceQBitName()
+   {
+      return (sourceQBitName);
+   }
+
+
+   /*******************************************************************************
+    ** QBit provenance, until core apps implement SourceQBitAware.
+    *******************************************************************************/
+   @Override
+   public void setSourceQBitName(String sourceQBitName)
+   {
+      this.sourceQBitName = sourceQBitName;
+   }
+
+
+   /*******************************************************************************
+    ** QBit provenance, until core apps implement SourceQBitAware.
+    *******************************************************************************/
+   @Override
+   public CrmAppMetaData withSourceQBitName(String sourceQBitName)
+   {
+      setSourceQBitName(sourceQBitName);
+      return (this);
+   }
+}

src/main/java/com/kingsrook/qbits/crm/CrmQBitConfig.java

@@ -43,6 +43,9 @@ public class CrmQBitConfig implements QBitConfig
 {
    public static final CrmQBitConfig DEFAULT_CONFIG = new CrmQBitConfig();
 
+   private String schedulerName;
+   private Boolean schedulesEnabled = true;
+
    private String messagingProviderName;
 
    private String publicBaseUrl;
@@ -51,6 +54,7 @@ public class CrmQBitConfig implements QBitConfig
 
    private String                                      defaultBackendNameForTables;
    private Boolean                                     inferBackendNames = true;
+   private Boolean                                     appEnabled = true;
    private MetaDataCustomizerInterface<QTableMetaData>  tableMetaDataCustomizer;
    private List<QFieldMetaData>                         securityFields;
    private List<RecordSecurityLock>                     recordSecurityLocks;
@@ -195,6 +199,24 @@ public class CrmQBitConfig implements QBitConfig
    }
 
 
+   /*******************************************************************************
+    ** Whether to publish the standalone CRM app.
+    *******************************************************************************/
+   public Boolean getAppEnabled()
+   {
+      return (appEnabled);
+   }
+
+
+   /*******************************************************************************
+    ** Allow a host to put CRM tables in its own app.
+    *******************************************************************************/
+   public CrmQBitConfig withAppEnabled(Boolean appEnabled)
+   {
+      this.appEnabled = appEnabled;
+      return (this);
+   }
+
 
    /*******************************************************************************
     ** Getter for tokenSecret
@@ -252,4 +274,38 @@ public class CrmQBitConfig implements QBitConfig
       return (this);
    }
 
+   /*******************************************************************************
+    ** Scheduler configuration: schedulerName.
+    *******************************************************************************/
+   public String getSchedulerName()
+   {
+      return (schedulerName);
+   }
+
+   /*******************************************************************************
+    ** Configure schedulerName.
+    *******************************************************************************/
+   public CrmQBitConfig withSchedulerName(String schedulerName)
+   {
+      this.schedulerName = schedulerName;
+      return (this);
+   }
+
+   /*******************************************************************************
+    ** Scheduler configuration: schedulesEnabled.
+    *******************************************************************************/
+   public Boolean getSchedulesEnabled()
+   {
+      return (schedulesEnabled);
+   }
+
+   /*******************************************************************************
+    ** Configure schedulesEnabled.
+    *******************************************************************************/
+   public CrmQBitConfig withSchedulesEnabled(Boolean schedulesEnabled)
+   {
+      this.schedulesEnabled = schedulesEnabled;
+      return (this);
+   }
+
 }

src/main/java/com/kingsrook/qbits/crm/CrmQBitProducer.java

@@ -26,7 +26,9 @@
 package com.kingsrook.qbits.crm;
 
 
+import java.util.List;
 import java.util.Locale;
+import java.util.Map;
 import com.kingsrook.qbits.crm.activities.customizers.ActivityLastDateCustomizer;
 import com.kingsrook.qbits.crm.activities.model.Activity;
 import com.kingsrook.qbits.crm.core.customizers.FormSubmissionAutoLinkCustomizer;
@@ -38,6 +40,8 @@ import com.kingsrook.qbits.crm.deals.customizers.DealProductRecalculationCustomi
 import com.kingsrook.qbits.crm.deals.customizers.DealStageChangeCustomizer;
 import com.kingsrook.qbits.crm.deals.model.Deal;
 import com.kingsrook.qbits.crm.deals.model.DealProduct;
+import com.kingsrook.qbits.crm.email.CrmEmailProcessIdentity;
+import com.kingsrook.qbits.crm.scoring.AutoAssignCustomizer;
 import com.kingsrook.qbits.crm.security.CrmPermissions;
 import com.kingsrook.qbits.crm.widgets.CrmWidgetMetaData;
 import com.kingsrook.qqq.backend.core.actions.customizers.MultiCustomizer;
@@ -53,6 +57,9 @@ import com.kingsrook.qqq.backend.core.model.metadata.audits.AuditLevel;
 import com.kingsrook.qqq.backend.core.model.metadata.audits.QAuditRules;
 import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
 import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReferenceWithProperties;
+import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordFeature;
+import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordFeaturesMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordOwnership;
 import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.joins.QJoinMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppSection;
@@ -60,7 +67,10 @@ import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
 import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitMetaDataProducer;
+import com.kingsrook.qqq.backend.core.model.metadata.scheduleing.QScheduleMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QFieldSection;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.Tier;
 
 
 /*******************************************************************************
@@ -106,17 +116,42 @@ public class CrmQBitProducer implements QBitMetaDataProducer<CrmQBitConfig>
       for(QWidgetMetaData widget : metaDataProducerMultiOutput.getEach(QWidgetMetaData.class))
       {
          widget.setName(snakeCase(widget.getName()));
-         widget.setPermissionRules(CrmPermissions.hasAccessRules(CrmPermissions.BASE));
+         QCodeReference permissionChecker = widget.getPermissionRules() == null ? null : widget.getPermissionRules().getCustomPermissionChecker();
+         widget.setPermissionRules(CrmPermissions.hasAccessRules(CrmPermissions.BASE).withCustomPermissionChecker(permissionChecker));
+         widget.setGridColumns(12);
          if(widget.getDefaultValues() != null && widget.getDefaultValues().get("joinName") instanceof String joinName)
          {
-            widget.withDefaultValue("joinName", snakeCase(joinName));
+            String normalizedJoinName = snakeCase(joinName);
+            widget.withDefaultValue("joinName", normalizedJoinName);
+            QJoinMetaData join = metaDataProducerMultiOutput.getEach(QJoinMetaData.class).stream()
+               .filter(candidate -> candidate.getName().equals(normalizedJoinName)).findFirst().orElseThrow();
+            QTableMetaData parent = metaDataProducerMultiOutput.get(QTableMetaData.class, join.getLeftTable());
+            boolean alreadyAttached = parent.getSections() != null && parent.getSections().stream()
+               .anyMatch(section -> section.getWidgetName() != null && snakeCase(section.getWidgetName()).equals(widget.getName()));
+            if(!alreadyAttached)
+            {
+               parent.withSection(new QFieldSection(widget.getName(), widget.getLabel(), new QIcon("list"), Tier.T2, List.of())
+                  .withWidgetName(widget.getName()));
+            }
          }
       }
 
+      metaDataProducerMultiOutput.get(QTableMetaData.class, "crm_form_submission").setLabel("Web Leads");
+      metaDataProducerMultiOutput.get(QTableMetaData.class, "crm_email_list").setLabel("Lists");
+      metaDataProducerMultiOutput.get(QTableMetaData.class, Contact.TABLE_NAME).withSection(
+         new QFieldSection("crm_contact_timeline", "Timeline", new QIcon("history"), Tier.T2, List.of())
+            .withWidgetName("crm_contact_timeline"));
+
       for(QProcessMetaData process : metaDataProducerMultiOutput.getEach(QProcessMetaData.class))
       {
          process.setPermissionRules(CrmPermissions.processRules(process.getName()));
          process.setSourceQBitName(getQBitMetaData().getName());
+         Integer seconds = Map.of("crm_process_sequence_steps", 300, "crm_calculate_lead_scores", 3600,
+            "crm_sync_email_delivery", 900, "crm_expire_consents", 86400).get(process.getName());
+         if(seconds != null && qBitConfig.getSchedulerName() != null && !Boolean.FALSE.equals(qBitConfig.getSchedulesEnabled()))
+         {
+            process.withSchedule(new QScheduleMetaData().withSchedulerName(qBitConfig.getSchedulerName()).withRepeatSeconds(seconds));
+         }
       }
 
       /////////////////////////////////////////////////////////////////////////
@@ -149,11 +184,26 @@ public class CrmQBitProducer implements QBitMetaDataProducer<CrmQBitConfig>
             || tableName.equals(Deal.TABLE_NAME) || tableName.equals(Activity.TABLE_NAME))
          {
             table.withAuditRules(new QAuditRules().withAuditLevel(AuditLevel.FIELD));
+            RecordFeaturesMetaData features = RecordFeaturesMetaData.ofOrWithNew(table)
+               .withOwnership(new RecordOwnership().withOwnerFieldName("ownerUserId"));
+            for(RecordFeature feature : List.of(RecordFeature.TAGS, RecordFeature.COMMENTS, RecordFeature.ATTACHMENTS,
+               RecordFeature.FOLLOW, RecordFeature.TIMELINE, RecordFeature.RECENTS, RecordFeature.FAVORITES))
+            {
+               features.withEnabled(feature, true);
+            }
+            if(!tableName.equals(Activity.TABLE_NAME))
+            {
+               features.withEnabled(RecordFeature.TASKS, true).withEnabled(RecordFeature.ASSIGNMENT, true);
+            }
          }
 
          ///////////////////////////////////////////////////////////////
          // wire activity timestamp propagation                       //
          ///////////////////////////////////////////////////////////////
+         if(tableName.equals(Contact.TABLE_NAME) || tableName.equals(Deal.TABLE_NAME))
+         {
+            addOrComposeCustomizer(table, TableCustomizers.POST_INSERT_RECORD, new QCodeReference(AutoAssignCustomizer.class));
+         }
          if(tableName.equals(Activity.TABLE_NAME))
          {
             addOrComposeCustomizer(table, TableCustomizers.POST_INSERT_RECORD, new QCodeReference(ActivityLastDateCustomizer.class));
@@ -236,10 +286,7 @@ public class CrmQBitProducer implements QBitMetaDataProducer<CrmQBitConfig>
     *******************************************************************************/
    public static QAppSection produceAppSection()
    {
-      return new QAppSection()
-         .withName("crm")
-         .withLabel("CRM")
-         .withIcon(new QIcon().withName("contacts"));
+      return (CrmAppMetaData.mainSection());
    }
 
 
@@ -295,12 +342,22 @@ public class CrmQBitProducer implements QBitMetaDataProducer<CrmQBitConfig>
          output.add(item instanceof QWidgetMetaData widget
             ? new CrmWidgetMetaData(widget).withSourceQBitName(getQBitMetaData().getName()) : item);
       }
+      if(!Boolean.FALSE.equals(qBitConfig.getAppEnabled()))
+      {
+         output.add(CrmAppMetaData.build(output, getQBitMetaData().getName()));
+      }
       output.add(new QAlertTypeMetaData().withName("crm.lead.received").withLabel("New web lead")
          .withDefaultSeverity(AlertSeverity.INFO).withSourceQBitName(getQBitMetaData().getName()));
       output.add(new QAlertTypeMetaData().withName("crm.sequence.failing").withLabel("Sequence needs attention")
          .withSourceQBitName(getQBitMetaData().getName()));
       output.add(new QAlertTypeMetaData().withName("crm.email.unavailable").withLabel("Email unavailable")
          .withSourceQBitName(getQBitMetaData().getName()));
+      for(String notification : List.of("crm.lead.assigned", "crm.deal.won", "crm.deal.rotting", "crm.task.due"))
+      {
+         output.add(new QAlertTypeMetaData().withName(notification).withLabel(notification)
+            .withDefaultSeverity(AlertSeverity.INFO).withSourceQBitName(getQBitMetaData().getName()));
+      }
+      qInstance.withProcessLifecycleListener(new QCodeReference(CrmEmailProcessIdentity.class));
       registerHooks(qInstance);
       return (output);
    }

src/main/java/com/kingsrook/qbits/crm/actions/CrmAction.java

@@ -0,0 +1,656 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2026.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+package com.kingsrook.qbits.crm.actions;
+
+
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import com.kingsrook.qbits.crm.activities.processes.LogActivityProcess;
+import com.kingsrook.qbits.crm.api.CrmApi;
+import com.kingsrook.qbits.crm.core.model.enums.CrmEnrollmentStatus;
+import com.kingsrook.qbits.crm.core.processes.ConvertFormToContactProcess;
+import com.kingsrook.qbits.crm.core.processes.GdprAnonymizeContactProcess;
+import com.kingsrook.qbits.crm.core.processes.MergeCompaniesProcess;
+import com.kingsrook.qbits.crm.core.processes.MergeContactsProcess;
+import com.kingsrook.qbits.crm.core.processes.SubscribeToListProcess;
+import com.kingsrook.qbits.crm.core.processes.UnsubscribeFromListProcess;
+import com.kingsrook.qbits.crm.deals.processes.CloneDealProcess;
+import com.kingsrook.qbits.crm.deals.processes.ConvertLeadProcess;
+import com.kingsrook.qbits.crm.deals.processes.ManageContactOnDealProcess;
+import com.kingsrook.qbits.crm.email.CrmEmailTransport;
+import com.kingsrook.qbits.crm.email.CrmEmails;
+import com.kingsrook.qbits.crm.email.processes.EnrollInSequenceProcess;
+import com.kingsrook.qbits.crm.email.processes.UnenrollFromSequenceProcess;
+import com.kingsrook.qbits.crm.scoring.processes.AutoAssignProcess;
+import com.kingsrook.qbits.crm.scoring.processes.CalculateLeadScoresProcess;
+import com.kingsrook.qqq.backend.core.actions.permissions.PermissionsHelper;
+import com.kingsrook.qqq.backend.core.actions.permissions.TablePermissionSubType;
+import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
+import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
+import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
+import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.exceptions.QUserFacingException;
+import com.kingsrook.qqq.backend.core.logging.QLogger;
+import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
+import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
+import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractLoadStep;
+import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractTransformStep;
+import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ExtractViaQueryStep;
+import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ProcessSummaryProviderInterface;
+import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
+import com.kingsrook.qqq.backend.core.utils.JsonUtils;
+import com.kingsrook.qqq.backend.core.utils.StringUtils;
+
+
+/*******************************************************************************
+ ** Shared streamed action lifecycle. Review never invokes a mutating operation.
+ *******************************************************************************/
+public final class CrmAction
+{
+   private static final QLogger LOG = QLogger.getLogger(CrmAction.class);
+
+
+
+   /*******************************************************************************
+    ** Utility holder.
+    *******************************************************************************/
+   private CrmAction()
+   {
+   }
+
+   /*******************************************************************************
+    ** Pin extraction to metadata and require an explicit record selection/filter.
+    *******************************************************************************/
+   public static class Extract extends ExtractViaQueryStep
+   {
+      @Override
+      public void preRun(RunBackendStepInput input, RunBackendStepOutput output) throws QException
+      {
+         String table = CrmActions.action(input.getProcessName()).table();
+         input.addValue(StreamedETLWithFrontendProcess.FIELD_SOURCE_TABLE, table);
+         input.addValue(StreamedETLWithFrontendProcess.FIELD_DESTINATION_TABLE, table);
+         super.preRun(input, output);
+      }
+
+      /*******************************************************************************
+       ** Reuse the core selection parser for preparation and merge validation.
+       *******************************************************************************/
+      QQueryFilter selection(RunBackendStepInput input) throws QException
+      {
+         if(!StringUtils.hasContent(input.getValueString("queryFilterJson"))
+            && (input.getCallback() == null || input.getCallback().getQueryFilter() == null))
+         {
+            throw (new QUserFacingException("Select records before running this action"));
+         }
+         return (super.getQueryFilter(input).clone());
+      }
+
+      @Override
+      protected QQueryFilter getQueryFilter(RunBackendStepInput input) throws QException
+      {
+         return (selection(input));
+      }
+   }
+
+   /*******************************************************************************
+    ** Populate read-only pipeline context and the merge survivor dropdown.
+    *******************************************************************************/
+   public static class Prepare implements BackendStep
+   {
+      @Override
+      public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
+      {
+         CrmActions.Action action = CrmActions.action(input.getProcessName());
+         QQueryFilter filter = new Extract().selection(input);
+         if(action.operation().equals("merge"))
+         {
+            List<QRecord> rows = selectedPair(action, input);
+            output.addValue("mergeIds", new ArrayList<>(rows.stream().map(r -> r.getValue("id")).toList()));
+            output.addValue("primaryId", rows.getFirst().getValue("id"));
+         }
+         else
+         {
+            List<QRecord> rows = new QueryAction().execute(new QueryInput(action.table()).withFilter(filter.withLimit(2))
+               .withFieldNamesToInclude(Set.of("pipelineId")).withSelectDistinct(true)).getRecords();
+            List<Serializable> pipelines = rows.stream().map(r -> r.getValue("pipelineId")).distinct().toList();
+            if(pipelines.size() != 1)
+            {
+               throw (new QUserFacingException("Select deals from one pipeline to choose a target stage"));
+            }
+            output.addValue("pipelineId", pipelines.getFirst());
+         }
+      }
+   }
+
+   /*******************************************************************************
+    ** Validate and describe each selected row, preserving keys for standard links.
+    *******************************************************************************/
+   public static class Transform extends AbstractTransformStep implements ProcessSummaryProviderInterface
+   {
+      /*******************************************************************************
+       ** Resolve the host transport; tests may supply a fake.
+       *******************************************************************************/
+      protected CrmEmailTransport emailTransport() throws QException
+      {
+         return (new CrmEmailTransport(CrmApi.config()));
+      }
+
+      private final Map<String, ProcessSummaryLine> lines = new LinkedHashMap<>();
+      private final Map<String, List<Serializable>> duplicateGroups = new LinkedHashMap<>();
+      private List<QRecord> pair;
+
+      @Override
+      public void preRun(RunBackendStepInput input, RunBackendStepOutput output) throws QException
+      {
+         CrmActions.Action action = CrmActions.action(input.getProcessName());
+         if(action.operation().equals("merge"))
+         {
+            pair = selectedPair(action, input);
+            if(pair.stream().noneMatch(r -> Objects.equals(r.getValue("id"), input.getValue("primaryId"))))
+            {
+               throw (new QUserFacingException("The record to keep must be one of the two selected records"));
+            }
+         }
+      }
+
+      @Override
+      public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
+      {
+         CrmActions.Action action = CrmActions.action(input.getProcessName());
+         for(QRecord extracted : input.getRecords())
+         {
+            Serializable id = extracted.getValue("id");
+            try
+            {
+               QRecord row = requireRow(action, input, id);
+               String reason = check(action, input, row, action.operation().equals("email") ? emailTransport() : null);
+               if(reason != null)
+               {
+                  add(lines, Status.WARNING, reason, reason, id);
+                  continue;
+               }
+               if(action.operation().equals("merge"))
+               {
+                  if(!Objects.equals(id, input.getValue("primaryId")))
+                  {
+                     continue;
+                  }
+                  Serializable secondary = pair.stream().map(r -> r.getValue("id")).filter(key -> !key.equals(id)).findFirst().orElseThrow();
+                  requireRow(action, input, secondary);
+                  row.setValue("mergeSecondaryId", secondary);
+               }
+               if(action.operation().equals("duplicates"))
+               {
+                  String key = row.getValueString(action.table().equals("crm_contact") ? "email" : "domain");
+                  if(StringUtils.hasContent(key))
+                  {
+                     duplicateGroups.computeIfAbsent(key.toLowerCase(Locale.ROOT), ignored -> new ArrayList<>()).add(id);
+                  }
+               }
+               String future = action.operation().equals("auto") ? "will be evaluated for assignment" : "will be " + action.past();
+               add(lines, Status.OK, future, "was " + action.past(), id);
+               output.addRecord(row);
+            }
+            catch(QUserFacingException e)
+            {
+               add(lines, Status.WARNING, e.getMessage(), e.getMessage(), id);
+            }
+            catch(QException | RuntimeException e)
+            {
+               LOG.warn("CRM action validation failed", e);
+               add(lines, Status.ERROR, "cannot be processed; validation failed (see server log)", "was not processed; validation failed (see server log)", id);
+            }
+         }
+      }
+
+      @Override
+      public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean result)
+      {
+         ArrayList<ProcessSummaryLineInterface> summary = new ArrayList<>(lines.values());
+         int group = 0;
+         for(List<Serializable> ids : duplicateGroups.values())
+         {
+            if(ids.size() > 1)
+            {
+               ProcessSummaryLine line = line(Status.WARNING, "will appear in duplicate group " + (++group), "appeared in duplicate group " + group);
+               ids.forEach(line::incrementCountAndAddPrimaryKey);
+               summary.add(line);
+            }
+         }
+         return (summary);
+      }
+   }
+
+   /*******************************************************************************
+    ** Reauthorize and validate at execution time; count only actual outcomes.
+    *******************************************************************************/
+   public static class Load extends AbstractLoadStep implements ProcessSummaryProviderInterface
+   {
+      /*******************************************************************************
+       ** Resolve the host transport; tests may supply a fake.
+       *******************************************************************************/
+      protected CrmEmailTransport emailTransport() throws QException
+      {
+         return (new CrmEmailTransport(CrmApi.config()));
+      }
+
+      private final Map<String, ProcessSummaryLine> lines = new LinkedHashMap<>();
+      private boolean duplicates;
+
+      @Override
+      public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
+      {
+         CrmActions.Action action = CrmActions.action(input.getProcessName());
+         duplicates = action.operation().equals("duplicates");
+         for(QRecord reviewed : input.getRecords())
+         {
+            Serializable id = reviewed.getValue("id");
+            try
+            {
+               QRecord row = requireRow(action, input, id);
+               String reason = check(action, input, row, action.operation().equals("email") ? emailTransport() : null);
+               if(reason != null)
+               {
+                  add(lines, Status.WARNING, reason, reason, id);
+                  continue;
+               }
+               row.setValue("mergeSecondaryId", reviewed.getValue("mergeSecondaryId"));
+               String skipped = perform(action, input, row, action.operation().equals("email") ? emailTransport() : null);
+               if(skipped == null)
+               {
+                  add(lines, Status.OK, "will be " + action.past(), "was " + action.past(), id);
+               }
+               else
+               {
+                  add(lines, Status.WARNING, skipped, skipped, id);
+               }
+            }
+            catch(QException | RuntimeException e)
+            {
+               LOG.warn("CRM action execution failed", e);
+               String reason = e instanceof QUserFacingException ? e.getMessage() : "see server log";
+               add(lines, Status.ERROR, "cannot be processed: " + reason, "failed: " + reason, id);
+            }
+         }
+      }
+
+      @Override
+      public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean result)
+      {
+         ArrayList<ProcessSummaryLineInterface> summary = new ArrayList<>(lines.values());
+         if(getTransformStep() instanceof Transform transform)
+         {
+            if(duplicates)
+            {
+               return (transform.getProcessSummary(output, result));
+            }
+            transform.lines.values().stream().filter(line -> line.getStatus() != Status.OK).forEach(summary::add);
+         }
+         return (summary);
+      }
+   }
+
+   /*******************************************************************************
+    ** Read current records under the session; read access alone cannot mutate.
+    *******************************************************************************/
+   private static QRecord requireRow(CrmActions.Action action, RunBackendStepInput input, Serializable id) throws QException
+   {
+      TablePermissionSubType permission = action.operation().equals("duplicates") ? TablePermissionSubType.READ : TablePermissionSubType.EDIT;
+      if(!PermissionsHelper.hasProcessPermission(input, action.name()) || !PermissionsHelper.hasTablePermission(input, action.table(), permission))
+      {
+         throw (new QUserFacingException("Permission denied for this action"));
+      }
+      QRecord row = GetAction.execute(action.table(), id);
+      if(row == null)
+      {
+         throw (new QUserFacingException("Record is unavailable"));
+      }
+      return (row);
+   }
+
+   /*******************************************************************************
+    ** Merge only an explicit pair, including when called outside the UI.
+    *******************************************************************************/
+   private static List<QRecord> selectedPair(CrmActions.Action action, RunBackendStepInput input) throws QException
+   {
+      QQueryFilter filter = new Extract().selection(input);
+      filter.setLimit(3);
+      List<QRecord> rows = new QueryAction().execute(new QueryInput(action.table()).withFilter(filter)).getRecords();
+      if(rows.size() != 2)
+      {
+         throw (new QUserFacingException("Select exactly two records to merge"));
+      }
+      return (rows);
+   }
+
+   /*******************************************************************************
+    ** Shared read-only checks for Review and Result.
+    *******************************************************************************/
+   private static String check(CrmActions.Action action, RunBackendStepInput input, QRecord row, CrmEmailTransport transport) throws QException
+   {
+      for(QFieldMetaData field : CrmActions.inputFields(action))
+      {
+         Serializable value = input.getValue(field.getName());
+         if(Boolean.TRUE.equals(field.getIsRequired()) && (value == null || value instanceof String text && text.isBlank()))
+         {
+            return ("Supply " + field.getLabel() + " before reviewing");
+         }
+         if(value != null && field.getPossibleValueSourceName() != null && !field.getName().equals("primaryId"))
+         {
+            if(GetAction.execute(field.getPossibleValueSourceName(), value) == null)
+            {
+               return (field.getLabel() + " is unavailable");
+            }
+         }
+      }
+      switch(action.operation())
+      {
+         case "email":
+            return (CrmEmails.skipReason(row, transport));
+         case "subscribe", "unsubscribe":
+            if(!StringUtils.hasContent(row.getValueString("email")))
+            {
+               return ("Contact has no email address");
+            }
+            break;
+         case "enroll":
+            if(Boolean.TRUE.equals(row.getValueBoolean("doNotEmail")))
+            {
+               return ("Contact has opted out of email");
+            }
+            if(!Boolean.TRUE.equals(GetAction.execute("crm_email_sequence", input.getValue("sequenceId")).getValueBoolean("isActive")))
+            {
+               return ("Sequence is not active");
+            }
+            if(!enrollments(input, row).isEmpty())
+            {
+               return ("Contact is already enrolled in this sequence");
+            }
+            break;
+         case "unenroll":
+            if(enrollments(input, row).isEmpty())
+            {
+               return ("Contact has no active enrollment in this sequence");
+            }
+            break;
+         case "stage", "won", "lost", "reopen":
+            QRecord target = targetStage(action, input, row);
+            if(target == null)
+            {
+               return ("No matching target stage exists in the deal's pipeline");
+            }
+            if(!Objects.equals(row.getValue("pipelineId"), target.getValue("pipelineId")))
+            {
+               return ("Target stage does not belong to this deal's pipeline");
+            }
+            if(Objects.equals(row.getValue("pipelineStageId"), target.getValue("id")))
+            {
+               return ("Deal is already in the target stage");
+            }
+            if(action.operation().equals("reopen"))
+            {
+               QRecord current = GetAction.execute("crm_pipeline_stage", row.getValue("pipelineStageId"));
+               if(current == null || !closed(current) || closed(target))
+               {
+                  return ("Reopen requires a closed deal and an open target stage");
+               }
+            }
+            if(StringUtils.hasContent(target.getValueString("requiredFieldsJson")))
+            {
+               try
+               {
+                  for(Object field : JsonUtils.toObject(target.getValueString("requiredFieldsJson"), List.class))
+                  {
+                     if(row.getValue(field.toString()) == null)
+                     {
+                        return ("Required field '" + field + "' must be set before moving to this stage");
+                     }
+                  }
+               }
+               catch(java.io.IOException e)
+               {
+                  throw (new QException("Invalid stage required fields", e));
+               }
+            }
+            break;
+         case "createDeal":
+            if(query("crm_pipeline_stage", "pipelineId", input.getValue("pipelineId")).isEmpty())
+            {
+               return ("The pipeline has no stages");
+            }
+            break;
+         default:
+            break;
+      }
+      return (null);
+   }
+
+   /*******************************************************************************
+    ** Terminal stage lookup stays within each selected deal's pipeline.
+    *******************************************************************************/
+   private static QRecord targetStage(CrmActions.Action action, RunBackendStepInput input, QRecord row) throws QException
+   {
+      if(List.of("stage", "reopen").contains(action.operation()))
+      {
+         return (GetAction.execute("crm_pipeline_stage", input.getValue("targetStageId")));
+      }
+      List<QRecord> stages = new QueryAction().execute(new QueryInput("crm_pipeline_stage").withFilter(new QQueryFilter()
+         .withCriteria(new QFilterCriteria("pipelineId", QCriteriaOperator.EQUALS, row.getValue("pipelineId")))
+         .withCriteria(new QFilterCriteria(action.operation().equals("won") ? "isClosedWon" : "isClosedLost", QCriteriaOperator.EQUALS, true))
+         .withOrderBy(new QFilterOrderBy("sortOrder")).withOrderBy(new QFilterOrderBy("id")).withLimit(1))).getRecords();
+      return (stages.isEmpty() ? null : stages.getFirst());
+   }
+
+   /*******************************************************************************
+    ** Closed-stage predicate.
+    *******************************************************************************/
+   private static boolean closed(QRecord stage)
+   {
+      return (Boolean.TRUE.equals(stage.getValueBoolean("isClosedWon")) || Boolean.TRUE.equals(stage.getValueBoolean("isClosedLost")));
+   }
+
+   /*******************************************************************************
+    ** Invoke the existing business operations only at Load time.
+    *******************************************************************************/
+   private static String perform(CrmActions.Action action, RunBackendStepInput input, QRecord row, CrmEmailTransport transport) throws QException
+   {
+      RunBackendStepInput call = new RunBackendStepInput().withProcessName(input.getProcessName()).withValues(new HashMap<>());
+      for(QFieldMetaData field : CrmActions.inputFields(action))
+      {
+         call.addValue(field.getName(), input.getValue(field.getName()));
+      }
+      Serializable id = row.getValue("id");
+      call.addValue("contactId", action.table().equals("crm_contact") ? id : input.getValue("contactId"));
+      call.addValue("companyId", action.table().equals("crm_company") ? id : null);
+      call.addValue("dealId", action.table().equals("crm_deal") ? id : null);
+      call.addValue("formSubmissionId", id);
+      RunBackendStepOutput result = new RunBackendStepOutput();
+      switch(action.operation())
+      {
+         case "email" ->
+         {
+            String processUUID = input.getValueString("crmEmailProcessUUID");
+            if(processUUID == null)
+            {
+               throw (new QUserFacingException("Email process identity is missing; start the Send Email action again"));
+            }
+            CrmEmails.send(transport, row, input.getValueInteger("emailTemplateId"), input.getValueString("subject"), input.getValueString("body"),
+               "email", id + ":" + processUUID, null);
+         }
+         case "assign" -> update(action.table(), new QRecord().withValue("id", id).withValue("ownerUserId", input.getValue("newOwnerUserId")));
+         case "auto" ->
+         {
+            call.addValue("entityType", "CONTACT");
+            call.addValue("recordId", id);
+            new AutoAssignProcess().run(call, result);
+            if(!"ASSIGNED".equals(result.getValue("action")))
+            {
+               return ("No assignment rule matched; owner unchanged");
+            }
+         }
+         case "subscribe", "unsubscribe" ->
+         {
+            call.addValue("email", row.getValue("email"));
+            call.addValue("firstName", row.getValue("firstName"));
+            call.addValue("subscriptionSource", "CRM record action");
+            if(action.operation().equals("subscribe"))
+            {
+               new SubscribeToListProcess().run(call, result);
+            }
+            else
+            {
+               new UnsubscribeFromListProcess().run(call, result);
+            }
+            if(List.of("NOOP", "NOT_FOUND").contains(result.getValue("action")))
+            {
+               return ("List membership unchanged");
+            }
+         }
+         case "enroll" -> new EnrollInSequenceProcess().run(call, result);
+         case "unenroll" ->
+         {
+            for(QRecord enrollment : enrollments(input, row))
+            {
+               call.addValue("enrollmentId", enrollment.getValue("id"));
+               new UnenrollFromSequenceProcess().run(call, result);
+            }
+         }
+         case "activity" -> new LogActivityProcess().run(call, result);
+         case "score" -> new CalculateLeadScoresProcess().recalculate(row);
+         case "merge" ->
+         {
+            Serializable secondary = row.getValue("mergeSecondaryId");
+            List<QRecord> pair = selectedPair(action, input);
+            if(secondary == null || secondary.equals(id) || pair.stream().noneMatch(r -> Objects.equals(r.getValue("id"), secondary)))
+            {
+               throw (new QUserFacingException("Merge selection changed; review it again"));
+            }
+            requireRow(action, input, secondary);
+            String entity = action.table().equals("crm_contact") ? "Contact" : "Company";
+            call.addValue("primary" + entity + "Id", id);
+            call.addValue("secondary" + entity + "Id", secondary);
+            if(entity.equals("Contact"))
+            {
+               new MergeContactsProcess().run(call, result);
+            }
+            else
+            {
+               new MergeCompaniesProcess().run(call, result);
+            }
+         }
+         case "duplicates" ->
+         {
+         }
+         case "createDeal" -> new ConvertLeadProcess().run(call, result);
+         case "erase" -> new GdprAnonymizeContactProcess().run(call, result);
+         case "stage", "won", "lost", "reopen" ->
+         {
+            QRecord change = new QRecord().withValue("id", id).withValue("pipelineStageId", targetStage(action, input, row).getValue("id"));
+            if(List.of("won", "lost").contains(action.operation()))
+            {
+               change.setValue("winLossReasonId", input.getValue("winLossReasonId"));
+            }
+            update(action.table(), change);
+         }
+         case "clone" -> new CloneDealProcess().run(call, result);
+         case "addContact" ->
+         {
+            call.addValue("action", "ADD");
+            new ManageContactOnDealProcess().run(call, result);
+         }
+         case "convert" -> new ConvertFormToContactProcess().run(call, result);
+         case "activate", "pause" -> update(action.table(), new QRecord().withValue("id", id).withValue("isActive", action.operation().equals("activate")));
+         default -> throw (new QUserFacingException("Action is not available"));
+      }
+      return (null);
+   }
+
+   /*******************************************************************************
+    ** Core DML can return record errors; never present those as successful writes.
+    *******************************************************************************/
+   private static void update(String table, QRecord change) throws QException
+   {
+      List<QRecord> records = new UpdateAction().execute(new UpdateInput(table).withRecord(change)).getRecords();
+      for(QRecord record : records)
+      {
+         if(record.getErrors() != null && !record.getErrors().isEmpty())
+         {
+            throw (new QUserFacingException("Update was rejected: " + record.getErrors()));
+         }
+      }
+   }
+
+   /*******************************************************************************
+    ** Active enrollments belonging to the selected contact and chosen sequence.
+    *******************************************************************************/
+   private static List<QRecord> enrollments(RunBackendStepInput input, QRecord row) throws QException
+   {
+      return (new QueryAction().execute(new QueryInput("crm_sequence_enrollment").withFilter(new QQueryFilter()
+         .withCriteria(new QFilterCriteria("contactId", QCriteriaOperator.EQUALS, row.getValue("id")))
+         .withCriteria(new QFilterCriteria("sequenceId", QCriteriaOperator.EQUALS, input.getValue("sequenceId")))
+         .withCriteria(new QFilterCriteria("status", QCriteriaOperator.EQUALS, CrmEnrollmentStatus.ACTIVE.getPossibleValueId())))).getRecords());
+   }
+
+   /*******************************************************************************
+    ** Session-scoped related rows.
+    *******************************************************************************/
+   private static List<QRecord> query(String table, String field, Serializable value) throws QException
+   {
+      return (new QueryAction().execute(new QueryInput(table).withFilter(new QQueryFilter(new QFilterCriteria(field, QCriteriaOperator.EQUALS, value)))).getRecords());
+   }
+
+   /*******************************************************************************
+    ** Standard future and past messages work on both Review and Result.
+    *******************************************************************************/
+   private static ProcessSummaryLine line(Status status, String future, String past)
+   {
+      return (new ProcessSummaryLine(status).withSingularFutureMessage(future).withPluralFutureMessage(future)
+         .withSingularPastMessage(past).withPluralPastMessage(past.startsWith("was ") ? "were " + past.substring(4) : past));
+   }
+
+   /*******************************************************************************
+    ** Group equal outcomes and retain primary keys for standard record links.
+    *******************************************************************************/
+   private static void add(Map<String, ProcessSummaryLine> lines, Status status, String future, String past, Serializable id)
+   {
+      lines.computeIfAbsent(status + ":" + future, ignored -> line(status, future, past)).incrementCountAndAddPrimaryKey(id);
+   }
+}

src/main/java/com/kingsrook/qbits/crm/actions/CrmActions.java

@@ -0,0 +1,201 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2026.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+package com.kingsrook.qbits.crm.actions;
+
+
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import com.kingsrook.qbits.crm.security.CrmPermissions;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
+import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerMultiOutput;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
+import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QComponentType;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendComponentMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendStepMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
+import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
+
+
+/*******************************************************************************
+ ** CRM record actions. The installed definition owns the table and operation.
+ *******************************************************************************/
+public class CrmActions implements MetaDataProducerInterface<MetaDataProducerMultiOutput>
+{
+   /*******************************************************************************
+    ** Immutable action definition; no session state is shared between runs.
+    *******************************************************************************/
+   public record Action(String name, String table, String label, String operation, String past)
+   {
+   }
+
+   public static final List<Action> ACTIONS = List.of(
+      new Action("crm_bulk_assign_owner", "crm_contact", "Assign Owner", "assign", "assigned a new owner"),
+      new Action("crm_auto_assign", "crm_contact", "Auto-Assign", "auto", "auto-assigned"),
+      new Action("crm_subscribe_to_list", "crm_contact", "Add to List", "subscribe", "added to the list"),
+      new Action("crm_unsubscribe_from_list", "crm_contact", "Remove from List", "unsubscribe", "removed from the list"),
+      new Action("crm_enroll_in_sequence", "crm_contact", "Enroll in Sequence", "enroll", "enrolled in the sequence"),
+      new Action("crm_unenroll_from_sequence", "crm_contact", "Unenroll from Sequence", "unenroll", "unenrolled from the sequence"),
+      new Action("crm_log_activity", "crm_contact", "Log Activity", "activity", "given a logged activity"),
+      new Action("crm_recalculate_score", "crm_contact", "Recalculate Score", "score", "rescored"),
+      new Action("crm_merge_contacts", "crm_contact", "Merge Contacts", "merge", "merged"),
+      new Action("crm_detect_duplicates", "crm_contact", "Find Duplicates", "duplicates", "checked for duplicates"),
+      new Action("crm_convert_lead", "crm_contact", "Create Deal", "createDeal", "given a new deal"),
+      new Action("crm_gdpr_anonymize_contact", "crm_contact", "Erase Personal Data", "erase", "anonymized"),
+      new Action("crm_send_email", "crm_contact", "Send Email", "email", "sent email"),
+      new Action("crm_assign_company_owner", "crm_company", "Assign Owner", "assign", "assigned a new owner"),
+      new Action("crm_merge_companies", "crm_company", "Merge Companies", "merge", "merged"),
+      new Action("crm_find_duplicate_companies", "crm_company", "Find Duplicates", "duplicates", "checked for duplicates"),
+      new Action("crm_log_company_activity", "crm_company", "Log Activity", "activity", "given a logged activity"),
+      new Action("crm_move_deal_stage", "crm_deal", "Move Stage", "stage", "moved to the target stage"),
+      new Action("crm_mark_deal_won", "crm_deal", "Mark Won", "won", "marked won"),
+      new Action("crm_mark_deal_lost", "crm_deal", "Mark Lost", "lost", "marked lost"),
+      new Action("crm_reopen_deal", "crm_deal", "Reopen", "reopen", "reopened"),
+      new Action("crm_clone_deal", "crm_deal", "Clone", "clone", "cloned"),
+      new Action("crm_transfer_ownership", "crm_deal", "Assign Owner", "assign", "assigned a new owner"),
+      new Action("crm_manage_contact_on_deal", "crm_deal", "Add Contact", "addContact", "linked to the contact"),
+      new Action("crm_log_deal_activity", "crm_deal", "Log Activity", "activity", "given a logged activity"),
+      new Action("crm_convert_form_to_contact", "crm_form_submission", "Convert to Contact", "convert", "converted to contacts"),
+      new Action("crm_activate_sequence", "crm_email_sequence", "Activate", "activate", "activated"),
+      new Action("crm_pause_sequence", "crm_email_sequence", "Pause", "pause", "paused"));
+
+   /*******************************************************************************
+    ** Resolve exclusively by the installed process name, never client values.
+    *******************************************************************************/
+   public static Action action(String name)
+   {
+      return (ACTIONS.stream().filter(a -> a.name().equals(name)).findFirst().orElseThrow());
+   }
+
+   /*******************************************************************************
+    ** Package-discovered metadata for every UI action.
+    *******************************************************************************/
+   @Override
+   public MetaDataProducerMultiOutput produce(QInstance instance)
+   {
+      MetaDataProducerMultiOutput output = new MetaDataProducerMultiOutput();
+      ACTIONS.forEach(action -> output.add(metadata(action)));
+      return (output);
+   }
+
+   /*******************************************************************************
+    ** Input details precede the standard ETL Review and Result screens.
+    *******************************************************************************/
+   public static QProcessMetaData metadata(Action action)
+   {
+      Map<String, Serializable> values = Map.of(
+         StreamedETLWithFrontendProcess.FIELD_SOURCE_TABLE, action.table(),
+         StreamedETLWithFrontendProcess.FIELD_DESTINATION_TABLE, action.table(),
+         StreamedETLWithFrontendProcess.FIELD_SUPPORTS_FULL_VALIDATION, true,
+         StreamedETLWithFrontendProcess.FIELD_DO_FULL_VALIDATION, true,
+         StreamedETLWithFrontendProcess.FIELD_TRANSACTION_LEVEL, StreamedETLWithFrontendProcess.TRANSACTION_LEVEL_AUTO_COMMIT,
+         StreamedETLWithFrontendProcess.FIELD_PREVIEW_MESSAGE, action.label() + " for the selected records.");
+      QProcessMetaData process = StreamedETLWithFrontendProcess.defineProcessMetaData(CrmAction.Extract.class, CrmAction.Transform.class, CrmAction.Load.class, values)
+         .withName(action.name()).withLabel(action.label()).withTableName(action.table()).withMinInputRecords(1)
+         .withIcon(new QIcon("play_arrow")).withPermissionRules(CrmPermissions.processRules(action.name()));
+      if(action.operation().equals("merge"))
+      {
+         process.withMinInputRecords(2).withMaxInputRecords(2);
+      }
+      List<QFieldMetaData> fields = inputFields(action);
+      if(!fields.isEmpty())
+      {
+         QFrontendStepMetaData details = new QFrontendStepMetaData().withName("details").withLabel(action.label())
+            .withComponent(new QFrontendComponentMetaData().withType(QComponentType.EDIT_FORM));
+         fields.forEach(details::withFormField);
+         process.withStep(0, details);
+         if(List.of("stage", "reopen", "merge").contains(action.operation()))
+         {
+            process.withStep(0, new QBackendStepMetaData().withName("prepare").withCode(new QCodeReference(CrmAction.Prepare.class)));
+         }
+      }
+      return (process);
+   }
+
+   /*******************************************************************************
+    ** Required user input, including server-validated contingent stage choices.
+    *******************************************************************************/
+   public static List<QFieldMetaData> inputFields(Action action)
+   {
+      List<QFieldMetaData> fields = new ArrayList<>();
+      switch(action.operation())
+      {
+         case "email" ->
+         {
+            fields.add(field("emailTemplateId", "Template", QFieldType.INTEGER, "crm_email_template").withIsRequired(false));
+            fields.add(field("subject", "Subject", QFieldType.STRING, null).withIsRequired(false));
+            fields.add(field("body", "Body", QFieldType.TEXT, null).withIsRequired(false));
+         }
+         case "assign" -> fields.add(field("newOwnerUserId", "Owner", QFieldType.STRING, null));
+         case "subscribe", "unsubscribe" -> fields.add(field("emailListId", "List", QFieldType.INTEGER, "crm_email_list"));
+         case "enroll", "unenroll" -> fields.add(field("sequenceId", "Sequence", QFieldType.INTEGER, "crm_email_sequence"));
+         case "merge" -> fields.add(field("primaryId", "Record to Keep", QFieldType.INTEGER, action.table())
+            .withPossibleValueSourceFilter(new QQueryFilter(new QFilterCriteria("id", QCriteriaOperator.IN, "${input.mergeIds}"))));
+         case "stage", "reopen" ->
+         {
+            fields.add(field("pipelineId", "Pipeline", QFieldType.INTEGER, "crm_pipeline").withIsEditable(false));
+            fields.add(field("targetStageId", "Target Stage", QFieldType.INTEGER, "crm_pipeline_stage")
+               .withPossibleValueSourceFilter(new QQueryFilter(new QFilterCriteria("pipelineId", QCriteriaOperator.EQUALS, "${input.pipelineId}"))));
+         }
+         case "won", "lost" -> fields.add(field("winLossReasonId", "Win/Loss Reason", QFieldType.INTEGER, "crm_win_loss_reason"));
+         case "createDeal" ->
+         {
+            fields.add(field("pipelineId", "Pipeline", QFieldType.INTEGER, "crm_pipeline"));
+            fields.add(field("dealName", "Deal Name", QFieldType.STRING, null));
+            fields.add(field("dealAmount", "Amount", QFieldType.DECIMAL, null).withIsRequired(false));
+         }
+         case "addContact" ->
+         {
+            fields.add(field("contactId", "Contact", QFieldType.INTEGER, "crm_contact"));
+            fields.add(field("contactRoleId", "Role", QFieldType.INTEGER, "crm_contact_role").withIsRequired(false));
+            fields.add(field("isPrimary", "Primary Contact", QFieldType.BOOLEAN, null).withIsRequired(false));
+         }
+         case "activity" ->
+         {
+            fields.add(field("activityTypeId", "Activity Type", QFieldType.INTEGER, "crm_activity_type"));
+            fields.add(field("subject", "Subject", QFieldType.STRING, null));
+            fields.add(field("description", "Description", QFieldType.TEXT, null).withIsRequired(false));
+         }
+         default ->
+         {
+         }
+      }
+      return (fields);
+   }
+
+   /*******************************************************************************
+    ** Consistent required field metadata.
+    *******************************************************************************/
+   private static QFieldMetaData field(String name, String label, QFieldType type, String pvs)
+   {
+      return (new QFieldMetaData(name, type).withLabel(label).withIsRequired(true).withPossibleValueSourceName(pvs));
+   }
+}

src/main/java/com/kingsrook/qbits/crm/activities/processes/FireRemindersProcess.java

@@ -1,121 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** Scheduled process that fires reminders for activities with isReminderSet
- ** = true and reminderDate <= now. Clears the reminder flag after processing.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.activities.processes;
-
-
-import java.time.Instant;
-import java.util.List;
-import com.kingsrook.qbits.crm.activities.model.Activity;
-import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
-import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
-import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
-import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
-
-
-/*******************************************************************************
- ** MetaDataProducer + BackendStep for firing reminders. Intended to run on a
- ** 5-minute schedule. Finds activities with active reminders whose
- ** reminderDate has passed, clears the flag, and would fire notifications
- ** in a production environment.
- *******************************************************************************/
-public class FireRemindersProcess implements BackendStep, MetaDataProducerInterface<QProcessMetaData>
-{
-   public static final String NAME = "crm_fire_reminders";
-
-
-
-   /*******************************************************************************
-    ** Produce the process metadata.
-    *******************************************************************************/
-   @Override
-   public QProcessMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QProcessMetaData()
-         .withName(NAME)
-         .withLabel("Fire Reminders")
-         .withIcon(new QIcon().withName("alarm"))
-         .withStepList(List.of(
-            new QBackendStepMetaData()
-               .withName("execute")
-               .withCode(new QCodeReference(FireRemindersProcess.class))
-         ));
-   }
-
-
-
-   /*******************************************************************************
-    ** Execute: find activities with due reminders and clear the reminder flag.
-    *******************************************************************************/
-   @Override
-   public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
-   {
-      ///////////////////////////////////////////////
-      // query activities with pending reminders   //
-      ///////////////////////////////////////////////
-      QueryOutput queryOutput = new QueryAction().execute(
-         new QueryInput(Activity.TABLE_NAME)
-            .withFilter(new QQueryFilter()
-               .withCriteria(new QFilterCriteria("isReminderSet", QCriteriaOperator.EQUALS, true))
-               .withCriteria(new QFilterCriteria("reminderDate", QCriteriaOperator.LESS_THAN_OR_EQUALS, Instant.now()))
-               .withCriteria(new QFilterCriteria("isCompleted", QCriteriaOperator.EQUALS, false))));
-
-      int firedCount = 0;
-      for(QRecord record : queryOutput.getRecords())
-      {
-         /////////////////////////////////////////////////////////////
-         // In production this would fire a notification. For now,  //
-         // just clear the reminder flag.                           //
-         /////////////////////////////////////////////////////////////
-         QRecord updateRecord = new QRecord()
-            .withValue("id", record.getValueInteger("id"))
-            .withValue("isReminderSet", false);
-
-         new UpdateAction().execute(
-            new UpdateInput(Activity.TABLE_NAME).withRecord(updateRecord));
-
-         firedCount++;
-      }
-
-      output.addValue("firedCount", firedCount);
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/activities/processes/LogActivityProcess.java

@@ -27,13 +27,7 @@
 package com.kingsrook.qbits.crm.activities.processes;
 
 
-import java.util.List;
 import com.kingsrook.qbits.crm.activities.model.Activity;
-import com.kingsrook.qbits.crm.activities.model.ActivityOutcome;
-import com.kingsrook.qbits.crm.activities.model.ActivityType;
-import com.kingsrook.qbits.crm.core.model.Company;
-import com.kingsrook.qbits.crm.core.model.Contact;
-import com.kingsrook.qbits.crm.core.model.enums.CrmDirection;
 import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
 import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
 import com.kingsrook.qqq.backend.core.context.QContext;
@@ -42,62 +36,18 @@ import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInpu
 import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertOutput;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
-import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 
 
 /*******************************************************************************
- ** MetaDataProducer + BackendStep that creates a crm_activity record from
+ ** Backend operation used by the streamed CRM actions that creates a crm_activity record from
  ** the provided input fields. Sets ownerUserId from the current session.
  *******************************************************************************/
-public class LogActivityProcess implements BackendStep, MetaDataProducerInterface<QProcessMetaData>
+public class LogActivityProcess implements BackendStep
 {
    public static final String NAME = "crm_log_activity";
 
 
 
-   /*******************************************************************************
-    ** Produce the process metadata with input fields for all activity fields.
-    *******************************************************************************/
-   @Override
-   public QProcessMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QProcessMetaData()
-         .withName(NAME)
-         .withLabel("Log Activity")
-         .withTableName(Activity.TABLE_NAME)
-         .withIcon(new QIcon().withName("event_note"))
-         .withStepList(List.of(
-            new QBackendStepMetaData()
-               .withName("execute")
-               .withCode(new QCodeReference(LogActivityProcess.class))
-               .withInputData(new QFunctionInputMetaData().withFieldList(List.of(
-                  new QFieldMetaData("activityTypeId", QFieldType.INTEGER).withIsRequired(true).withPossibleValueSourceName(ActivityType.TABLE_NAME),
-                  new QFieldMetaData("subject", QFieldType.STRING).withIsRequired(true),
-                  new QFieldMetaData("description", QFieldType.STRING),
-                  new QFieldMetaData("contactId", QFieldType.INTEGER).withPossibleValueSourceName(Contact.TABLE_NAME),
-                  new QFieldMetaData("companyId", QFieldType.INTEGER).withPossibleValueSourceName(Company.TABLE_NAME),
-                  new QFieldMetaData("dealId", QFieldType.INTEGER),
-                  new QFieldMetaData("direction", QFieldType.INTEGER).withPossibleValueSourceName(CrmDirection.NAME),
-                  new QFieldMetaData("durationMinutes", QFieldType.INTEGER),
-                  new QFieldMetaData("outcomeId", QFieldType.INTEGER).withPossibleValueSourceName(ActivityOutcome.TABLE_NAME),
-                  new QFieldMetaData("location", QFieldType.STRING),
-                  new QFieldMetaData("conferenceLink", QFieldType.STRING),
-                  new QFieldMetaData("startDate", QFieldType.DATE_TIME),
-                  new QFieldMetaData("endDate", QFieldType.DATE_TIME)
-               )))
-         ));
-   }
-
-
-
    /*******************************************************************************
     ** Execute the process: build an Activity entity from inputs and insert it.
     *******************************************************************************/

src/main/java/com/kingsrook/qbits/crm/activities/processes/ProcessOverdueTasksProcess.java

@@ -1,156 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** Scheduled process that finds overdue tasks and marks them with a system
- ** note. In production this would fire notifications; for now it logs a
- ** message on each overdue activity.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.activities.processes;
-
-
-import java.time.Instant;
-import java.util.List;
-import com.kingsrook.qbits.crm.activities.model.Activity;
-import com.kingsrook.qbits.crm.activities.model.ActivityType;
-import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
-import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
-import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
-import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
-
-
-/*******************************************************************************
- ** MetaDataProducer + BackendStep for processing overdue tasks. Intended to
- ** run on an hourly schedule. Finds activities of type "Task" with dueDate
- ** in the past and isCompleted=false.
- *******************************************************************************/
-public class ProcessOverdueTasksProcess implements BackendStep, MetaDataProducerInterface<QProcessMetaData>
-{
-   public static final String NAME = "crm_process_overdue_tasks";
-
-
-
-   /*******************************************************************************
-    ** Produce the process metadata.
-    *******************************************************************************/
-   @Override
-   public QProcessMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QProcessMetaData()
-         .withName(NAME)
-         .withLabel("Process Overdue Tasks")
-         .withIcon(new QIcon().withName("warning"))
-         .withStepList(List.of(
-            new QBackendStepMetaData()
-               .withName("execute")
-               .withCode(new QCodeReference(ProcessOverdueTasksProcess.class))
-         ));
-   }
-
-
-
-   /*******************************************************************************
-    ** Execute: find overdue tasks and mark them with a system note.
-    *******************************************************************************/
-   @Override
-   public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
-   {
-      ///////////////////////////////////////////////
-      // look up the "Task" activity type          //
-      ///////////////////////////////////////////////
-      Integer taskTypeId = lookupTaskActivityTypeId();
-      if(taskTypeId == null)
-      {
-         output.addValue("processedCount", 0);
-         return;
-      }
-
-      ///////////////////////////////////////////////
-      // query overdue, incomplete tasks           //
-      ///////////////////////////////////////////////
-      QueryOutput queryOutput = new QueryAction().execute(
-         new QueryInput(Activity.TABLE_NAME)
-            .withFilter(new QQueryFilter()
-               .withCriteria(new QFilterCriteria("activityTypeId", QCriteriaOperator.EQUALS, taskTypeId))
-               .withCriteria(new QFilterCriteria("dueDate", QCriteriaOperator.LESS_THAN, Instant.now()))
-               .withCriteria(new QFilterCriteria("isCompleted", QCriteriaOperator.EQUALS, false))));
-
-      int processedCount = 0;
-      for(QRecord record : queryOutput.getRecords())
-      {
-         /////////////////////////////////////////////////////////////////////
-         // In production this would create a notification. For now, add a  //
-         // system note to the description indicating it is overdue.        //
-         /////////////////////////////////////////////////////////////////////
-         String currentDescription = record.getValueString("description");
-         String note = "[OVERDUE] Task past due date.";
-         String newDescription = currentDescription != null ? currentDescription + " " + note : note;
-
-         QRecord updateRecord = new QRecord()
-            .withValue("id", record.getValueInteger("id"))
-            .withValue("description", newDescription);
-
-         new UpdateAction().execute(
-            new UpdateInput(Activity.TABLE_NAME).withRecord(updateRecord));
-
-         processedCount++;
-      }
-
-      output.addValue("processedCount", processedCount);
-   }
-
-
-
-   /***************************************************************************
-    ** Look up the "Task" activity type id. Returns null if not found.
-    ***************************************************************************/
-   private Integer lookupTaskActivityTypeId() throws QException
-   {
-      QueryOutput queryOutput = new QueryAction().execute(
-         new QueryInput(ActivityType.TABLE_NAME)
-            .withFilter(new QQueryFilter()
-               .withCriteria(new QFilterCriteria("name", QCriteriaOperator.EQUALS, "Task"))));
-
-      if(!queryOutput.getRecords().isEmpty())
-      {
-         return (queryOutput.getRecords().get(0).getValueInteger("id"));
-      }
-
-      return (null);
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/api/CrmIntakeApi.java

@@ -323,7 +323,7 @@ public class CrmIntakeApi
       }
       update.withValue("lastTouchDate", Instant.now());
       store.update("crm_contact", update);
-      if(contacts.isEmpty() && isUnassigned(contact.getValueString("ownerUserId")))
+      if(contacts.isEmpty() && isUnassigned(store.get("crm_contact", contact.getValueInteger("id")).getValueString("ownerUserId")))
       {
          CrmAutoAssign.assign(CrmAssignableEntity.CONTACT, contact.getValueInteger("id"));
       }

src/main/java/com/kingsrook/qbits/crm/core/CrmRecords.java

@@ -0,0 +1,72 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2026.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.crm.core;
+
+
+import java.util.List;
+import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
+import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+
+
+/*******************************************************************************
+ ** Checked writes for background work: record errors must not advance checkpoints.
+ *******************************************************************************/
+public final class CrmRecords
+{
+   /*******************************************************************************
+    ** Utility class.
+    *******************************************************************************/
+   private CrmRecords()
+   {
+   }
+
+   /*******************************************************************************
+    ** Insert and require a successful persisted row.
+    *******************************************************************************/
+   public static QRecord insert(String table, QRecord row) throws QException
+   {
+      return (check(new InsertAction().execute(new InsertInput(table).withRecord(row)).getRecords()));
+   }
+
+   /*******************************************************************************
+    ** Update and require a successful persisted row.
+    *******************************************************************************/
+   public static QRecord update(String table, QRecord row) throws QException
+   {
+      return (check(new UpdateAction().execute(new UpdateInput(table).withRecord(row)).getRecords()));
+   }
+
+   /*******************************************************************************
+    ** DML errors are failed work, never successful job progress.
+    *******************************************************************************/
+   private static QRecord check(List<QRecord> rows) throws QException
+   {
+      if(rows.isEmpty() || rows.stream().anyMatch(row -> row.getErrors() != null && !row.getErrors().isEmpty()))
+      {
+         throw (new QException("CRM write was rejected"));
+      }
+      return (rows.getFirst());
+   }
+}

src/main/java/com/kingsrook/qbits/crm/core/model/Company.java

@@ -30,6 +30,7 @@ import java.math.BigDecimal;
 import java.time.Instant;
 import java.util.List;
 import com.kingsrook.qbits.crm.CrmQBitConfig;
+import com.kingsrook.qbits.crm.activities.model.Activity;
 import com.kingsrook.qbits.crm.core.model.enums.CrmCompanyType;
 import com.kingsrook.qbits.crm.deals.model.Deal;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
@@ -63,6 +64,9 @@ import com.kingsrook.qqq.backend.core.utils.collections.MutableList;
    produceTableMetaData = true,
    tableMetaDataCustomizer = Company.TableMetaDataCustomizer.class,
    childTables = {
+      @ChildTable(childTableEntityClass = Activity.class, joinFieldName = "companyId",
+         childJoin = @ChildJoin(enabled = true),
+         childRecordListWidget = @ChildRecordListWidget(label = "Activities", enabled = true, maxRows = 25)),
       @ChildTable(childTableEntityClass = Contact.class, joinFieldName = "companyId",
          childJoin = @ChildJoin(enabled = true),
          childRecordListWidget = @ChildRecordListWidget(label = "Contacts", enabled = true, maxRows = 50)),

src/main/java/com/kingsrook/qbits/crm/core/model/Contact.java

@@ -30,7 +30,9 @@ import java.time.Instant;
 import java.util.List;
 import com.kingsrook.qbits.crm.CrmQBitConfig;
 import com.kingsrook.qbits.crm.activities.model.Activity;
-import com.kingsrook.qbits.crm.deals.model.DealContact;
+import com.kingsrook.qbits.crm.deals.model.Deal;
+import com.kingsrook.qbits.crm.email.model.EmailMessage;
+import com.kingsrook.qbits.crm.email.model.SequenceEnrollment;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.model.data.QField;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
@@ -62,12 +64,24 @@ import com.kingsrook.qqq.backend.core.utils.collections.MutableList;
    produceTableMetaData = true,
    tableMetaDataCustomizer = Contact.TableMetaDataCustomizer.class,
    childTables = {
+      @ChildTable(childTableEntityClass = Deal.class, joinFieldName = "primaryContactId",
+         childJoin = @ChildJoin(enabled = true),
+         childRecordListWidget = @ChildRecordListWidget(label = "Deals", enabled = true, maxRows = 25)),
+      @ChildTable(childTableEntityClass = EmailListMember.class, joinFieldName = "contactId",
+         childJoin = @ChildJoin(enabled = true),
+         childRecordListWidget = @ChildRecordListWidget(label = "List Memberships", enabled = true, maxRows = 25)),
+      @ChildTable(childTableEntityClass = SequenceEnrollment.class, joinFieldName = "contactId",
+         childJoin = @ChildJoin(enabled = true),
+         childRecordListWidget = @ChildRecordListWidget(label = "Enrollments", enabled = true, maxRows = 25)),
+      @ChildTable(childTableEntityClass = EmailMessage.class, joinFieldName = "contactId",
+         childJoin = @ChildJoin(enabled = true),
+         childRecordListWidget = @ChildRecordListWidget(label = "Email Messages", enabled = true, maxRows = 25)),
+      @ChildTable(childTableEntityClass = FormSubmission.class, joinFieldName = "contactId",
+         childJoin = @ChildJoin(enabled = true),
+         childRecordListWidget = @ChildRecordListWidget(label = "Web Leads", enabled = true, maxRows = 25)),
       @ChildTable(childTableEntityClass = Activity.class, joinFieldName = "contactId",
          childJoin = @ChildJoin(enabled = true),
          childRecordListWidget = @ChildRecordListWidget(label = "Activities", enabled = true, maxRows = 50)),
-      @ChildTable(childTableEntityClass = DealContact.class, joinFieldName = "contactId",
-         childJoin = @ChildJoin(enabled = true),
-         childRecordListWidget = @ChildRecordListWidget(label = "Deals", enabled = true, maxRows = 25)),
       @ChildTable(childTableEntityClass = ConsentRecord.class, joinFieldName = "contactId",
          childJoin = @ChildJoin(enabled = true),
          childRecordListWidget = @ChildRecordListWidget(label = "Consent Records", enabled = true, maxRows = 25))
@@ -227,7 +241,7 @@ public class Contact extends QRecordEntity
    @QField(possibleValueSourceName = LeadSource.TABLE_NAME)
    private Integer leadSourceId;
 
-   @QField(isRequired = true)
+   @QField()
    private String ownerUserId;
 
    @QField(maxLength = 250, valueTooLongBehavior = ValueTooLongBehavior.ERROR)

src/main/java/com/kingsrook/qbits/crm/core/model/EmailList.java

@@ -35,6 +35,9 @@ import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.ValueTooLongBehavior;
 import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
 import com.kingsrook.qqq.backend.core.model.metadata.producers.MetaDataCustomizerInterface;
+import com.kingsrook.qqq.backend.core.model.metadata.producers.annotations.ChildJoin;
+import com.kingsrook.qqq.backend.core.model.metadata.producers.annotations.ChildRecordListWidget;
+import com.kingsrook.qqq.backend.core.model.metadata.producers.annotations.ChildTable;
 import com.kingsrook.qqq.backend.core.model.metadata.producers.annotations.QMetaDataProducingEntity;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.SectionFactory;
@@ -48,7 +51,12 @@ import com.kingsrook.qqq.backend.core.model.metadata.tables.UniqueKey;
 @QMetaDataProducingEntity(
    producePossibleValueSource = true,
    produceTableMetaData = true,
-   tableMetaDataCustomizer = EmailList.TableMetaDataCustomizer.class
+   tableMetaDataCustomizer = EmailList.TableMetaDataCustomizer.class,
+   childTables = {
+      @ChildTable(childTableEntityClass = EmailListMember.class, joinFieldName = "emailListId",
+         childJoin = @ChildJoin(enabled = true),
+         childRecordListWidget = @ChildRecordListWidget(label = "Members", enabled = true, maxRows = 25))
+   }
 )
 public class EmailList extends QRecordEntity
 {

src/main/java/com/kingsrook/qbits/crm/core/model/enums/CrmEmailStatus.java

@@ -40,7 +40,13 @@ public enum CrmEmailStatus implements PossibleValueEnum<Integer>
    DRAFT(1, "Draft"),
    SENT(2, "Sent"),
    DELIVERED(3, "Delivered"),
-   FAILED(4, "Failed");
+   FAILED(4, "Failed"),
+   QUEUED(5, "Queued"),
+   BOUNCED(6, "Bounced"),
+   COMPLAINED(7, "Complained"),
+   OPENED(8, "Opened"),
+   CLICKED(9, "Clicked"),
+   SUPPRESSED(10, "Suppressed");
 
    private final Integer id;
    private final String  label;

src/main/java/com/kingsrook/qbits/crm/core/processes/BulkAssignOwnerProcess.java

@@ -1,124 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** Bulk action process to assign a new owner to selected records. Works on
- ** any table that has an ownerUserId field.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.core.processes;
-
-
-import java.util.ArrayList;
-import java.util.List;
-import com.kingsrook.qbits.crm.core.model.Contact;
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
-import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
-import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
-import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
-import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
-import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractTransformStep;
-import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
-
-
-/*******************************************************************************
- ** MetaDataProducer for the BulkAssignOwner table-level process. Uses the
- ** StreamedETL pattern with a transform step that sets ownerUserId on each
- ** record to the newOwnerUserId input value.
- *******************************************************************************/
-public class BulkAssignOwnerProcess implements MetaDataProducerInterface<QProcessMetaData>
-{
-   public static final String NAME = "crm_bulk_assign_owner";
-
-
-
-   /*******************************************************************************
-    ** Produce the process metadata.
-    *******************************************************************************/
-   @Override
-   public QProcessMetaData produce(QInstance qInstance) throws QException
-   {
-      QProcessMetaData process = StreamedETLWithFrontendProcess.processMetaDataBuilder()
-         .withName(NAME)
-         .withLabel("Bulk Assign Owner")
-         .withTableName(Contact.TABLE_NAME)
-         .withIcon(new QIcon().withName("person_pin"))
-         .withTransformStepClass(BulkAssignOwnerTransformStep.class)
-         .withPreviewMessage("Records to be assigned a new owner")
-         .withReviewStepRecordFields(List.of(
-            new QFieldMetaData("id", QFieldType.INTEGER),
-            new QFieldMetaData("ownerUserId", QFieldType.STRING).withLabel("Current Owner")
-         ))
-         .getProcessMetaData();
-
-      return (process);
-   }
-
-
-
-   /*******************************************************************************
-    ** Transform step that sets ownerUserId on each record.
-    *******************************************************************************/
-   public static class BulkAssignOwnerTransformStep extends AbstractTransformStep
-   {
-      private ProcessSummaryLine okLine = new ProcessSummaryLine(Status.OK)
-         .withSingularFutureMessage("will be assigned a new owner")
-         .withPluralFutureMessage("will be assigned a new owner")
-         .withSingularPastMessage("was assigned a new owner")
-         .withPluralPastMessage("were assigned a new owner");
-
-      /*******************************************************************************
-       ** Set ownerUserId on each record from the newOwnerUserId input.
-       *******************************************************************************/
-      @Override
-      public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
-      {
-         String newOwnerUserId = input.getValueString("newOwnerUserId");
-
-         for(QRecord record : input.getRecords())
-         {
-            record.setValue("ownerUserId", newOwnerUserId);
-            output.addRecord(record);
-            okLine.incrementCountAndAddPrimaryKey(record.getValueInteger("id"));
-         }
-      }
-
-      /*******************************************************************************
-       ** Provide process summary.
-       *******************************************************************************/
-      @Override
-      public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput runBackendStepOutput, boolean isForResultScreen)
-      {
-         ArrayList<ProcessSummaryLineInterface> rs = new ArrayList<>();
-         okLine.addSelfToListIfAnyCount(rs);
-         return (rs);
-      }
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/core/processes/ConvertFormToContactProcess.java

@@ -29,7 +29,6 @@ package com.kingsrook.qbits.crm.core.processes;
 
 
 import java.time.Instant;
-import java.util.List;
 import com.kingsrook.qbits.crm.activities.model.Activity;
 import com.kingsrook.qbits.crm.activities.model.ActivityType;
 import com.kingsrook.qbits.crm.core.model.Contact;
@@ -54,52 +53,20 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
-import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
 
 
 /*******************************************************************************
- ** MetaDataProducer + BackendStep for converting a form submission into a
+ ** Backend operation used by the streamed CRM actions for converting a form submission into a
  ** contact record. Deduplicates on email, updates the form with conversion
  ** data, and creates a Note activity on the contact.
  *******************************************************************************/
-public class ConvertFormToContactProcess implements BackendStep, MetaDataProducerInterface<QProcessMetaData>
+public class ConvertFormToContactProcess implements BackendStep
 {
    public static final String NAME = "crm_convert_form_to_contact";
 
 
 
-   /*******************************************************************************
-    ** Produce the process metadata.
-    *******************************************************************************/
-   @Override
-   public QProcessMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QProcessMetaData()
-         .withName(NAME)
-         .withLabel("Convert Form to Contact")
-         .withTableName(FormSubmission.TABLE_NAME)
-         .withIcon(new QIcon().withName("swap_horiz"))
-         .withStepList(List.of(
-            new QBackendStepMetaData()
-               .withName("execute")
-               .withCode(new QCodeReference(ConvertFormToContactProcess.class))
-               .withInputData(new QFunctionInputMetaData().withFieldList(List.of(
-                  new QFieldMetaData("formSubmissionId", QFieldType.INTEGER).withIsRequired(true)
-               )))
-         ));
-   }
-
-
-
    /*******************************************************************************
     ** Execute the conversion process.
     *******************************************************************************/

src/main/java/com/kingsrook/qbits/crm/core/processes/DetectDuplicatesProcess.java

@@ -41,49 +41,18 @@ import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutp
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
-import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
 
 
 /*******************************************************************************
- ** MetaDataProducer + BackendStep for detecting duplicate contacts or companies.
+ ** Backend operation used by the streamed CRM actions for detecting duplicate contacts or companies.
  *******************************************************************************/
-public class DetectDuplicatesProcess implements BackendStep, MetaDataProducerInterface<QProcessMetaData>
+public class DetectDuplicatesProcess implements BackendStep
 {
    public static final String NAME = "crm_detect_duplicates";
 
 
 
-   /*******************************************************************************
-    ** Produce the process metadata.
-    *******************************************************************************/
-   @Override
-   public QProcessMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QProcessMetaData()
-         .withName(NAME)
-         .withLabel("Detect Duplicates")
-         .withIcon(new QIcon().withName("find_replace"))
-         .withStepList(List.of(
-            new QBackendStepMetaData()
-               .withName("execute")
-               .withCode(new QCodeReference(DetectDuplicatesProcess.class))
-               .withInputData(new QFunctionInputMetaData().withFieldList(List.of(
-                  new QFieldMetaData("entityType", QFieldType.STRING).withIsRequired(true)
-               )))
-         ));
-   }
-
-
-
    /*******************************************************************************
     ** Execute the detect duplicates process.
     *******************************************************************************/

src/main/java/com/kingsrook/qbits/crm/core/processes/GdprAnonymizeContactProcess.java

@@ -29,7 +29,6 @@ package com.kingsrook.qbits.crm.core.processes;
 
 
 import java.time.Instant;
-import java.util.List;
 import com.kingsrook.qbits.crm.core.model.ConsentRecord;
 import com.kingsrook.qbits.crm.core.model.Contact;
 import com.kingsrook.qbits.crm.core.model.enums.CrmConsentStatus;
@@ -52,21 +51,12 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
-import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 
 
 /*******************************************************************************
- ** MetaDataProducer + BackendStep for GDPR anonymization of a contact.
+ ** Backend operation used by the streamed CRM actions for GDPR anonymization of a contact.
  *******************************************************************************/
-public class GdprAnonymizeContactProcess implements BackendStep, MetaDataProducerInterface<QProcessMetaData>
+public class GdprAnonymizeContactProcess implements BackendStep
 {
    private static final QLogger LOG = QLogger.getLogger(GdprAnonymizeContactProcess.class);
 
@@ -76,29 +66,6 @@ public class GdprAnonymizeContactProcess implements BackendStep, MetaDataProduce
 
 
 
-   /*******************************************************************************
-    ** Produce the process metadata.
-    *******************************************************************************/
-   @Override
-   public QProcessMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QProcessMetaData()
-         .withName(NAME)
-         .withLabel("GDPR Anonymize Contact")
-         .withTableName(Contact.TABLE_NAME)
-         .withIcon(new QIcon().withName("privacy_tip"))
-         .withStepList(List.of(
-            new QBackendStepMetaData()
-               .withName("execute")
-               .withCode(new QCodeReference(GdprAnonymizeContactProcess.class))
-               .withInputData(new QFunctionInputMetaData().withFieldList(List.of(
-                  new QFieldMetaData("contactId", QFieldType.INTEGER).withIsRequired(true)
-               )))
-         ));
-   }
-
-
-
    /*******************************************************************************
     ** Execute the GDPR anonymization process.
     *******************************************************************************/

src/main/java/com/kingsrook/qbits/crm/core/processes/ImportContactsProcess.java

@@ -1,159 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** Simplified import process for contacts. Takes a list of QRecords,
- ** validates required fields, checks for duplicates by email, inserts new
- ** contacts, with core table auditing.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.core.processes;
-
-
-import java.util.HashSet;
-import java.util.List;
-import java.util.Set;
-import com.kingsrook.qbits.crm.core.model.Contact;
-import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
-import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
-import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
-import com.kingsrook.qqq.backend.core.context.QContext;
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
-import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
-import com.kingsrook.qqq.backend.core.utils.StringUtils;
-
-
-/*******************************************************************************
- ** MetaDataProducer + BackendStep for importing contacts from QRecords.
- *******************************************************************************/
-public class ImportContactsProcess implements BackendStep, MetaDataProducerInterface<QProcessMetaData>
-{
-   public static final String NAME = "crm_import_contacts";
-
-
-
-   /*******************************************************************************
-    ** Produce the process metadata.
-    *******************************************************************************/
-   @Override
-   public QProcessMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QProcessMetaData()
-         .withName(NAME)
-         .withLabel("Import Contacts")
-         .withTableName(Contact.TABLE_NAME)
-         .withIcon(new QIcon().withName("upload"))
-         .withStepList(List.of(
-            new QBackendStepMetaData()
-               .withName("execute")
-               .withCode(new QCodeReference(ImportContactsProcess.class))
-         ));
-   }
-
-
-
-   /*******************************************************************************
-    ** Execute the import contacts process.
-    *******************************************************************************/
-   @Override
-   public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
-   {
-      List<QRecord> inputRecords = input.getRecords();
-      String sessionUserId = QContext.getQSession().getIdReference();
-
-      ////////////////////////////////////////////////////////////
-      // build set of existing emails for duplicate detection   //
-      ////////////////////////////////////////////////////////////
-      Set<String> existingEmails = new HashSet<>();
-      QueryOutput existingContacts = new QueryAction().execute(
-         new QueryInput(Contact.TABLE_NAME));
-      for(QRecord record : existingContacts.getRecords())
-      {
-         String email = record.getValueString("email");
-         if(StringUtils.hasContent(email))
-         {
-            existingEmails.add(email.toLowerCase());
-         }
-      }
-
-      int importedCount = 0;
-      int skippedCount  = 0;
-      int errorCount    = 0;
-
-      for(QRecord inputRecord : inputRecords)
-      {
-         String firstName = inputRecord.getValueString("firstName");
-         String lastName  = inputRecord.getValueString("lastName");
-         String email     = inputRecord.getValueString("email");
-
-         ///////////////////////////////////////////
-         // validate required fields              //
-         ///////////////////////////////////////////
-         if(!StringUtils.hasContent(firstName) || !StringUtils.hasContent(lastName) || !StringUtils.hasContent(email))
-         {
-            errorCount++;
-            continue;
-         }
-
-         ///////////////////////////////////////////
-         // check for duplicate by email          //
-         ///////////////////////////////////////////
-         if(existingEmails.contains(email.toLowerCase()))
-         {
-            skippedCount++;
-            continue;
-         }
-
-         ///////////////////////////////////////////
-         // insert the new contact                //
-         ///////////////////////////////////////////
-         QRecord newContact = new QRecord()
-            .withValue("firstName", firstName)
-            .withValue("lastName", lastName)
-            .withValue("email", email)
-            .withValue("phone", inputRecord.getValueString("phone"))
-            .withValue("ownerUserId", sessionUserId);
-
-         new InsertAction().execute(
-            new InsertInput(Contact.TABLE_NAME).withRecord(newContact));
-
-         existingEmails.add(email.toLowerCase());
-
-         importedCount++;
-      }
-
-      output.addValue("importedCount", importedCount);
-      output.addValue("skippedCount", skippedCount);
-      output.addValue("errorCount", errorCount);
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/core/processes/MergeCompaniesProcess.java

@@ -50,50 +50,17 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
-import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 
 
 /*******************************************************************************
- ** MetaDataProducer + BackendStep for merging two companies into one.
+ ** Backend operation used by the streamed CRM actions for merging two companies into one.
  *******************************************************************************/
-public class MergeCompaniesProcess implements BackendStep, MetaDataProducerInterface<QProcessMetaData>
+public class MergeCompaniesProcess implements BackendStep
 {
    public static final String NAME = "crm_merge_companies";
 
 
 
-   /*******************************************************************************
-    ** Produce the process metadata.
-    *******************************************************************************/
-   @Override
-   public QProcessMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QProcessMetaData()
-         .withName(NAME)
-         .withLabel("Merge Companies")
-         .withTableName(Company.TABLE_NAME)
-         .withIcon(new QIcon().withName("merge"))
-         .withStepList(List.of(
-            new QBackendStepMetaData()
-               .withName("execute")
-               .withCode(new QCodeReference(MergeCompaniesProcess.class))
-               .withInputData(new QFunctionInputMetaData().withFieldList(List.of(
-                  new QFieldMetaData("primaryCompanyId", QFieldType.INTEGER).withIsRequired(true),
-                  new QFieldMetaData("secondaryCompanyId", QFieldType.INTEGER).withIsRequired(true)
-               )))
-         ));
-   }
-
-
-
    /*******************************************************************************
     ** Execute the merge companies process.
     *******************************************************************************/

src/main/java/com/kingsrook/qbits/crm/core/processes/MergeContactsProcess.java

@@ -52,50 +52,17 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
-import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 
 
 /*******************************************************************************
- ** MetaDataProducer + BackendStep for merging two contacts into one.
+ ** Backend operation used by the streamed CRM actions for merging two contacts into one.
  *******************************************************************************/
-public class MergeContactsProcess implements BackendStep, MetaDataProducerInterface<QProcessMetaData>
+public class MergeContactsProcess implements BackendStep
 {
    public static final String NAME = "crm_merge_contacts";
 
 
 
-   /*******************************************************************************
-    ** Produce the process metadata.
-    *******************************************************************************/
-   @Override
-   public QProcessMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QProcessMetaData()
-         .withName(NAME)
-         .withLabel("Merge Contacts")
-         .withTableName(Contact.TABLE_NAME)
-         .withIcon(new QIcon().withName("merge"))
-         .withStepList(List.of(
-            new QBackendStepMetaData()
-               .withName("execute")
-               .withCode(new QCodeReference(MergeContactsProcess.class))
-               .withInputData(new QFunctionInputMetaData().withFieldList(List.of(
-                  new QFieldMetaData("primaryContactId", QFieldType.INTEGER).withIsRequired(true),
-                  new QFieldMetaData("secondaryContactId", QFieldType.INTEGER).withIsRequired(true)
-               )))
-         ));
-   }
-
-
-
    /*******************************************************************************
     ** Execute the merge contacts process.
     *******************************************************************************/
@@ -125,6 +92,8 @@ public class MergeContactsProcess implements BackendStep, MetaDataProducerInterf
       // Re-link activities from secondary         //
       ///////////////////////////////////////////////
       relinkRecords(Activity.TABLE_NAME, "contactId", secondaryContactId, primaryContactId);
+      relinkRecords("crm_email_message", "contactId", secondaryContactId, primaryContactId);
+      relinkRecords("crm_deal", "primaryContactId", secondaryContactId, primaryContactId);
 
       ///////////////////////////////////////////////
       // Re-link DealContact rows                  //

src/main/java/com/kingsrook/qbits/crm/core/processes/SubscribeToListProcess.java

@@ -28,7 +28,6 @@ package com.kingsrook.qbits.crm.core.processes;
 
 
 import java.time.Instant;
-import java.util.List;
 import com.kingsrook.qbits.crm.core.model.Contact;
 import com.kingsrook.qbits.crm.core.model.EmailListMember;
 import com.kingsrook.qbits.crm.core.model.enums.CrmListMemberStatus;
@@ -47,54 +46,20 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
-import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
 
 
 /*******************************************************************************
- ** MetaDataProducer + BackendStep to subscribe an email address to a list.
+ ** Backend operation used by the streamed CRM actions to subscribe an email address to a list.
  ** Handles: new subscription, resubscription after unsub, and no-op for
  ** existing active subscribers.
  *******************************************************************************/
-public class SubscribeToListProcess implements BackendStep, MetaDataProducerInterface<QProcessMetaData>
+public class SubscribeToListProcess implements BackendStep
 {
    public static final String NAME = "crm_subscribe_to_list";
 
 
 
-   /*******************************************************************************
-    ** Produce the process metadata.
-    *******************************************************************************/
-   @Override
-   public QProcessMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QProcessMetaData()
-         .withName(NAME)
-         .withLabel("Subscribe to List")
-         .withIcon(new QIcon().withName("person_add"))
-         .withStepList(List.of(
-            new QBackendStepMetaData()
-               .withName("execute")
-               .withCode(new QCodeReference(SubscribeToListProcess.class))
-               .withInputData(new QFunctionInputMetaData().withFieldList(List.of(
-                  new QFieldMetaData("email", QFieldType.STRING).withIsRequired(true),
-                  new QFieldMetaData("emailListId", QFieldType.INTEGER).withIsRequired(true),
-                  new QFieldMetaData("firstName", QFieldType.STRING),
-                  new QFieldMetaData("subscriptionSource", QFieldType.STRING)
-               )))
-         ));
-   }
-
-
-
    /*******************************************************************************
     ** Execute the subscription logic.
     *******************************************************************************/

src/main/java/com/kingsrook/qbits/crm/core/processes/UnsubscribeFromListProcess.java

@@ -28,7 +28,6 @@ package com.kingsrook.qbits.crm.core.processes;
 
 
 import java.time.Instant;
-import java.util.List;
 import com.kingsrook.qbits.crm.core.model.EmailListMember;
 import com.kingsrook.qbits.crm.core.model.enums.CrmListMemberStatus;
 import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
@@ -44,51 +43,19 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
-import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 
 
 /*******************************************************************************
- ** MetaDataProducer + BackendStep to unsubscribe an email from a list.
+ ** Backend operation used by the streamed CRM actions to unsubscribe an email from a list.
  ** Queries by emailListId + email, sets status to UNSUBSCRIBED and
  ** unsubscribedDate to now.
  *******************************************************************************/
-public class UnsubscribeFromListProcess implements BackendStep, MetaDataProducerInterface<QProcessMetaData>
+public class UnsubscribeFromListProcess implements BackendStep
 {
    public static final String NAME = "crm_unsubscribe_from_list";
 
 
 
-   /*******************************************************************************
-    ** Produce the process metadata.
-    *******************************************************************************/
-   @Override
-   public QProcessMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QProcessMetaData()
-         .withName(NAME)
-         .withLabel("Unsubscribe from List")
-         .withIcon(new QIcon().withName("unsubscribe"))
-         .withStepList(List.of(
-            new QBackendStepMetaData()
-               .withName("execute")
-               .withCode(new QCodeReference(UnsubscribeFromListProcess.class))
-               .withInputData(new QFunctionInputMetaData().withFieldList(List.of(
-                  new QFieldMetaData("email", QFieldType.STRING).withIsRequired(true),
-                  new QFieldMetaData("emailListId", QFieldType.INTEGER).withIsRequired(true)
-               )))
-         ));
-   }
-
-
-
    /*******************************************************************************
     ** Execute the unsubscription.
     *******************************************************************************/

src/main/java/com/kingsrook/qbits/crm/deals/CrmDealNotifications.java

@@ -0,0 +1,108 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2026.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.crm.deals;
+
+
+import java.time.Instant;
+import java.time.temporal.ChronoUnit;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import com.kingsrook.qbits.crm.core.CrmRecords;
+import com.kingsrook.qbits.crm.scoring.JobState;
+import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
+import com.kingsrook.qqq.backend.core.alerts.NotificationAudience;
+import com.kingsrook.qqq.backend.core.alerts.NotificationSignal;
+import com.kingsrook.qqq.backend.core.alerts.QAlerts;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+
+
+/*******************************************************************************
+ ** Deal age notifications with a persistent once-per-deal-and-stage checkpoint.
+ *******************************************************************************/
+public final class CrmDealNotifications
+{
+   /*******************************************************************************
+    ** Utility class.
+    *******************************************************************************/
+   private CrmDealNotifications()
+   {
+   }
+
+   /*******************************************************************************
+    ** Scan open deals in keyset pages, resolving stage metadata once per run.
+    *******************************************************************************/
+   public static void checkRotting() throws QException
+   {
+      Map<Integer, QRecord> stages = new HashMap<>();
+      for(QRecord stage : new QueryAction().execute(new QueryInput("crm_pipeline_stage")).getRecords())
+      {
+         stages.put(stage.getValueInteger("id"), stage);
+      }
+      Integer lastId = null;
+      Instant now = Instant.now();
+      while(true)
+      {
+         QQueryFilter filter = new QQueryFilter().withOrderBy(new QFilterOrderBy("id")).withLimit(200);
+         if(lastId != null)
+         {
+            filter.withCriteria(new QFilterCriteria("id", QCriteriaOperator.GREATER_THAN, lastId));
+         }
+         List<QRecord> page = new QueryAction().execute(new QueryInput("crm_deal").withFilter(filter)).getRecords();
+         for(QRecord deal : page)
+         {
+            lastId = deal.getValueInteger("id");
+            QRecord stage = stages.get(deal.getValueInteger("pipelineStageId"));
+            Instant entered = deal.getValueInstant("stageEnteredDate");
+            if(stage == null || entered == null || deal.getValueString("ownerUserId") == null
+               || Boolean.TRUE.equals(stage.getValueBoolean("isClosedWon")) || Boolean.TRUE.equals(stage.getValueBoolean("isClosedLost")))
+            {
+               continue;
+            }
+            Integer days = stage.getValueInteger("rotDays");
+            if(days == null || days <= 0 || !entered.isBefore(now.minus(days, ChronoUnit.DAYS)))
+            {
+               continue;
+            }
+            String key = "deal:" + lastId + ":stage:" + stage.getValue("id");
+            String checkpoint = "crm.deal.rotting:" + key;
+            if(new QueryAction().execute(new QueryInput(JobState.TABLE_NAME)
+               .withFilter(new QQueryFilter(new QFilterCriteria("name", QCriteriaOperator.EQUALS, checkpoint)))).getRecords().isEmpty())
+            {
+               QAlerts.notify(new NotificationSignal("Deal needs attention").withSourceName("crm.deal.rotting").withSourceKey(key)
+                  .withSourceTableName("crm_deal").withSourceRecordId(lastId.toString())
+                  .withAudience(NotificationAudience.user(deal.getValueString("ownerUserId"))));
+               CrmRecords.insert(JobState.TABLE_NAME, new QRecord().withValue("name", checkpoint).withValue("lastSuccess", now));
+            }
+         }
+         if(page.size() < 200)
+         {
+            break;
+         }
+      }
+   }
+}

src/main/java/com/kingsrook/qbits/crm/deals/customizers/DealStageChangeCustomizer.java

@@ -51,6 +51,9 @@ import com.kingsrook.qbits.crm.sync.CrmCurrencyUtils;
 import com.kingsrook.qqq.backend.core.actions.customizers.AbstractPreUpdateCustomizer;
 import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
 import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
+import com.kingsrook.qqq.backend.core.alerts.NotificationAudience;
+import com.kingsrook.qqq.backend.core.alerts.NotificationSignal;
+import com.kingsrook.qqq.backend.core.alerts.QAlerts;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.logging.QLogger;
 import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetInput;
@@ -192,6 +195,14 @@ public class DealStageChangeCustomizer extends AbstractPreUpdateCustomizer
       // insert DealStageHistory row           //
       ///////////////////////////////////////////
       insertStageHistoryRow(record, oldRecord, oldStageId, newStageId, now);
+      String owner = record.getValues().containsKey("ownerUserId") ? record.getValueString("ownerUserId") : oldRecord.getValueString("ownerUserId");
+      if(Boolean.TRUE.equals(newStage.getIsClosedWon()) && owner != null)
+      {
+         QAlerts.notify(new NotificationSignal("Deal won").withSourceName("crm.deal.won")
+            .withSourceKey("deal:" + record.getValue("id") + ":stage:" + newStageId + ":" + now)
+            .withSourceTableName(Deal.TABLE_NAME).withSourceRecordId(record.getValueString("id"))
+            .withAudience(NotificationAudience.user(owner)), updateInput == null ? null : updateInput.getTransaction());
+      }
    }
 
 

src/main/java/com/kingsrook/qbits/crm/deals/model/Deal.java

@@ -33,6 +33,7 @@ import java.util.List;
 import com.kingsrook.qbits.crm.CrmQBitConfig;
 import com.kingsrook.qbits.crm.activities.model.Activity;
 import com.kingsrook.qbits.crm.core.model.Company;
+import com.kingsrook.qbits.crm.core.model.Contact;
 import com.kingsrook.qbits.crm.core.model.LeadSource;
 import com.kingsrook.qbits.crm.core.model.enums.CrmDealPriority;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
@@ -118,7 +119,7 @@ public class Deal extends QRecordEntity
             .withRecordLabelFields("name")
             .withSection(t1Section)
             .withSection(new QFieldSection("details", new QIcon().withName("info"), Tier.T2,
-               List.of("companyId", "ownerUserId", "leadSourceId", "priority",
+               List.of("primaryContactId", "companyId", "ownerUserId", "leadSourceId", "priority",
                   "probabilityOverridePct", "expectedCloseDate", "actualCloseDate",
                   "winLossReasonId", "weightedAmount", "amountInBaseCurrency")))
             .withSection(SectionFactory.defaultT2("description"))
@@ -191,7 +192,10 @@ public class Deal extends QRecordEntity
    @QField(possibleValueSourceName = Company.TABLE_NAME)
    private Integer companyId;
 
-   @QField(isRequired = true)
+   @QField(possibleValueSourceName = Contact.TABLE_NAME)
+   private Integer primaryContactId;
+
+   @QField()
    private String ownerUserId;
 
    @QField(possibleValueSourceName = LeadSource.TABLE_NAME)
@@ -894,4 +898,32 @@ public class Deal extends QRecordEntity
       return (this);
    }
 
+
+   /*******************************************************************************
+    ** The contact primarily responsible for this opportunity.
+    *******************************************************************************/
+   public Integer getPrimaryContactId()
+   {
+      return (primaryContactId);
+   }
+
+
+   /*******************************************************************************
+    ** Set the primary contact.
+    *******************************************************************************/
+   public void setPrimaryContactId(Integer primaryContactId)
+   {
+      this.primaryContactId = primaryContactId;
+   }
+
+
+   /*******************************************************************************
+    ** Set the primary contact fluently.
+    *******************************************************************************/
+   public Deal withPrimaryContactId(Integer primaryContactId)
+   {
+      this.primaryContactId = primaryContactId;
+      return (this);
+   }
+
 }

src/main/java/com/kingsrook/qbits/crm/deals/processes/CloneDealProcess.java

@@ -29,7 +29,6 @@ package com.kingsrook.qbits.crm.deals.processes;
 
 
 import java.time.Instant;
-import java.util.List;
 import com.kingsrook.qbits.crm.deals.model.Deal;
 import com.kingsrook.qbits.crm.deals.model.DealContact;
 import com.kingsrook.qbits.crm.deals.model.DealProduct;
@@ -50,49 +49,17 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
-import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 
 
 /*******************************************************************************
- ** MetaDataProducer + BackendStep for cloning a deal and its child records.
+ ** Backend operation used by the streamed CRM actions for cloning a deal and its child records.
  *******************************************************************************/
-public class CloneDealProcess implements BackendStep, MetaDataProducerInterface<QProcessMetaData>
+public class CloneDealProcess implements BackendStep
 {
    public static final String NAME = "crm_clone_deal";
 
 
 
-   /*******************************************************************************
-    ** Produce the process metadata.
-    *******************************************************************************/
-   @Override
-   public QProcessMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QProcessMetaData()
-         .withName(NAME)
-         .withLabel("Clone Deal")
-         .withTableName(Deal.TABLE_NAME)
-         .withIcon(new QIcon().withName("content_copy"))
-         .withStepList(List.of(
-            new QBackendStepMetaData()
-               .withName("execute")
-               .withCode(new QCodeReference(CloneDealProcess.class))
-               .withInputData(new QFunctionInputMetaData().withFieldList(List.of(
-                  new QFieldMetaData("dealId", QFieldType.INTEGER).withIsRequired(true)
-               )))
-         ));
-   }
-
-
-
    /*******************************************************************************
     ** Execute the clone deal process.
     *******************************************************************************/
@@ -127,6 +94,7 @@ public class CloneDealProcess implements BackendStep, MetaDataProducerInterface<
          .withValue("amount", sourceDeal.getAmount())
          .withValue("currencyCode", sourceDeal.getCurrencyCode())
          .withValue("companyId", sourceDeal.getCompanyId())
+         .withValue("primaryContactId", sourceDeal.getPrimaryContactId())
          .withValue("ownerUserId", sourceDeal.getOwnerUserId())
          .withValue("leadSourceId", sourceDeal.getLeadSourceId())
          .withValue("expectedCloseDate", sourceDeal.getExpectedCloseDate())

src/main/java/com/kingsrook/qbits/crm/deals/processes/ConvertLeadProcess.java

@@ -30,7 +30,6 @@ package com.kingsrook.qbits.crm.deals.processes;
 
 import java.math.BigDecimal;
 import java.time.Instant;
-import java.util.List;
 import com.kingsrook.qbits.crm.core.model.Contact;
 import com.kingsrook.qbits.crm.core.model.LifecycleStage;
 import com.kingsrook.qbits.crm.deals.model.Deal;
@@ -57,55 +56,20 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
-import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 import com.kingsrook.qqq.backend.core.utils.ValueUtils;
 
 
 /*******************************************************************************
- ** MetaDataProducer + BackendStep for converting a lead (contact) into a
+ ** Backend operation used by the streamed CRM actions for converting a lead (contact) into a
  ** deal opportunity. Transitions the contact's lifecycle stage, creates
  ** a deal with the first pipeline stage, and links them via DealContact.
  *******************************************************************************/
-public class ConvertLeadProcess implements BackendStep, MetaDataProducerInterface<QProcessMetaData>
+public class ConvertLeadProcess implements BackendStep
 {
    public static final String NAME = "crm_convert_lead";
 
 
 
-   /*******************************************************************************
-    ** Produce the process metadata.
-    *******************************************************************************/
-   @Override
-   public QProcessMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QProcessMetaData()
-         .withName(NAME)
-         .withLabel("Convert Lead")
-         .withTableName(Contact.TABLE_NAME)
-         .withIcon(new QIcon().withName("trending_up"))
-         .withStepList(List.of(
-            new QBackendStepMetaData()
-               .withName("execute")
-               .withCode(new QCodeReference(ConvertLeadProcess.class))
-               .withInputData(new QFunctionInputMetaData().withFieldList(List.of(
-                  new QFieldMetaData("contactId", QFieldType.INTEGER).withIsRequired(true),
-                  new QFieldMetaData("pipelineId", QFieldType.INTEGER),
-                  new QFieldMetaData("dealName", QFieldType.STRING).withIsRequired(true),
-                  new QFieldMetaData("dealAmount", QFieldType.DECIMAL)
-               )))
-         ));
-   }
-
-
-
    /*******************************************************************************
     ** Execute the lead conversion process.
     *******************************************************************************/

src/main/java/com/kingsrook/qbits/crm/deals/processes/ManageContactOnDealProcess.java

@@ -33,6 +33,7 @@ import com.kingsrook.qbits.crm.deals.model.Deal;
 import com.kingsrook.qbits.crm.deals.model.DealContact;
 import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
 import com.kingsrook.qqq.backend.core.actions.tables.DeleteAction;
+import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
 import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
 import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
 import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
@@ -49,55 +50,19 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
-import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 
 
 /*******************************************************************************
- ** MetaDataProducer + BackendStep for adding or removing a contact on a deal.
+ ** Backend operation used by the streamed CRM actions for adding or removing a contact on a deal.
  ** ADD: inserts a DealContact row, with isPrimary enforcement.
  ** REMOVE: deletes the DealContact row matching dealId + contactId.
  *******************************************************************************/
-public class ManageContactOnDealProcess implements BackendStep, MetaDataProducerInterface<QProcessMetaData>
+public class ManageContactOnDealProcess implements BackendStep
 {
    public static final String NAME = "crm_manage_contact_on_deal";
 
 
 
-   /*******************************************************************************
-    ** Produce the process metadata.
-    *******************************************************************************/
-   @Override
-   public QProcessMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QProcessMetaData()
-         .withName(NAME)
-         .withLabel("Manage Contact on Deal")
-         .withTableName(Deal.TABLE_NAME)
-         .withIcon(new QIcon().withName("person_add"))
-         .withStepList(List.of(
-            new QBackendStepMetaData()
-               .withName("execute")
-               .withCode(new QCodeReference(ManageContactOnDealProcess.class))
-               .withInputData(new QFunctionInputMetaData().withFieldList(List.of(
-                  new QFieldMetaData("dealId", QFieldType.INTEGER).withIsRequired(true),
-                  new QFieldMetaData("contactId", QFieldType.INTEGER).withIsRequired(true),
-                  new QFieldMetaData("action", QFieldType.STRING).withIsRequired(true),
-                  new QFieldMetaData("contactRoleId", QFieldType.INTEGER),
-                  new QFieldMetaData("isPrimary", QFieldType.BOOLEAN)
-               )))
-         ));
-   }
-
-
-
    /*******************************************************************************
     ** Execute the manage contact process.
     *******************************************************************************/
@@ -131,6 +96,11 @@ public class ManageContactOnDealProcess implements BackendStep, MetaDataProducer
 
          new InsertAction().execute(
             new InsertInput(DealContact.TABLE_NAME).withRecordEntity(dealContact));
+         if(Boolean.TRUE.equals(isPrimary))
+         {
+            new UpdateAction().execute(new UpdateInput(Deal.TABLE_NAME).withRecord(new QRecord()
+               .withValue("id", dealId).withValue("primaryContactId", contactId)));
+         }
       }
       else if("REMOVE".equalsIgnoreCase(action))
       {
@@ -156,6 +126,12 @@ public class ManageContactOnDealProcess implements BackendStep, MetaDataProducer
          deleteInput.setTableName(DealContact.TABLE_NAME);
          deleteInput.setPrimaryKeys(idsToDelete.stream().map(id -> (java.io.Serializable) id).toList());
          new DeleteAction().execute(deleteInput);
+         QRecord deal = GetAction.execute(Deal.TABLE_NAME, dealId);
+         if(deal != null && contactId.equals(deal.getValueInteger("primaryContactId")))
+         {
+            new UpdateAction().execute(new UpdateInput(Deal.TABLE_NAME).withRecord(new QRecord()
+               .withValue("id", dealId).withValue("primaryContactId", null)));
+         }
       }
       else
       {

src/main/java/com/kingsrook/qbits/crm/deals/processes/MoveDealStageProcess.java

@@ -50,55 +50,22 @@ import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutp
 import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
-import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 import com.kingsrook.qqq.backend.core.utils.JsonUtils;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
 
 
 /*******************************************************************************
- ** MetaDataProducer + BackendStep for moving a deal to a new pipeline stage.
+ ** Backend operation used by the streamed CRM actions for moving a deal to a new pipeline stage.
  ** Enforces stage-belongs-to-pipeline validation, required field gating,
  ** weighted amount recalculation, close-date stamping, and stage history
  ** tracking.
  *******************************************************************************/
-public class MoveDealStageProcess implements BackendStep, MetaDataProducerInterface<QProcessMetaData>
+public class MoveDealStageProcess implements BackendStep
 {
    public static final String NAME = "crm_move_deal_stage";
 
 
 
-   /*******************************************************************************
-    ** Produce the process metadata.
-    *******************************************************************************/
-   @Override
-   public QProcessMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QProcessMetaData()
-         .withName(NAME)
-         .withLabel("Move Deal Stage")
-         .withTableName(Deal.TABLE_NAME)
-         .withIcon(new QIcon().withName("swap_horiz"))
-         .withStepList(List.of(
-            new QBackendStepMetaData()
-               .withName("execute")
-               .withCode(new QCodeReference(MoveDealStageProcess.class))
-               .withInputData(new QFunctionInputMetaData().withFieldList(List.of(
-                  new QFieldMetaData("dealId", QFieldType.INTEGER).withIsRequired(true),
-                  new QFieldMetaData("targetStageId", QFieldType.INTEGER).withIsRequired(true)
-               )))
-         ));
-   }
-
-
-
    /*******************************************************************************
     ** Execute the stage move process.
     *******************************************************************************/

src/main/java/com/kingsrook/qbits/crm/deals/processes/ReopenDealProcess.java

@@ -31,7 +31,6 @@ package com.kingsrook.qbits.crm.deals.processes;
 import java.math.BigDecimal;
 import java.time.Instant;
 import java.time.temporal.ChronoUnit;
-import java.util.List;
 import com.kingsrook.qbits.crm.deals.model.Deal;
 import com.kingsrook.qbits.crm.deals.model.DealStageHistory;
 import com.kingsrook.qbits.crm.deals.model.PipelineStage;
@@ -47,52 +46,19 @@ import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutp
 import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
-import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 
 
 /*******************************************************************************
- ** MetaDataProducer + BackendStep for reopening a closed deal by moving it
+ ** Backend operation used by the streamed CRM actions for reopening a closed deal by moving it
  ** to an active pipeline stage. Validates the deal is currently closed, the
  ** target stage is not closed, clears close data, and logs the transition.
  *******************************************************************************/
-public class ReopenDealProcess implements BackendStep, MetaDataProducerInterface<QProcessMetaData>
+public class ReopenDealProcess implements BackendStep
 {
    public static final String NAME = "crm_reopen_deal";
 
 
 
-   /*******************************************************************************
-    ** Produce the process metadata.
-    *******************************************************************************/
-   @Override
-   public QProcessMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QProcessMetaData()
-         .withName(NAME)
-         .withLabel("Reopen Deal")
-         .withTableName(Deal.TABLE_NAME)
-         .withIcon(new QIcon().withName("replay"))
-         .withStepList(List.of(
-            new QBackendStepMetaData()
-               .withName("execute")
-               .withCode(new QCodeReference(ReopenDealProcess.class))
-               .withInputData(new QFunctionInputMetaData().withFieldList(List.of(
-                  new QFieldMetaData("dealId", QFieldType.INTEGER).withIsRequired(true),
-                  new QFieldMetaData("targetStageId", QFieldType.INTEGER).withIsRequired(true)
-               )))
-         ));
-   }
-
-
-
    /*******************************************************************************
     ** Execute the reopen process.
     *******************************************************************************/

src/main/java/com/kingsrook/qbits/crm/deals/processes/TransferOwnershipProcess.java

@@ -27,7 +27,6 @@
 package com.kingsrook.qbits.crm.deals.processes;
 
 
-import java.util.List;
 import com.kingsrook.qbits.crm.core.model.Company;
 import com.kingsrook.qbits.crm.core.model.Contact;
 import com.kingsrook.qbits.crm.core.model.enums.CrmEntityType;
@@ -46,50 +45,18 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
-import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 
 
 /*******************************************************************************
- ** MetaDataProducer + BackendStep for transferring ownership of contacts,
+ ** Backend operation used by the streamed CRM actions for transferring ownership of contacts,
  ** companies, and deals from one user to another.
  *******************************************************************************/
-public class TransferOwnershipProcess implements BackendStep, MetaDataProducerInterface<QProcessMetaData>
+public class TransferOwnershipProcess implements BackendStep
 {
    public static final String NAME = "crm_transfer_ownership";
 
 
 
-   /*******************************************************************************
-    ** Produce the process metadata.
-    *******************************************************************************/
-   @Override
-   public QProcessMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QProcessMetaData()
-         .withName(NAME)
-         .withLabel("Transfer Ownership")
-         .withIcon(new QIcon().withName("swap_horiz"))
-         .withStepList(List.of(
-            new QBackendStepMetaData()
-               .withName("execute")
-               .withCode(new QCodeReference(TransferOwnershipProcess.class))
-               .withInputData(new QFunctionInputMetaData().withFieldList(List.of(
-                  new QFieldMetaData("fromUserId", QFieldType.STRING).withIsRequired(true),
-                  new QFieldMetaData("toUserId", QFieldType.STRING).withIsRequired(true)
-               )))
-         ));
-   }
-
-
-
    /*******************************************************************************
     ** Execute the transfer ownership process.
     *******************************************************************************/

src/main/java/com/kingsrook/qbits/crm/email/CrmEmailProcessIdentity.java

@@ -0,0 +1,55 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2026.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.crm.email;
+
+
+import com.kingsrook.qqq.backend.core.actions.processes.listeners.ProcessLifecycleListenerInterface;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessOutput;
+
+
+/*******************************************************************************
+ ** Carry the core process UUID across Review/Result and retries in process state.
+ *******************************************************************************/
+public class CrmEmailProcessIdentity implements ProcessLifecycleListenerInterface
+{
+   @Override
+   public boolean appliesTo(String processName)
+   {
+      return ("crm_send_email".equals(processName));
+   }
+
+   @Override
+   public void onProcessStarted(RunProcessInput input)
+   {
+      input.getProcessState().getValues().put("crmEmailProcessUUID", input.getProcessUUID());
+   }
+
+   @Override
+   public void onProcessCompleted(RunProcessInput input, RunProcessOutput output)
+   {
+   }
+
+   @Override
+   public void onProcessFailed(RunProcessInput input, Exception exception)
+   {
+   }
+}

src/main/java/com/kingsrook/qbits/crm/email/CrmEmailTransport.java

@@ -21,8 +21,11 @@
 package com.kingsrook.qbits.crm.email;
 
 
+import java.time.Instant;
+import java.util.Optional;
 import com.kingsrook.qbits.crm.CrmQBitConfig;
 import com.kingsrook.qbits.crm.api.CrmReadiness;
+import com.kingsrook.qbits.crm.core.model.enums.CrmEmailStatus;
 import com.kingsrook.qbits.email.api.EmailApi;
 import com.kingsrook.qbits.email.api.EmailApiCategory;
 import com.kingsrook.qbits.email.api.EmailApiEffectOutcome;
@@ -87,6 +90,14 @@ public class CrmEmailTransport
     ** Deliver one message, using a stable CRM purpose/id key for the durable email outbox.
     *******************************************************************************/
    public void send(String purpose, String id, Integer contactId, String address, String subject, String body, boolean marketing) throws QException
+   {
+      deliver(purpose, id, contactId, address, subject, body, marketing);
+   }
+
+   /*******************************************************************************
+    ** Return the transport result without guessing that queued email was sent.
+    *******************************************************************************/
+   private Delivery deliver(String purpose, String id, Integer contactId, String address, String subject, String body, boolean marketing) throws QException
    {
       if(emailInstalled())
       {
@@ -98,7 +109,9 @@ public class CrmEmailTransport
          {
             throw (new QUserFacingException("The email service could not accept this message."));
          }
-         return;
+         var message = result.message();
+         return (message == null ? new Delivery(null, CrmEmailStatus.QUEUED, null, null, null, null, null)
+            : new Delivery(String.valueOf(message.id()), status(message.status()), message.sentDate(), message.deliveredDate(), null, null, null));
       }
       if(readiness() != CrmReadiness.CORE_MESSAGING)
       {
@@ -107,5 +120,52 @@ public class CrmEmailTransport
       new SendMessageAction().execute(new SendMessageInput().withMessagingProviderName(config.getMessagingProviderName())
          .withTo(new Party().withAddress(address).withRole(EmailPartyRole.TO)).withSubject(subject)
          .withContent(new Content().withBody(body).withContentRole(EmailContentRole.TEXT)));
+      return (new Delivery(null, CrmEmailStatus.SENT, Instant.now(), null, null, null, null));
+   }
+
+   /*******************************************************************************
+    ** CRM-owned delivery snapshot; optional email types never escape this adapter.
+    *******************************************************************************/
+   public record Delivery(String messageId, CrmEmailStatus status, Instant sentDate, Instant deliveredDate,
+                          Instant openedDate, Instant clickedDate, Instant bouncedDate)
+   {
+   }
+
+   /*******************************************************************************
+    ** Send and return the actual outbox state, or the successful core send state.
+    *******************************************************************************/
+   public Delivery sendMessage(String purpose, String id, Integer contactId, String address, String subject, String body, boolean marketing) throws QException
+   {
+      send(purpose, id, contactId, address, subject, body, marketing);
+      return (lookup("crm:" + purpose + ":" + id).orElseGet(() ->
+         new Delivery(null, CrmEmailStatus.SENT, Instant.now(), null, null, null, null)));
+   }
+
+   /*******************************************************************************
+    ** Read delivery through the published API only. RC.3 exposes sent/delivered dates.
+    *******************************************************************************/
+   public Optional<Delivery> lookup(String key) throws QException
+   {
+      if(!emailInstalled() || key == null)
+      {
+         return (Optional.empty());
+      }
+      return (EmailApi.getMessageByIdempotencyKey(key).map(message -> new Delivery(String.valueOf(message.id()),
+         status(message.status()), message.sentDate(), message.deliveredDate(), null, null, null)));
+   }
+
+   /*******************************************************************************
+    ** Preserve known delivery states and leave unknown/pending states eligible for sync.
+    *******************************************************************************/
+   private CrmEmailStatus status(String status)
+   {
+      try
+      {
+         return (CrmEmailStatus.valueOf(status));
+      }
+      catch(IllegalArgumentException | NullPointerException e)
+      {
+         return (CrmEmailStatus.QUEUED);
+      }
    }
 }

src/main/java/com/kingsrook/qbits/crm/email/CrmEmails.java

@@ -0,0 +1,167 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2026.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.crm.email;
+
+
+import java.util.List;
+import java.util.Map;
+import com.kingsrook.qbits.crm.api.ActivityKind;
+import com.kingsrook.qbits.crm.api.CrmMarketingApi;
+import com.kingsrook.qbits.crm.api.CrmReadiness;
+import com.kingsrook.qbits.crm.core.CrmRecords;
+import com.kingsrook.qbits.crm.core.model.enums.CrmDirection;
+import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
+import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.exceptions.QUserFacingException;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+
+
+/*******************************************************************************
+ ** Shared CRM template rendering, transport delivery and timeline recording.
+ *******************************************************************************/
+public final class CrmEmails
+{
+   /*******************************************************************************
+    ** Utility class.
+    *******************************************************************************/
+   private CrmEmails()
+   {
+   }
+
+   /*******************************************************************************
+    ** A skip reason suitable for standard WARNING summary lines.
+    *******************************************************************************/
+   public static String skipReason(QRecord contact, CrmEmailTransport transport) throws QException
+   {
+      if(Boolean.TRUE.equals(contact.getValueBoolean("doNotEmail")))
+      {
+         return ("Contact has opted out of email");
+      }
+      if(contact.getValueString("email") == null || contact.getValueString("email").isBlank())
+      {
+         return ("Contact has no email address");
+      }
+      return (transport.readiness() == CrmReadiness.NOT_CONFIGURED ? "CRM email is not configured" : null);
+   }
+
+   /*******************************************************************************
+    ** Send once using the caller's stable purpose/id and record the transport outcome.
+    *******************************************************************************/
+   public static void send(CrmEmailTransport transport, QRecord contact, Integer templateId, String subject, String body,
+                           String purpose, String id, Integer enrollmentId) throws QException
+   {
+      String key = "crm:" + purpose + ":" + id;
+      List<QRecord> existing = new QueryAction().execute(new QueryInput("crm_email_message").withFilter(
+         new QQueryFilter(new QFilterCriteria("idempotencyKey", QCriteriaOperator.EQUALS, key)))).getRecords();
+      if(!existing.isEmpty())
+      {
+         recordActivity(existing.getFirst());
+         return;
+      }
+      if(templateId != null)
+      {
+         QRecord template = GetAction.execute("crm_email_template", templateId);
+         if(template == null)
+         {
+            throw (new QUserFacingException("Email template is unavailable"));
+         }
+         subject = subject == null || subject.isBlank() ? template.getValueString("subject") : subject;
+         body = body == null || body.isBlank() ? template.getValueString("bodyText") : body;
+         if(body == null)
+         {
+            body = template.getValueString("bodyHtml");
+         }
+      }
+      if(subject == null || subject.isBlank() || body == null || body.isBlank())
+      {
+         throw (new QUserFacingException("Email subject and body are required"));
+      }
+      subject = render(subject, contact);
+      body = render(body, contact);
+      CrmEmailTransport.Delivery delivery = transport.sendMessage(purpose, id, contact.getValueInteger("id"),
+         contact.getValueString("email"), subject, body, enrollmentId != null);
+      QRecord message = CrmRecords.insert("crm_email_message", new QRecord().withValue("contactId", contact.getValue("id"))
+         .withValue("emailTemplateId", templateId).withValue("sequenceEnrollmentId", enrollmentId)
+         .withValue("toAddresses", contact.getValue("email")).withValue("subject", subject)
+         .withValue("direction", CrmDirection.OUTBOUND.getId()).withValue("status", delivery.status().getId())
+         .withValue("idempotencyKey", key).withValue("emailMessageId", delivery.messageId())
+         .withValue("sentDate", delivery.sentDate()).withValue("deliveredDate", delivery.deliveredDate()));
+      recordActivity(message);
+   }
+
+   /*******************************************************************************
+    ** Repair a partially recorded send without sending again or duplicating its activity.
+    *******************************************************************************/
+   private static void recordActivity(QRecord message) throws QException
+   {
+      if(message.getValue("activityId") != null)
+      {
+         return;
+      }
+      String description = "Email: " + message.getValueString("idempotencyKey");
+      QQueryFilter filter = new QQueryFilter(new QFilterCriteria("contactId", QCriteriaOperator.EQUALS, message.getValue("contactId")),
+         new QFilterCriteria("description", QCriteriaOperator.EQUALS, description));
+      List<QRecord> activities = new QueryAction().execute(new QueryInput("crm_activity").withFilter(filter)).getRecords();
+      if(activities.isEmpty())
+      {
+         new CrmMarketingApi().recordActivity(message.getValueInteger("contactId"),
+            message.getValue("sequenceEnrollmentId") == null ? ActivityKind.EMAIL : ActivityKind.SEQUENCE_STEP,
+            message.getValueString("subject"), description);
+         activities = new QueryAction().execute(new QueryInput("crm_activity").withFilter(filter)).getRecords();
+      }
+      CrmRecords.update("crm_email_message", new QRecord().withValue("id", message.getValue("id"))
+         .withValue("activityId", activities.getFirst().getValue("id")));
+   }
+
+   /*******************************************************************************
+    ** Literal replacements avoid treating customer data as a template program.
+    *******************************************************************************/
+   public static String render(String text, QRecord contact) throws QException
+   {
+      QRecord company = contact.getValue("companyId") == null ? null : GetAction.execute("crm_company", contact.getValue("companyId"));
+      Map<String, String> fields = Map.of("contact.firstName", value(contact, "firstName"), "contact.lastName", value(contact, "lastName"),
+         "company.name", value(company, "name"), "owner.name", value(contact, "ownerUserId"));
+      // Replace each token in the original text only, so inserted values cannot introduce merge fields.
+      java.util.regex.Matcher matcher = java.util.regex.Pattern.compile("\\{\\{(contact.firstName|contact.lastName|company.name|owner.name|unsubscribeUrl)\\}\\}").matcher(text);
+      StringBuilder rendered = new StringBuilder();
+      while(matcher.find())
+      {
+         String replacement = "unsubscribeUrl".equals(matcher.group(1))
+            ? new CrmMarketingApi().buildUnsubscribeUrl(contact.getValueInteger("id"), null) : fields.get(matcher.group(1));
+         matcher.appendReplacement(rendered, java.util.regex.Matcher.quoteReplacement(replacement));
+      }
+      matcher.appendTail(rendered);
+      return (rendered.toString());
+   }
+
+   /*******************************************************************************
+    ** Missing merge values render as empty strings.
+    *******************************************************************************/
+   private static String value(QRecord row, String field)
+   {
+      return (row == null || row.getValueString(field) == null ? "" : row.getValueString(field));
+   }
+}

src/main/java/com/kingsrook/qbits/crm/email/model/EmailMessage.java

@@ -29,6 +29,7 @@ package com.kingsrook.qbits.crm.email.model;
 import java.time.Instant;
 import java.util.List;
 import com.kingsrook.qbits.crm.activities.model.Activity;
+import com.kingsrook.qbits.crm.core.model.Contact;
 import com.kingsrook.qbits.crm.core.model.enums.CrmBounceType;
 import com.kingsrook.qbits.crm.core.model.enums.CrmDirection;
 import com.kingsrook.qbits.crm.core.model.enums.CrmEmailStatus;
@@ -45,6 +46,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.tables.QFieldSection;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.SectionFactory;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.Tier;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.UniqueKey;
 
 
 /*******************************************************************************
@@ -74,6 +76,7 @@ public class EmailMessage extends QRecordEntity
       public QTableMetaData customizeMetaData(QInstance qInstance, QTableMetaData table) throws QException
       {
          table
+            .withUniqueKey(new UniqueKey("idempotencyKey"))
             .withIcon(new QIcon().withName("email"))
             .withRecordLabelFormat("%s")
             .withRecordLabelFields("subject")
@@ -84,10 +87,10 @@ public class EmailMessage extends QRecordEntity
                List.of("bodyHtml", "bodyText")))
             .withSection(new QFieldSection("tracking", new QIcon("visibility"), Tier.T2,
                List.of("isTracked", "openCount", "clickCount", "firstOpenedDate", "firstClickedDate",
-                  "bounceType", "isSpamComplaint")))
+                  "bounceType", "isSpamComplaint", "deliveredDate", "bouncedDate")))
             .withSection(new QFieldSection("references", new QIcon("link"), Tier.T2,
-               List.of("activityId", "emailTemplateId", "sequenceEnrollmentId",
-                  "threadId", "replyToEmailId")))
+               List.of("contactId", "activityId", "emailTemplateId", "sequenceEnrollmentId",
+                  "threadId", "replyToEmailId", "idempotencyKey", "emailMessageId")))
             .withSection(SectionFactory.defaultT3("createDate", "modifyDate"));
 
          return (table);
@@ -102,7 +105,10 @@ public class EmailMessage extends QRecordEntity
    @QField(possibleValueSourceName = Activity.TABLE_NAME)
    private Integer activityId;
 
-   @QField(isRequired = true, maxLength = 250, valueTooLongBehavior = ValueTooLongBehavior.ERROR)
+   @QField(possibleValueSourceName = Contact.TABLE_NAME)
+   private Integer contactId;
+
+   @QField(maxLength = 250, valueTooLongBehavior = ValueTooLongBehavior.ERROR)
    private String fromAddress;
 
    @QField(maxLength = 100, valueTooLongBehavior = ValueTooLongBehavior.ERROR)
@@ -168,6 +174,18 @@ public class EmailMessage extends QRecordEntity
    @QField()
    private Integer replyToEmailId;
 
+   @QField(maxLength = 250)
+   private String idempotencyKey;
+
+   @QField(maxLength = 100)
+   private String emailMessageId;
+
+   @QField()
+   private Instant deliveredDate;
+
+   @QField()
+   private Instant bouncedDate;
+
    @QField(isEditable = false)
    private Instant createDate;
 
@@ -999,4 +1017,131 @@ public class EmailMessage extends QRecordEntity
       return (this);
    }
 
+
+   /*******************************************************************************
+    ** Contact whose record includes this message.
+    *******************************************************************************/
+   public Integer getContactId()
+   {
+      return (contactId);
+   }
+
+
+   /*******************************************************************************
+    ** Set the message's contact.
+    *******************************************************************************/
+   public void setContactId(Integer contactId)
+   {
+      this.contactId = contactId;
+   }
+
+
+   /*******************************************************************************
+    ** Set the message's contact fluently.
+    *******************************************************************************/
+   public EmailMessage withContactId(Integer contactId)
+   {
+      this.contactId = contactId;
+      return (this);
+   }
+
+   /*******************************************************************************
+    ** Get idempotencyKey.
+    *******************************************************************************/
+   public String getIdempotencyKey()
+   {
+      return (idempotencyKey);
+   }
+
+   /*******************************************************************************
+    ** Set idempotencyKey.
+    *******************************************************************************/
+   public EmailMessage withIdempotencyKey(String idempotencyKey)
+   {
+      this.idempotencyKey = idempotencyKey;
+      return (this);
+   }
+
+   /*******************************************************************************
+    ** Get emailMessageId.
+    *******************************************************************************/
+   public String getEmailMessageId()
+   {
+      return (emailMessageId);
+   }
+
+   /*******************************************************************************
+    ** Set emailMessageId.
+    *******************************************************************************/
+   public EmailMessage withEmailMessageId(String emailMessageId)
+   {
+      this.emailMessageId = emailMessageId;
+      return (this);
+   }
+
+   /*******************************************************************************
+    ** Get deliveredDate.
+    *******************************************************************************/
+   public Instant getDeliveredDate()
+   {
+      return (deliveredDate);
+   }
+
+   /*******************************************************************************
+    ** Set deliveredDate.
+    *******************************************************************************/
+   public EmailMessage withDeliveredDate(Instant deliveredDate)
+   {
+      this.deliveredDate = deliveredDate;
+      return (this);
+   }
+
+   /*******************************************************************************
+    ** Get bouncedDate.
+    *******************************************************************************/
+   public Instant getBouncedDate()
+   {
+      return (bouncedDate);
+   }
+
+   /*******************************************************************************
+    ** Set bouncedDate.
+    *******************************************************************************/
+   public EmailMessage withBouncedDate(Instant bouncedDate)
+   {
+      this.bouncedDate = bouncedDate;
+      return (this);
+   }
+
+   /*******************************************************************************
+    ** Set idempotencyKey for entity mapping.
+    *******************************************************************************/
+   public void setIdempotencyKey(String idempotencyKey)
+   {
+      this.idempotencyKey = idempotencyKey;
+   }
+
+   /*******************************************************************************
+    ** Set emailMessageId for entity mapping.
+    *******************************************************************************/
+   public void setEmailMessageId(String emailMessageId)
+   {
+      this.emailMessageId = emailMessageId;
+   }
+
+   /*******************************************************************************
+    ** Set deliveredDate for entity mapping.
+    *******************************************************************************/
+   public void setDeliveredDate(Instant deliveredDate)
+   {
+      this.deliveredDate = deliveredDate;
+   }
+
+   /*******************************************************************************
+    ** Set bouncedDate for entity mapping.
+    *******************************************************************************/
+   public void setBouncedDate(Instant bouncedDate)
+   {
+      this.bouncedDate = bouncedDate;
+   }
 }

src/main/java/com/kingsrook/qbits/crm/email/model/SequenceEnrollment.java

@@ -28,6 +28,7 @@ package com.kingsrook.qbits.crm.email.model;
 
 import java.time.Instant;
 import java.util.List;
+import com.kingsrook.qbits.crm.core.model.Contact;
 import com.kingsrook.qbits.crm.core.model.enums.CrmEnrollmentStatus;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.model.data.QField;
@@ -97,7 +98,7 @@ public class SequenceEnrollment extends QRecordEntity
    @QField(isRequired = true, possibleValueSourceName = EmailSequence.TABLE_NAME)
    private Integer sequenceId;
 
-   @QField(isRequired = true)
+   @QField(isRequired = true, possibleValueSourceName = Contact.TABLE_NAME)
    private Integer contactId;
 
    @QField()

src/main/java/com/kingsrook/qbits/crm/email/processes/EnrollInSequenceProcess.java

@@ -28,7 +28,6 @@ package com.kingsrook.qbits.crm.email.processes;
 
 import java.time.Instant;
 import java.time.temporal.ChronoUnit;
-import java.util.List;
 import com.kingsrook.qbits.crm.core.model.Contact;
 import com.kingsrook.qbits.crm.core.model.enums.CrmEnrollmentStatus;
 import com.kingsrook.qbits.crm.email.model.EmailSequence;
@@ -51,52 +50,19 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
-import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 
 
 /*******************************************************************************
- ** MetaDataProducer + BackendStep to enroll a contact in an email sequence.
+ ** Backend operation used by the streamed CRM actions to enroll a contact in an email sequence.
  ** Validates that the contact is not already enrolled, the sequence is active,
  ** and the contact has not opted out of email.
  *******************************************************************************/
-public class EnrollInSequenceProcess implements BackendStep, MetaDataProducerInterface<QProcessMetaData>
+public class EnrollInSequenceProcess implements BackendStep
 {
    public static final String NAME = "crm_enroll_in_sequence";
 
 
 
-   /*******************************************************************************
-    ** Produce the process metadata.
-    *******************************************************************************/
-   @Override
-   public QProcessMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QProcessMetaData()
-         .withName(NAME)
-         .withLabel("Enroll in Sequence")
-         .withIcon(new QIcon().withName("playlist_add"))
-         .withStepList(List.of(
-            new QBackendStepMetaData()
-               .withName("execute")
-               .withCode(new QCodeReference(EnrollInSequenceProcess.class))
-               .withInputData(new QFunctionInputMetaData().withFieldList(List.of(
-                  new QFieldMetaData("sequenceId", QFieldType.INTEGER).withIsRequired(true),
-                  new QFieldMetaData("contactId", QFieldType.INTEGER).withIsRequired(true),
-                  new QFieldMetaData("dealId", QFieldType.INTEGER)
-               )))
-         ));
-   }
-
-
-
    /*******************************************************************************
     ** Execute the enrollment logic.
     *******************************************************************************/

src/main/java/com/kingsrook/qbits/crm/email/processes/ExpireConsentsProcess.java

@@ -0,0 +1,104 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2026.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.crm.email.processes;
+
+
+import java.time.Instant;
+import java.time.temporal.ChronoUnit;
+import java.util.List;
+import com.kingsrook.qbits.crm.core.CrmRecords;
+import com.kingsrook.qbits.crm.core.model.enums.CrmListMemberStatus;
+import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
+import com.kingsrook.qqq.backend.core.actions.tables.CountAction;
+import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.count.CountInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
+
+
+/*******************************************************************************
+ ** Scheduled CRM maintenance: crm_expire_consents.
+ *******************************************************************************/
+public class ExpireConsentsProcess implements BackendStep, MetaDataProducerInterface<QProcessMetaData>
+{
+   public static final String NAME = "crm_expire_consents";
+
+   @Override
+   public QProcessMetaData produce(QInstance instance)
+   {
+      return (new QProcessMetaData().withName(NAME).withLabel("ExpireConsents").withIsHidden(true)
+         .withStep(new QBackendStepMetaData().withName("execute").withCode(new QCodeReference(ExpireConsentsProcess.class))));
+   }
+
+   @Override
+   public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
+   {
+      Instant now = Instant.now();
+      Instant cutoff = now.minus(14, ChronoUnit.DAYS);
+      Integer lastId = null;
+      int expired = 0;
+      while(true)
+      {
+         QQueryFilter filter = new QQueryFilter(new QFilterCriteria("status", QCriteriaOperator.EQUALS, CrmListMemberStatus.PENDING_CONFIRMATION.getId()))
+            .withOrderBy(new QFilterOrderBy("id")).withLimit(200);
+         if(lastId != null)
+         {
+            filter.addCriteria(new QFilterCriteria("id", QCriteriaOperator.GREATER_THAN, lastId));
+         }
+         List<QRecord> page = new QueryAction().execute(new QueryInput("crm_email_list_member").withFilter(filter)).getRecords();
+         for(QRecord member : page)
+         {
+            lastId = member.getValueInteger("id");
+            Instant pendingSince = member.getValueInstant("confirmationSentDate");
+            if(pendingSince == null)
+            {
+               pendingSince = member.getValueInstant("createDate");
+            }
+            if(pendingSince != null && !pendingSince.isAfter(cutoff))
+            {
+               CrmRecords.update("crm_email_list_member", new QRecord().withValue("id", lastId)
+                  .withValue("status", CrmListMemberStatus.UNSUBSCRIBED.getId()).withValue("confirmationTokenHash", null)
+                  .withValue("unsubscribedDate", now));
+               expired++;
+            }
+         }
+         if(page.size() < 200)
+         {
+            break;
+         }
+      }
+      output.addValue("expiredPendingCount", expired);
+      output.addValue("expiredConsentCount", new CountAction().execute(new CountInput("crm_consent_record")
+         .withFilter(new QQueryFilter(new QFilterCriteria("expiryDate", QCriteriaOperator.LESS_THAN_OR_EQUALS, now)))).getCount());
+   }
+}

src/main/java/com/kingsrook/qbits/crm/email/processes/ProcessSequenceStepsProcess.java

@@ -34,9 +34,15 @@ import java.time.temporal.ChronoUnit;
 import java.util.List;
 import com.kingsrook.qbits.crm.activities.model.Activity;
 import com.kingsrook.qbits.crm.activities.model.ActivityType;
+import com.kingsrook.qbits.crm.api.CrmApi;
+import com.kingsrook.qbits.crm.api.CrmMarketingApi;
+import com.kingsrook.qbits.crm.api.CrmReadiness;
+import com.kingsrook.qbits.crm.core.CrmRecords;
 import com.kingsrook.qbits.crm.core.model.Contact;
 import com.kingsrook.qbits.crm.core.model.enums.CrmEnrollmentStatus;
 import com.kingsrook.qbits.crm.core.model.enums.CrmSequenceStepType;
+import com.kingsrook.qbits.crm.email.CrmEmailTransport;
+import com.kingsrook.qbits.crm.email.CrmEmails;
 import com.kingsrook.qbits.crm.email.model.EmailSequence;
 import com.kingsrook.qbits.crm.email.model.SequenceEnrollment;
 import com.kingsrook.qbits.crm.email.model.SequenceStep;
@@ -45,6 +51,8 @@ import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
 import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
 import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
 import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
+import com.kingsrook.qqq.backend.core.alerts.AlertSignal;
+import com.kingsrook.qqq.backend.core.alerts.QAlerts;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
 import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
@@ -53,6 +61,7 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetOutput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
@@ -78,6 +87,15 @@ public class ProcessSequenceStepsProcess implements BackendStep, MetaDataProduce
 
    private static final int MAX_FAILURES = 3;
 
+   /*******************************************************************************
+    ** Resolve the host transport; tests may supply a fake.
+    *******************************************************************************/
+   protected CrmEmailTransport emailTransport() throws QException
+   {
+      return (new CrmEmailTransport(CrmApi.config()));
+   }
+
+
 
 
    /*******************************************************************************
@@ -88,9 +106,9 @@ public class ProcessSequenceStepsProcess implements BackendStep, MetaDataProduce
    {
       return new QProcessMetaData()
          .withName(NAME)
+         .withIsHidden(true)
          .withLabel("Process Sequence Steps")
          .withIcon(new QIcon().withName("play_circle"))
-         // Scheduling is configured by the host application via ScheduledJob records
          .withStepList(List.of(
             new QBackendStepMetaData()
                .withName("execute")
@@ -109,29 +127,49 @@ public class ProcessSequenceStepsProcess implements BackendStep, MetaDataProduce
       ///////////////////////////////////////////////////////////
       // query enrollments where status=ACTIVE and due now     //
       ///////////////////////////////////////////////////////////
-      QueryOutput enrollmentQuery = new QueryAction().execute(
-         new QueryInput(SequenceEnrollment.TABLE_NAME)
-            .withFilter(new QQueryFilter()
-               .withCriteria(new QFilterCriteria("status", QCriteriaOperator.EQUALS, CrmEnrollmentStatus.ACTIVE.getPossibleValueId()))
-               .withCriteria(new QFilterCriteria("nextStepDate", QCriteriaOperator.LESS_THAN_OR_EQUALS, Instant.now()))));
-
-      int processedCount = 0;
-
-      for(QRecord enrollmentRecord : enrollmentQuery.getRecords())
+      CrmEmailTransport transport = emailTransport();
+      boolean ready = transport.readiness() != CrmReadiness.NOT_CONFIGURED;
+      boolean activeSequences = !new QueryAction().execute(new QueryInput(EmailSequence.TABLE_NAME)
+         .withFilter(new QQueryFilter(new QFilterCriteria("isActive", QCriteriaOperator.EQUALS, true)).withLimit(1))).getRecords().isEmpty();
+      if(!ready && activeSequences)
       {
-         SequenceEnrollment enrollment = new SequenceEnrollment(enrollmentRecord);
-
-         try
+         QAlerts.raise(new AlertSignal("crm.email.unavailable").withDedupKey("transport").withTitle("CRM email is unavailable"));
+      }
+      else if(ready)
+      {
+         QAlerts.resolve("crm.email.unavailable", "transport", "Email transport is ready");
+      }
+      int processedCount = 0;
+      Integer lastId = null;
+      Instant dueThrough = Instant.now();
+      while(true)
+      {
+         QQueryFilter filter = new QQueryFilter()
+            .withCriteria(new QFilterCriteria("status", QCriteriaOperator.EQUALS, CrmEnrollmentStatus.ACTIVE.getPossibleValueId()))
+            .withCriteria(new QFilterCriteria("nextStepDate", QCriteriaOperator.LESS_THAN_OR_EQUALS, dueThrough))
+            .withOrderBy(new QFilterOrderBy("id")).withLimit(200);
+         if(lastId != null)
          {
-            processEnrollment(enrollment);
-            processedCount++;
+            filter.withCriteria(new QFilterCriteria("id", QCriteriaOperator.GREATER_THAN, lastId));
          }
-         catch(Exception e)
+         List<QRecord> page = new QueryAction().execute(new QueryInput(SequenceEnrollment.TABLE_NAME).withFilter(filter)).getRecords();
+         for(QRecord enrollmentRecord : page)
          {
-            /////////////////////////////////////////////
-            // handle step failure                     //
-            /////////////////////////////////////////////
-            handleFailure(enrollment, e.getMessage());
+            SequenceEnrollment enrollment = new SequenceEnrollment(enrollmentRecord);
+            try
+            {
+               processEnrollment(enrollment, transport);
+               processedCount++;
+            }
+            catch(Exception e)
+            {
+               handleFailure(enrollment, e.getMessage());
+            }
+            lastId = enrollment.getId();
+         }
+         if(page.size() < 200)
+         {
+            break;
          }
       }
 
@@ -144,7 +182,7 @@ public class ProcessSequenceStepsProcess implements BackendStep, MetaDataProduce
     ** Process a single enrollment: load the next step and execute it.
     ** Respects businessDaysOnly and sendWindow settings on the sequence.
     ***************************************************************************/
-   private void processEnrollment(SequenceEnrollment enrollment) throws QException
+   private void processEnrollment(SequenceEnrollment enrollment, CrmEmailTransport transport) throws QException
    {
       Integer nextStepNumber = enrollment.getCurrentStepNumber() + 1;
 
@@ -157,6 +195,11 @@ public class ProcessSequenceStepsProcess implements BackendStep, MetaDataProduce
       EmailSequence sequence = sequenceGet.getRecord() != null
          ? new EmailSequence(sequenceGet.getRecord()) : null;
 
+      if(sequence == null || !Boolean.TRUE.equals(sequence.getIsActive()))
+      {
+         return;
+      }
+
       /////////////////////////////////////////////
       // load the contact for timezone           //
       /////////////////////////////////////////////
@@ -227,7 +270,27 @@ public class ProcessSequenceStepsProcess implements BackendStep, MetaDataProduce
       {
          switch(stepType)
          {
-            case EMAIL -> executeEmailStep(enrollment, step);
+            case EMAIL ->
+            {
+               var eligibility = new CrmMarketingApi().checkEmailEligibility(enrollment.getContactId());
+               if(!eligibility.eligible())
+               {
+                  CrmRecords.update(SequenceEnrollment.TABLE_NAME, new QRecord().withValue("id", enrollment.getId())
+                     .withValue("status", CrmEnrollmentStatus.PAUSED.getId()).withValue("lastFailureMessage", eligibility.reason())
+                     .withValue("nextStepDate", null));
+                  return;
+               }
+               if(transport.readiness() == CrmReadiness.NOT_CONFIGURED)
+               {
+                  return;
+               }
+               if(step.getEmailTemplateId() == null)
+               {
+                  throw (new QException("Sequence email template is required"));
+               }
+               CrmEmails.send(transport, contactGet.getRecord(), step.getEmailTemplateId(), null, null,
+                  "seq", enrollment.getId() + ":" + step.getId(), enrollment.getId());
+            }
             case CALL -> executeCallStep(enrollment, step);
             case TASK -> executeTaskStep(enrollment, step);
             case WAIT ->
@@ -247,50 +310,6 @@ public class ProcessSequenceStepsProcess implements BackendStep, MetaDataProduce
 
 
 
-   /***************************************************************************
-    ** Execute an EMAIL step: verify doNotEmail, create an activity record.
-    ***************************************************************************/
-   private void executeEmailStep(SequenceEnrollment enrollment, SequenceStep step) throws QException
-   {
-      ///////////////////////////////////////////////
-      // check doNotEmail                          //
-      ///////////////////////////////////////////////
-      GetOutput contactGet = new GetAction().execute(
-         new GetInput(Contact.TABLE_NAME).withPrimaryKey(enrollment.getContactId()));
-
-      if(contactGet.getRecord() != null)
-      {
-         Contact contact = new Contact(contactGet.getRecord());
-         if(Boolean.TRUE.equals(contact.getDoNotEmail()))
-         {
-            throw (new QException("Contact has opted out of email"));
-         }
-      }
-
-      ///////////////////////////////////////////////
-      // create activity for the email             //
-      ///////////////////////////////////////////////
-      String subject = "Sequence email: step " + step.getStepNumber();
-      if(step.getEmailTemplateId() != null)
-      {
-         subject += " (template #" + step.getEmailTemplateId() + ")";
-      }
-
-      Integer activityTypeId = lookupActivityTypeByName("Email");
-
-      new InsertAction().execute(
-         new InsertInput(Activity.TABLE_NAME).withRecordEntity(
-            new Activity()
-               .withActivityTypeId(activityTypeId)
-               .withContactId(enrollment.getContactId())
-               .withDealId(enrollment.getDealId())
-               .withSubject(subject)
-               .withOwnerUserId("system")
-               .withIsCompleted(true)));
-   }
-
-
-
    /***************************************************************************
     ** Execute a CALL step: create a task activity assigned to the contact owner.
     ***************************************************************************/
@@ -514,6 +533,8 @@ public class ProcessSequenceStepsProcess implements BackendStep, MetaDataProduce
       {
          updateRecord.withValue("status", CrmEnrollmentStatus.FAILED.getPossibleValueId());
          updateRecord.withValue("nextStepDate", null);
+         QAlerts.raise(new AlertSignal("crm.sequence.failing").withDedupKey("enrollment:" + enrollment.getId())
+            .withTitle("Sequence enrollment failed three times"));
       }
 
       new UpdateAction().execute(

src/main/java/com/kingsrook/qbits/crm/email/processes/SendEmailProcess.java

@@ -1,235 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** Process to send an email to a contact. Creates an Activity and an
- ** EmailMessage record, respecting doNotEmail flags and optional template
- ** lookups.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.email.processes;
-
-
-import java.time.Instant;
-import java.util.List;
-import com.kingsrook.qbits.crm.CrmSessionUtils;
-import com.kingsrook.qbits.crm.activities.model.Activity;
-import com.kingsrook.qbits.crm.activities.model.ActivityType;
-import com.kingsrook.qbits.crm.core.model.Contact;
-import com.kingsrook.qbits.crm.core.model.enums.CrmDirection;
-import com.kingsrook.qbits.crm.core.model.enums.CrmEmailStatus;
-import com.kingsrook.qbits.crm.email.model.EmailMessage;
-import com.kingsrook.qbits.crm.email.model.EmailTemplate;
-import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
-import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
-import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
-import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.exceptions.QUserFacingException;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertOutput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
-import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
-import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
-
-
-/*******************************************************************************
- ** MetaDataProducer + BackendStep for sending an email to a CRM contact.
- ** Validates doNotEmail, optionally loads a template, creates an Activity
- ** record, and creates a tracked EmailMessage record.
- *******************************************************************************/
-public class SendEmailProcess implements BackendStep, MetaDataProducerInterface<QProcessMetaData>
-{
-   public static final String NAME = "crm_send_email";
-
-
-
-   /*******************************************************************************
-    ** Produce the process metadata.
-    *******************************************************************************/
-   @Override
-   public QProcessMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QProcessMetaData()
-         .withName(NAME)
-         .withLabel("Send Email")
-         .withTableName(Contact.TABLE_NAME)
-         .withIcon(new QIcon().withName("email"))
-         .withStepList(List.of(
-            new QBackendStepMetaData()
-               .withName("execute")
-               .withCode(new QCodeReference(SendEmailProcess.class))
-               .withInputData(new QFunctionInputMetaData().withFieldList(List.of(
-                  new QFieldMetaData("contactId", QFieldType.INTEGER).withIsRequired(true),
-                  new QFieldMetaData("subject", QFieldType.STRING),
-                  new QFieldMetaData("bodyHtml", QFieldType.STRING),
-                  new QFieldMetaData("bodyText", QFieldType.STRING),
-                  new QFieldMetaData("emailTemplateId", QFieldType.INTEGER),
-                  new QFieldMetaData("dealId", QFieldType.INTEGER),
-                  new QFieldMetaData("ccAddresses", QFieldType.STRING),
-                  new QFieldMetaData("bccAddresses", QFieldType.STRING)
-               )))
-         ));
-   }
-
-
-
-   /*******************************************************************************
-    ** Execute the send-email process.
-    *******************************************************************************/
-   @Override
-   public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
-   {
-      Integer contactId       = input.getValueInteger("contactId");
-      String  subject         = input.getValueString("subject");
-      String  bodyHtml        = input.getValueString("bodyHtml");
-      String  bodyText        = input.getValueString("bodyText");
-      Integer emailTemplateId = input.getValueInteger("emailTemplateId");
-      Integer dealId          = input.getValueInteger("dealId");
-      String  ccAddresses     = input.getValueString("ccAddresses");
-      String  bccAddresses    = input.getValueString("bccAddresses");
-      String  sessionUserId   = CrmSessionUtils.getCurrentUserId();
-
-      ///////////////////////////////////////////
-      // Step 1: load and validate the contact //
-      ///////////////////////////////////////////
-      QRecord contactRecord = GetAction.execute(Contact.TABLE_NAME, contactId);
-      if(contactRecord == null)
-      {
-         throw new QException("Contact not found: " + contactId);
-      }
-
-      Contact contact = new Contact(contactRecord);
-      if(Boolean.TRUE.equals(contact.getDoNotEmail()))
-      {
-         throw new QUserFacingException("Cannot send email: contact has doNotEmail set to true.");
-      }
-
-      //////////////////////////////////////////////////////
-      // Step 2: if template provided, load and apply it  //
-      //////////////////////////////////////////////////////
-      if(emailTemplateId != null)
-      {
-         QRecord templateRecord = GetAction.execute(EmailTemplate.TABLE_NAME, emailTemplateId);
-         if(templateRecord != null)
-         {
-            EmailTemplate template = new EmailTemplate(templateRecord);
-            if(subject == null)
-            {
-               subject = template.getSubject();
-            }
-            if(bodyHtml == null)
-            {
-               bodyHtml = template.getBodyHtml();
-            }
-            if(bodyText == null)
-            {
-               bodyText = template.getBodyText();
-            }
-         }
-      }
-
-      /////////////////////////////////////////
-      // Step 3: look up Email activity type //
-      /////////////////////////////////////////
-      Integer emailTypeId = lookupEmailActivityTypeId();
-
-      //////////////////////////////////////////
-      // Step 4: create the Activity record   //
-      //////////////////////////////////////////
-      Activity activity = new Activity()
-         .withActivityTypeId(emailTypeId)
-         .withSubject(subject)
-         .withContactId(contactId)
-         .withDealId(dealId)
-         .withDirection(CrmDirection.OUTBOUND.getPossibleValueId())
-         .withOwnerUserId(sessionUserId)
-         .withIsCompleted(true);
-
-      InsertOutput activityInsert = new InsertAction().execute(
-         new InsertInput(Activity.TABLE_NAME).withRecordEntity(activity));
-      Integer activityId = activityInsert.getRecords().get(0).getValueInteger("id");
-
-      //////////////////////////////////////////////
-      // Step 5: create the EmailMessage record   //
-      //////////////////////////////////////////////
-      String toAddress = contact.getEmail() != null ? contact.getEmail() : "";
-
-      EmailMessage emailMessage = new EmailMessage()
-         .withActivityId(activityId)
-         .withFromAddress(sessionUserId)
-         .withFromName(sessionUserId)
-         .withToAddresses(toAddress)
-         .withCcAddresses(ccAddresses)
-         .withBccAddresses(bccAddresses)
-         .withSubject(subject != null ? subject : "")
-         .withBodyHtml(bodyHtml)
-         .withBodyText(bodyText)
-         .withDirection(CrmDirection.OUTBOUND.getPossibleValueId())
-         .withStatus(CrmEmailStatus.SENT.getPossibleValueId())
-         .withSentDate(Instant.now())
-         .withIsTracked(true)
-         .withOpenCount(0)
-         .withClickCount(0)
-         .withEmailTemplateId(emailTemplateId);
-
-      InsertOutput emailInsert = new InsertAction().execute(
-         new InsertInput(EmailMessage.TABLE_NAME).withRecordEntity(emailMessage));
-      Integer emailMessageId = emailInsert.getRecords().get(0).getValueInteger("id");
-
-      output.addValue("emailMessageId", emailMessageId);
-   }
-
-
-
-   /***************************************************************************
-    ** Look up the "Email" activity type id. Returns null if not found.
-    ***************************************************************************/
-   private Integer lookupEmailActivityTypeId() throws QException
-   {
-      QueryOutput queryOutput = new QueryAction().execute(
-         new QueryInput(ActivityType.TABLE_NAME)
-            .withFilter(new QQueryFilter()
-               .withCriteria(new QFilterCriteria("name", QCriteriaOperator.EQUALS, "Email"))));
-
-      if(!queryOutput.getRecords().isEmpty())
-      {
-         return (queryOutput.getRecords().get(0).getValueInteger("id"));
-      }
-
-      return (null);
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/email/processes/SyncEmailDeliveryProcess.java

@@ -0,0 +1,153 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2026.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.crm.email.processes;
+
+
+import java.time.Instant;
+import java.time.temporal.ChronoUnit;
+import java.util.List;
+import com.kingsrook.qbits.crm.api.CrmApi;
+import com.kingsrook.qbits.crm.core.CrmRecords;
+import com.kingsrook.qbits.crm.core.model.enums.CrmConsentStatus;
+import com.kingsrook.qbits.crm.core.model.enums.CrmConsentType;
+import com.kingsrook.qbits.crm.core.model.enums.CrmEmailStatus;
+import com.kingsrook.qbits.crm.email.CrmEmailTransport;
+import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
+import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
+
+
+/*******************************************************************************
+ ** Scheduled CRM maintenance: crm_sync_email_delivery.
+ *******************************************************************************/
+public class SyncEmailDeliveryProcess implements BackendStep, MetaDataProducerInterface<QProcessMetaData>
+{
+   public static final String NAME = "crm_sync_email_delivery";
+
+   @Override
+   public QProcessMetaData produce(QInstance instance)
+   {
+      return (new QProcessMetaData().withName(NAME).withLabel("SyncEmailDelivery").withIsHidden(true)
+         .withStep(new QBackendStepMetaData().withName("execute").withCode(new QCodeReference(SyncEmailDeliveryProcess.class))));
+   }
+
+   /*******************************************************************************
+    ** Resolve the host transport; an absent qbit-email yields empty lookups.
+    *******************************************************************************/
+   protected CrmEmailTransport emailTransport() throws QException
+   {
+      return (new CrmEmailTransport(CrmApi.config()));
+   }
+
+   @Override
+   public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
+   {
+      CrmEmailTransport transport = emailTransport();
+      Integer lastId = null;
+      int updated = 0;
+      Instant since = Instant.now().minus(30, ChronoUnit.DAYS);
+      while(true)
+      {
+         QQueryFilter filter = new QQueryFilter(new QFilterCriteria("createDate", QCriteriaOperator.GREATER_THAN_OR_EQUALS, since))
+            .withOrderBy(new QFilterOrderBy("id")).withLimit(200);
+         if(lastId != null)
+         {
+            filter.addCriteria(new QFilterCriteria("id", QCriteriaOperator.GREATER_THAN, lastId));
+         }
+         List<QRecord> page = new QueryAction().execute(new QueryInput("crm_email_message").withFilter(filter)).getRecords();
+         for(QRecord message : page)
+         {
+            lastId = message.getValueInteger("id");
+            if(message.getValueInteger("status") != null && List.of(CrmEmailStatus.BOUNCED.getId(), CrmEmailStatus.COMPLAINED.getId(), CrmEmailStatus.FAILED.getId(),
+               CrmEmailStatus.SUPPRESSED.getId()).contains(message.getValueInteger("status")))
+            {
+               continue;
+            }
+            var delivery = transport.lookup(message.getValueString("idempotencyKey"));
+            if(delivery.isEmpty())
+            {
+               continue;
+            }
+            CrmEmailTransport.Delivery state = delivery.get();
+            QRecord change = new QRecord().withValue("id", lastId).withValue("status", state.status().getId());
+            date(change, "sentDate", state.sentDate());
+            date(change, "deliveredDate", state.deliveredDate());
+            date(change, "firstOpenedDate", state.openedDate());
+            date(change, "firstClickedDate", state.clickedDate());
+            date(change, "bouncedDate", state.bouncedDate());
+            boolean complaint = state.status() == CrmEmailStatus.COMPLAINED;
+            if(complaint || state.status() == CrmEmailStatus.BOUNCED)
+            {
+               Integer contactId = message.getValueInteger("contactId");
+               if(contactId != null)
+               {
+                  CrmRecords.update("crm_contact", new QRecord().withValue("id", contactId).withValue("doNotEmail", true));
+                  String source = complaint ? "Complaint" : "Hard bounce";
+                  // The terminal message checkpoint is written last. A retry must not duplicate consent evidence.
+                  String evidence = "Email: " + message.getValueString("idempotencyKey");
+                  List<QRecord> recorded = new QueryAction().execute(new QueryInput("crm_consent_record").withFilter(new QQueryFilter(
+                     new QFilterCriteria("contactId", QCriteriaOperator.EQUALS, contactId),
+                     new QFilterCriteria("source", QCriteriaOperator.EQUALS, source),
+                     new QFilterCriteria("consentText", QCriteriaOperator.EQUALS, evidence)))).getRecords();
+                  if(recorded.isEmpty())
+                  {
+                     CrmRecords.insert("crm_consent_record", new QRecord().withValue("contactId", contactId)
+                        .withValue("consentType", CrmConsentType.EMAIL.getId()).withValue("status", CrmConsentStatus.WITHDRAWN.getId())
+                        .withValue("consentDate", Instant.now()).withValue("source", source).withValue("consentText", evidence));
+                  }
+               }
+               change.withValue("isSpamComplaint", complaint);
+            }
+            CrmRecords.update("crm_email_message", change);
+            updated++;
+         }
+         if(page.size() < 200)
+         {
+            break;
+         }
+      }
+      output.addValue("updatedCount", updated);
+   }
+
+   /*******************************************************************************
+    ** Never clear a timestamp when the provider has not exposed it.
+    *******************************************************************************/
+   private void date(QRecord row, String field, Instant value)
+   {
+      if(value != null)
+      {
+         row.setValue(field, value);
+      }
+   }
+}

src/main/java/com/kingsrook/qbits/crm/email/processes/TrackEmailEventProcess.java

@@ -1,222 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** Process to track email engagement events (open, click, bounce, spam, reply).
- ** Updates the EmailMessage record counters and flags, and may set doNotEmail
- ** on the linked contact for hard bounces and spam complaints.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.email.processes;
-
-
-import java.time.Instant;
-import java.util.List;
-import com.kingsrook.qbits.crm.core.model.Contact;
-import com.kingsrook.qbits.crm.core.model.enums.CrmBounceType;
-import com.kingsrook.qbits.crm.core.model.enums.CrmEnrollmentStatus;
-import com.kingsrook.qbits.crm.email.model.EmailMessage;
-import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
-import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
-import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
-import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
-import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
-import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
-
-
-/*******************************************************************************
- ** MetaDataProducer + BackendStep for tracking email engagement events.
- ** Handles OPEN, CLICK, BOUNCE, SPAM, and REPLY event types.
- *******************************************************************************/
-public class TrackEmailEventProcess implements BackendStep, MetaDataProducerInterface<QProcessMetaData>
-{
-   public static final String NAME = "crm_track_email_event";
-
-
-
-   /*******************************************************************************
-    ** Produce the process metadata.
-    *******************************************************************************/
-   @Override
-   public QProcessMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QProcessMetaData()
-         .withName(NAME)
-         .withLabel("Track Email Event")
-         .withTableName(EmailMessage.TABLE_NAME)
-         .withIcon(new QIcon().withName("track_changes"))
-         .withStepList(List.of(
-            new QBackendStepMetaData()
-               .withName("execute")
-               .withCode(new QCodeReference(TrackEmailEventProcess.class))
-               .withInputData(new QFunctionInputMetaData().withFieldList(List.of(
-                  new QFieldMetaData("emailMessageId", QFieldType.INTEGER).withIsRequired(true),
-                  new QFieldMetaData("eventType", QFieldType.STRING).withIsRequired(true),
-                  new QFieldMetaData("bounceTypeId", QFieldType.INTEGER)
-               )))
-         ));
-   }
-
-
-
-   /*******************************************************************************
-    ** Execute the tracking event process.
-    *******************************************************************************/
-   @Override
-   public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
-   {
-      Integer emailMessageId = input.getValueInteger("emailMessageId");
-      String  eventType      = input.getValueString("eventType");
-      Integer bounceTypeId   = input.getValueInteger("bounceTypeId");
-
-      //////////////////////////////////////////
-      // Step 1: load the email message       //
-      //////////////////////////////////////////
-      QRecord emailRecord = GetAction.execute(EmailMessage.TABLE_NAME, emailMessageId);
-      if(emailRecord == null)
-      {
-         throw new QException("Email message not found: " + emailMessageId);
-      }
-
-      EmailMessage email = new EmailMessage(emailRecord);
-
-      //////////////////////////////////////////
-      // Step 2: handle event type            //
-      //////////////////////////////////////////
-      QRecord updateRecord = new QRecord().withValue("id", emailMessageId);
-
-      switch(eventType.toUpperCase())
-      {
-         case "OPEN":
-         {
-            int currentOpenCount = email.getOpenCount() != null ? email.getOpenCount() : 0;
-            updateRecord.setValue("openCount", currentOpenCount + 1);
-            if(email.getFirstOpenedDate() == null)
-            {
-               updateRecord.setValue("firstOpenedDate", Instant.now());
-            }
-            break;
-         }
-
-         case "CLICK":
-         {
-            int currentClickCount = email.getClickCount() != null ? email.getClickCount() : 0;
-            updateRecord.setValue("clickCount", currentClickCount + 1);
-            if(email.getFirstClickedDate() == null)
-            {
-               updateRecord.setValue("firstClickedDate", Instant.now());
-            }
-            break;
-         }
-
-         case "BOUNCE":
-         {
-            updateRecord.setValue("bounceType", bounceTypeId);
-
-            /////////////////////////////////////////////////////
-            // if hard bounce, set contact doNotEmail to true   //
-            /////////////////////////////////////////////////////
-            if(bounceTypeId != null && bounceTypeId.equals(CrmBounceType.HARD.getPossibleValueId()))
-            {
-               setContactDoNotEmail(email);
-            }
-            break;
-         }
-
-         case "SPAM":
-         {
-            updateRecord.setValue("isSpamComplaint", true);
-            setContactDoNotEmail(email);
-            break;
-         }
-
-         case "REPLY":
-         {
-            //////////////////////////////////////////////////////////////
-            // if email is linked to a sequence enrollment, mark REPLIED //
-            //////////////////////////////////////////////////////////////
-            if(email.getSequenceEnrollmentId() != null)
-            {
-               QRecord enrollmentUpdate = new QRecord()
-                  .withValue("id", email.getSequenceEnrollmentId())
-                  .withValue("status", CrmEnrollmentStatus.REPLIED.getPossibleValueId());
-
-               new UpdateAction().execute(
-                  new UpdateInput("crm_sequence_enrollment").withRecord(enrollmentUpdate));
-            }
-            break;
-         }
-
-         default:
-            throw new QException("Unknown event type: " + eventType);
-      }
-
-      new UpdateAction().execute(
-         new UpdateInput(EmailMessage.TABLE_NAME).withRecord(updateRecord));
-   }
-
-
-
-   /***************************************************************************
-    ** Finds the contact linked to an email message (via activity) and sets
-    ** doNotEmail = true. Looks up by the toAddresses field.
-    ***************************************************************************/
-   private void setContactDoNotEmail(EmailMessage email) throws QException
-   {
-      if(email.getToAddresses() == null)
-      {
-         return;
-      }
-
-      QueryOutput contactQuery = new QueryAction().execute(
-         new QueryInput(Contact.TABLE_NAME)
-            .withFilter(new QQueryFilter()
-               .withCriteria(new QFilterCriteria("email", QCriteriaOperator.EQUALS, email.getToAddresses()))));
-
-      if(!contactQuery.getRecords().isEmpty())
-      {
-         QRecord contactUpdate = new QRecord()
-            .withValue("id", contactQuery.getRecords().get(0).getValueInteger("id"))
-            .withValue("doNotEmail", true);
-
-         new UpdateAction().execute(
-            new UpdateInput(Contact.TABLE_NAME).withRecord(contactUpdate));
-      }
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/email/processes/UnenrollFromSequenceProcess.java

@@ -27,7 +27,6 @@ package com.kingsrook.qbits.crm.email.processes;
 
 
 import java.time.Instant;
-import java.util.List;
 import com.kingsrook.qbits.crm.core.model.enums.CrmEnrollmentStatus;
 import com.kingsrook.qbits.crm.email.model.SequenceEnrollment;
 import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
@@ -41,49 +40,18 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetOutput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
-import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 
 
 /*******************************************************************************
- ** MetaDataProducer + BackendStep to unenroll a contact from an email sequence.
+ ** Backend operation used by the streamed CRM actions to unenroll a contact from an email sequence.
  ** Sets status to UNENROLLED and records the unenrolled date.
  *******************************************************************************/
-public class UnenrollFromSequenceProcess implements BackendStep, MetaDataProducerInterface<QProcessMetaData>
+public class UnenrollFromSequenceProcess implements BackendStep
 {
    public static final String NAME = "crm_unenroll_from_sequence";
 
 
 
-   /*******************************************************************************
-    ** Produce the process metadata.
-    *******************************************************************************/
-   @Override
-   public QProcessMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QProcessMetaData()
-         .withName(NAME)
-         .withLabel("Unenroll from Sequence")
-         .withIcon(new QIcon().withName("playlist_remove"))
-         .withStepList(List.of(
-            new QBackendStepMetaData()
-               .withName("execute")
-               .withCode(new QCodeReference(UnenrollFromSequenceProcess.class))
-               .withInputData(new QFunctionInputMetaData().withFieldList(List.of(
-                  new QFieldMetaData("enrollmentId", QFieldType.INTEGER).withIsRequired(true)
-               )))
-         ));
-   }
-
-
-
    /*******************************************************************************
     ** Execute the unenrollment logic.
     *******************************************************************************/

src/main/java/com/kingsrook/qbits/crm/scoring/AutoAssignCustomizer.java

@@ -0,0 +1,65 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2026.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.crm.scoring;
+
+
+import java.util.List;
+import com.kingsrook.qbits.crm.core.model.enums.CrmAssignableEntity;
+import com.kingsrook.qqq.backend.core.actions.customizers.AbstractPostInsertCustomizer;
+import com.kingsrook.qqq.backend.core.alerts.NotificationAudience;
+import com.kingsrook.qqq.backend.core.alerts.NotificationSignal;
+import com.kingsrook.qqq.backend.core.alerts.QAlerts;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+
+
+/*******************************************************************************
+ ** Apply assignment rules to new contacts and deals without replacing explicit owners.
+ *******************************************************************************/
+public class AutoAssignCustomizer extends AbstractPostInsertCustomizer
+{
+   @Override
+   public List<QRecord> apply(List<QRecord> records) throws QException
+   {
+      for(QRecord row : records)
+      {
+         if(row.getErrors() != null && !row.getErrors().isEmpty())
+         {
+            continue;
+         }
+         String owner = row.getValueString("ownerUserId");
+         if(owner == null || "system".equals(owner))
+         {
+            CrmAutoAssign.Result result = CrmAutoAssign.assign("crm_contact".equals(insertInput.getTableName())
+               ? CrmAssignableEntity.CONTACT : CrmAssignableEntity.DEAL, row.getValueInteger("id"));
+            if("ASSIGNED".equals(result.action()))
+            {
+               row.setValue("ownerUserId", result.assignedUserId());
+               QAlerts.notify(new NotificationSignal("CRM lead assigned").withSourceName("crm.lead.assigned")
+                  .withSourceKey(insertInput.getTableName() + ":" + row.getValue("id"))
+                  .withSourceTableName(insertInput.getTableName()).withSourceRecordId(row.getValueString("id"))
+                  .withAudience(NotificationAudience.user(result.assignedUserId())), insertInput.getTransaction());
+            }
+         }
+      }
+      return (records);
+   }
+}

src/main/java/com/kingsrook/qbits/crm/scoring/JobState.java

@@ -0,0 +1,49 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2026.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.crm.scoring;
+
+
+import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.UniqueKey;
+
+
+/*******************************************************************************
+ ** Small persistent maintenance checkpoints, including the last full score pass.
+ *******************************************************************************/
+public class JobState implements MetaDataProducerInterface<QTableMetaData>
+{
+   public static final String TABLE_NAME = "crm_job_state";
+
+   @Override
+   public QTableMetaData produce(QInstance instance)
+   {
+      return (new QTableMetaData().withName(TABLE_NAME).withLabel("Job State").withPrimaryKeyField("id")
+         .withField(new QFieldMetaData("id", QFieldType.INTEGER).withIsEditable(false))
+         .withField(new QFieldMetaData("name", QFieldType.STRING).withIsRequired(true).withMaxLength(250))
+         .withField(new QFieldMetaData("lastSuccess", QFieldType.DATE_TIME))
+         .withField(new QFieldMetaData("lastFullPass", QFieldType.DATE_TIME))
+         .withUniqueKey(new UniqueKey("name")));
+   }
+}

src/main/java/com/kingsrook/qbits/crm/scoring/processes/AutoAssignProcess.java

@@ -26,22 +26,12 @@
 package com.kingsrook.qbits.crm.scoring.processes;
 
 
-import java.util.List;
 import com.kingsrook.qbits.crm.core.model.enums.CrmAssignableEntity;
 import com.kingsrook.qbits.crm.scoring.CrmAutoAssign;
 import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
 import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerInterface;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
-import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 
 
 /*******************************************************************************
@@ -49,35 +39,12 @@ import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
  ** based on configured assignment rules. Evaluates rules in sortOrder; first
  ** match wins. Supports SPECIFIC_USER, ROUND_ROBIN, and LEAST_ACTIVE strategies.
  *******************************************************************************/
-public class AutoAssignProcess implements BackendStep, MetaDataProducerInterface<QProcessMetaData>
+public class AutoAssignProcess implements BackendStep
 {
    public static final String NAME = "crm_auto_assign";
 
 
 
-   /*******************************************************************************
-    ** Produce the process metadata.
-    *******************************************************************************/
-   @Override
-   public QProcessMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QProcessMetaData()
-         .withName(NAME)
-         .withLabel("Auto Assign")
-         .withIcon(new QIcon().withName("assignment_ind"))
-         .withStepList(List.of(
-            new QBackendStepMetaData()
-               .withName("execute")
-               .withCode(new QCodeReference(AutoAssignProcess.class))
-               .withInputData(new QFunctionInputMetaData().withFieldList(List.of(
-                  new QFieldMetaData("entityType", QFieldType.STRING).withIsRequired(true),
-                  new QFieldMetaData("recordId", QFieldType.INTEGER).withIsRequired(true)
-               )))
-         ));
-   }
-
-
-
    /*******************************************************************************
     ** Execute the assignment logic.
     *******************************************************************************/

src/main/java/com/kingsrook/qbits/crm/scoring/processes/CalculateLeadScoresProcess.java

@@ -26,22 +26,29 @@
 package com.kingsrook.qbits.crm.scoring.processes;
 
 
+import java.time.Instant;
+import java.time.LocalDate;
+import java.time.ZoneOffset;
+import java.time.temporal.ChronoUnit;
+import java.util.ArrayList;
+import java.util.HashMap;
 import java.util.List;
+import java.util.Map;
 import java.util.Objects;
+import com.kingsrook.qbits.crm.core.CrmRecords;
 import com.kingsrook.qbits.crm.core.model.Company;
 import com.kingsrook.qbits.crm.core.model.Contact;
 import com.kingsrook.qbits.crm.core.model.enums.CrmScoreOperator;
+import com.kingsrook.qbits.crm.deals.CrmDealNotifications;
+import com.kingsrook.qbits.crm.scoring.JobState;
 import com.kingsrook.qbits.crm.scoring.model.LeadScoreRule;
 import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
-import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
 import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
 import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.logging.QLogger;
 import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
 import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetOutput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
@@ -80,9 +87,9 @@ public class CalculateLeadScoresProcess implements BackendStep, MetaDataProducer
    {
       return new QProcessMetaData()
          .withName(NAME)
+         .withIsHidden(true)
          .withLabel("Calculate Lead Scores")
          .withIcon(new QIcon().withName("score"))
-         // Scheduling is configured by the host application via ScheduledJob records
          .withStepList(List.of(
             new QBackendStepMetaData()
                .withName("execute")
@@ -98,6 +105,14 @@ public class CalculateLeadScoresProcess implements BackendStep, MetaDataProducer
    @Override
    public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
    {
+      Instant startedAt = Instant.now();
+      List<QRecord> states = new QueryAction().execute(new QueryInput(JobState.TABLE_NAME)
+         .withFilter(new QQueryFilter(new QFilterCriteria("name", QCriteriaOperator.EQUALS, NAME)))).getRecords();
+      QRecord state = states.isEmpty() ? new QRecord().withValue("name", NAME) : states.getFirst();
+      Instant lastSuccess = state.getValueInstant("lastSuccess");
+      Instant lastFullPass = state.getValueInstant("lastFullPass");
+      boolean full = Boolean.TRUE.equals(input.getValueBoolean("fullRecalc")) || lastSuccess == null || lastFullPass == null
+         || !lastFullPass.isAfter(startedAt.minus(24, ChronoUnit.HOURS));
       ///////////////////////////////////////////////
       // load all active rules ordered by sort     //
       ///////////////////////////////////////////////
@@ -114,7 +129,7 @@ public class CalculateLeadScoresProcess implements BackendStep, MetaDataProducer
       ///////////////////////////////////////////////////////////////
       // paginate through all contacts to avoid OOM on large CRMs //
       ///////////////////////////////////////////////////////////////
-      int pageSize     = 500;
+      int pageSize     = 200;
       int updatedCount = 0;
       int totalContacts = 0;
       Integer lastId   = null;
@@ -130,6 +145,10 @@ public class CalculateLeadScoresProcess implements BackendStep, MetaDataProducer
             filter.withCriteria(new QFilterCriteria("id", QCriteriaOperator.GREATER_THAN, lastId));
          }
 
+         if(!full)
+         {
+            filter.withCriteria(new QFilterCriteria("modifyDate", QCriteriaOperator.GREATER_THAN_OR_EQUALS, lastSuccess));
+         }
          QueryOutput contactsQuery = new QueryAction().execute(
             new QueryInput(Contact.TABLE_NAME).withFilter(filter));
 
@@ -138,41 +157,20 @@ public class CalculateLeadScoresProcess implements BackendStep, MetaDataProducer
             break;
          }
 
+         Map<Integer, QRecord> companies = loadCompanies(contactsQuery.getRecords());
+         List<QRecord> changes = new ArrayList<>();
          for(QRecord contactRecord : contactsQuery.getRecords())
          {
-            Contact contact = new Contact(contactRecord);
             totalContacts++;
-            int newScore = 0;
-
-            ///////////////////////////////////////////////
-            // evaluate each rule against the contact    //
-            ///////////////////////////////////////////////
-            for(LeadScoreRule rule : rules)
+            QRecord change = calculateChange(contactRecord, rules, companies);
+            if(change != null)
             {
-               if(evaluateRule(rule, contactRecord))
-               {
-                  newScore += rule.getScoreAdjustment();
-               }
+               changes.add(change);
             }
-
-            ///////////////////////////////////////////////
-            // update only if score changed              //
-            ///////////////////////////////////////////////
-            Integer currentScore = contact.getLeadScore() != null ? contact.getLeadScore() : 0;
-            if(!Objects.equals(currentScore, newScore))
-            {
-               QRecord updateRecord = new QRecord()
-                  .withValue("id", contact.getId())
-                  .withValue("leadScore", newScore);
-
-               new UpdateAction().execute(
-                  new UpdateInput(Contact.TABLE_NAME).withRecord(updateRecord));
-
-               updatedCount++;
-            }
-
-            lastId = contact.getId();
+            lastId = contactRecord.getValueInteger("id");
          }
+         writeScores(changes);
+         updatedCount += changes.size();
 
          /////////////////////////////////////////////
          // if page was not full, we are done       //
@@ -183,6 +181,21 @@ public class CalculateLeadScoresProcess implements BackendStep, MetaDataProducer
          }
       }
 
+      CrmDealNotifications.checkRotting();
+      state.withValue("lastSuccess", startedAt);
+      if(full)
+      {
+         state.withValue("lastFullPass", startedAt);
+      }
+      if(states.isEmpty())
+      {
+         CrmRecords.insert(JobState.TABLE_NAME, state);
+      }
+      else
+      {
+         CrmRecords.update(JobState.TABLE_NAME, state);
+      }
+      output.addValue("fullRecalc", full);
       output.addValue("updatedCount", updatedCount);
       output.addValue("totalContacts", totalContacts);
       output.addValue("totalRules", rules.size());
@@ -190,11 +203,63 @@ public class CalculateLeadScoresProcess implements BackendStep, MetaDataProducer
 
 
 
+   /*******************************************************************************
+    ** Recalculate only the selected contact using the same rules as the job.
+    *******************************************************************************/
+   public void recalculate(QRecord contact) throws QException
+   {
+      List<LeadScoreRule> rules = new QueryAction().execute(new QueryInput(LeadScoreRule.TABLE_NAME)
+         .withFilter(new QQueryFilter(new QFilterCriteria("isActive", QCriteriaOperator.EQUALS, true))
+            .withOrderBy(new QFilterOrderBy("sortOrder")))).getRecords().stream().map(LeadScoreRule::new).toList();
+      QRecord change = calculateChange(contact, rules, loadCompanies(List.of(contact)));
+      writeScores(change == null ? List.of() : List.of(change));
+   }
+
+   /*******************************************************************************
+    ** Compute a score update only when the result changed.
+    *******************************************************************************/
+   private QRecord calculateChange(QRecord contact, List<LeadScoreRule> rules, Map<Integer, QRecord> companies) throws QException
+   {
+      int score = 0;
+      for(LeadScoreRule rule : rules)
+      {
+         if(evaluateRule(rule, contact, companies))
+         {
+            score += rule.getScoreAdjustment();
+         }
+      }
+      Integer current = contact.getValueInteger("leadScore");
+      if(Objects.equals(current == null ? 0 : current, score))
+      {
+         return (null);
+      }
+      return (new QRecord().withValue("id", contact.getValue("id")).withValue("leadScore", score));
+   }
+
+   /*******************************************************************************
+    ** Batch score writes so core audit label resolution also operates per page.
+    *******************************************************************************/
+   private void writeScores(List<QRecord> changes) throws QException
+   {
+      if(changes.isEmpty())
+      {
+         return;
+      }
+      UpdateInput update = new UpdateInput(Contact.TABLE_NAME).withOmitModifyDateUpdate(true);
+      update.setRecords(changes);
+      var updated = new UpdateAction().execute(update).getRecords();
+      if(updated.isEmpty() || updated.stream().anyMatch(row -> row.getErrors() != null && !row.getErrors().isEmpty()))
+      {
+         throw (new QException("Lead score update was rejected"));
+      }
+   }
+
+
    /***************************************************************************
     ** Evaluate a single rule against a contact record. Supports dot-notation
     ** field paths for related entities (e.g., "company.industry").
     ***************************************************************************/
-   private boolean evaluateRule(LeadScoreRule rule, QRecord contactRecord)
+   private boolean evaluateRule(LeadScoreRule rule, QRecord contactRecord, Map<Integer, QRecord> companies)
    {
       CrmScoreOperator operator = CrmScoreOperator.getById(rule.getOperator());
       if(operator == null)
@@ -203,7 +268,7 @@ public class CalculateLeadScoresProcess implements BackendStep, MetaDataProducer
       }
 
       String fieldPath = rule.getFieldPath();
-      String actualValue = resolveFieldValue(fieldPath, contactRecord);
+      String actualValue = resolveFieldValue(fieldPath, contactRecord, companies);
       String expectedValue = rule.getFieldValue();
 
       switch(operator)
@@ -223,6 +288,10 @@ public class CalculateLeadScoresProcess implements BackendStep, MetaDataProducer
          case LT:
             return (compareNumeric(actualValue, expectedValue) < 0);
 
+         case DAYS_AGO_GT:
+         case DAYS_AGO_LT:
+            return (compareAge(actualValue, expectedValue, operator));
+
          case IS_SET:
             return (StringUtils.hasContent(actualValue));
 
@@ -262,48 +331,54 @@ public class CalculateLeadScoresProcess implements BackendStep, MetaDataProducer
     ** for related entity fields (e.g., "company.industry" loads the contact's
     ** company record and returns its "industry" field).
     ***************************************************************************/
-   private String resolveFieldValue(String fieldPath, QRecord contactRecord)
+   private String resolveFieldValue(String fieldPath, QRecord contactRecord, Map<Integer, QRecord> companies)
    {
       if(fieldPath == null || !fieldPath.contains("."))
       {
          return (contactRecord.getValueString(fieldPath));
       }
-
       String[] parts = fieldPath.split("\\.", 2);
-      String prefix    = parts[0];
-      String fieldName = parts[1];
-
-      try
+      if("company".equalsIgnoreCase(parts[0]))
       {
-         if("company".equalsIgnoreCase(prefix))
-         {
-            Integer companyId = contactRecord.getValueInteger("companyId");
-            if(companyId == null)
-            {
-               return (null);
-            }
-
-            GetOutput companyOutput = new GetAction().execute(
-               new GetInput(Company.TABLE_NAME).withPrimaryKey(companyId));
-
-            if(companyOutput.getRecord() == null)
-            {
-               return (null);
-            }
-
-            return (companyOutput.getRecord().getValueString(fieldName));
-         }
-         else
-         {
-            LOG.warn("Unsupported dot-notation prefix in lead score rule fieldPath: " + prefix);
-            return (null);
-         }
-      }
-      catch(Exception e)
-      {
-         LOG.warn("Error resolving dot-notation field [" + fieldPath + "] for contact", e);
-         return (null);
+         QRecord company = companies.get(contactRecord.getValueInteger("companyId"));
+         return (company == null ? null : company.getValueString(parts[1]));
       }
+      LOG.warn("Unsupported dot-notation prefix in lead score rule fieldPath: " + parts[0]);
+      return (null);
    }
 
+   /*******************************************************************************
+    ** One company query per page, shared by every rule and contact in that page.
+    *******************************************************************************/
+   private Map<Integer, QRecord> loadCompanies(List<QRecord> contacts) throws QException
+   {
+      List<Integer> ids = contacts.stream().map(row -> row.getValueInteger("companyId")).filter(Objects::nonNull).distinct().toList();
+      Map<Integer, QRecord> companies = new HashMap<>();
+      if(!ids.isEmpty())
+      {
+         for(QRecord company : new QueryAction().execute(new QueryInput(Company.TABLE_NAME)
+            .withFilter(new QQueryFilter(new QFilterCriteria("id", QCriteriaOperator.IN, ids)))).getRecords())
+         {
+            companies.put(company.getValueInteger("id"), company);
+         }
+      }
+      return (companies);
+   }
+
+   /*******************************************************************************
+    ** Compare instants or UTC calendar dates with the strict N-days-ago boundary.
+    *******************************************************************************/
+   private boolean compareAge(String actual, String expected, CrmScoreOperator operator)
+   {
+      try
+      {
+         Instant date = actual.length() == 10 ? LocalDate.parse(actual).atStartOfDay(ZoneOffset.UTC).toInstant() : Instant.parse(actual);
+         Instant threshold = Instant.now().minus(Long.parseLong(expected), ChronoUnit.DAYS);
+         return (operator == CrmScoreOperator.DAYS_AGO_GT ? date.isBefore(threshold) : date.isAfter(threshold));
+      }
+      catch(RuntimeException e)
+      {
+         return (false);
+      }
+   }
 }

src/main/java/com/kingsrook/qbits/crm/security/CrmPermissions.java

@@ -49,7 +49,7 @@ public final class CrmPermissions
    public static final List<String> ALL = List.of(APP_ACCESS, READ, WRITE, MANAGE, MANAGE_READ, MANAGE_WRITE, ADMIN, ADMIN_READ, ADMIN_WRITE);
 
    private static final Set<String> SETTINGS_TABLES = Set.of(
-      "crm_pipeline", "crm_product", "crm_email_template", "crm_email_sequence",
+      "crm_job_state", "crm_pipeline", "crm_product", "crm_email_template", "crm_email_sequence",
       "crm_lead_score_rule", "crm_assignment_rule", "crm_sales_goal", "crm_lifecycle_stage",
       "crm_lead_source", "crm_industry", "crm_activity_type", "crm_activity_outcome",
       "crm_contact_role", "crm_win_loss_reason");
@@ -57,7 +57,8 @@ public final class CrmPermissions
    private static final Set<String> MANAGEMENT_PROCESSES = Set.of(
       "crm_merge_contacts", "crm_merge_companies", "crm_bulk_assign_owner",
       "crm_subscribe_to_list", "crm_unsubscribe_from_list", "crm_enroll_in_sequence",
-      "crm_unenroll_from_sequence", "crm_process_sequence_steps");
+      "crm_unenroll_from_sequence", "crm_process_sequence_steps", "crm_assign_company_owner",
+      "crm_transfer_ownership", "crm_activate_sequence", "crm_pause_sequence", "crm_sync_email_delivery", "crm_expire_consents", "crm_calculate_lead_scores");
 
 
 

src/main/java/com/kingsrook/qbits/crm/widgets/CrmActivityCountRenderer.java

@@ -1,128 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** Renderer for the CRM Activity Count widget.
- **
- ** Queries activities created today and groups them by activityTypeId to
- ** produce counts. Returns a MultiStatisticsData with one entry per known
- ** type: Call, Email, Meeting.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.widgets;
-
-
-import java.time.Instant;
-import java.time.LocalDate;
-import java.time.ZoneId;
-import java.time.ZonedDateTime;
-import java.util.ArrayList;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
-import com.kingsrook.qbits.crm.activities.model.Activity;
-import com.kingsrook.qbits.crm.activities.model.ActivityType;
-import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
-import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
-import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
-import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.MultiStatisticsData;
-import com.kingsrook.qqq.backend.core.model.data.QRecord;
-
-
-/*******************************************************************************
- ** Renders activity counts by type for today as a multi-statistics widget.
- *******************************************************************************/
-public class CrmActivityCountRenderer extends AbstractWidgetRenderer
-{
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Override
-   public RenderWidgetOutput render(RenderWidgetInput input) throws QException
-   {
-      ////////////////////////////////////////////////////////
-      // determine start-of-day in the system's time zone   //
-      ////////////////////////////////////////////////////////
-      ZoneId zoneId = ZoneId.systemDefault();
-      ZonedDateTime startOfDay = LocalDate.now(zoneId).atStartOfDay(zoneId);
-      Instant todayStart = startOfDay.toInstant();
-
-      ////////////////////////////////////////////////////////////////
-      // query all activity types for labeling                      //
-      ////////////////////////////////////////////////////////////////
-      Map<Integer, String> typeNameMap = new HashMap<>();
-      QueryOutput typeOutput = new QueryAction().execute(
-         new QueryInput(ActivityType.TABLE_NAME));
-      for(QRecord typeRecord : typeOutput.getRecords())
-      {
-         typeNameMap.put(typeRecord.getValueInteger("id"), typeRecord.getValueString("name"));
-      }
-
-      ////////////////////////////////////////////////////////////
-      // query activities created today                         //
-      ////////////////////////////////////////////////////////////
-      QueryOutput activityOutput = new QueryAction().execute(
-         new QueryInput(Activity.TABLE_NAME)
-            .withFilter(new QQueryFilter()
-               .withCriteria(new QFilterCriteria("createDate", QCriteriaOperator.GREATER_THAN_OR_EQUALS, todayStart))));
-
-      ////////////////////////////////////////////////////////////
-      // count by activityTypeId                                //
-      ////////////////////////////////////////////////////////////
-      Map<Integer, Integer> countByType = new HashMap<>();
-      for(QRecord record : activityOutput.getRecords())
-      {
-         Integer typeId = record.getValueInteger("activityTypeId");
-         if(typeId != null)
-         {
-            countByType.merge(typeId, 1, Integer::sum);
-         }
-      }
-
-      ////////////////////////////////////////////////////////////
-      // build statistics group data                            //
-      ////////////////////////////////////////////////////////////
-      List<MultiStatisticsData.StatisticsGroupData> groups = new ArrayList<>();
-      for(Map.Entry<Integer, Integer> entry : countByType.entrySet())
-      {
-         String typeName = typeNameMap.getOrDefault(entry.getKey(), "Type " + entry.getKey());
-         groups.add(new MultiStatisticsData.StatisticsGroupData()
-            .withHeader(typeName)
-            .withSubheader("today")
-            .withStatisticList(List.of(
-               new MultiStatisticsData.StatisticsGroupData.Statistic(typeName, entry.getValue(), null)
-            )));
-      }
-
-      MultiStatisticsData data = new MultiStatisticsData("Activity Counts (Today)", groups);
-
-      return (new RenderWidgetOutput(data));
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmActivityCountWidgetProducer.java

@@ -1,68 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** MetaData producer for the CRM Activity Count widget.
- **
- ** Provides a MULTI_STATISTICS view showing counts of activities by type for
- ** the current user today: calls, emails, meetings.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.widgets;
-
-
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducer;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
-
-
-/*******************************************************************************
- ** Produces metadata for the crm_activity_count widget (MULTI_STATISTICS type).
- *******************************************************************************/
-public class CrmActivityCountWidgetProducer extends MetaDataProducer<QWidgetMetaData>
-{
-   public static final String NAME = "crm_activity_count";
-
-
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Override
-   public QWidgetMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QWidgetMetaData()
-         .withName(NAME)
-         .withType(WidgetType.MULTI_STATISTICS.getType())
-         .withGridColumns(6)
-         .withIsCard(true)
-         .withLabel("Activity Counts (Today)")
-         .withTooltip("Activity counts by type for today")
-         .withShowReloadButton(true)
-         .withIcon("topRightInsideCard", new QIcon("analytics"))
-         .withCodeReference(new QCodeReference(CrmActivityCountRenderer.class));
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmActivityFeedRenderer.java

@@ -1,131 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** Renderer for the CRM Activity Feed widget.
- **
- ** Queries the most recent 50 activities (by createDate descending) and returns
- ** a table with columns: subject, activityType, contact, company, date.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.widgets;
-
-
-import java.time.Instant;
-import java.time.ZoneId;
-import java.time.format.DateTimeFormatter;
-import java.util.ArrayList;
-import java.util.List;
-import java.util.Map;
-import com.kingsrook.qbits.crm.activities.model.Activity;
-import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
-import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
-import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
-import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.TableData;
-import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.utils.collections.MapBuilder;
-
-
-/*******************************************************************************
- ** Renders the activity feed as a table widget showing recent activities.
- *******************************************************************************/
-public class CrmActivityFeedRenderer extends AbstractWidgetRenderer
-{
-   private static final DateTimeFormatter DATE_FORMATTER = DateTimeFormatter
-      .ofPattern("yyyy-MM-dd HH:mm")
-      .withZone(ZoneId.systemDefault());
-
-
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Override
-   public RenderWidgetOutput render(RenderWidgetInput input) throws QException
-   {
-      ///////////////////////////////////////////////////////////
-      // query recent activities, ordered by createDate desc   //
-      ///////////////////////////////////////////////////////////
-      QueryInput queryInput = new QueryInput(Activity.TABLE_NAME);
-      queryInput.setFilter(new QQueryFilter()
-         .withOrderBy(new QFilterOrderBy("createDate", false))
-         .withLimit(50));
-
-      QueryOutput queryOutput = new QueryAction().execute(queryInput);
-
-      ///////////////////////////
-      // build table columns   //
-      ///////////////////////////
-      List<TableData.Column> columns = List.of(
-         new TableData.Column("html", "Subject", "subject", "2fr", null),
-         new TableData.Column("html", "Activity Type", "activityType", "1fr", null),
-         new TableData.Column("html", "Contact", "contact", "1fr", null),
-         new TableData.Column("html", "Company", "company", "1fr", null),
-         new TableData.Column("html", "Date", "date", "1fr", "right")
-      );
-
-      //////////////////////////
-      // build table rows     //
-      //////////////////////////
-      List<Map<String, Object>> rows = new ArrayList<>();
-      for(QRecord record : queryOutput.getRecords())
-      {
-         String subject = record.getValueString("subject");
-         if(subject == null)
-         {
-            subject = "";
-         }
-
-         Integer activityTypeId = record.getValueInteger("activityTypeId");
-         String activityTypeDisplay = activityTypeId != null ? String.valueOf(activityTypeId) : "-";
-
-         Integer contactId = record.getValueInteger("contactId");
-         String contactDisplay = contactId != null ? String.valueOf(contactId) : "-";
-
-         Integer companyId = record.getValueInteger("companyId");
-         String companyDisplay = companyId != null ? String.valueOf(companyId) : "-";
-
-         Instant createDate = record.getValueInstant("createDate");
-         String dateDisplay = createDate != null ? DATE_FORMATTER.format(createDate) : "-";
-
-         rows.add(MapBuilder.of(
-            "subject", subject,
-            "activityType", activityTypeDisplay,
-            "contact", contactDisplay,
-            "company", companyDisplay,
-            "date", dateDisplay
-         ));
-      }
-
-      TableData tableData = new TableData(null, columns, rows)
-         .withRowsPerPage(50)
-         .withHidePaginationDropdown(true);
-
-      return (new RenderWidgetOutput(tableData));
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmActivityFeedWidgetProducer.java

@@ -1,67 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** MetaData producer for the CRM Activity Feed widget.
- **
- ** Provides a TABLE view showing the most recent 50 activities across the CRM
- ** with columns: subject, activityType, contact, company, date.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.widgets;
-
-
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducer;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
-
-
-/*******************************************************************************
- ** Produces metadata for the crm_activity_feed widget (TABLE type).
- *******************************************************************************/
-public class CrmActivityFeedWidgetProducer extends MetaDataProducer<QWidgetMetaData>
-{
-   public static final String NAME = "crm_activity_feed";
-
-
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Override
-   public QWidgetMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QWidgetMetaData()
-         .withName(NAME)
-         .withType(WidgetType.TABLE.getType())
-         .withGridColumns(6)
-         .withIsCard(true)
-         .withLabel("Activity Feed")
-         .withTooltip("Recent CRM activities")
-         .withShowReloadButton(true)
-         .withIcon("topRightInsideCard", new com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon("event_note"))
-         .withCodeReference(new QCodeReference(CrmActivityFeedRenderer.class));
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmAvgDealSizeRenderer.java

@@ -1,143 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** Renderer for the Average Deal Size widget.
- **
- ** Queries deals where actualCloseDate is set and stage is closedWon.
- ** Computes average amountInBaseCurrency and returns as a single-stat table.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.widgets;
-
-
-import java.math.BigDecimal;
-import java.math.RoundingMode;
-import java.util.ArrayList;
-import java.util.HashSet;
-import java.util.List;
-import java.util.Map;
-import java.util.Set;
-import com.kingsrook.qbits.crm.deals.model.Deal;
-import com.kingsrook.qbits.crm.deals.model.PipelineStage;
-import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
-import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
-import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
-import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.TableData;
-import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.utils.collections.MapBuilder;
-
-
-/*******************************************************************************
- ** Renders average deal size as a single-stat table widget.
- *******************************************************************************/
-public class CrmAvgDealSizeRenderer extends AbstractWidgetRenderer
-{
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Override
-   public RenderWidgetOutput render(RenderWidgetInput input) throws QException
-   {
-      ///////////////////////////////////////////////////////////
-      // find closedWon stage IDs                              //
-      ///////////////////////////////////////////////////////////
-      Set<Integer> closedWonStageIds = new HashSet<>();
-      QueryOutput stageOutput = new QueryAction().execute(
-         new QueryInput(PipelineStage.TABLE_NAME)
-            .withFilter(new QQueryFilter()
-               .withCriteria(new QFilterCriteria("isClosedWon", QCriteriaOperator.EQUALS, true))));
-
-      for(QRecord record : stageOutput.getRecords())
-      {
-         closedWonStageIds.add(record.getValueInteger("id"));
-      }
-
-      ///////////////////////////////////////////////////////////
-      // query deals with actualCloseDate set                  //
-      ///////////////////////////////////////////////////////////
-      QueryOutput dealOutput = new QueryAction().execute(
-         new QueryInput(Deal.TABLE_NAME)
-            .withFilter(new QQueryFilter()
-               .withCriteria(new QFilterCriteria("actualCloseDate", QCriteriaOperator.IS_NOT_BLANK))));
-
-      ///////////////////////////////////////////////////////////
-      // compute average for closedWon deals                   //
-      ///////////////////////////////////////////////////////////
-      BigDecimal totalAmount = BigDecimal.ZERO;
-      int count = 0;
-
-      for(QRecord record : dealOutput.getRecords())
-      {
-         Integer stageId = record.getValueInteger("pipelineStageId");
-         if(closedWonStageIds.contains(stageId))
-         {
-            BigDecimal amount = record.getValueBigDecimal("amountInBaseCurrency");
-            if(amount == null)
-            {
-               amount = record.getValueBigDecimal("amount");
-            }
-            if(amount != null)
-            {
-               totalAmount = totalAmount.add(amount);
-               count++;
-            }
-         }
-      }
-
-      BigDecimal avgDealSize = count > 0
-         ? totalAmount.divide(BigDecimal.valueOf(count), 2, RoundingMode.HALF_UP)
-         : BigDecimal.ZERO;
-
-      ///////////////////////////
-      // build table output    //
-      ///////////////////////////
-      List<TableData.Column> columns = List.of(
-         new TableData.Column("html", "Metric", "metric", "1fr", null),
-         new TableData.Column("html", "Value", "value", "1fr", "right")
-      );
-
-      List<Map<String, Object>> rows = new ArrayList<>();
-      rows.add(MapBuilder.of(
-         "metric", "Average Deal Size",
-         "value", "$" + avgDealSize.toPlainString()
-      ));
-      rows.add(MapBuilder.of(
-         "metric", "Closed Won Deals",
-         "value", String.valueOf(count)
-      ));
-
-      TableData tableData = new TableData(null, columns, rows)
-         .withRowsPerPage(10)
-         .withHidePaginationDropdown(true);
-
-      return (new RenderWidgetOutput(tableData));
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmAvgDealSizeWidgetProducer.java

@@ -1,66 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** MetaData producer for the Average Deal Size widget.
- **
- ** Shows the average amount of closed-won deals.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.widgets;
-
-
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducer;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
-
-
-/*******************************************************************************
- ** Produces metadata for the crm_avg_deal_size widget (TABLE type).
- *******************************************************************************/
-public class CrmAvgDealSizeWidgetProducer extends MetaDataProducer<QWidgetMetaData>
-{
-   public static final String NAME = "crm_avg_deal_size";
-
-
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Override
-   public QWidgetMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QWidgetMetaData()
-         .withName(NAME)
-         .withType(WidgetType.TABLE.getType())
-         .withGridColumns(3)
-         .withIsCard(true)
-         .withLabel("Average Deal Size")
-         .withTooltip("Average amount of closed-won deals")
-         .withShowReloadButton(true)
-         .withIcon("topRightInsideCard", new com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon("bar_chart"))
-         .withCodeReference(new QCodeReference(CrmAvgDealSizeRenderer.class));
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmContactTimelineRenderer.java

@@ -19,109 +19,60 @@
  */
 
 
-/*******************************************************************************
- ** Renderer for the CRM Contact Timeline widget.
- **
- ** Queries activities for a specific contactId (from widget input) and returns
- ** a StepperData with each activity as a chronological step. Activities are
- ** ordered by createDate ascending.
- *******************************************************************************/
-
 package com.kingsrook.qbits.crm.widgets;
 
 
 import java.time.Instant;
-import java.time.ZoneId;
-import java.time.format.DateTimeFormatter;
 import java.util.ArrayList;
+import java.util.Comparator;
 import java.util.List;
-import com.kingsrook.qbits.crm.activities.model.Activity;
+import java.util.Map;
+import java.util.Objects;
+import com.kingsrook.qqq.backend.core.actions.ai.output.AiLinkBuilder;
 import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
-import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
+import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
 import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
 import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.StepperData;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.utils.StringUtils;
-import com.kingsrook.qqq.backend.core.utils.ValueUtils;
 
 
 /*******************************************************************************
- ** Renders a contact's activity timeline as a stepper widget.
+ ** A contact's latest fifty recorded events across activities, sent messages,
+ ** web leads and consent. Each source is bounded before merging by create date.
  *******************************************************************************/
 public class CrmContactTimelineRenderer extends AbstractWidgetRenderer
 {
-   private static final DateTimeFormatter DATE_FORMATTER = DateTimeFormatter
-      .ofPattern("yyyy-MM-dd HH:mm")
-      .withZone(ZoneId.systemDefault());
-
-
-
    /*******************************************************************************
-    **
+    ** Record sections supply id; contactId remains useful for embedded hosts.
     *******************************************************************************/
    @Override
    public RenderWidgetOutput render(RenderWidgetInput input) throws QException
    {
-      String contactIdParam = input.getQueryParams().get("contactId");
-
-      if(!StringUtils.hasContent(contactIdParam))
+      List<Map<String, Object>> rows = new ArrayList<>();
+      String id = input.getQueryParams().getOrDefault("id", input.getQueryParams().get("contactId"));
+      if(id != null && !id.isBlank() && GetAction.execute("crm_contact", Integer.valueOf(id)) != null)
       {
-         return (new RenderWidgetOutput(new StepperData("Contact Timeline", 0, List.of())));
-      }
-
-      Integer contactId = ValueUtils.getValueAsInteger(contactIdParam);
-
-      //////////////////////////////////////////////////////////////
-      // query activities for this contact, ordered chronologically //
-      //////////////////////////////////////////////////////////////
-      QueryInput queryInput = new QueryInput(Activity.TABLE_NAME);
-      queryInput.setFilter(new QQueryFilter()
-         .withCriteria(new QFilterCriteria("contactId", QCriteriaOperator.EQUALS, contactId))
-         .withOrderBy(new QFilterOrderBy("createDate", true)));
-
-      QueryOutput queryOutput = new QueryAction().execute(queryInput);
-
-      ////////////////////////////////////
-      // build steps from activities    //
-      ////////////////////////////////////
-      List<StepperData.Step> steps = new ArrayList<>();
-      for(QRecord record : queryOutput.getRecords())
-      {
-         String subject = record.getValueString("subject");
-         if(subject == null)
+         List<QRecord> events = new ArrayList<>();
+         for(String table : List.of("crm_activity", "crm_email_message", "crm_form_submission", "crm_consent_record"))
          {
-            subject = "Activity";
+            for(QRecord record : CrmHomeRenderer.query(table, CrmHomeRenderer.equalsFilter("contactId", Integer.valueOf(id)), "createDate", false, 50))
+            {
+               record.setValue("timelineTable", table);
+               events.add(record);
+            }
+         }
+         events.sort(Comparator.comparing((QRecord record) -> Objects.requireNonNullElse(record.getValueInstant("createDate"), Instant.EPOCH)).reversed()
+            .thenComparing(record -> record.getValueString("timelineTable")).thenComparing(record -> record.getValueInteger("id")));
+         for(QRecord record : events.stream().limit(50).toList())
+         {
+            String table = record.getValueString("timelineTable");
+            String kind = Map.of("crm_activity", "Activity", "crm_email_message", "Email message",
+               "crm_form_submission", "Web lead", "crm_consent_record", "Consent record").get(table);
+            String detail = Objects.toString(record.getValue("subject"), Objects.toString(record.getValue("email"), kind));
+            rows.add(CrmHomeRenderer.row(AiLinkBuilder.recordHref(table, record.getValue("id")), record.getValue("createDate"), kind, detail));
          }
-
-         Instant createDate = record.getValueInstant("createDate");
-         String dateStr = createDate != null ? DATE_FORMATTER.format(createDate) : "";
-
-         String description = record.getValueString("description");
-         String linkText = StringUtils.hasContent(description)
-            ? description.substring(0, Math.min(description.length(), 100))
-            : dateStr;
-
-         steps.add(new StepperData.Step()
-            .withLabel(subject)
-            .withLinkText(linkText));
       }
-
-      //////////////////////////////////////////////////////////////
-      // set activeStep to the last step (most recent activity)   //
-      //////////////////////////////////////////////////////////////
-      Integer activeStep = steps.isEmpty() ? 0 : steps.size() - 1;
-
-      StepperData stepperData = new StepperData("Contact Timeline", activeStep, steps);
-
-      return (new RenderWidgetOutput(stepperData));
+      return (new RenderWidgetOutput(CrmHomeRenderer.table(rows, "Recorded", "Kind", "Details")));
    }
-
 }

src/main/java/com/kingsrook/qbits/crm/widgets/CrmContactTimelineWidgetProducer.java

@@ -22,13 +22,14 @@
 /*******************************************************************************
  ** MetaData producer for the CRM Contact Timeline widget.
  **
- ** Provides a STEPPER view showing a chronological timeline of activities for
- ** a specific contact. The contactId is passed as a widget input parameter.
+ ** Provides a standard table of activities, messages, web leads and consent for
+ ** the contact selected by the record screen id parameter.
  *******************************************************************************/
 
 package com.kingsrook.qbits.crm.widgets;
 
 
+import com.kingsrook.qbits.crm.security.CrmPermissions;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
 import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducer;
@@ -39,7 +40,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
 
 
 /*******************************************************************************
- ** Produces metadata for the crm_contact_timeline widget (STEPPER type).
+ ** Produces metadata for the crm_contact_timeline widget (TABLE type).
  *******************************************************************************/
 public class CrmContactTimelineWidgetProducer extends MetaDataProducer<QWidgetMetaData>
 {
@@ -55,8 +56,9 @@ public class CrmContactTimelineWidgetProducer extends MetaDataProducer<QWidgetMe
    {
       return new QWidgetMetaData()
          .withName(NAME)
-         .withType(WidgetType.STEPPER.getType())
+         .withType(WidgetType.TABLE.getType())
          .withGridColumns(12)
+         .withPermissionRules(CrmPermissions.hasAccessRules(CrmPermissions.BASE))
          .withIsCard(true)
          .withLabel("Contact Timeline")
          .withTooltip("Chronological activity timeline for this contact")

src/main/java/com/kingsrook/qbits/crm/widgets/CrmDealForecastRenderer.java

@@ -1,189 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** Renderer for the CRM Deal Forecast widget.
- **
- ** Queries open deals (not in closed stages) and groups them by expected close
- ** month. Returns a table with columns: month, dealCount, totalAmount,
- ** totalWeightedAmount.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.widgets;
-
-
-import java.math.BigDecimal;
-import java.time.LocalDate;
-import java.time.YearMonth;
-import java.time.format.DateTimeFormatter;
-import java.util.ArrayList;
-import java.util.List;
-import java.util.Map;
-import java.util.Set;
-import java.util.TreeMap;
-import java.util.stream.Collectors;
-import com.kingsrook.qbits.crm.deals.model.Deal;
-import com.kingsrook.qbits.crm.deals.model.PipelineStage;
-import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
-import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
-import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
-import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.TableData;
-import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.utils.collections.MapBuilder;
-
-
-/*******************************************************************************
- ** Renders the deal forecast as a table widget showing open deal aggregates
- ** grouped by expected close month.
- *******************************************************************************/
-public class CrmDealForecastRenderer extends AbstractWidgetRenderer
-{
-   private static final DateTimeFormatter MONTH_FORMAT = DateTimeFormatter.ofPattern("yyyy-MM");
-
-
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Override
-   public RenderWidgetOutput render(RenderWidgetInput input) throws QException
-   {
-      ///////////////////////////////////////////////////////////////
-      // find all closed stage IDs to exclude                      //
-      ///////////////////////////////////////////////////////////////
-      Set<Integer> closedStageIds = findClosedStageIds();
-
-      ///////////////////////////////////////////////////////////////
-      // query all deals that are NOT in closed stages             //
-      ///////////////////////////////////////////////////////////////
-      QQueryFilter filter = new QQueryFilter();
-      if(!closedStageIds.isEmpty())
-      {
-         filter.withCriteria(new QFilterCriteria("pipelineStageId", QCriteriaOperator.NOT_IN, new ArrayList<>(closedStageIds)));
-      }
-
-      QueryOutput dealOutput = new QueryAction().execute(
-         new QueryInput(Deal.TABLE_NAME).withFilter(filter));
-
-      //////////////////////////////////////////////////////////
-      // aggregate deals by expected close month              //
-      //////////////////////////////////////////////////////////
-      Map<YearMonth, Integer> countByMonth          = new TreeMap<>();
-      Map<YearMonth, BigDecimal> amountByMonth      = new TreeMap<>();
-      Map<YearMonth, BigDecimal> weightedByMonth    = new TreeMap<>();
-
-      for(QRecord record : dealOutput.getRecords())
-      {
-         LocalDate expectedCloseDate = record.getValueLocalDate("expectedCloseDate");
-         if(expectedCloseDate == null)
-         {
-            continue;
-         }
-
-         YearMonth yearMonth = YearMonth.from(expectedCloseDate);
-
-         countByMonth.merge(yearMonth, 1, Integer::sum);
-
-         BigDecimal amount = record.getValueBigDecimal("amount");
-         if(amount != null)
-         {
-            amountByMonth.merge(yearMonth, amount, BigDecimal::add);
-         }
-
-         BigDecimal weighted = record.getValueBigDecimal("weightedAmount");
-         if(weighted != null)
-         {
-            weightedByMonth.merge(yearMonth, weighted, BigDecimal::add);
-         }
-      }
-
-      ///////////////////////////
-      // build table columns   //
-      ///////////////////////////
-      List<TableData.Column> columns = List.of(
-         new TableData.Column("html", "Month", "month", "1fr", null),
-         new TableData.Column("html", "Deal Count", "dealCount", "1fr", "right"),
-         new TableData.Column("html", "Total Amount", "totalAmount", "1fr", "right"),
-         new TableData.Column("html", "Weighted Amount", "totalWeightedAmount", "1fr", "right")
-      );
-
-      //////////////////////////
-      // build table rows     //
-      //////////////////////////
-      List<Map<String, Object>> rows = new ArrayList<>();
-      for(YearMonth yearMonth : countByMonth.keySet())
-      {
-         rows.add(MapBuilder.of(
-            "month", yearMonth.format(MONTH_FORMAT),
-            "dealCount", countByMonth.getOrDefault(yearMonth, 0),
-            "totalAmount", amountByMonth.getOrDefault(yearMonth, BigDecimal.ZERO).toPlainString(),
-            "totalWeightedAmount", weightedByMonth.getOrDefault(yearMonth, BigDecimal.ZERO).toPlainString()
-         ));
-      }
-
-      TableData tableData = new TableData(null, columns, rows)
-         .withRowsPerPage(50)
-         .withHidePaginationDropdown(true);
-
-      return (new RenderWidgetOutput(tableData));
-   }
-
-
-
-   /***************************************************************************
-    ** Find all pipeline stage IDs that are closed (won or lost).
-    ***************************************************************************/
-   private Set<Integer> findClosedStageIds() throws QException
-   {
-      QQueryFilter filter = new QQueryFilter()
-         .withCriteria(new QFilterCriteria("isClosedWon", QCriteriaOperator.EQUALS, true));
-
-      QQueryFilter lostFilter = new QQueryFilter()
-         .withCriteria(new QFilterCriteria("isClosedLost", QCriteriaOperator.EQUALS, true));
-
-      ///////////////////////////////////////////////////////////////////////////
-      // query stages that are closed won OR closed lost                       //
-      ///////////////////////////////////////////////////////////////////////////
-      QueryOutput wonOutput = new QueryAction().execute(
-         new QueryInput(PipelineStage.TABLE_NAME).withFilter(filter));
-
-      QueryOutput lostOutput = new QueryAction().execute(
-         new QueryInput(PipelineStage.TABLE_NAME).withFilter(lostFilter));
-
-      Set<Integer> closedIds = wonOutput.getRecords().stream()
-         .map(r -> r.getValueInteger("id"))
-         .collect(Collectors.toSet());
-
-      lostOutput.getRecords().stream()
-         .map(r -> r.getValueInteger("id"))
-         .forEach(closedIds::add);
-
-      return (closedIds);
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmDealForecastWidgetProducer.java

@@ -1,67 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** MetaData producer for the CRM Deal Forecast widget.
- **
- ** Provides a TABLE view showing open deals grouped by expected close month
- ** with counts, total amounts, and weighted amounts.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.widgets;
-
-
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducer;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
-
-
-/*******************************************************************************
- ** Produces metadata for the crm_deal_forecast widget (TABLE type).
- *******************************************************************************/
-public class CrmDealForecastWidgetProducer extends MetaDataProducer<QWidgetMetaData>
-{
-   public static final String NAME = "crm_deal_forecast";
-
-
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Override
-   public QWidgetMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QWidgetMetaData()
-         .withName(NAME)
-         .withType(WidgetType.TABLE.getType())
-         .withGridColumns(6)
-         .withIsCard(true)
-         .withLabel("Deal Forecast")
-         .withTooltip("Open deals grouped by expected close month")
-         .withShowReloadButton(true)
-         .withIcon("topRightInsideCard", new com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon("trending_up"))
-         .withCodeReference(new QCodeReference(CrmDealForecastRenderer.class));
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmGoalsVisibility.java

@@ -0,0 +1,72 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2026.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+package com.kingsrook.qbits.crm.widgets;
+
+
+import java.time.LocalDate;
+import java.time.ZoneOffset;
+import com.kingsrook.qbits.crm.security.CrmPermissions;
+import com.kingsrook.qqq.backend.core.actions.permissions.CustomPermissionChecker;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
+import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.metadata.permissions.MetaDataWithPermissionRules;
+
+
+/*******************************************************************************
+ ** Hide Goals in per-session frontend metadata unless a current goal is visible.
+ ** This is evaluated at request time, never while building shared app metadata.
+ *******************************************************************************/
+public class CrmGoalsVisibility implements CustomPermissionChecker
+{
+   /*******************************************************************************
+    ** Preserve crm.hasAccess and check existence through a bounded scoped query.
+    *******************************************************************************/
+   @Override
+   public void checkPermissionsThrowing(AbstractActionInput input, MetaDataWithPermissionRules metadata) throws QPermissionDeniedException
+   {
+      if(QContext.getQSession() == null || !QContext.getQSession().hasPermission(CrmPermissions.APP_ACCESS))
+      {
+         throw (new QPermissionDeniedException("CRM access is required."));
+      }
+      LocalDate today = LocalDate.now(ZoneOffset.UTC);
+      QQueryFilter filter = new QQueryFilter()
+         .withCriteria(new QFilterCriteria("periodStart", QCriteriaOperator.LESS_THAN_OR_EQUALS, today))
+         .withCriteria(new QFilterCriteria("periodEnd", QCriteriaOperator.GREATER_THAN_OR_EQUALS, today));
+      try
+      {
+         if(!CrmHomeRenderer.query("crm_sales_goal", filter, "id", true, 1).isEmpty())
+         {
+            return;
+         }
+      }
+      catch(QException e)
+      {
+         throw (new QPermissionDeniedException("Current goals are not available.", e));
+      }
+      throw (new QPermissionDeniedException("No goals for the current period."));
+   }
+}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmHomeRenderer.java

@@ -0,0 +1,496 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2026.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+package com.kingsrook.qbits.crm.widgets;
+
+
+import java.io.Serializable;
+import java.math.BigDecimal;
+import java.math.RoundingMode;
+import java.time.Instant;
+import java.time.LocalDate;
+import java.time.ZoneOffset;
+import java.time.temporal.ChronoUnit;
+import java.util.ArrayList;
+import java.util.Comparator;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import com.kingsrook.qbits.crm.core.model.enums.CrmFormConversionStatus;
+import com.kingsrook.qbits.crm.core.model.enums.CrmFormType;
+import com.kingsrook.qbits.crm.sync.CrmCurrencyUtils;
+import com.kingsrook.qqq.backend.core.actions.ai.output.AiLinkBuilder;
+import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
+import com.kingsrook.qqq.backend.core.actions.tables.AggregateAction;
+import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.Aggregate;
+import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateOperator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateResult;
+import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.GroupBy;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
+import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
+import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.ParentWidgetData;
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.StatisticsData;
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.TableData;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
+
+
+/*******************************************************************************
+ ** Daily CRM views. Core actions retain the current session and record locks.
+ ** Monetary totals use base currency; weighted amounts are grouped by currency
+ ** before conversion. No query loads all deals, contacts or submissions.
+ *******************************************************************************/
+public class CrmHomeRenderer extends AbstractWidgetRenderer
+{
+   private static final String DEAL = "crm_deal";
+   private static final String STAGE = "crm_pipeline_stage";
+   private static final Aggregate COUNT = new Aggregate("id", AggregateOperator.COUNT);
+   private static final Aggregate AMOUNT = new Aggregate("amountInBaseCurrency", AggregateOperator.SUM);
+   private static final Aggregate WEIGHTED = new Aggregate("weightedAmount", AggregateOperator.SUM);
+   private static final GroupBy CURRENCY = new GroupBy(QFieldType.STRING, "currencyCode");
+
+
+   /*******************************************************************************
+    ** Render only the requested home component using UTC reporting boundaries.
+    *******************************************************************************/
+   @Override
+   public RenderWidgetOutput render(RenderWidgetInput input) throws QException
+   {
+      LocalDate today = LocalDate.now(ZoneOffset.UTC);
+      switch(input.getWidgetMetaData().getName())
+      {
+         case "crm_today":
+            return (new RenderWidgetOutput(new ParentWidgetData().withChildWidgetNameList(CrmHomeWidgets.TODAY).withLabel("Today")));
+         case "crm_open_pipeline":
+            return (new RenderWidgetOutput(statistic(sum(DEAL, dealStatus(false), AMOUNT), true)));
+         case "crm_won_month":
+            return (new RenderWidgetOutput(statistic(sum(DEAL, period(dealStatus(true), "actualCloseDate", today.withDayOfMonth(1), today.plusDays(1)), AMOUNT), true)));
+         case "crm_new_contacts":
+            return (new RenderWidgetOutput(statistic(countSince("crm_contact", 30), false)));
+         case "crm_new_web_leads":
+            return (new RenderWidgetOutput(statistic(countSince("crm_form_submission", 7), false)));
+         case "crm_overdue_activities":
+            return (new RenderWidgetOutput(overdue()));
+         case "crm_pipeline_summary":
+            return (new RenderWidgetOutput(pipeline(input)));
+         case "crm_deal_forecast":
+            return (new RenderWidgetOutput(forecast(today)));
+         case "crm_rotting_deals":
+            return (new RenderWidgetOutput(rotting()));
+         case "crm_recent_web_leads":
+            return (new RenderWidgetOutput(webLeads()));
+         case "crm_goals":
+            return (new RenderWidgetOutput(goals(today)));
+         case "crm_sales_leaderboard":
+            return (new RenderWidgetOutput(leaderboard(today)));
+         default:
+            throw new QException("Unknown CRM home widget: " + input.getWidgetMetaData().getName());
+      }
+   }
+
+
+   /*******************************************************************************
+    ** Preserve decimals and large counts in the standard statistics renderer.
+    *******************************************************************************/
+   private StatisticsData statistic(BigDecimal value, boolean currency)
+   {
+      return (new StatisticsData().withCount(value).withCountContext(currency ? "base currency" : null));
+   }
+
+
+   /*******************************************************************************
+    ** Rolling counts include the present instant and exclude future records.
+    *******************************************************************************/
+   private BigDecimal countSince(String table, int days) throws QException
+   {
+      Instant now = Instant.now();
+      return (sum(table, period(new QQueryFilter(), "createDate", now.minus(days, ChronoUnit.DAYS), now), COUNT));
+   }
+
+
+   /*******************************************************************************
+    ** An unassigned activity is never treated as belonging to an anonymous user.
+    *******************************************************************************/
+   private StatisticsData overdue() throws QException
+   {
+      String user = QContext.getQSession().getIdReference();
+      if(user == null || user.isBlank())
+      {
+         return (statistic(BigDecimal.ZERO, false));
+      }
+      QQueryFilter filter = equalsFilter("assignedToUserId", user)
+         .withCriteria(new QFilterCriteria("isCompleted", QCriteriaOperator.NOT_EQUALS_OR_IS_NULL, true))
+         .withCriteria(new QFilterCriteria("dueDate", QCriteriaOperator.LESS_THAN, Instant.now()));
+      return (statistic(sum("crm_activity", filter, COUNT), false));
+   }
+
+
+   /*******************************************************************************
+    ** Open and won stages are resolved through the current session as well.
+    *******************************************************************************/
+   private QQueryFilter dealStatus(boolean won) throws QException
+   {
+      QQueryFilter filter = won ? equalsFilter("isClosedWon", true) : new QQueryFilter()
+         .withCriteria(new QFilterCriteria("isClosedWon", QCriteriaOperator.NOT_EQUALS_OR_IS_NULL, true))
+         .withCriteria(new QFilterCriteria("isClosedLost", QCriteriaOperator.NOT_EQUALS_OR_IS_NULL, true));
+      List<Integer> ids = allStages(filter).stream().map(row -> row.getValueInteger("id")).toList();
+      return (new QQueryFilter().withCriteria(new QFilterCriteria("pipelineStageId", QCriteriaOperator.IN, ids)));
+   }
+
+
+   /*******************************************************************************
+    ** Stage dictionaries are paged; transactional facts are aggregated in SQL.
+    *******************************************************************************/
+   private List<QRecord> allStages(QQueryFilter filter) throws QException
+   {
+      return (dictionary(STAGE, filter));
+   }
+
+
+   /*******************************************************************************
+    ** Read small settings dictionaries in deterministic keyset pages.
+    *******************************************************************************/
+   private List<QRecord> dictionary(String table, QQueryFilter filter) throws QException
+   {
+      List<QRecord> stages = new ArrayList<>();
+      int lastId = 0;
+      while(true)
+      {
+         QQueryFilter page = filter.clone().withCriteria(new QFilterCriteria("id", QCriteriaOperator.GREATER_THAN, lastId));
+         List<QRecord> records = query(table, page, "id", true, 200);
+         stages.addAll(records);
+         if(records.size() < 200)
+         {
+            return (stages);
+         }
+         lastId = records.getLast().getValueInteger("id");
+      }
+   }
+
+
+   /*******************************************************************************
+    ** Stage and deal count share one column, keeping both monetary columns.
+    *******************************************************************************/
+   private TableData pipeline(RenderWidgetInput input) throws QException
+   {
+      List<QRecord> pipelines = dictionary("crm_pipeline", equalsFilter("isActive", true));
+      List<Map<String, Object>> rows = new ArrayList<>();
+      String selected = pipelines.isEmpty() ? null : pipelines.getFirst().getValueString("id");
+      String requested = input.getQueryParams().get("pipelineId");
+      if(pipelines.stream().anyMatch(pipeline -> pipeline.getValueString("id").equals(requested)))
+      {
+         selected = requested;
+      }
+      if(!pipelines.isEmpty())
+      {
+         Integer pipelineId = Integer.valueOf(selected);
+         List<QRecord> stages = allStages(equalsFilter("pipelineId", pipelineId));
+         stages.sort(Comparator.comparing(row -> Objects.requireNonNullElse(row.getValueInteger("sortOrder"), 0)));
+         GroupBy stageGroup = new GroupBy(QFieldType.INTEGER, "pipelineStageId");
+         Map<Serializable, BigDecimal[]> totals = groupedTotals(equalsFilter("pipelineId", pipelineId), stageGroup);
+         for(QRecord stage : stages)
+         {
+            BigDecimal[] values = totals.getOrDefault(stage.getValue("id"), new BigDecimal[] {BigDecimal.ZERO, BigDecimal.ZERO, BigDecimal.ZERO});
+            String href = AiLinkBuilder.recordHref(STAGE, stage.getValue("id"));
+            rows.add(row(href, stage.getValueString("name") + " (" + values[0].toPlainString() + " deals)", values[1], values[2]));
+         }
+      }
+      TableData data = table(rows, "Stage / deals", "Amount (base)", "Weighted amount (base)");
+      if(pipelines.size() > 1)
+      {
+         data.withDropdownNameList(List.of("pipelineId")).withDropdownLabelList(List.of("Pipeline"))
+            .withDropdownDefaultValueList(List.of(selected))
+            .withDropdownDataList(List.of(pipelines.stream().map(pipeline -> Map.of("id", pipeline.getValueString("id"),
+               "label", Objects.toString(pipeline.getValue("name"), "Pipeline " + pipeline.getValueString("id")))).toList()));
+      }
+      return (data);
+   }
+
+
+   /*******************************************************************************
+    ** Six calendar buckets, including the current month, with empty months kept.
+    *******************************************************************************/
+   private TableData forecast(LocalDate today) throws QException
+   {
+      List<Map<String, Object>> rows = new ArrayList<>();
+      for(int month = 0; month < 6; month++)
+      {
+         LocalDate start = today.withDayOfMonth(1).plusMonths(month);
+         QQueryFilter filter = period(dealStatus(false), "expectedCloseDate", start, start.plusMonths(1));
+         BigDecimal[] totals = totals(filter);
+         rows.add(row(AiLinkBuilder.filteredTableHref(DEAL, filter), start.toString().substring(0, 7), totals[0], totals[2]));
+      }
+      return (table(rows, "Month", "Deals", "Weighted amount (base)"));
+   }
+
+
+   /*******************************************************************************
+    ** Limit the candidates in each stage in SQL, then select the oldest ten.
+    *******************************************************************************/
+   private TableData rotting() throws QException
+   {
+      List<QRecord> candidates = new ArrayList<>();
+      Map<Integer, String> stages = new LinkedHashMap<>();
+      Instant now = Instant.now();
+      QQueryFilter stageFilter = new QQueryFilter()
+         .withCriteria(new QFilterCriteria("isClosedWon", QCriteriaOperator.NOT_EQUALS_OR_IS_NULL, true))
+         .withCriteria(new QFilterCriteria("isClosedLost", QCriteriaOperator.NOT_EQUALS_OR_IS_NULL, true))
+         .withCriteria(new QFilterCriteria("rotDays", QCriteriaOperator.GREATER_THAN, 0));
+      for(QRecord stage : allStages(stageFilter))
+      {
+         Integer id = stage.getValueInteger("id");
+         stages.put(id, stage.getValueString("name"));
+         QQueryFilter filter = equalsFilter("pipelineStageId", id)
+            .withCriteria(new QFilterCriteria("stageEnteredDate", QCriteriaOperator.LESS_THAN, now.minus(stage.getValueInteger("rotDays"), ChronoUnit.DAYS)));
+         candidates.addAll(query(DEAL, filter, "stageEnteredDate", true, 10));
+      }
+      candidates.sort(Comparator.comparing(row -> row.getValueInstant("stageEnteredDate")));
+      List<Map<String, Object>> rows = new ArrayList<>();
+      for(QRecord deal : candidates.stream().limit(10).toList())
+      {
+         rows.add(row(AiLinkBuilder.recordHref(DEAL, deal.getValue("id")), deal.getValueString("name"),
+            stages.get(deal.getValueInteger("pipelineStageId")), ChronoUnit.DAYS.between(deal.getValueInstant("stageEnteredDate"), now)));
+      }
+      return (table(rows, "Deal", "Stage", "Days in stage"));
+   }
+
+
+   /*******************************************************************************
+    ** Latest ten submissions, with escaped labels and links on every cell.
+    *******************************************************************************/
+   private TableData webLeads() throws QException
+   {
+      List<Map<String, Object>> rows = new ArrayList<>();
+      for(QRecord lead : query("crm_form_submission", new QQueryFilter(), "createDate", false, 10))
+      {
+         String name = (Objects.toString(lead.getValue("firstName"), "") + " " + Objects.toString(lead.getValue("lastName"), "")).trim();
+         if(name.isBlank())
+         {
+            name = Objects.toString(lead.getValue("email"), "Web lead");
+         }
+         String form = "Unknown";
+         for(CrmFormType value : CrmFormType.values())
+         {
+            if(value.getId().equals(lead.getValueInteger("formType")))
+            {
+               form = value.getLabel();
+            }
+         }
+         String status = "New";
+         for(CrmFormConversionStatus value : CrmFormConversionStatus.values())
+         {
+            if(value.getId().equals(lead.getValueInteger("conversionStatus")))
+            {
+               status = value.getLabel();
+            }
+         }
+         rows.add(row(AiLinkBuilder.recordHref("crm_form_submission", lead.getValue("id")), name, form, status));
+      }
+      return (table(rows, "Name or email", "Form", "Status"));
+   }
+
+
+   /*******************************************************************************
+    ** Current goals only; metadata visibility hides the widget when none exist.
+    *******************************************************************************/
+   private TableData goals(LocalDate today) throws QException
+   {
+      QQueryFilter filter = new QQueryFilter()
+         .withCriteria(new QFilterCriteria("periodStart", QCriteriaOperator.LESS_THAN_OR_EQUALS, today))
+         .withCriteria(new QFilterCriteria("periodEnd", QCriteriaOperator.GREATER_THAN_OR_EQUALS, today));
+      List<Map<String, Object>> rows = new ArrayList<>();
+      for(QRecord goal : query("crm_sales_goal", filter, "id", true, 50))
+      {
+         QQueryFilter deals = period(dealStatus(true), "actualCloseDate", goal.getValueLocalDate("periodStart"), goal.getValueLocalDate("periodEnd").plusDays(1));
+         deals.withCriteria(new QFilterCriteria("ownerUserId", QCriteriaOperator.EQUALS, goal.getValue("userId")));
+         if(goal.getValue("pipelineId") != null)
+         {
+            deals.withCriteria(new QFilterCriteria("pipelineId", QCriteriaOperator.EQUALS, goal.getValue("pipelineId")));
+         }
+         BigDecimal actual = sum(DEAL, deals, AMOUNT);
+         BigDecimal target = CrmCurrencyUtils.convertToBaseCurrency(goal.getValueBigDecimal("targetAmount"), goal.getValueString("currencyCode"));
+         String attainment = target != null && target.signum() > 0 ? actual.multiply(BigDecimal.valueOf(100)).divide(target, 1, RoundingMode.HALF_UP) + "%" : "No target";
+         rows.add(row(AiLinkBuilder.recordHref("crm_sales_goal", goal.getValue("id")),
+            goal.getValueString("userId") + " (" + attainment + ")", target, actual));
+      }
+      TableData data = table(rows, "Owner / attainment", "Target (base)", "Won (base)");
+      return (data);
+   }
+
+
+   /*******************************************************************************
+    ** Quarter-to-date won revenue grouped by owner, linking to matching deals.
+    *******************************************************************************/
+   private TableData leaderboard(LocalDate today) throws QException
+   {
+      LocalDate start = today.withMonth(((today.getMonthValue() - 1) / 3) * 3 + 1).withDayOfMonth(1);
+      QQueryFilter filter = period(dealStatus(true), "actualCloseDate", start, today.plusDays(1));
+      GroupBy owner = new GroupBy(QFieldType.STRING, "ownerUserId");
+      List<AggregateResult> results = aggregate(DEAL, filter, List.of(COUNT, AMOUNT), List.of(owner));
+      results.sort(Comparator.comparing((AggregateResult result) -> number(result.getAggregateValue(AMOUNT))).reversed());
+      List<Map<String, Object>> rows = new ArrayList<>();
+      for(AggregateResult result : results)
+      {
+         Serializable user = result.getGroupByValue(owner);
+         QQueryFilter ownerFilter = filter.clone().withCriteria(new QFilterCriteria("ownerUserId", user == null ? QCriteriaOperator.IS_BLANK : QCriteriaOperator.EQUALS, user));
+         rows.add(row(AiLinkBuilder.filteredTableHref(DEAL, ownerFilter), Objects.toString(user, "Unassigned"),
+            number(result.getAggregateValue(COUNT)), number(result.getAggregateValue(AMOUNT))));
+      }
+      return (table(rows, "Owner", "Won deals", "Won amount (base)"));
+   }
+
+
+   /*******************************************************************************
+    ** Aggregate currency groups instead of loading the underlying deals.
+    *******************************************************************************/
+   private Map<Serializable, BigDecimal[]> groupedTotals(QQueryFilter filter, GroupBy group) throws QException
+   {
+      List<GroupBy> groups = new ArrayList<>(List.of(CURRENCY));
+      if(group != null)
+      {
+         groups.add(group);
+      }
+      Map<Serializable, BigDecimal[]> totals = new LinkedHashMap<>();
+      for(AggregateResult result : aggregate(DEAL, filter, List.of(COUNT, AMOUNT, WEIGHTED), groups))
+      {
+         Serializable key = group == null ? "total" : result.getGroupByValue(group);
+         BigDecimal[] values = totals.computeIfAbsent(key, ignored -> new BigDecimal[] {BigDecimal.ZERO, BigDecimal.ZERO, BigDecimal.ZERO});
+         values[0] = values[0].add(number(result.getAggregateValue(COUNT)));
+         values[1] = values[1].add(number(result.getAggregateValue(AMOUNT)));
+         values[2] = values[2].add(CrmCurrencyUtils.convertToBaseCurrency(number(result.getAggregateValue(WEIGHTED)), (String) result.getGroupByValue(CURRENCY)));
+      }
+      return (totals);
+   }
+
+
+   /*******************************************************************************
+    ** Totals across all currency buckets.
+    *******************************************************************************/
+   private BigDecimal[] totals(QQueryFilter filter) throws QException
+   {
+      return (groupedTotals(filter, null).getOrDefault("total", new BigDecimal[] {BigDecimal.ZERO, BigDecimal.ZERO, BigDecimal.ZERO}));
+   }
+
+
+   /*******************************************************************************
+    ** A scalar aggregate with an explicit zero for an empty selection.
+    *******************************************************************************/
+   private BigDecimal sum(String table, QQueryFilter filter, Aggregate aggregate) throws QException
+   {
+      List<AggregateResult> results = aggregate(table, filter, List.of(aggregate), List.of());
+      return (results.isEmpty() ? BigDecimal.ZERO : number(results.getFirst().getAggregateValue(aggregate)));
+   }
+
+
+   /*******************************************************************************
+    ** Delegate all permission and record-lock filtering to the core action.
+    *******************************************************************************/
+   private List<AggregateResult> aggregate(String table, QQueryFilter filter, List<Aggregate> aggregates, List<GroupBy> groups) throws QException
+   {
+      return (new AggregateAction().execute(new AggregateInput(table).withFilter(filter).withAggregates(aggregates).withGroupBys(groups)).getResults());
+   }
+
+
+   /*******************************************************************************
+    ** Normalize aggregate values without lossy integer conversions.
+    *******************************************************************************/
+   private BigDecimal number(Serializable value)
+   {
+      return (value == null ? BigDecimal.ZERO : new BigDecimal(value.toString()));
+   }
+
+
+   /*******************************************************************************
+    ** A bounded query with deterministic ordering for tied timestamps.
+    *******************************************************************************/
+   static List<QRecord> query(String table, QQueryFilter filter, String order, boolean ascending, int limit) throws QException
+   {
+      return (new QueryAction().execute(new QueryInput(table).withFilter(filter.withOrderBy(new QFilterOrderBy(order, ascending))
+         .withOrderBy(new QFilterOrderBy("id", ascending)).withLimit(limit))).getRecords());
+   }
+
+
+   /*******************************************************************************
+    ** Equality filter.
+    *******************************************************************************/
+   static QQueryFilter equalsFilter(String field, Serializable value)
+   {
+      return (new QQueryFilter(new QFilterCriteria(field, QCriteriaOperator.EQUALS, value)));
+   }
+
+
+   /*******************************************************************************
+    ** Half-open reporting interval, avoiding end-of-month time rounding.
+    *******************************************************************************/
+   private QQueryFilter period(QQueryFilter filter, String field, Serializable start, Serializable end)
+   {
+      return (filter.withCriteria(new QFilterCriteria(field, QCriteriaOperator.GREATER_THAN_OR_EQUALS, start))
+         .withCriteria(new QFilterCriteria(field, QCriteriaOperator.LESS_THAN, end)));
+   }
+
+
+   /*******************************************************************************
+    ** Every cell navigates to its record, or to the records in an aggregate.
+    *******************************************************************************/
+   static Map<String, Object> row(String href, Object first, Object second, Object third)
+   {
+      return (Map.of("first", link(href, first), "second", link(href, second), "third", link(href, third)));
+   }
+
+
+   /*******************************************************************************
+    ** Standard table cells may contain links; user text is always escaped.
+    *******************************************************************************/
+   private static String link(String href, Object label)
+   {
+      return ("<a href=\"" + escape(href) + "\">" + escape(Objects.toString(label, "")) + "</a>");
+   }
+
+
+   /*******************************************************************************
+    ** Escape text and attributes in links.
+    *******************************************************************************/
+   private static String escape(String value)
+   {
+      return (value.replace("&", "&amp;").replace("<", "&lt;").replace(">", "&gt;").replace("\"", "&quot;").replace("'", "&#39;"));
+   }
+
+
+   /*******************************************************************************
+    ** Three columns, full-width metadata, and the standard paginated table.
+    *******************************************************************************/
+   static TableData table(List<Map<String, Object>> rows, String first, String second, String third)
+   {
+      return (new TableData(null, List.of(new TableData.Column("html", first, "first", "2fr", "left"),
+         new TableData.Column("html", second, "second", "1fr", "left"), new TableData.Column("html", third, "third", "1fr", "left")), rows)
+         .withRowsPerPage(10).withHidePaginationDropdown(true));
+   }
+}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmHomeWidgets.java

@@ -0,0 +1,81 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2026.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+package com.kingsrook.qbits.crm.widgets;
+
+
+import java.util.List;
+import com.kingsrook.qbits.crm.security.CrmPermissions;
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
+import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducer;
+import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerMultiOutput;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
+import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
+
+
+/*******************************************************************************
+ ** Standard widgets for the single CRM home. Today holds five statistics.
+ *******************************************************************************/
+public class CrmHomeWidgets extends MetaDataProducer<MetaDataProducerMultiOutput>
+{
+   public static final List<String> HOME = List.of("crm_today", "crm_pipeline_summary", "crm_deal_forecast", "crm_rotting_deals",
+      "crm_recent_web_leads", "crm_goals", "crm_sales_leaderboard");
+   public static final List<String> TODAY = List.of("crm_open_pipeline", "crm_won_month", "crm_new_contacts", "crm_new_web_leads", "crm_overdue_activities");
+
+
+   /*******************************************************************************
+    ** All metrics and tables use the same session-aware data renderer.
+    *******************************************************************************/
+   @Override
+   public MetaDataProducerMultiOutput produce(QInstance instance)
+   {
+      MetaDataProducerMultiOutput output = new MetaDataProducerMultiOutput();
+      List<String> labels = List.of("Today", "Pipeline by stage", "Forecast", "Needs attention", "Recent web leads", "Goals", "Leaderboard");
+      for(int i = 0; i < HOME.size(); i++)
+      {
+         output.add(widget(HOME.get(i), labels.get(i), i == 0 ? WidgetType.PARENT_WIDGET : WidgetType.TABLE));
+      }
+      List<String> statistics = List.of("Open pipeline (base currency)", "Won this month (base currency)", "New contacts (30 days)", "Web leads (7 days)", "My overdue activities");
+      for(int i = 0; i < TODAY.size(); i++)
+      {
+         output.add(widget(TODAY.get(i), statistics.get(i), WidgetType.STATISTICS));
+      }
+      return (output);
+   }
+
+
+   /*******************************************************************************
+    ** A full-width, permission-protected standard widget.
+    *******************************************************************************/
+   private QWidgetMetaData widget(String name, String label, WidgetType type)
+   {
+      QWidgetMetaData widget = new QWidgetMetaData().withName(name).withLabel(label).withType(type.getType()).withGridColumns(12)
+         .withIsCard(type != WidgetType.PARENT_WIDGET).withShowReloadButton(true)
+         .withPermissionRules(CrmPermissions.hasAccessRules(CrmPermissions.BASE))
+         .withCodeReference(new QCodeReference(CrmHomeRenderer.class));
+      if("crm_goals".equals(name))
+      {
+         widget.getPermissionRules().withCustomPermissionChecker(new QCodeReference(CrmGoalsVisibility.class));
+      }
+      return (widget);
+   }
+}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmPipelineConversionRenderer.java

@@ -1,144 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** Renderer for the Pipeline Conversion widget.
- **
- ** Queries DealStageHistory, counts transitions into each stage (entered)
- ** versus transitions out (progressed), and computes a conversion rate.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.widgets;
-
-
-import java.math.BigDecimal;
-import java.math.RoundingMode;
-import java.util.ArrayList;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
-import com.kingsrook.qbits.crm.deals.model.DealStageHistory;
-import com.kingsrook.qbits.crm.deals.model.PipelineStage;
-import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
-import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
-import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
-import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.TableData;
-import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.utils.collections.MapBuilder;
-
-
-/*******************************************************************************
- ** Renders pipeline conversion data as a table widget.
- *******************************************************************************/
-public class CrmPipelineConversionRenderer extends AbstractWidgetRenderer
-{
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Override
-   public RenderWidgetOutput render(RenderWidgetInput input) throws QException
-   {
-      ///////////////////////////////////////////////////////////
-      // query all deal stage history                          //
-      ///////////////////////////////////////////////////////////
-      QueryOutput historyOutput = new QueryAction().execute(
-         new QueryInput(DealStageHistory.TABLE_NAME));
-
-      ///////////////////////////////////////////////////////////
-      // count entered and progressed for each toStageId       //
-      ///////////////////////////////////////////////////////////
-      Map<Integer, Integer> enteredByStage    = new HashMap<>();
-      Map<Integer, Integer> progressedByStage = new HashMap<>();
-
-      for(QRecord record : historyOutput.getRecords())
-      {
-         Integer toStageId   = record.getValueInteger("toStageId");
-         Integer fromStageId = record.getValueInteger("fromStageId");
-
-         if(toStageId != null)
-         {
-            enteredByStage.merge(toStageId, 1, Integer::sum);
-         }
-         if(fromStageId != null)
-         {
-            progressedByStage.merge(fromStageId, 1, Integer::sum);
-         }
-      }
-
-      ///////////////////////////////////////////////////////////
-      // build stage name lookup                               //
-      ///////////////////////////////////////////////////////////
-      QueryOutput stageOutput = new QueryAction().execute(
-         new QueryInput(PipelineStage.TABLE_NAME));
-
-      Map<Integer, String> stageNames = new HashMap<>();
-      for(QRecord record : stageOutput.getRecords())
-      {
-         stageNames.put(record.getValueInteger("id"), record.getValueString("name"));
-      }
-
-      ///////////////////////////
-      // build table columns   //
-      ///////////////////////////
-      List<TableData.Column> columns = List.of(
-         new TableData.Column("html", "Stage", "stageName", "2fr", null),
-         new TableData.Column("html", "Entered", "entered", "1fr", "right"),
-         new TableData.Column("html", "Progressed", "progressed", "1fr", "right"),
-         new TableData.Column("html", "Conversion %", "conversionRate", "1fr", "right")
-      );
-
-      //////////////////////////
-      // build table rows     //
-      //////////////////////////
-      List<Map<String, Object>> rows = new ArrayList<>();
-      for(Map.Entry<Integer, Integer> entry : enteredByStage.entrySet())
-      {
-         Integer stageId    = entry.getKey();
-         int     entered    = entry.getValue();
-         int     progressed = progressedByStage.getOrDefault(stageId, 0);
-
-         BigDecimal conversionRate = entered > 0
-            ? BigDecimal.valueOf(progressed).multiply(BigDecimal.valueOf(100)).divide(BigDecimal.valueOf(entered), 1, RoundingMode.HALF_UP)
-            : BigDecimal.ZERO;
-
-         String stageName = stageNames.getOrDefault(stageId, "Stage " + stageId);
-
-         rows.add(MapBuilder.of(
-            "stageName", stageName,
-            "entered", entered,
-            "progressed", progressed,
-            "conversionRate", conversionRate + "%"
-         ));
-      }
-
-      TableData tableData = new TableData(null, columns, rows)
-         .withRowsPerPage(25)
-         .withHidePaginationDropdown(true);
-
-      return (new RenderWidgetOutput(tableData));
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmPipelineConversionWidgetProducer.java

@@ -1,67 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** MetaData producer for the Pipeline Conversion widget.
- **
- ** Shows conversion rates between pipeline stages based on DealStageHistory
- ** transition data.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.widgets;
-
-
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducer;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
-
-
-/*******************************************************************************
- ** Produces metadata for the crm_pipeline_conversion widget (TABLE type).
- *******************************************************************************/
-public class CrmPipelineConversionWidgetProducer extends MetaDataProducer<QWidgetMetaData>
-{
-   public static final String NAME = "crm_pipeline_conversion";
-
-
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Override
-   public QWidgetMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QWidgetMetaData()
-         .withName(NAME)
-         .withType(WidgetType.TABLE.getType())
-         .withGridColumns(6)
-         .withIsCard(true)
-         .withLabel("Pipeline Conversion")
-         .withTooltip("Stage transition conversion rates")
-         .withShowReloadButton(true)
-         .withIcon("topRightInsideCard", new com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon("funnel_chart"))
-         .withCodeReference(new QCodeReference(CrmPipelineConversionRenderer.class));
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmPipelineSummaryRenderer.java

@@ -1,179 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** Renderer for the CRM Pipeline Summary widget.
- **
- ** Queries deals grouped by pipelineStageId for a given pipelineId and returns
- ** a table with columns: stageName, dealCount, totalAmount, weightedAmount.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.widgets;
-
-
-import java.math.BigDecimal;
-import java.util.ArrayList;
-import java.util.LinkedHashMap;
-import java.util.List;
-import java.util.Map;
-import com.kingsrook.qbits.crm.deals.model.Deal;
-import com.kingsrook.qbits.crm.deals.model.PipelineStage;
-import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
-import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
-import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
-import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.TableData;
-import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.utils.collections.MapBuilder;
-
-
-/*******************************************************************************
- ** Renders the pipeline summary as a table widget showing deal counts and
- ** amounts grouped by pipeline stage.
- *******************************************************************************/
-public class CrmPipelineSummaryRenderer extends AbstractWidgetRenderer
-{
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Override
-   public RenderWidgetOutput render(RenderWidgetInput input) throws QException
-   {
-      ///////////////////////////////////////////////////////////////
-      // get pipelineId from widget input query params             //
-      ///////////////////////////////////////////////////////////////
-      Integer pipelineId = null;
-      if(input.getQueryParams() != null && input.getQueryParams().containsKey("pipelineId"))
-      {
-         pipelineId = Integer.valueOf(input.getQueryParams().get("pipelineId"));
-      }
-
-      ///////////////////////////////////////////////////////////////
-      // query all stages for the pipeline, ordered by sortOrder   //
-      ///////////////////////////////////////////////////////////////
-      QQueryFilter stageFilter = new QQueryFilter()
-         .withOrderBy(new QFilterOrderBy("sortOrder", true));
-      if(pipelineId != null)
-      {
-         stageFilter.withCriteria(new QFilterCriteria("pipelineId", QCriteriaOperator.EQUALS, pipelineId));
-      }
-
-      QueryOutput stageOutput = new QueryAction().execute(
-         new QueryInput(PipelineStage.TABLE_NAME).withFilter(stageFilter));
-
-      //////////////////////////////////////////////////////////
-      // build a map of stageId -> stageName for display      //
-      //////////////////////////////////////////////////////////
-      Map<Integer, String> stageNames = new LinkedHashMap<>();
-      for(QRecord stageRecord : stageOutput.getRecords())
-      {
-         stageNames.put(stageRecord.getValueInteger("id"), stageRecord.getValueString("name"));
-      }
-
-      ///////////////////////////////////////////////////////////////
-      // query deals for the pipeline                              //
-      ///////////////////////////////////////////////////////////////
-      QQueryFilter dealFilter = new QQueryFilter();
-      if(pipelineId != null)
-      {
-         dealFilter.withCriteria(new QFilterCriteria("pipelineId", QCriteriaOperator.EQUALS, pipelineId));
-      }
-
-      QueryOutput dealOutput = new QueryAction().execute(
-         new QueryInput(Deal.TABLE_NAME).withFilter(dealFilter));
-
-      //////////////////////////////////////////////////////////
-      // aggregate deals by stage                             //
-      //////////////////////////////////////////////////////////
-      Map<Integer, Integer> countByStage          = new LinkedHashMap<>();
-      Map<Integer, BigDecimal> totalAmountByStage = new LinkedHashMap<>();
-      Map<Integer, BigDecimal> weightedByStage    = new LinkedHashMap<>();
-
-      for(Integer stageId : stageNames.keySet())
-      {
-         countByStage.put(stageId, 0);
-         totalAmountByStage.put(stageId, BigDecimal.ZERO);
-         weightedByStage.put(stageId, BigDecimal.ZERO);
-      }
-
-      for(QRecord dealRecord : dealOutput.getRecords())
-      {
-         Integer stageId = dealRecord.getValueInteger("pipelineStageId");
-         if(stageId == null || !stageNames.containsKey(stageId))
-         {
-            continue;
-         }
-
-         countByStage.merge(stageId, 1, Integer::sum);
-
-         BigDecimal amount = dealRecord.getValueBigDecimal("amount");
-         if(amount != null)
-         {
-            totalAmountByStage.merge(stageId, amount, BigDecimal::add);
-         }
-
-         BigDecimal weighted = dealRecord.getValueBigDecimal("weightedAmount");
-         if(weighted != null)
-         {
-            weightedByStage.merge(stageId, weighted, BigDecimal::add);
-         }
-      }
-
-      ///////////////////////////
-      // build table columns   //
-      ///////////////////////////
-      List<TableData.Column> columns = List.of(
-         new TableData.Column("html", "Stage", "stageName", "2fr", null),
-         new TableData.Column("html", "Deal Count", "dealCount", "1fr", "right"),
-         new TableData.Column("html", "Total Amount", "totalAmount", "1fr", "right"),
-         new TableData.Column("html", "Weighted Amount", "weightedAmount", "1fr", "right")
-      );
-
-      //////////////////////////
-      // build table rows     //
-      //////////////////////////
-      List<Map<String, Object>> rows = new ArrayList<>();
-      for(Integer stageId : stageNames.keySet())
-      {
-         rows.add(MapBuilder.of(
-            "stageName", stageNames.get(stageId),
-            "dealCount", countByStage.getOrDefault(stageId, 0),
-            "totalAmount", totalAmountByStage.getOrDefault(stageId, BigDecimal.ZERO).toPlainString(),
-            "weightedAmount", weightedByStage.getOrDefault(stageId, BigDecimal.ZERO).toPlainString()
-         ));
-      }
-
-      TableData tableData = new TableData(null, columns, rows)
-         .withRowsPerPage(50)
-         .withHidePaginationDropdown(true);
-
-      return (new RenderWidgetOutput(tableData));
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmPipelineSummaryWidgetProducer.java

@@ -1,67 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** MetaData producer for the CRM Pipeline Summary widget.
- **
- ** Provides a TABLE view showing deal counts and amounts grouped by pipeline
- ** stage for a given pipeline.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.widgets;
-
-
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducer;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
-
-
-/*******************************************************************************
- ** Produces metadata for the crm_pipeline_summary widget (TABLE type).
- *******************************************************************************/
-public class CrmPipelineSummaryWidgetProducer extends MetaDataProducer<QWidgetMetaData>
-{
-   public static final String NAME = "crm_pipeline_summary";
-
-
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Override
-   public QWidgetMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QWidgetMetaData()
-         .withName(NAME)
-         .withType(WidgetType.TABLE.getType())
-         .withGridColumns(6)
-         .withIsCard(true)
-         .withLabel("Pipeline Summary")
-         .withTooltip("Deal counts and amounts by pipeline stage")
-         .withShowReloadButton(true)
-         .withIcon("topRightInsideCard", new com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon("linear_scale"))
-         .withCodeReference(new QCodeReference(CrmPipelineSummaryRenderer.class));
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmQuotaAttainmentRenderer.java

@@ -1,171 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** Renderer for the CRM Quota Attainment widget.
- **
- ** Queries the SalesGoal for the current user and current period, then queries
- ** won deals in that period to compute attainment percentage.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.widgets;
-
-
-import java.math.BigDecimal;
-import java.math.RoundingMode;
-import java.time.LocalDate;
-import java.util.ArrayList;
-import java.util.List;
-import com.kingsrook.qbits.crm.deals.model.Deal;
-import com.kingsrook.qbits.crm.deals.model.PipelineStage;
-import com.kingsrook.qbits.crm.scoring.model.SalesGoal;
-import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
-import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
-import com.kingsrook.qqq.backend.core.context.QContext;
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
-import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
-import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.MultiStatisticsData;
-import com.kingsrook.qqq.backend.core.model.data.QRecord;
-
-
-/*******************************************************************************
- ** Renders quota attainment as multi-statistics showing target, actual, and
- ** attainment percentage.
- *******************************************************************************/
-public class CrmQuotaAttainmentRenderer extends AbstractWidgetRenderer
-{
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Override
-   public RenderWidgetOutput render(RenderWidgetInput input) throws QException
-   {
-      String currentUserId = QContext.getQSession() != null
-         ? QContext.getQSession().getIdReference() : null;
-
-      LocalDate today = LocalDate.now();
-
-      ///////////////////////////////////////////////
-      // query SalesGoal for current user/period   //
-      ///////////////////////////////////////////////
-      QueryOutput goalQuery = new QueryAction().execute(
-         new QueryInput(SalesGoal.TABLE_NAME)
-            .withFilter(new QQueryFilter()
-               .withCriteria(new QFilterCriteria("userId", QCriteriaOperator.EQUALS, currentUserId))
-               .withCriteria(new QFilterCriteria("periodStart", QCriteriaOperator.LESS_THAN_OR_EQUALS, today))
-               .withCriteria(new QFilterCriteria("periodEnd", QCriteriaOperator.GREATER_THAN_OR_EQUALS, today))));
-
-      List<MultiStatisticsData.StatisticsGroupData> groups = new ArrayList<>();
-
-      if(goalQuery.getRecords().isEmpty())
-      {
-         ///////////////////////////////////////////////
-         // no goal found -- show empty state         //
-         ///////////////////////////////////////////////
-         groups.add(new MultiStatisticsData.StatisticsGroupData()
-            .withHeader("No Goal Set")
-            .withSubheader("current period")
-            .withStatisticList(List.of(
-               new MultiStatisticsData.StatisticsGroupData.Statistic("Attainment", 0, "%")
-            )));
-      }
-      else
-      {
-         SalesGoal goal = new SalesGoal(goalQuery.getRecords().get(0));
-
-         ///////////////////////////////////////////////
-         // find closed-won stage ids                 //
-         ///////////////////////////////////////////////
-         QueryOutput stageQuery = new QueryAction().execute(
-            new QueryInput(PipelineStage.TABLE_NAME)
-               .withFilter(new QQueryFilter()
-                  .withCriteria(new QFilterCriteria("isClosedWon", QCriteriaOperator.EQUALS, true))));
-
-         List<Integer> closedWonStageIds = stageQuery.getRecords().stream()
-            .map(r -> r.getValueInteger("id"))
-            .toList();
-
-         ///////////////////////////////////////////////
-         // query won deals in the period             //
-         ///////////////////////////////////////////////
-         BigDecimal totalWon = BigDecimal.ZERO;
-
-         if(!closedWonStageIds.isEmpty())
-         {
-            QQueryFilter dealFilter = new QQueryFilter()
-               .withCriteria(new QFilterCriteria("ownerUserId", QCriteriaOperator.EQUALS, currentUserId))
-               .withCriteria(new QFilterCriteria("pipelineStageId", QCriteriaOperator.IN, closedWonStageIds))
-               .withCriteria(new QFilterCriteria("actualCloseDate", QCriteriaOperator.GREATER_THAN_OR_EQUALS, goal.getPeriodStart()))
-               .withCriteria(new QFilterCriteria("actualCloseDate", QCriteriaOperator.LESS_THAN_OR_EQUALS, goal.getPeriodEnd()));
-
-            if(goal.getPipelineId() != null)
-            {
-               dealFilter.withCriteria(new QFilterCriteria("pipelineId", QCriteriaOperator.EQUALS, goal.getPipelineId()));
-            }
-
-            QueryOutput dealQuery = new QueryAction().execute(
-               new QueryInput(Deal.TABLE_NAME).withFilter(dealFilter));
-
-            for(QRecord dealRecord : dealQuery.getRecords())
-            {
-               BigDecimal amount = dealRecord.getValueBigDecimal("amountInBaseCurrency");
-               if(amount != null)
-               {
-                  totalWon = totalWon.add(amount);
-               }
-            }
-         }
-
-         ///////////////////////////////////////////////
-         // compute attainment percentage             //
-         ///////////////////////////////////////////////
-         int attainmentPct = 0;
-         if(goal.getTargetAmount() != null && goal.getTargetAmount().compareTo(BigDecimal.ZERO) > 0)
-         {
-            attainmentPct = totalWon
-               .multiply(new BigDecimal(100))
-               .divide(goal.getTargetAmount(), 0, RoundingMode.HALF_UP)
-               .intValue();
-         }
-
-         groups.add(new MultiStatisticsData.StatisticsGroupData()
-            .withHeader("Quota Attainment")
-            .withSubheader(goal.getPeriodStart() + " to " + goal.getPeriodEnd())
-            .withStatisticList(List.of(
-               new MultiStatisticsData.StatisticsGroupData.Statistic("Target", goal.getTargetAmount().intValue(), null),
-               new MultiStatisticsData.StatisticsGroupData.Statistic("Actual", totalWon.intValue(), null),
-               new MultiStatisticsData.StatisticsGroupData.Statistic("Attainment", attainmentPct, "%")
-            )));
-      }
-
-      MultiStatisticsData data = new MultiStatisticsData("Quota Attainment", groups);
-
-      return (new RenderWidgetOutput(data));
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmQuotaAttainmentWidgetProducer.java

@@ -1,68 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** MetaData producer for the CRM Quota Attainment widget.
- **
- ** Provides a MULTI_STATISTICS view showing quota attainment percentage for
- ** the current user and current period.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.widgets;
-
-
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducer;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
-
-
-/*******************************************************************************
- ** Produces metadata for the crm_quota_attainment widget (MULTI_STATISTICS type).
- *******************************************************************************/
-public class CrmQuotaAttainmentWidgetProducer extends MetaDataProducer<QWidgetMetaData>
-{
-   public static final String NAME = "crm_quota_attainment";
-
-
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Override
-   public QWidgetMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QWidgetMetaData()
-         .withName(NAME)
-         .withType(WidgetType.MULTI_STATISTICS.getType())
-         .withGridColumns(6)
-         .withIsCard(true)
-         .withLabel("Quota Attainment")
-         .withTooltip("Your quota attainment for the current period")
-         .withShowReloadButton(true)
-         .withIcon("topRightInsideCard", new QIcon("flag"))
-         .withCodeReference(new QCodeReference(CrmQuotaAttainmentRenderer.class));
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmRecentDealsRenderer.java

@@ -1,131 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** Renderer for the Recent Deals widget.
- **
- ** Queries deals with actualCloseDate set, ordered by actualCloseDate DESC,
- ** limited to 20 rows. Returns table with name, amount, owner, closedDate,
- ** winLossReason.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.widgets;
-
-
-import java.math.BigDecimal;
-import java.time.LocalDate;
-import java.util.ArrayList;
-import java.util.List;
-import java.util.Map;
-import com.kingsrook.qbits.crm.deals.model.Deal;
-import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
-import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
-import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
-import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.TableData;
-import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.utils.collections.MapBuilder;
-
-
-/*******************************************************************************
- ** Renders recent deals as a table widget.
- *******************************************************************************/
-public class CrmRecentDealsRenderer extends AbstractWidgetRenderer
-{
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Override
-   public RenderWidgetOutput render(RenderWidgetInput input) throws QException
-   {
-      ///////////////////////////////////////////////////////////
-      // query recent deals with actualCloseDate set           //
-      ///////////////////////////////////////////////////////////
-      QueryOutput queryOutput = new QueryAction().execute(
-         new QueryInput(Deal.TABLE_NAME)
-            .withFilter(new QQueryFilter()
-               .withCriteria(new QFilterCriteria("actualCloseDate", QCriteriaOperator.IS_NOT_BLANK))
-               .withOrderBy(new QFilterOrderBy("actualCloseDate", false))
-               .withLimit(20)));
-
-      ///////////////////////////
-      // build table columns   //
-      ///////////////////////////
-      List<TableData.Column> columns = List.of(
-         new TableData.Column("html", "Name", "name", "2fr", null),
-         new TableData.Column("html", "Amount", "amount", "1fr", "right"),
-         new TableData.Column("html", "Owner", "ownerUserId", "1fr", null),
-         new TableData.Column("html", "Closed Date", "closedDate", "1fr", "right"),
-         new TableData.Column("html", "Win/Loss Reason", "winLossReason", "1fr", null)
-      );
-
-      //////////////////////////
-      // build table rows     //
-      //////////////////////////
-      List<Map<String, Object>> rows = new ArrayList<>();
-      for(QRecord record : queryOutput.getRecords())
-      {
-         String name = record.getValueString("name");
-         if(name == null)
-         {
-            name = "";
-         }
-
-         BigDecimal amount = record.getValueBigDecimal("amount");
-         String amountDisplay = amount != null ? "$" + amount.toPlainString() : "-";
-
-         String ownerUserId = record.getValueString("ownerUserId");
-         if(ownerUserId == null)
-         {
-            ownerUserId = "-";
-         }
-
-         LocalDate closedDate = record.getValueLocalDate("actualCloseDate");
-         String closedDateDisplay = closedDate != null ? closedDate.toString() : "-";
-
-         Integer winLossReasonId = record.getValueInteger("winLossReasonId");
-         String winLossDisplay = winLossReasonId != null ? String.valueOf(winLossReasonId) : "-";
-
-         rows.add(MapBuilder.of(
-            "name", name,
-            "amount", amountDisplay,
-            "ownerUserId", ownerUserId,
-            "closedDate", closedDateDisplay,
-            "winLossReason", winLossDisplay
-         ));
-      }
-
-      TableData tableData = new TableData(null, columns, rows)
-         .withRowsPerPage(20)
-         .withHidePaginationDropdown(true);
-
-      return (new RenderWidgetOutput(tableData));
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmRecentDealsWidgetProducer.java

@@ -1,66 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** MetaData producer for the Recent Deals widget.
- **
- ** Shows the most recently closed deals with key details.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.widgets;
-
-
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducer;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
-
-
-/*******************************************************************************
- ** Produces metadata for the crm_recent_deals widget (TABLE type).
- *******************************************************************************/
-public class CrmRecentDealsWidgetProducer extends MetaDataProducer<QWidgetMetaData>
-{
-   public static final String NAME = "crm_recent_deals";
-
-
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Override
-   public QWidgetMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QWidgetMetaData()
-         .withName(NAME)
-         .withType(WidgetType.TABLE.getType())
-         .withGridColumns(6)
-         .withIsCard(true)
-         .withLabel("Recent Deals")
-         .withTooltip("Most recently closed deals")
-         .withShowReloadButton(true)
-         .withIcon("topRightInsideCard", new com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon("receipt_long"))
-         .withCodeReference(new QCodeReference(CrmRecentDealsRenderer.class));
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmRottingDealsRenderer.java

@@ -1,186 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** Renderer for the CRM Rotting Deals widget.
- **
- ** Queries deals and their pipeline stages, finds deals where the number of
- ** days since stageEnteredDate exceeds the stage's rotDays threshold.
- ** Returns a table with columns: dealName, stageName, daysInStage, rotDays,
- ** ownerUserId, amount.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.widgets;
-
-
-import java.math.BigDecimal;
-import java.time.Instant;
-import java.time.temporal.ChronoUnit;
-import java.util.ArrayList;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
-import com.kingsrook.qbits.crm.deals.model.Deal;
-import com.kingsrook.qbits.crm.deals.model.PipelineStage;
-import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
-import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
-import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
-import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.TableData;
-import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.utils.collections.MapBuilder;
-
-
-/*******************************************************************************
- ** Renders the rotting deals table showing deals past their stage rot threshold.
- *******************************************************************************/
-public class CrmRottingDealsRenderer extends AbstractWidgetRenderer
-{
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Override
-   public RenderWidgetOutput render(RenderWidgetInput input) throws QException
-   {
-      Instant now = Instant.now();
-
-      ///////////////////////////////////////////////////////////////
-      // query all stages that have a rotDays value set            //
-      ///////////////////////////////////////////////////////////////
-      QueryOutput stageOutput = new QueryAction().execute(
-         new QueryInput(PipelineStage.TABLE_NAME)
-            .withFilter(new QQueryFilter()
-               .withCriteria(new QFilterCriteria("rotDays", QCriteriaOperator.IS_NOT_BLANK))));
-
-      ///////////////////////////////////////////////////////////////
-      // build a map of stageId -> PipelineStage for lookup        //
-      ///////////////////////////////////////////////////////////////
-      Map<Integer, PipelineStage> stageMap = new HashMap<>();
-      for(QRecord stageRecord : stageOutput.getRecords())
-      {
-         PipelineStage stage = new PipelineStage(stageRecord);
-         stageMap.put(stage.getId(), stage);
-      }
-
-      if(stageMap.isEmpty())
-      {
-         //////////////////////////////////////////////////////////
-         // no stages with rot days -- return empty table        //
-         //////////////////////////////////////////////////////////
-         return (buildEmptyOutput());
-      }
-
-      ///////////////////////////////////////////////////////////////
-      // query deals in those stages                               //
-      ///////////////////////////////////////////////////////////////
-      QueryOutput dealOutput = new QueryAction().execute(
-         new QueryInput(Deal.TABLE_NAME)
-            .withFilter(new QQueryFilter()
-               .withCriteria(new QFilterCriteria("pipelineStageId", QCriteriaOperator.IN, new ArrayList<>(stageMap.keySet())))));
-
-      ///////////////////////////
-      // build table columns   //
-      ///////////////////////////
-      List<TableData.Column> columns = List.of(
-         new TableData.Column("html", "Deal", "dealName", "2fr", null),
-         new TableData.Column("html", "Stage", "stageName", "1fr", null),
-         new TableData.Column("html", "Days in Stage", "daysInStage", "1fr", "right"),
-         new TableData.Column("html", "Rot Days", "rotDays", "1fr", "right"),
-         new TableData.Column("html", "Owner", "ownerUserId", "1fr", null),
-         new TableData.Column("html", "Amount", "amount", "1fr", "right")
-      );
-
-      //////////////////////////////////////////////////////////
-      // filter to only deals past the rot threshold          //
-      //////////////////////////////////////////////////////////
-      List<Map<String, Object>> rows = new ArrayList<>();
-      for(QRecord dealRecord : dealOutput.getRecords())
-      {
-         Integer stageId = dealRecord.getValueInteger("pipelineStageId");
-         PipelineStage stage = stageMap.get(stageId);
-
-         if(stage == null || stage.getRotDays() == null)
-         {
-            continue;
-         }
-
-         Instant stageEnteredDate = dealRecord.getValueInstant("stageEnteredDate");
-         if(stageEnteredDate == null)
-         {
-            continue;
-         }
-
-         long daysInStage = ChronoUnit.DAYS.between(stageEnteredDate, now);
-         if(daysInStage > stage.getRotDays())
-         {
-            String dealName    = dealRecord.getValueString("name");
-            String ownerUserId = dealRecord.getValueString("ownerUserId");
-            BigDecimal amount  = dealRecord.getValueBigDecimal("amount");
-
-            rows.add(MapBuilder.of(
-               "dealName", dealName != null ? dealName : "",
-               "stageName", stage.getName(),
-               "daysInStage", (int) daysInStage,
-               "rotDays", stage.getRotDays(),
-               "ownerUserId", ownerUserId != null ? ownerUserId : "",
-               "amount", amount != null ? amount.toPlainString() : "0"
-            ));
-         }
-      }
-
-      TableData tableData = new TableData(null, columns, rows)
-         .withRowsPerPage(50)
-         .withHidePaginationDropdown(true);
-
-      return (new RenderWidgetOutput(tableData));
-   }
-
-
-
-   /***************************************************************************
-    ** Build an empty table output when there are no rotting deals.
-    ***************************************************************************/
-   private RenderWidgetOutput buildEmptyOutput()
-   {
-      List<TableData.Column> columns = List.of(
-         new TableData.Column("html", "Deal", "dealName", "2fr", null),
-         new TableData.Column("html", "Stage", "stageName", "1fr", null),
-         new TableData.Column("html", "Days in Stage", "daysInStage", "1fr", "right"),
-         new TableData.Column("html", "Rot Days", "rotDays", "1fr", "right"),
-         new TableData.Column("html", "Owner", "ownerUserId", "1fr", null),
-         new TableData.Column("html", "Amount", "amount", "1fr", "right")
-      );
-
-      TableData tableData = new TableData(null, columns, new ArrayList<>())
-         .withRowsPerPage(50)
-         .withHidePaginationDropdown(true);
-
-      return (new RenderWidgetOutput(tableData));
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmRottingDealsWidgetProducer.java

@@ -1,67 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** MetaData producer for the CRM Rotting Deals widget.
- **
- ** Provides a TABLE view showing deals that have exceeded their current
- ** stage's rotDays threshold, indicating they need attention.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.widgets;
-
-
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducer;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
-
-
-/*******************************************************************************
- ** Produces metadata for the crm_rotting_deals widget (TABLE type).
- *******************************************************************************/
-public class CrmRottingDealsWidgetProducer extends MetaDataProducer<QWidgetMetaData>
-{
-   public static final String NAME = "crm_rotting_deals";
-
-
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Override
-   public QWidgetMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QWidgetMetaData()
-         .withName(NAME)
-         .withType(WidgetType.TABLE.getType())
-         .withGridColumns(6)
-         .withIsCard(true)
-         .withLabel("Rotting Deals")
-         .withTooltip("Deals that have exceeded their stage's rot days threshold")
-         .withShowReloadButton(true)
-         .withIcon("topRightInsideCard", new com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon("warning"))
-         .withCodeReference(new QCodeReference(CrmRottingDealsRenderer.class));
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmSalesCycleLengthRenderer.java

@@ -1,147 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** Renderer for the Sales Cycle Length widget.
- **
- ** Queries closedWon deals and computes the average number of days from
- ** createDate to actualCloseDate.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.widgets;
-
-
-import java.math.BigDecimal;
-import java.math.RoundingMode;
-import java.time.Instant;
-import java.time.LocalDate;
-import java.time.ZoneId;
-import java.time.temporal.ChronoUnit;
-import java.util.ArrayList;
-import java.util.HashSet;
-import java.util.List;
-import java.util.Map;
-import java.util.Set;
-import com.kingsrook.qbits.crm.deals.model.Deal;
-import com.kingsrook.qbits.crm.deals.model.PipelineStage;
-import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
-import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
-import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
-import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.TableData;
-import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.utils.collections.MapBuilder;
-
-
-/*******************************************************************************
- ** Renders sales cycle length as a single-stat table widget.
- *******************************************************************************/
-public class CrmSalesCycleLengthRenderer extends AbstractWidgetRenderer
-{
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Override
-   public RenderWidgetOutput render(RenderWidgetInput input) throws QException
-   {
-      ///////////////////////////////////////////////////////////
-      // find closedWon stage IDs                              //
-      ///////////////////////////////////////////////////////////
-      Set<Integer> closedWonStageIds = new HashSet<>();
-      QueryOutput stageOutput = new QueryAction().execute(
-         new QueryInput(PipelineStage.TABLE_NAME)
-            .withFilter(new QQueryFilter()
-               .withCriteria(new QFilterCriteria("isClosedWon", QCriteriaOperator.EQUALS, true))));
-
-      for(QRecord record : stageOutput.getRecords())
-      {
-         closedWonStageIds.add(record.getValueInteger("id"));
-      }
-
-      ///////////////////////////////////////////////////////////
-      // query deals with actualCloseDate set                  //
-      ///////////////////////////////////////////////////////////
-      QueryOutput dealOutput = new QueryAction().execute(
-         new QueryInput(Deal.TABLE_NAME)
-            .withFilter(new QQueryFilter()
-               .withCriteria(new QFilterCriteria("actualCloseDate", QCriteriaOperator.IS_NOT_BLANK))));
-
-      ///////////////////////////////////////////////////////////
-      // compute average days for closedWon deals              //
-      ///////////////////////////////////////////////////////////
-      long totalDays = 0;
-      int count = 0;
-
-      for(QRecord record : dealOutput.getRecords())
-      {
-         Integer stageId = record.getValueInteger("pipelineStageId");
-         if(closedWonStageIds.contains(stageId))
-         {
-            Instant   createDate     = record.getValueInstant("createDate");
-            LocalDate actualCloseDate = record.getValueLocalDate("actualCloseDate");
-
-            if(createDate != null && actualCloseDate != null)
-            {
-               LocalDate createLocalDate = createDate.atZone(ZoneId.systemDefault()).toLocalDate();
-               long days = ChronoUnit.DAYS.between(createLocalDate, actualCloseDate);
-               totalDays += days;
-               count++;
-            }
-         }
-      }
-
-      BigDecimal avgCycleLength = count > 0
-         ? BigDecimal.valueOf(totalDays).divide(BigDecimal.valueOf(count), 1, RoundingMode.HALF_UP)
-         : BigDecimal.ZERO;
-
-      ///////////////////////////
-      // build table output    //
-      ///////////////////////////
-      List<TableData.Column> columns = List.of(
-         new TableData.Column("html", "Metric", "metric", "1fr", null),
-         new TableData.Column("html", "Value", "value", "1fr", "right")
-      );
-
-      List<Map<String, Object>> rows = new ArrayList<>();
-      rows.add(MapBuilder.of(
-         "metric", "Avg Sales Cycle (days)",
-         "value", avgCycleLength.toPlainString()
-      ));
-      rows.add(MapBuilder.of(
-         "metric", "Closed Won Deals",
-         "value", String.valueOf(count)
-      ));
-
-      TableData tableData = new TableData(null, columns, rows)
-         .withRowsPerPage(10)
-         .withHidePaginationDropdown(true);
-
-      return (new RenderWidgetOutput(tableData));
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmSalesCycleLengthWidgetProducer.java

@@ -1,66 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** MetaData producer for the Sales Cycle Length widget.
- **
- ** Shows the average number of days from deal creation to close for won deals.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.widgets;
-
-
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducer;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
-
-
-/*******************************************************************************
- ** Produces metadata for the crm_sales_cycle_length widget (TABLE type).
- *******************************************************************************/
-public class CrmSalesCycleLengthWidgetProducer extends MetaDataProducer<QWidgetMetaData>
-{
-   public static final String NAME = "crm_sales_cycle_length";
-
-
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Override
-   public QWidgetMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QWidgetMetaData()
-         .withName(NAME)
-         .withType(WidgetType.TABLE.getType())
-         .withGridColumns(3)
-         .withIsCard(true)
-         .withLabel("Sales Cycle Length")
-         .withTooltip("Average days from creation to close for won deals")
-         .withShowReloadButton(true)
-         .withIcon("topRightInsideCard", new com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon("schedule"))
-         .withCodeReference(new QCodeReference(CrmSalesCycleLengthRenderer.class));
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmSalesLeaderboardRenderer.java

@@ -1,137 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** Renderer for the CRM Sales Leaderboard widget.
- **
- ** Queries deals in closed-won stages, groups by ownerUserId, sums
- ** amountInBaseCurrency, and returns a ranked table.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.widgets;
-
-
-import java.math.BigDecimal;
-import java.util.ArrayList;
-import java.util.Comparator;
-import java.util.LinkedHashMap;
-import java.util.List;
-import java.util.Map;
-import com.kingsrook.qbits.crm.deals.model.Deal;
-import com.kingsrook.qbits.crm.deals.model.PipelineStage;
-import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
-import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
-import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
-import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.TableData;
-import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.utils.collections.MapBuilder;
-
-
-/*******************************************************************************
- ** Renders the sales leaderboard as a table widget, ranking reps by closed-won
- ** deal revenue.
- *******************************************************************************/
-public class CrmSalesLeaderboardRenderer extends AbstractWidgetRenderer
-{
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Override
-   public RenderWidgetOutput render(RenderWidgetInput input) throws QException
-   {
-      ///////////////////////////////////////////////////////////
-      // find closed-won stage ids                             //
-      ///////////////////////////////////////////////////////////
-      QueryOutput stageQuery = new QueryAction().execute(
-         new QueryInput(PipelineStage.TABLE_NAME)
-            .withFilter(new QQueryFilter()
-               .withCriteria(new QFilterCriteria("isClosedWon", QCriteriaOperator.EQUALS, true))));
-
-      List<Integer> closedWonStageIds = stageQuery.getRecords().stream()
-         .map(r -> r.getValueInteger("id"))
-         .toList();
-
-      ///////////////////////////////////////////////////////////
-      // query deals in closed-won stages                      //
-      ///////////////////////////////////////////////////////////
-      Map<String, BigDecimal> revenueByOwner = new LinkedHashMap<>();
-
-      if(!closedWonStageIds.isEmpty())
-      {
-         QueryOutput dealQuery = new QueryAction().execute(
-            new QueryInput(Deal.TABLE_NAME)
-               .withFilter(new QQueryFilter()
-                  .withCriteria(new QFilterCriteria("pipelineStageId", QCriteriaOperator.IN, closedWonStageIds))));
-
-         for(QRecord dealRecord : dealQuery.getRecords())
-         {
-            String ownerUserId = dealRecord.getValueString("ownerUserId");
-            BigDecimal amount = dealRecord.getValueBigDecimal("amountInBaseCurrency");
-            if(ownerUserId != null && amount != null)
-            {
-               revenueByOwner.merge(ownerUserId, amount, BigDecimal::add);
-            }
-         }
-      }
-
-      ///////////////////////////
-      // build table columns   //
-      ///////////////////////////
-      List<TableData.Column> columns = List.of(
-         new TableData.Column("html", "Rank", "rank", "0.5fr", null),
-         new TableData.Column("html", "Owner", "owner", "2fr", null),
-         new TableData.Column("html", "Revenue", "revenue", "1fr", "right")
-      );
-
-      //////////////////////////
-      // build ranked rows    //
-      //////////////////////////
-      List<Map.Entry<String, BigDecimal>> sorted = new ArrayList<>(revenueByOwner.entrySet());
-      sorted.sort(Comparator.comparing(Map.Entry<String, BigDecimal>::getValue).reversed());
-
-      List<Map<String, Object>> rows = new ArrayList<>();
-      int rank = 1;
-      for(Map.Entry<String, BigDecimal> entry : sorted)
-      {
-         rows.add(MapBuilder.of(
-            "rank", String.valueOf(rank),
-            "owner", entry.getKey(),
-            "revenue", entry.getValue().toPlainString()
-         ));
-         rank++;
-      }
-
-      TableData tableData = new TableData(null, columns, rows)
-         .withRowsPerPage(25)
-         .withHidePaginationDropdown(true);
-
-      return (new RenderWidgetOutput(tableData));
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmSalesLeaderboardWidgetProducer.java

@@ -1,68 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** MetaData producer for the CRM Sales Leaderboard widget.
- **
- ** Provides a TABLE view showing deals grouped by owner with total revenue
- ** from closed-won deals, ranked descending.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.widgets;
-
-
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducer;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
-
-
-/*******************************************************************************
- ** Produces metadata for the crm_sales_leaderboard widget (TABLE type).
- *******************************************************************************/
-public class CrmSalesLeaderboardWidgetProducer extends MetaDataProducer<QWidgetMetaData>
-{
-   public static final String NAME = "crm_sales_leaderboard";
-
-
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Override
-   public QWidgetMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QWidgetMetaData()
-         .withName(NAME)
-         .withType(WidgetType.TABLE.getType())
-         .withGridColumns(6)
-         .withIsCard(true)
-         .withLabel("Sales Leaderboard")
-         .withTooltip("Top performers by closed-won deal revenue")
-         .withShowReloadButton(true)
-         .withIcon("topRightInsideCard", new QIcon("leaderboard"))
-         .withCodeReference(new QCodeReference(CrmSalesLeaderboardRenderer.class));
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmStageDurationRenderer.java

@@ -1,145 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** Renderer for the Stage Duration widget.
- **
- ** Queries DealStageHistory where durationInFromStageDays is not null,
- ** computes average and median per fromStageId.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.widgets;
-
-
-import java.math.BigDecimal;
-import java.math.RoundingMode;
-import java.util.ArrayList;
-import java.util.Collections;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
-import com.kingsrook.qbits.crm.deals.model.DealStageHistory;
-import com.kingsrook.qbits.crm.deals.model.PipelineStage;
-import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
-import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
-import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
-import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.TableData;
-import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.utils.collections.MapBuilder;
-
-
-/*******************************************************************************
- ** Renders stage duration analytics as a table widget.
- *******************************************************************************/
-public class CrmStageDurationRenderer extends AbstractWidgetRenderer
-{
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Override
-   public RenderWidgetOutput render(RenderWidgetInput input) throws QException
-   {
-      ///////////////////////////////////////////////////////////
-      // query history with durationInFromStageDays populated  //
-      ///////////////////////////////////////////////////////////
-      QueryOutput historyOutput = new QueryAction().execute(
-         new QueryInput(DealStageHistory.TABLE_NAME)
-            .withFilter(new QQueryFilter()
-               .withCriteria(new QFilterCriteria("durationInFromStageDays", QCriteriaOperator.IS_NOT_BLANK))));
-
-      ///////////////////////////////////////////////////////////
-      // group durations by fromStageId                        //
-      ///////////////////////////////////////////////////////////
-      Map<Integer, List<Integer>> durationsByStage = new HashMap<>();
-      for(QRecord record : historyOutput.getRecords())
-      {
-         Integer fromStageId = record.getValueInteger("fromStageId");
-         Integer duration    = record.getValueInteger("durationInFromStageDays");
-         if(fromStageId != null && duration != null)
-         {
-            durationsByStage.computeIfAbsent(fromStageId, k -> new ArrayList<>()).add(duration);
-         }
-      }
-
-      ///////////////////////////////////////////////////////////
-      // build stage name lookup                               //
-      ///////////////////////////////////////////////////////////
-      QueryOutput stageOutput = new QueryAction().execute(
-         new QueryInput(PipelineStage.TABLE_NAME));
-
-      Map<Integer, String> stageNames = new HashMap<>();
-      for(QRecord record : stageOutput.getRecords())
-      {
-         stageNames.put(record.getValueInteger("id"), record.getValueString("name"));
-      }
-
-      ///////////////////////////
-      // build table columns   //
-      ///////////////////////////
-      List<TableData.Column> columns = List.of(
-         new TableData.Column("html", "Stage", "stageName", "2fr", null),
-         new TableData.Column("html", "Avg Days", "avgDays", "1fr", "right"),
-         new TableData.Column("html", "Median Days", "medianDays", "1fr", "right")
-      );
-
-      //////////////////////////
-      // build table rows     //
-      //////////////////////////
-      List<Map<String, Object>> rows = new ArrayList<>();
-      for(Map.Entry<Integer, List<Integer>> entry : durationsByStage.entrySet())
-      {
-         Integer       stageId   = entry.getKey();
-         List<Integer> durations = entry.getValue();
-
-         BigDecimal avg = BigDecimal.valueOf(durations.stream().mapToInt(Integer::intValue).average().orElse(0))
-            .setScale(1, RoundingMode.HALF_UP);
-
-         Collections.sort(durations);
-         int medianIndex = durations.size() / 2;
-         int median = durations.size() % 2 == 0
-            ? (durations.get(medianIndex - 1) + durations.get(medianIndex)) / 2
-            : durations.get(medianIndex);
-
-         String stageName = stageNames.getOrDefault(stageId, "Stage " + stageId);
-
-         rows.add(MapBuilder.of(
-            "stageName", stageName,
-            "avgDays", avg,
-            "medianDays", median
-         ));
-      }
-
-      TableData tableData = new TableData(null, columns, rows)
-         .withRowsPerPage(25)
-         .withHidePaginationDropdown(true);
-
-      return (new RenderWidgetOutput(tableData));
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmStageDurationWidgetProducer.java

@@ -1,66 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** MetaData producer for the Stage Duration widget.
- **
- ** Shows average and median days spent in each pipeline stage.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.widgets;
-
-
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducer;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
-
-
-/*******************************************************************************
- ** Produces metadata for the crm_stage_duration widget (TABLE type).
- *******************************************************************************/
-public class CrmStageDurationWidgetProducer extends MetaDataProducer<QWidgetMetaData>
-{
-   public static final String NAME = "crm_stage_duration";
-
-
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Override
-   public QWidgetMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QWidgetMetaData()
-         .withName(NAME)
-         .withType(WidgetType.TABLE.getType())
-         .withGridColumns(6)
-         .withIsCard(true)
-         .withLabel("Stage Duration")
-         .withTooltip("Average and median days per pipeline stage")
-         .withShowReloadButton(true)
-         .withIcon("topRightInsideCard", new com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon("timer"))
-         .withCodeReference(new QCodeReference(CrmStageDurationRenderer.class));
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmWinRateRenderer.java

@@ -1,170 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** Renderer for the Win Rate widget.
- **
- ** Queries DealStageHistory for transitions to closedWon and closedLost stages,
- ** groups by month, and computes win rate percentage.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.widgets;
-
-
-import java.math.BigDecimal;
-import java.math.RoundingMode;
-import java.time.Instant;
-import java.time.YearMonth;
-import java.time.ZoneId;
-import java.util.ArrayList;
-import java.util.HashSet;
-import java.util.List;
-import java.util.Map;
-import java.util.Set;
-import java.util.TreeMap;
-import com.kingsrook.qbits.crm.deals.model.DealStageHistory;
-import com.kingsrook.qbits.crm.deals.model.PipelineStage;
-import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
-import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
-import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
-import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.TableData;
-import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import com.kingsrook.qqq.backend.core.utils.collections.MapBuilder;
-
-
-/*******************************************************************************
- ** Renders win rate analytics as a table widget.
- *******************************************************************************/
-public class CrmWinRateRenderer extends AbstractWidgetRenderer
-{
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Override
-   public RenderWidgetOutput render(RenderWidgetInput input) throws QException
-   {
-      ///////////////////////////////////////////////////////////
-      // find closedWon and closedLost stage IDs               //
-      ///////////////////////////////////////////////////////////
-      Set<Integer> closedWonStageIds  = new HashSet<>();
-      Set<Integer> closedLostStageIds = new HashSet<>();
-
-      QueryOutput stageOutput = new QueryAction().execute(
-         new QueryInput(PipelineStage.TABLE_NAME));
-
-      for(QRecord record : stageOutput.getRecords())
-      {
-         if(Boolean.TRUE.equals(record.getValueBoolean("isClosedWon")))
-         {
-            closedWonStageIds.add(record.getValueInteger("id"));
-         }
-         if(Boolean.TRUE.equals(record.getValueBoolean("isClosedLost")))
-         {
-            closedLostStageIds.add(record.getValueInteger("id"));
-         }
-      }
-
-      ///////////////////////////////////////////////////////////
-      // query all deal stage history                          //
-      ///////////////////////////////////////////////////////////
-      QueryOutput historyOutput = new QueryAction().execute(
-         new QueryInput(DealStageHistory.TABLE_NAME));
-
-      ///////////////////////////////////////////////////////////
-      // group by month                                        //
-      ///////////////////////////////////////////////////////////
-      Map<YearMonth, int[]> monthData = new TreeMap<>(); // [won, lost]
-
-      for(QRecord record : historyOutput.getRecords())
-      {
-         Integer toStageId      = record.getValueInteger("toStageId");
-         Instant transitionDate = record.getValueInstant("transitionDate");
-
-         if(toStageId == null || transitionDate == null)
-         {
-            continue;
-         }
-
-         boolean isWon  = closedWonStageIds.contains(toStageId);
-         boolean isLost = closedLostStageIds.contains(toStageId);
-
-         if(!isWon && !isLost)
-         {
-            continue;
-         }
-
-         YearMonth month = YearMonth.from(transitionDate.atZone(ZoneId.systemDefault()));
-         int[] counts = monthData.computeIfAbsent(month, k -> new int[2]);
-         if(isWon)
-         {
-            counts[0]++;
-         }
-         if(isLost)
-         {
-            counts[1]++;
-         }
-      }
-
-      ///////////////////////////
-      // build table columns   //
-      ///////////////////////////
-      List<TableData.Column> columns = List.of(
-         new TableData.Column("html", "Month", "month", "1fr", null),
-         new TableData.Column("html", "Won", "won", "1fr", "right"),
-         new TableData.Column("html", "Lost", "lost", "1fr", "right"),
-         new TableData.Column("html", "Win Rate %", "winRate", "1fr", "right")
-      );
-
-      //////////////////////////
-      // build table rows     //
-      //////////////////////////
-      List<Map<String, Object>> rows = new ArrayList<>();
-      for(Map.Entry<YearMonth, int[]> entry : monthData.entrySet())
-      {
-         int won   = entry.getValue()[0];
-         int lost  = entry.getValue()[1];
-         int total = won + lost;
-
-         BigDecimal winRate = total > 0
-            ? BigDecimal.valueOf(won).multiply(BigDecimal.valueOf(100)).divide(BigDecimal.valueOf(total), 1, RoundingMode.HALF_UP)
-            : BigDecimal.ZERO;
-
-         rows.add(MapBuilder.of(
-            "month", entry.getKey().toString(),
-            "won", won,
-            "lost", lost,
-            "winRate", winRate + "%"
-         ));
-      }
-
-      TableData tableData = new TableData(null, columns, rows)
-         .withRowsPerPage(25)
-         .withHidePaginationDropdown(true);
-
-      return (new RenderWidgetOutput(tableData));
-   }
-
-}

src/main/java/com/kingsrook/qbits/crm/widgets/CrmWinRateWidgetProducer.java

@@ -1,66 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** MetaData producer for the Win Rate widget.
- **
- ** Shows won/lost counts and win rate percentage grouped by month.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.widgets;
-
-
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
-import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducer;
-import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
-
-
-/*******************************************************************************
- ** Produces metadata for the crm_win_rate widget (TABLE type).
- *******************************************************************************/
-public class CrmWinRateWidgetProducer extends MetaDataProducer<QWidgetMetaData>
-{
-   public static final String NAME = "crm_win_rate";
-
-
-
-   /*******************************************************************************
-    **
-    *******************************************************************************/
-   @Override
-   public QWidgetMetaData produce(QInstance qInstance) throws QException
-   {
-      return new QWidgetMetaData()
-         .withName(NAME)
-         .withType(WidgetType.TABLE.getType())
-         .withGridColumns(6)
-         .withIsCard(true)
-         .withLabel("Win Rate")
-         .withTooltip("Monthly win/loss counts and win rate")
-         .withShowReloadButton(true)
-         .withIcon("topRightInsideCard", new com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon("analytics"))
-         .withCodeReference(new QCodeReference(CrmWinRateRenderer.class));
-   }
-
-}

src/main/resources/db/liquibase/qbit-crm/crm.changelog.sql

@@ -1,7 +1,7 @@
 --liquibase formatted sql
 -- Copyright 2026 Kingsrook, LLC. Licensed under the Apache License, Version 2.0.
 -- https://www.apache.org/licenses/LICENSE-2.0
--- QBit CRM 0.2.0 P1 PostgreSQL reference schema (33 tables).
+-- QBit CRM 0.2.0 P1 PostgreSQL reference schema (34 tables).
 -- QQQ table names equal SQL names; Java field names map to snake_case columns.
 -- Unique constraints match QQQ metadata. Relationship columns are indexed;
 -- foreign-key constraints are deliberately not imposed (the QQQ/host layer owns
@@ -199,7 +199,7 @@ CREATE TABLE IF NOT EXISTS crm_contact (
    company_id INTEGER,
    lifecycle_stage_id INTEGER,
    lead_source_id INTEGER,
-   owner_user_id TEXT NOT NULL,
+   owner_user_id TEXT,
    address_line1 VARCHAR(250),
    address_line2 VARCHAR(250),
    city VARCHAR(100),
@@ -264,7 +264,8 @@ CREATE TABLE IF NOT EXISTS crm_deal (
    probability_override_pct INTEGER,
    weighted_amount DECIMAL(20,6),
    company_id INTEGER,
-   owner_user_id TEXT NOT NULL,
+   primary_contact_id INTEGER,
+   owner_user_id TEXT,
    lead_source_id INTEGER,
    expected_close_date DATE,
    actual_close_date DATE,
@@ -279,6 +280,7 @@ CREATE TABLE IF NOT EXISTS crm_deal (
 
 CREATE INDEX IF NOT EXISTS ix_crm_deal_pipeline_id ON crm_deal (pipeline_id);
 CREATE INDEX IF NOT EXISTS ix_crm_deal_pipeline_stage_id ON crm_deal (pipeline_stage_id);
+CREATE INDEX IF NOT EXISTS ix_crm_deal_primary_contact_id ON crm_deal (primary_contact_id);
 CREATE INDEX IF NOT EXISTS ix_crm_deal_company_id ON crm_deal (company_id);
 CREATE INDEX IF NOT EXISTS ix_crm_deal_owner_user_id ON crm_deal (owner_user_id);
 CREATE INDEX IF NOT EXISTS ix_crm_deal_lead_source_id ON crm_deal (lead_source_id);
@@ -381,8 +383,13 @@ CREATE INDEX IF NOT EXISTS ix_crm_email_list_member_form_submission_id ON crm_em
 --changeset qbit-crm:020-crm_email_message
 CREATE TABLE IF NOT EXISTS crm_email_message (
    id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
+   idempotency_key VARCHAR(250) UNIQUE,
+   email_message_id VARCHAR(100),
+   delivered_date TIMESTAMP WITHOUT TIME ZONE,
+   bounced_date TIMESTAMP WITHOUT TIME ZONE,
    activity_id INTEGER,
-   from_address VARCHAR(250) NOT NULL,
+   contact_id INTEGER,
+   from_address VARCHAR(250),
    from_name VARCHAR(100),
    to_addresses TEXT NOT NULL,
    cc_addresses TEXT,
@@ -408,6 +415,8 @@ CREATE TABLE IF NOT EXISTS crm_email_message (
    modify_date TIMESTAMP WITHOUT TIME ZONE
 );
 
+CREATE INDEX IF NOT EXISTS ix_crm_email_message_email_message_id ON crm_email_message (email_message_id);
+CREATE INDEX IF NOT EXISTS ix_crm_email_message_contact_id ON crm_email_message (contact_id);
 CREATE INDEX IF NOT EXISTS ix_crm_email_message_activity_id ON crm_email_message (activity_id);
 CREATE INDEX IF NOT EXISTS ix_crm_email_message_thread_id ON crm_email_message (thread_id);
 CREATE INDEX IF NOT EXISTS ix_crm_email_message_email_template_id ON crm_email_message (email_template_id);
@@ -670,3 +679,11 @@ CREATE TABLE IF NOT EXISTS crm_win_loss_reason (
 );
 
 CREATE INDEX IF NOT EXISTS ix_crm_consent_record_evidence_form_submission_id ON crm_consent_record (evidence_form_submission_id);
+
+--changeset qbit-crm:crm_job_state
+CREATE TABLE IF NOT EXISTS crm_job_state (
+   id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
+   name VARCHAR(250) NOT NULL UNIQUE,
+   last_success TIMESTAMP WITHOUT TIME ZONE,
+   last_full_pass TIMESTAMP WITHOUT TIME ZONE
+);

src/test/java/com/kingsrook/qbits/crm/BaseTest.java

@@ -124,4 +124,37 @@ public class BaseTest
          CrmQBitProducer.GROUP_ID + ":" + CrmQBitProducer.ARTIFACT_ID);
    }
 
+   /*******************************************************************************
+    ** Existing operation regressions run helpers directly. CrmActionsTest covers ETL.
+    *******************************************************************************/
+   protected com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput runBusinessStep(
+      com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput input) throws QException
+   {
+      com.kingsrook.qqq.backend.core.actions.processes.BackendStep step;
+      step = switch(input.getProcessName())
+      {
+         case "crm_auto_assign" -> new com.kingsrook.qbits.crm.scoring.processes.AutoAssignProcess();
+         case "crm_enroll_in_sequence" -> new com.kingsrook.qbits.crm.email.processes.EnrollInSequenceProcess();
+         case "crm_unenroll_from_sequence" -> new com.kingsrook.qbits.crm.email.processes.UnenrollFromSequenceProcess();
+         case "crm_clone_deal" -> new com.kingsrook.qbits.crm.deals.processes.CloneDealProcess();
+         case "crm_convert_lead" -> new com.kingsrook.qbits.crm.deals.processes.ConvertLeadProcess();
+         case "crm_manage_contact_on_deal" -> new com.kingsrook.qbits.crm.deals.processes.ManageContactOnDealProcess();
+         case "crm_move_deal_stage" -> new com.kingsrook.qbits.crm.deals.processes.MoveDealStageProcess();
+         case "crm_reopen_deal" -> new com.kingsrook.qbits.crm.deals.processes.ReopenDealProcess();
+         case "crm_transfer_ownership" -> new com.kingsrook.qbits.crm.deals.processes.TransferOwnershipProcess();
+         case "crm_convert_form_to_contact" -> new com.kingsrook.qbits.crm.core.processes.ConvertFormToContactProcess();
+         case "crm_detect_duplicates" -> new com.kingsrook.qbits.crm.core.processes.DetectDuplicatesProcess();
+         case "crm_gdpr_anonymize_contact" -> new com.kingsrook.qbits.crm.core.processes.GdprAnonymizeContactProcess();
+         case "crm_merge_companies" -> new com.kingsrook.qbits.crm.core.processes.MergeCompaniesProcess();
+         case "crm_merge_contacts" -> new com.kingsrook.qbits.crm.core.processes.MergeContactsProcess();
+         case "crm_subscribe_to_list" -> new com.kingsrook.qbits.crm.core.processes.SubscribeToListProcess();
+         case "crm_unsubscribe_from_list" -> new com.kingsrook.qbits.crm.core.processes.UnsubscribeFromListProcess();
+         case "crm_log_activity" -> new com.kingsrook.qbits.crm.activities.processes.LogActivityProcess();
+         default -> throw (new IllegalArgumentException(input.getProcessName()));
+      };
+      com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput output = new com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput();
+      step.run(input, output);
+      return (output);
+   }
+
 }

src/test/java/com/kingsrook/qbits/crm/ContingentPossibleValuesTest.java

@@ -48,7 +48,6 @@ import com.kingsrook.qqq.backend.core.model.data.QRecord;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.PossibleValueSearchFilterUseCase;
 import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValue;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
 import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryRecordStore;
 import com.kingsrook.qqq.backend.core.utils.collections.MapBuilder;
@@ -103,7 +102,7 @@ class ContingentPossibleValuesTest extends BaseTest
 
 
    /*******************************************************************************
-    ** Optional company links and backend-only process inputs stay unrestricted.
+    ** Optional activity links stay unrestricted; logging has standard ETL screens.
     *******************************************************************************/
    @Test
    void testSkippedFields()
@@ -112,8 +111,8 @@ class ContingentPossibleValuesTest extends BaseTest
       assertThat(activity.getField("companyId").getIsRequired()).isFalse();
       assertThat(activity.getField("contactId").getPossibleValueSourceFilter()).isNull();
       assertThat(activity.getField("dealId").getPossibleValueSourceFilter()).isNull();
-      assertThat(QContext.getQInstance().getProcess(LogActivityProcess.NAME).getStepList())
-         .allMatch(step -> step instanceof QBackendStepMetaData);
+      assertThat(QContext.getQInstance().getProcess(LogActivityProcess.NAME).getFrontendStep("details")).isNotNull();
+      assertThat(QContext.getQInstance().getProcess(LogActivityProcess.NAME).getFrontendStep("review")).isNotNull();
    }
 
 

src/test/java/com/kingsrook/qbits/crm/CrmAppMetaDataTest.java

@@ -0,0 +1,169 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2026.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+package com.kingsrook.qbits.crm;
+
+
+import java.util.Map;
+import java.util.Set;
+import java.util.stream.Collectors;
+import com.kingsrook.qbits.crm.deals.model.Deal;
+import com.kingsrook.qbits.crm.widgets.CrmHomeWidgets;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaDataInterface;
+import com.kingsrook.qqq.backend.core.model.metadata.joins.QJoinMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppSection;
+import com.kingsrook.qqq.backend.core.model.metadata.permissions.PermissionLevel;
+import com.kingsrook.qqq.backend.core.model.metadata.qbits.SourceQBitAware;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QFieldSection;
+import org.junit.jupiter.api.Test;
+import static org.assertj.core.api.Assertions.assertThat;
+
+
+/*******************************************************************************
+ ** App navigation and record sections are tested against the validated instance.
+ *******************************************************************************/
+class CrmAppMetaDataTest extends BaseTest
+{
+   /*******************************************************************************
+    ** Exactly six main and fifteen settings tables, no sub-tables or processes.
+    *******************************************************************************/
+   @Test
+   void app()
+   {
+      QAppMetaData app = QContext.getQInstance().getApp("crm");
+      assertThat(QContext.getQInstance().getApps()).hasSize(1);
+      assertThat(app.getLabel()).isEqualTo("CRM");
+      assertThat(app.getIcon().getName()).isEqualTo("contacts");
+      assertThat(app.getSections()).hasSize(2);
+      assertThat(app.getSections().get(0).getTables()).containsExactly("crm_contact", "crm_company", "crm_deal", "crm_activity", "crm_form_submission", "crm_email_list");
+      assertThat(app.getSections().get(1).getTables()).containsExactly("crm_pipeline", "crm_product", "crm_email_template", "crm_email_sequence",
+         "crm_lead_score_rule", "crm_assignment_rule", "crm_sales_goal", "crm_currency", "crm_lifecycle_stage", "crm_lead_source", "crm_industry",
+         "crm_activity_type", "crm_activity_outcome", "crm_contact_role", "crm_win_loss_reason");
+      assertThat(app.getSections().get(1).getLabel()).isEqualTo("Settings");
+      for(QAppSection section : app.getSections())
+      {
+         assertThat(section.getProcesses()).isNullOrEmpty();
+      }
+      assertThat(app.getChildren()).hasSize(21);
+      assertThat(app.getWidgets()).containsExactlyElementsOf(CrmHomeWidgets.HOME);
+      assertThat(app.getPermissionRules().getPermissionBaseName()).isEqualTo("crm");
+      assertThat(app.getPermissionRules().getLevel()).isEqualTo(PermissionLevel.HAS_ACCESS_PERMISSION);
+      assertThat(((SourceQBitAware) app).getSourceQBitName()).isEqualTo("com.kingsrook.qbits:crm");
+      assertThat(CrmQBitProducer.produceAppSection().getTables()).containsExactlyElementsOf(CrmAppMetaData.MAIN_TABLES);
+      assertThat(QContext.getQInstance().getTable("crm_form_submission").getLabel()).isEqualTo("Web Leads");
+      assertThat(QContext.getQInstance().getTable("crm_email_list").getLabel()).isEqualTo("Lists");
+   }
+
+
+   /*******************************************************************************
+    ** Hosts can embed tables without the standalone app; null keeps the default.
+    *******************************************************************************/
+   @Test
+   void optOut() throws Exception
+   {
+      assertThat(new CrmQBitConfig().getAppEnabled()).isTrue();
+      for(Boolean enabled : new Boolean[] {false, true, null})
+      {
+         QInstance instance = new QInstance();
+         new CrmQBitProducer().withQBitConfig(new CrmQBitConfig().withDefaultBackendNameForTables(BACKEND_NAME).withAppEnabled(enabled))
+            .produce(instance).addSelfToInstance(instance);
+         assertThat(instance.getApps()).hasSize(Boolean.FALSE.equals(enabled) ? 0 : 1);
+         assertThat(instance.getTables()).hasSize(34);
+      }
+   }
+
+
+   /*******************************************************************************
+    ** Every child widget has a real join and both joined fields exist.
+    *******************************************************************************/
+   @Test
+   void childLists()
+   {
+      Map<String, Set<String>> expected = Map.of(
+         "crm_contact", Set.of("crm_deal", "crm_activity", "crm_email_list_member", "crm_consent_record", "crm_sequence_enrollment", "crm_email_message", "crm_form_submission"),
+         "crm_company", Set.of("crm_contact", "crm_deal", "crm_activity"),
+         "crm_deal", Set.of("crm_deal_contact", "crm_deal_product", "crm_deal_stage_history", "crm_activity"),
+         "crm_pipeline", Set.of("crm_pipeline_stage"), "crm_email_sequence", Set.of("crm_sequence_step", "crm_sequence_enrollment"),
+         "crm_email_list", Set.of("crm_email_list_member"), "crm_assignment_rule", Set.of("crm_assignment_rule_member"),
+         "crm_activity", Set.of("crm_activity_participant", "crm_email_message"));
+      QInstance instance = QContext.getQInstance();
+      expected.forEach((parent, children) ->
+      {
+         Set<String> actual = instance.getTable(parent).getSections().stream().filter(section -> section.getWidgetName() != null)
+            .map(section -> instance.getWidget(section.getWidgetName()))
+            .filter(widget -> WidgetType.CHILD_RECORD_LIST.getType().equals(widget.getType()))
+            .map(widget ->
+            {
+               String joinName = (String) widget.getDefaultValues().get("joinName");
+               assertThat(joinName).matches("crm_[a-z0-9_]+");
+               QJoinMetaData join = instance.getJoin(joinName);
+               assertThat(join).isNotNull();
+               assertThat(join.getLeftTable()).isEqualTo(parent);
+               join.getJoinOns().forEach(on ->
+               {
+                  assertThat(instance.getTable(parent).getField(on.getLeftField())).isNotNull();
+                  assertThat(instance.getTable(join.getRightTable()).getField(on.getRightField())).isNotNull();
+               });
+               if(parent.equals("crm_contact") && join.getRightTable().equals("crm_deal"))
+               {
+                  assertThat(join.getJoinOns().getFirst().getRightField()).isEqualTo("primaryContactId");
+               }
+               return (join.getRightTable());
+            }).collect(Collectors.toSet());
+         assertThat(actual).isEqualTo(children);
+      });
+      assertThat(instance.getTable("crm_contact").getSections()).extracting(QFieldSection::getWidgetName).contains("crm_contact_timeline");
+   }
+
+
+   /*******************************************************************************
+    ** No retired trends; all widgets have explicit protection and full width.
+    *******************************************************************************/
+   @Test
+   void widgets()
+   {
+      QInstance instance = QContext.getQInstance();
+      assertThat(instance.getWidgets()).doesNotContainKeys("crm_win_rate", "crm_stage_duration", "crm_sales_cycle_length", "crm_pipeline_conversion", "crm_avg_deal_size", "crm_activity_feed");
+      for(QWidgetMetaDataInterface widget : instance.getWidgets().values())
+      {
+         assertThat(widget.getPermissionRules()).isNotNull();
+         assertThat(widget.getGridColumns()).isEqualTo(12);
+         assertThat(widget.getType()).isNotEqualTo(WidgetType.HTML.getType());
+      }
+   }
+
+
+   /*******************************************************************************
+    ** Primary-contact accessors round-trip the new child-list relationship.
+    *******************************************************************************/
+   @Test
+   void primaryContact()
+   {
+      Deal deal = new Deal().withPrimaryContactId(7);
+      assertThat(deal.getPrimaryContactId()).isEqualTo(7);
+      deal.setPrimaryContactId(8);
+      assertThat(deal.getPrimaryContactId()).isEqualTo(8);
+   }
+}

src/test/java/com/kingsrook/qbits/crm/CrmPermissionsTest.java

@@ -49,7 +49,7 @@ class CrmPermissionsTest extends BaseTest
       assertThat(PermissionsHelper.hasTablePermission(input, "crm_contact", TablePermissionSubType.EDIT)).isFalse();
       assertThat(PermissionsHelper.hasTablePermission(input, "crm_pipeline", TablePermissionSubType.READ)).isFalse();
       assertThat(PermissionsHelper.hasProcessPermission(input, "crm_gdpr_anonymize_contact")).isFalse();
-      assertThat(PermissionsHelper.hasWidgetPermission(input, "crm_activity_feed")).isFalse();
+      assertThat(PermissionsHelper.hasWidgetPermission(input, "crm_recent_web_leads")).isFalse();
    }
 
 
@@ -62,7 +62,7 @@ class CrmPermissionsTest extends BaseTest
    {
       QueryInput input = new QueryInput("crm_contact");
       QContext.getQSession().setPermissions(Set.of(CrmPermissions.APP_ACCESS, CrmPermissions.MANAGE_READ, CrmPermissions.MANAGE_WRITE, CrmPermissions.MANAGE));
-      assertThat(PermissionsHelper.hasWidgetPermission(input, "crm_activity_feed")).isTrue();
+      assertThat(PermissionsHelper.hasWidgetPermission(input, "crm_recent_web_leads")).isTrue();
       assertThat(PermissionsHelper.hasTablePermission(input, "crm_pipeline", TablePermissionSubType.EDIT)).isTrue();
       assertThat(PermissionsHelper.hasProcessPermission(input, "crm_merge_contacts")).isTrue();
       assertThat(PermissionsHelper.hasTablePermission(input, "crm_currency", TablePermissionSubType.READ)).isTrue();

src/test/java/com/kingsrook/qbits/crm/CrmQBitProducerTest.java

@@ -142,7 +142,7 @@ class CrmQBitProducerTest extends BaseTest
       {
          assertThat(QContext.getQInstance().getTable(tableName).getAuditRules().getAuditLevel()).isEqualTo(AuditLevel.FIELD);
       }
-      assertThat(QContext.getQInstance().getTables()).hasSize(33)
+      assertThat(QContext.getQInstance().getTables()).hasSize(34)
          .doesNotContainKeys("crm_tag", "crm_contact_tag", "crm_company_tag", "crm_deal_tag", "crm_attachment", "crm_audit_log");
       assertThat(QContext.getQInstance().getProcesses()).doesNotContainKeys("crm_bulk_add_tag", "crm_bulk_remove_tag");
    }

src/test/java/com/kingsrook/qbits/crm/CrmRdbmsTest.java

@@ -44,6 +44,7 @@ import com.kingsrook.qbits.crm.api.CrmIntakeApi;
 import com.kingsrook.qbits.crm.api.CrmMarketingApi;
 import com.kingsrook.qbits.crm.api.FormType;
 import com.kingsrook.qbits.crm.security.CrmPermissions;
+import com.kingsrook.qqq.backend.core.actions.dashboard.RenderWidgetAction;
 import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
 import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
 import com.kingsrook.qqq.backend.core.context.QContext;
@@ -54,6 +55,9 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperat
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
+import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.StatisticsData;
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.TableData;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
 import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
@@ -222,7 +226,7 @@ class CrmRdbmsTest
    @Test
    void testSchemaMatchesMetadata() throws Exception
    {
-      assertThat(qInstance.getTables()).hasSize(33);
+      assertThat(qInstance.getTables()).hasSize(34);
       for(QTableMetaData table : qInstance.getTables().values())
       {
          Set<String> columns = new HashSet<>();
@@ -274,7 +278,7 @@ class CrmRdbmsTest
       String ddl = Files.readString(Path.of("docs/ddl/postgres.sql"));
       String changelog = Files.readString(Path.of("src/main/resources/db/liquibase/qbit-crm/crm.changelog.sql"));
       assertThat(changelog).startsWith("--liquibase formatted sql");
-      assertThat(changelog.lines().filter(line -> line.startsWith("--changeset qbit-crm:")).count()).isEqualTo(33);
+      assertThat(changelog.lines().filter(line -> line.startsWith("--changeset qbit-crm:")).count()).isEqualTo(34);
       assertThat(stripComments(changelog)).isEqualTo(stripComments(ddl));
    }
 
@@ -334,4 +338,36 @@ class CrmRdbmsTest
          .collect(java.util.stream.Collectors.joining("\n")));
    }
 
+
+   /*******************************************************************************
+    ** Home sums and GROUP BY execute against the shipped SQL schema, too.
+    *******************************************************************************/
+   @Test
+   void testHomeAggregates() throws Exception
+   {
+      try(Statement statement = connection.createStatement())
+      {
+         statement.executeUpdate("INSERT INTO crm_pipeline (id, name) VALUES (1, 'Sales')");
+         statement.executeUpdate("INSERT INTO crm_pipeline_stage (id, pipeline_id, name, sort_order, probability_pct, is_closed_won, is_closed_lost) VALUES (1, 1, 'Open', 1, 50, FALSE, FALSE), (2, 1, 'Won', 2, 100, TRUE, FALSE)");
+         statement.executeUpdate("INSERT INTO crm_deal (name, pipeline_id, pipeline_stage_id, owner_user_id, amount_in_base_currency, weighted_amount, expected_close_date, actual_close_date) VALUES ('Open deal', 1, 1, 'Ada', 123.45, 61.73, CURRENT_DATE, NULL), ('Won deal', 1, 2, 'Ada', 20.25, 20.25, NULL, CURRENT_DATE)");
+      }
+      RenderWidgetAction renderer = new RenderWidgetAction();
+      RenderWidgetInput input = new RenderWidgetInput();
+      input.setWidgetMetaData(qInstance.getWidget("crm_open_pipeline"));
+      StatisticsData statistics = (StatisticsData) renderer.execute(input).getWidgetData();
+      assertThat(new BigDecimal(statistics.getCount().toString())).isEqualByComparingTo("123.45");
+      input.setWidgetMetaData(qInstance.getWidget("crm_pipeline_summary"));
+      TableData pipeline = (TableData) renderer.execute(input).getWidgetData();
+      assertThat(pipeline.getRows()).hasSize(2);
+      assertThat(pipeline.getRows().getFirst().get("first").toString()).endsWith(">Open (1 deals)</a>");
+      input.setWidgetMetaData(qInstance.getWidget("crm_deal_forecast"));
+      TableData forecast = (TableData) renderer.execute(input).getWidgetData();
+      assertThat(forecast.getRows()).hasSize(6);
+      assertThat(forecast.getRows().getFirst().get("second").toString()).endsWith(">1</a>");
+      input.setWidgetMetaData(qInstance.getWidget("crm_sales_leaderboard"));
+      TableData leaders = (TableData) renderer.execute(input).getWidgetData();
+      assertThat(leaders.getRows()).hasSize(1);
+      assertThat(leaders.getRows().getFirst().get("first").toString()).endsWith(">Ada</a>");
+   }
+
 }

src/test/java/com/kingsrook/qbits/crm/PlatformReuseTest.java

@@ -0,0 +1,790 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2026.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.crm;
+
+
+import java.time.Instant;
+import java.time.LocalDate;
+import java.time.temporal.ChronoUnit;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import com.kingsrook.qbits.crm.actions.CrmAction;
+import com.kingsrook.qbits.crm.actions.CrmActions;
+import com.kingsrook.qbits.crm.api.CrmApi;
+import com.kingsrook.qbits.crm.api.CrmReadiness;
+import com.kingsrook.qbits.crm.core.CrmRecords;
+import com.kingsrook.qbits.crm.core.model.enums.CrmAssignStrategy;
+import com.kingsrook.qbits.crm.core.model.enums.CrmAssignableEntity;
+import com.kingsrook.qbits.crm.core.model.enums.CrmConsentStatus;
+import com.kingsrook.qbits.crm.core.model.enums.CrmConsentType;
+import com.kingsrook.qbits.crm.core.model.enums.CrmEmailStatus;
+import com.kingsrook.qbits.crm.core.model.enums.CrmEnrollmentStatus;
+import com.kingsrook.qbits.crm.core.model.enums.CrmListMemberStatus;
+import com.kingsrook.qbits.crm.core.model.enums.CrmScoreOperator;
+import com.kingsrook.qbits.crm.core.model.enums.CrmSequenceStepType;
+import com.kingsrook.qbits.crm.deals.CrmDealNotifications;
+import com.kingsrook.qbits.crm.email.CrmEmailProcessIdentity;
+import com.kingsrook.qbits.crm.email.CrmEmailTransport;
+import com.kingsrook.qbits.crm.email.CrmEmails;
+import com.kingsrook.qbits.crm.email.processes.ExpireConsentsProcess;
+import com.kingsrook.qbits.crm.email.processes.ProcessSequenceStepsProcess;
+import com.kingsrook.qbits.crm.email.processes.SyncEmailDeliveryProcess;
+import com.kingsrook.qbits.crm.scoring.processes.CalculateLeadScoresProcess;
+import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
+import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
+import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
+import com.kingsrook.qqq.backend.core.alerts.AlertResolution;
+import com.kingsrook.qqq.backend.core.alerts.AlertSignal;
+import com.kingsrook.qqq.backend.core.alerts.AlertsInstanceMetaData;
+import com.kingsrook.qqq.backend.core.alerts.LoggingAlertSink;
+import com.kingsrook.qqq.backend.core.alerts.NotificationAudience;
+import com.kingsrook.qqq.backend.core.alerts.NotificationSignal;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
+import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordFeature;
+import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordFeaturesMetaData;
+import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryRecordStore;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+
+
+/*******************************************************************************
+ ** P4 contracts exercised through memory DML, fake transport and real QAlerts dispatch.
+ *******************************************************************************/
+class PlatformReuseTest extends BaseTest
+{
+   private FakeTransport transport;
+
+   /*******************************************************************************
+    ** Keep effects local to this test.
+    *******************************************************************************/
+   @BeforeEach
+   void configure() throws Exception
+   {
+      transport = new FakeTransport();
+      CrmApi.config().withPublicBaseUrl("https://crm.example.test").withTokenSecret("test-only-signing-key");
+      Sink.notifications.clear();
+      Sink.alerts.clear();
+      Sink.resolutions.clear();
+      AlertsInstanceMetaData.ofOrWithNew(QContext.getQInstance()).withSink(new QCodeReference(Sink.class));
+   }
+
+   /*******************************************************************************
+    ** Send action renders and records transactional delivery.
+    *******************************************************************************/
+   @Test
+   void sendActionRendersAndRecordsTransactionalDelivery() throws Exception
+   {
+      QRecord contact = contact(1).withValue("companyId", 1);
+      put("crm_company", new QRecord().withValue("id", 1).withValue("name", "Example"));
+      put("crm_contact", contact);
+      put("crm_email_template", new QRecord().withValue("id", 1).withValue("subject", "Hello {{contact.firstName}}")
+         .withValue("bodyText", "{{contact.lastName}} at {{company.name}} / {{owner.name}} / {{unsubscribeUrl}}"));
+      CrmAction.Load load = load();
+      RunBackendStepInput input = new RunBackendStepInput().withProcessName("crm_send_email").withRecords(List.of(contact))
+         .addValue("emailTemplateId", 1).addValue("crmEmailProcessUUID", "run-123");
+      load.runOnePage(input, new RunBackendStepOutput());
+      assertThat(transport.key).isEqualTo("crm:email:1:run-123");
+      assertThat(transport.marketing).isFalse();
+      assertThat(transport.subject).isEqualTo("Hello Alice");
+      assertThat(transport.body).startsWith("Smith at Example / owner / https://crm.example.test/qqq/hooks/crm/unsubscribe/");
+      assertThat(rows("crm_email_message")).singleElement().satisfies(row ->
+      {
+         assertThat(row.getValueInteger("status")).isEqualTo(CrmEmailStatus.QUEUED.getId());
+         assertThat(row.getValueString("emailMessageId")).isEqualTo("provider-1");
+         assertThat(row.getValueString("idempotencyKey")).isEqualTo(transport.key);
+      });
+      assertActivity("EMAIL");
+      load.runOnePage(input, new RunBackendStepOutput());
+      assertThat(transport.sends).isEqualTo(1);
+      assertThat(rows("crm_activity")).hasSize(1);
+   }
+
+   /*******************************************************************************
+    ** Overrides and missing templates and literal merge values.
+    *******************************************************************************/
+   @Test
+   void overridesAndMissingTemplatesAndLiteralMergeValues() throws Exception
+   {
+      QRecord contact = contact(1).withValue("firstName", "{{owner.name}}");
+      assertThat(CrmEmails.render("{{contact.firstName}} {{contact.lastName}}", contact)).isEqualTo("{{owner.name}} Smith");
+      assertThrows(QException.class, () -> CrmEmails.send(transport, contact, 999, null, null, "email", "missing", null));
+      assertThrows(QException.class, () -> CrmEmails.send(transport, contact, null, "subject", "", "email", "missing", null));
+      put("crm_email_template", new QRecord().withValue("id", 1).withValue("subject", "template").withValue("bodyText", "template"));
+      CrmEmails.send(transport, contact, 1, "override", "body", "email", "1:run", null);
+      assertThat(transport.subject).isEqualTo("override");
+      assertThat(transport.body).isEqualTo("body");
+   }
+
+   /*******************************************************************************
+    ** Skips opt out missing address and unavailable transport.
+    *******************************************************************************/
+   @Test
+   void skipsOptOutMissingAddressAndUnavailableTransport() throws Exception
+   {
+      List<QRecord> contacts = List.of(contact(1).withValue("doNotEmail", true), contact(2).withValue("email", null), contact(3));
+      for(QRecord contact : contacts)
+      {
+         put("crm_contact", contact);
+      }
+      transport.ready = false;
+      CrmAction.Load load = load();
+      RunBackendStepOutput output = new RunBackendStepOutput();
+      load.runOnePage(new RunBackendStepInput().withProcessName("crm_send_email").withRecords(contacts), output);
+      assertThat(load.getProcessSummary(output, true)).hasSize(3);
+      assertThat(transport.sends).isZero();
+      assertThat(rows("crm_email_message")).isEmpty();
+      assertThat(rows("crm_activity")).isEmpty();
+   }
+
+   /*******************************************************************************
+    ** Process identity uses core uuid.
+    *******************************************************************************/
+   @Test
+   void processIdentityUsesCoreUuid() throws Exception
+   {
+      var output = new RunProcessAction().execute(new RunProcessInput().withProcessName("crm_send_email"));
+      assertThat(output.getException()).isEmpty();
+      assertThat(output.getValues().get("crmEmailProcessUUID")).isEqualTo(output.getProcessUUID());
+      var listener = new CrmEmailProcessIdentity();
+      assertThat(listener.appliesTo("crm_send_email")).isTrue();
+      assertThat(listener.appliesTo("crm_other")).isFalse();
+      listener.onProcessCompleted(new RunProcessInput(), output);
+      listener.onProcessFailed(new RunProcessInput(), new Exception("test"));
+      assertThat(CrmActions.inputFields(CrmActions.action("crm_send_email"))).extracting(field -> field.getName())
+         .containsExactly("emailTemplateId", "subject", "body");
+   }
+
+   /*******************************************************************************
+    ** Active sequence sends marketing and advances once.
+    *******************************************************************************/
+   @Test
+   void activeSequenceSendsMarketingAndAdvancesOnce() throws Exception
+   {
+      seedSequence(true, true);
+      sequence();
+      assertThat(transport.key).isEqualTo("crm:seq:1:1");
+      assertThat(transport.marketing).isTrue();
+      assertThat(rows("crm_email_message")).hasSize(1);
+      assertActivity("SEQUENCE_STEP");
+      assertThat(GetAction.execute("crm_sequence_enrollment", 1).getValueInteger("status")).isEqualTo(CrmEnrollmentStatus.COMPLETED.getId());
+      sequence();
+      assertThat(transport.sends).isEqualTo(1);
+   }
+
+   /*******************************************************************************
+    ** Paused sequence does not advance or send.
+    *******************************************************************************/
+   @Test
+   void pausedSequenceDoesNotAdvanceOrSend() throws Exception
+   {
+      seedSequence(false, true);
+      sequence();
+      assertThat(transport.sends).isZero();
+      assertThat(GetAction.execute("crm_sequence_enrollment", 1).getValueInteger("currentStepNumber")).isZero();
+      assertThat(Sink.alerts).isEmpty();
+   }
+
+   /*******************************************************************************
+    ** Missing consent pauses enrollment with reason.
+    *******************************************************************************/
+   @Test
+   void missingConsentPausesEnrollmentWithReason() throws Exception
+   {
+      seedSequence(true, false);
+      sequence();
+      QRecord enrollment = GetAction.execute("crm_sequence_enrollment", 1);
+      assertThat(enrollment.getValueInteger("status")).isEqualTo(CrmEnrollmentStatus.PAUSED.getId());
+      assertThat(enrollment.getValueString("lastFailureMessage")).isEqualTo("no_email_consent");
+      assertThat(enrollment.getValueInteger("failureCount")).isZero();
+      assertThat(transport.sends).isZero();
+   }
+
+   /*******************************************************************************
+    ** Transport failures raise on third attempt.
+    *******************************************************************************/
+   @Test
+   void transportFailuresRaiseOnThirdAttempt() throws Exception
+   {
+      seedSequence(true, true);
+      transport.fail = true;
+      sequence();
+      sequence();
+      sequence();
+      assertThat(GetAction.execute("crm_sequence_enrollment", 1).getValueInteger("failureCount")).isEqualTo(3);
+      assertThat(GetAction.execute("crm_sequence_enrollment", 1).getValueInteger("status")).isEqualTo(CrmEnrollmentStatus.FAILED.getId());
+      assertThat(Sink.alerts).singleElement().satisfies(signal ->
+      {
+         assertThat(signal.getAlertTypeName()).isEqualTo("crm.sequence.failing");
+         assertThat(signal.getDedupKey()).isEqualTo("enrollment:1");
+      });
+      assertThat(rows("crm_email_message")).isEmpty();
+   }
+
+   /*******************************************************************************
+    ** Unavailable transport raises then resolves without failing enrollment.
+    *******************************************************************************/
+   @Test
+   void unavailableTransportRaisesThenResolvesWithoutFailingEnrollment() throws Exception
+   {
+      seedSequence(true, true);
+      transport.ready = false;
+      sequence();
+      assertThat(Sink.alerts).singleElement().satisfies(signal ->
+      {
+         assertThat(signal.getAlertTypeName()).isEqualTo("crm.email.unavailable");
+         assertThat(signal.getDedupKey()).isEqualTo("transport");
+      });
+      assertThat(GetAction.execute("crm_sequence_enrollment", 1).getValueInteger("failureCount")).isZero();
+      transport.ready = true;
+      sequence();
+      assertThat(Sink.resolutions).hasSize(1);
+      assertThat(transport.sends).isEqualTo(1);
+   }
+
+   /*******************************************************************************
+    ** Delivery sync uses recent nonfinal messages and preserves dates.
+    *******************************************************************************/
+   @Test
+   void deliverySyncUsesRecentNonfinalMessagesAndPreservesDates() throws Exception
+   {
+      contact(1);
+      message(1, CrmEmailStatus.SENT, Instant.now());
+      message(2, CrmEmailStatus.SENT, Instant.now().minus(31, ChronoUnit.DAYS));
+      message(3, CrmEmailStatus.FAILED, Instant.now());
+      Instant now = Instant.now();
+      transport.deliveries.put("key:1", new CrmEmailTransport.Delivery("provider", CrmEmailStatus.CLICKED, now, now, now, now, null));
+      sync();
+      assertThat(transport.lookups).containsExactly("key:1");
+      QRecord message = GetAction.execute("crm_email_message", 1);
+      assertThat(message.getValueInstant("deliveredDate")).isEqualTo(now);
+      assertThat(message.getValueInstant("firstOpenedDate")).isEqualTo(now);
+      assertThat(message.getValueInstant("firstClickedDate")).isEqualTo(now);
+      transport.deliveries.put("key:1", new CrmEmailTransport.Delivery("provider", CrmEmailStatus.CLICKED, null, null, null, null, null));
+      sync();
+      assertThat(GetAction.execute("crm_email_message", 1).getValueInstant("firstOpenedDate")).isEqualTo(now);
+   }
+
+   /*******************************************************************************
+    ** Bounce and complaint withdraw once and suppress contact.
+    *******************************************************************************/
+   @Test
+   void bounceAndComplaintWithdrawOnceAndSuppressContact() throws Exception
+   {
+      for(int id = 1; id <= 2; id++)
+      {
+         contact(id);
+         message(id, CrmEmailStatus.DELIVERED, Instant.now());
+         CrmEmailStatus status = id == 1 ? CrmEmailStatus.BOUNCED : CrmEmailStatus.COMPLAINED;
+         transport.deliveries.put("key:" + id, new CrmEmailTransport.Delivery("provider", status, null, null, null, null, Instant.now()));
+      }
+      sync();
+      sync();
+      assertThat(rows("crm_contact")).allSatisfy(row -> assertThat(row.getValueBoolean("doNotEmail")).isTrue());
+      assertThat(rows("crm_consent_record")).hasSize(2).allSatisfy(row ->
+      {
+         assertThat(row.getValueInteger("consentType")).isEqualTo(CrmConsentType.EMAIL.getId());
+         assertThat(row.getValueInteger("status")).isEqualTo(CrmConsentStatus.WITHDRAWN.getId());
+      });
+      assertThat(rows("crm_consent_record")).extracting(row -> row.getValueString("source")).containsExactlyInAnyOrder("Hard bounce", "Complaint");
+   }
+
+   /*******************************************************************************
+    ** Absent email qbit is no op.
+    *******************************************************************************/
+   @Test
+   void absentEmailQbitIsNoOp() throws Exception
+   {
+      message(1, CrmEmailStatus.SENT, Instant.now());
+      RunBackendStepOutput output = new RunBackendStepOutput();
+      new SyncEmailDeliveryProcess().run(new RunBackendStepInput(), output);
+      assertThat(output.getValueInteger("updatedCount")).isZero();
+      assertThat(new CrmEmailTransport(CrmApi.config()).lookup("key:1")).isEmpty();
+   }
+
+   /*******************************************************************************
+    ** Stale pending members expire without new consent rows.
+    *******************************************************************************/
+   @Test
+   void stalePendingMembersExpireWithoutNewConsentRows() throws Exception
+   {
+      for(int id = 1; id <= 3; id++)
+      {
+         put("crm_email_list_member", new QRecord().withValue("id", id).withValue("emailListId", 1).withValue("email", "a" + id + "@example.test")
+            .withValue("status", id == 3 ? CrmListMemberStatus.SUBSCRIBED.getId() : CrmListMemberStatus.PENDING_CONFIRMATION.getId())
+            .withValue("confirmationSentDate", Instant.now().minus(id == 2 ? 13 : 15, ChronoUnit.DAYS)).withValue("confirmationTokenHash", "hash"));
+      }
+      put("crm_consent_record", new QRecord().withValue("id", 1).withValue("expiryDate", Instant.now().minusSeconds(1)));
+      RunBackendStepOutput output = new RunBackendStepOutput();
+      new ExpireConsentsProcess().run(new RunBackendStepInput(), output);
+      assertThat(output.getValueInteger("expiredPendingCount")).isEqualTo(1);
+      assertThat(output.getValueInteger("expiredConsentCount")).isEqualTo(1);
+      assertThat(GetAction.execute("crm_email_list_member", 1).getValue("confirmationTokenHash")).isNull();
+      assertThat(GetAction.execute("crm_email_list_member", 1).getValueInteger("status")).isEqualTo(CrmListMemberStatus.UNSUBSCRIBED.getId());
+      assertThat(GetAction.execute("crm_email_list_member", 2).getValueString("confirmationTokenHash")).isEqualTo("hash");
+      assertThat(rows("crm_consent_record")).hasSize(1);
+   }
+
+   /*******************************************************************************
+    ** Schedules require scheduler and can be disabled.
+    *******************************************************************************/
+   @Test
+   void schedulesRequireSchedulerAndCanBeDisabled() throws Exception
+   {
+      Map<String, Integer> intervals = Map.of("crm_process_sequence_steps", 300, "crm_calculate_lead_scores", 3600,
+         "crm_sync_email_delivery", 900, "crm_expire_consents", 86400);
+      for(String name : intervals.keySet())
+      {
+         assertThat(QContext.getQInstance().getProcess(name).getSchedule()).isNull();
+      }
+      CrmQBitConfig config = new CrmQBitConfig().withDefaultBackendNameForTables(BACKEND_NAME).withSchedulerName("jobs");
+      var output = new CrmQBitProducer().withQBitConfig(config).produce(new com.kingsrook.qqq.backend.core.model.metadata.QInstance());
+      for(var process : output.getEach(com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData.class))
+      {
+         if(intervals.containsKey(process.getName()))
+         {
+            assertThat(process.getSchedule().getRepeatSeconds()).isEqualTo(intervals.get(process.getName()));
+            assertThat(process.getSchedule().getSchedulerName()).isEqualTo("jobs");
+         }
+      }
+      assertThat(config.getSchedulesEnabled()).isTrue();
+      output = new CrmQBitProducer().withQBitConfig(config.withSchedulesEnabled(false)).produce(new com.kingsrook.qqq.backend.core.model.metadata.QInstance());
+      assertThat(output.getEach(com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData.class))
+         .allSatisfy(process -> assertThat(process.getSchedule()).isNull());
+   }
+
+   /*******************************************************************************
+    ** Collaboration and alerts use core metadata.
+    *******************************************************************************/
+   @Test
+   void collaborationAndAlertsUseCoreMetadata() throws Exception
+   {
+      for(String table : List.of("crm_contact", "crm_company", "crm_deal", "crm_activity"))
+      {
+         var features = RecordFeaturesMetaData.of(QContext.getQInstance().getTable(table));
+         for(RecordFeature feature : List.of(RecordFeature.TAGS, RecordFeature.COMMENTS, RecordFeature.ATTACHMENTS,
+            RecordFeature.FOLLOW, RecordFeature.TIMELINE, RecordFeature.RECENTS, RecordFeature.FAVORITES))
+         {
+            assertThat(features.getEnabled().get(feature)).isTrue();
+         }
+         assertThat(features.getOwnership().getOwnerFieldName()).isEqualTo("ownerUserId");
+         assertThat(features.getEnabled().get(RecordFeature.TASKS)).isEqualTo("crm_activity".equals(table) ? null : true);
+         assertThat(features.getEnabled().get(RecordFeature.ASSIGNMENT)).isEqualTo("crm_activity".equals(table) ? null : true);
+      }
+      assertThat(AlertsInstanceMetaData.of(QContext.getQInstance()).getAlertTypes()).containsKeys("crm.sequence.failing", "crm.email.unavailable",
+         "crm.lead.assigned", "crm.deal.won", "crm.deal.rotting", "crm.task.due");
+   }
+
+   /*******************************************************************************
+    ** Scoring uses dates incremental changes and daily full pass.
+    *******************************************************************************/
+   @Test
+   void scoringUsesDatesIncrementalChangesAndDailyFullPass() throws Exception
+   {
+      put("crm_contact", contact(1).withValue("lastActivityDate", Instant.now().minus(20, ChronoUnit.DAYS)).withValue("modifyDate", Instant.now().minusSeconds(300)));
+      put("crm_contact", contact(2).withValue("lastActivityDate", Instant.now().minus(2, ChronoUnit.DAYS)).withValue("modifyDate", Instant.now().minusSeconds(300)));
+      rule(1, "lastActivityDate", CrmScoreOperator.DAYS_AGO_GT, "10", -5);
+      rule(2, "lastActivityDate", CrmScoreOperator.DAYS_AGO_LT, "10", 10);
+      assertThat(scores(false).getValueInteger("totalContacts")).isEqualTo(2);
+      assertThat(GetAction.execute("crm_contact", 1).getValueInteger("leadScore")).isEqualTo(-5);
+      assertThat(GetAction.execute("crm_contact", 2).getValueInteger("leadScore")).isEqualTo(10);
+      assertThat(scores(false).getValueInteger("totalContacts")).isZero();
+      CrmRecords.update("crm_contact", new QRecord().withValue("id", 2).withValue("firstName", "Changed"));
+      assertThat(scores(false).getValueInteger("totalContacts")).isEqualTo(1);
+      assertThat(scores(true).getValueInteger("totalContacts")).isEqualTo(2);
+      QRecord state = rows("crm_job_state").getFirst();
+      CrmRecords.update("crm_job_state", state.withValue("lastFullPass", Instant.now().minus(25, ChronoUnit.HOURS)));
+      assertThat(scores(false).getValueInteger("totalContacts")).isEqualTo(2);
+   }
+
+   /*******************************************************************************
+    ** Scoring handles calendar dates and malformed rules.
+    *******************************************************************************/
+   @Test
+   void scoringHandlesCalendarDatesAndMalformedRules() throws Exception
+   {
+      put("crm_contact", contact(1).withValue("lastName", LocalDate.now().minusDays(30).toString()));
+      rule(1, "lastName", CrmScoreOperator.DAYS_AGO_GT, "10", 7);
+      rule(2, "firstName", CrmScoreOperator.DAYS_AGO_LT, "bad", 100);
+      rule(3, "email", CrmScoreOperator.DAYS_AGO_GT, "10", 100);
+      scores(true);
+      assertThat(GetAction.execute("crm_contact", 1).getValueInteger("leadScore")).isEqualTo(7);
+   }
+
+   /*******************************************************************************
+    ** Insert assignment preserves owners and notifies.
+    *******************************************************************************/
+   @Test
+   void insertAssignmentPreservesOwnersAndNotifies() throws Exception
+   {
+      for(CrmAssignableEntity entity : CrmAssignableEntity.values())
+      {
+         put("crm_assignment_rule", new QRecord().withValue("id", entity.getId()).withValue("name", "Assign")
+            .withValue("entityType", entity.getId()).withValue("assignStrategy", CrmAssignStrategy.SPECIFIC_USER.getId())
+            .withValue("assignToUserId", "assigned").withValue("isActive", true).withValue("sortOrder", 1));
+      }
+      QRecord inserted = CrmRecords.insert("crm_contact", new QRecord().withValue("firstName", "New").withValue("lastName", "Contact"));
+      assertThat(GetAction.execute("crm_contact", inserted.getValue("id")).getValueString("ownerUserId")).isEqualTo("assigned");
+      CrmRecords.insert("crm_contact", new QRecord().withValue("firstName", "Already").withValue("lastName", "Owned").withValue("ownerUserId", "explicit"));
+      put("crm_pipeline", new QRecord().withValue("id", 1).withValue("name", "Sales"));
+      put("crm_pipeline_stage", new QRecord().withValue("id", 1).withValue("pipelineId", 1).withValue("name", "Open"));
+      QRecord deal = CrmRecords.insert("crm_deal", new QRecord().withValue("name", "New deal").withValue("pipelineId", 1)
+         .withValue("pipelineStageId", 1).withValue("ownerUserId", "system").withValue("currencyCode", "USD"));
+      assertThat(GetAction.execute("crm_deal", deal.getValue("id")).getValueString("ownerUserId")).isEqualTo("assigned");
+      assertThat(Sink.notifications).hasSize(2).allSatisfy(signal ->
+      {
+         assertThat(signal.getSourceName()).isEqualTo("crm.lead.assigned");
+         assertThat(signal.getAudience()).isEqualTo(NotificationAudience.user("assigned"));
+      });
+   }
+
+   /*******************************************************************************
+    ** Won and rotting notifications target owner and deduplicate stage.
+    *******************************************************************************/
+   @Test
+   void wonAndRottingNotificationsTargetOwnerAndDeduplicateStage() throws Exception
+   {
+      put("crm_pipeline_stage", new QRecord().withValue("id", 1).withValue("name", "Open").withValue("pipelineId", 1).withValue("rotDays", 5));
+      put("crm_pipeline_stage", new QRecord().withValue("id", 2).withValue("name", "Won").withValue("pipelineId", 1).withValue("isClosedWon", true));
+      put("crm_deal", new QRecord().withValue("id", 1).withValue("name", "Old deal").withValue("pipelineId", 1)
+         .withValue("pipelineStageId", 1).withValue("ownerUserId", "owner").withValue("stageEnteredDate", Instant.now().minus(10, ChronoUnit.DAYS)));
+      CrmDealNotifications.checkRotting();
+      CrmDealNotifications.checkRotting();
+      assertThat(Sink.notifications).singleElement().satisfies(signal -> assertThat(signal.getSourceKey()).isEqualTo("deal:1:stage:1"));
+      CrmRecords.update("crm_deal", new QRecord().withValue("id", 1).withValue("pipelineStageId", 2));
+      assertThat(Sink.notifications).hasSize(2);
+      assertThat(Sink.notifications.getLast().getSourceName()).isEqualTo("crm.deal.won");
+      assertThat(Sink.notifications.getLast().getAudience()).isEqualTo(NotificationAudience.user("owner"));
+      CrmRecords.update("crm_deal", new QRecord().withValue("id", 1).withValue("pipelineStageId", 2));
+      CrmDealNotifications.checkRotting();
+      assertThat(Sink.notifications).hasSize(2);
+   }
+
+
+   /*******************************************************************************
+    ** Company loading scales with pages, not contacts or rules.
+    *******************************************************************************/
+   @Test
+   void scoringLoadsCompaniesOncePerPage() throws Exception
+   {
+      put("crm_company", new QRecord().withValue("id", 1).withValue("name", "Example"));
+      for(int id = 1; id <= 201; id++)
+      {
+         put("crm_contact", contact(id).withValue("companyId", 1));
+      }
+      rule(1, "company.name", CrmScoreOperator.EQUALS, "Example", 3);
+      rule(2, "company.name", CrmScoreOperator.IS_SET, null, 2);
+      CompanyReads.count = 0;
+      QContext.getQInstance().getTable("crm_company").withCustomizer(
+         com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizers.POST_QUERY_RECORD, new QCodeReference(CompanyReads.class));
+      assertThat(scores(true).getValueInteger("updatedCount")).isEqualTo(201);
+      // Two explicit page loads, plus at most two core audit-label translations.
+      assertThat(CompanyReads.count).isBetween(2, 4);
+      assertThat(GetAction.execute("crm_contact", 201).getValueInteger("leadScore")).isEqualTo(5);
+   }
+
+   /*******************************************************************************
+    ** Failed scoring must leave the successful checkpoint unchanged.
+    *******************************************************************************/
+   @Test
+   void scoringFailureDoesNotAdvanceCheckpoint() throws Exception
+   {
+      contact(1);
+      scores(true);
+      Instant checkpoint = rows("crm_job_state").getFirst().getValueInstant("lastSuccess");
+      rule(1, "firstName", CrmScoreOperator.IS_SET, null, 1);
+      QContext.getQInstance().getTable("crm_contact").withCustomizer(
+         com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizers.PRE_UPDATE_RECORD, new QCodeReference(RejectScore.class));
+      assertThrows(QException.class, () -> scores(true));
+      assertThat(rows("crm_job_state").getFirst().getValueInstant("lastSuccess")).isEqualTo(checkpoint);
+   }
+
+   /*******************************************************************************
+    ** Retry repairs a missing activity without sending the persisted message again.
+    *******************************************************************************/
+   @Test
+   void retryRepairsActivityAfterPartialRecording() throws Exception
+   {
+      QRecord contact = contact(1);
+      put("crm_email_message", new QRecord().withValue("id", 1).withValue("contactId", 1).withValue("subject", "Already sent")
+         .withValue("idempotencyKey", "crm:email:1:retry").withValue("status", CrmEmailStatus.SENT.getId()));
+      CrmEmails.send(transport, contact, null, "Already sent", "body", "email", "1:retry", null);
+      assertThat(transport.sends).isZero();
+      assertActivity("EMAIL");
+      assertThat(GetAction.execute("crm_email_message", 1).getValue("activityId")).isNotNull();
+   }
+
+   /*******************************************************************************
+    ** Count actual backend company queries.
+    *******************************************************************************/
+   public static class CompanyReads extends com.kingsrook.qqq.backend.core.actions.customizers.AbstractPostQueryCustomizer
+   {
+      private static int count;
+
+      @Override
+      public List<QRecord> apply(List<QRecord> records)
+      {
+         count++;
+         return (records);
+      }
+   }
+
+   /*******************************************************************************
+    ** Return a record-level error to exercise checked background writes.
+    *******************************************************************************/
+   public static class RejectScore extends com.kingsrook.qqq.backend.core.actions.customizers.AbstractPreUpdateCustomizer
+   {
+      @Override
+      public List<QRecord> apply(List<QRecord> records)
+      {
+         records.forEach(row -> row.addError(new com.kingsrook.qqq.backend.core.model.statusmessages.BadInputStatusMessage("Rejected score")));
+         return (records);
+      }
+   }
+
+   /*******************************************************************************
+    ** Send action with a local fake.
+    *******************************************************************************/
+   private CrmAction.Load load()
+   {
+      return (new CrmAction.Load()
+      {
+         @Override
+         protected CrmEmailTransport emailTransport()
+         {
+            return (transport);
+         }
+      });
+   }
+
+   /*******************************************************************************
+    ** Sequence runner with a local fake.
+    *******************************************************************************/
+   private void sequence() throws QException
+   {
+      new ProcessSequenceStepsProcess()
+      {
+         @Override
+         protected CrmEmailTransport emailTransport()
+         {
+            return (transport);
+         }
+      }.run(new RunBackendStepInput(), new RunBackendStepOutput());
+   }
+
+   /*******************************************************************************
+    ** Sync with a local fake.
+    *******************************************************************************/
+   private void sync() throws QException
+   {
+      new SyncEmailDeliveryProcess()
+      {
+         @Override
+         protected CrmEmailTransport emailTransport()
+         {
+            return (transport);
+         }
+      }.run(new RunBackendStepInput(), new RunBackendStepOutput());
+   }
+
+   /*******************************************************************************
+    ** Run score work using normal QQQ actions.
+    *******************************************************************************/
+   private RunBackendStepOutput scores(boolean full) throws QException
+   {
+      RunBackendStepOutput output = new RunBackendStepOutput();
+      new CalculateLeadScoresProcess().run(new RunBackendStepInput().addValue("fullRecalc", full), output);
+      return (output);
+   }
+
+   /*******************************************************************************
+    ** Contact fixture.
+    *******************************************************************************/
+   private QRecord contact(int id) throws QException
+   {
+      QRecord row = new QRecord().withValue("id", id).withValue("firstName", "Alice").withValue("lastName", "Smith")
+         .withValue("email", "alice" + id + "@example.test").withValue("ownerUserId", "owner");
+      put("crm_contact", row);
+      return (row);
+   }
+
+   /*******************************************************************************
+    ** Sequence fixtures use enum identifiers.
+    *******************************************************************************/
+   private void seedSequence(boolean active, boolean eligible) throws QException
+   {
+      contact(1);
+      put("crm_email_sequence", new QRecord().withValue("id", 1).withValue("name", "Sequence").withValue("isActive", active).withValue("totalSteps", 1));
+      put("crm_email_template", new QRecord().withValue("id", 1).withValue("subject", "Hello {{contact.firstName}}").withValue("bodyText", "Welcome"));
+      put("crm_sequence_step", new QRecord().withValue("id", 1).withValue("sequenceId", 1).withValue("stepNumber", 1)
+         .withValue("stepType", CrmSequenceStepType.EMAIL.getId()).withValue("emailTemplateId", 1));
+      put("crm_sequence_enrollment", new QRecord().withValue("id", 1).withValue("sequenceId", 1).withValue("contactId", 1)
+         .withValue("status", CrmEnrollmentStatus.ACTIVE.getId()).withValue("currentStepNumber", 0).withValue("failureCount", 0)
+         .withValue("nextStepDate", Instant.now().minusSeconds(60)));
+      if(eligible)
+      {
+         put("crm_consent_record", new QRecord().withValue("id", 1).withValue("contactId", 1).withValue("consentType", CrmConsentType.EMAIL.getId())
+            .withValue("status", CrmConsentStatus.GRANTED.getId()).withValue("consentDate", Instant.now()));
+      }
+   }
+
+   /*******************************************************************************
+    ** Transport-linked message fixture.
+    *******************************************************************************/
+   private void message(int id, CrmEmailStatus status, Instant created) throws QException
+   {
+      put("crm_email_message", new QRecord().withValue("id", id).withValue("contactId", id).withValue("subject", "Test")
+         .withValue("idempotencyKey", "key:" + id).withValue("status", status.getId()).withValue("createDate", created));
+   }
+
+   /*******************************************************************************
+    ** Score rule fixture.
+    *******************************************************************************/
+   private void rule(int id, String field, CrmScoreOperator operator, String expected, int points) throws QException
+   {
+      put("crm_lead_score_rule", new QRecord().withValue("id", id).withValue("name", "Rule " + id).withValue("fieldPath", field)
+         .withValue("operator", operator.getId()).withValue("fieldValue", expected).withValue("scoreAdjustment", points).withValue("isActive", true));
+   }
+
+   /*******************************************************************************
+    ** Assert the activity's stable code.
+    *******************************************************************************/
+   private void assertActivity(String code) throws QException
+   {
+      QRecord activity = rows("crm_activity").getFirst();
+      assertThat(GetAction.execute("crm_activity_type", activity.getValue("activityTypeId")).getValueString("code")).isEqualTo(code);
+   }
+
+   /*******************************************************************************
+    ** Insert isolated fixture data.
+    *******************************************************************************/
+   private void put(String table, QRecord row) throws QException
+   {
+      if(row.getValue("id") != null && GetAction.execute(table, row.getValue("id")) != null)
+      {
+         MemoryRecordStore.getInstance().update(new com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput(table).withRecord(row), true);
+      }
+      else
+      {
+         MemoryRecordStore.getInstance().insert(new InsertInput(table).withRecord(row), true);
+      }
+   }
+
+   /*******************************************************************************
+    ** Query in the current session.
+    *******************************************************************************/
+   private List<QRecord> rows(String table) throws QException
+   {
+      return (new QueryAction().execute(new QueryInput(table)).getRecords());
+   }
+
+   /*******************************************************************************
+    ** Fake transport with independent send and delivery controls.
+    *******************************************************************************/
+   private static class FakeTransport extends CrmEmailTransport
+   {
+      private boolean ready = true;
+      private boolean fail;
+      private boolean marketing;
+      private int sends;
+      private String key;
+      private String subject;
+      private String body;
+      private final Map<String, Delivery> deliveries = new HashMap<>();
+      private final List<String> lookups = new ArrayList<>();
+
+      /*******************************************************************************
+       ** No external provider is installed.
+       *******************************************************************************/
+      FakeTransport()
+      {
+         super(new CrmQBitConfig());
+      }
+
+      @Override
+      public CrmReadiness readiness()
+      {
+         return (ready ? CrmReadiness.QBIT_EMAIL : CrmReadiness.NOT_CONFIGURED);
+      }
+
+      @Override
+      public Delivery sendMessage(String purpose, String id, Integer contactId, String address, String subject, String body, boolean marketing) throws QException
+      {
+         if(fail)
+         {
+            throw (new QException("Fake transport failure"));
+         }
+         sends++;
+         key = "crm:" + purpose + ":" + id;
+         this.subject = subject;
+         this.body = body;
+         this.marketing = marketing;
+         return (new Delivery("provider-1", CrmEmailStatus.QUEUED, null, null, null, null, null));
+      }
+
+      @Override
+      public Optional<Delivery> lookup(String key)
+      {
+         lookups.add(key);
+         return (Optional.ofNullable(deliveries.get(key)));
+      }
+   }
+
+   /*******************************************************************************
+    ** Capture the core dispatch without an alerts backend.
+    *******************************************************************************/
+   public static class Sink extends LoggingAlertSink
+   {
+      private static final List<NotificationSignal> notifications = new ArrayList<>();
+      private static final List<AlertSignal> alerts = new ArrayList<>();
+      private static final List<AlertResolution> resolutions = new ArrayList<>();
+
+      @Override
+      public void notify(NotificationSignal signal)
+      {
+         notifications.add(signal);
+      }
+
+      @Override
+      public void raise(AlertSignal signal)
+      {
+         alerts.add(signal);
+      }
+
+      @Override
+      public void resolve(AlertResolution resolution)
+      {
+         resolutions.add(resolution);
+      }
+   }
+}

src/test/java/com/kingsrook/qbits/crm/actions/CrmActionsTest.java

@@ -0,0 +1,662 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2026.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+package com.kingsrook.qbits.crm.actions;
+
+
+import java.io.Serializable;
+import java.math.BigDecimal;
+import java.time.Instant;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.TreeMap;
+import java.util.stream.Stream;
+import com.kingsrook.qbits.crm.BaseTest;
+import com.kingsrook.qbits.crm.security.CrmPermissions;
+import com.kingsrook.qqq.backend.core.actions.customizers.AbstractPostQueryCustomizer;
+import com.kingsrook.qqq.backend.core.actions.customizers.AbstractPreUpdateCustomizer;
+import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizers;
+import com.kingsrook.qqq.backend.core.actions.permissions.PermissionsHelper;
+import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
+import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
+import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessOutput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
+import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
+import com.kingsrook.qqq.backend.core.model.metadata.permissions.DenyBehavior;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
+import com.kingsrook.qqq.backend.core.model.statusmessages.BadInputStatusMessage;
+import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryRecordStore;
+import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
+import com.kingsrook.qqq.backend.core.utils.JsonUtils;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.MethodSource;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+
+/*******************************************************************************
+ ** Every UI action traverses real ETL Review and Result against persisted data.
+ *******************************************************************************/
+class CrmActionsTest extends BaseTest
+{
+   /*******************************************************************************
+    ** Cover the complete registered action inventory.
+    *******************************************************************************/
+   static Stream<String> actions()
+   {
+      return (CrmActions.ACTIONS.stream().map(CrmActions.Action::name));
+   }
+
+   /*******************************************************************************
+    ** Review is read-only; Result describes the actual effect on selected rows.
+    *******************************************************************************/
+   @ParameterizedTest
+   @MethodSource("actions")
+   void everyActionReviewsAndExecutes(String name) throws Exception
+   {
+      CrmActions.Action action = CrmActions.action(name);
+      seed(action);
+      Map<String, Serializable> inputs = inputs();
+      Map<String, String> before = snapshot();
+      RunProcessOutput review = review(name, inputs, List.of(1, 2));
+      assertThat(snapshot()).as("Review must not write any CRM table").isEqualTo(before);
+      List<ProcessSummaryLine> reviewLines = lines(review, false);
+      assertThat(reviewLines).isNotEmpty();
+      assertThat(reviewLines).allSatisfy(line ->
+      {
+         assertThat(line.getSingularFutureMessage()).isNotBlank();
+         assertThat(line.getPluralFutureMessage()).isNotBlank();
+         assertThat(line.getSingularPastMessage()).isNotBlank();
+         assertThat(line.getPluralPastMessage()).isNotBlank();
+         assertThat(line.getPrimaryKeys()).isNotEmpty();
+      });
+      RunProcessOutput result = resume(name, review, Map.of());
+      assertThat(result.getProcessState().getNextStepName()).hasValue("result");
+      List<ProcessSummaryLine> resultLines = lines(result, true);
+      assertThat(resultLines).isNotEmpty();
+      assertThat(resultLines).noneMatch(line -> line.getStatus() == Status.ERROR);
+      if(action.operation().equals("email"))
+      {
+         assertThat(resultLines).allMatch(line -> line.getStatus() == Status.WARNING);
+         assertThat(resultLines.getFirst().getSingularPastMessage()).isNotBlank();
+         assertThat(snapshot()).isEqualTo(before);
+      }
+      else
+      {
+         assertThat(reviewLines).anyMatch(line -> line.getStatus() == Status.OK);
+         assertThat(resultLines).anyMatch(line -> line.getStatus() == Status.OK);
+         verifyEffect(action, resultLines);
+      }
+      if(!List.of("merge", "convert").contains(action.operation()))
+      {
+         assertThat(GetAction.execute(action.table(), 3).getValueString("ownerUserId")).isEqualTo("original");
+      }
+   }
+
+   /*******************************************************************************
+    ** Bindings, input screens, permissions, selection limits and removed processes.
+    *******************************************************************************/
+   @Test
+   void metadataIsComplete()
+   {
+      assertThat(CrmActions.ACTIONS).hasSize(28);
+      assertThat(QContext.getQInstance().getProcesses().keySet().stream().filter(name -> !name.contains("."))).hasSize(32);
+      for(CrmActions.Action action : CrmActions.ACTIONS)
+      {
+         QProcessMetaData process = QContext.getQInstance().getProcess(action.name());
+         assertThat(process.getTableName()).isEqualTo(action.table());
+         assertThat(process.getMinInputRecords()).isPositive();
+         assertThat(process.getPermissionRules().getDenyBehavior()).isEqualTo(DenyBehavior.HIDDEN);
+         assertThat(process.getFrontendStep("review")).isNotNull();
+         assertThat(process.getFrontendStep("result")).isNotNull();
+         if(!CrmActions.inputFields(action).isEmpty())
+         {
+            assertThat(process.getFrontendStep("details")).isNotNull();
+            assertThat(process.getStepList().indexOf(process.getFrontendStep("details")))
+               .isLessThan(process.getStepList().indexOf(process.getFrontendStep("review")));
+         }
+         if(action.operation().equals("merge"))
+         {
+            assertThat(process.getMinInputRecords()).isEqualTo(2);
+            assertThat(process.getMaxInputRecords()).isEqualTo(2);
+         }
+      }
+      for(String name : List.of("crm_process_sequence_steps", "crm_calculate_lead_scores", "crm_sync_email_delivery", "crm_expire_consents"))
+      {
+         assertThat(QContext.getQInstance().getProcess(name).getTableName()).isNull();
+      }
+      assertThat(QContext.getQInstance().getProcesses()).doesNotContainKeys("crm_fire_reminders", "crm_process_overdue_tasks", "crm_import_contacts", "crm_track_email_event");
+      assertThat(QContext.getQInstance().getProcess("crm_move_deal_stage").getInputField("targetStageId").orElseThrow().getPossibleValueSourceFilter()).isNotNull();
+   }
+
+   /*******************************************************************************
+    ** Administrative and management actions have distinct permission grants.
+    *******************************************************************************/
+   @Test
+   void permissionBoundaries() throws Exception
+   {
+      QueryInput input = new QueryInput("crm_contact");
+      QContext.getQSession().setPermissions(Set.of(CrmPermissions.APP_ACCESS, CrmPermissions.READ, CrmPermissions.WRITE));
+      for(String name : List.of("crm_bulk_assign_owner", "crm_assign_company_owner", "crm_transfer_ownership", "crm_merge_contacts", "crm_merge_companies",
+         "crm_subscribe_to_list", "crm_enroll_in_sequence", "crm_activate_sequence", "crm_pause_sequence", "crm_gdpr_anonymize_contact"))
+      {
+         assertThat(PermissionsHelper.hasProcessPermission(input, name)).as(name).isFalse();
+      }
+      assertThat(PermissionsHelper.hasProcessPermission(input, "crm_log_activity")).isTrue();
+   }
+
+   /*******************************************************************************
+    ** No writes when the selected target belongs to another pipeline.
+    *******************************************************************************/
+   @Test
+   void crossPipelineIsWarningInReviewAndResult() throws Exception
+   {
+      seed(CrmActions.action("crm_move_deal_stage"));
+      Map<String, Serializable> values = inputs();
+      values.put("targetStageId", 5);
+      RunProcessOutput review = review("crm_move_deal_stage", values, List.of(1, 2));
+      assertThat(lines(review, false)).allMatch(line -> line.getStatus() == Status.WARNING);
+      RunProcessOutput result = resume("crm_move_deal_stage", review, Map.of());
+      assertThat(lines(result, true)).allMatch(line -> line.getStatus() == Status.WARNING);
+      assertThat(GetAction.execute("crm_deal", 1).getValueInteger("pipelineStageId")).isEqualTo(1);
+   }
+
+   /*******************************************************************************
+    ** Missing input is not silently accepted through the process API.
+    *******************************************************************************/
+   @Test
+   void missingOwnerIsWarning() throws Exception
+   {
+      seed(CrmActions.action("crm_bulk_assign_owner"));
+      RunProcessOutput review = review("crm_bulk_assign_owner", Map.of(), List.of(1, 2));
+      assertThat(lines(review, false)).allMatch(line -> line.getStatus() == Status.WARNING);
+      RunProcessOutput result = resume("crm_bulk_assign_owner", review, Map.of());
+      assertThat(lines(result, true)).allMatch(line -> line.getStatus() == Status.WARNING);
+      assertThat(GetAction.execute("crm_contact", 1).getValueString("ownerUserId")).isEqualTo("original");
+   }
+
+   /*******************************************************************************
+    ** Read-only users cannot mutate even if they can see the process.
+    *******************************************************************************/
+   @Test
+   void readOnlySelectionIsSkipped() throws Exception
+   {
+      seed(CrmActions.action("crm_log_activity"));
+      QContext.getQSession().setPermissions(Set.of(CrmPermissions.APP_ACCESS, CrmPermissions.READ, CrmPermissions.MANAGE_READ));
+      RunProcessOutput review = review("crm_log_activity", inputs(), List.of(1));
+      assertThat(lines(review, false)).allMatch(line -> line.getStatus() == Status.WARNING);
+      RunProcessOutput result = resume("crm_log_activity", review, Map.of());
+      assertThat(lines(result, true)).allMatch(line -> line.getStatus() == Status.WARNING);
+      assertThat(rows("crm_activity")).isEmpty();
+   }
+
+   /*******************************************************************************
+    ** API callers cannot merge one/three records or select an unrelated survivor.
+    *******************************************************************************/
+   @Test
+   void mergeRequiresExactlyTwoSelectedRecords() throws Exception
+   {
+      seed(CrmActions.action("crm_merge_contacts"));
+      Map<String, String> before = snapshot();
+      for(List<Integer> ids : List.of(List.of(1), List.of(1, 2, 3)))
+      {
+         assertThatThrownBy(() -> review("crm_merge_contacts", inputs(), ids)).isInstanceOf(QException.class)
+            .hasMessageContaining("exactly two");
+      }
+      Map<String, Serializable> values = inputs();
+      values.put("primaryId", 3);
+      assertThatThrownBy(() -> review("crm_merge_contacts", values, List.of(1, 2))).isInstanceOf(QException.class)
+         .hasMessageContaining("one of the two selected");
+      assertThat(snapshot()).isEqualTo(before);
+   }
+
+   /*******************************************************************************
+    ** Missing selections cannot accidentally process an entire table.
+    *******************************************************************************/
+   @Test
+   void selectionIsRequired() throws Exception
+   {
+      seed(CrmActions.action("crm_auto_assign"));
+      assertThatThrownBy(() -> new RunProcessAction().execute(new RunProcessInput().withProcessName("crm_auto_assign")))
+         .isInstanceOf(QException.class).hasMessageContaining("Select records");
+      assertThat(GetAction.execute("crm_contact", 1).getValueString("ownerUserId")).isEqualTo("original");
+   }
+
+   /*******************************************************************************
+    ** A failure on one row is reported as ERROR while other selected rows complete.
+    *******************************************************************************/
+   @Test
+   void resultReportsRejectedUpdates() throws Exception
+   {
+      seed(CrmActions.action("crm_assign_company_owner"));
+      RunProcessOutput review = review("crm_assign_company_owner", inputs(), List.of(1, 2));
+      assertThat(lines(review, false)).allMatch(line -> line.getStatus() == Status.OK);
+      QContext.getQInstance().getTable("crm_company").withCustomizer(TableCustomizers.PRE_UPDATE_RECORD, new QCodeReference(RejectFirstUpdate.class));
+      RunProcessOutput result = resume("crm_assign_company_owner", review, Map.of());
+      assertThat(lines(result, true)).anySatisfy(line ->
+      {
+         assertThat(line.getStatus()).isEqualTo(Status.ERROR);
+         assertThat(line.getPrimaryKeys()).containsExactly(1);
+         assertThat(line.getSingularPastMessage()).contains("rejected");
+      });
+      assertThat(lines(result, true)).anySatisfy(line ->
+      {
+         assertThat(line.getStatus()).isEqualTo(Status.OK);
+         assertThat(line.getPrimaryKeys()).containsExactly(2);
+      });
+      assertThat(GetAction.execute("crm_company", 1).getValueString("ownerUserId")).isEqualTo("original");
+      assertThat(GetAction.execute("crm_company", 2).getValueString("ownerUserId")).isEqualTo("new-owner");
+   }
+
+   /*******************************************************************************
+    ** Test-only backend rejection after the user has reviewed the selection.
+    *******************************************************************************/
+   public static class RejectFirstUpdate extends AbstractPreUpdateCustomizer
+   {
+      @Override
+      public List<QRecord> apply(List<QRecord> records)
+      {
+         records.stream().filter(row -> Integer.valueOf(1).equals(row.getValueInteger("id")))
+            .forEach(row -> row.addError(new BadInputStatusMessage("Update rejected for testing")));
+         return (records);
+      }
+   }
+
+   /*******************************************************************************
+    ** Revalidate consent-related suppression changed after the Review screen.
+    *******************************************************************************/
+   @Test
+   void enrollmentRechecksSuppressionAfterReview() throws Exception
+   {
+      seed(CrmActions.action("crm_enroll_in_sequence"));
+      RunProcessOutput review = review("crm_enroll_in_sequence", inputs(), List.of(1, 2));
+      new com.kingsrook.qqq.backend.core.actions.tables.UpdateAction().execute(
+         new com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput("crm_contact")
+            .withRecord(new QRecord().withValue("id", 1).withValue("doNotEmail", true)));
+      RunProcessOutput result = resume("crm_enroll_in_sequence", review, Map.of());
+      assertThat(lines(result, true)).anyMatch(line -> line.getStatus() == Status.WARNING && line.getPrimaryKeys().equals(List.of(1)));
+      assertThat(rows("crm_sequence_enrollment")).extracting(row -> row.getValueInteger("contactId")).containsExactly(2);
+   }
+
+   /*******************************************************************************
+    ** Required stage fields are checked before any writes.
+    *******************************************************************************/
+   @Test
+   void stageRequiredFieldIsReviewed() throws Exception
+   {
+      seed(CrmActions.action("crm_move_deal_stage"));
+      change("crm_pipeline_stage", new QRecord().withValue("id", 2).withValue("pipelineId", 1).withValue("name", "Needs company")
+         .withValue("requiredFieldsJson", "[\"companyId\"]"));
+      RunProcessOutput review = review("crm_move_deal_stage", inputs(), List.of(1, 2));
+      assertThat(lines(review, false)).allMatch(line -> line.getStatus() == Status.WARNING);
+      RunProcessOutput result = resume("crm_move_deal_stage", review, Map.of());
+      assertThat(lines(result, true)).allMatch(line -> line.getStatus() == Status.WARNING);
+      assertThat(rows("crm_deal_stage_history")).isEmpty();
+   }
+
+   /*******************************************************************************
+    ** Invalid configuration produces standard ERROR lines on both screens.
+    *******************************************************************************/
+   @Test
+   void validationErrorsAppearInReviewAndResult() throws Exception
+   {
+      seed(CrmActions.action("crm_move_deal_stage"));
+      change("crm_pipeline_stage", new QRecord().withValue("id", 2).withValue("requiredFieldsJson", "invalid json"));
+      RunProcessOutput review = review("crm_move_deal_stage", inputs(), List.of(1, 2));
+      assertThat(lines(review, false)).allMatch(line -> line.getStatus() == Status.ERROR);
+      RunProcessOutput result = resume("crm_move_deal_stage", review, Map.of());
+      assertThat(lines(result, true)).allMatch(line -> line.getStatus() == Status.ERROR);
+      assertThat(rows("crm_deal_stage_history")).isEmpty();
+   }
+
+   /*******************************************************************************
+    ** Request values cannot redirect an action to another table or record id.
+    *******************************************************************************/
+   @Test
+   void installedBindingOwnsTheSelection() throws Exception
+   {
+      seed(CrmActions.action("crm_bulk_assign_owner"));
+      Map<String, Serializable> values = inputs();
+      values.put("sourceTable", "crm_company");
+      values.put("destinationTable", "crm_company");
+      values.put("recordId", 3);
+      values.put("contactId", 3);
+      RunProcessOutput result = resume("crm_bulk_assign_owner", review("crm_bulk_assign_owner", values, List.of(1, 2)), Map.of());
+      assertThat(lines(result, true)).allMatch(line -> line.getStatus() == Status.OK);
+      assertThat(rows("crm_company")).allMatch(row -> "original".equals(row.getValueString("ownerUserId")));
+      assertThat(GetAction.execute("crm_contact", 1).getValueString("ownerUserId")).isEqualTo("new-owner");
+      assertThat(GetAction.execute("crm_contact", 3).getValueString("ownerUserId")).isEqualTo("original");
+   }
+
+   /*******************************************************************************
+    ** No rules produces an honest warning instead of a successful assignment.
+    *******************************************************************************/
+   @Test
+   void autoAssignNoMatchIsWarning() throws Exception
+   {
+      seed(CrmActions.action("crm_auto_assign"));
+      change("crm_assignment_rule", new QRecord().withValue("id", 1).withValue("isActive", false));
+      RunProcessOutput result = resume("crm_auto_assign", review("crm_auto_assign", inputs(), List.of(1, 2)), Map.of());
+      assertThat(lines(result, true)).allMatch(line -> line.getStatus() == Status.WARNING);
+      assertThat(GetAction.execute("crm_contact", 1).getValueString("ownerUserId")).isEqualTo("original");
+   }
+
+   /*******************************************************************************
+    ** Scoring crosses two full keyset pages, including non-contiguous ids.
+    *******************************************************************************/
+   @Test
+   void scoringPagesInBatchesOf200() throws Exception
+   {
+      seed(CrmActions.action("crm_recalculate_score"));
+      for(int i = 4; i <= 405; i++)
+      {
+         put("crm_contact", new QRecord().withValue("id", 10 + i * 2).withValue("lastName", "Contact").withValue("ownerUserId", "original"));
+      }
+      PageObserver.sizes.clear();
+      QContext.getQInstance().getTable("crm_contact").withCustomizer(TableCustomizers.POST_QUERY_RECORD, new QCodeReference(PageObserver.class));
+      RunProcessOutput result = new RunProcessAction().execute(new RunProcessInput().withProcessName("crm_calculate_lead_scores"));
+      assertThat(result.getValues().get("totalContacts")).isEqualTo(405);
+      assertThat(result.getValues().get("updatedCount")).isEqualTo(405);
+      assertThat(PageObserver.sizes).containsExactly(200, 200, 5);
+      QContext.getQInstance().getTable("crm_contact").getCustomizers().remove(TableCustomizers.POST_QUERY_RECORD.getRole());
+      assertThat(rows("crm_contact")).hasSize(405).allMatch(row -> row.getValueInteger("leadScore") == 10);
+   }
+
+   /*******************************************************************************
+    ** Changing enrollment status cannot shift an offset and skip the next page.
+    *******************************************************************************/
+   @Test
+   void sequencePagesInBatchesOf200() throws Exception
+   {
+      seed(CrmActions.action("crm_enroll_in_sequence"));
+      for(int i = 1; i <= 405; i++)
+      {
+         put("crm_sequence_enrollment", new QRecord().withValue("id", i * 2).withValue("sequenceId", 1).withValue("contactId", 1)
+            .withValue("status", 1).withValue("currentStepNumber", 0).withValue("failureCount", 0).withValue("nextStepDate", Instant.now().minusSeconds(60)));
+      }
+      put("crm_sequence_enrollment", new QRecord().withValue("id", 9999).withValue("sequenceId", 1).withValue("contactId", 1)
+         .withValue("status", 1).withValue("currentStepNumber", 0).withValue("nextStepDate", Instant.now().plusSeconds(86400)));
+      PageObserver.sizes.clear();
+      QContext.getQInstance().getTable("crm_sequence_enrollment").withCustomizer(TableCustomizers.POST_QUERY_RECORD, new QCodeReference(PageObserver.class));
+      RunProcessOutput result = new RunProcessAction().execute(new RunProcessInput().withProcessName("crm_process_sequence_steps"));
+      assertThat(result.getValues().get("processedCount")).isEqualTo(405);
+      assertThat(PageObserver.sizes).containsExactly(200, 200, 5);
+      QContext.getQInstance().getTable("crm_sequence_enrollment").getCustomizers().remove(TableCustomizers.POST_QUERY_RECORD.getRole());
+      assertThat(rows("crm_sequence_enrollment").stream().filter(row -> row.getValueInteger("id") != 9999))
+         .hasSize(405).allMatch(row -> row.getValueInteger("status") == 3);
+      assertThat(GetAction.execute("crm_sequence_enrollment", 9999).getValueInteger("status")).isEqualTo(1);
+   }
+
+   /*******************************************************************************
+    ** Observe the actual query bounds, not just the final job counts.
+    *******************************************************************************/
+   public static class PageObserver extends AbstractPostQueryCustomizer
+   {
+      private static final List<Integer> sizes = new ArrayList<>();
+
+      @Override
+      public List<QRecord> apply(List<QRecord> records)
+      {
+         if(input instanceof QueryInput query && query.getFilter().getOrderBys() != null && !query.getFilter().getOrderBys().isEmpty())
+         {
+            assertThat(query.getFilter().getLimit()).isEqualTo(200);
+            assertThat(query.getFilter().getOrderBys().getFirst().getFieldName()).isEqualTo("id");
+            sizes.add(records.size());
+         }
+         return (records);
+      }
+   }
+
+   /*******************************************************************************
+    ** Verify observable writes, including merge relationships and duplicate links.
+    *******************************************************************************/
+   private void verifyEffect(CrmActions.Action action, List<ProcessSummaryLine> summary) throws QException
+   {
+      QRecord first = GetAction.execute(action.table(), 1);
+      switch(action.operation())
+      {
+         case "assign" -> assertThat(first.getValueString("ownerUserId")).isEqualTo("new-owner");
+         case "auto" -> assertThat(first.getValueString("ownerUserId")).isEqualTo("auto-owner");
+         case "subscribe" -> assertThat(rows("crm_email_list_member")).hasSize(2);
+         case "unsubscribe" -> assertThat(rows("crm_email_list_member")).allMatch(row -> row.getValueInstant("unsubscribedDate") != null);
+         case "enroll" -> assertThat(rows("crm_sequence_enrollment")).hasSize(2);
+         case "unenroll" -> assertThat(rows("crm_sequence_enrollment")).allMatch(row -> row.getValueInstant("unenrolledDate") != null);
+         case "activity" ->
+         {
+            assertThat(rows("crm_activity")).hasSize(2);
+            String foreignKey = action.table().substring(4) + "Id";
+            assertThat(rows("crm_activity")).extracting(row -> row.getValueInteger(foreignKey)).containsExactlyInAnyOrder(1, 2);
+         }
+         case "score" -> assertThat(first.getValueInteger("leadScore")).isEqualTo(10);
+         case "merge" ->
+         {
+            assertThat(GetAction.execute(action.table(), 2)).isNull();
+            assertThat(GetAction.execute(action.table(), 3)).isNotNull();
+         }
+         case "duplicates" ->
+         {
+            assertThat(summary).anySatisfy(line ->
+            {
+               assertThat(line.getStatus()).isEqualTo(Status.WARNING);
+               assertThat(line.getPrimaryKeys()).containsExactlyInAnyOrder(1, 2);
+            });
+            assertThat(rows(action.table())).hasSize(3);
+         }
+         case "createDeal", "clone" -> assertThat(rows("crm_deal")).hasSize(5);
+         case "erase" ->
+         {
+            assertThat(first.getValueBoolean("doNotEmail")).isTrue();
+            assertThat(first.getValueString("email")).endsWith("@removed.invalid");
+         }
+         case "stage" -> assertThat(first.getValueInteger("pipelineStageId")).isEqualTo(2);
+         case "won" ->
+         {
+            assertThat(first.getValueInteger("pipelineStageId")).isEqualTo(3);
+            assertThat(first.getValueInteger("winLossReasonId")).isEqualTo(1);
+            assertThat(first.getValueLocalDate("actualCloseDate")).isNotNull();
+            assertThat(rows("crm_deal_stage_history")).hasSize(2);
+         }
+         case "lost" -> assertThat(first.getValueInteger("pipelineStageId")).isEqualTo(4);
+         case "reopen" ->
+         {
+            assertThat(first.getValueInteger("pipelineStageId")).isEqualTo(2);
+            assertThat(first.getValue("actualCloseDate")).isNull();
+            assertThat(first.getValue("winLossReasonId")).isNull();
+         }
+         case "addContact" -> assertThat(rows("crm_deal_contact")).hasSize(2);
+         case "convert" ->
+         {
+            assertThat(first.getValueInteger("contactId")).isNotNull();
+            assertThat(GetAction.execute("crm_form_submission", 2).getValueInteger("contactId")).isNotNull();
+            assertThat(rows("crm_contact")).hasSize(5);
+         }
+         case "activate" -> assertThat(first.getValueBoolean("isActive")).isTrue();
+         case "pause" -> assertThat(first.getValueBoolean("isActive")).isFalse();
+         default -> throw (new AssertionError(action.name()));
+      }
+   }
+
+   /*******************************************************************************
+    ** Start, complete input details, and stop at the actual standard Review screen.
+    *******************************************************************************/
+   private RunProcessOutput review(String name, Map<String, Serializable> values, List<Integer> ids) throws QException
+   {
+      RunProcessInput request = new RunProcessInput();
+      request.setProcessName(name);
+      request.setValues(new HashMap<>(values));
+      request.addValue("queryFilterJson", JsonUtils.toJson(new QQueryFilter(new QFilterCriteria("id", QCriteriaOperator.IN, ids))));
+      RunProcessOutput output = new RunProcessAction().execute(request);
+      assertThat(output.getException()).isEmpty();
+      if(output.getProcessState().getNextStepName().orElse("").equals("details"))
+      {
+         output = resume(name, output, values);
+      }
+      assertThat(output.getProcessState().getNextStepName()).hasValue("review");
+      return (output);
+   }
+
+   /*******************************************************************************
+    ** Resume the process just as the frontend submits its current screen.
+    *******************************************************************************/
+   private RunProcessOutput resume(String name, RunProcessOutput prior, Map<String, Serializable> values) throws QException
+   {
+      RunProcessInput request = new RunProcessInput();
+      request.setProcessName(name);
+      request.setProcessUUID(prior.getProcessUUID());
+      request.setStartAfterStep(prior.getProcessState().getNextStepName().orElseThrow());
+      request.setValues(new HashMap<>(values));
+      RunProcessOutput output = new RunProcessAction().execute(request);
+      assertThat(output.getException()).isEmpty();
+      return (output);
+   }
+
+   /*******************************************************************************
+    ** Read standard ETL summaries, not test-specific step state.
+    *******************************************************************************/
+   @SuppressWarnings("unchecked")
+   private List<ProcessSummaryLine> lines(RunProcessOutput output, boolean result)
+   {
+      return ((List<ProcessSummaryLine>) output.getValues().get(result ? StreamedETLWithFrontendProcess.FIELD_PROCESS_SUMMARY : StreamedETLWithFrontendProcess.FIELD_VALIDATION_SUMMARY));
+   }
+
+   /*******************************************************************************
+    ** Supply all action inputs; each operation accepts only its declared fields.
+    *******************************************************************************/
+   private Map<String, Serializable> inputs()
+   {
+      Map<String, Serializable> values = new HashMap<>();
+      values.put("newOwnerUserId", "new-owner");
+      values.put("emailListId", 1);
+      values.put("sequenceId", 1);
+      values.put("primaryId", 1);
+      values.put("pipelineId", 1);
+      values.put("targetStageId", 2);
+      values.put("winLossReasonId", 1);
+      values.put("dealName", "New deal");
+      values.put("dealAmount", new BigDecimal("250"));
+      values.put("contactId", 1);
+      values.put("activityTypeId", 1);
+      values.put("subject", "Phone conversation");
+      values.put("description", "Discussed next steps");
+      return (values);
+   }
+
+   /*******************************************************************************
+    ** Stable persisted fixtures, including a third record excluded from selections.
+    *******************************************************************************/
+   private void seed(CrmActions.Action action) throws QException
+   {
+      put("crm_pipeline", new QRecord().withValue("id", 1).withValue("name", "Sales").withValue("isDefault", true));
+      put("crm_pipeline", new QRecord().withValue("id", 2).withValue("name", "Other"));
+      for(int id = 1; id <= 5; id++)
+      {
+         put("crm_pipeline_stage", new QRecord().withValue("id", id).withValue("name", "Stage " + id).withValue("pipelineId", id == 5 ? 2 : 1)
+            .withValue("sortOrder", id).withValue("probabilityPct", id == 3 ? 100 : 50).withValue("isClosedWon", id == 3).withValue("isClosedLost", id == 4));
+      }
+      put("crm_win_loss_reason", new QRecord().withValue("id", 1).withValue("name", "Budget"));
+      put("crm_activity_type", new QRecord().withValue("id", 1).withValue("name", "Note"));
+      put("crm_email_list", new QRecord().withValue("id", 1).withValue("name", "Newsletter").withValue("isActive", true));
+      put("crm_assignment_rule", new QRecord().withValue("id", 1).withValue("name", "Assign").withValue("entityType", 1)
+         .withValue("assignStrategy", 1).withValue("assignToUserId", "auto-owner").withValue("isActive", true).withValue("sortOrder", 1));
+      put("crm_lead_score_rule", new QRecord().withValue("id", 1).withValue("name", "Name present").withValue("fieldPath", "lastName")
+         .withValue("operator", 6).withValue("scoreAdjustment", 10).withValue("isActive", true).withValue("sortOrder", 1));
+      for(int id = 1; id <= 3; id++)
+      {
+         String email = action.operation().equals("duplicates") && id < 3 ? (id == 1 ? "A@EXAMPLE.COM" : "a@example.com") : "person" + id + "@example.com";
+         put("crm_contact", new QRecord().withValue("id", id).withValue("firstName", "Person").withValue("lastName", "Contact " + id)
+            .withValue("email", email).withValue("ownerUserId", "original").withValue("leadScore", 0));
+         put("crm_company", new QRecord().withValue("id", id).withValue("name", "Company " + id).withValue("ownerUserId", "original")
+            .withValue("domain", id < 3 ? (id == 1 ? "EXAMPLE.COM" : "example.com") : "different.com"));
+         put("crm_deal", new QRecord().withValue("id", id).withValue("name", "Deal " + id).withValue("pipelineId", 1)
+            .withValue("pipelineStageId", action.operation().equals("reopen") ? 3 : 1).withValue("amount", new BigDecimal("100"))
+            .withValue("currencyCode", "USD").withValue("ownerUserId", "original").withValue("stageEnteredDate", Instant.now().minusSeconds(86400)));
+         put("crm_form_submission", new QRecord().withValue("id", id).withValue("email", "form" + id + "@example.com")
+            .withValue("firstName", "Web").withValue("lastName", "Lead " + id).withValue("ownerUserId", "original"));
+         put("crm_email_sequence", new QRecord().withValue("id", id).withValue("name", "Sequence " + id)
+            .withValue("isActive", !action.operation().equals("activate")).withValue("ownerUserId", "original"));
+         if(id <= 2 && action.operation().equals("unsubscribe"))
+         {
+            put("crm_email_list_member", new QRecord().withValue("id", id).withValue("emailListId", 1).withValue("contactId", id)
+               .withValue("email", email).withValue("status", 1));
+         }
+         if(id <= 2 && action.operation().equals("unenroll"))
+         {
+            put("crm_sequence_enrollment", new QRecord().withValue("id", id).withValue("sequenceId", 1).withValue("contactId", id)
+               .withValue("status", 1).withValue("currentStepNumber", 0).withValue("failureCount", 0));
+         }
+      }
+   }
+
+   /*******************************************************************************
+    ** Raw fixtures intentionally bypass insert automation and uniqueness for duplicates.
+    *******************************************************************************/
+   private void put(String table, QRecord row) throws QException
+   {
+      MemoryRecordStore.getInstance().insert(new InsertInput(table).withRecord(row), true);
+   }
+
+   /*******************************************************************************
+    ** Update an existing fixture through normal DML.
+    *******************************************************************************/
+   private void change(String table, QRecord row) throws QException
+   {
+      new com.kingsrook.qqq.backend.core.actions.tables.UpdateAction().execute(
+         new com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput(table).withRecord(row));
+   }
+
+   /*******************************************************************************
+    ** Query through the normal session.
+    *******************************************************************************/
+   private List<QRecord> rows(String table) throws QException
+   {
+      return (new QueryAction().execute(new QueryInput(table)).getRecords());
+   }
+
+   /*******************************************************************************
+    ** Capture persisted values in every table to detect accidental Review writes.
+    *******************************************************************************/
+   private Map<String, String> snapshot() throws QException
+   {
+      Map<String, String> snapshot = new TreeMap<>();
+      for(String table : QContext.getQInstance().getTables().keySet())
+      {
+         List<Map<String, Serializable>> values = new ArrayList<>();
+         rows(table).forEach(row -> values.add(new TreeMap<>(row.getValues())));
+         snapshot.put(table, JsonUtils.toJson(values));
+      }
+      return (snapshot);
+   }
+}

src/test/java/com/kingsrook/qbits/crm/activities/processes/FireRemindersProcessTest.java

@@ -1,139 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** Unit tests for FireRemindersProcess.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.activities.processes;
-
-
-import java.time.Instant;
-import java.time.temporal.ChronoUnit;
-import com.kingsrook.qbits.crm.BaseTest;
-import com.kingsrook.qbits.crm.activities.model.Activity;
-import com.kingsrook.qbits.crm.activities.model.ActivityType;
-import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
-import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
-import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessOutput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertOutput;
-import org.junit.jupiter.api.Test;
-import static org.junit.jupiter.api.Assertions.assertEquals;
-import static org.junit.jupiter.api.Assertions.assertFalse;
-
-
-/*******************************************************************************
- ** Tests for firing reminders: create activity with reminder, run process,
- ** verify the reminder flag is cleared.
- *******************************************************************************/
-class FireRemindersProcessTest extends BaseTest
-{
-
-   /*******************************************************************************
-    ** Create an activity with a due reminder and verify it gets cleared.
-    *******************************************************************************/
-   @Test
-   void testFireReminders() throws QException
-   {
-      Integer typeId = insertActivityType("Meeting");
-
-      ///////////////////////////////////////
-      // create activity with past reminder //
-      ///////////////////////////////////////
-      InsertOutput activityInsert = new InsertAction().execute(
-         new InsertInput(Activity.TABLE_NAME).withRecordEntity(
-            new Activity()
-               .withActivityTypeId(typeId)
-               .withSubject("Team Meeting")
-               .withIsReminderSet(true)
-               .withReminderDate(Instant.now().minus(1, ChronoUnit.HOURS))
-               .withIsCompleted(false)
-               .withOwnerUserId("user-001")));
-      Integer activityId = activityInsert.getRecords().get(0).getValueInteger("id");
-
-      ////////////////////////////////////
-      // run the process                //
-      ////////////////////////////////////
-      RunProcessInput processInput = new RunProcessInput();
-      processInput.setProcessName(FireRemindersProcess.NAME);
-
-      RunProcessOutput processOutput = new RunProcessAction().execute(processInput);
-
-      assertEquals(1, processOutput.getValues().get("firedCount"));
-
-      /////////////////////////////////////////////
-      // verify the reminder flag was cleared    //
-      /////////////////////////////////////////////
-      Activity updated = new Activity(GetAction.execute(Activity.TABLE_NAME, activityId));
-      assertFalse(updated.getIsReminderSet());
-   }
-
-
-
-   /*******************************************************************************
-    ** Future reminders should not be fired.
-    *******************************************************************************/
-   @Test
-   void testFutureReminderNotFired() throws QException
-   {
-      Integer typeId = insertActivityType("Call");
-
-      new InsertAction().execute(
-         new InsertInput(Activity.TABLE_NAME).withRecordEntity(
-            new Activity()
-               .withActivityTypeId(typeId)
-               .withSubject("Future Call")
-               .withIsReminderSet(true)
-               .withReminderDate(Instant.now().plus(1, ChronoUnit.DAYS))
-               .withIsCompleted(false)
-               .withOwnerUserId("user-001")));
-
-      RunProcessInput processInput = new RunProcessInput();
-      processInput.setProcessName(FireRemindersProcess.NAME);
-
-      RunProcessOutput processOutput = new RunProcessAction().execute(processInput);
-
-      assertEquals(0, processOutput.getValues().get("firedCount"));
-   }
-
-
-
-   /***************************************************************************
-    ** Helper: insert an ActivityType and return its id.
-    ***************************************************************************/
-   private Integer insertActivityType(String name) throws QException
-   {
-      InsertOutput output = new InsertAction().execute(
-         new InsertInput(ActivityType.TABLE_NAME)
-            .withRecordEntity(new ActivityType()
-               .withName(name)
-               .withIconName("alarm")
-               .withIsSystem(false)
-               .withSortOrder(1)
-               .withIsActive(true)));
-      return (output.getRecords().get(0).getValueInteger("id"));
-   }
-
-}

src/test/java/com/kingsrook/qbits/crm/activities/processes/LogActivityProcessTest.java

@@ -29,12 +29,11 @@ package com.kingsrook.qbits.crm.activities.processes;
 import com.kingsrook.qbits.crm.BaseTest;
 import com.kingsrook.qbits.crm.activities.model.Activity;
 import com.kingsrook.qbits.crm.activities.model.ActivityType;
-import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
 import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
 import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessOutput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertOutput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
@@ -65,7 +64,7 @@ class LogActivityProcessTest extends BaseTest
       ////////////////////////////////////
       // run the LogActivity process    //
       ////////////////////////////////////
-      RunProcessInput processInput = new RunProcessInput();
+      RunBackendStepInput processInput = new RunBackendStepInput();
       processInput.setProcessName(LogActivityProcess.NAME);
       processInput.addValue("activityTypeId", activityTypeId);
       processInput.addValue("subject", "Test call with client");
@@ -73,7 +72,7 @@ class LogActivityProcessTest extends BaseTest
       processInput.addValue("contactId", 42);
       processInput.addValue("durationMinutes", 15);
 
-      RunProcessOutput processOutput = new RunProcessAction().execute(processInput);
+      RunBackendStepOutput processOutput = runBusinessStep(processInput);
 
       ////////////////////////////////////////////
       // verify the activity record was created //
@@ -102,12 +101,12 @@ class LogActivityProcessTest extends BaseTest
    {
       Integer activityTypeId = insertActivityType("Note");
 
-      RunProcessInput processInput = new RunProcessInput();
+      RunBackendStepInput processInput = new RunBackendStepInput();
       processInput.setProcessName(LogActivityProcess.NAME);
       processInput.addValue("activityTypeId", activityTypeId);
       processInput.addValue("subject", "Quick note");
 
-      new RunProcessAction().execute(processInput);
+      runBusinessStep(processInput);
 
       QueryOutput queryOutput = new QueryAction().execute(
          new QueryInput(Activity.TABLE_NAME));

src/test/java/com/kingsrook/qbits/crm/activities/processes/ProcessOverdueTasksProcessTest.java

@@ -1,144 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** Unit tests for ProcessOverdueTasksProcess.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.activities.processes;
-
-
-import java.time.Instant;
-import java.time.temporal.ChronoUnit;
-import com.kingsrook.qbits.crm.BaseTest;
-import com.kingsrook.qbits.crm.activities.model.Activity;
-import com.kingsrook.qbits.crm.activities.model.ActivityType;
-import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
-import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
-import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessOutput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertOutput;
-import org.junit.jupiter.api.Test;
-import static org.junit.jupiter.api.Assertions.assertEquals;
-import static org.junit.jupiter.api.Assertions.assertNotNull;
-import static org.junit.jupiter.api.Assertions.assertTrue;
-
-
-/*******************************************************************************
- ** Tests for processing overdue tasks.
- *******************************************************************************/
-class ProcessOverdueTasksProcessTest extends BaseTest
-{
-
-   /*******************************************************************************
-    ** Create an overdue task, run process, verify it was marked.
-    *******************************************************************************/
-   @Test
-   void testProcessOverdueTasks() throws QException
-   {
-      ///////////////////////////////////////
-      // set up prerequisites              //
-      ///////////////////////////////////////
-      Integer taskTypeId = insertActivityType("Task");
-
-      ///////////////////////////////////////
-      // create an overdue incomplete task //
-      ///////////////////////////////////////
-      InsertOutput activityInsert = new InsertAction().execute(
-         new InsertInput(Activity.TABLE_NAME).withRecordEntity(
-            new Activity()
-               .withActivityTypeId(taskTypeId)
-               .withSubject("Overdue task")
-               .withDueDate(Instant.now().minus(2, ChronoUnit.DAYS))
-               .withIsCompleted(false)
-               .withOwnerUserId("user-001")));
-      Integer activityId = activityInsert.getRecords().get(0).getValueInteger("id");
-
-      ////////////////////////////////////
-      // run the process                //
-      ////////////////////////////////////
-      RunProcessInput processInput = new RunProcessInput();
-      processInput.setProcessName(ProcessOverdueTasksProcess.NAME);
-
-      RunProcessOutput processOutput = new RunProcessAction().execute(processInput);
-
-      assertEquals(1, processOutput.getValues().get("processedCount"));
-
-      /////////////////////////////////////////////
-      // verify the task description was updated //
-      /////////////////////////////////////////////
-      Activity updated = new Activity(GetAction.execute(Activity.TABLE_NAME, activityId));
-      assertNotNull(updated.getDescription());
-      assertTrue(updated.getDescription().contains("[OVERDUE]"));
-   }
-
-
-
-   /*******************************************************************************
-    ** Non-overdue tasks should not be processed.
-    *******************************************************************************/
-   @Test
-   void testNoOverdueTasks() throws QException
-   {
-      Integer taskTypeId = insertActivityType("Task");
-
-      ///////////////////////////////////////
-      // create a future task              //
-      ///////////////////////////////////////
-      new InsertAction().execute(
-         new InsertInput(Activity.TABLE_NAME).withRecordEntity(
-            new Activity()
-               .withActivityTypeId(taskTypeId)
-               .withSubject("Future task")
-               .withDueDate(Instant.now().plus(5, ChronoUnit.DAYS))
-               .withIsCompleted(false)
-               .withOwnerUserId("user-001")));
-
-      RunProcessInput processInput = new RunProcessInput();
-      processInput.setProcessName(ProcessOverdueTasksProcess.NAME);
-
-      RunProcessOutput processOutput = new RunProcessAction().execute(processInput);
-
-      assertEquals(0, processOutput.getValues().get("processedCount"));
-   }
-
-
-
-   /***************************************************************************
-    ** Helper: insert an ActivityType and return its id.
-    ***************************************************************************/
-   private Integer insertActivityType(String name) throws QException
-   {
-      InsertOutput output = new InsertAction().execute(
-         new InsertInput(ActivityType.TABLE_NAME)
-            .withRecordEntity(new ActivityType()
-               .withName(name)
-               .withIconName("task")
-               .withIsSystem(false)
-               .withSortOrder(1)
-               .withIsActive(true)));
-      return (output.getRecords().get(0).getValueInteger("id"));
-   }
-
-}

src/test/java/com/kingsrook/qbits/crm/core/model/ContactChildTableTest.java

@@ -76,14 +76,14 @@ class ContactChildTableTest extends BaseTest
 
 
    /*******************************************************************************
-    ** Verify the join from Contact to DealContact exists in the QInstance.
+    ** Verify the primary-contact join from Contact to Deal exists in the QInstance.
     *******************************************************************************/
    @Test
-   void testContactDealContactJoinExists()
+   void testContactPrimaryDealJoinExists()
    {
-      String joinName = QJoinMetaData.makeInferredJoinName(Contact.TABLE_NAME, DealContact.TABLE_NAME).replaceAll("([a-z0-9])([A-Z])", "$1_$2").toLowerCase(java.util.Locale.ROOT);
+      String joinName = QJoinMetaData.makeInferredJoinName(Contact.TABLE_NAME, Deal.TABLE_NAME).replaceAll("([a-z0-9])([A-Z])", "$1_$2").toLowerCase(java.util.Locale.ROOT);
       QJoinMetaData join = QContext.getQInstance().getJoin(joinName);
-      assertNotNull(join, "Expected join from Contact to DealContact to exist");
+      assertNotNull(join, "Expected join from Contact to Deal to exist");
    }
 
 

src/test/java/com/kingsrook/qbits/crm/core/processes/BulkAssignOwnerProcessTest.java

@@ -1,97 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** Unit tests for BulkAssignOwnerProcess.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.core.processes;
-
-
-import com.kingsrook.qbits.crm.BaseTest;
-import com.kingsrook.qbits.crm.core.model.Contact;
-import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
-import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
-import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import org.junit.jupiter.api.Test;
-import static org.junit.jupiter.api.Assertions.assertEquals;
-
-
-/*******************************************************************************
- ** Tests that BulkAssignOwnerProcess sets ownerUserId on multiple records.
- *******************************************************************************/
-class BulkAssignOwnerProcessTest extends BaseTest
-{
-
-   /*******************************************************************************
-    ** Assign owner on multiple contacts via the transform step.
-    *******************************************************************************/
-   @Test
-   void testBulkAssignOwner() throws QException
-   {
-      //////////////////////////////////
-      // insert two contacts           //
-      //////////////////////////////////
-      new InsertAction().execute(
-         new InsertInput(Contact.TABLE_NAME).withRecordEntity(
-            new Contact()
-               .withFirstName("Alice").withLastName("A").withOwnerUserId("old-owner").withEmail("alice@test.com")));
-      new InsertAction().execute(
-         new InsertInput(Contact.TABLE_NAME).withRecordEntity(
-            new Contact()
-               .withFirstName("Bob").withLastName("B").withOwnerUserId("old-owner").withEmail("bob@test.com")));
-
-      //////////////////////////////////////////////////////////
-      // query back the records                                //
-      //////////////////////////////////////////////////////////
-      QueryOutput queryBefore = new QueryAction().execute(
-         new QueryInput(Contact.TABLE_NAME));
-      assertEquals(2, queryBefore.getRecords().size());
-
-      //////////////////////////////////////////////////////////
-      // run the transform step directly                      //
-      //////////////////////////////////////////////////////////
-      BulkAssignOwnerProcess.BulkAssignOwnerTransformStep step = new BulkAssignOwnerProcess.BulkAssignOwnerTransformStep();
-
-      RunBackendStepInput stepInput = new RunBackendStepInput();
-      stepInput.addValue("newOwnerUserId", "new-owner");
-      stepInput.setRecords(queryBefore.getRecords());
-
-      RunBackendStepOutput stepOutput = new RunBackendStepOutput();
-      step.runOnePage(stepInput, stepOutput);
-
-      //////////////////////////////////////////////////////////
-      // verify all records had ownerUserId updated            //
-      //////////////////////////////////////////////////////////
-      assertEquals(2, stepOutput.getRecords().size());
-      for(QRecord record : stepOutput.getRecords())
-      {
-         assertEquals("new-owner", record.getValueString("ownerUserId"));
-      }
-   }
-
-}

src/test/java/com/kingsrook/qbits/crm/core/processes/ConvertFormToContactProcessTest.java

@@ -33,12 +33,11 @@ import com.kingsrook.qbits.crm.core.model.Contact;
 import com.kingsrook.qbits.crm.core.model.FormSubmission;
 import com.kingsrook.qbits.crm.core.model.LeadSource;
 import com.kingsrook.qbits.crm.core.model.enums.CrmFormConversionStatus;
-import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
 import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
 import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessOutput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertOutput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
@@ -78,11 +77,11 @@ class ConvertFormToContactProcessTest extends BaseTest
       ///////////////////////////////////////////////
       // run the conversion process               //
       ///////////////////////////////////////////////
-      RunProcessInput processInput = new RunProcessInput();
+      RunBackendStepInput processInput = new RunBackendStepInput();
       processInput.setProcessName(ConvertFormToContactProcess.NAME);
       processInput.addValue("formSubmissionId", formId);
 
-      RunProcessOutput processOutput = new RunProcessAction().execute(processInput);
+      RunBackendStepOutput processOutput = runBusinessStep(processInput);
 
       ///////////////////////////////////////////////
       // verify a new contact was created          //
@@ -154,11 +153,11 @@ class ConvertFormToContactProcessTest extends BaseTest
       /////////////////////////////////////////////
       // run conversion                          //
       /////////////////////////////////////////////
-      RunProcessInput processInput = new RunProcessInput();
+      RunBackendStepInput processInput = new RunBackendStepInput();
       processInput.setProcessName(ConvertFormToContactProcess.NAME);
       processInput.addValue("formSubmissionId", formId);
 
-      new RunProcessAction().execute(processInput);
+      runBusinessStep(processInput);
 
       /////////////////////////////////////////////
       // verify no new contact was created       //

src/test/java/com/kingsrook/qbits/crm/core/processes/DetectDuplicatesProcessTest.java

@@ -29,11 +29,10 @@ package com.kingsrook.qbits.crm.core.processes;
 import com.kingsrook.qbits.crm.BaseTest;
 import com.kingsrook.qbits.crm.core.model.Company;
 import com.kingsrook.qbits.crm.core.model.Contact;
-import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
 import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessOutput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
 import org.junit.jupiter.api.Test;
 import static org.junit.jupiter.api.Assertions.assertEquals;
@@ -62,11 +61,11 @@ class DetectDuplicatesProcessTest extends BaseTest
       ////////////////////////////////////
       // run detect duplicates          //
       ////////////////////////////////////
-      RunProcessInput processInput = new RunProcessInput();
+      RunBackendStepInput processInput = new RunBackendStepInput();
       processInput.setProcessName(DetectDuplicatesProcess.NAME);
       processInput.addValue("entityType", "CONTACT");
 
-      RunProcessOutput processOutput = new RunProcessAction().execute(processInput);
+      RunBackendStepOutput processOutput = runBusinessStep(processInput);
 
       assertEquals(1, processOutput.getValues().get("duplicateGroupCount"));
       assertEquals(1, processOutput.getRecords().size());
@@ -85,11 +84,11 @@ class DetectDuplicatesProcessTest extends BaseTest
       insertCompany("Corp B", "dupe.com");
       insertCompany("Corp C", "unique.com");
 
-      RunProcessInput processInput = new RunProcessInput();
+      RunBackendStepInput processInput = new RunBackendStepInput();
       processInput.setProcessName(DetectDuplicatesProcess.NAME);
       processInput.addValue("entityType", "COMPANY");
 
-      RunProcessOutput processOutput = new RunProcessAction().execute(processInput);
+      RunBackendStepOutput processOutput = runBusinessStep(processInput);
 
       assertEquals(1, processOutput.getValues().get("duplicateGroupCount"));
    }
@@ -105,11 +104,11 @@ class DetectDuplicatesProcessTest extends BaseTest
       insertContact("one@example.com", "One", "User");
       insertContact("two@example.com", "Two", "User");
 
-      RunProcessInput processInput = new RunProcessInput();
+      RunBackendStepInput processInput = new RunBackendStepInput();
       processInput.setProcessName(DetectDuplicatesProcess.NAME);
       processInput.addValue("entityType", "CONTACT");
 
-      RunProcessOutput processOutput = new RunProcessAction().execute(processInput);
+      RunBackendStepOutput processOutput = runBusinessStep(processInput);
 
       assertEquals(0, processOutput.getValues().get("duplicateGroupCount"));
       assertTrue(processOutput.getRecords().isEmpty());

src/test/java/com/kingsrook/qbits/crm/core/processes/GdprAnonymizeContactProcessTest.java

@@ -31,13 +31,12 @@ import com.kingsrook.qbits.crm.core.model.ConsentRecord;
 import com.kingsrook.qbits.crm.core.model.Contact;
 import com.kingsrook.qbits.crm.core.model.enums.CrmConsentStatus;
 import com.kingsrook.qbits.crm.core.model.enums.CrmConsentType;
-import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
 import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
 import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
 import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessOutput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertOutput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
@@ -69,11 +68,11 @@ class GdprAnonymizeContactProcessTest extends BaseTest
       ////////////////////////////////////
       // run the anonymize process      //
       ////////////////////////////////////
-      RunProcessInput processInput = new RunProcessInput();
+      RunBackendStepInput processInput = new RunBackendStepInput();
       processInput.setProcessName(GdprAnonymizeContactProcess.NAME);
       processInput.addValue("contactId", contactId);
 
-      RunProcessOutput processOutput = new RunProcessAction().execute(processInput);
+      RunBackendStepOutput processOutput = runBusinessStep(processInput);
 
       assertTrue((Boolean) processOutput.getValues().get("anonymized"));
 

src/test/java/com/kingsrook/qbits/crm/core/processes/ImportContactsProcessTest.java

@@ -1,154 +0,0 @@
-/*
- * QQQ - Low-code Application Framework for Engineers.
- * Copyright (C) 2021-2026.  Kingsrook, LLC
- * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
- * contact@kingsrook.com
- * https://github.com/Kingsrook/
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     https://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-
-/*******************************************************************************
- ** Unit tests for ImportContactsProcess.
- *******************************************************************************/
-
-package com.kingsrook.qbits.crm.core.processes;
-
-
-import java.util.ArrayList;
-import java.util.List;
-import com.kingsrook.qbits.crm.BaseTest;
-import com.kingsrook.qbits.crm.core.model.Contact;
-import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
-import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
-import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
-import com.kingsrook.qqq.backend.core.exceptions.QException;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
-import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessOutput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
-import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryOutput;
-import com.kingsrook.qqq.backend.core.model.data.QRecord;
-import org.junit.jupiter.api.Test;
-import static org.junit.jupiter.api.Assertions.assertEquals;
-
-
-/*******************************************************************************
- ** Tests for importing contacts: new inserts, duplicate skipping, validation.
- *******************************************************************************/
-class ImportContactsProcessTest extends BaseTest
-{
-
-   /*******************************************************************************
-    ** Import new contacts and verify they are created with audit logs.
-    *******************************************************************************/
-   @Test
-   void testImportNewContacts() throws QException
-   {
-      List<QRecord> records = new ArrayList<>();
-      records.add(new QRecord()
-         .withValue("firstName", "Alice")
-         .withValue("lastName", "Smith")
-         .withValue("email", "alice@example.com"));
-      records.add(new QRecord()
-         .withValue("firstName", "Bob")
-         .withValue("lastName", "Jones")
-         .withValue("email", "bob@example.com"));
-
-      RunProcessInput processInput = new RunProcessInput();
-      processInput.setProcessName(ImportContactsProcess.NAME);
-      processInput.setRecords(records);
-
-      RunProcessOutput processOutput = new RunProcessAction().execute(processInput);
-
-      assertEquals(2, processOutput.getValues().get("importedCount"));
-      assertEquals(0, processOutput.getValues().get("skippedCount"));
-      assertEquals(0, processOutput.getValues().get("errorCount"));
-
-      QueryOutput contactQuery = new QueryAction().execute(
-         new QueryInput(Contact.TABLE_NAME));
-      assertEquals(2, contactQuery.getRecords().size());
-
-   }
-
-
-
-   /*******************************************************************************
-    ** Test duplicate skipping: pre-existing contact with same email is skipped.
-    *******************************************************************************/
-   @Test
-   void testImportWithDuplicates() throws QException
-   {
-      /////////////////////////////////////////////
-      // insert existing contact                 //
-      /////////////////////////////////////////////
-      new InsertAction().execute(
-         new InsertInput(Contact.TABLE_NAME).withRecordEntity(
-            new Contact()
-               .withFirstName("Existing")
-               .withLastName("User")
-               .withEmail("alice@example.com")
-               .withOwnerUserId("user-001")));
-
-      List<QRecord> records = new ArrayList<>();
-      records.add(new QRecord()
-         .withValue("firstName", "Alice")
-         .withValue("lastName", "Smith")
-         .withValue("email", "alice@example.com")); // duplicate
-      records.add(new QRecord()
-         .withValue("firstName", "Charlie")
-         .withValue("lastName", "Brown")
-         .withValue("email", "charlie@example.com")); // new
-
-      RunProcessInput processInput = new RunProcessInput();
-      processInput.setProcessName(ImportContactsProcess.NAME);
-      processInput.setRecords(records);
-
-      RunProcessOutput processOutput = new RunProcessAction().execute(processInput);
-
-      assertEquals(1, processOutput.getValues().get("importedCount"));
-      assertEquals(1, processOutput.getValues().get("skippedCount"));
-      assertEquals(0, processOutput.getValues().get("errorCount"));
-   }
-
-
-
-   /*******************************************************************************
-    ** Test validation: records missing required fields are counted as errors.
-    *******************************************************************************/
-   @Test
-   void testImportWithValidationErrors() throws QException
-   {
-      List<QRecord> records = new ArrayList<>();
-      records.add(new QRecord()
-         .withValue("firstName", "")
-         .withValue("lastName", "NoFirst")
-         .withValue("email", "nofirst@example.com")); // missing firstName
-      records.add(new QRecord()
-         .withValue("firstName", "Valid")
-         .withValue("lastName", "User")
-         .withValue("email", "valid@example.com"));
-
-      RunProcessInput processInput = new RunProcessInput();
-      processInput.setProcessName(ImportContactsProcess.NAME);
-      processInput.setRecords(records);
-
-      RunProcessOutput processOutput = new RunProcessAction().execute(processInput);
-
-      assertEquals(1, processOutput.getValues().get("importedCount"));
-      assertEquals(0, processOutput.getValues().get("skippedCount"));
-      assertEquals(1, processOutput.getValues().get("errorCount"));
-   }
-
-}