From d732cef626595fb6a0062481d6041187c0f543ab Mon Sep 17 00:00:00 2001 From: James Maes Date: Wed, 14 Oct 2026 11:40:48 -0500 Subject: [PATCH 1/2] feat(crm): resolve qbit-crm's snake_case table names, legacy fallback qbit-crm is renaming its tables to crm_* snake_case. QbitCrmGateway now resolves each table from the instance at call time, preferring the snake_case name and falling back to the legacy camelCase one, so either qbit-crm generation works. Tests run the gateway against both schemes. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 1 + README.md | 2 +- docs/DESIGN.md | 2 +- .../qbits/marketing/crm/QbitCrmGateway.java | 125 +++++++-- .../qbits/marketing/QbitCrmGatewayTest.java | 237 ++++++++++++++++++ 5 files changed, 350 insertions(+), 17 deletions(-) create mode 100644 src/test/java/com/kingsrook/qbits/marketing/QbitCrmGatewayTest.java diff --git a/CHANGELOG.md b/CHANGELOG.md index a7511c6..49db3f3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,7 @@ All notable changes to this project are documented here. The format follows Keep ## [0.1.0] - Unreleased ### Changed +- `QbitCrmGateway` resolves qbit-crm table names at call time, preferring the new snake_case names (`crm_contact`, ...) and falling back to the legacy camelCase ones; the legacy names stay available as `LEGACY_*` constants. - Switch the optional qbit-email integration to the published `com.kingsrook.qbits.email.api` package in `0.1.0-RC.3` (fixes #13). Preserve MARKETING messages, idempotency keys, execution links and outcome mapping; use typed readiness failures so missing providers or brand postal addresses remain send-time rejections. Upgrade `qbit-build-parent` to `2.1.1` and remove all per-repository guardrail allow-lists. - Resolve architecture finding F5 (#9): depend on `qbit-assistant-provider-api` and `qbit-assistant-provider-openai` `0.1.0-RC.11`, with no full assistant dependency, including in tests. - Replace the local OpenAI-compatible implementation with the published provider and document `ProviderSettings`, environment-key resolution and provider-name migration. Agent tests use a local scripted double. diff --git a/README.md b/README.md index 527d773..3ed2a68 100644 --- a/README.md +++ b/README.md @@ -23,7 +23,7 @@ It lives inside the host QQQ application: its navigation, authentication, permis | **Execution** | Durable jobs with idempotency keys, schedules, pause, resume and cancel, and attempts with provider receipts. Failures that certainly did not send are retried with backoff. Ambiguous outcomes are reconciled and never retried blindly. Everything is re-checked at send time (approval, schedule, consent, suppression, channel rules, rate, kill switch). | | **Transports** | A complete controlled test transport, and email through the host's QQQ messaging provider. A channel that names a transport is not treated as a working connector unless that transport is registered and usable. | | **Measurement** | Delivery, awareness, engagement, activation, repeat use and qualified demand are reported separately. Each metric shows raw events and unique actors. "Not measured" is never shown as 0. Test data is kept apart. | -| **CRM** | Uses the host CRM's own contacts, organizations, consent, suppression, lists, form submissions and activities through a narrow gateway. A qbit-crm adapter is included. Nothing is copied, and nobody becomes a lead without the campaign's explicit rule. | +| **CRM** | Uses the host CRM's own contacts, organizations, consent, suppression, lists, form submissions and activities through a narrow gateway. A qbit-crm adapter is included; it works with qbit-crm's snake_case table names and falls back to the legacy camelCase ones. Nothing is copied, and nobody becomes a lead without the campaign's explicit rule. | | **Agent** | Plans campaigns, drafts and revises content, flags unsupported claims, stale sources and readiness gaps, and analyzes results to recommend the next experiment. It uses narrow typed tools, runs as the requesting user, sees data only as untrusted input, and sees no CRM personal data. Every run is recorded with its steps, provider, model, tokens and cost. It can create drafts and findings; it cannot approve, schedule or send. | | **Models** | Interchangeable providers through `qbit-assistant-provider-api`'s `AssistantModelProvider`: OpenAI-compatible endpoints through `qbit-assistant-provider-openai` (included), or a host-supplied provider such as Claude. Tests use a local scripted double. With no provider configured, the agent says so and everything else works. Installing marketing does not install the assistant runtime, tables or routes. | diff --git a/docs/DESIGN.md b/docs/DESIGN.md index 8165625..ca20e05 100644 --- a/docs/DESIGN.md +++ b/docs/DESIGN.md @@ -122,7 +122,7 @@ Most rows carry a denormalized `productId`, so the optional product security loc - `listFormSubmissions(utmCampaign)`: ids and times only - `recordActivity` -`QbitCrmGateway` implements it over qbit-crm 0.1.0 by table name, through the standard actions, so there is no compile dependency on the CRM: +`QbitCrmGateway` implements it over qbit-crm 0.1.0 by table name, through the standard actions, so there is no compile dependency on the CRM. Table names are resolved from the instance at call time, preferring qbit-crm's snake_case names (`crm_contact`, ...) and falling back to the legacy camelCase ones (`crmContact`, ...): - **Email consent:** the most recent Email consent record decides (Granted and not expired). No record means no consent. - **Suppression:** `doNotEmail`, an unsubscribe date, or a list membership that isn't Subscribed. diff --git a/src/main/java/com/kingsrook/qbits/marketing/crm/QbitCrmGateway.java b/src/main/java/com/kingsrook/qbits/marketing/crm/QbitCrmGateway.java index 01d696e..b72c114 100644 --- a/src/main/java/com/kingsrook/qbits/marketing/crm/QbitCrmGateway.java +++ b/src/main/java/com/kingsrook/qbits/marketing/crm/QbitCrmGateway.java @@ -40,21 +40,36 @@ import com.kingsrook.qqq.backend.core.utils.ValueUtils; ** compile dependency on the CRM and CRM permissions and locks apply. ** ** Rules it applies (from qbit-crm's own model): - ** - Email consent: the contact's most recent crmConsentRecord of type Email + ** - Email consent: the contact's most recent consent record (crm_consent_record, or legacy crmConsentRecord) of type Email ** decides (Granted, and not past its expiry date). No record means no ** consent: Marketing does not send unsolicited email. ** - Suppression: doNotEmail, an unsubscribeDate, or a list membership that ** is not Subscribed. ** - Activities are logged as qbit-crm "Note" activities. + ** + ** Table names are resolved from the QInstance at call time: the snake_case + ** name qbit-crm now uses is preferred, falling back to its legacy camelCase + ** name, so either qbit-crm generation works. *******************************************************************************/ public class QbitCrmGateway implements MarketingCrmGateway { - public static final String CONTACT_TABLE = "crmContact"; - public static final String CONSENT_TABLE = "crmConsentRecord"; - public static final String LIST_MEMBER_TABLE = "crmEmailListMember"; - public static final String FORM_SUBMISSION_TABLE = "crmFormSubmission"; - public static final String ACTIVITY_TABLE = "crmActivity"; - public static final String ACTIVITY_TYPE_TABLE = "crmActivityType"; + public static final String CONTACT_TABLE = "crm_contact"; + public static final String CONSENT_TABLE = "crm_consent_record"; + public static final String LIST_MEMBER_TABLE = "crm_email_list_member"; + public static final String FORM_SUBMISSION_TABLE = "crm_form_submission"; + public static final String ACTIVITY_TABLE = "crm_activity"; + public static final String ACTIVITY_TYPE_TABLE = "crm_activity_type"; + + /////////////////////////////////////////////////////////////////////// + // the names qbit-crm used before it moved to snake_case; still used // + // when the instance has the old tables // + /////////////////////////////////////////////////////////////////////// + public static final String LEGACY_CONTACT_TABLE = "crmContact"; + public static final String LEGACY_CONSENT_TABLE = "crmConsentRecord"; + public static final String LEGACY_LIST_MEMBER_TABLE = "crmEmailListMember"; + public static final String LEGACY_FORM_SUBMISSION_TABLE = "crmFormSubmission"; + public static final String LEGACY_ACTIVITY_TABLE = "crmActivity"; + public static final String LEGACY_ACTIVITY_TYPE_TABLE = "crmActivityType"; static final Integer CONSENT_TYPE_EMAIL = 1; static final Integer CONSENT_STATUS_GRANTED = 1; @@ -81,7 +96,7 @@ public class QbitCrmGateway implements MarketingCrmGateway public boolean isAvailable() { QInstance qInstance = QContext.getQInstance(); - return (qInstance != null && qInstance.getTable(CONTACT_TABLE) != null && qInstance.getTable(CONSENT_TABLE) != null && qInstance.getTable(FORM_SUBMISSION_TABLE) != null); + return (qInstance != null && contactTable() != null && consentTable() != null && formSubmissionTable() != null); } @@ -106,7 +121,7 @@ public class QbitCrmGateway implements MarketingCrmGateway else if(recipientScope.startsWith("crm-list:")) { Integer listId = parseId(recipientScope, "crm-list:"); - for(QRecord member : MarketingStore.queryBy(LIST_MEMBER_TABLE, "emailListId", listId)) + for(QRecord member : MarketingStore.queryBy(listMemberTable(), "emailListId", listId)) { Integer contactId = member.getValueInteger("contactId"); if(contactId == null) @@ -135,7 +150,7 @@ public class QbitCrmGateway implements MarketingCrmGateway String address = null; if(eligibility.eligible()) { - QRecord contact = MarketingStore.get(CONTACT_TABLE, contactId); + QRecord contact = MarketingStore.get(contactTable(), contactId); address = contact == null ? null : contact.getValueString("email"); } return (new CrmRecipient(contactId, address, eligibility)); @@ -149,7 +164,7 @@ public class QbitCrmGateway implements MarketingCrmGateway @Override public Eligibility checkEmailEligibility(Integer contactId) throws QException { - QRecord contact = MarketingStore.get(CONTACT_TABLE, contactId); + QRecord contact = MarketingStore.get(contactTable(), contactId); if(contact == null) { return (Eligibility.no("contact_not_found")); @@ -167,7 +182,7 @@ public class QbitCrmGateway implements MarketingCrmGateway return (Eligibility.no("no_email_address")); } - List consents = MarketingStore.query(CONSENT_TABLE, new QQueryFilter( + List consents = MarketingStore.query(consentTable(), new QQueryFilter( new QFilterCriteria("contactId", QCriteriaOperator.EQUALS, contactId), new QFilterCriteria("consentType", QCriteriaOperator.EQUALS, CONSENT_TYPE_EMAIL))); if(consents.isEmpty()) @@ -210,7 +225,7 @@ public class QbitCrmGateway implements MarketingCrmGateway { filter.addCriteria(new QFilterCriteria("createDate", QCriteriaOperator.GREATER_THAN_OR_EQUALS, since)); } - for(QRecord submission : MarketingStore.query(FORM_SUBMISSION_TABLE, filter)) + for(QRecord submission : MarketingStore.query(formSubmissionTable(), filter)) { list.add(new CrmFormSubmissionRef(submission.getValueInteger("id"), submission.getValueInteger("contactId"), ValueUtils.getValueAsInstant(submission.getValue("createDate")))); } @@ -225,7 +240,7 @@ public class QbitCrmGateway implements MarketingCrmGateway @Override public void recordActivity(Integer contactId, String subject, String description) throws QException { - List types = MarketingStore.queryBy(ACTIVITY_TYPE_TABLE, "name", NOTE_ACTIVITY_TYPE_NAME); + List types = MarketingStore.queryBy(activityTypeTable(), "name", NOTE_ACTIVITY_TYPE_NAME); if(types.isEmpty()) { throw (new QUserFacingException("The CRM has no \"" + NOTE_ACTIVITY_TYPE_NAME + "\" activity type to log marketing activities with.")); @@ -234,7 +249,7 @@ public class QbitCrmGateway implements MarketingCrmGateway ////////////////////////////////////////////////////////////////// // written with the user's CRM permissions, not Marketing's scope // ////////////////////////////////////////////////////////////////// - com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput insert = new com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput(ACTIVITY_TABLE); + com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput insert = new com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput(activityTable()); insert.withPermissionEnforcement(com.kingsrook.qqq.backend.core.actions.permissions.PermissionEnforcement.ENFORCE); insert.withRecord(new QRecord() .withValue("activityTypeId", types.get(0).getValueInteger("id")) @@ -253,6 +268,86 @@ public class QbitCrmGateway implements MarketingCrmGateway + /******************************************************************************* + ** The contact table's name in the current instance (null if neither naming exists). + *******************************************************************************/ + public static String contactTable() + { + return (resolve(CONTACT_TABLE, LEGACY_CONTACT_TABLE)); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + public static String consentTable() + { + return (resolve(CONSENT_TABLE, LEGACY_CONSENT_TABLE)); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + public static String listMemberTable() + { + return (resolve(LIST_MEMBER_TABLE, LEGACY_LIST_MEMBER_TABLE)); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + public static String formSubmissionTable() + { + return (resolve(FORM_SUBMISSION_TABLE, LEGACY_FORM_SUBMISSION_TABLE)); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + public static String activityTable() + { + return (resolve(ACTIVITY_TABLE, LEGACY_ACTIVITY_TABLE)); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + public static String activityTypeTable() + { + return (resolve(ACTIVITY_TYPE_TABLE, LEGACY_ACTIVITY_TYPE_TABLE)); + } + + + + /******************************************************************************* + ** Prefer the snake_case table, fall back to the legacy one; null if the + ** instance has neither. + *******************************************************************************/ + private static String resolve(String name, String legacyName) + { + QInstance qInstance = QContext.getQInstance(); + if(qInstance == null) + { + return (null); + } + if(qInstance.getTable(name) != null) + { + return (name); + } + return (qInstance.getTable(legacyName) != null ? legacyName : null); + } + + + /******************************************************************************* ** *******************************************************************************/ diff --git a/src/test/java/com/kingsrook/qbits/marketing/QbitCrmGatewayTest.java b/src/test/java/com/kingsrook/qbits/marketing/QbitCrmGatewayTest.java new file mode 100644 index 0000000..6691a4b --- /dev/null +++ b/src/test/java/com/kingsrook/qbits/marketing/QbitCrmGatewayTest.java @@ -0,0 +1,237 @@ +/* + * Copyright 2026 Kingsrook, LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.marketing; + + +import java.time.Instant; +import java.time.temporal.ChronoUnit; +import java.util.List; +import com.kingsrook.qbits.marketing.crm.MarketingCrmGateway.CrmFormSubmissionRef; +import com.kingsrook.qbits.marketing.crm.MarketingCrmGateway.CrmRecipient; +import com.kingsrook.qbits.marketing.crm.QbitCrmGateway; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType; +import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData; +import org.junit.jupiter.api.Test; +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; + + +/******************************************************************************* + ** QbitCrmGateway against fixture CRM tables under both qbit-crm naming + ** schemes: snake_case (current) and camelCase (legacy). + *******************************************************************************/ +class QbitCrmGatewayTest extends BaseMarketingTest +{ + + /******************************************************************************* + ** + *******************************************************************************/ + @Test + void testNotAvailableWithoutCrmTables() + { + assertThat(new QbitCrmGateway().isAvailable()).isFalse(); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + @Test + void testSnakeCaseTables() throws QException + { + defineSnakeCaseTables(); + assertThat(QbitCrmGateway.contactTable()).isEqualTo("crm_contact"); + exerciseGateway(); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + @Test + void testLegacyCamelCaseTables() throws QException + { + defineLegacyTables(); + assertThat(QbitCrmGateway.contactTable()).isEqualTo("crmContact"); + exerciseGateway(); + } + + + + /******************************************************************************* + ** The snake_case tables win when both generations are defined. + *******************************************************************************/ + @Test + void testSnakeCasePreferredWhenBothExist() + { + defineSnakeCaseTables(); + defineLegacyTables(); + + assertThat(QbitCrmGateway.contactTable()).isEqualTo(QbitCrmGateway.CONTACT_TABLE); + assertThat(QbitCrmGateway.activityTypeTable()).isEqualTo(QbitCrmGateway.ACTIVITY_TYPE_TABLE); + } + + + + /******************************************************************************* + ** The same flows, whichever naming the instance has. + *******************************************************************************/ + private void exerciseGateway() throws QException + { + QbitCrmGateway gateway = new QbitCrmGateway(); + assertThat(gateway.isAvailable()).isTrue(); + + Integer granted = insert(QbitCrmGateway.contactTable(), new QRecord().withValue("email", "yes@example.org").withValue("doNotEmail", false)); + Integer noConsent = insert(QbitCrmGateway.contactTable(), new QRecord().withValue("email", "none@example.org").withValue("doNotEmail", false)); + Integer suppressed = insert(QbitCrmGateway.contactTable(), new QRecord().withValue("email", "dne@example.org").withValue("doNotEmail", true)); + for(Integer contactId : List.of(granted, suppressed)) + { + insert(QbitCrmGateway.consentTable(), new QRecord().withValue("contactId", contactId).withValue("consentType", 1).withValue("status", 1) + .withValue("consentDate", Instant.now().minus(2, ChronoUnit.DAYS))); + } + + ///////////////////////////// + // checkEmailEligibility // + ///////////////////////////// + assertThat(gateway.checkEmailEligibility(granted).eligible()).isTrue(); + assertThat(gateway.checkEmailEligibility(noConsent).reason()).isEqualTo("no_email_consent"); + assertThat(gateway.checkEmailEligibility(suppressed).reason()).isEqualTo("do_not_email"); + assertThat(gateway.checkEmailEligibility(9999).reason()).isEqualTo("contact_not_found"); + + /////////////////////////////////////// + // resolveRecipients: contact scope // + /////////////////////////////////////// + List single = gateway.resolveRecipients("crm-contact:" + granted); + assertThat(single).hasSize(1); + assertThat(single.get(0).address()).isEqualTo("yes@example.org"); + + ///////////////////////////////////// + // resolveRecipients: list scope // + ///////////////////////////////////// + insert(QbitCrmGateway.listMemberTable(), new QRecord().withValue("emailListId", 7).withValue("contactId", granted).withValue("status", 1)); + insert(QbitCrmGateway.listMemberTable(), new QRecord().withValue("emailListId", 7).withValue("contactId", noConsent).withValue("status", 1)); + insert(QbitCrmGateway.listMemberTable(), new QRecord().withValue("emailListId", 7).withValue("contactId", suppressed).withValue("status", 2)); + List members = gateway.resolveRecipients("crm-list:7"); + assertThat(members).hasSize(3); + assertThat(members.stream().filter(r -> r.eligibility().eligible()).map(CrmRecipient::address)).containsExactly("yes@example.org"); + assertThat(members.stream().map(r -> r.eligibility().reason())).contains("no_email_consent", "list_member_not_subscribed"); + + /////////////////////////// + // listFormSubmissions // + /////////////////////////// + Instant created = Instant.now().minus(1, ChronoUnit.HOURS); + insert(QbitCrmGateway.formSubmissionTable(), new QRecord().withValue("contactId", granted).withValue("utmCampaign", "spring").withValue("createDate", created)); + insert(QbitCrmGateway.formSubmissionTable(), new QRecord().withValue("contactId", noConsent).withValue("utmCampaign", "other").withValue("createDate", created)); + List submissions = gateway.listFormSubmissions("spring", null); + assertThat(submissions).hasSize(1); + assertThat(submissions.get(0).contactId()).isEqualTo(granted); + assertThat(gateway.listFormSubmissions("spring", Instant.now())).isEmpty(); + + ////////////////////// + // recordActivity // + ////////////////////// + assertThatThrownBy(() -> gateway.recordActivity(granted, "Subject", "Body")).hasMessageContaining("Note"); + insert(QbitCrmGateway.activityTypeTable(), new QRecord().withValue("name", "Note")); + gateway.recordActivity(granted, "Subject", "Body"); + List activities = MarketingStore.query(QbitCrmGateway.activityTable(), new QQueryFilter()); + assertThat(activities).hasSize(1); + assertThat(activities.get(0).getValueString("subject")).isEqualTo("Subject"); + assertThat(activities.get(0).getValueInteger("contactId")).isEqualTo(granted); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + private Integer insert(String table, QRecord record) throws QException + { + return (MarketingStore.insert(table, record).getValueInteger("id")); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + private void defineSnakeCaseTables() + { + defineCrmTables(QbitCrmGateway.CONTACT_TABLE, QbitCrmGateway.CONSENT_TABLE, QbitCrmGateway.LIST_MEMBER_TABLE, + QbitCrmGateway.FORM_SUBMISSION_TABLE, QbitCrmGateway.ACTIVITY_TABLE, QbitCrmGateway.ACTIVITY_TYPE_TABLE); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + private void defineLegacyTables() + { + defineCrmTables(QbitCrmGateway.LEGACY_CONTACT_TABLE, QbitCrmGateway.LEGACY_CONSENT_TABLE, QbitCrmGateway.LEGACY_LIST_MEMBER_TABLE, + QbitCrmGateway.LEGACY_FORM_SUBMISSION_TABLE, QbitCrmGateway.LEGACY_ACTIVITY_TABLE, QbitCrmGateway.LEGACY_ACTIVITY_TYPE_TABLE); + } + + + + /******************************************************************************* + ** Define fixture tables (memory backend) shaped like qbit-crm's. + *******************************************************************************/ + private void defineCrmTables(String contact, String consent, String listMember, String formSubmission, String activity, String activityType) + { + defineTable(contact, field("email", QFieldType.STRING), field("doNotEmail", QFieldType.BOOLEAN), field("unsubscribeDate", QFieldType.DATE_TIME)); + defineTable(consent, field("contactId", QFieldType.INTEGER), field("consentType", QFieldType.INTEGER), field("status", QFieldType.INTEGER), + field("consentDate", QFieldType.DATE_TIME), field("expiryDate", QFieldType.DATE_TIME)); + defineTable(listMember, field("emailListId", QFieldType.INTEGER), field("contactId", QFieldType.INTEGER), field("status", QFieldType.INTEGER)); + defineTable(formSubmission, field("contactId", QFieldType.INTEGER), field("utmCampaign", QFieldType.STRING), field("createDate", QFieldType.DATE_TIME)); + defineTable(activity, field("activityTypeId", QFieldType.INTEGER), field("subject", QFieldType.STRING), field("description", QFieldType.STRING), + field("contactId", QFieldType.INTEGER), field("ownerUserId", QFieldType.STRING), field("isCompleted", QFieldType.BOOLEAN), field("completedDate", QFieldType.DATE_TIME)); + defineTable(activityType, field("name", QFieldType.STRING)); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + private static QFieldMetaData field(String name, QFieldType type) + { + return (new QFieldMetaData(name, type)); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + private void defineTable(String name, QFieldMetaData... fields) + { + QTableMetaData table = new QTableMetaData().withName(name).withBackendName(BACKEND).withPrimaryKeyField("id") + .withField(new QFieldMetaData("id", QFieldType.INTEGER)); + for(QFieldMetaData field : fields) + { + table.withField(field); + } + qInstance.addTable(table); + } + +} -- 2.54.0 From 5dd68eb20b38707da0cf309e40fd72cd5c335e5f Mon Sep 17 00:00:00 2001 From: James Maes Date: Wed, 14 Oct 2026 11:46:24 -0500 Subject: [PATCH 2/2] refactor(crm): crm_* table names only, no legacy fallback qbit-crm 0.2.0 renames its tables to crm_* with no compatibility shims (owner decision via the coordinator), so the gateway uses the snake_case names directly and drops the camelCase fallback and LEGACY_* constants. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 2 +- README.md | 2 +- docs/DESIGN.md | 2 +- .../qbits/marketing/crm/QbitCrmGateway.java | 113 ++---------------- .../qbits/marketing/QbitCrmGatewayTest.java | 71 +++-------- 5 files changed, 30 insertions(+), 160 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 49db3f3..20ef635 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,7 +5,7 @@ All notable changes to this project are documented here. The format follows Keep ## [0.1.0] - Unreleased ### Changed -- `QbitCrmGateway` resolves qbit-crm table names at call time, preferring the new snake_case names (`crm_contact`, ...) and falling back to the legacy camelCase ones; the legacy names stay available as `LEGACY_*` constants. +- `QbitCrmGateway` uses qbit-crm 0.2.0's `crm_*` snake_case table names (`crm_contact`, ...). qbit-crm 0.1.0's camelCase tables are no longer supported. - Switch the optional qbit-email integration to the published `com.kingsrook.qbits.email.api` package in `0.1.0-RC.3` (fixes #13). Preserve MARKETING messages, idempotency keys, execution links and outcome mapping; use typed readiness failures so missing providers or brand postal addresses remain send-time rejections. Upgrade `qbit-build-parent` to `2.1.1` and remove all per-repository guardrail allow-lists. - Resolve architecture finding F5 (#9): depend on `qbit-assistant-provider-api` and `qbit-assistant-provider-openai` `0.1.0-RC.11`, with no full assistant dependency, including in tests. - Replace the local OpenAI-compatible implementation with the published provider and document `ProviderSettings`, environment-key resolution and provider-name migration. Agent tests use a local scripted double. diff --git a/README.md b/README.md index 3ed2a68..245693e 100644 --- a/README.md +++ b/README.md @@ -23,7 +23,7 @@ It lives inside the host QQQ application: its navigation, authentication, permis | **Execution** | Durable jobs with idempotency keys, schedules, pause, resume and cancel, and attempts with provider receipts. Failures that certainly did not send are retried with backoff. Ambiguous outcomes are reconciled and never retried blindly. Everything is re-checked at send time (approval, schedule, consent, suppression, channel rules, rate, kill switch). | | **Transports** | A complete controlled test transport, and email through the host's QQQ messaging provider. A channel that names a transport is not treated as a working connector unless that transport is registered and usable. | | **Measurement** | Delivery, awareness, engagement, activation, repeat use and qualified demand are reported separately. Each metric shows raw events and unique actors. "Not measured" is never shown as 0. Test data is kept apart. | -| **CRM** | Uses the host CRM's own contacts, organizations, consent, suppression, lists, form submissions and activities through a narrow gateway. A qbit-crm adapter is included; it works with qbit-crm's snake_case table names and falls back to the legacy camelCase ones. Nothing is copied, and nobody becomes a lead without the campaign's explicit rule. | +| **CRM** | Uses the host CRM's own contacts, organizations, consent, suppression, lists, form submissions and activities through a narrow gateway. A qbit-crm adapter is included (qbit-crm 0.2.0's `crm_*` tables). Nothing is copied, and nobody becomes a lead without the campaign's explicit rule. | | **Agent** | Plans campaigns, drafts and revises content, flags unsupported claims, stale sources and readiness gaps, and analyzes results to recommend the next experiment. It uses narrow typed tools, runs as the requesting user, sees data only as untrusted input, and sees no CRM personal data. Every run is recorded with its steps, provider, model, tokens and cost. It can create drafts and findings; it cannot approve, schedule or send. | | **Models** | Interchangeable providers through `qbit-assistant-provider-api`'s `AssistantModelProvider`: OpenAI-compatible endpoints through `qbit-assistant-provider-openai` (included), or a host-supplied provider such as Claude. Tests use a local scripted double. With no provider configured, the agent says so and everything else works. Installing marketing does not install the assistant runtime, tables or routes. | diff --git a/docs/DESIGN.md b/docs/DESIGN.md index ca20e05..f099e4f 100644 --- a/docs/DESIGN.md +++ b/docs/DESIGN.md @@ -122,7 +122,7 @@ Most rows carry a denormalized `productId`, so the optional product security loc - `listFormSubmissions(utmCampaign)`: ids and times only - `recordActivity` -`QbitCrmGateway` implements it over qbit-crm 0.1.0 by table name, through the standard actions, so there is no compile dependency on the CRM. Table names are resolved from the instance at call time, preferring qbit-crm's snake_case names (`crm_contact`, ...) and falling back to the legacy camelCase ones (`crmContact`, ...): +`QbitCrmGateway` implements it over qbit-crm 0.2.0's `crm_*` tables (`crm_contact`, `crm_consent_record`, ...) by table name, through the standard actions, so there is no compile dependency on the CRM. It moves to qbit-crm's public `com.kingsrook.qbits.crm.api.CrmMarketingApi` once that is released: - **Email consent:** the most recent Email consent record decides (Granted and not expired). No record means no consent. - **Suppression:** `doNotEmail`, an unsubscribe date, or a list membership that isn't Subscribed. diff --git a/src/main/java/com/kingsrook/qbits/marketing/crm/QbitCrmGateway.java b/src/main/java/com/kingsrook/qbits/marketing/crm/QbitCrmGateway.java index b72c114..6a1f477 100644 --- a/src/main/java/com/kingsrook/qbits/marketing/crm/QbitCrmGateway.java +++ b/src/main/java/com/kingsrook/qbits/marketing/crm/QbitCrmGateway.java @@ -40,16 +40,15 @@ import com.kingsrook.qqq.backend.core.utils.ValueUtils; ** compile dependency on the CRM and CRM permissions and locks apply. ** ** Rules it applies (from qbit-crm's own model): - ** - Email consent: the contact's most recent consent record (crm_consent_record, or legacy crmConsentRecord) of type Email + ** - Email consent: the contact's most recent crm_consent_record of type Email ** decides (Granted, and not past its expiry date). No record means no ** consent: Marketing does not send unsolicited email. ** - Suppression: doNotEmail, an unsubscribeDate, or a list membership that ** is not Subscribed. ** - Activities are logged as qbit-crm "Note" activities. ** - ** Table names are resolved from the QInstance at call time: the snake_case - ** name qbit-crm now uses is preferred, falling back to its legacy camelCase - ** name, so either qbit-crm generation works. + ** It uses qbit-crm 0.2.0's crm_* snake_case tables (no fallback to the + ** 0.1.0 camelCase names). *******************************************************************************/ public class QbitCrmGateway implements MarketingCrmGateway { @@ -60,16 +59,6 @@ public class QbitCrmGateway implements MarketingCrmGateway public static final String ACTIVITY_TABLE = "crm_activity"; public static final String ACTIVITY_TYPE_TABLE = "crm_activity_type"; - /////////////////////////////////////////////////////////////////////// - // the names qbit-crm used before it moved to snake_case; still used // - // when the instance has the old tables // - /////////////////////////////////////////////////////////////////////// - public static final String LEGACY_CONTACT_TABLE = "crmContact"; - public static final String LEGACY_CONSENT_TABLE = "crmConsentRecord"; - public static final String LEGACY_LIST_MEMBER_TABLE = "crmEmailListMember"; - public static final String LEGACY_FORM_SUBMISSION_TABLE = "crmFormSubmission"; - public static final String LEGACY_ACTIVITY_TABLE = "crmActivity"; - public static final String LEGACY_ACTIVITY_TYPE_TABLE = "crmActivityType"; static final Integer CONSENT_TYPE_EMAIL = 1; static final Integer CONSENT_STATUS_GRANTED = 1; @@ -96,7 +85,7 @@ public class QbitCrmGateway implements MarketingCrmGateway public boolean isAvailable() { QInstance qInstance = QContext.getQInstance(); - return (qInstance != null && contactTable() != null && consentTable() != null && formSubmissionTable() != null); + return (qInstance != null && qInstance.getTable(CONTACT_TABLE) != null && qInstance.getTable(CONSENT_TABLE) != null && qInstance.getTable(FORM_SUBMISSION_TABLE) != null); } @@ -121,7 +110,7 @@ public class QbitCrmGateway implements MarketingCrmGateway else if(recipientScope.startsWith("crm-list:")) { Integer listId = parseId(recipientScope, "crm-list:"); - for(QRecord member : MarketingStore.queryBy(listMemberTable(), "emailListId", listId)) + for(QRecord member : MarketingStore.queryBy(LIST_MEMBER_TABLE, "emailListId", listId)) { Integer contactId = member.getValueInteger("contactId"); if(contactId == null) @@ -150,7 +139,7 @@ public class QbitCrmGateway implements MarketingCrmGateway String address = null; if(eligibility.eligible()) { - QRecord contact = MarketingStore.get(contactTable(), contactId); + QRecord contact = MarketingStore.get(CONTACT_TABLE, contactId); address = contact == null ? null : contact.getValueString("email"); } return (new CrmRecipient(contactId, address, eligibility)); @@ -164,7 +153,7 @@ public class QbitCrmGateway implements MarketingCrmGateway @Override public Eligibility checkEmailEligibility(Integer contactId) throws QException { - QRecord contact = MarketingStore.get(contactTable(), contactId); + QRecord contact = MarketingStore.get(CONTACT_TABLE, contactId); if(contact == null) { return (Eligibility.no("contact_not_found")); @@ -182,7 +171,7 @@ public class QbitCrmGateway implements MarketingCrmGateway return (Eligibility.no("no_email_address")); } - List consents = MarketingStore.query(consentTable(), new QQueryFilter( + List consents = MarketingStore.query(CONSENT_TABLE, new QQueryFilter( new QFilterCriteria("contactId", QCriteriaOperator.EQUALS, contactId), new QFilterCriteria("consentType", QCriteriaOperator.EQUALS, CONSENT_TYPE_EMAIL))); if(consents.isEmpty()) @@ -225,7 +214,7 @@ public class QbitCrmGateway implements MarketingCrmGateway { filter.addCriteria(new QFilterCriteria("createDate", QCriteriaOperator.GREATER_THAN_OR_EQUALS, since)); } - for(QRecord submission : MarketingStore.query(formSubmissionTable(), filter)) + for(QRecord submission : MarketingStore.query(FORM_SUBMISSION_TABLE, filter)) { list.add(new CrmFormSubmissionRef(submission.getValueInteger("id"), submission.getValueInteger("contactId"), ValueUtils.getValueAsInstant(submission.getValue("createDate")))); } @@ -240,7 +229,7 @@ public class QbitCrmGateway implements MarketingCrmGateway @Override public void recordActivity(Integer contactId, String subject, String description) throws QException { - List types = MarketingStore.queryBy(activityTypeTable(), "name", NOTE_ACTIVITY_TYPE_NAME); + List types = MarketingStore.queryBy(ACTIVITY_TYPE_TABLE, "name", NOTE_ACTIVITY_TYPE_NAME); if(types.isEmpty()) { throw (new QUserFacingException("The CRM has no \"" + NOTE_ACTIVITY_TYPE_NAME + "\" activity type to log marketing activities with.")); @@ -249,7 +238,7 @@ public class QbitCrmGateway implements MarketingCrmGateway ////////////////////////////////////////////////////////////////// // written with the user's CRM permissions, not Marketing's scope // ////////////////////////////////////////////////////////////////// - com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput insert = new com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput(activityTable()); + com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput insert = new com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput(ACTIVITY_TABLE); insert.withPermissionEnforcement(com.kingsrook.qqq.backend.core.actions.permissions.PermissionEnforcement.ENFORCE); insert.withRecord(new QRecord() .withValue("activityTypeId", types.get(0).getValueInteger("id")) @@ -268,86 +257,6 @@ public class QbitCrmGateway implements MarketingCrmGateway - /******************************************************************************* - ** The contact table's name in the current instance (null if neither naming exists). - *******************************************************************************/ - public static String contactTable() - { - return (resolve(CONTACT_TABLE, LEGACY_CONTACT_TABLE)); - } - - - - /******************************************************************************* - ** - *******************************************************************************/ - public static String consentTable() - { - return (resolve(CONSENT_TABLE, LEGACY_CONSENT_TABLE)); - } - - - - /******************************************************************************* - ** - *******************************************************************************/ - public static String listMemberTable() - { - return (resolve(LIST_MEMBER_TABLE, LEGACY_LIST_MEMBER_TABLE)); - } - - - - /******************************************************************************* - ** - *******************************************************************************/ - public static String formSubmissionTable() - { - return (resolve(FORM_SUBMISSION_TABLE, LEGACY_FORM_SUBMISSION_TABLE)); - } - - - - /******************************************************************************* - ** - *******************************************************************************/ - public static String activityTable() - { - return (resolve(ACTIVITY_TABLE, LEGACY_ACTIVITY_TABLE)); - } - - - - /******************************************************************************* - ** - *******************************************************************************/ - public static String activityTypeTable() - { - return (resolve(ACTIVITY_TYPE_TABLE, LEGACY_ACTIVITY_TYPE_TABLE)); - } - - - - /******************************************************************************* - ** Prefer the snake_case table, fall back to the legacy one; null if the - ** instance has neither. - *******************************************************************************/ - private static String resolve(String name, String legacyName) - { - QInstance qInstance = QContext.getQInstance(); - if(qInstance == null) - { - return (null); - } - if(qInstance.getTable(name) != null) - { - return (name); - } - return (qInstance.getTable(legacyName) != null ? legacyName : null); - } - - - /******************************************************************************* ** *******************************************************************************/ diff --git a/src/test/java/com/kingsrook/qbits/marketing/QbitCrmGatewayTest.java b/src/test/java/com/kingsrook/qbits/marketing/QbitCrmGatewayTest.java index 6691a4b..67d0256 100644 --- a/src/test/java/com/kingsrook/qbits/marketing/QbitCrmGatewayTest.java +++ b/src/test/java/com/kingsrook/qbits/marketing/QbitCrmGatewayTest.java @@ -35,8 +35,8 @@ import static org.assertj.core.api.Assertions.assertThatThrownBy; /******************************************************************************* - ** QbitCrmGateway against fixture CRM tables under both qbit-crm naming - ** schemes: snake_case (current) and camelCase (legacy). + ** QbitCrmGateway against fixture tables shaped like qbit-crm 0.2.0's crm_* + ** tables. *******************************************************************************/ class QbitCrmGatewayTest extends BaseMarketingTest { @@ -56,57 +56,29 @@ class QbitCrmGatewayTest extends BaseMarketingTest ** *******************************************************************************/ @Test - void testSnakeCaseTables() throws QException + void testCrmFlows() throws QException { defineSnakeCaseTables(); - assertThat(QbitCrmGateway.contactTable()).isEqualTo("crm_contact"); + assertThat(QbitCrmGateway.CONTACT_TABLE).isEqualTo("crm_contact"); exerciseGateway(); } /******************************************************************************* - ** - *******************************************************************************/ - @Test - void testLegacyCamelCaseTables() throws QException - { - defineLegacyTables(); - assertThat(QbitCrmGateway.contactTable()).isEqualTo("crmContact"); - exerciseGateway(); - } - - - - /******************************************************************************* - ** The snake_case tables win when both generations are defined. - *******************************************************************************/ - @Test - void testSnakeCasePreferredWhenBothExist() - { - defineSnakeCaseTables(); - defineLegacyTables(); - - assertThat(QbitCrmGateway.contactTable()).isEqualTo(QbitCrmGateway.CONTACT_TABLE); - assertThat(QbitCrmGateway.activityTypeTable()).isEqualTo(QbitCrmGateway.ACTIVITY_TYPE_TABLE); - } - - - - /******************************************************************************* - ** The same flows, whichever naming the instance has. + ** Eligibility, recipient scopes, form submissions and activities. *******************************************************************************/ private void exerciseGateway() throws QException { QbitCrmGateway gateway = new QbitCrmGateway(); assertThat(gateway.isAvailable()).isTrue(); - Integer granted = insert(QbitCrmGateway.contactTable(), new QRecord().withValue("email", "yes@example.org").withValue("doNotEmail", false)); - Integer noConsent = insert(QbitCrmGateway.contactTable(), new QRecord().withValue("email", "none@example.org").withValue("doNotEmail", false)); - Integer suppressed = insert(QbitCrmGateway.contactTable(), new QRecord().withValue("email", "dne@example.org").withValue("doNotEmail", true)); + Integer granted = insert(QbitCrmGateway.CONTACT_TABLE, new QRecord().withValue("email", "yes@example.org").withValue("doNotEmail", false)); + Integer noConsent = insert(QbitCrmGateway.CONTACT_TABLE, new QRecord().withValue("email", "none@example.org").withValue("doNotEmail", false)); + Integer suppressed = insert(QbitCrmGateway.CONTACT_TABLE, new QRecord().withValue("email", "dne@example.org").withValue("doNotEmail", true)); for(Integer contactId : List.of(granted, suppressed)) { - insert(QbitCrmGateway.consentTable(), new QRecord().withValue("contactId", contactId).withValue("consentType", 1).withValue("status", 1) + insert(QbitCrmGateway.CONSENT_TABLE, new QRecord().withValue("contactId", contactId).withValue("consentType", 1).withValue("status", 1) .withValue("consentDate", Instant.now().minus(2, ChronoUnit.DAYS))); } @@ -128,9 +100,9 @@ class QbitCrmGatewayTest extends BaseMarketingTest ///////////////////////////////////// // resolveRecipients: list scope // ///////////////////////////////////// - insert(QbitCrmGateway.listMemberTable(), new QRecord().withValue("emailListId", 7).withValue("contactId", granted).withValue("status", 1)); - insert(QbitCrmGateway.listMemberTable(), new QRecord().withValue("emailListId", 7).withValue("contactId", noConsent).withValue("status", 1)); - insert(QbitCrmGateway.listMemberTable(), new QRecord().withValue("emailListId", 7).withValue("contactId", suppressed).withValue("status", 2)); + insert(QbitCrmGateway.LIST_MEMBER_TABLE, new QRecord().withValue("emailListId", 7).withValue("contactId", granted).withValue("status", 1)); + insert(QbitCrmGateway.LIST_MEMBER_TABLE, new QRecord().withValue("emailListId", 7).withValue("contactId", noConsent).withValue("status", 1)); + insert(QbitCrmGateway.LIST_MEMBER_TABLE, new QRecord().withValue("emailListId", 7).withValue("contactId", suppressed).withValue("status", 2)); List members = gateway.resolveRecipients("crm-list:7"); assertThat(members).hasSize(3); assertThat(members.stream().filter(r -> r.eligibility().eligible()).map(CrmRecipient::address)).containsExactly("yes@example.org"); @@ -140,8 +112,8 @@ class QbitCrmGatewayTest extends BaseMarketingTest // listFormSubmissions // /////////////////////////// Instant created = Instant.now().minus(1, ChronoUnit.HOURS); - insert(QbitCrmGateway.formSubmissionTable(), new QRecord().withValue("contactId", granted).withValue("utmCampaign", "spring").withValue("createDate", created)); - insert(QbitCrmGateway.formSubmissionTable(), new QRecord().withValue("contactId", noConsent).withValue("utmCampaign", "other").withValue("createDate", created)); + insert(QbitCrmGateway.FORM_SUBMISSION_TABLE, new QRecord().withValue("contactId", granted).withValue("utmCampaign", "spring").withValue("createDate", created)); + insert(QbitCrmGateway.FORM_SUBMISSION_TABLE, new QRecord().withValue("contactId", noConsent).withValue("utmCampaign", "other").withValue("createDate", created)); List submissions = gateway.listFormSubmissions("spring", null); assertThat(submissions).hasSize(1); assertThat(submissions.get(0).contactId()).isEqualTo(granted); @@ -151,9 +123,9 @@ class QbitCrmGatewayTest extends BaseMarketingTest // recordActivity // ////////////////////// assertThatThrownBy(() -> gateway.recordActivity(granted, "Subject", "Body")).hasMessageContaining("Note"); - insert(QbitCrmGateway.activityTypeTable(), new QRecord().withValue("name", "Note")); + insert(QbitCrmGateway.ACTIVITY_TYPE_TABLE, new QRecord().withValue("name", "Note")); gateway.recordActivity(granted, "Subject", "Body"); - List activities = MarketingStore.query(QbitCrmGateway.activityTable(), new QQueryFilter()); + List activities = MarketingStore.query(QbitCrmGateway.ACTIVITY_TABLE, new QQueryFilter()); assertThat(activities).hasSize(1); assertThat(activities.get(0).getValueString("subject")).isEqualTo("Subject"); assertThat(activities.get(0).getValueInteger("contactId")).isEqualTo(granted); @@ -182,17 +154,6 @@ class QbitCrmGatewayTest extends BaseMarketingTest - /******************************************************************************* - ** - *******************************************************************************/ - private void defineLegacyTables() - { - defineCrmTables(QbitCrmGateway.LEGACY_CONTACT_TABLE, QbitCrmGateway.LEGACY_CONSENT_TABLE, QbitCrmGateway.LEGACY_LIST_MEMBER_TABLE, - QbitCrmGateway.LEGACY_FORM_SUBMISSION_TABLE, QbitCrmGateway.LEGACY_ACTIVITY_TABLE, QbitCrmGateway.LEGACY_ACTIVITY_TYPE_TABLE); - } - - - /******************************************************************************* ** Define fixture tables (memory backend) shaped like qbit-crm's. *******************************************************************************/ -- 2.54.0