Compare commits

..
Author SHA1 Message Date
james.maesandClaude Opus 5.5 78ad854f41 refactor(email): use qbit-email's public api package (fixes #13)
QbitEmailTransport and MarketingRuntime use com.kingsrook.qbits.email.api
from qbit-email 0.1.0-RC.3; getStatus() keeps its mapping through
EmailApiReadiness.failedChecks (NOT_CONFIGURED only for the base URL and
unsubscribe key; a missing provider or brand address is still REJECTED
at send). qbit-build-parent 2.1.1 allows qbit-email's api by default, so
the temporary email allowances are gone; only the existing assistant
provider SPI allowances remain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 10:30:56 -05:00
james.maes 62e12e9ebe Merge pull request #12 from QRun-IO/feature/arch-provider-api
refactor(providers): depend only on qbit-assistant-provider-api (#9)
2026-10-14 08:27:24 -05:00
james.maesandClaude Opus 5.5 e790a1091f Merge origin/develop into feature/arch-provider-api
Brings in the qbit-email transport (#11). Conflicts are resolved in
favour of the RC.6/provider-api side, keeping qbit-email.

qbit-email is a new cross-qbit implementation dependency that the
enforced G1 guardrail rejects. It is allow-listed explicitly and
temporarily until qbit-email publishes an API (#13).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 08:18:27 -05:00
james.maes 1c5bada863 Merge pull request #11 from QRun-IO/feature/email-qbit
feat(email): QbitEmailTransport for MARKETING sends through qbit-email
2026-10-14 08:10:59 -05:00
james.maesandClaude Opus 5.5 3ec8776c2a refactor(providers): depend only on qbit-assistant-provider-api and provider-openai
Marketing compiled against qbit-assistant internals, and installing it
pulled in the whole assistant runtime. It now depends on
qbit-assistant-provider-api (the provider SPI and ModelPricing) and on the
published qbit-assistant-provider-openai, both 0.1.0-RC.11. The local
OpenAiCompatibleModelProvider is gone, and tests use a local scripted
double, so the full assistant is not needed even in test scope.

Hosts register the published provider through ProviderSettings. Its
provider name is "openai-compatible", so Marketing Policy provider
overrides must use that name.

Also move to QQQ 4.1.0-RC.6 and qbit-build-parent 2.1.0 with the
guardrails enforced. The provider-api packages and provider-openai are
explicitly allow-listed.

Closes #9 (architecture review QRun-IO/qqq#1150, finding F5).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 08:00:04 -05:00
james.maes 4bd4d0f482 Merge pull request #10 from QRun-IO/feature/contingent-possible-values
feat: contingent possible values (product-scoped references)
2026-10-14 07:09:09 -05:00
james.maesandClaude Opus 5.5 fa98cc082b feat: contingent possible values for scoped foreign keys
Dependent choices follow their parent selection on the same form (QQQ possibleValueSourceFilter with ${input.x}); filter builders still list everything.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 20:25:11 -05:00
17 changed files with 627 additions and 595 deletions

No files matched your search

+2 -1
View File
@@ -46,7 +46,8 @@ Services: `approval.ApprovalService`, `execution.ExecutionService`, `measurement
## Dependencies
- Java 21.
- Parent `com.kingsrook:qbit-build-parent:2.0.0` (qqq 4.0.0). The default-active `qqq-snapshot` profile imports `qqq-bom-pom:4.1.0-RC.5` ahead of it, as in qbit-quick-search (ADR-0007). Remove that activation when the parent moves to 4.1.
- Parent `com.kingsrook:qbit-build-parent:2.1.0` (qqq 4.0.0), with architecture enforcement and strict UI API routes enabled. The default-active `qqq-snapshot` profile imports `qqq-bom-pom:4.1.0-RC.6` ahead of it, as in qbit-quick-search (ADR-0007). Remove that activation when the parent moves to 4.1.
- Assistant provider API and OpenAI-compatible provider `0.1.0-RC.11`; no full assistant dependency. For local verification before publication only, use `-Dqbit-assistant.version=0.1.0-ARCHLOCAL-SNAPSHOT -Dqbit.guardrails.enforce=false` and inspect guardrail findings.
- Tests: JUnit 5, AssertJ.
## Release
+8 -1
View File
@@ -4,7 +4,14 @@ All notable changes to this project are documented here. The format follows Keep
## [0.1.0] - Unreleased
### Changed
- Switch the optional qbit-email integration to the published `com.kingsrook.qbits.email.api` package in `0.1.0-RC.3` (fixes #13). Preserve MARKETING messages, idempotency keys, execution links and outcome mapping; use typed readiness failures so missing providers or brand postal addresses remain send-time rejections. Upgrade `qbit-build-parent` to `2.1.1` and remove all per-repository guardrail allow-lists.
- Resolve architecture finding F5 (#9): depend on `qbit-assistant-provider-api` and `qbit-assistant-provider-openai` `0.1.0-RC.11`, with no full assistant dependency, including in tests.
- Replace the local OpenAI-compatible implementation with the published provider and document `ProviderSettings`, environment-key resolution and provider-name migration. Agent tests use a local scripted double.
- Upgrade QQQ to `4.1.0-RC.6`; enable architecture enforcement and strict UI API routes.
### Added
- Product-contingent possible values for editable campaign, claim, finding, metric and outcome-event references.
- Repository scaffold: build, CI and the repository standard files.
- **Model:** 18 standard `marketing_*` tables, 21 enum possible-value sources, joins and child lists. The PostgreSQL reference DDL (`docs/ddl/postgres.sql`) and an H2 twin are generated from one spec (`scripts/generate-model.py`).
- **Products and evidence:**
@@ -34,7 +41,7 @@ All notable changes to this project are documented here. The format follows Keep
- Narrow typed tools (6 read, 3 write-proposal), run as the user with core permission enforcement, the untrusted-data envelope and redaction.
- Run and step records with usage and cost.
- Budgets, concurrency and rate limits, cancellation, and a plain failure reason when no provider is configured.
- **Model providers:** `OpenAiCompatibleModelProvider` (no SDK), plus qbit-assistant's Claude and scripted providers.
- **Model providers:** the published `OpenAiCompatibleModelProvider` (no SDK), host-supplied providers such as Claude, and a local scripted test double.
- **AI tool kernel:** the read tools are contributed through `AiToolRegistry` when QQQ has it (QRun-IO/qqq#1096).
- **UI:**
- The Marketing app; its home is the Marketing dashboard (Needs Attention, Campaigns, Agent Activity, Configuration Status).
+31 -6
View File
@@ -6,7 +6,7 @@
> Marketing for QQQ applications: products and their verified evidence, campaigns, content with revision history, exact human approval, controlled execution, honest measurement, and a marketing agent that prepares work but never publishes.
**Status:** 0.1.0, not yet published.
- Built against QQQ `4.1.0-RC.5`.
- Built against QQQ `4.1.0-RC.6` with `qbit-build-parent` `2.1.1`.
- Hosted in QRun-IO/Voyage next to qbit-crm, on a local branch.
- The real integrations are implemented but unconfigured. See [what is configured](#what-is-implemented-what-is-not-configured).
@@ -25,7 +25,7 @@ It lives inside the host QQQ application: its navigation, authentication, permis
| **Measurement** | Delivery, awareness, engagement, activation, repeat use and qualified demand are reported separately. Each metric shows raw events and unique actors. "Not measured" is never shown as 0. Test data is kept apart. |
| **CRM** | Uses the host CRM's own contacts, organizations, consent, suppression, lists, form submissions and activities through a narrow gateway. A qbit-crm adapter is included. Nothing is copied, and nobody becomes a lead without the campaign's explicit rule. |
| **Agent** | Plans campaigns, drafts and revises content, flags unsupported claims, stale sources and readiness gaps, and analyzes results to recommend the next experiment. It uses narrow typed tools, runs as the requesting user, sees data only as untrusted input, and sees no CRM personal data. Every run is recorded with its steps, provider, model, tokens and cost. It can create drafts and findings; it cannot approve, schedule or send. |
| **Models** | Interchangeable providers through qbit-assistant's `AssistantModelProvider`: Claude (`qbit-assistant-provider-anthropic`), any OpenAI-compatible endpoint (included), and a scripted provider for tests. With no provider configured, the agent says so and everything else works. |
| **Models** | Interchangeable providers through `qbit-assistant-provider-api`'s `AssistantModelProvider`: OpenAI-compatible endpoints through `qbit-assistant-provider-openai` (included), or a host-supplied provider such as Claude. Tests use a local scripted double. With no provider configured, the agent says so and everything else works. Installing marketing does not install the assistant runtime, tables or routes. |
## Install
@@ -38,14 +38,24 @@ It lives inside the host QQQ application: its navigation, authentication, permis
```
```java
import com.kingsrook.qbits.assistant.provider.ProviderSettings;
import com.kingsrook.qbits.assistant.provider.openai.OpenAiCompatibleModelProvider;
import com.kingsrook.qqq.backend.core.instances.QMetaDataVariableInterpreter;
var provider = new OpenAiCompatibleModelProvider(new ProviderSettings()
.withKind("OPENAI_COMPATIBLE")
.withBaseUrl("https://api.openai.com/v1")
.withApiKey(new QMetaDataVariableInterpreter().interpret("${env.MARKETING_OPENAI_API_KEY}")));
new MarketingQBitProducer().withQBitConfig(new MarketingQBitConfig()
.withBackendName("rdbms") // where the marketing_* tables live
.withCrmGateway(new QbitCrmGateway()) // optional: qbit-crm
.withSchedulerName("scheduler") // optional: runs the execution worker
.withTransport(new QqqMessagingEmailTransport() // optional: email via your QQQ messaging provider
.withMessagingProviderName("email").withFromAddress("news@example.org"))
.withModelProvider(AnthropicModelProvider.fromEnvironment()) // optional: only when ANTHROPIC_API_KEY is set
.withModel("claude-sonnet-5-5"))
.withModelProvider(provider) // optional: configure only for agent use
.withProviderName("openai-compatible")
.withModel("your-model-id"))
.produce(qInstance);
```
@@ -56,6 +66,11 @@ Then:
[`docs/OPERATIONS.md`](docs/OPERATIONS.md) covers every setting, the run-books and the limits.
The provider SPI, pricing and OpenAI-compatible implementation are pinned to `0.1.0-RC.11`.
Replace imports of marketing's former `agent.provider.OpenAiCompatibleModelProvider` with the published class above and replace its builder with `ProviderSettings`.
The host resolves the API key reference; `ProviderSettings.withApiKeyEnvName(...)` alone does not resolve it.
Set any existing Marketing Policy provider override to `openai-compatible` as well. The base URL, key environment variable name and model remain host-configurable; omit the key for a local endpoint that needs none.
## Permissions
| Permission | Grants |
@@ -92,7 +107,7 @@ When the optional [qbit-email](https://github.com/QRun-IO/qbit-email) qbit is pr
- Idempotency key `marketing:<content id>:<approval id>:<recipient key>`; the linked record is the `marketing_execution` row. A repeat is a duplicate, never a second send.
- Accepted means queued in the qbit-email outbox. A suppressed recipient is skipped, not failed (the result says how many); a refusal by qbit-email is a rejected execution with its reason.
- Needs qbit-email's `publicBaseUrl`, the `QQQ_EMAIL_UNSUBSCRIBE_KEY` secret, an enabled default provider and a default brand with a postal address. Without the first two the transport reports *Not configured* and says which is missing.
- qbit-email is an optional dependency (`qbit-email.version`, a SNAPSHOT until its first RC) and needs QQQ `4.1.0-RC.5`.
- qbit-email is an optional dependency pinned to `0.1.0-RC.3` from Maven Central (`qbit-email.version`). The integration uses only its published `com.kingsrook.qbits.email.api` package for installation checks, readiness, sends and lookups, with the parent's default guardrail allow-lists.
## What is implemented, what is not configured
@@ -102,7 +117,7 @@ When the optional [qbit-email](https://github.com/QRun-IO/qbit-email) qbit is pr
| Approval binding, invalidation, separation of duties | Email transport (needs the host's QQQ messaging provider; reports *Not configured* until then, and *Configured* is not *Operational*) |
| Durable execution, retries, reconciliation, leases | No social, community, blog or CMS connector exists: those channels are posted by hand |
| Controlled test transport (complete) | No analytics importer: awareness and activation metrics stay *Not measured* until something records outcome events |
| Agent loop, tools, budgets, limits, run records | QQQ AI tool kernel registration (needs QRun-IO/qqq#1096; the qbit works without it) |
| Agent loop, tools, budgets, limits, run records; read-tool registration in the QQQ RC.6 AI tool registry | |
| CRM gateway and qbit-crm adapter, explicit lead rules | |
| Persistence across restart (H2 file, and PostgreSQL in the Voyage host) | |
@@ -116,6 +131,16 @@ Scale gaps (single worker per JVM, in-memory rate limits) are listed in [`docs/O
Always use the Maven wrapper (Maven 3.9.11); Maven 3.10 cannot read the parent POM. The entities, enums and DDL are generated from one spec: `python -I scripts/generate-model.py .` (commit the output).
Architecture guardrails and strict UI API route checks are enabled. The artifact allow-list preserves the parent's defaults and adds only `qbit-assistant-provider-openai`; the package allow-list also permits the extracted provider SPI and pricing. Parent 2.1.0 matches package prefixes using `prefix + "."`, so the pricing allowance must be `com.kingsrook.qbits.assistant.engine`, not the class name `…engine.ModelPricing`. The API artifact contains only `ModelPricing` in that package; the full assistant artifact remains prohibited.
Until provider `0.1.0-RC.11` is available on Central, verify against the locally installed assistant develop build only with:
```bash
./mvnw -Dqbit-assistant.version=0.1.0-ARCHLOCAL-SNAPSHOT -Dqbit.guardrails.enforce=false verify
```
This runs the guardrails in report mode so G7 can report the local SNAPSHOT dependencies. Keep the release version and enforcement enabled in the POM.
## License
Apache License 2.0. See [LICENSE](LICENSE) and [NOTICE](NOTICE).
+1 -1
View File
@@ -19,7 +19,7 @@ The work is truthful and auditable: facts come from sources with dates, people a
| From | Reused for |
|---|---|
| **qbit-assistant** (0.1.0-RC.2) | The model-provider SPI: `AssistantModelProvider`, `ModelRequest` / `ModelResponse` / `ModelPart`, `StreamHandle`, `ModelProviderException` with its retryable kinds, and `ModelUsage`. Also the Claude provider (`qbit-assistant-provider-anthropic`), `ScriptedModelProvider` for tests and demos, and `ModelPricing` for cost. Marketing adds an OpenAI-compatible provider that implements the same SPI. There is no second provider abstraction. |
| **QQQ AI tool kernel** (qqq#1078, core) | The tool contract: `AbstractAiTool` / `AiToolKind` (READ, WRITE_PROPOSAL), `AiToolOutcome`, the `qqq_data` untrusted-data envelope (`AiDataEnvelope`), secret redaction (`AiRedactor`), strict input keys, and running as the user's session. The kernel's tool set is fixed in RC.4. So QRun-IO/qqq#1096 adds `AiToolRegistry`; Marketing registers its read tools there when QQQ has it and otherwise runs them in its own runner with the same contract. |
| **QQQ AI tool kernel** (qqq#1078, core) | The tool contract: `AbstractAiTool` / `AiToolKind` (READ, WRITE_PROPOSAL), `AiToolOutcome`, the `qqq_data` untrusted-data envelope (`AiDataEnvelope`), secret redaction (`AiRedactor`), strict input keys, and running as the user's session. The RC.6 baseline includes `AiToolRegistry` (QRun-IO/qqq#1096); Marketing contributes its read tools there and runs its own agent tools with the same contract. |
| **QQQ core permission enforcement** (qqq#1061) | Agent tools read with `PermissionEnforcement.ENFORCE`. Service entry points check their step's permission, then run as trusted internal work in an authorized scope, as QQQ does for calls nested in a core action. Record security locks always apply. |
| **qbit-assistant / sql-workbench metadata builders** | Explicit builder-produced tables (not annotation scanning), snake_case `marketing_*` names, QQQ standard permission suffixes, `DenyBehavior.HIDDEN`, read-only system tables, record locks, reference DDL in `docs/ddl`, `tableMetaDataCustomizer` and `inferBackendNames`. |
| **Website-Backend admin conventions** | Processes are table and record actions, never nav items, with plain labels. Record actions use `StreamedETLWithFrontendProcess` with standard summary lines. App homes are real-data dashboards built from ≤3-column full-width tables. Links use the Next route grammar (`AiLinkBuilder`). |
+1 -1
View File
@@ -48,7 +48,7 @@ Nothing in Marketing stores a credential, and every key is read from the host's
| Integration | What to provide |
|---|---|
| Claude | `ANTHROPIC_API_KEY` (or `AnthropicModelProvider.builder().apiKey(fromSecretStore)`), and a model id such as `claude-sonnet-5-5`. |
| OpenAI-compatible | A base URL (`https://api.openai.com/v1`, `http://localhost:11434/v1` for Ollama, ...), an API key reference such as `${env.MARKETING_OPENAI_API_KEY}` if the server needs one, and a model id. |
| OpenAI-compatible | Use `com.kingsrook.qbits.assistant.provider.openai.OpenAiCompatibleModelProvider` from `qbit-assistant-provider-openai` with `ProviderSettings`: a base URL (`https://api.openai.com/v1`, `http://localhost:11434/v1` for Ollama, ...), a resolved key if the server needs one, and a model id in `MarketingQBitConfig.withModel(...)`. Resolve `${env.MARKETING_OPENAI_API_KEY}` with `QMetaDataVariableInterpreter` before `withApiKey(...)`; setting `apiKeyEnvName` alone does not resolve a key. The provider name is `openai-compatible`, including in Marketing Policy overrides. See the [setup example](../README.md#install). |
| Email (qbit-email) | When qbit-email is produced in the instance and no transport named `qqq_email` is registered, email goes through its outbox (category MARKETING). It needs `publicBaseUrl`, the `QQQ_EMAIL_UNSUBSCRIBE_KEY` secret, an enabled default provider and a default brand with a postal address; a missing piece is reported as *Not configured* or as a plain rejected-execution reason. |
| Email (QQQ messaging) | A QQQ messaging provider in the host instance (SMTP or SES), its name, and a from-address. After verifying it end to end, `withVerifiedOperational(true)`. Until then the email transport reports *Configured, not verified* and executions are blocked. |
+18 -9
View File
@@ -22,19 +22,23 @@
<maven.compiler.source>21</maven.compiler.source>
<maven.compiler.target>21</maven.compiler.target>
<jacoco.version>0.8.15</jacoco.version>
<!-- The model-provider SPI (AssistantModelProvider), the scripted test -->
<!-- provider and the price table come from qbit-assistant; marketing -->
<!-- reuses them instead of a second provider abstraction. -->
<qbit-assistant.version>0.1.0-RC.2</qbit-assistant.version>
<!-- Share the provider SPI and pricing without installing the assistant runtime. -->
<qbit-assistant.version>0.1.0-RC.11</qbit-assistant.version>
<qbit.guardrails.enforce>true</qbit.guardrails.enforce>
<qbit.guardrails.strictUiApiRoutes>true</qbit.guardrails.strictUiApiRoutes>
<!-- qbit-email from Maven Central -->
<qbit-email.version>0.1.0-RC.1</qbit-email.version>
<!-- Assistant provider SPI only (not email): these replace the parent lists, so they repeat -->
<!-- the parent defaults (object storage, qbit-email's api, *-api) plus the assistant entries. -->
<qbit.guardrails.allowedApiPackages>com.kingsrook.qbits.objectstorage.api,com.kingsrook.qbits.email.api,com.kingsrook.qbits.assistant.provider,com.kingsrook.qbits.assistant.engine</qbit.guardrails.allowedApiPackages>
<qbit.guardrails.allowedQbitArtifacts>com.kingsrook.qbits:qbit-object-storage,com.kingsrook.qbits:qbit-email,com.kingsrook.qbits:*-api,com.kingsrook.qbits:qbit-assistant-provider-openai</qbit.guardrails.allowedQbitArtifacts>
<qbit-email.version>0.1.0-RC.3</qbit-email.version>
<h2.version>2.3.232</h2.version>
</properties>
<parent>
<groupId>com.kingsrook</groupId>
<artifactId>qbit-build-parent</artifactId>
<version>2.0.0</version>
<version>2.1.1</version>
</parent>
<groupId>com.kingsrook.qbits</groupId>
@@ -81,7 +85,12 @@
<dependency>
<groupId>com.kingsrook.qbits</groupId>
<artifactId>qbit-assistant</artifactId>
<artifactId>qbit-assistant-provider-api</artifactId>
<version>${qbit-assistant.version}</version>
</dependency>
<dependency>
<groupId>com.kingsrook.qbits</groupId>
<artifactId>qbit-assistant-provider-openai</artifactId>
<version>${qbit-assistant.version}</version>
</dependency>
@@ -187,14 +196,14 @@
<profiles>
<!-- Built against qqq 4.1 (QBitMetaDataProducer). Until qbit-build-parent -->
<!-- imports the released 4.1 BOM, this profile is active by default and -->
<!-- imports qqq-bom-pom 4.1.0-RC.5 ahead of the parent's 4.0.0 BOM. -->
<!-- imports qqq-bom-pom 4.1.0-RC.6 ahead of the parent's 4.0.0 BOM. -->
<profile>
<id>qqq-snapshot</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<qqq.snapshot.version>4.1.0-RC.5</qqq.snapshot.version>
<qqq.snapshot.version>4.1.0-RC.6</qqq.snapshot.version>
</properties>
<dependencyManagement>
<dependencies>
@@ -168,7 +168,7 @@ public class MarketingRuntime implements QSupplementalInstanceMetaData
{
try
{
Class.forName("com.kingsrook.qbits.email.outbox.EmailService");
Class.forName("com.kingsrook.qbits.email.api.EmailApi");
return (true);
}
catch(ClassNotFoundException | LinkageError e)
@@ -1,512 +0,0 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.marketing.agent.provider;
import java.io.IOException;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.net.http.HttpTimeoutException;
import java.time.Duration;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.CompletableFuture;
import com.fasterxml.jackson.core.type.TypeReference;
import com.kingsrook.qbits.assistant.provider.AssistantModelProvider;
import com.kingsrook.qbits.assistant.provider.CompletableStreamHandle;
import com.kingsrook.qbits.assistant.provider.ModelCapabilities;
import com.kingsrook.qbits.assistant.provider.ModelEventListener;
import com.kingsrook.qbits.assistant.provider.ModelMessage;
import com.kingsrook.qbits.assistant.provider.ModelPart;
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
import com.kingsrook.qbits.assistant.provider.ModelRequest;
import com.kingsrook.qbits.assistant.provider.ModelResponse;
import com.kingsrook.qbits.assistant.provider.ModelToolDefinition;
import com.kingsrook.qbits.assistant.provider.ModelUsage;
import com.kingsrook.qbits.assistant.provider.StreamHandle;
import com.kingsrook.qbits.assistant.provider.SystemBlock;
import com.kingsrook.qqq.backend.core.instances.QMetaDataVariableInterpreter;
import com.kingsrook.qqq.backend.core.utils.JsonUtils;
import com.kingsrook.qqq.backend.core.utils.StringUtils;
/*******************************************************************************
** A model provider for any OpenAI-compatible Chat Completions endpoint
** (POST {baseUrl}/chat/completions): OpenAI, and the many servers that speak
** the same API (OpenRouter, Ollama, vLLM, LM Studio, LiteLLM, ...).
** Implements qbit-assistant's AssistantModelProvider, so it plugs into the
** marketing agent (and the assistant) next to the Anthropic provider.
**
** - No SDK: java.net.http. Non-streaming request; the text is delivered to
** the listener at the end.
** - The API key is read from the environment / secret reference given to
** the builder ("${env.NAME}"); it is never logged, and toString never
** contains it. A provider without a key is allowed for local servers that
** need none (Ollama).
** - Errors map to ModelProviderException kinds: 401/403 AUTHENTICATION,
** 429 RATE_LIMITED, 5xx SERVER / OVERLOADED (503), 400/404/422
** INVALID_REQUEST, timeouts TIMEOUT, I/O NETWORK. Messages carry the HTTP
** status and the API's error type only.
*******************************************************************************/
public class OpenAiCompatibleModelProvider implements AssistantModelProvider
{
private final String name;
private final String baseUrl;
private final String apiKey;
private final Duration requestTimeout;
private final HttpClient httpClient;
private final Boolean strictTools;
/*******************************************************************************
** Use the builder.
*******************************************************************************/
private OpenAiCompatibleModelProvider(Builder builder)
{
this.name = builder.name;
this.baseUrl = builder.baseUrl.endsWith("/") ? builder.baseUrl.substring(0, builder.baseUrl.length() - 1) : builder.baseUrl;
String key = builder.apiKey;
if(key != null && key.startsWith("${"))
{
key = new QMetaDataVariableInterpreter().interpret(key);
}
this.apiKey = StringUtils.hasContent(key) ? key : null;
this.requestTimeout = builder.requestTimeout;
this.strictTools = builder.strictTools;
this.httpClient = HttpClient.newBuilder().connectTimeout(Duration.ofSeconds(10)).build();
}
/*******************************************************************************
** A builder.
*******************************************************************************/
public static Builder builder()
{
return (new Builder());
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public String getName()
{
return (name);
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public ModelCapabilities getCapabilities(String modelId)
{
return (new ModelCapabilities().withTools(true).withParallelToolCalls(true).withPromptCaching(false).withThinking(false).withMaxContextTokens(128_000));
}
/*******************************************************************************
** Does it have a key? (Local servers may need none.)
*******************************************************************************/
public boolean hasApiKey()
{
return (apiKey != null);
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public StreamHandle stream(ModelRequest request, ModelEventListener listener)
{
CompletableStreamHandle handle = new CompletableStreamHandle();
CompletableFuture<HttpResponse<String>> future;
try
{
HttpRequest.Builder http = HttpRequest.newBuilder(URI.create(baseUrl + "/chat/completions"))
.timeout(requestTimeout)
.header("Content-Type", "application/json")
.POST(HttpRequest.BodyPublishers.ofString(JsonUtils.toJson(toRequestBody(request))));
if(apiKey != null)
{
http.header("Authorization", "Bearer " + apiKey);
}
future = httpClient.sendAsync(http.build(), HttpResponse.BodyHandlers.ofString());
}
catch(Exception e)
{
handle.fail(new ModelProviderException(ModelProviderException.Kind.INVALID_REQUEST, null, "Could not build the request: " + e.getClass().getSimpleName()));
return (handle);
}
handle.setOnCancel(() -> future.cancel(true));
future.whenComplete((response, error) ->
{
if(error != null)
{
Throwable cause = error.getCause() != null ? error.getCause() : error;
if(cause instanceof HttpTimeoutException)
{
handle.fail(new ModelProviderException(ModelProviderException.Kind.TIMEOUT, null, "The request timed out."));
}
else if(cause instanceof IOException)
{
handle.fail(new ModelProviderException(ModelProviderException.Kind.NETWORK, null, "Network error: " + cause.getClass().getSimpleName()));
}
else
{
handle.fail(new ModelProviderException(ModelProviderException.Kind.OTHER, null, "Request failed: " + cause.getClass().getSimpleName()));
}
return;
}
try
{
if(response.statusCode() >= 300)
{
handle.fail(toException(response.statusCode(), response.body()));
return;
}
ModelResponse modelResponse = fromResponseBody(response.body());
String text = modelResponse.getText();
if(StringUtils.hasContent(text))
{
listener.onTextDelta(text);
}
handle.complete(modelResponse);
}
catch(Exception e)
{
handle.fail(new ModelProviderException(ModelProviderException.Kind.OTHER, response.statusCode(), "Unreadable response: " + e.getClass().getSimpleName()));
}
});
return (handle);
}
/*******************************************************************************
** ModelRequest -> Chat Completions request body.
*******************************************************************************/
Map<String, Object> toRequestBody(ModelRequest request)
{
List<Map<String, Object>> messages = new ArrayList<>();
StringBuilder system = new StringBuilder();
for(SystemBlock block : request.getSystem())
{
if(system.length() > 0)
{
system.append("\n\n");
}
system.append(block.getText());
}
if(system.length() > 0)
{
messages.add(Map.of("role", "system", "content", system.toString()));
}
for(ModelMessage message : request.getMessages())
{
if(message.getRole() == ModelMessage.Role.ASSISTANT)
{
Map<String, Object> assistant = new LinkedHashMap<>();
StringBuilder text = new StringBuilder();
List<Map<String, Object>> toolCalls = new ArrayList<>();
for(ModelPart part : message.getParts())
{
if(part.getType() == ModelPart.Type.TEXT && part.getText() != null)
{
text.append(part.getText());
}
else if(part.getType() == ModelPart.Type.TOOL_USE)
{
toolCalls.add(Map.of("id", part.getToolUseId(), "type", "function",
"function", Map.of("name", part.getToolName(), "arguments", JsonUtils.toJson(part.getInput() == null ? Map.of() : part.getInput()))));
}
}
assistant.put("role", "assistant");
assistant.put("content", text.length() == 0 ? null : text.toString());
if(!toolCalls.isEmpty())
{
assistant.put("tool_calls", toolCalls);
}
messages.add(assistant);
}
else
{
StringBuilder text = new StringBuilder();
for(ModelPart part : message.getParts())
{
if(part.getType() == ModelPart.Type.TOOL_RESULT)
{
messages.add(Map.of("role", "tool", "tool_call_id", part.getToolUseId(), "content", part.getText() == null ? "" : part.getText()));
}
else if(part.getText() != null)
{
text.append(part.getText());
}
}
if(text.length() > 0)
{
messages.add(Map.of("role", "user", "content", text.toString()));
}
}
}
Map<String, Object> body = new LinkedHashMap<>();
body.put("model", request.getModel());
body.put("messages", messages);
if(request.getMaxTokens() != null)
{
body.put("max_completion_tokens", request.getMaxTokens());
}
if(!request.getTools().isEmpty())
{
List<Map<String, Object>> tools = new ArrayList<>();
for(ModelToolDefinition tool : request.getTools())
{
Map<String, Object> function = new LinkedHashMap<>();
function.put("name", tool.getName());
function.put("description", tool.getDescription());
function.put("parameters", tool.getInputSchema());
////////////////////////////////////////////////////////////////////
// OpenAI's strict mode needs every property required; the //
// agent validates inputs itself, so strict is off by default. //
////////////////////////////////////////////////////////////////////
if(Boolean.TRUE.equals(strictTools))
{
function.put("strict", true);
}
tools.add(Map.of("type", "function", "function", function));
}
body.put("tools", tools);
body.put("parallel_tool_calls", Boolean.TRUE.equals(request.getParallelToolCalls()));
}
return (body);
}
/*******************************************************************************
** Chat Completions response body -> ModelResponse.
*******************************************************************************/
@SuppressWarnings("unchecked")
ModelResponse fromResponseBody(String json) throws Exception
{
Map<String, Object> root = JsonUtils.toObject(json, new TypeReference<>() {});
List<Map<String, Object>> choices = (List<Map<String, Object>>) root.get("choices");
if(choices == null || choices.isEmpty())
{
throw (new IllegalStateException("no choices"));
}
Map<String, Object> choice = choices.get(0);
Map<String, Object> message = (Map<String, Object>) choice.get("message");
ModelResponse response = new ModelResponse();
Object content = message == null ? null : message.get("content");
if(content instanceof String text && !text.isEmpty())
{
response.getParts().add(ModelPart.text(text));
}
List<Map<String, Object>> toolCalls = message == null ? null : (List<Map<String, Object>>) message.get("tool_calls");
if(toolCalls != null)
{
for(Map<String, Object> call : toolCalls)
{
Map<String, Object> function = (Map<String, Object>) call.get("function");
String arguments = function == null ? null : (String) function.get("arguments");
Map<String, Object> input = StringUtils.hasContent(arguments) ? JsonUtils.toObject(arguments, new TypeReference<Map<String, Object>>() {}) : new LinkedHashMap<>();
response.getParts().add(ModelPart.toolUse((String) call.get("id"), function == null ? null : (String) function.get("name"), input));
}
}
String finish = (String) choice.get("finish_reason");
response.setStopReason(switch(finish == null ? "" : finish)
{
case "tool_calls", "function_call" -> ModelResponse.STOP_TOOL_USE;
case "length" -> ModelResponse.STOP_MAX_TOKENS;
case "content_filter" -> ModelResponse.STOP_REFUSAL;
default -> response.getToolUses().isEmpty() ? ModelResponse.STOP_END_TURN : ModelResponse.STOP_TOOL_USE;
});
Map<String, Object> usage = (Map<String, Object>) root.get("usage");
if(usage != null)
{
long prompt = number(usage.get("prompt_tokens"));
long cached = 0;
if(usage.get("prompt_tokens_details") instanceof Map<?, ?> details)
{
cached = number(details.get("cached_tokens"));
}
response.setUsage(new ModelUsage().withInputTokens(Math.max(0, prompt - cached)).withCacheReadTokens(cached).withOutputTokens(number(usage.get("completion_tokens"))));
}
response.setRequestId((String) root.get("id"));
return (response);
}
/*******************************************************************************
** An HTTP error -> a provider exception (status and error type only).
*******************************************************************************/
@SuppressWarnings("unchecked")
static ModelProviderException toException(int status, String body)
{
String type = null;
try
{
Map<String, Object> root = JsonUtils.toObject(body, new TypeReference<>() {});
Object error = root.get("error");
if(error instanceof Map<?, ?> map)
{
type = map.get("type") instanceof String s ? s : (map.get("code") instanceof String c ? c : null);
}
}
catch(Exception e)
{
// not JSON: the status alone
}
ModelProviderException.Kind kind = switch(status)
{
case 401, 403 -> ModelProviderException.Kind.AUTHENTICATION;
case 429 -> ModelProviderException.Kind.RATE_LIMITED;
case 503, 529 -> ModelProviderException.Kind.OVERLOADED;
case 400, 404, 413, 422 -> ModelProviderException.Kind.INVALID_REQUEST;
case 408 -> ModelProviderException.Kind.TIMEOUT;
default -> status >= 500 ? ModelProviderException.Kind.SERVER : ModelProviderException.Kind.OTHER;
};
return (new ModelProviderException(kind, status, "HTTP " + status + (type == null ? "" : " (" + type + ")")));
}
/*******************************************************************************
**
*******************************************************************************/
private static long number(Object value)
{
return (value instanceof Number n ? n.longValue() : 0);
}
/*******************************************************************************
** Never includes the key.
*******************************************************************************/
@Override
public String toString()
{
return ("OpenAiCompatibleModelProvider{name=" + name + ", baseUrl=" + baseUrl + ", apiKey=" + (apiKey == null ? "none" : "***") + "}");
}
/*******************************************************************************
** Builder.
*******************************************************************************/
public static class Builder
{
private String name = "openai";
private String baseUrl = "https://api.openai.com/v1";
private String apiKey;
private Duration requestTimeout = Duration.ofSeconds(120);
private Boolean strictTools = false;
/*******************************************************************************
** The provider's registration name.
*******************************************************************************/
public Builder name(String name)
{
this.name = name;
return (this);
}
/*******************************************************************************
** e.g. https://api.openai.com/v1 or http://localhost:11434/v1
*******************************************************************************/
public Builder baseUrl(String baseUrl)
{
this.baseUrl = baseUrl;
return (this);
}
/*******************************************************************************
** The key, or a reference such as "${env.OPENAI_API_KEY}".
*******************************************************************************/
public Builder apiKey(String apiKey)
{
this.apiKey = apiKey;
return (this);
}
/*******************************************************************************
**
*******************************************************************************/
public Builder requestTimeout(Duration requestTimeout)
{
this.requestTimeout = requestTimeout;
return (this);
}
/*******************************************************************************
**
*******************************************************************************/
public Builder strictTools(Boolean strictTools)
{
this.strictTools = strictTools;
return (this);
}
/*******************************************************************************
**
*******************************************************************************/
public OpenAiCompatibleModelProvider build()
{
if(!StringUtils.hasContent(name) || !StringUtils.hasContent(baseUrl))
{
throw (new IllegalArgumentException("name and baseUrl are required"));
}
return (new OpenAiCompatibleModelProvider(this));
}
}
}
@@ -20,23 +20,19 @@ package com.kingsrook.qbits.marketing.email;
import java.util.EnumSet;
import java.util.List;
import java.util.Set;
import com.kingsrook.qbits.email.EmailQBitConfig;
import com.kingsrook.qbits.email.EmailRuntime;
import com.kingsrook.qbits.email.metadata.EmailTables;
import com.kingsrook.qbits.email.outbox.EmailCategory;
import com.kingsrook.qbits.email.outbox.EmailRequest;
import com.kingsrook.qbits.email.outbox.EmailSendResult;
import com.kingsrook.qbits.email.outbox.EmailService;
import com.kingsrook.qbits.email.support.SystemWrites;
import com.kingsrook.qbits.email.api.EmailApi;
import com.kingsrook.qbits.email.api.EmailApiCategory;
import com.kingsrook.qbits.email.api.EmailApiMessage;
import com.kingsrook.qbits.email.api.EmailApiReadiness;
import com.kingsrook.qbits.email.api.EmailApiReadinessCheck;
import com.kingsrook.qbits.email.api.EmailApiRequest;
import com.kingsrook.qbits.email.api.EmailApiResult;
import com.kingsrook.qbits.marketing.model.MarketingExecution;
import com.kingsrook.qbits.marketing.model.enums.ChannelType;
import com.kingsrook.qbits.marketing.transport.MarketingTransport;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.utils.StringUtils;
import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
@@ -88,8 +84,8 @@ public class QbitEmailTransport implements MarketingTransport
{
try
{
Class.forName("com.kingsrook.qbits.email.outbox.EmailService");
return (EmailRuntime.config(qInstance) != null);
Class.forName("com.kingsrook.qbits.email.api.EmailApi");
return (EmailApi.isInstalled(qInstance));
}
catch(ClassNotFoundException | LinkageError e)
{
@@ -119,28 +115,30 @@ public class QbitEmailTransport implements MarketingTransport
@Override
public TransportStatus getStatus()
{
EmailQBitConfig emailConfig = isInstalled(QContext.getQInstance()) ? EmailRuntime.config() : null;
if(emailConfig == null)
if(!isInstalled(QContext.getQInstance()))
{
return (new TransportStatus(TransportStatus.State.NOT_CONFIGURED, "The email qbit (qbit-email) is not installed in this application."));
}
if(!StringUtils.hasContent(emailConfig.getPublicBaseUrl()))
{
return (new TransportStatus(TransportStatus.State.NOT_CONFIGURED, "Marketing email needs the email qbit's publicBaseUrl (the unsubscribe link base)."));
}
String key = null;
try
{
key = emailConfig.getSecretResolver().apply(UNSUBSCRIBE_KEY_NAME);
EmailApiReadiness readiness = EmailApi.readiness(QContext.getQInstance(), EmailApiCategory.MARKETING);
if(readiness.failed(EmailApiReadinessCheck.NOT_INSTALLED))
{
return (new TransportStatus(TransportStatus.State.NOT_CONFIGURED, "The email qbit (qbit-email) is not installed in this application."));
}
if(readiness.failed(EmailApiReadinessCheck.PUBLIC_BASE_URL))
{
return (new TransportStatus(TransportStatus.State.NOT_CONFIGURED, "Marketing email needs the email qbit's publicBaseUrl (the unsubscribe link base)."));
}
if(readiness.failed(EmailApiReadinessCheck.UNSUBSCRIBE_KEY))
{
return (new TransportStatus(TransportStatus.State.NOT_CONFIGURED, "Marketing email needs the " + UNSUBSCRIBE_KEY_NAME + " secret (it signs the unsubscribe links)."));
}
}
catch(Exception e)
catch(QException e)
{
LOG.warn("Could not resolve the unsubscribe key", logPair("error", e.getClass().getSimpleName()));
}
if(!StringUtils.hasContent(key))
{
return (new TransportStatus(TransportStatus.State.NOT_CONFIGURED, "Marketing email needs the " + UNSUBSCRIBE_KEY_NAME + " secret (it signs the unsubscribe links)."));
LOG.warn("Could not check email qbit readiness", logPair("error", e.getClass().getSimpleName()));
return (new TransportStatus(TransportStatus.State.CONFIGURED, "Could not check the email qbit's configuration; retry when its configuration can be read."));
}
return (new TransportStatus(TransportStatus.State.OPERATIONAL, "Queues through the email qbit's outbox with its unsubscribe link and suppression list."));
}
@@ -183,7 +181,7 @@ public class QbitEmailTransport implements MarketingTransport
try
{
EmailSendResult result = EmailService.send(emailRequest(request, recipient));
EmailApiResult result = EmailApi.send(emailRequest(request, recipient));
switch(result.status())
{
case QUEUED -> queued++;
@@ -227,8 +225,8 @@ public class QbitEmailTransport implements MarketingTransport
{
try
{
String prefix = "marketing:" + stripPrefix(idempotencyKey) + ":";
List<QRecord> rows = SystemWrites.query(EmailTables.MESSAGE, new QQueryFilter(new QFilterCriteria("idempotencyKey", QCriteriaOperator.STARTS_WITH, prefix)).withLimit(1));
String prefix = "marketing:" + stripPrefix(idempotencyKey) + ":";
List<EmailApiMessage> rows = EmailApi.findMessagesByIdempotencyKeyPrefix(prefix, 1);
if(rows.isEmpty())
{
return (new TransportLookup(TransportLookup.State.NOT_FOUND, null));
@@ -247,7 +245,7 @@ public class QbitEmailTransport implements MarketingTransport
/*******************************************************************************
** One MARKETING message to one recipient.
*******************************************************************************/
private EmailRequest emailRequest(TransportRequest request, Recipient recipient)
private EmailApiRequest emailRequest(TransportRequest request, Recipient recipient)
{
String body = request.body() == null ? "" : request.body();
if(StringUtils.hasContent(request.destinationUrl()) && request.destinationUrl().matches("(?i)https?://\\S+"))
@@ -256,11 +254,11 @@ public class QbitEmailTransport implements MarketingTransport
body = body + "\n\n[" + label + "](" + request.destinationUrl() + ")";
}
return (new EmailRequest()
return (new EmailApiRequest()
.withTo(recipient.address())
.withSubject(request.title() == null ? "" : request.title().replaceAll("[\\r\\n]+", " ").trim())
.withMarkdownBody(body)
.withCategory(EmailCategory.MARKETING)
.withCategory(EmailApiCategory.MARKETING)
.withIdempotencyKey(baseKey(request) + ":" + recipient.key())
.withLinkedRecord(MarketingExecution.TABLE_NAME, request.executionId())
.withSender("marketing", "execution"));
@@ -72,6 +72,9 @@ import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizers;
import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.ChildRecordListRenderer;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.instances.QInstanceEnricher;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
@@ -224,6 +227,8 @@ public class MarketingMetaDataBuilder
tables.put(spec.name(), buildTable(spec));
}
addContingentPossibleValues();
////////////////////////////////////////////////////////////////////////
// relations: each parent shows its children as lists (owner rule: //
// sub-tables are relations and child lists on their parents) //
@@ -264,6 +269,59 @@ public class MarketingMetaDataBuilder
/*******************************************************************************
** Scope editable foreign keys by product. Content's product is read-only;
** system-written history tables have no editable record forms.
*******************************************************************************/
private void addContingentPossibleValues()
{
// Offer only audiences belonging to the selected product.
tables.get(MarketingCampaign.TABLE_NAME).getField("audienceId").withPossibleValueSourceFilter(
new QQueryFilter(new QFilterCriteria("productId", QCriteriaOperator.EQUALS, "${input.productId}")));
// Offer only evidence sources belonging to the selected product.
tables.get(MarketingClaim.TABLE_NAME).getField("sourceId").withPossibleValueSourceFilter(
new QQueryFilter(new QFilterCriteria("productId", QCriteriaOperator.EQUALS, "${input.productId}")));
// Offer only campaigns belonging to the selected product.
tables.get(MarketingFinding.TABLE_NAME).getField("campaignId").withPossibleValueSourceFilter(
new QQueryFilter(new QFilterCriteria("productId", QCriteriaOperator.EQUALS, "${input.productId}")));
// Offer only content belonging to the selected product.
tables.get(MarketingFinding.TABLE_NAME).getField("contentId").withPossibleValueSourceFilter(
new QQueryFilter(new QFilterCriteria("productId", QCriteriaOperator.EQUALS, "${input.productId}")));
// Offer only claims belonging to the selected product.
tables.get(MarketingFinding.TABLE_NAME).getField("claimId").withPossibleValueSourceFilter(
new QQueryFilter(new QFilterCriteria("productId", QCriteriaOperator.EQUALS, "${input.productId}")));
// Offer only evidence sources belonging to the selected product.
tables.get(MarketingFinding.TABLE_NAME).getField("sourceId").withPossibleValueSourceFilter(
new QQueryFilter(new QFilterCriteria("productId", QCriteriaOperator.EQUALS, "${input.productId}")));
// Offer only agent runs belonging to the selected product.
tables.get(MarketingFinding.TABLE_NAME).getField("agentRunId").withPossibleValueSourceFilter(
new QQueryFilter(new QFilterCriteria("productId", QCriteriaOperator.EQUALS, "${input.productId}")));
// Offer only campaigns belonging to the selected product.
tables.get(MarketingMetric.TABLE_NAME).getField("campaignId").withPossibleValueSourceFilter(
new QQueryFilter(new QFilterCriteria("productId", QCriteriaOperator.EQUALS, "${input.productId}")));
// Offer only campaigns belonging to the selected product.
tables.get(MarketingOutcomeEvent.TABLE_NAME).getField("campaignId").withPossibleValueSourceFilter(
new QQueryFilter(new QFilterCriteria("productId", QCriteriaOperator.EQUALS, "${input.productId}")));
// Offer only content belonging to the selected product.
tables.get(MarketingOutcomeEvent.TABLE_NAME).getField("contentId").withPossibleValueSourceFilter(
new QQueryFilter(new QFilterCriteria("productId", QCriteriaOperator.EQUALS, "${input.productId}")));
// Offer only executions belonging to the selected product.
tables.get(MarketingOutcomeEvent.TABLE_NAME).getField("executionId").withPossibleValueSourceFilter(
new QQueryFilter(new QFilterCriteria("productId", QCriteriaOperator.EQUALS, "${input.productId}")));
}
/*******************************************************************************
** One table.
*******************************************************************************/
@@ -21,7 +21,6 @@ import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
import com.kingsrook.qbits.assistant.provider.ModelUsage;
import com.kingsrook.qbits.assistant.provider.fake.ScriptedModelProvider;
import com.kingsrook.qbits.marketing.agent.MarketingAgentService;
import com.kingsrook.qbits.marketing.agent.tools.MarketingTools;
import com.kingsrook.qbits.marketing.approval.ApprovalService;
@@ -0,0 +1,143 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.marketing;
import java.io.Serializable;
import java.util.Map;
import com.kingsrook.qbits.marketing.metadata.MarketingMetaDataBuilder;
import com.kingsrook.qbits.marketing.model.MarketingAudience;
import com.kingsrook.qqq.backend.core.actions.values.SearchPossibleValueSourceAction;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.values.SearchPossibleValueSourceInput;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.PossibleValueSearchFilterUseCase;
import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValue;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import com.kingsrook.qqq.backend.core.utils.collections.MapBuilder;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** Product-contingent metadata and real memory-backend possible-value searches.
*******************************************************************************/
class ContingentPossibleValuesTest extends BaseMarketingTest
{
/*******************************************************************************
** Every declared dependency uses an editable parent and child and a real
** product column on the referenced table.
*******************************************************************************/
@ParameterizedTest
@CsvSource({
"marketing_campaign, audienceId, marketing_audience",
"marketing_claim, sourceId, marketing_evidence_source",
"marketing_finding, campaignId, marketing_campaign",
"marketing_finding, contentId, marketing_content",
"marketing_finding, claimId, marketing_claim",
"marketing_finding, sourceId, marketing_evidence_source",
"marketing_finding, agentRunId, marketing_agent_run",
"marketing_metric, campaignId, marketing_campaign",
"marketing_outcome_event, campaignId, marketing_campaign",
"marketing_outcome_event, contentId, marketing_content",
"marketing_outcome_event, executionId, marketing_execution"
})
void testMetadata(String tableName, String fieldName, String targetName)
{
QTableMetaData table = qInstance.getTable(tableName);
QFieldMetaData field = table.getField(fieldName);
assertThat(MarketingMetaDataBuilder.SYSTEM_WRITTEN_TABLE_NAMES).doesNotContain(tableName);
assertThat(table.getField("productId").getIsEditable()).isTrue();
assertThat(field.getIsEditable()).isTrue();
assertThat(field.getPossibleValueSourceName()).isEqualTo(targetName);
assertThat(qInstance.getTable(targetName).getField("productId")).isNotNull();
assertThat(field.getPossibleValueSourceFilter()).isEqualTo(
new QQueryFilter(new QFilterCriteria("productId", QCriteriaOperator.EQUALS, "${input.productId}")));
}
/*******************************************************************************
** Read-only dependencies stay unfiltered, and the inventory is exhaustive.
*******************************************************************************/
@Test
void testSkippedFields()
{
int filterCount = 0;
for(QTableMetaData table : qInstance.getTables().values())
{
for(QFieldMetaData field : table.getFields().values())
{
if(MarketingMetaDataBuilder.SYSTEM_WRITTEN_TABLE_NAMES.contains(table.getName()) || "marketing_content".equals(table.getName()))
{
assertThat(field.getPossibleValueSourceFilter()).as(table.getName() + "." + field.getName()).isNull();
}
if(field.getPossibleValueSourceFilter() != null)
{
filterCount++;
}
}
}
assertThat(filterCount).isEqualTo(11);
}
/*******************************************************************************
** Three different target tables: selected products isolate choices, while
** missing products give no form choices and all filter-builder choices.
*******************************************************************************/
@ParameterizedTest
@CsvSource({"marketing_campaign, audienceId", "marketing_claim, sourceId", "marketing_metric, campaignId"})
void testSearch(String tableName, String fieldName) throws Exception
{
Integer firstProduct = product("first");
Integer secondProduct = product("second");
campaign(firstProduct);
campaign(secondProduct);
MarketingStore.insert(new MarketingAudience().withProductId(firstProduct).withName("First audience"));
MarketingStore.insert(new MarketingAudience().withProductId(secondProduct).withName("Second audience"));
QFieldMetaData field = qInstance.getTable(tableName).getField(fieldName);
String targetName = field.getPossibleValueSourceName();
Serializable firstId = MarketingStore.queryBy(targetName, "productId", firstProduct).get(0).getValue("id");
Serializable secondId = MarketingStore.queryBy(targetName, "productId", secondProduct).get(0).getValue("id");
assertThat(search(field, Map.of("productId", firstProduct), PossibleValueSearchFilterUseCase.FORM)).containsExactly(firstId);
assertThat(search(field, Map.of("productId", secondProduct), PossibleValueSearchFilterUseCase.FORM)).containsExactly(secondId);
assertThat(search(field, Map.of(), PossibleValueSearchFilterUseCase.FORM)).isEmpty();
assertThat(search(field, Map.of(), PossibleValueSearchFilterUseCase.FILTER)).containsExactlyInAnyOrder(firstId, secondId);
assertThat(field.getPossibleValueSourceFilter().getCriteria().get(0).getValues()).containsExactly("${input.productId}");
}
/*******************************************************************************
** Interpret a clone as the UI does, then search the actual table PVS.
*******************************************************************************/
private java.util.List<Serializable> search(QFieldMetaData field, Map<String, Serializable> input, PossibleValueSearchFilterUseCase useCase) throws Exception
{
QQueryFilter filter = field.getPossibleValueSourceFilter().clone();
filter.interpretValues(MapBuilder.of("input", input), useCase);
return (new SearchPossibleValueSourceAction().execute(new SearchPossibleValueSourceInput()
.withPossibleValueSourceName(field.getPossibleValueSourceName()).withDefaultQueryFilter(filter))
.getResults().stream().map(QPossibleValue::getId).map(id -> (Serializable) id).toList());
}
}
@@ -22,7 +22,6 @@ import java.time.Instant;
import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.assistant.provider.ModelUsage;
import com.kingsrook.qbits.assistant.provider.fake.ScriptedModelProvider;
import com.kingsrook.qbits.marketing.agent.MarketingAgentService;
import com.kingsrook.qbits.marketing.agent.tools.MarketingTools;
import com.kingsrook.qbits.marketing.measurement.MeasurementReport;
@@ -21,8 +21,10 @@ import java.util.List;
import java.util.Set;
import java.util.stream.Collectors;
import com.kingsrook.qbits.marketing.agent.KernelToolRegistration;
import com.kingsrook.qbits.marketing.agent.tools.MarketingTools;
import com.kingsrook.qbits.marketing.metadata.MarketingMetaDataBuilder;
import com.kingsrook.qbits.marketing.security.MarketingPermissions;
import com.kingsrook.qqq.backend.core.actions.ai.AiToolRegistry;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppChildMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
@@ -111,11 +113,16 @@ class MarketingQBitProducerTest extends BaseMarketingTest
/*******************************************************************************
** Whether or not the kernel hook is available (absent on QQQ 4.1.0-RC.4), registering no tools registers nothing.
** RC.6 registers the read tools, excludes writes and tolerates repeat registration.
*******************************************************************************/
@Test
void testKernelHookRegistersNothingWithoutTools()
void testKernelHookAvailableOnRc6()
{
assertThat(KernelToolRegistration.isAvailable()).isTrue();
assertThat(AiToolRegistry.of(qInstance).getTools()).containsOnlyKeys(
MarketingTools.GET_CAMPAIGN_BRIEF, MarketingTools.LIST_PRODUCT_CLAIMS, MarketingTools.LIST_CAMPAIGN_CONTENT,
MarketingTools.GET_CAMPAIGN_RESULTS, MarketingTools.LIST_CHANNELS, MarketingTools.LIST_FINDINGS);
assertThat(KernelToolRegistration.register(qInstance, MarketingTools.all())).isZero();
assertThat(KernelToolRegistration.register(qInstance, List.of())).isZero();
}
@@ -0,0 +1,218 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.marketing;
import java.util.ArrayDeque;
import java.util.ArrayList;
import java.util.Deque;
import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.assistant.provider.AssistantModelProvider;
import com.kingsrook.qbits.assistant.provider.CompletableStreamHandle;
import com.kingsrook.qbits.assistant.provider.ModelCapabilities;
import com.kingsrook.qbits.assistant.provider.ModelEventListener;
import com.kingsrook.qbits.assistant.provider.ModelPart;
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
import com.kingsrook.qbits.assistant.provider.ModelRequest;
import com.kingsrook.qbits.assistant.provider.ModelResponse;
import com.kingsrook.qbits.assistant.provider.ModelUsage;
import com.kingsrook.qbits.assistant.provider.StreamHandle;
import com.kingsrook.qqq.backend.core.utils.JsonUtils;
/*******************************************************************************
** Synchronous scripted responses and request snapshots for marketing tests.
*******************************************************************************/
class ScriptedModelProvider implements AssistantModelProvider
{
private final String name;
private final Deque<Step> steps = new ArrayDeque<>();
private final List<ModelRequest> requests = new ArrayList<>();
private final List<String> requestJson = new ArrayList<>();
private ModelUsage usagePerCall = new ModelUsage();
/*******************************************************************************
** Register under the name selected by the test configuration.
*******************************************************************************/
ScriptedModelProvider(String name)
{
this.name = name;
}
/*******************************************************************************
** Queue a complete answer.
*******************************************************************************/
ScriptedModelProvider thenText(String text)
{
return (thenRespond(request -> new ModelResponse().withParts(List.of(ModelPart.text(text))).withStopReason(ModelResponse.STOP_END_TURN)));
}
/*******************************************************************************
** Queue a single tool call.
*******************************************************************************/
ScriptedModelProvider thenToolUse(String toolName, Map<String, Object> input)
{
return (thenToolUses(null, List.<Object[]>of(new Object[] { toolName, input })));
}
/*******************************************************************************
** Queue parallel tool calls using the test fixtures' name/input pairs.
*******************************************************************************/
@SuppressWarnings("unchecked")
ScriptedModelProvider thenToolUses(String preamble, List<Object[]> calls)
{
return (thenRespond(request ->
{
List<ModelPart> parts = new ArrayList<>();
if(preamble != null)
{
parts.add(ModelPart.text(preamble));
}
for(Object[] call : calls)
{
parts.add(ModelPart.toolUse("tool_" + requests.size() + "_" + parts.size(), (String) call[0], (Map<String, Object>) call[1]));
}
return (new ModelResponse().withParts(parts).withStopReason(ModelResponse.STOP_TOOL_USE));
}));
}
/*******************************************************************************
** Queue a provider failure, including usage already incurred.
*******************************************************************************/
ScriptedModelProvider thenError(ModelProviderException.Kind kind, String message)
{
return (thenRespond(request ->
{
throw (new ModelProviderException(kind, null, message).withPartialUsage(new ModelUsage().add(usagePerCall)));
}));
}
/*******************************************************************************
** Queue a response that can inspect the request or change test state.
*******************************************************************************/
ScriptedModelProvider thenRespond(Step step)
{
steps.add(step);
return (this);
}
/*******************************************************************************
** Set usage for budget and cost assertions.
*******************************************************************************/
ScriptedModelProvider withUsagePerCall(ModelUsage usage)
{
usagePerCall = usage;
return (this);
}
/*******************************************************************************
** Provider registration name.
*******************************************************************************/
@Override
public String getName()
{
return (name);
}
/*******************************************************************************
** Exercise the agent's prompt-cache hints.
*******************************************************************************/
@Override
public ModelCapabilities getCapabilities(String modelId)
{
return (new ModelCapabilities().withPromptCaching(true).withThinking(false));
}
/*******************************************************************************
** Record before responding, preserving each turn before the agent mutates it.
*******************************************************************************/
@Override
public StreamHandle stream(ModelRequest request, ModelEventListener listener)
{
requests.add(request.copy());
requestJson.add(JsonUtils.toJson(request));
CompletableStreamHandle handle = new CompletableStreamHandle();
try
{
ModelResponse response = steps.removeFirst().respond(request);
listener.onTextDelta(response.getText());
response.setUsage(new ModelUsage().add(usagePerCall));
response.setRequestId("request_" + requests.size());
handle.complete(response);
}
catch(ModelProviderException e)
{
handle.fail(e);
}
return (handle);
}
/*******************************************************************************
** Request copies for tool exposure assertions.
*******************************************************************************/
List<ModelRequest> getRequests()
{
return (requests);
}
/*******************************************************************************
** Serialized snapshots for data-boundary assertions.
*******************************************************************************/
List<String> getRequestJson()
{
return (requestJson);
}
/*******************************************************************************
** A scripted step may return a response or a checked provider failure.
*******************************************************************************/
@FunctionalInterface
interface Step
{
/*******************************************************************************
** Respond to one model request.
*******************************************************************************/
ModelResponse respond(ModelRequest request) throws ModelProviderException;
}
}
@@ -31,7 +31,11 @@ import com.kingsrook.qbits.assistant.provider.ModelProviderException;
import com.kingsrook.qbits.assistant.provider.ModelRequest;
import com.kingsrook.qbits.assistant.provider.ModelResponse;
import com.kingsrook.qbits.assistant.provider.ModelToolDefinition;
import com.kingsrook.qbits.assistant.provider.ProviderSettings;
import com.kingsrook.qbits.assistant.provider.SystemBlock;
import com.kingsrook.qbits.assistant.provider.openai.OpenAiCompatibleModelProvider;
import com.kingsrook.qbits.marketing.MarketingQBitConfig;
import com.kingsrook.qqq.backend.core.instances.QMetaDataVariableInterpreter;
import com.sun.net.httpserver.HttpServer;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
@@ -41,8 +45,8 @@ import static org.assertj.core.api.Assertions.assertThatThrownBy;
/*******************************************************************************
** The OpenAI-compatible provider against a local fixture server (no network,
** no real key): request mapping, tool calls, usage, error kinds, and the key
** The published OpenAI-compatible provider against a local fixture server
** with no real key: request mapping, tool calls, usage, error kinds, and the key
** never appearing in toString.
*******************************************************************************/
class OpenAiCompatibleModelProviderTest
@@ -94,8 +98,9 @@ class OpenAiCompatibleModelProviderTest
*******************************************************************************/
private OpenAiCompatibleModelProvider provider(String key)
{
return (OpenAiCompatibleModelProvider.builder().name("local").baseUrl("http://127.0.0.1:" + server.getAddress().getPort() + "/v1").apiKey(key)
.requestTimeout(Duration.ofSeconds(5)).build());
return (new OpenAiCompatibleModelProvider(new ProviderSettings().withKind("OPENAI_COMPATIBLE")
.withBaseUrl("http://127.0.0.1:" + server.getAddress().getPort() + "/v1")
.withApiKey(key == null ? null : new QMetaDataVariableInterpreter().interpret(key)).withRequestTimeoutSeconds(5)));
}
@@ -105,7 +110,7 @@ class OpenAiCompatibleModelProviderTest
*******************************************************************************/
private static ModelRequest request()
{
return (new ModelRequest().withModel("test-model").withMaxTokens(100)
return (new ModelRequest().withModel("test-model").withMaxTokens(100).withStreaming(false)
.withSystem(List.of(new SystemBlock("Be truthful.", null)))
.withTools(List.of(new ModelToolDefinition().withName("marketing_list_channels").withDescription("Channels")
.withInputSchema(Map.of("type", "object", "properties", Map.of()))))
@@ -129,7 +134,11 @@ class OpenAiCompatibleModelProviderTest
"usage":{"prompt_tokens":120,"completion_tokens":30,"prompt_tokens_details":{"cached_tokens":20}}}
""" });
ModelResponse response = provider("${env.MARKETING_TEST_KEY_THAT_IS_NOT_SET}").stream(request(), text -> {}).await(Duration.ofSeconds(5));
MarketingQBitConfig config = new MarketingQBitConfig()
.withModelProvider(provider("${env.MARKETING_TEST_KEY_THAT_IS_NOT_SET}"))
.withProviderName("openai-compatible").withModel("configured-model");
ModelResponse response = config.getModelProviders().get(config.getProviderName())
.stream(request().withModel(config.getModel()), text -> {}).await(Duration.ofSeconds(5));
assertThat(response.getStopReason()).isEqualTo(ModelResponse.STOP_TOOL_USE);
assertThat(response.getText()).isEqualTo("Checking.");
assertThat(response.getToolUses()).hasSize(1);
@@ -141,7 +150,7 @@ class OpenAiCompatibleModelProviderTest
String body = lastBody.get();
assertThat(body).contains("\"role\":\"system\"").contains("Be truthful.").contains("\"tool_calls\"").contains("\"role\":\"tool\"").contains("\"tool_call_id\":\"call_1\"")
.contains("\"max_completion_tokens\":100").contains("marketing_list_channels");
.contains("\"max_tokens\":100").contains("marketing_list_channels").contains("\"model\":\"configured-model\"");
assertThat(lastAuth.get()).as("an unset key reference means no Authorization header").isNull();
}
@@ -162,7 +171,8 @@ class OpenAiCompatibleModelProviderTest
.isInstanceOfSatisfying(ModelProviderException.class, e ->
{
assertThat(e.getKind()).isEqualTo(ModelProviderException.Kind.AUTHENTICATION);
assertThat(e.getMessage()).contains("401").contains("invalid_api_key").doesNotContain("sk-test");
assertThat(e.getHttpStatus()).isEqualTo(401);
assertThat(e.getMessage()).doesNotContain("sk-FAKE-TEST-KEY-000000");
});
assertThat(lastAuth.get()).isEqualTo("Bearer sk-FAKE-TEST-KEY-000000");
@@ -174,8 +184,9 @@ class OpenAiCompatibleModelProviderTest
assertThatThrownBy(() -> provider.stream(request(), text -> {}).await(Duration.ofSeconds(5)))
.isInstanceOfSatisfying(ModelProviderException.class, e -> assertThat(e.getKind()).isEqualTo(ModelProviderException.Kind.INVALID_REQUEST));
responses.add(new Object[] { 200, "{\"choices\":[{\"finish_reason\":\"content_filter\",\"message\":{\"content\":\"\"}}]}" });
assertThat(OpenAiCompatibleModelProvider.toException(503, "").getKind()).isEqualTo(ModelProviderException.Kind.OVERLOADED);
responses.add(new Object[] { 503, "" });
assertThatThrownBy(() -> provider.stream(request(), text -> {}).await(Duration.ofSeconds(5)))
.isInstanceOfSatisfying(ModelProviderException.class, e -> assertThat(e.getKind()).isEqualTo(ModelProviderException.Kind.OVERLOADED));
}
@@ -196,4 +207,29 @@ class OpenAiCompatibleModelProviderTest
assertThat(streamed).containsExactly("Done.");
}
/*******************************************************************************
** The agent's default streaming mode receives text and usage from SSE.
*******************************************************************************/
@Test
void testStreaming() throws Exception
{
responses.add(new Object[] { 200, """
data: {"choices":[{"index":0,"delta":{"content":"Done."},"finish_reason":null}]}
data: {"choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":12,"completion_tokens":3}}
data: [DONE]
""" });
List<String> streamed = new ArrayList<>();
ModelResponse response = provider(null).stream(request().withStreaming(true), streamed::add).await(Duration.ofSeconds(5));
assertThat(streamed).containsExactly("Done.");
assertThat(response.getText()).isEqualTo("Done.");
assertThat(response.getUsage().getInputTokens()).isEqualTo(12);
assertThat(response.getUsage().getOutputTokens()).isEqualTo(3);
assertThat(lastBody.get()).contains("\"stream\":true");
}
}
@@ -21,9 +21,9 @@ import java.util.ArrayList;
import java.util.List;
import com.kingsrook.qbits.email.EmailQBitConfig;
import com.kingsrook.qbits.email.EmailQBitProducer;
import com.kingsrook.qbits.email.metadata.EmailTables;
import com.kingsrook.qbits.email.permissions.EmailPermissions;
import com.kingsrook.qbits.email.support.SystemWrites;
import com.kingsrook.qbits.email.api.EmailApi;
import com.kingsrook.qbits.email.api.EmailApiCategory;
import com.kingsrook.qbits.email.api.EmailApiMessage;
import com.kingsrook.qbits.marketing.MarketingQBitConfig;
import com.kingsrook.qbits.marketing.MarketingQBitProducer;
import com.kingsrook.qbits.marketing.MarketingRuntime;
@@ -34,10 +34,13 @@ import com.kingsrook.qbits.marketing.transport.MarketingTransport.Recipient;
import com.kingsrook.qbits.marketing.transport.MarketingTransport.TransportRequest;
import com.kingsrook.qbits.marketing.transport.MarketingTransport.TransportResult;
import com.kingsrook.qbits.marketing.transport.QqqMessagingEmailTransport;
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.instances.QInstanceValidator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType;
import com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData;
@@ -90,7 +93,7 @@ class QbitEmailTransportTest
new QInstanceValidator().validate(qInstance);
List<String> permissions = new ArrayList<>(MarketingPermissions.ALL);
permissions.addAll(EmailPermissions.ALL);
permissions.addAll(List.of("email.view", "email.send", "email.manage", "email.admin", "email.hasAccess"));
QContext.init(qInstance, new QSession().withPermissions(permissions));
return (qInstance);
}
@@ -103,7 +106,7 @@ class QbitEmailTransportTest
private void provider() throws QException
{
brand();
SystemWrites.insertOne(EmailTables.PROVIDER, new QRecord().withValue("name", "relay").withValue("label", "Relay").withValue("kind", "SMTP").withValue("host", "127.0.0.1")
insert("email_provider", new QRecord().withValue("name", "relay").withValue("label", "Relay").withValue("kind", "SMTP").withValue("host", "127.0.0.1")
.withValue("port", 2525).withValue("security", "NONE").withValue("fromAddress", "sender@example.com").withValue("enabled", true).withValue("isDefault", true));
}
@@ -114,12 +117,23 @@ class QbitEmailTransportTest
*******************************************************************************/
private void brand() throws QException
{
SystemWrites.insertOne(EmailTables.BRAND, new QRecord().withValue("name", "default").withValue("label", "Example").withValue("isDefault", true)
insert("email_brand", new QRecord().withValue("name", "default").withValue("label", "Example").withValue("isDefault", true)
.withValue("postalAddress", "1 Main St, Springfield"));
}
/*******************************************************************************
** Configure the email qbit's editable tables through ordinary QQQ actions.
*******************************************************************************/
private void insert(String table, QRecord record) throws QException
{
var output = new InsertAction().execute(new InsertInput(table).withRecords(List.of(record)));
assertThat(output.getRecords()).hasSize(1).allSatisfy(row -> assertThat(row.getErrors()).isNullOrEmpty());
}
/*******************************************************************************
**
*******************************************************************************/
@@ -151,22 +165,32 @@ class QbitEmailTransportTest
assertThat(result.outcome()).as(result.message()).isEqualTo(TransportResult.Outcome.ACCEPTED);
assertThat(result.providerStatus()).isEqualTo("queued");
List<QRecord> rows = SystemWrites.query(EmailTables.MESSAGE, new QQueryFilter());
List<EmailApiMessage> rows = EmailApi.findMessagesByIdempotencyKeyPrefix("marketing:3:4:", 10);
assertThat(rows).hasSize(2);
assertThat(rows).extracting(r -> r.getValueString("idempotencyKey")).containsExactlyInAnyOrder("marketing:3:4:crm-contact:1", "marketing:3:4:crm-contact:2");
assertThat(rows).extracting(EmailApiMessage::idempotencyKey).containsExactlyInAnyOrder("marketing:3:4:crm-contact:1", "marketing:3:4:crm-contact:2");
assertThat(rows).allSatisfy(r ->
{
assertThat(r.getValueString("category")).isEqualTo("MARKETING");
assertThat(r.category()).isEqualTo(EmailApiCategory.MARKETING);
assertThat(r.linkedTableName()).isEqualTo("marketing_execution");
assertThat(r.linkedRecordId()).isEqualTo("7");
assertThat(r.templateName()).isNull();
assertThat(r.status()).isEqualTo("QUEUED");
});
// The public API intentionally omits content; inspect it as an email admin.
List<QRecord> messages = new QueryAction().execute(new QueryInput("email_message")
.withShouldFetchHeavyFields(true).withShouldOmitHiddenFields(false)).getRecords();
assertThat(messages).hasSize(2).allSatisfy(r ->
{
assertThat(r.getValueString("subject")).isEqualTo("Big news");
assertThat(r.getValueString("linkedTableName")).isEqualTo("marketing_execution");
assertThat(r.getValueString("linkedRecordId")).isEqualTo("7");
assertThat(r.getValueString("senderQbit")).isEqualTo("marketing");
assertThat(r.getValueString("senderProcess")).isEqualTo("execution");
assertThat(r.getValueString("htmlBody")).contains("https://example.org/try").contains("nsubscribe");
assertThat(r.getValueString("toAddresses")).doesNotContain(",");
});
assertThat(transport.send(request).outcome()).isEqualTo(TransportResult.Outcome.DUPLICATE);
assertThat(SystemWrites.query(EmailTables.MESSAGE, new QQueryFilter())).hasSize(2);
assertThat(EmailApi.findMessagesByIdempotencyKeyPrefix("marketing:3:4:", 10)).hasSize(2);
assertThat(transport.lookup("marketing-execution:3:4").state()).isEqualTo(MarketingTransport.TransportLookup.State.FOUND_ACCEPTED);
assertThat(transport.lookup("marketing-execution:9:9").state()).isEqualTo(MarketingTransport.TransportLookup.State.NOT_FOUND);
@@ -183,7 +207,7 @@ class QbitEmailTransportTest
{
instance(true, "https://app.example.org", "k", new MarketingQBitConfig());
provider();
SystemWrites.insertOne(EmailTables.SUPPRESSION, new QRecord().withValue("address", "gone@example.org").withValue("reason", "UNSUBSCRIBED").withValue("scope", "MARKETING"));
insert("email_suppression", new QRecord().withValue("address", "gone@example.org").withValue("reason", "UNSUBSCRIBED").withValue("scope", "MARKETING"));
MarketingTransport transport = new QbitEmailTransport();
TransportResult result = transport.send(request("marketing-execution:1:1", new Recipient("crm-contact:9", "gone@example.org")));
@@ -216,6 +240,7 @@ class QbitEmailTransportTest
assertThat(new QbitEmailTransport().getStatus().detail()).contains("publicBaseUrl");
instance(true, "https://app.example.org", "k", new MarketingQBitConfig());
assertThat(transport.getStatus().state()).isEqualTo(MarketingTransport.TransportStatus.State.OPERATIONAL);
TransportResult refused = new QbitEmailTransport().send(request("marketing-execution:5:5", new Recipient("a", "a@example.org")));
assertThat(refused.outcome()).isEqualTo(TransportResult.Outcome.REJECTED);
assertThat(refused.message()).contains("refused the message");
@@ -227,6 +252,25 @@ class QbitEmailTransportTest
/*******************************************************************************
** A missing postal address is a send-time refusal, not NOT_CONFIGURED.
*******************************************************************************/
@Test
void testMissingBrandIsRefusedAtSend() throws Exception
{
instance(true, "https://app.example.org", "k", new MarketingQBitConfig());
insert("email_provider", new QRecord().withValue("name", "relay").withValue("label", "Relay").withValue("kind", "SMTP").withValue("host", "127.0.0.1")
.withValue("port", 2525).withValue("security", "NONE").withValue("fromAddress", "sender@example.com").withValue("enabled", true).withValue("isDefault", true));
QbitEmailTransport transport = new QbitEmailTransport();
assertThat(transport.getStatus().state()).isEqualTo(MarketingTransport.TransportStatus.State.OPERATIONAL);
TransportResult result = transport.send(request("marketing-execution:8:8", new Recipient("a", "a@example.org")));
assertThat(result.outcome()).isEqualTo(TransportResult.Outcome.REJECTED);
assertThat(result.message()).contains("postal address");
}
/*******************************************************************************
** Without the email qbit in the instance the name does not resolve to the
** transport, and a host-registered transport always wins.