Compare commits

...
Author SHA1 Message Date
james.maesandClaude Opus 5.5 1a05444e61 feat(agent): count agent concurrency and model-call rate in the database (#4)
maxConcurrentAgentRuns and maxModelCallsPerMinute were per-JVM, in memory.
Preflight now fails stale RUNNING runs (no heartbeat for maxRunSeconds
plus 60s) and counts RUNNING runs across all products; the per-minute
limit counts model-call steps in the last minute and waits instead of
failing. Runs record heartbeatAt (new column) on each model call. The
policy's maxConcurrentAgentRuns override is now honoured.

The count-then-start window remains racy; the ConditionalUpdateAction
claim from qqq#1178 (QQQ 4.1.0-RC.7) closes it in a follow-up.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 11:46:46 -05:00
james.maesandClaude Opus 5.5 0088e3b77b build: assistant provider SPI RC.12 and parent 2.1.2, no allow-lists
Move to qbit-assistant-provider-api 0.1.0-RC.12, whose types now live in
com.kingsrook.qbits.assistant.spi (ModelPricing included), and to
qbit-build-parent 2.1.2, which allows that package by default. Drop the
per-repository guardrail allow-lists. qbit-assistant-provider-openai is
only used by tests, so it becomes test scope; hosts that register it add
it themselves.

Fix the stale README status (RCs are on Central; hosted in Voyage#113).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 11:35:23 -05:00
james.maes bd54add778 Merge pull request #14 from QRun-IO/feature/orb-0.6.9
ci: qqq-orb 0.6.9 (refresh SNAPSHOT dependencies)
2026-10-14 10:57:08 -05:00
james.maes e261400b87 Merge pull request #15 from QRun-IO/feature/email-api
refactor(email): use qbit-email's public api package (fixes #13)
2026-10-14 10:34:09 -05:00
james.maesandClaude Opus 5.5 78ad854f41 refactor(email): use qbit-email's public api package (fixes #13)
QbitEmailTransport and MarketingRuntime use com.kingsrook.qbits.email.api
from qbit-email 0.1.0-RC.3; getStatus() keeps its mapping through
EmailApiReadiness.failedChecks (NOT_CONFIGURED only for the base URL and
unsubscribe key; a missing provider or brand address is still REJECTED
at send). qbit-build-parent 2.1.1 allows qbit-email's api by default, so
the temporary email allowances are gone; only the existing assistant
provider SPI allowances remain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 10:30:56 -05:00
james.maesandClaude Opus 5.5 c0a6a00fa2 ci: qqq-orb 0.6.9 (refresh SNAPSHOT dependencies)
Moves qqq-orb@0.6.8 to qqq-orb@0.6.9, which passes --update-snapshots so cached SNAPSHOT dependencies are re-checked on every build (QRun-IO/qqq-orb#22).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 10:07:22 -05:00
james.maes 62e12e9ebe Merge pull request #12 from QRun-IO/feature/arch-provider-api
refactor(providers): depend only on qbit-assistant-provider-api (#9)
2026-10-14 08:27:24 -05:00
james.maesandClaude Opus 5.5 e790a1091f Merge origin/develop into feature/arch-provider-api
Brings in the qbit-email transport (#11). Conflicts are resolved in
favour of the RC.6/provider-api side, keeping qbit-email.

qbit-email is a new cross-qbit implementation dependency that the
enforced G1 guardrail rejects. It is allow-listed explicitly and
temporarily until qbit-email publishes an API (#13).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 08:18:27 -05:00
james.maes 1c5bada863 Merge pull request #11 from QRun-IO/feature/email-qbit
feat(email): QbitEmailTransport for MARKETING sends through qbit-email
2026-10-14 08:10:59 -05:00
james.maesandClaude Opus 5.5 3ec8776c2a refactor(providers): depend only on qbit-assistant-provider-api and provider-openai
Marketing compiled against qbit-assistant internals, and installing it
pulled in the whole assistant runtime. It now depends on
qbit-assistant-provider-api (the provider SPI and ModelPricing) and on the
published qbit-assistant-provider-openai, both 0.1.0-RC.11. The local
OpenAiCompatibleModelProvider is gone, and tests use a local scripted
double, so the full assistant is not needed even in test scope.

Hosts register the published provider through ProviderSettings. Its
provider name is "openai-compatible", so Marketing Policy provider
overrides must use that name.

Also move to QQQ 4.1.0-RC.6 and qbit-build-parent 2.1.0 with the
guardrails enforced. The provider-api packages and provider-openai are
explicitly allow-listed.

Closes #9 (architecture review QRun-IO/qqq#1150, finding F5).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 08:00:04 -05:00
james.maesandClaude Opus 5.5 b795e36c73 build: qbit-email 0.1.0-RC.1 from Maven Central
Moves from 0.1.0-SNAPSHOT to the first release candidate. Test schemas
gain the RC's email_provider_sender_address table and the message
from_address_id column where they copy the qbit's DDL.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 07:59:20 -05:00
james.maes 4bd4d0f482 Merge pull request #10 from QRun-IO/feature/contingent-possible-values
feat: contingent possible values (product-scoped references)
2026-10-14 07:09:09 -05:00
james.maesandClaude Opus 5.5 6ef32bcefe feat(email): QbitEmailTransport for MARKETING sends through qbit-email
Phase (d) of QRun-IO/qbit-email docs/DESIGN.md section 11. qbit-email is an
optional dependency (0.1.0-SNAPSHOT until its first RC) used through one
adapter class behind a classpath and instance guard; without it the
existing behaviour is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 23:12:44 -05:00
23 changed files with 1290 additions and 669 deletions

No files matched your search

+1 -1
View File
@@ -1,7 +1,7 @@
version: 2.1
orbs:
qqq-orb: kingsrook/qqq-orb@0.6.8
qqq-orb: kingsrook/qqq-orb@0.6.9
commands:
configure-package-install:
+2 -1
View File
@@ -46,7 +46,8 @@ Services: `approval.ApprovalService`, `execution.ExecutionService`, `measurement
## Dependencies
- Java 21.
- Parent `com.kingsrook:qbit-build-parent:2.0.0` (qqq 4.0.0). The default-active `qqq-snapshot` profile imports `qqq-bom-pom:4.1.0-RC.4` ahead of it, as in qbit-quick-search (ADR-0007). Remove that activation when the parent moves to 4.1.
- Parent `com.kingsrook:qbit-build-parent:2.1.2` (qqq 4.0.0), with architecture enforcement and strict UI API routes enabled. The default-active `qqq-snapshot` profile imports `qqq-bom-pom:4.1.0-RC.6` ahead of it, as in qbit-quick-search (ADR-0007). Remove that activation when the parent moves to 4.1.
- Assistant provider SPI `qbit-assistant-provider-api` `0.1.0-RC.12` (package `com.kingsrook.qbits.assistant.spi`); the OpenAI-compatible provider is test scope; no full assistant dependency. No per-repository guardrail allow-lists.
- Tests: JUnit 5, AssertJ.
## Release
+14 -1
View File
@@ -4,6 +4,14 @@ All notable changes to this project are documented here. The format follows Keep
## [0.1.0] - Unreleased
### Changed
- Make the agent limits distributed (#4): `maxConcurrentAgentRuns` counts Running `marketing_agent_run` rows in the database (a new `heartbeatAt` column, updated on each model call, lets runs whose server stopped be marked Failed first), and `maxModelCallsPerMinute` counts model-call steps in the last 60 seconds and makes a run wait rather than fail. The in-memory semaphore and rate window are gone; the settings and Marketing Policy override keep their names and defaults (the policy `maxConcurrentAgentRuns` override now applies). Hosts add `heartbeat_at` to `marketing_agent_run`. The count-then-insert window is still racy; a conditional-update claim (qqq#1178) is a follow-up.
- Move to the assistant provider SPI `0.1.0-RC.12` (package `com.kingsrook.qbits.assistant.spi`) and `qbit-build-parent` `2.1.2`, and drop the per-repository guardrail allow-lists. `qbit-assistant-provider-openai` becomes a test dependency; hosts that register it add it themselves.
- Switch the optional qbit-email integration to the published `com.kingsrook.qbits.email.api` package in `0.1.0-RC.3` (fixes #13). Preserve MARKETING messages, idempotency keys, execution links and outcome mapping; use typed readiness failures so missing providers or brand postal addresses remain send-time rejections. Upgrade `qbit-build-parent` to `2.1.1` and remove all per-repository guardrail allow-lists.
- Resolve architecture finding F5 (#9): depend on `qbit-assistant-provider-api` and `qbit-assistant-provider-openai` `0.1.0-RC.11`, with no full assistant dependency, including in tests.
- Replace the local OpenAI-compatible implementation with the published provider and document `ProviderSettings`, environment-key resolution and provider-name migration. Agent tests use a local scripted double.
- Upgrade QQQ to `4.1.0-RC.6`; enable architecture enforcement and strict UI API routes.
### Added
- Product-contingent possible values for editable campaign, claim, finding, metric and outcome-event references.
- Repository scaffold: build, CI and the repository standard files.
@@ -35,7 +43,7 @@ All notable changes to this project are documented here. The format follows Keep
- Narrow typed tools (6 read, 3 write-proposal), run as the user with core permission enforcement, the untrusted-data envelope and redaction.
- Run and step records with usage and cost.
- Budgets, concurrency and rate limits, cancellation, and a plain failure reason when no provider is configured.
- **Model providers:** `OpenAiCompatibleModelProvider` (no SDK), plus qbit-assistant's Claude and scripted providers.
- **Model providers:** the published `OpenAiCompatibleModelProvider` (no SDK), host-supplied providers such as Claude, and a local scripted test double.
- **AI tool kernel:** the read tools are contributed through `AiToolRegistry` when QQQ has it (QRun-IO/qqq#1096).
- **UI:**
- The Marketing app; its home is the Marketing dashboard (Needs Attention, Campaigns, Agent Activity, Configuration Status).
@@ -43,3 +51,8 @@ All notable changes to this project are documented here. The format follows Keep
- Streamed-ETL record actions with standard summary lines.
- **Starter data:** QQQ, gclo and MarkdownViewr for Windows, verified 2026-10-12, with honest starter campaigns and the four channels.
- **Docs:** README, DESIGN, OPERATIONS and STARTER-DATA.
### Changed
- Move to the assistant provider SPI `0.1.0-RC.12` (package `com.kingsrook.qbits.assistant.spi`) and `qbit-build-parent` `2.1.2`, and drop the per-repository guardrail allow-lists. `qbit-assistant-provider-openai` becomes a test dependency; hosts that register it add it themselves.
- **Email:** new `QbitEmailTransport` sends campaign email through qbit-email (one MARKETING message per recipient, unsubscribe and suppression enforced there, idempotent, linked to the execution). It is selected automatically for the `qqq_email` channel when qbit-email is produced and the host registered no such transport; `QqqMessagingEmailTransport` remains the fallback. qbit-email is an optional dependency; QQQ moves to `4.1.0-RC.5`.
+1 -1
View File
@@ -46,7 +46,7 @@ Services: `approval.ApprovalService`, `execution.ExecutionService`, `measurement
## Dependencies
- Java 21.
- Parent `com.kingsrook:qbit-build-parent:2.0.0` (qqq 4.0.0). The default-active `qqq-snapshot` profile imports `qqq-bom-pom:4.1.0-RC.4` ahead of it, as in qbit-quick-search (ADR-0007). Remove that activation when the parent moves to 4.1.
- Parent `com.kingsrook:qbit-build-parent:2.0.0` (qqq 4.0.0). The default-active `qqq-snapshot` profile imports `qqq-bom-pom:4.1.0-RC.5` ahead of it, as in qbit-quick-search (ADR-0007). Remove that activation when the parent moves to 4.1.
- Tests: JUnit 5, AssertJ.
## Release
+36 -8
View File
@@ -5,9 +5,9 @@
> Marketing for QQQ applications: products and their verified evidence, campaigns, content with revision history, exact human approval, controlled execution, honest measurement, and a marketing agent that prepares work but never publishes.
**Status:** 0.1.0, not yet published.
- Built against QQQ `4.1.0-RC.4`.
- Hosted in QRun-IO/Voyage next to qbit-crm, on a local branch.
**Status:** 0.1.0 release candidates on Maven Central (`com.kingsrook.qbits:qbit-marketing`, latest `0.1.0-RC.2`); 0.1.0 GA is not yet released.
- Built against QQQ `4.1.0-RC.6` with `qbit-build-parent` `2.1.2` and the assistant provider SPI `0.1.0-RC.12`.
- Hosted in QRun-IO/Voyage next to qbit-crm (QRun-IO/Voyage#113).
- The real integrations are implemented but unconfigured. See [what is configured](#what-is-implemented-what-is-not-configured).
It lives inside the host QQQ application: its navigation, authentication, permissions, API, record screens and widgets. It is not a separate marketing app.
@@ -25,7 +25,7 @@ It lives inside the host QQQ application: its navigation, authentication, permis
| **Measurement** | Delivery, awareness, engagement, activation, repeat use and qualified demand are reported separately. Each metric shows raw events and unique actors. "Not measured" is never shown as 0. Test data is kept apart. |
| **CRM** | Uses the host CRM's own contacts, organizations, consent, suppression, lists, form submissions and activities through a narrow gateway. A qbit-crm adapter is included. Nothing is copied, and nobody becomes a lead without the campaign's explicit rule. |
| **Agent** | Plans campaigns, drafts and revises content, flags unsupported claims, stale sources and readiness gaps, and analyzes results to recommend the next experiment. It uses narrow typed tools, runs as the requesting user, sees data only as untrusted input, and sees no CRM personal data. Every run is recorded with its steps, provider, model, tokens and cost. It can create drafts and findings; it cannot approve, schedule or send. |
| **Models** | Interchangeable providers through qbit-assistant's `AssistantModelProvider`: Claude (`qbit-assistant-provider-anthropic`), any OpenAI-compatible endpoint (included), and a scripted provider for tests. With no provider configured, the agent says so and everything else works. |
| **Models** | Interchangeable providers through `qbit-assistant-provider-api`'s `AssistantModelProvider`: OpenAI-compatible endpoints through `qbit-assistant-provider-openai` (added by the host), or a host-supplied provider such as Claude. Tests use a local scripted double. With no provider configured, the agent says so and everything else works. Installing marketing does not install the assistant runtime, tables or routes. |
## Install
@@ -38,14 +38,24 @@ It lives inside the host QQQ application: its navigation, authentication, permis
```
```java
import com.kingsrook.qbits.assistant.spi.ProviderSettings;
import com.kingsrook.qbits.assistant.provider.openai.OpenAiCompatibleModelProvider;
import com.kingsrook.qqq.backend.core.instances.QMetaDataVariableInterpreter;
var provider = new OpenAiCompatibleModelProvider(new ProviderSettings()
.withKind("OPENAI_COMPATIBLE")
.withBaseUrl("https://api.openai.com/v1")
.withApiKey(new QMetaDataVariableInterpreter().interpret("${env.MARKETING_OPENAI_API_KEY}")));
new MarketingQBitProducer().withQBitConfig(new MarketingQBitConfig()
.withBackendName("rdbms") // where the marketing_* tables live
.withCrmGateway(new QbitCrmGateway()) // optional: qbit-crm
.withSchedulerName("scheduler") // optional: runs the execution worker
.withTransport(new QqqMessagingEmailTransport() // optional: email via your QQQ messaging provider
.withMessagingProviderName("email").withFromAddress("news@example.org"))
.withModelProvider(AnthropicModelProvider.fromEnvironment()) // optional: only when ANTHROPIC_API_KEY is set
.withModel("claude-sonnet-5-5"))
.withModelProvider(provider) // optional: configure only for agent use
.withProviderName("openai-compatible")
.withModel("your-model-id"))
.produce(qInstance);
```
@@ -56,6 +66,11 @@ Then:
[`docs/OPERATIONS.md`](docs/OPERATIONS.md) covers every setting, the run-books and the limits.
Marketing depends only on the provider SPI (`qbit-assistant-provider-api` `0.1.0-RC.12`, package `com.kingsrook.qbits.assistant.spi`). To use the OpenAI-compatible provider, the host adds `com.kingsrook.qbits:qbit-assistant-provider-openai:0.1.0-RC.12` itself.
Replace imports of marketing's former `agent.provider.OpenAiCompatibleModelProvider` with the published class above and replace its builder with `ProviderSettings`. Hosts upgrading from RC.11 change `com.kingsrook.qbits.assistant.provider.*` and `…assistant.engine.ModelPricing` imports to `com.kingsrook.qbits.assistant.spi.*`.
The host resolves the API key reference; `ProviderSettings.withApiKeyEnvName(...)` alone does not resolve it.
Set any existing Marketing Policy provider override to `openai-compatible` as well. The base URL, key environment variable name and model remain host-configurable; omit the key for a local endpoint that needs none.
## Permissions
| Permission | Grants |
@@ -83,6 +98,17 @@ Then:
- Content: Proposed Publication.
- **Actions:** every action is a streamed-ETL record action (record input, standard Review and Result summary lines) in the table's or record's Actions menu. None are nav items.
## Email through qbit-email
When the optional [qbit-email](https://github.com/QRun-IO/qbit-email) qbit is produced in the same instance, email channels named `qqq_email` (the starter-data channel) send through it automatically, unless the host registered its own transport under that name. `QqqMessagingEmailTransport` stays the fallback; without qbit-email nothing changes.
- Each recipient is one `MARKETING` message. qbit-email adds the RFC 8058 unsubscribe link and headers, enforces suppression and owns retries.
- The campaign content is sent as an ad-hoc Markdown email (subject = title, body plus the call to action link); no qbit-email template is registered.
- Idempotency key `marketing:<content id>:<approval id>:<recipient key>`; the linked record is the `marketing_execution` row. A repeat is a duplicate, never a second send.
- Accepted means queued in the qbit-email outbox. A suppressed recipient is skipped, not failed (the result says how many); a refusal by qbit-email is a rejected execution with its reason.
- Needs qbit-email's `publicBaseUrl`, the `QQQ_EMAIL_UNSUBSCRIBE_KEY` secret, an enabled default provider and a default brand with a postal address. Without the first two the transport reports *Not configured* and says which is missing.
- qbit-email is an optional dependency pinned to `0.1.0-RC.3` from Maven Central (`qbit-email.version`). The integration uses only its published `com.kingsrook.qbits.email.api` package for installation checks, readiness, sends and lookups, with the parent's default guardrail allow-lists.
## What is implemented, what is not configured
| Implemented and tested | Implemented, not configured / verified live |
@@ -91,11 +117,11 @@ Then:
| Approval binding, invalidation, separation of duties | Email transport (needs the host's QQQ messaging provider; reports *Not configured* until then, and *Configured* is not *Operational*) |
| Durable execution, retries, reconciliation, leases | No social, community, blog or CMS connector exists: those channels are posted by hand |
| Controlled test transport (complete) | No analytics importer: awareness and activation metrics stay *Not measured* until something records outcome events |
| Agent loop, tools, budgets, limits, run records | QQQ AI tool kernel registration (needs QRun-IO/qqq#1096; the qbit works without it) |
| Agent loop, tools, budgets, limits, run records; read-tool registration in the QQQ RC.6 AI tool registry | |
| CRM gateway and qbit-crm adapter, explicit lead rules | |
| Persistence across restart (H2 file, and PostgreSQL in the Voyage host) | |
Scale gaps (single worker per JVM, in-memory rate limits) are listed in [`docs/OPERATIONS.md`](docs/OPERATIONS.md#scale-and-limits).
Scale gaps (single execution worker per JVM; agent limits are database-counted but still racy across replicas) are listed in [`docs/OPERATIONS.md`](docs/OPERATIONS.md#scale-and-limits).
## Build
@@ -105,6 +131,8 @@ Scale gaps (single worker per JVM, in-memory rate limits) are listed in [`docs/O
Always use the Maven wrapper (Maven 3.9.11); Maven 3.10 cannot read the parent POM. The entities, enums and DDL are generated from one spec: `python -I scripts/generate-model.py .` (commit the output).
Architecture guardrails and strict UI API route checks are enabled with the parent's default allow-lists only (no per-repository lists): `qbit-build-parent` 2.1.2 allows the `*-api` artifacts and the `com.kingsrook.qbits.assistant.spi` and `com.kingsrook.qbits.email.api` packages. `qbit-assistant-provider-openai` is a test dependency only, and the full assistant artifact is never a dependency.
## License
Apache License 2.0. See [LICENSE](LICENSE) and [NOTICE](NOTICE).
+2 -2
View File
@@ -18,8 +18,8 @@ The work is truthful and auditable: facts come from sources with dates, people a
| From | Reused for |
|---|---|
| **qbit-assistant** (0.1.0-RC.2) | The model-provider SPI: `AssistantModelProvider`, `ModelRequest` / `ModelResponse` / `ModelPart`, `StreamHandle`, `ModelProviderException` with its retryable kinds, and `ModelUsage`. Also the Claude provider (`qbit-assistant-provider-anthropic`), `ScriptedModelProvider` for tests and demos, and `ModelPricing` for cost. Marketing adds an OpenAI-compatible provider that implements the same SPI. There is no second provider abstraction. |
| **QQQ AI tool kernel** (qqq#1078, core) | The tool contract: `AbstractAiTool` / `AiToolKind` (READ, WRITE_PROPOSAL), `AiToolOutcome`, the `qqq_data` untrusted-data envelope (`AiDataEnvelope`), secret redaction (`AiRedactor`), strict input keys, and running as the user's session. The kernel's tool set is fixed in RC.4. So QRun-IO/qqq#1096 adds `AiToolRegistry`; Marketing registers its read tools there when QQQ has it and otherwise runs them in its own runner with the same contract. |
| **qbit-assistant** (0.1.0-RC.2) | The model-provider SPI: `AssistantModelProvider`, `ModelRequest` / `ModelResponse` / `ModelPart`, `StreamHandle`, `ModelProviderException` with its retryable kinds, and `ModelUsage`. Since RC.12 it lives in `com.kingsrook.qbits.assistant.spi` (`qbit-assistant-provider-api`), together with `ModelPricing` for cost; hosts register a provider such as `qbit-assistant-provider-anthropic` or `qbit-assistant-provider-openai`. There is no second provider abstraction. |
| **QQQ AI tool kernel** (qqq#1078, core) | The tool contract: `AbstractAiTool` / `AiToolKind` (READ, WRITE_PROPOSAL), `AiToolOutcome`, the `qqq_data` untrusted-data envelope (`AiDataEnvelope`), secret redaction (`AiRedactor`), strict input keys, and running as the user's session. The RC.6 baseline includes `AiToolRegistry` (QRun-IO/qqq#1096); Marketing contributes its read tools there and runs its own agent tools with the same contract. |
| **QQQ core permission enforcement** (qqq#1061) | Agent tools read with `PermissionEnforcement.ENFORCE`. Service entry points check their step's permission, then run as trusted internal work in an authorized scope, as QQQ does for calls nested in a core action. Record security locks always apply. |
| **qbit-assistant / sql-workbench metadata builders** | Explicit builder-produced tables (not annotation scanning), snake_case `marketing_*` names, QQQ standard permission suffixes, `DenyBehavior.HIDDEN`, read-only system tables, record locks, reference DDL in `docs/ddl`, `tableMetaDataCustomizer` and `inferBackendNames`. |
| **Website-Backend admin conventions** | Processes are table and record actions, never nav items, with plain labels. Record actions use `StreamedETLWithFrontendProcess` with standard summary lines. App homes are real-data dashboards built from ≤3-column full-width tables. Links use the Next route grammar (`AiLinkBuilder`). |
+5 -4
View File
@@ -12,7 +12,7 @@
| `providerName` / `model` / `effort` | none | Which provider and model the agent uses. The Marketing Policy can override them. |
| `maxOutputTokens` | 8000 | Per model call. |
| `maxToolCallsPerRun` / `maxModelCallsPerRun` / `maxRunSeconds` | 24 / 16 / 300 | Limits of one agent run. |
| `maxConcurrentAgentRuns` / `maxModelCallsPerMinute` | 2 / 30 | Agent concurrency and rate (per JVM). |
| `maxConcurrentAgentRuns` / `maxModelCallsPerMinute` | 2 / 30 | Agent concurrency and rate, counted in the database so they hold across replicas. |
| `maxRowsToModel` | 50 | Rows per read tool. |
| `pricing` | Anthropic list prices (qbit-assistant) | USD per million tokens, by model id. Models without a price show no cost. Prices are never invented. |
| `registerKernelTools` | `true` | Contribute the read tools to the QQQ AI tool kernel (when QQQ has `AiToolRegistry`). |
@@ -48,8 +48,9 @@ Nothing in Marketing stores a credential, and every key is read from the host's
| Integration | What to provide |
|---|---|
| Claude | `ANTHROPIC_API_KEY` (or `AnthropicModelProvider.builder().apiKey(fromSecretStore)`), and a model id such as `claude-sonnet-5-5`. |
| OpenAI-compatible | A base URL (`https://api.openai.com/v1`, `http://localhost:11434/v1` for Ollama, ...), an API key reference such as `${env.MARKETING_OPENAI_API_KEY}` if the server needs one, and a model id. |
| Email | A QQQ messaging provider in the host instance (SMTP or SES), its name, and a from-address. After verifying it end to end, `withVerifiedOperational(true)`. Until then the email transport reports *Configured, not verified* and executions are blocked. |
| OpenAI-compatible | Use `com.kingsrook.qbits.assistant.provider.openai.OpenAiCompatibleModelProvider` from `qbit-assistant-provider-openai` with `ProviderSettings`: a base URL (`https://api.openai.com/v1`, `http://localhost:11434/v1` for Ollama, ...), a resolved key if the server needs one, and a model id in `MarketingQBitConfig.withModel(...)`. Resolve `${env.MARKETING_OPENAI_API_KEY}` with `QMetaDataVariableInterpreter` before `withApiKey(...)`; setting `apiKeyEnvName` alone does not resolve a key. The provider name is `openai-compatible`, including in Marketing Policy overrides. See the [setup example](../README.md#install). |
| Email (qbit-email) | When qbit-email is produced in the instance and no transport named `qqq_email` is registered, email goes through its outbox (category MARKETING). It needs `publicBaseUrl`, the `QQQ_EMAIL_UNSUBSCRIBE_KEY` secret, an enabled default provider and a default brand with a postal address; a missing piece is reported as *Not configured* or as a plain rejected-execution reason. |
| Email (QQQ messaging) | A QQQ messaging provider in the host instance (SMTP or SES), its name, and a from-address. After verifying it end to end, `withVerifiedOperational(true)`. Until then the email transport reports *Configured, not verified* and executions are blocked. |
**Status:** check **Configuration Status** on the Marketing dashboard after any change.
@@ -94,7 +95,7 @@ Fix the cause. If the content changed, it needs a new approval. Then schedule ag
These are real limitations of 0.1.0. Nothing here is claimed to be production-scale.
- **Single execution worker per JVM.** The worker takes an in-process lock and claims a row by re-reading its status before marking it Running. Two replicas could both claim the same due execution in the instant between read and update, and the idempotency key is then the only protection (it fully protects the test transport, but not email). **Run Marketing's worker on one replica**, or add a database-level claim (`UPDATE ... WHERE status = SCHEDULED`, or `SELECT ... FOR UPDATE SKIP LOCKED`) before scaling out.
- **Rate limits and agent concurrency are per JVM** (in memory). The channel hourly rate is computed from the database, so it holds across replicas, but only up to the race above.
- **Agent concurrency and the model-call rate are counted in the database**, so they hold across replicas. Concurrency counts `marketing_agent_run` rows in Running (all products); a run records a heartbeat on each model call, and a Running run silent for `maxRunSeconds` plus 60 seconds is marked Failed ("Stopped responding; the server may have restarted.") before the count. The rate counts model-call steps in the last 60 seconds; at the limit a run waits (bounded by `maxRunSeconds`, and Cancel Run still works) instead of failing. The channel hourly rate is computed the same way. Counting and then inserting is still racy: two replicas can both see room, so a limit can be overshot briefly. A conditional-update claim (`ConditionalUpdateAction`, qqq#1178, QQQ 4.1.0-RC.7) will close that window. Hosts add the new `heartbeat_at` column to `marketing_agent_run` (see `docs/ddl/postgres.sql`).
- **Agent runs are synchronous** inside the record action, bounded by `maxRunSeconds`. A restart interrupts them: they are marked Failed and can be run again. There is no queue or resume.
- **The worker queries due executions page by page** (`workerBatchSize` per run) and scans channel attempts for the hourly rate. That suits hundreds of executions per hour, not bulk email. Bulk outreach is deliberately not supported.
- **Measurement is computed in memory** from a campaign's events on every view. Add a rollup table before campaigns have hundreds of thousands of events.
+1
View File
@@ -321,6 +321,7 @@ CREATE TABLE IF NOT EXISTS marketing_agent_run (
model VARCHAR(100),
requested_by_user_id VARCHAR(100),
started_at TIMESTAMP WITHOUT TIME ZONE,
heartbeat_at TIMESTAMP WITHOUT TIME ZONE,
finished_at TIMESTAMP WITHOUT TIME ZONE,
model_calls INTEGER,
tool_calls INTEGER,
+24 -8
View File
@@ -22,17 +22,18 @@
<maven.compiler.source>21</maven.compiler.source>
<maven.compiler.target>21</maven.compiler.target>
<jacoco.version>0.8.15</jacoco.version>
<!-- The model-provider SPI (AssistantModelProvider), the scripted test -->
<!-- provider and the price table come from qbit-assistant; marketing -->
<!-- reuses them instead of a second provider abstraction. -->
<qbit-assistant.version>0.1.0-RC.2</qbit-assistant.version>
<!-- The provider SPI (com.kingsrook.qbits.assistant.spi) only: no assistant runtime. -->
<qbit-assistant.version>0.1.0-RC.12</qbit-assistant.version>
<qbit.guardrails.enforce>true</qbit.guardrails.enforce>
<qbit.guardrails.strictUiApiRoutes>true</qbit.guardrails.strictUiApiRoutes>
<qbit-email.version>0.1.0-RC.3</qbit-email.version>
<h2.version>2.3.232</h2.version>
</properties>
<parent>
<groupId>com.kingsrook</groupId>
<artifactId>qbit-build-parent</artifactId>
<version>2.0.0</version>
<version>2.1.2</version>
</parent>
<groupId>com.kingsrook.qbits</groupId>
@@ -79,9 +80,24 @@
<dependency>
<groupId>com.kingsrook.qbits</groupId>
<artifactId>qbit-assistant</artifactId>
<artifactId>qbit-assistant-provider-api</artifactId>
<version>${qbit-assistant.version}</version>
</dependency>
<dependency>
<groupId>com.kingsrook.qbits</groupId>
<artifactId>qbit-assistant-provider-openai</artifactId>
<version>${qbit-assistant.version}</version>
<!-- tests only: hosts add the provider they register (README, Install) -->
<scope>test</scope>
</dependency>
<!-- optional: email through qbit-email when it is produced in the host instance (QbitEmailTransport) -->
<dependency>
<groupId>com.kingsrook.qbits</groupId>
<artifactId>qbit-email</artifactId>
<version>${qbit-email.version}</version>
<optional>true</optional>
</dependency>
<!-- tests: the RDBMS backend over H2 proves the DDL and persistence across a restart -->
<dependency>
@@ -177,14 +193,14 @@
<profiles>
<!-- Built against qqq 4.1 (QBitMetaDataProducer). Until qbit-build-parent -->
<!-- imports the released 4.1 BOM, this profile is active by default and -->
<!-- imports qqq-bom-pom 4.1.0-RC.4 ahead of the parent's 4.0.0 BOM. -->
<!-- imports qqq-bom-pom 4.1.0-RC.6 ahead of the parent's 4.0.0 BOM. -->
<profile>
<id>qqq-snapshot</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<qqq.snapshot.version>4.1.0-RC.4</qqq.snapshot.version>
<qqq.snapshot.version>4.1.0-RC.6</qqq.snapshot.version>
</properties>
<dependencyManagement>
<dependencies>
+1 -1
View File
@@ -171,7 +171,7 @@ TABLES = [
("MarketingAgentRun", "marketing_agent_run", "A run of the marketing agent: task, status, provider, model usage and outputs.", [
F("productId", "Fk:MarketingProduct"), F("campaignId", "Fk:MarketingCampaign"), F("contentId", "Fk:MarketingContent"),
F("task", "Enum:AgentTask", required=True), F("status", "Enum:AgentRunStatus", required=True), F("instructions", "Text"),
F("providerName", "String:64"), F("model", "String:100"), F("requestedByUserId", "String:100"), F("startedAt", "Instant"),
F("providerName", "String:64"), F("model", "String:100"), F("requestedByUserId", "String:100"), F("startedAt", "Instant"), F("heartbeatAt", "Instant"),
F("finishedAt", "Instant"), F("modelCalls", "Integer"), F("toolCalls", "Integer"), F("inputTokens", "Long"),
F("outputTokens", "Long"), F("cacheReadTokens", "Long"), F("costMicros", "Long"), F("draftsCreated", "Integer"),
F("findingsCreated", "Integer"), F("summary", "Text"), F("outputJson", "Text"), F("errorMessage", "Text")]),
@@ -21,8 +21,8 @@ import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.assistant.engine.ModelPricing;
import com.kingsrook.qbits.assistant.provider.AssistantModelProvider;
import com.kingsrook.qbits.assistant.spi.ModelPricing;
import com.kingsrook.qbits.assistant.spi.AssistantModelProvider;
import com.kingsrook.qbits.marketing.crm.MarketingCrmGateway;
import com.kingsrook.qbits.marketing.transport.MarketingTransport;
import com.kingsrook.qqq.backend.core.model.metadata.producers.MetaDataCustomizerInterface;
@@ -18,20 +18,19 @@ package com.kingsrook.qbits.marketing;
import java.net.InetAddress;
import java.util.ArrayDeque;
import java.util.ArrayList;
import java.util.Deque;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.UUID;
import java.util.concurrent.Semaphore;
import java.util.concurrent.locks.ReentrantLock;
import com.kingsrook.qbits.assistant.provider.AssistantModelProvider;
import com.kingsrook.qbits.assistant.spi.AssistantModelProvider;
import com.kingsrook.qbits.marketing.crm.MarketingCrmGateway;
import com.kingsrook.qbits.marketing.email.QbitEmailTransport;
import com.kingsrook.qbits.marketing.model.MarketingPolicy;
import com.kingsrook.qbits.marketing.transport.ControlledTestTransport;
import com.kingsrook.qbits.marketing.transport.MarketingTransport;
import com.kingsrook.qbits.marketing.transport.QqqMessagingEmailTransport;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
@@ -48,14 +47,15 @@ import com.kingsrook.qqq.backend.core.utils.StringUtils;
/*******************************************************************************
** The live parts of the Marketing QBit for one instance: the transports, the
** model providers, the CRM gateway, and the in-process limits (agent
** concurrency, model-call rate, the single execution worker).
** model providers, the CRM gateway, and the single execution worker's
** in-process lock. (Agent concurrency and the model-call rate are counted in
** the database by MarketingAgentService.)
**
** Registered as supplemental instance meta-data (not published to
** frontends), so process steps and widgets find it from QContext.
**
** Scale note: the limits here are per JVM. One replica is the supported shape
** today; see docs/OPERATIONS.md ("Scale and limits").
** Scale note: the execution worker is per JVM. One worker replica is the
** supported shape today; see docs/OPERATIONS.md ("Scale and limits").
*******************************************************************************/
public class MarketingRuntime implements QSupplementalInstanceMetaData
{
@@ -67,10 +67,9 @@ public class MarketingRuntime implements QSupplementalInstanceMetaData
private final MarketingQBitConfig config;
private final Map<String, MarketingTransport> transports = new LinkedHashMap<>();
private final Semaphore agentPermits;
private final ReentrantLock workerLock = new ReentrantLock();
private final Deque<Long> modelCallTimes = new ArrayDeque<>();
private final String workerName;
private volatile MarketingTransport qbitEmailTransport;
@@ -88,7 +87,6 @@ public class MarketingRuntime implements QSupplementalInstanceMetaData
{
transports.put(transport.getName(), transport);
}
this.agentPermits = new Semaphore(config.getMaxConcurrentAgentRuns());
this.workerName = hostName() + ":" + UUID.randomUUID().toString().substring(0, 8);
}
@@ -131,11 +129,47 @@ public class MarketingRuntime implements QSupplementalInstanceMetaData
/*******************************************************************************
** A transport by name, or null.
** A transport by name, or null. When the host registered no transport named
** qqq_email (the default email channel name) and qbit-email is produced in the
** instance, that name resolves to QbitEmailTransport; a host-registered
** transport always wins, and QqqMessagingEmailTransport remains the fallback
** the host can register.
*******************************************************************************/
public MarketingTransport getTransport(String name)
{
return (name == null ? null : transports.get(name));
if(name == null)
{
return (null);
}
MarketingTransport transport = transports.get(name);
if(transport == null && QqqMessagingEmailTransport.DEFAULT_NAME.equals(name) && emailQBitClassPresent() && QbitEmailTransport.isInstalled(QContext.getQInstance()))
{
if(qbitEmailTransport == null)
{
qbitEmailTransport = new QbitEmailTransport();
}
return (qbitEmailTransport);
}
return (transport);
}
/*******************************************************************************
** qbit-email is an optional dependency: touch QbitEmailTransport only when
** its classes are present.
*******************************************************************************/
private static boolean emailQBitClassPresent()
{
try
{
Class.forName("com.kingsrook.qbits.email.api.EmailApi");
return (true);
}
catch(ClassNotFoundException | LinkageError e)
{
return (false);
}
}
@@ -289,47 +323,6 @@ public class MarketingRuntime implements QSupplementalInstanceMetaData
/*******************************************************************************
** Try to start an agent run (bounded concurrency). Release with
** releaseAgentPermit.
*******************************************************************************/
public boolean tryAcquireAgentPermit()
{
return (agentPermits.tryAcquire());
}
/*******************************************************************************
**
*******************************************************************************/
public void releaseAgentPermit()
{
agentPermits.release();
}
/*******************************************************************************
** Reserve a model call within the per-minute rate (false = over the limit).
*******************************************************************************/
public synchronized boolean tryReserveModelCall()
{
long now = System.currentTimeMillis();
while(!modelCallTimes.isEmpty() && modelCallTimes.peekFirst() < now - 60_000)
{
modelCallTimes.pollFirst();
}
if(modelCallTimes.size() >= config.getMaxModelCallsPerMinute())
{
return (false);
}
modelCallTimes.addLast(now);
return (true);
}
/*******************************************************************************
** The single execution worker's lock in this JVM.
*******************************************************************************/
@@ -25,18 +25,18 @@ import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.assistant.engine.ModelPricing;
import com.kingsrook.qbits.assistant.provider.AssistantModelProvider;
import com.kingsrook.qbits.assistant.provider.CacheHint;
import com.kingsrook.qbits.assistant.provider.ModelEventListener;
import com.kingsrook.qbits.assistant.provider.ModelMessage;
import com.kingsrook.qbits.assistant.provider.ModelPart;
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
import com.kingsrook.qbits.assistant.provider.ModelRequest;
import com.kingsrook.qbits.assistant.provider.ModelResponse;
import com.kingsrook.qbits.assistant.provider.ModelToolDefinition;
import com.kingsrook.qbits.assistant.provider.ModelUsage;
import com.kingsrook.qbits.assistant.provider.SystemBlock;
import com.kingsrook.qbits.assistant.spi.ModelPricing;
import com.kingsrook.qbits.assistant.spi.AssistantModelProvider;
import com.kingsrook.qbits.assistant.spi.CacheHint;
import com.kingsrook.qbits.assistant.spi.ModelEventListener;
import com.kingsrook.qbits.assistant.spi.ModelMessage;
import com.kingsrook.qbits.assistant.spi.ModelPart;
import com.kingsrook.qbits.assistant.spi.ModelProviderException;
import com.kingsrook.qbits.assistant.spi.ModelRequest;
import com.kingsrook.qbits.assistant.spi.ModelResponse;
import com.kingsrook.qbits.assistant.spi.ModelToolDefinition;
import com.kingsrook.qbits.assistant.spi.ModelUsage;
import com.kingsrook.qbits.assistant.spi.SystemBlock;
import com.kingsrook.qbits.marketing.MarketingQBitConfig;
import com.kingsrook.qbits.marketing.MarketingRuntime;
import com.kingsrook.qbits.marketing.MarketingStore;
@@ -78,11 +78,15 @@ import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
** A run:
** 1. checks the policy (kill switch), the provider and model, the daily
** token and monthly cost budgets, and the concurrency limit - failing the
** run with a plain reason when any is missing (missing-provider behavior);
** run with a plain reason when any is missing (missing-provider behavior).
** The concurrency limit counts RUNNING runs in the database (after failing
** those whose heartbeat stopped), so it holds across replicas;
** 2. runs the deterministic readiness checks (a CHECK step);
** 3. loops model call -> tool calls, bounded by model calls, tool calls,
** wall-clock time and the per-minute model-call rate; it re-reads its own
** status before every model call so Cancel Run stops it;
** wall-clock time and the per-minute model-call rate (counted from the
** database; at the limit the run waits, it does not fail); it re-reads its
** own status before every model call so Cancel Run stops it, and records a
** heartbeat on each model call;
** 4. records every model call (tokens, stop reason) and tool call (input,
** outcome, enveloped result), the usage and cost, the drafts and findings
** it created, and its final summary.
@@ -98,8 +102,29 @@ public class MarketingAgentService
static final int MAX_STEP_JSON = 20_000;
/////////////////////////////////////////////////////////////////////////////
// a RUNNING run silent for maxRunSeconds plus this long is not alive //
/////////////////////////////////////////////////////////////////////////////
public static final int STALE_GRACE_SECONDS = 60;
private final AiRedactor redactor = new AiRedactor(List.of());
private Duration modelCallWindow = Duration.ofSeconds(60);
private Duration modelCallPoll = Duration.ofSeconds(1);
/*******************************************************************************
** Change the per-minute window and how often a waiting run re-checks it
** (the defaults are 60 seconds and 1 second; tests use short ones).
*******************************************************************************/
public MarketingAgentService withModelCallWindow(Duration window, Duration poll)
{
this.modelCallWindow = window;
this.modelCallPoll = poll;
return (this);
}
/*******************************************************************************
@@ -158,19 +183,7 @@ public class MarketingAgentService
return (finish(run, AgentRunStatus.FAILED, null, refusal, null, new MarketingToolContext(run, 1), new ModelUsage(), 0, 0));
}
if(!runtime.tryAcquireAgentPermit())
{
return (finish(run, AgentRunStatus.FAILED, null, "Too many agent runs at once (limit " + runtime.getConfig().getMaxConcurrentAgentRuns() + "); start it again shortly.",
null, new MarketingToolContext(run, 1), new ModelUsage(), 0, 0));
}
try
{
return (execute(runtime, policy, provider, model, run, task));
}
finally
{
runtime.releaseAgentPermit();
}
return (execute(runtime, policy, provider, model, run, task));
}
@@ -213,6 +226,101 @@ public class MarketingAgentService
return ("This month's agent cost budget is used up.");
}
}
/////////////////////////////////////////////////////////////////////////////
// concurrency is counted in the database, across every replica and //
// product. NOTE: count-then-mark-RUNNING is still racy - two replicas can //
// both see room and both start. A conditional-update claim //
// (ConditionalUpdateAction, qqq#1178, QQQ 4.1.0-RC.7) will close that //
// window as a follow-up. //
/////////////////////////////////////////////////////////////////////////////
failStaleRuns(runtime.getConfig());
int limit = policy.getMaxConcurrentAgentRuns() != null ? policy.getMaxConcurrentAgentRuns() : runtime.getConfig().getMaxConcurrentAgentRuns();
if(runningRuns() >= limit)
{
return ("Too many agent runs at once (limit " + limit + "); start it again shortly.");
}
return (null);
}
/*******************************************************************************
** RUNNING runs (all products) whose heartbeat - or start, for a run that
** never beat - is older than maxRunSeconds plus a grace period are failed:
** their server stopped. Returns how many.
*******************************************************************************/
int failStaleRuns(MarketingQBitConfig config) throws QException
{
Instant cutoff = Instant.now().minusSeconds((long) config.getMaxRunSeconds() + STALE_GRACE_SECONDS);
List<QRecord> running = MarketingStore.asSystem(() -> MarketingStore.query(MarketingAgentRun.TABLE_NAME,
new QQueryFilter(new QFilterCriteria("status", QCriteriaOperator.EQUALS, AgentRunStatus.RUNNING.getId()))));
int count = 0;
for(QRecord record : running)
{
Instant last = record.getValueInstant("heartbeatAt");
last = last != null ? last : record.getValueInstant("startedAt");
last = last != null ? last : record.getValueInstant("createDate");
if(last != null && last.isBefore(cutoff))
{
MarketingStore.updateInternal(MarketingAgentRun.TABLE_NAME, new QRecord().withValue("id", record.getValue("id"))
.withValue("status", AgentRunStatus.FAILED.getId()).withValue("finishedAt", Instant.now())
.withValue("errorMessage", "Stopped responding; the server may have restarted."));
count++;
}
}
return (count);
}
/*******************************************************************************
** RUNNING runs across all products, counted as the system user.
*******************************************************************************/
static int runningRuns() throws QException
{
return (MarketingStore.asSystem(() -> MarketingStore.count(MarketingAgentRun.TABLE_NAME,
new QQueryFilter(new QFilterCriteria("status", QCriteriaOperator.EQUALS, AgentRunStatus.RUNNING.getId())))));
}
/*******************************************************************************
** Model calls (steps of the model-call kind, all runs, system user) recorded
** since the instant.
*******************************************************************************/
static int modelCallsSince(Instant since) throws QException
{
return (MarketingStore.asSystem(() -> MarketingStore.count(MarketingAgentStep.TABLE_NAME, new QQueryFilter(
new QFilterCriteria("kind", QCriteriaOperator.EQUALS, AgentStepKind.MODEL_CALL.getId()),
new QFilterCriteria("createDate", QCriteriaOperator.GREATER_THAN_OR_EQUALS, since)))));
}
/*******************************************************************************
** Wait until the per-minute model-call limit has room. Returns null to go
** ahead, CANCELLED if a person cancelled the run while waiting, or FAILED
** if the run's time ran out first.
**
** NOTE: counting then calling is racy across replicas (see preflight); the
** conditional-update claim follow-up (qqq#1178) will close it.
*******************************************************************************/
private AgentRunStatus awaitModelCallSlot(MarketingAgentRun run, int limit, Instant deadline) throws QException, InterruptedException
{
while(modelCallsSince(Instant.now().minus(modelCallWindow)) >= limit)
{
if(isCancelled(run.getId()))
{
return (AgentRunStatus.CANCELLED);
}
if(Instant.now().isAfter(deadline))
{
return (AgentRunStatus.FAILED);
}
Thread.sleep(modelCallPoll.toMillis());
}
return (null);
}
@@ -229,7 +337,7 @@ public class MarketingAgentService
int maxTools = policy != null && policy.getMaxToolCallsPerRun() != null ? policy.getMaxToolCallsPerRun() : config.getMaxToolCallsPerRun();
MarketingStore.updateInternal(MarketingAgentRun.TABLE_NAME, new QRecord().withValue("id", run.getId())
.withValue("status", AgentRunStatus.RUNNING.getId()).withValue("startedAt", Instant.now()));
.withValue("status", AgentRunStatus.RUNNING.getId()).withValue("startedAt", Instant.now()).withValue("heartbeatAt", Instant.now()));
MarketingToolContext context = new MarketingToolContext(run, config.getMaxRowsToModel());
ModelUsage usage = new ModelUsage();
@@ -278,11 +386,21 @@ public class MarketingAgentService
{
return (finish(run, AgentRunStatus.FAILED, null, "Stopped: the run reached its limit of " + config.getMaxModelCallsPerRun() + " model calls.", null, context, usage, modelCalls, toolCalls));
}
if(!runtime.tryReserveModelCall())
/////////////////////////////////////////////////////////////////
// at the per-minute limit, wait (bounded by the run's time) //
/////////////////////////////////////////////////////////////////
AgentRunStatus blocked = awaitModelCallSlot(run, config.getMaxModelCallsPerMinute(), deadline);
if(blocked == AgentRunStatus.CANCELLED)
{
return (finish(run, AgentRunStatus.FAILED, null, "Stopped: the model-call rate limit (" + config.getMaxModelCallsPerMinute() + " per minute) was reached.", null, context, usage, modelCalls,
toolCalls));
return (finish(run, AgentRunStatus.CANCELLED, null, "Cancelled by a person.", null, context, usage, modelCalls, toolCalls));
}
if(blocked == AgentRunStatus.FAILED)
{
return (finish(run, AgentRunStatus.FAILED, null, "Stopped: the run exceeded " + config.getMaxRunSeconds() + " seconds, waiting for the model-call rate limit ("
+ config.getMaxModelCallsPerMinute() + " per minute).", null, context, usage, modelCalls, toolCalls));
}
MarketingStore.updateInternal(MarketingAgentRun.TABLE_NAME, new QRecord().withValue("id", run.getId()).withValue("heartbeatAt", Instant.now()));
ModelRequest request = new ModelRequest()
.withModel(model)
@@ -1,512 +0,0 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.marketing.agent.provider;
import java.io.IOException;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.net.http.HttpTimeoutException;
import java.time.Duration;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.CompletableFuture;
import com.fasterxml.jackson.core.type.TypeReference;
import com.kingsrook.qbits.assistant.provider.AssistantModelProvider;
import com.kingsrook.qbits.assistant.provider.CompletableStreamHandle;
import com.kingsrook.qbits.assistant.provider.ModelCapabilities;
import com.kingsrook.qbits.assistant.provider.ModelEventListener;
import com.kingsrook.qbits.assistant.provider.ModelMessage;
import com.kingsrook.qbits.assistant.provider.ModelPart;
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
import com.kingsrook.qbits.assistant.provider.ModelRequest;
import com.kingsrook.qbits.assistant.provider.ModelResponse;
import com.kingsrook.qbits.assistant.provider.ModelToolDefinition;
import com.kingsrook.qbits.assistant.provider.ModelUsage;
import com.kingsrook.qbits.assistant.provider.StreamHandle;
import com.kingsrook.qbits.assistant.provider.SystemBlock;
import com.kingsrook.qqq.backend.core.instances.QMetaDataVariableInterpreter;
import com.kingsrook.qqq.backend.core.utils.JsonUtils;
import com.kingsrook.qqq.backend.core.utils.StringUtils;
/*******************************************************************************
** A model provider for any OpenAI-compatible Chat Completions endpoint
** (POST {baseUrl}/chat/completions): OpenAI, and the many servers that speak
** the same API (OpenRouter, Ollama, vLLM, LM Studio, LiteLLM, ...).
** Implements qbit-assistant's AssistantModelProvider, so it plugs into the
** marketing agent (and the assistant) next to the Anthropic provider.
**
** - No SDK: java.net.http. Non-streaming request; the text is delivered to
** the listener at the end.
** - The API key is read from the environment / secret reference given to
** the builder ("${env.NAME}"); it is never logged, and toString never
** contains it. A provider without a key is allowed for local servers that
** need none (Ollama).
** - Errors map to ModelProviderException kinds: 401/403 AUTHENTICATION,
** 429 RATE_LIMITED, 5xx SERVER / OVERLOADED (503), 400/404/422
** INVALID_REQUEST, timeouts TIMEOUT, I/O NETWORK. Messages carry the HTTP
** status and the API's error type only.
*******************************************************************************/
public class OpenAiCompatibleModelProvider implements AssistantModelProvider
{
private final String name;
private final String baseUrl;
private final String apiKey;
private final Duration requestTimeout;
private final HttpClient httpClient;
private final Boolean strictTools;
/*******************************************************************************
** Use the builder.
*******************************************************************************/
private OpenAiCompatibleModelProvider(Builder builder)
{
this.name = builder.name;
this.baseUrl = builder.baseUrl.endsWith("/") ? builder.baseUrl.substring(0, builder.baseUrl.length() - 1) : builder.baseUrl;
String key = builder.apiKey;
if(key != null && key.startsWith("${"))
{
key = new QMetaDataVariableInterpreter().interpret(key);
}
this.apiKey = StringUtils.hasContent(key) ? key : null;
this.requestTimeout = builder.requestTimeout;
this.strictTools = builder.strictTools;
this.httpClient = HttpClient.newBuilder().connectTimeout(Duration.ofSeconds(10)).build();
}
/*******************************************************************************
** A builder.
*******************************************************************************/
public static Builder builder()
{
return (new Builder());
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public String getName()
{
return (name);
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public ModelCapabilities getCapabilities(String modelId)
{
return (new ModelCapabilities().withTools(true).withParallelToolCalls(true).withPromptCaching(false).withThinking(false).withMaxContextTokens(128_000));
}
/*******************************************************************************
** Does it have a key? (Local servers may need none.)
*******************************************************************************/
public boolean hasApiKey()
{
return (apiKey != null);
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public StreamHandle stream(ModelRequest request, ModelEventListener listener)
{
CompletableStreamHandle handle = new CompletableStreamHandle();
CompletableFuture<HttpResponse<String>> future;
try
{
HttpRequest.Builder http = HttpRequest.newBuilder(URI.create(baseUrl + "/chat/completions"))
.timeout(requestTimeout)
.header("Content-Type", "application/json")
.POST(HttpRequest.BodyPublishers.ofString(JsonUtils.toJson(toRequestBody(request))));
if(apiKey != null)
{
http.header("Authorization", "Bearer " + apiKey);
}
future = httpClient.sendAsync(http.build(), HttpResponse.BodyHandlers.ofString());
}
catch(Exception e)
{
handle.fail(new ModelProviderException(ModelProviderException.Kind.INVALID_REQUEST, null, "Could not build the request: " + e.getClass().getSimpleName()));
return (handle);
}
handle.setOnCancel(() -> future.cancel(true));
future.whenComplete((response, error) ->
{
if(error != null)
{
Throwable cause = error.getCause() != null ? error.getCause() : error;
if(cause instanceof HttpTimeoutException)
{
handle.fail(new ModelProviderException(ModelProviderException.Kind.TIMEOUT, null, "The request timed out."));
}
else if(cause instanceof IOException)
{
handle.fail(new ModelProviderException(ModelProviderException.Kind.NETWORK, null, "Network error: " + cause.getClass().getSimpleName()));
}
else
{
handle.fail(new ModelProviderException(ModelProviderException.Kind.OTHER, null, "Request failed: " + cause.getClass().getSimpleName()));
}
return;
}
try
{
if(response.statusCode() >= 300)
{
handle.fail(toException(response.statusCode(), response.body()));
return;
}
ModelResponse modelResponse = fromResponseBody(response.body());
String text = modelResponse.getText();
if(StringUtils.hasContent(text))
{
listener.onTextDelta(text);
}
handle.complete(modelResponse);
}
catch(Exception e)
{
handle.fail(new ModelProviderException(ModelProviderException.Kind.OTHER, response.statusCode(), "Unreadable response: " + e.getClass().getSimpleName()));
}
});
return (handle);
}
/*******************************************************************************
** ModelRequest -> Chat Completions request body.
*******************************************************************************/
Map<String, Object> toRequestBody(ModelRequest request)
{
List<Map<String, Object>> messages = new ArrayList<>();
StringBuilder system = new StringBuilder();
for(SystemBlock block : request.getSystem())
{
if(system.length() > 0)
{
system.append("\n\n");
}
system.append(block.getText());
}
if(system.length() > 0)
{
messages.add(Map.of("role", "system", "content", system.toString()));
}
for(ModelMessage message : request.getMessages())
{
if(message.getRole() == ModelMessage.Role.ASSISTANT)
{
Map<String, Object> assistant = new LinkedHashMap<>();
StringBuilder text = new StringBuilder();
List<Map<String, Object>> toolCalls = new ArrayList<>();
for(ModelPart part : message.getParts())
{
if(part.getType() == ModelPart.Type.TEXT && part.getText() != null)
{
text.append(part.getText());
}
else if(part.getType() == ModelPart.Type.TOOL_USE)
{
toolCalls.add(Map.of("id", part.getToolUseId(), "type", "function",
"function", Map.of("name", part.getToolName(), "arguments", JsonUtils.toJson(part.getInput() == null ? Map.of() : part.getInput()))));
}
}
assistant.put("role", "assistant");
assistant.put("content", text.length() == 0 ? null : text.toString());
if(!toolCalls.isEmpty())
{
assistant.put("tool_calls", toolCalls);
}
messages.add(assistant);
}
else
{
StringBuilder text = new StringBuilder();
for(ModelPart part : message.getParts())
{
if(part.getType() == ModelPart.Type.TOOL_RESULT)
{
messages.add(Map.of("role", "tool", "tool_call_id", part.getToolUseId(), "content", part.getText() == null ? "" : part.getText()));
}
else if(part.getText() != null)
{
text.append(part.getText());
}
}
if(text.length() > 0)
{
messages.add(Map.of("role", "user", "content", text.toString()));
}
}
}
Map<String, Object> body = new LinkedHashMap<>();
body.put("model", request.getModel());
body.put("messages", messages);
if(request.getMaxTokens() != null)
{
body.put("max_completion_tokens", request.getMaxTokens());
}
if(!request.getTools().isEmpty())
{
List<Map<String, Object>> tools = new ArrayList<>();
for(ModelToolDefinition tool : request.getTools())
{
Map<String, Object> function = new LinkedHashMap<>();
function.put("name", tool.getName());
function.put("description", tool.getDescription());
function.put("parameters", tool.getInputSchema());
////////////////////////////////////////////////////////////////////
// OpenAI's strict mode needs every property required; the //
// agent validates inputs itself, so strict is off by default. //
////////////////////////////////////////////////////////////////////
if(Boolean.TRUE.equals(strictTools))
{
function.put("strict", true);
}
tools.add(Map.of("type", "function", "function", function));
}
body.put("tools", tools);
body.put("parallel_tool_calls", Boolean.TRUE.equals(request.getParallelToolCalls()));
}
return (body);
}
/*******************************************************************************
** Chat Completions response body -> ModelResponse.
*******************************************************************************/
@SuppressWarnings("unchecked")
ModelResponse fromResponseBody(String json) throws Exception
{
Map<String, Object> root = JsonUtils.toObject(json, new TypeReference<>() {});
List<Map<String, Object>> choices = (List<Map<String, Object>>) root.get("choices");
if(choices == null || choices.isEmpty())
{
throw (new IllegalStateException("no choices"));
}
Map<String, Object> choice = choices.get(0);
Map<String, Object> message = (Map<String, Object>) choice.get("message");
ModelResponse response = new ModelResponse();
Object content = message == null ? null : message.get("content");
if(content instanceof String text && !text.isEmpty())
{
response.getParts().add(ModelPart.text(text));
}
List<Map<String, Object>> toolCalls = message == null ? null : (List<Map<String, Object>>) message.get("tool_calls");
if(toolCalls != null)
{
for(Map<String, Object> call : toolCalls)
{
Map<String, Object> function = (Map<String, Object>) call.get("function");
String arguments = function == null ? null : (String) function.get("arguments");
Map<String, Object> input = StringUtils.hasContent(arguments) ? JsonUtils.toObject(arguments, new TypeReference<Map<String, Object>>() {}) : new LinkedHashMap<>();
response.getParts().add(ModelPart.toolUse((String) call.get("id"), function == null ? null : (String) function.get("name"), input));
}
}
String finish = (String) choice.get("finish_reason");
response.setStopReason(switch(finish == null ? "" : finish)
{
case "tool_calls", "function_call" -> ModelResponse.STOP_TOOL_USE;
case "length" -> ModelResponse.STOP_MAX_TOKENS;
case "content_filter" -> ModelResponse.STOP_REFUSAL;
default -> response.getToolUses().isEmpty() ? ModelResponse.STOP_END_TURN : ModelResponse.STOP_TOOL_USE;
});
Map<String, Object> usage = (Map<String, Object>) root.get("usage");
if(usage != null)
{
long prompt = number(usage.get("prompt_tokens"));
long cached = 0;
if(usage.get("prompt_tokens_details") instanceof Map<?, ?> details)
{
cached = number(details.get("cached_tokens"));
}
response.setUsage(new ModelUsage().withInputTokens(Math.max(0, prompt - cached)).withCacheReadTokens(cached).withOutputTokens(number(usage.get("completion_tokens"))));
}
response.setRequestId((String) root.get("id"));
return (response);
}
/*******************************************************************************
** An HTTP error -> a provider exception (status and error type only).
*******************************************************************************/
@SuppressWarnings("unchecked")
static ModelProviderException toException(int status, String body)
{
String type = null;
try
{
Map<String, Object> root = JsonUtils.toObject(body, new TypeReference<>() {});
Object error = root.get("error");
if(error instanceof Map<?, ?> map)
{
type = map.get("type") instanceof String s ? s : (map.get("code") instanceof String c ? c : null);
}
}
catch(Exception e)
{
// not JSON: the status alone
}
ModelProviderException.Kind kind = switch(status)
{
case 401, 403 -> ModelProviderException.Kind.AUTHENTICATION;
case 429 -> ModelProviderException.Kind.RATE_LIMITED;
case 503, 529 -> ModelProviderException.Kind.OVERLOADED;
case 400, 404, 413, 422 -> ModelProviderException.Kind.INVALID_REQUEST;
case 408 -> ModelProviderException.Kind.TIMEOUT;
default -> status >= 500 ? ModelProviderException.Kind.SERVER : ModelProviderException.Kind.OTHER;
};
return (new ModelProviderException(kind, status, "HTTP " + status + (type == null ? "" : " (" + type + ")")));
}
/*******************************************************************************
**
*******************************************************************************/
private static long number(Object value)
{
return (value instanceof Number n ? n.longValue() : 0);
}
/*******************************************************************************
** Never includes the key.
*******************************************************************************/
@Override
public String toString()
{
return ("OpenAiCompatibleModelProvider{name=" + name + ", baseUrl=" + baseUrl + ", apiKey=" + (apiKey == null ? "none" : "***") + "}");
}
/*******************************************************************************
** Builder.
*******************************************************************************/
public static class Builder
{
private String name = "openai";
private String baseUrl = "https://api.openai.com/v1";
private String apiKey;
private Duration requestTimeout = Duration.ofSeconds(120);
private Boolean strictTools = false;
/*******************************************************************************
** The provider's registration name.
*******************************************************************************/
public Builder name(String name)
{
this.name = name;
return (this);
}
/*******************************************************************************
** e.g. https://api.openai.com/v1 or http://localhost:11434/v1
*******************************************************************************/
public Builder baseUrl(String baseUrl)
{
this.baseUrl = baseUrl;
return (this);
}
/*******************************************************************************
** The key, or a reference such as "${env.OPENAI_API_KEY}".
*******************************************************************************/
public Builder apiKey(String apiKey)
{
this.apiKey = apiKey;
return (this);
}
/*******************************************************************************
**
*******************************************************************************/
public Builder requestTimeout(Duration requestTimeout)
{
this.requestTimeout = requestTimeout;
return (this);
}
/*******************************************************************************
**
*******************************************************************************/
public Builder strictTools(Boolean strictTools)
{
this.strictTools = strictTools;
return (this);
}
/*******************************************************************************
**
*******************************************************************************/
public OpenAiCompatibleModelProvider build()
{
if(!StringUtils.hasContent(name) || !StringUtils.hasContent(baseUrl))
{
throw (new IllegalArgumentException("name and baseUrl are required"));
}
return (new OpenAiCompatibleModelProvider(this));
}
}
}
@@ -0,0 +1,287 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.marketing.email;
import java.util.EnumSet;
import java.util.List;
import java.util.Set;
import com.kingsrook.qbits.email.api.EmailApi;
import com.kingsrook.qbits.email.api.EmailApiCategory;
import com.kingsrook.qbits.email.api.EmailApiMessage;
import com.kingsrook.qbits.email.api.EmailApiReadiness;
import com.kingsrook.qbits.email.api.EmailApiReadinessCheck;
import com.kingsrook.qbits.email.api.EmailApiRequest;
import com.kingsrook.qbits.email.api.EmailApiResult;
import com.kingsrook.qbits.marketing.model.MarketingExecution;
import com.kingsrook.qbits.marketing.model.enums.ChannelType;
import com.kingsrook.qbits.marketing.transport.MarketingTransport;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.utils.StringUtils;
import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
/*******************************************************************************
** Email through qbit-email: the durable outbox, the RFC 8058 unsubscribe
** link and headers, and suppression enforcement all live there.
**
** This is the ONLY class in this qbit that references qbit-email classes
** (the dependency is optional). Callers reach it only after
** isInstalled(...) is true; otherwise QqqMessagingEmailTransport is the
** fallback.
**
** Content: a campaign's content row holds a title and a plain body (no
** template), so each message is an ad-hoc Markdown email: subject = title,
** body = body plus the call to action as a link. No qbit-email template is
** registered (nothing here needs a variable schema).
**
** One message per recipient, category MARKETING, idempotency key
** marketing:CONTENT:APPROVAL:RECIPIENT-KEY (the execution's own stable key
** plus the recipient key, never an address), and the execution row as the
** linked record.
**
** Outcomes: QUEUED is ACCEPTED (in qbit-email's outbox, not delivered).
** SUPPRESSED is a skip, not a failure: the address opted out or bounced, so
** nothing is sent for it (the result says how many were skipped; if every
** recipient was suppressed the result is ACCEPTED with providerStatus
** all_suppressed). REFUSED is REJECTED with qbit-email's reasons. An
** exception is AMBIGUOUS, reconcilable through lookup (the idempotency key
** means a repeat can never send twice).
*******************************************************************************/
public class QbitEmailTransport implements MarketingTransport
{
private static final QLogger LOG = QLogger.getLogger(QbitEmailTransport.class);
public static final String DEFAULT_NAME = "qbit_email";
private static final String UNSUBSCRIBE_KEY_NAME = "QQQ_EMAIL_UNSUBSCRIBE_KEY";
private String name = DEFAULT_NAME;
/*******************************************************************************
** Is qbit-email on the classpath AND produced in this instance?
*******************************************************************************/
public static boolean isInstalled(QInstance qInstance)
{
try
{
Class.forName("com.kingsrook.qbits.email.api.EmailApi");
return (EmailApi.isInstalled(qInstance));
}
catch(ClassNotFoundException | LinkageError e)
{
return (false);
}
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public String getName()
{
return (name);
}
/*******************************************************************************
** OPERATIONAL when the qbit-email outbox is produced and marketing's opt-out
** prerequisites (public base URL and unsubscribe key) are configured; the
** provider itself is checked by qbit-email at send time and a problem there
** comes back as a plain REJECTED reason.
*******************************************************************************/
@Override
public TransportStatus getStatus()
{
if(!isInstalled(QContext.getQInstance()))
{
return (new TransportStatus(TransportStatus.State.NOT_CONFIGURED, "The email qbit (qbit-email) is not installed in this application."));
}
try
{
EmailApiReadiness readiness = EmailApi.readiness(QContext.getQInstance(), EmailApiCategory.MARKETING);
if(readiness.failed(EmailApiReadinessCheck.NOT_INSTALLED))
{
return (new TransportStatus(TransportStatus.State.NOT_CONFIGURED, "The email qbit (qbit-email) is not installed in this application."));
}
if(readiness.failed(EmailApiReadinessCheck.PUBLIC_BASE_URL))
{
return (new TransportStatus(TransportStatus.State.NOT_CONFIGURED, "Marketing email needs the email qbit's publicBaseUrl (the unsubscribe link base)."));
}
if(readiness.failed(EmailApiReadinessCheck.UNSUBSCRIBE_KEY))
{
return (new TransportStatus(TransportStatus.State.NOT_CONFIGURED, "Marketing email needs the " + UNSUBSCRIBE_KEY_NAME + " secret (it signs the unsubscribe links)."));
}
}
catch(QException e)
{
LOG.warn("Could not check email qbit readiness", logPair("error", e.getClass().getSimpleName()));
return (new TransportStatus(TransportStatus.State.CONFIGURED, "Could not check the email qbit's configuration; retry when its configuration can be read."));
}
return (new TransportStatus(TransportStatus.State.OPERATIONAL, "Queues through the email qbit's outbox with its unsubscribe link and suppression list."));
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public Set<ChannelType> getSupportedChannelTypes()
{
return (EnumSet.of(ChannelType.EMAIL));
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public TransportResult send(TransportRequest request)
{
TransportStatus status = getStatus();
if(!status.isUsable())
{
return (new TransportResult(TransportResult.Outcome.RETRYABLE_FAILURE, null, "not_configured", status.detail()));
}
int queued = 0;
int duplicates = 0;
int suppressed = 0;
for(Recipient recipient : request.recipients())
{
if(!StringUtils.hasContent(recipient.address()))
{
return (new TransportResult(queued + duplicates == 0 ? TransportResult.Outcome.REJECTED : TransportResult.Outcome.AMBIGUOUS, null, "no_address",
"A recipient has no address" + (queued + duplicates == 0 ? "." : " after " + (queued + duplicates) + " message(s) were queued.")));
}
try
{
EmailApiResult result = EmailApi.send(emailRequest(request, recipient));
switch(result.status())
{
case QUEUED -> queued++;
case DUPLICATE -> duplicates++;
case SUPPRESSED -> suppressed++;
default ->
{
String reasons = result.reasons() == null || result.reasons().isEmpty() ? "no reason given" : String.join("; ", result.reasons());
LOG.info("Email qbit refused a marketing message", logPair("executionId", request.executionId()), logPair("recipientKey", recipient.key()));
return (new TransportResult(TransportResult.Outcome.REJECTED, null, "refused",
"The email qbit refused the message: " + reasons + " (" + queued + " queued and " + suppressed + " suppressed before it)."));
}
}
}
catch(Exception e)
{
LOG.warn("Email qbit enqueue failed; outcome unknown", logPair("executionId", request.executionId()), logPair("recipientKey", recipient.key()),
logPair("error", e.getClass().getSimpleName()));
return (new TransportResult(TransportResult.Outcome.AMBIGUOUS, null, "error",
"The email qbit raised " + e.getClass().getSimpleName() + " after " + (queued + duplicates) + " of " + request.recipients().size()
+ " message(s); a repeat cannot send twice, and lookup can confirm what was queued."));
}
}
String receipt = "qbit-email:" + baseKey(request);
String summary = queued + " queued, " + suppressed + " skipped as suppressed, " + duplicates + " already queued (queued is not proof of delivery).";
if(queued == 0 && suppressed == 0)
{
return (new TransportResult(TransportResult.Outcome.DUPLICATE, receipt, "already_queued", summary));
}
return (new TransportResult(TransportResult.Outcome.ACCEPTED, receipt, queued == 0 ? "all_suppressed" : "queued", summary));
}
/*******************************************************************************
** Finds what qbit-email queued for the execution (any message under its key).
*******************************************************************************/
@Override
public TransportLookup lookup(String idempotencyKey)
{
try
{
String prefix = "marketing:" + stripPrefix(idempotencyKey) + ":";
List<EmailApiMessage> rows = EmailApi.findMessagesByIdempotencyKeyPrefix(prefix, 1);
if(rows.isEmpty())
{
return (new TransportLookup(TransportLookup.State.NOT_FOUND, null));
}
return (new TransportLookup(TransportLookup.State.FOUND_ACCEPTED, "qbit-email:marketing:" + stripPrefix(idempotencyKey)));
}
catch(Exception e)
{
LOG.warn("Email qbit lookup failed", logPair("error", e.getClass().getSimpleName()));
return (new TransportLookup(TransportLookup.State.UNSUPPORTED, null));
}
}
/*******************************************************************************
** One MARKETING message to one recipient.
*******************************************************************************/
private EmailApiRequest emailRequest(TransportRequest request, Recipient recipient)
{
String body = request.body() == null ? "" : request.body();
if(StringUtils.hasContent(request.destinationUrl()) && request.destinationUrl().matches("(?i)https?://\\S+"))
{
String label = StringUtils.hasContent(request.callToAction()) ? request.callToAction().replaceAll("[\\[\\]\\r\\n]", " ").trim() : request.destinationUrl();
body = body + "\n\n[" + label + "](" + request.destinationUrl() + ")";
}
return (new EmailApiRequest()
.withTo(recipient.address())
.withSubject(request.title() == null ? "" : request.title().replaceAll("[\\r\\n]+", " ").trim())
.withMarkdownBody(body)
.withCategory(EmailApiCategory.MARKETING)
.withIdempotencyKey(baseKey(request) + ":" + recipient.key())
.withLinkedRecord(MarketingExecution.TABLE_NAME, request.executionId())
.withSender("marketing", "execution"));
}
/*******************************************************************************
** The per-execution part of the idempotency key.
*******************************************************************************/
private String baseKey(TransportRequest request)
{
return ("marketing:" + stripPrefix(request.idempotencyKey()));
}
/*******************************************************************************
** The execution key is marketing-execution:CONTENT:APPROVAL.
*******************************************************************************/
private static String stripPrefix(String key)
{
return (key == null ? "" : key.replaceFirst("^marketing-execution:", ""));
}
}
@@ -68,6 +68,9 @@ public class MarketingAgentRun extends QRecordEntity
@QField()
private Instant startedAt;
@QField()
private Instant heartbeatAt;
@QField()
private Instant finishedAt;
@@ -460,6 +463,36 @@ public class MarketingAgentRun extends QRecordEntity
}
/*******************************************************************************
** Getter for heartbeatAt
*******************************************************************************/
public Instant getHeartbeatAt()
{
return (this.heartbeatAt);
}
/*******************************************************************************
** Setter for heartbeatAt
*******************************************************************************/
public void setHeartbeatAt(Instant heartbeatAt)
{
this.heartbeatAt = heartbeatAt;
}
/*******************************************************************************
** Fluent setter for heartbeatAt
*******************************************************************************/
public MarketingAgentRun withHeartbeatAt(Instant heartbeatAt)
{
this.heartbeatAt = heartbeatAt;
return (this);
}
/*******************************************************************************
** Getter for finishedAt
*******************************************************************************/
@@ -17,11 +17,15 @@
package com.kingsrook.qbits.marketing;
import java.time.Duration;
import java.time.Instant;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
import com.kingsrook.qbits.assistant.provider.ModelUsage;
import com.kingsrook.qbits.assistant.provider.fake.ScriptedModelProvider;
import com.kingsrook.qbits.assistant.spi.ModelPart;
import com.kingsrook.qbits.assistant.spi.ModelProviderException;
import com.kingsrook.qbits.assistant.spi.ModelResponse;
import com.kingsrook.qbits.assistant.spi.ModelUsage;
import com.kingsrook.qbits.marketing.agent.MarketingAgentService;
import com.kingsrook.qbits.marketing.agent.tools.MarketingTools;
import com.kingsrook.qbits.marketing.approval.ApprovalService;
@@ -45,6 +49,8 @@ import com.kingsrook.qqq.backend.core.actions.permissions.PermissionEnforcement;
import com.kingsrook.qqq.backend.core.actions.processes.QProcessCallbackFactory;
import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.context.CapturedContext;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
@@ -281,11 +287,10 @@ class AgentAndSecurityTest extends BaseMarketingTest
/*******************************************************************************
** Runs stop at their model-call limit; a cancelled run stops; the
** concurrency limit refuses extra runs.
** Runs stop at their model-call limit; a cancelled run stops.
*******************************************************************************/
@Test
void testLimitsCancelAndConcurrency() throws QException
void testLimitsAndCancel() throws QException
{
config.withMaxModelCallsPerRun(2);
try
@@ -316,25 +321,112 @@ class AgentAndSecurityTest extends BaseMarketingTest
{
throw (new RuntimeException(e));
}
return (new com.kingsrook.qbits.assistant.provider.ModelResponse().withParts(new java.util.ArrayList<>(List.of(
com.kingsrook.qbits.assistant.provider.ModelPart.toolUse("t1", MarketingTools.LIST_CHANNELS, Map.of())))).withStopReason("tool_use"));
return (new com.kingsrook.qbits.assistant.spi.ModelResponse().withParts(new java.util.ArrayList<>(List.of(
com.kingsrook.qbits.assistant.spi.ModelPart.toolUse("t1", MarketingTools.LIST_CHANNELS, Map.of())))).withStopReason("tool_use"));
});
MarketingAgentRun cancelled = new MarketingAgentService().run(AgentTask.ANALYZE_RESULTS, campaignId, null, null);
assertThat(AgentRunStatus.getById(cancelled.getStatus())).isEqualTo(AgentRunStatus.CANCELLED);
}
MarketingRuntime runtime = MarketingRuntime.current();
assertThat(runtime.tryAcquireAgentPermit()).isTrue();
assertThat(runtime.tryAcquireAgentPermit()).isTrue();
try
/*******************************************************************************
** The concurrency limit is counted in the database: a second service
** instance (another replica on the same backend) is refused while the
** first one's run is RUNNING.
*******************************************************************************/
@Test
void testConcurrencyHoldsAcrossReplicas() throws QException
{
config.withMaxConcurrentAgentRuns(1);
CapturedContext context = QContext.capture();
List<MarketingAgentRun> refused = new ArrayList<>();
model.thenRespond(request ->
{
MarketingAgentRun busy = new MarketingAgentService().run(AgentTask.ANALYZE_RESULTS, campaignId, null, null);
assertThat(busy.getErrorMessage()).contains("Too many agent runs");
}
finally
{
runtime.releaseAgentPermit();
runtime.releaseAgentPermit();
}
try
{
QContext.init(context);
refused.add(new MarketingAgentService().run(AgentTask.ANALYZE_RESULTS, campaignId, null, null));
}
catch(QException e)
{
throw (new RuntimeException(e));
}
return (new ModelResponse().withParts(new ArrayList<>(List.of(ModelPart.text("Done.")))).withStopReason(ModelResponse.STOP_END_TURN));
});
MarketingAgentRun first = new MarketingAgentService().run(AgentTask.ANALYZE_RESULTS, campaignId, null, null);
assertThat(AgentRunStatus.getById(first.getStatus())).isEqualTo(AgentRunStatus.SUCCEEDED);
assertThat(first.getHeartbeatAt()).isNotNull();
assertThat(refused).hasSize(1);
assertThat(AgentRunStatus.getById(refused.get(0).getStatus())).isEqualTo(AgentRunStatus.FAILED);
assertThat(refused.get(0).getErrorMessage()).contains("Too many agent runs").contains("limit 1");
/////////////////////////////////////////////////////////////
// the first run finished, so the next one is let through //
/////////////////////////////////////////////////////////////
model.thenText("Again.");
assertThat(AgentRunStatus.getById(new MarketingAgentService().run(AgentTask.ANALYZE_RESULTS, campaignId, null, null).getStatus())).isEqualTo(AgentRunStatus.SUCCEEDED);
}
/*******************************************************************************
** A RUNNING run whose heartbeat is older than maxRunSeconds plus the grace
** is failed with a plain reason (and no longer counts); a fresh one stays.
*******************************************************************************/
@Test
void testStaleRunningRunIsReaped() throws QException
{
config.withMaxConcurrentAgentRuns(1);
Instant stale = Instant.now().minusSeconds(config.getMaxRunSeconds() + MarketingAgentService.STALE_GRACE_SECONDS + 30);
Integer staleId = MarketingStore.insertInternal(new MarketingAgentRun().withTask(AgentTask.ANALYZE_RESULTS.getId()).withStatus(AgentRunStatus.RUNNING.getId())
.withStartedAt(stale).withHeartbeatAt(stale)).getValueInteger("id");
model.thenText("Ran.");
MarketingAgentRun run = new MarketingAgentService().run(AgentTask.ANALYZE_RESULTS, campaignId, null, null);
assertThat(AgentRunStatus.getById(run.getStatus())).isEqualTo(AgentRunStatus.SUCCEEDED);
QRecord reaped = MarketingStore.get(MarketingAgentRun.TABLE_NAME, staleId);
assertThat(reaped.getValueInteger("status")).isEqualTo(AgentRunStatus.FAILED.getId());
assertThat(reaped.getValueString("errorMessage")).isEqualTo("Stopped responding; the server may have restarted.");
///////////////////////////////////////////////////////////////////
// a RUNNING run with a recent heartbeat is alive and counts //
///////////////////////////////////////////////////////////////////
MarketingStore.insertInternal(new MarketingAgentRun().withTask(AgentTask.ANALYZE_RESULTS.getId()).withStatus(AgentRunStatus.RUNNING.getId())
.withStartedAt(stale).withHeartbeatAt(Instant.now()));
MarketingAgentRun refused = new MarketingAgentService().run(AgentTask.ANALYZE_RESULTS, campaignId, null, null);
assertThat(refused.getErrorMessage()).contains("Too many agent runs");
}
/*******************************************************************************
** At the per-minute model-call limit a run waits for the window to clear
** rather than failing; the wait is bounded by maxRunSeconds.
*******************************************************************************/
@Test
void testModelCallRateWaitsInsteadOfFailing() throws QException
{
config.withMaxModelCallsPerMinute(1);
MarketingAgentService service = new MarketingAgentService().withModelCallWindow(Duration.ofMillis(1500), Duration.ofMillis(50));
model.thenText("One.").thenText("Two.");
assertThat(AgentRunStatus.getById(service.run(AgentTask.ANALYZE_RESULTS, campaignId, null, null).getStatus())).isEqualTo(AgentRunStatus.SUCCEEDED);
long start = System.currentTimeMillis();
MarketingAgentRun second = service.run(AgentTask.ANALYZE_RESULTS, campaignId, null, null);
assertThat(AgentRunStatus.getById(second.getStatus())).isEqualTo(AgentRunStatus.SUCCEEDED);
assertThat(System.currentTimeMillis() - start).isGreaterThanOrEqualTo(500);
///////////////////////////////////////////////////////////////////
// a wait longer than the run's time ends the run, not the JVM //
///////////////////////////////////////////////////////////////////
config.withMaxRunSeconds(1);
MarketingAgentService slow = new MarketingAgentService().withModelCallWindow(Duration.ofSeconds(60), Duration.ofMillis(50));
MarketingAgentRun timedOut = slow.run(AgentTask.ANALYZE_RESULTS, campaignId, null, null);
assertThat(AgentRunStatus.getById(timedOut.getStatus())).isEqualTo(AgentRunStatus.FAILED);
assertThat(timedOut.getErrorMessage()).contains("exceeded 1 seconds").contains("rate limit");
}
}
@@ -21,8 +21,7 @@ import java.io.Serializable;
import java.time.Instant;
import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.assistant.provider.ModelUsage;
import com.kingsrook.qbits.assistant.provider.fake.ScriptedModelProvider;
import com.kingsrook.qbits.assistant.spi.ModelUsage;
import com.kingsrook.qbits.marketing.agent.MarketingAgentService;
import com.kingsrook.qbits.marketing.agent.tools.MarketingTools;
import com.kingsrook.qbits.marketing.measurement.MeasurementReport;
@@ -21,8 +21,10 @@ import java.util.List;
import java.util.Set;
import java.util.stream.Collectors;
import com.kingsrook.qbits.marketing.agent.KernelToolRegistration;
import com.kingsrook.qbits.marketing.agent.tools.MarketingTools;
import com.kingsrook.qbits.marketing.metadata.MarketingMetaDataBuilder;
import com.kingsrook.qbits.marketing.security.MarketingPermissions;
import com.kingsrook.qqq.backend.core.actions.ai.AiToolRegistry;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppChildMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
@@ -111,12 +113,16 @@ class MarketingQBitProducerTest extends BaseMarketingTest
/*******************************************************************************
** On QQQ 4.1.0-RC.4 the kernel hook is absent; the qbit still produces.
** RC.6 registers the read tools, excludes writes and tolerates repeat registration.
*******************************************************************************/
@Test
void testKernelHookAbsentOnRc4()
void testKernelHookAvailableOnRc6()
{
assertThat(KernelToolRegistration.isAvailable()).isFalse();
assertThat(KernelToolRegistration.isAvailable()).isTrue();
assertThat(AiToolRegistry.of(qInstance).getTools()).containsOnlyKeys(
MarketingTools.GET_CAMPAIGN_BRIEF, MarketingTools.LIST_PRODUCT_CLAIMS, MarketingTools.LIST_CAMPAIGN_CONTENT,
MarketingTools.GET_CAMPAIGN_RESULTS, MarketingTools.LIST_CHANNELS, MarketingTools.LIST_FINDINGS);
assertThat(KernelToolRegistration.register(qInstance, MarketingTools.all())).isZero();
assertThat(KernelToolRegistration.register(qInstance, List.of())).isZero();
}
@@ -0,0 +1,218 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.marketing;
import java.util.ArrayDeque;
import java.util.ArrayList;
import java.util.Deque;
import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.assistant.spi.AssistantModelProvider;
import com.kingsrook.qbits.assistant.spi.CompletableStreamHandle;
import com.kingsrook.qbits.assistant.spi.ModelCapabilities;
import com.kingsrook.qbits.assistant.spi.ModelEventListener;
import com.kingsrook.qbits.assistant.spi.ModelPart;
import com.kingsrook.qbits.assistant.spi.ModelProviderException;
import com.kingsrook.qbits.assistant.spi.ModelRequest;
import com.kingsrook.qbits.assistant.spi.ModelResponse;
import com.kingsrook.qbits.assistant.spi.ModelUsage;
import com.kingsrook.qbits.assistant.spi.StreamHandle;
import com.kingsrook.qqq.backend.core.utils.JsonUtils;
/*******************************************************************************
** Synchronous scripted responses and request snapshots for marketing tests.
*******************************************************************************/
class ScriptedModelProvider implements AssistantModelProvider
{
private final String name;
private final Deque<Step> steps = new ArrayDeque<>();
private final List<ModelRequest> requests = new ArrayList<>();
private final List<String> requestJson = new ArrayList<>();
private ModelUsage usagePerCall = new ModelUsage();
/*******************************************************************************
** Register under the name selected by the test configuration.
*******************************************************************************/
ScriptedModelProvider(String name)
{
this.name = name;
}
/*******************************************************************************
** Queue a complete answer.
*******************************************************************************/
ScriptedModelProvider thenText(String text)
{
return (thenRespond(request -> new ModelResponse().withParts(List.of(ModelPart.text(text))).withStopReason(ModelResponse.STOP_END_TURN)));
}
/*******************************************************************************
** Queue a single tool call.
*******************************************************************************/
ScriptedModelProvider thenToolUse(String toolName, Map<String, Object> input)
{
return (thenToolUses(null, List.<Object[]>of(new Object[] { toolName, input })));
}
/*******************************************************************************
** Queue parallel tool calls using the test fixtures' name/input pairs.
*******************************************************************************/
@SuppressWarnings("unchecked")
ScriptedModelProvider thenToolUses(String preamble, List<Object[]> calls)
{
return (thenRespond(request ->
{
List<ModelPart> parts = new ArrayList<>();
if(preamble != null)
{
parts.add(ModelPart.text(preamble));
}
for(Object[] call : calls)
{
parts.add(ModelPart.toolUse("tool_" + requests.size() + "_" + parts.size(), (String) call[0], (Map<String, Object>) call[1]));
}
return (new ModelResponse().withParts(parts).withStopReason(ModelResponse.STOP_TOOL_USE));
}));
}
/*******************************************************************************
** Queue a provider failure, including usage already incurred.
*******************************************************************************/
ScriptedModelProvider thenError(ModelProviderException.Kind kind, String message)
{
return (thenRespond(request ->
{
throw (new ModelProviderException(kind, null, message).withPartialUsage(new ModelUsage().add(usagePerCall)));
}));
}
/*******************************************************************************
** Queue a response that can inspect the request or change test state.
*******************************************************************************/
ScriptedModelProvider thenRespond(Step step)
{
steps.add(step);
return (this);
}
/*******************************************************************************
** Set usage for budget and cost assertions.
*******************************************************************************/
ScriptedModelProvider withUsagePerCall(ModelUsage usage)
{
usagePerCall = usage;
return (this);
}
/*******************************************************************************
** Provider registration name.
*******************************************************************************/
@Override
public String getName()
{
return (name);
}
/*******************************************************************************
** Exercise the agent's prompt-cache hints.
*******************************************************************************/
@Override
public ModelCapabilities getCapabilities(String modelId)
{
return (new ModelCapabilities().withPromptCaching(true).withThinking(false));
}
/*******************************************************************************
** Record before responding, preserving each turn before the agent mutates it.
*******************************************************************************/
@Override
public StreamHandle stream(ModelRequest request, ModelEventListener listener)
{
requests.add(request.copy());
requestJson.add(JsonUtils.toJson(request));
CompletableStreamHandle handle = new CompletableStreamHandle();
try
{
ModelResponse response = steps.removeFirst().respond(request);
listener.onTextDelta(response.getText());
response.setUsage(new ModelUsage().add(usagePerCall));
response.setRequestId("request_" + requests.size());
handle.complete(response);
}
catch(ModelProviderException e)
{
handle.fail(e);
}
return (handle);
}
/*******************************************************************************
** Request copies for tool exposure assertions.
*******************************************************************************/
List<ModelRequest> getRequests()
{
return (requests);
}
/*******************************************************************************
** Serialized snapshots for data-boundary assertions.
*******************************************************************************/
List<String> getRequestJson()
{
return (requestJson);
}
/*******************************************************************************
** A scripted step may return a response or a checked provider failure.
*******************************************************************************/
@FunctionalInterface
interface Step
{
/*******************************************************************************
** Respond to one model request.
*******************************************************************************/
ModelResponse respond(ModelRequest request) throws ModelProviderException;
}
}
@@ -25,13 +25,17 @@ import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.concurrent.atomic.AtomicReference;
import com.kingsrook.qbits.assistant.provider.ModelMessage;
import com.kingsrook.qbits.assistant.provider.ModelPart;
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
import com.kingsrook.qbits.assistant.provider.ModelRequest;
import com.kingsrook.qbits.assistant.provider.ModelResponse;
import com.kingsrook.qbits.assistant.provider.ModelToolDefinition;
import com.kingsrook.qbits.assistant.provider.SystemBlock;
import com.kingsrook.qbits.assistant.spi.ModelMessage;
import com.kingsrook.qbits.assistant.spi.ModelPart;
import com.kingsrook.qbits.assistant.spi.ModelProviderException;
import com.kingsrook.qbits.assistant.spi.ModelRequest;
import com.kingsrook.qbits.assistant.spi.ModelResponse;
import com.kingsrook.qbits.assistant.spi.ModelToolDefinition;
import com.kingsrook.qbits.assistant.spi.ProviderSettings;
import com.kingsrook.qbits.assistant.spi.SystemBlock;
import com.kingsrook.qbits.assistant.provider.openai.OpenAiCompatibleModelProvider;
import com.kingsrook.qbits.marketing.MarketingQBitConfig;
import com.kingsrook.qqq.backend.core.instances.QMetaDataVariableInterpreter;
import com.sun.net.httpserver.HttpServer;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
@@ -41,8 +45,8 @@ import static org.assertj.core.api.Assertions.assertThatThrownBy;
/*******************************************************************************
** The OpenAI-compatible provider against a local fixture server (no network,
** no real key): request mapping, tool calls, usage, error kinds, and the key
** The published OpenAI-compatible provider against a local fixture server
** with no real key: request mapping, tool calls, usage, error kinds, and the key
** never appearing in toString.
*******************************************************************************/
class OpenAiCompatibleModelProviderTest
@@ -94,8 +98,9 @@ class OpenAiCompatibleModelProviderTest
*******************************************************************************/
private OpenAiCompatibleModelProvider provider(String key)
{
return (OpenAiCompatibleModelProvider.builder().name("local").baseUrl("http://127.0.0.1:" + server.getAddress().getPort() + "/v1").apiKey(key)
.requestTimeout(Duration.ofSeconds(5)).build());
return (new OpenAiCompatibleModelProvider(new ProviderSettings().withKind("OPENAI_COMPATIBLE")
.withBaseUrl("http://127.0.0.1:" + server.getAddress().getPort() + "/v1")
.withApiKey(key == null ? null : new QMetaDataVariableInterpreter().interpret(key)).withRequestTimeoutSeconds(5)));
}
@@ -105,7 +110,7 @@ class OpenAiCompatibleModelProviderTest
*******************************************************************************/
private static ModelRequest request()
{
return (new ModelRequest().withModel("test-model").withMaxTokens(100)
return (new ModelRequest().withModel("test-model").withMaxTokens(100).withStreaming(false)
.withSystem(List.of(new SystemBlock("Be truthful.", null)))
.withTools(List.of(new ModelToolDefinition().withName("marketing_list_channels").withDescription("Channels")
.withInputSchema(Map.of("type", "object", "properties", Map.of()))))
@@ -129,7 +134,11 @@ class OpenAiCompatibleModelProviderTest
"usage":{"prompt_tokens":120,"completion_tokens":30,"prompt_tokens_details":{"cached_tokens":20}}}
""" });
ModelResponse response = provider("${env.MARKETING_TEST_KEY_THAT_IS_NOT_SET}").stream(request(), text -> {}).await(Duration.ofSeconds(5));
MarketingQBitConfig config = new MarketingQBitConfig()
.withModelProvider(provider("${env.MARKETING_TEST_KEY_THAT_IS_NOT_SET}"))
.withProviderName("openai-compatible").withModel("configured-model");
ModelResponse response = config.getModelProviders().get(config.getProviderName())
.stream(request().withModel(config.getModel()), text -> {}).await(Duration.ofSeconds(5));
assertThat(response.getStopReason()).isEqualTo(ModelResponse.STOP_TOOL_USE);
assertThat(response.getText()).isEqualTo("Checking.");
assertThat(response.getToolUses()).hasSize(1);
@@ -141,7 +150,7 @@ class OpenAiCompatibleModelProviderTest
String body = lastBody.get();
assertThat(body).contains("\"role\":\"system\"").contains("Be truthful.").contains("\"tool_calls\"").contains("\"role\":\"tool\"").contains("\"tool_call_id\":\"call_1\"")
.contains("\"max_completion_tokens\":100").contains("marketing_list_channels");
.contains("\"max_tokens\":100").contains("marketing_list_channels").contains("\"model\":\"configured-model\"");
assertThat(lastAuth.get()).as("an unset key reference means no Authorization header").isNull();
}
@@ -162,7 +171,8 @@ class OpenAiCompatibleModelProviderTest
.isInstanceOfSatisfying(ModelProviderException.class, e ->
{
assertThat(e.getKind()).isEqualTo(ModelProviderException.Kind.AUTHENTICATION);
assertThat(e.getMessage()).contains("401").contains("invalid_api_key").doesNotContain("sk-test");
assertThat(e.getHttpStatus()).isEqualTo(401);
assertThat(e.getMessage()).doesNotContain("sk-FAKE-TEST-KEY-000000");
});
assertThat(lastAuth.get()).isEqualTo("Bearer sk-FAKE-TEST-KEY-000000");
@@ -174,8 +184,9 @@ class OpenAiCompatibleModelProviderTest
assertThatThrownBy(() -> provider.stream(request(), text -> {}).await(Duration.ofSeconds(5)))
.isInstanceOfSatisfying(ModelProviderException.class, e -> assertThat(e.getKind()).isEqualTo(ModelProviderException.Kind.INVALID_REQUEST));
responses.add(new Object[] { 200, "{\"choices\":[{\"finish_reason\":\"content_filter\",\"message\":{\"content\":\"\"}}]}" });
assertThat(OpenAiCompatibleModelProvider.toException(503, "").getKind()).isEqualTo(ModelProviderException.Kind.OVERLOADED);
responses.add(new Object[] { 503, "" });
assertThatThrownBy(() -> provider.stream(request(), text -> {}).await(Duration.ofSeconds(5)))
.isInstanceOfSatisfying(ModelProviderException.class, e -> assertThat(e.getKind()).isEqualTo(ModelProviderException.Kind.OVERLOADED));
}
@@ -196,4 +207,29 @@ class OpenAiCompatibleModelProviderTest
assertThat(streamed).containsExactly("Done.");
}
/*******************************************************************************
** The agent's default streaming mode receives text and usage from SSE.
*******************************************************************************/
@Test
void testStreaming() throws Exception
{
responses.add(new Object[] { 200, """
data: {"choices":[{"index":0,"delta":{"content":"Done."},"finish_reason":null}]}
data: {"choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":12,"completion_tokens":3}}
data: [DONE]
""" });
List<String> streamed = new ArrayList<>();
ModelResponse response = provider(null).stream(request().withStreaming(true), streamed::add).await(Duration.ofSeconds(5));
assertThat(streamed).containsExactly("Done.");
assertThat(response.getText()).isEqualTo("Done.");
assertThat(response.getUsage().getInputTokens()).isEqualTo(12);
assertThat(response.getUsage().getOutputTokens()).isEqualTo(3);
assertThat(lastBody.get()).contains("\"stream\":true");
}
}
@@ -0,0 +1,290 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.marketing.email;
import java.util.ArrayList;
import java.util.List;
import com.kingsrook.qbits.email.EmailQBitConfig;
import com.kingsrook.qbits.email.EmailQBitProducer;
import com.kingsrook.qbits.email.api.EmailApi;
import com.kingsrook.qbits.email.api.EmailApiCategory;
import com.kingsrook.qbits.email.api.EmailApiMessage;
import com.kingsrook.qbits.marketing.MarketingQBitConfig;
import com.kingsrook.qbits.marketing.MarketingQBitProducer;
import com.kingsrook.qbits.marketing.MarketingRuntime;
import com.kingsrook.qbits.marketing.model.enums.ChannelType;
import com.kingsrook.qbits.marketing.security.MarketingPermissions;
import com.kingsrook.qbits.marketing.transport.MarketingTransport;
import com.kingsrook.qbits.marketing.transport.MarketingTransport.Recipient;
import com.kingsrook.qbits.marketing.transport.MarketingTransport.TransportRequest;
import com.kingsrook.qbits.marketing.transport.MarketingTransport.TransportResult;
import com.kingsrook.qbits.marketing.transport.QqqMessagingEmailTransport;
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.instances.QInstanceValidator;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType;
import com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData;
import com.kingsrook.qqq.backend.core.model.session.QSession;
import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryBackendModule;
import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryRecordStore;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** QbitEmailTransport against a real qbit-email outbox (memory backend).
*******************************************************************************/
class QbitEmailTransportTest
{
private static final String BACKEND = "memory";
/*******************************************************************************
**
*******************************************************************************/
@AfterEach
void afterEach()
{
QContext.clear();
}
/*******************************************************************************
** An instance with marketing and (optionally) the email qbit.
*******************************************************************************/
private QInstance instance(boolean withEmail, String publicBaseUrl, String unsubscribeKey, MarketingQBitConfig marketingConfig) throws Exception
{
MemoryRecordStore.getInstance().reset();
QInstance qInstance = new QInstance();
qInstance.addBackend(new QBackendMetaData().withName(BACKEND).withBackendType(MemoryBackendModule.class));
qInstance.withInstanceDefaultAuthentication(new QAuthenticationMetaData().withName("anonymous").withType(QAuthenticationType.FULLY_ANONYMOUS));
if(withEmail)
{
EmailQBitConfig emailConfig = new EmailQBitConfig().withBackendName(BACKEND).withPublicBaseUrl(publicBaseUrl)
.withSecretResolver(name -> "QQQ_EMAIL_UNSUBSCRIBE_KEY".equals(name) ? unsubscribeKey : null);
new EmailQBitProducer().withQBitConfig(emailConfig).produce(qInstance);
}
new MarketingQBitProducer().withQBitConfig(marketingConfig.withBackendName(BACKEND)).produce(qInstance);
new QInstanceValidator().validate(qInstance);
List<String> permissions = new ArrayList<>(MarketingPermissions.ALL);
permissions.addAll(List.of("email.view", "email.send", "email.manage", "email.admin", "email.hasAccess"));
QContext.init(qInstance, new QSession().withPermissions(permissions));
return (qInstance);
}
/*******************************************************************************
**
*******************************************************************************/
private void provider() throws QException
{
brand();
insert("email_provider", new QRecord().withValue("name", "relay").withValue("label", "Relay").withValue("kind", "SMTP").withValue("host", "127.0.0.1")
.withValue("port", 2525).withValue("security", "NONE").withValue("fromAddress", "sender@example.com").withValue("enabled", true).withValue("isDefault", true));
}
/*******************************************************************************
** Marketing email needs a default brand with a postal address.
*******************************************************************************/
private void brand() throws QException
{
insert("email_brand", new QRecord().withValue("name", "default").withValue("label", "Example").withValue("isDefault", true)
.withValue("postalAddress", "1 Main St, Springfield"));
}
/*******************************************************************************
** Configure the email qbit's editable tables through ordinary QQQ actions.
*******************************************************************************/
private void insert(String table, QRecord record) throws QException
{
var output = new InsertAction().execute(new InsertInput(table).withRecords(List.of(record)));
assertThat(output.getRecords()).hasSize(1).allSatisfy(row -> assertThat(row.getErrors()).isNullOrEmpty());
}
/*******************************************************************************
**
*******************************************************************************/
private TransportRequest request(String key, Recipient... recipients)
{
return (new TransportRequest(key, 7, 1, "Email", ChannelType.EMAIL, "Big news", "Hello **there**.", "https://example.org/try", "Try it", List.of(recipients)));
}
/*******************************************************************************
** One MARKETING message per recipient with the right key and link; a repeat
** is DUPLICATE; the qqq_email name resolves to the transport.
*******************************************************************************/
@Test
void testSendsOneMarketingMessagePerRecipient() throws Exception
{
instance(true, "https://app.example.org", "k", new MarketingQBitConfig());
provider();
MarketingTransport transport = MarketingRuntime.current().getTransport(QqqMessagingEmailTransport.DEFAULT_NAME);
assertThat(transport).isInstanceOf(QbitEmailTransport.class);
assertThat(transport.getStatus().state()).isEqualTo(MarketingTransport.TransportStatus.State.OPERATIONAL);
assertThat(transport.getSupportedChannelTypes()).containsExactly(ChannelType.EMAIL);
TransportRequest request = request("marketing-execution:3:4", new Recipient("crm-contact:1", "a@example.org"), new Recipient("crm-contact:2", "b@example.org"));
TransportResult result = transport.send(request);
assertThat(result.message()).as(result.message()).isNotNull();
assertThat(result.outcome()).as(result.message()).isEqualTo(TransportResult.Outcome.ACCEPTED);
assertThat(result.providerStatus()).isEqualTo("queued");
List<EmailApiMessage> rows = EmailApi.findMessagesByIdempotencyKeyPrefix("marketing:3:4:", 10);
assertThat(rows).hasSize(2);
assertThat(rows).extracting(EmailApiMessage::idempotencyKey).containsExactlyInAnyOrder("marketing:3:4:crm-contact:1", "marketing:3:4:crm-contact:2");
assertThat(rows).allSatisfy(r ->
{
assertThat(r.category()).isEqualTo(EmailApiCategory.MARKETING);
assertThat(r.linkedTableName()).isEqualTo("marketing_execution");
assertThat(r.linkedRecordId()).isEqualTo("7");
assertThat(r.templateName()).isNull();
assertThat(r.status()).isEqualTo("QUEUED");
});
// The public API intentionally omits content; inspect it as an email admin.
List<QRecord> messages = new QueryAction().execute(new QueryInput("email_message")
.withShouldFetchHeavyFields(true).withShouldOmitHiddenFields(false)).getRecords();
assertThat(messages).hasSize(2).allSatisfy(r ->
{
assertThat(r.getValueString("subject")).isEqualTo("Big news");
assertThat(r.getValueString("senderQbit")).isEqualTo("marketing");
assertThat(r.getValueString("senderProcess")).isEqualTo("execution");
assertThat(r.getValueString("htmlBody")).contains("https://example.org/try").contains("nsubscribe");
assertThat(r.getValueString("toAddresses")).doesNotContain(",");
});
assertThat(transport.send(request).outcome()).isEqualTo(TransportResult.Outcome.DUPLICATE);
assertThat(EmailApi.findMessagesByIdempotencyKeyPrefix("marketing:3:4:", 10)).hasSize(2);
assertThat(transport.lookup("marketing-execution:3:4").state()).isEqualTo(MarketingTransport.TransportLookup.State.FOUND_ACCEPTED);
assertThat(transport.lookup("marketing-execution:9:9").state()).isEqualTo(MarketingTransport.TransportLookup.State.NOT_FOUND);
}
/*******************************************************************************
** A suppressed address is skipped, not failed; with every recipient
** suppressed the result is still ACCEPTED (all_suppressed).
*******************************************************************************/
@Test
void testSuppressedIsSkipped() throws Exception
{
instance(true, "https://app.example.org", "k", new MarketingQBitConfig());
provider();
insert("email_suppression", new QRecord().withValue("address", "gone@example.org").withValue("reason", "UNSUBSCRIBED").withValue("scope", "MARKETING"));
MarketingTransport transport = new QbitEmailTransport();
TransportResult result = transport.send(request("marketing-execution:1:1", new Recipient("crm-contact:9", "gone@example.org")));
assertThat(result.outcome()).isEqualTo(TransportResult.Outcome.ACCEPTED);
assertThat(result.providerStatus()).isEqualTo("all_suppressed");
assertThat(result.message()).contains("1 skipped as suppressed");
TransportResult mixed = transport.send(request("marketing-execution:2:1", new Recipient("crm-contact:9", "gone@example.org"), new Recipient("crm-contact:10", "ok@example.org")));
assertThat(mixed.providerStatus()).isEqualTo("queued");
assertThat(mixed.message()).contains("1 queued, 1 skipped");
}
/*******************************************************************************
** Without the unsubscribe key or public base URL the transport is not
** configured and says why; a refusal by the email qbit is REJECTED with
** its reason.
*******************************************************************************/
@Test
void testNotConfiguredAndRefused() throws Exception
{
instance(true, "https://app.example.org", null, new MarketingQBitConfig());
QbitEmailTransport transport = new QbitEmailTransport();
assertThat(transport.getStatus().state()).isEqualTo(MarketingTransport.TransportStatus.State.NOT_CONFIGURED);
assertThat(transport.getStatus().detail()).contains("QQQ_EMAIL_UNSUBSCRIBE_KEY");
assertThat(transport.send(request("k", new Recipient("a", "a@example.org"))).outcome()).isEqualTo(TransportResult.Outcome.RETRYABLE_FAILURE);
instance(true, null, "k", new MarketingQBitConfig());
assertThat(new QbitEmailTransport().getStatus().detail()).contains("publicBaseUrl");
instance(true, "https://app.example.org", "k", new MarketingQBitConfig());
assertThat(transport.getStatus().state()).isEqualTo(MarketingTransport.TransportStatus.State.OPERATIONAL);
TransportResult refused = new QbitEmailTransport().send(request("marketing-execution:5:5", new Recipient("a", "a@example.org")));
assertThat(refused.outcome()).isEqualTo(TransportResult.Outcome.REJECTED);
assertThat(refused.message()).contains("refused the message");
provider();
TransportResult noAddress = new QbitEmailTransport().send(request("marketing-execution:6:6", new Recipient("a", null)));
assertThat(noAddress.outcome()).isEqualTo(TransportResult.Outcome.REJECTED);
}
/*******************************************************************************
** A missing postal address is a send-time refusal, not NOT_CONFIGURED.
*******************************************************************************/
@Test
void testMissingBrandIsRefusedAtSend() throws Exception
{
instance(true, "https://app.example.org", "k", new MarketingQBitConfig());
insert("email_provider", new QRecord().withValue("name", "relay").withValue("label", "Relay").withValue("kind", "SMTP").withValue("host", "127.0.0.1")
.withValue("port", 2525).withValue("security", "NONE").withValue("fromAddress", "sender@example.com").withValue("enabled", true).withValue("isDefault", true));
QbitEmailTransport transport = new QbitEmailTransport();
assertThat(transport.getStatus().state()).isEqualTo(MarketingTransport.TransportStatus.State.OPERATIONAL);
TransportResult result = transport.send(request("marketing-execution:8:8", new Recipient("a", "a@example.org")));
assertThat(result.outcome()).isEqualTo(TransportResult.Outcome.REJECTED);
assertThat(result.message()).contains("postal address");
}
/*******************************************************************************
** Without the email qbit in the instance the name does not resolve to the
** transport, and a host-registered transport always wins.
*******************************************************************************/
@Test
void testFallbackAndHostTransportWin() throws Exception
{
instance(false, null, null, new MarketingQBitConfig());
assertThat(MarketingRuntime.current().getTransport(QqqMessagingEmailTransport.DEFAULT_NAME)).isNull();
assertThat(new QbitEmailTransport().getStatus().state()).isEqualTo(MarketingTransport.TransportStatus.State.NOT_CONFIGURED);
QqqMessagingEmailTransport host = new QqqMessagingEmailTransport().withMessagingProviderName("email").withFromAddress("n@example.org");
instance(true, "https://app.example.org", "k", new MarketingQBitConfig().withTransport(host));
assertThat(MarketingRuntime.current().getTransport(QqqMessagingEmailTransport.DEFAULT_NAME)).isSameAs(host);
}
}
+1
View File
@@ -314,6 +314,7 @@ CREATE TABLE IF NOT EXISTS marketing_agent_run (
model VARCHAR(100),
requested_by_user_id VARCHAR(100),
started_at TIMESTAMP,
heartbeat_at TIMESTAMP,
finished_at TIMESTAMP,
model_calls INTEGER,
tool_calls INTEGER,