diff --git a/AGENTS.md b/AGENTS.md index 3a6017f..42243c4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -46,7 +46,8 @@ Services: `approval.ApprovalService`, `execution.ExecutionService`, `measurement ## Dependencies - Java 21. -- Parent `com.kingsrook:qbit-build-parent:2.0.0` (qqq 4.0.0). The default-active `qqq-snapshot` profile imports `qqq-bom-pom:4.1.0-RC.5` ahead of it, as in qbit-quick-search (ADR-0007). Remove that activation when the parent moves to 4.1. +- Parent `com.kingsrook:qbit-build-parent:2.1.0` (qqq 4.0.0), with architecture enforcement and strict UI API routes enabled. The default-active `qqq-snapshot` profile imports `qqq-bom-pom:4.1.0-RC.6` ahead of it, as in qbit-quick-search (ADR-0007). Remove that activation when the parent moves to 4.1. +- Assistant provider API and OpenAI-compatible provider `0.1.0-RC.11`; no full assistant dependency. For local verification before publication only, use `-Dqbit-assistant.version=0.1.0-ARCHLOCAL-SNAPSHOT -Dqbit.guardrails.enforce=false` and inspect guardrail findings. - Tests: JUnit 5, AssertJ. ## Release diff --git a/CHANGELOG.md b/CHANGELOG.md index 25f9041..51dd6c9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,11 @@ All notable changes to this project are documented here. The format follows Keep ## [0.1.0] - Unreleased +### Changed +- Resolve architecture finding F5 (#9): depend on `qbit-assistant-provider-api` and `qbit-assistant-provider-openai` `0.1.0-RC.11`, with no full assistant dependency, including in tests. +- Replace the local OpenAI-compatible implementation with the published provider and document `ProviderSettings`, environment-key resolution and provider-name migration. Agent tests use a local scripted double. +- Upgrade QQQ to `4.1.0-RC.6` and `qbit-build-parent` to `2.1.0`; enable architecture enforcement and strict UI API routes with explicit provider API/artifact allowances. + ### Added - Product-contingent possible values for editable campaign, claim, finding, metric and outcome-event references. - Repository scaffold: build, CI and the repository standard files. @@ -35,7 +40,7 @@ All notable changes to this project are documented here. The format follows Keep - Narrow typed tools (6 read, 3 write-proposal), run as the user with core permission enforcement, the untrusted-data envelope and redaction. - Run and step records with usage and cost. - Budgets, concurrency and rate limits, cancellation, and a plain failure reason when no provider is configured. -- **Model providers:** `OpenAiCompatibleModelProvider` (no SDK), plus qbit-assistant's Claude and scripted providers. +- **Model providers:** the published `OpenAiCompatibleModelProvider` (no SDK), host-supplied providers such as Claude, and a local scripted test double. - **AI tool kernel:** the read tools are contributed through `AiToolRegistry` when QQQ has it (QRun-IO/qqq#1096). - **UI:** - The Marketing app; its home is the Marketing dashboard (Needs Attention, Campaigns, Agent Activity, Configuration Status). diff --git a/README.md b/README.md index 3425df0..6520953 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ > Marketing for QQQ applications: products and their verified evidence, campaigns, content with revision history, exact human approval, controlled execution, honest measurement, and a marketing agent that prepares work but never publishes. **Status:** 0.1.0, not yet published. -- Built against QQQ `4.1.0-RC.5`. +- Built against QQQ `4.1.0-RC.6` with `qbit-build-parent` `2.1.0`. - Hosted in QRun-IO/Voyage next to qbit-crm, on a local branch. - The real integrations are implemented but unconfigured. See [what is configured](#what-is-implemented-what-is-not-configured). @@ -25,7 +25,7 @@ It lives inside the host QQQ application: its navigation, authentication, permis | **Measurement** | Delivery, awareness, engagement, activation, repeat use and qualified demand are reported separately. Each metric shows raw events and unique actors. "Not measured" is never shown as 0. Test data is kept apart. | | **CRM** | Uses the host CRM's own contacts, organizations, consent, suppression, lists, form submissions and activities through a narrow gateway. A qbit-crm adapter is included. Nothing is copied, and nobody becomes a lead without the campaign's explicit rule. | | **Agent** | Plans campaigns, drafts and revises content, flags unsupported claims, stale sources and readiness gaps, and analyzes results to recommend the next experiment. It uses narrow typed tools, runs as the requesting user, sees data only as untrusted input, and sees no CRM personal data. Every run is recorded with its steps, provider, model, tokens and cost. It can create drafts and findings; it cannot approve, schedule or send. | -| **Models** | Interchangeable providers through qbit-assistant's `AssistantModelProvider`: Claude (`qbit-assistant-provider-anthropic`), any OpenAI-compatible endpoint (included), and a scripted provider for tests. With no provider configured, the agent says so and everything else works. | +| **Models** | Interchangeable providers through `qbit-assistant-provider-api`'s `AssistantModelProvider`: OpenAI-compatible endpoints through `qbit-assistant-provider-openai` (included), or a host-supplied provider such as Claude. Tests use a local scripted double. With no provider configured, the agent says so and everything else works. Installing marketing does not install the assistant runtime, tables or routes. | ## Install @@ -38,14 +38,24 @@ It lives inside the host QQQ application: its navigation, authentication, permis ``` ```java +import com.kingsrook.qbits.assistant.provider.ProviderSettings; +import com.kingsrook.qbits.assistant.provider.openai.OpenAiCompatibleModelProvider; +import com.kingsrook.qqq.backend.core.instances.QMetaDataVariableInterpreter; + +var provider = new OpenAiCompatibleModelProvider(new ProviderSettings() + .withKind("OPENAI_COMPATIBLE") + .withBaseUrl("https://api.openai.com/v1") + .withApiKey(new QMetaDataVariableInterpreter().interpret("${env.MARKETING_OPENAI_API_KEY}"))); + new MarketingQBitProducer().withQBitConfig(new MarketingQBitConfig() .withBackendName("rdbms") // where the marketing_* tables live .withCrmGateway(new QbitCrmGateway()) // optional: qbit-crm .withSchedulerName("scheduler") // optional: runs the execution worker .withTransport(new QqqMessagingEmailTransport() // optional: email via your QQQ messaging provider .withMessagingProviderName("email").withFromAddress("news@example.org")) - .withModelProvider(AnthropicModelProvider.fromEnvironment()) // optional: only when ANTHROPIC_API_KEY is set - .withModel("claude-sonnet-5-5")) + .withModelProvider(provider) // optional: configure only for agent use + .withProviderName("openai-compatible") + .withModel("your-model-id")) .produce(qInstance); ``` @@ -56,6 +66,11 @@ Then: [`docs/OPERATIONS.md`](docs/OPERATIONS.md) covers every setting, the run-books and the limits. +The provider SPI, pricing and OpenAI-compatible implementation are pinned to `0.1.0-RC.11`. +Replace imports of marketing's former `agent.provider.OpenAiCompatibleModelProvider` with the published class above and replace its builder with `ProviderSettings`. +The host resolves the API key reference; `ProviderSettings.withApiKeyEnvName(...)` alone does not resolve it. +Set any existing Marketing Policy provider override to `openai-compatible` as well. The base URL, key environment variable name and model remain host-configurable; omit the key for a local endpoint that needs none. + ## Permissions | Permission | Grants | @@ -102,7 +117,7 @@ When the optional [qbit-email](https://github.com/QRun-IO/qbit-email) qbit is pr | Approval binding, invalidation, separation of duties | Email transport (needs the host's QQQ messaging provider; reports *Not configured* until then, and *Configured* is not *Operational*) | | Durable execution, retries, reconciliation, leases | No social, community, blog or CMS connector exists: those channels are posted by hand | | Controlled test transport (complete) | No analytics importer: awareness and activation metrics stay *Not measured* until something records outcome events | -| Agent loop, tools, budgets, limits, run records | QQQ AI tool kernel registration (needs QRun-IO/qqq#1096; the qbit works without it) | +| Agent loop, tools, budgets, limits, run records; read-tool registration in the QQQ RC.6 AI tool registry | | | CRM gateway and qbit-crm adapter, explicit lead rules | | | Persistence across restart (H2 file, and PostgreSQL in the Voyage host) | | @@ -116,6 +131,16 @@ Scale gaps (single worker per JVM, in-memory rate limits) are listed in [`docs/O Always use the Maven wrapper (Maven 3.9.11); Maven 3.10 cannot read the parent POM. The entities, enums and DDL are generated from one spec: `python -I scripts/generate-model.py .` (commit the output). +Architecture guardrails and strict UI API route checks are enabled. The artifact allow-list preserves the parent's defaults and adds only `qbit-assistant-provider-openai`; the package allow-list also permits the extracted provider SPI and pricing. Parent 2.1.0 matches package prefixes using `prefix + "."`, so the pricing allowance must be `com.kingsrook.qbits.assistant.engine`, not the class name `…engine.ModelPricing`. The API artifact contains only `ModelPricing` in that package; the full assistant artifact remains prohibited. + +Until provider `0.1.0-RC.11` is available on Central, verify against the locally installed assistant develop build only with: + +```bash +./mvnw -Dqbit-assistant.version=0.1.0-ARCHLOCAL-SNAPSHOT -Dqbit.guardrails.enforce=false verify +``` + +This runs the guardrails in report mode so G7 can report the local SNAPSHOT dependencies. Keep the release version and enforcement enabled in the POM. + ## License Apache License 2.0. See [LICENSE](LICENSE) and [NOTICE](NOTICE). diff --git a/docs/DESIGN.md b/docs/DESIGN.md index fef7455..8165625 100644 --- a/docs/DESIGN.md +++ b/docs/DESIGN.md @@ -19,7 +19,7 @@ The work is truthful and auditable: facts come from sources with dates, people a | From | Reused for | |---|---| | **qbit-assistant** (0.1.0-RC.2) | The model-provider SPI: `AssistantModelProvider`, `ModelRequest` / `ModelResponse` / `ModelPart`, `StreamHandle`, `ModelProviderException` with its retryable kinds, and `ModelUsage`. Also the Claude provider (`qbit-assistant-provider-anthropic`), `ScriptedModelProvider` for tests and demos, and `ModelPricing` for cost. Marketing adds an OpenAI-compatible provider that implements the same SPI. There is no second provider abstraction. | -| **QQQ AI tool kernel** (qqq#1078, core) | The tool contract: `AbstractAiTool` / `AiToolKind` (READ, WRITE_PROPOSAL), `AiToolOutcome`, the `qqq_data` untrusted-data envelope (`AiDataEnvelope`), secret redaction (`AiRedactor`), strict input keys, and running as the user's session. The kernel's tool set is fixed in RC.4. So QRun-IO/qqq#1096 adds `AiToolRegistry`; Marketing registers its read tools there when QQQ has it and otherwise runs them in its own runner with the same contract. | +| **QQQ AI tool kernel** (qqq#1078, core) | The tool contract: `AbstractAiTool` / `AiToolKind` (READ, WRITE_PROPOSAL), `AiToolOutcome`, the `qqq_data` untrusted-data envelope (`AiDataEnvelope`), secret redaction (`AiRedactor`), strict input keys, and running as the user's session. The RC.6 baseline includes `AiToolRegistry` (QRun-IO/qqq#1096); Marketing contributes its read tools there and runs its own agent tools with the same contract. | | **QQQ core permission enforcement** (qqq#1061) | Agent tools read with `PermissionEnforcement.ENFORCE`. Service entry points check their step's permission, then run as trusted internal work in an authorized scope, as QQQ does for calls nested in a core action. Record security locks always apply. | | **qbit-assistant / sql-workbench metadata builders** | Explicit builder-produced tables (not annotation scanning), snake_case `marketing_*` names, QQQ standard permission suffixes, `DenyBehavior.HIDDEN`, read-only system tables, record locks, reference DDL in `docs/ddl`, `tableMetaDataCustomizer` and `inferBackendNames`. | | **Website-Backend admin conventions** | Processes are table and record actions, never nav items, with plain labels. Record actions use `StreamedETLWithFrontendProcess` with standard summary lines. App homes are real-data dashboards built from ≤3-column full-width tables. Links use the Next route grammar (`AiLinkBuilder`). | diff --git a/docs/OPERATIONS.md b/docs/OPERATIONS.md index a57754e..bdb2ca1 100644 --- a/docs/OPERATIONS.md +++ b/docs/OPERATIONS.md @@ -48,7 +48,7 @@ Nothing in Marketing stores a credential, and every key is read from the host's | Integration | What to provide | |---|---| | Claude | `ANTHROPIC_API_KEY` (or `AnthropicModelProvider.builder().apiKey(fromSecretStore)`), and a model id such as `claude-sonnet-5-5`. | -| OpenAI-compatible | A base URL (`https://api.openai.com/v1`, `http://localhost:11434/v1` for Ollama, ...), an API key reference such as `${env.MARKETING_OPENAI_API_KEY}` if the server needs one, and a model id. | +| OpenAI-compatible | Use `com.kingsrook.qbits.assistant.provider.openai.OpenAiCompatibleModelProvider` from `qbit-assistant-provider-openai` with `ProviderSettings`: a base URL (`https://api.openai.com/v1`, `http://localhost:11434/v1` for Ollama, ...), a resolved key if the server needs one, and a model id in `MarketingQBitConfig.withModel(...)`. Resolve `${env.MARKETING_OPENAI_API_KEY}` with `QMetaDataVariableInterpreter` before `withApiKey(...)`; setting `apiKeyEnvName` alone does not resolve a key. The provider name is `openai-compatible`, including in Marketing Policy overrides. See the [setup example](../README.md#install). | | Email (qbit-email) | When qbit-email is produced in the instance and no transport named `qqq_email` is registered, email goes through its outbox (category MARKETING). It needs `publicBaseUrl`, the `QQQ_EMAIL_UNSUBSCRIBE_KEY` secret, an enabled default provider and a default brand with a postal address; a missing piece is reported as *Not configured* or as a plain rejected-execution reason. | | Email (QQQ messaging) | A QQQ messaging provider in the host instance (SMTP or SES), its name, and a from-address. After verifying it end to end, `withVerifiedOperational(true)`. Until then the email transport reports *Configured, not verified* and executions are blocked. | diff --git a/pom.xml b/pom.xml index 0cef9a0..c625eae 100644 --- a/pom.xml +++ b/pom.xml @@ -22,10 +22,14 @@ 21 21 0.8.15 - - - - 0.1.0-RC.2 + + 0.1.0-RC.11 + true + true + + + com.kingsrook.qbits.objectstorage.api,com.kingsrook.qbits.assistant.provider,com.kingsrook.qbits.assistant.engine,com.kingsrook.qbits.email + com.kingsrook.qbits:qbit-object-storage,com.kingsrook.qbits:*-api,com.kingsrook.qbits:qbit-assistant-provider-openai,com.kingsrook.qbits:qbit-email 0.1.0-RC.1 2.3.232 @@ -34,7 +38,7 @@ com.kingsrook qbit-build-parent - 2.0.0 + 2.1.0 com.kingsrook.qbits @@ -81,7 +85,12 @@ com.kingsrook.qbits - qbit-assistant + qbit-assistant-provider-api + ${qbit-assistant.version} + + + com.kingsrook.qbits + qbit-assistant-provider-openai ${qbit-assistant.version} @@ -187,14 +196,14 @@ - + qqq-snapshot true - 4.1.0-RC.5 + 4.1.0-RC.6 diff --git a/src/main/java/com/kingsrook/qbits/marketing/agent/provider/OpenAiCompatibleModelProvider.java b/src/main/java/com/kingsrook/qbits/marketing/agent/provider/OpenAiCompatibleModelProvider.java deleted file mode 100644 index f5efcc3..0000000 --- a/src/main/java/com/kingsrook/qbits/marketing/agent/provider/OpenAiCompatibleModelProvider.java +++ /dev/null @@ -1,512 +0,0 @@ -/* - * Copyright 2026 Kingsrook, LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.kingsrook.qbits.marketing.agent.provider; - - -import java.io.IOException; -import java.net.URI; -import java.net.http.HttpClient; -import java.net.http.HttpRequest; -import java.net.http.HttpResponse; -import java.net.http.HttpTimeoutException; -import java.time.Duration; -import java.util.ArrayList; -import java.util.LinkedHashMap; -import java.util.List; -import java.util.Map; -import java.util.concurrent.CompletableFuture; -import com.fasterxml.jackson.core.type.TypeReference; -import com.kingsrook.qbits.assistant.provider.AssistantModelProvider; -import com.kingsrook.qbits.assistant.provider.CompletableStreamHandle; -import com.kingsrook.qbits.assistant.provider.ModelCapabilities; -import com.kingsrook.qbits.assistant.provider.ModelEventListener; -import com.kingsrook.qbits.assistant.provider.ModelMessage; -import com.kingsrook.qbits.assistant.provider.ModelPart; -import com.kingsrook.qbits.assistant.provider.ModelProviderException; -import com.kingsrook.qbits.assistant.provider.ModelRequest; -import com.kingsrook.qbits.assistant.provider.ModelResponse; -import com.kingsrook.qbits.assistant.provider.ModelToolDefinition; -import com.kingsrook.qbits.assistant.provider.ModelUsage; -import com.kingsrook.qbits.assistant.provider.StreamHandle; -import com.kingsrook.qbits.assistant.provider.SystemBlock; -import com.kingsrook.qqq.backend.core.instances.QMetaDataVariableInterpreter; -import com.kingsrook.qqq.backend.core.utils.JsonUtils; -import com.kingsrook.qqq.backend.core.utils.StringUtils; - - -/******************************************************************************* - ** A model provider for any OpenAI-compatible Chat Completions endpoint - ** (POST {baseUrl}/chat/completions): OpenAI, and the many servers that speak - ** the same API (OpenRouter, Ollama, vLLM, LM Studio, LiteLLM, ...). - ** Implements qbit-assistant's AssistantModelProvider, so it plugs into the - ** marketing agent (and the assistant) next to the Anthropic provider. - ** - ** - No SDK: java.net.http. Non-streaming request; the text is delivered to - ** the listener at the end. - ** - The API key is read from the environment / secret reference given to - ** the builder ("${env.NAME}"); it is never logged, and toString never - ** contains it. A provider without a key is allowed for local servers that - ** need none (Ollama). - ** - Errors map to ModelProviderException kinds: 401/403 AUTHENTICATION, - ** 429 RATE_LIMITED, 5xx SERVER / OVERLOADED (503), 400/404/422 - ** INVALID_REQUEST, timeouts TIMEOUT, I/O NETWORK. Messages carry the HTTP - ** status and the API's error type only. - *******************************************************************************/ -public class OpenAiCompatibleModelProvider implements AssistantModelProvider -{ - private final String name; - private final String baseUrl; - private final String apiKey; - private final Duration requestTimeout; - private final HttpClient httpClient; - private final Boolean strictTools; - - - - /******************************************************************************* - ** Use the builder. - *******************************************************************************/ - private OpenAiCompatibleModelProvider(Builder builder) - { - this.name = builder.name; - this.baseUrl = builder.baseUrl.endsWith("/") ? builder.baseUrl.substring(0, builder.baseUrl.length() - 1) : builder.baseUrl; - String key = builder.apiKey; - if(key != null && key.startsWith("${")) - { - key = new QMetaDataVariableInterpreter().interpret(key); - } - this.apiKey = StringUtils.hasContent(key) ? key : null; - this.requestTimeout = builder.requestTimeout; - this.strictTools = builder.strictTools; - this.httpClient = HttpClient.newBuilder().connectTimeout(Duration.ofSeconds(10)).build(); - } - - - - /******************************************************************************* - ** A builder. - *******************************************************************************/ - public static Builder builder() - { - return (new Builder()); - } - - - - /******************************************************************************* - ** - *******************************************************************************/ - @Override - public String getName() - { - return (name); - } - - - - /******************************************************************************* - ** - *******************************************************************************/ - @Override - public ModelCapabilities getCapabilities(String modelId) - { - return (new ModelCapabilities().withTools(true).withParallelToolCalls(true).withPromptCaching(false).withThinking(false).withMaxContextTokens(128_000)); - } - - - - /******************************************************************************* - ** Does it have a key? (Local servers may need none.) - *******************************************************************************/ - public boolean hasApiKey() - { - return (apiKey != null); - } - - - - /******************************************************************************* - ** - *******************************************************************************/ - @Override - public StreamHandle stream(ModelRequest request, ModelEventListener listener) - { - CompletableStreamHandle handle = new CompletableStreamHandle(); - CompletableFuture> future; - try - { - HttpRequest.Builder http = HttpRequest.newBuilder(URI.create(baseUrl + "/chat/completions")) - .timeout(requestTimeout) - .header("Content-Type", "application/json") - .POST(HttpRequest.BodyPublishers.ofString(JsonUtils.toJson(toRequestBody(request)))); - if(apiKey != null) - { - http.header("Authorization", "Bearer " + apiKey); - } - future = httpClient.sendAsync(http.build(), HttpResponse.BodyHandlers.ofString()); - } - catch(Exception e) - { - handle.fail(new ModelProviderException(ModelProviderException.Kind.INVALID_REQUEST, null, "Could not build the request: " + e.getClass().getSimpleName())); - return (handle); - } - - handle.setOnCancel(() -> future.cancel(true)); - future.whenComplete((response, error) -> - { - if(error != null) - { - Throwable cause = error.getCause() != null ? error.getCause() : error; - if(cause instanceof HttpTimeoutException) - { - handle.fail(new ModelProviderException(ModelProviderException.Kind.TIMEOUT, null, "The request timed out.")); - } - else if(cause instanceof IOException) - { - handle.fail(new ModelProviderException(ModelProviderException.Kind.NETWORK, null, "Network error: " + cause.getClass().getSimpleName())); - } - else - { - handle.fail(new ModelProviderException(ModelProviderException.Kind.OTHER, null, "Request failed: " + cause.getClass().getSimpleName())); - } - return; - } - - try - { - if(response.statusCode() >= 300) - { - handle.fail(toException(response.statusCode(), response.body())); - return; - } - ModelResponse modelResponse = fromResponseBody(response.body()); - String text = modelResponse.getText(); - if(StringUtils.hasContent(text)) - { - listener.onTextDelta(text); - } - handle.complete(modelResponse); - } - catch(Exception e) - { - handle.fail(new ModelProviderException(ModelProviderException.Kind.OTHER, response.statusCode(), "Unreadable response: " + e.getClass().getSimpleName())); - } - }); - return (handle); - } - - - - /******************************************************************************* - ** ModelRequest -> Chat Completions request body. - *******************************************************************************/ - Map toRequestBody(ModelRequest request) - { - List> messages = new ArrayList<>(); - StringBuilder system = new StringBuilder(); - for(SystemBlock block : request.getSystem()) - { - if(system.length() > 0) - { - system.append("\n\n"); - } - system.append(block.getText()); - } - if(system.length() > 0) - { - messages.add(Map.of("role", "system", "content", system.toString())); - } - - for(ModelMessage message : request.getMessages()) - { - if(message.getRole() == ModelMessage.Role.ASSISTANT) - { - Map assistant = new LinkedHashMap<>(); - StringBuilder text = new StringBuilder(); - List> toolCalls = new ArrayList<>(); - for(ModelPart part : message.getParts()) - { - if(part.getType() == ModelPart.Type.TEXT && part.getText() != null) - { - text.append(part.getText()); - } - else if(part.getType() == ModelPart.Type.TOOL_USE) - { - toolCalls.add(Map.of("id", part.getToolUseId(), "type", "function", - "function", Map.of("name", part.getToolName(), "arguments", JsonUtils.toJson(part.getInput() == null ? Map.of() : part.getInput())))); - } - } - assistant.put("role", "assistant"); - assistant.put("content", text.length() == 0 ? null : text.toString()); - if(!toolCalls.isEmpty()) - { - assistant.put("tool_calls", toolCalls); - } - messages.add(assistant); - } - else - { - StringBuilder text = new StringBuilder(); - for(ModelPart part : message.getParts()) - { - if(part.getType() == ModelPart.Type.TOOL_RESULT) - { - messages.add(Map.of("role", "tool", "tool_call_id", part.getToolUseId(), "content", part.getText() == null ? "" : part.getText())); - } - else if(part.getText() != null) - { - text.append(part.getText()); - } - } - if(text.length() > 0) - { - messages.add(Map.of("role", "user", "content", text.toString())); - } - } - } - - Map body = new LinkedHashMap<>(); - body.put("model", request.getModel()); - body.put("messages", messages); - if(request.getMaxTokens() != null) - { - body.put("max_completion_tokens", request.getMaxTokens()); - } - if(!request.getTools().isEmpty()) - { - List> tools = new ArrayList<>(); - for(ModelToolDefinition tool : request.getTools()) - { - Map function = new LinkedHashMap<>(); - function.put("name", tool.getName()); - function.put("description", tool.getDescription()); - function.put("parameters", tool.getInputSchema()); - //////////////////////////////////////////////////////////////////// - // OpenAI's strict mode needs every property required; the // - // agent validates inputs itself, so strict is off by default. // - //////////////////////////////////////////////////////////////////// - if(Boolean.TRUE.equals(strictTools)) - { - function.put("strict", true); - } - tools.add(Map.of("type", "function", "function", function)); - } - body.put("tools", tools); - body.put("parallel_tool_calls", Boolean.TRUE.equals(request.getParallelToolCalls())); - } - return (body); - } - - - - /******************************************************************************* - ** Chat Completions response body -> ModelResponse. - *******************************************************************************/ - @SuppressWarnings("unchecked") - ModelResponse fromResponseBody(String json) throws Exception - { - Map root = JsonUtils.toObject(json, new TypeReference<>() {}); - List> choices = (List>) root.get("choices"); - if(choices == null || choices.isEmpty()) - { - throw (new IllegalStateException("no choices")); - } - Map choice = choices.get(0); - Map message = (Map) choice.get("message"); - - ModelResponse response = new ModelResponse(); - Object content = message == null ? null : message.get("content"); - if(content instanceof String text && !text.isEmpty()) - { - response.getParts().add(ModelPart.text(text)); - } - List> toolCalls = message == null ? null : (List>) message.get("tool_calls"); - if(toolCalls != null) - { - for(Map call : toolCalls) - { - Map function = (Map) call.get("function"); - String arguments = function == null ? null : (String) function.get("arguments"); - Map input = StringUtils.hasContent(arguments) ? JsonUtils.toObject(arguments, new TypeReference>() {}) : new LinkedHashMap<>(); - response.getParts().add(ModelPart.toolUse((String) call.get("id"), function == null ? null : (String) function.get("name"), input)); - } - } - - String finish = (String) choice.get("finish_reason"); - response.setStopReason(switch(finish == null ? "" : finish) - { - case "tool_calls", "function_call" -> ModelResponse.STOP_TOOL_USE; - case "length" -> ModelResponse.STOP_MAX_TOKENS; - case "content_filter" -> ModelResponse.STOP_REFUSAL; - default -> response.getToolUses().isEmpty() ? ModelResponse.STOP_END_TURN : ModelResponse.STOP_TOOL_USE; - }); - - Map usage = (Map) root.get("usage"); - if(usage != null) - { - long prompt = number(usage.get("prompt_tokens")); - long cached = 0; - if(usage.get("prompt_tokens_details") instanceof Map details) - { - cached = number(details.get("cached_tokens")); - } - response.setUsage(new ModelUsage().withInputTokens(Math.max(0, prompt - cached)).withCacheReadTokens(cached).withOutputTokens(number(usage.get("completion_tokens")))); - } - response.setRequestId((String) root.get("id")); - return (response); - } - - - - /******************************************************************************* - ** An HTTP error -> a provider exception (status and error type only). - *******************************************************************************/ - @SuppressWarnings("unchecked") - static ModelProviderException toException(int status, String body) - { - String type = null; - try - { - Map root = JsonUtils.toObject(body, new TypeReference<>() {}); - Object error = root.get("error"); - if(error instanceof Map map) - { - type = map.get("type") instanceof String s ? s : (map.get("code") instanceof String c ? c : null); - } - } - catch(Exception e) - { - // not JSON: the status alone - } - - ModelProviderException.Kind kind = switch(status) - { - case 401, 403 -> ModelProviderException.Kind.AUTHENTICATION; - case 429 -> ModelProviderException.Kind.RATE_LIMITED; - case 503, 529 -> ModelProviderException.Kind.OVERLOADED; - case 400, 404, 413, 422 -> ModelProviderException.Kind.INVALID_REQUEST; - case 408 -> ModelProviderException.Kind.TIMEOUT; - default -> status >= 500 ? ModelProviderException.Kind.SERVER : ModelProviderException.Kind.OTHER; - }; - return (new ModelProviderException(kind, status, "HTTP " + status + (type == null ? "" : " (" + type + ")"))); - } - - - - /******************************************************************************* - ** - *******************************************************************************/ - private static long number(Object value) - { - return (value instanceof Number n ? n.longValue() : 0); - } - - - - /******************************************************************************* - ** Never includes the key. - *******************************************************************************/ - @Override - public String toString() - { - return ("OpenAiCompatibleModelProvider{name=" + name + ", baseUrl=" + baseUrl + ", apiKey=" + (apiKey == null ? "none" : "***") + "}"); - } - - - - /******************************************************************************* - ** Builder. - *******************************************************************************/ - public static class Builder - { - private String name = "openai"; - private String baseUrl = "https://api.openai.com/v1"; - private String apiKey; - private Duration requestTimeout = Duration.ofSeconds(120); - private Boolean strictTools = false; - - - - /******************************************************************************* - ** The provider's registration name. - *******************************************************************************/ - public Builder name(String name) - { - this.name = name; - return (this); - } - - - - /******************************************************************************* - ** e.g. https://api.openai.com/v1 or http://localhost:11434/v1 - *******************************************************************************/ - public Builder baseUrl(String baseUrl) - { - this.baseUrl = baseUrl; - return (this); - } - - - - /******************************************************************************* - ** The key, or a reference such as "${env.OPENAI_API_KEY}". - *******************************************************************************/ - public Builder apiKey(String apiKey) - { - this.apiKey = apiKey; - return (this); - } - - - - /******************************************************************************* - ** - *******************************************************************************/ - public Builder requestTimeout(Duration requestTimeout) - { - this.requestTimeout = requestTimeout; - return (this); - } - - - - /******************************************************************************* - ** - *******************************************************************************/ - public Builder strictTools(Boolean strictTools) - { - this.strictTools = strictTools; - return (this); - } - - - - /******************************************************************************* - ** - *******************************************************************************/ - public OpenAiCompatibleModelProvider build() - { - if(!StringUtils.hasContent(name) || !StringUtils.hasContent(baseUrl)) - { - throw (new IllegalArgumentException("name and baseUrl are required")); - } - return (new OpenAiCompatibleModelProvider(this)); - } - } - -} diff --git a/src/test/java/com/kingsrook/qbits/marketing/AgentAndSecurityTest.java b/src/test/java/com/kingsrook/qbits/marketing/AgentAndSecurityTest.java index 19b91f1..097437b 100644 --- a/src/test/java/com/kingsrook/qbits/marketing/AgentAndSecurityTest.java +++ b/src/test/java/com/kingsrook/qbits/marketing/AgentAndSecurityTest.java @@ -21,7 +21,6 @@ import java.util.List; import java.util.Map; import com.kingsrook.qbits.assistant.provider.ModelProviderException; import com.kingsrook.qbits.assistant.provider.ModelUsage; -import com.kingsrook.qbits.assistant.provider.fake.ScriptedModelProvider; import com.kingsrook.qbits.marketing.agent.MarketingAgentService; import com.kingsrook.qbits.marketing.agent.tools.MarketingTools; import com.kingsrook.qbits.marketing.approval.ApprovalService; diff --git a/src/test/java/com/kingsrook/qbits/marketing/EndToEndFlowTest.java b/src/test/java/com/kingsrook/qbits/marketing/EndToEndFlowTest.java index b5d4dd9..0a2b65b 100644 --- a/src/test/java/com/kingsrook/qbits/marketing/EndToEndFlowTest.java +++ b/src/test/java/com/kingsrook/qbits/marketing/EndToEndFlowTest.java @@ -22,7 +22,6 @@ import java.time.Instant; import java.util.List; import java.util.Map; import com.kingsrook.qbits.assistant.provider.ModelUsage; -import com.kingsrook.qbits.assistant.provider.fake.ScriptedModelProvider; import com.kingsrook.qbits.marketing.agent.MarketingAgentService; import com.kingsrook.qbits.marketing.agent.tools.MarketingTools; import com.kingsrook.qbits.marketing.measurement.MeasurementReport; diff --git a/src/test/java/com/kingsrook/qbits/marketing/MarketingQBitProducerTest.java b/src/test/java/com/kingsrook/qbits/marketing/MarketingQBitProducerTest.java index 8379896..6cb5da7 100644 --- a/src/test/java/com/kingsrook/qbits/marketing/MarketingQBitProducerTest.java +++ b/src/test/java/com/kingsrook/qbits/marketing/MarketingQBitProducerTest.java @@ -21,8 +21,10 @@ import java.util.List; import java.util.Set; import java.util.stream.Collectors; import com.kingsrook.qbits.marketing.agent.KernelToolRegistration; +import com.kingsrook.qbits.marketing.agent.tools.MarketingTools; import com.kingsrook.qbits.marketing.metadata.MarketingMetaDataBuilder; import com.kingsrook.qbits.marketing.security.MarketingPermissions; +import com.kingsrook.qqq.backend.core.actions.ai.AiToolRegistry; import com.kingsrook.qqq.backend.core.model.metadata.QInstance; import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppChildMetaData; import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData; @@ -111,11 +113,16 @@ class MarketingQBitProducerTest extends BaseMarketingTest /******************************************************************************* - ** Whether or not the kernel hook is available (absent on QQQ 4.1.0-RC.4), registering no tools registers nothing. + ** RC.6 registers the read tools, excludes writes and tolerates repeat registration. *******************************************************************************/ @Test - void testKernelHookRegistersNothingWithoutTools() + void testKernelHookAvailableOnRc6() { + assertThat(KernelToolRegistration.isAvailable()).isTrue(); + assertThat(AiToolRegistry.of(qInstance).getTools()).containsOnlyKeys( + MarketingTools.GET_CAMPAIGN_BRIEF, MarketingTools.LIST_PRODUCT_CLAIMS, MarketingTools.LIST_CAMPAIGN_CONTENT, + MarketingTools.GET_CAMPAIGN_RESULTS, MarketingTools.LIST_CHANNELS, MarketingTools.LIST_FINDINGS); + assertThat(KernelToolRegistration.register(qInstance, MarketingTools.all())).isZero(); assertThat(KernelToolRegistration.register(qInstance, List.of())).isZero(); } diff --git a/src/test/java/com/kingsrook/qbits/marketing/ScriptedModelProvider.java b/src/test/java/com/kingsrook/qbits/marketing/ScriptedModelProvider.java new file mode 100644 index 0000000..91563d4 --- /dev/null +++ b/src/test/java/com/kingsrook/qbits/marketing/ScriptedModelProvider.java @@ -0,0 +1,218 @@ +/* + * Copyright 2026 Kingsrook, LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.marketing; + + +import java.util.ArrayDeque; +import java.util.ArrayList; +import java.util.Deque; +import java.util.List; +import java.util.Map; +import com.kingsrook.qbits.assistant.provider.AssistantModelProvider; +import com.kingsrook.qbits.assistant.provider.CompletableStreamHandle; +import com.kingsrook.qbits.assistant.provider.ModelCapabilities; +import com.kingsrook.qbits.assistant.provider.ModelEventListener; +import com.kingsrook.qbits.assistant.provider.ModelPart; +import com.kingsrook.qbits.assistant.provider.ModelProviderException; +import com.kingsrook.qbits.assistant.provider.ModelRequest; +import com.kingsrook.qbits.assistant.provider.ModelResponse; +import com.kingsrook.qbits.assistant.provider.ModelUsage; +import com.kingsrook.qbits.assistant.provider.StreamHandle; +import com.kingsrook.qqq.backend.core.utils.JsonUtils; + + +/******************************************************************************* + ** Synchronous scripted responses and request snapshots for marketing tests. + *******************************************************************************/ +class ScriptedModelProvider implements AssistantModelProvider +{ + private final String name; + private final Deque steps = new ArrayDeque<>(); + private final List requests = new ArrayList<>(); + private final List requestJson = new ArrayList<>(); + private ModelUsage usagePerCall = new ModelUsage(); + + + + /******************************************************************************* + ** Register under the name selected by the test configuration. + *******************************************************************************/ + ScriptedModelProvider(String name) + { + this.name = name; + } + + + + /******************************************************************************* + ** Queue a complete answer. + *******************************************************************************/ + ScriptedModelProvider thenText(String text) + { + return (thenRespond(request -> new ModelResponse().withParts(List.of(ModelPart.text(text))).withStopReason(ModelResponse.STOP_END_TURN))); + } + + + + /******************************************************************************* + ** Queue a single tool call. + *******************************************************************************/ + ScriptedModelProvider thenToolUse(String toolName, Map input) + { + return (thenToolUses(null, List.of(new Object[] { toolName, input }))); + } + + + + /******************************************************************************* + ** Queue parallel tool calls using the test fixtures' name/input pairs. + *******************************************************************************/ + @SuppressWarnings("unchecked") + ScriptedModelProvider thenToolUses(String preamble, List calls) + { + return (thenRespond(request -> + { + List parts = new ArrayList<>(); + if(preamble != null) + { + parts.add(ModelPart.text(preamble)); + } + for(Object[] call : calls) + { + parts.add(ModelPart.toolUse("tool_" + requests.size() + "_" + parts.size(), (String) call[0], (Map) call[1])); + } + return (new ModelResponse().withParts(parts).withStopReason(ModelResponse.STOP_TOOL_USE)); + })); + } + + + + /******************************************************************************* + ** Queue a provider failure, including usage already incurred. + *******************************************************************************/ + ScriptedModelProvider thenError(ModelProviderException.Kind kind, String message) + { + return (thenRespond(request -> + { + throw (new ModelProviderException(kind, null, message).withPartialUsage(new ModelUsage().add(usagePerCall))); + })); + } + + + + /******************************************************************************* + ** Queue a response that can inspect the request or change test state. + *******************************************************************************/ + ScriptedModelProvider thenRespond(Step step) + { + steps.add(step); + return (this); + } + + + + /******************************************************************************* + ** Set usage for budget and cost assertions. + *******************************************************************************/ + ScriptedModelProvider withUsagePerCall(ModelUsage usage) + { + usagePerCall = usage; + return (this); + } + + + + /******************************************************************************* + ** Provider registration name. + *******************************************************************************/ + @Override + public String getName() + { + return (name); + } + + + + /******************************************************************************* + ** Exercise the agent's prompt-cache hints. + *******************************************************************************/ + @Override + public ModelCapabilities getCapabilities(String modelId) + { + return (new ModelCapabilities().withPromptCaching(true).withThinking(false)); + } + + + + /******************************************************************************* + ** Record before responding, preserving each turn before the agent mutates it. + *******************************************************************************/ + @Override + public StreamHandle stream(ModelRequest request, ModelEventListener listener) + { + requests.add(request.copy()); + requestJson.add(JsonUtils.toJson(request)); + CompletableStreamHandle handle = new CompletableStreamHandle(); + try + { + ModelResponse response = steps.removeFirst().respond(request); + listener.onTextDelta(response.getText()); + response.setUsage(new ModelUsage().add(usagePerCall)); + response.setRequestId("request_" + requests.size()); + handle.complete(response); + } + catch(ModelProviderException e) + { + handle.fail(e); + } + return (handle); + } + + + + /******************************************************************************* + ** Request copies for tool exposure assertions. + *******************************************************************************/ + List getRequests() + { + return (requests); + } + + + + /******************************************************************************* + ** Serialized snapshots for data-boundary assertions. + *******************************************************************************/ + List getRequestJson() + { + return (requestJson); + } + + + + /******************************************************************************* + ** A scripted step may return a response or a checked provider failure. + *******************************************************************************/ + @FunctionalInterface + interface Step + { + /******************************************************************************* + ** Respond to one model request. + *******************************************************************************/ + ModelResponse respond(ModelRequest request) throws ModelProviderException; + } +} diff --git a/src/test/java/com/kingsrook/qbits/marketing/agent/provider/OpenAiCompatibleModelProviderTest.java b/src/test/java/com/kingsrook/qbits/marketing/agent/provider/OpenAiCompatibleModelProviderTest.java index 4ce6f8e..94ade58 100644 --- a/src/test/java/com/kingsrook/qbits/marketing/agent/provider/OpenAiCompatibleModelProviderTest.java +++ b/src/test/java/com/kingsrook/qbits/marketing/agent/provider/OpenAiCompatibleModelProviderTest.java @@ -31,7 +31,11 @@ import com.kingsrook.qbits.assistant.provider.ModelProviderException; import com.kingsrook.qbits.assistant.provider.ModelRequest; import com.kingsrook.qbits.assistant.provider.ModelResponse; import com.kingsrook.qbits.assistant.provider.ModelToolDefinition; +import com.kingsrook.qbits.assistant.provider.ProviderSettings; import com.kingsrook.qbits.assistant.provider.SystemBlock; +import com.kingsrook.qbits.assistant.provider.openai.OpenAiCompatibleModelProvider; +import com.kingsrook.qbits.marketing.MarketingQBitConfig; +import com.kingsrook.qqq.backend.core.instances.QMetaDataVariableInterpreter; import com.sun.net.httpserver.HttpServer; import org.junit.jupiter.api.AfterEach; import org.junit.jupiter.api.BeforeEach; @@ -41,8 +45,8 @@ import static org.assertj.core.api.Assertions.assertThatThrownBy; /******************************************************************************* - ** The OpenAI-compatible provider against a local fixture server (no network, - ** no real key): request mapping, tool calls, usage, error kinds, and the key + ** The published OpenAI-compatible provider against a local fixture server + ** with no real key: request mapping, tool calls, usage, error kinds, and the key ** never appearing in toString. *******************************************************************************/ class OpenAiCompatibleModelProviderTest @@ -94,8 +98,9 @@ class OpenAiCompatibleModelProviderTest *******************************************************************************/ private OpenAiCompatibleModelProvider provider(String key) { - return (OpenAiCompatibleModelProvider.builder().name("local").baseUrl("http://127.0.0.1:" + server.getAddress().getPort() + "/v1").apiKey(key) - .requestTimeout(Duration.ofSeconds(5)).build()); + return (new OpenAiCompatibleModelProvider(new ProviderSettings().withKind("OPENAI_COMPATIBLE") + .withBaseUrl("http://127.0.0.1:" + server.getAddress().getPort() + "/v1") + .withApiKey(key == null ? null : new QMetaDataVariableInterpreter().interpret(key)).withRequestTimeoutSeconds(5))); } @@ -105,7 +110,7 @@ class OpenAiCompatibleModelProviderTest *******************************************************************************/ private static ModelRequest request() { - return (new ModelRequest().withModel("test-model").withMaxTokens(100) + return (new ModelRequest().withModel("test-model").withMaxTokens(100).withStreaming(false) .withSystem(List.of(new SystemBlock("Be truthful.", null))) .withTools(List.of(new ModelToolDefinition().withName("marketing_list_channels").withDescription("Channels") .withInputSchema(Map.of("type", "object", "properties", Map.of())))) @@ -129,7 +134,11 @@ class OpenAiCompatibleModelProviderTest "usage":{"prompt_tokens":120,"completion_tokens":30,"prompt_tokens_details":{"cached_tokens":20}}} """ }); - ModelResponse response = provider("${env.MARKETING_TEST_KEY_THAT_IS_NOT_SET}").stream(request(), text -> {}).await(Duration.ofSeconds(5)); + MarketingQBitConfig config = new MarketingQBitConfig() + .withModelProvider(provider("${env.MARKETING_TEST_KEY_THAT_IS_NOT_SET}")) + .withProviderName("openai-compatible").withModel("configured-model"); + ModelResponse response = config.getModelProviders().get(config.getProviderName()) + .stream(request().withModel(config.getModel()), text -> {}).await(Duration.ofSeconds(5)); assertThat(response.getStopReason()).isEqualTo(ModelResponse.STOP_TOOL_USE); assertThat(response.getText()).isEqualTo("Checking."); assertThat(response.getToolUses()).hasSize(1); @@ -141,7 +150,7 @@ class OpenAiCompatibleModelProviderTest String body = lastBody.get(); assertThat(body).contains("\"role\":\"system\"").contains("Be truthful.").contains("\"tool_calls\"").contains("\"role\":\"tool\"").contains("\"tool_call_id\":\"call_1\"") - .contains("\"max_completion_tokens\":100").contains("marketing_list_channels"); + .contains("\"max_tokens\":100").contains("marketing_list_channels").contains("\"model\":\"configured-model\""); assertThat(lastAuth.get()).as("an unset key reference means no Authorization header").isNull(); } @@ -162,7 +171,8 @@ class OpenAiCompatibleModelProviderTest .isInstanceOfSatisfying(ModelProviderException.class, e -> { assertThat(e.getKind()).isEqualTo(ModelProviderException.Kind.AUTHENTICATION); - assertThat(e.getMessage()).contains("401").contains("invalid_api_key").doesNotContain("sk-test"); + assertThat(e.getHttpStatus()).isEqualTo(401); + assertThat(e.getMessage()).doesNotContain("sk-FAKE-TEST-KEY-000000"); }); assertThat(lastAuth.get()).isEqualTo("Bearer sk-FAKE-TEST-KEY-000000"); @@ -174,8 +184,9 @@ class OpenAiCompatibleModelProviderTest assertThatThrownBy(() -> provider.stream(request(), text -> {}).await(Duration.ofSeconds(5))) .isInstanceOfSatisfying(ModelProviderException.class, e -> assertThat(e.getKind()).isEqualTo(ModelProviderException.Kind.INVALID_REQUEST)); - responses.add(new Object[] { 200, "{\"choices\":[{\"finish_reason\":\"content_filter\",\"message\":{\"content\":\"\"}}]}" }); - assertThat(OpenAiCompatibleModelProvider.toException(503, "").getKind()).isEqualTo(ModelProviderException.Kind.OVERLOADED); + responses.add(new Object[] { 503, "" }); + assertThatThrownBy(() -> provider.stream(request(), text -> {}).await(Duration.ofSeconds(5))) + .isInstanceOfSatisfying(ModelProviderException.class, e -> assertThat(e.getKind()).isEqualTo(ModelProviderException.Kind.OVERLOADED)); } @@ -196,4 +207,29 @@ class OpenAiCompatibleModelProviderTest assertThat(streamed).containsExactly("Done."); } + + + /******************************************************************************* + ** The agent's default streaming mode receives text and usage from SSE. + *******************************************************************************/ + @Test + void testStreaming() throws Exception + { + responses.add(new Object[] { 200, """ + data: {"choices":[{"index":0,"delta":{"content":"Done."},"finish_reason":null}]} + + data: {"choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":12,"completion_tokens":3}} + + data: [DONE] + + """ }); + List streamed = new ArrayList<>(); + ModelResponse response = provider(null).stream(request().withStreaming(true), streamed::add).await(Duration.ofSeconds(5)); + assertThat(streamed).containsExactly("Done."); + assertThat(response.getText()).isEqualTo("Done."); + assertThat(response.getUsage().getInputTokens()).isEqualTo(12); + assertThat(response.getUsage().getOutputTokens()).isEqualTo(3); + assertThat(lastBody.get()).contains("\"stream\":true"); + } + }