feat: provider and model registry, OpenAI-compatible adapter, discovery and health (P-a) #22

Merged
KofTwentyTwo merged 3 commits from feature/p1-provider-registry into develop 2026-10-13 22:59:53 +00:00
73 changed files with 6345 additions and 65 deletions

No files matched your search

+3
View File
@@ -19,3 +19,6 @@ dependency-reduced-pom.xml
/.env.local
/.cache/
.claude/settings.local.json
# Runtime output from every provider module
log/
+1
View File
@@ -42,6 +42,7 @@ Always use the wrapper: Maven 3.10 cannot read the parent POM. The committed def
- `processes/`: `AssistantRetentionStep` and `TestConnectionProcess` (record action on `assistant_provider`).
- `engine/ProviderService` (active provider from `assistant_provider` rows, key references, prices), `UsageRecorder` (one `assistant_usage` row per model call), `ConnectionTest`.
- `qbit-assistant-provider-anthropic`: `AnthropicModelProvider` and `AnthropicMapping` (anthropic-java streaming).
- `qbit-assistant-provider-openai`: OpenAI-compatible Chat Completions, model discovery and health using JDK HTTP and Jackson.
## Invariants (do not break)
+4
View File
@@ -4,6 +4,10 @@ All notable changes to this project are documented here. The format follows Keep
## [0.1.0] - Unreleased
- Add named, server-only CA bundles for HTTPS providers, additive JVM trust and sanitized TLS diagnostics; tolerate repeated/missing streamed tool identities and invalid arguments, with a golden tool round-trip test.
- Add the provider/model registry, per-model capabilities and prices, keyless local providers, discovery and health, and provider/model connection tests.
- Add the optional JDK HTTP OpenAI-compatible adapter for OpenAI, Azure, OpenRouter, Ollama, LM Studio, vLLM and llama.cpp, with strict-schema safeguards and localhost contract tests.
Phase 0/1 of the QQQ Assistant: read-only Q&A and navigation. Requires QQQ 4.1 with:
- the AI tool kernel (`qqq-backend-core` `actions.ai`, QRun-IO/qqq#1078)
- the developer-panel contract (QRun-IO/qqq#1076).
+1
View File
@@ -42,6 +42,7 @@ Always use the wrapper: Maven 3.10 cannot read the parent POM. The committed def
- `processes/`: `AssistantRetentionStep` and `TestConnectionProcess` (record action on `assistant_provider`).
- `engine/ProviderService` (active provider from `assistant_provider` rows, key references, prices), `UsageRecorder` (one `assistant_usage` row per model call), `ConnectionTest`.
- `qbit-assistant-provider-anthropic`: `AnthropicModelProvider` and `AnthropicMapping` (anthropic-java streaming).
- `qbit-assistant-provider-openai`: OpenAI-compatible Chat Completions, model discovery and health using JDK HTTP and Jackson.
## Invariants (do not break)
+58
View File
@@ -253,6 +253,64 @@ The **live** evals run the same golden tasks against Claude. They are skipped un
ANTHROPIC_API_KEY=... ./mvnw verify -Dassistant.live=true -Dtest.excludedGroups= -Dgroups=live -Djacoco.skip=true -pl qbit-assistant-provider-anthropic -am
```
## Providers
Add `com.kingsrook.qbits:qbit-assistant-provider-openai` at the same version as
the QBit and register the adapter alongside any Anthropic adapter:
```java
config.withProviderType(new OpenAiCompatibleProviderType());
```
`OpenAiCompatibleProviderType` is in
`com.kingsrook.qbits.assistant.provider.openai`. Its installed type id is
`openai-compatible`; provider rows choose OPENAI, AZURE_OPENAI, OPENROUTER,
OLLAMA, LM_STUDIO, VLLM, LLAMA_CPP or OPENAI_COMPATIBLE as their kind.
- Local kinds use their localhost endpoint by default and need no key. Leave
`apiKeyEnvName` empty. An auth proxy can use an explicitly allowed key env name.
- Cloud defaults use `OPENAI_API_KEY`, `AZURE_OPENAI_API_KEY` or
`OPENROUTER_API_KEY`. Values stay in the host's secret store.
- An override must match an entry in `config.withAllowedBaseUrls(Set.of(...))`.
Azure and OPENAI_COMPATIBLE have no default URL and always require an entry.
Redirects are not followed. Azure uses the deployment model id and defaults
`apiVersion` to `2024-10-21`.
- **Discover models** adds a provider's Models child list; existing edits survive.
Large cloud catalogs (over 20 models) start disabled. **Test connection** is
available on both providers and models and records a TEST usage row.
- A selected registered model must be enabled. With no model selected, the first
enabled registry model wins; providers with no model rows retain legacy selection.
Registry prices override type/model prices, then configured prices. Unpriced
local models cost zero. Capabilities can be adjusted per registered model.
For Ollama behind a TLS proxy with a private CA and bearer authentication:
```java
config.withProviderType(new OpenAiCompatibleProviderType())
.withAllowedBaseUrls(Set.of("https://llm.example.lan:8443/v1"))
.withAllowedApiKeyEnvNames(Set.of("LOCAL_LLM_API_KEY"))
.withTrustedCaBundleFile("local-llm", Path.of("/etc/app/llm-root-ca.pem"));
```
Set `LOCAL_LLM_API_KEY` in the server environment from your secret store. In the
provider record choose kind `OLLAMA`, base URL `https://llm.example.lan:8443/v1`,
`apiKeyEnvName = LOCAL_LLM_API_KEY`, and `tlsCaBundle = local-llm`. Only the names
are stored. `withTrustedCaBundle("local-llm", pem)` also accepts one or more PEM
certificates directly at startup. Invalid PEM fails configuration immediately.
The named bundle adds to the JVM roots; hostname verification stays enabled.
Leave `tlsCaBundle` empty to use JVM trust alone. Restart after changing a CA file.
Default tests use localhost mocks. To exercise an existing development server,
set `ASSISTANT_LIVE_OPENAI_BASE_URL` and `ASSISTANT_LIVE_OPENAI_MODEL`, optionally
`ASSISTANT_LIVE_OPENAI_KIND` (default OLLAMA), `ASSISTANT_LIVE_OPENAI_KEY_ENV`
(the variable name containing the key), and `ASSISTANT_LIVE_OPENAI_CA_FILE`
(a PEM bundle path). Unset or unreachable endpoints, including TLS failures, skip
the live test. Run:
```sh
./mvnw verify -pl qbit-assistant-provider-openai -am -Dtest.excludedGroups= -Dgroups=live -Djacoco.skip=true
```
## License
Apache 2.0.
+67
View File
@@ -118,3 +118,70 @@ Database (see `docs/schema-postgres.sql`):
Runtime: the tools call this server's own `/qqq/v1` on `127.0.0.1` and the port the first assistant request arrived on. If the server is not reachable that way (a different bind address, a TLS-only listener), set `withUiApiBaseUrl(...)`. Grant `assistant.admin` holders the new tables through the qbit's existing admin permission (no new permission names).
Behavior: nothing is writable until an admin adds a READ_WRITE data access row; before that the assistant reads exactly what it did before, now through the UI API.
## P-a: provider and model registry
Before starting the new version, database-backed hosts add the provider columns
and model sub-table below. Memory-backed hosts need no migration. Existing rows
derive kind and location from the adapter and keep their model behavior until
models are registered. Keys remain environment-variable references.
```sql
ALTER TABLE assistant_provider
ADD COLUMN kind VARCHAR(30),
ADD COLUMN tls_ca_bundle VARCHAR(100),
ADD COLUMN location VARCHAR(10),
ADD COLUMN hipaa_eligible BOOLEAN DEFAULT FALSE,
ADD COLUMN region VARCHAR(50),
ADD COLUMN api_version VARCHAR(30),
ADD COLUMN connect_timeout_seconds INTEGER DEFAULT 10 CHECK (connect_timeout_seconds BETWEEN 1 AND 600),
ADD COLUMN request_timeout_seconds INTEGER DEFAULT 120 CHECK (request_timeout_seconds BETWEEN 1 AND 600),
ADD COLUMN last_discovery_at TIMESTAMP WITH TIME ZONE,
ADD COLUMN last_discovery_status VARCHAR(16),
ADD COLUMN last_discovery_error VARCHAR(500);
ALTER TABLE assistant_provider ALTER COLUMN model TYPE VARCHAR(200);
ALTER TABLE assistant_provider ALTER COLUMN last_test_model TYPE VARCHAR(200);
ALTER TABLE assistant_usage ALTER COLUMN model TYPE VARCHAR(200);
ALTER TABLE assistant_message ALTER COLUMN model TYPE VARCHAR(200);
ALTER TABLE assistant_turn ALTER COLUMN model TYPE VARCHAR(200);
CREATE TABLE assistant_model
(
id SERIAL PRIMARY KEY,
provider_id INTEGER NOT NULL REFERENCES assistant_provider(id),
model_id VARCHAR(200) NOT NULL,
display_name VARCHAR(200),
enabled BOOLEAN,
supports_tools BOOLEAN,
supports_vision BOOLEAN,
supports_streaming BOOLEAN,
supports_reasoning_effort BOOLEAN,
context_window_tokens INTEGER,
max_output_tokens INTEGER,
input_usd_per_m_tok NUMERIC(12, 4),
output_usd_per_m_tok NUMERIC(12, 4),
source VARCHAR(16),
discovered_at TIMESTAMP WITH TIME ZONE,
last_test_at TIMESTAMP WITH TIME ZONE,
last_test_status VARCHAR(16),
last_test_latency_ms INTEGER,
last_test_error VARCHAR(500),
create_date TIMESTAMP WITH TIME ZONE,
modify_date TIMESTAMP WITH TIME ZONE,
UNIQUE (provider_id, model_id)
);
```
Register the optional `qbit-assistant-provider-openai` module and
`OpenAiCompatibleProviderType` to offer its kinds. Azure and generic endpoints
require an explicit `allowedBaseUrls` entry; local defaults may be keyless.
Discover models preserves existing rows. Cloud catalogs larger than 20 are added
disabled; enable the models to offer before leaving the provider's model empty.
HIPAA eligibility records an administrator's fact; P-a does not implement PHI routing.
Custom CA bundles are server configuration, never database content. Register them
with `withTrustedCaBundle(name, pem)` or `withTrustedCaBundleFile(name, path)` at
startup, then set the HTTPS provider row's `tlsCaBundle` to that name. Empty
references retain JVM default trust. The bundle adds trust without replacing JVM
roots; hostname verification remains enabled. If the P-a columns were already
applied, add only `tls_ca_bundle VARCHAR(100)` to `assistant_provider`.
+24
View File
@@ -309,6 +309,30 @@ AI exposure NONE is not enough, because the assistant acts as the user and an ad
- **scheduled assistant tasks**: recurring prompts that run as a designated user (with that user's stored credentials through the UI API, never a system session) and post results as notifications or records.
6. Everything goes through the `/qqq/v1` UI API, as the user, with the guarantees above.
### P-a as built
- Per-provider `tlsCaBundle` references startup-parsed, transient server CA bundles. OpenAI-compatible chat, health and discovery trust JVM roots plus the bundle, retain hostname verification, and report sanitized NETWORK errors with a trust hint.
- `assistant_provider` records kind, location, HIPAA eligibility, region, API
version, timeouts and discovery status. `assistant_model` is its admin-only
child list: enablement, capabilities, prices and test status, unique per provider/model.
- `ProviderSettings`, discovery and health extend the provider SPI through default
methods. Legacy `create(apiKey, baseUrl)` implementations keep working.
- The OpenAI-compatible module uses JDK HTTP and Jackson, Chat Completions with
SSE or single JSON replies, Azure deployment URLs and api-key auth, and keyless
local endpoints. Keys and URL overrides pass the existing allowlists first.
- Strict tools require explicit opt-in, OpenAI or Azure, and recursively strict
object schemas. A strict/schema/grammar 400 signals the existing one-retry fallback.
- Discovery only adds missing models; it preserves edits and disables catalogs
larger than 20 for cloud providers. Health uses a short models request. Provider
tests probe health then complete; model tests can probe a disabled model before
enabling it. Completion tests record TEST usage.
- Registry prices and capabilities override adapter/type defaults. Empty provider
model selects the first enabled registry model; an explicitly disabled selection
is a setup error. Rows without models retain the prior behavior.
- Localhost contract tests cover dialects, fragmented parallel tool calls, usage,
errors, timeouts, cancellation, discovery and a golden tool round trip. The live
smoke is environment-driven and excluded by default.
### Phases (each a green PR)
| | Scope | Depends on |
|---|---|---|
+42 -5
View File
@@ -30,7 +30,7 @@ CREATE TABLE assistant_message
status VARCHAR(16),
turn_id INTEGER,
seq INTEGER,
model VARCHAR(64),
model VARCHAR(200),
parts_json TEXT,
create_date TIMESTAMP WITH TIME ZONE,
modify_date TIMESTAMP WITH TIME ZONE
@@ -49,7 +49,7 @@ CREATE TABLE assistant_turn
stop_reason VARCHAR(64),
error_code VARCHAR(32),
cancel_reason VARCHAR(64),
model VARCHAR(64),
model VARCHAR(200),
started_at TIMESTAMP WITH TIME ZONE,
ended_at TIMESTAMP WITH TIME ZONE,
tool_call_count INTEGER,
@@ -173,10 +173,21 @@ CREATE TABLE assistant_provider
id SERIAL PRIMARY KEY,
name VARCHAR(100) NOT NULL UNIQUE,
provider_type VARCHAR(50) NOT NULL,
model VARCHAR(100),
kind VARCHAR(30),
location VARCHAR(10),
hipaa_eligible BOOLEAN DEFAULT FALSE,
region VARCHAR(50),
api_version VARCHAR(30),
connect_timeout_seconds INTEGER DEFAULT 10 CHECK (connect_timeout_seconds BETWEEN 1 AND 600),
request_timeout_seconds INTEGER DEFAULT 120 CHECK (request_timeout_seconds BETWEEN 1 AND 600),
last_discovery_at TIMESTAMP WITH TIME ZONE,
last_discovery_status VARCHAR(16),
last_discovery_error VARCHAR(500),
model VARCHAR(200),
enabled BOOLEAN,
is_default BOOLEAN,
base_url VARCHAR(255),
tls_ca_bundle VARCHAR(100),
api_key_env_name VARCHAR(100),
api_key_status VARCHAR(200),
monthly_cost_cap_usd NUMERIC(12, 2),
@@ -185,12 +196,38 @@ CREATE TABLE assistant_provider
last_test_at TIMESTAMP WITH TIME ZONE,
last_test_status VARCHAR(16),
last_test_latency_ms INTEGER,
last_test_model VARCHAR(100),
last_test_model VARCHAR(200),
last_test_error VARCHAR(500),
create_date TIMESTAMP WITH TIME ZONE,
modify_date TIMESTAMP WITH TIME ZONE
);
CREATE TABLE assistant_model
(
id SERIAL PRIMARY KEY,
provider_id INTEGER NOT NULL REFERENCES assistant_provider(id),
model_id VARCHAR(200) NOT NULL,
display_name VARCHAR(200),
enabled BOOLEAN,
supports_tools BOOLEAN,
supports_vision BOOLEAN,
supports_streaming BOOLEAN,
supports_reasoning_effort BOOLEAN,
context_window_tokens INTEGER,
max_output_tokens INTEGER,
input_usd_per_m_tok NUMERIC(12, 4),
output_usd_per_m_tok NUMERIC(12, 4),
source VARCHAR(16),
discovered_at TIMESTAMP WITH TIME ZONE,
last_test_at TIMESTAMP WITH TIME ZONE,
last_test_status VARCHAR(16),
last_test_latency_ms INTEGER,
last_test_error VARCHAR(500),
create_date TIMESTAMP WITH TIME ZONE,
modify_date TIMESTAMP WITH TIME ZONE,
UNIQUE (provider_id, model_id)
);
CREATE TABLE assistant_model_price
(
id SERIAL PRIMARY KEY,
@@ -215,7 +252,7 @@ CREATE TABLE assistant_usage
turn_id INTEGER,
provider_id INTEGER,
provider_type VARCHAR(50),
model VARCHAR(100),
model VARCHAR(200),
purpose VARCHAR(16),
status VARCHAR(16),
error_kind VARCHAR(32),
+12
View File
@@ -23,3 +23,15 @@ Light and dark, 390 and 1440 px:
- `panel-persona-*`: the panel with the configured persona. This is the Next UI build of qqq-frontend-next#94; the demo's admin matches the permission-scoped "Ada (analytics)" profile.
- `panel-pending-*`: the assistant proposed `update_record` on page 12, which is READ_WRITE. The card shows the change with Save and Cancel; nothing is saved yet.
- `panel-confirmed-*`: after Save, the write ran once through `PATCH /qqq/v1/table/page/12` with the user's session. The card says Done, and the record screen behind the panel shows the new value.
Planned P-a provider registry additions, taken against `OpenAiDemoServer` in the
OpenAI provider's test sources with the Next UI **1.0.0-RC.18** jar. See that
[demo's run instructions](../../qbit-assistant-provider-openai/src/test/java/com/kingsrook/qbits/assistant/provider/openai/demo/README.md).
Discovery and connection-test status, TEST usage and the assistant turn are real
executions against the localhost mock; replies/token counts are synthetic, not
live GPU measurements. No screenshots in this group have been added yet.
Light and dark, 390 and 1440 px:
- `p-provider-local-*`: Local GPU (Ollama), keyless LOCAL settings, discovery/test results and discovered models.
- `p-model-*`: a discovered model's capabilities, compute prices and connection-test result.
- `p-home-providers-*`: the home's provider summary and usage from the executed demo calls, including disabled OpenRouter with its key Not set.
Binary file not shown.

After

Width:  |  Height:  |  Size: 95 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 38 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 97 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 38 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 76 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 45 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 77 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 45 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 98 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 44 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 100 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 44 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 86 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 43 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 86 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 42 KiB

+6
View File
@@ -52,6 +52,7 @@
<modules>
<module>qbit-assistant</module>
<module>qbit-assistant-provider-anthropic</module>
<module>qbit-assistant-provider-openai</module>
</modules>
<licenses>
@@ -101,6 +102,11 @@
<version>${revision}</version>
<type>test-jar</type>
</dependency>
<dependency>
<groupId>com.kingsrook.qbits</groupId>
<artifactId>qbit-assistant-provider-openai</artifactId>
<version>${revision}</version>
</dependency>
<dependency>
<groupId>com.anthropic</groupId>
<artifactId>anthropic-java</artifactId>
+128
View File
@@ -0,0 +1,128 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ Copyright 2026 Kingsrook, LLC
~
~ Licensed under the Apache License, Version 2.0 (the "License");
~ you may not use this file except in compliance with the License.
~ You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>com.kingsrook.qbits</groupId>
<artifactId>qbit-assistant-parent</artifactId>
<version>${revision}</version>
</parent>
<artifactId>qbit-assistant-provider-openai</artifactId>
<packaging>jar</packaging>
<name>QBit Assistant - OpenAI-compatible provider</name>
<description>OpenAI-compatible Chat Completions provider using the JDK HTTP client</description>
<url>https://github.com/QRun-IO/qbit-assistant</url>
<dependencies>
<dependency>
<groupId>com.kingsrook.qbits</groupId>
<artifactId>qbit-assistant</artifactId>
</dependency>
<dependency>
<groupId>com.kingsrook.qqq</groupId>
<artifactId>qqq-backend-core</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</dependency>
<dependency>
<groupId>com.kingsrook.qbits</groupId>
<artifactId>qbit-assistant</artifactId>
<type>test-jar</type>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.kingsrook.qqq</groupId>
<artifactId>qqq-middleware-javalin</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.assertj</groupId>
<artifactId>assertj-core</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<!-- live evals call the real API: opt-in only -->
<excludedGroups>${test.excludedGroups}</excludedGroups>
<redirectTestOutputToFile>true</redirectTestOutputToFile>
</configuration>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<executions>
<execution>
<id>prepare-agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>report</id>
<phase>verify</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
<execution>
<id>check</id>
<goals>
<goal>check</goal>
</goals>
<configuration>
<rules>
<rule>
<element>BUNDLE</element>
<limits>
<limit>
<counter>INSTRUCTION</counter>
<value>COVEREDRATIO</value>
<minimum>0.70</minimum>
</limit>
<limit>
<counter>CLASS</counter>
<value>COVEREDRATIO</value>
<minimum>0.90</minimum>
</limit>
</limits>
</rule>
</rules>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
@@ -0,0 +1,429 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant.provider.openai;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.http.HttpResponse;
import java.net.http.HttpTimeoutException;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.TreeMap;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.atomic.AtomicReference;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.node.ObjectNode;
import com.kingsrook.qbits.assistant.provider.AssistantModelProvider;
import com.kingsrook.qbits.assistant.provider.CompletableStreamHandle;
import com.kingsrook.qbits.assistant.provider.ModelCapabilities;
import com.kingsrook.qbits.assistant.provider.ModelEventListener;
import com.kingsrook.qbits.assistant.provider.ModelPart;
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
import com.kingsrook.qbits.assistant.provider.ModelRequest;
import com.kingsrook.qbits.assistant.provider.ModelResponse;
import com.kingsrook.qbits.assistant.provider.ModelUsage;
import com.kingsrook.qbits.assistant.provider.ProviderSettings;
import com.kingsrook.qbits.assistant.provider.StreamHandle;
/*******************************************************************************
** JDK HTTP Chat Completions adapter. Only visible content is emitted or stored;
** reasoning deltas are ignored. Each call owns its cancellation and read guard.
*******************************************************************************/
public class OpenAiCompatibleModelProvider implements AssistantModelProvider
{
private final OpenAiHttp http;
private final boolean local;
/*******************************************************************************
** Build a client with resolved settings.
*******************************************************************************/
public OpenAiCompatibleModelProvider(ProviderSettings settings)
{
http = new OpenAiHttp(settings);
local = "LOCAL".equals(new OpenAiCompatibleProviderType().getDefaultLocation(http.settings.getKind()));
}
/*******************************************************************************
** Adapter identity.
*******************************************************************************/
@Override
public String getName()
{
return ("openai-compatible");
}
/*******************************************************************************
** Conservative defaults; registry rows can override model-specific facts.
*******************************************************************************/
@Override
public ModelCapabilities getCapabilities(String modelId)
{
return (new ModelCapabilities().withTools(true).withParallelToolCalls(true).withPromptCaching(false).withThinking(false)
.withMaxContextTokens(local ? 32_000 : 128_000).withStreaming(true).withVision(false).withReasoningEffort(false));
}
/*******************************************************************************
** Interrupting the worker aborts the HTTP call, including a blocked body read.
*******************************************************************************/
@Override
public StreamHandle stream(ModelRequest request, ModelEventListener listener)
{
CompletableStreamHandle handle = new CompletableStreamHandle();
AtomicReference<InputStream> input = new AtomicReference<>();
Thread worker = Thread.ofVirtual().unstarted(() -> run(request, listener, handle, input));
handle.setOnCancel(() ->
{
worker.interrupt();
close(input.get());
});
worker.start();
return (handle);
}
/*******************************************************************************
** Guard the entire request and idle reads, including after HTTP headers.
*******************************************************************************/
private void run(ModelRequest request, ModelEventListener listener, CompletableStreamHandle handle, AtomicReference<InputStream> input)
{
Thread worker = Thread.currentThread();
AtomicLong lastRead = new AtomicLong(System.nanoTime());
long started = lastRead.get();
long timeout = TimeUnit.SECONDS.toNanos(http.settings.getRequestTimeoutSeconds());
ScheduledExecutorService guard = Executors.newSingleThreadScheduledExecutor(Thread.ofVirtual().factory());
Accumulator result = new Accumulator(request.getModel(), listener);
handle.setPartialUsage(() -> result.usage);
guard.scheduleAtFixedRate(() ->
{
long now = System.nanoTime();
if(now - lastRead.get() >= timeout || now - started >= timeout)
{
handle.fail(new ModelProviderException(ModelProviderException.Kind.TIMEOUT, null, "The provider did not answer in time.").withPartialUsage(result.usage));
worker.interrupt();
close(input.get());
}
}, 100, 100, TimeUnit.MILLISECONDS);
try
{
ModelCapabilities caps = request.getCapabilities() == null ? getCapabilities(request.getModel()) : request.getCapabilities();
ObjectNode body = OpenAiMapping.request(request, http.settings.getKind(), caps);
boolean strict = false;
for(JsonNode tool : body.path("tools"))
{
strict |= tool.path("function").path("strict").asBoolean();
}
HttpResponse<InputStream> response = http.client.send(http.request(request.getModel(), http.settings.getRequestTimeoutSeconds())
.POST(java.net.http.HttpRequest.BodyPublishers.ofString(body.toString())).build(), HttpResponse.BodyHandlers.ofInputStream());
input.set(response.body());
result.requestId = OpenAiHttp.requestId(response);
if(handle.isCancelled())
{
return;
}
if(response.statusCode() != 200)
{
throw (http.error(response.statusCode(), new String(response.body().readNBytes(64 * 1024), StandardCharsets.UTF_8), result.requestId, strict));
}
if(Boolean.TRUE.equals(caps.getStreaming()))
{
readEvents(response.body(), lastRead, result, strict, handle);
}
else
{
result.accept(OpenAiHttp.JSON.readTree(response.body()), false, http, strict);
}
handle.complete(result.response());
}
catch(ModelProviderException e)
{
handle.fail(e.withPartialUsage(result.usage));
}
catch(HttpTimeoutException e)
{
handle.fail(new ModelProviderException(ModelProviderException.Kind.TIMEOUT, null, "The provider did not answer in time.").withPartialUsage(result.usage));
}
catch(InterruptedException e)
{
Thread.currentThread().interrupt();
if(!handle.isCancelled())
{
handle.fail(new ModelProviderException(ModelProviderException.Kind.NETWORK, null, "The provider request was interrupted."));
}
}
catch(IOException | RuntimeException e)
{
////////////////////////////////////////////////////////////////////////
// a transport error at or past the deadline is the timeout itself //
// (the JDK client or the guard closed the call), whichever saw it //
// first; report it as TIMEOUT so the kind does not depend on a race. //
////////////////////////////////////////////////////////////////////////
long now = System.nanoTime();
long margin = TimeUnit.MILLISECONDS.toNanos(200);
boolean late = now - started >= timeout - margin || now - lastRead.get() >= timeout - margin;
handle.fail((late ? new ModelProviderException(ModelProviderException.Kind.TIMEOUT, null, "The provider did not answer in time.") : OpenAiHttp.transportError(e)).withPartialUsage(result.usage));
}
finally
{
guard.shutdownNow();
close(input.get());
}
}
/*******************************************************************************
** SSE data fields, CRLF, comments, multi-line data and an optional DONE marker.
*******************************************************************************/
private void readEvents(InputStream stream, AtomicLong lastRead, Accumulator result, boolean strict, CompletableStreamHandle handle) throws IOException, ModelProviderException
{
BufferedReader reader = new BufferedReader(new InputStreamReader(stream, StandardCharsets.UTF_8));
StringBuilder data = new StringBuilder();
String line;
while(!handle.isCancelled() && (line = reader.readLine()) != null)
{
lastRead.set(System.nanoTime());
if(line.isEmpty())
{
if(data.length() > 0)
{
if("[DONE]".equals(data.toString().trim()))
{
return;
}
result.accept(OpenAiHttp.JSON.readTree(data.toString()), true, http, strict);
data.setLength(0);
}
}
else if(line.startsWith("data:"))
{
if(data.length() > 0)
{
data.append('\n');
}
data.append(line.substring(5).stripLeading());
}
}
if(data.length() > 0 && !"[DONE]".equals(data.toString().trim()))
{
result.accept(OpenAiHttp.JSON.readTree(data.toString()), true, http, strict);
}
}
/*******************************************************************************
** Closing an already cancelled stream is harmless.
*******************************************************************************/
private static void close(InputStream input)
{
if(input != null)
{
try
{
input.close();
}
catch(IOException e)
{
// The aborted HTTP call may already have closed the stream.
}
}
}
/*******************************************************************************
** Tool arguments arrive as JSON fragments, ordered by call index.
*******************************************************************************/
private static class Tool
{
private String id = "";
private String name = "";
private final StringBuilder arguments = new StringBuilder();
private boolean announced;
}
/*******************************************************************************
** Shared mapping for stream chunks and one-shot responses.
*******************************************************************************/
private static class Accumulator
{
private final ModelEventListener listener;
private final StringBuilder text = new StringBuilder();
private final Map<Integer, Tool> tools = new TreeMap<>();
private volatile ModelUsage usage = new ModelUsage();
private String model;
private String requestId;
private String stop = ModelResponse.STOP_END_TURN;
/*******************************************************************************
** One completion.
*******************************************************************************/
Accumulator(String model, ModelEventListener listener)
{
this.model = model;
this.listener = listener;
}
/*******************************************************************************
** Ignore reasoning fields; retain only text, tools and billable usage.
*******************************************************************************/
void accept(JsonNode chunk, boolean streaming, OpenAiHttp http, boolean strict) throws ModelProviderException
{
if(chunk.has("error"))
{
JsonNode error = chunk.path("error");
int status = error.path("status").asInt(error.path("code").asInt(500));
String type = error.path("type").asText();
if(status == 500)
{
status = switch(type)
{
case "invalid_request_error" -> 400;
case "authentication_error" -> 401;
case "rate_limit_error" -> 429;
case "overloaded_error" -> 503;
default -> 500;
};
}
throw (http.error(status, chunk.toString(), requestId, strict));
}
model = chunk.path("model").asText(model);
if(chunk.hasNonNull("usage"))
{
JsonNode used = chunk.get("usage");
long cached = used.path("prompt_tokens_details").path("cached_tokens").asLong();
usage = new ModelUsage().withInputTokens(Math.max(0, used.path("prompt_tokens").asLong() - cached))
.withOutputTokens(used.path("completion_tokens").asLong()).withCacheReadTokens(cached);
}
for(JsonNode choice : chunk.path("choices"))
{
if(choice.path("index").asInt() != 0)
{
continue;
}
JsonNode delta = choice.path(streaming ? "delta" : "message");
String content = delta.path("content").asText("");
if(!content.isEmpty())
{
text.append(content);
listener.onTextDelta(content);
}
int position = 0;
for(JsonNode call : delta.path("tool_calls"))
{
Tool tool = tools.computeIfAbsent(call.path("index").asInt(position++), k -> new Tool());
if(tool.id.isEmpty())
{
tool.id = call.path("id").asText("");
}
if(tool.name.isEmpty())
{
tool.name = call.path("function").path("name").asText("");
}
tool.arguments.append(call.path("function").path("arguments").asText(""));
if(!tool.announced && !tool.id.isEmpty() && !tool.name.isEmpty())
{
listener.onToolUseStart(tool.id, tool.name);
tool.announced = true;
}
}
if(choice.hasNonNull("finish_reason"))
{
stop = switch(choice.get("finish_reason").asText())
{
case "tool_calls", "function_call" -> ModelResponse.STOP_TOOL_USE;
case "length" -> ModelResponse.STOP_MAX_TOKENS;
case "content_filter" -> ModelResponse.STOP_REFUSAL;
default -> ModelResponse.STOP_END_TURN;
};
}
}
}
/*******************************************************************************
** Parse complete argument objects only, preserving parallel call order.
*******************************************************************************/
ModelResponse response()
{
List<ModelPart> parts = new ArrayList<>();
if(!text.isEmpty())
{
parts.add(ModelPart.text(text.toString()));
}
for(Map.Entry<Integer, Tool> entry : tools.entrySet())
{
Tool tool = entry.getValue();
if(tool.id.isEmpty())
{
tool.id = "call_" + entry.getKey();
}
if(!tool.announced)
{
listener.onToolUseStart(tool.id, tool.name);
tool.announced = true;
}
Map<String, Object> arguments;
try
{
arguments = tool.arguments.isEmpty() ? Map.of() : OpenAiHttp.JSON.readerFor(new TypeReference<Map<String, Object>>() {})
.with(com.fasterxml.jackson.databind.DeserializationFeature.FAIL_ON_TRAILING_TOKENS).readValue(tool.arguments.toString());
if(arguments == null)
{
throw (new IOException("Tool arguments must be an object."));
}
}
catch(IOException e)
{
arguments = Map.of("_invalid_arguments", tool.arguments.substring(0, Math.min(200, tool.arguments.length())));
}
parts.add(ModelPart.toolUse(tool.id, tool.name, arguments));
}
if(!tools.isEmpty() && ModelResponse.STOP_END_TURN.equals(stop))
{
stop = ModelResponse.STOP_TOOL_USE;
}
return (new ModelResponse().withModel(model).withRequestId(requestId).withParts(parts).withUsage(usage).withStopReason(stop));
}
}
}
@@ -0,0 +1,199 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant.provider.openai;
import java.util.List;
import com.kingsrook.qbits.assistant.provider.AssistantModelProvider;
import com.kingsrook.qbits.assistant.provider.AssistantProviderType;
import com.kingsrook.qbits.assistant.provider.DiscoveredModel;
import com.kingsrook.qbits.assistant.provider.HealthResult;
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
import com.kingsrook.qbits.assistant.provider.ProviderSettings;
/*******************************************************************************
** One installed adapter for the Chat Completions dialects.
*******************************************************************************/
public class OpenAiCompatibleProviderType implements AssistantProviderType
{
/*******************************************************************************
** Installed adapter id.
*******************************************************************************/
@Override
public String getType()
{
return ("openai-compatible");
}
/*******************************************************************************
** Settings label.
*******************************************************************************/
@Override
public String getLabel()
{
return ("OpenAI-compatible");
}
/*******************************************************************************
** Discover endpoint-specific models instead of guessing ids.
*******************************************************************************/
@Override
public List<String> getModels()
{
return (List.of());
}
/*******************************************************************************
** Extensible kind list.
*******************************************************************************/
@Override
public List<String> getKinds()
{
return (List.of("OPENAI", "AZURE_OPENAI", "OPENROUTER", "OLLAMA", "LM_STUDIO", "VLLM", "LLAMA_CPP", "OPENAI_COMPATIBLE"));
}
/*******************************************************************************
** The legacy factory uses the generic dialect.
*******************************************************************************/
@Override
public String getDefaultApiKeyEnvName()
{
return (null);
}
/*******************************************************************************
** The generic dialect requires an explicit URL.
*******************************************************************************/
@Override
public String getDefaultBaseUrl()
{
return (null);
}
/*******************************************************************************
** Built-in endpoints. Redirects are never followed.
*******************************************************************************/
@Override
public String getDefaultBaseUrl(String kind)
{
return (switch(kind)
{
case "OPENAI" -> "https://api.openai.com/v1";
case "OPENROUTER" -> "https://openrouter.ai/api/v1";
case "OLLAMA" -> "http://localhost:11434/v1";
case "LM_STUDIO" -> "http://localhost:1234/v1";
case "VLLM" -> "http://localhost:8000/v1";
case "LLAMA_CPP" -> "http://localhost:8080/v1";
default -> null;
});
}
/*******************************************************************************
** Default key references, never key values.
*******************************************************************************/
@Override
public String getDefaultApiKeyEnvName(String kind)
{
return (switch(kind)
{
case "OPENAI" -> "OPENAI_API_KEY";
case "AZURE_OPENAI" -> "AZURE_OPENAI_API_KEY";
case "OPENROUTER" -> "OPENROUTER_API_KEY";
default -> null;
});
}
/*******************************************************************************
** Location is an admin-editable fact, with a kind-based default.
*******************************************************************************/
@Override
public String getDefaultLocation(String kind)
{
return (List.of("OLLAMA", "LM_STUDIO", "VLLM", "LLAMA_CPP").contains(kind) ? "LOCAL" : "CLOUD");
}
/*******************************************************************************
** Backward-compatible factory.
*******************************************************************************/
@Override
public AssistantModelProvider create(String apiKey, String baseUrl)
{
return (create(new ProviderSettings().withKind("OPENAI_COMPATIBLE").withApiKey(apiKey).withBaseUrl(baseUrl)));
}
/*******************************************************************************
** Full connection settings.
*******************************************************************************/
@Override
public AssistantModelProvider create(ProviderSettings settings)
{
return (new OpenAiCompatibleModelProvider(settings));
}
/*******************************************************************************
** Only a models request, never a completion.
*******************************************************************************/
@Override
public List<DiscoveredModel> discoverModels(ProviderSettings settings) throws ModelProviderException
{
return (new OpenAiHttp(settings).models(settings.getRequestTimeoutSeconds()));
}
/*******************************************************************************
** A short models probe with latency and count.
*******************************************************************************/
@Override
public HealthResult checkHealth(ProviderSettings settings)
{
long start = System.nanoTime();
try
{
List<DiscoveredModel> models = new OpenAiHttp(settings).models(Math.min(10, settings.getRequestTimeoutSeconds()));
return (new HealthResult(true, (System.nanoTime() - start) / 1_000_000, models.size() + " models available"));
}
catch(ModelProviderException e)
{
return (new HealthResult(false, (System.nanoTime() - start) / 1_000_000,
e.getKind() + (e.getHttpStatus() == null ? "" : " HTTP " + e.getHttpStatus()) + ": " + e.getMessage()));
}
}
}
@@ -0,0 +1,267 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant.provider.openai;
import java.io.IOException;
import java.net.URI;
import java.net.URLEncoder;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.net.http.HttpTimeoutException;
import java.nio.charset.StandardCharsets;
import java.time.Duration;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import javax.net.ssl.SSLHandshakeException;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.kingsrook.qbits.assistant.provider.DiscoveredModel;
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
import com.kingsrook.qbits.assistant.provider.ProviderSettings;
/*******************************************************************************
** Shared HTTP/authentication/error mapping. ProviderService guards URLs before
** constructing this transport; redirects must not move keys to another host.
*******************************************************************************/
final class OpenAiHttp
{
static final ObjectMapper JSON = new ObjectMapper();
final HttpClient client;
final ProviderSettings settings;
private final String base;
/*******************************************************************************
** Copy settings so callers cannot mutate a cached client's credentials.
*******************************************************************************/
OpenAiHttp(ProviderSettings source)
{
settings = new ProviderSettings().withKind(source.getKind() == null ? "OPENAI_COMPATIBLE" : source.getKind())
.withTlsCaBundle(source.getTlsCaBundle()).withSslContext(source.getSslContext())
.withApiKey(source.getApiKey()).withBaseUrl(source.getBaseUrl()).withApiVersion(source.getApiVersion()).withRegion(source.getRegion())
.withConnectTimeoutSeconds(source.getConnectTimeoutSeconds()).withRequestTimeoutSeconds(source.getRequestTimeoutSeconds());
String url = settings.getBaseUrl() == null ? new OpenAiCompatibleProviderType().getDefaultBaseUrl(settings.getKind()) : settings.getBaseUrl();
if(url == null)
{
throw (new IllegalArgumentException("A base URL is required for this provider kind."));
}
base = url.replaceAll("/+$", "");
HttpClient.Builder builder = HttpClient.newBuilder().connectTimeout(Duration.ofSeconds(settings.getConnectTimeoutSeconds())).followRedirects(HttpClient.Redirect.NEVER);
if(settings.getSslContext() != null)
{
builder.sslContext(settings.getSslContext());
}
client = builder.build();
}
/*******************************************************************************
** Azure uses deployment paths and a version query parameter.
*******************************************************************************/
HttpRequest.Builder request(String model, Integer seconds) throws ModelProviderException
{
try
{
String path = model == null ? "/models" : "/chat/completions";
if("AZURE_OPENAI".equals(settings.getKind()))
{
path = model == null ? "/openai/models" : "/openai/deployments/" + encode(model) + "/chat/completions";
path += "?api-version=" + encode(settings.getApiVersion() == null || settings.getApiVersion().isBlank() ? "2024-10-21" : settings.getApiVersion());
}
HttpRequest.Builder builder = HttpRequest.newBuilder(URI.create(base + path)).timeout(Duration.ofSeconds(seconds)).header("Content-Type", "application/json");
if(settings.getApiKey() != null && !settings.getApiKey().isBlank())
{
if("AZURE_OPENAI".equals(settings.getKind()))
{
builder.header("api-key", settings.getApiKey());
}
else
{
builder.header("Authorization", "Bearer " + settings.getApiKey());
}
}
return (builder);
}
catch(IllegalArgumentException e)
{
throw (new ModelProviderException(ModelProviderException.Kind.INVALID_REQUEST, null, "The provider connection settings are invalid."));
}
}
/*******************************************************************************
** Models with optional server-specific context information.
*******************************************************************************/
List<DiscoveredModel> models(Integer seconds) throws ModelProviderException
{
CompletableFuture<HttpResponse<String>> pending = client.sendAsync(request(null, seconds).GET().build(), HttpResponse.BodyHandlers.ofString());
try
{
HttpResponse<String> response = pending.get(seconds, TimeUnit.SECONDS);
if(response.statusCode() != 200)
{
throw (error(response.statusCode(), response.body(), requestId(response), false));
}
JsonNode body = JSON.readTree(response.body());
if(body.has("error"))
{
throw (error(500, response.body(), requestId(response), false));
}
List<DiscoveredModel> result = new ArrayList<>();
for(JsonNode model : body.path("data"))
{
String id = model.path("id").asText();
if(id.isBlank())
{
continue;
}
int context = model.path("context_length").asInt(model.path("max_model_len").asInt(model.path("meta").path("n_ctx_train").asInt()));
result.add(new DiscoveredModel(id, model.path("name").asText(id), context > 0 ? context : null, model.path("owned_by").asText(null)));
}
return (result);
}
catch(TimeoutException e)
{
throw (new ModelProviderException(ModelProviderException.Kind.TIMEOUT, null, "The provider did not answer in time."));
}
catch(ExecutionException e)
{
throw (transportError(e));
}
catch(InterruptedException e)
{
Thread.currentThread().interrupt();
throw (new ModelProviderException(ModelProviderException.Kind.NETWORK, null, "The provider request was interrupted."));
}
catch(IOException e)
{
throw (transportError(e));
}
finally
{
pending.cancel(true);
client.close();
}
}
/*******************************************************************************
** Sanitize errors, explicitly removing even short or unusual API keys.
*******************************************************************************/
ModelProviderException error(int status, String body, String requestId, boolean strict)
{
String message = "The provider refused the request.";
String type = "openai-compatible";
try
{
JsonNode error = JSON.readTree(body).path("error");
message = error.path("message").asText(message);
type = error.path("type").asText(type);
}
catch(Exception e)
{
// Do not expose unstructured proxy pages or request bodies.
}
message = safe(message);
type = safe(type);
ModelProviderException.Kind kind = errorKind(status);
String lower = message == null ? "" : message.toLowerCase(Locale.ROOT);
return (new ModelProviderException(kind, status, message).withProviderDetails(type, message, safe(requestId))
.withStrictSchemaRejected(strict && status == 400 && (lower.contains("strict") || lower.contains("schema") || lower.contains("grammar"))));
}
/*******************************************************************************
** Credential-free diagnostics.
*******************************************************************************/
private String safe(String value)
{
String key = settings.getApiKey();
return (ModelProviderException.sanitizeProviderMessage(value == null || key == null || key.isEmpty() ? value : value.replace(key, "[redacted]"), 300));
}
/*******************************************************************************
** Common request-id headers.
*******************************************************************************/
static String requestId(HttpResponse<?> response)
{
return (response.headers().firstValue("x-request-id").orElse(response.headers().firstValue("apim-request-id").orElse(null)));
}
/*******************************************************************************
** Encode a single path segment or query value.
*******************************************************************************/
private static String encode(String value)
{
return (URLEncoder.encode(value, StandardCharsets.UTF_8).replace("+", "%20"));
}
/*******************************************************************************
** Map the provider error category.
*******************************************************************************/
private static ModelProviderException.Kind errorKind(int status)
{
return (switch(status)
{
case 401, 403 -> ModelProviderException.Kind.AUTHENTICATION;
case 429 -> ModelProviderException.Kind.RATE_LIMITED;
case 503, 529 -> ModelProviderException.Kind.OVERLOADED;
case 400, 404, 422 -> ModelProviderException.Kind.INVALID_REQUEST;
default -> status >= 500 ? ModelProviderException.Kind.SERVER : ModelProviderException.Kind.INVALID_REQUEST;
});
}
/*******************************************************************************
** Fixed diagnostics omit remote certificate details, URLs and credentials.
*******************************************************************************/
static ModelProviderException transportError(Throwable error)
{
for(Throwable cause = error; cause != null; cause = cause.getCause())
{
if(cause instanceof SSLHandshakeException)
{
return (new ModelProviderException(ModelProviderException.Kind.NETWORK, null,
"The provider TLS certificate is not trusted or does not match the host. Check tlsCaBundle and the HTTPS hostname."));
}
if(cause instanceof HttpTimeoutException)
{
return (new ModelProviderException(ModelProviderException.Kind.TIMEOUT, null, "The provider did not answer in time."));
}
}
return (new ModelProviderException(ModelProviderException.Kind.NETWORK, null, "Could not read the provider response."));
}
}
@@ -0,0 +1,190 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant.provider.openai;
import java.util.List;
import java.util.Locale;
import java.util.stream.Collectors;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.node.ArrayNode;
import com.fasterxml.jackson.databind.node.ObjectNode;
import com.kingsrook.qbits.assistant.provider.ModelCapabilities;
import com.kingsrook.qbits.assistant.provider.ModelMessage;
import com.kingsrook.qbits.assistant.provider.ModelPart;
import com.kingsrook.qbits.assistant.provider.ModelRequest;
import com.kingsrook.qbits.assistant.provider.ModelToolDefinition;
import com.kingsrook.qbits.assistant.provider.SystemBlock;
/*******************************************************************************
** Chat Completions request mapping and conservative strict-schema checks.
*******************************************************************************/
final class OpenAiMapping
{
/*******************************************************************************
** Utility class.
*******************************************************************************/
private OpenAiMapping()
{
}
/*******************************************************************************
** Map only capabilities the target dialect supports.
*******************************************************************************/
static ObjectNode request(ModelRequest request, String kind, ModelCapabilities capabilities) throws JsonProcessingException
{
ObjectNode body = OpenAiHttp.JSON.createObjectNode();
boolean streaming = Boolean.TRUE.equals(capabilities.getStreaming());
boolean nativeOpenAi = List.of("OPENAI", "AZURE_OPENAI").contains(kind);
body.put("model", request.getModel());
body.put("stream", streaming);
if(streaming)
{
body.putObject("stream_options").put("include_usage", true);
}
if(request.getMaxTokens() != null)
{
body.put(nativeOpenAi ? "max_completion_tokens" : "max_tokens", request.getMaxTokens());
}
if(Boolean.TRUE.equals(capabilities.getReasoningEffort()) && request.getEffort() != null)
{
String effort = request.getEffort().toLowerCase(Locale.ROOT);
body.put("reasoning_effort", List.of("low", "medium").contains(effort) ? effort : "high");
}
ArrayNode messages = body.putArray("messages");
if(!request.getSystem().isEmpty())
{
messages.addObject().put("role", "system").put("content", request.getSystem().stream().map(SystemBlock::getText).collect(Collectors.joining("\n\n")));
}
for(ModelMessage message : request.getMessages())
{
ObjectNode current = null;
for(ModelPart part : message.getParts())
{
if(part.getType() == ModelPart.Type.TOOL_RESULT)
{
current = null;
messages.addObject().put("role", "tool").put("tool_call_id", part.getToolUseId())
.put("content", (Boolean.TRUE.equals(part.getIsError()) ? "ERROR: " : "") + part.getText());
continue;
}
if(current == null)
{
current = messages.addObject().put("role", message.getRole() == ModelMessage.Role.ASSISTANT ? "assistant" : "user");
current.put("content", "");
}
if(part.getType() == ModelPart.Type.TEXT)
{
current.put("content", current.path("content").asText() + part.getText());
}
else if(part.getType() == ModelPart.Type.TOOL_USE)
{
ArrayNode calls = current.has("tool_calls") ? (ArrayNode) current.get("tool_calls") : current.putArray("tool_calls");
calls.addObject().put("id", part.getToolUseId()).put("type", "function").putObject("function")
.put("name", part.getToolName()).put("arguments", OpenAiHttp.JSON.writeValueAsString(part.getInput()));
}
}
}
if(!request.getTools().isEmpty() && Boolean.TRUE.equals(capabilities.getTools()))
{
ArrayNode tools = body.putArray("tools");
for(ModelToolDefinition tool : request.getTools())
{
JsonNode schema = OpenAiHttp.JSON.valueToTree(tool.getInputSchema());
ObjectNode function = tools.addObject().put("type", "function").putObject("function").put("name", tool.getName()).put("description", tool.getDescription());
function.set("parameters", schema);
if(nativeOpenAi && Boolean.TRUE.equals(tool.getStrict()) && strictCompatible(schema))
{
function.put("strict", true);
}
}
if(List.of("OPENAI", "AZURE_OPENAI", "OPENROUTER", "VLLM").contains(kind))
{
body.put("parallel_tool_calls", Boolean.TRUE.equals(request.getParallelToolCalls()) && Boolean.TRUE.equals(capabilities.getParallelToolCalls()));
}
}
return (body);
}
/*******************************************************************************
** Every object must forbid extras and require every declared property.
*******************************************************************************/
static boolean strictCompatible(JsonNode schema)
{
if(schema == null || !schema.isObject())
{
return (false);
}
JsonNode type = schema.path("type");
if("object".equals(type.asText()) || schema.has("properties") || (type.isArray() && type.toString().contains("\"object\"")))
{
if(!schema.has("additionalProperties") || !schema.get("additionalProperties").isBoolean() || schema.get("additionalProperties").asBoolean())
{
return (false);
}
JsonNode required = schema.path("required");
var names = schema.path("properties").fieldNames();
while(names.hasNext())
{
String name = names.next();
boolean found = false;
for(JsonNode value : required)
{
found |= name.equals(value.asText());
}
if(!found || !strictCompatible(schema.path("properties").get(name)))
{
return (false);
}
}
}
for(String keyword : List.of("items", "additionalProperties"))
{
if(schema.path(keyword).isObject() && !strictCompatible(schema.get(keyword)))
{
return (false);
}
}
for(String keyword : List.of("anyOf", "oneOf", "allOf", "prefixItems"))
{
for(JsonNode child : schema.path(keyword))
{
if(!strictCompatible(child))
{
return (false);
}
}
}
for(String keyword : List.of("$defs", "definitions"))
{
for(JsonNode child : schema.path(keyword))
{
if(!strictCompatible(child))
{
return (false);
}
}
}
return (true);
}
}
@@ -0,0 +1,77 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant.provider.openai;
import java.nio.file.Path;
import java.time.Duration;
import java.util.List;
import com.kingsrook.qbits.assistant.AssistantQBitConfig;
import com.kingsrook.qbits.assistant.provider.ModelMessage;
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
import com.kingsrook.qbits.assistant.provider.ModelRequest;
import com.kingsrook.qbits.assistant.provider.ProviderSettings;
import org.junit.jupiter.api.Assumptions;
import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** Opt-in smoke test; never contacts an endpoint in the default build.
*******************************************************************************/
@Tag("live")
class LiveOpenAiCompatibleTest
{
/*******************************************************************************
** All connection details come from environment references.
*******************************************************************************/
@Test
void live() throws Exception
{
String url = System.getenv("ASSISTANT_LIVE_OPENAI_BASE_URL");
String model = System.getenv("ASSISTANT_LIVE_OPENAI_MODEL");
Assumptions.assumeTrue(url != null && !url.isBlank() && model != null && !model.isBlank(), "Live URL and model must be set");
String kind = System.getenv("ASSISTANT_LIVE_OPENAI_KIND");
String keyEnv = System.getenv("ASSISTANT_LIVE_OPENAI_KEY_ENV");
ProviderSettings settings = new ProviderSettings().withKind(kind == null || kind.isBlank() ? "OLLAMA" : kind)
.withBaseUrl(url).withApiKey(keyEnv == null || keyEnv.isBlank() ? null : System.getenv(keyEnv));
String caFile = System.getenv("ASSISTANT_LIVE_OPENAI_CA_FILE");
if(caFile != null && !caFile.isBlank())
{
settings.withTlsCaBundle("live").withSslContext(new AssistantQBitConfig().withTrustedCaBundleFile("live", Path.of(caFile)).getTrustedCaBundles().get("live"));
}
try
{
OpenAiCompatibleProviderType type = new OpenAiCompatibleProviderType();
Assumptions.assumeTrue(Boolean.TRUE.equals(type.checkHealth(settings).healthy()), "Live health probe must pass");
assertThat(type.discoverModels(settings)).isNotEmpty();
var provider = type.create(settings);
StringBuilder text = new StringBuilder();
provider.stream(new ModelRequest().withModel(model).withMaxTokens(128).withMessages(List.of(ModelMessage.user("Reply with OK."))), text::append).await(Duration.ofSeconds(120));
assertThat(text).isNotEmpty();
var response = provider.stream(new ModelRequest().withModel(model).withMaxTokens(256).withTools(List.of(OpenAiContractTest.tool()))
.withMessages(List.of(ModelMessage.user("Call lookup with q set to hello. Do not answer without calling the tool."))), t -> {}).await(Duration.ofSeconds(120));
assertThat(response.getToolUses()).anyMatch(t -> "lookup".equals(t.getToolName()));
}
catch(ModelProviderException e)
{
Assumptions.assumeFalse(e.getKind() == ModelProviderException.Kind.NETWORK || e.getKind() == ModelProviderException.Kind.TIMEOUT, "Live endpoint unreachable (including TLS)");
throw (e);
}
}
}
@@ -0,0 +1,183 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant.provider.openai;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.charset.StandardCharsets;
import java.util.List;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.LinkedBlockingQueue;
import javax.net.ssl.SSLContext;
import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpServer;
import com.sun.net.httpserver.HttpsConfigurator;
import com.sun.net.httpserver.HttpsServer;
/*******************************************************************************
** Loopback-only scripted Chat Completions server with captured requests.
*******************************************************************************/
class MockOpenAiServer implements AutoCloseable
{
final List<Captured> requests = new CopyOnWriteArrayList<>();
private final LinkedBlockingQueue<Script> scripts = new LinkedBlockingQueue<>();
private final HttpServer server;
private final ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor();
/*******************************************************************************
** Captured wire data, only synthetic keys in default tests.
*******************************************************************************/
record Captured(String path, String authorization, String apiKey, String body)
{
}
/*******************************************************************************
** Delay can be before headers or after an SSE keep-alive.
*******************************************************************************/
record Script(int status, String body, long delayMs, boolean afterHeaders, CountDownLatch started)
{
}
/*******************************************************************************
** Bind only the loopback interface.
*******************************************************************************/
MockOpenAiServer() throws IOException
{
this(null);
}
/*******************************************************************************
** Optional TLS uses exactly the same scripted endpoints and request capture.
*******************************************************************************/
MockOpenAiServer(SSLContext tls) throws IOException
{
if(tls == null)
{
server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
}
else
{
HttpsServer https = HttpsServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
https.setHttpsConfigurator(new HttpsConfigurator(tls));
server = https;
}
server.createContext("/", this::serve);
server.setExecutor(executor);
server.start();
}
/*******************************************************************************
** Base URL deliberately has the v1 prefix.
*******************************************************************************/
String url()
{
return ((server instanceof HttpsServer ? "https://localhost:" : "http://127.0.0.1:") + server.getAddress().getPort() + "/v1");
}
/*******************************************************************************
** Queue a reply.
*******************************************************************************/
void enqueue(int status, String body)
{
scripts.add(new Script(status, body, 0, false, new CountDownLatch(1)));
}
/*******************************************************************************
** Queue a stalled response.
*******************************************************************************/
CountDownLatch stall(boolean afterHeaders)
{
CountDownLatch started = new CountDownLatch(1);
scripts.add(new Script(200, "data: [DONE]\n\n", 2500, afterHeaders, started));
return (started);
}
/*******************************************************************************
** Capture and reply, tolerating client cancellation.
*******************************************************************************/
private void serve(HttpExchange exchange) throws IOException
{
try(exchange)
{
requests.add(new Captured(exchange.getRequestURI().toString(), exchange.getRequestHeaders().getFirst("Authorization"),
exchange.getRequestHeaders().getFirst("api-key"), new String(exchange.getRequestBody().readAllBytes(), StandardCharsets.UTF_8)));
Script script = scripts.poll();
if(script == null)
{
exchange.sendResponseHeaders(500, -1);
return;
}
if(!script.afterHeaders())
{
script.started().countDown();
Thread.sleep(script.delayMs());
}
exchange.getResponseHeaders().set("Content-Type", script.body().startsWith("{") ? "application/json" : "text/event-stream");
exchange.getResponseHeaders().set("x-request-id", "mock-request");
exchange.sendResponseHeaders(script.status(), 0);
if(script.afterHeaders())
{
exchange.getResponseBody().write(": keep-alive\r\n\r\n".getBytes(StandardCharsets.UTF_8));
exchange.getResponseBody().flush();
script.started().countDown();
Thread.sleep(script.delayMs());
}
exchange.getResponseBody().write(script.body().getBytes(StandardCharsets.UTF_8));
}
catch(InterruptedException e)
{
Thread.currentThread().interrupt();
}
catch(IOException e)
{
// Cancellation closes the client socket while a script is writing.
}
}
/*******************************************************************************
** Stop all handlers.
*******************************************************************************/
@Override
public void close()
{
server.stop(0);
executor.shutdownNow();
}
}
@@ -0,0 +1,461 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant.provider.openai;
import java.time.Duration;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import com.fasterxml.jackson.databind.JsonNode;
import com.kingsrook.qbits.assistant.provider.ModelCapabilities;
import com.kingsrook.qbits.assistant.provider.ModelEventListener;
import com.kingsrook.qbits.assistant.provider.ModelMessage;
import com.kingsrook.qbits.assistant.provider.ModelPart;
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
import com.kingsrook.qbits.assistant.provider.ModelRequest;
import com.kingsrook.qbits.assistant.provider.ModelResponse;
import com.kingsrook.qbits.assistant.provider.ModelToolDefinition;
import com.kingsrook.qbits.assistant.provider.ProviderSettings;
import com.kingsrook.qbits.assistant.provider.SystemBlock;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import org.junit.jupiter.params.provider.ValueSource;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
/*******************************************************************************
** Contract tests exercise the wire protocol through localhost HTTP only.
*******************************************************************************/
class OpenAiContractTest
{
static final String TEXT = "data: {\"model\":\"test\",\"choices\":[{\"delta\":{\"content\":\"OK\"},\"finish_reason\":\"stop\"}]}\r\n\r\ndata: [DONE]\r\n\r\n";
/*******************************************************************************
** A minimal request, also shared by the opt-in live test.
*******************************************************************************/
static ModelRequest request()
{
return (new ModelRequest().withModel("test").withMaxTokens(32).withEffort("max").withMessages(List.of(ModelMessage.user("Hello"))));
}
/*******************************************************************************
** Synthetic settings for the local server.
*******************************************************************************/
static ProviderSettings settings(MockOpenAiServer server, String kind)
{
return (new ProviderSettings().withKind(kind).withBaseUrl(server.url()).withApiKey("synthetic-key"));
}
/*******************************************************************************
** Strict-compatible schema with one required property.
*******************************************************************************/
static ModelToolDefinition tool()
{
return (new ModelToolDefinition().withName("lookup").withDescription("Look up a value").withStrict(true)
.withInputSchema(Map.of("type", "object", "additionalProperties", false, "properties", Map.of("q", Map.of("type", "string")), "required", List.of("q"))));
}
/*******************************************************************************
** Kind-specific parameters, ordered tool results, auth and strict behavior.
*******************************************************************************/
@ParameterizedTest
@ValueSource(strings = { "OPENAI", "AZURE_OPENAI", "OPENROUTER", "OLLAMA", "LM_STUDIO", "VLLM", "LLAMA_CPP", "OPENAI_COMPATIBLE" })
void requestContract(String kind) throws Exception
{
try(MockOpenAiServer server = new MockOpenAiServer())
{
server.enqueue(200, TEXT);
boolean nativeKind = kind.equals("OPENAI") || kind.equals("AZURE_OPENAI");
ModelRequest request = request().withCapabilities(new ModelCapabilities().withReasoningEffort(true)).withTools(List.of(tool()))
.withSystem(List.of(new SystemBlock().withText("one"), new SystemBlock().withText("two")))
.withMessages(List.of(ModelMessage.user("hi"), new ModelMessage().withRole(ModelMessage.Role.ASSISTANT)
.withPart(ModelPart.toolUse("a", "lookup", Map.of("q", "a"))).withPart(ModelPart.toolUse("b", "lookup", Map.of("q", "b"))),
new ModelMessage().withRole(ModelMessage.Role.USER).withPart(ModelPart.toolResult("a", "first", false)).withPart(ModelPart.toolResult("b", "second", true))));
var provider = new OpenAiCompatibleModelProvider(settings(server, kind));
StringBuilder text = new StringBuilder();
assertThat(provider.stream(request, text::append).await(Duration.ofSeconds(5)).getText()).isEqualTo("OK");
assertThat(text.toString()).isEqualTo("OK");
var captured = server.requests.getFirst();
JsonNode body = OpenAiHttp.JSON.readTree(captured.body());
assertThat(body.path("messages").get(0).path("content").asText()).isEqualTo("one\n\ntwo");
assertThat(body.path("messages").get(2).path("tool_calls").get(0).path("function").path("arguments").asText()).isEqualTo("{\"q\":\"a\"}");
assertThat(body.path("messages").get(3).path("tool_call_id").asText()).isEqualTo("a");
assertThat(body.path("messages").get(4).path("content").asText()).isEqualTo("ERROR: second");
assertThat(body.path("tools").get(0).path("function").has("strict")).isEqualTo(nativeKind);
assertThat(body.has("max_completion_tokens")).isEqualTo(nativeKind);
assertThat(body.has("max_tokens")).isEqualTo(!nativeKind);
assertThat(body.path("reasoning_effort").asText()).isEqualTo("high");
assertThat(body.has("tool_choice")).isFalse();
assertThat(body.has("parallel_tool_calls")).isEqualTo(List.of("OPENAI", "AZURE_OPENAI", "OPENROUTER", "VLLM").contains(kind));
assertThat(provider.countTokens(request)).isNull();
if(kind.equals("AZURE_OPENAI"))
{
assertThat(captured.path()).isEqualTo("/v1/openai/deployments/test/chat/completions?api-version=2024-10-21");
assertThat(captured.apiKey()).isEqualTo("synthetic-key");
assertThat(captured.authorization()).isNull();
}
else
{
assertThat(captured.authorization()).isEqualTo("Bearer synthetic-key");
}
}
}
/*******************************************************************************
** Fragmented parallel tools, comments, cache usage and ignored reasoning.
*******************************************************************************/
@Test
void streamedToolsAndUsage() throws Exception
{
try(MockOpenAiServer server = new MockOpenAiServer())
{
StringBuilder script = new StringBuilder(": comment\r\n\r\n");
script.append(event(Map.of("choices", List.of(Map.of("delta", Map.of("reasoning_content", "never expose", "tool_calls", List.of(
Map.of("index", 1, "id", "b", "function", Map.of("name", "lookup", "arguments", "")),
Map.of("index", 0, "id", "a", "function", Map.of("name", "lookup", "arguments", "")))))))));
for(char c : "{\"q\":\"value\"}".toCharArray())
{
script.append(event(Map.of("choices", List.of(Map.of("delta", Map.of("tool_calls", List.of(
Map.of("index", 0, "function", Map.of("arguments", String.valueOf(c))), Map.of("index", 1, "function", Map.of("arguments", String.valueOf(c))))))))));
}
script.append(event(Map.of("choices", List.of(Map.of("delta", Map.of(), "finish_reason", "tool_calls")), "usage",
Map.of("prompt_tokens", 100, "completion_tokens", 8, "prompt_tokens_details", Map.of("cached_tokens", 70)))));
server.enqueue(200, script + "data: [DONE]\n\n");
List<String> starts = new ArrayList<>();
StringBuilder text = new StringBuilder();
ModelResponse response = new OpenAiCompatibleModelProvider(settings(server, "OLLAMA").withApiKey(null)).stream(request(), new ModelEventListener()
{
/*******************************************************************************
** Provider contract.
*******************************************************************************/
@Override
public void onTextDelta(String delta)
{
text.append(delta);
}
/*******************************************************************************
** Provider contract.
*******************************************************************************/
@Override
public void onToolUseStart(String id, String name)
{
starts.add(id);
}
}).await(Duration.ofSeconds(5));
assertThat(response.getToolUses()).extracting(ModelPart::getToolUseId).containsExactly("a", "b");
assertThat(response.getToolUses().getFirst().getInput()).containsEntry("q", "value");
assertThat(starts).containsExactly("b", "a");
assertThat(text).isEmpty();
assertThat(response.getUsage().getInputTokens()).isEqualTo(30L);
assertThat(response.getUsage().getCacheReadTokens()).isEqualTo(70L);
assertThat(response.getUsage().getOutputTokens()).isEqualTo(8L);
assertThat(response.getStopReason()).isEqualTo(ModelResponse.STOP_TOOL_USE);
assertThat(server.requests.getFirst().authorization()).isNull();
}
}
/*******************************************************************************
** Single JSON mode and defaults without optional request fields.
*******************************************************************************/
@Test
void nonStreamingAndNoUsage() throws Exception
{
try(MockOpenAiServer server = new MockOpenAiServer())
{
server.enqueue(200, "{\"choices\":[{\"message\":{\"content\":\"done\"},\"finish_reason\":\"length\"}]}");
var provider = new OpenAiCompatibleModelProvider(settings(server, "OPENAI"));
StringBuilder text = new StringBuilder();
ModelResponse response = provider.stream(request().withCapabilities(new ModelCapabilities().withStreaming(false)), text::append).await(Duration.ofSeconds(5));
assertThat(response.getStopReason()).isEqualTo(ModelResponse.STOP_MAX_TOKENS);
assertThat(text.toString()).isEqualTo("done");
assertThat(response.getUsage().billableTokens()).isZero();
JsonNode body = OpenAiHttp.JSON.readTree(server.requests.getFirst().body());
assertThat(body.path("stream").asBoolean()).isFalse();
assertThat(body.has("stream_options")).isFalse();
assertThat(body.has("reasoning_effort")).isFalse();
assertThat(body.has("parallel_tool_calls")).isFalse();
server.enqueue(200, TEXT);
assertThat(provider.stream(request(), text::append).await(Duration.ofSeconds(5)).getUsage().billableTokens()).isZero();
}
}
/*******************************************************************************
** Every HTTP error category carries safe details.
*******************************************************************************/
@ParameterizedTest
@CsvSource({ "401,AUTHENTICATION", "403,AUTHENTICATION", "429,RATE_LIMITED", "503,OVERLOADED", "529,OVERLOADED", "400,INVALID_REQUEST", "404,INVALID_REQUEST", "422,INVALID_REQUEST", "500,SERVER", "502,SERVER" })
void errors(int status, String kind) throws Exception
{
try(MockOpenAiServer server = new MockOpenAiServer())
{
server.enqueue(status, "{\"error\":{\"type\":\"bad_request\",\"message\":\"Rejected synthetic-key\"}}");
assertThatThrownBy(() -> new OpenAiCompatibleModelProvider(settings(server, "OPENAI")).stream(request(), t -> {}).await(Duration.ofSeconds(5)))
.isInstanceOfSatisfying(ModelProviderException.class, e ->
{
assertThat(e.getKind().name()).isEqualTo(kind);
assertThat(e.getHttpStatus()).isEqualTo(status);
assertThat(e.getRequestId()).isEqualTo("mock-request");
assertThat(e.getMessage()).doesNotContain("synthetic-key");
assertThat(e.getProviderMessage()).contains("[redacted]");
});
}
}
/*******************************************************************************
** Strict rejection is opt-in; incompatible nested object schemas are omitted.
*******************************************************************************/
@Test
void strictAndStreamErrors() throws Exception
{
try(MockOpenAiServer server = new MockOpenAiServer())
{
var provider = new OpenAiCompatibleModelProvider(settings(server, "OPENAI"));
String error = "{\"error\":{\"type\":\"invalid_request_error\",\"message\":\"strict schema grammar refused\"}}";
for(boolean sse : List.of(false, true))
{
server.enqueue(sse ? 200 : 400, sse ? "data: " + error + "\n\n" : error);
assertThatThrownBy(() -> provider.stream(request().withTools(List.of(tool())), t -> {}).await(Duration.ofSeconds(5)))
.isInstanceOfSatisfying(ModelProviderException.class, e -> assertThat(e.isStrictSchemaRejected()).isTrue());
}
server.enqueue(200, TEXT);
provider.stream(request().withTools(List.of(tool().withInputSchema(Map.of("type", "object", "properties", Map.of())))), t -> {}).await(Duration.ofSeconds(5));
assertThat(OpenAiHttp.JSON.readTree(server.requests.getLast().body()).path("tools").get(0).path("function").has("strict")).isFalse();
assertThat(OpenAiMapping.strictCompatible(OpenAiHttp.JSON.readTree("{\"type\":\"array\",\"items\":{\"type\":\"object\"}}"))).isFalse();
assertThat(OpenAiMapping.strictCompatible(OpenAiHttp.JSON.readTree("{\"anyOf\":[{\"type\":\"object\"}]}"))).isFalse();
}
}
/*******************************************************************************
** Header timeout, stalled body timeout, cancellation and network failure.
*******************************************************************************/
@Test
void timeoutCancelAndNetwork() throws Exception
{
try(MockOpenAiServer server = new MockOpenAiServer())
{
var provider = new OpenAiCompatibleModelProvider(settings(server, "OLLAMA").withRequestTimeoutSeconds(1));
for(boolean afterHeaders : List.of(false, true))
{
server.stall(afterHeaders);
assertThatThrownBy(() -> provider.stream(request(), t -> {}).await(Duration.ofSeconds(5)))
.isInstanceOfSatisfying(ModelProviderException.class, e -> assertThat(e.getKind()).isEqualTo(ModelProviderException.Kind.TIMEOUT));
}
var started = server.stall(true);
var handle = provider.stream(request(), t -> {});
assertThat(started.await(3, java.util.concurrent.TimeUnit.SECONDS)).isTrue();
handle.cancel();
assertThat(handle.await(Duration.ofSeconds(2)).getStopReason()).isEqualTo(ModelResponse.STOP_CANCELLED);
assertThat(handle.isCancelled()).isTrue();
}
String closed;
try(MockOpenAiServer server = new MockOpenAiServer())
{
closed = server.url();
}
var provider = new OpenAiCompatibleModelProvider(new ProviderSettings().withKind("OLLAMA").withBaseUrl(closed));
assertThatThrownBy(() -> provider.stream(request(), t -> {}).await(Duration.ofSeconds(5)))
.isInstanceOfSatisfying(ModelProviderException.class, e -> assertThat(e.getKind()).isEqualTo(ModelProviderException.Kind.NETWORK));
}
/*******************************************************************************
** Discovery timeouts include the body, and failed probes never complete.
*******************************************************************************/
@Test
void discoveryTimeoutAndMalformedResponse() throws Exception
{
try(MockOpenAiServer server = new MockOpenAiServer())
{
var type = new OpenAiCompatibleProviderType();
var settings = settings(server, "OLLAMA").withRequestTimeoutSeconds(1);
server.stall(true);
assertThatThrownBy(() -> type.discoverModels(settings)).isInstanceOfSatisfying(ModelProviderException.class,
e -> assertThat(e.getKind()).isEqualTo(ModelProviderException.Kind.TIMEOUT));
server.enqueue(200, "not JSON");
assertThat(type.checkHealth(settings).healthy()).isFalse();
server.enqueue(200, "{\"error\":{\"message\":\"failed\"}}");
assertThat(type.checkHealth(settings).healthy()).isFalse();
}
}
/*******************************************************************************
** JDK header validation must never expose even an invalid credential.
*******************************************************************************/
@Test
void invalidHeaderDoesNotLeakKey() throws Exception
{
try(MockOpenAiServer server = new MockOpenAiServer())
{
var settings = settings(server, "OPENAI").withApiKey("private\ncredential");
assertThatThrownBy(() -> new OpenAiCompatibleProviderType().discoverModels(settings))
.isInstanceOfSatisfying(ModelProviderException.class, e ->
{
assertThat(e.getMessage()).doesNotContain("private", "credential");
assertThat(e.getCause()).isNull();
});
assertThat(new OpenAiCompatibleProviderType().checkHealth(settings).healthy()).isFalse();
assertThat(server.requests).isEmpty();
}
}
/*******************************************************************************
** Non-streaming calls preserve tools and usage, including cached tokens.
*******************************************************************************/
@Test
void nonStreamingTools() throws Exception
{
try(MockOpenAiServer server = new MockOpenAiServer())
{
server.enqueue(200, """
{"choices":[{"message":{"tool_calls":[
{"id":"one","function":{"name":"lookup","arguments":"{\\"q\\":\\"x\\"}"}},
{"id":"two","function":{"name":"lookup","arguments":"{}"}}]},"finish_reason":"tool_calls"}],
"usage":{"prompt_tokens":12,"completion_tokens":4,"prompt_tokens_details":{"cached_tokens":2}}}
""");
var response = new OpenAiCompatibleModelProvider(settings(server, "OPENAI")).stream(
request().withCapabilities(new ModelCapabilities().withStreaming(false)), t -> {}).await(Duration.ofSeconds(5));
assertThat(response.getToolUses()).extracting(ModelPart::getToolUseId).containsExactly("one", "two");
assertThat(response.getToolUses().getFirst().getInput()).containsEntry("q", "x");
assertThat(response.getUsage().getInputTokens()).isEqualTo(10L);
}
}
/*******************************************************************************
** Model discovery context dialects, Azure path and health without completions.
*******************************************************************************/
@ParameterizedTest
@ValueSource(strings = { "OPENAI", "OPENROUTER", "AZURE_OPENAI", "VLLM", "LLAMA_CPP" })
void discoveryAndHealth(String kind) throws Exception
{
try(MockOpenAiServer server = new MockOpenAiServer())
{
String models = "{\"data\":[{\"id\":\"a\",\"owned_by\":\"owner\",\"context_length\":64000},{\"id\":\"b\",\"max_model_len\":32000},{\"id\":\"c\",\"meta\":{\"n_ctx_train\":16000}}]}";
server.enqueue(200, models);
var type = new OpenAiCompatibleProviderType();
var settings = settings(server, kind).withApiVersion("2024-10-21");
assertThat(type.discoverModels(settings)).extracting(m -> m.contextWindowTokens()).containsExactly(64000, 32000, 16000);
server.enqueue(200, models);
assertThat(type.checkHealth(settings).message()).isEqualTo("3 models available");
server.enqueue(401, "{\"error\":{\"message\":\"bad key\"}}");
assertThat(type.checkHealth(settings).healthy()).isFalse();
assertThat(server.requests).allSatisfy(r -> assertThat(r.path()).isEqualTo(kind.equals("AZURE_OPENAI") ? "/v1/openai/models?api-version=2024-10-21" : "/v1/models"));
assertThat(type.getKinds()).hasSize(8);
assertThat(type.getModels()).isEmpty();
assertThat(type.getDefaultBaseUrl()).isNull();
assertThat(type.getDefaultApiKeyEnvName()).isNull();
assertThat(type.getLabel()).isEqualTo("OpenAI-compatible");
for(String k : type.getKinds())
{
assertThat(type.getDefaultLocation(k)).isIn("LOCAL", "CLOUD");
type.getDefaultBaseUrl(k);
type.getDefaultApiKeyEnvName(k);
}
assertThat(type.create(null, server.url()).getName()).isEqualTo(type.getType());
}
}
/*******************************************************************************
** Serialize a scripted SSE event.
*******************************************************************************/
static String event(Object value) throws Exception
{
return ("data: " + OpenAiHttp.JSON.writeValueAsString(value) + "\n\n");
}
/*******************************************************************************
** Repeated identities, missing ids and older servers' stop reason are tolerated.
*******************************************************************************/
@Test
void repeatedToolIdentityAndMissingId() throws Exception
{
try(MockOpenAiServer server = new MockOpenAiServer())
{
StringBuilder script = new StringBuilder();
for(String fragment : List.of("{", "\"q\":", "\"hello\"}"))
{
script.append(event(Map.of("choices", List.of(Map.of("delta", Map.of("tool_calls", List.of(
Map.of("index", 0, "id", "repeated", "function", Map.of("name", "lookup", "arguments", fragment)),
Map.of("index", 1, "function", Map.of("name", "lookup", "arguments", fragment)))), "finish_reason", "stop")))));
}
server.enqueue(200, script.toString());
ModelResponse response = new OpenAiCompatibleModelProvider(settings(server, "OLLAMA")).stream(request(), t -> {}).await(Duration.ofSeconds(5));
assertThat(response.getStopReason()).isEqualTo(ModelResponse.STOP_TOOL_USE);
assertThat(response.getToolUses()).extracting(ModelPart::getToolUseId).containsExactly("repeated", "call_1");
assertThat(response.getToolUses()).allSatisfy(tool ->
{
assertThat(tool.getToolName()).isEqualTo("lookup");
assertThat(tool.getInput()).containsExactlyEntriesOf(Map.of("q", "hello"));
});
}
}
/*******************************************************************************
** Invalid JSON becomes bounded input for kernel validation, never NETWORK.
*******************************************************************************/
@ParameterizedTest
@ValueSource(strings = { "{broken", "null", "[]", "{} trailing", "long" })
void malformedArguments(String input) throws Exception
{
String arguments = input.equals("long") ? "x".repeat(250) : input;
try(MockOpenAiServer server = new MockOpenAiServer())
{
server.enqueue(200, event(Map.of("choices", List.of(Map.of("delta", Map.of("tool_calls", List.of(
Map.of("index", 0, "function", Map.of("name", "lookup", "arguments", arguments)))), "finish_reason", "stop")))));
ModelResponse response = new OpenAiCompatibleModelProvider(settings(server, "LLAMA_CPP")).stream(request(), t -> {}).await(Duration.ofSeconds(5));
assertThat(response.getStopReason()).isEqualTo(ModelResponse.STOP_TOOL_USE);
assertThat(response.getToolUses().getFirst().getInput()).containsExactlyEntriesOf(Map.of("_invalid_arguments", arguments.substring(0, Math.min(200, arguments.length()))));
}
}
}
@@ -0,0 +1,87 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant.provider.openai;
import java.util.List;
import java.util.Map;
import com.fasterxml.jackson.databind.JsonNode;
import com.kingsrook.qbits.assistant.AssistantQBitConfig;
import com.kingsrook.qbits.assistant.testing.AssistantTestFixture;
import com.kingsrook.qbits.assistant.testing.GoldenTasks;
import com.kingsrook.qbits.assistant.testing.TurnRunner;
import com.kingsrook.qqq.backend.core.context.QContext;
import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** Golden fixture task through the real adapter, engine and tool result mapping.
*******************************************************************************/
@Tag("golden")
class OpenAiGoldenTurnTest
{
/*******************************************************************************
** A golden task travels through the engine, real UI tools and a tool round trip.
*******************************************************************************/
@Test
void goldenToolRoundTrip() throws Exception
{
try(MockOpenAiServer server = new MockOpenAiServer())
{
Map<String, Object> arguments = Map.of("table", "blog_post", "filter", Map.of("booleanOperator", "AND", "criteria",
List.of(Map.of("field", "status", "operator", "EQUALS", "values", List.of("PUBLISHED")))));
Map<String, Object> call = Map.of("index", 0, "id", "count", "function", Map.of("name", "count_records", "arguments", OpenAiHttp.JSON.writeValueAsString(arguments)));
server.enqueue(200, OpenAiContractTest.event(Map.of("choices", List.of(Map.of("delta", Map.of("tool_calls", List.of(call)), "finish_reason", "stop")))));
server.enqueue(200, OpenAiContractTest.event(Map.of("choices", List.of(Map.of("delta", Map.of("content", "There are " + AssistantTestFixture.PUBLISHED_POSTS + " published posts. /app/blog_post"), "finish_reason", "stop")))));
var provider = new OpenAiCompatibleModelProvider(OpenAiContractTest.settings(server, "OLLAMA"));
var config = new AssistantQBitConfig().withProviderName(provider.getName()).withModelProvider(provider);
try
{
var instance = AssistantTestFixture.build(config);
var task = GoldenTasks.all().stream().filter(t -> t.id().equals("count-published-posts")).findFirst().orElseThrow();
var outcome = new TurnRunner(instance, config).run(task.userId(), task.prompt());
assertThat(outcome.errorCode()).isNull();
task.check().accept(outcome);
assertThat(outcome.text()).isEqualTo("There are " + AssistantTestFixture.PUBLISHED_POSTS + " published posts. /app/blog_post");
assertThat(outcome.end()).isNotEmpty();
assertThat(server.requests).hasSize(2);
JsonNode messages = OpenAiHttp.JSON.readTree(server.requests.get(1).body()).path("messages");
JsonNode assistant = messages.get(messages.size() - 2);
assertThat(assistant.path("role").asText()).isEqualTo("assistant");
assertThat(assistant.path("tool_calls")).hasSize(1);
assertThat(assistant.path("tool_calls").get(0).path("id").asText()).isEqualTo("count");
assertThat(assistant.path("tool_calls").get(0).path("function").path("name").asText()).isEqualTo("count_records");
assertThat(messages.get(messages.size() - 1).path("content").asText()).contains("count").doesNotStartWith("ERROR:");
assertThat(messages.get(messages.size() - 1).path("role").asText()).isEqualTo("tool");
assertThat(messages.get(messages.size() - 1).path("tool_call_id").asText()).isEqualTo("count");
}
finally
{
if(config.getRuntime() != null)
{
config.getRuntime().close();
}
QContext.clear();
}
}
}
}
@@ -0,0 +1,216 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant.provider.openai;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.security.KeyStore;
import java.time.Duration;
import java.util.Set;
import javax.net.ssl.KeyManagerFactory;
import javax.net.ssl.SSLContext;
import com.kingsrook.qbits.assistant.AssistantQBitConfig;
import com.kingsrook.qbits.assistant.model.AssistantProvider;
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
import com.kingsrook.qbits.assistant.provider.ProviderSettings;
import com.kingsrook.qbits.assistant.testing.AssistantTestFixture;
import com.kingsrook.qqq.backend.core.context.QContext;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
/*******************************************************************************
** Private CA coverage against loopback HTTPS; all key material is test-only.
*******************************************************************************/
class OpenAiTlsTest
{
@TempDir
Path temporary;
/*******************************************************************************
** The public test root, never a real deployment CA.
*******************************************************************************/
private String pem() throws Exception
{
try(InputStream input = getClass().getResourceAsStream("/tls/ca.pem"))
{
return (new String(input.readAllBytes(), StandardCharsets.US_ASCII));
}
}
/*******************************************************************************
** Leaf chain signed by the test root, valid for localhost but not its IP.
*******************************************************************************/
private SSLContext serverContext() throws Exception
{
KeyStore store = KeyStore.getInstance("PKCS12");
try(InputStream input = getClass().getResourceAsStream("/tls/server.p12"))
{
store.load(input, "test-only".toCharArray());
}
KeyManagerFactory keys = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
keys.init(store, "test-only".toCharArray());
SSLContext context = SSLContext.getInstance("TLS");
context.init(keys.getKeyManagers(), null, null);
return (context);
}
/*******************************************************************************
** Every operation rejects the unknown CA with a safe, actionable diagnostic.
*******************************************************************************/
@Test
void untrustedCertificate() throws Exception
{
try(MockOpenAiServer server = new MockOpenAiServer(serverContext()))
{
ProviderSettings settings = OpenAiContractTest.settings(server, "OLLAMA");
var type = new OpenAiCompatibleProviderType();
assertThatThrownBy(() -> type.discoverModels(settings)).isInstanceOfSatisfying(ModelProviderException.class, this::trustError);
assertThat(type.checkHealth(settings).healthy()).isFalse();
assertThat(type.checkHealth(settings).message()).contains("NETWORK", "tlsCaBundle");
assertThatThrownBy(() -> type.create(settings).stream(OpenAiContractTest.request(), t -> {}).await(Duration.ofSeconds(5)))
.isInstanceOfSatisfying(ModelProviderException.class, this::trustError);
assertThat(server.requests).isEmpty();
}
}
/*******************************************************************************
** No raw handshake details, keys, or certificate contents escape.
*******************************************************************************/
private void trustError(ModelProviderException error)
{
assertThat(error.getKind()).isEqualTo(ModelProviderException.Kind.NETWORK);
assertThat(error.getMessage()).contains("certificate is not trusted", "tlsCaBundle").doesNotContain("synthetic-key", "TEST ONLY", "BEGIN CERTIFICATE");
assertThat(error.getCause()).isNull();
}
/*******************************************************************************
** Shared trust enables all operations while endpoint identification stays on.
*******************************************************************************/
@Test
void trustedHealthDiscoveryChatAndHostnameCheck() throws Exception
{
var config = new AssistantQBitConfig().withTrustedCaBundle("test-root", pem());
try(MockOpenAiServer server = new MockOpenAiServer(serverContext()))
{
ProviderSettings settings = OpenAiContractTest.settings(server, "OLLAMA").withTlsCaBundle("test-root")
.withSslContext(config.getTrustedCaBundles().get("test-root"));
var type = new OpenAiCompatibleProviderType();
String models = "{\"data\":[{\"id\":\"test\"}]}";
server.enqueue(200, models);
assertThat(type.checkHealth(settings).healthy()).isTrue();
server.enqueue(200, models);
assertThat(type.discoverModels(settings)).extracting(m -> m.modelId()).containsExactly("test");
server.enqueue(200, OpenAiContractTest.TEXT);
assertThat(type.create(settings).stream(OpenAiContractTest.request(), t -> {}).await(Duration.ofSeconds(5)).getText()).isEqualTo("OK");
assertThat(server.requests).hasSize(3).allSatisfy(r -> assertThat(r.authorization()).isEqualTo("Bearer synthetic-key"));
settings.withBaseUrl(server.url().replace("localhost", "127.0.0.1"));
assertThatThrownBy(() -> type.discoverModels(settings)).isInstanceOfSatisfying(ModelProviderException.class, this::trustError);
assertThatThrownBy(() -> type.create(settings).stream(OpenAiContractTest.request(), t -> {}).await(Duration.ofSeconds(5)))
.isInstanceOfSatisfying(ModelProviderException.class, this::trustError);
assertThat(server.requests).hasSize(3);
}
}
/*******************************************************************************
** Parse at startup, accept multiple PEMs, omit contexts from serialized beans.
*******************************************************************************/
@Test
void eagerConfigAndSerialization() throws Exception
{
String pem = pem();
Path file = temporary.resolve("bundle.pem");
Files.writeString(file, pem + pem);
var config = new AssistantQBitConfig().withTrustedCaBundleFile("test-root", file);
Files.delete(file);
assertThat(config.getTrustedCaBundles()).containsKey("test-root");
assertThatThrownBy(() -> config.getTrustedCaBundles().clear()).isInstanceOf(UnsupportedOperationException.class);
assertThat(OpenAiHttp.JSON.writeValueAsString(config)).doesNotContain("trustedCaBundles", "BEGIN CERTIFICATE", "test-root");
var settings = new ProviderSettings().withSslContext(config.getTrustedCaBundles().get("test-root"));
assertThat(OpenAiHttp.JSON.writeValueAsString(settings)).doesNotContain("sslContext", "BEGIN CERTIFICATE");
for(String invalid : new String[] { null, "", "private-content", pem + "private-content", "-----BEGIN CERTIFICATE-----bad-----END CERTIFICATE-----" })
{
assertThatThrownBy(() -> config.withTrustedCaBundle("bad", invalid)).isInstanceOf(IllegalArgumentException.class)
.hasMessageContaining("Invalid CA bundle").hasMessageNotContaining("private-content").hasNoCause();
}
assertThatThrownBy(() -> config.withTrustedCaBundle(" ", pem)).hasMessageContaining("CA bundle name");
assertThatThrownBy(() -> config.withTrustedCaBundleFile("missing", file)).hasMessage("Could not read the configured CA bundle file.").hasNoCause();
}
/*******************************************************************************
** Settings saves reject unknown names and HTTP; changing names rebuilds clients.
*******************************************************************************/
@Test
void registryValidationAndClientCache() throws Exception
{
var config = new AssistantQBitConfig().withProviderType(new OpenAiCompatibleProviderType())
.withAllowedBaseUrls(Set.of("https://localhost:8443/v1"))
.withTrustedCaBundle("first", pem()).withTrustedCaBundle("second", pem());
try
{
var instance = AssistantTestFixture.build(config);
QContext.init(instance, AssistantTestFixture.sessionFor(AssistantTestFixture.ADMIN));
var service = config.getRuntime().getProviderService();
var row = new AssistantProvider().withName("Test TLS").withProviderType("openai-compatible").withKind("OLLAMA")
.withBaseUrl("https://localhost:8443/v1").withTlsCaBundle("unknown");
assertThat(service.validate(row)).anyMatch(e -> e.contains("configured on this server"));
assertThatThrownBy(() -> service.settings(row)).hasMessageContaining("TLS CA bundle");
assertThatThrownBy(() -> config.getRuntime().getStore().insert(row)).hasMessageContaining("TLS CA bundle");
row.withTlsCaBundle("first").withBaseUrl(null);
assertThat(service.validate(row)).anyMatch(e -> e.contains("HTTPS"));
row.withBaseUrl("https://localhost:8443/v1");
assertThat(service.validate(row)).isEmpty();
row.withId(config.getRuntime().getStore().insert(row).getId());
assertThat(row.getTlsCaBundle()).isEqualTo("first");
assertThat(service.settings(row).getSslContext()).isSameAs(config.getTrustedCaBundles().get("first"));
var first = service.forRow(row, "test").provider();
assertThat(service.forRow(row, "test").provider()).isSameAs(first);
row.withTlsCaBundle("second");
assertThat(service.forRow(row, "test").provider()).isNotSameAs(first);
row.withTlsCaBundle(null);
assertThat(service.settings(row).getSslContext()).isNull();
assertThat(instance.getTable(AssistantProvider.TABLE_NAME).getField("tlsCaBundle").getMaxLength()).isEqualTo(100);
}
finally
{
if(config.getRuntime() != null)
{
config.getRuntime().close();
}
QContext.clear();
}
}
}
@@ -0,0 +1,251 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant.provider.openai.demo;
import java.io.IOException;
import java.math.BigDecimal;
import java.net.InetSocketAddress;
import java.nio.charset.StandardCharsets;
import java.util.Map;
import java.util.Set;
import com.kingsrook.qbits.assistant.AssistantQBitConfig;
import com.kingsrook.qbits.assistant.metadata.AssistantMetaDataBuilder;
import com.kingsrook.qbits.assistant.model.AssistantModel;
import com.kingsrook.qbits.assistant.model.AssistantProvider;
import com.kingsrook.qbits.assistant.persistence.AssistantStore;
import com.kingsrook.qbits.assistant.processes.DiscoverModelsProcess;
import com.kingsrook.qbits.assistant.processes.TestConnectionProcess;
import com.kingsrook.qbits.assistant.processes.TestModelConnectionProcess;
import com.kingsrook.qbits.assistant.provider.AssistantProviderType;
import com.kingsrook.qbits.assistant.provider.fake.ScriptedModelProvider;
import com.kingsrook.qbits.assistant.provider.openai.OpenAiCompatibleProviderType;
import com.kingsrook.qbits.assistant.testing.AssistantTestFixture;
import com.kingsrook.qbits.assistant.testing.TurnRunner;
import com.kingsrook.qqq.backend.core.actions.processes.QProcessCallbackFactory;
import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.instances.AbstractQQQApplication;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.model.session.QSession;
import com.kingsrook.qqq.backend.core.modules.authentication.QAuthenticationModuleCustomizerInterface;
import com.kingsrook.qqq.middleware.javalin.QApplicationJavalinServer;
import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpServer;
/*******************************************************************************
** Memory-backed P-a screenshot demo. All discovery, tests and usage come from
** real application calls to a localhost mock; no external credentials are used.
*******************************************************************************/
public class OpenAiDemoServer extends AbstractQQQApplication
{
private static final String ADMIN = "admin-demo";
private static final String MODEL = "qwen3:30b-a3b";
private static QInstance instance;
/*******************************************************************************
** Give browser sessions the same admin identity used for demo traffic.
*******************************************************************************/
public static class DemoSessionCustomizer implements QAuthenticationModuleCustomizerInterface
{
/*******************************************************************************
**
*******************************************************************************/
@Override
public void customizeSession(QInstance qInstance, QSession qSession, Map<String, Object> context)
{
QSession admin = AssistantTestFixture.sessionFor(ADMIN);
qSession.setUser(admin.getUser());
qSession.setPermissions(admin.getPermissions());
qSession.withSecurityKeyValue(AssistantMetaDataBuilder.allAccessKeyName(AssistantQBitConfig.DEFAULT_OWNER_KEY_TYPE), true);
}
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public QInstance defineQInstance() throws QException
{
return (instance);
}
/*******************************************************************************
** Seed settings, execute the record actions and a turn, then serve Next UI.
*******************************************************************************/
public static void main(String[] args) throws Exception
{
int port = args.length > 0 ? Integer.parseInt(args[0]) : 8099;
HttpServer mock = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
mock.createContext("/", OpenAiDemoServer::serveMock);
mock.start();
Runtime.getRuntime().addShutdownHook(new Thread(() -> mock.stop(0)));
String baseUrl = "http://127.0.0.1:" + mock.getAddress().getPort() + "/v1";
AssistantQBitConfig config = new AssistantQBitConfig()
.withProviderName(ScriptedModelProvider.DEFAULT_NAME)
.withModelProvider(new ScriptedModelProvider())
.withProviderType(new OpenAiCompatibleProviderType())
.withAllowedBaseUrls(Set.of(baseUrl))
.withSecretResolver(name -> null)
.withAllowedWidgetNames(Set.of());
AssistantProviderType anthropic = optionalAnthropicType();
if(anthropic != null)
{
config.withProviderType(anthropic);
}
instance = AssistantTestFixture.build(config);
Integer providerId;
Integer modelId;
QContext.init(instance, AssistantTestFixture.sessionFor(ADMIN));
try
{
AssistantStore store = config.getRuntime().getStore();
AssistantProvider local = store.insert(new AssistantProvider().withName("Local GPU (Ollama)")
.withProviderType("openai-compatible").withKind("OLLAMA").withLocation("LOCAL").withHipaaEligible(true)
.withBaseUrl(baseUrl).withModel(MODEL).withEnabled(true).withIsDefault(true));
providerId = local.getId();
store.insert(new AssistantProvider().withName("OpenRouter").withProviderType("openai-compatible")
.withKind("OPENROUTER").withLocation("CLOUD").withEnabled(false).withIsDefault(false));
if(anthropic != null)
{
store.insert(new AssistantProvider().withName("Claude (production)").withProviderType(anthropic.getType())
.withModel("claude-sonnet-5-5").withEnabled(false).withIsDefault(false));
}
runProcess(DiscoverModelsProcess.NAME, providerId);
AssistantModel model = store.listModels().stream().filter(m -> providerId.equals(m.getProviderId()) && MODEL.equals(m.getModelId()))
.findFirst().orElseThrow(() -> new QException("Demo model discovery failed."));
modelId = model.getId();
store.update(model.withInputUsdPerMTok(new BigDecimal("0.05")).withOutputUsdPerMTok(new BigDecimal("0.10")));
runProcess(TestConnectionProcess.NAME, providerId);
runProcess(TestModelConnectionProcess.NAME, modelId);
if(!"OK".equals(store.getProvider(providerId).getLastTestStatus())
|| store.listModels().stream().noneMatch(m -> modelId.equals(m.getId()) && "OK".equals(m.getLastTestStatus())))
{
throw (new QException("Demo connection tests failed."));
}
}
finally
{
QContext.clear();
}
TurnRunner.Outcome outcome = new TurnRunner(instance, config).run(ADMIN, "Say hello from the local demo provider.");
if(outcome.errorCode() != null || outcome.text().isBlank())
{
throw (new QException("Demo assistant turn failed: " + outcome.errorCode()));
}
instance.withInstanceDefaultAuthentication(new QAuthenticationMetaData().withName("mock").withType(QAuthenticationType.MOCK)
.withCustomizer(new QCodeReference(DemoSessionCustomizer.class)));
instance.setHomeAppName(AssistantMetaDataBuilder.APP_NAME);
new QApplicationJavalinServer(new OpenAiDemoServer()).withPort(port).withServeFrontendNext(true).start();
System.out.println("Assistant home: http://localhost:" + port + "/app/" + AssistantMetaDataBuilder.APP_NAME);
System.out.println("Local provider: http://localhost:" + port + "/app/assistant_provider/" + providerId);
System.out.println("Local model: http://localhost:" + port + "/app/assistant_model/" + modelId);
}
/*******************************************************************************
** Reuse the Anthropic type only when that optional module is on the classpath.
*******************************************************************************/
private static AssistantProviderType optionalAnthropicType() throws ReflectiveOperationException
{
try
{
return ((AssistantProviderType) Class.forName("com.kingsrook.qbits.assistant.provider.anthropic.AnthropicProviderType")
.getDeclaredConstructor().newInstance());
}
catch(ClassNotFoundException e)
{
return (null);
}
}
/*******************************************************************************
** Execute the same streamed ETL process as the record action, skipping UI.
*******************************************************************************/
private static void runProcess(String process, Integer id) throws QException
{
RunProcessInput input = new RunProcessInput();
input.setProcessName(process);
input.setCallback(QProcessCallbackFactory.forPrimaryKey("id", id));
input.setFrontendStepBehavior(RunProcessInput.FrontendStepBehavior.SKIP);
new RunProcessAction().execute(input);
}
/*******************************************************************************
** Repeatable localhost endpoints for startup traffic and screenshot actions.
** Token counts are mock API reports, never measurements of GPU inference.
*******************************************************************************/
private static void serveMock(HttpExchange exchange) throws IOException
{
try(exchange)
{
exchange.getRequestBody().readAllBytes();
String path = exchange.getRequestURI().getPath();
String body;
if("GET".equals(exchange.getRequestMethod()) && "/v1/models".equals(path))
{
exchange.getResponseHeaders().set("Content-Type", "application/json");
body = """
{"object":"list","data":[{"id":"qwen3:30b-a3b","object":"model","owned_by":"local"},
{"id":"gpt-oss:20b","object":"model","owned_by":"local"},
{"id":"qwen3-coder:30b","object":"model","owned_by":"local"},
{"id":"bge-m3","object":"model","owned_by":"local"}]}
""";
}
else if("POST".equals(exchange.getRequestMethod()) && "/v1/chat/completions".equals(path))
{
exchange.getResponseHeaders().set("Content-Type", "text/event-stream");
body = """
data: {"id":"demo-completion","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant","content":"Hello from the local demo provider."}}]}
data: {"choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: {"choices":[],"usage":{"prompt_tokens":24,"completion_tokens":8,"total_tokens":32}}
data: [DONE]
""";
}
else
{
exchange.sendResponseHeaders(404, -1);
return;
}
exchange.sendResponseHeaders(200, 0);
exchange.getResponseBody().write(body.getBytes(StandardCharsets.UTF_8));
exchange.getResponseBody().flush();
}
}
}
@@ -0,0 +1,53 @@
# OpenAI-compatible provider registry demo
Run from the repository root in PowerShell with Java 21. This is a manual,
memory-backed screenshot server, not a Surefire test. Nothing is persisted
between runs. No API keys or external model services are needed: the secret
resolver deliberately returns no keys, including when the shell has credentials.
Build the test classpath (the local reactor class directories come first below,
so the demo uses this checkout's code and fixture):
```powershell
.\mvnw.cmd -q -pl qbit-assistant-provider-openai -am test-compile dependency:build-classpath '-Dmdep.includeScope=test' '-Dmdep.outputFile=target/demo-classpath.txt'
if ($LASTEXITCODE -ne 0) { throw 'Demo compilation/classpath generation failed' }
$demoDependencies = (Get-Content qbit-assistant-provider-openai/target/demo-classpath.txt -Raw).Trim()
$demoFrontend = "$env:USERPROFILE/.m2/repository/com/kingsrook/qqq/qqq-frontend-next/1.0.0-RC.18/qqq-frontend-next-1.0.0-RC.18.jar"
$demoClasspath = "qbit-assistant-provider-openai/target/test-classes;qbit-assistant-provider-openai/target/classes;qbit-assistant/target/test-classes;qbit-assistant/target/classes;$demoFrontend;$demoDependencies"
java -cp $demoClasspath com.kingsrook.qbits.assistant.provider.openai.demo.OpenAiDemoServer 8099
```
The frontend artifact is `com.kingsrook.qqq:qqq-frontend-next:1.0.0-RC.18`,
already in the local Maven repository. Adjust `$demoFrontend` if your Maven
local repository is elsewhere. Omit the final port argument for the default
8099, or use another free port (the smoke check used 18099). Stop with Ctrl+C.
Startup prints these URLs (with the selected port):
- Assistant home: `http://localhost:8099/app/assistantAdmin`
- Local provider: `http://localhost:8099/app/assistant_provider/1`
- Priced and tested model: `http://localhost:8099/app/assistant_model/1`
Every browser session is `admin-demo`. Startup inserts an enabled, default,
keyless Local GPU (Ollama) provider, marked LOCAL and HIPAA eligible for the
synthetic demo, and a disabled CLOUD OpenRouter provider whose key is Not set.
If the Anthropic module is explicitly added to the classpath, its provider type
and a disabled Claude (production) row are also included; otherwise it is skipped.
The normal Discover models process fetches four model IDs from the localhost
mock: `qwen3:30b-a3b`, `gpt-oss:20b`, `qwen3-coder:30b`, and `bge-m3`.
A normal update sets qwen3's input/output compute prices to $0.05/$0.10 per
million tokens. The provider and model Test connection processes then each
make a real HTTP completion call, recording two TEST usage rows and the
last-test fields. One assistant turn records a third usage row with purpose
TURN. Discovery timestamps/status also come from the actual process.
Replies and token counts are synthetic mock API responses (24 input / 8 output
tokens per completion), not live GPU measurements or billed cloud usage.
No usage/history rows are manually seeded. The mock remains available for
repeating discovery, connection tests and chat while taking screenshots.
Capture the P-a screens in light and dark at 390 and 1440 px; see
[`docs/screenshots/README.md`](../../../../../../../../../../../docs/screenshots/README.md)
for the planned screenshot prefixes.
@@ -0,0 +1,23 @@
# Test-only TLS fixtures
These certificates and the private key in `server.p12` are synthetic test material.
They must never be used for deployment or trusted outside tests. The PKCS12
password is `test-only`. The CA private key is not retained in the repository.
BouncyCastle was not on the test classpath, so fixtures were generated with the
JDK 21 **keytool executable**, not internal `sun.security` APIs.
The leaf is signed by `QBit TEST ONLY Root CA` and has only `DNS:localhost` in its
SAN; connecting to `127.0.0.1` deliberately fails hostname verification. Both
certificates start on 2020-01-01 and last 36,500 days to avoid date-dependent tests.
Each regeneration creates fresh random keys. Run in a temporary directory, then
copy only `ca.pem` and `server.p12` here:
```sh
keytool -genkeypair -alias ca -keyalg RSA -keysize 2048 -dname 'CN=QBit TEST ONLY Root CA' -ext bc:c=ca:true -ext ku:c=keyCertSign,cRLSign -startdate '2020/01/01 00:00:00' -validity 36500 -keystore ca.p12 -storepass test-only -storetype PKCS12 -noprompt
keytool -exportcert -rfc -alias ca -keystore ca.p12 -storepass test-only -file ca.pem
keytool -genkeypair -alias server -keyalg RSA -keysize 2048 -dname 'CN=localhost' -startdate '2020/01/01 00:00:00' -validity 36500 -keystore server.p12 -storetype PKCS12 -storepass test-only -noprompt
keytool -certreq -alias server -keystore server.p12 -storepass test-only -file server.csr
keytool -gencert -alias ca -keystore ca.p12 -storepass test-only -infile server.csr -outfile server.pem -rfc -startdate '2020/01/01 00:00:00' -validity 36500 -ext san=dns:localhost -ext eku=serverAuth -ext ku=digitalSignature,keyEncipherment
keytool -importcert -alias ca -file ca.pem -keystore server.p12 -storepass test-only -noprompt
keytool -importcert -alias server -file server.pem -keystore server.p12 -storepass test-only -noprompt
```
@@ -0,0 +1,19 @@
-----BEGIN CERTIFICATE-----
MIIDCDCCAfCgAwIBAgIII8Xlo9Lidn4wDQYJKoZIhvcNAQEMBQAwITEfMB0GA1UE
AxMWUUJpdCBURVNUIE9OTFkgUm9vdCBDQTAgFw0yMDAxMDEwNjAwMDBaGA8yMTE5
MTIwODA2MDAwMFowITEfMB0GA1UEAxMWUUJpdCBURVNUIE9OTFkgUm9vdCBDQTCC
ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMyuwvRX9gZCJoJW68JPf8HC
9rA/WAqSwIgwOi6JUVGfRFkp3DZGvfvfw/SiYkED+Zf0AeVrfOZbNZO4dXoe9Rxq
YBu4nr79jCfKnlmGsIr/YmWxePUzixvxJ7suHb4y1uah7C36h1EeQKAEYbpFhI7o
WVQkZonWDr4w0PmcstjS4lIT43lZdbAxX7RRhiBKe9Ph9P+sqbn1Uk4GOPo5YQkg
5eZSPAU4gVh4K07e4aVej+kRHMqMmP4joY4KHBrZXOHKN/mXDGkipys0bf+z7uti
/qowu6lNEdF/IihvCgU/fe5wVFbdh7E5t+mpRo+A9B0YrQAtqkJ6gw+Nrwr2TvEC
AwEAAaNCMEAwHQYDVR0OBBYEFMul052SYQj4mA4NgW+mX5mfSzgyMA4GA1UdDwEB
/wQEAwIBBjAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBDAUAA4IBAQCAKxnC
2iBt6cxyh3T/Z11fdBEEFASYU2C/nGqt1FnYhyfR9eluDerNJqsnHmP9pjmf5KAO
bGYiw9WodHEbLyM9etNPJSCJRouCHFmRyWarfQV95NjkMfVMXdA2zNWmWv0vxZ2y
dOSQUP3nCegLW1JqBoDsdJCJvzU8lcuUwVd9BNywcXmrQbL0iHETwUUWtIEACv2A
ja9ZNHYs1oZbaA9YotK3vCcCt0B9xHxMDza9Er7ljTWE6zexTh7IWJ/MbABUdQbQ
SGDVrVFlk2PoDIO5rwRwS8LwaKkdwuPcAHbhBRN3HVyr06SBD7AiVWhUV7g5R+h3
qHqj8SDW3k3QL63f
-----END CERTIFICATE-----
@@ -17,6 +17,9 @@
package com.kingsrook.qbits.assistant;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.LinkedHashSet;
@@ -25,10 +28,12 @@ import java.util.Map;
import java.util.Set;
import java.util.function.Function;
import java.util.regex.Pattern;
import javax.net.ssl.SSLContext;
import com.fasterxml.jackson.annotation.JsonIgnore;
import com.kingsrook.qbits.assistant.engine.ModelPricing;
import com.kingsrook.qbits.assistant.provider.AssistantModelProvider;
import com.kingsrook.qbits.assistant.provider.AssistantProviderType;
import com.kingsrook.qbits.assistant.provider.TrustedCaBundle;
import com.kingsrook.qbits.assistant.uiapi.UiApiGateway;
import com.kingsrook.qqq.backend.core.actions.ai.AiExposure;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
@@ -69,6 +74,9 @@ public class AssistantQBitConfig implements QBitConfig
@Deprecated(since = "0.1.0-RC.3")
public static final String LEGACY_DEFAULT_OWNER_KEY_TYPE = "assistantUserId";
@JsonIgnore
private final transient Map<String, SSLContext> trustedCaBundles = new LinkedHashMap<>();
private Boolean enabled = true;
private String backendName;
private String providerName = DEFAULT_PROVIDER_NAME;
@@ -1399,7 +1407,14 @@ public class AssistantQBitConfig implements QBitConfig
public Set<String> getEffectiveAllowedApiKeyEnvNames()
{
Set<String> names = new LinkedHashSet<>(allowedApiKeyEnvNames == null ? Set.of() : allowedApiKeyEnvNames);
providerTypes.values().forEach(t -> names.add(t.getDefaultApiKeyEnvName()));
providerTypes.values().forEach(t -> t.getKinds().forEach(k ->
{
String name = t.getDefaultApiKeyEnvName(k);
if(name != null)
{
names.add(name);
}
}));
return (names);
}
@@ -1580,4 +1595,45 @@ public class AssistantQBitConfig implements QBitConfig
return (this);
}
/*******************************************************************************
** Register server-only trust material, parsed eagerly without retaining PEM.
*******************************************************************************/
public AssistantQBitConfig withTrustedCaBundle(String name, String pem)
{
if(name == null || name.isBlank() || name.length() > 100 || !name.equals(name.trim()))
{
throw (new IllegalArgumentException("CA bundle name must be 1 to 100 characters without surrounding whitespace."));
}
trustedCaBundles.put(name, TrustedCaBundle.parse(pem));
return (this);
}
/*******************************************************************************
** Read the bundle at startup; no file content or path enters diagnostics.
*******************************************************************************/
public AssistantQBitConfig withTrustedCaBundleFile(String name, Path pemFile)
{
try
{
return (withTrustedCaBundle(name, Files.readString(pemFile)));
}
catch(IOException | SecurityException e)
{
throw (new IllegalArgumentException("Could not read the configured CA bundle file."));
}
}
/*******************************************************************************
** Server-only contexts; never expose trust material in metadata or JSON.
*******************************************************************************/
@JsonIgnore
public Map<String, SSLContext> getTrustedCaBundles()
{
return (Map.copyOf(trustedCaBundles));
}
}
@@ -25,7 +25,9 @@ import com.kingsrook.qbits.assistant.home.AssistantAccessWidget;
import com.kingsrook.qbits.assistant.home.AssistantHome;
import com.kingsrook.qbits.assistant.metadata.AssistantMetaDataBuilder;
import com.kingsrook.qbits.assistant.metadata.AssistantRecordScreens;
import com.kingsrook.qbits.assistant.processes.DiscoverModelsProcess;
import com.kingsrook.qbits.assistant.processes.TestConnectionProcess;
import com.kingsrook.qbits.assistant.processes.TestModelConnectionProcess;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerMultiOutput;
@@ -165,6 +167,11 @@ public class AssistantQBitProducer implements QBitMetaDataProducer<AssistantQBit
QProcessMetaData testConnection = TestConnectionProcess.process();
testConnection.setSourceQBitName(qBitMetaData.getName());
qInstance.addProcess(testConnection);
for(QProcessMetaData process : java.util.List.of(DiscoverModelsProcess.process(), TestModelConnectionProcess.process()))
{
process.setSourceQBitName(qBitMetaData.getName());
qInstance.addProcess(process);
}
qInstance.addWidget(AssistantHome.widget());
qInstance.addWidget(AssistantAccessWidget.widget());
@@ -21,11 +21,15 @@ import java.time.Duration;
import java.util.ArrayList;
import java.util.List;
import com.kingsrook.qbits.assistant.AssistantRuntime;
import com.kingsrook.qbits.assistant.model.AssistantModel;
import com.kingsrook.qbits.assistant.model.AssistantProvider;
import com.kingsrook.qbits.assistant.provider.AssistantProviderType;
import com.kingsrook.qbits.assistant.provider.HealthResult;
import com.kingsrook.qbits.assistant.provider.ModelMessage;
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
import com.kingsrook.qbits.assistant.provider.ModelRequest;
import com.kingsrook.qbits.assistant.provider.ModelResponse;
import com.kingsrook.qbits.assistant.provider.RegistryModelProvider;
import com.kingsrook.qbits.assistant.provider.StreamHandle;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
@@ -108,6 +112,16 @@ public final class ConnectionTest
return (new Result(STATUS_ERROR, null, null, e.getMessage()));
}
return (call(runtime, active, userId));
}
/*******************************************************************************
** Probe health, then make and account for a minimal completion.
*******************************************************************************/
private static Result call(AssistantRuntime runtime, ProviderService.Active active, String userId)
{
List<ModelMessage> messages = new ArrayList<>();
messages.add(ModelMessage.user(PING_PROMPT));
ModelRequest request = new ModelRequest()
@@ -117,10 +131,16 @@ public final class ConnectionTest
.withMessages(messages);
UsageRecorder.Call usageCall = new UsageRecorder.Call(userId, null, null, active, UsageRecorder.PURPOSE_TEST);
ModelPricing pricing = runtime.getProviderService().pricing(active.providerType(), active.model());
ModelPricing pricing = runtime.getProviderService().pricing(active, active.model());
long started = System.nanoTime();
try
{
AssistantProviderType type = runtime.getConfig().getProviderTypes().get(active.providerType());
HealthResult health = type.checkHealth(runtime.getProviderService().settings(active.row()));
if(Boolean.FALSE.equals(health.healthy()))
{
throw (new ModelProviderException(ModelProviderException.Kind.NETWORK, null, health.message()));
}
StreamHandle handle = active.provider().stream(request, text ->
{
// the reply text is not used
@@ -140,6 +160,10 @@ public final class ConnectionTest
UsageRecorder.save(runtime.getStore(), UsageRecorder.row(usageCall, null, e, latency, pricing));
return (new Result(STATUS_ERROR, (int) latency, null, describe(e)));
}
catch(ProviderService.ProviderSetupException e)
{
return (new Result(STATUS_ERROR, null, null, e.getMessage()));
}
catch(InterruptedException e)
{
Thread.currentThread().interrupt();
@@ -155,17 +179,7 @@ public final class ConnectionTest
*******************************************************************************/
static String describe(ModelProviderException e)
{
String hint = switch(e.getKind())
{
case AUTHENTICATION -> "The API key was refused. Check the key in the secret store.";
case RATE_LIMITED -> "Rate limited by the provider. Try again shortly.";
case OVERLOADED -> "The provider is overloaded. Try again shortly.";
case INVALID_REQUEST -> "The provider refused the request. Check the model id.";
case TIMEOUT -> "No answer in time.";
case NETWORK -> "Could not reach the provider. Check the base URL and the network.";
case SERVER -> "The provider had a server error.";
default -> "The call failed.";
};
String hint = hint(e.getKind());
return (hint + " (" + e.getKind() + (e.getHttpStatus() == null ? "" : ", HTTP " + e.getHttpStatus()) + (e.getMessage() == null ? "" : ": " + e.getMessage()) + ")");
}
@@ -179,4 +193,56 @@ public final class ConnectionTest
return (value == null || value.length() <= max ? value : value.substring(0, max));
}
/*******************************************************************************
** Test even a disabled registry model before an admin enables it.
*******************************************************************************/
public static Result runModel(AssistantRuntime runtime, AssistantModel model, String userId) throws QException
{
AssistantProvider row = runtime.getStore().getProvider(model.getProviderId());
Result result;
try
{
if(row == null)
{
throw (new ProviderService.ProviderSetupException("The model's provider no longer exists."));
}
AssistantProviderType type = runtime.getConfig().getProviderTypes().get(row.getProviderType());
var settings = runtime.getProviderService().settings(row);
var provider = new RegistryModelProvider(type.create(settings), model);
result = call(runtime, new ProviderService.Active(row, row.getProviderType(), model.getModelId(), provider), userId);
}
catch(ProviderService.ProviderSetupException e)
{
result = new Result(STATUS_ERROR, null, model.getModelId(), e.getMessage());
}
runtime.getStore().updateFields(AssistantModel.TABLE_NAME, new QRecord().withValue("id", model.getId())
.withValue("lastTestAt", runtime.getClock().instant()).withValue("lastTestStatus", result.status())
.withValue("lastTestLatencyMs", result.latencyMs()).withValue("lastTestError", truncate(result.error(), 500)));
return (result);
}
/*******************************************************************************
** Map the provider error category.
*******************************************************************************/
private static String hint(ModelProviderException.Kind kind)
{
return (switch(kind)
{
case AUTHENTICATION -> "The API key was refused. Check the key in the secret store.";
case RATE_LIMITED -> "Rate limited by the provider. Try again shortly.";
case OVERLOADED -> "The provider is overloaded. Try again shortly.";
case INVALID_REQUEST -> "The provider refused the request. Check the model id.";
case TIMEOUT -> "No answer in time.";
case NETWORK -> "Could not reach the provider. Check the base URL and the network.";
case SERVER -> "The provider had a server error.";
default -> "The call failed.";
});
}
}
@@ -0,0 +1,118 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant.engine;
import java.math.BigDecimal;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import com.kingsrook.qbits.assistant.AssistantRuntime;
import com.kingsrook.qbits.assistant.model.AssistantModel;
import com.kingsrook.qbits.assistant.model.AssistantProvider;
import com.kingsrook.qbits.assistant.provider.AssistantProviderType;
import com.kingsrook.qbits.assistant.provider.DiscoveredModel;
import com.kingsrook.qbits.assistant.provider.ModelCapabilities;
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
import com.kingsrook.qbits.assistant.provider.ProviderSettings;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** Additive discovery: never overwrite an administrator's model configuration.
*******************************************************************************/
public final class ModelDiscovery
{
/*******************************************************************************
** Summary for the standard process result screen.
*******************************************************************************/
public record Result(boolean ok, String message)
{
}
/*******************************************************************************
** Utility class.
*******************************************************************************/
private ModelDiscovery()
{
}
/*******************************************************************************
** Serialize local discovery runs; the table unique key also guards replicas.
*******************************************************************************/
public static Result run(AssistantRuntime runtime, AssistantProvider row) throws QException
{
ProviderService service = runtime.getProviderService();
synchronized(service)
{
Result result;
try
{
ProviderSettings settings = service.settings(row);
AssistantProviderType type = runtime.getConfig().getProviderTypes().get(row.getProviderType());
List<DiscoveredModel> discovered = type.discoverModels(settings);
Set<String> existing = new HashSet<>();
runtime.getStore().listModels().stream().filter(m -> row.getId().equals(m.getProviderId())).forEach(m -> existing.add(m.getModelId()));
boolean local = "LOCAL".equals(service.location(row));
var provider = type.create(settings);
int added = 0;
int present = 0;
for(DiscoveredModel model : discovered)
{
if(existing.contains(model.modelId()))
{
present++;
continue;
}
ModelCapabilities caps = provider.getCapabilities(model.modelId());
runtime.getStore().insert(new AssistantModel().withProviderId(row.getId()).withModelId(model.modelId())
.withDisplayName(model.displayName()).withEnabled(local || discovered.size() <= 20)
.withSource("DISCOVERED").withDiscoveredAt(runtime.getClock().instant())
.withSupportsTools(caps.getTools()).withSupportsVision(caps.getVision()).withSupportsStreaming(caps.getStreaming())
.withSupportsReasoningEffort(caps.getReasoningEffort())
.withContextWindowTokens(model.contextWindowTokens() == null ? caps.getMaxContextTokens() : model.contextWindowTokens())
.withInputUsdPerMTok(local ? BigDecimal.ZERO : null).withOutputUsdPerMTok(local ? BigDecimal.ZERO : null));
existing.add(model.modelId());
added++;
}
result = new Result(true, added + " models added, " + present + " already present");
}
catch(ProviderService.ProviderSetupException e)
{
result = new Result(false, e.getMessage());
}
catch(ModelProviderException e)
{
result = new Result(false, ConnectionTest.describe(e));
}
catch(QException | RuntimeException e)
{
result = new Result(false, "Could not save discovered models. Check the model registry and retry.");
}
runtime.getStore().updateFields(AssistantProvider.TABLE_NAME, new QRecord().withValue("id", row.getId())
.withValue("lastDiscoveryAt", runtime.getClock().instant()).withValue("lastDiscoveryStatus", result.ok() ? "OK" : "ERROR")
.withValue("lastDiscoveryError", result.ok() ? null : result.message().substring(0, Math.min(500, result.message().length()))));
service.invalidate();
return (result);
}
}
}
@@ -32,11 +32,14 @@ import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.Semaphore;
import com.kingsrook.qbits.assistant.AssistantQBitConfig;
import com.kingsrook.qbits.assistant.model.AssistantModel;
import com.kingsrook.qbits.assistant.model.AssistantModelPrice;
import com.kingsrook.qbits.assistant.model.AssistantProvider;
import com.kingsrook.qbits.assistant.persistence.AssistantStore;
import com.kingsrook.qbits.assistant.provider.AssistantModelProvider;
import com.kingsrook.qbits.assistant.provider.AssistantProviderType;
import com.kingsrook.qbits.assistant.provider.ProviderSettings;
import com.kingsrook.qbits.assistant.provider.RegistryModelProvider;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.utils.StringUtils;
@@ -75,6 +78,7 @@ public class ProviderService
private volatile List<AssistantProvider> cachedRows;
private volatile Map<String, ModelPricing> cachedPrices;
private volatile Instant cachedAt;
private volatile List<AssistantModel> cachedModels;
@@ -179,24 +183,41 @@ public class ProviderService
*******************************************************************************/
public Active forRow(AssistantProvider row, String fallbackModel) throws ProviderSetupException
{
List<String> errors = validate(row);
if(!errors.isEmpty())
{
throw (new ProviderSetupException(String.join(" ", errors)));
}
ProviderSettings settings = settings(row);
AssistantProviderType type = config.getProviderTypes().get(row.getProviderType());
String key = resolveKey(row);
if(!StringUtils.hasContent(key))
List<AssistantModel> models;
try
{
throw (new ProviderSetupException("No API key: the environment variable " + keyEnvName(row) + " is not set on this server."));
models = models(row.getId());
}
String model = StringUtils.hasContent(row.getModel()) ? row.getModel() : (StringUtils.hasContent(fallbackModel) ? fallbackModel : first(type.getModels()));
String baseUrl = StringUtils.hasContent(row.getBaseUrl()) ? row.getBaseUrl().trim() : null;
String cacheKey = row.getId() + "|" + row.getProviderType() + "|" + baseUrl + "|" + fingerprint(key);
AssistantModelProvider provider = built.computeIfAbsent(cacheKey, k -> type.create(key, baseUrl));
return (new Active(row, row.getProviderType(), model, provider));
catch(QException e)
{
throw (new ProviderSetupException("Could not read the provider's model registry."));
}
String model = StringUtils.hasContent(row.getModel()) ? row.getModel() : null;
if(model == null && !models.isEmpty())
{
model = models.stream().filter(m -> Boolean.TRUE.equals(m.getEnabled())).map(AssistantModel::getModelId).findFirst().orElse(null);
if(model == null)
{
throw (new ProviderSetupException("All models of this provider are disabled. Enable a model first."));
}
}
if(model == null)
{
model = StringUtils.hasContent(fallbackModel) ? fallbackModel : first(type.getModels());
}
String selected = model;
AssistantModel registry = models.stream().filter(m -> java.util.Objects.equals(selected, m.getModelId())).findFirst().orElse(null);
if(registry != null && !Boolean.TRUE.equals(registry.getEnabled()))
{
throw (new ProviderSetupException("The selected model is disabled in this provider's model registry."));
}
String cacheKey = row.getId() + "|" + row.getProviderType() + "|" + settings.getKind() + "|" + settings.getBaseUrl()
+ "|" + settings.getApiVersion() + "|" + settings.getRegion() + "|" + settings.getConnectTimeoutSeconds()
+ "|" + settings.getTlsCaBundle() + "|" + settings.getRequestTimeoutSeconds() + "|" + fingerprint(settings.getApiKey());
AssistantModelProvider provider = built.computeIfAbsent(cacheKey, k -> type.create(settings));
return (new Active(row, row.getProviderType(), model, registry == null ? provider : new RegistryModelProvider(provider, registry)));
}
@@ -216,11 +237,11 @@ public class ProviderService
}
String envName = keyEnvName(row);
if(!AssistantQBitConfig.ENV_NAME.matcher(envName).matches())
if(envName != null && !AssistantQBitConfig.ENV_NAME.matcher(envName).matches())
{
errors.add("API key variable must be the NAME of an environment variable (capital letters, digits and _), not the key itself.");
}
else if(!config.getEffectiveAllowedApiKeyEnvNames().contains(envName))
else if(envName != null && !config.getEffectiveAllowedApiKeyEnvNames().contains(envName))
{
errors.add("API key variable " + envName + " is not allowed on this server. Allowed: " + String.join(", ", config.getEffectiveAllowedApiKeyEnvNames()) + ".");
}
@@ -228,12 +249,44 @@ public class ProviderService
if(StringUtils.hasContent(row.getBaseUrl()))
{
String url = row.getBaseUrl().trim();
if(!url.equals(type.getDefaultBaseUrl()) && (config.getAllowedBaseUrls() == null || !config.getAllowedBaseUrls().contains(url)))
if(!url.equals(type.getDefaultBaseUrl(kind(row))) && (config.getAllowedBaseUrls() == null || !config.getAllowedBaseUrls().contains(url)))
{
errors.add("Base URL " + url + " is not allowed on this server; leave it empty to use " + type.getDefaultBaseUrl() + ".");
errors.add("Base URL " + url + " is not allowed on this server; leave it empty to use " + type.getDefaultBaseUrl(kind(row)) + ".");
}
}
if(StringUtils.hasContent(row.getTlsCaBundle()))
{
if(!config.getTrustedCaBundles().containsKey(row.getTlsCaBundle()))
{
errors.add("TLS CA bundle must name a bundle configured on this server (withTrustedCaBundle).");
}
String url = StringUtils.hasContent(row.getBaseUrl()) ? row.getBaseUrl().trim() : type.getDefaultBaseUrl(kind(row));
if(url == null || !url.regionMatches(true, 0, "https://", 0, 8))
{
errors.add("TLS CA bundle requires an HTTPS base URL.");
}
}
if(!type.getKinds().contains(kind(row)))
{
errors.add("The provider kind is not supported by the installed adapter.");
}
if(!List.of("LOCAL", "CLOUD").contains(location(row)))
{
errors.add("Location must be LOCAL or CLOUD.");
}
if(!StringUtils.hasContent(row.getBaseUrl()) && type.getDefaultBaseUrl(kind(row)) == null)
{
errors.add("An allow-listed base URL is required for this provider kind.");
}
for(Integer timeout : java.util.Arrays.asList(row.getConnectTimeoutSeconds(), row.getRequestTimeoutSeconds()))
{
if(timeout != null && (timeout < 1 || timeout > 600))
{
errors.add("Connection and request timeouts must be between 1 and 600 seconds.");
}
}
if(row.getMonthlyCostCapUsd() != null && row.getMonthlyCostCapUsd().signum() < 0)
{
errors.add("Monthly cost cap must not be negative.");
@@ -258,6 +311,10 @@ public class ProviderService
public String keyStatus(AssistantProvider row)
{
String envName = keyEnvName(row);
if(envName == null)
{
return ("Not needed (local, no key)");
}
if(!AssistantQBitConfig.ENV_NAME.matcher(envName).matches() || !config.getEffectiveAllowedApiKeyEnvNames().contains(envName))
{
return ("Not usable: the reference is not an allowed environment variable name");
@@ -298,7 +355,7 @@ public class ProviderService
return (row.getApiKeyEnvName().trim());
}
AssistantProviderType type = row.getProviderType() == null ? null : config.getProviderTypes().get(row.getProviderType());
return (type == null ? "" : type.getDefaultApiKeyEnvName());
return (type == null ? "" : type.getDefaultApiKeyEnvName(kind(row)));
}
@@ -309,6 +366,10 @@ public class ProviderService
private String resolveKey(AssistantProvider row)
{
String envName = keyEnvName(row);
if(envName == null)
{
return (null);
}
if(!AssistantQBitConfig.ENV_NAME.matcher(envName).matches() || !config.getEffectiveAllowedApiKeyEnvNames().contains(envName))
{
return (null);
@@ -447,11 +508,12 @@ public class ProviderService
/*******************************************************************************
** Drop the cached rows and prices (after a settings change).
*******************************************************************************/
public void invalidate()
public synchronized void invalidate()
{
cachedAt = null;
cachedRows = null;
cachedPrices = null;
cachedModels = null;
}
@@ -459,7 +521,7 @@ public class ProviderService
/*******************************************************************************
** The provider rows (cached for policyCacheSeconds).
*******************************************************************************/
public List<AssistantProvider> rows() throws QException
public synchronized List<AssistantProvider> rows() throws QException
{
refresh();
return (cachedRows);
@@ -470,7 +532,7 @@ public class ProviderService
/*******************************************************************************
**
*******************************************************************************/
private Map<String, ModelPricing> prices() throws QException
private synchronized Map<String, ModelPricing> prices() throws QException
{
refresh();
return (cachedPrices);
@@ -497,6 +559,7 @@ public class ProviderService
prices.put(priceKey(price.getProviderType(), price.getModel()), toPricing(price));
}
}
cachedModels = store.listModels();
cachedRows = rows;
cachedPrices = prices;
cachedAt = now;
@@ -521,7 +584,7 @@ public class ProviderService
{
try
{
return (HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(key.getBytes(StandardCharsets.UTF_8))));
return (HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest((key == null ? "" : key).getBytes(StandardCharsets.UTF_8))));
}
catch(Exception e)
{
@@ -549,4 +612,98 @@ public class ProviderService
return (value == null ? 0 : value.doubleValue());
}
/*******************************************************************************
** Effective kind, including legacy rows.
*******************************************************************************/
public String kind(AssistantProvider row)
{
return (StringUtils.hasContent(row.getKind()) ? row.getKind() : (row.getProviderType() == null ? "" : row.getProviderType().toUpperCase(java.util.Locale.ROOT).replace('-', '_')));
}
/*******************************************************************************
** Effective location, including legacy rows.
*******************************************************************************/
public String location(AssistantProvider row)
{
AssistantProviderType type = config.getProviderTypes().get(row.getProviderType());
return (StringUtils.hasContent(row.getLocation()) ? row.getLocation() : (type == null ? "CLOUD" : type.getDefaultLocation(kind(row))));
}
/*******************************************************************************
** Validate settings and resolve only an allowed key reference.
*******************************************************************************/
public ProviderSettings settings(AssistantProvider row) throws ProviderSetupException
{
List<String> errors = validate(row);
if(!errors.isEmpty())
{
throw (new ProviderSetupException(String.join(" ", errors)));
}
String key = resolveKey(row);
if(keyEnvName(row) != null && !StringUtils.hasContent(key))
{
throw (new ProviderSetupException("No API key: the environment variable " + keyEnvName(row) + " is not set on this server."));
}
return (new ProviderSettings().withKind(kind(row)).withApiKey(key)
.withBaseUrl(StringUtils.hasContent(row.getBaseUrl()) ? row.getBaseUrl().trim() : null)
.withApiVersion(row.getApiVersion()).withRegion(row.getRegion())
.withTlsCaBundle(row.getTlsCaBundle()).withSslContext(StringUtils.hasContent(row.getTlsCaBundle()) ? config.getTrustedCaBundles().get(row.getTlsCaBundle()) : null)
.withConnectTimeoutSeconds(row.getConnectTimeoutSeconds() == null ? 10 : row.getConnectTimeoutSeconds())
.withRequestTimeoutSeconds(row.getRequestTimeoutSeconds() == null ? 120 : row.getRequestTimeoutSeconds()));
}
/*******************************************************************************
** Cached models belonging to one provider.
*******************************************************************************/
public synchronized List<AssistantModel> models(Integer providerId) throws QException
{
refresh();
return (cachedModels.stream().filter(m -> java.util.Objects.equals(providerId, m.getProviderId())).toList());
}
/*******************************************************************************
** Registry price, type price, then config price; unpriced local models cost zero.
*******************************************************************************/
public ModelPricing pricing(Active active, String model)
{
if(active != null && active.row() != null)
{
try
{
for(AssistantModel row : models(active.providerId()))
{
if(java.util.Objects.equals(model, row.getModelId()) && (row.getInputUsdPerMTok() != null || row.getOutputUsdPerMTok() != null))
{
return (toPricing(new AssistantModelPrice().withInputUsdPerMTok(row.getInputUsdPerMTok()).withOutputUsdPerMTok(row.getOutputUsdPerMTok())));
}
}
ModelPricing price = prices().get(priceKey(active.providerType(), model));
if(price != null)
{
return (price);
}
if("LOCAL".equals(location(active.row())) && (config.getPricing() == null || !config.getPricing().containsKey(model)))
{
return (new ModelPricing().withInputUsdPerMTok(0.0).withOutputUsdPerMTok(0.0));
}
}
catch(QException e)
{
LOG.warn("Could not read model registry prices");
}
}
return (pricing(active == null ? null : active.providerType(), model));
}
}
@@ -1057,7 +1057,7 @@ public class TurnEngine implements Runnable
*******************************************************************************/
private ModelPricing pricing()
{
return (runtime.getProviderService().pricing(active == null ? null : active.providerType(), model));
return (runtime.getProviderService().pricing(active, model));
}
@@ -79,7 +79,7 @@ public class AssistantHomeData
/*******************************************************************************
** One provider's health.
*******************************************************************************/
public record ProviderHealth(AssistantProvider row, String typeLabel, String keyStatus, boolean keyPresent, List<String> problems, boolean active, long monthCostMicros, Long capMicros)
public record ProviderHealth(AssistantProvider row, String typeLabel, String keyStatus, boolean keyPresent, List<String> problems, boolean active, long monthCostMicros, Long capMicros, String kind, String location, long enabledModels)
{
}
@@ -275,8 +275,8 @@ public class AssistantHomeData
{
String status = service.keyStatus(row);
AssistantProviderType type = row.getProviderType() == null ? null : runtime.getConfig().getProviderTypes().get(row.getProviderType());
data.providers.add(new ProviderHealth(row, type == null ? row.getProviderType() : type.getLabel(), status, status.startsWith("Set"), service.validate(row), active != null && active.getId().equals(row.getId()),
providerMonth.getOrDefault(row.getId(), 0L), ProviderService.capMicros(row)));
data.providers.add(new ProviderHealth(row, type == null ? row.getProviderType() : type.getLabel(), status, (status.startsWith("Set") || status.startsWith("Not needed")), service.validate(row), active != null && active.getId().equals(row.getId()),
providerMonth.getOrDefault(row.getId(), 0L), ProviderService.capMicros(row), service.kind(row), service.location(row), service.models(row.getId()).stream().filter(m -> Boolean.TRUE.equals(m.getEnabled())).count()));
}
return (data);
}
@@ -28,6 +28,7 @@ import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import com.kingsrook.qbits.assistant.metadata.AssistantRecordScreens;
import com.kingsrook.qbits.assistant.model.AssistantConversation;
import com.kingsrook.qbits.assistant.model.AssistantModelPrice;
import com.kingsrook.qbits.assistant.model.AssistantProvider;
@@ -185,6 +186,13 @@ public final class AssistantHomeView
}
html.append("</div><div style=\"").append(CELLS).append("\">");
cell(html, "Type", nvl(p.typeLabel(), "Not set"));
cell(html, "Kind", AssistantRecordScreens.providerKindLabel(p.kind()));
cell(html, "Location", "LOCAL".equals(p.location()) ? "Local" : "Cloud");
if(Boolean.TRUE.equals(row.getHipaaEligible()))
{
cell(html, "Eligibility", "HIPAA-eligible");
}
cell(html, "Enabled models", String.valueOf(p.enabledModels()));
cell(html, "Model", nvl(row.getModel(), "Default model"));
cell(html, "API key", p.keyStatus());
cell(html, "Last test", lastTest(row));
@@ -26,6 +26,7 @@ import com.kingsrook.qbits.assistant.model.AssistantConversation;
import com.kingsrook.qbits.assistant.model.AssistantDataAccess;
import com.kingsrook.qbits.assistant.model.AssistantFeedback;
import com.kingsrook.qbits.assistant.model.AssistantMessage;
import com.kingsrook.qbits.assistant.model.AssistantModel;
import com.kingsrook.qbits.assistant.model.AssistantModelPrice;
import com.kingsrook.qbits.assistant.model.AssistantPendingAction;
import com.kingsrook.qbits.assistant.model.AssistantPolicy;
@@ -179,6 +180,10 @@ public class AssistantMetaDataBuilder
.withUniqueKey(new UniqueKey("name"))
.withRecordLabelFormat("%s").withRecordLabelFields("name"));
tables.add(settings(base(AssistantModel.TABLE_NAME, "Assistant Model", "memory", AssistantModel.class))
.withUniqueKey(new UniqueKey("providerId", "modelId"))
.withRecordLabelFormat("%s").withRecordLabelFields("modelId"));
tables.add(settings(base(AssistantModelPrice.TABLE_NAME, "Assistant Model Price", "sell", AssistantModelPrice.class))
.withUniqueKey(new UniqueKey("providerType", "model"))
.withRecordLabelFormat("%s").withRecordLabelFields("model"));
@@ -30,6 +30,7 @@ import com.kingsrook.qbits.assistant.model.AssistantConversation;
import com.kingsrook.qbits.assistant.model.AssistantDataAccess;
import com.kingsrook.qbits.assistant.model.AssistantFeedback;
import com.kingsrook.qbits.assistant.model.AssistantMessage;
import com.kingsrook.qbits.assistant.model.AssistantModel;
import com.kingsrook.qbits.assistant.model.AssistantModelPrice;
import com.kingsrook.qbits.assistant.model.AssistantPendingAction;
import com.kingsrook.qbits.assistant.model.AssistantPolicy;
@@ -165,6 +166,7 @@ public final class AssistantRecordScreens
case AssistantPolicy.TABLE_NAME -> policy(table);
case AssistantUsageDaily.TABLE_NAME -> usageDaily(table);
case AssistantProvider.TABLE_NAME -> provider(table);
case AssistantModel.TABLE_NAME -> model(table);
case AssistantModelPrice.TABLE_NAME -> price(table);
case AssistantUsage.TABLE_NAME -> usage(table);
case AssistantDataAccess.TABLE_NAME -> dataAccess(table);
@@ -217,6 +219,7 @@ public final class AssistantRecordScreens
QTableMetaData provider = qInstance.getTable(AssistantProvider.TABLE_NAME);
if(provider != null)
{
childList(qInstance, provider, AssistantModel.TABLE_NAME, "Models", "memory", List.of("modelId", "displayName", "enabled", "supportsTools", "lastTestStatus"), "providerId", "provider");
childList(qInstance, provider, AssistantUsage.TABLE_NAME, "Model Calls", "receipt_long", USAGE_LIST_FIELDS, "providerId", "provider");
}
}
@@ -248,6 +251,10 @@ public final class AssistantRecordScreens
////////////////////////////////////////////////////////////////////
types.add(new QPossibleValue<>(config.getProviderName(), config.getProviderName()));
}
List<QPossibleValue<?>> kinds = new ArrayList<>();
Set<String> kindNames = new LinkedHashSet<>(List.of("ANTHROPIC", "OPENAI", "AZURE_OPENAI", "OPENROUTER", "OLLAMA", "LM_STUDIO", "VLLM", "LLAMA_CPP", "OPENAI_COMPATIBLE"));
config.getProviderTypes().values().forEach(t -> kindNames.addAll(t.getKinds()));
kindNames.forEach(k -> kinds.add(new QPossibleValue<>(k, providerKindLabel(k))));
List<QPossibleValue<?>> modelValues = new ArrayList<>();
models.forEach(m -> modelValues.add(new QPossibleValue<>(m, m)));
@@ -256,11 +263,41 @@ public final class AssistantRecordScreens
.withValueFormatAndFields(PVSValueFormatAndFields.LABEL_ONLY).withEnumValues(types),
new QPossibleValueSource().withName(MODEL_PVS_NAME).withType(QPossibleValueSourceType.ENUM).withIdType(QFieldType.STRING)
.withValueFormatAndFields(PVSValueFormatAndFields.LABEL_ONLY).withEnumValues(modelValues),
enumSource("assistant_provider_kind", kinds),
enumSource("assistant_provider_location", List.of(new QPossibleValue<>("LOCAL", "Local"), new QPossibleValue<>("CLOUD", "Cloud"))),
enumSource("assistant_model_source", List.of(new QPossibleValue<>("DISCOVERED", "Discovered"), new QPossibleValue<>("MANUAL", "Manual"))),
QPossibleValueSource.newForTable(AssistantProvider.TABLE_NAME)));
}
/*******************************************************************************
** Display label shared by provider settings and home cards. Custom kinds
** retain their registered name.
*******************************************************************************/
public static String providerKindLabel(String kind)
{
if(kind == null)
{
return (null);
}
return (switch(kind)
{
case "ANTHROPIC" -> "Anthropic";
case "OPENAI" -> "OpenAI";
case "AZURE_OPENAI" -> "Azure OpenAI";
case "OPENROUTER" -> "OpenRouter";
case "OLLAMA" -> "Ollama";
case "LM_STUDIO" -> "LM Studio";
case "VLLM" -> "vLLM";
case "LLAMA_CPP" -> "llama.cpp server";
case "OPENAI_COMPATIBLE" -> "OpenAI-compatible (other)";
default -> kind;
});
}
/*******************************************************************************
** The provider settings screen.
*******************************************************************************/
@@ -268,12 +305,20 @@ public final class AssistantRecordScreens
{
pvs(table, "providerType", PROVIDER_TYPE_PVS_NAME, "Type");
pvs(table, "model", MODEL_PVS_NAME, "Model");
pvs(table, "kind", "assistant_provider_kind", "Kind");
pvs(table, "location", "assistant_provider_location", "Location");
table.getField("hipaaEligible").withLabel("HIPAA-eligible");
table.getField("apiVersion").withLabel("API version");
table.getField("connectTimeoutSeconds").withLabel("Connect timeout (s)");
table.getField("requestTimeoutSeconds").withLabel("Request timeout (s)");
help(table, "hipaaEligible", "Records a fact (local, or cloud under a signed BAA). It does not create a BAA.");
table.getField("isDefault").withLabel("Default");
table.getField("apiKeyEnvName").withLabel("API Key Variable");
table.getField("tlsCaBundle").withLabel("TLS CA bundle");
table.getField("baseUrl").withLabel("Base URL Override");
table.getField("monthlyCostCapUsd").withLabel("Monthly Cost Cap (USD)").withDisplayFormat(DisplayFormat.CURRENCY);
counts(table, "maxOutputTokensPerRequest", "lastTestLatencyMs");
table.getField("lastTestLatencyMs").withLabel("Last Test Latency (ms)");
table.getField("lastTestLatencyMs").withLabel("Last test latency (ms)");
help(table, "name", "A name for this provider, e.g. Claude (production).");
help(table, "providerType", "Which kind of provider this is. Only the types installed on this server are offered.");
help(table, "model", "The model answers come from. Empty uses the policy's (or the application's) main model.");
@@ -281,6 +326,7 @@ public final class AssistantRecordScreens
help(table, "isDefault", "The assistant uses the default enabled provider (else the first enabled one). Marking one default unmarks the others.");
help(table, "apiKeyEnvName", "The NAME of the environment variable that holds the API key (e.g. ANTHROPIC_API_KEY), never the key itself. "
+ "The key stays in the server's secret store; only allowed variable names are accepted. Empty uses the type's default variable. To replace the key, change it in the secret store.");
help(table, "tlsCaBundle", "Name of a CA bundle configured on the server (withTrustedCaBundle). Leave empty to use the JVM trust store.");
help(table, "baseUrl", "Leave empty to use the provider's own API. Other URLs must be allowed in the server's configuration.");
help(table, "monthlyCostCapUsd", "When this provider's cost this month (UTC) reaches the cap, new requests are refused. Empty: no cap.");
help(table, "maxOutputTokensPerRequest", "The most output tokens one model call may produce. Empty uses the configured limit.");
@@ -288,8 +334,10 @@ public final class AssistantRecordScreens
table.getField(ProviderService.KEY_STATUS_FIELD).withLabel("API Key").withHelpContent(new QHelpContent("Whether the key variable is set on this server, with the key's last four characters. The key itself is never shown or returned."));
sections(table,
section("provider", "Provider", "settings_input_component", Tier.T1, "id", "name", "providerType", "model", "enabled", "isDefault"),
section("connection", "Connection", "key", Tier.T2, "apiKeyEnvName", ProviderService.KEY_STATUS_FIELD, "baseUrl"),
section("connection", "Connection", "key", Tier.T2, "apiKeyEnvName", ProviderService.KEY_STATUS_FIELD, "baseUrl", "tlsCaBundle"),
section("limits", "Limits", "speed", Tier.T2, "monthlyCostCapUsd", "maxOutputTokensPerRequest", "maxConcurrentRequests"),
section("registry", "Provider settings", "settings", Tier.T2, "kind", "location", "hipaaEligible", "region", "apiVersion", "connectTimeoutSeconds", "requestTimeoutSeconds"),
section("discovery", "Last discovery", "memory", Tier.T2, "lastDiscoveryAt", "lastDiscoveryStatus", "lastDiscoveryError"),
section("lastTest", "Last connection test", "network_check", Tier.T2, "lastTestAt", "lastTestStatus", "lastTestLatencyMs", "lastTestModel", "lastTestError"));
}
@@ -305,8 +353,8 @@ public final class AssistantRecordScreens
{
help(table, field, "USD per million tokens.");
}
table.getField("inputUsdPerMTok").withLabel("Input (USD / MTok)");
table.getField("outputUsdPerMTok").withLabel("Output (USD / MTok)");
table.getField("inputUsdPerMTok").withLabel("Input (USD per million tokens)");
table.getField("outputUsdPerMTok").withLabel("Output (USD per million tokens)");
table.getField("cacheReadUsdPerMTok").withLabel("Cache Read (USD / MTok)");
table.getField("cacheWrite5mUsdPerMTok").withLabel("Cache Write 5m (USD / MTok)");
table.getField("cacheWrite1hUsdPerMTok").withLabel("Cache Write 1h (USD / MTok)");
@@ -785,11 +833,34 @@ public final class AssistantRecordScreens
qInstance.addWidget(ChildRecordListRenderer.widgetMetaDataBuilder(join)
.withLabel(label)
.withMaxRows(50)
.withCanAddChildRecord(false)
.withCanAddChildRecord(AssistantModel.TABLE_NAME.equals(childTableName))
.withOnlyIncludeFieldNames(shown)
.getWidgetMetaData());
String sectionName = sectionPrefix + label.replace(" ", "");
conversation.addSection(new QFieldSection(sectionName, new QIcon(icon), Tier.T2).withLabel(label).withWidgetName(join.getName()));
}
/*******************************************************************************
** The model sub-table record screen.
*******************************************************************************/
private static void model(QTableMetaData table)
{
link(table, "providerId", AssistantProvider.TABLE_NAME, "Provider");
pvs(table, "source", "assistant_model_source", "Source");
table.getField("contextWindowTokens").withLabel("Context window (tokens)");
table.getField("maxOutputTokens").withLabel("Max output tokens");
table.getField("supportsReasoningEffort").withLabel("Supports reasoning effort");
table.getField("inputUsdPerMTok").withLabel("Input (USD per million tokens)");
table.getField("outputUsdPerMTok").withLabel("Output (USD per million tokens)");
table.getField("lastTestLatencyMs").withLabel("Last test latency (ms)");
sections(table,
section("model", "Model", "memory", Tier.T1, "id", "providerId", "modelId", "displayName", "enabled", "source", "discoveredAt"),
section("capabilities", "Capabilities", "settings", Tier.T2, "supportsTools", "supportsVision", "supportsStreaming", "supportsReasoningEffort", "contextWindowTokens", "maxOutputTokens"),
section("prices", "Prices", "payments", Tier.T2, "inputUsdPerMTok", "outputUsdPerMTok"),
section("lastTest", "Last connection test", "network_check", Tier.T2, "lastTestAt", "lastTestStatus", "lastTestLatencyMs", "lastTestError"));
}
}
@@ -25,10 +25,10 @@ import com.kingsrook.qbits.assistant.AssistantRuntime;
import com.kingsrook.qbits.assistant.engine.ProviderService;
import com.kingsrook.qbits.assistant.model.AssistantProvider;
import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
import com.kingsrook.qqq.backend.core.model.actions.tables.QueryOrGetInputInterface;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.QueryOrGetInputInterface;
import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.statusmessages.BadInputStatusMessage;
@@ -77,6 +77,24 @@ public class AssistantSettingsCustomizer implements TableCustomizerInterface
public List<QRecord> preInsertOrUpdate(AbstractActionInput input, List<QRecord> records, boolean isPreview, Optional<List<QRecord>> oldRecordList) throws QException
{
ProviderService service = service();
if(com.kingsrook.qbits.assistant.model.AssistantModel.TABLE_NAME.equals(tableName(input)))
{
for(QRecord record : records)
{
if(record.getValue("id") == null)
{
if(record.getValue("source") == null)
{
record.setValue("source", "MANUAL");
}
if(record.getValue("enabled") == null)
{
record.setValue("enabled", true);
}
}
}
return (records);
}
if(service == null || !AssistantProvider.TABLE_NAME.equals(tableName(input)))
{
return (records);
@@ -95,7 +113,23 @@ public class AssistantSettingsCustomizer implements TableCustomizerInterface
old.map(m -> m.get(record.getValue("id"))).ifPresent(o -> merged.setValues(new java.util.LinkedHashMap<>(o.getValues())));
record.getValues().forEach(merged::setValue);
for(String error : service.validate(new AssistantProvider(merged)))
AssistantProvider provider = new AssistantProvider(merged);
if(merged.getValue("kind") == null)
{
record.setValue("kind", service.kind(provider));
}
if(merged.getValue("location") == null)
{
record.setValue("location", service.location(provider));
}
for(String field : List.of("hipaaEligible", "connectTimeoutSeconds", "requestTimeoutSeconds"))
{
if(merged.getValue(field) == null)
{
record.setValue(field, field.equals("hipaaEligible") ? Boolean.FALSE : (field.equals("connectTimeoutSeconds") ? 10 : 120));
}
}
for(String error : service.validate(provider))
{
record.addError(new BadInputStatusMessage(error));
}
@@ -169,7 +203,7 @@ public class AssistantSettingsCustomizer implements TableCustomizerInterface
for(Map.Entry<String, Serializable> entry : record.getValues().entrySet())
{
String name = entry.getKey();
boolean system = name.equals("id") || name.equals("modifyDate") || name.startsWith("lastTest")
boolean system = name.equals("id") || name.equals("modifyDate") || name.startsWith("lastTest") || name.startsWith("lastDiscovery")
|| (name.equals("isDefault") && Boolean.FALSE.equals(entry.getValue()));
if(!system)
{
@@ -63,7 +63,7 @@ public class AssistantMessage extends QRecordEntity
@QField(isEditable = false)
private Integer seq;
@QField(isEditable = false, maxLength = 64)
@QField(isEditable = false, maxLength = 200)
private String model;
@QField(isEditable = false)
@@ -0,0 +1,767 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant.model;
import java.math.BigDecimal;
import java.time.Instant;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.data.QField;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
/*******************************************************************************
** A provider-scoped model, its capabilities, prices and last connection test.
*******************************************************************************/
public class AssistantModel extends QRecordEntity
{
public static final String TABLE_NAME = "assistant_model";
@QField(isEditable = false, isPrimaryKey = true)
private Integer id;
@QField(isRequired = true)
private Integer providerId;
@QField(isRequired = true, maxLength = 200)
private String modelId;
@QField(maxLength = 200)
private String displayName;
@QField()
private Boolean enabled;
@QField()
private Boolean supportsTools;
@QField()
private Boolean supportsVision;
@QField()
private Boolean supportsStreaming;
@QField()
private Boolean supportsReasoningEffort;
@QField()
private Integer contextWindowTokens;
@QField()
private Integer maxOutputTokens;
@QField()
private BigDecimal inputUsdPerMTok;
@QField()
private BigDecimal outputUsdPerMTok;
@QField(maxLength = 16)
private String source;
@QField(isEditable = false)
private Instant discoveredAt;
@QField(isEditable = false)
private Instant lastTestAt;
@QField(isEditable = false, maxLength = 16)
private String lastTestStatus;
@QField(isEditable = false)
private Integer lastTestLatencyMs;
@QField(isEditable = false, maxLength = 500)
private String lastTestError;
@QField(isEditable = false)
private Instant createDate;
@QField(isEditable = false)
private Instant modifyDate;
/*******************************************************************************
** Default constructor.
*******************************************************************************/
public AssistantModel()
{
}
/*******************************************************************************
** Populate from a record.
*******************************************************************************/
public AssistantModel(QRecord record) throws QException
{
populateFromQRecord(record);
}
/*******************************************************************************
** Getter for id
*******************************************************************************/
public Integer getId()
{
return (this.id);
}
/*******************************************************************************
** Setter for id
*******************************************************************************/
public void setId(Integer id)
{
this.id = id;
}
/*******************************************************************************
** Fluent setter for id
*******************************************************************************/
public AssistantModel withId(Integer id)
{
this.id = id;
return (this);
}
/*******************************************************************************
** Getter for providerId
*******************************************************************************/
public Integer getProviderId()
{
return (this.providerId);
}
/*******************************************************************************
** Setter for providerId
*******************************************************************************/
public void setProviderId(Integer providerId)
{
this.providerId = providerId;
}
/*******************************************************************************
** Fluent setter for providerId
*******************************************************************************/
public AssistantModel withProviderId(Integer providerId)
{
this.providerId = providerId;
return (this);
}
/*******************************************************************************
** Getter for modelId
*******************************************************************************/
public String getModelId()
{
return (this.modelId);
}
/*******************************************************************************
** Setter for modelId
*******************************************************************************/
public void setModelId(String modelId)
{
this.modelId = modelId;
}
/*******************************************************************************
** Fluent setter for modelId
*******************************************************************************/
public AssistantModel withModelId(String modelId)
{
this.modelId = modelId;
return (this);
}
/*******************************************************************************
** Getter for displayName
*******************************************************************************/
public String getDisplayName()
{
return (this.displayName);
}
/*******************************************************************************
** Setter for displayName
*******************************************************************************/
public void setDisplayName(String displayName)
{
this.displayName = displayName;
}
/*******************************************************************************
** Fluent setter for displayName
*******************************************************************************/
public AssistantModel withDisplayName(String displayName)
{
this.displayName = displayName;
return (this);
}
/*******************************************************************************
** Getter for enabled
*******************************************************************************/
public Boolean getEnabled()
{
return (this.enabled);
}
/*******************************************************************************
** Setter for enabled
*******************************************************************************/
public void setEnabled(Boolean enabled)
{
this.enabled = enabled;
}
/*******************************************************************************
** Fluent setter for enabled
*******************************************************************************/
public AssistantModel withEnabled(Boolean enabled)
{
this.enabled = enabled;
return (this);
}
/*******************************************************************************
** Getter for supportsTools
*******************************************************************************/
public Boolean getSupportsTools()
{
return (this.supportsTools);
}
/*******************************************************************************
** Setter for supportsTools
*******************************************************************************/
public void setSupportsTools(Boolean supportsTools)
{
this.supportsTools = supportsTools;
}
/*******************************************************************************
** Fluent setter for supportsTools
*******************************************************************************/
public AssistantModel withSupportsTools(Boolean supportsTools)
{
this.supportsTools = supportsTools;
return (this);
}
/*******************************************************************************
** Getter for supportsVision
*******************************************************************************/
public Boolean getSupportsVision()
{
return (this.supportsVision);
}
/*******************************************************************************
** Setter for supportsVision
*******************************************************************************/
public void setSupportsVision(Boolean supportsVision)
{
this.supportsVision = supportsVision;
}
/*******************************************************************************
** Fluent setter for supportsVision
*******************************************************************************/
public AssistantModel withSupportsVision(Boolean supportsVision)
{
this.supportsVision = supportsVision;
return (this);
}
/*******************************************************************************
** Getter for supportsStreaming
*******************************************************************************/
public Boolean getSupportsStreaming()
{
return (this.supportsStreaming);
}
/*******************************************************************************
** Setter for supportsStreaming
*******************************************************************************/
public void setSupportsStreaming(Boolean supportsStreaming)
{
this.supportsStreaming = supportsStreaming;
}
/*******************************************************************************
** Fluent setter for supportsStreaming
*******************************************************************************/
public AssistantModel withSupportsStreaming(Boolean supportsStreaming)
{
this.supportsStreaming = supportsStreaming;
return (this);
}
/*******************************************************************************
** Getter for supportsReasoningEffort
*******************************************************************************/
public Boolean getSupportsReasoningEffort()
{
return (this.supportsReasoningEffort);
}
/*******************************************************************************
** Setter for supportsReasoningEffort
*******************************************************************************/
public void setSupportsReasoningEffort(Boolean supportsReasoningEffort)
{
this.supportsReasoningEffort = supportsReasoningEffort;
}
/*******************************************************************************
** Fluent setter for supportsReasoningEffort
*******************************************************************************/
public AssistantModel withSupportsReasoningEffort(Boolean supportsReasoningEffort)
{
this.supportsReasoningEffort = supportsReasoningEffort;
return (this);
}
/*******************************************************************************
** Getter for contextWindowTokens
*******************************************************************************/
public Integer getContextWindowTokens()
{
return (this.contextWindowTokens);
}
/*******************************************************************************
** Setter for contextWindowTokens
*******************************************************************************/
public void setContextWindowTokens(Integer contextWindowTokens)
{
this.contextWindowTokens = contextWindowTokens;
}
/*******************************************************************************
** Fluent setter for contextWindowTokens
*******************************************************************************/
public AssistantModel withContextWindowTokens(Integer contextWindowTokens)
{
this.contextWindowTokens = contextWindowTokens;
return (this);
}
/*******************************************************************************
** Getter for maxOutputTokens
*******************************************************************************/
public Integer getMaxOutputTokens()
{
return (this.maxOutputTokens);
}
/*******************************************************************************
** Setter for maxOutputTokens
*******************************************************************************/
public void setMaxOutputTokens(Integer maxOutputTokens)
{
this.maxOutputTokens = maxOutputTokens;
}
/*******************************************************************************
** Fluent setter for maxOutputTokens
*******************************************************************************/
public AssistantModel withMaxOutputTokens(Integer maxOutputTokens)
{
this.maxOutputTokens = maxOutputTokens;
return (this);
}
/*******************************************************************************
** Getter for inputUsdPerMTok
*******************************************************************************/
public BigDecimal getInputUsdPerMTok()
{
return (this.inputUsdPerMTok);
}
/*******************************************************************************
** Setter for inputUsdPerMTok
*******************************************************************************/
public void setInputUsdPerMTok(BigDecimal inputUsdPerMTok)
{
this.inputUsdPerMTok = inputUsdPerMTok;
}
/*******************************************************************************
** Fluent setter for inputUsdPerMTok
*******************************************************************************/
public AssistantModel withInputUsdPerMTok(BigDecimal inputUsdPerMTok)
{
this.inputUsdPerMTok = inputUsdPerMTok;
return (this);
}
/*******************************************************************************
** Getter for outputUsdPerMTok
*******************************************************************************/
public BigDecimal getOutputUsdPerMTok()
{
return (this.outputUsdPerMTok);
}
/*******************************************************************************
** Setter for outputUsdPerMTok
*******************************************************************************/
public void setOutputUsdPerMTok(BigDecimal outputUsdPerMTok)
{
this.outputUsdPerMTok = outputUsdPerMTok;
}
/*******************************************************************************
** Fluent setter for outputUsdPerMTok
*******************************************************************************/
public AssistantModel withOutputUsdPerMTok(BigDecimal outputUsdPerMTok)
{
this.outputUsdPerMTok = outputUsdPerMTok;
return (this);
}
/*******************************************************************************
** Getter for source
*******************************************************************************/
public String getSource()
{
return (this.source);
}
/*******************************************************************************
** Setter for source
*******************************************************************************/
public void setSource(String source)
{
this.source = source;
}
/*******************************************************************************
** Fluent setter for source
*******************************************************************************/
public AssistantModel withSource(String source)
{
this.source = source;
return (this);
}
/*******************************************************************************
** Getter for discoveredAt
*******************************************************************************/
public Instant getDiscoveredAt()
{
return (this.discoveredAt);
}
/*******************************************************************************
** Setter for discoveredAt
*******************************************************************************/
public void setDiscoveredAt(Instant discoveredAt)
{
this.discoveredAt = discoveredAt;
}
/*******************************************************************************
** Fluent setter for discoveredAt
*******************************************************************************/
public AssistantModel withDiscoveredAt(Instant discoveredAt)
{
this.discoveredAt = discoveredAt;
return (this);
}
/*******************************************************************************
** Getter for lastTestAt
*******************************************************************************/
public Instant getLastTestAt()
{
return (this.lastTestAt);
}
/*******************************************************************************
** Setter for lastTestAt
*******************************************************************************/
public void setLastTestAt(Instant lastTestAt)
{
this.lastTestAt = lastTestAt;
}
/*******************************************************************************
** Fluent setter for lastTestAt
*******************************************************************************/
public AssistantModel withLastTestAt(Instant lastTestAt)
{
this.lastTestAt = lastTestAt;
return (this);
}
/*******************************************************************************
** Getter for lastTestStatus
*******************************************************************************/
public String getLastTestStatus()
{
return (this.lastTestStatus);
}
/*******************************************************************************
** Setter for lastTestStatus
*******************************************************************************/
public void setLastTestStatus(String lastTestStatus)
{
this.lastTestStatus = lastTestStatus;
}
/*******************************************************************************
** Fluent setter for lastTestStatus
*******************************************************************************/
public AssistantModel withLastTestStatus(String lastTestStatus)
{
this.lastTestStatus = lastTestStatus;
return (this);
}
/*******************************************************************************
** Getter for lastTestLatencyMs
*******************************************************************************/
public Integer getLastTestLatencyMs()
{
return (this.lastTestLatencyMs);
}
/*******************************************************************************
** Setter for lastTestLatencyMs
*******************************************************************************/
public void setLastTestLatencyMs(Integer lastTestLatencyMs)
{
this.lastTestLatencyMs = lastTestLatencyMs;
}
/*******************************************************************************
** Fluent setter for lastTestLatencyMs
*******************************************************************************/
public AssistantModel withLastTestLatencyMs(Integer lastTestLatencyMs)
{
this.lastTestLatencyMs = lastTestLatencyMs;
return (this);
}
/*******************************************************************************
** Getter for lastTestError
*******************************************************************************/
public String getLastTestError()
{
return (this.lastTestError);
}
/*******************************************************************************
** Setter for lastTestError
*******************************************************************************/
public void setLastTestError(String lastTestError)
{
this.lastTestError = lastTestError;
}
/*******************************************************************************
** Fluent setter for lastTestError
*******************************************************************************/
public AssistantModel withLastTestError(String lastTestError)
{
this.lastTestError = lastTestError;
return (this);
}
/*******************************************************************************
** Getter for createDate
*******************************************************************************/
public Instant getCreateDate()
{
return (this.createDate);
}
/*******************************************************************************
** Setter for createDate
*******************************************************************************/
public void setCreateDate(Instant createDate)
{
this.createDate = createDate;
}
/*******************************************************************************
** Fluent setter for createDate
*******************************************************************************/
public AssistantModel withCreateDate(Instant createDate)
{
this.createDate = createDate;
return (this);
}
/*******************************************************************************
** Getter for modifyDate
*******************************************************************************/
public Instant getModifyDate()
{
return (this.modifyDate);
}
/*******************************************************************************
** Setter for modifyDate
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Fluent setter for modifyDate
*******************************************************************************/
public AssistantModel withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
}
@@ -35,6 +35,37 @@ import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
*******************************************************************************/
public class AssistantProvider extends QRecordEntity
{
@QField(maxLength = 30)
private String kind;
@QField(maxLength = 10)
private String location;
@QField()
private Boolean hipaaEligible;
@QField(maxLength = 50)
private String region;
@QField(maxLength = 30)
private String apiVersion;
@QField()
private Integer connectTimeoutSeconds;
@QField()
private Integer requestTimeoutSeconds;
@QField(isEditable = false)
private Instant lastDiscoveryAt;
@QField(isEditable = false, maxLength = 16)
private String lastDiscoveryStatus;
@QField(isEditable = false, maxLength = 500)
private String lastDiscoveryError;
public static final String TABLE_NAME = "assistant_provider";
@QField(isEditable = false, isPrimaryKey = true)
@@ -46,7 +77,7 @@ public class AssistantProvider extends QRecordEntity
@QField(isRequired = true, maxLength = 50)
private String providerType;
@QField(maxLength = 100)
@QField(maxLength = 200)
private String model;
@QField()
@@ -58,6 +89,9 @@ public class AssistantProvider extends QRecordEntity
@QField(maxLength = 255)
private String baseUrl;
@QField(maxLength = 100)
private String tlsCaBundle;
@QField(maxLength = 100)
private String apiKeyEnvName;
@@ -86,7 +120,7 @@ public class AssistantProvider extends QRecordEntity
@QField(isEditable = false)
private Integer lastTestLatencyMs;
@QField(isEditable = false, maxLength = 100)
@QField(isEditable = false, maxLength = 200)
private String lastTestModel;
@QField(isEditable = false, maxLength = 500)
@@ -706,4 +740,346 @@ public class AssistantProvider extends QRecordEntity
return (this);
}
/*******************************************************************************
** Getter for kind
*******************************************************************************/
public String getKind()
{
return (this.kind);
}
/*******************************************************************************
** Setter for kind
*******************************************************************************/
public void setKind(String kind)
{
this.kind = kind;
}
/*******************************************************************************
** Fluent setter for kind
*******************************************************************************/
public AssistantProvider withKind(String kind)
{
this.kind = kind;
return (this);
}
/*******************************************************************************
** Getter for location
*******************************************************************************/
public String getLocation()
{
return (this.location);
}
/*******************************************************************************
** Setter for location
*******************************************************************************/
public void setLocation(String location)
{
this.location = location;
}
/*******************************************************************************
** Fluent setter for location
*******************************************************************************/
public AssistantProvider withLocation(String location)
{
this.location = location;
return (this);
}
/*******************************************************************************
** Getter for hipaaEligible
*******************************************************************************/
public Boolean getHipaaEligible()
{
return (this.hipaaEligible);
}
/*******************************************************************************
** Setter for hipaaEligible
*******************************************************************************/
public void setHipaaEligible(Boolean hipaaEligible)
{
this.hipaaEligible = hipaaEligible;
}
/*******************************************************************************
** Fluent setter for hipaaEligible
*******************************************************************************/
public AssistantProvider withHipaaEligible(Boolean hipaaEligible)
{
this.hipaaEligible = hipaaEligible;
return (this);
}
/*******************************************************************************
** Getter for region
*******************************************************************************/
public String getRegion()
{
return (this.region);
}
/*******************************************************************************
** Setter for region
*******************************************************************************/
public void setRegion(String region)
{
this.region = region;
}
/*******************************************************************************
** Fluent setter for region
*******************************************************************************/
public AssistantProvider withRegion(String region)
{
this.region = region;
return (this);
}
/*******************************************************************************
** Getter for apiVersion
*******************************************************************************/
public String getApiVersion()
{
return (this.apiVersion);
}
/*******************************************************************************
** Setter for apiVersion
*******************************************************************************/
public void setApiVersion(String apiVersion)
{
this.apiVersion = apiVersion;
}
/*******************************************************************************
** Fluent setter for apiVersion
*******************************************************************************/
public AssistantProvider withApiVersion(String apiVersion)
{
this.apiVersion = apiVersion;
return (this);
}
/*******************************************************************************
** Getter for connectTimeoutSeconds
*******************************************************************************/
public Integer getConnectTimeoutSeconds()
{
return (this.connectTimeoutSeconds);
}
/*******************************************************************************
** Setter for connectTimeoutSeconds
*******************************************************************************/
public void setConnectTimeoutSeconds(Integer connectTimeoutSeconds)
{
this.connectTimeoutSeconds = connectTimeoutSeconds;
}
/*******************************************************************************
** Fluent setter for connectTimeoutSeconds
*******************************************************************************/
public AssistantProvider withConnectTimeoutSeconds(Integer connectTimeoutSeconds)
{
this.connectTimeoutSeconds = connectTimeoutSeconds;
return (this);
}
/*******************************************************************************
** Getter for requestTimeoutSeconds
*******************************************************************************/
public Integer getRequestTimeoutSeconds()
{
return (this.requestTimeoutSeconds);
}
/*******************************************************************************
** Setter for requestTimeoutSeconds
*******************************************************************************/
public void setRequestTimeoutSeconds(Integer requestTimeoutSeconds)
{
this.requestTimeoutSeconds = requestTimeoutSeconds;
}
/*******************************************************************************
** Fluent setter for requestTimeoutSeconds
*******************************************************************************/
public AssistantProvider withRequestTimeoutSeconds(Integer requestTimeoutSeconds)
{
this.requestTimeoutSeconds = requestTimeoutSeconds;
return (this);
}
/*******************************************************************************
** Getter for lastDiscoveryAt
*******************************************************************************/
public Instant getLastDiscoveryAt()
{
return (this.lastDiscoveryAt);
}
/*******************************************************************************
** Setter for lastDiscoveryAt
*******************************************************************************/
public void setLastDiscoveryAt(Instant lastDiscoveryAt)
{
this.lastDiscoveryAt = lastDiscoveryAt;
}
/*******************************************************************************
** Fluent setter for lastDiscoveryAt
*******************************************************************************/
public AssistantProvider withLastDiscoveryAt(Instant lastDiscoveryAt)
{
this.lastDiscoveryAt = lastDiscoveryAt;
return (this);
}
/*******************************************************************************
** Getter for lastDiscoveryStatus
*******************************************************************************/
public String getLastDiscoveryStatus()
{
return (this.lastDiscoveryStatus);
}
/*******************************************************************************
** Setter for lastDiscoveryStatus
*******************************************************************************/
public void setLastDiscoveryStatus(String lastDiscoveryStatus)
{
this.lastDiscoveryStatus = lastDiscoveryStatus;
}
/*******************************************************************************
** Fluent setter for lastDiscoveryStatus
*******************************************************************************/
public AssistantProvider withLastDiscoveryStatus(String lastDiscoveryStatus)
{
this.lastDiscoveryStatus = lastDiscoveryStatus;
return (this);
}
/*******************************************************************************
** Getter for lastDiscoveryError
*******************************************************************************/
public String getLastDiscoveryError()
{
return (this.lastDiscoveryError);
}
/*******************************************************************************
** Setter for lastDiscoveryError
*******************************************************************************/
public void setLastDiscoveryError(String lastDiscoveryError)
{
this.lastDiscoveryError = lastDiscoveryError;
}
/*******************************************************************************
** Fluent setter for lastDiscoveryError
*******************************************************************************/
public AssistantProvider withLastDiscoveryError(String lastDiscoveryError)
{
this.lastDiscoveryError = lastDiscoveryError;
return (this);
}
/*******************************************************************************
** Getter for tlsCaBundle
*******************************************************************************/
public String getTlsCaBundle()
{
return (this.tlsCaBundle);
}
/*******************************************************************************
** Setter for tlsCaBundle
*******************************************************************************/
public void setTlsCaBundle(String tlsCaBundle)
{
this.tlsCaBundle = tlsCaBundle;
}
/*******************************************************************************
** Fluent setter for tlsCaBundle
*******************************************************************************/
public AssistantProvider withTlsCaBundle(String tlsCaBundle)
{
this.tlsCaBundle = tlsCaBundle;
return (this);
}
}
@@ -65,7 +65,7 @@ public class AssistantTurn extends QRecordEntity
@QField(isEditable = false, maxLength = 64)
private String cancelReason;
@QField(isEditable = false, maxLength = 64)
@QField(isEditable = false, maxLength = 200)
private String model;
@QField(isEditable = false)
@@ -51,7 +51,7 @@ public class AssistantUsage extends QRecordEntity
@QField(isEditable = false, maxLength = 50)
private String providerType;
@QField(isEditable = false, maxLength = 100)
@QField(isEditable = false, maxLength = 200)
private String model;
@QField(isEditable = false, maxLength = 16)
@@ -28,6 +28,7 @@ import java.util.concurrent.ConcurrentHashMap;
import com.kingsrook.qbits.assistant.model.AssistantConversation;
import com.kingsrook.qbits.assistant.model.AssistantFeedback;
import com.kingsrook.qbits.assistant.model.AssistantMessage;
import com.kingsrook.qbits.assistant.model.AssistantModel;
import com.kingsrook.qbits.assistant.model.AssistantModelPrice;
import com.kingsrook.qbits.assistant.model.AssistantPendingAction;
import com.kingsrook.qbits.assistant.model.AssistantPolicy;
@@ -728,4 +729,15 @@ public class AssistantStore
return (records == null ? List.of() : records);
}
/*******************************************************************************
** All registry models in stable id order.
*******************************************************************************/
public List<AssistantModel> listModels() throws QException
{
return (query(AssistantModel.TABLE_NAME, new QQueryFilter().withOrderBy(new QFilterOrderBy("id", true)), AssistantModel.class));
}
}
@@ -0,0 +1,216 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant.processes;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.assistant.AssistantRuntime;
import com.kingsrook.qbits.assistant.engine.ModelDiscovery;
import com.kingsrook.qbits.assistant.engine.ProviderService;
import com.kingsrook.qbits.assistant.metadata.AssistantMetaDataBuilder;
import com.kingsrook.qbits.assistant.model.AssistantProvider;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractLoadStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractTransformStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ExtractViaQueryStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ProcessSummaryProviderInterface;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
/*******************************************************************************
** Discover models as a standard streamed ETL record action. Review inspects
** settings without network calls; Result adds missing models and records status.
*******************************************************************************/
public final class DiscoverModelsProcess
{
public static final String NAME = "assistantProviderDiscoverModels";
/*******************************************************************************
** Utility class.
*******************************************************************************/
private DiscoverModelsProcess()
{
}
/*******************************************************************************
** The process.
*******************************************************************************/
public static QProcessMetaData process()
{
Map<String, Serializable> values = new HashMap<>();
values.put(StreamedETLWithFrontendProcess.FIELD_SOURCE_TABLE, AssistantProvider.TABLE_NAME);
values.put(StreamedETLWithFrontendProcess.FIELD_DESTINATION_TABLE, AssistantProvider.TABLE_NAME);
values.put(StreamedETLWithFrontendProcess.FIELD_SUPPORTS_FULL_VALIDATION, true);
values.put(StreamedETLWithFrontendProcess.FIELD_DO_FULL_VALIDATION, true);
values.put(StreamedETLWithFrontendProcess.FIELD_PREVIEW_MESSAGE,
"This lists available models and adds missing registry rows. Existing models are preserved.");
QProcessMetaData process = StreamedETLWithFrontendProcess.defineProcessMetaData(ExtractViaQueryStep.class, Transform.class, Load.class, values);
process.setName(NAME);
process.setLabel("Discover models");
process.setIcon(new QIcon("network_check"));
process.setTableName(AssistantProvider.TABLE_NAME);
process.setPermissionRules(AssistantMetaDataBuilder.adminRules());
return (process);
}
/*******************************************************************************
** A summary line with future (Review) and past (Result) wording.
*******************************************************************************/
static ProcessSummaryLine line(Status status, String singularFuture, String pluralFuture, String singularPast, String pluralPast)
{
ProcessSummaryLine line = new ProcessSummaryLine(status, 0, null);
line.setSingularFutureMessage(singularFuture);
line.setPluralFutureMessage(pluralFuture);
line.setSingularPastMessage(singularPast);
line.setPluralPastMessage(pluralPast);
return (line);
}
/*******************************************************************************
** A one-record line with the same text on both screens.
*******************************************************************************/
static ProcessSummaryLine single(Status status, String text, Serializable primaryKey)
{
ProcessSummaryLine line = line(status, text, text, text, text);
line.incrementCountAndAddPrimaryKey(primaryKey);
return (line);
}
/*******************************************************************************
** Review: nothing is called yet.
*******************************************************************************/
public static class Transform extends AbstractTransformStep implements ProcessSummaryProviderInterface
{
private final ProcessSummaryLine willTest = line(Status.OK, "provider will be queried for models", "providers will be queried for models",
"provider was queried", "providers were queried");
private final ProcessSummaryLine disabled = line(Status.INFO, "provider is turned off (it is tested anyway)", "providers are turned off (they are tested anyway)",
"provider was turned off", "providers were turned off");
private final List<ProcessSummaryLineInterface> problems = new ArrayList<>();
/*******************************************************************************
**
*******************************************************************************/
@Override
public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
AssistantRuntime runtime = AssistantRuntime.of(QContext.getQInstance());
for(QRecord record : input.getRecords())
{
AssistantProvider row = new AssistantProvider(record);
willTest.incrementCountAndAddPrimaryKey(row.getId());
if(!Boolean.TRUE.equals(row.getEnabled()))
{
disabled.incrementCountAndAddPrimaryKey(row.getId());
}
if(runtime != null)
{
ProviderService service = runtime.getProviderService();
for(String error : service.validate(row))
{
problems.add(single(Status.ERROR, row.getName() + ": " + error, row.getId()));
}
problems.add(single(Status.INFO, row.getName() + ": API key " + service.keyStatus(row), row.getId()));
}
output.getRecords().add(record);
}
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean isForResultScreen)
{
ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>();
willTest.addSelfToListIfAnyCount(lines);
disabled.addSelfToListIfAnyCount(lines);
lines.addAll(problems);
return (lines);
}
}
/*******************************************************************************
** Result: one line per provider.
*******************************************************************************/
public static class Load extends AbstractLoadStep implements ProcessSummaryProviderInterface
{
private final List<ProcessSummaryLineInterface> lines = new ArrayList<>();
/*******************************************************************************
**
*******************************************************************************/
@Override
public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
AssistantRuntime runtime = AssistantRuntime.of(QContext.getQInstance());
if(runtime == null)
{
throw (new QException("The assistant is not running on this server."));
}
for(QRecord record : input.getRecords())
{
AssistantProvider row = new AssistantProvider(record);
ModelDiscovery.Result result = ModelDiscovery.run(runtime, row);
lines.add(single(result.ok() ? Status.OK : Status.ERROR, row.getName() + ": " + result.message(), row.getId()));
output.getRecords().add(record);
}
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean isForResultScreen)
{
return (new ArrayList<>(lines));
}
}
}
@@ -201,13 +201,14 @@ public final class TestConnectionProcess
{
AssistantProvider row = new AssistantProvider(record);
ConnectionTest.Result result = ConnectionTest.run(runtime, row, userId);
String label = row.getName() + " (" + runtime.getProviderService().kind(row) + ", " + runtime.getProviderService().location(row) + ")";
if(result.ok())
{
lines.add(single(Status.OK, row.getName() + ": connected to " + result.model() + " in " + result.latencyMs() + " ms", row.getId()));
lines.add(single(Status.OK, label + ": connected to " + result.model() + " in " + result.latencyMs() + " ms", row.getId()));
}
else
{
lines.add(single(Status.ERROR, row.getName() + ": " + result.error(), row.getId()));
lines.add(single(Status.ERROR, label + ": " + result.error(), row.getId()));
}
output.getRecords().add(record);
}
@@ -0,0 +1,225 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant.processes;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.assistant.AssistantRuntime;
import com.kingsrook.qbits.assistant.AssistantSecurity;
import com.kingsrook.qbits.assistant.engine.ConnectionTest;
import com.kingsrook.qbits.assistant.engine.ProviderService;
import com.kingsrook.qbits.assistant.metadata.AssistantMetaDataBuilder;
import com.kingsrook.qbits.assistant.model.AssistantModel;
import com.kingsrook.qbits.assistant.model.AssistantProvider;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractLoadStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractTransformStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ExtractViaQueryStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ProcessSummaryProviderInterface;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
/*******************************************************************************
** Test a registry model as a standard streamed ETL record action. Review
** inspects its parent settings; Result records the model test and TEST usage.
*******************************************************************************/
public final class TestModelConnectionProcess
{
public static final String NAME = "assistantModelTestConnection";
/*******************************************************************************
** Utility class.
*******************************************************************************/
private TestModelConnectionProcess()
{
}
/*******************************************************************************
** The process.
*******************************************************************************/
public static QProcessMetaData process()
{
Map<String, Serializable> values = new HashMap<>();
values.put(StreamedETLWithFrontendProcess.FIELD_SOURCE_TABLE, AssistantModel.TABLE_NAME);
values.put(StreamedETLWithFrontendProcess.FIELD_DESTINATION_TABLE, AssistantModel.TABLE_NAME);
values.put(StreamedETLWithFrontendProcess.FIELD_SUPPORTS_FULL_VALIDATION, true);
values.put(StreamedETLWithFrontendProcess.FIELD_DO_FULL_VALIDATION, true);
values.put(StreamedETLWithFrontendProcess.FIELD_PREVIEW_MESSAGE,
"This makes one short live call to each selected registry model (a few tokens) and saves the result on the model.");
QProcessMetaData process = StreamedETLWithFrontendProcess.defineProcessMetaData(ExtractViaQueryStep.class, Transform.class, Load.class, values);
process.setName(NAME);
process.setLabel("Test connection");
process.setIcon(new QIcon("network_check"));
process.setTableName(AssistantModel.TABLE_NAME);
process.setPermissionRules(AssistantMetaDataBuilder.adminRules());
return (process);
}
/*******************************************************************************
** A summary line with future (Review) and past (Result) wording.
*******************************************************************************/
static ProcessSummaryLine line(Status status, String singularFuture, String pluralFuture, String singularPast, String pluralPast)
{
ProcessSummaryLine line = new ProcessSummaryLine(status, 0, null);
line.setSingularFutureMessage(singularFuture);
line.setPluralFutureMessage(pluralFuture);
line.setSingularPastMessage(singularPast);
line.setPluralPastMessage(pluralPast);
return (line);
}
/*******************************************************************************
** A one-record line with the same text on both screens.
*******************************************************************************/
static ProcessSummaryLine single(Status status, String text, Serializable primaryKey)
{
ProcessSummaryLine line = line(status, text, text, text, text);
line.incrementCountAndAddPrimaryKey(primaryKey);
return (line);
}
/*******************************************************************************
** Review: nothing is called yet.
*******************************************************************************/
public static class Transform extends AbstractTransformStep implements ProcessSummaryProviderInterface
{
private final ProcessSummaryLine willTest = line(Status.OK, "model will be tested with a live call", "models will be tested with a live call",
"model was tested", "models were tested");
private final ProcessSummaryLine disabled = line(Status.INFO, "model is turned off (it is tested anyway)", "models are turned off (they are tested anyway)",
"model was turned off", "models were turned off");
private final List<ProcessSummaryLineInterface> problems = new ArrayList<>();
/*******************************************************************************
**
*******************************************************************************/
@Override
public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
AssistantRuntime runtime = AssistantRuntime.of(QContext.getQInstance());
for(QRecord record : input.getRecords())
{
AssistantModel model = new AssistantModel(record);
AssistantProvider row = runtime == null ? null : runtime.getStore().getProvider(model.getProviderId());
willTest.incrementCountAndAddPrimaryKey(model.getId());
if(!Boolean.TRUE.equals(model.getEnabled()))
{
disabled.incrementCountAndAddPrimaryKey(model.getId());
}
if(runtime != null && row != null)
{
ProviderService service = runtime.getProviderService();
for(String error : service.validate(row))
{
problems.add(single(Status.ERROR, model.getModelId() + ": " + error, model.getId()));
}
problems.add(single(Status.INFO, model.getModelId() + ": API key " + service.keyStatus(row), model.getId()));
}
if(row == null)
{
problems.add(single(Status.ERROR, "The model's provider no longer exists.", model.getId()));
}
output.getRecords().add(record);
}
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean isForResultScreen)
{
ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>();
willTest.addSelfToListIfAnyCount(lines);
disabled.addSelfToListIfAnyCount(lines);
lines.addAll(problems);
return (lines);
}
}
/*******************************************************************************
** Result: one line per provider.
*******************************************************************************/
public static class Load extends AbstractLoadStep implements ProcessSummaryProviderInterface
{
private final List<ProcessSummaryLineInterface> lines = new ArrayList<>();
/*******************************************************************************
**
*******************************************************************************/
@Override
public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
AssistantRuntime runtime = AssistantRuntime.of(QContext.getQInstance());
if(runtime == null)
{
throw (new QException("The assistant is not running on this server."));
}
String userId = AssistantSecurity.userId(QContext.getQSession());
for(QRecord record : input.getRecords())
{
AssistantModel row = new AssistantModel(record);
ConnectionTest.Result result = ConnectionTest.runModel(runtime, row, userId);
lines.add(single(result.ok() ? Status.OK : Status.ERROR, row.getModelId() + ": "
+ (result.ok() ? "connected in " + result.latencyMs() + " ms" : result.error()), row.getId()));
output.getRecords().add(record);
}
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean isForResultScreen)
{
return (new ArrayList<>(lines));
}
}
}
@@ -13,6 +13,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant.provider;
@@ -63,4 +64,75 @@ public interface AssistantProviderType
*******************************************************************************/
AssistantModelProvider create(String apiKey, String baseUrl);
/*******************************************************************************
** Backward-compatible provider registry extension.
*******************************************************************************/
default List<String> getKinds()
{
return (List.of(getType().toUpperCase(java.util.Locale.ROOT).replace('-', '_')));
}
/*******************************************************************************
** Backward-compatible provider registry extension.
*******************************************************************************/
default String getDefaultBaseUrl(String kind)
{
return (getDefaultBaseUrl());
}
/*******************************************************************************
** Backward-compatible provider registry extension.
*******************************************************************************/
default String getDefaultApiKeyEnvName(String kind)
{
return (getDefaultApiKeyEnvName());
}
/*******************************************************************************
** Backward-compatible provider registry extension.
*******************************************************************************/
default String getDefaultLocation(String kind)
{
return ("CLOUD");
}
/*******************************************************************************
** Backward-compatible provider registry extension.
*******************************************************************************/
default AssistantModelProvider create(ProviderSettings settings)
{
return (create(settings.getApiKey(), settings.getBaseUrl()));
}
/*******************************************************************************
** Backward-compatible provider registry extension.
*******************************************************************************/
default List<DiscoveredModel> discoverModels(ProviderSettings settings) throws ModelProviderException
{
return (getModels().stream().map(id -> new DiscoveredModel(id, id, null, null)).toList());
}
/*******************************************************************************
** Backward-compatible provider registry extension.
*******************************************************************************/
default HealthResult checkHealth(ProviderSettings settings)
{
return (new HealthResult(null, null, "Health check not supported"));
}
}
@@ -0,0 +1,25 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant.provider;
/*******************************************************************************
** A model advertised by the provider, without inferred capabilities.
*******************************************************************************/
public record DiscoveredModel(String modelId, String displayName, Integer contextWindowTokens, String ownedBy)
{
}
@@ -0,0 +1,32 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant.provider;
/*******************************************************************************
** A health probe result; null healthy means unsupported.
*******************************************************************************/
public record HealthResult(Boolean healthy, Long latencyMs, String message)
{
/*******************************************************************************
** Sanitize provider diagnostics.
*******************************************************************************/
public HealthResult
{
message = ModelProviderException.sanitizeProviderMessage(message, 300);
}
}
@@ -23,6 +23,16 @@ package com.kingsrook.qbits.assistant.provider;
*******************************************************************************/
public class ModelCapabilities
{
private Boolean vision = false;
private Boolean streaming = true;
private Boolean reasoningEffort = false;
private Boolean tools = true;
private Boolean parallelToolCalls = true;
private Boolean promptCaching = false;
@@ -182,4 +192,98 @@ public class ModelCapabilities
return (this);
}
/*******************************************************************************
** Getter for vision
*******************************************************************************/
public Boolean getVision()
{
return (this.vision);
}
/*******************************************************************************
** Setter for vision
*******************************************************************************/
public void setVision(Boolean vision)
{
this.vision = vision;
}
/*******************************************************************************
** Fluent setter for vision
*******************************************************************************/
public ModelCapabilities withVision(Boolean vision)
{
this.vision = vision;
return (this);
}
/*******************************************************************************
** Getter for streaming
*******************************************************************************/
public Boolean getStreaming()
{
return (this.streaming);
}
/*******************************************************************************
** Setter for streaming
*******************************************************************************/
public void setStreaming(Boolean streaming)
{
this.streaming = streaming;
}
/*******************************************************************************
** Fluent setter for streaming
*******************************************************************************/
public ModelCapabilities withStreaming(Boolean streaming)
{
this.streaming = streaming;
return (this);
}
/*******************************************************************************
** Getter for reasoningEffort
*******************************************************************************/
public Boolean getReasoningEffort()
{
return (this.reasoningEffort);
}
/*******************************************************************************
** Setter for reasoningEffort
*******************************************************************************/
public void setReasoningEffort(Boolean reasoningEffort)
{
this.reasoningEffort = reasoningEffort;
}
/*******************************************************************************
** Fluent setter for reasoningEffort
*******************************************************************************/
public ModelCapabilities withReasoningEffort(Boolean reasoningEffort)
{
this.reasoningEffort = reasoningEffort;
return (this);
}
}
@@ -32,6 +32,10 @@ import java.util.List;
*******************************************************************************/
public class ModelRequest
{
private ModelCapabilities capabilities;
private String model;
private String effort;
private Integer maxTokens;
@@ -49,6 +53,7 @@ public class ModelRequest
{
return (new ModelRequest()
.withModel(model)
.withCapabilities(capabilities)
.withEffort(effort)
.withMaxTokens(maxTokens)
.withParallelToolCalls(parallelToolCalls)
@@ -274,4 +279,36 @@ public class ModelRequest
return (this);
}
/*******************************************************************************
** Getter for capabilities
*******************************************************************************/
public ModelCapabilities getCapabilities()
{
return (this.capabilities);
}
/*******************************************************************************
** Setter for capabilities
*******************************************************************************/
public void setCapabilities(ModelCapabilities capabilities)
{
this.capabilities = capabilities;
}
/*******************************************************************************
** Fluent setter for capabilities
*******************************************************************************/
public ModelRequest withCapabilities(ModelCapabilities capabilities)
{
this.capabilities = capabilities;
return (this);
}
}
@@ -0,0 +1,330 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant.provider;
import javax.net.ssl.SSLContext;
import com.fasterxml.jackson.annotation.JsonIgnore;
/*******************************************************************************
** Resolved connection settings. The API key is never included in diagnostics.
*******************************************************************************/
public class ProviderSettings
{
private String tlsCaBundle;
@JsonIgnore
private transient SSLContext sslContext;
private String kind;
private String apiKey;
private String baseUrl;
private String apiVersion;
private String region;
private Integer connectTimeoutSeconds = 10;
private Integer requestTimeoutSeconds = 120;
/*******************************************************************************
** Getter for kind
*******************************************************************************/
public String getKind()
{
return (this.kind);
}
/*******************************************************************************
** Setter for kind
*******************************************************************************/
public void setKind(String kind)
{
this.kind = kind;
}
/*******************************************************************************
** Fluent setter for kind
*******************************************************************************/
public ProviderSettings withKind(String kind)
{
this.kind = kind;
return (this);
}
/*******************************************************************************
** Getter for apiKey
*******************************************************************************/
public String getApiKey()
{
return (this.apiKey);
}
/*******************************************************************************
** Setter for apiKey
*******************************************************************************/
public void setApiKey(String apiKey)
{
this.apiKey = apiKey;
}
/*******************************************************************************
** Fluent setter for apiKey
*******************************************************************************/
public ProviderSettings withApiKey(String apiKey)
{
this.apiKey = apiKey;
return (this);
}
/*******************************************************************************
** Getter for baseUrl
*******************************************************************************/
public String getBaseUrl()
{
return (this.baseUrl);
}
/*******************************************************************************
** Setter for baseUrl
*******************************************************************************/
public void setBaseUrl(String baseUrl)
{
this.baseUrl = baseUrl;
}
/*******************************************************************************
** Fluent setter for baseUrl
*******************************************************************************/
public ProviderSettings withBaseUrl(String baseUrl)
{
this.baseUrl = baseUrl;
return (this);
}
/*******************************************************************************
** Getter for apiVersion
*******************************************************************************/
public String getApiVersion()
{
return (this.apiVersion);
}
/*******************************************************************************
** Setter for apiVersion
*******************************************************************************/
public void setApiVersion(String apiVersion)
{
this.apiVersion = apiVersion;
}
/*******************************************************************************
** Fluent setter for apiVersion
*******************************************************************************/
public ProviderSettings withApiVersion(String apiVersion)
{
this.apiVersion = apiVersion;
return (this);
}
/*******************************************************************************
** Getter for region
*******************************************************************************/
public String getRegion()
{
return (this.region);
}
/*******************************************************************************
** Setter for region
*******************************************************************************/
public void setRegion(String region)
{
this.region = region;
}
/*******************************************************************************
** Fluent setter for region
*******************************************************************************/
public ProviderSettings withRegion(String region)
{
this.region = region;
return (this);
}
/*******************************************************************************
** Getter for connectTimeoutSeconds
*******************************************************************************/
public Integer getConnectTimeoutSeconds()
{
return (this.connectTimeoutSeconds);
}
/*******************************************************************************
** Setter for connectTimeoutSeconds
*******************************************************************************/
public void setConnectTimeoutSeconds(Integer connectTimeoutSeconds)
{
this.connectTimeoutSeconds = connectTimeoutSeconds;
}
/*******************************************************************************
** Fluent setter for connectTimeoutSeconds
*******************************************************************************/
public ProviderSettings withConnectTimeoutSeconds(Integer connectTimeoutSeconds)
{
this.connectTimeoutSeconds = connectTimeoutSeconds;
return (this);
}
/*******************************************************************************
** Getter for requestTimeoutSeconds
*******************************************************************************/
public Integer getRequestTimeoutSeconds()
{
return (this.requestTimeoutSeconds);
}
/*******************************************************************************
** Setter for requestTimeoutSeconds
*******************************************************************************/
public void setRequestTimeoutSeconds(Integer requestTimeoutSeconds)
{
this.requestTimeoutSeconds = requestTimeoutSeconds;
}
/*******************************************************************************
** Fluent setter for requestTimeoutSeconds
*******************************************************************************/
public ProviderSettings withRequestTimeoutSeconds(Integer requestTimeoutSeconds)
{
this.requestTimeoutSeconds = requestTimeoutSeconds;
return (this);
}
/*******************************************************************************
** Safe diagnostic representation.
*******************************************************************************/
@Override
public String toString()
{
return ("ProviderSettings[kind=" + kind + "]");
}
/*******************************************************************************
** Getter for tlsCaBundle
*******************************************************************************/
public String getTlsCaBundle()
{
return (this.tlsCaBundle);
}
/*******************************************************************************
** Setter for tlsCaBundle
*******************************************************************************/
public void setTlsCaBundle(String tlsCaBundle)
{
this.tlsCaBundle = tlsCaBundle;
}
/*******************************************************************************
** Fluent setter for tlsCaBundle
*******************************************************************************/
public ProviderSettings withTlsCaBundle(String tlsCaBundle)
{
this.tlsCaBundle = tlsCaBundle;
return (this);
}
/*******************************************************************************
** Getter for sslContext
*******************************************************************************/
@JsonIgnore
public SSLContext getSslContext()
{
return (this.sslContext);
}
/*******************************************************************************
** Setter for sslContext
*******************************************************************************/
public void setSslContext(SSLContext sslContext)
{
this.sslContext = sslContext;
}
/*******************************************************************************
** Fluent setter for sslContext
*******************************************************************************/
public ProviderSettings withSslContext(SSLContext sslContext)
{
this.sslContext = sslContext;
return (this);
}
}
@@ -0,0 +1,97 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant.provider;
import com.kingsrook.qbits.assistant.model.AssistantModel;
/*******************************************************************************
** Applies registry capabilities to both the engine and the adapter's request.
*******************************************************************************/
public class RegistryModelProvider implements AssistantModelProvider
{
private final AssistantModelProvider delegate;
private final AssistantModel model;
/*******************************************************************************
** One resolved model's overrides.
*******************************************************************************/
public RegistryModelProvider(AssistantModelProvider delegate, AssistantModel model)
{
this.delegate = delegate;
this.model = model;
}
/*******************************************************************************
** Adapter identity.
*******************************************************************************/
@Override
public String getName()
{
return (delegate.getName());
}
/*******************************************************************************
** Copy defaults without mutating a cached adapter capability object.
*******************************************************************************/
@Override
public ModelCapabilities getCapabilities(String modelId)
{
ModelCapabilities defaults = delegate.getCapabilities(modelId);
return (new ModelCapabilities()
.withTools(model.getSupportsTools() == null ? defaults.getTools() : model.getSupportsTools())
.withVision(model.getSupportsVision() == null ? defaults.getVision() : model.getSupportsVision())
.withStreaming(model.getSupportsStreaming() == null ? defaults.getStreaming() : model.getSupportsStreaming())
.withReasoningEffort(model.getSupportsReasoningEffort() == null ? defaults.getReasoningEffort() : model.getSupportsReasoningEffort())
.withMaxContextTokens(model.getContextWindowTokens() == null ? defaults.getMaxContextTokens() : model.getContextWindowTokens())
.withParallelToolCalls(defaults.getParallelToolCalls()).withThinking(defaults.getThinking()).withPromptCaching(defaults.getPromptCaching()));
}
/*******************************************************************************
** Pass effective capabilities and the output ceiling to the adapter.
*******************************************************************************/
@Override
public StreamHandle stream(ModelRequest request, ModelEventListener listener)
{
ModelRequest effective = request.copy().withCapabilities(getCapabilities(request.getModel()));
if(model.getMaxOutputTokens() != null)
{
effective.setMaxTokens(request.getMaxTokens() == null ? model.getMaxOutputTokens() : Math.min(request.getMaxTokens(), model.getMaxOutputTokens()));
}
return (delegate.stream(effective, listener));
}
/*******************************************************************************
** Preserve optional token counting.
*******************************************************************************/
@Override
public TokenCount countTokens(ModelRequest request)
{
return (delegate.countTokens(request));
}
}
@@ -0,0 +1,153 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant.provider;
import java.io.ByteArrayInputStream;
import java.nio.charset.StandardCharsets;
import java.security.KeyStore;
import java.security.cert.CertificateException;
import java.security.cert.CertificateFactory;
import java.security.cert.X509Certificate;
import java.util.Arrays;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.stream.Stream;
import javax.net.ssl.SSLContext;
import javax.net.ssl.TrustManager;
import javax.net.ssl.TrustManagerFactory;
import javax.net.ssl.X509TrustManager;
/*******************************************************************************
** Builds additive trust: JVM roots OR explicitly configured certificates.
** Endpoint identification remains the HTTP client's responsibility and is enabled.
*******************************************************************************/
public final class TrustedCaBundle implements X509TrustManager
{
private final X509TrustManager defaults;
private final X509TrustManager additional;
/*******************************************************************************
** Two independently validating trust managers, never a trust-all fallback.
*******************************************************************************/
TrustedCaBundle(X509TrustManager defaults, X509TrustManager additional)
{
this.defaults = defaults;
this.additional = additional;
}
/*******************************************************************************
** Accept only one or more complete PEM certificates; reject trailing garbage.
*******************************************************************************/
public static SSLContext parse(String pem)
{
try
{
KeyStore store = KeyStore.getInstance(KeyStore.getDefaultType());
store.load(null, null);
CertificateFactory factory = CertificateFactory.getInstance("X.509");
Matcher matcher = Pattern.compile("-----BEGIN CERTIFICATE-----[\\s\\S]*?-----END CERTIFICATE-----").matcher(pem == null ? "" : pem);
int end = 0;
int count = 0;
while(matcher.find())
{
if(!pem.substring(end, matcher.start()).isBlank())
{
throw (new CertificateException());
}
X509Certificate certificate = (X509Certificate) factory.generateCertificate(new ByteArrayInputStream(matcher.group().getBytes(StandardCharsets.US_ASCII)));
store.setCertificateEntry("ca-" + count++, certificate);
end = matcher.end();
}
if(count == 0 || !pem.substring(end).isBlank())
{
throw (new CertificateException());
}
SSLContext context = SSLContext.getInstance("TLS");
context.init(null, new TrustManager[] { new TrustedCaBundle(manager(null), manager(store)) }, null);
return (context);
}
catch(Exception e)
{
throw (new IllegalArgumentException("Invalid CA bundle: expected one or more PEM X.509 certificates."));
}
}
/*******************************************************************************
** A null store asks JSSE to load the JVM's configured default trust roots.
*******************************************************************************/
private static X509TrustManager manager(KeyStore store) throws Exception
{
TrustManagerFactory factory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
factory.init(store);
return (Arrays.stream(factory.getTrustManagers()).filter(X509TrustManager.class::isInstance).map(X509TrustManager.class::cast).findFirst().orElseThrow());
}
/*******************************************************************************
** Client validation uses the same additive trust rule.
*******************************************************************************/
@Override
public void checkClientTrusted(X509Certificate[] chain, String authType) throws CertificateException
{
try
{
defaults.checkClientTrusted(chain, authType);
}
catch(CertificateException e)
{
additional.checkClientTrusted(chain, authType);
}
}
/*******************************************************************************
** Try JVM roots first, then the named bundle, preserving chain validation.
*******************************************************************************/
@Override
public void checkServerTrusted(X509Certificate[] chain, String authType) throws CertificateException
{
try
{
defaults.checkServerTrusted(chain, authType);
}
catch(CertificateException e)
{
additional.checkServerTrusted(chain, authType);
}
}
/*******************************************************************************
** Both sets of issuers remain accepted.
*******************************************************************************/
@Override
public X509Certificate[] getAcceptedIssuers()
{
return (Stream.concat(Arrays.stream(defaults.getAcceptedIssuers()), Arrays.stream(additional.getAcceptedIssuers())).toArray(X509Certificate[]::new));
}
}
@@ -132,7 +132,10 @@ class AssistantRouteProviderTest extends BaseAssistantTest
List<Frame> frames = new ArrayList<>();
try(BufferedReader reader = new BufferedReader(new InputStreamReader(body, StandardCharsets.UTF_8)))
{
String id = null, event = null, data = null, line;
String id = null;
String event = null;
String data = null;
String line;
while((line = reader.readLine()) != null)
{
if(line.isEmpty())
@@ -379,7 +382,7 @@ class AssistantRouteProviderTest extends BaseAssistantTest
{
assertThat(AssistantApi.deriveTitle(" how many posts?\nmore lines")).isEqualTo("how many posts?");
String title = AssistantApi.deriveTitle("find every published blog post that is missing a meta description and an excerpt please");
assertThat(title).endsWith("\u2026").hasSizeLessThanOrEqualTo(AssistantApi.TITLE_MAX_CHARS);
assertThat(title).endsWith("…").hasSizeLessThanOrEqualTo(AssistantApi.TITLE_MAX_CHARS);
assertThat(AssistantApi.deriveTitle("x".repeat(100))).hasSize(AssistantApi.TITLE_MAX_CHARS);
}
@@ -287,7 +287,10 @@ class EngineComponentsTest extends BaseAssistantTest
});
CompletableStreamHandle cancelled = new CompletableStreamHandle();
cancelled.setOnCancel(() -> { throw (new IllegalStateException("closing")); });
cancelled.setOnCancel(() ->
{
throw (new IllegalStateException("closing"));
});
cancelled.cancel();
cancelled.cancel();
assertThat(cancelled.await(Duration.ofSeconds(1)).getStopReason()).isEqualTo(ModelResponse.STOP_CANCELLED);
@@ -327,7 +330,17 @@ class EngineComponentsTest extends BaseAssistantTest
Map<String, Object> bigFilter = ContextBuilder.sanitizePageContext(Map.of("tableName", "page", "filter", Map.of("criteria", "z".repeat(5000))));
assertThat(bigFilter).containsOnlyKeys("tableName");
assertThat(TurnEngine.map("a", 1, "b", null, "c", new Object() { public String toString() { return ("obj"); } })).containsEntry("a", 1).doesNotContainKey("b").containsEntry("c", "obj");
assertThat(TurnEngine.map("a", 1, "b", null, "c", new Object()
{
/*******************************************************************************
** A value rendered through toString.
*******************************************************************************/
@Override
public String toString()
{
return ("obj");
}
})).containsEntry("a", 1).doesNotContainKey("b").containsEntry("c", "obj");
}
}
@@ -0,0 +1,67 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant.provider;
import java.security.KeyStore;
import java.security.cert.X509Certificate;
import java.util.Arrays;
import java.util.Base64;
import javax.net.ssl.TrustManagerFactory;
import javax.net.ssl.X509TrustManager;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** Offline proof that adding a bundle does not replace the JVM's trust roots.
*******************************************************************************/
class TrustedCaBundleTest
{
/*******************************************************************************
** Use existing public JVM roots in memory; no network or fixture signing.
*******************************************************************************/
@Test
void keepsDefaultRoots() throws Exception
{
X509TrustManager defaults = manager(null);
X509Certificate[] roots = defaults.getAcceptedIssuers();
assertThat(roots.length).isGreaterThan(1);
KeyStore extra = KeyStore.getInstance(KeyStore.getDefaultType());
extra.load(null, null);
extra.setCertificateEntry("one-root", roots[0]);
TrustedCaBundle combined = new TrustedCaBundle(defaults, manager(extra));
assertThat(combined.getAcceptedIssuers()).contains(roots);
assertThat(combined.getAcceptedIssuers()).hasSize(roots.length + 1);
combined.checkServerTrusted(new X509Certificate[] { roots[1] }, roots[1].getPublicKey().getAlgorithm());
combined.checkClientTrusted(new X509Certificate[] { roots[1] }, roots[1].getPublicKey().getAlgorithm());
String pem = "-----BEGIN CERTIFICATE-----\n" + Base64.getMimeEncoder(64, new byte[] { '\n' }).encodeToString(roots[0].getEncoded()) + "\n-----END CERTIFICATE-----\n";
assertThat(TrustedCaBundle.parse(pem).getProtocol()).isEqualTo("TLS");
}
/*******************************************************************************
** Resolve a real PKIX manager from either the JVM or the small extra store.
*******************************************************************************/
private X509TrustManager manager(KeyStore store) throws Exception
{
TrustManagerFactory factory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
factory.init(store);
return (Arrays.stream(factory.getTrustManagers()).filter(X509TrustManager.class::isInstance).map(X509TrustManager.class::cast).findFirst().orElseThrow());
}
}
@@ -0,0 +1,432 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant.settings;
import java.math.BigDecimal;
import java.time.Duration;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.Set;
import com.kingsrook.qbits.assistant.AssistantQBitConfig;
import com.kingsrook.qbits.assistant.engine.ConnectionTest;
import com.kingsrook.qbits.assistant.engine.ModelDiscovery;
import com.kingsrook.qbits.assistant.engine.ModelPricing;
import com.kingsrook.qbits.assistant.engine.ProviderService;
import com.kingsrook.qbits.assistant.home.AssistantHomeData;
import com.kingsrook.qbits.assistant.metadata.AssistantMetaDataBuilder;
import com.kingsrook.qbits.assistant.model.AssistantModel;
import com.kingsrook.qbits.assistant.model.AssistantModelPrice;
import com.kingsrook.qbits.assistant.model.AssistantProvider;
import com.kingsrook.qbits.assistant.model.AssistantUsage;
import com.kingsrook.qbits.assistant.processes.DiscoverModelsProcess;
import com.kingsrook.qbits.assistant.processes.TestModelConnectionProcess;
import com.kingsrook.qbits.assistant.provider.AssistantModelProvider;
import com.kingsrook.qbits.assistant.provider.AssistantProviderType;
import com.kingsrook.qbits.assistant.provider.DiscoveredModel;
import com.kingsrook.qbits.assistant.provider.HealthResult;
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
import com.kingsrook.qbits.assistant.provider.ModelRequest;
import com.kingsrook.qbits.assistant.provider.ProviderSettings;
import com.kingsrook.qbits.assistant.testing.AssistantTestFixture;
import com.kingsrook.qbits.assistant.testing.BaseAssistantTest;
import com.kingsrook.qqq.backend.core.actions.processes.QProcessCallbackFactory;
import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
/*******************************************************************************
** Registry behavior through the memory-backed settings tables; no network.
*******************************************************************************/
class ProviderRegistryTest extends BaseAssistantTest
{
private final FakeType type = new FakeType();
private int creates;
private int discoveries;
private int healthChecks;
private boolean unhealthy;
private boolean discoveryError;
private int modelCount = 3;
private String secret;
/*******************************************************************************
** Test adapter with cloud and keyless local dialects.
*******************************************************************************/
class FakeType implements AssistantProviderType
{
/*******************************************************************************
** Provider contract.
*******************************************************************************/
@Override
public String getType()
{
return ("openai-compatible");
}
/*******************************************************************************
** Provider contract.
*******************************************************************************/
@Override
public String getLabel()
{
return ("Test registry");
}
/*******************************************************************************
** Provider contract.
*******************************************************************************/
@Override
public List<String> getModels()
{
return (List.of("fallback"));
}
/*******************************************************************************
** Provider contract.
*******************************************************************************/
@Override
public List<String> getKinds()
{
return (List.of("OLLAMA", "OPENAI", "AZURE_OPENAI", "OPENAI_COMPATIBLE"));
}
/*******************************************************************************
** Provider contract.
*******************************************************************************/
@Override
public String getDefaultApiKeyEnvName()
{
return (null);
}
/*******************************************************************************
** Provider contract.
*******************************************************************************/
@Override
public String getDefaultApiKeyEnvName(String kind)
{
return (kind.equals("OPENAI") ? "OPENAI_API_KEY" : (kind.equals("AZURE_OPENAI") ? "AZURE_OPENAI_API_KEY" : null));
}
/*******************************************************************************
** Provider contract.
*******************************************************************************/
@Override
public String getDefaultBaseUrl()
{
return (null);
}
/*******************************************************************************
** Provider contract.
*******************************************************************************/
@Override
public String getDefaultBaseUrl(String kind)
{
return (kind.equals("OLLAMA") ? "http://localhost:11434/v1" : (kind.equals("OPENAI") ? "https://api.openai.com/v1" : null));
}
/*******************************************************************************
** Provider contract.
*******************************************************************************/
@Override
public String getDefaultLocation(String kind)
{
return (kind.equals("OLLAMA") ? "LOCAL" : "CLOUD");
}
/*******************************************************************************
** Provider contract.
*******************************************************************************/
@Override
public AssistantModelProvider create(String key, String url)
{
creates++;
return (provider);
}
/*******************************************************************************
** Provider contract.
*******************************************************************************/
@Override
public List<DiscoveredModel> discoverModels(ProviderSettings settings) throws ModelProviderException
{
discoveries++;
if(discoveryError)
{
throw (new ModelProviderException(ModelProviderException.Kind.NETWORK, null, "Discovery unreachable"));
}
List<DiscoveredModel> models = new ArrayList<>();
for(int i = 0; i < modelCount; i++)
{
models.add(new DiscoveredModel("model-" + i, "Model " + i, 8192, "test"));
}
return (models);
}
/*******************************************************************************
** Provider contract.
*******************************************************************************/
@Override
public HealthResult checkHealth(ProviderSettings settings)
{
healthChecks++;
return (new HealthResult(!unhealthy, 1L, unhealthy ? "Unreachable" : "Healthy"));
}
}
/*******************************************************************************
** Configure explicit reference and URL allowlists.
*******************************************************************************/
@Override
protected void customizeConfig(AssistantQBitConfig config)
{
config.withProviderType(type).withSecretResolver(n -> secret).withAllowedApiKeyEnvNames(Set.of("LOCAL_PROXY_KEY"))
.withAllowedBaseUrls(Set.of("https://approved.example"));
}
/*******************************************************************************
** Create settings under the admin's own session.
*******************************************************************************/
private AssistantProvider row() throws Exception
{
QContext.init(qInstance, AssistantTestFixture.sessionFor(AssistantTestFixture.ADMIN));
return (config.getRuntime().getStore().insert(new AssistantProvider().withName("Local").withProviderType(type.getType()).withKind("OLLAMA").withEnabled(true)));
}
/*******************************************************************************
** Keyless rows, kind validation, URL guards, timeouts and client fingerprints.
*******************************************************************************/
@Test
void settingsAndCache() throws Exception
{
AssistantProvider row = row();
var service = config.getRuntime().getProviderService();
assertThat(row.getLocation()).isEqualTo("LOCAL");
assertThat(row.getHipaaEligible()).isFalse();
assertThat(row.getConnectTimeoutSeconds()).isEqualTo(10);
assertThat(row.getRequestTimeoutSeconds()).isEqualTo(120);
assertThat(service.keyStatus(row)).isEqualTo("Not needed (local, no key)");
assertThat(service.forRow(row, "legacy").model()).isEqualTo("legacy");
service.forRow(row, "legacy");
assertThat(creates).isEqualTo(1);
row.withApiVersion("v1");
service.forRow(row, "legacy");
row.withRegion("r1");
service.forRow(row, "legacy");
row.withConnectTimeoutSeconds(11);
service.forRow(row, "legacy");
row.withRequestTimeoutSeconds(121);
service.forRow(row, "legacy");
assertThat(creates).isEqualTo(5);
row.withApiKeyEnvName("LOCAL_PROXY_KEY");
assertThatThrownBy(() -> service.settings(row)).hasMessageContaining("not set");
secret = "private-test-key";
assertThat(service.settings(row).toString()).doesNotContain(secret);
service.forRow(row, "legacy");
secret = "rotated-test-key";
service.forRow(row, "legacy");
assertThat(creates).isEqualTo(7);
assertThat(service.validate(row.withKind("ANTHROPIC"))).anyMatch(e -> e.contains("kind"));
row.withKind("AZURE_OPENAI").withApiKeyEnvName(null);
assertThat(service.validate(row)).anyMatch(e -> e.contains("base URL"));
assertThat(service.validate(row.withBaseUrl("https://unapproved.example"))).anyMatch(e -> e.contains("not allowed"));
assertThat(service.validate(row.withBaseUrl("https://approved.example"))).isEmpty();
assertThat(config.getEffectiveAllowedApiKeyEnvNames()).contains("OPENAI_API_KEY", "AZURE_OPENAI_API_KEY").doesNotContainNull();
for(int invalid : List.of(0, 601))
{
assertThat(service.validate(row.withConnectTimeoutSeconds(invalid))).anyMatch(e -> e.contains("600"));
row.withConnectTimeoutSeconds(10);
assertThat(service.validate(row.withRequestTimeoutSeconds(invalid))).anyMatch(e -> e.contains("600"));
}
row.withRequestTimeoutSeconds(120).withApiKeyEnvName("DATABASE_PASSWORD");
assertThat(service.validate(row)).anyMatch(e -> e.contains("not allowed"));
row.withApiKeyEnvName("not a name");
assertThat(service.validate(row)).anyMatch(e -> e.contains("NAME"));
}
/*******************************************************************************
** Registry selection, invalidation, price precedence and call capabilities.
*******************************************************************************/
@Test
void modelsPricesAndCapabilities() throws Exception
{
AssistantProvider row = row();
var store = config.getRuntime().getStore();
var service = config.getRuntime().getProviderService();
AssistantModel disabled = store.insert(new AssistantModel().withProviderId(row.getId()).withModelId("disabled").withEnabled(false));
assertThatThrownBy(() -> service.forRow(row, "fallback")).hasMessageContaining("All models");
AssistantModel model = store.insert(new AssistantModel().withProviderId(row.getId()).withModelId("active").withEnabled(true)
.withSupportsTools(false).withSupportsVision(true).withSupportsStreaming(false).withSupportsReasoningEffort(true).withContextWindowTokens(4096).withMaxOutputTokens(16));
var active = service.forRow(row, "fallback");
assertThat(active.model()).isEqualTo("active");
var caps = active.provider().getCapabilities("active");
assertThat(caps.getTools()).isFalse();
assertThat(caps.getVision()).isTrue();
assertThat(caps.getStreaming()).isFalse();
assertThat(caps.getReasoningEffort()).isTrue();
assertThat(caps.getMaxContextTokens()).isEqualTo(4096);
provider.thenText("OK");
active.provider().stream(new ModelRequest().withModel("active").withMaxTokens(32), t -> {}).await(Duration.ofSeconds(2));
assertThat(provider.getRequests().getLast().getMaxTokens()).isEqualTo(16);
assertThat(provider.getRequests().getLast().getCapabilities().getStreaming()).isFalse();
assertThat(active.provider().countTokens(new ModelRequest())).isNull();
assertThatThrownBy(() -> service.forRow(row.withModel("disabled"), "fallback")).hasMessageContaining("disabled");
row.withModel(null);
assertThat(service.pricing(active, "active").getInputUsdPerMTok()).isZero();
config.getPricing().put("active", new ModelPricing().withInputUsdPerMTok(9.0));
assertThat(service.pricing(active, "active").getInputUsdPerMTok()).isEqualTo(9.0);
store.insert(new AssistantModelPrice().withProviderType(type.getType()).withModel("active").withInputUsdPerMTok(new BigDecimal("5")).withOutputUsdPerMTok(new BigDecimal("6")));
assertThat(service.pricing(active, "active").getInputUsdPerMTok()).isEqualTo(5.0);
store.update(model.withInputUsdPerMTok(new BigDecimal("2")).withOutputUsdPerMTok(new BigDecimal("3")));
assertThat(service.pricing(active, "active").getInputUsdPerMTok()).isEqualTo(2.0);
store.update(row.withModel("active"));
assertThat(service.forRow(store.getProvider(row.getId()), "fallback").model()).isEqualTo("active");
row.withModel(null);
store.update(disabled.withEnabled(true));
assertThat(service.forRow(row, "fallback").model()).isEqualTo("disabled");
}
/*******************************************************************************
** Discovery preserves edits, disables large cloud catalogs and records errors.
*******************************************************************************/
@Test
void discovery() throws Exception
{
AssistantProvider row = row();
var runtime = config.getRuntime();
var transform = new DiscoverModelsProcess.Transform();
var input = new RunBackendStepInput();
input.setRecords(List.of(row.toQRecord()));
var output = new RunBackendStepOutput();
transform.runOnePage(input, output);
assertThat(discoveries).isZero();
assertThat(creates).isZero();
assertThat(transform.getProcessSummary(output, false)).isNotEmpty();
runProcess(DiscoverModelsProcess.NAME, row.getId());
assertThat(discoveries).isEqualTo(1);
assertThat(runtime.getStore().listModels()).hasSize(3).allSatisfy(m ->
{
assertThat(m.getEnabled()).isTrue();
assertThat(m.getSource()).isEqualTo("DISCOVERED");
assertThat(m.getInputUsdPerMTok()).isEqualByComparingTo(BigDecimal.ZERO);
});
AssistantModel edited = runtime.getStore().listModels().getFirst().withDisplayName("Admin name").withEnabled(false).withInputUsdPerMTok(BigDecimal.TEN);
runtime.getStore().update(edited);
assertThat(ModelDiscovery.run(runtime, row).message()).isEqualTo("0 models added, 3 already present");
assertThat(runtime.getStore().listModels().getFirst().getDisplayName()).isEqualTo("Admin name");
modelCount = 23;
secret = "test-key";
row = runtime.getStore().insert(new AssistantProvider().withName("Cloud").withProviderType(type.getType()).withKind("OPENAI"));
assertThat(ModelDiscovery.run(runtime, row).ok()).isTrue();
assertThat(runtime.getProviderService().models(row.getId())).hasSize(23).allMatch(m -> !m.getEnabled());
discoveryError = true;
assertThat(ModelDiscovery.run(runtime, row).ok()).isFalse();
assertThat(runtime.getStore().getProvider(row.getId()).getLastDiscoveryError()).contains("Discovery unreachable");
}
/*******************************************************************************
** Model tests use the requested id, account usage and persist health failures.
*******************************************************************************/
@Test
void modelTestAndMetadata() throws Exception
{
AssistantProvider row = row();
var runtime = config.getRuntime();
AssistantModel model = runtime.getStore().insert(new AssistantModel().withProviderId(row.getId()).withModelId("disabled-test").withEnabled(false));
provider.thenText("OK");
runProcess(TestModelConnectionProcess.NAME, model.getId());
assertThat(healthChecks).isEqualTo(1);
assertThat(provider.getRequests().getLast().getModel()).isEqualTo("disabled-test");
assertThat(provider.getRequests().getLast().getMaxTokens()).isEqualTo(32);
assertThat(runtime.getStore().listModels().getFirst().getLastTestStatus()).isEqualTo("OK");
assertThat(runtime.getStore().queryRecords(AssistantUsage.TABLE_NAME, new QQueryFilter(), 10)).hasSize(1);
unhealthy = true;
assertThat(ConnectionTest.runModel(runtime, model, AssistantTestFixture.ADMIN).ok()).isFalse();
assertThat(runtime.getStore().listModels().getFirst().getLastTestError()).contains("Unreachable");
assertThat(provider.getRequests()).hasSize(1);
assertThat(AssistantMetaDataBuilder.NAV_TABLES).doesNotContain(AssistantModel.TABLE_NAME);
assertThat(qInstance.getTable(AssistantProvider.TABLE_NAME).getSections()).anyMatch(s -> s.getWidgetName() != null && s.getWidgetName().contains("model"));
assertThat(qInstance.getTable(AssistantModel.TABLE_NAME).getField("providerId").getPossibleValueSourceName()).isEqualTo(AssistantProvider.TABLE_NAME);
assertThat(qInstance.getTable(AssistantModel.TABLE_NAME).getField("lastTestError").getIsEditable()).isFalse();
assertThat(qInstance.getTable(AssistantProvider.TABLE_NAME).getField("lastDiscoveryAt").getIsEditable()).isFalse();
assertThat(AssistantHomeData.load(runtime).getProviders().getFirst().enabledModels()).isZero();
assertThat(AssistantHomeData.load(runtime).getProviders().getFirst().keyPresent()).isTrue();
}
/*******************************************************************************
** Standard streamed ETL with Review and Result enabled.
*******************************************************************************/
private void runProcess(String process, Integer id) throws Exception
{
RunProcessInput input = new RunProcessInput();
input.setProcessName(process);
input.setCallback(QProcessCallbackFactory.forPrimaryKey("id", id));
input.setFrontendStepBehavior(RunProcessInput.FrontendStepBehavior.SKIP);
new RunProcessAction().execute(input);
}
}