feat: provider and model registry, OpenAI-compatible adapter, discovery and health (P-a) #22
No files matched your search
@@ -19,3 +19,6 @@ dependency-reduced-pom.xml
|
||||
/.env.local
|
||||
/.cache/
|
||||
.claude/settings.local.json
|
||||
|
||||
# Runtime output from every provider module
|
||||
log/
|
||||
@@ -42,6 +42,7 @@ Always use the wrapper: Maven 3.10 cannot read the parent POM. The committed def
|
||||
- `processes/`: `AssistantRetentionStep` and `TestConnectionProcess` (record action on `assistant_provider`).
|
||||
- `engine/ProviderService` (active provider from `assistant_provider` rows, key references, prices), `UsageRecorder` (one `assistant_usage` row per model call), `ConnectionTest`.
|
||||
- `qbit-assistant-provider-anthropic`: `AnthropicModelProvider` and `AnthropicMapping` (anthropic-java streaming).
|
||||
- `qbit-assistant-provider-openai`: OpenAI-compatible Chat Completions, model discovery and health using JDK HTTP and Jackson.
|
||||
|
||||
## Invariants (do not break)
|
||||
|
||||
|
||||
@@ -4,6 +4,10 @@ All notable changes to this project are documented here. The format follows Keep
|
||||
|
||||
## [0.1.0] - Unreleased
|
||||
|
||||
- Add named, server-only CA bundles for HTTPS providers, additive JVM trust and sanitized TLS diagnostics; tolerate repeated/missing streamed tool identities and invalid arguments, with a golden tool round-trip test.
|
||||
- Add the provider/model registry, per-model capabilities and prices, keyless local providers, discovery and health, and provider/model connection tests.
|
||||
- Add the optional JDK HTTP OpenAI-compatible adapter for OpenAI, Azure, OpenRouter, Ollama, LM Studio, vLLM and llama.cpp, with strict-schema safeguards and localhost contract tests.
|
||||
|
||||
Phase 0/1 of the QQQ Assistant: read-only Q&A and navigation. Requires QQQ 4.1 with:
|
||||
- the AI tool kernel (`qqq-backend-core` `actions.ai`, QRun-IO/qqq#1078)
|
||||
- the developer-panel contract (QRun-IO/qqq#1076).
|
||||
|
||||
@@ -42,6 +42,7 @@ Always use the wrapper: Maven 3.10 cannot read the parent POM. The committed def
|
||||
- `processes/`: `AssistantRetentionStep` and `TestConnectionProcess` (record action on `assistant_provider`).
|
||||
- `engine/ProviderService` (active provider from `assistant_provider` rows, key references, prices), `UsageRecorder` (one `assistant_usage` row per model call), `ConnectionTest`.
|
||||
- `qbit-assistant-provider-anthropic`: `AnthropicModelProvider` and `AnthropicMapping` (anthropic-java streaming).
|
||||
- `qbit-assistant-provider-openai`: OpenAI-compatible Chat Completions, model discovery and health using JDK HTTP and Jackson.
|
||||
|
||||
## Invariants (do not break)
|
||||
|
||||
|
||||
@@ -253,6 +253,64 @@ The **live** evals run the same golden tasks against Claude. They are skipped un
|
||||
ANTHROPIC_API_KEY=... ./mvnw verify -Dassistant.live=true -Dtest.excludedGroups= -Dgroups=live -Djacoco.skip=true -pl qbit-assistant-provider-anthropic -am
|
||||
```
|
||||
|
||||
## Providers
|
||||
|
||||
Add `com.kingsrook.qbits:qbit-assistant-provider-openai` at the same version as
|
||||
the QBit and register the adapter alongside any Anthropic adapter:
|
||||
|
||||
```java
|
||||
config.withProviderType(new OpenAiCompatibleProviderType());
|
||||
```
|
||||
|
||||
`OpenAiCompatibleProviderType` is in
|
||||
`com.kingsrook.qbits.assistant.provider.openai`. Its installed type id is
|
||||
`openai-compatible`; provider rows choose OPENAI, AZURE_OPENAI, OPENROUTER,
|
||||
OLLAMA, LM_STUDIO, VLLM, LLAMA_CPP or OPENAI_COMPATIBLE as their kind.
|
||||
|
||||
- Local kinds use their localhost endpoint by default and need no key. Leave
|
||||
`apiKeyEnvName` empty. An auth proxy can use an explicitly allowed key env name.
|
||||
- Cloud defaults use `OPENAI_API_KEY`, `AZURE_OPENAI_API_KEY` or
|
||||
`OPENROUTER_API_KEY`. Values stay in the host's secret store.
|
||||
- An override must match an entry in `config.withAllowedBaseUrls(Set.of(...))`.
|
||||
Azure and OPENAI_COMPATIBLE have no default URL and always require an entry.
|
||||
Redirects are not followed. Azure uses the deployment model id and defaults
|
||||
`apiVersion` to `2024-10-21`.
|
||||
- **Discover models** adds a provider's Models child list; existing edits survive.
|
||||
Large cloud catalogs (over 20 models) start disabled. **Test connection** is
|
||||
available on both providers and models and records a TEST usage row.
|
||||
- A selected registered model must be enabled. With no model selected, the first
|
||||
enabled registry model wins; providers with no model rows retain legacy selection.
|
||||
Registry prices override type/model prices, then configured prices. Unpriced
|
||||
local models cost zero. Capabilities can be adjusted per registered model.
|
||||
|
||||
For Ollama behind a TLS proxy with a private CA and bearer authentication:
|
||||
|
||||
```java
|
||||
config.withProviderType(new OpenAiCompatibleProviderType())
|
||||
.withAllowedBaseUrls(Set.of("https://llm.example.lan:8443/v1"))
|
||||
.withAllowedApiKeyEnvNames(Set.of("LOCAL_LLM_API_KEY"))
|
||||
.withTrustedCaBundleFile("local-llm", Path.of("/etc/app/llm-root-ca.pem"));
|
||||
```
|
||||
|
||||
Set `LOCAL_LLM_API_KEY` in the server environment from your secret store. In the
|
||||
provider record choose kind `OLLAMA`, base URL `https://llm.example.lan:8443/v1`,
|
||||
`apiKeyEnvName = LOCAL_LLM_API_KEY`, and `tlsCaBundle = local-llm`. Only the names
|
||||
are stored. `withTrustedCaBundle("local-llm", pem)` also accepts one or more PEM
|
||||
certificates directly at startup. Invalid PEM fails configuration immediately.
|
||||
The named bundle adds to the JVM roots; hostname verification stays enabled.
|
||||
Leave `tlsCaBundle` empty to use JVM trust alone. Restart after changing a CA file.
|
||||
|
||||
Default tests use localhost mocks. To exercise an existing development server,
|
||||
set `ASSISTANT_LIVE_OPENAI_BASE_URL` and `ASSISTANT_LIVE_OPENAI_MODEL`, optionally
|
||||
`ASSISTANT_LIVE_OPENAI_KIND` (default OLLAMA), `ASSISTANT_LIVE_OPENAI_KEY_ENV`
|
||||
(the variable name containing the key), and `ASSISTANT_LIVE_OPENAI_CA_FILE`
|
||||
(a PEM bundle path). Unset or unreachable endpoints, including TLS failures, skip
|
||||
the live test. Run:
|
||||
|
||||
```sh
|
||||
./mvnw verify -pl qbit-assistant-provider-openai -am -Dtest.excludedGroups= -Dgroups=live -Djacoco.skip=true
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
Apache 2.0.
|
||||
@@ -118,3 +118,70 @@ Database (see `docs/schema-postgres.sql`):
|
||||
Runtime: the tools call this server's own `/qqq/v1` on `127.0.0.1` and the port the first assistant request arrived on. If the server is not reachable that way (a different bind address, a TLS-only listener), set `withUiApiBaseUrl(...)`. Grant `assistant.admin` holders the new tables through the qbit's existing admin permission (no new permission names).
|
||||
|
||||
Behavior: nothing is writable until an admin adds a READ_WRITE data access row; before that the assistant reads exactly what it did before, now through the UI API.
|
||||
|
||||
## P-a: provider and model registry
|
||||
|
||||
Before starting the new version, database-backed hosts add the provider columns
|
||||
and model sub-table below. Memory-backed hosts need no migration. Existing rows
|
||||
derive kind and location from the adapter and keep their model behavior until
|
||||
models are registered. Keys remain environment-variable references.
|
||||
|
||||
```sql
|
||||
ALTER TABLE assistant_provider
|
||||
ADD COLUMN kind VARCHAR(30),
|
||||
ADD COLUMN tls_ca_bundle VARCHAR(100),
|
||||
ADD COLUMN location VARCHAR(10),
|
||||
ADD COLUMN hipaa_eligible BOOLEAN DEFAULT FALSE,
|
||||
ADD COLUMN region VARCHAR(50),
|
||||
ADD COLUMN api_version VARCHAR(30),
|
||||
ADD COLUMN connect_timeout_seconds INTEGER DEFAULT 10 CHECK (connect_timeout_seconds BETWEEN 1 AND 600),
|
||||
ADD COLUMN request_timeout_seconds INTEGER DEFAULT 120 CHECK (request_timeout_seconds BETWEEN 1 AND 600),
|
||||
ADD COLUMN last_discovery_at TIMESTAMP WITH TIME ZONE,
|
||||
ADD COLUMN last_discovery_status VARCHAR(16),
|
||||
ADD COLUMN last_discovery_error VARCHAR(500);
|
||||
ALTER TABLE assistant_provider ALTER COLUMN model TYPE VARCHAR(200);
|
||||
ALTER TABLE assistant_provider ALTER COLUMN last_test_model TYPE VARCHAR(200);
|
||||
ALTER TABLE assistant_usage ALTER COLUMN model TYPE VARCHAR(200);
|
||||
ALTER TABLE assistant_message ALTER COLUMN model TYPE VARCHAR(200);
|
||||
ALTER TABLE assistant_turn ALTER COLUMN model TYPE VARCHAR(200);
|
||||
|
||||
CREATE TABLE assistant_model
|
||||
(
|
||||
id SERIAL PRIMARY KEY,
|
||||
provider_id INTEGER NOT NULL REFERENCES assistant_provider(id),
|
||||
model_id VARCHAR(200) NOT NULL,
|
||||
display_name VARCHAR(200),
|
||||
enabled BOOLEAN,
|
||||
supports_tools BOOLEAN,
|
||||
supports_vision BOOLEAN,
|
||||
supports_streaming BOOLEAN,
|
||||
supports_reasoning_effort BOOLEAN,
|
||||
context_window_tokens INTEGER,
|
||||
max_output_tokens INTEGER,
|
||||
input_usd_per_m_tok NUMERIC(12, 4),
|
||||
output_usd_per_m_tok NUMERIC(12, 4),
|
||||
source VARCHAR(16),
|
||||
discovered_at TIMESTAMP WITH TIME ZONE,
|
||||
last_test_at TIMESTAMP WITH TIME ZONE,
|
||||
last_test_status VARCHAR(16),
|
||||
last_test_latency_ms INTEGER,
|
||||
last_test_error VARCHAR(500),
|
||||
create_date TIMESTAMP WITH TIME ZONE,
|
||||
modify_date TIMESTAMP WITH TIME ZONE,
|
||||
UNIQUE (provider_id, model_id)
|
||||
);
|
||||
```
|
||||
|
||||
Register the optional `qbit-assistant-provider-openai` module and
|
||||
`OpenAiCompatibleProviderType` to offer its kinds. Azure and generic endpoints
|
||||
require an explicit `allowedBaseUrls` entry; local defaults may be keyless.
|
||||
Discover models preserves existing rows. Cloud catalogs larger than 20 are added
|
||||
disabled; enable the models to offer before leaving the provider's model empty.
|
||||
HIPAA eligibility records an administrator's fact; P-a does not implement PHI routing.
|
||||
|
||||
Custom CA bundles are server configuration, never database content. Register them
|
||||
with `withTrustedCaBundle(name, pem)` or `withTrustedCaBundleFile(name, path)` at
|
||||
startup, then set the HTTPS provider row's `tlsCaBundle` to that name. Empty
|
||||
references retain JVM default trust. The bundle adds trust without replacing JVM
|
||||
roots; hostname verification remains enabled. If the P-a columns were already
|
||||
applied, add only `tls_ca_bundle VARCHAR(100)` to `assistant_provider`.
|
||||
@@ -309,6 +309,30 @@ AI exposure NONE is not enough, because the assistant acts as the user and an ad
|
||||
- **scheduled assistant tasks**: recurring prompts that run as a designated user (with that user's stored credentials through the UI API, never a system session) and post results as notifications or records.
|
||||
6. Everything goes through the `/qqq/v1` UI API, as the user, with the guarantees above.
|
||||
|
||||
### P-a as built
|
||||
|
||||
- Per-provider `tlsCaBundle` references startup-parsed, transient server CA bundles. OpenAI-compatible chat, health and discovery trust JVM roots plus the bundle, retain hostname verification, and report sanitized NETWORK errors with a trust hint.
|
||||
- `assistant_provider` records kind, location, HIPAA eligibility, region, API
|
||||
version, timeouts and discovery status. `assistant_model` is its admin-only
|
||||
child list: enablement, capabilities, prices and test status, unique per provider/model.
|
||||
- `ProviderSettings`, discovery and health extend the provider SPI through default
|
||||
methods. Legacy `create(apiKey, baseUrl)` implementations keep working.
|
||||
- The OpenAI-compatible module uses JDK HTTP and Jackson, Chat Completions with
|
||||
SSE or single JSON replies, Azure deployment URLs and api-key auth, and keyless
|
||||
local endpoints. Keys and URL overrides pass the existing allowlists first.
|
||||
- Strict tools require explicit opt-in, OpenAI or Azure, and recursively strict
|
||||
object schemas. A strict/schema/grammar 400 signals the existing one-retry fallback.
|
||||
- Discovery only adds missing models; it preserves edits and disables catalogs
|
||||
larger than 20 for cloud providers. Health uses a short models request. Provider
|
||||
tests probe health then complete; model tests can probe a disabled model before
|
||||
enabling it. Completion tests record TEST usage.
|
||||
- Registry prices and capabilities override adapter/type defaults. Empty provider
|
||||
model selects the first enabled registry model; an explicitly disabled selection
|
||||
is a setup error. Rows without models retain the prior behavior.
|
||||
- Localhost contract tests cover dialects, fragmented parallel tool calls, usage,
|
||||
errors, timeouts, cancellation, discovery and a golden tool round trip. The live
|
||||
smoke is environment-driven and excluded by default.
|
||||
|
||||
### Phases (each a green PR)
|
||||
| | Scope | Depends on |
|
||||
|---|---|---|
|
||||
|
||||
@@ -30,7 +30,7 @@ CREATE TABLE assistant_message
|
||||
status VARCHAR(16),
|
||||
turn_id INTEGER,
|
||||
seq INTEGER,
|
||||
model VARCHAR(64),
|
||||
model VARCHAR(200),
|
||||
parts_json TEXT,
|
||||
create_date TIMESTAMP WITH TIME ZONE,
|
||||
modify_date TIMESTAMP WITH TIME ZONE
|
||||
@@ -49,7 +49,7 @@ CREATE TABLE assistant_turn
|
||||
stop_reason VARCHAR(64),
|
||||
error_code VARCHAR(32),
|
||||
cancel_reason VARCHAR(64),
|
||||
model VARCHAR(64),
|
||||
model VARCHAR(200),
|
||||
started_at TIMESTAMP WITH TIME ZONE,
|
||||
ended_at TIMESTAMP WITH TIME ZONE,
|
||||
tool_call_count INTEGER,
|
||||
@@ -173,10 +173,21 @@ CREATE TABLE assistant_provider
|
||||
id SERIAL PRIMARY KEY,
|
||||
name VARCHAR(100) NOT NULL UNIQUE,
|
||||
provider_type VARCHAR(50) NOT NULL,
|
||||
model VARCHAR(100),
|
||||
kind VARCHAR(30),
|
||||
location VARCHAR(10),
|
||||
hipaa_eligible BOOLEAN DEFAULT FALSE,
|
||||
region VARCHAR(50),
|
||||
api_version VARCHAR(30),
|
||||
connect_timeout_seconds INTEGER DEFAULT 10 CHECK (connect_timeout_seconds BETWEEN 1 AND 600),
|
||||
request_timeout_seconds INTEGER DEFAULT 120 CHECK (request_timeout_seconds BETWEEN 1 AND 600),
|
||||
last_discovery_at TIMESTAMP WITH TIME ZONE,
|
||||
last_discovery_status VARCHAR(16),
|
||||
last_discovery_error VARCHAR(500),
|
||||
model VARCHAR(200),
|
||||
enabled BOOLEAN,
|
||||
is_default BOOLEAN,
|
||||
base_url VARCHAR(255),
|
||||
tls_ca_bundle VARCHAR(100),
|
||||
api_key_env_name VARCHAR(100),
|
||||
api_key_status VARCHAR(200),
|
||||
monthly_cost_cap_usd NUMERIC(12, 2),
|
||||
@@ -185,12 +196,38 @@ CREATE TABLE assistant_provider
|
||||
last_test_at TIMESTAMP WITH TIME ZONE,
|
||||
last_test_status VARCHAR(16),
|
||||
last_test_latency_ms INTEGER,
|
||||
last_test_model VARCHAR(100),
|
||||
last_test_model VARCHAR(200),
|
||||
last_test_error VARCHAR(500),
|
||||
create_date TIMESTAMP WITH TIME ZONE,
|
||||
modify_date TIMESTAMP WITH TIME ZONE
|
||||
);
|
||||
|
||||
CREATE TABLE assistant_model
|
||||
(
|
||||
id SERIAL PRIMARY KEY,
|
||||
provider_id INTEGER NOT NULL REFERENCES assistant_provider(id),
|
||||
model_id VARCHAR(200) NOT NULL,
|
||||
display_name VARCHAR(200),
|
||||
enabled BOOLEAN,
|
||||
supports_tools BOOLEAN,
|
||||
supports_vision BOOLEAN,
|
||||
supports_streaming BOOLEAN,
|
||||
supports_reasoning_effort BOOLEAN,
|
||||
context_window_tokens INTEGER,
|
||||
max_output_tokens INTEGER,
|
||||
input_usd_per_m_tok NUMERIC(12, 4),
|
||||
output_usd_per_m_tok NUMERIC(12, 4),
|
||||
source VARCHAR(16),
|
||||
discovered_at TIMESTAMP WITH TIME ZONE,
|
||||
last_test_at TIMESTAMP WITH TIME ZONE,
|
||||
last_test_status VARCHAR(16),
|
||||
last_test_latency_ms INTEGER,
|
||||
last_test_error VARCHAR(500),
|
||||
create_date TIMESTAMP WITH TIME ZONE,
|
||||
modify_date TIMESTAMP WITH TIME ZONE,
|
||||
UNIQUE (provider_id, model_id)
|
||||
);
|
||||
|
||||
CREATE TABLE assistant_model_price
|
||||
(
|
||||
id SERIAL PRIMARY KEY,
|
||||
@@ -215,7 +252,7 @@ CREATE TABLE assistant_usage
|
||||
turn_id INTEGER,
|
||||
provider_id INTEGER,
|
||||
provider_type VARCHAR(50),
|
||||
model VARCHAR(100),
|
||||
model VARCHAR(200),
|
||||
purpose VARCHAR(16),
|
||||
status VARCHAR(16),
|
||||
error_kind VARCHAR(32),
|
||||
|
||||
@@ -23,3 +23,15 @@ Light and dark, 390 and 1440 px:
|
||||
- `panel-persona-*`: the panel with the configured persona. This is the Next UI build of qqq-frontend-next#94; the demo's admin matches the permission-scoped "Ada (analytics)" profile.
|
||||
- `panel-pending-*`: the assistant proposed `update_record` on page 12, which is READ_WRITE. The card shows the change with Save and Cancel; nothing is saved yet.
|
||||
- `panel-confirmed-*`: after Save, the write ran once through `PATCH /qqq/v1/table/page/12` with the user's session. The card says Done, and the record screen behind the panel shows the new value.
|
||||
|
||||
Planned P-a provider registry additions, taken against `OpenAiDemoServer` in the
|
||||
OpenAI provider's test sources with the Next UI **1.0.0-RC.18** jar. See that
|
||||
[demo's run instructions](../../qbit-assistant-provider-openai/src/test/java/com/kingsrook/qbits/assistant/provider/openai/demo/README.md).
|
||||
Discovery and connection-test status, TEST usage and the assistant turn are real
|
||||
executions against the localhost mock; replies/token counts are synthetic, not
|
||||
live GPU measurements. No screenshots in this group have been added yet.
|
||||
|
||||
Light and dark, 390 and 1440 px:
|
||||
- `p-provider-local-*`: Local GPU (Ollama), keyless LOCAL settings, discovery/test results and discovered models.
|
||||
- `p-model-*`: a discovered model's capabilities, compute prices and connection-test result.
|
||||
- `p-home-providers-*`: the home's provider summary and usage from the executed demo calls, including disabled OpenRouter with its key Not set.
|
||||
|
After Width: | Height: | Size: 95 KiB |
|
After Width: | Height: | Size: 38 KiB |
|
After Width: | Height: | Size: 97 KiB |
|
After Width: | Height: | Size: 38 KiB |
|
After Width: | Height: | Size: 76 KiB |
|
After Width: | Height: | Size: 45 KiB |
|
After Width: | Height: | Size: 77 KiB |
|
After Width: | Height: | Size: 45 KiB |
|
After Width: | Height: | Size: 98 KiB |
|
After Width: | Height: | Size: 44 KiB |
|
After Width: | Height: | Size: 100 KiB |
|
After Width: | Height: | Size: 44 KiB |
|
After Width: | Height: | Size: 86 KiB |
|
After Width: | Height: | Size: 43 KiB |
|
After Width: | Height: | Size: 86 KiB |
|
After Width: | Height: | Size: 42 KiB |
@@ -52,6 +52,7 @@
|
||||
<modules>
|
||||
<module>qbit-assistant</module>
|
||||
<module>qbit-assistant-provider-anthropic</module>
|
||||
<module>qbit-assistant-provider-openai</module>
|
||||
</modules>
|
||||
|
||||
<licenses>
|
||||
@@ -101,6 +102,11 @@
|
||||
<version>${revision}</version>
|
||||
<type>test-jar</type>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>com.kingsrook.qbits</groupId>
|
||||
<artifactId>qbit-assistant-provider-openai</artifactId>
|
||||
<version>${revision}</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>com.anthropic</groupId>
|
||||
<artifactId>anthropic-java</artifactId>
|
||||
|
||||
@@ -0,0 +1,128 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!--
|
||||
~ Copyright 2026 Kingsrook, LLC
|
||||
~
|
||||
~ Licensed under the Apache License, Version 2.0 (the "License");
|
||||
~ you may not use this file except in compliance with the License.
|
||||
~ You may obtain a copy of the License at
|
||||
~
|
||||
~ http://www.apache.org/licenses/LICENSE-2.0
|
||||
~
|
||||
~ Unless required by applicable law or agreed to in writing, software
|
||||
~ distributed under the License is distributed on an "AS IS" BASIS,
|
||||
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
~ See the License for the specific language governing permissions and
|
||||
~ limitations under the License.
|
||||
-->
|
||||
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
<parent>
|
||||
<groupId>com.kingsrook.qbits</groupId>
|
||||
<artifactId>qbit-assistant-parent</artifactId>
|
||||
<version>${revision}</version>
|
||||
</parent>
|
||||
|
||||
<artifactId>qbit-assistant-provider-openai</artifactId>
|
||||
<packaging>jar</packaging>
|
||||
|
||||
<name>QBit Assistant - OpenAI-compatible provider</name>
|
||||
<description>OpenAI-compatible Chat Completions provider using the JDK HTTP client</description>
|
||||
<url>https://github.com/QRun-IO/qbit-assistant</url>
|
||||
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>com.kingsrook.qbits</groupId>
|
||||
<artifactId>qbit-assistant</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>com.kingsrook.qqq</groupId>
|
||||
<artifactId>qqq-backend-core</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>com.fasterxml.jackson.core</groupId>
|
||||
<artifactId>jackson-databind</artifactId>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>com.kingsrook.qbits</groupId>
|
||||
<artifactId>qbit-assistant</artifactId>
|
||||
<type>test-jar</type>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>com.kingsrook.qqq</groupId>
|
||||
<artifactId>qqq-middleware-javalin</artifactId>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.junit.jupiter</groupId>
|
||||
<artifactId>junit-jupiter</artifactId>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.assertj</groupId>
|
||||
<artifactId>assertj-core</artifactId>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
|
||||
<build>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<groupId>org.apache.maven.plugins</groupId>
|
||||
<artifactId>maven-surefire-plugin</artifactId>
|
||||
<configuration>
|
||||
<!-- live evals call the real API: opt-in only -->
|
||||
<excludedGroups>${test.excludedGroups}</excludedGroups>
|
||||
<redirectTestOutputToFile>true</redirectTestOutputToFile>
|
||||
</configuration>
|
||||
</plugin>
|
||||
<plugin>
|
||||
<groupId>org.jacoco</groupId>
|
||||
<artifactId>jacoco-maven-plugin</artifactId>
|
||||
<executions>
|
||||
<execution>
|
||||
<id>prepare-agent</id>
|
||||
<goals>
|
||||
<goal>prepare-agent</goal>
|
||||
</goals>
|
||||
</execution>
|
||||
<execution>
|
||||
<id>report</id>
|
||||
<phase>verify</phase>
|
||||
<goals>
|
||||
<goal>report</goal>
|
||||
</goals>
|
||||
</execution>
|
||||
<execution>
|
||||
<id>check</id>
|
||||
<goals>
|
||||
<goal>check</goal>
|
||||
</goals>
|
||||
<configuration>
|
||||
<rules>
|
||||
<rule>
|
||||
<element>BUNDLE</element>
|
||||
<limits>
|
||||
<limit>
|
||||
<counter>INSTRUCTION</counter>
|
||||
<value>COVEREDRATIO</value>
|
||||
<minimum>0.70</minimum>
|
||||
</limit>
|
||||
<limit>
|
||||
<counter>CLASS</counter>
|
||||
<value>COVEREDRATIO</value>
|
||||
<minimum>0.90</minimum>
|
||||
</limit>
|
||||
</limits>
|
||||
</rule>
|
||||
</rules>
|
||||
</configuration>
|
||||
</execution>
|
||||
</executions>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
</project>
|
||||
@@ -0,0 +1,429 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.provider.openai;
|
||||
|
||||
|
||||
import java.io.BufferedReader;
|
||||
import java.io.IOException;
|
||||
import java.io.InputStream;
|
||||
import java.io.InputStreamReader;
|
||||
import java.net.http.HttpResponse;
|
||||
import java.net.http.HttpTimeoutException;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.TreeMap;
|
||||
import java.util.concurrent.Executors;
|
||||
import java.util.concurrent.ScheduledExecutorService;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
import java.util.concurrent.atomic.AtomicLong;
|
||||
import java.util.concurrent.atomic.AtomicReference;
|
||||
import com.fasterxml.jackson.core.type.TypeReference;
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.fasterxml.jackson.databind.node.ObjectNode;
|
||||
import com.kingsrook.qbits.assistant.provider.AssistantModelProvider;
|
||||
import com.kingsrook.qbits.assistant.provider.CompletableStreamHandle;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelCapabilities;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelEventListener;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelPart;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelRequest;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelResponse;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelUsage;
|
||||
import com.kingsrook.qbits.assistant.provider.ProviderSettings;
|
||||
import com.kingsrook.qbits.assistant.provider.StreamHandle;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** JDK HTTP Chat Completions adapter. Only visible content is emitted or stored;
|
||||
** reasoning deltas are ignored. Each call owns its cancellation and read guard.
|
||||
*******************************************************************************/
|
||||
public class OpenAiCompatibleModelProvider implements AssistantModelProvider
|
||||
{
|
||||
private final OpenAiHttp http;
|
||||
private final boolean local;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Build a client with resolved settings.
|
||||
*******************************************************************************/
|
||||
public OpenAiCompatibleModelProvider(ProviderSettings settings)
|
||||
{
|
||||
http = new OpenAiHttp(settings);
|
||||
local = "LOCAL".equals(new OpenAiCompatibleProviderType().getDefaultLocation(http.settings.getKind()));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Adapter identity.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getName()
|
||||
{
|
||||
return ("openai-compatible");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Conservative defaults; registry rows can override model-specific facts.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public ModelCapabilities getCapabilities(String modelId)
|
||||
{
|
||||
return (new ModelCapabilities().withTools(true).withParallelToolCalls(true).withPromptCaching(false).withThinking(false)
|
||||
.withMaxContextTokens(local ? 32_000 : 128_000).withStreaming(true).withVision(false).withReasoningEffort(false));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Interrupting the worker aborts the HTTP call, including a blocked body read.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public StreamHandle stream(ModelRequest request, ModelEventListener listener)
|
||||
{
|
||||
CompletableStreamHandle handle = new CompletableStreamHandle();
|
||||
AtomicReference<InputStream> input = new AtomicReference<>();
|
||||
Thread worker = Thread.ofVirtual().unstarted(() -> run(request, listener, handle, input));
|
||||
handle.setOnCancel(() ->
|
||||
{
|
||||
worker.interrupt();
|
||||
close(input.get());
|
||||
});
|
||||
worker.start();
|
||||
return (handle);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Guard the entire request and idle reads, including after HTTP headers.
|
||||
*******************************************************************************/
|
||||
private void run(ModelRequest request, ModelEventListener listener, CompletableStreamHandle handle, AtomicReference<InputStream> input)
|
||||
{
|
||||
Thread worker = Thread.currentThread();
|
||||
AtomicLong lastRead = new AtomicLong(System.nanoTime());
|
||||
long started = lastRead.get();
|
||||
long timeout = TimeUnit.SECONDS.toNanos(http.settings.getRequestTimeoutSeconds());
|
||||
ScheduledExecutorService guard = Executors.newSingleThreadScheduledExecutor(Thread.ofVirtual().factory());
|
||||
Accumulator result = new Accumulator(request.getModel(), listener);
|
||||
handle.setPartialUsage(() -> result.usage);
|
||||
guard.scheduleAtFixedRate(() ->
|
||||
{
|
||||
long now = System.nanoTime();
|
||||
if(now - lastRead.get() >= timeout || now - started >= timeout)
|
||||
{
|
||||
handle.fail(new ModelProviderException(ModelProviderException.Kind.TIMEOUT, null, "The provider did not answer in time.").withPartialUsage(result.usage));
|
||||
worker.interrupt();
|
||||
close(input.get());
|
||||
}
|
||||
}, 100, 100, TimeUnit.MILLISECONDS);
|
||||
try
|
||||
{
|
||||
ModelCapabilities caps = request.getCapabilities() == null ? getCapabilities(request.getModel()) : request.getCapabilities();
|
||||
ObjectNode body = OpenAiMapping.request(request, http.settings.getKind(), caps);
|
||||
boolean strict = false;
|
||||
for(JsonNode tool : body.path("tools"))
|
||||
{
|
||||
strict |= tool.path("function").path("strict").asBoolean();
|
||||
}
|
||||
HttpResponse<InputStream> response = http.client.send(http.request(request.getModel(), http.settings.getRequestTimeoutSeconds())
|
||||
.POST(java.net.http.HttpRequest.BodyPublishers.ofString(body.toString())).build(), HttpResponse.BodyHandlers.ofInputStream());
|
||||
input.set(response.body());
|
||||
result.requestId = OpenAiHttp.requestId(response);
|
||||
if(handle.isCancelled())
|
||||
{
|
||||
return;
|
||||
}
|
||||
if(response.statusCode() != 200)
|
||||
{
|
||||
throw (http.error(response.statusCode(), new String(response.body().readNBytes(64 * 1024), StandardCharsets.UTF_8), result.requestId, strict));
|
||||
}
|
||||
if(Boolean.TRUE.equals(caps.getStreaming()))
|
||||
{
|
||||
readEvents(response.body(), lastRead, result, strict, handle);
|
||||
}
|
||||
else
|
||||
{
|
||||
result.accept(OpenAiHttp.JSON.readTree(response.body()), false, http, strict);
|
||||
}
|
||||
handle.complete(result.response());
|
||||
}
|
||||
catch(ModelProviderException e)
|
||||
{
|
||||
handle.fail(e.withPartialUsage(result.usage));
|
||||
}
|
||||
catch(HttpTimeoutException e)
|
||||
{
|
||||
handle.fail(new ModelProviderException(ModelProviderException.Kind.TIMEOUT, null, "The provider did not answer in time.").withPartialUsage(result.usage));
|
||||
}
|
||||
catch(InterruptedException e)
|
||||
{
|
||||
Thread.currentThread().interrupt();
|
||||
if(!handle.isCancelled())
|
||||
{
|
||||
handle.fail(new ModelProviderException(ModelProviderException.Kind.NETWORK, null, "The provider request was interrupted."));
|
||||
}
|
||||
}
|
||||
catch(IOException | RuntimeException e)
|
||||
{
|
||||
////////////////////////////////////////////////////////////////////////
|
||||
// a transport error at or past the deadline is the timeout itself //
|
||||
// (the JDK client or the guard closed the call), whichever saw it //
|
||||
// first; report it as TIMEOUT so the kind does not depend on a race. //
|
||||
////////////////////////////////////////////////////////////////////////
|
||||
long now = System.nanoTime();
|
||||
long margin = TimeUnit.MILLISECONDS.toNanos(200);
|
||||
boolean late = now - started >= timeout - margin || now - lastRead.get() >= timeout - margin;
|
||||
handle.fail((late ? new ModelProviderException(ModelProviderException.Kind.TIMEOUT, null, "The provider did not answer in time.") : OpenAiHttp.transportError(e)).withPartialUsage(result.usage));
|
||||
}
|
||||
finally
|
||||
{
|
||||
guard.shutdownNow();
|
||||
close(input.get());
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** SSE data fields, CRLF, comments, multi-line data and an optional DONE marker.
|
||||
*******************************************************************************/
|
||||
private void readEvents(InputStream stream, AtomicLong lastRead, Accumulator result, boolean strict, CompletableStreamHandle handle) throws IOException, ModelProviderException
|
||||
{
|
||||
BufferedReader reader = new BufferedReader(new InputStreamReader(stream, StandardCharsets.UTF_8));
|
||||
StringBuilder data = new StringBuilder();
|
||||
String line;
|
||||
while(!handle.isCancelled() && (line = reader.readLine()) != null)
|
||||
{
|
||||
lastRead.set(System.nanoTime());
|
||||
if(line.isEmpty())
|
||||
{
|
||||
if(data.length() > 0)
|
||||
{
|
||||
if("[DONE]".equals(data.toString().trim()))
|
||||
{
|
||||
return;
|
||||
}
|
||||
result.accept(OpenAiHttp.JSON.readTree(data.toString()), true, http, strict);
|
||||
data.setLength(0);
|
||||
}
|
||||
}
|
||||
else if(line.startsWith("data:"))
|
||||
{
|
||||
if(data.length() > 0)
|
||||
{
|
||||
data.append('\n');
|
||||
}
|
||||
data.append(line.substring(5).stripLeading());
|
||||
}
|
||||
}
|
||||
if(data.length() > 0 && !"[DONE]".equals(data.toString().trim()))
|
||||
{
|
||||
result.accept(OpenAiHttp.JSON.readTree(data.toString()), true, http, strict);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Closing an already cancelled stream is harmless.
|
||||
*******************************************************************************/
|
||||
private static void close(InputStream input)
|
||||
{
|
||||
if(input != null)
|
||||
{
|
||||
try
|
||||
{
|
||||
input.close();
|
||||
}
|
||||
catch(IOException e)
|
||||
{
|
||||
// The aborted HTTP call may already have closed the stream.
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Tool arguments arrive as JSON fragments, ordered by call index.
|
||||
*******************************************************************************/
|
||||
private static class Tool
|
||||
{
|
||||
private String id = "";
|
||||
private String name = "";
|
||||
private final StringBuilder arguments = new StringBuilder();
|
||||
private boolean announced;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Shared mapping for stream chunks and one-shot responses.
|
||||
*******************************************************************************/
|
||||
private static class Accumulator
|
||||
{
|
||||
private final ModelEventListener listener;
|
||||
private final StringBuilder text = new StringBuilder();
|
||||
private final Map<Integer, Tool> tools = new TreeMap<>();
|
||||
private volatile ModelUsage usage = new ModelUsage();
|
||||
private String model;
|
||||
private String requestId;
|
||||
private String stop = ModelResponse.STOP_END_TURN;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** One completion.
|
||||
*******************************************************************************/
|
||||
Accumulator(String model, ModelEventListener listener)
|
||||
{
|
||||
this.model = model;
|
||||
this.listener = listener;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Ignore reasoning fields; retain only text, tools and billable usage.
|
||||
*******************************************************************************/
|
||||
void accept(JsonNode chunk, boolean streaming, OpenAiHttp http, boolean strict) throws ModelProviderException
|
||||
{
|
||||
if(chunk.has("error"))
|
||||
{
|
||||
JsonNode error = chunk.path("error");
|
||||
int status = error.path("status").asInt(error.path("code").asInt(500));
|
||||
String type = error.path("type").asText();
|
||||
if(status == 500)
|
||||
{
|
||||
status = switch(type)
|
||||
{
|
||||
case "invalid_request_error" -> 400;
|
||||
case "authentication_error" -> 401;
|
||||
case "rate_limit_error" -> 429;
|
||||
case "overloaded_error" -> 503;
|
||||
default -> 500;
|
||||
};
|
||||
}
|
||||
throw (http.error(status, chunk.toString(), requestId, strict));
|
||||
}
|
||||
model = chunk.path("model").asText(model);
|
||||
if(chunk.hasNonNull("usage"))
|
||||
{
|
||||
JsonNode used = chunk.get("usage");
|
||||
long cached = used.path("prompt_tokens_details").path("cached_tokens").asLong();
|
||||
usage = new ModelUsage().withInputTokens(Math.max(0, used.path("prompt_tokens").asLong() - cached))
|
||||
.withOutputTokens(used.path("completion_tokens").asLong()).withCacheReadTokens(cached);
|
||||
}
|
||||
for(JsonNode choice : chunk.path("choices"))
|
||||
{
|
||||
if(choice.path("index").asInt() != 0)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
JsonNode delta = choice.path(streaming ? "delta" : "message");
|
||||
String content = delta.path("content").asText("");
|
||||
if(!content.isEmpty())
|
||||
{
|
||||
text.append(content);
|
||||
listener.onTextDelta(content);
|
||||
}
|
||||
int position = 0;
|
||||
for(JsonNode call : delta.path("tool_calls"))
|
||||
{
|
||||
Tool tool = tools.computeIfAbsent(call.path("index").asInt(position++), k -> new Tool());
|
||||
if(tool.id.isEmpty())
|
||||
{
|
||||
tool.id = call.path("id").asText("");
|
||||
}
|
||||
if(tool.name.isEmpty())
|
||||
{
|
||||
tool.name = call.path("function").path("name").asText("");
|
||||
}
|
||||
tool.arguments.append(call.path("function").path("arguments").asText(""));
|
||||
if(!tool.announced && !tool.id.isEmpty() && !tool.name.isEmpty())
|
||||
{
|
||||
listener.onToolUseStart(tool.id, tool.name);
|
||||
tool.announced = true;
|
||||
}
|
||||
}
|
||||
if(choice.hasNonNull("finish_reason"))
|
||||
{
|
||||
stop = switch(choice.get("finish_reason").asText())
|
||||
{
|
||||
case "tool_calls", "function_call" -> ModelResponse.STOP_TOOL_USE;
|
||||
case "length" -> ModelResponse.STOP_MAX_TOKENS;
|
||||
case "content_filter" -> ModelResponse.STOP_REFUSAL;
|
||||
default -> ModelResponse.STOP_END_TURN;
|
||||
};
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Parse complete argument objects only, preserving parallel call order.
|
||||
*******************************************************************************/
|
||||
ModelResponse response()
|
||||
{
|
||||
List<ModelPart> parts = new ArrayList<>();
|
||||
if(!text.isEmpty())
|
||||
{
|
||||
parts.add(ModelPart.text(text.toString()));
|
||||
}
|
||||
for(Map.Entry<Integer, Tool> entry : tools.entrySet())
|
||||
{
|
||||
Tool tool = entry.getValue();
|
||||
if(tool.id.isEmpty())
|
||||
{
|
||||
tool.id = "call_" + entry.getKey();
|
||||
}
|
||||
if(!tool.announced)
|
||||
{
|
||||
listener.onToolUseStart(tool.id, tool.name);
|
||||
tool.announced = true;
|
||||
}
|
||||
Map<String, Object> arguments;
|
||||
try
|
||||
{
|
||||
arguments = tool.arguments.isEmpty() ? Map.of() : OpenAiHttp.JSON.readerFor(new TypeReference<Map<String, Object>>() {})
|
||||
.with(com.fasterxml.jackson.databind.DeserializationFeature.FAIL_ON_TRAILING_TOKENS).readValue(tool.arguments.toString());
|
||||
if(arguments == null)
|
||||
{
|
||||
throw (new IOException("Tool arguments must be an object."));
|
||||
}
|
||||
}
|
||||
catch(IOException e)
|
||||
{
|
||||
arguments = Map.of("_invalid_arguments", tool.arguments.substring(0, Math.min(200, tool.arguments.length())));
|
||||
}
|
||||
parts.add(ModelPart.toolUse(tool.id, tool.name, arguments));
|
||||
}
|
||||
if(!tools.isEmpty() && ModelResponse.STOP_END_TURN.equals(stop))
|
||||
{
|
||||
stop = ModelResponse.STOP_TOOL_USE;
|
||||
}
|
||||
return (new ModelResponse().withModel(model).withRequestId(requestId).withParts(parts).withUsage(usage).withStopReason(stop));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,199 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.provider.openai;
|
||||
|
||||
|
||||
import java.util.List;
|
||||
import com.kingsrook.qbits.assistant.provider.AssistantModelProvider;
|
||||
import com.kingsrook.qbits.assistant.provider.AssistantProviderType;
|
||||
import com.kingsrook.qbits.assistant.provider.DiscoveredModel;
|
||||
import com.kingsrook.qbits.assistant.provider.HealthResult;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
|
||||
import com.kingsrook.qbits.assistant.provider.ProviderSettings;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** One installed adapter for the Chat Completions dialects.
|
||||
*******************************************************************************/
|
||||
public class OpenAiCompatibleProviderType implements AssistantProviderType
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Installed adapter id.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getType()
|
||||
{
|
||||
return ("openai-compatible");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Settings label.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getLabel()
|
||||
{
|
||||
return ("OpenAI-compatible");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Discover endpoint-specific models instead of guessing ids.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public List<String> getModels()
|
||||
{
|
||||
return (List.of());
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Extensible kind list.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public List<String> getKinds()
|
||||
{
|
||||
return (List.of("OPENAI", "AZURE_OPENAI", "OPENROUTER", "OLLAMA", "LM_STUDIO", "VLLM", "LLAMA_CPP", "OPENAI_COMPATIBLE"));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The legacy factory uses the generic dialect.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getDefaultApiKeyEnvName()
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The generic dialect requires an explicit URL.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getDefaultBaseUrl()
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Built-in endpoints. Redirects are never followed.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getDefaultBaseUrl(String kind)
|
||||
{
|
||||
return (switch(kind)
|
||||
{
|
||||
case "OPENAI" -> "https://api.openai.com/v1";
|
||||
case "OPENROUTER" -> "https://openrouter.ai/api/v1";
|
||||
case "OLLAMA" -> "http://localhost:11434/v1";
|
||||
case "LM_STUDIO" -> "http://localhost:1234/v1";
|
||||
case "VLLM" -> "http://localhost:8000/v1";
|
||||
case "LLAMA_CPP" -> "http://localhost:8080/v1";
|
||||
default -> null;
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Default key references, never key values.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getDefaultApiKeyEnvName(String kind)
|
||||
{
|
||||
return (switch(kind)
|
||||
{
|
||||
case "OPENAI" -> "OPENAI_API_KEY";
|
||||
case "AZURE_OPENAI" -> "AZURE_OPENAI_API_KEY";
|
||||
case "OPENROUTER" -> "OPENROUTER_API_KEY";
|
||||
default -> null;
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Location is an admin-editable fact, with a kind-based default.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getDefaultLocation(String kind)
|
||||
{
|
||||
return (List.of("OLLAMA", "LM_STUDIO", "VLLM", "LLAMA_CPP").contains(kind) ? "LOCAL" : "CLOUD");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Backward-compatible factory.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public AssistantModelProvider create(String apiKey, String baseUrl)
|
||||
{
|
||||
return (create(new ProviderSettings().withKind("OPENAI_COMPATIBLE").withApiKey(apiKey).withBaseUrl(baseUrl)));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Full connection settings.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public AssistantModelProvider create(ProviderSettings settings)
|
||||
{
|
||||
return (new OpenAiCompatibleModelProvider(settings));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Only a models request, never a completion.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public List<DiscoveredModel> discoverModels(ProviderSettings settings) throws ModelProviderException
|
||||
{
|
||||
return (new OpenAiHttp(settings).models(settings.getRequestTimeoutSeconds()));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A short models probe with latency and count.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public HealthResult checkHealth(ProviderSettings settings)
|
||||
{
|
||||
long start = System.nanoTime();
|
||||
try
|
||||
{
|
||||
List<DiscoveredModel> models = new OpenAiHttp(settings).models(Math.min(10, settings.getRequestTimeoutSeconds()));
|
||||
return (new HealthResult(true, (System.nanoTime() - start) / 1_000_000, models.size() + " models available"));
|
||||
}
|
||||
catch(ModelProviderException e)
|
||||
{
|
||||
return (new HealthResult(false, (System.nanoTime() - start) / 1_000_000,
|
||||
e.getKind() + (e.getHttpStatus() == null ? "" : " HTTP " + e.getHttpStatus()) + ": " + e.getMessage()));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,267 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.provider.openai;
|
||||
|
||||
|
||||
import java.io.IOException;
|
||||
import java.net.URI;
|
||||
import java.net.URLEncoder;
|
||||
import java.net.http.HttpClient;
|
||||
import java.net.http.HttpRequest;
|
||||
import java.net.http.HttpResponse;
|
||||
import java.net.http.HttpTimeoutException;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.time.Duration;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Locale;
|
||||
import java.util.concurrent.CompletableFuture;
|
||||
import java.util.concurrent.ExecutionException;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
import java.util.concurrent.TimeoutException;
|
||||
import javax.net.ssl.SSLHandshakeException;
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.kingsrook.qbits.assistant.provider.DiscoveredModel;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
|
||||
import com.kingsrook.qbits.assistant.provider.ProviderSettings;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Shared HTTP/authentication/error mapping. ProviderService guards URLs before
|
||||
** constructing this transport; redirects must not move keys to another host.
|
||||
*******************************************************************************/
|
||||
final class OpenAiHttp
|
||||
{
|
||||
static final ObjectMapper JSON = new ObjectMapper();
|
||||
final HttpClient client;
|
||||
final ProviderSettings settings;
|
||||
private final String base;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Copy settings so callers cannot mutate a cached client's credentials.
|
||||
*******************************************************************************/
|
||||
OpenAiHttp(ProviderSettings source)
|
||||
{
|
||||
settings = new ProviderSettings().withKind(source.getKind() == null ? "OPENAI_COMPATIBLE" : source.getKind())
|
||||
.withTlsCaBundle(source.getTlsCaBundle()).withSslContext(source.getSslContext())
|
||||
.withApiKey(source.getApiKey()).withBaseUrl(source.getBaseUrl()).withApiVersion(source.getApiVersion()).withRegion(source.getRegion())
|
||||
.withConnectTimeoutSeconds(source.getConnectTimeoutSeconds()).withRequestTimeoutSeconds(source.getRequestTimeoutSeconds());
|
||||
String url = settings.getBaseUrl() == null ? new OpenAiCompatibleProviderType().getDefaultBaseUrl(settings.getKind()) : settings.getBaseUrl();
|
||||
if(url == null)
|
||||
{
|
||||
throw (new IllegalArgumentException("A base URL is required for this provider kind."));
|
||||
}
|
||||
base = url.replaceAll("/+$", "");
|
||||
HttpClient.Builder builder = HttpClient.newBuilder().connectTimeout(Duration.ofSeconds(settings.getConnectTimeoutSeconds())).followRedirects(HttpClient.Redirect.NEVER);
|
||||
if(settings.getSslContext() != null)
|
||||
{
|
||||
builder.sslContext(settings.getSslContext());
|
||||
}
|
||||
client = builder.build();
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Azure uses deployment paths and a version query parameter.
|
||||
*******************************************************************************/
|
||||
HttpRequest.Builder request(String model, Integer seconds) throws ModelProviderException
|
||||
{
|
||||
try
|
||||
{
|
||||
String path = model == null ? "/models" : "/chat/completions";
|
||||
if("AZURE_OPENAI".equals(settings.getKind()))
|
||||
{
|
||||
path = model == null ? "/openai/models" : "/openai/deployments/" + encode(model) + "/chat/completions";
|
||||
path += "?api-version=" + encode(settings.getApiVersion() == null || settings.getApiVersion().isBlank() ? "2024-10-21" : settings.getApiVersion());
|
||||
}
|
||||
HttpRequest.Builder builder = HttpRequest.newBuilder(URI.create(base + path)).timeout(Duration.ofSeconds(seconds)).header("Content-Type", "application/json");
|
||||
if(settings.getApiKey() != null && !settings.getApiKey().isBlank())
|
||||
{
|
||||
if("AZURE_OPENAI".equals(settings.getKind()))
|
||||
{
|
||||
builder.header("api-key", settings.getApiKey());
|
||||
}
|
||||
else
|
||||
{
|
||||
builder.header("Authorization", "Bearer " + settings.getApiKey());
|
||||
}
|
||||
}
|
||||
return (builder);
|
||||
}
|
||||
catch(IllegalArgumentException e)
|
||||
{
|
||||
throw (new ModelProviderException(ModelProviderException.Kind.INVALID_REQUEST, null, "The provider connection settings are invalid."));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Models with optional server-specific context information.
|
||||
*******************************************************************************/
|
||||
List<DiscoveredModel> models(Integer seconds) throws ModelProviderException
|
||||
{
|
||||
CompletableFuture<HttpResponse<String>> pending = client.sendAsync(request(null, seconds).GET().build(), HttpResponse.BodyHandlers.ofString());
|
||||
try
|
||||
{
|
||||
HttpResponse<String> response = pending.get(seconds, TimeUnit.SECONDS);
|
||||
if(response.statusCode() != 200)
|
||||
{
|
||||
throw (error(response.statusCode(), response.body(), requestId(response), false));
|
||||
}
|
||||
JsonNode body = JSON.readTree(response.body());
|
||||
if(body.has("error"))
|
||||
{
|
||||
throw (error(500, response.body(), requestId(response), false));
|
||||
}
|
||||
List<DiscoveredModel> result = new ArrayList<>();
|
||||
for(JsonNode model : body.path("data"))
|
||||
{
|
||||
String id = model.path("id").asText();
|
||||
if(id.isBlank())
|
||||
{
|
||||
continue;
|
||||
}
|
||||
int context = model.path("context_length").asInt(model.path("max_model_len").asInt(model.path("meta").path("n_ctx_train").asInt()));
|
||||
result.add(new DiscoveredModel(id, model.path("name").asText(id), context > 0 ? context : null, model.path("owned_by").asText(null)));
|
||||
}
|
||||
return (result);
|
||||
}
|
||||
catch(TimeoutException e)
|
||||
{
|
||||
throw (new ModelProviderException(ModelProviderException.Kind.TIMEOUT, null, "The provider did not answer in time."));
|
||||
}
|
||||
catch(ExecutionException e)
|
||||
{
|
||||
throw (transportError(e));
|
||||
}
|
||||
catch(InterruptedException e)
|
||||
{
|
||||
Thread.currentThread().interrupt();
|
||||
throw (new ModelProviderException(ModelProviderException.Kind.NETWORK, null, "The provider request was interrupted."));
|
||||
}
|
||||
catch(IOException e)
|
||||
{
|
||||
throw (transportError(e));
|
||||
}
|
||||
finally
|
||||
{
|
||||
pending.cancel(true);
|
||||
client.close();
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Sanitize errors, explicitly removing even short or unusual API keys.
|
||||
*******************************************************************************/
|
||||
ModelProviderException error(int status, String body, String requestId, boolean strict)
|
||||
{
|
||||
String message = "The provider refused the request.";
|
||||
String type = "openai-compatible";
|
||||
try
|
||||
{
|
||||
JsonNode error = JSON.readTree(body).path("error");
|
||||
message = error.path("message").asText(message);
|
||||
type = error.path("type").asText(type);
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
// Do not expose unstructured proxy pages or request bodies.
|
||||
}
|
||||
message = safe(message);
|
||||
type = safe(type);
|
||||
ModelProviderException.Kind kind = errorKind(status);
|
||||
String lower = message == null ? "" : message.toLowerCase(Locale.ROOT);
|
||||
return (new ModelProviderException(kind, status, message).withProviderDetails(type, message, safe(requestId))
|
||||
.withStrictSchemaRejected(strict && status == 400 && (lower.contains("strict") || lower.contains("schema") || lower.contains("grammar"))));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Credential-free diagnostics.
|
||||
*******************************************************************************/
|
||||
private String safe(String value)
|
||||
{
|
||||
String key = settings.getApiKey();
|
||||
return (ModelProviderException.sanitizeProviderMessage(value == null || key == null || key.isEmpty() ? value : value.replace(key, "[redacted]"), 300));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Common request-id headers.
|
||||
*******************************************************************************/
|
||||
static String requestId(HttpResponse<?> response)
|
||||
{
|
||||
return (response.headers().firstValue("x-request-id").orElse(response.headers().firstValue("apim-request-id").orElse(null)));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Encode a single path segment or query value.
|
||||
*******************************************************************************/
|
||||
private static String encode(String value)
|
||||
{
|
||||
return (URLEncoder.encode(value, StandardCharsets.UTF_8).replace("+", "%20"));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Map the provider error category.
|
||||
*******************************************************************************/
|
||||
private static ModelProviderException.Kind errorKind(int status)
|
||||
{
|
||||
return (switch(status)
|
||||
{
|
||||
case 401, 403 -> ModelProviderException.Kind.AUTHENTICATION;
|
||||
case 429 -> ModelProviderException.Kind.RATE_LIMITED;
|
||||
case 503, 529 -> ModelProviderException.Kind.OVERLOADED;
|
||||
case 400, 404, 422 -> ModelProviderException.Kind.INVALID_REQUEST;
|
||||
default -> status >= 500 ? ModelProviderException.Kind.SERVER : ModelProviderException.Kind.INVALID_REQUEST;
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fixed diagnostics omit remote certificate details, URLs and credentials.
|
||||
*******************************************************************************/
|
||||
static ModelProviderException transportError(Throwable error)
|
||||
{
|
||||
for(Throwable cause = error; cause != null; cause = cause.getCause())
|
||||
{
|
||||
if(cause instanceof SSLHandshakeException)
|
||||
{
|
||||
return (new ModelProviderException(ModelProviderException.Kind.NETWORK, null,
|
||||
"The provider TLS certificate is not trusted or does not match the host. Check tlsCaBundle and the HTTPS hostname."));
|
||||
}
|
||||
if(cause instanceof HttpTimeoutException)
|
||||
{
|
||||
return (new ModelProviderException(ModelProviderException.Kind.TIMEOUT, null, "The provider did not answer in time."));
|
||||
}
|
||||
}
|
||||
return (new ModelProviderException(ModelProviderException.Kind.NETWORK, null, "Could not read the provider response."));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,190 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.provider.openai;
|
||||
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Locale;
|
||||
import java.util.stream.Collectors;
|
||||
import com.fasterxml.jackson.core.JsonProcessingException;
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.fasterxml.jackson.databind.node.ArrayNode;
|
||||
import com.fasterxml.jackson.databind.node.ObjectNode;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelCapabilities;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelMessage;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelPart;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelRequest;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelToolDefinition;
|
||||
import com.kingsrook.qbits.assistant.provider.SystemBlock;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Chat Completions request mapping and conservative strict-schema checks.
|
||||
*******************************************************************************/
|
||||
final class OpenAiMapping
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private OpenAiMapping()
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Map only capabilities the target dialect supports.
|
||||
*******************************************************************************/
|
||||
static ObjectNode request(ModelRequest request, String kind, ModelCapabilities capabilities) throws JsonProcessingException
|
||||
{
|
||||
ObjectNode body = OpenAiHttp.JSON.createObjectNode();
|
||||
boolean streaming = Boolean.TRUE.equals(capabilities.getStreaming());
|
||||
boolean nativeOpenAi = List.of("OPENAI", "AZURE_OPENAI").contains(kind);
|
||||
body.put("model", request.getModel());
|
||||
body.put("stream", streaming);
|
||||
if(streaming)
|
||||
{
|
||||
body.putObject("stream_options").put("include_usage", true);
|
||||
}
|
||||
if(request.getMaxTokens() != null)
|
||||
{
|
||||
body.put(nativeOpenAi ? "max_completion_tokens" : "max_tokens", request.getMaxTokens());
|
||||
}
|
||||
if(Boolean.TRUE.equals(capabilities.getReasoningEffort()) && request.getEffort() != null)
|
||||
{
|
||||
String effort = request.getEffort().toLowerCase(Locale.ROOT);
|
||||
body.put("reasoning_effort", List.of("low", "medium").contains(effort) ? effort : "high");
|
||||
}
|
||||
ArrayNode messages = body.putArray("messages");
|
||||
if(!request.getSystem().isEmpty())
|
||||
{
|
||||
messages.addObject().put("role", "system").put("content", request.getSystem().stream().map(SystemBlock::getText).collect(Collectors.joining("\n\n")));
|
||||
}
|
||||
for(ModelMessage message : request.getMessages())
|
||||
{
|
||||
ObjectNode current = null;
|
||||
for(ModelPart part : message.getParts())
|
||||
{
|
||||
if(part.getType() == ModelPart.Type.TOOL_RESULT)
|
||||
{
|
||||
current = null;
|
||||
messages.addObject().put("role", "tool").put("tool_call_id", part.getToolUseId())
|
||||
.put("content", (Boolean.TRUE.equals(part.getIsError()) ? "ERROR: " : "") + part.getText());
|
||||
continue;
|
||||
}
|
||||
if(current == null)
|
||||
{
|
||||
current = messages.addObject().put("role", message.getRole() == ModelMessage.Role.ASSISTANT ? "assistant" : "user");
|
||||
current.put("content", "");
|
||||
}
|
||||
if(part.getType() == ModelPart.Type.TEXT)
|
||||
{
|
||||
current.put("content", current.path("content").asText() + part.getText());
|
||||
}
|
||||
else if(part.getType() == ModelPart.Type.TOOL_USE)
|
||||
{
|
||||
ArrayNode calls = current.has("tool_calls") ? (ArrayNode) current.get("tool_calls") : current.putArray("tool_calls");
|
||||
calls.addObject().put("id", part.getToolUseId()).put("type", "function").putObject("function")
|
||||
.put("name", part.getToolName()).put("arguments", OpenAiHttp.JSON.writeValueAsString(part.getInput()));
|
||||
}
|
||||
}
|
||||
}
|
||||
if(!request.getTools().isEmpty() && Boolean.TRUE.equals(capabilities.getTools()))
|
||||
{
|
||||
ArrayNode tools = body.putArray("tools");
|
||||
for(ModelToolDefinition tool : request.getTools())
|
||||
{
|
||||
JsonNode schema = OpenAiHttp.JSON.valueToTree(tool.getInputSchema());
|
||||
ObjectNode function = tools.addObject().put("type", "function").putObject("function").put("name", tool.getName()).put("description", tool.getDescription());
|
||||
function.set("parameters", schema);
|
||||
if(nativeOpenAi && Boolean.TRUE.equals(tool.getStrict()) && strictCompatible(schema))
|
||||
{
|
||||
function.put("strict", true);
|
||||
}
|
||||
}
|
||||
if(List.of("OPENAI", "AZURE_OPENAI", "OPENROUTER", "VLLM").contains(kind))
|
||||
{
|
||||
body.put("parallel_tool_calls", Boolean.TRUE.equals(request.getParallelToolCalls()) && Boolean.TRUE.equals(capabilities.getParallelToolCalls()));
|
||||
}
|
||||
}
|
||||
return (body);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Every object must forbid extras and require every declared property.
|
||||
*******************************************************************************/
|
||||
static boolean strictCompatible(JsonNode schema)
|
||||
{
|
||||
if(schema == null || !schema.isObject())
|
||||
{
|
||||
return (false);
|
||||
}
|
||||
JsonNode type = schema.path("type");
|
||||
if("object".equals(type.asText()) || schema.has("properties") || (type.isArray() && type.toString().contains("\"object\"")))
|
||||
{
|
||||
if(!schema.has("additionalProperties") || !schema.get("additionalProperties").isBoolean() || schema.get("additionalProperties").asBoolean())
|
||||
{
|
||||
return (false);
|
||||
}
|
||||
JsonNode required = schema.path("required");
|
||||
var names = schema.path("properties").fieldNames();
|
||||
while(names.hasNext())
|
||||
{
|
||||
String name = names.next();
|
||||
boolean found = false;
|
||||
for(JsonNode value : required)
|
||||
{
|
||||
found |= name.equals(value.asText());
|
||||
}
|
||||
if(!found || !strictCompatible(schema.path("properties").get(name)))
|
||||
{
|
||||
return (false);
|
||||
}
|
||||
}
|
||||
}
|
||||
for(String keyword : List.of("items", "additionalProperties"))
|
||||
{
|
||||
if(schema.path(keyword).isObject() && !strictCompatible(schema.get(keyword)))
|
||||
{
|
||||
return (false);
|
||||
}
|
||||
}
|
||||
for(String keyword : List.of("anyOf", "oneOf", "allOf", "prefixItems"))
|
||||
{
|
||||
for(JsonNode child : schema.path(keyword))
|
||||
{
|
||||
if(!strictCompatible(child))
|
||||
{
|
||||
return (false);
|
||||
}
|
||||
}
|
||||
}
|
||||
for(String keyword : List.of("$defs", "definitions"))
|
||||
{
|
||||
for(JsonNode child : schema.path(keyword))
|
||||
{
|
||||
if(!strictCompatible(child))
|
||||
{
|
||||
return (false);
|
||||
}
|
||||
}
|
||||
}
|
||||
return (true);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,77 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.provider.openai;
|
||||
|
||||
|
||||
import java.nio.file.Path;
|
||||
import java.time.Duration;
|
||||
import java.util.List;
|
||||
import com.kingsrook.qbits.assistant.AssistantQBitConfig;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelMessage;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelRequest;
|
||||
import com.kingsrook.qbits.assistant.provider.ProviderSettings;
|
||||
import org.junit.jupiter.api.Assumptions;
|
||||
import org.junit.jupiter.api.Tag;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Opt-in smoke test; never contacts an endpoint in the default build.
|
||||
*******************************************************************************/
|
||||
@Tag("live")
|
||||
class LiveOpenAiCompatibleTest
|
||||
{
|
||||
/*******************************************************************************
|
||||
** All connection details come from environment references.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void live() throws Exception
|
||||
{
|
||||
String url = System.getenv("ASSISTANT_LIVE_OPENAI_BASE_URL");
|
||||
String model = System.getenv("ASSISTANT_LIVE_OPENAI_MODEL");
|
||||
Assumptions.assumeTrue(url != null && !url.isBlank() && model != null && !model.isBlank(), "Live URL and model must be set");
|
||||
String kind = System.getenv("ASSISTANT_LIVE_OPENAI_KIND");
|
||||
String keyEnv = System.getenv("ASSISTANT_LIVE_OPENAI_KEY_ENV");
|
||||
ProviderSettings settings = new ProviderSettings().withKind(kind == null || kind.isBlank() ? "OLLAMA" : kind)
|
||||
.withBaseUrl(url).withApiKey(keyEnv == null || keyEnv.isBlank() ? null : System.getenv(keyEnv));
|
||||
String caFile = System.getenv("ASSISTANT_LIVE_OPENAI_CA_FILE");
|
||||
if(caFile != null && !caFile.isBlank())
|
||||
{
|
||||
settings.withTlsCaBundle("live").withSslContext(new AssistantQBitConfig().withTrustedCaBundleFile("live", Path.of(caFile)).getTrustedCaBundles().get("live"));
|
||||
}
|
||||
try
|
||||
{
|
||||
OpenAiCompatibleProviderType type = new OpenAiCompatibleProviderType();
|
||||
Assumptions.assumeTrue(Boolean.TRUE.equals(type.checkHealth(settings).healthy()), "Live health probe must pass");
|
||||
assertThat(type.discoverModels(settings)).isNotEmpty();
|
||||
var provider = type.create(settings);
|
||||
StringBuilder text = new StringBuilder();
|
||||
provider.stream(new ModelRequest().withModel(model).withMaxTokens(128).withMessages(List.of(ModelMessage.user("Reply with OK."))), text::append).await(Duration.ofSeconds(120));
|
||||
assertThat(text).isNotEmpty();
|
||||
var response = provider.stream(new ModelRequest().withModel(model).withMaxTokens(256).withTools(List.of(OpenAiContractTest.tool()))
|
||||
.withMessages(List.of(ModelMessage.user("Call lookup with q set to hello. Do not answer without calling the tool."))), t -> {}).await(Duration.ofSeconds(120));
|
||||
assertThat(response.getToolUses()).anyMatch(t -> "lookup".equals(t.getToolName()));
|
||||
}
|
||||
catch(ModelProviderException e)
|
||||
{
|
||||
Assumptions.assumeFalse(e.getKind() == ModelProviderException.Kind.NETWORK || e.getKind() == ModelProviderException.Kind.TIMEOUT, "Live endpoint unreachable (including TLS)");
|
||||
throw (e);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,183 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.provider.openai;
|
||||
|
||||
|
||||
import java.io.IOException;
|
||||
import java.net.InetSocketAddress;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.List;
|
||||
import java.util.concurrent.CopyOnWriteArrayList;
|
||||
import java.util.concurrent.CountDownLatch;
|
||||
import java.util.concurrent.ExecutorService;
|
||||
import java.util.concurrent.Executors;
|
||||
import java.util.concurrent.LinkedBlockingQueue;
|
||||
import javax.net.ssl.SSLContext;
|
||||
import com.sun.net.httpserver.HttpExchange;
|
||||
import com.sun.net.httpserver.HttpServer;
|
||||
import com.sun.net.httpserver.HttpsConfigurator;
|
||||
import com.sun.net.httpserver.HttpsServer;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Loopback-only scripted Chat Completions server with captured requests.
|
||||
*******************************************************************************/
|
||||
class MockOpenAiServer implements AutoCloseable
|
||||
{
|
||||
final List<Captured> requests = new CopyOnWriteArrayList<>();
|
||||
private final LinkedBlockingQueue<Script> scripts = new LinkedBlockingQueue<>();
|
||||
private final HttpServer server;
|
||||
private final ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor();
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Captured wire data, only synthetic keys in default tests.
|
||||
*******************************************************************************/
|
||||
record Captured(String path, String authorization, String apiKey, String body)
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Delay can be before headers or after an SSE keep-alive.
|
||||
*******************************************************************************/
|
||||
record Script(int status, String body, long delayMs, boolean afterHeaders, CountDownLatch started)
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Bind only the loopback interface.
|
||||
*******************************************************************************/
|
||||
MockOpenAiServer() throws IOException
|
||||
{
|
||||
this(null);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Optional TLS uses exactly the same scripted endpoints and request capture.
|
||||
*******************************************************************************/
|
||||
MockOpenAiServer(SSLContext tls) throws IOException
|
||||
{
|
||||
if(tls == null)
|
||||
{
|
||||
server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
|
||||
}
|
||||
else
|
||||
{
|
||||
HttpsServer https = HttpsServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
|
||||
https.setHttpsConfigurator(new HttpsConfigurator(tls));
|
||||
server = https;
|
||||
}
|
||||
server.createContext("/", this::serve);
|
||||
server.setExecutor(executor);
|
||||
server.start();
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Base URL deliberately has the v1 prefix.
|
||||
*******************************************************************************/
|
||||
String url()
|
||||
{
|
||||
return ((server instanceof HttpsServer ? "https://localhost:" : "http://127.0.0.1:") + server.getAddress().getPort() + "/v1");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Queue a reply.
|
||||
*******************************************************************************/
|
||||
void enqueue(int status, String body)
|
||||
{
|
||||
scripts.add(new Script(status, body, 0, false, new CountDownLatch(1)));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Queue a stalled response.
|
||||
*******************************************************************************/
|
||||
CountDownLatch stall(boolean afterHeaders)
|
||||
{
|
||||
CountDownLatch started = new CountDownLatch(1);
|
||||
scripts.add(new Script(200, "data: [DONE]\n\n", 2500, afterHeaders, started));
|
||||
return (started);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Capture and reply, tolerating client cancellation.
|
||||
*******************************************************************************/
|
||||
private void serve(HttpExchange exchange) throws IOException
|
||||
{
|
||||
try(exchange)
|
||||
{
|
||||
requests.add(new Captured(exchange.getRequestURI().toString(), exchange.getRequestHeaders().getFirst("Authorization"),
|
||||
exchange.getRequestHeaders().getFirst("api-key"), new String(exchange.getRequestBody().readAllBytes(), StandardCharsets.UTF_8)));
|
||||
Script script = scripts.poll();
|
||||
if(script == null)
|
||||
{
|
||||
exchange.sendResponseHeaders(500, -1);
|
||||
return;
|
||||
}
|
||||
if(!script.afterHeaders())
|
||||
{
|
||||
script.started().countDown();
|
||||
Thread.sleep(script.delayMs());
|
||||
}
|
||||
exchange.getResponseHeaders().set("Content-Type", script.body().startsWith("{") ? "application/json" : "text/event-stream");
|
||||
exchange.getResponseHeaders().set("x-request-id", "mock-request");
|
||||
exchange.sendResponseHeaders(script.status(), 0);
|
||||
if(script.afterHeaders())
|
||||
{
|
||||
exchange.getResponseBody().write(": keep-alive\r\n\r\n".getBytes(StandardCharsets.UTF_8));
|
||||
exchange.getResponseBody().flush();
|
||||
script.started().countDown();
|
||||
Thread.sleep(script.delayMs());
|
||||
}
|
||||
exchange.getResponseBody().write(script.body().getBytes(StandardCharsets.UTF_8));
|
||||
}
|
||||
catch(InterruptedException e)
|
||||
{
|
||||
Thread.currentThread().interrupt();
|
||||
}
|
||||
catch(IOException e)
|
||||
{
|
||||
// Cancellation closes the client socket while a script is writing.
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Stop all handlers.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void close()
|
||||
{
|
||||
server.stop(0);
|
||||
executor.shutdownNow();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,461 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.provider.openai;
|
||||
|
||||
|
||||
import java.time.Duration;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelCapabilities;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelEventListener;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelMessage;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelPart;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelRequest;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelResponse;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelToolDefinition;
|
||||
import com.kingsrook.qbits.assistant.provider.ProviderSettings;
|
||||
import com.kingsrook.qbits.assistant.provider.SystemBlock;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.params.ParameterizedTest;
|
||||
import org.junit.jupiter.params.provider.CsvSource;
|
||||
import org.junit.jupiter.params.provider.ValueSource;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Contract tests exercise the wire protocol through localhost HTTP only.
|
||||
*******************************************************************************/
|
||||
class OpenAiContractTest
|
||||
{
|
||||
static final String TEXT = "data: {\"model\":\"test\",\"choices\":[{\"delta\":{\"content\":\"OK\"},\"finish_reason\":\"stop\"}]}\r\n\r\ndata: [DONE]\r\n\r\n";
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A minimal request, also shared by the opt-in live test.
|
||||
*******************************************************************************/
|
||||
static ModelRequest request()
|
||||
{
|
||||
return (new ModelRequest().withModel("test").withMaxTokens(32).withEffort("max").withMessages(List.of(ModelMessage.user("Hello"))));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Synthetic settings for the local server.
|
||||
*******************************************************************************/
|
||||
static ProviderSettings settings(MockOpenAiServer server, String kind)
|
||||
{
|
||||
return (new ProviderSettings().withKind(kind).withBaseUrl(server.url()).withApiKey("synthetic-key"));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Strict-compatible schema with one required property.
|
||||
*******************************************************************************/
|
||||
static ModelToolDefinition tool()
|
||||
{
|
||||
return (new ModelToolDefinition().withName("lookup").withDescription("Look up a value").withStrict(true)
|
||||
.withInputSchema(Map.of("type", "object", "additionalProperties", false, "properties", Map.of("q", Map.of("type", "string")), "required", List.of("q"))));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Kind-specific parameters, ordered tool results, auth and strict behavior.
|
||||
*******************************************************************************/
|
||||
@ParameterizedTest
|
||||
@ValueSource(strings = { "OPENAI", "AZURE_OPENAI", "OPENROUTER", "OLLAMA", "LM_STUDIO", "VLLM", "LLAMA_CPP", "OPENAI_COMPATIBLE" })
|
||||
void requestContract(String kind) throws Exception
|
||||
{
|
||||
try(MockOpenAiServer server = new MockOpenAiServer())
|
||||
{
|
||||
server.enqueue(200, TEXT);
|
||||
boolean nativeKind = kind.equals("OPENAI") || kind.equals("AZURE_OPENAI");
|
||||
ModelRequest request = request().withCapabilities(new ModelCapabilities().withReasoningEffort(true)).withTools(List.of(tool()))
|
||||
.withSystem(List.of(new SystemBlock().withText("one"), new SystemBlock().withText("two")))
|
||||
.withMessages(List.of(ModelMessage.user("hi"), new ModelMessage().withRole(ModelMessage.Role.ASSISTANT)
|
||||
.withPart(ModelPart.toolUse("a", "lookup", Map.of("q", "a"))).withPart(ModelPart.toolUse("b", "lookup", Map.of("q", "b"))),
|
||||
new ModelMessage().withRole(ModelMessage.Role.USER).withPart(ModelPart.toolResult("a", "first", false)).withPart(ModelPart.toolResult("b", "second", true))));
|
||||
var provider = new OpenAiCompatibleModelProvider(settings(server, kind));
|
||||
StringBuilder text = new StringBuilder();
|
||||
assertThat(provider.stream(request, text::append).await(Duration.ofSeconds(5)).getText()).isEqualTo("OK");
|
||||
assertThat(text.toString()).isEqualTo("OK");
|
||||
var captured = server.requests.getFirst();
|
||||
JsonNode body = OpenAiHttp.JSON.readTree(captured.body());
|
||||
assertThat(body.path("messages").get(0).path("content").asText()).isEqualTo("one\n\ntwo");
|
||||
assertThat(body.path("messages").get(2).path("tool_calls").get(0).path("function").path("arguments").asText()).isEqualTo("{\"q\":\"a\"}");
|
||||
assertThat(body.path("messages").get(3).path("tool_call_id").asText()).isEqualTo("a");
|
||||
assertThat(body.path("messages").get(4).path("content").asText()).isEqualTo("ERROR: second");
|
||||
assertThat(body.path("tools").get(0).path("function").has("strict")).isEqualTo(nativeKind);
|
||||
assertThat(body.has("max_completion_tokens")).isEqualTo(nativeKind);
|
||||
assertThat(body.has("max_tokens")).isEqualTo(!nativeKind);
|
||||
assertThat(body.path("reasoning_effort").asText()).isEqualTo("high");
|
||||
assertThat(body.has("tool_choice")).isFalse();
|
||||
assertThat(body.has("parallel_tool_calls")).isEqualTo(List.of("OPENAI", "AZURE_OPENAI", "OPENROUTER", "VLLM").contains(kind));
|
||||
assertThat(provider.countTokens(request)).isNull();
|
||||
if(kind.equals("AZURE_OPENAI"))
|
||||
{
|
||||
assertThat(captured.path()).isEqualTo("/v1/openai/deployments/test/chat/completions?api-version=2024-10-21");
|
||||
assertThat(captured.apiKey()).isEqualTo("synthetic-key");
|
||||
assertThat(captured.authorization()).isNull();
|
||||
}
|
||||
else
|
||||
{
|
||||
assertThat(captured.authorization()).isEqualTo("Bearer synthetic-key");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fragmented parallel tools, comments, cache usage and ignored reasoning.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void streamedToolsAndUsage() throws Exception
|
||||
{
|
||||
try(MockOpenAiServer server = new MockOpenAiServer())
|
||||
{
|
||||
StringBuilder script = new StringBuilder(": comment\r\n\r\n");
|
||||
script.append(event(Map.of("choices", List.of(Map.of("delta", Map.of("reasoning_content", "never expose", "tool_calls", List.of(
|
||||
Map.of("index", 1, "id", "b", "function", Map.of("name", "lookup", "arguments", "")),
|
||||
Map.of("index", 0, "id", "a", "function", Map.of("name", "lookup", "arguments", "")))))))));
|
||||
for(char c : "{\"q\":\"value\"}".toCharArray())
|
||||
{
|
||||
script.append(event(Map.of("choices", List.of(Map.of("delta", Map.of("tool_calls", List.of(
|
||||
Map.of("index", 0, "function", Map.of("arguments", String.valueOf(c))), Map.of("index", 1, "function", Map.of("arguments", String.valueOf(c))))))))));
|
||||
}
|
||||
script.append(event(Map.of("choices", List.of(Map.of("delta", Map.of(), "finish_reason", "tool_calls")), "usage",
|
||||
Map.of("prompt_tokens", 100, "completion_tokens", 8, "prompt_tokens_details", Map.of("cached_tokens", 70)))));
|
||||
server.enqueue(200, script + "data: [DONE]\n\n");
|
||||
List<String> starts = new ArrayList<>();
|
||||
StringBuilder text = new StringBuilder();
|
||||
ModelResponse response = new OpenAiCompatibleModelProvider(settings(server, "OLLAMA").withApiKey(null)).stream(request(), new ModelEventListener()
|
||||
{
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Provider contract.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void onTextDelta(String delta)
|
||||
{
|
||||
text.append(delta);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Provider contract.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void onToolUseStart(String id, String name)
|
||||
{
|
||||
starts.add(id);
|
||||
}
|
||||
}).await(Duration.ofSeconds(5));
|
||||
assertThat(response.getToolUses()).extracting(ModelPart::getToolUseId).containsExactly("a", "b");
|
||||
assertThat(response.getToolUses().getFirst().getInput()).containsEntry("q", "value");
|
||||
assertThat(starts).containsExactly("b", "a");
|
||||
assertThat(text).isEmpty();
|
||||
assertThat(response.getUsage().getInputTokens()).isEqualTo(30L);
|
||||
assertThat(response.getUsage().getCacheReadTokens()).isEqualTo(70L);
|
||||
assertThat(response.getUsage().getOutputTokens()).isEqualTo(8L);
|
||||
assertThat(response.getStopReason()).isEqualTo(ModelResponse.STOP_TOOL_USE);
|
||||
assertThat(server.requests.getFirst().authorization()).isNull();
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Single JSON mode and defaults without optional request fields.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void nonStreamingAndNoUsage() throws Exception
|
||||
{
|
||||
try(MockOpenAiServer server = new MockOpenAiServer())
|
||||
{
|
||||
server.enqueue(200, "{\"choices\":[{\"message\":{\"content\":\"done\"},\"finish_reason\":\"length\"}]}");
|
||||
var provider = new OpenAiCompatibleModelProvider(settings(server, "OPENAI"));
|
||||
StringBuilder text = new StringBuilder();
|
||||
ModelResponse response = provider.stream(request().withCapabilities(new ModelCapabilities().withStreaming(false)), text::append).await(Duration.ofSeconds(5));
|
||||
assertThat(response.getStopReason()).isEqualTo(ModelResponse.STOP_MAX_TOKENS);
|
||||
assertThat(text.toString()).isEqualTo("done");
|
||||
assertThat(response.getUsage().billableTokens()).isZero();
|
||||
JsonNode body = OpenAiHttp.JSON.readTree(server.requests.getFirst().body());
|
||||
assertThat(body.path("stream").asBoolean()).isFalse();
|
||||
assertThat(body.has("stream_options")).isFalse();
|
||||
assertThat(body.has("reasoning_effort")).isFalse();
|
||||
assertThat(body.has("parallel_tool_calls")).isFalse();
|
||||
server.enqueue(200, TEXT);
|
||||
assertThat(provider.stream(request(), text::append).await(Duration.ofSeconds(5)).getUsage().billableTokens()).isZero();
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Every HTTP error category carries safe details.
|
||||
*******************************************************************************/
|
||||
@ParameterizedTest
|
||||
@CsvSource({ "401,AUTHENTICATION", "403,AUTHENTICATION", "429,RATE_LIMITED", "503,OVERLOADED", "529,OVERLOADED", "400,INVALID_REQUEST", "404,INVALID_REQUEST", "422,INVALID_REQUEST", "500,SERVER", "502,SERVER" })
|
||||
void errors(int status, String kind) throws Exception
|
||||
{
|
||||
try(MockOpenAiServer server = new MockOpenAiServer())
|
||||
{
|
||||
server.enqueue(status, "{\"error\":{\"type\":\"bad_request\",\"message\":\"Rejected synthetic-key\"}}");
|
||||
assertThatThrownBy(() -> new OpenAiCompatibleModelProvider(settings(server, "OPENAI")).stream(request(), t -> {}).await(Duration.ofSeconds(5)))
|
||||
.isInstanceOfSatisfying(ModelProviderException.class, e ->
|
||||
{
|
||||
assertThat(e.getKind().name()).isEqualTo(kind);
|
||||
assertThat(e.getHttpStatus()).isEqualTo(status);
|
||||
assertThat(e.getRequestId()).isEqualTo("mock-request");
|
||||
assertThat(e.getMessage()).doesNotContain("synthetic-key");
|
||||
assertThat(e.getProviderMessage()).contains("[redacted]");
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Strict rejection is opt-in; incompatible nested object schemas are omitted.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void strictAndStreamErrors() throws Exception
|
||||
{
|
||||
try(MockOpenAiServer server = new MockOpenAiServer())
|
||||
{
|
||||
var provider = new OpenAiCompatibleModelProvider(settings(server, "OPENAI"));
|
||||
String error = "{\"error\":{\"type\":\"invalid_request_error\",\"message\":\"strict schema grammar refused\"}}";
|
||||
for(boolean sse : List.of(false, true))
|
||||
{
|
||||
server.enqueue(sse ? 200 : 400, sse ? "data: " + error + "\n\n" : error);
|
||||
assertThatThrownBy(() -> provider.stream(request().withTools(List.of(tool())), t -> {}).await(Duration.ofSeconds(5)))
|
||||
.isInstanceOfSatisfying(ModelProviderException.class, e -> assertThat(e.isStrictSchemaRejected()).isTrue());
|
||||
}
|
||||
server.enqueue(200, TEXT);
|
||||
provider.stream(request().withTools(List.of(tool().withInputSchema(Map.of("type", "object", "properties", Map.of())))), t -> {}).await(Duration.ofSeconds(5));
|
||||
assertThat(OpenAiHttp.JSON.readTree(server.requests.getLast().body()).path("tools").get(0).path("function").has("strict")).isFalse();
|
||||
assertThat(OpenAiMapping.strictCompatible(OpenAiHttp.JSON.readTree("{\"type\":\"array\",\"items\":{\"type\":\"object\"}}"))).isFalse();
|
||||
assertThat(OpenAiMapping.strictCompatible(OpenAiHttp.JSON.readTree("{\"anyOf\":[{\"type\":\"object\"}]}"))).isFalse();
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Header timeout, stalled body timeout, cancellation and network failure.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void timeoutCancelAndNetwork() throws Exception
|
||||
{
|
||||
try(MockOpenAiServer server = new MockOpenAiServer())
|
||||
{
|
||||
var provider = new OpenAiCompatibleModelProvider(settings(server, "OLLAMA").withRequestTimeoutSeconds(1));
|
||||
for(boolean afterHeaders : List.of(false, true))
|
||||
{
|
||||
server.stall(afterHeaders);
|
||||
assertThatThrownBy(() -> provider.stream(request(), t -> {}).await(Duration.ofSeconds(5)))
|
||||
.isInstanceOfSatisfying(ModelProviderException.class, e -> assertThat(e.getKind()).isEqualTo(ModelProviderException.Kind.TIMEOUT));
|
||||
}
|
||||
var started = server.stall(true);
|
||||
var handle = provider.stream(request(), t -> {});
|
||||
assertThat(started.await(3, java.util.concurrent.TimeUnit.SECONDS)).isTrue();
|
||||
handle.cancel();
|
||||
assertThat(handle.await(Duration.ofSeconds(2)).getStopReason()).isEqualTo(ModelResponse.STOP_CANCELLED);
|
||||
assertThat(handle.isCancelled()).isTrue();
|
||||
}
|
||||
String closed;
|
||||
try(MockOpenAiServer server = new MockOpenAiServer())
|
||||
{
|
||||
closed = server.url();
|
||||
}
|
||||
var provider = new OpenAiCompatibleModelProvider(new ProviderSettings().withKind("OLLAMA").withBaseUrl(closed));
|
||||
assertThatThrownBy(() -> provider.stream(request(), t -> {}).await(Duration.ofSeconds(5)))
|
||||
.isInstanceOfSatisfying(ModelProviderException.class, e -> assertThat(e.getKind()).isEqualTo(ModelProviderException.Kind.NETWORK));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Discovery timeouts include the body, and failed probes never complete.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void discoveryTimeoutAndMalformedResponse() throws Exception
|
||||
{
|
||||
try(MockOpenAiServer server = new MockOpenAiServer())
|
||||
{
|
||||
var type = new OpenAiCompatibleProviderType();
|
||||
var settings = settings(server, "OLLAMA").withRequestTimeoutSeconds(1);
|
||||
server.stall(true);
|
||||
assertThatThrownBy(() -> type.discoverModels(settings)).isInstanceOfSatisfying(ModelProviderException.class,
|
||||
e -> assertThat(e.getKind()).isEqualTo(ModelProviderException.Kind.TIMEOUT));
|
||||
server.enqueue(200, "not JSON");
|
||||
assertThat(type.checkHealth(settings).healthy()).isFalse();
|
||||
server.enqueue(200, "{\"error\":{\"message\":\"failed\"}}");
|
||||
assertThat(type.checkHealth(settings).healthy()).isFalse();
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** JDK header validation must never expose even an invalid credential.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void invalidHeaderDoesNotLeakKey() throws Exception
|
||||
{
|
||||
try(MockOpenAiServer server = new MockOpenAiServer())
|
||||
{
|
||||
var settings = settings(server, "OPENAI").withApiKey("private\ncredential");
|
||||
assertThatThrownBy(() -> new OpenAiCompatibleProviderType().discoverModels(settings))
|
||||
.isInstanceOfSatisfying(ModelProviderException.class, e ->
|
||||
{
|
||||
assertThat(e.getMessage()).doesNotContain("private", "credential");
|
||||
assertThat(e.getCause()).isNull();
|
||||
});
|
||||
assertThat(new OpenAiCompatibleProviderType().checkHealth(settings).healthy()).isFalse();
|
||||
assertThat(server.requests).isEmpty();
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Non-streaming calls preserve tools and usage, including cached tokens.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void nonStreamingTools() throws Exception
|
||||
{
|
||||
try(MockOpenAiServer server = new MockOpenAiServer())
|
||||
{
|
||||
server.enqueue(200, """
|
||||
{"choices":[{"message":{"tool_calls":[
|
||||
{"id":"one","function":{"name":"lookup","arguments":"{\\"q\\":\\"x\\"}"}},
|
||||
{"id":"two","function":{"name":"lookup","arguments":"{}"}}]},"finish_reason":"tool_calls"}],
|
||||
"usage":{"prompt_tokens":12,"completion_tokens":4,"prompt_tokens_details":{"cached_tokens":2}}}
|
||||
""");
|
||||
var response = new OpenAiCompatibleModelProvider(settings(server, "OPENAI")).stream(
|
||||
request().withCapabilities(new ModelCapabilities().withStreaming(false)), t -> {}).await(Duration.ofSeconds(5));
|
||||
assertThat(response.getToolUses()).extracting(ModelPart::getToolUseId).containsExactly("one", "two");
|
||||
assertThat(response.getToolUses().getFirst().getInput()).containsEntry("q", "x");
|
||||
assertThat(response.getUsage().getInputTokens()).isEqualTo(10L);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Model discovery context dialects, Azure path and health without completions.
|
||||
*******************************************************************************/
|
||||
@ParameterizedTest
|
||||
@ValueSource(strings = { "OPENAI", "OPENROUTER", "AZURE_OPENAI", "VLLM", "LLAMA_CPP" })
|
||||
void discoveryAndHealth(String kind) throws Exception
|
||||
{
|
||||
try(MockOpenAiServer server = new MockOpenAiServer())
|
||||
{
|
||||
String models = "{\"data\":[{\"id\":\"a\",\"owned_by\":\"owner\",\"context_length\":64000},{\"id\":\"b\",\"max_model_len\":32000},{\"id\":\"c\",\"meta\":{\"n_ctx_train\":16000}}]}";
|
||||
server.enqueue(200, models);
|
||||
var type = new OpenAiCompatibleProviderType();
|
||||
var settings = settings(server, kind).withApiVersion("2024-10-21");
|
||||
assertThat(type.discoverModels(settings)).extracting(m -> m.contextWindowTokens()).containsExactly(64000, 32000, 16000);
|
||||
server.enqueue(200, models);
|
||||
assertThat(type.checkHealth(settings).message()).isEqualTo("3 models available");
|
||||
server.enqueue(401, "{\"error\":{\"message\":\"bad key\"}}");
|
||||
assertThat(type.checkHealth(settings).healthy()).isFalse();
|
||||
assertThat(server.requests).allSatisfy(r -> assertThat(r.path()).isEqualTo(kind.equals("AZURE_OPENAI") ? "/v1/openai/models?api-version=2024-10-21" : "/v1/models"));
|
||||
assertThat(type.getKinds()).hasSize(8);
|
||||
assertThat(type.getModels()).isEmpty();
|
||||
assertThat(type.getDefaultBaseUrl()).isNull();
|
||||
assertThat(type.getDefaultApiKeyEnvName()).isNull();
|
||||
assertThat(type.getLabel()).isEqualTo("OpenAI-compatible");
|
||||
for(String k : type.getKinds())
|
||||
{
|
||||
assertThat(type.getDefaultLocation(k)).isIn("LOCAL", "CLOUD");
|
||||
type.getDefaultBaseUrl(k);
|
||||
type.getDefaultApiKeyEnvName(k);
|
||||
}
|
||||
assertThat(type.create(null, server.url()).getName()).isEqualTo(type.getType());
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Serialize a scripted SSE event.
|
||||
*******************************************************************************/
|
||||
static String event(Object value) throws Exception
|
||||
{
|
||||
return ("data: " + OpenAiHttp.JSON.writeValueAsString(value) + "\n\n");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Repeated identities, missing ids and older servers' stop reason are tolerated.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void repeatedToolIdentityAndMissingId() throws Exception
|
||||
{
|
||||
try(MockOpenAiServer server = new MockOpenAiServer())
|
||||
{
|
||||
StringBuilder script = new StringBuilder();
|
||||
for(String fragment : List.of("{", "\"q\":", "\"hello\"}"))
|
||||
{
|
||||
script.append(event(Map.of("choices", List.of(Map.of("delta", Map.of("tool_calls", List.of(
|
||||
Map.of("index", 0, "id", "repeated", "function", Map.of("name", "lookup", "arguments", fragment)),
|
||||
Map.of("index", 1, "function", Map.of("name", "lookup", "arguments", fragment)))), "finish_reason", "stop")))));
|
||||
}
|
||||
server.enqueue(200, script.toString());
|
||||
ModelResponse response = new OpenAiCompatibleModelProvider(settings(server, "OLLAMA")).stream(request(), t -> {}).await(Duration.ofSeconds(5));
|
||||
assertThat(response.getStopReason()).isEqualTo(ModelResponse.STOP_TOOL_USE);
|
||||
assertThat(response.getToolUses()).extracting(ModelPart::getToolUseId).containsExactly("repeated", "call_1");
|
||||
assertThat(response.getToolUses()).allSatisfy(tool ->
|
||||
{
|
||||
assertThat(tool.getToolName()).isEqualTo("lookup");
|
||||
assertThat(tool.getInput()).containsExactlyEntriesOf(Map.of("q", "hello"));
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Invalid JSON becomes bounded input for kernel validation, never NETWORK.
|
||||
*******************************************************************************/
|
||||
@ParameterizedTest
|
||||
@ValueSource(strings = { "{broken", "null", "[]", "{} trailing", "long" })
|
||||
void malformedArguments(String input) throws Exception
|
||||
{
|
||||
String arguments = input.equals("long") ? "x".repeat(250) : input;
|
||||
try(MockOpenAiServer server = new MockOpenAiServer())
|
||||
{
|
||||
server.enqueue(200, event(Map.of("choices", List.of(Map.of("delta", Map.of("tool_calls", List.of(
|
||||
Map.of("index", 0, "function", Map.of("name", "lookup", "arguments", arguments)))), "finish_reason", "stop")))));
|
||||
ModelResponse response = new OpenAiCompatibleModelProvider(settings(server, "LLAMA_CPP")).stream(request(), t -> {}).await(Duration.ofSeconds(5));
|
||||
assertThat(response.getStopReason()).isEqualTo(ModelResponse.STOP_TOOL_USE);
|
||||
assertThat(response.getToolUses().getFirst().getInput()).containsExactlyEntriesOf(Map.of("_invalid_arguments", arguments.substring(0, Math.min(200, arguments.length()))));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,87 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.provider.openai;
|
||||
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.kingsrook.qbits.assistant.AssistantQBitConfig;
|
||||
import com.kingsrook.qbits.assistant.testing.AssistantTestFixture;
|
||||
import com.kingsrook.qbits.assistant.testing.GoldenTasks;
|
||||
import com.kingsrook.qbits.assistant.testing.TurnRunner;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import org.junit.jupiter.api.Tag;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Golden fixture task through the real adapter, engine and tool result mapping.
|
||||
*******************************************************************************/
|
||||
@Tag("golden")
|
||||
class OpenAiGoldenTurnTest
|
||||
{
|
||||
/*******************************************************************************
|
||||
** A golden task travels through the engine, real UI tools and a tool round trip.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void goldenToolRoundTrip() throws Exception
|
||||
{
|
||||
try(MockOpenAiServer server = new MockOpenAiServer())
|
||||
{
|
||||
Map<String, Object> arguments = Map.of("table", "blog_post", "filter", Map.of("booleanOperator", "AND", "criteria",
|
||||
List.of(Map.of("field", "status", "operator", "EQUALS", "values", List.of("PUBLISHED")))));
|
||||
Map<String, Object> call = Map.of("index", 0, "id", "count", "function", Map.of("name", "count_records", "arguments", OpenAiHttp.JSON.writeValueAsString(arguments)));
|
||||
server.enqueue(200, OpenAiContractTest.event(Map.of("choices", List.of(Map.of("delta", Map.of("tool_calls", List.of(call)), "finish_reason", "stop")))));
|
||||
server.enqueue(200, OpenAiContractTest.event(Map.of("choices", List.of(Map.of("delta", Map.of("content", "There are " + AssistantTestFixture.PUBLISHED_POSTS + " published posts. /app/blog_post"), "finish_reason", "stop")))));
|
||||
var provider = new OpenAiCompatibleModelProvider(OpenAiContractTest.settings(server, "OLLAMA"));
|
||||
var config = new AssistantQBitConfig().withProviderName(provider.getName()).withModelProvider(provider);
|
||||
try
|
||||
{
|
||||
var instance = AssistantTestFixture.build(config);
|
||||
var task = GoldenTasks.all().stream().filter(t -> t.id().equals("count-published-posts")).findFirst().orElseThrow();
|
||||
var outcome = new TurnRunner(instance, config).run(task.userId(), task.prompt());
|
||||
assertThat(outcome.errorCode()).isNull();
|
||||
task.check().accept(outcome);
|
||||
assertThat(outcome.text()).isEqualTo("There are " + AssistantTestFixture.PUBLISHED_POSTS + " published posts. /app/blog_post");
|
||||
assertThat(outcome.end()).isNotEmpty();
|
||||
assertThat(server.requests).hasSize(2);
|
||||
JsonNode messages = OpenAiHttp.JSON.readTree(server.requests.get(1).body()).path("messages");
|
||||
JsonNode assistant = messages.get(messages.size() - 2);
|
||||
assertThat(assistant.path("role").asText()).isEqualTo("assistant");
|
||||
assertThat(assistant.path("tool_calls")).hasSize(1);
|
||||
assertThat(assistant.path("tool_calls").get(0).path("id").asText()).isEqualTo("count");
|
||||
assertThat(assistant.path("tool_calls").get(0).path("function").path("name").asText()).isEqualTo("count_records");
|
||||
assertThat(messages.get(messages.size() - 1).path("content").asText()).contains("count").doesNotStartWith("ERROR:");
|
||||
assertThat(messages.get(messages.size() - 1).path("role").asText()).isEqualTo("tool");
|
||||
assertThat(messages.get(messages.size() - 1).path("tool_call_id").asText()).isEqualTo("count");
|
||||
}
|
||||
finally
|
||||
{
|
||||
if(config.getRuntime() != null)
|
||||
{
|
||||
config.getRuntime().close();
|
||||
}
|
||||
QContext.clear();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
}
|
||||
@@ -0,0 +1,216 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.provider.openai;
|
||||
|
||||
|
||||
import java.io.InputStream;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.security.KeyStore;
|
||||
import java.time.Duration;
|
||||
import java.util.Set;
|
||||
import javax.net.ssl.KeyManagerFactory;
|
||||
import javax.net.ssl.SSLContext;
|
||||
import com.kingsrook.qbits.assistant.AssistantQBitConfig;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantProvider;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
|
||||
import com.kingsrook.qbits.assistant.provider.ProviderSettings;
|
||||
import com.kingsrook.qbits.assistant.testing.AssistantTestFixture;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.api.io.TempDir;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Private CA coverage against loopback HTTPS; all key material is test-only.
|
||||
*******************************************************************************/
|
||||
class OpenAiTlsTest
|
||||
{
|
||||
@TempDir
|
||||
Path temporary;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The public test root, never a real deployment CA.
|
||||
*******************************************************************************/
|
||||
private String pem() throws Exception
|
||||
{
|
||||
try(InputStream input = getClass().getResourceAsStream("/tls/ca.pem"))
|
||||
{
|
||||
return (new String(input.readAllBytes(), StandardCharsets.US_ASCII));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Leaf chain signed by the test root, valid for localhost but not its IP.
|
||||
*******************************************************************************/
|
||||
private SSLContext serverContext() throws Exception
|
||||
{
|
||||
KeyStore store = KeyStore.getInstance("PKCS12");
|
||||
try(InputStream input = getClass().getResourceAsStream("/tls/server.p12"))
|
||||
{
|
||||
store.load(input, "test-only".toCharArray());
|
||||
}
|
||||
KeyManagerFactory keys = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
|
||||
keys.init(store, "test-only".toCharArray());
|
||||
SSLContext context = SSLContext.getInstance("TLS");
|
||||
context.init(keys.getKeyManagers(), null, null);
|
||||
return (context);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Every operation rejects the unknown CA with a safe, actionable diagnostic.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void untrustedCertificate() throws Exception
|
||||
{
|
||||
try(MockOpenAiServer server = new MockOpenAiServer(serverContext()))
|
||||
{
|
||||
ProviderSettings settings = OpenAiContractTest.settings(server, "OLLAMA");
|
||||
var type = new OpenAiCompatibleProviderType();
|
||||
assertThatThrownBy(() -> type.discoverModels(settings)).isInstanceOfSatisfying(ModelProviderException.class, this::trustError);
|
||||
assertThat(type.checkHealth(settings).healthy()).isFalse();
|
||||
assertThat(type.checkHealth(settings).message()).contains("NETWORK", "tlsCaBundle");
|
||||
assertThatThrownBy(() -> type.create(settings).stream(OpenAiContractTest.request(), t -> {}).await(Duration.ofSeconds(5)))
|
||||
.isInstanceOfSatisfying(ModelProviderException.class, this::trustError);
|
||||
assertThat(server.requests).isEmpty();
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** No raw handshake details, keys, or certificate contents escape.
|
||||
*******************************************************************************/
|
||||
private void trustError(ModelProviderException error)
|
||||
{
|
||||
assertThat(error.getKind()).isEqualTo(ModelProviderException.Kind.NETWORK);
|
||||
assertThat(error.getMessage()).contains("certificate is not trusted", "tlsCaBundle").doesNotContain("synthetic-key", "TEST ONLY", "BEGIN CERTIFICATE");
|
||||
assertThat(error.getCause()).isNull();
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Shared trust enables all operations while endpoint identification stays on.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void trustedHealthDiscoveryChatAndHostnameCheck() throws Exception
|
||||
{
|
||||
var config = new AssistantQBitConfig().withTrustedCaBundle("test-root", pem());
|
||||
try(MockOpenAiServer server = new MockOpenAiServer(serverContext()))
|
||||
{
|
||||
ProviderSettings settings = OpenAiContractTest.settings(server, "OLLAMA").withTlsCaBundle("test-root")
|
||||
.withSslContext(config.getTrustedCaBundles().get("test-root"));
|
||||
var type = new OpenAiCompatibleProviderType();
|
||||
String models = "{\"data\":[{\"id\":\"test\"}]}";
|
||||
server.enqueue(200, models);
|
||||
assertThat(type.checkHealth(settings).healthy()).isTrue();
|
||||
server.enqueue(200, models);
|
||||
assertThat(type.discoverModels(settings)).extracting(m -> m.modelId()).containsExactly("test");
|
||||
server.enqueue(200, OpenAiContractTest.TEXT);
|
||||
assertThat(type.create(settings).stream(OpenAiContractTest.request(), t -> {}).await(Duration.ofSeconds(5)).getText()).isEqualTo("OK");
|
||||
assertThat(server.requests).hasSize(3).allSatisfy(r -> assertThat(r.authorization()).isEqualTo("Bearer synthetic-key"));
|
||||
settings.withBaseUrl(server.url().replace("localhost", "127.0.0.1"));
|
||||
assertThatThrownBy(() -> type.discoverModels(settings)).isInstanceOfSatisfying(ModelProviderException.class, this::trustError);
|
||||
assertThatThrownBy(() -> type.create(settings).stream(OpenAiContractTest.request(), t -> {}).await(Duration.ofSeconds(5)))
|
||||
.isInstanceOfSatisfying(ModelProviderException.class, this::trustError);
|
||||
assertThat(server.requests).hasSize(3);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Parse at startup, accept multiple PEMs, omit contexts from serialized beans.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void eagerConfigAndSerialization() throws Exception
|
||||
{
|
||||
String pem = pem();
|
||||
Path file = temporary.resolve("bundle.pem");
|
||||
Files.writeString(file, pem + pem);
|
||||
var config = new AssistantQBitConfig().withTrustedCaBundleFile("test-root", file);
|
||||
Files.delete(file);
|
||||
assertThat(config.getTrustedCaBundles()).containsKey("test-root");
|
||||
assertThatThrownBy(() -> config.getTrustedCaBundles().clear()).isInstanceOf(UnsupportedOperationException.class);
|
||||
assertThat(OpenAiHttp.JSON.writeValueAsString(config)).doesNotContain("trustedCaBundles", "BEGIN CERTIFICATE", "test-root");
|
||||
var settings = new ProviderSettings().withSslContext(config.getTrustedCaBundles().get("test-root"));
|
||||
assertThat(OpenAiHttp.JSON.writeValueAsString(settings)).doesNotContain("sslContext", "BEGIN CERTIFICATE");
|
||||
for(String invalid : new String[] { null, "", "private-content", pem + "private-content", "-----BEGIN CERTIFICATE-----bad-----END CERTIFICATE-----" })
|
||||
{
|
||||
assertThatThrownBy(() -> config.withTrustedCaBundle("bad", invalid)).isInstanceOf(IllegalArgumentException.class)
|
||||
.hasMessageContaining("Invalid CA bundle").hasMessageNotContaining("private-content").hasNoCause();
|
||||
}
|
||||
assertThatThrownBy(() -> config.withTrustedCaBundle(" ", pem)).hasMessageContaining("CA bundle name");
|
||||
assertThatThrownBy(() -> config.withTrustedCaBundleFile("missing", file)).hasMessage("Could not read the configured CA bundle file.").hasNoCause();
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Settings saves reject unknown names and HTTP; changing names rebuilds clients.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void registryValidationAndClientCache() throws Exception
|
||||
{
|
||||
var config = new AssistantQBitConfig().withProviderType(new OpenAiCompatibleProviderType())
|
||||
.withAllowedBaseUrls(Set.of("https://localhost:8443/v1"))
|
||||
.withTrustedCaBundle("first", pem()).withTrustedCaBundle("second", pem());
|
||||
try
|
||||
{
|
||||
var instance = AssistantTestFixture.build(config);
|
||||
QContext.init(instance, AssistantTestFixture.sessionFor(AssistantTestFixture.ADMIN));
|
||||
var service = config.getRuntime().getProviderService();
|
||||
var row = new AssistantProvider().withName("Test TLS").withProviderType("openai-compatible").withKind("OLLAMA")
|
||||
.withBaseUrl("https://localhost:8443/v1").withTlsCaBundle("unknown");
|
||||
assertThat(service.validate(row)).anyMatch(e -> e.contains("configured on this server"));
|
||||
assertThatThrownBy(() -> service.settings(row)).hasMessageContaining("TLS CA bundle");
|
||||
assertThatThrownBy(() -> config.getRuntime().getStore().insert(row)).hasMessageContaining("TLS CA bundle");
|
||||
row.withTlsCaBundle("first").withBaseUrl(null);
|
||||
assertThat(service.validate(row)).anyMatch(e -> e.contains("HTTPS"));
|
||||
row.withBaseUrl("https://localhost:8443/v1");
|
||||
assertThat(service.validate(row)).isEmpty();
|
||||
row.withId(config.getRuntime().getStore().insert(row).getId());
|
||||
assertThat(row.getTlsCaBundle()).isEqualTo("first");
|
||||
assertThat(service.settings(row).getSslContext()).isSameAs(config.getTrustedCaBundles().get("first"));
|
||||
var first = service.forRow(row, "test").provider();
|
||||
assertThat(service.forRow(row, "test").provider()).isSameAs(first);
|
||||
row.withTlsCaBundle("second");
|
||||
assertThat(service.forRow(row, "test").provider()).isNotSameAs(first);
|
||||
row.withTlsCaBundle(null);
|
||||
assertThat(service.settings(row).getSslContext()).isNull();
|
||||
assertThat(instance.getTable(AssistantProvider.TABLE_NAME).getField("tlsCaBundle").getMaxLength()).isEqualTo(100);
|
||||
}
|
||||
finally
|
||||
{
|
||||
if(config.getRuntime() != null)
|
||||
{
|
||||
config.getRuntime().close();
|
||||
}
|
||||
QContext.clear();
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,251 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.provider.openai.demo;
|
||||
|
||||
|
||||
import java.io.IOException;
|
||||
import java.math.BigDecimal;
|
||||
import java.net.InetSocketAddress;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import com.kingsrook.qbits.assistant.AssistantQBitConfig;
|
||||
import com.kingsrook.qbits.assistant.metadata.AssistantMetaDataBuilder;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantModel;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantProvider;
|
||||
import com.kingsrook.qbits.assistant.persistence.AssistantStore;
|
||||
import com.kingsrook.qbits.assistant.processes.DiscoverModelsProcess;
|
||||
import com.kingsrook.qbits.assistant.processes.TestConnectionProcess;
|
||||
import com.kingsrook.qbits.assistant.processes.TestModelConnectionProcess;
|
||||
import com.kingsrook.qbits.assistant.provider.AssistantProviderType;
|
||||
import com.kingsrook.qbits.assistant.provider.fake.ScriptedModelProvider;
|
||||
import com.kingsrook.qbits.assistant.provider.openai.OpenAiCompatibleProviderType;
|
||||
import com.kingsrook.qbits.assistant.testing.AssistantTestFixture;
|
||||
import com.kingsrook.qbits.assistant.testing.TurnRunner;
|
||||
import com.kingsrook.qqq.backend.core.actions.processes.QProcessCallbackFactory;
|
||||
import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.instances.AbstractQQQApplication;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
|
||||
import com.kingsrook.qqq.backend.core.model.session.QSession;
|
||||
import com.kingsrook.qqq.backend.core.modules.authentication.QAuthenticationModuleCustomizerInterface;
|
||||
import com.kingsrook.qqq.middleware.javalin.QApplicationJavalinServer;
|
||||
import com.sun.net.httpserver.HttpExchange;
|
||||
import com.sun.net.httpserver.HttpServer;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Memory-backed P-a screenshot demo. All discovery, tests and usage come from
|
||||
** real application calls to a localhost mock; no external credentials are used.
|
||||
*******************************************************************************/
|
||||
public class OpenAiDemoServer extends AbstractQQQApplication
|
||||
{
|
||||
private static final String ADMIN = "admin-demo";
|
||||
private static final String MODEL = "qwen3:30b-a3b";
|
||||
private static QInstance instance;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Give browser sessions the same admin identity used for demo traffic.
|
||||
*******************************************************************************/
|
||||
public static class DemoSessionCustomizer implements QAuthenticationModuleCustomizerInterface
|
||||
{
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void customizeSession(QInstance qInstance, QSession qSession, Map<String, Object> context)
|
||||
{
|
||||
QSession admin = AssistantTestFixture.sessionFor(ADMIN);
|
||||
qSession.setUser(admin.getUser());
|
||||
qSession.setPermissions(admin.getPermissions());
|
||||
qSession.withSecurityKeyValue(AssistantMetaDataBuilder.allAccessKeyName(AssistantQBitConfig.DEFAULT_OWNER_KEY_TYPE), true);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public QInstance defineQInstance() throws QException
|
||||
{
|
||||
return (instance);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Seed settings, execute the record actions and a turn, then serve Next UI.
|
||||
*******************************************************************************/
|
||||
public static void main(String[] args) throws Exception
|
||||
{
|
||||
int port = args.length > 0 ? Integer.parseInt(args[0]) : 8099;
|
||||
HttpServer mock = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
|
||||
mock.createContext("/", OpenAiDemoServer::serveMock);
|
||||
mock.start();
|
||||
Runtime.getRuntime().addShutdownHook(new Thread(() -> mock.stop(0)));
|
||||
String baseUrl = "http://127.0.0.1:" + mock.getAddress().getPort() + "/v1";
|
||||
AssistantQBitConfig config = new AssistantQBitConfig()
|
||||
.withProviderName(ScriptedModelProvider.DEFAULT_NAME)
|
||||
.withModelProvider(new ScriptedModelProvider())
|
||||
.withProviderType(new OpenAiCompatibleProviderType())
|
||||
.withAllowedBaseUrls(Set.of(baseUrl))
|
||||
.withSecretResolver(name -> null)
|
||||
.withAllowedWidgetNames(Set.of());
|
||||
AssistantProviderType anthropic = optionalAnthropicType();
|
||||
if(anthropic != null)
|
||||
{
|
||||
config.withProviderType(anthropic);
|
||||
}
|
||||
instance = AssistantTestFixture.build(config);
|
||||
Integer providerId;
|
||||
Integer modelId;
|
||||
QContext.init(instance, AssistantTestFixture.sessionFor(ADMIN));
|
||||
try
|
||||
{
|
||||
AssistantStore store = config.getRuntime().getStore();
|
||||
AssistantProvider local = store.insert(new AssistantProvider().withName("Local GPU (Ollama)")
|
||||
.withProviderType("openai-compatible").withKind("OLLAMA").withLocation("LOCAL").withHipaaEligible(true)
|
||||
.withBaseUrl(baseUrl).withModel(MODEL).withEnabled(true).withIsDefault(true));
|
||||
providerId = local.getId();
|
||||
store.insert(new AssistantProvider().withName("OpenRouter").withProviderType("openai-compatible")
|
||||
.withKind("OPENROUTER").withLocation("CLOUD").withEnabled(false).withIsDefault(false));
|
||||
if(anthropic != null)
|
||||
{
|
||||
store.insert(new AssistantProvider().withName("Claude (production)").withProviderType(anthropic.getType())
|
||||
.withModel("claude-sonnet-5-5").withEnabled(false).withIsDefault(false));
|
||||
}
|
||||
|
||||
runProcess(DiscoverModelsProcess.NAME, providerId);
|
||||
AssistantModel model = store.listModels().stream().filter(m -> providerId.equals(m.getProviderId()) && MODEL.equals(m.getModelId()))
|
||||
.findFirst().orElseThrow(() -> new QException("Demo model discovery failed."));
|
||||
modelId = model.getId();
|
||||
store.update(model.withInputUsdPerMTok(new BigDecimal("0.05")).withOutputUsdPerMTok(new BigDecimal("0.10")));
|
||||
runProcess(TestConnectionProcess.NAME, providerId);
|
||||
runProcess(TestModelConnectionProcess.NAME, modelId);
|
||||
if(!"OK".equals(store.getProvider(providerId).getLastTestStatus())
|
||||
|| store.listModels().stream().noneMatch(m -> modelId.equals(m.getId()) && "OK".equals(m.getLastTestStatus())))
|
||||
{
|
||||
throw (new QException("Demo connection tests failed."));
|
||||
}
|
||||
}
|
||||
finally
|
||||
{
|
||||
QContext.clear();
|
||||
}
|
||||
TurnRunner.Outcome outcome = new TurnRunner(instance, config).run(ADMIN, "Say hello from the local demo provider.");
|
||||
if(outcome.errorCode() != null || outcome.text().isBlank())
|
||||
{
|
||||
throw (new QException("Demo assistant turn failed: " + outcome.errorCode()));
|
||||
}
|
||||
instance.withInstanceDefaultAuthentication(new QAuthenticationMetaData().withName("mock").withType(QAuthenticationType.MOCK)
|
||||
.withCustomizer(new QCodeReference(DemoSessionCustomizer.class)));
|
||||
instance.setHomeAppName(AssistantMetaDataBuilder.APP_NAME);
|
||||
new QApplicationJavalinServer(new OpenAiDemoServer()).withPort(port).withServeFrontendNext(true).start();
|
||||
System.out.println("Assistant home: http://localhost:" + port + "/app/" + AssistantMetaDataBuilder.APP_NAME);
|
||||
System.out.println("Local provider: http://localhost:" + port + "/app/assistant_provider/" + providerId);
|
||||
System.out.println("Local model: http://localhost:" + port + "/app/assistant_model/" + modelId);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Reuse the Anthropic type only when that optional module is on the classpath.
|
||||
*******************************************************************************/
|
||||
private static AssistantProviderType optionalAnthropicType() throws ReflectiveOperationException
|
||||
{
|
||||
try
|
||||
{
|
||||
return ((AssistantProviderType) Class.forName("com.kingsrook.qbits.assistant.provider.anthropic.AnthropicProviderType")
|
||||
.getDeclaredConstructor().newInstance());
|
||||
}
|
||||
catch(ClassNotFoundException e)
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Execute the same streamed ETL process as the record action, skipping UI.
|
||||
*******************************************************************************/
|
||||
private static void runProcess(String process, Integer id) throws QException
|
||||
{
|
||||
RunProcessInput input = new RunProcessInput();
|
||||
input.setProcessName(process);
|
||||
input.setCallback(QProcessCallbackFactory.forPrimaryKey("id", id));
|
||||
input.setFrontendStepBehavior(RunProcessInput.FrontendStepBehavior.SKIP);
|
||||
new RunProcessAction().execute(input);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Repeatable localhost endpoints for startup traffic and screenshot actions.
|
||||
** Token counts are mock API reports, never measurements of GPU inference.
|
||||
*******************************************************************************/
|
||||
private static void serveMock(HttpExchange exchange) throws IOException
|
||||
{
|
||||
try(exchange)
|
||||
{
|
||||
exchange.getRequestBody().readAllBytes();
|
||||
String path = exchange.getRequestURI().getPath();
|
||||
String body;
|
||||
if("GET".equals(exchange.getRequestMethod()) && "/v1/models".equals(path))
|
||||
{
|
||||
exchange.getResponseHeaders().set("Content-Type", "application/json");
|
||||
body = """
|
||||
{"object":"list","data":[{"id":"qwen3:30b-a3b","object":"model","owned_by":"local"},
|
||||
{"id":"gpt-oss:20b","object":"model","owned_by":"local"},
|
||||
{"id":"qwen3-coder:30b","object":"model","owned_by":"local"},
|
||||
{"id":"bge-m3","object":"model","owned_by":"local"}]}
|
||||
""";
|
||||
}
|
||||
else if("POST".equals(exchange.getRequestMethod()) && "/v1/chat/completions".equals(path))
|
||||
{
|
||||
exchange.getResponseHeaders().set("Content-Type", "text/event-stream");
|
||||
body = """
|
||||
data: {"id":"demo-completion","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant","content":"Hello from the local demo provider."}}]}
|
||||
|
||||
data: {"choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
|
||||
|
||||
data: {"choices":[],"usage":{"prompt_tokens":24,"completion_tokens":8,"total_tokens":32}}
|
||||
|
||||
data: [DONE]
|
||||
|
||||
""";
|
||||
}
|
||||
else
|
||||
{
|
||||
exchange.sendResponseHeaders(404, -1);
|
||||
return;
|
||||
}
|
||||
exchange.sendResponseHeaders(200, 0);
|
||||
exchange.getResponseBody().write(body.getBytes(StandardCharsets.UTF_8));
|
||||
exchange.getResponseBody().flush();
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
# OpenAI-compatible provider registry demo
|
||||
|
||||
Run from the repository root in PowerShell with Java 21. This is a manual,
|
||||
memory-backed screenshot server, not a Surefire test. Nothing is persisted
|
||||
between runs. No API keys or external model services are needed: the secret
|
||||
resolver deliberately returns no keys, including when the shell has credentials.
|
||||
|
||||
Build the test classpath (the local reactor class directories come first below,
|
||||
so the demo uses this checkout's code and fixture):
|
||||
|
||||
```powershell
|
||||
.\mvnw.cmd -q -pl qbit-assistant-provider-openai -am test-compile dependency:build-classpath '-Dmdep.includeScope=test' '-Dmdep.outputFile=target/demo-classpath.txt'
|
||||
if ($LASTEXITCODE -ne 0) { throw 'Demo compilation/classpath generation failed' }
|
||||
|
||||
$demoDependencies = (Get-Content qbit-assistant-provider-openai/target/demo-classpath.txt -Raw).Trim()
|
||||
$demoFrontend = "$env:USERPROFILE/.m2/repository/com/kingsrook/qqq/qqq-frontend-next/1.0.0-RC.18/qqq-frontend-next-1.0.0-RC.18.jar"
|
||||
$demoClasspath = "qbit-assistant-provider-openai/target/test-classes;qbit-assistant-provider-openai/target/classes;qbit-assistant/target/test-classes;qbit-assistant/target/classes;$demoFrontend;$demoDependencies"
|
||||
java -cp $demoClasspath com.kingsrook.qbits.assistant.provider.openai.demo.OpenAiDemoServer 8099
|
||||
```
|
||||
|
||||
The frontend artifact is `com.kingsrook.qqq:qqq-frontend-next:1.0.0-RC.18`,
|
||||
already in the local Maven repository. Adjust `$demoFrontend` if your Maven
|
||||
local repository is elsewhere. Omit the final port argument for the default
|
||||
8099, or use another free port (the smoke check used 18099). Stop with Ctrl+C.
|
||||
|
||||
Startup prints these URLs (with the selected port):
|
||||
|
||||
- Assistant home: `http://localhost:8099/app/assistantAdmin`
|
||||
- Local provider: `http://localhost:8099/app/assistant_provider/1`
|
||||
- Priced and tested model: `http://localhost:8099/app/assistant_model/1`
|
||||
|
||||
Every browser session is `admin-demo`. Startup inserts an enabled, default,
|
||||
keyless Local GPU (Ollama) provider, marked LOCAL and HIPAA eligible for the
|
||||
synthetic demo, and a disabled CLOUD OpenRouter provider whose key is Not set.
|
||||
If the Anthropic module is explicitly added to the classpath, its provider type
|
||||
and a disabled Claude (production) row are also included; otherwise it is skipped.
|
||||
|
||||
The normal Discover models process fetches four model IDs from the localhost
|
||||
mock: `qwen3:30b-a3b`, `gpt-oss:20b`, `qwen3-coder:30b`, and `bge-m3`.
|
||||
A normal update sets qwen3's input/output compute prices to $0.05/$0.10 per
|
||||
million tokens. The provider and model Test connection processes then each
|
||||
make a real HTTP completion call, recording two TEST usage rows and the
|
||||
last-test fields. One assistant turn records a third usage row with purpose
|
||||
TURN. Discovery timestamps/status also come from the actual process.
|
||||
|
||||
Replies and token counts are synthetic mock API responses (24 input / 8 output
|
||||
tokens per completion), not live GPU measurements or billed cloud usage.
|
||||
No usage/history rows are manually seeded. The mock remains available for
|
||||
repeating discovery, connection tests and chat while taking screenshots.
|
||||
|
||||
Capture the P-a screens in light and dark at 390 and 1440 px; see
|
||||
[`docs/screenshots/README.md`](../../../../../../../../../../../docs/screenshots/README.md)
|
||||
for the planned screenshot prefixes.
|
||||
@@ -0,0 +1,23 @@
|
||||
# Test-only TLS fixtures
|
||||
|
||||
These certificates and the private key in `server.p12` are synthetic test material.
|
||||
They must never be used for deployment or trusted outside tests. The PKCS12
|
||||
password is `test-only`. The CA private key is not retained in the repository.
|
||||
BouncyCastle was not on the test classpath, so fixtures were generated with the
|
||||
JDK 21 **keytool executable**, not internal `sun.security` APIs.
|
||||
|
||||
The leaf is signed by `QBit TEST ONLY Root CA` and has only `DNS:localhost` in its
|
||||
SAN; connecting to `127.0.0.1` deliberately fails hostname verification. Both
|
||||
certificates start on 2020-01-01 and last 36,500 days to avoid date-dependent tests.
|
||||
Each regeneration creates fresh random keys. Run in a temporary directory, then
|
||||
copy only `ca.pem` and `server.p12` here:
|
||||
|
||||
```sh
|
||||
keytool -genkeypair -alias ca -keyalg RSA -keysize 2048 -dname 'CN=QBit TEST ONLY Root CA' -ext bc:c=ca:true -ext ku:c=keyCertSign,cRLSign -startdate '2020/01/01 00:00:00' -validity 36500 -keystore ca.p12 -storepass test-only -storetype PKCS12 -noprompt
|
||||
keytool -exportcert -rfc -alias ca -keystore ca.p12 -storepass test-only -file ca.pem
|
||||
keytool -genkeypair -alias server -keyalg RSA -keysize 2048 -dname 'CN=localhost' -startdate '2020/01/01 00:00:00' -validity 36500 -keystore server.p12 -storetype PKCS12 -storepass test-only -noprompt
|
||||
keytool -certreq -alias server -keystore server.p12 -storepass test-only -file server.csr
|
||||
keytool -gencert -alias ca -keystore ca.p12 -storepass test-only -infile server.csr -outfile server.pem -rfc -startdate '2020/01/01 00:00:00' -validity 36500 -ext san=dns:localhost -ext eku=serverAuth -ext ku=digitalSignature,keyEncipherment
|
||||
keytool -importcert -alias ca -file ca.pem -keystore server.p12 -storepass test-only -noprompt
|
||||
keytool -importcert -alias server -file server.pem -keystore server.p12 -storepass test-only -noprompt
|
||||
```
|
||||
@@ -0,0 +1,19 @@
|
||||
-----BEGIN CERTIFICATE-----
|
||||
MIIDCDCCAfCgAwIBAgIII8Xlo9Lidn4wDQYJKoZIhvcNAQEMBQAwITEfMB0GA1UE
|
||||
AxMWUUJpdCBURVNUIE9OTFkgUm9vdCBDQTAgFw0yMDAxMDEwNjAwMDBaGA8yMTE5
|
||||
MTIwODA2MDAwMFowITEfMB0GA1UEAxMWUUJpdCBURVNUIE9OTFkgUm9vdCBDQTCC
|
||||
ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMyuwvRX9gZCJoJW68JPf8HC
|
||||
9rA/WAqSwIgwOi6JUVGfRFkp3DZGvfvfw/SiYkED+Zf0AeVrfOZbNZO4dXoe9Rxq
|
||||
YBu4nr79jCfKnlmGsIr/YmWxePUzixvxJ7suHb4y1uah7C36h1EeQKAEYbpFhI7o
|
||||
WVQkZonWDr4w0PmcstjS4lIT43lZdbAxX7RRhiBKe9Ph9P+sqbn1Uk4GOPo5YQkg
|
||||
5eZSPAU4gVh4K07e4aVej+kRHMqMmP4joY4KHBrZXOHKN/mXDGkipys0bf+z7uti
|
||||
/qowu6lNEdF/IihvCgU/fe5wVFbdh7E5t+mpRo+A9B0YrQAtqkJ6gw+Nrwr2TvEC
|
||||
AwEAAaNCMEAwHQYDVR0OBBYEFMul052SYQj4mA4NgW+mX5mfSzgyMA4GA1UdDwEB
|
||||
/wQEAwIBBjAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBDAUAA4IBAQCAKxnC
|
||||
2iBt6cxyh3T/Z11fdBEEFASYU2C/nGqt1FnYhyfR9eluDerNJqsnHmP9pjmf5KAO
|
||||
bGYiw9WodHEbLyM9etNPJSCJRouCHFmRyWarfQV95NjkMfVMXdA2zNWmWv0vxZ2y
|
||||
dOSQUP3nCegLW1JqBoDsdJCJvzU8lcuUwVd9BNywcXmrQbL0iHETwUUWtIEACv2A
|
||||
ja9ZNHYs1oZbaA9YotK3vCcCt0B9xHxMDza9Er7ljTWE6zexTh7IWJ/MbABUdQbQ
|
||||
SGDVrVFlk2PoDIO5rwRwS8LwaKkdwuPcAHbhBRN3HVyr06SBD7AiVWhUV7g5R+h3
|
||||
qHqj8SDW3k3QL63f
|
||||
-----END CERTIFICATE-----
|
||||
@@ -17,6 +17,9 @@
|
||||
package com.kingsrook.qbits.assistant;
|
||||
|
||||
|
||||
import java.io.IOException;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.util.ArrayList;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.LinkedHashSet;
|
||||
@@ -25,10 +28,12 @@ import java.util.Map;
|
||||
import java.util.Set;
|
||||
import java.util.function.Function;
|
||||
import java.util.regex.Pattern;
|
||||
import javax.net.ssl.SSLContext;
|
||||
import com.fasterxml.jackson.annotation.JsonIgnore;
|
||||
import com.kingsrook.qbits.assistant.engine.ModelPricing;
|
||||
import com.kingsrook.qbits.assistant.provider.AssistantModelProvider;
|
||||
import com.kingsrook.qbits.assistant.provider.AssistantProviderType;
|
||||
import com.kingsrook.qbits.assistant.provider.TrustedCaBundle;
|
||||
import com.kingsrook.qbits.assistant.uiapi.UiApiGateway;
|
||||
import com.kingsrook.qqq.backend.core.actions.ai.AiExposure;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
|
||||
@@ -69,6 +74,9 @@ public class AssistantQBitConfig implements QBitConfig
|
||||
@Deprecated(since = "0.1.0-RC.3")
|
||||
public static final String LEGACY_DEFAULT_OWNER_KEY_TYPE = "assistantUserId";
|
||||
|
||||
@JsonIgnore
|
||||
private final transient Map<String, SSLContext> trustedCaBundles = new LinkedHashMap<>();
|
||||
|
||||
private Boolean enabled = true;
|
||||
private String backendName;
|
||||
private String providerName = DEFAULT_PROVIDER_NAME;
|
||||
@@ -1399,7 +1407,14 @@ public class AssistantQBitConfig implements QBitConfig
|
||||
public Set<String> getEffectiveAllowedApiKeyEnvNames()
|
||||
{
|
||||
Set<String> names = new LinkedHashSet<>(allowedApiKeyEnvNames == null ? Set.of() : allowedApiKeyEnvNames);
|
||||
providerTypes.values().forEach(t -> names.add(t.getDefaultApiKeyEnvName()));
|
||||
providerTypes.values().forEach(t -> t.getKinds().forEach(k ->
|
||||
{
|
||||
String name = t.getDefaultApiKeyEnvName(k);
|
||||
if(name != null)
|
||||
{
|
||||
names.add(name);
|
||||
}
|
||||
}));
|
||||
return (names);
|
||||
}
|
||||
|
||||
@@ -1580,4 +1595,45 @@ public class AssistantQBitConfig implements QBitConfig
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Register server-only trust material, parsed eagerly without retaining PEM.
|
||||
*******************************************************************************/
|
||||
public AssistantQBitConfig withTrustedCaBundle(String name, String pem)
|
||||
{
|
||||
if(name == null || name.isBlank() || name.length() > 100 || !name.equals(name.trim()))
|
||||
{
|
||||
throw (new IllegalArgumentException("CA bundle name must be 1 to 100 characters without surrounding whitespace."));
|
||||
}
|
||||
trustedCaBundles.put(name, TrustedCaBundle.parse(pem));
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Read the bundle at startup; no file content or path enters diagnostics.
|
||||
*******************************************************************************/
|
||||
public AssistantQBitConfig withTrustedCaBundleFile(String name, Path pemFile)
|
||||
{
|
||||
try
|
||||
{
|
||||
return (withTrustedCaBundle(name, Files.readString(pemFile)));
|
||||
}
|
||||
catch(IOException | SecurityException e)
|
||||
{
|
||||
throw (new IllegalArgumentException("Could not read the configured CA bundle file."));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Server-only contexts; never expose trust material in metadata or JSON.
|
||||
*******************************************************************************/
|
||||
@JsonIgnore
|
||||
public Map<String, SSLContext> getTrustedCaBundles()
|
||||
{
|
||||
return (Map.copyOf(trustedCaBundles));
|
||||
}
|
||||
}
|
||||
@@ -25,7 +25,9 @@ import com.kingsrook.qbits.assistant.home.AssistantAccessWidget;
|
||||
import com.kingsrook.qbits.assistant.home.AssistantHome;
|
||||
import com.kingsrook.qbits.assistant.metadata.AssistantMetaDataBuilder;
|
||||
import com.kingsrook.qbits.assistant.metadata.AssistantRecordScreens;
|
||||
import com.kingsrook.qbits.assistant.processes.DiscoverModelsProcess;
|
||||
import com.kingsrook.qbits.assistant.processes.TestConnectionProcess;
|
||||
import com.kingsrook.qbits.assistant.processes.TestModelConnectionProcess;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.logging.QLogger;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerMultiOutput;
|
||||
@@ -165,6 +167,11 @@ public class AssistantQBitProducer implements QBitMetaDataProducer<AssistantQBit
|
||||
QProcessMetaData testConnection = TestConnectionProcess.process();
|
||||
testConnection.setSourceQBitName(qBitMetaData.getName());
|
||||
qInstance.addProcess(testConnection);
|
||||
for(QProcessMetaData process : java.util.List.of(DiscoverModelsProcess.process(), TestModelConnectionProcess.process()))
|
||||
{
|
||||
process.setSourceQBitName(qBitMetaData.getName());
|
||||
qInstance.addProcess(process);
|
||||
}
|
||||
|
||||
qInstance.addWidget(AssistantHome.widget());
|
||||
qInstance.addWidget(AssistantAccessWidget.widget());
|
||||
|
||||
@@ -21,11 +21,15 @@ import java.time.Duration;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import com.kingsrook.qbits.assistant.AssistantRuntime;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantModel;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantProvider;
|
||||
import com.kingsrook.qbits.assistant.provider.AssistantProviderType;
|
||||
import com.kingsrook.qbits.assistant.provider.HealthResult;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelMessage;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelRequest;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelResponse;
|
||||
import com.kingsrook.qbits.assistant.provider.RegistryModelProvider;
|
||||
import com.kingsrook.qbits.assistant.provider.StreamHandle;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
@@ -108,6 +112,16 @@ public final class ConnectionTest
|
||||
return (new Result(STATUS_ERROR, null, null, e.getMessage()));
|
||||
}
|
||||
|
||||
return (call(runtime, active, userId));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Probe health, then make and account for a minimal completion.
|
||||
*******************************************************************************/
|
||||
private static Result call(AssistantRuntime runtime, ProviderService.Active active, String userId)
|
||||
{
|
||||
List<ModelMessage> messages = new ArrayList<>();
|
||||
messages.add(ModelMessage.user(PING_PROMPT));
|
||||
ModelRequest request = new ModelRequest()
|
||||
@@ -117,10 +131,16 @@ public final class ConnectionTest
|
||||
.withMessages(messages);
|
||||
|
||||
UsageRecorder.Call usageCall = new UsageRecorder.Call(userId, null, null, active, UsageRecorder.PURPOSE_TEST);
|
||||
ModelPricing pricing = runtime.getProviderService().pricing(active.providerType(), active.model());
|
||||
ModelPricing pricing = runtime.getProviderService().pricing(active, active.model());
|
||||
long started = System.nanoTime();
|
||||
try
|
||||
{
|
||||
AssistantProviderType type = runtime.getConfig().getProviderTypes().get(active.providerType());
|
||||
HealthResult health = type.checkHealth(runtime.getProviderService().settings(active.row()));
|
||||
if(Boolean.FALSE.equals(health.healthy()))
|
||||
{
|
||||
throw (new ModelProviderException(ModelProviderException.Kind.NETWORK, null, health.message()));
|
||||
}
|
||||
StreamHandle handle = active.provider().stream(request, text ->
|
||||
{
|
||||
// the reply text is not used
|
||||
@@ -140,6 +160,10 @@ public final class ConnectionTest
|
||||
UsageRecorder.save(runtime.getStore(), UsageRecorder.row(usageCall, null, e, latency, pricing));
|
||||
return (new Result(STATUS_ERROR, (int) latency, null, describe(e)));
|
||||
}
|
||||
catch(ProviderService.ProviderSetupException e)
|
||||
{
|
||||
return (new Result(STATUS_ERROR, null, null, e.getMessage()));
|
||||
}
|
||||
catch(InterruptedException e)
|
||||
{
|
||||
Thread.currentThread().interrupt();
|
||||
@@ -155,17 +179,7 @@ public final class ConnectionTest
|
||||
*******************************************************************************/
|
||||
static String describe(ModelProviderException e)
|
||||
{
|
||||
String hint = switch(e.getKind())
|
||||
{
|
||||
case AUTHENTICATION -> "The API key was refused. Check the key in the secret store.";
|
||||
case RATE_LIMITED -> "Rate limited by the provider. Try again shortly.";
|
||||
case OVERLOADED -> "The provider is overloaded. Try again shortly.";
|
||||
case INVALID_REQUEST -> "The provider refused the request. Check the model id.";
|
||||
case TIMEOUT -> "No answer in time.";
|
||||
case NETWORK -> "Could not reach the provider. Check the base URL and the network.";
|
||||
case SERVER -> "The provider had a server error.";
|
||||
default -> "The call failed.";
|
||||
};
|
||||
String hint = hint(e.getKind());
|
||||
return (hint + " (" + e.getKind() + (e.getHttpStatus() == null ? "" : ", HTTP " + e.getHttpStatus()) + (e.getMessage() == null ? "" : ": " + e.getMessage()) + ")");
|
||||
}
|
||||
|
||||
@@ -179,4 +193,56 @@ public final class ConnectionTest
|
||||
return (value == null || value.length() <= max ? value : value.substring(0, max));
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Test even a disabled registry model before an admin enables it.
|
||||
*******************************************************************************/
|
||||
public static Result runModel(AssistantRuntime runtime, AssistantModel model, String userId) throws QException
|
||||
{
|
||||
AssistantProvider row = runtime.getStore().getProvider(model.getProviderId());
|
||||
Result result;
|
||||
try
|
||||
{
|
||||
if(row == null)
|
||||
{
|
||||
throw (new ProviderService.ProviderSetupException("The model's provider no longer exists."));
|
||||
}
|
||||
AssistantProviderType type = runtime.getConfig().getProviderTypes().get(row.getProviderType());
|
||||
var settings = runtime.getProviderService().settings(row);
|
||||
var provider = new RegistryModelProvider(type.create(settings), model);
|
||||
result = call(runtime, new ProviderService.Active(row, row.getProviderType(), model.getModelId(), provider), userId);
|
||||
}
|
||||
catch(ProviderService.ProviderSetupException e)
|
||||
{
|
||||
result = new Result(STATUS_ERROR, null, model.getModelId(), e.getMessage());
|
||||
}
|
||||
runtime.getStore().updateFields(AssistantModel.TABLE_NAME, new QRecord().withValue("id", model.getId())
|
||||
.withValue("lastTestAt", runtime.getClock().instant()).withValue("lastTestStatus", result.status())
|
||||
.withValue("lastTestLatencyMs", result.latencyMs()).withValue("lastTestError", truncate(result.error(), 500)));
|
||||
return (result);
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Map the provider error category.
|
||||
*******************************************************************************/
|
||||
private static String hint(ModelProviderException.Kind kind)
|
||||
{
|
||||
return (switch(kind)
|
||||
{
|
||||
case AUTHENTICATION -> "The API key was refused. Check the key in the secret store.";
|
||||
case RATE_LIMITED -> "Rate limited by the provider. Try again shortly.";
|
||||
case OVERLOADED -> "The provider is overloaded. Try again shortly.";
|
||||
case INVALID_REQUEST -> "The provider refused the request. Check the model id.";
|
||||
case TIMEOUT -> "No answer in time.";
|
||||
case NETWORK -> "Could not reach the provider. Check the base URL and the network.";
|
||||
case SERVER -> "The provider had a server error.";
|
||||
default -> "The call failed.";
|
||||
});
|
||||
}
|
||||
|
||||
}
|
||||
@@ -0,0 +1,118 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.engine;
|
||||
|
||||
|
||||
import java.math.BigDecimal;
|
||||
import java.util.HashSet;
|
||||
import java.util.List;
|
||||
import java.util.Set;
|
||||
import com.kingsrook.qbits.assistant.AssistantRuntime;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantModel;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantProvider;
|
||||
import com.kingsrook.qbits.assistant.provider.AssistantProviderType;
|
||||
import com.kingsrook.qbits.assistant.provider.DiscoveredModel;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelCapabilities;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
|
||||
import com.kingsrook.qbits.assistant.provider.ProviderSettings;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Additive discovery: never overwrite an administrator's model configuration.
|
||||
*******************************************************************************/
|
||||
public final class ModelDiscovery
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Summary for the standard process result screen.
|
||||
*******************************************************************************/
|
||||
public record Result(boolean ok, String message)
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private ModelDiscovery()
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Serialize local discovery runs; the table unique key also guards replicas.
|
||||
*******************************************************************************/
|
||||
public static Result run(AssistantRuntime runtime, AssistantProvider row) throws QException
|
||||
{
|
||||
ProviderService service = runtime.getProviderService();
|
||||
synchronized(service)
|
||||
{
|
||||
Result result;
|
||||
try
|
||||
{
|
||||
ProviderSettings settings = service.settings(row);
|
||||
AssistantProviderType type = runtime.getConfig().getProviderTypes().get(row.getProviderType());
|
||||
List<DiscoveredModel> discovered = type.discoverModels(settings);
|
||||
Set<String> existing = new HashSet<>();
|
||||
runtime.getStore().listModels().stream().filter(m -> row.getId().equals(m.getProviderId())).forEach(m -> existing.add(m.getModelId()));
|
||||
boolean local = "LOCAL".equals(service.location(row));
|
||||
var provider = type.create(settings);
|
||||
int added = 0;
|
||||
int present = 0;
|
||||
for(DiscoveredModel model : discovered)
|
||||
{
|
||||
if(existing.contains(model.modelId()))
|
||||
{
|
||||
present++;
|
||||
continue;
|
||||
}
|
||||
ModelCapabilities caps = provider.getCapabilities(model.modelId());
|
||||
runtime.getStore().insert(new AssistantModel().withProviderId(row.getId()).withModelId(model.modelId())
|
||||
.withDisplayName(model.displayName()).withEnabled(local || discovered.size() <= 20)
|
||||
.withSource("DISCOVERED").withDiscoveredAt(runtime.getClock().instant())
|
||||
.withSupportsTools(caps.getTools()).withSupportsVision(caps.getVision()).withSupportsStreaming(caps.getStreaming())
|
||||
.withSupportsReasoningEffort(caps.getReasoningEffort())
|
||||
.withContextWindowTokens(model.contextWindowTokens() == null ? caps.getMaxContextTokens() : model.contextWindowTokens())
|
||||
.withInputUsdPerMTok(local ? BigDecimal.ZERO : null).withOutputUsdPerMTok(local ? BigDecimal.ZERO : null));
|
||||
existing.add(model.modelId());
|
||||
added++;
|
||||
}
|
||||
result = new Result(true, added + " models added, " + present + " already present");
|
||||
}
|
||||
catch(ProviderService.ProviderSetupException e)
|
||||
{
|
||||
result = new Result(false, e.getMessage());
|
||||
}
|
||||
catch(ModelProviderException e)
|
||||
{
|
||||
result = new Result(false, ConnectionTest.describe(e));
|
||||
}
|
||||
catch(QException | RuntimeException e)
|
||||
{
|
||||
result = new Result(false, "Could not save discovered models. Check the model registry and retry.");
|
||||
}
|
||||
runtime.getStore().updateFields(AssistantProvider.TABLE_NAME, new QRecord().withValue("id", row.getId())
|
||||
.withValue("lastDiscoveryAt", runtime.getClock().instant()).withValue("lastDiscoveryStatus", result.ok() ? "OK" : "ERROR")
|
||||
.withValue("lastDiscoveryError", result.ok() ? null : result.message().substring(0, Math.min(500, result.message().length()))));
|
||||
service.invalidate();
|
||||
return (result);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -32,11 +32,14 @@ import java.util.Map;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import java.util.concurrent.Semaphore;
|
||||
import com.kingsrook.qbits.assistant.AssistantQBitConfig;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantModel;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantModelPrice;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantProvider;
|
||||
import com.kingsrook.qbits.assistant.persistence.AssistantStore;
|
||||
import com.kingsrook.qbits.assistant.provider.AssistantModelProvider;
|
||||
import com.kingsrook.qbits.assistant.provider.AssistantProviderType;
|
||||
import com.kingsrook.qbits.assistant.provider.ProviderSettings;
|
||||
import com.kingsrook.qbits.assistant.provider.RegistryModelProvider;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.logging.QLogger;
|
||||
import com.kingsrook.qqq.backend.core.utils.StringUtils;
|
||||
@@ -75,6 +78,7 @@ public class ProviderService
|
||||
private volatile List<AssistantProvider> cachedRows;
|
||||
private volatile Map<String, ModelPricing> cachedPrices;
|
||||
private volatile Instant cachedAt;
|
||||
private volatile List<AssistantModel> cachedModels;
|
||||
|
||||
|
||||
|
||||
@@ -179,24 +183,41 @@ public class ProviderService
|
||||
*******************************************************************************/
|
||||
public Active forRow(AssistantProvider row, String fallbackModel) throws ProviderSetupException
|
||||
{
|
||||
List<String> errors = validate(row);
|
||||
if(!errors.isEmpty())
|
||||
{
|
||||
throw (new ProviderSetupException(String.join(" ", errors)));
|
||||
}
|
||||
|
||||
ProviderSettings settings = settings(row);
|
||||
AssistantProviderType type = config.getProviderTypes().get(row.getProviderType());
|
||||
String key = resolveKey(row);
|
||||
if(!StringUtils.hasContent(key))
|
||||
List<AssistantModel> models;
|
||||
try
|
||||
{
|
||||
throw (new ProviderSetupException("No API key: the environment variable " + keyEnvName(row) + " is not set on this server."));
|
||||
models = models(row.getId());
|
||||
}
|
||||
|
||||
String model = StringUtils.hasContent(row.getModel()) ? row.getModel() : (StringUtils.hasContent(fallbackModel) ? fallbackModel : first(type.getModels()));
|
||||
String baseUrl = StringUtils.hasContent(row.getBaseUrl()) ? row.getBaseUrl().trim() : null;
|
||||
String cacheKey = row.getId() + "|" + row.getProviderType() + "|" + baseUrl + "|" + fingerprint(key);
|
||||
AssistantModelProvider provider = built.computeIfAbsent(cacheKey, k -> type.create(key, baseUrl));
|
||||
return (new Active(row, row.getProviderType(), model, provider));
|
||||
catch(QException e)
|
||||
{
|
||||
throw (new ProviderSetupException("Could not read the provider's model registry."));
|
||||
}
|
||||
String model = StringUtils.hasContent(row.getModel()) ? row.getModel() : null;
|
||||
if(model == null && !models.isEmpty())
|
||||
{
|
||||
model = models.stream().filter(m -> Boolean.TRUE.equals(m.getEnabled())).map(AssistantModel::getModelId).findFirst().orElse(null);
|
||||
if(model == null)
|
||||
{
|
||||
throw (new ProviderSetupException("All models of this provider are disabled. Enable a model first."));
|
||||
}
|
||||
}
|
||||
if(model == null)
|
||||
{
|
||||
model = StringUtils.hasContent(fallbackModel) ? fallbackModel : first(type.getModels());
|
||||
}
|
||||
String selected = model;
|
||||
AssistantModel registry = models.stream().filter(m -> java.util.Objects.equals(selected, m.getModelId())).findFirst().orElse(null);
|
||||
if(registry != null && !Boolean.TRUE.equals(registry.getEnabled()))
|
||||
{
|
||||
throw (new ProviderSetupException("The selected model is disabled in this provider's model registry."));
|
||||
}
|
||||
String cacheKey = row.getId() + "|" + row.getProviderType() + "|" + settings.getKind() + "|" + settings.getBaseUrl()
|
||||
+ "|" + settings.getApiVersion() + "|" + settings.getRegion() + "|" + settings.getConnectTimeoutSeconds()
|
||||
+ "|" + settings.getTlsCaBundle() + "|" + settings.getRequestTimeoutSeconds() + "|" + fingerprint(settings.getApiKey());
|
||||
AssistantModelProvider provider = built.computeIfAbsent(cacheKey, k -> type.create(settings));
|
||||
return (new Active(row, row.getProviderType(), model, registry == null ? provider : new RegistryModelProvider(provider, registry)));
|
||||
}
|
||||
|
||||
|
||||
@@ -216,11 +237,11 @@ public class ProviderService
|
||||
}
|
||||
|
||||
String envName = keyEnvName(row);
|
||||
if(!AssistantQBitConfig.ENV_NAME.matcher(envName).matches())
|
||||
if(envName != null && !AssistantQBitConfig.ENV_NAME.matcher(envName).matches())
|
||||
{
|
||||
errors.add("API key variable must be the NAME of an environment variable (capital letters, digits and _), not the key itself.");
|
||||
}
|
||||
else if(!config.getEffectiveAllowedApiKeyEnvNames().contains(envName))
|
||||
else if(envName != null && !config.getEffectiveAllowedApiKeyEnvNames().contains(envName))
|
||||
{
|
||||
errors.add("API key variable " + envName + " is not allowed on this server. Allowed: " + String.join(", ", config.getEffectiveAllowedApiKeyEnvNames()) + ".");
|
||||
}
|
||||
@@ -228,12 +249,44 @@ public class ProviderService
|
||||
if(StringUtils.hasContent(row.getBaseUrl()))
|
||||
{
|
||||
String url = row.getBaseUrl().trim();
|
||||
if(!url.equals(type.getDefaultBaseUrl()) && (config.getAllowedBaseUrls() == null || !config.getAllowedBaseUrls().contains(url)))
|
||||
if(!url.equals(type.getDefaultBaseUrl(kind(row))) && (config.getAllowedBaseUrls() == null || !config.getAllowedBaseUrls().contains(url)))
|
||||
{
|
||||
errors.add("Base URL " + url + " is not allowed on this server; leave it empty to use " + type.getDefaultBaseUrl() + ".");
|
||||
errors.add("Base URL " + url + " is not allowed on this server; leave it empty to use " + type.getDefaultBaseUrl(kind(row)) + ".");
|
||||
}
|
||||
}
|
||||
|
||||
if(StringUtils.hasContent(row.getTlsCaBundle()))
|
||||
{
|
||||
if(!config.getTrustedCaBundles().containsKey(row.getTlsCaBundle()))
|
||||
{
|
||||
errors.add("TLS CA bundle must name a bundle configured on this server (withTrustedCaBundle).");
|
||||
}
|
||||
String url = StringUtils.hasContent(row.getBaseUrl()) ? row.getBaseUrl().trim() : type.getDefaultBaseUrl(kind(row));
|
||||
if(url == null || !url.regionMatches(true, 0, "https://", 0, 8))
|
||||
{
|
||||
errors.add("TLS CA bundle requires an HTTPS base URL.");
|
||||
}
|
||||
}
|
||||
|
||||
if(!type.getKinds().contains(kind(row)))
|
||||
{
|
||||
errors.add("The provider kind is not supported by the installed adapter.");
|
||||
}
|
||||
if(!List.of("LOCAL", "CLOUD").contains(location(row)))
|
||||
{
|
||||
errors.add("Location must be LOCAL or CLOUD.");
|
||||
}
|
||||
if(!StringUtils.hasContent(row.getBaseUrl()) && type.getDefaultBaseUrl(kind(row)) == null)
|
||||
{
|
||||
errors.add("An allow-listed base URL is required for this provider kind.");
|
||||
}
|
||||
for(Integer timeout : java.util.Arrays.asList(row.getConnectTimeoutSeconds(), row.getRequestTimeoutSeconds()))
|
||||
{
|
||||
if(timeout != null && (timeout < 1 || timeout > 600))
|
||||
{
|
||||
errors.add("Connection and request timeouts must be between 1 and 600 seconds.");
|
||||
}
|
||||
}
|
||||
if(row.getMonthlyCostCapUsd() != null && row.getMonthlyCostCapUsd().signum() < 0)
|
||||
{
|
||||
errors.add("Monthly cost cap must not be negative.");
|
||||
@@ -258,6 +311,10 @@ public class ProviderService
|
||||
public String keyStatus(AssistantProvider row)
|
||||
{
|
||||
String envName = keyEnvName(row);
|
||||
if(envName == null)
|
||||
{
|
||||
return ("Not needed (local, no key)");
|
||||
}
|
||||
if(!AssistantQBitConfig.ENV_NAME.matcher(envName).matches() || !config.getEffectiveAllowedApiKeyEnvNames().contains(envName))
|
||||
{
|
||||
return ("Not usable: the reference is not an allowed environment variable name");
|
||||
@@ -298,7 +355,7 @@ public class ProviderService
|
||||
return (row.getApiKeyEnvName().trim());
|
||||
}
|
||||
AssistantProviderType type = row.getProviderType() == null ? null : config.getProviderTypes().get(row.getProviderType());
|
||||
return (type == null ? "" : type.getDefaultApiKeyEnvName());
|
||||
return (type == null ? "" : type.getDefaultApiKeyEnvName(kind(row)));
|
||||
}
|
||||
|
||||
|
||||
@@ -309,6 +366,10 @@ public class ProviderService
|
||||
private String resolveKey(AssistantProvider row)
|
||||
{
|
||||
String envName = keyEnvName(row);
|
||||
if(envName == null)
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
if(!AssistantQBitConfig.ENV_NAME.matcher(envName).matches() || !config.getEffectiveAllowedApiKeyEnvNames().contains(envName))
|
||||
{
|
||||
return (null);
|
||||
@@ -447,11 +508,12 @@ public class ProviderService
|
||||
/*******************************************************************************
|
||||
** Drop the cached rows and prices (after a settings change).
|
||||
*******************************************************************************/
|
||||
public void invalidate()
|
||||
public synchronized void invalidate()
|
||||
{
|
||||
cachedAt = null;
|
||||
cachedRows = null;
|
||||
cachedPrices = null;
|
||||
cachedModels = null;
|
||||
}
|
||||
|
||||
|
||||
@@ -459,7 +521,7 @@ public class ProviderService
|
||||
/*******************************************************************************
|
||||
** The provider rows (cached for policyCacheSeconds).
|
||||
*******************************************************************************/
|
||||
public List<AssistantProvider> rows() throws QException
|
||||
public synchronized List<AssistantProvider> rows() throws QException
|
||||
{
|
||||
refresh();
|
||||
return (cachedRows);
|
||||
@@ -470,7 +532,7 @@ public class ProviderService
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private Map<String, ModelPricing> prices() throws QException
|
||||
private synchronized Map<String, ModelPricing> prices() throws QException
|
||||
{
|
||||
refresh();
|
||||
return (cachedPrices);
|
||||
@@ -497,6 +559,7 @@ public class ProviderService
|
||||
prices.put(priceKey(price.getProviderType(), price.getModel()), toPricing(price));
|
||||
}
|
||||
}
|
||||
cachedModels = store.listModels();
|
||||
cachedRows = rows;
|
||||
cachedPrices = prices;
|
||||
cachedAt = now;
|
||||
@@ -521,7 +584,7 @@ public class ProviderService
|
||||
{
|
||||
try
|
||||
{
|
||||
return (HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(key.getBytes(StandardCharsets.UTF_8))));
|
||||
return (HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest((key == null ? "" : key).getBytes(StandardCharsets.UTF_8))));
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
@@ -549,4 +612,98 @@ public class ProviderService
|
||||
return (value == null ? 0 : value.doubleValue());
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Effective kind, including legacy rows.
|
||||
*******************************************************************************/
|
||||
public String kind(AssistantProvider row)
|
||||
{
|
||||
return (StringUtils.hasContent(row.getKind()) ? row.getKind() : (row.getProviderType() == null ? "" : row.getProviderType().toUpperCase(java.util.Locale.ROOT).replace('-', '_')));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Effective location, including legacy rows.
|
||||
*******************************************************************************/
|
||||
public String location(AssistantProvider row)
|
||||
{
|
||||
AssistantProviderType type = config.getProviderTypes().get(row.getProviderType());
|
||||
return (StringUtils.hasContent(row.getLocation()) ? row.getLocation() : (type == null ? "CLOUD" : type.getDefaultLocation(kind(row))));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Validate settings and resolve only an allowed key reference.
|
||||
*******************************************************************************/
|
||||
public ProviderSettings settings(AssistantProvider row) throws ProviderSetupException
|
||||
{
|
||||
List<String> errors = validate(row);
|
||||
if(!errors.isEmpty())
|
||||
{
|
||||
throw (new ProviderSetupException(String.join(" ", errors)));
|
||||
}
|
||||
String key = resolveKey(row);
|
||||
if(keyEnvName(row) != null && !StringUtils.hasContent(key))
|
||||
{
|
||||
throw (new ProviderSetupException("No API key: the environment variable " + keyEnvName(row) + " is not set on this server."));
|
||||
}
|
||||
return (new ProviderSettings().withKind(kind(row)).withApiKey(key)
|
||||
.withBaseUrl(StringUtils.hasContent(row.getBaseUrl()) ? row.getBaseUrl().trim() : null)
|
||||
.withApiVersion(row.getApiVersion()).withRegion(row.getRegion())
|
||||
.withTlsCaBundle(row.getTlsCaBundle()).withSslContext(StringUtils.hasContent(row.getTlsCaBundle()) ? config.getTrustedCaBundles().get(row.getTlsCaBundle()) : null)
|
||||
.withConnectTimeoutSeconds(row.getConnectTimeoutSeconds() == null ? 10 : row.getConnectTimeoutSeconds())
|
||||
.withRequestTimeoutSeconds(row.getRequestTimeoutSeconds() == null ? 120 : row.getRequestTimeoutSeconds()));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Cached models belonging to one provider.
|
||||
*******************************************************************************/
|
||||
public synchronized List<AssistantModel> models(Integer providerId) throws QException
|
||||
{
|
||||
refresh();
|
||||
return (cachedModels.stream().filter(m -> java.util.Objects.equals(providerId, m.getProviderId())).toList());
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Registry price, type price, then config price; unpriced local models cost zero.
|
||||
*******************************************************************************/
|
||||
public ModelPricing pricing(Active active, String model)
|
||||
{
|
||||
if(active != null && active.row() != null)
|
||||
{
|
||||
try
|
||||
{
|
||||
for(AssistantModel row : models(active.providerId()))
|
||||
{
|
||||
if(java.util.Objects.equals(model, row.getModelId()) && (row.getInputUsdPerMTok() != null || row.getOutputUsdPerMTok() != null))
|
||||
{
|
||||
return (toPricing(new AssistantModelPrice().withInputUsdPerMTok(row.getInputUsdPerMTok()).withOutputUsdPerMTok(row.getOutputUsdPerMTok())));
|
||||
}
|
||||
}
|
||||
ModelPricing price = prices().get(priceKey(active.providerType(), model));
|
||||
if(price != null)
|
||||
{
|
||||
return (price);
|
||||
}
|
||||
if("LOCAL".equals(location(active.row())) && (config.getPricing() == null || !config.getPricing().containsKey(model)))
|
||||
{
|
||||
return (new ModelPricing().withInputUsdPerMTok(0.0).withOutputUsdPerMTok(0.0));
|
||||
}
|
||||
}
|
||||
catch(QException e)
|
||||
{
|
||||
LOG.warn("Could not read model registry prices");
|
||||
}
|
||||
}
|
||||
return (pricing(active == null ? null : active.providerType(), model));
|
||||
}
|
||||
|
||||
}
|
||||
@@ -1057,7 +1057,7 @@ public class TurnEngine implements Runnable
|
||||
*******************************************************************************/
|
||||
private ModelPricing pricing()
|
||||
{
|
||||
return (runtime.getProviderService().pricing(active == null ? null : active.providerType(), model));
|
||||
return (runtime.getProviderService().pricing(active, model));
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -79,7 +79,7 @@ public class AssistantHomeData
|
||||
/*******************************************************************************
|
||||
** One provider's health.
|
||||
*******************************************************************************/
|
||||
public record ProviderHealth(AssistantProvider row, String typeLabel, String keyStatus, boolean keyPresent, List<String> problems, boolean active, long monthCostMicros, Long capMicros)
|
||||
public record ProviderHealth(AssistantProvider row, String typeLabel, String keyStatus, boolean keyPresent, List<String> problems, boolean active, long monthCostMicros, Long capMicros, String kind, String location, long enabledModels)
|
||||
{
|
||||
}
|
||||
|
||||
@@ -275,8 +275,8 @@ public class AssistantHomeData
|
||||
{
|
||||
String status = service.keyStatus(row);
|
||||
AssistantProviderType type = row.getProviderType() == null ? null : runtime.getConfig().getProviderTypes().get(row.getProviderType());
|
||||
data.providers.add(new ProviderHealth(row, type == null ? row.getProviderType() : type.getLabel(), status, status.startsWith("Set"), service.validate(row), active != null && active.getId().equals(row.getId()),
|
||||
providerMonth.getOrDefault(row.getId(), 0L), ProviderService.capMicros(row)));
|
||||
data.providers.add(new ProviderHealth(row, type == null ? row.getProviderType() : type.getLabel(), status, (status.startsWith("Set") || status.startsWith("Not needed")), service.validate(row), active != null && active.getId().equals(row.getId()),
|
||||
providerMonth.getOrDefault(row.getId(), 0L), ProviderService.capMicros(row), service.kind(row), service.location(row), service.models(row.getId()).stream().filter(m -> Boolean.TRUE.equals(m.getEnabled())).count()));
|
||||
}
|
||||
return (data);
|
||||
}
|
||||
|
||||
@@ -28,6 +28,7 @@ import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Locale;
|
||||
import java.util.Map;
|
||||
import com.kingsrook.qbits.assistant.metadata.AssistantRecordScreens;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantConversation;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantModelPrice;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantProvider;
|
||||
@@ -185,6 +186,13 @@ public final class AssistantHomeView
|
||||
}
|
||||
html.append("</div><div style=\"").append(CELLS).append("\">");
|
||||
cell(html, "Type", nvl(p.typeLabel(), "Not set"));
|
||||
cell(html, "Kind", AssistantRecordScreens.providerKindLabel(p.kind()));
|
||||
cell(html, "Location", "LOCAL".equals(p.location()) ? "Local" : "Cloud");
|
||||
if(Boolean.TRUE.equals(row.getHipaaEligible()))
|
||||
{
|
||||
cell(html, "Eligibility", "HIPAA-eligible");
|
||||
}
|
||||
cell(html, "Enabled models", String.valueOf(p.enabledModels()));
|
||||
cell(html, "Model", nvl(row.getModel(), "Default model"));
|
||||
cell(html, "API key", p.keyStatus());
|
||||
cell(html, "Last test", lastTest(row));
|
||||
|
||||
@@ -26,6 +26,7 @@ import com.kingsrook.qbits.assistant.model.AssistantConversation;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantDataAccess;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantFeedback;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantMessage;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantModel;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantModelPrice;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantPendingAction;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantPolicy;
|
||||
@@ -179,6 +180,10 @@ public class AssistantMetaDataBuilder
|
||||
.withUniqueKey(new UniqueKey("name"))
|
||||
.withRecordLabelFormat("%s").withRecordLabelFields("name"));
|
||||
|
||||
tables.add(settings(base(AssistantModel.TABLE_NAME, "Assistant Model", "memory", AssistantModel.class))
|
||||
.withUniqueKey(new UniqueKey("providerId", "modelId"))
|
||||
.withRecordLabelFormat("%s").withRecordLabelFields("modelId"));
|
||||
|
||||
tables.add(settings(base(AssistantModelPrice.TABLE_NAME, "Assistant Model Price", "sell", AssistantModelPrice.class))
|
||||
.withUniqueKey(new UniqueKey("providerType", "model"))
|
||||
.withRecordLabelFormat("%s").withRecordLabelFields("model"));
|
||||
|
||||
@@ -30,6 +30,7 @@ import com.kingsrook.qbits.assistant.model.AssistantConversation;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantDataAccess;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantFeedback;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantMessage;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantModel;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantModelPrice;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantPendingAction;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantPolicy;
|
||||
@@ -165,6 +166,7 @@ public final class AssistantRecordScreens
|
||||
case AssistantPolicy.TABLE_NAME -> policy(table);
|
||||
case AssistantUsageDaily.TABLE_NAME -> usageDaily(table);
|
||||
case AssistantProvider.TABLE_NAME -> provider(table);
|
||||
case AssistantModel.TABLE_NAME -> model(table);
|
||||
case AssistantModelPrice.TABLE_NAME -> price(table);
|
||||
case AssistantUsage.TABLE_NAME -> usage(table);
|
||||
case AssistantDataAccess.TABLE_NAME -> dataAccess(table);
|
||||
@@ -217,6 +219,7 @@ public final class AssistantRecordScreens
|
||||
QTableMetaData provider = qInstance.getTable(AssistantProvider.TABLE_NAME);
|
||||
if(provider != null)
|
||||
{
|
||||
childList(qInstance, provider, AssistantModel.TABLE_NAME, "Models", "memory", List.of("modelId", "displayName", "enabled", "supportsTools", "lastTestStatus"), "providerId", "provider");
|
||||
childList(qInstance, provider, AssistantUsage.TABLE_NAME, "Model Calls", "receipt_long", USAGE_LIST_FIELDS, "providerId", "provider");
|
||||
}
|
||||
}
|
||||
@@ -248,6 +251,10 @@ public final class AssistantRecordScreens
|
||||
////////////////////////////////////////////////////////////////////
|
||||
types.add(new QPossibleValue<>(config.getProviderName(), config.getProviderName()));
|
||||
}
|
||||
List<QPossibleValue<?>> kinds = new ArrayList<>();
|
||||
Set<String> kindNames = new LinkedHashSet<>(List.of("ANTHROPIC", "OPENAI", "AZURE_OPENAI", "OPENROUTER", "OLLAMA", "LM_STUDIO", "VLLM", "LLAMA_CPP", "OPENAI_COMPATIBLE"));
|
||||
config.getProviderTypes().values().forEach(t -> kindNames.addAll(t.getKinds()));
|
||||
kindNames.forEach(k -> kinds.add(new QPossibleValue<>(k, providerKindLabel(k))));
|
||||
List<QPossibleValue<?>> modelValues = new ArrayList<>();
|
||||
models.forEach(m -> modelValues.add(new QPossibleValue<>(m, m)));
|
||||
|
||||
@@ -256,11 +263,41 @@ public final class AssistantRecordScreens
|
||||
.withValueFormatAndFields(PVSValueFormatAndFields.LABEL_ONLY).withEnumValues(types),
|
||||
new QPossibleValueSource().withName(MODEL_PVS_NAME).withType(QPossibleValueSourceType.ENUM).withIdType(QFieldType.STRING)
|
||||
.withValueFormatAndFields(PVSValueFormatAndFields.LABEL_ONLY).withEnumValues(modelValues),
|
||||
enumSource("assistant_provider_kind", kinds),
|
||||
enumSource("assistant_provider_location", List.of(new QPossibleValue<>("LOCAL", "Local"), new QPossibleValue<>("CLOUD", "Cloud"))),
|
||||
enumSource("assistant_model_source", List.of(new QPossibleValue<>("DISCOVERED", "Discovered"), new QPossibleValue<>("MANUAL", "Manual"))),
|
||||
QPossibleValueSource.newForTable(AssistantProvider.TABLE_NAME)));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Display label shared by provider settings and home cards. Custom kinds
|
||||
** retain their registered name.
|
||||
*******************************************************************************/
|
||||
public static String providerKindLabel(String kind)
|
||||
{
|
||||
if(kind == null)
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
return (switch(kind)
|
||||
{
|
||||
case "ANTHROPIC" -> "Anthropic";
|
||||
case "OPENAI" -> "OpenAI";
|
||||
case "AZURE_OPENAI" -> "Azure OpenAI";
|
||||
case "OPENROUTER" -> "OpenRouter";
|
||||
case "OLLAMA" -> "Ollama";
|
||||
case "LM_STUDIO" -> "LM Studio";
|
||||
case "VLLM" -> "vLLM";
|
||||
case "LLAMA_CPP" -> "llama.cpp server";
|
||||
case "OPENAI_COMPATIBLE" -> "OpenAI-compatible (other)";
|
||||
default -> kind;
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The provider settings screen.
|
||||
*******************************************************************************/
|
||||
@@ -268,12 +305,20 @@ public final class AssistantRecordScreens
|
||||
{
|
||||
pvs(table, "providerType", PROVIDER_TYPE_PVS_NAME, "Type");
|
||||
pvs(table, "model", MODEL_PVS_NAME, "Model");
|
||||
pvs(table, "kind", "assistant_provider_kind", "Kind");
|
||||
pvs(table, "location", "assistant_provider_location", "Location");
|
||||
table.getField("hipaaEligible").withLabel("HIPAA-eligible");
|
||||
table.getField("apiVersion").withLabel("API version");
|
||||
table.getField("connectTimeoutSeconds").withLabel("Connect timeout (s)");
|
||||
table.getField("requestTimeoutSeconds").withLabel("Request timeout (s)");
|
||||
help(table, "hipaaEligible", "Records a fact (local, or cloud under a signed BAA). It does not create a BAA.");
|
||||
table.getField("isDefault").withLabel("Default");
|
||||
table.getField("apiKeyEnvName").withLabel("API Key Variable");
|
||||
table.getField("tlsCaBundle").withLabel("TLS CA bundle");
|
||||
table.getField("baseUrl").withLabel("Base URL Override");
|
||||
table.getField("monthlyCostCapUsd").withLabel("Monthly Cost Cap (USD)").withDisplayFormat(DisplayFormat.CURRENCY);
|
||||
counts(table, "maxOutputTokensPerRequest", "lastTestLatencyMs");
|
||||
table.getField("lastTestLatencyMs").withLabel("Last Test Latency (ms)");
|
||||
table.getField("lastTestLatencyMs").withLabel("Last test latency (ms)");
|
||||
help(table, "name", "A name for this provider, e.g. Claude (production).");
|
||||
help(table, "providerType", "Which kind of provider this is. Only the types installed on this server are offered.");
|
||||
help(table, "model", "The model answers come from. Empty uses the policy's (or the application's) main model.");
|
||||
@@ -281,6 +326,7 @@ public final class AssistantRecordScreens
|
||||
help(table, "isDefault", "The assistant uses the default enabled provider (else the first enabled one). Marking one default unmarks the others.");
|
||||
help(table, "apiKeyEnvName", "The NAME of the environment variable that holds the API key (e.g. ANTHROPIC_API_KEY), never the key itself. "
|
||||
+ "The key stays in the server's secret store; only allowed variable names are accepted. Empty uses the type's default variable. To replace the key, change it in the secret store.");
|
||||
help(table, "tlsCaBundle", "Name of a CA bundle configured on the server (withTrustedCaBundle). Leave empty to use the JVM trust store.");
|
||||
help(table, "baseUrl", "Leave empty to use the provider's own API. Other URLs must be allowed in the server's configuration.");
|
||||
help(table, "monthlyCostCapUsd", "When this provider's cost this month (UTC) reaches the cap, new requests are refused. Empty: no cap.");
|
||||
help(table, "maxOutputTokensPerRequest", "The most output tokens one model call may produce. Empty uses the configured limit.");
|
||||
@@ -288,8 +334,10 @@ public final class AssistantRecordScreens
|
||||
table.getField(ProviderService.KEY_STATUS_FIELD).withLabel("API Key").withHelpContent(new QHelpContent("Whether the key variable is set on this server, with the key's last four characters. The key itself is never shown or returned."));
|
||||
sections(table,
|
||||
section("provider", "Provider", "settings_input_component", Tier.T1, "id", "name", "providerType", "model", "enabled", "isDefault"),
|
||||
section("connection", "Connection", "key", Tier.T2, "apiKeyEnvName", ProviderService.KEY_STATUS_FIELD, "baseUrl"),
|
||||
section("connection", "Connection", "key", Tier.T2, "apiKeyEnvName", ProviderService.KEY_STATUS_FIELD, "baseUrl", "tlsCaBundle"),
|
||||
section("limits", "Limits", "speed", Tier.T2, "monthlyCostCapUsd", "maxOutputTokensPerRequest", "maxConcurrentRequests"),
|
||||
section("registry", "Provider settings", "settings", Tier.T2, "kind", "location", "hipaaEligible", "region", "apiVersion", "connectTimeoutSeconds", "requestTimeoutSeconds"),
|
||||
section("discovery", "Last discovery", "memory", Tier.T2, "lastDiscoveryAt", "lastDiscoveryStatus", "lastDiscoveryError"),
|
||||
section("lastTest", "Last connection test", "network_check", Tier.T2, "lastTestAt", "lastTestStatus", "lastTestLatencyMs", "lastTestModel", "lastTestError"));
|
||||
}
|
||||
|
||||
@@ -305,8 +353,8 @@ public final class AssistantRecordScreens
|
||||
{
|
||||
help(table, field, "USD per million tokens.");
|
||||
}
|
||||
table.getField("inputUsdPerMTok").withLabel("Input (USD / MTok)");
|
||||
table.getField("outputUsdPerMTok").withLabel("Output (USD / MTok)");
|
||||
table.getField("inputUsdPerMTok").withLabel("Input (USD per million tokens)");
|
||||
table.getField("outputUsdPerMTok").withLabel("Output (USD per million tokens)");
|
||||
table.getField("cacheReadUsdPerMTok").withLabel("Cache Read (USD / MTok)");
|
||||
table.getField("cacheWrite5mUsdPerMTok").withLabel("Cache Write 5m (USD / MTok)");
|
||||
table.getField("cacheWrite1hUsdPerMTok").withLabel("Cache Write 1h (USD / MTok)");
|
||||
@@ -785,11 +833,34 @@ public final class AssistantRecordScreens
|
||||
qInstance.addWidget(ChildRecordListRenderer.widgetMetaDataBuilder(join)
|
||||
.withLabel(label)
|
||||
.withMaxRows(50)
|
||||
.withCanAddChildRecord(false)
|
||||
.withCanAddChildRecord(AssistantModel.TABLE_NAME.equals(childTableName))
|
||||
.withOnlyIncludeFieldNames(shown)
|
||||
.getWidgetMetaData());
|
||||
|
||||
String sectionName = sectionPrefix + label.replace(" ", "");
|
||||
conversation.addSection(new QFieldSection(sectionName, new QIcon(icon), Tier.T2).withLabel(label).withWidgetName(join.getName()));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The model sub-table record screen.
|
||||
*******************************************************************************/
|
||||
private static void model(QTableMetaData table)
|
||||
{
|
||||
link(table, "providerId", AssistantProvider.TABLE_NAME, "Provider");
|
||||
pvs(table, "source", "assistant_model_source", "Source");
|
||||
table.getField("contextWindowTokens").withLabel("Context window (tokens)");
|
||||
table.getField("maxOutputTokens").withLabel("Max output tokens");
|
||||
table.getField("supportsReasoningEffort").withLabel("Supports reasoning effort");
|
||||
table.getField("inputUsdPerMTok").withLabel("Input (USD per million tokens)");
|
||||
table.getField("outputUsdPerMTok").withLabel("Output (USD per million tokens)");
|
||||
table.getField("lastTestLatencyMs").withLabel("Last test latency (ms)");
|
||||
sections(table,
|
||||
section("model", "Model", "memory", Tier.T1, "id", "providerId", "modelId", "displayName", "enabled", "source", "discoveredAt"),
|
||||
section("capabilities", "Capabilities", "settings", Tier.T2, "supportsTools", "supportsVision", "supportsStreaming", "supportsReasoningEffort", "contextWindowTokens", "maxOutputTokens"),
|
||||
section("prices", "Prices", "payments", Tier.T2, "inputUsdPerMTok", "outputUsdPerMTok"),
|
||||
section("lastTest", "Last connection test", "network_check", Tier.T2, "lastTestAt", "lastTestStatus", "lastTestLatencyMs", "lastTestError"));
|
||||
}
|
||||
|
||||
}
|
||||
@@ -25,10 +25,10 @@ import com.kingsrook.qbits.assistant.AssistantRuntime;
|
||||
import com.kingsrook.qbits.assistant.engine.ProviderService;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantProvider;
|
||||
import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.QueryOrGetInputInterface;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.QueryOrGetInputInterface;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.statusmessages.BadInputStatusMessage;
|
||||
@@ -77,6 +77,24 @@ public class AssistantSettingsCustomizer implements TableCustomizerInterface
|
||||
public List<QRecord> preInsertOrUpdate(AbstractActionInput input, List<QRecord> records, boolean isPreview, Optional<List<QRecord>> oldRecordList) throws QException
|
||||
{
|
||||
ProviderService service = service();
|
||||
if(com.kingsrook.qbits.assistant.model.AssistantModel.TABLE_NAME.equals(tableName(input)))
|
||||
{
|
||||
for(QRecord record : records)
|
||||
{
|
||||
if(record.getValue("id") == null)
|
||||
{
|
||||
if(record.getValue("source") == null)
|
||||
{
|
||||
record.setValue("source", "MANUAL");
|
||||
}
|
||||
if(record.getValue("enabled") == null)
|
||||
{
|
||||
record.setValue("enabled", true);
|
||||
}
|
||||
}
|
||||
}
|
||||
return (records);
|
||||
}
|
||||
if(service == null || !AssistantProvider.TABLE_NAME.equals(tableName(input)))
|
||||
{
|
||||
return (records);
|
||||
@@ -95,7 +113,23 @@ public class AssistantSettingsCustomizer implements TableCustomizerInterface
|
||||
old.map(m -> m.get(record.getValue("id"))).ifPresent(o -> merged.setValues(new java.util.LinkedHashMap<>(o.getValues())));
|
||||
record.getValues().forEach(merged::setValue);
|
||||
|
||||
for(String error : service.validate(new AssistantProvider(merged)))
|
||||
AssistantProvider provider = new AssistantProvider(merged);
|
||||
if(merged.getValue("kind") == null)
|
||||
{
|
||||
record.setValue("kind", service.kind(provider));
|
||||
}
|
||||
if(merged.getValue("location") == null)
|
||||
{
|
||||
record.setValue("location", service.location(provider));
|
||||
}
|
||||
for(String field : List.of("hipaaEligible", "connectTimeoutSeconds", "requestTimeoutSeconds"))
|
||||
{
|
||||
if(merged.getValue(field) == null)
|
||||
{
|
||||
record.setValue(field, field.equals("hipaaEligible") ? Boolean.FALSE : (field.equals("connectTimeoutSeconds") ? 10 : 120));
|
||||
}
|
||||
}
|
||||
for(String error : service.validate(provider))
|
||||
{
|
||||
record.addError(new BadInputStatusMessage(error));
|
||||
}
|
||||
@@ -169,7 +203,7 @@ public class AssistantSettingsCustomizer implements TableCustomizerInterface
|
||||
for(Map.Entry<String, Serializable> entry : record.getValues().entrySet())
|
||||
{
|
||||
String name = entry.getKey();
|
||||
boolean system = name.equals("id") || name.equals("modifyDate") || name.startsWith("lastTest")
|
||||
boolean system = name.equals("id") || name.equals("modifyDate") || name.startsWith("lastTest") || name.startsWith("lastDiscovery")
|
||||
|| (name.equals("isDefault") && Boolean.FALSE.equals(entry.getValue()));
|
||||
if(!system)
|
||||
{
|
||||
|
||||
@@ -63,7 +63,7 @@ public class AssistantMessage extends QRecordEntity
|
||||
@QField(isEditable = false)
|
||||
private Integer seq;
|
||||
|
||||
@QField(isEditable = false, maxLength = 64)
|
||||
@QField(isEditable = false, maxLength = 200)
|
||||
private String model;
|
||||
|
||||
@QField(isEditable = false)
|
||||
|
||||
@@ -0,0 +1,767 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.model;
|
||||
|
||||
|
||||
import java.math.BigDecimal;
|
||||
import java.time.Instant;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QField;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A provider-scoped model, its capabilities, prices and last connection test.
|
||||
*******************************************************************************/
|
||||
public class AssistantModel extends QRecordEntity
|
||||
{
|
||||
public static final String TABLE_NAME = "assistant_model";
|
||||
|
||||
@QField(isEditable = false, isPrimaryKey = true)
|
||||
private Integer id;
|
||||
|
||||
@QField(isRequired = true)
|
||||
private Integer providerId;
|
||||
|
||||
@QField(isRequired = true, maxLength = 200)
|
||||
private String modelId;
|
||||
|
||||
@QField(maxLength = 200)
|
||||
private String displayName;
|
||||
|
||||
@QField()
|
||||
private Boolean enabled;
|
||||
|
||||
@QField()
|
||||
private Boolean supportsTools;
|
||||
|
||||
@QField()
|
||||
private Boolean supportsVision;
|
||||
|
||||
@QField()
|
||||
private Boolean supportsStreaming;
|
||||
|
||||
@QField()
|
||||
private Boolean supportsReasoningEffort;
|
||||
|
||||
@QField()
|
||||
private Integer contextWindowTokens;
|
||||
|
||||
@QField()
|
||||
private Integer maxOutputTokens;
|
||||
|
||||
@QField()
|
||||
private BigDecimal inputUsdPerMTok;
|
||||
|
||||
@QField()
|
||||
private BigDecimal outputUsdPerMTok;
|
||||
|
||||
@QField(maxLength = 16)
|
||||
private String source;
|
||||
|
||||
@QField(isEditable = false)
|
||||
private Instant discoveredAt;
|
||||
|
||||
@QField(isEditable = false)
|
||||
private Instant lastTestAt;
|
||||
|
||||
@QField(isEditable = false, maxLength = 16)
|
||||
private String lastTestStatus;
|
||||
|
||||
@QField(isEditable = false)
|
||||
private Integer lastTestLatencyMs;
|
||||
|
||||
@QField(isEditable = false, maxLength = 500)
|
||||
private String lastTestError;
|
||||
|
||||
@QField(isEditable = false)
|
||||
private Instant createDate;
|
||||
|
||||
@QField(isEditable = false)
|
||||
private Instant modifyDate;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Default constructor.
|
||||
*******************************************************************************/
|
||||
public AssistantModel()
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Populate from a record.
|
||||
*******************************************************************************/
|
||||
public AssistantModel(QRecord record) throws QException
|
||||
{
|
||||
populateFromQRecord(record);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for id
|
||||
*******************************************************************************/
|
||||
public Integer getId()
|
||||
{
|
||||
return (this.id);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for id
|
||||
*******************************************************************************/
|
||||
public void setId(Integer id)
|
||||
{
|
||||
this.id = id;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for id
|
||||
*******************************************************************************/
|
||||
public AssistantModel withId(Integer id)
|
||||
{
|
||||
this.id = id;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for providerId
|
||||
*******************************************************************************/
|
||||
public Integer getProviderId()
|
||||
{
|
||||
return (this.providerId);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for providerId
|
||||
*******************************************************************************/
|
||||
public void setProviderId(Integer providerId)
|
||||
{
|
||||
this.providerId = providerId;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for providerId
|
||||
*******************************************************************************/
|
||||
public AssistantModel withProviderId(Integer providerId)
|
||||
{
|
||||
this.providerId = providerId;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for modelId
|
||||
*******************************************************************************/
|
||||
public String getModelId()
|
||||
{
|
||||
return (this.modelId);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for modelId
|
||||
*******************************************************************************/
|
||||
public void setModelId(String modelId)
|
||||
{
|
||||
this.modelId = modelId;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for modelId
|
||||
*******************************************************************************/
|
||||
public AssistantModel withModelId(String modelId)
|
||||
{
|
||||
this.modelId = modelId;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for displayName
|
||||
*******************************************************************************/
|
||||
public String getDisplayName()
|
||||
{
|
||||
return (this.displayName);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for displayName
|
||||
*******************************************************************************/
|
||||
public void setDisplayName(String displayName)
|
||||
{
|
||||
this.displayName = displayName;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for displayName
|
||||
*******************************************************************************/
|
||||
public AssistantModel withDisplayName(String displayName)
|
||||
{
|
||||
this.displayName = displayName;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for enabled
|
||||
*******************************************************************************/
|
||||
public Boolean getEnabled()
|
||||
{
|
||||
return (this.enabled);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for enabled
|
||||
*******************************************************************************/
|
||||
public void setEnabled(Boolean enabled)
|
||||
{
|
||||
this.enabled = enabled;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for enabled
|
||||
*******************************************************************************/
|
||||
public AssistantModel withEnabled(Boolean enabled)
|
||||
{
|
||||
this.enabled = enabled;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for supportsTools
|
||||
*******************************************************************************/
|
||||
public Boolean getSupportsTools()
|
||||
{
|
||||
return (this.supportsTools);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for supportsTools
|
||||
*******************************************************************************/
|
||||
public void setSupportsTools(Boolean supportsTools)
|
||||
{
|
||||
this.supportsTools = supportsTools;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for supportsTools
|
||||
*******************************************************************************/
|
||||
public AssistantModel withSupportsTools(Boolean supportsTools)
|
||||
{
|
||||
this.supportsTools = supportsTools;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for supportsVision
|
||||
*******************************************************************************/
|
||||
public Boolean getSupportsVision()
|
||||
{
|
||||
return (this.supportsVision);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for supportsVision
|
||||
*******************************************************************************/
|
||||
public void setSupportsVision(Boolean supportsVision)
|
||||
{
|
||||
this.supportsVision = supportsVision;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for supportsVision
|
||||
*******************************************************************************/
|
||||
public AssistantModel withSupportsVision(Boolean supportsVision)
|
||||
{
|
||||
this.supportsVision = supportsVision;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for supportsStreaming
|
||||
*******************************************************************************/
|
||||
public Boolean getSupportsStreaming()
|
||||
{
|
||||
return (this.supportsStreaming);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for supportsStreaming
|
||||
*******************************************************************************/
|
||||
public void setSupportsStreaming(Boolean supportsStreaming)
|
||||
{
|
||||
this.supportsStreaming = supportsStreaming;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for supportsStreaming
|
||||
*******************************************************************************/
|
||||
public AssistantModel withSupportsStreaming(Boolean supportsStreaming)
|
||||
{
|
||||
this.supportsStreaming = supportsStreaming;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for supportsReasoningEffort
|
||||
*******************************************************************************/
|
||||
public Boolean getSupportsReasoningEffort()
|
||||
{
|
||||
return (this.supportsReasoningEffort);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for supportsReasoningEffort
|
||||
*******************************************************************************/
|
||||
public void setSupportsReasoningEffort(Boolean supportsReasoningEffort)
|
||||
{
|
||||
this.supportsReasoningEffort = supportsReasoningEffort;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for supportsReasoningEffort
|
||||
*******************************************************************************/
|
||||
public AssistantModel withSupportsReasoningEffort(Boolean supportsReasoningEffort)
|
||||
{
|
||||
this.supportsReasoningEffort = supportsReasoningEffort;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for contextWindowTokens
|
||||
*******************************************************************************/
|
||||
public Integer getContextWindowTokens()
|
||||
{
|
||||
return (this.contextWindowTokens);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for contextWindowTokens
|
||||
*******************************************************************************/
|
||||
public void setContextWindowTokens(Integer contextWindowTokens)
|
||||
{
|
||||
this.contextWindowTokens = contextWindowTokens;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for contextWindowTokens
|
||||
*******************************************************************************/
|
||||
public AssistantModel withContextWindowTokens(Integer contextWindowTokens)
|
||||
{
|
||||
this.contextWindowTokens = contextWindowTokens;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for maxOutputTokens
|
||||
*******************************************************************************/
|
||||
public Integer getMaxOutputTokens()
|
||||
{
|
||||
return (this.maxOutputTokens);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for maxOutputTokens
|
||||
*******************************************************************************/
|
||||
public void setMaxOutputTokens(Integer maxOutputTokens)
|
||||
{
|
||||
this.maxOutputTokens = maxOutputTokens;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for maxOutputTokens
|
||||
*******************************************************************************/
|
||||
public AssistantModel withMaxOutputTokens(Integer maxOutputTokens)
|
||||
{
|
||||
this.maxOutputTokens = maxOutputTokens;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for inputUsdPerMTok
|
||||
*******************************************************************************/
|
||||
public BigDecimal getInputUsdPerMTok()
|
||||
{
|
||||
return (this.inputUsdPerMTok);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for inputUsdPerMTok
|
||||
*******************************************************************************/
|
||||
public void setInputUsdPerMTok(BigDecimal inputUsdPerMTok)
|
||||
{
|
||||
this.inputUsdPerMTok = inputUsdPerMTok;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for inputUsdPerMTok
|
||||
*******************************************************************************/
|
||||
public AssistantModel withInputUsdPerMTok(BigDecimal inputUsdPerMTok)
|
||||
{
|
||||
this.inputUsdPerMTok = inputUsdPerMTok;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for outputUsdPerMTok
|
||||
*******************************************************************************/
|
||||
public BigDecimal getOutputUsdPerMTok()
|
||||
{
|
||||
return (this.outputUsdPerMTok);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for outputUsdPerMTok
|
||||
*******************************************************************************/
|
||||
public void setOutputUsdPerMTok(BigDecimal outputUsdPerMTok)
|
||||
{
|
||||
this.outputUsdPerMTok = outputUsdPerMTok;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for outputUsdPerMTok
|
||||
*******************************************************************************/
|
||||
public AssistantModel withOutputUsdPerMTok(BigDecimal outputUsdPerMTok)
|
||||
{
|
||||
this.outputUsdPerMTok = outputUsdPerMTok;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for source
|
||||
*******************************************************************************/
|
||||
public String getSource()
|
||||
{
|
||||
return (this.source);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for source
|
||||
*******************************************************************************/
|
||||
public void setSource(String source)
|
||||
{
|
||||
this.source = source;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for source
|
||||
*******************************************************************************/
|
||||
public AssistantModel withSource(String source)
|
||||
{
|
||||
this.source = source;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for discoveredAt
|
||||
*******************************************************************************/
|
||||
public Instant getDiscoveredAt()
|
||||
{
|
||||
return (this.discoveredAt);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for discoveredAt
|
||||
*******************************************************************************/
|
||||
public void setDiscoveredAt(Instant discoveredAt)
|
||||
{
|
||||
this.discoveredAt = discoveredAt;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for discoveredAt
|
||||
*******************************************************************************/
|
||||
public AssistantModel withDiscoveredAt(Instant discoveredAt)
|
||||
{
|
||||
this.discoveredAt = discoveredAt;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for lastTestAt
|
||||
*******************************************************************************/
|
||||
public Instant getLastTestAt()
|
||||
{
|
||||
return (this.lastTestAt);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for lastTestAt
|
||||
*******************************************************************************/
|
||||
public void setLastTestAt(Instant lastTestAt)
|
||||
{
|
||||
this.lastTestAt = lastTestAt;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for lastTestAt
|
||||
*******************************************************************************/
|
||||
public AssistantModel withLastTestAt(Instant lastTestAt)
|
||||
{
|
||||
this.lastTestAt = lastTestAt;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for lastTestStatus
|
||||
*******************************************************************************/
|
||||
public String getLastTestStatus()
|
||||
{
|
||||
return (this.lastTestStatus);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for lastTestStatus
|
||||
*******************************************************************************/
|
||||
public void setLastTestStatus(String lastTestStatus)
|
||||
{
|
||||
this.lastTestStatus = lastTestStatus;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for lastTestStatus
|
||||
*******************************************************************************/
|
||||
public AssistantModel withLastTestStatus(String lastTestStatus)
|
||||
{
|
||||
this.lastTestStatus = lastTestStatus;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for lastTestLatencyMs
|
||||
*******************************************************************************/
|
||||
public Integer getLastTestLatencyMs()
|
||||
{
|
||||
return (this.lastTestLatencyMs);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for lastTestLatencyMs
|
||||
*******************************************************************************/
|
||||
public void setLastTestLatencyMs(Integer lastTestLatencyMs)
|
||||
{
|
||||
this.lastTestLatencyMs = lastTestLatencyMs;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for lastTestLatencyMs
|
||||
*******************************************************************************/
|
||||
public AssistantModel withLastTestLatencyMs(Integer lastTestLatencyMs)
|
||||
{
|
||||
this.lastTestLatencyMs = lastTestLatencyMs;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for lastTestError
|
||||
*******************************************************************************/
|
||||
public String getLastTestError()
|
||||
{
|
||||
return (this.lastTestError);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for lastTestError
|
||||
*******************************************************************************/
|
||||
public void setLastTestError(String lastTestError)
|
||||
{
|
||||
this.lastTestError = lastTestError;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for lastTestError
|
||||
*******************************************************************************/
|
||||
public AssistantModel withLastTestError(String lastTestError)
|
||||
{
|
||||
this.lastTestError = lastTestError;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for createDate
|
||||
*******************************************************************************/
|
||||
public Instant getCreateDate()
|
||||
{
|
||||
return (this.createDate);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for createDate
|
||||
*******************************************************************************/
|
||||
public void setCreateDate(Instant createDate)
|
||||
{
|
||||
this.createDate = createDate;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for createDate
|
||||
*******************************************************************************/
|
||||
public AssistantModel withCreateDate(Instant createDate)
|
||||
{
|
||||
this.createDate = createDate;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for modifyDate
|
||||
*******************************************************************************/
|
||||
public Instant getModifyDate()
|
||||
{
|
||||
return (this.modifyDate);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for modifyDate
|
||||
*******************************************************************************/
|
||||
public void setModifyDate(Instant modifyDate)
|
||||
{
|
||||
this.modifyDate = modifyDate;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for modifyDate
|
||||
*******************************************************************************/
|
||||
public AssistantModel withModifyDate(Instant modifyDate)
|
||||
{
|
||||
this.modifyDate = modifyDate;
|
||||
return (this);
|
||||
}
|
||||
|
||||
}
|
||||
@@ -35,6 +35,37 @@ import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
|
||||
*******************************************************************************/
|
||||
public class AssistantProvider extends QRecordEntity
|
||||
{
|
||||
@QField(maxLength = 30)
|
||||
private String kind;
|
||||
|
||||
@QField(maxLength = 10)
|
||||
private String location;
|
||||
|
||||
@QField()
|
||||
private Boolean hipaaEligible;
|
||||
|
||||
@QField(maxLength = 50)
|
||||
private String region;
|
||||
|
||||
@QField(maxLength = 30)
|
||||
private String apiVersion;
|
||||
|
||||
@QField()
|
||||
private Integer connectTimeoutSeconds;
|
||||
|
||||
@QField()
|
||||
private Integer requestTimeoutSeconds;
|
||||
|
||||
@QField(isEditable = false)
|
||||
private Instant lastDiscoveryAt;
|
||||
|
||||
@QField(isEditable = false, maxLength = 16)
|
||||
private String lastDiscoveryStatus;
|
||||
|
||||
@QField(isEditable = false, maxLength = 500)
|
||||
private String lastDiscoveryError;
|
||||
|
||||
|
||||
public static final String TABLE_NAME = "assistant_provider";
|
||||
|
||||
@QField(isEditable = false, isPrimaryKey = true)
|
||||
@@ -46,7 +77,7 @@ public class AssistantProvider extends QRecordEntity
|
||||
@QField(isRequired = true, maxLength = 50)
|
||||
private String providerType;
|
||||
|
||||
@QField(maxLength = 100)
|
||||
@QField(maxLength = 200)
|
||||
private String model;
|
||||
|
||||
@QField()
|
||||
@@ -58,6 +89,9 @@ public class AssistantProvider extends QRecordEntity
|
||||
@QField(maxLength = 255)
|
||||
private String baseUrl;
|
||||
|
||||
@QField(maxLength = 100)
|
||||
private String tlsCaBundle;
|
||||
|
||||
@QField(maxLength = 100)
|
||||
private String apiKeyEnvName;
|
||||
|
||||
@@ -86,7 +120,7 @@ public class AssistantProvider extends QRecordEntity
|
||||
@QField(isEditable = false)
|
||||
private Integer lastTestLatencyMs;
|
||||
|
||||
@QField(isEditable = false, maxLength = 100)
|
||||
@QField(isEditable = false, maxLength = 200)
|
||||
private String lastTestModel;
|
||||
|
||||
@QField(isEditable = false, maxLength = 500)
|
||||
@@ -706,4 +740,346 @@ public class AssistantProvider extends QRecordEntity
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for kind
|
||||
*******************************************************************************/
|
||||
public String getKind()
|
||||
{
|
||||
return (this.kind);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for kind
|
||||
*******************************************************************************/
|
||||
public void setKind(String kind)
|
||||
{
|
||||
this.kind = kind;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for kind
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withKind(String kind)
|
||||
{
|
||||
this.kind = kind;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for location
|
||||
*******************************************************************************/
|
||||
public String getLocation()
|
||||
{
|
||||
return (this.location);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for location
|
||||
*******************************************************************************/
|
||||
public void setLocation(String location)
|
||||
{
|
||||
this.location = location;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for location
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withLocation(String location)
|
||||
{
|
||||
this.location = location;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for hipaaEligible
|
||||
*******************************************************************************/
|
||||
public Boolean getHipaaEligible()
|
||||
{
|
||||
return (this.hipaaEligible);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for hipaaEligible
|
||||
*******************************************************************************/
|
||||
public void setHipaaEligible(Boolean hipaaEligible)
|
||||
{
|
||||
this.hipaaEligible = hipaaEligible;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for hipaaEligible
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withHipaaEligible(Boolean hipaaEligible)
|
||||
{
|
||||
this.hipaaEligible = hipaaEligible;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for region
|
||||
*******************************************************************************/
|
||||
public String getRegion()
|
||||
{
|
||||
return (this.region);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for region
|
||||
*******************************************************************************/
|
||||
public void setRegion(String region)
|
||||
{
|
||||
this.region = region;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for region
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withRegion(String region)
|
||||
{
|
||||
this.region = region;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for apiVersion
|
||||
*******************************************************************************/
|
||||
public String getApiVersion()
|
||||
{
|
||||
return (this.apiVersion);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for apiVersion
|
||||
*******************************************************************************/
|
||||
public void setApiVersion(String apiVersion)
|
||||
{
|
||||
this.apiVersion = apiVersion;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for apiVersion
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withApiVersion(String apiVersion)
|
||||
{
|
||||
this.apiVersion = apiVersion;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for connectTimeoutSeconds
|
||||
*******************************************************************************/
|
||||
public Integer getConnectTimeoutSeconds()
|
||||
{
|
||||
return (this.connectTimeoutSeconds);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for connectTimeoutSeconds
|
||||
*******************************************************************************/
|
||||
public void setConnectTimeoutSeconds(Integer connectTimeoutSeconds)
|
||||
{
|
||||
this.connectTimeoutSeconds = connectTimeoutSeconds;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for connectTimeoutSeconds
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withConnectTimeoutSeconds(Integer connectTimeoutSeconds)
|
||||
{
|
||||
this.connectTimeoutSeconds = connectTimeoutSeconds;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for requestTimeoutSeconds
|
||||
*******************************************************************************/
|
||||
public Integer getRequestTimeoutSeconds()
|
||||
{
|
||||
return (this.requestTimeoutSeconds);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for requestTimeoutSeconds
|
||||
*******************************************************************************/
|
||||
public void setRequestTimeoutSeconds(Integer requestTimeoutSeconds)
|
||||
{
|
||||
this.requestTimeoutSeconds = requestTimeoutSeconds;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for requestTimeoutSeconds
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withRequestTimeoutSeconds(Integer requestTimeoutSeconds)
|
||||
{
|
||||
this.requestTimeoutSeconds = requestTimeoutSeconds;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for lastDiscoveryAt
|
||||
*******************************************************************************/
|
||||
public Instant getLastDiscoveryAt()
|
||||
{
|
||||
return (this.lastDiscoveryAt);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for lastDiscoveryAt
|
||||
*******************************************************************************/
|
||||
public void setLastDiscoveryAt(Instant lastDiscoveryAt)
|
||||
{
|
||||
this.lastDiscoveryAt = lastDiscoveryAt;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for lastDiscoveryAt
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withLastDiscoveryAt(Instant lastDiscoveryAt)
|
||||
{
|
||||
this.lastDiscoveryAt = lastDiscoveryAt;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for lastDiscoveryStatus
|
||||
*******************************************************************************/
|
||||
public String getLastDiscoveryStatus()
|
||||
{
|
||||
return (this.lastDiscoveryStatus);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for lastDiscoveryStatus
|
||||
*******************************************************************************/
|
||||
public void setLastDiscoveryStatus(String lastDiscoveryStatus)
|
||||
{
|
||||
this.lastDiscoveryStatus = lastDiscoveryStatus;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for lastDiscoveryStatus
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withLastDiscoveryStatus(String lastDiscoveryStatus)
|
||||
{
|
||||
this.lastDiscoveryStatus = lastDiscoveryStatus;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for lastDiscoveryError
|
||||
*******************************************************************************/
|
||||
public String getLastDiscoveryError()
|
||||
{
|
||||
return (this.lastDiscoveryError);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for lastDiscoveryError
|
||||
*******************************************************************************/
|
||||
public void setLastDiscoveryError(String lastDiscoveryError)
|
||||
{
|
||||
this.lastDiscoveryError = lastDiscoveryError;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for lastDiscoveryError
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withLastDiscoveryError(String lastDiscoveryError)
|
||||
{
|
||||
this.lastDiscoveryError = lastDiscoveryError;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for tlsCaBundle
|
||||
*******************************************************************************/
|
||||
public String getTlsCaBundle()
|
||||
{
|
||||
return (this.tlsCaBundle);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for tlsCaBundle
|
||||
*******************************************************************************/
|
||||
public void setTlsCaBundle(String tlsCaBundle)
|
||||
{
|
||||
this.tlsCaBundle = tlsCaBundle;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for tlsCaBundle
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withTlsCaBundle(String tlsCaBundle)
|
||||
{
|
||||
this.tlsCaBundle = tlsCaBundle;
|
||||
return (this);
|
||||
}
|
||||
}
|
||||
@@ -65,7 +65,7 @@ public class AssistantTurn extends QRecordEntity
|
||||
@QField(isEditable = false, maxLength = 64)
|
||||
private String cancelReason;
|
||||
|
||||
@QField(isEditable = false, maxLength = 64)
|
||||
@QField(isEditable = false, maxLength = 200)
|
||||
private String model;
|
||||
|
||||
@QField(isEditable = false)
|
||||
|
||||
@@ -51,7 +51,7 @@ public class AssistantUsage extends QRecordEntity
|
||||
@QField(isEditable = false, maxLength = 50)
|
||||
private String providerType;
|
||||
|
||||
@QField(isEditable = false, maxLength = 100)
|
||||
@QField(isEditable = false, maxLength = 200)
|
||||
private String model;
|
||||
|
||||
@QField(isEditable = false, maxLength = 16)
|
||||
|
||||
@@ -28,6 +28,7 @@ import java.util.concurrent.ConcurrentHashMap;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantConversation;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantFeedback;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantMessage;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantModel;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantModelPrice;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantPendingAction;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantPolicy;
|
||||
@@ -728,4 +729,15 @@ public class AssistantStore
|
||||
return (records == null ? List.of() : records);
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** All registry models in stable id order.
|
||||
*******************************************************************************/
|
||||
public List<AssistantModel> listModels() throws QException
|
||||
{
|
||||
return (query(AssistantModel.TABLE_NAME, new QQueryFilter().withOrderBy(new QFilterOrderBy("id", true)), AssistantModel.class));
|
||||
}
|
||||
|
||||
}
|
||||
@@ -0,0 +1,216 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.processes;
|
||||
|
||||
|
||||
import java.io.Serializable;
|
||||
import java.util.ArrayList;
|
||||
import java.util.HashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import com.kingsrook.qbits.assistant.AssistantRuntime;
|
||||
import com.kingsrook.qbits.assistant.engine.ModelDiscovery;
|
||||
import com.kingsrook.qbits.assistant.engine.ProviderService;
|
||||
import com.kingsrook.qbits.assistant.metadata.AssistantMetaDataBuilder;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantProvider;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
|
||||
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractLoadStep;
|
||||
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractTransformStep;
|
||||
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ExtractViaQueryStep;
|
||||
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ProcessSummaryProviderInterface;
|
||||
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Discover models as a standard streamed ETL record action. Review inspects
|
||||
** settings without network calls; Result adds missing models and records status.
|
||||
*******************************************************************************/
|
||||
public final class DiscoverModelsProcess
|
||||
{
|
||||
public static final String NAME = "assistantProviderDiscoverModels";
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private DiscoverModelsProcess()
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The process.
|
||||
*******************************************************************************/
|
||||
public static QProcessMetaData process()
|
||||
{
|
||||
Map<String, Serializable> values = new HashMap<>();
|
||||
values.put(StreamedETLWithFrontendProcess.FIELD_SOURCE_TABLE, AssistantProvider.TABLE_NAME);
|
||||
values.put(StreamedETLWithFrontendProcess.FIELD_DESTINATION_TABLE, AssistantProvider.TABLE_NAME);
|
||||
values.put(StreamedETLWithFrontendProcess.FIELD_SUPPORTS_FULL_VALIDATION, true);
|
||||
values.put(StreamedETLWithFrontendProcess.FIELD_DO_FULL_VALIDATION, true);
|
||||
values.put(StreamedETLWithFrontendProcess.FIELD_PREVIEW_MESSAGE,
|
||||
"This lists available models and adds missing registry rows. Existing models are preserved.");
|
||||
QProcessMetaData process = StreamedETLWithFrontendProcess.defineProcessMetaData(ExtractViaQueryStep.class, Transform.class, Load.class, values);
|
||||
process.setName(NAME);
|
||||
process.setLabel("Discover models");
|
||||
process.setIcon(new QIcon("network_check"));
|
||||
process.setTableName(AssistantProvider.TABLE_NAME);
|
||||
process.setPermissionRules(AssistantMetaDataBuilder.adminRules());
|
||||
return (process);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A summary line with future (Review) and past (Result) wording.
|
||||
*******************************************************************************/
|
||||
static ProcessSummaryLine line(Status status, String singularFuture, String pluralFuture, String singularPast, String pluralPast)
|
||||
{
|
||||
ProcessSummaryLine line = new ProcessSummaryLine(status, 0, null);
|
||||
line.setSingularFutureMessage(singularFuture);
|
||||
line.setPluralFutureMessage(pluralFuture);
|
||||
line.setSingularPastMessage(singularPast);
|
||||
line.setPluralPastMessage(pluralPast);
|
||||
return (line);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A one-record line with the same text on both screens.
|
||||
*******************************************************************************/
|
||||
static ProcessSummaryLine single(Status status, String text, Serializable primaryKey)
|
||||
{
|
||||
ProcessSummaryLine line = line(status, text, text, text, text);
|
||||
line.incrementCountAndAddPrimaryKey(primaryKey);
|
||||
return (line);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Review: nothing is called yet.
|
||||
*******************************************************************************/
|
||||
public static class Transform extends AbstractTransformStep implements ProcessSummaryProviderInterface
|
||||
{
|
||||
private final ProcessSummaryLine willTest = line(Status.OK, "provider will be queried for models", "providers will be queried for models",
|
||||
"provider was queried", "providers were queried");
|
||||
private final ProcessSummaryLine disabled = line(Status.INFO, "provider is turned off (it is tested anyway)", "providers are turned off (they are tested anyway)",
|
||||
"provider was turned off", "providers were turned off");
|
||||
private final List<ProcessSummaryLineInterface> problems = new ArrayList<>();
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
|
||||
{
|
||||
AssistantRuntime runtime = AssistantRuntime.of(QContext.getQInstance());
|
||||
for(QRecord record : input.getRecords())
|
||||
{
|
||||
AssistantProvider row = new AssistantProvider(record);
|
||||
willTest.incrementCountAndAddPrimaryKey(row.getId());
|
||||
if(!Boolean.TRUE.equals(row.getEnabled()))
|
||||
{
|
||||
disabled.incrementCountAndAddPrimaryKey(row.getId());
|
||||
}
|
||||
if(runtime != null)
|
||||
{
|
||||
ProviderService service = runtime.getProviderService();
|
||||
for(String error : service.validate(row))
|
||||
{
|
||||
problems.add(single(Status.ERROR, row.getName() + ": " + error, row.getId()));
|
||||
}
|
||||
problems.add(single(Status.INFO, row.getName() + ": API key " + service.keyStatus(row), row.getId()));
|
||||
}
|
||||
output.getRecords().add(record);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean isForResultScreen)
|
||||
{
|
||||
ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>();
|
||||
willTest.addSelfToListIfAnyCount(lines);
|
||||
disabled.addSelfToListIfAnyCount(lines);
|
||||
lines.addAll(problems);
|
||||
return (lines);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Result: one line per provider.
|
||||
*******************************************************************************/
|
||||
public static class Load extends AbstractLoadStep implements ProcessSummaryProviderInterface
|
||||
{
|
||||
private final List<ProcessSummaryLineInterface> lines = new ArrayList<>();
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
|
||||
{
|
||||
AssistantRuntime runtime = AssistantRuntime.of(QContext.getQInstance());
|
||||
if(runtime == null)
|
||||
{
|
||||
throw (new QException("The assistant is not running on this server."));
|
||||
}
|
||||
for(QRecord record : input.getRecords())
|
||||
{
|
||||
AssistantProvider row = new AssistantProvider(record);
|
||||
ModelDiscovery.Result result = ModelDiscovery.run(runtime, row);
|
||||
lines.add(single(result.ok() ? Status.OK : Status.ERROR, row.getName() + ": " + result.message(), row.getId()));
|
||||
output.getRecords().add(record);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean isForResultScreen)
|
||||
{
|
||||
return (new ArrayList<>(lines));
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
@@ -201,13 +201,14 @@ public final class TestConnectionProcess
|
||||
{
|
||||
AssistantProvider row = new AssistantProvider(record);
|
||||
ConnectionTest.Result result = ConnectionTest.run(runtime, row, userId);
|
||||
String label = row.getName() + " (" + runtime.getProviderService().kind(row) + ", " + runtime.getProviderService().location(row) + ")";
|
||||
if(result.ok())
|
||||
{
|
||||
lines.add(single(Status.OK, row.getName() + ": connected to " + result.model() + " in " + result.latencyMs() + " ms", row.getId()));
|
||||
lines.add(single(Status.OK, label + ": connected to " + result.model() + " in " + result.latencyMs() + " ms", row.getId()));
|
||||
}
|
||||
else
|
||||
{
|
||||
lines.add(single(Status.ERROR, row.getName() + ": " + result.error(), row.getId()));
|
||||
lines.add(single(Status.ERROR, label + ": " + result.error(), row.getId()));
|
||||
}
|
||||
output.getRecords().add(record);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,225 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.processes;
|
||||
|
||||
|
||||
import java.io.Serializable;
|
||||
import java.util.ArrayList;
|
||||
import java.util.HashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import com.kingsrook.qbits.assistant.AssistantRuntime;
|
||||
import com.kingsrook.qbits.assistant.AssistantSecurity;
|
||||
import com.kingsrook.qbits.assistant.engine.ConnectionTest;
|
||||
import com.kingsrook.qbits.assistant.engine.ProviderService;
|
||||
import com.kingsrook.qbits.assistant.metadata.AssistantMetaDataBuilder;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantModel;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantProvider;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
|
||||
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractLoadStep;
|
||||
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractTransformStep;
|
||||
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ExtractViaQueryStep;
|
||||
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ProcessSummaryProviderInterface;
|
||||
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Test a registry model as a standard streamed ETL record action. Review
|
||||
** inspects its parent settings; Result records the model test and TEST usage.
|
||||
*******************************************************************************/
|
||||
public final class TestModelConnectionProcess
|
||||
{
|
||||
public static final String NAME = "assistantModelTestConnection";
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private TestModelConnectionProcess()
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The process.
|
||||
*******************************************************************************/
|
||||
public static QProcessMetaData process()
|
||||
{
|
||||
Map<String, Serializable> values = new HashMap<>();
|
||||
values.put(StreamedETLWithFrontendProcess.FIELD_SOURCE_TABLE, AssistantModel.TABLE_NAME);
|
||||
values.put(StreamedETLWithFrontendProcess.FIELD_DESTINATION_TABLE, AssistantModel.TABLE_NAME);
|
||||
values.put(StreamedETLWithFrontendProcess.FIELD_SUPPORTS_FULL_VALIDATION, true);
|
||||
values.put(StreamedETLWithFrontendProcess.FIELD_DO_FULL_VALIDATION, true);
|
||||
values.put(StreamedETLWithFrontendProcess.FIELD_PREVIEW_MESSAGE,
|
||||
"This makes one short live call to each selected registry model (a few tokens) and saves the result on the model.");
|
||||
QProcessMetaData process = StreamedETLWithFrontendProcess.defineProcessMetaData(ExtractViaQueryStep.class, Transform.class, Load.class, values);
|
||||
process.setName(NAME);
|
||||
process.setLabel("Test connection");
|
||||
process.setIcon(new QIcon("network_check"));
|
||||
process.setTableName(AssistantModel.TABLE_NAME);
|
||||
process.setPermissionRules(AssistantMetaDataBuilder.adminRules());
|
||||
return (process);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A summary line with future (Review) and past (Result) wording.
|
||||
*******************************************************************************/
|
||||
static ProcessSummaryLine line(Status status, String singularFuture, String pluralFuture, String singularPast, String pluralPast)
|
||||
{
|
||||
ProcessSummaryLine line = new ProcessSummaryLine(status, 0, null);
|
||||
line.setSingularFutureMessage(singularFuture);
|
||||
line.setPluralFutureMessage(pluralFuture);
|
||||
line.setSingularPastMessage(singularPast);
|
||||
line.setPluralPastMessage(pluralPast);
|
||||
return (line);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A one-record line with the same text on both screens.
|
||||
*******************************************************************************/
|
||||
static ProcessSummaryLine single(Status status, String text, Serializable primaryKey)
|
||||
{
|
||||
ProcessSummaryLine line = line(status, text, text, text, text);
|
||||
line.incrementCountAndAddPrimaryKey(primaryKey);
|
||||
return (line);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Review: nothing is called yet.
|
||||
*******************************************************************************/
|
||||
public static class Transform extends AbstractTransformStep implements ProcessSummaryProviderInterface
|
||||
{
|
||||
private final ProcessSummaryLine willTest = line(Status.OK, "model will be tested with a live call", "models will be tested with a live call",
|
||||
"model was tested", "models were tested");
|
||||
private final ProcessSummaryLine disabled = line(Status.INFO, "model is turned off (it is tested anyway)", "models are turned off (they are tested anyway)",
|
||||
"model was turned off", "models were turned off");
|
||||
private final List<ProcessSummaryLineInterface> problems = new ArrayList<>();
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
|
||||
{
|
||||
AssistantRuntime runtime = AssistantRuntime.of(QContext.getQInstance());
|
||||
for(QRecord record : input.getRecords())
|
||||
{
|
||||
AssistantModel model = new AssistantModel(record);
|
||||
AssistantProvider row = runtime == null ? null : runtime.getStore().getProvider(model.getProviderId());
|
||||
willTest.incrementCountAndAddPrimaryKey(model.getId());
|
||||
if(!Boolean.TRUE.equals(model.getEnabled()))
|
||||
{
|
||||
disabled.incrementCountAndAddPrimaryKey(model.getId());
|
||||
}
|
||||
if(runtime != null && row != null)
|
||||
{
|
||||
ProviderService service = runtime.getProviderService();
|
||||
for(String error : service.validate(row))
|
||||
{
|
||||
problems.add(single(Status.ERROR, model.getModelId() + ": " + error, model.getId()));
|
||||
}
|
||||
problems.add(single(Status.INFO, model.getModelId() + ": API key " + service.keyStatus(row), model.getId()));
|
||||
}
|
||||
if(row == null)
|
||||
{
|
||||
problems.add(single(Status.ERROR, "The model's provider no longer exists.", model.getId()));
|
||||
}
|
||||
output.getRecords().add(record);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean isForResultScreen)
|
||||
{
|
||||
ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>();
|
||||
willTest.addSelfToListIfAnyCount(lines);
|
||||
disabled.addSelfToListIfAnyCount(lines);
|
||||
lines.addAll(problems);
|
||||
return (lines);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Result: one line per provider.
|
||||
*******************************************************************************/
|
||||
public static class Load extends AbstractLoadStep implements ProcessSummaryProviderInterface
|
||||
{
|
||||
private final List<ProcessSummaryLineInterface> lines = new ArrayList<>();
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
|
||||
{
|
||||
AssistantRuntime runtime = AssistantRuntime.of(QContext.getQInstance());
|
||||
if(runtime == null)
|
||||
{
|
||||
throw (new QException("The assistant is not running on this server."));
|
||||
}
|
||||
String userId = AssistantSecurity.userId(QContext.getQSession());
|
||||
for(QRecord record : input.getRecords())
|
||||
{
|
||||
AssistantModel row = new AssistantModel(record);
|
||||
ConnectionTest.Result result = ConnectionTest.runModel(runtime, row, userId);
|
||||
lines.add(single(result.ok() ? Status.OK : Status.ERROR, row.getModelId() + ": "
|
||||
+ (result.ok() ? "connected in " + result.latencyMs() + " ms" : result.error()), row.getId()));
|
||||
output.getRecords().add(record);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean isForResultScreen)
|
||||
{
|
||||
return (new ArrayList<>(lines));
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
@@ -13,6 +13,7 @@
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.provider;
|
||||
|
||||
|
||||
@@ -63,4 +64,75 @@ public interface AssistantProviderType
|
||||
*******************************************************************************/
|
||||
AssistantModelProvider create(String apiKey, String baseUrl);
|
||||
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Backward-compatible provider registry extension.
|
||||
*******************************************************************************/
|
||||
default List<String> getKinds()
|
||||
{
|
||||
return (List.of(getType().toUpperCase(java.util.Locale.ROOT).replace('-', '_')));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Backward-compatible provider registry extension.
|
||||
*******************************************************************************/
|
||||
default String getDefaultBaseUrl(String kind)
|
||||
{
|
||||
return (getDefaultBaseUrl());
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Backward-compatible provider registry extension.
|
||||
*******************************************************************************/
|
||||
default String getDefaultApiKeyEnvName(String kind)
|
||||
{
|
||||
return (getDefaultApiKeyEnvName());
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Backward-compatible provider registry extension.
|
||||
*******************************************************************************/
|
||||
default String getDefaultLocation(String kind)
|
||||
{
|
||||
return ("CLOUD");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Backward-compatible provider registry extension.
|
||||
*******************************************************************************/
|
||||
default AssistantModelProvider create(ProviderSettings settings)
|
||||
{
|
||||
return (create(settings.getApiKey(), settings.getBaseUrl()));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Backward-compatible provider registry extension.
|
||||
*******************************************************************************/
|
||||
default List<DiscoveredModel> discoverModels(ProviderSettings settings) throws ModelProviderException
|
||||
{
|
||||
return (getModels().stream().map(id -> new DiscoveredModel(id, id, null, null)).toList());
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Backward-compatible provider registry extension.
|
||||
*******************************************************************************/
|
||||
default HealthResult checkHealth(ProviderSettings settings)
|
||||
{
|
||||
return (new HealthResult(null, null, "Health check not supported"));
|
||||
}
|
||||
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.provider;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A model advertised by the provider, without inferred capabilities.
|
||||
*******************************************************************************/
|
||||
public record DiscoveredModel(String modelId, String displayName, Integer contextWindowTokens, String ownedBy)
|
||||
{
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.provider;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A health probe result; null healthy means unsupported.
|
||||
*******************************************************************************/
|
||||
public record HealthResult(Boolean healthy, Long latencyMs, String message)
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Sanitize provider diagnostics.
|
||||
*******************************************************************************/
|
||||
public HealthResult
|
||||
{
|
||||
message = ModelProviderException.sanitizeProviderMessage(message, 300);
|
||||
}
|
||||
}
|
||||
@@ -23,6 +23,16 @@ package com.kingsrook.qbits.assistant.provider;
|
||||
*******************************************************************************/
|
||||
public class ModelCapabilities
|
||||
{
|
||||
|
||||
private Boolean vision = false;
|
||||
|
||||
|
||||
private Boolean streaming = true;
|
||||
|
||||
|
||||
private Boolean reasoningEffort = false;
|
||||
|
||||
|
||||
private Boolean tools = true;
|
||||
private Boolean parallelToolCalls = true;
|
||||
private Boolean promptCaching = false;
|
||||
@@ -182,4 +192,98 @@ public class ModelCapabilities
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for vision
|
||||
*******************************************************************************/
|
||||
public Boolean getVision()
|
||||
{
|
||||
return (this.vision);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for vision
|
||||
*******************************************************************************/
|
||||
public void setVision(Boolean vision)
|
||||
{
|
||||
this.vision = vision;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for vision
|
||||
*******************************************************************************/
|
||||
public ModelCapabilities withVision(Boolean vision)
|
||||
{
|
||||
this.vision = vision;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for streaming
|
||||
*******************************************************************************/
|
||||
public Boolean getStreaming()
|
||||
{
|
||||
return (this.streaming);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for streaming
|
||||
*******************************************************************************/
|
||||
public void setStreaming(Boolean streaming)
|
||||
{
|
||||
this.streaming = streaming;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for streaming
|
||||
*******************************************************************************/
|
||||
public ModelCapabilities withStreaming(Boolean streaming)
|
||||
{
|
||||
this.streaming = streaming;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for reasoningEffort
|
||||
*******************************************************************************/
|
||||
public Boolean getReasoningEffort()
|
||||
{
|
||||
return (this.reasoningEffort);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for reasoningEffort
|
||||
*******************************************************************************/
|
||||
public void setReasoningEffort(Boolean reasoningEffort)
|
||||
{
|
||||
this.reasoningEffort = reasoningEffort;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for reasoningEffort
|
||||
*******************************************************************************/
|
||||
public ModelCapabilities withReasoningEffort(Boolean reasoningEffort)
|
||||
{
|
||||
this.reasoningEffort = reasoningEffort;
|
||||
return (this);
|
||||
}
|
||||
|
||||
}
|
||||
@@ -32,6 +32,10 @@ import java.util.List;
|
||||
*******************************************************************************/
|
||||
public class ModelRequest
|
||||
{
|
||||
|
||||
private ModelCapabilities capabilities;
|
||||
|
||||
|
||||
private String model;
|
||||
private String effort;
|
||||
private Integer maxTokens;
|
||||
@@ -49,6 +53,7 @@ public class ModelRequest
|
||||
{
|
||||
return (new ModelRequest()
|
||||
.withModel(model)
|
||||
.withCapabilities(capabilities)
|
||||
.withEffort(effort)
|
||||
.withMaxTokens(maxTokens)
|
||||
.withParallelToolCalls(parallelToolCalls)
|
||||
@@ -274,4 +279,36 @@ public class ModelRequest
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for capabilities
|
||||
*******************************************************************************/
|
||||
public ModelCapabilities getCapabilities()
|
||||
{
|
||||
return (this.capabilities);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for capabilities
|
||||
*******************************************************************************/
|
||||
public void setCapabilities(ModelCapabilities capabilities)
|
||||
{
|
||||
this.capabilities = capabilities;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for capabilities
|
||||
*******************************************************************************/
|
||||
public ModelRequest withCapabilities(ModelCapabilities capabilities)
|
||||
{
|
||||
this.capabilities = capabilities;
|
||||
return (this);
|
||||
}
|
||||
|
||||
}
|
||||
@@ -0,0 +1,330 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.provider;
|
||||
|
||||
|
||||
import javax.net.ssl.SSLContext;
|
||||
import com.fasterxml.jackson.annotation.JsonIgnore;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Resolved connection settings. The API key is never included in diagnostics.
|
||||
*******************************************************************************/
|
||||
public class ProviderSettings
|
||||
{
|
||||
private String tlsCaBundle;
|
||||
@JsonIgnore
|
||||
private transient SSLContext sslContext;
|
||||
private String kind;
|
||||
private String apiKey;
|
||||
private String baseUrl;
|
||||
private String apiVersion;
|
||||
private String region;
|
||||
private Integer connectTimeoutSeconds = 10;
|
||||
private Integer requestTimeoutSeconds = 120;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for kind
|
||||
*******************************************************************************/
|
||||
public String getKind()
|
||||
{
|
||||
return (this.kind);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for kind
|
||||
*******************************************************************************/
|
||||
public void setKind(String kind)
|
||||
{
|
||||
this.kind = kind;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for kind
|
||||
*******************************************************************************/
|
||||
public ProviderSettings withKind(String kind)
|
||||
{
|
||||
this.kind = kind;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for apiKey
|
||||
*******************************************************************************/
|
||||
public String getApiKey()
|
||||
{
|
||||
return (this.apiKey);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for apiKey
|
||||
*******************************************************************************/
|
||||
public void setApiKey(String apiKey)
|
||||
{
|
||||
this.apiKey = apiKey;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for apiKey
|
||||
*******************************************************************************/
|
||||
public ProviderSettings withApiKey(String apiKey)
|
||||
{
|
||||
this.apiKey = apiKey;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for baseUrl
|
||||
*******************************************************************************/
|
||||
public String getBaseUrl()
|
||||
{
|
||||
return (this.baseUrl);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for baseUrl
|
||||
*******************************************************************************/
|
||||
public void setBaseUrl(String baseUrl)
|
||||
{
|
||||
this.baseUrl = baseUrl;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for baseUrl
|
||||
*******************************************************************************/
|
||||
public ProviderSettings withBaseUrl(String baseUrl)
|
||||
{
|
||||
this.baseUrl = baseUrl;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for apiVersion
|
||||
*******************************************************************************/
|
||||
public String getApiVersion()
|
||||
{
|
||||
return (this.apiVersion);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for apiVersion
|
||||
*******************************************************************************/
|
||||
public void setApiVersion(String apiVersion)
|
||||
{
|
||||
this.apiVersion = apiVersion;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for apiVersion
|
||||
*******************************************************************************/
|
||||
public ProviderSettings withApiVersion(String apiVersion)
|
||||
{
|
||||
this.apiVersion = apiVersion;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for region
|
||||
*******************************************************************************/
|
||||
public String getRegion()
|
||||
{
|
||||
return (this.region);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for region
|
||||
*******************************************************************************/
|
||||
public void setRegion(String region)
|
||||
{
|
||||
this.region = region;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for region
|
||||
*******************************************************************************/
|
||||
public ProviderSettings withRegion(String region)
|
||||
{
|
||||
this.region = region;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for connectTimeoutSeconds
|
||||
*******************************************************************************/
|
||||
public Integer getConnectTimeoutSeconds()
|
||||
{
|
||||
return (this.connectTimeoutSeconds);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for connectTimeoutSeconds
|
||||
*******************************************************************************/
|
||||
public void setConnectTimeoutSeconds(Integer connectTimeoutSeconds)
|
||||
{
|
||||
this.connectTimeoutSeconds = connectTimeoutSeconds;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for connectTimeoutSeconds
|
||||
*******************************************************************************/
|
||||
public ProviderSettings withConnectTimeoutSeconds(Integer connectTimeoutSeconds)
|
||||
{
|
||||
this.connectTimeoutSeconds = connectTimeoutSeconds;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for requestTimeoutSeconds
|
||||
*******************************************************************************/
|
||||
public Integer getRequestTimeoutSeconds()
|
||||
{
|
||||
return (this.requestTimeoutSeconds);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for requestTimeoutSeconds
|
||||
*******************************************************************************/
|
||||
public void setRequestTimeoutSeconds(Integer requestTimeoutSeconds)
|
||||
{
|
||||
this.requestTimeoutSeconds = requestTimeoutSeconds;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for requestTimeoutSeconds
|
||||
*******************************************************************************/
|
||||
public ProviderSettings withRequestTimeoutSeconds(Integer requestTimeoutSeconds)
|
||||
{
|
||||
this.requestTimeoutSeconds = requestTimeoutSeconds;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Safe diagnostic representation.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String toString()
|
||||
{
|
||||
return ("ProviderSettings[kind=" + kind + "]");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for tlsCaBundle
|
||||
*******************************************************************************/
|
||||
public String getTlsCaBundle()
|
||||
{
|
||||
return (this.tlsCaBundle);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for tlsCaBundle
|
||||
*******************************************************************************/
|
||||
public void setTlsCaBundle(String tlsCaBundle)
|
||||
{
|
||||
this.tlsCaBundle = tlsCaBundle;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for tlsCaBundle
|
||||
*******************************************************************************/
|
||||
public ProviderSettings withTlsCaBundle(String tlsCaBundle)
|
||||
{
|
||||
this.tlsCaBundle = tlsCaBundle;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for sslContext
|
||||
*******************************************************************************/
|
||||
@JsonIgnore
|
||||
public SSLContext getSslContext()
|
||||
{
|
||||
return (this.sslContext);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for sslContext
|
||||
*******************************************************************************/
|
||||
public void setSslContext(SSLContext sslContext)
|
||||
{
|
||||
this.sslContext = sslContext;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for sslContext
|
||||
*******************************************************************************/
|
||||
public ProviderSettings withSslContext(SSLContext sslContext)
|
||||
{
|
||||
this.sslContext = sslContext;
|
||||
return (this);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,97 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.provider;
|
||||
|
||||
|
||||
import com.kingsrook.qbits.assistant.model.AssistantModel;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Applies registry capabilities to both the engine and the adapter's request.
|
||||
*******************************************************************************/
|
||||
public class RegistryModelProvider implements AssistantModelProvider
|
||||
{
|
||||
private final AssistantModelProvider delegate;
|
||||
private final AssistantModel model;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** One resolved model's overrides.
|
||||
*******************************************************************************/
|
||||
public RegistryModelProvider(AssistantModelProvider delegate, AssistantModel model)
|
||||
{
|
||||
this.delegate = delegate;
|
||||
this.model = model;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Adapter identity.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getName()
|
||||
{
|
||||
return (delegate.getName());
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Copy defaults without mutating a cached adapter capability object.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public ModelCapabilities getCapabilities(String modelId)
|
||||
{
|
||||
ModelCapabilities defaults = delegate.getCapabilities(modelId);
|
||||
return (new ModelCapabilities()
|
||||
.withTools(model.getSupportsTools() == null ? defaults.getTools() : model.getSupportsTools())
|
||||
.withVision(model.getSupportsVision() == null ? defaults.getVision() : model.getSupportsVision())
|
||||
.withStreaming(model.getSupportsStreaming() == null ? defaults.getStreaming() : model.getSupportsStreaming())
|
||||
.withReasoningEffort(model.getSupportsReasoningEffort() == null ? defaults.getReasoningEffort() : model.getSupportsReasoningEffort())
|
||||
.withMaxContextTokens(model.getContextWindowTokens() == null ? defaults.getMaxContextTokens() : model.getContextWindowTokens())
|
||||
.withParallelToolCalls(defaults.getParallelToolCalls()).withThinking(defaults.getThinking()).withPromptCaching(defaults.getPromptCaching()));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Pass effective capabilities and the output ceiling to the adapter.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public StreamHandle stream(ModelRequest request, ModelEventListener listener)
|
||||
{
|
||||
ModelRequest effective = request.copy().withCapabilities(getCapabilities(request.getModel()));
|
||||
if(model.getMaxOutputTokens() != null)
|
||||
{
|
||||
effective.setMaxTokens(request.getMaxTokens() == null ? model.getMaxOutputTokens() : Math.min(request.getMaxTokens(), model.getMaxOutputTokens()));
|
||||
}
|
||||
return (delegate.stream(effective, listener));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Preserve optional token counting.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public TokenCount countTokens(ModelRequest request)
|
||||
{
|
||||
return (delegate.countTokens(request));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,153 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.provider;
|
||||
|
||||
|
||||
import java.io.ByteArrayInputStream;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.security.KeyStore;
|
||||
import java.security.cert.CertificateException;
|
||||
import java.security.cert.CertificateFactory;
|
||||
import java.security.cert.X509Certificate;
|
||||
import java.util.Arrays;
|
||||
import java.util.regex.Matcher;
|
||||
import java.util.regex.Pattern;
|
||||
import java.util.stream.Stream;
|
||||
import javax.net.ssl.SSLContext;
|
||||
import javax.net.ssl.TrustManager;
|
||||
import javax.net.ssl.TrustManagerFactory;
|
||||
import javax.net.ssl.X509TrustManager;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Builds additive trust: JVM roots OR explicitly configured certificates.
|
||||
** Endpoint identification remains the HTTP client's responsibility and is enabled.
|
||||
*******************************************************************************/
|
||||
public final class TrustedCaBundle implements X509TrustManager
|
||||
{
|
||||
private final X509TrustManager defaults;
|
||||
private final X509TrustManager additional;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Two independently validating trust managers, never a trust-all fallback.
|
||||
*******************************************************************************/
|
||||
TrustedCaBundle(X509TrustManager defaults, X509TrustManager additional)
|
||||
{
|
||||
this.defaults = defaults;
|
||||
this.additional = additional;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Accept only one or more complete PEM certificates; reject trailing garbage.
|
||||
*******************************************************************************/
|
||||
public static SSLContext parse(String pem)
|
||||
{
|
||||
try
|
||||
{
|
||||
KeyStore store = KeyStore.getInstance(KeyStore.getDefaultType());
|
||||
store.load(null, null);
|
||||
CertificateFactory factory = CertificateFactory.getInstance("X.509");
|
||||
Matcher matcher = Pattern.compile("-----BEGIN CERTIFICATE-----[\\s\\S]*?-----END CERTIFICATE-----").matcher(pem == null ? "" : pem);
|
||||
int end = 0;
|
||||
int count = 0;
|
||||
while(matcher.find())
|
||||
{
|
||||
if(!pem.substring(end, matcher.start()).isBlank())
|
||||
{
|
||||
throw (new CertificateException());
|
||||
}
|
||||
X509Certificate certificate = (X509Certificate) factory.generateCertificate(new ByteArrayInputStream(matcher.group().getBytes(StandardCharsets.US_ASCII)));
|
||||
store.setCertificateEntry("ca-" + count++, certificate);
|
||||
end = matcher.end();
|
||||
}
|
||||
if(count == 0 || !pem.substring(end).isBlank())
|
||||
{
|
||||
throw (new CertificateException());
|
||||
}
|
||||
SSLContext context = SSLContext.getInstance("TLS");
|
||||
context.init(null, new TrustManager[] { new TrustedCaBundle(manager(null), manager(store)) }, null);
|
||||
return (context);
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
throw (new IllegalArgumentException("Invalid CA bundle: expected one or more PEM X.509 certificates."));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A null store asks JSSE to load the JVM's configured default trust roots.
|
||||
*******************************************************************************/
|
||||
private static X509TrustManager manager(KeyStore store) throws Exception
|
||||
{
|
||||
TrustManagerFactory factory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
|
||||
factory.init(store);
|
||||
return (Arrays.stream(factory.getTrustManagers()).filter(X509TrustManager.class::isInstance).map(X509TrustManager.class::cast).findFirst().orElseThrow());
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Client validation uses the same additive trust rule.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void checkClientTrusted(X509Certificate[] chain, String authType) throws CertificateException
|
||||
{
|
||||
try
|
||||
{
|
||||
defaults.checkClientTrusted(chain, authType);
|
||||
}
|
||||
catch(CertificateException e)
|
||||
{
|
||||
additional.checkClientTrusted(chain, authType);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Try JVM roots first, then the named bundle, preserving chain validation.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void checkServerTrusted(X509Certificate[] chain, String authType) throws CertificateException
|
||||
{
|
||||
try
|
||||
{
|
||||
defaults.checkServerTrusted(chain, authType);
|
||||
}
|
||||
catch(CertificateException e)
|
||||
{
|
||||
additional.checkServerTrusted(chain, authType);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Both sets of issuers remain accepted.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public X509Certificate[] getAcceptedIssuers()
|
||||
{
|
||||
return (Stream.concat(Arrays.stream(defaults.getAcceptedIssuers()), Arrays.stream(additional.getAcceptedIssuers())).toArray(X509Certificate[]::new));
|
||||
}
|
||||
}
|
||||
@@ -132,7 +132,10 @@ class AssistantRouteProviderTest extends BaseAssistantTest
|
||||
List<Frame> frames = new ArrayList<>();
|
||||
try(BufferedReader reader = new BufferedReader(new InputStreamReader(body, StandardCharsets.UTF_8)))
|
||||
{
|
||||
String id = null, event = null, data = null, line;
|
||||
String id = null;
|
||||
String event = null;
|
||||
String data = null;
|
||||
String line;
|
||||
while((line = reader.readLine()) != null)
|
||||
{
|
||||
if(line.isEmpty())
|
||||
@@ -379,7 +382,7 @@ class AssistantRouteProviderTest extends BaseAssistantTest
|
||||
{
|
||||
assertThat(AssistantApi.deriveTitle(" how many posts?\nmore lines")).isEqualTo("how many posts?");
|
||||
String title = AssistantApi.deriveTitle("find every published blog post that is missing a meta description and an excerpt please");
|
||||
assertThat(title).endsWith("\u2026").hasSizeLessThanOrEqualTo(AssistantApi.TITLE_MAX_CHARS);
|
||||
assertThat(title).endsWith("…").hasSizeLessThanOrEqualTo(AssistantApi.TITLE_MAX_CHARS);
|
||||
assertThat(AssistantApi.deriveTitle("x".repeat(100))).hasSize(AssistantApi.TITLE_MAX_CHARS);
|
||||
}
|
||||
|
||||
|
||||
@@ -287,7 +287,10 @@ class EngineComponentsTest extends BaseAssistantTest
|
||||
});
|
||||
|
||||
CompletableStreamHandle cancelled = new CompletableStreamHandle();
|
||||
cancelled.setOnCancel(() -> { throw (new IllegalStateException("closing")); });
|
||||
cancelled.setOnCancel(() ->
|
||||
{
|
||||
throw (new IllegalStateException("closing"));
|
||||
});
|
||||
cancelled.cancel();
|
||||
cancelled.cancel();
|
||||
assertThat(cancelled.await(Duration.ofSeconds(1)).getStopReason()).isEqualTo(ModelResponse.STOP_CANCELLED);
|
||||
@@ -327,7 +330,17 @@ class EngineComponentsTest extends BaseAssistantTest
|
||||
Map<String, Object> bigFilter = ContextBuilder.sanitizePageContext(Map.of("tableName", "page", "filter", Map.of("criteria", "z".repeat(5000))));
|
||||
assertThat(bigFilter).containsOnlyKeys("tableName");
|
||||
|
||||
assertThat(TurnEngine.map("a", 1, "b", null, "c", new Object() { public String toString() { return ("obj"); } })).containsEntry("a", 1).doesNotContainKey("b").containsEntry("c", "obj");
|
||||
assertThat(TurnEngine.map("a", 1, "b", null, "c", new Object()
|
||||
{
|
||||
/*******************************************************************************
|
||||
** A value rendered through toString.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String toString()
|
||||
{
|
||||
return ("obj");
|
||||
}
|
||||
})).containsEntry("a", 1).doesNotContainKey("b").containsEntry("c", "obj");
|
||||
}
|
||||
|
||||
}
|
||||
@@ -0,0 +1,67 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.provider;
|
||||
|
||||
|
||||
import java.security.KeyStore;
|
||||
import java.security.cert.X509Certificate;
|
||||
import java.util.Arrays;
|
||||
import java.util.Base64;
|
||||
import javax.net.ssl.TrustManagerFactory;
|
||||
import javax.net.ssl.X509TrustManager;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Offline proof that adding a bundle does not replace the JVM's trust roots.
|
||||
*******************************************************************************/
|
||||
class TrustedCaBundleTest
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Use existing public JVM roots in memory; no network or fixture signing.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void keepsDefaultRoots() throws Exception
|
||||
{
|
||||
X509TrustManager defaults = manager(null);
|
||||
X509Certificate[] roots = defaults.getAcceptedIssuers();
|
||||
assertThat(roots.length).isGreaterThan(1);
|
||||
KeyStore extra = KeyStore.getInstance(KeyStore.getDefaultType());
|
||||
extra.load(null, null);
|
||||
extra.setCertificateEntry("one-root", roots[0]);
|
||||
TrustedCaBundle combined = new TrustedCaBundle(defaults, manager(extra));
|
||||
assertThat(combined.getAcceptedIssuers()).contains(roots);
|
||||
assertThat(combined.getAcceptedIssuers()).hasSize(roots.length + 1);
|
||||
combined.checkServerTrusted(new X509Certificate[] { roots[1] }, roots[1].getPublicKey().getAlgorithm());
|
||||
combined.checkClientTrusted(new X509Certificate[] { roots[1] }, roots[1].getPublicKey().getAlgorithm());
|
||||
String pem = "-----BEGIN CERTIFICATE-----\n" + Base64.getMimeEncoder(64, new byte[] { '\n' }).encodeToString(roots[0].getEncoded()) + "\n-----END CERTIFICATE-----\n";
|
||||
assertThat(TrustedCaBundle.parse(pem).getProtocol()).isEqualTo("TLS");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Resolve a real PKIX manager from either the JVM or the small extra store.
|
||||
*******************************************************************************/
|
||||
private X509TrustManager manager(KeyStore store) throws Exception
|
||||
{
|
||||
TrustManagerFactory factory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
|
||||
factory.init(store);
|
||||
return (Arrays.stream(factory.getTrustManagers()).filter(X509TrustManager.class::isInstance).map(X509TrustManager.class::cast).findFirst().orElseThrow());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,432 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.settings;
|
||||
|
||||
|
||||
import java.math.BigDecimal;
|
||||
import java.time.Duration;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import com.kingsrook.qbits.assistant.AssistantQBitConfig;
|
||||
import com.kingsrook.qbits.assistant.engine.ConnectionTest;
|
||||
import com.kingsrook.qbits.assistant.engine.ModelDiscovery;
|
||||
import com.kingsrook.qbits.assistant.engine.ModelPricing;
|
||||
import com.kingsrook.qbits.assistant.engine.ProviderService;
|
||||
import com.kingsrook.qbits.assistant.home.AssistantHomeData;
|
||||
import com.kingsrook.qbits.assistant.metadata.AssistantMetaDataBuilder;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantModel;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantModelPrice;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantProvider;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantUsage;
|
||||
import com.kingsrook.qbits.assistant.processes.DiscoverModelsProcess;
|
||||
import com.kingsrook.qbits.assistant.processes.TestModelConnectionProcess;
|
||||
import com.kingsrook.qbits.assistant.provider.AssistantModelProvider;
|
||||
import com.kingsrook.qbits.assistant.provider.AssistantProviderType;
|
||||
import com.kingsrook.qbits.assistant.provider.DiscoveredModel;
|
||||
import com.kingsrook.qbits.assistant.provider.HealthResult;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelRequest;
|
||||
import com.kingsrook.qbits.assistant.provider.ProviderSettings;
|
||||
import com.kingsrook.qbits.assistant.testing.AssistantTestFixture;
|
||||
import com.kingsrook.qbits.assistant.testing.BaseAssistantTest;
|
||||
import com.kingsrook.qqq.backend.core.actions.processes.QProcessCallbackFactory;
|
||||
import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Registry behavior through the memory-backed settings tables; no network.
|
||||
*******************************************************************************/
|
||||
class ProviderRegistryTest extends BaseAssistantTest
|
||||
{
|
||||
private final FakeType type = new FakeType();
|
||||
private int creates;
|
||||
private int discoveries;
|
||||
private int healthChecks;
|
||||
private boolean unhealthy;
|
||||
private boolean discoveryError;
|
||||
private int modelCount = 3;
|
||||
private String secret;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Test adapter with cloud and keyless local dialects.
|
||||
*******************************************************************************/
|
||||
class FakeType implements AssistantProviderType
|
||||
{
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Provider contract.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getType()
|
||||
{
|
||||
return ("openai-compatible");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Provider contract.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getLabel()
|
||||
{
|
||||
return ("Test registry");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Provider contract.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public List<String> getModels()
|
||||
{
|
||||
return (List.of("fallback"));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Provider contract.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public List<String> getKinds()
|
||||
{
|
||||
return (List.of("OLLAMA", "OPENAI", "AZURE_OPENAI", "OPENAI_COMPATIBLE"));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Provider contract.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getDefaultApiKeyEnvName()
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Provider contract.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getDefaultApiKeyEnvName(String kind)
|
||||
{
|
||||
return (kind.equals("OPENAI") ? "OPENAI_API_KEY" : (kind.equals("AZURE_OPENAI") ? "AZURE_OPENAI_API_KEY" : null));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Provider contract.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getDefaultBaseUrl()
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Provider contract.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getDefaultBaseUrl(String kind)
|
||||
{
|
||||
return (kind.equals("OLLAMA") ? "http://localhost:11434/v1" : (kind.equals("OPENAI") ? "https://api.openai.com/v1" : null));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Provider contract.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getDefaultLocation(String kind)
|
||||
{
|
||||
return (kind.equals("OLLAMA") ? "LOCAL" : "CLOUD");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Provider contract.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public AssistantModelProvider create(String key, String url)
|
||||
{
|
||||
creates++;
|
||||
return (provider);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Provider contract.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public List<DiscoveredModel> discoverModels(ProviderSettings settings) throws ModelProviderException
|
||||
{
|
||||
discoveries++;
|
||||
if(discoveryError)
|
||||
{
|
||||
throw (new ModelProviderException(ModelProviderException.Kind.NETWORK, null, "Discovery unreachable"));
|
||||
}
|
||||
List<DiscoveredModel> models = new ArrayList<>();
|
||||
for(int i = 0; i < modelCount; i++)
|
||||
{
|
||||
models.add(new DiscoveredModel("model-" + i, "Model " + i, 8192, "test"));
|
||||
}
|
||||
return (models);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Provider contract.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public HealthResult checkHealth(ProviderSettings settings)
|
||||
{
|
||||
healthChecks++;
|
||||
return (new HealthResult(!unhealthy, 1L, unhealthy ? "Unreachable" : "Healthy"));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Configure explicit reference and URL allowlists.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
protected void customizeConfig(AssistantQBitConfig config)
|
||||
{
|
||||
config.withProviderType(type).withSecretResolver(n -> secret).withAllowedApiKeyEnvNames(Set.of("LOCAL_PROXY_KEY"))
|
||||
.withAllowedBaseUrls(Set.of("https://approved.example"));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Create settings under the admin's own session.
|
||||
*******************************************************************************/
|
||||
private AssistantProvider row() throws Exception
|
||||
{
|
||||
QContext.init(qInstance, AssistantTestFixture.sessionFor(AssistantTestFixture.ADMIN));
|
||||
return (config.getRuntime().getStore().insert(new AssistantProvider().withName("Local").withProviderType(type.getType()).withKind("OLLAMA").withEnabled(true)));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Keyless rows, kind validation, URL guards, timeouts and client fingerprints.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void settingsAndCache() throws Exception
|
||||
{
|
||||
AssistantProvider row = row();
|
||||
var service = config.getRuntime().getProviderService();
|
||||
assertThat(row.getLocation()).isEqualTo("LOCAL");
|
||||
assertThat(row.getHipaaEligible()).isFalse();
|
||||
assertThat(row.getConnectTimeoutSeconds()).isEqualTo(10);
|
||||
assertThat(row.getRequestTimeoutSeconds()).isEqualTo(120);
|
||||
assertThat(service.keyStatus(row)).isEqualTo("Not needed (local, no key)");
|
||||
assertThat(service.forRow(row, "legacy").model()).isEqualTo("legacy");
|
||||
service.forRow(row, "legacy");
|
||||
assertThat(creates).isEqualTo(1);
|
||||
row.withApiVersion("v1");
|
||||
service.forRow(row, "legacy");
|
||||
row.withRegion("r1");
|
||||
service.forRow(row, "legacy");
|
||||
row.withConnectTimeoutSeconds(11);
|
||||
service.forRow(row, "legacy");
|
||||
row.withRequestTimeoutSeconds(121);
|
||||
service.forRow(row, "legacy");
|
||||
assertThat(creates).isEqualTo(5);
|
||||
row.withApiKeyEnvName("LOCAL_PROXY_KEY");
|
||||
assertThatThrownBy(() -> service.settings(row)).hasMessageContaining("not set");
|
||||
secret = "private-test-key";
|
||||
assertThat(service.settings(row).toString()).doesNotContain(secret);
|
||||
service.forRow(row, "legacy");
|
||||
secret = "rotated-test-key";
|
||||
service.forRow(row, "legacy");
|
||||
assertThat(creates).isEqualTo(7);
|
||||
assertThat(service.validate(row.withKind("ANTHROPIC"))).anyMatch(e -> e.contains("kind"));
|
||||
row.withKind("AZURE_OPENAI").withApiKeyEnvName(null);
|
||||
assertThat(service.validate(row)).anyMatch(e -> e.contains("base URL"));
|
||||
assertThat(service.validate(row.withBaseUrl("https://unapproved.example"))).anyMatch(e -> e.contains("not allowed"));
|
||||
assertThat(service.validate(row.withBaseUrl("https://approved.example"))).isEmpty();
|
||||
assertThat(config.getEffectiveAllowedApiKeyEnvNames()).contains("OPENAI_API_KEY", "AZURE_OPENAI_API_KEY").doesNotContainNull();
|
||||
for(int invalid : List.of(0, 601))
|
||||
{
|
||||
assertThat(service.validate(row.withConnectTimeoutSeconds(invalid))).anyMatch(e -> e.contains("600"));
|
||||
row.withConnectTimeoutSeconds(10);
|
||||
assertThat(service.validate(row.withRequestTimeoutSeconds(invalid))).anyMatch(e -> e.contains("600"));
|
||||
}
|
||||
row.withRequestTimeoutSeconds(120).withApiKeyEnvName("DATABASE_PASSWORD");
|
||||
assertThat(service.validate(row)).anyMatch(e -> e.contains("not allowed"));
|
||||
row.withApiKeyEnvName("not a name");
|
||||
assertThat(service.validate(row)).anyMatch(e -> e.contains("NAME"));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Registry selection, invalidation, price precedence and call capabilities.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void modelsPricesAndCapabilities() throws Exception
|
||||
{
|
||||
AssistantProvider row = row();
|
||||
var store = config.getRuntime().getStore();
|
||||
var service = config.getRuntime().getProviderService();
|
||||
AssistantModel disabled = store.insert(new AssistantModel().withProviderId(row.getId()).withModelId("disabled").withEnabled(false));
|
||||
assertThatThrownBy(() -> service.forRow(row, "fallback")).hasMessageContaining("All models");
|
||||
AssistantModel model = store.insert(new AssistantModel().withProviderId(row.getId()).withModelId("active").withEnabled(true)
|
||||
.withSupportsTools(false).withSupportsVision(true).withSupportsStreaming(false).withSupportsReasoningEffort(true).withContextWindowTokens(4096).withMaxOutputTokens(16));
|
||||
var active = service.forRow(row, "fallback");
|
||||
assertThat(active.model()).isEqualTo("active");
|
||||
var caps = active.provider().getCapabilities("active");
|
||||
assertThat(caps.getTools()).isFalse();
|
||||
assertThat(caps.getVision()).isTrue();
|
||||
assertThat(caps.getStreaming()).isFalse();
|
||||
assertThat(caps.getReasoningEffort()).isTrue();
|
||||
assertThat(caps.getMaxContextTokens()).isEqualTo(4096);
|
||||
provider.thenText("OK");
|
||||
active.provider().stream(new ModelRequest().withModel("active").withMaxTokens(32), t -> {}).await(Duration.ofSeconds(2));
|
||||
assertThat(provider.getRequests().getLast().getMaxTokens()).isEqualTo(16);
|
||||
assertThat(provider.getRequests().getLast().getCapabilities().getStreaming()).isFalse();
|
||||
assertThat(active.provider().countTokens(new ModelRequest())).isNull();
|
||||
assertThatThrownBy(() -> service.forRow(row.withModel("disabled"), "fallback")).hasMessageContaining("disabled");
|
||||
row.withModel(null);
|
||||
assertThat(service.pricing(active, "active").getInputUsdPerMTok()).isZero();
|
||||
config.getPricing().put("active", new ModelPricing().withInputUsdPerMTok(9.0));
|
||||
assertThat(service.pricing(active, "active").getInputUsdPerMTok()).isEqualTo(9.0);
|
||||
store.insert(new AssistantModelPrice().withProviderType(type.getType()).withModel("active").withInputUsdPerMTok(new BigDecimal("5")).withOutputUsdPerMTok(new BigDecimal("6")));
|
||||
assertThat(service.pricing(active, "active").getInputUsdPerMTok()).isEqualTo(5.0);
|
||||
store.update(model.withInputUsdPerMTok(new BigDecimal("2")).withOutputUsdPerMTok(new BigDecimal("3")));
|
||||
assertThat(service.pricing(active, "active").getInputUsdPerMTok()).isEqualTo(2.0);
|
||||
store.update(row.withModel("active"));
|
||||
assertThat(service.forRow(store.getProvider(row.getId()), "fallback").model()).isEqualTo("active");
|
||||
row.withModel(null);
|
||||
store.update(disabled.withEnabled(true));
|
||||
assertThat(service.forRow(row, "fallback").model()).isEqualTo("disabled");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Discovery preserves edits, disables large cloud catalogs and records errors.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void discovery() throws Exception
|
||||
{
|
||||
AssistantProvider row = row();
|
||||
var runtime = config.getRuntime();
|
||||
var transform = new DiscoverModelsProcess.Transform();
|
||||
var input = new RunBackendStepInput();
|
||||
input.setRecords(List.of(row.toQRecord()));
|
||||
var output = new RunBackendStepOutput();
|
||||
transform.runOnePage(input, output);
|
||||
assertThat(discoveries).isZero();
|
||||
assertThat(creates).isZero();
|
||||
assertThat(transform.getProcessSummary(output, false)).isNotEmpty();
|
||||
runProcess(DiscoverModelsProcess.NAME, row.getId());
|
||||
assertThat(discoveries).isEqualTo(1);
|
||||
assertThat(runtime.getStore().listModels()).hasSize(3).allSatisfy(m ->
|
||||
{
|
||||
assertThat(m.getEnabled()).isTrue();
|
||||
assertThat(m.getSource()).isEqualTo("DISCOVERED");
|
||||
assertThat(m.getInputUsdPerMTok()).isEqualByComparingTo(BigDecimal.ZERO);
|
||||
});
|
||||
AssistantModel edited = runtime.getStore().listModels().getFirst().withDisplayName("Admin name").withEnabled(false).withInputUsdPerMTok(BigDecimal.TEN);
|
||||
runtime.getStore().update(edited);
|
||||
assertThat(ModelDiscovery.run(runtime, row).message()).isEqualTo("0 models added, 3 already present");
|
||||
assertThat(runtime.getStore().listModels().getFirst().getDisplayName()).isEqualTo("Admin name");
|
||||
modelCount = 23;
|
||||
secret = "test-key";
|
||||
row = runtime.getStore().insert(new AssistantProvider().withName("Cloud").withProviderType(type.getType()).withKind("OPENAI"));
|
||||
assertThat(ModelDiscovery.run(runtime, row).ok()).isTrue();
|
||||
assertThat(runtime.getProviderService().models(row.getId())).hasSize(23).allMatch(m -> !m.getEnabled());
|
||||
discoveryError = true;
|
||||
assertThat(ModelDiscovery.run(runtime, row).ok()).isFalse();
|
||||
assertThat(runtime.getStore().getProvider(row.getId()).getLastDiscoveryError()).contains("Discovery unreachable");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Model tests use the requested id, account usage and persist health failures.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void modelTestAndMetadata() throws Exception
|
||||
{
|
||||
AssistantProvider row = row();
|
||||
var runtime = config.getRuntime();
|
||||
AssistantModel model = runtime.getStore().insert(new AssistantModel().withProviderId(row.getId()).withModelId("disabled-test").withEnabled(false));
|
||||
provider.thenText("OK");
|
||||
runProcess(TestModelConnectionProcess.NAME, model.getId());
|
||||
assertThat(healthChecks).isEqualTo(1);
|
||||
assertThat(provider.getRequests().getLast().getModel()).isEqualTo("disabled-test");
|
||||
assertThat(provider.getRequests().getLast().getMaxTokens()).isEqualTo(32);
|
||||
assertThat(runtime.getStore().listModels().getFirst().getLastTestStatus()).isEqualTo("OK");
|
||||
assertThat(runtime.getStore().queryRecords(AssistantUsage.TABLE_NAME, new QQueryFilter(), 10)).hasSize(1);
|
||||
unhealthy = true;
|
||||
assertThat(ConnectionTest.runModel(runtime, model, AssistantTestFixture.ADMIN).ok()).isFalse();
|
||||
assertThat(runtime.getStore().listModels().getFirst().getLastTestError()).contains("Unreachable");
|
||||
assertThat(provider.getRequests()).hasSize(1);
|
||||
assertThat(AssistantMetaDataBuilder.NAV_TABLES).doesNotContain(AssistantModel.TABLE_NAME);
|
||||
assertThat(qInstance.getTable(AssistantProvider.TABLE_NAME).getSections()).anyMatch(s -> s.getWidgetName() != null && s.getWidgetName().contains("model"));
|
||||
assertThat(qInstance.getTable(AssistantModel.TABLE_NAME).getField("providerId").getPossibleValueSourceName()).isEqualTo(AssistantProvider.TABLE_NAME);
|
||||
assertThat(qInstance.getTable(AssistantModel.TABLE_NAME).getField("lastTestError").getIsEditable()).isFalse();
|
||||
assertThat(qInstance.getTable(AssistantProvider.TABLE_NAME).getField("lastDiscoveryAt").getIsEditable()).isFalse();
|
||||
assertThat(AssistantHomeData.load(runtime).getProviders().getFirst().enabledModels()).isZero();
|
||||
assertThat(AssistantHomeData.load(runtime).getProviders().getFirst().keyPresent()).isTrue();
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Standard streamed ETL with Review and Result enabled.
|
||||
*******************************************************************************/
|
||||
private void runProcess(String process, Integer id) throws Exception
|
||||
{
|
||||
RunProcessInput input = new RunProcessInput();
|
||||
input.setProcessName(process);
|
||||
input.setCallback(QProcessCallbackFactory.forPrimaryKey("id", id));
|
||||
input.setFrontendStepBehavior(RunProcessInput.FrontendStepBehavior.SKIP);
|
||||
new RunProcessAction().execute(input);
|
||||
}
|
||||
}
|
||||