Assistant app home, provider settings, Test connection, per-call usage and prices #12
No files matched your search
@@ -31,14 +31,16 @@ Always use the wrapper: Maven 3.10 cannot read the parent POM. The committed def
|
||||
|
||||
- `qbit-assistant` (`com.kingsrook.qbits.assistant`):
|
||||
- `AssistantQBitProducer`, `AssistantQBitConfig` (with `AssistantModelRouting` and `AssistantBudgets`), `AssistantRuntime`, `AssistantSecurity`.
|
||||
- `metadata/`: tables, owner key type, the Assistant Admin app, the retention process, and the `assistant.admin` custom permission checker.
|
||||
- `metadata/`: tables, owner key type, the Assistant app (home + main tables in the nav), the retention process, the settings customizer (key-reference checks, masked key status), and the `assistant.admin` custom permission checker.
|
||||
- `home/`: the app home widget (`AssistantHomeData` loads, `AssistantHomeView` renders flow-layout HTML).
|
||||
- `model/`: entities for the `assistant*` tables.
|
||||
- `persistence/AssistantStore`: the **only** code that calls QQQ data actions, and only on the QBit's own tables, under the user's session.
|
||||
- `engine/`: `TurnEngine` (the loop), `ContextBuilder` (cache-prefix order), `SystemPrompt`, `PolicyService` (policy row, budgets), `RateLimiter`, the `TurnEvent` / `TurnEventLog` stream, and `MessageParts`.
|
||||
- `api/`: `AssistantApi` (transport-free) and `AssistantRouteProvider` (Javalin, SSE).
|
||||
- `provider/`: the provider-neutral SPI, plus `fake/ScriptedModelProvider` for tests, evals and demos.
|
||||
- `developer/`: the record Developer-screen panel, on the core developer-panel contract.
|
||||
- `processes/AssistantRetentionStep`.
|
||||
- `processes/`: `AssistantRetentionStep` and `TestConnectionProcess` (record action on `assistant_provider`).
|
||||
- `engine/ProviderService` (active provider from `assistant_provider` rows, key references, prices), `UsageRecorder` (one `assistant_usage` row per model call), `ConnectionTest`.
|
||||
- `qbit-assistant-provider-anthropic`: `AnthropicModelProvider` and `AnthropicMapping` (anthropic-java streaming).
|
||||
|
||||
## Invariants (do not break)
|
||||
@@ -56,6 +58,7 @@ Always use the wrapper: Maven 3.10 cannot read the parent POM. The committed def
|
||||
- Parallel tool results go back in ONE user message, in call order.
|
||||
- Errors (DENIED, NOT_FOUND, INVALID_INPUT, ERROR) are `is_error` results.
|
||||
- Unknown tools and over-limit calls also get error results.
|
||||
- **Keys:** never stored in a table. `assistant_provider.apiKeyEnvName` holds only an allowed environment variable NAME; `apiKeyStatus` is computed on read (last 4 at most) and stripped on save.
|
||||
- **Logging:** never log message content, prompts, tool results or keys. Structured end-of-turn log only.
|
||||
- **Thinking:** thinking is never streamed or stored. The Anthropic adapter keeps the raw assistant message (with signatures) only in memory, as `providerState`, for the tool loop.
|
||||
- **Links:** only `/app` or `/app/...` hrefs are emitted (`TurnEngine.isAppHref`).
|
||||
|
||||
@@ -8,6 +8,20 @@ Phase 0/1 of the QQQ Assistant: read-only Q&A and navigation. Requires QQQ 4.1 w
|
||||
- the AI tool kernel (`qqq-backend-core` `actions.ai`, QRun-IO/qqq#1078)
|
||||
- the developer-panel contract (QRun-IO/qqq#1076).
|
||||
|
||||
### Added (0.1.0-RC.4)
|
||||
- **Provider settings** (`assistant_provider`): type (registered `AssistantProviderType`s; `AnthropicProviderType` in the Anthropic module), model, enabled, default, base URL override, monthly cost cap, max output tokens per request, max concurrent requests, and the last connection test. The default enabled row is the active provider; with no rows the provider registered in code is used, so existing hosts keep working.
|
||||
- **Key handling:** the key is never stored. A row names the environment variable that holds it (`apiKeyEnvName`, allow-listed); `apiKeyStatus` is computed on read (`Set • …abcd`) and dropped on save. A pasted key, an unrelated variable or an unlisted base URL is refused. New config: `withProviderType`, `allowedApiKeyEnvNames`, `allowedBaseUrls`, `withSecretResolver`, `providerTestTimeoutSeconds`.
|
||||
- **Test connection**, a record action on providers (standard streamed ETL Review and Result lines): one short live call, result saved on the row and recorded as a `TEST` usage row.
|
||||
- **Per-call usage** (`assistant_usage`): one row per model call (turn steps and tests) with provider, model, status, error kind, HTTP status, latency, tokens and cost. Child list on conversations and providers.
|
||||
- **Model prices** (`assistant_model_price`), editable, used for every call's cost; the reference DDL seeds the Anthropic list prices of 2026-10. The built-in `pricing` defaults now cover every current Claude model.
|
||||
- **The app home** (`assistantHome` widget): health, providers, usage by day, costs against caps, by user, top tools and recent failures, from real rows with empty states.
|
||||
- `ModelResponse.model`: the model that answered (set by the Anthropic adapter).
|
||||
|
||||
### Changed (0.1.0-RC.4)
|
||||
- The app is labelled **Assistant** (name `assistantAdmin` unchanged). Its nav lists the home and the main tables only (Providers, Conversations, Model prices, Policy); other tables are child lists. `assistantRetention` is a table action on conversations instead of a nav entry.
|
||||
- Every `assistant_*` table has AI exposure `NONE`, so the assistant never reads its own settings or audit tables (before, only the `assistant.admin` permission kept them out).
|
||||
- `AssistantMetaDataBuilder.buildAdminApp(tables)` no longer takes the retention process.
|
||||
|
||||
### Changed (0.1.0-RC.3)
|
||||
- **Snake_case names** (QRun-IO naming rule, 2026-10-12). Metadata only; the physical tables were already `assistant_*`:
|
||||
- tables `assistantConversation` ... `assistantUsageDaily` are now `assistant_conversation`, `assistant_message`, `assistant_turn`, `assistant_turn_event`, `assistant_tool_call`, `assistant_feedback`, `assistant_policy` and `assistant_usage_daily`; the conversation possible-value source follows the table name
|
||||
|
||||
@@ -31,14 +31,16 @@ Always use the wrapper: Maven 3.10 cannot read the parent POM. The committed def
|
||||
|
||||
- `qbit-assistant` (`com.kingsrook.qbits.assistant`):
|
||||
- `AssistantQBitProducer`, `AssistantQBitConfig` (with `AssistantModelRouting` and `AssistantBudgets`), `AssistantRuntime`, `AssistantSecurity`.
|
||||
- `metadata/`: tables, owner key type, the Assistant Admin app, the retention process, and the `assistant.admin` custom permission checker.
|
||||
- `metadata/`: tables, owner key type, the Assistant app (home + main tables in the nav), the retention process, the settings customizer (key-reference checks, masked key status), and the `assistant.admin` custom permission checker.
|
||||
- `home/`: the app home widget (`AssistantHomeData` loads, `AssistantHomeView` renders flow-layout HTML).
|
||||
- `model/`: entities for the `assistant_*` tables.
|
||||
- `persistence/AssistantStore`: the **only** code that calls QQQ data actions, and only on the QBit's own tables, under the user's session.
|
||||
- `engine/`: `TurnEngine` (the loop), `ContextBuilder` (cache-prefix order), `SystemPrompt`, `PolicyService` (policy row, budgets), `RateLimiter`, the `TurnEvent` / `TurnEventLog` stream, and `MessageParts`.
|
||||
- `api/`: `AssistantApi` (transport-free) and `AssistantRouteProvider` (Javalin, SSE).
|
||||
- `provider/`: the provider-neutral SPI, plus `fake/ScriptedModelProvider` for tests, evals and demos.
|
||||
- `developer/`: the record Developer-screen panel, on the core developer-panel contract.
|
||||
- `processes/AssistantRetentionStep`.
|
||||
- `processes/`: `AssistantRetentionStep` and `TestConnectionProcess` (record action on `assistant_provider`).
|
||||
- `engine/ProviderService` (active provider from `assistant_provider` rows, key references, prices), `UsageRecorder` (one `assistant_usage` row per model call), `ConnectionTest`.
|
||||
- `qbit-assistant-provider-anthropic`: `AnthropicModelProvider` and `AnthropicMapping` (anthropic-java streaming).
|
||||
|
||||
## Invariants (do not break)
|
||||
@@ -56,6 +58,7 @@ Always use the wrapper: Maven 3.10 cannot read the parent POM. The committed def
|
||||
- Parallel tool results go back in ONE user message, in call order.
|
||||
- Errors (DENIED, NOT_FOUND, INVALID_INPUT, ERROR) are `is_error` results.
|
||||
- Unknown tools and over-limit calls also get error results.
|
||||
- **Keys:** never stored in a table. `assistant_provider.apiKeyEnvName` holds only an allowed environment variable NAME; `apiKeyStatus` is computed on read (last 4 at most) and stripped on save.
|
||||
- **Logging:** never log message content, prompts, tool results or keys. Structured end-of-turn log only.
|
||||
- **Thinking:** thinking is never streamed or stored. The Anthropic adapter keeps the raw assistant message (with signatures) only in memory, as `providerState`, for the tool loop.
|
||||
- **Links:** only `/app` or `/app/...` hrefs are emitted (`TurnEngine.isAppHref`).
|
||||
|
||||
@@ -37,14 +37,15 @@ The data access itself is done by the AI tool kernel in QQQ core (`com.kingsrook
|
||||
AssistantQBitConfig config = new AssistantQBitConfig()
|
||||
.withBackendName("rdbms") // where the assistant_* tables live
|
||||
.withTableMetaDataCustomizer(snakeCaseCustomizer) // optional: backend names for your schema
|
||||
.withModelProvider(AnthropicModelProvider.fromEnvironment()); // key from ANTHROPIC_API_KEY
|
||||
.withModelProvider(AnthropicModelProvider.fromEnvironment()) // fallback while no provider row exists
|
||||
.withProviderType(new AnthropicProviderType()); // offered on the Providers settings screen
|
||||
|
||||
new AssistantQBitProducer().withQBitConfig(config).produce(qInstance);
|
||||
```
|
||||
|
||||
`produce` does the following:
|
||||
- Registers the QBit.
|
||||
- Adds the tables, the `assistant_user_id` security key type, the `assistantRetention` process and the **Assistant Admin** app.
|
||||
- Adds the tables, the `assistant_user_id` security key type, the `assistantRetention` and **Test connection** processes, and the **Assistant** app with its home.
|
||||
- Builds the runtime.
|
||||
- Registers the `/qqq/v1/assistant/*` routes when Javalin is on the classpath.
|
||||
- Registers the record Developer-screen panel when QQQ core has the developer-panel contract.
|
||||
@@ -53,12 +54,48 @@ Create the tables in your database: see [Tables](#tables). Then grant the permis
|
||||
|
||||
### The API key
|
||||
|
||||
The key never goes in config, tables or logs. There are two ways to supply it:
|
||||
- `AnthropicModelProvider.fromEnvironment()` reads `ANTHROPIC_API_KEY`.
|
||||
- `AnthropicModelProvider.builder().apiKey(valueFromYourSecretStore).build()` takes the key from your own secret store. It also accepts the reference form `"${env.NAME}"`.
|
||||
The key never goes in config, tables or logs. It lives in the application's secret store and reaches the server as an environment variable (for kof22: Infisical, then an ExternalSecret, then the pod env).
|
||||
|
||||
- **Provider settings rows** (recommended) name the variable: `assistant_provider.apiKeyEnvName`, default the type's own (`ANTHROPIC_API_KEY`). The server reads the variable when it builds the client.
|
||||
- Without a row, the provider registered in code is used: `AnthropicModelProvider.fromEnvironment()` reads `ANTHROPIC_API_KEY`, and `AnthropicModelProvider.builder().apiKey(valueFromYourSecretStore).build()` takes the key from your own secret store (it also accepts `"${env.NAME}"`).
|
||||
|
||||
The provider's `toString` never contains the key. Its errors carry only the HTTP status and the API error type.
|
||||
|
||||
## Provider settings (the Providers table)
|
||||
|
||||
Admins set up providers on the **Providers** screen (`assistant_provider`), a standard QQQ table:
|
||||
|
||||
| Field | Meaning |
|
||||
|---|---|
|
||||
| Name, Type, Model | The type is one of the registered provider types (`withProviderType`; Anthropic now, others plug in the same way). The model list comes from the type and the price table. An empty model uses the policy's main model. |
|
||||
| Enabled, Default | The assistant uses the default enabled provider, else the first enabled one. Marking one default unmarks the others. With no enabled row, the provider registered in code is used. |
|
||||
| API Key Variable | The **name** of the environment variable that holds the key, never the key. Only allowed names are accepted: each registered type's default plus `allowedApiKeyEnvNames`. A pasted key (or `DATABASE_PASSWORD`) is refused and nothing is stored. |
|
||||
| API Key | Read-only, computed on every read: `Set • …abcd (from ANTHROPIC_API_KEY)` or `Not set (...)`. It shows the last four characters at most. No read path (screen, API, query, export) returns the key. |
|
||||
| Base URL Override | Empty uses the provider's own API. Any other URL must be listed in `allowedBaseUrls`, so a row cannot send the key to a host of an admin's choosing. |
|
||||
| Monthly Cost Cap (USD) | When this provider's cost this month (UTC) reaches the cap, new requests are refused (429 `BUDGET`) and a running answer stops before its next model call. |
|
||||
| Max Output Tokens Per Request, Max Concurrent Requests | Caps one model call's output, and how many calls run at once on this server. |
|
||||
| Last test | Status, time, latency, the model that answered, and the error text, from **Test connection**. |
|
||||
|
||||
**Why a reference and not an encrypted column.** QQQ core has no encrypted or secret field type. The key-provider and cipher work in the QQQ 4.1.0-RC.6 batch (QRun-IO/qqq#1123, #1107) encrypts backup artifacts, not table columns, and is not released. The design spec (§2.8) also says keys never go in tables. A reference keeps the key in the secret store that already holds it, so rotating it there is the "replace key" step, and the database never holds one.
|
||||
|
||||
**Test connection** is a record action on a provider (select one or more). It is a standard streamed ETL: Review lists what will be called and anything wrong with the row (missing key, disallowed variable or URL) before any call. Result has one line per provider: `connected to claude-sonnet-5-5 in 812 ms`, or the error with its kind and HTTP status. It makes one short live call (a 1-sentence prompt, at most 32 output tokens, effort low), saves the result on the row, and records the call in `assistant_usage` with purpose `TEST`.
|
||||
|
||||
## The app home
|
||||
|
||||
The **Assistant** app's home is the one place for the assistant's health, usage and cost (no other dashboard repeats it). All of it comes from the tables below, read as the viewing admin; an empty table gives an empty state, never sample data.
|
||||
|
||||
- **Health:** is it answering (kill switch, default provider, key present, last test), the 7-day error rate of model calls, cost and requests this month against the instance's monthly limit.
|
||||
- **Providers:** one row per provider: type, model, masked key status, last test, cost this month against its cap, limits, and any problem with its settings.
|
||||
- **Usage, last 30 days:** bars of model requests per day, totals (conversations, requests, tokens in and out, cost) and the last 7 days in words.
|
||||
- **Costs this month:** month to date against the policy limit and each provider cap, then by model. Cost is tokens times the model's price from **Model prices**.
|
||||
- **By user, this month**, **Top tools** (calls, failures, average time) and **Recent failures** (model calls and tool calls, linked).
|
||||
|
||||
The nav lists the home and the main tables: **Providers**, **Conversations**, **Model prices** and **Policy**. Messages, turns, tool calls, feedback and model calls are child lists on a conversation; model calls are also listed on their provider. Retention and Test connection are table and record actions, never nav entries.
|
||||
|
||||
## Model prices
|
||||
|
||||
`assistant_model_price` holds USD per million tokens per provider type and model: input, output, cache read, and cache writes (5 minute and 1 hour). Empty cache prices default to 0.1x, 1.25x and 2x the input price. The reference DDL seeds the Anthropic list prices of 2026-10. Edit a row to change what later calls cost; past rows keep the cost they were recorded with. A model with no row uses the `pricing` config map, else costs 0.
|
||||
|
||||
## Configuration
|
||||
|
||||
| Setting | Default | Meaning |
|
||||
@@ -82,6 +119,13 @@ The provider's `toString` never contains the key. Its errors carry only the HTTP
|
||||
| `hiddenWidgetNames` | none | Widgets that are never rendered, even when allowed. |
|
||||
| `customRedactionPatterns` | none | Extra regular expressions the kernel redacts from tool results. |
|
||||
| `killSwitchPollMillis` | 2000 | How often a running turn re-checks the kill switch while it waits on the model. |
|
||||
| `providerTypes` | none | `withProviderType(...)`: the types the Providers screen offers (`AnthropicProviderType`). |
|
||||
| `allowedApiKeyEnvNames` | none | Extra environment variable names a provider row may name as its key. Each type's default is always allowed. |
|
||||
| `allowedBaseUrls` | none | Base URL overrides a provider row may use. The type's own URL is always allowed. |
|
||||
| `secretResolver` | `System::getenv` | How a key variable name becomes the key (a host secret store, or tests). |
|
||||
| `providerTestTimeoutSeconds` | 30 | How long Test connection waits for the model. |
|
||||
| `homeMaxRows` | 20000 | Rows per table the app home reads for its period; it says when totals are partial. |
|
||||
| `tablePathPrefix` | `/app` | The front of the links the app home makes to tables and records. |
|
||||
|
||||
At runtime, an admin can override the kill switch, the main model and effort, and the budgets in the single **`assistant_policy`** row.
|
||||
|
||||
@@ -96,7 +140,7 @@ When a turn sees the switch on, it cancels every turn running on that replica. I
|
||||
| Permission | Grants |
|
||||
|---|---|
|
||||
| `assistant.hasAccess` | Use the panel. It grants **no data access**: the assistant reads only what the user can read anyway. |
|
||||
| `assistant.admin` | The Assistant Admin app, its tables and the retention process. |
|
||||
| `assistant.admin` | The Assistant app (home, settings and audit tables), Test connection and the retention process. |
|
||||
|
||||
There is no assistant-specific data permission. Everything the kernel reads is checked against the user's normal table permissions and record locks. The QBit never uses a system session (an ArchUnit test enforces this).
|
||||
|
||||
@@ -106,7 +150,7 @@ When the QBit is enabled, it registers the supplemental instance meta-data `assi
|
||||
|
||||
Conversation data belongs to its user. Every user-owned table has a record lock on `userId` with key type `assistant_user_id`:
|
||||
- The panel routes add the user's own id to their session, so users only ever see their own rows.
|
||||
- Admins see everyone's rows in the Assistant Admin app through the key type's all-access key, `assistant_user_id_all_access`. Grant that key to `assistant.admin` holders from your authentication customizer:
|
||||
- Admins see everyone's rows in the Assistant app through the key type's all-access key, `assistant_user_id_all_access`. Grant that key to `assistant.admin` holders from your authentication customizer:
|
||||
|
||||
```java
|
||||
// in your QAuthenticationModuleCustomizerInterface.customizeSession(...)
|
||||
@@ -174,7 +218,7 @@ The stream ends after `message.end`. Thinking is never streamed.
|
||||
|
||||
## Tables
|
||||
|
||||
All of these are standard, visible QQQ tables in the Assistant Admin app:
|
||||
All of these are standard, visible QQQ tables in the Assistant app. The model never reads any of them (AI exposure `NONE`).
|
||||
|
||||
| Table | Holds |
|
||||
|---|---|
|
||||
@@ -186,6 +230,9 @@ All of these are standard, visible QQQ tables in the Assistant Admin app:
|
||||
| `assistant_feedback` | Feedback on answers. |
|
||||
| `assistant_policy` | The kill switch and overrides. |
|
||||
| `assistant_usage_daily` | Usage rollups: per user per day, plus an instance row whose `userId` is null. |
|
||||
| `assistant_provider` | Provider settings: type, model, key variable (never the key), limits, last connection test. |
|
||||
| `assistant_model_price` | USD per million tokens per model, used for every call's cost. |
|
||||
| `assistant_usage` | One row per model call (a turn step or a connection test): user, conversation, turn, provider, model, status, latency, tokens, cost. Locked to its user like the other audit tables. |
|
||||
|
||||
Table names follow the QRun-IO naming rule: lowercase snake_case with the `assistant_` prefix, the same as the physical tables in `docs/schema-postgres.sql` (the reference DDL). Field names are camelCase. Map them to your columns with a table customizer. Upgrading from 0.1.0-RC.1, which used camelCase table names and the `assistantUserId` key type: see [UPGRADE.md](UPGRADE.md).
|
||||
|
||||
|
||||
+14
@@ -1,5 +1,19 @@
|
||||
# Upgrading qbit-assistant
|
||||
|
||||
## 0.1.0-RC.3 to 0.1.0-RC.4: provider settings, prices, per-call usage, app home
|
||||
|
||||
Three new tables. Create them before you deploy (see `docs/schema-postgres.sql`, which also seeds the Anthropic prices):
|
||||
- `assistant_provider` (with the `api_key_status` column: it is filled on read and never written, but the backend selects it)
|
||||
- `assistant_model_price`
|
||||
- `assistant_usage`
|
||||
|
||||
Then, to keep answering exactly as before:
|
||||
1. Register the provider type: `config.withProviderType(new AnthropicProviderType())`. Keep `withModelProvider(AnthropicModelProvider.fromEnvironment())` as the fallback.
|
||||
2. Add one provider row (or seed it in your migration): type `anthropic`, model = your current main model, enabled, default, `apiKeyEnvName` = `ANTHROPIC_API_KEY` (or leave it empty for the default). No key value is ever written.
|
||||
3. Run **Test connection** on it.
|
||||
|
||||
Until a row exists, the provider registered in code answers, as in RC.3. Other changes: the app label is **Assistant**; the nav no longer lists the retention process (it is a table action on conversations) or the audit tables (they are child lists). Grants on `assistant.admin` cover everything new.
|
||||
|
||||
## 0.1.0-RC.1 to 0.1.0-RC.3: snake_case names
|
||||
|
||||
The rename shipped in 0.1.0-RC.3. 0.1.0-RC.2 on Maven Central is an accidental republish of the RC.1 code (see CHANGELOG.md), so upgrading from RC.2 is the same as upgrading from RC.1.
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
# qbit-assistant: settings, usage and app home (design)
|
||||
|
||||
The assistant's overall design is the QQQ spec `QRun-IO/qqq` `docs/specs/2026-10-11-qqq-assistant-qbit-design.md` (owner decisions D1 to D7). This note covers what 0.1.0-RC.4 adds on top of it: provider settings, per-call usage, prices and the app home.
|
||||
|
||||
Owner request (2026-10-13): *"it needs a good homepage for the app for it, and it needs a settings screen where you setup the API keys, provider, test connectivity etc. and the app homepage should show health, usage, costs, etc."*
|
||||
|
||||
## Data
|
||||
|
||||
All of it is in standard QQQ tables with full record screens (D6), prefixed `assistant_`, secured by `assistant.admin` and, for user-owned rows, the `assistant_user_id` record lock. Every `assistant_*` table has AI exposure `NONE`.
|
||||
|
||||
| Table | Written by | Purpose |
|
||||
|---|---|---|
|
||||
| `assistant_provider` | admins | Which provider and model answer, the key reference, limits, the last connection test. |
|
||||
| `assistant_model_price` | admins (seeded) | USD per million tokens per model. |
|
||||
| `assistant_usage` | the qbit only | One row per model call: turn steps (`TURN`) and connection tests (`TEST`). |
|
||||
|
||||
`assistant_turn` keeps the per-turn totals and `assistant_usage_daily` the budget rollups, as before. `assistant_usage` adds the per-call detail the home needs: the provider, the failures and latency.
|
||||
|
||||
## API keys
|
||||
|
||||
Decision: **reference, not storage.** A provider row holds the NAME of an environment variable (`apiKeyEnvName`); `ProviderService` resolves it through `config.getSecretResolver()` (default `System::getenv`) when it builds a client.
|
||||
|
||||
Why:
|
||||
- QQQ core has no encrypted or secret field type. The key-provider SPI and Tink cipher in the RC.6 batch (qqq#1123, #1107) encrypt streamed backup artifacts, not columns, and are not released.
|
||||
- Spec §2.8: keys are `${env.*}` secrets and are never stored in tables.
|
||||
- Deployments already hold the key in a secret store (kof22: Infisical, then an ExternalSecret, then the pod env). Rotating it there is the "replace key" step; the database never has a copy to leak through a backup, an export or the API.
|
||||
|
||||
Guards:
|
||||
- Only allow-listed names: each registered type's default (`ANTHROPIC_API_KEY`) plus `allowedApiKeyEnvNames`. Otherwise an admin could aim a row at `DATABASE_PASSWORD` and send it to a provider.
|
||||
- A base URL override must be in `allowedBaseUrls`. Otherwise an admin could send the real key to their own host.
|
||||
- The variable name must look like one (`[A-Z_][A-Z0-9_]*`), so a key pasted into the field is refused before it is stored. The error never echoes the value.
|
||||
- `apiKeyStatus` is a real, read-only column filled by the post-query customizer: `Set • …abcd (from VAR)` (no characters for keys under 12) or `Not set`. Saves drop it, so it is always null in the database. (A virtual field cannot sit in a base section in QQQ 4.1, hence the column.)
|
||||
- Clients are cached by row settings and a SHA-256 of the key. Keys are never logged, and neither `toString` nor the errors contain one.
|
||||
|
||||
## Choosing the provider
|
||||
|
||||
`ProviderService.resolve(policyModel)`:
|
||||
1. The enabled row marked default (marking one default unmarks the others), else the first enabled row.
|
||||
2. With no enabled row, the provider registered in code (`withModelProvider`) and the policy's main model: hosts that never add a row keep working.
|
||||
3. A row that cannot be used (no key, a disallowed name or URL, an uninstalled type) makes the assistant report `NOT_CONFIGURED` rather than fall back silently.
|
||||
|
||||
The model is the row's, else the policy's main model. The rows and prices are cached for `policyCacheSeconds` and dropped on any change.
|
||||
|
||||
## Limits
|
||||
|
||||
- **Monthly cost cap (USD):** checked when a turn starts (429 `BUDGET`) and before each model call of a running turn, from the provider's `assistant_usage` cost since the 1st (UTC).
|
||||
- **Max output tokens per request:** lowers the request's `max_tokens`.
|
||||
- **Max concurrent requests:** a per-provider semaphore around each model call on this replica.
|
||||
|
||||
These sit next to the existing policy budgets (per user per day, per instance per month), which still apply.
|
||||
|
||||
## Test connection
|
||||
|
||||
A record action on `assistant_provider`, as a standard streamed ETL with Review and Result summary lines and no custom form.
|
||||
- **Review** lists the providers, the masked key status and every settings problem, without calling anything.
|
||||
- **Result** makes one call per provider: the prompt "Connection test. Reply with the single word OK.", at most 32 output tokens, effort low. It reports `connected to <model> in <n> ms`, or a hint, the kind, the HTTP status and the provider's own credential-free message.
|
||||
|
||||
The result goes on the row (`lastTest*`) and into `assistant_usage` (`TEST`), so a test's cost is counted like any other call.
|
||||
|
||||
## Cost
|
||||
|
||||
`cost = input × in + output × out + cacheRead × read + cacheWrite5m × w5 + cacheWrite1h × w1`, in micro-dollars, rounded up (`ModelPricing.costMicros`).
|
||||
|
||||
Prices come from the `assistant_model_price` row for (type, model). Empty cache prices are 0.1x, 1.25x and 2x the input price. A model without a row uses the `pricing` config map, else costs 0. A price change affects later calls only; each usage row keeps the cost it was recorded with.
|
||||
|
||||
## App home
|
||||
|
||||
One HTML widget (`assistantHome`) on the Assistant app. It is the only place this data is summarized: no other dashboard repeats it, and the developer panel stays per-record.
|
||||
|
||||
`AssistantHomeData` reads the tables as the viewing admin, at most `homeMaxRows` rows each, for the last 30 days (UTC) and the current month, and says when totals are partial. `AssistantHomeView` renders it in flow and grid layout only:
|
||||
- no tables, fixed widths or inner scroll areas
|
||||
- inherited text colors, so light and dark both work
|
||||
- a status row, one row per provider, then cards for usage by day, costs, users, tools and failures.
|
||||
|
||||
Every section has an empty state; nothing is invented.
|
||||
|
||||
The nav is the home plus the main tables: Providers, Conversations, Model prices, Policy. Messages, turns, tool calls, feedback and model calls are child lists on a conversation, and model calls also on a provider. Processes are actions on their tables.
|
||||
@@ -155,3 +155,87 @@ CREATE TABLE assistant_usage_daily
|
||||
);
|
||||
CREATE INDEX assistant_usage_daily_user_id ON assistant_usage_daily (user_id);
|
||||
CREATE INDEX assistant_usage_daily_date ON assistant_usage_daily (usage_date, scope);
|
||||
|
||||
-- 0.1.0-RC.4: provider settings, model prices and per-call usage.
|
||||
-- api_key_status is filled when a row is read (never written); the API key itself
|
||||
-- is never stored: api_key_env_name holds only the NAME of an environment variable.
|
||||
|
||||
CREATE TABLE assistant_provider
|
||||
(
|
||||
id SERIAL PRIMARY KEY,
|
||||
name VARCHAR(100) NOT NULL UNIQUE,
|
||||
provider_type VARCHAR(50) NOT NULL,
|
||||
model VARCHAR(100),
|
||||
enabled BOOLEAN,
|
||||
is_default BOOLEAN,
|
||||
base_url VARCHAR(255),
|
||||
api_key_env_name VARCHAR(100),
|
||||
api_key_status VARCHAR(200),
|
||||
monthly_cost_cap_usd NUMERIC(12, 2),
|
||||
max_output_tokens_per_request INTEGER,
|
||||
max_concurrent_requests INTEGER,
|
||||
last_test_at TIMESTAMP WITH TIME ZONE,
|
||||
last_test_status VARCHAR(16),
|
||||
last_test_latency_ms INTEGER,
|
||||
last_test_model VARCHAR(100),
|
||||
last_test_error VARCHAR(500),
|
||||
create_date TIMESTAMP WITH TIME ZONE,
|
||||
modify_date TIMESTAMP WITH TIME ZONE
|
||||
);
|
||||
|
||||
CREATE TABLE assistant_model_price
|
||||
(
|
||||
id SERIAL PRIMARY KEY,
|
||||
provider_type VARCHAR(50) NOT NULL,
|
||||
model VARCHAR(100) NOT NULL,
|
||||
input_usd_per_m_tok NUMERIC(12, 4) NOT NULL,
|
||||
output_usd_per_m_tok NUMERIC(12, 4) NOT NULL,
|
||||
cache_read_usd_per_m_tok NUMERIC(12, 4),
|
||||
cache_write5m_usd_per_m_tok NUMERIC(12, 4),
|
||||
cache_write1h_usd_per_m_tok NUMERIC(12, 4),
|
||||
notes VARCHAR(255),
|
||||
create_date TIMESTAMP WITH TIME ZONE,
|
||||
modify_date TIMESTAMP WITH TIME ZONE,
|
||||
UNIQUE (provider_type, model)
|
||||
);
|
||||
|
||||
CREATE TABLE assistant_usage
|
||||
(
|
||||
id SERIAL PRIMARY KEY,
|
||||
user_id VARCHAR(255),
|
||||
conversation_id INTEGER,
|
||||
turn_id INTEGER,
|
||||
provider_id INTEGER,
|
||||
provider_type VARCHAR(50),
|
||||
model VARCHAR(100),
|
||||
purpose VARCHAR(16),
|
||||
status VARCHAR(16),
|
||||
error_kind VARCHAR(32),
|
||||
http_status INTEGER,
|
||||
latency_ms INTEGER,
|
||||
input_tokens BIGINT,
|
||||
output_tokens BIGINT,
|
||||
cache_read_tokens BIGINT,
|
||||
cache_write_tokens BIGINT,
|
||||
cost_micros BIGINT,
|
||||
request_id VARCHAR(100),
|
||||
create_date TIMESTAMP WITH TIME ZONE
|
||||
);
|
||||
CREATE INDEX assistant_usage_create_date ON assistant_usage (create_date);
|
||||
CREATE INDEX assistant_usage_provider ON assistant_usage (provider_id, create_date);
|
||||
CREATE INDEX assistant_usage_conversation ON assistant_usage (conversation_id);
|
||||
CREATE INDEX assistant_usage_user_id ON assistant_usage (user_id);
|
||||
|
||||
-- Anthropic list prices, USD per million tokens (cached 2026-10-06). Cache writes:
|
||||
-- 1.25x input (5 minute) and 2x input (1 hour). Re-check before budgeting.
|
||||
INSERT INTO assistant_model_price (provider_type, model, input_usd_per_m_tok, output_usd_per_m_tok, cache_read_usd_per_m_tok, cache_write5m_usd_per_m_tok, cache_write1h_usd_per_m_tok, notes, create_date, modify_date)
|
||||
VALUES
|
||||
('anthropic', 'claude-sonnet-5-5', 2.00, 10.00, 0.20, 2.50, 4.00, 'Anthropic list price, 2026-10', NOW(), NOW()),
|
||||
('anthropic', 'claude-opus-5-5', 4.00, 20.00, 0.20, 5.00, 8.00, 'Anthropic list price, 2026-10', NOW(), NOW()),
|
||||
('anthropic', 'claude-haiku-5-5', 0.10, 0.50, 0.01, 0.125, 0.20, 'Anthropic list price, 2026-10 (prompts up to 100K tokens)', NOW(), NOW()),
|
||||
('anthropic', 'claude-fable-5-1', 10.00, 50.00, 0.25, 12.50, 20.00, 'Anthropic list price, 2026-10', NOW(), NOW()),
|
||||
('anthropic', 'claude-opus-5', 5.00, 25.00, 0.50, 6.25, 10.00, 'Anthropic list price, 2026-10', NOW(), NOW()),
|
||||
('anthropic', 'claude-sonnet-5', 2.00, 10.00, 0.20, 2.50, 4.00, 'Anthropic list price, 2026-10', NOW(), NOW()),
|
||||
('anthropic', 'claude-opus-4-8', 5.00, 25.00, 0.50, 6.25, 10.00, 'Anthropic list price, 2026-10', NOW(), NOW()),
|
||||
('anthropic', 'claude-sonnet-4-6', 3.00, 15.00, 0.30, 3.75, 6.00, 'Anthropic list price, 2026-10', NOW(), NOW()),
|
||||
('anthropic', 'claude-haiku-4-5', 1.00, 5.00, 0.10, 1.25, 2.00, 'Anthropic list price, 2026-10', NOW(), NOW());
|
||||
@@ -0,0 +1,12 @@
|
||||
# Assistant app home and provider settings screenshots
|
||||
|
||||
Taken 2026-10-13 against the local `AssistantDemoServer` (qbit-assistant-provider-anthropic test sources), never production.
|
||||
- Memory backend and the Next UI 1.0.0-RC.16 jar.
|
||||
- The Anthropic API is the recorded test server: real HTTP through the real adapter, engine, Test connection and usage recording, with canned answers and no key.
|
||||
|
||||
Light and dark, 390 and 1440 px:
|
||||
- `home-*`: the Assistant app home.
|
||||
- `provider-*`: the provider settings screen, with the masked key and the model-call child list.
|
||||
- `provider-not-set-*`: a provider whose key variable is not set.
|
||||
- `provider-edit-*`: the edit form.
|
||||
- `prices-*`: the model price table.
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 147 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 136 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 150 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 138 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 87 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 126 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 87 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 124 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 102 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 40 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 88 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 99 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 88 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 99 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 103 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 40 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 91 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 41 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 92 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 40 KiB |
+1
@@ -264,6 +264,7 @@ public class AnthropicMapping
|
||||
.withStopReason(message.stopReason().map(s -> s.asString()).orElse(ModelResponse.STOP_END_TURN))
|
||||
.withUsage(toUsage(message.usage()))
|
||||
.withRequestId(message.id())
|
||||
.withModel(message.model().asString())
|
||||
.withProviderState(message.toParam()));
|
||||
}
|
||||
|
||||
|
||||
+113
@@ -0,0 +1,113 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.provider.anthropic;
|
||||
|
||||
|
||||
import java.util.List;
|
||||
import com.kingsrook.qbits.assistant.provider.AssistantModelProvider;
|
||||
import com.kingsrook.qbits.assistant.provider.AssistantProviderType;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The Anthropic (Claude) provider type for the assistant_provider settings
|
||||
** table. Register it with AssistantQBitConfig.withProviderType(new
|
||||
** AnthropicProviderType()). The key comes from the row's key variable
|
||||
** (default ANTHROPIC_API_KEY).
|
||||
*******************************************************************************/
|
||||
public class AnthropicProviderType implements AssistantProviderType
|
||||
{
|
||||
public static final String DEFAULT_BASE_URL = "https://api.anthropic.com";
|
||||
|
||||
/*******************************************************************************
|
||||
** The current Claude models, the default (Sonnet 5.5) first.
|
||||
*******************************************************************************/
|
||||
public static final List<String> MODELS = List.of(
|
||||
"claude-sonnet-5-5", "claude-opus-5-5", "claude-haiku-5-5", "claude-fable-5-1",
|
||||
"claude-opus-5", "claude-sonnet-5", "claude-opus-4-8", "claude-sonnet-4-6", "claude-haiku-4-5");
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getType()
|
||||
{
|
||||
return (AnthropicModelProvider.NAME);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getLabel()
|
||||
{
|
||||
return ("Anthropic (Claude)");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public List<String> getModels()
|
||||
{
|
||||
return (MODELS);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getDefaultApiKeyEnvName()
|
||||
{
|
||||
return (AnthropicModelProvider.API_KEY_ENV);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getDefaultBaseUrl()
|
||||
{
|
||||
return (DEFAULT_BASE_URL);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A provider with the resolved key (passed as a literal, never logged).
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public AssistantModelProvider create(String apiKey, String baseUrl)
|
||||
{
|
||||
AnthropicModelProvider.Builder builder = AnthropicModelProvider.builder().apiKey(apiKey);
|
||||
if(baseUrl != null)
|
||||
{
|
||||
builder.baseUrl(baseUrl);
|
||||
}
|
||||
return (builder.build());
|
||||
}
|
||||
|
||||
}
|
||||
+174
@@ -0,0 +1,174 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.provider.anthropic;
|
||||
|
||||
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import com.kingsrook.qbits.assistant.AssistantQBitConfig;
|
||||
import com.kingsrook.qbits.assistant.engine.ConnectionTest;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantProvider;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantUsage;
|
||||
import com.kingsrook.qbits.assistant.provider.fake.ScriptedModelProvider;
|
||||
import com.kingsrook.qbits.assistant.testing.AssistantTestFixture;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
|
||||
import org.junit.jupiter.api.AfterEach;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The Anthropic provider type with Test connection against a recorded HTTP
|
||||
** server (no live calls): the key from the row's variable reaches the API as
|
||||
** x-api-key, the result and the TEST usage row are saved, and a 401 becomes
|
||||
** an admin-readable error without the key.
|
||||
*******************************************************************************/
|
||||
class AnthropicProviderTypeTest
|
||||
{
|
||||
private static final String KEY = "sk-ant-test-0000000000000000-abcd";
|
||||
|
||||
private AssistantQBitConfig config;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@AfterEach
|
||||
void tearDown()
|
||||
{
|
||||
if(config != null && config.getRuntime() != null)
|
||||
{
|
||||
config.getRuntime().close();
|
||||
}
|
||||
QContext.clear();
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** An instance with the Anthropic type allowed to use the recorded server.
|
||||
*******************************************************************************/
|
||||
private QInstance instance(String baseUrl) throws Exception
|
||||
{
|
||||
config = new AssistantQBitConfig()
|
||||
.withProviderName(ScriptedModelProvider.DEFAULT_NAME)
|
||||
.withModelProvider(new ScriptedModelProvider())
|
||||
.withProviderType(new AnthropicProviderType())
|
||||
.withAllowedBaseUrls(Set.of(baseUrl))
|
||||
.withSecretResolver(Map.of(AnthropicModelProvider.API_KEY_ENV, KEY)::get);
|
||||
QInstance qInstance = AssistantTestFixture.build(config);
|
||||
QContext.init(qInstance, AssistantTestFixture.sessionFor(AssistantTestFixture.ADMIN));
|
||||
return (qInstance);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private AssistantProvider insertProvider(String baseUrl) throws Exception
|
||||
{
|
||||
QRecord record = new InsertAction().execute(new InsertInput(AssistantProvider.TABLE_NAME).withRecord(new QRecord()
|
||||
.withValue("name", "Claude").withValue("providerType", "anthropic").withValue("model", "claude-sonnet-5-5")
|
||||
.withValue("enabled", true).withValue("isDefault", true).withValue("baseUrl", baseUrl))).getRecords().get(0);
|
||||
assertThat(record.getErrors()).isEmpty();
|
||||
return (config.getRuntime().getStore().getProvider(record.getValueInteger("id")));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A good connection.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void testConnectionOk() throws Exception
|
||||
{
|
||||
try(RecordedAnthropicServer server = new RecordedAnthropicServer())
|
||||
{
|
||||
instance(server.baseUrl());
|
||||
server.thenStream("text-with-thinking.sse", 0);
|
||||
AssistantProvider row = insertProvider(server.baseUrl());
|
||||
assertThat(row.getApiKeyStatus()).isEqualTo("Set • …abcd (from ANTHROPIC_API_KEY)");
|
||||
|
||||
ConnectionTest.Result result = ConnectionTest.run(config.getRuntime(), row, AssistantTestFixture.ADMIN);
|
||||
assertThat(result.ok()).as(result.error()).isTrue();
|
||||
assertThat(result.model()).isEqualTo("claude-sonnet-5-5");
|
||||
assertThat(server.getRequests()).hasSize(1);
|
||||
assertThat(server.getRequests().get(0).get("apiKey")).isEqualTo(KEY);
|
||||
assertThat(server.getRequests().get(0).get("body")).contains("\"max_tokens\":" + ConnectionTest.PING_MAX_TOKENS);
|
||||
|
||||
AssistantProvider saved = config.getRuntime().getStore().getProvider(row.getId());
|
||||
assertThat(saved.getLastTestStatus()).isEqualTo("OK");
|
||||
assertThat(saved.getLastTestModel()).isEqualTo("claude-sonnet-5-5");
|
||||
|
||||
QRecord usage = new QueryAction().execute(new QueryInput(AssistantUsage.TABLE_NAME).withFilter(new QQueryFilter())).getRecords().get(0);
|
||||
assertThat(usage.getValueString("purpose")).isEqualTo("TEST");
|
||||
assertThat(usage.getValueString("status")).isEqualTo("OK");
|
||||
assertThat(usage.getValueLong("inputTokens")).isPositive();
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A refused key.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void testConnectionUnauthorized() throws Exception
|
||||
{
|
||||
try(RecordedAnthropicServer server = new RecordedAnthropicServer())
|
||||
{
|
||||
instance(server.baseUrl());
|
||||
server.thenError(401, "unauthorized.json");
|
||||
AssistantProvider row = insertProvider(server.baseUrl());
|
||||
|
||||
ConnectionTest.Result result = ConnectionTest.run(config.getRuntime(), row, AssistantTestFixture.ADMIN);
|
||||
assertThat(result.ok()).isFalse();
|
||||
assertThat(result.error()).contains("API key was refused").contains("HTTP 401").doesNotContain(KEY);
|
||||
|
||||
AssistantProvider saved = config.getRuntime().getStore().getProvider(row.getId());
|
||||
assertThat(saved.getLastTestStatus()).isEqualTo("ERROR");
|
||||
assertThat(saved.getLastTestError()).doesNotContain(KEY);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The type itself.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void testType()
|
||||
{
|
||||
AnthropicProviderType type = new AnthropicProviderType();
|
||||
assertThat(type.getType()).isEqualTo("anthropic");
|
||||
assertThat(type.getModels().get(0)).isEqualTo("claude-sonnet-5-5");
|
||||
assertThat(type.getDefaultApiKeyEnvName()).isEqualTo("ANTHROPIC_API_KEY");
|
||||
assertThat(type.getDefaultBaseUrl()).isEqualTo("https://api.anthropic.com");
|
||||
assertThat(type.getLabel()).contains("Anthropic");
|
||||
assertThat(type.create(KEY, null).toString()).doesNotContain(KEY);
|
||||
}
|
||||
|
||||
}
|
||||
+172
@@ -0,0 +1,172 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.provider.anthropic.demo;
|
||||
|
||||
|
||||
import java.io.Serializable;
|
||||
import java.math.BigDecimal;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import com.kingsrook.qbits.assistant.AssistantQBitConfig;
|
||||
import com.kingsrook.qbits.assistant.engine.ConnectionTest;
|
||||
import com.kingsrook.qbits.assistant.engine.ModelPricing;
|
||||
import com.kingsrook.qbits.assistant.metadata.AssistantMetaDataBuilder;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantModelPrice;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantProvider;
|
||||
import com.kingsrook.qbits.assistant.provider.anthropic.AnthropicModelProvider;
|
||||
import com.kingsrook.qbits.assistant.provider.anthropic.AnthropicProviderType;
|
||||
import com.kingsrook.qbits.assistant.provider.anthropic.RecordedAnthropicServer;
|
||||
import com.kingsrook.qbits.assistant.provider.fake.ScriptedModelProvider;
|
||||
import com.kingsrook.qbits.assistant.testing.AssistantTestFixture;
|
||||
import com.kingsrook.qbits.assistant.testing.TurnRunner;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.instances.AbstractQQQApplication;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
|
||||
import com.kingsrook.qqq.backend.core.model.session.QSession;
|
||||
import com.kingsrook.qqq.backend.core.modules.authentication.QAuthenticationModuleCustomizerInterface;
|
||||
import com.kingsrook.qqq.middleware.javalin.QApplicationJavalinServer;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Local demo of the Assistant app home and provider settings, for
|
||||
** screenshots (never against production). The Anthropic API is the
|
||||
** recorded test server, so every call is a real HTTP call through the real
|
||||
** adapter, engine, Test connection and usage recording - with canned
|
||||
** answers and no key. Everyone is an admin.
|
||||
**
|
||||
** Run with the test classpath plus the qqq-frontend-next jar; open
|
||||
** http://localhost:8099/ and the Assistant app.
|
||||
*******************************************************************************/
|
||||
public class AssistantDemoServer extends AbstractQQQApplication
|
||||
{
|
||||
private static QInstance instance;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Every browser session is an admin with the all-access key.
|
||||
*******************************************************************************/
|
||||
public static class DemoSessionCustomizer implements QAuthenticationModuleCustomizerInterface
|
||||
{
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void customizeSession(QInstance qInstance, QSession qSession, Map<String, Object> context)
|
||||
{
|
||||
QSession admin = AssistantTestFixture.sessionFor("admin-demo");
|
||||
qSession.setUser(admin.getUser());
|
||||
qSession.setPermissions(admin.getPermissions());
|
||||
qSession.withSecurityKeyValue(AssistantMetaDataBuilder.allAccessKeyName(AssistantQBitConfig.DEFAULT_OWNER_KEY_TYPE), true);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public QInstance defineQInstance() throws QException
|
||||
{
|
||||
return (instance);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Build, run real traffic against the recorded API, then serve.
|
||||
*******************************************************************************/
|
||||
public static void main(String[] args) throws Exception
|
||||
{
|
||||
int port = args.length > 0 ? Integer.parseInt(args[0]) : 8099;
|
||||
|
||||
RecordedAnthropicServer api = new RecordedAnthropicServer();
|
||||
AssistantQBitConfig config = new AssistantQBitConfig()
|
||||
.withProviderName(ScriptedModelProvider.DEFAULT_NAME)
|
||||
.withModelProvider(new ScriptedModelProvider())
|
||||
.withProviderType(new AnthropicProviderType())
|
||||
.withAllowedBaseUrls(Set.of(api.baseUrl()))
|
||||
.withAllowedApiKeyEnvNames(Set.of("ANTHROPIC_API_KEY_STAGING"))
|
||||
.withSecretResolver(Map.of(AnthropicModelProvider.API_KEY_ENV, "sk-ant-demo-not-a-real-key-7f3a")::get)
|
||||
.withAllowedWidgetNames(Set.of());
|
||||
instance = AssistantTestFixture.build(config);
|
||||
|
||||
QContext.init(instance, AssistantTestFixture.sessionFor("admin-1"));
|
||||
Integer production = insert(AssistantProvider.TABLE_NAME, Map.of("name", "Claude (production)", "providerType", "anthropic", "model", "claude-sonnet-5-5",
|
||||
"enabled", true, "isDefault", true, "baseUrl", api.baseUrl(), "monthlyCostCapUsd", new BigDecimal("25.00"),
|
||||
"maxOutputTokensPerRequest", 16000, "maxConcurrentRequests", 4));
|
||||
Integer staging = insert(AssistantProvider.TABLE_NAME, Map.of("name", "Claude (staging key)", "providerType", "anthropic", "model", "claude-haiku-5-5",
|
||||
"enabled", false, "isDefault", false, "apiKeyEnvName", "ANTHROPIC_API_KEY_STAGING"));
|
||||
for(Map.Entry<String, ModelPricing> price : ModelPricing.defaults().entrySet())
|
||||
{
|
||||
ModelPricing p = price.getValue();
|
||||
insert(AssistantModelPrice.TABLE_NAME, Map.of("providerType", "anthropic", "model", price.getKey(),
|
||||
"inputUsdPerMTok", BigDecimal.valueOf(p.getInputUsdPerMTok()), "outputUsdPerMTok", BigDecimal.valueOf(p.getOutputUsdPerMTok()),
|
||||
"cacheReadUsdPerMTok", BigDecimal.valueOf(p.getCacheReadUsdPerMTok()), "cacheWrite5mUsdPerMTok", BigDecimal.valueOf(p.getCacheWrite5mUsdPerMTok()),
|
||||
"cacheWrite1hUsdPerMTok", BigDecimal.valueOf(p.getCacheWrite1hUsdPerMTok()), "notes", "Anthropic list price, 2026-10"));
|
||||
}
|
||||
|
||||
api.thenStream("text-with-thinking.sse", 0);
|
||||
ConnectionTest.run(config.getRuntime(), config.getRuntime().getStore().getProvider(production), "admin-1");
|
||||
ConnectionTest.run(config.getRuntime(), config.getRuntime().getStore().getProvider(staging), "admin-1");
|
||||
QContext.clear();
|
||||
|
||||
TurnRunner runner = new TurnRunner(instance, config);
|
||||
api.thenStream("parallel-tools.sse", 0).thenStream("text-with-thinking.sse", 0);
|
||||
runner.run(AssistantTestFixture.EDITOR, "How many published posts are there, and what tables can I see?");
|
||||
api.thenStream("text-with-thinking.sse", 0);
|
||||
runner.run(AssistantTestFixture.EDITOR, "Summarize this week's page changes.");
|
||||
api.thenStream("parallel-tools.sse", 0).thenStream("text-with-thinking.sse", 0);
|
||||
runner.run(AssistantTestFixture.ANALYST, "What were the pageviews yesterday?");
|
||||
api.thenError(529, "overloaded.json").thenError(529, "overloaded.json").thenError(529, "overloaded.json");
|
||||
runner.run(AssistantTestFixture.OTHER_EDITOR, "Draft a post outline.");
|
||||
|
||||
instance.withInstanceDefaultAuthentication(new QAuthenticationMetaData().withName("mock").withType(QAuthenticationType.MOCK)
|
||||
.withCustomizer(new QCodeReference(DemoSessionCustomizer.class)));
|
||||
instance.setHomeAppName(AssistantMetaDataBuilder.APP_NAME);
|
||||
|
||||
new QApplicationJavalinServer(new AssistantDemoServer()).withPort(port).withServeFrontendNext(true).start();
|
||||
System.out.println("Assistant demo on http://localhost:" + port + "/");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private static Integer insert(String table, Map<String, Serializable> values) throws QException
|
||||
{
|
||||
QRecord record = new QRecord();
|
||||
record.setValues(new LinkedHashMap<>(values));
|
||||
QRecord inserted = new InsertAction().execute(new InsertInput(table).withRecord(record)).getRecords().get(0);
|
||||
if(!inserted.getErrors().isEmpty())
|
||||
{
|
||||
throw (new QException("Demo insert failed: " + inserted.getErrors()));
|
||||
}
|
||||
return (inserted.getValueInteger("id"));
|
||||
}
|
||||
|
||||
}
|
||||
@@ -23,10 +23,12 @@ import java.util.LinkedHashSet;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import java.util.function.Function;
|
||||
import java.util.regex.Pattern;
|
||||
import com.fasterxml.jackson.annotation.JsonIgnore;
|
||||
import com.kingsrook.qbits.assistant.engine.ModelPricing;
|
||||
import com.kingsrook.qbits.assistant.provider.AssistantModelProvider;
|
||||
import com.kingsrook.qbits.assistant.provider.AssistantProviderType;
|
||||
import com.kingsrook.qqq.backend.core.actions.ai.AiExposure;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.producers.MetaDataCustomizerInterface;
|
||||
@@ -55,6 +57,11 @@ public class AssistantQBitConfig implements QBitConfig
|
||||
public static final String DEFAULT_SHORTCUT = "mod+j";
|
||||
public static final String DEFAULT_PROVIDER_NAME = "anthropic";
|
||||
|
||||
/*******************************************************************************
|
||||
** What an API key reference must look like: an environment variable name.
|
||||
*******************************************************************************/
|
||||
public static final Pattern ENV_NAME = Pattern.compile("^[A-Z_][A-Z0-9_]{0,99}$");
|
||||
|
||||
/*******************************************************************************
|
||||
** The owner key type used before 0.1.0-RC.3 (camelCase). See UPGRADE.md.
|
||||
*******************************************************************************/
|
||||
@@ -90,9 +97,16 @@ public class AssistantQBitConfig implements QBitConfig
|
||||
private Set<String> allowedWidgetNames = new LinkedHashSet<>();
|
||||
private List<String> customRedactionPatterns = new ArrayList<>();
|
||||
private Integer killSwitchPollMillis = 2000;
|
||||
private Set<String> allowedApiKeyEnvNames = new LinkedHashSet<>();
|
||||
private Set<String> allowedBaseUrls = new LinkedHashSet<>();
|
||||
private Integer providerTestTimeoutSeconds = 30;
|
||||
private Integer homeMaxRows = 20000;
|
||||
private String tablePathPrefix = "/app";
|
||||
|
||||
private transient MetaDataCustomizerInterface<QTableMetaData> tableMetaDataCustomizer;
|
||||
private transient Map<String, AssistantModelProvider> modelProviders = new LinkedHashMap<>();
|
||||
private transient Map<String, AssistantProviderType> providerTypes = new LinkedHashMap<>();
|
||||
private transient Function<String, String> secretResolver = System::getenv;
|
||||
private transient AssistantRuntime runtime;
|
||||
|
||||
|
||||
@@ -154,6 +168,15 @@ public class AssistantQBitConfig implements QBitConfig
|
||||
requirePositive(errors, "toolResultRetentionDays", toolResultRetentionDays);
|
||||
requirePositive(errors, "approvalRetentionDays", approvalRetentionDays);
|
||||
requirePositive(errors, "killSwitchPollMillis", killSwitchPollMillis);
|
||||
requirePositive(errors, "providerTestTimeoutSeconds", providerTestTimeoutSeconds);
|
||||
requirePositive(errors, "homeMaxRows", homeMaxRows);
|
||||
for(String name : allowedApiKeyEnvNames == null ? Set.<String>of() : allowedApiKeyEnvNames)
|
||||
{
|
||||
if(name == null || !ENV_NAME.matcher(name).matches())
|
||||
{
|
||||
errors.add("allowedApiKeyEnvNames: not an environment variable name: " + name);
|
||||
}
|
||||
}
|
||||
for(String pattern : customRedactionPatterns == null ? List.<String>of() : customRedactionPatterns)
|
||||
{
|
||||
try
|
||||
@@ -1196,4 +1219,221 @@ public class AssistantQBitConfig implements QBitConfig
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Register a provider type for the provider settings table.
|
||||
*******************************************************************************/
|
||||
public AssistantQBitConfig withProviderType(AssistantProviderType providerType)
|
||||
{
|
||||
if(providerType != null)
|
||||
{
|
||||
providerTypes.put(providerType.getType(), providerType);
|
||||
}
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The registered provider types, by type id.
|
||||
*******************************************************************************/
|
||||
@JsonIgnore
|
||||
public Map<String, AssistantProviderType> getProviderTypes()
|
||||
{
|
||||
return (providerTypes);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The environment variable names an assistant_provider row may use as its
|
||||
** API key reference: the configured allow-list plus each registered type's
|
||||
** default (e.g. ANTHROPIC_API_KEY). An admin can therefore never point a
|
||||
** provider at an unrelated secret such as a database password.
|
||||
*******************************************************************************/
|
||||
@JsonIgnore
|
||||
public Set<String> getEffectiveAllowedApiKeyEnvNames()
|
||||
{
|
||||
Set<String> names = new LinkedHashSet<>(allowedApiKeyEnvNames == null ? Set.of() : allowedApiKeyEnvNames);
|
||||
providerTypes.values().forEach(t -> names.add(t.getDefaultApiKeyEnvName()));
|
||||
return (names);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Resolve a key reference (an environment variable name) to the secret.
|
||||
*******************************************************************************/
|
||||
@JsonIgnore
|
||||
public Function<String, String> getSecretResolver()
|
||||
{
|
||||
return (secretResolver);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Use another secret lookup than System.getenv (a host secret store; tests).
|
||||
*******************************************************************************/
|
||||
public AssistantQBitConfig withSecretResolver(Function<String, String> secretResolver)
|
||||
{
|
||||
this.secretResolver = secretResolver;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for allowedApiKeyEnvNames (extra API key environment variable names a provider row may reference)
|
||||
*******************************************************************************/
|
||||
public Set<String> getAllowedApiKeyEnvNames()
|
||||
{
|
||||
return (this.allowedApiKeyEnvNames);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for allowedApiKeyEnvNames
|
||||
*******************************************************************************/
|
||||
public void setAllowedApiKeyEnvNames(Set<String> allowedApiKeyEnvNames)
|
||||
{
|
||||
this.allowedApiKeyEnvNames = allowedApiKeyEnvNames;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for allowedApiKeyEnvNames
|
||||
*******************************************************************************/
|
||||
public AssistantQBitConfig withAllowedApiKeyEnvNames(Set<String> allowedApiKeyEnvNames)
|
||||
{
|
||||
this.allowedApiKeyEnvNames = allowedApiKeyEnvNames;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for allowedBaseUrls (base URL overrides a provider row may use; empty allows only the type's own URL)
|
||||
*******************************************************************************/
|
||||
public Set<String> getAllowedBaseUrls()
|
||||
{
|
||||
return (this.allowedBaseUrls);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for allowedBaseUrls
|
||||
*******************************************************************************/
|
||||
public void setAllowedBaseUrls(Set<String> allowedBaseUrls)
|
||||
{
|
||||
this.allowedBaseUrls = allowedBaseUrls;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for allowedBaseUrls
|
||||
*******************************************************************************/
|
||||
public AssistantQBitConfig withAllowedBaseUrls(Set<String> allowedBaseUrls)
|
||||
{
|
||||
this.allowedBaseUrls = allowedBaseUrls;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for providerTestTimeoutSeconds (how long Test connection waits for the model)
|
||||
*******************************************************************************/
|
||||
public Integer getProviderTestTimeoutSeconds()
|
||||
{
|
||||
return (this.providerTestTimeoutSeconds);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for providerTestTimeoutSeconds
|
||||
*******************************************************************************/
|
||||
public void setProviderTestTimeoutSeconds(Integer providerTestTimeoutSeconds)
|
||||
{
|
||||
this.providerTestTimeoutSeconds = providerTestTimeoutSeconds;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for providerTestTimeoutSeconds
|
||||
*******************************************************************************/
|
||||
public AssistantQBitConfig withProviderTestTimeoutSeconds(Integer providerTestTimeoutSeconds)
|
||||
{
|
||||
this.providerTestTimeoutSeconds = providerTestTimeoutSeconds;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for homeMaxRows (rows per table the app home reads for one period)
|
||||
*******************************************************************************/
|
||||
public Integer getHomeMaxRows()
|
||||
{
|
||||
return (this.homeMaxRows);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for homeMaxRows
|
||||
*******************************************************************************/
|
||||
public void setHomeMaxRows(Integer homeMaxRows)
|
||||
{
|
||||
this.homeMaxRows = homeMaxRows;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for homeMaxRows
|
||||
*******************************************************************************/
|
||||
public AssistantQBitConfig withHomeMaxRows(Integer homeMaxRows)
|
||||
{
|
||||
this.homeMaxRows = homeMaxRows;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for tablePathPrefix (the front of the links the app home makes to tables and records)
|
||||
*******************************************************************************/
|
||||
public String getTablePathPrefix()
|
||||
{
|
||||
return (this.tablePathPrefix);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for tablePathPrefix
|
||||
*******************************************************************************/
|
||||
public void setTablePathPrefix(String tablePathPrefix)
|
||||
{
|
||||
this.tablePathPrefix = tablePathPrefix;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for tablePathPrefix
|
||||
*******************************************************************************/
|
||||
public AssistantQBitConfig withTablePathPrefix(String tablePathPrefix)
|
||||
{
|
||||
this.tablePathPrefix = tablePathPrefix;
|
||||
return (this);
|
||||
}
|
||||
|
||||
}
|
||||
+11
-2
@@ -21,8 +21,10 @@ import java.io.InputStream;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Properties;
|
||||
import com.kingsrook.qbits.assistant.home.AssistantHome;
|
||||
import com.kingsrook.qbits.assistant.metadata.AssistantMetaDataBuilder;
|
||||
import com.kingsrook.qbits.assistant.metadata.AssistantRecordScreens;
|
||||
import com.kingsrook.qbits.assistant.processes.TestConnectionProcess;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.logging.QLogger;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerMultiOutput;
|
||||
@@ -152,13 +154,20 @@ public class AssistantQBitProducer implements QBitMetaDataProducer<AssistantQBit
|
||||
}
|
||||
qInstance.addPossibleValueSource(builder.buildConversationPossibleValueSource());
|
||||
AssistantRecordScreens.buildPossibleValueSources().forEach(qInstance::addPossibleValueSource);
|
||||
AssistantRecordScreens.buildSettingsPossibleValueSources(qBitConfig).forEach(qInstance::addPossibleValueSource);
|
||||
AssistantRecordScreens.addConversationChildLists(qInstance);
|
||||
|
||||
QProcessMetaData retention = builder.buildRetentionProcess();
|
||||
retention.setSourceQBitName(qBitMetaData.getName());
|
||||
qInstance.addProcess(retention);
|
||||
|
||||
QAppMetaData app = builder.buildAdminApp(tables, retention);
|
||||
QProcessMetaData testConnection = TestConnectionProcess.process();
|
||||
testConnection.setSourceQBitName(qBitMetaData.getName());
|
||||
qInstance.addProcess(testConnection);
|
||||
|
||||
qInstance.addWidget(AssistantHome.widget());
|
||||
|
||||
QAppMetaData app = builder.buildAdminApp(tables);
|
||||
qInstance.addApp(app);
|
||||
|
||||
postProduceActions(new MetaDataProducerMultiOutput(), qInstance);
|
||||
@@ -201,7 +210,7 @@ public class AssistantQBitProducer implements QBitMetaDataProducer<AssistantQBit
|
||||
|
||||
LOG.info("Assistant QBit produced", logPair("version", VERSION), logPair("backendName", qBitConfig.getBackendName()), logPair("routes", routes), logPair("developerPanel", developerPanel),
|
||||
logPair("enabled", qBitConfig.isEffectivelyEnabled()), logPair("providerName", qBitConfig.getProviderName()),
|
||||
logPair("providerRegistered", qBitConfig.getActiveModelProvider() != null), logPair("mainModel", qBitConfig.getModels().getMainModel()));
|
||||
logPair("providerRegistered", qBitConfig.getActiveModelProvider() != null), logPair("providerTypes", qBitConfig.getProviderTypes().keySet()),logPair("mainModel", qBitConfig.getModels().getMainModel()));
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -25,6 +25,7 @@ import java.util.concurrent.ExecutorService;
|
||||
import java.util.concurrent.Executors;
|
||||
import com.kingsrook.qbits.assistant.engine.ContextBuilder;
|
||||
import com.kingsrook.qbits.assistant.engine.PolicyService;
|
||||
import com.kingsrook.qbits.assistant.engine.ProviderService;
|
||||
import com.kingsrook.qbits.assistant.engine.RateLimiter;
|
||||
import com.kingsrook.qbits.assistant.engine.RunningTurn;
|
||||
import com.kingsrook.qbits.assistant.persistence.AssistantStore;
|
||||
@@ -46,6 +47,7 @@ public class AssistantRuntime
|
||||
private final Clock clock;
|
||||
private final AssistantStore store;
|
||||
private final PolicyService policyService;
|
||||
private final ProviderService providerService;
|
||||
private final RateLimiter rateLimiter;
|
||||
private final AiToolKernel kernel;
|
||||
private final ContextBuilder contextBuilder;
|
||||
@@ -74,6 +76,7 @@ public class AssistantRuntime
|
||||
this.clock = clock;
|
||||
this.store = new AssistantStore();
|
||||
this.policyService = new PolicyService(config, store, clock);
|
||||
this.providerService = new ProviderService(config, store, clock);
|
||||
this.rateLimiter = new RateLimiter(clock);
|
||||
this.kernel = new AiToolKernel(qInstance, kernelSettings(config));
|
||||
this.contextBuilder = new ContextBuilder(config, kernel);
|
||||
@@ -271,4 +274,14 @@ public class AssistantRuntime
|
||||
return (executor);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for providerService
|
||||
*******************************************************************************/
|
||||
public ProviderService getProviderService()
|
||||
{
|
||||
return (providerService);
|
||||
}
|
||||
|
||||
}
|
||||
@@ -28,6 +28,7 @@ import com.kingsrook.qbits.assistant.AssistantSecurity;
|
||||
import com.kingsrook.qbits.assistant.engine.ContextBuilder;
|
||||
import com.kingsrook.qbits.assistant.engine.MessageParts;
|
||||
import com.kingsrook.qbits.assistant.engine.PolicyService;
|
||||
import com.kingsrook.qbits.assistant.engine.ProviderService;
|
||||
import com.kingsrook.qbits.assistant.engine.RunningTurn;
|
||||
import com.kingsrook.qbits.assistant.engine.TurnEngine;
|
||||
import com.kingsrook.qbits.assistant.engine.TurnEvent;
|
||||
@@ -96,7 +97,7 @@ public class AssistantApi
|
||||
{
|
||||
reason = "NO_PERMISSION";
|
||||
}
|
||||
else if(config.getActiveModelProvider() == null)
|
||||
else if(activeProvider() == null)
|
||||
{
|
||||
reason = "NOT_CONFIGURED";
|
||||
}
|
||||
@@ -111,7 +112,7 @@ public class AssistantApi
|
||||
result.put("reason", reason);
|
||||
}
|
||||
result.put("shortcut", config.getShortcut());
|
||||
result.put("model", reason == null ? runtime.getPolicyService().current(false).mainModel() : config.getModels().getMainModel());
|
||||
result.put("model", reason == null ? activeProvider().model() : config.getModels().getMainModel());
|
||||
result.put("limits", Map.of("maxTurnSeconds", config.getMaxTurnSeconds(), "maxToolCallsPerTurn", config.getMaxToolCallsPerTurn()));
|
||||
result.put("features", Map.of("writes", false));
|
||||
return (result);
|
||||
@@ -267,17 +268,23 @@ public class AssistantApi
|
||||
}
|
||||
Map<String, Object> pageContext = body.get("pageContext") instanceof Map<?, ?> m ? ContextBuilder.sanitizePageContext((Map<String, Object>) m) : Map.of();
|
||||
|
||||
if(config.getActiveModelProvider() == null)
|
||||
PolicyService.EffectivePolicy policy = runtime.getPolicyService().current(true);
|
||||
ProviderService.Active active = runtime.getProviderService().resolve(policy.mainModel());
|
||||
if(active == null)
|
||||
{
|
||||
throw (new AssistantApiException(503, "NOT_CONFIGURED", "The assistant is not configured on this server."));
|
||||
}
|
||||
|
||||
PolicyService.EffectivePolicy policy = runtime.getPolicyService().current(true);
|
||||
if(policy.killSwitch())
|
||||
{
|
||||
throw (new AssistantApiException(503, "KILL_SWITCH", "The assistant has been turned off by an administrator."));
|
||||
}
|
||||
|
||||
Long capMicros = ProviderService.capMicros(active.row());
|
||||
if(capMicros != null && runtime.getProviderService().monthToDateCostMicros(active.providerId()) >= capMicros)
|
||||
{
|
||||
throw (new AssistantApiException(429, "BUDGET", "The assistant has reached this provider's monthly cost cap. Ask an administrator."));
|
||||
}
|
||||
|
||||
PolicyService.BudgetStatus budget = runtime.getPolicyService().checkBudget(policy, userId, 0, 0);
|
||||
if(budget.hardExceeded())
|
||||
{
|
||||
@@ -326,7 +333,7 @@ public class AssistantApi
|
||||
.withUserId(userId)
|
||||
.withUserMessageId(userMessage.getId())
|
||||
.withStatus("RUNNING")
|
||||
.withModel(policy.mainModel())
|
||||
.withModel(active.model())
|
||||
.withStartedAt(runtime.getClock().instant())
|
||||
.withToolCallCount(0)
|
||||
.withModelCallCount(0));
|
||||
@@ -338,7 +345,7 @@ public class AssistantApi
|
||||
.withStatus("STREAMING")
|
||||
.withTurnId(turn.getId())
|
||||
.withSeq(seq + 1)
|
||||
.withModel(policy.mainModel())
|
||||
.withModel(active.model())
|
||||
.withPartsJson("[]"));
|
||||
|
||||
store.updateFields(AssistantMessage.TABLE_NAME, new QRecord().withValue("id", userMessage.getId()).withValue("turnId", turn.getId()));
|
||||
@@ -657,4 +664,14 @@ public class AssistantApi
|
||||
return (value == null || value.length() <= max ? value : value.substring(0, max));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The provider answers would come from now, or null.
|
||||
*******************************************************************************/
|
||||
private ProviderService.Active activeProvider() throws QException
|
||||
{
|
||||
return (runtime.getProviderService().resolve(runtime.getPolicyService().current(false).mainModel()));
|
||||
}
|
||||
|
||||
}
|
||||
+182
@@ -0,0 +1,182 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.engine;
|
||||
|
||||
|
||||
import java.time.Duration;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import com.kingsrook.qbits.assistant.AssistantRuntime;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantProvider;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelMessage;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelRequest;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelResponse;
|
||||
import com.kingsrook.qbits.assistant.provider.StreamHandle;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Test connection: one minimal live call to a provider row's model (a short
|
||||
** prompt, at most PING_MAX_TOKENS output tokens, low effort). The result
|
||||
** (status, latency, the model that answered, error text) is saved on the
|
||||
** row, and the call is recorded in assistant_usage with purpose TEST.
|
||||
** Error text comes from the provider exception, which never carries the key
|
||||
** or the request.
|
||||
*******************************************************************************/
|
||||
public final class ConnectionTest
|
||||
{
|
||||
public static final int PING_MAX_TOKENS = 32;
|
||||
public static final String PING_PROMPT = "Connection test. Reply with the single word OK.";
|
||||
|
||||
public static final String STATUS_OK = "OK";
|
||||
public static final String STATUS_ERROR = "ERROR";
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** What a test found.
|
||||
*******************************************************************************/
|
||||
public record Result(String status, Integer latencyMs, String model, String error)
|
||||
{
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
public boolean ok()
|
||||
{
|
||||
return (STATUS_OK.equals(status));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private ConnectionTest()
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Test one provider row and save the result on it.
|
||||
*******************************************************************************/
|
||||
public static Result run(AssistantRuntime runtime, AssistantProvider row, String userId) throws QException
|
||||
{
|
||||
Result result = call(runtime, row, userId);
|
||||
runtime.getStore().updateFields(AssistantProvider.TABLE_NAME, new QRecord()
|
||||
.withValue("id", row.getId())
|
||||
.withValue("lastTestAt", runtime.getClock().instant())
|
||||
.withValue("lastTestStatus", result.status())
|
||||
.withValue("lastTestLatencyMs", result.latencyMs())
|
||||
.withValue("lastTestModel", result.model())
|
||||
.withValue("lastTestError", truncate(result.error(), 500)));
|
||||
runtime.getProviderService().invalidate();
|
||||
return (result);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Make the call (and record its usage).
|
||||
*******************************************************************************/
|
||||
static Result call(AssistantRuntime runtime, AssistantProvider row, String userId)
|
||||
{
|
||||
ProviderService.Active active;
|
||||
try
|
||||
{
|
||||
active = runtime.getProviderService().forRow(row, runtime.getConfig().getModels().getMainModel());
|
||||
}
|
||||
catch(ProviderService.ProviderSetupException e)
|
||||
{
|
||||
return (new Result(STATUS_ERROR, null, null, e.getMessage()));
|
||||
}
|
||||
|
||||
List<ModelMessage> messages = new ArrayList<>();
|
||||
messages.add(ModelMessage.user(PING_PROMPT));
|
||||
ModelRequest request = new ModelRequest()
|
||||
.withModel(active.model())
|
||||
.withEffort("low")
|
||||
.withMaxTokens(PING_MAX_TOKENS)
|
||||
.withMessages(messages);
|
||||
|
||||
UsageRecorder.Call usageCall = new UsageRecorder.Call(userId, null, null, active, UsageRecorder.PURPOSE_TEST);
|
||||
ModelPricing pricing = runtime.getProviderService().pricing(active.providerType(), active.model());
|
||||
long started = System.nanoTime();
|
||||
try
|
||||
{
|
||||
StreamHandle handle = active.provider().stream(request, text ->
|
||||
{
|
||||
// the reply text is not used
|
||||
});
|
||||
ModelResponse response = handle.await(Duration.ofSeconds(runtime.getConfig().getProviderTestTimeoutSeconds()));
|
||||
long latency = (System.nanoTime() - started) / 1_000_000;
|
||||
UsageRecorder.save(runtime.getStore(), UsageRecorder.row(usageCall, response, null, latency, pricing));
|
||||
if(ModelResponse.STOP_CANCELLED.equals(response.getStopReason()))
|
||||
{
|
||||
return (new Result(STATUS_ERROR, (int) latency, null, "The call was cancelled."));
|
||||
}
|
||||
return (new Result(STATUS_OK, (int) latency, response.getModel() != null ? response.getModel() : active.model(), null));
|
||||
}
|
||||
catch(ModelProviderException e)
|
||||
{
|
||||
long latency = (System.nanoTime() - started) / 1_000_000;
|
||||
UsageRecorder.save(runtime.getStore(), UsageRecorder.row(usageCall, null, e, latency, pricing));
|
||||
return (new Result(STATUS_ERROR, (int) latency, null, describe(e)));
|
||||
}
|
||||
catch(InterruptedException e)
|
||||
{
|
||||
Thread.currentThread().interrupt();
|
||||
return (new Result(STATUS_ERROR, null, null, "The test was interrupted."));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** An admin-readable error: the kind, the HTTP status and the provider's
|
||||
** (credential-free) message.
|
||||
*******************************************************************************/
|
||||
static String describe(ModelProviderException e)
|
||||
{
|
||||
String hint = switch(e.getKind())
|
||||
{
|
||||
case AUTHENTICATION -> "The API key was refused. Check the key in the secret store.";
|
||||
case RATE_LIMITED -> "Rate limited by the provider. Try again shortly.";
|
||||
case OVERLOADED -> "The provider is overloaded. Try again shortly.";
|
||||
case INVALID_REQUEST -> "The provider refused the request. Check the model id.";
|
||||
case TIMEOUT -> "No answer in time.";
|
||||
case NETWORK -> "Could not reach the provider. Check the base URL and the network.";
|
||||
case SERVER -> "The provider had a server error.";
|
||||
default -> "The call failed.";
|
||||
};
|
||||
return (hint + " (" + e.getKind() + (e.getHttpStatus() == null ? "" : ", HTTP " + e.getHttpStatus()) + (e.getMessage() == null ? "" : ": " + e.getMessage()) + ")");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private static String truncate(String value, int max)
|
||||
{
|
||||
return (value == null || value.length() <= max ? value : value.substring(0, max));
|
||||
}
|
||||
|
||||
}
|
||||
@@ -66,6 +66,15 @@ public class ModelPricing implements Serializable
|
||||
map.put(AssistantModelRouting.SONNET_5_5, of(2.0, 10.0, 0.1));
|
||||
map.put(AssistantModelRouting.OPUS_5_5, of(4.0, 20.0, 0.05));
|
||||
map.put(AssistantModelRouting.HAIKU_5_5, of(0.10, 0.50, 0.1));
|
||||
map.put("claude-fable-5-1", of(10.0, 50.0, 0.025));
|
||||
map.put("claude-fable-5", of(10.0, 50.0, 0.1));
|
||||
map.put("claude-opus-5", of(5.0, 25.0, 0.1));
|
||||
map.put("claude-opus-4-8", of(5.0, 25.0, 0.1));
|
||||
map.put("claude-opus-4-7", of(5.0, 25.0, 0.1));
|
||||
map.put("claude-opus-4-6", of(5.0, 25.0, 0.1));
|
||||
map.put("claude-sonnet-5", of(2.0, 10.0, 0.1));
|
||||
map.put("claude-sonnet-4-6", of(3.0, 15.0, 0.1));
|
||||
map.put("claude-haiku-4-5", of(1.0, 5.0, 0.1));
|
||||
return (map);
|
||||
}
|
||||
|
||||
|
||||
+552
@@ -0,0 +1,552 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.engine;
|
||||
|
||||
|
||||
import java.math.BigDecimal;
|
||||
import java.math.RoundingMode;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.security.MessageDigest;
|
||||
import java.time.Clock;
|
||||
import java.time.Instant;
|
||||
import java.time.LocalDate;
|
||||
import java.time.ZoneOffset;
|
||||
import java.util.ArrayList;
|
||||
import java.util.HexFormat;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import java.util.concurrent.Semaphore;
|
||||
import com.kingsrook.qbits.assistant.AssistantQBitConfig;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantModelPrice;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantProvider;
|
||||
import com.kingsrook.qbits.assistant.persistence.AssistantStore;
|
||||
import com.kingsrook.qbits.assistant.provider.AssistantModelProvider;
|
||||
import com.kingsrook.qbits.assistant.provider.AssistantProviderType;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.logging.QLogger;
|
||||
import com.kingsrook.qqq.backend.core.utils.StringUtils;
|
||||
import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Which provider and model answer, from the assistant_provider settings rows,
|
||||
** and what a call costs, from the assistant_model_price rows.
|
||||
**
|
||||
** - The active provider is the enabled row marked default (lowest id), else
|
||||
** the lowest enabled row. With no enabled rows, the provider registered in
|
||||
** the QBit config (withModelProvider / providerName) is used, so hosts
|
||||
** that have not created a row keep working.
|
||||
** - A row's API key is never stored: apiKeyEnvName names an environment
|
||||
** variable (resolved through config.getSecretResolver()). Only names in
|
||||
** the allow-list may be used, and a base URL override must be allowed in
|
||||
** the config, so an admin cannot send another secret, or the key, to a
|
||||
** host of their choosing.
|
||||
** - Built providers are cached by row settings and a SHA-256 fingerprint of
|
||||
** the key, so a rotated key builds a new client. Keys are never logged.
|
||||
*******************************************************************************/
|
||||
public class ProviderService
|
||||
{
|
||||
private static final QLogger LOG = QLogger.getLogger(ProviderService.class);
|
||||
|
||||
public static final String KEY_STATUS_FIELD = "apiKeyStatus";
|
||||
|
||||
private final AssistantQBitConfig config;
|
||||
private final AssistantStore store;
|
||||
private final Clock clock;
|
||||
|
||||
private final Map<String, AssistantModelProvider> built = new ConcurrentHashMap<>();
|
||||
private final Map<Integer, Semaphore> concurrency = new ConcurrentHashMap<>();
|
||||
|
||||
private volatile List<AssistantProvider> cachedRows;
|
||||
private volatile Map<String, ModelPricing> cachedPrices;
|
||||
private volatile Instant cachedAt;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The provider a turn uses. row is null when the config provider is used.
|
||||
*******************************************************************************/
|
||||
public record Active(AssistantProvider row, String providerType, String model, AssistantModelProvider provider)
|
||||
{
|
||||
/*******************************************************************************
|
||||
** The row id, or null.
|
||||
*******************************************************************************/
|
||||
public Integer providerId()
|
||||
{
|
||||
return (row == null ? null : row.getId());
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A provider row that cannot be used (the message is safe to show).
|
||||
*******************************************************************************/
|
||||
public static class ProviderSetupException extends QException
|
||||
{
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
public ProviderSetupException(String message)
|
||||
{
|
||||
super(message);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
public ProviderService(AssistantQBitConfig config, AssistantStore store, Clock clock)
|
||||
{
|
||||
this.config = config;
|
||||
this.store = store;
|
||||
this.clock = clock;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The active provider for a model choice from the policy (used when the row
|
||||
** has no model), or null when nothing usable is configured.
|
||||
*******************************************************************************/
|
||||
public Active resolve(String policyModel) throws QException
|
||||
{
|
||||
AssistantProvider row = activeRow();
|
||||
if(row == null)
|
||||
{
|
||||
AssistantModelProvider provider = config.getActiveModelProvider();
|
||||
return (provider == null ? null : new Active(null, provider.getName(), policyModel, provider));
|
||||
}
|
||||
|
||||
try
|
||||
{
|
||||
return (forRow(row, policyModel));
|
||||
}
|
||||
catch(ProviderSetupException e)
|
||||
{
|
||||
LOG.warn("The default assistant provider cannot be used", e, logPair("providerId", row.getId()));
|
||||
return (null);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The default enabled provider row, or null.
|
||||
*******************************************************************************/
|
||||
public AssistantProvider activeRow() throws QException
|
||||
{
|
||||
AssistantProvider first = null;
|
||||
for(AssistantProvider row : rows())
|
||||
{
|
||||
if(Boolean.TRUE.equals(row.getEnabled()))
|
||||
{
|
||||
if(Boolean.TRUE.equals(row.getIsDefault()))
|
||||
{
|
||||
return (row);
|
||||
}
|
||||
if(first == null)
|
||||
{
|
||||
first = row;
|
||||
}
|
||||
}
|
||||
}
|
||||
return (first);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Build (or reuse) the provider of one row. Throws a ProviderSetupException
|
||||
** with a message for the admin when the row cannot be used.
|
||||
*******************************************************************************/
|
||||
public Active forRow(AssistantProvider row, String fallbackModel) throws ProviderSetupException
|
||||
{
|
||||
List<String> errors = validate(row);
|
||||
if(!errors.isEmpty())
|
||||
{
|
||||
throw (new ProviderSetupException(String.join(" ", errors)));
|
||||
}
|
||||
|
||||
AssistantProviderType type = config.getProviderTypes().get(row.getProviderType());
|
||||
String key = resolveKey(row);
|
||||
if(!StringUtils.hasContent(key))
|
||||
{
|
||||
throw (new ProviderSetupException("No API key: the environment variable " + keyEnvName(row) + " is not set on this server."));
|
||||
}
|
||||
|
||||
String model = StringUtils.hasContent(row.getModel()) ? row.getModel() : (StringUtils.hasContent(fallbackModel) ? fallbackModel : first(type.getModels()));
|
||||
String baseUrl = StringUtils.hasContent(row.getBaseUrl()) ? row.getBaseUrl().trim() : null;
|
||||
String cacheKey = row.getId() + "|" + row.getProviderType() + "|" + baseUrl + "|" + fingerprint(key);
|
||||
AssistantModelProvider provider = built.computeIfAbsent(cacheKey, k -> type.create(key, baseUrl));
|
||||
return (new Active(row, row.getProviderType(), model, provider));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** What is wrong with a row's settings (empty when it can be used, apart
|
||||
** from the key being present). Messages never contain a key.
|
||||
*******************************************************************************/
|
||||
public List<String> validate(AssistantProvider row)
|
||||
{
|
||||
List<String> errors = new ArrayList<>();
|
||||
AssistantProviderType type = row.getProviderType() == null ? null : config.getProviderTypes().get(row.getProviderType());
|
||||
if(type == null)
|
||||
{
|
||||
errors.add("Provider type " + row.getProviderType() + " is not installed on this server.");
|
||||
return (errors);
|
||||
}
|
||||
|
||||
String envName = keyEnvName(row);
|
||||
if(!AssistantQBitConfig.ENV_NAME.matcher(envName).matches())
|
||||
{
|
||||
errors.add("API key variable must be the NAME of an environment variable (capital letters, digits and _), not the key itself.");
|
||||
}
|
||||
else if(!config.getEffectiveAllowedApiKeyEnvNames().contains(envName))
|
||||
{
|
||||
errors.add("API key variable " + envName + " is not allowed on this server. Allowed: " + String.join(", ", config.getEffectiveAllowedApiKeyEnvNames()) + ".");
|
||||
}
|
||||
|
||||
if(StringUtils.hasContent(row.getBaseUrl()))
|
||||
{
|
||||
String url = row.getBaseUrl().trim();
|
||||
if(!url.equals(type.getDefaultBaseUrl()) && (config.getAllowedBaseUrls() == null || !config.getAllowedBaseUrls().contains(url)))
|
||||
{
|
||||
errors.add("Base URL " + url + " is not allowed on this server; leave it empty to use " + type.getDefaultBaseUrl() + ".");
|
||||
}
|
||||
}
|
||||
|
||||
if(row.getMonthlyCostCapUsd() != null && row.getMonthlyCostCapUsd().signum() < 0)
|
||||
{
|
||||
errors.add("Monthly cost cap must not be negative.");
|
||||
}
|
||||
if(row.getMaxOutputTokensPerRequest() != null && row.getMaxOutputTokensPerRequest() < 1)
|
||||
{
|
||||
errors.add("Max output tokens per request must be at least 1.");
|
||||
}
|
||||
if(row.getMaxConcurrentRequests() != null && row.getMaxConcurrentRequests() < 1)
|
||||
{
|
||||
errors.add("Max concurrent requests must be at least 1.");
|
||||
}
|
||||
return (errors);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The masked key status shown on the settings screen: whether the key is
|
||||
** set and its last four characters, never more.
|
||||
*******************************************************************************/
|
||||
public String keyStatus(AssistantProvider row)
|
||||
{
|
||||
String envName = keyEnvName(row);
|
||||
if(!AssistantQBitConfig.ENV_NAME.matcher(envName).matches() || !config.getEffectiveAllowedApiKeyEnvNames().contains(envName))
|
||||
{
|
||||
return ("Not usable: the reference is not an allowed environment variable name");
|
||||
}
|
||||
String key = resolveKey(row);
|
||||
if(!StringUtils.hasContent(key))
|
||||
{
|
||||
return ("Not set (" + envName + " is empty)");
|
||||
}
|
||||
return (mask(key) + " (from " + envName + ")");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** "Set • …abcd" for a key (only the last four characters, and none for a
|
||||
** key shorter than 12).
|
||||
*******************************************************************************/
|
||||
public static String mask(String key)
|
||||
{
|
||||
if(key == null || key.isEmpty())
|
||||
{
|
||||
return ("Not set");
|
||||
}
|
||||
String trimmed = key.trim();
|
||||
return (trimmed.length() < 12 ? "Set" : "Set • …" + trimmed.substring(trimmed.length() - 4));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The key variable of a row (its own, else the type's default).
|
||||
*******************************************************************************/
|
||||
public String keyEnvName(AssistantProvider row)
|
||||
{
|
||||
if(StringUtils.hasContent(row.getApiKeyEnvName()))
|
||||
{
|
||||
return (row.getApiKeyEnvName().trim());
|
||||
}
|
||||
AssistantProviderType type = row.getProviderType() == null ? null : config.getProviderTypes().get(row.getProviderType());
|
||||
return (type == null ? "" : type.getDefaultApiKeyEnvName());
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The key of a row (only for allowed variable names), or null.
|
||||
*******************************************************************************/
|
||||
private String resolveKey(AssistantProvider row)
|
||||
{
|
||||
String envName = keyEnvName(row);
|
||||
if(!AssistantQBitConfig.ENV_NAME.matcher(envName).matches() || !config.getEffectiveAllowedApiKeyEnvNames().contains(envName))
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
try
|
||||
{
|
||||
String value = config.getSecretResolver() == null ? null : config.getSecretResolver().apply(envName);
|
||||
return (value == null ? null : value.trim());
|
||||
}
|
||||
catch(RuntimeException e)
|
||||
{
|
||||
LOG.warn("Could not resolve an assistant API key reference", logPair("envName", envName));
|
||||
return (null);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The concurrency gate of a provider row (null: unlimited).
|
||||
*******************************************************************************/
|
||||
public Semaphore gate(AssistantProvider row)
|
||||
{
|
||||
if(row == null || row.getMaxConcurrentRequests() == null || row.getMaxConcurrentRequests() < 1)
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
int limit = row.getMaxConcurrentRequests();
|
||||
return (concurrency.compute(row.getId(), (id, existing) -> limitOf(existing) == limit ? existing : new LimitedSemaphore(limit)));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private static int limitOf(Semaphore semaphore)
|
||||
{
|
||||
return (semaphore instanceof LimitedSemaphore limited ? limited.limit : -1);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A semaphore that remembers its size (so a changed limit makes a new one).
|
||||
*******************************************************************************/
|
||||
private static class LimitedSemaphore extends Semaphore
|
||||
{
|
||||
private final int limit;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
LimitedSemaphore(int limit)
|
||||
{
|
||||
super(limit, true);
|
||||
this.limit = limit;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The monthly cost cap of a row in micro-dollars, or null.
|
||||
*******************************************************************************/
|
||||
public static Long capMicros(AssistantProvider row)
|
||||
{
|
||||
if(row == null || row.getMonthlyCostCapUsd() == null)
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
return (row.getMonthlyCostCapUsd().multiply(BigDecimal.valueOf(1_000_000)).setScale(0, RoundingMode.DOWN).longValueExact());
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** What a provider has cost since the start of the month (UTC), in
|
||||
** micro-dollars, from the usage rows the session can see.
|
||||
*******************************************************************************/
|
||||
public long monthToDateCostMicros(Integer providerId) throws QException
|
||||
{
|
||||
if(providerId == null)
|
||||
{
|
||||
return (0);
|
||||
}
|
||||
Instant monthStart = LocalDate.now(clock.withZone(ZoneOffset.UTC)).withDayOfMonth(1).atStartOfDay().toInstant(ZoneOffset.UTC);
|
||||
return (store.sumUsageCostSince(providerId, monthStart));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The price of a model: the assistant_model_price row for the type and
|
||||
** model, else the config price table, else zero.
|
||||
*******************************************************************************/
|
||||
public ModelPricing pricing(String providerType, String model)
|
||||
{
|
||||
try
|
||||
{
|
||||
ModelPricing row = prices().get(priceKey(providerType, model));
|
||||
if(row != null)
|
||||
{
|
||||
return (row);
|
||||
}
|
||||
}
|
||||
catch(QException e)
|
||||
{
|
||||
LOG.warn("Could not read the assistant price table; using the configured prices", e);
|
||||
}
|
||||
ModelPricing configured = config.getPricing() == null || model == null ? null : config.getPricing().get(model);
|
||||
return (configured == null ? new ModelPricing() : configured);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A price row as ModelPricing (cache prices default to the standard
|
||||
** multipliers of the input price when empty).
|
||||
*******************************************************************************/
|
||||
public static ModelPricing toPricing(AssistantModelPrice price)
|
||||
{
|
||||
double input = d(price.getInputUsdPerMTok());
|
||||
return (new ModelPricing()
|
||||
.withInputUsdPerMTok(input)
|
||||
.withOutputUsdPerMTok(d(price.getOutputUsdPerMTok()))
|
||||
.withCacheReadUsdPerMTok(price.getCacheReadUsdPerMTok() == null ? input * 0.1 : d(price.getCacheReadUsdPerMTok()))
|
||||
.withCacheWrite5mUsdPerMTok(price.getCacheWrite5mUsdPerMTok() == null ? input * 1.25 : d(price.getCacheWrite5mUsdPerMTok()))
|
||||
.withCacheWrite1hUsdPerMTok(price.getCacheWrite1hUsdPerMTok() == null ? input * 2.0 : d(price.getCacheWrite1hUsdPerMTok())));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Drop the cached rows and prices (after a settings change).
|
||||
*******************************************************************************/
|
||||
public void invalidate()
|
||||
{
|
||||
cachedAt = null;
|
||||
cachedRows = null;
|
||||
cachedPrices = null;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The provider rows (cached for policyCacheSeconds).
|
||||
*******************************************************************************/
|
||||
public List<AssistantProvider> rows() throws QException
|
||||
{
|
||||
refresh();
|
||||
return (cachedRows);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private Map<String, ModelPricing> prices() throws QException
|
||||
{
|
||||
refresh();
|
||||
return (cachedPrices);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private synchronized void refresh() throws QException
|
||||
{
|
||||
Instant now = clock.instant();
|
||||
if(cachedAt != null && cachedRows != null && cachedPrices != null && cachedAt.plusSeconds(config.getPolicyCacheSeconds()).isAfter(now))
|
||||
{
|
||||
return;
|
||||
}
|
||||
List<AssistantProvider> rows = store.listProviders();
|
||||
Map<String, ModelPricing> prices = new ConcurrentHashMap<>();
|
||||
for(AssistantModelPrice price : store.listPrices())
|
||||
{
|
||||
if(price.getModel() != null && price.getInputUsdPerMTok() != null && price.getOutputUsdPerMTok() != null)
|
||||
{
|
||||
prices.put(priceKey(price.getProviderType(), price.getModel()), toPricing(price));
|
||||
}
|
||||
}
|
||||
cachedRows = rows;
|
||||
cachedPrices = prices;
|
||||
cachedAt = now;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private static String priceKey(String providerType, String model)
|
||||
{
|
||||
return ((providerType == null ? "" : providerType) + "|" + model);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** SHA-256 of a key (hex), for the client cache key only.
|
||||
*******************************************************************************/
|
||||
static String fingerprint(String key)
|
||||
{
|
||||
try
|
||||
{
|
||||
return (HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(key.getBytes(StandardCharsets.UTF_8))));
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
throw (new IllegalStateException("SHA-256 is not available", e));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private static String first(List<String> list)
|
||||
{
|
||||
return (list == null || list.isEmpty() ? null : list.get(0));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private static double d(BigDecimal value)
|
||||
{
|
||||
return (value == null ? 0 : value.doubleValue());
|
||||
}
|
||||
|
||||
}
|
||||
@@ -123,6 +123,11 @@ public class TurnEngine implements Runnable
|
||||
private int modelCallCount = 0;
|
||||
private long costMicros = 0;
|
||||
|
||||
private ProviderService.Active active;
|
||||
private String model;
|
||||
private Long capMicros;
|
||||
private long monthCostAtStart = 0;
|
||||
|
||||
private volatile boolean finished = false;
|
||||
|
||||
|
||||
@@ -177,8 +182,20 @@ public class TurnEngine implements Runnable
|
||||
}
|
||||
persistPendingEvents();
|
||||
|
||||
AssistantModelProvider provider = config.getActiveModelProvider();
|
||||
ModelCapabilities caps = provider.getCapabilities(policy.mainModel());
|
||||
model = policy.mainModel();
|
||||
active = runtime.getProviderService().resolve(policy.mainModel());
|
||||
if(active == null)
|
||||
{
|
||||
throw (new ModelProviderException(ModelProviderException.Kind.AUTHENTICATION, null, "No usable assistant provider is configured."));
|
||||
}
|
||||
model = active.model();
|
||||
capMicros = ProviderService.capMicros(active.row());
|
||||
if(capMicros != null)
|
||||
{
|
||||
monthCostAtStart = runtime.getProviderService().monthToDateCostMicros(active.providerId());
|
||||
}
|
||||
AssistantModelProvider provider = active.provider();
|
||||
ModelCapabilities caps = provider.getCapabilities(model);
|
||||
|
||||
List<AssistantMessage> history = new ArrayList<>();
|
||||
for(AssistantMessage message : store.listMessages(userId, conversation.getId(), userMessage.getId(), config.getHistoryMessageLimit()))
|
||||
@@ -189,7 +206,12 @@ public class TurnEngine implements Runnable
|
||||
}
|
||||
}
|
||||
|
||||
ModelRequest request = runtime.getContextBuilder().build(session, policy.mainModel(), policy.mainEffort(), history, userText, pageContext);
|
||||
ModelRequest request = runtime.getContextBuilder().build(session, model, policy.mainEffort(), history, userText, pageContext);
|
||||
Integer maxPerRequest = active.row() == null ? null : active.row().getMaxOutputTokensPerRequest();
|
||||
if(maxPerRequest != null && (request.getMaxTokens() == null || request.getMaxTokens() > maxPerRequest))
|
||||
{
|
||||
request.setMaxTokens(maxPerRequest);
|
||||
}
|
||||
if(caps != null && Boolean.FALSE.equals(caps.getParallelToolCalls()))
|
||||
{
|
||||
request.setParallelToolCalls(false);
|
||||
@@ -209,6 +231,14 @@ public class TurnEngine implements Runnable
|
||||
break;
|
||||
}
|
||||
|
||||
if(capMicros != null && monthCostAtStart + costMicros >= capMicros)
|
||||
{
|
||||
status = "ERROR";
|
||||
errorCode = "BUDGET";
|
||||
errorText = "The assistant has reached this provider's monthly cost cap. Ask an administrator.";
|
||||
break;
|
||||
}
|
||||
|
||||
ModelResponse response = callModel(provider, request);
|
||||
stopReason = response.getStopReason();
|
||||
|
||||
@@ -297,7 +327,7 @@ public class TurnEngine implements Runnable
|
||||
if(e.getPartialUsage() != null)
|
||||
{
|
||||
usage.add(e.getPartialUsage());
|
||||
costMicros += pricing(policy.mainModel()).costMicros(e.getPartialUsage());
|
||||
costMicros += pricing().costMicros(e.getPartialUsage());
|
||||
}
|
||||
status = "ERROR";
|
||||
errorCode = e.getKind() == ModelProviderException.Kind.TIMEOUT ? "TIMEOUT" : "PROVIDER";
|
||||
@@ -455,24 +485,51 @@ public class TurnEngine implements Runnable
|
||||
}
|
||||
};
|
||||
|
||||
StreamHandle handle = provider.stream(request.copy(), listener);
|
||||
running.setCurrentHandle(handle);
|
||||
ModelResponse response;
|
||||
Semaphore gate = runtime.getProviderService().gate(active.row());
|
||||
if(gate != null)
|
||||
{
|
||||
Duration wait = Duration.between(runtime.getClock().instant(), deadline);
|
||||
if(!gate.tryAcquire(Math.max(1, wait.toMillis()), TimeUnit.MILLISECONDS))
|
||||
{
|
||||
throw (new ModelProviderException(ModelProviderException.Kind.RATE_LIMITED, null, "The provider's concurrency limit was reached."));
|
||||
}
|
||||
}
|
||||
|
||||
long started = System.nanoTime();
|
||||
ModelResponse response = null;
|
||||
try
|
||||
{
|
||||
Duration remaining = Duration.between(runtime.getClock().instant(), deadline);
|
||||
response = handle.await(remaining.isNegative() ? Duration.ofMillis(1) : remaining);
|
||||
StreamHandle handle = provider.stream(request.copy(), listener);
|
||||
running.setCurrentHandle(handle);
|
||||
try
|
||||
{
|
||||
Duration remaining = Duration.between(runtime.getClock().instant(), deadline);
|
||||
response = handle.await(remaining.isNegative() ? Duration.ofMillis(1) : remaining);
|
||||
}
|
||||
finally
|
||||
{
|
||||
running.setCurrentHandle(null);
|
||||
flushText(true);
|
||||
}
|
||||
}
|
||||
catch(ModelProviderException e)
|
||||
{
|
||||
recordUsage(null, e, started);
|
||||
throw (e);
|
||||
}
|
||||
finally
|
||||
{
|
||||
running.setCurrentHandle(null);
|
||||
flushText(true);
|
||||
if(gate != null)
|
||||
{
|
||||
gate.release();
|
||||
}
|
||||
}
|
||||
recordUsage(response, null, started);
|
||||
|
||||
if(response.getUsage() != null)
|
||||
{
|
||||
usage.add(response.getUsage());
|
||||
costMicros += pricing(policy.mainModel()).costMicros(response.getUsage());
|
||||
costMicros += pricing().costMicros(response.getUsage());
|
||||
}
|
||||
if(response.getRequestId() != null)
|
||||
{
|
||||
@@ -682,7 +739,7 @@ public class TurnEngine implements Runnable
|
||||
parts.addNotice("error", errorText);
|
||||
}
|
||||
|
||||
String model = policy == null ? null : policy.mainModel();
|
||||
String model = this.model != null ? this.model : (policy == null ? null : policy.mainModel());
|
||||
try
|
||||
{
|
||||
store.updateFields(AssistantMessage.TABLE_NAME, new QRecord()
|
||||
@@ -866,10 +923,21 @@ public class TurnEngine implements Runnable
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private ModelPricing pricing(String model)
|
||||
private ModelPricing pricing()
|
||||
{
|
||||
ModelPricing pricing = config.getPricing() == null ? null : config.getPricing().get(model);
|
||||
return (pricing == null ? new ModelPricing() : pricing);
|
||||
return (runtime.getProviderService().pricing(active == null ? null : active.providerType(), model));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** One assistant_usage row for a model call of this turn.
|
||||
*******************************************************************************/
|
||||
private void recordUsage(ModelResponse response, ModelProviderException error, long startedNanos)
|
||||
{
|
||||
long latencyMs = (System.nanoTime() - startedNanos) / 1_000_000;
|
||||
UsageRecorder.save(store, UsageRecorder.row(new UsageRecorder.Call(userId, conversation.getId(), turn.getId(), active, UsageRecorder.PURPOSE_TURN),
|
||||
response, error, latencyMs, pricing()));
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,133 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.engine;
|
||||
|
||||
|
||||
import com.kingsrook.qbits.assistant.model.AssistantUsage;
|
||||
import com.kingsrook.qbits.assistant.persistence.AssistantStore;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelResponse;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelUsage;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.logging.QLogger;
|
||||
import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Writes one assistant_usage row per model call (a turn step or a
|
||||
** connection test), with the tokens, the cost from the price table, the
|
||||
** latency and how the call ended. A failed write is logged and never fails
|
||||
** the call it describes.
|
||||
*******************************************************************************/
|
||||
public final class UsageRecorder
|
||||
{
|
||||
private static final QLogger LOG = QLogger.getLogger(UsageRecorder.class);
|
||||
|
||||
public static final String PURPOSE_TURN = "TURN";
|
||||
public static final String PURPOSE_TEST = "TEST";
|
||||
|
||||
public static final String STATUS_OK = "OK";
|
||||
public static final String STATUS_ERROR = "ERROR";
|
||||
public static final String STATUS_CANCELLED = "CANCELLED";
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Who and what a call was for.
|
||||
*******************************************************************************/
|
||||
public record Call(String userId, Integer conversationId, Integer turnId, ProviderService.Active active, String purpose)
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private UsageRecorder()
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Build the row of a finished call (response or error, one of them set).
|
||||
*******************************************************************************/
|
||||
public static AssistantUsage row(Call call, ModelResponse response, ModelProviderException error, long latencyMs, ModelPricing pricing)
|
||||
{
|
||||
ModelUsage usage = response != null ? response.getUsage() : (error == null ? null : error.getPartialUsage());
|
||||
String status = error != null ? STATUS_ERROR
|
||||
: (ModelResponse.STOP_CANCELLED.equals(response == null ? null : response.getStopReason()) ? STATUS_CANCELLED : STATUS_OK);
|
||||
|
||||
return (new AssistantUsage()
|
||||
.withUserId(call.userId())
|
||||
.withConversationId(call.conversationId())
|
||||
.withTurnId(call.turnId())
|
||||
.withProviderId(call.active() == null ? null : call.active().providerId())
|
||||
.withProviderType(call.active() == null ? null : call.active().providerType())
|
||||
.withModel(response != null && response.getModel() != null ? response.getModel() : (call.active() == null ? null : call.active().model()))
|
||||
.withPurpose(call.purpose())
|
||||
.withStatus(status)
|
||||
.withErrorKind(error == null ? null : String.valueOf(error.getKind()))
|
||||
.withHttpStatus(error == null ? null : error.getHttpStatus())
|
||||
.withLatencyMs((int) Math.min(Integer.MAX_VALUE, Math.max(0, latencyMs)))
|
||||
.withInputTokens(usage == null ? 0L : nz(usage.getInputTokens()))
|
||||
.withOutputTokens(usage == null ? 0L : nz(usage.getOutputTokens()))
|
||||
.withCacheReadTokens(usage == null ? 0L : nz(usage.getCacheReadTokens()))
|
||||
.withCacheWriteTokens(usage == null ? 0L : nz(usage.getCacheWriteTokens()))
|
||||
.withCostMicros(pricing == null ? 0L : pricing.costMicros(usage))
|
||||
.withRequestId(response == null ? null : truncate(response.getRequestId(), 100)));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Insert a row (under the session in QContext); never throws.
|
||||
*******************************************************************************/
|
||||
public static void save(AssistantStore store, AssistantUsage row)
|
||||
{
|
||||
try
|
||||
{
|
||||
store.insert(row);
|
||||
}
|
||||
catch(QException | RuntimeException e)
|
||||
{
|
||||
LOG.warn("Could not record assistant model-call usage", e, logPair("turnId", row.getTurnId()), logPair("purpose", row.getPurpose()));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private static long nz(Long value)
|
||||
{
|
||||
return (value == null ? 0 : value);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private static String truncate(String value, int max)
|
||||
{
|
||||
return (value == null || value.length() <= max ? value : value.substring(0, max));
|
||||
}
|
||||
|
||||
}
|
||||
@@ -0,0 +1,90 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.home;
|
||||
|
||||
|
||||
import com.kingsrook.qbits.assistant.AssistantRuntime;
|
||||
import com.kingsrook.qbits.assistant.metadata.AssistantMetaDataBuilder;
|
||||
import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
|
||||
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.RawHTML;
|
||||
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The Assistant app home: one full-width HTML widget, the single owner of the
|
||||
** assistant's health, usage and cost data (no other dashboard repeats it).
|
||||
*******************************************************************************/
|
||||
public final class AssistantHome
|
||||
{
|
||||
public static final String WIDGET_NAME = "assistantHome";
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private AssistantHome()
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The widget.
|
||||
*******************************************************************************/
|
||||
public static QWidgetMetaData widget()
|
||||
{
|
||||
return (new QWidgetMetaData()
|
||||
.withName(WIDGET_NAME)
|
||||
.withLabel("Overview")
|
||||
.withType(WidgetType.HTML.getType())
|
||||
.withCodeReference(new QCodeReference(Renderer.class))
|
||||
.withGridColumns(12)
|
||||
.withIsCard(false)
|
||||
.withPermissionRules(AssistantMetaDataBuilder.adminRules()));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Renders the home for the viewing admin.
|
||||
*******************************************************************************/
|
||||
public static class Renderer extends AbstractWidgetRenderer
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Render.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public RenderWidgetOutput render(RenderWidgetInput input) throws QException
|
||||
{
|
||||
AssistantRuntime runtime = AssistantRuntime.of(QContext.getQInstance());
|
||||
if(runtime == null)
|
||||
{
|
||||
return (new RenderWidgetOutput(new RawHTML(null, "<p>The assistant is not running on this server.</p>")));
|
||||
}
|
||||
AssistantHomeData data = AssistantHomeData.load(runtime);
|
||||
return (new RenderWidgetOutput(new RawHTML(null, AssistantHomeView.render(data, runtime.getConfig().getTablePathPrefix()))));
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
+481
@@ -0,0 +1,481 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.home;
|
||||
|
||||
|
||||
import java.time.Instant;
|
||||
import java.time.LocalDate;
|
||||
import java.time.ZoneOffset;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Comparator;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import com.kingsrook.qbits.assistant.AssistantRuntime;
|
||||
import com.kingsrook.qbits.assistant.engine.PolicyService;
|
||||
import com.kingsrook.qbits.assistant.engine.ProviderService;
|
||||
import com.kingsrook.qbits.assistant.engine.UsageRecorder;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantConversation;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantProvider;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantToolCall;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantUsage;
|
||||
import com.kingsrook.qbits.assistant.provider.AssistantProviderType;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Everything the Assistant home shows, computed from the qbit's own tables
|
||||
** (assistant_provider, assistant_usage, assistant_conversation,
|
||||
** assistant_tool_call, assistant_policy) as the viewing admin. Days are UTC.
|
||||
** No sample data: an empty table gives an empty section.
|
||||
*******************************************************************************/
|
||||
public class AssistantHomeData
|
||||
{
|
||||
public static final int DAYS = 30;
|
||||
public static final int TOP = 10;
|
||||
public static final int RECENT = 10;
|
||||
|
||||
private static final Set<String> TOOL_FAILURES = Set.of("DENIED", "NOT_FOUND", "INVALID_INPUT", "ERROR");
|
||||
|
||||
private final List<ProviderHealth> providers = new ArrayList<>();
|
||||
private final Map<LocalDate, DayUsage> days = new LinkedHashMap<>();
|
||||
private final List<Totals> byUser = new ArrayList<>();
|
||||
private final List<Totals> byModel = new ArrayList<>();
|
||||
private final List<ToolTotals> byTool = new ArrayList<>();
|
||||
private final List<QRecord> failedCalls = new ArrayList<>();
|
||||
private final List<QRecord> failedTools = new ArrayList<>();
|
||||
|
||||
private LocalDate today;
|
||||
private long monthCostMicros;
|
||||
private long monthRequests;
|
||||
private Long instanceMonthlyCapMicros;
|
||||
private boolean killSwitch;
|
||||
private int calls7d;
|
||||
private int errors7d;
|
||||
private boolean truncated;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** One provider's health.
|
||||
*******************************************************************************/
|
||||
public record ProviderHealth(AssistantProvider row, String typeLabel, String keyStatus, boolean keyPresent, List<String> problems, boolean active, long monthCostMicros, Long capMicros)
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** One day's usage.
|
||||
*******************************************************************************/
|
||||
public static class DayUsage
|
||||
{
|
||||
public int conversations;
|
||||
public int requests;
|
||||
public int errors;
|
||||
public long tokensIn;
|
||||
public long tokensOut;
|
||||
public long costMicros;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Totals for one key (a user, a model).
|
||||
*******************************************************************************/
|
||||
public static class Totals
|
||||
{
|
||||
public String key;
|
||||
public int requests;
|
||||
public long tokensIn;
|
||||
public long tokensOut;
|
||||
public long costMicros;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Totals for one tool.
|
||||
*******************************************************************************/
|
||||
public static class ToolTotals
|
||||
{
|
||||
public String name;
|
||||
public int calls;
|
||||
public int failures;
|
||||
public long totalMs;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Load it all for the session in QContext.
|
||||
*******************************************************************************/
|
||||
public static AssistantHomeData load(AssistantRuntime runtime) throws QException
|
||||
{
|
||||
AssistantHomeData data = new AssistantHomeData();
|
||||
data.today = LocalDate.now(runtime.getClock().withZone(ZoneOffset.UTC));
|
||||
LocalDate firstDay = data.today.minusDays(DAYS - 1);
|
||||
LocalDate monthStart = data.today.withDayOfMonth(1);
|
||||
LocalDate from = firstDay.isBefore(monthStart) ? firstDay : monthStart;
|
||||
Instant fromAt = from.atStartOfDay().toInstant(ZoneOffset.UTC);
|
||||
Instant monthAt = monthStart.atStartOfDay().toInstant(ZoneOffset.UTC);
|
||||
Instant weekAt = runtime.getClock().instant().minusSeconds(7L * 24 * 3600);
|
||||
int max = runtime.getConfig().getHomeMaxRows();
|
||||
|
||||
for(LocalDate d = firstDay; !d.isAfter(data.today); d = d.plusDays(1))
|
||||
{
|
||||
data.days.put(d, new DayUsage());
|
||||
}
|
||||
|
||||
PolicyService.EffectivePolicy policy = runtime.getPolicyService().current(false);
|
||||
data.killSwitch = policy.killSwitch();
|
||||
data.instanceMonthlyCapMicros = policy.budgets().getInstanceMonthlyCostMicrosHard();
|
||||
|
||||
////////////////////////////////////////
|
||||
// usage rows: by day, user, model //
|
||||
////////////////////////////////////////
|
||||
List<QRecord> usage = runtime.getStore().queryRecords(AssistantUsage.TABLE_NAME, new QQueryFilter()
|
||||
.withCriteria("createDate", QCriteriaOperator.GREATER_THAN_OR_EQUALS, fromAt)
|
||||
.withOrderBy(new QFilterOrderBy("id", false)), max);
|
||||
data.truncated |= usage.size() >= max;
|
||||
|
||||
Map<String, Totals> users = new LinkedHashMap<>();
|
||||
Map<String, Totals> models = new LinkedHashMap<>();
|
||||
Map<Integer, Long> providerMonth = new LinkedHashMap<>();
|
||||
for(QRecord row : usage)
|
||||
{
|
||||
Instant at = row.getValueInstant("createDate");
|
||||
long in = nz(row.getValueLong("inputTokens")) + nz(row.getValueLong("cacheReadTokens")) + nz(row.getValueLong("cacheWriteTokens"));
|
||||
long out = nz(row.getValueLong("outputTokens"));
|
||||
long cost = nz(row.getValueLong("costMicros"));
|
||||
boolean err = UsageRecorder.STATUS_ERROR.equals(row.getValueString("status"));
|
||||
boolean turn = UsageRecorder.PURPOSE_TURN.equals(row.getValueString("purpose"));
|
||||
if(at == null)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
DayUsage day = data.days.get(LocalDate.ofInstant(at, ZoneOffset.UTC));
|
||||
if(day != null)
|
||||
{
|
||||
day.requests++;
|
||||
day.errors += err ? 1 : 0;
|
||||
day.tokensIn += in;
|
||||
day.tokensOut += out;
|
||||
day.costMicros += cost;
|
||||
}
|
||||
|
||||
if(!at.isBefore(weekAt) && turn)
|
||||
{
|
||||
data.calls7d++;
|
||||
data.errors7d += err ? 1 : 0;
|
||||
}
|
||||
|
||||
if(err && data.failedCalls.size() < RECENT)
|
||||
{
|
||||
data.failedCalls.add(row);
|
||||
}
|
||||
|
||||
if(!at.isBefore(monthAt))
|
||||
{
|
||||
data.monthCostMicros += cost;
|
||||
data.monthRequests++;
|
||||
add(users, row.getValueString("userId") == null ? "(none)" : row.getValueString("userId"), in, out, cost);
|
||||
add(models, row.getValueString("model") == null ? "(unknown)" : row.getValueString("model"), in, out, cost);
|
||||
Integer providerId = row.getValueInteger("providerId");
|
||||
if(providerId != null)
|
||||
{
|
||||
providerMonth.merge(providerId, cost, Long::sum);
|
||||
}
|
||||
}
|
||||
}
|
||||
data.byUser.addAll(top(users));
|
||||
data.byModel.addAll(top(models));
|
||||
|
||||
////////////////////////////////////////
|
||||
// conversations started per day //
|
||||
////////////////////////////////////////
|
||||
Instant dayOneAt = firstDay.atStartOfDay().toInstant(ZoneOffset.UTC);
|
||||
List<QRecord> conversations = runtime.getStore().queryRecords(AssistantConversation.TABLE_NAME, new QQueryFilter()
|
||||
.withCriteria("createDate", QCriteriaOperator.GREATER_THAN_OR_EQUALS, dayOneAt), max);
|
||||
data.truncated |= conversations.size() >= max;
|
||||
for(QRecord row : conversations)
|
||||
{
|
||||
Instant at = row.getValueInstant("createDate");
|
||||
DayUsage day = at == null ? null : data.days.get(LocalDate.ofInstant(at, ZoneOffset.UTC));
|
||||
if(day != null)
|
||||
{
|
||||
day.conversations++;
|
||||
}
|
||||
}
|
||||
|
||||
////////////////////////////////////////
|
||||
// tools //
|
||||
////////////////////////////////////////
|
||||
List<QRecord> tools = runtime.getStore().queryRecords(AssistantToolCall.TABLE_NAME, new QQueryFilter()
|
||||
.withCriteria("createDate", QCriteriaOperator.GREATER_THAN_OR_EQUALS, dayOneAt)
|
||||
.withOrderBy(new QFilterOrderBy("id", false)), max);
|
||||
data.truncated |= tools.size() >= max;
|
||||
Map<String, ToolTotals> toolMap = new LinkedHashMap<>();
|
||||
for(QRecord row : tools)
|
||||
{
|
||||
String name = row.getValueString("toolName") == null ? "(unknown)" : row.getValueString("toolName");
|
||||
ToolTotals totals = toolMap.computeIfAbsent(name, k ->
|
||||
{
|
||||
ToolTotals t = new ToolTotals();
|
||||
t.name = k;
|
||||
return (t);
|
||||
});
|
||||
totals.calls++;
|
||||
totals.totalMs += nz(row.getValueLong("durationMs"));
|
||||
if(TOOL_FAILURES.contains(row.getValueString("outcome")))
|
||||
{
|
||||
totals.failures++;
|
||||
if(data.failedTools.size() < RECENT)
|
||||
{
|
||||
data.failedTools.add(row);
|
||||
}
|
||||
}
|
||||
}
|
||||
toolMap.values().stream()
|
||||
.sorted(Comparator.comparingInt((ToolTotals t) -> t.calls).reversed().thenComparing(t -> t.name))
|
||||
.limit(TOP)
|
||||
.forEach(data.byTool::add);
|
||||
|
||||
////////////////////////////////////////
|
||||
// providers //
|
||||
////////////////////////////////////////
|
||||
ProviderService service = runtime.getProviderService();
|
||||
AssistantProvider active = service.activeRow();
|
||||
for(AssistantProvider row : runtime.getStore().listProviders())
|
||||
{
|
||||
String status = service.keyStatus(row);
|
||||
AssistantProviderType type = row.getProviderType() == null ? null : runtime.getConfig().getProviderTypes().get(row.getProviderType());
|
||||
data.providers.add(new ProviderHealth(row, type == null ? row.getProviderType() : type.getLabel(), status, status.startsWith("Set"), service.validate(row), active != null && active.getId().equals(row.getId()),
|
||||
providerMonth.getOrDefault(row.getId(), 0L), ProviderService.capMicros(row)));
|
||||
}
|
||||
return (data);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private static void add(Map<String, Totals> map, String key, long in, long out, long cost)
|
||||
{
|
||||
Totals totals = map.computeIfAbsent(key, k ->
|
||||
{
|
||||
Totals t = new Totals();
|
||||
t.key = k;
|
||||
return (t);
|
||||
});
|
||||
totals.requests++;
|
||||
totals.tokensIn += in;
|
||||
totals.tokensOut += out;
|
||||
totals.costMicros += cost;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The top entries by cost, then requests.
|
||||
*******************************************************************************/
|
||||
private static List<Totals> top(Map<String, Totals> map)
|
||||
{
|
||||
return (map.values().stream()
|
||||
.sorted(Comparator.comparingLong((Totals t) -> t.costMicros).reversed().thenComparing(Comparator.comparingInt((Totals t) -> t.requests).reversed()).thenComparing(t -> t.key))
|
||||
.limit(TOP)
|
||||
.toList());
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private static long nz(Long value)
|
||||
{
|
||||
return (value == null ? 0 : value);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The error rate of turn model calls in the last 7 days (0-100), or null.
|
||||
*******************************************************************************/
|
||||
public Integer errorRatePercent7d()
|
||||
{
|
||||
return (calls7d == 0 ? null : (int) Math.round(errors7d * 100.0 / calls7d));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for providers
|
||||
*******************************************************************************/
|
||||
public List<ProviderHealth> getProviders()
|
||||
{
|
||||
return (providers);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for days (oldest first)
|
||||
*******************************************************************************/
|
||||
public Map<LocalDate, DayUsage> getDays()
|
||||
{
|
||||
return (days);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for byUser
|
||||
*******************************************************************************/
|
||||
public List<Totals> getByUser()
|
||||
{
|
||||
return (byUser);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for byModel
|
||||
*******************************************************************************/
|
||||
public List<Totals> getByModel()
|
||||
{
|
||||
return (byModel);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for byTool
|
||||
*******************************************************************************/
|
||||
public List<ToolTotals> getByTool()
|
||||
{
|
||||
return (byTool);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for failedCalls (newest first)
|
||||
*******************************************************************************/
|
||||
public List<QRecord> getFailedCalls()
|
||||
{
|
||||
return (failedCalls);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for failedTools (newest first)
|
||||
*******************************************************************************/
|
||||
public List<QRecord> getFailedTools()
|
||||
{
|
||||
return (failedTools);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for today
|
||||
*******************************************************************************/
|
||||
public LocalDate getToday()
|
||||
{
|
||||
return (today);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for monthCostMicros
|
||||
*******************************************************************************/
|
||||
public long getMonthCostMicros()
|
||||
{
|
||||
return (monthCostMicros);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for monthRequests
|
||||
*******************************************************************************/
|
||||
public long getMonthRequests()
|
||||
{
|
||||
return (monthRequests);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for instanceMonthlyCapMicros
|
||||
*******************************************************************************/
|
||||
public Long getInstanceMonthlyCapMicros()
|
||||
{
|
||||
return (instanceMonthlyCapMicros);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for killSwitch
|
||||
*******************************************************************************/
|
||||
public boolean isKillSwitch()
|
||||
{
|
||||
return (killSwitch);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for calls7d
|
||||
*******************************************************************************/
|
||||
public int getCalls7d()
|
||||
{
|
||||
return (calls7d);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for errors7d
|
||||
*******************************************************************************/
|
||||
public int getErrors7d()
|
||||
{
|
||||
return (errors7d);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for truncated (a table had more rows than the home reads)
|
||||
*******************************************************************************/
|
||||
public boolean isTruncated()
|
||||
{
|
||||
return (truncated);
|
||||
}
|
||||
|
||||
}
|
||||
+631
@@ -0,0 +1,631 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.home;
|
||||
|
||||
|
||||
import java.math.BigDecimal;
|
||||
import java.math.RoundingMode;
|
||||
import java.text.NumberFormat;
|
||||
import java.time.Instant;
|
||||
import java.time.LocalDate;
|
||||
import java.time.ZoneOffset;
|
||||
import java.time.format.DateTimeFormatter;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Locale;
|
||||
import java.util.Map;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantConversation;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantModelPrice;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantProvider;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantToolCall;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantUsage;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Renders the Assistant home: a status row, one row per provider across the
|
||||
** full width, then cards for usage by day, costs this month, usage by user,
|
||||
** top tools and recent failures. Every card has an honest empty state.
|
||||
** Layout is flow and grid only (no tables, no fixed widths, no inner
|
||||
** scrolling): rows wrap their cells, cards stack to one column on narrow
|
||||
** screens, long names wrap, and colors are inherited (only status dots and
|
||||
** bars use a fixed color), so it reads at every width in light and dark mode.
|
||||
*******************************************************************************/
|
||||
public final class AssistantHomeView
|
||||
{
|
||||
private static final DateTimeFormatter WHEN = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm 'UTC'").withZone(ZoneOffset.UTC);
|
||||
private static final DateTimeFormatter DAY = DateTimeFormatter.ofPattern("MMM d", Locale.US);
|
||||
|
||||
private static final String ROOT = "display:grid;gap:1rem;min-width:0;";
|
||||
private static final String GRID = "display:grid;grid-template-columns:repeat(auto-fit,minmax(min(100%,22rem),1fr));gap:1rem;align-items:start;min-width:0;";
|
||||
private static final String CARD = "border:1px solid rgba(127,127,127,.3);border-radius:8px;padding:1rem;min-width:0;overflow-wrap:anywhere;";
|
||||
private static final String ROW = "border:1px solid rgba(127,127,127,.25);border-radius:6px;padding:.75rem;margin:0 0 .75rem;min-width:0;";
|
||||
private static final String CELLS = "display:grid;grid-template-columns:repeat(auto-fit,minmax(9.5rem,1fr));gap:.6rem 1rem;margin:.6rem 0 0;font-size:.9rem;";
|
||||
private static final String LIST = "display:grid;grid-template-columns:minmax(6rem,auto) 1fr;gap:.25rem .75rem;margin:0 0 .75rem;font-size:.9rem;";
|
||||
private static final String ITEMS = "margin:0 0 .75rem;padding-left:1.25rem;list-style:disc;font-size:.9rem;display:grid;gap:.35rem;";
|
||||
private static final String LINE = "display:flex;flex-wrap:wrap;justify-content:space-between;gap:.15rem 1rem;padding:.35rem 0;border-bottom:1px solid rgba(127,127,127,.2);font-size:.9rem;";
|
||||
private static final String MUTED = "opacity:.75;";
|
||||
private static final String LINK = "color:inherit;text-decoration:underline;text-underline-offset:2px;";
|
||||
private static final String CHIP = "display:inline-flex;align-items:center;gap:.4rem;padding:.05rem .6rem;border-radius:999px;border:1px solid rgba(127,127,127,.4);font-size:.8rem;white-space:nowrap;";
|
||||
private static final String H = "margin:0 0 .75rem;font-size:1rem;font-weight:600;";
|
||||
|
||||
private static final String GREEN = "#2e9d5b";
|
||||
private static final String AMBER = "#c98a12";
|
||||
private static final String RED = "#d0453a";
|
||||
private static final String GREY = "#8a8a8a";
|
||||
private static final String BAR = "#4a7fd4";
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private AssistantHomeView()
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The whole home; prefix is the front of table and record links.
|
||||
*******************************************************************************/
|
||||
public static String render(AssistantHomeData data, String prefix)
|
||||
{
|
||||
StringBuilder html = new StringBuilder("<div class=\"assistant-home\" style=\"" + ROOT + "\">");
|
||||
status(html, data, prefix);
|
||||
providers(html, data, prefix);
|
||||
html.append("<div style=\"").append(GRID).append("\">");
|
||||
usageByDay(html, data, prefix);
|
||||
costs(html, data, prefix);
|
||||
byUser(html, data);
|
||||
tools(html, data, prefix);
|
||||
failures(html, data, prefix);
|
||||
html.append("</div>");
|
||||
if(data.isTruncated())
|
||||
{
|
||||
html.append("<p style=\"").append(MUTED).append("font-size:.85rem;margin:0;\">Some totals are partial: a table had more rows in this period than the home reads.</p>");
|
||||
}
|
||||
return (html.append("</div>").toString());
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The status row: is the assistant answering, with what, at what cost.
|
||||
*******************************************************************************/
|
||||
private static void status(StringBuilder html, AssistantHomeData data, String prefix)
|
||||
{
|
||||
open(html, "Health");
|
||||
AssistantHomeData.ProviderHealth active = data.getProviders().stream().filter(AssistantHomeData.ProviderHealth::active).findFirst().orElse(null);
|
||||
|
||||
html.append("<div style=\"display:flex;flex-wrap:wrap;gap:.5rem .75rem;align-items:center;\">");
|
||||
if(data.isKillSwitch())
|
||||
{
|
||||
html.append(chip(RED, "Turned off by the kill switch"));
|
||||
}
|
||||
else if(data.getProviders().isEmpty())
|
||||
{
|
||||
html.append(chip(GREY, "No provider configured here (the application's built-in provider is used, if any)"));
|
||||
}
|
||||
else if(active == null)
|
||||
{
|
||||
html.append(chip(RED, "No provider is turned on"));
|
||||
}
|
||||
else if(!active.keyPresent() || !active.problems().isEmpty())
|
||||
{
|
||||
html.append(chip(RED, "Default provider cannot be used"));
|
||||
}
|
||||
else if("ERROR".equals(active.row().getLastTestStatus()))
|
||||
{
|
||||
html.append(chip(AMBER, "Last connection test failed"));
|
||||
}
|
||||
else
|
||||
{
|
||||
html.append(chip(GREEN, "Answering"));
|
||||
}
|
||||
html.append("</div><div style=\"").append(CELLS).append("\">");
|
||||
cell(html, "Provider", active == null ? "None" : active.row().getName() + " (" + nvl(active.row().getModel(), "default model") + ")");
|
||||
cell(html, "API key", active == null ? "Not applicable" : active.keyStatus());
|
||||
cell(html, "Last test", active == null ? "Not applicable" : lastTest(active.row()));
|
||||
Integer rate = data.errorRatePercent7d();
|
||||
cell(html, "Error rate, 7 days", rate == null ? "No requests yet" : rate + "% of " + number(data.getCalls7d()) + " requests");
|
||||
cell(html, "Cost this month", money(data.getMonthCostMicros()) + (data.getInstanceMonthlyCapMicros() == null ? "" : " of " + money(data.getInstanceMonthlyCapMicros()) + " limit"));
|
||||
cell(html, "Requests this month", number(data.getMonthRequests()));
|
||||
html.append("</div>");
|
||||
close(html);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** One row per provider.
|
||||
*******************************************************************************/
|
||||
private static void providers(StringBuilder html, AssistantHomeData data, String prefix)
|
||||
{
|
||||
open(html, "Providers");
|
||||
if(data.getProviders().isEmpty())
|
||||
{
|
||||
empty(html, "No providers yet. Add one from Providers, New: pick the type and model, and name the environment variable that holds the API key.");
|
||||
}
|
||||
for(AssistantHomeData.ProviderHealth p : data.getProviders())
|
||||
{
|
||||
AssistantProvider row = p.row();
|
||||
html.append("<div style=\"").append(ROW).append("\"><div style=\"display:flex;flex-wrap:wrap;align-items:center;gap:.5rem .75rem;\">");
|
||||
html.append("<strong>").append(recordLink(prefix, AssistantProvider.TABLE_NAME, row.getId(), row.getName())).append("</strong>");
|
||||
if(p.active())
|
||||
{
|
||||
html.append(chip(GREEN, "In use"));
|
||||
}
|
||||
else if(Boolean.TRUE.equals(row.getIsDefault()))
|
||||
{
|
||||
html.append(chip(GREY, "Default"));
|
||||
}
|
||||
if(!Boolean.TRUE.equals(row.getEnabled()))
|
||||
{
|
||||
html.append(chip(GREY, "Off"));
|
||||
}
|
||||
if(row.getLastTestStatus() != null)
|
||||
{
|
||||
html.append(chip("OK".equals(row.getLastTestStatus()) ? GREEN : RED, "OK".equals(row.getLastTestStatus()) ? "Last test passed" : "Last test failed"));
|
||||
}
|
||||
html.append("</div><div style=\"").append(CELLS).append("\">");
|
||||
cell(html, "Type", nvl(p.typeLabel(), "Not set"));
|
||||
cell(html, "Model", nvl(row.getModel(), "Default model"));
|
||||
cell(html, "API key", p.keyStatus());
|
||||
cell(html, "Last test", lastTest(row));
|
||||
cell(html, "Cost this month", money(p.monthCostMicros()) + (p.capMicros() == null ? ", no cap" : " of " + money(p.capMicros()) + " cap"));
|
||||
List<String> limits = new ArrayList<>();
|
||||
if(row.getMaxOutputTokensPerRequest() != null)
|
||||
{
|
||||
limits.add(number(row.getMaxOutputTokensPerRequest()) + " output tokens per request");
|
||||
}
|
||||
if(row.getMaxConcurrentRequests() != null)
|
||||
{
|
||||
limits.add(row.getMaxConcurrentRequests() + " at a time");
|
||||
}
|
||||
cell(html, "Limits", limits.isEmpty() ? "None" : String.join(", ", limits));
|
||||
html.append("</div>");
|
||||
if(!p.problems().isEmpty())
|
||||
{
|
||||
html.append("<ul style=\"").append(ITEMS).append("margin-top:.6rem;\">");
|
||||
p.problems().forEach(problem -> html.append("<li>").append(dot(RED)).append(esc(problem)).append("</li>"));
|
||||
html.append("</ul>");
|
||||
}
|
||||
html.append("</div>");
|
||||
}
|
||||
html.append("<div style=\"display:flex;flex-wrap:wrap;gap:.5rem 1rem;\">");
|
||||
link(html, prefix, AssistantProvider.TABLE_NAME, "Providers");
|
||||
link(html, prefix, AssistantModelPrice.TABLE_NAME, "Model prices");
|
||||
html.append("</div>");
|
||||
close(html);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Usage by day: bars of requests, then the recent days in words.
|
||||
*******************************************************************************/
|
||||
private static void usageByDay(StringBuilder html, AssistantHomeData data, String prefix)
|
||||
{
|
||||
open(html, "Usage, last " + AssistantHomeData.DAYS + " days");
|
||||
Map<LocalDate, AssistantHomeData.DayUsage> days = data.getDays();
|
||||
long conversations = 0;
|
||||
long requests = 0;
|
||||
long tokensIn = 0;
|
||||
long tokensOut = 0;
|
||||
long cost = 0;
|
||||
long max = 1;
|
||||
for(AssistantHomeData.DayUsage day : days.values())
|
||||
{
|
||||
conversations += day.conversations;
|
||||
requests += day.requests;
|
||||
tokensIn += day.tokensIn;
|
||||
tokensOut += day.tokensOut;
|
||||
cost += day.costMicros;
|
||||
max = Math.max(max, day.requests);
|
||||
}
|
||||
|
||||
if(requests == 0 && conversations == 0)
|
||||
{
|
||||
empty(html, "No assistant use in the last " + AssistantHomeData.DAYS + " days.");
|
||||
}
|
||||
else
|
||||
{
|
||||
html.append("<div role=\"img\" aria-label=\"Model requests per day, last ").append(days.size()).append(" days\" style=\"display:flex;align-items:flex-end;gap:2px;height:3rem;margin:0 0 .25rem;\">");
|
||||
for(Map.Entry<LocalDate, AssistantHomeData.DayUsage> entry : days.entrySet())
|
||||
{
|
||||
AssistantHomeData.DayUsage day = entry.getValue();
|
||||
int pct = (int) Math.max(day.requests == 0 ? 0 : 4, Math.round(day.requests * 100.0 / max));
|
||||
html.append("<span title=\"").append(esc(DAY.format(entry.getKey()) + ": " + day.requests + " requests")).append("\" style=\"flex:1 1 0;min-width:0;height:").append(pct)
|
||||
.append("%;background:").append(day.errors > 0 ? AMBER : BAR).append(";border-radius:2px 2px 0 0;\"></span>");
|
||||
}
|
||||
html.append("</div><div style=\"display:flex;justify-content:space-between;font-size:.75rem;margin:0 0 .75rem;").append(MUTED).append("\"><span>")
|
||||
.append(esc(DAY.format(days.keySet().iterator().next()))).append("</span><span>").append(esc(DAY.format(data.getToday()))).append("</span></div>");
|
||||
|
||||
html.append("<dl style=\"").append(LIST).append("\">");
|
||||
row(html, "Conversations", number(conversations));
|
||||
row(html, "Requests", number(requests));
|
||||
row(html, "Tokens in", number(tokensIn));
|
||||
row(html, "Tokens out", number(tokensOut));
|
||||
row(html, "Cost", money(cost));
|
||||
html.append("</dl>");
|
||||
|
||||
List<LocalDate> recent = new ArrayList<>(days.keySet());
|
||||
recent = recent.subList(Math.max(0, recent.size() - 7), recent.size());
|
||||
html.append("<div style=\"margin:0 0 .75rem;\">");
|
||||
for(int i = recent.size() - 1; i >= 0; i--)
|
||||
{
|
||||
AssistantHomeData.DayUsage day = days.get(recent.get(i));
|
||||
html.append("<div style=\"").append(LINE).append("\"><span>").append(esc(DAY.format(recent.get(i)))).append("</span><span style=\"").append(MUTED).append("\">")
|
||||
.append(esc(day.conversations + " conv, " + day.requests + " req, " + compact(day.tokensIn) + " in, " + compact(day.tokensOut) + " out, " + money(day.costMicros)))
|
||||
.append("</span></div>");
|
||||
}
|
||||
html.append("</div>");
|
||||
}
|
||||
html.append("<div style=\"display:flex;flex-wrap:wrap;gap:.5rem 1rem;\">");
|
||||
link(html, prefix, AssistantUsage.TABLE_NAME, "All model calls");
|
||||
link(html, prefix, AssistantConversation.TABLE_NAME, "Conversations");
|
||||
html.append("</div>");
|
||||
close(html);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Costs this month: total against caps, then by model.
|
||||
*******************************************************************************/
|
||||
private static void costs(StringBuilder html, AssistantHomeData data, String prefix)
|
||||
{
|
||||
open(html, "Costs this month");
|
||||
html.append("<dl style=\"").append(LIST).append("\">");
|
||||
row(html, "Month to date", money(data.getMonthCostMicros()));
|
||||
if(data.getInstanceMonthlyCapMicros() != null)
|
||||
{
|
||||
row(html, "Monthly limit", money(data.getInstanceMonthlyCapMicros()) + " (" + percent(data.getMonthCostMicros(), data.getInstanceMonthlyCapMicros()) + " used)");
|
||||
}
|
||||
for(AssistantHomeData.ProviderHealth p : data.getProviders())
|
||||
{
|
||||
if(p.capMicros() != null)
|
||||
{
|
||||
row(html, p.row().getName() + " cap", money(p.capMicros()) + " (" + percent(p.monthCostMicros(), p.capMicros()) + " used)");
|
||||
}
|
||||
}
|
||||
html.append("</dl>");
|
||||
if(data.getByModel().isEmpty())
|
||||
{
|
||||
empty(html, "No model calls this month.");
|
||||
}
|
||||
else
|
||||
{
|
||||
html.append("<div style=\"margin:0 0 .75rem;\">");
|
||||
for(AssistantHomeData.Totals t : data.getByModel())
|
||||
{
|
||||
html.append("<div style=\"").append(LINE).append("\"><span>").append(esc(t.key)).append("</span><span>")
|
||||
.append(esc(money(t.costMicros) + ", " + number(t.requests) + " req, " + compact(t.tokensIn) + " in, " + compact(t.tokensOut) + " out")).append("</span></div>");
|
||||
}
|
||||
html.append("</div>");
|
||||
}
|
||||
html.append("<p style=\"").append(MUTED).append("font-size:.85rem;margin:0 0 .5rem;\">Cost is tokens times the model's price.</p>");
|
||||
link(html, prefix, AssistantModelPrice.TABLE_NAME, "Model prices");
|
||||
close(html);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Usage by user this month.
|
||||
*******************************************************************************/
|
||||
private static void byUser(StringBuilder html, AssistantHomeData data)
|
||||
{
|
||||
open(html, "By user, this month");
|
||||
if(data.getByUser().isEmpty())
|
||||
{
|
||||
empty(html, "No one has used the assistant this month.");
|
||||
}
|
||||
for(AssistantHomeData.Totals t : data.getByUser())
|
||||
{
|
||||
html.append("<div style=\"").append(LINE).append("\"><span>").append(esc(t.key)).append("</span><span>")
|
||||
.append(esc(money(t.costMicros) + ", " + number(t.requests) + " req, " + compact(t.tokensIn + t.tokensOut) + " tokens")).append("</span></div>");
|
||||
}
|
||||
close(html);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Top tools.
|
||||
*******************************************************************************/
|
||||
private static void tools(StringBuilder html, AssistantHomeData data, String prefix)
|
||||
{
|
||||
open(html, "Top tools, last " + AssistantHomeData.DAYS + " days");
|
||||
if(data.getByTool().isEmpty())
|
||||
{
|
||||
empty(html, "No tool calls yet.");
|
||||
}
|
||||
else
|
||||
{
|
||||
html.append("<div style=\"margin:0 0 .75rem;\">");
|
||||
for(AssistantHomeData.ToolTotals t : data.getByTool())
|
||||
{
|
||||
html.append("<div style=\"").append(LINE).append("\"><span>").append(esc(t.name)).append("</span><span>")
|
||||
.append(esc(number(t.calls) + " calls" + (t.failures == 0 ? "" : ", " + number(t.failures) + " failed") + ", " + (t.totalMs / Math.max(1, t.calls)) + " ms avg"))
|
||||
.append("</span></div>");
|
||||
}
|
||||
html.append("</div>");
|
||||
}
|
||||
link(html, prefix, AssistantToolCall.TABLE_NAME, "All tool calls");
|
||||
close(html);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Recent failures: model calls and tool calls.
|
||||
*******************************************************************************/
|
||||
private static void failures(StringBuilder html, AssistantHomeData data, String prefix)
|
||||
{
|
||||
open(html, "Recent failures");
|
||||
if(data.getFailedCalls().isEmpty() && data.getFailedTools().isEmpty())
|
||||
{
|
||||
empty(html, "No failed model calls or tool calls in the last " + AssistantHomeData.DAYS + " days.");
|
||||
}
|
||||
else
|
||||
{
|
||||
html.append("<ul style=\"").append(ITEMS).append("\">");
|
||||
for(QRecord call : data.getFailedCalls())
|
||||
{
|
||||
html.append("<li>").append(recordLink(prefix, AssistantUsage.TABLE_NAME, call.getValue("id"), "Model call #" + call.getValue("id"))).append(": ")
|
||||
.append(esc(nvl(call.getValueString("errorKind"), "Error") + (call.getValueInteger("httpStatus") == null ? "" : " (HTTP " + call.getValueInteger("httpStatus") + ")")
|
||||
+ ", " + nvl(call.getValueString("model"), "unknown model") + ", " + when(call.getValueInstant("createDate"))))
|
||||
.append("</li>");
|
||||
}
|
||||
for(QRecord tool : data.getFailedTools())
|
||||
{
|
||||
html.append("<li>").append(recordLink(prefix, AssistantToolCall.TABLE_NAME, tool.getValue("id"), nvl(tool.getValueString("toolName"), "Tool call"))).append(": ")
|
||||
.append(esc(label(tool.getValueString("outcome")) + ", " + when(tool.getValueInstant("createDate")))).append("</li>");
|
||||
}
|
||||
html.append("</ul>");
|
||||
}
|
||||
close(html);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A provider's last test in words.
|
||||
*******************************************************************************/
|
||||
static String lastTest(AssistantProvider row)
|
||||
{
|
||||
if(row.getLastTestAt() == null)
|
||||
{
|
||||
return ("Not tested yet: run Test connection");
|
||||
}
|
||||
if("OK".equals(row.getLastTestStatus()))
|
||||
{
|
||||
return ("Passed " + WHEN.format(row.getLastTestAt()) + ", " + nvl(row.getLastTestModel(), "model") + " in " + row.getLastTestLatencyMs() + " ms");
|
||||
}
|
||||
return ("Failed " + WHEN.format(row.getLastTestAt()) + ": " + nvl(row.getLastTestError(), "no detail"));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Micro-dollars as dollars ("$0.00", "<$0.01" for a little, "$12.34").
|
||||
*******************************************************************************/
|
||||
public static String money(long micros)
|
||||
{
|
||||
if(micros > 0 && micros < 10_000)
|
||||
{
|
||||
return ("<$0.01");
|
||||
}
|
||||
NumberFormat format = NumberFormat.getNumberInstance(Locale.US);
|
||||
format.setMinimumFractionDigits(2);
|
||||
format.setMaximumFractionDigits(2);
|
||||
return ("$" + format.format(BigDecimal.valueOf(micros).movePointLeft(6).setScale(2, RoundingMode.HALF_UP)));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private static String percent(long value, long of)
|
||||
{
|
||||
return (of <= 0 ? "n/a" : Math.round(value * 100.0 / of) + "%");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
static String number(long value)
|
||||
{
|
||||
return (NumberFormat.getIntegerInstance(Locale.US).format(value));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** 1234567 as 1.2M, 4500 as 4.5k.
|
||||
*******************************************************************************/
|
||||
static String compact(long value)
|
||||
{
|
||||
if(value >= 1_000_000)
|
||||
{
|
||||
return (BigDecimal.valueOf(value).movePointLeft(6).setScale(1, RoundingMode.HALF_UP).stripTrailingZeros().toPlainString() + "M");
|
||||
}
|
||||
if(value >= 1_000)
|
||||
{
|
||||
return (BigDecimal.valueOf(value).movePointLeft(3).setScale(1, RoundingMode.HALF_UP).stripTrailingZeros().toPlainString() + "k");
|
||||
}
|
||||
return (String.valueOf(value));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private static String when(Instant instant)
|
||||
{
|
||||
return (instant == null ? "time unknown" : WHEN.format(instant));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** SOME_STATUS to Some status.
|
||||
*******************************************************************************/
|
||||
private static String label(String value)
|
||||
{
|
||||
if(value == null)
|
||||
{
|
||||
return ("Unknown");
|
||||
}
|
||||
String lower = value.replace('_', ' ').toLowerCase(Locale.ROOT);
|
||||
return (Character.toUpperCase(lower.charAt(0)) + lower.substring(1));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private static void open(StringBuilder html, String title)
|
||||
{
|
||||
html.append("<section style=\"").append(CARD).append("\"><h3 style=\"").append(H).append("\">").append(esc(title)).append("</h3>");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private static void close(StringBuilder html)
|
||||
{
|
||||
html.append("</section>");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private static void empty(StringBuilder html, String text)
|
||||
{
|
||||
html.append("<p style=\"").append(MUTED).append("margin:0 0 .75rem;font-size:.9rem;\">").append(esc(text)).append("</p>");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private static void cell(StringBuilder html, String label, String value)
|
||||
{
|
||||
html.append("<div style=\"min-width:0;\"><div style=\"").append(MUTED).append("font-size:.8rem;\">").append(esc(label)).append("</div><div>").append(esc(value)).append("</div></div>");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private static void row(StringBuilder html, String label, String value)
|
||||
{
|
||||
html.append("<dt style=\"").append(MUTED).append("\">").append(esc(label)).append("</dt><dd style=\"margin:0;min-width:0;\">").append(esc(value)).append("</dd>");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private static String chip(String color, String text)
|
||||
{
|
||||
return ("<span style=\"" + CHIP + "\">" + dot(color) + esc(text) + "</span>");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private static String dot(String color)
|
||||
{
|
||||
return ("<span aria-hidden=\"true\" style=\"display:inline-block;width:.55rem;height:.55rem;border-radius:50%;margin-right:.35rem;background:" + color + ";\"></span>");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private static void link(StringBuilder html, String prefix, String table, String text)
|
||||
{
|
||||
html.append("<a style=\"").append(LINK).append("font-size:.9rem;\" href=\"").append(esc(prefix + "/" + table)).append("\">").append(esc(text)).append("</a>");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private static String recordLink(String prefix, String table, Object id, String text)
|
||||
{
|
||||
return ("<a style=\"" + LINK + "\" href=\"" + esc(prefix + "/" + table + "/" + id) + "\">" + esc(text == null ? String.valueOf(id) : text) + "</a>");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private static String nvl(String value, String fallback)
|
||||
{
|
||||
return (value == null || value.isBlank() ? fallback : value);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** HTML-escape.
|
||||
*******************************************************************************/
|
||||
static String esc(String value)
|
||||
{
|
||||
if(value == null)
|
||||
{
|
||||
return ("");
|
||||
}
|
||||
StringBuilder out = new StringBuilder(value.length());
|
||||
for(char c : value.toCharArray())
|
||||
{
|
||||
switch(c)
|
||||
{
|
||||
case '&' -> out.append("&");
|
||||
case '<' -> out.append("<");
|
||||
case '>' -> out.append(">");
|
||||
case '"' -> out.append(""");
|
||||
case '\'' -> out.append("'");
|
||||
default -> out.append(c);
|
||||
}
|
||||
}
|
||||
return (out.toString());
|
||||
}
|
||||
|
||||
}
|
||||
+59
-12
@@ -22,13 +22,20 @@ import java.util.List;
|
||||
import com.kingsrook.qbits.assistant.AssistantQBitConfig;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantConversation;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantFeedback;
|
||||
import com.kingsrook.qbits.assistant.home.AssistantHome;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantMessage;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantModelPrice;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantPolicy;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantProvider;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantToolCall;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantTurn;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantTurnEvent;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantUsage;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantUsageDaily;
|
||||
import com.kingsrook.qbits.assistant.processes.AssistantRetentionStep;
|
||||
import com.kingsrook.qqq.backend.core.actions.ai.AiExposure;
|
||||
import com.kingsrook.qqq.backend.core.actions.ai.AiTableMetaData;
|
||||
import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizers;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
|
||||
@@ -55,7 +62,7 @@ import com.kingsrook.qqq.backend.core.utils.StringUtils;
|
||||
** Builds the QBit's metadata (spec section 4, owner decision D6): standard,
|
||||
** visible QQQ tables on the configured backend, the owner security key type,
|
||||
** the conversation possible-value source, the retention process and the
|
||||
** "Assistant Admin" app.
|
||||
** "Assistant" app (its home plus the main tables).
|
||||
**
|
||||
** Security:
|
||||
** - Every user-owned table carries userId and a record security lock on it
|
||||
@@ -84,6 +91,14 @@ public class AssistantMetaDataBuilder
|
||||
@Deprecated(since = "0.1.0-RC.3")
|
||||
public static final String LEGACY_ALL_ACCESS_SUFFIX = "AllAccess";
|
||||
|
||||
/*******************************************************************************
|
||||
** The tables in the app's nav, in order (the main tables). The others are
|
||||
** child lists on these (messages, turns, tool calls, feedback and model
|
||||
** calls on a conversation; model calls on a provider) or reached from the
|
||||
** home. Processes are table and record actions, never nav entries.
|
||||
*******************************************************************************/
|
||||
public static final List<String> NAV_TABLES = List.of(AssistantProvider.TABLE_NAME, AssistantConversation.TABLE_NAME, AssistantModelPrice.TABLE_NAME, AssistantPolicy.TABLE_NAME);
|
||||
|
||||
private final AssistantQBitConfig config;
|
||||
|
||||
|
||||
@@ -154,12 +169,41 @@ public class AssistantMetaDataBuilder
|
||||
.withUniqueKey(new UniqueKey("scope", "userId", "usageDate"))
|
||||
.withRecordLabelFormat("%s %s").withRecordLabelFields("scope", "usageDate"));
|
||||
|
||||
tables.add(settings(base(AssistantProvider.TABLE_NAME, "Assistant Provider", "settings_input_component", AssistantProvider.class))
|
||||
.withUniqueKey(new UniqueKey("name"))
|
||||
.withRecordLabelFormat("%s").withRecordLabelFields("name"));
|
||||
|
||||
tables.add(settings(base(AssistantModelPrice.TABLE_NAME, "Assistant Model Price", "sell", AssistantModelPrice.class))
|
||||
.withUniqueKey(new UniqueKey("providerType", "model"))
|
||||
.withRecordLabelFormat("%s").withRecordLabelFields("model"));
|
||||
|
||||
tables.add(readOnly(owned(base(AssistantUsage.TABLE_NAME, "Assistant Model Call", "receipt_long", AssistantUsage.class), false))
|
||||
.withRecordLabelFormat("%s #%s").withRecordLabelFields("purpose", "id"));
|
||||
|
||||
tables.forEach(AssistantRecordScreens::apply);
|
||||
return (tables);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A settings table: the customizer checks rows before a save, masks the
|
||||
** key status after a read and drops the provider caches after a change.
|
||||
*******************************************************************************/
|
||||
private static QTableMetaData settings(QTableMetaData table)
|
||||
{
|
||||
QCodeReference customizer = new QCodeReference(AssistantSettingsCustomizer.class);
|
||||
return (table
|
||||
.withCustomizer(TableCustomizers.POST_QUERY_RECORD, customizer)
|
||||
.withCustomizer(TableCustomizers.PRE_INSERT_RECORD, customizer)
|
||||
.withCustomizer(TableCustomizers.PRE_UPDATE_RECORD, customizer)
|
||||
.withCustomizer(TableCustomizers.POST_INSERT_RECORD, customizer)
|
||||
.withCustomizer(TableCustomizers.POST_UPDATE_RECORD, customizer)
|
||||
.withCustomizer(TableCustomizers.POST_DELETE_RECORD, customizer));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Possible value source over conversations (for the conversationId fields).
|
||||
*******************************************************************************/
|
||||
@@ -180,6 +224,7 @@ public class AssistantMetaDataBuilder
|
||||
.withName(RETENTION_PROCESS_NAME)
|
||||
.withLabel("Assistant Retention")
|
||||
.withIcon(new QIcon("auto_delete"))
|
||||
.withTableName(AssistantConversation.TABLE_NAME)
|
||||
.withPermissionRules(adminRules())
|
||||
.withStep(new QBackendStepMetaData()
|
||||
.withName("purge")
|
||||
@@ -197,22 +242,19 @@ public class AssistantMetaDataBuilder
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The Assistant Admin app over the tables and the retention process.
|
||||
** The Assistant app: the home widget, and the main tables in the nav.
|
||||
*******************************************************************************/
|
||||
public QAppMetaData buildAdminApp(List<QTableMetaData> tables, QProcessMetaData retention)
|
||||
public QAppMetaData buildAdminApp(List<QTableMetaData> tables)
|
||||
{
|
||||
QAppMetaData app = new QAppMetaData()
|
||||
.withName(APP_NAME)
|
||||
.withLabel("Assistant Admin")
|
||||
.withLabel("Assistant")
|
||||
.withIcon(new QIcon("smart_toy"))
|
||||
.withPermissionRules(adminRules());
|
||||
for(QTableMetaData table : tables)
|
||||
.withPermissionRules(adminRules())
|
||||
.withWidgets(List.of(AssistantHome.WIDGET_NAME));
|
||||
for(String name : NAV_TABLES)
|
||||
{
|
||||
app.withChild(table);
|
||||
}
|
||||
if(retention != null)
|
||||
{
|
||||
app.withChild(retention);
|
||||
tables.stream().filter(t -> t.getName().equals(name)).findFirst().ifPresent(app::withChild);
|
||||
}
|
||||
return (app);
|
||||
}
|
||||
@@ -224,7 +266,7 @@ public class AssistantMetaDataBuilder
|
||||
** are explicit: left null, the enricher copies the host's default level
|
||||
** (often NOT_PROTECTED), and the meta-data path (app tree, table list) then
|
||||
** allows the object without ever calling the checker, so every user would
|
||||
** see the Assistant Admin app and its tables in the nav.
|
||||
** see the Assistant app and its tables in the nav.
|
||||
*******************************************************************************/
|
||||
public static QPermissionRules adminRules()
|
||||
{
|
||||
@@ -250,6 +292,11 @@ public class AssistantMetaDataBuilder
|
||||
.withPermissionRules(adminRules())
|
||||
.withFieldsFromEntity(entity);
|
||||
|
||||
//////////////////////////////////////////////////////////////////
|
||||
// the assistant never reads its own tables (settings, audit) //
|
||||
//////////////////////////////////////////////////////////////////
|
||||
AiTableMetaData.ofOrWithNew(table).withExposure(AiExposure.NONE);
|
||||
|
||||
QFieldMetaData conversationId = table.getFields().get("conversationId");
|
||||
if(conversationId != null)
|
||||
{
|
||||
|
||||
+164
-3
@@ -22,14 +22,20 @@ import java.util.LinkedHashSet;
|
||||
import java.util.List;
|
||||
import java.util.Set;
|
||||
import com.kingsrook.qbits.assistant.AssistantModelRouting;
|
||||
import com.kingsrook.qbits.assistant.AssistantQBitConfig;
|
||||
import com.kingsrook.qbits.assistant.engine.ProviderService;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantConversation;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantFeedback;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantMessage;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantModelPrice;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantPolicy;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantProvider;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantToolCall;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantTurn;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantTurnEvent;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantUsage;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantUsageDaily;
|
||||
import com.kingsrook.qbits.assistant.provider.AssistantProviderType;
|
||||
import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.ChildRecordListRenderer;
|
||||
import com.kingsrook.qqq.backend.core.instances.QInstanceEnricher;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
|
||||
@@ -74,7 +80,11 @@ public final class AssistantRecordScreens
|
||||
static final String MICROS_HELP = "In micro-dollars: 1,000,000 is $1.00.";
|
||||
static final String CHILD_LIST_PREFIX = "conversation";
|
||||
|
||||
private static final List<String> DATES = List.of("createDate", "modifyDate");
|
||||
public static final String PROVIDER_TYPE_PVS_NAME = "assistant_provider_type";
|
||||
public static final String MODEL_PVS_NAME = "assistant_model";
|
||||
|
||||
private static final List<String> DATES = List.of("createDate", "modifyDate");
|
||||
private static final List<String> USAGE_LIST_FIELDS = List.of("purpose", "status", "model", "inputTokens", "outputTokens", "costMicros", "latencyMs", "createDate");
|
||||
|
||||
|
||||
|
||||
@@ -126,6 +136,9 @@ public final class AssistantRecordScreens
|
||||
case AssistantFeedback.TABLE_NAME -> feedback(table);
|
||||
case AssistantPolicy.TABLE_NAME -> policy(table);
|
||||
case AssistantUsageDaily.TABLE_NAME -> usageDaily(table);
|
||||
case AssistantProvider.TABLE_NAME -> provider(table);
|
||||
case AssistantModelPrice.TABLE_NAME -> price(table);
|
||||
case AssistantUsage.TABLE_NAME -> usage(table);
|
||||
default ->
|
||||
{
|
||||
// not an assistant table
|
||||
@@ -151,6 +164,143 @@ public final class AssistantRecordScreens
|
||||
childList(qInstance, conversation, AssistantTurn.TABLE_NAME, "Turns", "sync_alt", List.of("status", "model", "toolCallCount", "costMicros", "startedAt"));
|
||||
childList(qInstance, conversation, AssistantToolCall.TABLE_NAME, "Tool Calls", "build", List.of("toolName", "outcome", "rowCount", "durationMs", "createDate"));
|
||||
childList(qInstance, conversation, AssistantFeedback.TABLE_NAME, "Feedback", "thumbs_up_down", List.of("rating", "reason", "createDate"));
|
||||
childList(qInstance, conversation, AssistantUsage.TABLE_NAME, "Model Calls", "receipt_long", USAGE_LIST_FIELDS);
|
||||
|
||||
QTableMetaData provider = qInstance.getTable(AssistantProvider.TABLE_NAME);
|
||||
if(provider != null)
|
||||
{
|
||||
childList(qInstance, provider, AssistantUsage.TABLE_NAME, "Model Calls", "receipt_long", USAGE_LIST_FIELDS, "providerId", "provider");
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Possible-value sources of the settings tables: the registered provider
|
||||
** types, the models they offer (plus any priced model), and the providers.
|
||||
*******************************************************************************/
|
||||
public static List<QPossibleValueSource> buildSettingsPossibleValueSources(AssistantQBitConfig config)
|
||||
{
|
||||
List<QPossibleValue<?>> types = new ArrayList<>();
|
||||
Set<String> models = new LinkedHashSet<>();
|
||||
for(AssistantProviderType type : config.getProviderTypes().values())
|
||||
{
|
||||
types.add(new QPossibleValue<>(type.getType(), type.getLabel()));
|
||||
models.addAll(type.getModels());
|
||||
}
|
||||
if(config.getPricing() != null)
|
||||
{
|
||||
models.addAll(config.getPricing().keySet());
|
||||
}
|
||||
if(types.isEmpty())
|
||||
{
|
||||
////////////////////////////////////////////////////////////////////
|
||||
// no type registered: offer the configured provider's name, so //
|
||||
// the field still has a value (rows of it cannot be used) //
|
||||
////////////////////////////////////////////////////////////////////
|
||||
types.add(new QPossibleValue<>(config.getProviderName(), config.getProviderName()));
|
||||
}
|
||||
List<QPossibleValue<?>> modelValues = new ArrayList<>();
|
||||
models.forEach(m -> modelValues.add(new QPossibleValue<>(m, m)));
|
||||
|
||||
return (List.of(
|
||||
new QPossibleValueSource().withName(PROVIDER_TYPE_PVS_NAME).withType(QPossibleValueSourceType.ENUM).withIdType(QFieldType.STRING)
|
||||
.withValueFormatAndFields(PVSValueFormatAndFields.LABEL_ONLY).withEnumValues(types),
|
||||
new QPossibleValueSource().withName(MODEL_PVS_NAME).withType(QPossibleValueSourceType.ENUM).withIdType(QFieldType.STRING)
|
||||
.withValueFormatAndFields(PVSValueFormatAndFields.LABEL_ONLY).withEnumValues(modelValues),
|
||||
QPossibleValueSource.newForTable(AssistantProvider.TABLE_NAME)));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The provider settings screen.
|
||||
*******************************************************************************/
|
||||
private static void provider(QTableMetaData table)
|
||||
{
|
||||
pvs(table, "providerType", PROVIDER_TYPE_PVS_NAME, "Type");
|
||||
pvs(table, "model", MODEL_PVS_NAME, "Model");
|
||||
table.getField("isDefault").withLabel("Default");
|
||||
table.getField("apiKeyEnvName").withLabel("API Key Variable");
|
||||
table.getField("baseUrl").withLabel("Base URL Override");
|
||||
table.getField("monthlyCostCapUsd").withLabel("Monthly Cost Cap (USD)").withDisplayFormat(DisplayFormat.CURRENCY);
|
||||
counts(table, "maxOutputTokensPerRequest", "lastTestLatencyMs");
|
||||
table.getField("lastTestLatencyMs").withLabel("Last Test Latency (ms)");
|
||||
help(table, "name", "A name for this provider, e.g. Claude (production).");
|
||||
help(table, "providerType", "Which kind of provider this is. Only the types installed on this server are offered.");
|
||||
help(table, "model", "The model answers come from. Empty uses the policy's (or the application's) main model.");
|
||||
help(table, "enabled", "Off: the assistant never uses this provider.");
|
||||
help(table, "isDefault", "The assistant uses the default enabled provider (else the first enabled one). Marking one default unmarks the others.");
|
||||
help(table, "apiKeyEnvName", "The NAME of the environment variable that holds the API key (e.g. ANTHROPIC_API_KEY), never the key itself. "
|
||||
+ "The key stays in the server's secret store; only allowed variable names are accepted. Empty uses the type's default variable. To replace the key, change it in the secret store.");
|
||||
help(table, "baseUrl", "Leave empty to use the provider's own API. Other URLs must be allowed in the server's configuration.");
|
||||
help(table, "monthlyCostCapUsd", "When this provider's cost this month (UTC) reaches the cap, new requests are refused. Empty: no cap.");
|
||||
help(table, "maxOutputTokensPerRequest", "The most output tokens one model call may produce. Empty uses the configured limit.");
|
||||
help(table, "maxConcurrentRequests", "How many model calls may run at the same time on this server. Empty: no limit.");
|
||||
table.getField(ProviderService.KEY_STATUS_FIELD).withLabel("API Key").withHelpContent(new QHelpContent("Whether the key variable is set on this server, with the key's last four characters. The key itself is never shown or returned."));
|
||||
sections(table,
|
||||
section("provider", "Provider", "settings_input_component", Tier.T1, "id", "name", "providerType", "model", "enabled", "isDefault"),
|
||||
section("connection", "Connection", "key", Tier.T2, "apiKeyEnvName", ProviderService.KEY_STATUS_FIELD, "baseUrl"),
|
||||
section("limits", "Limits", "speed", Tier.T2, "monthlyCostCapUsd", "maxOutputTokensPerRequest", "maxConcurrentRequests"),
|
||||
section("lastTest", "Last connection test", "network_check", Tier.T2, "lastTestAt", "lastTestStatus", "lastTestLatencyMs", "lastTestModel", "lastTestError"));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The price table.
|
||||
*******************************************************************************/
|
||||
private static void price(QTableMetaData table)
|
||||
{
|
||||
pvs(table, "providerType", PROVIDER_TYPE_PVS_NAME, "Provider Type");
|
||||
for(String field : List.of("inputUsdPerMTok", "outputUsdPerMTok", "cacheWrite5mUsdPerMTok", "cacheWrite1hUsdPerMTok"))
|
||||
{
|
||||
help(table, field, "USD per million tokens.");
|
||||
}
|
||||
table.getField("inputUsdPerMTok").withLabel("Input (USD / MTok)");
|
||||
table.getField("outputUsdPerMTok").withLabel("Output (USD / MTok)");
|
||||
table.getField("cacheReadUsdPerMTok").withLabel("Cache Read (USD / MTok)");
|
||||
table.getField("cacheWrite5mUsdPerMTok").withLabel("Cache Write 5m (USD / MTok)");
|
||||
table.getField("cacheWrite1hUsdPerMTok").withLabel("Cache Write 1h (USD / MTok)");
|
||||
help(table, "model", "The exact model id, e.g. claude-sonnet-5-5.");
|
||||
help(table, "cacheReadUsdPerMTok", "USD per million tokens. Empty: 0.1 x the input price.");
|
||||
sections(table,
|
||||
section("price", "Model", "sell", Tier.T1, "id", "providerType", "model"),
|
||||
section("prices", "Prices", "payments", Tier.T2, "inputUsdPerMTok", "outputUsdPerMTok", "cacheReadUsdPerMTok", "cacheWrite5mUsdPerMTok", "cacheWrite1hUsdPerMTok"),
|
||||
section("notes", "Notes", "notes", Tier.T3, "notes"));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** One model call.
|
||||
*******************************************************************************/
|
||||
private static void usage(QTableMetaData table)
|
||||
{
|
||||
link(table, "turnId", AssistantTurn.TABLE_NAME, "Turn");
|
||||
link(table, "providerId", AssistantProvider.TABLE_NAME, "Provider");
|
||||
micros(table, "costMicros");
|
||||
counts(table, "inputTokens", "outputTokens", "cacheReadTokens", "cacheWriteTokens", "latencyMs");
|
||||
table.getField("latencyMs").withLabel("Latency (ms)");
|
||||
help(table, "purpose", "TURN: a step of an answer. TEST: a Test connection call.");
|
||||
sections(table,
|
||||
section("call", "Model Call", "receipt_long", Tier.T1, "id", "purpose", "status", "model", "providerId", "providerType", "userId", "conversationId", "turnId"),
|
||||
section("result", "Result", "fact_check", Tier.T2, "latencyMs", "errorKind", "httpStatus", "requestId"),
|
||||
section("usage", "Tokens and cost", "query_stats", Tier.T2, "inputTokens", "outputTokens", "cacheReadTokens", "cacheWriteTokens", "costMicros"));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A field picked from a possible-value source.
|
||||
*******************************************************************************/
|
||||
private static void pvs(QTableMetaData table, String fieldName, String pvsName, String label)
|
||||
{
|
||||
QFieldMetaData field = table.getFields().get(fieldName);
|
||||
if(field != null)
|
||||
{
|
||||
field.withPossibleValueSourceName(pvsName).withLabel(label);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -457,6 +607,17 @@ public final class AssistantRecordScreens
|
||||
** children on the conversation's view (read-only).
|
||||
*******************************************************************************/
|
||||
private static void childList(QInstance qInstance, QTableMetaData conversation, String childTableName, String label, String icon, List<String> fields)
|
||||
{
|
||||
childList(qInstance, conversation, childTableName, label, icon, fields, "conversationId", CHILD_LIST_PREFIX);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Join a parent to a child table on one of the child's fields and list the
|
||||
** children on the parent's view (read-only).
|
||||
*******************************************************************************/
|
||||
private static void childList(QInstance qInstance, QTableMetaData conversation, String childTableName, String label, String icon, List<String> fields, String joinField, String sectionPrefix)
|
||||
{
|
||||
if(qInstance.getTable(childTableName) == null)
|
||||
{
|
||||
@@ -468,7 +629,7 @@ public final class AssistantRecordScreens
|
||||
.withRightTable(childTableName)
|
||||
.withInferredName()
|
||||
.withType(JoinType.ONE_TO_MANY)
|
||||
.withJoinOn(new JoinOn("id", "conversationId"));
|
||||
.withJoinOn(new JoinOn("id", joinField));
|
||||
if(qInstance.getJoin(join.getName()) != null)
|
||||
{
|
||||
return;
|
||||
@@ -484,7 +645,7 @@ public final class AssistantRecordScreens
|
||||
.withOnlyIncludeFieldNames(shown)
|
||||
.getWidgetMetaData());
|
||||
|
||||
String sectionName = CHILD_LIST_PREFIX + label.replace(" ", "");
|
||||
String sectionName = sectionPrefix + label.replace(" ", "");
|
||||
conversation.addSection(new QFieldSection(sectionName, new QIcon(icon), Tier.T2).withLabel(label).withWidgetName(join.getName()));
|
||||
}
|
||||
}
|
||||
+211
@@ -0,0 +1,211 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.metadata;
|
||||
|
||||
|
||||
import java.io.Serializable;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Optional;
|
||||
import com.kingsrook.qbits.assistant.AssistantRuntime;
|
||||
import com.kingsrook.qbits.assistant.engine.ProviderService;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantProvider;
|
||||
import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.QueryOrGetInputInterface;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.statusmessages.BadInputStatusMessage;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Customizer of the settings tables (assistant_provider, assistant_model_price).
|
||||
**
|
||||
** assistant_provider:
|
||||
** - Before a save, the row is checked (ProviderService.validate). Above all,
|
||||
** apiKeyEnvName must be the NAME of an allowed environment variable, so a
|
||||
** key pasted into it by mistake is refused and never stored.
|
||||
** - After a save, a row marked default makes every other row non-default.
|
||||
** - After a read, the virtual field apiKeyStatus says whether the key is set,
|
||||
** with its last four characters at most. No read path returns the key.
|
||||
**
|
||||
** Both tables: a change drops the cached provider rows and prices.
|
||||
*******************************************************************************/
|
||||
public class AssistantSettingsCustomizer implements TableCustomizerInterface
|
||||
{
|
||||
|
||||
/*******************************************************************************
|
||||
** Add the masked key status to provider rows.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public List<QRecord> postQuery(QueryOrGetInputInterface queryInput, List<QRecord> records) throws QException
|
||||
{
|
||||
ProviderService service = service();
|
||||
if(service == null || !AssistantProvider.TABLE_NAME.equals(queryInput.getTableName()))
|
||||
{
|
||||
return (records);
|
||||
}
|
||||
for(QRecord record : records)
|
||||
{
|
||||
record.setValue(ProviderService.KEY_STATUS_FIELD, service.keyStatus(new AssistantProvider(record)));
|
||||
}
|
||||
return (records);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Check provider rows before insert and update.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public List<QRecord> preInsertOrUpdate(AbstractActionInput input, List<QRecord> records, boolean isPreview, Optional<List<QRecord>> oldRecordList) throws QException
|
||||
{
|
||||
ProviderService service = service();
|
||||
if(service == null || !AssistantProvider.TABLE_NAME.equals(tableName(input)))
|
||||
{
|
||||
return (records);
|
||||
}
|
||||
|
||||
Optional<Map<Serializable, QRecord>> old = oldRecordListToMap("id", oldRecordList);
|
||||
for(QRecord record : records)
|
||||
{
|
||||
record.removeValue(ProviderService.KEY_STATUS_FIELD);
|
||||
if(isSystemUpdate(record))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
QRecord merged = new QRecord();
|
||||
old.map(m -> m.get(record.getValue("id"))).ifPresent(o -> merged.setValues(new java.util.LinkedHashMap<>(o.getValues())));
|
||||
record.getValues().forEach(merged::setValue);
|
||||
|
||||
for(String error : service.validate(new AssistantProvider(merged)))
|
||||
{
|
||||
record.addError(new BadInputStatusMessage(error));
|
||||
}
|
||||
if(record.getValue("enabled") == null && merged.getValue("enabled") == null)
|
||||
{
|
||||
record.setValue("enabled", true);
|
||||
}
|
||||
if(record.getValue("isDefault") == null && merged.getValue("isDefault") == null)
|
||||
{
|
||||
record.setValue("isDefault", false);
|
||||
}
|
||||
}
|
||||
return (records);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Keep one default provider; drop caches.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public List<QRecord> postInsertOrUpdate(AbstractActionInput input, List<QRecord> records, Optional<List<QRecord>> oldRecordList) throws QException
|
||||
{
|
||||
AssistantRuntime runtime = AssistantRuntime.of(QContext.getQInstance());
|
||||
if(runtime == null)
|
||||
{
|
||||
return (records);
|
||||
}
|
||||
if(AssistantProvider.TABLE_NAME.equals(tableName(input)))
|
||||
{
|
||||
for(QRecord record : records)
|
||||
{
|
||||
if(Boolean.TRUE.equals(record.getValueBoolean("isDefault")) && record.getValue("id") != null && record.getErrors().isEmpty())
|
||||
{
|
||||
runtime.getStore().clearOtherDefaultProviders(record.getValueInteger("id"));
|
||||
}
|
||||
}
|
||||
}
|
||||
runtime.getProviderService().invalidate();
|
||||
return (records);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Drop caches after a delete.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public List<QRecord> postDelete(DeleteInput deleteInput, List<QRecord> records) throws QException
|
||||
{
|
||||
ProviderService service = service();
|
||||
if(service != null)
|
||||
{
|
||||
service.invalidate();
|
||||
}
|
||||
return (records);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** An update by the qbit itself: only the last test result, or clearing the
|
||||
** default flag. These are saved even for a row whose settings are invalid.
|
||||
*******************************************************************************/
|
||||
static boolean isSystemUpdate(QRecord record)
|
||||
{
|
||||
if(record.getValue("id") == null)
|
||||
{
|
||||
return (false);
|
||||
}
|
||||
for(Map.Entry<String, Serializable> entry : record.getValues().entrySet())
|
||||
{
|
||||
String name = entry.getKey();
|
||||
boolean system = name.equals("id") || name.equals("modifyDate") || name.startsWith("lastTest")
|
||||
|| (name.equals("isDefault") && Boolean.FALSE.equals(entry.getValue()));
|
||||
if(!system)
|
||||
{
|
||||
return (false);
|
||||
}
|
||||
}
|
||||
return (true);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private static String tableName(AbstractActionInput input)
|
||||
{
|
||||
if(input instanceof com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput insert)
|
||||
{
|
||||
return (insert.getTableName());
|
||||
}
|
||||
if(input instanceof com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput update)
|
||||
{
|
||||
return (update.getTableName());
|
||||
}
|
||||
return (null);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private static ProviderService service()
|
||||
{
|
||||
AssistantRuntime runtime = AssistantRuntime.of(QContext.getQInstance());
|
||||
return (runtime == null ? null : runtime.getProviderService());
|
||||
}
|
||||
|
||||
}
|
||||
+429
@@ -0,0 +1,429 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.model;
|
||||
|
||||
|
||||
import java.math.BigDecimal;
|
||||
import java.time.Instant;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QField;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The price of one model, in USD per million tokens, used to compute the cost
|
||||
** of every model call. Seeded with the Anthropic list prices; editable.
|
||||
*******************************************************************************/
|
||||
public class AssistantModelPrice extends QRecordEntity
|
||||
{
|
||||
public static final String TABLE_NAME = "assistant_model_price";
|
||||
|
||||
@QField(isEditable = false, isPrimaryKey = true)
|
||||
private Integer id;
|
||||
|
||||
@QField(isRequired = true, maxLength = 50)
|
||||
private String providerType;
|
||||
|
||||
@QField(isRequired = true, maxLength = 100)
|
||||
private String model;
|
||||
|
||||
@QField(isRequired = true)
|
||||
private BigDecimal inputUsdPerMTok;
|
||||
|
||||
@QField(isRequired = true)
|
||||
private BigDecimal outputUsdPerMTok;
|
||||
|
||||
@QField()
|
||||
private BigDecimal cacheReadUsdPerMTok;
|
||||
|
||||
@QField()
|
||||
private BigDecimal cacheWrite5mUsdPerMTok;
|
||||
|
||||
@QField()
|
||||
private BigDecimal cacheWrite1hUsdPerMTok;
|
||||
|
||||
@QField(maxLength = 255)
|
||||
private String notes;
|
||||
|
||||
@QField(isEditable = false)
|
||||
private Instant createDate;
|
||||
|
||||
@QField(isEditable = false)
|
||||
private Instant modifyDate;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Default constructor.
|
||||
*******************************************************************************/
|
||||
public AssistantModelPrice()
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Constructor that populates from a QRecord.
|
||||
*******************************************************************************/
|
||||
public AssistantModelPrice(QRecord record) throws QException
|
||||
{
|
||||
populateFromQRecord(record);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for id
|
||||
*******************************************************************************/
|
||||
public Integer getId()
|
||||
{
|
||||
return (this.id);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for id
|
||||
*******************************************************************************/
|
||||
public void setId(Integer id)
|
||||
{
|
||||
this.id = id;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for id
|
||||
*******************************************************************************/
|
||||
public AssistantModelPrice withId(Integer id)
|
||||
{
|
||||
this.id = id;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for providerType
|
||||
*******************************************************************************/
|
||||
public String getProviderType()
|
||||
{
|
||||
return (this.providerType);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for providerType
|
||||
*******************************************************************************/
|
||||
public void setProviderType(String providerType)
|
||||
{
|
||||
this.providerType = providerType;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for providerType
|
||||
*******************************************************************************/
|
||||
public AssistantModelPrice withProviderType(String providerType)
|
||||
{
|
||||
this.providerType = providerType;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for model
|
||||
*******************************************************************************/
|
||||
public String getModel()
|
||||
{
|
||||
return (this.model);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for model
|
||||
*******************************************************************************/
|
||||
public void setModel(String model)
|
||||
{
|
||||
this.model = model;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for model
|
||||
*******************************************************************************/
|
||||
public AssistantModelPrice withModel(String model)
|
||||
{
|
||||
this.model = model;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for inputUsdPerMTok
|
||||
*******************************************************************************/
|
||||
public BigDecimal getInputUsdPerMTok()
|
||||
{
|
||||
return (this.inputUsdPerMTok);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for inputUsdPerMTok
|
||||
*******************************************************************************/
|
||||
public void setInputUsdPerMTok(BigDecimal inputUsdPerMTok)
|
||||
{
|
||||
this.inputUsdPerMTok = inputUsdPerMTok;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for inputUsdPerMTok
|
||||
*******************************************************************************/
|
||||
public AssistantModelPrice withInputUsdPerMTok(BigDecimal inputUsdPerMTok)
|
||||
{
|
||||
this.inputUsdPerMTok = inputUsdPerMTok;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for outputUsdPerMTok
|
||||
*******************************************************************************/
|
||||
public BigDecimal getOutputUsdPerMTok()
|
||||
{
|
||||
return (this.outputUsdPerMTok);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for outputUsdPerMTok
|
||||
*******************************************************************************/
|
||||
public void setOutputUsdPerMTok(BigDecimal outputUsdPerMTok)
|
||||
{
|
||||
this.outputUsdPerMTok = outputUsdPerMTok;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for outputUsdPerMTok
|
||||
*******************************************************************************/
|
||||
public AssistantModelPrice withOutputUsdPerMTok(BigDecimal outputUsdPerMTok)
|
||||
{
|
||||
this.outputUsdPerMTok = outputUsdPerMTok;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for cacheReadUsdPerMTok
|
||||
*******************************************************************************/
|
||||
public BigDecimal getCacheReadUsdPerMTok()
|
||||
{
|
||||
return (this.cacheReadUsdPerMTok);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for cacheReadUsdPerMTok
|
||||
*******************************************************************************/
|
||||
public void setCacheReadUsdPerMTok(BigDecimal cacheReadUsdPerMTok)
|
||||
{
|
||||
this.cacheReadUsdPerMTok = cacheReadUsdPerMTok;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for cacheReadUsdPerMTok
|
||||
*******************************************************************************/
|
||||
public AssistantModelPrice withCacheReadUsdPerMTok(BigDecimal cacheReadUsdPerMTok)
|
||||
{
|
||||
this.cacheReadUsdPerMTok = cacheReadUsdPerMTok;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for cacheWrite5mUsdPerMTok
|
||||
*******************************************************************************/
|
||||
public BigDecimal getCacheWrite5mUsdPerMTok()
|
||||
{
|
||||
return (this.cacheWrite5mUsdPerMTok);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for cacheWrite5mUsdPerMTok
|
||||
*******************************************************************************/
|
||||
public void setCacheWrite5mUsdPerMTok(BigDecimal cacheWrite5mUsdPerMTok)
|
||||
{
|
||||
this.cacheWrite5mUsdPerMTok = cacheWrite5mUsdPerMTok;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for cacheWrite5mUsdPerMTok
|
||||
*******************************************************************************/
|
||||
public AssistantModelPrice withCacheWrite5mUsdPerMTok(BigDecimal cacheWrite5mUsdPerMTok)
|
||||
{
|
||||
this.cacheWrite5mUsdPerMTok = cacheWrite5mUsdPerMTok;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for cacheWrite1hUsdPerMTok
|
||||
*******************************************************************************/
|
||||
public BigDecimal getCacheWrite1hUsdPerMTok()
|
||||
{
|
||||
return (this.cacheWrite1hUsdPerMTok);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for cacheWrite1hUsdPerMTok
|
||||
*******************************************************************************/
|
||||
public void setCacheWrite1hUsdPerMTok(BigDecimal cacheWrite1hUsdPerMTok)
|
||||
{
|
||||
this.cacheWrite1hUsdPerMTok = cacheWrite1hUsdPerMTok;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for cacheWrite1hUsdPerMTok
|
||||
*******************************************************************************/
|
||||
public AssistantModelPrice withCacheWrite1hUsdPerMTok(BigDecimal cacheWrite1hUsdPerMTok)
|
||||
{
|
||||
this.cacheWrite1hUsdPerMTok = cacheWrite1hUsdPerMTok;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for notes
|
||||
*******************************************************************************/
|
||||
public String getNotes()
|
||||
{
|
||||
return (this.notes);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for notes
|
||||
*******************************************************************************/
|
||||
public void setNotes(String notes)
|
||||
{
|
||||
this.notes = notes;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for notes
|
||||
*******************************************************************************/
|
||||
public AssistantModelPrice withNotes(String notes)
|
||||
{
|
||||
this.notes = notes;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for createDate
|
||||
*******************************************************************************/
|
||||
public Instant getCreateDate()
|
||||
{
|
||||
return (this.createDate);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for createDate
|
||||
*******************************************************************************/
|
||||
public void setCreateDate(Instant createDate)
|
||||
{
|
||||
this.createDate = createDate;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for createDate
|
||||
*******************************************************************************/
|
||||
public AssistantModelPrice withCreateDate(Instant createDate)
|
||||
{
|
||||
this.createDate = createDate;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for modifyDate
|
||||
*******************************************************************************/
|
||||
public Instant getModifyDate()
|
||||
{
|
||||
return (this.modifyDate);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for modifyDate
|
||||
*******************************************************************************/
|
||||
public void setModifyDate(Instant modifyDate)
|
||||
{
|
||||
this.modifyDate = modifyDate;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for modifyDate
|
||||
*******************************************************************************/
|
||||
public AssistantModelPrice withModifyDate(Instant modifyDate)
|
||||
{
|
||||
this.modifyDate = modifyDate;
|
||||
return (this);
|
||||
}
|
||||
|
||||
}
|
||||
+709
@@ -0,0 +1,709 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.model;
|
||||
|
||||
|
||||
import java.math.BigDecimal;
|
||||
import java.time.Instant;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QField;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A model provider the assistant can use: its type, model, connection,
|
||||
** limits and the result of its last connection test (the provider settings
|
||||
** screen). The API key itself is never stored: apiKeyEnvName names the
|
||||
** environment variable (or host secret) that holds it, resolved at runtime.
|
||||
** The read-only apiKeyStatus field (filled on read, never stored) shows whether it is set and its
|
||||
** last four characters.
|
||||
*******************************************************************************/
|
||||
public class AssistantProvider extends QRecordEntity
|
||||
{
|
||||
public static final String TABLE_NAME = "assistant_provider";
|
||||
|
||||
@QField(isEditable = false, isPrimaryKey = true)
|
||||
private Integer id;
|
||||
|
||||
@QField(isRequired = true, maxLength = 100)
|
||||
private String name;
|
||||
|
||||
@QField(isRequired = true, maxLength = 50)
|
||||
private String providerType;
|
||||
|
||||
@QField(maxLength = 100)
|
||||
private String model;
|
||||
|
||||
@QField()
|
||||
private Boolean enabled;
|
||||
|
||||
@QField()
|
||||
private Boolean isDefault;
|
||||
|
||||
@QField(maxLength = 255)
|
||||
private String baseUrl;
|
||||
|
||||
@QField(maxLength = 100)
|
||||
private String apiKeyEnvName;
|
||||
|
||||
/*******************************************************************************
|
||||
** Computed when a row is read (whether the key is set, and its last four
|
||||
** characters); never written - saves drop it.
|
||||
*******************************************************************************/
|
||||
@QField(isEditable = false, maxLength = 200)
|
||||
private String apiKeyStatus;
|
||||
|
||||
@QField()
|
||||
private BigDecimal monthlyCostCapUsd;
|
||||
|
||||
@QField()
|
||||
private Integer maxOutputTokensPerRequest;
|
||||
|
||||
@QField()
|
||||
private Integer maxConcurrentRequests;
|
||||
|
||||
@QField(isEditable = false)
|
||||
private Instant lastTestAt;
|
||||
|
||||
@QField(isEditable = false, maxLength = 16)
|
||||
private String lastTestStatus;
|
||||
|
||||
@QField(isEditable = false)
|
||||
private Integer lastTestLatencyMs;
|
||||
|
||||
@QField(isEditable = false, maxLength = 100)
|
||||
private String lastTestModel;
|
||||
|
||||
@QField(isEditable = false, maxLength = 500)
|
||||
private String lastTestError;
|
||||
|
||||
@QField(isEditable = false)
|
||||
private Instant createDate;
|
||||
|
||||
@QField(isEditable = false)
|
||||
private Instant modifyDate;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Default constructor.
|
||||
*******************************************************************************/
|
||||
public AssistantProvider()
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Constructor that populates from a QRecord.
|
||||
*******************************************************************************/
|
||||
public AssistantProvider(QRecord record) throws QException
|
||||
{
|
||||
populateFromQRecord(record);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for id
|
||||
*******************************************************************************/
|
||||
public Integer getId()
|
||||
{
|
||||
return (this.id);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for id
|
||||
*******************************************************************************/
|
||||
public void setId(Integer id)
|
||||
{
|
||||
this.id = id;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for id
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withId(Integer id)
|
||||
{
|
||||
this.id = id;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for name
|
||||
*******************************************************************************/
|
||||
public String getName()
|
||||
{
|
||||
return (this.name);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for name
|
||||
*******************************************************************************/
|
||||
public void setName(String name)
|
||||
{
|
||||
this.name = name;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for name
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withName(String name)
|
||||
{
|
||||
this.name = name;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for providerType
|
||||
*******************************************************************************/
|
||||
public String getProviderType()
|
||||
{
|
||||
return (this.providerType);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for providerType
|
||||
*******************************************************************************/
|
||||
public void setProviderType(String providerType)
|
||||
{
|
||||
this.providerType = providerType;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for providerType
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withProviderType(String providerType)
|
||||
{
|
||||
this.providerType = providerType;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for model
|
||||
*******************************************************************************/
|
||||
public String getModel()
|
||||
{
|
||||
return (this.model);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for model
|
||||
*******************************************************************************/
|
||||
public void setModel(String model)
|
||||
{
|
||||
this.model = model;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for model
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withModel(String model)
|
||||
{
|
||||
this.model = model;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for enabled
|
||||
*******************************************************************************/
|
||||
public Boolean getEnabled()
|
||||
{
|
||||
return (this.enabled);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for enabled
|
||||
*******************************************************************************/
|
||||
public void setEnabled(Boolean enabled)
|
||||
{
|
||||
this.enabled = enabled;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for enabled
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withEnabled(Boolean enabled)
|
||||
{
|
||||
this.enabled = enabled;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for isDefault
|
||||
*******************************************************************************/
|
||||
public Boolean getIsDefault()
|
||||
{
|
||||
return (this.isDefault);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for isDefault
|
||||
*******************************************************************************/
|
||||
public void setIsDefault(Boolean isDefault)
|
||||
{
|
||||
this.isDefault = isDefault;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for isDefault
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withIsDefault(Boolean isDefault)
|
||||
{
|
||||
this.isDefault = isDefault;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for baseUrl
|
||||
*******************************************************************************/
|
||||
public String getBaseUrl()
|
||||
{
|
||||
return (this.baseUrl);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for baseUrl
|
||||
*******************************************************************************/
|
||||
public void setBaseUrl(String baseUrl)
|
||||
{
|
||||
this.baseUrl = baseUrl;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for baseUrl
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withBaseUrl(String baseUrl)
|
||||
{
|
||||
this.baseUrl = baseUrl;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for apiKeyEnvName
|
||||
*******************************************************************************/
|
||||
public String getApiKeyEnvName()
|
||||
{
|
||||
return (this.apiKeyEnvName);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for apiKeyEnvName
|
||||
*******************************************************************************/
|
||||
public void setApiKeyEnvName(String apiKeyEnvName)
|
||||
{
|
||||
this.apiKeyEnvName = apiKeyEnvName;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for apiKeyEnvName
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withApiKeyEnvName(String apiKeyEnvName)
|
||||
{
|
||||
this.apiKeyEnvName = apiKeyEnvName;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for monthlyCostCapUsd
|
||||
*******************************************************************************/
|
||||
public BigDecimal getMonthlyCostCapUsd()
|
||||
{
|
||||
return (this.monthlyCostCapUsd);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for monthlyCostCapUsd
|
||||
*******************************************************************************/
|
||||
public void setMonthlyCostCapUsd(BigDecimal monthlyCostCapUsd)
|
||||
{
|
||||
this.monthlyCostCapUsd = monthlyCostCapUsd;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for monthlyCostCapUsd
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withMonthlyCostCapUsd(BigDecimal monthlyCostCapUsd)
|
||||
{
|
||||
this.monthlyCostCapUsd = monthlyCostCapUsd;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for maxOutputTokensPerRequest
|
||||
*******************************************************************************/
|
||||
public Integer getMaxOutputTokensPerRequest()
|
||||
{
|
||||
return (this.maxOutputTokensPerRequest);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for maxOutputTokensPerRequest
|
||||
*******************************************************************************/
|
||||
public void setMaxOutputTokensPerRequest(Integer maxOutputTokensPerRequest)
|
||||
{
|
||||
this.maxOutputTokensPerRequest = maxOutputTokensPerRequest;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for maxOutputTokensPerRequest
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withMaxOutputTokensPerRequest(Integer maxOutputTokensPerRequest)
|
||||
{
|
||||
this.maxOutputTokensPerRequest = maxOutputTokensPerRequest;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for maxConcurrentRequests
|
||||
*******************************************************************************/
|
||||
public Integer getMaxConcurrentRequests()
|
||||
{
|
||||
return (this.maxConcurrentRequests);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for maxConcurrentRequests
|
||||
*******************************************************************************/
|
||||
public void setMaxConcurrentRequests(Integer maxConcurrentRequests)
|
||||
{
|
||||
this.maxConcurrentRequests = maxConcurrentRequests;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for maxConcurrentRequests
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withMaxConcurrentRequests(Integer maxConcurrentRequests)
|
||||
{
|
||||
this.maxConcurrentRequests = maxConcurrentRequests;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for lastTestAt
|
||||
*******************************************************************************/
|
||||
public Instant getLastTestAt()
|
||||
{
|
||||
return (this.lastTestAt);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for lastTestAt
|
||||
*******************************************************************************/
|
||||
public void setLastTestAt(Instant lastTestAt)
|
||||
{
|
||||
this.lastTestAt = lastTestAt;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for lastTestAt
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withLastTestAt(Instant lastTestAt)
|
||||
{
|
||||
this.lastTestAt = lastTestAt;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for lastTestStatus
|
||||
*******************************************************************************/
|
||||
public String getLastTestStatus()
|
||||
{
|
||||
return (this.lastTestStatus);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for lastTestStatus
|
||||
*******************************************************************************/
|
||||
public void setLastTestStatus(String lastTestStatus)
|
||||
{
|
||||
this.lastTestStatus = lastTestStatus;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for lastTestStatus
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withLastTestStatus(String lastTestStatus)
|
||||
{
|
||||
this.lastTestStatus = lastTestStatus;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for lastTestLatencyMs
|
||||
*******************************************************************************/
|
||||
public Integer getLastTestLatencyMs()
|
||||
{
|
||||
return (this.lastTestLatencyMs);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for lastTestLatencyMs
|
||||
*******************************************************************************/
|
||||
public void setLastTestLatencyMs(Integer lastTestLatencyMs)
|
||||
{
|
||||
this.lastTestLatencyMs = lastTestLatencyMs;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for lastTestLatencyMs
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withLastTestLatencyMs(Integer lastTestLatencyMs)
|
||||
{
|
||||
this.lastTestLatencyMs = lastTestLatencyMs;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for lastTestModel
|
||||
*******************************************************************************/
|
||||
public String getLastTestModel()
|
||||
{
|
||||
return (this.lastTestModel);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for lastTestModel
|
||||
*******************************************************************************/
|
||||
public void setLastTestModel(String lastTestModel)
|
||||
{
|
||||
this.lastTestModel = lastTestModel;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for lastTestModel
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withLastTestModel(String lastTestModel)
|
||||
{
|
||||
this.lastTestModel = lastTestModel;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for lastTestError
|
||||
*******************************************************************************/
|
||||
public String getLastTestError()
|
||||
{
|
||||
return (this.lastTestError);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for lastTestError
|
||||
*******************************************************************************/
|
||||
public void setLastTestError(String lastTestError)
|
||||
{
|
||||
this.lastTestError = lastTestError;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for lastTestError
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withLastTestError(String lastTestError)
|
||||
{
|
||||
this.lastTestError = lastTestError;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for createDate
|
||||
*******************************************************************************/
|
||||
public Instant getCreateDate()
|
||||
{
|
||||
return (this.createDate);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for createDate
|
||||
*******************************************************************************/
|
||||
public void setCreateDate(Instant createDate)
|
||||
{
|
||||
this.createDate = createDate;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for createDate
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withCreateDate(Instant createDate)
|
||||
{
|
||||
this.createDate = createDate;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for modifyDate
|
||||
*******************************************************************************/
|
||||
public Instant getModifyDate()
|
||||
{
|
||||
return (this.modifyDate);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for modifyDate
|
||||
*******************************************************************************/
|
||||
public void setModifyDate(Instant modifyDate)
|
||||
{
|
||||
this.modifyDate = modifyDate;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for modifyDate
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withModifyDate(Instant modifyDate)
|
||||
{
|
||||
this.modifyDate = modifyDate;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for apiKeyStatus (masked key status, computed on read)
|
||||
*******************************************************************************/
|
||||
public String getApiKeyStatus()
|
||||
{
|
||||
return (this.apiKeyStatus);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for apiKeyStatus
|
||||
*******************************************************************************/
|
||||
public void setApiKeyStatus(String apiKeyStatus)
|
||||
{
|
||||
this.apiKeyStatus = apiKeyStatus;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for apiKeyStatus
|
||||
*******************************************************************************/
|
||||
public AssistantProvider withApiKeyStatus(String apiKeyStatus)
|
||||
{
|
||||
this.apiKeyStatus = apiKeyStatus;
|
||||
return (this);
|
||||
}
|
||||
|
||||
}
|
||||
@@ -0,0 +1,701 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.model;
|
||||
|
||||
|
||||
import java.time.Instant;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QField;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecordEntity;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** One model call: who, which provider and model, what for (a TURN or a
|
||||
** connection TEST), how it ended, how long it took, the tokens and the cost.
|
||||
** Written by the qbit only.
|
||||
*******************************************************************************/
|
||||
public class AssistantUsage extends QRecordEntity
|
||||
{
|
||||
public static final String TABLE_NAME = "assistant_usage";
|
||||
|
||||
@QField(isEditable = false, isPrimaryKey = true)
|
||||
private Integer id;
|
||||
|
||||
@QField(isEditable = false, maxLength = 255)
|
||||
private String userId;
|
||||
|
||||
@QField(isEditable = false)
|
||||
private Integer conversationId;
|
||||
|
||||
@QField(isEditable = false)
|
||||
private Integer turnId;
|
||||
|
||||
@QField(isEditable = false)
|
||||
private Integer providerId;
|
||||
|
||||
@QField(isEditable = false, maxLength = 50)
|
||||
private String providerType;
|
||||
|
||||
@QField(isEditable = false, maxLength = 100)
|
||||
private String model;
|
||||
|
||||
@QField(isEditable = false, maxLength = 16)
|
||||
private String purpose;
|
||||
|
||||
@QField(isEditable = false, maxLength = 16)
|
||||
private String status;
|
||||
|
||||
@QField(isEditable = false, maxLength = 32)
|
||||
private String errorKind;
|
||||
|
||||
@QField(isEditable = false)
|
||||
private Integer httpStatus;
|
||||
|
||||
@QField(isEditable = false)
|
||||
private Integer latencyMs;
|
||||
|
||||
@QField(isEditable = false)
|
||||
private Long inputTokens;
|
||||
|
||||
@QField(isEditable = false)
|
||||
private Long outputTokens;
|
||||
|
||||
@QField(isEditable = false)
|
||||
private Long cacheReadTokens;
|
||||
|
||||
@QField(isEditable = false)
|
||||
private Long cacheWriteTokens;
|
||||
|
||||
@QField(isEditable = false)
|
||||
private Long costMicros;
|
||||
|
||||
@QField(isEditable = false, maxLength = 100)
|
||||
private String requestId;
|
||||
|
||||
@QField(isEditable = false)
|
||||
private Instant createDate;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Default constructor.
|
||||
*******************************************************************************/
|
||||
public AssistantUsage()
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Constructor that populates from a QRecord.
|
||||
*******************************************************************************/
|
||||
public AssistantUsage(QRecord record) throws QException
|
||||
{
|
||||
populateFromQRecord(record);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for id
|
||||
*******************************************************************************/
|
||||
public Integer getId()
|
||||
{
|
||||
return (this.id);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for id
|
||||
*******************************************************************************/
|
||||
public void setId(Integer id)
|
||||
{
|
||||
this.id = id;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for id
|
||||
*******************************************************************************/
|
||||
public AssistantUsage withId(Integer id)
|
||||
{
|
||||
this.id = id;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for userId
|
||||
*******************************************************************************/
|
||||
public String getUserId()
|
||||
{
|
||||
return (this.userId);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for userId
|
||||
*******************************************************************************/
|
||||
public void setUserId(String userId)
|
||||
{
|
||||
this.userId = userId;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for userId
|
||||
*******************************************************************************/
|
||||
public AssistantUsage withUserId(String userId)
|
||||
{
|
||||
this.userId = userId;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for conversationId
|
||||
*******************************************************************************/
|
||||
public Integer getConversationId()
|
||||
{
|
||||
return (this.conversationId);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for conversationId
|
||||
*******************************************************************************/
|
||||
public void setConversationId(Integer conversationId)
|
||||
{
|
||||
this.conversationId = conversationId;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for conversationId
|
||||
*******************************************************************************/
|
||||
public AssistantUsage withConversationId(Integer conversationId)
|
||||
{
|
||||
this.conversationId = conversationId;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for turnId
|
||||
*******************************************************************************/
|
||||
public Integer getTurnId()
|
||||
{
|
||||
return (this.turnId);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for turnId
|
||||
*******************************************************************************/
|
||||
public void setTurnId(Integer turnId)
|
||||
{
|
||||
this.turnId = turnId;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for turnId
|
||||
*******************************************************************************/
|
||||
public AssistantUsage withTurnId(Integer turnId)
|
||||
{
|
||||
this.turnId = turnId;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for providerId
|
||||
*******************************************************************************/
|
||||
public Integer getProviderId()
|
||||
{
|
||||
return (this.providerId);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for providerId
|
||||
*******************************************************************************/
|
||||
public void setProviderId(Integer providerId)
|
||||
{
|
||||
this.providerId = providerId;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for providerId
|
||||
*******************************************************************************/
|
||||
public AssistantUsage withProviderId(Integer providerId)
|
||||
{
|
||||
this.providerId = providerId;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for providerType
|
||||
*******************************************************************************/
|
||||
public String getProviderType()
|
||||
{
|
||||
return (this.providerType);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for providerType
|
||||
*******************************************************************************/
|
||||
public void setProviderType(String providerType)
|
||||
{
|
||||
this.providerType = providerType;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for providerType
|
||||
*******************************************************************************/
|
||||
public AssistantUsage withProviderType(String providerType)
|
||||
{
|
||||
this.providerType = providerType;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for model
|
||||
*******************************************************************************/
|
||||
public String getModel()
|
||||
{
|
||||
return (this.model);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for model
|
||||
*******************************************************************************/
|
||||
public void setModel(String model)
|
||||
{
|
||||
this.model = model;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for model
|
||||
*******************************************************************************/
|
||||
public AssistantUsage withModel(String model)
|
||||
{
|
||||
this.model = model;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for purpose
|
||||
*******************************************************************************/
|
||||
public String getPurpose()
|
||||
{
|
||||
return (this.purpose);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for purpose
|
||||
*******************************************************************************/
|
||||
public void setPurpose(String purpose)
|
||||
{
|
||||
this.purpose = purpose;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for purpose
|
||||
*******************************************************************************/
|
||||
public AssistantUsage withPurpose(String purpose)
|
||||
{
|
||||
this.purpose = purpose;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for status
|
||||
*******************************************************************************/
|
||||
public String getStatus()
|
||||
{
|
||||
return (this.status);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for status
|
||||
*******************************************************************************/
|
||||
public void setStatus(String status)
|
||||
{
|
||||
this.status = status;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for status
|
||||
*******************************************************************************/
|
||||
public AssistantUsage withStatus(String status)
|
||||
{
|
||||
this.status = status;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for errorKind
|
||||
*******************************************************************************/
|
||||
public String getErrorKind()
|
||||
{
|
||||
return (this.errorKind);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for errorKind
|
||||
*******************************************************************************/
|
||||
public void setErrorKind(String errorKind)
|
||||
{
|
||||
this.errorKind = errorKind;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for errorKind
|
||||
*******************************************************************************/
|
||||
public AssistantUsage withErrorKind(String errorKind)
|
||||
{
|
||||
this.errorKind = errorKind;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for httpStatus
|
||||
*******************************************************************************/
|
||||
public Integer getHttpStatus()
|
||||
{
|
||||
return (this.httpStatus);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for httpStatus
|
||||
*******************************************************************************/
|
||||
public void setHttpStatus(Integer httpStatus)
|
||||
{
|
||||
this.httpStatus = httpStatus;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for httpStatus
|
||||
*******************************************************************************/
|
||||
public AssistantUsage withHttpStatus(Integer httpStatus)
|
||||
{
|
||||
this.httpStatus = httpStatus;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for latencyMs
|
||||
*******************************************************************************/
|
||||
public Integer getLatencyMs()
|
||||
{
|
||||
return (this.latencyMs);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for latencyMs
|
||||
*******************************************************************************/
|
||||
public void setLatencyMs(Integer latencyMs)
|
||||
{
|
||||
this.latencyMs = latencyMs;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for latencyMs
|
||||
*******************************************************************************/
|
||||
public AssistantUsage withLatencyMs(Integer latencyMs)
|
||||
{
|
||||
this.latencyMs = latencyMs;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for inputTokens
|
||||
*******************************************************************************/
|
||||
public Long getInputTokens()
|
||||
{
|
||||
return (this.inputTokens);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for inputTokens
|
||||
*******************************************************************************/
|
||||
public void setInputTokens(Long inputTokens)
|
||||
{
|
||||
this.inputTokens = inputTokens;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for inputTokens
|
||||
*******************************************************************************/
|
||||
public AssistantUsage withInputTokens(Long inputTokens)
|
||||
{
|
||||
this.inputTokens = inputTokens;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for outputTokens
|
||||
*******************************************************************************/
|
||||
public Long getOutputTokens()
|
||||
{
|
||||
return (this.outputTokens);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for outputTokens
|
||||
*******************************************************************************/
|
||||
public void setOutputTokens(Long outputTokens)
|
||||
{
|
||||
this.outputTokens = outputTokens;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for outputTokens
|
||||
*******************************************************************************/
|
||||
public AssistantUsage withOutputTokens(Long outputTokens)
|
||||
{
|
||||
this.outputTokens = outputTokens;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for cacheReadTokens
|
||||
*******************************************************************************/
|
||||
public Long getCacheReadTokens()
|
||||
{
|
||||
return (this.cacheReadTokens);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for cacheReadTokens
|
||||
*******************************************************************************/
|
||||
public void setCacheReadTokens(Long cacheReadTokens)
|
||||
{
|
||||
this.cacheReadTokens = cacheReadTokens;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for cacheReadTokens
|
||||
*******************************************************************************/
|
||||
public AssistantUsage withCacheReadTokens(Long cacheReadTokens)
|
||||
{
|
||||
this.cacheReadTokens = cacheReadTokens;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for cacheWriteTokens
|
||||
*******************************************************************************/
|
||||
public Long getCacheWriteTokens()
|
||||
{
|
||||
return (this.cacheWriteTokens);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for cacheWriteTokens
|
||||
*******************************************************************************/
|
||||
public void setCacheWriteTokens(Long cacheWriteTokens)
|
||||
{
|
||||
this.cacheWriteTokens = cacheWriteTokens;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for cacheWriteTokens
|
||||
*******************************************************************************/
|
||||
public AssistantUsage withCacheWriteTokens(Long cacheWriteTokens)
|
||||
{
|
||||
this.cacheWriteTokens = cacheWriteTokens;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for costMicros
|
||||
*******************************************************************************/
|
||||
public Long getCostMicros()
|
||||
{
|
||||
return (this.costMicros);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for costMicros
|
||||
*******************************************************************************/
|
||||
public void setCostMicros(Long costMicros)
|
||||
{
|
||||
this.costMicros = costMicros;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for costMicros
|
||||
*******************************************************************************/
|
||||
public AssistantUsage withCostMicros(Long costMicros)
|
||||
{
|
||||
this.costMicros = costMicros;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for requestId
|
||||
*******************************************************************************/
|
||||
public String getRequestId()
|
||||
{
|
||||
return (this.requestId);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for requestId
|
||||
*******************************************************************************/
|
||||
public void setRequestId(String requestId)
|
||||
{
|
||||
this.requestId = requestId;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for requestId
|
||||
*******************************************************************************/
|
||||
public AssistantUsage withRequestId(String requestId)
|
||||
{
|
||||
this.requestId = requestId;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for createDate
|
||||
*******************************************************************************/
|
||||
public Instant getCreateDate()
|
||||
{
|
||||
return (this.createDate);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for createDate
|
||||
*******************************************************************************/
|
||||
public void setCreateDate(Instant createDate)
|
||||
{
|
||||
this.createDate = createDate;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for createDate
|
||||
*******************************************************************************/
|
||||
public AssistantUsage withCreateDate(Instant createDate)
|
||||
{
|
||||
this.createDate = createDate;
|
||||
return (this);
|
||||
}
|
||||
|
||||
}
|
||||
+85
@@ -28,10 +28,13 @@ import java.util.concurrent.ConcurrentHashMap;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantConversation;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantFeedback;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantMessage;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantModelPrice;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantPolicy;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantProvider;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantToolCall;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantTurn;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantTurnEvent;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantUsage;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantUsageDaily;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelUsage;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.DeleteAction;
|
||||
@@ -70,6 +73,9 @@ public class AssistantStore
|
||||
public static final int MAX_CONVERSATIONS = 200;
|
||||
public static final int MAX_RUNNING_TURNS = 50;
|
||||
public static final int EVENT_PAGE_SIZE = 500;
|
||||
public static final int MAX_PROVIDERS = 100;
|
||||
public static final int MAX_PRICES = 500;
|
||||
public static final int MAX_SUM_ROWS = 100_000;
|
||||
|
||||
private static final Map<String, Object> USAGE_LOCKS = new ConcurrentHashMap<>();
|
||||
|
||||
@@ -610,4 +616,83 @@ public class AssistantStore
|
||||
return (value == null ? 0 : value);
|
||||
}
|
||||
|
||||
|
||||
/////////////////////////////////////////////////////////////////////////////
|
||||
// provider settings, prices and per-call usage //
|
||||
/////////////////////////////////////////////////////////////////////////////
|
||||
|
||||
/*******************************************************************************
|
||||
** All provider rows, by id.
|
||||
*******************************************************************************/
|
||||
public List<AssistantProvider> listProviders() throws QException
|
||||
{
|
||||
return (query(AssistantProvider.TABLE_NAME, new QQueryFilter().withOrderBy(new QFilterOrderBy("id", true)).withLimit(MAX_PROVIDERS), AssistantProvider.class));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** One provider row, or null.
|
||||
*******************************************************************************/
|
||||
public AssistantProvider getProvider(Integer id) throws QException
|
||||
{
|
||||
return (first(query(AssistantProvider.TABLE_NAME, new QQueryFilter().withCriteria("id", QCriteriaOperator.EQUALS, id), AssistantProvider.class)));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Make every provider row other than `keepId` non-default.
|
||||
*******************************************************************************/
|
||||
public void clearOtherDefaultProviders(Integer keepId) throws QException
|
||||
{
|
||||
List<AssistantProvider> others = query(AssistantProvider.TABLE_NAME, new QQueryFilter()
|
||||
.withCriteria("isDefault", QCriteriaOperator.EQUALS, true)
|
||||
.withCriteria("id", QCriteriaOperator.NOT_EQUALS, keepId), AssistantProvider.class);
|
||||
for(AssistantProvider other : others)
|
||||
{
|
||||
updateFields(AssistantProvider.TABLE_NAME, new QRecord().withValue("id", other.getId()).withValue("isDefault", false));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** All price rows.
|
||||
*******************************************************************************/
|
||||
public List<AssistantModelPrice> listPrices() throws QException
|
||||
{
|
||||
return (query(AssistantModelPrice.TABLE_NAME, new QQueryFilter().withOrderBy(new QFilterOrderBy("id", true)).withLimit(MAX_PRICES), AssistantModelPrice.class));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The cost (micro-dollars) of a provider's calls since a time.
|
||||
*******************************************************************************/
|
||||
public long sumUsageCostSince(Integer providerId, Instant since) throws QException
|
||||
{
|
||||
long total = 0;
|
||||
for(QRecord record : queryRecords(AssistantUsage.TABLE_NAME, new QQueryFilter()
|
||||
.withCriteria("providerId", QCriteriaOperator.EQUALS, providerId)
|
||||
.withCriteria("createDate", QCriteriaOperator.GREATER_THAN_OR_EQUALS, since), MAX_SUM_ROWS))
|
||||
{
|
||||
Long cost = record.getValueLong("costMicros");
|
||||
total += cost == null ? 0 : cost;
|
||||
}
|
||||
return (total);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Raw rows of one of the qbit's tables (the app home's reads), at most
|
||||
** `limit`.
|
||||
*******************************************************************************/
|
||||
public List<QRecord> queryRecords(String tableName, QQueryFilter filter, int limit) throws QException
|
||||
{
|
||||
List<QRecord> records = new QueryAction().execute(new QueryInput(tableName).withFilter(filter.withLimit(limit))).getRecords();
|
||||
return (records == null ? List.of() : records);
|
||||
}
|
||||
|
||||
}
|
||||
+228
@@ -0,0 +1,228 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.processes;
|
||||
|
||||
|
||||
import java.io.Serializable;
|
||||
import java.util.ArrayList;
|
||||
import java.util.HashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import com.kingsrook.qbits.assistant.AssistantRuntime;
|
||||
import com.kingsrook.qbits.assistant.AssistantSecurity;
|
||||
import com.kingsrook.qbits.assistant.engine.ConnectionTest;
|
||||
import com.kingsrook.qbits.assistant.engine.ProviderService;
|
||||
import com.kingsrook.qbits.assistant.metadata.AssistantMetaDataBuilder;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantProvider;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
|
||||
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractLoadStep;
|
||||
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractTransformStep;
|
||||
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ExtractViaQueryStep;
|
||||
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ProcessSummaryProviderInterface;
|
||||
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Test connection: a record action on assistant_provider. A standard
|
||||
** streamed ETL: Review says what will be called (and what is wrong with a
|
||||
** row before any call); Result has one line per provider with the status,
|
||||
** latency and model reached, or the error. The result is also saved on the
|
||||
** provider row and the call is recorded in assistant_usage (purpose TEST).
|
||||
*******************************************************************************/
|
||||
public final class TestConnectionProcess
|
||||
{
|
||||
public static final String NAME = "assistantProviderTestConnection";
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private TestConnectionProcess()
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The process.
|
||||
*******************************************************************************/
|
||||
public static QProcessMetaData process()
|
||||
{
|
||||
Map<String, Serializable> values = new HashMap<>();
|
||||
values.put(StreamedETLWithFrontendProcess.FIELD_SOURCE_TABLE, AssistantProvider.TABLE_NAME);
|
||||
values.put(StreamedETLWithFrontendProcess.FIELD_DESTINATION_TABLE, AssistantProvider.TABLE_NAME);
|
||||
values.put(StreamedETLWithFrontendProcess.FIELD_SUPPORTS_FULL_VALIDATION, true);
|
||||
values.put(StreamedETLWithFrontendProcess.FIELD_DO_FULL_VALIDATION, true);
|
||||
values.put(StreamedETLWithFrontendProcess.FIELD_PREVIEW_MESSAGE,
|
||||
"This makes one short live call to each selected provider's model (a few tokens) and saves the result on the provider.");
|
||||
QProcessMetaData process = StreamedETLWithFrontendProcess.defineProcessMetaData(ExtractViaQueryStep.class, Transform.class, Load.class, values);
|
||||
process.setName(NAME);
|
||||
process.setLabel("Test connection");
|
||||
process.setIcon(new QIcon("network_check"));
|
||||
process.setTableName(AssistantProvider.TABLE_NAME);
|
||||
process.setPermissionRules(AssistantMetaDataBuilder.adminRules());
|
||||
return (process);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A summary line with future (Review) and past (Result) wording.
|
||||
*******************************************************************************/
|
||||
static ProcessSummaryLine line(Status status, String singularFuture, String pluralFuture, String singularPast, String pluralPast)
|
||||
{
|
||||
ProcessSummaryLine line = new ProcessSummaryLine(status, 0, null);
|
||||
line.setSingularFutureMessage(singularFuture);
|
||||
line.setPluralFutureMessage(pluralFuture);
|
||||
line.setSingularPastMessage(singularPast);
|
||||
line.setPluralPastMessage(pluralPast);
|
||||
return (line);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A one-record line with the same text on both screens.
|
||||
*******************************************************************************/
|
||||
static ProcessSummaryLine single(Status status, String text, Serializable primaryKey)
|
||||
{
|
||||
ProcessSummaryLine line = line(status, text, text, text, text);
|
||||
line.incrementCountAndAddPrimaryKey(primaryKey);
|
||||
return (line);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Review: nothing is called yet.
|
||||
*******************************************************************************/
|
||||
public static class Transform extends AbstractTransformStep implements ProcessSummaryProviderInterface
|
||||
{
|
||||
private final ProcessSummaryLine willTest = line(Status.OK, "provider will be tested with a live call", "providers will be tested with a live call",
|
||||
"provider was tested", "providers were tested");
|
||||
private final ProcessSummaryLine disabled = line(Status.INFO, "provider is turned off (it is tested anyway)", "providers are turned off (they are tested anyway)",
|
||||
"provider was turned off", "providers were turned off");
|
||||
private final List<ProcessSummaryLineInterface> problems = new ArrayList<>();
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
|
||||
{
|
||||
AssistantRuntime runtime = AssistantRuntime.of(QContext.getQInstance());
|
||||
for(QRecord record : input.getRecords())
|
||||
{
|
||||
AssistantProvider row = new AssistantProvider(record);
|
||||
willTest.incrementCountAndAddPrimaryKey(row.getId());
|
||||
if(!Boolean.TRUE.equals(row.getEnabled()))
|
||||
{
|
||||
disabled.incrementCountAndAddPrimaryKey(row.getId());
|
||||
}
|
||||
if(runtime != null)
|
||||
{
|
||||
ProviderService service = runtime.getProviderService();
|
||||
for(String error : service.validate(row))
|
||||
{
|
||||
problems.add(single(Status.ERROR, row.getName() + ": " + error, row.getId()));
|
||||
}
|
||||
problems.add(single(Status.INFO, row.getName() + ": API key " + service.keyStatus(row), row.getId()));
|
||||
}
|
||||
output.getRecords().add(record);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean isForResultScreen)
|
||||
{
|
||||
ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>();
|
||||
willTest.addSelfToListIfAnyCount(lines);
|
||||
disabled.addSelfToListIfAnyCount(lines);
|
||||
lines.addAll(problems);
|
||||
return (lines);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Result: one line per provider.
|
||||
*******************************************************************************/
|
||||
public static class Load extends AbstractLoadStep implements ProcessSummaryProviderInterface
|
||||
{
|
||||
private final List<ProcessSummaryLineInterface> lines = new ArrayList<>();
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
|
||||
{
|
||||
AssistantRuntime runtime = AssistantRuntime.of(QContext.getQInstance());
|
||||
if(runtime == null)
|
||||
{
|
||||
throw (new QException("The assistant is not running on this server."));
|
||||
}
|
||||
String userId = AssistantSecurity.userId(QContext.getQSession());
|
||||
for(QRecord record : input.getRecords())
|
||||
{
|
||||
AssistantProvider row = new AssistantProvider(record);
|
||||
ConnectionTest.Result result = ConnectionTest.run(runtime, row, userId);
|
||||
if(result.ok())
|
||||
{
|
||||
lines.add(single(Status.OK, row.getName() + ": connected to " + result.model() + " in " + result.latencyMs() + " ms", row.getId()));
|
||||
}
|
||||
else
|
||||
{
|
||||
lines.add(single(Status.ERROR, row.getName() + ": " + result.error(), row.getId()));
|
||||
}
|
||||
output.getRecords().add(record);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean isForResultScreen)
|
||||
{
|
||||
return (new ArrayList<>(lines));
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
+66
@@ -0,0 +1,66 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package com.kingsrook.qbits.assistant.provider;
|
||||
|
||||
|
||||
import java.util.List;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A kind of model provider an admin can configure in the assistant_provider
|
||||
** table (Anthropic now; OpenAI-compatible, Bedrock and others plug in the
|
||||
** same way). Hosts register types with AssistantQBitConfig.withProviderType;
|
||||
** the provider settings screen offers the registered types and their models.
|
||||
**
|
||||
** create() receives the API key already resolved from its reference; it must
|
||||
** never log the key or put it in toString or exception messages.
|
||||
*******************************************************************************/
|
||||
public interface AssistantProviderType
|
||||
{
|
||||
|
||||
/*******************************************************************************
|
||||
** The type's id, stored in assistant_provider.providerType (e.g. "anthropic").
|
||||
*******************************************************************************/
|
||||
String getType();
|
||||
|
||||
/*******************************************************************************
|
||||
** A label for the settings screen.
|
||||
*******************************************************************************/
|
||||
String getLabel();
|
||||
|
||||
/*******************************************************************************
|
||||
** The models offered for this type, best default first.
|
||||
*******************************************************************************/
|
||||
List<String> getModels();
|
||||
|
||||
/*******************************************************************************
|
||||
** The environment variable that holds the key by default (e.g.
|
||||
** ANTHROPIC_API_KEY).
|
||||
*******************************************************************************/
|
||||
String getDefaultApiKeyEnvName();
|
||||
|
||||
/*******************************************************************************
|
||||
** The provider's own API base URL (shown when no override is set).
|
||||
*******************************************************************************/
|
||||
String getDefaultBaseUrl();
|
||||
|
||||
/*******************************************************************************
|
||||
** Build a provider. apiKey is the resolved secret (never null or empty);
|
||||
** baseUrl is null for the type's default.
|
||||
*******************************************************************************/
|
||||
AssistantModelProvider create(String apiKey, String baseUrl);
|
||||
|
||||
}
|
||||
+32
@@ -43,6 +43,7 @@ public class ModelResponse
|
||||
private String stopReason;
|
||||
private ModelUsage usage = new ModelUsage();
|
||||
private String requestId;
|
||||
private String model;
|
||||
|
||||
@JsonIgnore
|
||||
private transient Object providerState;
|
||||
@@ -213,4 +214,35 @@ public class ModelResponse
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Getter for model (the model that answered, when the provider reports it)
|
||||
*******************************************************************************/
|
||||
public String getModel()
|
||||
{
|
||||
return (this.model);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Setter for model
|
||||
*******************************************************************************/
|
||||
public void setModel(String model)
|
||||
{
|
||||
this.model = model;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fluent setter for model
|
||||
*******************************************************************************/
|
||||
public ModelResponse withModel(String model)
|
||||
{
|
||||
this.model = model;
|
||||
return (this);
|
||||
}
|
||||
|
||||
}
|
||||
+8
-2
@@ -62,7 +62,7 @@ class AssistantQBitProducerTest extends BaseAssistantTest
|
||||
@Test
|
||||
void testProducedMetaData()
|
||||
{
|
||||
List<String> owned = List.of("assistant_conversation", "assistant_message", "assistant_turn", "assistant_turn_event", "assistant_tool_call", "assistant_feedback", "assistant_usage_daily");
|
||||
List<String> owned = List.of("assistant_conversation", "assistant_message", "assistant_turn", "assistant_turn_event", "assistant_tool_call", "assistant_feedback", "assistant_usage_daily", "assistant_usage");
|
||||
for(String name : owned)
|
||||
{
|
||||
QTableMetaData table = qInstance.getTable(name);
|
||||
@@ -81,7 +81,13 @@ class AssistantQBitProducerTest extends BaseAssistantTest
|
||||
QSecurityKeyType keyType = qInstance.getSecurityKeyType(AssistantQBitConfig.DEFAULT_OWNER_KEY_TYPE);
|
||||
assertThat(keyType.getAllAccessKeyName()).isEqualTo("assistant_user_id_all_access");
|
||||
|
||||
assertThat(qInstance.getApp(AssistantMetaDataBuilder.APP_NAME).getChildren()).hasSize(9);
|
||||
//////////////////////////////////////////////////////////////////////
|
||||
// the nav: the home plus the main tables; processes are actions //
|
||||
//////////////////////////////////////////////////////////////////////
|
||||
assertThat(qInstance.getApp(AssistantMetaDataBuilder.APP_NAME).getChildren()).extracting(c -> c.getName())
|
||||
.containsExactly("assistant_provider", "assistant_conversation", "assistant_model_price", "assistant_policy");
|
||||
assertThat(qInstance.getApp(AssistantMetaDataBuilder.APP_NAME).getWidgets()).containsExactly("assistantHome");
|
||||
assertThat(qInstance.getProcess(AssistantMetaDataBuilder.RETENTION_PROCESS_NAME).getTableName()).isEqualTo("assistant_conversation");
|
||||
assertThat(qInstance.getProcess(AssistantMetaDataBuilder.RETENTION_PROCESS_NAME)).isNotNull();
|
||||
assertThat(qInstance.getProcess(AssistantMetaDataBuilder.RETENTION_PROCESS_NAME).getSchedule()).isNull();
|
||||
assertThat(qInstance.getPossibleValueSource(AssistantConversation.TABLE_NAME)).isNotNull();
|
||||
|
||||
+4
-3
@@ -101,14 +101,15 @@ class AssistantRecordScreensTest extends BaseAssistantTest
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A conversation lists its messages, turns, tool calls and feedback.
|
||||
** A conversation lists its messages, turns, tool calls, feedback and model
|
||||
** calls.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void testConversationChildLists()
|
||||
{
|
||||
QTableMetaData conversation = qInstance.getTable(AssistantConversation.TABLE_NAME);
|
||||
List<String> widgets = conversation.getSections().stream().map(QFieldSection::getWidgetName).filter(name -> name != null).toList();
|
||||
assertThat(widgets).hasSize(4);
|
||||
assertThat(widgets).hasSize(5);
|
||||
for(String widget : widgets)
|
||||
{
|
||||
assertThat(qInstance.getWidget(widget)).as(widget).isNotNull();
|
||||
@@ -118,7 +119,7 @@ class AssistantRecordScreensTest extends BaseAssistantTest
|
||||
// a second pass does not add them again //
|
||||
///////////////////////////////////////////////////
|
||||
AssistantRecordScreens.addConversationChildLists(qInstance);
|
||||
assertThat(conversation.getSections().stream().filter(section -> section.getWidgetName() != null)).hasSize(4);
|
||||
assertThat(conversation.getSections().stream().filter(section -> section.getWidgetName() != null)).hasSize(5);
|
||||
}
|
||||
|
||||
|
||||
|
||||
+573
@@ -0,0 +1,573 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.settings;
|
||||
|
||||
|
||||
import java.io.Serializable;
|
||||
import java.math.BigDecimal;
|
||||
import java.util.HashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import com.kingsrook.qbits.assistant.AssistantQBitConfig;
|
||||
import com.kingsrook.qbits.assistant.api.AssistantApiException;
|
||||
import com.kingsrook.qbits.assistant.engine.ConnectionTest;
|
||||
import com.kingsrook.qbits.assistant.engine.ModelPricing;
|
||||
import com.kingsrook.qbits.assistant.engine.ProviderService;
|
||||
import com.kingsrook.qbits.assistant.engine.UsageRecorder;
|
||||
import com.kingsrook.qbits.assistant.home.AssistantHome;
|
||||
import com.kingsrook.qbits.assistant.home.AssistantHomeData;
|
||||
import com.kingsrook.qbits.assistant.home.AssistantHomeView;
|
||||
import com.kingsrook.qbits.assistant.metadata.AssistantRecordScreens;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantModelPrice;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantProvider;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantUsage;
|
||||
import com.kingsrook.qbits.assistant.processes.TestConnectionProcess;
|
||||
import com.kingsrook.qbits.assistant.provider.AssistantModelProvider;
|
||||
import com.kingsrook.qbits.assistant.provider.AssistantProviderType;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelProviderException;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelUsage;
|
||||
import com.kingsrook.qbits.assistant.testing.AssistantTestFixture;
|
||||
import com.kingsrook.qbits.assistant.testing.BaseAssistantTest;
|
||||
import com.kingsrook.qbits.assistant.testing.TurnRunner;
|
||||
import com.kingsrook.qqq.backend.core.actions.ai.AiExposure;
|
||||
import com.kingsrook.qqq.backend.core.actions.ai.AiTableMetaData;
|
||||
import com.kingsrook.qqq.backend.core.actions.dashboard.RenderWidgetAction;
|
||||
import com.kingsrook.qqq.backend.core.actions.processes.QProcessCallbackFactory;
|
||||
import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessOutput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertOutput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
|
||||
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.RawHTML;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
|
||||
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
|
||||
import com.kingsrook.qqq.backend.core.utils.JsonUtils;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The provider settings, key handling, Test connection, per-call usage, cost
|
||||
** math and the app home.
|
||||
*******************************************************************************/
|
||||
class AssistantSettingsTest extends BaseAssistantTest
|
||||
{
|
||||
static final String FAKE_TYPE = "fake";
|
||||
static final String KEY = "sk-test-SECRET-VALUE-0123456789-wxyz";
|
||||
|
||||
private final Map<String, String> secrets = new HashMap<>();
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A provider type whose providers are the test's scripted provider.
|
||||
*******************************************************************************/
|
||||
class FakeType implements AssistantProviderType
|
||||
{
|
||||
String lastKey;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getType()
|
||||
{
|
||||
return (FAKE_TYPE);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getLabel()
|
||||
{
|
||||
return ("Fake");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public List<String> getModels()
|
||||
{
|
||||
return (List.of("fake-model-1", "fake-model-2"));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getDefaultApiKeyEnvName()
|
||||
{
|
||||
return ("FAKE_API_KEY");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public String getDefaultBaseUrl()
|
||||
{
|
||||
return ("https://fake.example");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public AssistantModelProvider create(String apiKey, String baseUrl)
|
||||
{
|
||||
lastKey = apiKey;
|
||||
return (provider);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
private final FakeType fakeType = new FakeType();
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
protected void customizeConfig(AssistantQBitConfig config)
|
||||
{
|
||||
secrets.put("FAKE_API_KEY", KEY);
|
||||
secrets.put("DATABASE_PASSWORD", "db-secret-should-never-leak");
|
||||
config.withProviderType(fakeType).withSecretResolver(secrets::get).withPolicyCacheSeconds(0);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private void asAdmin()
|
||||
{
|
||||
QContext.init(qInstance, AssistantTestFixture.sessionFor(AssistantTestFixture.ADMIN));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private InsertOutput insertProvider(Map<String, Serializable> values) throws Exception
|
||||
{
|
||||
QRecord record = new QRecord().withValue("name", "Fake").withValue("providerType", FAKE_TYPE).withValue("model", "fake-model-1")
|
||||
.withValue("enabled", true).withValue("isDefault", true);
|
||||
values.forEach(record::setValue);
|
||||
return (new InsertAction().execute(new InsertInput(AssistantProvider.TABLE_NAME).withRecord(record)));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private Integer providerId() throws Exception
|
||||
{
|
||||
InsertOutput output = insertProvider(Map.of());
|
||||
assertThat(output.getRecords().get(0).getErrors()).isEmpty();
|
||||
return (output.getRecords().get(0).getValueInteger("id"));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Tables, nav, actions, child lists and exposure.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void testMetaData()
|
||||
{
|
||||
for(String name : List.of(AssistantProvider.TABLE_NAME, AssistantModelPrice.TABLE_NAME, AssistantUsage.TABLE_NAME))
|
||||
{
|
||||
QTableMetaData table = qInstance.getTable(name);
|
||||
assertThat(table).as(name).isNotNull();
|
||||
assertThat(table.getIsHidden()).isFalse();
|
||||
assertThat(AiTableMetaData.of(table).getExposure()).isEqualTo(AiExposure.NONE);
|
||||
}
|
||||
assertThat(qInstance.getTable(AssistantUsage.TABLE_NAME).getRecordSecurityLocks()).hasSize(1);
|
||||
assertThat(qInstance.getTable(AssistantProvider.TABLE_NAME).getField("apiKeyStatus").getIsEditable()).isFalse();
|
||||
assertThat(qInstance.getProcess(TestConnectionProcess.NAME).getTableName()).isEqualTo(AssistantProvider.TABLE_NAME);
|
||||
assertThat(qInstance.getProcess(TestConnectionProcess.NAME).getLabel()).isEqualTo("Test connection");
|
||||
assertThat(qInstance.getWidget(AssistantHome.WIDGET_NAME)).isNotNull();
|
||||
assertThat(qInstance.getPossibleValueSource(AssistantRecordScreens.PROVIDER_TYPE_PVS_NAME).getEnumValues()).extracting(v -> String.valueOf(v.getId())).containsExactly(FAKE_TYPE);
|
||||
assertThat(qInstance.getPossibleValueSource(AssistantRecordScreens.MODEL_PVS_NAME).getEnumValues()).extracting(v -> String.valueOf(v.getId())).contains("fake-model-1", "claude-sonnet-5-5");
|
||||
assertThat(qInstance.getTable(AssistantProvider.TABLE_NAME).getSections()).anyMatch(s -> s.getWidgetName() != null);
|
||||
assertThat(qInstance.getTable(AssistantProvider.TABLE_NAME).getSections()).anyMatch(s -> s.getFieldNames() != null && s.getFieldNames().contains("apiKeyStatus"));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** No read path returns the key: query, get, the entity and its JSON hold
|
||||
** only the masked status.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void testKeyNeverReturned() throws Exception
|
||||
{
|
||||
asAdmin();
|
||||
Integer id = providerId();
|
||||
|
||||
List<QRecord> queried = new QueryAction().execute(new QueryInput(AssistantProvider.TABLE_NAME).withFilter(new QQueryFilter())).getRecords();
|
||||
QRecord got = new GetAction().execute(new GetInput(AssistantProvider.TABLE_NAME).withPrimaryKey(id)).getRecord();
|
||||
for(QRecord record : List.of(queried.get(0), got))
|
||||
{
|
||||
assertThat(record.getValueString("apiKeyStatus")).isEqualTo("Set • …wxyz (from FAKE_API_KEY)");
|
||||
assertThat(JsonUtils.toJson(record.getValues())).doesNotContain(KEY).doesNotContain("SECRET");
|
||||
assertThat(JsonUtils.toJson(new AssistantProvider(record))).doesNotContain(KEY);
|
||||
}
|
||||
|
||||
//////////////////////////////////////////////////////////////
|
||||
// the status is never stored, even if a caller sends one //
|
||||
//////////////////////////////////////////////////////////////
|
||||
new UpdateAction().execute(new UpdateInput(AssistantProvider.TABLE_NAME).withRecord(new QRecord().withValue("id", id).withValue("apiKeyStatus", "forged").withValue("name", "Renamed")));
|
||||
secrets.remove("FAKE_API_KEY");
|
||||
QRecord after = new GetAction().execute(new GetInput(AssistantProvider.TABLE_NAME).withPrimaryKey(id)).getRecord();
|
||||
assertThat(after.getValueString("apiKeyStatus")).isEqualTo("Not set (FAKE_API_KEY is empty)");
|
||||
assertThat(after.getValueString("name")).isEqualTo("Renamed");
|
||||
|
||||
assertThat(ProviderService.mask("short")).isEqualTo("Set");
|
||||
assertThat(ProviderService.mask(null)).isEqualTo("Not set");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A key pasted into the variable field, an unrelated secret's variable and
|
||||
** an unknown base URL are all refused (and nothing is stored).
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void testUnsafeSettingsRefused() throws Exception
|
||||
{
|
||||
asAdmin();
|
||||
InsertOutput pasted = insertProvider(Map.of("apiKeyEnvName", KEY));
|
||||
assertThat(pasted.getRecords().get(0).getErrors()).isNotEmpty();
|
||||
assertThat(pasted.getRecords().get(0).getErrors().get(0).getMessage()).contains("NAME of an environment variable").doesNotContain(KEY);
|
||||
|
||||
InsertOutput other = insertProvider(Map.of("name", "Other", "apiKeyEnvName", "DATABASE_PASSWORD"));
|
||||
assertThat(other.getRecords().get(0).getErrors().get(0).getMessage()).contains("not allowed");
|
||||
|
||||
InsertOutput url = insertProvider(Map.of("name", "Url", "baseUrl", "https://evil.example"));
|
||||
assertThat(url.getRecords().get(0).getErrors().get(0).getMessage()).contains("not allowed");
|
||||
|
||||
InsertOutput badType = insertProvider(Map.of("name", "Type", "providerType", "nope"));
|
||||
assertThat(badType.getRecords().get(0).getErrors().get(0).getMessage()).contains("not installed");
|
||||
|
||||
InsertOutput limits = insertProvider(Map.of("name", "Limits", "monthlyCostCapUsd", new BigDecimal("-1"), "maxOutputTokensPerRequest", 0, "maxConcurrentRequests", 0));
|
||||
assertThat(limits.getRecords().get(0).getErrors()).hasSize(3);
|
||||
|
||||
assertThat(new QueryAction().execute(new QueryInput(AssistantProvider.TABLE_NAME).withFilter(new QQueryFilter())).getRecords()).isEmpty();
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Marking a provider default unmarks the others; the default enabled one is
|
||||
** active; with no rows the configured provider is used.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void testActiveProvider() throws Exception
|
||||
{
|
||||
asAdmin();
|
||||
ProviderService service = config.getRuntime().getProviderService();
|
||||
ProviderService.Active fallback = service.resolve("main-model");
|
||||
assertThat(fallback.row()).isNull();
|
||||
assertThat(fallback.model()).isEqualTo("main-model");
|
||||
|
||||
Integer first = providerId();
|
||||
Integer second = insertProvider(Map.of("name", "Second", "model", "fake-model-2")).getRecords().get(0).getValueInteger("id");
|
||||
assertThat(service.activeRow().getId()).isEqualTo(second);
|
||||
assertThat(new GetAction().execute(new GetInput(AssistantProvider.TABLE_NAME).withPrimaryKey(first)).getRecord().getValueBoolean("isDefault")).isFalse();
|
||||
|
||||
ProviderService.Active active = service.resolve("main-model");
|
||||
assertThat(active.providerId()).isEqualTo(second);
|
||||
assertThat(active.model()).isEqualTo("fake-model-2");
|
||||
assertThat(active.provider()).isSameAs(provider);
|
||||
assertThat(fakeType.lastKey).isEqualTo(KEY);
|
||||
|
||||
new UpdateAction().execute(new UpdateInput(AssistantProvider.TABLE_NAME).withRecord(new QRecord().withValue("id", second).withValue("enabled", false)));
|
||||
assertThat(service.activeRow().getId()).isEqualTo(first);
|
||||
|
||||
secrets.remove("FAKE_API_KEY");
|
||||
assertThat(service.resolve("main-model")).isNull();
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Test connection (a record action): Result lines, the saved result, and a
|
||||
** TEST usage row - success and failure.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void testTestConnectionAction() throws Exception
|
||||
{
|
||||
asAdmin();
|
||||
Integer id = providerId();
|
||||
|
||||
provider.thenText("OK");
|
||||
List<ProcessSummaryLineInterface> lines = runTest(id);
|
||||
assertThat(lines).anyMatch(l -> l.getStatus() == Status.OK && l.getMessage().contains("connected to fake-model-1"));
|
||||
AssistantProvider row = config.getRuntime().getStore().getProvider(id);
|
||||
assertThat(row.getLastTestStatus()).isEqualTo("OK");
|
||||
assertThat(row.getLastTestModel()).isEqualTo("fake-model-1");
|
||||
assertThat(row.getLastTestLatencyMs()).isNotNull();
|
||||
assertThat(provider.getRequests().get(provider.getRequests().size() - 1).getMaxTokens()).isEqualTo(ConnectionTest.PING_MAX_TOKENS);
|
||||
|
||||
provider.thenError(ModelProviderException.Kind.AUTHENTICATION, "invalid x-api-key");
|
||||
lines = runTest(id);
|
||||
assertThat(lines).anyMatch(l -> l.getStatus() == Status.ERROR && l.getMessage().contains("API key was refused"));
|
||||
row = config.getRuntime().getStore().getProvider(id);
|
||||
assertThat(row.getLastTestStatus()).isEqualTo("ERROR");
|
||||
assertThat(row.getLastTestError()).contains("AUTHENTICATION").doesNotContain(KEY);
|
||||
|
||||
List<QRecord> usage = new QueryAction().execute(new QueryInput(AssistantUsage.TABLE_NAME).withFilter(new QQueryFilter()
|
||||
.withCriteria("purpose", QCriteriaOperator.EQUALS, UsageRecorder.PURPOSE_TEST))).getRecords();
|
||||
assertThat(usage).extracting(r -> r.getValueString("status")).containsExactlyInAnyOrder("OK", "ERROR");
|
||||
assertThat(usage).allMatch(r -> AssistantTestFixture.ADMIN.equals(r.getValueString("userId")) && id.equals(r.getValueInteger("providerId")));
|
||||
|
||||
////////////////////////////////////////////////////////////////
|
||||
// a row that cannot be used fails before any call //
|
||||
////////////////////////////////////////////////////////////////
|
||||
int calls = provider.getCallCount();
|
||||
secrets.remove("FAKE_API_KEY");
|
||||
lines = runTest(id);
|
||||
assertThat(lines).anyMatch(l -> l.getStatus() == Status.ERROR && l.getMessage().contains("No API key"));
|
||||
assertThat(provider.getCallCount()).isEqualTo(calls);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Run Test connection on one provider; the Result screen's lines.
|
||||
*******************************************************************************/
|
||||
@SuppressWarnings("unchecked")
|
||||
private List<ProcessSummaryLineInterface> runTest(Integer id) throws Exception
|
||||
{
|
||||
RunProcessInput input = new RunProcessInput();
|
||||
input.setProcessName(TestConnectionProcess.NAME);
|
||||
input.setCallback(QProcessCallbackFactory.forPrimaryKey("id", id));
|
||||
input.setFrontendStepBehavior(RunProcessInput.FrontendStepBehavior.SKIP);
|
||||
RunProcessOutput output = new RunProcessAction().execute(input);
|
||||
List<ProcessSummaryLineInterface> review = (List<ProcessSummaryLineInterface>) output.getValues().get(StreamedETLWithFrontendProcess.FIELD_VALIDATION_SUMMARY);
|
||||
assertThat(review).anyMatch(l -> l.getMessage() != null && l.getMessage().contains("API key"));
|
||||
return ((List<ProcessSummaryLineInterface>) output.getValues().get(StreamedETLWithFrontendProcess.FIELD_PROCESS_SUMMARY));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Every model call of a turn writes a usage row, priced from the price
|
||||
** table; the per-request output cap applies; the monthly cap stops turns.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void testTurnUsageAndLimits() throws Exception
|
||||
{
|
||||
asAdmin();
|
||||
Integer id = insertProvider(Map.of("maxOutputTokensPerRequest", 500)).getRecords().get(0).getValueInteger("id");
|
||||
new InsertAction().execute(new InsertInput(AssistantModelPrice.TABLE_NAME).withRecord(new QRecord()
|
||||
.withValue("providerType", FAKE_TYPE).withValue("model", "fake-model-1")
|
||||
.withValue("inputUsdPerMTok", new BigDecimal("3")).withValue("outputUsdPerMTok", new BigDecimal("15")).withValue("cacheReadUsdPerMTok", new BigDecimal("0.3"))));
|
||||
|
||||
provider.thenToolUse("list_tables", Map.of()).thenText("There are some tables.");
|
||||
TurnRunner.Outcome outcome = runner.run(AssistantTestFixture.EDITOR, "what tables are there?");
|
||||
assertThat(outcome.errorCode()).isNull();
|
||||
assertThat(provider.getRequests()).allMatch(r -> r.getMaxTokens() == 500 && "fake-model-1".equals(r.getModel()));
|
||||
|
||||
asAdmin();
|
||||
List<QRecord> usage = new QueryAction().execute(new QueryInput(AssistantUsage.TABLE_NAME).withFilter(new QQueryFilter()
|
||||
.withCriteria("turnId", QCriteriaOperator.EQUALS, outcome.turnId()))).getRecords();
|
||||
assertThat(usage).hasSize(2);
|
||||
for(QRecord row : usage)
|
||||
{
|
||||
assertThat(row.getValueString("purpose")).isEqualTo("TURN");
|
||||
assertThat(row.getValueString("status")).isEqualTo("OK");
|
||||
assertThat(row.getValueInteger("providerId")).isEqualTo(id);
|
||||
assertThat(row.getValueInteger("conversationId")).isEqualTo(outcome.conversationId());
|
||||
assertThat(row.getValueString("userId")).isEqualTo(AssistantTestFixture.EDITOR);
|
||||
//////////////////////////////////////////////////////////////
|
||||
// 1200 in x $3 + 150 out x $15 + 3000 cache read x $0.30 //
|
||||
//////////////////////////////////////////////////////////////
|
||||
assertThat(row.getValueLong("costMicros")).isEqualTo(3600 + 2250 + 900);
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////
|
||||
// month to date is $0.01350; a $0.01 cap refuses new turns //
|
||||
////////////////////////////////////////////////////////////////
|
||||
assertThat(config.getRuntime().getProviderService().monthToDateCostMicros(id)).isEqualTo(13_500);
|
||||
new UpdateAction().execute(new UpdateInput(AssistantProvider.TABLE_NAME).withRecord(new QRecord().withValue("id", id).withValue("monthlyCostCapUsd", new BigDecimal("0.01"))));
|
||||
provider.thenText("never");
|
||||
int calls = provider.getCallCount();
|
||||
assertThatThrownBy(() -> runner.run(AssistantTestFixture.EDITOR, "again"))
|
||||
.isInstanceOfSatisfying(AssistantApiException.class, e -> assertThat(e.getCode()).isEqualTo("BUDGET"));
|
||||
assertThat(provider.getCallCount()).isEqualTo(calls);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A failed model call is recorded with its kind.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void testFailedCallRecorded() throws Exception
|
||||
{
|
||||
asAdmin();
|
||||
providerId();
|
||||
provider.thenError(ModelProviderException.Kind.OVERLOADED, "overloaded");
|
||||
TurnRunner.Outcome outcome = runner.run(AssistantTestFixture.EDITOR, "hello");
|
||||
assertThat(outcome.errorCode()).isEqualTo("PROVIDER");
|
||||
|
||||
asAdmin();
|
||||
List<QRecord> usage = new QueryAction().execute(new QueryInput(AssistantUsage.TABLE_NAME).withFilter(new QQueryFilter())).getRecords();
|
||||
assertThat(usage).hasSize(1);
|
||||
assertThat(usage.get(0).getValueString("status")).isEqualTo("ERROR");
|
||||
assertThat(usage.get(0).getValueString("errorKind")).isEqualTo("OVERLOADED");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Cost math: price rows override the config, cache defaults, formatting.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void testCostMath() throws Exception
|
||||
{
|
||||
asAdmin();
|
||||
ProviderService service = config.getRuntime().getProviderService();
|
||||
ModelUsage usage = new ModelUsage().withInputTokens(1_000_000L).withOutputTokens(100_000L).withCacheReadTokens(1_000_000L).withCacheWriteTokens(0L);
|
||||
|
||||
////////////////////////////////////////////////
|
||||
// no row: config defaults (Sonnet 5.5 $2/$10) //
|
||||
////////////////////////////////////////////////
|
||||
assertThat(service.pricing("anthropic", "claude-sonnet-5-5").costMicros(usage)).isEqualTo(2_000_000 + 1_000_000 + 200_000);
|
||||
assertThat(service.pricing("x", "unknown-model").costMicros(usage)).isZero();
|
||||
|
||||
new InsertAction().execute(new InsertInput(AssistantModelPrice.TABLE_NAME).withRecord(new QRecord()
|
||||
.withValue("providerType", "anthropic").withValue("model", "claude-sonnet-5-5")
|
||||
.withValue("inputUsdPerMTok", new BigDecimal("1")).withValue("outputUsdPerMTok", new BigDecimal("5"))));
|
||||
ModelPricing row = service.pricing("anthropic", "claude-sonnet-5-5");
|
||||
assertThat(row.getCacheReadUsdPerMTok()).isEqualTo(0.1);
|
||||
assertThat(row.getCacheWrite5mUsdPerMTok()).isEqualTo(1.25);
|
||||
assertThat(row.costMicros(usage)).isEqualTo(1_000_000 + 500_000 + 100_000);
|
||||
assertThat(service.pricing("other-type", "claude-sonnet-5-5").costMicros(usage)).isEqualTo(3_200_000);
|
||||
|
||||
assertThat(ModelPricing.defaults()).containsKeys("claude-fable-5-1", "claude-haiku-4-5");
|
||||
assertThat(ProviderService.capMicros(new AssistantProvider().withMonthlyCostCapUsd(new BigDecimal("12.345678")))).isEqualTo(12_345_678L);
|
||||
assertThat(ProviderService.capMicros(null)).isNull();
|
||||
assertThat(AssistantHomeView.money(0)).isEqualTo("$0.00");
|
||||
assertThat(AssistantHomeView.money(5)).isEqualTo("<$0.01");
|
||||
assertThat(AssistantHomeView.money(12_345_678)).isEqualTo("$12.35");
|
||||
assertThat(AssistantHomeView.money(1_234_000_000L)).isEqualTo("$1,234.00");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The home with nothing in it: honest empty states, no tables.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void testHomeEmpty() throws Exception
|
||||
{
|
||||
asAdmin();
|
||||
String html = render();
|
||||
assertThat(html).contains("No providers yet").contains("No assistant use in the last 30 days").contains("No tool calls yet")
|
||||
.contains("No failed model calls").contains("No one has used the assistant this month").doesNotContain("<table");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The home with real rows: providers with health, usage, cost, tools and
|
||||
** failures.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void testHomeWithData() throws Exception
|
||||
{
|
||||
asAdmin();
|
||||
Integer id = insertProvider(Map.of("monthlyCostCapUsd", new BigDecimal("50"))).getRecords().get(0).getValueInteger("id");
|
||||
provider.thenText("OK");
|
||||
ConnectionTest.run(config.getRuntime(), config.getRuntime().getStore().getProvider(id), AssistantTestFixture.ADMIN);
|
||||
|
||||
provider.thenToolUse("list_tables", Map.of()).thenText("Here.");
|
||||
runner.run(AssistantTestFixture.EDITOR, "what tables?");
|
||||
provider.thenToolUse("describe_table", Map.of("table", "nope")).thenError(ModelProviderException.Kind.SERVER, "boom");
|
||||
runner.run(AssistantTestFixture.ANALYST, "describe nope");
|
||||
|
||||
asAdmin();
|
||||
AssistantHomeData data = AssistantHomeData.load(config.getRuntime());
|
||||
assertThat(data.getProviders()).hasSize(1);
|
||||
assertThat(data.getProviders().get(0).active()).isTrue();
|
||||
assertThat(data.getProviders().get(0).keyPresent()).isTrue();
|
||||
assertThat(data.getByUser()).extracting(t -> t.key).contains(AssistantTestFixture.EDITOR, AssistantTestFixture.ANALYST, AssistantTestFixture.ADMIN);
|
||||
assertThat(data.getByTool()).extracting(t -> t.name).contains("list_tables", "describe_table");
|
||||
assertThat(data.getFailedCalls()).hasSize(1);
|
||||
assertThat(data.getFailedTools()).hasSize(1);
|
||||
assertThat(data.errorRatePercent7d()).isEqualTo(25);
|
||||
assertThat(data.getDays().get(data.getToday()).conversations).isEqualTo(2);
|
||||
assertThat(data.getMonthRequests()).isEqualTo(5);
|
||||
assertThat(data.getMonthCostMicros()).isZero();
|
||||
|
||||
String html = render();
|
||||
assertThat(html).contains("Answering").contains("Fake").contains("In use").contains("Last test passed").contains("…wxyz")
|
||||
.contains("of $50.00 cap").contains("list_tables").contains("Model call #").contains("25% of 4 requests").doesNotContain(KEY).doesNotContain("<table");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Render the home widget as the current session.
|
||||
*******************************************************************************/
|
||||
private String render() throws Exception
|
||||
{
|
||||
RenderWidgetInput input = new RenderWidgetInput();
|
||||
input.setWidgetMetaData(qInstance.getWidget(AssistantHome.WIDGET_NAME));
|
||||
RenderWidgetOutput output = new RenderWidgetAction().execute(input);
|
||||
return (((RawHTML) output.getWidgetData()).getHtml());
|
||||
}
|
||||
|
||||
}
|
||||
Reference in new issue
Block a user