Compare commits
5
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
57673131ff
|
||
|
|
c27994c432
|
||
|
|
14bd4e377c | ||
|
|
cb7d5d4d5a
|
||
|
|
51e0692f46 |
No files matched your search
@@ -1,5 +1,14 @@
|
||||
# Changelog
|
||||
|
||||
## Unreleased - Answer presentation
|
||||
|
||||
- Add `present_results`: server-formatted metrics, tables, computed percentages,
|
||||
charts and validated in-app filter/record/action links.
|
||||
- Persist and stream presentation parts with a multi-line GFM fallback and
|
||||
record-data retention; add a static default presentation skill.
|
||||
- Add the owner's 81 JPEG / 20 PNG golden example, shared live eval coverage,
|
||||
permission/formatting/input tests and strict-schema budget coverage.
|
||||
|
||||
## Unreleased - P-e
|
||||
|
||||
- Add per-profile heuristic/classifier model routing, with budgeted ROUTE usage.
|
||||
@@ -11,6 +20,8 @@
|
||||
|
||||
## Unreleased - P-d
|
||||
|
||||
- Require QQQ RC.5 and send budget alerts directly through core QAlerts, with admin-visible alert type metadata and core sink tests.
|
||||
|
||||
- Admin budgets, expiring overrides, deduplicated warning/downgrade/stop events.
|
||||
- Per-dispatch limits preserving PHI routing and existing caps.
|
||||
- Home budget cards, 30-day cost/token bars and provider breakdowns.
|
||||
|
||||
@@ -9,7 +9,7 @@ Design spec: `QRun-IO/qqq` `docs/specs/2026-10-11-qqq-assistant-qbit-design.md`.
|
||||
| `com.kingsrook.qbits:qbit-assistant` | The QBit: conversation tables, turn engine, `/qqq/v1/assistant` routes (JSON + Server-Sent Events), budgets, kill switch, the provider SPI, and a scripted provider for tests and demos. |
|
||||
| `com.kingsrook.qbits:qbit-assistant-provider-anthropic` | The Claude provider (`com.anthropic:anthropic-java`). Add it to use Claude. The core QBit has no LLM SDK dependency. |
|
||||
|
||||
The data access itself is done by the AI tool kernel in QQQ core (`com.kingsrook.qqq.backend.core.actions.ai.AiToolKernel`). The kernel checks table permissions and exposure rules, applies record security, and caps and redacts what the model sees. Every tool result reaches the model wrapped as untrusted data.
|
||||
Data tools call the QQQ UI API through `UiApiToolbox` and `UiApiGateway` with the signed-in user's session. The UI enforces permissions and record security; the assistant further narrows access and caps and redacts returned values. Data tool results reach the model wrapped as untrusted data. Only record-free navigation and help use the core AI tool kernel.
|
||||
|
||||
## Requirements
|
||||
|
||||
@@ -203,6 +203,7 @@ Base path `/qqq/v1/assistant`. Every request is authenticated through the app's
|
||||
The stream sends these events, each with a `seq`:
|
||||
- `message.start`, `text.delta`
|
||||
- `tool.start`, `tool.result` (with `render` and `recordRefs`)
|
||||
- `presentation` (`messageId`, `part`: summary, blocks and server-built GFM markdown)
|
||||
- `ui.navigate`, `notice`
|
||||
- `usage`, `message.end`
|
||||
- `error`
|
||||
@@ -385,7 +386,7 @@ Apache 2.0.
|
||||
|
||||
## Data classification and HIPAA mode
|
||||
|
||||
On QQQ 4.1.0-RC.4, classify application tables/fields in the host configuration:
|
||||
On QQQ 4.1.0-RC.5, classify application tables/fields in the host configuration:
|
||||
|
||||
```java
|
||||
import com.kingsrook.qbits.assistant.classification.DataClassification;
|
||||
@@ -455,7 +456,7 @@ extra allowance or temporary unlimited use from the budget's Overrides child lis
|
||||
with a required reason and expiry; the session sets the grantor.
|
||||
|
||||
Budget state changes appear once per scope/period/state in the Events child list,
|
||||
as panel notices and, when the optional QQQ Alerts SPI is installed, bell alerts.
|
||||
as panel notices and, through core QAlerts, bell alerts when an alerts sink is installed (logging otherwise).
|
||||
The Assistant home shows budgets, overrides, daily cost and tokens, and breakdowns
|
||||
by user, model, provider and tool. Existing `AssistantBudgets`/policy limits and
|
||||
provider monthly caps continue to apply independently of these rows and overrides.
|
||||
@@ -487,3 +488,16 @@ NETWORK/TIMEOUT marks a provider DOWN for 60 seconds; success marks it UP. Local
|
||||
health is probed when unknown/stale with a two-second timeout. Provider cards show
|
||||
health and down-until, failures show fallback totals, and turn records show the
|
||||
route and final destination. See [UPGRADE.md](UPGRADE.md) for the additive schema.
|
||||
|
||||
### Answer presentation
|
||||
|
||||
The always-available `present_results` tool turns retrieved results into metrics,
|
||||
tables, charts and actionable in-app links. The server validates the user's UI API
|
||||
visibility, formats field values, computes shares and builds all links. It emits
|
||||
and stores a `presentation` part with a complete GFM Markdown fallback. Profile
|
||||
skills can extend the built-in presentation instructions. See the
|
||||
[part, stream and input contract](docs/answer-presentation.md).
|
||||
|
||||
The image-breakdown golden task also runs in the existing opt-in live provider suite:
|
||||
`./mvnw verify -Dassistant.live=true -Dtest.excludedGroups= -Dgroups=live -Djacoco.skip=true -pl qbit-assistant-provider-anthropic -am`
|
||||
(with `ANTHROPIC_API_KEY` set). Live calls are excluded from normal verification.
|
||||
@@ -28,6 +28,8 @@ All configuration remains assistant.admin-only, with AI exposure NONE.
|
||||
|
||||
## P-d: data-driven budgets
|
||||
|
||||
Requires QQQ 4.1.0-RC.5 for the core Alerts API and alert type metadata.
|
||||
|
||||
Create `assistant_budget`, `assistant_budget_override`, `assistant_budget_event`
|
||||
and `assistant_budget_usage`
|
||||
using the P-d DDL in [schema-postgres.sql](docs/schema-postgres.sql) before deploying.
|
||||
|
||||
+37
-2
@@ -387,8 +387,10 @@ AI exposure NONE is not enough, because the assistant acts as the user and an ad
|
||||
reason, session grantor and expiry; they never waive the other limit layers.
|
||||
- Start-turn checks use the existing 429 BUDGET path through PolicyService.
|
||||
Every dispatch, including strict retries, evaluates current usage and overrides.
|
||||
Events deduplicate WARNED, DOWNGRADED and STOPPED; optional reflective QAlerts
|
||||
signals accompany panel notices without a compile-time Alerts dependency.
|
||||
Events deduplicate WARNED, DOWNGRADED and STOPPED.
|
||||
- QQQ RC.5 core QAlerts sends budget signals alongside panel notices, with three
|
||||
admin-visible alert types and a default logging sink. Resolution awaits active
|
||||
alert/recurrence tracking so overrides cannot clear shared alerts incorrectly.
|
||||
- Explicit registry model selection leaves provider defaults untouched. Automatic
|
||||
downgrades prefer LOCAL, then price. PHI eligibility and destination provider caps
|
||||
still apply; a missing eligible target stops the turn.
|
||||
@@ -444,3 +446,36 @@ AI exposure NONE is not enough, because the assistant acts as the user and an ad
|
||||
- an "Explain access" action (for a user, and a table or a prompt, which operations the assistant could call and which it would be denied, with the reason);
|
||||
- a recent-denials widget on the app home.
|
||||
- **(d) Spec'd assistant endpoints:** move `/qqq/v1/assistant/...` (conversations, turns, events, feedback, cancel, config, actions) onto `AbstractEndpointSpec`s through the qbit spec extension point (`MiddlewareSpecExtension`, established prefix `/assistant`, QRun-IO/qqq#1128). The SSE turn and events streams stay raw routes until the spec layer can describe a stream.
|
||||
|
||||
## Answer presentation (owner priority 2026-10-13)
|
||||
|
||||
`present_results` is always offered and reads no records. It takes a direct answer,
|
||||
metrics, a table, an optional table-derived chart, filtered list links, record links
|
||||
and next-step actions. The server validates metadata and the user's UI API table
|
||||
visibility once per turn, formats values using QQQ field types/display formats,
|
||||
computes percentages and builds every `/app` href. Unknown or denied items are
|
||||
dropped with an explanation. The assistant must still retrieve every fact through
|
||||
data tools before presenting it.
|
||||
|
||||
The [presentation part and SSE contract](answer-presentation.md) defines `summary`,
|
||||
`blocks` and the complete server-built GFM `markdown` fallback. Each successful
|
||||
call emits `presentation` with `{messageId, part}` and persists that part; the tool
|
||||
response tells the model not to repeat its tables or links. Audit rows contain
|
||||
only accepted presentation data. Normal record-data retention removes these parts.
|
||||
|
||||
The built-in Presentation skill is compact static prompt text in the cached rules
|
||||
block. It asks for a one-sentence answer first, metrics for single numbers, tables
|
||||
for breakdowns/comparisons, charts for useful shares/trends and actual filter links
|
||||
whenever offering a list. Administrator profile skills append after it as before
|
||||
and extend it; there is no replacement boolean or extra permission surface.
|
||||
|
||||
The owner's one-line table was a frontend parsing defect: Next's `AssistantMarkdown`
|
||||
used react-markdown without `remark-gfm`, so table lines became a paragraph.
|
||||
The renderer fix belongs to `qqq-frontend-next`; this worktree preserves real
|
||||
newlines and provides structured blocks plus a proper multi-line GFM fallback.
|
||||
The frontend change is maintained separately and is not verified by this build.
|
||||
|
||||
The shared golden task asks "How many images do I have by type?": five UI count
|
||||
calls establish 81 JPEG, 20 PNG and zero other configured types, then a presentation
|
||||
with two rows, 80.2%/19.8% shares, two pie points and working MIME filter links.
|
||||
The same task is included in the opt-in `live` Anthropic eval suite.
|
||||
@@ -0,0 +1,56 @@
|
||||
# Assistant answer presentation contract (v1)
|
||||
|
||||
Owner priority (2026-10-13): answers must ALWAYS be easy to read and act on. Bad example the owner saw, rendered on ONE line:
|
||||
"| Type | Count | |---|---| | JPEG (image/jpeg) | 81 | | PNG (image/png) | 20 | | WebP, GIF, SVG | 0 |" followed by "You can open the lists filtered by JPEG or PNG" with NO links.
|
||||
|
||||
Root cause found in Next: `AssistantMarkdown` uses react-markdown WITHOUT `remark-gfm`, so GFM tables are not parsed and the table's lines collapse into one paragraph (soft line breaks). Backend streaming preserves `\n` (events are JSON).
|
||||
|
||||
## New message part: `presentation`
|
||||
|
||||
An assistant message's `parts` (persisted in `assistant_message.parts_json`, streamed as an SSE event, returned by the conversation API) may contain, besides `text`, `notice`, `tool`/`render` parts etc., ONE part per `present_results` call:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "presentation",
|
||||
"summary": "101 media files are images: 81 JPEG and 20 PNG.",
|
||||
"markdown": "<server-built GFM fallback of the whole presentation, for clients that do not render blocks (CLI/TUI)>",
|
||||
"blocks": [
|
||||
{ "kind": "metrics", "items": [ { "label": "Images", "value": 101, "display": "101", "href": "/app/media?filter=..." } ] },
|
||||
{ "kind": "table", "title": "Media by type",
|
||||
"columns": [ { "key": "type", "label": "Type", "align": "left" },
|
||||
{ "key": "count", "label": "Count", "align": "right" },
|
||||
{ "key": "share", "label": "Share", "align": "right" } ],
|
||||
"rows": [ { "cells": { "type": "JPEG (image/jpeg)", "count": 81, "share": 0.802 },
|
||||
"display": { "type": "JPEG (image/jpeg)", "count": "81", "share": "80.2%" },
|
||||
"href": "/app/media?filter=..." } ] },
|
||||
{ "kind": "chart", "chartType": "pie", "title": "Share by type",
|
||||
"points": [ { "label": "JPEG", "value": 81, "display": "81", "href": "/app/media?filter=..." },
|
||||
{ "label": "PNG", "value": 20, "display": "20", "href": "/app/media?filter=..." } ] },
|
||||
{ "kind": "filterLinks", "items": [ { "label": "Open JPEG media (81)", "tableName": "media", "href": "/app/media?filter=..." } ] },
|
||||
{ "kind": "recordLinks", "items": [ { "label": "Sunset.jpg", "tableName": "media", "recordId": "12", "href": "/app/media/12" } ] },
|
||||
{ "kind": "actions", "items": [ { "label": "Show the PNG files", "prompt": "Show me the PNG media files" },
|
||||
{ "label": "Open media", "href": "/app/media" } ] }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Rules:
|
||||
- `summary` is required, one sentence, the direct answer first.
|
||||
- `chartType` ∈ `bar`, `pie`, `line`; ≤ 12 points; values are numbers.
|
||||
- Every `href` is built SERVER-SIDE from metadata and is an `/app` path (same rules as `TurnEngine.isAppHref`). Filtered list links are `/app/<tableName>?filter=<URL-encoded JSON {"criteria":[{"fieldName":..,"operator":..,"values":[..]}],"booleanOperator":"AND"}>`, which Next's `deserializeFilter` accepts (JSON starting with `{`). The model never writes URLs.
|
||||
- `display` strings are formatted server-side from field metadata (`displayFormat`, e.g. `%,d`, `$%,.2f`; percent columns as `80.2%`); clients show `display` when present, else the raw value.
|
||||
- `actions` items have either `prompt` (send as the user's next message) or `href` (navigate), never both.
|
||||
- Unknown `kind`s must be ignored by clients (forward compatible); clients that do not render blocks render `markdown`.
|
||||
- The SSE event is `presentation` with `{ "messageId": <id>, "part": <the part above> }`; Next's reducer appends it to the assistant message's parts.
|
||||
|
||||
## Tool input and implementation
|
||||
|
||||
The input schema is [present-results.schema.json](../qbit-assistant/src/main/resources/present-results.schema.json). `table.rowKeys` supplies one raw filter value per row; `table.rowFilter` names the table, field and optional operator (EQUALS by default). Columns receive stable server keys `c0`, `c1`, etc. Input cells remain raw scalars; each row has a parallel display map. The example keys above are illustrative.
|
||||
|
||||
`PresentResultsTool` is instantiated per turn. It caches `GET /metaData` with that turn's session, checks table read visibility and instance fields, validates QQQ criteria operators and builds paths using RC.5 `AiLinkBuilder`. Filter links use URL-encoded JSON accepted by Next. Record ids are encoded as opaque path segments. Metadata denial or invalid item drops that item (an invalid table drops its table/chart), with a short explanation to the model; other items survive. No data query or permission elevation happens here.
|
||||
|
||||
Limits: summary 300 characters, display text 2,000 characters, table 20 columns/200 rows, chart 12 points, item lists 100 entries, filters 20 criteria/200 values. Chart truncation is reported to the model. Percent columns reference a raw numeric column; a zero total yields 0.0%. Numbers still must originate in earlier data tool results; presentation is formatting, not a new source of facts.
|
||||
|
||||
The schema uses small named, nonrecursive definitions to avoid deeply inlined structures. Each object component is at most two levels of inline containers; the resolved table/criteria structure necessarily has the nesting shown in the owner contract. Strict is off by default. Opting `present_results` into strict mode requires all properties (empty arrays/strings, empty table and -1 for unused percentage indexes), removes provider-unsupported length constraints, and retains server validation. This has zero optional parameters and one shared scalar union in the existing strict-budget test.
|
||||
|
||||
Presentation parts are persisted with the assistant message and replayed as `presentation` events with `{messageId, part}` (plus the standard event sequence). They are removed by record-data retention along with tool renders. Model history and text-only consumers can use the server-built Markdown. Unknown block kinds should be ignored. Plain labels must be rendered as text, never parsed as links or HTML; fallback Markdown escapes them. Tool audit input records the accepted presentation only, excluding dropped values.
|
||||
@@ -193,7 +193,7 @@
|
||||
</build>
|
||||
|
||||
<profiles>
|
||||
<!-- The assistant needs QQQ 4.1.0-RC.4: the AI tool kernel (package -->
|
||||
<!-- The assistant needs QQQ 4.1.0-RC.5: the AI tool kernel (package -->
|
||||
<!-- actions.ai, qqq#1078) and the developer panels (qqq#1076). Until -->
|
||||
<!-- qbit-build-parent imports the 4.1 BOM, this -->
|
||||
<!-- profile is active by default and imports qqq-bom-pom ahead of the -->
|
||||
@@ -205,7 +205,7 @@
|
||||
<activeByDefault>true</activeByDefault>
|
||||
</activation>
|
||||
<properties>
|
||||
<qqq.snapshot.version>4.1.0-RC.4</qqq.snapshot.version>
|
||||
<qqq.snapshot.version>4.1.0-RC.5</qqq.snapshot.version>
|
||||
</properties>
|
||||
<dependencyManagement>
|
||||
<dependencies>
|
||||
|
||||
+18
@@ -108,6 +108,24 @@ class AnthropicRequestContractTest
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Presentation remains inside the actual serialized strict request budget.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void testPresentationStrictOptIn() throws Exception
|
||||
{
|
||||
Map<String, Object> body = requestBody(new AssistantQBitConfig().withModelProvider(new ScriptedModelProvider()).withStrictToolNames(Set.of("present_results")));
|
||||
assertThat(violations(body)).isEmpty();
|
||||
assertThat(strictBudgetViolations(body)).isEmpty();
|
||||
assertThat(tools(body)).anySatisfy(tool ->
|
||||
{
|
||||
assertThat(tool.get("name")).isEqualTo("present_results");
|
||||
assertThat(tool.get("strict")).isEqualTo(true);
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The RC.3 shape (every tool strict) is recognized as over budget, and a
|
||||
** host can still opt a few tools in.
|
||||
|
||||
@@ -28,6 +28,9 @@ import com.kingsrook.qbits.assistant.metadata.AssistantRecordScreens;
|
||||
import com.kingsrook.qbits.assistant.processes.DiscoverModelsProcess;
|
||||
import com.kingsrook.qbits.assistant.processes.TestConnectionProcess;
|
||||
import com.kingsrook.qbits.assistant.processes.TestModelConnectionProcess;
|
||||
import com.kingsrook.qqq.backend.core.alerts.AlertSeverity;
|
||||
import com.kingsrook.qqq.backend.core.alerts.QAlertTypeMetaData;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantBudget;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.logging.QLogger;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerMultiOutput;
|
||||
@@ -155,6 +158,9 @@ public class AssistantQBitProducer implements QBitMetaDataProducer<AssistantQBit
|
||||
qInstance.addTable(customized);
|
||||
tables.add(customized);
|
||||
}
|
||||
qInstance.add(budgetAlertType("warning", "Assistant budget warning", "Assistant usage has reached the budget warning threshold.", AlertSeverity.WARNING));
|
||||
qInstance.add(budgetAlertType("downgraded", "Assistant budget downgraded", "Assistant usage has reached the budget limit and switched to a cheaper model.", AlertSeverity.WARNING));
|
||||
qInstance.add(budgetAlertType("stopped", "Assistant budget stopped", "Assistant dispatch has stopped because a budget limit was reached.", AlertSeverity.ERROR));
|
||||
qInstance.addPossibleValueSource(builder.buildConversationPossibleValueSource());
|
||||
AssistantRecordScreens.buildPossibleValueSources().forEach(qInstance::addPossibleValueSource);
|
||||
AssistantRecordScreens.buildSettingsPossibleValueSources(qBitConfig).forEach(qInstance::addPossibleValueSource);
|
||||
@@ -185,6 +191,18 @@ public class AssistantQBitProducer implements QBitMetaDataProducer<AssistantQBit
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Declare budget alerts as top-level core metadata for admin visibility.
|
||||
*******************************************************************************/
|
||||
private QAlertTypeMetaData budgetAlertType(String state, String label, String description, AlertSeverity severity)
|
||||
{
|
||||
return (new QAlertTypeMetaData().withName("assistant.budget." + state).withLabel(label).withDescription(description)
|
||||
.withDefaultSeverity(severity).withHowToFix("Raise the budget or grant an override in Assistant → Budgets")
|
||||
.withSourceQBitName(ARTIFACT_ID).withVisibilityPermission("assistant.admin").withSubjectTableName(AssistantBudget.TABLE_NAME));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Build the runtime and register the routes.
|
||||
*******************************************************************************/
|
||||
|
||||
+7
-59
@@ -17,23 +17,16 @@
|
||||
package com.kingsrook.qbits.assistant.engine;
|
||||
|
||||
|
||||
import com.kingsrook.qqq.backend.core.logging.QLogger;
|
||||
import java.lang.reflect.Method;
|
||||
import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
|
||||
import com.kingsrook.qqq.backend.core.alerts.AlertSeverity;
|
||||
import com.kingsrook.qqq.backend.core.alerts.AlertSignal;
|
||||
import com.kingsrook.qqq.backend.core.alerts.QAlerts;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Optional Alerts producer SPI, with log-only fallback and no dependency.
|
||||
** Budget signals through core's non-throwing Alerts API and default logging sink.
|
||||
*******************************************************************************/
|
||||
public final class BudgetAlerts
|
||||
{
|
||||
private static final QLogger LOG = QLogger.getLogger(BudgetAlerts.class);
|
||||
private static final String QALERTS = "com.kingsrook.qqq.backend.core.alerts.QAlerts";
|
||||
private static final String ALERT_SIGNAL = "com.kingsrook.qqq.backend.core.alerts.AlertSignal";
|
||||
private static final String SEVERITY = "com.kingsrook.qqq.backend.core.alerts.AlertSeverity";
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
@@ -44,55 +37,10 @@ public final class BudgetAlerts
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** True when the Alerts SPI is on the classpath.
|
||||
** Raise one deduplicated budget transition through the configured core sink.
|
||||
*******************************************************************************/
|
||||
public static boolean available()
|
||||
public static void raise(String type, String dedupKey, AlertSeverity severity, String title, String summary)
|
||||
{
|
||||
try
|
||||
{
|
||||
Class.forName(QALERTS, false, BudgetAlerts.class.getClassLoader());
|
||||
return (true);
|
||||
}
|
||||
catch(ClassNotFoundException | LinkageError e)
|
||||
{
|
||||
return (false);
|
||||
}
|
||||
QAlerts.raise(new AlertSignal(type).withDedupKey(dedupKey).withSeverity(severity).withTitle(title).withSummary(summary));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** QAlerts.raise(new AlertSignal(type).withDedupKey(..).withSeverity(..)
|
||||
** .withTitle(..).withSummary(..)), or a log line.
|
||||
*******************************************************************************/
|
||||
public static void raise(String type, String dedupKey, String severity, String title, String summary)
|
||||
{
|
||||
LOG.info("Assistant budget alert", logPair("type", type), logPair("dedupKey", dedupKey), logPair("title", title));
|
||||
if(!available())
|
||||
{
|
||||
return;
|
||||
}
|
||||
try
|
||||
{
|
||||
ClassLoader loader = BudgetAlerts.class.getClassLoader();
|
||||
Class<?> signalType = Class.forName(ALERT_SIGNAL, true, loader);
|
||||
Object signal = signalType.getConstructor(String.class).newInstance(type);
|
||||
signal = signalType.getMethod("withDedupKey", String.class).invoke(signal, dedupKey);
|
||||
Class<?> severityType = Class.forName(SEVERITY, true, loader);
|
||||
@SuppressWarnings({"unchecked", "rawtypes"})
|
||||
Object level = Enum.valueOf((Class<? extends Enum>) severityType.asSubclass(Enum.class), severity);
|
||||
signal = signalType.getMethod("withSeverity", severityType).invoke(signal, level);
|
||||
signal = signalType.getMethod("withTitle", String.class).invoke(signal, title);
|
||||
signal = signalType.getMethod("withSummary", String.class).invoke(signal, summary);
|
||||
Method raise = Class.forName(QALERTS, true, loader).getMethod("raise", signalType);
|
||||
raise.invoke(null, signal);
|
||||
}
|
||||
catch(Exception | LinkageError e)
|
||||
{
|
||||
LOG.info("Could not raise a assistant budget alert through the Alerts SPI", logPair("type", type), logPair("reason", e.getClass().getSimpleName()));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
}
|
||||
+7
-28
@@ -24,6 +24,7 @@ import com.kingsrook.qbits.assistant.model.AssistantBudgetUsage;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantModel;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantUsage;
|
||||
import com.kingsrook.qbits.assistant.persistence.AssistantStore;
|
||||
import com.kingsrook.qqq.backend.core.alerts.AlertSeverity;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
|
||||
@@ -48,21 +49,6 @@ public class BudgetService
|
||||
private final AssistantStore store;
|
||||
private final ProviderService providers;
|
||||
private final Clock clock;
|
||||
private final AlertRaiser alerts;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Injectable alert seam; production uses the optional Alerts producer SPI.
|
||||
*******************************************************************************/
|
||||
@FunctionalInterface
|
||||
public interface AlertRaiser
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Attempt to raise one deduplicated transition.
|
||||
*******************************************************************************/
|
||||
void raise(String type, String key, String severity, String title, String summary);
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -104,21 +90,10 @@ public class BudgetService
|
||||
** Production constructor.
|
||||
*******************************************************************************/
|
||||
public BudgetService(AssistantStore store, ProviderService providers, Clock clock)
|
||||
{
|
||||
this(store, providers, clock, BudgetAlerts::raise);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Constructor with an alert seam for tests.
|
||||
*******************************************************************************/
|
||||
public BudgetService(AssistantStore store, ProviderService providers, Clock clock, AlertRaiser alerts)
|
||||
{
|
||||
this.store = store;
|
||||
this.providers = providers;
|
||||
this.clock = clock;
|
||||
this.alerts = alerts;
|
||||
}
|
||||
|
||||
|
||||
@@ -318,6 +293,10 @@ public class BudgetService
|
||||
|
||||
/*******************************************************************************
|
||||
** The persisted unique key is also the warning/alert deduplication record.
|
||||
** Resolution on period rollover or override recovery needs an active-alert
|
||||
** lifecycle: these immutable events cannot re-raise a resolved state within
|
||||
** the same period, and user-specific overrides must not resolve shared alerts.
|
||||
** Track active keys and recurrence before calling QAlerts.resolve here.
|
||||
*******************************************************************************/
|
||||
private boolean transition(Snapshot snapshot, String state, String userId, String message) throws QException
|
||||
{
|
||||
@@ -333,8 +312,8 @@ public class BudgetService
|
||||
case "DOWNGRADED" -> "downgraded";
|
||||
default -> "stopped";
|
||||
};
|
||||
alerts.raise(type, snapshot.budget().getId() + ":" + snapshot.scopeKey() + ":" + snapshot.periodStart(),
|
||||
"STOPPED".equals(state) ? "ERROR" : "WARNING", "Assistant budget " + snapshot.budget().getName(), message);
|
||||
BudgetAlerts.raise(type, snapshot.budget().getId() + ":" + snapshot.scopeKey() + ":" + snapshot.periodStart(),
|
||||
"STOPPED".equals(state) ? AlertSeverity.ERROR : AlertSeverity.WARNING, "Assistant budget " + snapshot.budget().getName(), message);
|
||||
return (true);
|
||||
}
|
||||
|
||||
|
||||
@@ -34,6 +34,7 @@ import com.kingsrook.qbits.assistant.provider.ModelPart;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelRequest;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelToolDefinition;
|
||||
import com.kingsrook.qbits.assistant.provider.SystemBlock;
|
||||
import com.kingsrook.qbits.assistant.tools.PresentResultsTool;
|
||||
import com.kingsrook.qbits.assistant.tools.UiApiToolbox;
|
||||
import com.kingsrook.qqq.backend.core.actions.ai.AiToolDefinition;
|
||||
import com.kingsrook.qqq.backend.core.actions.ai.AiToolKernel;
|
||||
@@ -116,6 +117,7 @@ public class ContextBuilder
|
||||
.withStrict(isStrict(definition.getName(), !Boolean.FALSE.equals(definition.getStrict()))));
|
||||
}
|
||||
}
|
||||
byName.put(PresentResultsTool.NAME, PresentResultsTool.definition(isStrict(PresentResultsTool.NAME, true)));
|
||||
return (new ArrayList<>(byName.values()));
|
||||
}
|
||||
|
||||
|
||||
+19
-4
@@ -31,7 +31,7 @@ import com.kingsrook.qqq.backend.core.utils.StringUtils;
|
||||
** assistantMessage.partsJson and returned by the API:
|
||||
** {type:"text", text} | {type:"tool", toolCallId, name, summary, outcome, render?}
|
||||
** | {type:"record_refs", refs:[...]} | {type:"navigate", href}
|
||||
** | {type:"notice", level, text}
|
||||
** | {type:"notice", level, text} | {type:"presentation", summary, blocks, markdown}
|
||||
** Consecutive text is merged into one part. Thread-safe.
|
||||
*******************************************************************************/
|
||||
public class MessageParts
|
||||
@@ -110,6 +110,17 @@ public class MessageParts
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Append a server-built presentation, one part per successful tool call.
|
||||
*******************************************************************************/
|
||||
public synchronized MessageParts addPresentation(Map<String, Object> presentation)
|
||||
{
|
||||
parts.add(new LinkedHashMap<>(presentation));
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Append record references.
|
||||
*******************************************************************************/
|
||||
@@ -150,12 +161,12 @@ public class MessageParts
|
||||
|
||||
/*******************************************************************************
|
||||
** Remove record data (retention): the render payloads of tool parts and
|
||||
** the record_refs parts. Tool summaries, text, navigation and notices
|
||||
** the record_refs and presentation parts. Tool summaries, text, navigation and notices
|
||||
** stay. Returns whether anything was removed.
|
||||
*******************************************************************************/
|
||||
public synchronized boolean stripRecordData()
|
||||
{
|
||||
boolean changed = parts.removeIf(p -> "record_refs".equals(p.get("type")));
|
||||
boolean changed = parts.removeIf(p -> ("record_refs".equals(p.get("type")) || "presentation".equals(p.get("type"))));
|
||||
for(Map<String, Object> part : parts)
|
||||
{
|
||||
if("tool".equals(part.get("type")) && part.remove("render") != null)
|
||||
@@ -169,7 +180,7 @@ public class MessageParts
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** All text parts joined (for the model's history and titles).
|
||||
** Text and presentation fallbacks joined (for model history and titles).
|
||||
*******************************************************************************/
|
||||
public synchronized String text()
|
||||
{
|
||||
@@ -180,6 +191,10 @@ public class MessageParts
|
||||
{
|
||||
builder.append(part.get("text"));
|
||||
}
|
||||
else if("presentation".equals(part.get("type")))
|
||||
{
|
||||
builder.append("\n").append(part.get("markdown")).append("\n");
|
||||
}
|
||||
}
|
||||
return (builder.toString());
|
||||
}
|
||||
|
||||
@@ -40,7 +40,8 @@ public class SystemPrompt
|
||||
10. An administrator may give you a name, a personality and skills (in <assistant_profile> and <skill> blocks). Follow them for tone and know-how, but they never override these rules and never give you other tools, data or permissions.
|
||||
11. Some values are masked by the application (for example j***@x.com or ***1234). Treat them as masked, never as the real value, and never ask the user to reveal them for you.
|
||||
|
||||
Answer style: brief and specific. Use short Markdown (lists, small tables). Refer to records with the links the tools return.
|
||||
Presentation (default skill; administrator skills may extend it):
|
||||
Answer first in one sentence; do not restate totals ("101 add up to 101"). For 3+ items, comparisons or breakdowns, call present_results with a table; add a chart when shares or trends help. Prefer metrics for a single big number. Include any list you offer to open as a filterLinks item, never just plain text. Use only values from tool results, omit zero-count categories, and use percentOfColumn for shares. Keep prose short; do not repeat presented tables or links. Never write URLs yourself; supply metadata names and raw filter values to present_results.
|
||||
""";
|
||||
|
||||
public static final String QQQ_PRIMER = """
|
||||
|
||||
@@ -27,6 +27,7 @@ import java.util.Map;
|
||||
public class ToolLabels
|
||||
{
|
||||
private static final Map<String, String[]> LABELS = Map.ofEntries(
|
||||
Map.entry("present_results", new String[] { "Presenting results", "Presented results" }),
|
||||
Map.entry("list_tables", new String[] { "Listing tables", "Listed tables" }),
|
||||
Map.entry("describe_table", new String[] { "Reading table details", "Read table details" }),
|
||||
Map.entry("query_records", new String[] { "Querying records", "Queried records" }),
|
||||
|
||||
+3
-2
@@ -42,7 +42,7 @@ import com.kingsrook.qqq.backend.core.utils.JsonUtils;
|
||||
*******************************************************************************/
|
||||
public class TurnClassification
|
||||
{
|
||||
private static final Set<String> METADATA_TOOLS = Set.of("list_tables", "describe_table", "navigate", "search_help");
|
||||
private static final Set<String> METADATA_TOOLS = Set.of("list_tables", "describe_table", "navigate", "search_help", "present_results");
|
||||
private final ClassificationSource source;
|
||||
private final Set<String> tables = new LinkedHashSet<>();
|
||||
private DataClassification classification;
|
||||
@@ -65,7 +65,8 @@ public class TurnClassification
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Observe before appending any result to a model request. Metadata tools carry no record values.
|
||||
** Observe before appending results. Metadata reads no records; presentation
|
||||
** only reuses values whose originating data tools were already classified.
|
||||
*******************************************************************************/
|
||||
public void observe(ModelPart use, AiToolResult result)
|
||||
{
|
||||
|
||||
@@ -36,6 +36,7 @@ import com.kingsrook.qbits.assistant.provider.ModelRequest;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelResponse;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelUsage;
|
||||
import com.kingsrook.qbits.assistant.provider.StreamHandle;
|
||||
import com.kingsrook.qbits.assistant.tools.PresentResultsTool;
|
||||
import com.kingsrook.qbits.assistant.tools.UiApiToolbox;
|
||||
import com.kingsrook.qqq.backend.core.actions.ai.AiRecordRef;
|
||||
import com.kingsrook.qqq.backend.core.actions.ai.AiToolCall;
|
||||
@@ -101,6 +102,7 @@ public class TurnEngine implements Runnable
|
||||
|
||||
private static final Set<AiToolOutcome> ERROR_OUTCOMES = Set.of(AiToolOutcome.DENIED, AiToolOutcome.NOT_FOUND, AiToolOutcome.INVALID_INPUT, AiToolOutcome.ERROR);
|
||||
|
||||
private final PresentResultsTool presenter;
|
||||
private final AssistantRuntime runtime;
|
||||
private final AssistantQBitConfig config;
|
||||
private final AssistantStore store;
|
||||
@@ -146,6 +148,7 @@ public class TurnEngine implements Runnable
|
||||
AssistantTurn turn, String userText, Map<String, Object> pageContext, PolicyService.EffectivePolicy policy, RunningTurn running, String softBudgetNotice)
|
||||
{
|
||||
this.runtime = runtime;
|
||||
this.presenter = new PresentResultsTool(runtime.getQInstance());
|
||||
this.config = runtime.getConfig();
|
||||
this.store = runtime.getStore();
|
||||
this.session = session;
|
||||
@@ -689,7 +692,11 @@ public class TurnEngine implements Runnable
|
||||
{
|
||||
if(!running.isCancelled())
|
||||
{
|
||||
if(ContextBuilder.KERNEL_TOOLS.contains(call.getName()))
|
||||
if(PresentResultsTool.NAME.equals(call.getName()))
|
||||
{
|
||||
results[index] = presenter.execute(toolContext(), call.getId(), call.getInput());
|
||||
}
|
||||
else if(ContextBuilder.KERNEL_TOOLS.contains(call.getName()))
|
||||
{
|
||||
///////////////////////////////////////////////////
|
||||
// navigate / search_help: no records are read //
|
||||
@@ -798,7 +805,8 @@ public class TurnEngine implements Runnable
|
||||
}
|
||||
|
||||
Map<String, Object> data = map("messageId", assistantMessage.getId(), "toolCallId", use.getToolUseId(), "name", use.getToolName(), "outcome", outcome, "summary", summary);
|
||||
if(result.getRender() != null)
|
||||
Boolean presentation = PresentResultsTool.NAME.equals(use.getToolName()) && AiToolOutcome.OK.equals(result.getOutcome()) && result.getRender() != null;
|
||||
if(result.getRender() != null && !presentation)
|
||||
{
|
||||
data.put("render", result.getRender());
|
||||
}
|
||||
@@ -808,7 +816,12 @@ public class TurnEngine implements Runnable
|
||||
}
|
||||
emit(TurnEvent.TOOL_RESULT, data);
|
||||
|
||||
parts.addTool(use.getToolUseId(), use.getToolName(), summary, outcome, result.getRender());
|
||||
parts.addTool(use.getToolUseId(), use.getToolName(), summary, outcome, presentation ? null : result.getRender());
|
||||
if(presentation)
|
||||
{
|
||||
parts.addPresentation(result.getRender());
|
||||
emit(TurnEvent.PRESENTATION, map("messageId", assistantMessage.getId(), "part", result.getRender()));
|
||||
}
|
||||
parts.addRecordRefs(refs);
|
||||
|
||||
String href = result.getNavigateHref();
|
||||
@@ -826,7 +839,7 @@ public class TurnEngine implements Runnable
|
||||
.withUserId(userId)
|
||||
.withToolCallId(use.getToolUseId())
|
||||
.withToolName(use.getToolName())
|
||||
.withInputJson(truncate(redactInput(use.getInput()), MAX_INPUT_JSON_CHARS))
|
||||
.withInputJson(truncate(redactInput(PresentResultsTool.NAME.equals(use.getToolName()) ? (presentation ? result.getRender() : Map.of()) : use.getInput()), MAX_INPUT_JSON_CHARS))
|
||||
.withOutcome(outcome)
|
||||
.withPermissionCheck(result.getPermissionCheck())
|
||||
.withTableName(result.getTableName())
|
||||
|
||||
@@ -33,6 +33,7 @@ public class TurnEvent
|
||||
public static final String TEXT_DELTA = "text.delta";
|
||||
public static final String TOOL_START = "tool.start";
|
||||
public static final String TOOL_RESULT = "tool.result";
|
||||
public static final String PRESENTATION = "presentation";
|
||||
public static final String UI_NAVIGATE = "ui.navigate";
|
||||
public static final String NOTICE = "notice";
|
||||
public static final String USAGE = "usage";
|
||||
|
||||
+727
@@ -0,0 +1,727 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.tools;
|
||||
|
||||
|
||||
import java.io.InputStream;
|
||||
import java.math.BigDecimal;
|
||||
import java.math.MathContext;
|
||||
import java.net.URLEncoder;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.ArrayList;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Locale;
|
||||
import java.util.Map;
|
||||
import java.util.function.Function;
|
||||
import com.fasterxml.jackson.core.type.TypeReference;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelToolDefinition;
|
||||
import com.kingsrook.qbits.assistant.uiapi.UiApiCall;
|
||||
import com.kingsrook.qbits.assistant.uiapi.UiApiResult;
|
||||
import com.kingsrook.qqq.backend.core.actions.ai.AiToolOutcome;
|
||||
import com.kingsrook.qqq.backend.core.actions.ai.AiToolResult;
|
||||
import com.kingsrook.qqq.backend.core.actions.ai.output.AiLinkBuilder;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
|
||||
import com.kingsrook.qqq.backend.core.utils.JsonUtils;
|
||||
import com.kingsrook.qqq.backend.core.utils.ValueUtils;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Builds presentations from previously retrieved values, never reading records.
|
||||
** One instance belongs to one turn: UI metadata, including denials, is cached
|
||||
** only for that turn and session. All output links are built by the server.
|
||||
*******************************************************************************/
|
||||
public class PresentResultsTool
|
||||
{
|
||||
public static final String NAME = "present_results";
|
||||
|
||||
private final QInstance qInstance;
|
||||
private UiApiResult metadata;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Create a turn-scoped presenter.
|
||||
*******************************************************************************/
|
||||
public PresentResultsTool(QInstance qInstance)
|
||||
{
|
||||
this.qInstance = qInstance;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Stable schema. Strict opt-in requires empty placeholders instead of many
|
||||
** optional productions, keeping the existing strict grammar budget small.
|
||||
*******************************************************************************/
|
||||
public static ModelToolDefinition definition(Boolean strict)
|
||||
{
|
||||
try(InputStream stream = PresentResultsTool.class.getResourceAsStream("/present-results.schema.json"))
|
||||
{
|
||||
Map<String, Object> schema = JsonUtils.toObject(new String(stream.readAllBytes(), StandardCharsets.UTF_8), new TypeReference<Map<String, Object>>() {});
|
||||
if(strict)
|
||||
{
|
||||
requireProperties(schema);
|
||||
}
|
||||
return (new ModelToolDefinition().withName(NAME).withDescription("Present an answer as metrics, a table, a chart and actionable links. Use only data from tool results; never supply URLs.")
|
||||
.withInputSchema(schema).withStrict(strict));
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
throw (new IllegalStateException("Cannot load presentation schema", e));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Eliminate optional productions for strict mode without adding unions.
|
||||
*******************************************************************************/
|
||||
private static void requireProperties(Object node)
|
||||
{
|
||||
if(node instanceof Map<?, ?>)
|
||||
{
|
||||
Map<String, Object> schema = object(node);
|
||||
schema.remove("maxLength");
|
||||
if(schema.get("properties") instanceof Map<?, ?> properties)
|
||||
{
|
||||
schema.put("required", new ArrayList<>(properties.keySet()));
|
||||
}
|
||||
schema.values().forEach(PresentResultsTool::requireProperties);
|
||||
}
|
||||
else if(node instanceof List<?> list)
|
||||
{
|
||||
list.forEach(PresentResultsTool::requireProperties);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Preserve valid items when others are dropped; never echo rejected values.
|
||||
*******************************************************************************/
|
||||
public synchronized AiToolResult execute(UiApiToolbox.Context context, String callId, Map<String, Object> input)
|
||||
{
|
||||
AiToolResult result = new AiToolResult().withToolCallId(callId).withToolName(NAME);
|
||||
try
|
||||
{
|
||||
require(input != null && input.get("summary") instanceof String, "summary required");
|
||||
String summary = text(input.get("summary"));
|
||||
require(!summary.isBlank() && summary.length() <= 300, "summary must contain 1 to 300 characters");
|
||||
List<String> notes = new ArrayList<>();
|
||||
List<Map<String, Object>> blocks = new ArrayList<>();
|
||||
addItems(blocks, "metrics", input.get("metrics"), notes, item -> metric(context, item));
|
||||
Map<String, Object> table = null;
|
||||
if(input.get("table") != null)
|
||||
{
|
||||
try
|
||||
{
|
||||
table = table(context, object(input.get("table")));
|
||||
if(!list(table.get("columns")).isEmpty())
|
||||
{
|
||||
blocks.add(table);
|
||||
}
|
||||
}
|
||||
catch(IllegalArgumentException e)
|
||||
{
|
||||
notes.add("Dropped table: " + e.getMessage());
|
||||
}
|
||||
}
|
||||
if(input.get("chart") != null && table != null && !list(table.get("rows")).isEmpty())
|
||||
{
|
||||
try
|
||||
{
|
||||
blocks.add(chart(object(input.get("chart")), table, notes));
|
||||
}
|
||||
catch(IllegalArgumentException e)
|
||||
{
|
||||
notes.add("Dropped chart: " + e.getMessage());
|
||||
}
|
||||
}
|
||||
if(input.get("chart") != null && (table == null || list(table.get("rows")).isEmpty()))
|
||||
{
|
||||
notes.add("Dropped chart: requires a valid nonempty table.");
|
||||
}
|
||||
addItems(blocks, "filterLinks", input.get("filterLinks"), notes, item -> filterLink(context, item));
|
||||
addItems(blocks, "recordLinks", input.get("recordLinks"), notes, item -> recordLink(context, item));
|
||||
addItems(blocks, "actions", input.get("actions"), notes, item -> action(context, item));
|
||||
Map<String, Object> part = map("type", "presentation", "summary", summary, "blocks", blocks, "markdown", markdown(summary, blocks));
|
||||
return (result.withOutcome(AiToolOutcome.OK).withSummary("Presented results").withRender(part)
|
||||
.withPermissionCheck(metadata == null ? "presentation:no table references" : "uiApi:GET /metaData:" + metadata.status())
|
||||
.withModelContent("Presented to the user: " + summary + (summary.endsWith(".") ? "" : ".") + " Do not repeat the table or links in text." + (notes.isEmpty() ? "" : "\n" + String.join("\n", notes))));
|
||||
}
|
||||
catch(IllegalArgumentException e)
|
||||
{
|
||||
return (result.withOutcome(AiToolOutcome.INVALID_INPUT).withSummary("Invalid presentation").withModelContent(e.getMessage()));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Resolve tables through the same metadata operation and session as the UI.
|
||||
*******************************************************************************/
|
||||
private QTableMetaData visibleTable(UiApiToolbox.Context context, String name)
|
||||
{
|
||||
QTableMetaData table = qInstance.getTable(name);
|
||||
require(table != null, "unknown table " + name);
|
||||
if(metadata == null)
|
||||
{
|
||||
try
|
||||
{
|
||||
metadata = context.gateway().invoke(new UiApiCall("GET", "/metaData").withSessionUUID(context.sessionUUID()).withUserTimezone(context.userTimezone()));
|
||||
}
|
||||
catch(QException e)
|
||||
{
|
||||
metadata = new UiApiResult(503, "{}");
|
||||
}
|
||||
}
|
||||
require(metadata.isSuccess(), "GET /metaData answered HTTP " + metadata.status());
|
||||
Map<String, Object> tables = object(object(metadata.json()).get("tables"));
|
||||
require(tables.containsKey(name) && Boolean.TRUE.equals(object(tables.get(name)).get("readPermission")), "table not visible through UI API: " + name);
|
||||
return (table);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A field reference must be paired with a visible table reference.
|
||||
*******************************************************************************/
|
||||
private QFieldMetaData field(UiApiToolbox.Context context, Map<String, Object> item)
|
||||
{
|
||||
String tableName = text(item.get("tableName"));
|
||||
String fieldName = text(item.get("fieldName"));
|
||||
require(fieldName.isEmpty() || !tableName.isEmpty(), "fieldName requires tableName");
|
||||
QTableMetaData table = tableName.isEmpty() ? null : visibleTable(context, tableName);
|
||||
QFieldMetaData field = fieldName.isEmpty() ? null : table.getFields().get(fieldName);
|
||||
require(fieldName.isEmpty() || field != null, "unknown field " + fieldName);
|
||||
return (field);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A metric preserves its numeric value and adds the metadata display.
|
||||
*******************************************************************************/
|
||||
private Map<String, Object> metric(UiApiToolbox.Context context, Map<String, Object> item)
|
||||
{
|
||||
QFieldMetaData field = field(context, item);
|
||||
number(item.get("value"));
|
||||
return (map("label", label(item), "value", item.get("value"), "display", display(item.get("value"), field)));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Build table cells and percentages from raw values, never model displays.
|
||||
*******************************************************************************/
|
||||
private Map<String, Object> table(UiApiToolbox.Context context, Map<String, Object> input)
|
||||
{
|
||||
List<?> columns = list(input.get("columns"));
|
||||
List<?> rows = list(input.get("rows"));
|
||||
require(columns.size() <= 20 && rows.size() <= 200, "table exceeds 20 columns or 200 rows");
|
||||
require(rows.isEmpty() || !columns.isEmpty(), "rows require columns");
|
||||
List<QFieldMetaData> fields = new ArrayList<>();
|
||||
List<Map<String, Object>> outputColumns = new ArrayList<>();
|
||||
List<BigDecimal> totals = new ArrayList<>();
|
||||
for(Integer c = 0; c < columns.size(); c++)
|
||||
{
|
||||
Map<String, Object> column = object(columns.get(c));
|
||||
fields.add(field(context, column));
|
||||
BigDecimal total = BigDecimal.ZERO;
|
||||
Boolean numeric = !rows.isEmpty();
|
||||
for(Object row : rows)
|
||||
{
|
||||
require(list(row).size() == columns.size(), "row width must match columns");
|
||||
Object value = list(row).get(c);
|
||||
scalar(value);
|
||||
if(value instanceof Number)
|
||||
{
|
||||
total = total.add(number(value));
|
||||
}
|
||||
else
|
||||
{
|
||||
numeric = false;
|
||||
}
|
||||
}
|
||||
totals.add(numeric ? total : null);
|
||||
outputColumns.add(map("key", "c" + c, "label", label(column), "align", numeric || percentColumn(column) >= 0 ? "right" : "left"));
|
||||
}
|
||||
Map<String, Object> rowFilter = input.get("rowFilter") == null ? Map.of() : object(input.get("rowFilter"));
|
||||
List<?> rowKeys = input.get("rowKeys") == null ? List.of() : list(input.get("rowKeys"));
|
||||
Boolean hasFilter = !text(rowFilter.get("tableName")).isEmpty();
|
||||
require(!hasFilter || rowKeys.size() == rows.size(), "rowKeys must match rows");
|
||||
QFieldMetaData filterField = field(context, rowFilter);
|
||||
require(!hasFilter || filterField != null, "rowFilter requires a field");
|
||||
List<Map<String, Object>> outputRows = new ArrayList<>();
|
||||
for(Integer r = 0; r < rows.size(); r++)
|
||||
{
|
||||
Map<String, Object> cells = new LinkedHashMap<>();
|
||||
Map<String, Object> displays = new LinkedHashMap<>();
|
||||
for(Integer c = 0; c < columns.size(); c++)
|
||||
{
|
||||
Integer source = percentColumn(object(columns.get(c)));
|
||||
Object value = list(rows.get(r)).get(c);
|
||||
String formatted;
|
||||
if(source >= 0)
|
||||
{
|
||||
require(source < columns.size() && totals.get(source) != null && percentColumn(object(columns.get(source))) < 0, "percentOfColumn must reference a numeric raw column");
|
||||
BigDecimal share = totals.get(source).signum() == 0 ? BigDecimal.ZERO : number(list(rows.get(r)).get(source)).divide(totals.get(source), MathContext.DECIMAL64);
|
||||
value = share;
|
||||
formatted = String.format(Locale.US, "%.1f%%", share.multiply(BigDecimal.valueOf(100)));
|
||||
}
|
||||
else
|
||||
{
|
||||
formatted = display(value, fields.get(c));
|
||||
}
|
||||
cells.put("c" + c, value);
|
||||
displays.put("c" + c, formatted);
|
||||
}
|
||||
Map<String, Object> row = map("cells", cells, "display", displays);
|
||||
if(hasFilter)
|
||||
{
|
||||
Object key = rowKeys.get(r);
|
||||
scalar(key);
|
||||
Map<String, Object> criterion = map("fieldName", rowFilter.get("fieldName"), "operator", rowFilter.getOrDefault("operator", "EQUALS"), "values", java.util.Collections.singletonList(key));
|
||||
row.put("href", filterHref(context, text(rowFilter.get("tableName")), List.of(criterion)));
|
||||
}
|
||||
outputRows.add(row);
|
||||
}
|
||||
return (map("kind", "table", "title", text(input.get("title")), "columns", outputColumns, "rows", outputRows));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A chart uses the table's raw values, formatted values and row links.
|
||||
*******************************************************************************/
|
||||
private Map<String, Object> chart(Map<String, Object> input, Map<String, Object> table, List<String> notes)
|
||||
{
|
||||
String type = text(input.get("chartType"));
|
||||
require(List.of("bar", "pie", "line").contains(type), "invalid chartType");
|
||||
Integer labelColumn = index(input.get("labelColumn"));
|
||||
Integer valueColumn = index(input.get("valueColumn"));
|
||||
Integer width = list(table.get("columns")).size();
|
||||
require(labelColumn >= 0 && valueColumn >= 0 && labelColumn < width && valueColumn < width, "chart column index out of range");
|
||||
List<Map<String, Object>> points = new ArrayList<>();
|
||||
List<?> rows = list(table.get("rows"));
|
||||
for(Object entry : rows)
|
||||
{
|
||||
Map<String, Object> row = object(entry);
|
||||
Object value = object(row.get("cells")).get("c" + valueColumn);
|
||||
number(value);
|
||||
if(points.size() < 12)
|
||||
{
|
||||
Map<String, Object> point = map("label", object(row.get("display")).get("c" + labelColumn), "value", value, "display", object(row.get("display")).get("c" + valueColumn));
|
||||
if(row.containsKey("href"))
|
||||
{
|
||||
point.put("href", row.get("href"));
|
||||
}
|
||||
points.add(point);
|
||||
}
|
||||
}
|
||||
if(rows.size() > 12)
|
||||
{
|
||||
notes.add("Chart truncated to 12 points; the table retains all rows.");
|
||||
}
|
||||
return (map("kind", "chart", "chartType", type, "title", text(input.get("title")), "points", points));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A filtered list link, including only validated metadata and raw values.
|
||||
*******************************************************************************/
|
||||
private Map<String, Object> filterLink(UiApiToolbox.Context context, Map<String, Object> item)
|
||||
{
|
||||
String tableName = text(item.get("tableName"));
|
||||
return (map("label", label(item), "tableName", tableName, "href", filterHref(context, tableName, list(item.get("criteria")))));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Serialize the UI's JSON filter format. The table path uses core navigate's
|
||||
** helper; filter values stay raw inside the URL-encoded JSON.
|
||||
*******************************************************************************/
|
||||
private String filterHref(UiApiToolbox.Context context, String tableName, List<?> criteria)
|
||||
{
|
||||
visibleTable(context, tableName);
|
||||
require(!criteria.isEmpty() && criteria.size() <= 20, "filter requires 1 to 20 criteria");
|
||||
List<Map<String, Object>> output = new ArrayList<>();
|
||||
for(Object entry : criteria)
|
||||
{
|
||||
Map<String, Object> criterion = object(entry);
|
||||
String fieldName = text(criterion.get("fieldName"));
|
||||
require(!fieldName.isBlank(), "filter requires fieldName");
|
||||
field(context, map("tableName", tableName, "fieldName", fieldName));
|
||||
String operator = text(criterion.get("operator"));
|
||||
operator = operator.isEmpty() ? "EQUALS" : operator;
|
||||
try
|
||||
{
|
||||
QCriteriaOperator.valueOf(operator);
|
||||
}
|
||||
catch(IllegalArgumentException e)
|
||||
{
|
||||
throw (new IllegalArgumentException("invalid criteria operator"));
|
||||
}
|
||||
List<?> values = list(criterion.get("values"));
|
||||
require(values.size() <= 200, "too many filter values");
|
||||
values.forEach(PresentResultsTool::scalar);
|
||||
output.add(map("fieldName", fieldName, "operator", operator, "values", values));
|
||||
}
|
||||
return (safeHref(AiLinkBuilder.objectHref(tableName) + "?filter=" + URLEncoder.encode(JsonUtils.toJson(map("criteria", output, "booleanOperator", "AND")), StandardCharsets.UTF_8)));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Record ids are opaque path segments, encoded by core's link helper.
|
||||
*******************************************************************************/
|
||||
private Map<String, Object> recordLink(UiApiToolbox.Context context, Map<String, Object> item)
|
||||
{
|
||||
String tableName = text(item.get("tableName"));
|
||||
visibleTable(context, tableName);
|
||||
Object id = item.get("recordId");
|
||||
require(id instanceof String || id instanceof Number, "recordId must be a string or number");
|
||||
String recordId = text(id);
|
||||
require(!recordId.isBlank() && !List.of(".", "..").contains(recordId), "invalid recordId");
|
||||
return (map("label", label(item), "tableName", tableName, "recordId", recordId, "href", safeHref(AiLinkBuilder.recordHref(tableName, recordId))));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Actions are either a next-message prompt or a visible table destination.
|
||||
*******************************************************************************/
|
||||
private Map<String, Object> action(UiApiToolbox.Context context, Map<String, Object> item)
|
||||
{
|
||||
String prompt = text(item.get("prompt"));
|
||||
String destination = text(item.get("navigateTo"));
|
||||
require(prompt.isBlank() != destination.isBlank(), "action requires exactly one of prompt or navigateTo");
|
||||
if(!destination.isBlank())
|
||||
{
|
||||
visibleTable(context, destination);
|
||||
return (map("label", label(item), "href", safeHref(AiLinkBuilder.objectHref(destination))));
|
||||
}
|
||||
return (map("label", label(item), "prompt", prompt));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Preserve independently valid items and report dropped positions.
|
||||
*******************************************************************************/
|
||||
private static void addItems(List<Map<String, Object>> blocks, String kind, Object input, List<String> notes, Function<Map<String, Object>, Map<String, Object>> build)
|
||||
{
|
||||
if(input == null)
|
||||
{
|
||||
return;
|
||||
}
|
||||
List<?> entries = list(input);
|
||||
require(entries.size() <= 100, "too many " + kind);
|
||||
List<Map<String, Object>> items = new ArrayList<>();
|
||||
for(Integer i = 0; i < entries.size(); i++)
|
||||
{
|
||||
try
|
||||
{
|
||||
items.add(build.apply(object(entries.get(i))));
|
||||
}
|
||||
catch(IllegalArgumentException e)
|
||||
{
|
||||
notes.add("Dropped " + kind + "[" + i + "]: " + e.getMessage());
|
||||
}
|
||||
}
|
||||
if(!items.isEmpty())
|
||||
{
|
||||
blocks.add(map("kind", kind, "items", items));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Metadata formats use QQQ field coercion and Java display format strings.
|
||||
*******************************************************************************/
|
||||
private static String display(Object value, QFieldMetaData field)
|
||||
{
|
||||
if(value == null)
|
||||
{
|
||||
return ("");
|
||||
}
|
||||
if(field != null)
|
||||
{
|
||||
try
|
||||
{
|
||||
Object typed = ValueUtils.getValueAsFieldType(field.getType(), value);
|
||||
if(field.getDisplayFormat() != null)
|
||||
{
|
||||
return (String.format(Locale.US, field.getDisplayFormat(), typed));
|
||||
}
|
||||
value = typed;
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
throw (new IllegalArgumentException("value does not match field format"));
|
||||
}
|
||||
}
|
||||
if(value instanceof Number)
|
||||
{
|
||||
BigDecimal decimal = number(value);
|
||||
return (decimal.stripTrailingZeros().scale() <= 0 ? String.format(Locale.US, "%,d", decimal.toBigIntegerExact()) : decimal.toPlainString());
|
||||
}
|
||||
return (text(value));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Complete GFM fallback with real newlines; chart data is already in table.
|
||||
*******************************************************************************/
|
||||
private static String markdown(String summary, List<Map<String, Object>> blocks)
|
||||
{
|
||||
StringBuilder markdown = new StringBuilder(escape(summary)).append("\n");
|
||||
for(Map<String, Object> block : blocks)
|
||||
{
|
||||
if("table".equals(block.get("kind")))
|
||||
{
|
||||
markdown.append("\n").append(escape(text(block.get("title")))).append("\n\n|");
|
||||
List<?> columns = list(block.get("columns"));
|
||||
for(Object column : columns)
|
||||
{
|
||||
markdown.append(' ').append(escape(text(object(column).get("label")))).append(" |");
|
||||
}
|
||||
markdown.append("\n|");
|
||||
for(Object column : columns)
|
||||
{
|
||||
markdown.append("right".equals(object(column).get("align")) ? " ---: |" : " --- |");
|
||||
}
|
||||
for(Object entry : list(block.get("rows")))
|
||||
{
|
||||
Map<String, Object> row = object(entry);
|
||||
markdown.append("\n|");
|
||||
for(Integer c = 0; c < columns.size(); c++)
|
||||
{
|
||||
String value = escape(text(object(row.get("display")).get("c" + c)));
|
||||
markdown.append(' ').append(c == 0 && row.containsKey("href") ? "[" + value + "](" + row.get("href") + ")" : value).append(" |");
|
||||
}
|
||||
}
|
||||
markdown.append("\n");
|
||||
}
|
||||
else if(block.get("items") instanceof List<?> items)
|
||||
{
|
||||
markdown.append("\n");
|
||||
for(Object entry : items)
|
||||
{
|
||||
Map<String, Object> item = object(entry);
|
||||
String label = escape(text(item.get("label")));
|
||||
markdown.append("- ");
|
||||
if(item.containsKey("href"))
|
||||
{
|
||||
markdown.append('[').append(label).append("](").append(item.get("href")).append(')');
|
||||
}
|
||||
else
|
||||
{
|
||||
markdown.append(label);
|
||||
if(item.containsKey("display") || item.containsKey("prompt"))
|
||||
{
|
||||
markdown.append(": ").append(escape(text(item.getOrDefault("display", item.get("prompt")))));
|
||||
}
|
||||
}
|
||||
markdown.append("\n");
|
||||
}
|
||||
}
|
||||
}
|
||||
return (markdown.toString());
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Literal text only: no Markdown links, HTML or GFM URL autolinks from labels.
|
||||
*******************************************************************************/
|
||||
private static String escape(String value)
|
||||
{
|
||||
StringBuilder output = new StringBuilder();
|
||||
for(char character : value.toCharArray())
|
||||
{
|
||||
if("\\`*_{}[]()<>|!&#:.@~".indexOf(character) >= 0)
|
||||
{
|
||||
output.append("&#").append((int) character).append(';');
|
||||
}
|
||||
else
|
||||
{
|
||||
output.append(character);
|
||||
}
|
||||
}
|
||||
return (output.toString());
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Reject unsafe paths even if metadata contains unexpected names.
|
||||
*******************************************************************************/
|
||||
private static String safeHref(String href)
|
||||
{
|
||||
require(AiLinkBuilder.isSafeAppPath(href), "unsafe application path");
|
||||
return (href);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A column with no percentage uses -1 in strict mode.
|
||||
*******************************************************************************/
|
||||
private static Integer percentColumn(Map<String, Object> column)
|
||||
{
|
||||
return (column.get("percentOfColumn") == null ? -1 : index(column.get("percentOfColumn")));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Indexes must be integral; never silently truncate model input.
|
||||
*******************************************************************************/
|
||||
private static Integer index(Object value)
|
||||
{
|
||||
try
|
||||
{
|
||||
return (number(value).intValueExact());
|
||||
}
|
||||
catch(ArithmeticException e)
|
||||
{
|
||||
throw (new IllegalArgumentException("index must be an integer"));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Finite numeric input only.
|
||||
*******************************************************************************/
|
||||
private static BigDecimal number(Object value)
|
||||
{
|
||||
require(value instanceof Number, "numeric value required");
|
||||
try
|
||||
{
|
||||
return (new BigDecimal(value.toString()));
|
||||
}
|
||||
catch(NumberFormatException e)
|
||||
{
|
||||
throw (new IllegalArgumentException("finite numeric value required"));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Cells and filter values remain raw JSON scalars.
|
||||
*******************************************************************************/
|
||||
private static void scalar(Object value)
|
||||
{
|
||||
require(value == null || value instanceof String || value instanceof Number || value instanceof Boolean, "scalar value required");
|
||||
if(value instanceof Number)
|
||||
{
|
||||
number(value);
|
||||
}
|
||||
if(value instanceof String)
|
||||
{
|
||||
text(value);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Bounded single-line display text.
|
||||
*******************************************************************************/
|
||||
private static String text(Object value)
|
||||
{
|
||||
require(value == null || value instanceof String || value instanceof Number || value instanceof Boolean, "text or scalar required");
|
||||
String text = value == null ? "" : String.valueOf(value);
|
||||
require(text.length() <= 2000, "text exceeds 2000 characters");
|
||||
return (text.replaceAll("[\\p{Cntrl}\\u2028\\u2029]", " "));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Labels are mandatory plain text.
|
||||
*******************************************************************************/
|
||||
private static String label(Map<String, Object> item)
|
||||
{
|
||||
String label = text(item.get("label"));
|
||||
require(!label.isBlank(), "label required");
|
||||
return (label);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Validate a JSON object without converting arbitrary values to strings.
|
||||
*******************************************************************************/
|
||||
@SuppressWarnings("unchecked")
|
||||
private static Map<String, Object> object(Object value)
|
||||
{
|
||||
require(value instanceof Map<?, ?>, "object required");
|
||||
return ((Map<String, Object>) value);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Validate a JSON array.
|
||||
*******************************************************************************/
|
||||
private static List<?> list(Object value)
|
||||
{
|
||||
require(value instanceof List<?>, "array required");
|
||||
return ((List<?>) value);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Ordered output maps allow null raw cells.
|
||||
*******************************************************************************/
|
||||
private static Map<String, Object> map(Object... pairs)
|
||||
{
|
||||
Map<String, Object> map = new LinkedHashMap<>();
|
||||
for(Integer i = 0; i < pairs.length; i += 2)
|
||||
{
|
||||
map.put((String) pairs[i], pairs[i + 1]);
|
||||
}
|
||||
return (map);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Validation failures contain only server-authored explanations.
|
||||
*******************************************************************************/
|
||||
private static void require(Boolean condition, String message)
|
||||
{
|
||||
if(!condition)
|
||||
{
|
||||
throw (new IllegalArgumentException(message));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,280 @@
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"summary": {
|
||||
"type": "string",
|
||||
"maxLength": 300
|
||||
},
|
||||
"metrics": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/metric"
|
||||
}
|
||||
},
|
||||
"table": {
|
||||
"$ref": "#/$defs/table"
|
||||
},
|
||||
"chart": {
|
||||
"$ref": "#/$defs/chart"
|
||||
},
|
||||
"filterLinks": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/filterLink"
|
||||
}
|
||||
},
|
||||
"recordLinks": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/recordLink"
|
||||
}
|
||||
},
|
||||
"actions": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/action"
|
||||
}
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"summary"
|
||||
],
|
||||
"additionalProperties": false,
|
||||
"$defs": {
|
||||
"cell": {
|
||||
"type": [
|
||||
"string",
|
||||
"number",
|
||||
"boolean",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"metric": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"label": {
|
||||
"type": "string"
|
||||
},
|
||||
"value": {
|
||||
"type": "number"
|
||||
},
|
||||
"tableName": {
|
||||
"type": "string"
|
||||
},
|
||||
"fieldName": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"label",
|
||||
"value"
|
||||
],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"column": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"label": {
|
||||
"type": "string"
|
||||
},
|
||||
"tableName": {
|
||||
"type": "string"
|
||||
},
|
||||
"fieldName": {
|
||||
"type": "string"
|
||||
},
|
||||
"percentOfColumn": {
|
||||
"type": "integer"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"label"
|
||||
],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"rowFilter": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"tableName": {
|
||||
"type": "string"
|
||||
},
|
||||
"fieldName": {
|
||||
"type": "string"
|
||||
},
|
||||
"operator": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"tableName",
|
||||
"fieldName"
|
||||
],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"criterion": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"fieldName": {
|
||||
"type": "string"
|
||||
},
|
||||
"operator": {
|
||||
"type": "string"
|
||||
},
|
||||
"values": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/cell"
|
||||
}
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"fieldName",
|
||||
"operator",
|
||||
"values"
|
||||
],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"filterLink": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"label": {
|
||||
"type": "string"
|
||||
},
|
||||
"tableName": {
|
||||
"type": "string"
|
||||
},
|
||||
"criteria": {
|
||||
"$ref": "#/$defs/criteria"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"label",
|
||||
"tableName",
|
||||
"criteria"
|
||||
],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"recordLink": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"label": {
|
||||
"type": "string"
|
||||
},
|
||||
"tableName": {
|
||||
"type": "string"
|
||||
},
|
||||
"recordId": {
|
||||
"$ref": "#/$defs/cell"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"label",
|
||||
"tableName",
|
||||
"recordId"
|
||||
],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"action": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"label": {
|
||||
"type": "string"
|
||||
},
|
||||
"prompt": {
|
||||
"type": "string"
|
||||
},
|
||||
"navigateTo": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"label"
|
||||
],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"chart": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"chartType": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"bar",
|
||||
"pie",
|
||||
"line"
|
||||
]
|
||||
},
|
||||
"title": {
|
||||
"type": "string"
|
||||
},
|
||||
"labelColumn": {
|
||||
"type": "integer"
|
||||
},
|
||||
"valueColumn": {
|
||||
"type": "integer"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"chartType",
|
||||
"labelColumn",
|
||||
"valueColumn"
|
||||
],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"row": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/cell"
|
||||
}
|
||||
},
|
||||
"columns": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/column"
|
||||
}
|
||||
},
|
||||
"rows": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/row"
|
||||
}
|
||||
},
|
||||
"rowKeys": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/cell"
|
||||
}
|
||||
},
|
||||
"criteria": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/criterion"
|
||||
}
|
||||
},
|
||||
"table": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"title": {
|
||||
"type": "string"
|
||||
},
|
||||
"columns": {
|
||||
"$ref": "#/$defs/columns"
|
||||
},
|
||||
"rows": {
|
||||
"$ref": "#/$defs/rows"
|
||||
},
|
||||
"rowFilter": {
|
||||
"$ref": "#/$defs/rowFilter"
|
||||
},
|
||||
"rowKeys": {
|
||||
"$ref": "#/$defs/rowKeys"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"columns",
|
||||
"rows"
|
||||
],
|
||||
"additionalProperties": false
|
||||
}
|
||||
},
|
||||
"description": "Present only facts from previous tool results. Optional sections may be omitted. In strict mode use empty arrays, empty strings, empty table rows/columns and -1 for unused indexes; every property must be supplied. rowKeys belongs to table, one raw key per row. Percent columns may use null cells; the server computes them."
|
||||
}
|
||||
+25
@@ -26,6 +26,9 @@ import com.kingsrook.qbits.assistant.testing.AssistantTestFixture;
|
||||
import com.kingsrook.qbits.assistant.testing.BaseAssistantTest;
|
||||
import com.kingsrook.qqq.backend.core.actions.permissions.PermissionCheckResult;
|
||||
import com.kingsrook.qqq.backend.core.actions.permissions.PermissionsHelper;
|
||||
import com.kingsrook.qqq.backend.core.alerts.AlertsInstanceMetaData;
|
||||
import com.kingsrook.qqq.backend.core.alerts.AlertSeverity;
|
||||
import com.kingsrook.qqq.backend.core.alerts.QAlertTypeMetaData;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
|
||||
@@ -103,6 +106,28 @@ class AssistantQBitProducerTest extends BaseAssistantTest
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Budget types are discoverable through core with admin-only visibility.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void budgetAlertTypes()
|
||||
{
|
||||
var types = AlertsInstanceMetaData.of(qInstance).getAlertTypes();
|
||||
assertThat(types).containsOnlyKeys("assistant.budget.warning", "assistant.budget.downgraded", "assistant.budget.stopped");
|
||||
for(QAlertTypeMetaData type : types.values())
|
||||
{
|
||||
assertThat(type.getLabel()).isNotBlank();
|
||||
assertThat(type.getDescription()).isNotBlank();
|
||||
assertThat(type.getDefaultSeverity()).isEqualTo(type.getName().endsWith("stopped") ? AlertSeverity.ERROR : AlertSeverity.WARNING);
|
||||
assertThat(type.getHowToFix()).isEqualTo("Raise the budget or grant an override in Assistant → Budgets");
|
||||
assertThat(type.getSourceQBitName()).isEqualTo("assistant");
|
||||
assertThat(type.getVisibilityPermission()).isEqualTo("assistant.admin");
|
||||
assertThat(type.getSubjectTableName()).isEqualTo("assistant_budget");
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The admin app and tables: visible only with assistant.admin.
|
||||
*******************************************************************************/
|
||||
|
||||
+4
@@ -93,6 +93,10 @@ class ClassificationTest extends BaseAssistantTest
|
||||
assertThat(record.getClassification()).isEqualTo(DataClassification.PHI);
|
||||
page.observe(ModelPart.toolUse("1", "describe_table", Map.of("table", "page")), new AiToolResult());
|
||||
assertThat(page.getClassification()).isEqualTo(DataClassification.PUBLIC);
|
||||
page.observe(ModelPart.toolUse("present", "present_results", Map.of("summary", "Available results")), new AiToolResult());
|
||||
assertThat(page.getClassification()).isEqualTo(DataClassification.PUBLIC);
|
||||
record.observe(ModelPart.toolUse("present", "present_results", Map.of("summary", "Available results")), new AiToolResult());
|
||||
assertThat(record.getClassification()).isEqualTo(DataClassification.PHI);
|
||||
page.observe(ModelPart.toolUse("2", "search_records", Map.of()), new AiToolResult().withModelContent("<qqq_data>{\"results\":[{\"table\":\"media\"}]}</qqq_data>"));
|
||||
assertThat(page.getClassification()).isEqualTo(DataClassification.PII);
|
||||
page.observe(ModelPart.toolUse("3", "search_records", Map.of()), new AiToolResult());
|
||||
|
||||
+59
-8
@@ -35,6 +35,13 @@ import com.kingsrook.qbits.assistant.provider.AssistantProviderType;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelUsage;
|
||||
import com.kingsrook.qbits.assistant.testing.AssistantTestFixture;
|
||||
import com.kingsrook.qbits.assistant.testing.BaseAssistantTest;
|
||||
import com.kingsrook.qqq.backend.core.alerts.AlertSignal;
|
||||
import com.kingsrook.qqq.backend.core.alerts.AlertSeverity;
|
||||
import com.kingsrook.qqq.backend.core.alerts.AlertsInstanceMetaData;
|
||||
import com.kingsrook.qqq.backend.core.alerts.LoggingAlertSink;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReferenceWithProperties;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.code.InitializableViaCodeReference;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
@@ -172,12 +179,45 @@ class BudgetServiceTest extends BaseAssistantTest
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fixed-clock evaluator and alert seam.
|
||||
** Fixed-clock evaluator with the production core sink configuration.
|
||||
*******************************************************************************/
|
||||
private BudgetService service(List<String> alerts)
|
||||
private BudgetService service(List<AlertSignal> alerts)
|
||||
{
|
||||
return (new BudgetService(store(), config.getRuntime().getProviderService(), Clock.fixed(NOW, ZoneOffset.UTC),
|
||||
(type, key, severity, title, summary) -> alerts.add(type + ":" + severity)));
|
||||
AlertsInstanceMetaData.ofOrWithNew(qInstance).withSink(new QCodeReferenceWithProperties(TestAlertSink.class, Map.of("signals", (ArrayList<AlertSignal>) alerts)));
|
||||
return (new BudgetService(store(), config.getRuntime().getProviderService(), Clock.fixed(NOW, ZoneOffset.UTC)));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Core loads a separate sink per instance; captured signals stay test-local.
|
||||
*******************************************************************************/
|
||||
public static class TestAlertSink extends LoggingAlertSink implements InitializableViaCodeReference
|
||||
{
|
||||
private List<AlertSignal> signals;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Receive this test's capture list from the configured code reference.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
@SuppressWarnings("unchecked")
|
||||
public void initialize(QCodeReference codeReference)
|
||||
{
|
||||
signals = (List<AlertSignal>) ((QCodeReferenceWithProperties) codeReference).getProperties().get("signals");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Capture the signal delivered through core.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void raise(AlertSignal signal)
|
||||
{
|
||||
signals.add(signal);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -188,15 +228,22 @@ class BudgetServiceTest extends BaseAssistantTest
|
||||
@Test
|
||||
void warningBoundaryAndDedup() throws Exception
|
||||
{
|
||||
budget();
|
||||
AssistantBudget budget = budget();
|
||||
usage(790_000, NOW);
|
||||
List<String> alerts = new ArrayList<>();
|
||||
List<AlertSignal> alerts = new ArrayList<>();
|
||||
BudgetService service = service(alerts);
|
||||
assertThat(service.evaluate(USER, null, false, false).notices()).isEmpty();
|
||||
usage(10_000, NOW);
|
||||
assertThat(service.evaluate(USER, null, false, false).notices()).singleElement().asString().contains("80%", "Per user daily");
|
||||
assertThat(service.evaluate(USER, null, false, false).notices()).isEmpty();
|
||||
assertThat(alerts).containsExactly("assistant.budget.warning:WARNING");
|
||||
assertThat(alerts).singleElement().satisfies(signal ->
|
||||
{
|
||||
assertThat(signal.getAlertTypeName()).isEqualTo("assistant.budget.warning");
|
||||
assertThat(signal.getDedupKey()).isEqualTo(budget.getId() + ":USER:" + USER + ":2026-10-13T00:00:00Z");
|
||||
assertThat(signal.getSeverity()).isEqualTo(AlertSeverity.WARNING);
|
||||
assertThat(signal.getTitle()).isEqualTo("Assistant budget Per user daily");
|
||||
assertThat(signal.getSummary()).contains("80%", "Per user daily");
|
||||
});
|
||||
assertThat(store().query(AssistantBudgetEvent.TABLE_NAME, new QQueryFilter(), AssistantBudgetEvent.class)).singleElement()
|
||||
.extracting(AssistantBudgetEvent::getState).isEqualTo("WARNED");
|
||||
}
|
||||
@@ -215,10 +262,14 @@ class BudgetServiceTest extends BaseAssistantTest
|
||||
AssistantModel cheap = store().insert(new AssistantModel().withProviderId(local.getId()).withModelId("cheap").withEnabled(true).withInputUsdPerMTok(BigDecimal.ONE));
|
||||
store().insert(new AssistantModel().withProviderId(cloud.getId()).withModelId("main").withEnabled(true).withInputUsdPerMTok(BigDecimal.ZERO));
|
||||
usage(1_000_000, NOW);
|
||||
BudgetService service = service(new ArrayList<>());
|
||||
List<AlertSignal> alerts = new ArrayList<>();
|
||||
BudgetService service = service(alerts);
|
||||
ProviderService.Active active = config.getRuntime().getProviderService().resolve("main");
|
||||
assertThat(service.evaluate(USER, active, false, false).active().providerId()).isEqualTo(local.getId());
|
||||
assertThat(service.evaluate(USER, active, true, false).stopped()).isTrue();
|
||||
assertThat(alerts).extracting(AlertSignal::getAlertTypeName).containsExactly("assistant.budget.downgraded", "assistant.budget.stopped");
|
||||
assertThat(alerts).extracting(AlertSignal::getSeverity).containsExactly(AlertSeverity.WARNING, AlertSeverity.ERROR);
|
||||
assertThat(alerts).extracting(AlertSignal::getDedupKey).containsOnly(budget.getId() + ":USER:" + USER + ":2026-10-13T00:00:00Z");
|
||||
store().updateFields(AssistantBudget.TABLE_NAME, new QRecord().withValue("id", budget.getId()).withValue("downgradeModelId", cheap.getId()));
|
||||
assertThat(service.evaluate(USER, active, false, false).active().model()).isEqualTo("cheap");
|
||||
}
|
||||
|
||||
+7
-3
@@ -226,9 +226,13 @@ public class AssistantTestFixture
|
||||
update("page", new QRecord().withValue("id", 12).withValue("metaDescription", "Who I am and what I build."));
|
||||
update("page", new QRecord().withValue("id", 12).withValue("content", "Body of page 12, rewritten with a longer story about the home lab."));
|
||||
|
||||
insert("media", List.of(
|
||||
new QRecord().withValue("fileName", "cover.heic").withValue("mimeType", "image/heic"),
|
||||
new QRecord().withValue("fileName", "logo.png").withValue("mimeType", "image/png").withValue("width", 512)));
|
||||
List<QRecord> media = new ArrayList<>();
|
||||
for(Integer i = 0; i < 101; i++)
|
||||
{
|
||||
media.add(new QRecord().withValue("fileName", "image-" + i + (i < 81 ? ".jpg" : ".png"))
|
||||
.withValue("mimeType", i < 81 ? "image/jpeg" : "image/png").withValue("width", 512));
|
||||
}
|
||||
insert("media", media);
|
||||
|
||||
List<QRecord> stats = new ArrayList<>();
|
||||
for(int d = 1; d <= 5; d++)
|
||||
|
||||
@@ -180,6 +180,7 @@ public class GoldenTasks
|
||||
},
|
||||
o -> live(o, "search_records", "/app/blog_post/1")));
|
||||
|
||||
tasks.add(PresentationGoldenTask.task());
|
||||
return (tasks);
|
||||
}
|
||||
|
||||
|
||||
+171
@@ -0,0 +1,171 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.testing;
|
||||
|
||||
|
||||
import java.net.URLDecoder;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import com.fasterxml.jackson.core.type.TypeReference;
|
||||
import com.kingsrook.qbits.assistant.engine.TurnEvent;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelPart;
|
||||
import com.kingsrook.qbits.assistant.provider.ModelResponse;
|
||||
import com.kingsrook.qqq.backend.core.utils.JsonUtils;
|
||||
import static com.kingsrook.qbits.assistant.testing.GoldenTasks.map;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Owner's image breakdown, shared by deterministic and live provider evals.
|
||||
*******************************************************************************/
|
||||
public class PresentationGoldenTask
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Counts come from UI API tool results, including the omitted zero types.
|
||||
*******************************************************************************/
|
||||
public static GoldenTasks.Task task()
|
||||
{
|
||||
return (new GoldenTasks.Task("images-by-type-presentation", AssistantTestFixture.EDITOR, "How many images do I have by type?", provider ->
|
||||
{
|
||||
List<Object[]> calls = new ArrayList<>();
|
||||
for(String mime : List.of("image/jpeg", "image/png", "image/webp", "image/gif", "image/svg+xml"))
|
||||
{
|
||||
calls.add(new Object[] {"count_records", map("table", "media", "filter", GoldenTasks.filter("AND", GoldenTasks.criterion("mimeType", "EQUALS", mime)))});
|
||||
}
|
||||
provider.thenToolUses(null, calls).thenRespond(request ->
|
||||
{
|
||||
List<ModelPart> results = EvalSupport.lastToolResults(request);
|
||||
Integer jpeg = ((Number) EvalSupport.unwrap(results.get(0).getText()).get("count")).intValue();
|
||||
Integer png = ((Number) EvalSupport.unwrap(results.get(1).getText()).get("count")).intValue();
|
||||
for(Integer i = 2; i < results.size(); i++)
|
||||
{
|
||||
assertThat(((Number) EvalSupport.unwrap(results.get(i).getText()).get("count")).intValue()).isZero();
|
||||
}
|
||||
return (new ModelResponse().withParts(List.of(ModelPart.toolUse("present", "present_results", input(jpeg, png)))).withStopReason(ModelResponse.STOP_TOOL_USE));
|
||||
}).thenText("");
|
||||
}, PresentationGoldenTask::check, PresentationGoldenTask::check));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The presentation input, deliberately using null percentage placeholders.
|
||||
*******************************************************************************/
|
||||
public static Map<String, Object> input(Integer jpeg, Integer png)
|
||||
{
|
||||
return (map("summary", (jpeg + png) + " media files are images: " + jpeg + " JPEG and " + png + " PNG.",
|
||||
"metrics", List.of(map("label", "Images", "value", jpeg + png)),
|
||||
"table", map("title", "Media by type", "columns", List.of(map("label", "Type"), map("label", "Count"), map("label", "Share", "percentOfColumn", 1)),
|
||||
"rows", List.of(java.util.Arrays.asList("JPEG", jpeg, null), java.util.Arrays.asList("PNG", png, null)),
|
||||
"rowFilter", map("tableName", "media", "fieldName", "mimeType"), "rowKeys", List.of("image/jpeg", "image/png")),
|
||||
"chart", map("chartType", "pie", "title", "Share by type", "labelColumn", 0, "valueColumn", 1),
|
||||
"filterLinks", List.of(filterLink("JPEG", "image/jpeg"), filterLink("PNG", "image/png"))));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A raw MIME criterion for the presentation tool.
|
||||
*******************************************************************************/
|
||||
private static Map<String, Object> filterLink(String label, String mime)
|
||||
{
|
||||
return (map("label", "Open " + label + " media", "tableName", "media", "criteria", List.of(map("fieldName", "mimeType", "operator", "EQUALS", "values", List.of(mime)))));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Check actual block semantics, allowing a live model to choose labels and
|
||||
** column order. Values, shares and filter criteria must remain exact.
|
||||
*******************************************************************************/
|
||||
public static void check(TurnRunner.Outcome outcome)
|
||||
{
|
||||
assertThat(outcome.end()).containsEntry("status", "COMPLETE");
|
||||
assertThat(outcome.toolCalls()).anyMatch(call -> "count_records".equals(call.getToolName()) && "OK".equals(call.getOutcome()));
|
||||
assertThat(outcome.events(TurnEvent.PRESENTATION)).hasSize(1);
|
||||
Map<String, Object> part = object(outcome.events(TurnEvent.PRESENTATION).get(0).get("part"));
|
||||
assertPart(part);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Also used against the persisted part, not only the streamed event.
|
||||
*******************************************************************************/
|
||||
public static void assertPart(Map<String, Object> part)
|
||||
{
|
||||
assertThat(part.get("type")).isEqualTo("presentation");
|
||||
assertThat((String) part.get("summary")).isNotBlank().doesNotContain("\n").hasSizeLessThanOrEqualTo(300);
|
||||
assertThat((String) part.get("markdown")).contains("\n|");
|
||||
Map<String, Object> table = block(part, "table");
|
||||
List<?> rows = (List<?>) table.get("rows");
|
||||
assertThat(rows).hasSize(2);
|
||||
assertThat(rows).anySatisfy(row -> assertThat(object(object(row).get("display")).values()).contains("81", "80.2%"));
|
||||
assertThat(rows).anySatisfy(row -> assertThat(object(object(row).get("display")).values()).contains("20", "19.8%"));
|
||||
Map<String, Object> chart = block(part, "chart");
|
||||
assertThat(chart.get("chartType")).isEqualTo("pie");
|
||||
List<?> points = (List<?>) chart.get("points");
|
||||
assertThat(points).hasSize(2);
|
||||
assertThat(points.stream().map(p -> ((Number) object(p).get("value")).intValue()).toList()).containsExactlyInAnyOrder(81, 20);
|
||||
List<?> links = (List<?>) block(part, "filterLinks").get("items");
|
||||
assertThat(links).hasSize(2);
|
||||
List<Object> mimes = new ArrayList<>();
|
||||
for(Object entry : links)
|
||||
{
|
||||
String href = (String) object(entry).get("href");
|
||||
assertThat(href).startsWith("/app/media?filter=");
|
||||
try
|
||||
{
|
||||
Map<String, Object> filter = JsonUtils.toObject(URLDecoder.decode(href.substring(href.indexOf('=') + 1), StandardCharsets.UTF_8), new TypeReference<Map<String, Object>>() {});
|
||||
assertThat(filter).containsEntry("booleanOperator", "AND");
|
||||
List<?> criteria = (List<?>) filter.get("criteria");
|
||||
assertThat(criteria).hasSize(1);
|
||||
Map<String, Object> criterion = object(criteria.get(0));
|
||||
assertThat(criterion).containsEntry("fieldName", "mimeType").containsEntry("operator", "EQUALS");
|
||||
mimes.addAll((List<?>) criterion.get("values"));
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
throw (new AssertionError("Invalid filtered list href", e));
|
||||
}
|
||||
}
|
||||
assertThat(mimes).containsExactlyInAnyOrder("image/jpeg", "image/png");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Find one block of a given kind.
|
||||
*******************************************************************************/
|
||||
public static Map<String, Object> block(Map<String, Object> part, String kind)
|
||||
{
|
||||
return (((List<?>) part.get("blocks")).stream().map(PresentationGoldenTask::object).filter(block -> kind.equals(block.get("kind"))).findFirst().orElseThrow());
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Typed test view of a JSON object.
|
||||
*******************************************************************************/
|
||||
@SuppressWarnings("unchecked")
|
||||
public static Map<String, Object> object(Object value)
|
||||
{
|
||||
return ((Map<String, Object>) value);
|
||||
}
|
||||
}
|
||||
@@ -59,7 +59,8 @@ public class TurnRunner
|
||||
*******************************************************************************/
|
||||
public String text()
|
||||
{
|
||||
return (events.stream().filter(e -> TurnEvent.TEXT_DELTA.equals(e.getType())).map(e -> String.valueOf(e.getData().get("text"))).collect(Collectors.joining()));
|
||||
return (events.stream().filter(e -> TurnEvent.TEXT_DELTA.equals(e.getType()) || TurnEvent.PRESENTATION.equals(e.getType()))
|
||||
.map(e -> TurnEvent.PRESENTATION.equals(e.getType()) ? String.valueOf(((Map<?, ?>) e.getData().get("part")).get("markdown")) : String.valueOf(e.getData().get("text"))).collect(Collectors.joining()));
|
||||
}
|
||||
|
||||
|
||||
|
||||
+275
@@ -0,0 +1,275 @@
|
||||
/*
|
||||
* Copyright 2026 Kingsrook, LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.assistant.tools;
|
||||
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import com.kingsrook.qbits.assistant.engine.MessageParts;
|
||||
import com.kingsrook.qbits.assistant.engine.PolicyService;
|
||||
import com.kingsrook.qbits.assistant.engine.TurnEvent;
|
||||
import com.kingsrook.qbits.assistant.model.AssistantMessage;
|
||||
import com.kingsrook.qbits.assistant.testing.AssistantTestFixture;
|
||||
import com.kingsrook.qbits.assistant.testing.BaseAssistantTest;
|
||||
import com.kingsrook.qbits.assistant.testing.PresentationGoldenTask;
|
||||
import com.kingsrook.qbits.assistant.testing.TurnRunner;
|
||||
import com.kingsrook.qbits.assistant.uiapi.UiApiCall;
|
||||
import com.kingsrook.qbits.assistant.uiapi.UiApiGateway;
|
||||
import com.kingsrook.qbits.assistant.uiapi.UiApiResult;
|
||||
import com.kingsrook.qqq.backend.core.actions.ai.AiToolOutcome;
|
||||
import com.kingsrook.qqq.backend.core.actions.ai.AiToolResult;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import static com.kingsrook.qbits.assistant.testing.PresentationGoldenTask.block;
|
||||
import static com.kingsrook.qbits.assistant.testing.PresentationGoldenTask.object;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Presentation persistence, permission boundaries, formatting and input limits.
|
||||
*******************************************************************************/
|
||||
class PresentResultsToolTest extends BaseAssistantTest
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Streamed, replayed and stored presentations agree; retention removes data.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void testPersistenceAndReplay() throws Exception
|
||||
{
|
||||
PresentationGoldenTask.task().script().accept(provider);
|
||||
TurnRunner.Outcome outcome = runner.run(AssistantTestFixture.EDITOR, "How many images do I have by type?");
|
||||
PresentationGoldenTask.check(outcome);
|
||||
QContext.init(qInstance, AssistantTestFixture.ownerSession(AssistantTestFixture.EDITOR, config));
|
||||
Map<String, Object> event = outcome.events(TurnEvent.PRESENTATION).get(0);
|
||||
AssistantMessage message = runner.runtime().getStore().getMessage(AssistantTestFixture.EDITOR, (Integer) event.get("messageId"));
|
||||
MessageParts parts = MessageParts.fromJson(message.getPartsJson());
|
||||
Map<String, Object> part = parts.list().stream().filter(p -> "presentation".equals(p.get("type"))).findFirst().orElseThrow();
|
||||
PresentationGoldenTask.assertPart(part);
|
||||
assertThat(part).isEqualTo(com.kingsrook.qqq.backend.core.utils.JsonUtils.toObject(com.kingsrook.qqq.backend.core.utils.JsonUtils.toJson(event.get("part")), Map.class));
|
||||
List<?> rows = (List<?>) block(part, "table").get("rows");
|
||||
List<?> points = (List<?>) block(part, "chart").get("points");
|
||||
List<?> links = (List<?>) block(part, "filterLinks").get("items");
|
||||
for(Integer i = 0; i < 2; i++)
|
||||
{
|
||||
assertThat(object(rows.get(i)).get("href")).isEqualTo(object(links.get(i)).get("href"));
|
||||
assertThat(object(points.get(i)).get("href")).isEqualTo(object(rows.get(i)).get("href"));
|
||||
}
|
||||
assertThat(runner.runtime().getStore().listEvents(AssistantTestFixture.EDITOR, outcome.turnId(), 0, 1000))
|
||||
.anyMatch(e -> TurnEvent.PRESENTATION.equals(e.getEventType()));
|
||||
assertThat(parts.text()).contains("81", "20", "\n|");
|
||||
assertThat(parts.stripRecordData()).isTrue();
|
||||
assertThat(parts.list()).noneMatch(p -> "presentation".equals(p.get("type")));
|
||||
assertThat(outcome.toolCalls()).anySatisfy(call ->
|
||||
{
|
||||
assertThat(call.getToolName()).isEqualTo(PresentResultsTool.NAME);
|
||||
assertThat(call.getInputJson()).contains("presentation");
|
||||
assertThat(call.getResultJson()).contains("Do not repeat");
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The real UI metadata hides media from an analyst, as does presentation.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void testUiDenial() throws Exception
|
||||
{
|
||||
String user = AssistantTestFixture.ANALYST;
|
||||
UiApiResult metadata = runner.runtime().getUiApiGateway().invoke(new UiApiCall("GET", "/metaData").withSessionUUID(user));
|
||||
assertThat(object(metadata.json().get("tables"))).doesNotContainKey("media");
|
||||
provider.thenToolUse(PresentResultsTool.NAME, input("summary", "The list is unavailable.", "recordLinks", List.of(input("label", "Image", "tableName", "media", "recordId", "1")))).thenText("");
|
||||
TurnRunner.Outcome outcome = runner.run(user, "Open an image");
|
||||
assertThat(outcome.toolCalls().get(0).getResultJson()).contains("table not visible through UI API: media");
|
||||
Map<String, Object> part = object(outcome.events(TurnEvent.PRESENTATION).get(0).get("part"));
|
||||
assertThat((List<?>) part.get("blocks")).isEmpty();
|
||||
assertThat(outcome.toolCalls().get(0).getInputJson()).doesNotContain("recordId");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Cache metadata once per turn and send the original session UUID.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void testMetadataCache() throws Exception
|
||||
{
|
||||
List<UiApiCall> calls = new ArrayList<>();
|
||||
UiApiGateway gateway = new UiApiGateway()
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Fixture metadata gateway.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public boolean isInvocable(String method, String path)
|
||||
{
|
||||
return (true);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fixture metadata gateway.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public UiApiResult invoke(UiApiCall call)
|
||||
{
|
||||
calls.add(call);
|
||||
return (new UiApiResult(200, "{\"tables\":{\"media\":{\"readPermission\":true}}}"));
|
||||
}
|
||||
};
|
||||
UiApiToolbox.Context context = new UiApiToolbox.Context("editor-session", "UTC", gateway, PolicyService.ToolPolicy.defaults(), null);
|
||||
PresentResultsTool tool = new PresentResultsTool(qInstance);
|
||||
assertThat(tool.execute(context, "1", PresentationGoldenTask.input(81, 20)).getOutcome()).isEqualTo(AiToolOutcome.OK);
|
||||
tool.execute(context, "2", PresentationGoldenTask.input(81, 20));
|
||||
assertThat(calls).hasSize(1).allMatch(call -> "editor-session".equals(call.getSessionUUID()) && "/metaData".equals(call.getPathTemplate()));
|
||||
new PresentResultsTool(qInstance).execute(context, "3", PresentationGoldenTask.input(81, 20));
|
||||
assertThat(calls).hasSize(2);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Unknown metadata is dropped independently without persisting its values.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void testUnknownMetadata()
|
||||
{
|
||||
AiToolResult result = execute(input("summary", "Available results.", "metrics", List.of(
|
||||
input("label", "Unknown table", "value", 9876, "tableName", "missing"),
|
||||
input("label", "Unknown field", "value", 5432, "tableName", "media", "fieldName", "missing"),
|
||||
input("label", "Known", "value", 101))));
|
||||
assertThat(result.getModelContent()).contains("Dropped metrics[0]", "unknown table", "Dropped metrics[1]", "unknown field");
|
||||
assertThat(result.getRender().toString()).doesNotContain("9876", "5432");
|
||||
assertThat((List<?>) block(result.getRender(), "metrics").get("items")).hasSize(1);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Markdown punctuation, external URLs and pipes remain literal text.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void testUntrustedLabelsAndLinks()
|
||||
{
|
||||
AiToolResult result = execute(input("summary", "Available results.", "metrics", List.of(input("label", "[evil](https://evil.example) | <img>", "value", 1)),
|
||||
"recordLinks", List.of(input("label", "Open", "tableName", "media", "recordId", "a/b?x#y")),
|
||||
"actions", List.of(input("label", "Ask", "prompt", "Show images"), input("label", "Open media", "navigateTo", "media"), input("label", "Bad", "prompt", "Ask", "navigateTo", "media"))));
|
||||
String markdown = (String) result.getRender().get("markdown");
|
||||
assertThat(markdown).doesNotContain("https://", "<img>", "[evil]").contains("|", "[Open](/app/media/a%2Fb%3Fx%23y)", "- Ask: Show images", "[Open media](/app/media)");
|
||||
assertThat(result.getModelContent()).contains("exactly one");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Formatting follows metadata while raw values remain numeric.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void testFormatting()
|
||||
{
|
||||
qInstance.getTable("media").addField(new QFieldMetaData("price", QFieldType.DECIMAL).withDisplayFormat("$%,.2f"));
|
||||
AiToolResult result = execute(input("summary", "Amounts.", "metrics", List.of(input("label", "Amount", "value", 1234.5, "tableName", "media", "fieldName", "price"), input("label", "Count", "value", 12000)),
|
||||
"table", input("columns", List.of(input("label", "Amount", "tableName", "media", "fieldName", "price")), "rows", List.of(List.of(1234.5)))));
|
||||
assertThat(block(result.getRender(), "metrics").toString()).contains("$1,234.50", "12,000", "1234.5");
|
||||
assertThat(block(result.getRender(), "table").toString()).contains("$1,234.50");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Percentages cannot refer to labels, derived percentages or bad indexes.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void testInvalidPercentColumn()
|
||||
{
|
||||
for(Object source : List.of(0, 3, 1.5))
|
||||
{
|
||||
AiToolResult result = execute(input("summary", "Invalid share.", "table", input("columns", List.of(input("label", "Name"), input("label", "Share", "percentOfColumn", source)), "rows", List.of(List.of("JPEG", 0)))));
|
||||
assertThat(result.getModelContent()).contains("Dropped table");
|
||||
assertThat((List<?>) result.getRender().get("blocks")).isEmpty();
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Zero totals are defined, and charts never exceed twelve points.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void testChartLimitAndZeroTotal()
|
||||
{
|
||||
List<List<Object>> rows = new ArrayList<>();
|
||||
for(Integer i = 0; i < 13; i++)
|
||||
{
|
||||
rows.add(List.of("Item " + i, 0, 0));
|
||||
}
|
||||
AiToolResult result = execute(input("summary", "Thirteen items.", "table", input("columns", List.of(input("label", "Item"), input("label", "Count"), input("label", "Share", "percentOfColumn", 1)), "rows", rows),
|
||||
"chart", input("chartType", "bar", "labelColumn", 0, "valueColumn", 1)));
|
||||
assertThat((List<?>) block(result.getRender(), "chart").get("points")).hasSize(12);
|
||||
assertThat((List<?>) block(result.getRender(), "table").get("rows")).hasSize(13);
|
||||
assertThat(result.getModelContent()).contains("truncated to 12");
|
||||
assertThat(block(result.getRender(), "table").toString()).contains("0.0%");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Operator validation does not coerce raw numeric/boolean filter values.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void testCriteriaValidation()
|
||||
{
|
||||
AiToolResult result = execute(input("summary", "Links.", "filterLinks", List.of(
|
||||
input("label", "Invalid", "tableName", "media", "criteria", List.of(input("fieldName", "width", "operator", "INVENTED", "values", List.of(10)))),
|
||||
input("label", "Valid", "tableName", "media", "criteria", List.of(input("fieldName", "width", "operator", "EQUALS", "values", List.of(512)))))));
|
||||
assertThat(result.getModelContent()).contains("invalid criteria operator");
|
||||
List<?> links = (List<?>) block(result.getRender(), "filterLinks").get("items");
|
||||
String decoded = java.net.URLDecoder.decode((String) object(links.get(0)).get("href"), java.nio.charset.StandardCharsets.UTF_8);
|
||||
assertThat(decoded).contains("\"values\":[512]");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Invalid summaries produce an error tool result; no presentation is emitted.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void testInvalidSummaryAndAlwaysOffered() throws Exception
|
||||
{
|
||||
assertThat(runner.runtime().getContextBuilder().tools(new PolicyService.ToolPolicy(true, java.util.Set.of("navigate"), 20, true)))
|
||||
.anyMatch(tool -> PresentResultsTool.NAME.equals(tool.getName()) && !Boolean.TRUE.equals(tool.getStrict()));
|
||||
assertThat(execute(input("summary", "x".repeat(301))).getOutcome()).isEqualTo(AiToolOutcome.INVALID_INPUT);
|
||||
provider.thenToolUse(PresentResultsTool.NAME, Map.of()).thenText("Unable to present.");
|
||||
TurnRunner.Outcome outcome = runner.run(AssistantTestFixture.EDITOR, "Present");
|
||||
assertThat(outcome.events(TurnEvent.PRESENTATION)).isEmpty();
|
||||
assertThat(outcome.toolCalls().get(0).getOutcome()).isEqualTo("INVALID_INPUT");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Use the fixture's real UI gateway for unit-level presentation checks.
|
||||
*******************************************************************************/
|
||||
private AiToolResult execute(Map<String, Object> input)
|
||||
{
|
||||
return (new PresentResultsTool(qInstance).execute(new UiApiToolbox.Context(AssistantTestFixture.EDITOR, "UTC", runner.runtime().getUiApiGateway(), PolicyService.ToolPolicy.defaults(), null), "present", input));
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user