53 Commits
Author SHA1 Message Date
james.maesandClaude Fable 5.2 cc5fe2a700 feat(access): writes follow the signed-in person; no answer without a spending limit; fix: a fourth review of the tool boundary
maven / build (push) Successful in 7m33s
build / maven (push) Successful in 7m33s
Package A2 of the actions design (docs/actions-and-mcp.md, sections 2 and 3),
with two owner decisions of 2026-11-12, and five findings of an independent
review of the tool boundary, each closed by the guarantee it gives.

Writes follow the person
- The data access of a table or process is declared in versioned code:
  AssistantQBitConfig.withDefaultTableAccess / withDefaultProcessAccess (both
  READ_WRITE) and a pin per object (withTableAccess, withProcessAccess).
  READ_WRITE grants nothing: the assistant may propose what the signed-in
  person may do themselves, and every change waits for their confirmation.
- An administrator's assistant_data_access row can only narrow (READ, NONE).
  It never raises a code default, a pin or a table kept from AI.
- Every assistant_* table and every process of the QBit is NONE by a pin in
  code that no row and no host pin lifts.
- The write tools are offered to a person who may create, edit or delete
  records of some table (asked of the UI API as that person, once per offer,
  within the tool-call time limit; no answer means no write tool). GET /config
  features.writes is that answer for the person.
- Settings that cannot be read, and were never read, allow nothing.
- The table action "Add rows for every table" is retired. The changeset
  assistant-data-access-follow-user removes the unedited READ rows it wrote,
  and no other row.

No answer without a spending limit
- A person for whom no spending limit is in force (BudgetService.limitInForce)
  is refused: the turn before anything is stored, every model call, every tool
  call, every embedding request, a scheduled turn. Restoring a limit restores
  service on the next request.

Review findings
- A pending action is never marked expired over its outcome: expiry is a
  decision made through the same claim as confirming and cancelling.
- No action stays RUNNING for ever: retention ends one left running for more
  than an hour as FAILED ("outcome unknown, check the record"), with a log row.
- A person is the session the authentication module resolved, not the object
  passed in: a hand-made permission gives nothing, a revoked one is gone.
- The assistant's own reads get their context from the tool-call service, for
  a person it checked; there is no public way to make a tool context from a
  session id, a gateway and a policy.
- The published test-jar holds the testing package only.
- AssistantStore's decision methods are final.

See CHANGELOG.md and UPGRADE.md (what a host must apply before deploying).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-12 13:11:20 -06:00
james.maesandClaude Fable 5.2 f1cfea7734 Merge the actions and MCP design (docs/actions-and-mcp-design) into feat/writes-follow-the-person
The design this branch builds package A2 of. It is merged here so that the
sentences of the design that the code and the owner's decisions of 2026-11-12
have overtaken can be corrected in the same change.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-12 13:04:01 -06:00
james.maesandClaude Fable 5.2 0dc325cd2c fix(tools): one decision per pending action, a tool surface closed at compile time; feat: pending actions with before and after
maven / build (push) Successful in 6m19s
build / maven (push) Successful in 6m19s
Closes the findings of a third independent review of the tool boundary and
builds the pending-action package of the actions design on top.

Guarantees:
- A pending action is confirmed or cancelled once, by exactly one request,
  on one server or several: the decision is an insert under a unique key
  (new table assistant_pending_action_claim). Whatever fails after the claim
  ends in FAILED or EXECUTED with a tool-call row, never in RUNNING.
- A tool cannot be run around the tool-call service from code outside the
  assistant's packages: execute, log and withdraw are package-private; a
  tool call request, a confirmed tool call and a tool context have no public
  constructor and are made only with the service's permit; the runtime has
  no public getter for the tool registry, the UI API gateway or its tool
  families. A host's own ToolFamily and AssistantTool compile unchanged.
- A signed-in person is one person: a session with no HTTP request is made a
  person only when the authentication module resolves the session's own id
  to the user the session names. The turn engine no longer makes a person.
- Only the credential a request's session was resolved from is forwarded to
  the UI API.
- A tool call that was cut short leaves no proposal anybody could confirm.
- The policy a tool call runs under is read fresh on every call.
- The hard budget is read on every call with one aggregate query per budget.
- The input kept in the tool-call log is redacted with the administrator's
  custom patterns too.

Pending actions (design package A3):
- update_record and delete_record read the record as the person when they
  propose; the card shows each field before and after (a delete: what the
  record holds), after the exposure rules of get_record.
- What was approved is what is written: the proposal keeps a version of the
  record (a digest, never values); a confirmed update or delete reads the
  record again and refuses when it changed, is gone or cannot be read.
- actionKind (record_create, record_update, record_delete, or the proposing
  tool's name), GET /assistant/actions (the person's own list), and
  approvalRetentionDays is now read by the retention process.
- UI API extension assistant 1.9.0 (additive).

Schema: three columns and two indexes on assistant_pending_action, and the
table assistant_pending_action_claim (docs/schema-postgres.sql, UPGRADE.md).

ChatToolSetTest and ToolCallChecksTest pass unchanged.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-12 11:35:43 -06:00
james.maesandClaude Fable 5.2 7c9176fcc8 fix(tools): the tool-call service owns the policy, the identity and the confirmation
maven / build (push) Successful in 6m51s
build / maven (push) Successful in 6m51s
An independent security review of engine.ToolCallService (after PR 106)
found what a caller could still hand it. The chat's turn engine is the
only caller today and builds everything correctly, so none of it was
reachable; it is closed before an MCP endpoint calls the service. The
rule: every tool call runs as the signed-in person with their current
permissions and budget, never as the assistant; a refusal happens before
any data action; every call that ran has a log row with redacted input.

Numbered as in the review:

 1. A caller cannot supply a policy. The service reads it
    (PolicyService.current(false).tools(), narrowed for a scheduled
    task's conversation) and builds the tool Context itself; a caller
    hands it a tools.registry.ToolCall (person, id, name, input).
    Every change always shows a confirmation card (owner, 2026-11-12):
    ToolPolicy.writesRequireConfirmation is removed and UiApiToolbox
    always proposes. The assistant_policy column stays and is no longer
    read; no migration.
 2. New SignedInPerson binds a session to the credentials forwarded to
    the UI API. No public constructor: ofRequest(Context), only for
    AssistantSpecs (ArchUnit), or ofSession(session). The context's
    gateway is always runtime.getUiApiGateway(). AssistantApi and
    TurnEngine lose withCredentials.
 3. The hard budget is read on every call, not once per scope.
    executeAndLog(call, conversationId, hipaaRoute) makes the scope.
 4. confirm(person, pendingActionId) replaces executeConfirmed,
    confirmationRefusal and logConfirmed: the service loads the stored
    action of that person, checks it, claims it once, runs it, marks it
    and writes the row. AssistantApi.confirmAction is a thin caller and
    the only one (ArchUnit).
 5. The toolboxes cannot be called around the service: their entry
    points are package-private, the runtime has no getter for a toolbox
    (each family is handed its own), and ArchUnit forbids calls into
    tools.. from outside it except through the registry. The assistant's
    own reads (starter prompts, knowledge re-read) use the new read-only
    UiApiToolbox.read.
 6. TurnEngine.executeTools writes each started call's row in a per-call
    finally: after an interruption of the engine, after a runtime
    exception while a result is processed or logged, and for a call cut
    short by its time limit, with the real outcome when the worker
    finished. A row that cannot be written for a call that changed data
    is logged at ERROR.
 7. Whatever fails after a confirmation's claim ends in FAILED with a
    row, never in RUNNING.
 8. The service checks that the scope's conversation and turn belong to
    the call's person and works out the row's client channel itself
    (ToolCallScope.clientChannel is gone). hipaaRoute stays the
    caller's; the javadoc says why.
 9. AssistantQBitConfig.withToolFamily throws once the QBit is produced.

Tests: ToolCallServiceTest (one per item, on the service directly),
ToolCallRowsTest (item 6; all three fail on develop), and
ToolCallArchitectureTest (items 2, 4, 5). ChatToolSetTest and
ToolCallChecksTest pass with unchanged expectations.
UiApiToolsSecurityTest's two tests that ran writes with confirmation off
now propose and confirm.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-12 10:37:13 -06:00
james.maesandClaude Fable 5.2 694309f528 docs(design): name the open write-mode draft the design would replace
maven / build (push) Successful in 4m48s
build / maven (push) Successful in 4m48s
The write-mode switch is not on develop but exists as an open draft
pull request (#21). Say so, and say what happens to it if the owner
confirms that every write is confirmed.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 17:05:27 -06:00
james.maesandClaude Fable 5.2 798acd5541 docs(design): actions and MCP, the assistant does what the signed-in person may do
maven / build (push) Successful in 5m31s
build / maven (push) Successful in 5m31s
Draft design and work plan for owner review. Nothing is implemented here.

- Writes follow the person's permissions by default: data access defaults
  declared in code, an administrator can only restrict, every write is
  confirmed on a card that shows the values before and after.
- New tools: run a process the person may run, create and edit saved
  reports and saved analyses, schedule delivery by email.
- An MCP endpoint that serves the same tool registry as chat: read-only
  first, then writes as proposals approved in the application.
- Test strategy, repositories and release order, risks, decisions for the
  owner, and work packages that can be built in parallel.

docs/DESIGN.md gets a pointer under "Next phases".

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 17:04:23 -06:00
james.maesandClaude Fable 5.2 7c54a02a87 feat(tools): save and schedule an analysis with one confirmation; decided actions reach the conversation; ask before guessing
maven / build (push) Successful in 6m31s
build / maven (push) Successful in 6m31s
save_analysis proposes keeping an analyze_records question as a saved analysis the user owns
and, when asked, a daily schedule emailed to them. One pending action lists everything that
will be created with the numbers as they are now; nothing exists until the user confirms it
(never auto-confirmed). On confirmation both records are created through the data-analysis
tables' own insert operations with the confirming user's session. Offered only with the
data-analysis extension and data access READ_WRITE on data_analysis_saved (a schedule also
needs it on data_analysis_schedule); the user's own insert permission is read from
GET /metaData first.

A schedule that could never run is refused when proposed and again at confirmation, with
what is missing: the instance is read (a schedule table, a scheduler on
data_analysis_run_schedules) and the host is asked through
AssistantQBitConfig.withScheduledDeliveryCheck.

Confirming any pending action now adds a message of the assistant to the conversation
(what was done, links to the records), and each earlier proposal carries what became of it
in the next request (done, failed, cancelled, expired). Before, the model only ever saw
'waiting for confirmation'.

Rule 14 for every model: ask one short question when a request is ambiguous in a way that
changes the result (table, field, time zone); the user's browser time zone is stated in the
per-user context. Golden task clarify-ambiguous-field.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 07:34:51 -06:00
james.maesandClaude Fable 5.2 e634d6a192 feat(tools): analyze_records answers grouped questions through the data-analysis pivot
A new read tool: group a table by one or two fields (a date field optionally by a calendar
unit) and compute a count, sum, average, minimum, maximum or distinct count per group, with
the usual filter. It is one call of the data-analysis qbit's pivot operation
(POST /qbit/data-analysis/tables/{table}/pivot) through the same gateway and with the
signed-in user's session, so table read, the table's analyze permission, field access and
record locks decide as on the Analyses screens. The assistant only narrows it: data access
READ on the table, fields whose values it may see, the policy's row cap. Offered only where
the data-analysis extension is registered; no production dependency on that qbit.

Tests run against the released data-analysis qbit served by the real UI API.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 07:06:27 -06:00
james.maesandClaude Fable 5.1 ca9212731d fix(api): say NOT_CONFIGURED, with what to configure, when no model provider is usable (#83)
QRun-IO/ci-policy: ci / not-enrolled (push) Skipped
QRun-IO/ci-policy: ci / fork-guard (push) Skipped
QRun-IO/ci-policy: ci / gradle-0 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-2 (push) Skipped
QRun-IO/ci-policy: ci / node-3 (push) Skipped
QRun-IO/ci-policy: ci / node-4 (push) Skipped
QRun-IO/ci-policy: ci / source-config-5 (push) Skipped
QBit CodeQL / Analyze branch source (push) Canceled after 0s
CodeQL / analyze (push) Canceled after 0s
QRun-IO/ci-policy: Shared Java Maven CI with Docker / verify (push) Successful in 7m45s
QRun-IO/ci-policy: Shared Java Maven CI with Docker / secrets (push) Failing after 21s
QRun-IO/ci-policy: Shared Java Maven CI with Docker / gate (push) Failing after 8s
QRun-IO/ci-policy: ci / java-docker-1 (push) Failing after 37m15s
QRun-IO/ci-policy: ci / ci-result (push) Failing after 5s
The probe reports DISABLED, NOT_CONFIGURED, NO_PERMISSION, KILL_SWITCH in that order; GET /config gains an additive message naming what to configure; a user without access on an unconfigured server gets 503 NOT_CONFIGURED instead of 403. Extension 1.7.0.

Fixes #82

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 11:30:04 -06:00
james.maesandClaude Opus 5.5 e4f369a4e7 feat(config): build starter prompts per user from what they can access
GET /assistant/config returned the profile's starter prompts as static
text, so a user could be offered examples about tables they cannot see.
profile.starterPrompts (same field and shape) is now built per user:
up to three prompts generated from the tables the user can read through
the assistant that have rows (list_tables, count_records and
describe_table via the UI API with the user's own session), cached five
minutes, stable for the day, bounded and fail-soft. Configured prompts
stay only when tagged with tables the user can read.

Part of KofTwentyTwo/Website-Backend#303

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 15:34:35 -05:00
james.maesandClaude Opus 5.5 cfad90eaee fix(assistant): optional unreachable providers degrade health; add provider setup state
An enabled provider that is not the default and not the only route for any
configured routing (enabled profile simple/complex model, budget downgrade,
knowledge source embedding, with no fallback on another usable provider) is
optional. When it is unreachable its provider and model checks now report
DEGRADED with a message naming where requests go, instead of turning the
whole "assistant" component DOWN (central health rolls up worst-of; WARNING
severity does not cap DOWN). The default provider and sole routes stay DOWN.
Failed provider probes carry reachable=false so the circuit breaker still
refuses the provider when its central outcome is only DEGRADED.

New assistant_provider.setup_state (Active / Setting up, empty = Active):
a provider that is Setting up is never routed to (default, profile models,
fallbacks, budget downgrades, embeddings) and gets no provider or model
health checks. Additive guarded DDL in docs/schema-postgres.sql; see
UPGRADE.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 11:43:58 -05:00
james.maesandClaude Opus 5.5 38ac805fe7 feat(assistant): POST /assistant/client-errors with the assistant_client_error table and health check
UI API extension assistant 1.5.0 adds POST /qqq/v1/assistant/client-errors
(AssistantClientErrorReportV1 in, AssistantClientErrorResponseV1 out, 202).
Any signed-in user may report; scope and message are required (400
otherwise, also for a non-object body). Every field is bounded, message and
stacks are scrubbed (AiRedactor secrets, e-mails, digit runs over 6, mixed
tokens, query strings), the route keeps only its path, and conversation /
message / turn ids are kept only when they are the user's own. Over
clientErrorsPerUserPerMinute (default 30) the report is acknowledged with
accepted:false and dropped, never a 429.

Rows go to the new owner-locked assistant_client_error table (Assistant app
nav, record screens, bundle help, channel wall by pattern, SET NULL foreign
keys in the reference DDL) and are deleted by retention after
toolResultRetentionDays. The assistant.client-errors health check is
DEGRADED at clientErrorWarnPerHour (default 20) reports in the last hour, so
the Assistant health card shows it too. The deprecated AssistantErrorV1
description again says "since 1.3.0".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 20:38:12 -05:00
james.maesandClaude Opus 5.5 15938e07e1 Merge develop (durable turns, defaults, indexes, FKs) into feature/transcript-redaction
- Turn start keeps develop's supersede/concurrency, run tracker,
  register and forceFinalize, and attaches the TranscriptRedactor.
- The shared finalization flushes redaction after the final events and
  before the terminal turn update on every path (supersede, timeout,
  restart recovery, forceFinalize); the redactor always closes.
- Token-bearing columns widened to TEXT (H2-safe upgrade statements),
  entity length limits removed so tokens never truncate.
- Event reader race that could omit terminal events fixed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 18:59:03 -05:00
james.maesandClaude Opus 5.5 80d529e741 Merge feature/schema-indexes (with origin/develop) into feature/foreign-keys
Conflicts in CHANGELOG.md and qbit-assistant/pom.xml: kept develop's
durable-turns and starter-defaults entries and its H2/Liquibase test
dependencies alongside the Testcontainers/Postgres ones.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:55:19 -05:00
james.maesandClaude Opus 5.5 b50bc4a5c7 Merge origin/develop into feature/schema-indexes
Conflicts in UPGRADE.md and docs/schema-postgres.sql: kept develop's
durable-turns and starter-defaults notes and DDL; the index audit block
now follows the durable-turns block at the end of the schema.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:51:27 -05:00
james.maesandClaude Opus 5.5 00861af42c feat(turns): lease, heartbeat, cancel request and phase columns on assistant_turn
heartbeat_at, lease_owner, cancel_requested_at, phase and concurrency_mode,
a (user_id, status) index, and a partial unique index so a turn gets one
message.end even when two replicas reap it. Store helpers for a conditional
update of a still-running turn.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 16:36:39 -05:00
james.maesandClaude Opus 5.5 e8beb7444a Merge develop (spi package) into feature/transcript-redaction; build on released QQQ 4.1.0-RC.7
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 16:27:32 -05:00
james.maesandClaude Opus 5.5 fdf8d72367 feat(schema): add the missing foreign keys with orphan quarantine
Reference DDL gains a guarded, re-runnable foreign key block after the
index audit: the conversation tree CASCADEs (matching
AssistantStore.deleteConversation and retention), the message/turn cycle,
usage and the data audit are SET NULL so cost rows and the audit trail
survive, provider references from usage, budget accounting and the data
audit are NO ACTION, profile-skill links CASCADE, and
assistant_model.provider_id becomes CASCADE (fk_assistant_model_provider,
as on kof22). Orphans are recorded in qqq_fk_quarantine first. Every FK
column now leads an index.

SchemaForeignKeyPostgresTest applies the DDL on Postgres (Testcontainers)
and fails any declared relation without a foreign key, a wrong delete
rule, an unindexed FK column, or a broken conversation delete order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 16:23:25 -05:00
james.maesandClaude Opus 5.5 8aac6cf92d docs(schema): add the missing lookup, retention and FK indexes
The reference Postgres schema gains six IF NOT EXISTS indexes:
- assistant_usage (turn_id): ScheduledTurn.cost
- assistant_turn_event (create_date), assistant_tool_call (create_date) and
  assistant_conversation (modify_date): the retention filters
- assistant_scheduled_run (conversation_id) and (turn_id): the ON DELETE
  SET NULL foreign keys

UPGRADE.md notes them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 15:26:08 -05:00
james.maesandClaude Opus 5.5 6c36917beb feat: no raw PII/PHI in stored transcripts (secured data §15)
- Stored copies of tool results replace PII/PHI field values with
  ClassifiedValueTokens.forRecordField tokens (core, QRun-IO/qqq#1182);
  model-written text, persisted text deltas, presentation parts, tool
  inputs, run summaries and audit reasons have every classified value
  the turn saw replaced by its token. What the model receives (as the
  PHI routing allows) and the live SSE to the entitled user are
  unchanged.
- Pending writes that would carry a classified value are refused rather
  than stored (follow-up: an encrypted short-lived payload).
- A core log redaction hook masks seen classified values in assistant
  logs.
- Canary test: CANARY-PHI/PII markers never appear (nor any >4-char
  substring) in any assistant_* table, logs or audits, while the model
  did receive them in memory.
- Requires QQQ 4.1.0-RC.7.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 11:22:50 -05:00
james.maesandClaude Opus 5.5 8655ea21d4 refactor!: provider SPI in its own package com.kingsrook.qbits.assistant.spi
The qbit-assistant-provider-api artifact now holds exactly one package,
com.kingsrook.qbits.assistant.spi (23 types including ModelPricing), so
consumers such as qbit-marketing depend only on that artifact and the
qbit-bom G1 guardrails can allow it by package. BREAKING: imports change
from ...assistant.provider.X / ...assistant.engine.ModelPricing to
...assistant.spi.X (UPGRADE.md has the table). ArchUnit forbids split
packages between the API and the implementation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 10:51:34 -05:00
james.maesandClaude Opus 5.5 1020c3b324 feat: withRecordHistory(true) host opt-in for get_record_history
Default off (owner: hard wall; kof22 does not opt in). When on, calls
core's ClientChannelsMetaData.withAssistantAuditReads() (QRun-IO/qqq#1193:
audits readable, never writable, worm_audit_* still denied) through a
STAND-IN(core=QRun-IO/qqq#1193, removeBy=RC.8) lookup, falling back on
RC.6 to removing exactly the audit* pattern. Other patterns are never
touched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 08:53:10 -05:00
james.maesandClaude Opus 5.5 b0b80ead3d feat: the hard wall and audit attribution (X1/Y1)
X1:
- Every assistant_* table, process, report and widget (and every object
  the qbit produces) is denied to client channel assistant: the
  assistant_* pattern in ClientChannelsMetaData plus
  QPermissionRules.withDeniedClients on each object.
- Startup fails if any of them is not denied.
- Every UI API call carries assistant/<turnId> (scheduled:
  assistant/scheduled/<taskId>); confirm and explain-access carry the
  owning turn.
- Tests: assistant.admin through the assistant is denied (403, absent
  from list_tables and /metaData) on every configuration object, while the
  same admin through next-ui is allowed; the startup check fails on one
  undenied object; core's security-surface patterns are present.
Y1:
- assistant_tool_call.audit_href: an /app link to the written record's
  audit rows filtered by clientChannel, built with the shared filter-link
  builder.
- Tests: an assistant write audits the human actor with
  clientChannel assistant/<turnId>; the same write from the UI audits
  next-ui. Hosts enable the audit clientChannel field (documented).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 08:17:35 -05:00
james.maesandClaude Opus 5.5 8fab9a856e Merge develop (P-g scheduled tasks, provider-api) into #19
Scheduled runs go through the invoker as assistant/scheduled/<taskId>;
the designated-user manageSession login stays a plain UI-client login.
get_record_history is offered only when the host opts the audit tables
in (core's default wall denies audit* to the assistant). Reserved-header
filtering in the stand-in header invoker is tested.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 07:45:15 -05:00
james.maesandClaude Opus 5.5 6e7438a2da Merge develop into feature/spec-endpoints-core-invoker (QQQ RC.6)
- CoreUiApiGateway on UiApiInvoker for every tool (P tools, bulk-load,
  presentation), client channel assistant/<turnId>; form params mapped to
  withFormParam; LoopbackUiApiGateway deleted.
- Extra headers (X-QQQ-AI-Route) through HeaderedUiApiInvoker, a stand-in
  until core UiApiRequest has withHeader.
- Core DataClassification replaces the qbit stand-in; the default
  classification source merges core metadata with the config map.
- Bulk-load confirm dispatch kept on the spec'd confirm path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 07:31:39 -05:00
james.maes 9e1bda8dae Merge remote-tracking branch 'origin/develop' into feature/p7-scheduled-tasks
# Conflicts:
#	README.md
#	qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/engine/TurnEngine.java
#	qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantRecordScreens.java
2026-10-13 20:37:33 -05:00
james.maesandClaude Opus 5.5 907d943f8d feat: scheduled assistant tasks as a designated user through the UI API (P-g)
- assistant_scheduled_task (Scheduled tasks): prompt, run-as user,
  credential env-var reference (allow-listed, never stored), profile,
  Quartz cron + time zone, output mode, write-mode request and per-task
  admin approval (cleared when the prompt, user, credential, profile or
  request changes), per-run cost cap; assistant_scheduled_run child list.
- Runner (60 s schedule): claims a slot via a unique (task, scheduledFor)
  run key, logs in exactly like a UI client (POST /qqq/v1/manageSession
  with accessToken), checks the session's user is the run-as user, then
  runs a normal turn on a new conversation (channel assistant/scheduled,
  X-QQQ-Client assistant/scheduled/<taskId>). No system or constructed
  sessions.
- PHI routing, budgets, routing/fallback apply unchanged; output as the
  conversation and run record, a notification through core alerts, or
  both. Record actions: Approve write mode, Revoke write mode, Run now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 20:34:46 -05:00
james.maes 5e1c213fd6 Merge pull request #34 from QRun-IO/feature/presentation-demo-shots
test(demo): answer presentation demo turn and panel screenshots
2026-10-13 20:29:45 -05:00
james.maes 2e90fbdc5d Merge remote-tracking branch 'origin/develop' into feature/p6-rag
# Conflicts:
#	CHANGELOG.md
#	README.md
#	qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/engine/ContextBuilder.java
#	qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/engine/SystemPrompt.java
#	qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/engine/TurnEngine.java
2026-10-13 20:06:13 -05:00
james.maesandClaude Opus 5.5 6a26a7d4fa test(demo): answer presentation demo turn; panel screenshots
The demo's mock host answers "How many images do I have by type?" with
count_records through the UI API and then present_results, so the
panel shows the owner's example (summary, metric, table with shares and
row links, pie chart, filter links, action). Screenshots light/dark at
390/1440 px with the Next UI build of qqq-frontend-next#111.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 20:03:20 -05:00
james.maesandClaude Opus 5.5 c27994c432 feat: answer presentation: present_results tool, presentation parts and the default presentation skill
Owner priority: answers must be easy to read and act on.

- present_results tool: summary, metrics, table (computed percent
  columns, per-row filtered-list links), bar/pie/line chart from the
  table, filter and record links, follow-up actions. Every table and
  field is checked against what the user can see through the UI API;
  hrefs (/app/<table>?filter=<QQQ filter JSON>, records) and number
  formats are built server-side from metadata; the model never writes
  URLs.
- presentation message part (persisted, SSE event) with a server-built
  multi-line GFM markdown fallback for clients without blocks.
- Static system-prompt Presentation section (built-in default skill):
  answer first, no restating, tables for 3+ items, charts for shares and
  trends, filterLinks whenever a list is offered.
- Golden eval of the owner's example: summary, 2-row table with 80.2% /
  19.8%, a 2-point pie and two mimeType filter links; negative cases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 19:54:05 -05:00
james.maes 1947688c34 Merge remote-tracking branch 'origin/develop' into feature/p6-rag 2026-10-13 19:51:42 -05:00
james.maesandClaude Opus 5.5 c487c87051 feat: RAG over instance content, permission-filtered through the UI API (P-f)
- Embeddings: EmbeddingProvider SPI, OpenAI-compatible /embeddings
  (batching, Azure path), assistant_model.model_kind CHAT/EMBEDDING and
  embedding_dimensions; chat selection ignores embedding models.
- VectorStore SPI: in-memory store, and the new qbit-assistant-rag-pgvector
  module (one schema per instance, HNSW cosine index, parameterized SQL).
  The index holds NO record content: table, key, field, chunk, hash and
  vector only.
- assistant_rag_source (Knowledge sources) with Index now / Clear index
  record actions that read through the UI API as the clicking admin,
  skip unchanged chunks and drop removed records.
- search_knowledge tool: vector search, then every hit re-read through the
  UI API with the asking user's session (403/404 hits dropped), AI field
  exposure and masking applied, /app citations; PHI rule for embeddings;
  classification raised by results.
- pgvector container test (pgvector/pgvector:pg16) passes with Docker.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 19:51:40 -05:00
james.maesandClaude Opus 5.5 cb7d5d4d5a refactor: budget alerts through core QAlerts (QQQ 4.1.0-RC.5), alert types declared
Closes #28 (first half; the DataClassification and loopback gateway
stand-ins go with RC.6).

- Default QQQ is 4.1.0-RC.5.
- BudgetAlerts calls QAlerts.raise(AlertSignal...) directly instead of
  binding by reflection.
- The producer declares assistant.budget.warning, .downgraded and .stopped
  as QAlertTypeMetaData (assistant.admin, subject assistant_budget).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 19:30:01 -05:00
james.maesandClaude Opus 5.5 55804d9059 feat: simple/complex model routing and fallback chains across providers (P-e)
- Profiles gain routing (OFF, HEURISTIC, CLASSIFIER) with simple, complex
  and classifier models. The heuristic is a small documented rule set; the
  classifier is one 4-token call on the question text only (ROUTE usage,
  3 s timeout, heuristic on any failure, skipped for PHI on a
  non-eligible provider).
- assistant_model.fallback_model_id chains (cycle-checked, max 4 hops)
  across providers; retryable failures before any output fall back.
  Candidates that are disabled, down, over a STOP budget or fail the PHI
  rule are skipped; without a chain, LOCAL falls back only to LOCAL.
- Circuit breaker (provider health UP/DOWN, down_until) and a cached
  health probe for stale LOCAL providers.
- A mid-turn provider switch rebuilds history from neutral parts and drops
  adapter state (Anthropic thinking signatures).
- Turn route decision, source and fallback count; home shows health and
  fallbacks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 19:09:26 -05:00
james.maesandClaude Opus 5.5 51fcfa6a33 feat: data-driven budgets with warn, downgrade, stop and admin overrides (P-d)
- assistant_budget (scope INSTANCE/USER/ROLE/PROVIDER/MODEL, DAY/MONTH,
  cost or tokens, warn %, DOWNGRADE or STOP at 100%, hard ceiling),
  assistant_budget_override (reason, grantor, expiry) and
  assistant_budget_event (deduplicated WARNED/DOWNGRADED/STOPPED).
- BudgetService checks before every model call; strictest outcome wins.
  Warnings raise a panel notice and an optional Alerts signal; downgrade
  prefers a cheaper LOCAL model and never crosses the PHI rule.
- assistant_budget_usage: an owner-lock-free accounting projection of
  usage counters so shared-scope sums run under the caller's own session.
- App home: budgets card with state and overrides, 30-day cost and token
  bars with user, model, provider and tool breakdowns.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 18:43:45 -05:00
james.maesandClaude Opus 5.5 d15c7dce12 feat: data classification, PHI routing and HIPAA mode (P-c)
- DataClassification (PUBLIC < CONFIDENTIAL < PII < PHI): a qbit
  stand-in for core's (QRun-IO/qqq#1142) behind a pluggable
  ClassificationSource (config by default; core metadata or
  qbit-secured-data overrides later).
- Each turn tracks the highest classification its page context, history
  and tool results touched (turn.classification, conversation
  max_classification, never lowered).
- PhiRoutingGuard before EVERY model call: PHI only to hipaa_eligible
  provider rows; otherwise the turn ends PHI_BLOCKED with the reason and
  the tool results are never sent.
- HIPAA mode: no non-eligible providers (resolve, enable, code fallback),
  AES-256-GCM conversation storage (key by env-var reference, required),
  PHI retention period, no provider messages or stack traces in logs.
- assistant_data_audit: one row per model call that carried PII/PHI and
  per block (who, classification, tables, provider, model, decision).
- System rule 11: masked values stay masked; never ask to reveal them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 18:20:13 -05:00
james.maes 88e1081396 Merge remote-tracking branch 'origin/feature/p1-provider-registry' into feature/p2-bedrock-vertex 2026-10-13 17:47:42 -05:00
james.maesandClaude Opus 5.5 c9ccd4c79b feat: AWS Bedrock and Google Vertex adapters (P-b)
- qbit-assistant-provider-bedrock: Claude on Bedrock through the
  anthropic-java Bedrock backend (AnthropicMapping reused), credentials
  from the server's AWS default chain, region required, no key reference.
  IRSA/web identity needs the host to add software.amazon.awssdk:sts.
- qbit-assistant-provider-vertex: Claude on Vertex through the
  anthropic-java Vertex backend, Gemini through Vertex's OpenAI-compatible
  endpoint with an ADC bearer token refreshed per request (never strict).
  Region and the new cloud_project column are required.
- AnthropicModelProvider.Builder.client(...) and a bearer-token supplier
  on ProviderSettings / the OpenAI adapter.
- Mocked contract tests: a Bedrock event-stream mock with SigV4 checks,
  Vertex rawPredict and OpenAI-compatible mocks, validation, and a golden
  tool round trip per module.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 17:47:40 -05:00
james.maesandClaude Opus 5.5 a7be39db0f feat: readable labels for the provider registry; P-a demo server and screenshots
- Kind, source and field labels (HIPAA-eligible, API version, prices
  per million tokens, timeouts) instead of raw ids.
- Test-only OpenAiDemoServer: a mock OpenAI-compatible host, real
  Discover models / Test connection runs and one turn.
- Screenshots of the home providers card, a local provider, its Models
  child list and a model, light and dark at 390 and 1440 px.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 17:33:43 -05:00
james.maesandClaude Opus 5.5 8d869f65cd feat: provider and model registry, OpenAI-compatible adapter, discovery and health (P-a)
- assistant_provider gains kind, location, hipaa_eligible, region, api
  version, timeouts, a CA bundle reference and discovery status.
- New assistant_model table (child list on the provider): capabilities,
  prices per million tokens, enabled flag, last test.
- Provider SPI: ProviderSettings, kinds, discovery and health as default
  methods; create(apiKey, baseUrl) implementations keep working.
- New module qbit-assistant-provider-openai: OpenAI, Azure OpenAI,
  OpenRouter, Ollama, LM Studio, vLLM and llama.cpp over JDK HTTP, with
  streaming, tool calls, usage, timeouts, cancel, strict only where
  supported (and the existing fallback on refusal), keyless local
  endpoints and per-provider private CA trust.
- Record actions: Discover models (provider), Test connection (model);
  provider Test connection checks health first.
- Mock-server contract tests, a golden tool round trip, TLS tests and a
  config-driven live check (skipped unless configured).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 17:17:39 -05:00
james.maesandClaude Opus 5.5 f2eb20a8bc docs: spec for write mode, the config wall, audit attribution and providers
Owner decisions (2026-10-13), continued, with all recommendations
accepted:
- write mode per conversation (Read-only by default), user preference,
  admin policy, confirmed writes, auto-confirm rules;
- the hard wall: the assistant client channel is denied every assistant_*
  object and the security surfaces in core, plus a startup check;
- audit attribution: the human actor plus via=assistant/<turnId>;
- providers and models as data (OpenAI-compatible, Bedrock, Vertex),
  local models, HIPAA/PHI routing, budgets, routing, RAG and scheduled
  tasks.
Also the phase plan and its dependencies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 16:20:43 -05:00
james.maes 77bd8df7a2 Merge develop into feature/spec-endpoints-core-invoker 2026-10-13 16:07:23 -05:00
james.maesandClaude Opus 5.5 f6edea4b7b feat!: spec'd /qqq/v1 assistant endpoints; tools on core's UiApiInvoker
Phase (d). Needs QQQ with QRun-IO/qqq#1128 (extension point, invoker,
credentials forwarding).

- AssistantSpecs: the "assistant" UI API extension (1.0.0, established
  prefix /assistant): config, conversations, turns (text/event-stream),
  events, cancel, feedback, actions (get/confirm/reject) and access, with
  the same URLs, bodies and statuses. The raw routes are removed; the
  route provider only learns the local port.
- CoreUiApiGateway over UiApiInvoker replaces the loopback stand-in. The
  caller's credentials (UiApiCredentials.fromContext) flow from the spec
  to the tools; the assistant's own operations are never invocable.
- The OpenAPI fragment is committed and contract-tested with UiApiContract.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 14:44:34 -05:00
james.maesandClaude Opus 5.5 0f34c8a1dc test(demo): the panel proposes a page change; screenshots of the card
The demo queues recorded answers that propose update_record on page 12
(READ_WRITE), so a question in the panel shows the confirmation card;
Save runs it through PATCH /qqq/v1/table/page/12 with the user's session.
Screenshots of the persona, the pending card and the confirmed write
(light/dark, 390/1440), taken locally against the demo server.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 14:34:36 -05:00
james.maesandClaude Opus 5.5 6634d88286 fix: log denied tool calls against their table; demo seeds persona and data access
- The tool's table is recorded before the call, so a call the UI API
  refuses still names its table in the log and in Recent denials.
- AssistantDemoServer seeds a default profile (Ada), an analysts profile,
  two skills (one linked) and data access rows (page READ_WRITE,
  media NONE).
- Screenshots of the Recent denials card, Explain access widget, Data
  access table, profile, skill and policy (light/dark, 390/1440).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 14:24:30 -05:00
james.maesandClaude Opus 5.5 0babaa7022 feat: profiles, skills, explain access and recent denials
Owner (2026-10-13): set a name, personality and extra skills for the
assistant, and the assistant stays limited to the UI API and to what the
signed-in user could do through it.

- assistant_profile (persona: display name, icon, bounded personality,
  greeting, starter prompts; default, app- or permission-scoped) and
  assistant_skill (admin Markdown instruction packs, read only when
  relevant, bounded) linked by assistant_profile_skill. Text only: it
  goes into the per-user system block and never reaches tool code.
- Explain access: GET /assistant/access and the assistantAccess home
  widget, per table and tool: allowed or why not, from GET /metaData with
  the user's own session plus data access and policy.
- Recent denials card on the Assistant home.
- Hard guarantee: ArchUnit forbids the tool / UI API / access packages
  from QQQ actions, the AI kernel and the profile/skill classes, and the
  store refuses any non-assistant_ table. A denial matrix proves query,
  get, insert, update and delete are denied identically via the UI API
  and via the assistant.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 14:10:47 -05:00
james.maesandClaude Opus 5.5 01cc719c5b feat!: assistant tools go through the /qqq/v1 UI API with the user's session
Owner decisions (2026-10-13): tools via the UI API, configurable and
verifiable data access, and confirmation for writes.

- Every data tool runs a spec'd /qqq/v1 operation (UiApiGateway, a
  loopback request carrying the user's own sessionUUID cookie), so it
  passes the UI's authentication, permissions, record locks, validation,
  audit and error mapping. LoopbackUiApiGateway is a stand-in for core's
  UiApiInvoker (QRun-IO/qqq#1128). navigate and search_help (no records)
  stay on the core kernel.
- Data access (assistant_data_access): NONE / READ / READ_WRITE per table
  or process; sensitive tables NONE and cannot be raised; it only narrows.
- Writes (create/update/delete_record) propose a pending action
  (assistant_pending_action) the owner confirms; confirm runs it once
  through the UI API with the confirming request's session.
- Policy: writesRequireConfirmation, allowedTools, maxRowsPerCall,
  redactPii. Tool-call log: UI API operation, HTTP status, audit link,
  pending action; inputs stored with secrets redacted.
- UiApiToolsSecurityTest proves a user without a permission gets the same
  denial through the assistant as through the UI; golden and red-team
  evals run through a real application server.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 13:56:18 -05:00
james.maesandClaude Opus 5.5 38bb123140 fix: stop sending every tool strict; surface provider error reasons
Every kof22 turn failed on its first model call with Anthropic 400
invalid_request_error "The compiled grammar is too large ... reduce the
number of strict tools". All ten read tools were strict, and the API
compiles every strict schema of a request into one grammar.

- Tools are not strict by default; hosts opt in with strictToolNames.
  The kernel validates every tool input (INVALID_INPUT) anyway.
- If a provider still refuses strict schemas, the turn retries once
  without strict tools and logs a warning.
- Provider errors carry the API error type, a sanitized message and the
  request id: logged at WARN on 4xx, request id kept on the usage row,
  the reason shown to assistant.admin holders and in Test connection.
- TurnEngine also catches Errors, so a turn never ends silently.
- AnthropicRequestContractTest checks the exact request against the
  Messages API constraints and a conservative strict-tool budget.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 13:25:28 -05:00
james.maesandClaude Opus 5.5 e54de89a49 feat: provider settings, per-call usage, model prices and the app home
- assistant_provider: type (registered AssistantProviderType; Anthropic),
  model, enabled/default, base URL override, cost cap, output and
  concurrency limits, last connection test. The key is never stored: the
  row names an allow-listed environment variable; apiKeyStatus is
  computed on read (last 4 at most) and dropped on save.
- Test connection: a record action (standard streamed ETL Review/Result)
  making one short live call; result saved on the row and as a TEST
  usage row.
- assistant_usage: one row per model call (turn steps and tests);
  assistant_model_price: editable USD/MTok used for every call's cost.
- The Assistant app home (health, providers, usage by day, costs vs caps,
  by user, top tools, recent failures); nav = home + main tables.
- AI exposure NONE on every assistant_* table.
- Tests, README, DESIGN, UPGRADE, CHANGELOG, reference DDL with the
  Anthropic price seed, demo server and screenshots.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 10:01:33 -05:00