Compare commits

..
Author SHA1 Message Date
james.maesandCodex 36b831dd5a ci: use the shared branch-aware CodeQL worker
Co-Authored-By: Codex <noreply@openai.com>
2026-10-21 10:24:43 -05:00
14 changed files with 33 additions and 288 deletions

No files matched your search

+2 -2
View File
@@ -1,12 +1,12 @@
version: 2.1
setup: true
orbs:
qqq-orb: kingsrook/qqq-orb@0.8
qqq-orb: kingsrook/qqq-orb@0.7.12
workflows:
setup:
jobs:
- qqq-orb/generate_pipeline:
orb_version: '0.8'
orb_version: 0.7.12
filters:
branches:
ignore: automation/qbit-rc-pending
-7
View File
@@ -1,7 +0,0 @@
name: build
on:
push:
jobs:
maven:
uses: self:KofTwentyTwo/ci-workflows/.gitea/workflows/maven.yaml@v1
secrets: inherit
+1 -1
View File
@@ -1,5 +1,5 @@
profile: qbit
orb_version: '0.8'
orb_version: 0.7.12
default_branch: develop
java_version: '21'
browsers: true
-12
View File
@@ -1,16 +1,5 @@
# Changelog
## Unreleased - Answer visible at the end of the stream
- The turn's redacted transcript (the question and answer parts, pending actions, stored events) is now written before `message.end` is sent. The panel reloads the conversation the moment the stream ends, and used to read the empty placeholders (`[]`) the transcript redactor stores until its flush, so the newest question and answer showed as empty bubbles until the panel was closed and reopened.
## Unreleased - Clear reason without a model provider (UI API extension 1.7.0)
- `GET /config` checks the server-wide causes before the user's permission: `DISABLED`, then `NOT_CONFIGURED` (no usable model provider), then `NO_PERMISSION`, then `KILL_SWITCH`. A server with no provider no longer tells users to ask for access (#82).
- UI API extension assistant 1.7.0: `GET /config` has `message` whenever `reason` is set: a sentence for the user. For `NOT_CONFIGURED` it says what an administrator configures (an enabled row on **Assistant > Providers**, for example Anthropic with `ANTHROPIC_API_KEY` set in the server environment, or a local OpenAI-compatible server such as Ollama).
- Without a usable provider, calls from a user who also lacks `assistant.hasAccess` answer `503 NOT_CONFIGURED` with that message instead of `403 NO_PERMISSION`; users with access keep their conversations, and a new question answers `503 NOT_CONFIGURED`.
- Verified the OpenAI-compatible adapter (kind `OLLAMA`) live against a local Ollama `llama3.3:70b-instruct-q4_K_M`, including a tool call (`LiveOpenAiCompatibleTest`).
## Unreleased - Conversation management (UI API extension 1.6.0)
- `assistant_conversation` is open to every assistant user (`assistant.hasAccess` or `assistant.admin`, through the new `AssistantAccessPermissionChecker`) in the standard table UI. Rows stay locked to their owner by the `userId` record lock, so a user only ever sees, updates and deletes their own conversations; admins with the all-access key still see everyone's. The Assistant app (its nav) and every other assistant table stay admin-only.
@@ -38,7 +27,6 @@
- Provider SPI: `ModelPart.Type.IMAGE` (`mediaType`, base64 `data`), mapped by the Anthropic, Bedrock and OpenAI-compatible providers. Claude models report vision. `ModelPart.toString()` never includes text or image data.
- `GET /config`: `features.attachments`, `limits.maxAttachmentBytes`, `limits.maxAttachments`, `limits.attachmentTypes`.
- System prompt: when `create_record` / `update_record` are offered, the model proposes the change instead of saying it cannot create records.
||||||| parent of c5125f9 (fix(assistant): store the transcript before message.end so the panel shows the answer)
## Unreleased - Optional provider health
-11
View File
@@ -479,17 +479,6 @@ OLLAMA, LM_STUDIO, VLLM, LLAMA_CPP or OPENAI_COMPATIBLE as their kind.
Registry prices override type/model prices, then configured prices. Unpriced
local models cost zero. Capabilities can be adjusted per registered model.
For a local Ollama with no key (the live test ran against `llama3.3:70b-instruct-q4_K_M`):
```java
config.withProviderType(new OpenAiCompatibleProviderType());
// provider row: type openai-compatible, kind OLLAMA, model llama3.3:70b-instruct-q4_K_M,
// base URL empty (http://localhost:11434/v1) or an allow-listed override such as http://127.0.0.1:11434/v1
```
Without any usable provider `GET /config` answers `enabled: false, reason: NOT_CONFIGURED` with a
`message` that says what to configure, before any permission check.
For Ollama behind a TLS proxy with a private CA and bearer authentication:
```java
+1 -1
View File
@@ -44,7 +44,7 @@ Guards:
`ProviderService.resolve(policyModel)`:
1. The enabled row marked default (marking one default unmarks the others), else the first enabled row.
2. With no enabled row, the provider registered in code (`withModelProvider`) and the policy's main model: hosts that never add a row keep working.
3. A row that cannot be used (no key, a disallowed name or URL, an uninstalled type) makes the assistant report `NOT_CONFIGURED` rather than fall back silently. `GET /config` reports it before `NO_PERMISSION`, with a `message` naming what to configure (#82).
3. A row that cannot be used (no key, a disallowed name or URL, an uninstalled type) makes the assistant report `NOT_CONFIGURED` rather than fall back silently.
The model is the row's, else the policy's main model. The rows and prices are cached for `policyCacheSeconds` and dropped on any change.
+15 -3
View File
@@ -20,7 +20,7 @@
<!-- Multi-module build. The qqq-orb sets the revision here; the modules -->
<!-- inherit it, and the flatten plugin resolves it in the published POMs. -->
<properties>
<revision>0.2.0-SNAPSHOT</revision>
<revision>0.1.0-SNAPSHOT</revision>
<maven.compiler.source>21</maven.compiler.source>
<maven.compiler.target>21</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
@@ -37,7 +37,7 @@
<parent>
<groupId>com.kingsrook</groupId>
<artifactId>qbit-build-parent</artifactId>
<version>2.2.1</version>
<version>2.1.1</version>
</parent>
<groupId>com.kingsrook.qbits</groupId>
@@ -217,7 +217,7 @@
<properties>
<!-- RC.7 provides classified-value tokens, log redaction and native help bundles; -->
<!-- RC.8 provides core job tracking (JobTracking, the Jobs tray). -->
<qqq.snapshot.version>4.1.0-RC.25</qqq.snapshot.version>
<qqq.snapshot.version>4.1.0-RC.15</qqq.snapshot.version>
</properties>
<dependencyManagement>
<dependencies>
@@ -230,6 +230,18 @@
</dependency>
</dependencies>
</dependencyManagement>
<repositories>
<repository>
<id>sonatype-snapshots</id>
<url>https://central.sonatype.com/repository/maven-snapshots/</url>
<releases>
<enabled>false</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
</repositories>
</profile>
</profiles>
</project>
@@ -86,20 +86,6 @@ public class AssistantApi
public static final int TITLE_MAX_CHARS = 60;
/////////////////////////////////////////////////////////////////////////////
// GET /config reason (and error code) when no model provider is usable: //
// no enabled provider row that can be built (a missing key, a disallowed //
// variable or URL, an uninstalled type) and no provider registered in code //
/////////////////////////////////////////////////////////////////////////////
public static final String NOT_CONFIGURED = "NOT_CONFIGURED";
public static final String NO_MODEL_PROVIDER_MESSAGE = "No model provider is configured on this server. An administrator adds an enabled provider on "
+ "Assistant > Providers (for example type Anthropic, whose API key variable ANTHROPIC_API_KEY must be set in the server environment, or a local "
+ "OpenAI-compatible server such as Ollama).";
public static final String NO_PERMISSION_MESSAGE = "You do not have access to the assistant.";
public static final String DISABLED_MESSAGE = "The assistant is turned off for this application.";
public static final String KILL_SWITCH_MESSAGE = "The assistant has been turned off by an administrator.";
private final AssistantRuntime runtime;
private final AssistantQBitConfig config;
private final AssistantStore store;
@@ -148,39 +134,27 @@ public class AssistantApi
Map<String, Object> result = new LinkedHashMap<>();
String reason = null;
String message = null;
/////////////////////////////////////////////////////////////////////////
// server-wide causes come before the user's own permission, so a //
// server without a usable model provider says so (QRun-IO/qbit- //
// assistant#82) instead of sending the user to ask for access //
/////////////////////////////////////////////////////////////////////////
if(!config.isEffectivelyEnabled())
{
reason = "DISABLED";
message = DISABLED_MESSAGE;
}
else if(activeProvider() == null)
{
reason = NOT_CONFIGURED;
message = NO_MODEL_PROVIDER_MESSAGE;
}
else if(!hasAccess())
{
reason = "NO_PERMISSION";
message = NO_PERMISSION_MESSAGE;
}
else if(activeProvider() == null)
{
reason = "NOT_CONFIGURED";
}
else if(runtime.getPolicyService().current(false).killSwitch())
{
reason = "KILL_SWITCH";
message = KILL_SWITCH_MESSAGE;
}
result.put("enabled", reason == null);
if(reason != null)
{
result.put("reason", reason);
result.put("message", message);
}
result.put("shortcut", config.getShortcut());
result.put("model", reason == null ? activeProvider().model() : config.getModels().getMainModel());
@@ -451,11 +425,11 @@ public class AssistantApi
ProviderService.Active active = runtime.getProviderService().resolve(policy.mainModel());
if(active == null)
{
throw (new AssistantApiException(503, NOT_CONFIGURED, NO_MODEL_PROVIDER_MESSAGE));
throw (new AssistantApiException(503, "NOT_CONFIGURED", "The assistant is not configured on this server."));
}
if(policy.killSwitch())
{
throw (new AssistantApiException(503, "KILL_SWITCH", KILL_SWITCH_MESSAGE));
throw (new AssistantApiException(503, "KILL_SWITCH", "The assistant has been turned off by an administrator."));
}
Long capMicros = ProviderService.capMicros(active.row());
@@ -1191,33 +1165,11 @@ public class AssistantApi
{
if(!config.isEffectivelyEnabled())
{
throw (new AssistantApiException(503, "DISABLED", DISABLED_MESSAGE));
throw (new AssistantApiException(503, "DISABLED", "The assistant is turned off for this application."));
}
if(!hasAccess())
{
if(activeProviderOrNull() == null)
{
throw (new AssistantApiException(503, NOT_CONFIGURED, NO_MODEL_PROVIDER_MESSAGE));
}
throw (new AssistantApiException(403, "NO_PERMISSION", NO_PERMISSION_MESSAGE));
}
}
/*******************************************************************************
** The active provider, or null when none is usable or it cannot be read
** (only used to pick the clearer refusal).
*******************************************************************************/
private ProviderService.Active activeProviderOrNull()
{
try
{
return (activeProvider());
}
catch(QException | RuntimeException e)
{
return (null);
throw (new AssistantApiException(403, "NO_PERMISSION", "You do not have access to the assistant."));
}
}
@@ -63,7 +63,7 @@ import io.javalin.http.HttpStatus;
public final class AssistantSpecs
{
public static final String EXTENSION_NAME = "assistant";
public static final String EXTENSION_VERSION = "1.7.0";
public static final String EXTENSION_VERSION = "1.6.0";
public static final String PREFIX = "/assistant";
@@ -198,8 +198,7 @@ public final class AssistantSpecs
"starterPrompts", array(string(null), "Up to three suggested first questions in total: permitted configured questions first, then generated questions. Table, row and field access are checked on every request.")));
rs.put("AssistantConfigResponseV1", object("Whether the assistant is available to this user, and how.",
"enabled", bool("Whether the panel may be used."),
"reason", string("Why not: DISABLED, NOT_CONFIGURED (no usable model provider on the server), NO_PERMISSION or KILL_SWITCH, checked in that order.").withNullable(true),
"message", string("Since 1.7.0: when reason is set, a sentence for the user that names the cause (for NOT_CONFIGURED, what an administrator configures).").withNullable(true),
"reason", string("Why not: DISABLED, NO_PERMISSION, NOT_CONFIGURED or KILL_SWITCH.").withNullable(true),
"shortcut", string("Keyboard shortcut, e.g. mod+j."),
"model", string("The model that answers."),
"limits", freeObject("maxTurnSeconds and maxToolCallsPerTurn. When features.attachments is true also maxAttachmentBytes (the largest file read, in bytes), "
@@ -1343,17 +1343,6 @@ public class TurnEngine implements Runnable
String finalErrorText = errorText;
safely(() -> emit(TurnEvent.ERROR, map("code", finalErrorCode, "message", finalErrorText)));
}
///////////////////////////////////////////////////////////////////////
// the stream ends at message.end and the panel reloads the //
// conversation at once, so the redacted transcript (message parts, //
// pending actions, events) must be stored before that event is sent //
///////////////////////////////////////////////////////////////////////
if(!alreadyTerminal)
{
safely(this::persistPendingEvents);
safely(() -> transcript.flush(store));
}
safely(() -> emit(TurnEvent.MESSAGE_END, map("messageId", assistantMessage.getId(), "status", finalStatus, "stopReason", finalStopReason)));
if(!alreadyTerminal)
{
@@ -175,13 +175,7 @@ components:
description: "Whether the panel may be used."
type: "boolean"
reason:
description: "Why not: DISABLED, NOT_CONFIGURED (no usable model provider\
\ on the server), NO_PERMISSION or KILL_SWITCH, checked in that order."
nullable: true
type: "string"
message:
description: "Since 1.7.0: when reason is set, a sentence for the user that\
\ names the cause (for NOT_CONFIGURED, what an administrator configures)."
description: "Why not: DISABLED, NO_PERMISSION, NOT_CONFIGURED or KILL_SWITCH."
nullable: true
type: "string"
shortcut:
@@ -311,9 +305,7 @@ components:
nullable: true
type: "string"
starterPrompts:
description: "Up to three suggested first questions in total: permitted\
\ configured questions first, then generated questions. Table, row and\
\ field access are checked on every request."
description: "Up to three suggested first questions in total: permitted configured questions first, then generated questions. Table, row and field access are checked on every request."
items:
type: "string"
type: "array"
@@ -476,7 +468,7 @@ info:
description: "QQQ Assistant: conversations, streamed turns, feedback, pending actions\
\ and data access explanations"
title: "QQQ Middleware API extension: assistant"
version: "1.7.0"
version: "1.6.0"
openapi: "3.0.3"
paths:
/qqq/v1/assistant/config:
@@ -166,45 +166,6 @@ class AssistantRouteProviderTest extends BaseAssistantTest
/*******************************************************************************
** No usable model provider (QRun-IO/qbit-assistant#82): the probe names the
** server's configuration, for users with and without access, and calls
** answer 503 NOT_CONFIGURED instead of 403 NO_PERMISSION.
*******************************************************************************/
@Test
void testNoModelProviderIsNotAPermissionProblem() throws Exception
{
String providerName = config.getProviderName();
config.setProviderName("no-such-provider");
try
{
for(String user : new String[] {EDITOR, NO_ACCESS})
{
Map<String, Object> probe = json(send("GET", "/config", user, null));
assertThat(probe).containsEntry("enabled", false).containsEntry("reason", "NOT_CONFIGURED");
assertThat((String) probe.get("message")).contains("No model provider").contains("Assistant > Providers").contains("ANTHROPIC_API_KEY");
}
HttpResponse<String> denied = send("GET", "/conversations", NO_ACCESS, null);
assertThat(denied.statusCode()).isEqualTo(503);
assertThat(json(denied)).containsEntry("code", "NOT_CONFIGURED");
assertThat((String) json(denied).get("message")).contains("No model provider");
assertThat(send("GET", "/conversations", EDITOR, null).statusCode()).isEqualTo(200);
HttpResponse<String> turn = send("POST", "/conversations", EDITOR, "{}");
Object id = json(turn).get("id");
HttpResponse<String> ask = send("POST", "/conversations/" + id + "/turns", EDITOR, "{\"text\":\"hi\"}");
assertThat(ask.statusCode()).isEqualTo(503);
assertThat(json(ask)).containsEntry("code", "NOT_CONFIGURED");
}
finally
{
config.setProviderName(providerName);
}
}
/*******************************************************************************
**
*******************************************************************************/
@@ -217,8 +178,7 @@ class AssistantRouteProviderTest extends BaseAssistantTest
assertThat((Map<String, Object>) config.get("limits")).containsEntry("maxTurnSeconds", 300).containsEntry("maxToolCallsPerTurn", 25);
assertThat((Map<String, Object>) config.get("features")).containsEntry("writes", false);
assertThat(json(send("GET", "/config", NO_ACCESS, null))).containsEntry("enabled", false).containsEntry("reason", "NO_PERMISSION")
.containsEntry("message", "You do not have access to the assistant.");
assertThat(json(send("GET", "/config", NO_ACCESS, null))).containsEntry("enabled", false).containsEntry("reason", "NO_PERMISSION");
HttpResponse<String> denied = send("GET", "/conversations", NO_ACCESS, null);
assertThat(denied.statusCode()).isEqualTo(403);
assertThat(json(denied)).containsEntry("code", "NO_PERMISSION")
@@ -141,55 +141,6 @@ class TurnEngineTest extends BaseAssistantTest
/*******************************************************************************
** The panel reloads the conversation as soon as message.end arrives, so the
** question and the answer must already be stored then, not after it.
*******************************************************************************/
@Test
void testMessagesAreStoredBeforeMessageEnd() throws Exception
{
provider.thenText("Four posts are published.");
TurnRunner.Outcome outcome = runner.run(EDITOR, "how many published posts?");
assertThat(outcome.end()).containsEntry("status", "COMPLETE");
List<AssistantMessage> stored = runner.messagesAtEnd();
assertThat(stored).hasSize(2);
assertThat(stored).extracting(AssistantMessage::getPartsJson).allSatisfy(parts -> assertThat(parts).isNotEqualTo("[]"));
assertThat(stored.get(0).getPartsJson()).contains("how many published posts?");
assertThat(stored.get(1).getPartsJson()).contains("Four posts are published.");
}
/*******************************************************************************
** The live thread after a tool-using turn: when message.end arrives the panel
** reloads, and must find the real title, the question, the answer and the tool
** steps - not the transcript redactor's placeholders (changed:v1, [], {}).
*******************************************************************************/
@Test
void testLiveThreadIsNotPlaceholdersAtMessageEnd() throws Exception
{
provider.thenToolUse("count_records", input("table", "blog_post", "filter", filter("AND", "status", "EQUALS", List.of("PUBLISHED"))))
.thenText("Four posts are published.");
TurnRunner.Outcome outcome = runner.run(EDITOR, "how many published posts?");
assertThat(outcome.end()).containsEntry("status", "COMPLETE");
assertThat(runner.conversationAtEnd().getTitle()).isEqualTo("how many published posts?");
List<AssistantMessage> stored = runner.messagesAtEnd();
assertThat(stored).extracting(AssistantMessage::getPartsJson).noneMatch(parts -> parts.equals("[]") || parts.contains("changed:v1"));
assertThat(stored.get(0).getPartsJson()).contains("how many published posts?");
assertThat(stored.get(stored.size() - 1).getPartsJson()).contains("Four posts are published.");
assertThat(runner.toolCallsAtEnd()).isNotEmpty().allSatisfy(call ->
{
assertThat(call.getInputJson()).isNotEqualTo("{}");
assertThat(call.getResultSummary()).doesNotContain("changed:v1");
});
}
/*******************************************************************************
** Parallel tool calls: all results in ONE user message, in order.
*******************************************************************************/
@@ -27,8 +27,6 @@ import com.kingsrook.qbits.assistant.AssistantRuntime;
import com.kingsrook.qbits.assistant.api.AssistantApi;
import com.kingsrook.qbits.assistant.engine.RunningTurn;
import com.kingsrook.qbits.assistant.engine.TurnEvent;
import com.kingsrook.qbits.assistant.model.AssistantConversation;
import com.kingsrook.qbits.assistant.model.AssistantMessage;
import com.kingsrook.qbits.assistant.model.AssistantToolCall;
import com.kingsrook.qbits.assistant.persistence.AssistantStore;
import com.kingsrook.qqq.backend.core.context.QContext;
@@ -49,10 +47,6 @@ public class TurnRunner
private final QInstance qInstance;
private final AssistantQBitConfig config;
private List<AssistantMessage> messagesAtEnd = List.of();
private AssistantConversation conversationAtEnd;
private List<AssistantToolCall> toolCallsAtEnd = List.of();
/*******************************************************************************
@@ -175,12 +169,6 @@ public class TurnRunner
{
List<TurnEvent> batch = running.getLog().after(last, 1000);
events.addAll(batch);
if(batch.stream().anyMatch(e -> TurnEvent.MESSAGE_END.equals(e.getType())))
{
messagesAtEnd = messagesOf(userId, convId);
conversationAtEnd = conversationOf(userId, convId);
toolCallsAtEnd = toolCalls(userId, running.getTurnId());
}
if(!batch.isEmpty())
{
last = batch.get(batch.size() - 1).getSeq();
@@ -206,74 +194,6 @@ public class TurnRunner
/*******************************************************************************
** The conversation's stored messages, read the moment the run's message.end
** was seen: what the panel finds when it reloads the conversation at the end
** of the stream.
*******************************************************************************/
public List<AssistantMessage> messagesAtEnd()
{
return (messagesAtEnd);
}
/*******************************************************************************
** The conversation row as stored the moment message.end was seen.
*******************************************************************************/
public AssistantConversation conversationAtEnd()
{
return (conversationAtEnd);
}
/*******************************************************************************
** The audited tool calls as stored the moment message.end was seen.
*******************************************************************************/
public List<AssistantToolCall> toolCallsAtEnd()
{
return (toolCallsAtEnd);
}
/*******************************************************************************
** One conversation of the user, read as the user.
*******************************************************************************/
private AssistantConversation conversationOf(String userId, Integer conversationId) throws Exception
{
QContext.init(qInstance, AssistantTestFixture.ownerSession(userId, config));
try
{
return (new AssistantStore().getConversation(userId, conversationId));
}
finally
{
QContext.clear();
}
}
/*******************************************************************************
** The stored messages of a conversation (read as the user).
*******************************************************************************/
private List<AssistantMessage> messagesOf(String userId, Integer conversationId) throws Exception
{
QContext.init(qInstance, AssistantTestFixture.ownerSession(userId, config));
try
{
return (new AssistantStore().listMessages(userId, conversationId, null, 50));
}
finally
{
QContext.clear();
}
}
/*******************************************************************************
** The audited tool calls of a turn (read as the user).
*******************************************************************************/