From e4f369a4e7a1e7c1c5cc69435914b71649812bdd Mon Sep 17 00:00:00 2001 From: James Maes Date: Fri, 16 Oct 2026 15:34:35 -0500 Subject: [PATCH 1/3] feat(config): build starter prompts per user from what they can access GET /assistant/config returned the profile's starter prompts as static text, so a user could be offered examples about tables they cannot see. profile.starterPrompts (same field and shape) is now built per user: up to three prompts generated from the tables the user can read through the assistant that have rows (list_tables, count_records and describe_table via the UI API with the user's own session), cached five minutes, stable for the day, bounded and fail-soft. Configured prompts stay only when tagged with tables the user can read. Part of KofTwentyTwo/Website-Backend#303 Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 4 + UPGRADE.md | 21 + docs/DESIGN.md | 2 +- .../qbits/assistant/AssistantRuntime.java | 13 + .../qbits/assistant/api/AssistantApi.java | 24 +- .../assistant/api/specs/AssistantSpecs.java | 2 +- .../assistant/engine/StarterPrompts.java | 525 ++++++++++++++++++ .../resources/openapi/qqq-v1/assistant.yaml | 4 +- .../access/ProfilesSkillsAndAccessTest.java | 6 +- .../assistant/engine/StarterPromptsTest.java | 291 ++++++++++ 10 files changed, 886 insertions(+), 6 deletions(-) create mode 100644 qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/engine/StarterPrompts.java create mode 100644 qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/engine/StarterPromptsTest.java diff --git a/CHANGELOG.md b/CHANGELOG.md index 688af47..f55d92e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,9 @@ # Changelog +## Unreleased - Starter prompts follow access + +- `profile.starterPrompts` in `GET /assistant/config` is built per user: up to three prompts generated from tables the user can read that have rows (UI API metadata and count as the user, bounded, cached five minutes, stable for the day), plus configured prompts only when tagged with tables the user can read (`[table] prompt`, `[none] prompt`; untagged lines are dropped). See UPGRADE.md. + ## Unreleased - Optional provider health - An enabled provider that is not the default and not the only route for any configured routing (an enabled profile's simple/complex model, a budget downgrade model or a knowledge source's embedding model, with no fallback on another usable provider) is optional: when it is unreachable, its `assistant.provider.` and `assistant.model.` checks report DEGRADED with a message naming the provider requests use instead, so one unfinished local server no longer turns the whole `assistant` component DOWN. The default provider and sole routes still report DOWN. A failed provider probe still carries `reachable=false`, so the circuit breaker keeps refusing the provider. diff --git a/UPGRADE.md b/UPGRADE.md index b8d2206..cce0bcb 100644 --- a/UPGRADE.md +++ b/UPGRADE.md @@ -1,5 +1,26 @@ # Upgrading qbit-assistant +## Starter prompts follow the user's access (Unreleased) + +`GET /assistant/config` no longer returns the profile's starter prompts as written. +`profile.starterPrompts` (same field, still `string[]`) is now built per user: + +- Up to three prompts are generated from the tables the signed-in user can read + through the assistant and that have rows (a count as the user), using the table + labels, a date field when present, and "What changed in ... this week?" when the + change-history tool is offered. The pick is stable for the user for the day and + cached for five minutes. Any failure yields no generated prompts. +- A configured starter prompt (Assistant Profile > Starter prompts, one per line) is + shown only when it declares the tables it needs and the user can read all of + them. Declare them with a leading tag: `[blog_post] Which posts are drafts?`, or + `[blog_post, page] ...` for several. `[none] What can you help with?` names no + table and is always shown. A line without a tag is no longer shown, because + nothing says which tables it mentions: add a tag to keep it. +- Shown prompts are the permitted configured ones first, then generated ones, three + in all (a profile with three or more permitted prompts shows only those). + +No schema change and no configuration. The panel needs no change. + ## Provider setup state (Unreleased) Apply the **Provider setup state** block at the end of `docs/schema-postgres.sql` diff --git a/docs/DESIGN.md b/docs/DESIGN.md index d17a8af..a96a0c8 100644 --- a/docs/DESIGN.md +++ b/docs/DESIGN.md @@ -124,7 +124,7 @@ Owner: *"the agent should have to go through the same API that the UIUX does. th 4. **Persona and skills, never more access** (owner, 2026-10-13): *"in the config for the AI settings, we should be able to set a name and personality and extra skills etc for the agent - and again - the agent should be limited to only using our UIUX API and being able to only do the things that the currently logged in user could do with the same UIUX API via the UIUX"*. - **Profiles** (`assistant_profile`, admin-edited): display name, icon, personality (tone instructions, at most 2,000 characters), greeting and up to 6 starter prompts. - One profile is the default. Others are scoped to an app, to users holding a permission (a role), or to both. The most specific profile wins. - - `/assistant/config` returns the profile for the panel header, the greeting and the starter prompts. + - `/assistant/config` returns the profile for the panel header, the greeting and the starter prompts (built per user from the tables the user can read; configured ones need a `[table]` tag, see UPGRADE.md). - **Skills** (`assistant_skill`, linked to profiles through `assistant_profile_skill`): named Markdown instruction packs of at most 6,000 characters, with a description, an enabled flag, trigger keywords and apps. - A skill is read only when relevant: a keyword appears in the question, the user is in one of its apps, or it has neither, in which case it is always read. - All skills read for one question total at most 12,000 characters. diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/AssistantRuntime.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/AssistantRuntime.java index dc505e8..500650a 100644 --- a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/AssistantRuntime.java +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/AssistantRuntime.java @@ -34,6 +34,7 @@ import com.kingsrook.qbits.assistant.engine.ClientErrors; import com.kingsrook.qbits.assistant.engine.ContextBuilder; import com.kingsrook.qbits.assistant.engine.PolicyService; import com.kingsrook.qbits.assistant.engine.ProfileService; +import com.kingsrook.qbits.assistant.engine.StarterPrompts; import com.kingsrook.qbits.assistant.engine.ProviderService; import com.kingsrook.qbits.assistant.engine.RateLimiter; import com.kingsrook.qbits.assistant.engine.RunningTurn; @@ -92,6 +93,7 @@ public class AssistantRuntime private final ContextBuilder contextBuilder; private final DataAccessService dataAccessService; private final ProfileService profileService; + private final StarterPrompts starterPrompts; private final AccessExplainer accessExplainer; private final UiApiToolbox toolbox; private final UiApiGateway uiApiGateway; @@ -145,6 +147,7 @@ public class AssistantRuntime this.accessExplainer = new AccessExplainer(qInstance, dataAccessService); this.toolbox = new UiApiToolbox(qInstance, dataAccessService, exposurePolicy, new AiRedactor(settings.getCustomRedactionPatterns()), config.getMaxCellChars() == null ? AiToolKernelSettings.DEFAULT_MAX_CELL_CHARS : config.getMaxCellChars()); + this.starterPrompts = new StarterPrompts(toolbox, clock); UiApiGateway delegate = config.getUiApiGateway() != null ? config.getUiApiGateway() : new CoreUiApiGateway(this::uiApiBaseUrl, qInstance); this.uiApiGateway = new UiApiGateway() { @@ -255,6 +258,16 @@ public class AssistantRuntime + /******************************************************************************* + ** Getter for starterPrompts (the per-user example prompts, built from what the user may read) + *******************************************************************************/ + public StarterPrompts getStarterPrompts() + { + return (starterPrompts); + } + + + /******************************************************************************* ** Getter for profileService (personas and skills: text only, never tools) *******************************************************************************/ diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/api/AssistantApi.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/api/AssistantApi.java index ece2c1d..e3f685d 100644 --- a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/api/AssistantApi.java +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/api/AssistantApi.java @@ -157,13 +157,35 @@ public class AssistantApi profile.put("displayName", persona.displayName()); profile.put("icon", persona.icon()); profile.put("greeting", persona.greeting()); - profile.put("starterPrompts", persona.starterPrompts()); + profile.put("starterPrompts", reason == null ? starterPrompts(persona) : List.of()); result.put("profile", profile); return (result); } + /******************************************************************************* + ** The example prompts for this user (see StarterPrompts): built from the + ** tables the user may read, plus the configured ones they may see. Never + ** throws. + *******************************************************************************/ + private List starterPrompts(ProfileService.Persona persona) + { + try + { + PolicyService.EffectivePolicy policy = runtime.getPolicyService().current(false); + UiApiToolbox.Context context = new UiApiToolbox.Context(session.getUuid(), session.getValue(QSession.VALUE_KEY_USER_TIMEZONE), runtime.getUiApiGateway(), policy.tools(), null, credentials); + boolean historyOffered = runtime.getToolbox().definitions(policy.tools(), false).stream().anyMatch(d -> UiApiToolbox.RECORD_HISTORY.equals(d.getName())); + return (runtime.getStarterPrompts().forUser(context, userId, persona.starterPrompts(), historyOffered)); + } + catch(QException | RuntimeException e) + { + return (List.of()); + } + } + + + /******************************************************************************* ** GET /conversations *******************************************************************************/ diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/api/specs/AssistantSpecs.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/api/specs/AssistantSpecs.java index e2eea36..e7146c8 100644 --- a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/api/specs/AssistantSpecs.java +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/api/specs/AssistantSpecs.java @@ -195,7 +195,7 @@ public final class AssistantSpecs Map rs = new LinkedHashMap<>(); rs.put("AssistantProfileV1", object("How the assistant presents itself (admin-configured; text only, it never changes access).", "displayName", string("The assistant's name."), "icon", string("A Material icon name."), "greeting", string("Shown on a new conversation.").withNullable(true), - "starterPrompts", array(string(null), "Suggested first questions."))); + "starterPrompts", array(string(null), "Suggested first questions, built for this user: up to three generated from the tables the user can read that have data, plus any configured ones whose tables the user can read."))); rs.put("AssistantConfigResponseV1", object("Whether the assistant is available to this user, and how.", "enabled", bool("Whether the panel may be used."), "reason", string("Why not: DISABLED, NO_PERMISSION, NOT_CONFIGURED or KILL_SWITCH.").withNullable(true), diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/engine/StarterPrompts.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/engine/StarterPrompts.java new file mode 100644 index 0000000..95e9bd3 --- /dev/null +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/engine/StarterPrompts.java @@ -0,0 +1,525 @@ +/* + * Copyright 2026 Kingsrook, LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.assistant.engine; + + +import java.time.Clock; +import java.time.Duration; +import java.time.Instant; +import java.time.LocalDate; +import java.time.ZoneOffset; +import java.util.ArrayList; +import java.util.Collections; +import java.util.Comparator; +import java.util.LinkedHashMap; +import java.util.LinkedHashSet; +import java.util.List; +import java.util.Locale; +import java.util.Map; +import java.util.Set; +import java.util.concurrent.ConcurrentHashMap; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.kingsrook.qbits.assistant.tools.UiApiToolbox; +import com.kingsrook.qbits.assistant.uiapi.UiApiCall; +import com.kingsrook.qbits.assistant.uiapi.UiApiGateway; +import com.kingsrook.qbits.assistant.uiapi.UiApiResult; +import com.kingsrook.qqq.backend.core.actions.ai.AiToolOutcome; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.logging.QLogger; +import com.kingsrook.qqq.backend.core.utils.StringUtils; + + +/******************************************************************************* + ** The example prompts the panel offers on a new conversation, built per user + ** from what that user can actually reach. + ** + ** The tables come from the same tool the model uses (list_tables, which is the + ** UI API's metadata narrowed by data access), the row counts and fields from + ** count_records and describe_table, all with the user's own session. So a + ** prompt can only name a table the user may read through the assistant, and + ** only one that has rows. + ** + ** Configured profile prompts are kept only when they declare the tables they + ** need, as a leading tag: {@code [blog_post] Which posts are drafts?} or + ** {@code [blog_post, page] ...}, and the user can read every one. A line with + ** {@code [none]} names no table and always stays. A line without a tag is + ** dropped, because there is no way to tell which tables it mentions. + ** + ** The result is stable for a user for the day (the pick is seeded by user id + ** and date, not random), cached for a few minutes, bounded in work and time, + ** and never throws: any failure yields no generated prompts. + *******************************************************************************/ +public class StarterPrompts +{ + private static final QLogger LOG = QLogger.getLogger(StarterPrompts.class); + + public static final int MAX_PROMPTS = 3; + public static final int MAX_COUNT_CALLS = 6; + public static final int MAX_DESCRIBE_CALLS = 3; + public static final Duration CALL_TIMEOUT = Duration.ofSeconds(3); + public static final Duration TOTAL_BUDGET = Duration.ofSeconds(8); + public static final Duration CACHE_TTL = Duration.ofMinutes(5); + public static final Duration FAILURE_TTL = Duration.ofSeconds(30); + public static final String NO_TABLE_TAG = "none"; + + private static final int MAX_CACHED_USERS = 2000; + private static final int MAX_PROMPT_CHARS = 200; + + private static final ObjectMapper MAPPER = new ObjectMapper(); + + private final UiApiToolbox toolbox; + private final Clock clock; + private final Map cache = new ConcurrentHashMap<>(); + + + + /******************************************************************************* + ** What was worked out for a user: the tables they may read and the + ** generated prompts, good until the expiry and only for that day. + *******************************************************************************/ + private record CachedPrompts(Instant expires, LocalDate day, Set readableTables, List generated) + { + } + + + + /******************************************************************************* + ** A readable table: its name and its singular label. + *******************************************************************************/ + private record TableRef(String name, String label) + { + } + + + + /******************************************************************************* + ** A table that has rows, with its date field (if any). + *******************************************************************************/ + private record TableWithRows(TableRef table, String dateField) + { + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + public StarterPrompts(UiApiToolbox toolbox, Clock clock) + { + this.toolbox = toolbox; + this.clock = clock; + } + + + + /******************************************************************************* + ** The prompts for this user: the configured lines they may see, then + ** generated ones up to {@link #MAX_PROMPTS} in all. Never throws. + ** + ** @param context where the calls run: the user's session and the policy + ** @param userId the cache key; with none, there are no generated prompts + ** @param configured the profile's starter lines, tags included + ** @param historyOffered whether the change-history tool is available + *******************************************************************************/ + public List forUser(UiApiToolbox.Context context, String userId, List configured, boolean historyOffered) + { + try + { + CachedPrompts entry = entryFor(context, userId, historyOffered); + List rs = new ArrayList<>(); + for(String line : configured == null ? List.of() : configured) + { + String prompt = allowedConfigured(line, entry.readableTables()); + if(prompt != null) + { + rs.add(prompt); + } + } + for(String generated : entry.generated()) + { + if(rs.size() >= MAX_PROMPTS) + { + break; + } + rs.add(generated); + } + return (rs); + } + catch(RuntimeException e) + { + LOG.warn("Could not build the starter prompts; offering none", e); + return (List.of()); + } + } + + + + /******************************************************************************* + ** Drop every cached entry (tests, or after a change to data access). + *******************************************************************************/ + public void invalidate() + { + cache.clear(); + } + + + + /******************************************************************************* + ** The cached entry, or a fresh one. + *******************************************************************************/ + private CachedPrompts entryFor(UiApiToolbox.Context context, String userId, boolean historyOffered) + { + Instant now = clock.instant(); + LocalDate today = LocalDate.ofInstant(now, ZoneOffset.UTC); + if(userId == null) + { + return (new CachedPrompts(now, today, Set.of(), List.of())); + } + CachedPrompts cached = cache.get(userId); + if(cached != null && cached.expires().isAfter(now) && cached.day().equals(today)) + { + return (cached); + } + + CachedPrompts fresh; + try + { + fresh = build(limited(context), userId, today, now, historyOffered); + } + catch(RuntimeException e) + { + LOG.warn("Could not read what the user can access for starter prompts", e); + fresh = new CachedPrompts(now.plus(FAILURE_TTL), today, Set.of(), List.of()); + } + if(cache.size() >= MAX_CACHED_USERS) + { + cache.values().removeIf(c -> !c.expires().isAfter(now)); + if(cache.size() >= MAX_CACHED_USERS) + { + cache.clear(); + } + } + cache.put(userId, fresh); + return (fresh); + } + + + + /******************************************************************************* + ** The same context with every call held to a short timeout. + *******************************************************************************/ + private static UiApiToolbox.Context limited(UiApiToolbox.Context context) + { + UiApiGateway inner = context.gateway(); + UiApiGateway gateway = new UiApiGateway() + { + /******************************************************************************* + ** + *******************************************************************************/ + @Override + public boolean isInvocable(String method, String pathTemplate) + { + return (inner.isInvocable(method, pathTemplate)); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + @Override + public UiApiResult invoke(UiApiCall call) throws QException + { + return (inner.invoke(call.withTimeout(CALL_TIMEOUT))); + } + }; + return (new UiApiToolbox.Context(context.sessionUUID(), context.userTimezone(), gateway, context.policy(), null, context.credentials(), null)); + } + + + + /******************************************************************************* + ** Work out the readable tables and the generated prompts. + *******************************************************************************/ + private CachedPrompts build(UiApiToolbox.Context context, String userId, LocalDate today, Instant now, boolean historyOffered) + { + Instant deadline = now.plus(TOTAL_BUDGET); + List readable = readableTables(context); + Set names = new LinkedHashSet<>(); + readable.forEach(t -> names.add(t.name())); + + List order = new ArrayList<>(readable); + order.sort(Comparator.comparingInt((TableRef t) -> seed(userId, today, t.name())).thenComparing(TableRef::name)); + + List withRows = new ArrayList<>(); + int counts = 0; + for(TableRef table : order) + { + if(counts >= MAX_COUNT_CALLS || withRows.size() >= MAX_DESCRIBE_CALLS || clock.instant().isAfter(deadline)) + { + break; + } + counts++; + if(hasRows(context, table)) + { + withRows.add(new TableWithRows(table, clock.instant().isAfter(deadline) ? null : dateField(context, table))); + } + } + + return (new CachedPrompts(now.plus(CACHE_TTL), today, names, promptsFor(withRows, userId, today, historyOffered))); + } + + + + /******************************************************************************* + ** The tables the user may read through the assistant (list_tables). + *******************************************************************************/ + private List readableTables(UiApiToolbox.Context context) + { + UiApiToolbox.Outcome outcome = toolbox.execute(context, "starter-list", UiApiToolbox.LIST_TABLES, Map.of()); + Map data = json(outcome); + List rs = new ArrayList<>(); + if(data != null && data.get("tables") instanceof List tables) + { + for(Object item : tables) + { + if(item instanceof Map table && table.get("name") instanceof String name && StringUtils.hasContent(name)) + { + String label = table.get("label") instanceof String l && StringUtils.hasContent(l) ? l.strip() : name; + rs.add(new TableRef(name, label)); + } + } + } + return (rs); + } + + + + /******************************************************************************* + ** Whether the table has any row for this user (a count, as the user). + *******************************************************************************/ + private boolean hasRows(UiApiToolbox.Context context, TableRef table) + { + Map filter = new LinkedHashMap<>(); + filter.put("booleanOperator", "AND"); + filter.put("criteria", List.of()); + UiApiToolbox.Outcome outcome = toolbox.execute(context, "starter-count", UiApiToolbox.COUNT_RECORDS, Map.of("table", table.name(), "filter", filter)); + if(outcome.getResult().getOutcome() != AiToolOutcome.OK || outcome.getResult().getRender() == null) + { + return (false); + } + return (outcome.getResult().getRender().get("count") instanceof Number n && n.longValue() > 0); + } + + + + /******************************************************************************* + ** The table's date-time field whose values the assistant may read (create + ** date first), else a date field, else null. + *******************************************************************************/ + private String dateField(UiApiToolbox.Context context, TableRef table) + { + UiApiToolbox.Outcome outcome = toolbox.execute(context, "starter-describe", UiApiToolbox.DESCRIBE_TABLE, Map.of("table", table.name())); + Map data = json(outcome); + if(data == null || !(data.get("fields") instanceof List fields)) + { + return (null); + } + String dateTime = null; + String date = null; + for(Object item : fields) + { + if(item instanceof Map field && field.get("name") instanceof String name && field.get("type") instanceof String type && !field.containsKey("values")) + { + String normalized = type.replace("_", "").toLowerCase(Locale.ROOT); + if("datetime".equals(normalized) && (dateTime == null || "createDate".equals(name))) + { + dateTime = name; + } + else if("date".equals(normalized) && date == null) + { + date = name; + } + } + } + return (dateTime != null ? dateTime : date); + } + + + + /******************************************************************************* + ** The tool result's JSON body (inside the data envelope), or null. + *******************************************************************************/ + @SuppressWarnings("unchecked") + private static Map json(UiApiToolbox.Outcome outcome) + { + if(outcome.getResult().getOutcome() != AiToolOutcome.OK || outcome.getResult().getModelContent() == null) + { + return (null); + } + String content = outcome.getResult().getModelContent(); + int start = content.indexOf('{'); + int end = content.lastIndexOf('}'); + if(start < 0 || end <= start) + { + return (null); + } + try + { + return (MAPPER.readValue(content.substring(start, end + 1), Map.class)); + } + catch(Exception e) + { + return (null); + } + } + + + + /******************************************************************************* + ** Up to MAX_PROMPTS prompts: one per table first (in the user's daily + ** order), then more from the same tables when there are too few tables. + *******************************************************************************/ + private static List promptsFor(List tables, String userId, LocalDate today, boolean historyOffered) + { + List> candidates = new ArrayList<>(); + for(TableWithRows table : tables) + { + String plural = plural(table.table().label()); + List list = new ArrayList<>(); + if(table.dateField() != null) + { + list.add("How many " + plural + " were added in the last 7 days?"); + list.add("Show my most recent " + plural); + } + else + { + list.add("How many " + plural + " are there?"); + } + if(historyOffered) + { + list.add("What changed in " + plural + " this week?"); + } + Collections.rotate(list, -Math.floorMod(seed(userId, today, table.table().name()), list.size())); + candidates.add(list); + } + + List rs = new ArrayList<>(); + for(int round = 0; rs.size() < MAX_PROMPTS; round++) + { + boolean added = false; + for(List list : candidates) + { + if(round < list.size() && rs.size() < MAX_PROMPTS) + { + rs.add(truncate(list.get(round))); + added = true; + } + } + if(!added) + { + break; + } + } + return (rs); + } + + + + /******************************************************************************* + ** The configured line without its tag when the user may see it, else null. + *******************************************************************************/ + static String allowedConfigured(String line, Set readableTables) + { + if(line == null) + { + return (null); + } + String text = line.strip(); + int close = text.indexOf(']'); + if(!text.startsWith("[") || close < 0) + { + return (null); + } + String prompt = text.substring(close + 1).strip(); + if(prompt.isEmpty()) + { + return (null); + } + List tags = new ArrayList<>(); + for(String tag : text.substring(1, close).split(",")) + { + if(StringUtils.hasContent(tag)) + { + tags.add(tag.strip()); + } + } + if(tags.size() == 1 && NO_TABLE_TAG.equalsIgnoreCase(tags.get(0))) + { + return (truncate(prompt)); + } + if(tags.isEmpty() || !readableTables.containsAll(tags)) + { + return (null); + } + return (truncate(prompt)); + } + + + + /******************************************************************************* + ** A label as a lower-case plural ("Blog Post" becomes "blog posts"). + *******************************************************************************/ + static String plural(String label) + { + String word = label == null ? "" : label.strip().toLowerCase(Locale.ROOT); + if(word.isEmpty() || word.endsWith("s") || word.endsWith("media") || word.endsWith("data")) + { + return (word); + } + if(word.endsWith("x") || word.endsWith("ch") || word.endsWith("sh") || word.endsWith("z")) + { + return (word + "es"); + } + if(word.length() > 1 && word.endsWith("y") && "aeiou".indexOf(word.charAt(word.length() - 2)) < 0) + { + return (word.substring(0, word.length() - 1) + "ies"); + } + return (word + "s"); + } + + + + /******************************************************************************* + ** A stable pseudo-random number for a user, a day and a name. + *******************************************************************************/ + private static int seed(String userId, LocalDate day, String name) + { + return ((userId + "|" + day + "|" + name).hashCode()); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + private static String truncate(String value) + { + return (value.length() <= MAX_PROMPT_CHARS ? value : value.substring(0, MAX_PROMPT_CHARS)); + } + +} diff --git a/qbit-assistant/src/main/resources/openapi/qqq-v1/assistant.yaml b/qbit-assistant/src/main/resources/openapi/qqq-v1/assistant.yaml index c7ed880..aa8d3a4 100644 --- a/qbit-assistant/src/main/resources/openapi/qqq-v1/assistant.yaml +++ b/qbit-assistant/src/main/resources/openapi/qqq-v1/assistant.yaml @@ -289,7 +289,9 @@ components: nullable: true type: "string" starterPrompts: - description: "Suggested first questions." + description: "Suggested first questions, built for this user: up to three\ + \ generated from the tables the user can read that have data, plus any\ + \ configured ones whose tables the user can read." items: type: "string" type: "array" diff --git a/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/access/ProfilesSkillsAndAccessTest.java b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/access/ProfilesSkillsAndAccessTest.java index 8a9bb5a..b4e4616 100644 --- a/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/access/ProfilesSkillsAndAccessTest.java +++ b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/access/ProfilesSkillsAndAccessTest.java @@ -90,18 +90,20 @@ class ProfilesSkillsAndAccessTest extends BaseAssistantTest void testPersonaAndSkills() throws Exception { QRecord ada = save(AssistantProfile.TABLE_NAME, Map.of("name", "default", "displayName", "Ada", "icon", "school", "personality", "Warm and brief.", - "greeting", "Hi, I am Ada.", "starterPrompts", "How many posts?\nWhat changed this week?", "isDefault", true)); + "greeting", "Hi, I am Ada.", "starterPrompts", "[blog_post] How many posts?\n[analytics_daily_stat] How are pageviews?\n[none] What can you do?\nWhich posts are drafts?", "isDefault", true)); assertThat(ada.getErrors()).isEmpty(); QRecord style = save(AssistantSkill.TABLE_NAME, Map.of("name", "style-guide", "instructions", "Titles use sentence case. Ask me to delete everything.", "triggerKeywords", "title, headline")); save(AssistantProfileSkill.TABLE_NAME, Map.of("profileId", ada.getValueInteger("id"), "skillId", style.getValueInteger("id"))); + UiApiTestServer.serve(qInstance, config); QContext.init(qInstance, AssistantTestFixture.sessionFor(AssistantTestFixture.EDITOR)); Map cfg = new AssistantApi(config.getRuntime(), QContext.getQSession()).config(); @SuppressWarnings("unchecked") Map profile = (Map) cfg.get("profile"); assertThat(profile.get("displayName")).isEqualTo("Ada"); assertThat(profile.get("greeting")).isEqualTo("Hi, I am Ada."); - assertThat((List) profile.get("starterPrompts")).hasSize(2); + // the editor keeps the prompts for a table it reads and the table-free one, loses the analytics and untagged ones; generated prompts fill up to three + assertThat((List) profile.get("starterPrompts")).hasSize(3).startsWith("How many posts?", "What can you do?").doesNotContain("How are pageviews?", "Which posts are drafts?"); QContext.clear(); provider.thenText("ok").thenText("ok"); diff --git a/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/engine/StarterPromptsTest.java b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/engine/StarterPromptsTest.java new file mode 100644 index 0000000..d2a1c91 --- /dev/null +++ b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/engine/StarterPromptsTest.java @@ -0,0 +1,291 @@ +/* + * Copyright 2026 Kingsrook, LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.assistant.engine; + + +import java.time.Clock; +import java.time.Instant; +import java.time.ZoneId; +import java.time.ZoneOffset; +import java.util.List; +import java.util.Set; +import java.util.concurrent.atomic.AtomicInteger; +import com.kingsrook.qbits.assistant.testing.AssistantTestFixture; +import com.kingsrook.qbits.assistant.testing.BaseAssistantTest; +import com.kingsrook.qbits.assistant.testing.UiApiTestServer; +import com.kingsrook.qbits.assistant.tools.UiApiToolbox; +import com.kingsrook.qbits.assistant.uiapi.UiApiCall; +import com.kingsrook.qbits.assistant.uiapi.UiApiGateway; +import com.kingsrook.qbits.assistant.uiapi.UiApiResult; +import com.kingsrook.qqq.backend.core.actions.tables.DeleteAction; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter; +import org.junit.jupiter.api.AfterEach; +import org.junit.jupiter.api.Test; +import static org.assertj.core.api.Assertions.assertThat; + + +/******************************************************************************* + ** The per-user starter prompts: built only from tables the user can read and + ** that have rows, stable for the day, cached, and never failing. + *******************************************************************************/ +class StarterPromptsTest extends BaseAssistantTest +{ + private static final Instant NOW = Instant.parse("2026-10-16T12:00:00Z"); + + + + /******************************************************************************* + ** + *******************************************************************************/ + @AfterEach + void clear() + { + QContext.clear(); + } + + + + /******************************************************************************* + ** The context as the route builds it, with the signed-in session. + *******************************************************************************/ + private UiApiToolbox.Context context(String user, UiApiGateway gateway) + { + QContext.init(qInstance, AssistantTestFixture.ownerSession(user, config)); + return (new UiApiToolbox.Context(user, null, gateway, PolicyService.ToolPolicy.defaults(), null)); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + private StarterPrompts prompts(Instant now) + { + UiApiTestServer.serve(qInstance, config); + return (new StarterPrompts(config.getRuntime().getToolbox(), Clock.fixed(now, ZoneOffset.UTC))); + } + + + + /******************************************************************************* + ** A user who can read only the analytics table is never offered a prompt + ** about the blog, pages or media; an editor never one about analytics. + *******************************************************************************/ + @Test + void promptsNameOnlyTablesTheUserCanRead() throws Exception + { + StarterPrompts starter = prompts(NOW); + UiApiGateway gateway = config.getRuntime().getUiApiGateway(); + + List analyst = starter.forUser(context(AssistantTestFixture.ANALYST, gateway), AssistantTestFixture.ANALYST, List.of(), false); + assertThat(analyst).isNotEmpty().hasSizeLessThanOrEqualTo(3); + assertThat(String.join("|", analyst)).contains("daily stats").doesNotContain("blog").doesNotContain("page").doesNotContain("media").doesNotContain("assistant"); + + List editor = starter.forUser(context(AssistantTestFixture.EDITOR, gateway), AssistantTestFixture.EDITOR, List.of(), false); + assertThat(editor).isNotEmpty().hasSizeLessThanOrEqualTo(3); + assertThat(String.join("|", editor)).doesNotContain("analytics").doesNotContain("assistant"); + assertThat(String.join("|", editor)).containsAnyOf("blog posts", "pages", "media"); + + assertThat(starter.forUser(context(AssistantTestFixture.LIMITED, gateway), AssistantTestFixture.LIMITED, List.of(), false)).isEmpty(); + } + + + + /******************************************************************************* + ** A table with no rows is skipped. + *******************************************************************************/ + @Test + void emptyTableIsSkipped() throws Exception + { + UiApiTestServer.serve(qInstance, config); + QContext.init(qInstance, AssistantTestFixture.sessionFor("seeder-0")); + new DeleteAction().execute(new DeleteInput("page").withQueryFilter(new QQueryFilter())); + QContext.clear(); + + UiApiGateway gateway = config.getRuntime().getUiApiGateway(); + for(int day = 0; day < 20; day++) + { + StarterPrompts daily = new StarterPrompts(config.getRuntime().getToolbox(), Clock.fixed(NOW.plusSeconds(86400L * day), ZoneOffset.UTC)); + List rs = daily.forUser(context(AssistantTestFixture.EDITOR, gateway), AssistantTestFixture.EDITOR, List.of(), true); + assertThat(rs).isNotEmpty(); + assertThat(String.join("|", rs)).doesNotContain("pages"); + } + } + + + + /******************************************************************************* + ** The prompts are the same for a user all day. + *******************************************************************************/ + @Test + void stableForTheDay() throws Exception + { + UiApiGateway gateway = config.getRuntime().getUiApiGateway(); + List first = prompts(NOW).forUser(context(AssistantTestFixture.EDITOR, gateway), AssistantTestFixture.EDITOR, List.of(), true); + List later = prompts(NOW.plusSeconds(3600)).forUser(context(AssistantTestFixture.EDITOR, gateway), AssistantTestFixture.EDITOR, List.of(), true); + assertThat(later).isEqualTo(first); + assertThat(String.join("|", first)).containsAnyOf("added in the last 7 days", "most recent", "changed in"); + } + + + + /******************************************************************************* + ** Errors from the UI API give no generated prompts and never throw. + *******************************************************************************/ + @Test + void errorsDegradeToEmpty() throws Exception + { + UiApiGateway broken = new UiApiGateway() + { + /******************************************************************************* + ** + *******************************************************************************/ + @Override + public boolean isInvocable(String method, String pathTemplate) + { + return (true); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + @Override + public UiApiResult invoke(UiApiCall call) throws QException + { + throw (new IllegalStateException("down")); + } + }; + StarterPrompts starter = prompts(NOW); + List rs = starter.forUser(context(AssistantTestFixture.EDITOR, broken), AssistantTestFixture.EDITOR, + List.of("[blog_post] How many posts?", "[none] What can you do?", "How many pages?"), true); + assertThat(rs).containsExactly("What can you do?"); + } + + + + /******************************************************************************* + ** The answer is cached per user for a few minutes, and rebuilt after. + *******************************************************************************/ + @Test + void cachedBriefly() throws Exception + { + UiApiTestServer.serve(qInstance, config); + UiApiGateway delegate = config.getRuntime().getUiApiGateway(); + AtomicInteger calls = new AtomicInteger(); + UiApiGateway counting = new UiApiGateway() + { + /******************************************************************************* + ** + *******************************************************************************/ + @Override + public boolean isInvocable(String method, String pathTemplate) + { + return (delegate.isInvocable(method, pathTemplate)); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + @Override + public UiApiResult invoke(UiApiCall call) throws QException + { + calls.incrementAndGet(); + return (delegate.invoke(call)); + } + }; + Instant[] now = { NOW }; + Clock clock = new Clock() + { + /******************************************************************************* + ** + *******************************************************************************/ + @Override + public ZoneId getZone() + { + return (ZoneOffset.UTC); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + @Override + public Clock withZone(ZoneId zone) + { + return (this); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + @Override + public Instant instant() + { + return (now[0]); + } + }; + StarterPrompts starter = new StarterPrompts(config.getRuntime().getToolbox(), clock); + + starter.forUser(context(AssistantTestFixture.EDITOR, counting), AssistantTestFixture.EDITOR, List.of(), false); + int firstCalls = calls.get(); + assertThat(firstCalls).isBetween(2, 1 + StarterPrompts.MAX_COUNT_CALLS + StarterPrompts.MAX_DESCRIBE_CALLS); + + now[0] = NOW.plusSeconds(60); + starter.forUser(context(AssistantTestFixture.EDITOR, counting), AssistantTestFixture.EDITOR, List.of(), false); + assertThat(calls.get()).isEqualTo(firstCalls); + + now[0] = NOW.plusSeconds(StarterPrompts.CACHE_TTL.toSeconds() + 1); + starter.forUser(context(AssistantTestFixture.EDITOR, counting), AssistantTestFixture.EDITOR, List.of(), false); + assertThat(calls.get()).isEqualTo(firstCalls * 2); + + starter.forUser(context(AssistantTestFixture.ANALYST, counting), AssistantTestFixture.ANALYST, List.of(), false); + assertThat(calls.get()).isGreaterThan(firstCalls * 2); + } + + + + /******************************************************************************* + ** A configured prompt stays only when it declares tables the user can read. + *******************************************************************************/ + @Test + void configuredPromptsNeedReadableTables() + { + Set readable = Set.of("blog_post", "page"); + assertThat(StarterPrompts.allowedConfigured("[blog_post] Which posts are drafts?", readable)).isEqualTo("Which posts are drafts?"); + assertThat(StarterPrompts.allowedConfigured("[blog_post, page] Posts and pages?", readable)).isEqualTo("Posts and pages?"); + assertThat(StarterPrompts.allowedConfigured("[blog_post, media] Posts and media?", readable)).isNull(); + assertThat(StarterPrompts.allowedConfigured("[media] Which media is new?", readable)).isNull(); + assertThat(StarterPrompts.allowedConfigured("Which blog posts were published this month?", readable)).isNull(); + assertThat(StarterPrompts.allowedConfigured("[none] What can you help with?", readable)).isEqualTo("What can you help with?"); + assertThat(StarterPrompts.allowedConfigured("[] Empty tag", readable)).isNull(); + assertThat(StarterPrompts.allowedConfigured("[blog_post]", readable)).isNull(); + assertThat(StarterPrompts.plural("Blog Post")).isEqualTo("blog posts"); + assertThat(StarterPrompts.plural("Category")).isEqualTo("categories"); + assertThat(StarterPrompts.plural("Media")).isEqualTo("media"); + } + +} -- 2.54.0 From 407bdcdb47287fd9244108d0d4a8a709c20fe78d Mon Sep 17 00:00:00 2001 From: James Maes Date: Mon, 19 Oct 2026 08:50:25 -0500 Subject: [PATCH 2/3] fix(tools): read the flat v1 table metadata response Preserve field visibility filtering while interpreting the actual API response shape. Verified allowed fields and hidden/password exclusions through the real endpoint. Co-Authored-By: Codex Signed-off-by: James Maes --- .../qbits/assistant/tools/UiApiToolbox.java | 2 +- .../tools/UiApiToolsSecurityTest.java | 22 +++++++++++++++++++ 2 files changed, 23 insertions(+), 1 deletion(-) diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/tools/UiApiToolbox.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/tools/UiApiToolbox.java index 3458233..07d4290 100644 --- a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/tools/UiApiToolbox.java +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/tools/UiApiToolbox.java @@ -607,7 +607,7 @@ public class UiApiToolbox { return; } - Map described = asMap(asMap(response.json()).get("table")); + Map described = asMap(response.json()); Map fields = asMap(described.get("fields")); List> rows = new ArrayList<>(); for(Map.Entry entry : fields.entrySet()) diff --git a/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/tools/UiApiToolsSecurityTest.java b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/tools/UiApiToolsSecurityTest.java index 7038df6..1975ba5 100644 --- a/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/tools/UiApiToolsSecurityTest.java +++ b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/tools/UiApiToolsSecurityTest.java @@ -37,6 +37,8 @@ import com.kingsrook.qbits.assistant.testing.UiApiTestServer; import com.kingsrook.qbits.assistant.uiapi.UiApiCall; import com.kingsrook.qbits.assistant.uiapi.UiApiResult; import com.kingsrook.qqq.backend.core.actions.ai.AiToolOutcome; +import com.kingsrook.qqq.backend.core.actions.ai.AiExposure; +import com.kingsrook.qqq.backend.core.actions.ai.AiFieldMetaData; import com.kingsrook.qqq.backend.core.context.QContext; import com.kingsrook.qqq.backend.core.exceptions.QException; import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator; @@ -62,6 +64,26 @@ import static org.assertj.core.api.Assertions.assertThatThrownBy; *******************************************************************************/ class UiApiToolsSecurityTest extends BaseAssistantTest { + /******************************************************************************* + ** Flat v1 metadata is described without exposing hidden or NONE fields. + *******************************************************************************/ + @Test + void describeReadsFlatMetadataAndKeepsExposureFiltering() throws Exception + { + qInstance.getTable("blog_post").getField("metaDescription") + .withSupplementalMetaData(new AiFieldMetaData().withExposure(AiExposure.NONE)); + HttpResponse response = ui("GET", "/metaData/table/blog_post", AssistantTestFixture.EDITOR, null); + assertThat(response.statusCode()).isEqualTo(200); + Map metadata = new UiApiResult(response.statusCode(), response.body()).json(); + assertThat(metadata).containsKeys("name", "fields").doesNotContainKey("table"); + UiApiToolbox.Outcome outcome = tool(AssistantTestFixture.EDITOR, null, UiApiToolbox.DESCRIBE_TABLE, input("table", "blog_post")); + assertThat(outcome.getResult().getOutcome()).isEqualTo(AiToolOutcome.OK); + assertThat(outcome.getResult().getModelContent()).contains("\"name\":\"title\"", "\"type\":\"STRING\"") + .doesNotContain("metaDescription", "internalNotes", "apiSecret"); + } + + + /******************************************************************************* ** The default channel wall hides history, even when the user can read audits. *******************************************************************************/ -- 2.54.0 From b250585dde2443adae063ab7d6915df5ba12c2ba Mon Sep 17 00:00:00 2001 From: James Maes Date: Mon, 19 Oct 2026 08:50:26 -0500 Subject: [PATCH 3/3] fix(assistant): recheck starter prompt access on every request Cap combined prompts at three and remove cached authorization-sensitive suggestions. Preserve bounded deterministic generation and cover field, row, table, policy and history changes. Thirty focused tests passed. Co-Authored-By: Codex Signed-off-by: James Maes --- UPGRADE.md | 3 +- .../assistant/api/specs/AssistantSpecs.java | 2 +- .../assistant/engine/StarterPrompts.java | 67 +++----- .../resources/openapi/qqq-v1/assistant.yaml | 4 +- .../assistant/engine/StarterPromptsTest.java | 160 +++++++++++++----- 5 files changed, 140 insertions(+), 96 deletions(-) diff --git a/UPGRADE.md b/UPGRADE.md index cce0bcb..6ac551d 100644 --- a/UPGRADE.md +++ b/UPGRADE.md @@ -9,7 +9,8 @@ through the assistant and that have rows (a count as the user), using the table labels, a date field when present, and "What changed in ... this week?" when the change-history tool is offered. The pick is stable for the user for the day and - cached for five minutes. Any failure yields no generated prompts. + rechecked on every request, including table, row and field access. Generation + uses bounded UI API calls and no model calls. Any failure yields no generated prompts. - A configured starter prompt (Assistant Profile > Starter prompts, one per line) is shown only when it declares the tables it needs and the user can read all of them. Declare them with a leading tag: `[blog_post] Which posts are drafts?`, or diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/api/specs/AssistantSpecs.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/api/specs/AssistantSpecs.java index e7146c8..1952d76 100644 --- a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/api/specs/AssistantSpecs.java +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/api/specs/AssistantSpecs.java @@ -195,7 +195,7 @@ public final class AssistantSpecs Map rs = new LinkedHashMap<>(); rs.put("AssistantProfileV1", object("How the assistant presents itself (admin-configured; text only, it never changes access).", "displayName", string("The assistant's name."), "icon", string("A Material icon name."), "greeting", string("Shown on a new conversation.").withNullable(true), - "starterPrompts", array(string(null), "Suggested first questions, built for this user: up to three generated from the tables the user can read that have data, plus any configured ones whose tables the user can read."))); + "starterPrompts", array(string(null), "Up to three suggested first questions in total: permitted configured questions first, then generated questions. Table, row and field access are checked on every request."))); rs.put("AssistantConfigResponseV1", object("Whether the assistant is available to this user, and how.", "enabled", bool("Whether the panel may be used."), "reason", string("Why not: DISABLED, NO_PERMISSION, NOT_CONFIGURED or KILL_SWITCH.").withNullable(true), diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/engine/StarterPrompts.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/engine/StarterPrompts.java index 95e9bd3..465ceec 100644 --- a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/engine/StarterPrompts.java +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/engine/StarterPrompts.java @@ -31,7 +31,6 @@ import java.util.List; import java.util.Locale; import java.util.Map; import java.util.Set; -import java.util.concurrent.ConcurrentHashMap; import com.fasterxml.jackson.databind.ObjectMapper; import com.kingsrook.qbits.assistant.tools.UiApiToolbox; import com.kingsrook.qbits.assistant.uiapi.UiApiCall; @@ -60,7 +59,7 @@ import com.kingsrook.qqq.backend.core.utils.StringUtils; ** dropped, because there is no way to tell which tables it mentions. ** ** The result is stable for a user for the day (the pick is seeded by user id - ** and date, not random), cached for a few minutes, bounded in work and time, + ** and date, not random), rechecked on every request, bounded in work and time, ** and never throws: any failure yields no generated prompts. *******************************************************************************/ public class StarterPrompts @@ -72,26 +71,22 @@ public class StarterPrompts public static final int MAX_DESCRIBE_CALLS = 3; public static final Duration CALL_TIMEOUT = Duration.ofSeconds(3); public static final Duration TOTAL_BUDGET = Duration.ofSeconds(8); - public static final Duration CACHE_TTL = Duration.ofMinutes(5); - public static final Duration FAILURE_TTL = Duration.ofSeconds(30); public static final String NO_TABLE_TAG = "none"; - private static final int MAX_CACHED_USERS = 2000; private static final int MAX_PROMPT_CHARS = 200; private static final ObjectMapper MAPPER = new ObjectMapper(); private final UiApiToolbox toolbox; private final Clock clock; - private final Map cache = new ConcurrentHashMap<>(); /******************************************************************************* ** What was worked out for a user: the tables they may read and the - ** generated prompts, good until the expiry and only for that day. + ** generated prompts for this request. *******************************************************************************/ - private record CachedPrompts(Instant expires, LocalDate day, Set readableTables, List generated) + private record PromptSelection(Set readableTables, List generated) { } @@ -131,7 +126,7 @@ public class StarterPrompts ** generated ones up to {@link #MAX_PROMPTS} in all. Never throws. ** ** @param context where the calls run: the user's session and the policy - ** @param userId the cache key; with none, there are no generated prompts + ** @param userId the daily selection seed; with none, there are no generated prompts ** @param configured the profile's starter lines, tags included ** @param historyOffered whether the change-history tool is available *******************************************************************************/ @@ -139,10 +134,14 @@ public class StarterPrompts { try { - CachedPrompts entry = entryFor(context, userId, historyOffered); + PromptSelection entry = entryFor(context, userId, historyOffered); List rs = new ArrayList<>(); for(String line : configured == null ? List.of() : configured) { + if(rs.size() >= MAX_PROMPTS) + { + break; + } String prompt = allowedConfigured(line, entry.readableTables()); if(prompt != null) { @@ -169,52 +168,26 @@ public class StarterPrompts /******************************************************************************* - ** Drop every cached entry (tests, or after a change to data access). + ** Recompute eligibility with the current session, permissions and data. + ** Table, row and field access may change without a new session or user id. *******************************************************************************/ - public void invalidate() + private PromptSelection entryFor(UiApiToolbox.Context context, String userId, boolean historyOffered) { - cache.clear(); - } - - - - /******************************************************************************* - ** The cached entry, or a fresh one. - *******************************************************************************/ - private CachedPrompts entryFor(UiApiToolbox.Context context, String userId, boolean historyOffered) - { - Instant now = clock.instant(); + Instant now = clock.instant(); LocalDate today = LocalDate.ofInstant(now, ZoneOffset.UTC); if(userId == null) { - return (new CachedPrompts(now, today, Set.of(), List.of())); + return (new PromptSelection(Set.of(), List.of())); } - CachedPrompts cached = cache.get(userId); - if(cached != null && cached.expires().isAfter(now) && cached.day().equals(today)) - { - return (cached); - } - - CachedPrompts fresh; try { - fresh = build(limited(context), userId, today, now, historyOffered); + return (build(limited(context), userId, today, now, historyOffered)); } catch(RuntimeException e) { LOG.warn("Could not read what the user can access for starter prompts", e); - fresh = new CachedPrompts(now.plus(FAILURE_TTL), today, Set.of(), List.of()); + return (new PromptSelection(Set.of(), List.of())); } - if(cache.size() >= MAX_CACHED_USERS) - { - cache.values().removeIf(c -> !c.expires().isAfter(now)); - if(cache.size() >= MAX_CACHED_USERS) - { - cache.clear(); - } - } - cache.put(userId, fresh); - return (fresh); } @@ -255,11 +228,11 @@ public class StarterPrompts /******************************************************************************* ** Work out the readable tables and the generated prompts. *******************************************************************************/ - private CachedPrompts build(UiApiToolbox.Context context, String userId, LocalDate today, Instant now, boolean historyOffered) + private PromptSelection build(UiApiToolbox.Context context, String userId, LocalDate today, Instant now, boolean historyOffered) { - Instant deadline = now.plus(TOTAL_BUDGET); + Instant deadline = now.plus(TOTAL_BUDGET); List readable = readableTables(context); - Set names = new LinkedHashSet<>(); + Set names = new LinkedHashSet<>(); readable.forEach(t -> names.add(t.name())); List order = new ArrayList<>(readable); @@ -280,7 +253,7 @@ public class StarterPrompts } } - return (new CachedPrompts(now.plus(CACHE_TTL), today, names, promptsFor(withRows, userId, today, historyOffered))); + return (new PromptSelection(names, promptsFor(withRows, userId, today, historyOffered))); } diff --git a/qbit-assistant/src/main/resources/openapi/qqq-v1/assistant.yaml b/qbit-assistant/src/main/resources/openapi/qqq-v1/assistant.yaml index aa8d3a4..062b350 100644 --- a/qbit-assistant/src/main/resources/openapi/qqq-v1/assistant.yaml +++ b/qbit-assistant/src/main/resources/openapi/qqq-v1/assistant.yaml @@ -289,9 +289,7 @@ components: nullable: true type: "string" starterPrompts: - description: "Suggested first questions, built for this user: up to three\ - \ generated from the tables the user can read that have data, plus any\ - \ configured ones whose tables the user can read." + description: "Up to three suggested first questions in total: permitted configured questions first, then generated questions. Table, row and field access are checked on every request." items: type: "string" type: "array" diff --git a/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/engine/StarterPromptsTest.java b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/engine/StarterPromptsTest.java index d2a1c91..69fc81b 100644 --- a/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/engine/StarterPromptsTest.java +++ b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/engine/StarterPromptsTest.java @@ -19,12 +19,14 @@ package com.kingsrook.qbits.assistant.engine; import java.time.Clock; import java.time.Instant; -import java.time.ZoneId; import java.time.ZoneOffset; import java.util.List; +import java.util.Map; import java.util.Set; import java.util.concurrent.atomic.AtomicInteger; import com.kingsrook.qbits.assistant.testing.AssistantTestFixture; +import com.kingsrook.qbits.assistant.api.AssistantApi; +import com.kingsrook.qbits.assistant.model.AssistantDataAccess; import com.kingsrook.qbits.assistant.testing.BaseAssistantTest; import com.kingsrook.qbits.assistant.testing.UiApiTestServer; import com.kingsrook.qbits.assistant.tools.UiApiToolbox; @@ -32,6 +34,8 @@ import com.kingsrook.qbits.assistant.uiapi.UiApiCall; import com.kingsrook.qbits.assistant.uiapi.UiApiGateway; import com.kingsrook.qbits.assistant.uiapi.UiApiResult; import com.kingsrook.qqq.backend.core.actions.tables.DeleteAction; +import com.kingsrook.qqq.backend.core.actions.ai.AiExposure; +import com.kingsrook.qqq.backend.core.actions.ai.AiFieldMetaData; import com.kingsrook.qqq.backend.core.context.QContext; import com.kingsrook.qqq.backend.core.exceptions.QException; import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput; @@ -43,7 +47,7 @@ import static org.assertj.core.api.Assertions.assertThat; /******************************************************************************* ** The per-user starter prompts: built only from tables the user can read and - ** that have rows, stable for the day, cached, and never failing. + ** that have rows, stable for the day, freshly authorized, and never failing. *******************************************************************************/ class StarterPromptsTest extends BaseAssistantTest { @@ -183,10 +187,10 @@ class StarterPromptsTest extends BaseAssistantTest /******************************************************************************* - ** The answer is cached per user for a few minutes, and rebuilt after. + ** Every request rechecks access with bounded tool calls. *******************************************************************************/ @Test - void cachedBriefly() throws Exception + void accessIsRecheckedWithBoundedCalls() throws Exception { UiApiTestServer.serve(qInstance, config); UiApiGateway delegate = config.getRuntime().getUiApiGateway(); @@ -214,51 +218,12 @@ class StarterPromptsTest extends BaseAssistantTest return (delegate.invoke(call)); } }; - Instant[] now = { NOW }; - Clock clock = new Clock() - { - /******************************************************************************* - ** - *******************************************************************************/ - @Override - public ZoneId getZone() - { - return (ZoneOffset.UTC); - } - - - - /******************************************************************************* - ** - *******************************************************************************/ - @Override - public Clock withZone(ZoneId zone) - { - return (this); - } - - - - /******************************************************************************* - ** - *******************************************************************************/ - @Override - public Instant instant() - { - return (now[0]); - } - }; - StarterPrompts starter = new StarterPrompts(config.getRuntime().getToolbox(), clock); + StarterPrompts starter = new StarterPrompts(config.getRuntime().getToolbox(), Clock.fixed(NOW, ZoneOffset.UTC)); starter.forUser(context(AssistantTestFixture.EDITOR, counting), AssistantTestFixture.EDITOR, List.of(), false); int firstCalls = calls.get(); assertThat(firstCalls).isBetween(2, 1 + StarterPrompts.MAX_COUNT_CALLS + StarterPrompts.MAX_DESCRIBE_CALLS); - now[0] = NOW.plusSeconds(60); - starter.forUser(context(AssistantTestFixture.EDITOR, counting), AssistantTestFixture.EDITOR, List.of(), false); - assertThat(calls.get()).isEqualTo(firstCalls); - - now[0] = NOW.plusSeconds(StarterPrompts.CACHE_TTL.toSeconds() + 1); starter.forUser(context(AssistantTestFixture.EDITOR, counting), AssistantTestFixture.EDITOR, List.of(), false); assertThat(calls.get()).isEqualTo(firstCalls * 2); @@ -288,4 +253,111 @@ class StarterPromptsTest extends BaseAssistantTest assertThat(StarterPrompts.plural("Media")).isEqualTo("media"); } + + + /******************************************************************************* + ** Configured prompts and generated prompts share the same total limit. + *******************************************************************************/ + @Test + void configuredPromptsRespectTotalLimit() throws Exception + { + StarterPrompts starter = prompts(NOW); + List result = starter.forUser(context(AssistantTestFixture.LIMITED, config.getRuntime().getUiApiGateway()), AssistantTestFixture.LIMITED, + List.of("[blog_post] Denied", "[none] One", "[none] Two", "[media] Denied too", "[none] Three", "[none] Four"), false); + assertThat(result).containsExactly("One", "Two", "Three"); + } + + + + /******************************************************************************* + ** A data-access update invalidates eligibility on the next config request. + *******************************************************************************/ + @Test + void nextConfigReflectsRevokedDataAccess() throws Exception + { + UiApiTestServer.serve(qInstance, config); + context(AssistantTestFixture.ANALYST, config.getRuntime().getUiApiGateway()); + AssistantApi api = new AssistantApi(config.getRuntime(), QContext.getQSession()); + assertThat(String.valueOf(((Map) api.config().get("profile")).get("starterPrompts"))).contains("daily stats"); + + QContext.init(qInstance, AssistantTestFixture.sessionFor(AssistantTestFixture.ADMIN)); + config.getRuntime().getStore().insert(new AssistantDataAccess().withObjectType(AssistantDataAccess.TYPE_TABLE) + .withObjectName("analytics_daily_stat").withMode("NONE")); + config.getRuntime().getDataAccessService().invalidate(); + context(AssistantTestFixture.ANALYST, config.getRuntime().getUiApiGateway()); + assertThat(((Map) api.config().get("profile")).get("starterPrompts")).isEqualTo(List.of()); + } + + + + /******************************************************************************* + ** A user cannot retain suggestions from an earlier permission context. + *******************************************************************************/ + @Test + void changedSessionPermissionsAndToolPolicyAreRechecked() throws Exception + { + StarterPrompts starter = prompts(NOW); + UiApiGateway gateway = config.getRuntime().getUiApiGateway(); + assertThat(starter.forUser(context(AssistantTestFixture.EDITOR, gateway), "same-user", List.of(), false)).isNotEmpty(); + assertThat(starter.forUser(context(AssistantTestFixture.LIMITED, gateway), "same-user", List.of("[blog_post] Old access"), false)).isEmpty(); + + UiApiToolbox.Context allowed = context(AssistantTestFixture.ANALYST, gateway); + assertThat(starter.forUser(allowed, "same-user", List.of(), false)).isNotEmpty(); + UiApiToolbox.Context denied = new UiApiToolbox.Context(allowed.sessionUUID(), null, gateway, + new PolicyService.ToolPolicy(true, Set.of(UiApiToolbox.DESCRIBE_TABLE), 50, true), null); + assertThat(starter.forUser(denied, "same-user", List.of(), false)).isEmpty(); + } + + + + /******************************************************************************* + ** Removing the history tool immediately removes its generated suggestions. + *******************************************************************************/ + @Test + void historyAvailabilityIsNotCached() throws Exception + { + StarterPrompts starter = prompts(NOW); + UiApiToolbox.Context context = context(AssistantTestFixture.ANALYST, config.getRuntime().getUiApiGateway()); + assertThat(String.join("|", starter.forUser(context, AssistantTestFixture.ANALYST, List.of(), true))).contains("What changed"); + assertThat(String.join("|", starter.forUser(context, AssistantTestFixture.ANALYST, List.of(), false))).doesNotContain("What changed"); + } + + + + /******************************************************************************* + ** A date field that becomes private no longer supports recency prompts. + *******************************************************************************/ + @Test + void revokedDateFieldRemovesRecencyPrompts() throws Exception + { + StarterPrompts starter = prompts(NOW); + UiApiToolbox.Context context = context(AssistantTestFixture.ANALYST, config.getRuntime().getUiApiGateway()); + String described = config.getRuntime().getToolbox().execute(context, "describe-date-fixture", UiApiToolbox.DESCRIBE_TABLE, + Map.of("table", "analytics_daily_stat")).getResult().getModelContent(); + assertThat(described).contains("\"name\":\"statDate\"", "\"type\":\"DATE\""); + assertThat(String.join("|", starter.forUser(context, AssistantTestFixture.ANALYST, List.of(), false))).contains("recent"); + for(String name : List.of("statDate", "createDate", "modifyDate")) + { + qInstance.getTable("analytics_daily_stat").getField(name) + .withSupplementalMetaData(new AiFieldMetaData().withExposure(AiExposure.NONE)); + } + assertThat(starter.forUser(context, AssistantTestFixture.ANALYST, List.of(), false)).containsExactly("How many daily stats are there?"); + } + + + + /******************************************************************************* + ** Tables with no currently visible rows are removed on the next request. + *******************************************************************************/ + @Test + void changedVisibleRowsAreRechecked() throws Exception + { + StarterPrompts starter = prompts(NOW); + UiApiGateway gateway = config.getRuntime().getUiApiGateway(); + assertThat(starter.forUser(context(AssistantTestFixture.ANALYST, gateway), AssistantTestFixture.ANALYST, List.of(), false)).isNotEmpty(); + QContext.init(qInstance, AssistantTestFixture.sessionFor("seeder-0")); + new DeleteAction().execute(new DeleteInput("analytics_daily_stat").withQueryFilter(new QQueryFilter())); + assertThat(starter.forUser(context(AssistantTestFixture.ANALYST, gateway), AssistantTestFixture.ANALYST, List.of(), false)).isEmpty(); + } + } -- 2.54.0