diff --git a/CHANGELOG.md b/CHANGELOG.md index 3bd94ca..7c27843 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,19 @@ # Changelog +## Unreleased - P-g + +- Add admin Scheduled tasks, owner-locked run history, Quartz cron/timezones, + and a 60-second QQQ scheduled process using the host's scheduler context. +- Log in through the loopback UI API with a separately allow-listed credential + environment reference; verify the designated user before starting a normal turn. +- Add Review/Result Approve write mode, Revoke write mode and Run now actions. + Approval binds the task definition; scheduled writes always await confirmation. +- Attribute scheduled UI API requests and tool logs, apply per-run STOP caps, + and send optional Alerts notifications to the designated user. HIPAA summaries + contain only `see conversation`. +- Add unique schedule-slot claims, fixed-clock fixture integration tests and + additive Postgres DDL. No API routes or provider modules change. + ## Unreleased - P-f - Add OpenAI-compatible embeddings (including Azure), explicit CHAT/EMBEDDING diff --git a/CLAUDE.md b/CLAUDE.md index 8945998..9f92bfc 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -51,6 +51,7 @@ Always use the wrapper: Maven 3.10 cannot read the parent POM. The committed def ## Invariants (do not break) - **Sessions:** + - Scheduled runs log in as the designated user through `POST /qqq/v1/manageSession`; resolve only its returned UUID, verify the user id, and never construct or elevate a session. The QQQ-supplied scheduler context touches only assistant bookkeeping. - Never use `QSystemUserSession` or construct a `QSession` in main code (ArchUnit). - Every route uses `ExecutorSessionUtils.setupSession` (the user's own session). - `AssistantSecurity` only ever adds the user's OWN id key, or the all-access key for `assistant.admin` holders. @@ -67,6 +68,8 @@ Always use the wrapper: Maven 3.10 cannot read the parent POM. The committed def - Errors (DENIED, NOT_FOUND, INVALID_INPUT, ERROR) are `is_error` results. - Unknown tools and over-limit calls also get error results. - **Keys:** never stored in a table. `assistant_provider.apiKeyEnvName` holds only an allowed environment variable NAME; `apiKeyStatus` is computed on read (last 4 at most) and stripped on save. + - Scheduled task `credentialEnvName` is a separately allow-listed environment variable NAME. Tokens never enter rows, model context or logs. `credentialStatus` is computed and stripped on save. + - Scheduled write approval is invalidated by changes to prompt, run-as user, credential reference, profile or write request. Scheduled writes always require the designated user's panel confirmation. - **Logging:** never log message content, prompts, tool results or keys. Structured end-of-turn log only. - **Thinking:** thinking is never streamed or stored. The Anthropic adapter keeps the raw assistant message (with signatures) only in memory, as `providerState`, for the tool loop. - **Links:** only `/app` or `/app/...` hrefs are emitted (`TurnEngine.isAppHref`). diff --git a/README.md b/README.md index ad5a707..0bf9ff1 100644 --- a/README.md +++ b/README.md @@ -551,6 +551,45 @@ health is probed when unknown/stale with a two-second timeout. Provider cards sh health and down-until, failures show fallback totals, and turn records show the route and final destination. See [UPGRADE.md](UPGRADE.md) for the additive schema. +## Scheduled tasks + +Administrators configure recurring prompts in **Assistant → Scheduled tasks**. +Tasks use Quartz cron syntax (for example `0 0 8 * * ?`) and an IANA timezone +(for example `America/Chicago`; default `UTC`). Tasks start disabled. + +```java +config.withUiApiBaseUrl("http://127.0.0.1:8000") + .withScheduledTasksSchedulerName("hostScheduler") + .withAllowedScheduledCredentialEnvNames(Set.of("ASSISTANT_SCHEDULED_TOKEN")); +``` + +Provide a long-lived access token accepted by the host's authentication provider +in that environment variable. Store only its **name** on the task and set +`runAsUserId` to the corresponding QQQ user id. The runner posts the token to +`/qqq/v1/manageSession`, reads its session cookie and verifies the returned +session's user before starting a new conversation. No system session reads user +data. Configure the loopback URL at startup; a scheduled job cannot rely on a +browser having visited first. + +For kof22, provision a designated Authentik account with the intended application +permissions, obtain a token that kof22's QQQ authentication provider accepts as an +`accessToken`, and inject it through the existing secret-manager/ExternalSecret +pipeline. An Authentik management API token is suitable only if the host's auth +adapter explicitly accepts it. Rotate the token in the secret manager, never in +the task table. This feature does not provision an account or token. + +**Run now**, **Approve write mode** and **Revoke write mode** are Review/Result +record actions. Write approval names the designated user and displays the exact +prompt SHA-256. Changing prompt, user, credential reference, profile or write-mode +request clears approval. Scheduled writes always wait for that user's panel +confirmation, even if normal policy disables confirmation. + +Each run keeps an owner-locked audit row and a normal conversation. RECORD adds +no notification; NOTIFICATION and BOTH also notify the designated user through +the optional Alerts QBit SPI. Without Alerts, notification delivery is log-only. +HIPAA summaries are always `see conversation`. Cost caps stop at dispatch +boundaries, so the last completed call can cross the cap; no further call starts. +Apply the [P-g migration](UPGRADE.md#p-g-scheduled-tasks) before enabling jobs. ### Answer presentation The always-available `present_results` tool turns retrieved results into metrics, diff --git a/UPGRADE.md b/UPGRADE.md index fd719a8..de70f99 100644 --- a/UPGRADE.md +++ b/UPGRADE.md @@ -1,5 +1,36 @@ # Upgrading qbit-assistant +## P-g: scheduled tasks + +Apply the **P-g** DDL in [docs/schema-postgres.sql](docs/schema-postgres.sql) +before deploying. The unique `(task_id, scheduled_for)` constraint is required +for multi-replica safety; retain run rows as schedule-slot claim history. +Existing conversations default to channel `panel`. Tasks start disabled. + +Set `withScheduledTasksSchedulerName(...)` to an existing QQQ scheduler and +configure `withUiApiBaseUrl(...)` with this replica's own listener URL. The runner +polls every 60 seconds. QQQ must supply a scheduler session able to manage the +assistant tables; this QBit never creates that session. The configured backend +must enforce unique keys atomically (Postgres does). + +Allow only designated credential references with +`withAllowedScheduledCredentialEnvNames(Set.of("ASSISTANT_SCHEDULED_TOKEN"))`. +Provide that variable through the host's secret manager. It must hold a token +accepted by the host's `manageSession` authentication provider for `runAsUserId`. +The runner fails closed on missing/disallowed credentials or an identity mismatch. +See [scheduled-task setup](README.md#scheduled-tasks) for kof22 provisioning. + +QQQ RC.5 has no conditional UpdateInput. The runner uses a database-unique +schedule-slot insertion as its optimistic claim, then advances `nextRunAt` +before starting work. Crashes can skip a slot; they never retry it. No ArchUnit +exception or fabricated session is introduced. Core client-channel enforcement +arrives with the client-channel RC; the header and tool audit are already sent. + +Notifications use the optional Alerts QBit notification SPI with a single-user +audience. Without it, delivery logs run id/status only. All output modes retain +the owner-locked audit row and conversation. HIPAA run summaries never copy the +answer; they contain only `see conversation`. + ## P-f: knowledge retrieval Apply the **P-f** assistant-table DDL in diff --git a/docs/DESIGN.md b/docs/DESIGN.md index 2081531..40295e6 100644 --- a/docs/DESIGN.md +++ b/docs/DESIGN.md @@ -469,6 +469,92 @@ AI exposure NONE is not enough, because the assistant acts as the user and an ad per-record pgvector replacement is transactional, while scan reconciliation is not a snapshot of concurrently changing application data. +### P-g as built + +- `assistant_scheduled_task` is admin-only, AI exposure NONE, and appears as + Scheduled tasks. The owner-locked, read-only `assistant_scheduled_run` child + list retains run status, accounting and a bounded summary. Conversations gain + read-only `scheduledTaskId` and `channel` (`panel` or `assistant/scheduled`). + Task and run timestamps use the runtime clock; cron uses Quartz and a validated + ZoneId, including DST. Enablement and schedule edits recompute `nextRunAt`. +- Credentials are references, never stored tokens. `credentialEnvName` must + match `[A-Z_][A-Z0-9_]*` and the dedicated + `allowedScheduledCredentialEnvNames` allowlist. The existing secret resolver + supplies its value. Read-only `credentialStatus` reports presence without + exposing token characters and is stripped on every save. +- A run sends JSON `{accessToken: ...}` to the same loopback base URL's + `POST /qqq/v1/manageSession`. RC.5 with HTTP-only cookies returns the UUID in + `Set-Cookie: sessionUUID`; legacy JSON `uuid` is also accepted. Only then does + the runner resolve that UUID through the host authentication module, exactly + the UUID lookup used by the UI request pipeline. No direct token authentication, + automated-user session or constructed QSession exists in assistant code. + The resolved user id must equal `runAsUserId` before any model call. MOCK in + the integration fixture maps test tokens to the existing user-prefix sessions; + that token adapter is test code only. +- kof22 provisions an Authentik designated account with normal QQQ permissions + and injects an accepted long-lived token through its secret manager and + ExternalSecret into an allow-listed environment variable. The task holds the + variable name and QQQ user id. Host authentication must accept that token type + as `accessToken`; an Authentik management API token is not automatically an + application access token. Neither account nor credential is provisioned here. +- `assistantScheduledTasksRunner` registers a 60-second QScheduleMetaData when + `scheduledTasksSchedulerName` names a host scheduler. QQQ supplies its normal + scheduler context for assistant-table bookkeeping; no ArchUnit exception is + necessary. Configure `uiApiBaseUrl` at startup. No browser visit is required. +- **Claim compatibility:** RC.5 UpdateInput cannot express a conditional + `WHERE next_run_at = expected`. AssistantStore therefore claims the expected + schedule slot using the database-enforced unique `(taskId, scheduledFor)` run + key and advances `nextRunAt` before login/dispatch. This is a unique-insert + optimistic claim rather than a conditional update. The memory fixture also + serializes claim operations on its instance. The database constraint, not + that monitor, prevents cross-replica duplicate execution. Do not remove claim + history or deploy to a backend without atomic unique keys. A crash can lose a + run; it never retries a claimed slot. A later tick advances an already-claimed + slot left behind by a crash between claim and schedule update. +- The unchanged AssistantApi starts a new normal turn under the logged-in + designated user, preserving start gates, rate limiting, P-c classification, + P-d budgets and P-e routing/fallback. An internal context supplies the task's + selected profile and additional cost cap. Cost includes chat, classifier, + failed calls and knowledge-query embeddings. STOP is checked at dispatch + boundaries; an in-flight call can exceed the cap, and no next call is sent. +- Every scheduled UI API request, including metadata and knowledge retrieval, + carries `X-QQQ-Client: assistant/scheduled/`. The only uiapi change is + additive `LoopbackUiApiGateway.invokeWithClient`; custom gateways currently + fail closed for scheduled requests because their channel contract is unknown. + Tool-call rows also record `clientChannel`. Core channel deny enforcement + remains pending the client-channel RC; existing AI exposure/data-access + restrictions continue to apply on RC.5. +- Approve write mode, Revoke write mode and Run now are admin record actions + using standard streamed ETL Review/Result. Review shows the exact prompt's + SHA-256 and run-as user. Result rereads and compares the complete reviewed + definition before recording the actual administrator's user id and time. + A persisted approval hash additionally binds prompt, run-as user, credential + reference, profile and requested mode. Editing any of these clears approval; + ordinary saves cannot set approval or computed bookkeeping fields. +- This base has create/update/delete tools, but no W1 conversation mode/policy + switch. Scheduled runs explicitly narrow the existing tool policy: write tools + require both request and matching approval, a READ_WRITE table and the policy's + allowed-tools list. All scheduled writes require panel confirmation, including + when normal policy allows immediate writes. A pending proposal adds a waiting + notice to the run summary; the scheduler never confirms it. +- RECORD keeps the run and conversation; NOTIFICATION and BOTH also submit an + Alerts notification with `NotificationAudience.user(runAsUserId)` and an + `/app/...` conversation link. Reflection targets the installed Alerts QBit's + RC.5 notification SPI/sink, following WebhooksAlerts' optional-dependency + pattern. Without it, a log contains run id and status only. Notification text + contains no answer data. In HIPAA mode run summaries contain exactly + `see conversation`; conversation payloads retain the existing encryption. + The existing record URL is an admin-screen link. Non-admin owners can read + their scheduled conversations in the panel's conversation list; opening a + particular existing conversation from a notification still needs a frontend + deep-link contract (the current panel only supports open/new-prompt links). + This phase does not broaden existing conversation record-screen permissions. +- Failures close the run as FAILED or BLOCKED with sanitized errors and wait + until the next schedule. All output modes retain their audit record. Tests + exercise real manageSession and UI API requests with a fixed clock, designated + permissions and owner locks, headers, competing runners, approval invalidation, + notification modes, PHI refusal, encrypted HIPAA storage and cost STOP. + ### Phases (each a green PR) | | Scope | Depends on | |---|---|---| diff --git a/docs/schema-postgres.sql b/docs/schema-postgres.sql index 4eeae53..72a273e 100644 --- a/docs/schema-postgres.sql +++ b/docs/schema-postgres.sql @@ -524,3 +524,53 @@ CREATE TABLE assistant_rag_source ( -- ); -- CREATE INDEX IF NOT EXISTS assistant_rag_chunk_cosine -- ON assistant_rag.assistant_rag_chunk USING hnsw (embedding vector_cosine_ops); + +-- P-g: designated-user scheduled assistant tasks. Install before enabling jobs. +CREATE TABLE assistant_scheduled_task ( + id SERIAL PRIMARY KEY, + name VARCHAR(200) NOT NULL, + enabled BOOLEAN NOT NULL DEFAULT FALSE, + prompt TEXT NOT NULL CHECK (char_length(prompt) BETWEEN 1 AND 4000), + run_as_user_id VARCHAR(200) NOT NULL, + credential_env_name VARCHAR(200) NOT NULL, + credential_status VARCHAR(200), -- computed; always NULL in stored rows + profile_id INTEGER REFERENCES assistant_profile(id), + cron_expression VARCHAR(200) NOT NULL, + timezone VARCHAR(100) NOT NULL DEFAULT 'UTC', + next_run_at TIMESTAMPTZ, + output_mode VARCHAR(20) NOT NULL DEFAULT 'RECORD' CHECK (output_mode IN ('RECORD', 'NOTIFICATION', 'BOTH')), + write_mode_requested BOOLEAN NOT NULL DEFAULT FALSE, + write_mode_approved_by VARCHAR(200), + write_mode_approved_at TIMESTAMPTZ, + write_mode_approval_hash VARCHAR(64), + max_cost_usd_per_run NUMERIC(18,6) CHECK (max_cost_usd_per_run >= 0), + last_run_at TIMESTAMPTZ, + last_run_status VARCHAR(20), + last_run_error VARCHAR(500), + create_date TIMESTAMPTZ, + modify_date TIMESTAMPTZ +); +CREATE INDEX assistant_scheduled_task_due ON assistant_scheduled_task(next_run_at) WHERE enabled; + +ALTER TABLE assistant_conversation ADD COLUMN scheduled_task_id INTEGER REFERENCES assistant_scheduled_task(id); +ALTER TABLE assistant_conversation ADD COLUMN channel VARCHAR(64) NOT NULL DEFAULT 'panel'; +ALTER TABLE assistant_tool_call ADD COLUMN client_channel VARCHAR(64); + +CREATE TABLE assistant_scheduled_run ( + id SERIAL PRIMARY KEY, + task_id INTEGER NOT NULL REFERENCES assistant_scheduled_task(id), + user_id VARCHAR(200) NOT NULL, + scheduled_for TIMESTAMPTZ NOT NULL, + started_at TIMESTAMPTZ, + ended_at TIMESTAMPTZ, + status VARCHAR(20) NOT NULL CHECK (status IN ('RUNNING', 'OK', 'FAILED', 'SKIPPED', 'BLOCKED')), + conversation_id INTEGER REFERENCES assistant_conversation(id) ON DELETE SET NULL, + turn_id INTEGER REFERENCES assistant_turn(id) ON DELETE SET NULL, + summary VARCHAR(2000), + cost_micros BIGINT, + error VARCHAR(500), + create_date TIMESTAMPTZ, + modify_date TIMESTAMPTZ, + UNIQUE (task_id, scheduled_for) +); +CREATE INDEX assistant_scheduled_run_owner ON assistant_scheduled_run(user_id, started_at); diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/AssistantQBitConfig.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/AssistantQBitConfig.java index 75cce19..0c8b63c 100644 --- a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/AssistantQBitConfig.java +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/AssistantQBitConfig.java @@ -17,6 +17,7 @@ package com.kingsrook.qbits.assistant; +import javax.net.ssl.SSLContext; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Path; @@ -28,7 +29,6 @@ import java.util.Map; import java.util.Set; import java.util.function.Function; import java.util.regex.Pattern; -import javax.net.ssl.SSLContext; import com.fasterxml.jackson.annotation.JsonIgnore; import com.kingsrook.qbits.assistant.classification.ClassificationSource; import com.kingsrook.qbits.assistant.classification.ConfiguredClassificationSource; @@ -110,6 +110,9 @@ public class AssistantQBitConfig implements QBitConfig private Integer policyCacheSeconds = 30; private String ownerSecurityKeyType = DEFAULT_OWNER_KEY_TYPE; private String shortcut = DEFAULT_SHORTCUT; + private Set allowedScheduledCredentialEnvNames = new LinkedHashSet<>(); + private String scheduledTasksSchedulerName; + private String retentionSchedulerName; private Boolean registerRoutes = true; private Map pricing = ModelPricing.defaults(); @@ -214,6 +217,13 @@ public class AssistantQBitConfig implements QBitConfig requirePositive(errors, "killSwitchPollMillis", killSwitchPollMillis); requirePositive(errors, "providerTestTimeoutSeconds", providerTestTimeoutSeconds); requirePositive(errors, "homeMaxRows", homeMaxRows); + for(String name : allowedScheduledCredentialEnvNames == null ? Set.of() : allowedScheduledCredentialEnvNames) + { + if(name == null || !ENV_NAME.matcher(name).matches()) + { + errors.add("allowedScheduledCredentialEnvNames contains an invalid environment variable name."); + } + } for(String name : allowedApiKeyEnvNames == null ? Set.of() : allowedApiKeyEnvNames) { if(name == null || !ENV_NAME.matcher(name).matches()) @@ -1855,4 +1865,66 @@ public class AssistantQBitConfig implements QBitConfig this.vectorStore = vectorStore; return (this); } + + + + /******************************************************************************* + ** Getter for allowedScheduledCredentialEnvNames + *******************************************************************************/ + public Set getAllowedScheduledCredentialEnvNames() + { + return (allowedScheduledCredentialEnvNames); + } + + + + /******************************************************************************* + ** Setter for allowedScheduledCredentialEnvNames + *******************************************************************************/ + public void setAllowedScheduledCredentialEnvNames(Set allowedScheduledCredentialEnvNames) + { + this.allowedScheduledCredentialEnvNames = allowedScheduledCredentialEnvNames; + } + + + + /******************************************************************************* + ** Fluent setter for allowedScheduledCredentialEnvNames + *******************************************************************************/ + public AssistantQBitConfig withAllowedScheduledCredentialEnvNames(Set allowedScheduledCredentialEnvNames) + { + this.allowedScheduledCredentialEnvNames = allowedScheduledCredentialEnvNames; + return (this); + } + + + + /******************************************************************************* + ** Getter for scheduledTasksSchedulerName + *******************************************************************************/ + public String getScheduledTasksSchedulerName() + { + return (scheduledTasksSchedulerName); + } + + + + /******************************************************************************* + ** Setter for scheduledTasksSchedulerName + *******************************************************************************/ + public void setScheduledTasksSchedulerName(String scheduledTasksSchedulerName) + { + this.scheduledTasksSchedulerName = scheduledTasksSchedulerName; + } + + + + /******************************************************************************* + ** Fluent setter for scheduledTasksSchedulerName + *******************************************************************************/ + public AssistantQBitConfig withScheduledTasksSchedulerName(String scheduledTasksSchedulerName) + { + this.scheduledTasksSchedulerName = scheduledTasksSchedulerName; + return (this); + } } diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/AssistantQBitProducer.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/AssistantQBitProducer.java index 917dffa..97d1481 100644 --- a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/AssistantQBitProducer.java +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/AssistantQBitProducer.java @@ -21,17 +21,21 @@ import java.io.InputStream; import java.util.ArrayList; import java.util.List; import java.util.Properties; +import com.kingsrook.qbits.assistant.api.AssistantRoutes; +import com.kingsrook.qbits.assistant.developer.AssistantDeveloperPanels; import com.kingsrook.qbits.assistant.home.AssistantAccessWidget; import com.kingsrook.qbits.assistant.home.AssistantHome; +import com.kingsrook.qbits.assistant.metadata.AssistantInstanceMetaData; import com.kingsrook.qbits.assistant.metadata.AssistantMetaDataBuilder; import com.kingsrook.qbits.assistant.metadata.AssistantRecordScreens; +import com.kingsrook.qbits.assistant.model.AssistantBudget; import com.kingsrook.qbits.assistant.processes.DiscoverModelsProcess; import com.kingsrook.qbits.assistant.processes.KnowledgeIndexProcess; +import com.kingsrook.qbits.assistant.processes.ScheduledTaskProcess; import com.kingsrook.qbits.assistant.processes.TestConnectionProcess; import com.kingsrook.qbits.assistant.processes.TestModelConnectionProcess; import com.kingsrook.qqq.backend.core.alerts.AlertSeverity; import com.kingsrook.qqq.backend.core.alerts.QAlertTypeMetaData; -import com.kingsrook.qbits.assistant.model.AssistantBudget; import com.kingsrook.qqq.backend.core.exceptions.QException; import com.kingsrook.qqq.backend.core.logging.QLogger; import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerMultiOutput; @@ -174,7 +178,7 @@ public class AssistantQBitProducer implements QBitMetaDataProducer pageContext, PolicyService.EffectivePolicy policy, TurnClassification classification, RunningTurn running, Instant deadline, Consumer notice) throws QException, InterruptedException { - AssistantProfile profile = runtime.getProfileService().pick(session, runtime.appNameFor(pageContext)); + var scheduled = runtime.getScheduledTurn().task(conversation.getId()); + ScheduledTurn.checkCost(scheduled, 0L); + AssistantProfile profile = runtime.getProfileService().pick(session, runtime.appNameFor(pageContext), scheduled == null ? null : scheduled.getProfileId()); String mode = profile == null || profile.getRoutingMode() == null ? "OFF" : profile.getRoutingMode(); Boolean escalation = profile == null && Boolean.TRUE.equals(runtime.getConfig().getModels().getEscalationEnabled()); String decision = "OFF"; diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/engine/ProfileService.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/engine/ProfileService.java index 265be51..e56b912 100644 --- a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/engine/ProfileService.java +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/engine/ProfileService.java @@ -107,7 +107,17 @@ public class ProfileService *******************************************************************************/ public Persona persona(QSession session, String appName) { - AssistantProfile profile = pick(session, appName); + return (persona(session, appName, null)); + } + + + + /******************************************************************************* + ** An administrator-selected scheduled profile still respects role scope. + *******************************************************************************/ + public Persona persona(QSession session, String appName, Integer profileId) + { + AssistantProfile profile = pick(session, appName, profileId); if(profile == null) { return (new Persona(null, DEFAULT_DISPLAY_NAME, DEFAULT_ICON, null, null, List.of())); @@ -217,6 +227,22 @@ public class ProfileService *******************************************************************************/ public AssistantProfile pick(QSession session, String appName) { + return (pick(session, appName, null)); + } + + + + /******************************************************************************* + ** Explicit profiles supply instructions and routing, never permissions. + *******************************************************************************/ + public AssistantProfile pick(QSession session, String appName, Integer profileId) + { + if(profileId != null) + { + return (load().profiles().stream().filter(p -> profileId.equals(p.getId()) && !Boolean.FALSE.equals(p.getEnabled()) + && (!StringUtils.hasContent(p.getPermissionName()) || session.hasPermission(p.getPermissionName()))).findFirst().orElse(null)); + } + List enabled = load().profiles().stream().filter(p -> !Boolean.FALSE.equals(p.getEnabled())).toList(); return (enabled.stream() .filter(p -> appliesTo(p, session, appName)) diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/engine/TurnEngine.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/engine/TurnEngine.java index 6cf6f8b..06a8262 100644 --- a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/engine/TurnEngine.java +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/engine/TurnEngine.java @@ -27,16 +27,20 @@ import java.util.HashSet; import java.util.LinkedHashMap; import java.util.List; import java.util.Map; +import java.util.Objects; import java.util.Set; import java.util.concurrent.ConcurrentLinkedQueue; +import java.util.concurrent.ExecutionException; import java.util.concurrent.Future; import java.util.concurrent.Semaphore; import java.util.concurrent.TimeUnit; +import java.util.concurrent.TimeoutException; import com.kingsrook.qbits.assistant.AssistantQBitConfig; import com.kingsrook.qbits.assistant.AssistantRuntime; import com.kingsrook.qbits.assistant.model.AssistantConversation; import com.kingsrook.qbits.assistant.model.AssistantMessage; import com.kingsrook.qbits.assistant.model.AssistantPendingAction; +import com.kingsrook.qbits.assistant.model.AssistantScheduledTask; import com.kingsrook.qbits.assistant.model.AssistantToolCall; import com.kingsrook.qbits.assistant.model.AssistantTurn; import com.kingsrook.qbits.assistant.persistence.AssistantStore; @@ -51,6 +55,7 @@ import com.kingsrook.qbits.assistant.provider.ModelRequest; import com.kingsrook.qbits.assistant.provider.ModelResponse; import com.kingsrook.qbits.assistant.provider.ModelUsage; import com.kingsrook.qbits.assistant.provider.StreamHandle; +import com.kingsrook.qbits.assistant.scheduled.ScheduledTurn; import com.kingsrook.qbits.assistant.tools.KnowledgeTool; import com.kingsrook.qbits.assistant.tools.PresentResultsTool; import com.kingsrook.qbits.assistant.tools.bulkload.BulkLoadTools; @@ -127,6 +132,9 @@ public class TurnEngine implements Runnable private final StringBuilder textBuffer = new StringBuilder(); private long lastTextFlush = System.currentTimeMillis(); + private AssistantScheduledTask scheduledTask; + private PolicyService.ToolPolicy toolPolicy; + private Instant startedAt; private Instant deadline; private int toolCallCount = 0; @@ -185,6 +193,13 @@ public class TurnEngine implements Runnable try { QContext.init(runtime.getQInstance(), session); + scheduledTask = runtime.getScheduledTurn().task(conversation.getId()); + if(conversation.getScheduledTaskId() != null && scheduledTask == null) + { + throw (new QException("Scheduled execution context is no longer available; start a new conversation.")); + } + toolPolicy = scheduledTask == null ? policy.tools() : ScheduledTurn.tools(runtime, scheduledTask, policy.tools()); + ScheduledTurn.checkCost(runtime, conversation.getId(), turn.getId()); classification = new TurnClassification(config.getClassificationSource(), conversation, pageContext); store.saveClassification(userId, conversation.getId(), turn.getId(), classification); startKillSwitchWatcher(); @@ -230,9 +245,9 @@ public class TurnEngine implements Runnable } String appName = runtime.appNameFor(pageContext); - ProfileService.Persona persona = runtime.getProfileService().persona(session, appName); + ProfileService.Persona persona = runtime.getProfileService().persona(session, appName, scheduledTask == null ? null : scheduledTask.getProfileId()); String profileText = ProfileService.instructions(persona, runtime.getProfileService().relevantSkills(persona.profileId(), userText, appName)); - ModelRequest request = runtime.getContextBuilder().build(session, model, route.effort(), history, userText, pageContext, policy.tools(), profileText); + ModelRequest request = runtime.getContextBuilder().build(session, model, route.effort(), history, userText, pageContext, toolPolicy, profileText); Integer maxPerRequest = active.row() == null ? null : active.row().getMaxOutputTokensPerRequest(); if(maxPerRequest != null && (request.getMaxTokens() == null || request.getMaxTokens() > maxPerRequest)) { @@ -242,7 +257,7 @@ public class TurnEngine implements Runnable { request.setParallelToolCalls(false); } - Set allowedTools = runtime.getContextBuilder().toolNames(policy.tools()); + Set allowedTools = runtime.getContextBuilder().toolNames(toolPolicy); boolean toolLimitReached = false; while(true) @@ -259,6 +274,7 @@ public class TurnEngine implements Runnable ModelResponse response = fallback.call(request, () -> active, selected -> callModelWithStrictFallback(selected, request)); stopReason = response.getStopReason(); + ScheduledTurn.checkCost(runtime, conversation.getId(), turn.getId()); if(ModelResponse.STOP_CANCELLED.equals(stopReason)) { @@ -543,6 +559,7 @@ public class TurnEngine implements Runnable { return (new ModelResponse().withStopReason(ModelResponse.STOP_CANCELLED)); } + ScheduledTurn.checkCost(runtime, conversation.getId(), turn.getId()); PolicyService.BudgetStatus configuredBudget = runtime.getPolicyService().checkConfiguredBudget(policy, userId, usage.billableTokens(), costMicros); if(configuredBudget.hardExceeded()) { @@ -556,7 +573,7 @@ public class TurnEngine implements Runnable throw (new BudgetService.Blocked(budgetDecision.reason())); } ProviderService.Active selected = budgetDecision.active(); - if(!java.util.Objects.equals(active.providerId(), selected.providerId()) || !java.util.Objects.equals(active.model(), selected.model())) + if(!Objects.equals(active.providerId(), selected.providerId()) || !Objects.equals(active.model(), selected.model())) { active = selected; model = active.model(); @@ -761,11 +778,11 @@ public class TurnEngine implements Runnable { future.get(waitMillis, TimeUnit.MILLISECONDS); } - catch(java.util.concurrent.TimeoutException e) + catch(TimeoutException e) { future.cancel(true); } - catch(java.util.concurrent.ExecutionException e) + catch(ExecutionException e) { LOG.debug("tool future failed", hipaaSafe(e)); } @@ -853,6 +870,7 @@ public class TurnEngine implements Runnable .withUserId(userId) .withToolCallId(use.getToolUseId()) .withToolName(use.getToolName()) + .withClientChannel(scheduledTask == null ? null : "assistant/scheduled/" + scheduledTask.getId()) .withInputJson(truncate(redactInput(PresentResultsTool.NAME.equals(use.getToolName()) ? (presentation ? result.getRender() : Map.of()) : use.getInput()), MAX_INPUT_JSON_CHARS)) .withOutcome(outcome) .withPermissionCheck(result.getPermissionCheck()) @@ -899,7 +917,7 @@ public class TurnEngine implements Runnable *******************************************************************************/ private UiApiToolbox.Context toolContext() { - return (new UiApiToolbox.Context(session.getUuid(), userTimezone(), runtime.getUiApiGateway(), policy.tools(), this::proposeWrite)); + return (new UiApiToolbox.Context(session.getUuid(), userTimezone(), runtime.getUiApiGateway(), toolPolicy, this::proposeWrite)); } @@ -1073,6 +1091,10 @@ public class TurnEngine implements Runnable finally { running.getLog().close(); + if(scheduledTask != null) + { + runtime.getScheduledTurn().detach(conversation.getId(), session.getUuid()); + } runtime.unregister(turn.getId()); runtime.getRateLimiter().release(userId); } diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantMetaDataBuilder.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantMetaDataBuilder.java index 4b7972e..c270244 100644 --- a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantMetaDataBuilder.java +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantMetaDataBuilder.java @@ -17,6 +17,8 @@ package com.kingsrook.qbits.assistant.metadata; +import java.util.ArrayList; +import java.util.List; import com.kingsrook.qbits.assistant.AssistantQBitConfig; import com.kingsrook.qbits.assistant.home.AssistantAccessWidget; import com.kingsrook.qbits.assistant.home.AssistantHome; @@ -30,13 +32,15 @@ import com.kingsrook.qbits.assistant.model.AssistantDataAudit; import com.kingsrook.qbits.assistant.model.AssistantFeedback; import com.kingsrook.qbits.assistant.model.AssistantMessage; import com.kingsrook.qbits.assistant.model.AssistantModel; -import com.kingsrook.qbits.assistant.model.AssistantRagSource; import com.kingsrook.qbits.assistant.model.AssistantModelPrice; import com.kingsrook.qbits.assistant.model.AssistantPendingAction; import com.kingsrook.qbits.assistant.model.AssistantPolicy; import com.kingsrook.qbits.assistant.model.AssistantProfile; import com.kingsrook.qbits.assistant.model.AssistantProfileSkill; import com.kingsrook.qbits.assistant.model.AssistantProvider; +import com.kingsrook.qbits.assistant.model.AssistantRagSource; +import com.kingsrook.qbits.assistant.model.AssistantScheduledRun; +import com.kingsrook.qbits.assistant.model.AssistantScheduledTask; import com.kingsrook.qbits.assistant.model.AssistantSkill; import com.kingsrook.qbits.assistant.model.AssistantToolCall; import com.kingsrook.qbits.assistant.model.AssistantTurn; @@ -68,8 +72,6 @@ import com.kingsrook.qqq.backend.core.model.metadata.tables.Capability; import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData; import com.kingsrook.qqq.backend.core.model.metadata.tables.UniqueKey; import com.kingsrook.qqq.backend.core.utils.StringUtils; -import java.util.ArrayList; -import java.util.List; /******************************************************************************* @@ -111,7 +113,7 @@ public class AssistantMetaDataBuilder ** calls on a conversation; model calls on a provider) or reached from the ** home. Processes are table and record actions, never nav entries. *******************************************************************************/ - public static final List NAV_TABLES = List.of(AssistantRagSource.TABLE_NAME, AssistantBudget.TABLE_NAME, AssistantProvider.TABLE_NAME, AssistantProfile.TABLE_NAME, AssistantSkill.TABLE_NAME, AssistantDataAccess.TABLE_NAME, AssistantConversation.TABLE_NAME, AssistantModelPrice.TABLE_NAME, AssistantPolicy.TABLE_NAME); + public static final List NAV_TABLES = List.of(AssistantScheduledTask.TABLE_NAME, AssistantRagSource.TABLE_NAME, AssistantBudget.TABLE_NAME, AssistantProvider.TABLE_NAME, AssistantProfile.TABLE_NAME, AssistantSkill.TABLE_NAME, AssistantDataAccess.TABLE_NAME, AssistantConversation.TABLE_NAME, AssistantModelPrice.TABLE_NAME, AssistantPolicy.TABLE_NAME); private final AssistantQBitConfig config; @@ -187,6 +189,17 @@ public class AssistantMetaDataBuilder .withUniqueKey(new UniqueKey("name")) .withRecordLabelFormat("%s").withRecordLabelFields("name")); + QCodeReference scheduledCustomizer = new QCodeReference(AssistantScheduledTaskCustomizer.class); + tables.add(text(base(AssistantScheduledTask.TABLE_NAME, "Scheduled tasks", "schedule", AssistantScheduledTask.class), "prompt") + .withRecordLabelFormat("%s").withRecordLabelFields("name") + .withCustomizer(TableCustomizers.PRE_INSERT_RECORD, scheduledCustomizer).withCustomizer(TableCustomizers.PRE_UPDATE_RECORD, scheduledCustomizer) + .withCustomizer(TableCustomizers.POST_QUERY_RECORD, scheduledCustomizer)); + tables.add(readOnly(owned(base(AssistantScheduledRun.TABLE_NAME, "Scheduled runs", "history", AssistantScheduledRun.class), false)) + .withoutCapabilities(Capability.TABLE_DELETE) + .withPermissionRules(new QPermissionRules().withLevel(PermissionLevel.NOT_PROTECTED)) + .withUniqueKey(new UniqueKey("taskId", "scheduledFor")) + .withRecordLabelFormat("Run #%s").withRecordLabelFields("id")); + QCodeReference ragCustomizer = new QCodeReference(AssistantRagSourceCustomizer.class); tables.add(text(base(AssistantRagSource.TABLE_NAME, "Knowledge sources", "manage_search", AssistantRagSource.class), "filterJson") .withRecordLabelFormat("%s").withRecordLabelFields("name") diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantRecordScreens.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantRecordScreens.java index 75bb9d6..3ed6616 100644 --- a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantRecordScreens.java +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantRecordScreens.java @@ -43,6 +43,8 @@ import com.kingsrook.qbits.assistant.model.AssistantProfile; import com.kingsrook.qbits.assistant.model.AssistantProfileSkill; import com.kingsrook.qbits.assistant.model.AssistantProvider; import com.kingsrook.qbits.assistant.model.AssistantRagSource; +import com.kingsrook.qbits.assistant.model.AssistantScheduledRun; +import com.kingsrook.qbits.assistant.model.AssistantScheduledTask; import com.kingsrook.qbits.assistant.model.AssistantSkill; import com.kingsrook.qbits.assistant.model.AssistantToolCall; import com.kingsrook.qbits.assistant.model.AssistantTurn; @@ -52,9 +54,9 @@ import com.kingsrook.qbits.assistant.model.AssistantUsageDaily; import com.kingsrook.qbits.assistant.provider.AssistantProviderType; import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.ChildRecordListRenderer; import com.kingsrook.qqq.backend.core.instances.QInstanceEnricher; -import com.kingsrook.qqq.backend.core.model.metadata.QInstance; import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator; import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter; +import com.kingsrook.qqq.backend.core.model.metadata.QInstance; import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReferenceWithProperties; import com.kingsrook.qqq.backend.core.model.metadata.fields.AdornmentType; import com.kingsrook.qqq.backend.core.model.metadata.fields.DisplayFormat; @@ -132,6 +134,8 @@ public final class AssistantRecordScreens } return (List.of( + QPossibleValueSource.newForTable(AssistantScheduledTask.TABLE_NAME), + enumSource("assistant_scheduled_output", List.of(new QPossibleValue<>("RECORD", "Record"), new QPossibleValue<>("NOTIFICATION", "Notification"), new QPossibleValue<>("BOTH", "Both"))), QPossibleValueSource.newForTable(AssistantBudget.TABLE_NAME), QPossibleValueSource.newForTable(AssistantModel.TABLE_NAME).withName("assistant_registry_model"), QPossibleValueSource.newForTable(AssistantModel.TABLE_NAME).withName("assistant_budget_model"), @@ -173,6 +177,32 @@ public final class AssistantRecordScreens { switch(table.getName()) { + case AssistantScheduledTask.TABLE_NAME -> + { + link(table, "profileId", AssistantProfile.TABLE_NAME, "Profile"); + pvs(table, "outputMode", "assistant_scheduled_output", "Output mode"); + help(table, "credentialEnvName", "Name of an allowed environment variable holding this user's access token. Never enter the token here."); + help(table, "cronExpression", "Quartz cron expression, including seconds; for example 0 0 8 * * ? for 08:00 daily."); + help(table, "timezone", "IANA time zone, for example America/Chicago. Defaults to UTC; daylight saving time follows this zone."); + help(table, "writeModeRequested", "Requires administrator approval of this exact prompt and identity. Each proposed write still requires the user's confirmation."); + help(table, "maxCostUsdPerRun", "Optional USD STOP cap. Checked before each model or embedding call; a completed call may cross the cap."); + sections(table, + section("task", "Task", "schedule", Tier.T1, "id", "name", "enabled", "prompt", "profileId"), + section("identity", "Run as", "person", Tier.T2, "runAsUserId", "credentialEnvName", "credentialStatus"), + section("schedule", "Schedule and output", "event", Tier.T2, "cronExpression", "timezone", "nextRunAt", "outputMode", "maxCostUsdPerRun"), + section("approval", "Write approval", "verified_user", Tier.T2, "writeModeRequested", "writeModeApprovedBy", "writeModeApprovedAt", "writeModeApprovalHash"), + section("lastRun", "Last run", "history", Tier.T2, "lastRunAt", "lastRunStatus", "lastRunError")); + } + case AssistantScheduledRun.TABLE_NAME -> + { + link(table, "taskId", AssistantScheduledTask.TABLE_NAME, "Task"); + link(table, "conversationId", AssistantConversation.TABLE_NAME, "Conversation"); + link(table, "turnId", AssistantTurn.TABLE_NAME, "Turn"); + help(table, "costMicros", MICROS_HELP); + sections(table, + section("run", "Run", "history", Tier.T1, "id", "taskId", "userId", "status", "scheduledFor", "startedAt", "endedAt"), + section("result", "Result", "chat", Tier.T2, "conversationId", "turnId", "summary", "costMicros", "error")); + } case AssistantBudget.TABLE_NAME -> budget(table); case AssistantBudgetOverride.TABLE_NAME, AssistantBudgetEvent.TABLE_NAME -> { @@ -213,6 +243,11 @@ public final class AssistantRecordScreens *******************************************************************************/ public static void addConversationChildLists(QInstance qInstance) { + QTableMetaData scheduledTask = qInstance.getTable(AssistantScheduledTask.TABLE_NAME); + if(scheduledTask != null) + { + childList(qInstance, scheduledTask, AssistantScheduledRun.TABLE_NAME, "Runs", "history", List.of("startedAt", "status", "conversationId", "costMicros", "summary"), "taskId", "scheduled"); + } QTableMetaData conversation = qInstance.getTable(AssistantConversation.TABLE_NAME); if(conversation == null) { diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantScheduledTaskCustomizer.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantScheduledTaskCustomizer.java new file mode 100644 index 0000000..95fce21 --- /dev/null +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantScheduledTaskCustomizer.java @@ -0,0 +1,132 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2026. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.assistant.metadata; + + +import java.io.Serializable; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.Objects; +import java.util.Optional; +import com.kingsrook.qbits.assistant.AssistantRuntime; +import com.kingsrook.qbits.assistant.model.AssistantScheduledTask; +import com.kingsrook.qbits.assistant.persistence.AssistantStore; +import com.kingsrook.qbits.assistant.scheduled.ScheduledTasks; +import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterface; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput; +import com.kingsrook.qqq.backend.core.model.actions.tables.QueryOrGetInputInterface; +import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.statusmessages.BadInputStatusMessage; + + +/******************************************************************************* + ** Validate task edits and invalidate approvals before changed text is saved. + *******************************************************************************/ +public class AssistantScheduledTaskCustomizer implements TableCustomizerInterface +{ + /******************************************************************************* + ** Internal status writes bypass this customizer only inside AssistantStore. + *******************************************************************************/ + @Override + public List preInsertOrUpdate(AbstractActionInput input, List records, boolean isPreview, Optional> oldRecordList) throws QException + { + Optional> old = oldRecordListToMap("id", oldRecordList); + AssistantRuntime runtime = AssistantRuntime.of(QContext.getQInstance()); + for(QRecord record : records) + { + record.getValues().remove("credentialStatus"); + if(input instanceof UpdateInput update + && update.hasFlag(AssistantStore.ScheduledWrite.INTERNAL)) + { + continue; + } + QRecord previous = old.map(rows -> rows.get(record.getValue("id"))).orElse(null); + for(String field : List.of("writeModeApprovedBy", "writeModeApprovedAt", "writeModeApprovalHash", "nextRunAt", "lastRunAt", "lastRunStatus", "lastRunError")) + { + record.getValues().remove(field); + } + QRecord merged = new QRecord(); + merged.setValues(previous == null ? new LinkedHashMap<>() : new LinkedHashMap<>(previous.getValues())); + record.getValues().forEach(merged::setValue); + for(var entry : Map.of("enabled", false, "timezone", "UTC", "outputMode", "RECORD", "writeModeRequested", false).entrySet()) + { + if(merged.getValue(entry.getKey()) == null) + { + merged.setValue(entry.getKey(), entry.getValue()); + record.setValue(entry.getKey(), entry.getValue()); + } + } + if(previous == null || ScheduledTasks.APPROVAL_FIELDS.stream().anyMatch(field -> !Objects.equals(previous.getValue(field), merged.getValue(field)))) + { + record.setValue("writeModeApprovedBy", null); + record.setValue("writeModeApprovedAt", null); + record.setValue("writeModeApprovalHash", null); + } + try + { + AssistantScheduledTask task = new AssistantScheduledTask(merged); + ScheduledTasks.checkReference(runtime.getConfig(), task.getCredentialEnvName()); + if(task.getPrompt() == null || task.getPrompt().isBlank() || task.getPrompt().length() > 4000) + { + throw (new QException("The prompt must contain 1 to 4000 characters.")); + } + if(!List.of("RECORD", "NOTIFICATION", "BOTH").contains(task.getOutputMode())) + { + throw (new QException("Output mode must be RECORD, NOTIFICATION or BOTH.")); + } + if(task.getMaxCostUsdPerRun() != null && task.getMaxCostUsdPerRun().signum() < 0) + { + throw (new QException("The per-run cost cap cannot be negative.")); + } + var next = ScheduledTasks.next(task, runtime.getClock().instant()); + if(previous == null || List.of("enabled", "cronExpression", "timezone").stream().anyMatch(field -> !Objects.equals(previous.getValue(field), merged.getValue(field)))) + { + record.setValue("nextRunAt", Boolean.TRUE.equals(task.getEnabled()) ? next : null); + } + } + catch(QException e) + { + record.addError(new BadInputStatusMessage(e.getMessage())); + } + } + return (records); + } + + + + /******************************************************************************* + ** Computed status is never persisted. + *******************************************************************************/ + @Override + public List postQuery(QueryOrGetInputInterface input, List records) throws QException + { + AssistantRuntime runtime = AssistantRuntime.of(QContext.getQInstance()); + for(QRecord record : records) + { + record.setValue("credentialStatus", ScheduledTasks.credentialStatus(runtime.getConfig(), new AssistantScheduledTask(record))); + } + return (records); + } +} diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/model/AssistantConversation.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/model/AssistantConversation.java index 6ec033e..4d113f9 100644 --- a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/model/AssistantConversation.java +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/model/AssistantConversation.java @@ -38,6 +38,12 @@ public class AssistantConversation extends QRecordEntity @Deprecated(since = "0.1.0-RC.3") public static final String LEGACY_TABLE_NAME = "assistantConversation"; + @QField(isEditable = false) + private Integer scheduledTaskId; + + @QField(isEditable = false, maxLength = 64) + private String channel = "panel"; + @QField(isEditable = false, maxLength = 16) private String maxClassification; @@ -535,4 +541,66 @@ public class AssistantConversation extends QRecordEntity + + + + /******************************************************************************* + ** Getter for scheduledTaskId + *******************************************************************************/ + public Integer getScheduledTaskId() + { + return (scheduledTaskId); + } + + + + /******************************************************************************* + ** Setter for scheduledTaskId + *******************************************************************************/ + public void setScheduledTaskId(Integer scheduledTaskId) + { + this.scheduledTaskId = scheduledTaskId; + } + + + + /******************************************************************************* + ** Fluent setter for scheduledTaskId + *******************************************************************************/ + public AssistantConversation withScheduledTaskId(Integer scheduledTaskId) + { + this.scheduledTaskId = scheduledTaskId; + return (this); + } + + + + /******************************************************************************* + ** Getter for channel + *******************************************************************************/ + public String getChannel() + { + return (channel); + } + + + + /******************************************************************************* + ** Setter for channel + *******************************************************************************/ + public void setChannel(String channel) + { + this.channel = channel; + } + + + + /******************************************************************************* + ** Fluent setter for channel + *******************************************************************************/ + public AssistantConversation withChannel(String channel) + { + this.channel = channel; + return (this); + } } diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/model/AssistantScheduledRun.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/model/AssistantScheduledRun.java new file mode 100644 index 0000000..07bbe1e --- /dev/null +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/model/AssistantScheduledRun.java @@ -0,0 +1,532 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2026. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.assistant.model; + + +import java.time.Instant; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.model.data.QField; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.data.QRecordEntity; + + +/******************************************************************************* + ** Scheduled assistant bookkeeping; credentials are references only. + *******************************************************************************/ +public class AssistantScheduledRun extends QRecordEntity +{ + public static final String TABLE_NAME = "assistant_scheduled_run"; + + @QField(isEditable = false, isPrimaryKey = true) + private Integer id; + + @QField(isRequired = true) + private Integer taskId; + + @QField(isRequired = true, maxLength = 200) + private String userId; + + @QField() + private Instant scheduledFor; + + @QField() + private Instant startedAt; + + @QField() + private Instant endedAt; + + @QField(maxLength = 20) + private String status; + + @QField() + private Integer conversationId; + + @QField() + private Integer turnId; + + @QField(maxLength = 2000) + private String summary; + + @QField() + private Long costMicros; + + @QField(maxLength = 500) + private String error; + + @QField(isEditable = false) + private Instant createDate; + + @QField(isEditable = false) + private Instant modifyDate; + + + + /******************************************************************************* + ** Empty entity for framework mapping. + *******************************************************************************/ + public AssistantScheduledRun() + { + } + + + + /******************************************************************************* + ** Map a bookkeeping record. + *******************************************************************************/ + public AssistantScheduledRun(QRecord record) throws QException + { + populateFromQRecord(record); + } + + + + /******************************************************************************* + ** Getter for id + *******************************************************************************/ + public Integer getId() + { + return (id); + } + + + + /******************************************************************************* + ** Setter for id + *******************************************************************************/ + public void setId(Integer id) + { + this.id = id; + } + + + + /******************************************************************************* + ** Fluent setter for id + *******************************************************************************/ + public AssistantScheduledRun withId(Integer id) + { + this.id = id; + return (this); + } + + + + /******************************************************************************* + ** Getter for taskId + *******************************************************************************/ + public Integer getTaskId() + { + return (taskId); + } + + + + /******************************************************************************* + ** Setter for taskId + *******************************************************************************/ + public void setTaskId(Integer taskId) + { + this.taskId = taskId; + } + + + + /******************************************************************************* + ** Fluent setter for taskId + *******************************************************************************/ + public AssistantScheduledRun withTaskId(Integer taskId) + { + this.taskId = taskId; + return (this); + } + + + + /******************************************************************************* + ** Getter for userId + *******************************************************************************/ + public String getUserId() + { + return (userId); + } + + + + /******************************************************************************* + ** Setter for userId + *******************************************************************************/ + public void setUserId(String userId) + { + this.userId = userId; + } + + + + /******************************************************************************* + ** Fluent setter for userId + *******************************************************************************/ + public AssistantScheduledRun withUserId(String userId) + { + this.userId = userId; + return (this); + } + + + + /******************************************************************************* + ** Getter for scheduledFor + *******************************************************************************/ + public Instant getScheduledFor() + { + return (scheduledFor); + } + + + + /******************************************************************************* + ** Setter for scheduledFor + *******************************************************************************/ + public void setScheduledFor(Instant scheduledFor) + { + this.scheduledFor = scheduledFor; + } + + + + /******************************************************************************* + ** Fluent setter for scheduledFor + *******************************************************************************/ + public AssistantScheduledRun withScheduledFor(Instant scheduledFor) + { + this.scheduledFor = scheduledFor; + return (this); + } + + + + /******************************************************************************* + ** Getter for startedAt + *******************************************************************************/ + public Instant getStartedAt() + { + return (startedAt); + } + + + + /******************************************************************************* + ** Setter for startedAt + *******************************************************************************/ + public void setStartedAt(Instant startedAt) + { + this.startedAt = startedAt; + } + + + + /******************************************************************************* + ** Fluent setter for startedAt + *******************************************************************************/ + public AssistantScheduledRun withStartedAt(Instant startedAt) + { + this.startedAt = startedAt; + return (this); + } + + + + /******************************************************************************* + ** Getter for endedAt + *******************************************************************************/ + public Instant getEndedAt() + { + return (endedAt); + } + + + + /******************************************************************************* + ** Setter for endedAt + *******************************************************************************/ + public void setEndedAt(Instant endedAt) + { + this.endedAt = endedAt; + } + + + + /******************************************************************************* + ** Fluent setter for endedAt + *******************************************************************************/ + public AssistantScheduledRun withEndedAt(Instant endedAt) + { + this.endedAt = endedAt; + return (this); + } + + + + /******************************************************************************* + ** Getter for status + *******************************************************************************/ + public String getStatus() + { + return (status); + } + + + + /******************************************************************************* + ** Setter for status + *******************************************************************************/ + public void setStatus(String status) + { + this.status = status; + } + + + + /******************************************************************************* + ** Fluent setter for status + *******************************************************************************/ + public AssistantScheduledRun withStatus(String status) + { + this.status = status; + return (this); + } + + + + /******************************************************************************* + ** Getter for conversationId + *******************************************************************************/ + public Integer getConversationId() + { + return (conversationId); + } + + + + /******************************************************************************* + ** Setter for conversationId + *******************************************************************************/ + public void setConversationId(Integer conversationId) + { + this.conversationId = conversationId; + } + + + + /******************************************************************************* + ** Fluent setter for conversationId + *******************************************************************************/ + public AssistantScheduledRun withConversationId(Integer conversationId) + { + this.conversationId = conversationId; + return (this); + } + + + + /******************************************************************************* + ** Getter for turnId + *******************************************************************************/ + public Integer getTurnId() + { + return (turnId); + } + + + + /******************************************************************************* + ** Setter for turnId + *******************************************************************************/ + public void setTurnId(Integer turnId) + { + this.turnId = turnId; + } + + + + /******************************************************************************* + ** Fluent setter for turnId + *******************************************************************************/ + public AssistantScheduledRun withTurnId(Integer turnId) + { + this.turnId = turnId; + return (this); + } + + + + /******************************************************************************* + ** Getter for summary + *******************************************************************************/ + public String getSummary() + { + return (summary); + } + + + + /******************************************************************************* + ** Setter for summary + *******************************************************************************/ + public void setSummary(String summary) + { + this.summary = summary; + } + + + + /******************************************************************************* + ** Fluent setter for summary + *******************************************************************************/ + public AssistantScheduledRun withSummary(String summary) + { + this.summary = summary; + return (this); + } + + + + /******************************************************************************* + ** Getter for costMicros + *******************************************************************************/ + public Long getCostMicros() + { + return (costMicros); + } + + + + /******************************************************************************* + ** Setter for costMicros + *******************************************************************************/ + public void setCostMicros(Long costMicros) + { + this.costMicros = costMicros; + } + + + + /******************************************************************************* + ** Fluent setter for costMicros + *******************************************************************************/ + public AssistantScheduledRun withCostMicros(Long costMicros) + { + this.costMicros = costMicros; + return (this); + } + + + + /******************************************************************************* + ** Getter for error + *******************************************************************************/ + public String getError() + { + return (error); + } + + + + /******************************************************************************* + ** Setter for error + *******************************************************************************/ + public void setError(String error) + { + this.error = error; + } + + + + /******************************************************************************* + ** Fluent setter for error + *******************************************************************************/ + public AssistantScheduledRun withError(String error) + { + this.error = error; + return (this); + } + + + + /******************************************************************************* + ** Getter for createDate + *******************************************************************************/ + public Instant getCreateDate() + { + return (createDate); + } + + + + /******************************************************************************* + ** Setter for createDate + *******************************************************************************/ + public void setCreateDate(Instant createDate) + { + this.createDate = createDate; + } + + + + /******************************************************************************* + ** Fluent setter for createDate + *******************************************************************************/ + public AssistantScheduledRun withCreateDate(Instant createDate) + { + this.createDate = createDate; + return (this); + } + + + + /******************************************************************************* + ** Getter for modifyDate + *******************************************************************************/ + public Instant getModifyDate() + { + return (modifyDate); + } + + + + /******************************************************************************* + ** Setter for modifyDate + *******************************************************************************/ + public void setModifyDate(Instant modifyDate) + { + this.modifyDate = modifyDate; + } + + + + /******************************************************************************* + ** Fluent setter for modifyDate + *******************************************************************************/ + public AssistantScheduledRun withModifyDate(Instant modifyDate) + { + this.modifyDate = modifyDate; + return (this); + } +} diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/model/AssistantScheduledTask.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/model/AssistantScheduledTask.java new file mode 100644 index 0000000..cac9e69 --- /dev/null +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/model/AssistantScheduledTask.java @@ -0,0 +1,805 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2026. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.assistant.model; + + +import java.math.BigDecimal; +import java.time.Instant; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.model.data.QField; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.data.QRecordEntity; + + +/******************************************************************************* + ** Scheduled assistant bookkeeping; credentials are references only. + *******************************************************************************/ +public class AssistantScheduledTask extends QRecordEntity +{ + public static final String TABLE_NAME = "assistant_scheduled_task"; + + @QField(isEditable = false, isPrimaryKey = true) + private Integer id; + + @QField(isRequired = true, maxLength = 200) + private String name; + + @QField() + private Boolean enabled; + + @QField(isRequired = true, maxLength = 4000) + private String prompt; + + @QField(isRequired = true, maxLength = 200) + private String runAsUserId; + + @QField(isRequired = true, maxLength = 200) + private String credentialEnvName; + + @QField(isEditable = false, maxLength = 200) + private String credentialStatus; + + @QField() + private Integer profileId; + + @QField(isRequired = true, maxLength = 200) + private String cronExpression; + + @QField(maxLength = 100) + private String timezone; + + @QField(isEditable = false) + private Instant nextRunAt; + + @QField(maxLength = 20) + private String outputMode; + + @QField() + private Boolean writeModeRequested; + + @QField(isEditable = false, maxLength = 200) + private String writeModeApprovedBy; + + @QField(isEditable = false) + private Instant writeModeApprovedAt; + + @QField(isEditable = false, maxLength = 64) + private String writeModeApprovalHash; + + @QField() + private BigDecimal maxCostUsdPerRun; + + @QField(isEditable = false) + private Instant lastRunAt; + + @QField(isEditable = false, maxLength = 20) + private String lastRunStatus; + + @QField(isEditable = false, maxLength = 500) + private String lastRunError; + + @QField(isEditable = false) + private Instant createDate; + + @QField(isEditable = false) + private Instant modifyDate; + + + + /******************************************************************************* + ** Empty entity for framework mapping. + *******************************************************************************/ + public AssistantScheduledTask() + { + } + + + + /******************************************************************************* + ** Map a bookkeeping record. + *******************************************************************************/ + public AssistantScheduledTask(QRecord record) throws QException + { + populateFromQRecord(record); + } + + + + /******************************************************************************* + ** Getter for id + *******************************************************************************/ + public Integer getId() + { + return (id); + } + + + + /******************************************************************************* + ** Setter for id + *******************************************************************************/ + public void setId(Integer id) + { + this.id = id; + } + + + + /******************************************************************************* + ** Fluent setter for id + *******************************************************************************/ + public AssistantScheduledTask withId(Integer id) + { + this.id = id; + return (this); + } + + + + /******************************************************************************* + ** Getter for name + *******************************************************************************/ + public String getName() + { + return (name); + } + + + + /******************************************************************************* + ** Setter for name + *******************************************************************************/ + public void setName(String name) + { + this.name = name; + } + + + + /******************************************************************************* + ** Fluent setter for name + *******************************************************************************/ + public AssistantScheduledTask withName(String name) + { + this.name = name; + return (this); + } + + + + /******************************************************************************* + ** Getter for enabled + *******************************************************************************/ + public Boolean getEnabled() + { + return (enabled); + } + + + + /******************************************************************************* + ** Setter for enabled + *******************************************************************************/ + public void setEnabled(Boolean enabled) + { + this.enabled = enabled; + } + + + + /******************************************************************************* + ** Fluent setter for enabled + *******************************************************************************/ + public AssistantScheduledTask withEnabled(Boolean enabled) + { + this.enabled = enabled; + return (this); + } + + + + /******************************************************************************* + ** Getter for prompt + *******************************************************************************/ + public String getPrompt() + { + return (prompt); + } + + + + /******************************************************************************* + ** Setter for prompt + *******************************************************************************/ + public void setPrompt(String prompt) + { + this.prompt = prompt; + } + + + + /******************************************************************************* + ** Fluent setter for prompt + *******************************************************************************/ + public AssistantScheduledTask withPrompt(String prompt) + { + this.prompt = prompt; + return (this); + } + + + + /******************************************************************************* + ** Getter for runAsUserId + *******************************************************************************/ + public String getRunAsUserId() + { + return (runAsUserId); + } + + + + /******************************************************************************* + ** Setter for runAsUserId + *******************************************************************************/ + public void setRunAsUserId(String runAsUserId) + { + this.runAsUserId = runAsUserId; + } + + + + /******************************************************************************* + ** Fluent setter for runAsUserId + *******************************************************************************/ + public AssistantScheduledTask withRunAsUserId(String runAsUserId) + { + this.runAsUserId = runAsUserId; + return (this); + } + + + + /******************************************************************************* + ** Getter for credentialEnvName + *******************************************************************************/ + public String getCredentialEnvName() + { + return (credentialEnvName); + } + + + + /******************************************************************************* + ** Setter for credentialEnvName + *******************************************************************************/ + public void setCredentialEnvName(String credentialEnvName) + { + this.credentialEnvName = credentialEnvName; + } + + + + /******************************************************************************* + ** Fluent setter for credentialEnvName + *******************************************************************************/ + public AssistantScheduledTask withCredentialEnvName(String credentialEnvName) + { + this.credentialEnvName = credentialEnvName; + return (this); + } + + + + /******************************************************************************* + ** Getter for credentialStatus + *******************************************************************************/ + public String getCredentialStatus() + { + return (credentialStatus); + } + + + + /******************************************************************************* + ** Setter for credentialStatus + *******************************************************************************/ + public void setCredentialStatus(String credentialStatus) + { + this.credentialStatus = credentialStatus; + } + + + + /******************************************************************************* + ** Fluent setter for credentialStatus + *******************************************************************************/ + public AssistantScheduledTask withCredentialStatus(String credentialStatus) + { + this.credentialStatus = credentialStatus; + return (this); + } + + + + /******************************************************************************* + ** Getter for profileId + *******************************************************************************/ + public Integer getProfileId() + { + return (profileId); + } + + + + /******************************************************************************* + ** Setter for profileId + *******************************************************************************/ + public void setProfileId(Integer profileId) + { + this.profileId = profileId; + } + + + + /******************************************************************************* + ** Fluent setter for profileId + *******************************************************************************/ + public AssistantScheduledTask withProfileId(Integer profileId) + { + this.profileId = profileId; + return (this); + } + + + + /******************************************************************************* + ** Getter for cronExpression + *******************************************************************************/ + public String getCronExpression() + { + return (cronExpression); + } + + + + /******************************************************************************* + ** Setter for cronExpression + *******************************************************************************/ + public void setCronExpression(String cronExpression) + { + this.cronExpression = cronExpression; + } + + + + /******************************************************************************* + ** Fluent setter for cronExpression + *******************************************************************************/ + public AssistantScheduledTask withCronExpression(String cronExpression) + { + this.cronExpression = cronExpression; + return (this); + } + + + + /******************************************************************************* + ** Getter for timezone + *******************************************************************************/ + public String getTimezone() + { + return (timezone); + } + + + + /******************************************************************************* + ** Setter for timezone + *******************************************************************************/ + public void setTimezone(String timezone) + { + this.timezone = timezone; + } + + + + /******************************************************************************* + ** Fluent setter for timezone + *******************************************************************************/ + public AssistantScheduledTask withTimezone(String timezone) + { + this.timezone = timezone; + return (this); + } + + + + /******************************************************************************* + ** Getter for nextRunAt + *******************************************************************************/ + public Instant getNextRunAt() + { + return (nextRunAt); + } + + + + /******************************************************************************* + ** Setter for nextRunAt + *******************************************************************************/ + public void setNextRunAt(Instant nextRunAt) + { + this.nextRunAt = nextRunAt; + } + + + + /******************************************************************************* + ** Fluent setter for nextRunAt + *******************************************************************************/ + public AssistantScheduledTask withNextRunAt(Instant nextRunAt) + { + this.nextRunAt = nextRunAt; + return (this); + } + + + + /******************************************************************************* + ** Getter for outputMode + *******************************************************************************/ + public String getOutputMode() + { + return (outputMode); + } + + + + /******************************************************************************* + ** Setter for outputMode + *******************************************************************************/ + public void setOutputMode(String outputMode) + { + this.outputMode = outputMode; + } + + + + /******************************************************************************* + ** Fluent setter for outputMode + *******************************************************************************/ + public AssistantScheduledTask withOutputMode(String outputMode) + { + this.outputMode = outputMode; + return (this); + } + + + + /******************************************************************************* + ** Getter for writeModeRequested + *******************************************************************************/ + public Boolean getWriteModeRequested() + { + return (writeModeRequested); + } + + + + /******************************************************************************* + ** Setter for writeModeRequested + *******************************************************************************/ + public void setWriteModeRequested(Boolean writeModeRequested) + { + this.writeModeRequested = writeModeRequested; + } + + + + /******************************************************************************* + ** Fluent setter for writeModeRequested + *******************************************************************************/ + public AssistantScheduledTask withWriteModeRequested(Boolean writeModeRequested) + { + this.writeModeRequested = writeModeRequested; + return (this); + } + + + + /******************************************************************************* + ** Getter for writeModeApprovedBy + *******************************************************************************/ + public String getWriteModeApprovedBy() + { + return (writeModeApprovedBy); + } + + + + /******************************************************************************* + ** Setter for writeModeApprovedBy + *******************************************************************************/ + public void setWriteModeApprovedBy(String writeModeApprovedBy) + { + this.writeModeApprovedBy = writeModeApprovedBy; + } + + + + /******************************************************************************* + ** Fluent setter for writeModeApprovedBy + *******************************************************************************/ + public AssistantScheduledTask withWriteModeApprovedBy(String writeModeApprovedBy) + { + this.writeModeApprovedBy = writeModeApprovedBy; + return (this); + } + + + + /******************************************************************************* + ** Getter for writeModeApprovedAt + *******************************************************************************/ + public Instant getWriteModeApprovedAt() + { + return (writeModeApprovedAt); + } + + + + /******************************************************************************* + ** Setter for writeModeApprovedAt + *******************************************************************************/ + public void setWriteModeApprovedAt(Instant writeModeApprovedAt) + { + this.writeModeApprovedAt = writeModeApprovedAt; + } + + + + /******************************************************************************* + ** Fluent setter for writeModeApprovedAt + *******************************************************************************/ + public AssistantScheduledTask withWriteModeApprovedAt(Instant writeModeApprovedAt) + { + this.writeModeApprovedAt = writeModeApprovedAt; + return (this); + } + + + + /******************************************************************************* + ** Getter for writeModeApprovalHash + *******************************************************************************/ + public String getWriteModeApprovalHash() + { + return (writeModeApprovalHash); + } + + + + /******************************************************************************* + ** Setter for writeModeApprovalHash + *******************************************************************************/ + public void setWriteModeApprovalHash(String writeModeApprovalHash) + { + this.writeModeApprovalHash = writeModeApprovalHash; + } + + + + /******************************************************************************* + ** Fluent setter for writeModeApprovalHash + *******************************************************************************/ + public AssistantScheduledTask withWriteModeApprovalHash(String writeModeApprovalHash) + { + this.writeModeApprovalHash = writeModeApprovalHash; + return (this); + } + + + + /******************************************************************************* + ** Getter for maxCostUsdPerRun + *******************************************************************************/ + public BigDecimal getMaxCostUsdPerRun() + { + return (maxCostUsdPerRun); + } + + + + /******************************************************************************* + ** Setter for maxCostUsdPerRun + *******************************************************************************/ + public void setMaxCostUsdPerRun(BigDecimal maxCostUsdPerRun) + { + this.maxCostUsdPerRun = maxCostUsdPerRun; + } + + + + /******************************************************************************* + ** Fluent setter for maxCostUsdPerRun + *******************************************************************************/ + public AssistantScheduledTask withMaxCostUsdPerRun(BigDecimal maxCostUsdPerRun) + { + this.maxCostUsdPerRun = maxCostUsdPerRun; + return (this); + } + + + + /******************************************************************************* + ** Getter for lastRunAt + *******************************************************************************/ + public Instant getLastRunAt() + { + return (lastRunAt); + } + + + + /******************************************************************************* + ** Setter for lastRunAt + *******************************************************************************/ + public void setLastRunAt(Instant lastRunAt) + { + this.lastRunAt = lastRunAt; + } + + + + /******************************************************************************* + ** Fluent setter for lastRunAt + *******************************************************************************/ + public AssistantScheduledTask withLastRunAt(Instant lastRunAt) + { + this.lastRunAt = lastRunAt; + return (this); + } + + + + /******************************************************************************* + ** Getter for lastRunStatus + *******************************************************************************/ + public String getLastRunStatus() + { + return (lastRunStatus); + } + + + + /******************************************************************************* + ** Setter for lastRunStatus + *******************************************************************************/ + public void setLastRunStatus(String lastRunStatus) + { + this.lastRunStatus = lastRunStatus; + } + + + + /******************************************************************************* + ** Fluent setter for lastRunStatus + *******************************************************************************/ + public AssistantScheduledTask withLastRunStatus(String lastRunStatus) + { + this.lastRunStatus = lastRunStatus; + return (this); + } + + + + /******************************************************************************* + ** Getter for lastRunError + *******************************************************************************/ + public String getLastRunError() + { + return (lastRunError); + } + + + + /******************************************************************************* + ** Setter for lastRunError + *******************************************************************************/ + public void setLastRunError(String lastRunError) + { + this.lastRunError = lastRunError; + } + + + + /******************************************************************************* + ** Fluent setter for lastRunError + *******************************************************************************/ + public AssistantScheduledTask withLastRunError(String lastRunError) + { + this.lastRunError = lastRunError; + return (this); + } + + + + /******************************************************************************* + ** Getter for createDate + *******************************************************************************/ + public Instant getCreateDate() + { + return (createDate); + } + + + + /******************************************************************************* + ** Setter for createDate + *******************************************************************************/ + public void setCreateDate(Instant createDate) + { + this.createDate = createDate; + } + + + + /******************************************************************************* + ** Fluent setter for createDate + *******************************************************************************/ + public AssistantScheduledTask withCreateDate(Instant createDate) + { + this.createDate = createDate; + return (this); + } + + + + /******************************************************************************* + ** Getter for modifyDate + *******************************************************************************/ + public Instant getModifyDate() + { + return (modifyDate); + } + + + + /******************************************************************************* + ** Setter for modifyDate + *******************************************************************************/ + public void setModifyDate(Instant modifyDate) + { + this.modifyDate = modifyDate; + } + + + + /******************************************************************************* + ** Fluent setter for modifyDate + *******************************************************************************/ + public AssistantScheduledTask withModifyDate(Instant modifyDate) + { + this.modifyDate = modifyDate; + return (this); + } +} diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/model/AssistantToolCall.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/model/AssistantToolCall.java index 687a745..37c3a3c 100644 --- a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/model/AssistantToolCall.java +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/model/AssistantToolCall.java @@ -40,6 +40,9 @@ public class AssistantToolCall extends QRecordEntity @Deprecated(since = "0.1.0-RC.3") public static final String LEGACY_TABLE_NAME = "assistantToolCall"; + @QField(isEditable = false, maxLength = 64) + private String clientChannel; + @QField(isEditable = false, isPrimaryKey = true) private Integer id; @@ -803,4 +806,35 @@ public class AssistantToolCall extends QRecordEntity return (this); } + + + + /******************************************************************************* + ** Getter for clientChannel + *******************************************************************************/ + public String getClientChannel() + { + return (clientChannel); + } + + + + /******************************************************************************* + ** Setter for clientChannel + *******************************************************************************/ + public void setClientChannel(String clientChannel) + { + this.clientChannel = clientChannel; + } + + + + /******************************************************************************* + ** Fluent setter for clientChannel + *******************************************************************************/ + public AssistantToolCall withClientChannel(String clientChannel) + { + this.clientChannel = clientChannel; + return (this); + } } diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/persistence/AssistantStore.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/persistence/AssistantStore.java index 1c5c3ad..ef195d9 100644 --- a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/persistence/AssistantStore.java +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/persistence/AssistantStore.java @@ -17,8 +17,20 @@ package com.kingsrook.qbits.assistant.persistence; +import java.io.Serializable; +import java.time.Instant; +import java.time.LocalDate; +import java.util.ArrayList; +import java.util.Collections; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.Objects; +import java.util.concurrent.ConcurrentHashMap; import com.kingsrook.qbits.assistant.AssistantQBitConfig; import com.kingsrook.qbits.assistant.AssistantRuntime; +import com.kingsrook.qbits.assistant.classification.DataClassification; +import com.kingsrook.qbits.assistant.engine.TurnClassification; import com.kingsrook.qbits.assistant.model.AssistantBudget; import com.kingsrook.qbits.assistant.model.AssistantBudgetEvent; import com.kingsrook.qbits.assistant.model.AssistantBudgetOverride; @@ -32,18 +44,22 @@ import com.kingsrook.qbits.assistant.model.AssistantModelPrice; import com.kingsrook.qbits.assistant.model.AssistantPendingAction; import com.kingsrook.qbits.assistant.model.AssistantPolicy; import com.kingsrook.qbits.assistant.model.AssistantProvider; +import com.kingsrook.qbits.assistant.model.AssistantScheduledRun; +import com.kingsrook.qbits.assistant.model.AssistantScheduledTask; import com.kingsrook.qbits.assistant.model.AssistantToolCall; import com.kingsrook.qbits.assistant.model.AssistantTurn; import com.kingsrook.qbits.assistant.model.AssistantTurnEvent; import com.kingsrook.qbits.assistant.model.AssistantUsage; import com.kingsrook.qbits.assistant.model.AssistantUsageDaily; import com.kingsrook.qbits.assistant.provider.ModelUsage; +import com.kingsrook.qbits.assistant.scheduled.ScheduledTasks; import com.kingsrook.qqq.backend.core.actions.tables.DeleteAction; import com.kingsrook.qqq.backend.core.actions.tables.InsertAction; import com.kingsrook.qqq.backend.core.actions.tables.QueryAction; import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction; import com.kingsrook.qqq.backend.core.context.QContext; import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.model.actions.tables.ActionFlag; import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput; import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput; import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertOutput; @@ -56,15 +72,6 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateOutput; import com.kingsrook.qqq.backend.core.model.data.QRecord; import com.kingsrook.qqq.backend.core.model.data.QRecordEntity; import com.kingsrook.qqq.backend.core.utils.CollectionUtils; -import java.io.Serializable; -import java.time.Instant; -import java.time.LocalDate; -import java.util.ArrayList; -import java.util.Collections; -import java.util.LinkedHashMap; -import java.util.List; -import java.util.Map; -import java.util.concurrent.ConcurrentHashMap; /******************************************************************************* @@ -773,7 +780,7 @@ public class AssistantStore /******************************************************************************* ** Save the turn level and never lower persisted conversation history. *******************************************************************************/ - public synchronized void saveClassification(String userId, Integer conversationId, Integer turnId, com.kingsrook.qbits.assistant.engine.TurnClassification classification) throws QException + public synchronized void saveClassification(String userId, Integer conversationId, Integer turnId, TurnClassification classification) throws QException { AssistantConversation current = getConversation(userId, conversationId); classification.inherit(current); @@ -781,8 +788,8 @@ public class AssistantStore .withValue("classification", classification.getClassification().name()).withValue("classificationTables", classification.getTables())); if(current != null) { - var maximum = com.kingsrook.qbits.assistant.classification.DataClassification.max( - com.kingsrook.qbits.assistant.classification.DataClassification.stored(current.getMaxClassification()), classification.getClassification()); + var maximum = DataClassification.max( + DataClassification.stored(current.getMaxClassification()), classification.getClassification()); if(!maximum.name().equals(current.getMaxClassification())) { updateFields(AssistantConversation.TABLE_NAME, new QRecord().withValue("id", conversationId).withValue("maxClassification", maximum.name())); @@ -934,4 +941,98 @@ public class AssistantStore updateFields(AssistantBudgetUsage.TABLE_NAME, projection.withValue("id", existing.get(0).getId())); } } + + + + /******************************************************************************* + ** Internal bookkeeping flag; never accepted from a UI request body. + *******************************************************************************/ + public enum ScheduledWrite implements ActionFlag + { + INTERNAL + } + + + + /******************************************************************************* + ** Update computed task fields through QQQ under the caller's session. + *******************************************************************************/ + public void scheduledUpdate(QRecord record) throws QException + { + UpdateOutput output = new UpdateAction().execute(new UpdateInput(AssistantScheduledTask.TABLE_NAME) + .withRecord(record).withFlag(ScheduledWrite.INTERNAL)); + if(CollectionUtils.nullSafeHasContents(output.getRecords().get(0).getErrors())) + { + throw (new QException("Could not update scheduled task bookkeeping.")); + } + } + + + + /******************************************************************************* + ** Read a task definition, never application data. + *******************************************************************************/ + public AssistantScheduledTask scheduledTask(Integer id) throws QException + { + return (first(query(AssistantScheduledTask.TABLE_NAME, + new QQueryFilter().withCriteria("id", QCriteriaOperator.EQUALS, id), AssistantScheduledTask.class))); + } + + + + /******************************************************************************* + ** Optimistic slot claim. The database unique(taskId, scheduledFor) is the + ** cross-replica arbiter; the monitor also covers the fixture memory backend. + ** Advance before dispatch. A crash can skip a slot, never run it twice. + *******************************************************************************/ + public ScheduledClaim claimScheduled( + AssistantScheduledTask candidate, Instant now, Boolean manual) throws QException + { + synchronized(QContext.getQInstance()) + { + var task = scheduledTask(candidate.getId()); + if(task == null || (!manual && (!Boolean.TRUE.equals(task.getEnabled()) || task.getNextRunAt() == null + || task.getNextRunAt().isAfter(now) || !Objects.equals(task.getNextRunAt(), candidate.getNextRunAt())))) + { + return (null); + } + Instant slot = manual ? now : task.getNextRunAt(); + Instant next = Boolean.TRUE.equals(task.getEnabled()) ? ScheduledTasks.next(task, now) : null; + var filter = new QQueryFilter().withCriteria("taskId", QCriteriaOperator.EQUALS, task.getId()).withCriteria("scheduledFor", QCriteriaOperator.EQUALS, slot); + if(!query(AssistantScheduledRun.TABLE_NAME, filter, AssistantScheduledRun.class).isEmpty()) + { + if(!manual) + { + scheduledUpdate(new QRecord().withValue("id", task.getId()).withValue("nextRunAt", next)); + } + return (null); + } + AssistantScheduledRun run; + try + { + run = insert(new AssistantScheduledRun().withTaskId(task.getId()).withUserId(task.getRunAsUserId()) + .withScheduledFor(slot).withStartedAt(now).withStatus("RUNNING").withCostMicros(0L)); + } + catch(QException e) + { + if(!query(AssistantScheduledRun.TABLE_NAME, filter, AssistantScheduledRun.class).isEmpty()) + { + return (null); + } + throw (e); + } + scheduledUpdate(new QRecord().withValue("id", task.getId()).withValue("nextRunAt", next)); + return (new ScheduledClaim(task, run)); + } + } + + + + /******************************************************************************* + ** The claimed task snapshot and its run owner must remain consistent. + *******************************************************************************/ + public record ScheduledClaim(AssistantScheduledTask task, AssistantScheduledRun run) + { + } + } diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/processes/ScheduledTaskProcess.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/processes/ScheduledTaskProcess.java new file mode 100644 index 0000000..3dbf332 --- /dev/null +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/processes/ScheduledTaskProcess.java @@ -0,0 +1,241 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2026. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.assistant.processes; + + +import java.io.Serializable; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import com.kingsrook.qbits.assistant.AssistantQBitConfig; +import com.kingsrook.qbits.assistant.AssistantRuntime; +import com.kingsrook.qbits.assistant.AssistantSecurity; +import com.kingsrook.qbits.assistant.metadata.AssistantMetaDataBuilder; +import com.kingsrook.qbits.assistant.model.AssistantScheduledTask; +import com.kingsrook.qbits.assistant.scheduled.ScheduledTaskRunner; +import com.kingsrook.qbits.assistant.scheduled.ScheduledTasks; +import com.kingsrook.qqq.backend.core.actions.processes.BackendStep; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface; +import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput; +import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput; +import com.kingsrook.qqq.backend.core.model.actions.processes.Status; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference; +import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon; +import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.scheduleing.QScheduleMetaData; +import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractLoadStep; +import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractTransformStep; +import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ExtractViaQueryStep; +import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ProcessSummaryProviderInterface; +import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess; + + +/******************************************************************************* + ** Admin record actions and the scheduler's bookkeeping step. No nav processes. + *******************************************************************************/ +public final class ScheduledTaskProcess +{ + public static final String RUNNER = "assistantScheduledTasksRunner"; + + + + /******************************************************************************* + ** Utility class. + *******************************************************************************/ + private ScheduledTaskProcess() + { + } + + + + /******************************************************************************* + ** QQQ supplies the scheduler session. No session is created here. + *******************************************************************************/ + public static List processes(AssistantQBitConfig config) + { + List result = new ArrayList<>(); + QProcessMetaData runner = new QProcessMetaData().withName(RUNNER).withLabel("Run due assistant tasks") + .withPermissionRules(AssistantMetaDataBuilder.adminRules()) + .withStep(new QBackendStepMetaData().withName("runDue").withCode(new QCodeReference(Runner.class))); + if(config.getScheduledTasksSchedulerName() != null && !config.getScheduledTasksSchedulerName().isBlank()) + { + runner.withSchedule(new QScheduleMetaData().withSchedulerName(config.getScheduledTasksSchedulerName()).withRepeatSeconds(60)); + } + result.add(runner); + for(String action : List.of("Approve", "Revoke", "RunNow")) + { + String label = "Approve".equals(action) ? "Approve write mode" : "Revoke".equals(action) ? "Revoke write mode" : "Run now"; + Map values = new HashMap<>(); + values.put(StreamedETLWithFrontendProcess.FIELD_SOURCE_TABLE, AssistantScheduledTask.TABLE_NAME); + values.put(StreamedETLWithFrontendProcess.FIELD_DESTINATION_TABLE, AssistantScheduledTask.TABLE_NAME); + values.put(StreamedETLWithFrontendProcess.FIELD_SUPPORTS_FULL_VALIDATION, true); + values.put(StreamedETLWithFrontendProcess.FIELD_DO_FULL_VALIDATION, true); + values.put(StreamedETLWithFrontendProcess.FIELD_PREVIEW_MESSAGE, label + " for the selected tasks."); + values.put("scheduledAction", action); + QProcessMetaData process = StreamedETLWithFrontendProcess.defineProcessMetaData(ExtractViaQueryStep.class, Review.class, Result.class, values); + process.setName("assistantScheduledTask" + action); + process.setLabel(label); + process.setIcon(new QIcon("schedule")); + process.setTableName(AssistantScheduledTask.TABLE_NAME); + process.setPermissionRules(AssistantMetaDataBuilder.adminRules()); + result.add(process); + } + return (result); + } + + + + /******************************************************************************* + ** Actions must enforce permission even when called outside the UI executor. + *******************************************************************************/ + private static AssistantRuntime admin() throws QException + { + if(QContext.getQSession() == null || !QContext.getQSession().hasPermission(AssistantQBitConfig.ADMIN_PERMISSION)) + { + throw (new QException("assistant.admin is required for scheduled task actions.")); + } + return (AssistantRuntime.of(QContext.getQInstance())); + } + + + + /******************************************************************************* + ** Background scheduler step uses only the context supplied by QQQ. + *******************************************************************************/ + public static class Runner implements BackendStep + { + /******************************************************************************* + ** Dispatch the due page once. + *******************************************************************************/ + @Override + public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException + { + output.addValue("runs", new ScheduledTaskRunner(AssistantRuntime.of(QContext.getQInstance())).runDue()); + } + } + + + + /******************************************************************************* + ** Review binds the displayed prompt hash and user to the exact settings. + *******************************************************************************/ + public static class Review extends AbstractTransformStep implements ProcessSummaryProviderInterface + { + private final ArrayList lines = new ArrayList<>(); + + + + /******************************************************************************* + ** No login or model call on Review. + *******************************************************************************/ + @Override + public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException + { + admin(); + for(QRecord record : input.getRecords()) + { + AssistantScheduledTask task = new AssistantScheduledTask(record); + record.setValue("reviewedHash", ScheduledTasks.approvalHash(task)); + lines.add(TestConnectionProcess.single(Status.INFO, "Run as " + task.getRunAsUserId() + "; prompt SHA-256 " + ScheduledTasks.hash(task.getPrompt()) + + ". Write mode permits policy-allowed create, update and delete proposals; writes wait for this user's panel confirmation.", task.getId())); + output.getRecords().add(record); + } + } + + + + /******************************************************************************* + ** Standard streamed ETL Review lines. + *******************************************************************************/ + @Override + public ArrayList getProcessSummary(RunBackendStepOutput output, boolean isForResultScreen) + { + return (lines); + } + } + + + + /******************************************************************************* + ** Result rereads tasks; a change since Review requires a fresh review. + *******************************************************************************/ + public static class Result extends AbstractLoadStep implements ProcessSummaryProviderInterface + { + private final ArrayList lines = new ArrayList<>(); + + + + /******************************************************************************* + ** Approval records the actual administrator's session user id. + *******************************************************************************/ + @Override + public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException + { + AssistantRuntime runtime = admin(); + String action = input.getValueString("scheduledAction"); + for(QRecord record : input.getRecords()) + { + AssistantScheduledTask task = runtime.getStore().scheduledTask(record.getValueInteger("id")); + if(task == null || !ScheduledTasks.approvalHash(task).equals(record.getValueString("reviewedHash"))) + { + throw (new QException("The task changed after Review. Review it again.")); + } + String summary; + if("RunNow".equals(action)) + { + var run = new ScheduledTaskRunner(runtime).run(task, true); + summary = run == null ? "Task already claimed." : "Run " + run.getId() + ": " + run.getStatus(); + } + else if("Approve".equals(action) || "Revoke".equals(action)) + { + Boolean approve = "Approve".equals(action); + runtime.getStore().scheduledUpdate(new QRecord().withValue("id", task.getId()) + .withValue("writeModeApprovedBy", approve ? AssistantSecurity.userId(QContext.getQSession()) : null) + .withValue("writeModeApprovedAt", approve ? runtime.getClock().instant() : null) + .withValue("writeModeApprovalHash", approve ? ScheduledTasks.approvalHash(task) : null)); + summary = approve ? "Write mode approved." : "Write mode revoked."; + } + else + { + throw (new QException("Unknown scheduled task action.")); + } + lines.add(TestConnectionProcess.single(Status.INFO, summary, task.getId())); + output.getRecords().add(record); + } + } + + + + /******************************************************************************* + ** Standard streamed ETL Result lines. + *******************************************************************************/ + @Override + public ArrayList getProcessSummary(RunBackendStepOutput output, boolean isForResultScreen) + { + return (lines); + } + } +} diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/rag/EmbeddingService.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/rag/EmbeddingService.java index 8aed0d7..7f98e75 100644 --- a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/rag/EmbeddingService.java +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/rag/EmbeddingService.java @@ -35,6 +35,7 @@ import com.kingsrook.qbits.assistant.provider.EmbeddingRequest; import com.kingsrook.qbits.assistant.provider.EmbeddingResponse; import com.kingsrook.qbits.assistant.provider.ModelProviderException; import com.kingsrook.qbits.assistant.provider.ModelResponse; +import com.kingsrook.qbits.assistant.scheduled.ScheduledTurn; import com.kingsrook.qqq.backend.core.context.QContext; import com.kingsrook.qqq.backend.core.exceptions.QException; @@ -106,6 +107,7 @@ public class EmbeddingService private List batch(Integer modelId, List texts, DataClassification classification, String tables, Integer conversationId, Integer turnId) throws QException { String userId = AssistantSecurity.userId(QContext.getQSession()); + ScheduledTurn.checkCost(runtime, conversationId, turnId); ProviderService.Active active = resolve(modelId, classification, tables, conversationId, turnId); PolicyService.EffectivePolicy policy = runtime.getPolicyService().current(true); BudgetService.Decision budget = runtime.getBudgetService().evaluate(userId, active, classification == DataClassification.PHI, false); diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/scheduled/ScheduledAlerts.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/scheduled/ScheduledAlerts.java new file mode 100644 index 0000000..08d18a6 --- /dev/null +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/scheduled/ScheduledAlerts.java @@ -0,0 +1,72 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2026. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.assistant.scheduled; + + +import com.kingsrook.qbits.assistant.model.AssistantScheduledRun; +import com.kingsrook.qqq.backend.core.logging.QLogger; +import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair; + + +/******************************************************************************* + ** Optional Alerts notification SPI bridge. The audience is exactly one user; + ** notification bodies contain only status and a link, never answer text. + *******************************************************************************/ +public class ScheduledAlerts +{ + private static final QLogger LOG = QLogger.getLogger(ScheduledAlerts.class); + + + + /******************************************************************************* + ** Reflection keeps the optional notification implementation out of the ABI. + *******************************************************************************/ + public void raise(AssistantScheduledRun run) + { + LOG.info("Scheduled assistant result", logPair("runId", run.getId()), logPair("status", run.getStatus())); + try + { + String prefix = "com.kingsrook.qbits.alerts.spi."; + Class type = Class.forName(prefix + "NotificationSignal"); + Object signal = type.getConstructor(String.class).newInstance("Scheduled assistant task finished"); + type.getMethod("withSourceName", String.class).invoke(signal, "assistant"); + type.getMethod("withSourceKey", String.class).invoke(signal, "assistant/scheduled/run/" + run.getId()); + type.getMethod("withBody", String.class).invoke(signal, "Status: " + run.getStatus() + ". See conversation."); + Class audienceType = Class.forName(prefix + "NotificationAudience"); + Object audience = audienceType.getMethod("user", String.class).invoke(null, run.getUserId()); + type.getMethod("withAudience", audienceType).invoke(signal, audience); + if(run.getConversationId() != null) + { + Class linkType = Class.forName(prefix + "AlertLink"); + Object link = linkType.getMethod("toPath", String.class, String.class).invoke(null, + "/app/assistantAdmin/assistant_conversation/" + run.getConversationId(), "See conversation"); + type.getMethod("withLink", linkType).invoke(signal, link); + } + Object runtime = Class.forName("com.kingsrook.qbits.alerts.AlertsRuntime").getMethod("get").invoke(null); + Object sink = runtime.getClass().getMethod("sink").invoke(runtime); + sink.getClass().getMethod("notify", type).invoke(sink, signal); + } + catch(Exception | LinkageError e) + { + LOG.info("Scheduled assistant notification unavailable", logPair("runId", run.getId())); + } + } +} diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/scheduled/ScheduledLogin.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/scheduled/ScheduledLogin.java new file mode 100644 index 0000000..b1d58ea --- /dev/null +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/scheduled/ScheduledLogin.java @@ -0,0 +1,119 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2026. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.assistant.scheduled; + + +import java.net.HttpCookie; +import java.net.URI; +import java.net.http.HttpClient; +import java.net.http.HttpRequest; +import java.net.http.HttpResponse; +import java.nio.charset.StandardCharsets; +import java.time.Duration; +import java.util.Map; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.kingsrook.qbits.assistant.AssistantRuntime; +import com.kingsrook.qbits.assistant.AssistantSecurity; +import com.kingsrook.qbits.assistant.model.AssistantScheduledTask; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType; +import com.kingsrook.qqq.backend.core.model.session.QSession; +import com.kingsrook.qqq.backend.core.modules.authentication.QAuthenticationModuleDispatcher; + + +/******************************************************************************* + ** Log in exactly as a UI client, then resolve the returned session UUID with + ** the host's authentication module. Never authenticate directly with a token + ** in process, create an automated session, or manufacture session permissions. + *******************************************************************************/ +public class ScheduledLogin +{ + /******************************************************************************* + ** No response body or transport exception can leak an access token. + *******************************************************************************/ + public QSession login(AssistantRuntime runtime, AssistantScheduledTask task) throws QException + { + String token = ScheduledTasks.credential(runtime.getConfig(), task); + String base = runtime.uiApiBaseUrl(); + if(base == null || base.isBlank()) + { + throw (new QException("Configure uiApiBaseUrl before scheduling assistant tasks.")); + } + String uuid; + try + { + HttpRequest request = HttpRequest.newBuilder(URI.create(base.replaceAll("/+$", "") + "/qqq/v1/manageSession")) + .timeout(Duration.ofSeconds(30)).header("Content-Type", "application/json") + .header("X-QQQ-Client", "assistant/scheduled/" + task.getId()) + .POST(HttpRequest.BodyPublishers.ofString(new ObjectMapper().writeValueAsString(Map.of("accessToken", token)), StandardCharsets.UTF_8)).build(); + HttpResponse response = HttpClient.newBuilder().connectTimeout(Duration.ofSeconds(5)).build() + .send(request, HttpResponse.BodyHandlers.ofString(StandardCharsets.UTF_8)); + if(response.statusCode() != 200) + { + throw (new QException("Scheduled login manageSession answered HTTP " + response.statusCode() + ".")); + } + uuid = response.headers().allValues("Set-Cookie").stream().flatMap(value -> HttpCookie.parse(value).stream()) + .filter(cookie -> "sessionUUID".equals(cookie.getName())).map(HttpCookie::getValue).findFirst().orElse(""); + if(uuid.isBlank()) + { + uuid = new ObjectMapper().readTree(response.body()).path("uuid").asText(); + } + if(uuid.isBlank()) + { + throw (new QException("Scheduled login returned no session UUID.")); + } + } + catch(QException e) + { + throw (e); + } + catch(InterruptedException e) + { + Thread.currentThread().interrupt(); + throw (new QException("Scheduled login interrupted.")); + } + catch(Exception e) + { + throw (new QException("Scheduled login failed.")); + } + QSession session; + try + { + var instance = runtime.getQInstance(); + String key = QAuthenticationType.MOCK.equals(instance.getAuthentication().getType()) ? "sessionId" : "sessionUUID"; + session = new QAuthenticationModuleDispatcher().getQModule(instance.getAuthentication()).createSession(instance, Map.of(key, uuid)); + } + catch(Exception e) + { + throw (new QException("Scheduled login session could not be resolved.")); + } + if(!task.getRunAsUserId().equals(AssistantSecurity.userId(session))) + { + throw (new QException("The credential belongs to another user.")); + } + if(!uuid.equals(session.getUuid())) + { + throw (new QException("The authentication module did not preserve the logged-in session UUID.")); + } + AssistantSecurity.applySessionKeys(session, runtime.getConfig()); + return (session); + } +} diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/scheduled/ScheduledTaskRunner.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/scheduled/ScheduledTaskRunner.java new file mode 100644 index 0000000..c840e2b --- /dev/null +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/scheduled/ScheduledTaskRunner.java @@ -0,0 +1,218 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2026. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.assistant.scheduled; + + +import java.time.Instant; +import java.util.Map; +import com.kingsrook.qbits.assistant.AssistantRuntime; +import com.kingsrook.qbits.assistant.api.AssistantApi; +import com.kingsrook.qbits.assistant.api.AssistantApiException; +import com.kingsrook.qbits.assistant.engine.MessageParts; +import com.kingsrook.qbits.assistant.engine.ProviderService; +import com.kingsrook.qbits.assistant.engine.RunningTurn; +import com.kingsrook.qbits.assistant.model.AssistantConversation; +import com.kingsrook.qbits.assistant.model.AssistantPendingAction; +import com.kingsrook.qbits.assistant.model.AssistantScheduledRun; +import com.kingsrook.qbits.assistant.model.AssistantScheduledTask; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.session.QSession; + + +/******************************************************************************* + ** Scheduler bookkeeping uses its supplied QContext. Each actual turn switches + ** to the designated user's freshly authenticated UI session, restoring context + ** afterwards. No user data is fetched by the scheduler or the store. + *******************************************************************************/ +public class ScheduledTaskRunner +{ + private final AssistantRuntime runtime; + private final ScheduledLogin login; + private final ScheduledAlerts alerts; + + + + /******************************************************************************* + ** Default production collaborators. + *******************************************************************************/ + public ScheduledTaskRunner(AssistantRuntime runtime) + { + this(runtime, new ScheduledLogin(), new ScheduledAlerts()); + } + + + + /******************************************************************************* + ** Notification seam for hosts and tests; authentication stays explicit. + *******************************************************************************/ + public ScheduledTaskRunner(AssistantRuntime runtime, ScheduledLogin login, ScheduledAlerts alerts) + { + this.runtime = runtime; + this.login = login; + this.alerts = alerts; + } + + + + /******************************************************************************* + ** One bounded page; each slot is advanced before any authentication/model. + *******************************************************************************/ + public Integer runDue() throws QException + { + Instant now = runtime.getClock().instant(); + Integer count = 0; + for(AssistantScheduledTask task : runtime.getStore().query(AssistantScheduledTask.TABLE_NAME, new QQueryFilter() + .withCriteria("enabled", QCriteriaOperator.EQUALS, true).withCriteria("nextRunAt", QCriteriaOperator.LESS_THAN_OR_EQUALS, now) + .withOrderBy(new QFilterOrderBy("nextRunAt", true)).withLimit(100), AssistantScheduledTask.class)) + { + if(run(task, false) != null) + { + count++; + } + } + return (count); + } + + + + /******************************************************************************* + ** Claim, authenticate, start the normal API turn, and record its result. + *******************************************************************************/ + public AssistantScheduledRun run(AssistantScheduledTask candidate, Boolean manual) throws QException + { + var store = runtime.getStore(); + var claim = store.claimScheduled(candidate, runtime.getClock().instant(), manual); + if(claim == null) + { + return (null); + } + AssistantScheduledRun run = claim.run(); + AssistantScheduledTask task = claim.task(); + var original = QContext.capture(); + QSession session = null; + Boolean attached = false; + try + { + session = login.login(runtime, task); + QContext.init(runtime.getQInstance(), session); + AssistantApi api = new AssistantApi(runtime, session); + Integer conversationId = ((Number) api.createConversation(Map.of("title", task.getName())).get("id")).intValue(); + run.setConversationId(conversationId); + store.updateFields(AssistantConversation.TABLE_NAME, new QRecord().withValue("id", conversationId) + .withValue("scheduledTaskId", task.getId()).withValue("channel", "assistant/scheduled")); + runtime.getScheduledTurn().attach(conversationId, session.getUuid(), task); + attached = true; + RunningTurn running = api.startTurn(conversationId, Map.of("text", task.getPrompt())); + run.setTurnId(running.getTurnId()); + while(runtime.getRunningTurn(running.getTurnId()) != null) + { + Thread.sleep(20); + } + var turn = store.getTurn(task.getRunAsUserId(), run.getTurnId()); + if(turn == null || "RUNNING".equals(turn.getStatus())) + { + throw (new QException("The scheduled turn did not persist a terminal result.")); + } + run.setCostMicros(ScheduledTurn.cost(runtime, turn.getId())); + run.setStatus("COMPLETE".equals(turn.getStatus()) ? "OK" : blocked(turn.getErrorCode()) ? "BLOCKED" : "FAILED"); + run.setError("OK".equals(run.getStatus()) ? null : "Turn stopped: " + (turn.getErrorCode() == null ? turn.getStatus() : turn.getErrorCode())); + String summary = MessageParts.fromJson(store.getMessage(task.getRunAsUserId(), turn.getAssistantMessageId()).getPartsJson()).text(); + Boolean pending = !store.query(AssistantPendingAction.TABLE_NAME, new QQueryFilter().withCriteria("turnId", QCriteriaOperator.EQUALS, turn.getId()) + .withCriteria("userId", QCriteriaOperator.EQUALS, task.getRunAsUserId()).withCriteria("status", QCriteriaOperator.EQUALS, "PENDING").withLimit(1), AssistantPendingAction.class).isEmpty(); + run.setSummary(Boolean.TRUE.equals(runtime.getConfig().getHipaaMode()) ? "see conversation" + : truncate((pending ? "Waiting for your confirmation in the panel. " : "") + summary, 2000)); + } + catch(AssistantApiException e) + { + run.withStatus(blocked(e.getCode()) ? "BLOCKED" : "FAILED").withError("Turn could not start: " + e.getCode()); + } + catch(InterruptedException e) + { + if(run.getTurnId() != null && runtime.getRunningTurn(run.getTurnId()) != null) + { + runtime.getRunningTurn(run.getTurnId()).cancel("SCHEDULER_INTERRUPTED"); + } + Thread.currentThread().interrupt(); + run.withStatus("FAILED").withError("Scheduled runner interrupted."); + } + catch(QException e) + { + if(e instanceof ProviderService.ProviderSetupException && blocked(e.getMessage())) + { + run.withStatus("BLOCKED").withError("Provider blocked by PHI/HIPAA policy."); + } + else + { + run.withStatus("FAILED").withError(run.getConversationId() == null ? truncate(e.getMessage(), 500) : "Scheduled turn failed; see conversation."); + } + } + catch(RuntimeException e) + { + run.withStatus("FAILED").withError("Scheduled task failed."); + } + finally + { + if(attached && (run.getTurnId() == null || runtime.getRunningTurn(run.getTurnId()) == null)) + { + runtime.getScheduledTurn().detach(run.getConversationId(), session.getUuid()); + } + QContext.init(original); + } + run.setEndedAt(runtime.getClock().instant()); + if(Boolean.TRUE.equals(runtime.getConfig().getHipaaMode())) + { + run.setSummary("see conversation"); + } + store.update(run); + store.scheduledUpdate(new QRecord().withValue("id", task.getId()).withValue("lastRunAt", run.getStartedAt()) + .withValue("lastRunStatus", run.getStatus()).withValue("lastRunError", run.getError())); + if(!"RECORD".equals(task.getOutputMode())) + { + alerts.raise(run); + } + return (run); + } + + + + /******************************************************************************* + ** Policy denials are distinct from operational failures. + *******************************************************************************/ + private static Boolean blocked(String code) + { + return (code != null && (code.contains("BUDGET") || code.contains("PHI") || code.contains("HIPAA") || code.contains("KILL_SWITCH"))); + } + + + + /******************************************************************************* + ** Bound stored summaries and sanitized errors. + *******************************************************************************/ + private static String truncate(String value, Integer limit) + { + return (value == null || value.length() <= limit ? value : value.substring(0, limit)); + } +} diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/scheduled/ScheduledTasks.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/scheduled/ScheduledTasks.java new file mode 100644 index 0000000..888b1ad --- /dev/null +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/scheduled/ScheduledTasks.java @@ -0,0 +1,164 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2026. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.assistant.scheduled; + + +import java.nio.charset.StandardCharsets; +import java.security.MessageDigest; +import java.time.Instant; +import java.time.ZoneId; +import java.util.Date; +import java.util.HexFormat; +import java.util.List; +import java.util.Objects; +import java.util.TimeZone; +import com.kingsrook.qbits.assistant.AssistantQBitConfig; +import com.kingsrook.qbits.assistant.model.AssistantScheduledTask; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.utils.JsonUtils; +import org.quartz.CronExpression; + + +/******************************************************************************* + ** Schedule and credential validation shared by saves, reviews and execution. + *******************************************************************************/ +public final class ScheduledTasks +{ + public static final List APPROVAL_FIELDS = List.of("prompt", "runAsUserId", "credentialEnvName", "profileId", "writeModeRequested"); + + + + /******************************************************************************* + ** Utility class. + *******************************************************************************/ + private ScheduledTasks() + { + } + + + + /******************************************************************************* + ** Quartz computes wall-clock schedules in a validated IANA timezone. + *******************************************************************************/ + public static Instant next(AssistantScheduledTask task, Instant after) throws QException + { + try + { + CronExpression cron = new CronExpression(task.getCronExpression()); + cron.setTimeZone(TimeZone.getTimeZone(ZoneId.of(task.getTimezone() == null ? "UTC" : task.getTimezone()))); + Date next = cron.getNextValidTimeAfter(Date.from(after)); + return (next == null ? null : next.toInstant()); + } + catch(Exception e) + { + throw (new QException("Invalid Quartz cron expression or timezone.")); + } + } + + + + /******************************************************************************* + ** Refuse a pasted token or an unrelated host secret without echoing it. + *******************************************************************************/ + public static void checkReference(AssistantQBitConfig config, String name) throws QException + { + if(name == null || !name.matches("[A-Z_][A-Z0-9_]*")) + { + throw (new QException("The scheduled credential must be an environment variable name.")); + } + if(config.getAllowedScheduledCredentialEnvNames() == null || !config.getAllowedScheduledCredentialEnvNames().contains(name)) + { + throw (new QException("The scheduled credential environment variable is not allowed.")); + } + } + + + + /******************************************************************************* + ** Resolve only after the reference passes the dedicated allowlist. + *******************************************************************************/ + public static String credential(AssistantQBitConfig config, AssistantScheduledTask task) throws QException + { + checkReference(config, task.getCredentialEnvName()); + String token = config.getSecretResolver().apply(task.getCredentialEnvName()); + if(token == null || token.isBlank()) + { + throw (new QException("The scheduled credential environment variable is not set.")); + } + return (token); + } + + + + /******************************************************************************* + ** Presence only; no token characters are exposed in record screens. + *******************************************************************************/ + public static String credentialStatus(AssistantQBitConfig config, AssistantScheduledTask task) + { + try + { + credential(config, task); + return ("Set (from " + task.getCredentialEnvName() + ")"); + } + catch(QException e) + { + return (e.getMessage()); + } + } + + + + /******************************************************************************* + ** Bind approval to all security-relevant settings, including null values. + *******************************************************************************/ + public static String approvalHash(AssistantScheduledTask task) throws QException + { + return (hash(JsonUtils.toJson(APPROVAL_FIELDS.stream().map(name -> task.toQRecord().getValue(name)).toList()))); + } + + + + /******************************************************************************* + ** SHA-256 displayed on Review and checked again on Result. + *******************************************************************************/ + public static String hash(String text) throws QException + { + try + { + return (HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(text.getBytes(StandardCharsets.UTF_8)))); + } + catch(Exception e) + { + throw (new QException("Could not hash the scheduled task.")); + } + } + + + + /******************************************************************************* + ** An approval for an older task definition never authorizes writes. + *******************************************************************************/ + public static Boolean writesApproved(AssistantScheduledTask task) throws QException + { + return (Boolean.TRUE.equals(task.getWriteModeRequested()) && task.getWriteModeApprovedAt() != null + && task.getWriteModeApprovedBy() != null && Objects.equals(task.getWriteModeApprovalHash(), approvalHash(task))); + } +} diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/scheduled/ScheduledTurn.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/scheduled/ScheduledTurn.java new file mode 100644 index 0000000..6e82463 --- /dev/null +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/scheduled/ScheduledTurn.java @@ -0,0 +1,173 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2026. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.assistant.scheduled; + + +import java.math.BigDecimal; +import java.util.Map; +import java.util.Set; +import java.util.concurrent.ConcurrentHashMap; +import java.util.stream.Collectors; +import com.kingsrook.qbits.assistant.AssistantRuntime; +import com.kingsrook.qbits.assistant.engine.BudgetService; +import com.kingsrook.qbits.assistant.engine.PolicyService; +import com.kingsrook.qbits.assistant.model.AssistantScheduledTask; +import com.kingsrook.qbits.assistant.model.AssistantUsage; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter; + + +/******************************************************************************* + ** Server-owned per-conversation execution settings. They never come from a + ** panel request or prompt. Scoped to one runtime and removed after completion. + *******************************************************************************/ +public class ScheduledTurn +{ + private final Map tasks = new ConcurrentHashMap<>(); + private final Map channels = new ConcurrentHashMap<>(); + + + + /******************************************************************************* + ** Attach a snapshot before the unchanged API starts its normal turn. + *******************************************************************************/ + public void attach(Integer conversationId, String uuid, AssistantScheduledTask task) throws QException + { + if(channels.putIfAbsent(uuid, "assistant/scheduled/" + task.getId()) != null) + { + throw (new QException("This login session already has a scheduled task in progress.")); + } + tasks.put(conversationId, task); + } + + + + /******************************************************************************* + ** Remove references to the completed run's context. + *******************************************************************************/ + public void detach(Integer conversationId, String uuid) + { + if(conversationId != null) + { + AssistantScheduledTask task = tasks.remove(conversationId); + if(task != null) + { + channels.remove(uuid, "assistant/scheduled/" + task.getId()); + } + } + } + + + + /******************************************************************************* + ** Snapshot of task settings for this conversation. + *******************************************************************************/ + public AssistantScheduledTask task(Integer conversationId) + { + return (tasks.get(conversationId)); + } + + + + /******************************************************************************* + ** Client attribution also covers metadata and knowledge retrieval calls. + *******************************************************************************/ + public String channel(String uuid) + { + return (uuid == null ? null : channels.get(uuid)); + } + + + + /******************************************************************************* + ** Narrow the normal policy. Scheduled writes always require confirmation. + *******************************************************************************/ + public static PolicyService.ToolPolicy tools(AssistantRuntime runtime, AssistantScheduledTask task, PolicyService.ToolPolicy policy) throws QException + { + Set allowed = runtime.getContextBuilder().toolNames(policy).stream() + .filter(name -> !Set.of("create_record", "update_record", "delete_record").contains(name) || approved(task)) + .collect(Collectors.toSet()); + if(allowed.isEmpty()) + { + allowed = Set.of("__no_tools__"); + } + return (new PolicyService.ToolPolicy(true, allowed, policy.maxRowsPerCall(), policy.redactPii())); + } + + + + /******************************************************************************* + ** Approval failure closes write mode. + *******************************************************************************/ + private static Boolean approved(AssistantScheduledTask task) + { + try + { + return (ScheduledTasks.writesApproved(task)); + } + catch(QException e) + { + return (false); + } + } + + + + /******************************************************************************* + ** Extra STOP cap at dispatch boundaries; a completed call can cross it. + *******************************************************************************/ + public static void checkCost(AssistantScheduledTask task, Long costMicros) throws BudgetService.Blocked + { + if(task != null && task.getMaxCostUsdPerRun() != null + && BigDecimal.valueOf(costMicros, 6).compareTo(task.getMaxCostUsdPerRun()) >= 0) + { + throw (new BudgetService.Blocked("The scheduled task reached its per-run cost cap.")); + } + } + + + + /******************************************************************************* + ** Include chat, classifier, failed attempts and knowledge-query embeddings. + *******************************************************************************/ + public static Long cost(AssistantRuntime runtime, Integer turnId) throws QException + { + return (runtime.getStore().query(AssistantUsage.TABLE_NAME, + new QQueryFilter().withCriteria("turnId", + QCriteriaOperator.EQUALS, turnId), + AssistantUsage.class).stream().mapToLong(row -> row.getCostMicros() == null ? 0 : row.getCostMicros()).sum()); + } + + + + /******************************************************************************* + ** Check persisted accounting before each embedding and model dispatch. + *******************************************************************************/ + public static void checkCost(AssistantRuntime runtime, Integer conversationId, Integer turnId) throws QException + { + AssistantScheduledTask task = conversationId == null ? null : runtime.getScheduledTurn().task(conversationId); + if(task != null) + { + checkCost(task, cost(runtime, turnId)); + } + } +} diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/uiapi/LoopbackUiApiGateway.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/uiapi/LoopbackUiApiGateway.java index 435b353..5a0e71d 100644 --- a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/uiapi/LoopbackUiApiGateway.java +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/uiapi/LoopbackUiApiGateway.java @@ -165,6 +165,35 @@ public class LoopbackUiApiGateway implements UiApiGateway + /******************************************************************************* + ** Add server-owned client attribution for a scheduled turn. All operation, + ** session, encoding and transport rules remain those of this gateway. + *******************************************************************************/ + public UiApiResult invokeWithClient(UiApiCall call, String clientChannel) throws QException + { + if(call == null || !isInvocable(call.getMethod(), call.getPathTemplate()) || !StringUtils.hasContent(call.getSessionUUID())) + { + throw (new QException("A scheduled UI API call requires a spec'd operation and a session.")); + } + try + { + HttpRequest request = HttpRequest.newBuilder(buildRequest(call), (name, value) -> true).setHeader("X-QQQ-Client", clientChannel).build(); + HttpResponse response = httpClient.send(request, HttpResponse.BodyHandlers.ofString(StandardCharsets.UTF_8)); + return (new UiApiResult(response.statusCode(), response.body())); + } + catch(InterruptedException e) + { + Thread.currentThread().interrupt(); + throw (new QException("Scheduled UI API request interrupted.")); + } + catch(IOException e) + { + throw (new QException("Scheduled UI API request failed.")); + } + } + + + /******************************************************************************* ** The HTTP request: path parameters filled and encoded, query string, JSON ** body, the session cookie and the marker headers. diff --git a/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/AssistantQBitProducerTest.java b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/AssistantQBitProducerTest.java index 38a5c19..df30caf 100644 --- a/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/AssistantQBitProducerTest.java +++ b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/AssistantQBitProducerTest.java @@ -17,7 +17,10 @@ package com.kingsrook.qbits.assistant; +import java.util.ArrayList; +import java.util.List; import com.kingsrook.qbits.assistant.metadata.AssistantAdminPermissionChecker; +import com.kingsrook.qbits.assistant.metadata.AssistantInstanceMetaData; import com.kingsrook.qbits.assistant.metadata.AssistantMetaDataBuilder; import com.kingsrook.qbits.assistant.model.AssistantConversation; import com.kingsrook.qbits.assistant.model.AssistantPolicy; @@ -26,8 +29,8 @@ import com.kingsrook.qbits.assistant.testing.AssistantTestFixture; import com.kingsrook.qbits.assistant.testing.BaseAssistantTest; import com.kingsrook.qqq.backend.core.actions.permissions.PermissionCheckResult; import com.kingsrook.qqq.backend.core.actions.permissions.PermissionsHelper; -import com.kingsrook.qqq.backend.core.alerts.AlertsInstanceMetaData; import com.kingsrook.qqq.backend.core.alerts.AlertSeverity; +import com.kingsrook.qqq.backend.core.alerts.AlertsInstanceMetaData; import com.kingsrook.qqq.backend.core.alerts.QAlertTypeMetaData; import com.kingsrook.qqq.backend.core.context.QContext; import com.kingsrook.qqq.backend.core.exceptions.QException; @@ -44,9 +47,8 @@ import com.kingsrook.qqq.backend.core.model.metadata.security.QSecurityKeyType; import com.kingsrook.qqq.backend.core.model.metadata.security.RecordSecurityLock; import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData; import com.kingsrook.qqq.backend.core.model.session.QSession; +import com.kingsrook.qqq.backend.core.model.session.QUser; import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryBackendModule; -import java.util.ArrayList; -import java.util.List; import org.junit.jupiter.api.Test; import static org.assertj.core.api.Assertions.assertThat; import static org.assertj.core.api.Assertions.assertThatThrownBy; @@ -88,7 +90,7 @@ class AssistantQBitProducerTest extends BaseAssistantTest // the nav: the home plus the main tables; processes are actions // ////////////////////////////////////////////////////////////////////// assertThat(qInstance.getApp(AssistantMetaDataBuilder.APP_NAME).getChildren()).extracting(c -> c.getName()) - .containsExactly("assistant_rag_source", "assistant_budget", "assistant_provider", "assistant_profile", "assistant_skill", "assistant_data_access", "assistant_conversation", "assistant_model_price", "assistant_policy"); + .containsExactly("assistant_scheduled_task", "assistant_rag_source", "assistant_budget", "assistant_provider", "assistant_profile", "assistant_skill", "assistant_data_access", "assistant_conversation", "assistant_model_price", "assistant_policy"); assertThat(qInstance.getApp(AssistantMetaDataBuilder.APP_NAME).getWidgets()).containsExactly("assistantHome", "assistantAccess"); assertThat(qInstance.getProcess(AssistantMetaDataBuilder.RETENTION_PROCESS_NAME).getTableName()).isEqualTo("assistant_conversation"); assertThat(qInstance.getProcess(AssistantMetaDataBuilder.RETENTION_PROCESS_NAME)).isNotNull(); @@ -96,9 +98,9 @@ class AssistantQBitProducerTest extends BaseAssistantTest assertThat(qInstance.getPossibleValueSource(AssistantConversation.TABLE_NAME)).isNotNull(); assertThat(qInstance.getQBits().values()).anyMatch(q -> "assistant".equals(q.getArtifactId())); - assertThat(com.kingsrook.qbits.assistant.metadata.AssistantInstanceMetaData.of(qInstance)).isNotNull(); - assertThat(com.kingsrook.qbits.assistant.metadata.AssistantInstanceMetaData.of(qInstance).getVersion()).isEqualTo(1); - assertThat(com.kingsrook.qbits.assistant.metadata.AssistantInstanceMetaData.of(qInstance).getBasePath()).isEqualTo("/qqq/v1/assistant"); + assertThat(AssistantInstanceMetaData.of(qInstance)).isNotNull(); + assertThat(AssistantInstanceMetaData.of(qInstance).getVersion()).isEqualTo(1); + assertThat(AssistantInstanceMetaData.of(qInstance).getBasePath()).isEqualTo("/qqq/v1/assistant"); assertThat(config.getRuntime()).isNotNull(); assertThat(AssistantRuntime.of(qInstance)).isSameAs(config.getRuntime()); assertThat(AssistantQBitProducer.getVersion()).isNotBlank(); @@ -192,7 +194,7 @@ class AssistantQBitProducerTest extends BaseAssistantTest assertThat(editor.getSecurityKeyValues(AssistantQBitConfig.DEFAULT_OWNER_KEY_TYPE)).containsExactly(AssistantTestFixture.EDITOR); assertThat(editor.getSecurityKeyValues("assistant_user_id_all_access")).isNullOrEmpty(); - QSession admin = new QSession().withUser(new com.kingsrook.qqq.backend.core.model.session.QUser().withIdReference("boss")); + QSession admin = new QSession().withUser(new QUser().withIdReference("boss")); admin.withPermission(AssistantQBitConfig.ADMIN_PERMISSION); AssistantSecurity.applySessionKeys(admin, config); assertThat(admin.getSecurityKeyValues("assistant_user_id_all_access")).containsExactly(true); @@ -277,7 +279,7 @@ class AssistantQBitProducerTest extends BaseAssistantTest AssistantQBitConfig hostConfig = new AssistantQBitConfig().withBackendName("memory").withOwnerSecurityKeyType("userKey").withRetentionSchedulerName("quartz").withRegisterRoutes(false); hostConfig.setEnabled(false); new AssistantQBitProducer().withQBitConfig(hostConfig).produce(instance); - assertThat(com.kingsrook.qbits.assistant.metadata.AssistantInstanceMetaData.of(instance)).as("not advertised when disabled").isNull(); + assertThat(AssistantInstanceMetaData.of(instance)).as("not advertised when disabled").isNull(); assertThat(instance.getSecurityKeyType("userKey").getAllAccessKeyName()).isEqualTo("userKeyAll"); assertThat(instance.getProcess(AssistantMetaDataBuilder.RETENTION_PROCESS_NAME).getSchedule().getSchedulerName()).isEqualTo("quartz"); hostConfig.getRuntime().close(); diff --git a/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/scheduled/ScheduledTasksTest.java b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/scheduled/ScheduledTasksTest.java new file mode 100644 index 0000000..d33e05c --- /dev/null +++ b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/scheduled/ScheduledTasksTest.java @@ -0,0 +1,557 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2026. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.assistant.scheduled; + + +import java.math.BigDecimal; +import java.time.Clock; +import java.time.Instant; +import java.time.ZoneOffset; +import java.util.ArrayList; +import java.util.Base64; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.Objects; +import java.util.Set; +import java.util.concurrent.Callable; +import java.util.concurrent.CyclicBarrier; +import java.util.concurrent.Executors; +import java.util.stream.Stream; +import com.kingsrook.qbits.assistant.AssistantQBitConfig; +import com.kingsrook.qbits.assistant.AssistantRuntime; +import com.kingsrook.qbits.assistant.AssistantSecurity; +import com.kingsrook.qbits.assistant.classification.DataClassification; +import com.kingsrook.qbits.assistant.model.AssistantDataAccess; +import com.kingsrook.qbits.assistant.model.AssistantPendingAction; +import com.kingsrook.qbits.assistant.model.AssistantPolicy; +import com.kingsrook.qbits.assistant.model.AssistantProfile; +import com.kingsrook.qbits.assistant.model.AssistantProvider; +import com.kingsrook.qbits.assistant.model.AssistantScheduledRun; +import com.kingsrook.qbits.assistant.model.AssistantScheduledTask; +import com.kingsrook.qbits.assistant.model.AssistantToolCall; +import com.kingsrook.qbits.assistant.processes.ScheduledTaskProcess; +import com.kingsrook.qbits.assistant.provider.AssistantModelProvider; +import com.kingsrook.qbits.assistant.provider.AssistantProviderType; +import com.kingsrook.qbits.assistant.provider.ModelProviderException; +import com.kingsrook.qbits.assistant.testing.AssistantTestFixture; +import com.kingsrook.qbits.assistant.testing.BaseAssistantTest; +import com.kingsrook.qbits.assistant.testing.UiApiTestServer; +import com.kingsrook.qbits.assistant.uiapi.UiApiCall; +import com.kingsrook.qqq.backend.core.actions.tables.QueryAction; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.exceptions.QAuthenticationException; +import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput; +import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.QInstance; +import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference; +import com.kingsrook.qqq.backend.core.model.metadata.security.RecordSecurityLock; +import com.kingsrook.qqq.backend.core.model.session.QSession; +import com.kingsrook.qqq.backend.core.modules.authentication.implementations.MockAuthenticationModule; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; + + + +/******************************************************************************* + ** Scheduled runs use real manageSession and UI API requests on the fixture. + *******************************************************************************/ +class ScheduledTasksTest extends BaseAssistantTest +{ + private static final Instant NOW = Instant.parse("2026-03-08T06:55:00Z"); + private final Map secrets = new HashMap<>(); + private final List notifications = new ArrayList<>(); + private AssistantRuntime runtime; + + + + /******************************************************************************* + ** Dedicated environment references; the test resolver contains fake tokens. + *******************************************************************************/ + @Override + protected void customizeConfig(AssistantQBitConfig config) + { + config.withAllowedScheduledCredentialEnvNames(Set.of("SCHEDULED_TOKEN")).withSecretResolver(secrets::get) + .withDataClassification("patient", DataClassification.PHI); + } + + + + /******************************************************************************* + ** Fixed time for due selection, cron, budgets and persisted timestamps. + *******************************************************************************/ + @BeforeEach + void setupSchedule() + { + config.getRuntime().close(); + runtime = new AssistantRuntime(qInstance, config, Clock.fixed(NOW, ZoneOffset.UTC)); + config.setRuntime(runtime); + qInstance.getAuthentication().setModuleCodeReference(new QCodeReference(TokenMock.class)); + secrets.put("SCHEDULED_TOKEN", AssistantTestFixture.EDITOR); + admin(); + } + + + + /******************************************************************************* + ** RC.5 MOCK maps sessionId but ignores accessToken. This test-only module + ** maps login tokens to that same fixture identity, preserving all grants. + *******************************************************************************/ + public static class TokenMock extends MockAuthenticationModule + { + /******************************************************************************* + ** Login and subsequent UUID resolution use identical fixture sessions. + *******************************************************************************/ + @Override + public QSession createSession(QInstance instance, Map context) throws QAuthenticationException + { + Map mapped = new HashMap<>(context); + if(context.get("accessToken") != null) + { + mapped.put("sessionId", context.get("accessToken")); + } + QSession session = super.createSession(instance, mapped); + AssistantSecurity.applySessionKeys(session, AssistantRuntime.of(instance).getConfig()); + return (session); + } + } + + + + /******************************************************************************* + ** Bookkeeping starts in the scheduler/admin context supplied by the host. + *******************************************************************************/ + private void admin() + { + QContext.init(qInstance, AssistantTestFixture.ownerSession(AssistantTestFixture.ADMIN, config)); + } + + + + /******************************************************************************* + ** A due daily task, crossing Chicago's spring daylight-saving transition. + *******************************************************************************/ + private AssistantScheduledTask task() throws Exception + { + AssistantScheduledTask task = runtime.getStore().insert(new AssistantScheduledTask().withName("Morning report").withEnabled(true) + .withPrompt("Summarize today's posts").withRunAsUserId(AssistantTestFixture.EDITOR).withCredentialEnvName("SCHEDULED_TOKEN") + .withCronExpression("0 0 8 * * ?").withTimezone("America/Chicago").withOutputMode("RECORD").withWriteModeRequested(false)); + runtime.getStore().scheduledUpdate(new QRecord().withValue("id", task.getId()).withValue("nextRunAt", NOW.minusSeconds(60))); + return (runtime.getStore().scheduledTask(task.getId())); + } + + + + /******************************************************************************* + ** Capture notification delivery without changing the runner. + *******************************************************************************/ + private ScheduledTaskRunner scheduledRunner() + { + return (new ScheduledTaskRunner(runtime, new ScheduledLogin(), new ScheduledAlerts() + { + /******************************************************************************* + ** Delegate using this instance's configured collaborator. + *******************************************************************************/ + @Override + public void raise(AssistantScheduledRun run) + { + notifications.add(run); + } + })); + } + + + + /******************************************************************************* + ** One due slot, one owned conversation; next wall-clock occurrence uses DST. + *******************************************************************************/ + @Test + void dueRunsOnceAndAdvancesAcrossDst() throws Exception + { + AssistantScheduledTask task = task(); + provider.thenText("Today's report"); + assertThat(scheduledRunner().runDue()).isEqualTo(1); + assertThat(scheduledRunner().runDue()).isZero(); + var runs = runtime.getStore().query(AssistantScheduledRun.TABLE_NAME, new QQueryFilter(), AssistantScheduledRun.class); + assertThat(runs).hasSize(1); + AssistantScheduledRun run = runs.get(0); + assertThat(run.getStatus()).as(run.getError()).isEqualTo("OK"); + assertThat(run.getSummary()).isEqualTo("Today's report"); + assertThat(run.getStartedAt()).isEqualTo(NOW); + var conversation = runtime.getStore().getConversation(AssistantTestFixture.EDITOR, run.getConversationId()); + assertThat(conversation.getUserId()).isEqualTo(AssistantTestFixture.EDITOR); + assertThat(conversation.getChannel()).isEqualTo("assistant/scheduled"); + assertThat(conversation.getScheduledTaskId()).isEqualTo(task.getId()); + assertThat(runtime.getStore().scheduledTask(task.getId()).getNextRunAt()).isEqualTo(Instant.parse("2026-03-08T13:00:00Z")); + assertThat(QContext.getQSession().getUser().getIdReference()).isEqualTo(AssistantTestFixture.ADMIN); + assertThat(notifications).isEmpty(); + } + + + + /******************************************************************************* + ** Every tool HTTP call carries both the designated UUID and client channel. + *******************************************************************************/ + @Test + void designatedUserDenialAndClientHeaders() throws Exception + { + AssistantScheduledTask task = task(); + provider.thenToolUse("query_records", input("table", "analytics_daily_stat", "filter", filter("AND"))).thenText("Access denied"); + AssistantScheduledRun run = scheduledRunner().run(task, false); + assertThat(run.getStatus()).as(run.getError()).isEqualTo("OK"); + var calls = runtime.getStore().query(AssistantToolCall.TABLE_NAME, new QQueryFilter().withCriteria("turnId", QCriteriaOperator.EQUALS, run.getTurnId()), AssistantToolCall.class); + assertThat(calls).hasSize(1); + assertThat(calls.get(0).getHttpStatus()).isEqualTo(403); + assertThat(calls.get(0).getClientChannel()).isEqualTo("assistant/scheduled/" + task.getId()); + assertThat(UiApiTestServer.SCHEDULED_REQUESTS).isNotEmpty().allSatisfy(request -> + { + assertThat(request.get("client")).isEqualTo("assistant/scheduled/" + task.getId()); + assertThat(request.get("session")).isEqualTo(AssistantTestFixture.EDITOR); + }); + QContext.init(qInstance, AssistantTestFixture.ownerSession("editor-other", config)); + assertThat(runtime.getStore().query(AssistantScheduledRun.TABLE_NAME, new QQueryFilter(), AssistantScheduledRun.class)).isEmpty(); + assertThat(runtime.getStore().getConversation("editor-other", run.getConversationId())).isNull(); + } + + + + /******************************************************************************* + ** Credential failures occur before a model call and never expose a token. + *******************************************************************************/ + @Test + void credentialFailures() throws Exception + { + AssistantScheduledTask task = task(); + secrets.clear(); + assertThat(scheduledRunner().run(task, false).getError()).contains("not set"); + secrets.put("SCHEDULED_TOKEN", "analyst-other"); + task = task(); + assertThat(scheduledRunner().run(task, false).getError()).contains("another user"); + secrets.put("SCHEDULED_TOKEN", "Deny"); + task = task(); + assertThat(scheduledRunner().run(task, false).getError()).contains("HTTP"); + task = task(); + config.setAllowedScheduledCredentialEnvNames(Set.of()); + AssistantScheduledRun run = scheduledRunner().run(task, false); + assertThat(run.getStatus()).as(run.getError()).isEqualTo("FAILED"); + assertThat(run.getError()).contains("not allowed"); + assertThat(provider.getRequests()).isEmpty(); + } + + + + /******************************************************************************* + ** Independent runner instances compete for the same durable schedule slot. + *******************************************************************************/ + @Test + void racingRunnersClaimOnlyOnce() throws Exception + { + AssistantScheduledTask task = task(); + provider.thenText("Once"); + CyclicBarrier barrier = new CyclicBarrier(2); + try(var executor = Executors.newVirtualThreadPerTaskExecutor()) + { + var work = (Callable) () -> + { + admin(); + barrier.await(); + return (scheduledRunner().run(task, false)); + }; + var one = executor.submit(work); + var two = executor.submit(work); + assertThat(Stream.of(one.get(), two.get()).filter(Objects::nonNull)).hasSize(1); + } + assertThat(provider.getRequests()).hasSize(1); + } + + + + /******************************************************************************* + ** Per-run caps stop before further dispatch, including a zero-cost allowance. + *******************************************************************************/ + @Test + void costCapStopsRun() throws Exception + { + AssistantScheduledTask task = task(); + runtime.getStore().updateFields(AssistantScheduledTask.TABLE_NAME, new QRecord().withValue("id", task.getId()).withValue("maxCostUsdPerRun", BigDecimal.ZERO)); + AssistantScheduledRun run = scheduledRunner().run(task, false); + assertThat(run.getStatus()).as(run.getError()).isEqualTo("BLOCKED"); + assertThat(run.getError()).contains("BUDGET"); + assertThat(provider.getRequests()).isEmpty(); + task = task(); + runtime.getStore().updateFields(AssistantScheduledTask.TABLE_NAME, new QRecord().withValue("id", task.getId()).withValue("maxCostUsdPerRun", new BigDecimal("0.000001"))); + provider.thenToolUse("count_records", input("table", "page", "filter", filter("AND"))).thenText("Must not run"); + assertThat(scheduledRunner().run(task, false).getStatus()).isEqualTo("BLOCKED"); + assertThat(provider.getRequests()).hasSize(1); + } + + + + /******************************************************************************* + ** Approved writes are pending proposals even when normal auto-confirm is on. + *******************************************************************************/ + @Test + void approvalAndPromptInvalidation() throws Exception + { + AssistantScheduledTask task = task(); + secrets.put("SCHEDULED_TOKEN", AssistantTestFixture.ADMIN); + runtime.getStore().updateFields(AssistantScheduledTask.TABLE_NAME, new QRecord().withValue("id", task.getId()).withValue("runAsUserId", AssistantTestFixture.ADMIN).withValue("writeModeRequested", true)); + runtime.getStore().insert(new AssistantDataAccess().withObjectType("TABLE").withObjectName("page").withMode("READ_WRITE")); + runtime.getDataAccessService().invalidate(); + provider.thenText("Read only"); + assertThat(scheduledRunner().run(task, false).getStatus()).isEqualTo("OK"); + assertThat(provider.getRequests().get(0).getTools()).noneMatch(tool -> tool.getName().equals("create_record")); + task = runtime.getStore().scheduledTask(task.getId()); + action(task, "Approve"); + assertThat(ScheduledTasks.writesApproved(runtime.getStore().scheduledTask(task.getId()))).isTrue(); + provider.thenToolUse("create_record", input("table", "page", "values", Map.of("title", "Scheduled proposal"))).thenText("Please confirm"); + AssistantScheduledRun run = scheduledRunner().run(task, true); + assertThat(run.getStatus()).as(run.getError()).isEqualTo("OK"); + assertThat(run.getSummary()).contains("Waiting for your confirmation"); + assertThat(provider.getRequests().get(1).getTools()).anyMatch(tool -> tool.getName().equals("create_record")); + assertThat(runtime.getStore().query(AssistantPendingAction.TABLE_NAME, new QQueryFilter(), AssistantPendingAction.class)).hasSize(1); + runtime.getStore().updateFields(AssistantScheduledTask.TABLE_NAME, new QRecord().withValue("id", task.getId()).withValue("prompt", "Changed prompt")); + assertThat(runtime.getStore().scheduledTask(task.getId()).getWriteModeApprovedAt()).isNull(); + assertThat(runtime.getStore().scheduledTask(task.getId()).getWriteModeApprovalHash()).isNull(); + action(runtime.getStore().scheduledTask(task.getId()), "Approve"); + action(runtime.getStore().scheduledTask(task.getId()), "Revoke"); + assertThat(runtime.getStore().scheduledTask(task.getId()).getWriteModeApprovedAt()).isNull(); + QContext.init(qInstance, AssistantTestFixture.ownerSession(AssistantTestFixture.EDITOR, config)); + AssistantScheduledTask selected = task; + assertThatThrownBy(() -> action(selected, "Approve")).hasMessageContaining("assistant.admin"); + } + + + + /******************************************************************************* + ** Exercise the same Review/Result steps the record action uses. + *******************************************************************************/ + private void action(AssistantScheduledTask task, String action) throws Exception + { + RunBackendStepInput input = new RunBackendStepInput(); + input.setRecords(List.of(task.toQRecord())); + input.addValue("scheduledAction", action); + RunBackendStepOutput reviewOutput = new RunBackendStepOutput(); + ScheduledTaskProcess.Review review = new ScheduledTaskProcess.Review(); + review.runOnePage(input, reviewOutput); + assertThat(review.getProcessSummary(reviewOutput, false)).isNotEmpty(); + input.setRecords(reviewOutput.getRecords()); + RunBackendStepOutput resultOutput = new RunBackendStepOutput(); + ScheduledTaskProcess.Result result = new ScheduledTaskProcess.Result(); + result.runOnePage(input, resultOutput); + assertThat(result.getProcessSummary(resultOutput, true)).isNotEmpty(); + } + + + + /******************************************************************************* + ** Output modes retain minimal run history and route notifications by owner. + *******************************************************************************/ + @Test + void outputModesAndModelFailure() throws Exception + { + for(String mode : List.of("RECORD", "NOTIFICATION", "BOTH")) + { + AssistantScheduledTask task = task(); + runtime.getStore().updateFields(AssistantScheduledTask.TABLE_NAME, new QRecord().withValue("id", task.getId()).withValue("outputMode", mode)); + provider.thenText("Summary"); + assertThat(scheduledRunner().run(task, false).getStatus()).isEqualTo("OK"); + } + assertThat(notifications).hasSize(2).allSatisfy(run -> assertThat(run.getUserId()).isEqualTo(AssistantTestFixture.EDITOR)); + provider.thenError(ModelProviderException.Kind.OTHER, "Private provider detail"); + AssistantScheduledRun failed = scheduledRunner().run(task(), false); + assertThat(failed.getStatus()).isEqualTo("FAILED"); + assertThat(failed.getError()).doesNotContain("Private provider detail"); + new ScheduledAlerts().raise(failed); + } + + + + /******************************************************************************* + ** Validation rejects malformed schedules and secret-looking references. + *******************************************************************************/ + @Test + void validationAndManualProcess() throws Exception + { + AssistantScheduledTask task = task(); + assertThatThrownBy(() -> ScheduledTasks.next(task.withTimezone("No/SuchZone"), NOW)).hasMessageContaining("timezone"); + assertThatThrownBy(() -> ScheduledTasks.next(task.withTimezone("UTC").withCronExpression("invalid"), NOW)).hasMessageContaining("cron"); + assertThatThrownBy(() -> ScheduledTasks.checkReference(config, "token-SECRET")).hasMessageNotContaining("SECRET"); + assertThatThrownBy(() -> runtime.getStore().updateFields(AssistantScheduledTask.TABLE_NAME, new QRecord().withValue("id", task.getId()).withValue("prompt", " "))).hasMessageContaining("Prompt"); + AssistantScheduledTask saved = runtime.getStore().scheduledTask(task.getId()); + assertThat(saved.getCredentialStatus()).contains("Set").doesNotContain(AssistantTestFixture.EDITOR); + provider.thenText("Manual"); + action(saved, "RunNow"); + config.setScheduledTasksSchedulerName("hostScheduler"); + assertThat(ScheduledTaskProcess.processes(config).get(0).getSchedule().getRepeatSeconds()).isEqualTo(60); + new ScheduledTaskProcess.Runner().run(new RunBackendStepInput(), new RunBackendStepOutput()); + } + + + + /******************************************************************************* + ** The scheduled profile supplies persona text and normal PHI refusal applies. + *******************************************************************************/ + @Test + void selectedProfileAndPhiBlock() throws Exception + { + AssistantProfile profile = runtime.getStore().insert(new AssistantProfile().withName("Scheduled").withDisplayName("Daily helper").withEnabled(true) + .withPersonality("Use a concise daily briefing style.")); + AssistantScheduledTask task = task(); + runtime.getStore().updateFields(AssistantScheduledTask.TABLE_NAME, new QRecord().withValue("id", task.getId()).withValue("profileId", profile.getId())); + provider.thenToolUse("get_record", input("table", "patient", "id", 1)).thenText("Must not reach a noneligible provider"); + AssistantScheduledRun run = scheduledRunner().run(task, false); + assertThat(run.getStatus()).as(run.getError()).isEqualTo("BLOCKED"); + assertThat(provider.getRequests()).hasSize(1); + assertThat(provider.getRequests().get(0).getSystem().get(1).getText()).contains("Daily helper"); + } + + + + /******************************************************************************* + ** HIPAA retains the answer only in encrypted conversation storage. + *******************************************************************************/ + @Test + void hipaaSummaryAndEncryption() throws Exception + { + secrets.put("CONVERSATION_KEY", Base64.getEncoder().encodeToString(new byte[32])); + config.withHipaaMode(true).withConversationEncryptionKeyEnvName("CONVERSATION_KEY"); + config.withProviderType(new AssistantProviderType() + { + /******************************************************************************* + ** Delegate using this instance's configured collaborator. + *******************************************************************************/ + @Override + public String getType() + { + return ("scheduled-test"); + } + + /******************************************************************************* + ** Delegate using this instance's configured collaborator. + *******************************************************************************/ + @Override + public String getLabel() + { + return ("Scheduled test"); + } + + /******************************************************************************* + ** Delegate using this instance's configured collaborator. + *******************************************************************************/ + @Override + public List getModels() + { + return (List.of("test-model")); + } + + /******************************************************************************* + ** Delegate using this instance's configured collaborator. + *******************************************************************************/ + @Override + public String getDefaultApiKeyEnvName() + { + return (null); + } + + /******************************************************************************* + ** Delegate using this instance's configured collaborator. + *******************************************************************************/ + @Override + public String getDefaultBaseUrl() + { + return ("http://localhost"); + } + + /******************************************************************************* + ** Delegate using this instance's configured collaborator. + *******************************************************************************/ + @Override + public AssistantModelProvider create(String key, String base) + { + return (provider); + } + }); + runtime.close(); + runtime = new AssistantRuntime(qInstance, config, Clock.fixed(NOW, ZoneOffset.UTC)); + config.setRuntime(runtime); + AssistantScheduledRun blockedRun = scheduledRunner().run(task(), false); + assertThat(blockedRun.getStatus()).isEqualTo("BLOCKED"); + assertThat(blockedRun.getSummary()).isEqualTo("see conversation"); + assertThat(provider.getRequests()).isEmpty(); + runtime.getStore().insert(new AssistantProvider().withName("Eligible local") + .withProviderType("scheduled-test").withModel("test-model").withEnabled(true).withIsDefault(true).withLocation("LOCAL").withHipaaEligible(true)); + provider.thenText("PHI answer sentinel"); + AssistantScheduledRun run = scheduledRunner().run(task(), false); + assertThat(run.getStatus()).as(run.getError()).isEqualTo("OK"); + assertThat(run.getSummary()).isEqualTo("see conversation"); + var records = new QueryAction().execute( + new QueryInput("assistant_message")).getRecords(); + assertThat(records).isNotEmpty().allSatisfy(record -> assertThat(record.getValueString("partsJson")).startsWith("enc:v1:").doesNotContain("sentinel")); + } + + + + /******************************************************************************* + ** Instance/user budgets and the kill switch also stop scheduled starts. + *******************************************************************************/ + @Test + void normalPolicyStillApplies() throws Exception + { + AssistantScheduledTask task = task(); + runtime.getStore().insert(new AssistantPolicy().withName("Stop").withKillSwitch(true)); + assertThat(scheduledRunner().run(task, false).getStatus()).isEqualTo("BLOCKED"); + assertThat(provider.getRequests()).isEmpty(); + runtime.getStore().delete("assistant_policy", new QQueryFilter()); + config.getBudgets().setUserDailyCostMicrosHard(0L); + AssistantScheduledRun budgetRun = scheduledRunner().run(task(), false); + assertThat(budgetRun.getStatus()).isEqualTo("BLOCKED"); + assertThat(budgetRun.getError()).contains("BUDGET"); + assertThat(provider.getRequests()).isEmpty(); + } + + + + /******************************************************************************* + ** Application record locks stay effective with a designated-user login. + *******************************************************************************/ + @Test + void applicationRowsRemainOwnerLocked() throws Exception + { + qInstance.getTable("blog_post").withRecordSecurityLock(new RecordSecurityLock() + .withFieldName("slug").withSecurityKeyType(config.getOwnerSecurityKeyType())); + assertThat(runtime.getUiApiGateway().invoke(new UiApiCall("PATCH", "/table/{tableName}/{primaryKey}") + .withPathParam("tableName", "blog_post").withPathParam("primaryKey", "1").withSessionUUID(AssistantTestFixture.ADMIN) + .withJsonBody(Map.of("slug", AssistantTestFixture.EDITOR))).status()).isEqualTo(200); + provider.thenToolUse("query_records", input("table", "blog_post", "filter", filter("AND"))).thenText("Your one post"); + AssistantScheduledRun run = scheduledRunner().run(task(), false); + assertThat(run.getStatus()).as(run.getError()).isEqualTo("OK"); + var calls = runtime.getStore().query(AssistantToolCall.TABLE_NAME, new QQueryFilter().withCriteria("turnId", QCriteriaOperator.EQUALS, run.getTurnId()), AssistantToolCall.class); + assertThat(calls).singleElement().satisfies(call -> assertThat(call.getRowCount()).isEqualTo(1)); + } +} diff --git a/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/testing/UiApiTestServer.java b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/testing/UiApiTestServer.java index 4d40bef..05d4ea6 100644 --- a/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/testing/UiApiTestServer.java +++ b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/testing/UiApiTestServer.java @@ -19,6 +19,9 @@ package com.kingsrook.qbits.assistant.testing; import java.io.IOException; import java.net.ServerSocket; +import java.util.List; +import java.util.Map; +import java.util.concurrent.CopyOnWriteArrayList; import com.kingsrook.qbits.assistant.AssistantQBitConfig; import com.kingsrook.qqq.backend.core.exceptions.QException; import com.kingsrook.qqq.backend.core.instances.AbstractQQQApplication; @@ -34,6 +37,8 @@ import com.kingsrook.qqq.middleware.javalin.QApplicationJavalinServer; *******************************************************************************/ public final class UiApiTestServer { + public static final List> SCHEDULED_REQUESTS = new CopyOnWriteArrayList<>(); + private static QApplicationJavalinServer current; private static QInstance currentInstance; private static int currentPort; @@ -89,6 +94,7 @@ public final class UiApiTestServer if(current == null || currentInstance != qInstance) { stop(); + SCHEDULED_REQUESTS.clear(); try { currentPort = freePort(); @@ -96,7 +102,15 @@ public final class UiApiTestServer .withPort(currentPort) .withServeFrontendNext(false) .withServeFrontendMaterialDashboard(false) - .withServeLegacyUnversionedMiddlewareAPI(false); + .withServeLegacyUnversionedMiddlewareAPI(false) + .withJavalinConfigCustomizer(app -> app.routes.before(context -> + { + String client = context.header("X-QQQ-Client"); + if(client != null && !context.path().endsWith("manageSession")) + { + SCHEDULED_REQUESTS.add(Map.of("client", client, "session", String.valueOf(context.cookie("sessionUUID")), "path", context.path())); + } + })); current.start(); currentInstance = qInstance; }