Compare commits

...
Author SHA1 Message Date
CircleCI 6294c52eee Bump version to 1.0.0-RC.7 [skip ci] 2026-10-14 22:33:48 +00:00
james.maesandClaude Opus 5.5 422094fecb Merge develop into release/1.0 for 1.0.0-RC.7 (#25 setup card, seeds, help)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:30:44 -05:00
james.maes b281741d2e Merge pull request #25 from QRun-IO/feature/tables-populated-defaults
SQL Workbench: table help, guardrail test, audit retention
2026-10-14 17:19:50 -05:00
james.maesandClaude Sonnet 5.5 47a3e62a45 fix(audit): prune in bounded background batches, spare RUNNING rows, delay first pass
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-14 17:16:05 -05:00
james.maesandClaude Opus 5.5 7e9017c376 feat(help): table help, guardrail test and audit retention
Adds help/sql-workbench.yaml (empty-state help for all six tables) applied by
SqlWorkbenchHelp, a tables guardrail test, and the auditRetentionDays config
that DESIGN.md described but did not exist (default null keeps everything).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 16:58:04 -05:00
james.maes 268499aa74 Merge pull request #22 from QRun-IO/feature/orb-0.6.9
ci: qqq-orb 0.6.9 (refresh SNAPSHOT dependencies)
2026-10-14 10:46:39 -05:00
james.maesandClaude Opus 5.5 d864eb996d ci: qqq-orb 0.6.9 (refresh SNAPSHOT dependencies)
Moves qqq-orb@0.6.8 to qqq-orb@0.6.9, which passes --update-snapshots so cached SNAPSHOT dependencies are re-checked on every build (QRun-IO/qqq-orb#22).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 10:07:53 -05:00
CircleCI aa13c1367e Bump version to 1.0.0-RC.6 [skip ci] 2026-10-14 12:26:21 +00:00
james.maesandClaude Opus 5.5 660278e020 Merge develop into release/1.0 for 1.0.0-RC.6 (#21 UI API extension, guardrails)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 07:22:53 -05:00
james.maes c59b2ae72f Merge pull request #21 from QRun-IO/feature/arch-ui-api-specs
feat(api): sql-workbench routes as spec'd UI API extension endpoints (#20)
2026-10-14 07:21:36 -05:00
CircleCI 6a0eae112c Bump version to 1.0.0-RC.5 [skip ci] 2026-10-12 21:28:20 +00:00
james.maesandClaude Opus 5.5 bb43871212 Merge develop into release/1.0 for 1.0.0-RC.5
Users get sql_workbench_result_snapshot.insert, so pinning results works
under core permission enforcement (#16).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 16:26:05 -05:00
CircleCI a0e3c222f7 Bump version to 1.0.0-RC.4 [skip ci] 2026-10-12 21:13:23 +00:00
james.maesandClaude Opus 5.5 b531d9c046 Merge develop into release/1.0 for 1.0.0-RC.4
snake_case names (#7), app home and Runner (#9), record developer panel
(#6) and record-screen polish (#14). 1.0.0-RC.3 on Central is an
accidental republish of RC.2, so this candidate is RC.4.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 16:10:45 -05:00
james.maesandClaude Opus 5.5 763c0e4352 ci: fix the GA tag filter YAML (no brackets in the flow sequence) [skip ci]
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 16:01:51 -05:00
james.maesandClaude Opus 5.5 d0153ebd9c Set version to 1.0.0-RC.3 and publish only GA tags [skip ci]
1.0.0-RC.3 is already on Maven Central (an RC tag build republished the previous code), so the next release-branch build must publish the following RC. RC tags no longer trigger a publish (/^v[0-9]+\.[0-9]+\.[0-9]+$/ only).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 15:56:55 -05:00
james.maesandClaude Opus 5.5 0d8af84fbc Bump version to 1.0.0-RC.2 [skip ci]
The orb published 1.0.0-RC.2 but could not push this commit (the
CircleCI deploy key is read only).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 12:39:06 -05:00
james.maesandClaude Opus 5.5 0d06a04953 Merge develop into release/1.0 for 1.0.0-RC.2
Sets <revision> to 1.0.0-RC.1, the version already on Maven Central,
because RC.1's version bump never landed on release/1.0. The orb
increments an existing RC, so this publishes 1.0.0-RC.2 instead of a
duplicate RC.1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 12:27:56 -05:00
14 changed files with 647 additions and 4 deletions

No files matched your search

+1 -1
View File
@@ -1,7 +1,7 @@
version: 2.1
orbs:
qqq-orb: kingsrook/qqq-orb@0.6.8
qqq-orb: kingsrook/qqq-orb@0.6.9
commands:
configure-package-install:
+2
View File
@@ -5,6 +5,8 @@ All notable changes to this project are documented here. The format follows Keep
## [1.0.0-RC.4] - Unreleased
### Added
- Table help: the SQL Workbench app and every `sql_workbench_*` table carry help saying what the table is for and how its rows get created (`help/sql-workbench.yaml`, QQQ help-bundle format, applied by `SqlWorkbenchHelp`; host help is never overridden). `SqlWorkbenchTablesGuardrailTest` fails when a table is unused by runtime code, unexplained, or has no such help. All six tables are user content or logs and start empty by design; nothing is seeded.
- `auditRetentionDays` config (default null: keep forever, no change for existing hosts): when set, execution audit rows older than that many days are deleted, in the background in bounded batches, at most once an hour per process (first pass 5 minutes after start), triggered as a run is audited; rows still RUNNING are kept. DESIGN.md already described this setting; it did not exist before.
- OpenAPI coverage for all 21 SQL Workbench UI API operations, including streaming CSV/JSON/XLSX export. The `sql-workbench` extension preserves `/qqq/v1/sql-workbench`, existing request/response shapes, permission and ownership checks, exact decimal parameters, execution limits and audit. The generated fragment is in `src/main/resources/openapi/qqq-v1/sql-workbench.yaml`; strict-server security tests and a fragment contract test guard it.
- **App home dashboard.** The `sqlWorkbench` app's home is now an information dashboard of real data, read in the viewer's session (permissions and record locks apply): summary tiles and links, runs per day, recent notebooks, recent runs, slowest and failed queries, most-used notebooks, shared notebooks, committed writes, and (workbench admins only) activity by user. Each widget has an honest empty state.
- **Runner.** The notebook editor (`SQL_WORKBENCH` widget) moves to its own nav entry: the `sqlWorkbenchRunner` child app, labeled Runner. Open a notebook with `/app/sqlWorkbenchRunner?notebook=<id>`.
+6
View File
@@ -101,6 +101,12 @@ public class SnakeCaseCustomizer implements MetaDataCustomizerInterface<QTableMe
}
```
## Starter defaults and retention
All six tables are user content (notebooks, cells, shares, pinned snapshots) or logs (versions, the execution audit), so none is seeded: they start empty, and each table's help (`help/sql-workbench.yaml`, shown on the table screen) says how its rows get created. No host include or action is needed.
Retention: `notebookVersionRetention` (default 100) prunes the oldest versions of a notebook on save. `auditRetentionDays` (default null: keep forever) deletes execution audit rows older than that many days, in the background in bounded batches, at most once an hour per process, triggered as a run is audited.
## Storage upgrade from 1.0.0-RC.1
RC.2 adds `sql_workbench_notebook_share`. The DDL creates it and backfills an `all` share (the shared library) for every notebook whose `visibility` is `SHARED`; it is additive and idempotent, and owners (`owner_user_id`) are unchanged. Run it before deploying RC.2: without the share rows, published notebooks are visible only to their owners.
+4
View File
@@ -1,5 +1,9 @@
# Upgrading
## Table help and `auditRetentionDays` (unreleased)
No action needed. The qbit now fills empty `help` slots on its app and tables (help a host already set is kept). The new `auditRetentionDays` config defaults to null, which keeps every audit row as before; set it to bound the size of `sql_workbench_execution`. `notebookVersionRetention` (default 100 per notebook) was already enforced. There is no DDL change and no seed.
## To 1.0.0-RC.4: snake_case names (metadata only)
The rename shipped in 1.0.0-RC.4. 1.0.0-RC.3 on Maven Central is an accidental republish of the RC.2 code (see CHANGELOG.md), so upgrading from RC.3 is the same as upgrading from RC.2.
+1 -1
View File
@@ -190,7 +190,7 @@ Publishing in the notebook UI inserts (or deletes) the `ALL` share; the share cu
| clientIp | STRING(64) | from the request, when available |
| createDate | DATE_TIME | |
The audit table has its insert, edit, and delete capabilities disabled in the QQQ metadata, so the UI and generic API do not offer those operations, and its customizer rejects them in the core actions for every session but the system user (admins included). Its record lock shows each user their own rows (admins all). The engine writes it through QQQ actions with a system session context. The row is inserted **before** execution (`RUNNING`). If that insert fails, the SQL **does not run** (fail closed). The row is updated at the end. Retention: `auditRetentionDays` (default unlimited; the host prunes).
The audit table has its insert, edit, and delete capabilities disabled in the QQQ metadata, so the UI and generic API do not offer those operations, and its customizer rejects them in the core actions for every session but the system user (admins included). Its record lock shows each user their own rows (admins all). The engine writes it through QQQ actions with a system session context. The row is inserted **before** execution (`RUNNING`). If that insert fails, the SQL **does not run** (fail closed). The row is updated at the end. Retention: `auditRetentionDays` (default unlimited; when set, rows whose run started earlier than that many days ago are deleted, in the background in bounded batches, at most once an hour per process (first pass 5 minutes after start), triggered as a run is audited; rows still RUNNING are kept).
---
+1 -1
View File
@@ -20,7 +20,7 @@
<properties>
<qbit.guardrails.enforce>true</qbit.guardrails.enforce>
<qbit.guardrails.strictUiApiRoutes>true</qbit.guardrails.strictUiApiRoutes>
<revision>1.0.0-SNAPSHOT</revision>
<revision>1.0.0-RC.7</revision>
<maven.compiler.source>21</maven.compiler.source>
<maven.compiler.target>21</maven.compiler.target>
<jacoco.version>0.8.15</jacoco.version>
@@ -122,6 +122,7 @@ public class SqlWorkbenchQBitConfig implements QBitConfig
private Integer maxSchemaTables = 2000;
private Integer snapshotMaxRows = 1000;
private Integer notebookVersionRetention = 100;
private Integer auditRetentionDays = null;
private Boolean registerRoutes = true;
private transient MetaDataCustomizerInterface<QTableMetaData> tableMetaDataCustomizer;
@@ -215,6 +216,10 @@ public class SqlWorkbenchQBitConfig implements QBitConfig
requirePositive(errors, "maxSchemaTables", maxSchemaTables);
requirePositive(errors, "snapshotMaxRows", snapshotMaxRows);
requirePositive(errors, "notebookVersionRetention", notebookVersionRetention);
if(auditRetentionDays != null)
{
requirePositive(errors, "auditRetentionDays", auditRetentionDays);
}
if(defaultPageSize != null && maxPageSize != null && defaultPageSize > maxPageSize)
{
@@ -1480,6 +1485,39 @@ public class SqlWorkbenchQBitConfig implements QBitConfig
}
/*******************************************************************************
** Getter for auditRetentionDays: execution audit rows older than this many
** days are deleted (at most once an hour, when a run is audited); null (the
** default) keeps them forever.
*******************************************************************************/
public Integer getAuditRetentionDays()
{
return (this.auditRetentionDays);
}
/*******************************************************************************
** Setter for auditRetentionDays
*******************************************************************************/
public void setAuditRetentionDays(Integer auditRetentionDays)
{
this.auditRetentionDays = auditRetentionDays;
}
/*******************************************************************************
** Fluent setter for auditRetentionDays
*******************************************************************************/
public SqlWorkbenchQBitConfig withAuditRetentionDays(Integer auditRetentionDays)
{
this.auditRetentionDays = auditRetentionDays;
return (this);
}
/*******************************************************************************
** Getter for registerRoutes
*******************************************************************************/
@@ -28,6 +28,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitMetaDataProducer;
import com.kingsrook.qqq.backend.core.utils.StringUtils;
import com.kingsrook.qbits.sqlworkbench.metadata.SqlWorkbenchHelp;
import com.kingsrook.qbits.sqlworkbench.metadata.SqlWorkbenchAppMetaDataProducer;
import com.kingsrook.qbits.sqlworkbench.metadata.SqlWorkbenchTableMetaDataHelper;
import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
@@ -182,6 +183,11 @@ public class SqlWorkbenchQBitProducer implements QBitMetaDataProducer<SqlWorkben
MetaDataProducerMultiOutput output = QBitMetaDataProducer.super.produce(qInstance);
output.addSelfToInstance(qInstance);
/////////////////////////////////////////////////////////////////////////
// after the tables and app are in the instance: fill their empty help //
/////////////////////////////////////////////////////////////////////////
SqlWorkbenchHelp.apply(qInstance, qBitConfig);
return (new MetaDataProducerMultiOutput());
}
@@ -28,7 +28,7 @@ import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** Notebook versions and the execution audit are append-only: only the qbit
** itself (the system user) inserts them, and only its retention pruning
** itself (the system user) inserts them, and only its retention pruning (auditRetentionDays, notebookVersionRetention)
** deletes them. The tables also lack the insert / update / delete
** capabilities, which hides those operations in the UI and API; this
** customizer enforces it in the core actions, whoever calls them (admins
@@ -19,14 +19,27 @@ package com.kingsrook.qbits.sqlworkbench.engine;
import java.time.Duration;
import java.time.Instant;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.atomic.AtomicLong;
import com.kingsrook.qqq.backend.core.actions.tables.DeleteAction;
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.session.QSystemUserSession;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertOutput;
@@ -50,7 +63,20 @@ public class ExecutionAuditor
public static final int MAX_ERROR_LENGTH = 4000;
public static final long PRUNE_INTERVAL_MILLIS = 60L * 60 * 1000;
public static final long FIRST_PRUNE_DELAY_MILLIS = 5L * 60 * 1000;
public static final int BATCH_SIZE = 1000;
public static final int MAX_BATCHES = 20;
private static final ExecutorService PRUNER = Executors.newSingleThreadExecutor(runnable ->
{
Thread thread = new Thread(runnable, "sql-workbench-audit-prune");
thread.setDaemon(true);
return (thread);
});
private final SqlWorkbenchQBitConfig config;
private final AtomicLong lastPruneMillis = new AtomicLong(System.currentTimeMillis() - PRUNE_INTERVAL_MILLIS + FIRST_PRUNE_DELAY_MILLIS);
@@ -92,11 +118,114 @@ public class ExecutionAuditor
throwIfNotInserted(inserted[0], "Could not write the SQL Workbench audit record; the SQL was not run.");
execution.setId(inserted[0].getValueInteger("id"));
pruneIfDue();
return (execution.getId());
}
/*******************************************************************************
** For tests: make the next pruneIfDue start a pass now.
*******************************************************************************/
void allowPruneNow()
{
lastPruneMillis.set(0);
}
/*******************************************************************************
** Apply auditRetentionDays in the background, at most once an hour per
** process, and never the moment a process starts (the first pass is
** FIRST_PRUNE_DELAY_MILLIS after construction, so restarts and rolling
** deploys do not each trigger a pass). Never delays or fails the run being
** audited: a failed pass is logged and tried again an hour later.
**
** Across replicas: the qbit has no table for shared state, so each replica
** keeps its own hourly limit. That is safe because a pass is bounded
** (MAX_BATCHES batches of BATCH_SIZE rows) and idempotent: replicas that
** overlap delete rows another replica already deleted, which costs one
** cheap query. N replicas therefore cost at most N bounded passes an hour.
**
** @return the pending pass, or null when none was started
*******************************************************************************/
Future<?> pruneIfDue()
{
if(config.getAuditRetentionDays() == null)
{
return (null);
}
long now = System.currentTimeMillis();
long last = lastPruneMillis.get();
if(now - last < PRUNE_INTERVAL_MILLIS || !lastPruneMillis.compareAndSet(last, now))
{
return (null);
}
QInstance qInstance = QContext.getQInstance();
Instant cutoff = Instant.ofEpochMilli(now).minus(Duration.ofDays(config.getAuditRetentionDays()));
return (PRUNER.submit(() ->
{
QContext.init(qInstance, new QSystemUserSession());
try
{
pruneOlderThan(cutoff);
}
catch(Exception e)
{
LOG.warn("Could not prune old SQL Workbench audit rows", e);
}
finally
{
QContext.clear();
}
}));
}
/*******************************************************************************
** Delete audit rows that started before a cutoff (system user; the table is
** otherwise append-only), in batches of BATCH_SIZE ids (so a large backlog
** never loads whole rows, with their SQL text, or builds an unbounded IN
** list), at most MAX_BATCHES batches per call; the rest goes on the next
** pass. Rows still RUNNING are kept.
**
** @return the number of rows deleted
*******************************************************************************/
public int pruneOlderThan(Instant cutoff) throws QException
{
int[] deleted = new int[1];
asSystem(() ->
{
for(int batch = 0; batch < MAX_BATCHES; batch++)
{
QQueryFilter filter = new QQueryFilter(new QFilterCriteria("startTime", QCriteriaOperator.LESS_THAN, cutoff))
.withCriteria(new QFilterCriteria("status", QCriteriaOperator.NOT_EQUALS, ExecutionStatus.RUNNING.name()))
.withOrderBy(new QFilterOrderBy("id"))
.withLimit(BATCH_SIZE);
List<Serializable> ids = new ArrayList<>();
for(QRecord record : new QueryAction().execute(new QueryInput(tableName()).withFilter(filter)).getRecords())
{
ids.add(record.getValueInteger("id"));
}
if(ids.isEmpty())
{
return;
}
deleted[0] += new DeleteAction().execute(new DeleteInput(tableName()).withPrimaryKeys(ids)).getDeletedRecordCount();
if(ids.size() < BATCH_SIZE)
{
return;
}
}
});
return (deleted[0]);
}
/*******************************************************************************
** Record the outcome.
*******************************************************************************/
@@ -0,0 +1,134 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.sqlworkbench.metadata;
import java.io.IOException;
import java.io.InputStream;
import java.util.LinkedHashMap;
import java.util.Map;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.dataformat.yaml.YAMLFactory;
import com.kingsrook.qbits.sqlworkbench.SqlWorkbenchQBitConfig;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.help.QHelpContent;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
/*******************************************************************************
** Applies help/sql-workbench.yaml (the QQQ help-bundle format) to the SQL
** Workbench app and its sql_workbench_* tables: what each table is for and
** how its rows get created, shown on the table screen and when it is empty.
** STAND-IN(core=QRun-IO/qqq#1198, removeBy=4.1.0-RC.9): replace with HelpBundles.register once the pinned QQQ
** release includes it; the bundle file stays the same.
*******************************************************************************/
public final class SqlWorkbenchHelp
{
public static final String RESOURCE = "/help/sql-workbench.yaml";
public static final String SLOT = "help";
/*******************************************************************************
** Utility class.
*******************************************************************************/
private SqlWorkbenchHelp()
{
}
/*******************************************************************************
** The bundle's entries by key, validated (fails startup if malformed).
*******************************************************************************/
public static Map<String, JsonNode> entries() throws QException
{
try(InputStream stream = SqlWorkbenchHelp.class.getResourceAsStream(RESOURCE))
{
if(stream == null)
{
throw (new IOException("missing resource"));
}
JsonNode bundle = new ObjectMapper(new YAMLFactory()).readTree(stream);
if(!"sql-workbench".equals(bundle.path("bundle").asText()) || !"QBIT".equals(bundle.path("layer").asText()))
{
throw (new IOException("invalid bundle header"));
}
Map<String, JsonNode> entries = new LinkedHashMap<>();
for(JsonNode entry : bundle.path("entries"))
{
String key = entry.path("key").asText();
String summary = entry.path("summary").asText();
if(key.isBlank() || summary.isBlank() || summary.length() > 120 || entry.path("content").asText().isBlank() || entries.put(key, entry) != null)
{
throw (new IOException("invalid or duplicate entry: " + key));
}
}
return (entries);
}
catch(IOException | RuntimeException e)
{
throw (new QException("Could not load " + RESOURCE, e));
}
}
/*******************************************************************************
** Apply the help to the app and every sql_workbench_* table in the instance
** (names carry the config's prefix, if any). A host's own help on a table
** is kept (the bundle only fills the slot).
*******************************************************************************/
public static void apply(QInstance qInstance, SqlWorkbenchQBitConfig config) throws QException
{
for(Map.Entry<String, JsonNode> entry : entries().entrySet())
{
String key = entry.getKey();
String name = key.substring(key.indexOf(':') + 1, key.indexOf(';'));
if(key.startsWith("table:"))
{
QTableMetaData table = qInstance.getTable(config.applyPrefix(name));
if(table != null && (table.getHelpContent() == null || !table.getHelpContent().containsKey(SLOT)))
{
table.withHelpContent(SLOT, content(entry.getValue()));
}
}
else if(key.startsWith("app:"))
{
QAppMetaData app = qInstance.getApp(config.applyPrefix(name));
if(app != null && (app.getHelpContent() == null || !app.getHelpContent().containsKey(SLOT)))
{
app.withHelpContent(SLOT, content(entry.getValue()));
}
}
}
}
/*******************************************************************************
** Summary as the first paragraph, then the content, as markdown.
*******************************************************************************/
static QHelpContent content(JsonNode entry)
{
return (new QHelpContent().withContentAsMarkdown(entry.path("summary").asText().strip() + "\n\n" + entry.path("content").asText().strip()));
}
}
@@ -0,0 +1,43 @@
bundle: sql-workbench
version: 1
layer: QBIT
# Help for the SQL Workbench app and every sql_workbench_* table, in the QQQ
# help-bundle format (qqq#1198). Each table entry says what the table is for and
# how its rows get created, so an empty table explains itself. SqlWorkbenchHelp
# applies it until the core HelpBundles registry ships in a QQQ release.
# Summaries stay under 120 characters (SqlWorkbenchTablesGuardrailTest).
entries:
- key: app:sqlWorkbench;slot:help
summary: "Run SQL against your backends in notebooks: read-only by default, write mode gated, everything audited."
content: |
Nothing needs to be set up: every table here holds notebooks you write or a record of what ran. Open the Runner to write your first query and save it as a notebook. The Dashboard lists your notebooks and recent runs.
- key: table:sql_workbench_notebook;slot:help
summary: "Saved SQL notebooks: a title, the target backend, parameters and sharing."
content: |
**How rows get created:** you create them. Open the Runner, write your SQL and Save, or use New here. Duplicate Notebook makes a personal copy of one you can read, and Share Notebook lets others read or edit yours. A new installation has none; this is your own content.
- key: table:sql_workbench_cell;slot:help
summary: "The SQL and text cells that make up each notebook, in order."
content: |
**How rows get created:** by the notebook editor. Each cell you add in the Runner (SQL or markdown) is saved here with its notebook. You rarely edit this table directly.
- key: table:sql_workbench_notebook_version;slot:help
summary: "Automatic history of each notebook: one version per saved change, newest kept."
content: |
**How rows get created:** by the workbench, never by hand. Every save of a notebook writes a version, and the oldest are pruned beyond notebookVersionRetention (100 per notebook by default). Use Restore Version to bring an old one back as a new version.
- key: table:sql_workbench_notebook_share;slot:help
summary: "Who a notebook is shared with: a user, a group, or everyone, and whether they can edit."
content: |
**How rows get created:** use Share Notebook on a notebook you own. Each share adds a row for the user, group or everyone. Notebooks are private until shared.
- key: table:sql_workbench_result_snapshot;slot:help
summary: "Result tables you pinned to a notebook so the data is kept with it."
content: |
**How rows get created:** run a query in the Runner and pin its result to the notebook (Snapshot). The snapshot keeps the audited SQL and up to snapshotMaxRows rows. It stays empty until someone pins a result.
- key: table:sql_workbench_execution;slot:help
summary: "The audit of every statement run: who, where, the SQL, the outcome and how long it took."
content: |
**How rows get created:** by the workbench itself, before each run, including rejected requests; nobody can add, edit or delete rows. It stays empty until someone runs a query. Set auditRetentionDays in the qbit config to delete rows older than that many days (the default keeps them forever).
@@ -0,0 +1,188 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.sqlworkbench;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.stream.Stream;
import com.kingsrook.qbits.sqlworkbench.metadata.SqlWorkbenchHelp;
import com.kingsrook.qqq.backend.core.model.metadata.help.QHelpContent;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** Guardrail (owner requirement 2026-10-14: "all tables should be populated and
** the qbit should be fully using them"). For every sql_workbench_* table:
** - runtime code (not just its model, metadata or home dashboard) reads or
** writes it, so no table exists only on paper;
** - it is listed in STARTS_EMPTY with why (every table here is user content or
** a log, so none is seeded);
** - it has bundle help that says what it is for and how its rows get created.
** A new table fails this test until it is wired and explained.
*******************************************************************************/
class SqlWorkbenchTablesGuardrailTest extends BaseSqlWorkbenchTest
{
private static final Path MAIN = Path.of("src", "main", "java", "com", "kingsrook", "qbits", "sqlworkbench");
/*******************************************************************************
** Every table legitimately starts empty, and where its rows come from.
*******************************************************************************/
static final Map<String, String> STARTS_EMPTY = Map.of(
"sql_workbench_notebook", "user content: someone saves a notebook",
"sql_workbench_cell", "user content: the cells of a saved notebook",
"sql_workbench_notebook_version", "runtime: every notebook save",
"sql_workbench_notebook_share", "user content: someone shares a notebook",
"sql_workbench_result_snapshot", "user content: someone pins a result",
"sql_workbench_execution", "runtime: every statement run (the audit)");
/*******************************************************************************
** Every table is used by runtime code.
*******************************************************************************/
@Test
void everyTableIsUsedAtRuntime() throws Exception
{
Map<String, String> entityByTable = entityByTable();
List<String> runtimeSource;
try(Stream<Path> files = Files.walk(MAIN))
{
runtimeSource = files.filter(p -> p.toString().endsWith(".java"))
.filter(p -> !p.startsWith(MAIN.resolve("model")) && !p.startsWith(MAIN.resolve("metadata")) && !p.startsWith(MAIN.resolve("home")))
.map(SqlWorkbenchTablesGuardrailTest::read).toList();
}
for(String table : workbenchTables())
{
String entity = entityByTable.get(table);
assertThat(entity).as("entity class for " + table).isNotNull();
Pattern use = Pattern.compile("\\b" + entity + "(\\.TABLE_NAME|\\.class|\\(|::)");
assertThat(runtimeSource.stream().anyMatch(source -> use.matcher(source).find()))
.as(table + " (" + entity + ") must be read or written by runtime code, not only declared").isTrue();
}
}
/*******************************************************************************
** Every table is explained as starting empty.
*******************************************************************************/
@Test
void everyTableIsExplained()
{
assertThat(workbenchTables()).isEqualTo(STARTS_EMPTY.keySet().stream().sorted().toList());
}
/*******************************************************************************
** Every table and the app have bundle help that says how rows get created.
*******************************************************************************/
@Test
void everyTableHasEmptyStateHelp() throws Exception
{
assertThat(SqlWorkbenchHelp.entries()).allSatisfy((key, entry) -> assertThat(entry.path("summary").asText().length()).as(key).isLessThanOrEqualTo(120));
for(String table : workbenchTables())
{
QTableMetaData meta = qInstance.getTable(table);
List<QHelpContent> help = meta.getHelpContent() == null ? null : meta.getHelpContent().get(SqlWorkbenchHelp.SLOT);
assertThat(help).as(table + " table help").isNotEmpty();
assertThat(help.get(0).getContent()).as(table + " help says how rows get created").contains("How rows get created");
}
assertThat(qInstance.getApp("sqlWorkbench").getHelpContent()).containsKey(SqlWorkbenchHelp.SLOT);
assertThat(SqlWorkbenchHelp.entries().keySet()).allMatch(key -> !key.startsWith("table:") || qInstance.getTable(key.substring(6, key.indexOf(';'))) != null,
"help names only real tables");
}
/*******************************************************************************
** Help a host already wrote is kept; the bundle only fills empty slots.
*******************************************************************************/
@Test
void hostHelpIsNotOverridden() throws Exception
{
QTableMetaData table = qInstance.getTable("sql_workbench_notebook");
table.getHelpContent().remove(SqlWorkbenchHelp.SLOT);
table.withHelpContent(SqlWorkbenchHelp.SLOT, new QHelpContent("host words"));
SqlWorkbenchHelp.apply(qInstance, config);
assertThat(table.getHelpContent().get(SqlWorkbenchHelp.SLOT).get(0).getContent()).isEqualTo("host words");
}
/*******************************************************************************
**
*******************************************************************************/
private List<String> workbenchTables()
{
List<String> tables = qInstance.getTables().values().stream().map(QTableMetaData::getName).filter(n -> n.startsWith("sql_workbench_")).sorted().toList();
assertThat(tables).hasSize(6);
return (tables);
}
/*******************************************************************************
** model/X.java declares TABLE_NAME = "x".
*******************************************************************************/
private static Map<String, String> entityByTable() throws Exception
{
Map<String, String> rs = new HashMap<>();
Pattern p = Pattern.compile("TABLE_NAME\\s*=\\s*\"([a-z_]+)\"");
try(Stream<Path> files = Files.list(MAIN.resolve("model")))
{
for(Path file : files.toList())
{
Matcher m = p.matcher(read(file));
if(m.find())
{
rs.put(m.group(1), file.getFileName().toString().replace(".java", ""));
}
}
}
return (rs);
}
/*******************************************************************************
**
*******************************************************************************/
private static String read(Path path)
{
try
{
return (Files.readString(path, StandardCharsets.UTF_8));
}
catch(Exception e)
{
throw (new IllegalStateException(e));
}
}
}
@@ -490,6 +490,99 @@ class SqlExecutionEngineTest extends BaseSqlWorkbenchTest
/*******************************************************************************
** auditRetentionDays deletes audit rows older than the window, in the
** background, and only those: a 40-day-old RUNNING row survives. The default
** keeps everything, and a process does not prune the moment it starts.
*******************************************************************************/
@Test
void testAuditRetentionPrunesOldRows() throws Exception
{
ExecutionAuditor auditor = new ExecutionAuditor(config);
java.time.Instant old = java.time.Instant.now().minus(java.time.Duration.ofDays(40));
for(String[] row : new String[][] { { "a1", "REJECTED" }, { "a2", "REJECTED" }, { "r1", "RUNNING" } })
{
auditor.record(new SqlWorkbenchExecution().withExecutionUuid("00000000-0000-0000-0000-0000000000" + row[0]).withUserId(READER).withBackendName(BACKEND_NAME)
.withMode(ExecutionMode.READ_ONLY.name()).withStatus(row[1]).withSqlText("SELECT 1").withStartTime(old));
}
assertThat(audits()).hasSize(3);
////////////////////////////////////////////////////////////
// default (null): nothing is ever pruned //
////////////////////////////////////////////////////////////
run("SELECT 1", Map.of(), READER);
auditor.allowPruneNow();
assertThat(auditor.pruneIfDue()).isNull();
assertThat(audits()).hasSize(4);
////////////////////////////////////////////////////////////////////
// 30 days: a fresh auditor waits out its start-up delay //
////////////////////////////////////////////////////////////////////
config.setAuditRetentionDays(30);
ExecutionAuditor pruning = new ExecutionAuditor(config);
assertThat(pruning.pruneIfDue()).as("no pass right after start-up").isNull();
pruning.allowPruneNow();
pruning.pruneIfDue().get();
assertThat(audits()).extracting(SqlWorkbenchExecution::getExecutionUuid)
.doesNotContain("00000000-0000-0000-0000-0000000000a1", "00000000-0000-0000-0000-0000000000a2")
.contains("00000000-0000-0000-0000-0000000000r1");
assertThat(audits()).as("the old RUNNING row and the fresh run").hasSize(2);
assertThat(pruning.pruneIfDue()).as("at most hourly").isNull();
assertThat(pruning.pruneOlderThan(java.time.Instant.now().minus(java.time.Duration.ofDays(30)))).as("RUNNING rows are never deleted").isZero();
////////////////////////////////////////////////////////////////////////////
// the pass that start() triggers runs in the background and spares RUNNING //
////////////////////////////////////////////////////////////////////////////
auditor.record(new SqlWorkbenchExecution().withExecutionUuid("00000000-0000-0000-0000-0000000000a3").withUserId(READER).withBackendName(BACKEND_NAME)
.withMode(ExecutionMode.READ_ONLY.name()).withStatus(ExecutionStatus.REJECTED.name()).withSqlText("SELECT 1").withStartTime(old));
pruning.allowPruneNow();
pruning.start(new SqlWorkbenchExecution().withExecutionUuid("00000000-0000-0000-0000-0000000000b1").withUserId(READER).withBackendName(BACKEND_NAME)
.withMode(ExecutionMode.READ_ONLY.name()).withSqlText("SELECT 2").withStartTime(java.time.Instant.now()));
for(int i = 0; i < 100 && audits().stream().anyMatch(a -> a.getExecutionUuid().endsWith("a3")); i++)
{
Thread.sleep(50);
}
assertThat(audits()).extracting(SqlWorkbenchExecution::getExecutionUuid).doesNotContain("00000000-0000-0000-0000-0000000000a3")
.contains("00000000-0000-0000-0000-0000000000r1");
}
/*******************************************************************************
** A backlog bigger than one batch is deleted in bounded batches.
*******************************************************************************/
@Test
void testAuditRetentionDeletesInBatches() throws Exception
{
config.setAuditRetentionDays(30);
ExecutionAuditor auditor = new ExecutionAuditor(config);
java.time.Instant old = java.time.Instant.now().minus(java.time.Duration.ofDays(40));
int total = ExecutionAuditor.BATCH_SIZE + 5;
for(int i = 0; i < total; i++)
{
auditor.record(new SqlWorkbenchExecution().withExecutionUuid(java.util.UUID.randomUUID().toString()).withUserId(READER).withBackendName(BACKEND_NAME)
.withMode(ExecutionMode.READ_ONLY.name()).withStatus(ExecutionStatus.REJECTED.name()).withSqlText("SELECT 1").withStartTime(old));
}
assertThat(auditor.pruneOlderThan(java.time.Instant.now().minus(java.time.Duration.ofDays(30)))).isEqualTo(total);
assertThat(audits()).isEmpty();
}
/*******************************************************************************
** auditRetentionDays must be positive when set.
*******************************************************************************/
@Test
void testAuditRetentionValidation()
{
java.util.List<String> errors = new java.util.ArrayList<>();
new com.kingsrook.qbits.sqlworkbench.SqlWorkbenchQBitConfig().withBackendName(BACKEND_NAME).withAuditRetentionDays(0).validate(null, errors);
assertThat(errors).anyMatch(e -> e.contains("auditRetentionDays"));
}
/*******************************************************************************
** A connection that cannot be opened answers with a generic message: the
** driver's (URL, host, user) stays in the server log.