Compare commits
6
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2b66492571
|
||
|
|
062a95e30a | ||
|
|
4d2a735992
|
||
|
|
a2fad5103d
|
||
|
|
4194837610
|
||
|
|
5736bf3e1f |
No files matched your search
@@ -6,6 +6,8 @@ All notable changes are documented here, following [Keep a Changelog](https://ke
|
||||
|
||||
### Added
|
||||
|
||||
- Foreign keys (`worm-audit:11-12`), `ON DELETE RESTRICT` only: chain link -> core audit header, sealed export -> its first and last checkpoints, export verification -> its export, with their indexes. Link references (ranges, break link, purge anchor) stay unconstrained because the retention purge deletes the links they outlive. Pre-existing orphans are copied to `qqq_fk_quarantine` (`KEPT`), never modified, and leave that constraint `NOT VALID`. Re-apply the rendered DDL (idempotent).
|
||||
- Guided empty states: `help/qbit-worm-audit.yaml` (QQQ help-bundle format) gives the app and all six tables a summary and a "How rows get created" section, applied by the `WormAuditHelp` stand-in (`STAND-IN(core=QRun-IO/qqq#1198, removeBy=4.1.0-RC.7)`) without replacing host help. `WormAuditTablesGuardrailTest` pins runtime use, seeding or a documented reason, and help for every table; `WormAuditSettingsSeedTest` runs the script's settings seed on H2.
|
||||
- RC.2 settings: editable, FIELD-audited singleton with insert-only defaults, validation, separate edit permission and a 30-second effective-settings cache. Schedules load at startup.
|
||||
- GOVERNANCE exports with 30-day default retention; per-export retention verification, administrator-bypass diagnostics, home warnings and documented COMPLIANCE transition.
|
||||
- New `worm-audit:10` settings migration; historical changesets remain unchanged.
|
||||
@@ -26,3 +28,7 @@ All notable changes are documented here, following [Keep a Changelog](https://ke
|
||||
### Fixed
|
||||
|
||||
- Keep runtime signing/trust objects and storage suppliers out of serialized metadata so signed-chain record child lists render successfully.
|
||||
|
||||
### Deprecated
|
||||
|
||||
- `WormAuditSettings.seed(config)` is deprecated for PostgreSQL hosts, whose database script (`worm-audit:10`) already seeds row 1; it remains supported for memory and H2 hosts. `read()` falls back to the configured values while the row is absent.
|
||||
@@ -80,6 +80,12 @@ check and database constraint require ID 1. Every edit uses core FIELD-level aud
|
||||
the change enters the audit chain. `modify_date` and `modified_by` are stamped by the server.
|
||||
Keep notes free of sensitive data; notes are audited too.
|
||||
|
||||
### Starter defaults and empty screens
|
||||
|
||||
The only seeded table is `worm_audit_settings`, by the database script (`WormAuditDdl.postgres(config)`, changeset `worm-audit:10`): `INSERT ... SELECT ... WHERE NOT EXISTS`, so it adds row 1 from the host's configured values only when absent and never overwrites an edit. There is no separate defaults changelog and no extra include line. If the row is absent the qbit runs on the configured values (`WormAuditSettings.read()` falls back), so nothing stops, but run the script to make the values visible and editable. The chain, checkpoint, export, verification and purge tables are logs written by the qbit's jobs and the database owner's purge function, and start empty.
|
||||
|
||||
Every table, and the app, ships help (`help/qbit-worm-audit.yaml`, the QQQ help-bundle format) saying what it is for and how its rows get created; the Next UI shows it as the empty state. A host's own help is never replaced. `WormAuditTablesGuardrailTest` and `WormAuditSettingsSeedTest` pin this.
|
||||
|
||||
| Field | Default | Validation / effect |
|
||||
|---|---|---|
|
||||
| `audit_retention_years` | 6 | 1–100; home next eligible purge date |
|
||||
@@ -104,8 +110,9 @@ absent. Historical checkpoints remain valid after changing their creation cadenc
|
||||
|
||||
Changeset `worm-audit:10` creates and seeds the table with `INSERT ... SELECT ... WHERE NOT
|
||||
EXISTS`; rerunning it never overwrites an administrator change. The renderer seeds from the
|
||||
host config. Memory hosts and the demo can call `WormAuditSettings.seed(config)` for the same
|
||||
insert-only behavior during single-threaded startup.
|
||||
host config. Memory and H2 hosts, whose settings table cannot take generic inserts, can still call
|
||||
`WormAuditSettings.seed(config)` for the same insert-only behavior during single-threaded
|
||||
startup (deprecated for PostgreSQL hosts, which get the row from the script).
|
||||
|
||||
`audit_retention_years` does **not** rewrite the database purge function. The home warns if
|
||||
it differs from the compiled interval; changing that interval requires a reviewed migration.
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
# Upgrading qbit-worm-audit
|
||||
|
||||
## Table help bundle (unreleased)
|
||||
|
||||
Nothing to do. The qbit applies `help/qbit-worm-audit.yaml` at the end of `produce()`, filling only help slots the host has not set. No schema or seed changes: the settings row is still seeded by the database script (changeset `worm-audit:10`) and nothing else is seeded.
|
||||
|
||||
`WormAuditSettings.seed(config)` is now `@Deprecated` for PostgreSQL hosts (the script seeds row 1). It still works and remains the way for memory and H2 hosts to create row 1. The settings reader falls back to the configured values while the row is absent.
|
||||
@@ -111,6 +111,17 @@ file into their changelog (kof22: `changesets/features/YYYY-MM-DD-worm-audit.sql
|
||||
qbit tables, the functions, the triggers, the purger role (`CREATE ROLE ... NOLOGIN` when it is
|
||||
missing) and the grants. It is idempotent (`CREATE OR REPLACE`, `DROP TRIGGER IF EXISTS`).
|
||||
|
||||
Foreign keys (`worm-audit:11-12`) are `ON DELETE RESTRICT` only, so nothing cascades into or out
|
||||
of evidence: `worm_audit_chain.audit_id` -> the core audit header (the purge deletes details, then
|
||||
links, then headers, in that order), `worm_audit_export.from/to_checkpoint_id` -> checkpoints and
|
||||
`worm_audit_verification.export_id` -> exports (neither is ever deleted). References to chain links
|
||||
(checkpoint, export and verification ranges, `first_break_link_id`, the purge anchor) are not
|
||||
foreign keys: the purge deletes the sealed prefix those rows outlive. On upgrade, rows whose parent
|
||||
is already missing are evidence and are never changed: they are copied to `qqq_fk_quarantine`
|
||||
(action `KEPT`) and that constraint stays `NOT VALID` (new rows are still checked). With the
|
||||
foreign key, a plain `TRUNCATE` of `audit` (or of checkpoints or exports) is refused by PostgreSQL
|
||||
before the guard trigger; `TRUNCATE ... CASCADE` reaches the guards, which refuse it.
|
||||
|
||||
### 4.5 Threat model summary
|
||||
|
||||
| Actor | Can modify audit rows? | Detected by |
|
||||
|
||||
@@ -351,3 +351,37 @@ INSERT INTO public.worm_audit_settings
|
||||
export_cron, verify_cron)
|
||||
SELECT 1, 6, 'GOVERNANCE', 30, 10000, 60, 30, 60, 24, '0 0 2 * * ?', '0 0 3 * * ?'
|
||||
WHERE NOT EXISTS (SELECT 1 FROM public.worm_audit_settings WHERE id = 1);
|
||||
|
||||
-- changeset worm-audit:11
|
||||
-- comment: Index the foreign key columns added by worm-audit:12 (worm_audit_chain.audit_id already leads worm_audit_chain_audit_all).
|
||||
CREATE INDEX IF NOT EXISTS worm_audit_export_from_checkpoint ON public.worm_audit_export(from_checkpoint_id);
|
||||
CREATE INDEX IF NOT EXISTS worm_audit_export_to_checkpoint ON public.worm_audit_export(to_checkpoint_id);
|
||||
CREATE INDEX IF NOT EXISTS worm_audit_verification_export ON public.worm_audit_verification(export_id);
|
||||
|
||||
-- changeset worm-audit:12 splitStatements:false
|
||||
-- comment: Foreign keys, ON DELETE RESTRICT only: nothing cascades into or out of audit evidence. A chain link needs its audit header (the purge deletes links before headers); a sealed export needs its signed checkpoints and an export verification its export (neither is ever deleted). Links referenced by checkpoints, exports, verifications and purges are not foreign keys: the purge deletes the sealed prefix those rows outlive. Rows whose parent is already missing are evidence and are never changed: they are copied to qqq_fk_quarantine (action KEPT) and the constraint is left NOT VALID (enforced for new rows only).
|
||||
DO $$
|
||||
DECLARE
|
||||
fk record;
|
||||
orphans bigint;
|
||||
BEGIN
|
||||
CREATE TABLE IF NOT EXISTS qqq_fk_quarantine (id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, table_name VARCHAR(100) NOT NULL, column_name VARCHAR(100) NOT NULL, row_id VARCHAR(100), missing_parent_id VARCHAR(100), action VARCHAR(20) NOT NULL, row_data TEXT, changeset VARCHAR(200), quarantined_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP);
|
||||
FOR fk IN SELECT * FROM (VALUES
|
||||
('fk_worm_audit_chain_audit_id', 'worm_audit_chain', 'audit_id', '"audit"'),
|
||||
('fk_worm_audit_export_from_checkpoint_id', 'worm_audit_export', 'from_checkpoint_id', 'public.worm_audit_checkpoint'),
|
||||
('fk_worm_audit_export_to_checkpoint_id', 'worm_audit_export', 'to_checkpoint_id', 'public.worm_audit_checkpoint'),
|
||||
('fk_worm_audit_verification_export_id', 'worm_audit_verification', 'export_id', 'public.worm_audit_export')
|
||||
) AS t(name, child, col, parent) LOOP
|
||||
IF NOT EXISTS (SELECT FROM pg_constraint WHERE conname = fk.name) THEN
|
||||
EXECUTE format('INSERT INTO qqq_fk_quarantine (table_name, column_name, row_id, missing_parent_id, action, row_data, changeset)'
|
||||
|| ' SELECT %L, %L, c.id::text, c.%I::text, ''KEPT'', row_to_json(c)::text, ''worm-audit:12'' FROM public.%I c'
|
||||
|| ' WHERE c.%I IS NOT NULL AND NOT EXISTS (SELECT FROM %s p WHERE p.id = c.%I)', fk.child, fk.col, fk.col, fk.child, fk.col, fk.parent, fk.col);
|
||||
GET DIAGNOSTICS orphans = ROW_COUNT;
|
||||
EXECUTE format('ALTER TABLE public.%I ADD CONSTRAINT %I FOREIGN KEY (%I) REFERENCES %s (id) ON DELETE RESTRICT NOT VALID', fk.child, fk.name, fk.col, fk.parent);
|
||||
IF orphans = 0 THEN
|
||||
EXECUTE format('ALTER TABLE public.%I VALIDATE CONSTRAINT %I', fk.child, fk.name);
|
||||
END IF;
|
||||
END IF;
|
||||
END LOOP;
|
||||
END;
|
||||
$$;
|
||||
@@ -26,6 +26,7 @@ import com.kingsrook.qbits.wormaudit.app.WormAuditApp;
|
||||
import com.kingsrook.qbits.wormaudit.ddl.WormAuditDdl;
|
||||
import com.kingsrook.qbits.wormaudit.guard.AppendOnlyGuard;
|
||||
import com.kingsrook.qbits.wormaudit.model.LinkRangeWidget;
|
||||
import com.kingsrook.qbits.wormaudit.model.WormAuditHelp;
|
||||
import com.kingsrook.qbits.wormaudit.model.WormAuditTables;
|
||||
import com.kingsrook.qbits.wormaudit.processes.ExportProcesses;
|
||||
import com.kingsrook.qbits.wormaudit.processes.WormAuditProcesses;
|
||||
@@ -230,6 +231,7 @@ public class WormAuditQBitProducer implements QBitMetaDataProducer<WormAuditQBit
|
||||
.withDefaultMonitor(AlertDefaultMonitor.heartbeat(name, name, 24 * 3600, 2 * 3600)));
|
||||
}
|
||||
output.addSelfToInstance(qInstance);
|
||||
WormAuditHelp.apply(qInstance, qBitConfig);
|
||||
denyAssistant(qInstance);
|
||||
return (new MetaDataProducerMultiOutput());
|
||||
}
|
||||
|
||||
@@ -91,7 +91,14 @@ public class WormAuditSettings
|
||||
|
||||
/*******************************************************************************
|
||||
** Seed the singleton only when absent; never overwrite an administrator edit.
|
||||
** The database script (changeset worm-audit:10) is the normal seed; this remains
|
||||
** for memory and H2 hosts, whose settings table cannot take generic inserts.
|
||||
**
|
||||
** @deprecated for PostgreSQL hosts, which get row 1 from the database script;
|
||||
** still the supported way to create row 1 on memory and H2 hosts. Reads already
|
||||
** fall back to the configured values while the row is absent.
|
||||
*******************************************************************************/
|
||||
@Deprecated
|
||||
public static void seed(WormAuditQBitConfig config) throws com.kingsrook.qqq.backend.core.exceptions.QException
|
||||
{
|
||||
var store = new AuditStore(null);
|
||||
|
||||
@@ -0,0 +1,139 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.wormaudit.model;
|
||||
|
||||
|
||||
import java.io.IOException;
|
||||
import java.io.InputStream;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.Map;
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.fasterxml.jackson.dataformat.yaml.YAMLFactory;
|
||||
import com.kingsrook.qbits.wormaudit.WormAuditQBitConfig;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.help.QHelpContent;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Applies help/qbit-worm-audit.yaml (the QQQ help-bundle format) to the WORM
|
||||
** Audit app and its worm_audit_* tables: what each table is for and how its
|
||||
** rows get created, shown on the table screen and when it is empty.
|
||||
** The bundle's app entry is keyed to the default app name; a host that renames
|
||||
** the app gets the same text.
|
||||
** STAND-IN(core=QRun-IO/qqq#1198, removeBy=4.1.0-RC.7): replace with
|
||||
** HelpBundles.register once the pinned QQQ release includes it; the bundle
|
||||
** file stays the same.
|
||||
*******************************************************************************/
|
||||
public final class WormAuditHelp
|
||||
{
|
||||
public static final String RESOURCE = "/help/qbit-worm-audit.yaml";
|
||||
public static final String SLOT = "help";
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Utility class.
|
||||
*******************************************************************************/
|
||||
private WormAuditHelp()
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The bundle's entries by key, validated (fails startup if malformed).
|
||||
*******************************************************************************/
|
||||
public static Map<String, JsonNode> entries() throws QException
|
||||
{
|
||||
try(InputStream stream = WormAuditHelp.class.getResourceAsStream(RESOURCE))
|
||||
{
|
||||
if(stream == null)
|
||||
{
|
||||
throw (new IOException("missing resource"));
|
||||
}
|
||||
JsonNode bundle = new ObjectMapper(new YAMLFactory()).readTree(stream);
|
||||
if(!"qbit-worm-audit".equals(bundle.path("bundle").asText()) || !"QBIT".equals(bundle.path("layer").asText()))
|
||||
{
|
||||
throw (new IOException("invalid bundle header"));
|
||||
}
|
||||
Map<String, JsonNode> entries = new LinkedHashMap<>();
|
||||
for(JsonNode entry : bundle.path("entries"))
|
||||
{
|
||||
String key = entry.path("key").asText();
|
||||
String summary = entry.path("summary").asText();
|
||||
if(key.isBlank() || summary.isBlank() || summary.length() > 120 || entry.path("content").asText().isBlank() || entries.put(key, entry) != null)
|
||||
{
|
||||
throw (new IOException("invalid or duplicate entry: " + key));
|
||||
}
|
||||
}
|
||||
return (entries);
|
||||
}
|
||||
catch(IOException | RuntimeException e)
|
||||
{
|
||||
throw (new QException("Could not load " + RESOURCE, e));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Apply the help to the app and every worm_audit_* table. A host's own help on
|
||||
** a table or the app is kept (the bundle only fills the slot).
|
||||
*******************************************************************************/
|
||||
public static void apply(QInstance qInstance, WormAuditQBitConfig config) throws QException
|
||||
{
|
||||
for(Map.Entry<String, JsonNode> entry : entries().entrySet())
|
||||
{
|
||||
String key = entry.getKey();
|
||||
if(key.startsWith("table:"))
|
||||
{
|
||||
QTableMetaData table = qInstance.getTable(key.substring("table:".length(), key.indexOf(';')));
|
||||
if(table != null && (table.getHelpContent() == null || !table.getHelpContent().containsKey(SLOT)))
|
||||
{
|
||||
table.withHelpContent(SLOT, content(entry.getValue()));
|
||||
}
|
||||
}
|
||||
else if(key.startsWith("app:"))
|
||||
{
|
||||
QAppMetaData app = qInstance.getApp(config.getAppName());
|
||||
if(app != null && (app.getHelpContent() == null || !app.getHelpContent().containsKey(SLOT)))
|
||||
{
|
||||
app.withHelpContent(SLOT, content(entry.getValue()));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Summary as the first paragraph, then the content, as markdown.
|
||||
*******************************************************************************/
|
||||
static QHelpContent content(JsonNode entry)
|
||||
{
|
||||
return (new QHelpContent().withContentAsMarkdown(entry.path("summary").asText().strip() + "\n\n" + entry.path("content").asText().strip()));
|
||||
}
|
||||
|
||||
}
|
||||
@@ -351,3 +351,37 @@ INSERT INTO public.worm_audit_settings
|
||||
export_cron, verify_cron)
|
||||
SELECT ${settings_values}
|
||||
WHERE NOT EXISTS (SELECT 1 FROM public.worm_audit_settings WHERE id = 1);
|
||||
|
||||
-- changeset worm-audit:11
|
||||
-- comment: Index the foreign key columns added by worm-audit:12 (worm_audit_chain.audit_id already leads worm_audit_chain_audit_all).
|
||||
CREATE INDEX IF NOT EXISTS worm_audit_export_from_checkpoint ON public.worm_audit_export(from_checkpoint_id);
|
||||
CREATE INDEX IF NOT EXISTS worm_audit_export_to_checkpoint ON public.worm_audit_export(to_checkpoint_id);
|
||||
CREATE INDEX IF NOT EXISTS worm_audit_verification_export ON public.worm_audit_verification(export_id);
|
||||
|
||||
-- changeset worm-audit:12 splitStatements:false
|
||||
-- comment: Foreign keys, ON DELETE RESTRICT only: nothing cascades into or out of audit evidence. A chain link needs its audit header (the purge deletes links before headers); a sealed export needs its signed checkpoints and an export verification its export (neither is ever deleted). Links referenced by checkpoints, exports, verifications and purges are not foreign keys: the purge deletes the sealed prefix those rows outlive. Rows whose parent is already missing are evidence and are never changed: they are copied to qqq_fk_quarantine (action KEPT) and the constraint is left NOT VALID (enforced for new rows only).
|
||||
DO $$
|
||||
DECLARE
|
||||
fk record;
|
||||
orphans bigint;
|
||||
BEGIN
|
||||
CREATE TABLE IF NOT EXISTS qqq_fk_quarantine (id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, table_name VARCHAR(100) NOT NULL, column_name VARCHAR(100) NOT NULL, row_id VARCHAR(100), missing_parent_id VARCHAR(100), action VARCHAR(20) NOT NULL, row_data TEXT, changeset VARCHAR(200), quarantined_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP);
|
||||
FOR fk IN SELECT * FROM (VALUES
|
||||
('fk_worm_audit_chain_audit_id', 'worm_audit_chain', 'audit_id', '${audit}'),
|
||||
('fk_worm_audit_export_from_checkpoint_id', 'worm_audit_export', 'from_checkpoint_id', 'public.worm_audit_checkpoint'),
|
||||
('fk_worm_audit_export_to_checkpoint_id', 'worm_audit_export', 'to_checkpoint_id', 'public.worm_audit_checkpoint'),
|
||||
('fk_worm_audit_verification_export_id', 'worm_audit_verification', 'export_id', 'public.worm_audit_export')
|
||||
) AS t(name, child, col, parent) LOOP
|
||||
IF NOT EXISTS (SELECT FROM pg_constraint WHERE conname = fk.name) THEN
|
||||
EXECUTE format('INSERT INTO qqq_fk_quarantine (table_name, column_name, row_id, missing_parent_id, action, row_data, changeset)'
|
||||
|| ' SELECT %L, %L, c.id::text, c.%I::text, ''KEPT'', row_to_json(c)::text, ''worm-audit:12'' FROM public.%I c'
|
||||
|| ' WHERE c.%I IS NOT NULL AND NOT EXISTS (SELECT FROM %s p WHERE p.id = c.%I)', fk.child, fk.col, fk.col, fk.child, fk.col, fk.parent, fk.col);
|
||||
GET DIAGNOSTICS orphans = ROW_COUNT;
|
||||
EXECUTE format('ALTER TABLE public.%I ADD CONSTRAINT %I FOREIGN KEY (%I) REFERENCES %s (id) ON DELETE RESTRICT NOT VALID', fk.child, fk.name, fk.col, fk.parent);
|
||||
IF orphans = 0 THEN
|
||||
EXECUTE format('ALTER TABLE public.%I VALIDATE CONSTRAINT %I', fk.child, fk.name);
|
||||
END IF;
|
||||
END IF;
|
||||
END LOOP;
|
||||
END;
|
||||
$$;
|
||||
@@ -0,0 +1,45 @@
|
||||
bundle: qbit-worm-audit
|
||||
version: 1
|
||||
layer: QBIT
|
||||
# Help for the WORM Audit app and every worm_audit_* table, in the QQQ help-bundle
|
||||
# format (qqq#1198). Each table entry says what the table is for and how its rows
|
||||
# get created, so an empty table explains itself. WormAuditHelp applies it until
|
||||
# the core HelpBundles registry ships in a QQQ release
|
||||
# (STAND-IN(core=QRun-IO/qqq#1198, removeBy=4.1.0-RC.7)). Summaries stay under 120
|
||||
# characters. Only the settings table is seeded (by the database script, see the
|
||||
# README); every other table is written by the qbit's jobs and starts empty.
|
||||
entries:
|
||||
- key: app:worm_audit;slot:help
|
||||
summary: "Tamper-evident audit: a hash chain over every audit record, signed checkpoints, and sealed exports."
|
||||
content: |
|
||||
Install the database script (the Setup warnings on the home list what is missing) and the one settings row is created with it. Everything else fills by itself once the first audit record has settled: the chain job links records every 30 seconds by default, checkpoints follow, the daily job verifies and exports. Use Verify audit chain to check at any time.
|
||||
|
||||
- key: table:worm_audit_settings;slot:help
|
||||
summary: "The one settings row: retention, export lock, checkpoint and chain cadence, and the job schedules."
|
||||
content: |
|
||||
**How rows get created:** the database script inserts the one row (ID 1) with the host's configured values, only when it is absent; it can never be added to or deleted. Edit record 1 to change a value; changes are audited and take effect within 30 seconds, except the schedules and chain interval, which need a restart. If the row is missing the qbit runs on the host's configured values, so nothing stops, but run the script to make them visible and editable here. Audit retention here is the next purge date only; the purge interval in the database changes through a reviewed migration.
|
||||
|
||||
- key: table:worm_audit_chain;slot:help
|
||||
summary: "One link per core audit record: a hash that includes the previous link, so any change breaks the chain."
|
||||
content: |
|
||||
**How rows get created:** the Chain settled audit records job appends links for audit records older than the settle horizon (every 30 seconds by default, set in Settings). Nobody adds, edits or deletes a link; the database refuses. An empty table means no audit record has settled yet, or the chain job is not running.
|
||||
|
||||
- key: table:worm_audit_checkpoint;slot:help
|
||||
summary: "Signed ranges of links (Ed25519), proving the chain's head at a point in time."
|
||||
content: |
|
||||
**How rows get created:** the chain job writes one after the configured number of links or minutes (Settings), signing the range. Sealed exports and verification check against them. An empty table means the chain is too short for a first checkpoint yet.
|
||||
|
||||
- key: table:worm_audit_export;slot:help
|
||||
summary: "Sealed exports of chain ranges to object-lock storage, with their retention and manifest."
|
||||
content: |
|
||||
**How rows get created:** the Daily sealed audit export job (schedule in Settings) and the Export now action seal the links since the last export into the configured bucket, then record it here. Verify sealed export re-reads one. An empty table means no export has run: check the bucket setup warning on the home.
|
||||
|
||||
- key: table:worm_audit_verification;slot:help
|
||||
summary: "Every run of the chain verification: what was checked and whether it passed or broke."
|
||||
content: |
|
||||
**How rows get created:** the Verify audit chain action and the Daily audit chain verification job (schedule in Settings) each add a row, with the first break found, if any. A break also raises an alert. An empty table means verification has not run yet.
|
||||
|
||||
- key: table:worm_audit_purge;slot:help
|
||||
summary: "Record of each retention purge: the cutoff, what was removed and the new anchor link."
|
||||
content: |
|
||||
**How rows get created:** only the database's owner role can purge, by calling the purge function after the retention date has passed; the function writes this row. The application can never delete audit rows. An empty table means nothing has been purged, which is normal until records pass retention.
|
||||
@@ -229,11 +229,21 @@ class PostgresAcceptanceTest
|
||||
assertThat(((SQLException) e).getSQLState())
|
||||
.isEqualTo("42501"));
|
||||
sql(c, "GRANT UPDATE, DELETE, TRUNCATE ON " + table + " TO app");
|
||||
for(String operation :
|
||||
List.of(
|
||||
"UPDATE " + table + " SET id=id",
|
||||
"DELETE FROM " + table,
|
||||
"TRUNCATE " + table))
|
||||
// A table a foreign key references (audit, checkpoint, export) refuses TRUNCATE before
|
||||
// its guard runs; TRUNCATE ... CASCADE (owner) reaches the guards, which refuse it.
|
||||
if(count(c, "SELECT count(*) FROM pg_constraint WHERE contype='f' AND confrelid='" + table + "'::regclass") > 0)
|
||||
{
|
||||
assertThatThrownBy(() -> sql(runtime, "TRUNCATE " + table))
|
||||
.isInstanceOf(SQLException.class)
|
||||
.hasMessageContaining("referenced in a foreign key constraint");
|
||||
assertThatThrownBy(() -> sql(c, "TRUNCATE " + table + " CASCADE"))
|
||||
.isInstanceOf(SQLException.class)
|
||||
.hasMessageContaining("worm_audit:");
|
||||
}
|
||||
boolean referenced = count(c, "SELECT count(*) FROM pg_constraint WHERE contype='f' AND confrelid='" + table + "'::regclass") > 0;
|
||||
for(String operation : referenced
|
||||
? List.of("UPDATE " + table + " SET id=id", "DELETE FROM " + table)
|
||||
: List.of("UPDATE " + table + " SET id=id", "DELETE FROM " + table, "TRUNCATE " + table))
|
||||
{
|
||||
assertThatThrownBy(() -> sql(runtime, operation))
|
||||
.isInstanceOf(SQLException.class)
|
||||
@@ -583,6 +593,91 @@ class PostgresAcceptanceTest
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Referential integrity guardrail: every table reference of the WORM tables
|
||||
** (possible-value fields, and the chain's audit header) has a RESTRICT
|
||||
** foreign key, or an exemption with its reason. Nothing may cascade into or
|
||||
** out of audit evidence.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void everyReferenceHasARestrictForeignKeyOrAReason() throws Exception
|
||||
{
|
||||
String links = "the retention purge deletes the sealed prefix of chain links; this evidence row outlives the links it names";
|
||||
java.util.Map<String, String> exempt = java.util.Map.of(
|
||||
"worm_audit_checkpoint.from_link_id", links,
|
||||
"worm_audit_checkpoint.to_link_id", links,
|
||||
"worm_audit_export.from_link_id", links,
|
||||
"worm_audit_export.to_link_id", links,
|
||||
"worm_audit_verification.from_link_id", links,
|
||||
"worm_audit_verification.to_link_id", links,
|
||||
"worm_audit_verification.first_break_link_id", links + "; a break may name a link that is missing",
|
||||
"worm_audit_purge.anchor_link_id", "the anchor is the last link the same purge deleted");
|
||||
|
||||
java.util.Set<String> references = new java.util.TreeSet<>(List.of("worm_audit_chain.audit_id"));
|
||||
for(var table : instance.getTables().values())
|
||||
{
|
||||
if(!table.getName().startsWith("worm_audit_"))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
for(var field : table.getFields().values())
|
||||
{
|
||||
if(field.getPossibleValueSourceName() != null && instance.getTable(field.getPossibleValueSourceName()) != null)
|
||||
{
|
||||
references.add(table.getName() + "." + field.getBackendName());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
java.util.Map<String, String> foreignKeys = new java.util.HashMap<>();
|
||||
try(var c = owner(); var s = c.createStatement(); var r = s.executeQuery("""
|
||||
SELECT c.conrelid::regclass::text || '.' || a.attname, c.confdeltype::text || c.convalidated::text
|
||||
FROM pg_constraint c JOIN pg_attribute a ON a.attrelid = c.conrelid AND a.attnum = c.conkey[1]
|
||||
WHERE c.contype = 'f' AND c.conrelid::regclass::text LIKE 'worm_audit_%'"""))
|
||||
{
|
||||
while(r.next())
|
||||
{
|
||||
foreignKeys.put(r.getString(1), r.getString(2));
|
||||
}
|
||||
}
|
||||
assertThat(references.stream().filter(ref -> !foreignKeys.containsKey(ref) && !exempt.containsKey(ref)).toList())
|
||||
.as("references without a foreign key (add one to worm-audit.sql, or an exemption with a reason)").isEmpty();
|
||||
assertThat(foreignKeys).as("RESTRICT and validated").containsOnlyKeys("worm_audit_chain.audit_id", "worm_audit_export.from_checkpoint_id",
|
||||
"worm_audit_export.to_checkpoint_id", "worm_audit_verification.export_id").allSatisfy((k, v) -> assertThat(v).isEqualTo("rtrue"));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Upgrading a schema whose chain names a missing audit header: the link is
|
||||
** evidence, so it is copied to qqq_fk_quarantine (KEPT), never changed, and
|
||||
** the constraint is added NOT VALID (new links are still checked). A re-run
|
||||
** changes nothing.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void orphanEvidenceIsKeptAndTheConstraintIsNotValidated() throws Exception
|
||||
{
|
||||
audit(1);
|
||||
new WormAuditChainer(config).chainOnce();
|
||||
try(var c = owner())
|
||||
{
|
||||
sql(c, "ALTER TABLE worm_audit_chain DROP CONSTRAINT fk_worm_audit_chain_audit_id");
|
||||
sql(c, "ALTER TABLE audit DISABLE TRIGGER worm_audit_row_guard; DELETE FROM audit WHERE id = 1; ALTER TABLE audit ENABLE TRIGGER worm_audit_row_guard");
|
||||
sql(c, WormAuditDdl.postgres(config));
|
||||
sql(c, WormAuditDdl.postgres(config));
|
||||
assertThat(count(c, "SELECT count(*) FROM qqq_fk_quarantine WHERE table_name = 'worm_audit_chain' AND column_name = 'audit_id'"
|
||||
+ " AND row_id = '1' AND missing_parent_id = '1' AND action = 'KEPT' AND row_data LIKE '%\"link_hash\"%'")).isEqualTo(1);
|
||||
assertThat(count(c, "SELECT count(*) FROM qqq_fk_quarantine")).isEqualTo(1);
|
||||
assertThat(count(c, "SELECT count(*) FROM worm_audit_chain WHERE id = 1 AND audit_id = 1")).isEqualTo(1);
|
||||
assertThat(count(c, "SELECT count(*) FROM pg_constraint WHERE conname = 'fk_worm_audit_chain_audit_id' AND NOT convalidated")).isEqualTo(1);
|
||||
assertThatThrownBy(() -> sql(c, "INSERT INTO worm_audit_chain(id,kind,audit_id,detail_ids,prev_hash,link_hash,chained_at)"
|
||||
+ " VALUES(99,'AUDIT',404,'',repeat('0',64),repeat('0',64),now())"))
|
||||
.hasMessageContaining("fk_worm_audit_chain_audit_id");
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The settings row (seeded by the DDL) overrides config: change it there.
|
||||
*******************************************************************************/
|
||||
|
||||
@@ -0,0 +1,140 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.wormaudit;
|
||||
|
||||
|
||||
import java.sql.Connection;
|
||||
import java.sql.DriverManager;
|
||||
import java.sql.ResultSet;
|
||||
import java.sql.Statement;
|
||||
import java.util.UUID;
|
||||
import java.util.regex.Matcher;
|
||||
import java.util.regex.Pattern;
|
||||
import com.kingsrook.qbits.wormaudit.ddl.WormAuditDdl;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The settings row the shipped database script seeds (ddl/postgres/worm-audit.sql,
|
||||
** changeset worm-audit:10, rendered with the host's configured values).
|
||||
** The whole script is PostgreSQL-only (triggers, roles, plpgsql), so it cannot
|
||||
** run on H2; this runs the settings table's CREATE and its seed statement as
|
||||
** the script renders them, on H2 in PostgreSQL mode: the row appears with the
|
||||
** host's values, a second run inserts nothing, and an edit survives.
|
||||
** WormAuditSettings.seed() is deprecated (nothing in src/main calls it) but still
|
||||
** works for memory and H2 hosts; WormAuditSettings.read() falls back to the
|
||||
** configured values while the row is absent.
|
||||
*******************************************************************************/
|
||||
class WormAuditSettingsSeedTest
|
||||
{
|
||||
|
||||
/*******************************************************************************
|
||||
** Fresh table, seed twice, edit survives, values follow the host config.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void seedStatementIsInsertOnlyAndUsesHostValues() throws Exception
|
||||
{
|
||||
var config = new WormAuditQBitConfig().withRetentionYears(8).withExportRetentionDays(45).withCheckpointEveryLinks(12);
|
||||
String ddl = WormAuditDdl.postgres(config);
|
||||
String create = statement(ddl, "CREATE TABLE IF NOT EXISTS public.worm_audit_settings");
|
||||
String seed = statement(ddl, "INSERT INTO public.worm_audit_settings");
|
||||
|
||||
try(Connection connection = DriverManager.getConnection("jdbc:h2:mem:" + UUID.randomUUID() + ";MODE=PostgreSQL;DB_CLOSE_DELAY=-1");
|
||||
Statement sql = connection.createStatement())
|
||||
{
|
||||
sql.execute(create.replace("timestamptz", "timestamp with time zone").replace("now()", "CURRENT_TIMESTAMP").replace("current_user", "'app'"));
|
||||
sql.execute(seed);
|
||||
assertThat(scalar(sql, "SELECT COUNT(*) FROM public.worm_audit_settings")).isEqualTo("1");
|
||||
assertThat(scalar(sql, "SELECT audit_retention_years FROM public.worm_audit_settings")).isEqualTo("8");
|
||||
assertThat(scalar(sql, "SELECT export_retention_days FROM public.worm_audit_settings")).isEqualTo("45");
|
||||
assertThat(scalar(sql, "SELECT checkpoint_every_links FROM public.worm_audit_settings")).isEqualTo("12");
|
||||
assertThat(scalar(sql, "SELECT export_lock_mode FROM public.worm_audit_settings")).isEqualTo(config.getExportLockMode());
|
||||
assertThat(scalar(sql, "SELECT export_cron FROM public.worm_audit_settings")).isEqualTo(config.getExportCron());
|
||||
|
||||
sql.execute("UPDATE public.worm_audit_settings SET export_retention_days = 90 WHERE id = 1");
|
||||
sql.execute(seed);
|
||||
assertThat(scalar(sql, "SELECT COUNT(*) FROM public.worm_audit_settings")).isEqualTo("1");
|
||||
assertThat(scalar(sql, "SELECT export_retention_days FROM public.worm_audit_settings")).isEqualTo("90");
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The deprecated public seed() still creates row 1 once and keeps edits.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
@SuppressWarnings("deprecation")
|
||||
void deprecatedSeedStillWorksForMemoryHosts() throws Exception
|
||||
{
|
||||
var config = new WormAuditQBitConfig().withRetentionYears(8);
|
||||
WormAuditTestSupport.init(config);
|
||||
WormAuditSettings.seed(config);
|
||||
assertThat(WormAuditTestSupport.STORE.get("worm_audit_settings", 1).getValueInteger("audit_retention_years")).isEqualTo(8);
|
||||
WormAuditSettings.seed(config);
|
||||
assertThat(WormAuditTestSupport.STORE.get("worm_audit_settings", 1)).isNotNull();
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The script seeds nothing but the settings table: its only top-level INSERT is the
|
||||
** settings seed (the indented purge INSERTs are the purge function writing its log).
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void onlySettingsAreSeeded()
|
||||
{
|
||||
String ddl = WormAuditDdl.postgres(new WormAuditQBitConfig());
|
||||
Matcher matcher = Pattern.compile("(?m)^INSERT INTO\\s+(\\S+)").matcher(ddl);
|
||||
while(matcher.find())
|
||||
{
|
||||
assertThat(matcher.group(1)).as("every top-level INSERT in the script targets the settings table, not a log").isEqualTo("public.worm_audit_settings");
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** From the start text up to the statement's closing semicolon.
|
||||
*******************************************************************************/
|
||||
private static String statement(String ddl, String start)
|
||||
{
|
||||
int from = ddl.indexOf(start);
|
||||
assertThat(from).as(start).isGreaterThanOrEqualTo(0);
|
||||
return (ddl.substring(from, ddl.indexOf(";", from)));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private static String scalar(Statement sql, String query) throws Exception
|
||||
{
|
||||
try(ResultSet result = sql.executeQuery(query))
|
||||
{
|
||||
assertThat(result.next()).isTrue();
|
||||
return (result.getString(1));
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
@@ -119,7 +119,7 @@ class WormAuditSettingsTest
|
||||
var reader = new WormAuditSettings(config, clock);
|
||||
assertThat(reader.read().audit_retention_years()).isEqualTo(8);
|
||||
assertThat(reader.read().export_retention_days()).isEqualTo(45);
|
||||
WormAuditSettings.seed(config);
|
||||
WormAuditTestSupport.seedSettings(config);
|
||||
editor();
|
||||
update("export_retention_days", 90);
|
||||
assertThat(reader.read().export_retention_days()).isEqualTo(90);
|
||||
@@ -128,7 +128,7 @@ class WormAuditSettingsTest
|
||||
assertThat(reader.read().export_retention_days()).isEqualTo(90);
|
||||
clock.now = clock.now.plusSeconds(1);
|
||||
assertThat(reader.read().export_retention_days()).isEqualTo(120);
|
||||
WormAuditSettings.seed(config);
|
||||
WormAuditTestSupport.seedSettings(config);
|
||||
assertThat(STORE.get(TABLE, 1).getValueInteger("export_retention_days")).isEqualTo(120);
|
||||
assertThat(reader.read().checkpoint_every_links()).isEqualTo(12);
|
||||
instance.getTables().remove(TABLE);
|
||||
@@ -181,7 +181,7 @@ class WormAuditSettingsTest
|
||||
assertThat(table.isCapabilityEnabled(instance.getBackendForTable(TABLE), Capability.TABLE_UPDATE)).isTrue();
|
||||
assertThat(table.isCapabilityEnabled(instance.getBackendForTable(TABLE), Capability.TABLE_INSERT)).isFalse();
|
||||
assertThat(table.isCapabilityEnabled(instance.getBackendForTable(TABLE), Capability.TABLE_DELETE)).isFalse();
|
||||
WormAuditSettings.seed(config);
|
||||
WormAuditTestSupport.seedSettings(config);
|
||||
editor();
|
||||
assertThatThrownBy(() -> new DeleteAction().execute(new DeleteInput(TABLE).withPrimaryKeys(List.of(1)))).isInstanceOf(Exception.class);
|
||||
assertThatThrownBy(() -> new InsertAction().execute(new InsertInput(TABLE).withRecord(new QRecord().withValue("id", 2)))).isInstanceOf(Exception.class);
|
||||
@@ -218,7 +218,7 @@ class WormAuditSettingsTest
|
||||
{
|
||||
var config = ChainVerificationTest.signed().withSettleHorizonSeconds(0);
|
||||
init(config);
|
||||
WormAuditSettings.seed(config);
|
||||
WormAuditTestSupport.seedSettings(config);
|
||||
QContext.getQInstance().withEnforcePermissionsInCoreActions(true);
|
||||
QContext.getQSession().withPermissions(Set.of(TABLE + ".read", TABLE + ".edit"));
|
||||
assertThatThrownBy(() -> update("export_lock_mode", "COMPLIANCE")).hasMessageContaining("worm_audit.settings");
|
||||
@@ -243,7 +243,7 @@ class WormAuditSettingsTest
|
||||
{
|
||||
var config = ChainVerificationTest.signed();
|
||||
var instance = init(config);
|
||||
WormAuditSettings.seed(config);
|
||||
WormAuditTestSupport.seedSettings(config);
|
||||
editor();
|
||||
update("chain_interval_seconds", 45);
|
||||
update("export_cron", "0 15 4 * * ?");
|
||||
@@ -275,7 +275,7 @@ class WormAuditSettingsTest
|
||||
{
|
||||
var config = ChainVerificationTest.signed().withCheckpointEveryLinks(2).withSettleHorizonSeconds(0);
|
||||
init(config);
|
||||
WormAuditSettings.seed(config);
|
||||
WormAuditTestSupport.seedSettings(config);
|
||||
var storage = new SealedExportTest.MemorySealedStore();
|
||||
audit(1);
|
||||
audit(2);
|
||||
|
||||
@@ -0,0 +1,169 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.wormaudit;
|
||||
|
||||
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.stream.Stream;
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.kingsrook.qbits.wormaudit.ddl.WormAuditDdl;
|
||||
import com.kingsrook.qbits.wormaudit.model.WormAuditHelp;
|
||||
import com.kingsrook.qbits.wormaudit.model.WormAuditTables;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.help.QHelpContent;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Guardrail (owner requirement 2026-10-14: "all tables should be populated and
|
||||
** the qbit should be fully using them"). For every worm_audit_* table:
|
||||
** - runtime code (not just its model, metadata or home screen) reads or writes
|
||||
** it;
|
||||
** - it is either seeded by the shipped database script, or listed here with
|
||||
** why it starts empty;
|
||||
** - it has bundle help that says what it is for and how its rows get created
|
||||
** (shown on the table screen and when it is empty).
|
||||
** A new table fails this test until it is wired, seeded or explained.
|
||||
*******************************************************************************/
|
||||
class WormAuditTablesGuardrailTest
|
||||
{
|
||||
private static final Path MAIN = Path.of("src", "main", "java", "com", "kingsrook", "qbits", "wormaudit");
|
||||
|
||||
/*******************************************************************************
|
||||
** Tables the shipped database script (ddl/postgres/worm-audit.sql) seeds.
|
||||
*******************************************************************************/
|
||||
static final List<String> SEEDED = List.of(WormAuditTables.SETTINGS);
|
||||
|
||||
/*******************************************************************************
|
||||
** Tables that legitimately start empty, and where their rows come from.
|
||||
*******************************************************************************/
|
||||
static final Map<String, String> STARTS_EMPTY = Map.of(
|
||||
WormAuditTables.CHAIN, "log: the chain job links each settled audit record",
|
||||
WormAuditTables.CHECKPOINT, "log: the chain job signs a range every N links or minutes",
|
||||
WormAuditTables.EXPORT, "log: the daily export or Export now seals a range",
|
||||
WormAuditTables.VERIFICATION, "log: every verification run, manual or daily",
|
||||
WormAuditTables.PURGE, "log: the database owner's purge function, after retention");
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Every table is used by runtime code.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void everyTableIsUsedAtRuntime() throws Exception
|
||||
{
|
||||
List<String> runtimeSource;
|
||||
try(Stream<Path> files = Files.walk(MAIN))
|
||||
{
|
||||
runtimeSource = files.filter(p -> p.toString().endsWith(".java"))
|
||||
.filter(p -> !p.startsWith(MAIN.resolve("model")) && !p.startsWith(MAIN.resolve("app")) && !p.startsWith(MAIN.resolve("ddl")))
|
||||
.filter(p -> !p.endsWith("WormAuditQBitProducer.java"))
|
||||
.map(WormAuditTablesGuardrailTest::read).toList();
|
||||
}
|
||||
for(String table : WormAuditTables.ALL)
|
||||
{
|
||||
String constant = "WormAuditTables." + table.substring("worm_audit_".length()).toUpperCase();
|
||||
assertThat(runtimeSource.stream().anyMatch(source -> source.contains(constant)))
|
||||
.as(table + " (" + constant + ") must be read or written by runtime code, not only declared").isTrue();
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Every table is seeded by the script or explained.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void everyTableIsSeededOrExplained() throws Exception
|
||||
{
|
||||
String ddl = WormAuditDdl.postgres(new WormAuditQBitConfig());
|
||||
for(String table : WormAuditTables.ALL)
|
||||
{
|
||||
boolean seeded = SEEDED.contains(table);
|
||||
assertThat(seeded || STARTS_EMPTY.containsKey(table)).as(table + " must be seeded by the database script or listed in STARTS_EMPTY").isTrue();
|
||||
assertThat(seeded && STARTS_EMPTY.containsKey(table)).as(table + " cannot be both seeded and starting empty").isFalse();
|
||||
assertThat(ddl.contains("INSERT INTO public." + table + "\n") || ddl.contains("INSERT INTO public." + table + " ")).as(table + " seeded iff the script inserts it").isEqualTo(seeded);
|
||||
}
|
||||
assertThat(STARTS_EMPTY.keySet()).allMatch(WormAuditTables.ALL::contains, "STARTS_EMPTY names only real tables");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Every table and the app have bundle help that says how rows get created.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void everyTableHasEmptyStateHelp() throws Exception
|
||||
{
|
||||
QInstance instance = WormAuditTestSupport.init(new WormAuditQBitConfig());
|
||||
assertThat(WormAuditHelp.entries()).allSatisfy((key, entry) -> assertThat(entry.path("summary").asText().length()).as(key).isLessThanOrEqualTo(120));
|
||||
for(String table : WormAuditTables.ALL)
|
||||
{
|
||||
List<QHelpContent> help = instance.getTable(table).getHelpContent() == null ? null : instance.getTable(table).getHelpContent().get(WormAuditHelp.SLOT);
|
||||
assertThat(help).as(table + " table help").isNotEmpty();
|
||||
assertThat(help.get(0).getContent()).as(table + " help says how rows get created").contains("How rows get created");
|
||||
}
|
||||
assertThat(instance.getApp("worm_audit").getHelpContent()).containsKey(WormAuditHelp.SLOT);
|
||||
for(Map.Entry<String, JsonNode> entry : WormAuditHelp.entries().entrySet())
|
||||
{
|
||||
String key = entry.getKey();
|
||||
assertThat(key.startsWith("app:") || WormAuditTables.ALL.contains(key.substring(6, key.indexOf(';')))).as("help names only real tables: " + key).isTrue();
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A host's own help is never replaced.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void hostHelpIsKept() throws Exception
|
||||
{
|
||||
QInstance instance = WormAuditTestSupport.init(new WormAuditQBitConfig());
|
||||
instance.getTable(WormAuditTables.CHAIN).getHelpContent().put(WormAuditHelp.SLOT, List.of(new QHelpContent().withContentAsMarkdown("host text")));
|
||||
WormAuditHelp.apply(instance, new WormAuditQBitConfig());
|
||||
assertThat(instance.getTable(WormAuditTables.CHAIN).getHelpContent().get(WormAuditHelp.SLOT)).hasSize(1);
|
||||
assertThat(instance.getTable(WormAuditTables.CHAIN).getHelpContent().get(WormAuditHelp.SLOT).get(0).getContent()).isEqualTo("host text");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
**
|
||||
*******************************************************************************/
|
||||
private static String read(Path path)
|
||||
{
|
||||
try
|
||||
{
|
||||
return (Files.readString(path, StandardCharsets.UTF_8));
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
throw (new IllegalStateException(e));
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
@@ -37,7 +37,7 @@ import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.Mem
|
||||
import java.time.Instant;
|
||||
import java.util.List;
|
||||
|
||||
class WormAuditTestSupport {
|
||||
public class WormAuditTestSupport {
|
||||
static final Instant OLD = Instant.parse("2020-01-02T03:04:05.123456Z");
|
||||
static final AuditStore STORE = new AuditStore(null);
|
||||
|
||||
@@ -130,6 +130,16 @@ class WormAuditTestSupport {
|
||||
false);
|
||||
}
|
||||
|
||||
/** What the shipped database script does for the singleton: insert row 1 from the host config, only when absent. */
|
||||
public static void seedSettings(WormAuditQBitConfig config) throws Exception {
|
||||
if (STORE.get(com.kingsrook.qbits.wormaudit.model.WormAuditTables.SETTINGS, 1) == null) {
|
||||
STORE.insert(
|
||||
com.kingsrook.qbits.wormaudit.model.WormAuditTables.SETTINGS,
|
||||
List.of(WormAuditSettings.defaults(config)));
|
||||
}
|
||||
config.invalidateSettings();
|
||||
}
|
||||
|
||||
static void remove(String table, long id) throws Exception {
|
||||
MemoryRecordStore.getInstance().delete(new DeleteInput(table).withPrimaryKeys(List.of(id)));
|
||||
}
|
||||
|
||||
@@ -176,7 +176,7 @@ public class WormAuditDemoServer extends AbstractQQQApplication
|
||||
public void seed(QInstance instance) throws Exception
|
||||
{
|
||||
QContext.init(instance, new QSession());
|
||||
com.kingsrook.qbits.wormaudit.WormAuditSettings.seed(config);
|
||||
com.kingsrook.qbits.wormaudit.WormAuditTestSupport.seedSettings(config);
|
||||
AuditStore store = new AuditStore(null);
|
||||
Instant today = Instant.now().minus(Duration.ofHours(2));
|
||||
try
|
||||
|
||||
Reference in new issue
Block a user