Compare commits

...
Author SHA1 Message Date
james.maesandCodex a5d7192b95 feat: allow host-owned Audit app composition
Keep standalone defaults while permitting embedded WORM metadata and authorized widgets without replacing host navigation or help.

Co-Authored-By: Codex <noreply@openai.com>
Signed-off-by: James Maes <james@kof22.com>
2026-10-19 10:01:20 -05:00
james.maes d78ecc1bc4 Merge pull request #17 from QRun-IO/ci/release-rc-secrets-inherit
ci(release-rc): pass secrets so the shared workflow can sign the RC tag
2026-10-16 08:04:06 -05:00
james.maesandClaude Opus 5.5 e81c8ba37e ci(release-rc): pass secrets to the shared workflow so it can sign the RC tag
QRun-IO/.github#2 adds a tag-and-release job that signs vX with the
QRUN_TAG_SIGNING_KEY secret after Maven Central confirms the artifact.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 16:20:12 -05:00
james.maes a9e985a86c Merge pull request #16 from QRun-IO/feature/purge-replace-as-owner
fix(ddl): replace worm_audit_purge when the owner revoked its own EXECUTE
2026-10-15 09:16:36 -05:00
james.maesandClaude Opus 5.5 bde8ff416f fix(ddl): replace worm_audit_purge when the owner revoked its own EXECUTE
PostgreSQL refuses CREATE OR REPLACE FUNCTION when the owning role has no
EXECUTE on it, which is the CNPG shape (the runtime role owns the schema
and worm-audit:6 revokes its EXECUTE). worm-audit:13 now grants EXECUTE
back to the migrating role only when missing, replaces the function, and
revokes it again inside the same changeset transaction.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 09:13:59 -05:00
james.maes 92c5fc9ca4 Merge pull request #15 from QRun-IO/feature/ddl-comment-not-changeset
fix(ddl): no comment line may start with -- changeset
2026-10-15 07:58:54 -05:00
james.maesandClaude Opus 5.5 86f9929cd2 fix(ddl): no comment line may start with -- changeset
Liquibase formatted SQL parses any line beginning with '-- changeset' as a
directive, so the worm-audit:13 comment broke the changelog. Reword it and
guard the rendered DDL in CanonicalAndCryptoTest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 07:56:47 -05:00
james.maes 4842cb174d Merge pull request #14 from QRun-IO/feature/chain-coverage-perf
perf(chain): normalized detail coverage table and sweep floors
2026-10-15 07:42:38 -05:00
11 changed files with 139 additions and 7 deletions

No files matched your search

+1
View File
@@ -21,5 +21,6 @@ jobs:
release-rc:
if: github.event_name == 'workflow_dispatch' || (github.event.pull_request.merged == true && contains(github.event.pull_request.labels.*.name, 'release:rc'))
uses: QRun-IO/.github/.github/workflows/qbit-release-rc.yml@main
secrets: inherit
with:
dry_run: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run }}
+2
View File
@@ -6,6 +6,8 @@ All notable changes are documented here, following [Keep a Changelog](https://ke
### Fixed
- `worm-audit:13` failed with `permission denied for function worm_audit_purge` when the migrating role owns the function but had revoked its own EXECUTE (a runtime role that owns the schema, as on CNPG; PostgreSQL refuses `CREATE OR REPLACE` then). The changeset now grants EXECUTE back to the migrating role only if it lacks it, replaces the function, and revokes it again in the same transaction. Found by Website-Backend's CNPG-shaped changelog test.
- `worm-audit:13` had a comment line starting with `-- changeset`, which Liquibase formatted-SQL parsing rejects as a malformed directive. The comment is reworded, and `CanonicalAndCryptoTest` now requires every `-- changeset` line to be a real directive.
- Chain writer database load. Each 30-second tick re-read the whole late-detail lookback (24 h of `audit_detail`) and probed the GIN array index once per detail (18.4 s and 1.78M buffer hits per tick on a 120k-detail production database), and re-scanned every settled audit header for the unchained anti-join. Coverage now lives in `worm_audit_chain_detail` (`worm-audit:13`): one insert-only row per covered detail, written by the chain trigger in the same statement as its link and backfilled from the existing chain, so the late-detail check is a primary-key probe. Each tick also reads only ids above per-JVM sweep floors (rows settled since the last complete sweep); after a restart and once an hour a tick sweeps everything, as before. Link hash inputs, `detail_ids`, the verifier and exports are unchanged.
- The chain trigger's overlap check (`EXISTS ... && ...`) ran as a cached generic plan that sequentially scanned all of `worm_audit_chain` for every inserted link (55k sequential scans, 1.2B tuples read in production, about 80 ms per link and growing). The trigger now inserts the coverage rows and lets their primary key reject a detail covered twice.
- Verification counts detail coverage by aggregating the chain's own detail lists once instead of one index probe per detail, and breaks when the coverage table disagrees with the chain.
+6
View File
@@ -137,6 +137,12 @@ Open `/app/worm_audit` (or `/app/admin/worm_audit` with `parentAppName="admin"`)
The full-width `wormAuditHome` HTML widget uses the standard `/qqq/v1/widget` API.
Hosts can add `wormAuditHealthCard` to their own dashboard. Both widgets and the app
require read permission on at least one WORM table, using the host's table permission rules.
To embed WORM in an existing host app, set `withIncludeApp(false)` and configure
`appName`, `appLabel` and optional `parentAppName` for that host app. The host owns
its navigation, home widget placement and app help; the qbit does not create or replace
the app or fill its help. Both widgets and all table help, permissions, guards,
processes and schedules remain registered. The health card links to the configured
host app. Standalone app creation remains enabled by default.
The aggregate home shows only identifiers, counts, times, shortened hashes and reasons;
it never renders audit messages or field values. Record links retain the target table's permissions.
+20 -2
View File
@@ -469,8 +469,8 @@ BEGIN
END;
$fn$;
-- The array index worm_audit_chain_detail_coverage is no longer used by this release. It stays until a later
-- changeset drops it, so a previous-release writer still running during a rolling deploy keeps its indexed probe.
-- The array index worm_audit_chain_detail_coverage is no longer used by this release. It stays until
-- a later migration drops it, so a previous-release writer still running during a rolling deploy keeps its indexed probe.
DROP TRIGGER IF EXISTS worm_audit_row_guard ON public.worm_audit_chain_detail;
CREATE TRIGGER worm_audit_row_guard BEFORE UPDATE OR DELETE ON public.worm_audit_chain_detail
@@ -482,6 +482,16 @@ REVOKE UPDATE, DELETE, TRUNCATE ON public.worm_audit_chain_detail FROM "app";
GRANT SELECT, INSERT ON public.worm_audit_chain_detail TO "app";
-- Same function as worm-audit:5, except that coverage rows are locked and purged before their links.
-- The migrating role may have revoked its own EXECUTE (a runtime role that owns the schema, as on CNPG);
-- PostgreSQL then refuses CREATE OR REPLACE. Grant it back for this statement only, then restore.
DO $$
BEGIN
IF NOT has_function_privilege(current_user, 'public.worm_audit_purge(timestamptz)', 'EXECUTE') THEN
EXECUTE format('GRANT EXECUTE ON FUNCTION public.worm_audit_purge(timestamptz) TO %I', current_user);
PERFORM set_config('worm_audit.purge_regrant', 'on', true);
END IF;
END;
$$;
CREATE OR REPLACE FUNCTION public.worm_audit_purge(p_cutoff timestamptz) RETURNS void
LANGUAGE plpgsql SECURITY DEFINER SET search_path = pg_catalog, public, pg_temp SET timezone = 'UTC' AS $fn$
DECLARE
@@ -536,3 +546,11 @@ BEGIN
VALUES(now(), session_user, p_cutoff, '6 years', 'PURGED', audits_count, details_count, links_count, boundary, anchor);
END;
$fn$;
DO $$
BEGIN
IF current_setting('worm_audit.purge_regrant', true) = 'on' THEN
EXECUTE format('REVOKE EXECUTE ON FUNCTION public.worm_audit_purge(timestamptz) FROM %I', current_user);
PERFORM set_config('worm_audit.purge_regrant', 'off', true);
END IF;
END;
$$;
@@ -48,6 +48,7 @@ public class WormAuditQBitConfig implements QBitConfig
private String exportLockMode = "GOVERNANCE";
private int exportRetentionDays = 30;
private final WormAuditSettings settings = new WormAuditSettings(this);
private boolean includeApp = true;
private String appName = "worm_audit";
private String appLabel = "WORM Audit";
private String parentAppName;
@@ -131,7 +132,26 @@ public class WormAuditQBitConfig implements QBitConfig
/*******************************************************************************
** Read the app name.
** Whether to create the standalone app. Widgets remain available when false.
*******************************************************************************/
public boolean getIncludeApp()
{
return (includeApp);
}
/*******************************************************************************
** Disable standalone app creation when the host owns the Audit app and help.
*******************************************************************************/
public WormAuditQBitConfig withIncludeApp(boolean includeApp)
{
this.includeApp = includeApp;
return (this);
}
/*******************************************************************************
** Read the app name (the host-owned destination when includeApp is false).
*******************************************************************************/
public String getAppName()
{
@@ -214,7 +214,10 @@ public class WormAuditQBitProducer implements QBitMetaDataProducer<WormAuditQBit
{
qInstance.addScheduler(new QuartzSchedulerMetaData().withName(qBitConfig.getSchedulerName()));
}
output.add(WormAuditApp.app(qBitConfig, producedTables));
if(qBitConfig.getIncludeApp())
{
output.add(WormAuditApp.app(qBitConfig, producedTables));
}
for(QWidgetMetaData widget : WormAuditApp.widgets())
{
output.add(widget);
@@ -115,7 +115,7 @@ public final class WormAuditHelp
table.withHelpContent(SLOT, content(entry.getValue()));
}
}
else if(key.startsWith("app:"))
else if(config.getIncludeApp() && key.startsWith("app:"))
{
QAppMetaData app = qInstance.getApp(config.getAppName());
if(app != null && (app.getHelpContent() == null || !app.getHelpContent().containsKey(SLOT)))
@@ -469,8 +469,8 @@ BEGIN
END;
$fn$;
-- The array index worm_audit_chain_detail_coverage is no longer used by this release. It stays until a later
-- changeset drops it, so a previous-release writer still running during a rolling deploy keeps its indexed probe.
-- The array index worm_audit_chain_detail_coverage is no longer used by this release. It stays until
-- a later migration drops it, so a previous-release writer still running during a rolling deploy keeps its indexed probe.
DROP TRIGGER IF EXISTS worm_audit_row_guard ON public.worm_audit_chain_detail;
CREATE TRIGGER worm_audit_row_guard BEFORE UPDATE OR DELETE ON public.worm_audit_chain_detail
@@ -482,6 +482,16 @@ REVOKE UPDATE, DELETE, TRUNCATE ON public.worm_audit_chain_detail FROM ${app_rol
GRANT SELECT, INSERT ON public.worm_audit_chain_detail TO ${app_role};
-- Same function as worm-audit:5, except that coverage rows are locked and purged before their links.
-- The migrating role may have revoked its own EXECUTE (a runtime role that owns the schema, as on CNPG);
-- PostgreSQL then refuses CREATE OR REPLACE. Grant it back for this statement only, then restore.
DO $$
BEGIN
IF NOT has_function_privilege(current_user, 'public.worm_audit_purge(timestamptz)', 'EXECUTE') THEN
EXECUTE format('GRANT EXECUTE ON FUNCTION public.worm_audit_purge(timestamptz) TO %I', current_user);
PERFORM set_config('worm_audit.purge_regrant', 'on', true);
END IF;
END;
$$;
CREATE OR REPLACE FUNCTION public.worm_audit_purge(p_cutoff timestamptz) RETURNS void
LANGUAGE plpgsql SECURITY DEFINER SET search_path = pg_catalog, public, pg_temp SET timezone = 'UTC' AS $fn$
DECLARE
@@ -536,3 +546,11 @@ BEGIN
VALUES(now(), session_user, p_cutoff, '${retention}', 'PURGED', audits_count, details_count, links_count, boundary, anchor);
END;
$fn$;
DO $$
BEGIN
IF current_setting('worm_audit.purge_regrant', true) = 'on' THEN
EXECUTE format('REVOKE EXECUTE ON FUNCTION public.worm_audit_purge(timestamptz) FROM %I', current_user);
PERFORM set_config('worm_audit.purge_regrant', 'off', true);
END IF;
END;
$$;
@@ -167,6 +167,9 @@ class CanonicalAndCryptoTest
java.nio.file.Files.readString(
java.nio.file.Path.of("docs/ddl/worm-audit-postgres.sql")));
assertThat(sql).doesNotContain("${");
// Liquibase parses every line starting with "-- changeset" as a directive, comments included.
assertThat(sql.lines().filter(line -> line.startsWith("-- changeset")))
.allMatch(line -> line.matches("-- changeset worm-audit:[0-9]+( splitStatements:false)?"));
assertThat(sql).contains("GET DIAGNOSTICS stack = PG_CONTEXT", "function worm_audit_purge(timestamp with time zone)", "SET timezone = 'UTC'");
// The documented migration contract deliberately uses physical snake_case columns.
assertThat(sql).contains("OLD.audit_id", "OLD.timestamp", "NEW.name", "GRANT UPDATE (label, create_date, modify_date)");
@@ -278,6 +278,29 @@ class WormAuditHomeTest
}
/*******************************************************************************
** Embedded widgets retain direct authorization without a standalone app.
*******************************************************************************/
@Test
void embeddedWidgetsRequireReadPermission() throws Exception
{
var config = ChainVerificationTest.signed().withIncludeApp(false).withAppName("audit").withAppLabel("Audit");
var instance = init(config);
assertThat(instance.getApp("worm_audit")).isNull();
assertThat(instance.getApp("audit")).isNull();
var input = new RenderWidgetInput().withWidgetMetaData(instance.getWidget(WormAuditApp.HEALTH_CARD_WIDGET));
QContext.getQSession().withPermissions("admin.hasAccess");
assertThatThrownBy(() -> new WormAuditApp.HealthCardRenderer().render(input)).isInstanceOf(QPermissionDeniedException.class);
assertThatThrownBy(() -> new WormAuditApp.HomeRenderer().render(input)).isInstanceOf(QPermissionDeniedException.class);
QContext.getQSession().withPermissions(WormAuditTables.CHAIN + ".read");
String card = ((RawHTML) new WormAuditApp.HealthCardRenderer().render(input).getWidgetData()).getHtml();
assertThat(card).contains("href=\"/app/audit\"");
assertThat(new WormAuditApp.HomeRenderer().render(input).getWidgetData()).isInstanceOf(RawHTML.class);
QContext.setQSession(new QSession());
assertThatThrownBy(() -> new WormAuditApp.HealthCardRenderer().render(input)).isInstanceOf(QPermissionDeniedException.class);
}
/*******************************************************************************
** Exercise the real SQL anti-join without Docker, including its exact boundary.
*******************************************************************************/
@@ -21,10 +21,15 @@
package com.kingsrook.qbits.wormaudit;
import java.util.List;
import com.kingsrook.qbits.wormaudit.app.WormAuditApp;
import com.kingsrook.qbits.wormaudit.model.WormAuditHelp;
import com.kingsrook.qbits.wormaudit.ddl.WormAuditDdl;
import com.kingsrook.qbits.wormaudit.model.WormAuditTables;
import com.kingsrook.qbits.wormaudit.processes.WormAuditProcesses;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryBackendModule;
import com.kingsrook.qqq.backend.core.model.metadata.tables.Capability;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
@@ -43,6 +48,7 @@ class WormAuditQBitProducerTest
{
var config = new WormAuditQBitConfig();
QInstance instance = WormAuditTestSupport.init(config);
assertThat(config.getIncludeApp()).isTrue();
assertThat(config.getRetentionYears()).isEqualTo(6);
assertThat(config.getTableMetaDataCustomizer()).isNull();
assertThat(config.getSigningKey()).isNull();
@@ -160,4 +166,36 @@ class WormAuditQBitProducerTest
() -> WormAuditDdl.resolveNames(new WormAuditQBitConfig(), new QInstance()))
.isInstanceOf(IllegalArgumentException.class);
}
/*******************************************************************************
** Embedded production keeps the host app intact and registers all evidence metadata.
*******************************************************************************/
@Test
void embeddedMetadataPreservesHostApp() throws Exception
{
var config = new WormAuditQBitConfig().withBackendName("memory").withIncludeApp(false).withAppName("audit");
var instance = WormAuditTestSupport.core(new QBackendMetaData().withName("memory").withBackendType(MemoryBackendModule.class));
var host = new QAppMetaData().withName("audit").withLabel("Combined Audit").withWidgets(List.of("hostSummary"));
instance.addApp(host);
new WormAuditQBitProducer().withQBitConfig(config).produce(instance);
assertThat(config.getIncludeApp()).isFalse();
assertThat(instance.getApp("worm_audit")).isNull();
assertThat(instance.getApp("audit")).isSameAs(host);
assertThat(host.getLabel()).isEqualTo("Combined Audit");
assertThat(host.getWidgets()).containsExactly("hostSummary");
assertThat(host.getHelpContent()).isNullOrEmpty();
assertThat(instance.getWidget(WormAuditApp.HEALTH_CARD_WIDGET)).isNotNull();
assertThat(instance.getWidget(WormAuditApp.HOME_WIDGET)).isNotNull();
for(String name : WormAuditTables.ALL)
{
assertThat(instance.getTable(name).getHelpContent()).containsKey(WormAuditHelp.SLOT);
assertThat(instance.getTable(name).getPermissionRules().getPermissionBaseName()).isEqualTo(name);
}
assertThat(instance.getTable(WormAuditTables.CHAIN).getDisabledCapabilities()).contains(Capability.TABLE_UPDATE, Capability.TABLE_DELETE, Capability.TABLE_INSERT);
assertThat(instance.getTable("audit").getDisabledCapabilities()).contains(Capability.TABLE_UPDATE, Capability.TABLE_DELETE);
assertThat(instance.getProcess(WormAuditProcesses.VERIFY)).isNotNull();
assertThat(instance.getProcess(WormAuditProcesses.CHAIN).getSchedule().getRepeatSeconds()).isEqualTo(config.getChainIntervalSeconds());
assertThat(instance.getProcess(WormAuditProcesses.DAILY).getSchedule().getCronExpression()).isEqualTo(config.getVerifyCron());
}
}