Compare commits

..
Author SHA1 Message Date
james.maes 46afa2e533 Merge pull request 'ci: build on the shared ci-workflows container workflow' (#47) from ci/ci-workflows into develop
container / build (push) Successful in 7m6s
build / container (push) Successful in 7m6s
2026-11-07 20:21:12 +00:00
James MaesandClaude Opus 5.5 0740fab027 build(image): apply Ubuntu security updates in the runtime image
container / build (push) Successful in 6m57s
build / container (push) Successful in 6m57s
The eclipse-temurin:21-jre-jammy base ships apt 2.4.14 (HIGH
CVE-2026-109518, fixed in 2.4.15ubuntu0.1), which the CI Trivy gate
blocks. Upgrade installed packages when building the image.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 14:13:31 -06:00
James MaesandClaude Opus 5.5 30820a6d4f ci: keep the script tests green on Gitea CI
container / build (push) Failing after 7m28s
build / container (push) Failing after 7m28s
The CI-contract test now checks the Gitea workflow smokes the Maven
final jar (the orb .qqq-ci.yml is gone). The release-bundle test renders
compose.yaml with docker compose config, so CI installs a checksum-pinned
Compose binary behind a docker compose-only shim.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 14:05:00 -06:00
James MaesandClaude Opus 5.5 22bb23613f ci: one command per line for the Python install
container / build (push) Failing after 26s
build / container (push) Failing after 26s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 14:03:38 -06:00
James MaesandClaude Opus 5.5 0e80a5e736 ci: install checksum-pinned Python for the smoke scripts
container / build (push) Failing after 24s
build / container (push) Failing after 24s
The job image runs unprivileged without python3, so apt is unavailable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 14:03:07 -06:00
James MaesandClaude Opus 5.5 2b04d936cf ci: build on the shared ci-workflows container workflow
container / build (push) Failing after 16s
build / container (push) Failing after 16s
Replace the retired CircleCI qqq-orb pipeline and the GitHub release
workflow (qqq-orb caller) with KofTwentyTwo/ci-workflows container.yaml@v1.
Every push runs the script unit tests, mvn verify and the packaged core
HTTP smoke check, then builds and Trivy-scans the image; develop pushes
git.kof22.com/qrun-io/qqq-all (ci-<sha>-run-<n> and develop).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 14:01:29 -06:00
james.maes 23b67c51f7 Merge pull request 'ci: remove workflows that call the retired CI system' (#46) from retire/old-ci into develop 2026-11-07 13:29:05 +00:00
james.maesandClaude Opus 5.5 a710d56e8c ci: remove workflows that call the retired CI system
Owner decision 2026-11-07: only KofTwentyTwo/ci-workflows remains.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 13:29:03 +00:00
forge-recovery 006dbb4600 ci: enroll original aggregate container lifecycle
Original QQQ all container lifecycle / release (push) Skipped
QRun-IO/ci-policy: ci / fork-guard (push) Skipped
QRun-IO/ci-policy: ci / application-template-0 (push) Skipped
QRun-IO/ci-policy: ci / gradle-1 (push) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (push) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (push) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-6 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (push) Skipped
QRun-IO/ci-policy: ci / node-8 (push) Skipped
QRun-IO/ci-policy: ci / node-9 (push) Skipped
QRun-IO/ci-policy: ci / node-10 (push) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (push) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (push) Skipped
QRun-IO/ci-policy: ci / source-config-13 (push) Skipped
QRun-IO/ci-policy: ci / source-config-14 (push) Skipped
QRun-IO/ci-policy: ci / android-0 (push) Skipped
QRun-IO/ci-policy: ci / original-business (push) Skipped
QRun-IO/ci-policy: ci / original-mcp (push) Skipped
QRun-IO/ci-policy: ci / original-voyage (push) Skipped
QRun-IO/ci-policy: ci / not-enrolled (push) Skipped
QRun-IO/ci-policy: native / fork-deny (push) Skipped
QRun-IO/ci-policy: native / dotnet-0 (push) Skipped
QRun-IO/ci-policy: native / native-result (push) Successful in 5s
QRun-IO/ci-policy: ci / head-profile (push) Successful in 11s
QRun-IO/ci-policy: ci / repository-policy (push) Skipped
QRun-IO/ci-policy: ci / java-core-2-head (push) Skipped
QRun-IO/ci-policy: ci / java-docker-3-head (push) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (push) Skipped
Shared QQQ all CI (original verify-and-smoke and develop image capture) / build-and-test (push) Successful in 4m19s
QRun-IO/ci-policy: Shared QQQ all CI (original verify-and-smoke and develop image capture) / build-and-test (push) Successful in 4m9s
Shared QQQ all CI (original verify-and-smoke and develop image capture) / core (push) Successful in 30s
QRun-IO/ci-policy: Shared QQQ all CI (original verify-and-smoke and develop image capture) / core (push) Successful in 32s
Shared QQQ all CI (original verify-and-smoke and develop image capture) / full (push) Successful in 8m25s
QRun-IO/ci-policy: Shared QQQ all CI (original verify-and-smoke and develop image capture) / full (push) Successful in 8m20s
Shared QQQ all CI (original verify-and-smoke and develop image capture) / Develop Container (build and retain the OCI image for the trusted publisher) (push) Successful in 2m34s
Shared QQQ all CI (original verify-and-smoke and develop image capture) / gate (push) Successful in 9s
Original QQQ all container lifecycle / develop (push) Successful in 15m34s
QRun-IO/ci-policy: Shared QQQ all CI (original verify-and-smoke and develop image capture) / Develop Container (build and retain the OCI image for the trusted publisher) (push) Successful in 3m19s
QRun-IO/ci-policy: Shared QQQ all CI (original verify-and-smoke and develop image capture) / gate (push) Successful in 9s
QRun-IO/ci-policy: ci / java-maven-7-head (push) Successful in 16m3s
QRun-IO/ci-policy: ci / ci-result (push) Successful in 3s
2026-11-04 04:27:58 +00:00
forge-recoveryandCodex 7a39277072 ci: enroll original aggregate container lifecycle
QRun-IO/ci-policy: ci / fork-guard (pull_request) Skipped
QRun-IO/ci-policy: ci / application-template-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / gradle-1 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-8 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-9 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-10 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-13 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-14 (pull_request) Skipped
QRun-IO/ci-policy: ci / android-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-business (pull_request) Skipped
QRun-IO/ci-policy: ci / original-mcp (pull_request) Skipped
QRun-IO/ci-policy: ci / original-voyage (pull_request) Skipped
QRun-IO/ci-policy: ci / not-enrolled (pull_request) Skipped
QRun-IO/ci-policy: native / fork-deny (pull_request) Skipped
QRun-IO/ci-policy: native / dotnet-0 (pull_request) Skipped
QRun-IO/ci-policy: native / native-result (pull_request) Successful in 4s
QRun-IO/ci-policy: ci / head-profile (pull_request) Successful in 12s
QRun-IO/ci-policy: ci / repository-policy (pull_request) Skipped
QRun-IO/ci-policy: ci / java-core-2-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (pull_request) Skipped
QRun-IO/ci-policy: Shared QQQ all CI (original verify-and-smoke and develop image capture) / build-and-test (pull_request) Successful in 4m24s
QRun-IO/ci-policy: Shared QQQ all CI (original verify-and-smoke and develop image capture) / core (pull_request) Successful in 28s
QRun-IO/ci-policy: Shared QQQ all CI (original verify-and-smoke and develop image capture) / full (pull_request) Successful in 7m39s
QRun-IO/ci-policy: Shared QQQ all CI (original verify-and-smoke and develop image capture) / Develop Container (build and retain the OCI image for the trusted publisher) (pull_request) Skipped
QRun-IO/ci-policy: Shared QQQ all CI (original verify-and-smoke and develop image capture) / gate (pull_request) Successful in 8s
QRun-IO/ci-policy: ci / java-maven-7-head (pull_request) Successful in 12m18s
QRun-IO/ci-policy: ci / ci-result (pull_request) Successful in 4s
Co-Authored-By: Codex <noreply@openai.com>
2026-11-04 04:09:57 +00:00
james.maesandClaude Fable 5.1 778a5d904c Merge chore/drop-guarded-column-stats: core guards column stats (#1367); full-profile reportStorage SFTP details
QRun-IO/ci-policy: Shared Java Maven CI / gate (push) Failing after 9s
QRun-IO/ci-policy: ci / java-maven-2 (push) Failing after 18m48s
QRun-IO/ci-policy: ci / ci-result (push) Failing after 5s
QRun-IO/ci-policy: ci / fork-guard (push) Skipped
QRun-IO/ci-policy: ci / gradle-0 (push) Skipped
QRun-IO/ci-policy: ci / java-docker-1 (push) Skipped
QRun-IO/ci-policy: ci / node-3 (push) Skipped
QRun-IO/ci-policy: ci / node-4 (push) Skipped
QRun-IO/ci-policy: ci / source-config-5 (push) Skipped
QRun-IO/ci-policy: ci / not-enrolled (push) Skipped
QRun-IO/ci-policy: Shared Java Maven CI / verify (push) Failing after 2m28s
QRun-IO/ci-policy: Shared Java Maven CI / secrets (push) Successful in 1m12s
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 17:35:15 -06:00
james.maesandClaude Fable 5.1 a2200d736e chore(enterprise): drop GuardedColumnStatsStep now that core guards column stats
QRun-IO/qqq#1367 is fixed in core (2ac4fda45): ColumnStatsStep always
refuses statistics for a field the caller cannot read in full and tags
its result records with the analysed table. The host's guarded subclass
and the code swap in EnterpriseFeatures are gone; the enterprise profile
keeps registering the plain columnStats process.

GuardedColumnStatsStepTest becomes EnterpriseColumnStatsAccessTest and
now proves the same through core's step: a field behind a field security
lock is refused without the key, returns statistics (tagged with the
table) with it, and an unrestricted field is allowed.

The same core sweep made filesystem tables without backend details fail
validation (qqq#1364), which broke the full profile: its reportStorage on
the SFTP backend had none. It now gets one-file-per-record SFTP details
(reportStorage/, reference + contents), as the enterprise profile's local
files backend already did.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 17:32:56 -06:00
james.maesandClaude Fable 5.1 13408c3cfa Merge fix/enterprise-custom-fields-403: admin can read and manage Custom Fields
QRun-IO/ci-policy: ci / fork-guard (push) Skipped
QRun-IO/ci-policy: ci / gradle-0 (push) Skipped
QRun-IO/ci-policy: ci / java-docker-1 (push) Skipped
QRun-IO/ci-policy: ci / node-3 (push) Skipped
QRun-IO/ci-policy: ci / node-4 (push) Skipped
QRun-IO/ci-policy: ci / source-config-5 (push) Skipped
QRun-IO/ci-policy: ci / not-enrolled (push) Skipped
QRun-IO/ci-policy: Shared Java Maven CI / gate (push) Canceled after 0s
QRun-IO/ci-policy: Shared Java Maven CI / secrets (push) Canceled after 0s
QRun-IO/ci-policy: Shared Java Maven CI / verify (push) Canceled after 0s
QRun-IO/ci-policy: ci / java-maven-2 (push) Canceled after 0s
QRun-IO/ci-policy: ci / ci-result (push) Canceled after 0s
Fixes #47. The enterprise session gets the collaboration permissions that FieldSecurity checks by name, and keeps the frontend user value on customized sessions. 37 tests green.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 17:27:22 -06:00
james.maesandClaude Fable 5.1 1c4a8cc115 fix(enterprise): keep the frontend user value on customized sessions
39c28a4 moved the enterprise grants into DemoUsers.grantEnterpriseAccess
and dropped the session.withValueForFrontend("user", …) line between
the two enterprise blocks, so manageSession answered {} and clients
showed no signed-in user name. Restored, with a test that a customized
session still carries the user for the frontend.

Refs #47

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 16:35:47 -06:00
james.maesandClaude Fable 5.1 39c28a4e00 fix(enterprise): let the admin read and manage collaboration Custom Fields
The Custom Fields table (collaboration_field_definition) answered 403
"This needs collaboration.view." to the all-access admin. Its permission
rules are NOT_PROTECTED with qbit-collaboration's FieldSecurity checker,
which tests collaboration.view / collaboration.admin (and
collaboration.fields.manage for writes and processes) by literal name, so
PermissionsHelper.getAllAvailablePermissionNames never lists them; the
same pattern as the jobs and assistant names the profile already grants.

The enterprise session now also holds collaboration.view, .admin,
.fields.manage and .tags.manage. The grants moved into
DemoUsers.grantEnterpriseAccess so EnterpriseAccessTest can run the real
FieldSecurity checker against them: refused without, read and insert
allowed with.

GuardedColumnStatsStep gets a TODO to delete it once QRun-IO/qqq#1367
lands in core.

Fixes #47

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 16:26:11 -06:00
james.maesandClaude Fable 5.1 8015b8b090 feat(enterprise): store sessions for mock sign-ins (#52)
QRun-IO/ci-policy: ci / fork-guard (push) Skipped
QRun-IO/ci-policy: ci / gradle-0 (push) Skipped
QRun-IO/ci-policy: ci / java-docker-1 (push) Skipped
QRun-IO/ci-policy: ci / node-3 (push) Skipped
QRun-IO/ci-policy: ci / node-4 (push) Skipped
QRun-IO/ci-policy: ci / source-config-5 (push) Skipped
QRun-IO/ci-policy: ci / not-enrolled (push) Skipped
QRun-IO/ci-policy: Shared Java Maven CI / gate (push) Canceled after 0s
QRun-IO/ci-policy: Shared Java Maven CI / secrets (push) Canceled after 0s
QRun-IO/ci-policy: Shared Java Maven CI / verify (push) Canceled after 0s
QRun-IO/ci-policy: ci / java-maven-2 (push) Canceled after 0s
QRun-IO/ci-policy: ci / ci-result (push) Canceled after 0s
Only the OAuth2 module wrote to the session store; mock sign-in built a fresh session per request and stored nothing, so the terminal clients' Online count could not see signed-in terminals. The profile's mock session customizer now records each session uuid once through qbit-session-store (identity only, expiry) and touches it at most once a minute.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 13:34:49 -06:00
james.maesandClaude Fable 5.1 2c8abf6a64 feat(enterprise): the assistant answers, with Claude when keyed or a local Ollama (#51)
QRun-IO/ci-policy: ci / fork-guard (push) Skipped
QRun-IO/ci-policy: ci / gradle-0 (push) Skipped
QRun-IO/ci-policy: ci / java-docker-1 (push) Skipped
QRun-IO/ci-policy: ci / node-3 (push) Skipped
QRun-IO/ci-policy: ci / node-4 (push) Skipped
QRun-IO/ci-policy: ci / source-config-5 (push) Skipped
QRun-IO/ci-policy: ci / not-enrolled (push) Skipped
CodeQL / analyze (push) Canceled after 0s
QBit CodeQL / Analyze branch source (push) Canceled after 0s
QRun-IO/ci-policy: Shared Java Maven CI / verify (push) Failing after 2m27s
QRun-IO/ci-policy: Shared Java Maven CI / secrets (push) Successful in 13s
QRun-IO/ci-policy: Shared Java Maven CI / gate (push) Failing after 12s
QRun-IO/ci-policy: ci / java-maven-2 (push) Failing after 15m16s
QRun-IO/ci-policy: ci / ci-result (push) Failing after 5s
EnterpriseAssistant registers both provider types and keeps two provider rows, Anthropic (Claude) when ANTHROPIC_API_KEY is set (referenced by env-var name, never stored) else Local Ollama when it answers on 127.0.0.1:11434, else NOT_CONFIGURED; enterprise sessions hold assistant.hasAccess and assistant.admin; timeouts allow a local 70B model.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 11:30:09 -06:00
james.maesandClaude Fable 5.1 fb70b5f7c2 feat(enterprise): quick search covers every demo and sample table (#50)
Registers every demo table and every visible sample table with a record label through qbit-quick-search 1.1's own registration and self-priming (STRING fields only: label fields plus the obvious text fields), labels unlabelled tables by name, and shortens the index sync to 60 s so new tables build at startup. 17 tables, 57 documents on the enterprise profile; 'Illinois' and 'ada' both resolve.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 10:48:41 -06:00
james.maesandClaude Fable 5.1 d91d996289 feat(enterprise): column statistics, field audits, object storage on MinIO, guarded columnStats (#49)
Turns QUERY_STATS on for backends that can aggregate (H2, SQLite, the sample's in-memory H2, memory), registers the columnStats process behind a GuardedColumnStatsStep that refuses fields the caller cannot read in full and tags results with the analysed table so field-access rules apply (core follow-up: QRun-IO/qqq#1367), adds field-level audits to orderLine and the sample tables, and installs the Object Storage QBit on MinIO when S3_ENDPOINT is set so collaboration attachments work.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 07:01:13 -06:00
james.maesandClaude Fable 5.1 74837f5db2 feat: enterprise profile with the full QBit set for terminal-client testing (#48)
Adds QQQ_ALL_PROFILE=enterprise composing the QBit set an enterprise rollout would run (jobs, collaboration, alerts, data-analysis, db-performance, sql-workbench, help, release-notes, qa-capture, secured-data, email, slack, cluster, cdn, backups, file-gateway, session-store, worm-audit, assistant, quick-search 1.1.0, redis realtime, the core profile's seven QBits, the sample app, core audits/scripts/saved views/reports/bulk-load/sharing/query stats/durable jobs/health) with MOCK auth and an in-memory Quartz scheduler. Fixes found by the sweep: health advertised, calculateOrderTotal input form, record labels, mutable health indicators, quoted H2 reserved words, webhook test-event ordering workaround, jobs-permission workaround, quick-search admin permission for the full profile's smoke.

Closes #44, closes #45, closes #46. Refs #47, QRun-IO/qqq#1361, QRun-IO/qqq#1362.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 06:21:34 -06:00
james.maesandCodex 0252c6811b ci: adopt managed orb 0.8 patch track (#40)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 16:04:22 -05:00
james.maesandCodex 73bdcc1a6f ci: adopt shared orb 0.7.17 (#39)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 14:25:53 -05:00
james.maesandCodex d937b19050 ci: use central source scanning (#37)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 11:52:26 -05:00
james.maesandCodex 2b75c6beb9 ci: adopt central orb 0.7.12 and stable gates (#36)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 03:36:27 -05:00
james.maesandCodex 12ae927c33 ci: align canonical shared orb workers (#35)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 01:33:08 -05:00
james.maesandCodex 6bcc28e332 ci: adopt shared orb 0.7.11 (#34)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 00:58:52 -05:00
james.maesandCodex cc89f73680 ci: centralize All builds smoke checks and releases (#33)
* ci: centralize All builds smoke checks and releases

Use orb 0.7.9 and its immutable shared release worker. Build once, preserve core/full verification and conditional public GHCR delivery, and consume public Circle provenance without exporting a token.

Co-Authored-By: Codex <noreply@openai.com>

* fix: use shared SDK v2 S3 support for MinIO

Replace obsolete SDK v1 client overrides with upstream endpoint and path-style metadata. Preserve endpoint coverage for every filesystem action and local HTTP listing/write assertions.

Co-Authored-By: Codex <noreply@openai.com>

* test: verify the HttpOnly OIDC session contract

Current QQQ omits the session UUID from JSON for HttpOnly cookies. Require the nonempty HttpOnly cookie and reject exposed UUIDs, retaining authenticated backend and permission checks.

Co-Authored-By: Codex <noreply@openai.com>

* fix: accept leading-dash Mongo credentials in Compose

Use explicit username/password option assignments in health and replica-set initialization. Reproduced the original failure and verified authenticated ping with the corrected arguments against mongo:7 and a leading-dash fixture password.

Co-Authored-By: Codex <noreply@openai.com>

---------

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:56:33 -05:00
28 changed files with 1853 additions and 92 deletions

No files matched your search

-14
View File
@@ -1,14 +0,0 @@
version: 2.1
setup: true
orbs:
qqq-orb: kingsrook/qqq-orb@0.7.9
workflows:
setup:
jobs:
- qqq-orb/generate_pipeline:
orb_version: 0.7.9
filters:
branches:
ignore: automation/qbit-rc-pending
tags:
only: /.*/
+33
View File
@@ -0,0 +1,33 @@
name: build
on:
push:
jobs:
container:
uses: self:KofTwentyTwo/ci-workflows/.gitea/workflows/container.yaml@v1
secrets: inherit
with:
# Same gates as the old orb build-and-test + core smoke: script unit tests,
# the full reactor (BOM tests, packaged integration tests), then the packaged
# core jar smoked over HTTP. The Dockerfile copies that verified jar.
prebuild: |
set -euo pipefail
# The smoke scripts need Python 3 (stdlib only); the job image has none and no root for apt.
mkdir -p /tmp/python
curl -fsSLo /tmp/python.tgz https://github.com/astral-sh/python-build-standalone/releases/download/20261013/cpython-3.12.15%2B20261013-x86_64-unknown-linux-gnu-install_only.tar.gz
echo "5d546fc004fc2435fa142eb8f42ed8de5eb4725963dc22f7e6671014a80eb560 /tmp/python.tgz" | sha256sum -c -
tar -xzf /tmp/python.tgz -C /tmp/python --strip-components=1 && rm /tmp/python.tgz
# The release-bundle and full-smoke config tests render compose.yaml with `docker compose config`
# (no daemon). Provide the Compose binary behind a `docker compose`-only shim.
curl -fsSLo /tmp/python/bin/docker-compose https://github.com/docker/compose/releases/download/v5.6.1/docker-compose-linux-x86_64
echo "47737f4236e6d3ccea7291995203f8f018058783b4c3a9469119811a0b5db8c0 /tmp/python/bin/docker-compose" | sha256sum -c -
printf '%s\n' '#!/bin/sh' '[ "$1" = compose ] || { echo "only docker compose is installed" >&2; exit 1; }' 'shift' 'exec docker-compose "$@"' > /tmp/python/bin/docker
chmod +x /tmp/python/bin/docker-compose /tmp/python/bin/docker
export PATH=/tmp/python/bin:$PATH
python3 -m unittest discover -s scripts -p 'test_*.py' -v
mvn verify
python3 scripts/run_core_smoke.py qqq-all-app/target/qqq-all-app.jar
images: |
qqq-all|Dockerfile|.
# develop pushes git.kof22.com/qrun-io/qqq-all:ci-<sha>-run-<n> plus the moving develop tag.
publish-branch: develop
extra-tags: develop
+23
View File
@@ -0,0 +1,23 @@
name: CodeQL
'on':
push:
branches:
- main
- develop
- release/**
- hotfix/**
pull_request: {}
schedule:
- cron: 41 16 * * 0
workflow_dispatch: {}
permissions:
contents: read
actions: read
security-events: write
packages: read
concurrency:
group: codeql-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs:
analyze:
uses: QRun-IO/qqq-orb/.github/workflows/qbit-codeql.yml@c288a8864fe9054f68410e1d78f5863aea4c4d22
-53
View File
@@ -1,53 +0,0 @@
name: Release 4.1.0
'on':
workflow_dispatch:
inputs:
version:
description: Reviewed source version (GA or exact coordinated RC)
required: true
type: choice
default: 4.1.0
options:
- 4.1.0
- 4.1.0-RC.1
release_sha:
description: Full develop commit SHA already signed and tagged with the selected
version
required: true
type: string
smoke_run_id:
description: Successful Smoke workflow run ID for that exact develop SHA
required: false
type: string
default: ''
smoke_evidence:
description: Evidence provider; github preserves the original Smoke workflow
validation
type: choice
default: circle
options:
- github
- circle
required: true
circle_workflow_id:
description: Exact successful Circle workflow UUID when circle evidence is
selected
type: string
required: false
default: ''
permissions:
actions: read
contents: write
packages: write
checks: read
jobs:
release:
uses: QRun-IO/qqq-orb/.github/workflows/qqq-all-release.yml@65265e48a4176d8fb0f4f744e2919bad587bd5ca
with:
version: ${{ inputs.version }}
release_sha: ${{ inputs.release_sha }}
smoke_run_id: ${{ inputs.smoke_run_id }}
smoke_evidence: ${{ inputs.smoke_evidence }}
circle_workflow_id: ${{ inputs.circle_workflow_id }}
secrets:
CIRCLECI_TOKEN: ${{ secrets.CIRCLECI_TOKEN }}
-6
View File
@@ -1,6 +0,0 @@
profile: all
orb_version: 0.7.9
default_branch: develop
publish:
snapshot: true
publish_context: qqq-maven-registry-credentials
+1
View File
@@ -1,6 +1,7 @@
FROM eclipse-temurin:21-jre-jammy
RUN apt-get update \
&& apt-get upgrade -y --no-install-recommends \
&& apt-get install -y --no-install-recommends curl \
&& rm -rf /var/lib/apt/lists/* \
&& groupadd --system --gid 10001 qqq \
+2 -2
View File
@@ -41,9 +41,9 @@ Full adds sample tables on PostgreSQL, MySQL, MongoDB, MinIO S3, and SFTP. It in
To stop the full stack, run `docker compose --profile full down`. Add `--volumes` only when you intend to erase demo data. For a fresh Artemis volume, the configured login is required; old volumes created for anonymous access should be recreated. This reference stack uses demo credentials and is not a production deployment.
The [canonical QQQ pipeline](.circleci/config.yml), configured by [.qqq-ci.yml](.qqq-ci.yml), builds once and runs core/full smoke checks through the shared orb. Its develop-image job publishes to GHCR only after the container package exists and its visibility is public. Until then, image publication is skipped and local Compose remains available.
CI runs [.gitea/workflows/build.yaml](.gitea/workflows/build.yaml) through the shared `KofTwentyTwo/ci-workflows` container workflow: script unit tests, `mvn verify`, and the packaged core HTTP smoke check on every push, then the image build and Trivy scan. Pushes to `develop` publish `git.kof22.com/qrun-io/qqq-all` (`ci-<sha>-run-<n>` and `develop`). The full Compose smoke check runs locally with `python3 scripts/run_full_smoke.py`.
The [4.1.0 release procedure](docs/release.md) describes the separate Maven parent/BOM publication path and the manually gated, exact-commit [release workflow](.github/workflows/release.yml). The workflow will not release the current snapshot build.
The [4.1.0 release procedure](docs/release.md) describes the separate Maven parent/BOM publication path. The GitHub release workflow that called the retired qqq-orb was removed; a versioned release path on Gitea is not defined yet.
## Contributing and license
+1 -1
View File
@@ -1,6 +1,6 @@
# 4.1.0 release preparation
This procedure prepares a release; this repository currently remains on `4.1.0-SNAPSHOT`. The [manual release workflow](../.github/workflows/release.yml) publishes the versioned GHCR image and a GitHub release containing an image-based Compose bundle plus `compose.yaml`. It does **not** deploy Maven artifacts or create a tag. Publication is authorized once the release gates pass. Keep the workflow disabled until Apache-2.0 alignment, public artifact verification and GHCR visibility are complete.
This procedure prepares a release; this repository currently remains on `4.1.0-SNAPSHOT`. The GitHub manual release workflow (GHCR image plus Compose bundle, via the retired qqq-orb) was removed with the move to Gitea CI; `develop` images now publish to `git.kof22.com/qrun-io/qqq-all`. A versioned release path on Gitea is not defined yet.
## Exact coordinated RC under QQQ #921
@@ -0,0 +1,13 @@
-- Quick Search 1.x protects its admin processes (quickSearchFullReindex and friends) and its
-- admin app with the quick_search_admin permission; the full-profile admin runs the reindex.
INSERT INTO permission (name, object_type, object_label)
VALUES
('quick_search_admin.hasAccess', 'process', 'Quick Search Admin')
ON CONFLICT (name) DO NOTHING;
INSERT INTO role_permission_int (role_id, permission_id)
SELECT role.id, permission.id
FROM role CROSS JOIN permission
WHERE permission.name = 'quick_search_admin.hasAccess'
AND role.name = 'admin'
ON CONFLICT (role_id, permission_id) DO NOTHING;
+16
View File
@@ -15,6 +15,18 @@ Core uses mock authentication. The `sessionId` cookie selects the two fixed demo
The core demo seeds a customer table view, a shipping address linked to the geo country/state/city tables, and an order review workflow. The `RunRecordWorkflow` process can run that workflow against an order; `syncOrder` writes a process trace after an order event. It also seeds an active `orderStored` webhook subscription to its receiver at `/demo/order-webhook`. Insert an order as Admin, run `SendWebhookEvent` as Admin, and inspect `/demo/order-webhook-receipts` to see the delivered payload. Demo User can read webhook records but cannot edit their configuration or invoke the delivery process. The receiver is available only in the core profile, which Compose publishes on host loopback.
## Enterprise profile: the assistant
`QQQ_ALL_PROFILE=enterprise` (the terminal-client test server, started by qqq-cli's
`scripts/enterprise-server.sh`) installs qbit-assistant with two model provider rows, reconciled at every
start: **Anthropic (Claude)** (key variable `ANTHROPIC_API_KEY`; only the name is stored) and **Local
Ollama** (OpenAI-compatible, kind OLLAMA, `QQQ_ASSISTANT_OLLAMA_URL` default `http://127.0.0.1:11434/v1`,
model `QQQ_ASSISTANT_OLLAMA_MODEL` default `llama3.3:70b-instruct-q4_K_M`, no key). The default is
Anthropic when `ANTHROPIC_API_KEY` is set, else Ollama when it answers and lists the model, else neither
(the assistant reports `NOT_CONFIGURED` with what to configure). Every enterprise session holds
`assistant.hasAccess` and `assistant.admin`. A turn may take 900 s and one Ollama call 600 s, for a local
70B model.
## Full profile
Set `QQQ_ALL_PROFILE=full` to use external services. The full profile requires an explicit `QQQ_ALL_BIND_HOST`; use `127.0.0.1` for local-only access or deliberately set another interface when a reverse proxy needs it. `QQQ_ALL_PORT` defaults to `8080`. The full profile replaces core's mock authentication with Keycloak OIDC. It does not start an embedded broker.
@@ -108,6 +120,10 @@ For an existing volume, apply the idempotent grants and then sign in again:
COMPOSE_PROFILES=full docker compose exec -T postgres sh -c 'psql -v ON_ERROR_STOP=1 -U "$POSTGRES_USER" -d "$POSTGRES_DB"' < infra/postgres/05-runtime-demo-permissions.sql
```
Quick Search 1.x protects `quickSearchFullReindex` and its other admin processes with
`quick_search_admin.hasAccess`; fresh volumes grant it to the admin role through
`infra/postgres/06-quick-search-permissions.sql`. Apply that file the same way on an existing volume.
`mvn -pl qqq-all-app -am verify` includes native local API, calculation/permission,
memory restart, and packaged-JAR checks. `scripts/run_core_smoke.py` and
`scripts/run_full_smoke.py` verify these routes with actual Admin and restricted
+25 -1
View File
@@ -42,13 +42,37 @@
<dependency><groupId>com.kingsrook.qqq</groupId><artifactId>qqq-esb</artifactId></dependency>
<dependency><groupId>com.kingsrook.qqq</groupId><artifactId>qqq-frontend-next</artifactId></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-user-role-permissions</artifactId></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-quick-search</artifactId></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-quick-search</artifactId><version>1.1.0-SNAPSHOT</version></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-customizable-table-views</artifactId></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-standard-process-trace</artifactId></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-webhooks</artifactId></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-workflows</artifactId></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-geo-data</artifactId></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-sftp-data-integration</artifactId></dependency>
<!-- Enterprise profile (QQQ_ALL_PROFILE=enterprise): every locally runnable qbit, for terminal-client testing. -->
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-alerts</artifactId><version>0.1.0-SNAPSHOT</version></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-assistant</artifactId><version>0.1.0-SNAPSHOT</version></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-assistant-provider-anthropic</artifactId><version>0.1.0-SNAPSHOT</version></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-assistant-provider-openai</artifactId><version>0.1.0-SNAPSHOT</version></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-backups</artifactId><version>0.1.0-SNAPSHOT</version></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-cdn</artifactId><version>0.1.0-SNAPSHOT</version></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-cluster</artifactId><version>0.1.0-SNAPSHOT</version></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-collaboration</artifactId><version>0.1.0-SNAPSHOT</version></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-data-analysis</artifactId><version>0.1.0-SNAPSHOT</version></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-db-performance</artifactId><version>0.1.0-SNAPSHOT</version></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-email</artifactId><version>0.1.0-SNAPSHOT</version></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-file-gateway</artifactId><version>0.1.0-SNAPSHOT</version></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-help</artifactId><version>0.1.0-SNAPSHOT</version></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-jobs</artifactId><version>0.1.0-SNAPSHOT</version></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-object-storage</artifactId><version>0.1.0-SNAPSHOT</version></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-qa-capture</artifactId><version>0.1.0-SNAPSHOT</version></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-redis</artifactId><version>0.1.0-SNAPSHOT</version></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-release-notes</artifactId><version>0.1.0-SNAPSHOT</version></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-secured-data</artifactId><version>0.1.0-SNAPSHOT</version></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-session-store</artifactId><version>0.1.0-SNAPSHOT</version></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-slack</artifactId><version>0.1.0-SNAPSHOT</version></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-sql-workbench</artifactId><version>1.0.0-SNAPSHOT</version></dependency>
<dependency><groupId>com.kingsrook.qbits</groupId><artifactId>qbit-worm-audit</artifactId><version>0.1.0-SNAPSHOT</version></dependency>
<dependency><groupId>org.postgresql</groupId><artifactId>postgresql</artifactId></dependency>
<dependency><groupId>com.mysql</groupId><artifactId>mysql-connector-j</artifactId></dependency>
<dependency><groupId>com.h2database</groupId><artifactId>h2</artifactId></dependency>
@@ -66,6 +66,10 @@ final class CoreRuntime implements AutoCloseable
try
{
WebhookSubscriptionsHelper.clearMemoizations();
if(EnterpriseFeatures.isEnabled())
{
EnterpriseAssistant.reconcileAtStartup(launcher.getQInstance());
}
}
finally
{
@@ -34,6 +34,18 @@ final class DemoQBitData
"redirect_state", "user", "role", "permission", "role_permission_int", "user_role_int",
"user_permission_int");
/** H2 (MySQL mode) keywords that qbit tables use as column or table names. */
private static final Set<String> H2_RESERVED_WORDS = Set.of("VALUE", "KEY", "VALUES", "USER", "ORDER", "GROUP",
"LIMIT", "OFFSET", "YEAR", "MONTH", "DAY", "HOUR", "MINUTE", "SECOND", "ROW", "ROWS", "END", "FROM", "TO",
"ALL", "AND", "OR", "NOT", "NULL", "TRUE", "FALSE", "IF", "IN", "IS", "AS", "ON", "BY", "SELECT", "WHERE",
"TABLE", "INTERVAL", "DEFAULT", "CHECK", "CONSTRAINT", "PRIMARY", "UNIQUE", "FOREIGN", "INDEX", "JOIN",
"LEFT", "RIGHT", "INNER", "OUTER", "CROSS", "NATURAL", "UNION", "EXCEPT", "INTERSECT", "MINUS", "DISTINCT",
"HAVING", "QUALIFY", "WINDOW", "CASE", "WHEN", "THEN", "ELSE", "FETCH", "FOR", "WITH", "CURRENT_DATE",
"CURRENT_TIME", "CURRENT_TIMESTAMP", "CURRENT_USER", "LOCALTIME", "LOCALTIMESTAMP", "ARRAY", "BETWEEN",
"EXISTS", "LIKE", "ILIKE", "REGEXP", "SET", "UNKNOWN", "USING", "TOP", "SYSDATE", "SYSTIME", "TODAY",
"_ROWID_", "ROWNUM", "LISTAGG", "INTERSECTS", "ANY", "SOME", "ASYMMETRIC", "SYMMETRIC", "AUTHORIZATION",
"CURRENT_CATALOG", "CURRENT_SCHEMA", "CURRENT_PATH", "CURRENT_ROLE", "LATERAL", "UESCAPE");
private DemoQBitData()
{
}
@@ -231,7 +243,7 @@ final class DemoQBitData
{
throw new IllegalArgumentException("Unsupported demo schema identifier");
}
if("value".equalsIgnoreCase(value) || "key".equalsIgnoreCase(value))
if(H2_RESERVED_WORDS.contains(value.toUpperCase()))
{
return "`" + value + "`";
}
@@ -11,6 +11,7 @@ import com.kingsrook.qbits.customizabletableviews.CustomizableTableViewsQBitConf
import com.kingsrook.qbits.customizabletableviews.CustomizableTableViewsQBitProducer;
import com.kingsrook.qbits.quicksearch.QuickSearchQBitConfig;
import com.kingsrook.qbits.quicksearch.QuickSearchQBitProducer;
import com.kingsrook.qbits.quicksearch.QuickSearchStartupMode;
import com.kingsrook.qbits.quicksearch.SearchableFieldConfig;
import com.kingsrook.qbits.sftpdataintegration.SFTPDataIntegrationQBitConfig;
import com.kingsrook.qbits.sftpdataintegration.SFTPDataIntegrationQBitProducer;
@@ -34,8 +35,11 @@ import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.qbits.ProvidedOrSuppliedTableConfig;
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendStepMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QStepMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValue;
import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSource;
import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSourceType;
@@ -47,6 +51,8 @@ import com.kingsrook.qqq.backend.core.model.savedreports.SavedReportsMetaDataPro
import com.kingsrook.qqq.backend.core.model.scheduledjobs.ScheduledJobsMetaDataProvider;
import com.kingsrook.qqq.backend.core.model.common.TimeZonePossibleValueSourceMetaDataProvider;
import com.kingsrook.qqq.backend.core.model.metadata.scheduleing.simple.SimpleSchedulerMetaData;
import com.kingsrook.qqq.backend.module.filesystem.base.model.metadata.Cardinality;
import com.kingsrook.qqq.backend.module.filesystem.sftp.model.metadata.SFTPTableBackendDetails;
import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSTableBackendDetails;
/** Registers the locally usable qbits against the demo's relational backend. */
@@ -111,6 +117,8 @@ final class DemoQBits
.withCategory(WebhookEventCategory.STORE).withTableName("order")
.withCustomizer(new QCodeReference(DemoOrderWebhookContent.class)));
moveTestEventFormAfterPreview(instance);
new WorkflowsQBitProducer()
.withQBitConfig(new WorkflowsQBitConfig().withTableMetaDataCustomizer(tableCustomizer))
.produce(instance).addSelfToInstance(instance);
@@ -156,14 +164,23 @@ final class DemoQBits
.withOpensearchUsername(settings.get("OPENSEARCH_USER"))
.withOpensearchPassword(settings.get("OPENSEARCH_PASSWORD"))
.withUseSsl(Boolean.parseBoolean(settings.getOrDefault("OPENSEARCH_SSL", "false")))
// quick search 1.x connects at startup unless degraded; 0.2.x did not (the enterprise profile bumped it)
.withStartupMode(QuickSearchStartupMode.DEGRADED)
.withSearchableTable("customer", List.of(new SearchableFieldConfig("name"),
new SearchableFieldConfig("email")));
////////////////////////////////////////////////////////////////////
// quick search 1.x registers its own QBitMetaData when produced //
////////////////////////////////////////////////////////////////////
new QuickSearchQBitProducer().withConfig(searchConfig).produce(instance);
instance.addQBit(new QBitMetaData().withGroupId("com.kingsrook.qbits")
.withArtifactId("quick-search").withVersion("0.2.1-SNAPSHOT").withConfig(searchConfig));
new SavedBulkLoadProfileMetaDataProvider().defineAll(instance, "postgres", null);
new SavedReportsMetaDataProvider().defineAll(instance, "postgres", "sftp", null);
///////////////////////////////////////////////////////////////////////////
// rendered reports are files, one per record, on the SFTP server: core //
// now refuses a filesystem table without backend details (qqq#1364) //
///////////////////////////////////////////////////////////////////////////
instance.getTable("reportStorage").setBackendDetails(new SFTPTableBackendDetails().withBasePath("reportStorage")
.withCardinality(Cardinality.ONE).withFileNameFieldName("reference").withContentsFieldName("contents"));
new SFTPDataIntegrationQBitProducer().withSftpDataIntegrationQBitConfig(new SFTPDataIntegrationQBitConfig()
.withSourceFileTableConfig(ProvidedOrSuppliedTableConfig.provideTableUsingBackendNamed("sftp"))
.withDestinationFileTableConfig(ProvidedOrSuppliedTableConfig.provideTableUsingBackendNamed("sftp"))
@@ -181,4 +198,27 @@ final class DemoQBits
return table;
})).produce(instance, "demo-import");
}
/**
* qbit-webhooks 0.x puts SendTestEventToWebhook's input form before the
* preview step, so the records chosen at init are lost and the run fails
* with "No records were selected" (QRun-IO/qqq#1361; fixed on qbit-webhooks
* fix/test-event-records). Until a release carries it, move the form after
* the first backend step here.
*/
static void moveTestEventFormAfterPreview(QInstance instance)
{
QProcessMetaData process = instance.getProcess("SendTestEventToWebhook");
if(process == null || process.getStepList() == null || process.getStepList().size() < 2)
{
return;
}
List<QStepMetaData> steps = process.getStepList();
if(steps.get(0) instanceof QFrontendStepMetaData form && "input".equals(form.getName()) && steps.get(1) instanceof QBackendStepMetaData)
{
steps.remove(0);
steps.add(1, form);
}
}
}
@@ -21,6 +21,10 @@ import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.PermissionLevel;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRules;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendComponentMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendStepMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QComponentType;
import com.kingsrook.qqq.backend.core.model.metadata.help.QHelpContent;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionOutputMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
@@ -62,13 +66,25 @@ final class DemoRuntimeFeatures
instance.withRuntimeService(new QCodeReference(TransientNotesService.class));
instance.addProcess(new QProcessMetaData().withName("calculateOrderTotal").withLabel("Calculate Order Total (JavaScript)")
.withPermissionRules(new QPermissionRules().withLevel(PermissionLevel.HAS_ACCESS_PERMISSION))
// the inputs as a form, so interactive clients ask for them (QRun-IO/qqq-all#45)
.withStep(new QFrontendStepMetaData().withName("input").withLabel("Order line")
.withComponent(new QFrontendComponentMetaData().withType(QComponentType.EDIT_FORM))
.withFormField(new QFieldMetaData("quantity", QFieldType.INTEGER).withLabel("Quantity").withIsRequired(true)
.withHelpContent(new QHelpContent("1 to 1000")))
.withFormField(new QFieldMetaData("unitPriceCents", QFieldType.INTEGER).withLabel("Unit price (cents)").withIsRequired(true)
.withHelpContent(new QHelpContent("0 to 1000000"))))
.withStep(new QBackendStepMetaData().withName("calculate")
.withCode(new QCodeReference(CalculateOrderTotalStep.class))
.withInputData(new QFunctionInputMetaData().withFieldList(List.of(
new QFieldMetaData("quantity", QFieldType.INTEGER),
new QFieldMetaData("unitPriceCents", QFieldType.INTEGER))))
.withOutputMetaData(new QFunctionOutputMetaData().withFieldList(List.of(
new QFieldMetaData("totalCents", QFieldType.INTEGER))))));
new QFieldMetaData("totalCents", QFieldType.INTEGER)))))
.withStep(new QFrontendStepMetaData().withName("result").withLabel("Total")
.withComponent(new QFrontendComponentMetaData().withType(QComponentType.VIEW_FORM))
.withViewField(new QFieldMetaData("quantity", QFieldType.INTEGER).withLabel("Quantity"))
.withViewField(new QFieldMetaData("unitPriceCents", QFieldType.INTEGER).withLabel("Unit price (cents)"))
.withViewField(new QFieldMetaData("totalCents", QFieldType.INTEGER).withLabel("Total (cents)"))));
QJavalinMetaData.ofOrWithNew(instance).withAdditionalRouteProviderReference(new QCodeReference(DemoCatalogRoutes.class));
}
@@ -9,7 +9,9 @@ package com.kingsrook.qqq.all;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import com.kingsrook.qqq.backend.core.actions.permissions.PermissionsHelper;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.security.QSecurityKeyType;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.PermissionLevel;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRules;
import com.kingsrook.qqq.backend.core.model.session.QSession;
@@ -54,7 +56,7 @@ public class DemoUsers implements QAuthenticationModuleCustomizerInterface
public void customizeSession(QInstance instance, QSession session, Map<String, Object> context)
{
session.withPermissions("apiCatalog.read", "demoNote.read");
Boolean admin = ADMIN_SESSION.equals(session.getUuid());
Boolean admin = ADMIN_SESSION.equals(session.getUuid()) || EnterpriseFeatures.isEnabled();
String name = admin ? "Admin" : "Demo User";
String userId = admin ? "core:admin" : "core:demo";
session.getUser().setIdReference(userId);
@@ -84,6 +86,50 @@ public class DemoUsers implements QAuthenticationModuleCustomizerInterface
session.withPermission(processName + ".hasAccess");
}
}
if(EnterpriseFeatures.isEnabled())
{
grantEnterpriseAccess(instance, session, userId);
}
session.withValueForFrontend("user", new LinkedHashMap<>(Map.of("name", name, "email", userId + "@example.test")));
if(EnterpriseFeatures.isEnabled())
{
EnterpriseSessions.record(session);
}
}
/**
* The enterprise test server: every session is an all-access admin, so the
* terminal clients can reach every qbit surface.
*/
static void grantEnterpriseAccess(QInstance instance, QSession session, String userId)
{
//////////////////////////////////////////////////////////////////////
// enterprise test server: every session is an all-access admin, so //
// the terminal clients can reach every qbit surface //
//////////////////////////////////////////////////////////////////////
session.withPermissions(PermissionsHelper.getAllAvailablePermissionNames(instance));
//////////////////////////////////////////////////////////////////////
// permissions some modules check by literal name, outside the //
// table/process permission rules (QRun-IO/qqq#1362) //
//////////////////////////////////////////////////////////////////////
session.withPermissions("jobs_job.read", "jobs_admin", "data_analysis.admin", "assistant.hasAccess", "assistant.admin");
//////////////////////////////////////////////////////////////////////
// qbit-collaboration's custom checkers (Custom Fields, tags, the //
// admin tables) test these names; without them the Custom Fields //
// table answered 403 "This needs collaboration.view." (#47) //
//////////////////////////////////////////////////////////////////////
session.withPermissions("collaboration.view", "collaboration.admin", "collaboration.fields.manage", "collaboration.tags.manage");
for(QSecurityKeyType keyType : instance.getSecurityKeyTypes().values())
{
if(keyType.getAllAccessKeyName() != null)
{
session.withSecurityKeyValue(keyType.getAllAccessKeyName(), true);
}
//////////////////////////////////////////////////////////////////
// per-user keys (owner locks of qbit tables) take the user id //
//////////////////////////////////////////////////////////////////
session.withSecurityKeyValue(keyType.getName(), userId);
}
}
}
@@ -0,0 +1,258 @@
/*
* Copyright 2026 Kingsrook, LLC
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0
*/
package com.kingsrook.qqq.all;
import java.io.Serializable;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.time.Duration;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.function.Function;
import java.util.function.Predicate;
import com.kingsrook.qbits.assistant.AssistantQBitConfig;
import com.kingsrook.qbits.assistant.model.AssistantProvider;
import com.kingsrook.qbits.assistant.provider.anthropic.AnthropicModelProvider;
import com.kingsrook.qbits.assistant.provider.anthropic.AnthropicProviderType;
import com.kingsrook.qbits.assistant.provider.openai.OpenAiCompatibleProviderType;
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
/*******************************************************************************
** The assistant on the enterprise test server: two provider rows, kept in
** step with the environment at every start.
**
** - "Anthropic (Claude)": type anthropic, key variable ANTHROPIC_API_KEY
** (only the NAME is stored; enterprise-server.sh passes the value from its
** private 0600 secrets file). Enabled when the variable is set.
** - "Local Ollama": type openai-compatible, kind OLLAMA, base URL
** http://127.0.0.1:11434/v1, model QQQ_ASSISTANT_OLLAMA_MODEL (default
** llama3.3:70b-instruct-q4_K_M), no key. Enabled when Ollama answers and
** lists that model.
**
** The default is Anthropic when its key is set, else Ollama when reachable;
** with neither, both rows are disabled and GET /assistant/config answers
** NOT_CONFIGURED with a message that says what to configure.
*******************************************************************************/
final class EnterpriseAssistant
{
private static final QLogger LOG = QLogger.getLogger(EnterpriseAssistant.class);
static final String ANTHROPIC_ROW = "Anthropic (Claude)";
static final String OLLAMA_ROW = "Local Ollama";
static final String ANTHROPIC_KEY_ENV = "ANTHROPIC_API_KEY";
static final String OLLAMA_MODEL_ENV = "QQQ_ASSISTANT_OLLAMA_MODEL";
static final String OLLAMA_URL_ENV = "QQQ_ASSISTANT_OLLAMA_URL";
static final String OLLAMA_BASE_URL = "http://127.0.0.1:11434/v1";
static final String OLLAMA_MODEL = "llama3.3:70b-instruct-q4_K_M";
/////////////////////////////////////////////////////////////////////////////
// a local 70B model reads the tool catalog slowly: give a turn and one //
// model call room (the qbit defaults are 300 s and 120 s) //
/////////////////////////////////////////////////////////////////////////////
static final Integer MAX_TURN_SECONDS = 900;
static final Integer OLLAMA_REQUEST_SECONDS = 600;
/*******************************************************************************
** Which provider answers, from the environment at startup.
*******************************************************************************/
enum Choice
{
ANTHROPIC,
OLLAMA,
NONE
}
private EnterpriseAssistant()
{
}
/*******************************************************************************
** The QBit config: both provider types, the Ollama URL allowed as a base
** URL override, and the code-registered Anthropic fallback.
*******************************************************************************/
static AssistantQBitConfig config(String backendName, String uiApiBaseUrl, Function<String, String> env)
{
return new AssistantQBitConfig()
.withBackendName(backendName).withUiApiBaseUrl(uiApiBaseUrl)
.withProviderType(new AnthropicProviderType())
.withProviderType(new OpenAiCompatibleProviderType())
.withAllowedBaseUrls(Set.of(ollamaBaseUrl(env)))
.withMaxTurnSeconds(MAX_TURN_SECONDS);
}
/*******************************************************************************
** Anthropic when its key is set, else Ollama when reachable, else none.
*******************************************************************************/
static Choice choose(Function<String, String> env, Predicate<String> ollamaHasModel)
{
String key = env.apply(ANTHROPIC_KEY_ENV);
if(key != null && !key.isBlank())
{
return Choice.ANTHROPIC;
}
return ollamaHasModel.test(ollamaModel(env)) ? Choice.OLLAMA : Choice.NONE;
}
/*******************************************************************************
** The Ollama model (QQQ_ASSISTANT_OLLAMA_MODEL, default llama3.3 70B).
*******************************************************************************/
static String ollamaModel(Function<String, String> env)
{
String model = env.apply(OLLAMA_MODEL_ENV);
return model == null || model.isBlank() ? OLLAMA_MODEL : model.trim();
}
/*******************************************************************************
** The Ollama base URL (QQQ_ASSISTANT_OLLAMA_URL, default 127.0.0.1:11434/v1).
*******************************************************************************/
static String ollamaBaseUrl(Function<String, String> env)
{
String url = env.apply(OLLAMA_URL_ENV);
return url == null || url.isBlank() ? OLLAMA_BASE_URL : url.trim();
}
/*******************************************************************************
** The two provider rows for a choice (id left out).
*******************************************************************************/
static List<QRecord> rows(Choice choice, Function<String, String> env)
{
QRecord anthropic = new QRecord()
.withValue("name", ANTHROPIC_ROW)
.withValue("providerType", AnthropicModelProvider.NAME)
.withValue("apiKeyEnvName", ANTHROPIC_KEY_ENV)
.withValue("enabled", choice == Choice.ANTHROPIC)
.withValue("isDefault", choice == Choice.ANTHROPIC);
QRecord ollama = new QRecord()
.withValue("name", OLLAMA_ROW)
.withValue("providerType", new OpenAiCompatibleProviderType().getType())
.withValue("kind", "OLLAMA")
.withValue("location", "LOCAL")
.withValue("baseUrl", ollamaBaseUrl(env))
.withValue("model", ollamaModel(env))
.withValue("requestTimeoutSeconds", OLLAMA_REQUEST_SECONDS)
.withValue("enabled", choice == Choice.OLLAMA)
.withValue("isDefault", choice == Choice.OLLAMA);
return List.of(anthropic, ollama);
}
/*******************************************************************************
** Insert or update the two rows (matched by name) for this start. Runs
** with the caller's QContext (the system session at startup).
*******************************************************************************/
static Choice reconcile(QInstance instance, Function<String, String> env, Predicate<String> ollamaHasModel) throws QException
{
if(instance.getTable(AssistantProvider.TABLE_NAME) == null)
{
return Choice.NONE;
}
Choice choice = choose(env, ollamaHasModel);
List<QRecord> wanted = rows(choice, env);
QueryInput queryInput = new QueryInput(AssistantProvider.TABLE_NAME);
queryInput.setFilter(new QQueryFilter(new QFilterCriteria("name", QCriteriaOperator.IN, List.of(ANTHROPIC_ROW, OLLAMA_ROW))));
Map<Serializable, QRecord> existing = new java.util.HashMap<>();
for(QRecord record : new QueryAction().execute(queryInput).getRecords())
{
existing.putIfAbsent(record.getValueString("name"), record);
}
//////////////////////////////////////////////////////////////////////////
// the non-default row first, so marking the default one last leaves it //
// as the only default (the qbit clears other defaults on save) //
//////////////////////////////////////////////////////////////////////////
List<QRecord> ordered = wanted.stream().sorted((a, b) -> Boolean.compare(Boolean.TRUE.equals(a.getValueBoolean("isDefault")), Boolean.TRUE.equals(b.getValueBoolean("isDefault")))).toList();
for(QRecord row : ordered)
{
QRecord current = existing.get(row.getValueString("name"));
if(current == null)
{
new InsertAction().execute(new InsertInput(AssistantProvider.TABLE_NAME).withRecord(row));
}
else
{
row.setValue("id", current.getValue("id"));
new UpdateAction().execute(new UpdateInput(AssistantProvider.TABLE_NAME).withRecord(row));
}
}
return choice;
}
/*******************************************************************************
** Reconcile from the process environment; a failure is logged and leaves
** the server running (the assistant then reports NOT_CONFIGURED).
*******************************************************************************/
static void reconcileAtStartup(QInstance instance)
{
try
{
Choice choice = reconcile(instance, System::getenv, model -> ollamaHasModel(ollamaBaseUrl(System::getenv), model));
LOG.info("Enterprise assistant model provider: " + choice);
}
catch(QException | RuntimeException e)
{
LOG.warn("Could not set up the enterprise assistant provider rows", e);
}
}
/*******************************************************************************
** True when Ollama answers GET {base}/models within 3 s and lists the model.
*******************************************************************************/
static boolean ollamaHasModel(String baseUrl, String model)
{
try
{
HttpClient client = HttpClient.newBuilder().connectTimeout(Duration.ofSeconds(2)).build();
HttpRequest request = HttpRequest.newBuilder(URI.create(baseUrl + "/models")).timeout(Duration.ofSeconds(3)).GET().build();
HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
return response.statusCode() == 200 && response.body().contains("\"" + model + "\"");
}
catch(Exception e)
{
if(e instanceof InterruptedException)
{
Thread.currentThread().interrupt();
}
return false;
}
}
}
@@ -0,0 +1,706 @@
/*
* Copyright 2026 Kingsrook, LLC
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0
*/
package com.kingsrook.qqq.all;
import java.nio.file.Files;
import java.nio.file.Path;
import java.security.KeyPair;
import java.security.KeyPairGenerator;
import java.time.Duration;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashSet;
import java.util.LinkedHashSet;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.TreeMap;
import java.util.Properties;
import java.util.Set;
import java.util.function.Consumer;
import java.util.stream.Collectors;
import com.kingsrook.qbits.alerts.AlertsQBitConfig;
import com.kingsrook.qbits.alerts.AlertsQBitProducer;
import com.kingsrook.qbits.assistant.AssistantQBitProducer;
import com.kingsrook.qbits.backups.BackupsQBitConfig;
import com.kingsrook.qbits.backups.BackupsQBitProducer;
import com.kingsrook.qbits.backups.storage.InMemoryBackupStorage;
import com.kingsrook.qbits.cdn.CdnQBitConfig;
import com.kingsrook.qbits.cdn.CdnQBitProducer;
import com.kingsrook.qbits.cluster.ClusterQBitConfig;
import com.kingsrook.qbits.cluster.ClusterQBitProducer;
import com.kingsrook.qbits.collaboration.CollaborationQBitConfig;
import com.kingsrook.qbits.collaboration.CollaborationQBitProducer;
import com.kingsrook.qbits.dataanalysis.DataAnalysisQBitConfig;
import com.kingsrook.qbits.dataanalysis.DataAnalysisQBitProducer;
import com.kingsrook.qbits.dbperformance.DbPerfQBitConfig;
import com.kingsrook.qbits.dbperformance.DbPerfQBitProducer;
import com.kingsrook.qbits.email.EmailQBitConfig;
import com.kingsrook.qbits.email.EmailQBitProducer;
import com.kingsrook.qbits.filegateway.FileGatewayQBitConfig;
import com.kingsrook.qbits.filegateway.FileGatewayQBitProducer;
import com.kingsrook.qbits.help.HelpQBitConfig;
import com.kingsrook.qbits.help.HelpQBitProducer;
import com.kingsrook.qbits.jobs.JobsQBitConfig;
import com.kingsrook.qbits.jobs.JobsQBitProducer;
import com.kingsrook.qbits.objectstorage.ObjectStorageQBitConfig;
import com.kingsrook.qbits.objectstorage.ObjectStorageQBitProducer;
import com.kingsrook.qbits.objectstorage.config.AddressingStyle;
import com.kingsrook.qbits.objectstorage.config.BucketRole;
import com.kingsrook.qbits.objectstorage.config.ProviderType;
import com.kingsrook.qbits.objectstorage.config.StorageBucketConfig;
import com.kingsrook.qbits.objectstorage.config.StorageConnectionConfig;
import com.kingsrook.qbits.qacapture.QaCaptureQBitConfig;
import com.kingsrook.qbits.qacapture.QaCaptureQBitProducer;
import com.kingsrook.qbits.quicksearch.QuickSearchQBitConfig;
import com.kingsrook.qbits.quicksearch.QuickSearchQBitProducer;
import com.kingsrook.qbits.quicksearch.SearchableFieldConfig;
import com.kingsrook.qbits.quicksearch.SearchableTableConfig;
import com.kingsrook.qbits.redis.RedisConnectionConfig;
import com.kingsrook.qbits.redis.RedisQBitConfig;
import com.kingsrook.qbits.redis.RedisQBitProducer;
import com.kingsrook.qbits.releasenotes.ReleaseNotesQBitConfig;
import com.kingsrook.qbits.releasenotes.ReleaseNotesQBitProducer;
import com.kingsrook.qbits.releasenotes.sources.ComponentVersionsSource;
import com.kingsrook.qbits.secureddata.SecuredDataQBitConfig;
import com.kingsrook.qbits.secureddata.SecuredDataQBitProducer;
import com.kingsrook.qbits.sessionstore.QSessionStoreProviderType;
import com.kingsrook.qbits.sessionstore.QSessionStoreQBitConfig;
import com.kingsrook.qbits.sessionstore.QSessionStoreQBitProducer;
import com.kingsrook.qbits.slack.SlackQBitConfig;
import com.kingsrook.qbits.slack.SlackQBitProducer;
import com.kingsrook.qbits.sqlworkbench.SqlWorkbenchQBitConfig;
import com.kingsrook.qbits.sqlworkbench.SqlWorkbenchQBitProducer;
import com.kingsrook.qbits.wormaudit.WormAuditQBitConfig;
import com.kingsrook.qbits.wormaudit.WormAuditQBitProducer;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.instances.QInstanceEnricher;
import com.kingsrook.qqq.backend.core.health.HealthMetaDataProvider;
import com.kingsrook.qqq.backend.core.jobs.DurableJobsMetaData;
import com.kingsrook.qqq.backend.core.jobs.DurableJobsMetaDataProvider;
import com.kingsrook.qqq.backend.core.model.audits.AuditsMetaDataProvider;
import com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.tables.Capability;
import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryBackendModule;
import com.kingsrook.qqq.backend.core.processes.implementations.columnstats.ColumnStatsStep;
import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.audits.AuditLevel;
import com.kingsrook.qqq.backend.core.model.metadata.audits.QAuditRules;
import com.kingsrook.qqq.backend.core.model.metadata.producers.MetaDataCustomizerInterface;
import com.kingsrook.qqq.backend.core.model.metadata.scheduleing.quartz.QuartzSchedulerMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppChildMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppSection;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import com.kingsrook.qqq.backend.core.model.querystats.QueryStatMetaDataProvider;
import com.kingsrook.qqq.backend.core.model.savedbulkloadprofiles.SavedBulkLoadProfileMetaDataProvider;
import com.kingsrook.qqq.backend.core.model.savedreports.SavedReportsMetaDataProvider;
import com.kingsrook.qqq.backend.core.model.savedviews.SavedViewsMetaDataProvider;
import com.kingsrook.qqq.backend.core.model.scripts.ScriptsMetaDataProvider;
import com.kingsrook.qqq.backend.core.actions.preferences.UserPreferencesKeyValueStoreMetaDataProvider;
import com.kingsrook.qqq.backend.core.processes.implementations.sharing.SharingMetaDataProvider;
import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordFeature;
import com.kingsrook.qqq.backend.module.filesystem.base.model.metadata.AbstractFilesystemTableBackendDetails;
import com.kingsrook.qqq.backend.module.filesystem.base.model.metadata.Cardinality;
import com.kingsrook.qqq.backend.module.filesystem.local.model.metadata.FilesystemBackendMetaData;
import com.kingsrook.qqq.backend.module.filesystem.local.model.metadata.FilesystemTableBackendDetails;
import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSTableBackendDetails;
/*******************************************************************************
** The "enterprise" profile (QQQ_ALL_PROFILE=enterprise): the core profile plus
** every qbit and core module an enterprise rollout runs, all on the local H2
** backend, for exercising the terminal clients against a large instance.
**
** Each feature is named; QQQ_ALL_ENTERPRISE_EXCLUDE (comma separated names)
** leaves features out. Quick search needs QQQ_ALL_OPENSEARCH_HOST and redis
** needs QQQ_ALL_REDIS_HOST; without them those two are left out.
*******************************************************************************/
final class EnterpriseFeatures
{
static final String PROFILE = "enterprise";
private static final String SAMPLE_PROVIDER = "com.kingsrook.sampleapp.metadata.SampleMetaDataProvider";
private interface Feature
{
void add(QInstance instance) throws Exception;
}
private EnterpriseFeatures()
{
}
/*******************************************************************************
** True when the process runs the enterprise profile.
*******************************************************************************/
static boolean isEnabled()
{
return PROFILE.equals(System.getenv("QQQ_ALL_PROFILE"));
}
/*******************************************************************************
** Names the operator excluded.
*******************************************************************************/
static Set<String> excluded()
{
String value = System.getenv().getOrDefault("QQQ_ALL_ENTERPRISE_EXCLUDE", "");
return Arrays.stream(value.split(",")).map(String::trim).filter(s -> !s.isEmpty()).collect(Collectors.toSet());
}
/*******************************************************************************
** Add every enterprise feature that is not excluded.
*******************************************************************************/
static void add(QInstance instance, String backendName, int port) throws QException
{
String schedulerName = "demoScheduler";
String baseUrl = "http://127.0.0.1:" + port;
//////////////////////////////////////////////////////////////////////////
// qbits schedule with cron expressions, which the simple scheduler of //
// the core profile cannot run: swap it for an in-memory Quartz //
//////////////////////////////////////////////////////////////////////////
Properties quartzProperties = new Properties();
quartzProperties.put("org.quartz.scheduler.instanceName", "qqqAllEnterprise");
quartzProperties.put("org.quartz.threadPool.threadCount", "4");
quartzProperties.put("org.quartz.jobStore.class", "org.quartz.simpl.RAMJobStore");
instance.getSchedulers().put(schedulerName, new QuartzSchedulerMetaData().withProperties(quartzProperties).withName(schedulerName));
MetaDataCustomizerInterface<QTableMetaData> tableCustomizer = (i, table) ->
{
if(table.getBackendName() == null || backendName.equals(table.getBackendName()))
{
table.setBackendName(backendName);
table.setBackendDetails(new RDBMSTableBackendDetails()
.withTableName(QInstanceEnricher.inferBackendName(table.getName())));
QInstanceEnricher.setInferredFieldBackendNames(table);
}
return table;
};
Consumer<QTableMetaData> enricher = table ->
{
try
{
tableCustomizer.customizeMetaData(instance, table);
}
catch(QException e)
{
throw new IllegalStateException(e);
}
};
Map<String, Feature> features = new LinkedHashMap<>();
/////////////////////////
// core qqq modules //
/////////////////////////
features.put("audits", i ->
{
new AuditsMetaDataProvider().defineAll(i, backendName, enricher);
for(String tableName : List.of("customer", "order", "address"))
{
i.getTable(tableName).setAuditRules(new QAuditRules().withAuditLevel(AuditLevel.FIELD));
}
});
features.put("scripts", i ->
{
////////////////////////////////////////////////////////////////////
// the scripts provider defines its own "tables" possible values //
////////////////////////////////////////////////////////////////////
i.getPossibleValueSources().remove("tables");
new ScriptsMetaDataProvider().defineAll(i, backendName, enricher);
});
features.put("savedViews", i -> new SavedViewsMetaDataProvider().defineAll(i, backendName, enricher));
features.put("savedReports", i ->
{
new SavedReportsMetaDataProvider().defineAll(i, backendName, "files", enricher);
//////////////////////////////////////////////////////////////////
// rendered reports are files, one per record, under the files //
// backend (the provider leaves the storage details to the host) //
//////////////////////////////////////////////////////////////////
QTableMetaData reportStorage = i.getTable("reportStorage");
if(reportStorage != null && reportStorage.getBackendDetails() == null)
{
reportStorage.setBackendDetails(new FilesystemTableBackendDetails().withBasePath("reportStorage").withCardinality(Cardinality.ONE).withFileNameFieldName("reference").withContentsFieldName("contents"));
}
ensureFilesystemDirectories(i);
});
features.put("bulkLoadProfiles", i -> new SavedBulkLoadProfileMetaDataProvider().defineAll(i, backendName, enricher));
features.put("sharing", i -> new SharingMetaDataProvider().defineAll(i, null));
features.put("queryStats", i -> new QueryStatMetaDataProvider().defineAll(i, backendName, enricher));
features.put("userPreferences", i -> new UserPreferencesKeyValueStoreMetaDataProvider().defineAll(i, backendName, null));
//////////////////////////////////////////////////////////////////////////
// central health: its tables turn on the health UI API extension, so //
// /version lists it (QRun-IO/qqq-all#44). Enterprise only for now: in //
// the core profile its backend checks share the cached sqlite //
// connection across the test suite's runtimes //
//////////////////////////////////////////////////////////////////////////
features.put("health", i -> new HealthMetaDataProvider().defineAll(i, backendName, enricher));
features.put("durableJobs", i ->
{
new DurableJobsMetaDataProvider().defineAll(i, backendName, enricher);
i.withSupplementalMetaData(new DurableJobsMetaData().withWorkerThreads(2));
});
/////////////////////////
// qbits //
/////////////////////////
//////////////////////////////////////////////////////////////////////////
// object storage on a local MinIO (enterprise-server.sh starts it): //
// collaboration attachments live in its private "collaboration" //
// bucket. Credentials are references to the server's env only. //
//////////////////////////////////////////////////////////////////////////
String s3Endpoint = System.getenv("S3_ENDPOINT");
if(s3Endpoint != null && !s3Endpoint.isBlank())
{
features.put("objectStorage", i -> new ObjectStorageQBitProducer().withQBitConfig(new ObjectStorageQBitConfig()
.withBackendName(backendName).withSchedulerName(schedulerName).withAdminApp(true)
.withConnection(new StorageConnectionConfig("local").withProviderType(ProviderType.MINIO)
.withEndpoint("${env.S3_ENDPOINT}").withRegion("us-east-1").withAddressingStyle(AddressingStyle.PATH)
.withAccessKeyRef("${env.S3_ACCESS_KEY}").withSecretKeyRef("${env.S3_SECRET_KEY}"))
.withBucket(new StorageBucketConfig("collaboration").withConnection("local")
.withBucketName("${env.S3_ATTACHMENTS_BUCKET}").withRole(BucketRole.PRIVATE).withIndexed(false)))
.produce(i).addSelfToInstance(i));
}
features.put("jobs", i -> new JobsQBitProducer().withQBitConfig(new JobsQBitConfig()
.withSchedulerName(schedulerName)).produce(i).addSelfToInstance(i));
features.put("collaboration", i -> new CollaborationQBitProducer().withQBitConfig(new CollaborationQBitConfig()
.withBackendName(backendName).withSchedulerName(schedulerName).withUiApi(true).withAdminApp(true)
.withDefaultFeatures(RecordFeature.values())
.withFrontendBaseUrl(baseUrl)).produce(i).addSelfToInstance(i));
features.put("alerts", i -> new AlertsQBitProducer().withQBitConfig(new AlertsQBitConfig()
.withBackendName(backendName).withSchedulerName(schedulerName)
.withPublicBaseUrl(baseUrl)).produce(i).addSelfToInstance(i));
features.put("dataAnalysis", i -> new DataAnalysisQBitProducer().withQBitConfig(new DataAnalysisQBitConfig()
.withBackendName(backendName).withSchedulerName(schedulerName)).produce(i).addSelfToInstance(i));
features.put("dbPerformance", i -> new DbPerfQBitProducer().withQBitConfig(new DbPerfQBitConfig()
.withBackendName(backendName).withSchedulerName(schedulerName)).produce(i).addSelfToInstance(i));
features.put("sqlWorkbench", i -> new SqlWorkbenchQBitProducer().withQBitConfig(new SqlWorkbenchQBitConfig()
.withBackendName(backendName).withTableMetaDataCustomizer(tableCustomizer)
.withAllowedBackendNames(List.of(backendName))).produce(i).addSelfToInstance(i));
features.put("help", i -> new HelpQBitProducer().withQBitConfig(new HelpQBitConfig()
.withBackendName(backendName).withTableMetaDataCustomizer(tableCustomizer)).produce(i).addSelfToInstance(i));
features.put("releaseNotes", i -> new ReleaseNotesQBitProducer().withQBitConfig(new ReleaseNotesQBitConfig()
.withBackendName(backendName).withApplicationName("qqqAll").withApplicationLabel("QQQ All")
.withAppVersion("4.1.0").withAllowNonReleaseVersions(true)
.withSource(new ComponentVersionsSource())).produce(i).addSelfToInstance(i));
features.put("qaCapture", i -> new QaCaptureQBitProducer().withQBitConfig(new QaCaptureQBitConfig()
.withBackendName(backendName).withEnvironment("local").withBackendVersion("4.1.0-SNAPSHOT"))
.produce(i).addSelfToInstance(i));
features.put("securedData", i -> new SecuredDataQBitProducer().withQBitConfig(new SecuredDataQBitConfig()
.withBackendName(backendName).withSchedulerName(schedulerName)
.withTableMetaDataCustomizer(tableCustomizer)).produce(i).addSelfToInstance(i));
features.put("email", i -> new EmailQBitProducer().withQBitConfig(new EmailQBitConfig()
.withBackendName(backendName).withSchedulerName(schedulerName).withPublicBaseUrl(baseUrl)
.withTableMetaDataCustomizer(tableCustomizer)).produce(i).addSelfToInstance(i));
features.put("slack", i -> new SlackQBitProducer().withQBitConfig(new SlackQBitConfig()
.withBackendName(backendName).withSchedulerName(schedulerName).withPublicBaseUrl(baseUrl)
.withTableMetaDataCustomizer(tableCustomizer)).produce(i).addSelfToInstance(i));
features.put("cluster", i -> new ClusterQBitProducer().withQBitConfig(new ClusterQBitConfig()
.withBackendName(backendName)).produce(i).addSelfToInstance(i));
features.put("cdn", i -> new CdnQBitProducer().withQBitConfig(new CdnQBitConfig()
.withBackendName(backendName).withSchedulerName(schedulerName)
.withPurgeableTables(List.of("customer")).withTableMetaDataCustomizer(tableCustomizer))
.produce(i).addSelfToInstance(i));
features.put("backups", i -> new BackupsQBitProducer().withQBitConfig(new BackupsQBitConfig()
.withBackendName(backendName).withSchedulerName(schedulerName)
.withStorageResolver(name -> new InMemoryBackupStorage())).produce(i).addSelfToInstance(i));
features.put("fileGateway", i -> new FileGatewayQBitProducer().withQBitConfig(new FileGatewayQBitConfig()
.withBackendName(backendName).withSftpEnabled(false).withSchedulerName(schedulerName)
.withTableMetaDataCustomizer(tableCustomizer)).produce(i).addSelfToInstance(i));
features.put("sessionStore", i ->
{
new QSessionStoreQBitProducer().withConfig(new QSessionStoreQBitConfig()
.withProviderType(QSessionStoreProviderType.TABLE_BASED).withBackendName(backendName)
.withDefaultTtl(Duration.ofHours(8)).withEnableSlidingExpiration(true)).produce(i, "session-store");
if(i.getProcess("cleanExpiredSessions") != null && i.getProcess("cleanExpiredSessions").getSchedule() != null)
{
i.getProcess("cleanExpiredSessions").getSchedule().setSchedulerName(schedulerName);
}
});
features.put("wormAudit", i ->
{
KeyPairGenerator generator = KeyPairGenerator.getInstance("Ed25519");
KeyPair keyPair = generator.generateKeyPair();
new WormAuditQBitProducer().withQBitConfig(new WormAuditQBitConfig()
.withBackendName(backendName).withSchedulerName(schedulerName).withAppRole("app")
.withSigningKey(() -> new WormAuditQBitConfig.SigningKey("local", keyPair.getPrivate()))
.withTrustedPublicKeys(Map.of("local", keyPair.getPublic()))
.withTableMetaDataCustomizer(tableCustomizer)).produce(i).addSelfToInstance(i);
});
features.put("assistant", i -> new AssistantQBitProducer().withQBitConfig(EnterpriseAssistant.config(backendName, baseUrl, System::getenv)
.withTableMetaDataCustomizer(tableCustomizer)).produce(i).addSelfToInstance(i));
if(isSampleAppOnClasspath())
{
features.put("sampleApp", EnterpriseFeatures::mergeSampleApp);
}
String opensearchHost = System.getenv("QQQ_ALL_OPENSEARCH_HOST");
if(opensearchHost != null && !opensearchHost.isBlank())
{
features.put("quickSearch", i ->
{
QuickSearchQBitConfig searchConfig = new QuickSearchQBitConfig()
.withBackendName(backendName)
.withOpensearchHost(opensearchHost)
.withOpensearchPort(Integer.valueOf(System.getenv().getOrDefault("QQQ_ALL_OPENSEARCH_PORT", "9200")))
.withOpensearchIndexName(System.getenv().getOrDefault("QQQ_ALL_OPENSEARCH_INDEX", "qqq-all-enterprise"))
.withUseSsl(false)
.withAllowPlaintextCredentials(true)
.withSchedulerName(schedulerName)
.withTableMetaDataCustomizer(tableCustomizer)
// new tables are queued for their first build, which the basepull runs: keep it quick
.withBasepullRepeatSeconds(60);
for(SearchableTableConfig table : searchableTables(i))
{
searchConfig.withSearchableTable(table);
}
new QuickSearchQBitProducer().withConfig(searchConfig).produce(i).addSelfToInstance(i);
});
}
String redisHost = System.getenv("QQQ_ALL_REDIS_HOST");
if(redisHost != null && !redisHost.isBlank())
{
features.put("redis", i -> new RedisQBitProducer().withQBitConfig(new RedisQBitConfig()
.withConnection("realtime", new RedisConnectionConfig().withHost(redisHost)
.withPort(Integer.valueOf(System.getenv().getOrDefault("QQQ_ALL_REDIS_PORT", "6379")))
.withTls(false))
.withRealtime("realtime", "local", "qqq-all")).produce(i).addSelfToInstance(i));
}
Set<String> excluded = excluded();
for(Map.Entry<String, Feature> entry : features.entrySet())
{
if(excluded.contains(entry.getKey()))
{
System.out.println("[enterprise] skipped " + entry.getKey());
continue;
}
try
{
entry.getValue().add(instance);
System.out.println("[enterprise] added " + entry.getKey());
}
catch(Exception e)
{
if("true".equals(System.getenv("QQQ_ALL_ENTERPRISE_LENIENT")))
{
System.out.println("[enterprise] FAILED " + entry.getKey() + ": " + e);
Arrays.stream(e.getStackTrace()).limit(6).forEach(frame -> System.out.println("[enterprise] at " + frame));
continue;
}
throw new QException("Enterprise feature " + entry.getKey() + " failed: " + e.getMessage(), e);
}
}
enableColumnStatistics(instance);
auditDataTables(instance);
addNavigationApps(instance);
}
/*******************************************************************************
** QQQ turns QUERY_STATS (column statistics: counts, distinct values, min/max
** and averages per field) off on every backend by default. Turn it on for the
** backends whose module can aggregate, RDBMS (H2, SQLite, the sample's
** in-memory H2) and the memory backend, so their tables offer statistics as
** the web UI and the terminal clients expect; filesystem and API backends
** cannot aggregate and stay without it. Tables of those backends that list
** QUERY_STATS as disabled keep it off. The columnStats process that serves
** the statistics is added when the instance lacks it; core guards it by field
** access (QRun-IO/qqq#1367: only fully readable fields, results tagged).
*******************************************************************************/
static void enableColumnStatistics(QInstance instance)
{
////////////////////////////////////////////////////////////////////
// the UI API serves column statistics through this process //
////////////////////////////////////////////////////////////////////
if(instance.getProcess(ColumnStatsStep.getProcessMetaData().getName()) == null)
{
instance.addProcess(ColumnStatsStep.getProcessMetaData());
}
for(QBackendMetaData backend : instance.getBackends().values())
{
if(backend instanceof RDBMSBackendMetaData || new MemoryBackendModule().getBackendType().equals(backend.getBackendType()))
{
backend.withCapability(Capability.QUERY_STATS);
}
}
}
/*******************************************************************************
** Field-level audits on the demo and sample data tables (customer, order and
** address get them with the audits feature), when audits are installed.
*******************************************************************************/
static void auditDataTables(QInstance instance)
{
if(instance.getTable("audit") == null)
{
return;
}
for(String tableName : List.of("customer", "order", "orderLine", "address", "person", "pet", "petNote", "carrier"))
{
QTableMetaData table = instance.getTable(tableName);
if(table != null && table.getAuditRules() == null)
{
table.setAuditRules(new QAuditRules().withAuditLevel(AuditLevel.FIELD));
}
}
}
/** Text fields worth searching besides a table's record label fields. */
private static final Set<String> SEARCHABLE_FIELD_NAMES = Set.of("name", "label", "title", "description", "email",
"firstName", "lastName", "city", "state", "country", "code", "orderNo", "status");
/** The backends of the demo and sample data (QBit admin tables live on h2 too, and are left out). */
private static final Set<String> SAMPLE_BACKENDS = Set.of("rdbms", "memory");
private static final List<String> DEMO_TABLES = List.of("customer", "order", "address", "shipping_country",
"shipping_stateProvince", "shipping_city");
/*******************************************************************************
** Quick Search registrations for the demo tables and every visible sample-app
** table with a record label: the label fields plus the obvious text fields,
** STRING fields only (OpenSearch indexes them as searchable text; dates,
** numbers and booleans stay out). Self-priming creates their index rows and
** queues each for its first build.
*******************************************************************************/
static List<SearchableTableConfig> searchableTables(QInstance instance)
{
List<SearchableTableConfig> tables = new ArrayList<>();
for(QTableMetaData table : instance.getTables().values())
{
boolean demo = DEMO_TABLES.contains(table.getName());
boolean sample = SAMPLE_BACKENDS.contains(table.getBackendName()) && !table.getIsHidden()
&& table.getRecordLabelFields() != null && !table.getRecordLabelFields().isEmpty();
if(!demo && !sample)
{
continue;
}
Set<String> names = new LinkedHashSet<>();
for(String labelField : Objects.requireNonNullElse(table.getRecordLabelFields(), List.<String>of()))
{
names.add(labelField);
}
names.addAll(SEARCHABLE_FIELD_NAMES);
List<SearchableFieldConfig> fields = new ArrayList<>();
for(String name : names)
{
QFieldMetaData field = table.getFields().get(name);
if(field != null && field.getType() == QFieldType.STRING)
{
fields.add(new SearchableFieldConfig(name).withWeight(Objects.requireNonNullElse(table.getRecordLabelFields(), List.<String>of()).contains(name) ? 3 : 1));
}
}
if(fields.isEmpty())
{
continue;
}
SearchableTableConfig config = new SearchableTableConfig(table.getName(), fields);
if((table.getRecordLabelFields() == null || table.getRecordLabelFields().isEmpty()) && fields.stream().anyMatch(f -> "name".equals(f.getFieldName())))
{
///////////////////////////////////////////////////////////////
// label hits by name where the table itself has no label //
///////////////////////////////////////////////////////////////
config.withRecordLabelFormat("%s", "name");
}
if(!table.getFields().containsKey("modifyDate"))
{
config.withBasepullTimestampField(null);
}
tables.add(config);
}
return (tables);
}
/*******************************************************************************
** The core profile has no apps and several qbits only add tables. Put the
** demo tables in a "QQQ All" app and every visible table that no app reaches
** into an "Enterprise Admin" app, one section per table-name prefix, so the
** clients' app tree reaches everything.
*******************************************************************************/
private static void addNavigationApps(QInstance instance)
{
List<String> demoTables = List.of("customer", "order", "orderLine", "product", "address", "apiCatalog", "demoNote",
"shipping_country", "shipping_stateProvince", "shipping_city");
List<QAppChildMetaData> demoChildren = new ArrayList<>();
demoTables.stream().map(instance::getTable).filter(Objects::nonNull).forEach(demoChildren::add);
List.of("syncOrder", "calculateOrderTotal").stream().map(instance::getProcess).filter(Objects::nonNull).forEach(demoChildren::add);
instance.addApp(new QAppMetaData().withName("qqqAll").withLabel("QQQ All").withIcon(new QIcon("storefront"))
.withSectionOfChildren(new QAppSection().withName("demoData").withLabel("Demo Data").withIcon(new QIcon("table_chart")), demoChildren));
Set<String> reached = new HashSet<>();
for(QAppMetaData app : instance.getApps().values())
{
if(app.getChildren() != null)
{
app.getChildren().stream().filter(c -> c instanceof QTableMetaData).forEach(c -> reached.add(c.getName()));
}
if(app.getSections() != null)
{
app.getSections().stream().filter(section -> section.getTables() != null).forEach(section -> reached.addAll(section.getTables()));
}
}
Map<String, List<QAppChildMetaData>> byPrefix = new TreeMap<>();
for(QTableMetaData table : instance.getTables().values())
{
if(reached.contains(table.getName()) || table.getIsHidden())
{
continue;
}
String backendName = QInstanceEnricher.inferBackendName(table.getName());
String prefix = backendName.contains("_") ? backendName.substring(0, backendName.indexOf('_')) : backendName;
byPrefix.computeIfAbsent(prefix, k -> new ArrayList<>()).add(table);
}
QAppMetaData admin = new QAppMetaData().withName("enterpriseAdmin").withLabel("Enterprise Admin").withIcon(new QIcon("admin_panel_settings"));
byPrefix.forEach((prefix, tables) -> admin.withSectionOfChildren(new QAppSection().withName("admin_" + prefix)
.withLabel(QInstanceEnricher.nameToLabel(prefix)).withIcon(new QIcon("folder")), tables));
instance.addApp(admin);
}
/*******************************************************************************
** Filesystem tables (rendered report storage) answer 500 when their directory
** does not exist yet: create each one under the files backend.
*******************************************************************************/
private static void ensureFilesystemDirectories(QInstance instance) throws Exception
{
if(!(instance.getBackend("files") instanceof FilesystemBackendMetaData files))
{
return;
}
for(QTableMetaData table : instance.getTables().values())
{
if("files".equals(table.getBackendName()) && table.getBackendDetails() instanceof AbstractFilesystemTableBackendDetails details && details.getBasePath() != null)
{
Files.createDirectories(Path.of(files.getBasePath(), details.getBasePath()));
}
}
}
/*******************************************************************************
** True when the qqq sample project's classes are on the classpath (the start
** script adds qqq-sample-project/target/classes).
*******************************************************************************/
private static boolean isSampleAppOnClasspath()
{
try
{
Class.forName(SAMPLE_PROVIDER);
return true;
}
catch(ClassNotFoundException e)
{
return false;
}
}
/*******************************************************************************
** Merge the standard qqq sample app (person, pet, carrier, city files, the
** field and adornment labs, its processes, widgets and apps) into this
** instance. Its ESB wiring is dropped (this instance has its own broker) and
** names this instance already defines win. Reflection keeps the sample
** project out of the build; it is a classpath add-on.
*******************************************************************************/
private static void mergeSampleApp(QInstance instance) throws Exception
{
Class<?> provider = Class.forName(SAMPLE_PROVIDER);
QInstance sample = (QInstance) provider.getMethod("defineInstance").invoke(null);
provider.getMethod("primeTestDatabase", String.class).invoke(null, "prime-test-database.sql");
//////////////////////////////////////////////////////////////////////////
// the sample's sales module defines its own "customer"; this instance //
// already has one, so the sales module stays out //
//////////////////////////////////////////////////////////////////////////
Set<String> skip = Set.of("sales", "customer", "sku", "tag", "sales_order", "sales_order_line", "sales_order_tag",
"salesOrderJoinLine", "salesOrderJoinTag", "salesOrderLines", "salesOrderTags");
sample.getTables().keySet().removeAll(skip);
sample.getPossibleValueSources().keySet().removeAll(skip);
sample.getJoins().keySet().removeAll(skip);
sample.getWidgets().keySet().removeAll(skip);
sample.getApps().keySet().removeAll(skip);
sample.getApps().values().forEach(app ->
{
if(app.getChildren() != null)
{
app.getChildren().removeIf(child -> skip.contains(child.getName()));
}
if(app.getSections() != null)
{
app.getSections().forEach(section ->
{
if(section.getTables() != null)
{
section.getTables().removeIf(skip::contains);
}
});
}
});
sample.getBackends().forEach((name, value) -> instance.getBackends().putIfAbsent(name, value));
sample.getPossibleValueSources().forEach((name, value) -> instance.getPossibleValueSources().putIfAbsent(name, value));
sample.getSecurityKeyTypes().forEach((name, value) -> instance.getSecurityKeyTypes().putIfAbsent(name, value));
sample.getTables().forEach((name, table) ->
{
if(table.getSupplementalMetaData() != null)
{
table.getSupplementalMetaData().values().removeIf(EnterpriseFeatures::isEsb);
}
instance.getTables().putIfAbsent(name, table);
});
sample.getProcesses().forEach((name, process) ->
{
if(process.getSupplementalMetaData() != null)
{
process.getSupplementalMetaData().values().removeIf(EnterpriseFeatures::isEsb);
}
instance.getProcesses().putIfAbsent(name, process);
});
sample.getJoins().forEach((name, value) -> instance.getJoins().putIfAbsent(name, value));
sample.getReports().forEach((name, value) -> instance.getReports().putIfAbsent(name, value));
sample.getWidgets().forEach((name, value) -> instance.getWidgets().putIfAbsent(name, value));
sample.getApps().forEach((name, value) -> instance.getApps().putIfAbsent(name, value));
}
/*******************************************************************************
** ESB supplemental meta data of the sample points at its own destinations.
*******************************************************************************/
private static boolean isEsb(Object supplemental)
{
return supplemental != null && supplemental.getClass().getName().startsWith("com.kingsrook.qqq.esb.");
}
}
@@ -0,0 +1,83 @@
/*
* Copyright 2026 Kingsrook, LLC
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0
*/
package com.kingsrook.qqq.all;
import java.time.Duration;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import com.kingsrook.qqq.backend.core.modules.authentication.QSessionStoreHelper;
import com.kingsrook.qqq.backend.core.model.session.QSession;
import com.kingsrook.qqq.backend.core.model.session.QUser;
/**
* Records mock sign-ins in the session store (qbit-session-store) for the
* enterprise profile.
*
* Only the OAuth2 module writes to the session store; MOCK authentication
* builds a fresh session on every request and stores nothing, so the
* storedSession table stayed empty and "who is online" could not count
* terminals. This is the smallest hook: the mock module's session customizer
* calls {@link #record(QSession)}, which stores one row per session uuid
* (user id, a lean copy of the session, expiresAt) the first time this JVM
* sees it and touches it (sliding expiry) at most once a minute after that.
*/
final class EnterpriseSessions
{
static final Duration TOUCH_INTERVAL = Duration.ofMinutes(1);
private static final Map<String, Long> LAST_SEEN = new ConcurrentHashMap<>();
private EnterpriseSessions()
{
}
static void record(QSession session)
{
String uuid = session == null ? null : session.getUuid();
if(uuid == null || !QSessionStoreHelper.isSessionStoreAvailable())
{
return;
}
long now = System.currentTimeMillis();
Long last = LAST_SEEN.get(uuid);
if(last != null && now - last < TOUCH_INTERVAL.toMillis())
{
return;
}
LAST_SEEN.put(uuid, now);
if(last == null && QSessionStoreHelper.loadSession(uuid).isEmpty())
{
QSessionStoreHelper.storeSession(uuid, lean(session), QSessionStoreHelper.getDefaultTtl());
}
else
{
QSessionStoreHelper.touchSession(uuid);
}
}
/** The identity only: the admin session's thousand permissions do not belong in a row. */
static QSession lean(QSession session)
{
QSession lean = new QSession();
lean.setUuid(session.getUuid());
lean.setIdReference(session.getUuid());
if(session.getUser() != null)
{
QUser user = new QUser();
user.setIdReference(session.getUser().getIdReference());
user.setFullName(session.getUser().getFullName());
lean.setUser(user);
}
return (lean);
}
/** Tests start from a clean slate. */
static void forget()
{
LAST_SEEN.clear();
}
}
@@ -26,9 +26,9 @@ public final class ProfileLauncher
static String profile(Map<String, String> environment)
{
String selected = environment.getOrDefault("QQQ_ALL_PROFILE", "core");
if(!"core".equals(selected) && !"full".equals(selected))
if(!"core".equals(selected) && !"full".equals(selected) && !EnterpriseFeatures.PROFILE.equals(selected))
{
throw new IllegalArgumentException("QQQ_ALL_PROFILE must be core or full");
throw new IllegalArgumentException("QQQ_ALL_PROFILE must be core, full or enterprise");
}
return selected;
}
@@ -38,7 +38,7 @@ public final class ProfileLauncher
String selected = profile(environment);
Path dataDirectory = Path.of(environment.getOrDefault("QQQ_ALL_DATA_DIR", "./data"));
Integer port = Integer.valueOf(environment.getOrDefault("QQQ_ALL_PORT", "8080"));
if("core".equals(selected))
if("core".equals(selected) || EnterpriseFeatures.PROFILE.equals(selected))
{
CoreRuntime.launch(dataDirectory, port, environment.getOrDefault("QQQ_ALL_BIND_HOST", "127.0.0.1"));
return;
@@ -7,6 +7,7 @@
package com.kingsrook.qqq.all;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.atomic.AtomicInteger;
import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
@@ -110,12 +111,13 @@ public final class QqqAllApplication extends AbstractQQQApplication
instance.addTable(new QTableMetaData().withName("customer").withLabel("Customer")
.withBackendName("h2").withPrimaryKeyField("id")
.withRecordLabelFormat("%s").withRecordLabelFields("name")
.withField(new QFieldMetaData("id", QFieldType.INTEGER))
.withField(new QFieldMetaData("name", QFieldType.STRING))
.withField(new QFieldMetaData("email", QFieldType.STRING)));
instance.addTable(new QTableMetaData().withName("order").withLabel("Order")
.withBackendName("h2").withBackendDetails(new RDBMSTableBackendDetails().withTableName("orders"))
.withPrimaryKeyField("id")
.withPrimaryKeyField("id").withRecordLabelFormat("%s").withRecordLabelFields("orderNo")
.withField(new QFieldMetaData("id", QFieldType.INTEGER))
.withField(new QFieldMetaData("orderNo", QFieldType.STRING).withBackendName("order_no"))
.withField(new QFieldMetaData("customerId", QFieldType.INTEGER).withBackendName("customer_id"))
@@ -156,12 +158,16 @@ public final class QqqAllApplication extends AbstractQQQApplication
new EsbJavalinMetaDataProducer().produce(instance);
instance.withSupplementalMetaData(new HealthCheckMetaData().withEnabled(true)
.withEndpointPath("/health").withIndicators(List.of(new MemoryHealthIndicator().withThreshold(99))));
.withEndpointPath("/health").withIndicators(new ArrayList<>(List.of(new MemoryHealthIndicator().withThreshold(99)))));
QJavalinMetaData.ofOrWithNew(instance)
.withAdditionalRouteProviderReference(new QCodeReference(JavalinHealthRouteProvider.class));
if(includeDemoQbits)
{
DemoQBits.addCore(instance, "h2");
if(EnterpriseFeatures.isEnabled())
{
EnterpriseFeatures.add(instance, "h2", httpPort);
}
DemoUsers.protectDemoTables(instance);
QJavalinMetaData.ofOrWithNew(instance)
.withAdditionalRouteProviderReference(new QCodeReference(DemoOrderWebhookReceiver.class));
@@ -0,0 +1,63 @@
/*
* Copyright 2026 Kingsrook, LLC
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0
*/
package com.kingsrook.qqq.all;
import java.nio.file.Path;
import java.util.Map;
import com.kingsrook.qbits.collaboration.customfields.FieldSecurity;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.session.QSession;
import com.kingsrook.qqq.backend.core.model.session.QUser;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
/** The enterprise admin passes qbit-collaboration's Custom Fields checker (qqq-all#47). */
class EnterpriseAccessTest
{
@TempDir Path dataDirectory;
@AfterEach
void tearDown()
{
QContext.clear();
}
@Test
void adminCanReadAndManageCustomFields() throws Exception
{
QInstance instance = new QqqAllApplication(dataDirectory, "tcp://127.0.0.1:61616").defineQInstance();
QSession plain = new QSession();
QContext.init(instance, plain);
assertThrows(QPermissionDeniedException.class,
() -> new FieldSecurity().checkPermissionsThrowing(new QueryInput("collaboration_field_definition"), null));
QSession admin = new QSession();
DemoUsers.grantEnterpriseAccess(instance, admin, "core:admin");
QContext.init(instance, admin);
assertDoesNotThrow(() -> new FieldSecurity().checkPermissionsThrowing(new QueryInput("collaboration_field_definition"), null));
assertDoesNotThrow(() -> new FieldSecurity().checkPermissionsThrowing(new InsertInput("collaboration_field_definition"), null));
}
@Test
void customizedSessionsStillCarryTheUserForTheFrontend() throws Exception
{
QInstance instance = new QqqAllApplication(dataDirectory, "tcp://127.0.0.1:61616").defineQInstance();
QSession session = new QSession();
session.setUuid(DemoUsers.ADMIN_SESSION);
session.setUser(new QUser());
new DemoUsers().customizeSession(instance, session, Map.of());
assertEquals("Admin", ((Map<?, ?>) session.getValuesForFrontend().get("user")).get("name"));
}
}
@@ -0,0 +1,118 @@
/*
* Copyright 2026 Kingsrook, LLC
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0
*/
package com.kingsrook.qqq.all;
import java.nio.file.Path;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.function.Function;
import java.util.stream.Collectors;
import com.kingsrook.qbits.assistant.AssistantQBitConfig;
import com.kingsrook.qbits.assistant.AssistantQBitProducer;
import com.kingsrook.qbits.assistant.model.AssistantProvider;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.session.QSystemUserSession;
import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryRecordStore;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
/** The enterprise assistant's provider rows follow the environment: Anthropic with a key, else Ollama, else none. */
class EnterpriseAssistantTest
{
@TempDir Path dataDirectory;
@AfterEach
void clear()
{
QContext.clear();
MemoryRecordStore.fullReset();
}
@Test
void choiceFollowsKeyThenOllama()
{
assertEquals(EnterpriseAssistant.Choice.ANTHROPIC, EnterpriseAssistant.choose(env(Map.of("ANTHROPIC_API_KEY", "sk-test")), model -> true));
assertEquals(EnterpriseAssistant.Choice.OLLAMA, EnterpriseAssistant.choose(env(Map.of("ANTHROPIC_API_KEY", " ")), model -> true));
assertEquals(EnterpriseAssistant.Choice.NONE, EnterpriseAssistant.choose(env(Map.of()), model -> false));
}
@Test
void ollamaModelAndUrlFromEnvironment()
{
List<String> asked = new java.util.ArrayList<>();
EnterpriseAssistant.choose(env(Map.of("QQQ_ASSISTANT_OLLAMA_MODEL", "qwen3-coder:30b")), model -> asked.add(model));
assertEquals(List.of("qwen3-coder:30b"), asked);
assertEquals("llama3.3:70b-instruct-q4_K_M", EnterpriseAssistant.ollamaModel(env(Map.of())));
assertEquals("http://127.0.0.1:11434/v1", EnterpriseAssistant.ollamaBaseUrl(env(Map.of())));
AssistantQBitConfig config = EnterpriseAssistant.config("h2", "http://127.0.0.1:8078", env(Map.of("QQQ_ASSISTANT_OLLAMA_URL", "http://127.0.0.1:9999/v1")));
assertTrue(config.getAllowedBaseUrls().contains("http://127.0.0.1:9999/v1"));
assertTrue(config.getProviderTypes().containsKey("anthropic"));
assertTrue(config.getProviderTypes().containsKey("openai-compatible"));
}
@Test
void rowsNeverHoldAKey()
{
List<QRecord> rows = EnterpriseAssistant.rows(EnterpriseAssistant.Choice.ANTHROPIC, env(Map.of("ANTHROPIC_API_KEY", "sk-secret")));
for(QRecord row : rows)
{
assertFalse(row.getValues().values().stream().anyMatch(v -> String.valueOf(v).contains("sk-secret")));
}
assertEquals("ANTHROPIC_API_KEY", rows.get(0).getValueString("apiKeyEnvName"));
assertNull(rows.get(1).getValue("apiKeyEnvName"));
}
@Test
void reconcileInsertsThenSwitchesTheDefault() throws Exception
{
QInstance instance = new QqqAllApplication(dataDirectory, "tcp://127.0.0.1:61616").defineQInstance();
new AssistantQBitProducer().withQBitConfig(EnterpriseAssistant.config("transientMemory", "http://127.0.0.1:1", env(Map.of()))).produce(instance);
QContext.init(instance, new QSystemUserSession());
assertEquals(EnterpriseAssistant.Choice.OLLAMA, EnterpriseAssistant.reconcile(instance, env(Map.of()), model -> true));
Map<String, QRecord> rows = providerRows();
assertEquals(2, rows.size());
assertTrue(rows.get(EnterpriseAssistant.OLLAMA_ROW).getValueBoolean("isDefault"));
assertTrue(rows.get(EnterpriseAssistant.OLLAMA_ROW).getValueBoolean("enabled"));
assertEquals("OLLAMA", rows.get(EnterpriseAssistant.OLLAMA_ROW).getValueString("kind"));
assertEquals("llama3.3:70b-instruct-q4_K_M", rows.get(EnterpriseAssistant.OLLAMA_ROW).getValueString("model"));
assertFalse(rows.get(EnterpriseAssistant.ANTHROPIC_ROW).getValueBoolean("enabled"));
assertEquals(EnterpriseAssistant.Choice.ANTHROPIC, EnterpriseAssistant.reconcile(instance, env(Map.of("ANTHROPIC_API_KEY", "sk-test")), model -> true));
rows = providerRows();
assertEquals(2, rows.size());
assertTrue(rows.get(EnterpriseAssistant.ANTHROPIC_ROW).getValueBoolean("isDefault"));
assertTrue(rows.get(EnterpriseAssistant.ANTHROPIC_ROW).getValueBoolean("enabled"));
assertFalse(rows.get(EnterpriseAssistant.OLLAMA_ROW).getValueBoolean("isDefault"));
assertFalse(rows.get(EnterpriseAssistant.OLLAMA_ROW).getValueBoolean("enabled"));
assertEquals(EnterpriseAssistant.Choice.NONE, EnterpriseAssistant.reconcile(instance, env(Map.of()), model -> false));
assertTrue(providerRows().values().stream().noneMatch(row -> row.getValueBoolean("enabled")));
}
private static Map<String, QRecord> providerRows() throws Exception
{
return new QueryAction().execute(new QueryInput(AssistantProvider.TABLE_NAME)).getRecords().stream()
.collect(Collectors.toMap(row -> row.getValueString("name"), row -> row));
}
private static Function<String, String> env(Map<String, String> values)
{
return new HashMap<>(values)::get;
}
}
@@ -0,0 +1,117 @@
/*
* Copyright 2026 Kingsrook, LLC
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0
*/
package com.kingsrook.qqq.all;
import java.math.BigDecimal;
import java.util.List;
import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessOutput;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType;
import com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.authentication.AuthScope;
import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.security.FieldSecurityLock;
import com.kingsrook.qqq.backend.core.model.metadata.security.QSecurityKeyType;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import com.kingsrook.qqq.backend.core.model.session.QSession;
import com.kingsrook.qqq.backend.core.model.session.QSystemUserSession;
import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryBackendModule;
import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryRecordStore;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
/**
* The enterprise profile's columnStats process (core's step, QRun-IO/qqq#1367)
* refuses statistics for a field the caller cannot read in full and tags its
* results with the analysed table.
*/
class EnterpriseColumnStatsAccessTest
{
private static final String KEY = "salaryViewer";
private QInstance instance;
@BeforeEach
void setUp() throws QException
{
MemoryRecordStore.fullReset();
instance = new QInstance();
instance.registerAuthenticationProvider(AuthScope.instanceDefault(), new QAuthenticationMetaData().withName("mock").withType(QAuthenticationType.MOCK));
instance.addBackend(new QBackendMetaData().withName("memory").withBackendType(MemoryBackendModule.class));
instance.addSecurityKeyType(new QSecurityKeyType().withName(KEY));
instance.addTable(new QTableMetaData().withName("employee").withBackendName("memory").withPrimaryKeyField("id")
.withField(new QFieldMetaData("id", QFieldType.INTEGER))
.withField(new QFieldMetaData("name", QFieldType.STRING))
.withField(new QFieldMetaData("salary", QFieldType.DECIMAL).withLabel("Salary")
.withFieldSecurityLock(new FieldSecurityLock().withSecurityKeyType(KEY)
.withDefaultBehavior(FieldSecurityLock.Behavior.MASK).withOverrideValues(List.of(true)))));
EnterpriseFeatures.enableColumnStatistics(instance);
QContext.init(instance, new QSystemUserSession());
new InsertAction().execute(new InsertInput("employee").withRecords(List.of(
new QRecord().withValue("name", "Ada").withValue("salary", new BigDecimal("100")),
new QRecord().withValue("name", "Grace").withValue("salary", new BigDecimal("200")))));
}
@AfterEach
void tearDown()
{
QContext.clear();
MemoryRecordStore.fullReset();
}
@Test
void restrictedFieldIsRefusedWithoutTheTier() throws QException
{
QContext.init(instance, new QSession());
QPermissionDeniedException denied = assertThrows(QPermissionDeniedException.class, () -> run("salary"));
assertTrue(denied.getMessage() != null && !denied.getMessage().isBlank());
}
@Test
void restrictedFieldIsAllowedWithTheTier() throws QException
{
QContext.init(instance, new QSession().withSecurityKeyValue(KEY, true));
RunProcessOutput output = run("salary");
assertTrue(output.getException().isEmpty(), () -> output.getException().get().getMessage());
QRecord stats = (QRecord) output.getValues().get("statsRecord");
assertEquals(2, stats.getValueInteger("count"));
assertEquals("employee", stats.getTableName());
}
@Test
void unrestrictedFieldIsAllowed() throws QException
{
QContext.init(instance, new QSession());
RunProcessOutput output = run("name");
assertTrue(output.getException().isEmpty(), () -> output.getException().get().getMessage());
}
private RunProcessOutput run(String fieldName) throws QException
{
RunProcessInput input = new RunProcessInput();
input.setProcessName("columnStats");
input.setFrontendStepBehavior(RunProcessInput.FrontendStepBehavior.SKIP);
input.addValue("tableName", "employee");
input.addValue("fieldName", fieldName);
return (new RunProcessAction().execute(input));
}
}
@@ -0,0 +1,81 @@
/*
* Copyright 2026 Kingsrook, LLC
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0
*/
package com.kingsrook.qqq.all;
import java.nio.file.Path;
import com.kingsrook.qqq.backend.core.model.audits.AuditsMetaDataProvider;
import com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.audits.AuditLevel;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.tables.Capability;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
/** The enterprise profile offers column statistics wherever the backend can aggregate. */
class EnterpriseColumnStatsTest
{
@TempDir Path dataDirectory;
@Test
void queryStatsOnAggregatingBackendsOnly() throws Exception
{
QInstance instance = new QqqAllApplication(dataDirectory, "tcp://127.0.0.1:61616").defineQInstance();
for(String tableName : new String[] {"customer", "order", "orderLine", "demoNote"})
{
assertFalse(hasQueryStats(instance, tableName), tableName + " before");
}
EnterpriseFeatures.enableColumnStatistics(instance);
assertNotNull(instance.getProcess("columnStats"));
///////////////////////////////////////////////////////////////////
// H2 (customer, order), SQLite (orderLine) and memory (demoNote) //
///////////////////////////////////////////////////////////////////
for(String tableName : new String[] {"customer", "order", "orderLine", "demoNote"})
{
assertTrue(hasQueryStats(instance, tableName), tableName);
assertTrue(hasCapability(instance, tableName, Capability.TABLE_COUNT), tableName);
assertTrue(hasCapability(instance, tableName, Capability.TABLE_EXPORT), tableName);
}
////////////////////////////////////////////////////////////////
// the filesystem (product) and API (apiCatalog) backends do //
// not aggregate, so they stay without statistics //
////////////////////////////////////////////////////////////////
assertFalse(hasQueryStats(instance, "product"));
assertFalse(hasQueryStats(instance, "apiCatalog"));
}
@Test
void dataTablesAreAuditedWhenAuditsAreInstalled() throws Exception
{
QInstance instance = new QqqAllApplication(dataDirectory, "tcp://127.0.0.1:61616").defineQInstance();
new AuditsMetaDataProvider().defineAll(instance, "h2", null);
EnterpriseFeatures.auditDataTables(instance);
for(String tableName : new String[] {"customer", "order", "orderLine", "address"})
{
assertEquals(AuditLevel.FIELD, instance.getTable(tableName).getAuditRules().getAuditLevel(), tableName);
}
}
private static boolean hasQueryStats(QInstance instance, String tableName)
{
return hasCapability(instance, tableName, Capability.QUERY_STATS);
}
private static boolean hasCapability(QInstance instance, String tableName, Capability capability)
{
QTableMetaData table = instance.getTable(tableName);
QBackendMetaData backend = instance.getBackendForTable(tableName);
return table.isCapabilityEnabled(backend, capability);
}
}
@@ -0,0 +1,59 @@
/*
* Copyright 2026 Kingsrook, LLC
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0
*/
package com.kingsrook.qqq.all;
import java.nio.file.Path;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import com.kingsrook.qbits.quicksearch.SearchableFieldConfig;
import com.kingsrook.qbits.quicksearch.SearchableTableConfig;
import com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryBackendModule;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
/** Quick Search covers the demo and sample tables, with their text fields only. */
class EnterpriseQuickSearchTablesTest
{
@TempDir Path dataDirectory;
@Test
void demoAndSampleTablesWithTheirTextFields() throws Exception
{
QInstance instance = new QqqAllApplication(dataDirectory, "tcp://127.0.0.1:61616").defineQInstance();
///////////////////////////////////////////////////////////////////////
// a stand-in for a sample-app table: labelled, on the sample's //
// backend, with fields OpenSearch must not index as text //
///////////////////////////////////////////////////////////////////////
instance.addBackend(new QBackendMetaData().withName("rdbms").withBackendType(MemoryBackendModule.class));
instance.addTable(new QTableMetaData().withName("countryState").withBackendName("rdbms").withPrimaryKeyField("id")
.withRecordLabelFormat("%s").withRecordLabelFields("name")
.withField(new QFieldMetaData("id", QFieldType.INTEGER))
.withField(new QFieldMetaData("name", QFieldType.STRING))
.withField(new QFieldMetaData("description", QFieldType.STRING))
.withField(new QFieldMetaData("code", QFieldType.INTEGER))
.withField(new QFieldMetaData("createDate", QFieldType.DATE_TIME)));
Map<String, List<String>> tables = EnterpriseFeatures.searchableTables(instance).stream()
.collect(Collectors.toMap(SearchableTableConfig::getTableName,
table -> table.getFields().stream().map(SearchableFieldConfig::getFieldName).toList()));
assertEquals(List.of("name", "email"), tables.get("customer").stream().sorted((a, b) -> a.equals("name") ? -1 : 1).toList());
assertTrue(tables.get("shipping_stateProvince").contains("name"));
assertEquals(List.of("name", "description"), tables.get("countryState"));
assertFalse(tables.containsKey("webhook"), "QBit admin tables on h2 stay out");
assertFalse(tables.containsKey("product"), "the filesystem product table is not a demo search table");
}
}
@@ -0,0 +1,115 @@
/*
* Copyright 2026 Kingsrook, LLC
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0
*/
package com.kingsrook.qqq.all;
import java.time.Duration;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import com.kingsrook.qqq.backend.core.model.session.QSession;
import com.kingsrook.qqq.backend.core.model.session.QUser;
import com.kingsrook.qqq.backend.core.modules.authentication.QSessionStoreProviderInterface;
import com.kingsrook.qqq.backend.core.modules.authentication.QSessionStoreRegistry;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNull;
/** Mock sign-ins become one stored session per uuid, touched rather than re-stored. */
class EnterpriseSessionsTest
{
private final RecordingStore store = new RecordingStore();
@BeforeEach
void setUp()
{
EnterpriseSessions.forget();
QSessionStoreRegistry.getInstance().clear();
QSessionStoreRegistry.getInstance().register(store);
}
@AfterEach
void tearDown()
{
QSessionStoreRegistry.getInstance().clear();
EnterpriseSessions.forget();
}
@Test
void oneRowPerSessionUuid()
{
EnterpriseSessions.record(session("aaaa", "core:admin"));
EnterpriseSessions.record(session("aaaa", "core:admin"));
EnterpriseSessions.record(session("bbbb", "core:admin"));
assertEquals(List.of("aaaa", "bbbb"), new ArrayList<>(store.sessions.keySet()));
assertEquals("core:admin", store.sessions.get("bbbb").getUser().getIdReference());
assertNull(store.sessions.get("aaaa").getPermissions(), "only the identity is stored");
}
@Test
void aKnownSessionIsTouchedNotStoredAgain()
{
store.sessions.put("cccc", session("cccc", "core:admin"));
EnterpriseSessions.record(session("cccc", "core:admin"));
assertEquals(0, store.stores);
assertEquals(1, store.touches);
}
private static QSession session(String uuid, String userId)
{
QSession session = new QSession();
session.setUuid(uuid);
QUser user = new QUser();
user.setIdReference(userId);
session.setUser(user);
session.withPermissions("customer.read", "customer.edit");
return (session);
}
/** A session store that keeps what it is given and counts calls. */
private static class RecordingStore implements QSessionStoreProviderInterface
{
private final Map<String, QSession> sessions = new LinkedHashMap<>();
private int stores;
private int touches;
@Override
public void store(String sessionUuid, QSession session, Duration ttl)
{
stores++;
sessions.put(sessionUuid, session);
}
@Override
public Optional<QSession> load(String sessionUuid)
{
return (Optional.ofNullable(sessions.get(sessionUuid)));
}
@Override
public void remove(String sessionUuid)
{
sessions.remove(sessionUuid);
}
@Override
public void touch(String sessionUuid)
{
touches++;
}
@Override
public Duration getDefaultTtl()
{
return (Duration.ofHours(1));
}
}
}
+4 -4
View File
@@ -148,11 +148,11 @@ class SmokeTest(unittest.TestCase):
pom = ET.parse(ROOT / "qqq-all-app" / "pom.xml")
final_name = pom.findtext(".//{http://maven.apache.org/POM/4.0.0}build/"
"{http://maven.apache.org/POM/4.0.0}finalName")
# The shared all profile owns the smoke command; this repository owns
# the Maven artifact it consumes. Keep that cross-repository contract.
# CI smokes the exact jar Maven builds; keep the build output and the
# CI smoke command on the same artifact path.
self.assertEqual("qqq-all-app", final_name)
manifest = (ROOT / ".qqq-ci.yml").read_text()
self.assertRegex(manifest, r"(?m)^profile: all$")
workflow = (ROOT / ".gitea" / "workflows" / "build.yaml").read_text()
self.assertIn("python3 scripts/run_core_smoke.py qqq-all-app/target/qqq-all-app.jar", workflow)
def test_standalone_full_smoke_requires_explicit_search_endpoint(self):