Compare commits

...
Author SHA1 Message Date
james.maes fbce2e5d6e Merge pull request 'fix(targets): ask whether a table is readable as a query, not with an untyped input' (#69) from fix/target-read-asked-as-query into develop
build / maven (push) Successful in 6m55s
maven / build (push) Successful in 6m55s
2026-11-12 17:20:43 +00:00
james.maesandClaude Fable 5.2 70f6184a7a fix(targets): ask whether a table is readable as a query, not with an untyped input
build / maven (push) Successful in 8m46s
maven / build (push) Successful in 8m41s
TargetResolver.canReadTable asked PermissionsHelper.hasTablePermission with a
bare AbstractActionInput. Core hands a table's custom permission checker the
action input and not the permission asked, so a checker that tells a read
from a write by the input's type (a query, a get or a count reads; anything
else is refused) refused the question. For such a table every recents and
favorites call answered "Not found" for people the checker lets read it:
the UI sends POST /recents on every list and record screen, so each of them
showed "Resource not found POST /qqq/v1/qbit/collaboration/recents (HTTP
404)".

The read is now asked with a QueryInput for the table, the read it stands
for; ParentRecordAccess already asks with a GetInput. Who may read a table
does not change: the checker still decides, and a person it refuses is
still answered "Not found" (RecentsServiceTest
.tableWithAnInputTypedPermissionCheckerIsATarget, which failed before with
QNotFoundException on the first call).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-12 10:53:22 -06:00
james.maes 16bd4f768f Merge pull request 'perf: skip tag lookups for tables with no tag assignments (forward-port of #67)' (#68) from fix/read-path-statements-develop into develop
build / maven (push) Successful in 7m34s
maven / build (push) Successful in 7m34s
2026-11-10 22:11:02 +00:00
james.maesandClaude Fable 5.2 04d04adc6e perf(security): authorize each distinct parent once per child result page
build / maven (push) Successful in 7m15s
maven / build (push) Successful in 7m15s
ParentRowCustomizer and AttachmentPreviewCustomizer read the parent of
every child row to decide whether the row is visible, so rows that share a
parent repeated the same read: 61 tasks on two parents ran 61 parent reads
(TagLookupStatementBudgetTest), now 2. The answer is kept only for the one
postQuery call, where the session and transaction cannot change.

Also correct the README: the Tags field is selectable.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 15:58:11 -06:00
james.maesandClaude Fable 5.2 de922d7e86 perf(tags): no assignment lookup for tables that have no tags
Core selects the Tags virtual field for every query or get that names no
fields, so every such read of a TAGS-enabled table ran one
collaboration_record_tag statement, including reads made one record at a
time and tables that were never tagged. Measured with 600 added parents
(TagLookupStatementBudgetTest): a 602-row list ran 2 statements, 600 gets
ran 600 and 100 single-record updates ran 100; now 1 probe, then 0.

TaggedTables remembers, per node and per table, whether any assignment
exists (core Memoization registered with ClusterCaches, 30 s by default,
withTagLookupCacheSeconds, 0 disables). Writing an assignment clears it at
once and after commit and publishes memo-invalidate. The probe ignores row
locks and returns no data, so the answer is the same for every session.
Tables with assignments are read as before: one lookup per 500 parents.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 15:58:11 -06:00
james.maes c6b077321c Merge pull request 'fix(upload-journal): accept Amazon Aurora PostgreSQL (full_page_writes=off)' (#65) from fix/aurora-full-page-writes-develop into develop
build / maven (push) Successful in 7m35s
maven / build (push) Successful in 7m35s
2026-11-10 15:27:45 +00:00
james.maesandClaude Fable 5.2 2837f4152a fix(upload-journal): accept Amazon Aurora PostgreSQL (full_page_writes=off)
build / maven (push) Successful in 9m8s
maven / build (push) Successful in 9m8s
The PostgreSQL 17 journal qualification refused every call on Amazon Aurora
PostgreSQL, which reports full_page_writes=off and does not expose the
parameter in its cluster parameter group.

- The settings query gains an eleventh column,
  to_regprocedure('aurora_version()') IS NOT NULL, which is true on Aurora,
  false on stock PostgreSQL 17 and never raises.
- full_page_writes=off is accepted only when that column is true. Stock
  PostgreSQL with full_page_writes=off is still refused, and synchronous_commit,
  fsync, recovery, isolation, time zone and lock timeout are unchanged on Aurora.
- The decision is the pure function tornPagesPrevented, covered by an always-on
  unit test. The opt-in real PostgreSQL 17 suite simulates Aurora with a
  function named aurora_version() and full_page_writes=off on the server.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-09 20:26:14 -06:00
james.maes f0f1db7642 Merge pull request 'build(deps): align with Gitea registry (parent 2.2.1, QQQ 4.1.0-RC.26)' (#63) from deps/align-gitea into develop
build / maven (push) Successful in 7m59s
maven / build (push) Successful in 7m59s
2026-11-07 18:03:38 +00:00
james.maesandClaude Opus 5.5 c8150bc9cf build(deps): align with Gitea registry (parent 2.2.1, QQQ 4.1.0-RC.26)
build / maven (push) Successful in 7m48s
maven / build (push) Successful in 7m48s
pom.xml: parent -> 2.2.1
pom.xml: qqq.version: 4.1.0-RC.22 -> 4.1.0-RC.26
pom.xml: qbit-object-storage: 0.1.0-RC.4 -> 0.1.0-RC.10
pom.xml: qbit-redis.version: 0.1.0-RC.1 -> 0.2.0-RC.1
pom.xml: qbit-email.version: 0.1.0-RC.2 -> 0.1.0-RC.11
pom.xml: revision -> 1.2.0-SNAPSHOT

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 17:55:19 +00:00
james.maes 1c6ae14f59 fix(security): check parent-table read permission with a real GetInput (#62)
QRun-IO/ci-policy: native / dotnet-0 (push) Skipped
QRun-IO/ci-policy: ci / java-core-2-head (push) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (push) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (push) Skipped
QRun-IO/ci-policy: ci / source-config-14 (push) Skipped
QRun-IO/ci-policy: ci / original-voyage (push) Skipped
QRun-IO/ci-policy: ci / original-cli (push) Skipped
QRun-IO/ci-policy: ci / ci-result (push) Successful in 10s
build / maven (push) Successful in 15m15s
QRun-IO/ci-policy: ci / fork-guard (push) Skipped
QRun-IO/ci-policy: native / fork-deny (push) Skipped
QRun-IO/ci-policy: ci / head-profile (push) Skipped
QRun-IO/ci-policy: ci / repository-policy (push) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (push) Skipped
QRun-IO/ci-policy: native / native-result (push) Successful in 13s
QRun-IO/ci-policy: ci / java-maven-7-head (push) Skipped
QRun-IO/ci-policy: ci / node-8 (push) Skipped
QRun-IO/ci-policy: ci / node-9 (push) Skipped
QRun-IO/ci-policy: ci / node-10 (push) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (push) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (push) Skipped
QRun-IO/ci-policy: ci / source-config-13 (push) Skipped
QRun-IO/ci-policy: ci / original-business (push) Skipped
QRun-IO/ci-policy: ci / application-template-0 (push) Skipped
QRun-IO/ci-policy: ci / gradle-1 (push) Skipped
QRun-IO/ci-policy: ci / java-docker-3-head (push) Skipped
QRun-IO/ci-policy: ci / java-maven-6 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (push) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (push) Skipped
QRun-IO/ci-policy: ci / android-0 (push) Skipped
QRun-IO/ci-policy: ci / not-enrolled (push) Skipped
QRun-IO/ci-policy: ci / Scoped workflow [workflow 539bec410f7926fec3ef7fabe51934a91e0248be] (push) Successful in 14s
QRun-IO/ci-policy: ci / shared-pipeline (push) Skipped
maven / build (push) Successful in 15m15s
QRun-IO/ci-policy: ci / original-mcp (push) Skipped
2026-11-06 19:30:40 +00:00
James Maes 9248f3f32c test: keep develop's CollaborationApiHttpTest unchanged
QRun-IO/ci-policy: ci / fork-guard (pull_request) Skipped
QRun-IO/ci-policy: ci / Scoped workflow [workflow 539bec410f7926fec3ef7fabe51934a91e0248be] (pull_request) Successful in 11s
QRun-IO/ci-policy: ci / head-profile (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy (pull_request) Skipped
QRun-IO/ci-policy: ci / application-template-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-8 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-business (pull_request) Skipped
QRun-IO/ci-policy: ci / original-voyage (pull_request) Skipped
QRun-IO/ci-policy: ci / original-cli (pull_request) Skipped
build / maven (push) Successful in 10m50s
QRun-IO/ci-policy: native / fork-deny (pull_request) Skipped
QRun-IO/ci-policy: native / dotnet-0 (pull_request) Skipped
QRun-IO/ci-policy: native / native-result (pull_request) Successful in 9s
QRun-IO/ci-policy: ci / java-core-2-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7-head (pull_request) Skipped
QRun-IO/ci-policy: ci / android-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-mcp (pull_request) Skipped
QRun-IO/ci-policy: ci / not-enrolled (pull_request) Skipped
QRun-IO/ci-policy: ci / ci-result (pull_request) Successful in 10s
QRun-IO/ci-policy: ci / gradle-1 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-10 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-13 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-14 (pull_request) Skipped
QRun-IO/ci-policy: ci / shared-pipeline (pull_request) Skipped
maven / build (push) Successful in 10m51s
QRun-IO/ci-policy: ci / node-9 (pull_request) Skipped
2026-11-06 13:16:55 -06:00
James MaesandClaude Fable 5.1 468dc42751 fix(security): check parent-table read permission with a real GetInput
QRun-IO/ci-policy: ci / fork-guard (pull_request) Skipped
QRun-IO/ci-policy: ci / Scoped workflow [workflow 539bec410f7926fec3ef7fabe51934a91e0248be] (pull_request) Successful in 11s
QRun-IO/ci-policy: ci / head-profile (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy (pull_request) Skipped
QRun-IO/ci-policy: ci / application-template-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-8 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-business (pull_request) Skipped
QRun-IO/ci-policy: ci / original-voyage (pull_request) Skipped
QRun-IO/ci-policy: ci / original-cli (pull_request) Skipped
build / maven (push) Successful in 11m14s
QRun-IO/ci-policy: native / fork-deny (pull_request) Skipped
QRun-IO/ci-policy: native / dotnet-0 (pull_request) Skipped
QRun-IO/ci-policy: native / native-result (pull_request) Successful in 8s
QRun-IO/ci-policy: ci / java-core-2-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7-head (pull_request) Skipped
QRun-IO/ci-policy: ci / android-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-mcp (pull_request) Skipped
QRun-IO/ci-policy: ci / not-enrolled (pull_request) Skipped
QRun-IO/ci-policy: ci / ci-result (pull_request) Successful in 8s
QRun-IO/ci-policy: ci / gradle-1 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-10 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-13 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-14 (pull_request) Skipped
QRun-IO/ci-policy: ci / shared-pipeline (pull_request) Skipped
maven / build (push) Successful in 11m14s
QRun-IO/ci-policy: ci / node-9 (pull_request) Skipped
ParentRecordAccess.canReadTable passed a bare AbstractActionInput to
PermissionsHelper.hasTablePermission, so the READ check did not run as an actual
table read. Pass a GetInput for the table instead, which is what a host's table
permission rules evaluate. Ported from the GG Forge parent-read qualification.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 13:16:06 -06:00
james.maes 73681b3a2e Merge pull request 'ci: pin shared workflow to v1' (#60) from ci/pin-v1-develop into develop
QRun-IO/ci-policy: native / dotnet-0 (push) Skipped
QRun-IO/ci-policy: ci / java-core-2-head (push) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (push) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (push) Skipped
QRun-IO/ci-policy: ci / source-config-14 (push) Skipped
QRun-IO/ci-policy: ci / original-voyage (push) Skipped
QRun-IO/ci-policy: ci / original-cli (push) Skipped
QRun-IO/ci-policy: ci / ci-result (push) Waiting to run
build / maven (push) Successful in 9m44s
QRun-IO/ci-policy: ci / fork-guard (push) Skipped
QRun-IO/ci-policy: native / fork-deny (push) Skipped
QRun-IO/ci-policy: ci / Scoped workflow [workflow 1e25bb5a243dfdcd95d4b9a4f780200ff8c8778a] (push) Successful in 14s
QRun-IO/ci-policy: native / native-result (push) Successful in 12s
QRun-IO/ci-policy: ci / head-profile (push) Skipped
QRun-IO/ci-policy: ci / repository-policy (push) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-7-head (push) Skipped
QRun-IO/ci-policy: ci / node-8 (push) Skipped
QRun-IO/ci-policy: ci / node-9 (push) Skipped
QRun-IO/ci-policy: ci / node-10 (push) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (push) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (push) Skipped
QRun-IO/ci-policy: ci / source-config-13 (push) Skipped
QRun-IO/ci-policy: ci / original-business (push) Skipped
QRun-IO/ci-policy: ci / application-template-0 (push) Skipped
QRun-IO/ci-policy: ci / gradle-1 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-6 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (push) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (push) Skipped
QRun-IO/ci-policy: ci / android-0 (push) Skipped
QRun-IO/ci-policy: ci / not-enrolled (push) Skipped
QRun-IO/ci-policy: ci / shared-pipeline (push) Skipped
maven / build (push) Successful in 9m44s
QRun-IO/ci-policy: ci / java-docker-3-head (push) Skipped
QRun-IO/ci-policy: ci / original-mcp (push) Skipped
2026-11-06 16:04:53 +00:00
james.maes 483dc271fe Merge pull request 'fix(custom-fields): reject over-long STRING values and reload help after publication' (#61) from fix/custom-field-length-and-help-20261106 into develop
QRun-IO/ci-policy: native / dotnet-0 (push) Skipped
QRun-IO/ci-policy: ci / java-core-2-head (push) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (push) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (push) Skipped
QRun-IO/ci-policy: ci / source-config-14 (push) Skipped
QRun-IO/ci-policy: ci / original-voyage (push) Skipped
QRun-IO/ci-policy: ci / original-cli (push) Skipped
QRun-IO/ci-policy: ci / ci-result (push) Successful in 8s
build / maven (push) Successful in 10m55s
QRun-IO/ci-policy: ci / fork-guard (push) Skipped
QRun-IO/ci-policy: native / fork-deny (push) Skipped
QRun-IO/ci-policy: ci / Scoped workflow [workflow 1e25bb5a243dfdcd95d4b9a4f780200ff8c8778a] (push) Successful in 10s
QRun-IO/ci-policy: ci / head-profile (push) Skipped
QRun-IO/ci-policy: native / native-result (push) Successful in 8s
QRun-IO/ci-policy: ci / repository-policy (push) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-7-head (push) Skipped
QRun-IO/ci-policy: ci / node-8 (push) Skipped
QRun-IO/ci-policy: ci / node-9 (push) Skipped
QRun-IO/ci-policy: ci / node-10 (push) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (push) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (push) Skipped
QRun-IO/ci-policy: ci / source-config-13 (push) Skipped
QRun-IO/ci-policy: ci / original-business (push) Skipped
QRun-IO/ci-policy: ci / application-template-0 (push) Skipped
QRun-IO/ci-policy: ci / gradle-1 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-6 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (push) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (push) Skipped
QRun-IO/ci-policy: ci / android-0 (push) Skipped
QRun-IO/ci-policy: ci / not-enrolled (push) Skipped
QRun-IO/ci-policy: ci / shared-pipeline (push) Skipped
maven / build (push) Successful in 10m55s
QRun-IO/ci-policy: ci / java-docker-3-head (push) Skipped
QRun-IO/ci-policy: ci / original-mcp (push) Skipped
2026-11-06 15:32:01 +00:00
james.maes 813894f8c6 Merge pull request 'Integrate attachments, upload journal, presence and Redis presence for KOF22' (#59) from kof22/collab-v2 into develop
QRun-IO/ci-policy: native / dotnet-0 (push) Skipped
QRun-IO/ci-policy: ci / java-core-2-head (push) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (push) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (push) Skipped
QRun-IO/ci-policy: ci / source-config-14 (push) Skipped
QRun-IO/ci-policy: ci / original-voyage (push) Skipped
QRun-IO/ci-policy: ci / original-cli (push) Skipped
QRun-IO/ci-policy: ci / ci-result (push) Successful in 8s
build / maven (push) Failing after 10m42s
QRun-IO/ci-policy: ci / fork-guard (push) Skipped
QRun-IO/ci-policy: native / fork-deny (push) Skipped
QRun-IO/ci-policy: ci / Scoped workflow [workflow 1e25bb5a243dfdcd95d4b9a4f780200ff8c8778a] (push) Successful in 9s
QRun-IO/ci-policy: native / native-result (push) Successful in 7s
QRun-IO/ci-policy: ci / head-profile (push) Skipped
QRun-IO/ci-policy: ci / repository-policy (push) Skipped
QRun-IO/ci-policy: ci / java-maven-7-head (push) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (push) Skipped
QRun-IO/ci-policy: ci / node-8 (push) Skipped
QRun-IO/ci-policy: ci / node-9 (push) Skipped
QRun-IO/ci-policy: ci / node-10 (push) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (push) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (push) Skipped
QRun-IO/ci-policy: ci / source-config-13 (push) Skipped
QRun-IO/ci-policy: ci / original-business (push) Skipped
QRun-IO/ci-policy: ci / application-template-0 (push) Skipped
QRun-IO/ci-policy: ci / gradle-1 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-6 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (push) Skipped
QRun-IO/ci-policy: ci / android-0 (push) Skipped
QRun-IO/ci-policy: ci / not-enrolled (push) Skipped
QRun-IO/ci-policy: ci / shared-pipeline (push) Skipped
maven / build (push) Failing after 10m42s
QRun-IO/ci-policy: ci / java-docker-3-head (push) Skipped
QRun-IO/ci-policy: ci / original-mcp (push) Skipped
2026-11-06 14:52:51 +00:00
james.maesandClaude Fable 5.1 ef625ca300 fix(custom-fields): reject over-long STRING values and reload help after publication
QRun-IO/ci-policy: ci / fork-guard (pull_request) Skipped
QRun-IO/ci-policy: ci / head-profile (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy (pull_request) Skipped
QRun-IO/ci-policy: ci / application-template-0 (pull_request) Skipped
build / maven (push) Successful in 5m44s
QRun-IO/ci-policy: ci / java-maven-6 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-8 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-business (pull_request) Skipped
QRun-IO/ci-policy: ci / original-voyage (pull_request) Skipped
QRun-IO/ci-policy: native / fork-deny (pull_request) Skipped
QRun-IO/ci-policy: native / dotnet-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / Scoped workflow [workflow 1e25bb5a243dfdcd95d4b9a4f780200ff8c8778a] (pull_request) Successful in 7s
QRun-IO/ci-policy: native / native-result (pull_request) Successful in 6s
QRun-IO/ci-policy: ci / java-core-2-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7-head (pull_request) Skipped
QRun-IO/ci-policy: ci / android-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-mcp (pull_request) Skipped
QRun-IO/ci-policy: ci / original-cli (pull_request) Skipped
QRun-IO/ci-policy: ci / not-enrolled (pull_request) Skipped
QRun-IO/ci-policy: ci / ci-result (pull_request) Successful in 6s
maven / build (push) Successful in 5m44s
QRun-IO/ci-policy: ci / gradle-1 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-10 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-13 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-14 (pull_request) Skipped
QRun-IO/ci-policy: ci / shared-pipeline (pull_request) Skipped
QRun-IO/ci-policy: ci / node-9 (pull_request) Skipped
Bounded STRING custom fields now carry QQQ's native ValueTooLongBehavior.ERROR,
so a write longer than maxLength is rejected with a field error and the stored
value is preserved. The QQQ default is PASS_THROUGH, which let over-long values
through unchecked.

After the runtime metadata overlay refreshes (immediately, or in the after-commit
callback of the originating transaction), persisted help content is reloaded so
sections and fields created by publishing a definition receive their stored
helpContent rows without a restart.

Two CustomFieldsTest regressions cover invalid-write preservation and stored
help on a newly published section. Carried locally by GG Forge as
collaboration-custom-fields.patch (supersedes collaboration-field-length.patch).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: James Maes <james@kof22.com>
2026-11-06 08:39:48 -06:00
james.maesandClaude Opus 5.5 6060b737c5 ci: pin shared workflow to v1
QRun-IO/ci-policy: ci / fork-guard (pull_request) Skipped
build / maven (push) Successful in 3m35s
QRun-IO/ci-policy: ci / head-profile (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy (pull_request) Skipped
QRun-IO/ci-policy: ci / application-template-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-8 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-business (pull_request) Skipped
QRun-IO/ci-policy: ci / original-voyage (pull_request) Skipped
QRun-IO/ci-policy: ci / original-cli (pull_request) Skipped
QRun-IO/ci-policy: native / fork-deny (pull_request) Skipped
QRun-IO/ci-policy: native / dotnet-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / Scoped workflow [workflow 1e25bb5a243dfdcd95d4b9a4f780200ff8c8778a] (pull_request) Successful in 9s
QRun-IO/ci-policy: native / native-result (pull_request) Successful in 9s
QRun-IO/ci-policy: ci / java-core-2-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7-head (pull_request) Skipped
QRun-IO/ci-policy: ci / android-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-mcp (pull_request) Skipped
QRun-IO/ci-policy: ci / not-enrolled (pull_request) Skipped
QRun-IO/ci-policy: ci / ci-result (pull_request) Waiting to run
maven / build (push) Successful in 3m36s
QRun-IO/ci-policy: ci / gradle-1 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-10 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-13 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-14 (pull_request) Skipped
QRun-IO/ci-policy: ci / shared-pipeline (pull_request) Skipped
QRun-IO/ci-policy: ci / node-9 (pull_request) Skipped
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 14:06:32 +00:00
james.maesandClaude Sonnet 5.5 024cfc21a9 release: prepare Collaboration 1.0 (Core RC.22 BOM, Redis RC.1, current status docs)
QRun-IO/ci-policy: ci / node-8 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-business (pull_request) Skipped
QRun-IO/ci-policy: ci / original-voyage (pull_request) Skipped
QRun-IO/ci-policy: ci / original-cli (pull_request) Skipped
QRun-IO/ci-policy: ci / fork-guard (pull_request) Skipped
build / maven (push) Successful in 11m4s
QRun-IO/ci-policy: ci / head-profile (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy (pull_request) Skipped
QRun-IO/ci-policy: ci / application-template-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (pull_request) Skipped
QRun-IO/ci-policy: ci / android-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-mcp (pull_request) Skipped
QRun-IO/ci-policy: ci / not-enrolled (pull_request) Skipped
QRun-IO/ci-policy: ci / Scoped workflow [workflow 1e25bb5a243dfdcd95d4b9a4f780200ff8c8778a] (pull_request) Successful in 12s
QRun-IO/ci-policy: native / native-result (pull_request) Successful in 9s
QRun-IO/ci-policy: ci / ci-result (pull_request) Successful in 6s
QRun-IO/ci-policy: native / fork-deny (pull_request) Skipped
QRun-IO/ci-policy: native / dotnet-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-core-2-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7-head (pull_request) Skipped
QRun-IO/ci-policy: ci / node-10 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-13 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-14 (pull_request) Skipped
QRun-IO/ci-policy: ci / shared-pipeline (pull_request) Skipped
maven / build (push) Successful in 11m4s
QRun-IO/ci-policy: ci / gradle-1 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-9 (pull_request) Skipped
Replace the default-active Core snapshot profile with the released Core
4.1.0-RC.22 BOM, pin qbit-redis to its first release candidate, and replace
the snapshot-era status text with the current 1.0 status.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-05 17:25:50 -06:00
James MaesandClaude Sonnet 5.5 63245b14e0 feat(collaboration): integrate presence, Redis presence, virtual-field and fixture work on the attachment base
QRun-IO/ci-policy: ci / fork-guard (pull_request) Skipped
QRun-IO/ci-policy: ci / Scoped workflow [workflow d20a578c7d818bc7ca32a2ac2c675da4e4f27385] (pull_request) Successful in 5s
QRun-IO/ci-policy: ci / head-profile (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy (pull_request) Skipped
QRun-IO/ci-policy: ci / application-template-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-8 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-business (pull_request) Skipped
QRun-IO/ci-policy: ci / original-voyage (pull_request) Skipped
QRun-IO/ci-policy: ci / original-cli (pull_request) Skipped
build / maven (push) Failing after 42s
QRun-IO/ci-policy: native / fork-deny (pull_request) Skipped
QRun-IO/ci-policy: native / dotnet-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-core-2-head (pull_request) Skipped
QRun-IO/ci-policy: native / native-result (pull_request) Successful in 4s
QRun-IO/ci-policy: ci / java-maven-7-head (pull_request) Skipped
QRun-IO/ci-policy: ci / android-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-mcp (pull_request) Skipped
QRun-IO/ci-policy: ci / not-enrolled (pull_request) Skipped
QRun-IO/ci-policy: ci / ci-result (pull_request) Successful in 5s
QRun-IO/ci-policy: ci / gradle-1 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-10 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-13 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-14 (pull_request) Skipped
QRun-IO/ci-policy: ci / shared-pipeline (pull_request) Skipped
maven / build (push) Failing after 42s
QRun-IO/ci-policy: ci / node-9 (pull_request) Skipped
One reviewable integration on current develop 7d61f29e. It starts from
feat/record-files-standalone (attachment lifecycle, upload journal, retention
SPI; descends from develop) and adds the owner stack content from
feature/collab, feature/collab-int-20261029, feature/field-values-1196,
feature/presence-recovery-1300, test/presence-capacity-1302 and the
fix/attachment-transactions-1289 / fix/mywork-query-bounds-1291 work, plus
KOF22 adaptations: CollaborationQBitConfig.attachmentLifecycle,
CollaborationRecords.insertActivity/query(transaction), ActivityWriter source
feature routing, and the Redis/fixture tests.

Build: develop CI files are preserved. pom.xml only adds the Redis API
guardrail allow-list and an optional qbit-redis dependency; the core and
qbit-redis versions stay on the canonical SNAPSHOT lines with markers to repin
after the core and redis integration PRs are published (order CORE -> REDIS ->
COLLAB). No PRIVATE or feature-snapshot coordinate is introduced. Not yet
compiled: it needs the core integration artifact (virtual-field providers).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-05 17:19:49 -06:00
James MaesandGPT-6 Codex fb7c6316bf docs: record successful Record Files consumer verification
Co-Authored-By: GPT-6 Codex <noreply@openai.com>
2026-11-05 17:19:48 -06:00
James MaesandGPT-6 Codex 9fbe20c9d6 fix: handle upload recovery metadata and align Core dependency
Verify all 269 Collaboration tests, including PostgreSQL attachment recovery and MySQL/PostgreSQL journals, against the compatible Core snapshot.

Co-Authored-By: GPT-6 Codex <noreply@openai.com>
2026-11-05 17:19:47 -06:00
1c05422901 chore: preserve paused QBit work and document remaining issues
Save the current implementation and README, status, and known-issue documentation at the user-requested stop. No completed release or deployment is claimed.

Co-Authored-By: Claude <noreply@anthropic.com>

Co-Authored-By: GPT-6 Codex <noreply@openai.com>
2026-11-05 17:19:47 -06:00
James MaesandClaude Sonnet 5.5 89e77b67b9 feat(attachments): reconcile reviewed attachment lifecycle, upload journal and retention SPI
Selective three-way reconciliation onto develop 7d61f29 (base c4eb972 = original
ancestor; theirs = reviewed b6fb2382 third-party-src/qbit-collaboration).
Attachments package (main and tests), upload journal (H2/MySQL/PostgreSQL),
cleanup queue, recovery/adoption processes, attachment schema fields, tables,
pagination, delivery schema, help text and docs. Upstream presence work, CI
workflows and release convention are untouched. Redis presence, my-work bounds,
notifier and custom-field changes in b6 are deliberately not included.

Targets qqq 4.1.0-RC.21 BOM. Needs core QBackendTransaction after-rollback
callbacks (not in RC.21): see docs/CORE-DEPENDENCY.md. Not compiled against RC.21.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-05 17:19:46 -06:00
james.maes 9c51710f09 Merge pull request 'ci: use shared ci-workflows Maven workflow' (#57) from ci/ci-workflows into develop
QRun-IO/ci-policy: native / dotnet-0 (push) Skipped
QRun-IO/ci-policy: ci / java-core-2-head (push) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (push) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (push) Skipped
QRun-IO/ci-policy: ci / source-config-14 (push) Skipped
QRun-IO/ci-policy: ci / original-voyage (push) Skipped
QRun-IO/ci-policy: ci / original-cli (push) Skipped
QRun-IO/ci-policy: ci / ci-result (push) Successful in 7s
build / maven (push) Successful in 3m3s
QRun-IO/ci-policy: ci / fork-guard (push) Skipped
QRun-IO/ci-policy: native / fork-deny (push) Skipped
QRun-IO/ci-policy: ci / head-profile (push) Skipped
QRun-IO/ci-policy: native / native-result (push) Successful in 7s
QRun-IO/ci-policy: ci / repository-policy (push) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-7-head (push) Skipped
QRun-IO/ci-policy: ci / node-8 (push) Skipped
QRun-IO/ci-policy: ci / node-9 (push) Skipped
QRun-IO/ci-policy: ci / node-10 (push) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (push) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (push) Skipped
QRun-IO/ci-policy: ci / source-config-13 (push) Skipped
QRun-IO/ci-policy: ci / original-business (push) Skipped
QRun-IO/ci-policy: ci / Scoped workflow [workflow d20a578c7d818bc7ca32a2ac2c675da4e4f27385] (push) Successful in 8s
QRun-IO/ci-policy: ci / application-template-0 (push) Skipped
QRun-IO/ci-policy: ci / gradle-1 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-6 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (push) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (push) Skipped
QRun-IO/ci-policy: ci / android-0 (push) Skipped
QRun-IO/ci-policy: ci / not-enrolled (push) Skipped
QRun-IO/ci-policy: ci / shared-pipeline (push) Skipped
maven / build (push) Successful in 3m3s
QRun-IO/ci-policy: ci / java-docker-3-head (push) Skipped
QRun-IO/ci-policy: ci / original-mcp (push) Skipped
2026-11-05 23:15:25 +00:00
james.maesandClaude Opus 5.5 a0f9a172d0 ci: use shared ci-workflows Maven workflow
QRun-IO/ci-policy: ci / fork-guard (pull_request) Skipped
QRun-IO/ci-policy: ci / Scoped workflow [workflow 1b3648d9421543350067e95e54959dfb471b26d5] (pull_request) Successful in 7s
QRun-IO/ci-policy: ci / head-profile (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy (pull_request) Skipped
QRun-IO/ci-policy: ci / application-template-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-8 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-business (pull_request) Skipped
QRun-IO/ci-policy: ci / original-voyage (pull_request) Skipped
QRun-IO/ci-policy: ci / original-cli (pull_request) Skipped
build / maven (push) Successful in 3m9s
QRun-IO/ci-policy: Shared Java Maven CI / verify (pull_request) Successful in 3m25s
QRun-IO/ci-policy: Shared Java Maven CI / secrets (pull_request) Failing after 1s
QRun-IO/ci-policy: Shared Java Maven CI / gate (pull_request) Failing after 13s
QRun-IO/ci-policy: ci / java-maven-6 (pull_request) Failing after 17m15s
QRun-IO/ci-policy: native / fork-deny (pull_request) Skipped
QRun-IO/ci-policy: native / dotnet-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-core-2-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7-head (pull_request) Skipped
QRun-IO/ci-policy: native / native-result (pull_request) Successful in 6s
QRun-IO/ci-policy: ci / android-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-mcp (pull_request) Skipped
QRun-IO/ci-policy: ci / not-enrolled (pull_request) Skipped
QRun-IO/ci-policy: ci / ci-result (pull_request) Failing after 8s
QRun-IO/ci-policy: ci / gradle-1 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-10 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-13 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-14 (pull_request) Skipped
QRun-IO/ci-policy: ci / shared-pipeline (pull_request) Skipped
maven / build (push) Successful in 3m8s
QRun-IO/ci-policy: ci / node-9 (pull_request) Skipped
Replace the platform-ci release callers with one caller of
KofTwentyTwo/ci-workflows maven.yaml (qqq-orb port).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 22:03:19 +00:00
forge-recovery 7d61f29e92 ci: enroll shared Gitea lifecycle
qrun-release / postpublish (push) Failing after 10m38s
QRun-IO/ci-policy: ci / ci-result (push) Successful in 7s
QRun-IO/ci-policy: ci / java-docker-3 (push) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (push) Skipped
QRun-IO/ci-policy: ci / source-config-14 (push) Skipped
QRun-IO/ci-policy: native / dotnet-0 (push) Skipped
Shared Java Maven CI / secrets (push) Successful in 35s
QRun-IO/ci-policy: ci / java-core-2-head (push) Skipped
Shared Java Maven CI / verify (push) Successful in 2m52s
QRun-IO/ci-policy: ci / fork-guard (push) Skipped
QRun-IO/ci-policy: ci / head-profile (push) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (push) Skipped
QRun-IO/ci-policy: ci / node-8 (push) Skipped
QRun-IO/ci-policy: ci / node-9 (push) Skipped
QRun-IO/ci-policy: ci / node-10 (push) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (push) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (push) Skipped
QRun-IO/ci-policy: ci / source-config-13 (push) Skipped
QRun-IO/ci-policy: native / fork-deny (push) Skipped
QRun-IO/ci-policy: native / native-result (push) Successful in 10s
QRun-IO/ci-policy: Shared Java Maven CI / secrets (push) Successful in 34s
QRun-IO/ci-policy: ci / repository-policy (push) Skipped
QRun-IO/ci-policy: ci / java-maven-7-head (push) Skipped
QRun-IO/ci-policy: Shared Java Maven CI / verify (push) Successful in 2m9s
Shared QRun source version / version (push) Failing after 10m38s
QRun-IO/ci-policy: ci / application-template-0 (push) Skipped
QRun-IO/ci-policy: ci / gradle-1 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (push) Skipped
QRun-IO/ci-policy: ci / android-0 (push) Skipped
QRun-IO/ci-policy: ci / not-enrolled (push) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (push) Skipped
Shared Java Maven CI / gate (push) Successful in 18s
QRun-IO/ci-policy: Shared Java Maven CI / gate (push) Successful in 17s
qrun-release / release (push) Successful in 24m44s
QRun-IO/ci-policy: ci / java-maven-6 (push) Successful in 24m42s
QRun-IO/ci-policy: ci / original-mcp (push) Skipped
QRun-IO/ci-policy: ci / java-docker-3-head (push) Skipped
2026-11-02 22:12:48 +00:00
forge-recovery 6e60585af1 ci: enroll shared Gitea lifecycle
QRun-IO/ci-policy: ci / java-docker-3 (push) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (push) Skipped
QRun-IO/ci-policy: ci / source-config-14 (push) Skipped
QRun-IO/ci-policy: native / dotnet-0 (push) Skipped
Shared Java Maven CI / secrets (push) Successful in 41s
QRun-IO/ci-policy: ci / java-core-2-head (push) Skipped
Shared Java Maven CI / verify (push) Successful in 6m4s
QRun-IO/ci-policy: ci / ci-result (push) Successful in 6s
QRun-IO/ci-policy: ci / fork-guard (push) Skipped
QRun-IO/ci-policy: ci / head-profile (push) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (push) Skipped
QRun-IO/ci-policy: ci / node-8 (push) Skipped
QRun-IO/ci-policy: ci / node-9 (push) Skipped
QRun-IO/ci-policy: ci / node-10 (push) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (push) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (push) Skipped
QRun-IO/ci-policy: ci / source-config-13 (push) Skipped
QRun-IO/ci-policy: native / fork-deny (push) Skipped
QRun-IO/ci-policy: Shared Java Maven CI / secrets (push) Successful in 45s
QRun-IO/ci-policy: ci / repository-policy (push) Skipped
QRun-IO/ci-policy: ci / java-maven-7-head (push) Skipped
QRun-IO/ci-policy: native / native-result (push) Successful in 13s
QRun-IO/ci-policy: Shared Java Maven CI / verify (push) Successful in 3m1s
QRun-IO/ci-policy: ci / application-template-0 (push) Skipped
QRun-IO/ci-policy: ci / gradle-1 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (push) Skipped
QRun-IO/ci-policy: ci / android-0 (push) Skipped
QRun-IO/ci-policy: ci / not-enrolled (push) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (push) Skipped
Shared Java Maven CI / gate (push) Successful in 19s
qrun-release / release (push) Successful in 9m16s
QRun-IO/ci-policy: Shared Java Maven CI / gate (push) Successful in 19s
QRun-IO/ci-policy: ci / java-maven-6 (push) Successful in 9m33s
QRun-IO/ci-policy: ci / original-mcp (push) Skipped
QRun-IO/ci-policy: ci / java-docker-3-head (push) Skipped
2026-11-02 21:31:05 +00:00
forge-recoveryandCodex 594e755da4 ci: enroll shared Gitea release-candidate lifecycle
QRun-IO/ci-policy: ci / fork-guard (pull_request) Skipped
QRun-IO/ci-policy: ci / head-profile (pull_request) Skipped
QRun-IO/ci-policy: ci / application-template-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-8 (pull_request) Skipped
QRun-IO/ci-policy: Shared Java Maven CI / secrets (pull_request) Successful in 14s
QRun-IO/ci-policy: ci / repository-policy (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (pull_request) Skipped
QRun-IO/ci-policy: Shared Java Maven CI / verify (pull_request) Successful in 2m9s
QRun-IO/ci-policy: Shared Java Maven CI / gate (pull_request) Successful in 13s
QRun-IO/ci-policy: ci / java-maven-6 (pull_request) Successful in 57m16s
Release candidate / release-rc (pull_request) Skipped
QRun-IO/ci-policy: ci / android-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / not-enrolled (pull_request) Skipped
QRun-IO/ci-policy: native / fork-deny (pull_request) Skipped
QRun-IO/ci-policy: native / dotnet-0 (pull_request) Skipped
QRun-IO/ci-policy: native / native-result (pull_request) Successful in 6s
QRun-IO/ci-policy: ci / java-core-2-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7-head (pull_request) Skipped
QRun-IO/ci-policy: ci / ci-result (pull_request) Successful in 6s
QRun-IO/ci-policy: ci / gradle-1 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-10 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-13 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-14 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3-head (pull_request) Skipped
QRun-IO/ci-policy: ci / node-9 (pull_request) Skipped
Co-Authored-By: Codex <noreply@openai.com>
2026-11-02 20:14:28 +00:00
4679821cb4 ci: add the isolated postpublish job to the QRun release caller
QRun-IO/ci-policy: ci / fork-guard (pull_request) Skipped
QRun-IO/ci-policy: ci / head-profile (pull_request) Skipped
QRun-IO/ci-policy: ci / application-template-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-8 (pull_request) Skipped
QRun-IO/ci-policy: Shared Java Maven CI / secrets (pull_request) Successful in 17s
QRun-IO/ci-policy: ci / repository-policy (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (pull_request) Skipped
Shared Java Maven CI / secrets (push) Successful in 28s
Shared Java Maven CI / verify (push) Successful in 2m10s
qrun-release / postpublish (push) Skipped
QRun-IO/ci-policy: Shared Java Maven CI / verify (pull_request) Successful in 2m19s
QRun-IO/ci-policy: Shared Java Maven CI / gate (pull_request) Successful in 13s
QRun-IO/ci-policy: ci / java-maven-6 (pull_request) Successful in 2h42m10s
Release candidate / release-rc (pull_request) Skipped
QRun-IO/ci-policy: ci / not-enrolled (pull_request) Skipped
QRun-IO/ci-policy: native / fork-deny (pull_request) Skipped
QRun-IO/ci-policy: native / dotnet-0 (pull_request) Skipped
QRun-IO/ci-policy: native / native-result (pull_request) Successful in 5s
QRun-IO/ci-policy: ci / java-core-2-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7-head (pull_request) Skipped
QRun-IO/ci-policy: ci / ci-result (pull_request) Successful in 5s
QRun-IO/ci-policy: ci / gradle-1 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-10 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-13 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-14 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3-head (pull_request) Skipped
Shared Java Maven CI / gate (push) Successful in 10s
qrun-release / release (push) Successful in 2h42m0s
QRun-IO/ci-policy: ci / node-9 (pull_request) Skipped
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Codex <noreply@openai.com>
2026-11-02 15:47:51 +00:00
forge-recovery f8337989e8 ci: add the QRun release caller (#53)
QRun-IO/ci-policy: ci / ci-result (push) Failing after 1s
QRun-IO/ci-policy: ci / java-docker-2 (push) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-3 (push) Skipped
QRun-IO/ci-policy: ci / node-playwright-7 (push) Skipped
Shared Java Maven CI / verify (push) Failing after 1s
Shared Java Maven CI / secrets (push) Failing after 0s
QRun-IO/ci-policy: ci / java-maven-4 (push) Failing after 58s
QRun-IO/ci-policy: ci / fork-guard (push) Skipped
QRun-IO/ci-policy: ci / node-6 (push) Skipped
QRun-IO/ci-policy: ci / source-config-8 (push) Skipped
QRun-IO/ci-policy: Shared Java Maven CI / verify (push) Failing after 1s
QRun-IO/ci-policy: Shared Java Maven CI / secrets (push) Failing after 1s
QRun-IO/ci-policy: ci / application-template-0 (push) Skipped
QRun-IO/ci-policy: ci / gradle-1 (push) Skipped
QRun-IO/ci-policy: ci / node-5 (push) Skipped
QRun-IO/ci-policy: ci / not-enrolled (push) Skipped
Shared Java Maven CI / gate (push) Failing after 0s
qrun-release / release (push) Failing after 59s
QRun-IO/ci-policy: Shared Java Maven CI / gate (push) Failing after 0s
2026-11-02 04:17:58 +00:00
james.maesandClaude Fable 5.1 6cf98bf1fd Merge feat/per-session-presence: per-connection viewers and counts (extension 1.2.0)
QRun-IO/ci-policy: ci / ci-result (push) Successful in 3s
QRun-IO/ci-policy: ci / java-docker-1 (push) Skipped
QRun-IO/ci-policy: ci / node-3 (push) Skipped
QRun-IO/ci-policy: Shared Java Maven CI / verify (push) Successful in 3m40s
QRun-IO/ci-policy: ci / fork-guard (push) Skipped
QRun-IO/ci-policy: ci / gradle-0 (push) Skipped
QRun-IO/ci-policy: Shared Java Maven CI / secrets (push) Successful in 28s
QRun-IO/ci-policy: Shared Java Maven CI / gate (push) Successful in 15s
QRun-IO/ci-policy: ci / java-maven-2 (push) Successful in 20m24s
QRun-IO/ci-policy: ci / node-4 (push) Skipped
QRun-IO/ci-policy: ci / source-config-5 (push) Skipped
QRun-IO/ci-policy: ci / not-enrolled (push) Skipped
Fixes #54. Each merged viewer carries connections; GET /presence?perConnection=true adds one lease per connection with opaque ids; presence.changed carries leaseCount and leases.

Security: the raw session uuid is never stored, replicated or returned. The heartbeat uses an HMAC-SHA256 derived sessionId keyed by QQQ_PRESENCE_SESSION_SECRET (random per-process secret when unset, so set it on every pod). The public leases() accessor and Lease record are removed. 162 tests green, including a test that the uuid is absent from every bus message and snapshot.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 17:27:25 -06:00
james.maesandClaude Fable 5.1 0ae1e62b7b fix(presence): never store or publish the session uuid
The session uuid is a bearer credential (clients resume sessions with
it as a cookie or bearer token). The first cut kept it on the lease,
replicated it on the internal presence bus and exposed it through a
public leases() accessor.

Derive an opaque session id instead: "s" + the first 16 hex digits of
HMAC-SHA256(secret, sessionUuid), with QQQ_PRESENCE_SESSION_SECRET
(shared across pods) or a random per-process secret. The raw uuid is
dropped at heartbeat; only the opaque id is stored, replicated and
returned (per-connection viewers and presence.changed leases). The
leases() accessor and Lease record are removed. Tests assert the raw
uuid never appears in bus messages, snapshots or HTTP responses.

Refs #54

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 16:35:11 -06:00
james.maesandClaude Fable 5.1 47cbc7c72d feat(presence): per-connection viewers and connection counts
Two terminals or tabs of one user collapsed into a single viewer, so
clients could not tell "Admin, twice" from "Admin". The registry
already kept one lease per connection; expose it additively:

- merged viewers carry connections (live leases behind the viewer)
- snapshots and presence.changed carry leaseCount
- GET /presence?perConnection=true adds leases, one viewer per lease
  with an opaque connection id, mode, field and lease start;
  presence.changed always carries leases (subscribers cannot opt in)
- leases record the server-side session uuid, never sent to clients
- counts() still counts users; connectionCounts() counts leases and
  the Who's here card shows both

Extension collaboration 1.2.0; regenerated committed document.

Fixes #54

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 16:31:37 -06:00
james.maesandClaude Fable 5.1 1ae6508793 fix(presence): always send viewers, empty after the last leave (#53)
QRun-IO/ci-policy: ci / node-3 (push) Skipped
QRun-IO/ci-policy: ci / java-docker-1 (push) Skipped
QRun-IO/ci-policy: ci / ci-result (push) Successful in 5s
QRun-IO/ci-policy: ci / fork-guard (push) Skipped
QRun-IO/ci-policy: ci / gradle-0 (push) Skipped
QBit CodeQL / Analyze branch source (push) Canceled after 0s
CodeQL / analyze (push) Canceled after 0s
QRun-IO/ci-policy: Shared Java Maven CI / verify (push) Successful in 2m51s
QRun-IO/ci-policy: Shared Java Maven CI / secrets (push) Successful in 16s
QRun-IO/ci-policy: ci / node-4 (push) Skipped
QRun-IO/ci-policy: ci / source-config-5 (push) Skipped
QRun-IO/ci-policy: ci / not-enrolled (push) Skipped
QRun-IO/ci-policy: Shared Java Maven CI / gate (push) Successful in 8s
QRun-IO/ci-policy: ci / java-maven-2 (push) Successful in 26m47s
The presence payload omitted viewers once the last viewer left, so clients kept showing the previous list. Viewers are now always serialized, as an empty list after the last leave.

Fixes #52

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 04:25:29 -06:00
james.maesandCodex 03407ae79a ci: adopt managed orb 0.8 patch track (#43)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 15:26:43 -05:00
james.maesandCodex 9d6767e731 ci: adopt shared orb 0.7.18 (#42)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 14:43:27 -05:00
james.maesandCodex aa6a4a3b58 ci: adopt shared orb 0.7.17 (#41)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 14:15:33 -05:00
james.maesandCodex d9a46afbf9 ci: use the shared branch-aware CodeQL worker (#39)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 10:35:35 -05:00
james.maesandCodex b5308a8bab ci: adopt central orb 0.7.12 and stable gates (#37)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 08:50:12 -05:00
james.maesandCodex 0caaa3717e ci: centralize rc release completion in qqq orb (#36)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 02:23:46 -05:00
james.maesandCodex 464db76bc3 ci: adopt shared orb 0.7.11 (#35)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 00:54:30 -05:00
james.maes 19a06166f5 Merge pull request #34 from QRun-IO/feature/central-ci
ci: adopt shared QQQ orb 0.7.4
2026-10-20 19:32:48 -05:00
james.maesandCodex 61060e2793 ci: adopt central QQQ orb 0.7.4
Co-Authored-By: Codex <noreply@openai.com>
2026-10-20 19:26:47 -05:00
james.maes 4e88ed8303 Merge pull request #31 from QRun-IO/ci/release-rc-secrets-inherit
ci(release-rc): pass secrets so the shared workflow can sign the RC tag
2026-10-16 08:03:57 -05:00
james.maesandClaude Opus 5.5 7a7b804106 ci(release-rc): pass secrets to the shared workflow so it can sign the RC tag
QRun-IO/.github#2 adds a tag-and-release job that signs vX with the
QRUN_TAG_SIGNING_KEY secret after Maven Central confirms the artifact.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 16:19:25 -05:00
james.maes 4c95151a86 Merge pull request #29 from QRun-IO/feature/ui-api-1.6
feat(ui-api): extension for QQQ UI API 1.6.0 (operationIds, BasicErrorResponseV1, QQQ 4.1.0-RC.15)
2026-10-15 16:12:53 -05:00
james.maesandClaude Opus 5.5 41bbf2c428 chore: target QQQ 4.1.0-RC.15 (UI API 1.6.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 12:27:04 -05:00
james.maesandClaude Opus 5.5 2f6be765f2 feat(ui-api): extension for QQQ UI API 1.6.0 (operationIds, BasicErrorResponseV1, QQQ 4.1.0-RC.14)
Regenerated against UI API 1.6.0: every operation has an operationId and
the standard error responses (BasicErrorResponseV1: error, code, message,
fieldErrors, traceId); the extension's own error schema is kept but
deprecated; hand-written error bodies go through QJavalinUtils so they carry
code/message/traceId. Extension MINOR bumped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 11:43:37 -05:00
james.maes 92dd901409 Merge pull request #28 from QRun-IO/fix/producer-version-from-build
fix: qbit metadata version comes from the build
2026-10-15 01:42:56 -05:00
james.maesandClaude Opus 5.5 6a4ea584f5 fix: the qbit metadata version comes from the build, not a hardcoded constant
The producer read a literal version, so released artifacts reported a stale
or -SNAPSHOT version. qbit-collaboration.properties (version=${project.version}) is
Maven-filtered and the producer loads it; tests assert producer == bundle ==
artifact version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 01:27:45 -05:00
james.maes 03b3bbaca2 Merge pull request #27 from QRun-IO/fix/help-bundle-version-test
test(help): bundle version test compares with the artifact version
2026-10-14 23:18:42 -05:00
james.maesandClaude Opus 5.5 31ce528064 test(help): compare the help bundle version with the artifact version, not a hardcoded SNAPSHOT
The bundle version is now Maven-filtered, so a literal SNAPSHOT in the test
fails every release build. Surefire passes the artifact version in
qbit.artifactVersion.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 23:07:29 -05:00
james.maes 90b3956cf7 Merge pull request #26 from QRun-IO/fix/help-bundle-version
fix(help): help bundle version comes from the build, not a hardcoded SNAPSHOT
2026-10-14 22:37:20 -05:00
james.maesandClaude Opus 5.5 e4f92192c6 fix(help): the help bundle carries the artifact version (Maven-filtered) instead of a hardcoded SNAPSHOT
On release branches the bundle's version and the producer's differed, so the
bundle test that asserts they match failed, and released bundles recorded
SNAPSHOT as their provenance. help/*.yaml is now filtered:
version: ${project.version}.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 22:04:44 -05:00
james.maes 675d30568e Merge pull request #25 from QRun-IO/feature/help-bundle
feat(help): qbit-collaboration help bundle via QQQ help bundles, with coverage gate
2026-10-14 20:45:30 -05:00
james.maesandClaude Opus 5.5 3df535453e feat(help): ship the qbit-collaboration help bundle through QQQ help bundles, with coverage gate
Host-neutral help (deployment-specific facts removed), registered with
HelpBundles, gated by HelpBundleAssertions. QQQ 4.1.0-RC.7.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 20:27:03 -05:00
james.maes d302f1dc1f Merge pull request #24 from QRun-IO/feature/tables-populated-defaults
feat: Create rows from configuration, Setup card and table help
2026-10-14 17:44:57 -05:00
james.maesandClaude Opus 5.5 58c87b0378 fix: load feature settings in one transaction, correct scope wording, count only eligible tables
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:39:52 -05:00
james.maesandClaude Opus 5.5 e5647e6020 chore: well-formed STAND-IN marker for the help applier
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:30:31 -05:00
james.maesandClaude Opus 5.5 3d2a1ded72 feat: Create rows from configuration, Setup card and table help so Feature Settings is never a mystery
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:08:52 -05:00
james.maes 662a9f1a6d Merge pull request #23 from QRun-IO/feature/qqq-rc7
feat: QQQ RC.7 followers provider and secured-data label rules
2026-10-14 16:33:33 -05:00
james.maesandClaude Opus 5.5 e014c00428 feat: QQQ 4.1.0-RC.7: record followers provider and secured-data label rules
CollaborationFollowersProvider implements core RecordFollowersProviderInterface
over collaboration_follow (CHANGE/ALERT -> notifyOnChange, COMMENT ->
notifyOnComment, muted excluded), registered via RecordFeaturesMetaData, so
qbit-alerts can notify followers. Cached favorite/recent labels are stored
only when FieldAccessHelper.isRecordLabelFullAccess holds; notification
titles fall back to the table label and id for recipients without FULL
label access. Timeline masking (MASKED shows the mask, HIDDEN omitted,
canary never leaks) is covered by tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 16:29:09 -05:00
james.maes ea0b74f94c Merge pull request #22 from QRun-IO/fix/issues-20-21
fix: context-free attachment steps (#20) and no fixed home column widths (#21)
2026-10-14 14:59:18 -05:00
james.maesandClaude Opus 5.5 e051c74273 fix: context-free attachment action steps and no fixed home column widths
DeleteAttachmentsProcess Review/Delete create their AttachmentsService on
first use instead of in the constructor, so QQQ can build them during
context-free instance validation (fixes #20). Home table columns no
longer set widths, so narrow cards wrap instead of scrolling (fixes #21).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 14:56:09 -05:00
james.maes 680fd5fa02 Merge pull request #18 from QRun-IO/feature/orb-0.6.9
ci: qqq-orb 0.6.9 (refresh SNAPSHOT dependencies)
2026-10-14 10:56:55 -05:00
james.maes 1e0a5773bb Merge pull request #19 from QRun-IO/feature/email-api
refactor(email): use qbit-email's public api package; no G1 allow-list
2026-10-14 10:25:07 -05:00
james.maesandClaude Opus 5.5 d5adcec567 refactor(email): use qbit-email's public api package; no G1 allow-list
Moves the optional email integration from qbit-email internals to
com.kingsrook.qbits.email.api (qbit-email 0.1.0-RC.2) with identical
behaviour, and moves to qbit-build-parent 2.1.1, whose default G1
allow-lists include qbit-email's API (QRun-IO/qbit-bom#12), so the
per-repo allow-lists are gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 10:20:32 -05:00
james.maesandClaude Opus 5.5 83b253a94f ci: qqq-orb 0.6.9 (refresh SNAPSHOT dependencies)
Moves qqq-orb@0.6.8 to qqq-orb@0.6.9, which passes --update-snapshots so cached SNAPSHOT dependencies are re-checked on every build (QRun-IO/qqq-orb#22).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 10:06:36 -05:00
james.maes 4eb5b70ade Merge pull request #17 from QRun-IO/feature/object-storage-rc4
feat(attachments): bucket upload cap via object storage api (RC.4)
2026-10-14 09:58:45 -05:00
james.maesandClaude Opus 5.5 da932fb309 feat(attachments): honor the bucket upload cap via object storage's api (qbit-object-storage 0.1.0-RC.4)
AttachmentStorage.maxBytes now reads StorageBucket.maxUploadBytes() from
the api package (null = no bucket limit); uploads stop at the smaller of
the bucket cap and the per-table cap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 09:55:21 -05:00
james.maes 66189ec110 Merge pull request #15 from QRun-IO/feature/email-qbit
feat(email): email mentions, shares and task notices through qbit-email
2026-10-14 08:27:25 -05:00
james.maesandClaude Opus 5.5 23d3365437 Merge develop into feature/email-qbit
Keeps develop's changes and re-applies the optional qbit-email 0.1.0-RC.1
delivery on top (adapted to develop's refactors; guardrail allow-lists
for the optional integration where the build parent enforces them).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 08:19:03 -05:00
james.maesandClaude Opus 5.5 f0b8fc9141 build: qbit-email 0.1.0-RC.1 from Maven Central
Moves from 0.1.0-SNAPSHOT to the first release candidate. Test schemas
gain the RC's email_provider_sender_address table and the message
from_address_id column where they copy the qbit's DDL.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 07:58:55 -05:00
james.maes 6cd78a803e Merge pull request #16 from QRun-IO/fix/live-review
fix: live-review fixes (tag id filter, home widgets, display names, demo multi-user)
2026-10-14 07:09:47 -05:00
james.maes b4032baf70 Merge pull request #14 from QRun-IO/feature/demo-dev-server
test: collaboration demo dev server with seeded data
2026-10-14 07:09:35 -05:00
james.maes d2ad22eb97 Merge pull request #13 from QRun-IO/feature/presence-custom-fields
feat: custom field designer and presence
2026-10-14 07:09:21 -05:00
james.maes 66b5bf2bea Merge pull request #8 from QRun-IO/feature/ui-api-endpoints
feat: adopt core collaboration contracts and the UI API extension endpoints
2026-10-14 07:09:09 -05:00
james.maesandClaude Opus 5.5 d828d1677d feat: friendly audit summaries on the activity timeline
Core's "Record was Edited/Inserted/Deleted" audit messages become "Edited by
Alice Morgan: Title, Total", "Created by Alice Morgan" and "Deleted by ...",
using personalized field labels (first four, then "and N more"). Detail rows
and unrecognized messages are untouched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 23:28:43 -05:00
james.maesandClaude Opus 5.5 ef2ee45b5a feat: demo server picks the user per browser session and sends the real name
A ?user= request parameter (remembered in the collab_dev_user cookie) chooses the
demo identity for that browser, falling back to -Dcollab.dev.user. The session
now carries values.user (name and email) so the frontend shows Alice Morgan
instead of Anonymous. README documents opening a second browser to see presence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 23:27:08 -05:00
james.maesandClaude Opus 5.5 03669391d0 fix: exclude the caller from recipient and mention search; demo links are absolute
People search (share recipients and mention candidates) never returns the
current user. The record link already honors frontendBaseUrl; the demo server
now sets it to http://localhost:<port> and the test asserts the absolute URL.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 23:23:44 -05:00
james.maesandClaude Opus 5.5 7e63314413 feat: display names beside user ids in API payloads and admin tables
Tasks, attachments, comments' revisions, followers, shares, access requests and
assignments gain additive *DisplayName properties, resolved with one directory
query per response (unknown ids fall back to the id). User-id fields on the
collaboration tables use the collaboration_user possible value source, keyed on
userId, so admin screens show names. Contract regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 23:22:00 -05:00
james.maesandClaude Opus 5.5 68fe96c22f feat: rebuild the Collaboration home from standard widgets
Replace the HTML wall of text with a KPI block (comments this week, open
threads, open and overdue tasks, open access requests, attachments with a
human-readable size, active custom fields) and full-width Tags, Custom fields
and Who's here tables, each with a real empty state and no hard-coded colors.
Counts still use the caller's session. Tests updated to assert widget data.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 23:14:27 -05:00
james.maesandClaude Opus 5.5 c9954efef4 feat(email): email mentions, shares and task notices through qbit-email
Phase (d) of QRun-IO/qbit-email docs/DESIGN.md section 11. qbit-email is an
optional dependency (0.1.0-SNAPSHOT until its first RC) used through one
adapter class behind a classpath and instance guard; without it the
existing behaviour is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 23:12:39 -05:00
james.maesandClaude Opus 5.5 e696d9edaf fix: demo Admin parent app is reachable so Collaboration lands in the app tree
The producer already nests the app under its parent, or leaves it top-level.
QQQ builds the appTree from apps whose parents survive permission checks, so a
denied parent silently orphaned Collaboration (present in apps, absent from the
tree). The demo granted admin.access while the Admin app requires
admin.hasAccess. Tests assert the tree for both placements and for the demo.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 23:07:19 -05:00
james.maesandClaude Opus 5.5 42cf51325a fix: tag filter accepts ids as any JSON number or numeric string
TagCriteriaExpander treated non-Number values as names, so ids arriving
as strings from v1 JSON matched nothing. Numbers and numeric strings that
match an existing tag are ids; everything else is a normalized name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 23:04:54 -05:00
james.maes 95506ed706 Merge branch 'feature/presence-custom-fields' into feature/demo-dev-server 2026-10-13 22:19:51 -05:00
james.maesandClaude Opus 5.5 7b7c921079 test: collaboration demo dev server with seeded Sales data, plus a SELECT custom-field refresh fix
CollaborationDevServer serves the Next dashboard over an in-memory
instance (people, orders, every record feature on, seeded comments,
files, tags, custom fields, tasks, shares, audit history) with mock users
alice, bob and admin. CustomFieldsProvider now registers generated SELECT
possible value sources on refresh.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 22:04:03 -05:00
207 changed files with 29593 additions and 780 deletions

No files matched your search

+11 -57
View File
@@ -1,60 +1,14 @@
version: 2.1
setup: true
orbs:
qqq-orb: kingsrook/qqq-orb@0.6.8
commands:
configure-package-install:
steps:
- run:
name: Keep package installation unattended
command: |
# These build machines are discarded after the job.
sudo install -d /etc/needrestart/conf.d
printf '%s\n' '$nrconf{restart} = "l";' | sudo tee /etc/needrestart/conf.d/99-qqq-ci.conf >/dev/null
qqq-orb: kingsrook/qqq-orb@0.8
workflows:
test_only:
jobs:
- qqq-orb/mvn_test_only:
pre-steps:
- configure-package-install
context: [qqq-maven-registry-credentials]
filters:
branches:
ignore: /(develop|release\/.*)/
publish_snapshot:
jobs:
- qqq-orb/mvn_publish:
pre-steps:
- configure-package-install
context: [qqq-maven-registry-credentials]
branch_type: snapshot
filters:
branches:
only: [develop]
publish_release_candidate:
jobs:
- qqq-orb/mvn_publish:
pre-steps:
- configure-package-install
context: [qqq-maven-registry-credentials]
branch_type: release_candidate
filters:
branches:
only: [/release\/.*/]
publish_release:
jobs:
- qqq-orb/mvn_publish:
pre-steps:
- configure-package-install
context: [qqq-maven-registry-credentials]
branch_type: release
filters:
branches:
ignore: /.*/
tags:
only: [/^v\d+\.\d+\.\d+$/]
setup:
jobs:
- qqq-orb/generate_pipeline:
orb_version: '0.8'
filters:
branches:
ignore: automation/qbit-rc-pending
tags:
only: /^v\d+\.\d+\.\d+$/
+7
View File
@@ -0,0 +1,7 @@
name: build
on:
push:
jobs:
maven:
uses: self:KofTwentyTwo/ci-workflows/.gitea/workflows/maven.yaml@v1
secrets: inherit
+23
View File
@@ -0,0 +1,23 @@
name: CodeQL
'on':
push:
branches:
- main
- develop
- release/**
- hotfix/**
pull_request: {}
schedule:
- cron: 41 16 * * 0
workflow_dispatch: {}
permissions:
contents: read
actions: read
security-events: write
packages: read
concurrency:
group: codeql-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs:
analyze:
uses: QRun-IO/qqq-orb/.github/workflows/qbit-codeql.yml@c288a8864fe9054f68410e1d78f5863aea4c4d22
+62 -13
View File
@@ -1,25 +1,74 @@
# Cut the next release candidate (QRun-IO/qqq docs/QRUN-QBIT-REPO-STANDARD.md, C6):
# run this workflow by hand, or merge a develop PR labelled `release:rc`. It merges develop into
# the release line and the release/* push runs CircleCI publish_release_candidate.
name: Release candidate
on:
'on':
workflow_dispatch:
inputs:
dry_run:
description: Print the plan without merging or creating branches
description: Plan a cut without changing release state
type: boolean
default: false
finalize:
description: Retry finalization of an existing cut; do not cut another RC
type: boolean
default: false
recover:
description: Recover the same published RC after a deployment timeout; never
reupload
type: boolean
default: false
circle_build:
description: Original Circle publisher build number (recovery only)
type: string
default: ''
merge_sha:
description: Existing cut merge_sha (finalize retry only)
type: string
default: ''
bump_sha:
description: Existing cut bump_sha (finalize retry only)
type: string
default: ''
branch:
description: Existing cut branch (finalize retry only)
type: string
default: ''
version:
description: Existing cut version (finalize retry only)
type: string
default: ''
pull_request:
types: [closed]
branches: [develop]
types:
- closed
branches:
- develop
repository_dispatch:
types:
- qbit-rc-published
permissions:
contents: write
actions: read
checks: read
jobs:
release-rc:
if: github.event_name == 'workflow_dispatch' || (github.event.pull_request.merged == true && contains(github.event.pull_request.labels.*.name, 'release:rc'))
uses: QRun-IO/.github/.github/workflows/qbit-release-rc.yml@main
if: github.event_name == 'workflow_dispatch' || github.event_name == 'repository_dispatch'
|| (github.event_name == 'pull_request' && github.event.pull_request.merged
== true && github.event.pull_request.base.ref == 'develop' && contains(github.event.pull_request.labels.*.name,
'release:rc'))
uses: QRun-IO/qqq-orb/.github/workflows/qbit-release-rc.yml@cc30f599e63c8e41629c99d23395bac1adb8bc44
with:
dry_run: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run }}
dry_run: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run || false
}}
finalize: ${{ github.event_name == 'repository_dispatch' || (github.event_name
== 'workflow_dispatch' && inputs.finalize) }}
recover: ${{ github.event_name == 'workflow_dispatch' && inputs.recover || false
}}
circle_build: ${{ github.event_name == 'workflow_dispatch' && inputs.circle_build
|| '' }}
merge_sha: ${{ github.event_name == 'repository_dispatch' && github.event.client_payload.merge_sha
|| inputs.merge_sha || '' }}
bump_sha: ${{ github.event_name == 'repository_dispatch' && github.event.client_payload.bump_sha
|| inputs.bump_sha || '' }}
branch: ${{ github.event_name == 'repository_dispatch' && github.event.client_payload.branch
|| inputs.branch || '' }}
version: ${{ github.event_name == 'repository_dispatch' && github.event.client_payload.version
|| inputs.version || '' }}
secrets: inherit
+16
View File
@@ -0,0 +1,16 @@
profile: qbit
orb_version: '0.8'
default_branch: develop
java_version: '21'
browsers: true
api_checks: false
docs_only: false
release_on_main: false
release_tag_policy: stable
test_branch_policy: non-release
publish:
snapshot: true
release_candidate: true
release: true
asciidoc: false
release_completion: true
+1 -1
View File
@@ -42,7 +42,7 @@ This file provides guidance to Codex when working with code in this repository.
## Dependencies
- Java 21.
- Parent `com.kingsrook:qbit-build-parent:2.0.0` (qqq 4.0.0). The default-active `qqq-snapshot` profile imports `qqq-bom-pom:4.1.0-RC.5` ahead of it, as in qbit-quick-search (ADR-0007). Remove that activation when the parent moves to 4.1.
- Parent `com.kingsrook:qbit-build-parent:2.1.1`. The default-active `qqq-snapshot` profile imports `qqq-bom-pom:4.1.0-SNAPSHOT`; build the matching Core integration branch first (see `docs/CORE-DEPENDENCY.md`). Public RC.21 lacks required rollback callbacks.
- Tests: JUnit 5, AssertJ.
## Release
+42 -1
View File
@@ -4,6 +4,47 @@ All notable changes are documented here, following [Keep a Changelog](https://ke
## [Unreleased]
### Added
- Attachment lifecycle SPI (`AttachmentLifecyclePolicy`, `CollaborationQBitConfig.withAttachmentLifecycle`) for retention, registration and mandatory access events; durable upload journal (`collaboration_upload_journal`, H2/MySQL/PostgreSQL), upload recovery and existing-attachment adoption processes, durable cleanup queue with operator settlement, verified/authenticated delivery (`AttachmentDownload`, `AttachmentContentIntegrityException`), media policy, aggregate guard and bounded attachment pagination. See `docs/ATTACHMENT-LIFECYCLE.md`, `docs/ATTACHMENT-RECOVERY.md`, `docs/RETAINED-UPLOAD-JOURNAL.md`, `docs/ATTACHMENT-PAGINATION.md`.
- Requires a core whose `QBackendTransaction` provides `supportsRollbackCallbacks()` and `addAfterRollbackCallback(Runnable)` (not in 4.1.0-RC.21); see `docs/CORE-DEPENDENCY.md`.
- Feature Settings table action **Create rows from configuration**: writes the code configuration out as editable `collaboration_feature_setting` rows (insert-only, only when the table is empty, no change in behavior).
- Setup card on the Collaboration home: each record feature, whether it is on, for which tables, and where that comes from.
- Host-neutral QBIT help bundle covering tables, fields, sections, built-in actions, the app, widgets and possible-value sources; core registration and `HelpBundleAssertions` coverage replace the temporary help loader.
- `CollaborationTablesGuardrailTest` and `FeatureSettingsDefaultsTest`.
### Changed
- UI API extension collaboration 1.2.0, additive (#54): presence viewers carry `connections` (live leases merged into the viewer), snapshots carry `leaseCount`, and `GET /presence?perConnection=true` adds `leases`, one viewer per connection with an opaque `connectionId`, `mode`, `fieldName`, `sessionId` and `since`. `presence.changed` events always carry `leaseCount` and `leases`. Leases keep an opaque HMAC-SHA256 session id (secret `QQQ_PRESENCE_SESSION_SECRET` or random per process), never the session uuid, which is a bearer credential. `PresenceRegistry.connectionCounts()` counts leases next to the unchanged user `counts()`; the Who's here card shows both.
- UI API extension collaboration 1.1.0 (requires QQQ 4.1.0-RC.15, UI API 1.6.0): operationIds, standard error responses with BasicErrorResponseV1, extension tags and published presence request/snapshot/viewer schemas to resolve dangling references. Retain CollaborationErrorV1 as deprecated.
- Upgrade the default QQQ BOM to `4.1.0-RC.7` from Maven Central.
- Register the core followers provider for alerts and other qbits, with event preferences, future-mute exclusion, system reads scoped to one record, and caller-owned recipient access checks.
- Cache favorite/recent record labels only for FULL field access. Resolve comment/share notification labels per recipient with a table-label-and-id fallback. Use the typed core label-access API for email labels as well.
- Verify RC.7 timeline audit masking and HIDDEN-field omission through enforced viewer queries, including canaries in values and messages.
- Switch the optional qbit-email integration to 0.1.0-RC.2 and its published `com.kingsrook.qbits.email.api` package, preserving templates, categories, idempotency keys, linked records and enqueue-status handling. Narrow the email package allowance to the public API.
- Email through qbit-email 0.1.0-RC.1: new optional dependency and `EmailCollaborationNotifier` (templates `collaboration_mention`, `collaboration_share`, `collaboration_task_reminder`), the default notifier when qbit-email is produced in the instance and no `notifier` is configured; `NoOpNotifier` otherwise. One NOTIFICATION message per recipient, no comment bodies, record labels only when field access allows. New `recipientEmailResolver` config hook; `CollaborationNotification` gains optional `sourceId` and `actorUserId`.
### Fixed
- Favorites and recents accept a table whose custom permission checker tells a read from a write by the type of the action input. `TargetResolver` asked whether a table is readable with an untyped `AbstractActionInput`; such a checker refuses an input it does not recognise, so `POST /recents` (which the UI sends on every list and record screen) and starring answered `404 Not found` for people the checker lets read the table. The read is now asked as the query it stands for, as `ParentRecordAccess` already asks with a get. Who may read a table is unchanged.
- Reads of a table that has no tag assignment no longer query `collaboration_record_tag`. Core selects the Tags field for every query or get that names no fields, so each of them (including per-record gets and the stored-record read before an update) ran one assignment statement whether or not the table had ever been tagged: 600 gets of an untagged table ran 600 statements, now none. Each node remembers which tables have no assignment (`TaggedTables`, core `Memoization` registered with `ClusterCaches`) for `CollaborationQBitConfig.withTagLookupCacheSeconds` (default 30, 0 disables); writing an assignment clears it on the node and through the ClusterBus. Tables with assignments are read as before, one lookup per 500 parents. No API or schema change. See "What the Tags field costs a read" in the README.
- Browsing one of the qbit's child tables (comments, tasks, record tags, attachments and the other parent-referencing tables) reads each distinct parent once per result page to authorize its rows, instead of once per row: 61 tasks on two parents ran 61 parent reads, now 2. The check itself and its outcome are unchanged.
- The retained upload journal's PostgreSQL 17 qualification accepts Amazon Aurora PostgreSQL, which reports `full_page_writes=off` and does not let a deployment change it. `off` is accepted only when the settings query detects Aurora (`to_regprocedure('aurora_version()') IS NOT NULL`); stock PostgreSQL with `full_page_writes=off` is still refused, and every other requirement is unchanged on Aurora. See `docs/RETAINED-UPLOAD-JOURNAL.md`.
- Bounded STRING custom fields carry QQQ's native `ValueTooLongBehavior.ERROR`, so a write longer than `maxLength` is rejected with a field error and the stored value is preserved, instead of passing through unchecked (the QQQ default `PASS_THROUGH`).
- Persisted help content is reloaded after each committed runtime metadata refresh, so sections and fields created by publishing a custom-field definition receive their stored `helpContent` rows without a restart.
- QBit metadata reports the Maven artifact version from a filtered resource, including release versions.
- `presence.changed` events and presence snapshots always carry `viewers`, sending `[]` once the last viewer leaves, as the schema requires (#52).
- Timeline audit summaries read "Edited by Alice Morgan: Title, Total" or "Created by Alice Morgan" instead of "Record was Edited"; detail rows are unchanged.
- The demo server chooses its user per browser session (`?user=bob`, remembered in a cookie) instead of once per JVM, and sends the person's name to the frontend so it no longer shows Anonymous.
- The share recipient search and mention candidates no longer offer the caller to themself.
- The demo server configures `frontendBaseUrl`, so record links are absolute URLs (the API already returned absolute links once it is set).
- Tag filters accept tag ids as any JSON number or numeric string; other strings remain names.
- Collaboration home rebuilt from standard KPI and table widgets with empty states; the demo Admin parent app is reachable so Collaboration appears in the app tree.
- Dependent possible values for comment parents (thread root/replies), feature-setting scope names (apps/eligible tables/instance), and comment field anchors (session-visible fields), with RC.5 action-level tests.
### Added
@@ -23,7 +64,7 @@ All notable changes are documented here, following [Keep a Changelog](https://ke
- Adopt core feature overrides, record capabilities and implied WRITE locks, reference metadata, mirrored locks, after-commit cleanup and summaries. Core registers feature permissions in the policy editor.
- Replace `OtherUserAccess`/`UserSessionSupplier` with `CollaborationRecordAccessChecker` and core `UserSessionProviderInterface`; configure `withUserSessionProvider`.
- Remove `ParentTableRestriction` and `TagFilterTranslator`; runtime `collaborationTags` filtering/search uses core subqueries. Retain the unpinned visibility fallback still missing from core.
- Publish identifier-only realtime hints after commit. Selectable Tags output and the optional followers SPI await missing core hooks.
- Publish identifier-only realtime hints after commit. Selectable Tags output awaits a core hook; the followers SPI is adopted above.
- Build against locally installed QQQ `4.1.0-SNAPSHOT` until RC.6 is released; `qqq-middleware-javalin` is a compile dependency. Object storage remains pinned to `0.1.0-RC.2`.
+173 -19
View File
@@ -1,9 +1,29 @@
# QBit Collaboration
> **Version 1.0.** Published to the KOF22 Gitea Maven registry (`com.kingsrook.qbits`). Requires QQQ Core `4.1.0-RC.22` or later. See [current status](docs/QBIT-RELEASE-STATUS.md).
Record collaboration for QQQ: attachments, tags, custom fields, comments with mentions, presence, sharing, favorites, follow, timeline and tasks. **Implemented now:** feature settings and people tables, favorites and recents services (phase b), per-user security, daily recents retention, phase-c attachments (upload/list/download/rename/delete, previews and parent cleanup), and the Collaboration app home with real session-scoped counts. Phase e adds comments/mentions, follow actions and digests, the audit/activity timeline, and automatic people-directory updates. Phase d adds tags, vocabulary policies, management and bulk actions, and core-integrated tag filtering/search. Phases h/i add sharing/access requests, tasks/reminders, assignments and My work services. Phases f/g add the custom-field designer and ephemeral replicated presence. Runtime overrides and TAGS mode settings resolve through core.
Design: [QRun-IO/qqq PR #1136](https://github.com/QRun-IO/qqq/pull/1136), `docs/COLLABORATION.md`. See [the phase table](docs/DESIGN.md).
## In-app help
The qbit registers [help/qbit-collaboration.yaml](src/main/resources/help/qbit-collaboration.yaml)
with QQQ's help-bundle registry. Core startup sync applies it when the host has the
`helpContent` table; registration is safe without that table. Update the bundle's
literal version alongside the POM version when releasing.
To customize text, register a separate bundle with `layer: HOST` and the same entry
key and role using `HelpBundles.register(instance, "help/my-host.yaml")`. HOST text
takes precedence over QBIT text; startup sync preserves user edits. Hosts should
supply help for their configured table-specific follow, tag and assignment actions,
and for a renamed Collaboration app.
`CollaborationHelpBundleTest` runs `HelpBundleAssertions` against the produced
metadata and checks widget and possible-value-source coverage. The table guardrail
also checks that every table explains how its rows are created. Run `./mvnw -q verify`.
## Dependent fields
Comment parents list the selected thread's root and replies. Feature-setting scope names follow `scopeType` (apps, eligible tables, or `instance`). Comment field anchors follow `tableName` and offer non-hidden fields visible to the session. These use RC.5 filter variables and custom providers receiving `otherValues`; automatic dependency refresh awaits core `dependsOn`. See [dependent-field details](docs/CONTINGENT-POSSIBLE-VALUES.md).
@@ -11,10 +31,100 @@ Comment parents list the selected thread's root and replies. Feature-setting sco
## Requirements
- Java 21.
- QQQ `4.1.0-SNAPSHOT` installed from develop, including UI API extension point #1128. The build will move to `4.1.0-RC.6` once released; middleware Javalin is a compile dependency.
- A configured QQQ backend supporting query, get, insert, update, delete, count and aggregate actions. PostgreSQL 15+ is the intended persistent store (the generated personal-target indexes use `NULLS NOT DISTINCT`); tests use the memory backend.
- QQQ `4.1.0-SNAPSHOT` built from the compatible Core integration branch; see [Core dependency setup](docs/CORE-DEPENDENCY.md). Middleware Javalin is a compile dependency. Public RC.21 lacks the required rollback API.
- A configured QQQ backend supporting query, get, insert, update, delete, count and aggregate actions. PostgreSQL 15+ is the intended persistent store (the generated personal-target indexes use `NULLS NOT DISTINCT`); tests use memory, H2 and disposable PostgreSQL/MySQL containers.
- Attachments require `qbit-object-storage:0.1.0-RC.2` (a pinned dependency), installed with a PRIVATE bucket alias matching `attachmentsBucketName`. Configure its connection and credentials in the host; this qbit does not provision buckets.
- Core feature resolution, permissions, reference security, summaries, query expansion and realtime use QQQ `4.1.0-SNAPSHOT`, targeting RC.6.
- Core feature resolution, permissions, reference security, summaries, query expansion and realtime use the same matching Core snapshot.
## Followers and secured data
The instance registers a core `RecordFollowersProviderInterface` backed by
`collaboration_follow`, so alerts and other qbits can use `RecordFollowers.get`.
CHANGE and ALERT select `notifyOnChange`; COMMENT selects `notifyOnComment`.
Future mutes are excluded, and disabling FOLLOW returns no subscribers. The
provider reads only the requested table/record as the system. Consumers must
check each recipient's record access before sending data.
Favorites and recents cache a record label only when every label field is FULL
for the current session; otherwise the stored label is null. Lists continue to
re-read labels as the viewer. Comment and share notification titles resolve
labels in each recipient's session and fall back to the table label and id
(for example, `Order #1042`) when label access is restricted or unavailable.
Timeline audit reads use core's enforced query boundary: MASKED values are
masked, HIDDEN details are omitted, and restricted audit messages are cleared.
Comment bodies remain user content; realtime field-name filtering is handled by core.
## Demo server
`CollaborationDevServer` is a disposable, memory-backed Sales CRM for trying the
Next dashboard and taking screenshots. It needs Java 21 and the matching Core integration snapshot. No database, Docker or MinIO is needed. It binds
to `127.0.0.1`; stopping the JVM discards edits and uploaded attachments.
Build the Next static export in `qqq-frontend-next`:
```bash
QQQ_NEXT_OUTPUT=export pnpm build
```
In PowerShell, use `$env:QQQ_NEXT_OUTPUT = 'export'; pnpm build` for that command.
Then, from this repository, prepare the test classpath and place the export under
a classpath directory named `next-dashboard`, following StorageDevServer's layout:
```powershell
.\mvnw.cmd -o -q test-compile dependency:build-classpath '-Dmdep.includeScope=test' '-Dmdep.outputFile=target/demo-classpath.txt'
New-Item -ItemType Directory -Force target/demo-frontend/next-dashboard | Out-Null
Copy-Item -Path ../qqq-frontend-next/out/* -Destination target/demo-frontend/next-dashboard -Recurse -Force
$demoDeps = (Get-Content target/demo-classpath.txt -Raw).Trim()
$demoClasspath = "target/test-classes;target/classes;$demoDeps;target/demo-frontend"
java '-Dqqq.javalin.frontend=next' '-Dcollab.dev.port=8098' '-Dcollab.dev.user=alice' -cp $demoClasspath com.kingsrook.qbits.collaboration.devserver.CollaborationDevServer
```
The equivalent Java command is
`java -Dqqq.javalin.frontend=next -Dcollab.dev.port=8098 -Dcollab.dev.user=alice -cp "target/test-classes;target/classes;DEPS;EXPORT" com.kingsrook.qbits.collaboration.devserver.CollaborationDevServer`.
`DEPS` is the generated classpath file's contents; `EXPORT` is the folder **containing**
`next-dashboard/`, not the export itself. On macOS/Linux use `./mvnw` and `:` instead
of `;` between classpath entries.
Open [the demo](http://localhost:8098/?createMockSession=true), then **Sales → People**
or **Sales orders**. [Person 1](http://localhost:8098/app/person/1) and
[order 1](http://localhost:8098/app/sales_order/1) have the richest collaboration panels.
Pick the user per browser: add `user=` to the page load, for example
`http://localhost:8098/?createMockSession=true&user=bob`. The server remembers the choice in a
`collab_dev_user` cookie, so **open a second browser with `?user=bob` to see presence** (your
first browser keeps its own user; browsers on the same profile share the cookie). Without a
choice, `-Dcollab.dev.user=alice|bob|admin` (one value, not the literal list) is the default.
The MOCK session customizer supplies stable identities, display names (the sidebar shows
Alice Morgan, not Anonymous) and real permissions, also used by recipient access checks.
Shared record links are absolute (`frontendBaseUrl` is `http://localhost:<port>`):
| User | Permissions |
| --- | --- |
| `alice` — Alice Morgan (default) | Full Sales access and comment, attach, tag, share, task and assign permissions |
| `bob` — Bob Chen | People read/comment; no Sales order access |
| `admin` — Demo Administrator | Full Sales access plus `collaboration.admin`, vocabulary/field management and Admin → Collaboration |
Both tables enable every record feature. People use FREE tags and host owner/assignee
fields; orders use APPROVED_ONLY tags. The seed contains 25 fictional contacts at
10 companies, 40 orders with deterministic amounts/dates, eight approved tags,
two ACTIVE custom fields (Preferred contact; searchable Purchase order no.), six
comments with three mentions and a resolved thread, three generated PNG/PDF attachments,
follows, four tasks (one overdue and one completed), two order assignments, two shares, Bob's open order-access
request, and six favorites/recents for Alice. Ordinary record updates create FIELD
audit history. Task deadlines are relative to startup so the overdue example stays useful.
Attachment previews/downloads use real in-memory bytes and expiring local URLs.
Notifications are recorded in memory; they do not send email. Presence is live and
ephemeral: the current browser supplies its heartbeat; there are no fake persistent viewers.
Two current permission edge cases remain visible in this snapshot: Bob can start a
comment thread, but replying to another author's thread fails its activity update;
an access request carrying a share ID for an unreadable parent returns 404. The seed
uses Bob's own thread and a direct access request without a share ID.
The test builds and seeds without frontend assets, validates the instance, then checks
the version/collaboration endpoints, downloads and Bob's restricted access on a free port:
```powershell
.\mvnw.cmd -o -q verify
```
## Install
@@ -43,6 +153,15 @@ new CollaborationQBitProducer()
Omit `withParentAppName` for a top-level app. Run normal QQQ instance validation after all producers.
## Starter defaults and setup
Collaboration ships no SQL seed. The one configuration table, `collaboration_feature_setting`, depends on the host's configuration (`withDefaultFeatures`, `withTableFeatures`, the default tag mode, features declared on apps and tables), so its rows are computed at run time:
- Open Feature Settings and run the table action **Create rows from configuration** (collaboration administrators). It writes one Instance row per feature, plus an App or Table row wherever the code declares something at that scope. The rows say exactly what the code already says, so nothing changes; `FeatureSettingsDefaultsTest` proves every table resolves identically with and without them.
- It only creates rows when the table is empty, so it never touches an administrator's rows. Rows are applied after all code configuration. An Instance row overrides what code declares on any app or table that has no row of its own, so once rows exist, change features in the rows, not in code.
- The **Setup** card on the Collaboration home shows each feature, whether it is on, for which tables, and whether that comes from Feature Settings rows or the configuration.
- Every collaboration table has help (shown when it is empty) that says how its rows get created. The other tables hold user content or runtime data and start empty by design.
## Tables
| Table | Purpose |
@@ -66,7 +185,7 @@ Omit `withParentAppName` for a top-level app. Run normal QQQ instance validation
All tables include `id`, `createDate` and `modifyDate`, identity/details/dates sections, and table possible-value sources. SQL table names match metadata table names; SQL columns use snake_case. Use a stable nonempty scope name (convention: `instance` for INSTANCE; the metadata app/table name otherwise). JSON validation and override resolution arrive with the provider, not in this scaffold.
The home reports persisted enabled settings by scope type, people known and people seen within the inclusive last seven days, plus Favorites and Recently viewed row counts and distinct user counts, attachment count and total bytes, comments in the last seven days, open root threads, and follower subscriptions (admins). Attachment aggregates are explicitly approximate until core can mirror parent locks. Recent activity is based on `lastSeenAt`, independently of the directory's `active` account flag. Empty states explain missing data. These counts do not represent resolved feature enablement on host tables.
The home is four full-width standard widgets (no custom HTML, no hard-coded colors, so dark mode works): an Overview of KPIs (comments this week, open threads, open and overdue tasks, open access requests, attachments with a human-readable total size such as 4.7 KB, active custom fields), a Tags table (label, table or All tables, records), a Custom fields table (name, table, status, index state) and an admin-only Who's here table of live viewer counts per table. Every block has a real empty-state message. Counts use the caller's session and permissions; attachment aggregates remain approximate until core can mirror parent locks.
## Permissions
@@ -196,7 +315,8 @@ UI API endpoints are described below. Runtime feature enablement, mirrored WRITE
| `defaultMaxBytes` | `104857600` (100 MiB) | Positive byte limit, capped by the bucket's `maxUploadBytes`. |
| `blockedExtensions` | Empty | Case-insensitive extension denylist, with or without leading dots. |
| `recentsPerUser` | `200` | Positive retention count per user. Enforced by the trim process. |
| `notifier` | `QCodeReference(NoOpNotifier.class)` | Implements `CollaborationNotifier`; replace with a QAlerts.notify adapter when QRun-IO/qqq#1125 is released. |
| `notifier` | None | Implements `CollaborationNotifier`; replace with a QAlerts.notify adapter when QRun-IO/qqq#1125 is released. When unset: `EmailCollaborationNotifier` if qbit-email is on the classpath and produced in the instance, otherwise `NoOpNotifier`. |
| `recipientEmailResolver` | People directory | `Function<String, String>` from user id to email address for the email notifier. Unset reads the `email` of the matching `collaboration_user` row; no address means no email. |
| `userSessionProvider` | None | `QCodeReference` implementing `UserSessionProviderInterface`; missing user sessions mean unknown access. |
| `followActionTables` | Empty | Existing host tables to receive standard Follow and Unfollow Review/Result processes. |
| `followChangeTables` | Empty | Existing host tables whose UPDATE events should produce five-minute follow digests. |
@@ -204,6 +324,18 @@ UI API endpoints are described below. Runtime feature enablement, mirrored WRITE
| `parentAppName` | None | Optional existing parent, such as `admin`. |
| `adminApp` | `true` | Set false to omit the app/home. Record child-list widgets remain available. |
## Email through qbit-email
qbit-email 0.1.0-RC.2 is an optional dependency (`<optional>true</optional>`). The integration uses its published `com.kingsrook.qbits.email.api` package for installation detection, template registration and sending. When it is on the classpath, produced in the same instance, and `notifier` is not configured, mentions, shares and task notifications are emailed by `EmailCollaborationNotifier`. Without qbit-email nothing changes: `NoOpNotifier` sends nothing. A configured `notifier` always wins.
The guardrail allowlist properties retain the object-storage API and `*-api` artifact allowances and add only the qbit-email artifact and `com.kingsrook.qbits.email.api` package. The current parent 2.0.0 does not execute guardrails; these properties are ready for a parent that does. Tests retain `EmailQBitProducer`/`EmailQBitConfig` only to assemble the host instance; G1 checks production classes and sources.
- Templates (NOTIFICATION, Markdown, owner `qbit-collaboration`), registered on first send and never overwritten: `collaboration_mention`, `collaboration_share`, `collaboration_task_reminder` (task assignment, reminder and overdue). Follow digests are not emailed.
- One message per recipient, idempotency key `collaboration:<mention|share|task>:<event id>:<recipient user id>`, linked to the parent record, sender `qbit-collaboration`. A repeat is a duplicate. The event id is the mention, share, task or assignment row id (task reminders add the due field); `CollaborationNotification` carries it as `sourceId`, with the acting user as `actorUserId`.
- No comment bodies, task titles, notes or share messages are emailed, since they may hold PHI. The email says who did what on which record, with a link. The record label is read as the recipient (needs `userSessionProvider`) and used only when `FieldAccessHelper.isRecordLabelFullAccess` allows it (reflective guard); otherwise the text says "a record in <table label>". Variables are classified CONFIDENTIAL so the templates need no PHI approval.
- The link is sent only when absolute: set `frontendBaseUrl`.
- Recipient addresses come from `withRecipientEmailResolver(Function<userId, String>)`, else the people directory email. Refused or suppressed sends are logged by user id and status only; delivery problems never fail the collaboration action.
## Adopting in another app
1. Register the host backend and optional Admin app, then invoke the producer above.
@@ -277,17 +409,26 @@ size, pending suggestions, and the five most-used approved tags.
`collaborationTags` is a virtual STRING field labeled Tags with possible-value source `collaboration_tag`. `TagCriteriaExpander` supports IN, NOT_IN, EQUALS, NOT_EQUALS, IS_BLANK, IS_NOT_BLANK and CONTAINS_ALL using core `QSubQuery` membership filters. Ordinary quick search matches labels; `#name` matches normalized names. Numeric criteria are tag IDs; string criteria are names. The old ID materializer is removed. Core owns SQL execution or bounded cross-backend materialization; its bound applies to inner rows, not final parent matches.
The runtime provider adds this field and its search entry where TAGS is enabled. It is queryable but not selectable: the overlay has no virtual-field post-query hook and existing host customizers must be preserved.
The runtime provider adds this field and its search entry where TAGS is enabled. It is queryable and selectable (`TagValueProvider` fills it after host customizers have run).
### What the Tags field costs a read
Core selects every selectable virtual field for a query or get that names no fields, so each such read of a table with TAGS enabled runs `TagValueProvider`, including the reads other code makes one record at a time (a get, the stored-record read before an update, a lookup made per row). The provider therefore costs:
- nothing for a table that has no tag assignment. Each node remembers that one fact per table (`TaggedTables`) for `withTagLookupCacheSeconds` (default 30; 0 looks every time) and learns it with one indexed `LIMIT 1` statement;
- one assignment statement and one vocabulary statement per 500 parents for a table that has assignments. Record tags themselves are never cached.
A node that writes an assignment forgets the table's answer at once and again after commit, and publishes core's `memo-invalidate` cluster event (`ClusterCaches`, name `collaboration.taggedTables`). Hosts with a ClusterBus transport see a table's first tag on every node immediately; without one, another node shows it after at most the configured seconds. The same bound covers assignments written outside QQQ. The answer does not depend on the session (the probe ignores row locks and returns no data), so it cannot change which records or tags a caller sees. A failed probe is thrown, not remembered.
## UI API
The producer registers extension `collaboration` version `1.0.0` at
The producer registers extension `collaboration` version `1.2.0` at
`/qqq/v1/qbit/collaboration`. It uses the signed-in user's session and existing
services. Disable registration with `new CollaborationQBitConfig().withUiApi(false)`.
Presence, shares, tasks, my-work, direct upload and thumbnail routes are not part
of this extension version.
Requires QQQ `4.1.0-RC.15` and UI API `1.6.0`. Operations carry stable operationIds,
the collaboration tag and standard error responses.
All successes return HTTP 200. All bodies have named `Collaboration*V1` schemas
All successes return HTTP 200. Success and request bodies have named `Collaboration*V1` schemas
in [the extension contract](src/main/resources/openapi/qqq-v1/collaboration.yaml).
The server publishes these alongside core at `/qqq/v1/openapi.yaml` and `.json`.
@@ -349,9 +490,11 @@ attachment filename; preview requests inline content. QQQ's current OpenAPI
`Response` model cannot declare the `Location` header, so this contract uses the
JSON URL alternative consistently.
A missing or invisible parent returns identical `404 {"error":"Not found"}`.
Missing write permissions return 403; invalid input returns 400 with an `error`
message through QQQ's exception mapper. Provider/backend failures remain server
A missing or invisible parent returns 404 with the same `Not found` error message.
Missing write permissions return 403; invalid input returns 400 through QQQ's
exception mapper. Errors use `BasicErrorResponseV1` with `error`, `code`, `message`,
`traceId` and optional `fieldErrors`. `CollaborationErrorV1` remains defined but is
deprecated since 1.1.0. Provider/backend failures remain server
errors. Services continue to own author, record-lock and permission checks.
Mention candidates expose `canRead` as the string enum `"true"`, `"false"`,
`"unknown"`; unknown does not imply the recipient can read the record.
@@ -477,7 +620,7 @@ host integrations as other collaboration tables.
## Core contracts and completed UI API
The qbit targets QQQ RC.6 using the locally installed `4.1.0-SNAPSHOT`. Defaults enable FAVORITES, RECENTS and TIMELINE for eligible host tables; all other features start off. Configure before producing metadata:
The qbit uses QQQ `4.1.0-RC.7` from Maven Central. Defaults enable FAVORITES, RECENTS and TIMELINE for eligible host tables; all other features start off. Configure before producing metadata:
```java
config.withDefaultFeatures(RecordFeature.FAVORITES, RecordFeature.RECENTS, RecordFeature.TIMELINE)
@@ -507,9 +650,9 @@ New routes, all under `/qqq/v1/qbit/collaboration`:
| PUT | `/records/{table}/{pk}/assignment` |
| GET | `/my-work` |
Typed `CollaborationWorkSchemas` describe requests and results. Task PATCH preserves omitted fields, accepts OPEN/DONE/CANCELLED transitions, and permits explicit null for optional fields; null `remindAt` selects the service's due-date default. Assignment PUT replaces owner and assignee; null clears. Invalid input is 400, denied visible-record operations are 403, and missing/invisible parents are 404. Access requests deliberately never inspect parent existence. The extension stays at unreleased **1.0.0**; contract tooling accepts these additions without a bump.
Typed `CollaborationWorkSchemas` describe requests and results. Task PATCH preserves omitted fields, accepts OPEN/DONE/CANCELLED transitions, and permits explicit null for optional fields; null `remindAt` selects the service's due-date default. Assignment PUT replaces owner and assignee; null clears. Invalid input is 400, denied visible-record operations are 403, and missing/invisible parents are 404. Access requests deliberately never inspect parent existence. The extension is now **1.1.0**, including the UI API 1.6.0 error contract and published presence schemas.
`CollaborationRealtime` publishes `comment.added/updated/deleted`, `attachment.changed`, `tags.changed` and `task.changed` on `record:{table}:{pk}` after commit. Successful notification delivery publishes `notification.created` on `user:{userId}`. Payloads contain identifiers only. The optional followers provider is skipped because `RecordFollowersProviderInterface` is absent from the installed jar. Selectable Tags values remain unavailable because the overlay has no output hook.
`CollaborationRealtime` publishes `comment.added/updated/deleted`, `attachment.changed`, `tags.changed` and `task.changed` on `record:{table}:{pk}` after commit. Successful notification delivery publishes `notification.created` on `user:{userId}`. Payloads contain identifiers only. `CollaborationFollowersProvider` supplies the core followers SPI for alerts and other consumers. Selectable Tags values remain unavailable because the overlay has no output hook.
## Custom fields (phase f)
@@ -590,7 +733,7 @@ Enable `RecordFeature.PRESENCE` on the table. The typed endpoints below
| --- | --- |
| `POST /presence` | `{connectionId, table, pk?, mode: "VIEWING"|"EDITING", fieldName?}` |
| `DELETE /presence` | `{connectionId, table, pk?}` |
| `GET /presence?table=...&pk=...` | Omit pk for a table snapshot |
| `GET /presence?table=...&pk=...&perConnection=true` | Omit pk for a table snapshot; `perConnection` is optional |
Session identity supplies userId and displayName. Every call rechecks table READ and record
GetAction visibility. Disabled features and denied/missing parents return the same 404. A
@@ -602,6 +745,17 @@ sweep and expiry checks on reads. The cap is 50,000 leases per pod: new targets
a warning while existing leases can renew. Tabs merge per user; EDITING wins and the latest
editor supplies focus. Nothing is persisted, including directory rows.
Per-connection presence (extension 1.2.0, #54): each merged viewer carries `connections`, the
number of live leases (tabs, terminals) behind it, and every snapshot carries `leaseCount`, so
two terminals of one user read as one person with two connections. `perConnection=true` adds
`leases`, one viewer per lease with an opaque `connectionId` (`c` and the first 16 hex digits of
SHA-256 of `userId`, a newline and the client connection id), its `mode`, `fieldName` (record
targets only), `sessionId` and `since`, the lease start. `presence.changed` events always carry
`leaseCount` and `leases`. Session uuids are bearer credentials, so a lease never stores or
publishes one: it keeps `sessionId`, `s` and the first 16 hex digits of HMAC-SHA256 of the
session uuid under `QQQ_PRESENCE_SESSION_SECRET` (system property or environment; set the same
value on every pod so ids agree) or else a random per-process secret. `PresenceRegistry.counts()` still counts users; `connectionCounts()` counts leases.
Every heartbeat/leave replicates over core's bus on `instance:presence`, a channel refused by
core client subscriptions. `presence.changed` is emitted to record and table channels. Table
snapshots aggregate users without record ids or focused field names; those stay on record
@@ -610,5 +764,5 @@ refetch after reconnecting.
The initial overlay refresh starts the registry on every pod, including those without local
viewers. Endpoints also initialize it lazily. At shutdown call `PresenceRegistry.close(instance)`
before `RealtimeBus.close(instance)`. The admin-only **Who's here** home card shows user counts
per table, and core record-feature summaries include the presence snapshot.
before `RealtimeBus.close(instance)`. The admin-only **Who's here** home card shows user and
connection counts per table, and core record-feature summaries include the presence snapshot.
+9
View File
@@ -0,0 +1,9 @@
# Upgrading
## Unreleased
No schema change and nothing required of hosts.
- New table action **Create rows from configuration** on Feature Settings. Run it once if you want the effective feature settings visible and editable as rows. It changes no behavior and does nothing once the table has any row. Note that after rows exist they win over later changes to `withDefaultFeatures` or `withTableFeatures` in code, so change those settings in the rows from then on.
- Reads of tables without any tag assignment skip the assignment lookup (`withTagLookupCacheSeconds`, default 30 seconds, 0 restores a lookup on every read). A host running several nodes without a ClusterBus transport shows a table's very first tag on the other nodes after at most that long; later tags on that table show at once, as before.
- New **Setup** widget first on the Collaboration home, and table help text. Hosts that override help on a table keep their own.
+81
View File
@@ -0,0 +1,81 @@
# Attachment lifecycle extension
Collaboration owns standard attachment storage and record access. A host can configure `CollaborationQBitConfig.withAttachmentLifecycle(new QCodeReference(HostPolicy.class))` to enforce retention, registration and mandatory access-event persistence. `HostPolicy` extends `AttachmentLifecyclePolicy`. Collaboration has no dependency on Record Files, generation, scripting or a particular event store.
The default policy preserves legacy behavior for ordinary attachments. Invalid policy loading fails before storage mutation. This is a trusted Java extension, never an entry point for untrusted user scripts. Policy implementations must be safe for concurrent requests and may not depend on per-instance state surviving between operations.
## Contract
| Hook | Timing and meaning |
| --- | --- |
| `isRetained(attachment)` | Called before object storage during intake with complete metadata but no generated id. Its result is persisted in the server-controlled `retained` field. Also consulted for legacy rows during deletion and display hints. A persisted true value always wins. |
| `beforeStored(attachment)` | Called with canonical metadata and actual checksum immediately before the first object PUT, after matching retry replay returns. Retained files default to refusal; a mandatory provider commits storage intent independently. Retained uploads reject ambient transactions. |
| `afterStored(attachment)` | Called after object and attachment metadata insertion, before upload returns. Register the retained identity and ingestion status. Failure preserves the object and inserted row for reconciliation when the enclosing transaction permits it. |
| `beforeAccess(attachment, inline)` | Called after parent authorization, before signing either standard preview or download URLs. Throw to deny. Runtime policy failures are converted to checked failures so preview code cannot silently swallow them. |
| `beforeMetadataChange(attachment, changes)` | Called after parent/ownership checks, before applying filename/description changes. The second record contains id and changed fields only. This represents an intent, not a committed change. |
Callbacks receive record copies; scalar attachment metadata is the contract. They must not mutate the passed records to influence storage or authorization. They may throw `QException` to fail the operation. A failed registration must remain inaccessible through the policy until reconciliation succeeds; absence of a retained registry entry must never mean unrestricted access.
## Persistent retention
`collaboration_attachment.retained` is a required boolean with false default for existing ordinary attachments. Apply the generated PostgreSQL schema migration through the host's normal migration tooling before installing this version. The migration is `ALTER TABLE collaboration_attachment ADD COLUMN IF NOT EXISTS retained BOOLEAN NOT NULL DEFAULT FALSE`. It does not automatically classify pre-existing files; any historical backfill requires a deliberate host migration.
There is no public API to clear retention. Direct user table writes remain denied. Deletion first authorizes and checks retention for the complete selection; it checks retention again immediately before storage deletion. Parent cleanup tombstones retained attachment rows and excludes completed tombstones from subsequent pages. The original bytes remain. The ordinary parent-based UI remains inaccessible after parent deletion; a retained-history view must supply its own preserved authorization policy. This extension does not grant archive access.
Removing the configured provider cannot restore deletion of rows already marked retained. The no-op provider also refuses signed URLs and metadata edits for those retained rows. Replacing a provider is a trusted host operation: the replacement must still enforce audit and quarantine for every row marked retained, even if its table is no longer configured for new retention.
Attachment DTOs expose separate `canEdit` and `canDelete` hints. Both require current ownership/admin and attach capability; retention disables deletion. Existing constructor overloads preserve source compatibility. The server rechecks all permissions on mutation. The old `canDelete` flag must no longer control client rename/description actions.
## Verified content for integrations
`AttachmentsService.readContent(attachmentId, maximumBytes)` authorizes the actual parent/current actor, enforces both content-read policy phases, validates stored length and SHA-256, closes storage on all paths and returns immutable `AttachmentContent(attachmentId, fileName, contentType, sha256, bytes)`. No storage coordinates or signed URL escape. The caller chooses a limit up to 20 MiB; ordinary large-file download remains a separate streaming/presigned path. No bytes leave this method before integrity verification and the result audit hook succeed.
`beforeContentRead(attachment)` must authorize and durably record the attempt before storage opens. `afterContentRead(attachment, bytesRead)` must record verified server-read completion before returning content. Both default to refusal for retained files until the lifecycle provider implements them. These facts are distinct from URL issuance, user download, printer spooling and physical output.
## Retryable intake
An upload overload adds `intakeKey` and `manifestSha256` after the ordinary upload arguments. Both are server-coordinator SHA-256 identifiers. The owner additionally hashes actual authenticated actor and canonical parent into the storage scope; never trust a client key as authority. The intake fingerprint includes the caller's manifest, filename, declared type, description, actual length and actual content hash. Every retry reads and checks its supplied bytes. A matching attachment is returned and its idempotent registration hook is retried; changed inputs conflict.
This path rejects ambient transactions. It conditionally creates a deterministic storage key with fingerprint metadata and never overwrites or deletes an uncertain original. If storage succeeded but metadata insertion failed, the next attempt verifies the existing bytes/fingerprint and completes registration. Host migration must add nullable `intake_key`/`intake_fingerprint` columns and the unique nonnull intake-key index from `CollaborationSchema`. That index is the database race boundary; sequential memory fixtures do not prove its concurrency behavior. The calling coordinator still owns durable request reservation, lifecycle and reconciliation scheduling.
Retryable files are bounded to 20 MiB and use a conditional single-object PUT. The inspected object-storage multipart implementation does not carry conditional-create semantics through completion; it is deliberately not used here. Larger retryable originals need a separately qualified conditional multipart path before this bound changes. This is an advertised intake limit, not a claim of unlimited file sizes or storage immutability. Real storage conditional behavior, object version/lock and backup retention still require provider acceptance.
## Atomic display metadata auditing
`beforeMetadataChange` records an independent attempt. Before the update, `validateMetadataTransaction(attachment, transaction)` validates the actual transaction and audit backend; retained files default to denial. After the update, `afterMetadataChange(attachment, changes, transaction)` inserts the success event into that same transaction and also defaults to denial for retained files. The provider must not commit or close the supplied transaction. Reject no-op transactions before mutation; memory tests only establish callback behavior.
When no caller transaction exists, Collaboration opens, commits, rolls back and closes the owner transaction. If a caller supplied a transaction, that caller owns completion and must roll back on any propagated failure. Successful operation and its event become durable together on commit. Record Files must qualify this with a real supported JDBC backend; this generic owner SPI cannot prove provider durability.
## Durability boundaries and remaining integration
The callback is not an audit database. Before returning from `beforeAccess`, a mandatory audit provider must have durably committed its intent independently of a caller transaction that may later roll back. An event means URL issuance was requested, not completed download or physical printing. Storage signing can still fail after the intent is recorded. Signed URLs can be reused until expiration; each use is not observed by Collaboration. Storage access telemetry or a serving gateway is required for transfer events.
Retained originals are never deleted when metadata insertion fails; ordinary non-retained failed inserts retain legacy cleanup. A committed STORAGE_WRITE_ATTEMPT can identify an orphan even when no attachment row was inserted.
An object put and QQQ metadata insert are not an atomic transaction. If a surrounding transaction rolls back after object creation, reconciliation must recover an orphaned object. The retained-only journal integration below provides explicit registration recovery; it does not provide malware scanning, WORM storage protection, bucket lifecycle protection or backup verification. Those remain deployment requirements.
QQQ record-reference write validation issues an internal GetAction using global metadata. Attachment previews use the existing `QueryOrGetInputInterface.getShouldGenerateDisplayValues()` contract: visibility filtering always runs, while URL generation runs only when display values were requested. Core validation GETs default to false; the standard middleware's display GET/query executors explicitly request true. This avoids incidental signing during validation without removing validators or modifying QQQ core. Trusted callers requesting presentation URLs directly through GetAction/QueryAction must explicitly enable display values; `AttachmentsService.downloadUrl` remains an explicit authorized access operation.
## Aggregate visibility policy
The optional configuration-only `requiresPinnedAggregateParent()` hook defaults to false. Owners whose parent visibility relies on dynamic archive/project authorization opt in. AttachmentAggregateGuard preserves the normal collaboration permission checker and additionally requires Count/Aggregate inputs to contain one literal top-level AND equality for both `tableName` and `recordId`. It checks current native parent visibility before SQL; joins, set-operation filters, OR roots, missing and conflicting pins are rejected. It does not grant view access.
Opt-in table metadata hides generic TABLE_COUNT and QUERY_STATS capabilities in both instance and table metadata. The current frontend's use-filter-setup hook already skips totals when TABLE_COUNT is absent and represents them as unknown; record listings and the attachment panel remain available. Native CountAction still permits an authorized pinned count. Do not substitute zero for an unavailable total or issue an unpinned aggregate and rely on a post-query filter: aggregate SQL cannot use that filter. Ordinary policies preserve the existing count capability. Verification is pending the new focused guard suite.
## Retained upload journal integration (verification in progress)
Public attachment services always use JdbcUploadJournal for retained originals. Ordinary non-retained streaming behavior is unchanged. Apply UploadJournalTable.h2Script() as an owner migration before enabling this path: the immutable physical manifest includes private storage coordinates and lease tokens which deliberately are not QQQ metadata fields. Generating SQL from metadata alone is insufficient. Only shared file-H2 with actual WRITE_DELAY=0 is currently qualified; memory journals are explicitly injected test fixtures only.
Retained uploads are capped at20MiB and use conditional single-request creation. The journal commits the candidate, fences a lease, and records UNKNOWN before PUT. A conditional collision or uncertain response is accepted only after reading the bounded original and verifying actual length and SHA-256. Metadata insertion and journal REGISTERED checkpoint share one real owner transaction. The mandatory idempotent afterStored lifecycle callback runs after that commit; COMPLETE follows only after it succeeds. A failed callback leaves the same attachment id recoverable. Source archival refuses transfer while the owner checkpoint is incomplete.
Renewal occurs at synchronous stage boundaries. A storage call that outlives its lease cannot register under its expired fence; a newer recovery may inspect an existing matching original but never overwrite or delete it. No background task synthesizes or carries a browser session. Provider request deadlines and operational supervision remain host acceptance requirements.
The native Retained Upload Journal table exposes status to its original actor only, subject to current source visibility; after a completed archive transfer, visibility follows the exact attachment's current parent. Native totals are disabled and private-field predicate/order/unique-key/join probes are denied. The Recover retained upload action reauthorizes the original actor and current attach capability, verifies existing bytes and retries registration. It does not accept bytes, a locator, another actor or a coordinator scope. Coordinated intakeKey entries must retry through their original coordinator, preserving origin/version/provenance. The beforeRecovery(candidate) lifecycle hook is default-deny for retained originals and records a recovery read attempt separately from STORAGE_WRITE_ATTEMPT and user downloads.
Missing bytes or corrupt bytes remain unresolved, never timeout into cancellation or permission to delete possible evidence. A disabled/deleted original actor has no administrative impersonation escape; an explicitly audited trusted administrative reconciliation path remains to be designed. Existing attachment panels also need discoverable pending-upload recovery navigation; the standard table action alone is not a complete panel UX.
### Recovery navigation metadata
Retention owners can call public UploadRecoveryNavigation.configure(sourceTable) after producing source metadata. It adds ATTACHMENTS.recoveryView containing the native owner table, label and actor/status filter. The generic Files panel adds literal canonical tableName/recordId AND criteria and navigates to the standard QQQ table and its recovery action. CurrentUser expressions are accepted only for createdByUserId/EQUALS with one value. Filters aid discovery and never replace original-actor/current-parent authorization. Missing or malformed metadata hides this optional navigation on ordinary installations. An interrupted-upload message does not claim saved/completed bytes; recovery still verifies the owner journal and exact object.
+45
View File
@@ -0,0 +1,45 @@
# Standard attachment list pagination
The standard `GET /records/{table}/{pk}/attachments` endpoint now applies its
page offset and limit inside the native QQQ attachment query. It reads at most
`limit + 1` rows (maximum 101), projects the requested page, and uses the extra
authorized row only to determine whether to emit the next cursor. It performs no
count query and does not load the complete attachment list before slicing it.
The route, response schema, default limit 20, allowed limits 1–100, and existing
version-1 scoped offset cursor bytes are unchanged. Sorting is still descending
`createDate`, then descending `id`; equal timestamps therefore retain a stable tie
order. These are offset pages, not snapshots: concurrent insertion or deletion can
shift offsets. High offsets can require backend scanning; bounded returned rows
and host allocation are not a constant-time database performance guarantee.
Each request still touches the current person, resolves and authorizes the actual
source record, canonicalizes its primary key, excludes deleted-parent attachments,
and uses the existing per-entry edit/delete policy. The same entry projection is
evaluated for the lookahead row. There is no generic post-filter refill or inference
from a denied row. Sources transferred to retained archive records use the exact
current parent identity and its native authorization policy. This owner artifact
does not import Record Files or reconstruct its archive policy.
Metadata visibility is separate from content release. Listing retained or
quarantined metadata does not issue a download, preview or print capability; those
paths retain their lifecycle and current authorization checks. The cursor is a
routing/offset hint, not a signed authorization token. Current source access is
required even when the caller presents a previously valid cursor.
`AttachmentsService.listPage` is the bounded public service operation. The older
unpaged `list` method remains for compatibility; new page consumers should use
`listPage`. Cursor parsing/encoding is shared with the existing complete-list
pagination helper, so other Collaboration endpoints keep their previous behavior.
Verification: local focused run `target/attachment-pagination-focused.log` passed
40 tests on 2026-10-31 at 02:01:44, with zero failures/errors/skips and Checkstyle.
Two new tests execute the actual QQQ RDBMS action against H2 and observe native
offset/limit and returned-row bounds, equal timestamp ordering, deleted/other
source exclusion, retained flags, terminal pages and current revocation before
attachment SQL. The other 38 tests cover existing attachment service behavior and
the unchanged OpenAPI fixture. Root HTTP run47446 also passed all12 tests, including
the new regression covering three pages,
cursor scope, malformed/oversized input and a denied actor. Cross-qbit archive integration, full release verification and installation
of this new owner snapshot remain separate gates. No installed or frozen consumer
artifact has been replaced by this change.
+37
View File
@@ -0,0 +1,37 @@
# Attachment cleanup recovery
Deletion writes a standard `collaboration_attachment_cleanup` row and removes attachment metadata in the same backend transaction. Only after commit does it delete the stored object, then acknowledge the cleanup row. PostgreSQL rejection of metadata deletion leaves both the original row and bytes intact. A provider failure or lost callback leaves the cleanup locator available after restart. The service returns a failure when immediate cleanup fails; the logical attachment deletion may already have committed.
The cleanup table is visible in the Collaboration app to administrators. Direct table mutations are denied, including to administrators, so callers cannot forge deletion targets. Recovery runs through `collaborationRetryAttachmentCleanup`, guarded for system sessions or `collaboration.admin`. With the host's configured scheduler, it retries a bounded batch of 100 every minute. Kof22 already supplies a scheduler name. Without a scheduler, operators must invoke the recovery process; it is not safe to assume background retries exist.
Apply the generated PostgreSQL schema before enabling this version; it creates the additional cleanup table and due-time index. Attachment ID is optional because rejected uploads may never have a committed ID. The upgrade preserves existing cleanup rows and removes the earlier attachment-ID uniqueness constraint. Do not remove that table during application rollback while cleanup remains pending. Existing releases do not know how to drain it.
Verification covers real PostgreSQL metadata rejection, outer upload/delete rollback, commit, provider failure followed by a new recovery service, loss of post-commit callbacks, failed queue acknowledgement and repeat replay. Deterministic storage faults also cover a provider that writes bytes before reporting upload failure and two concurrent cleanup workers. A real MinIO fixture, accessed through the published object-storage QBit, verifies deletion replay in plain and versioned buckets while preserving another object's exact bytes. This is not evidence for every object-storage provider. The separate abrupt-JVM test described below uses fixture file storage.
Failed upload rollback compensation now records its locator independently of the rolled-back transaction before attempting storage cleanup. Failed worker attempts persist a next-attempt time at least 60 seconds later; this lets healthy requests beyond a failing batch proceed. Tests verify later recovery once the failed requests become due.
Upload I/O and runtime failures attempt durable compensation and preserve the original exception, adding cleanup failure as a suppressed exception. Deferred provider completion is covered by the settlement protocol below.
Full Maven verification including operator settlement passes 206 tests, zero failures, errors or skips, plus packaging, help coverage, architecture guardrails and JaCoCo checks. PostgreSQL tests cover the retained receipt, repeated requests, active owners, existing references, rejected receipt writes, provider failure and the real form/Review/Result flow with permission revocation. Coordinated host adoption and staging rollback still require separate verification.
On backends supporting transactional FOR UPDATE SKIP LOCKED (including the PostgreSQL host), upload now commits a cleanup locator before calling storage and holds its row lock for the upload transaction. Metadata insertion and removal of that locator share the same transaction. Recovery workers skip active uploads and hold the same lock while deleting abandoned bytes and acknowledging cleanup. An existing caller transaction retains ownership until it ends; otherwise the service starts its own transaction after spooling the input. Backends without this locking capability retain callback-based compensation and are not covered by this ownership guarantee.
Focused PostgreSQL tests verify the locator exists before provider I/O, cleanup skips an active upload, and recovery succeeds after an interrupted upload whose rollback callbacks were deliberately suppressed. That test simulates callback loss. A separate child JVM executes the actual upload service against the disposable PostgreSQL database, persists fixture bytes, then calls Runtime.halt before returning from the provider. The parent verifies exit code 73, absent metadata, retained intent and bytes, and successful idempotent recovery using the original locator. This proves abrupt JVM termination recovery, not power-loss durability or late remote-provider completion. Another test rejects intent insertion in PostgreSQL and verifies zero provider writes or deletes; a completed upload in an open outer transaction remains protected from cleanup until commit. Concurrent workers verify that only one worker enters provider deletion for a locator.
Issue #1289 remains open for coordinated host adoption and application rollback acceptance. Row locking does not make external object storage and the database one atomic transaction.
## Unreleased late-upload settlement change
A PostgreSQL regression reproduced a PUT finishing after timeout rollback had deleted the still-empty location and acknowledged its cleanup row. Upload intents (no attachment ID) now use AttachmentStorage.cleanupUpload, which removes current bytes and reports whether an earlier PUT can still complete. The generic adapter cannot prove settlement and returns false. Recovery retains the locator for later sweeps, with the existing 60-second backoff and 100-row batch. Ordinary committed deletion and successful upload acknowledgement remain unchanged.
Adapters may report settlement only from authoritative completion/cancellation information, never merely object absence. A deterministic deferred-write fixture verifies the late-completion ordering; this does not establish remote S3 settlement. Generic uncertain-upload records remain pending until the provider proves settlement or an administrator performs the explicit operation below. There is no age-based expiry.
## Resolve a settled upload
In the Attachment Cleanup table, select upload intents and run **Resolve settled uploads**. An identified user with `collaboration.admin` must supply provider completion or cancellation evidence covering every selected locator, then confirm the standard Review screen. A support case or request receipt establishing completion is suitable; object absence, an elapsed timeout, or an application process stopping does not establish that an external provider has stopped writing. If settlement cannot be established, leave the intent pending.
Apply rechecks permission, locks each row, rejects active ownership and existing attachment references, and deletes current bytes before saving completion. Provider or database failures leave the intent unresolved for retry. Committed attachment deletions use the ordinary cleanup retry operation instead.
The service-managed row retains `resolvedAt`, `resolvedBy` and `settlementEvidence`, together with the original locator. Repeated settlement does not overwrite that receipt; scheduled retries exclude resolved rows. Direct edits and deletes remain denied even to administrators. These fields provide the durable operational receipt without relying on the framework's best-effort audit writer. Store a reference to provider evidence, never credentials or private object contents.
Apply the additive settlement columns and pending-row index before this code runs. Kof22's corresponding migration is `2026-10-26-collaboration-upload-settlement.sql`. Retain receipts and pending intents during rollback. Do not start an earlier cleanup worker that ignores `resolvedAt`: it could consume retained receipts. Roll back to the pre-cleanup application with cleanup processing disabled, or use a version that understands settlement receipts.
+76
View File
@@ -0,0 +1,76 @@
# Coordinated collaboration snapshot
## Combined integration follow-up under qualification
PR #51 combines current attachment settlement, My Work, presence recovery,
contributed-field and capacity follow-ups. Its source inventory and gate history
are in [INTEGRATION-20261029.md](INTEGRATION-20261029.md). The candidate selects
verified published Redis `0.1.0-redis-integration-20261029-551f1d5-SNAPSHOT` and
the existing core `4.1.0-virtual-field-values-1196-cc3981bb3-SNAPSHOT`.
The combined collaboration artifact is not yet published. Historical snapshot
and acceptance evidence below is not proof of this candidate's deployment.
## Selectable-field follow-up snapshot
The following focused upgrade is published for QQQ #1196:
| Component | Version |
| --- | --- |
| QQQ BOM and modules | `4.1.0-virtual-field-values-1196-cc3981bb3-SNAPSHOT` |
| qbit-collaboration | `0.1.0-field-values-1196-ecff347-SNAPSHOT` |
| qbit-redis (unchanged prerequisite) | `0.1.0-redis-7fe5492-SNAPSHOT` |
Import this QQQ BOM in the consuming host. Collaboration publisher 168 passed
from `ecff347efe2bb84949b8452a3383e3fdeb4bcede`; its POM and JAR resolve to
timestamp `20261026.133203-1`. Full collaboration verification against the
published core passed 198 tests with no failures, errors or skips, plus packaging
and coverage gates. An independent empty-cache Java 21 consumer downloaded the
pair and compiled both collaboration value providers against the core API.
See [the upgrade evidence](VIRTUAL-FIELD-UPGRADE.md) for checksums and scope.
This is not a combined release of the separate attachment, My Work and presence
recovery follow-ups, nor a Kof22 deployment. The earlier candidate's browser
acceptance below must not be treated as browser qualification of this new pair.
A compatible packaged frontend and host staging qualification remain required.
## Earlier main-line candidate
The earlier main-line candidate is published. Its coordinated versions are:
| Component | Version |
| --- | --- |
| QQQ BOM and modules | `4.1.0-collab-33464d37e-SNAPSHOT` |
| qbit-redis | `0.1.0-redis-7fe5492-SNAPSHOT` |
| qbit-collaboration | `0.1.0-collab-8f348da-SNAPSHOT` |
| qbit-session-store | `0.1.0-session-b5276ab-SNAPSHOT` |
| Next frontend source | `8d372142f8af0c41a75a353edc6d150c33ccf613` |
Maven snapshot repository: https://central.sonatype.com/repository/maven-snapshots/ . Import the candidate QQQ BOM in the host to align transitive QQQ modules. The standalone consumer was verified from an empty cache. This is a feature snapshot, not a stable/GA release or a production deployment.
Frontend build and full browser acceptance: https://github.com/QRun-IO/qqq-frontend-next/actions/runs/39683690943 . Its acceptance-build artifact contains the tested export. No dashboard container image publication is implied. Install the export through the normal host frontend packaging path; upgrade frontend and backend together because the shared-presence issued-lease protocol changed.
## Host requirements
- Configure a named Redis connection through qbit-redis and enable CollaborationQBitConfig.redisPresence. Use explicit deployment/application namespaces and a PresenceScopeProvider derived from authoritative authenticated tenant context.
- Start Redis infrastructure before dependent services and stop dependent services before closing its pools. Session-store may select SHARED_REDIS to reuse the named connection; its session semantics remain owned by session-store.
- Configure TLS/authentication and bounded timeouts/pools for the actual environment. The live acceptance fixture's plaintext loopback Redis and mock authentication are test-only.
- Deliver SSE without proxy buffering. The stream and its subscription updates must reach the owning web node; configure affinity for that connection. Shared Redis does not make node-local SSE connection IDs portable.
- Configure record versions on supported backend paths for stale-save protection. Presence is advisory and never replaces database compare-and-write. Unversioned records do not gain conflict protection merely by enabling presence.
- Use explicit QInstance registration when startup code runs outside QContext. Follow core docs/misc/SessionStoreIsolation.md and session-store docs/SESSION-REGISTRATION.md. JDBC shutdown and health operations must target the owning application; see core docs/lifecycle/application-shutdown.md.
- Apply generated schema/migrations through the host's migration tooling. Do not infer that publishing this library migrated an existing database or an external deployed host.
## Verification
All four standard backend publishers and the three QBit source CI runs passed. Tests against published prerequisites: Redis23, collaboration177, session-store96; zero failures/errors/skips and configured coverage gates passed. Core verification4209 and frontend acceptance39683690943 passed. Frontend local3664 tests and TypeScript passed.
Independent JAR/POM downloads, exact timestamped versions, SHA-256 hashes and an empty-cache standalone consumer passed. Final browser checks used published backend JARs, the exact CI frontend export, two independent JVMs and one Redis endpoint. Cross-node avatars, self exclusion, concurrent editor field focus, preserved drafts, table presence and departure passed. Redis admin pool/subscription metrics, persisted framework UP health history and health-record navigation passed.
The two-node fixture uses independent memory records and mock users; it proves published-artifact presence/admin integration. Database conflict and authentication behavior have separate core/frontend acceptance evidence. Publication does not prove unknown deployed hosts have migrated.
Full evidence and issue reconciliation: https://github.com/QRun-IO/qqq/issues/1293#issuecomment-6329618640 .
## Supported milestone and optional work
The owner requested the main-line flow on N web nodes sharing one configured Redis endpoint. Browser delivery remains SSE plus HTTP lease writes; Redis Pub/Sub supplies invalidation hints. A new WebSocket protocol is not required.
Data-loss/restore qualification (#1300), additional failover topologies (#1301), sustained scale/failure injection (#1302), Windows tooling qualifications (#1304/#1305/#1309), CI mirror resilience (#1307), search layout polish (#1310), long-version preflight (#1312) and test-only AssertJ maintenance (#1314) remain tracked separately in the QQQ project. No measured production capacity or disaster-recovery guarantee is claimed by this snapshot.
+34
View File
@@ -0,0 +1,34 @@
# Core dependency: after-rollback callbacks
Collaboration's retained-upload and cleanup paths compensate external object storage only after a transaction's rollback is confirmed. They need two methods on `com.kingsrook.qqq.backend.core.actions.QBackendTransaction`. These are included in QQQ Core `4.1.0-RC.22`, which this module imports through the BOM; Core `4.1.0-RC.21` does not declare them.
## Required API (reviewed implementation: `b6fb2382`, `third-party-src/qqq-core`)
`qqq-backend-core`, class `QBackendTransaction`:
```java
public boolean supportsRollbackCallbacks() // default: false
public void addAfterRollbackCallback(Runnable callback) // throws UnsupportedOperationException unless supportsRollbackCallbacks(); Objects.requireNonNull(callback)
protected void runAfterRollbackCallbacks() // run once, in order, then clear; exceptions (incl. LinkageError) logged and swallowed
```
Pending rollback callbacks are preserved when a required operation marks the transaction rollback-only, and run only after confirmed JDBC rollback. A failed physical commit clears them because its outcome is uncertain. Successful commit discards them.
`qqq-backend-module-rdbms`, class `RDBMSTransaction`:
```java
@Override public boolean supportsRollbackCallbacks() // true
@Override public void addAfterRollbackCallback(Runnable callback) // super.add...; then rolledBackSinceCommit = false
// rollback()/close(): after connection.rollback() succeeds -> rolledBackSinceCommit = true; runAfterRollbackCallbacks()
// close() rolls back (and so runs callbacks) unless a rollback already happened since the last commit / callback registration
```
## Callers in this module
- `attachments/AttachmentsService.storeUpload`: refuses an ambient transaction where `!supportsRollbackCallbacks()`; registers storage compensation.
- `attachments/AttachmentCleanupQueue.beginUpload`: registers the same cleanup on both `addAfterRollbackCallback` and `addAfterCommitCallback`.
## Core tests to carry with the change
- Base class: `supportsRollbackCallbacks()` is false; `addAfterRollbackCallback` throws `UnsupportedOperationException`.
- RDBMS: callback runs after `rollback()`; does not run after `commit()`; runs when a transaction with a registered callback is closed without commit;
a callback registered after an earlier rollback still runs on close; a throwing callback does not stop later callbacks.
- Integration (already in this module, needs the core): `attachments/AttachmentTransactionsPostgresTest` (needs Docker) incl. `interruptedUploadSurvivesLostRollbackCallback`.
+51
View File
@@ -0,0 +1,51 @@
# Coordinated collaboration integration candidate
This branch combines the current heads of the attachment, My Work, presence
recovery, virtual-field and capacity PRs. It is a qualification candidate, not
a published release or a deployed host.
| PR | Included source |
| --- | --- |
| #45 attachment settlement | `dc65a826ce362416d55d91136fa39225f4a7cf94` |
| #46 bounded My Work | `cdcf7ee36509c7a1fc6f22fa864209eb80a6ff34` |
| #47 presence recovery | `776ebe3fcb6492881bc9d7ec3f6ca6adfa320200` |
| #48 contributed field values | `1da538cf43ba1ed80e3f8fd599729b5691563cbf` |
| #49 capacity qualification | `56a8b760d9c1a1862cb11e80fad07cf5cdc20b3a` |
All five heads are ancestors of integration commit
`44a89d77b1ad3a39d26c512ada78af0884872013`. The stacked My Work and capacity
branches did not contain the latest heads of their base PRs, so those base
heads were merged explicitly as well. No production source was rewritten during
integration. The core prerequisite remains
`4.1.0-virtual-field-values-1196-cc3981bb3-SNAPSHOT`. The candidate now selects
published Redis `0.1.0-redis-integration-20261029-551f1d5-SNAPSHOT` after native
publisher job42 succeeded and independent timestamped POM/JAR downloads matched
their declared version and repository checksums. The timestamp is
`20261029.165102-1`; JAR SHA256 is
`385f69c5d84362dbab40a3044261fb9820e25126785d7f22901121302e6ce076`.
The caller and manifest both select released `kingsrook/qqq-orb@0.8`. The
canonical validator extracted from released 0.8.32 and CircleCI schema validation
pass; no unpublished orb changes are used.
Local `./mvnw.cmd verify -Dqqq.mywork.load=true` ran 238 tests with no skips, but
reported three failures and four errors. Failures include Redis command read
timeouts/rejoin assertions and PostgreSQL connection I/O during the attachment
fixture. Docker Desktop's published-port path has independently exhibited
300–900 ms replies after idle, compared with 0–58 ms on a direct Docker network.
This observation is not proof of every failure's cause. No timeout, assertion,
security boundary, or quality gate was weakened. Native hosted Linux results
are required before publication; the local run is **not** a passing gate.
Native CircleCI job192 subsequently passed the full verification at integration
head `02085baff928c374e3b0d77367fbcc8182f81f93`; CodeQL also passed. Those checks
used the earlier Redis pin. The new published Redis pin above requires its own
current-head native verification before this collaboration artifact is published.
Use an explicit short feature branch (`feature/collab-int-20261029`) and dry-run
the released version calculator: generated help-bundle versions must remain at
most 50 characters. The longer `collaboration-integration-20261029` name would
produce 57 characters and is not a valid publication route.
The successful earlier combined-source checks remain historical evidence only.
Host TLS/ACL, actual identity/proxy/provider acceptance, maintenance/restore
enforcement, artifact publication, and deployment are separate gates.
+92
View File
@@ -0,0 +1,92 @@
# My Work summary query bounds
The summary previously materialized every matching task, assignment and mention.
A regression fixture measured 601 parent lookups for 601 tasks on two parents.
The summary now scans each category in primary-key pages of at most 200 rows.
It retains at most ten display tasks and sorts them using the existing due-date
order (earliest first, null last, then ID). Counts still include every visible
matching row, rather than silently truncating at a display limit. Assignment and
mention filters remain scoped to the signed-in user.
Each page caches parent visibility only for that page, using the existing caller
permission and record-lock checks. Missing or revoked parents contribute neither
rows nor counts; unexpected authorization/backend failures propagate. No decision
is shared across requests. The starting maximum ID bounds each category scan so
concurrent inserts cannot extend it indefinitely. Like the earlier summary, this
is not a transactionally consistent snapshot across the three categories.
This bounds row retention, not total work independently of data size. For N
matching rows, each category needs one high-watermark query plus approximately
ceil(N/200) page queries, and one parent lookup per distinct parent per page.
Many unique parents still require linear authorization work. Per-record heavy
field size is also outside this row-count bound. Database and browser latency,
N-node presence, saturation and production capacity remain part of #1291/#1302.
Regression evidence: the 601-task case passes a budget of at most eight parent
queries, with no task query above 200 rows; correct top-ten ordering and revoked
access on a subsequent request are checked. Another case crosses assignment and
mention page boundaries and excludes hidden parents, other users and undelivered
mentions. These tests use the framework's real memory backend instrumentation.
`MyWorkPostgresTest` repeats the 601-row case using PostgreSQL 17 and the generated
production schema. Independent `pg_stat_statements` observations verify exactly
five task queries and between one and eight parent queries. It also verifies the
same top ten, visible count and next-request permission revocation against SQL.
The fixture passes; these query counts are not sustained throughput or
deployment-capacity measurements.
## Reproducible local workload
Run `mvn -Dtest=MyWorkPostgresTest -Dqqq.mywork.load=true test` with Java21 and
Docker. The fixture owns a disposable PostgreSQL17 container, uses the generated
schema and real authorization checks, and writes `target/mywork-load.csv`.
It never connects to an application database. The optional workload runs after
the normal query-budget regression so ordinary CI keeps the smaller test.
Each scenario has 2,000 tasks: one uses ten shared parents, the other 2,000 distinct
parents. Every tenth parent is inaccessible. Two warmups precede ten measured
sequential summaries. Every sample verifies 1,800 visible tasks and the exact top
ten. SQL calls and server execution time are collected independently using
pg_stat_statements after warmup. Latency includes service execution and database
round trips, excluding fixture setup and assertions.
CSV percentiles use nearest rank. With only ten samples, p95 and p99 both equal
the maximum observed sample; they are descriptive, not stable tail estimates.
SQL figures are totals for all ten samples. This is a single-JVM local workload,
not HTTP/browser latency, sustained concurrency, pool saturation, multi-node
presence or a deployment-capacity claim. Do not derive production alert thresholds
from these numbers. No wall-clock performance assertion is added to CI.
The fixture explicitly uses the framework C3P0 connection provider, matching the
host's provider, with initial/minimum1, maximum3 and a two-second checkout timeout.
It verifies the pool stays within that cap, has no checkout timeouts and returns
all borrowed connections after each scenario. The original fixture used the
unpooled default; the expanded workload failed with a Windows socket-allocation
error after repeatedly opening connections. This was a fixture mismatch, not
proof of a Kof22 outage. The same workload completed with pooling.
Observed local run (Java21, PostgreSQL17 in Docker on Windows, ten sequential
samples after two warmups): shared-parent p50=91.192ms, max=112.105ms; distinct-parent
p50=960.565ms, max=1100.260ms. SQL calls per summary were113 and2013 respectively.
This confirms linear distinct-parent authorization work remains despite bounded
row retention. The existing exact-count semantics still require examining every
matching task. No runtime optimization or production-capacity claim follows from
this fixture measurement alone.
## Compatibility with the later attachment changes
On October 26, an isolated source export combined attachment commit
`dc65a826ce362416d55d91136fa39225f4a7cf94` with the four My Work files from
`2ac067bea4af199486f90e2dc5212baf8910b8fb`. Both branches share base
`15cb8cbf72aa9a5e033280cf5a38384d1295fc7d`; their changed files do not overlap.
This was a source-overlay verification, not a branch merge or publication.
Java 21 `./mvnw.cmd verify -Dqqq.mywork.load=true` passed all 209 tests with zero
failures, errors or skips, including the 29 attachment transaction tests.
Packaging and JaCoCo gates passed. The optional PostgreSQL workload retained the
same visible-count, top-ten, query-budget and pool assertions. Shared-parent
p50/max were 76.301/96.044 ms; distinct-parent p50/max were 939.896/1035.402 ms.
SQL counts remained 113 and 2013 per summary. These local measurements preserve
the limitations above; they do not qualify an integrated published artifact or
Kof22 deployment. The eventual integrated branch still needs its own CI gates.
+179
View File
@@ -0,0 +1,179 @@
# Presence recovery candidate — QQQ #1300
This is an unreleased implementation candidate in collaboration draft PR47.
The existing store/service suites and CI pass; host adoption and enforced
maintenance remain incomplete. It is not part of the published snapshots.
## Contract under qualification
Each deployment/application/tenant/table has a persistent recovery guard in the
same Redis hash slot as its bounded room projections. Each Lua operation reads
that guard atomically with its room work. The guard contains a generation, a
readiness deadline and the latest issued expiry. A missing guard, or complete
loss of table projections before that expiry, starts a fresh generation and a
recovery interval of one lease lifetime. Existing live browsers can reserve and
renew replacement leases during that interval. Snapshots report recovery rather
than a successful empty room until the interval finishes.
Lease entries carry the guard generation. Recovery removes invalid-generation
projections from touched rooms so they cannot consume all capacity and prevent
re-registration. Existing authorization remains in PresenceService; the guard
does not grant record, tenant or field access. The shared adapter maps recovery
to a sanitized service-unavailable response. Record-version conflict checks are
independent of this advisory state.
An untouched new namespace is conservatively unknown too. Its first snapshot
therefore starts a recovery interval. Normal idle expiry retains the guard so
later empty snapshots can be known-empty. The guard has no TTL; tenant-retirement
operations and long-term guard storage must be included in operational capacity
qualification.
## Controlled restore
The host configuration has a bounded `recoveryGeneration`, default `initial`,
included in both the storage namespace and realtime audience. It belongs in
deployment configuration outside the Redis backup. All nodes must use the same
value. A restored guard can otherwise restore old authority; Redis process IDs
cannot detect arbitrary same-process data restoration.
The required maintenance procedure is: stop admission and drain all old web
nodes; restore Redis; choose a never-reused generation outside Redis; start only
nodes with that generation; verify recovery and authenticated re-registration;
then reopen traffic. Do not roll back the generation when rolling back an app.
The host tooling that enforces and verifies this sequence is **not implemented**
yet. The branch does not claim transparent arbitrary restore or split-brain
recovery. Native Sentinel/Cluster routing remains the separate Redis issue #1301.
The storage prefix is v2 so old executables cannot treat generation-bound entries
as old-format authority. This requires a coordinated deployment, not a mixed
version rolling deployment. Existing published v1 snapshots are unchanged.
## Evidence and remaining work
- Before the patch, a two-client disposable Redis test failed because total loss
returned an empty snapshot.
- Seven focused tests now pass: full loss, loss of volatile projections while the
guard survives, restored guard/lease bytes followed by a host generation
change, bounded generation validation, and rejection of inconsistent lease
lifetimes across nodes, guard loss across previously unvisited records, and
shared cold readiness followed by normal idle expiry. The latter verifies
that peer reads do not extend the deadline, the guard remains persistent,
expired projections remain known-empty and expired leases cannot renew.
These use independent clients in
one JVM, not a deployed HA cluster.
- Full wrapper verification passed on October 26: 184 tests, no failures,
errors or skips; packaging and coverage gates passed. This includes the
original seven store tests and two-node six-viewer/two-editor scenario.
The admin dashboard reads every table, so the fixture initializes every
dashboard table for both tenants through the real backend and waits for
recovery without writing Redis guard state.
- The service fixture socket read timeout is one second; its previous 200 ms
limit caused a Redis EVALSHA read timeout. Production timeouts are unchanged.
- The full run includes a real HTTP recovery test: real Redis
FLUSHDB produces 503 during recovery, the former lease returns 409 with
PRESENCE_LEASE_EXPIRED, hidden records and unauthorized viewers remain
inaccessible, and a new lease rejoins through the production recovery window.
This uses a loopback HTTP server with fixture authentication, not a browser
or a deployed host. The focused test and the combined full suite both pass.
- The store suite checks loss of either state or expiry data in both record
and table projections. Server-issued lease renewal must fail unavailable
without changing the surviving projection. It also covers complete loss
of either projection pair and stale departure ordering.
- Still required: deployed-host qualification and enforced host maintenance/reset. Local
per-operation cost measurements at 1,000 and 10,000 leases are recorded in
`acceptance/RECOVERY-COST.md`; sustained production capacity is unqualified.
The implementation uses no INFO call or lifecycle polling cache.
- This is an unreleased draft candidate. Publication and production deployment
have not occurred; the remaining acceptance work above is still required.
## Independent-process acceptance
`docs/acceptance/presence_recovery_processes.py` passed on October 26 using
two independent `SharedPresenceDevServer` JVMs and one disposable Redis container.
Each user renews through the opposite node. After FLUSHDB, both nodes return 503
for presence and 409 for the former leases; fresh leases restore both viewers on
both nodes. The harness terminates only its owned processes and container.
This is fixture authentication with independently seeded memory records, not a
deployed HA application or a browser acceptance result.
Reproduce from the repository root after `./mvnw test-compile`:
```text
./mvnw dependency:build-classpath -Dmdep.includeScope=test -Dmdep.outputFile=target/recovery-process-classpath.txt
python docs/acceptance/presence_recovery_processes.py --java /path/to/java --repo /path/to/repository
```
Requires Python 3, Java 21 and Docker. JVM output and argument files stay under
`target/recovery-processes-*` for diagnosis. The harness uses the production
45-second recovery window, with bounded startup and recovery deadlines.
## Browser recovery acceptance
The same harness accepts `--frontend-repo /path/to/qqq-frontend-next`. It copies
that checkout's existing `out` export into the owned test output directory and
serves it through the real middleware on both JVMs. Playwright is loaded from
the frontend checkout; no API mocks are installed. Both browser users first
see the other viewer, then Redis FLUSHDB triggers the unavailable message and
removes stale people. Both viewer badges return without reloading either page.
The final fixed-candidate run passed all five profiles on October 26 against
commit `4b0d6a32d6bd8ecba023857d3661d9f3f5980975`: Chromium, Firefox, WebKit,
Pixel 7 emulation and iPad emulation. Source/compiled Lua hashes were checked
before the run and after each profile. Traces and screenshots remain under
each `target/recovery-processes-*` directory. The earlier Chromium screenshot
in `target/recovery-processes-229f305e83` was also visually inspected.
Select profiles using `--browser-profile chromium|firefox|webkit|mobile|tablet`.
These tests use the local export and do not establish provenance of a newly
published frontend artifact, physical-device behavior or deployed Kof22
acceptance. Fixture frontend serving is opt-in via `collab.dev.frontend`;
its default remains false.
## Controlled restore acceptance
`acceptance/presence_restore_processes.py` exercises a disposable Redis RDB
restore with two independent JVMs. It saves the populated dataset, verifies
both old JVMs have terminated and their listeners have closed, clears the
dataset, reloads the saved image and starts both nodes with a shared fresh
generation supplied outside Redis. It checks recovery responses, rejects old
leases on both nodes, and requires two fresh viewers to rejoin across nodes.
The final run passed on October 26; logs are retained in
`target/restore-processes-aec5288289`. Both leases are explicitly renewed before
saving, and rejection must complete within 40 seconds of that renewal, before
their normal 45-second expiry. Old leases are rejected again after fresh
viewers have rejoined.
The fixture enables Redis DEBUG RELOAD only on its owned loopback container.
Run after test compilation and classpath preparation:
```text
python docs/acceptance/presence_restore_processes.py --java /path/to/java --repo /path/to/repository
```
This is a protocol acceptance fixture, not host maintenance automation. Actual
host admission closure, replacement prevention and generation configuration
still require implementation and staging validation.
To qualify a packaged backend, add `--backend-jar /path/to/qbit-collaboration.jar`.
This excludes production `target/classes` from the JVM classpath, rejects a
duplicate collaboration dependency JAR, and records the artifact SHA-256 in
`target/restore-processes-*/result.json`. The receipt includes the elapsed time
for rejection of old leases (required to remain below their normal expiry),
the restore/rejoin checks and fixture cleanup. Cleanup failure fails the run.
The test repository still supplies compiled fixture classes and its dependency
classpath; this is not a standalone production launcher.
## Packaged browser qualification
The process harness accepts `--backend-jar` and `--frontend-jar` for artifact
verification. With a backend JAR, production `target/classes` is omitted from the
runtime classpath; test classes provide only the disposable server fixture. The
frontend JAR is served directly instead of copying a local `out` export.
`--frontend-repo` remains required for Playwright tooling. Dependency classpaths
must not contain a second collaboration or frontend artifact.
Each run writes `result.json` with the selected artifact SHA-256 hashes, browser
profile, result and cleanup outcome. A successful test with failed fixture cleanup
exits unsuccessfully. This records local artifact qualification, not publication
or deployment provenance. Supply `--browser-profile` to qualify each required
browser independently.
+6
View File
@@ -0,0 +1,6 @@
# Known issues
- Deployment-specific object storage, database failover, backup/restore and retention policies need host qualification.
- Docker must be running for the attachment transaction tests; enable both `mysqlJournalTests` and `postgresJournalTests` for the additional journal-provider cases. On Windows use `-Dexec.skip=true` for the inherited POSIX console-summary command.
- Redis presence needs `qbit-redis` on the classpath; it is an optional dependency.
- The matching Next UI and host configuration must be verified before rollout.
+13
View File
@@ -0,0 +1,13 @@
# Collaboration — status
**Version 1.0**, built and published by the shared `ci-workflows` Maven pipeline to the KOF22 Gitea Maven registry (`com.kingsrook.qbits`); `mvn deploy` runs the tests once.
## What it contains
Attachment lifecycle with the retained upload journal, recovery and cleanup APIs; presence including Redis-backed presence; tags, custom fields, comments with mentions, sharing, favorites, follow, timeline and tasks; virtual-field and fixture support.
## Dependencies
QQQ Core `4.1.0-RC.22` (imported through its BOM; it provides the rollback callbacks the upload paths need). `qbit-redis` is optional and only needed for Redis presence. Build with Java 21 and `./mvnw verify`; Docker must be running for the attachment transaction tests (set `mysqlJournalTests` and `postgresJournalTests` for the extra journal-provider cases).
Host qualification of object storage, database failover, backup/restore and retention policy remains the deploying team's responsibility. See [known issues](QBIT-KNOWN-ISSUES.md).
+47
View File
@@ -0,0 +1,47 @@
# Shared Redis presence candidate
Status: coordinated candidate artifacts are published; remote CI, empty-cache consumer resolution and published-artifact two-node browser checks passed. See [CANDIDATE-RELEASE.md](CANDIDATE-RELEASE.md) for exact versions and deployment requirements. This guide describes the current main-line scope. See MAINLINE-ACCEPTANCE.md for evidence and SESSION-STATE.md for live release checkpoints.
## Host configuration and dependency ownership
Multiple QQQ web nodes share one configured Redis endpoint. The host enables CollaborationQBitConfig.redisPresence, supplies a started named Redis connection, explicit deployment/application names, and a PresenceScopeProvider deriving one tenant scope from authenticated host context. Browser input cannot choose the tenant.
Collaboration owns presence semantics, Lua scripts, record/table projections and authorization. qbit-redis owns connections, namespaces, script execution, pool lifecycle, admin management and framework health integration. Collaboration production code consumes only the Redis public API and does not import Jedis. Without shared configuration, the local presence backend remains available for a single node; its issued authority is not distributed. Runtime Redis failure never silently switches an HA host to local presence.
Browser delivery uses the existing SSE stream with HTTP lease writes. Redis Pub/Sub carries scoped invalidation hints between nodes; a new WebSocket transport is not required. Hints contain table/record identifiers, not record contents or viewer lists. The recipient rereads through the authorized presence API. Core checks the authenticated recipient audience during live delivery and replay. Coordinate core/frontend/QBit versions and do not share a topic with audience-unaware hubs.
## Issued lease lifecycle
The candidate collaboration extension version is 1.3.0:
- POST /presence/leases authorizes the table/record and creates an invisible, expiring reservation, returning a server-issued connectionId.
- POST /presence/leases/renew requires that issued handle and a positive safe-integer sequence. It activates or updates existing live authority; it cannot recreate an expired or departed lease.
- DELETE /presence performs ordered departure. Delayed renewals cannot reopen the departed issued handle.
- HTTP 409 with code PRESENCE_LEASE_EXPIRED tells an active browser to open a fresh lease. Input, permission and infrastructure failures retain their standard mappings. Unavailable shared storage returns 503 rather than an empty-room result.
- Shared presence rejects legacy heartbeat upserts. Local legacy behavior remains for compatibility.
The Next candidate awaits open before activation, uses the handle across focus changes, and releases it on hide/unmount. It opens again on pageshow or explicit expiry. Generation checks discard old asynchronous responses; a late open result is released without activation. One immediate rejoin is allowed after expiry, then retries follow the normal 20-second heartbeat. Infrastructure errors do not churn handles. The UI preserves drafts and shows unavailable status when presence cannot be read or renewed.
## State and permissions
Record and table rooms use bounded expiry indexes and state hashes in the same Redis cluster hash slot for their deployment/application/tenant/feature namespace. Lua updates both projections atomically using Redis TIME. User/connection/record identifiers are encoded independently. No record values are stored.
Snapshots merge tabs by user. EDITING takes precedence over VIEWING; focus follows the latest renewal within the selected mode. Table projections omit focused fields. Expired entries are pruned on reads/mutations and idle keys expire. Invisible reservations count toward capacity. Scripts use EVALSHA with fallback only for NOSCRIPT, through the Redis QBit API.
Reads and renewals recheck parent and field access. Canonical record identifiers and recipient field filtering are enforced. Admin counts are limited to known rooms in the administrator's authenticated tenant; non-admins receive no counts. Capacity and TTL limits bound individual operations; they are not measured sustained production throughput guarantees.
## Verified behavior
- Real Redis tests cover six users on a record with two editors, duplicate-tab aggregation, ordering, departure, expiry, isolation, capacity and script-cache recovery.
- Service and real HTTP tests cover issued open/activation/renew/departure, 409 expiry, authorization, canonical keys, field projection and unavailable responses.
- Clean release-source collaboration verify passed 177 tests with coverage/architecture gates; focused frontend suites passed 34 tests, TypeScript and changed-file lint.
- Live browser acceptance with mocks disabled used three identities on two independent QQQ JVMs sharing Redis. It verified avatars, field focus, drafts, table projection and departure. Demo records were independently seeded memory data; this is presence evidence, not cross-node database save-consistency evidence.
- Live Redis admin acceptance verified pool/subscription status, persisted framework health results and navigation to standard health records.
Presence is advisory. Atomic optimistic record-version checks protect saves independently. Coordinated cold-consumer acceptance against published artifacts remains required; local Maven cache success does not prove remote artifact availability.
## Deferred work
The user narrowed this release to the normal flow. Optional qualification is tracked in QQQ issues #1300 (data loss/backup restore), #1301 (failover/disaster topologies), and #1302 (sustained scale/failure injection), all on the QQQ Roadmap backlog.
Issued leases do not provide disaster-recovery fencing: complete state loss can temporarily look empty, and restoring an old live lease can restore its authority. No recovery generation guard is implemented in this candidate. The deferred investigation is tracked in [QQQ #1300](https://github.com/QRun-IO/qqq/issues/1300).
+52
View File
@@ -0,0 +1,52 @@
# Retained upload journal
The Collaboration-owned journal is wired into retained `AttachmentsService` upload and recovery. It persists candidate metadata before object creation, owns the attachment/checkpoint transaction, and exposes native read-only recovery navigation. It has no Record Files dependency and stores no second copy of file bytes.
The integrated owner snapshot passed all **214 tests**, including 10 real MySQL provider cases and 11 H2 journal regressions, and was installed under `0.1.0-record-files-SNAPSHOT`. The subsequent doublewrite/session-read-only delta compiles and awaits its provider rerun. This document does not qualify Aurora, other SQL vendors, external object storage, disks, backups, or a host deployment.
## Ownership and recovery
1. `prepare` independently commits the immutable server-built candidate before the first PUT. Matching retries reuse it; changed manifests conflict.
2. `reserve` locks the row and grants a bounded lease with a monotonically increasing fence. `renew` extends only a still-live exact reservation at synchronous stage boundaries.
3. `recordStorage(UNKNOWN)` precedes conditional object creation. `VERIFIED` means the storage owner proved the exact length and SHA-256.
4. `register` passes its real transaction to the attachment registrar. The attachment insert and REGISTERED checkpoint commit together; callback failure rolls both back.
5. The retained lifecycle registration runs after commit and is retried idempotently. The journal becomes COMPLETE only after it succeeds. Completed coordinated retries still invoke required lifecycle registration without adding a second attachment or activity event.
6. Recovery reauthorizes the original actor and current source. It verifies the stored original without overwriting it and resumes registration. Missing or unreadable bytes remain uncertain; timeout never proves absence.
The owner rejects ambient/no-op transactions. SQL values are bound; mutable operations lock rows, check the token/fence/expiry, and require exactly one guarded update. Reads reconstruct manifest hashes. Private locator and lease columns are absent from QQQ metadata and forbidden in native filters/projections; native direct writes and unscoped totals are denied. Current actor/source authorization remains in the owner service and native journal security, not in the low-level JDBC port.
Disabled or deleted original actors still need an explicitly designed administrative reconciliation flow. No fabricated session or broad archive permission substitutes for that flow. Broad post-filtered listings can contain short pages; source-pinned native recovery navigation is the supported user entry point.
## Database boundaries
Both owner tables must use the same QQQ backend. Hosts apply DDL through their migrations; runtime code never creates tables or changes server-global settings.
| Backend | Runtime requirements | Schema |
| --- | --- | --- |
| File H2 | Native `h2`, `jdbc:h2:file:`, actual configured connection, writable real transaction, live `WRITE_DELAY=0` | `UploadJournalTable.h2Script()` |
| MySQL 8.4 | Native `mysql`, actual MySQL 8.4 product/version, exact configured URL target and catalog, writable real transaction, both owner tables InnoDB, fixed owner table mappings, journal `utf8mb4_0900_bin` columns/table, single-column unique locator/intake indexes, synchronous commit settings | `UploadJournalTable.mysqlScript()` |
| PostgreSQL 17 | Native `postgres` (vendor and backend type), actual PostgreSQL 17 product/version, exact configured `jdbc:postgresql://host/catalog` target and catalog, writable real transaction, not in recovery, `synchronous_commit` `on` or `remote_apply`, `fsync=on`, `full_page_writes=on` (see the Aurora rule below), session `TimeZone` UTC, `lock_timeout` from 1 to 5000 ms, READ COMMITTED, both owner tables permanent ordinary tables with fixed mappings, no non-deterministic collation, complete unique indexes on `locator_hash` and `intake_key` | `UploadJournalTable.postgresScript()` |
| Amazon Aurora PostgreSQL 17 | Every PostgreSQL 17 requirement above, except that `full_page_writes=off` is accepted | `UploadJournalTable.postgresScript()` |
| Aurora MySQL, MariaDB, other versions/providers | Refused; no inferred compatibility or durability claim | None |
Aurora rule: stock PostgreSQL must report `full_page_writes=on`, because recovery repairs a partly written page from the full page image in the write-ahead log. Amazon Aurora PostgreSQL reports `full_page_writes=off` and the parameter is not in its cluster parameter group, so a deployment cannot turn it on; its storage layer does not write torn pages, so the protection is not needed there. The qualification therefore accepts `off` only when the same settings query reports `to_regprocedure('aurora_version()') IS NOT NULL`, which is true on Aurora, false on stock PostgreSQL 17 and never raises. Nothing else is relaxed on Aurora: `synchronous_commit`, `fsync`, recovery state, isolation, time zone and lock timeout are checked exactly as on stock PostgreSQL. The detection is a name lookup, so a function named `aurora_version()` on the session's search path of a stock server is reported as Aurora; creating one is a schema change inside the host's trust boundary, like the settings themselves.
The MySQL adapter resolves explicit or native-generated URLs through QQQ `ConnectionManager`. It accepts a single ordinary `jdbc:mysql://host:port/catalog` target with a simple catalog identifier. Proxy/load-balancing URLs, alternate catalogs or table remapping fail closed. Error messages do not echo URLs or credentials. Driver options remain host-owned; TLS and credentials are not changed by the journal.
Each MySQL operation checks `innodb_flush_log_at_trx_commit=1`, and `sync_binlog=1` when binary logging is enabled; a read-only server or session is refused. It also requires `innodb_doublewrite` to be `ON` or `DETECT_AND_RECOVER`; [detect-only mode does not retain recoverable page copies](https://dev.mysql.com/doc/refman/8.4/en/innodb-doublewrite-buffer.html). These settings follow the [MySQL 8.4 InnoDB durability documentation](https://dev.mysql.com/doc/refman/8.4/en/innodb-parameters.html#sysvar_innodb_flush_log_at_trx_commit) and [binary-log durability documentation](https://dev.mysql.com/doc/refman/8.4/en/replication-options-binary-log.html#sysvar_sync_binlog). Hardware that misreports flush completion can still lose data; replication/failover, backups, restore and disk guarantees remain deployment qualifications. Runtime settings are checked at transaction entry; an administrator changing server settings concurrently is outside the application trust boundary.
The MySQL journal uses millisecond timestamps with explicit UTC JDBC calendars, InnoDB, enforced CHECK constraints and binary NO PAD string comparisons. Its live session must report UTC (+00:00/UTC, or SYSTEM with a UTC system zone); the journal does not silently change a pooled session timezone. Hosts should configure the native JDBC driver timezone consistently for generic QQQ date fields as well. The host must configure each journal connection with `innodb_lock_wait_timeout` between 1 and 5 seconds, through its backend JDBC sessionVariables/pool setup (or a dedicated database default). Qualification reads and preserves that session setting; it never changes it or other pooled-session/global settings. Each statement also has a ten-second query timeout. Duplicate error 1062 is distinguished from unrelated integrity failures; deadlock/lock-timeout failures remain typed BUSY responses, never fabricated success.
## Reproducible tests
Use the repository Maven wrapper and the same revision/QQQ dependency overrides as the integrated build. The default suite remains Docker-independent; explicitly enable the provider fixture:
```powershell
.\mvnw.cmd '-Drevision=0.1.0-record-files-SNAPSHOT' '-DmysqlJournalTests=true' '-Dtest=MySqlUploadJournalTest,UploadJournalTest' test
```
`MySqlUploadJournalTest` creates only its own Testcontainers MySQL 8.4.13 container, random synthetic credentials and private anonymous data volume, and destroys that container on completion. It does not use existing host services. The image is pinned to `mysql@sha256:a0225d1ea6631c8bc2385dc4c2faba17bbf1834ecf369e1659a9a382bd8d5dd7`. Credentials are not printed by the fixture.
The eleven provider cases cover acknowledged prepare surviving a real database process restart, attachment/checkpoint rollback and completion replay, simultaneous prepare/lease fencing, wrong backend/catalog/mapping/ambient/Aurora refusal, actual non-durable server settings and MyISAM refusal, binary identifier comparisons and native write denial, actual lock-timeout/BUSY behavior, column/index drift and immutable conflict, and lease instants across differing JVM default zones, preservation/refusal of host session lock-timeout settings, and refusal of detect-only doublewrite/read-only sessions without changing their settings. The attachment schema in this suite is deliberately a minimal transaction fixture; this is not an end-to-end MySQL object-storage or tenant/archive qualification. Record Files owns its separate provider integration tests.
First provider run compiled but failed its restart fixture because Docker reassigned the dynamic port and Testcontainers retained a cached mapping; two subsequent setup errors followed that stale endpoint. The fixture now refreshes the inspected binding after restart and uses bounded JDBC readiness checks. Docker wire logs are suppressed before container creation. Expanded snapshot verification: **20 tests passed (9 MySQL, 11 H2), zero failures/errors/skips**, `target/mysql-journal-focused-verification.log`, completed 2026-10-31 00:01:44. The tenth provider case and full 214-test owner quality gates subsequently passed; the corrected development-GAV artifact SHA-256 is `72D6E36E77B92BE3ECFE9BE7042D292C76B308AA05A9E43BD490B229BD1A75E2`. The eleventh provider regression and doublewrite/read-only delta await fresh verification.
+76
View File
@@ -0,0 +1,76 @@
# Selectable collaboration fields: published feature snapshot
Tracks [QQQ #1196](https://github.com/QRun-IO/qqq/issues/1196).
This source branch requires the new core virtual-field value-provider API. Its
default BOM pin is now the published
`4.1.0-virtual-field-values-1196-cc3981bb3-SNAPSHOT`. Core publisher job 9716
passed from commit `cc3981bb3c95e8f2dd7ef2a02e2233db7d0e2a79`; an independent
consumer with an initially empty Maven repository downloaded its BOM and core
and compiled the provider interface on October 26, 2026. Both artifacts resolve
to timestamp `20261026.130631-1` from Sonatype's snapshot repository.
Published artifact SHA-256 receipts:
| Artifact | SHA-256 |
| --- | --- |
| BOM POM | `d68cf837d7e07513c4e70977829f99e6236cb3e89f8af85fc75239d5b52564a5` |
| Core POM | `d89d491aaa9312051ec462e1cf165b948ee3e88c9aec2721ff25a00e956c431b` |
| Core JAR | `ae6dbeba8d0e4bc55f2714303ee3482dbf6a0cd34c03663c6009d8fe2c1da54d` |
The collaboration follow-up is published as
`0.1.0-field-values-1196-ecff347-SNAPSHOT`, timestamp
`20261026.133203-1`. Publisher job 168 passed from commit
`ecff347efe2bb84949b8452a3383e3fdeb4bcede`, using signed tag
`publish-values1196-20261026-ecff347`.
| Collaboration artifact | SHA-256 |
| --- | --- |
| POM | `cc12f59cbdfd4bc53ba545a9f2b6d7c14a316034b107ef5541ddc48df45fe6af` |
| JAR | `284fa370504dd82cee8c87d2b8f5c0cecba8c4b15b32a4f9d3d62a0a062e549d` |
On October 26, an independent Java 21 consumer downloaded this pair into an
initially empty Maven repository and compiled both `TagValueProvider` and
`CustomFieldValueProvider` as `VirtualFieldValueProviderInterface` instances.
Maven recorded Sonatype `central-portal-snapshots` as the artifact source. The
downloaded collaboration POM pins the core version above; the independently
downloaded core JAR matches the receipt above. This checks published dependency
resolution and API compatibility; runtime behavior is covered by the 198-test
verification below, not by this compilation probe.
This snapshot does not include the separate attachment, My Work or presence
recovery follow-up PRs. It has not been deployed to Kof22.
Earlier local verification used core artifacts installed as
`4.1.0-values1196-SNAPSHOT`, with
`./mvnw.cmd -Dqqq.snapshot.version=4.1.0-values1196-SNAPSHOT verify`.
That version exists only in the local Maven repository; it is not an adoption pin.
## Behavior
- `collaborationTags` returns sorted, distinct approved labels from global or
matching-table vocabulary. Merged tags and deleted assignments are excluded.
The picker and filters retain their tag IDs; exports contain one Tags column.
- EAV custom fields return typed existing values and ordinary possible-value
display labels. Missing values are null internally and omitted from HTTP value
maps. This change does not enable EAV writes.
- Providers process batches and use the originating transaction. Parent record
authorization and field visibility remain enforced by core. Existing host
post-query customizers remain installed.
- SQL DISTINCT on provider values is explicitly unsupported. Backend primary-key
identity is required; joined heavy primary keys must be fetched explicitly.
## Release sequence
1. Publish and verify the coordinated core artifact containing the provider API (done).
2. Replace this branch's core BOM pin with that published version (done).
3. Run full collaboration verification against the published prerequisite
(done October 26: 198 tests, no failures/errors/skips; packaging and coverage
passed, including HTTP, PostgreSQL transaction and Redis tests).
4. Publish the collaboration snapshot and verify an independent consumer using
published artifacts (done; exact pair recorded in CANDIDATE-RELEASE.md).
5. Update Kof22 Website-Backend adoption work and qualify it in the selected
isolated staging environment before requesting production deployment.
The published pair contains this implementation. Kof22 adoption, a compatible
packaged frontend and staging qualification remain separate release steps.
+56
View File
@@ -0,0 +1,56 @@
# Recovery cost diagnostic — October 26, 2026
One Java 21 process, one Redis connection, disposable loopback Redis 7.4-alpine,
1,000 issued leases, ten viewers per record, no TLS or HTTP. The probe keeps
all leases alive through the real 45-second recovery window and verifies that
the final table snapshot contains all 1,000 viewers. Each phase samples 200
renewals with increasing sequence numbers. This is not a capacity or SLA claim.
| Renewal latency (ms) | During recovery | Ready |
|---|---:|---:|
| p50 | 3.6779 | 0.5134 |
| p95 | 4.1956 | 0.7226 |
| p99 | 4.5447 | 0.9122 |
| Maximum | 6.8271 | 1.4886 |
Population took 4,640.799 ms. The run passed all correctness checks. This first
implementation scanned touched projections on every request during recovery.
After changing table cleanup to occur atomically only when creating its recovery
generation (individual record cleanup remains lazy), the same diagnostic passed:
| Renewal latency (ms) | Optimized recovery | Ready |
|---|---:|---:|
| p50 | 0.6375 | 0.5029 |
| p95 | 0.8150 | 0.5762 |
| p99 | 0.9406 | 0.7500 |
| Maximum | 4.2280 | 1.2251 |
Optimized population took 1,634.486 ms. All 1,000 leases survived. The focused
recovery/store suite also passed 13 tests, including guard loss with previously
unvisited record rooms. Full optimized verification passed 184 tests with no failures/errors/skips and coverage gates passed. These are
individual local runs and need further qualification before claiming supported
production capacity. They do not isolate every source
of latency, compare against an earlier release, or measure sustained concurrent
load, saturation, TLS, HTTP or browser propagation.
At 10,000 entries the optimized diagnostic also passed, retaining all leases.
Population took 10,687.2105 ms; the same 200-sample phases measured:
| Renewal latency (ms) | Optimized recovery, 10,000 entries | Ready |
|---|---:|---:|
| p50 | 0.4978 | 0.5341 |
| p95 | 0.5914 | 0.7108 |
| p99 | 0.7255 | 1.2964 |
| Maximum | 0.7544 | 1.8799 |
Do not infer that larger tables are faster from these separate local runs.
JIT warmup, scheduling and concurrent machine activity were not controlled.
The useful result is that the optimization avoids a repeated whole-table scan
during recovery and both populated-table runs preserved lease correctness.
The explicit `RedisPresenceRecoveryCostProbe` test-source main class is not
automatically run by the unit suite and imposes no machine-dependent CI gate.
After test compilation, run it on the test classpath with argument `1000`.
It fails if the real recovery window ends before the cold measurement, rather
than silently reporting ready-state latency as recovery latency.
@@ -0,0 +1,52 @@
// Real browser recovery against the disposable two-JVM harness. No API mocks.
import { createRequire } from 'node:module'
import { execFileSync } from 'node:child_process'
import path from 'node:path'
const [frontend, first, second, container, output, profile = 'chromium'] = process.argv.slice(2)
const require = createRequire(path.join(frontend, 'package.json'))
const { chromium, firefox, webkit, devices, expect } = require('@playwright/test')
const profiles = {
chromium: [chromium, devices['Desktop Chrome']],
firefox: [firefox, devices['Desktop Firefox']],
webkit: [webkit, devices['Desktop Safari']],
mobile: [chromium, devices['Pixel 7']],
tablet: [webkit, devices['iPad (gen 7)']],
}
if (!profiles[profile]) throw new Error(`Unknown browser profile: ${profile}`)
const [engine, device] = profiles[profile]
const browser = await engine.launch({ headless: true })
const contexts = []
try {
const pages = []
for (const [port, user] of [[first, 'alice'], [second, 'bob']]) {
const context = await browser.newContext(device)
contexts.push(context)
await context.tracing.start({ screenshots: true, snapshots: true })
const page = await context.newPage()
await page.goto(`http://127.0.0.1:${port}/?createMockSession=true&user=${user}`)
// The root export redirects after hydration; finish it before record navigation.
await page.waitForURL(url => /^\/app\/?$/.test(url.pathname), { waitUntil: 'domcontentloaded' })
await page.goto(`http://127.0.0.1:${port}/app/person/1`)
pages.push(page)
}
const unavailable = page => page.getByText('Presence temporarily unavailable', { exact: true })
await expect(pages[0].locator('[data-qqq-id="presence-bob"]')).toBeVisible({ timeout: 90000 })
await expect(pages[1].locator('[data-qqq-id="presence-alice"]')).toBeVisible({ timeout: 90000 })
execFileSync('docker', ['exec', container, 'redis-cli', 'FLUSHDB'])
for (const page of pages) await expect(unavailable(page)).toBeVisible({ timeout: 35000 })
await expect(pages[0].locator('[data-qqq-id="presence-bob"]')).toHaveCount(0)
await expect(pages[1].locator('[data-qqq-id="presence-alice"]')).toHaveCount(0)
await expect(pages[0].locator('[data-qqq-id="presence-bob"]')).toBeVisible({ timeout: 90000 })
await expect(pages[1].locator('[data-qqq-id="presence-alice"]')).toBeVisible({ timeout: 90000 })
for (const [index, page] of pages.entries()) {
await expect(unavailable(page)).toHaveCount(0)
await page.screenshot({ path: path.join(output, `recovered-${index}.png`) })
}
console.log(`PASS: ${profile} shows recovery unavailable, clears stale people, and rejoins both JVMs without reload.`)
} finally {
for (const [index, context] of contexts.entries()) {
await context.tracing.stop({ path: path.join(output, `browser-${index}.zip`) })
}
await browser.close()
}
@@ -0,0 +1,172 @@
"""Exercise Redis loss through two independent SharedPresenceDevServer JVMs.
Run after test-compile and dependency:build-classpath with test scope and output
target/recovery-process-classpath.txt. Uses only disposable loopback fixtures.
"""
import argparse
import hashlib
import json
import os
from pathlib import Path
import socket
import shutil
import subprocess
import time
import urllib.error
import urllib.request
import uuid
def free_port():
"""Reserve a candidate loopback port; startup detects any bind race."""
with socket.socket() as listener:
listener.bind(("127.0.0.1", 0))
return listener.getsockname()[1]
def request(port, user, method, path, body=None):
"""Use the fixture's mock session through the actual HTTP boundary."""
separator = "&" if "?" in path else "?"
url = f"http://127.0.0.1:{port}/qqq/v1/qbit/collaboration{path}{separator}user={user}"
data = None if body is None else json.dumps(body).encode()
req = urllib.request.Request(url, data=data, method=method, headers={
"Cookie": f"sessionUUID={user}-recovery-processes",
"Content-Type": "application/json",
})
try:
response = urllib.request.urlopen(req, timeout=10)
except urllib.error.HTTPError as error:
response = error
with response:
return response.status, json.load(response)
def require(status, result):
"""Fail with fixture-only response evidence when transport differs."""
assert result[0] == status, result
return result[1]
def settle(ports, leases):
"""Renew on the opposite node until both nodes report both people."""
deadline = time.monotonic() + 60
sequence = 0
while time.monotonic() < deadline:
sequence += 1
for index, (user, lease) in enumerate(leases.items()):
result = request(ports[1 - index], user, "POST", "/presence/leases/renew", {
"connectionId": lease, "table": "person", "pk": "1",
"mode": "VIEWING", "sequence": sequence,
})
assert result[0] in (200, 503), result
if result[0] == 503:
assert "recovering" in result[1]["message"], result
snapshots = [request(port, "alice", "GET", "/presence?table=person&pk=1") for port in ports]
if all(status == 200 for status, _ in snapshots):
assert all(len(body["viewers"]) == 2 for _, body in snapshots), snapshots
return
assert all(status == 503 for status, _ in snapshots), snapshots
time.sleep(1)
raise AssertionError("Two-process recovery exceeded 60 seconds")
def main():
"""Own and clean up the exact child JVMs and disposable Redis container."""
parser = argparse.ArgumentParser()
parser.add_argument("--java", required=True)
parser.add_argument("--repo", type=Path, required=True)
parser.add_argument("--frontend-repo", type=Path)
parser.add_argument("--frontend-jar", type=Path, help="Serve this packaged frontend instead of a local export")
parser.add_argument("--backend-jar", type=Path, help="Use this backend artifact instead of target/classes")
parser.add_argument("--browser-profile", choices=("chromium", "firefox", "webkit", "mobile", "tablet"), default="chromium")
args = parser.parse_args()
repo = args.repo.resolve()
output = repo / "target" / ("recovery-processes-" + uuid.uuid4().hex[:10])
output.mkdir()
if args.frontend_jar and not args.frontend_repo:
parser.error("frontend-jar requires frontend-repo for browser tooling")
dependencies = (repo / "target/recovery-process-classpath.txt").read_text().strip().split(os.pathsep)
assert not any(Path(item).name.startswith(("qbit-collaboration-", "qqq-frontend-next-")) for item in dependencies)
backend = args.backend_jar.resolve(strict=True) if args.backend_jar else repo / "target/classes"
classpath = os.pathsep.join([str(repo / "target/test-classes"), str(backend), *dependencies])
evidence = {"status": "FAIL", "browser_profile": args.browser_profile, "cleanup": False}
if args.backend_jar:
evidence["backend_jar_sha256"] = hashlib.sha256(backend.read_bytes()).hexdigest()
if args.frontend_jar:
frontend = args.frontend_jar.resolve(strict=True)
evidence["frontend_jar_sha256"] = hashlib.sha256(frontend.read_bytes()).hexdigest()
classpath = str(frontend) + os.pathsep + classpath
elif args.frontend_repo:
shutil.copytree(args.frontend_repo.resolve() / "out", output / "next-dashboard")
classpath = str(output) + os.pathsep + classpath
container = "qqq-recovery-" + uuid.uuid4().hex[:12]
processes, logs = [], []
try:
subprocess.run(["docker", "run", "-d", "--name", container, "-p", "127.0.0.1::6379", "redis:7.4-alpine"], check=True, capture_output=True)
binding = subprocess.check_output(["docker", "port", container, "6379"], text=True).strip()
redis_port = int(binding.rsplit(":", 1)[1])
ports = [free_port(), free_port()]
for port in ports:
argfile = output / f"node-{port}.args"
argfile.write_text('-Dcollab.dev.frontend=' + str(bool(args.frontend_repo)).lower() + '\n-cp\n"' + classpath.replace("\\", "/") + '"\n'
'com.kingsrook.qbits.collaboration.devserver.SharedPresenceDevServer\n'
f'{port}\n{redis_port}\n', encoding="utf-8")
log = (output / f"node-{port}.log").open("w")
logs.append(log)
processes.append(subprocess.Popen([args.java, "@" + str(argfile)], cwd=repo,
stdout=log, stderr=subprocess.STDOUT,
creationflags=getattr(subprocess, "CREATE_NO_WINDOW", 0)))
deadline = time.monotonic() + 60
for port, process in zip(ports, processes):
while f"SHARED_PRESENCE_READY {port}" not in (output / f"node-{port}.log").read_text(errors="replace"):
assert process.poll() is None, f"Node exited; see {output}"
assert time.monotonic() < deadline, f"Startup timed out; see {output}"
time.sleep(0.2)
if args.frontend_repo:
subprocess.run(["node", str(repo / "docs/acceptance/presence_recovery_browser.mjs"),
str(args.frontend_repo.resolve()), str(ports[0]), str(ports[1]), container, str(output), args.browser_profile], check=True)
evidence["status"] = "PASS"
return
def join():
"""Acquire each person's new lease on a different node."""
return {user: require(200, request(port, user, "POST", "/presence/leases", {
"table": "person", "pk": "1"}))["connectionId"] for port, user in zip(ports, ("alice", "bob"))}
original = join()
settle(ports, original)
subprocess.run(["docker", "exec", container, "redis-cli", "FLUSHDB"], check=True, capture_output=True)
for port in ports:
require(503, request(port, "alice", "GET", "/presence?table=person&pk=1"))
for user, lease in original.items():
error = require(409, request(port, user, "POST", "/presence/leases/renew", {
"connectionId": lease, "table": "person", "pk": "1", "mode": "VIEWING", "sequence": 1000}))
assert error["code"] == "PRESENCE_LEASE_EXPIRED", error
replacement = join()
assert all(replacement[user] != original[user] for user in original)
settle(ports, replacement)
print(f"PASS: two independent JVMs recovered two viewers after Redis loss. Logs: {output}")
evidence["status"] = "PASS"
except Exception as error:
evidence["failure"] = str(error)
raise
finally:
for process in processes:
process.terminate()
for process in processes:
try:
process.wait(timeout=10)
except subprocess.TimeoutExpired:
process.kill()
process.wait(timeout=10)
for log in logs:
log.close()
cleanup = subprocess.run(["docker", "rm", "-f", container], capture_output=True, check=False)
evidence["cleanup"] = cleanup.returncode == 0 and all(process.poll() is not None for process in processes)
(output / "result.json").write_text(json.dumps(evidence, indent=2), encoding="utf-8")
print(output, flush=True)
if evidence["status"] == "PASS" and not evidence["cleanup"]:
raise RuntimeError("Recovery passed but owned fixture cleanup failed")
if __name__ == "__main__":
main()
@@ -0,0 +1,166 @@
"""Restore a disposable Redis image while both fixture JVMs are stopped.
This verifies the controlled-restore protocol, not production orchestration.
Requires the same compiled test classpath as presence_recovery_processes.py.
"""
import argparse
import hashlib
import json
import os
from pathlib import Path
import socket
import subprocess
import time
import uuid
from presence_recovery_processes import free_port, request, require, settle
def main():
"""Own a Redis fixture and two generations of independently running nodes."""
parser = argparse.ArgumentParser()
parser.add_argument("--java", required=True)
parser.add_argument("--repo", type=Path, required=True)
parser.add_argument("--backend-jar", type=Path, help="Verify this artifact without production target/classes")
args = parser.parse_args()
repo = args.repo.resolve()
output = repo / "target" / ("restore-processes-" + uuid.uuid4().hex[:10])
output.mkdir()
dependencies = (repo / "target/recovery-process-classpath.txt").read_text().strip().split(os.pathsep)
assert all(Path(item).is_file() and item.endswith('.jar') for item in dependencies)
assert not any(Path(item).name.startswith("qbit-collaboration-") for item in dependencies)
backend = args.backend_jar.resolve(strict=True) if args.backend_jar else repo / "target/classes"
classpath = os.pathsep.join([str(repo / "target/test-classes"), str(backend), *dependencies])
evidence = {"status": "FAIL", "cleanup": False, "checks": []}
if args.backend_jar:
evidence["backend_jar_sha256"] = hashlib.sha256(backend.read_bytes()).hexdigest()
container = "qqq-restore-" + uuid.uuid4().hex[:12]
container_created = False
processes, logs = [], []
ports = [free_port(), free_port()]
def redis(*command):
"""Run commands only against this owned, disposable Redis container."""
return subprocess.check_output(["docker", "exec", container, "redis-cli", *command], text=True).strip()
def stop_nodes():
"""Require process termination and closed old listeners before restore."""
for process in processes:
if process.poll() is None:
process.terminate()
for process in processes:
try:
process.wait(timeout=10)
except subprocess.TimeoutExpired:
process.kill()
process.wait(timeout=10)
processes.clear()
for port in ports:
with socket.socket() as connection:
connection.settimeout(1)
assert connection.connect_ex(("127.0.0.1", port)) != 0, "Old listener remains open"
def start_nodes(generation, redis_port):
"""Supply the shared generation outside the Redis image, on both JVMs."""
for port in ports:
argfile = output / f"{generation}-{port}.args"
argfile.write_text(f'-Dcollab.dev.recoveryGeneration={generation}\n-cp\n"'
+ classpath.replace("\\", "/") + '"\n'
'com.kingsrook.qbits.collaboration.devserver.SharedPresenceDevServer\n'
f'{port}\n{redis_port}\n', encoding="utf-8")
logfile = output / f"{generation}-{port}.log"
log = logfile.open("w")
logs.append(log)
processes.append(subprocess.Popen([args.java, "@" + str(argfile)], cwd=repo,
stdout=log, stderr=subprocess.STDOUT,
creationflags=getattr(subprocess, "CREATE_NO_WINDOW", 0)))
deadline = time.monotonic() + 60
for port, process in zip(ports, processes):
logfile = output / f"{generation}-{port}.log"
while f"SHARED_PRESENCE_READY {port}" not in logfile.read_text(errors="replace"):
assert process.poll() is None, f"Node exited; see {output}"
assert time.monotonic() < deadline, f"Startup timed out; see {output}"
time.sleep(0.2)
def join():
"""Issue each person's lease on a separate node."""
return {user: require(200, request(port, user, "POST", "/presence/leases", {
"table": "person", "pk": "1"}))["connectionId"] for port, user in zip(ports, ("alice", "bob"))}
def reject_old(leases, sequence):
"""Restored lease authority must fail on both new-generation nodes."""
for port in ports:
for user, lease in leases.items():
error = require(409, request(port, user, "POST", "/presence/leases/renew", {
"connectionId": lease, "table": "person", "pk": "1",
"mode": "VIEWING", "sequence": sequence}))
assert error["code"] == "PRESENCE_LEASE_EXPIRED", error
try:
# DEBUG RELOAD is enabled only on this disposable loopback fixture.
subprocess.run(["docker", "run", "-d", "--name", container, "-p", "127.0.0.1::6379",
"redis:7.4-alpine", "redis-server", "--enable-debug-command", "yes", "--save", ""],
check=True, capture_output=True)
container_created = True
binding = subprocess.check_output(["docker", "port", container, "6379"], text=True).strip()
redis_port = int(binding.rsplit(":", 1)[1])
start_nodes("before-restore", redis_port)
original = join()
settle(ports, original)
last_renewal = time.monotonic()
for port, (user, lease) in zip(ports, original.items()):
require(200, request(port, user, "POST", "/presence/leases/renew", {
"connectionId": lease, "table": "person", "pk": "1",
"mode": "VIEWING", "sequence": 100}))
assert redis("SAVE") == "OK"
before = int(redis("DBSIZE"))
assert before > 0
stop_nodes()
evidence["checks"].append("old JVMs stopped and listeners closed before restore")
assert redis("FLUSHDB") == "OK"
assert redis("DBSIZE") == "0"
assert redis("DEBUG", "RELOAD", "NOSAVE") == "OK"
assert int(redis("DBSIZE")) > 0, "Saved Redis image was not restored"
evidence["checks"].append("nonempty saved Redis image restored")
start_nodes("restored-" + uuid.uuid4().hex, redis_port)
reject_old(original, 1000)
# Avoid a false pass caused only by the old 45-second lease expiring.
evidence["old_lease_rejection_seconds"] = time.monotonic() - last_renewal
assert evidence["old_lease_rejection_seconds"] < 40, "Restore exceeded the live-lease proof window"
evidence["checks"].append("both nodes reject old leases before normal expiry")
for port in ports:
result = require(503, request(port, "alice", "GET", "/presence?table=person&pk=1"))
assert "recovering" in result["message"], result
replacement = join()
assert all(replacement[user] != original[user] for user in original)
settle(ports, replacement)
reject_old(original, 1001)
evidence["checks"].append("both nodes recover with fresh viewers and still reject old leases")
if args.backend_jar:
assert hashlib.sha256(backend.read_bytes()).hexdigest() == evidence["backend_jar_sha256"]
evidence["status"] = "PASS"
print(f"PASS: restored Redis image cannot authorize old leases on either new-generation JVM. Logs: {output}")
except Exception as error:
evidence["failure"] = str(error)
raise
finally:
try:
try:
stop_nodes()
finally:
for log in logs:
log.close()
if container_created:
subprocess.run(["docker", "rm", "-f", container], capture_output=True, check=True)
evidence["cleanup"] = True
except Exception as error:
evidence["status"] = "FAIL"
evidence["cleanup_failure"] = str(error)
raise
finally:
(output / "result.json").write_text(json.dumps(evidence, indent=2), encoding="utf-8")
if __name__ == "__main__":
main()
+264
View File
@@ -0,0 +1,264 @@
# Independent-node presence workload
This fixture targets QQQ #1302/#1291. It runs 2–8 independent JVM HTTP servers
against an owned loopback Redis container. Every node serves the same configured
mock identities across 2–8 tenant scopes. The scope comes from the authenticated
fixture identity, never a supplied tenant parameter. Records are independently
seeded in memory; this does not qualify shared database or production IAM behavior.
Build with Java 21:
```powershell
./mvnw.cmd test-compile dependency:build-classpath '-Dmdep.includeScope=test' '-Dmdep.outputFile=target/capacity-classpath.txt'
python docs/performance/presence_nodes.py --java /path/to/java --nodes 3 --tenants 2 --users 6 --tabs 4 --rounds 100
```
Four tabs per user means two duplicate tabs on each of two records. Every round
renews each issued lease on a rotating node, with sequential per-tab sequences.
All nodes must then return exactly the expected tenant's distinct people and
exactly two editors on both records. Any mismatch or request failure fails the
run. Requests use bounded worker concurrency and ten-second timeouts. Each node
has eight pooled Redis connections, a 500 ms pool wait and a 384 MiB heap ceiling.
Results and per-node logs are retained under `target/capacity-nodes-<id>`.
The result records parameters, node PIDs, elapsed time, sample counts and
nearest-rank HTTP p50/p95/p99/max. Renewal throughput includes snapshot checks
and is not maximum server throughput. All owned JVMs and the Redis container are
removed in `finally`; the receipt reports cleanup separately from test success.
Each fixture JVM emits resource samples every 100 ms using Java management APIs
and the Redis QBit's existing pool metrics. The receipt summarizes sampled heap
used/committed bytes, active/idle/waiting pool counts and the configured maximum.
CPU core equivalents are process CPU time divided by observed wall time; they
can exceed one for a multithreaded JVM. Observation begins after server startup
and includes lease setup and workload execution. Unsupported CPU counters are
reported as null. Linux fixtures also read process `VmRSS` from `/proc/self/status`
every 100 ms and record `residentBytes_sampled_max`; unsupported platforms report
null. Linux-container runs require resident-memory samples. Heap is not total
resident memory, and process RSS is not the container's complete cgroup footprint.
Sampled peaks may miss
short bursts and must not be represented as exact maxima. Raw samples remain in
node logs; missing node samples or a sampled pool-bound violation fail the run.
The final Redis INFO capture records `used_memory`, lifetime `used_memory_peak`,
and current `used_memory_rss`, plus process user/system CPU seconds since Redis
startup. Redis RSS is an end-of-run observation, not a sampled peak. These fresh,
owned Redis instances include fixture setup and diagnostics in their lifetime;
do not interpret those CPU values as workload-only measurements or add independent
process peaks and call the sum a measured simultaneous maximum. Missing Redis
memory/CPU diagnostics fail acceptance while cleanup still runs.
The [100-user resource run](results/linux-100-users-resources.json) used three
Linux JVMs, two tenants with 50 people each, four leases per person, 16 request
workers and 200 rounds. All 80,000 renewals and 2,400 exact-membership snapshots
passed in 102.568 workload seconds. Renewal p50/p95/p99/max were
11.326/21.765/25.881/1,007.404 ms; snapshot p99 was 19.424 ms. The one-second
renewal maximum is retained, not discarded. Cleanup passed.
Sampled JVM RSS peaks were 662,786,048 / 662,188,032 / 703,479,808 bytes
(approximately 632–671 MiB), while used-heap peaks were about 171–177 MiB.
CPU observations were 1.73–1.83 core equivalents per JVM; sampled pool-active
peaks were 7/7/6 of eight with no sampled waiters. Redis peak allocated memory
was 2,579,896 bytes; final RSS was 8,777,728 bytes. Redis process CPU since
startup was 23.738 system plus 35.526 user seconds. These measurements include
fixture overhead, omit browser load, and do not establish deployment limits.
Initial smoke run: three JVMs, two tenants, six users each, four tabs per user,
three rounds: 144 renewals and 36 snapshots passed with tenant isolation,
duplicate-tab deduplication and two editors retained. Cleanup passed. This short
run validates the harness, not sustained capacity or operational thresholds.
A second run with 12 users per tenant and 100 rounds passed 9,600 renewals and
1,200 snapshots in 23.110 seconds. Renewal p50/p95/p99 were 7.246/16.454/21.853 ms;
snapshot p50/p95/p99 were 12.908/15.776/18.239 ms. Every round checked exact
tenant membership, deduplication and two editors on both records at every node.
Cleanup passed. These are measurements from one local Windows/Docker run;
they do not establish deployment limits or browser propagation latency.
## Sustained-run limitation under investigation
The 1,000-round workload has intermittently failed with HTTP 503 on lease
renewal. Four runs failed; one completed 96,000 renewals and 12,000 snapshots.
The passing run does not resolve the failures. Temporary local diagnostics on
the third failure identified `JedisConnectionException` caused by
`SocketTimeoutException` while reading the `EVALSHA` response in
`RedisPresenceStore.mutate`. The configured socket timeout remained two seconds.
The temporary production-file diagnostics were removed after identifying this
path. No timeout increase, hidden request retry or production fix is included.
Resource samples did not show pool waiters, but cannot rule out short bursts.
Redis slow-command timings and latency events are now captured before fixture
cleanup, alongside Redis CPU, memory and command counters, to distinguish server
execution cost from transport or scheduling delays. Slow-command summaries omit
command arguments. These diagnostics belong only to the disposable fixture.
Until the cause is established and qualified, this workload establishes no
supported sustained throughput or production alert thresholds.
The timing-instrumented Windows-JVM run failed at round 480, with no commands
at or above the configured one-millisecond Redis slow-log threshold and no
latency-monitor events. Redis reported about 93 microseconds per `EVALSHA` on
average. This weakens a slow-Lua explanation but does not identify the delayed
component or exclude time spent outside command execution.
## Linux comparison
`--node-image eclipse-temurin:21-jre` optionally runs each JVM in its own Linux
container on an owned Docker network shared with Redis. The HTTP driver remains
on the host. Only loopback HTTP ports are published. Compiled fixture classes
and copied dependency JARs are mounted read-only; Redis remains disposable.
The existing loopback defaults remain unchanged when this option is absent.
Container mode records Docker client PIDs separately from container identities.
It removes the Windows-JVM-to-published-Redis-port path, but also changes the JVM
platform and scheduling environment; it is a diagnostic comparison, not proof
of a specific operating-system or Docker defect. Cleanup includes every owned
node container and the owned network.
`--churn-every 20` departs all tabs belonging to the last user in each tenant
every 20 rounds. Every departed lease must reject a later renewal with HTTP 409
and `PRESENCE_LEASE_EXPIRED` on another node. All surviving nodes must show that
user absent on both records, then show the original exact membership after
fresh, distinct leases are acquired and activated. Only this explicit rejoin
path acquires new leases; the node-crash path preserves existing ones.
`--pause-redis` adds a controlled failure phase after the ordinary rounds. It
freezes only the owned Redis container, sends up to 64 concurrent renewal
requests, and requires HTTP 503 within an eight-second fixture bound. Snapshots
must also fail explicitly rather than return an empty room. Redis is unpaused
in `finally`; once reachable, higher-sequence renewals must recover the same
leases and exact membership. The test requires observed pool waiters and rejects
sampled active counts above the configured maximum. It does not retry failed
requests invisibly or raise production timeouts.
The [Linux Redis-stall receipt](results/linux-redis-stall.json) passed with three
nodes, two tenants, 24 users and 96 tab leases. All 96 stalled renewals and all
three node snapshots returned the expected unavailable response. Existing
leases recovered after Redis resumed, and cleanup passed. This proves bounded
failure and recovery for an injected stall; it does not explain the intermittent
Windows timeout or establish a saturation throughput threshold.
```powershell
python docs/performance/presence_nodes.py --java java --node-image eclipse-temurin:21-jre --nodes 3 --tenants 2 --users 12 --tabs 4 --rounds 1000
```
The first Linux comparison completed 1,000 rounds in 250.811 seconds: 96,000
renewals and 12,000 snapshots, with exact membership/editor assertions and
cleanup passing. Renewal p95/p99 were 15.331/17.760 ms. The sampled active pool
peak was five of eight, with no sampled waiters. This one run is separate from
the unresolved Windows failures and does not establish a production SLA.
`--crash-round 50` abruptly kills the first node before that round, then sends
all existing leases through surviving nodes. It does not acquire replacement
leases or hide failed requests with retries. Exact tenant membership, duplicate
tabs and editor counts must remain correct on every surviving node. This tests
HTTP lease continuity after process loss, not browser/SSE reconnect, ingress
failover, database durability or a Redis outage.
The Linux crash run passed 100 rounds (9,600 renewals and 996 snapshots) in
25.148 seconds. The first JVM exited with code 137 at round 50; the remaining
two retained all issued leases and expected membership. Cleanup passed.
A [combined churn and node-loss run](results/linux-node-crash-churn.json) passed
100 rounds with the first node killed at round 50 and churn every 20 rounds.
Five cycles rejected all 40 departed handles and checked absence/restored
membership across both tenants and records on every surviving node. Cleanup
passed. This is bounded HTTP lifecycle qualification; browser reconnect and
missed-event behavior remain separate requirements.
Recorded receipts retain both successful and failing evidence:
- [Windows timeout with Redis diagnostics](results/windows-timeout.json)
- [Linux 1,000-round run](results/linux-1000-rounds.json)
- [Linux node crash](results/linux-node-crash.json)
Only the Java executable path is normalized in these receipts. The Linux image
used for these runs was
`eclipse-temurin@sha256:227118764cccd3c3294617061aaa93a08309c63a395eefa83e6b35032992ea99`.
## Browser propagation baseline
`browser_subscription_affinity.mjs FRONTEND_REPO BASE_URL RESULT_PATH` verifies
the current owning-node subscription contract against the two-node fixture. It
uses the connection ID from the real browser's successful subscription request,
then preserves the same session cookies while changing only the test routing
cookie. The [published-artifact receipt](results/browser-subscription-affinity.json)
records owner 200, other node 404, owner 200, with the requested channel active
on both successful responses and the unsaved draft retained. This proves why
subscription requests need owning-node routing even when presence state is shared.
It does not test the deployed Traefik cookie, HTTPS, buffering or failover.
The packaged runner accepts this script through `--affinity-script`.
`browser_presence_latency.mjs` measures Chromium field-focus changes until a
different editor sees the corresponding remote marker. Supply a caller-owned
two-node loopback fixture serving the published frontend with `alice`/`admin`
mock identities and the acceptance router's `presence_node=first/second`
affinity cookies:
```powershell
node docs/performance/browser_presence_latency.mjs /path/to/frontend http://localhost:PORT result.json 25
```
Use `presence_router.mjs FIRST_NODE_PORT SECOND_NODE_PORT ROUTER_PORT` for this
test. It binds only to loopback and adds a test-only POST endpoint that severs
first-node SSE responses. Never deploy this fault-injection router. The packaged
acceptance fixture accepts it through `--router-script` alongside `--latency-script`.
The test verifies the target marker is absent before every change, the old
marker disappears afterward, both real SSE streams opened successfully,
issued-lease writes reached distinct nodes, and unsaved drafts survive. API
responses are not mocked. The fixture owns backend/router/Redis cleanup; this
script owns and closes its browser.
After timing the normal transitions, the script takes the observer offline and
explicitly severs its existing SSE response through the fixture router. It requires
an observed stream failure, a failed offline fetch, three acknowledged sender focus
changes, and absence of the new marker while disconnected. After network restoration
it requires a new successful SSE response and the current marker with the previous
marker removed, without reload or loss of either unsaved draft. Recovery may use
the shipped reconnect and polling behavior; this does not prove event replay.
The [disconnect/recovery receipt](results/browser-reconnect.json) passed against
the public RC37 JAR: one stream severed, three missed changes, recovery in 1,858 ms,
both drafts preserved. The accompanying six-user and Redis admin checks passed;
fixture cleanup passed. This is one Chromium disconnect case without concurrent
load, not a reconnect storm, lease-expiry test, or production recovery objective.
Append `load` to the browser command (or use `--latency-load` on the packaged
fixture) to add 16 concurrent HTTP lease workers: four mock identities with four
leases each on the same record, split between both nodes. Each worker renews
sequentially with a 100 ms pause after each response. The browser observations
are spread over at least 60 seconds; every observation interval must include
completed background renewals. The workload continues during disconnect/reconnect.
Any request or lease-cleanup failure fails the test. Responses are disposed after
reading, and workers stop before the contexts close. The receipt reports completed
renewals, observed node counts, elapsed time and any errors. These are HTTP load
workers, not 16 additional browser tabs or distinct people.
The [bounded load/reconnect receipt](results/browser-load-reconnect.json) passed
9,046 renewals over 69.728 seconds, with no worker or lease-release errors.
Across 25 observed focus transitions, p50/p95/p99 were 32.384/85.391/785.495 ms;
p99 is the maximum of this small sample, not a reliable tail-latency estimate.
The observer recovered in 1,855.887 ms after missing three updates while load
continued. Both drafts survived. Six-user and Redis admin acceptance also passed
against the public RC37 JAR; fixture cleanup passed. An initial run failed only
because the new load helper used the wrong lease-release endpoint; that receipt
is retained in the local packaged fixture history, not counted as a passing run.
The [25-transition receipt](results/browser-focus-25.json) passed against the
downloaded public RC37 JAR and published backend snapshots. Focus-to-observed-DOM
p50/p95/p99 were 31.648/33.078/36.092 ms; p99 is the maximum in this small sample.
There were no page errors or failed lease responses. Six-user presence and Redis
admin browser checks also passed in the same fixture run; fixture cleanup passed.
This baseline includes browser automation, UI debounce, network, invalidation
and rendering. It is not a server-only transport measurement, an SSE-only causal
proof, an under-load result, or a production latency objective. The separate load
receipt above covers one bounded workload and disconnect scenario; larger loads,
reconnect storms and deployment-specific behavior remain unqualified.
Remaining qualification includes longer soak and larger workloads, container
resource limits, reconnect storms and the actual deployment's capacity and alert
thresholds. The receipts above cover specific bounded HTTP, browser, saturation,
churn, missed-update and process-crash cases, not every combination. The native
Windows intermittent Redis read timeout remains unresolved. Loopback HTTP latency must not be reported
as browser propagation or a production SLA. No deployed capacity threshold is
established by this fixture.
@@ -0,0 +1,145 @@
// Real UI observations against a caller-owned two-node acceptance fixture.
import { createRequire } from 'node:module';
import { resolve } from 'node:path';
import { writeFile } from 'node:fs/promises';
import { performance } from 'node:perf_hooks';
import { setTimeout as delay } from 'node:timers/promises';
import { startPresenceLoad } from './browser_presence_load.mjs';
const [frontendRepo, base, output, countValue = '25', loadMode] = process.argv.slice(2);
if (loadMode && loadMode !== 'load') throw new Error('Optional final argument must be load');
const samples = Number(countValue);
if (!frontendRepo || !base || !output || !Number.isInteger(samples) || samples < 5 || samples > 100) {
throw new Error('Expected frontend repo, loopback base URL, result path and 5..100 samples');
}
if (!['localhost', '127.0.0.1'].includes(new URL(base).hostname)) {
throw new Error('Only the disposable loopback fixture is supported');
}
const require = createRequire(resolve(frontendRepo, 'package.json'));
const { chromium, expect } = require('@playwright/test');
const result = { status: 'FAIL', samples: [], streams: [], requests: [], errors: [],
scope: 'Chromium focus-to-remote-DOM observation, including automation and UI debounce; no concurrent load' };
const browser = await chromium.launch({ headless: true });
let load;
try {
async function editor(user, node) {
const context = await browser.newContext();
await context.addCookies([
{ name: 'collab_dev_user', value: user, url: base },
{ name: 'presence_node', value: node, url: base },
]);
const page = await context.newPage();
page.setDefaultTimeout(15000);
page.on('pageerror', error => result.errors.push(error.message));
page.on('response', response => {
const path = new URL(response.url()).pathname;
if (path.endsWith('/realtime/stream')) {
result.streams.push({ user, status: response.status(), type: response.headers()['content-type'] });
}
if (path.includes('/presence/leases')) {
result.requests.push({ user, status: response.status(), node: response.headers()['x-acceptance-node'] });
}
});
await page.goto(`${base}/app/person/1/edit?createMockSession=true`);
await expect(page.locator('[data-qqq-id="firstName"]')).toBeVisible();
return page;
}
const observer = await editor('alice', 'first');
const sender = await editor('admin', 'second');
await observer.locator('[data-qqq-id="firstName"]').fill('Retained observer draft');
await sender.locator('[data-qqq-id="firstName"]').fill('Retained sender draft');
await expect(observer.locator('[data-presence-field="firstName"]')).toContainText('Demo Administrator');
if (loadMode) {
load = await startPresenceLoad(browser, base, expect);
result.load = load.state;
result.scope = 'Chromium focus-to-remote-DOM and reconnect under 16 concurrent HTTP lease workers; loopback, four background identities';
}
for (let index = 0; index < samples; index++) {
const previousCompleted = load?.state.completed;
if (load) await delay(60000 / samples);
const field = index % 2 === 0 ? 'lastName' : 'firstName';
const previous = field === 'firstName' ? 'lastName' : 'firstName';
await expect(observer.locator(`[data-presence-field="${field}"]`)).not.toContainText('Demo Administrator');
const before = performance.now();
await sender.locator(`[data-qqq-id="${field}"]`).focus();
await expect(observer.locator(`[data-presence-field="${field}"]`)).toContainText('Demo Administrator');
result.samples.push({ field, propagation_ms: performance.now() - before });
await expect(observer.locator(`[data-presence-field="${previous}"]`)).not.toContainText('Demo Administrator');
if (load) {
expect(load.state.errors).toEqual([]);
expect(load.state.completed).toBeGreaterThan(previousCompleted);
}
}
// Drop a real browser connection, miss multiple focus updates, then converge
// through the shipped reconnect/polling behavior without reloading the form.
const oldField = samples % 2 === 0 ? 'firstName' : 'lastName';
const newField = oldField === 'firstName' ? 'lastName' : 'firstName';
const streamsBefore = result.streams.filter(stream => stream.user === 'alice').length;
result.reconnect = { status: 'FAIL', missed_changes: 0 };
const interrupted = observer.waitForEvent('requestfailed', {
predicate: request => new URL(request.url()).pathname.endsWith('/realtime/stream'),
timeout: 10000,
});
await observer.context().setOffline(true);
const disconnect = await fetch(`${base}/__acceptance/disconnect-first`, { method: 'POST' });
expect(disconnect.status).toBe(200);
result.reconnect.disconnected_streams = (await disconnect.json()).disconnected;
expect(result.reconnect.disconnected_streams).toBeGreaterThan(0);
const failedStream = await interrupted;
result.reconnect.stream_failure = failedStream.failure()?.errorText;
const offlineFetch = await observer.evaluate(async () => {
try {
await fetch(`/app/?offlineProbe=${Date.now()}`, { cache: 'no-store' });
return false;
} catch { return true; }
});
expect(offlineFetch).toBe(true);
for (const field of [newField, oldField, newField]) {
const renewed = sender.waitForResponse(response =>
new URL(response.url()).pathname.endsWith('/presence/leases/renew') &&
response.request().method() === 'POST' && response.status() === 200 &&
response.request().postDataJSON()?.fieldName === field);
await sender.locator(`[data-qqq-id="${field}"]`).focus();
await renewed;
result.reconnect.missed_changes++;
}
await expect(observer.locator(`[data-presence-field="${newField}"]`)).not.toContainText('Demo Administrator');
const resumed = performance.now();
await observer.context().setOffline(false);
await expect.poll(() => result.streams.filter(stream => stream.user === 'alice' &&
stream.status === 200 && stream.type?.includes('text/event-stream')).length,
{ timeout: 30000 }).toBeGreaterThan(streamsBefore);
await expect(observer.locator(`[data-presence-field="${newField}"]`)).toContainText('Demo Administrator', { timeout: 30000 });
await expect(observer.locator(`[data-presence-field="${oldField}"]`)).not.toContainText('Demo Administrator');
result.reconnect.recovery_ms = performance.now() - resumed;
result.reconnect.status = 'PASS';
if (load) {
await load.stop();
expect(load.state.errors).toEqual([]);
expect(load.state.elapsed_ms).toBeGreaterThanOrEqual(60000);
expect(load.state.nodes.first).toBeGreaterThan(0);
expect(load.state.nodes.second).toBeGreaterThan(0);
load = undefined;
}
await expect(observer.locator('[data-qqq-id="firstName"]')).toHaveValue('Retained observer draft');
await expect(sender.locator('[data-qqq-id="firstName"]')).toHaveValue('Retained sender draft');
for (const user of ['alice', 'admin']) {
expect(result.streams.some(stream => stream.user === user && stream.status === 200 && stream.type?.includes('text/event-stream'))).toBe(true);
}
expect(result.requests.some(request => request.user === 'alice' && request.node === 'first')).toBe(true);
expect(result.requests.some(request => request.user === 'admin' && request.node === 'second')).toBe(true);
expect(result.requests.every(request => request.status === 200)).toBe(true);
expect(result.errors).toEqual([]);
const ordered = result.samples.map(sample => sample.propagation_ms).sort((a, b) => a - b);
result.percentiles_ms = Object.fromEntries([['p50', .5], ['p95', .95], ['p99', .99], ['max', 1]]
.map(([name, quantile]) => [name, ordered[Math.ceil(ordered.length * quantile) - 1]]));
result.status = 'PASS';
} catch (error) {
result.failure = String(error);
process.exitCode = 1;
} finally {
if (load) await load.stop();
await browser.close();
await writeFile(output, JSON.stringify(result, null, 2));
console.log(JSON.stringify(result, null, 2));
}
@@ -0,0 +1,75 @@
// Bounded real HTTP load using caller-owned authenticated browser contexts.
import { performance } from 'node:perf_hooks';
import { setTimeout as delay } from 'node:timers/promises';
export async function startPresenceLoad(browser, base, expect) {
const state = { workers: 16, completed: 0, errors: [], nodes: {}, elapsed_ms: 0 };
const contexts = [];
const leases = [];
let running = true;
let stopped = false;
let tasks = [];
const began = performance.now();
const endpoint = `${base}/qqq/v1/qbit/collaboration/presence/leases`;
async function post(context, path, data) {
const response = await context.request.post(endpoint + path, { data, timeout: 10000 });
try {
expect(response.status()).toBe(200);
const node = response.headers()['x-acceptance-node'];
state.nodes[node] = (state.nodes[node] ?? 0) + 1;
return await response.json();
} finally { await response.dispose(); }
}
async function stop() {
if (stopped) return;
stopped = true;
running = false;
await Promise.all(tasks);
state.elapsed_ms = performance.now() - began;
for (const lease of leases) {
try {
const response = await lease.context.request.delete(`${base}/qqq/v1/qbit/collaboration/presence`, {
data: { ...lease.target, connectionId: lease.id, sequence: ++lease.sequence }, timeout: 10000,
});
try { expect(response.status()).toBe(200); } finally { await response.dispose(); }
} catch (error) { state.errors.push(String(error)); }
}
for (const context of contexts) await context.close();
}
try {
for (const [index, user] of ['bob', 'carol', 'dave', 'erin'].entries()) {
const context = await browser.newContext();
contexts.push(context);
await context.addCookies([
{ name: 'collab_dev_user', value: user, url: base },
{ name: 'presence_node', value: index % 2 ? 'second' : 'first', url: base },
]);
const page = await context.newPage();
await page.goto(`${base}/app/person/1?createMockSession=true`);
await expect(page.locator('[data-qqq-id="record-view-title-person"]')).toBeVisible();
await page.close();
for (let tab = 0; tab < 4; tab++) {
const target = { table: 'person', pk: '1' };
const opened = await post(context, '', target);
expect(typeof opened.connectionId).toBe('string');
leases.push({ context, target, id: opened.connectionId, sequence: 0, completed: 0 });
}
}
tasks = leases.map(async lease => {
try {
while (running) {
await post(lease.context, '/renew', { ...lease.target, connectionId: lease.id,
sequence: ++lease.sequence, mode: 'VIEWING' });
lease.completed++;
state.completed++;
await delay(100);
}
} catch (error) { state.errors.push(String(error)); running = false; }
});
await expect.poll(() => leases.filter(lease => lease.completed > 0).length).toBe(16);
return { state, stop };
} catch (error) {
await stop();
throw error;
}
}
@@ -0,0 +1,60 @@
// Verify owning-node routing with real published UI and backend artifacts.
import { createRequire } from 'node:module';
import { resolve } from 'node:path';
import { writeFile } from 'node:fs/promises';
const [frontendRepo, base, output] = process.argv.slice(2);
if (!frontendRepo || !output || !['localhost', '127.0.0.1'].includes(new URL(base).hostname)) {
throw new Error('Expected frontend repo, owned loopback URL and result path');
}
const require = createRequire(resolve(frontendRepo, 'package.json'));
const { chromium, expect } = require('@playwright/test');
const result = { status: 'FAIL', checks: [], errors: [] };
const browser = await chromium.launch({ headless: true });
try {
const context = await browser.newContext();
await context.addCookies([
{ name: 'collab_dev_user', value: 'alice', url: base },
{ name: 'presence_node', value: 'first', url: base },
]);
const page = await context.newPage();
page.on('pageerror', error => result.errors.push(error.message));
const subscribed = page.waitForResponse(response =>
new URL(response.url()).pathname.endsWith('/realtime/subscriptions') && response.status() === 200);
await page.goto(`${base}/app/person/1/edit?createMockSession=true`);
await expect(page.locator('[data-qqq-id="firstName"]')).toBeVisible();
await page.locator('[data-qqq-id="firstName"]').fill('Keep this unsaved draft');
const response = await subscribed;
expect(response.headers()['x-acceptance-node']).toBe('first');
const { connectionId } = response.request().postDataJSON();
expect(typeof connectionId).toBe('string');
for (const [node, expected] of [['first', 200], ['second', 404], ['first', 200]]) {
// Keep the same session/user cookies and change only the test routing cookie.
const cookies = (await context.cookies(base)).map(cookie =>
`${cookie.name}=${cookie.name === 'presence_node' ? node : cookie.value}`).join('; ');
const changed = await context.request.post(`${base}/qqq/v1/realtime/subscriptions`, {
headers: { Cookie: cookies },
data: { connectionId, add: ['record:person:1'], remove: [] }, timeout: 10000,
});
try {
result.checks.push({ node: changed.headers()['x-acceptance-node'], status: changed.status() });
expect(changed.headers()['x-acceptance-node']).toBe(node);
expect(changed.status()).toBe(expected);
if (expected === 200) {
const body = await changed.json();
expect(body.active).toContain('record:person:1');
expect(body.refused ?? []).toEqual([]);
}
} finally { await changed.dispose(); }
}
await expect(page.locator('[data-qqq-id="firstName"]')).toHaveValue('Keep this unsaved draft');
expect(result.errors).toEqual([]);
result.status = 'PASS';
} catch (error) {
result.failure = String(error);
process.exitCode = 1;
} finally {
await browser.close();
await writeFile(output, JSON.stringify(result, null, 2));
console.log(JSON.stringify(result, null, 2));
}
+343
View File
@@ -0,0 +1,343 @@
"""Bounded HTTP presence workload across independent, disposable JVMs.
This measures loopback HTTP service latency, not browser propagation or a
production capacity limit. Run test-compile and dependency:build-classpath with
test scope and output target/capacity-classpath.txt before invoking this script.
"""
import argparse
from concurrent.futures import ThreadPoolExecutor
import json
import math
import os
from pathlib import Path
import shutil
import socket
import subprocess
import sys
import time
import urllib.error
import urllib.request
import uuid
def port():
"""Choose a loopback port; child startup detects a competing bind."""
with socket.socket() as listener:
listener.bind(("127.0.0.1", 0))
return listener.getsockname()[1]
def request(node, user, path, body=None, tab=0, method=None, expected=200):
"""Use real HTTP and the explicit mock identity; no tenant parameter."""
separator = "&" if "?" in path else "?"
req = urllib.request.Request(
f"http://127.0.0.1:{node}/qqq/v1/qbit/collaboration{path}{separator}user={user}",
data=None if body is None else json.dumps(body).encode(),
method=method,
headers={"Content-Type": "application/json",
"Cookie": f"sessionUUID={user}-capacity-{tab}"})
try:
response = urllib.request.urlopen(req, timeout=10)
except urllib.error.HTTPError as error:
if error.code == expected:
with error:
return json.load(error)
with error:
detail = error.read(4096).decode(errors="replace")
raise RuntimeError(f"HTTP {error.code} node={node} user={user} path={path}: {detail}") from error
with response:
if response.status != expected:
raise AssertionError(f"Unexpected status {response.status}")
return json.load(response)
def distribution(values):
"""Nearest-rank milliseconds, with no discarded slow or failed samples."""
ordered = sorted(values)
return {"samples": len(ordered), **{
name: ordered[max(0, math.ceil(len(ordered) * fraction) - 1)]
for name, fraction in (("p50_ms", .5), ("p95_ms", .95), ("p99_ms", .99), ("max_ms", 1))}}
def main():
"""Own child JVMs/container and retain results even when a check fails."""
parser = argparse.ArgumentParser()
parser.add_argument("--java", required=True)
parser.add_argument("--node-image", help="Optional Linux Java21 image; isolate Redis traffic on an owned Docker network")
parser.add_argument("--nodes", type=int, default=3)
parser.add_argument("--tenants", type=int, default=2)
parser.add_argument("--users", type=int, default=6)
parser.add_argument("--tabs", type=int, default=4)
parser.add_argument("--rounds", type=int, default=10)
parser.add_argument("--workers", type=int, default=8)
parser.add_argument("--crash-round", type=int, default=0, help="Kill the first node before this round; zero disables")
parser.add_argument("--churn-every", type=int, default=0, help="Leave/rejoin the last user's tabs every N rounds; zero disables")
parser.add_argument("--pause-redis", action="store_true", help="After the workload, freeze owned Redis and verify bounded failures/recovery")
args = parser.parse_args()
for name, lower, upper in (("nodes", 2, 8), ("tenants", 2, 8), ("users", 2, 100),
("tabs", 2, 8), ("rounds", 1, 1000), ("workers", 1, 64)):
if not lower <= getattr(args, name) <= upper:
parser.error(f"{name} must be {lower}..{upper}")
if args.crash_round and not 2 <= args.crash_round <= args.rounds:
parser.error("crash-round must be zero or 2..rounds")
if args.churn_every and not 1 <= args.churn_every <= args.rounds:
parser.error("churn-every must be zero or 1..rounds")
repo = Path(__file__).resolve().parents[2]
output = repo / "target" / ("capacity-nodes-" + uuid.uuid4().hex[:10])
output.mkdir(parents=True)
classpath = os.pathsep.join([str(repo / "target/test-classes"), str(repo / "target/classes"),
(repo / "target/capacity-classpath.txt").read_text().strip()])
processes, logs = [], []
container = "qqq-capacity-" + uuid.uuid4().hex[:12]
network = container + "-net"
node_containers = []
network_created = False
result = {"status": "FAIL", "parameters": vars(args), "scope": "loopback HTTP, mock identities, memory records"}
try:
network_args = []
if args.node_image:
result["node_image_id"] = subprocess.check_output(
["docker", "image", "inspect", args.node_image, "--format", "{{.Id}}"], text=True).strip()
subprocess.run(["docker", "network", "create", network], check=True, capture_output=True)
network_created = True
network_args = ["--network", network]
dependencies = output / "dependencies"
dependencies.mkdir()
linux_classpath = ["/fixture/target/test-classes", "/fixture/target/classes"]
for index, dependency in enumerate((repo / "target/capacity-classpath.txt").read_text().strip().split(os.pathsep)):
destination = dependencies / f"{index}.jar"
shutil.copyfile(dependency, destination)
linux_classpath.append("/fixture/" + destination.relative_to(repo).as_posix())
classpath = ":".join(linux_classpath)
subprocess.run(["docker", "run", "-d", "--name", container, "-p", "127.0.0.1::6379",
*network_args,
"redis:7.4-alpine", "redis-server", "--slowlog-log-slower-than", "1000",
"--slowlog-max-len", "128", "--latency-monitor-threshold", "1"], check=True, capture_output=True)
redis_port = int(subprocess.check_output(["docker", "port", container, "6379"], text=True).strip().rsplit(":", 1)[1])
nodes = [port() for _ in range(args.nodes)]
if len(set(nodes)) != len(nodes):
raise RuntimeError("Port allocation collision; no nodes started")
for node in nodes:
argfile = output / f"node-{node}.args"
connection_properties = (f"-Dcollab.capacity.redisHost={container}\n-Dcollab.dev.bindAddress=0.0.0.0\n"
if args.node_image else "")
argfile.write_text(f'-Xmx384m\n-Dcollab.capacity.tenants={args.tenants}\n-Dcollab.capacity.users={args.users}\n'
+ connection_properties +
'-cp\n"' + classpath.replace("\\", "/") + '"\n'
'com.kingsrook.qbits.collaboration.devserver.CapacityPresenceDevServer\n'
+ (f'8080\n6379\n' if args.node_image else f'{node}\n{redis_port}\n'), encoding="utf-8")
log = (output / f"node-{node}.log").open("w")
logs.append(log)
command = [args.java, "@" + str(argfile)]
if args.node_image:
node_container = f"{container}-node-{node}"
node_containers.append(node_container)
command = ["docker", "run", "--rm", "--name", node_container, "--network", network,
"-p", f"127.0.0.1:{node}:8080", "--mount", f"type=bind,source={repo},target=/fixture,readonly",
args.node_image, "java", "@/fixture/" + argfile.relative_to(repo).as_posix()]
processes.append(subprocess.Popen(command, cwd=repo,
stdout=log, stderr=subprocess.STDOUT,
creationflags=getattr(subprocess, "CREATE_NO_WINDOW", 0)))
deadline = time.monotonic() + 120
for node, process in zip(nodes, processes):
while "CAPACITY_PRESENCE_READY " not in (output / f"node-{node}.log").read_text(errors="replace"):
if process.poll() is not None or time.monotonic() > deadline:
raise RuntimeError(f"Node {node} failed startup; inspect {output}")
time.sleep(.2)
actors = []
for tenant in range(args.tenants):
for user in range(args.users):
for tab in range(args.tabs):
identity = f"load-t{tenant}-u{user}"
record = str(1 + tab % 2)
node = nodes[len(actors) % len(nodes)]
lease = request(node, identity, "/presence/leases", {"table": "person", "pk": record}, tab)["connectionId"]
actors.append((identity, record, tab, lease, user))
renewals, snapshots = [], []
def check_rooms(excluded=None):
"""Check every tenant/record on every surviving node, including absence."""
for tenant in range(args.tenants):
expected_users = {f"load-t{tenant}-u{user}" for user in range(args.users) if user != excluded}
for node in nodes:
for record in ("1", "2"):
before = time.perf_counter()
viewers = request(node, f"load-t{tenant}-u0", f"/presence?table=person&pk={record}")["viewers"]
snapshots.append((time.perf_counter() - before) * 1000)
actual = {viewer["userId"] for viewer in viewers}
if actual != expected_users or len(viewers) != len(expected_users):
raise AssertionError(f"Tenant isolation/deduplication mismatch: {actual} != {expected_users}")
if sum(viewer["mode"] == "EDITING" for viewer in viewers) != sum(user != excluded for user in range(2)):
raise AssertionError("Unexpected editor count")
def renew(item, sequence):
"""Keep each tab sequential while moving its writes between nodes."""
index, (identity, record, tab, lease, user) = item
before = time.perf_counter()
request(nodes[(index + sequence) % len(nodes)], identity, "/presence/leases/renew",
{"connectionId": lease, "table": "person", "pk": record,
"mode": "EDITING" if user < 2 else "VIEWING", "sequence": sequence}, tab)
return (time.perf_counter() - before) * 1000
started = time.perf_counter()
with ThreadPoolExecutor(max_workers=args.workers) as workers:
for sequence in range(1, args.rounds + 1):
result["current_round"] = sequence
if sequence == args.crash_round:
crashed = nodes.pop(0)
if args.node_image:
subprocess.run(["docker", "kill", node_containers[0]], check=True, capture_output=True)
else:
processes[0].kill()
processes[0].wait(timeout=10)
result["crash"] = {"round": sequence, "http_port": crashed,
"surviving_nodes": len(nodes), "process_exit": processes[0].returncode}
renewals.extend(workers.map(lambda item: renew(item, sequence), enumerate(actors)))
check_rooms()
if args.churn_every and sequence % args.churn_every == 0:
departing = [(index, actor) for index, actor in enumerate(actors) if actor[4] == args.users - 1]
for index, (identity, record, tab, lease, user) in departing:
request(nodes[index % len(nodes)], identity, "/presence",
{"connectionId": lease, "table": "person", "pk": record, "sequence": sequence + 1}, tab, method="DELETE")
response = request(nodes[(index + 1) % len(nodes)], identity, "/presence/leases/renew",
{"connectionId": lease, "table": "person", "pk": record,
"mode": "VIEWING", "sequence": sequence + 2}, tab, expected=409)
if response.get("code") != "PRESENCE_LEASE_EXPIRED":
raise AssertionError("Departed lease was not rejected as expired")
check_rooms(excluded=args.users - 1)
for index, (identity, record, tab, lease, user) in departing:
replacement = request(nodes[(index + 1) % len(nodes)], identity, "/presence/leases",
{"table": "person", "pk": record}, tab)["connectionId"]
if replacement == lease:
raise AssertionError("Rejoin reused departed authority")
actors[index] = (identity, record, tab, replacement, user)
renew((index, actors[index]), sequence)
check_rooms()
result["churn_cycles"] = result.get("churn_cycles", 0) + 1
result["departed_leases_rejected"] = result.get("departed_leases_rejected", 0) + len(departing)
result["completed_rounds"] = sequence
if args.pause_redis:
failures = []
def stalled_renew(item):
"""A frozen authority must fail explicitly, never acknowledge a write."""
index, (identity, record, tab, lease, user) = item
before = time.perf_counter()
response = request(nodes[index % len(nodes)], identity, "/presence/leases/renew",
{"connectionId": lease, "table": "person", "pk": record,
"mode": "EDITING" if user < 2 else "VIEWING", "sequence": args.rounds + 1},
tab, expected=503)
if "viewers" in response or "temporarily unavailable" not in response.get("message", ""):
raise AssertionError("Stalled authority did not return the expected unavailable response")
duration = (time.perf_counter() - before) * 1000
if duration > 8000:
raise AssertionError("Unavailable response exceeded the eight-second fixture bound")
return duration
subprocess.run(["docker", "pause", container], check=True, capture_output=True)
try:
with ThreadPoolExecutor(max_workers=64) as burst:
failures.extend(burst.map(stalled_renew, enumerate(actors)))
for node in nodes:
response = request(node, "load-t0-u0", "/presence?table=person&pk=1", expected=503)
if "viewers" in response:
raise AssertionError("Stalled authority was presented as a room snapshot")
finally:
subprocess.run(["docker", "unpause", container], check=True, capture_output=True)
subprocess.run(["docker", "exec", container, "redis-cli", "PING"], check=True, capture_output=True, timeout=10)
for item in enumerate(actors):
renew(item, args.rounds + 2)
check_rooms()
result["redis_stall"] = {"rejected_renewals": len(failures), "failed_snapshot_nodes": len(nodes),
"response_times": distribution(failures), "same_leases_recovered": True}
elapsed = time.perf_counter() - started
result.update(status="PASS", seconds=elapsed, renewals=distribution(renewals),
snapshots=distribution(snapshots), renewal_throughput=len(renewals) / elapsed,
node_pids=[process.pid for process in processes],
pid_kind="docker-client" if args.node_image else "JVM", node_containers=node_containers)
except Exception as error:
result["error"] = str(error)
raise
finally:
failure_active = sys.exc_info()[0] is not None
try:
slow = json.loads(subprocess.check_output(
["docker", "exec", container, "redis-cli", "--json", "SLOWLOG", "GET", "128"], text=True, timeout=10))
result["redis_slow_commands"] = [{"unix_time": entry[1], "duration_us": entry[2], "command": entry[3][0]}
for entry in slow]
result["redis_latency_events"] = json.loads(subprocess.check_output(
["docker", "exec", container, "redis-cli", "--json", "LATENCY", "LATEST"], text=True, timeout=10))
for section in ("cpu", "memory", "commandstats", "stats"):
value = subprocess.check_output(["docker", "exec", container, "redis-cli", "INFO", section], text=True, timeout=10)
(output / f"redis-{section}.txt").write_text(value)
parsed = dict(line.split(":", 1) for line in value.splitlines() if ":" in line and not line.startswith("#"))
if section == "memory":
result["redis_memory"] = {key: int(parsed[key]) for key in
("used_memory", "used_memory_peak", "used_memory_rss")}
if section == "cpu":
result["redis_cpu_seconds_since_start"] = {key: float(parsed[key]) for key in
("used_cpu_sys", "used_cpu_user")}
except (subprocess.SubprocessError, ValueError, KeyError) as error:
result["redis_diagnostic_error"] = type(error).__name__
node_cleanup = True
for node_container in node_containers:
removed = subprocess.run(["docker", "rm", "-f", node_container], capture_output=True)
if removed.returncode != 0:
remaining = subprocess.run(["docker", "ps", "-aq", "--filter", f"name=^{node_container}$"], capture_output=True, text=True)
node_cleanup = node_cleanup and remaining.returncode == 0 and not remaining.stdout.strip()
for process in processes:
if process.poll() is None:
process.terminate()
for process in processes:
try:
process.wait(timeout=10)
except subprocess.TimeoutExpired:
process.kill()
process.wait(timeout=10)
for log in logs:
log.close()
resources = []
for node_log in sorted(output.glob("node-*.log")):
samples = [json.loads(line.split("CAPACITY_METRIC ", 1)[1])
for line in node_log.read_text(errors="replace").splitlines()
if line.startswith("CAPACITY_METRIC ")]
if samples:
first, last = samples[0], samples[-1]
wall_ms = last["uptimeMillis"] - first["uptimeMillis"]
cpu_ns = last["cpuNanos"] - first["cpuNanos"]
resources.append({"node_log": node_log.name, "samples": len(samples),
"observed_wall_ms": wall_ms,
"cpu_seconds": cpu_ns / 1e9 if first["cpuNanos"] >= 0 else None,
"cpu_core_equivalents": cpu_ns / (wall_ms * 1e6) if wall_ms > 0 and first["cpuNanos"] >= 0 else None,
**{key + "_sampled_max": max(sample[key] for sample in samples)
for key in ("heapUsedBytes", "heapCommittedBytes", "poolActive", "poolIdle", "poolWaiting", "poolMaximum")},
"residentBytes_sampled_max": max(sample["residentBytes"] for sample in samples)
if all(sample.get("residentBytes", -1) >= 0 for sample in samples) else None})
if any(sample["poolActive"] > sample["poolMaximum"] for sample in samples):
result.update(status="FAIL", error="Sampled pool active count exceeded configured maximum")
result["resources"] = resources
if result["status"] == "PASS" and args.node_image and any(node["residentBytes_sampled_max"] is None for node in resources):
result.update(status="FAIL", error="Linux resident-memory samples missing")
if result["status"] == "PASS" and any(key not in result for key in ("redis_memory", "redis_cpu_seconds_since_start")):
result.update(status="FAIL", error="Redis memory/CPU diagnostics missing")
if result["status"] == "PASS" and len(resources) != args.nodes:
result.update(status="FAIL", error="Missing resource samples for a node")
if result["status"] == "PASS" and args.pause_redis and not any(node["poolWaiting_sampled_max"] > 0 for node in resources):
result.update(status="FAIL", error="Redis stall did not demonstrate pool waiters")
cleanup = subprocess.run(["docker", "rm", "-f", container], capture_output=True)
network_cleanup = True
if network_created:
network_cleanup = subprocess.run(["docker", "network", "rm", network], capture_output=True).returncode == 0
result["cleanup"] = cleanup.returncode == 0 and node_cleanup and network_cleanup and all(process.poll() is not None for process in processes)
(output / "result.json").write_text(json.dumps(result, indent=2))
print(json.dumps(result, indent=2))
print(output)
if result["status"] == "PASS" and not result["cleanup"]:
raise RuntimeError("Workload passed but owned resource cleanup failed")
if result["status"] == "FAIL" and not failure_active:
raise RuntimeError(result.get("error", "Capacity resource validation failed"))
if __name__ == "__main__":
main()
+42
View File
@@ -0,0 +1,42 @@
// Disposable loopback fixture only; never deploy this fault-injection router.
import http from 'node:http';
const [first, second, listen] = process.argv.slice(2).map(Number);
if (![first, second, listen].every(port => Number.isInteger(port) && port > 0 && port < 65536)) {
throw new Error('Expected first-node, second-node and router ports');
}
const firstStreams = new Set();
const server = http.createServer((request, response) => {
if (request.method === 'POST' && request.url === '/__acceptance/disconnect-first') {
const disconnected = firstStreams.size;
for (const stream of firstStreams) stream.destroy();
response.writeHead(200, { 'content-type': 'application/json' });
response.end(JSON.stringify({ disconnected }));
return;
}
const isSecond = /(?:^|;\s*)presence_node=second(?:;|$)/.test(request.headers.cookie ?? '');
const port = isSecond ? second : first;
const upstream = http.request({
hostname: '127.0.0.1', port, path: request.url, method: request.method,
headers: { ...request.headers, host: `127.0.0.1:${port}` },
}, incoming => {
response.writeHead(incoming.statusCode, {
...incoming.headers, 'x-acceptance-node': isSecond ? 'second' : 'first',
});
if (!isSecond && incoming.headers['content-type']?.includes('text/event-stream')) {
firstStreams.add(response);
}
incoming.pipe(response);
});
upstream.on('error', () => {
if (!response.headersSent) response.writeHead(502);
response.end('Acceptance node unavailable');
});
response.on('close', () => {
firstStreams.delete(response);
upstream.destroy();
});
request.pipe(upstream);
});
server.listen(listen, '127.0.0.1', () => console.log(`PRESENCE_ROUTER_READY ${listen}`));
for (const signal of ['SIGINT', 'SIGTERM']) process.on(signal, () => server.close());
@@ -0,0 +1,285 @@
{
"status": "PASS",
"samples": [
{
"field": "lastName",
"propagation_ms": 36.092099999999846
},
{
"field": "firstName",
"propagation_ms": 32.51870000000008
},
{
"field": "lastName",
"propagation_ms": 31.758600000000115
},
{
"field": "firstName",
"propagation_ms": 31.090299999999843
},
{
"field": "lastName",
"propagation_ms": 31.64980000000014
},
{
"field": "firstName",
"propagation_ms": 31.648199999999633
},
{
"field": "lastName",
"propagation_ms": 31.395500000000084
},
{
"field": "firstName",
"propagation_ms": 32.36750000000029
},
{
"field": "lastName",
"propagation_ms": 31.601799999999912
},
{
"field": "firstName",
"propagation_ms": 31.90599999999995
},
{
"field": "lastName",
"propagation_ms": 31.44150000000036
},
{
"field": "firstName",
"propagation_ms": 31.186699999999746
},
{
"field": "lastName",
"propagation_ms": 31.115800000000036
},
{
"field": "firstName",
"propagation_ms": 32.539400000000114
},
{
"field": "lastName",
"propagation_ms": 31.188000000000102
},
{
"field": "firstName",
"propagation_ms": 33.021200000000135
},
{
"field": "lastName",
"propagation_ms": 33.07759999999962
},
{
"field": "firstName",
"propagation_ms": 32.7176000000004
},
{
"field": "lastName",
"propagation_ms": 31.457300000000032
},
{
"field": "firstName",
"propagation_ms": 32.920900000000074
},
{
"field": "lastName",
"propagation_ms": 31.89130000000023
},
{
"field": "firstName",
"propagation_ms": 30.847200000000157
},
{
"field": "lastName",
"propagation_ms": 30.158300000000054
},
{
"field": "firstName",
"propagation_ms": 31.48239999999987
},
{
"field": "lastName",
"propagation_ms": 31.337600000000293
}
],
"streams": [
{
"user": "alice",
"status": 200,
"type": "text/event-stream;charset=utf-8"
},
{
"user": "admin",
"status": 200,
"type": "text/event-stream;charset=utf-8"
}
],
"requests": [
{
"user": "alice",
"status": 200,
"node": "first"
},
{
"user": "alice",
"status": 200,
"node": "first"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "alice",
"status": 200,
"node": "first"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
}
],
"errors": [],
"scope": "Chromium focus-to-remote-DOM observation, including automation and UI debounce; no concurrent load",
"percentiles_ms": {
"p50": 31.648199999999633,
"p95": 33.07759999999962,
"p99": 36.092099999999846,
"max": 36.092099999999846
},
"frontend_jar_sha256": "c9fa2eb2381a47c8104f82f4e940d2bfa8ba67e2341298dde0de4e21ad3ff1ab",
"fixture_cleanup": true,
"backend_source": "Published core cc3981bb3/collaboration ecff347/Redis7fe5492 snapshots; two independent JVMs, mock identities and memory records."
}
@@ -0,0 +1,357 @@
{
"status": "PASS",
"samples": [
{
"field": "lastName",
"propagation_ms": 36.560600000000704
},
{
"field": "firstName",
"propagation_ms": 33.44050000000061
},
{
"field": "lastName",
"propagation_ms": 32.50890000000072
},
{
"field": "firstName",
"propagation_ms": 785.4951000000001
},
{
"field": "lastName",
"propagation_ms": 85.39069999999992
},
{
"field": "firstName",
"propagation_ms": 30.589599999999336
},
{
"field": "lastName",
"propagation_ms": 32.756099999998696
},
{
"field": "firstName",
"propagation_ms": 38.17399999999907
},
{
"field": "lastName",
"propagation_ms": 30.243099999999686
},
{
"field": "firstName",
"propagation_ms": 29.600799999992887
},
{
"field": "lastName",
"propagation_ms": 29.57220000000234
},
{
"field": "firstName",
"propagation_ms": 29.197800000001735
},
{
"field": "lastName",
"propagation_ms": 33.05130000000645
},
{
"field": "firstName",
"propagation_ms": 32.38390000000072
},
{
"field": "lastName",
"propagation_ms": 31.52850000000035
},
{
"field": "firstName",
"propagation_ms": 34.66169999999693
},
{
"field": "lastName",
"propagation_ms": 31.258300000001327
},
{
"field": "firstName",
"propagation_ms": 44.19029999999475
},
{
"field": "lastName",
"propagation_ms": 36.56189999999333
},
{
"field": "firstName",
"propagation_ms": 30.240600000004633
},
{
"field": "lastName",
"propagation_ms": 29.79899999999907
},
{
"field": "firstName",
"propagation_ms": 29.621500000001106
},
{
"field": "lastName",
"propagation_ms": 31.66189999999915
},
{
"field": "firstName",
"propagation_ms": 29.198399999993853
},
{
"field": "lastName",
"propagation_ms": 32.931200000006356
}
],
"streams": [
{
"user": "alice",
"status": 200,
"type": "text/event-stream;charset=utf-8"
},
{
"user": "admin",
"status": 200,
"type": "text/event-stream;charset=utf-8"
},
{
"user": "alice",
"status": 200,
"type": "text/event-stream;charset=utf-8"
}
],
"requests": [
{
"user": "alice",
"status": 200,
"node": "first"
},
{
"user": "alice",
"status": 200,
"node": "first"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "alice",
"status": 200,
"node": "first"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "alice",
"status": 200,
"node": "first"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "alice",
"status": 200,
"node": "first"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "alice",
"status": 200,
"node": "first"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
}
],
"errors": [],
"scope": "Chromium focus-to-remote-DOM and reconnect under 16 concurrent HTTP lease workers; loopback, four background identities",
"load": {
"workers": 16,
"completed": 9046,
"errors": [],
"nodes": {
"first": 4540,
"second": 4522
},
"elapsed_ms": 69728.3579
},
"reconnect": {
"status": "PASS",
"missed_changes": 3,
"disconnected_streams": 1,
"stream_failure": "net::ERR_INCOMPLETE_CHUNKED_ENCODING",
"recovery_ms": 1855.8868999999977
},
"percentiles_ms": {
"p50": 32.38390000000072,
"p95": 85.39069999999992,
"p99": 785.4951000000001,
"max": 785.4951000000001
},
"frontend_jar_sha256": "c9fa2eb2381a47c8104f82f4e940d2bfa8ba67e2341298dde0de4e21ad3ff1ab",
"fixture_cleanup": true,
"backend_source": "Published core cc3981bb3/collaboration ecff347/Redis7fe5492 snapshots; two independent JVMs, mock identities and memory records.",
"fixture_checks": [
"browser-six-presence",
"browser-redis-admin",
"browser-presence-latency"
]
}
@@ -0,0 +1,317 @@
{
"status": "PASS",
"samples": [
{
"field": "lastName",
"propagation_ms": 37.25770000000011
},
{
"field": "firstName",
"propagation_ms": 32.14499999999998
},
{
"field": "lastName",
"propagation_ms": 32.40950000000021
},
{
"field": "firstName",
"propagation_ms": 33.52500000000009
},
{
"field": "lastName",
"propagation_ms": 31.228200000000015
},
{
"field": "firstName",
"propagation_ms": 30.430999999999585
},
{
"field": "lastName",
"propagation_ms": 31.493199999999888
},
{
"field": "firstName",
"propagation_ms": 33.14539999999988
},
{
"field": "lastName",
"propagation_ms": 32.11259999999993
},
{
"field": "firstName",
"propagation_ms": 33.23129999999992
},
{
"field": "lastName",
"propagation_ms": 31.06800000000021
},
{
"field": "firstName",
"propagation_ms": 30.109199999999873
},
{
"field": "lastName",
"propagation_ms": 31.534299999999803
},
{
"field": "firstName",
"propagation_ms": 33.770800000000236
},
{
"field": "lastName",
"propagation_ms": 31.654199999999946
},
{
"field": "firstName",
"propagation_ms": 33.31700000000001
},
{
"field": "lastName",
"propagation_ms": 33.108600000000024
},
{
"field": "firstName",
"propagation_ms": 32.27370000000019
},
{
"field": "lastName",
"propagation_ms": 31.452499999999873
},
{
"field": "firstName",
"propagation_ms": 31.86220000000003
},
{
"field": "lastName",
"propagation_ms": 31.734399999999823
},
{
"field": "firstName",
"propagation_ms": 32.04489999999987
},
{
"field": "lastName",
"propagation_ms": 31.051899999999932
},
{
"field": "firstName",
"propagation_ms": 30.876300000000356
},
{
"field": "lastName",
"propagation_ms": 31.56970000000001
}
],
"streams": [
{
"user": "alice",
"status": 200,
"type": "text/event-stream;charset=utf-8"
},
{
"user": "admin",
"status": 200,
"type": "text/event-stream;charset=utf-8"
},
{
"user": "alice",
"status": 200,
"type": "text/event-stream;charset=utf-8"
}
],
"requests": [
{
"user": "alice",
"status": 200,
"node": "first"
},
{
"user": "alice",
"status": 200,
"node": "first"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "alice",
"status": 200,
"node": "first"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
}
],
"errors": [],
"scope": "Chromium focus-to-remote-DOM observation, including automation and UI debounce; no concurrent load",
"reconnect": {
"status": "PASS",
"missed_changes": 3,
"disconnected_streams": 1,
"stream_failure": "net::ERR_INCOMPLETE_CHUNKED_ENCODING",
"recovery_ms": 1858.2972000000004
},
"percentiles_ms": {
"p50": 31.86220000000003,
"p95": 33.770800000000236,
"p99": 37.25770000000011,
"max": 37.25770000000011
},
"frontend_jar_sha256": "c9fa2eb2381a47c8104f82f4e940d2bfa8ba67e2341298dde0de4e21ad3ff1ab",
"fixture_cleanup": true,
"backend_source": "Published core cc3981bb3/collaboration ecff347/Redis7fe5492 snapshots; two independent JVMs, mock identities and memory records.",
"fixture_checks": [
"browser-six-presence",
"browser-redis-admin",
"browser-presence-latency"
]
}
@@ -0,0 +1,14 @@
{
"status": "PASS",
"checks": [
{ "node": "first", "status": 200 },
{ "node": "second", "status": 404 },
{ "node": "first", "status": 200 }
],
"errors": [],
"fixture_cleanup": true,
"fixture_checks": ["browser-six-presence", "browser-redis-admin", "browser-subscription-affinity"],
"frontend_jar_sha256": "c9fa2eb2381a47c8104f82f4e940d2bfa8ba67e2341298dde0de4e21ad3ff1ab",
"backend_source": "Published core cc3981bb3/collaboration ecff347/Redis7fe5492 snapshots; two independent JVMs, mock identities and memory records.",
"scope": "Same browser session and live SSE connection ID; change only test-router affinity. Not deployed Traefik qualification."
}
@@ -0,0 +1,181 @@
{
"status": "PASS",
"parameters": {
"java": "/path/to/java21",
"node_image": "eclipse-temurin@sha256:227118764cccd3c3294617061aaa93a08309c63a395eefa83e6b35032992ea99",
"nodes": 3,
"tenants": 2,
"users": 50,
"tabs": 4,
"rounds": 200,
"workers": 16,
"crash_round": 0,
"churn_every": 0,
"pause_redis": false
},
"scope": "loopback HTTP, mock identities, memory records",
"node_image_id": "sha256:227118764cccd3c3294617061aaa93a08309c63a395eefa83e6b35032992ea99",
"current_round": 200,
"completed_rounds": 200,
"seconds": 102.56801009993069,
"renewals": {
"samples": 80000,
"p50_ms": 11.325599974952638,
"p95_ms": 21.764500066637993,
"p99_ms": 25.881099980324507,
"max_ms": 1007.4040000326931
},
"snapshots": {
"samples": 2400,
"p50_ms": 15.130399959161878,
"p95_ms": 16.613700077868998,
"p99_ms": 19.423600053414702,
"max_ms": 28.864299994893372
},
"renewal_throughput": 779.9702843221492,
"node_pids": [
105696,
4828,
117588
],
"pid_kind": "docker-client",
"node_containers": [
"qqq-capacity-f58f3ddcf5f3-node-59675",
"qqq-capacity-f58f3ddcf5f3-node-59676",
"qqq-capacity-f58f3ddcf5f3-node-59677"
],
"redis_slow_commands": [
{
"unix_time": 1793033347,
"duration_us": 1432,
"command": "EVALSHA"
},
{
"unix_time": 1793033347,
"duration_us": 1612,
"command": "EVALSHA"
},
{
"unix_time": 1793033340,
"duration_us": 1877,
"command": "EVALSHA"
},
{
"unix_time": 1793033340,
"duration_us": 1313,
"command": "ZRANGE"
},
{
"unix_time": 1793033336,
"duration_us": 1316,
"command": "EVALSHA"
},
{
"unix_time": 1793033305,
"duration_us": 1459,
"command": "EVALSHA"
},
{
"unix_time": 1793033300,
"duration_us": 1767,
"command": "EVALSHA"
},
{
"unix_time": 1793033290,
"duration_us": 1180,
"command": "EVALSHA"
},
{
"unix_time": 1793033283,
"duration_us": 1421,
"command": "EVALSHA"
},
{
"unix_time": 1793033260,
"duration_us": 1291,
"command": "EVALSHA"
},
{
"unix_time": 1793033260,
"duration_us": 1551,
"command": "EVALSHA"
},
{
"unix_time": 1793033260,
"duration_us": 1083,
"command": "ZRANGE"
},
{
"unix_time": 1793033255,
"duration_us": 1039,
"command": "EVALSHA"
},
{
"unix_time": 1793033254,
"duration_us": 1215,
"command": "EVALSHA"
}
],
"redis_latency_events": [
[
"command",
1793033347,
1,
1
]
],
"redis_cpu_seconds_since_start": {
"used_cpu_sys": 23.737706,
"used_cpu_user": 35.526214
},
"redis_memory": {
"used_memory": 2415952,
"used_memory_peak": 2579896,
"used_memory_rss": 8777728
},
"resources": [
{
"node_log": "node-59675.log",
"samples": 1112,
"observed_wall_ms": 111099,
"cpu_seconds": 203.14,
"cpu_core_equivalents": 1.828459302063925,
"heapUsedBytes_sampled_max": 181098896,
"heapCommittedBytes_sampled_max": 211812352,
"poolActive_sampled_max": 7,
"poolIdle_sampled_max": 7,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8,
"residentBytes_sampled_max": 662786048
},
{
"node_log": "node-59676.log",
"samples": 1145,
"observed_wall_ms": 114399,
"cpu_seconds": 200.44,
"cpu_core_equivalents": 1.7521132177728826,
"heapUsedBytes_sampled_max": 185167416,
"heapCommittedBytes_sampled_max": 219152384,
"poolActive_sampled_max": 7,
"poolIdle_sampled_max": 7,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8,
"residentBytes_sampled_max": 662188032
},
{
"node_log": "node-59677.log",
"samples": 1159,
"observed_wall_ms": 115800,
"cpu_seconds": 200.16,
"cpu_core_equivalents": 1.728497409326425,
"heapUsedBytes_sampled_max": 179378632,
"heapCommittedBytes_sampled_max": 210763776,
"poolActive_sampled_max": 6,
"poolIdle_sampled_max": 7,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8,
"residentBytes_sampled_max": 703479808
}
],
"cleanup": true
}
@@ -0,0 +1,101 @@
{
"status": "PASS",
"parameters": {
"java": "java (Java 21)",
"node_image": "eclipse-temurin:21-jre",
"nodes": 3,
"tenants": 2,
"users": 12,
"tabs": 4,
"rounds": 1000,
"workers": 8
},
"scope": "loopback HTTP, mock identities, memory records",
"current_round": 1000,
"completed_rounds": 1000,
"seconds": 250.81072249996942,
"renewals": {
"samples": 96000,
"p50_ms": 6.353199947625399,
"p95_ms": 15.330899972468615,
"p99_ms": 17.760399961844087,
"max_ms": 326.8311999272555
},
"snapshots": {
"samples": 12000,
"p50_ms": 13.105699908919632,
"p95_ms": 14.261199976317585,
"p99_ms": 16.519399941898882,
"max_ms": 915.3468000004068
},
"renewal_throughput": 382.7587554595546,
"node_pids": [
64000,
40240,
115400
],
"pid_kind": "docker-client",
"node_containers": [
"qqq-capacity-0a8b7956499d-node-64951",
"qqq-capacity-0a8b7956499d-node-64952",
"qqq-capacity-0a8b7956499d-node-64953"
],
"redis_slow_commands": [
{
"unix_time": 1793030517,
"duration_us": 1117,
"command": "EVALSHA"
}
],
"redis_latency_events": [
[
"command",
1793030517,
1,
1
]
],
"resources": [
{
"node_log": "node-64951.log",
"samples": 2580,
"observed_wall_ms": 257899,
"cpu_seconds": 165.55,
"cpu_core_equivalents": 0.6419179601316795,
"heapUsedBytes_sampled_max": 166667368,
"heapCommittedBytes_sampled_max": 188743680,
"poolActive_sampled_max": 4,
"poolIdle_sampled_max": 5,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8
},
{
"node_log": "node-64952.log",
"samples": 2569,
"observed_wall_ms": 256799,
"cpu_seconds": 165.85,
"cpu_core_equivalents": 0.6458358482704372,
"heapUsedBytes_sampled_max": 169216944,
"heapCommittedBytes_sampled_max": 192937984,
"poolActive_sampled_max": 5,
"poolIdle_sampled_max": 4,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8
},
{
"node_log": "node-64953.log",
"samples": 2608,
"observed_wall_ms": 260700,
"cpu_seconds": 164.72,
"cpu_core_equivalents": 0.631837360951285,
"heapUsedBytes_sampled_max": 166319008,
"heapCommittedBytes_sampled_max": 188743680,
"poolActive_sampled_max": 4,
"poolIdle_sampled_max": 5,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8
}
],
"cleanup": true,
"receipt_note": "Java executable path normalized; measurements unchanged. Disposable fixture identities only."
}
@@ -0,0 +1,117 @@
{
"status": "PASS",
"parameters": {
"java": "java (Java 21)",
"node_image": "eclipse-temurin@sha256:227118764cccd3c3294617061aaa93a08309c63a395eefa83e6b35032992ea99",
"nodes": 3,
"tenants": 2,
"users": 12,
"tabs": 4,
"rounds": 100,
"workers": 8,
"crash_round": 50,
"churn_every": 20
},
"scope": "loopback HTTP, mock identities, memory records",
"node_image_id": "sha256:227118764cccd3c3294617061aaa93a08309c63a395eefa83e6b35032992ea99",
"current_round": 100,
"completed_rounds": 100,
"churn_cycles": 5,
"departed_leases_rejected": 40,
"crash": {
"round": 50,
"http_port": 52824,
"surviving_nodes": 2,
"process_exit": 137
},
"seconds": 28.090077199973166,
"renewals": {
"samples": 9600,
"p50_ms": 7.532799965701997,
"p95_ms": 17.53149996511638,
"p99_ms": 29.359900043345988,
"max_ms": 88.13189994543791
},
"snapshots": {
"samples": 1092,
"p50_ms": 13.582199928350747,
"p95_ms": 15.386099927127361,
"p99_ms": 18.502700026147068,
"max_ms": 22.144700051285326
},
"renewal_throughput": 341.75769371004685,
"node_pids": [
22844,
66652,
61920
],
"pid_kind": "docker-client",
"node_containers": [
"qqq-capacity-bb9ba87919d2-node-52824",
"qqq-capacity-bb9ba87919d2-node-52825",
"qqq-capacity-bb9ba87919d2-node-52827"
],
"redis_slow_commands": [
{
"unix_time": 1793031681,
"duration_us": 1206,
"command": "EVALSHA"
},
{
"unix_time": 1793031681,
"duration_us": 1147,
"command": "EVALSHA"
}
],
"redis_latency_events": [
[
"command",
1793031681,
1,
1
]
],
"resources": [
{
"node_log": "node-52824.log",
"samples": 212,
"observed_wall_ms": 21099,
"cpu_seconds": 29.48,
"cpu_core_equivalents": 1.3972226171856486,
"heapUsedBytes_sampled_max": 163641024,
"heapCommittedBytes_sampled_max": 188743680,
"poolActive_sampled_max": 3,
"poolIdle_sampled_max": 4,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8
},
{
"node_log": "node-52825.log",
"samples": 354,
"observed_wall_ms": 35299,
"cpu_seconds": 43.09,
"cpu_core_equivalents": 1.22071446783195,
"heapUsedBytes_sampled_max": 168178592,
"heapCommittedBytes_sampled_max": 192937984,
"poolActive_sampled_max": 5,
"poolIdle_sampled_max": 5,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8
},
{
"node_log": "node-52827.log",
"samples": 340,
"observed_wall_ms": 33900,
"cpu_seconds": 42.42,
"cpu_core_equivalents": 1.2513274336283187,
"heapUsedBytes_sampled_max": 166291160,
"heapCommittedBytes_sampled_max": 188743680,
"poolActive_sampled_max": 5,
"poolIdle_sampled_max": 5,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8
}
],
"cleanup": true,
"receipt_note": "Java executable path normalized; measurements unchanged. Disposable fixture identities only."
}
@@ -0,0 +1,96 @@
{
"status": "PASS",
"parameters": {
"java": "java (Java 21)",
"node_image": "eclipse-temurin@sha256:227118764cccd3c3294617061aaa93a08309c63a395eefa83e6b35032992ea99",
"nodes": 3,
"tenants": 2,
"users": 12,
"tabs": 4,
"rounds": 100,
"workers": 8,
"crash_round": 50
},
"scope": "loopback HTTP, mock identities, memory records",
"node_image_id": "sha256:227118764cccd3c3294617061aaa93a08309c63a395eefa83e6b35032992ea99",
"current_round": 100,
"completed_rounds": 100,
"crash": {
"round": 50,
"http_port": 54039,
"surviving_nodes": 2,
"process_exit": 137
},
"seconds": 25.147547999978997,
"renewals": {
"samples": 9600,
"p50_ms": 7.125200005248189,
"p95_ms": 17.407900071702898,
"p99_ms": 27.217599912546575,
"max_ms": 74.53640003222972
},
"snapshots": {
"samples": 996,
"p50_ms": 13.538200058974326,
"p95_ms": 15.59239998459816,
"p99_ms": 20.58999997097999,
"max_ms": 315.0679999962449
},
"renewal_throughput": 381.7469599822622,
"node_pids": [
85832,
90280,
67764
],
"pid_kind": "docker-client",
"node_containers": [
"qqq-capacity-e599dca8e163-node-54039",
"qqq-capacity-e599dca8e163-node-54040",
"qqq-capacity-e599dca8e163-node-54041"
],
"redis_slow_commands": [],
"redis_latency_events": [],
"resources": [
{
"node_log": "node-54039.log",
"samples": 198,
"observed_wall_ms": 19699,
"cpu_seconds": 28.58,
"cpu_core_equivalents": 1.4508350677699375,
"heapUsedBytes_sampled_max": 163478640,
"heapCommittedBytes_sampled_max": 188743680,
"poolActive_sampled_max": 3,
"poolIdle_sampled_max": 4,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8
},
{
"node_log": "node-54040.log",
"samples": 300,
"observed_wall_ms": 29900,
"cpu_seconds": 41.85,
"cpu_core_equivalents": 1.399665551839465,
"heapUsedBytes_sampled_max": 165177008,
"heapCommittedBytes_sampled_max": 188743680,
"poolActive_sampled_max": 4,
"poolIdle_sampled_max": 5,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8
},
{
"node_log": "node-54041.log",
"samples": 331,
"observed_wall_ms": 33000,
"cpu_seconds": 41.52,
"cpu_core_equivalents": 1.2581818181818183,
"heapUsedBytes_sampled_max": 168546936,
"heapCommittedBytes_sampled_max": 192937984,
"poolActive_sampled_max": 4,
"poolIdle_sampled_max": 5,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8
}
],
"cleanup": true,
"receipt_note": "Java executable path normalized; measurements unchanged. Disposable fixture identities only."
}
@@ -0,0 +1,117 @@
{
"status": "PASS",
"parameters": {
"java": "java (Java 21)",
"node_image": "eclipse-temurin@sha256:227118764cccd3c3294617061aaa93a08309c63a395eefa83e6b35032992ea99",
"nodes": 3,
"tenants": 2,
"users": 12,
"tabs": 4,
"rounds": 30,
"workers": 8,
"crash_round": 0,
"churn_every": 0,
"pause_redis": true
},
"scope": "loopback HTTP, mock identities, memory records",
"node_image_id": "sha256:227118764cccd3c3294617061aaa93a08309c63a395eefa83e6b35032992ea99",
"current_round": 30,
"completed_rounds": 30,
"redis_stall": {
"rejected_renewals": 96,
"failed_snapshot_nodes": 3,
"response_times": {
"samples": 96,
"p50_ms": 1013.8123999349773,
"p95_ms": 4014.4751999760047,
"p99_ms": 4015.4822999611497,
"max_ms": 4015.4822999611497
},
"same_leases_recovered": true
},
"seconds": 20.33888840000145,
"renewals": {
"samples": 2880,
"p50_ms": 8.835799992084503,
"p95_ms": 23.678299970924854,
"p99_ms": 35.42319999542087,
"max_ms": 102.84149996004999
},
"snapshots": {
"samples": 372,
"p50_ms": 13.98350007366389,
"p95_ms": 16.946100047789514,
"p99_ms": 20.020700059831142,
"max_ms": 20.943500101566315
},
"renewal_throughput": 141.6006589622565,
"node_pids": [
88080,
78036,
121352
],
"pid_kind": "docker-client",
"node_containers": [
"qqq-capacity-68ad627e7be5-node-58312",
"qqq-capacity-68ad627e7be5-node-58314",
"qqq-capacity-68ad627e7be5-node-58315"
],
"redis_slow_commands": [
{
"unix_time": 1793032016,
"duration_us": 2198,
"command": "EVALSHA"
}
],
"redis_latency_events": [
[
"command",
1793032016,
2,
2
]
],
"resources": [
{
"node_log": "node-58312.log",
"samples": 280,
"observed_wall_ms": 27899,
"cpu_seconds": 23.96,
"cpu_core_equivalents": 0.8588121438044375,
"heapUsedBytes_sampled_max": 164221400,
"heapCommittedBytes_sampled_max": 188743680,
"poolActive_sampled_max": 6,
"poolIdle_sampled_max": 5,
"poolWaiting_sampled_max": 15,
"poolMaximum_sampled_max": 8
},
{
"node_log": "node-58314.log",
"samples": 289,
"observed_wall_ms": 28800,
"cpu_seconds": 23.2,
"cpu_core_equivalents": 0.8055555555555556,
"heapUsedBytes_sampled_max": 162849016,
"heapCommittedBytes_sampled_max": 192937984,
"poolActive_sampled_max": 4,
"poolIdle_sampled_max": 3,
"poolWaiting_sampled_max": 14,
"poolMaximum_sampled_max": 8
},
{
"node_log": "node-58315.log",
"samples": 272,
"observed_wall_ms": 27100,
"cpu_seconds": 23.85,
"cpu_core_equivalents": 0.8800738007380073,
"heapUsedBytes_sampled_max": 165779696,
"heapCommittedBytes_sampled_max": 188743680,
"poolActive_sampled_max": 5,
"poolIdle_sampled_max": 4,
"poolWaiting_sampled_max": 14,
"poolMaximum_sampled_max": 8
}
],
"cleanup": true,
"receipt_note": "Java executable path normalized; measurements unchanged. Disposable fixture identities only."
}
@@ -0,0 +1,61 @@
{
"status": "FAIL",
"parameters": {
"java": "java (Java 21)",
"nodes": 3,
"tenants": 2,
"users": 12,
"tabs": 4,
"rounds": 1000,
"workers": 8
},
"scope": "loopback HTTP, mock identities, memory records",
"current_round": 480,
"completed_rounds": 479,
"error": "HTTP 503 node=56028 user=load-t0-u5 path=/presence/leases/renew: {\"error\":\"Presence is temporarily unavailable. Please retry.\",\"code\":\"ERROR\",\"message\":\"Presence is temporarily unavailable. Please retry.\",\"traceId\":\"cea30b82-e82b-49ec-af8a-64d8b6bd5af6\"}",
"redis_slow_commands": [],
"redis_latency_events": [],
"resources": [
{
"node_log": "node-56027.log",
"samples": 1067,
"observed_wall_ms": 106600,
"cpu_seconds": 94.328125,
"cpu_core_equivalents": 0.8848792213883677,
"heapUsedBytes_sampled_max": 160559696,
"heapCommittedBytes_sampled_max": 178257920,
"poolActive_sampled_max": 5,
"poolIdle_sampled_max": 5,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8
},
{
"node_log": "node-56028.log",
"samples": 1066,
"observed_wall_ms": 106501,
"cpu_seconds": 95.8125,
"cpu_core_equivalents": 0.8996394400052582,
"heapUsedBytes_sampled_max": 160251104,
"heapCommittedBytes_sampled_max": 178257920,
"poolActive_sampled_max": 5,
"poolIdle_sampled_max": 4,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8
},
{
"node_log": "node-56029.log",
"samples": 1066,
"observed_wall_ms": 106500,
"cpu_seconds": 95.25,
"cpu_core_equivalents": 0.8943661971830986,
"heapUsedBytes_sampled_max": 157995216,
"heapCommittedBytes_sampled_max": 175112192,
"poolActive_sampled_max": 5,
"poolIdle_sampled_max": 4,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8
}
],
"cleanup": true,
"receipt_note": "Java executable path normalized; measurements unchanged. Disposable fixture identities only."
}
+121 -29
View File
@@ -18,16 +18,25 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<properties>
<revision>0.1.0-SNAPSHOT</revision>
<!-- Only the published Redis API package is a permitted cross-qbit boundary. -->
<qbit.guardrails.allowedApiPackages>com.kingsrook.qbits.objectstorage.api,com.kingsrook.qbits.email.api,com.kingsrook.qbits.redis.api</qbit.guardrails.allowedApiPackages>
<qbit.guardrails.allowedQbitArtifacts>com.kingsrook.qbits:qbit-object-storage,com.kingsrook.qbits:qbit-email,com.kingsrook.qbits:*-api,com.kingsrook.qbits:qbit-redis</qbit.guardrails.allowedQbitArtifacts>
<revision>1.2.0-SNAPSHOT</revision>
<maven.compiler.source>21</maven.compiler.source>
<maven.compiler.target>21</maven.compiler.target>
<jacoco.version>0.8.15</jacoco.version>
<!-- qbit-email from Maven Central -->
<qbit-email.version>0.1.0-RC.11</qbit-email.version>
<!-- Core RC.22 (QBitMetaDataProducer, VirtualFieldValueProviderInterface, QServiceUnavailableException); BLOCKED until it resolves from Central/the registry in CI. -->
<qqq.version>4.1.0-RC.26</qqq.version>
<!-- qbit-redis first release candidate of release/0.1 (producer run 5119 did not publish it yet): BLOCKED, verify the published artifact before pushing. -->
<qbit-redis.version>0.2.0-RC.1</qbit-redis.version>
</properties>
<parent>
<groupId>com.kingsrook</groupId>
<artifactId>qbit-build-parent</artifactId>
<version>2.0.0</version>
<version>2.2.1</version>
</parent>
<groupId>com.kingsrook.qbits</groupId>
@@ -66,20 +75,51 @@
<tag>HEAD</tag>
</scm>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.kingsrook.qqq</groupId>
<artifactId>qqq-bom-pom</artifactId>
<version>${qqq.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>com.kingsrook.qqq</groupId>
<artifactId>qqq-backend-module-rdbms</artifactId>
</dependency>
<dependency>
<groupId>com.kingsrook.qbits</groupId>
<artifactId>qbit-redis</artifactId>
<version>${qbit-redis.version}</version>
<optional>true</optional>
</dependency>
<dependency>
<groupId>com.kingsrook.qbits</groupId>
<artifactId>qbit-object-storage</artifactId>
<version>0.1.0-RC.2</version>
<version>0.1.0-RC.10</version>
</dependency>
<dependency>
<groupId>com.kingsrook.qqq</groupId>
<artifactId>qqq-backend-core</artifactId>
</dependency>
<!-- optional: email through qbit-email when it is produced in the host instance (EmailCollaborationNotifier) -->
<dependency>
<groupId>com.kingsrook.qbits</groupId>
<artifactId>qbit-email</artifactId>
<version>${qbit-email.version}</version>
<optional>true</optional>
<exclusions>
<exclusion>
<groupId>com.kingsrook.qqq</groupId>
<artifactId>*</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.kingsrook.qqq</groupId>
<artifactId>qqq-middleware-javalin</artifactId>
@@ -95,10 +135,88 @@
<version>3.27.7</version>
<scope>test</scope>
</dependency>
<!-- upload journal and attachment transaction suites: real JDBC (H2, MySQL and PostgreSQL 17 containers) -->
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<version>2.4.240</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers</artifactId>
<version>2.0.3</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers-mysql</artifactId>
<version>2.0.3</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>42.7.8</version>
<scope>test</scope>
</dependency>
<!-- native PostgreSQL module (vendor/type "postgres"), version managed by the qqq BOM -->
<dependency>
<groupId>com.kingsrook.qqq</groupId>
<artifactId>qqq-backend-module-postgres</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>com.kingsrook.qqq</groupId>
<artifactId>qqq-backend-module-rdbms</artifactId>
</exclusion>
<exclusion>
<groupId>com.kingsrook.qqq</groupId>
<artifactId>qqq-backend-core</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
<build>
<resources>
<!-- help bundles carry the artifact version (QQQ help bundle provenance) -->
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
<includes>
<include>help/*.yaml</include>
<include>qbit-collaboration.properties</include>
</includes>
</resource>
<resource>
<directory>src/main/resources</directory>
<filtering>false</filtering>
<excludes>
<exclude>help/*.yaml</exclude>
<exclude>qbit-collaboration.properties</exclude>
</excludes>
</resource>
</resources>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId><artifactId>maven-resources-plugin</artifactId><version>3.3.1</version>
<executions><execution><id>package-license-notices</id><phase>process-resources</phase><goals><goal>copy-resources</goal></goals>
<configuration><outputDirectory>${project.build.outputDirectory}/META-INF</outputDirectory><resources><resource>
<directory>${project.basedir}</directory><filtering>false</filtering><includes><include>LICENSE</include><include>NOTICE</include></includes>
</resource></resources></configuration>
</execution></executions>
</plugin>
<!-- the artifact version, for the help bundle version test -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<systemPropertyVariables combine.children="append">
<qbit.artifactVersion>${project.version}</qbit.artifactVersion>
</systemPropertyVariables>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-checkstyle-plugin</artifactId>
@@ -177,30 +295,4 @@
</plugin>
</plugins>
</build>
<profiles>
<!-- Built against qqq 4.1 (QBitMetaDataProducer). Until qbit-build-parent -->
<!-- imports the released 4.1 BOM, this profile is active by default and -->
<!-- imports qqq-bom-pom 4.1.0-RC.6 ahead of the parent's 4.0.0 BOM. -->
<profile>
<id>qqq-snapshot</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<qqq.snapshot.version>4.1.0-RC.6</qqq.snapshot.version>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.kingsrook.qqq</groupId>
<artifactId>qqq-bom-pom</artifactId>
<version>${qqq.snapshot.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
</profile>
</profiles>
</project>
@@ -25,7 +25,7 @@ package com.kingsrook.qbits.collaboration;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.collaboration.notify.NoOpNotifier;
import java.util.function.Function;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordFeature;
@@ -46,9 +46,12 @@ public class CollaborationQBitConfig implements QBitConfig
private Map<String, List<RecordFeature>> tableFeatures = new LinkedHashMap<>();
private List<String> mirroredSecurityKeyTypes = List.of();
private QCodeReference notifier = new QCodeReference(NoOpNotifier.class);
private QCodeReference notifier = null;
private com.kingsrook.qbits.collaboration.presence.RedisPresenceConfig redisPresence;
private Function<String, String> recipientEmailResolver = null;
private QCodeReference userSessionProvider = null;
private String defaultTagMode = "FREE";
private Integer tagLookupCacheSeconds = 30;
private List<String> tagActionTables = List.of();
private List<String> followActionTables = List.of();
private List<String> followChangeTables = List.of();
@@ -63,6 +66,7 @@ public class CollaborationQBitConfig implements QBitConfig
private Boolean adminApp = true;
private Boolean uiApi = true;
private QCodeReference attachmentStorage = new QCodeReference(com.kingsrook.qbits.collaboration.attachments.AttachmentStorage.class);
private QCodeReference attachmentLifecycle = new QCodeReference(com.kingsrook.qbits.collaboration.attachments.AttachmentLifecyclePolicy.class);
private Integer recentsPerUser = 200;
private String attachmentsBucketName = "collaboration";
private Long defaultMaxBytes = 100L * 1024 * 1024;
@@ -102,6 +106,23 @@ public class CollaborationQBitConfig implements QBitConfig
return (this);
}
/*******************************************************************************
** Trusted host policy for retained attachments and access auditing.
*******************************************************************************/
public QCodeReference getAttachmentLifecycle()
{
return (attachmentLifecycle);
}
/*******************************************************************************
** Configure lifecycle enforcement without coupling to a consuming qbit.
*******************************************************************************/
public CollaborationQBitConfig withAttachmentLifecycle(QCodeReference attachmentLifecycle)
{
this.attachmentLifecycle = java.util.Objects.requireNonNull(attachmentLifecycle, "attachmentLifecycle");
return (this);
}
/*******************************************************************************
@@ -296,12 +317,16 @@ public class CollaborationQBitConfig implements QBitConfig
@Override
public void validate(QInstance instance, List<String> errors)
{
if(redisPresence != null)
{
redisPresence.validate(errors);
}
if(assertCondition(StringUtils.hasContent(backendName), "backendName is required for Collaboration", errors))
{
assertCondition(instance.getBackend(backendName) != null, "Unknown Collaboration backend: " + backendName, errors);
}
assertCondition(List.of("FREE", "APPROVED_ONLY").contains(defaultTagMode == null ? "" : defaultTagMode), "defaultTagMode must be FREE or APPROVED_ONLY", errors);
assertCondition(notifier != null, "notifier is required", errors);
assertCondition(tagLookupCacheSeconds != null && tagLookupCacheSeconds >= 0, "tagLookupCacheSeconds must be 0 (off) or more", errors);
for(List<String> tables : List.of(followActionTables, followChangeTables, tagActionTables, assignActionTables))
{
for(String table : tables)
@@ -424,13 +449,32 @@ public class CollaborationQBitConfig implements QBitConfig
}
/*******************************************************************************
** Get notifier.
** Get notifier. Null (the default) means: email through qbit-email when it is
** produced in the instance, otherwise no delivery transport.
*******************************************************************************/
public QCodeReference getNotifier()
{
return (notifier);
}
/*******************************************************************************
** Get the user id to email address lookup used by the email notifier.
*******************************************************************************/
public Function<String, String> getRecipientEmailResolver()
{
return (recipientEmailResolver);
}
/*******************************************************************************
** Configure how the email notifier finds a recipient address from a user id.
** Null (the default) uses the email stored in the collaboration people directory.
*******************************************************************************/
public CollaborationQBitConfig withRecipientEmailResolver(Function<String, String> recipientEmailResolver)
{
this.recipientEmailResolver = recipientEmailResolver;
return (this);
}
/*******************************************************************************
** Configure notifier.
*******************************************************************************/
@@ -508,6 +552,27 @@ public class CollaborationQBitConfig implements QBitConfig
return (this);
}
/*******************************************************************************
** How long a node remembers whether a table has any tag assignment.
*******************************************************************************/
public Integer getTagLookupCacheSeconds()
{
return (tagLookupCacheSeconds);
}
/*******************************************************************************
** Reads of a table without any tag assignment skip the assignment lookup while
** this node remembers that (default 30 seconds; 0 looks every time). A node
** that writes an assignment forgets at once and tells the others over the
** ClusterBus; without a cluster transport another node shows a table's first
** tag after at most this long. Tables that have assignments are always read.
*******************************************************************************/
public CollaborationQBitConfig withTagLookupCacheSeconds(Integer tagLookupCacheSeconds)
{
this.tagLookupCacheSeconds = tagLookupCacheSeconds;
return (this);
}
/*******************************************************************************
** Host tables receiving Add tags and Remove tags actions.
*******************************************************************************/
@@ -655,4 +720,21 @@ public class CollaborationQBitConfig implements QBitConfig
{
this.managedIndexes = managedIndexes;
}
/***************************************************************************
** Optional shared Redis presence; absent retains the existing memory provider.
***************************************************************************/
public com.kingsrook.qbits.collaboration.presence.RedisPresenceConfig getRedisPresence()
{
return (redisPresence);
}
/***************************************************************************
** Select shared presence with explicit host-derived tenant scope.
***************************************************************************/
public CollaborationQBitConfig withRedisPresence(com.kingsrook.qbits.collaboration.presence.RedisPresenceConfig value)
{
redisPresence = value;
return (this);
}
}
@@ -22,6 +22,9 @@
package com.kingsrook.qbits.collaboration;
import java.io.IOException;
import java.io.InputStream;
import java.util.Properties;
import com.kingsrook.qbits.collaboration.app.CollaborationHome;
import com.kingsrook.qbits.collaboration.attachments.DeleteAttachmentsProcess;
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
@@ -29,6 +32,7 @@ import com.kingsrook.qbits.collaboration.recents.TrimRecentsProcess;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerMultiOutput;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.help.bundles.HelpBundles;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSource;
import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitMetaData;
@@ -43,13 +47,41 @@ public class CollaborationQBitProducer implements QBitMetaDataProducer<Collabora
{
public static final String GROUP_ID = "com.kingsrook.qbits";
public static final String ARTIFACT_ID = "qbit-collaboration";
public static final String VERSION = "0.1.0";
public static final String VERSION = loadVersion();
private CollaborationQBitConfig qBitConfig;
private String namespace;
/*******************************************************************************
** Read the artifact version from the Maven-filtered properties resource.
*******************************************************************************/
private static String loadVersion()
{
try(InputStream inputStream = CollaborationQBitProducer.class.getClassLoader().getResourceAsStream("qbit-collaboration.properties"))
{
if(inputStream == null)
{
throw new IllegalStateException("Missing qbit-collaboration.properties; build with Maven to generate the artifact version.");
}
Properties properties = new Properties();
properties.load(inputStream);
String version = properties.getProperty("version");
if(version == null || version.isBlank() || version.contains("${"))
{
throw new IllegalStateException("Missing or unfiltered version in qbit-collaboration.properties; build with Maven resource filtering.");
}
return (version);
}
catch(IOException e)
{
throw new IllegalStateException("Could not read the artifact version from qbit-collaboration.properties.", e);
}
}
/*******************************************************************************
** Configure the qbit.
*******************************************************************************/
@@ -123,6 +155,10 @@ public class CollaborationQBitProducer implements QBitMetaDataProducer<Collabora
public MetaDataProducerMultiOutput produce(QInstance instance) throws QException
{
qBitConfig.validate(instance);
var realtime = com.kingsrook.qqq.backend.core.realtime.RealtimeMetaData.of(instance);
realtime.withAudienceProvider(com.kingsrook.qbits.collaboration.presence.RedisPresenceAudience.EVENT,
new com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference(com.kingsrook.qbits.collaboration.presence.RedisPresenceAudience.class));
instance.withSupplementalMetaData(realtime);
QBitMetaData qBit = getQBitMetaData();
instance.addQBit(qBit);
MetaDataProducerMultiOutput output = new MetaDataProducerMultiOutput();
@@ -151,6 +187,11 @@ public class CollaborationQBitProducer implements QBitMetaDataProducer<Collabora
com.kingsrook.qbits.collaboration.tags.TagProcesses.processesFor(table).forEach(output::add);
}
output.add(DeleteAttachmentsProcess.metadata());
output.add(com.kingsrook.qbits.collaboration.attachments.RetryAttachmentCleanupProcess.metadata(qBitConfig));
output.add(com.kingsrook.qbits.collaboration.attachments.ResolveAttachmentUploadProcess.metadata());
output.add(com.kingsrook.qbits.collaboration.attachments.RecoverRetainedUploadProcess.metadata());
output.add(com.kingsrook.qbits.collaboration.attachments.AdoptRetainedAttachmentProcess.metadata());
output.add(FeatureSettingsDefaultsProcess.process());
for(QPossibleValueSource source : CollaborationTables.possibleValueSources())
{
output.add(source);
@@ -162,7 +203,7 @@ public class CollaborationQBitProducer implements QBitMetaDataProducer<Collabora
}
if(Boolean.TRUE.equals(qBitConfig.getAdminApp()))
{
output.add(CollaborationHome.widget());
CollaborationHome.widgets().forEach(output::add);
output.add(CollaborationHome.app(qBitConfig));
}
output.addSelfToInstance(instance);
@@ -178,6 +219,8 @@ public class CollaborationQBitProducer implements QBitMetaDataProducer<Collabora
QAppMetaData parent = instance.getApp(qBitConfig.getParentAppName());
parent.withChild(instance.getApp(qBitConfig.getAppName()));
}
// Registration is metadata-only; core startup sync skips hosts without helpContent.
HelpBundles.register(instance, "help/qbit-collaboration.yaml");
return (new MetaDataProducerMultiOutput());
}
}
@@ -33,7 +33,8 @@ import com.kingsrook.qqq.backend.core.realtime.RealtimeBus;
import com.kingsrook.qqq.backend.core.realtime.RealtimeMessage;
/*******************************************************************************
** Identifier-only hints, published after successful commit.
** Identifier-only hints, published after successful commit. A tag assignment
** also clears what this node remembers about untagged tables (TaggedTables).
*******************************************************************************/
public class CollaborationRealtime implements RecordChangeListenerInterface
{
@@ -55,6 +56,7 @@ public class CollaborationRealtime implements RecordChangeListenerInterface
public void onRecordsChanged(RecordChangeEvent change) throws QException
{
var instance = QContext.getQInstance();
java.util.Set<String> newlyTaggedTables = new java.util.HashSet<>();
for(int index = 0; index < change.getRecords().size(); index++)
{
var row = new com.kingsrook.qqq.backend.core.model.data.QRecord(change.getRecords().get(index));
@@ -70,6 +72,13 @@ public class CollaborationRealtime implements RecordChangeListenerInterface
{
continue;
}
if(CollaborationTables.RECORD_TAG.equals(change.getTableName()) && change.getType() != RecordChangeType.DELETE && newlyTaggedTables.add(table))
{
////////////////////////////////////////////////////////////////////////
// the Tags field skips tables known to have no assignment: forget it //
////////////////////////////////////////////////////////////////////////
com.kingsrook.qbits.collaboration.tags.TaggedTables.assignmentsChanged(table, change.getTransaction());
}
String event = switch(change.getTableName())
{
case CollaborationTables.COMMENT -> change.getType() == RecordChangeType.DELETE || row.getValue("deletedAt") != null
@@ -23,6 +23,8 @@ package com.kingsrook.qbits.collaboration;
import java.io.Serializable;
import java.util.List;
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
import com.kingsrook.qqq.backend.core.actions.tables.DeleteAction;
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
@@ -36,6 +38,8 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordFeature;
import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordReferenceTableMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
/*******************************************************************************
@@ -74,7 +78,15 @@ public final class CollaborationRecords
*******************************************************************************/
public static List<QRecord> query(String table, QQueryFilter filter) throws QException
{
QueryInput input = new QueryInput(table).withFilter(filter).withShouldFetchHeavyFields(true).withTransaction(QContext.getQBackendTransaction());
return (query(table, filter, QContext.getQBackendTransaction()));
}
/*******************************************************************************
** Provider reads participate in the originating query's explicit transaction.
*******************************************************************************/
public static List<QRecord> query(String table, QQueryFilter filter, QBackendTransaction transaction) throws QException
{
QueryInput input = new QueryInput(table).withFilter(filter).withShouldFetchHeavyFields(true).withTransaction(transaction);
input.setTableMetaData(serviceMetadata(table));
input.withPermissionBypass("Trusted collaboration service performs parent/ownership authorization.");
return (new QueryAction().execute(input).getRecords().stream().map(QRecord::new).toList());
@@ -85,8 +97,34 @@ public final class CollaborationRecords
*******************************************************************************/
public static QRecord insert(String table, QRecord row) throws QException
{
InsertInput input = new InsertInput(table).withRecords(List.of(row)).withTransaction(QContext.getQBackendTransaction());
input.setTableMetaData(serviceMetadata(table));
return (insert(serviceMetadata(table), row));
}
/*******************************************************************************
** Activity belongs to the authorized source operation; timeline controls its
** presentation. Preserve native parent checks and mirrored security keys.
*******************************************************************************/
public static QRecord insertActivity(QRecord row, RecordFeature sourceFeature) throws QException
{
QTableMetaData metadata = serviceMetadata(CollaborationTables.ACTIVITY);
RecordReferenceTableMetaData original = RecordReferenceTableMetaData.of(metadata);
if(original == null || sourceFeature == null)
{
throw (new QException("Activity source authorization metadata is required."));
}
RecordReferenceTableMetaData reference = (RecordReferenceTableMetaData) original.clone();
reference.withFeature(sourceFeature);
metadata.withSupplementalMetaData(reference);
return (insert(metadata, row));
}
/*******************************************************************************
** A private metadata copy retains the same transaction and validation path.
*******************************************************************************/
private static QRecord insert(QTableMetaData metadata, QRecord row) throws QException
{
InsertInput input = new InsertInput(metadata.getName()).withRecords(List.of(row)).withTransaction(QContext.getQBackendTransaction());
input.setTableMetaData(metadata);
input.withPermissionBypass("Trusted collaboration service authorized this write.");
List<QRecord> result = new InsertAction().execute(input).getRecords();
check(result);
@@ -143,7 +181,10 @@ public final class CollaborationRecords
public static QTableMetaData cleanupMetadata(String table)
{
QTableMetaData metadata = serviceMetadata(table);
metadata.getSupplementalMetaData().remove(com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordReferenceTableMetaData.NAME);
if(metadata.getSupplementalMetaData() != null)
{
metadata.getSupplementalMetaData().remove(com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordReferenceTableMetaData.NAME);
}
metadata.setRecordSecurityLocks(null);
return (metadata);
}
@@ -57,7 +57,7 @@ public final class CollaborationSchema
for(QFieldMetaData field : table.getFields().values())
{
String type = field.getName().equals(table.getPrimaryKeyField()) ? "INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY" : sqlType(field);
columns.add(" " + field.getBackendName() + " " + type + (field.getName().equals("parentDeleted") ? " DEFAULT FALSE" : "") + (Boolean.TRUE.equals(field.getIsRequired()) ? " NOT NULL" : ""));
columns.add(" " + field.getBackendName() + " " + type + (field.getName().equals("parentDeleted") || field.getName().equals("retained") ? " DEFAULT FALSE" : "") + (Boolean.TRUE.equals(field.getIsRequired()) ? " NOT NULL" : ""));
}
statements.add("CREATE TABLE IF NOT EXISTS " + table.getName() + " (\n" + String.join(",\n", columns) + "\n)");
for(QFieldMetaData field : table.getFields().values())
@@ -78,6 +78,10 @@ public final class CollaborationSchema
}
statements.add("CREATE INDEX IF NOT EXISTS idx_collaboration_record_tag_tag ON collaboration_record_tag (tag_id)");
statements.add("CREATE INDEX IF NOT EXISTS idx_collaboration_attachment_parent ON collaboration_attachment (table_name, record_id)");
statements.add("ALTER TABLE collaboration_attachment ADD COLUMN IF NOT EXISTS retained BOOLEAN NOT NULL DEFAULT FALSE");
statements.add("ALTER TABLE collaboration_attachment ADD COLUMN IF NOT EXISTS intake_key VARCHAR(64)");
statements.add("ALTER TABLE collaboration_attachment ADD COLUMN IF NOT EXISTS intake_fingerprint VARCHAR(64)");
statements.add("CREATE UNIQUE INDEX IF NOT EXISTS uq_collaboration_attachment_intake ON collaboration_attachment (intake_key) WHERE intake_key IS NOT NULL");
statements.add("CREATE INDEX IF NOT EXISTS idx_collaboration_comment_parent ON collaboration_comment (table_name, record_id, last_activity_at, id)");
statements.add("CREATE INDEX IF NOT EXISTS idx_collaboration_comment_created ON collaboration_comment (table_name, record_id, create_date)");
statements.add("CREATE INDEX IF NOT EXISTS idx_collaboration_comment_thread ON collaboration_comment (thread_id, create_date, id)");
@@ -95,6 +99,15 @@ public final class CollaborationSchema
statements.add("CREATE INDEX IF NOT EXISTS idx_collaboration_assignment_owner ON collaboration_record_assignment (owner_user_id)");
statements.add("CREATE INDEX IF NOT EXISTS idx_collaboration_assignment_assignee ON collaboration_record_assignment (assignee_user_id)");
statements.add("ALTER TABLE collaboration_mention ADD COLUMN IF NOT EXISTS read_at TIMESTAMPTZ");
statements.add("ALTER TABLE collaboration_attachment_cleanup ADD COLUMN IF NOT EXISTS next_attempt_at TIMESTAMPTZ");
statements.add("ALTER TABLE collaboration_attachment_cleanup ADD COLUMN IF NOT EXISTS resolved_at TIMESTAMPTZ");
statements.add("ALTER TABLE collaboration_attachment_cleanup ADD COLUMN IF NOT EXISTS resolved_by VARCHAR(255)");
statements.add("ALTER TABLE collaboration_attachment_cleanup ADD COLUMN IF NOT EXISTS settlement_evidence VARCHAR(2000)");
statements.add("ALTER TABLE collaboration_attachment_cleanup ALTER COLUMN attachment_id DROP NOT NULL");
statements.add("DROP INDEX IF EXISTS uq_collaboration_attachment_cleanup_1");
statements.add("CREATE INDEX IF NOT EXISTS idx_collaboration_attachment_storage_key ON collaboration_attachment USING HASH (storage_key)");
statements.add("CREATE INDEX IF NOT EXISTS idx_collaboration_attachment_cleanup_due ON collaboration_attachment_cleanup (next_attempt_at, id)");
statements.add("CREATE INDEX IF NOT EXISTS idx_collaboration_attachment_cleanup_pending ON collaboration_attachment_cleanup (next_attempt_at, id) WHERE resolved_at IS NULL");
statements.add("CREATE INDEX IF NOT EXISTS idx_collaboration_mention_unread ON collaboration_mention (mentioned_user_id) WHERE notified_at IS NOT NULL AND read_at IS NULL");
return (String.join(";\n", statements) + ";\n");
}
@@ -49,7 +49,7 @@ public class CollaborationSummaryProvider implements RecordFeatureSummaryProvide
String user = PersonalRecords.currentUserId();
if(features.isEnabled(RecordFeature.PRESENCE))
{
summary.setPresence(com.kingsrook.qbits.collaboration.presence.PresenceRegistry.get(QContext.getQInstance()).snapshot(ref.tableName(), id).stream().map(viewer ->
summary.setPresence(new com.kingsrook.qbits.collaboration.presence.PresenceService().snapshot(ref.tableName(), id).stream().map(viewer ->
{
java.util.Map<String, java.io.Serializable> entry = new java.util.LinkedHashMap<>();
entry.put("userId", viewer.userId());
@@ -68,6 +68,7 @@ public final class CoreCollaboration
{
config.applyFeatures(instance);
var features = RecordFeaturesMetaData.ofOrWithNew(instance);
features.withFollowersProvider(new QCodeReference(com.kingsrook.qbits.collaboration.follow.CollaborationFollowersProvider.class));
features.withOverrideProvider(new QCodeReference(FeatureSettingsProvider.class));
features.getRuntimeProviders().add(new QCodeReference(com.kingsrook.qbits.collaboration.tags.TagsMetaDataProvider.class));
features.getRuntimeProviders().add(new QCodeReference(com.kingsrook.qbits.collaboration.customfields.CustomFieldsProvider.class));
@@ -29,10 +29,12 @@ import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterfa
import com.kingsrook.qqq.backend.core.context.CapturedContext;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.instances.QInstanceHelpContentManager;
import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
/*******************************************************************************
** Publish committed policy changes through the core overlay.
@@ -77,7 +79,7 @@ public class FeatureSettingsCustomizer implements TableCustomizerInterface
var instance = QContext.getQInstance();
if(transaction == null)
{
RuntimeMetaDataOverlay.refresh(instance);
refreshMetadata(instance);
}
else
{
@@ -86,7 +88,7 @@ public class FeatureSettingsCustomizer implements TableCustomizerInterface
{
try
{
QContext.withTemporaryContext(context, () -> RuntimeMetaDataOverlay.refresh(instance));
QContext.withTemporaryContext(context, () -> refreshMetadata(instance));
}
catch(QException e)
{
@@ -95,4 +97,13 @@ public class FeatureSettingsCustomizer implements TableCustomizerInterface
});
}
}
/*******************************************************************************
** Bind persisted help after the overlay creates its dynamic fields and sections.
*******************************************************************************/
private static void refreshMetadata(QInstance instance) throws QException
{
RuntimeMetaDataOverlay.refresh(instance);
QInstanceHelpContentManager.reloadHelpContent(instance);
}
}
@@ -0,0 +1,278 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.TreeSet;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.collaboration.FeaturePermissionMode;
import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordFeature;
import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordFeaturesMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
/*******************************************************************************
** The collaboration_feature_setting rows that say, in the database, exactly
** what the code configuration already says: one INSTANCE row per feature
** (enabled for the configured default features, disabled for the rest, with
** the default permission mode and the TAGS mode), plus an APP or TABLE row
** wherever the code declares something at that scope.
**
** Why rows computed at run time and not a SQL seed: the right rows depend on
** the host's configuration (withDefaultFeatures, withTableFeatures, the
** default tag mode, features declared on apps and tables), which SQL cannot
** see.
**
** Why every declaring scope gets its own row: runtime overrides are applied
** AFTER the code declarations, instance first, then app, then table. A bare
** INSTANCE row would therefore overwrite what a table declares in code.
** Restating each declaring scope in its own row keeps the resolved result of
** every table identical with and without the rows (FeatureSettingsDefaultsTest
** proves that for several configurations).
**
** Rows are only ever created when the table is empty, so they can never
** disagree with rows an administrator already wrote.
*******************************************************************************/
public final class FeatureSettingsDefaults
{
public static final String INSTANCE_SCOPE_NAME = "instance";
private static final ObjectMapper MAPPER = new ObjectMapper();
/*******************************************************************************
** Utility class.
*******************************************************************************/
private FeatureSettingsDefaults()
{
}
/*******************************************************************************
** The rows to create: the configured rows when the table is empty, none
** when an administrator already has rows.
*******************************************************************************/
public static List<QRecord> missingRows(QInstance instance, CollaborationQBitConfig config) throws QException
{
if(!CollaborationRecords.query(CollaborationTables.FEATURE_SETTING, new QQueryFilter().withLimit(1)).isEmpty())
{
return (List.of());
}
return (configuredRows(instance, config));
}
/*******************************************************************************
** Every row the configuration implies (whether or not it exists yet).
*******************************************************************************/
public static List<QRecord> configuredRows(QInstance instance, CollaborationQBitConfig config)
{
List<QRecord> rows = new ArrayList<>();
///////////////////////////////////////////////////////////
// instance: every feature, so each one has a visible row //
///////////////////////////////////////////////////////////
RecordFeaturesMetaData instanceLevel = RecordFeaturesMetaData.of(instance);
for(RecordFeature feature : RecordFeature.values())
{
Map<String, Serializable> settings = new LinkedHashMap<>(declaredSettings(instanceLevel, feature));
if(feature == RecordFeature.TAGS)
{
settings.putIfAbsent("mode", config.getDefaultTagMode());
}
FeaturePermissionMode mode = declaredMode(instanceLevel, feature);
rows.add(row("INSTANCE", INSTANCE_SCOPE_NAME, feature, enabledThrough(feature, instanceLevel), mode == null ? FeaturePermissionMode.EDIT_OR_GRANTED : mode, settings));
}
/////////////////////////////////////////////////////////////////////////////
// apps and tables: only where the code declares something for the feature //
/////////////////////////////////////////////////////////////////////////////
for(QAppMetaData app : CollectionUtils.nonNullMap(instance.getApps()).values())
{
RecordFeaturesMetaData level = RecordFeaturesMetaData.of(app);
for(RecordFeature feature : declared(level))
{
rows.add(row("APP", app.getName(), feature, enabledThrough(feature, instanceLevel, level), modeThrough(feature, instanceLevel, level), declaredSettings(level, feature)));
}
}
for(QTableMetaData table : CollectionUtils.nonNullMap(instance.getTables()).values())
{
if(table.getName().startsWith("collaboration_"))
{
continue;
}
RecordFeaturesMetaData level = RecordFeaturesMetaData.of(table);
List<RecordFeaturesMetaData> apps = new ArrayList<>();
for(QAppMetaData app : CollectionUtils.nonNullMap(instance.getApps()).values())
{
boolean contains = CollectionUtils.nonNullList(app.getSections()).stream().anyMatch(section -> CollectionUtils.nonNullList(section.getTables()).contains(table.getName()));
if(contains && RecordFeaturesMetaData.of(app) != null)
{
apps.add(RecordFeaturesMetaData.of(app));
}
}
///////////////////////////////////////////////////////////////////////////
// a table row restates the whole chain (instance, its apps, the table). //
// Needed where the table declares something itself, and where several //
// apps declare the same feature: one row per app cannot restate the //
// combination, but the last override, the table row, can. //
///////////////////////////////////////////////////////////////////////////
Set<RecordFeature> features = new TreeSet<>(declared(level));
if(apps.size() > 1)
{
apps.forEach(app -> features.addAll(declared(app)));
}
List<RecordFeaturesMetaData> chain = new ArrayList<>();
chain.add(instanceLevel);
chain.addAll(apps);
chain.add(level);
RecordFeaturesMetaData[] levels = chain.toArray(new RecordFeaturesMetaData[0]);
for(RecordFeature feature : features)
{
Map<String, Serializable> settings = new LinkedHashMap<>();
for(RecordFeaturesMetaData each : levels)
{
settings.putAll(declaredSettings(each, feature));
}
rows.add(row("TABLE", table.getName(), feature, enabledThrough(feature, levels), modeThrough(feature, levels), settings));
}
}
return (rows);
}
/*******************************************************************************
** The features a scope declares anything about (enabled, mode or settings).
*******************************************************************************/
private static Set<RecordFeature> declared(RecordFeaturesMetaData level)
{
if(level == null)
{
return (Set.of());
}
return (Stream.of(level.getEnabled().keySet(), level.getPermissionModes().keySet(), level.getSettings().keySet())
.flatMap(Set::stream).collect(Collectors.toCollection(TreeSet::new)));
}
/*******************************************************************************
** Enabled as the code resolves it through the given levels, in order, the
** same way core does (a later level that declares the feature wins).
*******************************************************************************/
private static boolean enabledThrough(RecordFeature feature, RecordFeaturesMetaData... levels)
{
boolean enabled = false;
for(RecordFeaturesMetaData level : levels)
{
if(level != null && level.getEnabled().containsKey(feature))
{
enabled = Boolean.TRUE.equals(level.getEnabled().get(feature));
}
}
return (enabled);
}
/*******************************************************************************
** The permission mode through the given levels (a later declaration wins),
** EDIT_OR_GRANTED when none declares one, as core resolves it.
*******************************************************************************/
private static FeaturePermissionMode modeThrough(RecordFeature feature, RecordFeaturesMetaData... levels)
{
FeaturePermissionMode mode = FeaturePermissionMode.EDIT_OR_GRANTED;
for(RecordFeaturesMetaData level : levels)
{
if(declaredMode(level, feature) != null)
{
mode = declaredMode(level, feature);
}
}
return (mode);
}
/*******************************************************************************
**
*******************************************************************************/
private static FeaturePermissionMode declaredMode(RecordFeaturesMetaData level, RecordFeature feature)
{
return (level == null ? null : level.getPermissionModes().get(feature));
}
/*******************************************************************************
**
*******************************************************************************/
private static Map<String, Serializable> declaredSettings(RecordFeaturesMetaData level, RecordFeature feature)
{
return (level == null || level.getSettings().get(feature) == null ? Map.of() : level.getSettings().get(feature));
}
/*******************************************************************************
** One feature_setting record; no settings means a null settings column.
*******************************************************************************/
private static QRecord row(String scopeType, String scopeName, RecordFeature feature, boolean enabled, FeaturePermissionMode mode, Map<String, Serializable> settings)
{
QRecord record = new QRecord().withValue("scopeType", scopeType).withValue("scopeName", scopeName).withValue("feature", feature.name())
.withValue("enabled", enabled).withValue("permissionMode", mode == null ? null : mode.name());
if(!settings.isEmpty())
{
try
{
record.withValue("settings", MAPPER.writeValueAsString(settings));
}
catch(JsonProcessingException e)
{
throw (new IllegalStateException("Feature settings are not serializable", e));
}
}
return (record);
}
}
@@ -0,0 +1,274 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
import com.kingsrook.qbits.collaboration.permissions.CollaborationPermissions;
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
import com.kingsrook.qqq.backend.core.actions.permissions.CustomPermissionChecker;
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.MetaDataWithPermissionRules;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.PermissionLevel;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRules;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
import com.kingsrook.qqq.backend.core.model.session.QSession;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractExtractStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractLoadStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractTransformStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ProcessSummaryProviderInterface;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
/*******************************************************************************
** "Create rows from configuration": a table action on Feature Settings that
** writes the code configuration out as editable rows (FeatureSettingsDefaults).
** It changes no behavior on its own, only makes the effective settings visible
** and editable. It does nothing once the table has any row. Standard streamed
** ETL: Review lists the rows that will be created, Result the rows created.
*******************************************************************************/
public final class FeatureSettingsDefaultsProcess
{
public static final String NAME = "collaborationCreateFeatureSettings";
/*******************************************************************************
** Utility class.
*******************************************************************************/
private FeatureSettingsDefaultsProcess()
{
}
/*******************************************************************************
** The process: a table-level action (no records need to be selected).
*******************************************************************************/
public static QProcessMetaData process()
{
Map<String, Serializable> values = new HashMap<>();
values.put(StreamedETLWithFrontendProcess.FIELD_SOURCE_TABLE, CollaborationTables.FEATURE_SETTING);
values.put(StreamedETLWithFrontendProcess.FIELD_DESTINATION_TABLE, CollaborationTables.FEATURE_SETTING);
values.put(StreamedETLWithFrontendProcess.FIELD_SUPPORTS_FULL_VALIDATION, false);
values.put(StreamedETLWithFrontendProcess.FIELD_PREVIEW_MESSAGE,
"These rows will be created from the current configuration. Nothing changes: the rows say what the configuration already says, and you can edit them afterwards.");
QProcessMetaData process = StreamedETLWithFrontendProcess.defineProcessMetaData(Extract.class, Transform.class, Load.class, values);
process.setName(NAME);
process.setLabel("Create rows from configuration");
process.setIcon(new QIcon("playlist_add"));
process.setTableName(CollaborationTables.FEATURE_SETTING);
process.setMinInputRecords(0);
process.setPermissionRules(new QPermissionRules().withLevel(PermissionLevel.HAS_ACCESS_PERMISSION).withPermissionBaseName("collaboration")
.withCustomPermissionChecker(new QCodeReference(AdminOnly.class)));
return (process);
}
/*******************************************************************************
** Collaboration administrators only (the table itself is admin-write).
*******************************************************************************/
public static class AdminOnly implements CustomPermissionChecker
{
/*******************************************************************************
**
*******************************************************************************/
@Override
public void checkPermissionsThrowing(AbstractActionInput input, MetaDataWithPermissionRules metadata) throws QPermissionDeniedException
{
QSession session = QContext.getQSession();
if(session == null || !session.hasPermission(CollaborationPermissions.ADMIN))
{
throw (new QPermissionDeniedException("This needs the " + CollaborationPermissions.ADMIN + " permission."));
}
}
}
/*******************************************************************************
** Extract: the configured rows (none when the table already has rows).
*******************************************************************************/
public static class Extract extends AbstractExtractStep
{
/*******************************************************************************
**
*******************************************************************************/
@Override
public Integer doCount(RunBackendStepInput input) throws QException
{
return (FeatureSettingsDefaults.missingRows(QContext.getQInstance(), CollaborationRecords.config()).size());
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
for(QRecord row : FeatureSettingsDefaults.missingRows(QContext.getQInstance(), CollaborationRecords.config()))
{
getRecordPipe().addRecord(row);
}
}
}
/*******************************************************************************
** Transform: counts by scope for the review screen.
*******************************************************************************/
public static class Transform extends AbstractTransformStep implements ProcessSummaryProviderInterface
{
private final Map<String, ProcessSummaryLine> byScope = new HashMap<>();
/*******************************************************************************
**
*******************************************************************************/
@Override
public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
for(QRecord record : input.getRecords())
{
String scope = record.getValueString("scopeType");
byScope.computeIfAbsent(scope, s -> line(Status.OK, scope + " setting will be created", scope + " settings will be created",
scope + " setting was created", scope + " settings were created")).incrementCount();
output.getRecords().add(record);
}
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean isForResultScreen)
{
ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>();
byScope.values().forEach(line -> line.addSelfToListIfAnyCount(lines));
if(lines.isEmpty())
{
lines.add(new ProcessSummaryLine(Status.INFO, null, "Feature settings already has rows, so nothing was created. Edit the rows directly."));
}
return (lines);
}
}
/*******************************************************************************
** Load: insert through the standard action with the administrator's own session.
*******************************************************************************/
public static class Load extends AbstractLoadStep implements ProcessSummaryProviderInterface
{
private final ProcessSummaryLine created = line(Status.OK, "setting was created", "settings were created", "setting was created", "settings were created");
/*******************************************************************************
** One transaction for the whole load: the override refresh waits for the
** commit, so a failure on a later page can never leave some rows (for
** example Instance rows without the App and Table rows that restate the
** code) in effect.
*******************************************************************************/
@Override
public Optional<QBackendTransaction> openTransaction(RunBackendStepInput input) throws QException
{
return (Optional.of(QBackendTransaction.openFor(new InsertInput(CollaborationTables.FEATURE_SETTING))));
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
List<QRecord> inserted = new InsertAction().execute(new InsertInput(CollaborationTables.FEATURE_SETTING).withRecords(input.getRecords())
.withTransaction(getTransaction().orElse(null))).getRecords();
for(QRecord record : inserted)
{
if(record.getErrors() != null && !record.getErrors().isEmpty())
{
throw (new QException("Could not create feature settings: " + record.getErrors()));
}
created.incrementCount();
output.getRecords().add(record);
}
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean isForResultScreen)
{
ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>();
created.addSelfToListIfAnyCount(lines);
return (lines);
}
}
/*******************************************************************************
** A summary line with its four messages.
*******************************************************************************/
private static ProcessSummaryLine line(Status status, String singularFuture, String pluralFuture, String singularPast, String pluralPast)
{
ProcessSummaryLine line = new ProcessSummaryLine(status, 0, null);
line.setSingularFutureMessage(singularFuture);
line.setPluralFutureMessage(pluralFuture);
line.setSingularPastMessage(singularPast);
line.setPluralPastMessage(pluralPast);
return (line);
}
}
@@ -35,6 +35,7 @@ import com.kingsrook.qbits.collaboration.favorites.FavoriteEntry;
import com.kingsrook.qqq.backend.core.actions.permissions.PermissionEnforcement;
import com.kingsrook.qqq.backend.core.actions.permissions.PermissionsHelper;
import com.kingsrook.qqq.backend.core.actions.permissions.TablePermissionSubType;
import com.kingsrook.qqq.backend.core.actions.permissions.fieldaccess.FieldAccessHelper;
import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.context.QContext;
@@ -55,7 +56,7 @@ import com.kingsrook.qqq.backend.core.utils.ValueUtils;
/*******************************************************************************
** Resolves targets only with released RC.5 APIs. Each list batches record label
** Resolves targets through core permission and field-access APIs. Each list batches record label
** refreshes by table, preserving table permissions and record security locks.
** Next accepts the flat /app/name/ routes, including dotted process names.
*******************************************************************************/
@@ -131,6 +132,17 @@ public final class TargetResolver
/*******************************************************************************
** Persist record labels only when all label fields are FULL for this session.
*******************************************************************************/
public static String cacheableLabel(FavoriteEntry target)
{
return ("RECORD".equals(target.targetType())
&& !FieldAccessHelper.isRecordLabelFullAccess(QContext.getQInstance().getTable(target.targetName())) ? null : target.label());
}
/*******************************************************************************
** One QueryAction per referenced record table; no cached labels escape.
*******************************************************************************/
@@ -276,12 +288,16 @@ public final class TargetResolver
/*******************************************************************************
** Guard existence before invoking the core table permission helper.
** Guard existence before invoking the core table permission helper. The read
** is asked as the query it stands for: core hands a table's custom permission
** checker the action input and not the permission asked, so a checker that
** tells a read from a write by the input's type refuses an untyped input, and
** its table was then no target for the people it lets read it.
*******************************************************************************/
private static boolean canReadTable(String table)
{
return (table != null && QContext.getQInstance().getTable(table) != null
&& PermissionsHelper.hasTablePermission(new AbstractActionInput(), table, TablePermissionSubType.READ));
&& PermissionsHelper.hasTablePermission(new QueryInput(table), table, TablePermissionSubType.READ));
}
@@ -28,6 +28,7 @@ import com.kingsrook.qbits.collaboration.model.CollaborationTables;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordFeature;
/*******************************************************************************
** Internal activity writer; the calling service has already checked the parent.
@@ -46,8 +47,28 @@ public final class ActivityWriter
*******************************************************************************/
public static void write(String table, String recordId, String type, String summary, String refTable, String refId) throws QException
{
CollaborationRecords.insert(CollaborationTables.ACTIVITY, new QRecord().withValue("tableName", table).withValue("recordId", recordId)
CollaborationRecords.insertActivity(new QRecord().withValue("tableName", table).withValue("recordId", recordId)
.withValue("at", Instant.now()).withValue("actorUserId", CollaborationSecurity.userId(QContext.getQSession()))
.withValue("type", type).withValue("summary", summary).withValue("refTableName", refTable).withValue("refId", refId).withValue("parentDeleted", false));
.withValue("type", type).withValue("summary", summary).withValue("refTableName", refTable).withValue("refId", refId).withValue("parentDeleted", false), sourceFeature(refTable));
}
/*******************************************************************************
** Source-specific activity requires that feature; generic events retain the
** existing timeline authorization contract.
*******************************************************************************/
private static RecordFeature sourceFeature(String table)
{
return (switch(table == null ? "" : table)
{
case CollaborationTables.ATTACHMENT -> RecordFeature.ATTACHMENTS;
case CollaborationTables.COMMENT, CollaborationTables.COMMENT_REVISION, CollaborationTables.MENTION -> RecordFeature.COMMENTS;
case CollaborationTables.TAG, CollaborationTables.RECORD_TAG -> RecordFeature.TAGS;
case CollaborationTables.FIELD_DEFINITION, CollaborationTables.FIELD_VALUE -> RecordFeature.CUSTOM_FIELDS;
case CollaborationTables.TASK -> RecordFeature.TASKS;
case CollaborationTables.RECORD_ASSIGNMENT -> RecordFeature.ASSIGNMENT;
case CollaborationTables.FOLLOW, CollaborationTables.FOLLOW_DIGEST -> RecordFeature.FOLLOW;
case CollaborationTables.SHARE -> RecordFeature.SHARING;
default -> RecordFeature.TIMELINE;
});
}
}
@@ -52,9 +52,10 @@ public final class CollaborationApi
*******************************************************************************/
public static MiddlewareSpecExtension extension()
{
return (new MiddlewareSpecExtension().withName("collaboration").withSemanticVersion("1.0.0")
return (new MiddlewareSpecExtension().withName("collaboration").withSemanticVersion("1.3.0")
.withDescription("Record collaboration: favorites, recents, attachments, tags, conversations, following, timeline, people and presence")
.withSpec(PostPresenceSpec.class).withSpec(DeletePresenceSpec.class).withSpec(GetPresenceSpec.class)
.withSpec(OpenPresenceLeaseSpec.class).withSpec(RenewPresenceLeaseSpec.class)
.withSpec(ListFavoritesSpec.class)
.withSpec(StarFavoriteSpec.class)
.withSpec(UnstarFavoriteSpec.class)
@@ -96,6 +97,7 @@ public final class CollaborationApi
.withSpec(ListMyTasksSpec.class)
.withSpec(PutAssignmentSpec.class)
.withSpec(GetMyWorkSpec.class)
.withSpec(MarkMentionsReadSpec.class)
.withSpec(UploadAttachmentSpec.class));
}
}
@@ -46,6 +46,7 @@ import com.kingsrook.qqq.backend.core.exceptions.QBadRequestException;
import com.kingsrook.qqq.middleware.javalin.specs.BasicOperation;
import com.kingsrook.qqq.middleware.javalin.specs.BasicResponse;
import com.kingsrook.qqq.middleware.javalin.specs.extensions.AbstractJsonEndpointSpec;
import com.kingsrook.qqq.middleware.javalin.specs.v1.responses.BasicErrorResponseV1;
import com.kingsrook.qqq.openapi.model.HttpMethod;
import com.kingsrook.qqq.openapi.model.In;
import com.kingsrook.qqq.openapi.model.Parameter;
@@ -112,10 +113,11 @@ abstract class CollaborationJsonSpec extends AbstractJsonEndpointSpec
*******************************************************************************/
static List<BasicResponse> errors()
{
return (List.of(new BasicResponse(HttpStatus.BAD_REQUEST, "Invalid input", "CollaborationErrorV1"),
new BasicResponse(HttpStatus.UNAUTHORIZED, "Sign in required", "CollaborationErrorV1"),
new BasicResponse(HttpStatus.FORBIDDEN, "Permission denied", "CollaborationErrorV1"),
new BasicResponse(HttpStatus.NOT_FOUND, "Not found (including invisible parents)", "CollaborationErrorV1")));
return (List.of(new BasicResponse(HttpStatus.BAD_REQUEST, "Invalid input", BasicErrorResponseV1.class.getSimpleName()),
new BasicResponse(HttpStatus.UNAUTHORIZED, "Sign in required", BasicErrorResponseV1.class.getSimpleName()),
new BasicResponse(HttpStatus.FORBIDDEN, "Permission denied", BasicErrorResponseV1.class.getSimpleName()),
new BasicResponse(HttpStatus.NOT_FOUND, "Not found (including invisible parents)", BasicErrorResponseV1.class.getSimpleName()),
new BasicResponse(HttpStatus.SERVICE_UNAVAILABLE, "Service temporarily unavailable", BasicErrorResponseV1.class.getSimpleName())));
}
/*******************************************************************************
@@ -212,6 +214,19 @@ abstract class CollaborationJsonSpec extends AbstractJsonEndpointSpec
protected static <T> Page<T> page(JsonInput input, List<T> values) throws QBadRequestException
{
int limit = limit(input);
PagePosition position = pagePosition(input);
int start = Math.min(position.offset(), values.size());
int end = start + Math.min(limit, values.size() - start);
String next = end < values.size() ? position.cursor(end) : null;
return (new Page<>(values.subList(start, end), next));
}
/*******************************************************************************
** Shared existing offset cursor contract, also used by bounded native lists.
** Scope is a routing check, never authorization or a signed capability.
*******************************************************************************/
protected static PagePosition pagePosition(JsonInput input) throws QBadRequestException
{
int offset = 0;
String scope = input.getSpec().defineBasicOperation().getPath() + input.getPathParams().toString()
+ ":" + input.getQueryParam("table") + ":" + input.getQueryParam("pk") + ":" + input.getQueryParam("q");
@@ -237,10 +252,25 @@ abstract class CollaborationJsonSpec extends AbstractJsonEndpointSpec
throw (new QBadRequestException("Invalid cursor"));
}
}
int start = Math.min(offset, values.size());
int end = start + Math.min(limit, values.size() - start);
String next = end < values.size() ? Base64.getUrlEncoder().withoutPadding().encodeToString((prefix + end).getBytes(StandardCharsets.UTF_8)) : null;
return (new Page<>(values.subList(start, end), next));
return (new PagePosition(offset, prefix));
}
/*******************************************************************************
** Same version, scope and offset bytes as previously emitted by page().
*******************************************************************************/
protected record PagePosition(int offset, String prefix)
{
/*******************************************************************************
** Encode only a representable nonnegative continuation offset.
*******************************************************************************/
String cursor(long next) throws QBadRequestException
{
if(next < 0 || next > Integer.MAX_VALUE)
{
throw (new QBadRequestException("Invalid cursor"));
}
return (Base64.getUrlEncoder().withoutPadding().encodeToString((prefix + next).getBytes(StandardCharsets.UTF_8)));
}
}
/*******************************************************************************
@@ -41,11 +41,67 @@ public final class CollaborationPresenceSchemas
{
}
/*******************************************************************************
** Target for invisible server-issued renewal authority.
*******************************************************************************/
public record CollaborationPresenceOpenV1(String table, String pk) implements ToSchema
{
/***************************************************************************
** Opening alone never advertises a viewer.
***************************************************************************/
@Override
public Schema toSchema()
{
return (new Schema().withType(Type.OBJECT).withProperties(new TreeMap<>(Map.of(
"table", text("Parent table name.", false), "pk", text("Optional record key.", true)))));
}
}
/*******************************************************************************
** Opaque handle bound to the authenticated caller and target.
*******************************************************************************/
public record CollaborationPresenceLeaseV1(String connectionId) implements ToSchema
{
/***************************************************************************
** Activate with a positive sequence; acquire a new handle after expiry.
***************************************************************************/
@Override
public Schema toSchema()
{
return (new Schema().withType(Type.OBJECT).withProperties(Map.of(
"connectionId", text("Server-issued handle. Renew before 45 seconds; departure is terminal.", false))));
}
}
/*******************************************************************************
** Stable machine-readable response for missing renewal authority.
*******************************************************************************/
public record CollaborationPresenceExpiredV1(String code) implements ToSchema
{
/***************************************************************************
** Clients rejoin only if their surface is still active and visible.
***************************************************************************/
@Override
public Schema toSchema()
{
return (new Schema().withType(Type.OBJECT).withProperties(Map.of(
"code", text("Open a fresh lease before retrying renewal.", false).withEnumValues(List.of("PRESENCE_LEASE_EXPIRED")))));
}
}
/*******************************************************************************
** All identity information is derived from the authenticated session.
*******************************************************************************/
public record CollaborationPresenceRequestV1(String connectionId, String table, String pk, String mode, String fieldName) implements ToSchema
public record CollaborationPresenceRequestV1(String connectionId, String table, String pk, String mode, String fieldName, Long sequence) implements ToSchema
{
/*******************************************************************************
** Preserve source callers that do not yet send sequence numbers.
*******************************************************************************/
public CollaborationPresenceRequestV1(String connectionId, String table, String pk, String mode, String fieldName)
{
this(connectionId, table, pk, mode, fieldName, null);
}
/*******************************************************************************
** Clients renew every twenty seconds; the lease expires after forty-five.
*******************************************************************************/
@@ -57,15 +113,24 @@ public final class CollaborationPresenceSchemas
"connectionId", text("Client connection identity.", false), "table", text("Parent table name.", false),
"pk", text("Optional parent key; omit for a table view.", true),
"mode", text("VIEWING or EDITING.", false).withEnumValues(List.of("VIEWING", "EDITING")),
"fieldName", text("Optional focused field on a record.", true)))));
"fieldName", text("Optional focused field on a record.", true),
"sequence", sequenceSchema()))));
}
}
/*******************************************************************************
** Leave removes only this authenticated user's connection and target.
*******************************************************************************/
public record CollaborationPresenceLeaveV1(String connectionId, String table, String pk) implements ToSchema
public record CollaborationPresenceLeaveV1(String connectionId, String table, String pk, Long sequence) implements ToSchema
{
/*******************************************************************************
** Preserve source callers that do not yet send sequence numbers.
*******************************************************************************/
public CollaborationPresenceLeaveV1(String connectionId, String table, String pk)
{
this(connectionId, table, pk, null);
}
/*******************************************************************************
** Leave does not need a mode or a field.
*******************************************************************************/
@@ -75,7 +140,7 @@ public final class CollaborationPresenceSchemas
return (new Schema().withType(Type.OBJECT).withDescription("Remove the caller's lease on this connection and target.")
.withProperties(new TreeMap<>(Map.of(
"connectionId", text("Client connection identity.", false), "table", text("Parent table name.", false),
"pk", text("Optional parent key.", true)))));
"pk", text("Optional parent key.", true), "sequence", sequenceSchema()))));
}
}
@@ -122,4 +187,13 @@ public final class CollaborationPresenceSchemas
{
return (new Schema().withType(Type.STRING).withDescription(description).withNullable(nullable));
}
/*******************************************************************************
** Optional for legacy clients; new clients increment on every renewal and leave.
*******************************************************************************/
private static Schema sequenceSchema()
{
return (new Schema().withType(Type.INTEGER).withNullable(true)
.withDescription("Monotonically increasing nonnegative safe integer per connection and target; orders delayed requests."));
}
}
@@ -70,8 +70,17 @@ public final class CollaborationSchemas
/*******************************************************************************
** An attachment visible through its parent record; no storage coordinates are exposed.
*******************************************************************************/
public record CollaborationAttachmentV1(Integer id, String tableName, String recordId, String fileName, String contentType, Long sizeBytes, String sha256, String description, String createdByUserId, Instant createDate, Boolean canDelete) implements ToSchema
public record CollaborationAttachmentV1(Integer id, String tableName, String recordId, String fileName, String contentType, Long sizeBytes, String sha256, String description, String createdByUserId, Instant createDate, Boolean canDelete, String createdByDisplayName, Boolean canEdit) implements ToSchema
{
/*******************************************************************************
** Preserve construction compatibility with the earlier attachment body.
*******************************************************************************/
public CollaborationAttachmentV1(Integer id, String tableName, String recordId, String fileName, String contentType,
Long sizeBytes, String sha256, String description, String createdByUserId, Instant createDate, Boolean canDelete, String createdByDisplayName)
{
this(id, tableName, recordId, fileName, contentType, sizeBytes, sha256, description, createdByUserId, createDate, canDelete, createdByDisplayName, canDelete);
}
/*******************************************************************************
** Describe this versioned body for generated clients.
*******************************************************************************/
@@ -89,7 +98,9 @@ public final class CollaborationSchemas
Map.entry("description", new Schema().withType(Type.STRING).withDescription("Optional description.").withNullable(true)),
Map.entry("createdByUserId", new Schema().withType(Type.STRING).withDescription("Uploader identity.").withNullable(true)),
Map.entry("createDate", new Schema().withType(Type.STRING).withFormat("date-time").withDescription("Upload time.").withNullable(true)),
Map.entry("canDelete", new Schema().withType(Type.BOOLEAN).withDescription("Whether the caller may change or delete this attachment.").withNullable(true))))));
Map.entry("canDelete", new Schema().withType(Type.BOOLEAN).withDescription("Whether the caller may delete this attachment; retained files cannot be deleted.").withNullable(true)),
Map.entry("canEdit", new Schema().withType(Type.BOOLEAN).withDescription("Whether the caller may change the attachment filename or description.").withNullable(true)),
Map.entry("createdByDisplayName", new Schema().withType(Type.STRING).withDescription("Uploader display name.").withNullable(true))))));
}
}
@@ -201,7 +212,7 @@ public final class CollaborationSchemas
/*******************************************************************************
** A previous comment body.
*******************************************************************************/
public record CollaborationCommentRevisionV1(Integer id, String body, Instant editedAt, String editedByUserId) implements ToSchema
public record CollaborationCommentRevisionV1(Integer id, String body, Instant editedAt, String editedByUserId, String editedByDisplayName) implements ToSchema
{
/*******************************************************************************
** Describe this versioned body for generated clients.
@@ -213,14 +224,15 @@ public final class CollaborationSchemas
Map.entry("id", new Schema().withType(Type.INTEGER).withDescription("Revision identifier.").withNullable(true)),
Map.entry("body", new Schema().withType(Type.STRING).withDescription("Previous body text.").withNullable(true)),
Map.entry("editedAt", new Schema().withType(Type.STRING).withFormat("date-time").withDescription("Time the body was replaced.").withNullable(true)),
Map.entry("editedByUserId", new Schema().withType(Type.STRING).withDescription("Editor identity.").withNullable(true))))));
Map.entry("editedByUserId", new Schema().withType(Type.STRING).withDescription("Editor identity.").withNullable(true)),
Map.entry("editedByDisplayName", new Schema().withType(Type.STRING).withDescription("Editor display name.").withNullable(true))))));
}
}
/*******************************************************************************
** A subscription to record changes and comments.
*******************************************************************************/
public record CollaborationFollowV1(Integer id, String userId, String reason, Boolean notifyOnChange, Boolean notifyOnComment, Instant mutedUntil) implements ToSchema
public record CollaborationFollowV1(Integer id, String userId, String reason, Boolean notifyOnChange, Boolean notifyOnComment, Instant mutedUntil, String userDisplayName) implements ToSchema
{
/*******************************************************************************
** Describe this versioned body for generated clients.
@@ -234,7 +246,8 @@ public final class CollaborationSchemas
Map.entry("reason", new Schema().withType(Type.STRING).withEnumValues(List.of("MANUAL", "AUTHOR", "COMMENTED", "MENTIONED", "ASSIGNED", "SHARED")).withDescription("How following began.").withNullable(true)),
Map.entry("notifyOnChange", new Schema().withType(Type.BOOLEAN).withDescription("Receive record change digests.").withNullable(true)),
Map.entry("notifyOnComment", new Schema().withType(Type.BOOLEAN).withDescription("Receive new comment notifications.").withNullable(true)),
Map.entry("mutedUntil", new Schema().withType(Type.STRING).withFormat("date-time").withDescription("Notifications muted until this time, if set.").withNullable(true))))));
Map.entry("mutedUntil", new Schema().withType(Type.STRING).withFormat("date-time").withDescription("Notifications muted until this time, if set.").withNullable(true)),
Map.entry("userDisplayName", new Schema().withType(Type.STRING).withDescription("Follower display name.").withNullable(true))))));
}
}
@@ -487,8 +500,16 @@ public final class CollaborationSchemas
/*******************************************************************************
** Temporary access to file bytes. Clients follow url.
*******************************************************************************/
public record CollaborationDownloadV1(String url) implements ToSchema
public record CollaborationDownloadV1(String url, String delivery) implements ToSchema
{
/***************************************************************************
** Existing callers retain the storage URL contract.
***************************************************************************/
public CollaborationDownloadV1(String url)
{
this(url, "SIGNED_URL");
}
/*******************************************************************************
** Describe this versioned body for generated clients.
*******************************************************************************/
@@ -496,7 +517,8 @@ public final class CollaborationSchemas
public Schema toSchema()
{
return (new Schema().withType(Type.OBJECT).withDescription("Temporary access to file bytes. Clients follow url.").withProperties(new TreeMap<>(Map.ofEntries(
Map.entry("url", new Schema().withType(Type.STRING).withDescription("Presigned URL, valid for five minutes. Clients follow url; never persist it.").withNullable(true))))));
Map.entry("url", new Schema().withType(Type.STRING).withDescription("Temporary delivery URL; never persist it.").withNullable(true)),
Map.entry("delivery", new Schema().withType(Type.STRING).withDescription("SIGNED_URL or AUTHENTICATED_FETCH. Authenticated fetch requires the trusted native API transport and a durable receipt check.").withNullable(true))))));
}
}
@@ -511,7 +533,7 @@ public final class CollaborationSchemas
@Override
public Schema toSchema()
{
return (new Schema().withType(Type.OBJECT).withDescription("Standard QQQ error response.").withProperties(new TreeMap<>(Map.ofEntries(
return (new Schema().withType(Type.OBJECT).withDescription("Deprecated since 1.1.0: errors use BasicErrorResponseV1.").withProperties(new TreeMap<>(Map.ofEntries(
Map.entry("error", new Schema().withType(Type.STRING).withDescription("Human-readable reason the operation failed.").withNullable(true))))));
}
}
@@ -678,14 +700,14 @@ public final class CollaborationSchemas
{
Map<String, Schema> schemas = new TreeMap<>();
schemas.put("CollaborationFavoriteV1", new CollaborationFavoriteV1(null, null, null, null, null, null, null, null, null).toSchema());
schemas.put("CollaborationAttachmentV1", new CollaborationAttachmentV1(null, null, null, null, null, null, null, null, null, null, null).toSchema());
schemas.put("CollaborationAttachmentV1", new CollaborationAttachmentV1(null, null, null, null, null, null, null, null, null, null, null, null).toSchema());
schemas.put("CollaborationTagV1", new CollaborationTagV1(null, null, null, null, null, null, null).toSchema());
schemas.put("CollaborationPersonV1", new CollaborationPersonV1(null, null, null, null).toSchema());
schemas.put("CollaborationMentionV1", new CollaborationMentionV1(null, null).toSchema());
schemas.put("CollaborationMentionCandidateV1", new CollaborationMentionCandidateV1(null, null).toSchema());
schemas.put("CollaborationCommentV1", new CollaborationCommentV1(null, null, null, null, null, null, null, null, null, null, null, null, null, null, null).toSchema());
schemas.put("CollaborationCommentRevisionV1", new CollaborationCommentRevisionV1(null, null, null, null).toSchema());
schemas.put("CollaborationFollowV1", new CollaborationFollowV1(null, null, null, null, null, null).toSchema());
schemas.put("CollaborationCommentRevisionV1", new CollaborationCommentRevisionV1(null, null, null, null, null).toSchema());
schemas.put("CollaborationFollowV1", new CollaborationFollowV1(null, null, null, null, null, null, null).toSchema());
schemas.put("CollaborationFollowStateV1", new CollaborationFollowStateV1(null, null).toSchema());
schemas.put("CollaborationTimelineDetailV1", new CollaborationTimelineDetailV1(null, null, null, null).toSchema());
schemas.put("CollaborationTimelineItemV1", new CollaborationTimelineItemV1(null, null, null, null, null, null, null, null, null).toSchema());
@@ -711,6 +733,13 @@ public final class CollaborationSchemas
schemas.put("CollaborationCommentPostRequestV1", new CollaborationCommentPostRequestV1(null, null, null).toSchema());
schemas.put("CollaborationCommentEditRequestV1", new CollaborationCommentEditRequestV1(null).toSchema());
schemas.put("CollaborationFollowRequestV1", new CollaborationFollowRequestV1(null, null).toSchema());
schemas.put("CollaborationPresenceRequestV1", new CollaborationPresenceSchemas.CollaborationPresenceRequestV1(null, null, null, null, null).toSchema());
schemas.put("CollaborationPresenceLeaveV1", new CollaborationPresenceSchemas.CollaborationPresenceLeaveV1(null, null, null).toSchema());
schemas.put("CollaborationPresenceSnapshotV1", new CollaborationPresenceSchemas.CollaborationPresenceSnapshotV1(null, null, null).toSchema());
schemas.put("CollaborationPresenceViewerV1", new CollaborationPresenceSchemas.CollaborationPresenceViewerV1(null, null, null, null).toSchema());
schemas.put("CollaborationPresenceOpenV1", new CollaborationPresenceSchemas.CollaborationPresenceOpenV1(null, null).toSchema());
schemas.put("CollaborationPresenceLeaseV1", new CollaborationPresenceSchemas.CollaborationPresenceLeaseV1(null).toSchema());
schemas.put("CollaborationPresenceExpiredV1", new CollaborationPresenceSchemas.CollaborationPresenceExpiredV1(null).toSchema());
schemas.putAll(CollaborationWorkSchemas.components());
return (schemas);
}
@@ -22,6 +22,10 @@
package com.kingsrook.qbits.collaboration.api;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.stream.Stream;
import com.kingsrook.qbits.collaboration.api.CollaborationSchemas.CollaborationAttachmentV1;
import com.kingsrook.qbits.collaboration.api.CollaborationSchemas.CollaborationCommentRevisionV1;
import com.kingsrook.qbits.collaboration.api.CollaborationSchemas.CollaborationCommentV1;
@@ -40,10 +44,13 @@ import com.kingsrook.qbits.collaboration.comments.MentionCandidate;
import com.kingsrook.qbits.collaboration.comments.MentionEntry;
import com.kingsrook.qbits.collaboration.favorites.FavoriteEntry;
import com.kingsrook.qbits.collaboration.follow.FollowEntry;
import com.kingsrook.qbits.collaboration.people.PeopleService;
import com.kingsrook.qbits.collaboration.people.PersonEntry;
import com.kingsrook.qbits.collaboration.tags.TagEntry;
import com.kingsrook.qbits.collaboration.timeline.TimelineDetail;
import com.kingsrook.qbits.collaboration.timeline.TimelineItem;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
@@ -67,13 +74,39 @@ final class CollaborationViews
value.id(), value.targetType(), value.targetName(), value.recordId(), value.label(), value.folder(), value.sortOrder(), value.available(), value.href()));
}
/*******************************************************************************
** One display name per distinct user id, from a single directory query.
*******************************************************************************/
static Map<String, String> names(String... userIds) throws QException
{
return (new PeopleService().displayNames(Arrays.asList(userIds)));
}
/*******************************************************************************
** The display name for an id; null stays null, unknown ids show themselves.
*******************************************************************************/
private static String name(Map<String, String> names, String userId)
{
return (userId == null ? null : names.getOrDefault(userId, userId));
}
/*******************************************************************************
** Project attachment display data.
*******************************************************************************/
static CollaborationAttachmentV1 attachment(AttachmentEntry value)
static CollaborationAttachmentV1 attachment(AttachmentEntry value) throws QException
{
return (new CollaborationAttachmentV1(
value.id(), value.tableName(), value.recordId(), value.fileName(), value.contentType(), value.sizeBytes(), value.sha256(), value.description(), value.createdByUserId(), value.createDate(), value.canDelete()));
return (attachments(List.of(value)).getFirst());
}
/*******************************************************************************
** Project attachments, resolving uploader names in one batch.
*******************************************************************************/
static List<CollaborationAttachmentV1> attachments(List<AttachmentEntry> values) throws QException
{
Map<String, String> names = names(values.stream().map(AttachmentEntry::createdByUserId).toArray(String[]::new));
return (values.stream().map(value -> new CollaborationAttachmentV1(
value.id(), value.tableName(), value.recordId(), value.fileName(), value.contentType(), value.sizeBytes(), value.sha256(), value.description(), value.createdByUserId(), value.createDate(), value.canDelete(),
name(names, value.createdByUserId()), value.canEdit())).toList());
}
/*******************************************************************************
@@ -115,19 +148,29 @@ final class CollaborationViews
/*******************************************************************************
** Project commentrevision display data.
*******************************************************************************/
static CollaborationCommentRevisionV1 commentRevision(CommentRevision value)
static List<CollaborationCommentRevisionV1> commentRevisions(List<CommentRevision> values) throws QException
{
return (new CollaborationCommentRevisionV1(
value.id(), value.body(), value.editedAt(), value.editedByUserId()));
Map<String, String> names = names(values.stream().map(CommentRevision::editedByUserId).toArray(String[]::new));
return (values.stream().map(value -> new CollaborationCommentRevisionV1(
value.id(), value.body(), value.editedAt(), value.editedByUserId(), name(names, value.editedByUserId()))).toList());
}
/*******************************************************************************
** Project follow display data.
*******************************************************************************/
static CollaborationFollowV1 follow(FollowEntry value)
static CollaborationFollowV1 follow(FollowEntry value) throws QException
{
return (new CollaborationFollowV1(
value.id(), value.userId(), value.reason(), value.notifyOnChange(), value.notifyOnComment(), value.mutedUntil()));
return (follows(List.of(value)).getFirst());
}
/*******************************************************************************
** Project followers, resolving names in one batch.
*******************************************************************************/
static List<CollaborationFollowV1> follows(List<FollowEntry> values) throws QException
{
Map<String, String> names = names(values.stream().map(FollowEntry::userId).toArray(String[]::new));
return (values.stream().map(value -> new CollaborationFollowV1(
value.id(), value.userId(), value.reason(), value.notifyOnChange(), value.notifyOnComment(), value.mutedUntil(), name(names, value.userId()))).toList());
}
/*******************************************************************************
@@ -159,33 +202,70 @@ final class CollaborationViews
/*******************************************************************************
** Project Share public fields.
*******************************************************************************/
static CollaborationWorkSchemas.CollaborationShareV1 share(com.kingsrook.qqq.backend.core.model.data.QRecord row)
static CollaborationWorkSchemas.CollaborationShareV1 share(QRecord row) throws QException
{
return (new CollaborationWorkSchemas.CollaborationShareV1(row.getValueInteger("id"), row.getValueString("tableName"), row.getValueString("recordId"), row.getValueString("fromUserId"), row.getValueString("toUserId"), row.getValueString("message"), row.getValueBoolean("recipientCouldRead"), row.getValueInstant("sentAt")));
return (shares(List.of(row)).getFirst());
}
/*******************************************************************************
** Project shares, resolving sender and recipient names in one batch.
*******************************************************************************/
static List<CollaborationWorkSchemas.CollaborationShareV1> shares(List<QRecord> rows) throws QException
{
Map<String, String> names = names(rows.stream().flatMap(row -> Stream.of(row.getValueString("fromUserId"), row.getValueString("toUserId"))).toArray(String[]::new));
return (rows.stream().map(row -> new CollaborationWorkSchemas.CollaborationShareV1(row.getValueInteger("id"), row.getValueString("tableName"), row.getValueString("recordId"),
row.getValueString("fromUserId"), row.getValueString("toUserId"), row.getValueString("message"), row.getValueBoolean("recipientCouldRead"), row.getValueInstant("sentAt"),
name(names, row.getValueString("fromUserId")), name(names, row.getValueString("toUserId")))).toList());
}
/*******************************************************************************
** Project AccessRequest public fields.
*******************************************************************************/
static CollaborationWorkSchemas.CollaborationAccessRequestV1 accessRequest(com.kingsrook.qqq.backend.core.model.data.QRecord row)
static CollaborationWorkSchemas.CollaborationAccessRequestV1 accessRequest(QRecord row) throws QException
{
return (new CollaborationWorkSchemas.CollaborationAccessRequestV1(row.getValueInteger("id"), row.getValueString("tableName"), row.getValueString("recordId"), row.getValueString("requesterUserId"), row.getValueString("message"), row.getValueString("status"), row.getValueInstant("handledAt"), row.getValueString("note")));
return (accessRequests(List.of(row)).getFirst());
}
/*******************************************************************************
** Project access requests, resolving requester and handler names in one batch.
*******************************************************************************/
static List<CollaborationWorkSchemas.CollaborationAccessRequestV1> accessRequests(List<QRecord> rows) throws QException
{
Map<String, String> names = names(rows.stream().flatMap(row -> Stream.of(row.getValueString("requesterUserId"), row.getValueString("handledByUserId"))).toArray(String[]::new));
return (rows.stream().map(row -> new CollaborationWorkSchemas.CollaborationAccessRequestV1(row.getValueInteger("id"), row.getValueString("tableName"), row.getValueString("recordId"),
row.getValueString("requesterUserId"), row.getValueString("message"), row.getValueString("status"), row.getValueInstant("handledAt"), row.getValueString("note"),
name(names, row.getValueString("requesterUserId")), name(names, row.getValueString("handledByUserId")))).toList());
}
/*******************************************************************************
** Project Task public fields.
*******************************************************************************/
static CollaborationWorkSchemas.CollaborationTaskV1 task(com.kingsrook.qqq.backend.core.model.data.QRecord row)
static CollaborationWorkSchemas.CollaborationTaskV1 task(QRecord row) throws QException
{
return (new CollaborationWorkSchemas.CollaborationTaskV1(row.getValueInteger("id"), row.getValueString("tableName"), row.getValueString("recordId"), row.getValueString("title"), row.getValueString("notes"), row.getValueString("assigneeUserId"), row.getValueInstant("dueAt"), row.getValueInstant("remindAt"), row.getValueString("status"), row.getValueString("createdByUserId"), row.getValueInstant("completedAt"), row.getValueString("completedByUserId")));
return (tasks(List.of(row)).getFirst());
}
/*******************************************************************************
** Project tasks, resolving assignee, creator and completer names in one batch.
*******************************************************************************/
static List<CollaborationWorkSchemas.CollaborationTaskV1> tasks(List<QRecord> rows) throws QException
{
Map<String, String> names = names(rows.stream().flatMap(row -> Stream.of(row.getValueString("assigneeUserId"), row.getValueString("createdByUserId"),
row.getValueString("completedByUserId"))).toArray(String[]::new));
return (rows.stream().map(row -> new CollaborationWorkSchemas.CollaborationTaskV1(row.getValueInteger("id"), row.getValueString("tableName"), row.getValueString("recordId"),
row.getValueString("title"), row.getValueString("notes"), row.getValueString("assigneeUserId"), row.getValueInstant("dueAt"), row.getValueInstant("remindAt"), row.getValueString("status"),
row.getValueString("createdByUserId"), row.getValueInstant("completedAt"), row.getValueString("completedByUserId"),
name(names, row.getValueString("assigneeUserId")), name(names, row.getValueString("createdByUserId")), name(names, row.getValueString("completedByUserId")))).toList());
}
/*******************************************************************************
** Project Assignment public fields.
*******************************************************************************/
static CollaborationWorkSchemas.CollaborationAssignmentV1 assignment(com.kingsrook.qqq.backend.core.model.data.QRecord row)
static CollaborationWorkSchemas.CollaborationAssignmentV1 assignment(QRecord row) throws QException
{
return (new CollaborationWorkSchemas.CollaborationAssignmentV1(row.getValueInteger("id"), row.getValueString("tableName"), row.getValueString("recordId"), row.getValueString("ownerUserId"), row.getValueString("assigneeUserId")));
Map<String, String> names = names(row.getValueString("ownerUserId"), row.getValueString("assigneeUserId"));
return (new CollaborationWorkSchemas.CollaborationAssignmentV1(row.getValueInteger("id"), row.getValueString("tableName"), row.getValueString("recordId"),
row.getValueString("ownerUserId"), row.getValueString("assigneeUserId"), name(names, row.getValueString("ownerUserId")), name(names, row.getValueString("assigneeUserId"))));
}
}
@@ -95,7 +95,7 @@ public final class CollaborationWorkSchemas
/*******************************************************************************
** Versioned Share body.
*******************************************************************************/
public record CollaborationShareV1(Integer id, String tableName, String recordId, String fromUserId, String toUserId, String message, Boolean recipientCouldRead, Instant sentAt) implements ToSchema
public record CollaborationShareV1(Integer id, String tableName, String recordId, String fromUserId, String toUserId, String message, Boolean recipientCouldRead, Instant sentAt, String fromDisplayName, String toDisplayName) implements ToSchema
{
/*******************************************************************************
** Explicit schema for generated clients.
@@ -111,7 +111,9 @@ public final class CollaborationWorkSchemas
Map.entry("toUserId", new Schema().withType(Type.STRING).withDescription("toUserId.").withNullable(true)),
Map.entry("message", new Schema().withType(Type.STRING).withDescription("message.").withNullable(true)),
Map.entry("recipientCouldRead", new Schema().withType(Type.BOOLEAN).withDescription("recipientCouldRead.").withNullable(true)),
Map.entry("sentAt", new Schema().withType(Type.STRING).withFormat("date-time").withDescription("sentAt.").withNullable(true))))));
Map.entry("sentAt", new Schema().withType(Type.STRING).withFormat("date-time").withDescription("sentAt.").withNullable(true)),
Map.entry("fromDisplayName", new Schema().withType(Type.STRING).withDescription("Sender display name.").withNullable(true)),
Map.entry("toDisplayName", new Schema().withType(Type.STRING).withDescription("Recipient display name.").withNullable(true))))));
}
}
@@ -151,7 +153,7 @@ public final class CollaborationWorkSchemas
/*******************************************************************************
** Versioned AccessRequest body.
*******************************************************************************/
public record CollaborationAccessRequestV1(Integer id, String tableName, String recordId, String requesterUserId, String message, String status, Instant handledAt, String note) implements ToSchema
public record CollaborationAccessRequestV1(Integer id, String tableName, String recordId, String requesterUserId, String message, String status, Instant handledAt, String note, String requesterDisplayName, String handledByDisplayName) implements ToSchema
{
/*******************************************************************************
** Explicit schema for generated clients.
@@ -167,7 +169,9 @@ public final class CollaborationWorkSchemas
Map.entry("message", new Schema().withType(Type.STRING).withDescription("message.").withNullable(true)),
Map.entry("status", new Schema().withType(Type.STRING).withDescription("status.").withNullable(true)),
Map.entry("handledAt", new Schema().withType(Type.STRING).withFormat("date-time").withDescription("handledAt.").withNullable(true)),
Map.entry("note", new Schema().withType(Type.STRING).withDescription("note.").withNullable(true))))));
Map.entry("note", new Schema().withType(Type.STRING).withDescription("note.").withNullable(true)),
Map.entry("requesterDisplayName", new Schema().withType(Type.STRING).withDescription("Requester display name.").withNullable(true)),
Map.entry("handledByDisplayName", new Schema().withType(Type.STRING).withDescription("Display name of the person who handled the request.").withNullable(true))))));
}
}
@@ -210,7 +214,7 @@ public final class CollaborationWorkSchemas
/*******************************************************************************
** Versioned Task body.
*******************************************************************************/
public record CollaborationTaskV1(Integer id, String tableName, String recordId, String title, String notes, String assigneeUserId, Instant dueAt, Instant remindAt, String status, String createdByUserId, Instant completedAt, String completedByUserId) implements ToSchema
public record CollaborationTaskV1(Integer id, String tableName, String recordId, String title, String notes, String assigneeUserId, Instant dueAt, Instant remindAt, String status, String createdByUserId, Instant completedAt, String completedByUserId, String assigneeDisplayName, String createdByDisplayName, String completedByDisplayName) implements ToSchema
{
/*******************************************************************************
** Explicit schema for generated clients.
@@ -230,7 +234,10 @@ public final class CollaborationWorkSchemas
Map.entry("status", new Schema().withType(Type.STRING).withDescription("status.").withNullable(true)),
Map.entry("createdByUserId", new Schema().withType(Type.STRING).withDescription("createdByUserId.").withNullable(true)),
Map.entry("completedAt", new Schema().withType(Type.STRING).withFormat("date-time").withDescription("completedAt.").withNullable(true)),
Map.entry("completedByUserId", new Schema().withType(Type.STRING).withDescription("completedByUserId.").withNullable(true))))));
Map.entry("completedByUserId", new Schema().withType(Type.STRING).withDescription("completedByUserId.").withNullable(true)),
Map.entry("assigneeDisplayName", new Schema().withType(Type.STRING).withDescription("Assignee display name.").withNullable(true)),
Map.entry("createdByDisplayName", new Schema().withType(Type.STRING).withDescription("Creator display name.").withNullable(true)),
Map.entry("completedByDisplayName", new Schema().withType(Type.STRING).withDescription("Display name of the person who completed the task.").withNullable(true))))));
}
}
@@ -312,7 +319,7 @@ public final class CollaborationWorkSchemas
/*******************************************************************************
** Versioned Assignment body.
*******************************************************************************/
public record CollaborationAssignmentV1(Integer id, String tableName, String recordId, String ownerUserId, String assigneeUserId) implements ToSchema
public record CollaborationAssignmentV1(Integer id, String tableName, String recordId, String ownerUserId, String assigneeUserId, String ownerDisplayName, String assigneeDisplayName) implements ToSchema
{
/*******************************************************************************
** Explicit schema for generated clients.
@@ -325,7 +332,9 @@ public final class CollaborationWorkSchemas
Map.entry("tableName", new Schema().withType(Type.STRING).withDescription("tableName.").withNullable(true)),
Map.entry("recordId", new Schema().withType(Type.STRING).withDescription("recordId.").withNullable(true)),
Map.entry("ownerUserId", new Schema().withType(Type.STRING).withDescription("ownerUserId.").withNullable(true)),
Map.entry("assigneeUserId", new Schema().withType(Type.STRING).withDescription("assigneeUserId.").withNullable(true))))));
Map.entry("assigneeUserId", new Schema().withType(Type.STRING).withDescription("assigneeUserId.").withNullable(true)),
Map.entry("ownerDisplayName", new Schema().withType(Type.STRING).withDescription("Owner display name.").withNullable(true)),
Map.entry("assigneeDisplayName", new Schema().withType(Type.STRING).withDescription("Assignee display name.").withNullable(true))))));
}
}
@@ -357,18 +366,18 @@ public final class CollaborationWorkSchemas
schemas.put("CollaborationLinkV1", new CollaborationLinkV1(null, null).toSchema());
schemas.put("CollaborationShareRecipientV1", new CollaborationShareRecipientV1(null, null, null).toSchema());
schemas.put("CollaborationSharePreviewV1", new CollaborationSharePreviewV1(null).toSchema());
schemas.put("CollaborationShareV1", new CollaborationShareV1(null, null, null, null, null, null, null, null).toSchema());
schemas.put("CollaborationShareV1", new CollaborationShareV1(null, null, null, null, null, null, null, null, null, null).toSchema());
schemas.put("CollaborationShareListV1", new CollaborationShareListV1(null).toSchema());
schemas.put("CollaborationShareRequestV1", new CollaborationShareRequestV1(null, null).toSchema());
schemas.put("CollaborationAccessRequestV1", new CollaborationAccessRequestV1(null, null, null, null, null, null, null, null).toSchema());
schemas.put("CollaborationAccessRequestV1", new CollaborationAccessRequestV1(null, null, null, null, null, null, null, null, null, null).toSchema());
schemas.put("CollaborationAccessRequestCreateRequestV1", new CollaborationAccessRequestCreateRequestV1(null, null, null, null).toSchema());
schemas.put("CollaborationAccessRequestListV1", new CollaborationAccessRequestListV1(null, null).toSchema());
schemas.put("CollaborationTaskV1", new CollaborationTaskV1(null, null, null, null, null, null, null, null, null, null, null, null).toSchema());
schemas.put("CollaborationTaskV1", new CollaborationTaskV1(null, null, null, null, null, null, null, null, null, null, null, null, null, null, null).toSchema());
schemas.put("CollaborationTaskCreateRequestV1", new CollaborationTaskCreateRequestV1(null, null, null, null, null).toSchema());
schemas.put("CollaborationTaskPatchRequestV1", new CollaborationTaskPatchRequestV1(null, null, null, null, null, null).toSchema());
schemas.put("CollaborationTaskListV1", new CollaborationTaskListV1(null, null).toSchema());
schemas.put("CollaborationAssignmentRequestV1", new CollaborationAssignmentRequestV1(null, null).toSchema());
schemas.put("CollaborationAssignmentV1", new CollaborationAssignmentV1(null, null, null, null, null).toSchema());
schemas.put("CollaborationAssignmentV1", new CollaborationAssignmentV1(null, null, null, null, null, null, null).toSchema());
schemas.put("CollaborationMyWorkV1", new CollaborationMyWorkV1(null, null, null, null).toSchema());
return (schemas);
}
@@ -69,6 +69,6 @@ public class CommentHistorySpec extends CollaborationJsonSpec
protected Object handle(JsonInput input) throws Exception
{
var page = page(input, new CommentsService().history(id(input)));
return (new CollaborationCommentHistoryV1(page.items().stream().map(CollaborationViews::commentRevision).toList(), page.nextCursor()));
return (new CollaborationCommentHistoryV1(CollaborationViews.commentRevisions(page.items()), page.nextCursor()));
}
}
@@ -75,8 +75,8 @@ public class CreateSharesSpec extends CollaborationJsonSpec
protected Object handle(JsonInput input) throws Exception
{
var body = body(input, CollaborationWorkSchemas.CollaborationShareRequestV1.class);
return (new CollaborationWorkSchemas.CollaborationShareListV1(new com.kingsrook.qbits.collaboration.sharing.SharingService()
.share(table(input), pk(input), recipients(body.toUserIds()), body.message()).stream().map(CollaborationViews::share).toList()));
return (new CollaborationWorkSchemas.CollaborationShareListV1(CollaborationViews.shares(new com.kingsrook.qbits.collaboration.sharing.SharingService()
.share(table(input), pk(input), recipients(body.toUserIds()), body.message()))));
}
}
@@ -76,7 +76,7 @@ public class DeletePresenceSpec extends CollaborationJsonSpec
protected Object handle(JsonInput input) throws Exception
{
var body = body(input, CollaborationPresenceSchemas.CollaborationPresenceLeaveV1.class);
new PresenceService().leave(body.connectionId(), required(body.table(), "table"), body.pk());
new PresenceService().leave(body.connectionId(), required(body.table(), "table"), body.pk(), body.sequence());
return (success());
}
}
@@ -68,6 +68,7 @@ public class DownloadAttachmentSpec extends CollaborationJsonSpec
@Override
protected Object handle(JsonInput input) throws Exception
{
return (new CollaborationDownloadV1(new AttachmentsService().downloadUrl(id(input), false).toExternalForm()));
var delivery = new AttachmentsService().downloadDelivery(id(input));
return (new CollaborationDownloadV1(delivery.url().toExternalForm(), delivery.authenticated() ? "AUTHENTICATED_FETCH" : "SIGNED_URL"));
}
}
@@ -69,6 +69,6 @@ public class GetFollowSpec extends CollaborationJsonSpec
protected Object handle(JsonInput input) throws Exception
{
var state = new FollowService().state(table(input), pk(input));
return (new CollaborationFollowStateV1(state.isPresent(), state.map(CollaborationViews::follow).orElse(null)));
return (new CollaborationFollowStateV1(state.isPresent(), state.isPresent() ? CollaborationViews.follow(state.get()) : null));
}
}
@@ -65,7 +65,7 @@ public class GetMyWorkSpec extends CollaborationJsonSpec
protected Object handle(JsonInput input) throws Exception
{
var summary = new com.kingsrook.qbits.collaboration.mywork.MyWorkService().summary();
return (new CollaborationWorkSchemas.CollaborationMyWorkV1(summary.openTaskCount(), summary.tasks().stream().map(CollaborationViews::task).toList(),
return (new CollaborationWorkSchemas.CollaborationMyWorkV1(summary.openTaskCount(), CollaborationViews.tasks(summary.tasks()),
summary.assignedRecordsByTable(), summary.unreadMentionCount()));
}
@@ -68,7 +68,10 @@ public class ListAttachmentsSpec extends CollaborationJsonSpec
@Override
protected Object handle(JsonInput input) throws Exception
{
var page = page(input, new AttachmentsService().list(table(input), pk(input)));
return (new CollaborationAttachmentListV1(page.items().stream().map(CollaborationViews::attachment).toList(), page.nextCursor()));
int limit = limit(input);
var position = pagePosition(input);
var page = new AttachmentsService().listPage(table(input), pk(input), position.offset(), limit);
String next = page.hasMore() ? position.cursor((long) position.offset() + page.items().size()) : null;
return (new CollaborationAttachmentListV1(CollaborationViews.attachments(page.items()), next));
}
}
@@ -69,6 +69,6 @@ public class ListFollowersSpec extends CollaborationJsonSpec
protected Object handle(JsonInput input) throws Exception
{
var page = page(input, new FollowService().followers(table(input), pk(input)));
return (new CollaborationFollowerListV1(page.items().stream().map(CollaborationViews::follow).toList(), page.nextCursor()));
return (new CollaborationFollowerListV1(CollaborationViews.follows(page.items()), page.nextCursor()));
}
}
@@ -66,7 +66,7 @@ public class ListMyAccessRequestsSpec extends CollaborationJsonSpec
{
var values = new com.kingsrook.qbits.collaboration.sharing.AccessRequestService().mine();
var page = page(input, values);
return (new CollaborationWorkSchemas.CollaborationAccessRequestListV1(page.items().stream().map(CollaborationViews::accessRequest).toList(), page.nextCursor()));
return (new CollaborationWorkSchemas.CollaborationAccessRequestListV1(CollaborationViews.accessRequests(page.items()), page.nextCursor()));
}
}
@@ -65,7 +65,7 @@ public class ListMyTasksSpec extends CollaborationJsonSpec
protected Object handle(JsonInput input) throws Exception
{
var page = new com.kingsrook.qbits.collaboration.tasks.TasksService().mine(input.getQueryParam("status"), limit(input), input.getQueryParam("cursor"));
return (new CollaborationWorkSchemas.CollaborationTaskListV1(page.tasks().stream().map(CollaborationViews::task).toList(), page.nextCursor()));
return (new CollaborationWorkSchemas.CollaborationTaskListV1(CollaborationViews.tasks(page.tasks()), page.nextCursor()));
}
}
@@ -70,7 +70,7 @@ public class ListTasksSpec extends CollaborationJsonSpec
throw (new com.kingsrook.qqq.backend.core.exceptions.QBadRequestException("includeDone must be true or false"));
}
var page = page(input, new com.kingsrook.qbits.collaboration.tasks.TasksService().list(table(input), pk(input), Boolean.valueOf(includeDone)));
return (new CollaborationWorkSchemas.CollaborationTaskListV1(page.items().stream().map(CollaborationViews::task).toList(), page.nextCursor()));
return (new CollaborationWorkSchemas.CollaborationTaskListV1(CollaborationViews.tasks(page.items()), page.nextCursor()));
}
}
@@ -0,0 +1,71 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.api;
import java.util.List;
import com.kingsrook.qqq.middleware.javalin.specs.BasicOperation;
import com.kingsrook.qqq.openapi.model.HttpMethod;
import com.kingsrook.qqq.openapi.model.Parameter;
/*******************************************************************************
** Mark all currently visible unread mentions read.
*******************************************************************************/
public class MarkMentionsReadSpec extends CollaborationJsonSpec
{
/*******************************************************************************
** Describe this operation.
*******************************************************************************/
@Override
public BasicOperation defineBasicOperation()
{
return (operation(HttpMethod.POST, "/my-work/mentions/read", "Mark my visible mentions read"));
}
/*******************************************************************************
** Declare inputs.
*******************************************************************************/
@Override
public List<Parameter> defineRequestParameters()
{
return (parameters());
}
/*******************************************************************************
** Name the response.
*******************************************************************************/
@Override
protected String getResponseSchemaName()
{
return ("CollaborationMutationResultV1");
}
/*******************************************************************************
** Run the service as the authenticated caller.
*******************************************************************************/
@Override
protected Object handle(JsonInput input) throws Exception
{
new com.kingsrook.qbits.collaboration.mywork.MyWorkService().markMentionsRead(null);
return (success());
}
}
@@ -0,0 +1,81 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.api;
import java.util.List;
import com.kingsrook.qbits.collaboration.presence.PresenceService;
import com.kingsrook.qqq.middleware.javalin.specs.BasicOperation;
import com.kingsrook.qqq.openapi.model.HttpMethod;
import com.kingsrook.qqq.openapi.model.Parameter;
import com.kingsrook.qqq.openapi.model.Schema;
/*******************************************************************************
** Renew presence with fresh parent access checks.
*******************************************************************************/
public class OpenPresenceLeaseSpec extends CollaborationJsonSpec
{
/*******************************************************************************
** Describe the extension operation.
*******************************************************************************/
@Override
public BasicOperation defineBasicOperation()
{
return (operation(HttpMethod.POST, "/presence/leases", "Open presence lease"));
}
/*******************************************************************************
** Explicit query parameters for polling.
*******************************************************************************/
@Override
public List<Parameter> defineRequestParameters()
{
return (parameters());
}
/*******************************************************************************
** Name the versioned response schema.
*******************************************************************************/
@Override
protected String getResponseSchemaName()
{
return ("CollaborationPresenceLeaseV1");
}
/*******************************************************************************
** Declare a typed JSON request body.
*******************************************************************************/
@Override
protected Schema defineRequestBodySchema()
{
return (new Schema().withRefToSchema("CollaborationPresenceOpenV1"));
}
/*******************************************************************************
** Services derive identity from QContext and enforce visibility.
*******************************************************************************/
@Override
protected Object handle(JsonInput input) throws Exception
{
var body = body(input, CollaborationPresenceSchemas.CollaborationPresenceOpenV1.class);
return (new CollaborationPresenceSchemas.CollaborationPresenceLeaseV1(new PresenceService().openLease(required(body.table(), "table"), body.pk())));
}
}
@@ -77,6 +77,6 @@ public class PostPresenceSpec extends CollaborationJsonSpec
{
var body = body(input, CollaborationPresenceSchemas.CollaborationPresenceRequestV1.class);
return (new CollaborationPresenceSchemas.CollaborationPresenceSnapshotV1(body.table(), body.pk(),
new PresenceService().heartbeat(body.connectionId(), required(body.table(), "table"), body.pk(), body.mode(), body.fieldName())));
new PresenceService().heartbeat(body.connectionId(), required(body.table(), "table"), body.pk(), body.mode(), body.fieldName(), body.sequence())));
}
}
@@ -68,6 +68,7 @@ public class PreviewAttachmentSpec extends CollaborationJsonSpec
@Override
protected Object handle(JsonInput input) throws Exception
{
return (new CollaborationDownloadV1(new AttachmentsService().downloadUrl(id(input), true).toExternalForm()));
var delivery = new AttachmentsService().downloadDelivery(id(input), true);
return (new CollaborationDownloadV1(delivery.url().toExternalForm(), delivery.authenticated() ? "AUTHENTICATED_FETCH" : "SIGNED_URL"));
}
}
@@ -76,7 +76,7 @@ public class PutAssignmentSpec extends CollaborationJsonSpec
{
var body = body(input, CollaborationWorkSchemas.CollaborationAssignmentRequestV1.class);
return (CollaborationViews.assignment(new com.kingsrook.qbits.collaboration.assignment.AssignmentService()
.assign(table(input), pk(input), body.ownerUserId(), body.assigneeUserId())));
.assignRoles(table(input), pk(input), body.ownerUserId(), body.assigneeUserId(), input.getBody().keySet())));
}
}
@@ -0,0 +1,112 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.api;
import java.util.List;
import com.kingsrook.qbits.collaboration.presence.PresenceService;
import com.kingsrook.qqq.middleware.javalin.specs.BasicOperation;
import com.kingsrook.qqq.openapi.model.HttpMethod;
import com.kingsrook.qqq.openapi.model.Parameter;
import com.kingsrook.qqq.openapi.model.Schema;
/*******************************************************************************
** Renew presence with fresh parent access checks.
*******************************************************************************/
public class RenewPresenceLeaseSpec extends CollaborationJsonSpec
{
/*******************************************************************************
** Expiry is distinct from permissions, invalid input and service outages.
*******************************************************************************/
@Override
public List<com.kingsrook.qqq.middleware.javalin.specs.BasicResponse> defineAdditionalBasicResponses()
{
var responses = new java.util.ArrayList<>(super.defineAdditionalBasicResponses());
responses.add(new com.kingsrook.qqq.middleware.javalin.specs.BasicResponse(io.javalin.http.HttpStatus.CONFLICT,
"Presence lease expired; open a new lease.", "CollaborationPresenceExpiredV1"));
return (responses);
}
/*******************************************************************************
** Preserve standard framework mapping for every other exception.
*******************************************************************************/
@Override
protected void handleException(io.javalin.http.Context context, Exception exception)
{
if(com.kingsrook.qqq.backend.core.utils.ExceptionUtils.findClassInRootChain(exception,
com.kingsrook.qbits.collaboration.presence.PresenceLeaseExpiredException.class) != null)
{
context.status(409).contentType("application/json").result(com.kingsrook.qqq.backend.core.utils.JsonUtils.toJson(
new CollaborationPresenceSchemas.CollaborationPresenceExpiredV1("PRESENCE_LEASE_EXPIRED")));
}
else
{
super.handleException(context, exception);
}
}
/*******************************************************************************
** Describe the extension operation.
*******************************************************************************/
@Override
public BasicOperation defineBasicOperation()
{
return (operation(HttpMethod.POST, "/presence/leases/renew", "Renew issued presence lease"));
}
/*******************************************************************************
** Explicit query parameters for polling.
*******************************************************************************/
@Override
public List<Parameter> defineRequestParameters()
{
return (parameters());
}
/*******************************************************************************
** Name the versioned response schema.
*******************************************************************************/
@Override
protected String getResponseSchemaName()
{
return ("CollaborationPresenceSnapshotV1");
}
/*******************************************************************************
** Declare a typed JSON request body.
*******************************************************************************/
@Override
protected Schema defineRequestBodySchema()
{
return (new Schema().withRefToSchema("CollaborationPresenceRequestV1"));
}
/*******************************************************************************
** Services derive identity from QContext and enforce visibility.
*******************************************************************************/
@Override
protected Object handle(JsonInput input) throws Exception
{
var body = body(input, CollaborationPresenceSchemas.CollaborationPresenceRequestV1.class);
return (new CollaborationPresenceSchemas.CollaborationPresenceSnapshotV1(body.table(), body.pk(),
new PresenceService().renewLease(body.connectionId(), required(body.table(), "table"), body.pk(), body.mode(), body.fieldName(), body.sequence())));
}
}
@@ -24,15 +24,22 @@ package com.kingsrook.qbits.collaboration.app;
import java.time.Instant;
import java.time.temporal.ChronoUnit;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import com.kingsrook.qbits.collaboration.CollaborationQBitConfig;
import com.kingsrook.qbits.collaboration.CollaborationRecords;
import com.kingsrook.qbits.collaboration.CollaborationSecurity;
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
import com.kingsrook.qbits.collaboration.permissions.CollaborationPermissions;
import com.kingsrook.qbits.collaboration.presence.PresenceRegistry;
import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
import com.kingsrook.qqq.backend.core.actions.permissions.PermissionEnforcement;
import com.kingsrook.qqq.backend.core.actions.tables.AggregateAction;
import com.kingsrook.qqq.backend.core.actions.tables.CountAction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.Aggregate;
@@ -41,27 +48,45 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateOp
import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateResult;
import com.kingsrook.qqq.backend.core.model.actions.tables.count.CountInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.RawHTML;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.MultiStatisticsData;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.MultiStatisticsData.StatisticsGroupData;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.MultiStatisticsData.StatisticsGroupData.Statistic;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.TableData;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppSection;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
/*******************************************************************************
** The Collaboration app and its home. Counts use QQQ actions with the current
** QContext session, preserving permissions and record security.
** The Collaboration app and its home, built from standard QQQ widgets: a KPI
** block and three full-width tables (tags, custom fields, who's here). Counts
** use QQQ actions with the current QContext session, so permissions and record
** security apply, and every block carries a real empty-state message. No colors
** are specified, so the frontend theme (including dark mode) decides.
*******************************************************************************/
public final class CollaborationHome
{
public static final String HOME_WIDGET = "collaborationHome";
public static final String SETUP_WIDGET = "collaborationSetup";
public static final String KPI_WIDGET = "collaborationKpis";
public static final String TAGS_WIDGET = "collaborationTags";
public static final String FIELDS_WIDGET = "collaborationCustomFields";
public static final String PRESENCE_WIDGET = "collaborationWhosHere";
public static final List<String> HOME_WIDGETS = List.of(SETUP_WIDGET, KPI_WIDGET, TAGS_WIDGET, FIELDS_WIDGET, PRESENCE_WIDGET);
public static final String NOBODY_HERE = "Nobody else is here right now.";
public static final String NO_TAGS = "No tags yet. Tags appear here once someone creates one on a record.";
public static final String NO_FIELDS = "No custom fields yet. Define one under Custom fields.";
public static final String PRESENCE_ADMIN_ONLY = "Who's here is visible to collaboration administrators.";
@@ -81,9 +106,9 @@ public final class CollaborationHome
{
QAppMetaData app = new QAppMetaData().withName(config.getAppName()).withLabel("Collaboration").withIcon(new QIcon("forum"))
.withPermissionRules(CollaborationPermissions.permissionRules())
.withWidgets(List.of(HOME_WIDGET))
.withWidgets(HOME_WIDGETS)
.withSection(new QAppSection().withName("collaboration").withLabel("Collaboration").withIcon(new QIcon("forum"))
.withTables(List.of(CollaborationTables.COMMENT, CollaborationTables.ATTACHMENT, CollaborationTables.TAG, CollaborationTables.FIELD_DEFINITION, CollaborationTables.TASK, CollaborationTables.SHARE, CollaborationTables.ACCESS_REQUEST, CollaborationTables.RECORD_ASSIGNMENT, CollaborationTables.ACTIVITY, CollaborationTables.FEATURE_SETTING, CollaborationTables.USER)));
.withTables(List.of(CollaborationTables.COMMENT, CollaborationTables.ATTACHMENT, CollaborationTables.ATTACHMENT_CLEANUP, CollaborationTables.TAG, CollaborationTables.FIELD_DEFINITION, CollaborationTables.TASK, CollaborationTables.SHARE, CollaborationTables.ACCESS_REQUEST, CollaborationTables.RECORD_ASSIGNMENT, CollaborationTables.ACTIVITY, CollaborationTables.FEATURE_SETTING, CollaborationTables.USER)));
app.setParentAppName(config.getParentAppName());
return (app);
}
@@ -91,141 +116,285 @@ public final class CollaborationHome
/*******************************************************************************
** Full-width HTML widget, following the reference Storage home renderer.
** The four full-width home widgets, in display order.
*******************************************************************************/
public static QWidgetMetaData widget()
public static List<QWidgetMetaData> widgets()
{
return (new QWidgetMetaData().withName(HOME_WIDGET).withLabel("Overview").withType(WidgetType.HTML.getType())
.withCodeReference(new QCodeReference(HomeRenderer.class)).withGridColumns(12).withIsCard(false)
return (List.of(
widget(SETUP_WIDGET, "Setup", WidgetType.TABLE, SetupRenderer.class),
widget(KPI_WIDGET, "Overview", WidgetType.MULTI_STATISTICS, KpiRenderer.class),
widget(TAGS_WIDGET, "Tags", WidgetType.TABLE, TagsRenderer.class),
widget(FIELDS_WIDGET, "Custom fields", WidgetType.TABLE, FieldsRenderer.class),
widget(PRESENCE_WIDGET, "Who's here", WidgetType.TABLE, PresenceRenderer.class)));
}
/*******************************************************************************
** One full-width card-less widget guarded by the shared permission rules.
*******************************************************************************/
private static QWidgetMetaData widget(String name, String label, WidgetType type, Class<? extends AbstractWidgetRenderer> renderer)
{
return (new QWidgetMetaData().withName(name).withLabel(label).withType(type.getType())
.withCodeReference(new QCodeReference(renderer)).withGridColumns(12).withIsCard(false)
.withPermissionRules(CollaborationPermissions.permissionRules()));
}
/*******************************************************************************
** Show persisted enabled overrides by scope type, and recent directory use.
** These are settings counts, not a claim about resolved host-table features.
** The Setup card: each record feature, whether it is on, for which tables,
** where that comes from, and the next step.
*******************************************************************************/
public static String render(Instant now) throws QException
public static TableData setup() throws QException
{
CollaborationSecurity.ensureSessionKeys(QContext.getQInstance(), QContext.getQSession());
StringBuilder html = new StringBuilder("<div style=\"display:flex;flex-direction:column;gap:1rem;width:100%;overflow-wrap:anywhere;\"><section><h3>Feature settings</h3>");
long settings = count(CollaborationTables.FEATURE_SETTING, new QQueryFilter());
if(settings == 0)
List<Map<String, Object>> rows = new ArrayList<>();
for(CollaborationSetupStatus.Item item : CollaborationSetupStatus.load(QContext.getQInstance()))
{
html.append("<p>No feature settings yet: features follow the code defaults</p>");
Map<String, Object> row = new LinkedHashMap<>();
row.put("feature", item.feature());
row.put("state", item.state());
row.put("where", item.where());
row.put("source", item.source());
row.put("next", item.nextStep() == null ? "" : item.nextStep());
rows.add(row);
}
else
{
html.append("<p>Enabled feature settings per scope:</p><ul>");
for(String scope : List.of("INSTANCE", "APP", "TABLE"))
{
long enabled = count(CollaborationTables.FEATURE_SETTING, new QQueryFilter()
.withCriteria(new QFilterCriteria("scopeType", QCriteriaOperator.EQUALS, scope))
.withCriteria(new QFilterCriteria("enabled", QCriteriaOperator.EQUALS, true)));
html.append("<li>").append(scope).append(": ").append(enabled).append("</li>");
}
html.append("</ul><p>Feature settings take effect after saving.</p>");
}
long people = count(CollaborationTables.USER, new QQueryFilter());
long recent = count(CollaborationTables.USER, new QQueryFilter()
.withCriteria(new QFilterCriteria("lastSeenAt", QCriteriaOperator.GREATER_THAN_OR_EQUALS, now.minus(7, ChronoUnit.DAYS)))
.withCriteria(new QFilterCriteria("lastSeenAt", QCriteriaOperator.LESS_THAN_OR_EQUALS, now)));
html.append("</section><section><h3>People</h3><p>People known: ").append(people)
.append("</p><p>Active in the last 7 days: ").append(recent).append("</p>");
if(people == 0)
{
html.append("<p>No people yet: the directory will be filled by collaboration calls when the feature services are installed.</p>");
}
html.append("</section>");
personalSummary(html, CollaborationTables.FAVORITE, "Favorites");
personalSummary(html, CollaborationTables.RECENT, "Recently viewed");
attachmentSummary(html);
tagSummary(html);
fieldSummary(html);
html.append("<section><h3>Open access requests</h3><p>").append(count(CollaborationTables.ACCESS_REQUEST,
new QQueryFilter().withCriteria("status", QCriteriaOperator.EQUALS, "OPEN"))).append("</p></section>");
html.append("<section><h3>Open tasks</h3><p>").append(count(CollaborationTables.TASK,
new QQueryFilter().withCriteria("status", QCriteriaOperator.EQUALS, "OPEN"))).append("</p></section>");
html.append("<section><h3>Overdue tasks</h3><p>").append(count(CollaborationTables.TASK,
new QQueryFilter().withCriteria("status", QCriteriaOperator.EQUALS, "OPEN").withCriteria("dueAt", QCriteriaOperator.LESS_THAN, now))).append("</p></section>");
html.append("<section><h3>Shares this week</h3><p>").append(count(CollaborationTables.SHARE,
new QQueryFilter().withCriteria("sentAt", QCriteriaOperator.GREATER_THAN_OR_EQUALS, now.minus(7, ChronoUnit.DAYS))
.withCriteria("sentAt", QCriteriaOperator.LESS_THAN_OR_EQUALS, now))).append("</p></section>");
html.append("<section><h3>Comments this week</h3><p>").append(count(CollaborationTables.COMMENT, new QQueryFilter()
.withCriteria("createDate", QCriteriaOperator.GREATER_THAN_OR_EQUALS, now.minus(7, ChronoUnit.DAYS))
.withCriteria("createDate", QCriteriaOperator.LESS_THAN_OR_EQUALS, now).withCriteria("deletedAt", QCriteriaOperator.IS_BLANK))).append("</p></section>");
html.append("<section><h3>Open threads</h3><p>").append(count(CollaborationTables.COMMENT, new QQueryFilter()
.withCriteria("threadId", QCriteriaOperator.IS_BLANK).withCriteria("resolvedAt", QCriteriaOperator.IS_BLANK)
.withCriteria("deletedAt", QCriteriaOperator.IS_BLANK))).append("</p></section>");
if(QContext.getQSession().hasPermission(CollaborationPermissions.ADMIN))
{
html.append("<section><h3>Followers</h3><p>").append(count(CollaborationTables.FOLLOW, new QQueryFilter())).append("</p></section>");
}
html.append("<p>Conversation totals are approximate where parent record locks require post-filtering.</p>");
return (html.append("</div>").toString());
return (table("Setup", "Nothing to set up.", rows, column("Feature", "feature", "left"), column("State", "state", "left"), column("Where", "where", "left"),
column("From", "source", "left"), column("Next step", "next", "left")));
}
/*******************************************************************************
** Caller-filtered vocabulary totals and five most used active tags.
** The KPI groups: conversations, work, files and custom fields.
*******************************************************************************/
private static void tagSummary(StringBuilder html) throws QException
public static MultiStatisticsData kpis(Instant now) throws QException
{
QQueryFilter active = new QQueryFilter().withCriteria("mergedIntoTagId", QCriteriaOperator.IS_BLANK);
html.append("<section><h3>Tags</h3><p>Vocabulary size: ").append(count(CollaborationTables.TAG, active))
.append("</p><p>Pending suggestions: ").append(count(CollaborationTables.TAG, new QQueryFilter()
.withCriteria("mergedIntoTagId", QCriteriaOperator.IS_BLANK).withCriteria("approved", QCriteriaOperator.EQUALS, false)))
.append("</p><ul>");
com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput input = new com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput(CollaborationTables.TAG)
.withFilter(active.withCriteria("approved", QCriteriaOperator.EQUALS, true).withOrderBy(new QFilterOrderBy("usageCount", false)).withOrderBy(new QFilterOrderBy("label")).withLimit(5));
input.setPermissionEnforcement(PermissionEnforcement.ENFORCE);
for(com.kingsrook.qqq.backend.core.model.data.QRecord tag : new com.kingsrook.qqq.backend.core.actions.tables.QueryAction().execute(input).getRecords())
{
html.append("<li>").append(tag.getValueString("label").replace("&", "&amp;").replace("<", "&lt;").replace(">", "&gt;"))
.append(": ").append(tag.getValueInteger("usageCount")).append("</li>");
}
html.append("</ul></section>");
}
/*******************************************************************************
** Real table aggregates, restricted to readable parent tables. Parent locks
** are post-filtered on rows, so these aggregate totals are approximate.
*******************************************************************************/
private static void attachmentSummary(StringBuilder html) throws QException
{
Aggregate count = new Aggregate().withFieldName("id").withOperator(AggregateOperator.COUNT);
CollaborationSecurity.ensureSessionKeys(QContext.getQInstance(), QContext.getQSession());
Instant weekAgo = now.minus(7, ChronoUnit.DAYS);
int commentsThisWeek = count(CollaborationTables.COMMENT, new QQueryFilter()
.withCriteria("createDate", QCriteriaOperator.GREATER_THAN_OR_EQUALS, weekAgo)
.withCriteria("createDate", QCriteriaOperator.LESS_THAN_OR_EQUALS, now).withCriteria("deletedAt", QCriteriaOperator.IS_BLANK));
int openThreads = count(CollaborationTables.COMMENT, new QQueryFilter().withCriteria("threadId", QCriteriaOperator.IS_BLANK)
.withCriteria("resolvedAt", QCriteriaOperator.IS_BLANK).withCriteria("deletedAt", QCriteriaOperator.IS_BLANK));
int openTasks = count(CollaborationTables.TASK, new QQueryFilter().withCriteria("status", QCriteriaOperator.EQUALS, "OPEN"));
int overdue = count(CollaborationTables.TASK, new QQueryFilter().withCriteria("status", QCriteriaOperator.EQUALS, "OPEN")
.withCriteria("dueAt", QCriteriaOperator.LESS_THAN, now));
int openRequests = count(CollaborationTables.ACCESS_REQUEST, new QQueryFilter().withCriteria("status", QCriteriaOperator.EQUALS, "OPEN"));
int activeFields = count(CollaborationTables.FIELD_DEFINITION, new QQueryFilter().withCriteria("status", QCriteriaOperator.EQUALS, "ACTIVE"));
Aggregate files = new Aggregate().withFieldName("id").withOperator(AggregateOperator.COUNT);
Aggregate bytes = new Aggregate().withFieldName("sizeBytes").withOperator(AggregateOperator.SUM);
AggregateInput input = new AggregateInput(CollaborationTables.ATTACHMENT).withAggregates(List.of(count, bytes));
input.setPermissionEnforcement(PermissionEnforcement.ENFORCE);
List<AggregateResult> results = new AggregateAction().execute(input).getResults();
Object rowCount = results.isEmpty() ? 0 : results.getFirst().getAggregateValue(count);
Object totalBytes = results.isEmpty() ? 0 : results.getFirst().getAggregateValue(bytes);
html.append("<section><h3>Attachments</h3><p>Attachments: ").append(rowCount)
.append("</p><p>Total size (bytes): ").append(totalBytes == null ? 0 : totalBytes)
.append("</p><p>Approximate totals: parent record locks are applied when browsing attachments.</p>");
if("0".equals(String.valueOf(rowCount)))
{
html.append("<p>No attachments yet.</p>");
}
html.append("</section>");
AggregateInput aggregateInput = new AggregateInput(CollaborationTables.ATTACHMENT).withAggregates(List.of(files, bytes));
aggregateInput.setPermissionEnforcement(PermissionEnforcement.ENFORCE);
List<AggregateResult> results = new AggregateAction().execute(aggregateInput).getResults();
int attachments = results.isEmpty() ? 0 : numberOf(results.getFirst().getAggregateValue(files)).intValue();
long totalBytes = results.isEmpty() ? 0 : numberOf(results.getFirst().getAggregateValue(bytes)).longValue();
String fileSubheader = attachments == 0 ? "No attachments yet" : humanSize(totalBytes) + " in total";
return (new MultiStatisticsData("Collaboration", List.of(
new StatisticsGroupData("forum", null, "Conversations", "Last 7 days and open", List.of(
new Statistic("Comments this week", commentsThisWeek, path(CollaborationTables.COMMENT)),
new Statistic("Open threads", openThreads, path(CollaborationTables.COMMENT)))),
new StatisticsGroupData("task_alt", null, "Work", openTasks == 0 && openRequests == 0 ? "Nothing waiting" : "Waiting on someone", List.of(
new Statistic("Open tasks", openTasks, path(CollaborationTables.TASK)),
new Statistic("Overdue tasks", overdue, path(CollaborationTables.TASK)),
new Statistic("Open access requests", openRequests, path(CollaborationTables.ACCESS_REQUEST)))),
new StatisticsGroupData("attach_file", null, "Attachments", fileSubheader, List.of(
new Statistic("Attachments", attachments, path(CollaborationTables.ATTACHMENT)))),
new StatisticsGroupData("tune", null, "Custom fields", activeFields == 0 ? "None active yet" : "Defined by administrators", List.of(
new Statistic("Active custom fields", activeFields, path(CollaborationTables.FIELD_DEFINITION)))))));
}
/*******************************************************************************
** Aggregate rows and distinct owners without loading high-volume personal data.
** Approved and pending tags, most used first: label, table label, records.
*******************************************************************************/
private static void personalSummary(StringBuilder html, String table, String label) throws QException
public static TableData tags() throws QException
{
Aggregate rows = new Aggregate().withFieldName("id").withOperator(AggregateOperator.COUNT);
Aggregate users = new Aggregate().withFieldName("userId").withOperator(AggregateOperator.COUNT_DISTINCT);
AggregateInput input = new AggregateInput(table).withAggregates(List.of(rows, users));
CollaborationSecurity.ensureSessionKeys(QContext.getQInstance(), QContext.getQSession());
QueryInput input = new QueryInput(CollaborationTables.TAG).withFilter(new QQueryFilter().withCriteria("mergedIntoTagId", QCriteriaOperator.IS_BLANK)
.withOrderBy(new QFilterOrderBy("usageCount", false)).withOrderBy(new QFilterOrderBy("label")).withLimit(200));
input.setPermissionEnforcement(PermissionEnforcement.ENFORCE);
List<AggregateResult> results = new AggregateAction().execute(input).getResults();
html.append("<section><h3>").append(label).append("</h3><p>Rows: ").append(results.isEmpty() ? 0 : results.getFirst().getAggregateValue(rows))
.append("</p><p>Distinct users: ").append(results.isEmpty() ? 0 : results.getFirst().getAggregateValue(users)).append("</p></section>");
List<Map<String, Object>> rows = new ArrayList<>();
for(QRecord tag : new QueryAction().execute(input).getRecords())
{
Map<String, Object> row = new LinkedHashMap<>();
row.put("label", tag.getValueString("label"));
row.put("table", tableLabel(tag.getValueString("tableName")));
row.put("records", tag.getValueInteger("usageCount") == null ? 0 : tag.getValueInteger("usageCount"));
rows.add(row);
}
return (table("Tags", NO_TAGS, rows, column("Label", "label", "left"), column("Table", "table", "left"), column("Records", "records", "right")));
}
/*******************************************************************************
** Custom field definitions: name, table, status and index state.
*******************************************************************************/
public static TableData fields() throws QException
{
CollaborationSecurity.ensureSessionKeys(QContext.getQInstance(), QContext.getQSession());
QueryInput input = new QueryInput(CollaborationTables.FIELD_DEFINITION).withFilter(new QQueryFilter()
.withOrderBy(new QFilterOrderBy("tableName")).withOrderBy(new QFilterOrderBy("name")).withLimit(200));
input.setPermissionEnforcement(PermissionEnforcement.ENFORCE);
List<Map<String, Object>> rows = new ArrayList<>();
for(QRecord definition : new QueryAction().execute(input).getRecords())
{
Map<String, Object> row = new LinkedHashMap<>();
row.put("name", definition.getValueString("name"));
row.put("table", tableLabel(definition.getValueString("tableName")));
row.put("status", friendly(definition.getValueString("status")));
row.put("indexState", friendly(definition.getValueString("indexState")));
rows.add(row);
}
return (table("Custom fields", NO_FIELDS, rows, column("Name", "name", "left"), column("Table", "table", "left"),
column("Status", "status", "left"), column("Index state", "indexState", "left")));
}
/*******************************************************************************
** Live viewer counts per table. Only administrators see them (they cover
** records the viewer may not be able to open); others get an explanation.
*******************************************************************************/
public static TableData presence() throws QException
{
CollaborationSecurity.ensureSessionKeys(QContext.getQInstance(), QContext.getQSession());
boolean admin = QContext.getQSession().hasPermission(CollaborationPermissions.ADMIN);
List<Map<String, Object>> rows = new ArrayList<>();
if(admin)
{
var instance = QContext.getQInstance();
var redis = com.kingsrook.qbits.collaboration.presence.RedisPresenceConfig.forInstance(instance);
Map<String, Integer> counts;
if(redis == null)
{
counts = PresenceRegistry.get(instance).counts();
}
else
{
counts = new java.util.TreeMap<>();
var backend = new com.kingsrook.qbits.collaboration.presence.RedisPresenceBackend(redis);
// Read only known table rooms in the administrator's authenticated tenant.
// Never scan Redis keys or silently substitute local counts on failure.
for(String tableName : instance.getTables().keySet())
{
int viewers = backend.snapshot(tableName, null).size();
if(viewers > 0)
{
counts.put(tableName, viewers);
}
}
}
counts.forEach((table, viewers) ->
{
Map<String, Object> row = new LinkedHashMap<>();
row.put("table", tableLabel(table));
row.put("viewers", viewers);
rows.add(row);
});
}
return (table("Who's here", admin ? NOBODY_HERE : PRESENCE_ADMIN_ONLY, rows, column("Table", "table", "left"), column("People", "viewers", "right")));
}
/*******************************************************************************
** Assemble a table widget whose empty state is a sentence, not a blank grid.
*******************************************************************************/
private static TableData table(String label, String emptyMessage, List<Map<String, Object>> rows, TableData.Column... columns)
{
return (new TableData(label, List.of(columns), rows).withNoRowsFoundHTML(emptyMessage).withRowsPerPage(10));
}
/*******************************************************************************
** A plain column; the frontend styles it.
*******************************************************************************/
private static TableData.Column column(String header, String accessor, String align)
{
//////////////////////////////////////////////////////////////////
// no width: frontends size the columns to fit narrow (390 px) cards //
//////////////////////////////////////////////////////////////////
return (new TableData.Column("default", header, accessor, null, align));
}
/*******************************************************************************
** A table's display label, "All tables" when there is none (global).
*******************************************************************************/
private static String tableLabel(String name)
{
if(name == null || name.isBlank())
{
return ("All tables");
}
QTableMetaData table = QContext.getQInstance().getTable(name);
return (table == null || table.getLabel() == null ? name : table.getLabel());
}
/*******************************************************************************
** ACTIVE becomes Active; a missing value becomes a dash.
*******************************************************************************/
private static String friendly(String value)
{
if(value == null || value.isBlank())
{
return ("-");
}
String lower = value.toLowerCase(Locale.ROOT).replace('_', ' ');
return (Character.toUpperCase(lower.charAt(0)) + lower.substring(1));
}
/*******************************************************************************
** Binary sizes with one decimal, like 4.7 KB; whole bytes below 1 KB.
*******************************************************************************/
public static String humanSize(long bytes)
{
if(bytes < 1024)
{
return (bytes + " B");
}
String[] units = { "KB", "MB", "GB", "TB", "PB" };
double value = bytes;
int unit = -1;
while(value >= 1024 && unit < units.length - 1)
{
value /= 1024;
unit++;
}
return (String.format(Locale.ROOT, "%.1f %s", value, units[unit]));
}
/*******************************************************************************
** Aggregate values come back as any numeric type, or null for empty sets.
*******************************************************************************/
private static Number numberOf(Object value)
{
return (value instanceof Number number ? number : 0);
}
/*******************************************************************************
** The path of a Collaboration table page, under the parent app when configured.
*******************************************************************************/
private static String path(String table)
{
CollaborationQBitConfig config = CollaborationRecords.config();
String prefix = config.getParentAppName() == null ? "/" + config.getAppName() : "/" + config.getParentAppName() + "/" + config.getAppName();
return (prefix + "/" + table);
}
@@ -233,7 +402,7 @@ public final class CollaborationHome
/*******************************************************************************
** Count without materializing rows or substituting a privileged session.
*******************************************************************************/
private static long count(String table, QQueryFilter filter) throws QException
private static int count(String table, QQueryFilter filter) throws QException
{
CountInput input = new CountInput(table).withFilter(filter);
input.setPermissionEnforcement(PermissionEnforcement.ENFORCE);
@@ -243,38 +412,26 @@ public final class CollaborationHome
/*******************************************************************************
** Real definition status and index totals, plus ephemeral admin-only counts.
** Setup renderer instantiated by QQQ.
*******************************************************************************/
private static void fieldSummary(StringBuilder html) throws QException
public static class SetupRenderer extends AbstractWidgetRenderer
{
html.append("<section><h3>Custom fields</h3><ul>");
for(String status : List.of("DRAFT", "ACTIVE", "ARCHIVED"))
/*******************************************************************************
** Render using the current session.
*******************************************************************************/
@Override
public RenderWidgetOutput render(RenderWidgetInput input) throws QException
{
html.append("<li>").append(status).append(": ").append(count(CollaborationTables.FIELD_DEFINITION,
new QQueryFilter().withCriteria("status", QCriteriaOperator.EQUALS, status))).append("</li>");
}
for(String state : List.of("NONE", "BUILDING", "READY", "FAILED"))
{
html.append("<li>Index ").append(state).append(": ").append(count(CollaborationTables.FIELD_DEFINITION,
new QQueryFilter().withCriteria("indexState", QCriteriaOperator.EQUALS, state))).append("</li>");
}
html.append("</ul><p>Definitions with storage or index messages: ").append(count(CollaborationTables.FIELD_DEFINITION,
new QQueryFilter().withCriteria("statusMessage", QCriteriaOperator.IS_NOT_BLANK))).append(". See Custom Fields for details.</p></section>");
if(QContext.getQSession().hasPermission(CollaborationPermissions.ADMIN))
{
html.append("<section><h3>Who's here</h3><ul>");
com.kingsrook.qbits.collaboration.presence.PresenceRegistry.get(QContext.getQInstance()).counts().forEach((table, count) ->
html.append("<li>").append(table.replace("&", "&amp;").replace("<", "&lt;").replace(">", "&gt;")).append(": ").append(count).append("</li>"));
html.append("</ul></section>");
return (new RenderWidgetOutput(setup()));
}
}
/*******************************************************************************
** Widget renderer instantiated by QQQ.
** KPI renderer instantiated by QQQ.
*******************************************************************************/
public static class HomeRenderer extends AbstractWidgetRenderer
public static class KpiRenderer extends AbstractWidgetRenderer
{
/*******************************************************************************
** Render using the current session and time.
@@ -282,7 +439,58 @@ public final class CollaborationHome
@Override
public RenderWidgetOutput render(RenderWidgetInput input) throws QException
{
return (new RenderWidgetOutput(new RawHTML(null, CollaborationHome.render(Instant.now()))));
return (new RenderWidgetOutput(kpis(Instant.now())));
}
}
/*******************************************************************************
** Tags renderer instantiated by QQQ.
*******************************************************************************/
public static class TagsRenderer extends AbstractWidgetRenderer
{
/*******************************************************************************
** Render using the current session.
*******************************************************************************/
@Override
public RenderWidgetOutput render(RenderWidgetInput input) throws QException
{
return (new RenderWidgetOutput(tags()));
}
}
/*******************************************************************************
** Custom fields renderer instantiated by QQQ.
*******************************************************************************/
public static class FieldsRenderer extends AbstractWidgetRenderer
{
/*******************************************************************************
** Render using the current session.
*******************************************************************************/
@Override
public RenderWidgetOutput render(RenderWidgetInput input) throws QException
{
return (new RenderWidgetOutput(fields()));
}
}
/*******************************************************************************
** Presence renderer instantiated by QQQ.
*******************************************************************************/
public static class PresenceRenderer extends AbstractWidgetRenderer
{
/*******************************************************************************
** Render using the current session.
*******************************************************************************/
@Override
public RenderWidgetOutput render(RenderWidgetInput input) throws QException
{
return (new RenderWidgetOutput(presence()));
}
}
}
@@ -0,0 +1,129 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.app;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.Locale;
import java.util.Set;
import com.kingsrook.qbits.collaboration.CollaborationRecords;
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
import com.kingsrook.qqq.backend.core.actions.collaboration.RecordFeaturesResolver;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordFeature;
import com.kingsrook.qqq.backend.core.model.metadata.tables.Capability;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
/*******************************************************************************
** The Setup card on the Collaboration home: which record features are on, for
** which tables, and whether that comes from Feature Settings rows or from the
** code defaults. Uses the same resolution the runtime uses
** (RecordFeaturesResolver: code declarations, then the committed rows), so the
** card never reports something the runtime would not do.
*******************************************************************************/
public final class CollaborationSetupStatus
{
private static final int NAMED_TABLES = 5;
/*******************************************************************************
** One line of the card.
*******************************************************************************/
public record Item(String feature, String state, String where, String source, String nextStep)
{
}
/*******************************************************************************
** Utility class.
*******************************************************************************/
private CollaborationSetupStatus()
{
}
/*******************************************************************************
** One line for Feature Settings itself, then one per record feature.
*******************************************************************************/
public static List<Item> load(QInstance instance) throws QException
{
List<QRecord> rows = CollaborationRecords.query(CollaborationTables.FEATURE_SETTING, new QQueryFilter().withLimit(1000));
Set<String> featuresWithRows = new HashSet<>();
rows.forEach(row -> featuresWithRows.add(row.getValueString("feature")));
List<Item> items = new ArrayList<>();
items.add(rows.isEmpty()
? new Item("Feature settings", "Using code defaults", "-", "Configuration", "Run Create rows from configuration on Feature Settings to see and edit every setting.")
: new Item("Feature settings", "Set up", rows.size() + " row" + (rows.size() == 1 ? "" : "s"), "Feature Settings", null));
List<QTableMetaData> hostTables = CollectionUtils.nonNullMap(instance.getTables()).values().stream()
.filter(table -> !table.getName().startsWith("collaboration_") && !Boolean.TRUE.equals(table.getIsHidden()) && table.getPrimaryKeyField() != null
&& (table.isCapabilityEnabled(instance.getBackend(table.getBackendName()), Capability.TABLE_GET) || table.isCapabilityEnabled(instance.getBackend(table.getBackendName()), Capability.TABLE_QUERY)))
.sorted((a, b) -> a.getName().compareTo(b.getName())).toList();
for(RecordFeature feature : RecordFeature.values())
{
List<String> on = new ArrayList<>();
for(QTableMetaData table : hostTables)
{
if(RecordFeaturesResolver.resolve(instance, table).getFeatures().get(feature).getEnabled())
{
on.add(table.getLabel() == null ? table.getName() : table.getLabel());
}
}
String source = featuresWithRows.contains(feature.name()) ? "Feature Settings" : "Configuration";
String name = friendly(feature);
if(on.isEmpty())
{
items.add(new Item(name, "Off", "No table", source, "Add a Feature Settings row (scope INSTANCE, or TABLE for one table) and turn it on."));
}
else
{
boolean all = on.size() == hostTables.size();
String where = all ? "Every table" : String.join(", ", on.subList(0, Math.min(NAMED_TABLES, on.size()))) + (on.size() > NAMED_TABLES ? " and " + (on.size() - NAMED_TABLES) + " more" : "");
items.add(new Item(name, all ? "On" : "On for " + on.size() + " of " + hostTables.size() + " tables", where, source, null));
}
}
return (items);
}
/*******************************************************************************
** TAGS becomes Tags, CUSTOM_FIELDS becomes Custom fields.
*******************************************************************************/
static String friendly(RecordFeature feature)
{
String lower = feature.name().toLowerCase(Locale.ROOT).replace('_', ' ');
return (Character.toUpperCase(lower.charAt(0)) + lower.substring(1));
}
}
@@ -53,15 +53,42 @@ public class AssignmentService
** Full replacement of both roles; null clears a role.
*******************************************************************************/
public QRecord assign(String table, Serializable recordId, String ownerUserId, String assigneeUserId) throws QException
{
return (assignRoles(table, recordId, ownerUserId, assigneeUserId, java.util.Set.of("ownerUserId", "assigneeUserId")));
}
/*******************************************************************************
** Omitted roles remain unchanged; an explicitly supplied null clears a role.
*******************************************************************************/
public QRecord assignRoles(String table, Serializable recordId, String ownerUserId, String assigneeUserId, java.util.Set<String> roles) throws QException
{
ParentRecordAccess.requireFeature(table, recordId, "assign");
String id = FollowService.canonicalId(table, recordId);
if(roles == null || roles.isEmpty() || !java.util.Set.of("ownerUserId", "assigneeUserId").containsAll(roles))
{
throw (new com.kingsrook.qqq.backend.core.exceptions.QBadRequestException("Supply ownerUserId or assigneeUserId; omitted roles remain unchanged."));
}
var instance = com.kingsrook.qqq.backend.core.context.QContext.getQInstance();
var ownership = com.kingsrook.qqq.backend.core.actions.collaboration.RecordFeaturesResolver.resolve(instance, instance.getTable(table)).getOwnership();
if(ownership != null && ((roles.contains("ownerUserId") && ownership.getOwnerFieldName() != null)
|| (roles.contains("assigneeUserId") && ownership.getAssigneeFieldName() != null)))
{
throw (new com.kingsrook.qqq.backend.core.exceptions.QBadRequestException("Update native ownership fields through the parent record's versioned update endpoint."));
}
WorkSupport.requireReader(ownerUserId, table, id);
WorkSupport.requireReader(assigneeUserId, table, id);
synchronized(CollaborationRecords.config())
{
Optional<QRecord> old = get(table, id);
QRecord change = new QRecord().withValue("ownerUserId", ownerUserId).withValue("assigneeUserId", assigneeUserId);
QRecord change = new QRecord();
if(roles.contains("ownerUserId"))
{
change.setValue("ownerUserId", ownerUserId);
}
if(roles.contains("assigneeUserId"))
{
change.setValue("assigneeUserId", assigneeUserId);
}
QRecord row;
if(old.isEmpty())
{
@@ -73,13 +100,13 @@ public class AssignmentService
row = get(table, id).orElseThrow();
}
ActivityWriter.write(table, id, "ASSIGNED", "Record assignment changed", CollaborationTables.RECORD_ASSIGNMENT, row.getValueString("id"));
if(assigneeUserId != null && (old.isEmpty() || !Objects.equals(old.get().getValueString("assigneeUserId"), assigneeUserId)))
if(roles.contains("assigneeUserId") && assigneeUserId != null && (old.isEmpty() || !Objects.equals(old.get().getValueString("assigneeUserId"), assigneeUserId)))
{
new FollowService().ensureFollower(assigneeUserId, table, id, "ASSIGNED");
if(!assigneeUserId.equals(PersonalRecords.currentUserId()))
{
Notifications.send(new CollaborationNotification(assigneeUserId, "A record was assigned to you", null,
CollaborationNotification.Category.TASK, table, id, Notifications.link(table, id)));
CollaborationNotification.Category.TASK, table, id, Notifications.link(table, id)).withEvent(row.getValue("id"), PersonalRecords.currentUserId()));
}
}
return (row);
@@ -0,0 +1,121 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.attachments;
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
import com.kingsrook.qbits.collaboration.permissions.CollaborationPermissions;
import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
/*******************************************************************************
** A visible native adoption action for an existing original. It never
** accepts replacement bytes, a storage path, an actor, or an intake scope.
*******************************************************************************/
public final class AdoptRetainedAttachmentProcess
{
public static final String NAME = "collaborationAdoptRetainedAttachment";
/*******************************************************************************
** Metadata utility.
*******************************************************************************/
private AdoptRetainedAttachmentProcess()
{
}
/*******************************************************************************
** Standard QQQ confirmation and result screens, attached to the attachment table.
*******************************************************************************/
public static QProcessMetaData metadata()
{
return (new QProcessMetaData().withName(NAME).withTableName(CollaborationTables.ATTACHMENT).withLabel("Retain existing original")
.withMinInputRecords(1).withMaxInputRecords(1).withPermissionRules(CollaborationPermissions.permissionRules())
.withStep(new com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData().withName("prepare").withCode(new QCodeReference(Prepare.class))
.withInputData(new com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMetaData()
.withRecordListMetaData(new com.kingsrook.qqq.backend.core.model.metadata.processes.QRecordListMetaData().withTableName(CollaborationTables.ATTACHMENT))))
.withStep(new com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendStepMetaData().withName("confirm").withLabel("Retain this original")
.withComponent(new com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendComponentMetaData()
.withType(com.kingsrook.qqq.backend.core.model.metadata.processes.QComponentType.EDIT_FORM))
.withFormField(new QFieldMetaData("confirmAdoption", QFieldType.BOOLEAN).withLabel("Verify and retain the existing original without copying bytes").withDefaultValue(false).withIsRequired(true)))
.withStep(new com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData().withName("adopt").withCode(new QCodeReference(Adopt.class)))
.withStep(new com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendStepMetaData().withName("result").withLabel("Original retained")
.withComponent(new com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendComponentMetaData()
.withType(com.kingsrook.qqq.backend.core.model.metadata.processes.QComponentType.VIEW_FORM))
.withViewField(new QFieldMetaData("attachmentId", QFieldType.INTEGER).withLabel("Retained attachment")
.withPossibleValueSourceName(com.kingsrook.qbits.collaboration.model.CollaborationTables.ATTACHMENT))));
}
/*******************************************************************************
** Resolve one visible attachment selection before displaying confirmation.
*******************************************************************************/
public static final class Prepare implements BackendStep
{
/*******************************************************************************
** Read-only preview under current actor and parent visibility.
*******************************************************************************/
@Override
public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
if(input.getRecords() == null || input.getRecords().size() != 1)
{
throw (new QException("Select exactly one attachment."));
}
var row = new AttachmentsService().requireAdoptable(input.getRecords().getFirst().getValueInteger("id"));
if(row == null)
{
throw (new QException("Attachment is unavailable."));
}
if(row.getValueString("intakeKey") != null)
{
throw (new QException("Retry this upload through its original intake request to preserve its context."));
}
output.addValue("attachmentId", row.getValue("id"));
output.addValue("confirmAdoption", false);
}
}
/*******************************************************************************
** Execution reauthorizes independently of the earlier preview.
*******************************************************************************/
public static final class Adopt implements BackendStep
{
/*******************************************************************************
** The owner returns only after mandatory lifecycle registration completes.
*******************************************************************************/
@Override
public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
if(!Boolean.TRUE.equals(input.getValueBoolean("confirmAdoption")))
{
throw (new QException("Confirm adoption of this existing original to continue."));
}
output.addValue("attachmentId", new AttachmentsService().adoptRetainedAttachment(input.getValueInteger("attachmentId")).id());
}
}
}
@@ -0,0 +1,201 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.attachments;
import java.io.Serializable;
import java.util.List;
import com.kingsrook.qbits.collaboration.CollaborationQBitConfig;
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
import com.kingsrook.qbits.collaboration.security.ParentRecordAccess;
import com.kingsrook.qqq.backend.core.actions.customizers.QCodeLoader;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.count.CountInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.MetaDataWithPermissionRules;
import com.kingsrook.qqq.backend.core.model.metadata.tables.Capability;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QSupplementalTableMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
/*******************************************************************************
** Optional owner policy for parents whose dynamic visibility cannot be expressed
** by aggregate SQL. Count/aggregate require one currently authorized parent.
** Ordinary collaboration policies retain their existing count behavior.
*******************************************************************************/
public final class AttachmentAggregateGuard extends AttachmentSecurity
{
private static final String PINNED_POLICY = "collaboration.attachmentPinnedAggregateParent";
/*******************************************************************************
** Hide unscoped list totals in both instance and table metadata. Native count
** remains available to callers supplying the required authorized parent pin.
*******************************************************************************/
public static QTableMetaData configure(QTableMetaData table, CollaborationQBitConfig config)
{
if(policy(config).requiresPinnedAggregateParent())
{
table.withSupplementalMetaData(new PinnedParentPolicy());
}
if(table.getSupplementalMetaData(PINNED_POLICY) != null)
{
table.withoutCapabilities(Capability.TABLE_COUNT, Capability.QUERY_STATS);
}
return (table);
}
/*******************************************************************************
** Preserve ordinary view/write rules, then narrow aggregate inputs only.
*******************************************************************************/
@Override
public void checkPermissionsThrowing(AbstractActionInput input, MetaDataWithPermissionRules metadata) throws QPermissionDeniedException
{
super.checkPermissionsThrowing(input, metadata);
if(!(input instanceof CountInput) && !(input instanceof AggregateInput))
{
return;
}
try
{
CollaborationQBitConfig config = (CollaborationQBitConfig) QContext.getQInstance().getQBits().values().stream()
.filter(qbit -> qbit.getConfig() instanceof CollaborationQBitConfig).findFirst().orElseThrow().getConfig();
QTableMetaData tableMetadata = QContext.getQInstance().getTable(CollaborationTables.ATTACHMENT);
if(tableMetadata.getSupplementalMetaData(PINNED_POLICY) == null && !policy(config).requiresPinnedAggregateParent() && !hasRetainedOriginal())
{
return;
}
QQueryFilter filter = input instanceof CountInput count ? count.getFilter() : ((AggregateInput) input).getFilter();
var joins = input instanceof CountInput count ? count.getQueryJoins() : ((AggregateInput) input).getQueryJoins();
if(joins != null && !joins.isEmpty() || filter == null || filter.getBooleanOperator() != QQueryFilter.BooleanOperator.AND)
{
throw (new QException("A single pinned attachment parent is required for counts and aggregates."));
}
rejectSetOperations(filter, 0, new int[]{0});
Serializable table = literal(filter, "tableName");
Serializable record = literal(filter, "recordId");
if(!(table instanceof String name) || name.isBlank() || record == null)
{
throw (new QException("A single pinned attachment parent is required for counts and aggregates."));
}
ParentRecordAccess.requireVisibleParent(name, record);
}
catch(QException | RuntimeException denied)
{
throw (new QPermissionDeniedException("Counts and aggregates require one currently readable attachment parent."));
}
}
/*******************************************************************************
** A literal top-level AND equality constrains every row, even when other
** independent subfilters contain OR. Conflicting or computed pins are refused.
*******************************************************************************/
private static Serializable literal(QQueryFilter filter, String field) throws QException
{
Serializable result = null;
for(var criterion : filter.getCriteria() == null ? List.<com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria>of() : filter.getCriteria())
{
if(field.equals(criterion.getFieldName()) && criterion.getOperator() == QCriteriaOperator.EQUALS)
{
if(criterion.getValues() == null || criterion.getValues().size() != 1)
{
throw (new QException("An exact parent equality is required."));
}
Serializable value = criterion.getValues().getFirst();
if(!(value instanceof String) && !(value instanceof Number) || result != null && !result.equals(value))
{
throw (new QException("Computed or conflicting parent equality is not supported."));
}
result = value;
}
}
return (result);
}
/*******************************************************************************
** Set-operation filters may combine independent branches; never assume their
** SQL obeys an ordinary root AND pin. Bound traversal before SQL execution.
*******************************************************************************/
private static void rejectSetOperations(QQueryFilter filter, int depth, int[] visited) throws QException
{
if(depth > 64 || ++visited[0] > 4096 || filter.getSubFilterSetOperator() != null)
{
throw (new QException("Unsupported aggregate filter shape."));
}
for(QQueryFilter child : filter.getSubFilters() == null ? List.<QQueryFilter>of() : filter.getSubFilters())
{
rejectSetOperations(child, depth + 1, visited);
}
}
/*******************************************************************************
** Capability policy is configuration-only and must not depend on a request.
*******************************************************************************/
private static AttachmentLifecyclePolicy policy(CollaborationQBitConfig config)
{
return (java.util.Objects.requireNonNull(QCodeLoader.getAdHoc(AttachmentLifecyclePolicy.class, config.getAttachmentLifecycle()), "Attachment lifecycle policy is unavailable."));
}
/*******************************************************************************
** A fresh instance may have lost its former lifecycle configuration. Inspect
** only existence through private owner metadata, never return retained rows.
** Failure propagates to aggregate denial; there is no permissive fallback.
*******************************************************************************/
private static boolean hasRetainedOriginal() throws QException
{
var input = new com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput(CollaborationTables.ATTACHMENT)
.withFilter(new QQueryFilter().withCriteria("retained", QCriteriaOperator.EQUALS, true).withLimit(1))
.withShouldFetchHeavyFields(false).withShouldGenerateDisplayValues(false);
input.setTableMetaData(com.kingsrook.qbits.collaboration.CollaborationRecords.cleanupMetadata(CollaborationTables.ATTACHMENT));
input.withPermissionBypass("Owner aggregate guard checks only whether retained originals require pinned-parent authorization; no row data is exposed.");
return (!new com.kingsrook.qqq.backend.core.actions.tables.QueryAction().execute(input).getRecords().isEmpty());
}
/*******************************************************************************
** An installed capability restriction survives subsequent lifecycle changes
** and metadata clones. It carries no request or authorization state.
*******************************************************************************/
public static final class PinnedParentPolicy extends QSupplementalTableMetaData
{
/*******************************************************************************
** Stable owner metadata identity.
*******************************************************************************/
@Override
public String getType()
{
return (PINNED_POLICY);
}
/*******************************************************************************
** The marker is immutable and has no nested mutable state.
*******************************************************************************/
@Override
protected QSupplementalTableMetaData finishClone(QSupplementalTableMetaData clone)
{
return (clone);
}
}
}
@@ -66,7 +66,8 @@ public class AttachmentCleanup
while(true)
{
List<QRecord> rows = com.kingsrook.qbits.collaboration.CollaborationRecords.cleanupQuery(com.kingsrook.qbits.collaboration.model.CollaborationTables.ATTACHMENT, new QQueryFilter().withCriteria("tableName", QCriteriaOperator.EQUALS, table)
.withCriteria("recordId", QCriteriaOperator.IN, ids.stream().map(Object::toString).toList()).withLimit(200));
.withCriteria("recordId", QCriteriaOperator.IN, ids.stream().map(Object::toString).toList())
.withCriteria("parentDeleted", QCriteriaOperator.NOT_EQUALS, true).withLimit(200));
if(rows.isEmpty())
{
return;
@@ -0,0 +1,527 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.attachments;
import java.io.Serializable;
import java.time.Instant;
import java.util.List;
import com.kingsrook.qbits.collaboration.CollaborationRecords;
import com.kingsrook.qbits.collaboration.CollaborationSecurity;
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
import com.kingsrook.qbits.collaboration.permissions.CollaborationPermissions;
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
import com.kingsrook.qqq.backend.core.actions.permissions.CustomPermissionChecker;
import com.kingsrook.qqq.backend.core.actions.tables.DeleteAction;
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
import com.kingsrook.qqq.backend.core.context.CapturedContext;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryLockMode;
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.MetaDataWithPermissionRules;
import com.kingsrook.qqq.backend.core.model.session.QSystemUserSession;
import com.kingsrook.qqq.backend.core.modules.backend.QBackendModuleDispatcher;
/*******************************************************************************
** Durable deletion intent, committed with metadata removal before deleting bytes.
*******************************************************************************/
public class AttachmentCleanupQueue implements CustomPermissionChecker
{
private final AttachmentStorage storage;
/*******************************************************************************
** Metadata permission checks do not create a storage client.
*******************************************************************************/
public AttachmentCleanupQueue()
{
this(new AttachmentStorage());
}
/*******************************************************************************
** Use the same provider boundary as the authorized attachment operation.
*******************************************************************************/
AttachmentCleanupQueue(AttachmentStorage storage)
{
this.storage = storage;
}
/*******************************************************************************
** Only admins may inspect recovery locations; no direct mutations are allowed.
*******************************************************************************/
@Override
public void checkPermissionsThrowing(AbstractActionInput input, MetaDataWithPermissionRules metadata) throws QPermissionDeniedException
{
if(!(input instanceof QueryInput) && !(input instanceof com.kingsrook.qqq.backend.core.model.actions.tables.get.GetInput)
&& !(input instanceof com.kingsrook.qqq.backend.core.model.actions.tables.count.CountInput))
{
throw (new QPermissionDeniedException("Attachment cleanup records are service-managed."));
}
requireOperator();
}
/*******************************************************************************
** The caller has already authorized the parent and attachment owner.
*******************************************************************************/
void delete(QRecord row, boolean cleanup) throws QException
{
QBackendTransaction outer = QContext.getQBackendTransaction();
if(outer != null)
{
QRecord pending = enqueueAndDelete(row, cleanup);
CapturedContext context = new CapturedContext(QContext.getQInstance(), QContext.getQSession());
outer.addAfterCommitCallback(() ->
{
try
{
QContext.withTemporaryContext(context, () -> finish(pending));
}
catch(QException e)
{
throw (new IllegalStateException("Attachment cleanup remains durably queued.", e));
}
});
return;
}
QRecord pending;
try(QBackendTransaction transaction = QBackendTransaction.openFor(new DeleteInput(CollaborationTables.ATTACHMENT)))
{
QContext.setTransaction(transaction);
try
{
pending = enqueueAndDelete(row, cleanup);
transaction.commit();
}
catch(QException | RuntimeException failure)
{
transaction.rollback();
throw (failure);
}
finally
{
QContext.clearTransaction();
}
}
finish(pending);
}
/*******************************************************************************
** Store the recovery locator in the same database transaction as row deletion.
*******************************************************************************/
private QRecord enqueueAndDelete(QRecord row, boolean cleanup) throws QException
{
QRecord intent = new QRecord().withValue("attachmentId", row.getValue("id"))
.withValue("bucketName", row.getValue("bucketName")).withValue("storageKey", row.getValue("storageKey"));
InsertInput insert = new InsertInput(CollaborationTables.ATTACHMENT_CLEANUP).withRecords(List.of(intent))
.withTransaction(QContext.getQBackendTransaction());
insert.withPermissionBypass("Authorized attachment deletion records durable storage cleanup.");
List<QRecord> inserted = new InsertAction().execute(insert).getRecords();
AttachmentRecords.checkErrors(inserted);
DeleteInput delete = new DeleteInput(CollaborationTables.ATTACHMENT).withPrimaryKeys(List.of(row.getValue("id")))
.withTransaction(QContext.getQBackendTransaction());
delete.setTableMetaData(cleanup ? CollaborationRecords.cleanupMetadata(CollaborationTables.ATTACHMENT) : AttachmentRecords.storageTable());
delete.withPermissionBypass("Attachment deletion was authorized before recording cleanup.");
AttachmentRecords.checkErrors(new DeleteAction().execute(delete).getRecordsWithErrors());
return (inserted.getFirst());
}
/*******************************************************************************
** Durable upload ownership requires real transactional row locking.
*******************************************************************************/
static boolean supportsUploadOwnership() throws QException
{
var backend = QContext.getQInstance().getBackendForTable(CollaborationTables.ATTACHMENT_CLEANUP);
return (new QBackendModuleDispatcher().getQBackendModule(backend).supportsQueryLockMode(backend, QueryLockMode.FOR_UPDATE_SKIP_LOCKED));
}
/*******************************************************************************
** Commit the locator independently, then lock it before any provider write.
** If a worker claimed it first, abort without uploading any bytes.
*******************************************************************************/
QRecord beginUpload(String bucket, String key) throws QException
{
if(!supportsUploadOwnership())
{
return (null);
}
QBackendTransaction transaction = QContext.getQBackendTransaction();
if(transaction == null)
{
throw (new QException("Durable attachment uploads require an active transaction."));
}
CapturedContext context = new CapturedContext(QContext.getQInstance(), QContext.getQSession());
QRecord[] pending = new QRecord[1];
QContext.withTemporaryContext(context, () ->
{
InsertInput insert = new InsertInput(CollaborationTables.ATTACHMENT_CLEANUP).withRecords(List.of(
new QRecord().withValue("bucketName", bucket).withValue("storageKey", key)));
insert.withPermissionBypass("Authorized upload records its immutable locator before external I/O.");
List<QRecord> inserted = new InsertAction().execute(insert).getRecords();
AttachmentRecords.checkErrors(inserted);
pending[0] = inserted.getFirst();
});
if(lock(pending[0], transaction) == null)
{
throw (new QException("Upload cleanup intent was already claimed; retry the upload."));
}
Runnable cleanup = () ->
{
try
{
QContext.withTemporaryContext(context, () -> finish(pending[0]));
}
catch(QException e)
{
throw (new IllegalStateException("Upload cleanup remains durably queued.", e));
}
};
transaction.addAfterRollbackCallback(cleanup);
transaction.addAfterCommitCallback(cleanup);
return (pending[0]);
}
/*******************************************************************************
** Publish metadata and retire its upload intent in the same transaction.
*******************************************************************************/
void completeUpload(QRecord pending) throws QException
{
if(pending != null)
{
DeleteInput delete = new DeleteInput(CollaborationTables.ATTACHMENT_CLEANUP).withPrimaryKey(pending.getValue("id"))
.withTransaction(QContext.getQBackendTransaction());
delete.withPermissionBypass("Attachment metadata now owns the uploaded object in this transaction.");
AttachmentRecords.checkErrors(new DeleteAction().execute(delete).getRecordsWithErrors());
}
}
/*******************************************************************************
** Skip active owners without waiting or updating their locked intent.
*******************************************************************************/
private QRecord lock(QRecord pending, QBackendTransaction transaction) throws QException
{
QueryInput query = new QueryInput(CollaborationTables.ATTACHMENT_CLEANUP).withTransaction(transaction)
.withLockMode(QueryLockMode.FOR_UPDATE_SKIP_LOCKED)
.withFilter(new QQueryFilter().withCriteria("id", QCriteriaOperator.EQUALS, pending.getValue("id")));
query.withPermissionBypass("Internal attachment lifecycle owns this immutable cleanup locator.");
List<QRecord> rows = new QueryAction().execute(query).getRecords();
return (rows.isEmpty() ? null : rows.getFirst());
}
/*******************************************************************************
** Distinguish an active owner from a committed row requiring delayed retry.
*******************************************************************************/
private enum CleanupOutcome
{
COMPLETED, ACTIVE, REFERENCED, UNCERTAIN_UPLOAD
}
/*******************************************************************************
** Hold the same row lock as an uploader until deletion and acknowledgement end.
*******************************************************************************/
private CleanupOutcome finish(QRecord pending) throws QException
{
if(!supportsUploadOwnership())
{
return (finishUnlocked(pending));
}
CleanupOutcome[] outcome = { CleanupOutcome.ACTIVE };
CapturedContext context = new CapturedContext(QContext.getQInstance(), QContext.getQSession());
QContext.withTemporaryContext(context, () ->
{
try(QBackendTransaction transaction = QBackendTransaction.openFor(new DeleteInput(CollaborationTables.ATTACHMENT_CLEANUP)))
{
QContext.setTransaction(transaction);
try
{
QRecord locked = lock(pending, transaction);
if(locked != null)
{
outcome[0] = finishUnlocked(locked);
}
transaction.commit();
}
catch(QException | RuntimeException failure)
{
transaction.rollback();
throw (failure);
}
finally
{
QContext.clearTransaction();
}
}
});
return (outcome[0]);
}
/*******************************************************************************
** Never erase bytes while metadata still exists, including nontransactional hosts.
** Provider or queue deletion failure retains the intent for idempotent replay.
*******************************************************************************/
private CleanupOutcome finishUnlocked(QRecord pending) throws QException
{
if(pending.getValue("resolvedAt") != null)
{
return (CleanupOutcome.COMPLETED);
}
if(!AttachmentRecords.query(new QQueryFilter()
.withCriteria("bucketName", QCriteriaOperator.EQUALS, pending.getValue("bucketName"))
.withCriteria("storageKey", QCriteriaOperator.EQUALS, pending.getValue("storageKey"))).isEmpty())
{
return (CleanupOutcome.REFERENCED);
}
if(pending.getValue("attachmentId") == null)
{
if(!storage.cleanupUpload(pending.getValueString("bucketName"), pending.getValueString("storageKey")))
{
return (CleanupOutcome.UNCERTAIN_UPLOAD);
}
}
else
{
storage.delete(pending.getValueString("bucketName"), pending.getValueString("storageKey"));
}
DeleteInput delete = new DeleteInput(CollaborationTables.ATTACHMENT_CLEANUP).withPrimaryKeys(List.of(pending.getValue("id"))).withTransaction(QContext.getQBackendTransaction());
delete.withPermissionBypass("Storage cleanup completed; remove its durable intent.");
AttachmentRecords.checkErrors(new DeleteAction().execute(delete).getRecordsWithErrors());
return (CleanupOutcome.COMPLETED);
}
/*******************************************************************************
** Rollback compensation must survive a provider outage. Persist independently
** of the failed transaction, then attempt cleanup with the original context.
*******************************************************************************/
void compensateUpload(String bucket, String key)
{
CapturedContext context = new CapturedContext(QContext.getQInstance(), QContext.getQSession());
try
{
QContext.withTemporaryContext(context, () ->
{
InsertInput insert = new InsertInput(CollaborationTables.ATTACHMENT_CLEANUP).withRecords(List.of(
new QRecord().withValue("bucketName", bucket).withValue("storageKey", key)));
insert.withPermissionBypass("Failed authorized upload retains its immutable storage locator for compensation.");
List<QRecord> inserted = new InsertAction().execute(insert).getRecords();
AttachmentRecords.checkErrors(inserted);
finish(inserted.getFirst());
});
}
catch(QException e)
{
throw (new IllegalStateException("Upload compensation could not complete.", e));
}
}
/*******************************************************************************
** Bounded recovery from committed intents, independent of the original service.
*******************************************************************************/
public int retryPending() throws QException
{
requireOperator();
if(QContext.getQBackendTransaction() != null)
{
throw (new QException("Attachment cleanup must run outside a caller transaction."));
}
QueryInput query = new QueryInput(CollaborationTables.ATTACHMENT_CLEANUP).withFilter(new QQueryFilter()
.withCriteria("resolvedAt", QCriteriaOperator.IS_BLANK).withSubFilter(new QQueryFilter()
.withBooleanOperator(QQueryFilter.BooleanOperator.OR)
.withCriteria("nextAttemptAt", QCriteriaOperator.IS_BLANK)
.withCriteria("nextAttemptAt", QCriteriaOperator.LESS_THAN_OR_EQUALS, Instant.now()))
.withOrderBy(new QFilterOrderBy("id")).withLimit(100));
query.withPermissionBypass("Authorized operator retries committed storage cleanup.");
List<QRecord> pending = new QueryAction().execute(query).getRecords();
int completed = 0;
QException failure = null;
for(QRecord row : pending)
{
try
{
CleanupOutcome outcome = finish(row);
if(outcome == CleanupOutcome.COMPLETED)
{
completed++;
}
else if(outcome == CleanupOutcome.REFERENCED || outcome == CleanupOutcome.UNCERTAIN_UPLOAD)
{
defer(row);
}
}
catch(QException | RuntimeException e)
{
try
{
defer(row);
}
catch(QException | RuntimeException deferFailure)
{
e.addSuppressed(deferFailure);
}
if(failure == null)
{
failure = new QException("Attachment cleanup remains pending; retry after the provider or database recovers.", e);
}
}
}
if(failure != null)
{
throw (failure);
}
return (completed);
}
/*******************************************************************************
** Persist backoff so failing entries do not monopolize the next bounded batch.
*******************************************************************************/
private void defer(QRecord pending) throws QException
{
UpdateInput update = new UpdateInput(CollaborationTables.ATTACHMENT_CLEANUP).withRecords(List.of(
new QRecord().withValue("id", pending.getValue("id")).withValue("nextAttemptAt", Instant.now().plusSeconds(60))));
update.withPermissionBypass("Committed cleanup failed or is not yet safe; retain it for a later attempt.");
AttachmentRecords.checkErrors(new UpdateAction().execute(update).getRecords());
}
/*******************************************************************************
** An identified administrator attests provider settlement; absence or age alone
** is not evidence. Keep the receipt on the service-managed row permanently.
*******************************************************************************/
public void resolveUpload(Serializable id, String evidence) throws QException
{
String actor = requireSettlementOperator();
validateEvidence(evidence);
if(id == null || QContext.getQBackendTransaction() != null || !supportsUploadOwnership())
{
throw (new QException("Upload settlement requires a record ID and an independent locking transaction."));
}
try(QBackendTransaction transaction = QBackendTransaction.openFor(new UpdateInput(CollaborationTables.ATTACHMENT_CLEANUP)))
{
QContext.setTransaction(transaction);
try
{
QRecord pending = lock(new QRecord().withValue("id", id), transaction);
if(pending == null)
{
throw (new QException("Cleanup record is active or no longer exists. Retry after its owner finishes."));
}
if(pending.getValue("resolvedAt") != null)
{
transaction.commit();
return;
}
if(pending.getValue("attachmentId") != null)
{
throw (new QException("Use normal cleanup retry for committed attachment deletions."));
}
if(!AttachmentRecords.query(new QQueryFilter()
.withCriteria("bucketName", QCriteriaOperator.EQUALS, pending.getValue("bucketName"))
.withCriteria("storageKey", QCriteriaOperator.EQUALS, pending.getValue("storageKey"))).isEmpty())
{
throw (new QException("An attachment still references this object; settlement is not allowed."));
}
storage.delete(pending.getValueString("bucketName"), pending.getValueString("storageKey"));
UpdateInput update = new UpdateInput(CollaborationTables.ATTACHMENT_CLEANUP).withTransaction(transaction)
.withRecords(List.of(new QRecord().withValue("id", pending.getValue("id"))
.withValue("resolvedAt", Instant.now()).withValue("resolvedBy", actor)
.withValue("settlementEvidence", evidence.strip())));
update.withPermissionBypass("Identified collaboration administrator records confirmed upload settlement after storage cleanup.");
AttachmentRecords.checkErrors(new UpdateAction().execute(update).getRecords());
transaction.commit();
}
catch(QException | RuntimeException failure)
{
transaction.rollback();
throw (failure);
}
finally
{
QContext.clearTransaction();
}
}
}
/*******************************************************************************
** The form and service both reject missing or oversized evidence.
*******************************************************************************/
static void validateEvidence(String evidence) throws QException
{
if(evidence == null || evidence.isBlank() || evidence.strip().length() > 2000)
{
throw (new QException("Record provider completion or cancellation evidence (1–2000 characters)."));
}
}
/*******************************************************************************
** Settlement is a deliberate, attributable admin action, never scheduled work.
*******************************************************************************/
static String requireSettlementOperator() throws QPermissionDeniedException
{
var session = QContext.getQSession();
String actor = CollaborationSecurity.userId(session);
if(session == null || session instanceof QSystemUserSession || !session.hasPermission(CollaborationPermissions.ADMIN)
|| actor == null || actor.isBlank() || actor.length() > 255)
{
throw (new QPermissionDeniedException("An identified collaboration administrator must confirm upload settlement."));
}
return (actor);
}
/*******************************************************************************
** Scheduled system work and explicit admin recovery share one authorization rule.
*******************************************************************************/
static void requireOperator() throws QPermissionDeniedException
{
var session = QContext.getQSession();
if(!(session instanceof QSystemUserSession) && (session == null || !session.hasPermission(CollaborationPermissions.ADMIN)))
{
throw (new QPermissionDeniedException("This needs the collaboration.admin permission."));
}
}
}
@@ -0,0 +1,50 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.attachments;
/*******************************************************************************
** Verified bytes returned only after authorization, integrity and audit checks.
** Storage coordinates never cross this boundary. Content is bounded to 20 MiB.
*******************************************************************************/
public record AttachmentContent(Integer attachmentId, String fileName, String contentType, String sha256, byte[] bytes)
{
/*******************************************************************************
** Own the byte array so a caller cannot change a retained receipt's content.
*******************************************************************************/
public AttachmentContent
{
if(bytes == null || bytes.length > 20 * 1024 * 1024)
{
throw (new IllegalArgumentException("Attachment content is missing or exceeds the supported limit."));
}
bytes = bytes.clone();
}
/*******************************************************************************
** Return caller-owned bytes without exposing the immutable content buffer.
*******************************************************************************/
@Override
public byte[] bytes()
{
return (bytes.clone());
}
}
@@ -0,0 +1,41 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.attachments;
import com.kingsrook.qqq.backend.core.exceptions.QException;
/*******************************************************************************
** Exact stored content failed its immutable metadata check. Consumers must not
** retry this as a transport outage or release the unverified bytes.
*******************************************************************************/
public class AttachmentContentIntegrityException extends QException
{
/*******************************************************************************
** Fixed public diagnostic, with no object locator or provider details.
*******************************************************************************/
public AttachmentContentIntegrityException()
{
super("Attachment content failed integrity verification.");
}
}
@@ -0,0 +1,54 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.attachments;
import java.net.URL;
import java.util.Objects;
/*******************************************************************************
** A download destination may require the application's authenticated transport.
** The URL is sensitive and never attests that bytes were served.
*******************************************************************************/
public record AttachmentDownload(URL url, boolean authenticated)
{
/***************************************************************************
** Only ordinary HTTP destinations are eligible for attachment delivery.
***************************************************************************/
public AttachmentDownload
{
Objects.requireNonNull(url, "url");
if((!"https".equalsIgnoreCase(url.getProtocol()) && !"http".equalsIgnoreCase(url.getProtocol()))
|| url.getUserInfo() != null || url.getRef() != null || url.getHost() == null || url.getHost().isBlank())
{
throw new IllegalArgumentException("Attachment download requires HTTP or HTTPS");
}
}
/***************************************************************************
** Never disclose storage signatures through ordinary object logging.
***************************************************************************/
@Override
public String toString()
{
return ("AttachmentDownload[authenticated=" + authenticated + ", url=<redacted>]");
}
}
@@ -27,6 +27,14 @@ import java.time.Instant;
** Panel DTO. Storage locations and ephemeral signed URLs stay server-side.
*******************************************************************************/
public record AttachmentEntry(Integer id, String tableName, String recordId, String fileName, String contentType,
Long sizeBytes, String sha256, String description, String createdByUserId, Instant createDate, Boolean canDelete)
Long sizeBytes, String sha256, String description, String createdByUserId, Instant createDate, Boolean canDelete, Boolean canEdit)
{
/*******************************************************************************
** Preserve source compatibility for callers using the earlier DTO constructor.
*******************************************************************************/
public AttachmentEntry(Integer id, String tableName, String recordId, String fileName, String contentType,
Long sizeBytes, String sha256, String description, String createdByUserId, Instant createDate, Boolean canDelete)
{
this(id, tableName, recordId, fileName, contentType, sizeBytes, sha256, description, createdByUserId, createDate, canDelete, canDelete);
}
}
@@ -0,0 +1,214 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.attachments;
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** Trusted host extension for attachment lifecycle policy. The default preserves
** ordinary Collaboration behavior. Implementations must be concurrency-safe and
** must not retain request context. Callbacks receive defensive record copies.
** This is not an untrusted scripting API or a new file-storage abstraction.
*******************************************************************************/
public class AttachmentLifecyclePolicy
{
/*******************************************************************************
** Configuration-only capability policy, evaluated during metadata production
** without a session. Dynamic parent visibility owners opt in to pinned counts.
*******************************************************************************/
public boolean requiresPinnedAggregateParent()
{
return (false);
}
/*******************************************************************************
** An installed retention owner must authorize an exact archival reference
** transfer in the caller's real transaction. Ordinary policies deny this.
*******************************************************************************/
public void beforeReferenceTransfer(QRecord attachment, String tableName, String recordId, QBackendTransaction transaction) throws QException
{
throw (new QException("Retained reference transfer requires an archive policy."));
}
/*******************************************************************************
** Persist the reference-transfer outcome in the same transaction. The policy
** must not commit or close the transaction; failure requires caller rollback.
*******************************************************************************/
public void afterReferenceTransfer(QRecord attachment, String tableName, String recordId, QBackendTransaction transaction) throws QException
{
throw (new QException("Retained reference transfer requires an atomic archive audit policy."));
}
/*******************************************************************************
** Called with canonical source, actual digest/size and persisted retention
** decision immediately before the first object write. There is no row ID yet.
** Retained originals require an independently durable intent before storage.
*******************************************************************************/
public void beforeStored(QRecord attachment) throws QException
{
if(Boolean.TRUE.equals(attachment.getValueBoolean("retained")))
{
throw (new QException("Retained storage requires a durable write-intent policy."));
}
}
/*******************************************************************************
** Recheck current adoption authority after external operations, without
** creating another intent or granting access to the original's bytes.
*******************************************************************************/
public void authorizeAdoption(QRecord attachment) throws QException
{
throw (new QException("Existing-original adoption is not configured."));
}
/*******************************************************************************
** Explicit adoption of an existing owner original, not a new upload. The
** policy must authorize the current source and commit its intent before read.
*******************************************************************************/
public void beforeAdoption(QRecord attachment) throws QException
{
throw (new QException("Existing-original adoption requires an audited retention policy."));
}
/*******************************************************************************
** Record the retention transition in the owner's transaction. Registration
** follows separately; failure after commit leaves a retained retryable row.
*******************************************************************************/
public void afterAdoption(QRecord attachment, QBackendTransaction transaction) throws QException
{
throw (new QException("Existing-original adoption requires an atomic retention audit."));
}
/*******************************************************************************
** Called after storage and metadata insertion, before upload returns. Failure
** leaves the original intact for reconciliation; it is not a rollback hook.
** Enforce incomplete registration/quarantine in beforeAccess as appropriate.
*******************************************************************************/
public void afterStored(QRecord attachment) throws QException
{
}
/*******************************************************************************
** Before reading an existing original for registration recovery. This is not
** a write attempt or a user download; retained owners must audit it explicitly.
*******************************************************************************/
public void beforeRecovery(QRecord candidate) throws QException
{
if(Boolean.TRUE.equals(candidate.getValueBoolean("retained")))
{
throw (new QException("Retained recovery requires a durable recovery-attempt policy."));
}
}
/*******************************************************************************
** Called after parent authorization and before either preview or download URL
** signing. Throw to deny. A successful audit here means access was requested,
** not that bytes were downloaded. Mandatory audit must commit independently
** before returning; the caller transaction may still roll back afterwards.
*******************************************************************************/
public void beforeAccess(QRecord attachment, boolean inline) throws QException
{
}
/*******************************************************************************
** Optional authenticated delivery after ordinary parent/lifecycle checks.
** Return null to retain the existing storage URL. No alternate lookup occurs.
*******************************************************************************/
public AttachmentDownload downloadDelivery(QRecord attachment) throws QException
{
return (null);
}
/*******************************************************************************
** Authorize and durably audit a server-side content read, distinct from URL
** issuance. Retained content requires an implementation of this new contract.
*******************************************************************************/
public void beforeContentRead(QRecord attachment) throws QException
{
if(Boolean.TRUE.equals(attachment.getValueBoolean("retained")))
{
throw (new QException("Retained content requires an audited content-read policy."));
}
}
/*******************************************************************************
** Persist the verified server-read result before content leaves the service.
** This does not attest user download, printer spooling or physical output.
*******************************************************************************/
public void afterContentRead(QRecord attachment, Long bytesRead) throws QException
{
if(Boolean.TRUE.equals(attachment.getValueBoolean("retained")))
{
throw (new QException("Retained content requires an audited read-result policy."));
}
}
/*******************************************************************************
** Return true to prohibit deletion and preserve bytes on parent cleanup.
** Called during intake before storage with complete metadata but no row id;
** true is persisted on the attachment and cannot be cleared by this API.
** It is also queried for legacy rows during deletion/read hints. Failure
** denies deletion. Do not depend on parent existence or caller permissions.
*******************************************************************************/
public boolean isRetained(QRecord attachment) throws QException
{
return (false);
}
/*******************************************************************************
** Called after ownership/parent checks and before applying display metadata.
** The second record contains id and proposed fields, not the complete row.
** This records an intent only; it does not attest that a transaction committed.
*******************************************************************************/
public void beforeMetadataChange(QRecord attachment, QRecord changes) throws QException
{
}
/*******************************************************************************
** Validate the actual owner transaction before applying display metadata.
** Required providers must reject unsupported/no-op transactions and incompatible
** audit backends before the first write. This method does not commit anything.
*******************************************************************************/
public void validateMetadataTransaction(QRecord attachment, QBackendTransaction transaction) throws QException
{
if(Boolean.TRUE.equals(attachment.getValueBoolean("retained")))
{
throw (new QException("Retained metadata requires an atomic audit transaction policy."));
}
}
/*******************************************************************************
** Record the successful display metadata change in the supplied transaction
** before its owner commits. Required providers must validate real transaction
** and backend compatibility, insert their event using this transaction and
** throw on failure. Never commit or close the supplied owner transaction.
*******************************************************************************/
public void afterMetadataChange(QRecord attachment, QRecord changes, QBackendTransaction transaction) throws QException
{
if(Boolean.TRUE.equals(attachment.getValueBoolean("retained")))
{
throw (new QException("Retained metadata requires an atomic audit-result policy."));
}
}
}
@@ -0,0 +1,87 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.attachments;
import java.util.Locale;
import java.util.Set;
/*******************************************************************************
** What an uploaded object is allowed to tell a browser. The declared media type
** is client-controlled: it is kept as display metadata, but the object itself is
** stored with an inert type and an inline disposition only when the type is a
** known passive image, PDF, plain-text, audio or video format. Anything else (HTML, SVG,
** XML, script, unknown) is stored as an opaque attachment, so a signed storage
** link can never render or execute uploaded active content inline.
*******************************************************************************/
final class AttachmentMediaPolicy
{
static final String OPAQUE = "application/octet-stream";
private static final Set<String> PASSIVE_INLINE = Set.of(
"image/png", "image/jpeg", "image/gif", "image/webp", "application/pdf", "text/plain",
"audio/mpeg", "audio/mp4", "audio/ogg", "audio/wav", "audio/webm",
"video/mp4", "video/webm", "video/ogg");
/***************************************************************************
** Static policy only.
***************************************************************************/
private AttachmentMediaPolicy()
{
}
/***************************************************************************
** The base media type of a declared value: lowercase, without parameters.
** Anything that is not a plain type/subtype token is treated as unknown.
***************************************************************************/
static String baseType(String declared)
{
if(declared == null)
{
return (OPAQUE);
}
String base = declared.split(";", 2)[0].trim().toLowerCase(Locale.ROOT);
return (base.matches("[a-z0-9][a-z0-9!#$&^_.+-]{0,126}/[a-z0-9][a-z0-9!#$&^_.+-]{0,126}") ? base : OPAQUE);
}
/***************************************************************************
** The media type the stored object will be served with.
***************************************************************************/
static String storedType(String declared)
{
String base = baseType(declared);
return (PASSIVE_INLINE.contains(base) ? base : OPAQUE);
}
/***************************************************************************
** Inline only for passive types; every other object is an attachment.
***************************************************************************/
static String disposition(String declared)
{
return (PASSIVE_INLINE.contains(baseType(declared)) ? "inline" : "attachment");
}
}
@@ -46,15 +46,16 @@ public class AttachmentPreviewCustomizer implements TableCustomizerInterface
@Override
public List<QRecord> postQuery(QueryOrGetInputInterface input, List<QRecord> records) throws QException
{
return (filter(records, input instanceof GetInput, new AttachmentsService()));
return (filter(records, input instanceof GetInput, input.getShouldGenerateDisplayValues(), new AttachmentsService()));
}
/*******************************************************************************
** Keep filtering every row even when the preview generation cap is reached.
*******************************************************************************/
List<QRecord> filter(List<QRecord> records, boolean isGet, AttachmentsService service) throws QException
List<QRecord> filter(List<QRecord> records, boolean isGet, boolean generateUrls, AttachmentsService service) throws QException
{
List<QRecord> visible = new ArrayList<>();
ParentRecordAccess.ResultSetParents parents = new ParentRecordAccess.ResultSetParents();
for(QRecord record : records)
{
try
@@ -63,7 +64,7 @@ public class AttachmentPreviewCustomizer implements TableCustomizerInterface
{
throw (new QNotFoundException("Not found"));
}
ParentRecordAccess.requireVisibleParent(record.getValueString("tableName"), record.getValueString("recordId"));
parents.requireVisible(record.getValueString("tableName"), record.getValueString("recordId"));
}
catch(QNotFoundException expected)
{
@@ -77,15 +78,21 @@ public class AttachmentPreviewCustomizer implements TableCustomizerInterface
QRecord row = new QRecord(record);
row.setValue("previewUrl", null);
row.setValue("downloadUrl", null);
row.setValue("previewDelivery", null);
row.setValue("downloadDelivery", null);
row.withDisplayValue("downloadUrl", null).withDisplayValue("downloadUrl:downloadUrlDynamic", null);
visible.add(row);
if(visible.size() <= 200)
if(generateUrls && visible.size() <= 200)
{
try
{
row.setValue("previewUrl", service.url(row, true).toString());
String downloadUrl = service.url(row, false).toString();
AttachmentDownload preview = service.delivery(row, true);
AttachmentDownload download = service.delivery(row, false);
row.setValue("previewUrl", preview.url().toString());
row.setValue("previewDelivery", preview.authenticated() ? "AUTHENTICATED_FETCH" : "SIGNED_URL");
String downloadUrl = download.url().toString();
row.setValue("downloadUrl", downloadUrl);
row.setValue("downloadDelivery", download.authenticated() ? "AUTHENTICATED_FETCH" : "SIGNED_URL");
row.withDisplayValue("downloadUrl", row.getValueString("fileName"))
.withDisplayValue("downloadUrl:downloadUrlDynamic", downloadUrl);
}
@@ -22,13 +22,17 @@
package com.kingsrook.qbits.collaboration.attachments;
import java.net.URL;
import java.util.Optional;
import java.util.UUID;
import com.kingsrook.qbits.objectstorage.api.DeleteMode;
import com.kingsrook.qbits.objectstorage.api.DeleteRequest;
import com.kingsrook.qbits.objectstorage.api.ObjectStorage;
import com.kingsrook.qbits.objectstorage.api.ObjectStream;
import com.kingsrook.qbits.objectstorage.api.PresignGet;
import com.kingsrook.qbits.objectstorage.api.PutRequest;
import com.kingsrook.qbits.objectstorage.api.Range;
import com.kingsrook.qbits.objectstorage.api.StorageBucket;
import com.kingsrook.qbits.objectstorage.api.StoredObject;
/*******************************************************************************
** Narrow storage boundary. The caller must check collaboration access first.
@@ -45,14 +49,13 @@ public class AttachmentStorage
}
/*******************************************************************************
** The bucket's own upload cap, when object storage exposes it through its api
** package. It does not yet (requested from qbit-object-storage), so this
** returns Long.MAX_VALUE: collaboration enforces its configured per-table cap,
** and object storage still enforces the bucket's cap when the object is put.
** The bucket's own upload cap, from object storage's api package; Long.MAX_VALUE
** when the bucket has no limit. Collaboration also enforces its per-table cap.
*******************************************************************************/
public long maxBytes(String name)
{
return (Long.MAX_VALUE);
Long max = bucket(name).maxUploadBytes();
return (max == null ? Long.MAX_VALUE : max);
}
/*******************************************************************************
@@ -63,6 +66,22 @@ public class AttachmentStorage
bucket(name).put(request);
}
/*******************************************************************************
** Open content for an already-authorized caller. The service owns closing it.
*******************************************************************************/
public ObjectStream open(String name, String key)
{
return (bucket(name).get(key, Range.all()));
}
/*******************************************************************************
** Inspect a deterministic intake key after uncertain conditional creation.
*******************************************************************************/
public Optional<StoredObject> head(String name, String key)
{
return (bucket(name).head(key));
}
/*******************************************************************************
** Prefer soft deletion; allow permanent deletion only where no soft mode exists.
*******************************************************************************/
@@ -74,6 +93,18 @@ public class AttachmentStorage
}
}
/*******************************************************************************
** Remove failed-upload bytes and report whether no earlier PUT can still finish.
** Generic object storage cannot prove remote settlement after a timeout. Keep
** its durable locator for later sweeps. Adapters may return true only with an
** authoritative completion/cancellation guarantee, never from object absence.
*******************************************************************************/
public boolean cleanupUpload(String name, String key)
{
delete(name, key);
return (false);
}
/*******************************************************************************
** Create a temporary link, never a persistent table value.
*******************************************************************************/
@@ -21,6 +21,8 @@
package com.kingsrook.qbits.collaboration.attachments;
import java.io.ByteArrayOutputStream;
import java.io.DataOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
@@ -37,14 +39,17 @@ import java.util.ArrayList;
import java.util.HexFormat;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.UUID;
import com.kingsrook.qbits.collaboration.CollaborationQBitConfig;
import com.kingsrook.qbits.collaboration.CollaborationSecurity;
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
import com.kingsrook.qbits.collaboration.permissions.CollaborationPermissions;
import com.kingsrook.qbits.collaboration.security.ParentRecordAccess;
import com.kingsrook.qbits.objectstorage.api.ObjectStream;
import com.kingsrook.qbits.objectstorage.api.PresignGet;
import com.kingsrook.qbits.objectstorage.api.PutRequest;
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
import com.kingsrook.qqq.backend.core.actions.tables.DeleteAction;
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
@@ -68,6 +73,8 @@ public class AttachmentsService
{
private final CollaborationQBitConfig config;
private final AttachmentStorage storage;
private final AttachmentLifecyclePolicy lifecycle;
private final com.kingsrook.qbits.collaboration.attachments.uploadjournal.UploadJournal uploadJournal;
/*******************************************************************************
** Resolve the installed configuration without a global mutable singleton.
@@ -77,15 +84,46 @@ public class AttachmentsService
this.config = (CollaborationQBitConfig) QContext.getQInstance().getQBits().values().stream()
.filter(qbit -> qbit.getConfig() instanceof CollaborationQBitConfig).findFirst().orElseThrow().getConfig();
this.storage = com.kingsrook.qqq.backend.core.actions.customizers.QCodeLoader.getAdHoc(AttachmentStorage.class, config.getAttachmentStorage());
this.lifecycle = resolveLifecycle(config);
this.uploadJournal = new com.kingsrook.qbits.collaboration.attachments.uploadjournal.JdbcUploadJournal();
}
/*******************************************************************************
** Injectable storage boundary for deterministic service tests.
*******************************************************************************/
public AttachmentsService(CollaborationQBitConfig config, AttachmentStorage storage)
{
this(config, storage, resolveLifecycle(config));
}
/*******************************************************************************
** Package-local deterministic lifecycle seam for service tests.
*******************************************************************************/
AttachmentsService(CollaborationQBitConfig config, AttachmentStorage storage, AttachmentLifecyclePolicy lifecycle)
{
this(config, storage, lifecycle, new com.kingsrook.qbits.collaboration.attachments.uploadjournal.JdbcUploadJournal());
}
/*******************************************************************************
** Explicit test-only package seam; public services always use the durable
** qualified journal. It cannot be replaced through application configuration.
*******************************************************************************/
AttachmentsService(CollaborationQBitConfig config, AttachmentStorage storage, AttachmentLifecyclePolicy lifecycle,
com.kingsrook.qbits.collaboration.attachments.uploadjournal.UploadJournal uploadJournal)
{
this.config = config;
this.storage = storage;
this.lifecycle = java.util.Objects.requireNonNull(lifecycle, "lifecycle");
this.uploadJournal = java.util.Objects.requireNonNull(uploadJournal, "uploadJournal");
}
/*******************************************************************************
** Invalid policy configuration must fail before any storage side effect.
*******************************************************************************/
private static AttachmentLifecyclePolicy resolveLifecycle(CollaborationQBitConfig config)
{
return (java.util.Objects.requireNonNull(com.kingsrook.qqq.backend.core.actions.customizers.QCodeLoader
.getAdHoc(AttachmentLifecyclePolicy.class, config.getAttachmentLifecycle()), "Could not load attachment lifecycle policy."));
}
/*******************************************************************************
@@ -93,6 +131,36 @@ public class AttachmentsService
** The caller owns the input stream. Oversize files never reach object storage.
*******************************************************************************/
public AttachmentEntry upload(String tableName, Serializable recordId, String fileName, String contentType, InputStream stream, String description) throws QException
{
return (uploadInternal(tableName, recordId, fileName, contentType, stream, description, null, null));
}
/*******************************************************************************
** Retryable intake for a server coordinator. Keys and manifests are SHA-256
** identifiers; keys are additionally scoped to actual actor and parent here.
** This path owns persistence and refuses ambient transactions. Replays verify
** supplied bytes and rerun registration; they never create another original.
*******************************************************************************/
public AttachmentEntry upload(String tableName, Serializable recordId, String fileName, String contentType, InputStream stream, String description,
String intakeKey, String manifestSha256) throws QException
{
if(intakeKey == null || !intakeKey.matches("[0-9a-f]{64}") || manifestSha256 == null || !manifestSha256.matches("[0-9a-f]{64}"))
{
throw (new QBadRequestException("Intake key and manifest must be SHA-256 identifiers."));
}
if(QContext.getQBackendTransaction() != null)
{
throw (new QException("Retryable attachment intake cannot join an ambient transaction."));
}
return (uploadInternal(tableName, recordId, fileName, contentType, stream, description, intakeKey, manifestSha256));
}
/*******************************************************************************
** Shared bounded intake; the retryable path preserves uncertain objects for
** reconciliation under a deterministic key and never overwrites them.
*******************************************************************************/
private AttachmentEntry uploadInternal(String tableName, Serializable recordId, String fileName, String contentType, InputStream stream, String description,
String intakeKey, String manifestSha256) throws QException
{
new com.kingsrook.qbits.collaboration.people.PeopleService().touch();
ParentRecordAccess.requireFeature(tableName, recordId, "attach");
@@ -100,13 +168,19 @@ public class AttachmentsService
String id = canonicalId(tableName, recordId);
String name = sanitize(fileName);
String bucket = config.getAttachmentsBucketName();
String key = "attachments/" + segment(tableName) + "/" + segment(id) + "/" + UUID.randomUUID() + "/" + name;
String scopedKey = intakeKey == null ? null : tupleHash("attachment-intake-v1", user, tableName, id, intakeKey);
String key = scopedKey == null ? "attachments/" + segment(tableName) + "/" + segment(id) + "/" + UUID.randomUUID() + "/" + name
: "attachments/intake/" + scopedKey;
if(tableName.length() > 128 || id.length() > 64 || user.length() > 255 || bucket.length() > 255 || key.length() > 1024
|| (contentType != null && contentType.length() > 255))
{
throw (new QBadRequestException("Attachment metadata exceeds the supported field lengths."));
}
long limit = Math.min(config.getDefaultMaxBytes(), storage.maxBytes(bucket));
if(scopedKey != null)
{
limit = Math.min(limit, 20L * 1024 * 1024);
}
Path temporary = null;
try
{
@@ -129,30 +203,69 @@ public class AttachmentsService
}
}
String type = contentType == null || contentType.isBlank() ? "application/octet-stream" : contentType;
try(InputStream upload = Files.newInputStream(temporary))
QRecord row = new QRecord().withValue("tableName", tableName).withValue("recordId", id).withValue("fileName", name)
.withValue("contentType", type).withValue("sizeBytes", size).withValue("sha256", HexFormat.of().formatHex(digest.digest()))
.withValue("bucketName", bucket).withValue("storageKey", key).withValue("description", description)
.withValue("createdByUserId", user).withValue("parentDeleted", false);
row.withValue("retained", lifecycle.isRetained(new QRecord(row)));
if(scopedKey != null)
{
storage.put(bucket, PutRequest.of(key, upload).withContentLength(size).withContentType(type).withContentDisposition("inline"));
String fingerprint = tupleHash("attachment-manifest-v1", manifestSha256, name, type, description, String.valueOf(size), row.getValueString("sha256"));
row.withValue("intakeKey", scopedKey).withValue("intakeFingerprint", fingerprint);
QRecord prior = findIntake(scopedKey);
if(prior != null)
{
if(Boolean.TRUE.equals(prior.getValueBoolean("retained")) && uploadJournal.findByLocation(bucket, key).isPresent())
{
return (journaledUpload(row, temporary));
}
return (replayIntake(prior, fingerprint));
}
if(Boolean.TRUE.equals(row.getValueBoolean("retained")))
{
return (journaledUpload(row, temporary));
}
beforeObjectWrite(row);
createIntakeObject(bucket, key, Files.readAllBytes(temporary), type, fingerprint);
}
else
{
if(Boolean.TRUE.equals(row.getValueBoolean("retained")))
{
return (journaledUpload(row, temporary));
}
beforeObjectWrite(row);
////////////////////////////////////////////////////////////////////////
// ordinary uploads keep the transactional cleanup-queue path (355b06d) //
////////////////////////////////////////////////////////////////////////
return (storeUpload(row, temporary));
}
boolean rowSaved = false;
try
{
QRecord row = new QRecord().withValue("tableName", tableName).withValue("recordId", id).withValue("fileName", name)
.withValue("contentType", type).withValue("sizeBytes", size).withValue("sha256", HexFormat.of().formatHex(digest.digest()))
.withValue("bucketName", bucket).withValue("storageKey", key).withValue("description", description)
.withValue("createdByUserId", user).withValue("parentDeleted", false);
InsertInput input = new InsertInput(CollaborationTables.ATTACHMENT).withTransaction(QContext.getQBackendTransaction()).withRecords(List.of(row));
input.withPermissionBypass("AttachmentsService checked parent and attach permission before upload.");
List<QRecord> records = new InsertAction().execute(input).getRecords();
AttachmentRecords.checkErrors(records);
rowSaved = true;
lifecycle.afterStored(new QRecord(records.getFirst()));
com.kingsrook.qbits.collaboration.activity.ActivityWriter.write(tableName, id, "FILE_ADDED", "File added", CollaborationTables.ATTACHMENT, records.getFirst().getValueString("id"));
return (entry(records.getFirst()));
}
catch(QException | RuntimeException failure)
{
if(scopedKey != null)
{
QRecord prior = findIntake(scopedKey);
if(!rowSaved && prior != null)
{
return (replayIntake(prior, row.getValueString("intakeFingerprint")));
}
throw (failure);
}
try
{
if(!rowSaved)
if(!rowSaved && !Boolean.TRUE.equals(row.getValueBoolean("retained")))
{
storage.delete(bucket, key);
}
@@ -184,23 +297,317 @@ public class AttachmentsService
}
}
/*******************************************************************************
** On transactional locking backends, own a transaction only after spooling
** the bounded input. An existing outer transaction retains the upload lock.
*******************************************************************************/
private AttachmentEntry storeUpload(QRecord row, Path temporary) throws QException, IOException
{
var transaction = QContext.getQBackendTransaction();
if(transaction == null && AttachmentCleanupQueue.supportsUploadOwnership())
{
try(var owned = com.kingsrook.qqq.backend.core.actions.QBackendTransaction.openFor(new InsertInput(CollaborationTables.ATTACHMENT)))
{
QContext.setTransaction(owned);
try
{
AttachmentEntry result = storeUpload(row, temporary);
owned.commit();
return (result);
}
catch(QException | IOException | RuntimeException failure)
{
try
{
owned.rollback();
}
catch(QException rollbackFailure)
{
failure.addSuppressed(rollbackFailure);
}
throw (failure);
}
finally
{
QContext.clearTransaction();
}
}
}
if(transaction != null && !transaction.supportsRollbackCallbacks())
{
throw (new QException("Attachment uploads require a transaction with confirmed rollback support."));
}
String bucket = row.getValueString("bucketName");
String key = row.getValueString("storageKey");
AttachmentCleanupQueue cleanup = new AttachmentCleanupQueue(storage);
QRecord pending = cleanup.beginUpload(bucket, key);
try(InputStream upload = Files.newInputStream(temporary))
{
storage.put(bucket, PutRequest.of(key, upload).withContentLength(row.getValueLong("sizeBytes"))
.withContentType(AttachmentMediaPolicy.storedType(row.getValueString("contentType")))
.withContentDisposition(AttachmentMediaPolicy.disposition(row.getValueString("contentType"))));
}
catch(IOException | RuntimeException failure)
{
if(pending == null)
{
try
{
cleanup.compensateUpload(bucket, key);
}
catch(RuntimeException cleanupFailure)
{
failure.addSuppressed(cleanupFailure);
}
}
throw (failure);
}
if(transaction != null && pending == null)
{
var context = new com.kingsrook.qqq.backend.core.context.CapturedContext(QContext.getQInstance(), QContext.getQSession());
transaction.addAfterRollbackCallback(() -> QContext.withTemporaryContext(context,
() -> cleanup.compensateUpload(bucket, key)));
}
boolean rowSaved = false;
try
{
InsertInput input = new InsertInput(CollaborationTables.ATTACHMENT).withTransaction(QContext.getQBackendTransaction()).withRecords(List.of(row));
input.withPermissionBypass("AttachmentsService checked parent and attach permission before upload.");
List<QRecord> records = new InsertAction().execute(input).getRecords();
AttachmentRecords.checkErrors(records);
rowSaved = true;
cleanup.completeUpload(pending);
lifecycle.afterStored(new QRecord(records.getFirst()));
com.kingsrook.qbits.collaboration.activity.ActivityWriter.write(row.getValueString("tableName"), row.getValueString("recordId"),
"FILE_ADDED", "File added", CollaborationTables.ATTACHMENT, records.getFirst().getValueString("id"));
return (entry(records.getFirst()));
}
catch(QException | RuntimeException failure)
{
if(!rowSaved && pending == null)
{
try
{
cleanup.compensateUpload(bucket, key);
}
catch(RuntimeException cleanupFailure)
{
failure.addSuppressed(cleanupFailure);
}
}
throw (failure);
}
}
/*******************************************************************************
** Retained bytes cannot be written before durable intent or under an ambient
** transaction whose later rollback could lose their metadata association.
*******************************************************************************/
private void beforeObjectWrite(QRecord row) throws QException
{
if(Boolean.TRUE.equals(row.getValueBoolean("retained")) && QContext.getQBackendTransaction() != null)
{
throw (new QException("Retained attachment storage cannot join an ambient transaction."));
}
lifecycle.beforeStored(new QRecord(row));
}
/*******************************************************************************
** Retained originals use qualified bounded conditional creation and durable
** owner recovery; ordinary uploads retain the established streaming behavior.
*******************************************************************************/
private AttachmentEntry journaledUpload(QRecord row, Path temporary) throws QException, IOException
{
if(row.getValueLong("sizeBytes") > JournaledAttachmentUpload.MAX_BYTES)
{
throw (new QBadRequestException("Retained uploads are limited to 20 MiB by the qualified conditional storage path."));
}
var receipt = new JournaledAttachmentUpload(storage, lifecycle, uploadJournal).store(row, Files.readAllBytes(temporary));
QRecord stored = receipt.attachment();
if(!receipt.alreadyComplete())
{
com.kingsrook.qbits.collaboration.activity.ActivityWriter.write(stored.getValueString("tableName"), stored.getValueString("recordId"),
"FILE_ADDED", "File added", CollaborationTables.ATTACHMENT, stored.getValueString("id"));
}
return (entry(stored));
}
/*******************************************************************************
** Native confirmation discovery checks the configured current adoption policy.
** No original bytes are read and no audit success is asserted during review.
*******************************************************************************/
public QRecord requireAdoptable(Integer attachmentId) throws QException
{
QRecord row = requireMutable(attachmentId);
lifecycle.authorizeAdoption(new QRecord(row));
return (row);
}
/*******************************************************************************
** Adopt this exact visible original without uploading, copying or deleting
** storage. Registration retries retain the same owner attachment identity.
*******************************************************************************/
public AttachmentEntry adoptRetainedAttachment(Integer attachmentId) throws QException
{
new ExistingAttachmentAdoption(this, storage, lifecycle).adopt(attachmentId);
return (entry(requireRow(attachmentId)));
}
/*******************************************************************************
** Recover a visible retained upload without accepting bytes, locators or an
** actor from the client. Original-actor and current-parent checks are mandatory.
*******************************************************************************/
public AttachmentEntry recoverRetainedUpload(Integer journalId) throws QException
{
return (entry(new JournaledAttachmentUpload(storage, lifecycle, uploadJournal).recover(journalId)));
}
/*******************************************************************************
** Conditional single-request puts retain the same original across retries.
** Byte arrays deliberately avoid a multipart provider path whose conditional
** completion support is not qualified. The caller enforces a 20 MiB limit.
*******************************************************************************/
private void createIntakeObject(String bucket, String key, byte[] bytes, String type, String fingerprint) throws QException
{
try
{
storage.put(bucket, PutRequest.of(key, bytes).withContentLength((long) bytes.length).withContentType(AttachmentMediaPolicy.storedType(type))
.withContentDisposition("attachment").withIfNoneMatch(true).withMetadata(Map.of("qqq-intake-fingerprint", fingerprint)));
}
catch(RuntimeException uncertain)
{
try
{
if(storage.head(bucket, key).isEmpty())
{
throw (uncertain);
}
try(ObjectStream existing = storage.open(bucket, key))
{
byte[] original = existing.getInputStream().readNBytes(bytes.length + 1);
if(existing.getHeaders() == null || existing.getHeaders().getUserMetadata() == null
|| !fingerprint.equals(existing.getHeaders().getUserMetadata().get("qqq-intake-fingerprint")) || !MessageDigest.isEqual(original, bytes))
{
throw (new QException("An intake request cannot replace an existing original."));
}
}
}
catch(IOException | RuntimeException verificationFailure)
{
throw (new QException("Could not reconcile attachment intake storage.", verificationFailure));
}
}
}
/*******************************************************************************
** Internal lookup is keyed by authenticated actor and canonical parent scope.
*******************************************************************************/
private QRecord findIntake(String scopedKey) throws QException
{
List<QRecord> rows = AttachmentRecords.query(new QQueryFilter().withCriteria("intakeKey", QCriteriaOperator.EQUALS, scopedKey).withLimit(1));
return (rows.isEmpty() ? null : rows.getFirst());
}
/*******************************************************************************
** Reauthorize persisted identity and retry idempotent owner registration.
*******************************************************************************/
private AttachmentEntry replayIntake(QRecord row, String fingerprint) throws QException
{
if(Boolean.TRUE.equals(row.getValueBoolean("parentDeleted")))
{
throw (new QNotFoundException("Not found"));
}
ParentRecordAccess.requireFeature(row.getValueString("tableName"), row.getValueString("recordId"), "attach");
if(!fingerprint.equals(row.getValueString("intakeFingerprint")))
{
throw (new QBadRequestException("Intake request conflicts with its original manifest."));
}
lifecycle.afterStored(new QRecord(row));
return (entry(row));
}
/*******************************************************************************
** Length-prefix all tuple members, including null, so delimiters cannot collide.
*******************************************************************************/
private String tupleHash(String... values) throws QException
{
try
{
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
try(DataOutputStream output = new DataOutputStream(buffer))
{
for(String value : values)
{
byte[] bytes = value == null ? new byte[0] : value.getBytes(StandardCharsets.UTF_8);
output.writeInt(value == null ? -1 : bytes.length);
output.write(bytes);
}
}
return (HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(buffer.toByteArray())));
}
catch(IOException | NoSuchAlgorithmException e)
{
throw (new QException("Could not identify attachment intake.", e));
}
}
/*******************************************************************************
** Newest first, independently of collaboration.view.
*******************************************************************************/
public List<AttachmentEntry> list(String tableName, Serializable recordId) throws QException
{
new com.kingsrook.qbits.collaboration.people.PeopleService().touch();
String id = canonicalId(tableName, recordId);
List<AttachmentEntry> entries = new ArrayList<>();
for(QRecord row : AttachmentRecords.query(new QQueryFilter().withCriteria("tableName", QCriteriaOperator.EQUALS, tableName)
.withCriteria("recordId", QCriteriaOperator.EQUALS, id).withCriteria("parentDeleted", QCriteriaOperator.NOT_EQUALS, true)
.withOrderBy(new QFilterOrderBy("createDate", false)).withOrderBy(new QFilterOrderBy("id", false))))
for(QRecord row : AttachmentRecords.query(listFilter(tableName, recordId)))
{
entries.add(entry(row));
}
return (entries);
}
/*******************************************************************************
** Standard Files-tab pages use one bounded native query, with one same-parent
** lookahead row. Authorization is current on every request; offsets are not
** snapshots across concurrent insertions or deletions.
*******************************************************************************/
public AttachmentPage listPage(String tableName, Serializable recordId, int offset, int limit) throws QException
{
if(offset < 0 || limit < 1 || limit > 100)
{
throw (new QBadRequestException("Offset must be nonnegative and limit must be between 1 and 100."));
}
List<QRecord> rows = AttachmentRecords.query(listFilter(tableName, recordId).withSkip(offset).withLimit(limit + 1));
List<AttachmentEntry> entries = new ArrayList<>();
for(QRecord row : rows)
{
// Preserve the same metadata-policy evaluation for the lookahead row.
AttachmentEntry visible = entry(row);
if(entries.size() < limit)
{
entries.add(visible);
}
}
return (new AttachmentPage(List.copyOf(entries), rows.size() > limit));
}
/*******************************************************************************
** Only authorized display metadata and continuation, never a raw total.
*******************************************************************************/
public record AttachmentPage(List<AttachmentEntry> items, boolean hasMore)
{
}
/*******************************************************************************
** Shared canonical source authority, deletion filter and deterministic order.
*******************************************************************************/
private QQueryFilter listFilter(String tableName, Serializable recordId) throws QException
{
new com.kingsrook.qbits.collaboration.people.PeopleService().touch();
String id = canonicalId(tableName, recordId);
return (new QQueryFilter().withCriteria("tableName", QCriteriaOperator.EQUALS, tableName)
.withCriteria("recordId", QCriteriaOperator.EQUALS, id).withCriteria("parentDeleted", QCriteriaOperator.NOT_EQUALS, true)
.withOrderBy(new QFilterOrderBy("createDate", false)).withOrderBy(new QFilterOrderBy("id", false)));
}
/*******************************************************************************
** Re-check the attachment's actual parent before signing a five-minute GET.
** Inline uses the stored inline header; RC.2 only overrides attachment names.
@@ -211,17 +618,106 @@ public class AttachmentsService
return (url(requireRow(attachmentId), inline));
}
/*******************************************************************************
** Standard download clients preserve the selected delivery authentication mode.
*******************************************************************************/
public AttachmentDownload downloadDelivery(Serializable attachmentId) throws QException
{
new com.kingsrook.qbits.collaboration.people.PeopleService().touch();
return (downloadDelivery(attachmentId, false));
}
/***************************************************************************
** Preserve the explicit transport mode for preview and download consumers.
***************************************************************************/
public AttachmentDownload downloadDelivery(Serializable attachmentId, boolean inline) throws QException
{
return (delivery(requireRow(attachmentId), inline));
}
/*******************************************************************************
** Read a bounded exact original for trusted integrations such as printing.
** Authorize the actual parent, enforce lifecycle policy and verify persisted
** length and SHA-256 before releasing any bytes. No signed URL is involved.
*******************************************************************************/
public AttachmentContent readContent(Serializable attachmentId, Integer maximumBytes) throws QException
{
QRecord row = requireRow(attachmentId);
requireUser();
requireRetainedPolicy(row);
Long size = row.getValueLong("sizeBytes");
String sha256 = row.getValueString("sha256");
if(maximumBytes == null || maximumBytes < 1 || maximumBytes > 20 * 1024 * 1024)
{
throw (new QBadRequestException("Content read limit must be between 1 byte and 20 MiB."));
}
if(size == null || size < 0 || size > maximumBytes || sha256 == null || !sha256.matches("[0-9a-f]{64}"))
{
throw (new QBadRequestException("Attachment exceeds the content read limit or lacks integrity metadata."));
}
lifecycle.beforeContentRead(new QRecord(row));
byte[] bytes;
try(ObjectStream content = storage.open(row.getValueString("bucketName"), row.getValueString("storageKey")))
{
if(content.getContentLength() != null && !size.equals(content.getContentLength()))
{
throw (new AttachmentContentIntegrityException());
}
bytes = content.getInputStream().readNBytes(size.intValue() + 1);
String actualHash = HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(bytes));
if(bytes.length != size.longValue() || !sha256.equals(actualHash))
{
throw (new AttachmentContentIntegrityException());
}
}
catch(IOException | NoSuchAlgorithmException | RuntimeException e)
{
throw (new QException("Could not read attachment content.", e));
}
lifecycle.afterContentRead(new QRecord(row), size);
return (new AttachmentContent(row.getValueInteger("id"), row.getValueString("fileName"), row.getValueString("contentType"), sha256, bytes));
}
/*******************************************************************************
** Sign an already-authorized row, also used by the preview customizer.
*******************************************************************************/
URL url(QRecord row, boolean inline)
URL url(QRecord row, boolean inline) throws QException
{
return (delivery(row, inline).url());
}
/*******************************************************************************
** Resolve only the configured owner policy, after exact row authorization.
*******************************************************************************/
AttachmentDownload delivery(QRecord row, boolean inline) throws QException
{
requireRetainedPolicy(row);
try
{
lifecycle.beforeAccess(new QRecord(row), inline);
}
catch(RuntimeException e)
{
// Preview generation may ignore storage failures, never policy failures.
throw (new QException("Attachment access policy failed.", e));
}
AttachmentDownload configured = lifecycle.downloadDelivery(new QRecord(row));
if(configured != null)
{
return (configured);
}
PresignGet request = PresignGet.of(row.getValueString("storageKey")).ttl(Duration.ofMinutes(5));
if(!inline)
///////////////////////////////////////////////////////////////////////////////////////////
// Objects stored before AttachmentMediaPolicy keep their original headers (possibly an //
// active type served inline). The link therefore forces an attachment disposition for //
// every type that is not a known passive format, whatever the caller asked for; passive //
// previews stay inline. //
///////////////////////////////////////////////////////////////////////////////////////////
if(!inline || !"inline".equals(AttachmentMediaPolicy.disposition(row.getValueString("contentType"))))
{
request.downloadAs(row.getValueString("fileName"));
}
return (storage.presign(row.getValueString("bucketName"), request));
return (new AttachmentDownload(storage.presign(row.getValueString("bucketName"), request), false));
}
/*******************************************************************************
@@ -233,7 +729,7 @@ public class AttachmentsService
List<QRecord> rows = new ArrayList<>();
for(Serializable id : attachmentIds.stream().distinct().toList())
{
rows.add(requireMutable(id));
rows.add(requireDeletable(id));
}
for(QRecord row : rows)
{
@@ -250,6 +746,7 @@ public class AttachmentsService
{
new com.kingsrook.qbits.collaboration.people.PeopleService().touch();
QRecord row = requireMutable(attachmentId);
requireRetainedPolicy(row);
QRecord change = new QRecord().withValue("id", row.getValue("id"));
if(fileName != null)
{
@@ -259,10 +756,41 @@ public class AttachmentsService
{
change.setValue("description", description);
}
lifecycle.beforeMetadataChange(new QRecord(row), new QRecord(change));
UpdateInput input = new UpdateInput(CollaborationTables.ATTACHMENT).withTransaction(QContext.getQBackendTransaction()).withRecords(List.of(change));
input.setTableMetaData(AttachmentRecords.storageTable());
input.withPermissionBypass("AttachmentsService checked the actual parent and author/admin before metadata update.");
AttachmentRecords.checkErrors(new UpdateAction().execute(input).getRecords());
QBackendTransaction callerTransaction = QContext.getQBackendTransaction();
try(QBackendTransaction ownedTransaction = callerTransaction == null ? QBackendTransaction.openFor(input) : null)
{
QBackendTransaction transaction = ownedTransaction == null ? callerTransaction : ownedTransaction;
input.setTransaction(transaction);
try
{
lifecycle.validateMetadataTransaction(new QRecord(row), transaction);
AttachmentRecords.checkErrors(new UpdateAction().execute(input).getRecords());
lifecycle.afterMetadataChange(new QRecord(row), new QRecord(change), transaction);
if(ownedTransaction != null)
{
ownedTransaction.commit();
}
}
catch(QException | RuntimeException failure)
{
if(ownedTransaction != null)
{
try
{
ownedTransaction.rollback();
}
catch(QException rollbackFailure)
{
failure.addSuppressed(rollbackFailure);
}
}
throw (failure);
}
}
return (entry(requireRow(attachmentId)));
}
@@ -274,6 +802,46 @@ public class AttachmentsService
return (rename(attachmentId, null, description));
}
/*******************************************************************************
** Transfer a retained attachment's reference to an authorized archive parent.
** The caller owns the transaction and must roll it back on any failure. Bytes,
** hashes, author, storage coordinates and the retained flag never change.
*******************************************************************************/
public void transferRetainedReference(Serializable attachmentId, String tableName, Serializable recordId, QBackendTransaction transaction) throws QException
{
requireUser();
if(transaction == null || (QContext.getQBackendTransaction() != null && QContext.getQBackendTransaction() != transaction))
{
throw (new QException("Archive transfer requires the caller's explicit transaction."));
}
QRecord row = requireRow(attachmentId);
if(!Boolean.TRUE.equals(row.getValueBoolean("retained")) || tableName == null || recordId == null)
{
throw (new QException("Only a retained attachment can be transferred to an archive."));
}
var candidate = uploadJournal.findByLocation(row.getValueString("bucketName"), row.getValueString("storageKey"));
if(candidate.isPresent() && candidate.get().state() != com.kingsrook.qbits.collaboration.attachments.uploadjournal.UploadJournalState.COMPLETE)
{
throw (new QException("Retained upload registration must complete before archival reference transfer."));
}
QRecord target = com.kingsrook.qqq.backend.core.actions.collaboration.RecordReferenceSecurityHelper.visibleParent(tableName, recordId, transaction);
if(target == null)
{
throw (new QNotFoundException("Not found"));
}
String canonical = target.getValueString(QContext.getQInstance().getTable(tableName).getPrimaryKeyField());
lifecycle.beforeReferenceTransfer(new QRecord(row), tableName, canonical, transaction);
QRecord change = new QRecord().withValue("id", row.getValue("id")).withValue("tableName", tableName).withValue("recordId", canonical);
com.kingsrook.qqq.backend.core.actions.collaboration.RecordReferenceSecurityHelper.copyMirroredLocks(
QContext.getQInstance().getTable(tableName), target, QContext.getQInstance().getTable(CollaborationTables.ATTACHMENT), change);
UpdateInput input = new UpdateInput(CollaborationTables.ATTACHMENT).withTransaction(transaction)
.withRecords(List.of(change));
input.setTableMetaData(com.kingsrook.qbits.collaboration.CollaborationRecords.cleanupMetadata(CollaborationTables.ATTACHMENT));
input.withPermissionBypass("Installed retention owner authorized this exact archival reference transfer and transaction.");
AttachmentRecords.checkErrors(new UpdateAction().execute(input).getRecords());
lifecycle.afterReferenceTransfer(new QRecord(row), tableName, canonical, transaction);
}
/*******************************************************************************
** Process Review uses persisted identity rather than trusting client rows.
*******************************************************************************/
@@ -282,7 +850,7 @@ public class AttachmentsService
new com.kingsrook.qbits.collaboration.people.PeopleService().touch();
QRecord row = requireRow(attachmentId);
requireUser();
if(!canDelete(row))
if(!canModify(row))
{
throw (new QPermissionDeniedException("Only the author or collaboration.admin can change this attachment."));
}
@@ -291,7 +859,17 @@ public class AttachmentsService
}
/*******************************************************************************
** Delete storage first so a provider failure leaves a retryable row.
** Deletion reviews must include retention as well as parent and ownership.
*******************************************************************************/
public QRecord requireDeletable(Serializable attachmentId) throws QException
{
QRecord row = requireMutable(attachmentId);
requireNotRetained(row);
return (row);
}
/*******************************************************************************
** Atomically remove metadata and retain a durable storage cleanup request.
*******************************************************************************/
void deleteStoredRow(QRecord row) throws QException
{
@@ -303,11 +881,20 @@ public class AttachmentsService
*******************************************************************************/
void deleteStoredRow(QRecord row, boolean cleanup) throws QException
{
storage.delete(row.getValueString("bucketName"), row.getValueString("storageKey"));
DeleteInput input = new DeleteInput(CollaborationTables.ATTACHMENT).withTransaction(QContext.getQBackendTransaction()).withPrimaryKeys(List.of(row.getValue("id")));
input.setTableMetaData(cleanup ? com.kingsrook.qbits.collaboration.CollaborationRecords.cleanupMetadata(CollaborationTables.ATTACHMENT) : AttachmentRecords.storageTable());
input.withPermissionBypass("Attachment service authorized deletion, or trusted parent cleanup selected this row.");
AttachmentRecords.checkErrors(new DeleteAction().execute(input).getRecordsWithErrors());
if(isRetained(row))
{
if(!cleanup)
{
throw (new QPermissionDeniedException("This attachment is retained and cannot be deleted."));
}
UpdateInput mark = new UpdateInput(CollaborationTables.ATTACHMENT).withTransaction(QContext.getQBackendTransaction())
.withRecords(List.of(new QRecord().withValue("id", row.getValue("id")).withValue("parentDeleted", true)));
mark.setTableMetaData(com.kingsrook.qbits.collaboration.CollaborationRecords.cleanupMetadata(CollaborationTables.ATTACHMENT));
mark.withPermissionBypass("Trusted parent cleanup retains the original and marks its parent deleted.");
AttachmentRecords.checkErrors(new UpdateAction().execute(mark).getRecords());
return;
}
new AttachmentCleanupQueue(storage).delete(row, cleanup);
}
/*******************************************************************************
@@ -336,7 +923,7 @@ public class AttachmentsService
/*******************************************************************************
** Ownership is layered on parent visibility; admins still need that visibility.
*******************************************************************************/
private boolean canDelete(QRecord row)
private boolean canModify(QRecord row)
{
String user = CollaborationSecurity.userId(QContext.getQSession());
return (user != null && (isAdmin() || user.equals(row.getValueString("createdByUserId"))));
@@ -366,11 +953,53 @@ public class AttachmentsService
/*******************************************************************************
** Return display data, without internal storage coordinates.
*******************************************************************************/
private AttachmentEntry entry(QRecord row)
private AttachmentEntry entry(QRecord row) throws QException
{
boolean canEdit = canModify(row) && !(Boolean.TRUE.equals(row.getValueBoolean("retained")) && lifecycle.getClass().equals(AttachmentLifecyclePolicy.class));
if(canEdit)
{
try
{
ParentRecordAccess.requireFeature(row.getValueString("tableName"), row.getValueString("recordId"), "attach");
}
catch(QPermissionDeniedException e)
{
canEdit = false;
}
}
return (new AttachmentEntry(row.getValueInteger("id"), row.getValueString("tableName"), row.getValueString("recordId"),
row.getValueString("fileName"), row.getValueString("contentType"), row.getValueLong("sizeBytes"), row.getValueString("sha256"),
row.getValueString("description"), row.getValueString("createdByUserId"), row.getValueInstant("createDate"), canDelete(row)));
row.getValueString("description"), row.getValueString("createdByUserId"), row.getValueInstant("createDate"), canEdit && !isRetained(row), canEdit));
}
/*******************************************************************************
** Preflight every selected row before deleting any storage object.
*******************************************************************************/
private void requireNotRetained(QRecord row) throws QException
{
if(isRetained(row))
{
throw (new QPermissionDeniedException("This attachment is retained and cannot be deleted."));
}
}
/*******************************************************************************
** Persisted retention survives removal or reconfiguration of the provider.
*******************************************************************************/
private boolean isRetained(QRecord row) throws QException
{
return (Boolean.TRUE.equals(row.getValueBoolean("retained")) || lifecycle.isRetained(new QRecord(row)));
}
/*******************************************************************************
** Removing the configured provider must not permit unaudited retained access.
*******************************************************************************/
private void requireRetainedPolicy(QRecord row) throws QException
{
if(Boolean.TRUE.equals(row.getValueBoolean("retained")) && lifecycle.getClass().equals(AttachmentLifecyclePolicy.class))
{
throw (new QException("Retained attachment access requires its lifecycle policy."));
}
}
/*******************************************************************************
@@ -92,7 +92,7 @@ public final class DeleteAttachmentsProcess
*******************************************************************************/
public static class Review extends AbstractTransformStep implements ProcessSummaryProviderInterface
{
private final AttachmentsService service;
private AttachmentsService service;
private final ProcessSummaryLine selected = line(Status.OK, "will be deleted", "deleted");
private final ProcessSummaryLine skipped = line(Status.WARNING, "cannot be deleted (skipped)", "skipped");
@@ -101,7 +101,11 @@ public final class DeleteAttachmentsProcess
*******************************************************************************/
public Review()
{
this(new AttachmentsService());
//////////////////////////////////////////////////////////////////////////
// no QContext here: QQQ constructs steps during context-free instance //
// validation, so the service is created on first use (in a run) //
//////////////////////////////////////////////////////////////////////////
this(null);
}
/*******************************************************************************
@@ -112,6 +116,20 @@ public final class DeleteAttachmentsProcess
this.service = service;
}
/*******************************************************************************
** The attachments service, created on first use inside a run (QContext exists).
*******************************************************************************/
private AttachmentsService service()
{
if(service == null)
{
service = new AttachmentsService();
}
return (service);
}
/*******************************************************************************
** Dry run only; no object or row mutations.
*******************************************************************************/
@@ -122,7 +140,7 @@ public final class DeleteAttachmentsProcess
{
try
{
QRecord checked = service.requireMutable(row.getValue("id"));
QRecord checked = service().requireDeletable(row.getValue("id"));
output.getRecords().add(checked);
selected.incrementCountAndAddPrimaryKey(checked.getValue("id"));
}
@@ -151,7 +169,7 @@ public final class DeleteAttachmentsProcess
*******************************************************************************/
public static class Delete extends AbstractLoadStep implements ProcessSummaryProviderInterface
{
private final AttachmentsService service;
private AttachmentsService service;
private final ProcessSummaryLine deleted = line(Status.OK, "will be deleted", "deleted");
private final ProcessSummaryLine failed = line(Status.ERROR, "could not be deleted", "could not be deleted (retry required)");
@@ -160,7 +178,11 @@ public final class DeleteAttachmentsProcess
*******************************************************************************/
public Delete()
{
this(new AttachmentsService());
//////////////////////////////////////////////////////////////////////////
// no QContext here: QQQ constructs steps during context-free instance //
// validation, so the service is created on first use (in a run) //
//////////////////////////////////////////////////////////////////////////
this(null);
}
/*******************************************************************************
@@ -171,6 +193,20 @@ public final class DeleteAttachmentsProcess
this.service = service;
}
/*******************************************************************************
** The attachments service, created on first use inside a run (QContext exists).
*******************************************************************************/
private AttachmentsService service()
{
if(service == null)
{
service = new AttachmentsService();
}
return (service);
}
/*******************************************************************************
** Count success only after both storage and row deletion succeed.
*******************************************************************************/
@@ -181,7 +217,7 @@ public final class DeleteAttachmentsProcess
{
try
{
service.delete(List.of(row.getValue("id")));
service().delete(List.of(row.getValue("id")));
deleted.incrementCountAndAddPrimaryKey(row.getValue("id"));
}
catch(QException | RuntimeException expected)
@@ -0,0 +1,291 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.attachments;
import java.io.IOException;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.HexFormat;
import java.util.List;
import java.util.Objects;
import com.kingsrook.qbits.collaboration.CollaborationSecurity;
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
import com.kingsrook.qbits.objectstorage.api.ObjectStream;
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.exceptions.QNotFoundException;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.module.rdbms.actions.RDBMSTransaction;
import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSTableBackendDetails;
/*******************************************************************************
** Owner-only adoption of existing bytes. A durable intent precedes verification;
** retention and its audit commit together, then idempotent registration follows.
*******************************************************************************/
final class ExistingAttachmentAdoption
{
private static final long MAX_BYTES = 20L * 1024 * 1024;
private final AttachmentsService service;
private final AttachmentStorage storage;
private final AttachmentLifecyclePolicy lifecycle;
/***************************************************************************
** Dependencies are the actual configured owner boundaries.
***************************************************************************/
ExistingAttachmentAdoption(AttachmentsService service, AttachmentStorage storage, AttachmentLifecyclePolicy lifecycle)
{
this.service = service;
this.storage = storage;
this.lifecycle = lifecycle;
}
/***************************************************************************
** No external transaction, caller locator, bytes, receipt or force option.
***************************************************************************/
void adopt(Integer id) throws QException
{
var instance = QContext.getQInstance();
var session = QContext.getQSession();
String actor = CollaborationSecurity.userId(session);
Check check = () ->
{
if(QContext.getQBackendTransaction() != null || QContext.getQInstance() != instance || QContext.getQSession() != session
|| actor == null || !actor.equals(CollaborationSecurity.userId(session)))
{
throw (new QException("Adoption requires the unchanged initiating context and its own transaction."));
}
};
check.run();
QRecord original = service.requireMutable(id);
if(original.getValueString("intakeKey") != null)
{
throw (new QException("Coordinated intake must recover through its original request."));
}
requireIntegrity(original);
lifecycle.authorizeAdoption(new QRecord(original));
check.run();
lifecycle.beforeAdoption(new QRecord(original));
check.run();
same(original, service.requireMutable(id));
QRecord retained = new QRecord(original).withValue("retained", true);
UpdateInput input = new UpdateInput(CollaborationTables.ATTACHMENT)
.withRecords(List.of(new QRecord().withValue("id", id).withValue("retained", true)));
input.setTableMetaData(AttachmentRecords.storageTable());
input.withPermissionBypass("Attachment owner rechecked current parent, author/admin and audited retention policy.");
try(QBackendTransaction transaction = QBackendTransaction.openFor(input))
{
try
{
lifecycle.validateMetadataTransaction(retained, transaction);
check.run();
lock(transaction, id);
same(original, read(transaction, id));
verify(original);
check.run();
same(original, service.requireMutable(id));
lifecycle.authorizeAdoption(new QRecord(original));
check.run();
if(!Boolean.TRUE.equals(original.getValueBoolean("retained")))
{
input.setTransaction(transaction);
AttachmentRecords.checkErrors(new UpdateAction().execute(input).getRecords());
lifecycle.afterAdoption(retained, transaction);
}
check.run();
service.requireMutable(id);
lifecycle.authorizeAdoption(new QRecord(retained));
check.run();
transaction.commit();
}
catch(QException | RuntimeException | Error failure)
{
try
{
transaction.rollback();
}
catch(QException | RuntimeException cleanup)
{
if(cleanup != failure)
{
failure.addSuppressed(cleanup);
}
}
throw failure;
}
}
check.run();
same(retained, service.requireMutable(id));
lifecycle.authorizeAdoption(new QRecord(retained));
check.run();
lifecycle.afterStored(new QRecord(retained));
check.run();
same(retained, service.requireMutable(id));
lifecycle.authorizeAdoption(new QRecord(retained));
check.run();
}
/***************************************************************************
** Native metadata supplies identifiers; no caller text becomes SQL.
***************************************************************************/
private void lock(QBackendTransaction transaction, Integer id) throws QException
{
if(!(transaction instanceof RDBMSTransaction jdbc))
{
throw (new QException("Existing-original adoption requires a real relational transaction."));
}
var table = AttachmentRecords.storageTable();
String tableName = table.getBackendDetails() instanceof RDBMSTableBackendDetails details ? details.getTableName() : table.getName();
String key = table.getField(table.getPrimaryKeyField()).getBackendName();
key = key == null ? table.getPrimaryKeyField() : key;
if(tableName == null || !tableName.matches("[A-Za-z_][A-Za-z0-9_]*") || !key.matches("[A-Za-z_][A-Za-z0-9_]*"))
{
throw (new QException("Unsupported attachment storage mapping."));
}
try
{
String quote = jdbc.getConnection().getMetaData().getIdentifierQuoteString().trim();
if(!List.of("\"", "`").contains(quote))
{
throw (new QException("Unsupported attachment identifier quoting."));
}
try(var statement = jdbc.getConnection().prepareStatement("SELECT " + quote + key + quote + " FROM " + quote + tableName + quote
+ " WHERE " + quote + key + quote + " = ? FOR UPDATE"))
{
statement.setQueryTimeout(5);
statement.setObject(1, id);
try(var result = statement.executeQuery())
{
if(!result.next())
{
throw (new QNotFoundException("Not found"));
}
}
}
}
catch(java.sql.SQLException failure)
{
throw (new QException("Could not lock the original attachment.", failure));
}
}
/***************************************************************************
** Read the locked original in the same transaction without URL customizers.
***************************************************************************/
private QRecord read(QBackendTransaction transaction, Integer id) throws QException
{
QueryInput input = new QueryInput(CollaborationTables.ATTACHMENT).withTransaction(transaction).withShouldFetchHeavyFields(true)
.withFilter(new QQueryFilter().withCriteria("id", QCriteriaOperator.EQUALS, id).withLimit(1));
input.setTableMetaData(AttachmentRecords.storageTable());
input.withPermissionBypass("Owner adoption already authorized the exact parent and attachment.");
var rows = new QueryAction().execute(input).getRecords();
if(rows.size() != 1)
{
throw (new QNotFoundException("Not found"));
}
return (rows.getFirst());
}
/***************************************************************************
** Compare persisted metadata, allowing only the native modification timestamp.
** The caller supplies the exact expected retention state at each boundary.
***************************************************************************/
private void same(QRecord expected, QRecord actual) throws QException
{
for(var field : AttachmentRecords.storageTable().getFields().keySet())
{
if(!"modifyDate".equals(field) && !Objects.equals(expected.getValue(field), actual.getValue(field)))
{
throw (new QException("Attachment metadata changed during adoption."));
}
}
}
/***************************************************************************
** Reject ambiguous or oversized originals before opening storage.
***************************************************************************/
private void requireIntegrity(QRecord row) throws QException
{
Long size = row.getValueLong("sizeBytes");
if(size == null || size < 0 || size > MAX_BYTES || row.getValueString("sha256") == null
|| !row.getValueString("sha256").matches("[0-9a-f]{64}"))
{
throw (new QException("Original integrity metadata is missing or outside the adoption bound."));
}
}
/***************************************************************************
** Stream and hash the existing original without releasing or writing bytes.
***************************************************************************/
private void verify(QRecord row) throws QException
{
try(ObjectStream object = storage.open(row.getValueString("bucketName"), row.getValueString("storageKey")))
{
long expected = row.getValueLong("sizeBytes");
if(object.getContentLength() != null && object.getContentLength() != expected)
{
throw (new AttachmentContentIntegrityException());
}
MessageDigest digest = MessageDigest.getInstance("SHA-256");
byte[] buffer = new byte[8192];
long total = 0;
int count;
while((count = object.getInputStream().read(buffer, 0, (int) Math.min(buffer.length, expected - total + 1))) != -1)
{
total += count;
if(total > expected)
{
throw (new AttachmentContentIntegrityException());
}
digest.update(buffer, 0, count);
}
if(total != expected || !row.getValueString("sha256").equals(HexFormat.of().formatHex(digest.digest())))
{
throw (new AttachmentContentIntegrityException());
}
}
catch(IOException | NoSuchAlgorithmException | RuntimeException failure)
{
throw (new QException("Could not verify the existing original.", failure));
}
}
/***************************************************************************
** Checked context guard without installing a different session or authority.
***************************************************************************/
@FunctionalInterface
private interface Check
{
/***********************************************************************
** Recheck the initiating context after external callbacks.
***********************************************************************/
void run() throws QException;
}
}
@@ -0,0 +1,305 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.attachments;
import java.io.IOException;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.time.Duration;
import java.util.HexFormat;
import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.collaboration.PersonalRecords;
import com.kingsrook.qbits.collaboration.attachments.uploadjournal.UploadCandidate;
import com.kingsrook.qbits.collaboration.attachments.uploadjournal.UploadEntry;
import com.kingsrook.qbits.collaboration.attachments.uploadjournal.UploadJournal;
import com.kingsrook.qbits.collaboration.attachments.uploadjournal.UploadJournalState;
import com.kingsrook.qbits.collaboration.attachments.uploadjournal.UploadReservation;
import com.kingsrook.qbits.collaboration.attachments.uploadjournal.UploadStorageState;
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
import com.kingsrook.qbits.collaboration.security.ParentRecordAccess;
import com.kingsrook.qbits.objectstorage.api.ObjectStream;
import com.kingsrook.qbits.objectstorage.api.PutRequest;
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.exceptions.QNotFoundException;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** Retained-only coordination of one immutable original and its durable owner
** manifest. All database ownership stays in Collaboration; lifecycle registration
** may commit in its own module after the attachment checkpoint is durable.
*******************************************************************************/
final class JournaledAttachmentUpload
{
/*******************************************************************************
** The caller can avoid repeating non-mandatory timeline activity on replay.
*******************************************************************************/
record Receipt(QRecord attachment, boolean alreadyComplete) {}
static final long MAX_BYTES = 20L * 1024 * 1024;
private static final Duration LEASE = Duration.ofMinutes(5);
private final AttachmentStorage storage;
private final AttachmentLifecyclePolicy lifecycle;
private final UploadJournal journal;
/*******************************************************************************
** Explicit owner dependencies; the journal never receives file bytes.
*******************************************************************************/
JournaledAttachmentUpload(AttachmentStorage storage, AttachmentLifecyclePolicy lifecycle, UploadJournal journal)
{
this.storage = storage;
this.lifecycle = lifecycle;
this.journal = journal;
}
/*******************************************************************************
** Persist intent before any PUT. Conditional creation never replaces an
** original. A late or expired PUT cannot authorize metadata registration.
*******************************************************************************/
Receipt store(QRecord row, byte[] bytes) throws QException
{
requireIndependent();
UploadCandidate candidate = candidate(row);
authorize(candidate);
if(bytes.length > MAX_BYTES || bytes.length != candidate.sizeBytes() || !hash(bytes).equals(candidate.sha256()))
{
throw (new QException("Retained original exceeds the qualified bound or changed before storage."));
}
UploadEntry entry = journal.prepare(candidate);
if(entry.state() == UploadJournalState.COMPLETE)
{
QRecord existing = stored(entry);
lifecycle.afterStored(new QRecord(existing));
return (new Receipt(existing, true));
}
UploadReservation reservation = journal.reserve(entry.id(), LEASE);
try
{
lifecycle.beforeStored(new QRecord(row));
reservation = journal.renew(reservation, LEASE);
if(entry.storageState() != UploadStorageState.VERIFIED)
{
journal.recordStorage(reservation, UploadStorageState.UNKNOWN);
try
{
storage.put(candidate.bucketName(), PutRequest.of(candidate.storageKey(), bytes).withContentLength(candidate.sizeBytes())
.withContentType(candidate.contentType()).withContentDisposition("attachment").withIfNoneMatch(true)
.withMetadata(Map.of("qqq-upload-manifest", candidate.manifestHash())));
}
catch(RuntimeException uncertain)
{
try
{
verify(candidate);
}
catch(QException verification)
{
verification.addSuppressed(uncertain);
throw (verification);
}
}
}
else
{
verify(candidate);
}
reservation = journal.renew(reservation, LEASE);
journal.recordStorage(reservation, UploadStorageState.VERIFIED);
return (new Receipt(register(reservation), false));
}
finally
{
journal.release(reservation);
}
}
/*******************************************************************************
** Standard recovery reads an already stored original and never writes bytes.
** Coordinated intake must retry through its original coordinator and scope.
*******************************************************************************/
QRecord recover(Integer journalId) throws QException
{
requireIndependent();
UploadEntry entry = journal.find(journalId).orElseThrow(() -> new QNotFoundException("Not found"));
if(!PersonalRecords.currentUserId().equals(entry.candidate().actorId()))
{
throw (new QNotFoundException("Not found"));
}
if(entry.state() == UploadJournalState.COMPLETE)
{
QRecord row = stored(entry);
ParentRecordAccess.requireFeature(row.getValueString("tableName"), row.getValueString("recordId"), "attach");
return (row);
}
authorize(entry.candidate());
if(entry.candidate().intakeKey() != null)
{
throw (new QException("Retry this upload through its original intake request to preserve its registration context."));
}
UploadReservation reservation = journal.reserve(entry.id(), LEASE);
try
{
lifecycle.beforeRecovery(row(entry.candidate()));
verify(entry.candidate());
reservation = journal.renew(reservation, LEASE);
journal.recordStorage(reservation, UploadStorageState.VERIFIED);
return (register(reservation));
}
finally
{
journal.release(reservation);
}
}
/*******************************************************************************
** Attachment and journal checkpoint commit together. Replays use the same
** immutable attachment id and retry the mandatory idempotent lifecycle hook.
*******************************************************************************/
private QRecord register(UploadReservation reservation) throws QException
{
authorize(reservation.candidate());
UploadEntry entry = journal.register(reservation, (transaction, candidate, journalId) ->
{
InsertInput input = new InsertInput(CollaborationTables.ATTACHMENT).withTransaction(transaction).withRecords(List.of(row(candidate)));
input.withPermissionBypass("Retained owner journal reauthorized the original actor and current parent attach capability.");
List<QRecord> records = new InsertAction().execute(input).getRecords();
AttachmentRecords.checkErrors(records);
if(records.size() != 1)
{
throw (new QException("Retained registration did not create exactly one owner row."));
}
return (records.getFirst().getValueInteger("id"));
});
QRecord stored = stored(entry);
lifecycle.afterStored(new QRecord(stored));
reservation = journal.renew(reservation, LEASE);
journal.complete(reservation);
return (stored);
}
/*******************************************************************************
** Exact original bytes must be readable; missing or ambiguous storage never
** becomes a cancellation or permission to delete the source record.
*******************************************************************************/
private void verify(UploadCandidate candidate) throws QException
{
if(candidate.sizeBytes() > MAX_BYTES)
{
throw (new QException("Retained recovery exceeds the qualified content bound."));
}
try(ObjectStream object = storage.open(candidate.bucketName(), candidate.storageKey()))
{
byte[] bytes = object.getInputStream().readNBytes(Math.toIntExact(candidate.sizeBytes()) + 1);
if(bytes.length != candidate.sizeBytes() || !hash(bytes).equals(candidate.sha256()))
{
throw (new QException("Stored original does not match its immutable journal manifest."));
}
}
catch(IOException | RuntimeException failure)
{
throw (new QException("Original storage remains unresolved; no absence or deletion is inferred.", failure));
}
}
/*******************************************************************************
** Resolve the checkpoint without exposing storage coordinates to clients.
*******************************************************************************/
private QRecord stored(UploadEntry entry) throws QException
{
List<QRecord> rows = AttachmentRecords.query(new QQueryFilter().withCriteria("id", QCriteriaOperator.EQUALS, entry.attachmentId()).withLimit(1));
if(rows.size() != 1 || !entry.candidate().sha256().equals(rows.getFirst().getValueString("sha256"))
|| !entry.candidate().sizeBytes().equals(rows.getFirst().getValueLong("sizeBytes"))
|| !entry.candidate().actorId().equals(rows.getFirst().getValueString("createdByUserId")))
{
throw (new QException("Retained journal checkpoint does not match its immutable attachment."));
}
return (rows.getFirst());
}
/*******************************************************************************
** Authorization is current and actor-specific; the manifest cannot impersonate.
*******************************************************************************/
private void authorize(UploadCandidate candidate) throws QException
{
if(!PersonalRecords.currentUserId().equals(candidate.actorId()))
{
throw (new QNotFoundException("Not found"));
}
ParentRecordAccess.requireFeature(candidate.tableName(), candidate.recordId(), "attach");
}
/*******************************************************************************
** Retained registration cannot participate in an unrelated caller transaction.
*******************************************************************************/
private void requireIndependent() throws QException
{
if(QContext.getQBackendTransaction() != null)
{
throw (new QException("Retained upload recovery requires its own committed transactions."));
}
}
/*******************************************************************************
** Server-owned values only; no client-supplied locator or actor is accepted.
*******************************************************************************/
private UploadCandidate candidate(QRecord row)
{
return (new UploadCandidate(row.getValueString("tableName"), row.getValueString("recordId"), row.getValueString("createdByUserId"),
row.getValueString("fileName"), row.getValueString("contentType"), row.getValueString("description"), row.getValueLong("sizeBytes"),
row.getValueString("sha256"), row.getValueString("bucketName"), row.getValueString("storageKey"), row.getValueBoolean("retained"),
row.getValueString("intakeKey"), row.getValueString("intakeFingerprint")));
}
/*******************************************************************************
** Reconstruct only owner metadata from the immutable saved manifest.
*******************************************************************************/
private QRecord row(UploadCandidate candidate)
{
return (new QRecord().withValue("tableName", candidate.tableName()).withValue("recordId", candidate.recordId())
.withValue("createdByUserId", candidate.actorId()).withValue("fileName", candidate.fileName()).withValue("contentType", candidate.contentType())
.withValue("description", candidate.description()).withValue("sizeBytes", candidate.sizeBytes()).withValue("sha256", candidate.sha256())
.withValue("bucketName", candidate.bucketName()).withValue("storageKey", candidate.storageKey()).withValue("retained", true)
.withValue("parentDeleted", false).withValue("intakeKey", candidate.intakeKey()).withValue("intakeFingerprint", candidate.intakeFingerprint()));
}
/*******************************************************************************
** SHA-256 of actual bounded bytes, never a storage metadata assertion.
*******************************************************************************/
private String hash(byte[] bytes) throws QException
{
try
{
return (HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(bytes)));
}
catch(NoSuchAlgorithmException failure)
{
throw (new QException("SHA-256 is unavailable.", failure));
}
}
}
@@ -0,0 +1,123 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.attachments;
import com.kingsrook.qbits.collaboration.attachments.uploadjournal.UploadJournalTable;
import com.kingsrook.qbits.collaboration.permissions.CollaborationPermissions;
import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetInput;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
/*******************************************************************************
** A visible native recovery action for retained original registration. It never
** accepts replacement bytes, a storage path, an actor, or an intake scope.
*******************************************************************************/
public final class RecoverRetainedUploadProcess
{
public static final String NAME = "collaborationRecoverRetainedUpload";
/*******************************************************************************
** Metadata utility.
*******************************************************************************/
private RecoverRetainedUploadProcess()
{
}
/*******************************************************************************
** Standard QQQ confirmation and result screens, attached to the journal table.
*******************************************************************************/
public static QProcessMetaData metadata()
{
return (new QProcessMetaData().withName(NAME).withTableName(UploadJournalTable.NAME).withLabel("Recover retained upload")
.withMinInputRecords(1).withMaxInputRecords(1).withPermissionRules(CollaborationPermissions.permissionRules())
.withStep(new com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData().withName("prepare").withCode(new QCodeReference(Prepare.class))
.withInputData(new com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMetaData()
.withRecordListMetaData(new com.kingsrook.qqq.backend.core.model.metadata.processes.QRecordListMetaData().withTableName(UploadJournalTable.NAME))))
.withStep(new com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendStepMetaData().withName("confirm").withLabel("Recover the original upload")
.withComponent(new com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendComponentMetaData()
.withType(com.kingsrook.qqq.backend.core.model.metadata.processes.QComponentType.EDIT_FORM))
.withFormField(new QFieldMetaData("confirmRecovery", QFieldType.BOOLEAN).withLabel("Verify the stored original and retry registration").withDefaultValue(false).withIsRequired(true)))
.withStep(new com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData().withName("recover").withCode(new QCodeReference(Recover.class)))
.withStep(new com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendStepMetaData().withName("result").withLabel("Original registration saved")
.withComponent(new com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendComponentMetaData()
.withType(com.kingsrook.qqq.backend.core.model.metadata.processes.QComponentType.VIEW_FORM))
.withViewField(new QFieldMetaData("attachmentId", QFieldType.INTEGER).withLabel("Retained attachment")
.withPossibleValueSourceName(com.kingsrook.qbits.collaboration.model.CollaborationTables.ATTACHMENT))));
}
/*******************************************************************************
** Resolve one visible journal selection before displaying confirmation.
*******************************************************************************/
public static final class Prepare implements BackendStep
{
/*******************************************************************************
** Read-only preview under current actor and parent visibility.
*******************************************************************************/
@Override
public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
if(input.getRecords() == null || input.getRecords().size() != 1)
{
throw (new QException("Select exactly one retained upload."));
}
var row = new GetAction().execute(new GetInput(UploadJournalTable.NAME).withPrimaryKey(input.getRecords().getFirst().getValue("id"))).getRecord();
if(row == null)
{
throw (new QException("Retained upload is unavailable."));
}
if(row.getValueString("intakeKey") != null)
{
throw (new QException("Retry this upload through its original intake request to preserve its context."));
}
output.addValue("journalId", row.getValue("id"));
output.addValue("confirmRecovery", false);
}
}
/*******************************************************************************
** Execution reauthorizes independently of the earlier preview.
*******************************************************************************/
public static final class Recover implements BackendStep
{
/*******************************************************************************
** The owner returns only after mandatory lifecycle registration completes.
*******************************************************************************/
@Override
public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
if(!Boolean.TRUE.equals(input.getValue("confirmRecovery")))
{
throw (new QException("Confirm retained upload recovery to continue."));
}
output.addValue("attachmentId", new AttachmentsService().recoverRetainedUpload(input.getValueInteger("journalId")).id());
}
}
}
@@ -0,0 +1,227 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.attachments;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.Map;
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
import com.kingsrook.qqq.backend.core.actions.permissions.CustomPermissionChecker;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.MetaDataWithPermissionRules;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.PermissionLevel;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRules;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QComponentType;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendComponentMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendStepMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractLoadStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractTransformStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ExtractViaQueryStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ProcessSummaryProviderInterface;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
/*******************************************************************************
** Explicit provider settlement through the standard table action and Review flow.
*******************************************************************************/
public class ResolveAttachmentUploadProcess implements CustomPermissionChecker
{
public static final String NAME = "collaborationResolveAttachmentUpload";
public static final String EVIDENCE = "settlementEvidence";
/*******************************************************************************
** Administrative table action, never a navigation entry or scheduled process.
*******************************************************************************/
public static QProcessMetaData metadata()
{
Map<String, Serializable> values = new HashMap<>();
values.put(StreamedETLWithFrontendProcess.FIELD_SOURCE_TABLE, CollaborationTables.ATTACHMENT_CLEANUP);
values.put(StreamedETLWithFrontendProcess.FIELD_DESTINATION_TABLE, CollaborationTables.ATTACHMENT_CLEANUP);
values.put(StreamedETLWithFrontendProcess.FIELD_SUPPORTS_FULL_VALIDATION, true);
values.put(StreamedETLWithFrontendProcess.FIELD_DO_FULL_VALIDATION, true);
values.put(StreamedETLWithFrontendProcess.FIELD_PREVIEW_MESSAGE,
"Confirm that the provider has completed or cancelled every selected upload. Object absence or elapsed time alone is not proof. Current bytes will be deleted; the settlement receipt will be retained.");
return (StreamedETLWithFrontendProcess.defineProcessMetaData(ExtractViaQueryStep.class, Review.class, Apply.class, values)
.withName(NAME).withLabel("Resolve settled uploads").withTableName(CollaborationTables.ATTACHMENT_CLEANUP).withIcon(new QIcon("check"))
.withPermissionRules(new QPermissionRules().withLevel(PermissionLevel.NOT_PROTECTED)
.withCustomPermissionChecker(new QCodeReference(ResolveAttachmentUploadProcess.class)))
.withStep(0, new QFrontendStepMetaData().withName("evidence").withLabel("Confirm provider settlement")
.withFormField(new QFieldMetaData(EVIDENCE, QFieldType.STRING).withLabel("Provider completion or cancellation evidence")
.withMaxLength(2000).withIsRequired(true))
.withComponent(new QFrontendComponentMetaData().withType(QComponentType.EDIT_FORM))));
}
/*******************************************************************************
** Guard direct process invocation as well as menu visibility.
*******************************************************************************/
@Override
public void checkPermissionsThrowing(AbstractActionInput input, MetaDataWithPermissionRules metadata) throws QPermissionDeniedException
{
AttachmentCleanupQueue.requireSettlementOperator();
}
/*******************************************************************************
** Shared standard summary language for Review and Result.
*******************************************************************************/
private static ProcessSummaryLine line(Status status, String future, String past)
{
ProcessSummaryLine line = new ProcessSummaryLine(status);
line.setSingularFutureMessage(future);
line.setPluralFutureMessage(future);
line.setSingularPastMessage(past);
line.setPluralPastMessage(past);
return (line);
}
/*******************************************************************************
** Review never changes provider state or records a completed settlement.
*******************************************************************************/
public static class Review extends AbstractTransformStep implements ProcessSummaryProviderInterface
{
private final ProcessSummaryLine selected = line(Status.OK, "uploads will be resolved", "uploads selected");
private final ProcessSummaryLine skipped = line(Status.WARNING, "resolved records or committed deletions will be skipped", "records skipped");
/*******************************************************************************
** Require evidence before allowing the administrator to review selected rows.
*******************************************************************************/
@Override
public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
AttachmentCleanupQueue.requireSettlementOperator();
AttachmentCleanupQueue.validateEvidence(input.getValueString(EVIDENCE));
for(QRecord row : input.getRecords())
{
if(row.getValue("attachmentId") != null || row.getValue("resolvedAt") != null)
{
skipped.incrementCountAndAddPrimaryKey(row.getValue("id"));
}
else
{
output.getRecords().add(row);
selected.incrementCountAndAddPrimaryKey(row.getValue("id"));
}
}
}
/*******************************************************************************
** Keep streamed totals across pages.
*******************************************************************************/
@Override
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean isForResultScreen)
{
ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>();
selected.addSelfToListIfAnyCount(lines);
skipped.addSelfToListIfAnyCount(lines);
return (lines);
}
}
/*******************************************************************************
** Apply checks current authorization, row ownership and references again.
*******************************************************************************/
public static class Apply extends AbstractLoadStep implements ProcessSummaryProviderInterface
{
private final AttachmentCleanupQueue queue;
private final ProcessSummaryLine done = line(Status.OK, "uploads will be resolved", "uploads resolved");
private final ProcessSummaryLine failed = line(Status.ERROR, "uploads cannot be resolved", "uploads unresolved; inspect and retry");
/*******************************************************************************
** Construction is also used during context-free metadata validation.
*******************************************************************************/
public Apply()
{
this(new AttachmentCleanupQueue());
}
/*******************************************************************************
** Use the same service with a fixture storage provider in process tests.
*******************************************************************************/
public Apply(AttachmentCleanupQueue queue)
{
this.queue = queue;
}
/*******************************************************************************
** Each locator has its own transaction and immutable settlement receipt.
*******************************************************************************/
@Override
public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
AttachmentCleanupQueue.requireSettlementOperator();
AttachmentCleanupQueue.validateEvidence(input.getValueString(EVIDENCE));
for(QRecord row : input.getRecords())
{
try
{
queue.resolveUpload(row.getValue("id"), input.getValueString(EVIDENCE));
done.incrementCountAndAddPrimaryKey(row.getValue("id"));
}
catch(QException | RuntimeException failure)
{
failed.incrementCountAndAddPrimaryKey(row.getValue("id"));
}
}
}
/*******************************************************************************
** Report actual application results and retain review exclusions.
*******************************************************************************/
@Override
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean isForResultScreen)
{
ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>();
done.addSelfToListIfAnyCount(lines);
failed.addSelfToListIfAnyCount(lines);
if(getTransformStep() instanceof Review review)
{
review.skipped.addSelfToListIfAnyCount(lines);
}
return (lines);
}
}
}
@@ -0,0 +1,99 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.attachments;
import com.kingsrook.qbits.collaboration.CollaborationQBitConfig;
import com.kingsrook.qbits.collaboration.permissions.CollaborationPermissions;
import com.kingsrook.qqq.backend.core.actions.permissions.CustomPermissionChecker;
import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.MetaDataWithPermissionRules;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.PermissionLevel;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRules;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.scheduleing.QScheduleMetaData;
import com.kingsrook.qqq.backend.core.model.session.QSession;
import com.kingsrook.qqq.backend.core.model.session.QSystemUserSession;
/*******************************************************************************
** Scheduled durable attachment cleanup. Runs with its caller's session, which must be a
** scheduler system session or collaboration admin; never elevates a user.
*******************************************************************************/
public class RetryAttachmentCleanupProcess implements BackendStep, CustomPermissionChecker
{
public static final String NAME = "collaborationRetryAttachmentCleanup";
/*******************************************************************************
** Always register the backend-only process; scheduling is opt-in.
*******************************************************************************/
public static QProcessMetaData metadata(CollaborationQBitConfig config)
{
QProcessMetaData process = new QProcessMetaData().withName(NAME).withLabel("Retry attachment cleanup").withIsHidden(true).withIcon(new QIcon("history"))
.withPermissionRules(new QPermissionRules().withLevel(PermissionLevel.HAS_ACCESS_PERMISSION)
.withPermissionBaseName("collaboration").withCustomPermissionChecker(new QCodeReference(RetryAttachmentCleanupProcess.class)))
.withStep(new QBackendStepMetaData().withName("run").withCode(new QCodeReference(RetryAttachmentCleanupProcess.class)));
if(config.getSchedulerName() != null)
{
process.withSchedule(new QScheduleMetaData().withSchedulerName(config.getSchedulerName()).withRepeatSeconds(60).withInitialDelaySeconds(60));
}
return (process);
}
/*******************************************************************************
** Hidden is a presentation setting, so explicitly guard execution as well.
*******************************************************************************/
@Override
public void checkPermissionsThrowing(AbstractActionInput input, MetaDataWithPermissionRules metadata) throws QPermissionDeniedException
{
QSession session = QContext.getQSession();
if(!(session instanceof QSystemUserSession) && (session == null || !session.hasPermission(CollaborationPermissions.ADMIN)))
{
throw (new QPermissionDeniedException("This needs the collaboration.admin permission."));
}
}
/*******************************************************************************
** Retry committed cleanup intents after provider or process recovery.
*******************************************************************************/
@Override
public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
checkPermissionsThrowing(input, null);
new AttachmentCleanupQueue().retryPending();
}
}
Loaded 100 of 207 files, more files were not shown because too many files have changed in this diff. Show more