Compare commits

..
Author SHA1 Message Date
james.maes fbce2e5d6e Merge pull request 'fix(targets): ask whether a table is readable as a query, not with an untyped input' (#69) from fix/target-read-asked-as-query into develop
build / maven (push) Successful in 6m55s
maven / build (push) Successful in 6m55s
2026-11-12 17:20:43 +00:00
james.maesandClaude Fable 5.2 70f6184a7a fix(targets): ask whether a table is readable as a query, not with an untyped input
build / maven (push) Successful in 8m46s
maven / build (push) Successful in 8m41s
TargetResolver.canReadTable asked PermissionsHelper.hasTablePermission with a
bare AbstractActionInput. Core hands a table's custom permission checker the
action input and not the permission asked, so a checker that tells a read
from a write by the input's type (a query, a get or a count reads; anything
else is refused) refused the question. For such a table every recents and
favorites call answered "Not found" for people the checker lets read it:
the UI sends POST /recents on every list and record screen, so each of them
showed "Resource not found POST /qqq/v1/qbit/collaboration/recents (HTTP
404)".

The read is now asked with a QueryInput for the table, the read it stands
for; ParentRecordAccess already asks with a GetInput. Who may read a table
does not change: the checker still decides, and a person it refuses is
still answered "Not found" (RecentsServiceTest
.tableWithAnInputTypedPermissionCheckerIsATarget, which failed before with
QNotFoundException on the first call).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-12 10:53:22 -06:00
james.maes 16bd4f768f Merge pull request 'perf: skip tag lookups for tables with no tag assignments (forward-port of #67)' (#68) from fix/read-path-statements-develop into develop
build / maven (push) Successful in 7m34s
maven / build (push) Successful in 7m34s
2026-11-10 22:11:02 +00:00
james.maesandClaude Fable 5.2 04d04adc6e perf(security): authorize each distinct parent once per child result page
build / maven (push) Successful in 7m15s
maven / build (push) Successful in 7m15s
ParentRowCustomizer and AttachmentPreviewCustomizer read the parent of
every child row to decide whether the row is visible, so rows that share a
parent repeated the same read: 61 tasks on two parents ran 61 parent reads
(TagLookupStatementBudgetTest), now 2. The answer is kept only for the one
postQuery call, where the session and transaction cannot change.

Also correct the README: the Tags field is selectable.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 15:58:11 -06:00
james.maesandClaude Fable 5.2 de922d7e86 perf(tags): no assignment lookup for tables that have no tags
Core selects the Tags virtual field for every query or get that names no
fields, so every such read of a TAGS-enabled table ran one
collaboration_record_tag statement, including reads made one record at a
time and tables that were never tagged. Measured with 600 added parents
(TagLookupStatementBudgetTest): a 602-row list ran 2 statements, 600 gets
ran 600 and 100 single-record updates ran 100; now 1 probe, then 0.

TaggedTables remembers, per node and per table, whether any assignment
exists (core Memoization registered with ClusterCaches, 30 s by default,
withTagLookupCacheSeconds, 0 disables). Writing an assignment clears it at
once and after commit and publishes memo-invalidate. The probe ignores row
locks and returns no data, so the answer is the same for every session.
Tables with assignments are read as before: one lookup per 500 parents.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 15:58:11 -06:00
james.maes c6b077321c Merge pull request 'fix(upload-journal): accept Amazon Aurora PostgreSQL (full_page_writes=off)' (#65) from fix/aurora-full-page-writes-develop into develop
build / maven (push) Successful in 7m35s
maven / build (push) Successful in 7m35s
2026-11-10 15:27:45 +00:00
james.maesandClaude Fable 5.2 2837f4152a fix(upload-journal): accept Amazon Aurora PostgreSQL (full_page_writes=off)
build / maven (push) Successful in 9m8s
maven / build (push) Successful in 9m8s
The PostgreSQL 17 journal qualification refused every call on Amazon Aurora
PostgreSQL, which reports full_page_writes=off and does not expose the
parameter in its cluster parameter group.

- The settings query gains an eleventh column,
  to_regprocedure('aurora_version()') IS NOT NULL, which is true on Aurora,
  false on stock PostgreSQL 17 and never raises.
- full_page_writes=off is accepted only when that column is true. Stock
  PostgreSQL with full_page_writes=off is still refused, and synchronous_commit,
  fsync, recovery, isolation, time zone and lock timeout are unchanged on Aurora.
- The decision is the pure function tornPagesPrevented, covered by an always-on
  unit test. The opt-in real PostgreSQL 17 suite simulates Aurora with a
  function named aurora_version() and full_page_writes=off on the server.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-09 20:26:14 -06:00
james.maes f0f1db7642 Merge pull request 'build(deps): align with Gitea registry (parent 2.2.1, QQQ 4.1.0-RC.26)' (#63) from deps/align-gitea into develop
build / maven (push) Successful in 7m59s
maven / build (push) Successful in 7m59s
2026-11-07 18:03:38 +00:00
james.maesandClaude Opus 5.5 c8150bc9cf build(deps): align with Gitea registry (parent 2.2.1, QQQ 4.1.0-RC.26)
build / maven (push) Successful in 7m48s
maven / build (push) Successful in 7m48s
pom.xml: parent -> 2.2.1
pom.xml: qqq.version: 4.1.0-RC.22 -> 4.1.0-RC.26
pom.xml: qbit-object-storage: 0.1.0-RC.4 -> 0.1.0-RC.10
pom.xml: qbit-redis.version: 0.1.0-RC.1 -> 0.2.0-RC.1
pom.xml: qbit-email.version: 0.1.0-RC.2 -> 0.1.0-RC.11
pom.xml: revision -> 1.2.0-SNAPSHOT

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 17:55:19 +00:00
james.maes 1c6ae14f59 fix(security): check parent-table read permission with a real GetInput (#62)
QRun-IO/ci-policy: native / dotnet-0 (push) Skipped
QRun-IO/ci-policy: ci / head-profile (push) Skipped
QRun-IO/ci-policy: ci / java-core-2-head (push) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (push) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (push) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (push) Skipped
QRun-IO/ci-policy: native / native-result (push) Successful in 13s
QRun-IO/ci-policy: ci / java-maven-7-head (push) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (push) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (push) Skipped
QRun-IO/ci-policy: ci / source-config-13 (push) Skipped
QRun-IO/ci-policy: ci / source-config-14 (push) Skipped
QRun-IO/ci-policy: ci / original-voyage (push) Skipped
QRun-IO/ci-policy: ci / original-cli (push) Skipped
QRun-IO/ci-policy: ci / ci-result (push) Successful in 10s
build / maven (push) Successful in 15m15s
QRun-IO/ci-policy: ci / fork-guard (push) Skipped
QRun-IO/ci-policy: native / fork-deny (push) Skipped
QRun-IO/ci-policy: ci / repository-policy (push) Skipped
QRun-IO/ci-policy: ci / application-template-0 (push) Skipped
QRun-IO/ci-policy: ci / gradle-1 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-6 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (push) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (push) Skipped
QRun-IO/ci-policy: ci / node-8 (push) Skipped
QRun-IO/ci-policy: ci / node-9 (push) Skipped
QRun-IO/ci-policy: ci / node-10 (push) Skipped
QRun-IO/ci-policy: ci / original-business (push) Skipped
QRun-IO/ci-policy: ci / Scoped workflow [workflow 539bec410f7926fec3ef7fabe51934a91e0248be] (push) Successful in 14s
QRun-IO/ci-policy: ci / shared-pipeline (push) Skipped
maven / build (push) Successful in 15m15s
QRun-IO/ci-policy: ci / java-docker-3-head (push) Skipped
QRun-IO/ci-policy: ci / android-0 (push) Skipped
QRun-IO/ci-policy: ci / original-mcp (push) Skipped
QRun-IO/ci-policy: ci / not-enrolled (push) Skipped
2026-11-06 19:30:40 +00:00
James Maes 9248f3f32c test: keep develop's CollaborationApiHttpTest unchanged
QRun-IO/ci-policy: ci / fork-guard (pull_request) Skipped
QRun-IO/ci-policy: native / fork-deny (pull_request) Skipped
QRun-IO/ci-policy: ci / Scoped workflow [workflow 539bec410f7926fec3ef7fabe51934a91e0248be] (pull_request) Successful in 11s
QRun-IO/ci-policy: ci / head-profile (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy (pull_request) Skipped
QRun-IO/ci-policy: ci / application-template-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7-head (pull_request) Skipped
QRun-IO/ci-policy: ci / node-8 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-business (pull_request) Skipped
QRun-IO/ci-policy: ci / original-voyage (pull_request) Skipped
QRun-IO/ci-policy: ci / original-cli (pull_request) Skipped
QRun-IO/ci-policy: ci / ci-result (pull_request) Successful in 10s
build / maven (push) Successful in 10m50s
QRun-IO/ci-policy: native / dotnet-0 (pull_request) Skipped
QRun-IO/ci-policy: native / native-result (pull_request) Successful in 9s
QRun-IO/ci-policy: ci / gradle-1 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-core-2-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-10 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-13 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-14 (pull_request) Skipped
QRun-IO/ci-policy: ci / android-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-mcp (pull_request) Skipped
QRun-IO/ci-policy: ci / not-enrolled (pull_request) Skipped
QRun-IO/ci-policy: ci / shared-pipeline (pull_request) Skipped
maven / build (push) Successful in 10m51s
QRun-IO/ci-policy: ci / node-9 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (pull_request) Skipped
2026-11-06 13:16:55 -06:00
James MaesandClaude Fable 5.1 468dc42751 fix(security): check parent-table read permission with a real GetInput
QRun-IO/ci-policy: ci / fork-guard (pull_request) Skipped
QRun-IO/ci-policy: native / fork-deny (pull_request) Skipped
QRun-IO/ci-policy: ci / Scoped workflow [workflow 539bec410f7926fec3ef7fabe51934a91e0248be] (pull_request) Successful in 11s
QRun-IO/ci-policy: ci / head-profile (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy (pull_request) Skipped
QRun-IO/ci-policy: ci / application-template-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7-head (pull_request) Skipped
QRun-IO/ci-policy: ci / node-8 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-business (pull_request) Skipped
QRun-IO/ci-policy: ci / original-voyage (pull_request) Skipped
QRun-IO/ci-policy: ci / original-cli (pull_request) Skipped
QRun-IO/ci-policy: ci / ci-result (pull_request) Successful in 8s
build / maven (push) Successful in 11m14s
QRun-IO/ci-policy: native / dotnet-0 (pull_request) Skipped
QRun-IO/ci-policy: native / native-result (pull_request) Successful in 8s
QRun-IO/ci-policy: ci / gradle-1 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-core-2-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-10 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-13 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-14 (pull_request) Skipped
QRun-IO/ci-policy: ci / android-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-mcp (pull_request) Skipped
QRun-IO/ci-policy: ci / not-enrolled (pull_request) Skipped
QRun-IO/ci-policy: ci / shared-pipeline (pull_request) Skipped
maven / build (push) Successful in 11m14s
QRun-IO/ci-policy: ci / node-9 (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (pull_request) Skipped
ParentRecordAccess.canReadTable passed a bare AbstractActionInput to
PermissionsHelper.hasTablePermission, so the READ check did not run as an actual
table read. Pass a GetInput for the table instead, which is what a host's table
permission rules evaluate. Ported from the GG Forge parent-read qualification.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 13:16:06 -06:00
james.maes 73681b3a2e Merge pull request 'ci: pin shared workflow to v1' (#60) from ci/pin-v1-develop into develop
QRun-IO/ci-policy: native / dotnet-0 (push) Skipped
QRun-IO/ci-policy: native / native-result (push) Successful in 12s
QRun-IO/ci-policy: ci / head-profile (push) Skipped
QRun-IO/ci-policy: ci / java-core-2-head (push) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (push) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (push) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-7-head (push) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (push) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (push) Skipped
QRun-IO/ci-policy: ci / source-config-13 (push) Skipped
QRun-IO/ci-policy: ci / source-config-14 (push) Skipped
QRun-IO/ci-policy: ci / original-voyage (push) Skipped
QRun-IO/ci-policy: ci / original-cli (push) Skipped
QRun-IO/ci-policy: ci / ci-result (push) Waiting to run
build / maven (push) Successful in 9m44s
QRun-IO/ci-policy: ci / fork-guard (push) Skipped
QRun-IO/ci-policy: native / fork-deny (push) Skipped
QRun-IO/ci-policy: ci / Scoped workflow [workflow 1e25bb5a243dfdcd95d4b9a4f780200ff8c8778a] (push) Successful in 14s
QRun-IO/ci-policy: ci / repository-policy (push) Skipped
QRun-IO/ci-policy: ci / application-template-0 (push) Skipped
QRun-IO/ci-policy: ci / gradle-1 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-6 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (push) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (push) Skipped
QRun-IO/ci-policy: ci / node-8 (push) Skipped
QRun-IO/ci-policy: ci / node-9 (push) Skipped
QRun-IO/ci-policy: ci / node-10 (push) Skipped
QRun-IO/ci-policy: ci / original-business (push) Skipped
QRun-IO/ci-policy: ci / shared-pipeline (push) Skipped
maven / build (push) Successful in 9m44s
QRun-IO/ci-policy: ci / java-docker-3-head (push) Skipped
QRun-IO/ci-policy: ci / android-0 (push) Skipped
QRun-IO/ci-policy: ci / original-mcp (push) Skipped
QRun-IO/ci-policy: ci / not-enrolled (push) Skipped
2026-11-06 16:04:53 +00:00
james.maes 483dc271fe Merge pull request 'fix(custom-fields): reject over-long STRING values and reload help after publication' (#61) from fix/custom-field-length-and-help-20261106 into develop
QRun-IO/ci-policy: native / dotnet-0 (push) Skipped
QRun-IO/ci-policy: ci / head-profile (push) Skipped
QRun-IO/ci-policy: native / native-result (push) Successful in 8s
QRun-IO/ci-policy: ci / java-core-2-head (push) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (push) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (push) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-7-head (push) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (push) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (push) Skipped
QRun-IO/ci-policy: ci / source-config-13 (push) Skipped
QRun-IO/ci-policy: ci / source-config-14 (push) Skipped
QRun-IO/ci-policy: ci / original-voyage (push) Skipped
QRun-IO/ci-policy: ci / original-cli (push) Skipped
QRun-IO/ci-policy: ci / ci-result (push) Successful in 8s
build / maven (push) Successful in 10m55s
QRun-IO/ci-policy: ci / fork-guard (push) Skipped
QRun-IO/ci-policy: native / fork-deny (push) Skipped
QRun-IO/ci-policy: ci / Scoped workflow [workflow 1e25bb5a243dfdcd95d4b9a4f780200ff8c8778a] (push) Successful in 10s
QRun-IO/ci-policy: ci / repository-policy (push) Skipped
QRun-IO/ci-policy: ci / application-template-0 (push) Skipped
QRun-IO/ci-policy: ci / gradle-1 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-6 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (push) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (push) Skipped
QRun-IO/ci-policy: ci / node-8 (push) Skipped
QRun-IO/ci-policy: ci / node-9 (push) Skipped
QRun-IO/ci-policy: ci / node-10 (push) Skipped
QRun-IO/ci-policy: ci / original-business (push) Skipped
QRun-IO/ci-policy: ci / shared-pipeline (push) Skipped
maven / build (push) Successful in 10m55s
QRun-IO/ci-policy: ci / java-docker-3-head (push) Skipped
QRun-IO/ci-policy: ci / android-0 (push) Skipped
QRun-IO/ci-policy: ci / original-mcp (push) Skipped
QRun-IO/ci-policy: ci / not-enrolled (push) Skipped
2026-11-06 15:32:01 +00:00
james.maes 813894f8c6 Merge pull request 'Integrate attachments, upload journal, presence and Redis presence for KOF22' (#59) from kof22/collab-v2 into develop
QRun-IO/ci-policy: native / dotnet-0 (push) Skipped
QRun-IO/ci-policy: native / native-result (push) Successful in 7s
QRun-IO/ci-policy: ci / java-core-2-head (push) Skipped
QRun-IO/ci-policy: ci / head-profile (push) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (push) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-7-head (push) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (push) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (push) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (push) Skipped
QRun-IO/ci-policy: ci / source-config-13 (push) Skipped
QRun-IO/ci-policy: ci / source-config-14 (push) Skipped
QRun-IO/ci-policy: ci / original-voyage (push) Skipped
QRun-IO/ci-policy: ci / original-cli (push) Skipped
QRun-IO/ci-policy: ci / ci-result (push) Successful in 8s
build / maven (push) Failing after 10m42s
QRun-IO/ci-policy: ci / fork-guard (push) Skipped
QRun-IO/ci-policy: native / fork-deny (push) Skipped
QRun-IO/ci-policy: ci / Scoped workflow [workflow 1e25bb5a243dfdcd95d4b9a4f780200ff8c8778a] (push) Successful in 9s
QRun-IO/ci-policy: ci / repository-policy (push) Skipped
QRun-IO/ci-policy: ci / application-template-0 (push) Skipped
QRun-IO/ci-policy: ci / gradle-1 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-6 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (push) Skipped
QRun-IO/ci-policy: ci / node-8 (push) Skipped
QRun-IO/ci-policy: ci / node-9 (push) Skipped
QRun-IO/ci-policy: ci / node-10 (push) Skipped
QRun-IO/ci-policy: ci / original-business (push) Skipped
QRun-IO/ci-policy: ci / shared-pipeline (push) Skipped
maven / build (push) Failing after 10m42s
QRun-IO/ci-policy: ci / java-docker-3-head (push) Skipped
QRun-IO/ci-policy: ci / android-0 (push) Skipped
QRun-IO/ci-policy: ci / original-mcp (push) Skipped
QRun-IO/ci-policy: ci / not-enrolled (push) Skipped
2026-11-06 14:52:51 +00:00
james.maesandClaude Fable 5.1 ef625ca300 fix(custom-fields): reject over-long STRING values and reload help after publication
QRun-IO/ci-policy: ci / fork-guard (pull_request) Skipped
QRun-IO/ci-policy: native / fork-deny (pull_request) Skipped
QRun-IO/ci-policy: ci / head-profile (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy (pull_request) Skipped
QRun-IO/ci-policy: ci / application-template-0 (pull_request) Skipped
build / maven (push) Successful in 5m44s
QRun-IO/ci-policy: ci / java-maven-6 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7-head (pull_request) Skipped
QRun-IO/ci-policy: ci / node-8 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-business (pull_request) Skipped
QRun-IO/ci-policy: ci / original-voyage (pull_request) Skipped
QRun-IO/ci-policy: ci / original-cli (pull_request) Skipped
QRun-IO/ci-policy: ci / ci-result (pull_request) Successful in 6s
QRun-IO/ci-policy: native / dotnet-0 (pull_request) Skipped
maven / build (push) Successful in 5m44s
QRun-IO/ci-policy: ci / Scoped workflow [workflow 1e25bb5a243dfdcd95d4b9a4f780200ff8c8778a] (pull_request) Successful in 7s
QRun-IO/ci-policy: native / native-result (pull_request) Successful in 6s
QRun-IO/ci-policy: ci / gradle-1 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-core-2-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-10 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-13 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-14 (pull_request) Skipped
QRun-IO/ci-policy: ci / android-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-mcp (pull_request) Skipped
QRun-IO/ci-policy: ci / not-enrolled (pull_request) Skipped
QRun-IO/ci-policy: ci / shared-pipeline (pull_request) Skipped
QRun-IO/ci-policy: ci / node-9 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (pull_request) Skipped
Bounded STRING custom fields now carry QQQ's native ValueTooLongBehavior.ERROR,
so a write longer than maxLength is rejected with a field error and the stored
value is preserved. The QQQ default is PASS_THROUGH, which let over-long values
through unchecked.

After the runtime metadata overlay refreshes (immediately, or in the after-commit
callback of the originating transaction), persisted help content is reloaded so
sections and fields created by publishing a definition receive their stored
helpContent rows without a restart.

Two CustomFieldsTest regressions cover invalid-write preservation and stored
help on a newly published section. Carried locally by GG Forge as
collaboration-custom-fields.patch (supersedes collaboration-field-length.patch).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: James Maes <james@kof22.com>
2026-11-06 08:39:48 -06:00
james.maesandClaude Opus 5.5 6060b737c5 ci: pin shared workflow to v1
QRun-IO/ci-policy: ci / fork-guard (pull_request) Skipped
build / maven (push) Successful in 3m35s
QRun-IO/ci-policy: native / fork-deny (pull_request) Skipped
QRun-IO/ci-policy: ci / head-profile (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy (pull_request) Skipped
QRun-IO/ci-policy: ci / application-template-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7-head (pull_request) Skipped
QRun-IO/ci-policy: ci / node-8 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-business (pull_request) Skipped
QRun-IO/ci-policy: ci / original-voyage (pull_request) Skipped
QRun-IO/ci-policy: ci / original-cli (pull_request) Skipped
QRun-IO/ci-policy: ci / ci-result (pull_request) Waiting to run
maven / build (push) Successful in 3m36s
QRun-IO/ci-policy: native / dotnet-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / Scoped workflow [workflow 1e25bb5a243dfdcd95d4b9a4f780200ff8c8778a] (pull_request) Successful in 9s
QRun-IO/ci-policy: native / native-result (pull_request) Successful in 9s
QRun-IO/ci-policy: ci / gradle-1 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-core-2-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-10 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-13 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-14 (pull_request) Skipped
QRun-IO/ci-policy: ci / android-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-mcp (pull_request) Skipped
QRun-IO/ci-policy: ci / not-enrolled (pull_request) Skipped
QRun-IO/ci-policy: ci / shared-pipeline (pull_request) Skipped
QRun-IO/ci-policy: ci / node-9 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (pull_request) Skipped
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 14:06:32 +00:00
james.maesandClaude Sonnet 5.5 024cfc21a9 release: prepare Collaboration 1.0 (Core RC.22 BOM, Redis RC.1, current status docs)
QRun-IO/ci-policy: ci / node-8 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-business (pull_request) Skipped
QRun-IO/ci-policy: ci / original-voyage (pull_request) Skipped
QRun-IO/ci-policy: ci / original-cli (pull_request) Skipped
QRun-IO/ci-policy: ci / fork-guard (pull_request) Skipped
build / maven (push) Successful in 11m4s
QRun-IO/ci-policy: ci / ci-result (pull_request) Successful in 6s
QRun-IO/ci-policy: native / fork-deny (pull_request) Skipped
QRun-IO/ci-policy: ci / head-profile (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy (pull_request) Skipped
QRun-IO/ci-policy: ci / application-template-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7-head (pull_request) Skipped
QRun-IO/ci-policy: ci / node-10 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-13 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-14 (pull_request) Skipped
QRun-IO/ci-policy: ci / android-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-mcp (pull_request) Skipped
QRun-IO/ci-policy: ci / not-enrolled (pull_request) Skipped
QRun-IO/ci-policy: ci / Scoped workflow [workflow 1e25bb5a243dfdcd95d4b9a4f780200ff8c8778a] (pull_request) Successful in 12s
QRun-IO/ci-policy: ci / shared-pipeline (pull_request) Skipped
QRun-IO/ci-policy: native / native-result (pull_request) Successful in 9s
maven / build (push) Successful in 11m4s
QRun-IO/ci-policy: native / dotnet-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / gradle-1 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-core-2-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-9 (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (pull_request) Skipped
Replace the default-active Core snapshot profile with the released Core
4.1.0-RC.22 BOM, pin qbit-redis to its first release candidate, and replace
the snapshot-era status text with the current 1.0 status.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-05 17:25:50 -06:00
James MaesandClaude Sonnet 5.5 63245b14e0 feat(collaboration): integrate presence, Redis presence, virtual-field and fixture work on the attachment base
QRun-IO/ci-policy: ci / fork-guard (pull_request) Skipped
QRun-IO/ci-policy: native / fork-deny (pull_request) Skipped
QRun-IO/ci-policy: ci / Scoped workflow [workflow d20a578c7d818bc7ca32a2ac2c675da4e4f27385] (pull_request) Successful in 5s
QRun-IO/ci-policy: ci / head-profile (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy (pull_request) Skipped
QRun-IO/ci-policy: ci / application-template-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7-head (pull_request) Skipped
QRun-IO/ci-policy: ci / node-8 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-business (pull_request) Skipped
QRun-IO/ci-policy: ci / original-voyage (pull_request) Skipped
QRun-IO/ci-policy: ci / original-cli (pull_request) Skipped
QRun-IO/ci-policy: ci / ci-result (pull_request) Successful in 5s
build / maven (push) Failing after 42s
QRun-IO/ci-policy: native / dotnet-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / gradle-1 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-core-2-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (pull_request) Skipped
QRun-IO/ci-policy: native / native-result (pull_request) Successful in 4s
QRun-IO/ci-policy: ci / java-docker-3-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-10 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-13 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-14 (pull_request) Skipped
QRun-IO/ci-policy: ci / android-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-mcp (pull_request) Skipped
QRun-IO/ci-policy: ci / not-enrolled (pull_request) Skipped
QRun-IO/ci-policy: ci / shared-pipeline (pull_request) Skipped
maven / build (push) Failing after 42s
QRun-IO/ci-policy: ci / node-9 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (pull_request) Skipped
One reviewable integration on current develop 7d61f29e. It starts from
feat/record-files-standalone (attachment lifecycle, upload journal, retention
SPI; descends from develop) and adds the owner stack content from
feature/collab, feature/collab-int-20261029, feature/field-values-1196,
feature/presence-recovery-1300, test/presence-capacity-1302 and the
fix/attachment-transactions-1289 / fix/mywork-query-bounds-1291 work, plus
KOF22 adaptations: CollaborationQBitConfig.attachmentLifecycle,
CollaborationRecords.insertActivity/query(transaction), ActivityWriter source
feature routing, and the Redis/fixture tests.

Build: develop CI files are preserved. pom.xml only adds the Redis API
guardrail allow-list and an optional qbit-redis dependency; the core and
qbit-redis versions stay on the canonical SNAPSHOT lines with markers to repin
after the core and redis integration PRs are published (order CORE -> REDIS ->
COLLAB). No PRIVATE or feature-snapshot coordinate is introduced. Not yet
compiled: it needs the core integration artifact (virtual-field providers).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-05 17:19:49 -06:00
James MaesandGPT-6 Codex fb7c6316bf docs: record successful Record Files consumer verification
Co-Authored-By: GPT-6 Codex <noreply@openai.com>
2026-11-05 17:19:48 -06:00
James MaesandGPT-6 Codex 9fbe20c9d6 fix: handle upload recovery metadata and align Core dependency
Verify all 269 Collaboration tests, including PostgreSQL attachment recovery and MySQL/PostgreSQL journals, against the compatible Core snapshot.

Co-Authored-By: GPT-6 Codex <noreply@openai.com>
2026-11-05 17:19:47 -06:00
1c05422901 chore: preserve paused QBit work and document remaining issues
Save the current implementation and README, status, and known-issue documentation at the user-requested stop. No completed release or deployment is claimed.

Co-Authored-By: Claude <noreply@anthropic.com>

Co-Authored-By: GPT-6 Codex <noreply@openai.com>
2026-11-05 17:19:47 -06:00
James MaesandClaude Sonnet 5.5 89e77b67b9 feat(attachments): reconcile reviewed attachment lifecycle, upload journal and retention SPI
Selective three-way reconciliation onto develop 7d61f29 (base c4eb972 = original
ancestor; theirs = reviewed b6fb2382 third-party-src/qbit-collaboration).
Attachments package (main and tests), upload journal (H2/MySQL/PostgreSQL),
cleanup queue, recovery/adoption processes, attachment schema fields, tables,
pagination, delivery schema, help text and docs. Upstream presence work, CI
workflows and release convention are untouched. Redis presence, my-work bounds,
notifier and custom-field changes in b6 are deliberately not included.

Targets qqq 4.1.0-RC.21 BOM. Needs core QBackendTransaction after-rollback
callbacks (not in RC.21): see docs/CORE-DEPENDENCY.md. Not compiled against RC.21.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-05 17:19:46 -06:00
james.maes 9c51710f09 Merge pull request 'ci: use shared ci-workflows Maven workflow' (#57) from ci/ci-workflows into develop
QRun-IO/ci-policy: native / dotnet-0 (push) Skipped
QRun-IO/ci-policy: ci / head-profile (push) Skipped
QRun-IO/ci-policy: native / native-result (push) Successful in 7s
QRun-IO/ci-policy: ci / java-core-2-head (push) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (push) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (push) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-7-head (push) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (push) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (push) Skipped
QRun-IO/ci-policy: ci / source-config-13 (push) Skipped
QRun-IO/ci-policy: ci / source-config-14 (push) Skipped
QRun-IO/ci-policy: ci / original-voyage (push) Skipped
QRun-IO/ci-policy: ci / original-cli (push) Skipped
QRun-IO/ci-policy: ci / ci-result (push) Successful in 7s
build / maven (push) Successful in 3m3s
QRun-IO/ci-policy: ci / fork-guard (push) Skipped
QRun-IO/ci-policy: native / fork-deny (push) Skipped
QRun-IO/ci-policy: ci / Scoped workflow [workflow d20a578c7d818bc7ca32a2ac2c675da4e4f27385] (push) Successful in 8s
QRun-IO/ci-policy: ci / repository-policy (push) Skipped
QRun-IO/ci-policy: ci / application-template-0 (push) Skipped
QRun-IO/ci-policy: ci / gradle-1 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-6 (push) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (push) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (push) Skipped
QRun-IO/ci-policy: ci / node-8 (push) Skipped
QRun-IO/ci-policy: ci / node-9 (push) Skipped
QRun-IO/ci-policy: ci / node-10 (push) Skipped
QRun-IO/ci-policy: ci / original-business (push) Skipped
QRun-IO/ci-policy: ci / shared-pipeline (push) Skipped
maven / build (push) Successful in 3m3s
QRun-IO/ci-policy: ci / java-docker-3-head (push) Skipped
QRun-IO/ci-policy: ci / android-0 (push) Skipped
QRun-IO/ci-policy: ci / original-mcp (push) Skipped
QRun-IO/ci-policy: ci / not-enrolled (push) Skipped
2026-11-05 23:15:25 +00:00
james.maesandClaude Opus 5.5 a0f9a172d0 ci: use shared ci-workflows Maven workflow
QRun-IO/ci-policy: ci / fork-guard (pull_request) Skipped
QRun-IO/ci-policy: native / fork-deny (pull_request) Skipped
QRun-IO/ci-policy: ci / Scoped workflow [workflow 1b3648d9421543350067e95e54959dfb471b26d5] (pull_request) Successful in 7s
QRun-IO/ci-policy: ci / head-profile (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy (pull_request) Skipped
QRun-IO/ci-policy: ci / application-template-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-6-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-maven-7-head (pull_request) Skipped
QRun-IO/ci-policy: ci / node-8 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-business (pull_request) Skipped
QRun-IO/ci-policy: ci / original-voyage (pull_request) Skipped
QRun-IO/ci-policy: ci / original-cli (pull_request) Skipped
build / maven (push) Successful in 3m9s
QRun-IO/ci-policy: Shared Java Maven CI / verify (pull_request) Successful in 3m25s
QRun-IO/ci-policy: Shared Java Maven CI / secrets (pull_request) Failing after 1s
QRun-IO/ci-policy: Shared Java Maven CI / gate (pull_request) Failing after 13s
QRun-IO/ci-policy: ci / java-maven-6 (pull_request) Failing after 17m15s
QRun-IO/ci-policy: ci / ci-result (pull_request) Failing after 8s
QRun-IO/ci-policy: native / dotnet-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / gradle-1 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-core-2-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-3-head (pull_request) Skipped
QRun-IO/ci-policy: ci / java-docker-4 (pull_request) Skipped
QRun-IO/ci-policy: ci / java-frontend-browser-5 (pull_request) Skipped
QRun-IO/ci-policy: native / native-result (pull_request) Successful in 6s
QRun-IO/ci-policy: ci / node-10 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-13 (pull_request) Skipped
QRun-IO/ci-policy: ci / source-config-14 (pull_request) Skipped
QRun-IO/ci-policy: ci / android-0 (pull_request) Skipped
QRun-IO/ci-policy: ci / original-mcp (pull_request) Skipped
QRun-IO/ci-policy: ci / not-enrolled (pull_request) Skipped
QRun-IO/ci-policy: ci / shared-pipeline (pull_request) Skipped
maven / build (push) Successful in 3m8s
QRun-IO/ci-policy: ci / node-9 (pull_request) Skipped
QRun-IO/ci-policy: ci / node-playwright-11 (pull_request) Skipped
QRun-IO/ci-policy: ci / repository-policy-12 (pull_request) Skipped
Replace the platform-ci release callers with one caller of
KofTwentyTwo/ci-workflows maven.yaml (qqq-orb port).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 22:03:19 +00:00
118 changed files with 10850 additions and 830 deletions

No files matched your search

+7
View File
@@ -0,0 +1,7 @@
name: build
on:
push:
jobs:
maven:
uses: self:KofTwentyTwo/ci-workflows/.gitea/workflows/maven.yaml@v1
secrets: inherit
-24
View File
@@ -1,24 +0,0 @@
# Standard QRun release caller (v2, thin contract): every push and manual dispatch delegates to the protected qrun-release-v1 channel. Do not edit per repository.
name: qrun-release
on:
push: {}
workflow_dispatch: {}
permissions:
contents: read
actions: read
jobs:
release:
uses: self:KofTwentyTwo/platform-ci/.gitea/workflows/java-maven.yaml@qrun-release-v1
with:
runner: linux-small
browsers: true
browser-runner: browser-build
postpublish:
if: ${{ needs.release.outputs.capture == 'true' }}
needs: release
permissions:
contents: write
actions: write
uses: self:KofTwentyTwo/platform-ci/.gitea/workflows/qrun-source-version.yaml@qrun-release-v1
with:
runner: linux-small
-72
View File
@@ -1,72 +0,0 @@
name: Release candidate
'on':
workflow_dispatch:
inputs:
dry_run:
description: Plan a cut without changing release state
type: boolean
default: false
finalize:
description: Retry finalization of an existing cut; do not cut another RC
type: boolean
default: false
recover:
description: Recover the same published RC after a deployment timeout; never
reupload
type: boolean
default: false
circle_build:
description: Original release run id (recovery only)
type: string
default: ''
merge_sha:
description: Existing cut merge_sha (finalize retry only)
type: string
default: ''
bump_sha:
description: Existing cut bump_sha (finalize retry only)
type: string
default: ''
branch:
description: Existing cut branch (finalize retry only)
type: string
default: ''
version:
description: Existing cut version (finalize retry only)
type: string
default: ''
pull_request:
types:
- closed
branches:
- develop
permissions:
contents: read
jobs:
release-rc:
# The old qbit-rc-published repository_dispatch is replaced by the already-proven workflow_dispatch
# completion: the publisher's own ActionsWRITE job dispatches this workflow with finalize=true plus
# merge_sha/bump_sha/branch/version (same inputs as the manual finalize retry).
permissions:
contents: write
actions: read
pull-requests: write
if: github.event_name == 'workflow_dispatch'
|| (github.event_name == 'pull_request' && github.event.pull_request.merged
== true && github.event.pull_request.base.ref == 'develop' && contains(github.event.pull_request.labels.*.name,
'release:rc'))
uses: self:KofTwentyTwo/platform-ci/.gitea/workflows/qbit-release-rc.yaml@qrun-release-v1
with:
dry_run: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run || false
}}
finalize: ${{ github.event_name == 'workflow_dispatch' && inputs.finalize }}
recover: ${{ github.event_name == 'workflow_dispatch' && inputs.recover || false
}}
circle_build: ${{ github.event_name == 'workflow_dispatch' && inputs.circle_build
|| '' }}
merge_sha: ${{ inputs.merge_sha || '' }}
bump_sha: ${{ inputs.bump_sha || '' }}
branch: ${{ inputs.branch || '' }}
version: ${{ inputs.version || '' }}
secrets:
QRUN_TAG_SIGNING_KEY: ${{ secrets.QRUN_TAG_SIGNING_KEY }}
+6
View File
@@ -29,6 +29,12 @@ All notable changes are documented here, following [Keep a Changelog](https://ke
### Fixed
- Favorites and recents accept a table whose custom permission checker tells a read from a write by the type of the action input. `TargetResolver` asked whether a table is readable with an untyped `AbstractActionInput`; such a checker refuses an input it does not recognise, so `POST /recents` (which the UI sends on every list and record screen) and starring answered `404 Not found` for people the checker lets read the table. The read is now asked as the query it stands for, as `ParentRecordAccess` already asks with a get. Who may read a table is unchanged.
- Reads of a table that has no tag assignment no longer query `collaboration_record_tag`. Core selects the Tags field for every query or get that names no fields, so each of them (including per-record gets and the stored-record read before an update) ran one assignment statement whether or not the table had ever been tagged: 600 gets of an untagged table ran 600 statements, now none. Each node remembers which tables have no assignment (`TaggedTables`, core `Memoization` registered with `ClusterCaches`) for `CollaborationQBitConfig.withTagLookupCacheSeconds` (default 30, 0 disables); writing an assignment clears it on the node and through the ClusterBus. Tables with assignments are read as before, one lookup per 500 parents. No API or schema change. See "What the Tags field costs a read" in the README.
- Browsing one of the qbit's child tables (comments, tasks, record tags, attachments and the other parent-referencing tables) reads each distinct parent once per result page to authorize its rows, instead of once per row: 61 tasks on two parents ran 61 parent reads, now 2. The check itself and its outcome are unchanged.
- The retained upload journal's PostgreSQL 17 qualification accepts Amazon Aurora PostgreSQL, which reports `full_page_writes=off` and does not let a deployment change it. `off` is accepted only when the settings query detects Aurora (`to_regprocedure('aurora_version()') IS NOT NULL`); stock PostgreSQL with `full_page_writes=off` is still refused, and every other requirement is unchanged on Aurora. See `docs/RETAINED-UPLOAD-JOURNAL.md`.
- Bounded STRING custom fields carry QQQ's native `ValueTooLongBehavior.ERROR`, so a write longer than `maxLength` is rejected with a field error and the stored value is preserved, instead of passing through unchecked (the QQQ default `PASS_THROUGH`).
- Persisted help content is reloaded after each committed runtime metadata refresh, so sections and fields created by publishing a custom-field definition receive their stored `helpContent` rows without a restart.
- QBit metadata reports the Maven artifact version from a filtered resource, including release versions.
- `presence.changed` events and presence snapshots always carry `viewers`, sending `[]` once the last viewer leaves, as the schema requires (#52).
+11 -2
View File
@@ -1,6 +1,6 @@
# QBit Collaboration
> **Core integration verified locally (2026-11-04).** All 269 Collaboration tests pass against the matching Core snapshot, including attachment recovery and both journal providers. See [current integration status](docs/QBIT-RELEASE-STATUS.md), [dependency setup](docs/CORE-DEPENDENCY.md) and [remaining release requirements](docs/QBIT-KNOWN-ISSUES.md). This feature branch is not a published release.
> **Version 1.0.** Published to the KOF22 Gitea Maven registry (`com.kingsrook.qbits`). Requires QQQ Core `4.1.0-RC.22` or later. See [current status](docs/QBIT-RELEASE-STATUS.md).
Record collaboration for QQQ: attachments, tags, custom fields, comments with mentions, presence, sharing, favorites, follow, timeline and tasks. **Implemented now:** feature settings and people tables, favorites and recents services (phase b), per-user security, daily recents retention, phase-c attachments (upload/list/download/rename/delete, previews and parent cleanup), and the Collaboration app home with real session-scoped counts. Phase e adds comments/mentions, follow actions and digests, the audit/activity timeline, and automatic people-directory updates. Phase d adds tags, vocabulary policies, management and bulk actions, and core-integrated tag filtering/search. Phases h/i add sharing/access requests, tasks/reminders, assignments and My work services. Phases f/g add the custom-field designer and ephemeral replicated presence. Runtime overrides and TAGS mode settings resolve through core.
@@ -409,7 +409,16 @@ size, pending suggestions, and the five most-used approved tags.
`collaborationTags` is a virtual STRING field labeled Tags with possible-value source `collaboration_tag`. `TagCriteriaExpander` supports IN, NOT_IN, EQUALS, NOT_EQUALS, IS_BLANK, IS_NOT_BLANK and CONTAINS_ALL using core `QSubQuery` membership filters. Ordinary quick search matches labels; `#name` matches normalized names. Numeric criteria are tag IDs; string criteria are names. The old ID materializer is removed. Core owns SQL execution or bounded cross-backend materialization; its bound applies to inner rows, not final parent matches.
The runtime provider adds this field and its search entry where TAGS is enabled. It is queryable but not selectable: the overlay has no virtual-field post-query hook and existing host customizers must be preserved.
The runtime provider adds this field and its search entry where TAGS is enabled. It is queryable and selectable (`TagValueProvider` fills it after host customizers have run).
### What the Tags field costs a read
Core selects every selectable virtual field for a query or get that names no fields, so each such read of a table with TAGS enabled runs `TagValueProvider`, including the reads other code makes one record at a time (a get, the stored-record read before an update, a lookup made per row). The provider therefore costs:
- nothing for a table that has no tag assignment. Each node remembers that one fact per table (`TaggedTables`) for `withTagLookupCacheSeconds` (default 30; 0 looks every time) and learns it with one indexed `LIMIT 1` statement;
- one assignment statement and one vocabulary statement per 500 parents for a table that has assignments. Record tags themselves are never cached.
A node that writes an assignment forgets the table's answer at once and again after commit, and publishes core's `memo-invalidate` cluster event (`ClusterCaches`, name `collaboration.taggedTables`). Hosts with a ClusterBus transport see a table's first tag on every node immediately; without one, another node shows it after at most the configured seconds. The same bound covers assignments written outside QQQ. The answer does not depend on the session (the probe ignores row locks and returns no data), so it cannot change which records or tags a caller sees. A failed probe is thrown, not remembered.
## UI API
+1
View File
@@ -5,4 +5,5 @@
No schema change and nothing required of hosts.
- New table action **Create rows from configuration** on Feature Settings. Run it once if you want the effective feature settings visible and editable as rows. It changes no behavior and does nothing once the table has any row. Note that after rows exist they win over later changes to `withDefaultFeatures` or `withTableFeatures` in code, so change those settings in the rows from then on.
- Reads of tables without any tag assignment skip the assignment lookup (`withTagLookupCacheSeconds`, default 30 seconds, 0 restores a lookup on every read). A host running several nodes without a ClusterBus transport shows a table's very first tag on the other nodes after at most that long; later tags on that table show at once, as before.
- New **Setup** widget first on the Collaboration home, and table help text. Hosts that override help on a table keep their own.
+76
View File
@@ -0,0 +1,76 @@
# Coordinated collaboration snapshot
## Combined integration follow-up under qualification
PR #51 combines current attachment settlement, My Work, presence recovery,
contributed-field and capacity follow-ups. Its source inventory and gate history
are in [INTEGRATION-20261029.md](INTEGRATION-20261029.md). The candidate selects
verified published Redis `0.1.0-redis-integration-20261029-551f1d5-SNAPSHOT` and
the existing core `4.1.0-virtual-field-values-1196-cc3981bb3-SNAPSHOT`.
The combined collaboration artifact is not yet published. Historical snapshot
and acceptance evidence below is not proof of this candidate's deployment.
## Selectable-field follow-up snapshot
The following focused upgrade is published for QQQ #1196:
| Component | Version |
| --- | --- |
| QQQ BOM and modules | `4.1.0-virtual-field-values-1196-cc3981bb3-SNAPSHOT` |
| qbit-collaboration | `0.1.0-field-values-1196-ecff347-SNAPSHOT` |
| qbit-redis (unchanged prerequisite) | `0.1.0-redis-7fe5492-SNAPSHOT` |
Import this QQQ BOM in the consuming host. Collaboration publisher 168 passed
from `ecff347efe2bb84949b8452a3383e3fdeb4bcede`; its POM and JAR resolve to
timestamp `20261026.133203-1`. Full collaboration verification against the
published core passed 198 tests with no failures, errors or skips, plus packaging
and coverage gates. An independent empty-cache Java 21 consumer downloaded the
pair and compiled both collaboration value providers against the core API.
See [the upgrade evidence](VIRTUAL-FIELD-UPGRADE.md) for checksums and scope.
This is not a combined release of the separate attachment, My Work and presence
recovery follow-ups, nor a Kof22 deployment. The earlier candidate's browser
acceptance below must not be treated as browser qualification of this new pair.
A compatible packaged frontend and host staging qualification remain required.
## Earlier main-line candidate
The earlier main-line candidate is published. Its coordinated versions are:
| Component | Version |
| --- | --- |
| QQQ BOM and modules | `4.1.0-collab-33464d37e-SNAPSHOT` |
| qbit-redis | `0.1.0-redis-7fe5492-SNAPSHOT` |
| qbit-collaboration | `0.1.0-collab-8f348da-SNAPSHOT` |
| qbit-session-store | `0.1.0-session-b5276ab-SNAPSHOT` |
| Next frontend source | `8d372142f8af0c41a75a353edc6d150c33ccf613` |
Maven snapshot repository: https://central.sonatype.com/repository/maven-snapshots/ . Import the candidate QQQ BOM in the host to align transitive QQQ modules. The standalone consumer was verified from an empty cache. This is a feature snapshot, not a stable/GA release or a production deployment.
Frontend build and full browser acceptance: https://github.com/QRun-IO/qqq-frontend-next/actions/runs/39683690943 . Its acceptance-build artifact contains the tested export. No dashboard container image publication is implied. Install the export through the normal host frontend packaging path; upgrade frontend and backend together because the shared-presence issued-lease protocol changed.
## Host requirements
- Configure a named Redis connection through qbit-redis and enable CollaborationQBitConfig.redisPresence. Use explicit deployment/application namespaces and a PresenceScopeProvider derived from authoritative authenticated tenant context.
- Start Redis infrastructure before dependent services and stop dependent services before closing its pools. Session-store may select SHARED_REDIS to reuse the named connection; its session semantics remain owned by session-store.
- Configure TLS/authentication and bounded timeouts/pools for the actual environment. The live acceptance fixture's plaintext loopback Redis and mock authentication are test-only.
- Deliver SSE without proxy buffering. The stream and its subscription updates must reach the owning web node; configure affinity for that connection. Shared Redis does not make node-local SSE connection IDs portable.
- Configure record versions on supported backend paths for stale-save protection. Presence is advisory and never replaces database compare-and-write. Unversioned records do not gain conflict protection merely by enabling presence.
- Use explicit QInstance registration when startup code runs outside QContext. Follow core docs/misc/SessionStoreIsolation.md and session-store docs/SESSION-REGISTRATION.md. JDBC shutdown and health operations must target the owning application; see core docs/lifecycle/application-shutdown.md.
- Apply generated schema/migrations through the host's migration tooling. Do not infer that publishing this library migrated an existing database or an external deployed host.
## Verification
All four standard backend publishers and the three QBit source CI runs passed. Tests against published prerequisites: Redis23, collaboration177, session-store96; zero failures/errors/skips and configured coverage gates passed. Core verification4209 and frontend acceptance39683690943 passed. Frontend local3664 tests and TypeScript passed.
Independent JAR/POM downloads, exact timestamped versions, SHA-256 hashes and an empty-cache standalone consumer passed. Final browser checks used published backend JARs, the exact CI frontend export, two independent JVMs and one Redis endpoint. Cross-node avatars, self exclusion, concurrent editor field focus, preserved drafts, table presence and departure passed. Redis admin pool/subscription metrics, persisted framework UP health history and health-record navigation passed.
The two-node fixture uses independent memory records and mock users; it proves published-artifact presence/admin integration. Database conflict and authentication behavior have separate core/frontend acceptance evidence. Publication does not prove unknown deployed hosts have migrated.
Full evidence and issue reconciliation: https://github.com/QRun-IO/qqq/issues/1293#issuecomment-6329618640 .
## Supported milestone and optional work
The owner requested the main-line flow on N web nodes sharing one configured Redis endpoint. Browser delivery remains SSE plus HTTP lease writes; Redis Pub/Sub supplies invalidation hints. A new WebSocket protocol is not required.
Data-loss/restore qualification (#1300), additional failover topologies (#1301), sustained scale/failure injection (#1302), Windows tooling qualifications (#1304/#1305/#1309), CI mirror resilience (#1307), search layout polish (#1310), long-version preflight (#1312) and test-only AssertJ maintenance (#1314) remain tracked separately in the QQQ project. No measured production capacity or disaster-recovery guarantee is claimed by this snapshot.
+1 -4
View File
@@ -1,9 +1,6 @@
# Core dependency: after-rollback callbacks
Collaboration's retained-upload and cleanup paths compensate external object storage only after a transaction's
rollback is confirmed. They need two methods on `com.kingsrook.qqq.backend.core.actions.QBackendTransaction` that the
released core `4.1.0-RC.21` does not declare. This module does not compile against RC.21 until a core release that has them
is available. This branch now defaults to `4.1.0-SNAPSHOT`; the verified source is Core commit `551397c7c59b0f77833fc81e195f4c01cb1c282f`. Build/install that Core branch first for local development. No compatibility shim is provided (a shim could not confirm rollback and would weaken the guarantee).
Collaboration's retained-upload and cleanup paths compensate external object storage only after a transaction's rollback is confirmed. They need two methods on `com.kingsrook.qqq.backend.core.actions.QBackendTransaction`. These are included in QQQ Core `4.1.0-RC.22`, which this module imports through the BOM; Core `4.1.0-RC.21` does not declare them.
## Required API (reviewed implementation: `b6fb2382`, `third-party-src/qqq-core`)
+51
View File
@@ -0,0 +1,51 @@
# Coordinated collaboration integration candidate
This branch combines the current heads of the attachment, My Work, presence
recovery, virtual-field and capacity PRs. It is a qualification candidate, not
a published release or a deployed host.
| PR | Included source |
| --- | --- |
| #45 attachment settlement | `dc65a826ce362416d55d91136fa39225f4a7cf94` |
| #46 bounded My Work | `cdcf7ee36509c7a1fc6f22fa864209eb80a6ff34` |
| #47 presence recovery | `776ebe3fcb6492881bc9d7ec3f6ca6adfa320200` |
| #48 contributed field values | `1da538cf43ba1ed80e3f8fd599729b5691563cbf` |
| #49 capacity qualification | `56a8b760d9c1a1862cb11e80fad07cf5cdc20b3a` |
All five heads are ancestors of integration commit
`44a89d77b1ad3a39d26c512ada78af0884872013`. The stacked My Work and capacity
branches did not contain the latest heads of their base PRs, so those base
heads were merged explicitly as well. No production source was rewritten during
integration. The core prerequisite remains
`4.1.0-virtual-field-values-1196-cc3981bb3-SNAPSHOT`. The candidate now selects
published Redis `0.1.0-redis-integration-20261029-551f1d5-SNAPSHOT` after native
publisher job42 succeeded and independent timestamped POM/JAR downloads matched
their declared version and repository checksums. The timestamp is
`20261029.165102-1`; JAR SHA256 is
`385f69c5d84362dbab40a3044261fb9820e25126785d7f22901121302e6ce076`.
The caller and manifest both select released `kingsrook/qqq-orb@0.8`. The
canonical validator extracted from released 0.8.32 and CircleCI schema validation
pass; no unpublished orb changes are used.
Local `./mvnw.cmd verify -Dqqq.mywork.load=true` ran 238 tests with no skips, but
reported three failures and four errors. Failures include Redis command read
timeouts/rejoin assertions and PostgreSQL connection I/O during the attachment
fixture. Docker Desktop's published-port path has independently exhibited
300–900 ms replies after idle, compared with 0–58 ms on a direct Docker network.
This observation is not proof of every failure's cause. No timeout, assertion,
security boundary, or quality gate was weakened. Native hosted Linux results
are required before publication; the local run is **not** a passing gate.
Native CircleCI job192 subsequently passed the full verification at integration
head `02085baff928c374e3b0d77367fbcc8182f81f93`; CodeQL also passed. Those checks
used the earlier Redis pin. The new published Redis pin above requires its own
current-head native verification before this collaboration artifact is published.
Use an explicit short feature branch (`feature/collab-int-20261029`) and dry-run
the released version calculator: generated help-bundle versions must remain at
most 50 characters. The longer `collaboration-integration-20261029` name would
produce 57 characters and is not a valid publication route.
The successful earlier combined-source checks remain historical evidence only.
Host TLS/ACL, actual identity/proxy/provider acceptance, maintenance/restore
enforcement, artifact publication, and deployment are separate gates.
+92
View File
@@ -0,0 +1,92 @@
# My Work summary query bounds
The summary previously materialized every matching task, assignment and mention.
A regression fixture measured 601 parent lookups for 601 tasks on two parents.
The summary now scans each category in primary-key pages of at most 200 rows.
It retains at most ten display tasks and sorts them using the existing due-date
order (earliest first, null last, then ID). Counts still include every visible
matching row, rather than silently truncating at a display limit. Assignment and
mention filters remain scoped to the signed-in user.
Each page caches parent visibility only for that page, using the existing caller
permission and record-lock checks. Missing or revoked parents contribute neither
rows nor counts; unexpected authorization/backend failures propagate. No decision
is shared across requests. The starting maximum ID bounds each category scan so
concurrent inserts cannot extend it indefinitely. Like the earlier summary, this
is not a transactionally consistent snapshot across the three categories.
This bounds row retention, not total work independently of data size. For N
matching rows, each category needs one high-watermark query plus approximately
ceil(N/200) page queries, and one parent lookup per distinct parent per page.
Many unique parents still require linear authorization work. Per-record heavy
field size is also outside this row-count bound. Database and browser latency,
N-node presence, saturation and production capacity remain part of #1291/#1302.
Regression evidence: the 601-task case passes a budget of at most eight parent
queries, with no task query above 200 rows; correct top-ten ordering and revoked
access on a subsequent request are checked. Another case crosses assignment and
mention page boundaries and excludes hidden parents, other users and undelivered
mentions. These tests use the framework's real memory backend instrumentation.
`MyWorkPostgresTest` repeats the 601-row case using PostgreSQL 17 and the generated
production schema. Independent `pg_stat_statements` observations verify exactly
five task queries and between one and eight parent queries. It also verifies the
same top ten, visible count and next-request permission revocation against SQL.
The fixture passes; these query counts are not sustained throughput or
deployment-capacity measurements.
## Reproducible local workload
Run `mvn -Dtest=MyWorkPostgresTest -Dqqq.mywork.load=true test` with Java21 and
Docker. The fixture owns a disposable PostgreSQL17 container, uses the generated
schema and real authorization checks, and writes `target/mywork-load.csv`.
It never connects to an application database. The optional workload runs after
the normal query-budget regression so ordinary CI keeps the smaller test.
Each scenario has 2,000 tasks: one uses ten shared parents, the other 2,000 distinct
parents. Every tenth parent is inaccessible. Two warmups precede ten measured
sequential summaries. Every sample verifies 1,800 visible tasks and the exact top
ten. SQL calls and server execution time are collected independently using
pg_stat_statements after warmup. Latency includes service execution and database
round trips, excluding fixture setup and assertions.
CSV percentiles use nearest rank. With only ten samples, p95 and p99 both equal
the maximum observed sample; they are descriptive, not stable tail estimates.
SQL figures are totals for all ten samples. This is a single-JVM local workload,
not HTTP/browser latency, sustained concurrency, pool saturation, multi-node
presence or a deployment-capacity claim. Do not derive production alert thresholds
from these numbers. No wall-clock performance assertion is added to CI.
The fixture explicitly uses the framework C3P0 connection provider, matching the
host's provider, with initial/minimum1, maximum3 and a two-second checkout timeout.
It verifies the pool stays within that cap, has no checkout timeouts and returns
all borrowed connections after each scenario. The original fixture used the
unpooled default; the expanded workload failed with a Windows socket-allocation
error after repeatedly opening connections. This was a fixture mismatch, not
proof of a Kof22 outage. The same workload completed with pooling.
Observed local run (Java21, PostgreSQL17 in Docker on Windows, ten sequential
samples after two warmups): shared-parent p50=91.192ms, max=112.105ms; distinct-parent
p50=960.565ms, max=1100.260ms. SQL calls per summary were113 and2013 respectively.
This confirms linear distinct-parent authorization work remains despite bounded
row retention. The existing exact-count semantics still require examining every
matching task. No runtime optimization or production-capacity claim follows from
this fixture measurement alone.
## Compatibility with the later attachment changes
On October 26, an isolated source export combined attachment commit
`dc65a826ce362416d55d91136fa39225f4a7cf94` with the four My Work files from
`2ac067bea4af199486f90e2dc5212baf8910b8fb`. Both branches share base
`15cb8cbf72aa9a5e033280cf5a38384d1295fc7d`; their changed files do not overlap.
This was a source-overlay verification, not a branch merge or publication.
Java 21 `./mvnw.cmd verify -Dqqq.mywork.load=true` passed all 209 tests with zero
failures, errors or skips, including the 29 attachment transaction tests.
Packaging and JaCoCo gates passed. The optional PostgreSQL workload retained the
same visible-count, top-ten, query-budget and pool assertions. Shared-parent
p50/max were 76.301/96.044 ms; distinct-parent p50/max were 939.896/1035.402 ms.
SQL counts remained 113 and 2013 per summary. These local measurements preserve
the limitations above; they do not qualify an integrated published artifact or
Kof22 deployment. The eventual integrated branch still needs its own CI gates.
+179
View File
@@ -0,0 +1,179 @@
# Presence recovery candidate — QQQ #1300
This is an unreleased implementation candidate in collaboration draft PR47.
The existing store/service suites and CI pass; host adoption and enforced
maintenance remain incomplete. It is not part of the published snapshots.
## Contract under qualification
Each deployment/application/tenant/table has a persistent recovery guard in the
same Redis hash slot as its bounded room projections. Each Lua operation reads
that guard atomically with its room work. The guard contains a generation, a
readiness deadline and the latest issued expiry. A missing guard, or complete
loss of table projections before that expiry, starts a fresh generation and a
recovery interval of one lease lifetime. Existing live browsers can reserve and
renew replacement leases during that interval. Snapshots report recovery rather
than a successful empty room until the interval finishes.
Lease entries carry the guard generation. Recovery removes invalid-generation
projections from touched rooms so they cannot consume all capacity and prevent
re-registration. Existing authorization remains in PresenceService; the guard
does not grant record, tenant or field access. The shared adapter maps recovery
to a sanitized service-unavailable response. Record-version conflict checks are
independent of this advisory state.
An untouched new namespace is conservatively unknown too. Its first snapshot
therefore starts a recovery interval. Normal idle expiry retains the guard so
later empty snapshots can be known-empty. The guard has no TTL; tenant-retirement
operations and long-term guard storage must be included in operational capacity
qualification.
## Controlled restore
The host configuration has a bounded `recoveryGeneration`, default `initial`,
included in both the storage namespace and realtime audience. It belongs in
deployment configuration outside the Redis backup. All nodes must use the same
value. A restored guard can otherwise restore old authority; Redis process IDs
cannot detect arbitrary same-process data restoration.
The required maintenance procedure is: stop admission and drain all old web
nodes; restore Redis; choose a never-reused generation outside Redis; start only
nodes with that generation; verify recovery and authenticated re-registration;
then reopen traffic. Do not roll back the generation when rolling back an app.
The host tooling that enforces and verifies this sequence is **not implemented**
yet. The branch does not claim transparent arbitrary restore or split-brain
recovery. Native Sentinel/Cluster routing remains the separate Redis issue #1301.
The storage prefix is v2 so old executables cannot treat generation-bound entries
as old-format authority. This requires a coordinated deployment, not a mixed
version rolling deployment. Existing published v1 snapshots are unchanged.
## Evidence and remaining work
- Before the patch, a two-client disposable Redis test failed because total loss
returned an empty snapshot.
- Seven focused tests now pass: full loss, loss of volatile projections while the
guard survives, restored guard/lease bytes followed by a host generation
change, bounded generation validation, and rejection of inconsistent lease
lifetimes across nodes, guard loss across previously unvisited records, and
shared cold readiness followed by normal idle expiry. The latter verifies
that peer reads do not extend the deadline, the guard remains persistent,
expired projections remain known-empty and expired leases cannot renew.
These use independent clients in
one JVM, not a deployed HA cluster.
- Full wrapper verification passed on October 26: 184 tests, no failures,
errors or skips; packaging and coverage gates passed. This includes the
original seven store tests and two-node six-viewer/two-editor scenario.
The admin dashboard reads every table, so the fixture initializes every
dashboard table for both tenants through the real backend and waits for
recovery without writing Redis guard state.
- The service fixture socket read timeout is one second; its previous 200 ms
limit caused a Redis EVALSHA read timeout. Production timeouts are unchanged.
- The full run includes a real HTTP recovery test: real Redis
FLUSHDB produces 503 during recovery, the former lease returns 409 with
PRESENCE_LEASE_EXPIRED, hidden records and unauthorized viewers remain
inaccessible, and a new lease rejoins through the production recovery window.
This uses a loopback HTTP server with fixture authentication, not a browser
or a deployed host. The focused test and the combined full suite both pass.
- The store suite checks loss of either state or expiry data in both record
and table projections. Server-issued lease renewal must fail unavailable
without changing the surviving projection. It also covers complete loss
of either projection pair and stale departure ordering.
- Still required: deployed-host qualification and enforced host maintenance/reset. Local
per-operation cost measurements at 1,000 and 10,000 leases are recorded in
`acceptance/RECOVERY-COST.md`; sustained production capacity is unqualified.
The implementation uses no INFO call or lifecycle polling cache.
- This is an unreleased draft candidate. Publication and production deployment
have not occurred; the remaining acceptance work above is still required.
## Independent-process acceptance
`docs/acceptance/presence_recovery_processes.py` passed on October 26 using
two independent `SharedPresenceDevServer` JVMs and one disposable Redis container.
Each user renews through the opposite node. After FLUSHDB, both nodes return 503
for presence and 409 for the former leases; fresh leases restore both viewers on
both nodes. The harness terminates only its owned processes and container.
This is fixture authentication with independently seeded memory records, not a
deployed HA application or a browser acceptance result.
Reproduce from the repository root after `./mvnw test-compile`:
```text
./mvnw dependency:build-classpath -Dmdep.includeScope=test -Dmdep.outputFile=target/recovery-process-classpath.txt
python docs/acceptance/presence_recovery_processes.py --java /path/to/java --repo /path/to/repository
```
Requires Python 3, Java 21 and Docker. JVM output and argument files stay under
`target/recovery-processes-*` for diagnosis. The harness uses the production
45-second recovery window, with bounded startup and recovery deadlines.
## Browser recovery acceptance
The same harness accepts `--frontend-repo /path/to/qqq-frontend-next`. It copies
that checkout's existing `out` export into the owned test output directory and
serves it through the real middleware on both JVMs. Playwright is loaded from
the frontend checkout; no API mocks are installed. Both browser users first
see the other viewer, then Redis FLUSHDB triggers the unavailable message and
removes stale people. Both viewer badges return without reloading either page.
The final fixed-candidate run passed all five profiles on October 26 against
commit `4b0d6a32d6bd8ecba023857d3661d9f3f5980975`: Chromium, Firefox, WebKit,
Pixel 7 emulation and iPad emulation. Source/compiled Lua hashes were checked
before the run and after each profile. Traces and screenshots remain under
each `target/recovery-processes-*` directory. The earlier Chromium screenshot
in `target/recovery-processes-229f305e83` was also visually inspected.
Select profiles using `--browser-profile chromium|firefox|webkit|mobile|tablet`.
These tests use the local export and do not establish provenance of a newly
published frontend artifact, physical-device behavior or deployed Kof22
acceptance. Fixture frontend serving is opt-in via `collab.dev.frontend`;
its default remains false.
## Controlled restore acceptance
`acceptance/presence_restore_processes.py` exercises a disposable Redis RDB
restore with two independent JVMs. It saves the populated dataset, verifies
both old JVMs have terminated and their listeners have closed, clears the
dataset, reloads the saved image and starts both nodes with a shared fresh
generation supplied outside Redis. It checks recovery responses, rejects old
leases on both nodes, and requires two fresh viewers to rejoin across nodes.
The final run passed on October 26; logs are retained in
`target/restore-processes-aec5288289`. Both leases are explicitly renewed before
saving, and rejection must complete within 40 seconds of that renewal, before
their normal 45-second expiry. Old leases are rejected again after fresh
viewers have rejoined.
The fixture enables Redis DEBUG RELOAD only on its owned loopback container.
Run after test compilation and classpath preparation:
```text
python docs/acceptance/presence_restore_processes.py --java /path/to/java --repo /path/to/repository
```
This is a protocol acceptance fixture, not host maintenance automation. Actual
host admission closure, replacement prevention and generation configuration
still require implementation and staging validation.
To qualify a packaged backend, add `--backend-jar /path/to/qbit-collaboration.jar`.
This excludes production `target/classes` from the JVM classpath, rejects a
duplicate collaboration dependency JAR, and records the artifact SHA-256 in
`target/restore-processes-*/result.json`. The receipt includes the elapsed time
for rejection of old leases (required to remain below their normal expiry),
the restore/rejoin checks and fixture cleanup. Cleanup failure fails the run.
The test repository still supplies compiled fixture classes and its dependency
classpath; this is not a standalone production launcher.
## Packaged browser qualification
The process harness accepts `--backend-jar` and `--frontend-jar` for artifact
verification. With a backend JAR, production `target/classes` is omitted from the
runtime classpath; test classes provide only the disposable server fixture. The
frontend JAR is served directly instead of copying a local `out` export.
`--frontend-repo` remains required for Playwright tooling. Dependency classpaths
must not contain a second collaboration or frontend artifact.
Each run writes `result.json` with the selected artifact SHA-256 hashes, browser
profile, result and cleanup outcome. A successful test with failed fixture cleanup
exits unsuccessfully. This records local artifact qualification, not publication
or deployment provenance. Supply `--browser-profile` to qualify each required
browser independently.
+5 -8
View File
@@ -1,9 +1,6 @@
# QBit integration known issues
# Known issues
Updated: 2026-11-04.
- This branch requires the Core integration snapshot described in [CORE-DEPENDENCY.md](CORE-DEPENDENCY.md). Public Core RC.21 lacks rollback callbacks; public Collaboration RC.6 lacks the retained-upload SPI. A compatible published dependency set remains a release requirement.
- Local owner verification is complete: 269 tests and style/coverage gates pass, including both journal providers. Deployment-specific object storage, database failover, backup/restore and retention policies still require host qualification.
- Docker must be running for attachment transaction tests. Enable both `mysqlJournalTests` and `postgresJournalTests` for the additional journal-provider cases. On Windows use `-Dexec.skip=true` for the inherited POSIX console-summary command; all verification gates still run.
- Redis presence and unrelated private-cohort features were intentionally excluded. This attachment integration has no compile-time dependency on Record Files, Scripting or Document Generation.
- No release tag, public artifact publication or deployment is claimed by this branch. The matching Next UI and host configuration must be verified before rollout.
- Deployment-specific object storage, database failover, backup/restore and retention policies need host qualification.
- Docker must be running for the attachment transaction tests; enable both `mysqlJournalTests` and `postgresJournalTests` for the additional journal-provider cases. On Windows use `-Dexec.skip=true` for the inherited POSIX console-summary command.
- Redis presence needs `qbit-redis` on the classpath; it is an optional dependency.
- The matching Next UI and host configuration must be verified before rollout.
+7 -21
View File
@@ -1,27 +1,13 @@
# Collaboration attachment integration status
# Collaboration — status
Updated: 2026-11-04. Implementation and local owner verification complete; this feature branch is not a published release.
**Version 1.0**, built and published by the shared `ci-workflows` Maven pipeline to the KOF22 Gitea Maven registry (`com.kingsrook.qbits`); `mvn deploy` runs the tests once.
## What changed
## What it contains
The attachment lifecycle, retained upload journal, recovery and cleanup APIs are preserved on canonical Collaboration. The default Core dependency is now the compatible `4.1.0-SNAPSHOT` from Core commit `551397c7c59b0f77833fc81e195f4c01cb1c282f` (see [setup](CORE-DEPENDENCY.md)). Recovery queries now handle tables without supplemental metadata instead of throwing a null-pointer exception.
Attachment lifecycle with the retained upload journal, recovery and cleanup APIs; presence including Redis-backed presence; tags, custom fields, comments with mentions, sharing, favorites, follow, timeline and tasks; virtual-field and fixture support.
## Verification
## Dependencies
Using Java 21, the Maven wrapper, Docker and that locally installed Core snapshot:
QQQ Core `4.1.0-RC.22` (imported through its BOM; it provides the rollback callbacks the upload paths need). `qbit-redis` is optional and only needed for Redis presence. Build with Java 21 and `./mvnw verify`; Docker must be running for the attachment transaction tests (set `mysqlJournalTests` and `postgresJournalTests` for the extra journal-provider cases).
```powershell
.\mvnw.cmd '-DmysqlJournalTests=true' '-DpostgresJournalTests=true' '-Dexec.skip=true' install
```
All **269 tests passed**, with zero failures, errors or skips. Checkstyle and the JaCoCo coverage gates passed. This includes 29 PostgreSQL attachment transaction/recovery cases, 11 MySQL journal cases, 12 PostgreSQL journal cases, HTTP API tests and dependency-boundary guardrails. Each provider fixture creates and removes its own disposable services; no existing database is used. The first run exposed the fixed null handling and one transient Windows socket allocation failure; the complete corrected run passed.
`-Dexec.skip=true` avoids the inherited POSIX console-summary command on Windows; it does not skip tests, Checkstyle or coverage. The artifact was installed into an isolated local Maven repository as `com.kingsrook.qbits:qbit-collaboration:0.1.0-SNAPSHOT`.
## Downstream consumer verification
Record Files `develop` at `162af9a7b8cd9c2673c26c88f00323cd90e85e48` compiled and passed its default `verify` lifecycle against this locally installed Collaboration snapshot and Core `4.1.0-SNAPSHOT` using `-Dqqq.version=4.1.0-SNAPSHOT`. Surefire reported 278 tests: 220 executed successfully and 58 optional MySQL/PostgreSQL cases skipped. Style/coverage gates passed. This proves consumer API compatibility and the exercised HTTP, retention and recovery paths; it does not claim the skipped Record Files provider qualification or its full release.
## Release boundary
Core and Collaboration must be published as a compatible dependency set before adoption from public Maven coordinates. Passing local fixtures does not qualify a deployment's storage, backup/restore or database failover. See [known issues and remaining release requirements](QBIT-KNOWN-ISSUES.md).
Host qualification of object storage, database failover, backup/restore and retention policy remains the deploying team's responsibility. See [known issues](QBIT-KNOWN-ISSUES.md).
+47
View File
@@ -0,0 +1,47 @@
# Shared Redis presence candidate
Status: coordinated candidate artifacts are published; remote CI, empty-cache consumer resolution and published-artifact two-node browser checks passed. See [CANDIDATE-RELEASE.md](CANDIDATE-RELEASE.md) for exact versions and deployment requirements. This guide describes the current main-line scope. See MAINLINE-ACCEPTANCE.md for evidence and SESSION-STATE.md for live release checkpoints.
## Host configuration and dependency ownership
Multiple QQQ web nodes share one configured Redis endpoint. The host enables CollaborationQBitConfig.redisPresence, supplies a started named Redis connection, explicit deployment/application names, and a PresenceScopeProvider deriving one tenant scope from authenticated host context. Browser input cannot choose the tenant.
Collaboration owns presence semantics, Lua scripts, record/table projections and authorization. qbit-redis owns connections, namespaces, script execution, pool lifecycle, admin management and framework health integration. Collaboration production code consumes only the Redis public API and does not import Jedis. Without shared configuration, the local presence backend remains available for a single node; its issued authority is not distributed. Runtime Redis failure never silently switches an HA host to local presence.
Browser delivery uses the existing SSE stream with HTTP lease writes. Redis Pub/Sub carries scoped invalidation hints between nodes; a new WebSocket transport is not required. Hints contain table/record identifiers, not record contents or viewer lists. The recipient rereads through the authorized presence API. Core checks the authenticated recipient audience during live delivery and replay. Coordinate core/frontend/QBit versions and do not share a topic with audience-unaware hubs.
## Issued lease lifecycle
The candidate collaboration extension version is 1.3.0:
- POST /presence/leases authorizes the table/record and creates an invisible, expiring reservation, returning a server-issued connectionId.
- POST /presence/leases/renew requires that issued handle and a positive safe-integer sequence. It activates or updates existing live authority; it cannot recreate an expired or departed lease.
- DELETE /presence performs ordered departure. Delayed renewals cannot reopen the departed issued handle.
- HTTP 409 with code PRESENCE_LEASE_EXPIRED tells an active browser to open a fresh lease. Input, permission and infrastructure failures retain their standard mappings. Unavailable shared storage returns 503 rather than an empty-room result.
- Shared presence rejects legacy heartbeat upserts. Local legacy behavior remains for compatibility.
The Next candidate awaits open before activation, uses the handle across focus changes, and releases it on hide/unmount. It opens again on pageshow or explicit expiry. Generation checks discard old asynchronous responses; a late open result is released without activation. One immediate rejoin is allowed after expiry, then retries follow the normal 20-second heartbeat. Infrastructure errors do not churn handles. The UI preserves drafts and shows unavailable status when presence cannot be read or renewed.
## State and permissions
Record and table rooms use bounded expiry indexes and state hashes in the same Redis cluster hash slot for their deployment/application/tenant/feature namespace. Lua updates both projections atomically using Redis TIME. User/connection/record identifiers are encoded independently. No record values are stored.
Snapshots merge tabs by user. EDITING takes precedence over VIEWING; focus follows the latest renewal within the selected mode. Table projections omit focused fields. Expired entries are pruned on reads/mutations and idle keys expire. Invisible reservations count toward capacity. Scripts use EVALSHA with fallback only for NOSCRIPT, through the Redis QBit API.
Reads and renewals recheck parent and field access. Canonical record identifiers and recipient field filtering are enforced. Admin counts are limited to known rooms in the administrator's authenticated tenant; non-admins receive no counts. Capacity and TTL limits bound individual operations; they are not measured sustained production throughput guarantees.
## Verified behavior
- Real Redis tests cover six users on a record with two editors, duplicate-tab aggregation, ordering, departure, expiry, isolation, capacity and script-cache recovery.
- Service and real HTTP tests cover issued open/activation/renew/departure, 409 expiry, authorization, canonical keys, field projection and unavailable responses.
- Clean release-source collaboration verify passed 177 tests with coverage/architecture gates; focused frontend suites passed 34 tests, TypeScript and changed-file lint.
- Live browser acceptance with mocks disabled used three identities on two independent QQQ JVMs sharing Redis. It verified avatars, field focus, drafts, table projection and departure. Demo records were independently seeded memory data; this is presence evidence, not cross-node database save-consistency evidence.
- Live Redis admin acceptance verified pool/subscription status, persisted framework health results and navigation to standard health records.
Presence is advisory. Atomic optimistic record-version checks protect saves independently. Coordinated cold-consumer acceptance against published artifacts remains required; local Maven cache success does not prove remote artifact availability.
## Deferred work
The user narrowed this release to the normal flow. Optional qualification is tracked in QQQ issues #1300 (data loss/backup restore), #1301 (failover/disaster topologies), and #1302 (sustained scale/failure injection), all on the QQQ Roadmap backlog.
Issued leases do not provide disaster-recovery fencing: complete state loss can temporarily look empty, and restoring an old live lease can restore its authority. No recovery generation guard is implemented in this candidate. The deferred investigation is tracked in [QQQ #1300](https://github.com/QRun-IO/qqq/issues/1300).
+8 -5
View File
@@ -2,7 +2,7 @@
The Collaboration-owned journal is wired into retained `AttachmentsService` upload and recovery. It persists candidate metadata before object creation, owns the attachment/checkpoint transaction, and exposes native read-only recovery navigation. It has no Record Files dependency and stores no second copy of file bytes.
Historical private-cohort evidence (superseded by [current verification](QBIT-RELEASE-STATUS.md)): the integrated owner snapshot passed all **214 tests**, including 10 real MySQL provider cases and 11 H2 journal regressions, and was installed under `0.1.0-record-files-SNAPSHOT`. The subsequent doublewrite/session-read-only delta compiles and awaits its provider rerun. This document does not qualify Aurora, other SQL vendors, external object storage, disks, backups, or a host deployment.
The integrated owner snapshot passed all **214 tests**, including 10 real MySQL provider cases and 11 H2 journal regressions, and was installed under `0.1.0-record-files-SNAPSHOT`. The subsequent doublewrite/session-read-only delta compiles and awaits its provider rerun. This document does not qualify Aurora, other SQL vendors, external object storage, disks, backups, or a host deployment.
## Ownership and recovery
@@ -25,9 +25,12 @@ Both owner tables must use the same QQQ backend. Hosts apply DDL through their m
| --- | --- | --- |
| File H2 | Native `h2`, `jdbc:h2:file:`, actual configured connection, writable real transaction, live `WRITE_DELAY=0` | `UploadJournalTable.h2Script()` |
| MySQL 8.4 | Native `mysql`, actual MySQL 8.4 product/version, exact configured URL target and catalog, writable real transaction, both owner tables InnoDB, fixed owner table mappings, journal `utf8mb4_0900_bin` columns/table, single-column unique locator/intake indexes, synchronous commit settings | `UploadJournalTable.mysqlScript()` |
| PostgreSQL 17 | Native `postgres` backend, configured database, permanent owner tables, deterministic comparison, READ COMMITTED, UTC, 1–5000 ms lock timeout, synchronous commits with fsync/full_page_writes enabled | `UploadJournalTable.postgresScript()` |
| PostgreSQL 17 | Native `postgres` (vendor and backend type), actual PostgreSQL 17 product/version, exact configured `jdbc:postgresql://host/catalog` target and catalog, writable real transaction, not in recovery, `synchronous_commit` `on` or `remote_apply`, `fsync=on`, `full_page_writes=on` (see the Aurora rule below), session `TimeZone` UTC, `lock_timeout` from 1 to 5000 ms, READ COMMITTED, both owner tables permanent ordinary tables with fixed mappings, no non-deterministic collation, complete unique indexes on `locator_hash` and `intake_key` | `UploadJournalTable.postgresScript()` |
| Amazon Aurora PostgreSQL 17 | Every PostgreSQL 17 requirement above, except that `full_page_writes=off` is accepted | `UploadJournalTable.postgresScript()` |
| Aurora MySQL, MariaDB, other versions/providers | Refused; no inferred compatibility or durability claim | None |
Aurora rule: stock PostgreSQL must report `full_page_writes=on`, because recovery repairs a partly written page from the full page image in the write-ahead log. Amazon Aurora PostgreSQL reports `full_page_writes=off` and the parameter is not in its cluster parameter group, so a deployment cannot turn it on; its storage layer does not write torn pages, so the protection is not needed there. The qualification therefore accepts `off` only when the same settings query reports `to_regprocedure('aurora_version()') IS NOT NULL`, which is true on Aurora, false on stock PostgreSQL 17 and never raises. Nothing else is relaxed on Aurora: `synchronous_commit`, `fsync`, recovery state, isolation, time zone and lock timeout are checked exactly as on stock PostgreSQL. The detection is a name lookup, so a function named `aurora_version()` on the session's search path of a stock server is reported as Aurora; creating one is a schema change inside the host's trust boundary, like the settings themselves.
The MySQL adapter resolves explicit or native-generated URLs through QQQ `ConnectionManager`. It accepts a single ordinary `jdbc:mysql://host:port/catalog` target with a simple catalog identifier. Proxy/load-balancing URLs, alternate catalogs or table remapping fail closed. Error messages do not echo URLs or credentials. Driver options remain host-owned; TLS and credentials are not changed by the journal.
Each MySQL operation checks `innodb_flush_log_at_trx_commit=1`, and `sync_binlog=1` when binary logging is enabled; a read-only server or session is refused. It also requires `innodb_doublewrite` to be `ON` or `DETECT_AND_RECOVER`; [detect-only mode does not retain recoverable page copies](https://dev.mysql.com/doc/refman/8.4/en/innodb-doublewrite-buffer.html). These settings follow the [MySQL 8.4 InnoDB durability documentation](https://dev.mysql.com/doc/refman/8.4/en/innodb-parameters.html#sysvar_innodb_flush_log_at_trx_commit) and [binary-log durability documentation](https://dev.mysql.com/doc/refman/8.4/en/replication-options-binary-log.html#sysvar_sync_binlog). Hardware that misreports flush completion can still lose data; replication/failover, backups, restore and disk guarantees remain deployment qualifications. Runtime settings are checked at transaction entry; an administrator changing server settings concurrently is outside the application trust boundary.
@@ -36,14 +39,14 @@ The MySQL journal uses millisecond timestamps with explicit UTC JDBC calendars,
## Reproducible tests
Use Java 21, the Maven wrapper and the matching Core integration snapshot. Docker is required by the attachment transaction tests. Enable both additional journal-provider fixtures for complete verification:
Use the repository Maven wrapper and the same revision/QQQ dependency overrides as the integrated build. The default suite remains Docker-independent; explicitly enable the provider fixture:
```powershell
.\mvnw.cmd '-DmysqlJournalTests=true' '-DpostgresJournalTests=true' '-Dexec.skip=true' verify
.\mvnw.cmd '-Drevision=0.1.0-record-files-SNAPSHOT' '-DmysqlJournalTests=true' '-Dtest=MySqlUploadJournalTest,UploadJournalTest' test
```
`MySqlUploadJournalTest` creates only its own Testcontainers MySQL 8.4.13 container, random synthetic credentials and private anonymous data volume, and destroys that container on completion. It does not use existing host services. The image is pinned to `mysql@sha256:a0225d1ea6631c8bc2385dc4c2faba17bbf1834ecf369e1659a9a382bd8d5dd7`. Credentials are not printed by the fixture.
The eleven provider cases cover acknowledged prepare surviving a real database process restart, attachment/checkpoint rollback and completion replay, simultaneous prepare/lease fencing, wrong backend/catalog/mapping/ambient/Aurora refusal, actual non-durable server settings and MyISAM refusal, binary identifier comparisons and native write denial, actual lock-timeout/BUSY behavior, column/index drift and immutable conflict, and lease instants across differing JVM default zones, preservation/refusal of host session lock-timeout settings, and refusal of detect-only doublewrite/read-only sessions without changing their settings. The attachment schema in this suite is deliberately a minimal transaction fixture; this is not an end-to-end MySQL object-storage or tenant/archive qualification. Record Files owns its separate provider integration tests.
First provider run compiled but failed its restart fixture because Docker reassigned the dynamic port and Testcontainers retained a cached mapping; two subsequent setup errors followed that stale endpoint. The fixture now refreshes the inspected binding after restart and uses bounded JDBC readiness checks. Docker wire logs are suppressed before container creation. Expanded snapshot verification: **20 tests passed (9 MySQL, 11 H2), zero failures/errors/skips**, `target/mysql-journal-focused-verification.log`, completed 2026-10-31 00:01:44. The tenth provider case and full 214-test owner quality gates subsequently passed; the corrected development-GAV artifact SHA-256 is `72D6E36E77B92BE3ECFE9BE7042D292C76B308AA05A9E43BD490B229BD1A75E2`. Fresh verification on 2026-11-04 passed all 269 tests, including all 11 MySQL journal cases, 12 PostgreSQL journal cases and 29 PostgreSQL attachment transaction cases. Checkstyle and coverage gates passed against Core commit `551397c7c59b0f77833fc81e195f4c01cb1c282f`; no provider cases were skipped.
First provider run compiled but failed its restart fixture because Docker reassigned the dynamic port and Testcontainers retained a cached mapping; two subsequent setup errors followed that stale endpoint. The fixture now refreshes the inspected binding after restart and uses bounded JDBC readiness checks. Docker wire logs are suppressed before container creation. Expanded snapshot verification: **20 tests passed (9 MySQL, 11 H2), zero failures/errors/skips**, `target/mysql-journal-focused-verification.log`, completed 2026-10-31 00:01:44. The tenth provider case and full 214-test owner quality gates subsequently passed; the corrected development-GAV artifact SHA-256 is `72D6E36E77B92BE3ECFE9BE7042D292C76B308AA05A9E43BD490B229BD1A75E2`. The eleventh provider regression and doublewrite/read-only delta await fresh verification.
+76
View File
@@ -0,0 +1,76 @@
# Selectable collaboration fields: published feature snapshot
Tracks [QQQ #1196](https://github.com/QRun-IO/qqq/issues/1196).
This source branch requires the new core virtual-field value-provider API. Its
default BOM pin is now the published
`4.1.0-virtual-field-values-1196-cc3981bb3-SNAPSHOT`. Core publisher job 9716
passed from commit `cc3981bb3c95e8f2dd7ef2a02e2233db7d0e2a79`; an independent
consumer with an initially empty Maven repository downloaded its BOM and core
and compiled the provider interface on October 26, 2026. Both artifacts resolve
to timestamp `20261026.130631-1` from Sonatype's snapshot repository.
Published artifact SHA-256 receipts:
| Artifact | SHA-256 |
| --- | --- |
| BOM POM | `d68cf837d7e07513c4e70977829f99e6236cb3e89f8af85fc75239d5b52564a5` |
| Core POM | `d89d491aaa9312051ec462e1cf165b948ee3e88c9aec2721ff25a00e956c431b` |
| Core JAR | `ae6dbeba8d0e4bc55f2714303ee3482dbf6a0cd34c03663c6009d8fe2c1da54d` |
The collaboration follow-up is published as
`0.1.0-field-values-1196-ecff347-SNAPSHOT`, timestamp
`20261026.133203-1`. Publisher job 168 passed from commit
`ecff347efe2bb84949b8452a3383e3fdeb4bcede`, using signed tag
`publish-values1196-20261026-ecff347`.
| Collaboration artifact | SHA-256 |
| --- | --- |
| POM | `cc12f59cbdfd4bc53ba545a9f2b6d7c14a316034b107ef5541ddc48df45fe6af` |
| JAR | `284fa370504dd82cee8c87d2b8f5c0cecba8c4b15b32a4f9d3d62a0a062e549d` |
On October 26, an independent Java 21 consumer downloaded this pair into an
initially empty Maven repository and compiled both `TagValueProvider` and
`CustomFieldValueProvider` as `VirtualFieldValueProviderInterface` instances.
Maven recorded Sonatype `central-portal-snapshots` as the artifact source. The
downloaded collaboration POM pins the core version above; the independently
downloaded core JAR matches the receipt above. This checks published dependency
resolution and API compatibility; runtime behavior is covered by the 198-test
verification below, not by this compilation probe.
This snapshot does not include the separate attachment, My Work or presence
recovery follow-up PRs. It has not been deployed to Kof22.
Earlier local verification used core artifacts installed as
`4.1.0-values1196-SNAPSHOT`, with
`./mvnw.cmd -Dqqq.snapshot.version=4.1.0-values1196-SNAPSHOT verify`.
That version exists only in the local Maven repository; it is not an adoption pin.
## Behavior
- `collaborationTags` returns sorted, distinct approved labels from global or
matching-table vocabulary. Merged tags and deleted assignments are excluded.
The picker and filters retain their tag IDs; exports contain one Tags column.
- EAV custom fields return typed existing values and ordinary possible-value
display labels. Missing values are null internally and omitted from HTTP value
maps. This change does not enable EAV writes.
- Providers process batches and use the originating transaction. Parent record
authorization and field visibility remain enforced by core. Existing host
post-query customizers remain installed.
- SQL DISTINCT on provider values is explicitly unsupported. Backend primary-key
identity is required; joined heavy primary keys must be fetched explicitly.
## Release sequence
1. Publish and verify the coordinated core artifact containing the provider API (done).
2. Replace this branch's core BOM pin with that published version (done).
3. Run full collaboration verification against the published prerequisite
(done October 26: 198 tests, no failures/errors/skips; packaging and coverage
passed, including HTTP, PostgreSQL transaction and Redis tests).
4. Publish the collaboration snapshot and verify an independent consumer using
published artifacts (done; exact pair recorded in CANDIDATE-RELEASE.md).
5. Update Kof22 Website-Backend adoption work and qualify it in the selected
isolated staging environment before requesting production deployment.
The published pair contains this implementation. Kof22 adoption, a compatible
packaged frontend and staging qualification remain separate release steps.
+56
View File
@@ -0,0 +1,56 @@
# Recovery cost diagnostic — October 26, 2026
One Java 21 process, one Redis connection, disposable loopback Redis 7.4-alpine,
1,000 issued leases, ten viewers per record, no TLS or HTTP. The probe keeps
all leases alive through the real 45-second recovery window and verifies that
the final table snapshot contains all 1,000 viewers. Each phase samples 200
renewals with increasing sequence numbers. This is not a capacity or SLA claim.
| Renewal latency (ms) | During recovery | Ready |
|---|---:|---:|
| p50 | 3.6779 | 0.5134 |
| p95 | 4.1956 | 0.7226 |
| p99 | 4.5447 | 0.9122 |
| Maximum | 6.8271 | 1.4886 |
Population took 4,640.799 ms. The run passed all correctness checks. This first
implementation scanned touched projections on every request during recovery.
After changing table cleanup to occur atomically only when creating its recovery
generation (individual record cleanup remains lazy), the same diagnostic passed:
| Renewal latency (ms) | Optimized recovery | Ready |
|---|---:|---:|
| p50 | 0.6375 | 0.5029 |
| p95 | 0.8150 | 0.5762 |
| p99 | 0.9406 | 0.7500 |
| Maximum | 4.2280 | 1.2251 |
Optimized population took 1,634.486 ms. All 1,000 leases survived. The focused
recovery/store suite also passed 13 tests, including guard loss with previously
unvisited record rooms. Full optimized verification passed 184 tests with no failures/errors/skips and coverage gates passed. These are
individual local runs and need further qualification before claiming supported
production capacity. They do not isolate every source
of latency, compare against an earlier release, or measure sustained concurrent
load, saturation, TLS, HTTP or browser propagation.
At 10,000 entries the optimized diagnostic also passed, retaining all leases.
Population took 10,687.2105 ms; the same 200-sample phases measured:
| Renewal latency (ms) | Optimized recovery, 10,000 entries | Ready |
|---|---:|---:|
| p50 | 0.4978 | 0.5341 |
| p95 | 0.5914 | 0.7108 |
| p99 | 0.7255 | 1.2964 |
| Maximum | 0.7544 | 1.8799 |
Do not infer that larger tables are faster from these separate local runs.
JIT warmup, scheduling and concurrent machine activity were not controlled.
The useful result is that the optimization avoids a repeated whole-table scan
during recovery and both populated-table runs preserved lease correctness.
The explicit `RedisPresenceRecoveryCostProbe` test-source main class is not
automatically run by the unit suite and imposes no machine-dependent CI gate.
After test compilation, run it on the test classpath with argument `1000`.
It fails if the real recovery window ends before the cold measurement, rather
than silently reporting ready-state latency as recovery latency.
@@ -0,0 +1,52 @@
// Real browser recovery against the disposable two-JVM harness. No API mocks.
import { createRequire } from 'node:module'
import { execFileSync } from 'node:child_process'
import path from 'node:path'
const [frontend, first, second, container, output, profile = 'chromium'] = process.argv.slice(2)
const require = createRequire(path.join(frontend, 'package.json'))
const { chromium, firefox, webkit, devices, expect } = require('@playwright/test')
const profiles = {
chromium: [chromium, devices['Desktop Chrome']],
firefox: [firefox, devices['Desktop Firefox']],
webkit: [webkit, devices['Desktop Safari']],
mobile: [chromium, devices['Pixel 7']],
tablet: [webkit, devices['iPad (gen 7)']],
}
if (!profiles[profile]) throw new Error(`Unknown browser profile: ${profile}`)
const [engine, device] = profiles[profile]
const browser = await engine.launch({ headless: true })
const contexts = []
try {
const pages = []
for (const [port, user] of [[first, 'alice'], [second, 'bob']]) {
const context = await browser.newContext(device)
contexts.push(context)
await context.tracing.start({ screenshots: true, snapshots: true })
const page = await context.newPage()
await page.goto(`http://127.0.0.1:${port}/?createMockSession=true&user=${user}`)
// The root export redirects after hydration; finish it before record navigation.
await page.waitForURL(url => /^\/app\/?$/.test(url.pathname), { waitUntil: 'domcontentloaded' })
await page.goto(`http://127.0.0.1:${port}/app/person/1`)
pages.push(page)
}
const unavailable = page => page.getByText('Presence temporarily unavailable', { exact: true })
await expect(pages[0].locator('[data-qqq-id="presence-bob"]')).toBeVisible({ timeout: 90000 })
await expect(pages[1].locator('[data-qqq-id="presence-alice"]')).toBeVisible({ timeout: 90000 })
execFileSync('docker', ['exec', container, 'redis-cli', 'FLUSHDB'])
for (const page of pages) await expect(unavailable(page)).toBeVisible({ timeout: 35000 })
await expect(pages[0].locator('[data-qqq-id="presence-bob"]')).toHaveCount(0)
await expect(pages[1].locator('[data-qqq-id="presence-alice"]')).toHaveCount(0)
await expect(pages[0].locator('[data-qqq-id="presence-bob"]')).toBeVisible({ timeout: 90000 })
await expect(pages[1].locator('[data-qqq-id="presence-alice"]')).toBeVisible({ timeout: 90000 })
for (const [index, page] of pages.entries()) {
await expect(unavailable(page)).toHaveCount(0)
await page.screenshot({ path: path.join(output, `recovered-${index}.png`) })
}
console.log(`PASS: ${profile} shows recovery unavailable, clears stale people, and rejoins both JVMs without reload.`)
} finally {
for (const [index, context] of contexts.entries()) {
await context.tracing.stop({ path: path.join(output, `browser-${index}.zip`) })
}
await browser.close()
}
@@ -0,0 +1,172 @@
"""Exercise Redis loss through two independent SharedPresenceDevServer JVMs.
Run after test-compile and dependency:build-classpath with test scope and output
target/recovery-process-classpath.txt. Uses only disposable loopback fixtures.
"""
import argparse
import hashlib
import json
import os
from pathlib import Path
import socket
import shutil
import subprocess
import time
import urllib.error
import urllib.request
import uuid
def free_port():
"""Reserve a candidate loopback port; startup detects any bind race."""
with socket.socket() as listener:
listener.bind(("127.0.0.1", 0))
return listener.getsockname()[1]
def request(port, user, method, path, body=None):
"""Use the fixture's mock session through the actual HTTP boundary."""
separator = "&" if "?" in path else "?"
url = f"http://127.0.0.1:{port}/qqq/v1/qbit/collaboration{path}{separator}user={user}"
data = None if body is None else json.dumps(body).encode()
req = urllib.request.Request(url, data=data, method=method, headers={
"Cookie": f"sessionUUID={user}-recovery-processes",
"Content-Type": "application/json",
})
try:
response = urllib.request.urlopen(req, timeout=10)
except urllib.error.HTTPError as error:
response = error
with response:
return response.status, json.load(response)
def require(status, result):
"""Fail with fixture-only response evidence when transport differs."""
assert result[0] == status, result
return result[1]
def settle(ports, leases):
"""Renew on the opposite node until both nodes report both people."""
deadline = time.monotonic() + 60
sequence = 0
while time.monotonic() < deadline:
sequence += 1
for index, (user, lease) in enumerate(leases.items()):
result = request(ports[1 - index], user, "POST", "/presence/leases/renew", {
"connectionId": lease, "table": "person", "pk": "1",
"mode": "VIEWING", "sequence": sequence,
})
assert result[0] in (200, 503), result
if result[0] == 503:
assert "recovering" in result[1]["message"], result
snapshots = [request(port, "alice", "GET", "/presence?table=person&pk=1") for port in ports]
if all(status == 200 for status, _ in snapshots):
assert all(len(body["viewers"]) == 2 for _, body in snapshots), snapshots
return
assert all(status == 503 for status, _ in snapshots), snapshots
time.sleep(1)
raise AssertionError("Two-process recovery exceeded 60 seconds")
def main():
"""Own and clean up the exact child JVMs and disposable Redis container."""
parser = argparse.ArgumentParser()
parser.add_argument("--java", required=True)
parser.add_argument("--repo", type=Path, required=True)
parser.add_argument("--frontend-repo", type=Path)
parser.add_argument("--frontend-jar", type=Path, help="Serve this packaged frontend instead of a local export")
parser.add_argument("--backend-jar", type=Path, help="Use this backend artifact instead of target/classes")
parser.add_argument("--browser-profile", choices=("chromium", "firefox", "webkit", "mobile", "tablet"), default="chromium")
args = parser.parse_args()
repo = args.repo.resolve()
output = repo / "target" / ("recovery-processes-" + uuid.uuid4().hex[:10])
output.mkdir()
if args.frontend_jar and not args.frontend_repo:
parser.error("frontend-jar requires frontend-repo for browser tooling")
dependencies = (repo / "target/recovery-process-classpath.txt").read_text().strip().split(os.pathsep)
assert not any(Path(item).name.startswith(("qbit-collaboration-", "qqq-frontend-next-")) for item in dependencies)
backend = args.backend_jar.resolve(strict=True) if args.backend_jar else repo / "target/classes"
classpath = os.pathsep.join([str(repo / "target/test-classes"), str(backend), *dependencies])
evidence = {"status": "FAIL", "browser_profile": args.browser_profile, "cleanup": False}
if args.backend_jar:
evidence["backend_jar_sha256"] = hashlib.sha256(backend.read_bytes()).hexdigest()
if args.frontend_jar:
frontend = args.frontend_jar.resolve(strict=True)
evidence["frontend_jar_sha256"] = hashlib.sha256(frontend.read_bytes()).hexdigest()
classpath = str(frontend) + os.pathsep + classpath
elif args.frontend_repo:
shutil.copytree(args.frontend_repo.resolve() / "out", output / "next-dashboard")
classpath = str(output) + os.pathsep + classpath
container = "qqq-recovery-" + uuid.uuid4().hex[:12]
processes, logs = [], []
try:
subprocess.run(["docker", "run", "-d", "--name", container, "-p", "127.0.0.1::6379", "redis:7.4-alpine"], check=True, capture_output=True)
binding = subprocess.check_output(["docker", "port", container, "6379"], text=True).strip()
redis_port = int(binding.rsplit(":", 1)[1])
ports = [free_port(), free_port()]
for port in ports:
argfile = output / f"node-{port}.args"
argfile.write_text('-Dcollab.dev.frontend=' + str(bool(args.frontend_repo)).lower() + '\n-cp\n"' + classpath.replace("\\", "/") + '"\n'
'com.kingsrook.qbits.collaboration.devserver.SharedPresenceDevServer\n'
f'{port}\n{redis_port}\n', encoding="utf-8")
log = (output / f"node-{port}.log").open("w")
logs.append(log)
processes.append(subprocess.Popen([args.java, "@" + str(argfile)], cwd=repo,
stdout=log, stderr=subprocess.STDOUT,
creationflags=getattr(subprocess, "CREATE_NO_WINDOW", 0)))
deadline = time.monotonic() + 60
for port, process in zip(ports, processes):
while f"SHARED_PRESENCE_READY {port}" not in (output / f"node-{port}.log").read_text(errors="replace"):
assert process.poll() is None, f"Node exited; see {output}"
assert time.monotonic() < deadline, f"Startup timed out; see {output}"
time.sleep(0.2)
if args.frontend_repo:
subprocess.run(["node", str(repo / "docs/acceptance/presence_recovery_browser.mjs"),
str(args.frontend_repo.resolve()), str(ports[0]), str(ports[1]), container, str(output), args.browser_profile], check=True)
evidence["status"] = "PASS"
return
def join():
"""Acquire each person's new lease on a different node."""
return {user: require(200, request(port, user, "POST", "/presence/leases", {
"table": "person", "pk": "1"}))["connectionId"] for port, user in zip(ports, ("alice", "bob"))}
original = join()
settle(ports, original)
subprocess.run(["docker", "exec", container, "redis-cli", "FLUSHDB"], check=True, capture_output=True)
for port in ports:
require(503, request(port, "alice", "GET", "/presence?table=person&pk=1"))
for user, lease in original.items():
error = require(409, request(port, user, "POST", "/presence/leases/renew", {
"connectionId": lease, "table": "person", "pk": "1", "mode": "VIEWING", "sequence": 1000}))
assert error["code"] == "PRESENCE_LEASE_EXPIRED", error
replacement = join()
assert all(replacement[user] != original[user] for user in original)
settle(ports, replacement)
print(f"PASS: two independent JVMs recovered two viewers after Redis loss. Logs: {output}")
evidence["status"] = "PASS"
except Exception as error:
evidence["failure"] = str(error)
raise
finally:
for process in processes:
process.terminate()
for process in processes:
try:
process.wait(timeout=10)
except subprocess.TimeoutExpired:
process.kill()
process.wait(timeout=10)
for log in logs:
log.close()
cleanup = subprocess.run(["docker", "rm", "-f", container], capture_output=True, check=False)
evidence["cleanup"] = cleanup.returncode == 0 and all(process.poll() is not None for process in processes)
(output / "result.json").write_text(json.dumps(evidence, indent=2), encoding="utf-8")
print(output, flush=True)
if evidence["status"] == "PASS" and not evidence["cleanup"]:
raise RuntimeError("Recovery passed but owned fixture cleanup failed")
if __name__ == "__main__":
main()
@@ -0,0 +1,166 @@
"""Restore a disposable Redis image while both fixture JVMs are stopped.
This verifies the controlled-restore protocol, not production orchestration.
Requires the same compiled test classpath as presence_recovery_processes.py.
"""
import argparse
import hashlib
import json
import os
from pathlib import Path
import socket
import subprocess
import time
import uuid
from presence_recovery_processes import free_port, request, require, settle
def main():
"""Own a Redis fixture and two generations of independently running nodes."""
parser = argparse.ArgumentParser()
parser.add_argument("--java", required=True)
parser.add_argument("--repo", type=Path, required=True)
parser.add_argument("--backend-jar", type=Path, help="Verify this artifact without production target/classes")
args = parser.parse_args()
repo = args.repo.resolve()
output = repo / "target" / ("restore-processes-" + uuid.uuid4().hex[:10])
output.mkdir()
dependencies = (repo / "target/recovery-process-classpath.txt").read_text().strip().split(os.pathsep)
assert all(Path(item).is_file() and item.endswith('.jar') for item in dependencies)
assert not any(Path(item).name.startswith("qbit-collaboration-") for item in dependencies)
backend = args.backend_jar.resolve(strict=True) if args.backend_jar else repo / "target/classes"
classpath = os.pathsep.join([str(repo / "target/test-classes"), str(backend), *dependencies])
evidence = {"status": "FAIL", "cleanup": False, "checks": []}
if args.backend_jar:
evidence["backend_jar_sha256"] = hashlib.sha256(backend.read_bytes()).hexdigest()
container = "qqq-restore-" + uuid.uuid4().hex[:12]
container_created = False
processes, logs = [], []
ports = [free_port(), free_port()]
def redis(*command):
"""Run commands only against this owned, disposable Redis container."""
return subprocess.check_output(["docker", "exec", container, "redis-cli", *command], text=True).strip()
def stop_nodes():
"""Require process termination and closed old listeners before restore."""
for process in processes:
if process.poll() is None:
process.terminate()
for process in processes:
try:
process.wait(timeout=10)
except subprocess.TimeoutExpired:
process.kill()
process.wait(timeout=10)
processes.clear()
for port in ports:
with socket.socket() as connection:
connection.settimeout(1)
assert connection.connect_ex(("127.0.0.1", port)) != 0, "Old listener remains open"
def start_nodes(generation, redis_port):
"""Supply the shared generation outside the Redis image, on both JVMs."""
for port in ports:
argfile = output / f"{generation}-{port}.args"
argfile.write_text(f'-Dcollab.dev.recoveryGeneration={generation}\n-cp\n"'
+ classpath.replace("\\", "/") + '"\n'
'com.kingsrook.qbits.collaboration.devserver.SharedPresenceDevServer\n'
f'{port}\n{redis_port}\n', encoding="utf-8")
logfile = output / f"{generation}-{port}.log"
log = logfile.open("w")
logs.append(log)
processes.append(subprocess.Popen([args.java, "@" + str(argfile)], cwd=repo,
stdout=log, stderr=subprocess.STDOUT,
creationflags=getattr(subprocess, "CREATE_NO_WINDOW", 0)))
deadline = time.monotonic() + 60
for port, process in zip(ports, processes):
logfile = output / f"{generation}-{port}.log"
while f"SHARED_PRESENCE_READY {port}" not in logfile.read_text(errors="replace"):
assert process.poll() is None, f"Node exited; see {output}"
assert time.monotonic() < deadline, f"Startup timed out; see {output}"
time.sleep(0.2)
def join():
"""Issue each person's lease on a separate node."""
return {user: require(200, request(port, user, "POST", "/presence/leases", {
"table": "person", "pk": "1"}))["connectionId"] for port, user in zip(ports, ("alice", "bob"))}
def reject_old(leases, sequence):
"""Restored lease authority must fail on both new-generation nodes."""
for port in ports:
for user, lease in leases.items():
error = require(409, request(port, user, "POST", "/presence/leases/renew", {
"connectionId": lease, "table": "person", "pk": "1",
"mode": "VIEWING", "sequence": sequence}))
assert error["code"] == "PRESENCE_LEASE_EXPIRED", error
try:
# DEBUG RELOAD is enabled only on this disposable loopback fixture.
subprocess.run(["docker", "run", "-d", "--name", container, "-p", "127.0.0.1::6379",
"redis:7.4-alpine", "redis-server", "--enable-debug-command", "yes", "--save", ""],
check=True, capture_output=True)
container_created = True
binding = subprocess.check_output(["docker", "port", container, "6379"], text=True).strip()
redis_port = int(binding.rsplit(":", 1)[1])
start_nodes("before-restore", redis_port)
original = join()
settle(ports, original)
last_renewal = time.monotonic()
for port, (user, lease) in zip(ports, original.items()):
require(200, request(port, user, "POST", "/presence/leases/renew", {
"connectionId": lease, "table": "person", "pk": "1",
"mode": "VIEWING", "sequence": 100}))
assert redis("SAVE") == "OK"
before = int(redis("DBSIZE"))
assert before > 0
stop_nodes()
evidence["checks"].append("old JVMs stopped and listeners closed before restore")
assert redis("FLUSHDB") == "OK"
assert redis("DBSIZE") == "0"
assert redis("DEBUG", "RELOAD", "NOSAVE") == "OK"
assert int(redis("DBSIZE")) > 0, "Saved Redis image was not restored"
evidence["checks"].append("nonempty saved Redis image restored")
start_nodes("restored-" + uuid.uuid4().hex, redis_port)
reject_old(original, 1000)
# Avoid a false pass caused only by the old 45-second lease expiring.
evidence["old_lease_rejection_seconds"] = time.monotonic() - last_renewal
assert evidence["old_lease_rejection_seconds"] < 40, "Restore exceeded the live-lease proof window"
evidence["checks"].append("both nodes reject old leases before normal expiry")
for port in ports:
result = require(503, request(port, "alice", "GET", "/presence?table=person&pk=1"))
assert "recovering" in result["message"], result
replacement = join()
assert all(replacement[user] != original[user] for user in original)
settle(ports, replacement)
reject_old(original, 1001)
evidence["checks"].append("both nodes recover with fresh viewers and still reject old leases")
if args.backend_jar:
assert hashlib.sha256(backend.read_bytes()).hexdigest() == evidence["backend_jar_sha256"]
evidence["status"] = "PASS"
print(f"PASS: restored Redis image cannot authorize old leases on either new-generation JVM. Logs: {output}")
except Exception as error:
evidence["failure"] = str(error)
raise
finally:
try:
try:
stop_nodes()
finally:
for log in logs:
log.close()
if container_created:
subprocess.run(["docker", "rm", "-f", container], capture_output=True, check=True)
evidence["cleanup"] = True
except Exception as error:
evidence["status"] = "FAIL"
evidence["cleanup_failure"] = str(error)
raise
finally:
(output / "result.json").write_text(json.dumps(evidence, indent=2), encoding="utf-8")
if __name__ == "__main__":
main()
+264
View File
@@ -0,0 +1,264 @@
# Independent-node presence workload
This fixture targets QQQ #1302/#1291. It runs 2–8 independent JVM HTTP servers
against an owned loopback Redis container. Every node serves the same configured
mock identities across 2–8 tenant scopes. The scope comes from the authenticated
fixture identity, never a supplied tenant parameter. Records are independently
seeded in memory; this does not qualify shared database or production IAM behavior.
Build with Java 21:
```powershell
./mvnw.cmd test-compile dependency:build-classpath '-Dmdep.includeScope=test' '-Dmdep.outputFile=target/capacity-classpath.txt'
python docs/performance/presence_nodes.py --java /path/to/java --nodes 3 --tenants 2 --users 6 --tabs 4 --rounds 100
```
Four tabs per user means two duplicate tabs on each of two records. Every round
renews each issued lease on a rotating node, with sequential per-tab sequences.
All nodes must then return exactly the expected tenant's distinct people and
exactly two editors on both records. Any mismatch or request failure fails the
run. Requests use bounded worker concurrency and ten-second timeouts. Each node
has eight pooled Redis connections, a 500 ms pool wait and a 384 MiB heap ceiling.
Results and per-node logs are retained under `target/capacity-nodes-<id>`.
The result records parameters, node PIDs, elapsed time, sample counts and
nearest-rank HTTP p50/p95/p99/max. Renewal throughput includes snapshot checks
and is not maximum server throughput. All owned JVMs and the Redis container are
removed in `finally`; the receipt reports cleanup separately from test success.
Each fixture JVM emits resource samples every 100 ms using Java management APIs
and the Redis QBit's existing pool metrics. The receipt summarizes sampled heap
used/committed bytes, active/idle/waiting pool counts and the configured maximum.
CPU core equivalents are process CPU time divided by observed wall time; they
can exceed one for a multithreaded JVM. Observation begins after server startup
and includes lease setup and workload execution. Unsupported CPU counters are
reported as null. Linux fixtures also read process `VmRSS` from `/proc/self/status`
every 100 ms and record `residentBytes_sampled_max`; unsupported platforms report
null. Linux-container runs require resident-memory samples. Heap is not total
resident memory, and process RSS is not the container's complete cgroup footprint.
Sampled peaks may miss
short bursts and must not be represented as exact maxima. Raw samples remain in
node logs; missing node samples or a sampled pool-bound violation fail the run.
The final Redis INFO capture records `used_memory`, lifetime `used_memory_peak`,
and current `used_memory_rss`, plus process user/system CPU seconds since Redis
startup. Redis RSS is an end-of-run observation, not a sampled peak. These fresh,
owned Redis instances include fixture setup and diagnostics in their lifetime;
do not interpret those CPU values as workload-only measurements or add independent
process peaks and call the sum a measured simultaneous maximum. Missing Redis
memory/CPU diagnostics fail acceptance while cleanup still runs.
The [100-user resource run](results/linux-100-users-resources.json) used three
Linux JVMs, two tenants with 50 people each, four leases per person, 16 request
workers and 200 rounds. All 80,000 renewals and 2,400 exact-membership snapshots
passed in 102.568 workload seconds. Renewal p50/p95/p99/max were
11.326/21.765/25.881/1,007.404 ms; snapshot p99 was 19.424 ms. The one-second
renewal maximum is retained, not discarded. Cleanup passed.
Sampled JVM RSS peaks were 662,786,048 / 662,188,032 / 703,479,808 bytes
(approximately 632–671 MiB), while used-heap peaks were about 171–177 MiB.
CPU observations were 1.73–1.83 core equivalents per JVM; sampled pool-active
peaks were 7/7/6 of eight with no sampled waiters. Redis peak allocated memory
was 2,579,896 bytes; final RSS was 8,777,728 bytes. Redis process CPU since
startup was 23.738 system plus 35.526 user seconds. These measurements include
fixture overhead, omit browser load, and do not establish deployment limits.
Initial smoke run: three JVMs, two tenants, six users each, four tabs per user,
three rounds: 144 renewals and 36 snapshots passed with tenant isolation,
duplicate-tab deduplication and two editors retained. Cleanup passed. This short
run validates the harness, not sustained capacity or operational thresholds.
A second run with 12 users per tenant and 100 rounds passed 9,600 renewals and
1,200 snapshots in 23.110 seconds. Renewal p50/p95/p99 were 7.246/16.454/21.853 ms;
snapshot p50/p95/p99 were 12.908/15.776/18.239 ms. Every round checked exact
tenant membership, deduplication and two editors on both records at every node.
Cleanup passed. These are measurements from one local Windows/Docker run;
they do not establish deployment limits or browser propagation latency.
## Sustained-run limitation under investigation
The 1,000-round workload has intermittently failed with HTTP 503 on lease
renewal. Four runs failed; one completed 96,000 renewals and 12,000 snapshots.
The passing run does not resolve the failures. Temporary local diagnostics on
the third failure identified `JedisConnectionException` caused by
`SocketTimeoutException` while reading the `EVALSHA` response in
`RedisPresenceStore.mutate`. The configured socket timeout remained two seconds.
The temporary production-file diagnostics were removed after identifying this
path. No timeout increase, hidden request retry or production fix is included.
Resource samples did not show pool waiters, but cannot rule out short bursts.
Redis slow-command timings and latency events are now captured before fixture
cleanup, alongside Redis CPU, memory and command counters, to distinguish server
execution cost from transport or scheduling delays. Slow-command summaries omit
command arguments. These diagnostics belong only to the disposable fixture.
Until the cause is established and qualified, this workload establishes no
supported sustained throughput or production alert thresholds.
The timing-instrumented Windows-JVM run failed at round 480, with no commands
at or above the configured one-millisecond Redis slow-log threshold and no
latency-monitor events. Redis reported about 93 microseconds per `EVALSHA` on
average. This weakens a slow-Lua explanation but does not identify the delayed
component or exclude time spent outside command execution.
## Linux comparison
`--node-image eclipse-temurin:21-jre` optionally runs each JVM in its own Linux
container on an owned Docker network shared with Redis. The HTTP driver remains
on the host. Only loopback HTTP ports are published. Compiled fixture classes
and copied dependency JARs are mounted read-only; Redis remains disposable.
The existing loopback defaults remain unchanged when this option is absent.
Container mode records Docker client PIDs separately from container identities.
It removes the Windows-JVM-to-published-Redis-port path, but also changes the JVM
platform and scheduling environment; it is a diagnostic comparison, not proof
of a specific operating-system or Docker defect. Cleanup includes every owned
node container and the owned network.
`--churn-every 20` departs all tabs belonging to the last user in each tenant
every 20 rounds. Every departed lease must reject a later renewal with HTTP 409
and `PRESENCE_LEASE_EXPIRED` on another node. All surviving nodes must show that
user absent on both records, then show the original exact membership after
fresh, distinct leases are acquired and activated. Only this explicit rejoin
path acquires new leases; the node-crash path preserves existing ones.
`--pause-redis` adds a controlled failure phase after the ordinary rounds. It
freezes only the owned Redis container, sends up to 64 concurrent renewal
requests, and requires HTTP 503 within an eight-second fixture bound. Snapshots
must also fail explicitly rather than return an empty room. Redis is unpaused
in `finally`; once reachable, higher-sequence renewals must recover the same
leases and exact membership. The test requires observed pool waiters and rejects
sampled active counts above the configured maximum. It does not retry failed
requests invisibly or raise production timeouts.
The [Linux Redis-stall receipt](results/linux-redis-stall.json) passed with three
nodes, two tenants, 24 users and 96 tab leases. All 96 stalled renewals and all
three node snapshots returned the expected unavailable response. Existing
leases recovered after Redis resumed, and cleanup passed. This proves bounded
failure and recovery for an injected stall; it does not explain the intermittent
Windows timeout or establish a saturation throughput threshold.
```powershell
python docs/performance/presence_nodes.py --java java --node-image eclipse-temurin:21-jre --nodes 3 --tenants 2 --users 12 --tabs 4 --rounds 1000
```
The first Linux comparison completed 1,000 rounds in 250.811 seconds: 96,000
renewals and 12,000 snapshots, with exact membership/editor assertions and
cleanup passing. Renewal p95/p99 were 15.331/17.760 ms. The sampled active pool
peak was five of eight, with no sampled waiters. This one run is separate from
the unresolved Windows failures and does not establish a production SLA.
`--crash-round 50` abruptly kills the first node before that round, then sends
all existing leases through surviving nodes. It does not acquire replacement
leases or hide failed requests with retries. Exact tenant membership, duplicate
tabs and editor counts must remain correct on every surviving node. This tests
HTTP lease continuity after process loss, not browser/SSE reconnect, ingress
failover, database durability or a Redis outage.
The Linux crash run passed 100 rounds (9,600 renewals and 996 snapshots) in
25.148 seconds. The first JVM exited with code 137 at round 50; the remaining
two retained all issued leases and expected membership. Cleanup passed.
A [combined churn and node-loss run](results/linux-node-crash-churn.json) passed
100 rounds with the first node killed at round 50 and churn every 20 rounds.
Five cycles rejected all 40 departed handles and checked absence/restored
membership across both tenants and records on every surviving node. Cleanup
passed. This is bounded HTTP lifecycle qualification; browser reconnect and
missed-event behavior remain separate requirements.
Recorded receipts retain both successful and failing evidence:
- [Windows timeout with Redis diagnostics](results/windows-timeout.json)
- [Linux 1,000-round run](results/linux-1000-rounds.json)
- [Linux node crash](results/linux-node-crash.json)
Only the Java executable path is normalized in these receipts. The Linux image
used for these runs was
`eclipse-temurin@sha256:227118764cccd3c3294617061aaa93a08309c63a395eefa83e6b35032992ea99`.
## Browser propagation baseline
`browser_subscription_affinity.mjs FRONTEND_REPO BASE_URL RESULT_PATH` verifies
the current owning-node subscription contract against the two-node fixture. It
uses the connection ID from the real browser's successful subscription request,
then preserves the same session cookies while changing only the test routing
cookie. The [published-artifact receipt](results/browser-subscription-affinity.json)
records owner 200, other node 404, owner 200, with the requested channel active
on both successful responses and the unsaved draft retained. This proves why
subscription requests need owning-node routing even when presence state is shared.
It does not test the deployed Traefik cookie, HTTPS, buffering or failover.
The packaged runner accepts this script through `--affinity-script`.
`browser_presence_latency.mjs` measures Chromium field-focus changes until a
different editor sees the corresponding remote marker. Supply a caller-owned
two-node loopback fixture serving the published frontend with `alice`/`admin`
mock identities and the acceptance router's `presence_node=first/second`
affinity cookies:
```powershell
node docs/performance/browser_presence_latency.mjs /path/to/frontend http://localhost:PORT result.json 25
```
Use `presence_router.mjs FIRST_NODE_PORT SECOND_NODE_PORT ROUTER_PORT` for this
test. It binds only to loopback and adds a test-only POST endpoint that severs
first-node SSE responses. Never deploy this fault-injection router. The packaged
acceptance fixture accepts it through `--router-script` alongside `--latency-script`.
The test verifies the target marker is absent before every change, the old
marker disappears afterward, both real SSE streams opened successfully,
issued-lease writes reached distinct nodes, and unsaved drafts survive. API
responses are not mocked. The fixture owns backend/router/Redis cleanup; this
script owns and closes its browser.
After timing the normal transitions, the script takes the observer offline and
explicitly severs its existing SSE response through the fixture router. It requires
an observed stream failure, a failed offline fetch, three acknowledged sender focus
changes, and absence of the new marker while disconnected. After network restoration
it requires a new successful SSE response and the current marker with the previous
marker removed, without reload or loss of either unsaved draft. Recovery may use
the shipped reconnect and polling behavior; this does not prove event replay.
The [disconnect/recovery receipt](results/browser-reconnect.json) passed against
the public RC37 JAR: one stream severed, three missed changes, recovery in 1,858 ms,
both drafts preserved. The accompanying six-user and Redis admin checks passed;
fixture cleanup passed. This is one Chromium disconnect case without concurrent
load, not a reconnect storm, lease-expiry test, or production recovery objective.
Append `load` to the browser command (or use `--latency-load` on the packaged
fixture) to add 16 concurrent HTTP lease workers: four mock identities with four
leases each on the same record, split between both nodes. Each worker renews
sequentially with a 100 ms pause after each response. The browser observations
are spread over at least 60 seconds; every observation interval must include
completed background renewals. The workload continues during disconnect/reconnect.
Any request or lease-cleanup failure fails the test. Responses are disposed after
reading, and workers stop before the contexts close. The receipt reports completed
renewals, observed node counts, elapsed time and any errors. These are HTTP load
workers, not 16 additional browser tabs or distinct people.
The [bounded load/reconnect receipt](results/browser-load-reconnect.json) passed
9,046 renewals over 69.728 seconds, with no worker or lease-release errors.
Across 25 observed focus transitions, p50/p95/p99 were 32.384/85.391/785.495 ms;
p99 is the maximum of this small sample, not a reliable tail-latency estimate.
The observer recovered in 1,855.887 ms after missing three updates while load
continued. Both drafts survived. Six-user and Redis admin acceptance also passed
against the public RC37 JAR; fixture cleanup passed. An initial run failed only
because the new load helper used the wrong lease-release endpoint; that receipt
is retained in the local packaged fixture history, not counted as a passing run.
The [25-transition receipt](results/browser-focus-25.json) passed against the
downloaded public RC37 JAR and published backend snapshots. Focus-to-observed-DOM
p50/p95/p99 were 31.648/33.078/36.092 ms; p99 is the maximum in this small sample.
There were no page errors or failed lease responses. Six-user presence and Redis
admin browser checks also passed in the same fixture run; fixture cleanup passed.
This baseline includes browser automation, UI debounce, network, invalidation
and rendering. It is not a server-only transport measurement, an SSE-only causal
proof, an under-load result, or a production latency objective. The separate load
receipt above covers one bounded workload and disconnect scenario; larger loads,
reconnect storms and deployment-specific behavior remain unqualified.
Remaining qualification includes longer soak and larger workloads, container
resource limits, reconnect storms and the actual deployment's capacity and alert
thresholds. The receipts above cover specific bounded HTTP, browser, saturation,
churn, missed-update and process-crash cases, not every combination. The native
Windows intermittent Redis read timeout remains unresolved. Loopback HTTP latency must not be reported
as browser propagation or a production SLA. No deployed capacity threshold is
established by this fixture.
@@ -0,0 +1,145 @@
// Real UI observations against a caller-owned two-node acceptance fixture.
import { createRequire } from 'node:module';
import { resolve } from 'node:path';
import { writeFile } from 'node:fs/promises';
import { performance } from 'node:perf_hooks';
import { setTimeout as delay } from 'node:timers/promises';
import { startPresenceLoad } from './browser_presence_load.mjs';
const [frontendRepo, base, output, countValue = '25', loadMode] = process.argv.slice(2);
if (loadMode && loadMode !== 'load') throw new Error('Optional final argument must be load');
const samples = Number(countValue);
if (!frontendRepo || !base || !output || !Number.isInteger(samples) || samples < 5 || samples > 100) {
throw new Error('Expected frontend repo, loopback base URL, result path and 5..100 samples');
}
if (!['localhost', '127.0.0.1'].includes(new URL(base).hostname)) {
throw new Error('Only the disposable loopback fixture is supported');
}
const require = createRequire(resolve(frontendRepo, 'package.json'));
const { chromium, expect } = require('@playwright/test');
const result = { status: 'FAIL', samples: [], streams: [], requests: [], errors: [],
scope: 'Chromium focus-to-remote-DOM observation, including automation and UI debounce; no concurrent load' };
const browser = await chromium.launch({ headless: true });
let load;
try {
async function editor(user, node) {
const context = await browser.newContext();
await context.addCookies([
{ name: 'collab_dev_user', value: user, url: base },
{ name: 'presence_node', value: node, url: base },
]);
const page = await context.newPage();
page.setDefaultTimeout(15000);
page.on('pageerror', error => result.errors.push(error.message));
page.on('response', response => {
const path = new URL(response.url()).pathname;
if (path.endsWith('/realtime/stream')) {
result.streams.push({ user, status: response.status(), type: response.headers()['content-type'] });
}
if (path.includes('/presence/leases')) {
result.requests.push({ user, status: response.status(), node: response.headers()['x-acceptance-node'] });
}
});
await page.goto(`${base}/app/person/1/edit?createMockSession=true`);
await expect(page.locator('[data-qqq-id="firstName"]')).toBeVisible();
return page;
}
const observer = await editor('alice', 'first');
const sender = await editor('admin', 'second');
await observer.locator('[data-qqq-id="firstName"]').fill('Retained observer draft');
await sender.locator('[data-qqq-id="firstName"]').fill('Retained sender draft');
await expect(observer.locator('[data-presence-field="firstName"]')).toContainText('Demo Administrator');
if (loadMode) {
load = await startPresenceLoad(browser, base, expect);
result.load = load.state;
result.scope = 'Chromium focus-to-remote-DOM and reconnect under 16 concurrent HTTP lease workers; loopback, four background identities';
}
for (let index = 0; index < samples; index++) {
const previousCompleted = load?.state.completed;
if (load) await delay(60000 / samples);
const field = index % 2 === 0 ? 'lastName' : 'firstName';
const previous = field === 'firstName' ? 'lastName' : 'firstName';
await expect(observer.locator(`[data-presence-field="${field}"]`)).not.toContainText('Demo Administrator');
const before = performance.now();
await sender.locator(`[data-qqq-id="${field}"]`).focus();
await expect(observer.locator(`[data-presence-field="${field}"]`)).toContainText('Demo Administrator');
result.samples.push({ field, propagation_ms: performance.now() - before });
await expect(observer.locator(`[data-presence-field="${previous}"]`)).not.toContainText('Demo Administrator');
if (load) {
expect(load.state.errors).toEqual([]);
expect(load.state.completed).toBeGreaterThan(previousCompleted);
}
}
// Drop a real browser connection, miss multiple focus updates, then converge
// through the shipped reconnect/polling behavior without reloading the form.
const oldField = samples % 2 === 0 ? 'firstName' : 'lastName';
const newField = oldField === 'firstName' ? 'lastName' : 'firstName';
const streamsBefore = result.streams.filter(stream => stream.user === 'alice').length;
result.reconnect = { status: 'FAIL', missed_changes: 0 };
const interrupted = observer.waitForEvent('requestfailed', {
predicate: request => new URL(request.url()).pathname.endsWith('/realtime/stream'),
timeout: 10000,
});
await observer.context().setOffline(true);
const disconnect = await fetch(`${base}/__acceptance/disconnect-first`, { method: 'POST' });
expect(disconnect.status).toBe(200);
result.reconnect.disconnected_streams = (await disconnect.json()).disconnected;
expect(result.reconnect.disconnected_streams).toBeGreaterThan(0);
const failedStream = await interrupted;
result.reconnect.stream_failure = failedStream.failure()?.errorText;
const offlineFetch = await observer.evaluate(async () => {
try {
await fetch(`/app/?offlineProbe=${Date.now()}`, { cache: 'no-store' });
return false;
} catch { return true; }
});
expect(offlineFetch).toBe(true);
for (const field of [newField, oldField, newField]) {
const renewed = sender.waitForResponse(response =>
new URL(response.url()).pathname.endsWith('/presence/leases/renew') &&
response.request().method() === 'POST' && response.status() === 200 &&
response.request().postDataJSON()?.fieldName === field);
await sender.locator(`[data-qqq-id="${field}"]`).focus();
await renewed;
result.reconnect.missed_changes++;
}
await expect(observer.locator(`[data-presence-field="${newField}"]`)).not.toContainText('Demo Administrator');
const resumed = performance.now();
await observer.context().setOffline(false);
await expect.poll(() => result.streams.filter(stream => stream.user === 'alice' &&
stream.status === 200 && stream.type?.includes('text/event-stream')).length,
{ timeout: 30000 }).toBeGreaterThan(streamsBefore);
await expect(observer.locator(`[data-presence-field="${newField}"]`)).toContainText('Demo Administrator', { timeout: 30000 });
await expect(observer.locator(`[data-presence-field="${oldField}"]`)).not.toContainText('Demo Administrator');
result.reconnect.recovery_ms = performance.now() - resumed;
result.reconnect.status = 'PASS';
if (load) {
await load.stop();
expect(load.state.errors).toEqual([]);
expect(load.state.elapsed_ms).toBeGreaterThanOrEqual(60000);
expect(load.state.nodes.first).toBeGreaterThan(0);
expect(load.state.nodes.second).toBeGreaterThan(0);
load = undefined;
}
await expect(observer.locator('[data-qqq-id="firstName"]')).toHaveValue('Retained observer draft');
await expect(sender.locator('[data-qqq-id="firstName"]')).toHaveValue('Retained sender draft');
for (const user of ['alice', 'admin']) {
expect(result.streams.some(stream => stream.user === user && stream.status === 200 && stream.type?.includes('text/event-stream'))).toBe(true);
}
expect(result.requests.some(request => request.user === 'alice' && request.node === 'first')).toBe(true);
expect(result.requests.some(request => request.user === 'admin' && request.node === 'second')).toBe(true);
expect(result.requests.every(request => request.status === 200)).toBe(true);
expect(result.errors).toEqual([]);
const ordered = result.samples.map(sample => sample.propagation_ms).sort((a, b) => a - b);
result.percentiles_ms = Object.fromEntries([['p50', .5], ['p95', .95], ['p99', .99], ['max', 1]]
.map(([name, quantile]) => [name, ordered[Math.ceil(ordered.length * quantile) - 1]]));
result.status = 'PASS';
} catch (error) {
result.failure = String(error);
process.exitCode = 1;
} finally {
if (load) await load.stop();
await browser.close();
await writeFile(output, JSON.stringify(result, null, 2));
console.log(JSON.stringify(result, null, 2));
}
@@ -0,0 +1,75 @@
// Bounded real HTTP load using caller-owned authenticated browser contexts.
import { performance } from 'node:perf_hooks';
import { setTimeout as delay } from 'node:timers/promises';
export async function startPresenceLoad(browser, base, expect) {
const state = { workers: 16, completed: 0, errors: [], nodes: {}, elapsed_ms: 0 };
const contexts = [];
const leases = [];
let running = true;
let stopped = false;
let tasks = [];
const began = performance.now();
const endpoint = `${base}/qqq/v1/qbit/collaboration/presence/leases`;
async function post(context, path, data) {
const response = await context.request.post(endpoint + path, { data, timeout: 10000 });
try {
expect(response.status()).toBe(200);
const node = response.headers()['x-acceptance-node'];
state.nodes[node] = (state.nodes[node] ?? 0) + 1;
return await response.json();
} finally { await response.dispose(); }
}
async function stop() {
if (stopped) return;
stopped = true;
running = false;
await Promise.all(tasks);
state.elapsed_ms = performance.now() - began;
for (const lease of leases) {
try {
const response = await lease.context.request.delete(`${base}/qqq/v1/qbit/collaboration/presence`, {
data: { ...lease.target, connectionId: lease.id, sequence: ++lease.sequence }, timeout: 10000,
});
try { expect(response.status()).toBe(200); } finally { await response.dispose(); }
} catch (error) { state.errors.push(String(error)); }
}
for (const context of contexts) await context.close();
}
try {
for (const [index, user] of ['bob', 'carol', 'dave', 'erin'].entries()) {
const context = await browser.newContext();
contexts.push(context);
await context.addCookies([
{ name: 'collab_dev_user', value: user, url: base },
{ name: 'presence_node', value: index % 2 ? 'second' : 'first', url: base },
]);
const page = await context.newPage();
await page.goto(`${base}/app/person/1?createMockSession=true`);
await expect(page.locator('[data-qqq-id="record-view-title-person"]')).toBeVisible();
await page.close();
for (let tab = 0; tab < 4; tab++) {
const target = { table: 'person', pk: '1' };
const opened = await post(context, '', target);
expect(typeof opened.connectionId).toBe('string');
leases.push({ context, target, id: opened.connectionId, sequence: 0, completed: 0 });
}
}
tasks = leases.map(async lease => {
try {
while (running) {
await post(lease.context, '/renew', { ...lease.target, connectionId: lease.id,
sequence: ++lease.sequence, mode: 'VIEWING' });
lease.completed++;
state.completed++;
await delay(100);
}
} catch (error) { state.errors.push(String(error)); running = false; }
});
await expect.poll(() => leases.filter(lease => lease.completed > 0).length).toBe(16);
return { state, stop };
} catch (error) {
await stop();
throw error;
}
}
@@ -0,0 +1,60 @@
// Verify owning-node routing with real published UI and backend artifacts.
import { createRequire } from 'node:module';
import { resolve } from 'node:path';
import { writeFile } from 'node:fs/promises';
const [frontendRepo, base, output] = process.argv.slice(2);
if (!frontendRepo || !output || !['localhost', '127.0.0.1'].includes(new URL(base).hostname)) {
throw new Error('Expected frontend repo, owned loopback URL and result path');
}
const require = createRequire(resolve(frontendRepo, 'package.json'));
const { chromium, expect } = require('@playwright/test');
const result = { status: 'FAIL', checks: [], errors: [] };
const browser = await chromium.launch({ headless: true });
try {
const context = await browser.newContext();
await context.addCookies([
{ name: 'collab_dev_user', value: 'alice', url: base },
{ name: 'presence_node', value: 'first', url: base },
]);
const page = await context.newPage();
page.on('pageerror', error => result.errors.push(error.message));
const subscribed = page.waitForResponse(response =>
new URL(response.url()).pathname.endsWith('/realtime/subscriptions') && response.status() === 200);
await page.goto(`${base}/app/person/1/edit?createMockSession=true`);
await expect(page.locator('[data-qqq-id="firstName"]')).toBeVisible();
await page.locator('[data-qqq-id="firstName"]').fill('Keep this unsaved draft');
const response = await subscribed;
expect(response.headers()['x-acceptance-node']).toBe('first');
const { connectionId } = response.request().postDataJSON();
expect(typeof connectionId).toBe('string');
for (const [node, expected] of [['first', 200], ['second', 404], ['first', 200]]) {
// Keep the same session/user cookies and change only the test routing cookie.
const cookies = (await context.cookies(base)).map(cookie =>
`${cookie.name}=${cookie.name === 'presence_node' ? node : cookie.value}`).join('; ');
const changed = await context.request.post(`${base}/qqq/v1/realtime/subscriptions`, {
headers: { Cookie: cookies },
data: { connectionId, add: ['record:person:1'], remove: [] }, timeout: 10000,
});
try {
result.checks.push({ node: changed.headers()['x-acceptance-node'], status: changed.status() });
expect(changed.headers()['x-acceptance-node']).toBe(node);
expect(changed.status()).toBe(expected);
if (expected === 200) {
const body = await changed.json();
expect(body.active).toContain('record:person:1');
expect(body.refused ?? []).toEqual([]);
}
} finally { await changed.dispose(); }
}
await expect(page.locator('[data-qqq-id="firstName"]')).toHaveValue('Keep this unsaved draft');
expect(result.errors).toEqual([]);
result.status = 'PASS';
} catch (error) {
result.failure = String(error);
process.exitCode = 1;
} finally {
await browser.close();
await writeFile(output, JSON.stringify(result, null, 2));
console.log(JSON.stringify(result, null, 2));
}
+343
View File
@@ -0,0 +1,343 @@
"""Bounded HTTP presence workload across independent, disposable JVMs.
This measures loopback HTTP service latency, not browser propagation or a
production capacity limit. Run test-compile and dependency:build-classpath with
test scope and output target/capacity-classpath.txt before invoking this script.
"""
import argparse
from concurrent.futures import ThreadPoolExecutor
import json
import math
import os
from pathlib import Path
import shutil
import socket
import subprocess
import sys
import time
import urllib.error
import urllib.request
import uuid
def port():
"""Choose a loopback port; child startup detects a competing bind."""
with socket.socket() as listener:
listener.bind(("127.0.0.1", 0))
return listener.getsockname()[1]
def request(node, user, path, body=None, tab=0, method=None, expected=200):
"""Use real HTTP and the explicit mock identity; no tenant parameter."""
separator = "&" if "?" in path else "?"
req = urllib.request.Request(
f"http://127.0.0.1:{node}/qqq/v1/qbit/collaboration{path}{separator}user={user}",
data=None if body is None else json.dumps(body).encode(),
method=method,
headers={"Content-Type": "application/json",
"Cookie": f"sessionUUID={user}-capacity-{tab}"})
try:
response = urllib.request.urlopen(req, timeout=10)
except urllib.error.HTTPError as error:
if error.code == expected:
with error:
return json.load(error)
with error:
detail = error.read(4096).decode(errors="replace")
raise RuntimeError(f"HTTP {error.code} node={node} user={user} path={path}: {detail}") from error
with response:
if response.status != expected:
raise AssertionError(f"Unexpected status {response.status}")
return json.load(response)
def distribution(values):
"""Nearest-rank milliseconds, with no discarded slow or failed samples."""
ordered = sorted(values)
return {"samples": len(ordered), **{
name: ordered[max(0, math.ceil(len(ordered) * fraction) - 1)]
for name, fraction in (("p50_ms", .5), ("p95_ms", .95), ("p99_ms", .99), ("max_ms", 1))}}
def main():
"""Own child JVMs/container and retain results even when a check fails."""
parser = argparse.ArgumentParser()
parser.add_argument("--java", required=True)
parser.add_argument("--node-image", help="Optional Linux Java21 image; isolate Redis traffic on an owned Docker network")
parser.add_argument("--nodes", type=int, default=3)
parser.add_argument("--tenants", type=int, default=2)
parser.add_argument("--users", type=int, default=6)
parser.add_argument("--tabs", type=int, default=4)
parser.add_argument("--rounds", type=int, default=10)
parser.add_argument("--workers", type=int, default=8)
parser.add_argument("--crash-round", type=int, default=0, help="Kill the first node before this round; zero disables")
parser.add_argument("--churn-every", type=int, default=0, help="Leave/rejoin the last user's tabs every N rounds; zero disables")
parser.add_argument("--pause-redis", action="store_true", help="After the workload, freeze owned Redis and verify bounded failures/recovery")
args = parser.parse_args()
for name, lower, upper in (("nodes", 2, 8), ("tenants", 2, 8), ("users", 2, 100),
("tabs", 2, 8), ("rounds", 1, 1000), ("workers", 1, 64)):
if not lower <= getattr(args, name) <= upper:
parser.error(f"{name} must be {lower}..{upper}")
if args.crash_round and not 2 <= args.crash_round <= args.rounds:
parser.error("crash-round must be zero or 2..rounds")
if args.churn_every and not 1 <= args.churn_every <= args.rounds:
parser.error("churn-every must be zero or 1..rounds")
repo = Path(__file__).resolve().parents[2]
output = repo / "target" / ("capacity-nodes-" + uuid.uuid4().hex[:10])
output.mkdir(parents=True)
classpath = os.pathsep.join([str(repo / "target/test-classes"), str(repo / "target/classes"),
(repo / "target/capacity-classpath.txt").read_text().strip()])
processes, logs = [], []
container = "qqq-capacity-" + uuid.uuid4().hex[:12]
network = container + "-net"
node_containers = []
network_created = False
result = {"status": "FAIL", "parameters": vars(args), "scope": "loopback HTTP, mock identities, memory records"}
try:
network_args = []
if args.node_image:
result["node_image_id"] = subprocess.check_output(
["docker", "image", "inspect", args.node_image, "--format", "{{.Id}}"], text=True).strip()
subprocess.run(["docker", "network", "create", network], check=True, capture_output=True)
network_created = True
network_args = ["--network", network]
dependencies = output / "dependencies"
dependencies.mkdir()
linux_classpath = ["/fixture/target/test-classes", "/fixture/target/classes"]
for index, dependency in enumerate((repo / "target/capacity-classpath.txt").read_text().strip().split(os.pathsep)):
destination = dependencies / f"{index}.jar"
shutil.copyfile(dependency, destination)
linux_classpath.append("/fixture/" + destination.relative_to(repo).as_posix())
classpath = ":".join(linux_classpath)
subprocess.run(["docker", "run", "-d", "--name", container, "-p", "127.0.0.1::6379",
*network_args,
"redis:7.4-alpine", "redis-server", "--slowlog-log-slower-than", "1000",
"--slowlog-max-len", "128", "--latency-monitor-threshold", "1"], check=True, capture_output=True)
redis_port = int(subprocess.check_output(["docker", "port", container, "6379"], text=True).strip().rsplit(":", 1)[1])
nodes = [port() for _ in range(args.nodes)]
if len(set(nodes)) != len(nodes):
raise RuntimeError("Port allocation collision; no nodes started")
for node in nodes:
argfile = output / f"node-{node}.args"
connection_properties = (f"-Dcollab.capacity.redisHost={container}\n-Dcollab.dev.bindAddress=0.0.0.0\n"
if args.node_image else "")
argfile.write_text(f'-Xmx384m\n-Dcollab.capacity.tenants={args.tenants}\n-Dcollab.capacity.users={args.users}\n'
+ connection_properties +
'-cp\n"' + classpath.replace("\\", "/") + '"\n'
'com.kingsrook.qbits.collaboration.devserver.CapacityPresenceDevServer\n'
+ (f'8080\n6379\n' if args.node_image else f'{node}\n{redis_port}\n'), encoding="utf-8")
log = (output / f"node-{node}.log").open("w")
logs.append(log)
command = [args.java, "@" + str(argfile)]
if args.node_image:
node_container = f"{container}-node-{node}"
node_containers.append(node_container)
command = ["docker", "run", "--rm", "--name", node_container, "--network", network,
"-p", f"127.0.0.1:{node}:8080", "--mount", f"type=bind,source={repo},target=/fixture,readonly",
args.node_image, "java", "@/fixture/" + argfile.relative_to(repo).as_posix()]
processes.append(subprocess.Popen(command, cwd=repo,
stdout=log, stderr=subprocess.STDOUT,
creationflags=getattr(subprocess, "CREATE_NO_WINDOW", 0)))
deadline = time.monotonic() + 120
for node, process in zip(nodes, processes):
while "CAPACITY_PRESENCE_READY " not in (output / f"node-{node}.log").read_text(errors="replace"):
if process.poll() is not None or time.monotonic() > deadline:
raise RuntimeError(f"Node {node} failed startup; inspect {output}")
time.sleep(.2)
actors = []
for tenant in range(args.tenants):
for user in range(args.users):
for tab in range(args.tabs):
identity = f"load-t{tenant}-u{user}"
record = str(1 + tab % 2)
node = nodes[len(actors) % len(nodes)]
lease = request(node, identity, "/presence/leases", {"table": "person", "pk": record}, tab)["connectionId"]
actors.append((identity, record, tab, lease, user))
renewals, snapshots = [], []
def check_rooms(excluded=None):
"""Check every tenant/record on every surviving node, including absence."""
for tenant in range(args.tenants):
expected_users = {f"load-t{tenant}-u{user}" for user in range(args.users) if user != excluded}
for node in nodes:
for record in ("1", "2"):
before = time.perf_counter()
viewers = request(node, f"load-t{tenant}-u0", f"/presence?table=person&pk={record}")["viewers"]
snapshots.append((time.perf_counter() - before) * 1000)
actual = {viewer["userId"] for viewer in viewers}
if actual != expected_users or len(viewers) != len(expected_users):
raise AssertionError(f"Tenant isolation/deduplication mismatch: {actual} != {expected_users}")
if sum(viewer["mode"] == "EDITING" for viewer in viewers) != sum(user != excluded for user in range(2)):
raise AssertionError("Unexpected editor count")
def renew(item, sequence):
"""Keep each tab sequential while moving its writes between nodes."""
index, (identity, record, tab, lease, user) = item
before = time.perf_counter()
request(nodes[(index + sequence) % len(nodes)], identity, "/presence/leases/renew",
{"connectionId": lease, "table": "person", "pk": record,
"mode": "EDITING" if user < 2 else "VIEWING", "sequence": sequence}, tab)
return (time.perf_counter() - before) * 1000
started = time.perf_counter()
with ThreadPoolExecutor(max_workers=args.workers) as workers:
for sequence in range(1, args.rounds + 1):
result["current_round"] = sequence
if sequence == args.crash_round:
crashed = nodes.pop(0)
if args.node_image:
subprocess.run(["docker", "kill", node_containers[0]], check=True, capture_output=True)
else:
processes[0].kill()
processes[0].wait(timeout=10)
result["crash"] = {"round": sequence, "http_port": crashed,
"surviving_nodes": len(nodes), "process_exit": processes[0].returncode}
renewals.extend(workers.map(lambda item: renew(item, sequence), enumerate(actors)))
check_rooms()
if args.churn_every and sequence % args.churn_every == 0:
departing = [(index, actor) for index, actor in enumerate(actors) if actor[4] == args.users - 1]
for index, (identity, record, tab, lease, user) in departing:
request(nodes[index % len(nodes)], identity, "/presence",
{"connectionId": lease, "table": "person", "pk": record, "sequence": sequence + 1}, tab, method="DELETE")
response = request(nodes[(index + 1) % len(nodes)], identity, "/presence/leases/renew",
{"connectionId": lease, "table": "person", "pk": record,
"mode": "VIEWING", "sequence": sequence + 2}, tab, expected=409)
if response.get("code") != "PRESENCE_LEASE_EXPIRED":
raise AssertionError("Departed lease was not rejected as expired")
check_rooms(excluded=args.users - 1)
for index, (identity, record, tab, lease, user) in departing:
replacement = request(nodes[(index + 1) % len(nodes)], identity, "/presence/leases",
{"table": "person", "pk": record}, tab)["connectionId"]
if replacement == lease:
raise AssertionError("Rejoin reused departed authority")
actors[index] = (identity, record, tab, replacement, user)
renew((index, actors[index]), sequence)
check_rooms()
result["churn_cycles"] = result.get("churn_cycles", 0) + 1
result["departed_leases_rejected"] = result.get("departed_leases_rejected", 0) + len(departing)
result["completed_rounds"] = sequence
if args.pause_redis:
failures = []
def stalled_renew(item):
"""A frozen authority must fail explicitly, never acknowledge a write."""
index, (identity, record, tab, lease, user) = item
before = time.perf_counter()
response = request(nodes[index % len(nodes)], identity, "/presence/leases/renew",
{"connectionId": lease, "table": "person", "pk": record,
"mode": "EDITING" if user < 2 else "VIEWING", "sequence": args.rounds + 1},
tab, expected=503)
if "viewers" in response or "temporarily unavailable" not in response.get("message", ""):
raise AssertionError("Stalled authority did not return the expected unavailable response")
duration = (time.perf_counter() - before) * 1000
if duration > 8000:
raise AssertionError("Unavailable response exceeded the eight-second fixture bound")
return duration
subprocess.run(["docker", "pause", container], check=True, capture_output=True)
try:
with ThreadPoolExecutor(max_workers=64) as burst:
failures.extend(burst.map(stalled_renew, enumerate(actors)))
for node in nodes:
response = request(node, "load-t0-u0", "/presence?table=person&pk=1", expected=503)
if "viewers" in response:
raise AssertionError("Stalled authority was presented as a room snapshot")
finally:
subprocess.run(["docker", "unpause", container], check=True, capture_output=True)
subprocess.run(["docker", "exec", container, "redis-cli", "PING"], check=True, capture_output=True, timeout=10)
for item in enumerate(actors):
renew(item, args.rounds + 2)
check_rooms()
result["redis_stall"] = {"rejected_renewals": len(failures), "failed_snapshot_nodes": len(nodes),
"response_times": distribution(failures), "same_leases_recovered": True}
elapsed = time.perf_counter() - started
result.update(status="PASS", seconds=elapsed, renewals=distribution(renewals),
snapshots=distribution(snapshots), renewal_throughput=len(renewals) / elapsed,
node_pids=[process.pid for process in processes],
pid_kind="docker-client" if args.node_image else "JVM", node_containers=node_containers)
except Exception as error:
result["error"] = str(error)
raise
finally:
failure_active = sys.exc_info()[0] is not None
try:
slow = json.loads(subprocess.check_output(
["docker", "exec", container, "redis-cli", "--json", "SLOWLOG", "GET", "128"], text=True, timeout=10))
result["redis_slow_commands"] = [{"unix_time": entry[1], "duration_us": entry[2], "command": entry[3][0]}
for entry in slow]
result["redis_latency_events"] = json.loads(subprocess.check_output(
["docker", "exec", container, "redis-cli", "--json", "LATENCY", "LATEST"], text=True, timeout=10))
for section in ("cpu", "memory", "commandstats", "stats"):
value = subprocess.check_output(["docker", "exec", container, "redis-cli", "INFO", section], text=True, timeout=10)
(output / f"redis-{section}.txt").write_text(value)
parsed = dict(line.split(":", 1) for line in value.splitlines() if ":" in line and not line.startswith("#"))
if section == "memory":
result["redis_memory"] = {key: int(parsed[key]) for key in
("used_memory", "used_memory_peak", "used_memory_rss")}
if section == "cpu":
result["redis_cpu_seconds_since_start"] = {key: float(parsed[key]) for key in
("used_cpu_sys", "used_cpu_user")}
except (subprocess.SubprocessError, ValueError, KeyError) as error:
result["redis_diagnostic_error"] = type(error).__name__
node_cleanup = True
for node_container in node_containers:
removed = subprocess.run(["docker", "rm", "-f", node_container], capture_output=True)
if removed.returncode != 0:
remaining = subprocess.run(["docker", "ps", "-aq", "--filter", f"name=^{node_container}$"], capture_output=True, text=True)
node_cleanup = node_cleanup and remaining.returncode == 0 and not remaining.stdout.strip()
for process in processes:
if process.poll() is None:
process.terminate()
for process in processes:
try:
process.wait(timeout=10)
except subprocess.TimeoutExpired:
process.kill()
process.wait(timeout=10)
for log in logs:
log.close()
resources = []
for node_log in sorted(output.glob("node-*.log")):
samples = [json.loads(line.split("CAPACITY_METRIC ", 1)[1])
for line in node_log.read_text(errors="replace").splitlines()
if line.startswith("CAPACITY_METRIC ")]
if samples:
first, last = samples[0], samples[-1]
wall_ms = last["uptimeMillis"] - first["uptimeMillis"]
cpu_ns = last["cpuNanos"] - first["cpuNanos"]
resources.append({"node_log": node_log.name, "samples": len(samples),
"observed_wall_ms": wall_ms,
"cpu_seconds": cpu_ns / 1e9 if first["cpuNanos"] >= 0 else None,
"cpu_core_equivalents": cpu_ns / (wall_ms * 1e6) if wall_ms > 0 and first["cpuNanos"] >= 0 else None,
**{key + "_sampled_max": max(sample[key] for sample in samples)
for key in ("heapUsedBytes", "heapCommittedBytes", "poolActive", "poolIdle", "poolWaiting", "poolMaximum")},
"residentBytes_sampled_max": max(sample["residentBytes"] for sample in samples)
if all(sample.get("residentBytes", -1) >= 0 for sample in samples) else None})
if any(sample["poolActive"] > sample["poolMaximum"] for sample in samples):
result.update(status="FAIL", error="Sampled pool active count exceeded configured maximum")
result["resources"] = resources
if result["status"] == "PASS" and args.node_image and any(node["residentBytes_sampled_max"] is None for node in resources):
result.update(status="FAIL", error="Linux resident-memory samples missing")
if result["status"] == "PASS" and any(key not in result for key in ("redis_memory", "redis_cpu_seconds_since_start")):
result.update(status="FAIL", error="Redis memory/CPU diagnostics missing")
if result["status"] == "PASS" and len(resources) != args.nodes:
result.update(status="FAIL", error="Missing resource samples for a node")
if result["status"] == "PASS" and args.pause_redis and not any(node["poolWaiting_sampled_max"] > 0 for node in resources):
result.update(status="FAIL", error="Redis stall did not demonstrate pool waiters")
cleanup = subprocess.run(["docker", "rm", "-f", container], capture_output=True)
network_cleanup = True
if network_created:
network_cleanup = subprocess.run(["docker", "network", "rm", network], capture_output=True).returncode == 0
result["cleanup"] = cleanup.returncode == 0 and node_cleanup and network_cleanup and all(process.poll() is not None for process in processes)
(output / "result.json").write_text(json.dumps(result, indent=2))
print(json.dumps(result, indent=2))
print(output)
if result["status"] == "PASS" and not result["cleanup"]:
raise RuntimeError("Workload passed but owned resource cleanup failed")
if result["status"] == "FAIL" and not failure_active:
raise RuntimeError(result.get("error", "Capacity resource validation failed"))
if __name__ == "__main__":
main()
+42
View File
@@ -0,0 +1,42 @@
// Disposable loopback fixture only; never deploy this fault-injection router.
import http from 'node:http';
const [first, second, listen] = process.argv.slice(2).map(Number);
if (![first, second, listen].every(port => Number.isInteger(port) && port > 0 && port < 65536)) {
throw new Error('Expected first-node, second-node and router ports');
}
const firstStreams = new Set();
const server = http.createServer((request, response) => {
if (request.method === 'POST' && request.url === '/__acceptance/disconnect-first') {
const disconnected = firstStreams.size;
for (const stream of firstStreams) stream.destroy();
response.writeHead(200, { 'content-type': 'application/json' });
response.end(JSON.stringify({ disconnected }));
return;
}
const isSecond = /(?:^|;\s*)presence_node=second(?:;|$)/.test(request.headers.cookie ?? '');
const port = isSecond ? second : first;
const upstream = http.request({
hostname: '127.0.0.1', port, path: request.url, method: request.method,
headers: { ...request.headers, host: `127.0.0.1:${port}` },
}, incoming => {
response.writeHead(incoming.statusCode, {
...incoming.headers, 'x-acceptance-node': isSecond ? 'second' : 'first',
});
if (!isSecond && incoming.headers['content-type']?.includes('text/event-stream')) {
firstStreams.add(response);
}
incoming.pipe(response);
});
upstream.on('error', () => {
if (!response.headersSent) response.writeHead(502);
response.end('Acceptance node unavailable');
});
response.on('close', () => {
firstStreams.delete(response);
upstream.destroy();
});
request.pipe(upstream);
});
server.listen(listen, '127.0.0.1', () => console.log(`PRESENCE_ROUTER_READY ${listen}`));
for (const signal of ['SIGINT', 'SIGTERM']) process.on(signal, () => server.close());
@@ -0,0 +1,285 @@
{
"status": "PASS",
"samples": [
{
"field": "lastName",
"propagation_ms": 36.092099999999846
},
{
"field": "firstName",
"propagation_ms": 32.51870000000008
},
{
"field": "lastName",
"propagation_ms": 31.758600000000115
},
{
"field": "firstName",
"propagation_ms": 31.090299999999843
},
{
"field": "lastName",
"propagation_ms": 31.64980000000014
},
{
"field": "firstName",
"propagation_ms": 31.648199999999633
},
{
"field": "lastName",
"propagation_ms": 31.395500000000084
},
{
"field": "firstName",
"propagation_ms": 32.36750000000029
},
{
"field": "lastName",
"propagation_ms": 31.601799999999912
},
{
"field": "firstName",
"propagation_ms": 31.90599999999995
},
{
"field": "lastName",
"propagation_ms": 31.44150000000036
},
{
"field": "firstName",
"propagation_ms": 31.186699999999746
},
{
"field": "lastName",
"propagation_ms": 31.115800000000036
},
{
"field": "firstName",
"propagation_ms": 32.539400000000114
},
{
"field": "lastName",
"propagation_ms": 31.188000000000102
},
{
"field": "firstName",
"propagation_ms": 33.021200000000135
},
{
"field": "lastName",
"propagation_ms": 33.07759999999962
},
{
"field": "firstName",
"propagation_ms": 32.7176000000004
},
{
"field": "lastName",
"propagation_ms": 31.457300000000032
},
{
"field": "firstName",
"propagation_ms": 32.920900000000074
},
{
"field": "lastName",
"propagation_ms": 31.89130000000023
},
{
"field": "firstName",
"propagation_ms": 30.847200000000157
},
{
"field": "lastName",
"propagation_ms": 30.158300000000054
},
{
"field": "firstName",
"propagation_ms": 31.48239999999987
},
{
"field": "lastName",
"propagation_ms": 31.337600000000293
}
],
"streams": [
{
"user": "alice",
"status": 200,
"type": "text/event-stream;charset=utf-8"
},
{
"user": "admin",
"status": 200,
"type": "text/event-stream;charset=utf-8"
}
],
"requests": [
{
"user": "alice",
"status": 200,
"node": "first"
},
{
"user": "alice",
"status": 200,
"node": "first"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "alice",
"status": 200,
"node": "first"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
}
],
"errors": [],
"scope": "Chromium focus-to-remote-DOM observation, including automation and UI debounce; no concurrent load",
"percentiles_ms": {
"p50": 31.648199999999633,
"p95": 33.07759999999962,
"p99": 36.092099999999846,
"max": 36.092099999999846
},
"frontend_jar_sha256": "c9fa2eb2381a47c8104f82f4e940d2bfa8ba67e2341298dde0de4e21ad3ff1ab",
"fixture_cleanup": true,
"backend_source": "Published core cc3981bb3/collaboration ecff347/Redis7fe5492 snapshots; two independent JVMs, mock identities and memory records."
}
@@ -0,0 +1,357 @@
{
"status": "PASS",
"samples": [
{
"field": "lastName",
"propagation_ms": 36.560600000000704
},
{
"field": "firstName",
"propagation_ms": 33.44050000000061
},
{
"field": "lastName",
"propagation_ms": 32.50890000000072
},
{
"field": "firstName",
"propagation_ms": 785.4951000000001
},
{
"field": "lastName",
"propagation_ms": 85.39069999999992
},
{
"field": "firstName",
"propagation_ms": 30.589599999999336
},
{
"field": "lastName",
"propagation_ms": 32.756099999998696
},
{
"field": "firstName",
"propagation_ms": 38.17399999999907
},
{
"field": "lastName",
"propagation_ms": 30.243099999999686
},
{
"field": "firstName",
"propagation_ms": 29.600799999992887
},
{
"field": "lastName",
"propagation_ms": 29.57220000000234
},
{
"field": "firstName",
"propagation_ms": 29.197800000001735
},
{
"field": "lastName",
"propagation_ms": 33.05130000000645
},
{
"field": "firstName",
"propagation_ms": 32.38390000000072
},
{
"field": "lastName",
"propagation_ms": 31.52850000000035
},
{
"field": "firstName",
"propagation_ms": 34.66169999999693
},
{
"field": "lastName",
"propagation_ms": 31.258300000001327
},
{
"field": "firstName",
"propagation_ms": 44.19029999999475
},
{
"field": "lastName",
"propagation_ms": 36.56189999999333
},
{
"field": "firstName",
"propagation_ms": 30.240600000004633
},
{
"field": "lastName",
"propagation_ms": 29.79899999999907
},
{
"field": "firstName",
"propagation_ms": 29.621500000001106
},
{
"field": "lastName",
"propagation_ms": 31.66189999999915
},
{
"field": "firstName",
"propagation_ms": 29.198399999993853
},
{
"field": "lastName",
"propagation_ms": 32.931200000006356
}
],
"streams": [
{
"user": "alice",
"status": 200,
"type": "text/event-stream;charset=utf-8"
},
{
"user": "admin",
"status": 200,
"type": "text/event-stream;charset=utf-8"
},
{
"user": "alice",
"status": 200,
"type": "text/event-stream;charset=utf-8"
}
],
"requests": [
{
"user": "alice",
"status": 200,
"node": "first"
},
{
"user": "alice",
"status": 200,
"node": "first"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "alice",
"status": 200,
"node": "first"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "alice",
"status": 200,
"node": "first"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "alice",
"status": 200,
"node": "first"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "alice",
"status": 200,
"node": "first"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
}
],
"errors": [],
"scope": "Chromium focus-to-remote-DOM and reconnect under 16 concurrent HTTP lease workers; loopback, four background identities",
"load": {
"workers": 16,
"completed": 9046,
"errors": [],
"nodes": {
"first": 4540,
"second": 4522
},
"elapsed_ms": 69728.3579
},
"reconnect": {
"status": "PASS",
"missed_changes": 3,
"disconnected_streams": 1,
"stream_failure": "net::ERR_INCOMPLETE_CHUNKED_ENCODING",
"recovery_ms": 1855.8868999999977
},
"percentiles_ms": {
"p50": 32.38390000000072,
"p95": 85.39069999999992,
"p99": 785.4951000000001,
"max": 785.4951000000001
},
"frontend_jar_sha256": "c9fa2eb2381a47c8104f82f4e940d2bfa8ba67e2341298dde0de4e21ad3ff1ab",
"fixture_cleanup": true,
"backend_source": "Published core cc3981bb3/collaboration ecff347/Redis7fe5492 snapshots; two independent JVMs, mock identities and memory records.",
"fixture_checks": [
"browser-six-presence",
"browser-redis-admin",
"browser-presence-latency"
]
}
@@ -0,0 +1,317 @@
{
"status": "PASS",
"samples": [
{
"field": "lastName",
"propagation_ms": 37.25770000000011
},
{
"field": "firstName",
"propagation_ms": 32.14499999999998
},
{
"field": "lastName",
"propagation_ms": 32.40950000000021
},
{
"field": "firstName",
"propagation_ms": 33.52500000000009
},
{
"field": "lastName",
"propagation_ms": 31.228200000000015
},
{
"field": "firstName",
"propagation_ms": 30.430999999999585
},
{
"field": "lastName",
"propagation_ms": 31.493199999999888
},
{
"field": "firstName",
"propagation_ms": 33.14539999999988
},
{
"field": "lastName",
"propagation_ms": 32.11259999999993
},
{
"field": "firstName",
"propagation_ms": 33.23129999999992
},
{
"field": "lastName",
"propagation_ms": 31.06800000000021
},
{
"field": "firstName",
"propagation_ms": 30.109199999999873
},
{
"field": "lastName",
"propagation_ms": 31.534299999999803
},
{
"field": "firstName",
"propagation_ms": 33.770800000000236
},
{
"field": "lastName",
"propagation_ms": 31.654199999999946
},
{
"field": "firstName",
"propagation_ms": 33.31700000000001
},
{
"field": "lastName",
"propagation_ms": 33.108600000000024
},
{
"field": "firstName",
"propagation_ms": 32.27370000000019
},
{
"field": "lastName",
"propagation_ms": 31.452499999999873
},
{
"field": "firstName",
"propagation_ms": 31.86220000000003
},
{
"field": "lastName",
"propagation_ms": 31.734399999999823
},
{
"field": "firstName",
"propagation_ms": 32.04489999999987
},
{
"field": "lastName",
"propagation_ms": 31.051899999999932
},
{
"field": "firstName",
"propagation_ms": 30.876300000000356
},
{
"field": "lastName",
"propagation_ms": 31.56970000000001
}
],
"streams": [
{
"user": "alice",
"status": 200,
"type": "text/event-stream;charset=utf-8"
},
{
"user": "admin",
"status": 200,
"type": "text/event-stream;charset=utf-8"
},
{
"user": "alice",
"status": 200,
"type": "text/event-stream;charset=utf-8"
}
],
"requests": [
{
"user": "alice",
"status": 200,
"node": "first"
},
{
"user": "alice",
"status": 200,
"node": "first"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "alice",
"status": 200,
"node": "first"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
},
{
"user": "admin",
"status": 200,
"node": "second"
}
],
"errors": [],
"scope": "Chromium focus-to-remote-DOM observation, including automation and UI debounce; no concurrent load",
"reconnect": {
"status": "PASS",
"missed_changes": 3,
"disconnected_streams": 1,
"stream_failure": "net::ERR_INCOMPLETE_CHUNKED_ENCODING",
"recovery_ms": 1858.2972000000004
},
"percentiles_ms": {
"p50": 31.86220000000003,
"p95": 33.770800000000236,
"p99": 37.25770000000011,
"max": 37.25770000000011
},
"frontend_jar_sha256": "c9fa2eb2381a47c8104f82f4e940d2bfa8ba67e2341298dde0de4e21ad3ff1ab",
"fixture_cleanup": true,
"backend_source": "Published core cc3981bb3/collaboration ecff347/Redis7fe5492 snapshots; two independent JVMs, mock identities and memory records.",
"fixture_checks": [
"browser-six-presence",
"browser-redis-admin",
"browser-presence-latency"
]
}
@@ -0,0 +1,14 @@
{
"status": "PASS",
"checks": [
{ "node": "first", "status": 200 },
{ "node": "second", "status": 404 },
{ "node": "first", "status": 200 }
],
"errors": [],
"fixture_cleanup": true,
"fixture_checks": ["browser-six-presence", "browser-redis-admin", "browser-subscription-affinity"],
"frontend_jar_sha256": "c9fa2eb2381a47c8104f82f4e940d2bfa8ba67e2341298dde0de4e21ad3ff1ab",
"backend_source": "Published core cc3981bb3/collaboration ecff347/Redis7fe5492 snapshots; two independent JVMs, mock identities and memory records.",
"scope": "Same browser session and live SSE connection ID; change only test-router affinity. Not deployed Traefik qualification."
}
@@ -0,0 +1,181 @@
{
"status": "PASS",
"parameters": {
"java": "/path/to/java21",
"node_image": "eclipse-temurin@sha256:227118764cccd3c3294617061aaa93a08309c63a395eefa83e6b35032992ea99",
"nodes": 3,
"tenants": 2,
"users": 50,
"tabs": 4,
"rounds": 200,
"workers": 16,
"crash_round": 0,
"churn_every": 0,
"pause_redis": false
},
"scope": "loopback HTTP, mock identities, memory records",
"node_image_id": "sha256:227118764cccd3c3294617061aaa93a08309c63a395eefa83e6b35032992ea99",
"current_round": 200,
"completed_rounds": 200,
"seconds": 102.56801009993069,
"renewals": {
"samples": 80000,
"p50_ms": 11.325599974952638,
"p95_ms": 21.764500066637993,
"p99_ms": 25.881099980324507,
"max_ms": 1007.4040000326931
},
"snapshots": {
"samples": 2400,
"p50_ms": 15.130399959161878,
"p95_ms": 16.613700077868998,
"p99_ms": 19.423600053414702,
"max_ms": 28.864299994893372
},
"renewal_throughput": 779.9702843221492,
"node_pids": [
105696,
4828,
117588
],
"pid_kind": "docker-client",
"node_containers": [
"qqq-capacity-f58f3ddcf5f3-node-59675",
"qqq-capacity-f58f3ddcf5f3-node-59676",
"qqq-capacity-f58f3ddcf5f3-node-59677"
],
"redis_slow_commands": [
{
"unix_time": 1793033347,
"duration_us": 1432,
"command": "EVALSHA"
},
{
"unix_time": 1793033347,
"duration_us": 1612,
"command": "EVALSHA"
},
{
"unix_time": 1793033340,
"duration_us": 1877,
"command": "EVALSHA"
},
{
"unix_time": 1793033340,
"duration_us": 1313,
"command": "ZRANGE"
},
{
"unix_time": 1793033336,
"duration_us": 1316,
"command": "EVALSHA"
},
{
"unix_time": 1793033305,
"duration_us": 1459,
"command": "EVALSHA"
},
{
"unix_time": 1793033300,
"duration_us": 1767,
"command": "EVALSHA"
},
{
"unix_time": 1793033290,
"duration_us": 1180,
"command": "EVALSHA"
},
{
"unix_time": 1793033283,
"duration_us": 1421,
"command": "EVALSHA"
},
{
"unix_time": 1793033260,
"duration_us": 1291,
"command": "EVALSHA"
},
{
"unix_time": 1793033260,
"duration_us": 1551,
"command": "EVALSHA"
},
{
"unix_time": 1793033260,
"duration_us": 1083,
"command": "ZRANGE"
},
{
"unix_time": 1793033255,
"duration_us": 1039,
"command": "EVALSHA"
},
{
"unix_time": 1793033254,
"duration_us": 1215,
"command": "EVALSHA"
}
],
"redis_latency_events": [
[
"command",
1793033347,
1,
1
]
],
"redis_cpu_seconds_since_start": {
"used_cpu_sys": 23.737706,
"used_cpu_user": 35.526214
},
"redis_memory": {
"used_memory": 2415952,
"used_memory_peak": 2579896,
"used_memory_rss": 8777728
},
"resources": [
{
"node_log": "node-59675.log",
"samples": 1112,
"observed_wall_ms": 111099,
"cpu_seconds": 203.14,
"cpu_core_equivalents": 1.828459302063925,
"heapUsedBytes_sampled_max": 181098896,
"heapCommittedBytes_sampled_max": 211812352,
"poolActive_sampled_max": 7,
"poolIdle_sampled_max": 7,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8,
"residentBytes_sampled_max": 662786048
},
{
"node_log": "node-59676.log",
"samples": 1145,
"observed_wall_ms": 114399,
"cpu_seconds": 200.44,
"cpu_core_equivalents": 1.7521132177728826,
"heapUsedBytes_sampled_max": 185167416,
"heapCommittedBytes_sampled_max": 219152384,
"poolActive_sampled_max": 7,
"poolIdle_sampled_max": 7,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8,
"residentBytes_sampled_max": 662188032
},
{
"node_log": "node-59677.log",
"samples": 1159,
"observed_wall_ms": 115800,
"cpu_seconds": 200.16,
"cpu_core_equivalents": 1.728497409326425,
"heapUsedBytes_sampled_max": 179378632,
"heapCommittedBytes_sampled_max": 210763776,
"poolActive_sampled_max": 6,
"poolIdle_sampled_max": 7,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8,
"residentBytes_sampled_max": 703479808
}
],
"cleanup": true
}
@@ -0,0 +1,101 @@
{
"status": "PASS",
"parameters": {
"java": "java (Java 21)",
"node_image": "eclipse-temurin:21-jre",
"nodes": 3,
"tenants": 2,
"users": 12,
"tabs": 4,
"rounds": 1000,
"workers": 8
},
"scope": "loopback HTTP, mock identities, memory records",
"current_round": 1000,
"completed_rounds": 1000,
"seconds": 250.81072249996942,
"renewals": {
"samples": 96000,
"p50_ms": 6.353199947625399,
"p95_ms": 15.330899972468615,
"p99_ms": 17.760399961844087,
"max_ms": 326.8311999272555
},
"snapshots": {
"samples": 12000,
"p50_ms": 13.105699908919632,
"p95_ms": 14.261199976317585,
"p99_ms": 16.519399941898882,
"max_ms": 915.3468000004068
},
"renewal_throughput": 382.7587554595546,
"node_pids": [
64000,
40240,
115400
],
"pid_kind": "docker-client",
"node_containers": [
"qqq-capacity-0a8b7956499d-node-64951",
"qqq-capacity-0a8b7956499d-node-64952",
"qqq-capacity-0a8b7956499d-node-64953"
],
"redis_slow_commands": [
{
"unix_time": 1793030517,
"duration_us": 1117,
"command": "EVALSHA"
}
],
"redis_latency_events": [
[
"command",
1793030517,
1,
1
]
],
"resources": [
{
"node_log": "node-64951.log",
"samples": 2580,
"observed_wall_ms": 257899,
"cpu_seconds": 165.55,
"cpu_core_equivalents": 0.6419179601316795,
"heapUsedBytes_sampled_max": 166667368,
"heapCommittedBytes_sampled_max": 188743680,
"poolActive_sampled_max": 4,
"poolIdle_sampled_max": 5,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8
},
{
"node_log": "node-64952.log",
"samples": 2569,
"observed_wall_ms": 256799,
"cpu_seconds": 165.85,
"cpu_core_equivalents": 0.6458358482704372,
"heapUsedBytes_sampled_max": 169216944,
"heapCommittedBytes_sampled_max": 192937984,
"poolActive_sampled_max": 5,
"poolIdle_sampled_max": 4,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8
},
{
"node_log": "node-64953.log",
"samples": 2608,
"observed_wall_ms": 260700,
"cpu_seconds": 164.72,
"cpu_core_equivalents": 0.631837360951285,
"heapUsedBytes_sampled_max": 166319008,
"heapCommittedBytes_sampled_max": 188743680,
"poolActive_sampled_max": 4,
"poolIdle_sampled_max": 5,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8
}
],
"cleanup": true,
"receipt_note": "Java executable path normalized; measurements unchanged. Disposable fixture identities only."
}
@@ -0,0 +1,117 @@
{
"status": "PASS",
"parameters": {
"java": "java (Java 21)",
"node_image": "eclipse-temurin@sha256:227118764cccd3c3294617061aaa93a08309c63a395eefa83e6b35032992ea99",
"nodes": 3,
"tenants": 2,
"users": 12,
"tabs": 4,
"rounds": 100,
"workers": 8,
"crash_round": 50,
"churn_every": 20
},
"scope": "loopback HTTP, mock identities, memory records",
"node_image_id": "sha256:227118764cccd3c3294617061aaa93a08309c63a395eefa83e6b35032992ea99",
"current_round": 100,
"completed_rounds": 100,
"churn_cycles": 5,
"departed_leases_rejected": 40,
"crash": {
"round": 50,
"http_port": 52824,
"surviving_nodes": 2,
"process_exit": 137
},
"seconds": 28.090077199973166,
"renewals": {
"samples": 9600,
"p50_ms": 7.532799965701997,
"p95_ms": 17.53149996511638,
"p99_ms": 29.359900043345988,
"max_ms": 88.13189994543791
},
"snapshots": {
"samples": 1092,
"p50_ms": 13.582199928350747,
"p95_ms": 15.386099927127361,
"p99_ms": 18.502700026147068,
"max_ms": 22.144700051285326
},
"renewal_throughput": 341.75769371004685,
"node_pids": [
22844,
66652,
61920
],
"pid_kind": "docker-client",
"node_containers": [
"qqq-capacity-bb9ba87919d2-node-52824",
"qqq-capacity-bb9ba87919d2-node-52825",
"qqq-capacity-bb9ba87919d2-node-52827"
],
"redis_slow_commands": [
{
"unix_time": 1793031681,
"duration_us": 1206,
"command": "EVALSHA"
},
{
"unix_time": 1793031681,
"duration_us": 1147,
"command": "EVALSHA"
}
],
"redis_latency_events": [
[
"command",
1793031681,
1,
1
]
],
"resources": [
{
"node_log": "node-52824.log",
"samples": 212,
"observed_wall_ms": 21099,
"cpu_seconds": 29.48,
"cpu_core_equivalents": 1.3972226171856486,
"heapUsedBytes_sampled_max": 163641024,
"heapCommittedBytes_sampled_max": 188743680,
"poolActive_sampled_max": 3,
"poolIdle_sampled_max": 4,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8
},
{
"node_log": "node-52825.log",
"samples": 354,
"observed_wall_ms": 35299,
"cpu_seconds": 43.09,
"cpu_core_equivalents": 1.22071446783195,
"heapUsedBytes_sampled_max": 168178592,
"heapCommittedBytes_sampled_max": 192937984,
"poolActive_sampled_max": 5,
"poolIdle_sampled_max": 5,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8
},
{
"node_log": "node-52827.log",
"samples": 340,
"observed_wall_ms": 33900,
"cpu_seconds": 42.42,
"cpu_core_equivalents": 1.2513274336283187,
"heapUsedBytes_sampled_max": 166291160,
"heapCommittedBytes_sampled_max": 188743680,
"poolActive_sampled_max": 5,
"poolIdle_sampled_max": 5,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8
}
],
"cleanup": true,
"receipt_note": "Java executable path normalized; measurements unchanged. Disposable fixture identities only."
}
@@ -0,0 +1,96 @@
{
"status": "PASS",
"parameters": {
"java": "java (Java 21)",
"node_image": "eclipse-temurin@sha256:227118764cccd3c3294617061aaa93a08309c63a395eefa83e6b35032992ea99",
"nodes": 3,
"tenants": 2,
"users": 12,
"tabs": 4,
"rounds": 100,
"workers": 8,
"crash_round": 50
},
"scope": "loopback HTTP, mock identities, memory records",
"node_image_id": "sha256:227118764cccd3c3294617061aaa93a08309c63a395eefa83e6b35032992ea99",
"current_round": 100,
"completed_rounds": 100,
"crash": {
"round": 50,
"http_port": 54039,
"surviving_nodes": 2,
"process_exit": 137
},
"seconds": 25.147547999978997,
"renewals": {
"samples": 9600,
"p50_ms": 7.125200005248189,
"p95_ms": 17.407900071702898,
"p99_ms": 27.217599912546575,
"max_ms": 74.53640003222972
},
"snapshots": {
"samples": 996,
"p50_ms": 13.538200058974326,
"p95_ms": 15.59239998459816,
"p99_ms": 20.58999997097999,
"max_ms": 315.0679999962449
},
"renewal_throughput": 381.7469599822622,
"node_pids": [
85832,
90280,
67764
],
"pid_kind": "docker-client",
"node_containers": [
"qqq-capacity-e599dca8e163-node-54039",
"qqq-capacity-e599dca8e163-node-54040",
"qqq-capacity-e599dca8e163-node-54041"
],
"redis_slow_commands": [],
"redis_latency_events": [],
"resources": [
{
"node_log": "node-54039.log",
"samples": 198,
"observed_wall_ms": 19699,
"cpu_seconds": 28.58,
"cpu_core_equivalents": 1.4508350677699375,
"heapUsedBytes_sampled_max": 163478640,
"heapCommittedBytes_sampled_max": 188743680,
"poolActive_sampled_max": 3,
"poolIdle_sampled_max": 4,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8
},
{
"node_log": "node-54040.log",
"samples": 300,
"observed_wall_ms": 29900,
"cpu_seconds": 41.85,
"cpu_core_equivalents": 1.399665551839465,
"heapUsedBytes_sampled_max": 165177008,
"heapCommittedBytes_sampled_max": 188743680,
"poolActive_sampled_max": 4,
"poolIdle_sampled_max": 5,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8
},
{
"node_log": "node-54041.log",
"samples": 331,
"observed_wall_ms": 33000,
"cpu_seconds": 41.52,
"cpu_core_equivalents": 1.2581818181818183,
"heapUsedBytes_sampled_max": 168546936,
"heapCommittedBytes_sampled_max": 192937984,
"poolActive_sampled_max": 4,
"poolIdle_sampled_max": 5,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8
}
],
"cleanup": true,
"receipt_note": "Java executable path normalized; measurements unchanged. Disposable fixture identities only."
}
@@ -0,0 +1,117 @@
{
"status": "PASS",
"parameters": {
"java": "java (Java 21)",
"node_image": "eclipse-temurin@sha256:227118764cccd3c3294617061aaa93a08309c63a395eefa83e6b35032992ea99",
"nodes": 3,
"tenants": 2,
"users": 12,
"tabs": 4,
"rounds": 30,
"workers": 8,
"crash_round": 0,
"churn_every": 0,
"pause_redis": true
},
"scope": "loopback HTTP, mock identities, memory records",
"node_image_id": "sha256:227118764cccd3c3294617061aaa93a08309c63a395eefa83e6b35032992ea99",
"current_round": 30,
"completed_rounds": 30,
"redis_stall": {
"rejected_renewals": 96,
"failed_snapshot_nodes": 3,
"response_times": {
"samples": 96,
"p50_ms": 1013.8123999349773,
"p95_ms": 4014.4751999760047,
"p99_ms": 4015.4822999611497,
"max_ms": 4015.4822999611497
},
"same_leases_recovered": true
},
"seconds": 20.33888840000145,
"renewals": {
"samples": 2880,
"p50_ms": 8.835799992084503,
"p95_ms": 23.678299970924854,
"p99_ms": 35.42319999542087,
"max_ms": 102.84149996004999
},
"snapshots": {
"samples": 372,
"p50_ms": 13.98350007366389,
"p95_ms": 16.946100047789514,
"p99_ms": 20.020700059831142,
"max_ms": 20.943500101566315
},
"renewal_throughput": 141.6006589622565,
"node_pids": [
88080,
78036,
121352
],
"pid_kind": "docker-client",
"node_containers": [
"qqq-capacity-68ad627e7be5-node-58312",
"qqq-capacity-68ad627e7be5-node-58314",
"qqq-capacity-68ad627e7be5-node-58315"
],
"redis_slow_commands": [
{
"unix_time": 1793032016,
"duration_us": 2198,
"command": "EVALSHA"
}
],
"redis_latency_events": [
[
"command",
1793032016,
2,
2
]
],
"resources": [
{
"node_log": "node-58312.log",
"samples": 280,
"observed_wall_ms": 27899,
"cpu_seconds": 23.96,
"cpu_core_equivalents": 0.8588121438044375,
"heapUsedBytes_sampled_max": 164221400,
"heapCommittedBytes_sampled_max": 188743680,
"poolActive_sampled_max": 6,
"poolIdle_sampled_max": 5,
"poolWaiting_sampled_max": 15,
"poolMaximum_sampled_max": 8
},
{
"node_log": "node-58314.log",
"samples": 289,
"observed_wall_ms": 28800,
"cpu_seconds": 23.2,
"cpu_core_equivalents": 0.8055555555555556,
"heapUsedBytes_sampled_max": 162849016,
"heapCommittedBytes_sampled_max": 192937984,
"poolActive_sampled_max": 4,
"poolIdle_sampled_max": 3,
"poolWaiting_sampled_max": 14,
"poolMaximum_sampled_max": 8
},
{
"node_log": "node-58315.log",
"samples": 272,
"observed_wall_ms": 27100,
"cpu_seconds": 23.85,
"cpu_core_equivalents": 0.8800738007380073,
"heapUsedBytes_sampled_max": 165779696,
"heapCommittedBytes_sampled_max": 188743680,
"poolActive_sampled_max": 5,
"poolIdle_sampled_max": 4,
"poolWaiting_sampled_max": 14,
"poolMaximum_sampled_max": 8
}
],
"cleanup": true,
"receipt_note": "Java executable path normalized; measurements unchanged. Disposable fixture identities only."
}
@@ -0,0 +1,61 @@
{
"status": "FAIL",
"parameters": {
"java": "java (Java 21)",
"nodes": 3,
"tenants": 2,
"users": 12,
"tabs": 4,
"rounds": 1000,
"workers": 8
},
"scope": "loopback HTTP, mock identities, memory records",
"current_round": 480,
"completed_rounds": 479,
"error": "HTTP 503 node=56028 user=load-t0-u5 path=/presence/leases/renew: {\"error\":\"Presence is temporarily unavailable. Please retry.\",\"code\":\"ERROR\",\"message\":\"Presence is temporarily unavailable. Please retry.\",\"traceId\":\"cea30b82-e82b-49ec-af8a-64d8b6bd5af6\"}",
"redis_slow_commands": [],
"redis_latency_events": [],
"resources": [
{
"node_log": "node-56027.log",
"samples": 1067,
"observed_wall_ms": 106600,
"cpu_seconds": 94.328125,
"cpu_core_equivalents": 0.8848792213883677,
"heapUsedBytes_sampled_max": 160559696,
"heapCommittedBytes_sampled_max": 178257920,
"poolActive_sampled_max": 5,
"poolIdle_sampled_max": 5,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8
},
{
"node_log": "node-56028.log",
"samples": 1066,
"observed_wall_ms": 106501,
"cpu_seconds": 95.8125,
"cpu_core_equivalents": 0.8996394400052582,
"heapUsedBytes_sampled_max": 160251104,
"heapCommittedBytes_sampled_max": 178257920,
"poolActive_sampled_max": 5,
"poolIdle_sampled_max": 4,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8
},
{
"node_log": "node-56029.log",
"samples": 1066,
"observed_wall_ms": 106500,
"cpu_seconds": 95.25,
"cpu_core_equivalents": 0.8943661971830986,
"heapUsedBytes_sampled_max": 157995216,
"heapCommittedBytes_sampled_max": 175112192,
"poolActive_sampled_max": 5,
"poolIdle_sampled_max": 4,
"poolWaiting_sampled_max": 0,
"poolMaximum_sampled_max": 8
}
],
"cleanup": true,
"receipt_note": "Java executable path normalized; measurements unchanged. Disposable fixture identities only."
}
+29 -30
View File
@@ -18,18 +18,25 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<properties>
<revision>0.1.0-SNAPSHOT</revision>
<!-- Only the published Redis API package is a permitted cross-qbit boundary. -->
<qbit.guardrails.allowedApiPackages>com.kingsrook.qbits.objectstorage.api,com.kingsrook.qbits.email.api,com.kingsrook.qbits.redis.api</qbit.guardrails.allowedApiPackages>
<qbit.guardrails.allowedQbitArtifacts>com.kingsrook.qbits:qbit-object-storage,com.kingsrook.qbits:qbit-email,com.kingsrook.qbits:*-api,com.kingsrook.qbits:qbit-redis</qbit.guardrails.allowedQbitArtifacts>
<revision>1.2.0-SNAPSHOT</revision>
<maven.compiler.source>21</maven.compiler.source>
<maven.compiler.target>21</maven.compiler.target>
<jacoco.version>0.8.15</jacoco.version>
<!-- qbit-email from Maven Central -->
<qbit-email.version>0.1.0-RC.2</qbit-email.version>
<qbit-email.version>0.1.0-RC.11</qbit-email.version>
<!-- Core RC.22 (QBitMetaDataProducer, VirtualFieldValueProviderInterface, QServiceUnavailableException); BLOCKED until it resolves from Central/the registry in CI. -->
<qqq.version>4.1.0-RC.26</qqq.version>
<!-- qbit-redis first release candidate of release/0.1 (producer run 5119 did not publish it yet): BLOCKED, verify the published artifact before pushing. -->
<qbit-redis.version>0.2.0-RC.1</qbit-redis.version>
</properties>
<parent>
<groupId>com.kingsrook</groupId>
<artifactId>qbit-build-parent</artifactId>
<version>2.1.1</version>
<version>2.2.1</version>
</parent>
<groupId>com.kingsrook.qbits</groupId>
@@ -68,15 +75,33 @@
<tag>HEAD</tag>
</scm>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.kingsrook.qqq</groupId>
<artifactId>qqq-bom-pom</artifactId>
<version>${qqq.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>com.kingsrook.qqq</groupId>
<artifactId>qqq-backend-module-rdbms</artifactId>
</dependency>
<dependency>
<groupId>com.kingsrook.qbits</groupId>
<artifactId>qbit-redis</artifactId>
<version>${qbit-redis.version}</version>
<optional>true</optional>
</dependency>
<dependency>
<groupId>com.kingsrook.qbits</groupId>
<artifactId>qbit-object-storage</artifactId>
<version>0.1.0-RC.4</version>
<version>0.1.0-RC.10</version>
</dependency>
<dependency>
<groupId>com.kingsrook.qqq</groupId>
@@ -270,30 +295,4 @@
</plugin>
</plugins>
</build>
<profiles>
<!-- Built against qqq 4.1 (QBitMetaDataProducer). Until qbit-build-parent -->
<!-- imports the released 4.1 BOM, this profile is active by default and -->
<!-- imports the Core integration snapshot with confirmed rollback callbacks. -->
<profile>
<id>qqq-snapshot</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<qqq.snapshot.version>4.1.0-SNAPSHOT</qqq.snapshot.version>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.kingsrook.qqq</groupId>
<artifactId>qqq-bom-pom</artifactId>
<version>${qqq.snapshot.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
</profile>
</profiles>
</project>
@@ -47,9 +47,11 @@ public class CollaborationQBitConfig implements QBitConfig
private List<String> mirroredSecurityKeyTypes = List.of();
private QCodeReference notifier = null;
private com.kingsrook.qbits.collaboration.presence.RedisPresenceConfig redisPresence;
private Function<String, String> recipientEmailResolver = null;
private QCodeReference userSessionProvider = null;
private String defaultTagMode = "FREE";
private Integer tagLookupCacheSeconds = 30;
private List<String> tagActionTables = List.of();
private List<String> followActionTables = List.of();
private List<String> followChangeTables = List.of();
@@ -315,11 +317,16 @@ public class CollaborationQBitConfig implements QBitConfig
@Override
public void validate(QInstance instance, List<String> errors)
{
if(redisPresence != null)
{
redisPresence.validate(errors);
}
if(assertCondition(StringUtils.hasContent(backendName), "backendName is required for Collaboration", errors))
{
assertCondition(instance.getBackend(backendName) != null, "Unknown Collaboration backend: " + backendName, errors);
}
assertCondition(List.of("FREE", "APPROVED_ONLY").contains(defaultTagMode == null ? "" : defaultTagMode), "defaultTagMode must be FREE or APPROVED_ONLY", errors);
assertCondition(tagLookupCacheSeconds != null && tagLookupCacheSeconds >= 0, "tagLookupCacheSeconds must be 0 (off) or more", errors);
for(List<String> tables : List.of(followActionTables, followChangeTables, tagActionTables, assignActionTables))
{
for(String table : tables)
@@ -545,6 +552,27 @@ public class CollaborationQBitConfig implements QBitConfig
return (this);
}
/*******************************************************************************
** How long a node remembers whether a table has any tag assignment.
*******************************************************************************/
public Integer getTagLookupCacheSeconds()
{
return (tagLookupCacheSeconds);
}
/*******************************************************************************
** Reads of a table without any tag assignment skip the assignment lookup while
** this node remembers that (default 30 seconds; 0 looks every time). A node
** that writes an assignment forgets at once and tells the others over the
** ClusterBus; without a cluster transport another node shows a table's first
** tag after at most this long. Tables that have assignments are always read.
*******************************************************************************/
public CollaborationQBitConfig withTagLookupCacheSeconds(Integer tagLookupCacheSeconds)
{
this.tagLookupCacheSeconds = tagLookupCacheSeconds;
return (this);
}
/*******************************************************************************
** Host tables receiving Add tags and Remove tags actions.
*******************************************************************************/
@@ -692,4 +720,21 @@ public class CollaborationQBitConfig implements QBitConfig
{
this.managedIndexes = managedIndexes;
}
/***************************************************************************
** Optional shared Redis presence; absent retains the existing memory provider.
***************************************************************************/
public com.kingsrook.qbits.collaboration.presence.RedisPresenceConfig getRedisPresence()
{
return (redisPresence);
}
/***************************************************************************
** Select shared presence with explicit host-derived tenant scope.
***************************************************************************/
public CollaborationQBitConfig withRedisPresence(com.kingsrook.qbits.collaboration.presence.RedisPresenceConfig value)
{
redisPresence = value;
return (this);
}
}
@@ -155,6 +155,10 @@ public class CollaborationQBitProducer implements QBitMetaDataProducer<Collabora
public MetaDataProducerMultiOutput produce(QInstance instance) throws QException
{
qBitConfig.validate(instance);
var realtime = com.kingsrook.qqq.backend.core.realtime.RealtimeMetaData.of(instance);
realtime.withAudienceProvider(com.kingsrook.qbits.collaboration.presence.RedisPresenceAudience.EVENT,
new com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference(com.kingsrook.qbits.collaboration.presence.RedisPresenceAudience.class));
instance.withSupplementalMetaData(realtime);
QBitMetaData qBit = getQBitMetaData();
instance.addQBit(qBit);
MetaDataProducerMultiOutput output = new MetaDataProducerMultiOutput();
@@ -33,7 +33,8 @@ import com.kingsrook.qqq.backend.core.realtime.RealtimeBus;
import com.kingsrook.qqq.backend.core.realtime.RealtimeMessage;
/*******************************************************************************
** Identifier-only hints, published after successful commit.
** Identifier-only hints, published after successful commit. A tag assignment
** also clears what this node remembers about untagged tables (TaggedTables).
*******************************************************************************/
public class CollaborationRealtime implements RecordChangeListenerInterface
{
@@ -55,6 +56,7 @@ public class CollaborationRealtime implements RecordChangeListenerInterface
public void onRecordsChanged(RecordChangeEvent change) throws QException
{
var instance = QContext.getQInstance();
java.util.Set<String> newlyTaggedTables = new java.util.HashSet<>();
for(int index = 0; index < change.getRecords().size(); index++)
{
var row = new com.kingsrook.qqq.backend.core.model.data.QRecord(change.getRecords().get(index));
@@ -70,6 +72,13 @@ public class CollaborationRealtime implements RecordChangeListenerInterface
{
continue;
}
if(CollaborationTables.RECORD_TAG.equals(change.getTableName()) && change.getType() != RecordChangeType.DELETE && newlyTaggedTables.add(table))
{
////////////////////////////////////////////////////////////////////////
// the Tags field skips tables known to have no assignment: forget it //
////////////////////////////////////////////////////////////////////////
com.kingsrook.qbits.collaboration.tags.TaggedTables.assignmentsChanged(table, change.getTransaction());
}
String event = switch(change.getTableName())
{
case CollaborationTables.COMMENT -> change.getType() == RecordChangeType.DELETE || row.getValue("deletedAt") != null
@@ -23,6 +23,8 @@ package com.kingsrook.qbits.collaboration;
import java.io.Serializable;
import java.util.List;
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
import com.kingsrook.qqq.backend.core.actions.tables.DeleteAction;
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
@@ -36,6 +38,8 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordFeature;
import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordReferenceTableMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
/*******************************************************************************
@@ -74,7 +78,15 @@ public final class CollaborationRecords
*******************************************************************************/
public static List<QRecord> query(String table, QQueryFilter filter) throws QException
{
QueryInput input = new QueryInput(table).withFilter(filter).withShouldFetchHeavyFields(true).withTransaction(QContext.getQBackendTransaction());
return (query(table, filter, QContext.getQBackendTransaction()));
}
/*******************************************************************************
** Provider reads participate in the originating query's explicit transaction.
*******************************************************************************/
public static List<QRecord> query(String table, QQueryFilter filter, QBackendTransaction transaction) throws QException
{
QueryInput input = new QueryInput(table).withFilter(filter).withShouldFetchHeavyFields(true).withTransaction(transaction);
input.setTableMetaData(serviceMetadata(table));
input.withPermissionBypass("Trusted collaboration service performs parent/ownership authorization.");
return (new QueryAction().execute(input).getRecords().stream().map(QRecord::new).toList());
@@ -85,8 +97,34 @@ public final class CollaborationRecords
*******************************************************************************/
public static QRecord insert(String table, QRecord row) throws QException
{
InsertInput input = new InsertInput(table).withRecords(List.of(row)).withTransaction(QContext.getQBackendTransaction());
input.setTableMetaData(serviceMetadata(table));
return (insert(serviceMetadata(table), row));
}
/*******************************************************************************
** Activity belongs to the authorized source operation; timeline controls its
** presentation. Preserve native parent checks and mirrored security keys.
*******************************************************************************/
public static QRecord insertActivity(QRecord row, RecordFeature sourceFeature) throws QException
{
QTableMetaData metadata = serviceMetadata(CollaborationTables.ACTIVITY);
RecordReferenceTableMetaData original = RecordReferenceTableMetaData.of(metadata);
if(original == null || sourceFeature == null)
{
throw (new QException("Activity source authorization metadata is required."));
}
RecordReferenceTableMetaData reference = (RecordReferenceTableMetaData) original.clone();
reference.withFeature(sourceFeature);
metadata.withSupplementalMetaData(reference);
return (insert(metadata, row));
}
/*******************************************************************************
** A private metadata copy retains the same transaction and validation path.
*******************************************************************************/
private static QRecord insert(QTableMetaData metadata, QRecord row) throws QException
{
InsertInput input = new InsertInput(metadata.getName()).withRecords(List.of(row)).withTransaction(QContext.getQBackendTransaction());
input.setTableMetaData(metadata);
input.withPermissionBypass("Trusted collaboration service authorized this write.");
List<QRecord> result = new InsertAction().execute(input).getRecords();
check(result);
@@ -49,14 +49,13 @@ public class CollaborationSummaryProvider implements RecordFeatureSummaryProvide
String user = PersonalRecords.currentUserId();
if(features.isEnabled(RecordFeature.PRESENCE))
{
summary.setPresence(com.kingsrook.qbits.collaboration.presence.PresenceRegistry.get(QContext.getQInstance()).snapshot(ref.tableName(), id).stream().map(viewer ->
summary.setPresence(new com.kingsrook.qbits.collaboration.presence.PresenceService().snapshot(ref.tableName(), id).stream().map(viewer ->
{
java.util.Map<String, java.io.Serializable> entry = new java.util.LinkedHashMap<>();
entry.put("userId", viewer.userId());
entry.put("displayName", viewer.displayName());
entry.put("mode", viewer.mode());
entry.put("fieldName", viewer.fieldName());
entry.put("connections", viewer.connections());
return (entry);
}).toList());
}
@@ -29,10 +29,12 @@ import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizerInterfa
import com.kingsrook.qqq.backend.core.context.CapturedContext;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.instances.QInstanceHelpContentManager;
import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
/*******************************************************************************
** Publish committed policy changes through the core overlay.
@@ -77,7 +79,7 @@ public class FeatureSettingsCustomizer implements TableCustomizerInterface
var instance = QContext.getQInstance();
if(transaction == null)
{
RuntimeMetaDataOverlay.refresh(instance);
refreshMetadata(instance);
}
else
{
@@ -86,7 +88,7 @@ public class FeatureSettingsCustomizer implements TableCustomizerInterface
{
try
{
QContext.withTemporaryContext(context, () -> RuntimeMetaDataOverlay.refresh(instance));
QContext.withTemporaryContext(context, () -> refreshMetadata(instance));
}
catch(QException e)
{
@@ -95,4 +97,13 @@ public class FeatureSettingsCustomizer implements TableCustomizerInterface
});
}
}
/*******************************************************************************
** Bind persisted help after the overlay creates its dynamic fields and sections.
*******************************************************************************/
private static void refreshMetadata(QInstance instance) throws QException
{
RuntimeMetaDataOverlay.refresh(instance);
QInstanceHelpContentManager.reloadHelpContent(instance);
}
}
@@ -288,12 +288,16 @@ public final class TargetResolver
/*******************************************************************************
** Guard existence before invoking the core table permission helper.
** Guard existence before invoking the core table permission helper. The read
** is asked as the query it stands for: core hands a table's custom permission
** checker the action input and not the permission asked, so a checker that
** tells a read from a write by the input's type refuses an untyped input, and
** its table was then no target for the people it lets read it.
*******************************************************************************/
private static boolean canReadTable(String table)
{
return (table != null && QContext.getQInstance().getTable(table) != null
&& PermissionsHelper.hasTablePermission(new AbstractActionInput(), table, TablePermissionSubType.READ));
&& PermissionsHelper.hasTablePermission(new QueryInput(table), table, TablePermissionSubType.READ));
}
@@ -28,6 +28,7 @@ import com.kingsrook.qbits.collaboration.model.CollaborationTables;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordFeature;
/*******************************************************************************
** Internal activity writer; the calling service has already checked the parent.
@@ -46,8 +47,28 @@ public final class ActivityWriter
*******************************************************************************/
public static void write(String table, String recordId, String type, String summary, String refTable, String refId) throws QException
{
CollaborationRecords.insert(CollaborationTables.ACTIVITY, new QRecord().withValue("tableName", table).withValue("recordId", recordId)
CollaborationRecords.insertActivity(new QRecord().withValue("tableName", table).withValue("recordId", recordId)
.withValue("at", Instant.now()).withValue("actorUserId", CollaborationSecurity.userId(QContext.getQSession()))
.withValue("type", type).withValue("summary", summary).withValue("refTableName", refTable).withValue("refId", refId).withValue("parentDeleted", false));
.withValue("type", type).withValue("summary", summary).withValue("refTableName", refTable).withValue("refId", refId).withValue("parentDeleted", false), sourceFeature(refTable));
}
/*******************************************************************************
** Source-specific activity requires that feature; generic events retain the
** existing timeline authorization contract.
*******************************************************************************/
private static RecordFeature sourceFeature(String table)
{
return (switch(table == null ? "" : table)
{
case CollaborationTables.ATTACHMENT -> RecordFeature.ATTACHMENTS;
case CollaborationTables.COMMENT, CollaborationTables.COMMENT_REVISION, CollaborationTables.MENTION -> RecordFeature.COMMENTS;
case CollaborationTables.TAG, CollaborationTables.RECORD_TAG -> RecordFeature.TAGS;
case CollaborationTables.FIELD_DEFINITION, CollaborationTables.FIELD_VALUE -> RecordFeature.CUSTOM_FIELDS;
case CollaborationTables.TASK -> RecordFeature.TASKS;
case CollaborationTables.RECORD_ASSIGNMENT -> RecordFeature.ASSIGNMENT;
case CollaborationTables.FOLLOW, CollaborationTables.FOLLOW_DIGEST -> RecordFeature.FOLLOW;
case CollaborationTables.SHARE -> RecordFeature.SHARING;
default -> RecordFeature.TIMELINE;
});
}
}
@@ -52,9 +52,10 @@ public final class CollaborationApi
*******************************************************************************/
public static MiddlewareSpecExtension extension()
{
return (new MiddlewareSpecExtension().withName("collaboration").withSemanticVersion("1.2.0")
return (new MiddlewareSpecExtension().withName("collaboration").withSemanticVersion("1.3.0")
.withDescription("Record collaboration: favorites, recents, attachments, tags, conversations, following, timeline, people and presence")
.withSpec(PostPresenceSpec.class).withSpec(DeletePresenceSpec.class).withSpec(GetPresenceSpec.class)
.withSpec(OpenPresenceLeaseSpec.class).withSpec(RenewPresenceLeaseSpec.class)
.withSpec(ListFavoritesSpec.class)
.withSpec(StarFavoriteSpec.class)
.withSpec(UnstarFavoriteSpec.class)
@@ -96,6 +97,7 @@ public final class CollaborationApi
.withSpec(ListMyTasksSpec.class)
.withSpec(PutAssignmentSpec.class)
.withSpec(GetMyWorkSpec.class)
.withSpec(MarkMentionsReadSpec.class)
.withSpec(UploadAttachmentSpec.class));
}
}
@@ -116,7 +116,8 @@ abstract class CollaborationJsonSpec extends AbstractJsonEndpointSpec
return (List.of(new BasicResponse(HttpStatus.BAD_REQUEST, "Invalid input", BasicErrorResponseV1.class.getSimpleName()),
new BasicResponse(HttpStatus.UNAUTHORIZED, "Sign in required", BasicErrorResponseV1.class.getSimpleName()),
new BasicResponse(HttpStatus.FORBIDDEN, "Permission denied", BasicErrorResponseV1.class.getSimpleName()),
new BasicResponse(HttpStatus.NOT_FOUND, "Not found (including invisible parents)", BasicErrorResponseV1.class.getSimpleName())));
new BasicResponse(HttpStatus.NOT_FOUND, "Not found (including invisible parents)", BasicErrorResponseV1.class.getSimpleName()),
new BasicResponse(HttpStatus.SERVICE_UNAVAILABLE, "Service temporarily unavailable", BasicErrorResponseV1.class.getSimpleName())));
}
/*******************************************************************************
@@ -161,14 +162,13 @@ abstract class CollaborationJsonSpec extends AbstractJsonEndpointSpec
case "cursor" -> "Opaque nextCursor returned by the previous page; omit for the first page.";
case "before" -> "Timeline continuation token; alias for cursor. Supply only one.";
case "includeDone" -> "Include completed and cancelled tasks; default false.";
case "perConnection" -> "Also return one viewer per live connection (lease) in leases; default false. Since collaboration 1.2.0.";
case "status" -> "Task status: OPEN (default), DONE or CANCELLED.";
case "types" -> "Comma-separated AUDIT, ACTIVITY or activity type names (for example COMMENTED).";
case "q" -> "Text to search in display names or vocabulary.";
default -> name;
};
return (new Parameter().withName(name).withIn(location).withRequired(required).withDescription(description)
.withSchema(new Schema().withType(name.equals("limit") ? Type.INTEGER : name.equals("includeDone") || name.equals("perConnection") ? Type.BOOLEAN : Type.STRING)));
.withSchema(new Schema().withType(name.equals("limit") ? Type.INTEGER : name.equals("includeDone") ? Type.BOOLEAN : Type.STRING)));
}
/*******************************************************************************
@@ -24,7 +24,6 @@ package com.kingsrook.qbits.collaboration.api;
import java.util.List;
import java.util.Map;
import java.util.TreeMap;
import com.fasterxml.jackson.annotation.JsonInclude;
import com.kingsrook.qbits.collaboration.presence.PresenceRegistry;
import com.kingsrook.qqq.middleware.javalin.schemabuilder.ToSchema;
import com.kingsrook.qqq.openapi.model.Schema;
@@ -42,11 +41,67 @@ public final class CollaborationPresenceSchemas
{
}
/*******************************************************************************
** Target for invisible server-issued renewal authority.
*******************************************************************************/
public record CollaborationPresenceOpenV1(String table, String pk) implements ToSchema
{
/***************************************************************************
** Opening alone never advertises a viewer.
***************************************************************************/
@Override
public Schema toSchema()
{
return (new Schema().withType(Type.OBJECT).withProperties(new TreeMap<>(Map.of(
"table", text("Parent table name.", false), "pk", text("Optional record key.", true)))));
}
}
/*******************************************************************************
** Opaque handle bound to the authenticated caller and target.
*******************************************************************************/
public record CollaborationPresenceLeaseV1(String connectionId) implements ToSchema
{
/***************************************************************************
** Activate with a positive sequence; acquire a new handle after expiry.
***************************************************************************/
@Override
public Schema toSchema()
{
return (new Schema().withType(Type.OBJECT).withProperties(Map.of(
"connectionId", text("Server-issued handle. Renew before 45 seconds; departure is terminal.", false))));
}
}
/*******************************************************************************
** Stable machine-readable response for missing renewal authority.
*******************************************************************************/
public record CollaborationPresenceExpiredV1(String code) implements ToSchema
{
/***************************************************************************
** Clients rejoin only if their surface is still active and visible.
***************************************************************************/
@Override
public Schema toSchema()
{
return (new Schema().withType(Type.OBJECT).withProperties(Map.of(
"code", text("Open a fresh lease before retrying renewal.", false).withEnumValues(List.of("PRESENCE_LEASE_EXPIRED")))));
}
}
/*******************************************************************************
** All identity information is derived from the authenticated session.
*******************************************************************************/
public record CollaborationPresenceRequestV1(String connectionId, String table, String pk, String mode, String fieldName) implements ToSchema
public record CollaborationPresenceRequestV1(String connectionId, String table, String pk, String mode, String fieldName, Long sequence) implements ToSchema
{
/*******************************************************************************
** Preserve source callers that do not yet send sequence numbers.
*******************************************************************************/
public CollaborationPresenceRequestV1(String connectionId, String table, String pk, String mode, String fieldName)
{
this(connectionId, table, pk, mode, fieldName, null);
}
/*******************************************************************************
** Clients renew every twenty seconds; the lease expires after forty-five.
*******************************************************************************/
@@ -58,15 +113,24 @@ public final class CollaborationPresenceSchemas
"connectionId", text("Client connection identity.", false), "table", text("Parent table name.", false),
"pk", text("Optional parent key; omit for a table view.", true),
"mode", text("VIEWING or EDITING.", false).withEnumValues(List.of("VIEWING", "EDITING")),
"fieldName", text("Optional focused field on a record.", true)))));
"fieldName", text("Optional focused field on a record.", true),
"sequence", sequenceSchema()))));
}
}
/*******************************************************************************
** Leave removes only this authenticated user's connection and target.
*******************************************************************************/
public record CollaborationPresenceLeaveV1(String connectionId, String table, String pk) implements ToSchema
public record CollaborationPresenceLeaveV1(String connectionId, String table, String pk, Long sequence) implements ToSchema
{
/*******************************************************************************
** Preserve source callers that do not yet send sequence numbers.
*******************************************************************************/
public CollaborationPresenceLeaveV1(String connectionId, String table, String pk)
{
this(connectionId, table, pk, null);
}
/*******************************************************************************
** Leave does not need a mode or a field.
*******************************************************************************/
@@ -76,27 +140,15 @@ public final class CollaborationPresenceSchemas
return (new Schema().withType(Type.OBJECT).withDescription("Remove the caller's lease on this connection and target.")
.withProperties(new TreeMap<>(Map.of(
"connectionId", text("Client connection identity.", false), "table", text("Parent table name.", false),
"pk", text("Optional parent key.", true)))));
"pk", text("Optional parent key.", true), "sequence", sequenceSchema()))));
}
}
/*******************************************************************************
** The snapshot is also the data shape of presence.changed events. Viewers are
** included even when empty, whatever inclusion the serializer defaults to.
** The snapshot is also the data shape of presence.changed events.
*******************************************************************************/
public record CollaborationPresenceSnapshotV1(String table, String pk, @JsonInclude(JsonInclude.Include.ALWAYS) List<PresenceRegistry.Viewer> viewers,
Integer leaseCount, @JsonInclude(JsonInclude.Include.NON_NULL) List<PresenceRegistry.Viewer> leases) implements ToSchema
public record CollaborationPresenceSnapshotV1(String table, String pk, List<PresenceRegistry.Viewer> viewers) implements ToSchema
{
/*******************************************************************************
** The merged snapshot without per-connection detail.
*******************************************************************************/
public CollaborationPresenceSnapshotV1(String table, String pk, List<PresenceRegistry.Viewer> viewers)
{
this(table, pk, viewers, null, null);
}
/*******************************************************************************
** A table snapshot deliberately excludes record ids and focused field names.
*******************************************************************************/
@@ -106,17 +158,14 @@ public final class CollaborationPresenceSchemas
return (new Schema().withType(Type.OBJECT).withDescription("Live viewers, merged across tabs; presence.changed uses this payload.")
.withProperties(new TreeMap<>(Map.of(
"table", text("Parent table name.", false), "pk", text("Record key; absent for table events.", true),
"viewers", new Schema().withType(Type.ARRAY).withDescription("Live viewers; always present, empty once the last viewer leaves.").withNullable(false).withItems(new Schema().withRefToSchema("CollaborationPresenceViewerV1")),
"leaseCount", new Schema().withType(Type.INTEGER).withDescription("Live connections (leases) behind viewers; a user with two terminals counts twice. Since 1.2.0.").withNullable(true),
"leases", new Schema().withType(Type.ARRAY).withDescription("One viewer per live connection, with connectionId and since. On GET only with perConnection=true; always on presence.changed. Since 1.2.0.")
.withNullable(true).withItems(new Schema().withRefToSchema("CollaborationPresenceViewerV1"))))));
"viewers", new Schema().withType(Type.ARRAY).withItems(new Schema().withRefToSchema("CollaborationPresenceViewerV1"))))));
}
}
/*******************************************************************************
** Only public identity and focus are carried, never field contents.
*******************************************************************************/
public record CollaborationPresenceViewerV1(String userId, String displayName, String mode, String fieldName, Integer connections, String connectionId, String sessionId, String since) implements ToSchema
public record CollaborationPresenceViewerV1(String userId, String displayName, String mode, String fieldName) implements ToSchema
{
/*******************************************************************************
** Editing takes precedence when one user has several open tabs.
@@ -124,14 +173,10 @@ public final class CollaborationPresenceSchemas
@Override
public Schema toSchema()
{
return (new Schema().withType(Type.OBJECT).withDescription("One user merged across live tabs (EDITING takes precedence), or one live connection in leases.")
return (new Schema().withType(Type.OBJECT).withDescription("One user merged across live tabs; EDITING takes precedence.")
.withProperties(new TreeMap<>(Map.of(
"userId", text("Stable user identity.", false), "displayName", text("Session display name.", false),
"mode", text("VIEWING or EDITING.", false).withEnumValues(List.of("VIEWING", "EDITING")), "fieldName", text("Focused field; null on table snapshots.", true),
"connections", new Schema().withType(Type.INTEGER).withDescription("Live connections (tabs, terminals) merged into this viewer; 1 in leases. Since 1.2.0.").withNullable(true),
"connectionId", text("Opaque per-connection id, only in leases: c + the first 16 hex digits of SHA-256(userId + newline + connectionId). Since 1.2.0.", true),
"sessionId", text("Opaque id of the signed-in session behind this connection, only in leases (HMAC of the server session; never the session itself). Since 1.2.0.", true),
"since", text("When this connection's lease started (ISO-8601), only in leases. Since 1.2.0.", true).withFormat("date-time")))));
"mode", text("VIEWING or EDITING.", false).withEnumValues(List.of("VIEWING", "EDITING")), "fieldName", text("Focused field; null on table snapshots.", true)))));
}
}
@@ -142,4 +187,13 @@ public final class CollaborationPresenceSchemas
{
return (new Schema().withType(Type.STRING).withDescription(description).withNullable(nullable));
}
/*******************************************************************************
** Optional for legacy clients; new clients increment on every renewal and leave.
*******************************************************************************/
private static Schema sequenceSchema()
{
return (new Schema().withType(Type.INTEGER).withNullable(true)
.withDescription("Monotonically increasing nonnegative safe integer per connection and target; orders delayed requests."));
}
}
@@ -736,7 +736,10 @@ public final class CollaborationSchemas
schemas.put("CollaborationPresenceRequestV1", new CollaborationPresenceSchemas.CollaborationPresenceRequestV1(null, null, null, null, null).toSchema());
schemas.put("CollaborationPresenceLeaveV1", new CollaborationPresenceSchemas.CollaborationPresenceLeaveV1(null, null, null).toSchema());
schemas.put("CollaborationPresenceSnapshotV1", new CollaborationPresenceSchemas.CollaborationPresenceSnapshotV1(null, null, null).toSchema());
schemas.put("CollaborationPresenceViewerV1", new CollaborationPresenceSchemas.CollaborationPresenceViewerV1(null, null, null, null, null, null, null, null).toSchema());
schemas.put("CollaborationPresenceViewerV1", new CollaborationPresenceSchemas.CollaborationPresenceViewerV1(null, null, null, null).toSchema());
schemas.put("CollaborationPresenceOpenV1", new CollaborationPresenceSchemas.CollaborationPresenceOpenV1(null, null).toSchema());
schemas.put("CollaborationPresenceLeaseV1", new CollaborationPresenceSchemas.CollaborationPresenceLeaseV1(null).toSchema());
schemas.put("CollaborationPresenceExpiredV1", new CollaborationPresenceSchemas.CollaborationPresenceExpiredV1(null).toSchema());
schemas.putAll(CollaborationWorkSchemas.components());
return (schemas);
}
@@ -106,7 +106,7 @@ final class CollaborationViews
Map<String, String> names = names(values.stream().map(AttachmentEntry::createdByUserId).toArray(String[]::new));
return (values.stream().map(value -> new CollaborationAttachmentV1(
value.id(), value.tableName(), value.recordId(), value.fileName(), value.contentType(), value.sizeBytes(), value.sha256(), value.description(), value.createdByUserId(), value.createDate(), value.canDelete(),
name(names, value.createdByUserId()))).toList());
name(names, value.createdByUserId()), value.canEdit())).toList());
}
/*******************************************************************************
@@ -76,7 +76,7 @@ public class DeletePresenceSpec extends CollaborationJsonSpec
protected Object handle(JsonInput input) throws Exception
{
var body = body(input, CollaborationPresenceSchemas.CollaborationPresenceLeaveV1.class);
new PresenceService().leave(body.connectionId(), required(body.table(), "table"), body.pk());
new PresenceService().leave(body.connectionId(), required(body.table(), "table"), body.pk(), body.sequence());
return (success());
}
}
@@ -23,7 +23,6 @@ package com.kingsrook.qbits.collaboration.api;
import java.util.List;
import com.kingsrook.qbits.collaboration.presence.PresenceService;
import com.kingsrook.qqq.backend.core.exceptions.QBadRequestException;
import com.kingsrook.qqq.middleware.javalin.specs.BasicOperation;
import com.kingsrook.qqq.openapi.model.HttpMethod;
import com.kingsrook.qqq.openapi.model.Parameter;
@@ -49,7 +48,7 @@ public class GetPresenceSpec extends CollaborationJsonSpec
@Override
public List<Parameter> defineRequestParameters()
{
return (parameters("table!", "pk", "perConnection"));
return (parameters("table!", "pk"));
}
/*******************************************************************************
@@ -69,14 +68,6 @@ public class GetPresenceSpec extends CollaborationJsonSpec
{
String table = required(input.getQueryParam("table"), "table");
String pk = input.getQueryParam("pk");
String perConnection = input.getQueryParam("perConnection");
if(perConnection != null && !perConnection.equalsIgnoreCase("true") && !perConnection.equalsIgnoreCase("false"))
{
throw (new QBadRequestException("perConnection must be true or false"));
}
var service = new PresenceService();
var viewers = service.snapshot(table, pk);
var leases = service.connections(table, pk);
return (new CollaborationPresenceSchemas.CollaborationPresenceSnapshotV1(table, pk, viewers, leases.size(), "true".equalsIgnoreCase(perConnection) ? leases : null));
return (new CollaborationPresenceSchemas.CollaborationPresenceSnapshotV1(table, pk, new PresenceService().snapshot(table, pk)));
}
}
@@ -0,0 +1,71 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.api;
import java.util.List;
import com.kingsrook.qqq.middleware.javalin.specs.BasicOperation;
import com.kingsrook.qqq.openapi.model.HttpMethod;
import com.kingsrook.qqq.openapi.model.Parameter;
/*******************************************************************************
** Mark all currently visible unread mentions read.
*******************************************************************************/
public class MarkMentionsReadSpec extends CollaborationJsonSpec
{
/*******************************************************************************
** Describe this operation.
*******************************************************************************/
@Override
public BasicOperation defineBasicOperation()
{
return (operation(HttpMethod.POST, "/my-work/mentions/read", "Mark my visible mentions read"));
}
/*******************************************************************************
** Declare inputs.
*******************************************************************************/
@Override
public List<Parameter> defineRequestParameters()
{
return (parameters());
}
/*******************************************************************************
** Name the response.
*******************************************************************************/
@Override
protected String getResponseSchemaName()
{
return ("CollaborationMutationResultV1");
}
/*******************************************************************************
** Run the service as the authenticated caller.
*******************************************************************************/
@Override
protected Object handle(JsonInput input) throws Exception
{
new com.kingsrook.qbits.collaboration.mywork.MyWorkService().markMentionsRead(null);
return (success());
}
}
@@ -0,0 +1,81 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.api;
import java.util.List;
import com.kingsrook.qbits.collaboration.presence.PresenceService;
import com.kingsrook.qqq.middleware.javalin.specs.BasicOperation;
import com.kingsrook.qqq.openapi.model.HttpMethod;
import com.kingsrook.qqq.openapi.model.Parameter;
import com.kingsrook.qqq.openapi.model.Schema;
/*******************************************************************************
** Renew presence with fresh parent access checks.
*******************************************************************************/
public class OpenPresenceLeaseSpec extends CollaborationJsonSpec
{
/*******************************************************************************
** Describe the extension operation.
*******************************************************************************/
@Override
public BasicOperation defineBasicOperation()
{
return (operation(HttpMethod.POST, "/presence/leases", "Open presence lease"));
}
/*******************************************************************************
** Explicit query parameters for polling.
*******************************************************************************/
@Override
public List<Parameter> defineRequestParameters()
{
return (parameters());
}
/*******************************************************************************
** Name the versioned response schema.
*******************************************************************************/
@Override
protected String getResponseSchemaName()
{
return ("CollaborationPresenceLeaseV1");
}
/*******************************************************************************
** Declare a typed JSON request body.
*******************************************************************************/
@Override
protected Schema defineRequestBodySchema()
{
return (new Schema().withRefToSchema("CollaborationPresenceOpenV1"));
}
/*******************************************************************************
** Services derive identity from QContext and enforce visibility.
*******************************************************************************/
@Override
protected Object handle(JsonInput input) throws Exception
{
var body = body(input, CollaborationPresenceSchemas.CollaborationPresenceOpenV1.class);
return (new CollaborationPresenceSchemas.CollaborationPresenceLeaseV1(new PresenceService().openLease(required(body.table(), "table"), body.pk())));
}
}
@@ -77,6 +77,6 @@ public class PostPresenceSpec extends CollaborationJsonSpec
{
var body = body(input, CollaborationPresenceSchemas.CollaborationPresenceRequestV1.class);
return (new CollaborationPresenceSchemas.CollaborationPresenceSnapshotV1(body.table(), body.pk(),
new PresenceService().heartbeat(body.connectionId(), required(body.table(), "table"), body.pk(), body.mode(), body.fieldName())));
new PresenceService().heartbeat(body.connectionId(), required(body.table(), "table"), body.pk(), body.mode(), body.fieldName(), body.sequence())));
}
}
@@ -76,7 +76,7 @@ public class PutAssignmentSpec extends CollaborationJsonSpec
{
var body = body(input, CollaborationWorkSchemas.CollaborationAssignmentRequestV1.class);
return (CollaborationViews.assignment(new com.kingsrook.qbits.collaboration.assignment.AssignmentService()
.assign(table(input), pk(input), body.ownerUserId(), body.assigneeUserId())));
.assignRoles(table(input), pk(input), body.ownerUserId(), body.assigneeUserId(), input.getBody().keySet())));
}
}
@@ -0,0 +1,112 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.api;
import java.util.List;
import com.kingsrook.qbits.collaboration.presence.PresenceService;
import com.kingsrook.qqq.middleware.javalin.specs.BasicOperation;
import com.kingsrook.qqq.openapi.model.HttpMethod;
import com.kingsrook.qqq.openapi.model.Parameter;
import com.kingsrook.qqq.openapi.model.Schema;
/*******************************************************************************
** Renew presence with fresh parent access checks.
*******************************************************************************/
public class RenewPresenceLeaseSpec extends CollaborationJsonSpec
{
/*******************************************************************************
** Expiry is distinct from permissions, invalid input and service outages.
*******************************************************************************/
@Override
public List<com.kingsrook.qqq.middleware.javalin.specs.BasicResponse> defineAdditionalBasicResponses()
{
var responses = new java.util.ArrayList<>(super.defineAdditionalBasicResponses());
responses.add(new com.kingsrook.qqq.middleware.javalin.specs.BasicResponse(io.javalin.http.HttpStatus.CONFLICT,
"Presence lease expired; open a new lease.", "CollaborationPresenceExpiredV1"));
return (responses);
}
/*******************************************************************************
** Preserve standard framework mapping for every other exception.
*******************************************************************************/
@Override
protected void handleException(io.javalin.http.Context context, Exception exception)
{
if(com.kingsrook.qqq.backend.core.utils.ExceptionUtils.findClassInRootChain(exception,
com.kingsrook.qbits.collaboration.presence.PresenceLeaseExpiredException.class) != null)
{
context.status(409).contentType("application/json").result(com.kingsrook.qqq.backend.core.utils.JsonUtils.toJson(
new CollaborationPresenceSchemas.CollaborationPresenceExpiredV1("PRESENCE_LEASE_EXPIRED")));
}
else
{
super.handleException(context, exception);
}
}
/*******************************************************************************
** Describe the extension operation.
*******************************************************************************/
@Override
public BasicOperation defineBasicOperation()
{
return (operation(HttpMethod.POST, "/presence/leases/renew", "Renew issued presence lease"));
}
/*******************************************************************************
** Explicit query parameters for polling.
*******************************************************************************/
@Override
public List<Parameter> defineRequestParameters()
{
return (parameters());
}
/*******************************************************************************
** Name the versioned response schema.
*******************************************************************************/
@Override
protected String getResponseSchemaName()
{
return ("CollaborationPresenceSnapshotV1");
}
/*******************************************************************************
** Declare a typed JSON request body.
*******************************************************************************/
@Override
protected Schema defineRequestBodySchema()
{
return (new Schema().withRefToSchema("CollaborationPresenceRequestV1"));
}
/*******************************************************************************
** Services derive identity from QContext and enforce visibility.
*******************************************************************************/
@Override
protected Object handle(JsonInput input) throws Exception
{
var body = body(input, CollaborationPresenceSchemas.CollaborationPresenceRequestV1.class);
return (new CollaborationPresenceSchemas.CollaborationPresenceSnapshotV1(body.table(), body.pk(),
new PresenceService().renewLease(body.connectionId(), required(body.table(), "table"), body.pk(), body.mode(), body.fieldName(), body.sequence())));
}
}
@@ -267,19 +267,37 @@ public final class CollaborationHome
List<Map<String, Object>> rows = new ArrayList<>();
if(admin)
{
var registry = PresenceRegistry.get(QContext.getQInstance());
Map<String, Integer> connections = registry.connectionCounts();
registry.counts().forEach((table, viewers) ->
var instance = QContext.getQInstance();
var redis = com.kingsrook.qbits.collaboration.presence.RedisPresenceConfig.forInstance(instance);
Map<String, Integer> counts;
if(redis == null)
{
counts = PresenceRegistry.get(instance).counts();
}
else
{
counts = new java.util.TreeMap<>();
var backend = new com.kingsrook.qbits.collaboration.presence.RedisPresenceBackend(redis);
// Read only known table rooms in the administrator's authenticated tenant.
// Never scan Redis keys or silently substitute local counts on failure.
for(String tableName : instance.getTables().keySet())
{
int viewers = backend.snapshot(tableName, null).size();
if(viewers > 0)
{
counts.put(tableName, viewers);
}
}
}
counts.forEach((table, viewers) ->
{
Map<String, Object> row = new LinkedHashMap<>();
row.put("table", tableLabel(table));
row.put("viewers", viewers);
row.put("connections", connections.getOrDefault(table, viewers));
rows.add(row);
});
}
return (table("Who's here", admin ? NOBODY_HERE : PRESENCE_ADMIN_ONLY, rows, column("Table", "table", "left"), column("People", "viewers", "right"),
column("Connections", "connections", "right")));
return (table("Who's here", admin ? NOBODY_HERE : PRESENCE_ADMIN_ONLY, rows, column("Table", "table", "left"), column("People", "viewers", "right")));
}
@@ -53,15 +53,42 @@ public class AssignmentService
** Full replacement of both roles; null clears a role.
*******************************************************************************/
public QRecord assign(String table, Serializable recordId, String ownerUserId, String assigneeUserId) throws QException
{
return (assignRoles(table, recordId, ownerUserId, assigneeUserId, java.util.Set.of("ownerUserId", "assigneeUserId")));
}
/*******************************************************************************
** Omitted roles remain unchanged; an explicitly supplied null clears a role.
*******************************************************************************/
public QRecord assignRoles(String table, Serializable recordId, String ownerUserId, String assigneeUserId, java.util.Set<String> roles) throws QException
{
ParentRecordAccess.requireFeature(table, recordId, "assign");
String id = FollowService.canonicalId(table, recordId);
if(roles == null || roles.isEmpty() || !java.util.Set.of("ownerUserId", "assigneeUserId").containsAll(roles))
{
throw (new com.kingsrook.qqq.backend.core.exceptions.QBadRequestException("Supply ownerUserId or assigneeUserId; omitted roles remain unchanged."));
}
var instance = com.kingsrook.qqq.backend.core.context.QContext.getQInstance();
var ownership = com.kingsrook.qqq.backend.core.actions.collaboration.RecordFeaturesResolver.resolve(instance, instance.getTable(table)).getOwnership();
if(ownership != null && ((roles.contains("ownerUserId") && ownership.getOwnerFieldName() != null)
|| (roles.contains("assigneeUserId") && ownership.getAssigneeFieldName() != null)))
{
throw (new com.kingsrook.qqq.backend.core.exceptions.QBadRequestException("Update native ownership fields through the parent record's versioned update endpoint."));
}
WorkSupport.requireReader(ownerUserId, table, id);
WorkSupport.requireReader(assigneeUserId, table, id);
synchronized(CollaborationRecords.config())
{
Optional<QRecord> old = get(table, id);
QRecord change = new QRecord().withValue("ownerUserId", ownerUserId).withValue("assigneeUserId", assigneeUserId);
QRecord change = new QRecord();
if(roles.contains("ownerUserId"))
{
change.setValue("ownerUserId", ownerUserId);
}
if(roles.contains("assigneeUserId"))
{
change.setValue("assigneeUserId", assigneeUserId);
}
QRecord row;
if(old.isEmpty())
{
@@ -73,7 +100,7 @@ public class AssignmentService
row = get(table, id).orElseThrow();
}
ActivityWriter.write(table, id, "ASSIGNED", "Record assignment changed", CollaborationTables.RECORD_ASSIGNMENT, row.getValueString("id"));
if(assigneeUserId != null && (old.isEmpty() || !Objects.equals(old.get().getValueString("assigneeUserId"), assigneeUserId)))
if(roles.contains("assigneeUserId") && assigneeUserId != null && (old.isEmpty() || !Objects.equals(old.get().getValueString("assigneeUserId"), assigneeUserId)))
{
new FollowService().ensureFollower(assigneeUserId, table, id, "ASSIGNED");
if(!assigneeUserId.equals(PersonalRecords.currentUserId()))
@@ -55,6 +55,7 @@ public class AttachmentPreviewCustomizer implements TableCustomizerInterface
List<QRecord> filter(List<QRecord> records, boolean isGet, boolean generateUrls, AttachmentsService service) throws QException
{
List<QRecord> visible = new ArrayList<>();
ParentRecordAccess.ResultSetParents parents = new ParentRecordAccess.ResultSetParents();
for(QRecord record : records)
{
try
@@ -63,7 +64,7 @@ public class AttachmentPreviewCustomizer implements TableCustomizerInterface
{
throw (new QNotFoundException("Not found"));
}
ParentRecordAccess.requireVisibleParent(record.getValueString("tableName"), record.getValueString("recordId"));
parents.requireVisible(record.getValueString("tableName"), record.getValueString("recordId"));
}
catch(QNotFoundException expected)
{
@@ -47,14 +47,25 @@ import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSTableBackendDe
** pair, on the native QQQ postgres backend (vendor and backend type "postgres").
** Observation only: no schema migration and no session or server setting is
** changed at runtime.
**
** Amazon Aurora PostgreSQL 17 is the one qualified variant: it is held to every
** rule here except full_page_writes, which Aurora reports as "off" and does not
** let a deployment change (see tornPagesPrevented).
*******************************************************************************/
final class PostgresJournalDatabase
{
static final String VENDOR = "postgres";
private static final String SETTINGS = "SELECT current_setting('server_version_num')::int, current_setting('transaction_read_only'), pg_is_in_recovery(),"
/////////////////////////////////////////////////////////////////////////////////////////////
// Column 11 detects Amazon Aurora PostgreSQL inside the same statement. to_regprocedure //
// returns NULL when no aurora_version() function resolves, so it is true on Aurora, false //
// on stock PostgreSQL 17 and never raises. Calling aurora_version() directly would abort //
// the transaction on stock PostgreSQL, and pg_proc does not list the function on Aurora. //
/////////////////////////////////////////////////////////////////////////////////////////////
static final String SETTINGS = "SELECT current_setting('server_version_num')::int, current_setting('transaction_read_only'), pg_is_in_recovery(),"
+ " current_setting('synchronous_commit'), current_setting('fsync'), current_setting('full_page_writes'), current_setting('TimeZone'),"
+ " (SELECT setting::bigint FROM pg_settings WHERE name='lock_timeout'), current_setting('transaction_isolation'), current_database()";
+ " (SELECT setting::bigint FROM pg_settings WHERE name='lock_timeout'), current_setting('transaction_isolation'), current_database(),"
+ " to_regprocedure('aurora_version()') IS NOT NULL";
private static final String TABLES = "SELECT c.relname, c.relkind, c.relpersistence FROM pg_class c JOIN pg_namespace n ON n.oid=c.relnamespace"
+ " WHERE n.nspname=current_schema() AND c.relname = ANY(?)";
@@ -124,7 +135,8 @@ final class PostgresJournalDatabase
try(ResultSet settings = statement.executeQuery(SETTINGS))
{
if(!settings.next() || settings.getInt(1) / 10000 != 17 || !"off".equals(settings.getString(2)) || settings.getBoolean(3)
|| !Set.of("on", "remote_apply").contains(settings.getString(4)) || !"on".equals(settings.getString(5)) || !"on".equals(settings.getString(6))
|| !Set.of("on", "remote_apply").contains(settings.getString(4)) || !"on".equals(settings.getString(5))
|| !tornPagesPrevented(settings.getString(6), settings.getBoolean(11))
|| !Set.of("UTC", "Etc/UTC").contains(settings.getString(7)) || settings.getLong(8) < 1 || settings.getLong(8) > 5000
|| !"read committed".equals(settings.getString(9)) || !catalog.equals(settings.getString(10)))
{
@@ -147,6 +159,28 @@ final class PostgresJournalDatabase
/*******************************************************************************
** The torn-page rule, as a pure decision over the observed full_page_writes
** value and the Aurora detection column.
**
** Stock PostgreSQL must report full_page_writes "on": after a crash that leaves
** a partly written page, recovery repairs it from the full page image in the
** write-ahead log, so "off" there can lose committed journal rows.
**
** Amazon Aurora PostgreSQL reports full_page_writes "off" (measured on engine
** 17.9: source default, and the parameter is not in the cluster parameter group,
** so a deployment cannot turn it on). Its storage layer does not write torn
** pages, so the protection is not needed there. "off" is therefore accepted
** only when the server is Aurora; every other rule (synchronous_commit, fsync,
** recovery, isolation, time zone, lock timeout) stays as strict on Aurora.
*******************************************************************************/
static boolean tornPagesPrevented(String fullPageWrites, boolean aurora)
{
return ("on".equals(fullPageWrites) || (aurora && "off".equals(fullPageWrites)));
}
/*******************************************************************************
** Permanent (logged) ordinary tables only, deterministic text comparison, and
** complete valid non-partial unique indexes for the journal's two identities.
@@ -83,7 +83,7 @@ public final class CustomFieldStorage
}
if(!postgres(table))
{
return (new Availability(false, "Backend does not support JsonPathStorage; EAV fields are read-only filters (values are not selected)."));
return (new Availability(false, "Backend does not support JsonPathStorage; EAV fields support read-only values and filters."));
}
try(Connection connection = connection(table); var statement = connection.createStatement())
{
@@ -92,7 +92,7 @@ public final class CustomFieldStorage
}
catch(SQLException | IllegalArgumentException e)
{
return (new Availability(false, "JSON column unavailable; EAV fields are read-only filters. " + e.getMessage()));
return (new Availability(false, "JSON column unavailable; EAV fields support read-only values and filters. " + e.getMessage()));
}
}
@@ -0,0 +1,101 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.customfields;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.collaboration.CollaborationRecords;
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.actions.tables.query.VirtualFieldValueProviderInterface;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordReferenceTableMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
import com.kingsrook.qqq.backend.core.utils.ValueUtils;
/*******************************************************************************
** Supply EAV values for the parent records selected by the core query pipeline.
** Do not re-read parents here: that would recursively execute this provider.
*******************************************************************************/
public class CustomFieldValueProvider implements VirtualFieldValueProviderInterface
{
/*******************************************************************************
** Bound each lookup and retain the caller's transaction through service storage.
*******************************************************************************/
@Override
public void fill(QTableMetaData table, String fieldName, List<QRecord> records, QueryInput input) throws QException
{
String name = fieldName.substring(3);
var definitions = CollaborationRecords.query(CollaborationTables.FIELD_DEFINITION, new QQueryFilter()
.withCriteria("tableName", QCriteriaOperator.EQUALS, table.getName()).withCriteria("name", QCriteriaOperator.EQUALS, name)
.withCriteria("status", QCriteriaOperator.EQUALS, "ACTIVE"), input.getTransaction());
if(definitions.size() != 1)
{
throw (new QException("Custom field is unavailable"));
}
String column = FieldCriteriaExpander.valueColumn(definitions.getFirst().getValueString("type"));
for(List<QRecord> page : CollectionUtils.getPages(records, 500))
{
List<String> ids = page.stream().map(record -> String.valueOf(record.getValue(table.getPrimaryKeyField()))).distinct().toList();
Map<String, QRecord> values = new HashMap<>();
for(QRecord value : valuesForSelectedParents(new QQueryFilter()
.withCriteria("tableName", QCriteriaOperator.EQUALS, table.getName()).withCriteria("fieldName", QCriteriaOperator.EQUALS, name)
.withCriteria("parentDeleted", QCriteriaOperator.EQUALS, false).withCriteria("recordId", QCriteriaOperator.IN, ids), input))
{
if(values.put(value.getValueString("recordId"), value) != null)
{
throw (new QException("Duplicate custom field values"));
}
}
for(QRecord record : page)
{
QRecord value = values.get(String.valueOf(record.getValue(table.getPrimaryKeyField())));
record.setValue(fieldName, value == null ? null : ValueUtils.getValueAsFieldType(table.getVirtualField(fieldName).getType(), value.getValue(column)));
}
}
}
/*******************************************************************************
** The enclosing query already selected these parents under its access policy.
** Remove only recursive parent verification on a private child metadata clone;
** preserve child locks and never install this metadata on the instance.
*******************************************************************************/
private List<QRecord> valuesForSelectedParents(QQueryFilter filter, QueryInput origin) throws QException
{
QTableMetaData metadata = QContext.getQInstance().getTable(CollaborationTables.FIELD_VALUE).clone();
metadata.withCustomizers(null);
metadata.getSupplementalMetaData().remove(RecordReferenceTableMetaData.NAME);
QueryInput input = new QueryInput(CollaborationTables.FIELD_VALUE).withFilter(filter)
.withShouldFetchHeavyFields(true).withTransaction(origin.getTransaction());
input.setTableMetaData(metadata);
input.withPermissionBypass("EAV provider reads only parents selected by the enclosing query.");
return (new QueryAction().execute(input).getRecords());
}
}
@@ -43,6 +43,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QVirtualFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.ValueRangeBehavior;
import com.kingsrook.qqq.backend.core.model.metadata.fields.ValueTooLongBehavior;
import com.kingsrook.qqq.backend.core.model.metadata.help.QHelpContent;
import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSource;
import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSourceType;
@@ -58,7 +59,10 @@ public class CustomFieldsProvider implements RuntimeMetaDataProviderInterface
@Override
public RuntimeMetaDataContribution getContribution(QInstance instance) throws QException
{
com.kingsrook.qbits.collaboration.presence.PresenceRegistry.get(instance);
if(com.kingsrook.qbits.collaboration.presence.RedisPresenceConfig.forInstance(instance) == null)
{
com.kingsrook.qbits.collaboration.presence.PresenceRegistry.get(instance);
}
var result = new RuntimeMetaDataContribution();
var overrides = new FeatureSettingsProvider().getOverrides(instance);
List<QRecord> definitions = CollaborationRecords.query(CollaborationTables.FIELD_DEFINITION,
@@ -119,10 +123,15 @@ public class CustomFieldsProvider implements RuntimeMetaDataProviderInterface
String name = "cf_" + definition.getValueString("name");
QFieldType type = type(definition.getValueString("type"));
QFieldMetaData field = json ? new QFieldMetaData(name, type).withBackendStorage(new JsonPathStorage().withColumnName("qqq_custom_fields").withKey(definition.getValueString("name")))
: new QVirtualFieldMetaData(name, type).withIsEditable(false).withIsQueryCriteria(true).withIsQuerySelectable(false)
: new QVirtualFieldMetaData(name, type).withIsEditable(false).withIsQueryCriteria(true).withIsQuerySelectable(true)
.withValueProvider(new QCodeReference(CustomFieldValueProvider.class))
.withCriteriaExpander(new QCodeReference(FieldCriteriaExpander.class));
field.withLabel(definition.getValueString("label")).withIsRequired(json && Boolean.TRUE.equals(definition.getValueBoolean("required")))
.withMaxLength(definition.getValueInteger("maxLength"));
if(type == QFieldType.STRING && definition.getValueInteger("maxLength") != null)
{
field.withBehavior(ValueTooLongBehavior.ERROR);
}
if(definition.getValue("defaultValue") != null && json)
{
field.withDefaultValue(com.kingsrook.qqq.backend.core.utils.ValueUtils.getValueAsFieldType(type, definition.getValue("defaultValue")));
@@ -89,11 +89,20 @@ public final class EmailCollaborationNotifier implements CollaborationNotifier
*******************************************************************************/
@Override
public void notify(CollaborationNotification n)
{
deliver(n);
}
/*******************************************************************************
** A receipt means the queue accepted this event, including an existing duplicate.
*******************************************************************************/
@Override
public boolean deliver(CollaborationNotification n)
{
EmailApiTemplate template = templateFor(n.category());
if(template == null)
{
return;
return (false);
}
try
{
@@ -101,7 +110,7 @@ public final class EmailCollaborationNotifier implements CollaborationNotifier
if(!StringUtils.hasContent(address))
{
LOG.info("Collaboration email skipped: no address for recipient", logPair("userId", n.recipientUserId()));
return;
return (false);
}
EmailApi.ensureTemplate(template);
String eventId = n.sourceId() != null ? n.sourceId() : n.sourceRecordId();
@@ -123,10 +132,12 @@ public final class EmailCollaborationNotifier implements CollaborationNotifier
LOG.info("Collaboration email not queued", logPair("userId", n.recipientUserId()), logPair("status", result.status()),
logPair("messageId", result.messageId()));
}
return (result.status() == EmailApiStatus.QUEUED || result.status() == EmailApiStatus.DUPLICATE);
}
catch(QException | RuntimeException e)
{
LOG.warn("Collaboration email failed", e, logPair("userId", n.recipientUserId()), logPair("category", n.category()));
return (false);
}
}
@@ -183,6 +194,7 @@ public final class EmailCollaborationNotifier implements CollaborationNotifier
{
case MENTION -> "mention";
case SHARE -> "share";
case FOLLOW -> "follow";
default -> "task";
});
}
@@ -199,6 +211,7 @@ public final class EmailCollaborationNotifier implements CollaborationNotifier
case MENTION -> mentionTemplate();
case SHARE -> shareTemplate();
case TASK -> taskReminderTemplate();
case FOLLOW -> definition("collaboration_follow", "Followed record changed", "Changes to {{recordLabel}}", "A record you follow has changed.\n\n[View record]({{link}})");
default -> null;
});
}
@@ -193,12 +193,17 @@ public class FollowChangeBatcher implements RecordChangeListenerInterface
String user = batch.getValueString("userId");
List<QRecord> follows = CollaborationRecords.query(CollaborationTables.FOLLOW,
CollaborationRecords.parent(table, id).withCriteria("userId", QCriteriaOperator.EQUALS, user));
if(!follows.isEmpty() && eligible(follows.getFirst(), now))
if(!follows.isEmpty() && eligible(follows.getFirst(), now)
&& new com.kingsrook.qbits.collaboration.security.CollaborationRecordAccessChecker().canRead(user, table, id).orElse(false))
{
Notifications.send(new CollaborationNotification(user, "Changes to a followed record", batch.getValueString("changes"),
CollaborationNotification.Category.FOLLOW, table, id, Notifications.link(table, id)));
CollaborationNotification.Category.FOLLOW, table, id, Notifications.link(table, id)).withEvent(batch.getValue("id"), null),
() -> CollaborationRecords.delete(CollaborationTables.FOLLOW_DIGEST, batch.getValue("id")));
}
else
{
CollaborationRecords.delete(CollaborationTables.FOLLOW_DIGEST, batch.getValue("id"));
}
CollaborationRecords.delete(CollaborationTables.FOLLOW_DIGEST, batch.getValue("id"));
}
}
}
@@ -23,18 +23,19 @@ package com.kingsrook.qbits.collaboration.mywork;
import java.time.Instant;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.HashMap;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.collaboration.CollaborationRecords;
import com.kingsrook.qbits.collaboration.PersonalRecords;
import com.kingsrook.qbits.collaboration.assignment.AssignmentService;
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
import com.kingsrook.qbits.collaboration.security.ParentRecordAccess;
import com.kingsrook.qbits.collaboration.tasks.TasksService;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.exceptions.QNotFoundException;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
@@ -43,18 +44,94 @@ import com.kingsrook.qqq.backend.core.model.data.QRecord;
*******************************************************************************/
public class MyWorkService
{
private static final int PAGE_SIZE = 200;
private static final Comparator<QRecord> TASK_ORDER = Comparator.comparing((QRecord row) -> row.getValueInstant("dueAt"), Comparator.nullsLast(Comparator.naturalOrder()))
.thenComparing(row -> row.getValueInteger("id"));
/*******************************************************************************
** Open task count and first ten, assigned record counts, and unread mentions.
*******************************************************************************/
public Summary summary() throws QException
{
List<QRecord> tasks = new TasksService().mineRows("OPEN");
String user = PersonalRecords.currentUserId();
List<QRecord> tasks = new ArrayList<>();
long taskCount = scanVisible(CollaborationTables.TASK, new QQueryFilter()
.withCriteria("assigneeUserId", QCriteriaOperator.EQUALS, user)
.withCriteria("status", QCriteriaOperator.EQUALS, "OPEN"), row ->
{
tasks.add(row);
tasks.sort(TASK_ORDER);
if(tasks.size() > 10)
{
tasks.removeLast();
}
});
Map<String, Long> assignments = new LinkedHashMap<>();
for(QRecord row : new AssignmentService().mineRows(null, true))
scanVisible(CollaborationTables.RECORD_ASSIGNMENT, new QQueryFilter()
.withCriteria("assigneeUserId", QCriteriaOperator.EQUALS, user),
row -> assignments.merge(row.getValueString("tableName"), 1L, Long::sum));
long mentions = scanVisible(CollaborationTables.MENTION, new QQueryFilter()
.withCriteria("mentionedUserId", QCriteriaOperator.EQUALS, user)
.withCriteria("notifiedAt", QCriteriaOperator.IS_NOT_BLANK).withCriteria("readAt", QCriteriaOperator.IS_BLANK), row ->
{
// Only the visible count is needed for mentions.
});
return (new Summary(taskCount, List.copyOf(tasks), assignments, mentions));
}
/*******************************************************************************
** Bound retained rows and visibility decisions to one page. A starting high
** watermark prevents concurrent inserts from extending this scan indefinitely.
** Counts remain exact for visible rows scanned; no cross-request auth cache.
*******************************************************************************/
private long scanVisible(String table, QQueryFilter filter, java.util.function.Consumer<QRecord> consumer) throws QException
{
filter.withCriteria("parentDeleted", QCriteriaOperator.EQUALS, false);
List<QRecord> last = CollaborationRecords.query(table, filter.clone().withOrderBy(new QFilterOrderBy("id", false)).withLimit(1));
if(last.isEmpty())
{
assignments.merge(row.getValueString("tableName"), 1L, Long::sum);
return (0);
}
int maximum = last.getFirst().getValueInteger("id");
Integer cursor = null;
long count = 0;
while(true)
{
QQueryFilter pageFilter = filter.clone().withCriteria("id", QCriteriaOperator.LESS_THAN_OR_EQUALS, maximum)
.withOrderBy(new QFilterOrderBy("id")).withLimit(PAGE_SIZE);
if(cursor != null)
{
pageFilter.withCriteria("id", QCriteriaOperator.GREATER_THAN, cursor);
}
List<QRecord> rows = CollaborationRecords.query(table, pageFilter);
Map<List<String>, Boolean> parents = new HashMap<>();
for(QRecord row : rows)
{
List<String> parent = java.util.Arrays.asList(row.getValueString("tableName"), row.getValueString("recordId"));
if(!parents.containsKey(parent))
{
try
{
ParentRecordAccess.requireVisibleParent(parent.get(0), parent.get(1));
parents.put(parent, true);
}
catch(QNotFoundException expected)
{
parents.put(parent, false);
}
}
if(parents.get(parent))
{
consumer.accept(row);
count++;
}
}
if(rows.size() < PAGE_SIZE || rows.getLast().getValueInteger("id") >= maximum)
{
return (count);
}
cursor = rows.getLast().getValueInteger("id");
}
return (new Summary((long) tasks.size(), tasks.stream().limit(10).toList(), assignments, (long) unread().size()));
}
/*******************************************************************************
@@ -31,4 +31,13 @@ public interface CollaborationNotifier
** Deliver a recipient-authorized notification.
*******************************************************************************/
void notify(CollaborationNotification n);
/*******************************************************************************
** Legacy transports signal failure by throwing; adapters may explicitly refuse.
*******************************************************************************/
default boolean deliver(CollaborationNotification notification)
{
notify(notification);
return (true);
}
}
@@ -34,4 +34,13 @@ public class NoOpNotifier implements CollaborationNotifier
public void notify(CollaborationNotification n)
{
}
/*******************************************************************************
** No transport cannot acknowledge delivery.
*******************************************************************************/
@Override
public boolean deliver(CollaborationNotification notification)
{
return (false);
}
}
@@ -102,7 +102,10 @@ public final class Notifications
if(!checkParent || new CollaborationRecordAccessChecker().canRead(notification.recipientUserId(), notification.sourceTableName(), notification.sourceRecordId()).orElse(false))
{
CollaborationNotifier notifier = notifier();
notifier.notify(notification);
if(!notifier.deliver(notification))
{
return;
}
if(!(notifier instanceof NoOpNotifier))
{
try
@@ -0,0 +1,56 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.presence;
import java.util.List;
import com.kingsrook.qqq.backend.core.exceptions.QException;
/*******************************************************************************
** Ephemeral storage boundary behind the permission-checking presence service.
*******************************************************************************/
public interface PresenceBackend
{
/***************************************************************************
** Reserve invisible renewal authority; null means capacity refusal.
***************************************************************************/
String openLease(String user, String name, String table, String pk) throws QException;
/***************************************************************************
** Renew existing authority or throw PresenceLeaseExpiredException.
***************************************************************************/
boolean renewLease(String user, String name, String connection, String table, String pk, String mode, String field, Long sequence) throws QException;
/***************************************************************************
** Renew a connection's ordered lease.
***************************************************************************/
boolean heartbeat(String user, String name, String connection, String table, String pk, String mode, String field, Long sequence) throws QException;
/***************************************************************************
** Release only the authenticated user's connection.
***************************************************************************/
void leave(String user, String connection, String table, String pk, Long sequence) throws QException;
/***************************************************************************
** Read a snapshot for subsequent recipient-specific permission filtering.
***************************************************************************/
List<PresenceRegistry.Viewer> snapshot(String table, String pk) throws QException;
}
@@ -0,0 +1,36 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.presence;
/*******************************************************************************
** Expired or absent renewal authority requires opening a fresh presence lease.
*******************************************************************************/
public final class PresenceLeaseExpiredException extends IllegalStateException
{
/***************************************************************************
** Never include caller identifiers or stored data in this message.
***************************************************************************/
public PresenceLeaseExpiredException()
{
super("Presence lease expired; open a new lease");
}
}
@@ -0,0 +1,36 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.presence;
/*******************************************************************************
** Shared presence is rebuilding after unknown or lost Redis state.
*******************************************************************************/
public final class PresenceRecoveringException extends IllegalStateException
{
/***************************************************************************
** Never include caller identifiers or stored data in this message.
***************************************************************************/
public PresenceRecoveringException()
{
super("Presence is recovering; active viewers are rejoining");
}
}
@@ -21,33 +21,18 @@
package com.kingsrook.qbits.collaboration.presence;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import java.io.IOException;
import java.io.Serializable;
import java.nio.charset.StandardCharsets;
import java.security.InvalidKeyException;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
import java.time.Clock;
import java.time.Instant;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.TreeMap;
import java.util.WeakHashMap;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
import com.fasterxml.jackson.annotation.JsonInclude;
import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.databind.SerializerProvider;
import com.fasterxml.jackson.databind.annotation.JsonSerialize;
import com.fasterxml.jackson.databind.ser.std.StdSerializer;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.realtime.RealtimeBus;
@@ -58,13 +43,12 @@ import com.kingsrook.qqq.backend.core.realtime.RealtimeMessage;
** Ephemeral per-pod presence replicated over a channel core refuses to clients.
** Publish outside the monitor: the in-memory bus waits for listener threads.
*******************************************************************************/
public final class PresenceRegistry implements AutoCloseable
public final class PresenceRegistry implements PresenceBackend, AutoCloseable
{
public static final String INTERNAL_CHANNEL = "instance:presence";
public static final long TTL_MILLIS = 45000;
public static final int MAX_ENTRIES = 50000;
private static final QLogger LOG = QLogger.getLogger(PresenceRegistry.class);
private static final byte[] SESSION_SECRET = sessionSecret();
private static final Map<QInstance, PresenceRegistry> REGISTRIES = new WeakHashMap<>();
private final Map<Key, Entry> entries = new HashMap<>();
private final RealtimeBusInterface bus;
@@ -133,32 +117,17 @@ public final class PresenceRegistry implements AutoCloseable
}
/*******************************************************************************
** Each tab has its own lease; user views merge the live leases. The lease
** keeps an opaque id of the server-side session (never the session uuid,
** which is a bearer credential) and the time it started (renewals keep it).
** Each tab has its own lease; user views merge the live leases.
*******************************************************************************/
private record Entry(String displayName, String mode, String fieldName, long at, String sessionId, long since)
private record Entry(String displayName, String mode, String fieldName, long at, Long sequence)
{
}
/*******************************************************************************
** Transport-independent snapshot entry, never a persisted directory row.
** Merged per user, connections counts the live leases behind it; per
** connection (connections = 1), connectionId is an opaque per-lease id,
** sessionId an opaque id of the signed-in session (HMAC, never reversible to
** the session uuid) and since the lease start (ISO-8601). None of the three
** is set on merged viewers.
*******************************************************************************/
public record Viewer(String userId, String displayName, String mode, String fieldName, Integer connections, @JsonInclude(JsonInclude.Include.NON_NULL) String connectionId,
@JsonInclude(JsonInclude.Include.NON_NULL) String sessionId, @JsonInclude(JsonInclude.Include.NON_NULL) String since) implements Serializable
public record Viewer(String userId, String displayName, String mode, String fieldName) implements Serializable
{
/*******************************************************************************
** A merged viewer behind a number of live leases.
*******************************************************************************/
public Viewer(String userId, String displayName, String mode, String fieldName, Integer connections)
{
this(userId, displayName, mode, fieldName, connections, null, null, null);
}
}
/*******************************************************************************
@@ -166,19 +135,21 @@ public final class PresenceRegistry implements AutoCloseable
*******************************************************************************/
public boolean heartbeat(String user, String displayName, String connection, String table, String pk, String mode, String fieldName)
{
return (heartbeat(user, displayName, connection, null, table, pk, mode, fieldName));
return (heartbeat(user, displayName, connection, table, pk, mode, fieldName, null));
}
/*******************************************************************************
** Renew a lease tied to the caller's server-side session. Only an opaque HMAC
** of the session uuid is kept and replicated; the uuid itself is dropped here.
** Client sequence orders delayed HTTP requests independently of arrival time.
*******************************************************************************/
public boolean heartbeat(String user, String displayName, String connection, String sessionUuid, String table, String pk, String mode, String fieldName)
public boolean heartbeat(String user, String displayName, String connection, String table, String pk, String mode, String fieldName, Long sequence)
{
if(connection != null && connection.startsWith("lease:"))
{
throw (new PresenceLeaseExpiredException());
}
expire();
var key = new Key(user, connection, table, pk);
long now = clock.millis();
var entry = new Entry(displayName, mode, fieldName, now, opaqueSessionId(sessionUuid), now);
var entry = new Entry(displayName, mode, fieldName, clock.millis(), sequence);
if(!apply(key, entry, false))
{
return (false);
@@ -187,14 +158,47 @@ public final class PresenceRegistry implements AutoCloseable
return (true);
}
/***************************************************************************
** Local mode issues local authority; HA applications must use shared Redis.
***************************************************************************/
@Override
public String openLease(String user, String displayName, String table, String pk)
{
expire();
String connection = "lease:" + java.util.UUID.randomUUID();
return (apply(new Key(user, connection, table, pk), new Entry(displayName, "PENDING", null, clock.millis(), 0L), false) ? connection : null);
}
/***************************************************************************
** Check renewal and update under one monitor; no late request creates a lease.
***************************************************************************/
@Override
public boolean renewLease(String user, String displayName, String connection, String table, String pk, String mode, String fieldName, Long sequence)
{
if(connection == null || !connection.startsWith("lease:") || sequence == null || sequence < 1 || sequence > 9007199254740991L
|| !List.of("VIEWING", "EDITING").contains(mode == null ? "" : mode))
{
throw (new IllegalArgumentException("A server-issued lease, positive safe sequence and valid mode are required"));
}
expire();
return (apply(new Key(user, connection, table, pk), new Entry(displayName, mode, fieldName, clock.millis(), sequence), false, true));
}
/*******************************************************************************
** Leaves use the same internal channel so every pod removes the lease.
*******************************************************************************/
public void leave(String user, String connection, String table, String pk)
{
leave(user, connection, table, pk, null);
}
/*******************************************************************************
** Departure participates in the same ordered client lease.
*******************************************************************************/
public void leave(String user, String connection, String table, String pk, Long sequence)
{
var key = new Key(user, connection, table, pk);
long now = clock.millis();
var entry = new Entry(null, null, null, now, null, now);
var entry = new Entry(null, null, null, clock.millis(), sequence);
apply(key, entry, true);
publish(key, entry, true);
}
@@ -204,6 +208,10 @@ public final class PresenceRegistry implements AutoCloseable
*******************************************************************************/
private void publish(Key key, Entry entry, boolean leave)
{
if(key.connection().startsWith("lease:"))
{
return;
}
Map<String, Serializable> data = new LinkedHashMap<>();
data.put("userId", key.user());
data.put("connectionId", key.connection());
@@ -213,8 +221,8 @@ public final class PresenceRegistry implements AutoCloseable
data.put("mode", entry.mode());
data.put("fieldName", entry.fieldName());
data.put("at", entry.at());
data.put("sessionId", entry.sessionId());
data.put("leave", leave);
data.put("sequence", entry.sequence());
bus.publish(new RealtimeMessage().withChannel(INTERNAL_CHANNEL).withEvent("presence.heartbeat").withData(data));
}
@@ -235,51 +243,61 @@ public final class PresenceRegistry implements AutoCloseable
}
expire();
long receivedAt = Math.min(at.longValue(), clock.millis());
// Best-effort replication cannot grant or restore issued renewal authority.
if(connection.startsWith("lease:"))
{
return;
}
if(receivedAt + TTL_MILLIS <= clock.millis())
{
return;
}
apply(new Key(user, connection, table, (String) data.get("pk")),
new Entry((String) data.get("displayName"), (String) data.get("mode"), (String) data.get("fieldName"), receivedAt, (String) data.get("sessionId"), receivedAt), Boolean.TRUE.equals(data.get("leave")));
new Entry((String) data.get("displayName"), (String) data.get("mode"), (String) data.get("fieldName"), receivedAt,
data.get("sequence") instanceof Number sequence ? sequence.longValue() : null), Boolean.TRUE.equals(data.get("leave")));
}
/*******************************************************************************
** Unchanged renewals extend TTL without generating client event traffic.
*******************************************************************************/
private boolean apply(Key key, Entry entry, boolean leave)
{
return (apply(key, entry, leave, false));
}
/*******************************************************************************
** Local issued authority is checked atomically with the mutation.
*******************************************************************************/
private boolean apply(Key key, Entry entry, boolean leave, boolean requireLive)
{
boolean changed;
synchronized(this)
{
Entry old = entries.get(key);
if(old != null && old.at() > entry.at())
if(requireLive && (old == null || old.mode() == null || old.at() + TTL_MILLIS <= clock.millis()))
{
throw (new PresenceLeaseExpiredException());
}
if(old != null && old.sequence() != null
&& (entry.sequence() == null || entry.sequence() < old.sequence()
|| (entry.sequence().equals(old.sequence()) && (!leave || old.mode() == null))))
{
return (true);
}
if(!leave && old == null && entries.size() >= capacity)
if(old != null && old.sequence() == null && entry.sequence() == null
&& (old.at() > entry.at() || (old.at() == entry.at() && old.mode() == null && !leave)))
{
return (true);
}
if(old == null && entries.size() >= capacity)
{
LOG.warn("Presence capacity reached; refusing a new target");
return (false);
}
List<Viewer> before = projection(key.table(), key.pk());
if(leave)
{
entries.remove(key);
}
else
{
/////////////////////////////////////////////////////////////////
// a renewal keeps the lease start (pods may differ by a little: //
// the earliest start wins) and the first known session id //
/////////////////////////////////////////////////////////////////
if(old != null)
{
entry = new Entry(entry.displayName(), entry.mode(), entry.fieldName(), entry.at(),
old.sessionId() != null ? old.sessionId() : entry.sessionId(), Math.min(old.since(), entry.since()));
}
entries.put(key, entry);
}
changed = !before.equals(projection(key.table(), key.pk()));
List<Viewer> before = snapshotInternal(key.table(), key.pk());
// Keep a bounded departure marker until old messages can no longer be fresh.
entries.put(key, entry);
changed = !before.equals(snapshotInternal(key.table(), key.pk()));
}
if(changed)
{
@@ -326,137 +344,16 @@ public final class PresenceRegistry implements AutoCloseable
*******************************************************************************/
private List<Viewer> snapshotInternal(String table, String pk)
{
Map<String, Entry> users = new TreeMap<>();
Map<String, Integer> connections = new TreeMap<>();
for(var row : live(table, pk))
{
users.merge(row.getKey().user(), row.getValue(), (old, next) ->
(!Objects.equals(old.mode(), next.mode()) ? "EDITING".equals(next.mode()) : next.at() >= old.at()) ? next : old);
connections.merge(row.getKey().user(), 1, Integer::sum);
}
return (users.entrySet().stream().map(row -> new Viewer(row.getKey(), row.getValue().displayName(), row.getValue().mode(),
pk == null ? null : row.getValue().fieldName(), connections.get(row.getKey()))).toList());
}
/*******************************************************************************
** One viewer per live lease, ordered by user, lease start and opaque id. A
** table target counts each connection once per record it holds a lease on.
*******************************************************************************/
private List<Viewer> connectionsInternal(String table, String pk)
{
return (live(table, pk).stream()
.map(row -> new Viewer(row.getKey().user(), row.getValue().displayName(), row.getValue().mode(), pk == null ? null : row.getValue().fieldName(), 1,
opaqueConnectionId(row.getKey().user(), row.getKey().connection()), row.getValue().sessionId(), Instant.ofEpochMilli(row.getValue().since()).toString()))
.sorted(java.util.Comparator.comparing(Viewer::userId).thenComparing(Viewer::since).thenComparing(Viewer::connectionId))
.toList());
}
/*******************************************************************************
** Live leases on a table (any record when pk is null) or on one record.
*******************************************************************************/
private List<Map.Entry<Key, Entry>> live(String table, String pk)
{
List<Map.Entry<Key, Entry>> result = new ArrayList<>();
Map<String, Entry> users = new java.util.TreeMap<>();
for(var row : entries.entrySet())
{
if(row.getKey().table().equals(table) && (pk == null || Objects.equals(row.getKey().pk(), pk)) && row.getValue().at() + TTL_MILLIS > clock.millis())
if(row.getValue().mode() != null && !"PENDING".equals(row.getValue().mode()) && row.getKey().table().equals(table) && (pk == null || Objects.equals(row.getKey().pk(), pk)) && row.getValue().at() + TTL_MILLIS > clock.millis())
{
result.add(row);
users.merge(row.getKey().user(), row.getValue(), (old, next) ->
(!Objects.equals(old.mode(), next.mode()) ? "EDITING".equals(next.mode()) : next.at() >= old.at()) ? next : old);
}
}
return (result);
}
/*******************************************************************************
** What clients can observe: merged viewers and per-connection viewers. Lease
** renewals that change neither generate no event.
*******************************************************************************/
private List<Viewer> projection(String table, String pk)
{
List<Viewer> result = new ArrayList<>(snapshotInternal(table, pk));
result.addAll(connectionsInternal(table, pk));
return (result);
}
/*******************************************************************************
** Opaque, stable per-lease id: "c" and the first 16 hex digits of
** SHA-256(userId + "\n" + connectionId). A client can recognise its own
** connection; nobody learns another client's connection id.
*******************************************************************************/
public static String opaqueConnectionId(String userId, String connectionId)
{
try
{
byte[] digest = MessageDigest.getInstance("SHA-256").digest((userId + "\n" + connectionId).getBytes(StandardCharsets.UTF_8));
StringBuilder result = new StringBuilder("c");
for(int i = 0; i < 8; i++)
{
result.append(String.format("%02x", digest[i]));
}
return (result.toString());
}
catch(NoSuchAlgorithmException e)
{
throw (new IllegalStateException("SHA-256 is unavailable", e));
}
}
/*******************************************************************************
** Opaque session id: "s" and the first 16 hex digits of
** HMAC-SHA256(secret, sessionUuid), or null without a session. The secret is
** QQQ_PRESENCE_SESSION_SECRET (system property or environment; share it across
** pods for ids that agree between pods) or else random per process. Session
** uuids are bearer credentials and must never be stored or published.
*******************************************************************************/
public static String opaqueSessionId(String sessionUuid)
{
if(sessionUuid == null || sessionUuid.isBlank())
{
return (null);
}
try
{
Mac mac = Mac.getInstance("HmacSHA256");
mac.init(new SecretKeySpec(SESSION_SECRET, "HmacSHA256"));
byte[] digest = mac.doFinal(sessionUuid.getBytes(StandardCharsets.UTF_8));
StringBuilder result = new StringBuilder("s");
for(int i = 0; i < 8; i++)
{
result.append(String.format("%02x", digest[i]));
}
return (result.toString());
}
catch(NoSuchAlgorithmException | InvalidKeyException e)
{
throw (new IllegalStateException("HmacSHA256 is unavailable", e));
}
}
/*******************************************************************************
** Configured shared secret, or 32 random bytes for this process.
*******************************************************************************/
private static byte[] sessionSecret()
{
String configured = System.getProperty("QQQ_PRESENCE_SESSION_SECRET", System.getenv("QQQ_PRESENCE_SESSION_SECRET"));
if(configured != null && !configured.isBlank())
{
return (configured.getBytes(StandardCharsets.UTF_8));
}
byte[] random = new byte[32];
new SecureRandom().nextBytes(random);
return (random);
}
/*******************************************************************************
** One viewer per live lease (opt-in per-connection presence, #54).
*******************************************************************************/
public List<Viewer> connections(String table, String pk)
{
expire();
synchronized(this)
{
return (connectionsInternal(table, pk));
}
return (users.entrySet().stream().map(row -> new Viewer(row.getKey(), row.getValue().displayName(), row.getValue().mode(), pk == null ? null : row.getValue().fieldName())).toList());
}
/*******************************************************************************
@@ -467,27 +364,12 @@ public final class PresenceRegistry implements AutoCloseable
expire();
synchronized(this)
{
Map<String, Integer> result = new TreeMap<>();
Map<String, Integer> result = new java.util.TreeMap<>();
entries.keySet().stream().map(Key::table).distinct().forEach(table -> result.put(table, snapshotInternal(table, null).size()));
return (result);
}
}
/*******************************************************************************
** Live lease (connection) counts per table, next to the user counts above.
*******************************************************************************/
public Map<String, Integer> connectionCounts()
{
expire();
synchronized(this)
{
Map<String, Integer> result = new TreeMap<>();
entries.keySet().stream().map(Key::table).distinct().forEach(table -> result.put(table, live(table, null).size()));
result.values().removeIf(count -> count == 0);
return (result);
}
}
/*******************************************************************************
** Record focus stays on its record channel; table events contain table snapshots.
*******************************************************************************/
@@ -511,97 +393,23 @@ public final class PresenceRegistry implements AutoCloseable
{
data.put("pk", pk);
}
Viewers viewers = new Viewers();
Viewers leases = new Viewers();
ArrayList<LinkedHashMap<String, Serializable>> viewers = new ArrayList<>();
synchronized(this)
{
snapshotInternal(table, pk).forEach(viewer -> viewers.add(toMap(viewer)));
connectionsInternal(table, pk).forEach(viewer -> leases.add(toMap(viewer)));
for(Viewer viewer : snapshotInternal(table, pk))
{
LinkedHashMap<String, Serializable> value = new LinkedHashMap<>();
value.put("userId", viewer.userId());
value.put("displayName", viewer.displayName());
value.put("mode", viewer.mode());
value.put("fieldName", viewer.fieldName());
viewers.add(value);
}
}
data.put("viewers", viewers);
data.put("leaseCount", leases.size());
data.put("leases", leases);
bus.publish(new RealtimeMessage().withChannel(channel).withEvent("presence.changed").withData(data));
}
/*******************************************************************************
** Event shape of a viewer; per-connection fields only when set.
*******************************************************************************/
private static LinkedHashMap<String, Serializable> toMap(Viewer viewer)
{
LinkedHashMap<String, Serializable> value = new LinkedHashMap<>();
value.put("userId", viewer.userId());
value.put("displayName", viewer.displayName());
value.put("mode", viewer.mode());
value.put("fieldName", viewer.fieldName());
value.put("connections", viewer.connections());
if(viewer.connectionId() != null)
{
value.put("connectionId", viewer.connectionId());
if(viewer.sessionId() != null)
{
value.put("sessionId", viewer.sessionId());
}
value.put("since", viewer.since());
}
return (value);
}
/*******************************************************************************
** Event viewers list that is never omitted: the SSE writer serializes event data
** with NON_EMPTY inclusion, which would drop an empty list, but the schema
** declares viewers a required array, so the last leave must send [] (#52).
*******************************************************************************/
@JsonSerialize(using = ViewersSerializer.class)
public static final class Viewers extends ArrayList<LinkedHashMap<String, Serializable>>
{
}
/*******************************************************************************
** Writes each viewer with the caller's mapper settings and reports never empty.
*******************************************************************************/
public static final class ViewersSerializer extends StdSerializer<Viewers>
{
/*******************************************************************************
** Jackson instantiates the serializer reflectively.
*******************************************************************************/
public ViewersSerializer()
{
super(Viewers.class);
}
/*******************************************************************************
** Always an array, including the empty array after the last viewer leaves.
*******************************************************************************/
@Override
public void serialize(Viewers viewers, JsonGenerator generator, SerializerProvider provider) throws IOException
{
generator.writeStartArray(viewers, viewers.size());
for(LinkedHashMap<String, Serializable> viewer : viewers)
{
provider.defaultSerializeValue(viewer, generator);
}
generator.writeEndArray();
}
/*******************************************************************************
** NON_EMPTY inclusion consults this; an empty viewer list is still a value.
*******************************************************************************/
@Override
public boolean isEmpty(SerializerProvider provider, Viewers viewers)
{
return (false);
}
}
/*******************************************************************************
** Release listeners and timer on host shutdown or in tests.
*******************************************************************************/
@@ -0,0 +1,33 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.presence;
/*******************************************************************************
** Host resolves tenant scope from authenticated context, never request parameters.
*******************************************************************************/
public interface PresenceScopeProvider
{
/***************************************************************************
** Return a stable nonblank scope; single-tenant hosts explicitly return one.
***************************************************************************/
String tenantScope();
}
@@ -30,19 +30,62 @@ import com.kingsrook.qqq.backend.core.exceptions.QBadRequestException;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.exceptions.QNotFoundException;
import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordFeature;
import com.kingsrook.qqq.backend.core.realtime.PresenceFieldVisibility;
/*******************************************************************************
** Authenticated, permission-checked boundary for presence; nothing is persisted.
*******************************************************************************/
public class PresenceService
{
/*******************************************************************************
** Reserve invisible authority only for an authenticated, visible target.
*******************************************************************************/
public String openLease(String table, String pk) throws QException
{
String user = PersonalRecords.currentUserId();
String canonical = authorize(table, pk);
String name = QContext.getQSession().getUser().getFullName();
String lease = registry().openLease(user, name == null || name.isBlank() ? user : name, table, canonical);
if(lease == null)
{
throw (new com.kingsrook.qqq.backend.core.exceptions.QServiceUnavailableException("Presence capacity reached; retry later"));
}
return (lease);
}
/*******************************************************************************
** Issued renewals share the same identity, parent and field access checks.
*******************************************************************************/
public List<PresenceRegistry.Viewer> renewLease(String connection, String table, String pk, String mode, String field, Long sequence) throws QException
{
requireIssuedLease(connection, sequence);
return (renew(connection, table, pk, mode, field, sequence, true));
}
/*******************************************************************************
** Recheck visibility on every renewal and remove a refused caller's old lease.
*******************************************************************************/
public List<PresenceRegistry.Viewer> heartbeat(String connection, String table, String pk, String mode, String field) throws QException
{
return (heartbeat(connection, table, pk, mode, field, null));
}
/*******************************************************************************
** Ordered clients supply a monotonically increasing sequence per connection.
*******************************************************************************/
public List<PresenceRegistry.Viewer> heartbeat(String connection, String table, String pk, String mode, String field, Long sequence) throws QException
{
return (renew(connection, table, pk, mode, field, sequence, false));
}
/*******************************************************************************
** Keep legacy and issued requests on one permission and projection boundary.
*******************************************************************************/
private List<PresenceRegistry.Viewer> renew(String connection, String table, String pk, String mode, String field, Long sequence, boolean issued) throws QException
{
String user = PersonalRecords.currentUserId();
validateConnection(connection);
validateSequence(sequence);
pk = canonicalKey(table, pk);
try
{
@@ -50,34 +93,63 @@ public class PresenceService
}
catch(QException e)
{
registry().leave(user, connection, table, pk);
try
{
registry().leave(user, connection, table, pk, sequence);
}
catch(QException cleanupFailure)
{
// Preserve the access refusal; an unreachable store expires the old lease.
}
throw (e);
}
if(!List.of("VIEWING", "EDITING").contains(mode == null ? "" : mode))
{
throw (new QBadRequestException("Mode must be VIEWING or EDITING"));
}
if(field != null && (pk == null || QContext.getQInstance().getTable(table).getFields().get(field) == null))
if(field != null && (pk == null || !PresenceFieldVisibility.visibleFieldNames(table).contains(field)))
{
throw (new QBadRequestException("Unknown focused field"));
}
String name = QContext.getQSession().getUser().getFullName();
if(!registry().heartbeat(user, name == null || name.isBlank() ? user : name, connection, QContext.getQSession().getUuid(), table, pk, mode, field))
if(!issued && RedisPresenceConfig.forInstance(QContext.getQInstance()) != null)
{
throw (new QBadRequestException("Presence capacity reached; retry later"));
throw (new QBadRequestException("Shared presence requires a server-issued lease; use the presence/leases endpoints"));
}
return (registry().snapshot(table, pk));
String name = QContext.getQSession().getUser().getFullName();
String displayName = name == null || name.isBlank() ? user : name;
var backend = registry();
boolean accepted = issued ? backend.renewLease(user, displayName, connection, table, pk, mode, field, sequence)
: backend.heartbeat(user, displayName, connection, table, pk, mode, field, sequence);
if(!accepted)
{
throw (new com.kingsrook.qqq.backend.core.exceptions.QServiceUnavailableException("Presence capacity reached; retry later"));
}
return (visibleSnapshot(table, pk));
}
/*******************************************************************************
** Namespacing by user prevents one user from removing another user's connection.
*******************************************************************************/
public void leave(String connection, String table, String pk) throws QException
{
leave(connection, table, pk, null);
}
/*******************************************************************************
** Ordered departure cannot be undone by an earlier renewal arriving late.
*******************************************************************************/
public void leave(String connection, String table, String pk, Long sequence) throws QException
{
String user = PersonalRecords.currentUserId();
validateConnection(connection);
String canonical = authorize(table, pk);
registry().leave(user, connection, table, canonical);
validateSequence(sequence);
if(connection.startsWith("lease:"))
{
requireIssuedLease(connection, sequence);
}
// Removal is scoped to the caller; revoked readers must still release their lease.
String canonical = canonicalKey(table, pk);
registry().leave(user, connection, table, canonical, sequence);
}
/*******************************************************************************
@@ -86,16 +158,17 @@ public class PresenceService
public List<PresenceRegistry.Viewer> snapshot(String table, String pk) throws QException
{
PersonalRecords.currentUserId();
return (registry().snapshot(table, authorize(table, pk)));
return (visibleSnapshot(table, authorize(table, pk)));
}
/*******************************************************************************
** One viewer per live lease, behind the same access boundary as snapshot.
** Project a fresh recipient view without mutating shared registry entries.
*******************************************************************************/
public List<PresenceRegistry.Viewer> connections(String table, String pk) throws QException
private List<PresenceRegistry.Viewer> visibleSnapshot(String table, String pk) throws QException
{
PersonalRecords.currentUserId();
return (registry().connections(table, authorize(table, pk)));
var visible = PresenceFieldVisibility.visibleFieldNames(table);
return (registry().snapshot(table, pk).stream().map(viewer -> new PresenceRegistry.Viewer(viewer.userId(), viewer.displayName(), viewer.mode(),
viewer.fieldName() != null && visible.contains(viewer.fieldName()) ? viewer.fieldName() : null)).toList());
}
/*******************************************************************************
@@ -120,10 +193,6 @@ public class PresenceService
*******************************************************************************/
private String canonicalKey(String tableName, String pk) throws QNotFoundException
{
if(pk == null)
{
return (null);
}
try
{
var table = QContext.getQInstance().getTable(tableName);
@@ -131,6 +200,10 @@ public class PresenceService
{
throw (new IllegalArgumentException("Unknown table"));
}
if(pk == null)
{
return (null);
}
return (com.kingsrook.qqq.backend.core.utils.ValueUtils.getValueAsFieldType(table.getField(table.getPrimaryKeyField()).getType(), pk).toString());
}
catch(RuntimeException e)
@@ -151,10 +224,39 @@ public class PresenceService
}
/*******************************************************************************
** All endpoints on a pod share the eagerly installed instance registry.
** Issued handles require ordered activation and departure requests.
*******************************************************************************/
private PresenceRegistry registry()
private void requireIssuedLease(String connection, Long sequence) throws QBadRequestException
{
validateConnection(connection);
validateSequence(sequence);
if(!connection.startsWith("lease:") || sequence == null || sequence < 1)
{
throw (new QBadRequestException("A server-issued lease and positive sequence are required"));
}
}
/*******************************************************************************
** Restrict ordered leases to exact nonnegative JavaScript integers.
*******************************************************************************/
private void validateSequence(Long sequence) throws QBadRequestException
{
if(sequence != null && (sequence < 0 || sequence > 9007199254740991L))
{
throw (new QBadRequestException("Presence sequence must be a nonnegative safe integer"));
}
}
/*******************************************************************************
** Select the configured backend without creating an unused local registry.
*******************************************************************************/
private PresenceBackend registry() throws QException
{
var config = RedisPresenceConfig.forInstance(QContext.getQInstance());
if(config != null)
{
return (new RedisPresenceBackend(config));
}
return (PresenceRegistry.get(QContext.getQInstance()));
}
}
@@ -0,0 +1,54 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.presence;
import com.kingsrook.qbits.redis.api.RedisNamespace;
import com.kingsrook.qqq.backend.core.actions.customizers.QCodeLoader;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.realtime.RealtimeAudienceProvider;
/*******************************************************************************
** Presence hints are visible only within the authenticated storage namespace.
*******************************************************************************/
public class RedisPresenceAudience implements RealtimeAudienceProvider
{
public static final String EVENT = "presence.invalidated";
/***************************************************************************
** Recompute recipient scope on every delivery and replay; never trust the hint.
***************************************************************************/
@Override
public boolean allows(String audience) throws Exception
{
var config = RedisPresenceConfig.forInstance(QContext.getQInstance());
return (config != null && namespace(config).key("audience").equals(audience));
}
/***************************************************************************
** Storage and delivery must derive exactly the same host-owned tenant scope.
***************************************************************************/
public static RedisNamespace namespace(RedisPresenceConfig config) throws Exception
{
var scope = QCodeLoader.getAdHoc(PresenceScopeProvider.class, config.getScopeProvider()).tenantScope();
return (new RedisNamespace(config.getDeployment(), config.getApplication(), scope, "presence:" + config.getRecoveryGeneration()));
}
}
@@ -0,0 +1,190 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.presence;
import java.time.Duration;
import java.util.List;
import com.kingsrook.qbits.redis.api.RedisAccess;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QServiceUnavailableException;
/*******************************************************************************
** Request-scoped Redis adapter; tenant identity is re-derived for every request.
*******************************************************************************/
public class RedisPresenceBackend implements PresenceBackend
{
private final RedisPresenceStore store;
private final String audience;
/***************************************************************************
** Require a running named pool and an explicit host-derived tenant namespace.
***************************************************************************/
public RedisPresenceBackend(RedisPresenceConfig config) throws QServiceUnavailableException
{
try
{
var namespace = RedisPresenceAudience.namespace(config);
audience = namespace.key("audience");
var client = RedisAccess.scripts(QContext.getQInstance(), config.getConnectionName());
store = new RedisPresenceStore(client, namespace, Duration.ofMillis(PresenceRegistry.TTL_MILLIS), config.getCapacity());
}
catch(Exception failure)
{
throw (unavailable());
}
}
/***************************************************************************
** Reservations are invisible and do not require a client refresh hint.
***************************************************************************/
@Override
public String openLease(String user, String name, String table, String pk) throws QServiceUnavailableException
{
try
{
return (store.openLease(user, name, table, pk));
}
catch(RuntimeException failure)
{
throw (unavailable());
}
}
/***************************************************************************
** Preserve expiry as a rejoin signal, distinct from infrastructure failure.
***************************************************************************/
@Override
public boolean renewLease(String user, String name, String connection, String table, String pk, String mode, String field, Long sequence) throws QServiceUnavailableException
{
try
{
boolean accepted = store.renewLease(user, name, connection, table, pk, mode, field, sequence);
if(accepted)
{
changed(table, pk);
}
return (accepted);
}
catch(PresenceLeaseExpiredException failure)
{
throw (failure);
}
catch(RuntimeException failure)
{
throw (unavailable());
}
}
/***************************************************************************
** Do not fall back to isolated local state when a shared store is unavailable.
***************************************************************************/
@Override
public boolean heartbeat(String user, String name, String connection, String table, String pk, String mode, String field, Long sequence) throws QServiceUnavailableException
{
try
{
boolean accepted = store.heartbeat(user, name, connection, table, pk, mode, field, sequence);
if(accepted)
{
changed(table, pk);
}
return (accepted);
}
catch(RuntimeException failure)
{
throw (unavailable());
}
}
/***************************************************************************
** If unavailable, the existing lease still expires using Redis server time.
***************************************************************************/
@Override
public void leave(String user, String connection, String table, String pk, Long sequence) throws QServiceUnavailableException
{
try
{
if(!store.leave(user, connection, table, pk, sequence))
{
throw (unavailable());
}
changed(table, pk);
}
catch(RuntimeException failure)
{
throw (unavailable());
}
}
/***************************************************************************
** Failure is explicit; an empty result is reserved for a successfully read room.
***************************************************************************/
@Override
public List<PresenceRegistry.Viewer> snapshot(String table, String pk) throws QServiceUnavailableException
{
try
{
return (store.snapshot(table, pk));
}
catch(PresenceRecoveringException recovering)
{
throw (new QServiceUnavailableException("Presence is recovering; active viewers are rejoining. Please retry."));
}
catch(RuntimeException failure)
{
throw (unavailable());
}
}
/***************************************************************************
** Never expose client exceptions, endpoint details or credentials to browsers.
***************************************************************************/
private static QServiceUnavailableException unavailable()
{
return (new QServiceUnavailableException("Presence is temporarily unavailable. Please retry."));
}
/***************************************************************************
** Best-effort hints carry identifiers only; authorized snapshots remain truth.
***************************************************************************/
private void changed(String table, String pk)
{
try
{
var bus = com.kingsrook.qqq.backend.core.realtime.RealtimeBus.get(QContext.getQInstance());
bus.publish(new com.kingsrook.qqq.backend.core.realtime.RealtimeMessage()
.withChannel("table:" + table).withEvent(RedisPresenceAudience.EVENT).withAudience(audience)
.withData(java.util.Map.of("table", table)));
if(pk != null)
{
bus.publish(new com.kingsrook.qqq.backend.core.realtime.RealtimeMessage()
.withChannel("record:" + table + ":" + pk).withEvent(RedisPresenceAudience.EVENT).withAudience(audience)
.withData(java.util.Map.of("table", table, "pk", pk)));
}
}
catch(RuntimeException failure)
{
com.kingsrook.qqq.backend.core.logging.QLogger.getLogger(RedisPresenceBackend.class)
.warn("Presence hint delivery failed; clients must reconcile through authorized reads");
}
}
}
@@ -0,0 +1,174 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.presence;
import java.io.Serializable;
import java.util.List;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
/*******************************************************************************
** Host configuration for shared presence, including authoritative tenant scope.
*******************************************************************************/
public class RedisPresenceConfig implements Serializable
{
private String connectionName = null;
private String deployment = null;
private String application = null;
private QCodeReference scopeProvider = null;
private Integer capacity = 2000;
private String recoveryGeneration = "initial";
/***************************************************************************
** Inspect configuration without creating a runtime pool or local registry.
***************************************************************************/
public static RedisPresenceConfig forInstance(com.kingsrook.qqq.backend.core.model.metadata.QInstance instance)
{
var config = instance.getQBits().values().stream().map(bit -> bit.getConfig())
.filter(com.kingsrook.qbits.collaboration.CollaborationQBitConfig.class::isInstance)
.map(com.kingsrook.qbits.collaboration.CollaborationQBitConfig.class::cast).findFirst().orElse(null);
return (config == null ? null : config.getRedisPresence());
}
/***************************************************************************
** Explicit scope prevents accidental sharing between deployments or tenants.
***************************************************************************/
public void validate(List<String> errors)
{
if(connectionName == null || connectionName.isBlank() || deployment == null || deployment.isBlank()
|| application == null || application.isBlank() || scopeProvider == null)
{
errors.add("Redis presence requires a named connection, deployment, application and tenant scope provider");
}
if(capacity == null || capacity < 1 || capacity > 10000)
{
errors.add("Redis presence capacity must be between 1 and 10000");
}
if(recoveryGeneration == null || !recoveryGeneration.matches("[A-Za-z0-9_-]{1,64}"))
{
errors.add("Redis presence recovery generation must contain 1 to 64 letters, digits, underscores or hyphens");
}
}
/***************************************************************************
** Return connectionName.
***************************************************************************/
public String getConnectionName()
{
return (connectionName);
}
/***************************************************************************
** Configure connectionName.
***************************************************************************/
public RedisPresenceConfig withConnectionName(String value)
{
connectionName = value;
return (this);
}
/***************************************************************************
** Return deployment.
***************************************************************************/
public String getDeployment()
{
return (deployment);
}
/***************************************************************************
** Configure deployment.
***************************************************************************/
public RedisPresenceConfig withDeployment(String value)
{
deployment = value;
return (this);
}
/***************************************************************************
** Return application.
***************************************************************************/
public String getApplication()
{
return (application);
}
/***************************************************************************
** Configure application.
***************************************************************************/
public RedisPresenceConfig withApplication(String value)
{
application = value;
return (this);
}
/***************************************************************************
** Return scopeProvider.
***************************************************************************/
public QCodeReference getScopeProvider()
{
return (scopeProvider);
}
/***************************************************************************
** Configure scopeProvider.
***************************************************************************/
public RedisPresenceConfig withScopeProvider(QCodeReference value)
{
scopeProvider = value;
return (this);
}
/***************************************************************************
** Return capacity.
***************************************************************************/
public Integer getCapacity()
{
return (capacity);
}
/***************************************************************************
** Configure capacity.
***************************************************************************/
public RedisPresenceConfig withCapacity(Integer value)
{
capacity = value;
return (this);
}
/***************************************************************************
** Host-owned generation, stored outside Redis and changed after restoring data.
***************************************************************************/
public String getRecoveryGeneration()
{
return (recoveryGeneration);
}
/***************************************************************************
** All nodes must use the same new generation before maintenance traffic resumes.
***************************************************************************/
public RedisPresenceConfig withRecoveryGeneration(String value)
{
recoveryGeneration = value;
return (this);
}
}
@@ -0,0 +1,254 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.presence;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.time.Duration;
import java.util.ArrayList;
import java.util.Base64;
import java.util.HexFormat;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.TreeMap;
import com.kingsrook.qbits.redis.api.RedisNamespace;
import com.kingsrook.qbits.redis.api.RedisScript;
import com.kingsrook.qbits.redis.api.RedisScriptExecutor;
import com.kingsrook.qqq.backend.core.utils.JsonUtils;
/*******************************************************************************
** Shared ephemeral presence. Callers must authorize identity, scope and fields.
** Scripts atomically update record/table membership using the Redis server clock.
*******************************************************************************/
public final class RedisPresenceStore
{
private static final RedisScript MUTATE = new RedisScript(resource("redis-recovery.lua") + "\n" + resource("redis-mutate.lua"));
private static final RedisScript SNAPSHOT = new RedisScript(resource("redis-recovery.lua") + "\n" + resource("redis-snapshot.lua"));
private final RedisScriptExecutor client;
private final RedisNamespace namespace;
private final long ttlMillis;
private final int capacity;
/***************************************************************************
** Limit each indexed room, including departure markers, to bound script work.
***************************************************************************/
public RedisPresenceStore(RedisScriptExecutor client, RedisNamespace namespace, Duration ttl, int capacity)
{
this.client = Objects.requireNonNull(client);
this.namespace = Objects.requireNonNull(namespace);
this.ttlMillis = Objects.requireNonNull(ttl).toMillis();
if(ttlMillis < 1 || ttlMillis > Duration.ofMinutes(5).toMillis() || capacity < 1 || capacity > 10000)
{
throw (new IllegalArgumentException("Presence requires a positive TTL of at most five minutes and capacity of 1 to 10000"));
}
this.capacity = capacity;
}
/***************************************************************************
** Renew a connection; duplicate or delayed sequences never extend its lease.
***************************************************************************/
public boolean heartbeat(String user, String displayName, String connection, String table, String pk, String mode, String field, Long sequence)
{
if(connection != null && connection.startsWith("lease:"))
{
throw (new PresenceLeaseExpiredException());
}
if(!List.of("VIEWING", "EDITING").contains(mode == null ? "" : mode))
{
throw (new IllegalArgumentException("Invalid presence mode"));
}
return (mutate(user, displayName, connection, table, pk, mode, field, sequence, false, "legacy"));
}
/***************************************************************************
** Reserve an invisible server-issued lease. Null means room capacity refused.
** Only a subsequent renewal makes it visible; lost responses expire silently.
***************************************************************************/
public String openLease(String user, String displayName, String table, String pk)
{
String connection = "lease:" + java.util.UUID.randomUUID();
return (mutate(user, displayName, connection, table, pk, "VIEWING", null, 0L, false, "open") ? connection : null);
}
/***************************************************************************
** Activate or renew existing authority, never recreate missing/departed state.
** Callers must reauthorize identity, parent visibility and focused fields.
***************************************************************************/
public boolean renewLease(String user, String displayName, String connection, String table, String pk, String mode, String field, Long sequence)
{
if(connection == null || !connection.startsWith("lease:") || sequence == null || sequence < 1)
{
throw (new IllegalArgumentException("A server-issued lease and positive sequence are required"));
}
if(!List.of("VIEWING", "EDITING").contains(mode == null ? "" : mode))
{
throw (new IllegalArgumentException("Invalid presence mode"));
}
return (mutate(user, displayName, connection, table, pk, mode, field, sequence, false, "renew"));
}
/***************************************************************************
** Retain a bounded departure marker so older HTTP renewals cannot undo leave.
***************************************************************************/
public boolean leave(String user, String connection, String table, String pk, Long sequence)
{
return (mutate(user, user, connection, table, pk, "VIEWING", null, sequence, true, "legacy"));
}
/***************************************************************************
** Redis failures propagate; they must never be presented as an empty room.
***************************************************************************/
public List<PresenceRegistry.Viewer> snapshot(String table, String pk)
{
List<String> keys = roomKeys(table, pk, true);
Object result = client.execute(SNAPSHOT, keys, List.of(pk == null ? "" : ":" + encode(pk, 1024),
Long.toString(ttlMillis), java.util.UUID.randomUUID().toString()));
if(result instanceof Number number && number.longValue() == -2)
{
throw (new PresenceRecoveringException());
}
Map<String, Stored> users = new TreeMap<>();
for(Object value : (List<?>) result)
{
try
{
Stored entry = JsonUtils.toObject((String) value, Stored.class);
if(!entry.departed() && !entry.pending())
{
users.merge(entry.userId(), entry, (old, next) ->
(!Objects.equals(old.mode(), next.mode()) ? "EDITING".equals(next.mode()) : next.at() >= old.at()) ? next : old);
}
}
catch(IOException e)
{
throw (new IllegalStateException("Invalid stored presence data", e));
}
}
return (users.values().stream().map(entry -> new PresenceRegistry.Viewer(entry.userId(), entry.displayName(), entry.mode(),
pk == null ? null : entry.fieldName())).toList());
}
/***************************************************************************
** Validate bounded inputs before sending any Lua command to Redis.
***************************************************************************/
private boolean mutate(String user, String displayName, String connection, String table, String pk, String mode, String field, Long sequence, boolean departed, String operation)
{
String member = encode(user, 255) + ":" + encode(connection, 255) + ":" + (pk == null ? "table" : encode(pk, 1024));
encode(displayName, 1024);
if(field != null)
{
encode(field, 255);
}
if(sequence != null && (sequence < 0 || sequence > 9007199254740991L))
{
throw (new IllegalArgumentException("Presence sequence must be a nonnegative safe integer"));
}
Stored value = new Stored(user, displayName, mode, pk == null ? null : field, 0, sequence, departed, false, false, null);
Object result = client.execute(MUTATE, roomKeys(table, pk, true), List.of(member, JsonUtils.toJson(value),
sequence == null ? "" : sequence.toString(), departed ? "1" : "0", Long.toString(ttlMillis), Integer.toString(capacity), pk == null ? "1" : "0", operation,
java.util.UUID.randomUUID().toString()));
if(((Number) result).longValue() == -1)
{
throw (new PresenceLeaseExpiredException());
}
return (((Number) result).longValue() == 1);
}
/***************************************************************************
** One table hash tag keeps record/table updates atomic on Redis Cluster slots.
***************************************************************************/
private List<String> roomKeys(String table, String pk, boolean includeTable)
{
String base = "qqq:presence:v2:{" + digest("SHA-256", namespace.key(table)) + "}";
List<String> keys = new ArrayList<>();
if(pk != null)
{
String record = base + ":record:" + encode(pk, 1024);
keys.add(record + ":expiry");
keys.add(record + ":state");
}
if(pk == null || includeTable)
{
keys.add(base + ":table:expiry");
keys.add(base + ":table:state");
}
keys.add(base + ":recovery");
return (keys);
}
/***************************************************************************
** Distinguish arbitrary identifiers without allowing delimiter injection.
***************************************************************************/
private static String encode(String value, int maximumBytes)
{
if(value == null || value.isBlank() || value.getBytes(StandardCharsets.UTF_8).length > maximumBytes)
{
throw (new IllegalArgumentException("Invalid or oversized presence identifier"));
}
return (Base64.getUrlEncoder().withoutPadding().encodeToString(value.getBytes(StandardCharsets.UTF_8)));
}
/***************************************************************************
** Hash public routing identifiers and scripts using JDK algorithms.
***************************************************************************/
private static String digest(String algorithm, String value)
{
try
{
return (HexFormat.of().formatHex(MessageDigest.getInstance(algorithm).digest(value.getBytes(StandardCharsets.UTF_8))));
}
catch(NoSuchAlgorithmException e)
{
throw (new IllegalStateException("Required digest algorithm unavailable", e));
}
}
/***************************************************************************
** Load immutable scripts from the application artifact.
***************************************************************************/
private static String resource(String name)
{
try(var stream = RedisPresenceStore.class.getResourceAsStream("/presence/" + name))
{
if(stream == null)
{
throw (new IllegalStateException("Missing presence script"));
}
return (new String(stream.readAllBytes(), StandardCharsets.UTF_8));
}
catch(IOException e)
{
throw (new IllegalStateException("Cannot load presence script", e));
}
}
/***************************************************************************
** Internal lease data, never returned directly to a browser.
***************************************************************************/
public record Stored(String userId, String displayName, String mode, String fieldName, long at, Long sequence, boolean departed, boolean pending, boolean leased, String generation)
{
}
}
@@ -29,7 +29,7 @@ import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.exceptions.QNotFoundException;
import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
@@ -52,7 +52,7 @@ public final class ParentRecordAccess
{
QTableMetaData table = tableName == null ? null : QContext.getQInstance().getTable(tableName);
return (table != null && table.getPrimaryKeyField() != null && !tableName.startsWith(CollaborationQBitConfig.TABLE_NAME_PREFIX)
&& PermissionsHelper.hasTablePermission(new AbstractActionInput(), tableName, TablePermissionSubType.READ));
&& PermissionsHelper.hasTablePermission(new GetInput(tableName), tableName, TablePermissionSubType.READ));
}
/*******************************************************************************
@@ -81,6 +81,42 @@ public final class ParentRecordAccess
}
}
/*******************************************************************************
** One result set's parent checks: rows that share a parent cost one lookup.
** The session and transaction do not change within the call that creates it,
** so never keep an instance beyond that call.
*******************************************************************************/
public static final class ResultSetParents
{
private final java.util.Map<java.util.List<String>, Boolean> visible = new java.util.HashMap<>();
/*******************************************************************************
** Same outcome as ParentRecordAccess.requireVisibleParent for every row.
*******************************************************************************/
public void requireVisible(String tableName, String recordId) throws QException
{
java.util.List<String> parent = java.util.Arrays.asList(tableName, recordId);
Boolean known = visible.get(parent);
if(known == null)
{
try
{
requireVisibleParent(tableName, recordId);
known = true;
}
catch(QNotFoundException expected)
{
known = false;
}
visible.put(parent, known);
}
if(!known)
{
throw (new QNotFoundException("Not found"));
}
}
}
/*******************************************************************************
** Core record capability checks include parent WRITE locks for implied grants.
*******************************************************************************/
@@ -43,6 +43,7 @@ public class ParentRowCustomizer implements TableCustomizerInterface
public List<QRecord> postQuery(QueryOrGetInputInterface input, List<QRecord> records) throws QException
{
List<QRecord> visible = new ArrayList<>();
ParentRecordAccess.ResultSetParents parents = new ParentRecordAccess.ResultSetParents();
for(QRecord record : records)
{
try
@@ -51,7 +52,7 @@ public class ParentRowCustomizer implements TableCustomizerInterface
{
throw (new QNotFoundException("Not found"));
}
ParentRecordAccess.requireVisibleParent(record.getValueString("tableName"), record.getValueString("recordId"));
parents.requireVisible(record.getValueString("tableName"), record.getValueString("recordId"));
QRecord row = new QRecord(record);
if(row.getValue("deletedAt") != null)
{
@@ -0,0 +1,100 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.tags;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.collaboration.CollaborationRecords;
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.actions.tables.query.VirtualFieldValueProviderInterface;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordReferenceTableMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import com.kingsrook.qqq.backend.core.utils.CollectionUtils;
/*******************************************************************************
** Return approved tag labels for parents selected by the enclosing query.
*******************************************************************************/
public class TagValueProvider implements VirtualFieldValueProviderInterface
{
/*******************************************************************************
** Batch both assignments and vocabulary; never recursively load parent records.
** A table without any assignment costs no statement: core selects this field
** for every read that names no fields, so untagged tables must stay free.
*******************************************************************************/
@Override
public void fill(QTableMetaData table, String fieldName, List<QRecord> records, QueryInput origin) throws QException
{
if(!TaggedTables.mayHaveAssignments(table.getName(), origin.getTransaction()))
{
for(QRecord record : records)
{
record.setValue(fieldName, null);
}
return;
}
QTableMetaData metadata = QContext.getQInstance().getTable(CollaborationTables.RECORD_TAG).clone();
metadata.withCustomizers(null);
metadata.getSupplementalMetaData().remove(RecordReferenceTableMetaData.NAME);
for(List<QRecord> page : CollectionUtils.getPages(records, 500))
{
List<String> ids = page.stream().map(record -> String.valueOf(record.getValue(table.getPrimaryKeyField()))).distinct().toList();
QueryInput input = new QueryInput(CollaborationTables.RECORD_TAG).withTransaction(origin.getTransaction()).withFilter(new QQueryFilter()
.withCriteria("tableName", QCriteriaOperator.EQUALS, table.getName()).withCriteria("recordId", QCriteriaOperator.IN, ids)
.withCriteria("parentDeleted", QCriteriaOperator.EQUALS, false));
input.setTableMetaData(metadata);
input.withPermissionBypass("Tag provider reads only parents selected by the enclosing query.");
List<QRecord> assignments = new QueryAction().execute(input).getRecords();
Map<Integer, String> labels = new HashMap<>();
List<Integer> tagIds = assignments.stream().map(row -> row.getValueInteger("tagId")).distinct().toList();
for(List<Integer> tags : CollectionUtils.getPages(tagIds, 500))
{
for(QRecord tag : CollaborationRecords.query(CollaborationTables.TAG, TagsService.vocabularyFilter(table.getName())
.withCriteria("approved", QCriteriaOperator.EQUALS, true).withCriteria("id", QCriteriaOperator.IN, tags), origin.getTransaction()))
{
labels.put(tag.getValueInteger("id"), tag.getValueString("label"));
}
}
Map<String, java.util.SortedSet<String>> byParent = new HashMap<>();
for(QRecord assignment : assignments)
{
String label = labels.get(assignment.getValueInteger("tagId"));
if(label != null)
{
byParent.computeIfAbsent(assignment.getValueString("recordId"), key -> new java.util.TreeSet<>()).add(label);
}
}
for(QRecord record : page)
{
var values = byParent.get(String.valueOf(record.getValue(table.getPrimaryKeyField())));
record.setValue(fieldName, values == null ? null : String.join(", ", values));
}
}
}
}
@@ -0,0 +1,184 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.tags;
import java.time.Duration;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.WeakHashMap;
import com.kingsrook.qbits.collaboration.CollaborationRecords;
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.cluster.ClusterBus;
import com.kingsrook.qqq.backend.core.cluster.ClusterCaches;
import com.kingsrook.qqq.backend.core.cluster.ClusterEvent;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.utils.memoization.Memoization;
/*******************************************************************************
** Which host tables have any tag assignment at all. Every table with TAGS
** enabled carries the Tags field, so every read of such a table would otherwise
** look for assignments, including the tables nobody ever tagged.
**
** Only that one fact per table is remembered, never a record's tags: a table
** known to have assignments is always read from the database. The answer is the
** same for every session (the probe ignores row locks and returns no data), so
** it cannot widen or narrow what a caller sees.
**
** Each node keeps its own answers. An assignment written here clears the
** table's answer on this node at once and again after commit, and on the other
** nodes through the ClusterBus (memo-invalidate). The timeout bounds everything
** that event cannot reach: hosts without a cluster transport, writes made
** outside QQQ, and a probe that raced an uncommitted assignment.
*******************************************************************************/
public final class TaggedTables
{
public static final String CACHE_NAME = "collaboration.taggedTables";
private static final int MAX_TABLES = 10_000;
private static final Map<QInstance, Memoization<String, Boolean>> CACHES = ClusterCaches.shared(CACHE_NAME,
new WeakHashMap<QInstance, Memoization<String, Boolean>>(), TaggedTables::clearAll, TaggedTables::clearTable);
/*******************************************************************************
** Static lookups only.
*******************************************************************************/
private TaggedTables()
{
}
/*******************************************************************************
** False only when the table is known to have no assignment. A failed probe is
** thrown, never remembered. With the cache disabled the answer is always true.
*******************************************************************************/
public static boolean mayHaveAssignments(String tableName, QBackendTransaction transaction) throws QException
{
Integer seconds = CollaborationRecords.config().getTagLookupCacheSeconds();
if(seconds == null || seconds <= 0)
{
return (true);
}
return (cache(QContext.getQInstance(), seconds).getResultThrowing(tableName, name -> hasAssignments(name, transaction)).orElse(true));
}
/*******************************************************************************
** An assignment was written for this parent table. Forget the answer now, so
** the writing transaction reads its own assignment, and again after commit, so
** a probe that ran in between and saw nothing cannot outlive the commit.
*******************************************************************************/
public static void assignmentsChanged(String tableName, QBackendTransaction transaction)
{
QInstance instance = QContext.getQInstance();
forget(instance, tableName);
if(transaction != null)
{
transaction.addAfterCommitCallback(() -> forget(instance, tableName));
}
}
/*******************************************************************************
** Clear on this node and tell the others; a transport failure is logged by the
** bus and the timeout covers the nodes that did not hear.
*******************************************************************************/
private static void forget(QInstance instance, String tableName)
{
ClusterBus.publish(instance, new ClusterEvent(ClusterEvent.TYPE_MEMO_INVALIDATE, CACHE_NAME, tableName));
}
/*******************************************************************************
** One indexed row at most, in the caller's transaction, as the same answer for
** every session: no row locks, no parent check and no customizers.
*******************************************************************************/
private static Boolean hasAssignments(String tableName, QBackendTransaction transaction) throws QException
{
QueryInput input = new QueryInput(CollaborationTables.RECORD_TAG).withTransaction(transaction).withFieldNamesToInclude(Set.of("id"))
.withFilter(new QQueryFilter().withCriteria("tableName", QCriteriaOperator.EQUALS, tableName).withLimit(1));
input.setTableMetaData(CollaborationRecords.cleanupMetadata(CollaborationTables.RECORD_TAG));
input.withPermissionBypass("Tag provider asks only whether a table has any assignment.");
return (!new QueryAction().execute(input).getRecords().isEmpty());
}
/*******************************************************************************
** The instance's answers, following the configured timeout.
*******************************************************************************/
private static Memoization<String, Boolean> cache(QInstance instance, int seconds)
{
synchronized(CACHES)
{
Memoization<String, Boolean> cache = CACHES.computeIfAbsent(instance, key -> new Memoization<String, Boolean>(MAX_TABLES).withMayStoreNullValues(false));
cache.setTimeout(Duration.ofSeconds(seconds));
return (cache);
}
}
/*******************************************************************************
** Every instance in this JVM: an extra probe is cheaper than a stale answer.
*******************************************************************************/
private static List<Memoization<String, Boolean>> caches()
{
synchronized(CACHES)
{
return (List.copyOf(CACHES.values()));
}
}
/*******************************************************************************
** ClusterCaches: forget every table.
*******************************************************************************/
private static void clearAll()
{
caches().forEach(Memoization::clear);
}
/*******************************************************************************
** ClusterCaches: forget one table.
*******************************************************************************/
private static void clearTable(String tableName)
{
caches().forEach(cache -> cache.clearKey(tableName));
}
}
@@ -56,7 +56,8 @@ public class TagsMetaDataProvider implements RuntimeMetaDataProviderInterface
{
contribution.getTables().put(table.getName(), new RuntimeTableMetaDataContribution()
.withVirtualFields(List.of(new QVirtualFieldMetaData("collaborationTags", QFieldType.STRING).withLabel("Tags")
.withPossibleValueSourceName(CollaborationTables.TAG).withIsQueryCriteria(true).withIsQuerySelectable(false)
.withPossibleValueSourceName(CollaborationTables.TAG).withIsQueryCriteria(true).withIsQuerySelectable(true)
.withValueProvider(new QCodeReference(TagValueProvider.class)).withShouldTranslatePossibleValues(false)
.withCriteriaExpander(new QCodeReference(TagCriteriaExpander.class))))
.withSearchFields(List.of("collaborationTags")));
}
@@ -712,6 +712,23 @@ components:
nullable: true
type: "string"
type: "object"
CollaborationPresenceExpiredV1:
properties:
code:
description: "Open a fresh lease before retrying renewal."
enum:
- "PRESENCE_LEASE_EXPIRED"
nullable: false
type: "string"
type: "object"
CollaborationPresenceLeaseV1:
properties:
connectionId:
description: "Server-issued handle. Renew before 45 seconds; departure is\
\ terminal."
nullable: false
type: "string"
type: "object"
CollaborationPresenceLeaveV1:
description: "Remove the caller's lease on this connection and target."
properties:
@@ -723,6 +740,22 @@ components:
description: "Optional parent key."
nullable: true
type: "string"
sequence:
description: "Monotonically increasing nonnegative safe integer per connection\
\ and target; orders delayed requests."
nullable: true
type: "integer"
table:
description: "Parent table name."
nullable: false
type: "string"
type: "object"
CollaborationPresenceOpenV1:
properties:
pk:
description: "Optional record key."
nullable: true
type: "string"
table:
description: "Parent table name."
nullable: false
@@ -750,6 +783,11 @@ components:
description: "Optional parent key; omit for a table view."
nullable: true
type: "string"
sequence:
description: "Monotonically increasing nonnegative safe integer per connection\
\ and target; orders delayed requests."
nullable: true
type: "integer"
table:
description: "Parent table name."
nullable: false
@@ -758,19 +796,6 @@ components:
CollaborationPresenceSnapshotV1:
description: "Live viewers, merged across tabs; presence.changed uses this payload."
properties:
leaseCount:
description: "Live connections (leases) behind viewers; a user with two\
\ terminals counts twice. Since 1.2.0."
nullable: true
type: "integer"
leases:
description: "One viewer per live connection, with connectionId and since.\
\ On GET only with perConnection=true; always on presence.changed. Since\
\ 1.2.0."
items:
$ref: "#/components/schemas/CollaborationPresenceViewerV1"
nullable: true
type: "array"
pk:
description: "Record key; absent for table events."
nullable: true
@@ -780,26 +805,13 @@ components:
nullable: false
type: "string"
viewers:
description: "Live viewers; always present, empty once the last viewer leaves."
items:
$ref: "#/components/schemas/CollaborationPresenceViewerV1"
nullable: false
type: "array"
type: "object"
CollaborationPresenceViewerV1:
description: "One user merged across live tabs (EDITING takes precedence), or\
\ one live connection in leases."
description: "One user merged across live tabs; EDITING takes precedence."
properties:
connectionId:
description: "Opaque per-connection id, only in leases: c + the first 16\
\ hex digits of SHA-256(userId + newline + connectionId). Since 1.2.0."
nullable: true
type: "string"
connections:
description: "Live connections (tabs, terminals) merged into this viewer;\
\ 1 in leases. Since 1.2.0."
nullable: true
type: "integer"
displayName:
description: "Session display name."
nullable: false
@@ -815,18 +827,6 @@ components:
- "EDITING"
nullable: false
type: "string"
sessionId:
description: "Opaque id of the signed-in session behind this connection,\
\ only in leases (HMAC of the server session; never the session itself).\
\ Since 1.2.0."
nullable: true
type: "string"
since:
description: "When this connection's lease started (ISO-8601), only in leases.\
\ Since 1.2.0."
format: "date-time"
nullable: true
type: "string"
userId:
description: "Stable user identity."
nullable: false
@@ -1297,7 +1297,7 @@ info:
description: "Record collaboration: favorites, recents, attachments, tags, conversations,\
\ following, timeline, people and presence"
title: "QQQ Middleware API extension: collaboration"
version: "1.2.0"
version: "1.3.0"
openapi: "3.0.3"
paths:
/qqq/v1/qbit/collaboration/presence:
@@ -1336,6 +1336,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -1384,13 +1390,6 @@ paths:
required: false
schema:
type: "string"
- description: "Also return one viewer per live connection (lease) in leases;\
\ default false. Since collaboration 1.2.0."
in: "query"
name: "perConnection"
required: false
schema:
type: "boolean"
responses:
400:
content:
@@ -1416,6 +1415,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -1482,6 +1487,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -1513,6 +1524,151 @@ paths:
summary: "Renew presence"
tags:
- "collaboration"
/qqq/v1/qbit/collaboration/presence/leases:
post:
description: "Open presence lease. Uses the signed-in caller's permissions and\
\ record visibility."
operationId: "postQbitCollaborationPresenceLeases"
requestBody:
content:
application/json:
schema:
$ref: "#/components/schemas/CollaborationPresenceOpenV1"
required: true
responses:
400:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Invalid input"
401:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Sign in required"
403:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Permission denied"
404:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
schema:
$ref: "#/components/schemas/CollaborationPresenceLeaseV1"
description: "Successful Response"
409:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Conflict with the current state, e.g. an edit conflict or\
\ an invalid transition (code CONFLICT)."
429:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Too many requests (code RATE_LIMITED); retry later."
security:
- sessionUuidCookie:
- "N/A"
- bearerAuth:
- "N/A"
- oauth2:
- "openid"
- "profile"
- "email"
summary: "Open presence lease"
tags:
- "collaboration"
/qqq/v1/qbit/collaboration/presence/leases/renew:
post:
description: "Renew issued presence lease. Uses the signed-in caller's permissions\
\ and record visibility."
operationId: "postQbitCollaborationPresenceLeasesRenew"
requestBody:
content:
application/json:
schema:
$ref: "#/components/schemas/CollaborationPresenceRequestV1"
required: true
responses:
400:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Invalid input"
401:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Sign in required"
403:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Permission denied"
404:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
schema:
$ref: "#/components/schemas/CollaborationPresenceSnapshotV1"
description: "Successful Response"
409:
content:
application/json:
schema:
$ref: "#/components/schemas/CollaborationPresenceExpiredV1"
description: "Presence lease expired; open a new lease."
429:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Too many requests (code RATE_LIMITED); retry later."
security:
- sessionUuidCookie:
- "N/A"
- bearerAuth:
- "N/A"
- oauth2:
- "openid"
- "profile"
- "email"
summary: "Renew issued presence lease"
tags:
- "collaboration"
/qqq/v1/qbit/collaboration/favorites:
delete:
description: "Unstar a target. Uses the signed-in caller's permissions and record\
@@ -1549,6 +1705,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -1623,6 +1785,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -1689,6 +1857,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -1764,6 +1938,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -1830,6 +2010,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -1897,6 +2083,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -1984,6 +2176,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -2063,6 +2261,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -2131,6 +2335,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -2199,6 +2409,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -2267,6 +2483,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -2340,6 +2562,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -2420,6 +2648,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -2486,6 +2720,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -2573,6 +2813,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -2652,6 +2898,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -2739,6 +2991,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -2818,6 +3076,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -2886,6 +3150,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -2959,6 +3229,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -3027,6 +3303,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -3095,6 +3377,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -3176,6 +3464,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -3269,6 +3563,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -3343,6 +3643,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -3416,6 +3722,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -3495,6 +3807,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -3582,6 +3900,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -3682,6 +4006,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -3756,6 +4086,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -3824,6 +4160,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -3898,6 +4240,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -3978,6 +4326,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -4058,6 +4412,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -4125,6 +4485,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -4200,6 +4566,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -4293,6 +4665,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -4372,6 +4750,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -4446,6 +4830,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -4527,6 +4917,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -4607,6 +5003,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -4668,6 +5070,12 @@ paths:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
@@ -4699,6 +5107,73 @@ paths:
summary: "Get my work summary"
tags:
- "collaboration"
/qqq/v1/qbit/collaboration/my-work/mentions/read:
post:
description: "Mark my visible mentions read. Uses the signed-in caller's permissions\
\ and record visibility."
operationId: "postQbitCollaborationMyWorkMentionsRead"
responses:
400:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Invalid input"
401:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Sign in required"
403:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Permission denied"
404:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found (including invisible parents)"
503:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Service temporarily unavailable"
200:
content:
application/json:
schema:
$ref: "#/components/schemas/CollaborationMutationResultV1"
description: "Successful Response"
409:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Conflict with the current state, e.g. an edit conflict or\
\ an invalid transition (code CONFLICT)."
429:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Too many requests (code RATE_LIMITED); retry later."
security:
- sessionUuidCookie:
- "N/A"
- bearerAuth:
- "N/A"
- oauth2:
- "openid"
- "profile"
- "email"
summary: "Mark my visible mentions read"
tags:
- "collaboration"
servers:
- description: "The QQQ server that serves this document (paths include /qqq/v1)."
url: "/"
@@ -0,0 +1,80 @@
-- All room keys share a table hash tag. Redis TIME owns lease expiry.
local clock = redis.call('TIME')
local now = tonumber(clock[1]) * 1000 + math.floor(tonumber(clock[2]) / 1000)
local guard = presenceRecovery(KEYS, tonumber(ARGV[5]), ARGV[9], now)
local rooms = #KEYS - 1
-- Validate every pair before changing either record or table membership.
for i = 1, rooms, 2 do
if redis.call('ZCARD', KEYS[i]) ~= redis.call('HLEN', KEYS[i + 1]) then
return redis.error_reply('Presence room state is incomplete')
end
end
local member = ARGV[1]
local ttl = tonumber(ARGV[5])
local capacity = tonumber(ARGV[6])
local sequence = tonumber(ARGV[3])
local departed = ARGV[4] == '1'
for i = 1, rooms, 2 do
local expired = redis.call('ZRANGEBYSCORE', KEYS[i], '-inf', now)
for _, id in ipairs(expired) do
redis.call('ZREM', KEYS[i], id)
redis.call('HDEL', KEYS[i + 1], id)
end
end
local previous = redis.call('HGET', KEYS[2], member)
-- A record lease and its table projection are written atomically. If one is
-- missing while the other is live, do not erase surviving ordering/tombstones.
if rooms == 4 then
local tablePrevious = redis.call('HGET', KEYS[4], member)
if (previous and not tablePrevious) or (tablePrevious and not previous) then
return redis.error_reply('Presence room state is incomplete')
end
end
local old = previous and cjson.decode(previous) or nil
local operation = ARGV[8] or 'legacy'
-- Renewal authority is issued by open, not inferred from a caller's sequence.
-- Missing/expired/departed leases cannot be recreated by delayed HTTP requests.
if operation == 'renew' and (not old or not old.leased or old.departed or old.generation ~= guard.generation) then
return -1
end
if operation == 'open' and old then return -1 end
if operation == 'legacy' and not departed and old and old.leased then return -1 end
if old then
local oldSequence = old.sequence
if oldSequence == cjson.null then oldSequence = nil end
if oldSequence then
if not sequence or sequence < oldSequence or (sequence == oldSequence and (not departed or old.departed)) then
return 1
end
elseif not sequence and old.at == now and old.departed and not departed then
return 1
end
end
-- Check every room before writing any membership, avoiding a partial join.
for i = 1, rooms, 2 do
if redis.call('HEXISTS', KEYS[i + 1], member) == 0 and redis.call('ZCARD', KEYS[i]) >= capacity then
return 0
end
end
local data = cjson.decode(ARGV[2])
data.at = now
data.sequence = sequence or cjson.null
data.departed = departed
data.pending = operation == 'open'
data.leased = operation ~= 'legacy' or (old and old.leased) or false
data.generation = guard.generation
for i = 1, rooms, 2 do
-- Record scope comes first; table scope must never carry focused-field data.
if i > 1 or ARGV[7] == '1' then data.fieldName = cjson.null end
redis.call('HSET', KEYS[i + 1], member, cjson.encode(data))
redis.call('ZADD', KEYS[i], now + ttl, member)
redis.call('PEXPIRE', KEYS[i], ttl)
redis.call('PEXPIRE', KEYS[i + 1], ttl)
end
guard.liveUntil = now + ttl
redis.call('SET', KEYS[#KEYS], cjson.encode(guard))
return 1
@@ -0,0 +1,43 @@
-- This prelude is executed atomically with the room operation, in the same slot.
-- The guard deliberately outlives idle leases: missing means unknown, not empty.
local function presenceRecovery(keys, ttl, candidate, now)
local encoded = redis.call('GET', keys[#keys])
local guard
local created = false
if encoded then
guard = cjson.decode(encoded)
if type(guard.generation) ~= 'string' or type(guard.readyAt) ~= 'number' or type(guard.liveUntil) ~= 'number' or type(guard.ttl) ~= 'number' then
error('Invalid presence recovery guard')
end
if guard.ttl ~= ttl then error('Presence lease lifetime differs across nodes') end
-- All volatile projections disappearing before the last issued expiry is
-- data loss too, even when a persistent guard survives an eviction policy.
if guard.liveUntil > now and redis.call('ZCARD', keys[#keys - 2]) == 0
and redis.call('HLEN', keys[#keys - 1]) == 0 then
guard = nil
end
end
if not guard then
created = true
guard = { generation = candidate, readyAt = now + ttl, liveUntil = 0, ttl = ttl }
redis.call('SET', keys[#keys], cjson.encode(guard))
end
-- During recovery, discard only projections from an invalidated generation.
-- The table is cleaned atomically when its generation is created. Subsequent
-- requests only need to clean the touched record; all new table writes carry
-- the current generation. Avoid rescanning the whole table on every renewal.
if now < guard.readyAt then
for i = 1, #keys - 1, 2 do
if created or i < #keys - 2 then
for _, member in ipairs(redis.call('HKEYS', keys[i + 1])) do
local data = cjson.decode(redis.call('HGET', keys[i + 1], member))
if data.generation ~= guard.generation then
redis.call('HDEL', keys[i + 1], member)
redis.call('ZREM', keys[i], member)
end
end
end
end
end
return guard
end
@@ -0,0 +1,45 @@
-- A partially evicted room is unavailable, not empty. Do not silently discard it.
local clock = redis.call('TIME')
local now = tonumber(clock[1]) * 1000 + math.floor(tonumber(clock[2]) / 1000)
local guard = presenceRecovery(KEYS, tonumber(ARGV[2]), ARGV[3], now)
local rooms = #KEYS - 1
for i = 1, rooms, 2 do
if redis.call('ZCARD', KEYS[i]) ~= redis.call('HLEN', KEYS[i + 1]) then
return redis.error_reply('Presence room state is incomplete')
end
end
for i = 1, rooms, 2 do
local expired = redis.call('ZRANGEBYSCORE', KEYS[i], '-inf', now)
for _, id in ipairs(expired) do
redis.call('ZREM', KEYS[i], id)
redis.call('HDEL', KEYS[i + 1], id)
end
end
local members = redis.call('ZRANGE', KEYS[1], 0, -1)
-- Table membership is bounded by the configured capacity. Compare the record's
-- projection without key discovery or cross-slot reads.
if rooms == 4 then
local projected = 0
for _, id in ipairs(redis.call('ZRANGE', KEYS[3], 0, -1)) do
if string.sub(id, -string.len(ARGV[1])) == ARGV[1] then
projected = projected + 1
if not redis.call('ZSCORE', KEYS[1], id) then
return redis.error_reply('Presence room state is incomplete')
end
end
end
if projected ~= #members then
return redis.error_reply('Presence room state is incomplete')
end
end
local result = {}
if now < guard.readyAt then return -2 end
for _, id in ipairs(members) do
local data = redis.call('HGET', KEYS[2], id)
if not data then return redis.error_reply('Presence room state is incomplete') end
if cjson.decode(data).generation ~= guard.generation then
return redis.error_reply('Presence generation is inconsistent')
end
table.insert(result, data)
end
return result
@@ -0,0 +1,97 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration;
import com.kingsrook.qbits.collaboration.activity.ActivityWriter;
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
import com.kingsrook.qqq.backend.core.actions.collaboration.RecordFeaturesResolver;
import com.kingsrook.qqq.backend.core.actions.collaboration.RuntimeMetaDataOverlay;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordFeature;
import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordReferenceTableMetaData;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
/*******************************************************************************
** Service activity follows its originating feature without enabling timeline UI.
*******************************************************************************/
class ActivityWriterFeatureIsolationTest extends PersonalServiceFixture
{
/***************************************************************************
** An authorized attachment mutation must retain its event with timeline off.
***************************************************************************/
@Test
void attachmentActivityDoesNotRequireTimeline() throws Exception
{
features(RecordFeature.ATTACHMENTS);
assertThat(RecordFeaturesResolver.resolve(instance, instance.getTable("document")).isEnabled(RecordFeature.TIMELINE)).isFalse();
ActivityWriter.write("document", "1", "FILE_ADDED", "File added", CollaborationTables.ATTACHMENT, "41");
assertThat(RecordReferenceTableMetaData.of(instance.getTable(CollaborationTables.ACTIVITY)).getFeature()).isEqualTo(RecordFeature.TIMELINE);
assertThat(RecordFeaturesResolver.resolve(instance, instance.getTable("document")).isEnabled(RecordFeature.TIMELINE)).isFalse();
features(RecordFeature.ATTACHMENTS, RecordFeature.TIMELINE);
assertThat(CollaborationRecords.query(CollaborationTables.ACTIVITY,
CollaborationRecords.parent("document", "1"))).singleElement().satisfies(row ->
{
assertThat(row.getValueString("type")).isEqualTo("FILE_ADDED");
assertThat(row.getValueString("actorUserId")).isEqualTo("alice");
});
}
/***************************************************************************
** Enabling timeline cannot authorize a disabled attachment operation.
***************************************************************************/
@Test
void originatingFeatureRemainsRequired() throws Exception
{
features(RecordFeature.TIMELINE);
assertThatThrownBy(() -> ActivityWriter.write("document", "1", "FILE_ADDED", "File added", CollaborationTables.ATTACHMENT, "41"))
.isInstanceOf(QException.class);
assertThat(CollaborationRecords.query(CollaborationTables.ACTIVITY, new QQueryFilter())).isEmpty();
}
/***************************************************************************
** Service persistence must preserve native parent tenant authorization.
***************************************************************************/
@Test
void inaccessibleParentCannotAcquireActivity() throws Exception
{
features(RecordFeature.ATTACHMENTS);
assertThatThrownBy(() -> ActivityWriter.write("document", "3", "FILE_ADDED", "File added", CollaborationTables.ATTACHMENT, "41"))
.isInstanceOf(QException.class);
}
/***************************************************************************
** Use the supported refresh boundary so effective metadata changes atomically.
***************************************************************************/
private void features(RecordFeature... enabled) throws Exception
{
config.withTableFeatures("document", enabled).applyFeatures(instance);
RuntimeMetaDataOverlay.refresh(instance);
}
}
@@ -60,7 +60,7 @@ class CollaborationApiContractTest
assertThat(model.getComponents().getSchemas().keySet()).allMatch(name -> name.startsWith("Collaboration") || name.equals(BasicErrorResponseV1.class.getSimpleName()));
assertThat(model.getPaths().keySet()).allMatch(pathName -> pathName.startsWith("/qqq/v1/qbit/collaboration/"));
assertThat(generated).contains("multipart/form-data", "format: \"binary\"", "clients follow url", "nextCursor");
assertThat(extension.getSpecs()).hasSize(45);
assertThat(extension.getSpecs()).hasSize(48);
}
/*******************************************************************************
@@ -39,6 +39,12 @@ import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.node.ObjectNode;
import com.kingsrook.qbits.collaboration.attachments.AttachmentStorage;
import com.kingsrook.qbits.collaboration.customfields.FieldDefinitions;
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData;
import com.kingsrook.qbits.objectstorage.api.PresignGet;
import com.kingsrook.qbits.objectstorage.api.PutRequest;
import com.kingsrook.qqq.backend.core.actions.collaboration.RuntimeMetaDataOverlay;
@@ -63,6 +69,86 @@ import static org.assertj.core.api.Assertions.assertThat;
*******************************************************************************/
class CollaborationApiHttpTest extends PersonalServiceFixture
{
/***************************************************************************
** Data loss is unavailable over HTTP until a fresh lease has rejoined.
***************************************************************************/
@Test
void redisLossRequiresHttpRejoin() throws Exception
{
try(var redis = RedisFixtures.redis())
{
redis.start();
new com.kingsrook.qbits.redis.RedisQBitProducer().withQBitConfig(new com.kingsrook.qbits.redis.RedisQBitConfig()
.withConnection("presence", new com.kingsrook.qbits.redis.RedisConnectionConfig()
.withHost(RedisFixtures.host(redis)).withPort(RedisFixtures.port(redis)).withTls(false))).produce(instance);
config.withRedisPresence(new com.kingsrook.qbits.collaboration.presence.RedisPresenceConfig()
.withConnectionName("presence").withDeployment("test").withApplication("http-recovery")
.withScopeProvider(new QCodeReference(RedisPresenceServiceTest.TenantScope.class)));
var runtime = new com.kingsrook.qbits.redis.RedisRuntimeService();
try
{
runtime.start(instance);
String original = ok("POST", "/presence/leases", "{\"table\":\"document\",\"pk\":\"1\"}").get("connectionId").asText();
awaitHttpPresence(original);
assertThat(redis.execInContainer("redis-cli", "FLUSHDB").getExitCode()).isZero();
assertError(503, "GET", "/presence?table=document&pk=1", null, "alice");
assertError(404, "GET", "/presence?table=document&pk=3", null, "alice");
String oldRenewal = JSON.writeValueAsString(Map.of("connectionId", original, "table", "document", "pk", "1", "mode", "VIEWING", "sequence", 1000));
assertThat(json(request("POST", "/presence/leases/renew", oldRenewal, "alice"), 409).path("code").asText())
.isEqualTo("PRESENCE_LEASE_EXPIRED");
assertError(404, "POST", "/presence/leases/renew", oldRenewal, "outsider");
String replacement = ok("POST", "/presence/leases", "{\"table\":\"document\",\"pk\":\"1\"}").get("connectionId").asText();
assertThat(replacement).isNotEqualTo(original);
awaitHttpPresence(replacement);
assertThat(ok("GET", "/presence?table=document&pk=1", null).path("viewers")).hasSize(1);
}
finally
{
runtime.stop();
runtime.afterApplicationStop();
}
}
}
/***************************************************************************
** Keep a real HTTP lease alive while the production recovery window elapses.
***************************************************************************/
private void awaitHttpPresence(String lease) throws Exception
{
long deadline = System.nanoTime() + Duration.ofSeconds(55).toNanos();
long sequence = 0;
while(true)
{
String body = JSON.writeValueAsString(Map.of("connectionId", lease, "table", "document", "pk", "1", "mode", "VIEWING", "sequence", ++sequence));
var response = request("POST", "/presence/leases/renew", body, "alice");
if(response.statusCode() == 200)
{
assertThat(json(response, 200).path("viewers")).hasSize(1);
return;
}
assertThat(json(response, 503).path("message").asText()).contains("recovering");
assertThat(System.nanoTime()).as("HTTP presence recovery deadline").isLessThan(deadline);
Thread.sleep(1000);
}
}
/***************************************************************************
** Shared presence has an explicit unavailable response, never a local fallback.
***************************************************************************/
@Test
void unavailablePresenceHttp() throws Exception
{
config.withRedisPresence(new com.kingsrook.qbits.collaboration.presence.RedisPresenceConfig()
.withConnectionName("not-started").withDeployment("test").withApplication("fixture")
.withScopeProvider(new QCodeReference(RedisPresenceServiceTest.TenantScope.class)));
assertError(503, "GET", "/presence?table=document&pk=1", null, "alice");
assertError(503, "POST", "/presence/leases", "{\"table\":\"document\",\"pk\":\"1\"}", "alice");
assertError(503, "POST", "/presence/leases/renew", "{\"connectionId\":\"lease:missing\",\"table\":\"document\",\"pk\":\"1\",\"mode\":\"VIEWING\",\"sequence\":1}", "alice");
assertError(400, "POST", "/presence", "{\"connectionId\":\"tab\",\"table\":\"document\",\"pk\":\"1\",\"mode\":\"VIEWING\"}", "alice");
assertError(503, "DELETE", "/presence", "{\"connectionId\":\"tab\",\"table\":\"document\",\"pk\":\"1\"}", "alice");
assertError(404, "GET", "/presence?table=document&pk=3", null, "alice");
}
private static final ObjectMapper JSON = new ObjectMapper();
private final HttpClient client = HttpClient.newBuilder().followRedirects(HttpClient.Redirect.NEVER).build();
private QApplicationJavalinServer server;
@@ -115,6 +201,99 @@ class CollaborationApiHttpTest extends PersonalServiceFixture
MemoryStorage.objects.clear();
}
/*******************************************************************************
** Native record reads preserve authenticated transports and publish generic UI metadata.
*******************************************************************************/
@Test
void standardAttachmentRecordPreservesDeliveryModes() throws Exception
{
int id = json(upload("alice", "retained.txt", "original", "/records/document/1/attachments"), 200).get("id").asInt();
NativeDelivery.accessCalls = 0;
NativeDelivery.authenticated = true;
config.withAttachmentLifecycle(new QCodeReference(NativeDelivery.class));
MemoryStorage.lastGet = null;
assertThat(nativeGet("/table/collaboration_attachment/" + id, "alice").statusCode()).isEqualTo(403);
assertThat(NativeDelivery.accessCalls).isZero();
JsonNode record = json(nativeGet("/table/collaboration_attachment/" + id, "nativeview"), 200).get("record");
assertThat(record.at("/values/previewDelivery").asText()).isEqualTo("AUTHENTICATED_FETCH");
assertThat(record.at("/values/downloadDelivery").asText()).isEqualTo("AUTHENTICATED_FETCH");
assertThat(record.at("/values/previewUrl").asText()).contains("/qbit/record-files/files/1/downloads/");
assertThat(record.at("/values/downloadUrl").asText()).contains("/qbit/record-files/files/1/downloads/");
assertThat(MemoryStorage.lastGet).isNull();
assertThat(NativeDelivery.accessCalls).isEqualTo(2);
JsonNode metadata = json(nativeGet("/metaData/table/collaboration_attachment", "nativeview"), 200);
for(String field : List.of("preview", "download"))
{
boolean declared = false;
for(JsonNode adornment : metadata.at("/virtualFields/" + field + "Url/adornments"))
{
if("FILE_DOWNLOAD".equals(adornment.path("type").asText()))
{
assertThat(adornment.at("/values/deliveryModeField").asText()).isEqualTo(field + "Delivery");
declared = true;
}
}
assertThat(declared).as("Native %s field declares transport mode", field).isTrue();
}
assertThat(nativeGet("/table/collaboration_attachment/" + id, "nativeoutsider").statusCode()).isEqualTo(404);
assertThat(NativeDelivery.accessCalls).isEqualTo(2);
NativeDelivery.authenticated = false;
JsonNode signed = json(nativeGet("/table/collaboration_attachment/" + id, "nativeview"), 200).get("record");
assertThat(signed.at("/values/previewDelivery").asText()).isEqualTo("SIGNED_URL");
assertThat(signed.at("/values/downloadDelivery").asText()).isEqualTo("SIGNED_URL");
assertThat(signed.at("/values/downloadUrl").asText()).startsWith("https://files.example.invalid/");
}
/*******************************************************************************
** Use the supported native record and metadata routes, with the original actor cookie.
*******************************************************************************/
private HttpResponse<String> nativeGet(String path, String caller) throws Exception
{
String serverUrl = baseUrl.substring(0, baseUrl.indexOf("/qbit/"));
return (client.send(HttpRequest.newBuilder(URI.create(serverUrl + path)).timeout(Duration.ofSeconds(20))
.header("Cookie", "sessionUUID=" + caller + sessionSuffix).GET().build(), HttpResponse.BodyHandlers.ofString()));
}
/*******************************************************************************
** A trusted transport fixture proves owner metadata without a Record Files dependency.
*******************************************************************************/
public static class NativeDelivery extends com.kingsrook.qbits.collaboration.attachments.AttachmentLifecyclePolicy
{
private static int accessCalls;
private static boolean authenticated;
/*******************************************************************************
** Access must be authorized before the transport is returned.
*******************************************************************************/
@Override
public void beforeAccess(com.kingsrook.qqq.backend.core.model.data.QRecord attachment, boolean inline)
{
accessCalls++;
}
/*******************************************************************************
** The authenticated descriptor remains data, never a presigned storage capability.
*******************************************************************************/
@Override
public com.kingsrook.qbits.collaboration.attachments.AttachmentDownload downloadDelivery(com.kingsrook.qqq.backend.core.model.data.QRecord attachment)
throws com.kingsrook.qqq.backend.core.exceptions.QException
{
if(!authenticated)
{
return (null);
}
try
{
return (new com.kingsrook.qbits.collaboration.attachments.AttachmentDownload(URI.create(
"https://app.example/qqq/v1/qbit/record-files/files/1/downloads/00000000-0000-4000-8000-000000000001").toURL(), true));
}
catch(Exception failure)
{
throw (new com.kingsrook.qqq.backend.core.exceptions.QException("Invalid fixture URL", failure));
}
}
}
/*******************************************************************************
** Sharing validates recipients and access requests never reveal parent existence.
*******************************************************************************/
@@ -223,21 +402,6 @@ class CollaborationApiHttpTest extends PersonalServiceFixture
assertThat(ok("POST", "/presence", body).get("viewers")).hasSize(1);
assertThat(ok("GET", "/presence?table=document&pk=1", null).at("/viewers/0/mode").asText()).isEqualTo("EDITING");
assertThat(ok("GET", "/presence?table=document", null).at("/viewers/0/fieldName").isNull()).isTrue();
ok("POST", "/presence", body.replace("tab-one", "tab-two").replace("EDITING", "VIEWING"));
JsonNode merged = ok("GET", "/presence?table=document&pk=1", null);
assertThat(merged.get("viewers")).hasSize(1);
assertThat(merged.at("/viewers/0/connections").asInt()).isEqualTo(2);
assertThat(merged.get("leaseCount").asInt()).isEqualTo(2);
assertThat(merged.has("leases")).isFalse();
JsonNode perConnection = ok("GET", "/presence?table=document&pk=1&perConnection=true", null);
assertThat(perConnection.get("viewers")).hasSize(1);
assertThat(perConnection.get("leases")).hasSize(2);
assertThat(perConnection.at("/leases/0/connectionId").asText()).startsWith("c").doesNotContain("tab-");
assertThat(perConnection.at("/leases/0/since").asText()).isNotBlank();
assertThat(perConnection.at("/leases/0/sessionId").asText()).startsWith("s");
assertThat(perConnection.toString()).doesNotContain("alice" + sessionSuffix).doesNotContain(sessionSuffix);
assertError(400, "GET", "/presence?table=document&perConnection=maybe", null, "alice");
ok("DELETE", "/presence", "{\"connectionId\":\"tab-two\",\"table\":\"document\",\"pk\":\"1\"}");
assertError(404, "GET", "/presence?table=document&pk=3", null, "alice");
assertError(404, "POST", "/presence", body, "outsider");
assertError(400, "POST", "/presence", "{\"connectionId\":\"x\",\"table\":\"document\",\"mode\":\"BAD\"}", "alice");
@@ -246,6 +410,27 @@ class CollaborationApiHttpTest extends PersonalServiceFixture
assertThat(ok("GET", "/presence?table=document&pk=1", null).get("viewers")).isEmpty();
}
/*******************************************************************************
** HTTP-issued leases are invisible until activated and expire explicitly.
*******************************************************************************/
@Test
void issuedPresenceHttp() throws Exception
{
String lease = ok("POST", "/presence/leases", "{\"table\":\"document\",\"pk\":\"01\"}").get("connectionId").asText();
assertThat(lease).startsWith("lease:");
assertThat(ok("GET", "/presence?table=document&pk=1", null).get("viewers")).isEmpty();
String body = new ObjectMapper().writeValueAsString(Map.of("connectionId", lease, "table", "document", "pk", "1", "mode", "EDITING", "fieldName", "title", "sequence", 1));
assertThat(ok("POST", "/presence/leases/renew", body).get("viewers")).hasSize(1);
assertError(404, "POST", "/presence/leases", "{\"table\":\"document\",\"pk\":\"3\"}", "alice");
assertError(404, "POST", "/presence/leases/renew", body, "outsider");
assertError(400, "POST", "/presence/leases/renew", body.replace("\"sequence\":1", "\"sequence\":0"), "alice");
String leave = new ObjectMapper().writeValueAsString(Map.of("connectionId", lease, "table", "document", "pk", "1", "sequence", 2));
ok("DELETE", "/presence", leave);
var expired = json(request("POST", "/presence/leases/renew", body, "alice"), 409);
assertThat(expired.get("code").asText()).isEqualTo("PRESENCE_LEASE_EXPIRED");
assertThat(ok("GET", "/presence?table=document&pk=1", null).get("viewers")).isEmpty();
}
/*******************************************************************************
** Every personal operation and offset cursor runs through the HTTP boundary.
*******************************************************************************/
@@ -316,6 +501,36 @@ class CollaborationApiHttpTest extends PersonalServiceFixture
assertError(404, "GET", "/attachments/" + id + "/preview", null, "alice");
}
/*******************************************************************************
** Existing offset cursors retain wire scope while native lists fetch pages.
*******************************************************************************/
@Test
void attachmentPagesPreserveCursorScopeAndCurrentParentAccess() throws Exception
{
List<Integer> ids = new java.util.ArrayList<>();
for(int i = 0; i < 5; i++)
{
ids.add(json(upload("alice", "file-" + i + ".txt", "x", "/records/document/1/attachments"), 200).get("id").asInt());
}
JsonNode first = ok("GET", "/records/document/1/attachments?limit=2", null);
assertThat(first.get("items")).hasSize(2);
assertThat(first.at("/items/0/id").asInt()).isEqualTo(ids.get(4));
String cursor = first.get("nextCursor").asText();
String decoded = new String(java.util.Base64.getUrlDecoder().decode(cursor), StandardCharsets.UTF_8);
assertThat(decoded).startsWith("1:").endsWith(":2");
JsonNode second = ok("GET", "/records/document/1/attachments?limit=2&cursor=" + cursor, null);
assertThat(second.get("items")).hasSize(2);
assertThat(second.at("/items/0/id").asInt()).isEqualTo(ids.get(2));
JsonNode last = ok("GET", "/records/document/1/attachments?limit=2&cursor=" + second.get("nextCursor").asText(), null);
assertThat(last.get("items")).hasSize(1);
assertThat(last.get("nextCursor").isNull()).isTrue();
assertError(400, "GET", "/records/document/2/attachments?cursor=" + cursor, null, "alice");
assertError(400, "GET", "/records/document/1/attachments?limit=101", null, "alice");
assertError(400, "GET", "/records/document/1/attachments?cursor=invalid", null, "alice");
assertError(404, "GET", "/records/document/1/attachments?cursor=" + cursor, null, "outsider");
assertThat(ok("GET", "/records/document/2/attachments", null).get("items")).isEmpty();
}
/*******************************************************************************
** Vocabulary creation and record associations preserve caller permissions.
*******************************************************************************/
@@ -361,6 +576,80 @@ class CollaborationApiHttpTest extends PersonalServiceFixture
assertThat(tagQuery("IN", "[\"alpha\"]")).containsExactlyInAnyOrder(1, 2);
}
/*******************************************************************************
** EAV choices retain their stored ids and translated labels on the HTTP wire.
*******************************************************************************/
@Test
void customFieldValuesThroughTableReads() throws Exception
{
admin();
instance.addBackend(new RDBMSBackendMetaData().withName("fallback").withVendor("h2"));
instance.getTable("document").setBackendName("fallback");
QRecord definition = new InsertAction().execute(new InsertInput(CollaborationTables.FIELD_DEFINITION).withRecords(List.of(
new QRecord().withValue("tableName", "document").withValue("name", "choice").withValue("label", "Choice").withValue("type", "SELECT")
.withValue("options", "[{\"id\":\"x\",\"label\":\"Choice X\"}]")))).getRecords().getFirst();
assertThat(definition.getErrors()).isNullOrEmpty();
FieldDefinitions.apply(definition.getValue("id"), "Publish");
instance.getTable("document").setBackendName("memory");
CollaborationRecords.insert(CollaborationTables.FIELD_VALUE, new QRecord().withValue("tableName", "document").withValue("recordId", "1")
.withValue("fieldName", "choice").withValue("valueText", "x").withValue("parentDeleted", false));
login("alice");
String serverUrl = baseUrl.substring(0, baseUrl.indexOf("/qbit/"));
HttpResponse<String> response = client.send(HttpRequest.newBuilder(URI.create(serverUrl + "/table/document/query")).timeout(Duration.ofSeconds(20))
.header("Cookie", "sessionUUID=alice" + sessionSuffix).header("Content-Type", "application/json")
.POST(HttpRequest.BodyPublishers.ofString("{\"filter\":{\"orderBys\":[{\"fieldName\":\"id\"}]}}")).build(), HttpResponse.BodyHandlers.ofString());
JsonNode records = json(response, 200).get("records");
assertThat(records).hasSize(2);
assertThat(records.get(0).at("/values/id").asInt()).isEqualTo(1);
assertThat(records.get(0).at("/values/cf_choice").asText()).isEqualTo("x");
assertThat(records.get(0).at("/displayValues/cf_choice").asText()).isEqualTo("Choice X");
assertThat(records.get(1).at("/values/cf_choice").isMissingNode()).isTrue();
response = client.send(HttpRequest.newBuilder(URI.create(serverUrl + "/table/document/1")).timeout(Duration.ofSeconds(20))
.header("Cookie", "sessionUUID=alice" + sessionSuffix).GET().build(), HttpResponse.BodyHandlers.ofString());
JsonNode record = json(response, 200).get("record");
assertThat(record.at("/values/cf_choice").asText()).isEqualTo("x");
assertThat(record.at("/displayValues/cf_choice").asText()).isEqualTo("Choice X");
response = client.send(HttpRequest.newBuilder(URI.create(serverUrl + "/table/document/1")).timeout(Duration.ofSeconds(20))
.header("Cookie", "sessionUUID=outsider" + sessionSuffix).GET().build(), HttpResponse.BodyHandlers.ofString());
assertThat(response.statusCode()).isEqualTo(404);
assertThat(response.body()).doesNotContain("Choice X", "First document");
}
/*******************************************************************************
** Table reads expose tag labels while retaining parent-record authorization.
*******************************************************************************/
@Test
void tagValuesThroughTableReads() throws Exception
{
RuntimeMetaDataOverlay.refresh(instance);
int tag = ok("POST", "/tags", "{\"table\":\"document\",\"label\":\"Review, urgent\"}").get("id").asInt();
ok("PUT", "/records/document/1/tags", "{\"add\":[" + tag + "]}");
String serverUrl = baseUrl.substring(0, baseUrl.indexOf("/qbit/"));
HttpResponse<String> response = client.send(HttpRequest.newBuilder(URI.create(serverUrl + "/table/document/query")).timeout(Duration.ofSeconds(20))
.header("Cookie", "sessionUUID=alice" + sessionSuffix).header("Content-Type", "application/json")
.POST(HttpRequest.BodyPublishers.ofString("{\"filter\":{\"orderBys\":[{\"fieldName\":\"id\"}]}}")).build(), HttpResponse.BodyHandlers.ofString());
JsonNode records = json(response, 200).get("records");
assertThat(records).hasSize(2);
assertThat(records.get(0).at("/values/id").asInt()).isEqualTo(1);
assertThat(records.get(0).at("/values/collaborationTags").asText()).isEqualTo("Review, urgent");
assertThat(records.get(0).at("/displayValues/collaborationTags").asText()).isEqualTo("Review, urgent");
assertThat(records.get(1).at("/values/id").asInt()).isEqualTo(2);
assertThat(records.get(1).at("/values/collaborationTags").isMissingNode()).as(records.toString()).isTrue();
response = client.send(HttpRequest.newBuilder(URI.create(serverUrl + "/table/document/1")).timeout(Duration.ofSeconds(20))
.header("Cookie", "sessionUUID=alice" + sessionSuffix).GET().build(), HttpResponse.BodyHandlers.ofString());
JsonNode record = json(response, 200).get("record");
assertThat(record.at("/values/collaborationTags").asText()).isEqualTo("Review, urgent");
assertThat(record.at("/displayValues/collaborationTags").asText()).isEqualTo("Review, urgent");
response = client.send(HttpRequest.newBuilder(URI.create(serverUrl + "/table/document/1")).timeout(Duration.ofSeconds(20))
.header("Cookie", "sessionUUID=outsider" + sessionSuffix).GET().build(), HttpResponse.BodyHandlers.ofString());
assertThat(response.body()).doesNotContain("Review, urgent", "First document");
assertThat(response.statusCode()).isEqualTo(404);
}
/*******************************************************************************
** Run the v1 table query filtered on the virtual tags field; return matching ids.
*******************************************************************************/
@@ -417,6 +706,21 @@ class CollaborationApiHttpTest extends PersonalServiceFixture
JsonNode candidate = ok("GET", "/mentions/candidates?table=document&pk=1&q=carol", null).at("/items/0");
assertThat(candidate.at("/person/userId").asText()).isEqualTo("carol");
assertThat(candidate.get("canRead").asText()).isEqualTo("true");
json(request("POST", "/records/document/1/comments", "{\"body\":\"Review @[Alice](user:alice)\"}", "carol"), 200);
ok("POST", "/records/document/1/comments", "{\"body\":\"Review @[Carol](user:carol)\"}");
// Delivery is external to this HTTP fixture; persist its receipt before testing read state.
for(var mention : CollaborationRecords.query(com.kingsrook.qbits.collaboration.model.CollaborationTables.MENTION,
new com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter()))
{
CollaborationRecords.update(com.kingsrook.qbits.collaboration.model.CollaborationTables.MENTION,
new com.kingsrook.qqq.backend.core.model.data.QRecord().withValue("id", mention.getValue("id")).withValue("notifiedAt", java.time.Instant.now()));
}
assertThat(ok("GET", "/my-work", null).get("unreadMentionCount").asLong()).isPositive();
assertThat(json(request("GET", "/my-work", null, "carol"), 200).get("unreadMentionCount").asLong()).isEqualTo(1);
ok("POST", "/my-work/mentions/read", null);
assertThat(ok("GET", "/my-work", null).get("unreadMentionCount").asLong()).isZero();
assertThat(json(request("GET", "/my-work", null, "carol"), 200).get("unreadMentionCount").asLong()).isEqualTo(1);
ok("POST", "/my-work/mentions/read", null);
assertThat(ok("GET", "/records/document/1/follow", null).at("/follow/reason").asText()).isEqualTo("COMMENTED");
JsonNode state = ok("PUT", "/records/document/1/follow", "{\"notifyOnChange\":false,\"notifyOnComment\":true}");
assertThat(state.at("/follow/notifyOnChange").asBoolean()).isFalse();
@@ -524,7 +828,7 @@ class CollaborationApiHttpTest extends PersonalServiceFixture
JsonNode version = json(client.send(HttpRequest.newBuilder(URI.create(serverUrl + "/qqq/v1/version")).GET().build(),
HttpResponse.BodyHandlers.ofString()), 200);
assertThat(version.at("/extensions/0/name").asText()).isEqualTo("collaboration");
assertThat(version.at("/extensions/0/version").asText()).isEqualTo("1.2.0");
assertThat(version.at("/extensions/0/version").asText()).isEqualTo("1.3.0");
}
/*******************************************************************************
@@ -568,7 +872,11 @@ class CollaborationApiHttpTest extends PersonalServiceFixture
case "carol" -> "Carol Diaz";
default -> caller;
}));
session.withPermissions("document.read").withSecurityKeyValue("tenant", caller.equals("outsider") ? "b" : "a");
session.withPermissions("document.read").withSecurityKeyValue("tenant", (caller.equals("outsider") || caller.equals("nativeoutsider")) ? "b" : "a");
if(caller.startsWith("native"))
{
session.withPermissions("collaboration.view");
}
if(caller.equals("alice") || caller.equals("carol"))
{
session.withPermissions("document.write");
@@ -84,7 +84,8 @@ class CoreContractsTest extends PersonalServiceFixture
RuntimeMetaDataOverlay.refresh(instance);
long generation = RuntimeMetaDataOverlay.getGeneration(instance);
assertThat(instance.getTable("document").getVirtualFields()).containsKey("collaborationTags");
assertThat(instance.getTable("document").getVirtualField("collaborationTags").getIsQuerySelectable()).isFalse();
assertThat(instance.getTable("document").getVirtualField("collaborationTags").getIsQuerySelectable()).isTrue();
assertThat(instance.getTable("document").getVirtualField("collaborationTags").getValueProvider()).isNotNull();
QBackendTransaction tx = new QBackendTransaction();
var row = new InsertAction().execute(new InsertInput(CollaborationTables.FEATURE_SETTING).withTransaction(tx).withRecords(List.of(
new QRecord().withValue("scopeType", "TABLE").withValue("scopeName", "document").withValue("feature", "TAGS")
@@ -0,0 +1,188 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration;
import java.sql.Connection;
import java.sql.DriverManager;
import java.util.Set;
import com.kingsrook.qbits.collaboration.customfields.CustomFieldValueProvider;
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QVirtualFieldMetaData;
import com.kingsrook.qqq.backend.module.postgres.model.metadata.PostgreSQLBackendMetaData;
import com.kingsrook.qqq.backend.module.rdbms.actions.RDBMSTransaction;
import com.kingsrook.qqq.backend.module.rdbms.jdbc.ConnectionManager;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ValueSource;
import org.testcontainers.containers.GenericContainer;
import org.testcontainers.containers.wait.strategy.Wait;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** Verify provider reads using a real database and explicit caller transactions.
*******************************************************************************/
class CustomFieldValuesPostgresTest extends PersonalServiceFixture
{
private static GenericContainer<?> postgres;
/*******************************************************************************
** Own an isolated database; never connect to an application database.
*******************************************************************************/
@BeforeAll
static void startPostgres()
{
postgres = new GenericContainer<>("postgres:17-alpine").withEnv("POSTGRES_PASSWORD", "eav-fixture-only")
.withExposedPorts(5432).waitingFor(Wait.forLogMessage(".*database system is ready to accept connections.*", 2));
postgres.start();
}
/*******************************************************************************
** Stop only the fixture's database.
*******************************************************************************/
@AfterAll
static void stopPostgres()
{
if(postgres != null)
{
postgres.stop();
}
}
/*******************************************************************************
** Reuse the secured host metadata with real PostgreSQL storage.
*******************************************************************************/
@Override
@BeforeEach
public void setUp() throws Exception
{
super.setUp();
instance.getBackends().remove("memory");
instance.addBackend(new PostgreSQLBackendMetaData().withName("memory").withHostName(postgres.getHost())
.withPort(postgres.getMappedPort(5432)).withDatabaseName("postgres").withUsername("postgres").withPassword("eav-fixture-only"));
try(Connection connection = connect(); var sql = connection.createStatement())
{
sql.execute("DROP SCHEMA public CASCADE; CREATE SCHEMA public");
sql.execute(CollaborationSchema.postgresScript(config));
sql.execute("CREATE TABLE document (id INTEGER PRIMARY KEY, title TEXT, tenant TEXT)");
sql.execute("INSERT INTO document VALUES (1, 'Visible', 'a'), (3, 'Private', 'b')");
}
CollaborationRecords.insert(CollaborationTables.FIELD_DEFINITION, new QRecord().withValue("tableName", "document")
.withValue("name", "rating").withValue("label", "Rating").withValue("type", "INTEGER").withValue("status", "ACTIVE"));
instance.getTable("document").withVirtualField(new QVirtualFieldMetaData("cf_rating", QFieldType.INTEGER).withLabel("Rating")
.withIsQuerySelectable(true).withValueProvider(new QCodeReference(CustomFieldValueProvider.class)));
}
/*******************************************************************************
** Release transaction state and pooled connections before parent fixture cleanup.
*******************************************************************************/
@Override
public void tearDown()
{
QContext.clearTransaction();
ConnectionManager.closeConnectionProviders(instance);
super.tearDown();
}
/*******************************************************************************
** A separate connection establishes committed visibility independently.
*******************************************************************************/
private Connection connect() throws Exception
{
return (DriverManager.getConnection("jdbc:postgresql://" + postgres.getHost() + ":" + postgres.getMappedPort(5432)
+ "/postgres", "postgres", "eav-fixture-only"));
}
/*******************************************************************************
** Both vocabulary and assignments use the explicit transaction's visibility.
*******************************************************************************/
@ParameterizedTest
@ValueSource(booleans = {false, true})
void tagLabelsSeePendingVocabularyAndCompletion(boolean commit) throws Exception
{
instance.getTable("document").withVirtualField(new QVirtualFieldMetaData("collaborationTags", QFieldType.STRING).withLabel("Tags")
.withIsQuerySelectable(true).withPossibleValueSourceName(CollaborationTables.TAG).withShouldTranslatePossibleValues(false)
.withValueProvider(new QCodeReference(com.kingsrook.qbits.collaboration.tags.TagValueProvider.class)));
var tags = new com.kingsrook.qbits.collaboration.tags.TagsService();
try(var transaction = new RDBMSTransaction(connect()))
{
QContext.setTransaction(transaction);
var tag = tags.create("document", "Pending transaction label", null);
tags.apply("document", 1, java.util.List.of(tag.id()), java.util.List.of());
QContext.clearTransaction();
assertThat(QContext.getQBackendTransaction()).isNull();
var pending = new QueryAction().execute(new QueryInput("document").withFieldNamesToInclude(Set.of("collaborationTags"))
.withShouldTranslatePossibleValues(true).withShouldGenerateDisplayValues(true).withTransaction(transaction)).getRecords();
assertThat(pending).hasSize(1);
assertThat(pending.getFirst().getValues()).containsOnlyKeys("collaborationTags").containsEntry("collaborationTags", "Pending transaction label");
assertThat(pending.getFirst().getDisplayValue("collaborationTags")).isEqualTo("Pending transaction label");
assertThat(new QueryAction().execute(new QueryInput("document")).getRecords().getFirst().getValue("collaborationTags")).isNull();
if(commit)
{
transaction.commit();
}
else
{
transaction.rollback();
}
}
assertThat(new QueryAction().execute(new QueryInput("document")).getRecords().getFirst().getValue("collaborationTags"))
.isEqualTo(commit ? "Pending transaction label" : null);
}
/*******************************************************************************
** Explicit query transactions see pending values even without thread context.
*******************************************************************************/
@ParameterizedTest
@ValueSource(booleans = {false, true})
void explicitTransactionSeesPendingValueAndCompletion(boolean commit) throws Exception
{
try(var transaction = new RDBMSTransaction(connect()))
{
QContext.setTransaction(transaction);
CollaborationRecords.insert(CollaborationTables.FIELD_VALUE, new QRecord().withValue("tableName", "document").withValue("recordId", "1")
.withValue("fieldName", "rating").withValue("valueNumber", 42).withValue("parentDeleted", false));
QContext.clearTransaction();
assertThat(QContext.getQBackendTransaction()).isNull();
var pending = new QueryAction().execute(new QueryInput("document").withFieldNamesToInclude(Set.of("cf_rating"))
.withTransaction(transaction)).getRecords();
assertThat(pending).hasSize(1);
assertThat(pending.getFirst().getValues()).containsOnlyKeys("cf_rating").containsEntry("cf_rating", 42);
assertThat(new QueryAction().execute(new QueryInput("document")).getRecords().getFirst().getValue("cf_rating")).isNull();
if(commit)
{
transaction.commit();
}
else
{
transaction.rollback();
}
}
assertThat(new QueryAction().execute(new QueryInput("document")).getRecords().getFirst().getValue("cf_rating")).isEqualTo(commit ? 42 : null);
}
}
@@ -37,6 +37,7 @@ import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
import com.kingsrook.qqq.backend.core.actions.tables.DeleteAction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.instances.QInstanceEnricher;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
@@ -44,13 +45,19 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.helpcontent.HelpContentMetaDataProvider;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.fields.JsonPathStorage;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.PermissionLevel;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRules;
import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData;
import com.kingsrook.qbits.collaboration.app.CollaborationHome;
import com.kingsrook.qqq.backend.core.model.dashboard.widgets.TableData;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.Arguments;
import org.junit.jupiter.params.provider.MethodSource;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
@@ -59,6 +66,48 @@ import static org.assertj.core.api.Assertions.assertThatThrownBy;
*******************************************************************************/
public class CustomFieldsTest extends PersonalServiceFixture
{
/*******************************************************************************
** Publication binds stored guidance to sections that did not exist at startup.
*******************************************************************************/
@Test
void publishedSectionReceivesStoredHelp() throws Exception
{
new HelpContentMetaDataProvider().defineAll(instance, "memory", table ->
table.withPermissionRules(new QPermissionRules()
.withLevel(PermissionLevel.NOT_PROTECTED)));
new QInstanceEnricher(instance).enrich();
admin();
var help = new InsertAction().execute(new InsertInput("helpContent").withRecords(List.of(new QRecord()
.withValue("key", "table:document;section:customFields").withValue("role", "ALL_SCREENS")
.withValue("format", "MARKDOWN").withValue("content", "Synthetic dynamic section guidance.")))).getRecords().getFirst();
assertThat(help.getErrors()).isNullOrEmpty();
QRecord definition = draft("guided", "STRING");
FieldDefinitions.apply(definition.getValue("id"), "Publish");
assertThat(instance.getTable("document").getSection("customFields").getHelpContents())
.anySatisfy(content -> assertThat(content.getContent()).contains("Synthetic dynamic section guidance."));
}
/*******************************************************************************
** A configured string bound rejects writes without replacing persisted values.
*******************************************************************************/
@Test
void publishedStringRejectsValuesBeyondItsMaximumLength() throws Exception
{
admin();
QRecord definition = draft("note", "STRING");
new UpdateAction().execute(new UpdateInput(CollaborationTables.FIELD_DEFINITION).withRecords(List.of(
new QRecord().withValue("id", definition.getValue("id")).withValue("maxLength", 5))));
FieldDefinitions.apply(definition.getValue("id"), "Publish");
login("alice");
var valid = new UpdateAction().execute(new UpdateInput("document").withRecords(List.of(
new QRecord().withValue("id", 1).withValue("cf_note", "valid")))).getRecords().getFirst();
assertThat(valid.getErrors()).isNullOrEmpty();
var invalid = new UpdateAction().execute(new UpdateInput("document").withRecords(List.of(
new QRecord().withValue("id", 1).withValue("cf_note", "too long")))).getRecords().getFirst();
assertThat(invalid.getErrors()).isNotEmpty();
assertThat(query("cf_note", QCriteriaOperator.EQUALS, "valid")).hasSize(1);
}
/*******************************************************************************
** Refreshing a SELECT must replace its generated vocabulary and allow later fields.
*******************************************************************************/
@@ -242,6 +291,59 @@ public class CustomFieldsTest extends PersonalServiceFixture
assertThat(FieldProcesses.metadata()).extracting(process -> process.getLabel()).containsExactly("Publish field", "Archive field", "Rebuild index");
}
/*******************************************************************************
** Typed EAV storage returns the declared Java value through an actual query.
*******************************************************************************/
@ParameterizedTest
@MethodSource("eavValueTypes")
void eavValueTypes(String type, String column, java.io.Serializable stored, java.io.Serializable expected) throws Exception
{
admin();
instance.addBackend(new RDBMSBackendMetaData().withName("fallback").withVendor("h2"));
instance.getTable("document").setBackendName("fallback");
QRecord draft = new QRecord().withValue("tableName", "document").withValue("name", "typed").withValue("label", "Typed").withValue("type", type);
if(List.of("SELECT", "MULTI_SELECT").contains(type))
{
draft.withValue("options", "[{\"id\":\"x\",\"label\":\"Choice X\"}]");
}
if("RECORD_LINK".equals(type))
{
draft.withValue("linkTableName", CollaborationTables.USER);
}
QRecord definition = new InsertAction().execute(new InsertInput(CollaborationTables.FIELD_DEFINITION).withRecords(List.of(draft))).getRecords().getFirst();
assertThat(definition.getErrors()).isNullOrEmpty();
FieldDefinitions.apply(definition.getValue("id"), "Publish");
instance.getTable("document").setBackendName("memory");
CollaborationRecords.insert(CollaborationTables.FIELD_VALUE, new QRecord().withValue("tableName", "document").withValue("recordId", "1")
.withValue("fieldName", "typed").withValue(column, stored).withValue("parentDeleted", false));
login("alice");
var records = new QueryAction().execute(new QueryInput("document").withFieldNamesToInclude(java.util.Set.of("cf_typed"))).getRecords();
assertThat(records).hasSize(2);
assertThat(records).allSatisfy(record -> assertThat(record.getValues()).containsOnlyKeys("cf_typed"));
assertThat(records).extracting(record -> record.getValue("cf_typed")).containsExactlyInAnyOrder(expected, null);
Object actual = records.stream().map(record -> record.getValue("cf_typed")).filter(java.util.Objects::nonNull).findFirst().orElseThrow();
assertThat(actual).isExactlyInstanceOf(expected.getClass());
}
/*******************************************************************************
** Keep expectations independent of the provider's column-mapping helper.
*******************************************************************************/
static java.util.stream.Stream<Arguments> eavValueTypes()
{
return (java.util.stream.Stream.of(
Arguments.of("INTEGER", "valueNumber", new BigDecimal("7.0"), 7),
Arguments.of("DECIMAL", "valueNumber", new BigDecimal("12.345"), new BigDecimal("12.345")),
Arguments.of("BOOLEAN", "valueBoolean", false, false),
Arguments.of("DATE", "valueDate", java.time.LocalDate.of(2026, 10, 22), java.time.LocalDate.of(2026, 10, 22)),
Arguments.of("DATE_TIME", "valueDateTime", java.time.Instant.parse("2026-10-22T12:34:56Z"), java.time.Instant.parse("2026-10-22T12:34:56Z")),
Arguments.of("SELECT", "valueText", "x", "x"),
Arguments.of("MULTI_SELECT", "valueText", "[\"x\"]", "[\"x\"]"),
Arguments.of("USER", "valueText", "alice", "alice"),
Arguments.of("RECORD_LINK", "valueText", "alice", "alice"),
Arguments.of("STRING", "valueText", "A, B", "A, B"),
Arguments.of("TEXT", "valueText", "First\nSecond", "First\nSecond")));
}
/*******************************************************************************
** An unsupported JSON backend gets typed, read-only subquery filters.
*******************************************************************************/
@@ -256,18 +358,50 @@ public class CustomFieldsTest extends PersonalServiceFixture
instance.getTable("document").setBackendName("memory");
var virtual = instance.getTable("document").getVirtualField("cf_rating");
assertThat(virtual).isNotNull();
assertThat(virtual.getIsQuerySelectable()).isFalse();
assertThat(virtual.getIsQuerySelectable()).isTrue();
assertThat(virtual.getIsEditable()).isFalse();
assertThat(FieldDefinitions.get(definition.getValue("id")).getValueString("storage")).isEqualTo("EAV");
CollaborationRecords.insert(CollaborationTables.FIELD_VALUE, new QRecord().withValue("tableName", "document").withValue("recordId", "1")
.withValue("fieldName", "rating").withValue("valueNumber", new BigDecimal("7")).withValue("parentDeleted", false));
QContext.getQSession().withSecurityKeyValue("allTenants", true);
CollaborationRecords.insert(CollaborationTables.FIELD_VALUE, new QRecord().withValue("tableName", "document").withValue("recordId", "3")
.withValue("fieldName", "rating").withValue("valueNumber", new BigDecimal("99")).withValue("parentDeleted", false));
login("alice");
var selected = new QueryAction().execute(new QueryInput("document").withFieldNamesToInclude(java.util.Set.of("cf_rating"))).getRecords();
assertThat(selected).hasSize(2);
assertThat(selected).allSatisfy(row -> assertThat(row.getValues()).containsOnlyKeys("cf_rating"));
assertThat(selected).extracting(row -> row.getValue("cf_rating")).containsExactlyInAnyOrder(7, null);
assertThat(query("cf_rating", QCriteriaOperator.GREATER_THAN, 5)).extracting(row -> row.getValueInteger("id")).containsExactly(1);
assertThat(query("cf_rating", QCriteriaOperator.NOT_EQUALS, 7)).extracting(row -> row.getValueInteger("id")).containsExactly(2);
assertThat(new QueryAction().execute(new QueryInput("document").withFilter(new QQueryFilter().withCriteria("cf_rating", QCriteriaOperator.IS_BLANK))).getRecords())
.extracting(row -> row.getValueInteger("id")).containsExactly(2);
// Reuse the same session and query to prove results are not cached across access changes.
QueryInput projected = new QueryInput("document").withFieldNamesToInclude(java.util.Set.of("cf_rating"));
QContext.getQSession().getSecurityKeyValues().put("tenant", List.of("b"));
assertThat(new QueryAction().execute(projected).getRecords()).extracting(row -> row.getValue("cf_rating")).containsExactly(99);
QContext.getQSession().getSecurityKeyValues().put("tenant", List.of("a"));
assertThat(new QueryAction().execute(projected).getRecords()).extracting(row -> row.getValue("cf_rating")).containsExactlyInAnyOrder(7, null);
QContext.getQSession().setPermissions(java.util.Set.of());
assertThatThrownBy(() -> new QueryAction().execute(projected)).isInstanceOf(com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException.class);
login("alice");
assertThat(com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordReferenceTableMetaData.of(instance.getTable(CollaborationTables.FIELD_VALUE))).isNotNull();
assertThat(new QueryAction().execute(new QueryInput(CollaborationTables.FIELD_VALUE)
.withFilter(new QQueryFilter().withCriteria("tableName", QCriteriaOperator.EQUALS, "document")
.withCriteria("recordId", QCriteriaOperator.IN, List.of("1", "3")))).getRecords())
.extracting(row -> row.getValueString("recordId")).containsExactly("1");
virtual.setIsHidden(true);
assertThat(new QueryAction().execute(new QueryInput("document")).getRecords()).allSatisfy(row -> assertThat(row.getValues()).doesNotContainKey("cf_rating"));
virtual.setIsHidden(false);
admin();
QContext.getQSession().withSecurityKeyValue("allTenants", true);
assertThat(FieldDefinitions.valueCount(FieldDefinitions.get(definition.getValue("id")))).isEqualTo(2);
QRecord deleted = CollaborationRecords.insert(CollaborationTables.FIELD_VALUE, new QRecord().withValue("tableName", "document").withValue("recordId", "2")
.withValue("fieldName", "rating").withValue("valueNumber", new BigDecimal("88")).withValue("parentDeleted", true));
assertThat(deleted.getValueBoolean("parentDeleted")).isTrue();
login("alice");
assertThat(new QueryAction().execute(new QueryInput("document").withFieldNamesToInclude(java.util.Set.of("cf_rating"))).getRecords())
.extracting(row -> row.getValue("cf_rating")).containsExactlyInAnyOrder(7, null);
admin();
assertThat(FieldDefinitions.valueCount(FieldDefinitions.get(definition.getValue("id")))).isEqualTo(1);
var expander = new com.kingsrook.qbits.collaboration.customfields.FieldCriteriaExpander();
assertThat(expander.expandSearchTerm(instance.getTable("document"), "text", com.kingsrook.qqq.backend.core.actions.tables.query.SearchTermKind.TEXT, null)).isNotNull();
assertThat(com.kingsrook.qbits.collaboration.customfields.FieldCriteriaExpander.valueColumn("DATE")).isEqualTo("valueDate");
@@ -0,0 +1,214 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import com.github.dockerjava.api.command.InspectContainerResponse;
import com.github.dockerjava.api.model.ExposedPort;
import com.github.dockerjava.api.model.PortBinding;
import com.github.dockerjava.api.model.Ports;
import org.testcontainers.DockerClientFactory;
import org.testcontainers.containers.GenericContainer;
/*******************************************************************************
** Where a disposable test container is reachable FROM THIS JVM.
**
** The original behavior is Testcontainers' own: the host it reports (which honors
** TESTCONTAINERS_HOST_OVERRIDE) and the published port. The bridge address of the
** sibling is used in exactly one topology: a JVM inside a build container that the
** SAME daemon reports on its default bridge (a local full verify through the host
** socket). Being inside some container is not enough: on the native runner the job
** container is not a container of the isolated test daemon at all, so a sibling's
** bridge address (a private address of that other daemon) is unreachable and the
** published endpoint at the configured host must be dialed.
**
** The choice is made from the daemon's own description of the build container's
** networks (no connection probing). If the build container cannot be identified or
** is not on the default bridge (a job network, host networking, a remote or
** separate daemon), the published endpoint is used. Timeouts, scenarios and
** assertions are unchanged. The address is re-read from the daemon on every call
** (a restarted container may get another bridge IP or published port).
*******************************************************************************/
final class FixtureEndpoint
{
private static final String DEFAULT_BRIDGE = "bridge";
private static final Pattern MOUNTINFO_CONTAINER_ID = Pattern.compile("/docker/containers/([0-9a-f]{64})/");
private static volatile Boolean buildContainerOnDefaultBridge;
/***************************************************************************
** Host and port to dial.
***************************************************************************/
record Address(String host, int port)
{
}
/***************************************************************************
** Utility class.
***************************************************************************/
private FixtureEndpoint()
{
}
/***************************************************************************
** Resolve the endpoint of one exposed port of a running container, from a fresh
** inspection (not Testcontainers' cached mapping).
***************************************************************************/
static Address of(GenericContainer<?> container, int internalPort)
{
InspectContainerResponse inspected = container.getDockerClient().inspectContainerCmd(container.getContainerId()).exec();
var networks = inspected.getNetworkSettings().getNetworks();
String bridgeIp = networks == null || networks.get(DEFAULT_BRIDGE) == null ? null : networks.get(DEFAULT_BRIDGE).getIpAddress();
Integer publishedPort = null;
var bindings = inspected.getNetworkSettings().getPorts().getBindings().get(new ExposedPort(internalPort));
if(bindings != null && bindings.length > 0 && bindings[0].getHostPortSpec() != null)
{
publishedPort = Integer.valueOf(bindings[0].getHostPortSpec());
}
return (choose(onBridgeWithBuildContainer(), bridgeIp, internalPort, container.getHost(), publishedPort));
}
/***************************************************************************
** Pure decision, separately tested. With the build container on the default
** bridge: the sibling's bridge address and its own port. Otherwise the published
** endpoint exactly as Testcontainers reports it.
***************************************************************************/
static Address choose(boolean buildContainerOnDefaultBridge, String bridgeIp, int internalPort, String publishedHost, Integer publishedPort)
{
if(buildContainerOnDefaultBridge)
{
if(bridgeIp == null || bridgeIp.isBlank())
{
throw new IllegalStateException("FIXTURE_ENDPOINT_NO_BRIDGE_ADDRESS: the build container cannot reach a sibling without its bridge address");
}
return (new Address(bridgeIp, internalPort));
}
if(publishedHost == null || publishedHost.isBlank() || publishedPort == null)
{
throw new IllegalStateException("FIXTURE_ENDPOINT_NOT_PUBLISHED: no published endpoint to dial");
}
return (new Address(publishedHost, publishedPort));
}
/***************************************************************************
** How the fixture publishes its port: the historical loopback-only binding
** wherever a loopback dial can work, otherwise the daemon's own default binding.
***************************************************************************/
static PortBinding publishedBinding(int internalPort)
{
boolean loopback = loopbackPublication(onBridgeWithBuildContainer(), DockerClientFactory.instance().dockerHostIpAddress());
return (new PortBinding(loopback ? Ports.Binding.bindIpAndPort("127.0.0.1", 0) : Ports.Binding.empty(), new ExposedPort(internalPort)));
}
/***************************************************************************
** Pure decision, separately tested. A loopback-only publication is reachable
** when the build container dials the bridge address (publication unused) or the
** Docker host is this machine; if Testcontainers dials another host (a remote or
** separate daemon, TESTCONTAINERS_HOST_OVERRIDE) a loopback binding could never
** be reached, so the daemon's configured default binding is used instead.
***************************************************************************/
static boolean loopbackPublication(boolean buildContainerOnDefaultBridge, String dockerHost)
{
return (buildContainerOnDefaultBridge || dockerHost == null || dockerHost.isBlank()
|| dockerHost.equals("localhost") || dockerHost.equals("127.0.0.1") || dockerHost.equals("::1"));
}
/***************************************************************************
** Whether this JVM runs in a container that the daemon reports on the default
** bridge. Decided once per JVM from the daemon's description of this container.
***************************************************************************/
private static boolean onBridgeWithBuildContainer()
{
Boolean known = buildContainerOnDefaultBridge;
if(known == null)
{
known = decideBuildContainerOnDefaultBridge();
buildContainerOnDefaultBridge = known;
}
return (known);
}
/***************************************************************************
** Inspect the build container; anything that cannot be established means the
** original published endpoint (false).
***************************************************************************/
private static boolean decideBuildContainerOnDefaultBridge()
{
if(!Files.exists(Path.of("/.dockerenv")))
{
return (false);
}
try
{
String id = ownContainerId(Files.readString(Path.of("/proc/self/mountinfo")), System.getenv("HOSTNAME"));
if(id == null)
{
return (false);
}
var networks = DockerClientFactory.instance().client().inspectContainerCmd(id).exec().getNetworkSettings().getNetworks();
return (networks != null && networks.containsKey(DEFAULT_BRIDGE));
}
catch(Exception e)
{
return (false);
}
}
/***************************************************************************
** The id of the container this JVM runs in: from the mount table when it names
** the container directory, otherwise the host name (the container id unless
** overridden); null when neither exists.
***************************************************************************/
static String ownContainerId(String mountInfo, String hostName)
{
Matcher matcher = MOUNTINFO_CONTAINER_ID.matcher(mountInfo == null ? "" : mountInfo);
if(matcher.find())
{
return (matcher.group(1));
}
return (hostName == null || hostName.isBlank() ? null : hostName.trim());
}
}
@@ -0,0 +1,114 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
/*******************************************************************************
** The topology decision of the sibling-container fixtures.
*******************************************************************************/
class FixtureEndpointTest
{
private static final String CONTAINER_ID = "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef";
/***************************************************************************
** Build container on the default bridge: dial the sibling bridge IP and its own port, never the published port.
***************************************************************************/
@Test
void buildContainerOnTheBridgeUsesTheBridgeAddress()
{
assertEquals(new FixtureEndpoint.Address("172.17.0.9", 5432), FixtureEndpoint.choose(true, "172.17.0.9", 5432, "localhost", 41111));
assertEquals(new FixtureEndpoint.Address("172.17.0.9", 5432), FixtureEndpoint.choose(true, "172.17.0.9", 5432, null, null));
}
/***************************************************************************
** Every other topology (JVM on the Docker host, a separate or remote daemon such as the native runner's isolated
** test daemon, job network, host networking, unidentified build container) dials the published endpoint
** at the host Testcontainers reports (including TESTCONTAINERS_HOST_OVERRIDE), never a private bridge address.
***************************************************************************/
@Test
void everyOtherTopologyDialsThePublishedEndpoint()
{
assertEquals(new FixtureEndpoint.Address("localhost", 41111), FixtureEndpoint.choose(false, "172.17.0.9", 5432, "localhost", 41111));
assertEquals(new FixtureEndpoint.Address("172.17.0.1", 32768), FixtureEndpoint.choose(false, "172.17.0.3", 6379, "172.17.0.1", 32768));
assertEquals(new FixtureEndpoint.Address("host.docker.internal", 40000), FixtureEndpoint.choose(false, null, 5432, "host.docker.internal", 40000));
}
/***************************************************************************
** An unknown endpoint is an error, never a guessed address.
***************************************************************************/
@Test
void missingEndpointFailsClosed()
{
assertThrows(IllegalStateException.class, () -> FixtureEndpoint.choose(true, null, 5432, "localhost", 41111));
assertThrows(IllegalStateException.class, () -> FixtureEndpoint.choose(true, " ", 5432, "localhost", 41111));
assertThrows(IllegalStateException.class, () -> FixtureEndpoint.choose(false, "172.17.0.9", 5432, "localhost", null));
assertThrows(IllegalStateException.class, () -> FixtureEndpoint.choose(false, "172.17.0.9", 5432, " ", 41111));
}
/***************************************************************************
** The historical loopback-only publication stays wherever a loopback dial can work; when Testcontainers dials
** another host (the native runner's separate daemon) the daemon's default binding is used so the dial can succeed.
***************************************************************************/
@Test
void loopbackPublicationOnlyWhereALoopbackDialCanWork()
{
assertTrue(FixtureEndpoint.loopbackPublication(false, "localhost"));
assertTrue(FixtureEndpoint.loopbackPublication(false, "127.0.0.1"));
assertTrue(FixtureEndpoint.loopbackPublication(false, "::1"));
assertTrue(FixtureEndpoint.loopbackPublication(false, null));
assertTrue(FixtureEndpoint.loopbackPublication(true, "172.17.0.1"));
assertFalse(FixtureEndpoint.loopbackPublication(false, "172.17.0.1"));
assertFalse(FixtureEndpoint.loopbackPublication(false, "host.docker.internal"));
assertFalse(FixtureEndpoint.loopbackPublication(false, "docker.example.test"));
}
/***************************************************************************
** The build container id comes from the mount table when present, else the host name.
***************************************************************************/
@Test
void ownContainerIdPrefersTheMountTableThenTheHostName()
{
String mountInfo = "700 699 0:50 /docker/containers/" + CONTAINER_ID + "/hostname /etc/hostname rw,relatime - ext4 /dev/sda1 rw";
assertEquals(CONTAINER_ID, FixtureEndpoint.ownContainerId(mountInfo, "unrelated"));
assertEquals("abc123", FixtureEndpoint.ownContainerId("no container mounts here", " abc123 "));
assertNull(FixtureEndpoint.ownContainerId("", null));
assertNull(FixtureEndpoint.ownContainerId(null, " "));
}
}
@@ -0,0 +1,231 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration;
import java.nio.file.Files;
import java.nio.file.Path;
import java.sql.Connection;
import java.util.Arrays;
import java.util.Locale;
import java.sql.DriverManager;
import java.util.ArrayList;
import java.util.List;
import com.kingsrook.qbits.collaboration.mywork.MyWorkService;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.module.postgres.model.metadata.PostgreSQLBackendMetaData;
import com.kingsrook.qqq.backend.module.rdbms.jdbc.ConnectionManager;
import com.kingsrook.qqq.backend.module.rdbms.jdbc.C3P0PooledConnectionProvider;
import com.kingsrook.qqq.backend.module.rdbms.model.metadata.ConnectionPoolSettings;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.testcontainers.containers.GenericContainer;
import org.testcontainers.containers.wait.strategy.Wait;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** Real PostgreSQL query counts for the bounded, caller-authorized My Work scan.
*******************************************************************************/
class MyWorkPostgresTest extends PersonalServiceFixture
{
private static GenericContainer<?> postgres;
/*******************************************************************************
** Collect SQL counts in an owned disposable database, never an application DB.
*******************************************************************************/
@BeforeAll
static void startPostgres()
{
postgres = new GenericContainer<>("postgres:17-alpine")
.withEnv("POSTGRES_PASSWORD", "mywork-fixture-only").withExposedPorts(5432)
.withCommand("postgres", "-c", "shared_preload_libraries=pg_stat_statements")
.waitingFor(Wait.forLogMessage(".*database system is ready to accept connections.*", 2));
postgres.start();
}
/*******************************************************************************
** Stop only the database owned by this class.
*******************************************************************************/
@AfterAll
static void stopPostgres()
{
if(postgres != null)
{
postgres.stop();
}
}
/*******************************************************************************
** Reuse the real authorization fixture with PostgreSQL persistence.
*******************************************************************************/
@Override
@BeforeEach
public void setUp() throws Exception
{
super.setUp();
instance.getBackends().remove("memory");
instance.addBackend(new PostgreSQLBackendMetaData().withName("memory").withHostName(postgres.getHost())
.withPort(postgres.getMappedPort(5432)).withDatabaseName("postgres")
.withUsername("postgres").withPassword("mywork-fixture-only"));
var backend = (PostgreSQLBackendMetaData) instance.getBackend("memory");
backend.setConnectionProvider(new QCodeReference(C3P0PooledConnectionProvider.class));
backend.setConnectionPoolSettings(new ConnectionPoolSettings().withInitialPoolSize(1).withMinPoolSize(1)
.withMaxPoolSize(3).withAcquireIncrement(1).withCheckoutTimeoutSeconds(2));
try(Connection connection = connect(); var sql = connection.createStatement())
{
sql.execute("CREATE EXTENSION pg_stat_statements");
sql.execute(CollaborationSchema.postgresScript(config));
sql.execute("CREATE TABLE document (id INTEGER PRIMARY KEY, title TEXT, tenant TEXT)");
sql.execute("INSERT INTO document VALUES (1, 'Visible', 'a'), (3, 'Private', 'b')");
sql.execute("INSERT INTO collaboration_task (id, table_name, record_id, title, assignee_user_id, created_by_user_id, status, parent_deleted, due_at) "
+ "SELECT n, 'document', CASE WHEN n=601 THEN '3' ELSE '1' END, 'Task ' || n, 'alice', 'alice', 'OPEN', false, "
+ "CASE WHEN n%2=0 THEN TIMESTAMPTZ '2026-10-22 00:00:00Z' + (601-n)*INTERVAL '1 second' ELSE NULL END FROM generate_series(1,601) n");
sql.execute("SELECT pg_stat_statements_reset()");
}
}
/*******************************************************************************
** Release owner pools before container teardown.
*******************************************************************************/
@Override
public void tearDown()
{
ConnectionManager.closeConnectionProviders(instance);
super.tearDown();
}
/*******************************************************************************
** Exact counts/order and bounded SQL parent/page queries on the real backend.
*******************************************************************************/
@Test
void summaryPreservesVisibilityWithBoundedSqlQueries() throws Exception
{
var summary = new MyWorkService().summary();
assertThat(summary.openTaskCount()).isEqualTo(600);
assertThat(summary.tasks()).extracting(row -> row.getValueInteger("id"))
.containsExactly(600, 598, 596, 594, 592, 590, 588, 586, 584, 582);
try(Connection connection = connect(); var sql = connection.createStatement();
var rows = sql.executeQuery("SELECT query, calls FROM pg_stat_statements WHERE query ILIKE 'SELECT%'") )
{
long parents = 0;
long tasks = 0;
while(rows.next())
{
String query = rows.getString(1);
if(query.contains("\"document\""))
{
parents += rows.getLong(2);
}
if(query.contains("\"collaboration_task\""))
{
tasks += rows.getLong(2);
}
}
assertThat(parents).as("SQL parent queries").isBetween(1L, 8L);
assertThat(tasks).as("SQL task page queries").isEqualTo(5);
}
QContext.getQSession().getSecurityKeyValues().put("tenant", new ArrayList<>(List.of("b")));
assertThat(new MyWorkService().summary().openTaskCount()).isEqualTo(1);
if(Boolean.getBoolean("qqq.mywork.load"))
{
QContext.getQSession().getSecurityKeyValues().put("tenant", new ArrayList<>(List.of("a")));
measureLoad();
}
}
/*******************************************************************************
** Opt-in reproducible SQL work measurement, not a deployment capacity claim.
*******************************************************************************/
private void measureLoad() throws Exception
{
final int tasks = 2000;
final int samples = 10;
List<String> report = new ArrayList<>();
report.add("tasks,parents,samples,p50_ms,p95_ms,p99_ms,sql_calls,sql_execution_ms");
for(int parents : List.of(10, tasks))
{
try(Connection connection = connect(); var sql = connection.createStatement())
{
sql.execute("TRUNCATE collaboration_task, document");
sql.execute("INSERT INTO document SELECT n, 'Document ' || n, CASE WHEN n%10=0 THEN 'b' ELSE 'a' END FROM generate_series(1," + parents + ") n");
sql.execute("INSERT INTO collaboration_task (id, table_name, record_id, title, assignee_user_id, created_by_user_id, status, parent_deleted) "
+ "SELECT n, 'document', (((n-1)%" + parents + ")+1)::text, 'Task ' || n, 'alice', 'alice', 'OPEN', false FROM generate_series(1," + tasks + ") n");
}
// Warm the JVM and pool without counting these calls in the samples.
for(int warmup = 0; warmup < 2; warmup++)
{
assertThat(new MyWorkService().summary().openTaskCount()).isEqualTo(1800);
}
try(Connection connection = connect(); var sql = connection.createStatement())
{
sql.execute("SELECT pg_stat_statements_reset()");
}
double[] elapsed = new double[samples];
for(int sample = 0; sample < samples; sample++)
{
long start = System.nanoTime();
var summary = new MyWorkService().summary();
elapsed[sample] = (System.nanoTime() - start) / 1_000_000.0;
assertThat(summary.openTaskCount()).isEqualTo(1800);
assertThat(summary.tasks()).extracting(row -> row.getValueInteger("id"))
.containsExactly(1, 2, 3, 4, 5, 6, 7, 8, 9, 11);
}
var backend = (PostgreSQLBackendMetaData) instance.getBackend("memory");
var pool = ConnectionManager.getPoolMetrics(instance, backend);
// C3P0 returns connections asynchronously; allow a bounded check-in interval.
long deadline = System.nanoTime() + 2_000_000_000L;
while(pool.get("connectionsBusy").intValue() != 0 && System.nanoTime() < deadline)
{
Thread.sleep(10);
pool = ConnectionManager.getPoolMetrics(instance, backend);
}
assertThat(pool.get("connectionsTotal").intValue()).isBetween(1, 3);
assertThat(pool.get("connectionsBusy").intValue()).isZero();
assertThat(pool.get("checkoutTimeouts").longValue()).isZero();
Arrays.sort(elapsed);
try(Connection connection = connect(); var sql = connection.createStatement();
var rows = sql.executeQuery("SELECT COALESCE(sum(calls),0), COALESCE(sum(total_exec_time),0) FROM pg_stat_statements "
+ "WHERE query ILIKE 'SELECT%' AND (query LIKE '%\"document\"%' OR query LIKE '%\"collaboration_task\"%' "
+ "OR query LIKE '%\"collaboration_record_assignment\"%' OR query LIKE '%\"collaboration_mention\"%')"))
{
assertThat(rows.next()).isTrue();
report.add(String.format(Locale.ROOT, "%d,%d,%d,%.3f,%.3f,%.3f,%d,%.3f", tasks, parents, samples,
elapsed[4], elapsed[9], elapsed[9], rows.getLong(1), rows.getDouble(2)));
}
}
Path output = Path.of("target/mywork-load.csv");
Files.createDirectories(output.getParent());
Files.write(output, report);
System.out.println("My Work local PostgreSQL workload: " + output.toAbsolutePath());
report.forEach(System.out::println);
}
/*******************************************************************************
** A fixture-owned connection for setup and independent SQL observations.
*******************************************************************************/
private Connection connect() throws Exception
{
return (DriverManager.getConnection("jdbc:postgresql://" + postgres.getHost() + ":" + postgres.getMappedPort(5432) + "/postgres", "postgres", "mywork-fixture-only"));
}
}
@@ -0,0 +1,132 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration;
import java.time.Instant;
import java.util.ArrayList;
import java.util.List;
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
import com.kingsrook.qbits.collaboration.mywork.MyWorkService;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryRecordStore;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** Measure backend queries while preserving exact, caller-visible summary counts.
*******************************************************************************/
class MyWorkQueryBudgetTest extends PersonalServiceFixture
{
/*******************************************************************************
** Counts cross page boundaries without including another user's or hidden work.
*******************************************************************************/
@Test
void summaryPagesAssignmentsAndMentionsWithoutLeakingCounts() throws Exception
{
List<QRecord> parents = new ArrayList<>();
List<QRecord> assignments = new ArrayList<>();
List<QRecord> mentions = new ArrayList<>();
for(int id = 1; id <= 208; id++)
{
if(id > 3)
{
parents.add(new QRecord().withValue("id", id).withValue("title", "Parent " + id).withValue("tenant", id == 208 ? "b" : "a"));
}
assignments.add(new QRecord().withValue("id", id).withValue("tableName", "document").withValue("recordId", Integer.toString(id))
.withValue("assigneeUserId", id == 207 ? "bob" : "alice").withValue("parentDeleted", false));
}
for(int id = 1; id <= 601; id++)
{
mentions.add(new QRecord().withValue("id", id).withValue("tableName", "document").withValue("recordId", Integer.toString(id % 3 + 1))
.withValue("mentionedUserId", "alice").withValue("notifiedAt", Instant.now()).withValue("parentDeleted", false));
}
mentions.add(new QRecord().withValue("id", 602).withValue("tableName", "document").withValue("recordId", "1")
.withValue("mentionedUserId", "bob").withValue("notifiedAt", Instant.now()).withValue("parentDeleted", false));
mentions.add(new QRecord().withValue("id", 603).withValue("tableName", "document").withValue("recordId", "1")
.withValue("mentionedUserId", "alice").withValue("parentDeleted", false));
MemoryRecordStore.getInstance().insert(new InsertInput("document").withRecords(parents), true);
MemoryRecordStore.getInstance().insert(new InsertInput(CollaborationTables.RECORD_ASSIGNMENT).withRecords(assignments), true);
MemoryRecordStore.getInstance().insert(new InsertInput(CollaborationTables.MENTION).withRecords(mentions), true);
MemoryRecordStore.resetStatistics();
MemoryRecordStore.setCollectStatistics(true);
try
{
var summary = new MyWorkService().summary();
assertThat(summary.assignedRecordsByTable()).containsOnlyKeys("document").containsEntry("document", 205L);
assertThat(summary.unreadMentionCount()).isEqualTo(401);
assertThat(summary.openTaskCount()).isZero();
assertThat(summary.tasks()).isEmpty();
MemoryRecordStore.getActionInputs().get(QueryInput.class).stream().map(input -> (QueryInput) input)
.filter(input -> input.getTableName().startsWith("collaboration_"))
.forEach(input -> assertThat(input.getFilter().getLimit()).isNotNull().isLessThanOrEqualTo(200));
}
finally
{
MemoryRecordStore.setCollectStatistics(false);
MemoryRecordStore.resetStatistics();
}
}
/*******************************************************************************
** Hundreds of tasks on one parent must not cause hundreds of parent lookups.
*******************************************************************************/
@Test
void summaryBoundsPagesAndRepeatedParentLookups() throws Exception
{
List<QRecord> rows = new ArrayList<>();
for(int id = 1; id <= 601; id++)
{
rows.add(new QRecord().withValue("id", id).withValue("tableName", "document")
.withValue("recordId", id == 601 ? "3" : "1").withValue("title", "Task " + id)
.withValue("assigneeUserId", "alice").withValue("createdByUserId", "alice")
.withValue("status", "OPEN").withValue("parentDeleted", false)
.withValue("dueAt", id % 2 == 0 ? Instant.parse("2026-10-22T00:00:00Z").plusSeconds(601 - id) : null));
}
// Populate historical rows directly so current parent authorization is tested on read.
MemoryRecordStore.getInstance().insert(new InsertInput(CollaborationTables.TASK).withRecords(rows), true);
MemoryRecordStore.resetStatistics();
MemoryRecordStore.setCollectStatistics(true);
try
{
var summary = new MyWorkService().summary();
assertThat(summary.openTaskCount()).isEqualTo(600);
assertThat(summary.tasks()).extracting(row -> row.getValueInteger("id"))
.containsExactly(600, 598, 596, 594, 592, 590, 588, 586, 584, 582);
List<QueryInput> queries = MemoryRecordStore.getActionInputs().get(QueryInput.class).stream()
.map(input -> (QueryInput) input).toList();
assertThat(queries.stream().filter(input -> "document".equals(input.getTableName())).count()).isLessThanOrEqualTo(8);
assertThat(queries.stream().filter(input -> CollaborationTables.TASK.equals(input.getTableName())))
.allSatisfy(input -> assertThat(input.getFilter().getLimit()).isNotNull().isLessThanOrEqualTo(200));
// A second request must recheck current permissions rather than reuse cached decisions.
QContext.getQSession().getSecurityKeyValues().put("tenant", new ArrayList<>(List.of("b")));
assertThat(new MyWorkService().summary().openTaskCount()).isEqualTo(1);
}
finally
{
MemoryRecordStore.setCollectStatistics(false);
MemoryRecordStore.resetStatistics();
}
}
}
@@ -0,0 +1,144 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration;
import java.net.URL;
import java.net.URLClassLoader;
import java.nio.file.Path;
import java.util.Arrays;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;
/*******************************************************************************
** Exercise a consumer classpath without the optional Redis implementation/driver.
*******************************************************************************/
class OptionalRedisConsumerTest
{
/***************************************************************************
** An isolated loader must not resolve Redis from the test runner's parent.
***************************************************************************/
@Test
void memoryPresenceWithoutRedis() throws Exception
{
String classpath = System.getProperty("surefire.test.class.path", System.getProperty("java.class.path"));
URL[] urls = Arrays.stream(classpath.split(java.io.File.pathSeparator))
.filter(entry -> !entry.replace('\\', '/').contains("/qbit-redis/")
&& !entry.replace('\\', '/').contains("/qbit-email/")
&& !Path.of(entry).getFileName().toString().startsWith("jedis-"))
.map(entry ->
{
try
{
return (Path.of(entry).toUri().toURL());
}
catch(Exception e)
{
throw (new IllegalArgumentException(e));
}
}).toArray(URL[]::new);
ClassLoader previous = Thread.currentThread().getContextClassLoader();
try(var loader = new URLClassLoader(urls, ClassLoader.getPlatformClassLoader()))
{
assertThrows(ClassNotFoundException.class, () -> loader.loadClass("com.kingsrook.qbits.redis.api.RedisNamespace"));
assertThrows(ClassNotFoundException.class, () -> loader.loadClass("redis.clients.jedis.RedisClient"));
assertThrows(ClassNotFoundException.class, () -> loader.loadClass("com.kingsrook.qbits.email.api.EmailApi"));
Thread.currentThread().setContextClassLoader(loader);
loader.loadClass(Probe.class.getName()).getMethod("exercise").invoke(null);
}
finally
{
Thread.currentThread().setContextClassLoader(previous);
}
}
/***************************************************************************
** Loaded in the isolated consumer, including metadata enrichment and delivery.
***************************************************************************/
public static class Probe extends PersonalServiceFixture
{
/************************************************************************
** Real host setup and authorized presence work without Redis artifacts.
************************************************************************/
public static void exercise() throws Exception
{
var probe = new Probe();
try
{
probe.setUp();
var service = new com.kingsrook.qbits.collaboration.presence.PresenceService();
String lease = service.openLease("document", "1");
assertTrue(service.snapshot("document", "1").isEmpty());
assertEquals(1, service.renewLease(lease, "document", "1", "VIEWING", null, 1L).size());
service.leave(lease, "document", "1", 2L);
assertTrue(service.snapshot("document", "1").isEmpty());
assertFalse(new com.kingsrook.qbits.collaboration.presence.RedisPresenceAudience().allows("untrusted"));
probe.config.withUserSessionProvider(new com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference(RecipientSessions.class));
assertEquals(java.util.Optional.of(true), new com.kingsrook.qbits.collaboration.security.CollaborationRecordAccessChecker()
.canRead("alice", "document", "1"));
var notice = new com.kingsrook.qbits.collaboration.notify.CollaborationNotification("alice", "Test", "Test",
com.kingsrook.qbits.collaboration.notify.CollaborationNotification.Category.MENTION, "document", "1", "/app/document/1");
var receipts = new java.util.concurrent.atomic.AtomicInteger();
com.kingsrook.qbits.collaboration.notify.Notifications.send(notice, () -> receipts.incrementAndGet());
assertEquals(0, receipts.get(), "Missing email must not acknowledge delivery");
probe.config.withNotifier(new com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference(AcceptingNotifier.class));
com.kingsrook.qbits.collaboration.notify.Notifications.send(notice, () -> receipts.incrementAndGet());
assertEquals(1, receipts.get(), "Explicit host notifier must still work without email");
}
finally
{
probe.tearDown();
}
}
}
/***************************************************************************
** Host session lookup makes the notification eligible for actual delivery.
***************************************************************************/
public static class RecipientSessions implements com.kingsrook.qqq.backend.core.actions.collaboration.UserSessionProviderInterface
{
/************************************************************************
** Fixture recipient has access to the parent record in tenant a.
************************************************************************/
@Override
public java.util.Optional<com.kingsrook.qqq.backend.core.model.session.QSession> sessionForUser(String userId)
{
return (java.util.Optional.of(new com.kingsrook.qqq.backend.core.model.session.QSession()
.withUser(new com.kingsrook.qqq.backend.core.model.session.QUser().withIdReference(userId))
.withPermissions("document.read").withSecurityKeyValue("tenant", "a")));
}
}
/***************************************************************************
** A host transport that accepts delivery without the optional email adapter.
***************************************************************************/
public static class AcceptingNotifier implements com.kingsrook.qbits.collaboration.notify.CollaborationNotifier
{
/************************************************************************
** Test transport has no external delivery side effects.
************************************************************************/
@Override
public void notify(com.kingsrook.qbits.collaboration.notify.CollaborationNotification notification)
{
assertEquals("alice", notification.recipientUserId());
}
}
}
Loaded 100 of 118 files, more files were not shown because too many files have changed in this diff. Show more