Compare commits
10
Commits
v1.2.0-RC.5
...
develop
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
3c865a5c97
|
||
|
|
c28f839658
|
||
|
|
551c4c6f87
|
||
|
|
fbce2e5d6e
|
||
|
|
70f6184a7a
|
||
|
|
16bd4f768f
|
||
|
|
04d04adc6e
|
||
|
|
de922d7e86
|
||
|
|
c6b077321c
|
||
|
|
2837f4152a
|
No files matched your search
@@ -29,6 +29,12 @@ All notable changes are documented here, following [Keep a Changelog](https://ke
|
||||
|
||||
### Fixed
|
||||
|
||||
- Record search no longer fails for a person who may read a table but may not query the tag or custom field value tables. The Tags field (and each searchable custom field kept in the value table) is a search field of its table, and its search-term expansion is a sub-query on `collaboration_record_tag` and `collaboration_tag` (or `collaboration_field_value`). With the default rules every reader may query those tables, but a host can require a permission of its own on them, and core refuses a whole search for one refused sub-query table: every search of such a table answered `403`, whatever the text. `TagCriteriaExpander.expandSearchTerm` and `FieldCriteriaExpander.expandSearchTerm` now ask core whether the session may query those tables (the table READ check core applies to a sub-query) and contribute nothing when it may not, so the table's other search fields still match, and `#tag` matches nothing. Nobody gains access: an explicit Tags or custom field criterion on a record list is still refused for such a person, and a person who may query the tables searches as before.
|
||||
- The record features summary (`GET /table/{tableName}/{primaryKey}/features`) no longer fails when shared presence is recovering or unreachable. `CollaborationSummaryProvider` let the presence backend's service-unavailable answer escape, so the whole summary answered `503` (counts, favorite, follow, owner and assignee with it) for one lease lifetime after the first touch of each table's room in a store that had lost its state, and for as long as the store was unreachable. The summary now answers with every other part intact, leaves `presence` unset and reports `extensions.presence.status = "unavailable"`; an empty or absent `presence` without that extension still means a room that was read. The Who's here card says presence is temporarily unavailable instead of failing, and one visit now starts the recovery interval of every table's room (it stopped at the first recovering room, so the card needed one visit per table, a lease lifetime apart, before it could show counts). The presence endpoints (`/presence`, `/presence/leases`, `/presence/leases/renew`) are unchanged and keep answering `503`.
|
||||
- Favorites and recents accept a table whose custom permission checker tells a read from a write by the type of the action input. `TargetResolver` asked whether a table is readable with an untyped `AbstractActionInput`; such a checker refuses an input it does not recognise, so `POST /recents` (which the UI sends on every list and record screen) and starring answered `404 Not found` for people the checker lets read the table. The read is now asked as the query it stands for, as `ParentRecordAccess` already asks with a get. Who may read a table is unchanged.
|
||||
- Reads of a table that has no tag assignment no longer query `collaboration_record_tag`. Core selects the Tags field for every query or get that names no fields, so each of them (including per-record gets and the stored-record read before an update) ran one assignment statement whether or not the table had ever been tagged: 600 gets of an untagged table ran 600 statements, now none. Each node remembers which tables have no assignment (`TaggedTables`, core `Memoization` registered with `ClusterCaches`) for `CollaborationQBitConfig.withTagLookupCacheSeconds` (default 30, 0 disables); writing an assignment clears it on the node and through the ClusterBus. Tables with assignments are read as before, one lookup per 500 parents. No API or schema change. See "What the Tags field costs a read" in the README.
|
||||
- Browsing one of the qbit's child tables (comments, tasks, record tags, attachments and the other parent-referencing tables) reads each distinct parent once per result page to authorize its rows, instead of once per row: 61 tasks on two parents ran 61 parent reads, now 2. The check itself and its outcome are unchanged.
|
||||
- The retained upload journal's PostgreSQL 17 qualification accepts Amazon Aurora PostgreSQL, which reports `full_page_writes=off` and does not let a deployment change it. `off` is accepted only when the settings query detects Aurora (`to_regprocedure('aurora_version()') IS NOT NULL`); stock PostgreSQL with `full_page_writes=off` is still refused, and every other requirement is unchanged on Aurora. See `docs/RETAINED-UPLOAD-JOURNAL.md`.
|
||||
- Bounded STRING custom fields carry QQQ's native `ValueTooLongBehavior.ERROR`, so a write longer than `maxLength` is rejected with a field error and the stored value is preserved, instead of passing through unchecked (the QQQ default `PASS_THROUGH`).
|
||||
- Persisted help content is reloaded after each committed runtime metadata refresh, so sections and fields created by publishing a custom-field definition receive their stored `helpContent` rows without a restart.
|
||||
- QBit metadata reports the Maven artifact version from a filtered resource, including release versions.
|
||||
|
||||
@@ -409,7 +409,16 @@ size, pending suggestions, and the five most-used approved tags.
|
||||
|
||||
`collaborationTags` is a virtual STRING field labeled Tags with possible-value source `collaboration_tag`. `TagCriteriaExpander` supports IN, NOT_IN, EQUALS, NOT_EQUALS, IS_BLANK, IS_NOT_BLANK and CONTAINS_ALL using core `QSubQuery` membership filters. Ordinary quick search matches labels; `#name` matches normalized names. Numeric criteria are tag IDs; string criteria are names. The old ID materializer is removed. Core owns SQL execution or bounded cross-backend materialization; its bound applies to inner rows, not final parent matches.
|
||||
|
||||
The runtime provider adds this field and its search entry where TAGS is enabled. It is queryable but not selectable: the overlay has no virtual-field post-query hook and existing host customizers must be preserved.
|
||||
The runtime provider adds this field and its search entry where TAGS is enabled. It is queryable and selectable (`TagValueProvider` fills it after host customizers have run).
|
||||
|
||||
### What the Tags field costs a read
|
||||
|
||||
Core selects every selectable virtual field for a query or get that names no fields, so each such read of a table with TAGS enabled runs `TagValueProvider`, including the reads other code makes one record at a time (a get, the stored-record read before an update, a lookup made per row). The provider therefore costs:
|
||||
|
||||
- nothing for a table that has no tag assignment. Each node remembers that one fact per table (`TaggedTables`) for `withTagLookupCacheSeconds` (default 30; 0 looks every time) and learns it with one indexed `LIMIT 1` statement;
|
||||
- one assignment statement and one vocabulary statement per 500 parents for a table that has assignments. Record tags themselves are never cached.
|
||||
|
||||
A node that writes an assignment forgets the table's answer at once and again after commit, and publishes core's `memo-invalidate` cluster event (`ClusterCaches`, name `collaboration.taggedTables`). Hosts with a ClusterBus transport see a table's first tag on every node immediately; without one, another node shows it after at most the configured seconds. The same bound covers assignments written outside QQQ. The answer does not depend on the session (the probe ignores row locks and returns no data), so it cannot change which records or tags a caller sees. A failed probe is thrown, not remembered.
|
||||
|
||||
## UI API
|
||||
|
||||
|
||||
@@ -5,4 +5,5 @@
|
||||
No schema change and nothing required of hosts.
|
||||
|
||||
- New table action **Create rows from configuration** on Feature Settings. Run it once if you want the effective feature settings visible and editable as rows. It changes no behavior and does nothing once the table has any row. Note that after rows exist they win over later changes to `withDefaultFeatures` or `withTableFeatures` in code, so change those settings in the rows from then on.
|
||||
- Reads of tables without any tag assignment skip the assignment lookup (`withTagLookupCacheSeconds`, default 30 seconds, 0 restores a lookup on every read). A host running several nodes without a ClusterBus transport shows a table's very first tag on the other nodes after at most that long; later tags on that table show at once, as before.
|
||||
- New **Setup** widget first on the Collaboration home, and table help text. Hosts that override help on a table keep their own.
|
||||
@@ -28,6 +28,40 @@ later empty snapshots can be known-empty. The guard has no TTL; tenant-retiremen
|
||||
operations and long-term guard storage must be included in operational capacity
|
||||
qualification.
|
||||
|
||||
## Scope of the unavailable answer
|
||||
|
||||
Recovery and store failures are answered as service-unavailable only by the
|
||||
presence endpoints (`/presence`, `/presence/leases`, `/presence/leases/renew`).
|
||||
Responses that merely include presence do not fail with it. The core record
|
||||
features summary is built as usual, leaves `presence` unset and carries
|
||||
`extensions.presence.status = "unavailable"`; without that extension an empty
|
||||
or absent `presence` means a room that was read. The Who's here card shows a
|
||||
sentence instead of counts, never partial or node-local counts. The card reads
|
||||
every table's room; a recovering room does not stop that reading, so one visit
|
||||
starts the interval of every room, while any other failure stops it at once.
|
||||
|
||||
## When the interval starts
|
||||
|
||||
The interval starts on the first touch of a table's room when its guard is
|
||||
missing, or when the table projections are empty before the guard's latest
|
||||
issued expiry. Normal use cannot produce the second case: every write that
|
||||
advances that expiry (reservation, renewal, legacy heartbeat or departure)
|
||||
also leaves a table member expiring at exactly that instant (a departure
|
||||
leaves a marker, it does not remove the member), and the key lifetimes are set
|
||||
in the same script. A person who opens a table and leaves, a
|
||||
reservation that is never activated and ordinary idle expiry all end in a
|
||||
known-empty room under the same generation
|
||||
(`RedisPresenceRecoveryTest.normalUseNeverRestartsRecovery`).
|
||||
|
||||
The first touch of a table that has no guard still starts an interval, even in
|
||||
a store that has never held presence. An empty store that never had viewers and
|
||||
a store that just lost its viewers are the same bytes: anything that could tell
|
||||
them apart lives in the same keyspace and is lost with it, and a marker shared
|
||||
by all tables would sit in another hash slot than the table's room, outside
|
||||
what one script may read on Redis Cluster. Treating a missing guard as
|
||||
known-empty would report a viewer who is still on a page as absent after a
|
||||
loss, which is what the guard exists to prevent.
|
||||
|
||||
## Controlled restore
|
||||
|
||||
The host configuration has a bounded `recoveryGeneration`, default `initial`,
|
||||
|
||||
@@ -25,7 +25,11 @@ Both owner tables must use the same QQQ backend. Hosts apply DDL through their m
|
||||
| --- | --- | --- |
|
||||
| File H2 | Native `h2`, `jdbc:h2:file:`, actual configured connection, writable real transaction, live `WRITE_DELAY=0` | `UploadJournalTable.h2Script()` |
|
||||
| MySQL 8.4 | Native `mysql`, actual MySQL 8.4 product/version, exact configured URL target and catalog, writable real transaction, both owner tables InnoDB, fixed owner table mappings, journal `utf8mb4_0900_bin` columns/table, single-column unique locator/intake indexes, synchronous commit settings | `UploadJournalTable.mysqlScript()` |
|
||||
| Aurora MySQL, MariaDB, PostgreSQL, other versions/providers | Refused; no inferred compatibility or durability claim | None |
|
||||
| PostgreSQL 17 | Native `postgres` (vendor and backend type), actual PostgreSQL 17 product/version, exact configured `jdbc:postgresql://host/catalog` target and catalog, writable real transaction, not in recovery, `synchronous_commit` `on` or `remote_apply`, `fsync=on`, `full_page_writes=on` (see the Aurora rule below), session `TimeZone` UTC, `lock_timeout` from 1 to 5000 ms, READ COMMITTED, both owner tables permanent ordinary tables with fixed mappings, no non-deterministic collation, complete unique indexes on `locator_hash` and `intake_key` | `UploadJournalTable.postgresScript()` |
|
||||
| Amazon Aurora PostgreSQL 17 | Every PostgreSQL 17 requirement above, except that `full_page_writes=off` is accepted | `UploadJournalTable.postgresScript()` |
|
||||
| Aurora MySQL, MariaDB, other versions/providers | Refused; no inferred compatibility or durability claim | None |
|
||||
|
||||
Aurora rule: stock PostgreSQL must report `full_page_writes=on`, because recovery repairs a partly written page from the full page image in the write-ahead log. Amazon Aurora PostgreSQL reports `full_page_writes=off` and the parameter is not in its cluster parameter group, so a deployment cannot turn it on; its storage layer does not write torn pages, so the protection is not needed there. The qualification therefore accepts `off` only when the same settings query reports `to_regprocedure('aurora_version()') IS NOT NULL`, which is true on Aurora, false on stock PostgreSQL 17 and never raises. Nothing else is relaxed on Aurora: `synchronous_commit`, `fsync`, recovery state, isolation, time zone and lock timeout are checked exactly as on stock PostgreSQL. The detection is a name lookup, so a function named `aurora_version()` on the session's search path of a stock server is reported as Aurora; creating one is a schema change inside the host's trust boundary, like the settings themselves.
|
||||
|
||||
The MySQL adapter resolves explicit or native-generated URLs through QQQ `ConnectionManager`. It accepts a single ordinary `jdbc:mysql://host:port/catalog` target with a simple catalog identifier. Proxy/load-balancing URLs, alternate catalogs or table remapping fail closed. Error messages do not echo URLs or credentials. Driver options remain host-owned; TLS and credentials are not changed by the journal.
|
||||
|
||||
|
||||
@@ -51,6 +51,7 @@ public class CollaborationQBitConfig implements QBitConfig
|
||||
private Function<String, String> recipientEmailResolver = null;
|
||||
private QCodeReference userSessionProvider = null;
|
||||
private String defaultTagMode = "FREE";
|
||||
private Integer tagLookupCacheSeconds = 30;
|
||||
private List<String> tagActionTables = List.of();
|
||||
private List<String> followActionTables = List.of();
|
||||
private List<String> followChangeTables = List.of();
|
||||
@@ -325,6 +326,7 @@ public class CollaborationQBitConfig implements QBitConfig
|
||||
assertCondition(instance.getBackend(backendName) != null, "Unknown Collaboration backend: " + backendName, errors);
|
||||
}
|
||||
assertCondition(List.of("FREE", "APPROVED_ONLY").contains(defaultTagMode == null ? "" : defaultTagMode), "defaultTagMode must be FREE or APPROVED_ONLY", errors);
|
||||
assertCondition(tagLookupCacheSeconds != null && tagLookupCacheSeconds >= 0, "tagLookupCacheSeconds must be 0 (off) or more", errors);
|
||||
for(List<String> tables : List.of(followActionTables, followChangeTables, tagActionTables, assignActionTables))
|
||||
{
|
||||
for(String table : tables)
|
||||
@@ -550,6 +552,27 @@ public class CollaborationQBitConfig implements QBitConfig
|
||||
return (this);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** How long a node remembers whether a table has any tag assignment.
|
||||
*******************************************************************************/
|
||||
public Integer getTagLookupCacheSeconds()
|
||||
{
|
||||
return (tagLookupCacheSeconds);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Reads of a table without any tag assignment skip the assignment lookup while
|
||||
** this node remembers that (default 30 seconds; 0 looks every time). A node
|
||||
** that writes an assignment forgets at once and tells the others over the
|
||||
** ClusterBus; without a cluster transport another node shows a table's first
|
||||
** tag after at most this long. Tables that have assignments are always read.
|
||||
*******************************************************************************/
|
||||
public CollaborationQBitConfig withTagLookupCacheSeconds(Integer tagLookupCacheSeconds)
|
||||
{
|
||||
this.tagLookupCacheSeconds = tagLookupCacheSeconds;
|
||||
return (this);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Host tables receiving Add tags and Remove tags actions.
|
||||
*******************************************************************************/
|
||||
|
||||
@@ -33,7 +33,8 @@ import com.kingsrook.qqq.backend.core.realtime.RealtimeBus;
|
||||
import com.kingsrook.qqq.backend.core.realtime.RealtimeMessage;
|
||||
|
||||
/*******************************************************************************
|
||||
** Identifier-only hints, published after successful commit.
|
||||
** Identifier-only hints, published after successful commit. A tag assignment
|
||||
** also clears what this node remembers about untagged tables (TaggedTables).
|
||||
*******************************************************************************/
|
||||
public class CollaborationRealtime implements RecordChangeListenerInterface
|
||||
{
|
||||
@@ -55,6 +56,7 @@ public class CollaborationRealtime implements RecordChangeListenerInterface
|
||||
public void onRecordsChanged(RecordChangeEvent change) throws QException
|
||||
{
|
||||
var instance = QContext.getQInstance();
|
||||
java.util.Set<String> newlyTaggedTables = new java.util.HashSet<>();
|
||||
for(int index = 0; index < change.getRecords().size(); index++)
|
||||
{
|
||||
var row = new com.kingsrook.qqq.backend.core.model.data.QRecord(change.getRecords().get(index));
|
||||
@@ -70,6 +72,13 @@ public class CollaborationRealtime implements RecordChangeListenerInterface
|
||||
{
|
||||
continue;
|
||||
}
|
||||
if(CollaborationTables.RECORD_TAG.equals(change.getTableName()) && change.getType() != RecordChangeType.DELETE && newlyTaggedTables.add(table))
|
||||
{
|
||||
////////////////////////////////////////////////////////////////////////
|
||||
// the Tags field skips tables known to have no assignment: forget it //
|
||||
////////////////////////////////////////////////////////////////////////
|
||||
com.kingsrook.qbits.collaboration.tags.TaggedTables.assignmentsChanged(table, change.getTransaction());
|
||||
}
|
||||
String event = switch(change.getTableName())
|
||||
{
|
||||
case CollaborationTables.COMMENT -> change.getType() == RecordChangeType.DELETE || row.getValue("deletedAt") != null
|
||||
|
||||
@@ -21,23 +21,35 @@
|
||||
|
||||
package com.kingsrook.qbits.collaboration;
|
||||
|
||||
import java.io.Serializable;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.Map;
|
||||
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
|
||||
import com.kingsrook.qbits.collaboration.presence.PresenceService;
|
||||
import com.kingsrook.qqq.backend.core.actions.collaboration.RecordFeatureSummaryProviderInterface;
|
||||
import com.kingsrook.qqq.backend.core.actions.collaboration.RecordFeaturesResolver;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QServiceUnavailableException;
|
||||
import com.kingsrook.qqq.backend.core.logging.QLogger;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordFeature;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordFeatureSummary;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordReference;
|
||||
import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
|
||||
|
||||
/*******************************************************************************
|
||||
** Caller-scoped values for the core record features endpoint.
|
||||
*******************************************************************************/
|
||||
public class CollaborationSummaryProvider implements RecordFeatureSummaryProviderInterface
|
||||
{
|
||||
public static final String PRESENCE_EXTENSION = "presence";
|
||||
public static final String PRESENCE_UNAVAILABLE = "unavailable";
|
||||
|
||||
private static final QLogger LOG = QLogger.getLogger(CollaborationSummaryProvider.class);
|
||||
|
||||
/*******************************************************************************
|
||||
** Core has already checked the parent; omit values for disabled features.
|
||||
*******************************************************************************/
|
||||
@@ -49,15 +61,7 @@ public class CollaborationSummaryProvider implements RecordFeatureSummaryProvide
|
||||
String user = PersonalRecords.currentUserId();
|
||||
if(features.isEnabled(RecordFeature.PRESENCE))
|
||||
{
|
||||
summary.setPresence(new com.kingsrook.qbits.collaboration.presence.PresenceService().snapshot(ref.tableName(), id).stream().map(viewer ->
|
||||
{
|
||||
java.util.Map<String, java.io.Serializable> entry = new java.util.LinkedHashMap<>();
|
||||
entry.put("userId", viewer.userId());
|
||||
entry.put("displayName", viewer.displayName());
|
||||
entry.put("mode", viewer.mode());
|
||||
entry.put("fieldName", viewer.fieldName());
|
||||
return (entry);
|
||||
}).toList());
|
||||
contributePresence(ref.tableName(), id, summary);
|
||||
}
|
||||
if(features.isEnabled(RecordFeature.FAVORITES))
|
||||
{
|
||||
@@ -110,4 +114,32 @@ public class CollaborationSummaryProvider implements RecordFeatureSummaryProvide
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Presence is advisory: a recovering or unreachable shared store must not fail
|
||||
** the whole summary. The list stays unset, because an empty list means a room
|
||||
** that was read and is empty, and the provider extension says why. Presence's
|
||||
** own endpoints keep answering service-unavailable.
|
||||
*******************************************************************************/
|
||||
private static void contributePresence(String tableName, String id, RecordFeatureSummary summary) throws QException
|
||||
{
|
||||
try
|
||||
{
|
||||
summary.setPresence(new PresenceService().snapshot(tableName, id).stream().map(viewer ->
|
||||
{
|
||||
Map<String, Serializable> entry = new LinkedHashMap<>();
|
||||
entry.put("userId", viewer.userId());
|
||||
entry.put("displayName", viewer.displayName());
|
||||
entry.put("mode", viewer.mode());
|
||||
entry.put("fieldName", viewer.fieldName());
|
||||
return (entry);
|
||||
}).toList());
|
||||
}
|
||||
catch(QServiceUnavailableException unavailable)
|
||||
{
|
||||
summary.setPresence(null);
|
||||
summary.getExtensions().put(PRESENCE_EXTENSION, new LinkedHashMap<>(Map.of("status", PRESENCE_UNAVAILABLE)));
|
||||
LOG.debug("Presence is unavailable; the record features summary reports it as unavailable", logPair("tableName", tableName));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -288,12 +288,16 @@ public final class TargetResolver
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Guard existence before invoking the core table permission helper.
|
||||
** Guard existence before invoking the core table permission helper. The read
|
||||
** is asked as the query it stands for: core hands a table's custom permission
|
||||
** checker the action input and not the permission asked, so a checker that
|
||||
** tells a read from a write by the input's type refuses an untyped input, and
|
||||
** its table was then no target for the people it lets read it.
|
||||
*******************************************************************************/
|
||||
private static boolean canReadTable(String table)
|
||||
{
|
||||
return (table != null && QContext.getQInstance().getTable(table) != null
|
||||
&& PermissionsHelper.hasTablePermission(new AbstractActionInput(), table, TablePermissionSubType.READ));
|
||||
&& PermissionsHelper.hasTablePermission(new QueryInput(table), table, TablePermissionSubType.READ));
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -87,6 +87,7 @@ public final class CollaborationHome
|
||||
public static final String NO_TAGS = "No tags yet. Tags appear here once someone creates one on a record.";
|
||||
public static final String NO_FIELDS = "No custom fields yet. Define one under Custom fields.";
|
||||
public static final String PRESENCE_ADMIN_ONLY = "Who's here is visible to collaboration administrators.";
|
||||
public static final String PRESENCE_UNAVAILABLE = "Who's here is temporarily unavailable. Try again shortly.";
|
||||
|
||||
|
||||
|
||||
@@ -277,16 +278,9 @@ public final class CollaborationHome
|
||||
else
|
||||
{
|
||||
counts = new java.util.TreeMap<>();
|
||||
var backend = new com.kingsrook.qbits.collaboration.presence.RedisPresenceBackend(redis);
|
||||
// Read only known table rooms in the administrator's authenticated tenant.
|
||||
// Never scan Redis keys or silently substitute local counts on failure.
|
||||
for(String tableName : instance.getTables().keySet())
|
||||
if(!readSharedCounts(instance, redis, counts))
|
||||
{
|
||||
int viewers = backend.snapshot(tableName, null).size();
|
||||
if(viewers > 0)
|
||||
{
|
||||
counts.put(tableName, viewers);
|
||||
}
|
||||
return (table("Who's here", PRESENCE_UNAVAILABLE, List.of(), column("Table", "table", "left"), column("People", "viewers", "right")));
|
||||
}
|
||||
}
|
||||
counts.forEach((table, viewers) ->
|
||||
@@ -302,6 +296,49 @@ public final class CollaborationHome
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Read only known table rooms in the administrator's authenticated tenant;
|
||||
** never scan Redis keys or substitute node-local counts. False means the
|
||||
** counts are incomplete and must not be shown. A recovering room does not
|
||||
** stop the reading, so every room's interval starts on the same visit instead
|
||||
** of one room per visit; any other failure stops it at once.
|
||||
*******************************************************************************/
|
||||
private static boolean readSharedCounts(com.kingsrook.qqq.backend.core.model.metadata.QInstance instance,
|
||||
com.kingsrook.qbits.collaboration.presence.RedisPresenceConfig redis, Map<String, Integer> counts)
|
||||
{
|
||||
boolean complete = true;
|
||||
try
|
||||
{
|
||||
var backend = new com.kingsrook.qbits.collaboration.presence.RedisPresenceBackend(redis);
|
||||
for(String tableName : instance.getTables().keySet())
|
||||
{
|
||||
try
|
||||
{
|
||||
int viewers = backend.snapshot(tableName, null).size();
|
||||
if(viewers > 0)
|
||||
{
|
||||
counts.put(tableName, viewers);
|
||||
}
|
||||
}
|
||||
catch(com.kingsrook.qqq.backend.core.exceptions.QServiceUnavailableException unavailable)
|
||||
{
|
||||
if(!com.kingsrook.qbits.collaboration.presence.RedisPresenceBackend.isRecovering(unavailable))
|
||||
{
|
||||
throw (unavailable);
|
||||
}
|
||||
complete = false;
|
||||
}
|
||||
}
|
||||
}
|
||||
catch(com.kingsrook.qqq.backend.core.exceptions.QServiceUnavailableException unavailable)
|
||||
{
|
||||
complete = false;
|
||||
}
|
||||
return (complete);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Assemble a table widget whose empty state is a sentence, not a blank grid.
|
||||
*******************************************************************************/
|
||||
|
||||
+2
-1
@@ -55,6 +55,7 @@ public class AttachmentPreviewCustomizer implements TableCustomizerInterface
|
||||
List<QRecord> filter(List<QRecord> records, boolean isGet, boolean generateUrls, AttachmentsService service) throws QException
|
||||
{
|
||||
List<QRecord> visible = new ArrayList<>();
|
||||
ParentRecordAccess.ResultSetParents parents = new ParentRecordAccess.ResultSetParents();
|
||||
for(QRecord record : records)
|
||||
{
|
||||
try
|
||||
@@ -63,7 +64,7 @@ public class AttachmentPreviewCustomizer implements TableCustomizerInterface
|
||||
{
|
||||
throw (new QNotFoundException("Not found"));
|
||||
}
|
||||
ParentRecordAccess.requireVisibleParent(record.getValueString("tableName"), record.getValueString("recordId"));
|
||||
parents.requireVisible(record.getValueString("tableName"), record.getValueString("recordId"));
|
||||
}
|
||||
catch(QNotFoundException expected)
|
||||
{
|
||||
|
||||
+37
-3
@@ -47,14 +47,25 @@ import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSTableBackendDe
|
||||
** pair, on the native QQQ postgres backend (vendor and backend type "postgres").
|
||||
** Observation only: no schema migration and no session or server setting is
|
||||
** changed at runtime.
|
||||
**
|
||||
** Amazon Aurora PostgreSQL 17 is the one qualified variant: it is held to every
|
||||
** rule here except full_page_writes, which Aurora reports as "off" and does not
|
||||
** let a deployment change (see tornPagesPrevented).
|
||||
*******************************************************************************/
|
||||
final class PostgresJournalDatabase
|
||||
{
|
||||
static final String VENDOR = "postgres";
|
||||
|
||||
private static final String SETTINGS = "SELECT current_setting('server_version_num')::int, current_setting('transaction_read_only'), pg_is_in_recovery(),"
|
||||
/////////////////////////////////////////////////////////////////////////////////////////////
|
||||
// Column 11 detects Amazon Aurora PostgreSQL inside the same statement. to_regprocedure //
|
||||
// returns NULL when no aurora_version() function resolves, so it is true on Aurora, false //
|
||||
// on stock PostgreSQL 17 and never raises. Calling aurora_version() directly would abort //
|
||||
// the transaction on stock PostgreSQL, and pg_proc does not list the function on Aurora. //
|
||||
/////////////////////////////////////////////////////////////////////////////////////////////
|
||||
static final String SETTINGS = "SELECT current_setting('server_version_num')::int, current_setting('transaction_read_only'), pg_is_in_recovery(),"
|
||||
+ " current_setting('synchronous_commit'), current_setting('fsync'), current_setting('full_page_writes'), current_setting('TimeZone'),"
|
||||
+ " (SELECT setting::bigint FROM pg_settings WHERE name='lock_timeout'), current_setting('transaction_isolation'), current_database()";
|
||||
+ " (SELECT setting::bigint FROM pg_settings WHERE name='lock_timeout'), current_setting('transaction_isolation'), current_database(),"
|
||||
+ " to_regprocedure('aurora_version()') IS NOT NULL";
|
||||
|
||||
private static final String TABLES = "SELECT c.relname, c.relkind, c.relpersistence FROM pg_class c JOIN pg_namespace n ON n.oid=c.relnamespace"
|
||||
+ " WHERE n.nspname=current_schema() AND c.relname = ANY(?)";
|
||||
@@ -124,7 +135,8 @@ final class PostgresJournalDatabase
|
||||
try(ResultSet settings = statement.executeQuery(SETTINGS))
|
||||
{
|
||||
if(!settings.next() || settings.getInt(1) / 10000 != 17 || !"off".equals(settings.getString(2)) || settings.getBoolean(3)
|
||||
|| !Set.of("on", "remote_apply").contains(settings.getString(4)) || !"on".equals(settings.getString(5)) || !"on".equals(settings.getString(6))
|
||||
|| !Set.of("on", "remote_apply").contains(settings.getString(4)) || !"on".equals(settings.getString(5))
|
||||
|| !tornPagesPrevented(settings.getString(6), settings.getBoolean(11))
|
||||
|| !Set.of("UTC", "Etc/UTC").contains(settings.getString(7)) || settings.getLong(8) < 1 || settings.getLong(8) > 5000
|
||||
|| !"read committed".equals(settings.getString(9)) || !catalog.equals(settings.getString(10)))
|
||||
{
|
||||
@@ -147,6 +159,28 @@ final class PostgresJournalDatabase
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The torn-page rule, as a pure decision over the observed full_page_writes
|
||||
** value and the Aurora detection column.
|
||||
**
|
||||
** Stock PostgreSQL must report full_page_writes "on": after a crash that leaves
|
||||
** a partly written page, recovery repairs it from the full page image in the
|
||||
** write-ahead log, so "off" there can lose committed journal rows.
|
||||
**
|
||||
** Amazon Aurora PostgreSQL reports full_page_writes "off" (measured on engine
|
||||
** 17.9: source default, and the parameter is not in the cluster parameter group,
|
||||
** so a deployment cannot turn it on). Its storage layer does not write torn
|
||||
** pages, so the protection is not needed there. "off" is therefore accepted
|
||||
** only when the server is Aurora; every other rule (synchronous_commit, fsync,
|
||||
** recovery, isolation, time zone, lock timeout) stays as strict on Aurora.
|
||||
*******************************************************************************/
|
||||
static boolean tornPagesPrevented(String fullPageWrites, boolean aurora)
|
||||
{
|
||||
return ("on".equals(fullPageWrites) || (aurora && "off".equals(fullPageWrites)));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Permanent (logged) ordinary tables only, deterministic text comparison, and
|
||||
** complete valid non-partial unique indexes for the journal's two identities.
|
||||
|
||||
+7
@@ -87,10 +87,17 @@ public class FieldCriteriaExpander implements VirtualFieldCriteriaExpanderInterf
|
||||
|
||||
/*******************************************************************************
|
||||
** Core's search callback has no field argument, so combine searchable EAV text.
|
||||
** A session that may not query the value table gets no custom field matches:
|
||||
** core refuses a whole search for one refused sub-query table. An explicit
|
||||
** custom field criterion is still refused by core.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public QQueryFilter expandSearchTerm(QTableMetaData table, String term, com.kingsrook.qqq.backend.core.actions.tables.query.SearchTermKind kind, CriteriaExpansionContext context) throws QException
|
||||
{
|
||||
if(!com.kingsrook.qbits.collaboration.security.ParentRecordAccess.sessionMayQuery(CollaborationTables.FIELD_VALUE))
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
QQueryFilter result = new QQueryFilter().withBooleanOperator(QQueryFilter.BooleanOperator.OR);
|
||||
for(var definition : CollaborationRecords.query(CollaborationTables.FIELD_DEFINITION, new QQueryFilter()
|
||||
.withCriteria("tableName", QCriteriaOperator.EQUALS, table.getName()).withCriteria("status", QCriteriaOperator.EQUALS, "ACTIVE")
|
||||
|
||||
@@ -32,6 +32,8 @@ import com.kingsrook.qqq.backend.core.exceptions.QServiceUnavailableException;
|
||||
*******************************************************************************/
|
||||
public class RedisPresenceBackend implements PresenceBackend
|
||||
{
|
||||
private static final String RECOVERING = "Presence is recovering; active viewers are rejoining. Please retry.";
|
||||
|
||||
private final RedisPresenceStore store;
|
||||
private final String audience;
|
||||
|
||||
@@ -147,7 +149,7 @@ public class RedisPresenceBackend implements PresenceBackend
|
||||
}
|
||||
catch(PresenceRecoveringException recovering)
|
||||
{
|
||||
throw (new QServiceUnavailableException("Presence is recovering; active viewers are rejoining. Please retry."));
|
||||
throw (new QServiceUnavailableException(RECOVERING));
|
||||
}
|
||||
catch(RuntimeException failure)
|
||||
{
|
||||
@@ -155,6 +157,14 @@ public class RedisPresenceBackend implements PresenceBackend
|
||||
}
|
||||
}
|
||||
|
||||
/***************************************************************************
|
||||
** Recovery ends by itself after one lease lifetime; other failures do not.
|
||||
***************************************************************************/
|
||||
public static boolean isRecovering(QServiceUnavailableException failure)
|
||||
{
|
||||
return (RECOVERING.equals(failure.getMessage()));
|
||||
}
|
||||
|
||||
/***************************************************************************
|
||||
** Never expose client exceptions, endpoint details or credentials to browsers.
|
||||
***************************************************************************/
|
||||
|
||||
@@ -23,6 +23,7 @@ package com.kingsrook.qbits.collaboration.security;
|
||||
|
||||
import java.io.Serializable;
|
||||
import com.kingsrook.qbits.collaboration.CollaborationQBitConfig;
|
||||
import com.kingsrook.qqq.backend.core.actions.permissions.PermissionEnforcement;
|
||||
import com.kingsrook.qqq.backend.core.actions.permissions.PermissionsHelper;
|
||||
import com.kingsrook.qqq.backend.core.actions.permissions.TablePermissionSubType;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
@@ -30,6 +31,7 @@ import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QNotFoundException;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
|
||||
|
||||
@@ -55,6 +57,27 @@ public final class ParentRecordAccess
|
||||
&& PermissionsHelper.hasTablePermission(new GetInput(tableName), tableName, TablePermissionSubType.READ));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Whether core would let the current session query every one of these tables.
|
||||
** This is the table READ check, asked with an enforced query input, that core
|
||||
** applies to the tables of a sub-query. A host may protect this qbit's tables
|
||||
** more strictly than their defaults; a filter that reaches them through a
|
||||
** sub-query is then refused as a whole.
|
||||
*******************************************************************************/
|
||||
public static boolean sessionMayQuery(String... tableNames)
|
||||
{
|
||||
for(String tableName : tableNames)
|
||||
{
|
||||
QueryInput probe = new QueryInput(tableName);
|
||||
probe.setPermissionEnforcement(PermissionEnforcement.ENFORCE);
|
||||
if(!PermissionsHelper.hasTablePermission(probe, tableName, TablePermissionSubType.READ))
|
||||
{
|
||||
return (false);
|
||||
}
|
||||
}
|
||||
return (true);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Get as the caller, enforcing table permissions and the parent's read locks.
|
||||
*******************************************************************************/
|
||||
@@ -81,6 +104,42 @@ public final class ParentRecordAccess
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** One result set's parent checks: rows that share a parent cost one lookup.
|
||||
** The session and transaction do not change within the call that creates it,
|
||||
** so never keep an instance beyond that call.
|
||||
*******************************************************************************/
|
||||
public static final class ResultSetParents
|
||||
{
|
||||
private final java.util.Map<java.util.List<String>, Boolean> visible = new java.util.HashMap<>();
|
||||
|
||||
/*******************************************************************************
|
||||
** Same outcome as ParentRecordAccess.requireVisibleParent for every row.
|
||||
*******************************************************************************/
|
||||
public void requireVisible(String tableName, String recordId) throws QException
|
||||
{
|
||||
java.util.List<String> parent = java.util.Arrays.asList(tableName, recordId);
|
||||
Boolean known = visible.get(parent);
|
||||
if(known == null)
|
||||
{
|
||||
try
|
||||
{
|
||||
requireVisibleParent(tableName, recordId);
|
||||
known = true;
|
||||
}
|
||||
catch(QNotFoundException expected)
|
||||
{
|
||||
known = false;
|
||||
}
|
||||
visible.put(parent, known);
|
||||
}
|
||||
if(!known)
|
||||
{
|
||||
throw (new QNotFoundException("Not found"));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Core record capability checks include parent WRITE locks for implied grants.
|
||||
*******************************************************************************/
|
||||
|
||||
@@ -43,6 +43,7 @@ public class ParentRowCustomizer implements TableCustomizerInterface
|
||||
public List<QRecord> postQuery(QueryOrGetInputInterface input, List<QRecord> records) throws QException
|
||||
{
|
||||
List<QRecord> visible = new ArrayList<>();
|
||||
ParentRecordAccess.ResultSetParents parents = new ParentRecordAccess.ResultSetParents();
|
||||
for(QRecord record : records)
|
||||
{
|
||||
try
|
||||
@@ -51,7 +52,7 @@ public class ParentRowCustomizer implements TableCustomizerInterface
|
||||
{
|
||||
throw (new QNotFoundException("Not found"));
|
||||
}
|
||||
ParentRecordAccess.requireVisibleParent(record.getValueString("tableName"), record.getValueString("recordId"));
|
||||
parents.requireVisible(record.getValueString("tableName"), record.getValueString("recordId"));
|
||||
QRecord row = new QRecord(record);
|
||||
if(row.getValue("deletedAt") != null)
|
||||
{
|
||||
|
||||
@@ -25,6 +25,7 @@ import java.io.Serializable;
|
||||
import java.util.List;
|
||||
import com.kingsrook.qbits.collaboration.CollaborationRecords;
|
||||
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
|
||||
import com.kingsrook.qbits.collaboration.security.ParentRecordAccess;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.query.CriteriaExpansionContext;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.query.SearchTermKind;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.query.VirtualFieldCriteriaExpanderInterface;
|
||||
@@ -136,10 +137,17 @@ public class TagCriteriaExpander implements VirtualFieldCriteriaExpanderInterfac
|
||||
|
||||
/*******************************************************************************
|
||||
** Ordinary search matches labels; hash search matches an exact normalized name.
|
||||
** A session that may not query the tag tables gets no tag matches: core refuses
|
||||
** a whole search for one refused sub-query table, and a text search must not
|
||||
** fail because of tags. An explicit Tags criterion is still refused by core.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public QQueryFilter expandSearchTerm(QTableMetaData table, String term, SearchTermKind kind, CriteriaExpansionContext context) throws QException
|
||||
{
|
||||
if(!ParentRecordAccess.sessionMayQuery(CollaborationTables.RECORD_TAG, CollaborationTables.TAG))
|
||||
{
|
||||
return (null);
|
||||
}
|
||||
QQueryFilter tags = TagsService.vocabularyFilter(table.getName()).withCriteria("approved", QCriteriaOperator.EQUALS, true)
|
||||
.withCriteria(kind == SearchTermKind.TAG ? "name" : "label", kind == SearchTermKind.TAG ? QCriteriaOperator.EQUALS : QCriteriaOperator.CONTAINS,
|
||||
kind == SearchTermKind.TAG ? TagsService.normalize(term) : term);
|
||||
|
||||
@@ -45,16 +45,26 @@ public class TagValueProvider implements VirtualFieldValueProviderInterface
|
||||
{
|
||||
/*******************************************************************************
|
||||
** Batch both assignments and vocabulary; never recursively load parent records.
|
||||
** A table without any assignment costs no statement: core selects this field
|
||||
** for every read that names no fields, so untagged tables must stay free.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void fill(QTableMetaData table, String fieldName, List<QRecord> records, QueryInput origin) throws QException
|
||||
{
|
||||
if(!TaggedTables.mayHaveAssignments(table.getName(), origin.getTransaction()))
|
||||
{
|
||||
for(QRecord record : records)
|
||||
{
|
||||
record.setValue(fieldName, null);
|
||||
}
|
||||
return;
|
||||
}
|
||||
QTableMetaData metadata = QContext.getQInstance().getTable(CollaborationTables.RECORD_TAG).clone();
|
||||
metadata.withCustomizers(null);
|
||||
metadata.getSupplementalMetaData().remove(RecordReferenceTableMetaData.NAME);
|
||||
for(List<QRecord> page : CollectionUtils.getPages(records, 500))
|
||||
{
|
||||
List<String> ids = page.stream().map(record -> String.valueOf(record.getValue(table.getPrimaryKeyField()))).distinct().toList();
|
||||
QTableMetaData metadata = QContext.getQInstance().getTable(CollaborationTables.RECORD_TAG).clone();
|
||||
metadata.withCustomizers(null);
|
||||
metadata.getSupplementalMetaData().remove(RecordReferenceTableMetaData.NAME);
|
||||
QueryInput input = new QueryInput(CollaborationTables.RECORD_TAG).withTransaction(origin.getTransaction()).withFilter(new QQueryFilter()
|
||||
.withCriteria("tableName", QCriteriaOperator.EQUALS, table.getName()).withCriteria("recordId", QCriteriaOperator.IN, ids)
|
||||
.withCriteria("parentDeleted", QCriteriaOperator.EQUALS, false));
|
||||
|
||||
@@ -0,0 +1,184 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
|
||||
package com.kingsrook.qbits.collaboration.tags;
|
||||
|
||||
import java.time.Duration;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import java.util.WeakHashMap;
|
||||
import com.kingsrook.qbits.collaboration.CollaborationRecords;
|
||||
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
|
||||
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
|
||||
import com.kingsrook.qqq.backend.core.cluster.ClusterBus;
|
||||
import com.kingsrook.qqq.backend.core.cluster.ClusterCaches;
|
||||
import com.kingsrook.qqq.backend.core.cluster.ClusterEvent;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
|
||||
import com.kingsrook.qqq.backend.core.utils.memoization.Memoization;
|
||||
|
||||
/*******************************************************************************
|
||||
** Which host tables have any tag assignment at all. Every table with TAGS
|
||||
** enabled carries the Tags field, so every read of such a table would otherwise
|
||||
** look for assignments, including the tables nobody ever tagged.
|
||||
**
|
||||
** Only that one fact per table is remembered, never a record's tags: a table
|
||||
** known to have assignments is always read from the database. The answer is the
|
||||
** same for every session (the probe ignores row locks and returns no data), so
|
||||
** it cannot widen or narrow what a caller sees.
|
||||
**
|
||||
** Each node keeps its own answers. An assignment written here clears the
|
||||
** table's answer on this node at once and again after commit, and on the other
|
||||
** nodes through the ClusterBus (memo-invalidate). The timeout bounds everything
|
||||
** that event cannot reach: hosts without a cluster transport, writes made
|
||||
** outside QQQ, and a probe that raced an uncommitted assignment.
|
||||
*******************************************************************************/
|
||||
public final class TaggedTables
|
||||
{
|
||||
public static final String CACHE_NAME = "collaboration.taggedTables";
|
||||
|
||||
private static final int MAX_TABLES = 10_000;
|
||||
|
||||
private static final Map<QInstance, Memoization<String, Boolean>> CACHES = ClusterCaches.shared(CACHE_NAME,
|
||||
new WeakHashMap<QInstance, Memoization<String, Boolean>>(), TaggedTables::clearAll, TaggedTables::clearTable);
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Static lookups only.
|
||||
*******************************************************************************/
|
||||
private TaggedTables()
|
||||
{
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** False only when the table is known to have no assignment. A failed probe is
|
||||
** thrown, never remembered. With the cache disabled the answer is always true.
|
||||
*******************************************************************************/
|
||||
public static boolean mayHaveAssignments(String tableName, QBackendTransaction transaction) throws QException
|
||||
{
|
||||
Integer seconds = CollaborationRecords.config().getTagLookupCacheSeconds();
|
||||
if(seconds == null || seconds <= 0)
|
||||
{
|
||||
return (true);
|
||||
}
|
||||
return (cache(QContext.getQInstance(), seconds).getResultThrowing(tableName, name -> hasAssignments(name, transaction)).orElse(true));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** An assignment was written for this parent table. Forget the answer now, so
|
||||
** the writing transaction reads its own assignment, and again after commit, so
|
||||
** a probe that ran in between and saw nothing cannot outlive the commit.
|
||||
*******************************************************************************/
|
||||
public static void assignmentsChanged(String tableName, QBackendTransaction transaction)
|
||||
{
|
||||
QInstance instance = QContext.getQInstance();
|
||||
forget(instance, tableName);
|
||||
if(transaction != null)
|
||||
{
|
||||
transaction.addAfterCommitCallback(() -> forget(instance, tableName));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Clear on this node and tell the others; a transport failure is logged by the
|
||||
** bus and the timeout covers the nodes that did not hear.
|
||||
*******************************************************************************/
|
||||
private static void forget(QInstance instance, String tableName)
|
||||
{
|
||||
ClusterBus.publish(instance, new ClusterEvent(ClusterEvent.TYPE_MEMO_INVALIDATE, CACHE_NAME, tableName));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** One indexed row at most, in the caller's transaction, as the same answer for
|
||||
** every session: no row locks, no parent check and no customizers.
|
||||
*******************************************************************************/
|
||||
private static Boolean hasAssignments(String tableName, QBackendTransaction transaction) throws QException
|
||||
{
|
||||
QueryInput input = new QueryInput(CollaborationTables.RECORD_TAG).withTransaction(transaction).withFieldNamesToInclude(Set.of("id"))
|
||||
.withFilter(new QQueryFilter().withCriteria("tableName", QCriteriaOperator.EQUALS, tableName).withLimit(1));
|
||||
input.setTableMetaData(CollaborationRecords.cleanupMetadata(CollaborationTables.RECORD_TAG));
|
||||
input.withPermissionBypass("Tag provider asks only whether a table has any assignment.");
|
||||
return (!new QueryAction().execute(input).getRecords().isEmpty());
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The instance's answers, following the configured timeout.
|
||||
*******************************************************************************/
|
||||
private static Memoization<String, Boolean> cache(QInstance instance, int seconds)
|
||||
{
|
||||
synchronized(CACHES)
|
||||
{
|
||||
Memoization<String, Boolean> cache = CACHES.computeIfAbsent(instance, key -> new Memoization<String, Boolean>(MAX_TABLES).withMayStoreNullValues(false));
|
||||
cache.setTimeout(Duration.ofSeconds(seconds));
|
||||
return (cache);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Every instance in this JVM: an extra probe is cheaper than a stale answer.
|
||||
*******************************************************************************/
|
||||
private static List<Memoization<String, Boolean>> caches()
|
||||
{
|
||||
synchronized(CACHES)
|
||||
{
|
||||
return (List.copyOf(CACHES.values()));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** ClusterCaches: forget every table.
|
||||
*******************************************************************************/
|
||||
private static void clearAll()
|
||||
{
|
||||
caches().forEach(Memoization::clear);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** ClusterCaches: forget one table.
|
||||
*******************************************************************************/
|
||||
private static void clearTable(String tableName)
|
||||
{
|
||||
caches().forEach(cache -> cache.clearKey(tableName));
|
||||
}
|
||||
}
|
||||
@@ -138,6 +138,13 @@ class CollaborationApiHttpTest extends PersonalServiceFixture
|
||||
@Test
|
||||
void unavailablePresenceHttp() throws Exception
|
||||
{
|
||||
String tab = "{\"connectionId\":\"tab\",\"table\":\"document\",\"pk\":\"1\",\"mode\":\"VIEWING\"}";
|
||||
ok("POST", "/presence", tab);
|
||||
JsonNode local = json(nativeGet("/table/document/1/features", "alice"), 200);
|
||||
assertThat(local.path("presence")).hasSize(1);
|
||||
assertThat(local.at("/presence/0/userId").asText()).isEqualTo("alice");
|
||||
assertThat(local.path("extensions").has(CollaborationSummaryProvider.PRESENCE_EXTENSION)).isFalse();
|
||||
ok("DELETE", "/presence", "{\"connectionId\":\"tab\",\"table\":\"document\",\"pk\":\"1\"}");
|
||||
config.withRedisPresence(new com.kingsrook.qbits.collaboration.presence.RedisPresenceConfig()
|
||||
.withConnectionName("not-started").withDeployment("test").withApplication("fixture")
|
||||
.withScopeProvider(new QCodeReference(RedisPresenceServiceTest.TenantScope.class)));
|
||||
@@ -147,6 +154,75 @@ class CollaborationApiHttpTest extends PersonalServiceFixture
|
||||
assertError(400, "POST", "/presence", "{\"connectionId\":\"tab\",\"table\":\"document\",\"pk\":\"1\",\"mode\":\"VIEWING\"}", "alice");
|
||||
assertError(503, "DELETE", "/presence", "{\"connectionId\":\"tab\",\"table\":\"document\",\"pk\":\"1\"}", "alice");
|
||||
assertError(404, "GET", "/presence?table=document&pk=3", null, "alice");
|
||||
assertFeaturesReportPresenceUnavailable(0, 0);
|
||||
assertThat(nativeGet("/table/document/3/features", "alice").statusCode()).isEqualTo(404);
|
||||
}
|
||||
|
||||
/***************************************************************************
|
||||
** Presence trouble never fails the core record features summary: the other
|
||||
** parts are served and presence is reported unavailable, while presence's
|
||||
** own endpoint keeps answering service-unavailable.
|
||||
***************************************************************************/
|
||||
@Test
|
||||
void recordFeaturesSurvivePresenceRecoveryAndFailure() throws Exception
|
||||
{
|
||||
ok("POST", "/records/document/1/comments", "{\"body\":\"First\"}");
|
||||
ok("POST", "/records/document/1/tasks", "{\"title\":\"Review\"}");
|
||||
try(var redis = RedisFixtures.redis())
|
||||
{
|
||||
redis.start();
|
||||
new com.kingsrook.qbits.redis.RedisQBitProducer().withQBitConfig(new com.kingsrook.qbits.redis.RedisQBitConfig()
|
||||
.withConnection("presence", new com.kingsrook.qbits.redis.RedisConnectionConfig()
|
||||
.withHost(RedisFixtures.host(redis)).withPort(RedisFixtures.port(redis)).withTls(false)
|
||||
.withConnectionTimeoutMillis(200).withSocketTimeoutMillis(1000))).produce(instance);
|
||||
config.withRedisPresence(new com.kingsrook.qbits.collaboration.presence.RedisPresenceConfig()
|
||||
.withConnectionName("presence").withDeployment("test").withApplication("features-summary")
|
||||
.withScopeProvider(new QCodeReference(RedisPresenceServiceTest.TenantScope.class)));
|
||||
var runtime = new com.kingsrook.qbits.redis.RedisRuntimeService();
|
||||
try
|
||||
{
|
||||
runtime.start(instance);
|
||||
// The first touch of a room in a store with no state is recovering.
|
||||
assertFeaturesReportPresenceUnavailable(1, 1);
|
||||
assertThat(json(request("GET", "/presence?table=document&pk=1", null, "alice"), 503).path("message").asText()).contains("recovering");
|
||||
admin();
|
||||
var recovering = com.kingsrook.qbits.collaboration.app.CollaborationHome.presence();
|
||||
assertThat(recovering.getRows()).isEmpty();
|
||||
assertThat(recovering.getNoRowsFoundHTML()).isEqualTo(com.kingsrook.qbits.collaboration.app.CollaborationHome.PRESENCE_UNAVAILABLE);
|
||||
// One visit starts the interval of every table's room, not one room per visit.
|
||||
assertThat(com.kingsrook.qbits.redis.RedisRuntimeService.connection(instance, "presence").client().keys("*:recovery"))
|
||||
.hasSize(instance.getTables().size());
|
||||
redis.stop();
|
||||
assertFeaturesReportPresenceUnavailable(1, 1);
|
||||
assertThat(json(request("GET", "/presence?table=document&pk=1", null, "alice"), 503).path("message").asText()).contains("temporarily unavailable");
|
||||
}
|
||||
finally
|
||||
{
|
||||
runtime.stop();
|
||||
runtime.afterApplicationStop();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/***************************************************************************
|
||||
** The summary answers 200 with every other part intact; presence is unset
|
||||
** and the provider extension says it is unavailable.
|
||||
***************************************************************************/
|
||||
private void assertFeaturesReportPresenceUnavailable(int openComments, int openTasks) throws Exception
|
||||
{
|
||||
JsonNode summary = json(nativeGet("/table/document/1/features", "alice"), 200);
|
||||
assertThat(summary.hasNonNull("presence")).isFalse();
|
||||
assertThat(summary.at("/extensions/" + CollaborationSummaryProvider.PRESENCE_EXTENSION + "/status").asText()).isEqualTo(CollaborationSummaryProvider.PRESENCE_UNAVAILABLE);
|
||||
assertThat(summary.path("tableName").asText()).isEqualTo("document");
|
||||
assertThat(summary.path("primaryKey").asText()).isEqualTo("1");
|
||||
assertThat(summary.path("features").toString()).contains("PRESENCE").contains("COMMENTS");
|
||||
assertThat(summary.at("/counts/openComments").asInt(-1)).isEqualTo(openComments);
|
||||
assertThat(summary.at("/counts/openTasks").asInt(-1)).isEqualTo(openTasks);
|
||||
assertThat(summary.at("/counts/attachments").asInt(-1)).isZero();
|
||||
assertThat(summary.at("/counts/tags").asInt(-1)).isZero();
|
||||
assertThat(summary.path("isFavorite").isBoolean()).isTrue();
|
||||
assertThat(summary.path("isFollowing").isBoolean()).isTrue();
|
||||
assertThat(summary.path("capabilities").size()).isPositive();
|
||||
}
|
||||
|
||||
private static final ObjectMapper JSON = new ObjectMapper();
|
||||
|
||||
@@ -22,27 +22,46 @@ package com.kingsrook.qbits.collaboration;
|
||||
|
||||
import java.sql.Connection;
|
||||
import java.sql.DriverManager;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Set;
|
||||
import com.kingsrook.qbits.collaboration.customfields.CustomFieldValueProvider;
|
||||
import com.kingsrook.qbits.collaboration.customfields.FieldCriteriaExpander;
|
||||
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
|
||||
import com.kingsrook.qbits.collaboration.permissions.CollaborationPermissions;
|
||||
import com.kingsrook.qbits.collaboration.tags.TagCriteriaExpander;
|
||||
import com.kingsrook.qbits.collaboration.tags.TagValueProvider;
|
||||
import com.kingsrook.qbits.collaboration.tags.TagsService;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.RecordSearchAction;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.QInputSource;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.search.RecordSearchInput;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.fields.QVirtualFieldMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.permissions.PermissionLevel;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRules;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
|
||||
import com.kingsrook.qqq.backend.module.postgres.model.metadata.PostgreSQLBackendMetaData;
|
||||
import com.kingsrook.qqq.backend.module.rdbms.actions.RDBMSTransaction;
|
||||
import com.kingsrook.qqq.backend.module.rdbms.jdbc.ConnectionManager;
|
||||
import org.junit.jupiter.api.AfterAll;
|
||||
import org.junit.jupiter.api.BeforeAll;
|
||||
import org.junit.jupiter.api.BeforeEach;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.params.ParameterizedTest;
|
||||
import org.junit.jupiter.params.provider.ValueSource;
|
||||
import org.testcontainers.containers.GenericContainer;
|
||||
import org.testcontainers.containers.wait.strategy.Wait;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
/*******************************************************************************
|
||||
** Verify provider reads using a real database and explicit caller transactions.
|
||||
@@ -155,6 +174,71 @@ class CustomFieldValuesPostgresTest extends PersonalServiceFixture
|
||||
.isEqualTo(commit ? "Pending transaction label" : null);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Native sub-queries: a reader whom the host does not let query the tag or
|
||||
** value tables still searches the fields of the parent table. Tag and custom
|
||||
** field terms match nothing and nothing is refused; an explicit Tags criterion
|
||||
** stays refused; a reader who holds the permission searches as before. The
|
||||
** parent has a text key, which PostgreSQL compares with the text record id.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void searchSurvivesHostProtectedCollaborationTables() throws Exception
|
||||
{
|
||||
instance.addTable(new QTableMetaData().withName("asset").withLabel("Assets").withBackendName("memory").withPrimaryKeyField("id")
|
||||
.withPermissionRules(new QPermissionRules().withLevel(PermissionLevel.READ_WRITE_PERMISSIONS))
|
||||
.withField(new QFieldMetaData("id", QFieldType.STRING)).withField(new QFieldMetaData("title", QFieldType.STRING))
|
||||
.withRecordLabelFormatAndFields("%s", "title"));
|
||||
try(Connection connection = connect(); var sql = connection.createStatement())
|
||||
{
|
||||
sql.execute("CREATE TABLE asset (id VARCHAR(40) PRIMARY KEY, title TEXT)");
|
||||
sql.execute("INSERT INTO asset VALUES ('A1', 'Visible'), ('A2', 'Other')");
|
||||
}
|
||||
login("alice", "asset.read", "asset.write");
|
||||
CollaborationRecords.insert(CollaborationTables.FIELD_DEFINITION, new QRecord().withValue("tableName", "asset").withValue("name", "note")
|
||||
.withValue("label", "Note").withValue("type", "STRING").withValue("status", "ACTIVE").withValue("maxLength", 30).withValue("searchable", true));
|
||||
CollaborationRecords.insert(CollaborationTables.FIELD_VALUE, new QRecord().withValue("tableName", "asset").withValue("recordId", "A2")
|
||||
.withValue("fieldName", "note").withValue("valueText", "Quarterly").withValue("parentDeleted", false));
|
||||
instance.getTable("asset").withVirtualField(new QVirtualFieldMetaData("collaborationTags", QFieldType.STRING).withLabel("Tags")
|
||||
.withIsQueryCriteria(true).withIsQuerySelectable(true).withPossibleValueSourceName(CollaborationTables.TAG).withShouldTranslatePossibleValues(false)
|
||||
.withValueProvider(new QCodeReference(TagValueProvider.class)).withCriteriaExpander(new QCodeReference(TagCriteriaExpander.class)));
|
||||
instance.getTable("asset").withVirtualField(new QVirtualFieldMetaData("cf_note", QFieldType.STRING).withLabel("Note").withIsEditable(false)
|
||||
.withIsQueryCriteria(true).withIsQuerySelectable(true).withValueProvider(new QCodeReference(CustomFieldValueProvider.class))
|
||||
.withCriteriaExpander(new QCodeReference(FieldCriteriaExpander.class)));
|
||||
instance.getTable("asset").withSearchFields(new ArrayList<>(List.of("title", "collaborationTags", "cf_note")));
|
||||
var tags = new TagsService();
|
||||
var tag = tags.create("asset", "Urgent", null);
|
||||
tags.apply("asset", "A1", List.of(tag.id()), List.of());
|
||||
QQueryFilter tagCriteria = new QQueryFilter().withCriteria("collaborationTags", QCriteriaOperator.EQUALS, tag.id());
|
||||
assertThat(search("Visible")).containsExactly("A1");
|
||||
assertThat(search("Urgent")).containsExactly("A1");
|
||||
assertThat(search("#urgent")).containsExactly("A1");
|
||||
assertThat(search("Quarterly")).containsExactly("A2");
|
||||
assertThat(new QueryAction().execute(new QueryInput("asset").withFilter(tagCriteria)).getRecords()).hasSize(1);
|
||||
|
||||
SearchWithoutCollaborationReadTest.protect(instance, CollaborationTables.RECORD_TAG, CollaborationTables.TAG, CollaborationTables.FIELD_VALUE);
|
||||
assertThat(search("Visible")).containsExactly("A1");
|
||||
assertThat(search("Urgent")).isEmpty();
|
||||
assertThat(search("#urgent")).isEmpty();
|
||||
assertThat(search("Quarterly")).isEmpty();
|
||||
assertThatThrownBy(() -> new QueryAction().execute(new QueryInput("asset").withFilter(tagCriteria))).isInstanceOf(QPermissionDeniedException.class);
|
||||
|
||||
login("alice", "asset.read", "asset.write", CollaborationPermissions.VIEW);
|
||||
assertThat(search("Visible")).containsExactly("A1");
|
||||
assertThat(search("Urgent")).containsExactly("A1");
|
||||
assertThat(search("#urgent")).containsExactly("A1");
|
||||
assertThat(search("Quarterly")).containsExactly("A2");
|
||||
assertThat(new QueryAction().execute(new QueryInput("asset").withFilter(tagCriteria)).getRecords()).hasSize(1);
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Global record search as a signed-in person, limited to the text-keyed table.
|
||||
*******************************************************************************/
|
||||
private List<String> search(String term) throws Exception
|
||||
{
|
||||
return (new RecordSearchAction().execute(new RecordSearchInput().withTableNames(List.of("asset")).withSearchTerm(term).withInputSource(QInputSource.USER))
|
||||
.getResults().stream().map(result -> String.valueOf(result.getRecordId())).sorted().toList());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Explicit query transactions see pending values even without thread context.
|
||||
*******************************************************************************/
|
||||
|
||||
@@ -47,6 +47,8 @@ import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
*******************************************************************************/
|
||||
class RedisPresenceRecoveryTest extends PersonalServiceFixture
|
||||
{
|
||||
private static final com.fasterxml.jackson.databind.ObjectMapper JSON = new com.fasterxml.jackson.databind.ObjectMapper();
|
||||
|
||||
/***************************************************************************
|
||||
** Peer reads cannot prolong cold recovery; normal expiry stays known-empty.
|
||||
***************************************************************************/
|
||||
@@ -209,6 +211,117 @@ class RedisPresenceRecoveryTest extends PersonalServiceFixture
|
||||
}
|
||||
|
||||
|
||||
/***************************************************************************
|
||||
** Normal use cannot restart recovery. Every write that advances the guard's
|
||||
** latest issued expiry also leaves a member (a departure marker included)
|
||||
** expiring at exactly that instant, so the table room is never empty before
|
||||
** it. Only real loss of the projections starts a new interval.
|
||||
***************************************************************************/
|
||||
@Test
|
||||
void normalUseNeverRestartsRecovery() throws Exception
|
||||
{
|
||||
try(var redis = RedisFixtures.redis())
|
||||
{
|
||||
redis.start();
|
||||
var address = new HostAndPort(RedisFixtures.host(redis), RedisFixtures.port(redis));
|
||||
var config = DefaultJedisClientConfig.builder().build();
|
||||
try(var first = new RedisConnection(address, config, 2, Duration.ofMillis(500));
|
||||
var second = new RedisConnection(address, config, 2, Duration.ofMillis(500)))
|
||||
{
|
||||
var namespace = new RedisNamespace("test", "no-rearm", "tenant", "presence");
|
||||
var a = new RedisPresenceStore(first.scripts(), namespace, Duration.ofSeconds(2), 10);
|
||||
var b = new RedisPresenceStore(second.scripts(), namespace, Duration.ofSeconds(2), 10);
|
||||
assertThatThrownBy(() -> a.snapshot("document", null)).isInstanceOf(PresenceRecoveringException.class);
|
||||
String guardKey = first.client().keys("*:recovery").iterator().next();
|
||||
var warm = JSON.readTree(first.client().get(guardKey));
|
||||
String generation = warm.path("generation").asText();
|
||||
long readyAt = warm.path("readyAt").asLong();
|
||||
long deadline = System.nanoTime() + Duration.ofSeconds(6).toNanos();
|
||||
while(true)
|
||||
{
|
||||
try
|
||||
{
|
||||
assertThat(b.snapshot("document", null)).isEmpty();
|
||||
break;
|
||||
}
|
||||
catch(PresenceRecoveringException recovering)
|
||||
{
|
||||
assertThat(System.nanoTime()).isLessThan(deadline);
|
||||
Thread.sleep(50);
|
||||
}
|
||||
}
|
||||
|
||||
// One person opens the table and leaves within the lease lifetime;
|
||||
// the next visitor reads a known-empty room, not a recovering one.
|
||||
String tableLease = a.openLease("alice", "Alice", "document", null);
|
||||
assertGuardCoversTableRoom(first, guardKey, generation, readyAt);
|
||||
a.renewLease("alice", "Alice", tableLease, "document", null, "VIEWING", null, 1L);
|
||||
assertThat(b.snapshot("document", null)).extracting("userId").containsExactly("alice");
|
||||
a.leave("alice", tableLease, "document", null, 2L);
|
||||
assertThat(b.snapshot("document", null)).isEmpty();
|
||||
assertThat(b.snapshot("document", "7")).isEmpty();
|
||||
assertGuardCoversTableRoom(first, guardKey, generation, readyAt);
|
||||
|
||||
// The same holds for a record lease, a legacy connection and a
|
||||
// reservation that is never activated.
|
||||
String recordLease = b.openLease("bob", "Bob", "document", "1");
|
||||
b.renewLease("bob", "Bob", recordLease, "document", "1", "EDITING", "title", 1L);
|
||||
b.leave("bob", recordLease, "document", "1", 2L);
|
||||
assertThat(a.snapshot("document", "1")).isEmpty();
|
||||
assertGuardCoversTableRoom(first, guardKey, generation, readyAt);
|
||||
a.heartbeat("carol", "Carol", "tab", "document", "2", "VIEWING", null, 1L);
|
||||
a.leave("carol", "tab", "document", "2", 2L);
|
||||
assertThat(b.snapshot("document", null)).isEmpty();
|
||||
assertGuardCoversTableRoom(first, guardKey, generation, readyAt);
|
||||
assertThat(a.openLease("dave", "Dave", "document", "3")).isNotNull();
|
||||
assertGuardCoversTableRoom(first, guardKey, generation, readyAt);
|
||||
|
||||
// Read continuously while every marker and reservation expires: the
|
||||
// boundary between "markers present" and "idle" is never recovering.
|
||||
int reads = 0;
|
||||
deadline = System.nanoTime() + Duration.ofSeconds(6).toNanos();
|
||||
while(first.client().keys("*").size() > 1)
|
||||
{
|
||||
assertThat(System.nanoTime()).isLessThan(deadline);
|
||||
assertThat(a.snapshot("document", null)).isEmpty();
|
||||
assertThat(b.snapshot("document", "1")).isEmpty();
|
||||
reads++;
|
||||
}
|
||||
assertThat(reads).isPositive();
|
||||
assertThat(a.snapshot("document", null)).isEmpty();
|
||||
assertThat(b.snapshot("document", "1")).isEmpty();
|
||||
var idle = JSON.readTree(first.client().get(guardKey));
|
||||
assertThat(idle.path("generation").asText()).isEqualTo(generation);
|
||||
assertThat(idle.path("readyAt").asLong()).isEqualTo(readyAt);
|
||||
|
||||
// Contrast: projections lost before the latest issued expiry are real
|
||||
// loss, and that does start a new generation and interval.
|
||||
assertThat(a.openLease("erin", "Erin", "document", "1")).isNotNull();
|
||||
first.client().del(first.client().keys("*").stream().filter(key -> !key.endsWith(":recovery")).toArray(String[]::new));
|
||||
assertThatThrownBy(() -> b.snapshot("document", null)).isInstanceOf(PresenceRecoveringException.class);
|
||||
assertThat(JSON.readTree(first.client().get(guardKey)).path("generation").asText()).isNotEqualTo(generation);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/***************************************************************************
|
||||
** The invariant that rules out a restart: same generation and readiness, and
|
||||
** the newest table member expires exactly at the guard's latest issued expiry.
|
||||
***************************************************************************/
|
||||
private void assertGuardCoversTableRoom(RedisConnection connection, String guardKey, String generation, long readyAt) throws Exception
|
||||
{
|
||||
var guard = JSON.readTree(connection.client().get(guardKey));
|
||||
assertThat(guard.path("generation").asText()).isEqualTo(generation);
|
||||
assertThat(guard.path("readyAt").asLong()).isEqualTo(readyAt);
|
||||
String index = guardKey.substring(0, guardKey.length() - "recovery".length()) + "table:expiry";
|
||||
var newest = connection.client().zrangeWithScores(index, -1, -1);
|
||||
assertThat(newest).hasSize(1);
|
||||
assertThat((long) newest.getFirst().getScore()).isEqualTo(guard.path("liveUntil").asLong());
|
||||
assertThat(connection.client().pttl(index)).isPositive();
|
||||
}
|
||||
|
||||
|
||||
/***************************************************************************
|
||||
** Restore must happen in maintenance, followed by a fresh host-owned generation.
|
||||
** Replaying both old guard and lease bytes cannot authorize the new deployment.
|
||||
|
||||
@@ -309,8 +309,15 @@ class RedisPresenceServiceTest extends PersonalServiceFixture
|
||||
assertThatThrownBy(() -> service.heartbeat("tab", "document", "3", "VIEWING", null, 1L))
|
||||
.isInstanceOf(QNotFoundException.class);
|
||||
admin();
|
||||
assertThatThrownBy(com.kingsrook.qbits.collaboration.app.CollaborationHome::presence)
|
||||
.isInstanceOf(QServiceUnavailableException.class);
|
||||
var unavailable = com.kingsrook.qbits.collaboration.app.CollaborationHome.presence();
|
||||
assertThat(unavailable.getRows()).isEmpty();
|
||||
assertThat(unavailable.getNoRowsFoundHTML()).isEqualTo(com.kingsrook.qbits.collaboration.app.CollaborationHome.PRESENCE_UNAVAILABLE);
|
||||
var summary = com.kingsrook.qqq.backend.core.actions.collaboration.RecordFeatureSummaryHelper.summarize(instance.getTable("document"),
|
||||
com.kingsrook.qbits.collaboration.security.ParentRecordAccess.requireVisibleParent("document", 1));
|
||||
assertThat(summary.getPresence()).isNull();
|
||||
assertThat(summary.getExtensions().get(CollaborationSummaryProvider.PRESENCE_EXTENSION))
|
||||
.isEqualTo(java.util.Map.of("status", CollaborationSummaryProvider.PRESENCE_UNAVAILABLE));
|
||||
assertThat(summary.getCounts()).containsKeys("openComments", "openTasks", "tags", "attachments", "followers");
|
||||
}
|
||||
finally
|
||||
{
|
||||
|
||||
+250
@@ -0,0 +1,250 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package com.kingsrook.qbits.collaboration;
|
||||
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import com.kingsrook.qbits.collaboration.customfields.CustomFieldValueProvider;
|
||||
import com.kingsrook.qbits.collaboration.customfields.FieldCriteriaExpander;
|
||||
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
|
||||
import com.kingsrook.qbits.collaboration.permissions.CollaborationPermissions;
|
||||
import com.kingsrook.qbits.collaboration.tags.TagsService;
|
||||
import com.kingsrook.qqq.backend.core.actions.collaboration.RuntimeMetaDataOverlay;
|
||||
import com.kingsrook.qqq.backend.core.actions.customizers.QCodeLoader;
|
||||
import com.kingsrook.qqq.backend.core.actions.permissions.CustomPermissionChecker;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.RecordSearchAction;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.QInputSource;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.search.RecordSearchInput;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.fields.QVirtualFieldMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.permissions.MetaDataWithPermissionRules;
|
||||
import com.kingsrook.qqq.backend.core.utils.ValueUtils;
|
||||
import org.junit.jupiter.api.BeforeEach;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Record search for a person who may read a table, on a host that does not let
|
||||
** that person query the tag or custom field value tables: no tag or custom
|
||||
** field matches, never a refused search.
|
||||
*******************************************************************************/
|
||||
class SearchWithoutCollaborationReadTest extends PersonalServiceFixture
|
||||
{
|
||||
private int tagId;
|
||||
|
||||
|
||||
|
||||
/***************************************************************************
|
||||
** Document 1 is tagged "Urgent"; document 2 has the searchable custom field
|
||||
** "note" set to "Quarterly". Titles are searchable too. The Tags field is the
|
||||
** real contribution; the custom field is the virtual field contributed for a
|
||||
** backend whose values live in the value table (the memory backend used here
|
||||
** would otherwise store them on the parent, with no sub-query).
|
||||
***************************************************************************/
|
||||
@BeforeEach
|
||||
void searchableDocuments() throws Exception
|
||||
{
|
||||
instance.getTable("document").withSearchFields(new ArrayList<>(List.of("title")));
|
||||
RuntimeMetaDataOverlay.refresh(instance);
|
||||
admin();
|
||||
CollaborationRecords.insert(CollaborationTables.FIELD_DEFINITION, new QRecord().withValue("tableName", "document").withValue("name", "note")
|
||||
.withValue("label", "Note").withValue("type", "STRING").withValue("status", "ACTIVE").withValue("maxLength", 30).withValue("searchable", true));
|
||||
CollaborationRecords.insert(CollaborationTables.FIELD_VALUE, new QRecord().withValue("tableName", "document").withValue("recordId", "2")
|
||||
.withValue("fieldName", "note").withValue("valueText", "Quarterly").withValue("parentDeleted", false));
|
||||
login("alice");
|
||||
var tag = new TagsService().create("document", "Urgent", null);
|
||||
new TagsService().apply("document", 1, List.of(tag.id()), List.of());
|
||||
tagId = tag.id();
|
||||
instance.getTable("document").withVirtualField(new QVirtualFieldMetaData("cf_note", QFieldType.STRING).withLabel("Note").withIsEditable(false)
|
||||
.withIsQueryCriteria(true).withIsQuerySelectable(true).withValueProvider(new QCodeReference(CustomFieldValueProvider.class))
|
||||
.withCriteriaExpander(new QCodeReference(FieldCriteriaExpander.class)));
|
||||
instance.getTable("document").getSearchFields().add("cf_note");
|
||||
assertThat(instance.getTable("document").getSearchFields()).contains("title", "collaborationTags", "cf_note");
|
||||
assertThat(instance.getTable("document").getVirtualFields()).containsKeys("collaborationTags", "cf_note");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/***************************************************************************
|
||||
** With the default rules every reader of the table searches by tag and by
|
||||
** custom field, as before.
|
||||
***************************************************************************/
|
||||
@Test
|
||||
void defaultRulesSearchTagsAndCustomFields() throws Exception
|
||||
{
|
||||
assertThat(search("First")).containsExactly(1);
|
||||
assertThat(search("Urgent")).containsExactly(1);
|
||||
assertThat(search("#urgent")).containsExactly(1);
|
||||
assertThat(search("Quarterly")).containsExactly(2);
|
||||
assertThat(tagFilter()).containsExactly(1);
|
||||
assertThat(noteFilter()).containsExactly(2);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/***************************************************************************
|
||||
** A host that requires a permission on the link tables: a reader without it
|
||||
** still searches the fields of the table itself; tag and custom field terms
|
||||
** match nothing; explicit criteria on those fields stay refused; nothing
|
||||
** changes for a reader who holds the permission.
|
||||
***************************************************************************/
|
||||
@Test
|
||||
void protectedTablesContributeNothingInsteadOfRefusingTheSearch() throws Exception
|
||||
{
|
||||
protect(instance, CollaborationTables.RECORD_TAG, CollaborationTables.TAG, CollaborationTables.FIELD_VALUE);
|
||||
assertThat(search("First")).containsExactly(1);
|
||||
assertThat(search("document")).containsExactly(1, 2);
|
||||
assertThat(search("Urgent")).isEmpty();
|
||||
assertThat(search("#urgent")).isEmpty();
|
||||
assertThat(search("Quarterly")).isEmpty();
|
||||
assertThatThrownBy(this::tagFilter).isInstanceOf(QPermissionDeniedException.class);
|
||||
assertThatThrownBy(this::noteFilter).isInstanceOf(QPermissionDeniedException.class);
|
||||
|
||||
login("alice", CollaborationPermissions.VIEW);
|
||||
assertThat(search("First")).containsExactly(1);
|
||||
assertThat(search("Urgent")).containsExactly(1);
|
||||
assertThat(search("#urgent")).containsExactly(1);
|
||||
assertThat(search("Quarterly")).containsExactly(2);
|
||||
assertThat(tagFilter()).containsExactly(1);
|
||||
assertThat(noteFilter()).containsExactly(2);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/***************************************************************************
|
||||
** Either tag table being refused is enough: both are sub-query tables.
|
||||
***************************************************************************/
|
||||
@Test
|
||||
void eitherProtectedTagTableContributesNothing() throws Exception
|
||||
{
|
||||
var defaultTagRules = instance.getTable(CollaborationTables.TAG).getPermissionRules();
|
||||
protect(instance, CollaborationTables.TAG);
|
||||
assertThat(search("First")).containsExactly(1);
|
||||
assertThat(search("#urgent")).isEmpty();
|
||||
assertThat(search("Quarterly")).containsExactly(2);
|
||||
instance.getTable(CollaborationTables.TAG).withPermissionRules(defaultTagRules);
|
||||
assertThat(search("#urgent")).containsExactly(1);
|
||||
protect(instance, CollaborationTables.RECORD_TAG);
|
||||
assertThat(search("First")).containsExactly(1);
|
||||
assertThat(search("Urgent")).isEmpty();
|
||||
assertThat(search("Quarterly")).containsExactly(2);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/***************************************************************************
|
||||
** What a host does when it puts its own permission on these tables: the
|
||||
** permission is required first, then the table keeps its own checker.
|
||||
***************************************************************************/
|
||||
static void protect(QInstance instance, String... tableNames)
|
||||
{
|
||||
for(String tableName : tableNames)
|
||||
{
|
||||
var rules = instance.getTable(tableName).getPermissionRules();
|
||||
HostPermission.DELEGATES.put(tableName, rules.getCustomPermissionChecker());
|
||||
instance.getTable(tableName).withPermissionRules(rules.clone().withCustomPermissionChecker(new QCodeReference(HostPermission.class)));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/***************************************************************************
|
||||
** A host permission in front of a table's own checker.
|
||||
***************************************************************************/
|
||||
public static class HostPermission implements CustomPermissionChecker
|
||||
{
|
||||
private static final Map<String, QCodeReference> DELEGATES = new ConcurrentHashMap<>();
|
||||
|
||||
|
||||
|
||||
/************************************************************************
|
||||
** Refuse a session without the permission; otherwise ask the table.
|
||||
************************************************************************/
|
||||
@Override
|
||||
public void checkPermissionsThrowing(AbstractActionInput input, MetaDataWithPermissionRules metadata) throws QPermissionDeniedException
|
||||
{
|
||||
if(!QContext.getQSession().hasPermission(CollaborationPermissions.VIEW))
|
||||
{
|
||||
throw (new QPermissionDeniedException("This needs the " + CollaborationPermissions.VIEW + " permission."));
|
||||
}
|
||||
QCodeLoader.getAdHoc(CustomPermissionChecker.class, DELEGATES.get(metadata.getName())).checkPermissionsThrowing(input, metadata);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/***************************************************************************
|
||||
** Global record search as a signed-in person, limited to the host table.
|
||||
***************************************************************************/
|
||||
private List<Integer> search(String term) throws QException
|
||||
{
|
||||
return (new RecordSearchAction().execute(new RecordSearchInput().withTableNames(List.of("document")).withSearchTerm(term).withInputSource(QInputSource.USER))
|
||||
.getResults().stream().map(result -> ValueUtils.getValueAsInteger(result.getRecordId())).sorted().toList());
|
||||
}
|
||||
|
||||
|
||||
|
||||
/***************************************************************************
|
||||
** An explicit Tags criterion on a record list.
|
||||
***************************************************************************/
|
||||
private List<Integer> tagFilter() throws QException
|
||||
{
|
||||
return (ids(new QQueryFilter().withCriteria("collaborationTags", QCriteriaOperator.EQUALS, tagId)));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/***************************************************************************
|
||||
** An explicit custom field criterion on a record list.
|
||||
***************************************************************************/
|
||||
private List<Integer> noteFilter() throws QException
|
||||
{
|
||||
return (ids(new QQueryFilter().withCriteria("cf_note", QCriteriaOperator.EQUALS, "Quarterly")));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/***************************************************************************
|
||||
** Matching document ids through the real query action.
|
||||
***************************************************************************/
|
||||
private List<Integer> ids(QQueryFilter filter) throws QException
|
||||
{
|
||||
QueryInput input = new QueryInput("document").withFilter(filter);
|
||||
input.setInputSource(QInputSource.USER);
|
||||
return (new QueryAction().execute(input).getRecords().stream().map(row -> row.getValueInteger("id")).sorted().toList());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,368 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
|
||||
package com.kingsrook.qbits.collaboration;
|
||||
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.TreeMap;
|
||||
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
|
||||
import com.kingsrook.qbits.collaboration.permissions.CollaborationPermissions;
|
||||
import com.kingsrook.qbits.collaboration.tags.TagEntry;
|
||||
import com.kingsrook.qbits.collaboration.tags.TaggedTables;
|
||||
import com.kingsrook.qbits.collaboration.tags.TagsService;
|
||||
import com.kingsrook.qqq.backend.core.actions.QBackendTransaction;
|
||||
import com.kingsrook.qqq.backend.core.actions.collaboration.RuntimeMetaDataOverlay;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
|
||||
import com.kingsrook.qqq.backend.core.cluster.ClusterCacheAudit;
|
||||
import com.kingsrook.qqq.backend.core.cluster.ClusterCaches;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordReferenceSecurityConfig;
|
||||
import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryRecordStore;
|
||||
import com.kingsrook.qqq.backend.core.utils.lambdas.UnsafeVoidVoidMethod;
|
||||
import org.junit.jupiter.api.BeforeEach;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Count the backend queries this qbit adds to ordinary reads of a host table.
|
||||
** Every backend query is one SQL statement (and one pool checkout) on an RDBMS
|
||||
** backend, and core selects the Tags field for every read that names no fields:
|
||||
** a table nobody tagged must not pay for it, and a tagged table pays per page.
|
||||
**
|
||||
** Before TaggedTables (1.2.0-RC.2), with 600 added parents: a 602-row list ran
|
||||
** 2 assignment queries, 600 gets ran 600 and 100 single-record updates ran 100.
|
||||
*******************************************************************************/
|
||||
class TagLookupStatementBudgetTest extends PersonalServiceFixture
|
||||
{
|
||||
private static final int PARENTS = 600;
|
||||
|
||||
private final TagsService tags = new TagsService();
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** 600 more visible parents, and the Tags field every enabled table receives.
|
||||
*******************************************************************************/
|
||||
@BeforeEach
|
||||
void parents() throws Exception
|
||||
{
|
||||
List<QRecord> parents = new ArrayList<>();
|
||||
for(int id = 4; id <= PARENTS + 3; id++)
|
||||
{
|
||||
parents.add(new QRecord().withValue("id", id).withValue("title", "Document " + id).withValue("tenant", "a"));
|
||||
}
|
||||
MemoryRecordStore.getInstance().insert(new InsertInput("document").withRecords(parents), true);
|
||||
RuntimeMetaDataOverlay.refresh(instance);
|
||||
assertThat(instance.getTable("document").getVirtualFields()).containsKey("collaborationTags");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A table nobody tagged: one probe, then lists, per-record gets and per-record
|
||||
** updates (which read the stored record first) add nothing at all.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void untaggedTableCostsOneProbeThenNothing() throws Exception
|
||||
{
|
||||
assertThat(collaborationQueries(this::listWithoutTags)).isEqualTo(Map.of(CollaborationTables.RECORD_TAG, 1L));
|
||||
assertThat(collaborationQueries(this::listWithoutTags)).isEmpty();
|
||||
assertThat(collaborationQueries(() ->
|
||||
{
|
||||
for(int id = 4; id <= PARENTS + 3; id++)
|
||||
{
|
||||
assertThat(new GetAction().executeForRecord(new GetInput("document").withPrimaryKey(id)).getValue("collaborationTags")).isNull();
|
||||
}
|
||||
})).isEmpty();
|
||||
assertThat(collaborationQueries(() ->
|
||||
{
|
||||
for(int id = 4; id <= 103; id++)
|
||||
{
|
||||
new UpdateAction().execute(new UpdateInput("document").withRecords(List.of(new QRecord().withValue("id", id).withValue("title", "Renamed " + id))));
|
||||
}
|
||||
})).isEmpty();
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A tagged table: one assignment lookup and one vocabulary lookup per 500
|
||||
** parents, however many records the caller asked for, plus the probe whenever
|
||||
** the node has to ask again.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void taggedTableCostsOneLookupPerPage() throws Exception
|
||||
{
|
||||
TagEntry tag = tags.create("document", "Boundary", null);
|
||||
tags.apply("document", 1, List.of(tag.id()), List.of());
|
||||
tags.apply("document", 504, List.of(tag.id()), List.of());
|
||||
assertThat(collaborationQueries(() -> assertThat(taggedIds()).containsExactlyInAnyOrder(1, 504)))
|
||||
.isEqualTo(Map.of(CollaborationTables.RECORD_TAG, 2L, CollaborationTables.TAG, 2L));
|
||||
ClusterCaches.invalidate(TaggedTables.CACHE_NAME);
|
||||
assertThat(collaborationQueries(() -> assertThat(taggedIds()).containsExactlyInAnyOrder(1, 504)))
|
||||
.isEqualTo(Map.of(CollaborationTables.RECORD_TAG, 3L, CollaborationTables.TAG, 2L));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A table's first assignment shows at once on the node that wrote it, and the
|
||||
** last removal leaves the field empty again.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void firstAssignmentShowsAtOnce() throws Exception
|
||||
{
|
||||
listWithoutTags();
|
||||
TagEntry tag = tags.create("document", "Urgent", null);
|
||||
tags.apply("document", 2, List.of(tag.id()), List.of());
|
||||
assertThat(taggedIds()).containsExactly(2);
|
||||
assertThat(new GetAction().executeForRecord(new GetInput("document").withPrimaryKey(2)).getValueString("collaborationTags")).isEqualTo("Urgent");
|
||||
tags.apply("document", 2, List.of(), List.of(tag.id()));
|
||||
listWithoutTags();
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The answer is forgotten when the assignment is written, so its transaction
|
||||
** reads its own tag (the service already asked again, so the list does not),
|
||||
** and again when it commits, so a probe that ran in between and could not see
|
||||
** the assignment yet does not outlive the commit.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void commitForgetsAgain() throws Exception
|
||||
{
|
||||
listWithoutTags();
|
||||
TagEntry tag = tags.create("document", "Urgent", null);
|
||||
QBackendTransaction transaction = new QBackendTransaction();
|
||||
QContext.setTransaction(transaction);
|
||||
try
|
||||
{
|
||||
tags.apply("document", 2, List.of(tag.id()), List.of());
|
||||
assertThat(assignmentQueries(() -> assertThat(taggedIds()).containsExactly(2))).isEqualTo(2L);
|
||||
transaction.commit();
|
||||
}
|
||||
finally
|
||||
{
|
||||
QContext.clearTransaction();
|
||||
}
|
||||
assertThat(assignmentQueries(() -> assertThat(taggedIds()).containsExactly(2))).isEqualTo(3L);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** An assignment this node did not write (another node, or SQL outside QQQ)
|
||||
** shows when the ClusterBus says so.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void anotherNodesAssignmentShowsAfterClusterInvalidation() throws Exception
|
||||
{
|
||||
listWithoutTags();
|
||||
assignOutsideThisNode(tags.create("document", "Elsewhere", null));
|
||||
listWithoutTags();
|
||||
ClusterCaches.invalidateKey(TaggedTables.CACHE_NAME, "document");
|
||||
assertThat(taggedIds()).containsExactly(1);
|
||||
|
||||
ClusterCaches.invalidate(TaggedTables.CACHE_NAME);
|
||||
assertThat(assignmentQueries(() -> assertThat(taggedIds()).containsExactly(1))).isEqualTo(3L);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Without a cluster transport the timeout is the bound, and 0 turns the
|
||||
** memory off: every read looks, as before.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void timeoutBoundsWhatNoEventReachesAndZeroDisables() throws Exception
|
||||
{
|
||||
config.withTagLookupCacheSeconds(1);
|
||||
listWithoutTags();
|
||||
assignOutsideThisNode(tags.create("document", "Elsewhere", null));
|
||||
listWithoutTags();
|
||||
Thread.sleep(1100);
|
||||
assertThat(taggedIds()).containsExactly(1);
|
||||
|
||||
config.withTagLookupCacheSeconds(0);
|
||||
assertThat(assignmentQueries(() -> assertThat(taggedIds()).containsExactly(1))).isEqualTo(2L);
|
||||
|
||||
List<String> errors = new ArrayList<>();
|
||||
config.withTagLookupCacheSeconds(-1).validate(instance, errors);
|
||||
assertThat(errors).contains("tagLookupCacheSeconds must be 0 (off) or more");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The remembered answer is the same for every session: a caller who may not
|
||||
** see a table's only tagged record must not hide its tags from one who may.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void answerDoesNotDependOnTheCallersRowAccess() throws Exception
|
||||
{
|
||||
config.withMirroredSecurityKeyTypes(List.of("tenant"));
|
||||
RecordReferenceSecurityConfig.of(instance).withMirroredSecurityKeyTypes(List.of("tenant"));
|
||||
for(var table : instance.getTables().values())
|
||||
{
|
||||
CoreCollaboration.reference(table, config);
|
||||
}
|
||||
QContext.getQSession().withSecurityKeyValue("allTenants", true);
|
||||
TagEntry tag = tags.create("document", "Secret", null);
|
||||
tags.apply("document", 3, List.of(tag.id()), List.of());
|
||||
assertThat(CollaborationRecords.cleanupQuery(CollaborationTables.RECORD_TAG, new QQueryFilter()).getFirst().getValueString("lockKey_tenant")).isEqualTo("b");
|
||||
|
||||
login("alice");
|
||||
ClusterCaches.invalidate(TaggedTables.CACHE_NAME);
|
||||
assertThat(assignmentQueries(this::listWithoutTags)).isEqualTo(3L);
|
||||
QContext.getQSession().getSecurityKeyValues().put("tenant", new ArrayList<>(List.of("b")));
|
||||
assertThat(new QueryAction().execute(new QueryInput("document")).getRecords()).extracting(row -> row.getValueString("collaborationTags")).containsExactly("Secret");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Browsing one of this qbit's child tables authorizes each distinct parent
|
||||
** once, not once per row (61 rows on two parents ran 61 parent reads before).
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void childRowsSharingAParentCostOneParentLookup() throws Exception
|
||||
{
|
||||
List<QRecord> tasks = new ArrayList<>();
|
||||
for(int id = 1; id <= 61; id++)
|
||||
{
|
||||
tasks.add(new QRecord().withValue("id", id).withValue("tableName", "document").withValue("recordId", id == 61 ? "2" : "1")
|
||||
.withValue("title", "Task " + id).withValue("assigneeUserId", "alice").withValue("createdByUserId", "alice")
|
||||
.withValue("status", "OPEN").withValue("parentDeleted", false));
|
||||
}
|
||||
MemoryRecordStore.getInstance().insert(new InsertInput(CollaborationTables.TASK).withRecords(tasks), true);
|
||||
login("alice", CollaborationPermissions.VIEW);
|
||||
assertThat(backendQueries(() -> assertThat(new QueryAction().execute(new QueryInput(CollaborationTables.TASK)).getRecords()).hasSize(61)).get("document")).isEqualTo(2L);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A static cache that is not registered would go stale on the other nodes.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void cachesAreRegisteredForClusterInvalidation() throws Exception
|
||||
{
|
||||
listWithoutTags();
|
||||
assertThat(ClusterCacheAudit.unregisteredStaticCaches("com.kingsrook.qbits.collaboration.tags")).isEmpty();
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Every visible parent, none of them tagged.
|
||||
*******************************************************************************/
|
||||
private void listWithoutTags() throws Exception
|
||||
{
|
||||
assertThat(new QueryAction().execute(new QueryInput("document")).getRecords()).hasSize(PARENTS + 2)
|
||||
.allSatisfy(row -> assertThat(row.getValue("collaborationTags")).isNull());
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The ids of the visible parents that carry a tag label.
|
||||
*******************************************************************************/
|
||||
private List<Integer> taggedIds() throws Exception
|
||||
{
|
||||
return (new QueryAction().execute(new QueryInput("document")).getRecords().stream()
|
||||
.filter(row -> row.getValue("collaborationTags") != null).map(row -> row.getValueInteger("id")).toList());
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Store an assignment for document 1 without any action or listener running.
|
||||
*******************************************************************************/
|
||||
private void assignOutsideThisNode(TagEntry tag) throws Exception
|
||||
{
|
||||
MemoryRecordStore.getInstance().insert(new InsertInput(CollaborationTables.RECORD_TAG).withRecords(List.of(new QRecord()
|
||||
.withValue("tableName", "document").withValue("recordId", "1").withValue("tagId", tag.id())
|
||||
.withValue("createdByUserId", "alice").withValue("parentDeleted", false))), true);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Backend queries against the assignment table while the work runs.
|
||||
*******************************************************************************/
|
||||
private long assignmentQueries(UnsafeVoidVoidMethod<Exception> work) throws Exception
|
||||
{
|
||||
return (collaborationQueries(work).getOrDefault(CollaborationTables.RECORD_TAG, 0L));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Backend queries against this qbit's tables while the work runs, by table.
|
||||
*******************************************************************************/
|
||||
private Map<String, Long> collaborationQueries(UnsafeVoidVoidMethod<Exception> work) throws Exception
|
||||
{
|
||||
Map<String, Long> result = backendQueries(work);
|
||||
result.keySet().removeIf(table -> !table.startsWith(CollaborationQBitConfig.TABLE_NAME_PREFIX));
|
||||
return (result);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Every backend query while the work runs, by table.
|
||||
*******************************************************************************/
|
||||
private Map<String, Long> backendQueries(UnsafeVoidVoidMethod<Exception> work) throws Exception
|
||||
{
|
||||
MemoryRecordStore.resetStatistics();
|
||||
MemoryRecordStore.setCollectStatistics(true);
|
||||
try
|
||||
{
|
||||
work.run();
|
||||
Map<String, Long> result = new TreeMap<>();
|
||||
List<AbstractActionInput> inputs = MemoryRecordStore.getActionInputs().get(QueryInput.class);
|
||||
for(AbstractActionInput input : inputs == null ? List.<AbstractActionInput>of() : inputs)
|
||||
{
|
||||
result.merge(((QueryInput) input).getTableName(), 1L, Long::sum);
|
||||
}
|
||||
return (result);
|
||||
}
|
||||
finally
|
||||
{
|
||||
MemoryRecordStore.setCollectStatistics(false);
|
||||
MemoryRecordStore.resetStatistics();
|
||||
}
|
||||
}
|
||||
}
|
||||
+83
@@ -0,0 +1,83 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
|
||||
package com.kingsrook.qbits.collaboration.attachments.uploadjournal;
|
||||
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The torn-page decision of the PostgreSQL journal qualification, without a
|
||||
** database: the rule is a pure function of the observed full_page_writes value
|
||||
** and the Aurora detection column. The real-server cases, including the
|
||||
** detection column itself, are in the opt-in PostgresUploadJournalTest.
|
||||
*******************************************************************************/
|
||||
class PostgresJournalDatabaseTest
|
||||
{
|
||||
|
||||
/*******************************************************************************
|
||||
** "on" qualifies everywhere; "off" qualifies only on Amazon Aurora PostgreSQL.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void fullPageWritesOffIsAcceptedOnlyOnAurora()
|
||||
{
|
||||
assertThat(PostgresJournalDatabase.tornPagesPrevented("on", false)).isTrue();
|
||||
assertThat(PostgresJournalDatabase.tornPagesPrevented("on", true)).isTrue();
|
||||
assertThat(PostgresJournalDatabase.tornPagesPrevented("off", true)).isTrue();
|
||||
assertThat(PostgresJournalDatabase.tornPagesPrevented("off", false)).isFalse();
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Anything that is neither "on" nor Aurora's "off" is refused, Aurora or not:
|
||||
** a missing value, another spelling or an unexpected value.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void unknownFullPageWritesValuesAreRefused()
|
||||
{
|
||||
for(boolean aurora : new boolean[] { false, true })
|
||||
{
|
||||
assertThat(PostgresJournalDatabase.tornPagesPrevented(null, aurora)).isFalse();
|
||||
assertThat(PostgresJournalDatabase.tornPagesPrevented("", aurora)).isFalse();
|
||||
assertThat(PostgresJournalDatabase.tornPagesPrevented("OFF", aurora)).isFalse();
|
||||
assertThat(PostgresJournalDatabase.tornPagesPrevented("ON", aurora)).isFalse();
|
||||
assertThat(PostgresJournalDatabase.tornPagesPrevented("false", aurora)).isFalse();
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The settings query detects Aurora only through to_regprocedure, as its last
|
||||
** column. It never calls aurora_version() directly, which would abort the
|
||||
** transaction on stock PostgreSQL.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void settingsQueryDetectsAuroraOnlyThroughToRegprocedure()
|
||||
{
|
||||
String detection = "to_regprocedure('aurora_version()') IS NOT NULL";
|
||||
assertThat(PostgresJournalDatabase.SETTINGS).endsWith(", " + detection);
|
||||
assertThat(PostgresJournalDatabase.SETTINGS.replace(detection, "")).doesNotContain("aurora");
|
||||
}
|
||||
}
|
||||
+144
-1
@@ -81,6 +81,11 @@ import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
** dynamic port, random synthetic passwords); the application connects as a
|
||||
** DML-only role, and the superuser is used only for fixture DDL and fault
|
||||
** injection. The attachment row is a minimal stand-in, not a full intake fixture.
|
||||
**
|
||||
** No Amazon Aurora server is used. The Aurora cases run on this stock PostgreSQL
|
||||
** 17 server and simulate exactly two things: a function named aurora_version()
|
||||
** created in the suite's database (so to_regprocedure resolves, as it does on
|
||||
** Aurora), and full_page_writes turned off on the server (as Aurora reports it).
|
||||
*******************************************************************************/
|
||||
@EnabledIfSystemProperty(named = "postgresJournalTests", matches = "true")
|
||||
class PostgresUploadJournalTest
|
||||
@@ -107,6 +112,9 @@ class PostgresUploadJournalTest
|
||||
/*******************************************************************************
|
||||
** Start only the disposable server with explicit durable settings. pgjdbc
|
||||
** announces the JVM zone as the session TimeZone, so the suite runs in UTC.
|
||||
** fsync and full_page_writes are set through ALTER SYSTEM rather than on the
|
||||
** command line, because a command-line value cannot be changed by a reload and
|
||||
** the Aurora and durability cases need to turn each one off and back on.
|
||||
*******************************************************************************/
|
||||
@BeforeAll
|
||||
static void start() throws Exception
|
||||
@@ -115,7 +123,7 @@ class PostgresUploadJournalTest
|
||||
TimeZone.setDefault(TimeZone.getTimeZone("UTC"));
|
||||
postgres = new GenericContainer<>("postgres:17")
|
||||
.withEnv("POSTGRES_PASSWORD", ROOT_PASSWORD).withEnv("POSTGRES_DB", DATABASE)
|
||||
.withCommand("postgres", "-c", "fsync=on", "-c", "synchronous_commit=on", "-c", "full_page_writes=on")
|
||||
.withCommand("postgres", "-c", "synchronous_commit=on")
|
||||
.withExposedPorts(5432)
|
||||
.withCreateContainerCmdModifier(command ->
|
||||
{
|
||||
@@ -138,6 +146,8 @@ class PostgresUploadJournalTest
|
||||
statement.execute("ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO " + APP_ROLE);
|
||||
statement.execute("ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT USAGE, SELECT ON SEQUENCES TO " + APP_ROLE);
|
||||
}
|
||||
serverSetting("fsync", "on");
|
||||
serverSetting("full_page_writes", "on");
|
||||
}
|
||||
|
||||
|
||||
@@ -174,6 +184,9 @@ class PostgresUploadJournalTest
|
||||
execute("ALTER ROLE " + APP_ROLE + " RESET synchronous_commit");
|
||||
execute("ALTER ROLE " + APP_ROLE + " RESET default_transaction_isolation");
|
||||
execute("ALTER ROLE " + APP_ROLE + " RESET default_transaction_read_only");
|
||||
serverSetting("fsync", "on");
|
||||
serverSetting("full_page_writes", "on");
|
||||
execute("DROP FUNCTION IF EXISTS aurora_version()");
|
||||
backend = new PostgreSQLBackendMetaData();
|
||||
backend.setName("postgres_journal");
|
||||
backend.setDatabaseName(DATABASE);
|
||||
@@ -440,6 +453,67 @@ class PostgresUploadJournalTest
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The settings query carries the Aurora detection as its eleventh column. As
|
||||
** the DML-only application role on a stock server it reports false and does not
|
||||
** raise, so the transaction stays usable; once aurora_version() resolves (the
|
||||
** simulated Aurora) it reports true.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void settingsQueryDetectsAuroraWithoutRaising() throws Exception
|
||||
{
|
||||
assertThat(settingsQueryReportsAurora()).isFalse();
|
||||
simulateAurora();
|
||||
assertThat(settingsQueryReportsAurora()).isTrue();
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** full_page_writes=off is refused on stock PostgreSQL and accepted only once
|
||||
** the server is detected as Aurora (simulated); removing the detection refuses
|
||||
** it again, with no row written by a refused call.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void fullPageWritesOffIsAcceptedOnlyOnAurora() throws Exception
|
||||
{
|
||||
serverSetting("full_page_writes", "off");
|
||||
assertUnsupported();
|
||||
assertThat(count("collaboration_upload_journal")).isZero();
|
||||
simulateAurora();
|
||||
ConnectionManager.closeConnectionProviders();
|
||||
assertThat(journal.prepare(candidate("aurora")).id()).isPositive();
|
||||
assertThat(count("collaboration_upload_journal")).isEqualTo(1);
|
||||
execute("DROP FUNCTION aurora_version()");
|
||||
assertUnsupported();
|
||||
assertThat(count("collaboration_upload_journal")).isEqualTo(1);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Aurora (simulated, full_page_writes=off) gets no other allowance: with
|
||||
** synchronous_commit off, or with fsync off, it is refused like any server.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void auroraIsStillRefusedWithoutSynchronousCommitOrFsync() throws Exception
|
||||
{
|
||||
simulateAurora();
|
||||
serverSetting("full_page_writes", "off");
|
||||
assertThat(journal.prepare(candidate("aurora-baseline")).id()).isPositive();
|
||||
execute("ALTER ROLE " + APP_ROLE + " SET synchronous_commit='off'");
|
||||
assertUnsupported();
|
||||
execute("ALTER ROLE " + APP_ROLE + " RESET synchronous_commit");
|
||||
serverSetting("fsync", "off");
|
||||
assertUnsupported();
|
||||
serverSetting("fsync", "on");
|
||||
ConnectionManager.closeConnectionProviders();
|
||||
assertThat(journal.prepare(candidate("aurora-restored")).id()).isPositive();
|
||||
assertThat(count("collaboration_upload_journal")).isEqualTo(2);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Physical drift is refused: an unlogged owner table (lost in crash recovery)
|
||||
** and a non-deterministic collation on a journal column.
|
||||
@@ -565,6 +639,75 @@ class PostgresUploadJournalTest
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Make this stock server look like Amazon Aurora PostgreSQL to the detection:
|
||||
** a function named aurora_version() in the suite's schema. Nothing else about
|
||||
** Aurora is simulated by this call.
|
||||
*******************************************************************************/
|
||||
private void simulateAurora() throws SQLException
|
||||
{
|
||||
execute("CREATE FUNCTION aurora_version() RETURNS text LANGUAGE sql AS 'SELECT ''17.9''::text'");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Run the production settings query as the application role inside one
|
||||
** transaction, then prove that transaction was not aborted by it.
|
||||
*******************************************************************************/
|
||||
private boolean settingsQueryReportsAurora() throws SQLException
|
||||
{
|
||||
try(Connection connection = DriverManager.getConnection(appUrl(), APP_ROLE, APP_PASSWORD); Statement statement = connection.createStatement())
|
||||
{
|
||||
connection.setAutoCommit(false);
|
||||
boolean aurora;
|
||||
try(ResultSet settings = statement.executeQuery(PostgresJournalDatabase.SETTINGS))
|
||||
{
|
||||
assertThat(settings.getMetaData().getColumnCount()).isEqualTo(11);
|
||||
assertThat(settings.next()).isTrue();
|
||||
aurora = settings.getBoolean(11);
|
||||
assertThat(settings.wasNull()).isFalse();
|
||||
}
|
||||
try(ResultSet stillUsable = statement.executeQuery("SELECT 1"))
|
||||
{
|
||||
assertThat(stillUsable.next()).isTrue();
|
||||
}
|
||||
connection.rollback();
|
||||
return (aurora);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Change a reload-level (sighup) setting of the disposable server and wait
|
||||
** until a new session reports it, so the next pooled session observes it too.
|
||||
*******************************************************************************/
|
||||
private static void serverSetting(String name, String value) throws Exception
|
||||
{
|
||||
try(Connection connection = root(); Statement statement = connection.createStatement())
|
||||
{
|
||||
statement.execute("ALTER SYSTEM SET " + name + "=" + value);
|
||||
statement.execute("SELECT pg_reload_conf()");
|
||||
}
|
||||
long deadline = System.nanoTime() + Duration.ofSeconds(30).toNanos();
|
||||
while(System.nanoTime() < deadline)
|
||||
{
|
||||
try(Connection connection = root(); Statement statement = connection.createStatement();
|
||||
ResultSet setting = statement.executeQuery("SELECT current_setting('" + name + "')"))
|
||||
{
|
||||
if(setting.next() && value.equals(setting.getString(1)))
|
||||
{
|
||||
return;
|
||||
}
|
||||
}
|
||||
Thread.sleep(100);
|
||||
}
|
||||
throw (new IllegalStateException("The disposable server did not apply " + name + "=" + value + "."));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Prepare a leased manifest whose stored bytes have been verified by owner.
|
||||
*******************************************************************************/
|
||||
|
||||
@@ -35,14 +35,19 @@ import com.kingsrook.qbits.collaboration.PersonalRecords;
|
||||
import com.kingsrook.qbits.collaboration.PersonalServiceFixture;
|
||||
import com.kingsrook.qbits.collaboration.favorites.FavoriteEntry;
|
||||
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
|
||||
import com.kingsrook.qqq.backend.core.actions.permissions.CustomPermissionChecker;
|
||||
import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
|
||||
import com.kingsrook.qqq.backend.core.instances.QInstanceValidator;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.count.CountInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
|
||||
@@ -51,11 +56,17 @@ import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.audits.AuditLevel;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.permissions.MetaDataWithPermissionRules;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.permissions.PermissionLevel;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRules;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.security.FieldSecurityLock;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.security.QSecurityKeyType;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.scheduleing.simple.SimpleSchedulerMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.session.QSession;
|
||||
import com.kingsrook.qqq.backend.core.model.session.QSystemUserSession;
|
||||
import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryBackendModule;
|
||||
@@ -158,6 +169,70 @@ class RecentsServiceTest extends PersonalServiceFixture
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A table whose permission checker goes by the action input it is handed (a
|
||||
** query, a get or a count reads; anything else is refused as a write) is a
|
||||
** target for whoever that checker lets read it, and for nobody else: the
|
||||
** visibility check asks as the query it stands for, never with an untyped
|
||||
** input the checker cannot recognise.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void tableWithAnInputTypedPermissionCheckerIsATarget() throws Exception
|
||||
{
|
||||
instance.addTable(new QTableMetaData().withName("ledger").withLabel("Ledger").withBackendName("memory").withPrimaryKeyField("id")
|
||||
.withPermissionRules(new QPermissionRules().withLevel(PermissionLevel.HAS_ACCESS_PERMISSION)
|
||||
.withCustomPermissionChecker(new QCodeReference(ReadsByInputType.class)))
|
||||
.withField(new QFieldMetaData("id", QFieldType.INTEGER))
|
||||
.withField(new QFieldMetaData("title", QFieldType.STRING))
|
||||
.withRecordLabelFormatAndFields("%s", "title"));
|
||||
InsertInput seed = new InsertInput("ledger").withRecords(List.of(new QRecord().withValue("id", 7).withValue("title", "Seventh entry")));
|
||||
seed.withPermissionBypass("Test fixture seeds a row of a table nobody may write.");
|
||||
assertThat(new InsertAction().execute(seed).getRecords()).allSatisfy(row -> assertThat(row.getErrors()).isNullOrEmpty());
|
||||
|
||||
login("alice", ReadsByInputType.PERMISSION);
|
||||
at(START);
|
||||
RecentsService.recordView("TABLE", "ledger", null);
|
||||
at(START.plusSeconds(1));
|
||||
RecentsService.recordView("RECORD", "ledger", "7");
|
||||
assertThat(RecentsService.list(50)).extracting(FavoriteEntry::label).containsExactly("Seventh entry", "Ledger");
|
||||
|
||||
login("bob");
|
||||
assertThatThrownBy(() -> RecentsService.recordView("TABLE", "ledger", null)).hasMessage("Not found");
|
||||
assertThatThrownBy(() -> RecentsService.recordView("RECORD", "ledger", "7")).hasMessage("Not found");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A checker of the kind hosts and other qbits write for read-only tables: QQQ
|
||||
** hands it the action input and not the permission asked, so it reads the
|
||||
** input's type. A query, a get and a count need its permission; every other
|
||||
** input is refused to everyone.
|
||||
*******************************************************************************/
|
||||
public static class ReadsByInputType implements CustomPermissionChecker
|
||||
{
|
||||
static final String PERMISSION = "ledger.view";
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Refuse whatever is not a recognised read, then require the permission.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
public void checkPermissionsThrowing(AbstractActionInput actionInput, MetaDataWithPermissionRules metaDataWithPermissionRules) throws QPermissionDeniedException
|
||||
{
|
||||
if(!(actionInput instanceof QueryInput || actionInput instanceof GetInput || actionInput instanceof CountInput))
|
||||
{
|
||||
throw (new QPermissionDeniedException("This table is read-only."));
|
||||
}
|
||||
if(!QContext.getQSession().hasPermission(PERMISSION))
|
||||
{
|
||||
throw (new QPermissionDeniedException("Permission denied."));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Newest-first ordering is stable; locked rows are dropped before the limit.
|
||||
*******************************************************************************/
|
||||
|
||||
Reference in new issue
Block a user