From 551c4c6f87581662a629520ace4fb9b1ef1fdc6c Mon Sep 17 00:00:00 2001 From: James Maes Date: Tue, 17 Nov 2026 13:38:31 -0600 Subject: [PATCH 1/2] fix(presence): the record features summary no longer fails when presence is unavailable CollaborationSummaryProvider let the shared presence backend's service-unavailable answer escape, so core's record features summary (GET /table/{tableName}/{primaryKey}/features) answered 503 as a whole: counts, favorite, follow, owner and assignee went with it. With the Redis backend that happened for one lease lifetime after the first touch of each table's room in a store that had lost its state (the room is "recovering"), and for as long as the store was unreachable. The summary now catches only that case, builds every other part as usual, leaves presence unset and reports extensions.presence.status = "unavailable". The wire omits empty collections, so an empty and an unset presence list look the same; the extension is what tells them apart. The presence endpoints (/presence, /presence/leases, /presence/leases/renew) are unchanged and keep answering 503. The Who's here card had the same shape: it now says presence is temporarily unavailable instead of failing. It also stopped at the first recovering room, so it needed one visit per table, a lease lifetime apart, before it could show counts; a recovering room no longer stops the reading (any other failure still stops it at once), so one visit starts every room's interval. The recovery guard is unchanged. RedisPresenceRecoveryTest .normalUseNeverRestartsRecovery shows that normal use cannot restart the interval: every write that advances the guard's latest issued expiry also leaves a table member (a departure marker included) expiring at exactly that instant, so the room is never empty before it. Open and leave within the lease lifetime, an abandoned reservation and idle expiry all end in a known-empty room under the same generation; only real loss of the projections starts a new one. Co-Authored-By: Claude Fable 5.2 --- CHANGELOG.md | 1 + docs/PRESENCE-RECOVERY.md | 34 ++++++ .../CollaborationSummaryProvider.java | 50 ++++++-- .../collaboration/app/CollaborationHome.java | 55 +++++++-- .../presence/RedisPresenceBackend.java | 12 +- .../CollaborationApiHttpTest.java | 76 ++++++++++++ .../RedisPresenceRecoveryTest.java | 113 ++++++++++++++++++ .../RedisPresenceServiceTest.java | 11 +- 8 files changed, 331 insertions(+), 21 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e949416..1ae573e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -29,6 +29,7 @@ All notable changes are documented here, following [Keep a Changelog](https://ke ### Fixed +- The record features summary (`GET /table/{tableName}/{primaryKey}/features`) no longer fails when shared presence is recovering or unreachable. `CollaborationSummaryProvider` let the presence backend's service-unavailable answer escape, so the whole summary answered `503` (counts, favorite, follow, owner and assignee with it) for one lease lifetime after the first touch of each table's room in a store that had lost its state, and for as long as the store was unreachable. The summary now answers with every other part intact, leaves `presence` unset and reports `extensions.presence.status = "unavailable"`; an empty or absent `presence` without that extension still means a room that was read. The Who's here card says presence is temporarily unavailable instead of failing, and one visit now starts the recovery interval of every table's room (it stopped at the first recovering room, so the card needed one visit per table, a lease lifetime apart, before it could show counts). The presence endpoints (`/presence`, `/presence/leases`, `/presence/leases/renew`) are unchanged and keep answering `503`. - Favorites and recents accept a table whose custom permission checker tells a read from a write by the type of the action input. `TargetResolver` asked whether a table is readable with an untyped `AbstractActionInput`; such a checker refuses an input it does not recognise, so `POST /recents` (which the UI sends on every list and record screen) and starring answered `404 Not found` for people the checker lets read the table. The read is now asked as the query it stands for, as `ParentRecordAccess` already asks with a get. Who may read a table is unchanged. - Reads of a table that has no tag assignment no longer query `collaboration_record_tag`. Core selects the Tags field for every query or get that names no fields, so each of them (including per-record gets and the stored-record read before an update) ran one assignment statement whether or not the table had ever been tagged: 600 gets of an untagged table ran 600 statements, now none. Each node remembers which tables have no assignment (`TaggedTables`, core `Memoization` registered with `ClusterCaches`) for `CollaborationQBitConfig.withTagLookupCacheSeconds` (default 30, 0 disables); writing an assignment clears it on the node and through the ClusterBus. Tables with assignments are read as before, one lookup per 500 parents. No API or schema change. See "What the Tags field costs a read" in the README. - Browsing one of the qbit's child tables (comments, tasks, record tags, attachments and the other parent-referencing tables) reads each distinct parent once per result page to authorize its rows, instead of once per row: 61 tasks on two parents ran 61 parent reads, now 2. The check itself and its outcome are unchanged. diff --git a/docs/PRESENCE-RECOVERY.md b/docs/PRESENCE-RECOVERY.md index 43035ae..186d51b 100644 --- a/docs/PRESENCE-RECOVERY.md +++ b/docs/PRESENCE-RECOVERY.md @@ -28,6 +28,40 @@ later empty snapshots can be known-empty. The guard has no TTL; tenant-retiremen operations and long-term guard storage must be included in operational capacity qualification. +## Scope of the unavailable answer + +Recovery and store failures are answered as service-unavailable only by the +presence endpoints (`/presence`, `/presence/leases`, `/presence/leases/renew`). +Responses that merely include presence do not fail with it. The core record +features summary is built as usual, leaves `presence` unset and carries +`extensions.presence.status = "unavailable"`; without that extension an empty +or absent `presence` means a room that was read. The Who's here card shows a +sentence instead of counts, never partial or node-local counts. The card reads +every table's room; a recovering room does not stop that reading, so one visit +starts the interval of every room, while any other failure stops it at once. + +## When the interval starts + +The interval starts on the first touch of a table's room when its guard is +missing, or when the table projections are empty before the guard's latest +issued expiry. Normal use cannot produce the second case: every write that +advances that expiry (reservation, renewal, legacy heartbeat or departure) +also leaves a table member expiring at exactly that instant (a departure +leaves a marker, it does not remove the member), and the key lifetimes are set +in the same script. A person who opens a table and leaves, a +reservation that is never activated and ordinary idle expiry all end in a +known-empty room under the same generation +(`RedisPresenceRecoveryTest.normalUseNeverRestartsRecovery`). + +The first touch of a table that has no guard still starts an interval, even in +a store that has never held presence. An empty store that never had viewers and +a store that just lost its viewers are the same bytes: anything that could tell +them apart lives in the same keyspace and is lost with it, and a marker shared +by all tables would sit in another hash slot than the table's room, outside +what one script may read on Redis Cluster. Treating a missing guard as +known-empty would report a viewer who is still on a page as absent after a +loss, which is what the guard exists to prevent. + ## Controlled restore The host configuration has a bounded `recoveryGeneration`, default `initial`, diff --git a/src/main/java/com/kingsrook/qbits/collaboration/CollaborationSummaryProvider.java b/src/main/java/com/kingsrook/qbits/collaboration/CollaborationSummaryProvider.java index c0e20a3..2712d54 100644 --- a/src/main/java/com/kingsrook/qbits/collaboration/CollaborationSummaryProvider.java +++ b/src/main/java/com/kingsrook/qbits/collaboration/CollaborationSummaryProvider.java @@ -21,23 +21,35 @@ package com.kingsrook.qbits.collaboration; +import java.io.Serializable; +import java.util.LinkedHashMap; +import java.util.Map; import com.kingsrook.qbits.collaboration.model.CollaborationTables; +import com.kingsrook.qbits.collaboration.presence.PresenceService; import com.kingsrook.qqq.backend.core.actions.collaboration.RecordFeatureSummaryProviderInterface; import com.kingsrook.qqq.backend.core.actions.collaboration.RecordFeaturesResolver; import com.kingsrook.qqq.backend.core.context.QContext; import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.exceptions.QServiceUnavailableException; +import com.kingsrook.qqq.backend.core.logging.QLogger; import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator; import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter; import com.kingsrook.qqq.backend.core.model.data.QRecord; import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordFeature; import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordFeatureSummary; import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordReference; +import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair; /******************************************************************************* ** Caller-scoped values for the core record features endpoint. *******************************************************************************/ public class CollaborationSummaryProvider implements RecordFeatureSummaryProviderInterface { + public static final String PRESENCE_EXTENSION = "presence"; + public static final String PRESENCE_UNAVAILABLE = "unavailable"; + + private static final QLogger LOG = QLogger.getLogger(CollaborationSummaryProvider.class); + /******************************************************************************* ** Core has already checked the parent; omit values for disabled features. *******************************************************************************/ @@ -49,15 +61,7 @@ public class CollaborationSummaryProvider implements RecordFeatureSummaryProvide String user = PersonalRecords.currentUserId(); if(features.isEnabled(RecordFeature.PRESENCE)) { - summary.setPresence(new com.kingsrook.qbits.collaboration.presence.PresenceService().snapshot(ref.tableName(), id).stream().map(viewer -> - { - java.util.Map entry = new java.util.LinkedHashMap<>(); - entry.put("userId", viewer.userId()); - entry.put("displayName", viewer.displayName()); - entry.put("mode", viewer.mode()); - entry.put("fieldName", viewer.fieldName()); - return (entry); - }).toList()); + contributePresence(ref.tableName(), id, summary); } if(features.isEnabled(RecordFeature.FAVORITES)) { @@ -110,4 +114,32 @@ public class CollaborationSummaryProvider implements RecordFeatureSummaryProvide } } } + + /******************************************************************************* + ** Presence is advisory: a recovering or unreachable shared store must not fail + ** the whole summary. The list stays unset, because an empty list means a room + ** that was read and is empty, and the provider extension says why. Presence's + ** own endpoints keep answering service-unavailable. + *******************************************************************************/ + private static void contributePresence(String tableName, String id, RecordFeatureSummary summary) throws QException + { + try + { + summary.setPresence(new PresenceService().snapshot(tableName, id).stream().map(viewer -> + { + Map entry = new LinkedHashMap<>(); + entry.put("userId", viewer.userId()); + entry.put("displayName", viewer.displayName()); + entry.put("mode", viewer.mode()); + entry.put("fieldName", viewer.fieldName()); + return (entry); + }).toList()); + } + catch(QServiceUnavailableException unavailable) + { + summary.setPresence(null); + summary.getExtensions().put(PRESENCE_EXTENSION, new LinkedHashMap<>(Map.of("status", PRESENCE_UNAVAILABLE))); + LOG.debug("Presence is unavailable; the record features summary reports it as unavailable", logPair("tableName", tableName)); + } + } } diff --git a/src/main/java/com/kingsrook/qbits/collaboration/app/CollaborationHome.java b/src/main/java/com/kingsrook/qbits/collaboration/app/CollaborationHome.java index 086c03e..aef90da 100644 --- a/src/main/java/com/kingsrook/qbits/collaboration/app/CollaborationHome.java +++ b/src/main/java/com/kingsrook/qbits/collaboration/app/CollaborationHome.java @@ -87,6 +87,7 @@ public final class CollaborationHome public static final String NO_TAGS = "No tags yet. Tags appear here once someone creates one on a record."; public static final String NO_FIELDS = "No custom fields yet. Define one under Custom fields."; public static final String PRESENCE_ADMIN_ONLY = "Who's here is visible to collaboration administrators."; + public static final String PRESENCE_UNAVAILABLE = "Who's here is temporarily unavailable. Try again shortly."; @@ -277,16 +278,9 @@ public final class CollaborationHome else { counts = new java.util.TreeMap<>(); - var backend = new com.kingsrook.qbits.collaboration.presence.RedisPresenceBackend(redis); - // Read only known table rooms in the administrator's authenticated tenant. - // Never scan Redis keys or silently substitute local counts on failure. - for(String tableName : instance.getTables().keySet()) + if(!readSharedCounts(instance, redis, counts)) { - int viewers = backend.snapshot(tableName, null).size(); - if(viewers > 0) - { - counts.put(tableName, viewers); - } + return (table("Who's here", PRESENCE_UNAVAILABLE, List.of(), column("Table", "table", "left"), column("People", "viewers", "right"))); } } counts.forEach((table, viewers) -> @@ -302,6 +296,49 @@ public final class CollaborationHome + /******************************************************************************* + ** Read only known table rooms in the administrator's authenticated tenant; + ** never scan Redis keys or substitute node-local counts. False means the + ** counts are incomplete and must not be shown. A recovering room does not + ** stop the reading, so every room's interval starts on the same visit instead + ** of one room per visit; any other failure stops it at once. + *******************************************************************************/ + private static boolean readSharedCounts(com.kingsrook.qqq.backend.core.model.metadata.QInstance instance, + com.kingsrook.qbits.collaboration.presence.RedisPresenceConfig redis, Map counts) + { + boolean complete = true; + try + { + var backend = new com.kingsrook.qbits.collaboration.presence.RedisPresenceBackend(redis); + for(String tableName : instance.getTables().keySet()) + { + try + { + int viewers = backend.snapshot(tableName, null).size(); + if(viewers > 0) + { + counts.put(tableName, viewers); + } + } + catch(com.kingsrook.qqq.backend.core.exceptions.QServiceUnavailableException unavailable) + { + if(!com.kingsrook.qbits.collaboration.presence.RedisPresenceBackend.isRecovering(unavailable)) + { + throw (unavailable); + } + complete = false; + } + } + } + catch(com.kingsrook.qqq.backend.core.exceptions.QServiceUnavailableException unavailable) + { + complete = false; + } + return (complete); + } + + + /******************************************************************************* ** Assemble a table widget whose empty state is a sentence, not a blank grid. *******************************************************************************/ diff --git a/src/main/java/com/kingsrook/qbits/collaboration/presence/RedisPresenceBackend.java b/src/main/java/com/kingsrook/qbits/collaboration/presence/RedisPresenceBackend.java index 2bdb5cf..52d6c0e 100644 --- a/src/main/java/com/kingsrook/qbits/collaboration/presence/RedisPresenceBackend.java +++ b/src/main/java/com/kingsrook/qbits/collaboration/presence/RedisPresenceBackend.java @@ -32,6 +32,8 @@ import com.kingsrook.qqq.backend.core.exceptions.QServiceUnavailableException; *******************************************************************************/ public class RedisPresenceBackend implements PresenceBackend { + private static final String RECOVERING = "Presence is recovering; active viewers are rejoining. Please retry."; + private final RedisPresenceStore store; private final String audience; @@ -147,7 +149,7 @@ public class RedisPresenceBackend implements PresenceBackend } catch(PresenceRecoveringException recovering) { - throw (new QServiceUnavailableException("Presence is recovering; active viewers are rejoining. Please retry.")); + throw (new QServiceUnavailableException(RECOVERING)); } catch(RuntimeException failure) { @@ -155,6 +157,14 @@ public class RedisPresenceBackend implements PresenceBackend } } + /*************************************************************************** + ** Recovery ends by itself after one lease lifetime; other failures do not. + ***************************************************************************/ + public static boolean isRecovering(QServiceUnavailableException failure) + { + return (RECOVERING.equals(failure.getMessage())); + } + /*************************************************************************** ** Never expose client exceptions, endpoint details or credentials to browsers. ***************************************************************************/ diff --git a/src/test/java/com/kingsrook/qbits/collaboration/CollaborationApiHttpTest.java b/src/test/java/com/kingsrook/qbits/collaboration/CollaborationApiHttpTest.java index 41b839a..a96495b 100644 --- a/src/test/java/com/kingsrook/qbits/collaboration/CollaborationApiHttpTest.java +++ b/src/test/java/com/kingsrook/qbits/collaboration/CollaborationApiHttpTest.java @@ -138,6 +138,13 @@ class CollaborationApiHttpTest extends PersonalServiceFixture @Test void unavailablePresenceHttp() throws Exception { + String tab = "{\"connectionId\":\"tab\",\"table\":\"document\",\"pk\":\"1\",\"mode\":\"VIEWING\"}"; + ok("POST", "/presence", tab); + JsonNode local = json(nativeGet("/table/document/1/features", "alice"), 200); + assertThat(local.path("presence")).hasSize(1); + assertThat(local.at("/presence/0/userId").asText()).isEqualTo("alice"); + assertThat(local.path("extensions").has(CollaborationSummaryProvider.PRESENCE_EXTENSION)).isFalse(); + ok("DELETE", "/presence", "{\"connectionId\":\"tab\",\"table\":\"document\",\"pk\":\"1\"}"); config.withRedisPresence(new com.kingsrook.qbits.collaboration.presence.RedisPresenceConfig() .withConnectionName("not-started").withDeployment("test").withApplication("fixture") .withScopeProvider(new QCodeReference(RedisPresenceServiceTest.TenantScope.class))); @@ -147,6 +154,75 @@ class CollaborationApiHttpTest extends PersonalServiceFixture assertError(400, "POST", "/presence", "{\"connectionId\":\"tab\",\"table\":\"document\",\"pk\":\"1\",\"mode\":\"VIEWING\"}", "alice"); assertError(503, "DELETE", "/presence", "{\"connectionId\":\"tab\",\"table\":\"document\",\"pk\":\"1\"}", "alice"); assertError(404, "GET", "/presence?table=document&pk=3", null, "alice"); + assertFeaturesReportPresenceUnavailable(0, 0); + assertThat(nativeGet("/table/document/3/features", "alice").statusCode()).isEqualTo(404); + } + + /*************************************************************************** + ** Presence trouble never fails the core record features summary: the other + ** parts are served and presence is reported unavailable, while presence's + ** own endpoint keeps answering service-unavailable. + ***************************************************************************/ + @Test + void recordFeaturesSurvivePresenceRecoveryAndFailure() throws Exception + { + ok("POST", "/records/document/1/comments", "{\"body\":\"First\"}"); + ok("POST", "/records/document/1/tasks", "{\"title\":\"Review\"}"); + try(var redis = RedisFixtures.redis()) + { + redis.start(); + new com.kingsrook.qbits.redis.RedisQBitProducer().withQBitConfig(new com.kingsrook.qbits.redis.RedisQBitConfig() + .withConnection("presence", new com.kingsrook.qbits.redis.RedisConnectionConfig() + .withHost(RedisFixtures.host(redis)).withPort(RedisFixtures.port(redis)).withTls(false) + .withConnectionTimeoutMillis(200).withSocketTimeoutMillis(1000))).produce(instance); + config.withRedisPresence(new com.kingsrook.qbits.collaboration.presence.RedisPresenceConfig() + .withConnectionName("presence").withDeployment("test").withApplication("features-summary") + .withScopeProvider(new QCodeReference(RedisPresenceServiceTest.TenantScope.class))); + var runtime = new com.kingsrook.qbits.redis.RedisRuntimeService(); + try + { + runtime.start(instance); + // The first touch of a room in a store with no state is recovering. + assertFeaturesReportPresenceUnavailable(1, 1); + assertThat(json(request("GET", "/presence?table=document&pk=1", null, "alice"), 503).path("message").asText()).contains("recovering"); + admin(); + var recovering = com.kingsrook.qbits.collaboration.app.CollaborationHome.presence(); + assertThat(recovering.getRows()).isEmpty(); + assertThat(recovering.getNoRowsFoundHTML()).isEqualTo(com.kingsrook.qbits.collaboration.app.CollaborationHome.PRESENCE_UNAVAILABLE); + // One visit starts the interval of every table's room, not one room per visit. + assertThat(com.kingsrook.qbits.redis.RedisRuntimeService.connection(instance, "presence").client().keys("*:recovery")) + .hasSize(instance.getTables().size()); + redis.stop(); + assertFeaturesReportPresenceUnavailable(1, 1); + assertThat(json(request("GET", "/presence?table=document&pk=1", null, "alice"), 503).path("message").asText()).contains("temporarily unavailable"); + } + finally + { + runtime.stop(); + runtime.afterApplicationStop(); + } + } + } + + /*************************************************************************** + ** The summary answers 200 with every other part intact; presence is unset + ** and the provider extension says it is unavailable. + ***************************************************************************/ + private void assertFeaturesReportPresenceUnavailable(int openComments, int openTasks) throws Exception + { + JsonNode summary = json(nativeGet("/table/document/1/features", "alice"), 200); + assertThat(summary.hasNonNull("presence")).isFalse(); + assertThat(summary.at("/extensions/" + CollaborationSummaryProvider.PRESENCE_EXTENSION + "/status").asText()).isEqualTo(CollaborationSummaryProvider.PRESENCE_UNAVAILABLE); + assertThat(summary.path("tableName").asText()).isEqualTo("document"); + assertThat(summary.path("primaryKey").asText()).isEqualTo("1"); + assertThat(summary.path("features").toString()).contains("PRESENCE").contains("COMMENTS"); + assertThat(summary.at("/counts/openComments").asInt(-1)).isEqualTo(openComments); + assertThat(summary.at("/counts/openTasks").asInt(-1)).isEqualTo(openTasks); + assertThat(summary.at("/counts/attachments").asInt(-1)).isZero(); + assertThat(summary.at("/counts/tags").asInt(-1)).isZero(); + assertThat(summary.path("isFavorite").isBoolean()).isTrue(); + assertThat(summary.path("isFollowing").isBoolean()).isTrue(); + assertThat(summary.path("capabilities").size()).isPositive(); } private static final ObjectMapper JSON = new ObjectMapper(); diff --git a/src/test/java/com/kingsrook/qbits/collaboration/RedisPresenceRecoveryTest.java b/src/test/java/com/kingsrook/qbits/collaboration/RedisPresenceRecoveryTest.java index 7143023..e299fe3 100644 --- a/src/test/java/com/kingsrook/qbits/collaboration/RedisPresenceRecoveryTest.java +++ b/src/test/java/com/kingsrook/qbits/collaboration/RedisPresenceRecoveryTest.java @@ -47,6 +47,8 @@ import static org.assertj.core.api.Assertions.assertThatThrownBy; *******************************************************************************/ class RedisPresenceRecoveryTest extends PersonalServiceFixture { + private static final com.fasterxml.jackson.databind.ObjectMapper JSON = new com.fasterxml.jackson.databind.ObjectMapper(); + /*************************************************************************** ** Peer reads cannot prolong cold recovery; normal expiry stays known-empty. ***************************************************************************/ @@ -209,6 +211,117 @@ class RedisPresenceRecoveryTest extends PersonalServiceFixture } + /*************************************************************************** + ** Normal use cannot restart recovery. Every write that advances the guard's + ** latest issued expiry also leaves a member (a departure marker included) + ** expiring at exactly that instant, so the table room is never empty before + ** it. Only real loss of the projections starts a new interval. + ***************************************************************************/ + @Test + void normalUseNeverRestartsRecovery() throws Exception + { + try(var redis = RedisFixtures.redis()) + { + redis.start(); + var address = new HostAndPort(RedisFixtures.host(redis), RedisFixtures.port(redis)); + var config = DefaultJedisClientConfig.builder().build(); + try(var first = new RedisConnection(address, config, 2, Duration.ofMillis(500)); + var second = new RedisConnection(address, config, 2, Duration.ofMillis(500))) + { + var namespace = new RedisNamespace("test", "no-rearm", "tenant", "presence"); + var a = new RedisPresenceStore(first.scripts(), namespace, Duration.ofSeconds(2), 10); + var b = new RedisPresenceStore(second.scripts(), namespace, Duration.ofSeconds(2), 10); + assertThatThrownBy(() -> a.snapshot("document", null)).isInstanceOf(PresenceRecoveringException.class); + String guardKey = first.client().keys("*:recovery").iterator().next(); + var warm = JSON.readTree(first.client().get(guardKey)); + String generation = warm.path("generation").asText(); + long readyAt = warm.path("readyAt").asLong(); + long deadline = System.nanoTime() + Duration.ofSeconds(6).toNanos(); + while(true) + { + try + { + assertThat(b.snapshot("document", null)).isEmpty(); + break; + } + catch(PresenceRecoveringException recovering) + { + assertThat(System.nanoTime()).isLessThan(deadline); + Thread.sleep(50); + } + } + + // One person opens the table and leaves within the lease lifetime; + // the next visitor reads a known-empty room, not a recovering one. + String tableLease = a.openLease("alice", "Alice", "document", null); + assertGuardCoversTableRoom(first, guardKey, generation, readyAt); + a.renewLease("alice", "Alice", tableLease, "document", null, "VIEWING", null, 1L); + assertThat(b.snapshot("document", null)).extracting("userId").containsExactly("alice"); + a.leave("alice", tableLease, "document", null, 2L); + assertThat(b.snapshot("document", null)).isEmpty(); + assertThat(b.snapshot("document", "7")).isEmpty(); + assertGuardCoversTableRoom(first, guardKey, generation, readyAt); + + // The same holds for a record lease, a legacy connection and a + // reservation that is never activated. + String recordLease = b.openLease("bob", "Bob", "document", "1"); + b.renewLease("bob", "Bob", recordLease, "document", "1", "EDITING", "title", 1L); + b.leave("bob", recordLease, "document", "1", 2L); + assertThat(a.snapshot("document", "1")).isEmpty(); + assertGuardCoversTableRoom(first, guardKey, generation, readyAt); + a.heartbeat("carol", "Carol", "tab", "document", "2", "VIEWING", null, 1L); + a.leave("carol", "tab", "document", "2", 2L); + assertThat(b.snapshot("document", null)).isEmpty(); + assertGuardCoversTableRoom(first, guardKey, generation, readyAt); + assertThat(a.openLease("dave", "Dave", "document", "3")).isNotNull(); + assertGuardCoversTableRoom(first, guardKey, generation, readyAt); + + // Read continuously while every marker and reservation expires: the + // boundary between "markers present" and "idle" is never recovering. + int reads = 0; + deadline = System.nanoTime() + Duration.ofSeconds(6).toNanos(); + while(first.client().keys("*").size() > 1) + { + assertThat(System.nanoTime()).isLessThan(deadline); + assertThat(a.snapshot("document", null)).isEmpty(); + assertThat(b.snapshot("document", "1")).isEmpty(); + reads++; + } + assertThat(reads).isPositive(); + assertThat(a.snapshot("document", null)).isEmpty(); + assertThat(b.snapshot("document", "1")).isEmpty(); + var idle = JSON.readTree(first.client().get(guardKey)); + assertThat(idle.path("generation").asText()).isEqualTo(generation); + assertThat(idle.path("readyAt").asLong()).isEqualTo(readyAt); + + // Contrast: projections lost before the latest issued expiry are real + // loss, and that does start a new generation and interval. + assertThat(a.openLease("erin", "Erin", "document", "1")).isNotNull(); + first.client().del(first.client().keys("*").stream().filter(key -> !key.endsWith(":recovery")).toArray(String[]::new)); + assertThatThrownBy(() -> b.snapshot("document", null)).isInstanceOf(PresenceRecoveringException.class); + assertThat(JSON.readTree(first.client().get(guardKey)).path("generation").asText()).isNotEqualTo(generation); + } + } + } + + + /*************************************************************************** + ** The invariant that rules out a restart: same generation and readiness, and + ** the newest table member expires exactly at the guard's latest issued expiry. + ***************************************************************************/ + private void assertGuardCoversTableRoom(RedisConnection connection, String guardKey, String generation, long readyAt) throws Exception + { + var guard = JSON.readTree(connection.client().get(guardKey)); + assertThat(guard.path("generation").asText()).isEqualTo(generation); + assertThat(guard.path("readyAt").asLong()).isEqualTo(readyAt); + String index = guardKey.substring(0, guardKey.length() - "recovery".length()) + "table:expiry"; + var newest = connection.client().zrangeWithScores(index, -1, -1); + assertThat(newest).hasSize(1); + assertThat((long) newest.getFirst().getScore()).isEqualTo(guard.path("liveUntil").asLong()); + assertThat(connection.client().pttl(index)).isPositive(); + } + + /*************************************************************************** ** Restore must happen in maintenance, followed by a fresh host-owned generation. ** Replaying both old guard and lease bytes cannot authorize the new deployment. diff --git a/src/test/java/com/kingsrook/qbits/collaboration/RedisPresenceServiceTest.java b/src/test/java/com/kingsrook/qbits/collaboration/RedisPresenceServiceTest.java index acbef84..e3dc35f 100644 --- a/src/test/java/com/kingsrook/qbits/collaboration/RedisPresenceServiceTest.java +++ b/src/test/java/com/kingsrook/qbits/collaboration/RedisPresenceServiceTest.java @@ -309,8 +309,15 @@ class RedisPresenceServiceTest extends PersonalServiceFixture assertThatThrownBy(() -> service.heartbeat("tab", "document", "3", "VIEWING", null, 1L)) .isInstanceOf(QNotFoundException.class); admin(); - assertThatThrownBy(com.kingsrook.qbits.collaboration.app.CollaborationHome::presence) - .isInstanceOf(QServiceUnavailableException.class); + var unavailable = com.kingsrook.qbits.collaboration.app.CollaborationHome.presence(); + assertThat(unavailable.getRows()).isEmpty(); + assertThat(unavailable.getNoRowsFoundHTML()).isEqualTo(com.kingsrook.qbits.collaboration.app.CollaborationHome.PRESENCE_UNAVAILABLE); + var summary = com.kingsrook.qqq.backend.core.actions.collaboration.RecordFeatureSummaryHelper.summarize(instance.getTable("document"), + com.kingsrook.qbits.collaboration.security.ParentRecordAccess.requireVisibleParent("document", 1)); + assertThat(summary.getPresence()).isNull(); + assertThat(summary.getExtensions().get(CollaborationSummaryProvider.PRESENCE_EXTENSION)) + .isEqualTo(java.util.Map.of("status", CollaborationSummaryProvider.PRESENCE_UNAVAILABLE)); + assertThat(summary.getCounts()).containsKeys("openComments", "openTasks", "tags", "attachments", "followers"); } finally { -- 2.54.0 From c28f839658bbf476fc9575bb72d7ec1ee9d72c2f Mon Sep 17 00:00:00 2001 From: James Maes Date: Tue, 17 Nov 2026 13:54:31 -0600 Subject: [PATCH 2/2] fix(search): no tag or custom field matches, instead of a refused search, without read of their tables The Tags field is a search field of every taggable table, and so is each searchable custom field kept in the value table. Core asks each of them to expand the search term; the expansion is a sub-query on collaboration_record_tag and collaboration_tag (or collaboration_field_value). With the default rules every reader may query those tables, but a host can require a permission of its own on them. Core checks table READ for each sub-query table and one refusal refuses the whole query, so every record search of such a table answered 403 for a person without that permission, whatever the text. TagCriteriaExpander.expandSearchTerm and FieldCriteriaExpander.expandSearchTerm now ask core the same question first (PermissionsHelper.hasTablePermission with an enforced QueryInput for the table, the check core applies to a sub-query) and return null when the session may not query the tables. Core's contract for null is "this field does not match the term": the other search fields of the table still match, and a #tag search matches nothing. Nobody gains access. An explicit Tags or custom field criterion on a record list is unchanged: it is still refused by core for such a person. A silent empty result would be wrong for the negative operators (is not, is blank) and misleading for the positive ones, and the refusal already names the missing permission. A person who may query the tables searches and filters as before. SearchWithoutCollaborationReadTest (memory backend, materialized sub-queries) and CustomFieldValuesPostgresTest .searchSurvivesHostProtectedCollaborationTables (PostgreSQL, native sub-queries) put a host permission in front of each table's own checker: without it a text search returns the table's own matches, tag and custom field terms return nothing, nothing throws and the explicit criterion is refused; with it the results are the same as with the default rules. Both fail before this change with QPermissionDeniedException on the first search. Co-Authored-By: Claude Fable 5.2 --- CHANGELOG.md | 1 + .../customfields/FieldCriteriaExpander.java | 7 + .../security/ParentRecordAccess.java | 23 ++ .../tags/TagCriteriaExpander.java | 8 + .../CustomFieldValuesPostgresTest.java | 84 ++++++ .../SearchWithoutCollaborationReadTest.java | 250 ++++++++++++++++++ 6 files changed, 373 insertions(+) create mode 100644 src/test/java/com/kingsrook/qbits/collaboration/SearchWithoutCollaborationReadTest.java diff --git a/CHANGELOG.md b/CHANGELOG.md index 1ae573e..826188b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -29,6 +29,7 @@ All notable changes are documented here, following [Keep a Changelog](https://ke ### Fixed +- Record search no longer fails for a person who may read a table but may not query the tag or custom field value tables. The Tags field (and each searchable custom field kept in the value table) is a search field of its table, and its search-term expansion is a sub-query on `collaboration_record_tag` and `collaboration_tag` (or `collaboration_field_value`). With the default rules every reader may query those tables, but a host can require a permission of its own on them, and core refuses a whole search for one refused sub-query table: every search of such a table answered `403`, whatever the text. `TagCriteriaExpander.expandSearchTerm` and `FieldCriteriaExpander.expandSearchTerm` now ask core whether the session may query those tables (the table READ check core applies to a sub-query) and contribute nothing when it may not, so the table's other search fields still match, and `#tag` matches nothing. Nobody gains access: an explicit Tags or custom field criterion on a record list is still refused for such a person, and a person who may query the tables searches as before. - The record features summary (`GET /table/{tableName}/{primaryKey}/features`) no longer fails when shared presence is recovering or unreachable. `CollaborationSummaryProvider` let the presence backend's service-unavailable answer escape, so the whole summary answered `503` (counts, favorite, follow, owner and assignee with it) for one lease lifetime after the first touch of each table's room in a store that had lost its state, and for as long as the store was unreachable. The summary now answers with every other part intact, leaves `presence` unset and reports `extensions.presence.status = "unavailable"`; an empty or absent `presence` without that extension still means a room that was read. The Who's here card says presence is temporarily unavailable instead of failing, and one visit now starts the recovery interval of every table's room (it stopped at the first recovering room, so the card needed one visit per table, a lease lifetime apart, before it could show counts). The presence endpoints (`/presence`, `/presence/leases`, `/presence/leases/renew`) are unchanged and keep answering `503`. - Favorites and recents accept a table whose custom permission checker tells a read from a write by the type of the action input. `TargetResolver` asked whether a table is readable with an untyped `AbstractActionInput`; such a checker refuses an input it does not recognise, so `POST /recents` (which the UI sends on every list and record screen) and starring answered `404 Not found` for people the checker lets read the table. The read is now asked as the query it stands for, as `ParentRecordAccess` already asks with a get. Who may read a table is unchanged. - Reads of a table that has no tag assignment no longer query `collaboration_record_tag`. Core selects the Tags field for every query or get that names no fields, so each of them (including per-record gets and the stored-record read before an update) ran one assignment statement whether or not the table had ever been tagged: 600 gets of an untagged table ran 600 statements, now none. Each node remembers which tables have no assignment (`TaggedTables`, core `Memoization` registered with `ClusterCaches`) for `CollaborationQBitConfig.withTagLookupCacheSeconds` (default 30, 0 disables); writing an assignment clears it on the node and through the ClusterBus. Tables with assignments are read as before, one lookup per 500 parents. No API or schema change. See "What the Tags field costs a read" in the README. diff --git a/src/main/java/com/kingsrook/qbits/collaboration/customfields/FieldCriteriaExpander.java b/src/main/java/com/kingsrook/qbits/collaboration/customfields/FieldCriteriaExpander.java index 9d308f7..e4e5cce 100644 --- a/src/main/java/com/kingsrook/qbits/collaboration/customfields/FieldCriteriaExpander.java +++ b/src/main/java/com/kingsrook/qbits/collaboration/customfields/FieldCriteriaExpander.java @@ -87,10 +87,17 @@ public class FieldCriteriaExpander implements VirtualFieldCriteriaExpanderInterf /******************************************************************************* ** Core's search callback has no field argument, so combine searchable EAV text. + ** A session that may not query the value table gets no custom field matches: + ** core refuses a whole search for one refused sub-query table. An explicit + ** custom field criterion is still refused by core. *******************************************************************************/ @Override public QQueryFilter expandSearchTerm(QTableMetaData table, String term, com.kingsrook.qqq.backend.core.actions.tables.query.SearchTermKind kind, CriteriaExpansionContext context) throws QException { + if(!com.kingsrook.qbits.collaboration.security.ParentRecordAccess.sessionMayQuery(CollaborationTables.FIELD_VALUE)) + { + return (null); + } QQueryFilter result = new QQueryFilter().withBooleanOperator(QQueryFilter.BooleanOperator.OR); for(var definition : CollaborationRecords.query(CollaborationTables.FIELD_DEFINITION, new QQueryFilter() .withCriteria("tableName", QCriteriaOperator.EQUALS, table.getName()).withCriteria("status", QCriteriaOperator.EQUALS, "ACTIVE") diff --git a/src/main/java/com/kingsrook/qbits/collaboration/security/ParentRecordAccess.java b/src/main/java/com/kingsrook/qbits/collaboration/security/ParentRecordAccess.java index 35703f5..18e992e 100644 --- a/src/main/java/com/kingsrook/qbits/collaboration/security/ParentRecordAccess.java +++ b/src/main/java/com/kingsrook/qbits/collaboration/security/ParentRecordAccess.java @@ -23,6 +23,7 @@ package com.kingsrook.qbits.collaboration.security; import java.io.Serializable; import com.kingsrook.qbits.collaboration.CollaborationQBitConfig; +import com.kingsrook.qqq.backend.core.actions.permissions.PermissionEnforcement; import com.kingsrook.qqq.backend.core.actions.permissions.PermissionsHelper; import com.kingsrook.qqq.backend.core.actions.permissions.TablePermissionSubType; import com.kingsrook.qqq.backend.core.context.QContext; @@ -30,6 +31,7 @@ import com.kingsrook.qqq.backend.core.exceptions.QException; import com.kingsrook.qqq.backend.core.exceptions.QNotFoundException; import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException; import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetInput; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput; import com.kingsrook.qqq.backend.core.model.data.QRecord; import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData; @@ -55,6 +57,27 @@ public final class ParentRecordAccess && PermissionsHelper.hasTablePermission(new GetInput(tableName), tableName, TablePermissionSubType.READ)); } + /******************************************************************************* + ** Whether core would let the current session query every one of these tables. + ** This is the table READ check, asked with an enforced query input, that core + ** applies to the tables of a sub-query. A host may protect this qbit's tables + ** more strictly than their defaults; a filter that reaches them through a + ** sub-query is then refused as a whole. + *******************************************************************************/ + public static boolean sessionMayQuery(String... tableNames) + { + for(String tableName : tableNames) + { + QueryInput probe = new QueryInput(tableName); + probe.setPermissionEnforcement(PermissionEnforcement.ENFORCE); + if(!PermissionsHelper.hasTablePermission(probe, tableName, TablePermissionSubType.READ)) + { + return (false); + } + } + return (true); + } + /******************************************************************************* ** Get as the caller, enforcing table permissions and the parent's read locks. *******************************************************************************/ diff --git a/src/main/java/com/kingsrook/qbits/collaboration/tags/TagCriteriaExpander.java b/src/main/java/com/kingsrook/qbits/collaboration/tags/TagCriteriaExpander.java index 2f1ae25..cf6213e 100644 --- a/src/main/java/com/kingsrook/qbits/collaboration/tags/TagCriteriaExpander.java +++ b/src/main/java/com/kingsrook/qbits/collaboration/tags/TagCriteriaExpander.java @@ -25,6 +25,7 @@ import java.io.Serializable; import java.util.List; import com.kingsrook.qbits.collaboration.CollaborationRecords; import com.kingsrook.qbits.collaboration.model.CollaborationTables; +import com.kingsrook.qbits.collaboration.security.ParentRecordAccess; import com.kingsrook.qqq.backend.core.actions.tables.query.CriteriaExpansionContext; import com.kingsrook.qqq.backend.core.actions.tables.query.SearchTermKind; import com.kingsrook.qqq.backend.core.actions.tables.query.VirtualFieldCriteriaExpanderInterface; @@ -136,10 +137,17 @@ public class TagCriteriaExpander implements VirtualFieldCriteriaExpanderInterfac /******************************************************************************* ** Ordinary search matches labels; hash search matches an exact normalized name. + ** A session that may not query the tag tables gets no tag matches: core refuses + ** a whole search for one refused sub-query table, and a text search must not + ** fail because of tags. An explicit Tags criterion is still refused by core. *******************************************************************************/ @Override public QQueryFilter expandSearchTerm(QTableMetaData table, String term, SearchTermKind kind, CriteriaExpansionContext context) throws QException { + if(!ParentRecordAccess.sessionMayQuery(CollaborationTables.RECORD_TAG, CollaborationTables.TAG)) + { + return (null); + } QQueryFilter tags = TagsService.vocabularyFilter(table.getName()).withCriteria("approved", QCriteriaOperator.EQUALS, true) .withCriteria(kind == SearchTermKind.TAG ? "name" : "label", kind == SearchTermKind.TAG ? QCriteriaOperator.EQUALS : QCriteriaOperator.CONTAINS, kind == SearchTermKind.TAG ? TagsService.normalize(term) : term); diff --git a/src/test/java/com/kingsrook/qbits/collaboration/CustomFieldValuesPostgresTest.java b/src/test/java/com/kingsrook/qbits/collaboration/CustomFieldValuesPostgresTest.java index 34c07e7..f81fd70 100644 --- a/src/test/java/com/kingsrook/qbits/collaboration/CustomFieldValuesPostgresTest.java +++ b/src/test/java/com/kingsrook/qbits/collaboration/CustomFieldValuesPostgresTest.java @@ -22,27 +22,46 @@ package com.kingsrook.qbits.collaboration; import java.sql.Connection; import java.sql.DriverManager; +import java.util.ArrayList; +import java.util.List; import java.util.Set; import com.kingsrook.qbits.collaboration.customfields.CustomFieldValueProvider; +import com.kingsrook.qbits.collaboration.customfields.FieldCriteriaExpander; import com.kingsrook.qbits.collaboration.model.CollaborationTables; +import com.kingsrook.qbits.collaboration.permissions.CollaborationPermissions; +import com.kingsrook.qbits.collaboration.tags.TagCriteriaExpander; +import com.kingsrook.qbits.collaboration.tags.TagValueProvider; +import com.kingsrook.qbits.collaboration.tags.TagsService; import com.kingsrook.qqq.backend.core.actions.tables.QueryAction; +import com.kingsrook.qqq.backend.core.actions.tables.RecordSearchAction; import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException; +import com.kingsrook.qqq.backend.core.model.actions.tables.QInputSource; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter; import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput; +import com.kingsrook.qqq.backend.core.model.actions.tables.search.RecordSearchInput; import com.kingsrook.qqq.backend.core.model.data.QRecord; import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference; +import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData; import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType; import com.kingsrook.qqq.backend.core.model.metadata.fields.QVirtualFieldMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.permissions.PermissionLevel; +import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRules; +import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData; import com.kingsrook.qqq.backend.module.postgres.model.metadata.PostgreSQLBackendMetaData; import com.kingsrook.qqq.backend.module.rdbms.actions.RDBMSTransaction; import com.kingsrook.qqq.backend.module.rdbms.jdbc.ConnectionManager; import org.junit.jupiter.api.AfterAll; import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; import org.junit.jupiter.params.ParameterizedTest; import org.junit.jupiter.params.provider.ValueSource; import org.testcontainers.containers.GenericContainer; import org.testcontainers.containers.wait.strategy.Wait; import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; /******************************************************************************* ** Verify provider reads using a real database and explicit caller transactions. @@ -155,6 +174,71 @@ class CustomFieldValuesPostgresTest extends PersonalServiceFixture .isEqualTo(commit ? "Pending transaction label" : null); } + /******************************************************************************* + ** Native sub-queries: a reader whom the host does not let query the tag or + ** value tables still searches the fields of the parent table. Tag and custom + ** field terms match nothing and nothing is refused; an explicit Tags criterion + ** stays refused; a reader who holds the permission searches as before. The + ** parent has a text key, which PostgreSQL compares with the text record id. + *******************************************************************************/ + @Test + void searchSurvivesHostProtectedCollaborationTables() throws Exception + { + instance.addTable(new QTableMetaData().withName("asset").withLabel("Assets").withBackendName("memory").withPrimaryKeyField("id") + .withPermissionRules(new QPermissionRules().withLevel(PermissionLevel.READ_WRITE_PERMISSIONS)) + .withField(new QFieldMetaData("id", QFieldType.STRING)).withField(new QFieldMetaData("title", QFieldType.STRING)) + .withRecordLabelFormatAndFields("%s", "title")); + try(Connection connection = connect(); var sql = connection.createStatement()) + { + sql.execute("CREATE TABLE asset (id VARCHAR(40) PRIMARY KEY, title TEXT)"); + sql.execute("INSERT INTO asset VALUES ('A1', 'Visible'), ('A2', 'Other')"); + } + login("alice", "asset.read", "asset.write"); + CollaborationRecords.insert(CollaborationTables.FIELD_DEFINITION, new QRecord().withValue("tableName", "asset").withValue("name", "note") + .withValue("label", "Note").withValue("type", "STRING").withValue("status", "ACTIVE").withValue("maxLength", 30).withValue("searchable", true)); + CollaborationRecords.insert(CollaborationTables.FIELD_VALUE, new QRecord().withValue("tableName", "asset").withValue("recordId", "A2") + .withValue("fieldName", "note").withValue("valueText", "Quarterly").withValue("parentDeleted", false)); + instance.getTable("asset").withVirtualField(new QVirtualFieldMetaData("collaborationTags", QFieldType.STRING).withLabel("Tags") + .withIsQueryCriteria(true).withIsQuerySelectable(true).withPossibleValueSourceName(CollaborationTables.TAG).withShouldTranslatePossibleValues(false) + .withValueProvider(new QCodeReference(TagValueProvider.class)).withCriteriaExpander(new QCodeReference(TagCriteriaExpander.class))); + instance.getTable("asset").withVirtualField(new QVirtualFieldMetaData("cf_note", QFieldType.STRING).withLabel("Note").withIsEditable(false) + .withIsQueryCriteria(true).withIsQuerySelectable(true).withValueProvider(new QCodeReference(CustomFieldValueProvider.class)) + .withCriteriaExpander(new QCodeReference(FieldCriteriaExpander.class))); + instance.getTable("asset").withSearchFields(new ArrayList<>(List.of("title", "collaborationTags", "cf_note"))); + var tags = new TagsService(); + var tag = tags.create("asset", "Urgent", null); + tags.apply("asset", "A1", List.of(tag.id()), List.of()); + QQueryFilter tagCriteria = new QQueryFilter().withCriteria("collaborationTags", QCriteriaOperator.EQUALS, tag.id()); + assertThat(search("Visible")).containsExactly("A1"); + assertThat(search("Urgent")).containsExactly("A1"); + assertThat(search("#urgent")).containsExactly("A1"); + assertThat(search("Quarterly")).containsExactly("A2"); + assertThat(new QueryAction().execute(new QueryInput("asset").withFilter(tagCriteria)).getRecords()).hasSize(1); + + SearchWithoutCollaborationReadTest.protect(instance, CollaborationTables.RECORD_TAG, CollaborationTables.TAG, CollaborationTables.FIELD_VALUE); + assertThat(search("Visible")).containsExactly("A1"); + assertThat(search("Urgent")).isEmpty(); + assertThat(search("#urgent")).isEmpty(); + assertThat(search("Quarterly")).isEmpty(); + assertThatThrownBy(() -> new QueryAction().execute(new QueryInput("asset").withFilter(tagCriteria))).isInstanceOf(QPermissionDeniedException.class); + + login("alice", "asset.read", "asset.write", CollaborationPermissions.VIEW); + assertThat(search("Visible")).containsExactly("A1"); + assertThat(search("Urgent")).containsExactly("A1"); + assertThat(search("#urgent")).containsExactly("A1"); + assertThat(search("Quarterly")).containsExactly("A2"); + assertThat(new QueryAction().execute(new QueryInput("asset").withFilter(tagCriteria)).getRecords()).hasSize(1); + } + + /******************************************************************************* + ** Global record search as a signed-in person, limited to the text-keyed table. + *******************************************************************************/ + private List search(String term) throws Exception + { + return (new RecordSearchAction().execute(new RecordSearchInput().withTableNames(List.of("asset")).withSearchTerm(term).withInputSource(QInputSource.USER)) + .getResults().stream().map(result -> String.valueOf(result.getRecordId())).sorted().toList()); + } + /******************************************************************************* ** Explicit query transactions see pending values even without thread context. *******************************************************************************/ diff --git a/src/test/java/com/kingsrook/qbits/collaboration/SearchWithoutCollaborationReadTest.java b/src/test/java/com/kingsrook/qbits/collaboration/SearchWithoutCollaborationReadTest.java new file mode 100644 index 0000000..5b3ad32 --- /dev/null +++ b/src/test/java/com/kingsrook/qbits/collaboration/SearchWithoutCollaborationReadTest.java @@ -0,0 +1,250 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.collaboration; + + +import java.util.ArrayList; +import java.util.List; +import java.util.Map; +import java.util.concurrent.ConcurrentHashMap; +import com.kingsrook.qbits.collaboration.customfields.CustomFieldValueProvider; +import com.kingsrook.qbits.collaboration.customfields.FieldCriteriaExpander; +import com.kingsrook.qbits.collaboration.model.CollaborationTables; +import com.kingsrook.qbits.collaboration.permissions.CollaborationPermissions; +import com.kingsrook.qbits.collaboration.tags.TagsService; +import com.kingsrook.qqq.backend.core.actions.collaboration.RuntimeMetaDataOverlay; +import com.kingsrook.qqq.backend.core.actions.customizers.QCodeLoader; +import com.kingsrook.qqq.backend.core.actions.permissions.CustomPermissionChecker; +import com.kingsrook.qqq.backend.core.actions.tables.QueryAction; +import com.kingsrook.qqq.backend.core.actions.tables.RecordSearchAction; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException; +import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput; +import com.kingsrook.qqq.backend.core.model.actions.tables.QInputSource; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput; +import com.kingsrook.qqq.backend.core.model.actions.tables.search.RecordSearchInput; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.QInstance; +import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference; +import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType; +import com.kingsrook.qqq.backend.core.model.metadata.fields.QVirtualFieldMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.permissions.MetaDataWithPermissionRules; +import com.kingsrook.qqq.backend.core.utils.ValueUtils; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; + + +/******************************************************************************* + ** Record search for a person who may read a table, on a host that does not let + ** that person query the tag or custom field value tables: no tag or custom + ** field matches, never a refused search. + *******************************************************************************/ +class SearchWithoutCollaborationReadTest extends PersonalServiceFixture +{ + private int tagId; + + + + /*************************************************************************** + ** Document 1 is tagged "Urgent"; document 2 has the searchable custom field + ** "note" set to "Quarterly". Titles are searchable too. The Tags field is the + ** real contribution; the custom field is the virtual field contributed for a + ** backend whose values live in the value table (the memory backend used here + ** would otherwise store them on the parent, with no sub-query). + ***************************************************************************/ + @BeforeEach + void searchableDocuments() throws Exception + { + instance.getTable("document").withSearchFields(new ArrayList<>(List.of("title"))); + RuntimeMetaDataOverlay.refresh(instance); + admin(); + CollaborationRecords.insert(CollaborationTables.FIELD_DEFINITION, new QRecord().withValue("tableName", "document").withValue("name", "note") + .withValue("label", "Note").withValue("type", "STRING").withValue("status", "ACTIVE").withValue("maxLength", 30).withValue("searchable", true)); + CollaborationRecords.insert(CollaborationTables.FIELD_VALUE, new QRecord().withValue("tableName", "document").withValue("recordId", "2") + .withValue("fieldName", "note").withValue("valueText", "Quarterly").withValue("parentDeleted", false)); + login("alice"); + var tag = new TagsService().create("document", "Urgent", null); + new TagsService().apply("document", 1, List.of(tag.id()), List.of()); + tagId = tag.id(); + instance.getTable("document").withVirtualField(new QVirtualFieldMetaData("cf_note", QFieldType.STRING).withLabel("Note").withIsEditable(false) + .withIsQueryCriteria(true).withIsQuerySelectable(true).withValueProvider(new QCodeReference(CustomFieldValueProvider.class)) + .withCriteriaExpander(new QCodeReference(FieldCriteriaExpander.class))); + instance.getTable("document").getSearchFields().add("cf_note"); + assertThat(instance.getTable("document").getSearchFields()).contains("title", "collaborationTags", "cf_note"); + assertThat(instance.getTable("document").getVirtualFields()).containsKeys("collaborationTags", "cf_note"); + } + + + + /*************************************************************************** + ** With the default rules every reader of the table searches by tag and by + ** custom field, as before. + ***************************************************************************/ + @Test + void defaultRulesSearchTagsAndCustomFields() throws Exception + { + assertThat(search("First")).containsExactly(1); + assertThat(search("Urgent")).containsExactly(1); + assertThat(search("#urgent")).containsExactly(1); + assertThat(search("Quarterly")).containsExactly(2); + assertThat(tagFilter()).containsExactly(1); + assertThat(noteFilter()).containsExactly(2); + } + + + + /*************************************************************************** + ** A host that requires a permission on the link tables: a reader without it + ** still searches the fields of the table itself; tag and custom field terms + ** match nothing; explicit criteria on those fields stay refused; nothing + ** changes for a reader who holds the permission. + ***************************************************************************/ + @Test + void protectedTablesContributeNothingInsteadOfRefusingTheSearch() throws Exception + { + protect(instance, CollaborationTables.RECORD_TAG, CollaborationTables.TAG, CollaborationTables.FIELD_VALUE); + assertThat(search("First")).containsExactly(1); + assertThat(search("document")).containsExactly(1, 2); + assertThat(search("Urgent")).isEmpty(); + assertThat(search("#urgent")).isEmpty(); + assertThat(search("Quarterly")).isEmpty(); + assertThatThrownBy(this::tagFilter).isInstanceOf(QPermissionDeniedException.class); + assertThatThrownBy(this::noteFilter).isInstanceOf(QPermissionDeniedException.class); + + login("alice", CollaborationPermissions.VIEW); + assertThat(search("First")).containsExactly(1); + assertThat(search("Urgent")).containsExactly(1); + assertThat(search("#urgent")).containsExactly(1); + assertThat(search("Quarterly")).containsExactly(2); + assertThat(tagFilter()).containsExactly(1); + assertThat(noteFilter()).containsExactly(2); + } + + + + /*************************************************************************** + ** Either tag table being refused is enough: both are sub-query tables. + ***************************************************************************/ + @Test + void eitherProtectedTagTableContributesNothing() throws Exception + { + var defaultTagRules = instance.getTable(CollaborationTables.TAG).getPermissionRules(); + protect(instance, CollaborationTables.TAG); + assertThat(search("First")).containsExactly(1); + assertThat(search("#urgent")).isEmpty(); + assertThat(search("Quarterly")).containsExactly(2); + instance.getTable(CollaborationTables.TAG).withPermissionRules(defaultTagRules); + assertThat(search("#urgent")).containsExactly(1); + protect(instance, CollaborationTables.RECORD_TAG); + assertThat(search("First")).containsExactly(1); + assertThat(search("Urgent")).isEmpty(); + assertThat(search("Quarterly")).containsExactly(2); + } + + + + /*************************************************************************** + ** What a host does when it puts its own permission on these tables: the + ** permission is required first, then the table keeps its own checker. + ***************************************************************************/ + static void protect(QInstance instance, String... tableNames) + { + for(String tableName : tableNames) + { + var rules = instance.getTable(tableName).getPermissionRules(); + HostPermission.DELEGATES.put(tableName, rules.getCustomPermissionChecker()); + instance.getTable(tableName).withPermissionRules(rules.clone().withCustomPermissionChecker(new QCodeReference(HostPermission.class))); + } + } + + + + /*************************************************************************** + ** A host permission in front of a table's own checker. + ***************************************************************************/ + public static class HostPermission implements CustomPermissionChecker + { + private static final Map DELEGATES = new ConcurrentHashMap<>(); + + + + /************************************************************************ + ** Refuse a session without the permission; otherwise ask the table. + ************************************************************************/ + @Override + public void checkPermissionsThrowing(AbstractActionInput input, MetaDataWithPermissionRules metadata) throws QPermissionDeniedException + { + if(!QContext.getQSession().hasPermission(CollaborationPermissions.VIEW)) + { + throw (new QPermissionDeniedException("This needs the " + CollaborationPermissions.VIEW + " permission.")); + } + QCodeLoader.getAdHoc(CustomPermissionChecker.class, DELEGATES.get(metadata.getName())).checkPermissionsThrowing(input, metadata); + } + } + + + + /*************************************************************************** + ** Global record search as a signed-in person, limited to the host table. + ***************************************************************************/ + private List search(String term) throws QException + { + return (new RecordSearchAction().execute(new RecordSearchInput().withTableNames(List.of("document")).withSearchTerm(term).withInputSource(QInputSource.USER)) + .getResults().stream().map(result -> ValueUtils.getValueAsInteger(result.getRecordId())).sorted().toList()); + } + + + + /*************************************************************************** + ** An explicit Tags criterion on a record list. + ***************************************************************************/ + private List tagFilter() throws QException + { + return (ids(new QQueryFilter().withCriteria("collaborationTags", QCriteriaOperator.EQUALS, tagId))); + } + + + + /*************************************************************************** + ** An explicit custom field criterion on a record list. + ***************************************************************************/ + private List noteFilter() throws QException + { + return (ids(new QQueryFilter().withCriteria("cf_note", QCriteriaOperator.EQUALS, "Quarterly"))); + } + + + + /*************************************************************************** + ** Matching document ids through the real query action. + ***************************************************************************/ + private List ids(QQueryFilter filter) throws QException + { + QueryInput input = new QueryInput("document").withFilter(filter); + input.setInputSource(QInputSource.USER); + return (new QueryAction().execute(input).getRecords().stream().map(row -> row.getValueInteger("id")).sorted().toList()); + } +} -- 2.54.0