fix: presence trouble no longer fails the record features summary; record search no longer refused without read of the tag tables #71

Merged
james.maes merged 2 commits from fix/features-summary-presence-unavailable into develop 2026-11-17 20:54:06 +00:00
Owner

Two defects where a problem in one collaboration feature failed a request that is not about that feature. One commit each.

1. The record features summary failed when presence was unavailable

A host's record screens call core's record features summary (GET /table/{tableName}/{primaryKey}/features). CollaborationSummaryProvider fills its presence part from PresenceService.snapshot and let the shared presence backend's QServiceUnavailableException escape. The whole summary then answered 503, taking the comment, tag, task and file counts, favorite, follow, owner and assignee with it. With the Redis backend that happens for one lease lifetime (45 seconds by default) after the first touch of each table's room in a store that lost its state (the room answers "recovering" on purpose), and for as long as the store is unreachable.

Fix

  • CollaborationSummaryProvider catches only the service-unavailable case around the presence part. The rest of the summary is built as usual, presence is left unset and extensions.presence.status = "unavailable" is set. extensions is the provider-defined map core already serves; no new field. The wire omits empty collections, so an empty and an unset presence look the same there: the extension is what tells "unavailable" from "nobody here". One debug line per occurrence, table name only.
  • The presence endpoints (/presence, /presence/leases, /presence/leases/renew) are unchanged and keep answering 503.
  • The Who's here card (CollaborationHome.presence) had the same shape and now shows "Who's here is temporarily unavailable. Try again shortly." instead of failing; it never shows partial or node-local counts. It also stopped at the first recovering room, so after a store restart it needed one visit per table, a lease lifetime apart, before it could show counts. A recovering room no longer stops the reading (any other failure still stops it at once, so an outage does not cost one timeout per table), and one visit starts every room's interval. This is the one part that goes beyond the summary; it is separable if you would rather not take it here.

Also checked, no change needed

  • The other parts of the summary (favorite, follow, comments, attachments, tags, tasks, assignment): database reads only.
  • Possible-value providers, virtual field value providers, CustomFieldsProvider, TagsMetaDataProvider, FeatureSettingsProvider, CollaborationFollowersProvider, the record access checker and the other home cards: none reads presence or Redis (CustomFieldsProvider only creates the node-local registry when shared presence is not configured).
  • Realtime hints (CollaborationRealtime, Notifications, RedisPresenceBackend.changed): each publication already catches and logs, so a failing bus does not fail the write. RedisPresenceAudience.allows derives the tenant scope without a store call.
  • Not changed: node-local presence constructed for the first time while the host's realtime bus cannot be subscribed would fail with a runtime exception, not a service-unavailable one. That is a different failure and the catch is deliberately narrow.

Can the warm-up restart during normal use? No, and the Lua is unchanged.

The guard starts a new interval when liveUntil > now and the table projections are empty. Every write that advances liveUntil (reservation, renewal, legacy heartbeat, departure) also writes a table member with score now + ttl, the same value, and sets the key lifetimes in the same script; a departure leaves a marker, it does not remove the member. So the table room cannot be empty before liveUntil, and when its last member expires liveUntil > now is already false.

RedisPresenceRecoveryTest.normalUseNeverRestartsRecovery (real Redis) proves it on the unchanged scripts: open and leave within the lease lifetime, a record lease, a legacy connection and an abandoned reservation are each followed by a known-empty snapshot under the same generation and readyAt, with the newest table member's score equal to liveUntil; snapshots read continuously across the moment everything expires never answer "recovering"; and, as the contrast, deleting the projections before liveUntil does start a new generation.

First touch of a table the store has never seen: left as it is

An empty store that never had viewers and a store that just lost its viewers hold the same bytes. Anything that could tell them apart lives in the same keyspace and is lost with it; a marker shared by all tables would sit in another hash slot than the table's room, outside what one script may read on Redis Cluster; server uptime does not see a flush. Treating a missing guard as known-empty would report a viewer who is still on a page as absent after a loss, which is the property the guard exists for (totalLossRequiresSharedRecoveryBeforeKnownEmpty pins it). What remains is that the presence endpoints answer 503 for one lease lifetime on the first touch of each table after a restart without persistence; with this change nothing else does. Shortening the interval to the client renewal period, or keeping guards across restarts with persistence, are design choices for a separate change.

Tests

  • CollaborationApiHttpTest.recordFeaturesSurvivePresenceRecoveryAndFailure (real HTTP, real Redis): with presence recovering and then with the store stopped, the summary answers 200 with counts, favorite, follow and capabilities intact, presence unset and the extension set; GET /presence answers 503 in both cases; the card says unavailable and one visit creates every table's guard.
  • CollaborationApiHttpTest.unavailablePresenceHttp: a viewer is listed in the summary when presence is available (no extension); with the connection not started the summary answers 200 and reports unavailable, a record the caller cannot see is still 404, and the presence routes are still 503.
  • RedisPresenceServiceTest.permissionsIsolationAndUnavailable: the card and the summary with the store stopped (the card assertion was "throws" before).
  • RedisPresenceRecoveryTest.normalUseNeverRestartsRecovery: above. The existing lifecycle and recovery tests are unchanged and pass.

2. Record search was refused for a person who may not query the tag tables

The Tags field is a search field of every taggable table, and so is each searchable custom field kept in the value table. Core's record search asks each of them to expand the search term, and the expansion is a sub-query on collaboration_record_tag and collaboration_tag (or collaboration_field_value). With the default rules every reader may query those tables (CollaborationRowSecurity and TagSecurity allow it on purpose), but a host can require a permission of its own on them. Core checks table READ for each sub-query table and one refusal refuses the whole query, so every record search of such a table answered 403 for a person without that permission, whatever the text.

Fix

  • TagCriteriaExpander.expandSearchTerm and FieldCriteriaExpander.expandSearchTerm first ask core whether the session may query those tables (ParentRecordAccess.sessionMayQuery: PermissionsHelper.hasTablePermission with an enforced QueryInput for the table, the check core applies to a sub-query; no permission name is assumed) and return null when it may not. Core's contract for null is "this field does not match the term": the other search fields of the table still match, and a #tag search matches nothing.
  • Nobody gains access. A person who may query the tables searches and filters exactly as before.
  • An explicit Tags or custom field criterion on a record list is deliberately unchanged: it stays refused by core for such a person, with the message of the refusing table's checker. Matching nothing instead would be wrong for the negative operators (is not, is blank), and a silent empty list for an explicit filter misleads. Note for whoever decides the host rule: the Tags column itself shows labels to anyone who can read the parent record (TagValueProvider reads under a documented bypass), so such a person sees tags on records but cannot search or filter by them; changing that is a permission design question, not part of this fix.

Tests (a host permission is put in front of each table's own checker)

  • SearchWithoutCollaborationReadTest (memory backend, sub-queries materialized through QueryAction): without the permission a text search returns the table's own matches, tag text, #tag and custom field text return nothing, nothing throws, and the explicit Tags and custom field criteria are refused; with the permission, and with the default rules, results are unchanged; refusing either tag table alone is enough.
  • CustomFieldValuesPostgresTest.searchSurvivesHostProtectedCollaborationTables (PostgreSQL, native sub-queries): the same on a text-keyed parent.
  • Both fail before the change with QPermissionDeniedException on the first search.

Build

Full ./mvnw -B verify on this head: BUILD SUCCESS, 351 tests, 0 failures, 0 errors, 26 skipped (the MySQL and PostgreSQL upload journal suites, which skip here as they do on develop), 0 Checkstyle violations, guardrails and coverage gates passed.

🤖 Generated with Claude Code

Two defects where a problem in one collaboration feature failed a request that is not about that feature. One commit each. ## 1. The record features summary failed when presence was unavailable A host's record screens call core's record features summary (`GET /table/{tableName}/{primaryKey}/features`). `CollaborationSummaryProvider` fills its presence part from `PresenceService.snapshot` and let the shared presence backend's `QServiceUnavailableException` escape. The whole summary then answered 503, taking the comment, tag, task and file counts, favorite, follow, owner and assignee with it. With the Redis backend that happens for one lease lifetime (45 seconds by default) after the first touch of each table's room in a store that lost its state (the room answers "recovering" on purpose), and for as long as the store is unreachable. **Fix** - `CollaborationSummaryProvider` catches only the service-unavailable case around the presence part. The rest of the summary is built as usual, `presence` is left unset and `extensions.presence.status = "unavailable"` is set. `extensions` is the provider-defined map core already serves; no new field. The wire omits empty collections, so an empty and an unset `presence` look the same there: the extension is what tells "unavailable" from "nobody here". One debug line per occurrence, table name only. - The presence endpoints (`/presence`, `/presence/leases`, `/presence/leases/renew`) are unchanged and keep answering 503. - The Who's here card (`CollaborationHome.presence`) had the same shape and now shows "Who's here is temporarily unavailable. Try again shortly." instead of failing; it never shows partial or node-local counts. It also stopped at the first recovering room, so after a store restart it needed one visit per table, a lease lifetime apart, before it could show counts. A recovering room no longer stops the reading (any other failure still stops it at once, so an outage does not cost one timeout per table), and one visit starts every room's interval. This is the one part that goes beyond the summary; it is separable if you would rather not take it here. **Also checked, no change needed** - The other parts of the summary (favorite, follow, comments, attachments, tags, tasks, assignment): database reads only. - Possible-value providers, virtual field value providers, `CustomFieldsProvider`, `TagsMetaDataProvider`, `FeatureSettingsProvider`, `CollaborationFollowersProvider`, the record access checker and the other home cards: none reads presence or Redis (`CustomFieldsProvider` only creates the node-local registry when shared presence is not configured). - Realtime hints (`CollaborationRealtime`, `Notifications`, `RedisPresenceBackend.changed`): each publication already catches and logs, so a failing bus does not fail the write. `RedisPresenceAudience.allows` derives the tenant scope without a store call. - Not changed: node-local presence constructed for the first time while the host's realtime bus cannot be subscribed would fail with a runtime exception, not a service-unavailable one. That is a different failure and the catch is deliberately narrow. **Can the warm-up restart during normal use? No, and the Lua is unchanged.** The guard starts a new interval when `liveUntil > now` and the table projections are empty. Every write that advances `liveUntil` (reservation, renewal, legacy heartbeat, departure) also writes a table member with score `now + ttl`, the same value, and sets the key lifetimes in the same script; a departure leaves a marker, it does not remove the member. So the table room cannot be empty before `liveUntil`, and when its last member expires `liveUntil > now` is already false. `RedisPresenceRecoveryTest.normalUseNeverRestartsRecovery` (real Redis) proves it on the unchanged scripts: open and leave within the lease lifetime, a record lease, a legacy connection and an abandoned reservation are each followed by a known-empty snapshot under the same generation and `readyAt`, with the newest table member's score equal to `liveUntil`; snapshots read continuously across the moment everything expires never answer "recovering"; and, as the contrast, deleting the projections before `liveUntil` does start a new generation. **First touch of a table the store has never seen: left as it is** An empty store that never had viewers and a store that just lost its viewers hold the same bytes. Anything that could tell them apart lives in the same keyspace and is lost with it; a marker shared by all tables would sit in another hash slot than the table's room, outside what one script may read on Redis Cluster; server uptime does not see a flush. Treating a missing guard as known-empty would report a viewer who is still on a page as absent after a loss, which is the property the guard exists for (`totalLossRequiresSharedRecoveryBeforeKnownEmpty` pins it). What remains is that the presence endpoints answer 503 for one lease lifetime on the first touch of each table after a restart without persistence; with this change nothing else does. Shortening the interval to the client renewal period, or keeping guards across restarts with persistence, are design choices for a separate change. **Tests** - `CollaborationApiHttpTest.recordFeaturesSurvivePresenceRecoveryAndFailure` (real HTTP, real Redis): with presence recovering and then with the store stopped, the summary answers 200 with counts, favorite, follow and capabilities intact, `presence` unset and the extension set; `GET /presence` answers 503 in both cases; the card says unavailable and one visit creates every table's guard. - `CollaborationApiHttpTest.unavailablePresenceHttp`: a viewer is listed in the summary when presence is available (no extension); with the connection not started the summary answers 200 and reports unavailable, a record the caller cannot see is still 404, and the presence routes are still 503. - `RedisPresenceServiceTest.permissionsIsolationAndUnavailable`: the card and the summary with the store stopped (the card assertion was "throws" before). - `RedisPresenceRecoveryTest.normalUseNeverRestartsRecovery`: above. The existing lifecycle and recovery tests are unchanged and pass. ## 2. Record search was refused for a person who may not query the tag tables The Tags field is a search field of every taggable table, and so is each searchable custom field kept in the value table. Core's record search asks each of them to expand the search term, and the expansion is a sub-query on `collaboration_record_tag` and `collaboration_tag` (or `collaboration_field_value`). With the default rules every reader may query those tables (`CollaborationRowSecurity` and `TagSecurity` allow it on purpose), but a host can require a permission of its own on them. Core checks table READ for each sub-query table and one refusal refuses the whole query, so every record search of such a table answered 403 for a person without that permission, whatever the text. **Fix** - `TagCriteriaExpander.expandSearchTerm` and `FieldCriteriaExpander.expandSearchTerm` first ask core whether the session may query those tables (`ParentRecordAccess.sessionMayQuery`: `PermissionsHelper.hasTablePermission` with an enforced `QueryInput` for the table, the check core applies to a sub-query; no permission name is assumed) and return `null` when it may not. Core's contract for `null` is "this field does not match the term": the other search fields of the table still match, and a `#tag` search matches nothing. - Nobody gains access. A person who may query the tables searches and filters exactly as before. - An explicit Tags or custom field criterion on a record list is deliberately unchanged: it stays refused by core for such a person, with the message of the refusing table's checker. Matching nothing instead would be wrong for the negative operators (is not, is blank), and a silent empty list for an explicit filter misleads. Note for whoever decides the host rule: the Tags column itself shows labels to anyone who can read the parent record (`TagValueProvider` reads under a documented bypass), so such a person sees tags on records but cannot search or filter by them; changing that is a permission design question, not part of this fix. **Tests** (a host permission is put in front of each table's own checker) - `SearchWithoutCollaborationReadTest` (memory backend, sub-queries materialized through `QueryAction`): without the permission a text search returns the table's own matches, tag text, `#tag` and custom field text return nothing, nothing throws, and the explicit Tags and custom field criteria are refused; with the permission, and with the default rules, results are unchanged; refusing either tag table alone is enough. - `CustomFieldValuesPostgresTest.searchSurvivesHostProtectedCollaborationTables` (PostgreSQL, native sub-queries): the same on a text-keyed parent. - Both fail before the change with `QPermissionDeniedException` on the first search. ## Build Full `./mvnw -B verify` on this head: BUILD SUCCESS, 351 tests, 0 failures, 0 errors, 26 skipped (the MySQL and PostgreSQL upload journal suites, which skip here as they do on develop), 0 Checkstyle violations, guardrails and coverage gates passed. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
james.maes added 2 commits 2026-11-17 20:01:16 +00:00
CollaborationSummaryProvider let the shared presence backend's
service-unavailable answer escape, so core's record features summary
(GET /table/{tableName}/{primaryKey}/features) answered 503 as a whole:
counts, favorite, follow, owner and assignee went with it. With the Redis
backend that happened for one lease lifetime after the first touch of each
table's room in a store that had lost its state (the room is "recovering"),
and for as long as the store was unreachable.

The summary now catches only that case, builds every other part as usual,
leaves presence unset and reports extensions.presence.status =
"unavailable". The wire omits empty collections, so an empty and an unset
presence list look the same; the extension is what tells them apart. The
presence endpoints (/presence, /presence/leases, /presence/leases/renew)
are unchanged and keep answering 503.

The Who's here card had the same shape: it now says presence is
temporarily unavailable instead of failing. It also stopped at the first
recovering room, so it needed one visit per table, a lease lifetime apart,
before it could show counts; a recovering room no longer stops the reading
(any other failure still stops it at once), so one visit starts every
room's interval.

The recovery guard is unchanged. RedisPresenceRecoveryTest
.normalUseNeverRestartsRecovery shows that normal use cannot restart the
interval: every write that advances the guard's latest issued expiry also
leaves a table member (a departure marker included) expiring at exactly
that instant, so the room is never empty before it. Open and leave within
the lease lifetime, an abandoned reservation and idle expiry all end in a
known-empty room under the same generation; only real loss of the
projections starts a new one.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
fix(search): no tag or custom field matches, instead of a refused search, without read of their tables
maven / build (push) Successful in 7m0s
build / maven (push) Successful in 7m0s
c28f839658
The Tags field is a search field of every taggable table, and so is each
searchable custom field kept in the value table. Core asks each of them to
expand the search term; the expansion is a sub-query on
collaboration_record_tag and collaboration_tag (or
collaboration_field_value). With the default rules every reader may query
those tables, but a host can require a permission of its own on them. Core
checks table READ for each sub-query table and one refusal refuses the
whole query, so every record search of such a table answered 403 for a
person without that permission, whatever the text.

TagCriteriaExpander.expandSearchTerm and
FieldCriteriaExpander.expandSearchTerm now ask core the same question
first (PermissionsHelper.hasTablePermission with an enforced QueryInput
for the table, the check core applies to a sub-query) and return null when
the session may not query the tables. Core's contract for null is "this
field does not match the term": the other search fields of the table still
match, and a #tag search matches nothing.

Nobody gains access. An explicit Tags or custom field criterion on a
record list is unchanged: it is still refused by core for such a person. A
silent empty result would be wrong for the negative operators (is not,
is blank) and misleading for the positive ones, and the refusal already
names the missing permission. A person who may query the tables searches
and filters as before.

SearchWithoutCollaborationReadTest (memory backend, materialized
sub-queries) and CustomFieldValuesPostgresTest
.searchSurvivesHostProtectedCollaborationTables (PostgreSQL, native
sub-queries) put a host permission in front of each table's own checker:
without it a text search returns the table's own matches, tag and custom
field terms return nothing, nothing throws and the explicit criterion is
refused; with it the results are the same as with the default rules. Both
fail before this change with QPermissionDeniedException on the first
search.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
james.maes merged commit 3c865a5c97 into develop 2026-11-17 20:54:06 +00:00
Sign in to join this conversation.