fix: a rule read that asks for its own rows no longer recurses #43

2026-11-14T07:57:29Z

james.maes commented

2026-11-14 07:57:29 +00:00

The defect

A host composes this QBit with another QBit that adds a virtual field (with a value provider) to tables, including the secured-data tables. After one row was inserted into secured_data_classification, the next query with filter criteria threw StackOverflowError, and so did every query after it.

The repeating frames:

SecuredDataAuditProtector.classificationOf
core ClassifiedValues.isClassified
core ValueBehaviorApplier.validateClassifiedCriteriaValues      <- a query with criteria ...
core QueryAction.executeInAuthorizedScope
(the other QBit's value provider)                               <- ... run by a value provider ...
core VirtualFieldValueProviders.fill
core QueryAction.postRecordActions
SystemWrites.query                                              <- ... inside the read of the classification rows
SecuredDataRuntime.classificationRows
SecuredDataClassifications.row / effective
SecuredDataAuditProtector.classificationOf                      <- and round again

classificationRows() recorded the version of the rows only after the read returned. A question asked from inside the read therefore found the rows out of date and started the read again. Nothing was ever recorded, so every later request did the same. One row is enough because core runs no value provider for an empty result.

The provider is only the code that happened to do it. A post-query customizer or a record-security lookup on a secured-data table could do the same. The active retention rules (activeRetentionRules()) and the policies (SecuredDataResolver.state()) were read in the same way, and so was a transaction's read of its own retention rule edits.

The fix

System reads name their fields. SystemWrites.query and the audit key lookup ask for the stored fields of the table (SystemWrites.storedFields). Core's VirtualFieldValueProviders selects a virtual field only when fieldNamesToInclude is null or contains it, so no value provider runs inside a system read. The rows returned are unchanged: the QBit's tables have no virtual fields of their own.
A read cannot be started from inside itself. RuleRows holds one kind of rule rows and is used for all three. While a read is running, a question on the reading thread is answered from the rows held when the read began (none before the first read) and starts no other read. This covers what point 1 cannot switch off: a post-query customizer runs whatever fields a query names.
A failed read leaves nothing behind. The version is recorded only after a successful read and the reading mark is cleared in a finally, so the next question reads again. This also holds for an Error such as a stack overflow. The rows held before are not given to a request in place of rows that could not be read, which is the behaviour rulesThatCannotBeReadAreNeverAnEmptyAnswer already required.
RetentionRules.active: a question asked from inside a transaction's read of its own rule edits is answered from the rules every thread shares, instead of reading the edits again.

What a nested question is told, and who can be given that answer

This is the part to review most closely, because the classification decides masking, and core refuses to filter on classified values.

The answer. The rows this node held when the read began: the same answer every request was given up to that moment. Before the first read that is no rows, so the code-level classification applies. The answer can be lower than the settled one (a new row raises a field) or higher (a row lowers it). The tests cover both.

Who can be given it. Only code that runs on the reading thread while the read is in progress:

The methods that call RuleRows.get are synchronized on their owner (the runtime for classifications and retention rules, the resolver for policies). While a read is running, only the thread running it can enter. Every other thread waits for the monitor and is answered from the rows the read returns.
The reading mark is set only around the read itself, which is SystemWrites.query. Everything nested in it runs inside SystemWrites.call: the session is the system session and SystemWrites.active() is true.
In that state SecuredDataResolver.resolve returns FULL before it looks at any classification (existing behaviour). So no field access decision, for a person or for the system, is made from the nested answer.
What does use it is core's classified-value checks for operations nested in the read: operand validation of a nested query's criteria, and the protection of audit values for a nested write.
The request that started the read gets the new rows, or the exception if the read failed.

So a person's request never gets the nested answer. It gets the settled rows or an error.

Alternative I did not take. Answering "most sensitive" (PHI) to a nested question would make the nested answer never lower than the settled one. I left it out because nothing a person receives depends on the nested answer, and because retention rules and policies have no equivalent "most restrictive" value, so the three kinds of rows would behave differently. It is a small change in SecuredDataAuditProtector.classificationOf if a reviewer prefers it.

Not changed

The monitors. Classifications and retention rules stay under the one runtime monitor, because reading either may ask for the other; a lock per kind would allow a deadlock.
The rule read still passes the thread's open transaction, as before.

Tests

SecuredDataRuleReloadTest, 11 tests. Without the main-code changes 9 of them end in StackOverflowError (run on this branch with src/main reverted).

Test	Proves
`aValueProviderOnTheClassificationTableDoesNotRunInsideTheRuleRead`	The reported case: provider that queries with criteria, one row inserted through the QBit. The provider is never run by a system read.
`anOrdinaryQueryOnTheClassificationTableStillFillsTheVirtualField`	The virtual field still works for a query that selects every field, and its nested question no longer recurses.
`aQuestionAskedInsideTheClassificationReadDoesNotStartAnotherRead`	Post-query customizer (not switched off by naming fields): one read, not a recursion.
`aQuestionAskedInsideTheReadIsAnsweredFromTheRowsHeldBefore`	The nested answer in both directions of a change; it is asked as the system session inside `SystemWrites`; the resolver answers `FULL` there; the outer question gets the new rows.
`aRequestOnAnotherThreadWaitsForTheReadAndIsGivenItsRows`	A second thread is not answered while the read runs, then gets the new rows.
`aClassificationReadThatFailsIsTriedAgainByTheNextQuestion`	A read that fails with an exception, then with an `Error`, is read again and answers with the current rows.
`aValueProviderOnTheRetentionRuleTableDoesNotRunInsideTheRuleRead`, `aQuestionAskedInsideTheRetentionRuleReadIsAnsweredFromTheRulesHeldBefore`	The same for the active retention rules, including a failed read.
`aQuestionAskedInsideAReadOfATransactionsOwnEditsDoesNotReadThemAgain`	The own-edits read.
`aQuestionAskedInsideThePolicyReadDoesNotStartAnotherRead`	The same for policies, including a failed read.
`ruleRowsAreReadOncePerVersionAndNeverFromInsideTheirOwnRead`	`RuleRows` alone: nothing held before the first read, failure, version change.

The tests change rows "as another node writes them" (straight into the store, then the version moves), so that the read under test is the one the test observes.

Not proven here

The tests use the in-memory backend. The defect and the fix are in how the rows are cached, not in SQL, so I did not add an RDBMS variant.
Reaching the policy read from inside itself needs code that calls the resolver's policy questions from within the policy read. Core does not do that for a system read, so the test does it directly from a customizer.

Build

./mvnw -B verify on this head: 112 tests, 0 failures, Checkstyle and the JaCoCo gates green. Nothing skipped.

A port of this commit to release/0.1 follows in its own pull request.

🤖 Generated with Claude Code

## The defect A host composes this QBit with another QBit that adds a virtual field (with a value provider) to tables, including the secured-data tables. After one row was inserted into `secured_data_classification`, the next query with filter criteria threw `StackOverflowError`, and so did every query after it. The repeating frames: ``` SecuredDataAuditProtector.classificationOf core ClassifiedValues.isClassified core ValueBehaviorApplier.validateClassifiedCriteriaValues <- a query with criteria ... core QueryAction.executeInAuthorizedScope (the other QBit's value provider) <- ... run by a value provider ... core VirtualFieldValueProviders.fill core QueryAction.postRecordActions SystemWrites.query <- ... inside the read of the classification rows SecuredDataRuntime.classificationRows SecuredDataClassifications.row / effective SecuredDataAuditProtector.classificationOf <- and round again ``` `classificationRows()` recorded the version of the rows only after the read returned. A question asked from inside the read therefore found the rows out of date and started the read again. Nothing was ever recorded, so every later request did the same. One row is enough because core runs no value provider for an empty result. The provider is only the code that happened to do it. A post-query customizer or a record-security lookup on a secured-data table could do the same. The active retention rules (`activeRetentionRules()`) and the policies (`SecuredDataResolver.state()`) were read in the same way, and so was a transaction's read of its own retention rule edits. ## The fix 1. **System reads name their fields.** `SystemWrites.query` and the audit key lookup ask for the stored fields of the table (`SystemWrites.storedFields`). Core's `VirtualFieldValueProviders` selects a virtual field only when `fieldNamesToInclude` is null or contains it, so no value provider runs inside a system read. The rows returned are unchanged: the QBit's tables have no virtual fields of their own. 2. **A read cannot be started from inside itself.** `RuleRows` holds one kind of rule rows and is used for all three. While a read is running, a question on the reading thread is answered from the rows held when the read began (none before the first read) and starts no other read. This covers what point 1 cannot switch off: a post-query customizer runs whatever fields a query names. 3. **A failed read leaves nothing behind.** The version is recorded only after a successful read and the reading mark is cleared in a `finally`, so the next question reads again. This also holds for an `Error` such as a stack overflow. The rows held before are not given to a request in place of rows that could not be read, which is the behaviour `rulesThatCannotBeReadAreNeverAnEmptyAnswer` already required. 4. `RetentionRules.active`: a question asked from inside a transaction's read of its own rule edits is answered from the rules every thread shares, instead of reading the edits again. ## What a nested question is told, and who can be given that answer This is the part to review most closely, because the classification decides masking, and core refuses to filter on classified values. **The answer.** The rows this node held when the read began: the same answer every request was given up to that moment. Before the first read that is no rows, so the code-level classification applies. The answer can be lower than the settled one (a new row raises a field) or higher (a row lowers it). The tests cover both. **Who can be given it.** Only code that runs on the reading thread while the read is in progress: - The methods that call `RuleRows.get` are synchronized on their owner (the runtime for classifications and retention rules, the resolver for policies). While a read is running, only the thread running it can enter. Every other thread waits for the monitor and is answered from the rows the read returns. - The reading mark is set only around the read itself, which is `SystemWrites.query`. Everything nested in it runs inside `SystemWrites.call`: the session is the system session and `SystemWrites.active()` is true. - In that state `SecuredDataResolver.resolve` returns `FULL` before it looks at any classification (existing behaviour). So no field access decision, for a person or for the system, is made from the nested answer. - What does use it is core's classified-value checks for operations nested in the read: operand validation of a nested query's criteria, and the protection of audit values for a nested write. - The request that started the read gets the new rows, or the exception if the read failed. So a person's request never gets the nested answer. It gets the settled rows or an error. **Alternative I did not take.** Answering "most sensitive" (PHI) to a nested question would make the nested answer never lower than the settled one. I left it out because nothing a person receives depends on the nested answer, and because retention rules and policies have no equivalent "most restrictive" value, so the three kinds of rows would behave differently. It is a small change in `SecuredDataAuditProtector.classificationOf` if a reviewer prefers it. ## Not changed - The monitors. Classifications and retention rules stay under the one runtime monitor, because reading either may ask for the other; a lock per kind would allow a deadlock. - The rule read still passes the thread's open transaction, as before. ## Tests `SecuredDataRuleReloadTest`, 11 tests. Without the main-code changes 9 of them end in `StackOverflowError` (run on this branch with `src/main` reverted). | Test | Proves | |---|---| | `aValueProviderOnTheClassificationTableDoesNotRunInsideTheRuleRead` | The reported case: provider that queries with criteria, one row inserted through the QBit. The provider is never run by a system read. | | `anOrdinaryQueryOnTheClassificationTableStillFillsTheVirtualField` | The virtual field still works for a query that selects every field, and its nested question no longer recurses. | | `aQuestionAskedInsideTheClassificationReadDoesNotStartAnotherRead` | Post-query customizer (not switched off by naming fields): one read, not a recursion. | | `aQuestionAskedInsideTheReadIsAnsweredFromTheRowsHeldBefore` | The nested answer in both directions of a change; it is asked as the system session inside `SystemWrites`; the resolver answers `FULL` there; the outer question gets the new rows. | | `aRequestOnAnotherThreadWaitsForTheReadAndIsGivenItsRows` | A second thread is not answered while the read runs, then gets the new rows. | | `aClassificationReadThatFailsIsTriedAgainByTheNextQuestion` | A read that fails with an exception, then with an `Error`, is read again and answers with the current rows. | | `aValueProviderOnTheRetentionRuleTableDoesNotRunInsideTheRuleRead`, `aQuestionAskedInsideTheRetentionRuleReadIsAnsweredFromTheRulesHeldBefore` | The same for the active retention rules, including a failed read. | | `aQuestionAskedInsideAReadOfATransactionsOwnEditsDoesNotReadThemAgain` | The own-edits read. | | `aQuestionAskedInsideThePolicyReadDoesNotStartAnotherRead` | The same for policies, including a failed read. | | `ruleRowsAreReadOncePerVersionAndNeverFromInsideTheirOwnRead` | `RuleRows` alone: nothing held before the first read, failure, version change. | The tests change rows "as another node writes them" (straight into the store, then the version moves), so that the read under test is the one the test observes. ## Not proven here - The tests use the in-memory backend. The defect and the fix are in how the rows are cached, not in SQL, so I did not add an RDBMS variant. - Reaching the policy read from inside itself needs code that calls the resolver's policy questions from within the policy read. Core does not do that for a system read, so the test does it directly from a customizer. ## Build `./mvnw -B verify` on this head: 112 tests, 0 failures, Checkstyle and the JaCoCo gates green. Nothing skipped. A port of this commit to `release/0.1` follows in its own pull request. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

james.maes added 1 commit 2026-11-14 07:57:29 +00:00

fix: a rule read that asks for its own rows no longer recurses

maven / build (push) Successful in 1m17s

Details

build / maven (push) Successful in 1m17s

Details

d99db4738f

Reading the rule rows a node keeps (classifications, active retention rules,
policies) is a query on a secured-data table, and other code runs inside that
query: a virtual field value provider or a post-query customizer that another
QBit or the host put on the table. When that code runs a query with filter
criteria, core validates the operands of classified fields and asks this QBit
for the classification of the field, which needs the rows that are being
read. The version of the rows was recorded only after the read returned, so
the nested question started the read again, and that read asked again:
StackOverflowError, and the same on every later request, since nothing was
ever recorded. One classification row was enough (core runs no value provider
for an empty result).

- System reads of the QBit's own tables (SystemWrites.query, the audit key
  lookup) name the stored fields of the table. Core fills a virtual field only
  for a query that selects it, so no value provider runs inside a system read.
- RuleRows holds one kind of rule rows and is used for all three kinds, which
  had the same shape. A question asked from inside a read is answered from the
  rows held when the read began (none before the first read) and never starts
  another read. The owner's monitor makes that answer private to the reading
  thread: every other thread waits and gets the rows the read returns.
- A read that fails records nothing and clears the reading mark, so the next
  question reads again. The rows held before are not given to a request in
  place of rows that could not be read.
- RetentionRules.active: a question asked from inside a transaction's read of
  its own rule edits is answered from the rules every thread shares instead
  of reading the edits again.

Who can be given the nested answer: only code on the reading thread, inside
SystemWrites.call, where the session is the system session and the resolver
answers FULL without looking at a classification. No field access decision
for a person is made from it; the request that started the read, and every
request on another thread, gets the rows the read returns.

Tests (SecuredDataRuleReloadTest): 9 of the 11 end in StackOverflowError
without the fix. They cover a value provider and a post-query customizer on
the classification, retention rule and policy tables, the answer a nested
question gets in both directions of a change, the session it is asked in, a
second thread waiting for the read, and failed reads (an exception and an
Error) being read again.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

james.maes referenced this pull request

2026-11-14 07:58:02 +00:00

fix: a rule read that asks for its own rows no longer recurses (release/0.1 port of #43) #44

james.maes merged commit 7f1b5988af into develop

2026-11-14 08:23:03 +00:00

james.maes referenced this issue from a commit

2026-11-14 08:23:09 +00:00

Merge pull request 'fix: a rule read that asks for its own rows no longer recurses (release/0.1 port of #43)' (#44) from fix/reentrant-rule-reload-0.1 into release/0.1

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: QRun-IO/qbit-secured-data#43