Files
qbit-secured-data/src/main/java/com/kingsrook/qbits/secureddata/SecuredDataResolver.java
T
james.maesandClaude Fable 5.2 db26a1dc2e
maven / build (push) Successful in 1m22s
build / maven (push) Successful in 1m22s
fix: a rule read that asks for its own rows no longer recurses
Reading the rule rows a node keeps (classifications, active retention rules,
policies) is a query on a secured-data table, and other code runs inside that
query: a virtual field value provider or a post-query customizer that another
QBit or the host put on the table. When that code runs a query with filter
criteria, core validates the operands of classified fields and asks this QBit
for the classification of the field, which needs the rows that are being
read. The version of the rows was recorded only after the read returned, so
the nested question started the read again, and that read asked again:
StackOverflowError, and the same on every later request, since nothing was
ever recorded. One classification row was enough (core runs no value provider
for an empty result).

- System reads of the QBit's own tables (SystemWrites.query, the audit key
  lookup) name the stored fields of the table. Core fills a virtual field only
  for a query that selects it, so no value provider runs inside a system read.
- RuleRows holds one kind of rule rows and is used for all three kinds, which
  had the same shape. A question asked from inside a read is answered from the
  rows held when the read began (none before the first read) and never starts
  another read. The owner's monitor makes that answer private to the reading
  thread: every other thread waits and gets the rows the read returns.
- A read that fails records nothing and clears the reading mark, so the next
  question reads again. The rows held before are not given to a request in
  place of rows that could not be read.
- RetentionRules.active: a question asked from inside a transaction's read of
  its own rule edits is answered from the rules every thread shares instead
  of reading the edits again.

Who can be given the nested answer: only code on the reading thread, inside
SystemWrites.call, where the session is the system session and the resolver
answers FULL without looking at a classification. No field access decision
for a person is made from it; the request that started the read, and every
request on another thread, gets the rows the read returns.

Tests (SecuredDataRuleReloadTest): 9 of the 11 end in StackOverflowError
without the fix. They cover a value provider and a post-query customizer on
the classification, retention rule and policy tables, the answer a nested
question gets in both directions of a change, the session it is asked in, a
second thread waiting for the read, and failed reads (an exception and an
Error) being read again.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-14 01:56:49 -06:00

590 lines
27 KiB
Java

/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.secureddata;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import java.io.Serializable;
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.time.Instant;
import java.util.Base64;
import java.util.HashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Objects;
import java.util.TreeSet;
import com.kingsrook.qbits.secureddata.metadata.SecuredDataTables;
import com.kingsrook.qbits.secureddata.permissions.SecuredDataPermissions;
import com.kingsrook.qbits.secureddata.support.SecuredDataMetadata;
import com.kingsrook.qbits.secureddata.support.SystemWrites;
import com.kingsrook.qqq.backend.core.actions.ai.DataClassification;
import com.kingsrook.qqq.backend.core.actions.permissions.fieldaccess.FieldAccessDecision;
import com.kingsrook.qqq.backend.core.actions.permissions.fieldaccess.FieldAccessEvent;
import com.kingsrook.qqq.backend.core.actions.permissions.fieldaccess.FieldAccessResolverInterface;
import com.kingsrook.qqq.backend.core.actions.permissions.fieldaccess.FieldAccessTier;
import com.kingsrook.qqq.backend.core.actions.permissions.fieldaccess.FieldMaskStyle;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import com.kingsrook.qqq.backend.core.model.session.QSession;
import com.kingsrook.qqq.backend.core.model.session.QSystemUserSession;
import com.kingsrook.qqq.backend.core.utils.JsonUtils;
/*******************************************************************************
** Seven-step policy resolution with instance-versioned, bounded per-user state.
*******************************************************************************/
public class SecuredDataResolver implements FieldAccessResolverInterface
{
private final SecuredDataRuntime runtime;
private final byte[] fingerprintKey;
private final ThreadLocal<Boolean> resolving = ThreadLocal.withInitial(() -> false);
private final Map<String, RevealCache> reveals = new HashMap<>();
/*******************************************************************************
** Bounded per-user grant cache, with exact read-time expiry.
*******************************************************************************/
private record RevealCache(Instant until, List<QRecord> rows)
{
}
/** Used only inside methods synchronized on this resolver. */
private final RuleRows policyRows = new RuleRows();
private List<QRecord> policies = List.of();
private final Map<String, UserState> users = new HashMap<>();
/** Request-scoped (QContext) holder of the access fingerprint. */
private static final String FINGERPRINT_KEY = SecuredDataResolver.class.getName() + ".accessFingerprint";
/*******************************************************************************
** Cached policy matches and break-glass rows. Expiration is checked on every use.
*******************************************************************************/
private record UserState(List<QRecord> policies, List<QRecord> breakGlass)
{
}
/*******************************************************************************
** Zero thresholds disable that guardrail; positive thresholds combine by minimum.
*******************************************************************************/
public record Guardrails(int exactMatchPerHour, int missAlertConsecutive, int distinctValueAlert, boolean minEntropyEnabled)
{
}
/*******************************************************************************
** Capture the supplied key once; default keys are shared within this JVM.
*******************************************************************************/
public SecuredDataResolver(SecuredDataRuntime runtime)
{
this.runtime = runtime;
this.fingerprintKey = Objects.requireNonNull(runtime.config().getFingerprintKeySupplier().get()).clone();
if(fingerprintKey.length < 16)
{
throw (new IllegalArgumentException("Fingerprint keys must contain at least 16 bytes"));
}
}
/*******************************************************************************
** Resolve classification, grants, policies, channel caps and entropy.
*******************************************************************************/
@Override
public FieldAccessDecision resolve(QTableMetaData table, QFieldMetaData field)
{
if(SystemWrites.active() || QContext.getQSession() instanceof QSystemUserSession
|| Objects.equals(table.getPrimaryKeyField(), field.getName()))
{
return (FieldAccessDecision.FULL);
}
if(resolving.get())
{
throw (new IllegalStateException("Recursive secured-data resolution outside a system lookup"));
}
resolving.set(true);
try
{
DataClassification classification = SecuredDataClassifications.effective(table, field.getName());
if(classification == null || classification == DataClassification.PUBLIC)
{
return (FieldAccessDecision.FULL);
}
QSession session = QContext.getQSession();
if(SecuredDataTables.ALL.contains(table.getName()) && session != null && session.hasPermission(SecuredDataPermissions.ADMIN))
{
return (FieldAccessDecision.FULL);
}
List<QRecord> matches = matching(table, classification);
FieldAccessTier tier = null;
for(QRecord row : matches)
{
if(blank(row.getValueString("client_channel")))
{
FieldAccessTier candidate = FieldAccessTier.valueOf(row.getValueString("tier"));
tier = tier == null ? candidate : tier.leastRestrictive(candidate);
}
}
if(tier == null)
{
tier = runtime.config().defaultTier(classification);
}
if(session != null && SecuredDataMetadata.hasFeature(table, SecuredDataPermissions.REVEAL))
{
tier = tier.leastRestrictive(FieldAccessTier.REVEALABLE);
}
if(activeBreakGlass(table.getName(), classification) != null
|| (session != null && SecuredDataMetadata.hasFeature(table, SecuredDataPermissions.FULL)))
{
tier = FieldAccessTier.FULL;
}
for(QRecord row : matches)
{
if(!blank(row.getValueString("client_channel")))
{
tier = tier.mostRestrictive(FieldAccessTier.valueOf(row.getValueString("tier")));
}
}
QRecord override = SecuredDataClassifications.row(table, field.getName());
String style = override == null ? null : override.getValueString("mask_style");
boolean exactAllowed = !guardrails(matches).minEntropyEnabled() || !lowCardinality(field, override);
return (new FieldAccessDecision(tier, blank(style) ? null : FieldMaskStyle.valueOf(style), "Secured-data policy", exactAllowed));
}
finally
{
resolving.remove();
}
}
/*******************************************************************************
** The field's matching policies also drive enumeration controls.
*******************************************************************************/
public Guardrails guardrails(QTableMetaData table, String fieldName)
{
return (guardrails(matching(table, SecuredDataClassifications.effective(table, fieldName))));
}
/*******************************************************************************
** Most restrictive enabled setting across matching rows; config when none match.
*******************************************************************************/
private Guardrails guardrails(List<QRecord> matches)
{
SecuredDataQBitConfig config = runtime.config();
if(matches.isEmpty())
{
return (new Guardrails(config.getExactMatchPerHour(), config.getMissAlertConsecutive(), config.getDistinctValueAlert(), config.getMinEntropyEnabled()));
}
int rate = 0;
int misses = 0;
int distinct = 0;
boolean entropy = false;
for(QRecord row : matches)
{
rate = minimumEnabled(rate, setting(row, "exact_match_per_hour", config.getExactMatchPerHour()));
misses = minimumEnabled(misses, setting(row, "miss_alert_consecutive", config.getMissAlertConsecutive()));
distinct = minimumEnabled(distinct, setting(row, "distinct_value_alert", config.getDistinctValueAlert()));
entropy |= row.getValueBoolean("min_entropy_enabled") == null ? config.getMinEntropyEnabled() : row.getValueBoolean("min_entropy_enabled");
}
return (new Guardrails(rate, misses, distinct, entropy));
}
/*******************************************************************************
** Read an optional numeric setting.
*******************************************************************************/
private static int setting(QRecord row, String name, int fallback)
{
Integer value = row.getValueInteger(name);
return (value == null ? fallback : value);
}
/*******************************************************************************
** Zero means off, so an enabled row always wins over a disabled row.
*******************************************************************************/
private static int minimumEnabled(int left, int right)
{
return (left == 0 ? right : right == 0 ? left : Math.min(left, right));
}
/*******************************************************************************
** Explicit true/false wins over the field type/name heuristic.
*******************************************************************************/
static boolean lowCardinality(QFieldMetaData field, QRecord override)
{
Boolean explicit = override == null ? null : override.getValueBoolean("low_cardinality");
if(explicit != null)
{
return (explicit);
}
QFieldType type = field.getType();
String name = (field.getName() + " " + field.getLabel()).replaceAll("([a-z])([A-Z])", "$1 $2").toLowerCase(Locale.ROOT);
return (type == QFieldType.BOOLEAN || type == QFieldType.DATE || type == QFieldType.DATE_TIME
|| field.getPossibleValueSourceName() != null || field.getInlinePossibleValueSource() != null
|| (type == QFieldType.INTEGER && name.contains("year"))
|| name.matches(".*(?:^|[^a-z])(state|region|province|zip3)(?:[^a-z]|$).*")
|| (type == QFieldType.STRING && field.getMaxLength() != null && field.getMaxLength() < 6));
}
/*******************************************************************************
** Policies for this user, classification, table and current normalized channel.
*******************************************************************************/
private List<QRecord> matching(QTableMetaData table, DataClassification classification)
{
String channel = SecuredDataMetadata.channel(QContext.getQSession());
return (state().policies().stream().filter(row -> classification != null && classification.name().equals(row.getValueString("classification")))
.filter(row -> blank(row.getValueString("table_name")) || table.getName().equals(row.getValueString("table_name")))
.filter(row -> blank(row.getValueString("client_channel")) || Objects.equals(channel, row.getValueString("client_channel"))).toList());
}
/*******************************************************************************
** Granting rows only: channel rows cap tiers and never grant reveal settings.
*******************************************************************************/
public List<QRecord> revealPolicies(QTableMetaData table, String fieldName)
{
return (matching(table, SecuredDataClassifications.effective(table, fieldName)).stream()
.filter(row -> blank(row.getValueString("client_channel")) && "REVEALABLE".equals(row.getValueString("tier"))).toList());
}
/*******************************************************************************
** Refresh policy rows once per version and matched state per relevant permissions.
** A read that fails throws and is tried again by the next call. A question
** asked from inside the read itself (see RuleRows) is answered from the
** policies held when the read began; whatever it matched for its user is
** dropped with every other user's state when the read returns.
*******************************************************************************/
private synchronized UserState state()
{
policies = policyRows.get(runtime.policyVersion(), () ->
{
List<QRecord> rows = SystemWrites.query(SecuredDataTables.POLICY, new QQueryFilter());
users.clear();
return (rows);
});
QSession session = QContext.getQSession();
TreeSet<String> held = new TreeSet<>();
for(QRecord policy : policies)
{
String permission = policy.getValueString("permission_name");
if(session != null && session.hasPermission(permission))
{
held.add(permission);
}
}
String key = userId() + '\u0000' + String.join("\u0000", held);
UserState state = users.get(key);
if(state == null)
{
if(users.size() >= 1024)
{
users.clear();
}
List<QRecord> matching = policies.stream().filter(row -> Boolean.TRUE.equals(row.getValueBoolean("is_active")))
.filter(row -> held.contains(row.getValueString("permission_name"))).toList();
List<QRecord> breaks = SystemWrites.query(SecuredDataTables.BREAK_GLASS, new QQueryFilter()
.withCriteria(new QFilterCriteria("user_id", QCriteriaOperator.EQUALS, userId())));
state = new UserState(matching, List.copyOf(breaks));
users.put(key, state);
}
return (state);
}
/*******************************************************************************
** Exact expiry is checked at use time, independent of the future expiry job.
*******************************************************************************/
public List<QRecord> activeBreakGlass()
{
Instant now = runtime.config().getClock().instant();
return (state().breakGlass().stream().filter(row -> "ACTIVE".equals(row.getValueString("status")))
.filter(row -> row.getValueInstant("expires_at") != null && row.getValueInstant("expires_at").isAfter(now)).toList());
}
/*******************************************************************************
** A break-glass grant must cover both the table and classification.
*******************************************************************************/
QRecord activeBreakGlass(String table, DataClassification classification)
{
return (activeBreakGlass().stream().filter(row -> blank(row.getValueString("table_name")) || table.equals(row.getValueString("table_name")))
.filter(row -> DataClassification.valueOf(Objects.requireNonNullElse(row.getValueString("classification_ceiling"), "PHI")).isAtLeast(classification))
.findFirst().orElse(null));
}
/*******************************************************************************
** Stable across unrelated permissions/records; includes channel and exact expiry.
*******************************************************************************/
@Override
public String getAccessFingerprint()
{
/////////////////////////////////////////////////////////////////////////////
// core asks for the fingerprint before every field decision (once per //
// field of every record of a result), so the value is kept for the //
// request - and only while everything it is derived from is unchanged //
/////////////////////////////////////////////////////////////////////////////
QSession session = QContext.getQSession();
String channel = Objects.requireNonNullElse(SecuredDataMetadata.channel(session), "");
Instant now = runtime.config().getClock().instant();
RequestFingerprint kept = QContext.getObject(FINGERPRINT_KEY, RequestFingerprint.class).orElse(null);
if(kept != null && kept.matches(this, session, channel, runtime.policyVersion(), runtime.classificationVersion(), now))
{
return (kept.value());
}
RequestFingerprint computed = computeAccessFingerprint(session, channel, now);
QContext.setObject(FINGERPRINT_KEY, computed);
return (computed.value());
}
/*******************************************************************************
** The fingerprint and what it was derived from. It stops matching when the
** session, its permissions, the channel, a policy or classification version
** changes, or when the first active break-glass grant expires.
*******************************************************************************/
private record RequestFingerprint(int resolver, QSession session, int permissionsIdentity, int permissionCount, String channel, long policyVersion,
long classificationVersion, Instant validUntil, String value) implements Serializable
{
/*******************************************************************************
** Whether the kept value is still the one a fresh computation would give.
*******************************************************************************/
boolean matches(SecuredDataResolver owner, QSession current, String currentChannel, long currentPolicyVersion, long currentClassificationVersion, Instant now)
{
return (resolver == System.identityHashCode(owner) && session == current && permissionsIdentity == identityOfPermissions(current) && permissionCount == countOfPermissions(current)
&& channel.equals(currentChannel) && policyVersion == currentPolicyVersion && classificationVersion == currentClassificationVersion && now.isBefore(validUntil));
}
}
/*******************************************************************************
** The permission set object of a session (it is replaced, not edited, when a
** session is given new permissions).
*******************************************************************************/
private static int identityOfPermissions(QSession session)
{
return (session == null || session.getPermissions() == null ? 0 : System.identityHashCode(session.getPermissions()));
}
/*******************************************************************************
** How many permissions a session holds (an edit in place changes it).
*******************************************************************************/
private static int countOfPermissions(QSession session)
{
return (session == null || session.getPermissions() == null ? -1 : session.getPermissions().size());
}
/*******************************************************************************
** Stable across unrelated permissions/records; includes channel and exact expiry.
*******************************************************************************/
private synchronized RequestFingerprint computeAccessFingerprint(QSession session, String channel, Instant now)
{
/////////////////////////////////////////////////////////////////////////////
// versions are read before the rows they describe, so a change during the //
// computation leaves a kept value that no longer matches //
/////////////////////////////////////////////////////////////////////////////
long policyVersion = runtime.policyVersion();
long classificationVersion = runtime.classificationVersion();
TreeSet<String> relevant = new TreeSet<>();
List<QRecord> breakGlass = state().breakGlass();
for(QRecord row : policies)
{
String permission = row.getValueString("permission_name");
if(session != null && session.hasPermission(permission))
{
relevant.add(permission);
}
}
for(QTableMetaData table : QContext.getQInstance().getTables().values())
{
String base = SecuredDataMetadata.permissionBase(table);
for(String suffix : List.of(SecuredDataPermissions.FULL, SecuredDataPermissions.REVEAL))
{
if(session != null && session.hasPermission(base + "." + suffix))
{
relevant.add(base + "." + suffix);
}
}
}
List<QRecord> activeBreaks = breakGlass.stream().filter(row -> "ACTIVE".equals(row.getValueString("status")))
.filter(row -> row.getValueInstant("expires_at") != null && row.getValueInstant("expires_at").isAfter(now)).toList();
List<String> breaks = activeBreaks.stream().map(row -> row.getValueString("id")).sorted().toList();
Instant validUntil = activeBreaks.stream().map(row -> row.getValueInstant("expires_at")).min(Instant::compareTo).orElse(Instant.MAX);
String payload = JsonUtils.toJson(List.of(userId(), relevant, policyVersion, classificationVersion, breaks, channel));
try
{
String value = Base64.getUrlEncoder().withoutPadding().encodeToString(MessageDigest.getInstance("SHA-256").digest(payload.getBytes(StandardCharsets.UTF_8)));
return (new RequestFingerprint(System.identityHashCode(this), session, identityOfPermissions(session), countOfPermissions(session), channel, policyVersion, classificationVersion, validUntil, value));
}
catch(java.security.NoSuchAlgorithmException e)
{
throw (new IllegalStateException(e));
}
}
/*******************************************************************************
** Existing grants are read as system; phase (c) owns their creation.
*******************************************************************************/
@Override
public boolean isRevealActive(String tableName, String fieldName, Serializable primaryKey)
{
synchronized(reveals)
{
String user = userId();
Instant now = runtime.config().getClock().instant();
RevealCache cached = reveals.get(user);
if(cached == null || !cached.until().isAfter(now))
{
if(reveals.size() >= 1024)
{
reveals.clear();
}
cached = new RevealCache(now.plusSeconds(5), SystemWrites.query(SecuredDataTables.REVEAL_GRANT, new QQueryFilter()
.withCriteria(new QFilterCriteria("user_id", QCriteriaOperator.EQUALS, user))
.withCriteria(new QFilterCriteria("expires_at", QCriteriaOperator.GREATER_THAN, now))));
reveals.put(user, cached);
}
return (cached.rows().stream().anyMatch(row -> tableName.equals(row.getValueString("table_name"))
&& fieldName.equals(row.getValueString("field_name")) && String.valueOf(primaryKey).equals(row.getValueString("record_id"))
&& row.getValueInstant("expires_at").isAfter(now)));
}
}
/*******************************************************************************
** Publish a newly created grant to all sessions on this instance immediately.
*******************************************************************************/
public void invalidateReveals()
{
synchronized(reveals)
{
reveals.clear();
}
}
/*******************************************************************************
** Logging never throws into a core read.
*******************************************************************************/
@Override
public void onAccess(FieldAccessEvent event)
{
runtime.logger().onAccess(event);
}
/*******************************************************************************
** Preflight enumeration limits; core reports the REFUSED outcome exactly once.
*******************************************************************************/
@Override
public void checkExactMatch(FieldAccessEvent event) throws QPermissionDeniedException
{
runtime.logger().checkExactMatch(event);
}
/*******************************************************************************
** A keyed, non-reversible search identifier. No raw value leaves this method.
*******************************************************************************/
@Override
public String fingerprintSearchValue(Serializable value)
{
try
{
Mac mac = Mac.getInstance("HmacSHA256");
mac.init(new SecretKeySpec(fingerprintKey, "HmacSHA256"));
///////////////////////////////////////////////////////////////////////////////
// keep only 16 bits: enough to count distinct searched values in an hour, //
// far too few to confirm a guessed value (each fingerprint matches ~1 in //
// 65536 of the value space), so a persisted fingerprint is not a derivative //
// of the searched value that could be brute-forced with the key. //
///////////////////////////////////////////////////////////////////////////////
byte[] digest = mac.doFinal(JsonUtils.toJson(value).getBytes(StandardCharsets.UTF_8));
return (String.format("%02x%02x", digest[0], digest[1]));
}
catch(java.security.GeneralSecurityException e)
{
throw (new IllegalStateException("Unable to fingerprint search", e));
}
}
/*******************************************************************************
** Stable user identity for all of that user's sessions.
*******************************************************************************/
public static String userId()
{
QSession session = QContext.getQSession();
return (session == null || session.getUser() == null ? "anonymous" : Objects.requireNonNullElse(session.getUser().getIdReference(), "anonymous"));
}
/*******************************************************************************
** Treat empty optional selectors like null.
*******************************************************************************/
private static boolean blank(String value)
{
return (value == null || value.isBlank());
}
}