Compare commits

...
Author SHA1 Message Date
james.maesandClaude Opus 5.5 ef5c096e84 fix: resolve virtual fields and keep the access-log worker off unfinished instances
- SecuredDataClassifications.effective looked fields up with
  QTableMetaData.getField, which throws for a virtual (computed) field, so
  any user read of a table with one failed (kof22's permalink sitePath and
  siteUrl on every content table, and API validation). Virtual fields now
  take the table classification.
- The access-log worker opened a QContext on every tick, even with an empty
  queue. QContext validates an unvalidated instance, and the worker starts
  while the host is still building it, so the first tick validated (and
  enriched) a half-built instance on another thread; one failure cancelled
  every later tick for good. Idle ticks now return before opening a
  context, and a failed tick is logged and retried.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 08:42:05 -05:00
james.maes 75b5f0b64f Merge pull request #17 from QRun-IO/ci/release-rc-secrets-inherit
ci(release-rc): pass secrets so the shared workflow can sign the RC tag
2026-10-16 08:05:16 -05:00
james.maesandClaude Opus 5.5 2cb37a067d ci(release-rc): pass secrets to the shared workflow so it can sign the RC tag
QRun-IO/.github#2 adds a tag-and-release job that signs vX with the
QRUN_TAG_SIGNING_KEY secret after Maven Central confirms the artifact.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 16:18:59 -05:00
james.maes 971ddbed2d Merge pull request #16 from QRun-IO/feature/ui-api-1.6
feat(ui-api): extension for QQQ UI API 1.6.0 (operationIds, BasicErrorResponseV1, QQQ 4.1.0-RC.15)
2026-10-15 16:17:33 -05:00
james.maesandClaude Opus 5.5 e2f5199e06 test(retention): canary leak scan without false positives
The canary was 16 random letters g-v and any 5-letter window counted as a
leak, so ordinary JSON words built from those letters (options, joins)
matched by chance and RetentionProofTest failed intermittently. The canary
now has 24 random letters and a leak is the whole marker or a 12-letter
window of its random part (16^12 values): chance matches are impossible,
truncated or re-encoded fragments of a real leak are still caught. A new
test pins both directions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 16:15:09 -05:00
james.maesandClaude Opus 5.5 214bd75298 chore: target QQQ 4.1.0-RC.15 (UI API 1.6.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 12:27:07 -05:00
james.maesandClaude Opus 5.5 6bb55f332a feat(ui-api): extension for QQQ UI API 1.6.0 (operationIds, BasicErrorResponseV1, QQQ 4.1.0-RC.14)
Regenerated against UI API 1.6.0: every operation has an operationId and
the standard error responses (BasicErrorResponseV1: error, code, message,
fieldErrors, traceId); the extension's own error schema is kept but
deprecated; hand-written error bodies go through QJavalinUtils so they carry
code/message/traceId. Extension MINOR bumped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 11:48:13 -05:00
james.maes 781bae3efe Merge pull request #15 from QRun-IO/feature/remove-standins
build: QQQ 4.1.0-RC.7; remove core stand-ins
2026-10-14 16:33:33 -05:00
16 changed files with 360 additions and 39 deletions

No files matched your search

+1
View File
@@ -21,5 +21,6 @@ jobs:
release-rc:
if: github.event_name == 'workflow_dispatch' || (github.event.pull_request.merged == true && contains(github.event.pull_request.labels.*.name, 'release:rc'))
uses: QRun-IO/.github/.github/workflows/qbit-release-rc.yml@main
secrets: inherit
with:
dry_run: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run }}
+3
View File
@@ -4,6 +4,9 @@ All notable changes are documented here, following [Keep a Changelog](https://ke
## [Unreleased]
### Changed: UI API 1.6.0
- UI API extension secured-data 1.1.0 (requires QQQ 4.1.0-RC.15, UI API 1.6.0): operationIds, standard error responses with BasicErrorResponseV1, extension tags, and retained SecuredDataErrorV1 schema deprecated since 1.1.0.
### Changed — direct core field access and audit protection
- Remove all #1172/#1182 stand-ins and reflective bridges; use core field-access, classified-value token, audit protection, wipe flag and shredded-event APIs directly.
- Register the resolver, key source and audit protector through core supplemental metadata.
+2 -2
View File
@@ -4,7 +4,7 @@ Phases (b), (c), (e) and (f) of [SECURED-DATA](https://github.com/QRun-IO/qqq/bl
## Compatibility
Java 21, Maven wrapper 3.9.11, and **QQQ 4.1.0-RC.7** or later (it ships field access #1172 and audit value protection #1182).
Java 21, Maven wrapper 3.9.11, and **QQQ 4.1.0-RC.15** or later (UI API 1.6.0, field access #1172 and audit value protection #1182).
Core classifications, AI metadata, client channels, table-feature permissions and UI API extension points are used directly. The UI API and its contract test are part of the main build; the contract test calls `UiApiContract` directly.
@@ -121,7 +121,7 @@ Configure `.withSchedulerName("yourScheduler")` to register the hidden `secured_
## UI API extension
The `secured-data` extension is version **1.0.0**:
The `secured-data` extension is version **1.1.0**:
- `POST /qqq/v1/qbit/secured-data/reveal`
- `GET /qqq/v1/qbit/secured-data/reveal/options?tableName=&fieldName=`
+2 -2
View File
@@ -22,8 +22,8 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<properties>
<!-- QQQ 4.1.0-RC.7 ships field access (#1172) and audit value protection (#1182) -->
<qqq.snapshot.version>4.1.0-RC.7</qqq.snapshot.version>
<!-- QQQ 4.1.0-RC.15 ships UI API 1.6.0, field access and audit value protection -->
<qqq.snapshot.version>4.1.0-RC.15</qqq.snapshot.version>
<revision>0.1.0-SNAPSHOT</revision>
<maven.compiler.source>21</maven.compiler.source>
<maven.compiler.target>21</maven.compiler.target>
@@ -104,7 +104,7 @@ public class SecuredDataAccessLogger implements AutoCloseable
return (thread);
});
long interval = runtime.config().getAccessLogFlushIntervalMillis();
worker.scheduleWithFixedDelay(this::flush, interval, interval, TimeUnit.MILLISECONDS);
worker.scheduleWithFixedDelay(this::scheduledFlush, interval, interval, TimeUnit.MILLISECONDS);
shutdownHook = new Thread(this::close, "secured-data-access-log-shutdown");
Runtime.getRuntime().addShutdownHook(shutdownHook);
}
@@ -332,13 +332,38 @@ public class SecuredDataAccessLogger implements AutoCloseable
/*******************************************************************************
** The worker's tick. An exception would cancel every later run of a
** scheduled task, so a failed drain is logged (value-free) and retried on
** the next tick; the rows stay queued.
*******************************************************************************/
void scheduledFlush()
{
try
{
flush();
}
catch(RuntimeException e)
{
LOG.warn("Unable to drain the secured-data access log; retrying on the next tick");
}
}
/*******************************************************************************
** Drain safely both from the worker and explicit test/host shutdown calls.
** An empty queue opens no context: QContext validates an unvalidated
** instance, and the worker starts while the host is still building it.
*******************************************************************************/
public void flush()
{
synchronized(flushLock)
{
if(queue.isEmpty())
{
return;
}
QContext.withTemporaryContext(new CapturedContext(instance, new QSystemUserSession(instance)), () ->
{
PendingLog pending;
@@ -54,7 +54,11 @@ public final class SecuredDataClassifications
{
return (DataClassification.valueOf(row.getValueString("classification")));
}
QFieldMetaData field = table.getField(fieldName);
////////////////////////////////////////////////////////////////////
// getFields, not getField: a virtual field is not in the map and //
// getField throws, while it still takes the table classification //
////////////////////////////////////////////////////////////////////
QFieldMetaData field = table.getFields() == null || fieldName == null ? null : table.getFields().get(fieldName);
DataClassification classification = field == null ? null : SecuredDataMetadata.classification(table, field);
return (classification == null ? SecuredDataMetadata.classification(table, null) : classification);
}
@@ -37,11 +37,11 @@ public final class SecuredDataExtension
}
/*******************************************************************************
** The secured-data UI API, version 1.0.0.
** The secured-data UI API, version 1.1.0.
*******************************************************************************/
public static MiddlewareSpecExtension defineExtension()
{
return (new MiddlewareSpecExtension().withName("secured-data").withSemanticVersion("1.0.0")
return (new MiddlewareSpecExtension().withName("secured-data").withSemanticVersion("1.1.0")
.withDescription("Reveal grants and break-glass status")
.withSpec(SecuredDataRevealSpecV1.class).withSpec(SecuredDataRevealOptionsSpecV1.class).withSpec(SecuredDataStatusSpecV1.class));
}
@@ -41,7 +41,7 @@ public class SecuredDataRevealOptionsSpecV1 extends AbstractJsonEndpointSpec
public BasicOperation defineBasicOperation()
{
return (new BasicOperation().withPath("/qbit/secured-data/reveal/options").withHttpMethod(HttpMethod.GET)
.withTag(() -> "Secured data").withShortSummary("RevealOptions"));
.withTag(() -> "secured-data").withShortSummary("RevealOptions"));
}
/*******************************************************************************
@@ -28,6 +28,7 @@ import com.kingsrook.qbits.secureddata.reveal.RevealService;
import com.kingsrook.qqq.backend.core.exceptions.QBadRequestException;
import com.kingsrook.qqq.middleware.javalin.specs.BasicOperation;
import com.kingsrook.qqq.middleware.javalin.specs.extensions.AbstractJsonEndpointSpec;
import com.kingsrook.qqq.middleware.javalin.specs.v1.responses.BasicErrorResponseV1;
import com.kingsrook.qqq.openapi.model.HttpMethod;
import com.kingsrook.qqq.openapi.model.Parameter;
import com.kingsrook.qqq.openapi.model.Schema;
@@ -44,7 +45,7 @@ public class SecuredDataRevealSpecV1 extends AbstractJsonEndpointSpec
public BasicOperation defineBasicOperation()
{
return (new BasicOperation().withPath("/qbit/secured-data/reveal").withHttpMethod(HttpMethod.POST)
.withTag(() -> "Secured data").withShortSummary("Reveal"));
.withTag(() -> "secured-data").withShortSummary("Reveal"));
}
/*******************************************************************************
@@ -81,10 +82,10 @@ public class SecuredDataRevealSpecV1 extends AbstractJsonEndpointSpec
@Override
public List<com.kingsrook.qqq.middleware.javalin.specs.BasicResponse> defineAdditionalBasicResponses()
{
return (List.of(new com.kingsrook.qqq.middleware.javalin.specs.BasicResponse(io.javalin.http.HttpStatus.BAD_REQUEST, "Invalid reveal request", "SecuredDataErrorV1"),
new com.kingsrook.qqq.middleware.javalin.specs.BasicResponse(io.javalin.http.HttpStatus.FORBIDDEN, "Reveal denied", "SecuredDataErrorV1"),
new com.kingsrook.qqq.middleware.javalin.specs.BasicResponse(io.javalin.http.HttpStatus.NOT_FOUND, "Record not found or locked", "SecuredDataErrorV1"),
new com.kingsrook.qqq.middleware.javalin.specs.BasicResponse(io.javalin.http.HttpStatus.TOO_MANY_REQUESTS, "Daily reveal cap reached", "SecuredDataErrorV1")));
return (List.of(new com.kingsrook.qqq.middleware.javalin.specs.BasicResponse(io.javalin.http.HttpStatus.BAD_REQUEST, "Invalid reveal request", BasicErrorResponseV1.class.getSimpleName()),
new com.kingsrook.qqq.middleware.javalin.specs.BasicResponse(io.javalin.http.HttpStatus.FORBIDDEN, "Reveal denied", BasicErrorResponseV1.class.getSimpleName()),
new com.kingsrook.qqq.middleware.javalin.specs.BasicResponse(io.javalin.http.HttpStatus.NOT_FOUND, "Record not found or locked", BasicErrorResponseV1.class.getSimpleName()),
new com.kingsrook.qqq.middleware.javalin.specs.BasicResponse(io.javalin.http.HttpStatus.TOO_MANY_REQUESTS, "Daily reveal cap reached", BasicErrorResponseV1.class.getSimpleName())));
}
/*******************************************************************************
@@ -58,7 +58,8 @@ public final class SecuredDataSchemas
schemas.put("SecuredDataRevealAllowanceV1", object("tableName", type(Type.STRING), "fieldName", type(Type.STRING), "remainingDailyAllowance", type(Type.INTEGER).withNullable(true)));
schemas.put("SecuredDataStatusResponseV1", object("breakGlass", new Schema().withRefToSchema("SecuredDataBreakGlassV1").withNullable(true),
"revealsUsedToday", type(Type.INTEGER), "revealAllowances", type(Type.ARRAY).withItems(new Schema().withRefToSchema("SecuredDataRevealAllowanceV1"))));
schemas.put("SecuredDataErrorV1", object("error", type(Type.STRING)));
schemas.put("SecuredDataErrorV1", object("error", type(Type.STRING))
.withDescription("Deprecated since 1.1.0: errors use BasicErrorResponseV1."));
return (schemas);
}
@@ -41,7 +41,7 @@ public class SecuredDataStatusSpecV1 extends AbstractJsonEndpointSpec
public BasicOperation defineBasicOperation()
{
return (new BasicOperation().withPath("/qbit/secured-data/status").withHttpMethod(HttpMethod.GET)
.withTag(() -> "Secured data").withShortSummary("Status"));
.withTag(() -> "secured-data").withShortSummary("Status"));
}
/*******************************************************************************
@@ -1,5 +1,44 @@
components:
schemas:
BasicErrorResponseV1:
description: "The error body of every UI API operation (core and extensions)."
properties:
error:
description: "The error message (the same as message; kept for clients of\
\ UI API 1.0 to 1.5)."
type: "string"
code:
description: "A stable, machine-readable code. Well-known values: BAD_REQUEST\
\ (400), UNAUTHENTICATED (401),\nPERMISSION_DENIED (403), NOT_FOUND (404),\
\ CONFLICT (409), RATE_LIMITED (429), INTERNAL_ERROR (500),\nERROR (other).\
\ Extensions may add codes; treat an unknown code by the HTTP status.\
\ Since 1.6.0."
type: "string"
message:
description: "A message to show the user. Since 1.6.0."
type: "string"
fieldErrors:
description: "Errors about particular request fields, when known. Since\
\ 1.6.0."
items:
properties:
field:
description: "The field (or parameter) name."
type: "string"
message:
description: "The error about that field."
type: "string"
code:
description: "An optional machine-readable code, e.g. REQUIRED, TOO_LONG,\
\ PATTERN."
type: "string"
type: "object"
type: "array"
traceId:
description: "Identifies this request in the server logs (also the X-QQQ-Trace-Id\
\ response header); quote it when reporting a problem. Since 1.6.0."
type: "string"
type: "object"
SecuredDataBreakGlassV1:
properties:
id:
@@ -16,6 +55,7 @@ components:
type: "string"
type: "object"
SecuredDataErrorV1:
description: "Deprecated since 1.1.0: errors use BasicErrorResponseV1."
properties:
error:
type: "string"
@@ -96,19 +136,53 @@ components:
type: "object"
securitySchemes:
sessionUuidCookie:
description: "The session cookie set by POST /qqq/v1/manageSession (web clients)."
in: "cookie"
name: "sessionUUID"
type: "apiKey"
bearerAuth:
bearerFormat: "JWT or session UUID"
description: "`Authorization: Bearer <token>`, where the token is either\n-\
\ an OAuth2 access token (a signed JWT) issued by the app's identity provider\
\ to the app's client,\n or to its native client (`values.nativeClientId`\
\ in GET /qqq/v1/metaData/authentication), for\n example from the device-authorization\
\ grant or auth-code + PKCE. It is verified against the\n provider's keys\
\ (issuer, audience, expiry, signature) and gives the same user and permissions\n\
\ as signing in on the web. When it expires (401), refresh it at the provider\
\ and retry; or\n- a session UUID returned by POST /qqq/v1/manageSession,\
\ which the server keeps refreshed."
scheme: "bearer"
type: "http"
oauth2:
description: "The app's OAuth2 / OpenID Connect provider. The real endpoints\
\ come from the provider's discovery\ndocument: `{values.baseUrl}.well-known/openid-configuration`,\
\ with values from\nGET /qqq/v1/metaData/authentication. Native clients use\
\ `values.nativeClientId` (a public client)\nwith auth-code + PKCE (loopback\
\ or custom-scheme redirect URI registered at the provider) or the\ndevice-authorization\
\ grant (`device_authorization_endpoint`, RFC 8628), then send the access\
\ token\nas `Authorization: Bearer`."
flows:
authorizationCode:
authorizationUrl: "/authorize"
scopes:
email: "Email address"
offline_access: "A refresh token, to renew the access token without signing\
\ in again"
openid: "OpenID Connect sign-in"
profile: "Name, username and groups (permissions come from the groups)"
tokenUrl: "/token"
type: "oauth2"
info:
contact:
email: "contact@kingsrook.com"
description: "Reveal grants and break-glass status"
title: "QQQ Middleware API extension: secured-data"
version: "1.0.0"
version: "1.1.0"
openapi: "3.0.3"
paths:
/qqq/v1/qbit/secured-data/reveal:
post:
operationId: "postQbitSecuredDataReveal"
requestBody:
content:
application/json:
@@ -120,19 +194,19 @@ paths:
content:
application/json:
schema:
$ref: "#/components/schemas/SecuredDataErrorV1"
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Invalid reveal request"
403:
content:
application/json:
schema:
$ref: "#/components/schemas/SecuredDataErrorV1"
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Reveal denied"
404:
content:
application/json:
schema:
$ref: "#/components/schemas/SecuredDataErrorV1"
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Record not found or locked"
200:
content:
@@ -144,16 +218,37 @@ paths:
content:
application/json:
schema:
$ref: "#/components/schemas/SecuredDataErrorV1"
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Daily reveal cap reached"
401:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not authenticated, or the session or token is invalid or expired\
\ (code UNAUTHENTICATED). Sign in again, or refresh the token."
409:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Conflict with the current state, e.g. an edit conflict or\
\ an invalid transition (code CONFLICT)."
security:
- sessionUuidCookie:
- "N/A"
- bearerAuth:
- "N/A"
- oauth2:
- "openid"
- "profile"
- "email"
summary: "Reveal"
tags:
- "Secured data"
- "secured-data"
/qqq/v1/qbit/secured-data/reveal/options:
get:
operationId: "getQbitSecuredDataRevealOptions"
parameters:
- in: "query"
name: "tableName"
@@ -172,14 +267,61 @@ paths:
schema:
$ref: "#/components/schemas/SecuredDataRevealOptionsResponseV1"
description: "Successful Response"
400:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Bad request: a parameter or the body is invalid (code BAD_REQUEST);\
\ fieldErrors may name the fields."
401:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not authenticated, or the session or token is invalid or expired\
\ (code UNAUTHENTICATED). Sign in again, or refresh the token."
403:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "The user (or the credential's scope) may not do this (code\
\ PERMISSION_DENIED)."
404:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found, or not visible to this user (code NOT_FOUND)."
409:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Conflict with the current state, e.g. an edit conflict or\
\ an invalid transition (code CONFLICT)."
429:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Too many requests (code RATE_LIMITED); retry later."
security:
- sessionUuidCookie:
- "N/A"
- bearerAuth:
- "N/A"
- oauth2:
- "openid"
- "profile"
- "email"
summary: "RevealOptions"
tags:
- "Secured data"
- "secured-data"
/qqq/v1/qbit/secured-data/status:
get:
operationId: "getQbitSecuredDataStatus"
responses:
200:
content:
@@ -187,9 +329,61 @@ paths:
schema:
$ref: "#/components/schemas/SecuredDataStatusResponseV1"
description: "Successful Response"
400:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Bad request: a parameter or the body is invalid (code BAD_REQUEST);\
\ fieldErrors may name the fields."
401:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not authenticated, or the session or token is invalid or expired\
\ (code UNAUTHENTICATED). Sign in again, or refresh the token."
403:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "The user (or the credential's scope) may not do this (code\
\ PERMISSION_DENIED)."
404:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Not found, or not visible to this user (code NOT_FOUND)."
409:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Conflict with the current state, e.g. an edit conflict or\
\ an invalid transition (code CONFLICT)."
429:
content:
application/json:
schema:
$ref: "#/components/schemas/BasicErrorResponseV1"
description: "Too many requests (code RATE_LIMITED); retry later."
security:
- sessionUuidCookie:
- "N/A"
- bearerAuth:
- "N/A"
- oauth2:
- "openid"
- "profile"
- "email"
summary: "Status"
tags:
- "Secured data"
- "secured-data"
servers:
- description: "The QQQ server that serves this document (paths include /qqq/v1)."
url: "/"
tags:
- description: "Operations of the secured-data extension."
name: "secured-data"
@@ -84,13 +84,23 @@ class RetentionProofTest
private static final String CANARY_PREFIX = "CANARY-PHI-";
private static final int CANARY_RANDOM = 24;
private static final int MIN_LEAK_FRAGMENT = 12;
/*******************************************************************************
** Letters g-v only, so a marker can never coincide with hex digests or tokens.
** A fixed prefix and 24 random letters g-v (so it never coincides with hex
** digests). Leaks are matched on the whole marker or on 12-letter windows of
** its random part: 16^12 possibilities, so ordinary text, JSON words or
** base64 tokens cannot match by chance, while a truncated or re-encoded
** fragment of a real leak still does.
*******************************************************************************/
private static String canary()
{
StringBuilder marker = new StringBuilder("CANARY-PHI-");
for(int i = 0; i < 16; i++)
StringBuilder marker = new StringBuilder(CANARY_PREFIX);
for(int i = 0; i < CANARY_RANDOM; i++)
{
marker.append((char) ('g' + RANDOM.nextInt(16)));
}
@@ -100,21 +110,25 @@ class RetentionProofTest
/*******************************************************************************
** The marker, or any substring of it longer than four characters.
** The whole marker, or any MIN_LEAK_FRAGMENT-letter window of its random part.
*******************************************************************************/
private static List<String> leaks(String haystack, String... markers)
{
List<String> found = new ArrayList<>();
for(String marker : markers)
{
for(int length = 5; length <= marker.length(); length++)
if(haystack.contains(marker))
{
for(int start = 0; start + length <= marker.length(); start++)
found.add("whole marker");
continue;
}
String random = marker.substring(CANARY_PREFIX.length());
for(int start = 0; start + MIN_LEAK_FRAGMENT <= random.length(); start++)
{
if(haystack.contains(random.substring(start, start + MIN_LEAK_FRAGMENT)))
{
if(haystack.contains(marker.substring(start, start + length)))
{
found.add("length " + length + " at " + start);
}
found.add("fragment at " + start);
}
}
}
@@ -123,6 +137,23 @@ class RetentionProofTest
/*******************************************************************************
** The leak scan: whole markers and 12-letter fragments are found; the shared
** prefix and ordinary words made of the same letters are not.
*******************************************************************************/
@Test
void leakScanHasNoFalsePositives()
{
String marker = canary();
String random = marker.substring(CANARY_PREFIX.length());
assertThat(leaks("x" + marker + "x", marker)).containsExactly("whole marker");
assertThat(leaks("{\"v\":\"" + random.substring(5, 5 + MIN_LEAK_FRAGMENT) + "\"}", marker)).isNotEmpty();
assertThat(leaks(CANARY_PREFIX + " options joins strings rows points", marker)).isEmpty();
assertThat(leaks(random.substring(0, MIN_LEAK_FRAGMENT - 1), marker)).isEmpty();
}
/*******************************************************************************
** Canary through CRUD, a failing validation, a wipe run, a masked export and
** the access log: it may appear only in the host table itself.
@@ -46,6 +46,7 @@ import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import com.kingsrook.qqq.backend.core.utils.JsonUtils;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Test;
@@ -359,4 +360,31 @@ class SecuredDataAccessLoggerTest
{
}
}
/*******************************************************************************
** The worker starts while the host is still building its instance. An idle
** tick opens no context, so it never validates (or enriches) an unfinished
** instance on the worker thread, and a failing tick is logged instead of
** cancelling every later tick.
*******************************************************************************/
@Test
void idleTicksNeverValidateAnUnfinishedInstance() throws Exception
{
setup();
SecuredDataRuntime runtime = SecuredDataRuntime.current();
QInstance unfinished = SecuredDataTestSupport.base();
unfinished.addTable(new QTableMetaData().withName("unfinished").withBackendName("not_registered_yet"));
SecuredDataAccessLogger idle = new SecuredDataAccessLogger(unfinished, runtime);
try
{
idle.flush();
idle.scheduledFlush();
assertThat(unfinished.getHasBeenValidated()).isFalse();
}
finally
{
idle.close();
}
}
}
@@ -109,11 +109,25 @@ class SecuredDataHttpTest
var status = request("/qbit/secured-data/status", null, false);
assertThat(status.statusCode()).as(status.body()).isEqualTo(200);
assertThat(new JSONObject(status.body()).getInt("revealsUsedToday")).isEqualTo(1);
assertThat(request("/qbit/secured-data/reveal", body, false).statusCode()).isEqualTo(429);
assertThat(request("/qbit/secured-data/reveal", body, true).statusCode()).isEqualTo(403);
assertThat(request("/qbit/secured-data/reveal", "{}", false).statusCode()).isEqualTo(400);
assertThat(request("/qbit/secured-data/reveal", "{\"primaryKey\":1}", false).statusCode()).isEqualTo(400);
assertThat(request("/qbit/secured-data/reveal/options?tableName=person&fieldName=missing", null, false).statusCode()).isEqualTo(404);
assertErrorResponse(request("/qbit/secured-data/reveal", body, false), 429);
assertErrorResponse(request("/qbit/secured-data/reveal", body, true), 403);
assertErrorResponse(request("/qbit/secured-data/reveal", "{}", false), 400);
assertErrorResponse(request("/qbit/secured-data/reveal", "{\"primaryKey\":1}", false), 400);
assertErrorResponse(request("/qbit/secured-data/reveal/options?tableName=person&fieldName=missing", null, false), 404);
}
/*******************************************************************************
** Verify standard error bodies without changing endpoint status semantics.
*******************************************************************************/
private void assertErrorResponse(HttpResponse<String> response, int status)
{
assertThat(response.statusCode()).as(response.body()).isEqualTo(status);
JSONObject body = new JSONObject(response.body());
assertThat(body.keySet()).contains("error", "code", "message", "traceId");
assertThat(body.getString("error")).isNotBlank();
assertThat(body.getString("code")).isNotBlank();
assertThat(body.getString("message")).isNotBlank();
assertThat(body.getString("traceId")).isNotBlank();
}
@Test
@@ -33,6 +33,7 @@ import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QVirtualFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
@@ -326,4 +327,22 @@ class SecuredDataResolverTest
CLOCK.now = NOW.plusSeconds(10);
assertThat(resolver.isRevealActive("person", "email", 7)).isFalse();
}
/*******************************************************************************
** A virtual (computed) field is not in getFields, and getField throws for
** it: it resolves through the table classification instead of failing the
** whole read.
*******************************************************************************/
@Test
void virtualFieldsTakeTheTableClassification() throws Exception
{
QVirtualFieldMetaData display = new QVirtualFieldMetaData("displayName", QFieldType.STRING);
person.withVirtualFields(List.of(display));
assertThat(SecuredDataClassifications.effective(person, "displayName")).isNull();
assertThat(resolver.resolve(person, display).tier()).isEqualTo(FieldAccessTier.FULL);
person.withSupplementalMetaData(new AiTableMetaData().withClassification(DataClassification.PII));
assertThat(SecuredDataClassifications.effective(person, "displayName")).isEqualTo(DataClassification.PII);
assertThat(resolver.resolve(person, display).tier()).isEqualTo(FieldAccessTier.MASKED);
}
}