diff --git a/CHANGELOG.md b/CHANGELOG.md index 58809e8..da4bb7b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,19 @@ All notable changes are documented here, following [Keep a Changelog](https://ke ### Added +- Phase (e) configurable reveal-median, distinct-record and exact-match anomaly detection, with system access-log counters, bounded daily median caching, per-user/day alerts, and flags retained without an alert sink. +- Break-glass start and capped-reveal anomaly flags alongside the existing alerts. +- One Secured Data app home with six live KPIs, six full-width responsive lists, filtered View all links, theme tokens, compact dates, clamped text/tooltips and explicit empty states. +- Transaction-aware, value-free policy/classification change audit rows from post-write customizers and two access-log indexes for anomaly/home queries. +- Boundary, alert, home data, rendering and navigation tests plus a synthetic Next demo harness for coordinator screenshot capture. + +- Phase (c) HTTP-independent reveal options, reason validation, UTC daily limits, grants, audit links, cap alerts and assistant refusal. +- Five-second per-user grant cache with immediate invalidation and read-time expiration. +- Start/End break-glass table and record actions, streamed Review/Result summaries, owner/admin authorization, scheduled expiration, alerts and optional admin bell notification. +- Caller status and the versioned `secured-data` UI API extension behind the temporary `uiapi` Maven profile, with committed OpenAPI and contract/HTTP tests. +- Contingent classification field picker support for snake_case and camelCase table selections; unknown-table rejection. +- Memory and real-core coverage for reveal, break-glass, record locks and expiry. Reason columns support 500 characters; existing RDBMS installations must widen the three `reason` columns to VARCHAR(500). + - Phase (b) secured-data classification, policy, reveal-grant, break-glass and insert-only access-log tables, metadata, permissions, pickers and app navigation. - Versioned classification and policy resolution with channel caps, table features, break-glass expiry and access fingerprints. - Bounded asynchronous value-free logging, HMAC search fingerprints, rate limits, entropy rules and enumeration alerts. diff --git a/README.md b/README.md index 916e9ba..42619ea 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # QBit Secured Data -Phase (b) of [SECURED-DATA](https://github.com/QRun-IO/qqq/blob/develop/docs/SECURED-DATA.md): classification overrides, policy resolution, enumeration guardrails, and value-free access logging. QQQ core owns masking and query enforcement. Reveal/Start/End operations, home widgets, retention and audit-value protection are later phases. +Phases (b), (c), and (e) of [SECURED-DATA](https://github.com/QRun-IO/qqq/blob/develop/docs/SECURED-DATA.md): classification overrides, policy resolution, enumeration guardrails, and value-free access logging. QQQ core owns masking and query enforcement. Reveal, break-glass, the app home and anomaly alerts are implemented. Retention and audit-value protection remain later phases. ## Compatibility @@ -107,6 +107,32 @@ One daemon worker drains a bounded queue as a system session. Enqueueing does no Flagged rows raise the optional alerts producer signal `secured_data.enumeration`, deduplicated by `secured_data.enumeration::.:`. RC.5 already provides `QAlerts`; without an installed sink it logs the signal. Its catalog validator rejects underscores in type names, so this mandated type is sent through the producer without catalog registration until that core restriction is reconciled. No values or fingerprints are sent to the alert sink. +## Reveal and break-glass + +`new RevealService().options(tableName, fieldName)` describes the reason choices, free-text rules, duration and remaining allowance. `reveal(tableName, primaryKey, fieldName, reason, reasonText)` performs an explicitly permission-enforced Get as the caller before a system read. It returns `value`, core-formatted `displayValue`, `expiresAt`, and `revealId`. Grants and their linked audit rows are system-written; no raw value is copied into the access log. + +The effective tier must be REVEALABLE. The chosen granting policy is the row with the highest daily cap (null means unlimited), then the longest duration, then lowest lexical id. Channel-only cap rows never grant reveal settings. A feature/default-based REVEALABLE tier with no granting row uses five minutes and an allowed free-text reason. Limits count grants across all tables for the user in the UTC day. Reason and reason text each have a 500-character limit. A required explanation is accepted alongside a listed reason even if arbitrary free-text reasons are disabled. Capped requests record `REVEAL_DENIED/CAPPED` and raise `secured_data.reveal_capped`; assistant-channel requests record `REVEAL_DENIED/DENIED` and return 403. + +Grant lookups cache identifiers and expiration times for at most five seconds per user, check expiration on each use, and invalidate immediately after a new grant. The service serializes cap-check/write and duplicate-session checks within one runtime. Multi-replica deployments must coordinate these operations externally for a globally atomic cap and single active session; the in-process lock is not a distributed lock. + +`secured_data_break_glass_start` is a table action, labeled **Start break-glass**, requiring `secured_data_break_glass_start.hasAccess`. Its reason must contain 20-500 characters; duration defaults to 60 minutes and accepts 1-480. Table scope is optional and classification ceiling defaults to PHI. `secured_data_break_glass_end` is a record action with the standard streamed ETL Review/Result screens; ownership/admin authorization is checked at review and again at execution. Neither action is a navigation entry. The current admin-only activity table permissions remain in place; an owner can invoke End using the session id returned by status without obtaining admin access to activity tables. + +Start raises `secured_data.break_glass` at core's `HIGH` mapping (`AlertSeverity.ERROR`). End and expiry resolve the same per-session alert. Bell delivery targets holders of `secured_data.admin` through core's optional notification SPI #1125. RC.5 and the installed secure-data snapshot lack that SPI: a clear warning is logged. TODO: switch to a direct notification call when #1125 ships and verify against a notification-capable core. + +Configure `.withSchedulerName("yourScheduler")` to register the hidden `secured_data_break_glass_expire` process every minute. Without a scheduler, the host must invoke that process itself. Tier resolution checks expiration independently, so delayed maintenance cannot prolong access. `new StatusService().status()` returns the caller's active session or null, grants used today, and the allowance for each readable REVEALABLE field. + +## UI API extension + +The `secured-data` extension is version **1.0.0**: + +- `POST /qqq/v1/qbit/secured-data/reveal` +- `GET /qqq/v1/qbit/secured-data/reveal/options?tableName=&fieldName=` +- `GET /qqq/v1/qbit/secured-data/status` + +Build with `-Duiapi=true` and a compatible core to compile `src/main/java-uiapi` and `src/test/java-uiapi`. The producer registers the extension only when both its classes and core's extension metadata exist. The committed contract is [secured-data.yaml](src/main/resources/openapi/qqq-v1/secured-data.yaml); regenerate intentionally with `-Dqqq.uiApi.regenerate=true`. The supplied core exposes `UiApiContract.check`, so a local test-only `UiApiContractAssertions` adapter supplies the documented assertion facade. All UI API roots and adapters are `STAND-IN(core=QRun-IO/qqq#1128, removeBy=RC.6)` and fold into the normal roots at RC.6. The core OpenAPI model cannot mark object properties required; the request schema documents required properties and the handler validates them. + +The classification field picker uses CUSTOM possible values, reading `table_name` or `tableName` from the form's other values. It returns only the selected readable table's labeled fields, returns none without a selection, and rejects an unknown table. TODO: add `dependsOn("table_name")` when the core metadata API supports it. + ## Configuration | Fluent option | Default / purpose | @@ -124,6 +150,7 @@ Flagged rows raise the optional alerts producer signal `secured_data.enumeration | `withMissAlertConsecutive(int)` | 5; zero disables | | `withDistinctValueAlert(int)` | 15; zero disables | | `withMinEntropyEnabled(boolean)` | true | +| `withSchedulerName(String)` | Host scheduler for the 60-second break-glass expiry job; null means host-managed invocation | | `withClock(Clock)` | UTC system clock; injectable for deterministic expiry tests | Configure before producing the QBit; runtime infrastructure and HMAC keys are captured at startup. Classification/policy/break-glass lookups always run as system, with a thread-local recursion guard. No reveal values are cached. @@ -131,15 +158,56 @@ Configure before producing the QBit; runtime infrastructure and HMAC keys are ca ## Build and tests ```powershell -./mvnw.cmd -q -o verify -./mvnw.cmd -q -o -Dqqq.snapshot.version=4.1.0-SECUREDATA-SNAPSHOT verify +./mvnw.cmd -q -o clean verify +./mvnw.cmd -q -o clean verify -Dqqq.snapshot.version=4.1.0-SECUREDATA-SNAPSHOT -Duiapi=true ``` The direct suite uses memory and H2. `CoreFieldAccessBridgeIT` is included by Surefire and uses JUnit assumptions to skip unless the actual core resolver metadata is present. With the installed core snapshot it exercises Query/Get, policy tiers, masking, exact matches and outcomes, sorting denial, rate/entropy refusal and the assistant wall. Checkstyle and JaCoCo remain 70% instructions / 90% classes; stand-ins are not excluded. +## Anomaly monitoring (phase e) + +Application REVEAL and queued EXACT_MATCH_FILTER audit writes evaluate persisted access-log counters as system, counting the current row once. Successful reveals flag above three times the user's median across the previous 30 complete UTC days (including zero days), with at least ten reveals today. More than 20 distinct `(table, record)` pairs within the last hour also flag. More than 50 exact-match attempts on one restricted field within the last hour flag independently of enumeration controls. The default rate limit of 20 usually refuses searches first; refused attempts still count. + +Break-glass starts and capped reveal refusals also flag. Each raises `secured_data.anomaly` with dedup key `secured_data.anomaly::` through the optional alerts SPI. Existing enumeration, break-glass and cap alerts remain intact. No qbit-alerts dependency is required; flags remain visible without a sink. The sink owns deduplication. End and expiry events remain in the break-glass activity list rather than representing new session anomalies. + +Thresholds are positive integers: + +| Config method | Default | +| --- | ---: | +| `withAnomalyDailyMedianMultiplier` | 3 | +| `withAnomalyMinimumDailyReveals` | 10 | +| `withAnomalyDistinctRecordsPerHour` | 20 | +| `withAnomalyExactMatchesPerHour` | 50 | + +Medians use 30 scalar indexed counts on first qualifying use per user/day, cached for at most 2,048 users per instance and cleared at UTC day rollover. Today's and hourly counts read persisted history; distinct-record queries return at most the threshold plus one grouped rows. Counter evaluation and insertion serialize within an instance; other nodes become visible through persisted rows. This monitors activity rather than providing a distributed enforcement lock. Historical backfills after caching take effect next UTC day or after runtime restart. Existing deployments should apply the two new access-log indexes in `docs/ddl/secured-data-postgres.sql`. + +## Secured Data home + +The single Secured Data app owns one full-width HTML home widget, following qbit-webhooks' data/view/renderer pattern. It reads as the viewer with normal permissions and record security: + +- Six KPIs: reveals today, unexpired grants, active and unexpired break-glass sessions, anomaly flags and denied filters over seven days, and this runtime's access-log queue drops since startup. Denied filters include refused exact-match attempts. +- Six full-width lists: recent reveals (10), top revealers and reasons (30-day backend aggregates, top 10), recent break-glass starts/ends/expiries (10), policy/classification changes (10), and seven-day anomaly flags (10). +- Each list has a filtered access-log `View all` link without the widget's ten-row limit. Configure the host table route with `withTablePathPrefix("/app")`. + +Lists order newest first with id tie-breaking; rankings order by count descending then group alphabetically. Narrow screens use wrapping labelled card rows. Dates stay compact on one line, text is ellipsized with full tooltips, and colors inherit the host with `--qqq-grid-border` for borders. There are no inner scrollers. Empty lists explicitly say there is no activity. + +Navigation contains Classification, Policy, Access log and Break-glass sessions. Reveal grants remain a child list; retention is not implemented. Successful policy/classification inserts, updates and deletes produce value-free change rows with actor, record id and action in the source transaction. No configuration values are copied into these rows. + +### Local screenshot harness + +`SecuredDataDemoServer` is a test-only memory-backed Next demo adapted from qbit-webhooks. It seeds synthetic activity for all six lists, including long reasons, sessions and anomaly flags. It uses a demo administrator and must only be run locally. After a test run, launch in PowerShell: + +```powershell +[xml]$report = Get-Content target/surefire-reports/TEST-com.kingsrook.qbits.secureddata.SecuredDataHomeTest.xml +$demoClasspath = ($report.testsuite.properties.property | Where-Object name -eq 'java.class.path').value +java -cp $demoClasspath com.kingsrook.qbits.secureddata.SecuredDataDemoServer +``` + +Open `http://localhost:8098/app/secured_data`. Stop the demo before `clean` on Windows. Capture light and dark at 390, 768, 1024, 1440, 1920 and 3200 px under `target/screenshots/`. This session started the server, but screenshot capture was blocked by Chromium `spawn EPERM` in the sandbox and two connected-browser timeouts. The coordinator still needs to capture and inspect the twelve images through Next. + ## Remaining phases -RC.6 migration removes temporary types and reflection. Phase (c) adds Reveal and Start/End break-glass operations and expiry scheduling; this phase already reads active grants/sessions. Phase (e) adds the app home and broader anomaly rules. Retention, wipe and audit-value protection remain phases (f)/(g). +RC.6 migration removes temporary types and reflection. Retention, wipe and audit-value protection remain phases (f)/(g). ## License diff --git a/docs/ddl/secured-data-postgres.sql b/docs/ddl/secured-data-postgres.sql index f50bffd..2bcf9c8 100644 --- a/docs/ddl/secured-data-postgres.sql +++ b/docs/ddl/secured-data-postgres.sql @@ -47,7 +47,7 @@ CREATE TABLE secured_data_reveal_grant ( table_name VARCHAR(255) NOT NULL, record_id VARCHAR(255) NOT NULL, field_name VARCHAR(255) NOT NULL, - reason VARCHAR(255), + reason VARCHAR(500), reason_text TEXT, granted_at TIMESTAMP WITH TIME ZONE NOT NULL, expires_at TIMESTAMP WITH TIME ZONE NOT NULL, @@ -58,7 +58,7 @@ CREATE TABLE secured_data_break_glass ( id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, user_id VARCHAR(255) NOT NULL, user_name VARCHAR(255), - reason VARCHAR(255), + reason VARCHAR(500), reason_text TEXT, classification_ceiling VARCHAR(255) DEFAULT 'PHI', table_name VARCHAR(255), @@ -79,7 +79,7 @@ CREATE TABLE secured_data_access_log ( field_name TEXT, record_count INTEGER, action_name VARCHAR(255), - reason VARCHAR(255), + reason VARCHAR(500), reason_text TEXT, client_channel VARCHAR(255), outcome VARCHAR(255), @@ -93,3 +93,7 @@ CREATE INDEX secured_data_policy_lookup ON secured_data_policy (is_active, permi CREATE INDEX secured_data_break_glass_lookup ON secured_data_break_glass (user_id, status, expires_at); CREATE INDEX secured_data_reveal_lookup ON secured_data_reveal_grant (user_id, table_name, field_name, record_id, expires_at); CREATE INDEX secured_data_access_lookup ON secured_data_access_log (user_id, table_name, field_name, event_type, created_at); + +-- Phase (e): bounded user counters and home time windows. +CREATE INDEX secured_data_anomaly_lookup ON secured_data_access_log (user_id, event_type, created_at); +CREATE INDEX secured_data_home_lookup ON secured_data_access_log (event_type, created_at); diff --git a/pom.xml b/pom.xml index b7513cb..1e89ae3 100644 --- a/pom.xml +++ b/pom.xml @@ -22,6 +22,7 @@ + 4.1.0-RC.5 1.0.0-SNAPSHOT 21 21 @@ -203,28 +204,44 @@ + + + + com.kingsrook.qqq + qqq-bom-pom + ${qqq.snapshot.version} + pom + import + + + - + - qqq-snapshot - - true - - - - 4.1.0-RC.5 - - - - - com.kingsrook.qqq - qqq-bom-pom - ${qqq.snapshot.version} - pom - import - - - + uiapi + uiapitrue + + org.apache.maven.plugins + maven-checkstyle-plugin + + + ${project.basedir}/src/main/java + ${project.basedir}/src/main/java-uiapi + + + + org.codehaus.mojo + build-helper-maven-plugin + 3.6.1 + + uiapi-sourcegenerate-sourcesadd-source + src/main/java-uiapi + + uiapi-testsgenerate-test-sourcesadd-test-source + src/test/java-uiapi + + + diff --git a/src/main/java-uiapi/com/kingsrook/qbits/secureddata/uiapi/SecuredDataExtension.java b/src/main/java-uiapi/com/kingsrook/qbits/secureddata/uiapi/SecuredDataExtension.java new file mode 100644 index 0000000..189aa0f --- /dev/null +++ b/src/main/java-uiapi/com/kingsrook/qbits/secureddata/uiapi/SecuredDataExtension.java @@ -0,0 +1,56 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.secureddata.uiapi; + +import com.kingsrook.qqq.backend.core.model.metadata.QInstance; +import com.kingsrook.qqq.middleware.javalin.specs.extensions.MiddlewareSpecExtension; +import com.kingsrook.qqq.middleware.javalin.specs.extensions.MiddlewareSpecExtensionsMetaData; + +/******************************************************************************* + ** STAND-IN(core=QRun-IO/qqq#1128, removeBy=RC.6): move to main source root. + *******************************************************************************/ +public final class SecuredDataExtension +{ + /******************************************************************************* + ** Utility class. + *******************************************************************************/ + private SecuredDataExtension() + { + } + + /******************************************************************************* + ** The secured-data UI API, version 1.0.0. + *******************************************************************************/ + public static MiddlewareSpecExtension defineExtension() + { + return (new MiddlewareSpecExtension().withName("secured-data").withSemanticVersion("1.0.0") + .withDescription("Reveal grants and break-glass status") + .withSpec(SecuredDataRevealSpecV1.class).withSpec(SecuredDataRevealOptionsSpecV1.class).withSpec(SecuredDataStatusSpecV1.class)); + } + + /******************************************************************************* + ** Called reflectively by the producer. + *******************************************************************************/ + public static void register(QInstance instance) + { + MiddlewareSpecExtensionsMetaData.ofOrWithNew(instance).withExtension(defineExtension()); + } +} diff --git a/src/main/java-uiapi/com/kingsrook/qbits/secureddata/uiapi/SecuredDataRevealOptionsSpecV1.java b/src/main/java-uiapi/com/kingsrook/qbits/secureddata/uiapi/SecuredDataRevealOptionsSpecV1.java new file mode 100644 index 0000000..87c041b --- /dev/null +++ b/src/main/java-uiapi/com/kingsrook/qbits/secureddata/uiapi/SecuredDataRevealOptionsSpecV1.java @@ -0,0 +1,84 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.secureddata.uiapi; + +import java.util.List; +import java.util.Map; +import com.kingsrook.qbits.secureddata.reveal.RevealService; +import com.kingsrook.qqq.middleware.javalin.specs.BasicOperation; +import com.kingsrook.qqq.middleware.javalin.specs.extensions.AbstractJsonEndpointSpec; +import com.kingsrook.qqq.openapi.model.HttpMethod; +import com.kingsrook.qqq.openapi.model.Parameter; +import com.kingsrook.qqq.openapi.model.Schema; + +/******************************************************************************* + ** GET secured-data/reveal/options. + *******************************************************************************/ +public class SecuredDataRevealOptionsSpecV1 extends AbstractJsonEndpointSpec +{ + /******************************************************************************* + ** Operation metadata. + *******************************************************************************/ + @Override + public BasicOperation defineBasicOperation() + { + return (new BasicOperation().withPath("/qbit/secured-data/reveal/options").withHttpMethod(HttpMethod.GET) + .withTag(() -> "Secured data").withShortSummary("RevealOptions")); + } + + /******************************************************************************* + ** Request query parameters. + *******************************************************************************/ + @Override + public List defineRequestParameters() + { + return (List.of(new Parameter().withName("tableName").withIn(com.kingsrook.qqq.openapi.model.In.QUERY).withRequired(true).withSchema(new Schema().withType(com.kingsrook.qqq.openapi.model.Type.STRING)), + new Parameter().withName("fieldName").withIn(com.kingsrook.qqq.openapi.model.In.QUERY).withRequired(true).withSchema(new Schema().withType(com.kingsrook.qqq.openapi.model.Type.STRING)))); + } + + /******************************************************************************* + ** Published component schemas. + *******************************************************************************/ + @Override + public Map defineComponentSchemas() + { + Map schemas = SecuredDataSchemas.all(); + return (Map.of("SecuredDataRevealOptionsResponseV1", schemas.get("SecuredDataRevealOptionsResponseV1"))); + } + + /******************************************************************************* + ** Response component. + *******************************************************************************/ + @Override + protected String getResponseSchemaName() + { + return ("SecuredDataRevealOptionsResponseV1"); + } + + /******************************************************************************* + ** Execute with the caller session supplied by the core middleware. + *******************************************************************************/ + @Override + protected Object handle(JsonInput input) throws Exception + { + return (new RevealService().options(input.getQueryParam("tableName"), input.getQueryParam("fieldName"))); + } +} diff --git a/src/main/java-uiapi/com/kingsrook/qbits/secureddata/uiapi/SecuredDataRevealSpecV1.java b/src/main/java-uiapi/com/kingsrook/qbits/secureddata/uiapi/SecuredDataRevealSpecV1.java new file mode 100644 index 0000000..aa0b467 --- /dev/null +++ b/src/main/java-uiapi/com/kingsrook/qbits/secureddata/uiapi/SecuredDataRevealSpecV1.java @@ -0,0 +1,143 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.secureddata.uiapi; + +import java.io.Serializable; +import java.util.List; +import java.util.Map; +import com.kingsrook.qbits.secureddata.reveal.RevealCappedException; +import com.kingsrook.qbits.secureddata.reveal.RevealService; +import com.kingsrook.qqq.backend.core.exceptions.QBadRequestException; +import com.kingsrook.qqq.middleware.javalin.specs.BasicOperation; +import com.kingsrook.qqq.middleware.javalin.specs.extensions.AbstractJsonEndpointSpec; +import com.kingsrook.qqq.openapi.model.HttpMethod; +import com.kingsrook.qqq.openapi.model.Parameter; +import com.kingsrook.qqq.openapi.model.Schema; + +/******************************************************************************* + ** POST secured-data/reveal. + *******************************************************************************/ +public class SecuredDataRevealSpecV1 extends AbstractJsonEndpointSpec +{ + /******************************************************************************* + ** Operation metadata. + *******************************************************************************/ + @Override + public BasicOperation defineBasicOperation() + { + return (new BasicOperation().withPath("/qbit/secured-data/reveal").withHttpMethod(HttpMethod.POST) + .withTag(() -> "Secured data").withShortSummary("Reveal")); + } + + /******************************************************************************* + ** Request query parameters. + *******************************************************************************/ + @Override + public List defineRequestParameters() + { + return (List.of()); + } + + /******************************************************************************* + ** Published component schemas. + *******************************************************************************/ + @Override + public Map defineComponentSchemas() + { + Map schemas = SecuredDataSchemas.all(); + return (Map.of("SecuredDataRevealRequestV1", schemas.get("SecuredDataRevealRequestV1"), "SecuredDataRevealResponseV1", schemas.get("SecuredDataRevealResponseV1"), "SecuredDataErrorV1", schemas.get("SecuredDataErrorV1"))); + } + + /******************************************************************************* + ** Response component. + *******************************************************************************/ + @Override + protected String getResponseSchemaName() + { + return ("SecuredDataRevealResponseV1"); + } + + /******************************************************************************* + ** Standard error responses, including the daily-limit refusal. + *******************************************************************************/ + @Override + public List defineAdditionalBasicResponses() + { + return (List.of(new com.kingsrook.qqq.middleware.javalin.specs.BasicResponse(io.javalin.http.HttpStatus.BAD_REQUEST, "Invalid reveal request", "SecuredDataErrorV1"), + new com.kingsrook.qqq.middleware.javalin.specs.BasicResponse(io.javalin.http.HttpStatus.FORBIDDEN, "Reveal denied", "SecuredDataErrorV1"), + new com.kingsrook.qqq.middleware.javalin.specs.BasicResponse(io.javalin.http.HttpStatus.NOT_FOUND, "Record not found or locked", "SecuredDataErrorV1"), + new com.kingsrook.qqq.middleware.javalin.specs.BasicResponse(io.javalin.http.HttpStatus.TOO_MANY_REQUESTS, "Daily reveal cap reached", "SecuredDataErrorV1"))); + } + + /******************************************************************************* + ** Required JSON request. + *******************************************************************************/ + @Override + protected Schema defineRequestBodySchema() + { + return (new Schema().withRefToSchema("SecuredDataRevealRequestV1")); + } + + /******************************************************************************* + ** Reject wrong JSON types with the standard 400 response. + *******************************************************************************/ + private String string(Map body, String key) throws QBadRequestException + { + Object value = body.get(key); + if(value != null && !(value instanceof String)) + { + throw (new QBadRequestException(key + " must be a string.")); + } + return ((String) value); + } + + /******************************************************************************* + ** Execute with the caller session supplied by the core middleware. + *******************************************************************************/ + @Override + protected Object handle(JsonInput input) throws Exception + { + Map body = input.getBody(); + Object pk = body.get("primaryKey"); + if(!(pk instanceof String || pk instanceof Number)) + { + throw (new QBadRequestException("primaryKey must be a string or number.")); + } + return (new RevealService().reveal(string(body, "tableName"), (Serializable) pk, string(body, "fieldName"), + string(body, "reason"), string(body, "reasonText"))); + } + + /******************************************************************************* + ** Preserve core error bodies and use 429 for exhausted daily allowances. + *******************************************************************************/ + @Override + protected void handleException(io.javalin.http.Context context, Exception e) + { + if(e instanceof RevealCappedException) + { + com.kingsrook.qqq.middleware.javalin.QJavalinUtils.respondWithError(context, org.eclipse.jetty.http.HttpStatus.Code.TOO_MANY_REQUESTS, e.getMessage()); + } + else + { + super.handleException(context, e); + } + } +} diff --git a/src/main/java-uiapi/com/kingsrook/qbits/secureddata/uiapi/SecuredDataSchemas.java b/src/main/java-uiapi/com/kingsrook/qbits/secureddata/uiapi/SecuredDataSchemas.java new file mode 100644 index 0000000..152dc0d --- /dev/null +++ b/src/main/java-uiapi/com/kingsrook/qbits/secureddata/uiapi/SecuredDataSchemas.java @@ -0,0 +1,85 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.secureddata.uiapi; + +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import com.kingsrook.qqq.openapi.model.Schema; +import com.kingsrook.qqq.openapi.model.Type; + +/******************************************************************************* + ** Published shapes of service responses; nullable allowances mean unlimited. + *******************************************************************************/ +public final class SecuredDataSchemas +{ + /******************************************************************************* + ** Utility class. + *******************************************************************************/ + private SecuredDataSchemas() + { + } + + /******************************************************************************* + ** Every request and response component carries the extension prefix and version. + *******************************************************************************/ + public static Map all() + { + Map schemas = new LinkedHashMap<>(); + schemas.put("SecuredDataRevealRequestV1", object("tableName", type(Type.STRING), "fieldName", type(Type.STRING), + "primaryKey", new Schema().withAnyOf(List.of(type(Type.STRING), type(Type.NUMBER))).withDescription("Record primary key (string or number)."), + "reason", type(Type.STRING).withNullable(true).withMaxLength(500), "reasonText", type(Type.STRING).withNullable(true).withMaxLength(500)) + .withDescription("tableName, primaryKey and fieldName are required. Supply a policy reason or allowed free text; maximum 500 characters.")); + schemas.put("SecuredDataRevealResponseV1", object("value", new Schema().withNullable(true).withDescription("Authorized raw field value."), + "displayValue", type(Type.STRING).withNullable(true), "expiresAt", type(Type.STRING).withFormat("date-time"), "revealId", type(Type.INTEGER))); + schemas.put("SecuredDataRevealOptionsResponseV1", object("reasonOptions", type(Type.ARRAY).withItems(type(Type.STRING)), + "allowFreeTextReason", type(Type.BOOLEAN), "requireFreeText", type(Type.BOOLEAN), "revealMinutes", type(Type.INTEGER), + "remainingDailyAllowance", type(Type.INTEGER).withNullable(true))); + schemas.put("SecuredDataBreakGlassV1", object("id", type(Type.INTEGER), "reason", type(Type.STRING), "expiresAt", type(Type.STRING).withFormat("date-time"), + "scope", type(Type.STRING).withNullable(true), "ceiling", type(Type.STRING))); + schemas.put("SecuredDataRevealAllowanceV1", object("tableName", type(Type.STRING), "fieldName", type(Type.STRING), "remainingDailyAllowance", type(Type.INTEGER).withNullable(true))); + schemas.put("SecuredDataStatusResponseV1", object("breakGlass", new Schema().withRefToSchema("SecuredDataBreakGlassV1").withNullable(true), + "revealsUsedToday", type(Type.INTEGER), "revealAllowances", type(Type.ARRAY).withItems(new Schema().withRefToSchema("SecuredDataRevealAllowanceV1")))); + schemas.put("SecuredDataErrorV1", object("error", type(Type.STRING))); + return (schemas); + } + + /******************************************************************************* + ** Object properties in stable order. + *******************************************************************************/ + private static Schema object(Object... pairs) + { + Map properties = new LinkedHashMap<>(); + for(int i = 0; i < pairs.length; i += 2) + { + properties.put((String) pairs[i], (Schema) pairs[i + 1]); + } + return (type(Type.OBJECT).withProperties(properties)); + } + + /******************************************************************************* + ** Typed schema. + *******************************************************************************/ + private static Schema type(Type type) + { + return (new Schema().withType(type)); + } +} diff --git a/src/main/java-uiapi/com/kingsrook/qbits/secureddata/uiapi/SecuredDataStatusSpecV1.java b/src/main/java-uiapi/com/kingsrook/qbits/secureddata/uiapi/SecuredDataStatusSpecV1.java new file mode 100644 index 0000000..b858f43 --- /dev/null +++ b/src/main/java-uiapi/com/kingsrook/qbits/secureddata/uiapi/SecuredDataStatusSpecV1.java @@ -0,0 +1,83 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.secureddata.uiapi; + +import java.util.List; +import java.util.Map; +import com.kingsrook.qbits.secureddata.StatusService; +import com.kingsrook.qqq.middleware.javalin.specs.BasicOperation; +import com.kingsrook.qqq.middleware.javalin.specs.extensions.AbstractJsonEndpointSpec; +import com.kingsrook.qqq.openapi.model.HttpMethod; +import com.kingsrook.qqq.openapi.model.Parameter; +import com.kingsrook.qqq.openapi.model.Schema; + +/******************************************************************************* + ** GET secured-data/status. + *******************************************************************************/ +public class SecuredDataStatusSpecV1 extends AbstractJsonEndpointSpec +{ + /******************************************************************************* + ** Operation metadata. + *******************************************************************************/ + @Override + public BasicOperation defineBasicOperation() + { + return (new BasicOperation().withPath("/qbit/secured-data/status").withHttpMethod(HttpMethod.GET) + .withTag(() -> "Secured data").withShortSummary("Status")); + } + + /******************************************************************************* + ** Request query parameters. + *******************************************************************************/ + @Override + public List defineRequestParameters() + { + return (List.of()); + } + + /******************************************************************************* + ** Published component schemas. + *******************************************************************************/ + @Override + public Map defineComponentSchemas() + { + Map schemas = SecuredDataSchemas.all(); + return (Map.of("SecuredDataStatusResponseV1", schemas.get("SecuredDataStatusResponseV1"), "SecuredDataBreakGlassV1", schemas.get("SecuredDataBreakGlassV1"), "SecuredDataRevealAllowanceV1", schemas.get("SecuredDataRevealAllowanceV1"))); + } + + /******************************************************************************* + ** Response component. + *******************************************************************************/ + @Override + protected String getResponseSchemaName() + { + return ("SecuredDataStatusResponseV1"); + } + + /******************************************************************************* + ** Execute with the caller session supplied by the core middleware. + *******************************************************************************/ + @Override + protected Object handle(JsonInput input) throws Exception + { + return (new StatusService().status()); + } +} diff --git a/src/main/java/com/kingsrook/qbits/secureddata/SecuredDataAnomalies.java b/src/main/java/com/kingsrook/qbits/secureddata/SecuredDataAnomalies.java new file mode 100644 index 0000000..7b15018 --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/secureddata/SecuredDataAnomalies.java @@ -0,0 +1,163 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.secureddata; + +import java.time.Instant; +import java.time.temporal.ChronoUnit; +import java.util.Arrays; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import com.kingsrook.qbits.secureddata.metadata.SecuredDataTables; +import com.kingsrook.qbits.secureddata.support.SystemWrites; +import com.kingsrook.qqq.backend.core.actions.tables.AggregateAction; +import com.kingsrook.qqq.backend.core.actions.tables.CountAction; +import com.kingsrook.qqq.backend.core.alerts.AlertSignal; +import com.kingsrook.qqq.backend.core.alerts.QAlerts; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.Aggregate; +import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateInput; +import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateOperator; +import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.GroupBy; +import com.kingsrook.qqq.backend.core.model.actions.tables.count.CountInput; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType; + +/******************************************************************************* + ** System-only access-log counters. No protected values or unbounded row reads. + *******************************************************************************/ +public class SecuredDataAnomalies +{ + private final SecuredDataQBitConfig config; + private final Map medians = new LinkedHashMap<>(); + private Instant medianDay; + + /******************************************************************************* + ** Instance-scoped bounded daily cache. + *******************************************************************************/ + public SecuredDataAnomalies(SecuredDataQBitConfig config) + { + this.config = config; + } + + /******************************************************************************* + ** Evaluate before insertion, including this row exactly once. + *******************************************************************************/ + public synchronized boolean evaluate(QRecord row) throws QException + { + String event = row.getValueString("event_type"); + if("BREAK_GLASS_START".equals(event) || ("REVEAL_DENIED".equals(event) && "CAPPED".equals(row.getValueString("outcome")))) + { + return (true); + } + Instant now = row.getValueInstant("created_at"); + if(!List.of("REVEAL", "EXACT_MATCH_FILTER").contains(event) || now == null) + { + return (false); + } + String user = row.getValueString("user_id"); + QQueryFilter hour = filter(user, event, now.minus(1, ChronoUnit.HOURS), now, QCriteriaOperator.LESS_THAN_OR_EQUALS); + hour.getCriteria().stream().filter(criteria -> "created_at".equals(criteria.getFieldName()) + && criteria.getOperator() == QCriteriaOperator.GREATER_THAN_OR_EQUALS).forEach(criteria -> criteria.setOperator(QCriteriaOperator.GREATER_THAN)); + if("EXACT_MATCH_FILTER".equals(event)) + { + hour.withCriteria(new QFilterCriteria("table_name", QCriteriaOperator.EQUALS, row.getValueString("table_name"))) + .withCriteria(new QFilterCriteria("field_name", QCriteriaOperator.EQUALS, row.getValueString("field_name"))); + return (count(hour) >= config.getAnomalyExactMatchesPerHour()); + } + Instant day = now.truncatedTo(ChronoUnit.DAYS); + int today = count(filter(user, event, day, now, QCriteriaOperator.LESS_THAN_OR_EQUALS)) + 1; + if(today >= config.getAnomalyMinimumDailyReveals() && today > config.getAnomalyDailyMedianMultiplier() * median(user, day)) + { + return (true); + } + GroupBy table = new GroupBy(QFieldType.STRING, "table_name"); + GroupBy record = new GroupBy(QFieldType.STRING, "record_id"); + var groups = SystemWrites.call(() -> new AggregateAction().execute(new AggregateInput(SecuredDataTables.ACCESS_LOG) + .withFilter(hour).withGroupBy(table).withGroupBy(record).withAggregate(new Aggregate("id", AggregateOperator.COUNT)) + .withLimit(config.getAnomalyDistinctRecordsPerHour() + 1)).getResults()); + long distinct = groups.size(); + boolean alreadySeen = groups.stream().anyMatch(group -> java.util.Objects.equals(group.getGroupByValue(table), row.getValue("table_name")) + && java.util.Objects.equals(group.getGroupByValue(record), row.getValue("record_id"))); + return (distinct + (alreadySeen ? 0 : 1) > config.getAnomalyDistinctRecordsPerHour()); + } + + /******************************************************************************* + ** Previous thirty complete UTC days, including zeros; even median averages middle days. + ** Thirty scalar counts once per user/day; at most 2048 cached users. + *******************************************************************************/ + private double median(String user, Instant day) throws QException + { + if(!day.equals(medianDay)) + { + medians.clear(); + medianDay = day; + } + if(!medians.containsKey(user)) + { + int[] counts = new int[30]; + for(int i = 0; i < counts.length; i++) + { + counts[i] = count(filter(user, "REVEAL", day.minus(i + 1, ChronoUnit.DAYS), day.minus(i, ChronoUnit.DAYS), QCriteriaOperator.LESS_THAN)); + } + Arrays.sort(counts); + if(medians.size() >= 2048) + { + medians.remove(medians.keySet().iterator().next()); + } + medians.put(user, (counts[14] + (double) counts[15]) / 2); + } + return (medians.get(user)); + } + + /******************************************************************************* + ** Scalar database count, always as system. + *******************************************************************************/ + private static int count(QQueryFilter filter) throws QException + { + return (SystemWrites.call(() -> new CountAction().execute(new CountInput(SecuredDataTables.ACCESS_LOG).withFilter(filter)).getCount())); + } + + /******************************************************************************* + ** Bounded time window and user, compatible with indexed access-log lookups. + *******************************************************************************/ + private static QQueryFilter filter(String user, String event, Instant from, Instant until, QCriteriaOperator endOperator) + { + return (new QQueryFilter().withCriteria(new QFilterCriteria("user_id", QCriteriaOperator.EQUALS, user)) + .withCriteria(new QFilterCriteria("event_type", QCriteriaOperator.EQUALS, event)) + .withCriteria(new QFilterCriteria("created_at", QCriteriaOperator.GREATER_THAN_OR_EQUALS, from)) + .withCriteria(new QFilterCriteria("created_at", endOperator, until))); + } + + /******************************************************************************* + ** Optional SPI: flags remain useful without a sink. Deduplication belongs to the sink. + *******************************************************************************/ + public static void raise(QRecord row) + { + QAlerts.raise(new AlertSignal("secured_data.anomaly") + .withDedupKey("secured_data.anomaly:" + row.getValueString("user_id") + ":" + row.getValueInstant("created_at").toString().substring(0, 10)) + .withOccurredAt(row.getValueInstant("created_at")).withTitle("Unusual secured-data access") + .withSummary("Review the flagged secured-data access-log row.")); + } +} diff --git a/src/main/java/com/kingsrook/qbits/secureddata/SecuredDataQBitConfig.java b/src/main/java/com/kingsrook/qbits/secureddata/SecuredDataQBitConfig.java index 5108154..949b94c 100644 --- a/src/main/java/com/kingsrook/qbits/secureddata/SecuredDataQBitConfig.java +++ b/src/main/java/com/kingsrook/qbits/secureddata/SecuredDataQBitConfig.java @@ -43,6 +43,12 @@ public class SecuredDataQBitConfig implements QBitConfig { private static final byte[] JVM_KEY = new SecureRandom().generateSeed(32); private Map defaultTierByClassification = new HashMap<>(Map.of("CONFIDENTIAL", "FULL", "PII", "MASKED", "PHI", "MASKED")); + private int anomalyDailyMedianMultiplier = 3; + private int anomalyMinimumDailyReveals = 10; + private int anomalyDistinctRecordsPerHour = 20; + private int anomalyExactMatchesPerHour = 50; + private String tablePathPrefix = "/app"; + private String schedulerName; private String backendName = "memory"; private Clock clock = Clock.systemUTC(); private int accessLogQueueCapacity = 4096; @@ -58,6 +64,23 @@ public class SecuredDataQBitConfig implements QBitConfig + /******************************************************************************* + ** Scheduler to run the once-per-minute break-glass expiry job. + *******************************************************************************/ + public String getSchedulerName() + { + return (schedulerName); + } + + /******************************************************************************* + ** Configure the host scheduler; null leaves manual scheduling to the host. + *******************************************************************************/ + public SecuredDataQBitConfig withSchedulerName(String schedulerName) + { + this.schedulerName = schedulerName; + return (this); + } + /******************************************************************************* ** Get backendName. *******************************************************************************/ @@ -364,6 +387,8 @@ public class SecuredDataQBitConfig implements QBitConfig public void validate() { if(backendName == null || backendName.isBlank() || accessLogQueueCapacity < 1 || accessLogFlushIntervalMillis < 1 + || anomalyDailyMedianMultiplier < 1 || anomalyMinimumDailyReveals < 1 || anomalyDistinctRecordsPerHour < 1 + || anomalyExactMatchesPerHour < 1 || tablePathPrefix == null || !tablePathPrefix.startsWith("/") || tablePathPrefix.startsWith("//") || exactMatchPerHour < 0 || missAlertConsecutive < 0 || distinctValueAlert < 0) { throw (new IllegalArgumentException("Invalid secured-data configuration")); @@ -372,4 +397,89 @@ public class SecuredDataQBitConfig implements QBitConfig Objects.requireNonNull(viewLogClassifications); Objects.requireNonNull(fingerprintKeySupplier); } + + /******************************************************************************* + ** Get anomalyDailyMedianMultiplier. + *******************************************************************************/ + public int getAnomalyDailyMedianMultiplier() + { + return (anomalyDailyMedianMultiplier); + } + + /******************************************************************************* + ** Configure anomalyDailyMedianMultiplier. + *******************************************************************************/ + public SecuredDataQBitConfig withAnomalyDailyMedianMultiplier(int value) + { + this.anomalyDailyMedianMultiplier = value; + return (this); + } + + /******************************************************************************* + ** Get anomalyMinimumDailyReveals. + *******************************************************************************/ + public int getAnomalyMinimumDailyReveals() + { + return (anomalyMinimumDailyReveals); + } + + /******************************************************************************* + ** Configure anomalyMinimumDailyReveals. + *******************************************************************************/ + public SecuredDataQBitConfig withAnomalyMinimumDailyReveals(int value) + { + this.anomalyMinimumDailyReveals = value; + return (this); + } + + /******************************************************************************* + ** Get anomalyDistinctRecordsPerHour. + *******************************************************************************/ + public int getAnomalyDistinctRecordsPerHour() + { + return (anomalyDistinctRecordsPerHour); + } + + /******************************************************************************* + ** Configure anomalyDistinctRecordsPerHour. + *******************************************************************************/ + public SecuredDataQBitConfig withAnomalyDistinctRecordsPerHour(int value) + { + this.anomalyDistinctRecordsPerHour = value; + return (this); + } + + /******************************************************************************* + ** Get anomalyExactMatchesPerHour. + *******************************************************************************/ + public int getAnomalyExactMatchesPerHour() + { + return (anomalyExactMatchesPerHour); + } + + /******************************************************************************* + ** Configure anomalyExactMatchesPerHour. + *******************************************************************************/ + public SecuredDataQBitConfig withAnomalyExactMatchesPerHour(int value) + { + this.anomalyExactMatchesPerHour = value; + return (this); + } + + /******************************************************************************* + ** Get tablePathPrefix. + *******************************************************************************/ + public String getTablePathPrefix() + { + return (tablePathPrefix); + } + + /******************************************************************************* + ** Configure tablePathPrefix. + *******************************************************************************/ + public SecuredDataQBitConfig withTablePathPrefix(String value) + { + this.tablePathPrefix = value; + return (this); + } } diff --git a/src/main/java/com/kingsrook/qbits/secureddata/SecuredDataQBitProducer.java b/src/main/java/com/kingsrook/qbits/secureddata/SecuredDataQBitProducer.java index 0cd1100..1a450ec 100644 --- a/src/main/java/com/kingsrook/qbits/secureddata/SecuredDataQBitProducer.java +++ b/src/main/java/com/kingsrook/qbits/secureddata/SecuredDataQBitProducer.java @@ -27,7 +27,6 @@ import java.util.Locale; import java.util.Map; import com.kingsrook.qbits.secureddata.metadata.SecuredDataTableCustomizers; import com.kingsrook.qbits.secureddata.metadata.SecuredDataTables; -import com.kingsrook.qbits.secureddata.permissions.SecuredDataPermissions; import com.kingsrook.qbits.secureddata.standin.DataClassification; import com.kingsrook.qbits.secureddata.support.CoreMetadataBridge; import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizers; @@ -36,8 +35,6 @@ import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerMultiOutput import com.kingsrook.qqq.backend.core.model.metadata.QInstance; import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference; import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData; -import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData; -import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon; import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitMetaData; import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitMetaDataProducer; import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData; @@ -138,8 +135,7 @@ public class SecuredDataQBitProducer implements QBitMetaDataProducer tables = new LinkedHashMap<>(); for(QTableMetaData definition : SecuredDataTables.tables(qBitConfig)) { @@ -154,12 +150,14 @@ public class SecuredDataQBitProducer implements QBitMetaDataProducer app.withChild(tables.get(name))); - output.add(app); + output.add(com.kingsrook.qbits.secureddata.app.SecuredDataApp.app(tables)); + output.add(com.kingsrook.qbits.secureddata.app.SecuredDataApp.widget()); + com.kingsrook.qbits.secureddata.processes.BreakGlassProcesses.processes(qBitConfig).forEach(output::add); output.add(SecuredDataTables.revealsJoin()); output.add(SecuredDataTables.revealsWidget()); output.addSelfToInstance(qInstance); CoreMetadataBridge.register(qInstance); + com.kingsrook.qbits.secureddata.support.UiApiBridge.register(qInstance); if(qBitConfig.getNameHeuristics()) { seedHeuristics(qInstance); diff --git a/src/main/java/com/kingsrook/qbits/secureddata/SecuredDataResolver.java b/src/main/java/com/kingsrook/qbits/secureddata/SecuredDataResolver.java index 4ec8761..ab86274 100644 --- a/src/main/java/com/kingsrook/qbits/secureddata/SecuredDataResolver.java +++ b/src/main/java/com/kingsrook/qbits/secureddata/SecuredDataResolver.java @@ -66,6 +66,15 @@ public class SecuredDataResolver implements FieldAccessResolverInterface private final SecuredDataRuntime runtime; private final byte[] fingerprintKey; private final ThreadLocal resolving = ThreadLocal.withInitial(() -> false); + private final Map reveals = new HashMap<>(); + + /******************************************************************************* + ** Bounded per-user grant cache, with exact read-time expiry. + *******************************************************************************/ + private record RevealCache(Instant until, List rows) + { + } + private long cachedVersion; private List policies = List.of(); private final Map users = new HashMap<>(); @@ -267,6 +276,17 @@ public class SecuredDataResolver implements FieldAccessResolverInterface + /******************************************************************************* + ** Granting rows only: channel rows cap tiers and never grant reveal settings. + *******************************************************************************/ + public List revealPolicies(QTableMetaData table, String fieldName) + { + return (matching(table, SecuredDataClassifications.effective(table, fieldName)).stream() + .filter(row -> blank(row.getValueString("client_channel")) && "REVEALABLE".equals(row.getValueString("tier"))).toList()); + } + + + /******************************************************************************* ** Refresh policy rows once per version and matched state per relevant permissions. *******************************************************************************/ @@ -312,7 +332,7 @@ public class SecuredDataResolver implements FieldAccessResolverInterface /******************************************************************************* ** Exact expiry is checked at use time, independent of the future expiry job. *******************************************************************************/ - private List activeBreakGlass() + public List activeBreakGlass() { Instant now = runtime.config().getClock().instant(); return (state().breakGlass().stream().filter(row -> "ACTIVE".equals(row.getValueString("status"))) @@ -382,12 +402,39 @@ public class SecuredDataResolver implements FieldAccessResolverInterface @Override public boolean isRevealActive(String tableName, String fieldName, Serializable primaryKey) { - return (SystemWrites.query(SecuredDataTables.REVEAL_GRANT, new QQueryFilter() - .withCriteria(new QFilterCriteria("user_id", QCriteriaOperator.EQUALS, userId())) - .withCriteria(new QFilterCriteria("table_name", QCriteriaOperator.EQUALS, tableName)) - .withCriteria(new QFilterCriteria("field_name", QCriteriaOperator.EQUALS, fieldName)) - .withCriteria(new QFilterCriteria("record_id", QCriteriaOperator.EQUALS, String.valueOf(primaryKey))) - .withCriteria(new QFilterCriteria("expires_at", QCriteriaOperator.GREATER_THAN, runtime.config().getClock().instant()))).size() > 0); + synchronized(reveals) + { + String user = userId(); + Instant now = runtime.config().getClock().instant(); + RevealCache cached = reveals.get(user); + if(cached == null || !cached.until().isAfter(now)) + { + if(reveals.size() >= 1024) + { + reveals.clear(); + } + cached = new RevealCache(now.plusSeconds(5), SystemWrites.query(SecuredDataTables.REVEAL_GRANT, new QQueryFilter() + .withCriteria(new QFilterCriteria("user_id", QCriteriaOperator.EQUALS, user)) + .withCriteria(new QFilterCriteria("expires_at", QCriteriaOperator.GREATER_THAN, now)))); + reveals.put(user, cached); + } + return (cached.rows().stream().anyMatch(row -> tableName.equals(row.getValueString("table_name")) + && fieldName.equals(row.getValueString("field_name")) && String.valueOf(primaryKey).equals(row.getValueString("record_id")) + && row.getValueInstant("expires_at").isAfter(now))); + } + } + + + + /******************************************************************************* + ** Publish a newly created grant to all sessions on this instance immediately. + *******************************************************************************/ + public void invalidateReveals() + { + synchronized(reveals) + { + reveals.clear(); + } } diff --git a/src/main/java/com/kingsrook/qbits/secureddata/SecuredDataRuntime.java b/src/main/java/com/kingsrook/qbits/secureddata/SecuredDataRuntime.java index 7e036c0..af8ac2f 100644 --- a/src/main/java/com/kingsrook/qbits/secureddata/SecuredDataRuntime.java +++ b/src/main/java/com/kingsrook/qbits/secureddata/SecuredDataRuntime.java @@ -40,6 +40,7 @@ public class SecuredDataRuntime implements QSupplementalInstanceMetaData, AutoCl private final transient SecuredDataQBitConfig config; private final transient SecuredDataResolver resolver; private final transient SecuredDataAccessLogger logger; + private final transient SecuredDataAnomalies anomalies; private transient long cachedClassificationVersion; private transient List classificationRows = List.of(); @@ -52,6 +53,7 @@ public class SecuredDataRuntime implements QSupplementalInstanceMetaData, AutoCl { this.config = config; this.resolver = new SecuredDataResolver(this); + this.anomalies = new SecuredDataAnomalies(config); this.logger = new SecuredDataAccessLogger(instance, this); } @@ -173,4 +175,12 @@ public class SecuredDataRuntime implements QSupplementalInstanceMetaData, AutoCl { logger.close(); } + + /******************************************************************************* + ** Anomaly counters shared by synchronous reveals and queued searches. + *******************************************************************************/ + public SecuredDataAnomalies anomalies() + { + return (anomalies); + } } diff --git a/src/main/java/com/kingsrook/qbits/secureddata/StatusService.java b/src/main/java/com/kingsrook/qbits/secureddata/StatusService.java new file mode 100644 index 0000000..89da2ac --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/secureddata/StatusService.java @@ -0,0 +1,90 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.secureddata; + +import java.io.Serializable; +import java.time.Instant; +import java.util.ArrayList; +import java.util.List; +import com.kingsrook.qbits.secureddata.reveal.RevealOptions; +import com.kingsrook.qbits.secureddata.reveal.RevealService; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData; + +/******************************************************************************* + ** Caller-only banner and per-field reveal allowance summary. + *******************************************************************************/ +public class StatusService +{ + /******************************************************************************* + ** Active session projection. + *******************************************************************************/ + public record BreakGlassStatus(Serializable id, String reason, Instant expiresAt, String scope, String ceiling) + { + } + + /******************************************************************************* + ** Allowances differ by granting policy; null remaining means unlimited. + *******************************************************************************/ + public record Allowance(String tableName, String fieldName, Integer remainingDailyAllowance) + { + } + + /******************************************************************************* + ** Status response. + *******************************************************************************/ + public record Status(BreakGlassStatus breakGlass, int revealsUsedToday, List revealAllowances) + { + } + + /******************************************************************************* + ** Only readable revealable fields are included, without any record values. + *******************************************************************************/ + public Status status() throws QException + { + List active = SecuredDataRuntime.current().resolver().activeBreakGlass(); + QRecord row = active.isEmpty() ? null : active.getFirst(); + BreakGlassStatus session = row == null ? null : new BreakGlassStatus(row.getValue("id"), row.getValueString("reason"), row.getValueInstant("expires_at"), + row.getValueString("table_name"), row.getValueString("classification_ceiling")); + RevealService service = new RevealService(); + List allowances = new ArrayList<>(); + for(QTableMetaData table : QContext.getQInstance().getTables().values()) + { + for(QFieldMetaData field : table.getFields().values()) + { + try + { + RevealOptions options = service.options(table.getName(), field.getName()); + allowances.add(new Allowance(table.getName(), field.getName(), options.remainingDailyAllowance())); + } + catch(QPermissionDeniedException expected) + { + // This field offers no reveal operation to the caller. + } + } + } + return (new Status(session, service.usedToday(), List.copyOf(allowances))); + } +} diff --git a/src/main/java/com/kingsrook/qbits/secureddata/app/SecuredDataApp.java b/src/main/java/com/kingsrook/qbits/secureddata/app/SecuredDataApp.java new file mode 100644 index 0000000..448d4e7 --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/secureddata/app/SecuredDataApp.java @@ -0,0 +1,88 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.secureddata.app; + +import java.util.List; +import java.util.Map; +import com.kingsrook.qbits.secureddata.metadata.SecuredDataTables; +import com.kingsrook.qbits.secureddata.permissions.SecuredDataPermissions; +import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput; +import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput; +import com.kingsrook.qqq.backend.core.model.dashboard.widgets.RawHTML; +import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType; +import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference; +import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon; +import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData; + +/******************************************************************************* + ** One app and one home. Grants remain a child list; processes stay off the nav. + *******************************************************************************/ +public final class SecuredDataApp +{ + public static final String HOME_WIDGET = "securedDataHome"; + + /******************************************************************************* + ** Utility class. + *******************************************************************************/ + private SecuredDataApp() + { + } + + /******************************************************************************* + ** Admin-only home spanning the host's full grid. + *******************************************************************************/ + public static QWidgetMetaData widget() + { + return (new QWidgetMetaData().withName(HOME_WIDGET).withLabel("Overview").withType(WidgetType.HTML.getType()) + .withCodeReference(new QCodeReference(HomeRenderer.class)).withGridColumns(12).withIsCard(false) + .withPermissionRules(SecuredDataPermissions.rules())); + } + + /******************************************************************************* + ** Exactly four navigation tables. + *******************************************************************************/ + public static QAppMetaData app(Map tables) + { + QAppMetaData app = new QAppMetaData().withName("secured_data").withLabel("Secured Data") + .withIcon(new QIcon("shield")).withPermissionRules(SecuredDataPermissions.rules()).withWidgets(List.of(HOME_WIDGET)); + SecuredDataTables.MAIN.forEach(name -> app.withChild(tables.get(name))); + return (app); + } + + /******************************************************************************* + ** Standard HTML widget, following the webhooks home pattern. + *******************************************************************************/ + public static class HomeRenderer extends AbstractWidgetRenderer + { + /******************************************************************************* + ** Render real viewer-scoped data. + *******************************************************************************/ + @Override + public RenderWidgetOutput render(RenderWidgetInput input) throws QException + { + return (new RenderWidgetOutput(new RawHTML(null, SecuredDataHomeView.render(SecuredDataHomeData.load())))); + } + } +} diff --git a/src/main/java/com/kingsrook/qbits/secureddata/app/SecuredDataHomeData.java b/src/main/java/com/kingsrook/qbits/secureddata/app/SecuredDataHomeData.java new file mode 100644 index 0000000..161b851 --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/secureddata/app/SecuredDataHomeData.java @@ -0,0 +1,185 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.secureddata.app; + +import java.net.URLEncoder; +import java.nio.charset.StandardCharsets; +import java.time.Instant; +import java.time.temporal.ChronoUnit; +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import com.kingsrook.qbits.secureddata.SecuredDataRuntime; +import com.kingsrook.qbits.secureddata.metadata.SecuredDataTables; +import com.kingsrook.qqq.backend.core.actions.tables.AggregateAction; +import com.kingsrook.qqq.backend.core.actions.tables.CountAction; +import com.kingsrook.qqq.backend.core.actions.tables.QueryAction; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.Aggregate; +import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateInput; +import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateOperator; +import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.GroupBy; +import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.QFilterOrderByAggregate; +import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.QFilterOrderByGroupBy; +import com.kingsrook.qqq.backend.core.model.actions.tables.count.CountInput; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType; +import com.kingsrook.qqq.backend.core.utils.JsonUtils; + +/******************************************************************************* + ** Real home data, read as the viewer. Counts and rankings aggregate in the backend. + *******************************************************************************/ +public class SecuredDataHomeData +{ + private final Map kpis = new LinkedHashMap<>(); + private final List
sections = new ArrayList<>(); + + /******************************************************************************* + ** A full-width table and the exact filter behind its View all link. + *******************************************************************************/ + public record Section(String title, String empty, List fields, List labels, List rows, String href) + { + } + + /******************************************************************************* + ** Six KPIs and six lists; no samples or hidden truncation of aggregate counts. + *******************************************************************************/ + public static SecuredDataHomeData load() throws QException + { + SecuredDataHomeData data = new SecuredDataHomeData(); + Instant now = SecuredDataRuntime.current().config().getClock().instant(); + Instant week = now.minus(7, ChronoUnit.DAYS); + Instant month = now.minus(30, ChronoUnit.DAYS); + data.kpis.put("Reveals today", count(SecuredDataTables.ACCESS_LOG, events(now.truncatedTo(ChronoUnit.DAYS), now, "REVEAL"))); + data.kpis.put("Active reveal grants", count(SecuredDataTables.REVEAL_GRANT, new QQueryFilter() + .withCriteria(new QFilterCriteria("expires_at", QCriteriaOperator.GREATER_THAN, now)))); + data.kpis.put("Active break-glass sessions", count(SecuredDataTables.BREAK_GLASS, new QQueryFilter() + .withCriteria(new QFilterCriteria("status", QCriteriaOperator.EQUALS, "ACTIVE")) + .withCriteria(new QFilterCriteria("expires_at", QCriteriaOperator.GREATER_THAN, now)))); + QQueryFilter anomalies = window(week, now).withCriteria(new QFilterCriteria("anomaly_flag", QCriteriaOperator.EQUALS, true)); + data.kpis.put("Anomaly flags (7 days)", count(SecuredDataTables.ACCESS_LOG, anomalies)); + QQueryFilter denied = window(week, now).withSubFilter(new QQueryFilter().withBooleanOperator(QQueryFilter.BooleanOperator.OR) + .withCriteria(new QFilterCriteria("event_type", QCriteriaOperator.EQUALS, "FILTER_DENIED")) + .withSubFilter(new QQueryFilter().withCriteria(new QFilterCriteria("event_type", QCriteriaOperator.EQUALS, "EXACT_MATCH_FILTER")) + .withCriteria(new QFilterCriteria("outcome", QCriteriaOperator.EQUALS, "REFUSED")))); + data.kpis.put("Denied filters (7 days)", count(SecuredDataTables.ACCESS_LOG, denied)); + data.kpis.put("Access-log queue drops", SecuredDataRuntime.current().logger().getDroppedCount()); + data.recent("Recent reveals", "No reveals yet", events(null, now, "REVEAL"), + List.of("user_name", "table_name", "record_id", "field_name", "reason", "created_at", "client_channel"), + List.of("User", "Table", "Record", "Field", "Reason", "When", "Channel")); + data.top("Top revealers (30 days)", "No reveals in the last 30 days", "user_id", "User", events(month, now, "REVEAL")); + data.top("Top reasons (30 days)", "No reveal reasons in the last 30 days", "reason", "Reason", events(month, now, "REVEAL")); + data.recent("Break-glass events", "No break-glass events yet", events(null, now, "BREAK_GLASS_START", "BREAK_GLASS_END", "BREAK_GLASS_EXPIRED"), + List.of("user_name", "event_type", "reason", "created_at"), List.of("User", "Event", "Reason", "When")); + data.recent("Policy and classification changes", "No policy or classification changes yet", events(null, now, "POLICY_CHANGE", "CLASSIFICATION_CHANGE"), + List.of("user_name", "table_name", "record_id", "action_name", "created_at"), List.of("User", "Table", "Record", "Action", "When")); + data.recent("Anomaly flags", "No anomaly flags in the last 7 days", anomalies, + List.of("user_name", "event_type", "table_name", "field_name", "created_at"), List.of("User", "Event", "Table", "Field", "When")); + return (data); + } + + /******************************************************************************* + ** Scalar counts preserve viewer permissions. + *******************************************************************************/ + private static long count(String table, QQueryFilter filter) throws QException + { + return (new CountAction().execute(new CountInput(table).withFilter(filter)).getCount()); + } + + /******************************************************************************* + ** Newest ten, with deterministic id ordering for equal timestamps. + *******************************************************************************/ + private void recent(String title, String empty, QQueryFilter filter, List fields, List labels) throws QException + { + filter.withOrderBy(new QFilterOrderBy("created_at", false)).withOrderBy(new QFilterOrderBy("id", false)); + String href = link(filter); + List rows = new QueryAction().execute(new QueryInput(SecuredDataTables.ACCESS_LOG).withFilter(filter.withLimit(10))).getRecords(); + sections.add(new Section(title, empty, fields, labels, rows, href)); + } + + /******************************************************************************* + ** Top ten complete counts, with stable alphabetical tie ordering. + *******************************************************************************/ + private void top(String title, String empty, String field, String label, QQueryFilter filter) throws QException + { + String href = link(filter); + GroupBy group = new GroupBy(QFieldType.STRING, field); + Aggregate count = new Aggregate("id", AggregateOperator.COUNT); + filter.withOrderBy(new QFilterOrderByAggregate(count, false)).withOrderBy(new QFilterOrderByGroupBy(group)); + var results = new AggregateAction().execute(new AggregateInput(SecuredDataTables.ACCESS_LOG).withFilter(filter) + .withGroupBy(group).withAggregate(count).withLimit(10)).getResults(); + List rows = results.stream().map(result -> new QRecord().withValue(field, result.getGroupByValue(group)) + .withValue("count", result.getAggregateValue(count))).toList(); + sections.add(new Section(title, empty, List.of(field, "count"), List.of(label, "Reveals"), rows, href)); + } + + /******************************************************************************* + ** Next's table filter wire format, URL encoded; no widget row limit in the link. + *******************************************************************************/ + private static String link(QQueryFilter filter) + { + return (SecuredDataRuntime.current().config().getTablePathPrefix() + "/" + SecuredDataTables.ACCESS_LOG + + "?filter=" + URLEncoder.encode(JsonUtils.toJson(filter), StandardCharsets.UTF_8)); + } + + /******************************************************************************* + ** Event filter scoped to a time window. + *******************************************************************************/ + private static QQueryFilter events(Instant from, Instant now, String... events) + { + return (window(from, now).withCriteria(new QFilterCriteria("event_type", QCriteriaOperator.IN, java.util.Arrays.asList(events)))); + } + + /******************************************************************************* + ** Exclude future timestamps from all home activity windows. + *******************************************************************************/ + private static QQueryFilter window(Instant from, Instant now) + { + QQueryFilter filter = new QQueryFilter().withCriteria(new QFilterCriteria("created_at", QCriteriaOperator.LESS_THAN_OR_EQUALS, now)); + if(from != null) + { + filter.withCriteria(new QFilterCriteria("created_at", QCriteriaOperator.GREATER_THAN_OR_EQUALS, from)); + } + return (filter); + } + + /******************************************************************************* + ** KPI row. + *******************************************************************************/ + public Map getKpis() + { + return (kpis); + } + + /******************************************************************************* + ** Full-width lists in display order. + *******************************************************************************/ + public List
getSections() + { + return (sections); + } +} diff --git a/src/main/java/com/kingsrook/qbits/secureddata/app/SecuredDataHomeView.java b/src/main/java/com/kingsrook/qbits/secureddata/app/SecuredDataHomeView.java new file mode 100644 index 0000000..3b1b255 --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/secureddata/app/SecuredDataHomeView.java @@ -0,0 +1,101 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.secureddata.app; + +import java.time.ZoneOffset; +import java.time.format.DateTimeFormatter; +import java.util.Locale; +import com.kingsrook.qqq.backend.core.model.data.QRecord; + +/******************************************************************************* + ** Inline styles survive Next's sanitizer. Full-width labelled card rows wrap at + ** narrow widths; no inner scrollers. Inherited host text and theme border tokens. + *******************************************************************************/ +public final class SecuredDataHomeView +{ + private static final DateTimeFormatter WHEN = DateTimeFormatter.ofPattern("MMM d, HH:mm 'UTC'", Locale.ENGLISH).withZone(ZoneOffset.UTC); + private static final String GRID = "display:grid;grid-template-columns:repeat(auto-fit,minmax(min(100%,10rem),1fr));gap:.75rem;min-width:0;"; + private static final String CARD = "border:1px solid var(--qqq-grid-border,currentColor);border-radius:.5rem;padding:1rem;min-width:0;"; + + /******************************************************************************* + ** Utility class. + *******************************************************************************/ + private SecuredDataHomeView() + { + } + + /******************************************************************************* + ** KPI row followed by six full-width tables, never side by side. + *******************************************************************************/ + public static String render(SecuredDataHomeData data) + { + StringBuilder html = new StringBuilder("
"); + html.append("
"); + data.getKpis().forEach((label, value) -> html.append("
").append(value) + .append("
").append(esc(label)).append("
")); + html.append("
"); + for(SecuredDataHomeData.Section section : data.getSections()) + { + html.append("

") + .append(esc(section.title())).append("

View all

"); + if(section.rows().isEmpty()) + { + html.append("

").append(esc(section.empty())).append("

"); + } + for(QRecord row : section.rows()) + { + html.append("
"); + for(int i = 0; i < section.fields().size(); i++) + { + String field = section.fields().get(i); + String value = row.getValueString(field); + boolean date = "created_at".equals(field); + if(date && row.getValueInstant(field) != null) + { + value = WHEN.format(row.getValueInstant(field)); + } + if(value == null || value.isBlank()) + { + value = "—"; + } + html.append("
") + .append(esc(section.labels().get(i))).append("") + .append(esc(value)).append("
"); + } + html.append("
"); + } + html.append("
"); + } + return (html.append("
").toString()); + } + + /******************************************************************************* + ** Escape every text, tooltip and URL attribute. + *******************************************************************************/ + private static String esc(String value) + { + return (value.replace("&", "&").replace("<", "<").replace(">", ">").replace("\"", """).replace("'", "'")); + } +} diff --git a/src/main/java/com/kingsrook/qbits/secureddata/breakglass/BreakGlassService.java b/src/main/java/com/kingsrook/qbits/secureddata/breakglass/BreakGlassService.java new file mode 100644 index 0000000..1a5a87c --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/secureddata/breakglass/BreakGlassService.java @@ -0,0 +1,243 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.secureddata.breakglass; + +import java.io.Serializable; +import java.time.Instant; +import java.util.List; +import java.util.Objects; +import com.kingsrook.qbits.secureddata.SecuredDataResolver; +import com.kingsrook.qbits.secureddata.SecuredDataRuntime; +import com.kingsrook.qbits.secureddata.metadata.SecuredDataTables; +import com.kingsrook.qbits.secureddata.permissions.SecuredDataPermissions; +import com.kingsrook.qbits.secureddata.standin.DataClassification; +import com.kingsrook.qbits.secureddata.support.ActivityLog; +import com.kingsrook.qbits.secureddata.support.CoreMetadataBridge; +import com.kingsrook.qbits.secureddata.support.SystemWrites; +import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction; +import com.kingsrook.qqq.backend.core.alerts.AlertSeverity; +import com.kingsrook.qqq.backend.core.alerts.AlertSignal; +import com.kingsrook.qqq.backend.core.alerts.QAlerts; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.exceptions.QBadRequestException; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.exceptions.QNotFoundException; +import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException; +import com.kingsrook.qqq.backend.core.logging.QLogger; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter; +import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput; +import com.kingsrook.qqq.backend.core.model.data.QRecord; + +/******************************************************************************* + ** Explicit, audited emergency access. No implicit admin access to user data. + *******************************************************************************/ +public class BreakGlassService +{ + public static final String START = "secured_data_break_glass_start"; + public static final String END = "secured_data_break_glass_end"; + public static final String EXPIRE = "secured_data_break_glass_expire"; + private static final QLogger LOG = QLogger.getLogger(BreakGlassService.class); + + /******************************************************************************* + ** Validate first, then create a system-owned session and publish its policy version. + *******************************************************************************/ + public QRecord start(String reason, Integer minutes, String table, DataClassification ceiling) throws QException + { + checkChannel(); + if(!QContext.getQSession().hasPermission(START + ".hasAccess")) + { + throw (new QPermissionDeniedException("Permission denied.")); + } + int duration = minutes == null ? 60 : minutes; + if(reason == null || reason.trim().length() < 20 || reason.length() > 500 || duration < 1 || duration > 480) + { + throw (new QBadRequestException("Reason must contain 20-500 characters; duration must be 1-480 minutes.")); + } + if(table != null && table.isBlank()) + { + table = null; + } + if(table != null && QContext.getQInstance().getTable(table) == null) + { + throw (new QBadRequestException("Unknown table scope.")); + } + SecuredDataRuntime runtime = SecuredDataRuntime.current(); + synchronized(runtime) + { + if(!runtime.resolver().activeBreakGlass().isEmpty()) + { + throw (new QBadRequestException("An active break-glass session already exists.")); + } + Instant now = runtime.config().getClock().instant(); + QRecord row = SystemWrites.insertOne(SecuredDataTables.BREAK_GLASS, new QRecord() + .withValue("user_id", SecuredDataResolver.userId()).withValue("user_name", QContext.getQSession().getUser().getFullName()) + .withValue("reason", reason).withValue("table_name", table).withValue("classification_ceiling", Objects.requireNonNullElse(ceiling, DataClassification.PHI).name()) + .withValue("started_at", now).withValue("expires_at", now.plusSeconds(duration * 60L)).withValue("status", "ACTIVE")); + runtime.invalidatePolicies(); + log(row, "BREAK_GLASS_START"); + QAlerts.raise(new AlertSignal("secured_data.break_glass").withSeverity(AlertSeverity.fromLabel("HIGH", AlertSeverity.ERROR)).withDedupKey(dedup(row)) + .withTitle("Break-glass access started").withSummary("Review the secured-data break-glass session.").withOccurredAt(now)); + notifyAdmins(row); + return (row); + } + } + + /******************************************************************************* + ** Owner or admin may end an active session, never an assistant channel. + *******************************************************************************/ + public QRecord end(Serializable id) throws QException + { + checkChannel(); + synchronized(SecuredDataRuntime.current()) + { + QRecord row = find(id); + checkOwner(row); + if(!"ACTIVE".equals(row.getValueString("status"))) + { + throw (new QBadRequestException("Session is already finished.")); + } + return (finish(row, "ENDED", "BREAK_GLASS_END")); + } + } + + /******************************************************************************* + ** Review uses the same ownership check; load rechecks against persisted state. + *******************************************************************************/ + public void checkOwner(QRecord row) throws QPermissionDeniedException + { + checkChannel(); + if(!Objects.equals(SecuredDataResolver.userId(), row.getValueString("user_id")) + && !QContext.getQSession().hasPermission(SecuredDataPermissions.ADMIN)) + { + throw (new QPermissionDeniedException("Only the owner or a secured-data administrator can end this session.")); + } + } + + /******************************************************************************* + ** All rows are looked up as system; authorization remains the caller responsibility. + *******************************************************************************/ + public QRecord find(Serializable id) throws QNotFoundException + { + List rows = SystemWrites.query(SecuredDataTables.BREAK_GLASS, new QQueryFilter(new QFilterCriteria("id", QCriteriaOperator.EQUALS, id))); + if(rows.isEmpty()) + { + throw (new QNotFoundException("No record was found")); + } + return (rows.getFirst()); + } + + /******************************************************************************* + ** Scheduled maintenance always performs its lookups and changes as system. + *******************************************************************************/ + public int expire() throws QException + { + synchronized(SecuredDataRuntime.current()) + { + return (SystemWrites.call(() -> + { + List rows = SystemWrites.query(SecuredDataTables.BREAK_GLASS, new QQueryFilter() + .withCriteria(new QFilterCriteria("status", QCriteriaOperator.EQUALS, "ACTIVE")) + .withCriteria(new QFilterCriteria("expires_at", QCriteriaOperator.LESS_THAN_OR_EQUALS, SecuredDataRuntime.current().config().getClock().instant()))); + for(QRecord row : rows) + { + finish(row, "EXPIRED", "BREAK_GLASS_EXPIRED"); + } + return (rows.size()); + })); + } + } + + /******************************************************************************* + ** Update, audit and resolve using one stable per-session alert key. + *******************************************************************************/ + private QRecord finish(QRecord row, String status, String event) throws QException + { + QRecord update = new QRecord().withValue("id", row.getValue("id")).withValue("status", status) + .withValue("ended_at", SecuredDataRuntime.current().config().getClock().instant()).withValue("ended_by", SecuredDataResolver.userId()); + QRecord result = SystemWrites.call(() -> new UpdateAction().execute(new UpdateInput(SecuredDataTables.BREAK_GLASS).withRecord(update)).getRecords().getFirst()); + if(result.getErrors() != null && !result.getErrors().isEmpty()) + { + throw (new QException("Unable to finish break-glass session.")); + } + SecuredDataRuntime.current().invalidatePolicies(); + log(row, event); + QAlerts.resolve("secured_data.break_glass", dedup(row), status); + return (result); + } + + /******************************************************************************* + ** Audit the actor and affected session without copying protected field values. + *******************************************************************************/ + private void log(QRecord row, String event) throws QException + { + SystemWrites.insertOne(SecuredDataTables.ACCESS_LOG, ActivityLog.row(event, SecuredDataTables.BREAK_GLASS, "ALLOWED") + .withValue("break_glass_id", row.getValue("id")).withValue("record_id", row.getValueString("id")).withValue("reason", row.getValue("reason"))); + } + + /******************************************************************************* + ** Stable alert identity. + *******************************************************************************/ + private String dedup(QRecord row) + { + return ("secured_data.break_glass:" + row.getValueString("id")); + } + + /******************************************************************************* + ** RC.5 lacks #1125; use the notify SPI when installed without a qbit-alerts dependency. + *******************************************************************************/ + private void notifyAdmins(QRecord row) + { + try + { + Class signalClass = Class.forName("com.kingsrook.qqq.backend.core.alerts.NotificationSignal"); + Class audienceClass = Class.forName("com.kingsrook.qqq.backend.core.alerts.NotificationAudience"); + Object signal = signalClass.getConstructor(String.class).newInstance("Break-glass access started"); + signalClass.getMethod("withAudience", audienceClass).invoke(signal, audienceClass.getMethod("permission", String.class).invoke(null, SecuredDataPermissions.ADMIN)); + signalClass.getMethod("withSourceName", String.class).invoke(signal, "secured_data"); + signalClass.getMethod("withSourceKey", String.class).invoke(signal, dedup(row)); + QAlerts.class.getMethod("notify", signalClass).invoke(null, signal); + } + catch(ClassNotFoundException | NoSuchMethodException expected) + { + // TODO(core #1125): remove reflection at RC.6 and use the notification SPI directly. + LOG.warn("Break-glass admin bell notification unavailable: core notification SPI #1125 is not installed."); + } + catch(ReflectiveOperationException e) + { + LOG.warn("Unable to deliver break-glass admin bell notification."); + } + } + + /******************************************************************************* + ** Deny unauthenticated and assistant invocations in direct service calls too. + *******************************************************************************/ + private void checkChannel() throws QPermissionDeniedException + { + if(QContext.getQSession() == null || QContext.getQSession().getUser() == null + || "anonymous".equals(SecuredDataResolver.userId()) || SecuredDataResolver.userId().isBlank() + || "assistant".equals(CoreMetadataBridge.channel(QContext.getQSession()))) + { + throw (new QPermissionDeniedException("Permission denied.")); + } + } +} diff --git a/src/main/java/com/kingsrook/qbits/secureddata/metadata/FieldsOfTablePossibleValueSource.java b/src/main/java/com/kingsrook/qbits/secureddata/metadata/FieldsOfTablePossibleValueSource.java index 3779476..f72be10 100644 --- a/src/main/java/com/kingsrook/qbits/secureddata/metadata/FieldsOfTablePossibleValueSource.java +++ b/src/main/java/com/kingsrook/qbits/secureddata/metadata/FieldsOfTablePossibleValueSource.java @@ -58,9 +58,21 @@ public class FieldsOfTablePossibleValueSource implements QCustomPossibleValuePro public List> search(SearchPossibleValueSourceInput input) throws QException { Object tableName = input.getOtherValues() == null ? null : input.getOtherValues().get("table_name"); + if(tableName == null && input.getOtherValues() != null) + { + tableName = input.getOtherValues().get("tableName"); + } List> values = new ArrayList<>(); - QTableMetaData table = tableName == null ? null : QContext.getQInstance().getTable(tableName.toString()); - if(table != null && PermissionsHelper.hasTablePermission(new GetInput(table.getName()), table.getName(), TablePermissionSubType.READ)) + if(tableName == null || tableName.toString().isBlank()) + { + return (values); + } + QTableMetaData table = QContext.getQInstance().getTable(tableName.toString()); + if(table == null) + { + throw (new QException("Unknown table.")); + } + if(PermissionsHelper.hasTablePermission(new GetInput(table.getName()), table.getName(), TablePermissionSubType.READ)) { table.getFields().values().forEach(field -> values.add(new QPossibleValue<>(field.getName(), field.getLabel()))); } diff --git a/src/main/java/com/kingsrook/qbits/secureddata/metadata/SecuredDataTableCustomizers.java b/src/main/java/com/kingsrook/qbits/secureddata/metadata/SecuredDataTableCustomizers.java index cd4a541..9fd19ce 100644 --- a/src/main/java/com/kingsrook/qbits/secureddata/metadata/SecuredDataTableCustomizers.java +++ b/src/main/java/com/kingsrook/qbits/secureddata/metadata/SecuredDataTableCustomizers.java @@ -165,15 +165,17 @@ public class SecuredDataTableCustomizers implements TableCustomizerInterface ** Only successful mutations invalidate; transactions invalidate after commit. *******************************************************************************/ @Override - public List postInsertOrUpdate(AbstractActionInput input, List records, Optional> oldRecords) + public List postInsertOrUpdate(AbstractActionInput input, List records, Optional> oldRecords) throws QException { if(input instanceof InsertInput insert) { changed(insert.getTableName(), records, insert.getTransaction()); + audit(insert.getTableName(), records, "INSERT", insert.getTransaction()); } else if(input instanceof UpdateInput update) { changed(update.getTableName(), records, update.getTransaction()); + audit(update.getTableName(), records, "UPDATE", update.getTransaction()); } return (records); } @@ -184,9 +186,10 @@ public class SecuredDataTableCustomizers implements TableCustomizerInterface ** Deletions change the same versions as inserts and updates. *******************************************************************************/ @Override - public List postDelete(DeleteInput input, List records) + public List postDelete(DeleteInput input, List records) throws QException { changed(input.getTableName(), records, input.getTransaction()); + audit(input.getTableName(), records, "DELETE", input.getTransaction()); return (records); } @@ -217,4 +220,25 @@ public class SecuredDataTableCustomizers implements TableCustomizerInterface } } } + + /******************************************************************************* + ** Value-free mutation audit in the same transaction as the changed metadata. + *******************************************************************************/ + private static void audit(String table, List records, String action, QBackendTransaction transaction) throws QException + { + if(!List.of(SecuredDataTables.POLICY, SecuredDataTables.CLASSIFICATION).contains(table)) + { + return; + } + for(QRecord record : records) + { + if(record.getErrors() == null || record.getErrors().isEmpty()) + { + SystemWrites.insertOne(SecuredDataTables.ACCESS_LOG, com.kingsrook.qbits.secureddata.support.ActivityLog.row( + SecuredDataTables.POLICY.equals(table) ? "POLICY_CHANGE" : "CLASSIFICATION_CHANGE", table, "ALLOWED") + .withValue("record_id", record.getValueString("id")).withValue("action_name", action), + transaction == null ? QContext.getQBackendTransaction() : transaction); + } + } + } } diff --git a/src/main/java/com/kingsrook/qbits/secureddata/metadata/SecuredDataTables.java b/src/main/java/com/kingsrook/qbits/secureddata/metadata/SecuredDataTables.java index 9a3d1f1..698ddd8 100644 --- a/src/main/java/com/kingsrook/qbits/secureddata/metadata/SecuredDataTables.java +++ b/src/main/java/com/kingsrook/qbits/secureddata/metadata/SecuredDataTables.java @@ -120,12 +120,7 @@ public final class SecuredDataTables private static QFieldMetaData field(String name, QFieldType type) { QFieldMetaData field = new QFieldMetaData(name, type).withBackendName(name).withLabel(name.replace('_', ' ')); - if("field_name".equals(name)) - { - field.withPossibleValueSourceFilter(new com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter() - .withCriteria(new com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria("table_name", - com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator.EQUALS, "${input.table_name}"))); - } + // TODO(contingent): dependsOn("table_name") once the core RC has it return (field); } @@ -199,7 +194,7 @@ public final class SecuredDataTables .withField(field("table_name", QFieldType.STRING).withMaxLength(255).withIsRequired(true).withPossibleValueSourceName("tables")) .withField(field("record_id", QFieldType.STRING).withMaxLength(255).withIsRequired(true)) .withField(field("field_name", QFieldType.STRING).withMaxLength(255).withIsRequired(true).withPossibleValueSourceName("secured_data_fields")) - .withField(field("reason", QFieldType.STRING).withMaxLength(255)) + .withField(field("reason", QFieldType.STRING).withMaxLength(500)) .withField(field("reason_text", QFieldType.TEXT)) .withField(field("granted_at", QFieldType.DATE_TIME).withIsRequired(true)) .withField(field("expires_at", QFieldType.DATE_TIME).withIsRequired(true)) @@ -220,7 +215,7 @@ public final class SecuredDataTables QTableMetaData table = base(config, BREAK_GLASS, "Break-glass session") .withField(field("user_id", QFieldType.STRING).withMaxLength(255).withIsRequired(true)) .withField(field("user_name", QFieldType.STRING).withMaxLength(255)) - .withField(field("reason", QFieldType.STRING).withMaxLength(255)) + .withField(field("reason", QFieldType.STRING).withMaxLength(500)) .withField(field("reason_text", QFieldType.TEXT)) .withField(field("classification_ceiling", QFieldType.STRING).withMaxLength(255).withDefaultValue("PHI").withPossibleValueSourceName("secured_data_classification")) .withField(field("table_name", QFieldType.STRING).withMaxLength(255).withPossibleValueSourceName("tables")) @@ -251,7 +246,7 @@ public final class SecuredDataTables .withField(field("field_name", QFieldType.TEXT)) .withField(field("record_count", QFieldType.INTEGER)) .withField(field("action_name", QFieldType.STRING).withMaxLength(255)) - .withField(field("reason", QFieldType.STRING).withMaxLength(255)) + .withField(field("reason", QFieldType.STRING).withMaxLength(500)) .withField(field("reason_text", QFieldType.TEXT)) .withField(field("client_channel", QFieldType.STRING).withMaxLength(255)) .withField(field("outcome", QFieldType.STRING).withMaxLength(255).withPossibleValueSourceName("secured_data_outcome")) diff --git a/src/main/java/com/kingsrook/qbits/secureddata/processes/BreakGlassProcesses.java b/src/main/java/com/kingsrook/qbits/secureddata/processes/BreakGlassProcesses.java new file mode 100644 index 0000000..dfb8047 --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/secureddata/processes/BreakGlassProcesses.java @@ -0,0 +1,233 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.secureddata.processes; + +import java.io.Serializable; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import com.kingsrook.qbits.secureddata.SecuredDataQBitConfig; +import com.kingsrook.qbits.secureddata.breakglass.BreakGlassService; +import com.kingsrook.qbits.secureddata.metadata.SecuredDataTables; +import com.kingsrook.qbits.secureddata.standin.DataClassification; +import com.kingsrook.qbits.secureddata.support.SystemWrites; +import com.kingsrook.qqq.backend.core.actions.processes.BackendStep; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine; +import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface; +import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput; +import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput; +import com.kingsrook.qqq.backend.core.model.actions.processes.Status; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference; +import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType; +import com.kingsrook.qqq.backend.core.model.metadata.permissions.PermissionLevel; +import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRules; +import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.processes.QComponentType; +import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendComponentMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendStepMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.scheduleing.QScheduleMetaData; +import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractLoadStep; +import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractTransformStep; +import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ExtractViaQueryStep; +import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ProcessSummaryProviderInterface; +import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess; + +/******************************************************************************* + ** Table/record actions and the expiry schedule. Never added to navigation. + *******************************************************************************/ +public final class BreakGlassProcesses +{ + /******************************************************************************* + ** Utility class. + *******************************************************************************/ + private BreakGlassProcesses() + { + } + + /******************************************************************************* + ** A start form, a streamed Review/Result end action, and a hidden scheduled job. + *******************************************************************************/ + public static List processes(SecuredDataQBitConfig config) + { + QProcessMetaData start = new QProcessMetaData().withName(BreakGlassService.START).withLabel("Start break-glass") + .withTableName(SecuredDataTables.BREAK_GLASS).withMinInputRecords(0).withMaxInputRecords(0) + .withPermissionRules(new QPermissionRules().withLevel(PermissionLevel.HAS_ACCESS_PERMISSION)) + .withStep(new QFrontendStepMetaData().withName("input").withLabel("Start break-glass") + .withComponent(new QFrontendComponentMetaData().withType(QComponentType.EDIT_FORM)) + .withFormField(new QFieldMetaData("reason", QFieldType.STRING).withLabel("Reason").withIsRequired(true).withMaxLength(500)) + .withFormField(new QFieldMetaData("duration_minutes", QFieldType.INTEGER).withLabel("Duration minutes").withDefaultValue(60)) + .withFormField(new QFieldMetaData("table_name", QFieldType.STRING).withLabel("Table scope").withPossibleValueSourceName("tables")) + .withFormField(new QFieldMetaData("classification_ceiling", QFieldType.STRING).withLabel("Classification ceiling") + .withDefaultValue("PHI").withPossibleValueSourceName("secured_data_classification"))) + .withStep(new QBackendStepMetaData().withName("start").withCode(new QCodeReference(Start.class))) + .withStep(new QFrontendStepMetaData().withName("result").withLabel("Result") + .withComponent(new QFrontendComponentMetaData().withType(QComponentType.VIEW_FORM)) + .withViewField(new QFieldMetaData("message", QFieldType.STRING))); + Map values = new HashMap<>(); + values.put(StreamedETLWithFrontendProcess.FIELD_SOURCE_TABLE, SecuredDataTables.BREAK_GLASS); + values.put(StreamedETLWithFrontendProcess.FIELD_DESTINATION_TABLE, SecuredDataTables.BREAK_GLASS); + values.put(StreamedETLWithFrontendProcess.FIELD_SUPPORTS_FULL_VALIDATION, true); + values.put(StreamedETLWithFrontendProcess.FIELD_DO_FULL_VALIDATION, true); + values.put(StreamedETLWithFrontendProcess.FIELD_PREVIEW_MESSAGE, "The selected break-glass sessions will be ended."); + QProcessMetaData end = StreamedETLWithFrontendProcess.defineProcessMetaData(EndExtract.class, EndReview.class, EndLoad.class, values) + .withName(BreakGlassService.END).withLabel("End break-glass").withTableName(SecuredDataTables.BREAK_GLASS).withMinInputRecords(1) + .withPermissionRules(new QPermissionRules().withLevel(PermissionLevel.NOT_PROTECTED)); + QProcessMetaData expire = new QProcessMetaData().withName(BreakGlassService.EXPIRE).withLabel("Expire break-glass sessions").withIsHidden(true) + .withPermissionRules(new QPermissionRules().withLevel(PermissionLevel.HAS_ACCESS_PERMISSION)) + .withStep(new QBackendStepMetaData().withName("expire").withCode(new QCodeReference(Expire.class))); + if(config.getSchedulerName() != null) + { + expire.withSchedule(new QScheduleMetaData().withSchedulerName(config.getSchedulerName()).withRepeatSeconds(60).withInitialDelaySeconds(30)); + } + return (List.of(start, end, expire)); + } + + /******************************************************************************* + ** Start after the form is submitted. + *******************************************************************************/ + public static class Start implements BackendStep + { + @Override + public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException + { + String ceiling = input.getValueString("classification_ceiling"); + DataClassification classification; + try + { + classification = ceiling == null ? null : DataClassification.valueOf(ceiling); + } + catch(IllegalArgumentException e) + { + throw (new com.kingsrook.qqq.backend.core.exceptions.QBadRequestException("Unknown classification ceiling.")); + } + new BreakGlassService().start(input.getValueString("reason"), input.getValueInteger("duration_minutes"), input.getValueString("table_name"), classification); + output.addValue("message", "Break-glass access is active."); + } + } + + /******************************************************************************* + ** Extract with system visibility; ownership is checked before any row enters the pipe. + *******************************************************************************/ + public static class EndExtract extends ExtractViaQueryStep + { + @Override + public Integer doCount(RunBackendStepInput input) + { + return (null); + } + + @Override + public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException + { + for(QRecord row : SystemWrites.query(SecuredDataTables.BREAK_GLASS, getQueryFilter(input))) + { + new BreakGlassService().checkOwner(row); + getRecordPipe().addRecord(row); + } + } + } + + /******************************************************************************* + ** Review validates ownership and reports the change in future and past tense. + *******************************************************************************/ + public static class EndReview extends AbstractTransformStep implements ProcessSummaryProviderInterface + { + private final ProcessSummaryLine line = summary(); + + @Override + public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException + { + for(QRecord row : input.getRecords()) + { + new BreakGlassService().checkOwner(row); + if("ACTIVE".equals(row.getValueString("status"))) + { + line.incrementCountAndAddPrimaryKey(row.getValue("id")); + output.getRecords().add(row); + } + } + } + + @Override + public ArrayList getProcessSummary(RunBackendStepOutput output, boolean isForResultScreen) + { + ArrayList lines = new ArrayList<>(); + line.addSelfToListIfAnyCount(lines); + return (lines); + } + } + + /******************************************************************************* + ** Apply the reviewed operation, rechecking persisted ownership at execution time. + *******************************************************************************/ + public static class EndLoad extends AbstractLoadStep implements ProcessSummaryProviderInterface + { + private final ProcessSummaryLine line = summary(); + + @Override + public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException + { + for(QRecord row : input.getRecords()) + { + output.getRecords().add(new BreakGlassService().end(row.getValue("id"))); + line.incrementCountAndAddPrimaryKey(row.getValue("id")); + } + } + + @Override + public ArrayList getProcessSummary(RunBackendStepOutput output, boolean isForResultScreen) + { + ArrayList lines = new ArrayList<>(); + line.addSelfToListIfAnyCount(lines); + return (lines); + } + } + + /******************************************************************************* + ** Standard ETL summary wording. + *******************************************************************************/ + private static ProcessSummaryLine summary() + { + ProcessSummaryLine line = new ProcessSummaryLine(Status.OK, 0, null); + line.setSingularFutureMessage("session will be ended"); + line.setPluralFutureMessage("sessions will be ended"); + line.setSingularPastMessage("session was ended"); + line.setPluralPastMessage("sessions were ended"); + return (line); + } + + /******************************************************************************* + ** Expiry job; system context and writes are established by the service. + *******************************************************************************/ + public static class Expire implements BackendStep + { + @Override + public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException + { + output.addValue("expired", new BreakGlassService().expire()); + } + } +} diff --git a/src/main/java/com/kingsrook/qbits/secureddata/reveal/RevealCappedException.java b/src/main/java/com/kingsrook/qbits/secureddata/reveal/RevealCappedException.java new file mode 100644 index 0000000..c620e9b --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/secureddata/reveal/RevealCappedException.java @@ -0,0 +1,37 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.secureddata.reveal; + +import com.kingsrook.qqq.backend.core.exceptions.QException; + +/******************************************************************************* + ** Transport-independent refusal, mapped to 429 by the UI API. + *******************************************************************************/ +public class RevealCappedException extends QException +{ + /******************************************************************************* + ** Daily allowance exhausted. + *******************************************************************************/ + public RevealCappedException() + { + super("Daily reveal allowance exhausted."); + } +} diff --git a/src/main/java/com/kingsrook/qbits/secureddata/reveal/RevealOptions.java b/src/main/java/com/kingsrook/qbits/secureddata/reveal/RevealOptions.java new file mode 100644 index 0000000..25eac3f --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/secureddata/reveal/RevealOptions.java @@ -0,0 +1,31 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.secureddata.reveal; + +import java.util.List; + +/******************************************************************************* + ** Policy-driven reveal dialog. Null remainingDailyAllowance means unlimited. + *******************************************************************************/ +public record RevealOptions(List reasonOptions, boolean allowFreeTextReason, boolean requireFreeText, + int revealMinutes, Integer remainingDailyAllowance) +{ +} diff --git a/src/main/java/com/kingsrook/qbits/secureddata/reveal/RevealResult.java b/src/main/java/com/kingsrook/qbits/secureddata/reveal/RevealResult.java new file mode 100644 index 0000000..410197a --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/secureddata/reveal/RevealResult.java @@ -0,0 +1,31 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.secureddata.reveal; + +import java.io.Serializable; +import java.time.Instant; + +/******************************************************************************* + ** The authorized value and its grant; never persist this response in an audit row. + *******************************************************************************/ +public record RevealResult(Serializable value, String displayValue, Instant expiresAt, Serializable revealId) +{ +} diff --git a/src/main/java/com/kingsrook/qbits/secureddata/reveal/RevealService.java b/src/main/java/com/kingsrook/qbits/secureddata/reveal/RevealService.java new file mode 100644 index 0000000..4b8448b --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/secureddata/reveal/RevealService.java @@ -0,0 +1,215 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.secureddata.reveal; + +import java.io.Serializable; +import java.time.Instant; +import java.time.ZoneOffset; +import java.util.Arrays; +import java.util.Comparator; +import java.util.List; +import java.util.Objects; +import com.kingsrook.qbits.secureddata.SecuredDataResolver; +import com.kingsrook.qbits.secureddata.SecuredDataRuntime; +import com.kingsrook.qbits.secureddata.metadata.SecuredDataTables; +import com.kingsrook.qbits.secureddata.standin.FieldAccessTier; +import com.kingsrook.qbits.secureddata.support.ActivityLog; +import com.kingsrook.qbits.secureddata.support.CoreMetadataBridge; +import com.kingsrook.qbits.secureddata.support.SystemWrites; +import com.kingsrook.qqq.backend.core.actions.permissions.CorePermissionEnforcer; +import com.kingsrook.qqq.backend.core.actions.permissions.PermissionEnforcement; +import com.kingsrook.qqq.backend.core.actions.permissions.TablePermissionSubType; +import com.kingsrook.qqq.backend.core.actions.tables.GetAction; +import com.kingsrook.qqq.backend.core.alerts.AlertSignal; +import com.kingsrook.qqq.backend.core.alerts.QAlerts; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.exceptions.QBadRequestException; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.exceptions.QNotFoundException; +import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException; +import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetInput; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData; + +/******************************************************************************* + ** Reveal authorization and grant creation independent of any HTTP transport. + *******************************************************************************/ +public class RevealService +{ + /******************************************************************************* + ** Use the current instance runtime, including its injectable clock. + *******************************************************************************/ + public RevealOptions options(String tableName, String fieldName) throws QException + { + requireActor(); + QRecord policy = policy(tableName, fieldName); + String reasons = policy.getValueString("reason_options"); + Integer cap = policy.getValueInteger("daily_reveal_cap"); + return (new RevealOptions(reasons == null ? List.of() : Arrays.stream(reasons.split("\\R")).map(String::trim).filter(s -> !s.isEmpty()).toList(), + !Boolean.FALSE.equals(policy.getValueBoolean("allow_free_text_reason")), Boolean.TRUE.equals(policy.getValueBoolean("require_free_text")), + Objects.requireNonNullElse(policy.getValueInteger("reveal_minutes"), 5), cap == null ? null : Math.max(0, cap - usedToday()))); + } + + /******************************************************************************* + ** A single coherent policy: unlimited/highest cap, then longest duration, then id. + *******************************************************************************/ + private QRecord policy(String tableName, String fieldName) throws QException + { + requireTarget(tableName, fieldName); + if("assistant".equals(CoreMetadataBridge.channel(QContext.getQSession()))) + { + throw (new QPermissionDeniedException("Reveal is unavailable on the assistant channel.")); + } + QTableMetaData table = QContext.getQInstance().getTable(tableName); + if(table == null || !table.getFields().containsKey(fieldName)) + { + throw (new QNotFoundException("Table or field not found.")); + } + GetInput input = new GetInput(tableName); + input.setPermissionEnforcement(PermissionEnforcement.ENFORCE); + CorePermissionEnforcer.checkTablePermissions(input, TablePermissionSubType.READ); + if(SecuredDataRuntime.current().resolver().resolve(table, table.getField(fieldName)).tier() != FieldAccessTier.REVEALABLE) + { + throw (new QPermissionDeniedException("Field is not revealable.")); + } + return (SecuredDataRuntime.current().resolver().revealPolicies(table, fieldName).stream() + .max(Comparator.comparingInt((QRecord row) -> Objects.requireNonNullElse(row.getValueInteger("daily_reveal_cap"), Integer.MAX_VALUE)) + .thenComparingInt(row -> Objects.requireNonNullElse(row.getValueInteger("reveal_minutes"), 5)) + .thenComparing(row -> row.getValueString("id"), Comparator.reverseOrder())) + .orElse(new QRecord().withValue("reveal_minutes", 5).withValue("allow_free_text_reason", true))); + } + + /******************************************************************************* + ** UTC daily count across all tables, including expired grants. + *******************************************************************************/ + public int usedToday() + { + Instant start = SecuredDataRuntime.current().config().getClock().instant().atOffset(ZoneOffset.UTC).toLocalDate().atStartOfDay().toInstant(ZoneOffset.UTC); + return (SystemWrites.query(SecuredDataTables.REVEAL_GRANT, new QQueryFilter() + .withCriteria(new QFilterCriteria("user_id", QCriteriaOperator.EQUALS, SecuredDataResolver.userId())) + .withCriteria(new QFilterCriteria("granted_at", QCriteriaOperator.GREATER_THAN_OR_EQUALS, start))).size()); + } + + /******************************************************************************* + ** Serialize the cap check and write within this runtime; first fetch always as caller. + *******************************************************************************/ + public RevealResult reveal(String tableName, Serializable primaryKey, String fieldName, String reason, String reasonText) throws QException + { + requireActor(); + SecuredDataRuntime runtime = SecuredDataRuntime.current(); + synchronized(runtime) + { + if("assistant".equals(CoreMetadataBridge.channel(QContext.getQSession()))) + { + denied(tableName, primaryKey, fieldName, "DENIED"); + throw (new QPermissionDeniedException("Reveal is unavailable on the assistant channel.")); + } + if(primaryKey == null) + { + throw (new QBadRequestException("primaryKey is required.")); + } + requireTarget(tableName, fieldName); + GetInput input = new GetInput(tableName).withPrimaryKey(primaryKey).withShouldFetchHeavyFields(true); + input.setPermissionEnforcement(PermissionEnforcement.ENFORCE); + if(new GetAction().execute(input).getRecord() == null) + { + throw (new QNotFoundException("No record was found")); + } + RevealOptions options = options(tableName, fieldName); + validateReason(options, reason, reasonText); + if(options.remainingDailyAllowance() != null && options.remainingDailyAllowance() == 0) + { + denied(tableName, primaryKey, fieldName, "CAPPED"); + QAlerts.raise(new AlertSignal("secured_data.reveal_capped").withDedupKey("secured_data.reveal_capped:" + SecuredDataResolver.userId()) + .withTitle("Daily reveal allowance exhausted").withSummary("Review the secured-data access log.")); + throw (new RevealCappedException()); + } + QRecord raw = SystemWrites.call(() -> new GetAction().execute(new GetInput(tableName).withPrimaryKey(primaryKey) + .withShouldFetchHeavyFields(true).withShouldTranslatePossibleValues(true)).getRecord()); + if(raw == null) + { + throw (new QNotFoundException("No record was found")); + } + com.kingsrook.qqq.backend.core.actions.values.QValueFormatter.setDisplayValuesInRecords(QContext.getQInstance().getTable(tableName), List.of(raw)); + Instant now = runtime.config().getClock().instant(); + Instant expiry = now.plusSeconds(options.revealMinutes() * 60L); + QRecord audit = SystemWrites.insertOne(SecuredDataTables.ACCESS_LOG, ActivityLog.row("REVEAL", tableName, "ALLOWED") + .withValue("record_id", String.valueOf(primaryKey)).withValue("field_name", fieldName).withValue("reason", reason).withValue("reason_text", reasonText)); + QRecord grant = SystemWrites.insertOne(SecuredDataTables.REVEAL_GRANT, new QRecord() + .withValue("user_id", SecuredDataResolver.userId()).withValue("user_name", QContext.getQSession().getUser().getFullName()) + .withValue("table_name", tableName).withValue("field_name", fieldName).withValue("record_id", String.valueOf(primaryKey)) + .withValue("reason", reason).withValue("reason_text", reasonText).withValue("granted_at", now).withValue("expires_at", expiry) + .withValue("access_log_id", audit.getValue("id"))); + runtime.resolver().invalidateReveals(); + return (new RevealResult(raw.getValue(fieldName), raw.getDisplayValue(fieldName), expiry, grant.getValue("id"))); + } + } + + /******************************************************************************* + ** Grants require a stable authenticated user identity across sessions. + *******************************************************************************/ + private void requireActor() throws QPermissionDeniedException + { + if("anonymous".equals(SecuredDataResolver.userId()) || SecuredDataResolver.userId().isBlank()) + { + throw (new QPermissionDeniedException("Reveal requires an authenticated user.")); + } + } + + /******************************************************************************* + ** Missing target identifiers are malformed input, not an internal lookup error. + *******************************************************************************/ + private void requireTarget(String table, String field) throws QBadRequestException + { + if(table == null || table.isBlank() || field == null || field.isBlank()) + { + throw (new QBadRequestException("tableName and fieldName are required.")); + } + } + + /******************************************************************************* + ** Persist refusals without user supplied text or protected values. + *******************************************************************************/ + private void denied(String table, Serializable pk, String field, String outcome) throws QException + { + SystemWrites.insertOne(SecuredDataTables.ACCESS_LOG, ActivityLog.row("REVEAL_DENIED", table, outcome) + .withValue("record_id", String.valueOf(pk)).withValue("field_name", field)); + } + + /******************************************************************************* + ** Reasons are a listed option or allowed free text; both inputs are bounded. + *******************************************************************************/ + private void validateReason(RevealOptions options, String reason, String text) throws QBadRequestException + { + boolean hasText = text != null && !text.isBlank(); + boolean listed = reason != null && options.reasonOptions().contains(reason); + if((reason != null && reason.length() > 500) || (text != null && text.length() > 500) + || (options.requireFreeText() && !hasText) + || (!options.allowFreeTextReason() && hasText && !options.requireFreeText()) + || (!listed && !(options.allowFreeTextReason() && (hasText || (reason != null && !reason.isBlank()))))) + { + throw (new QBadRequestException("Supply an allowed reason and required explanation, each at most 500 characters.")); + } + } +} diff --git a/src/main/java/com/kingsrook/qbits/secureddata/support/ActivityLog.java b/src/main/java/com/kingsrook/qbits/secureddata/support/ActivityLog.java new file mode 100644 index 0000000..89a1294 --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/secureddata/support/ActivityLog.java @@ -0,0 +1,52 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.secureddata.support; + +import com.kingsrook.qbits.secureddata.SecuredDataResolver; +import com.kingsrook.qbits.secureddata.SecuredDataRuntime; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.model.data.QRecord; + +/******************************************************************************* + ** Synchronous activity audit rows for state-changing operations. + *******************************************************************************/ +public final class ActivityLog +{ + /******************************************************************************* + ** Utility class. + *******************************************************************************/ + private ActivityLog() + { + } + + /******************************************************************************* + ** Capture the actor before entering the system session. No field values. + *******************************************************************************/ + public static QRecord row(String event, String table, String outcome) + { + return (new QRecord().withValue("event_type", event).withValue("table_name", table) + .withValue("user_id", SecuredDataResolver.userId()) + .withValue("user_name", QContext.getQSession().getUser() == null ? "system" : QContext.getQSession().getUser().getFullName()) + .withValue("client_channel", CoreMetadataBridge.channel(QContext.getQSession())) + .withValue("created_at", SecuredDataRuntime.current().config().getClock().instant()) + .withValue("outcome", outcome).withValue("anomaly_flag", false)); + } +} diff --git a/src/main/java/com/kingsrook/qbits/secureddata/support/SystemWrites.java b/src/main/java/com/kingsrook/qbits/secureddata/support/SystemWrites.java index e6abacd..a18d3c9 100644 --- a/src/main/java/com/kingsrook/qbits/secureddata/support/SystemWrites.java +++ b/src/main/java/com/kingsrook/qbits/secureddata/support/SystemWrites.java @@ -22,6 +22,7 @@ package com.kingsrook.qbits.secureddata.support; import java.util.List; +import com.kingsrook.qqq.backend.core.actions.QBackendTransaction; import com.kingsrook.qqq.backend.core.actions.permissions.CorePermissionEnforcer; import com.kingsrook.qqq.backend.core.actions.tables.InsertAction; import com.kingsrook.qqq.backend.core.actions.tables.QueryAction; @@ -120,11 +121,53 @@ public final class SystemWrites *******************************************************************************/ public static QRecord insertOne(String table, QRecord record) throws QException { - QRecord inserted = call(() -> new InsertAction().execute(new InsertInput(table).withRecord(record)).getRecords().getFirst()); + return (insertOne(table, record, QContext.getQBackendTransaction())); + } + + /******************************************************************************* + ** Explicit transaction for post-write customizers; audit rolls back with its source. + *******************************************************************************/ + public static QRecord insertOne(String table, QRecord record, QBackendTransaction transaction) throws QException + { + if(com.kingsrook.qbits.secureddata.metadata.SecuredDataTables.ACCESS_LOG.equals(table)) + { + synchronized(com.kingsrook.qbits.secureddata.SecuredDataRuntime.current().anomalies()) + { + return (insertAndAlert(table, record, transaction)); + } + } + return (insertAndAlert(table, record, transaction)); + } + + /******************************************************************************* + ** Serialize same-node counter evaluation and insertion. Preserve enumeration flags. + *******************************************************************************/ + private static QRecord insertAndAlert(String table, QRecord record, QBackendTransaction transaction) throws QException + { + boolean anomaly = com.kingsrook.qbits.secureddata.metadata.SecuredDataTables.ACCESS_LOG.equals(table) + && com.kingsrook.qbits.secureddata.SecuredDataRuntime.current().anomalies().evaluate(record); + if(anomaly) + { + record.setValue("anomaly_flag", true); + } + QRecord inserted = call(() -> new InsertAction().execute(new InsertInput(table).withRecord(record).withTransaction(transaction)).getRecords().getFirst()); if(inserted.getErrors() != null && !inserted.getErrors().isEmpty()) { throw (new QException("Secured-data system insert failed")); } + if(anomaly) + { + if(transaction == null) + { + com.kingsrook.qbits.secureddata.SecuredDataAnomalies.raise(inserted); + } + else + { + var context = new com.kingsrook.qqq.backend.core.context.CapturedContext(QContext.getQInstance(), QContext.getQSession()); + transaction.addAfterCommitCallback(() -> QContext.withTemporaryContext(context, + () -> com.kingsrook.qbits.secureddata.SecuredDataAnomalies.raise(inserted))); + } + } return (inserted); } } diff --git a/src/main/java/com/kingsrook/qbits/secureddata/support/UiApiBridge.java b/src/main/java/com/kingsrook/qbits/secureddata/support/UiApiBridge.java new file mode 100644 index 0000000..cdeaf74 --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/secureddata/support/UiApiBridge.java @@ -0,0 +1,56 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.secureddata.support; + +import com.kingsrook.qqq.backend.core.model.metadata.QInstance; + +/******************************************************************************* + ** Optional UI API registration, with no linkage to unreleased middleware on RC.5. + *******************************************************************************/ +public final class UiApiBridge +{ + /******************************************************************************* + ** Utility class. + *******************************************************************************/ + private UiApiBridge() + { + } + + /******************************************************************************* + ** STAND-IN(core=QRun-IO/qqq#1128, removeBy=RC.6): register directly at RC.6. + *******************************************************************************/ + public static void register(QInstance instance) + { + try + { + Class.forName("com.kingsrook.qqq.middleware.javalin.specs.extensions.MiddlewareSpecExtensionsMetaData"); + Class.forName("com.kingsrook.qbits.secureddata.uiapi.SecuredDataExtension").getMethod("register", QInstance.class).invoke(null, instance); + } + catch(ClassNotFoundException expected) + { + // RC.5 or a build without the uiapi source profile. + } + catch(ReflectiveOperationException e) + { + throw (new IllegalStateException("Unable to register secured-data UI API extension", e)); + } + } +} diff --git a/src/main/resources/openapi/qqq-v1/secured-data.yaml b/src/main/resources/openapi/qqq-v1/secured-data.yaml new file mode 100644 index 0000000..bb9a63d --- /dev/null +++ b/src/main/resources/openapi/qqq-v1/secured-data.yaml @@ -0,0 +1,195 @@ +components: + schemas: + SecuredDataBreakGlassV1: + properties: + id: + type: "integer" + reason: + type: "string" + expiresAt: + format: "date-time" + type: "string" + scope: + nullable: true + type: "string" + ceiling: + type: "string" + type: "object" + SecuredDataErrorV1: + properties: + error: + type: "string" + type: "object" + SecuredDataRevealAllowanceV1: + properties: + tableName: + type: "string" + fieldName: + type: "string" + remainingDailyAllowance: + nullable: true + type: "integer" + type: "object" + SecuredDataRevealOptionsResponseV1: + properties: + reasonOptions: + items: + type: "string" + type: "array" + allowFreeTextReason: + type: "boolean" + requireFreeText: + type: "boolean" + revealMinutes: + type: "integer" + remainingDailyAllowance: + nullable: true + type: "integer" + type: "object" + SecuredDataRevealRequestV1: + description: "tableName, primaryKey and fieldName are required. Supply a policy\ + \ reason or allowed free text; maximum 500 characters." + properties: + tableName: + type: "string" + fieldName: + type: "string" + primaryKey: + anyOf: + - type: "string" + - type: "number" + description: "Record primary key (string or number)." + reason: + maxLength: 500 + nullable: true + type: "string" + reasonText: + maxLength: 500 + nullable: true + type: "string" + type: "object" + SecuredDataRevealResponseV1: + properties: + value: + description: "Authorized raw field value." + nullable: true + displayValue: + nullable: true + type: "string" + expiresAt: + format: "date-time" + type: "string" + revealId: + type: "integer" + type: "object" + SecuredDataStatusResponseV1: + properties: + breakGlass: + $ref: "#/components/schemas/SecuredDataBreakGlassV1" + nullable: true + revealsUsedToday: + type: "integer" + revealAllowances: + items: + $ref: "#/components/schemas/SecuredDataRevealAllowanceV1" + type: "array" + type: "object" + securitySchemes: + sessionUuidCookie: + in: "cookie" + name: "sessionUUID" + type: "apiKey" +info: + contact: + email: "contact@kingsrook.com" + description: "Reveal grants and break-glass status" + title: "QQQ Middleware API extension: secured-data" + version: "1.0.0" +openapi: "3.0.3" +paths: + /qqq/v1/qbit/secured-data/reveal: + post: + requestBody: + content: + application/json: + schema: + $ref: "#/components/schemas/SecuredDataRevealRequestV1" + required: true + responses: + 400: + content: + application/json: + schema: + $ref: "#/components/schemas/SecuredDataErrorV1" + description: "Invalid reveal request" + 403: + content: + application/json: + schema: + $ref: "#/components/schemas/SecuredDataErrorV1" + description: "Reveal denied" + 404: + content: + application/json: + schema: + $ref: "#/components/schemas/SecuredDataErrorV1" + description: "Record not found or locked" + 200: + content: + application/json: + schema: + $ref: "#/components/schemas/SecuredDataRevealResponseV1" + description: "Successful Response" + 429: + content: + application/json: + schema: + $ref: "#/components/schemas/SecuredDataErrorV1" + description: "Daily reveal cap reached" + security: + - sessionUuidCookie: + - "N/A" + summary: "Reveal" + tags: + - "Secured data" + /qqq/v1/qbit/secured-data/reveal/options: + get: + parameters: + - in: "query" + name: "tableName" + required: true + schema: + type: "string" + - in: "query" + name: "fieldName" + required: true + schema: + type: "string" + responses: + 200: + content: + application/json: + schema: + $ref: "#/components/schemas/SecuredDataRevealOptionsResponseV1" + description: "Successful Response" + security: + - sessionUuidCookie: + - "N/A" + summary: "RevealOptions" + tags: + - "Secured data" + /qqq/v1/qbit/secured-data/status: + get: + responses: + 200: + content: + application/json: + schema: + $ref: "#/components/schemas/SecuredDataStatusResponseV1" + description: "Successful Response" + security: + - sessionUuidCookie: + - "N/A" + summary: "Status" + tags: + - "Secured data" diff --git a/src/test/java-uiapi/com/kingsrook/qbits/secureddata/SecuredDataHttpTest.java b/src/test/java-uiapi/com/kingsrook/qbits/secureddata/SecuredDataHttpTest.java new file mode 100644 index 0000000..640d0b4 --- /dev/null +++ b/src/test/java-uiapi/com/kingsrook/qbits/secureddata/SecuredDataHttpTest.java @@ -0,0 +1,147 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.secureddata; + +import java.net.URI; +import java.net.http.HttpClient; +import java.net.http.HttpRequest; +import java.net.http.HttpResponse; +import java.time.Instant; +import java.util.List; +import java.util.Map; +import com.kingsrook.qbits.secureddata.breakglass.BreakGlassService; +import com.kingsrook.qbits.secureddata.metadata.SecuredDataTables; +import com.kingsrook.qqq.backend.core.alerts.AlertResolution; +import com.kingsrook.qqq.backend.core.alerts.AlertSignal; +import com.kingsrook.qqq.backend.core.alerts.AlertSinkInterface; +import com.kingsrook.qqq.backend.core.alerts.AlertsInstanceMetaData; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.instances.AbstractQQQApplication; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType; +import com.kingsrook.qqq.backend.core.model.metadata.QInstance; +import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference; +import com.kingsrook.qqq.backend.core.model.session.QSession; +import com.kingsrook.qqq.backend.core.modules.authentication.QAuthenticationModuleCustomizerInterface; +import com.kingsrook.qqq.middleware.javalin.QApplicationJavalinServer; +import com.kingsrook.qqq.middleware.javalin.QJavalinImplementation; +import org.json.JSONObject; +import org.junit.jupiter.api.AfterEach; +import org.junit.jupiter.api.Test; +import static com.kingsrook.qbits.secureddata.SecuredDataTestSupport.*; +import static org.assertj.core.api.Assertions.*; + +class SecuredDataHttpTest +{ + QApplicationJavalinServer server; + int port; + + @AfterEach + void cleanup() + { + if(server != null) server.stop(); + QJavalinImplementation.setSessionCookieHttpOnly(false); + close(); + } + + public static class Sessions implements QAuthenticationModuleCustomizerInterface + { + public void customizeSession(QInstance instance, QSession session, Map context) + { + session.setPermissions(java.util.Set.of("person.read", "cs_admin")); + } + } + + void start() throws Exception + { + QInstance instance = setup(); + instance.getAuthentication().setType(QAuthenticationType.MOCK); + instance.getAuthentication().setCustomizer(new QCodeReference(Sessions.class)); + insert(SecuredDataTables.POLICY, policyRow("cs_admin", "PII", "REVEALABLE").withValue("daily_reveal_cap", 1)); + insert("person", new QRecord().withValue("id", 1).withValue("email", "HTTP-CANARY@example.invalid")); + server = new QApplicationJavalinServer(new AbstractQQQApplication() + { + public QInstance defineQInstance() { return instance; } + }).withPort(0).withServeFrontendNext(false).withServeFrontendMaterialDashboard(false).withServeLegacyUnversionedMiddlewareAPI(false).withStrictUiApiRoutes(true); + server.start(); + var serviceField = QApplicationJavalinServer.class.getDeclaredField("service"); + serviceField.setAccessible(true); + port = ((io.javalin.Javalin) serviceField.get(server)).port(); + } + + HttpResponse request(String path, String body, boolean assistant) throws Exception + { + var builder = HttpRequest.newBuilder(URI.create("http://127.0.0.1:" + port + "/qqq/v1" + path)).header("Cookie", "sessionUUID=phase-c-user"); + if(assistant) builder.header("X-QQQ-Client", "assistant"); + if(body != null) builder.header("Content-Type", "application/json").POST(HttpRequest.BodyPublishers.ofString(body)); + return HttpClient.newHttpClient().send(builder.build(), HttpResponse.BodyHandlers.ofString()); + } + + @Test + void optionsRevealStatusAndCapThroughRealMiddleware() throws Exception + { + start(); + var options = request("/qbit/secured-data/reveal/options?tableName=person&fieldName=email", null, false); + assertThat(options.statusCode()).as(options.body()).isEqualTo(200); + assertThat(new JSONObject(options.body()).getInt("remainingDailyAllowance")).isEqualTo(1); + String body = "{\"tableName\":\"person\",\"primaryKey\":1,\"fieldName\":\"email\",\"reason\":\"Investigation\"}"; + var reveal = request("/qbit/secured-data/reveal", body, false); + assertThat(reveal.statusCode()).as(reveal.body()).isEqualTo(200); + assertThat(new JSONObject(reveal.body()).getString("value")).isEqualTo("HTTP-CANARY@example.invalid"); + var status = request("/qbit/secured-data/status", null, false); + assertThat(status.statusCode()).as(status.body()).isEqualTo(200); + assertThat(new JSONObject(status.body()).getInt("revealsUsedToday")).isEqualTo(1); + assertThat(request("/qbit/secured-data/reveal", body, false).statusCode()).isEqualTo(429); + assertThat(request("/qbit/secured-data/reveal", body, true).statusCode()).isEqualTo(403); + assertThat(request("/qbit/secured-data/reveal", "{}", false).statusCode()).isEqualTo(400); + assertThat(request("/qbit/secured-data/reveal", "{\"primaryKey\":1}", false).statusCode()).isEqualTo(400); + assertThat(request("/qbit/secured-data/reveal/options?tableName=person&fieldName=missing", null, false).statusCode()).isEqualTo(404); + } + + @Test + void notificationTargetsOnlySecuredDataAdmins() throws Exception + { + Class signalType; + try { signalType = Class.forName("com.kingsrook.qqq.backend.core.alerts.NotificationSignal"); } + catch(ClassNotFoundException expected) { org.junit.jupiter.api.Assumptions.assumeTrue(false, "Installed core lacks notification SPI #1125"); return; } + var instance = setup(); + Bell.signal = null; + AlertsInstanceMetaData.ofOrWithNew(instance).withSink(new QCodeReference(Bell.class)); + var metadata = AlertsInstanceMetaData.ofOrWithNew(instance); + var setter = metadata.getClass().getDeclaredMethod("setLoadedSink", AlertSinkInterface.class); + setter.setAccessible(true); + setter.invoke(metadata, java.lang.reflect.Proxy.newProxyInstance(AlertSinkInterface.class.getClassLoader(), new Class[] {AlertSinkInterface.class}, + (proxy, method, args) -> { if(method.getName().equals("notify")) Bell.signal = args[0]; return null; })); + QContext.setQSession(user("u1", BreakGlassService.START + ".hasAccess")); + new BreakGlassService().start("Urgent incident investigation", 60, null, null); + assertThat(Bell.signal).isNotNull(); + Object audience = signalType.getMethod("getAudience").invoke(Bell.signal); + assertThat((List) audience.getClass().getMethod("getValues").invoke(audience)).isEqualTo(List.of("secured_data.admin")); + } + + public static class Bell implements AlertSinkInterface + { + static Object signal; + public void raise(AlertSignal ignored) { } + public void resolve(AlertResolution ignored) { } + public void heartbeat(String key, Instant at) { } + } +} diff --git a/src/test/java-uiapi/com/kingsrook/qbits/secureddata/SecuredDataUiApiContractTest.java b/src/test/java-uiapi/com/kingsrook/qbits/secureddata/SecuredDataUiApiContractTest.java new file mode 100644 index 0000000..420bfa5 --- /dev/null +++ b/src/test/java-uiapi/com/kingsrook/qbits/secureddata/SecuredDataUiApiContractTest.java @@ -0,0 +1,34 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.secureddata; + +import java.nio.file.Path; +import com.kingsrook.qbits.secureddata.uiapi.SecuredDataExtension; +import org.junit.jupiter.api.Test; + +class SecuredDataUiApiContractTest +{ + @Test + void committedContractMatches() throws Exception + { + UiApiContractAssertions.assertExtensionMatchesCommitted(SecuredDataExtension.defineExtension(), Path.of("src/main/resources/openapi/qqq-v1/secured-data.yaml")); + } +} diff --git a/src/test/java-uiapi/com/kingsrook/qbits/secureddata/UiApiContractAssertions.java b/src/test/java-uiapi/com/kingsrook/qbits/secureddata/UiApiContractAssertions.java new file mode 100644 index 0000000..47683c8 --- /dev/null +++ b/src/test/java-uiapi/com/kingsrook/qbits/secureddata/UiApiContractAssertions.java @@ -0,0 +1,43 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.secureddata; + +import java.nio.file.Files; +import java.nio.file.Path; +import com.kingsrook.qqq.middleware.javalin.specs.extensions.MiddlewareSpecExtension; +import com.kingsrook.qqq.middleware.javalin.specs.v1.MiddlewareVersionV1; +import com.kingsrook.qqq.middleware.javalin.tools.UiApiContract; +import static org.assertj.core.api.Assertions.assertThat; + +/** STAND-IN(core=QRun-IO/qqq#1128, removeBy=RC.6): supplied worktree exposes UiApiContract, not the documented assertion facade. */ +final class UiApiContractAssertions +{ + static void assertExtensionMatchesCommitted(MiddlewareSpecExtension extension, Path path) throws Exception + { + String generated = UiApiContract.generateExtensionYaml(new MiddlewareVersionV1(), extension); + if(Boolean.getBoolean("qqq.uiApi.regenerate")) + { + Files.createDirectories(path.getParent()); + Files.writeString(path, generated); + } + assertThat(UiApiContract.check(generated, Files.readString(path), null, 1, "Regenerate secured-data.yaml with -Dqqq.uiApi.regenerate=true")).isEmpty(); + } +} diff --git a/src/test/java/com/kingsrook/qbits/secureddata/CoreFieldAccessBridgeIT.java b/src/test/java/com/kingsrook/qbits/secureddata/CoreFieldAccessBridgeIT.java index 6bc6cba..cfa577a 100644 --- a/src/test/java/com/kingsrook/qbits/secureddata/CoreFieldAccessBridgeIT.java +++ b/src/test/java/com/kingsrook/qbits/secureddata/CoreFieldAccessBridgeIT.java @@ -166,6 +166,38 @@ class CoreFieldAccessBridgeIT + @Test + void revealUnmasksOnlyGrantedRecordAndExpiresInCoreGet() throws Exception + { + setup(); + policy("cs_admin", "PII", "REVEALABLE"); + QRecord row = insert("person", new QRecord().withValue("email", EMAIL)); + QRecord other = insert("person", new QRecord().withValue("email", "OTHER-CANARY")); + GetInput get = new GetInput("person").withPrimaryKey(row.getValue("id")); + assertThat(new GetAction().execute(get).getRecord().getValueString("email")).isNotEqualTo(EMAIL); + var reveal = new com.kingsrook.qbits.secureddata.reveal.RevealService().reveal("person", row.getValue("id"), "email", "Investigating", null); + assertThat(reveal.value()).isEqualTo(EMAIL); + assertThat(new GetAction().execute(get).getRecord().getValueString("email")).isEqualTo(EMAIL); + assertThat(new GetAction().execute(new GetInput("person").withPrimaryKey(other.getValue("id"))).getRecord().getValueString("email")).isNotEqualTo("OTHER-CANARY"); + SecuredDataTestSupport.CLOCK.now = SecuredDataTestSupport.NOW.plusSeconds(300); + assertThat(new GetAction().execute(get).getRecord().getValueString("email")).isNotEqualTo(EMAIL); + } + + @Test + void revealPreservesCoreRecordLockDenial() throws Exception + { + QInstance instance = setup(); + policy("cs_admin", "PII", "REVEALABLE"); + QRecord row = insert("person", new QRecord().withValue("email", EMAIL).withValue("notes", "tenant-a")); + instance.addSecurityKeyType(new com.kingsrook.qqq.backend.core.model.metadata.security.QSecurityKeyType().withName("tenant")); + instance.getTable("person").withRecordSecurityLock(new com.kingsrook.qqq.backend.core.model.metadata.security.RecordSecurityLock().withSecurityKeyType("tenant").withFieldName("notes")); + QContext.getQSession().withSecurityKeyValue("tenant", "tenant-b"); + assertThat(new GetAction().execute(new GetInput("person").withPrimaryKey(row.getValue("id"))).getRecord()).isNull(); + assertThatThrownBy(() -> new com.kingsrook.qbits.secureddata.reveal.RevealService().reveal("person", row.getValue("id"), "email", "Investigating", null)) + .isInstanceOf(com.kingsrook.qqq.backend.core.exceptions.QNotFoundException.class); + assertThat(com.kingsrook.qbits.secureddata.support.SystemWrites.query(SecuredDataTables.REVEAL_GRANT, new QQueryFilter())).isEmpty(); + } + /******************************************************************************* ** Search. *******************************************************************************/ diff --git a/src/test/java/com/kingsrook/qbits/secureddata/ProcessRunner.java b/src/test/java/com/kingsrook/qbits/secureddata/ProcessRunner.java new file mode 100644 index 0000000..423e4d1 --- /dev/null +++ b/src/test/java/com/kingsrook/qbits/secureddata/ProcessRunner.java @@ -0,0 +1,109 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.secureddata; + + +import java.io.Serializable; +import java.util.Arrays; +import java.util.List; +import java.util.stream.Collectors; +import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction; +import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine; +import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface; +import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput; +import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessOutput; +import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess; + + +/******************************************************************************* + ** Runs a streamed-ETL table or record action the way the UI does: the + ** review screen, then the result. + *******************************************************************************/ +public final class ProcessRunner +{ + /******************************************************************************* + ** Review and result lines, as shown. + *******************************************************************************/ + public record Lines(List review, List result) + { + } + + + + /******************************************************************************* + ** Utility class. + *******************************************************************************/ + private ProcessRunner() + { + } + + + + /******************************************************************************* + ** Run on some record ids. + *******************************************************************************/ + @SuppressWarnings("unchecked") + public static Lines run(String process, Serializable... ids) throws Exception + { + RunProcessInput input = new RunProcessInput(); + input.setProcessName(process); + input.addValue("recordsParam", "recordIds"); + input.addValue("recordIds", Arrays.stream(ids).map(String::valueOf).collect(Collectors.joining(","))); + input.setFrontendStepBehavior(RunProcessInput.FrontendStepBehavior.BREAK); + RunProcessOutput review = new RunProcessAction().execute(input); + if(review.getException().isPresent()) + { + throw (review.getException().get()); + } + List reviewLines = (List) review.getValues().get(StreamedETLWithFrontendProcess.FIELD_VALIDATION_SUMMARY); + + RunProcessInput go = new RunProcessInput(); + go.setProcessName(process); + go.setProcessUUID(review.getProcessUUID()); + go.setStartAfterStep(StreamedETLWithFrontendProcess.STEP_NAME_REVIEW); + go.setFrontendStepBehavior(RunProcessInput.FrontendStepBehavior.SKIP); + RunProcessOutput result = new RunProcessAction().execute(go); + if(result.getException().isPresent()) + { + throw (result.getException().get()); + } + List resultLines = (List) result.getValues().get(StreamedETLWithFrontendProcess.FIELD_PROCESS_SUMMARY); + return (new Lines(text(reviewLines, false), text(resultLines, true))); + } + + + + /******************************************************************************* + ** Lines as text. + *******************************************************************************/ + private static List text(List lines, boolean past) + { + return (lines == null ? List.of() : lines.stream().map(line -> + { + if(line instanceof ProcessSummaryLine summary) + { + summary.pickMessage(past); + return ((summary.getCount() == null ? "" : summary.getCount() + " ") + summary.getMessage()); + } + return (String.valueOf(line)); + }).toList()); + } +} diff --git a/src/test/java/com/kingsrook/qbits/secureddata/RevealAndBreakGlassTest.java b/src/test/java/com/kingsrook/qbits/secureddata/RevealAndBreakGlassTest.java new file mode 100644 index 0000000..9ab3520 --- /dev/null +++ b/src/test/java/com/kingsrook/qbits/secureddata/RevealAndBreakGlassTest.java @@ -0,0 +1,319 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.secureddata; + +import java.io.Serializable; +import java.time.Instant; +import java.util.ArrayList; +import java.util.List; +import java.util.Map; +import com.kingsrook.qbits.secureddata.breakglass.BreakGlassService; +import com.kingsrook.qbits.secureddata.metadata.FieldsOfTablePossibleValueSource; +import com.kingsrook.qbits.secureddata.metadata.SecuredDataTables; +import com.kingsrook.qbits.secureddata.processes.BreakGlassProcesses; +import com.kingsrook.qbits.secureddata.reveal.RevealCappedException; +import com.kingsrook.qbits.secureddata.reveal.RevealService; +import com.kingsrook.qbits.secureddata.standin.DataClassification; +import com.kingsrook.qbits.secureddata.standin.FieldAccessTier; +import com.kingsrook.qbits.secureddata.support.SystemWrites; +import com.kingsrook.qqq.backend.core.actions.tables.GetAction; +import com.kingsrook.qqq.backend.core.alerts.AlertResolution; +import com.kingsrook.qqq.backend.core.alerts.AlertSeverity; +import com.kingsrook.qqq.backend.core.alerts.AlertSignal; +import com.kingsrook.qqq.backend.core.alerts.AlertSinkInterface; +import com.kingsrook.qqq.backend.core.alerts.AlertsInstanceMetaData; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.exceptions.QBadRequestException; +import com.kingsrook.qqq.backend.core.exceptions.QNotFoundException; +import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException; +import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput; +import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput; +import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetInput; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter; +import com.kingsrook.qqq.backend.core.model.actions.values.SearchPossibleValueSourceInput; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference; +import com.kingsrook.qqq.backend.core.model.session.QSession; +import org.junit.jupiter.api.AfterEach; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import static com.kingsrook.qbits.secureddata.SecuredDataTestSupport.*; +import static org.assertj.core.api.Assertions.*; + +class RevealAndBreakGlassTest +{ + RevealService reveals = new RevealService(); + BreakGlassService breaks = new BreakGlassService(); + Serializable pk; + QRecord policy; + + @BeforeEach + void init() throws Exception + { + var instance = setup(); + policy = insert(SecuredDataTables.POLICY, policyRow("cs_admin", "PII", "REVEALABLE") + .withValue("reason_options", "Support\nInvestigation").withValue("daily_reveal_cap", 2).withValue("allow_free_text_reason", false)); + pk = insert("person", new QRecord().withValue("email", "secret@example.invalid").withValue("ssn", "123456789")).getValue("id"); + Sink.signals.clear(); Sink.resolutions.clear(); + AlertsInstanceMetaData.ofOrWithNew(instance).withSink(new QCodeReference(Sink.class)); + } + + @AfterEach + void cleanup() { close(); } + + @Test + void revealValueAuditLinkCacheAndExactExpiry() throws Exception + { + var resolver = SecuredDataRuntime.current().resolver(); + assertThat(resolver.isRevealActive("person", "email", pk)).isFalse(); + var result = reveals.reveal("person", pk, "email", "Support", null); + assertThat(result.value()).isEqualTo("secret@example.invalid"); + assertThat(result.displayValue()).isEqualTo("secret@example.invalid"); + assertThat(result.expiresAt()).isEqualTo(NOW.plusSeconds(300)); + assertThat(resolver.isRevealActive("person", "email", pk)).isTrue(); + assertThat(resolver.isRevealActive("person", "ssn", pk)).isFalse(); + assertThat(resolver.isRevealActive("person", "email", 999)).isFalse(); + QRecord grant = SystemWrites.query(SecuredDataTables.REVEAL_GRANT, new QQueryFilter()).getFirst(); + QRecord log = logs().stream().filter(r -> "REVEAL".equals(r.getValueString("event_type"))).findFirst().orElseThrow(); + assertThat(grant.getValue("access_log_id")).isEqualTo(log.getValue("id")); + assertThat(log.getValueString("reason")).isEqualTo("Support"); + QContext.setQSession(user("u2", "cs_admin")); + assertThat(resolver.isRevealActive("person", "email", pk)).isFalse(); + QContext.setQSession(user("u1", "cs_admin")); + assertThat(resolver.isRevealActive("person", "email", pk)).isTrue(); + CLOCK.now = NOW.plusSeconds(300); + assertThat(resolver.isRevealActive("person", "email", pk)).isFalse(); + } + + @Test + void optionsSelectOneMostPermissiveGrantingPolicy() throws Exception + { + insert(SecuredDataTables.POLICY, policyRow("cs_admin", "PII", "REVEALABLE").withValue("daily_reveal_cap", 10) + .withValue("reveal_minutes", 15).withValue("reason_options", "Other policy").withValue("require_free_text", true)); + var options = reveals.options("person", "email"); + assertThat(options.remainingDailyAllowance()).isEqualTo(10); + assertThat(options.revealMinutes()).isEqualTo(15); + assertThat(options.reasonOptions()).containsExactly("Other policy"); + assertThat(options.requireFreeText()).isTrue(); + insert(SecuredDataTables.POLICY, policyRow("cs_admin", "PII", "REVEALABLE").withValue("client_channel", "web").withValue("reveal_minutes", 60)); + QContext.setQSession(channelUser("web", "cs_admin")); + assertThat(reveals.options("person", "email").revealMinutes()).isEqualTo(15); + } + + @Test + void wrongTierAndReadPermissionAreRefused() throws Exception + { + assertThatThrownBy(() -> reveals.reveal("person", pk, "ssn", "Support", null)).isInstanceOf(QPermissionDeniedException.class); + QContext.setQSession(new QSession().withUser(user("u1").getUser()).withPermissions(List.of("cs_admin"))); + assertThatThrownBy(() -> reveals.reveal("person", pk, "email", "Support", null)).isInstanceOf(QPermissionDeniedException.class); + assertThat(SystemWrites.query(SecuredDataTables.REVEAL_GRANT, new QQueryFilter())).isEmpty(); + } + + @Test + void missingRecordAndFieldAreNotFound() throws Exception + { + assertThatThrownBy(() -> reveals.reveal("person", 999, "email", "Support", null)).isInstanceOf(QNotFoundException.class); + assertThatThrownBy(() -> reveals.options("person", "bogus")).isInstanceOf(QNotFoundException.class); + assertThatThrownBy(() -> reveals.reveal("person", null, "email", "Support", null)).isInstanceOf(QBadRequestException.class); + } + + @Test + void invalidReasonsAndLengthAreRejected() throws Exception + { + assertThatThrownBy(() -> reveals.reveal("person", pk, "email", "Bogus", null)).isInstanceOf(QBadRequestException.class); + assertThatThrownBy(() -> reveals.reveal("person", pk, "email", "Support", "x".repeat(501))).isInstanceOf(QBadRequestException.class); + assertThatThrownBy(() -> reveals.reveal("person", pk, "email", "x".repeat(501), null)).isInstanceOf(QBadRequestException.class); + assertThatThrownBy(() -> reveals.reveal("person", pk, "email", null, null)).isInstanceOf(QBadRequestException.class); + } + + @Test + void freeTextRequiredAndFiveHundredCharactersAccepted() throws Exception + { + update(SecuredDataTables.POLICY, new QRecord().withValue("id", policy.getValue("id")).withValue("allow_free_text_reason", true).withValue("require_free_text", true)); + assertThatThrownBy(() -> reveals.reveal("person", pk, "email", "Support", null)).isInstanceOf(QBadRequestException.class); + assertThat(reveals.reveal("person", pk, "email", null, "x".repeat(500)).value()).isEqualTo("secret@example.invalid"); + } + + @Test + void dailyCapAcrossTablesAlertsAndResetsAtUtcMidnight() throws Exception + { + reveals.reveal("person", pk, "email", "Support", null); + QContext.getQInstance().addTable(person("memory").withName("customer")); + QContext.setQSession(user("u1", "cs_admin", "customer.read")); + Serializable other = insert("customer", new QRecord().withValue("email", "second@example.invalid")).getValue("id"); + reveals.reveal("customer", other, "email", "Support", null); + assertThat(reveals.options("person", "email").remainingDailyAllowance()).isZero(); + assertThatThrownBy(() -> reveals.reveal("person", pk, "email", "Support", null)).isInstanceOf(RevealCappedException.class); + assertThat(logs()).anyMatch(r -> "CAPPED".equals(r.getValueString("outcome")) && "REVEAL_DENIED".equals(r.getValueString("event_type"))); + assertThat(Sink.signals).anyMatch(s -> "secured_data.reveal_capped".equals(s.getAlertTypeName())); + CLOCK.now = NOW.plusSeconds(86400); + assertThat(reveals.options("person", "email").remainingDailyAllowance()).isEqualTo(2); + } + + @Test + void assistantRefusedAndLogged() throws Exception + { + QContext.setQSession(channelUser("assistant/thread", "cs_admin")); + assertThatThrownBy(() -> reveals.reveal("person", pk, "email", "Support", null)).isInstanceOf(QPermissionDeniedException.class); + assertThat(logs()).anyMatch(r -> "REVEAL_DENIED".equals(r.getValueString("event_type")) && "assistant".equals(r.getValueString("client_channel"))); + } + + void enableStart() { QContext.setQSession(user("u1", "cs_admin", BreakGlassService.START + ".hasAccess")); } + + @Test + void breakGlassStartsImmediatelyAlertsAndReportsStatus() throws Exception + { + enableStart(); + String fingerprint = SecuredDataRuntime.current().resolver().getAccessFingerprint(); + QRecord row = breaks.start("Investigating urgent incident", null, null, null); + assertThat(tier("email")).isEqualTo(FieldAccessTier.FULL); + assertThat(tier("ssn")).isEqualTo(FieldAccessTier.FULL); + assertThat(SecuredDataRuntime.current().resolver().getAccessFingerprint()).isNotEqualTo(fingerprint); + assertThat(Sink.signals).filteredOn(s -> "secured_data.break_glass".equals(s.getAlertTypeName())).singleElement().satisfies(s -> { assertThat(s.getSeverity()).isEqualTo(AlertSeverity.ERROR); assertThat(s.getAlertTypeName()).isEqualTo("secured_data.break_glass"); }); + var status = new StatusService().status(); + assertThat(status.breakGlass().id()).isEqualTo(row.getValue("id")); + assertThat(status.breakGlass().expiresAt()).isEqualTo(NOW.plusSeconds(3600)); + assertThat(status.breakGlass().ceiling()).isEqualTo("PHI"); + assertThatThrownBy(() -> breaks.start("Another urgent investigation", 60, null, null)).isInstanceOf(QBadRequestException.class); + } + + FieldAccessTier tier(String field) { var table = QContext.getQInstance().getTable("person"); return SecuredDataRuntime.current().resolver().resolve(table, table.getField(field)).tier(); } + + @Test + void ceilingScopeAndReadTimeExpiry() throws Exception + { + enableStart(); + breaks.start("Investigating urgent incident", 1, "person", DataClassification.PII); + assertThat(tier("email")).isEqualTo(FieldAccessTier.FULL); + assertThat(tier("ssn")).isEqualTo(FieldAccessTier.MASKED); + CLOCK.now = NOW.plusSeconds(60); + assertThat(tier("email")).isEqualTo(FieldAccessTier.REVEALABLE); + assertThat(new StatusService().status().breakGlass()).isNull(); + breaks.start("Investigating another incident", 1, SecuredDataTables.POLICY, null); + assertThat(tier("email")).isEqualTo(FieldAccessTier.REVEALABLE); + } + + @Test + void ownerAndAdminEndButOthersCannot() throws Exception + { + enableStart(); QRecord row = breaks.start("Investigating urgent incident", 60, null, null); + Serializable firstId = row.getValue("id"); + QContext.setQSession(user("u2")); + assertThatThrownBy(() -> breaks.end(firstId)).isInstanceOf(QPermissionDeniedException.class); + QContext.setQSession(user("u2", "secured_data.admin")); + breaks.end(row.getValue("id")); + assertThat(breaks.find(row.getValue("id")).getValueString("ended_by")).isEqualTo("u2"); + assertThat(Sink.resolutions).hasSize(1); + enableStart(); row = breaks.start("Investigating urgent incident", 60, null, null); + breaks.end(row.getValue("id")); + assertThat(tier("email")).isEqualTo(FieldAccessTier.REVEALABLE); + assertThat(logs()).anyMatch(r -> "BREAK_GLASS_END".equals(r.getValueString("event_type"))); + } + + @Test + void expiryJobIsIdempotentAndResolvesAlert() throws Exception + { + enableStart(); QRecord row = breaks.start("Investigating urgent incident", 1, null, null); + CLOCK.now = NOW.plusSeconds(60); + var output = new RunBackendStepOutput(); + new BreakGlassProcesses.Expire().run(new RunBackendStepInput(), output); + assertThat(breaks.find(row.getValue("id")).getValueString("status")).isEqualTo("EXPIRED"); + assertThat(breaks.expire()).isZero(); + assertThat(Sink.resolutions).hasSize(1); + assertThat(logs()).anyMatch(r -> "BREAK_GLASS_EXPIRED".equals(r.getValueString("event_type"))); + } + + @Test + void breakGlassValidationAndPermission() throws Exception + { + assertThatThrownBy(() -> breaks.start("Investigating urgent incident", 60, null, null)).isInstanceOf(QPermissionDeniedException.class); + enableStart(); + assertThatThrownBy(() -> breaks.start("short", 60, null, null)).isInstanceOf(QBadRequestException.class); + assertThatThrownBy(() -> breaks.start("Investigating urgent incident", 481, null, null)).isInstanceOf(QBadRequestException.class); + assertThatThrownBy(() -> breaks.start("Investigating urgent incident", 0, null, null)).isInstanceOf(QBadRequestException.class); + assertThatThrownBy(() -> breaks.start("Investigating urgent incident", 60, "missing", null)).isInstanceOf(QBadRequestException.class); + QContext.setQSession(channelUser("assistant", BreakGlassService.START + ".hasAccess")); + assertThatThrownBy(() -> breaks.start("Investigating urgent incident", 60, null, null)).isInstanceOf(QPermissionDeniedException.class); + } + + @Test + void statusAllowanceAndPicker() throws Exception + { + var status = new StatusService().status(); + assertThat(status.breakGlass()).isNull(); + assertThat(status.revealAllowances()).singleElement().satisfies(a -> { assertThat(a.tableName()).isEqualTo("person"); assertThat(a.fieldName()).isEqualTo("email"); assertThat(a.remainingDailyAllowance()).isEqualTo(2); }); + var provider = new FieldsOfTablePossibleValueSource(); + assertThat(provider.search(new SearchPossibleValueSourceInput())).isEmpty(); + assertThat(provider.search(new SearchPossibleValueSourceInput().withOtherValues(Map.of("table_name", "")))).isEmpty(); + var fields = provider.search(new SearchPossibleValueSourceInput().withOtherValues(Map.of("table_name", "person"))); + assertThat(fields).extracting(v -> v.getId()).containsExactlyInAnyOrder("id", "email", "ssn", "notes"); + assertThat(fields).allMatch(v -> v.getLabel() != null); + assertThat(provider.search(new SearchPossibleValueSourceInput().withOtherValues(Map.of("tableName", "person")))).hasSize(4); + assertThatThrownBy(() -> provider.search(new SearchPossibleValueSourceInput().withOtherValues(Map.of("tableName", "unknown")))).hasMessageContaining("Unknown table"); + } + + @Test + void endReviewAndResultRecheckOwnership() throws Exception + { + enableStart(); QRecord row = breaks.start("Investigating urgent incident", 60, null, null); + var input = new RunBackendStepInput(); input.setRecords(List.of(row)); + var output = new RunBackendStepOutput(); + var review = new BreakGlassProcesses.EndReview(); review.runOnePage(input, output); + assertThat(review.getProcessSummary(output, false)).hasSize(1); + var load = new BreakGlassProcesses.EndLoad(); load.runOnePage(input, new RunBackendStepOutput()); + assertThat(load.getProcessSummary(output, true)).hasSize(1); + assertThat(breaks.find(row.getValue("id")).getValueString("status")).isEqualTo("ENDED"); + } + + @Test + void startAndEndThroughProcessEngine() throws Exception + { + enableStart(); + var input = new RunBackendStepInput(); + input.addValue("reason", "Investigating urgent incident"); + input.addValue("duration_minutes", 60); + new BreakGlassProcesses.Start().run(input, new RunBackendStepOutput()); + QRecord row = SecuredDataRuntime.current().resolver().activeBreakGlass().getFirst(); + var lines = ProcessRunner.run(BreakGlassService.END, row.getValue("id")); + assertThat(lines.review()).anyMatch(line -> line.contains("will be ended")); + assertThat(lines.result()).anyMatch(line -> line.contains("was ended")); + assertThat(breaks.find(row.getValue("id")).getValueString("status")).isEqualTo("ENDED"); + } + + @Test + void expiryScheduleUsesHostSchedulerAndSixtySeconds() + { + var config = new SecuredDataQBitConfig().withSchedulerName("host"); + var process = BreakGlassProcesses.processes(config).getLast(); + assertThat(process.getSchedule().getRepeatSeconds()).isEqualTo(60); + assertThat(process.getSchedule().getSchedulerName()).isEqualTo("host"); + assertThat(process.getIsHidden()).isTrue(); + } + + public static class Sink implements AlertSinkInterface + { + static final List signals = new ArrayList<>(); + static final List resolutions = new ArrayList<>(); + public void raise(AlertSignal signal) { signals.add(signal); } + public void resolve(AlertResolution resolution) { resolutions.add(resolution); } + public void heartbeat(String key, Instant at) { } + } +} diff --git a/src/test/java/com/kingsrook/qbits/secureddata/SecuredDataAccessLoggerTest.java b/src/test/java/com/kingsrook/qbits/secureddata/SecuredDataAccessLoggerTest.java index 8794521..08023cb 100644 --- a/src/test/java/com/kingsrook/qbits/secureddata/SecuredDataAccessLoggerTest.java +++ b/src/test/java/com/kingsrook/qbits/secureddata/SecuredDataAccessLoggerTest.java @@ -116,7 +116,7 @@ class SecuredDataAccessLoggerTest .hasMessage("Exact-match search limit reached for [email]. Try again later or ask an administrator."); resolver.onAccess(refused); logger.flush(); - assertThat(logs()).hasSize(21); + assertThat(logs()).filteredOn(row -> "EXACT_MATCH_FILTER".equals(row.getValueString("event_type"))).hasSize(21); assertThat(logs().stream().filter(row -> "REFUSED".equals(row.getValueString("outcome")))).hasSize(1); assertThat(JsonUtils.toJson(logs())).doesNotContain("sentinel").doesNotContain("known"); CLOCK.now = NOW.plusSeconds(3601); @@ -204,7 +204,7 @@ class SecuredDataAccessLoggerTest classification("notes", "CONFIDENTIAL"); logger.onAccess(event(FieldAccessEventType.VIEW, null, null).withFieldNames(List.of("notes", "email"))); logger.flush(); - assertThat(logs()).hasSize(3); + assertThat(logs()).filteredOn(row -> !"CLASSIFICATION_CHANGE".equals(row.getValueString("event_type"))).hasSize(3); } /******************************************************************************* diff --git a/src/test/java/com/kingsrook/qbits/secureddata/SecuredDataAnomaliesTest.java b/src/test/java/com/kingsrook/qbits/secureddata/SecuredDataAnomaliesTest.java new file mode 100644 index 0000000..7075d90 --- /dev/null +++ b/src/test/java/com/kingsrook/qbits/secureddata/SecuredDataAnomaliesTest.java @@ -0,0 +1,186 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.secureddata; + +import java.time.temporal.ChronoUnit; +import java.util.List; +import com.kingsrook.qbits.secureddata.breakglass.BreakGlassService; +import com.kingsrook.qbits.secureddata.metadata.SecuredDataTables; +import com.kingsrook.qbits.secureddata.reveal.RevealService; +import com.kingsrook.qbits.secureddata.standin.FieldAccessEventType; +import com.kingsrook.qbits.secureddata.standin.FieldAccessOutcome; +import com.kingsrook.qbits.secureddata.support.ActivityLog; +import com.kingsrook.qqq.backend.core.actions.tables.InsertAction; +import com.kingsrook.qqq.backend.core.alerts.AlertsInstanceMetaData; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference; +import com.kingsrook.qbits.secureddata.support.SystemWrites; +import org.junit.jupiter.api.AfterEach; +import org.junit.jupiter.api.Test; +import static com.kingsrook.qbits.secureddata.SecuredDataTestSupport.*; +import static org.assertj.core.api.Assertions.*; + +class SecuredDataAnomaliesTest +{ + @AfterEach void cleanup() { close(); } + + void init(java.util.function.Consumer config) throws Exception + { + var instance = setup(config); + SecuredDataAccessLoggerTest.AlertsSink.signals.clear(); + AlertsInstanceMetaData.ofOrWithNew(instance).withSink(new QCodeReference(SecuredDataAccessLoggerTest.AlertsSink.class)); + } + + QRecord reveal(int record) throws Exception + { + return insert(SecuredDataTables.ACCESS_LOG, ActivityLog.row("REVEAL", "person", "ALLOWED") + .withValue("record_id", String.valueOf(record)).withValue("field_name", "email")); + } + + static void seed(QRecord row) throws Exception + { + SystemWrites.call(() -> new InsertAction().execute(new InsertInput(SecuredDataTables.ACCESS_LOG).withRecord(row))); + } + + void alertCount(int count) + { + assertThat(SecuredDataAccessLoggerTest.AlertsSink.signals).filteredOn(s -> "secured_data.anomaly".equals(s.getAlertTypeName())).hasSize(count) + .allSatisfy(s -> assertThat(s.getDedupKey()).isEqualTo("secured_data.anomaly:u1:2026-10-13")); + } + + @Test void minimumTenAndZeroDaysMedian() throws Exception + { + init(c -> {}); + for(int i = 0; i < 9; i++) assertThat(reveal(1).getValueBoolean("anomaly_flag")).isFalse(); + alertCount(0); + assertThat(reveal(1).getValueBoolean("anomaly_flag")).isTrue(); + alertCount(1); + } + + @Test void strictlyAboveThreeTimesMedianAndDailyCacheRollover() throws Exception + { + init(c -> {}); + for(int day = 1; day <= 30; day++) + for(int i = 0; i < 4; i++) seed(ActivityLog.row("REVEAL", "person", "ALLOWED").withValue("record_id", "1") + .withValue("created_at", NOW.truncatedTo(ChronoUnit.DAYS).minus(day, ChronoUnit.DAYS).plusSeconds(i))); + for(int i = 0; i < 12; i++) assertThat(reveal(1).getValueBoolean("anomaly_flag")).isFalse(); + alertCount(0); + assertThat(reveal(1).getValueBoolean("anomaly_flag")).isTrue(); + alertCount(1); + CLOCK.now = NOW.plus(1, ChronoUnit.DAYS); + assertThat(reveal(1).getValueBoolean("anomaly_flag")).isFalse(); + } + + @Test void distinctRecordBoundaryDuplicatesOtherUsersAndExpiredRows() throws Exception + { + init(c -> c.withAnomalyMinimumDailyReveals(1000)); + for(int i = 0; i < 20; i++) assertThat(reveal(i).getValueBoolean("anomaly_flag")).isFalse(); + assertThat(reveal(0).getValueBoolean("anomaly_flag")).isFalse(); + seed(ActivityLog.row("REVEAL", "person", "ALLOWED").withValue("user_id", "other").withValue("record_id", "999")); + seed(ActivityLog.row("REVEAL", "person", "ALLOWED").withValue("record_id", "998").withValue("created_at", NOW.minusSeconds(3600))); + alertCount(0); + assertThat(reveal(20).getValueBoolean("anomaly_flag")).isTrue(); + alertCount(1); + CLOCK.now = NOW.plusSeconds(3601); + assertThat(reveal(21).getValueBoolean("anomaly_flag")).isFalse(); + } + + @Test void exactMatchFiftyFirstQueuedSearchFlagsWithoutEnumerationAlert() throws Exception + { + init(c -> c.withExactMatchPerHour(0).withMissAlertConsecutive(0).withDistinctValueAlert(0)); + var logger = SecuredDataRuntime.current().logger(); + for(int i = 0; i < 50; i++) logger.onAccess(SecuredDataAccessLoggerTest.event(FieldAccessEventType.EXACT_MATCH_FILTER, FieldAccessOutcome.MATCH, "hash")); + logger.flush(); + assertThat(logs()).allSatisfy(row -> assertThat(row.getValueBoolean("anomaly_flag")).isFalse()); + alertCount(0); + logger.onAccess(SecuredDataAccessLoggerTest.event(FieldAccessEventType.EXACT_MATCH_FILTER, FieldAccessOutcome.MATCH, "hash")); + logger.flush(); + assertThat(logs().getLast().getValueBoolean("anomaly_flag")).isTrue(); + alertCount(1); + } + + @Test void configurableThresholdsAndMissingAlertSink() throws Exception + { + setup(c -> c.withAnomalyDailyMedianMultiplier(2).withAnomalyMinimumDailyReveals(3).withAnomalyDistinctRecordsPerHour(2).withAnomalyExactMatchesPerHour(2)); + assertThat(reveal(1).getValueBoolean("anomaly_flag")).isFalse(); + assertThat(reveal(2).getValueBoolean("anomaly_flag")).isFalse(); + assertThat(reveal(3).getValueBoolean("anomaly_flag")).isTrue(); + QContext.setQSession(user("u2")); + for(int i = 0; i < 3; i++) + { + var row = insert(SecuredDataTables.ACCESS_LOG, ActivityLog.row("EXACT_MATCH_FILTER", "person", "MATCH").withValue("field_name", "email")); + assertThat(row.getValueBoolean("anomaly_flag")).isEqualTo(i == 2); + } + } + + @Test void actualRevealServiceBreakGlassAndCapFlagAndAlert() throws Exception + { + init(c -> c.withAnomalyMinimumDailyReveals(1)); + insert(SecuredDataTables.POLICY, policyRow("cs_admin", "PII", "REVEALABLE").withValue("daily_reveal_cap", 1)); + var pk = insert("person", new QRecord().withValue("email", "private@example.invalid")).getValue("id"); + new RevealService().reveal("person", pk, "email", "Support", null); + assertThat(logs()).filteredOn(r -> "REVEAL".equals(r.getValueString("event_type"))).singleElement() + .satisfies(r -> assertThat(r.getValueBoolean("anomaly_flag")).isTrue()); + alertCount(1); + assertThatThrownBy(() -> new RevealService().reveal("person", pk, "email", "Support", null)).isInstanceOf(com.kingsrook.qbits.secureddata.reveal.RevealCappedException.class); + alertCount(2); + QContext.setQSession(user("u1", BreakGlassService.START + ".hasAccess")); + new BreakGlassService().start("Investigating urgent customer incident", 5, null, null); + alertCount(3); + assertThat(logs()).filteredOn(r -> "BREAK_GLASS_START".equals(r.getValueString("event_type"))).singleElement() + .satisfies(r -> assertThat(r.getValueBoolean("anomaly_flag")).isTrue()); + } + + @Test void invalidThresholdsRejected() throws Exception + { + for(var config : List.of(new SecuredDataQBitConfig().withAnomalyDailyMedianMultiplier(0), new SecuredDataQBitConfig().withAnomalyMinimumDailyReveals(0), + new SecuredDataQBitConfig().withAnomalyDistinctRecordsPerHour(0), new SecuredDataQBitConfig().withAnomalyExactMatchesPerHour(0))) + assertThatThrownBy(config::validate).isInstanceOf(IllegalArgumentException.class); + } + @Test void exactMatchCounterIsScopedToFieldTableUserAndHour() throws Exception + { + init(c -> c.withAnomalyExactMatchesPerHour(2)); + seed(ActivityLog.row("EXACT_MATCH_FILTER", "person", "MATCH").withValue("field_name", "email").withValue("created_at", NOW.minusSeconds(3600))); + seed(ActivityLog.row("EXACT_MATCH_FILTER", "person", "MATCH").withValue("field_name", "ssn")); + seed(ActivityLog.row("EXACT_MATCH_FILTER", "person", "MATCH").withValue("field_name", "email").withValue("user_id", "u2")); + for(int i = 0; i < 3; i++) + { + QRecord row = insert(SecuredDataTables.ACCESS_LOG, ActivityLog.row("EXACT_MATCH_FILTER", "person", "MATCH").withValue("field_name", "email")); + assertThat(row.getValueBoolean("anomaly_flag")).isEqualTo(i == 2); + } + alertCount(1); + } + + @Test void configurableMedianMultiplierUsesHistoricalCounts() throws Exception + { + init(c -> c.withAnomalyDailyMedianMultiplier(2).withAnomalyMinimumDailyReveals(3)); + for(int day = 1; day <= 30; day++) + for(int i = 0; i < 2; i++) seed(ActivityLog.row("REVEAL", "person", "ALLOWED").withValue("record_id", "1") + .withValue("created_at", NOW.minus(day, ChronoUnit.DAYS))); + for(int i = 0; i < 4; i++) assertThat(reveal(1).getValueBoolean("anomaly_flag")).isFalse(); + alertCount(0); + assertThat(reveal(1).getValueBoolean("anomaly_flag")).isTrue(); + alertCount(1); + } + +} diff --git a/src/test/java/com/kingsrook/qbits/secureddata/SecuredDataDdlTest.java b/src/test/java/com/kingsrook/qbits/secureddata/SecuredDataDdlTest.java index 6c77ad6..e2bd08a 100644 --- a/src/test/java/com/kingsrook/qbits/secureddata/SecuredDataDdlTest.java +++ b/src/test/java/com/kingsrook/qbits/secureddata/SecuredDataDdlTest.java @@ -25,7 +25,9 @@ import java.nio.file.Files; import java.nio.file.Path; import java.sql.Connection; import java.sql.DriverManager; +import com.kingsrook.qbits.secureddata.metadata.SecuredDataTables; import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.model.data.QRecord; import com.kingsrook.qqq.backend.core.model.metadata.QInstance; import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData; import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSTableBackendDetails; @@ -37,6 +39,7 @@ import static com.kingsrook.qbits.secureddata.SecuredDataTestSupport.RUNTIMES; import static com.kingsrook.qbits.secureddata.SecuredDataTestSupport.base; import static com.kingsrook.qbits.secureddata.SecuredDataTestSupport.classification; import static com.kingsrook.qbits.secureddata.SecuredDataTestSupport.close; +import static com.kingsrook.qbits.secureddata.SecuredDataTestSupport.insert; import static com.kingsrook.qbits.secureddata.SecuredDataTestSupport.logs; import static com.kingsrook.qbits.secureddata.SecuredDataTestSupport.person; import static com.kingsrook.qbits.secureddata.SecuredDataTestSupport.policy; @@ -97,7 +100,21 @@ class SecuredDataDdlTest SecuredDataRuntime.current().logger().onAccess(SecuredDataAccessLoggerTest.event(com.kingsrook.qbits.secureddata.standin.FieldAccessEventType.EXACT_MATCH_FILTER, com.kingsrook.qbits.secureddata.standin.FieldAccessOutcome.MATCH, "hmac")); SecuredDataRuntime.current().logger().flush(); - assertThat(logs()).hasSize(1); + assertThat(logs()).extracting(row -> row.getValueString("event_type")).containsExactly("CLASSIFICATION_CHANGE", "POLICY_CHANGE", "EXACT_MATCH_FILTER"); + // Equal timestamps must be counted by SQL as well as the memory backend. + SecuredDataRuntime.current().config().withAnomalyMinimumDailyReveals(1000).withAnomalyDistinctRecordsPerHour(2).withAnomalyExactMatchesPerHour(2); + for(int i = 0; i < 3; i++) + { + QRecord audit = insert(SecuredDataTables.ACCESS_LOG, com.kingsrook.qbits.secureddata.support.ActivityLog.row("REVEAL", "person", "ALLOWED") + .withValue("record_id", String.valueOf(i)).withValue("field_name", "email")); + assertThat(audit.getValueBoolean("anomaly_flag")).isEqualTo(i == 2); + } + for(int i = 0; i < 2; i++) + { + QRecord audit = insert(SecuredDataTables.ACCESS_LOG, com.kingsrook.qbits.secureddata.support.ActivityLog.row("EXACT_MATCH_FILTER", "person", "MATCH") + .withValue("field_name", "email")); + assertThat(audit.getValueBoolean("anomaly_flag")).isEqualTo(i == 1); + } assertThatThrownBy(() -> connection.createStatement().execute("INSERT INTO secured_data_classification (table_name, classification) VALUES ('person', 'PHI')")) .isInstanceOf(java.sql.SQLException.class); assertThatThrownBy(() -> connection.createStatement().execute("INSERT INTO secured_data_policy (name, permission_name, classification, tier, reveal_minutes) VALUES ('x','x','PII','FULL',61)")) diff --git a/src/test/java/com/kingsrook/qbits/secureddata/SecuredDataDemoServer.java b/src/test/java/com/kingsrook/qbits/secureddata/SecuredDataDemoServer.java new file mode 100644 index 0000000..61a7273 --- /dev/null +++ b/src/test/java/com/kingsrook/qbits/secureddata/SecuredDataDemoServer.java @@ -0,0 +1,117 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.secureddata; + +import java.util.Map; +import java.util.Set; +import com.kingsrook.qbits.secureddata.metadata.SecuredDataTables; +import com.kingsrook.qbits.secureddata.support.ActivityLog; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.instances.AbstractQQQApplication; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType; +import com.kingsrook.qqq.backend.core.model.metadata.QInstance; +import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference; +import com.kingsrook.qqq.backend.core.model.session.QSession; +import com.kingsrook.qqq.backend.core.model.session.QUser; +import com.kingsrook.qqq.backend.core.modules.authentication.QAuthenticationModuleCustomizerInterface; +import com.kingsrook.qqq.middleware.javalin.QApplicationJavalinServer; +import static com.kingsrook.qbits.secureddata.SecuredDataTestSupport.*; + +/******************************************************************************* + ** Test-only local Next demo, following WebhooksDemoServer. Synthetic identifiers + ** and reasons only; never shipped. Run on port 8098 after test compilation. + *******************************************************************************/ +public class SecuredDataDemoServer +{ + /******************************************************************************* + ** Local demo administrator; not an application authentication implementation. + *******************************************************************************/ + public static class Sessions implements QAuthenticationModuleCustomizerInterface + { + /******************************************************************************* + ** Give the local demo access to its home and tables. + *******************************************************************************/ + public void customizeSession(QInstance instance, QSession session, Map context) + { + session.setUser(new QUser().withIdReference("demo-admin").withFullName("Demo Administrator")); + session.setPermissions(Set.of("secured_data.admin", "secured_data.hasAccess", "person.read")); + } + } + + /******************************************************************************* + ** Seed every home section with realistic synthetic rows. + *******************************************************************************/ + public static void seedDemo() throws Exception + { + String[] names = {"Avery Morgan", "Jordan Patel", "Sam Rivera", "Taylor Chen"}; + String[] reasons = {"Customer identity verification", "Support case follow-up", "Account recovery investigation", "Review of an escalated access request with the customer support team"}; + for(int i = 0; i < 48; i++) + { + SecuredDataAnomaliesTest.seed(ActivityLog.row("REVEAL", "person", "ALLOWED") + .withValue("user_id", "operator-" + i % 4).withValue("user_name", names[i % 4]) + .withValue("record_id", String.valueOf(4200 + i)).withValue("field_name", i % 2 == 0 ? "email" : "ssn") + .withValue("reason", reasons[i % 4]).withValue("client_channel", i % 3 == 0 ? "api" : "web") + .withValue("created_at", NOW.minusSeconds(i < 15 ? i * 180L : i * 36000L)).withValue("anomaly_flag", i < 2)); + } + for(int i = 0; i < 3; i++) + { + String event = new String[] {"BREAK_GLASS_START", "BREAK_GLASS_END", "BREAK_GLASS_EXPIRED"}[i]; + SecuredDataAnomaliesTest.seed(ActivityLog.row(event, SecuredDataTables.BREAK_GLASS, "ALLOWED") + .withValue("user_name", names[i]).withValue("reason", "Urgent investigation of customer support incident SD-2026-1042") + .withValue("created_at", NOW.minusSeconds(i * 1200L)).withValue("anomaly_flag", i == 0)); + } + for(int i = 0; i < 4; i++) + { + insert(SecuredDataTables.REVEAL_GRANT, new QRecord().withValue("user_id", "operator-" + i).withValue("user_name", names[i]) + .withValue("table_name", "person").withValue("record_id", "4200").withValue("field_name", "email") + .withValue("granted_at", NOW.minusSeconds(60)).withValue("expires_at", NOW.plusSeconds(240)).withValue("access_log_id", i + 1)); + } + insert(SecuredDataTables.BREAK_GLASS, new QRecord().withValue("user_id", "operator-0").withValue("user_name", names[0]) + .withValue("started_at", NOW.minusSeconds(300)).withValue("expires_at", NOW.plusSeconds(3300)).withValue("status", "ACTIVE")); + for(int i = 0; i < 6; i++) + SecuredDataAnomaliesTest.seed(ActivityLog.row("FILTER_DENIED", "person", "DENIED").withValue("user_name", names[i % 4])); + policy("support_specialist", "PII", "REVEALABLE"); + classification("email", "PII"); + } + + /******************************************************************************* + ** Run the real bundled Next frontend and middleware against memory data. + *******************************************************************************/ + public static void main(String[] args) throws Exception + { + QInstance instance = setup(); + instance.withInstanceDefaultAuthentication(new QAuthenticationMetaData().withName("mock").withType(QAuthenticationType.MOCK)); + instance.getAuthentication().setCustomizer(new QCodeReference(Sessions.class)); + instance.setHomeAppName("secured_data"); + QContext.setQSession(user("demo-admin", "secured_data.admin")); + seedDemo(); + new QApplicationJavalinServer(new AbstractQQQApplication() + { + public QInstance defineQInstance() + { + return (instance); + } + }).withPort(args.length == 0 ? 8098 : Integer.parseInt(args[0])).withServeFrontendNext(true).start(); + System.out.println("Secured Data demo: http://localhost:8098/app/secured_data"); + } +} diff --git a/src/test/java/com/kingsrook/qbits/secureddata/SecuredDataHomeTest.java b/src/test/java/com/kingsrook/qbits/secureddata/SecuredDataHomeTest.java new file mode 100644 index 0000000..553225e --- /dev/null +++ b/src/test/java/com/kingsrook/qbits/secureddata/SecuredDataHomeTest.java @@ -0,0 +1,143 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.secureddata; + +import java.net.URLDecoder; +import java.nio.charset.StandardCharsets; +import java.util.List; +import com.kingsrook.qbits.secureddata.app.SecuredDataApp; +import com.kingsrook.qbits.secureddata.app.SecuredDataHomeData; +import com.kingsrook.qbits.secureddata.metadata.SecuredDataTables; +import com.kingsrook.qbits.secureddata.support.ActivityLog; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput; +import com.kingsrook.qqq.backend.core.model.dashboard.widgets.RawHTML; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import org.junit.jupiter.api.AfterEach; +import org.junit.jupiter.api.Test; +import static com.kingsrook.qbits.secureddata.SecuredDataTestSupport.*; +import static org.assertj.core.api.Assertions.*; + +class SecuredDataHomeTest +{ + @AfterEach void cleanup() { close(); } + + String home() throws Exception + { + return ((RawHTML) new SecuredDataApp.HomeRenderer().render(new RenderWidgetInput()).getWidgetData()).getHtml(); + } + + @Test void emptyStateAndNav() throws Exception + { + var instance = setup(); + QContext.setQSession(user("admin", "secured_data.admin")); + var data = SecuredDataHomeData.load(); + assertThat(data.getKpis().values()).containsOnly(0L); + assertThat(data.getSections()).hasSize(6).allSatisfy(s -> assertThat(s.rows()).isEmpty()); + assertThat(home()).contains("No reveals yet", "No reveals in the last 30 days", "No reveal reasons in the last 30 days", + "No break-glass events yet", "No policy or classification changes yet", "No anomaly flags in the last 7 days"); + var app = instance.getApp("secured_data"); + assertThat(app.getWidgets()).containsExactly(SecuredDataApp.HOME_WIDGET); + assertThat(app.getChildren()).extracting(c -> c.getName()).containsExactly(SecuredDataTables.CLASSIFICATION, SecuredDataTables.POLICY, + SecuredDataTables.ACCESS_LOG, SecuredDataTables.BREAK_GLASS); + assertThat(instance.getTable(SecuredDataTables.ACCESS_LOG).getSections()).anyMatch(s -> SecuredDataTables.REVEALS_WIDGET.equals(s.getWidgetName())); + } + + @Test void seededKpisOrderingLimitsLinksAndEveryWidget() throws Exception + { + setup(c -> c.withTablePathPrefix("/admin/data").withAccessLogQueueCapacity(1)); + QContext.setQSession(user("admin", "secured_data.admin")); + for(int i = 0; i < 15; i++) + SecuredDataAnomaliesTest.seed(ActivityLog.row("REVEAL", "person", "ALLOWED").withValue("user_id", i < 12 ? "alice" : "bob") + .withValue("user_name", i < 12 ? "Alice " : "Bob").withValue("reason", i < 11 ? "Customer support" : "Case review") + .withValue("record_id", String.valueOf(i)).withValue("field_name", "email").withValue("client_channel", "web") + .withValue("created_at", NOW.minusSeconds(i)).withValue("anomaly_flag", i == 0)); + SecuredDataAnomaliesTest.seed(ActivityLog.row("REVEAL", "person", "ALLOWED").withValue("created_at", NOW.minusSeconds(31 * 86400L))); + SecuredDataAnomaliesTest.seed(ActivityLog.row("REVEAL", "person", "ALLOWED").withValue("created_at", NOW.plusSeconds(60))); + for(String type : List.of("BREAK_GLASS_START", "BREAK_GLASS_END", "BREAK_GLASS_EXPIRED", "FILTER_DENIED")) + SecuredDataAnomaliesTest.seed(ActivityLog.row(type, "person", "ALLOWED")); + SecuredDataAnomaliesTest.seed(ActivityLog.row("EXACT_MATCH_FILTER", "person", "REFUSED")); + SecuredDataAnomaliesTest.seed(ActivityLog.row("FILTER_DENIED", "person", "DENIED").withValue("created_at", NOW.minusSeconds(8 * 86400L))); + for(int i = 0; i < 2; i++) + insert(SecuredDataTables.REVEAL_GRANT, new QRecord().withValue("user_id", "alice").withValue("table_name", "person") + .withValue("record_id", "1").withValue("field_name", "email").withValue("granted_at", NOW.minusSeconds(60)) + .withValue("expires_at", NOW.plusSeconds(i == 0 ? 60 : 0)).withValue("access_log_id", 1)); + for(int i = 0; i < 3; i++) + insert(SecuredDataTables.BREAK_GLASS, new QRecord().withValue("user_id", "alice").withValue("started_at", NOW.minusSeconds(60)) + .withValue("expires_at", NOW.plusSeconds(i == 0 ? 60 : 0)).withValue("status", i == 2 ? "ENDED" : "ACTIVE")); + var policy = policy("role", "PII", "MASKED"); + update(SecuredDataTables.POLICY, new QRecord().withValue("id", policy.getValue("id")).withValue("name", "Changed")); + delete(SecuredDataTables.POLICY, policy); + classification("email", "PII"); + var logger = SecuredDataRuntime.current().logger(); + logger.onAccess(SecuredDataAccessLoggerTest.event(com.kingsrook.qbits.secureddata.standin.FieldAccessEventType.VIEW, null, null)); + logger.onAccess(SecuredDataAccessLoggerTest.event(com.kingsrook.qbits.secureddata.standin.FieldAccessEventType.VIEW, null, null)); + var data = SecuredDataHomeData.load(); + assertThat(data.getKpis()).containsEntry("Reveals today", 15L).containsEntry("Active reveal grants", 1L) + .containsEntry("Active break-glass sessions", 1L).containsEntry("Anomaly flags (7 days)", 1L) + .containsEntry("Denied filters (7 days)", 2L).containsEntry("Access-log queue drops", 1L); + assertThat(data.getSections().get(0).rows()).hasSize(10).extracting(r -> r.getValueString("record_id")) + .containsExactly("0", "1", "2", "3", "4", "5", "6", "7", "8", "9"); + assertThat(data.getSections().get(1).rows()).extracting(r -> r.getValueString("user_id")).containsExactly("alice", "bob"); + assertThat(data.getSections().get(1).rows().getFirst().getValueInteger("count")).isEqualTo(12); + assertThat(data.getSections().get(2).rows().getFirst().getValueString("reason")).isEqualTo("Customer support"); + assertThat(data.getSections().get(3).rows()).hasSize(3); + assertThat(data.getSections().get(4).rows()).hasSize(4).extracting(r -> r.getValueString("action_name")).containsExactly("INSERT", "DELETE", "UPDATE", "INSERT"); + assertThat(data.getSections().get(5).rows()).hasSize(1); + for(var section : data.getSections()) + { + assertThat(section.href()).startsWith("/admin/data/secured_data_access_log?filter="); + String filter = URLDecoder.decode(section.href().split("filter=")[1], StandardCharsets.UTF_8); + assertThat(filter).contains("created_at").doesNotContain("\"limit\":10"); + } + String html = home(); + assertThat(html).contains("Recent reveals", "Top revealers (30 days)", "Top reasons (30 days)", "Break-glass events", "Policy and classification changes", "Anomaly flags") + .contains("Alice <b>", "Customer support", "BREAK_GLASS_EXPIRED", "CLASSIFICATION_CHANGE", "View all", "UTC", "title=", "text-overflow:ellipsis", "white-space:nowrap", "minmax(min(100%", "var(--qqq-grid-border") + .doesNotContain("Alice ", "overflow:auto", "overflow-x", " { + assertThat(r.getValueString("user_id")).isEqualTo("admin"); + assertThat(r.getValueString("record_id")).isEqualTo(row.getValueString("id")); + assertThat(r.getValueString("event_type")).isEqualTo("CLASSIFICATION_CHANGE"); + assertThat(r.getValue("reason")).isNull(); + }); + assertThat(logs()).extracting(r -> r.getValueString("action_name")).containsExactly("INSERT", "UPDATE", "DELETE"); + } +}