Compare commits

..
24 Commits
Author SHA1 Message Date
CI 152fc98ae7 Bump version to 0.1.0-RC.9 [skip ci] 2026-11-20 16:47:44 +00:00
james.maes 960ce55f8b Merge pull request 'feat: a create date and a modify date on every table (release/0.1 port of #47)' (#48) from feat/table-standard-dates-0.1 into release/0.1
maven / build (push) Successful in 2m46s
build / maven (push) Successful in 2m46s
2026-11-20 16:45:01 +00:00
james.maesandClaude Fable 5.2 0a8c0fba6f fix: an edit over HTTP is an edit of its row, not a new row
maven / build (push) Successful in 1m55s
build / maven (push) Successful in 1m55s
The table customizer finds the stored row of an update by its id, and
compared the two ids as they came: the id of a request over HTTP is text
(it is the id in the path), the stored id is a number. They were never
equal, so an edit made through the API or the UI had no stored row and was
validated and stamped as an insert. The uniqueness check of a
classification compared ids the same way. Both now compare the ids as text.

This is older than the standard dates, but they build on it: the customizer
gives a new row its create date, and a client that knows record versions
edits a table with a modify date by sending only the changed values and the
version it read.

What an edit over HTTP did before:
- A classification, policy or retention rule got a new create_date.
- Releasing a legal hold overwrote when and by whom it was placed (and,
  since the hold has one, its create date).
- Editing the reason of a legal hold without sending its status failed with
  a server error (the validation read the status of a row it did not have).
- Editing a classification while naming its table and field was refused as
  a second classification of that field.
- Editing a retention rule without sending every field was refused.

Tests (SecuredDataHttpEditTest, through the real middleware on the shipped
schema, a day after the rows were written): the reason of a hold is edited
with the changed value and the version, and the stale version is then
refused; a hold is released and keeps when and by whom it was placed and
its create date; a classification is edited naming its table and field; a
policy and a retention rule are renamed and keep their create dates; a row
with neither date is read and edited. All five fail without the fix. They
run in UTC: where the default zone of the JVM is not UTC, core refuses
every update that sends a version, which is core's to fix and is kept out
of these tests.

Notes for an upgrade (shipped DDL block and README): set a lock_timeout
before the ALTER statements and run the block again when one is cancelled;
on a large access log run its backfill UPDATE in ranges of id, or leave it
out, since rows without the two dates are read and written like any
others. Changelog: the fix, that an update naming the create date still
sets it, and that a wipe moves the record version of its retention rule.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-20 10:40:49 -06:00
james.maesandClaude Fable 5.2 4e20070ae1 feat: a create date and a modify date on every table
maven / build (push) Successful in 1m56s
build / maven (push) Successful in 1m55s
Every table of the QBit has a create date and a modify date: DATE_TIME, not
editable, with core's CREATE_DATE and MODIFY_DATE behaviors, in the columns
create_date and modify_date.

Seven tables did not have them: reveal grant, break-glass session, access
log, wipe run, wipe marker, legal hold and audit key. Their fields are named
createDate and modifyDate. Classification, policy and retention rule already
had create_date and modify_date, set by the table customizer. They keep those
field names, because a rename would change what a host reads and what their
audits record, and the two fields are now not editable and have the
behaviors.

Writes. Every write of the QBit is a core insert or update action, so the
dates are in the statement that writes the row. The table customizer sets
them, as it did for the three tables: a new row gets both, equal, from the
QBit's clock, and a changed row a new modify date. The access-log writer
still runs one insert per row and no other statement, lock or transaction.
The access log and wipe runs are only ever inserted, so their dates stay
equal.

Core uses a DATE_TIME field with the MODIFY_DATE behavior as the record
version of its table. That now holds for all ten tables: on an update core
sets the modify date itself, after the customizer has run, and an update
that sends the version it read is refused when the row has changed since.
Updates that send no version, the QBit's own among them, behave as before.

Schema. docs/ddl/secured-data-postgres.sql has a new last block that adds
the two columns to the seven tables: nullable, defaulting to the current
time, with the rows that were already there given the row's own time. No
CREATE TABLE statement is edited. An existing database runs that block
alone, before this release is deployed. The block rewrites no table, can be
run while the release before this one is running (its inserts name neither
column and get both from the default) and can be run again. This release
does not run without the columns. README: "Upgrading an existing database".

Unchanged. The dates have no classification, policy or assistant exposure of
their own: they are CONFIDENTIAL with their table and get the tier of its
other fields. Nothing signed or verified covers them: a wrapped data key is
bound to its table, field and record id, and the fingerprints of a log row
are made from search values. A changed policy or classification writes the
same core audit, field by field, and the same POLICY_CHANGE or
CLASSIFICATION_CHANGE row.

Tests. SecuredDataStandardDatesTest: every table the producer declares has
the two dates (the tables are taken from the instance, so a table added
later without them fails); the dates resolve to the tier of the other fields
of their table for five kinds of session; every writer sets both and only a
change moves the modify date (access-log writer, policy, classification,
legal hold, reveal, break-glass start, end and expiry on the memory backend;
audit key, wipe run, wipe marker and retention rule in the shipped schema);
the access-log writer runs exactly one insert per row; a changed policy and
classification write the audit and log rows they wrote before; rows written
before the columns existed are unchanged by the upgrade block, their wrapped
keys unwrap to the same data keys and their audit token verifies the same
value and no other. SecuredDataDdlTest: the shipped DDL gives every declared
table the two columns, the upgrade block adds them where they were missing,
an insert that names neither succeeds, and the block can be run twice. Seven
of the eight new tests fail without the change; the eighth is the record of
what a changed policy wrote before and passes on both sides.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-20 09:01:43 -06:00
CI 84ec915772 Bump version to 0.1.0-RC.8 [skip ci] 2026-11-14 14:27:35 +00:00
james.maes d72dd6addd Merge pull request 'fix: a reveal and the access-log writer no longer wait for each other (release/0.1 port of #45)' (#46) from fix/reveal-access-log-deadlock-0.1 into release/0.1
maven / build (push) Successful in 1m43s
build / maven (push) Successful in 1m43s
2026-11-14 14:25:59 +00:00
james.maesandClaude Fable 5.2 c2fde973a5 fix: a reveal and the access-log writer no longer wait for each other
maven / build (push) Successful in 2m0s
build / maven (push) Successful in 2m0s
A reveal, and the start, end and expiry of a break-glass session, hold the
runtime's monitor from their first check to their last write. Asking for a
classification (SecuredDataRuntime.classificationRows, and
activeRetentionRules) took the same monitor, and every thread asks: core
asks for every field it decides on and for every filter criterion of every
query, including the queries made inside this QBit's own writes.

So a thread that asked while holding another lock of this QBit could not be
answered during a reveal, and if the reveal needed that lock next, both
waited for ever. The access-log writer is such a thread. It holds the
anomaly counters while it inserts a row; the insert counts earlier rows and
writes an audit, both queries with criteria. A reveal inserts its own
access-log row.

    reveal:  runtime's monitor, then the anomaly counters
    writer:  the anomaly counters, then the runtime's monitor

Seen in a host on PostgreSQL: a person read a record with a masked field and
revealed it at once; the writer's tick drained the read's log rows at the
same moment. The request and the writer deadlocked (thread dump: the request
in SystemWrites.insertOne under RevealService.reveal, the writer in
SecuredDataRuntime.classificationRows under AuditAction.fetchIdFromName
under its own insert), and every later request blocked in
classificationRows. The same order existed between a reveal and the
resolver's monitor, and the key source's.

- The rule rows have a lock of their own (ruleLock), held only while they
  are answered or read. A read is a query made as the system and takes none
  of the QBit's other locks, so nothing that holds ruleLock waits for one.
- The runtime's monitor is left to what it was for: one reveal or
  break-glass change at a time.

Tests (SecuredDataLockOrderTest, on H2 in PostgreSQL mode with the audit
tables): a thread is parked inside a reveal, and inside the start of a
break-glass session, holding the runtime's monitor; the access-log writer
must drain its queue and another thread must be told a classification and
the active retention rules meanwhile, and then the parked operation must
finish. Both tests fail without the fix (neither thread finishes).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-14 03:40:00 -06:00
CI 82d780c05d Bump version to 0.1.0-RC.7 [skip ci] 2026-11-14 08:25:01 +00:00
james.maes 1c5304992d Merge pull request 'fix: a rule read that asks for its own rows no longer recurses (release/0.1 port of #43)' (#44) from fix/reentrant-rule-reload-0.1 into release/0.1
maven / build (push) Successful in 1m54s
build / maven (push) Successful in 1m54s
2026-11-14 08:23:08 +00:00
james.maesandClaude Fable 5.2 db26a1dc2e fix: a rule read that asks for its own rows no longer recurses
maven / build (push) Successful in 1m22s
build / maven (push) Successful in 1m22s
Reading the rule rows a node keeps (classifications, active retention rules,
policies) is a query on a secured-data table, and other code runs inside that
query: a virtual field value provider or a post-query customizer that another
QBit or the host put on the table. When that code runs a query with filter
criteria, core validates the operands of classified fields and asks this QBit
for the classification of the field, which needs the rows that are being
read. The version of the rows was recorded only after the read returned, so
the nested question started the read again, and that read asked again:
StackOverflowError, and the same on every later request, since nothing was
ever recorded. One classification row was enough (core runs no value provider
for an empty result).

- System reads of the QBit's own tables (SystemWrites.query, the audit key
  lookup) name the stored fields of the table. Core fills a virtual field only
  for a query that selects it, so no value provider runs inside a system read.
- RuleRows holds one kind of rule rows and is used for all three kinds, which
  had the same shape. A question asked from inside a read is answered from the
  rows held when the read began (none before the first read) and never starts
  another read. The owner's monitor makes that answer private to the reading
  thread: every other thread waits and gets the rows the read returns.
- A read that fails records nothing and clears the reading mark, so the next
  question reads again. The rows held before are not given to a request in
  place of rows that could not be read.
- RetentionRules.active: a question asked from inside a transaction's read of
  its own rule edits is answered from the rules every thread shares instead
  of reading the edits again.

Who can be given the nested answer: only code on the reading thread, inside
SystemWrites.call, where the session is the system session and the resolver
answers FULL without looking at a classification. No field access decision
for a person is made from it; the request that started the read, and every
request on another thread, gets the rows the read returns.

Tests (SecuredDataRuleReloadTest): 9 of the 11 end in StackOverflowError
without the fix. They cover a value provider and a post-query customizer on
the classification, retention rule and policy tables, the answer a nested
question gets in both directions of a change, the session it is asked in, a
second thread waiting for the read, and failed reads (an exception and an
Error) being read again.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-14 01:56:49 -06:00
CI f7c7ada8ca Bump version to 0.1.0-RC.6 [skip ci] 2026-11-10 21:39:44 +00:00
james.maes 726a84889d Merge pull request 'Merge develop into release/0.1 for 0.1.0-RC.6 (#41 performance fix)' (#42) from merge/develop-into-release-0.1-rc6 into release/0.1
maven / build (push) Successful in 1m37s
build / maven (push) Successful in 1m37s
2026-11-10 21:38:14 +00:00
james.maesandClaude Fable 5.2 41b39d535b Merge develop into release/0.1 for 0.1.0-RC.6 (#41 request-scoped fingerprint, cached rule rows)
maven / build (push) Successful in 1m27s
build / maven (push) Successful in 1m27s
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 15:37:45 -06:00
CI 8ae8326f4e Bump version to 0.1.0-RC.5 [skip ci] 2026-11-07 18:29:11 +00:00
james.maes 92c54535ce Merge pull request 'Merge develop into release/0.1 (Gitea-aligned RC)' (#40) from align/develop-into-release-0.1 into release/0.1
maven / build (push) Successful in 3m28s
build / maven (push) Successful in 8m29s
2026-11-07 18:16:52 +00:00
james.maesandClaude Opus 5.5 65025ad61f Merge develop into release/0.1 (Gitea-aligned RC)
maven / build (push) Canceled after 0s
build / maven (push) Canceled after 0s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 12:16:29 -06:00
james.maesandCodex cb52c31cd8 ci: centralize release branch verification and workflows (#27)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 12:34:53 -05:00
CircleCI feb3082c1b Bump version to 0.1.0-RC.4 [skip ci] 2026-10-16 16:54:47 +00:00
james.maesandClaude Opus 5.5 1343c0925e Merge develop into release/0.1 for 0.1.0-RC.4 (#19 lifecycle leak fix)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 11:51:58 -05:00
CircleCI c669c44b61 Bump version to 0.1.0-RC.3 [skip ci] 2026-10-16 16:41:56 +00:00
james.maesandClaude Opus 5.5 fdd5fb42e6 Merge develop into release/0.1 for 0.1.0-RC.3 (#18 virtual-field and access-log worker startup fixes)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 11:38:49 -05:00
CircleCI fd60abaffa Bump version to 0.1.0-RC.2 [skip ci] 2026-10-15 21:38:55 +00:00
james.maesandClaude Opus 5.5 f7003e10af Merge develop into release/0.1 for 0.1.0-RC.2 (#16 UI API 1.6.0 extension 1.1.0, QQQ RC.15)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 16:35:07 -05:00
CircleCI 8417cfe032 Bump version to 0.1.0-RC.1 [skip ci] 2026-10-14 21:47:14 +00:00
+1 -1
View File
@@ -25,7 +25,7 @@
<qqq.version>4.1.0-RC.26</qqq.version> <!-- feeds qbit-build-parent's qqq-bom-pom import -->
<!-- QQQ 4.1.0-RC.15 ships UI API 1.6.0, field access and audit value protection -->
<qqq.snapshot.version>4.1.0-RC.26</qqq.snapshot.version>
<revision>0.1.0-SNAPSHOT</revision>
<revision>0.1.0-RC.9</revision>
<maven.compiler.source>21</maven.compiler.source>
<maven.compiler.target>21</maven.compiler.target>
<jacoco.version>0.8.15</jacoco.version>