feat: a create date and a modify date on every table (release/0.1 port of #47) #48

Merged
james.maes merged 2 commits from feat/table-standard-dates-0.1 into release/0.1 2026-11-20 16:45:02 +00:00
Owner

Port of #47 to release/0.1: the same commit, cherry-picked. The two lines differ only in the version in pom.xml, as they did before.

Every table of the QBit has a create date and a modify date: DATE_TIME, not editable, with core's CREATE_DATE and MODIFY_DATE behaviors, in the columns create_date and modify_date.

What changes

  • Seven tables get the two dates: reveal grant, break-glass session, access log, wipe run, wipe marker, legal hold, audit key. Their fields are named createDate and modifyDate.
  • Three tables already had them: classification, policy and retention rule keep the field names create_date and modify_date (a rename would change what a host reads and what their audits record). The two fields are now not editable and have the behaviors.
  • Writes. Every write of the QBit is a core insert or update action, so the dates are in the statement that writes the row. The table customizer sets them, as it did for the three tables: a new row gets both, equal, from the QBit's clock; a changed row gets a new modify date. The access log and wipe runs are only ever inserted, so their dates stay equal.
  • The access-log writer still runs one insert per row and no other statement, lock or transaction. A test counts its statements.
  • Schema. docs/ddl/secured-data-postgres.sql has a new last block that adds the two columns to the seven tables. No CREATE TABLE statement is edited.

Upgrade: run the DDL first

An existing database runs the last block of the DDL (from the line -- Standard dates: to the end) before this release is deployed. This release does not run without the columns. The release before it goes on working once they are there.

  • The columns are nullable and default to the current time, so an insert that names neither still succeeds and gets both.
  • No table is rewritten: adding a nullable column and setting a default change the catalog only.
  • Rows that were already there get the row's own time (created_at, granted_at, started_at and so on; the README has the table). The UPDATE touches only rows whose create_date is null, takes row locks only, and can be run in ranges of id on a large access log.
  • The block can be run again and then changes nothing.

One thing that comes with the behavior

Core uses a DATE_TIME field with the MODIFY_DATE behavior as the record version of its table. That now holds for all ten tables, classification, policy and retention rule included:

  • On an update core sets the modify date itself, after the customizer has run.
  • An update that sends the version it read (expectedVersion) is refused when the row has changed since.
  • Updates that send no version, the QBit's own among them, behave as before.

What does not change

  • The dates have no classification, policy or assistant exposure of their own. They are CONFIDENTIAL with their table and are given the tier of its other fields.
  • Nothing signed or verified covers them. A wrapped data key is bound to its table, field and record id; the fingerprints of a log row are made from search values.
  • A changed policy or classification writes the same core audit, field by field, and the same POLICY_CHANGE or CLASSIFICATION_CHANGE row.

Tests

SecuredDataStandardDatesTest (7) and one new test in SecuredDataDdlTest:

  • Every table the producer declares has the two dates. The tables are taken from the instance, so a table added later without them fails.
  • The dates resolve to the tier of the other fields of their table, for the default tier, a hiding policy, a showing policy, a channel cap and the administrator.
  • Every writer sets both and only a change moves the modify date: access-log writer, policy, classification, legal hold, reveal, break-glass start, end and expiry (memory backend); audit key, wipe run, wipe marker, retention rule (shipped schema on H2).
  • The access-log writer runs exactly one insert per row.
  • A changed policy and classification write the audit and log rows they wrote before. This test passes with and without the change.
  • Rows written before the columns existed are unchanged by the upgrade block: the same rows, the same wrapped keys, which unwrap to the same data keys, and the same audit token, which verifies the same value and no other.
  • The shipped DDL gives every declared table the two columns; the block adds them where they were missing, an insert that names neither succeeds, and the block can be run twice.

Seven of the eight new tests fail without the change. ./mvnw verify: 122 tests, 0 failures, 0 Checkstyle violations, coverage checks met.

🤖 Generated with Claude Code

Port of #47 to `release/0.1`: the same commit, cherry-picked. The two lines differ only in the version in `pom.xml`, as they did before. Every table of the QBit has a create date and a modify date: `DATE_TIME`, not editable, with core's `CREATE_DATE` and `MODIFY_DATE` behaviors, in the columns `create_date` and `modify_date`. ## What changes - **Seven tables get the two dates**: reveal grant, break-glass session, access log, wipe run, wipe marker, legal hold, audit key. Their fields are named `createDate` and `modifyDate`. - **Three tables already had them**: classification, policy and retention rule keep the field names `create_date` and `modify_date` (a rename would change what a host reads and what their audits record). The two fields are now not editable and have the behaviors. - **Writes.** Every write of the QBit is a core insert or update action, so the dates are in the statement that writes the row. The table customizer sets them, as it did for the three tables: a new row gets both, equal, from the QBit's clock; a changed row gets a new modify date. The access log and wipe runs are only ever inserted, so their dates stay equal. - **The access-log writer** still runs one insert per row and no other statement, lock or transaction. A test counts its statements. - **Schema.** `docs/ddl/secured-data-postgres.sql` has a new last block that adds the two columns to the seven tables. No `CREATE TABLE` statement is edited. ## Upgrade: run the DDL first An existing database runs the last block of the DDL (from the line `-- Standard dates:` to the end) **before** this release is deployed. This release does not run without the columns. The release before it goes on working once they are there. - The columns are nullable and default to the current time, so an insert that names neither still succeeds and gets both. - No table is rewritten: adding a nullable column and setting a default change the catalog only. - Rows that were already there get the row's own time (`created_at`, `granted_at`, `started_at` and so on; the README has the table). The `UPDATE` touches only rows whose `create_date` is null, takes row locks only, and can be run in ranges of `id` on a large access log. - The block can be run again and then changes nothing. ## One thing that comes with the behavior Core uses a `DATE_TIME` field with the `MODIFY_DATE` behavior as the record version of its table. That now holds for all ten tables, classification, policy and retention rule included: - On an update core sets the modify date itself, after the customizer has run. - An update that sends the version it read (`expectedVersion`) is refused when the row has changed since. - Updates that send no version, the QBit's own among them, behave as before. ## What does not change - The dates have no classification, policy or assistant exposure of their own. They are CONFIDENTIAL with their table and are given the tier of its other fields. - Nothing signed or verified covers them. A wrapped data key is bound to its table, field and record id; the fingerprints of a log row are made from search values. - A changed policy or classification writes the same core audit, field by field, and the same `POLICY_CHANGE` or `CLASSIFICATION_CHANGE` row. ## Tests `SecuredDataStandardDatesTest` (7) and one new test in `SecuredDataDdlTest`: - Every table the producer declares has the two dates. The tables are taken from the instance, so a table added later without them fails. - The dates resolve to the tier of the other fields of their table, for the default tier, a hiding policy, a showing policy, a channel cap and the administrator. - Every writer sets both and only a change moves the modify date: access-log writer, policy, classification, legal hold, reveal, break-glass start, end and expiry (memory backend); audit key, wipe run, wipe marker, retention rule (shipped schema on H2). - The access-log writer runs exactly one insert per row. - A changed policy and classification write the audit and log rows they wrote before. This test passes with and without the change. - Rows written before the columns existed are unchanged by the upgrade block: the same rows, the same wrapped keys, which unwrap to the same data keys, and the same audit token, which verifies the same value and no other. - The shipped DDL gives every declared table the two columns; the block adds them where they were missing, an insert that names neither succeeds, and the block can be run twice. Seven of the eight new tests fail without the change. `./mvnw verify`: 122 tests, 0 failures, 0 Checkstyle violations, coverage checks met. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
james.maes added 1 commit 2026-11-20 15:03:03 +00:00
feat: a create date and a modify date on every table
maven / build (push) Successful in 1m56s
build / maven (push) Successful in 1m55s
4e20070ae1
Every table of the QBit has a create date and a modify date: DATE_TIME, not
editable, with core's CREATE_DATE and MODIFY_DATE behaviors, in the columns
create_date and modify_date.

Seven tables did not have them: reveal grant, break-glass session, access
log, wipe run, wipe marker, legal hold and audit key. Their fields are named
createDate and modifyDate. Classification, policy and retention rule already
had create_date and modify_date, set by the table customizer. They keep those
field names, because a rename would change what a host reads and what their
audits record, and the two fields are now not editable and have the
behaviors.

Writes. Every write of the QBit is a core insert or update action, so the
dates are in the statement that writes the row. The table customizer sets
them, as it did for the three tables: a new row gets both, equal, from the
QBit's clock, and a changed row a new modify date. The access-log writer
still runs one insert per row and no other statement, lock or transaction.
The access log and wipe runs are only ever inserted, so their dates stay
equal.

Core uses a DATE_TIME field with the MODIFY_DATE behavior as the record
version of its table. That now holds for all ten tables: on an update core
sets the modify date itself, after the customizer has run, and an update
that sends the version it read is refused when the row has changed since.
Updates that send no version, the QBit's own among them, behave as before.

Schema. docs/ddl/secured-data-postgres.sql has a new last block that adds
the two columns to the seven tables: nullable, defaulting to the current
time, with the rows that were already there given the row's own time. No
CREATE TABLE statement is edited. An existing database runs that block
alone, before this release is deployed. The block rewrites no table, can be
run while the release before this one is running (its inserts name neither
column and get both from the default) and can be run again. This release
does not run without the columns. README: "Upgrading an existing database".

Unchanged. The dates have no classification, policy or assistant exposure of
their own: they are CONFIDENTIAL with their table and get the tier of its
other fields. Nothing signed or verified covers them: a wrapped data key is
bound to its table, field and record id, and the fingerprints of a log row
are made from search values. A changed policy or classification writes the
same core audit, field by field, and the same POLICY_CHANGE or
CLASSIFICATION_CHANGE row.

Tests. SecuredDataStandardDatesTest: every table the producer declares has
the two dates (the tables are taken from the instance, so a table added
later without them fails); the dates resolve to the tier of the other fields
of their table for five kinds of session; every writer sets both and only a
change moves the modify date (access-log writer, policy, classification,
legal hold, reveal, break-glass start, end and expiry on the memory backend;
audit key, wipe run, wipe marker and retention rule in the shipped schema);
the access-log writer runs exactly one insert per row; a changed policy and
classification write the audit and log rows they wrote before; rows written
before the columns existed are unchanged by the upgrade block, their wrapped
keys unwrap to the same data keys and their audit token verifies the same
value and no other. SecuredDataDdlTest: the shipped DDL gives every declared
table the two columns, the upgrade block adds them where they were missing,
an insert that names neither succeeds, and the block can be run twice. Seven
of the eight new tests fail without the change; the eighth is the record of
what a changed policy wrote before and passes on both sides.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
james.maes added 1 commit 2026-11-20 16:42:12 +00:00
fix: an edit over HTTP is an edit of its row, not a new row
maven / build (push) Successful in 1m55s
build / maven (push) Successful in 1m55s
0a8c0fba6f
The table customizer finds the stored row of an update by its id, and
compared the two ids as they came: the id of a request over HTTP is text
(it is the id in the path), the stored id is a number. They were never
equal, so an edit made through the API or the UI had no stored row and was
validated and stamped as an insert. The uniqueness check of a
classification compared ids the same way. Both now compare the ids as text.

This is older than the standard dates, but they build on it: the customizer
gives a new row its create date, and a client that knows record versions
edits a table with a modify date by sending only the changed values and the
version it read.

What an edit over HTTP did before:
- A classification, policy or retention rule got a new create_date.
- Releasing a legal hold overwrote when and by whom it was placed (and,
  since the hold has one, its create date).
- Editing the reason of a legal hold without sending its status failed with
  a server error (the validation read the status of a row it did not have).
- Editing a classification while naming its table and field was refused as
  a second classification of that field.
- Editing a retention rule without sending every field was refused.

Tests (SecuredDataHttpEditTest, through the real middleware on the shipped
schema, a day after the rows were written): the reason of a hold is edited
with the changed value and the version, and the stale version is then
refused; a hold is released and keeps when and by whom it was placed and
its create date; a classification is edited naming its table and field; a
policy and a retention rule are renamed and keep their create dates; a row
with neither date is read and edited. All five fail without the fix. They
run in UTC: where the default zone of the JVM is not UTC, core refuses
every update that sends a version, which is core's to fix and is kept out
of these tests.

Notes for an upgrade (shipped DDL block and README): set a lock_timeout
before the ALTER statements and run the block again when one is cancelled;
on a large access log run its backfill UPDATE in ranges of id, or leave it
out, since rows without the two dates are read and written like any
others. Changelog: the fix, that an update naming the create date still
sets it, and that a wipe moves the record version of its retention rule.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
james.maes merged commit 960ce55f8b into release/0.1 2026-11-20 16:45:02 +00:00
Sign in to join this conversation.