Compare commits

...
2 Commits
Author SHA1 Message Date
james.maesandClaude Fable 5.2 faeca595f6 fix(schema): a row written once has modifyDate equal to createDate exactly
maven / build (push) Successful in 11m52s
build / maven (push) Successful in 11m52s
QQQ's createDate and modifyDate behaviors each read the clock, so the two
values of one insert differ by a few microseconds on a clock finer than the
machine this was first run on. The decision claim and the budget event are
written once and never change; an insert customizer on those two tables makes
modifyDate the same instant as createDate. The test of the other tables no
longer asserts equality they cannot give (they are rewritten when a
transcript is redacted).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-20 09:13:31 -06:00
james.maesandClaude Fable 5.2 57798a67fc feat(schema): every table has a create date and a modify date
maven / build (push) Failing after 11m24s
build / maven (push) Failing after 11m24s
Nine tables lacked a standard date: eight had createDate but no modifyDate
(turn events, tool calls, feedback, model calls, data audit, budget events,
client errors, pending action claims) and the budget accounting table had
neither. All nine now declare both, date-time, not editable, set by QQQ.

Every write the QBit makes goes through QQQ's own insert and update actions,
so an insert sets both and an update (an incremented daily usage counter, a
decided pending action, a cleared tool result, a scheduled run that ends, the
accounting row that follows a usage row) moves modifyDate. A table that is
only inserted has modifyDate equal to createDate.

The reference DDL gets an additive block (nullable columns defaulting to the
time of the write, so the previous release keeps working; existing rows get
the table's own creation time). Tests iterate the declared tables, exercise
the store's insert and update paths, and check the DDL creates the columns.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-20 08:45:59 -06:00
16 changed files with 889 additions and 0 deletions

No files matched your search

+6
View File
@@ -1,5 +1,11 @@
# Changelog
## Unreleased - Every table has a create date and a modify date
- Nine tables lacked a standard date: `assistant_turn_event`, `assistant_tool_call`, `assistant_feedback`, `assistant_usage`, `assistant_data_audit`, `assistant_budget_event`, `assistant_client_error` and `assistant_pending_action_claim` had `createDate` but no `modifyDate`, and `assistant_budget_usage` had neither. All nine now declare both, `DATE_TIME`, not editable, set by QQQ (`CREATE_DATE` and `MODIFY_DATE` behaviors); every other table already had them.
- Every write the QBit makes goes through QQQ's insert and update actions, so an insert sets both dates and an update (an incremented daily usage counter, a decided pending action, a cleared tool result, a scheduled run that ends, the budget accounting row that follows a usage row) moves `modifyDate` and leaves `createDate`. A row of a table whose rows never change (`assistant_pending_action_claim`, `assistant_budget_event`) has `modifyDate` equal to `createDate` exactly: QQQ's two date behaviors read the clock separately, so a new insert customizer (`AssistantInsertOnlyDatesCustomizer`, registered by the QBit on those two tables) makes the two one instant. The other new `modifyDate` values are QQQ's own (turn events, feedback, client errors and the data audit are inserted once, and rewritten only when a transcript is redacted).
- `docs/schema-postgres.sql` has a new block at its end (additive; see UPGRADE.md): the columns are nullable and default to the time of the write, so the previous release keeps working against the same database while a host migrates.
## Unreleased - The assistant can read the files of records, where a host turns that on; a link in an answer is one the application gave
Design: `docs/file-contents-and-links.md`. **Off by default: an upgrade changes nothing until a host turns file reading on in code (see `UPGRADE.md`).**
+21
View File
@@ -1,5 +1,26 @@
# Upgrading qbit-assistant
## Every table has create and modify dates (Unreleased)
Nine tables gain columns: `modify_date` on `assistant_turn_event`, `assistant_tool_call`,
`assistant_feedback`, `assistant_usage`, `assistant_data_audit`, `assistant_budget_event`,
`assistant_client_error` and `assistant_pending_action_claim`, and `create_date` and `modify_date` on
`assistant_budget_usage` (the names an application gets when it derives snake_case backend names from
field names; otherwise `modifyDate` and `createDate`). A host that writes its own migrations runs
the block at the end of `docs/schema-postgres.sql` ("Table standard") **before** deploying this release:
- It is safe while the previous release is still running against the same database: the columns are
nullable and default to `CURRENT_TIMESTAMP`, so rows the previous release inserts are given a value.
On PostgreSQL 11 and later adding a column with that (stable) default does not rewrite the table.
- The `UPDATE` statements give existing rows the table's own creation time (`occurred_at` for
`assistant_budget_usage`). They touch every row: on a large table (`assistant_usage`,
`assistant_tool_call`, `assistant_turn_event`) run each in id ranges, and run them before the new
release starts writing, or they overwrite real `modify_date` values.
- `assistant_pending_action_claim` and `assistant_budget_event` rows are written once: their `modifyDate` equals their `createDate` and never moves.
Nothing else changes: no configuration, no code to change. A host whose database is made from
the reference DDL, or whose own migrations run the block, needs no customizer.
## The assistant can read the files of records (Unreleased)
**Nothing changes on upgrade: reading the content of files is off until a host turns it on in code.** Turned on, the assistant reads the content of a file a record holds, for a person who may open it. An answer keeps no address of the application that the application did not give. No schema change, no permission, no API change. Design: `docs/file-contents-and-links.md`.
+34
View File
@@ -777,3 +777,37 @@ CREATE TABLE IF NOT EXISTS assistant_pending_action_claim
CONSTRAINT assistant_pending_action_claim_action UNIQUE (pending_action_id)
);
CREATE INDEX IF NOT EXISTS assistant_pending_action_claim_user_id ON assistant_pending_action_claim (user_id);
-- Table standard: every table has create_date and modify_date. Eight tables had only
-- create_date and assistant_budget_usage had neither. Additive and safe while the
-- previous release is still running against the same database:
-- * the columns are nullable and default to the time of the write, so rows the previous
-- release inserts (it does not know the columns) are still given a value;
-- * on PostgreSQL 11 and later adding a column whose default is CURRENT_TIMESTAMP
-- (STABLE, evaluated once at ALTER time) does not rewrite the table and takes only a
-- short lock;
-- * the UPDATE statements give existing rows the table's own creation time (for
-- assistant_budget_usage, occurred_at, the usage row's creation time). They touch every
-- row, so on a large table (assistant_usage, assistant_tool_call, assistant_turn_event)
-- run each one in id ranges (AND id BETWEEN a AND b). Run them BEFORE the new release
-- starts writing: afterwards they would overwrite real modify_date values.
-- Safe to re-run.
ALTER TABLE assistant_turn_event ADD COLUMN IF NOT EXISTS modify_date TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP;
UPDATE assistant_turn_event SET modify_date = create_date WHERE create_date IS NOT NULL AND modify_date IS DISTINCT FROM create_date;
ALTER TABLE assistant_tool_call ADD COLUMN IF NOT EXISTS modify_date TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP;
UPDATE assistant_tool_call SET modify_date = create_date WHERE create_date IS NOT NULL AND modify_date IS DISTINCT FROM create_date;
ALTER TABLE assistant_feedback ADD COLUMN IF NOT EXISTS modify_date TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP;
UPDATE assistant_feedback SET modify_date = create_date WHERE create_date IS NOT NULL AND modify_date IS DISTINCT FROM create_date;
ALTER TABLE assistant_usage ADD COLUMN IF NOT EXISTS modify_date TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP;
UPDATE assistant_usage SET modify_date = create_date WHERE create_date IS NOT NULL AND modify_date IS DISTINCT FROM create_date;
ALTER TABLE assistant_data_audit ADD COLUMN IF NOT EXISTS modify_date TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP;
UPDATE assistant_data_audit SET modify_date = create_date WHERE create_date IS NOT NULL AND modify_date IS DISTINCT FROM create_date;
ALTER TABLE assistant_budget_event ADD COLUMN IF NOT EXISTS modify_date TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP;
UPDATE assistant_budget_event SET modify_date = create_date WHERE create_date IS NOT NULL AND modify_date IS DISTINCT FROM create_date;
ALTER TABLE assistant_client_error ADD COLUMN IF NOT EXISTS modify_date TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP;
UPDATE assistant_client_error SET modify_date = create_date WHERE create_date IS NOT NULL AND modify_date IS DISTINCT FROM create_date;
ALTER TABLE assistant_pending_action_claim ADD COLUMN IF NOT EXISTS modify_date TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP;
UPDATE assistant_pending_action_claim SET modify_date = create_date WHERE create_date IS NOT NULL AND modify_date IS DISTINCT FROM create_date;
ALTER TABLE assistant_budget_usage ADD COLUMN IF NOT EXISTS create_date TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP;
ALTER TABLE assistant_budget_usage ADD COLUMN IF NOT EXISTS modify_date TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP;
UPDATE assistant_budget_usage SET create_date = occurred_at, modify_date = occurred_at WHERE create_date IS DISTINCT FROM occurred_at OR modify_date IS DISTINCT FROM occurred_at;
@@ -0,0 +1,63 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant.metadata;
import java.util.List;
import com.kingsrook.qqq.backend.core.actions.customizers.AbstractPreInsertCustomizer;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** For a table whose rows are written once and never change (a decision claim,
** a budget event): modifyDate equals createDate, exactly. QQQ's two date
** behaviors each read the clock on their own, so left alone the two values of
** one insert differ by a few microseconds. This runs after the behaviors and
** before the row is stored, and makes the second the same instant as the first.
*******************************************************************************/
public class AssistantInsertOnlyDatesCustomizer extends AbstractPreInsertCustomizer
{
/*******************************************************************************
** After the date behaviors ran (they run before the unique-key checks).
*******************************************************************************/
@Override
public WhenToRun getWhenToRun()
{
return (WhenToRun.BEFORE_UNIQUE_KEY_CHECKS);
}
/*******************************************************************************
** modifyDate = createDate on every record.
*******************************************************************************/
@Override
public List<QRecord> apply(List<QRecord> records) throws QException
{
for(QRecord record : records)
{
if(record.getValue("createDate") != null)
{
record.setValue("modifyDate", record.getValue("createDate"));
}
}
return (records);
}
}
@@ -165,6 +165,11 @@ public class AssistantMetaDataBuilder
{
List<QTableMetaData> tables = new ArrayList<>();
//////////////////////////////////////////////////////////////////////////////////
// tables whose rows are written once and never change: modifyDate = createDate //
//////////////////////////////////////////////////////////////////////////////////
QCodeReference insertOnlyDates = new QCodeReference(AssistantInsertOnlyDatesCustomizer.class);
QCodeReference conversationCustomizer = new QCodeReference(AssistantConversationCustomizer.class);
tables.add(owned(base(AssistantConversation.TABLE_NAME, "Assistant Conversation", "forum", AssistantConversation.class), false)
.withRecordLabelFormat("%s")
@@ -261,6 +266,7 @@ public class AssistantMetaDataBuilder
///////////////////////////////////////////////////////////////////////
tables.add(readOnly(owned(base(AssistantPendingActionClaim.TABLE_NAME, "Assistant Pending Action Claim", "task_alt", AssistantPendingActionClaim.class), false))
.withUniqueKey(new UniqueKey("pendingActionId"))
.withCustomizer(TableCustomizers.PRE_INSERT_RECORD, insertOnlyDates)
.withRecordLabelFormat("%s #%s").withRecordLabelFields("decision", "pendingActionId"));
tables.add(readOnly(owned(base(AssistantDataAudit.TABLE_NAME, "Assistant Data Audit", "shield", AssistantDataAudit.class), false))
@@ -301,6 +307,7 @@ public class AssistantMetaDataBuilder
.withCustomizer(TableCustomizers.PRE_UPDATE_RECORD, budgetCustomizer));
tables.add(readOnly(base(AssistantBudgetEvent.TABLE_NAME, "Budget Event", "notifications", AssistantBudgetEvent.class))
.withoutCapabilities(Capability.TABLE_DELETE)
.withCustomizer(TableCustomizers.PRE_INSERT_RECORD, insertOnlyDates)
.withUniqueKey(new UniqueKey("budgetId", "scopeKey", "periodStart", "state")));
tables.forEach(AssistantRecordScreens::apply);
return (tables);
@@ -59,6 +59,9 @@ public class AssistantBudgetEvent extends QRecordEntity
@QField(isEditable = false)
private Instant createDate;
@QField(isEditable = false)
private Instant modifyDate;
/*******************************************************************************
** Default constructor.
@@ -356,4 +359,35 @@ public class AssistantBudgetEvent extends QRecordEntity
return (this);
}
/*******************************************************************************
** Getter for modifyDate
*******************************************************************************/
public Instant getModifyDate()
{
return (this.modifyDate);
}
/*******************************************************************************
** Setter for modifyDate
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Fluent setter for modifyDate
*******************************************************************************/
public AssistantBudgetEvent withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
}
@@ -66,6 +66,12 @@ public class AssistantBudgetUsage extends QRecordEntity
@QField()
private Long cacheWriteTokens;
@QField(isEditable = false)
private Instant createDate;
@QField(isEditable = false)
private Instant modifyDate;
/*******************************************************************************
** Default constructor.
@@ -425,4 +431,66 @@ public class AssistantBudgetUsage extends QRecordEntity
return (this);
}
/*******************************************************************************
** Getter for createDate
*******************************************************************************/
public Instant getCreateDate()
{
return (this.createDate);
}
/*******************************************************************************
** Setter for createDate
*******************************************************************************/
public void setCreateDate(Instant createDate)
{
this.createDate = createDate;
}
/*******************************************************************************
** Fluent setter for createDate
*******************************************************************************/
public AssistantBudgetUsage withCreateDate(Instant createDate)
{
this.createDate = createDate;
return (this);
}
/*******************************************************************************
** Getter for modifyDate
*******************************************************************************/
public Instant getModifyDate()
{
return (this.modifyDate);
}
/*******************************************************************************
** Setter for modifyDate
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Fluent setter for modifyDate
*******************************************************************************/
public AssistantBudgetUsage withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
}
@@ -85,6 +85,9 @@ public class AssistantClientError extends QRecordEntity
@QField(isEditable = false)
private Instant createDate;
@QField(isEditable = false)
private Instant modifyDate;
/*******************************************************************************
@@ -631,4 +634,35 @@ public class AssistantClientError extends QRecordEntity
return (this);
}
/*******************************************************************************
** Getter for modifyDate
*******************************************************************************/
public Instant getModifyDate()
{
return (this.modifyDate);
}
/*******************************************************************************
** Setter for modifyDate
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Fluent setter for modifyDate
*******************************************************************************/
public AssistantClientError withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
}
@@ -72,6 +72,9 @@ public class AssistantDataAudit extends QRecordEntity
@QField(isEditable = false)
private Instant createDate;
@QField(isEditable = false)
private Instant modifyDate;
/*******************************************************************************
@@ -477,4 +480,33 @@ public class AssistantDataAudit extends QRecordEntity
/*******************************************************************************
** Getter for modifyDate
*******************************************************************************/
public Instant getModifyDate()
{
return (this.modifyDate);
}
/*******************************************************************************
** Setter for modifyDate
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Fluent setter for modifyDate
*******************************************************************************/
public AssistantDataAudit withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
}
@@ -61,6 +61,9 @@ public class AssistantFeedback extends QRecordEntity
@QField(isEditable = false)
private Instant createDate;
@QField(isEditable = false)
private Instant modifyDate;
/*******************************************************************************
@@ -328,4 +331,35 @@ public class AssistantFeedback extends QRecordEntity
return (this);
}
/*******************************************************************************
** Getter for modifyDate
*******************************************************************************/
public Instant getModifyDate()
{
return (this.modifyDate);
}
/*******************************************************************************
** Setter for modifyDate
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Fluent setter for modifyDate
*******************************************************************************/
public AssistantFeedback withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
}
@@ -50,6 +50,9 @@ public class AssistantPendingActionClaim extends QRecordEntity
@QField(isEditable = false)
private Instant createDate;
@QField(isEditable = false)
private Instant modifyDate;
/*******************************************************************************
@@ -224,4 +227,35 @@ public class AssistantPendingActionClaim extends QRecordEntity
return (this);
}
/*******************************************************************************
** Getter for modifyDate
*******************************************************************************/
public Instant getModifyDate()
{
return (this.modifyDate);
}
/*******************************************************************************
** Setter for modifyDate
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Fluent setter for modifyDate
*******************************************************************************/
public AssistantPendingActionClaim withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
}
@@ -109,6 +109,9 @@ public class AssistantToolCall extends QRecordEntity
@QField(isEditable = false)
private Instant createDate;
@QField(isEditable = false)
private Instant modifyDate;
/*******************************************************************************
@@ -837,4 +840,36 @@ public class AssistantToolCall extends QRecordEntity
this.clientChannel = clientChannel;
return (this);
}
/*******************************************************************************
** Getter for modifyDate
*******************************************************************************/
public Instant getModifyDate()
{
return (this.modifyDate);
}
/*******************************************************************************
** Setter for modifyDate
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Fluent setter for modifyDate
*******************************************************************************/
public AssistantToolCall withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
}
@@ -59,6 +59,9 @@ public class AssistantTurnEvent extends QRecordEntity
@QField(isEditable = false)
private Instant createDate;
@QField(isEditable = false)
private Instant modifyDate;
/*******************************************************************************
@@ -295,4 +298,35 @@ public class AssistantTurnEvent extends QRecordEntity
return (this);
}
/*******************************************************************************
** Getter for modifyDate
*******************************************************************************/
public Instant getModifyDate()
{
return (this.modifyDate);
}
/*******************************************************************************
** Setter for modifyDate
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Fluent setter for modifyDate
*******************************************************************************/
public AssistantTurnEvent withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
}
@@ -90,6 +90,9 @@ public class AssistantUsage extends QRecordEntity
@QField(isEditable = false)
private Instant createDate;
@QField(isEditable = false)
private Instant modifyDate;
/*******************************************************************************
@@ -698,4 +701,35 @@ public class AssistantUsage extends QRecordEntity
return (this);
}
/*******************************************************************************
** Getter for modifyDate
*******************************************************************************/
public Instant getModifyDate()
{
return (this.modifyDate);
}
/*******************************************************************************
** Setter for modifyDate
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Fluent setter for modifyDate
*******************************************************************************/
public AssistantUsage withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
}
@@ -0,0 +1,90 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.Statement;
import java.util.HashSet;
import java.util.Set;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import com.kingsrook.qbits.assistant.metadata.AssistantMetaDataBuilder;
import com.kingsrook.qbits.assistant.testing.BaseAssistantTest;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** The reference DDL (docs/schema-postgres.sql) gives every table the QBit
** declares the columns create_date and modify_date, for a database made from
** it (the creates, then the later additions, as the file says). The
** PostgreSQL-only parts are run by SchemaForeignKeyPostgresTest.
*******************************************************************************/
class TableStandardSchemaTest extends BaseAssistantTest
{
/*******************************************************************************
** Every declared table has both columns after the reference DDL ran.
*******************************************************************************/
@Test
void everyDeclaredTableHasBothDateColumns() throws Exception
{
Set<String> declared = new HashSet<>();
for(QTableMetaData table : new AssistantMetaDataBuilder(config).buildTables())
{
declared.add(table.getName());
}
try(Connection connection = DriverManager.getConnection("jdbc:h2:mem:tablestandard" + System.nanoTime() + ";MODE=PostgreSQL;DEFAULT_NULL_ORDERING=HIGH"))
{
String ddl = Files.readString(Path.of("..", "docs", "schema-postgres.sql"), StandardCharsets.UTF_8);
Pattern pattern = Pattern.compile("^(CREATE TABLE|ALTER TABLE)( IF NOT EXISTS)? (assistant_[a-z_]+)\\b(.*)", Pattern.DOTALL);
try(Statement statement = connection.createStatement())
{
for(String sql : AssistantDefaultsChangelogTest.statements(ddl))
{
Matcher m = pattern.matcher(sql);
if(m.find() && (m.group(1).startsWith("CREATE") || m.group(4).strip().startsWith("ADD COLUMN")))
{
statement.execute(sql.replace("TIMESTAMPTZ", "TIMESTAMP WITH TIME ZONE"));
}
}
}
for(String table : declared)
{
Set<String> columns = new HashSet<>();
try(Statement statement = connection.createStatement(); ResultSet rs = statement.executeQuery("SELECT LOWER(column_name) FROM information_schema.columns WHERE LOWER(table_name) = '" + table + "'"))
{
while(rs.next())
{
columns.add(rs.getString(1));
}
}
assertThat(columns).as(table + " columns").contains("create_date", "modify_date");
}
}
}
}
@@ -0,0 +1,329 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant.persistence;
import java.math.BigDecimal;
import java.time.Instant;
import java.time.LocalDate;
import java.util.List;
import com.kingsrook.qbits.assistant.metadata.AssistantMetaDataBuilder;
import com.kingsrook.qbits.assistant.model.AssistantBudgetEvent;
import com.kingsrook.qbits.assistant.model.AssistantBudgetUsage;
import com.kingsrook.qbits.assistant.model.AssistantClientError;
import com.kingsrook.qbits.assistant.model.AssistantDataAudit;
import com.kingsrook.qbits.assistant.model.AssistantFeedback;
import com.kingsrook.qbits.assistant.model.AssistantPendingAction;
import com.kingsrook.qbits.assistant.model.AssistantPendingActionClaim;
import com.kingsrook.qbits.assistant.model.AssistantScheduledRun;
import com.kingsrook.qbits.assistant.model.AssistantToolCall;
import com.kingsrook.qbits.assistant.model.AssistantTurnEvent;
import com.kingsrook.qbits.assistant.model.AssistantUsage;
import com.kingsrook.qbits.assistant.model.AssistantUsageDaily;
import com.kingsrook.qbits.assistant.spi.ModelUsage;
import com.kingsrook.qbits.assistant.testing.AssistantTestFixture;
import com.kingsrook.qbits.assistant.testing.BaseAssistantTest;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.fields.DynamicDefaultValueBehavior;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** The table standard: every table the QBit declares has a createDate and a
** modifyDate (date-time, not editable, set by QQQ), and every write path of
** the store sets them: an insert sets both, an update moves modifyDate and
** leaves createDate. A row of a table that is only ever inserted (a log, an
** audit row, a claim) has a modifyDate equal to its createDate, always.
*******************************************************************************/
class TableStandardDatesTest extends BaseAssistantTest
{
private static final String USER = AssistantTestFixture.ADMIN;
/*******************************************************************************
** Every table the producer declares, found from the metadata and not listed
** by hand, has both dates as the standard says.
*******************************************************************************/
@Test
void everyDeclaredTableHasBothDates() throws Exception
{
List<QTableMetaData> declared = new AssistantMetaDataBuilder(config).buildTables();
assertThat(declared).hasSizeGreaterThan(20);
for(QTableMetaData declaredTable : declared)
{
QTableMetaData table = qInstance.getTable(declaredTable.getName());
assertThat(table).as(declaredTable.getName() + " is in the instance").isNotNull();
assertDate(table, "createDate", DynamicDefaultValueBehavior.CREATE_DATE);
assertDate(table, "modifyDate", DynamicDefaultValueBehavior.MODIFY_DATE);
}
}
/*******************************************************************************
**
*******************************************************************************/
private static void assertDate(QTableMetaData table, String name, DynamicDefaultValueBehavior behavior)
{
QFieldMetaData field = table.getFields().get(name);
assertThat(field).as(table.getName() + "." + name).isNotNull();
assertThat(field.getType()).as(table.getName() + "." + name + " type").isEqualTo(QFieldType.DATE_TIME);
assertThat(field.getIsEditable()).as(table.getName() + "." + name + " editable").isFalse();
assertThat(field.getBehaviors()).as(table.getName() + "." + name + " behaviors").contains(behavior);
}
/*******************************************************************************
** A usage counter row that is incremented moves its modifyDate each time.
*******************************************************************************/
@Test
void incrementedUsageCounterMovesModifyDate() throws Exception
{
AssistantStore store = store();
LocalDate day = LocalDate.of(2026, 1, 15);
ModelUsage usage = new ModelUsage();
store.addUsage(USER, day, usage, 10L, true);
AssistantUsageDaily first = store.getUserUsage(USER, day);
assertThat(first.getCreateDate()).isNotNull();
assertThat(first.getModifyDate()).isNotNull();
pause();
store.addUsage(USER, day, usage, 10L, true);
AssistantUsageDaily second = store.getUserUsage(USER, day);
assertThat(second.getTurnCount()).isEqualTo(2);
assertThat(second.getCreateDate()).isEqualTo(first.getCreateDate());
assertThat(second.getModifyDate()).isAfter(first.getModifyDate());
}
/*******************************************************************************
** Rows written once (a decision claim, a budget event) have modifyDate equal
** to createDate, exactly: QQQ's two behaviors read the clock separately, so
** the table's insert customizer makes them one instant. Rows of the tables
** that are inserted and, only when a transcript is redacted, rewritten, have
** both dates set, modifyDate not before createDate.
*******************************************************************************/
@Test
void insertOnlyRowsHaveEqualDates() throws Exception
{
AssistantStore store = store();
Instant now = Instant.parse("2026-02-03T04:05:06Z");
for(int i = 0; i < 20; i++)
{
AssistantBudgetEvent budgetEvent = store.insert(new AssistantBudgetEvent().withBudgetId(1).withScopeKey("k" + i).withPeriodStart(now).withState("WARN")
.withUsedValue(BigDecimal.ONE).withLimitValue(BigDecimal.TEN));
assertEqualDates(budgetEvent.getCreateDate(), budgetEvent.getModifyDate());
AssistantPendingActionClaim claim = store.insert(new AssistantPendingActionClaim().withPendingActionId(100 + i).withUserId(USER).withDecision("EXECUTED"));
assertEqualDates(claim.getCreateDate(), claim.getModifyDate());
}
AssistantTurnEvent event = store.insert(new AssistantTurnEvent().withTurnId(1).withUserId(USER).withSeq(1).withEventType("text.delta").withDataJson("{}"));
assertBothSet(event.getCreateDate(), event.getModifyDate());
AssistantFeedback feedback = store.insert(new AssistantFeedback().withUserId(USER).withRating(1));
assertBothSet(feedback.getCreateDate(), feedback.getModifyDate());
AssistantClientError error = store.insert(new AssistantClientError().withUserId(USER).withScope("test").withMessage("m"));
assertBothSet(error.getCreateDate(), error.getModifyDate());
AssistantDataAudit audit = store.insert(new AssistantDataAudit().withUserId(USER).withDecision("ALLOWED"));
assertBothSet(audit.getCreateDate(), audit.getModifyDate());
}
/*******************************************************************************
**
*******************************************************************************/
private static void assertBothSet(Instant createDate, Instant modifyDate)
{
assertThat(createDate).isNotNull();
assertThat(modifyDate).isNotNull().isAfterOrEqualTo(createDate);
}
/*******************************************************************************
** A decision of a pending action: the action moves its modifyDate, and the
** claim row is written once with equal dates.
*******************************************************************************/
@Test
void pendingActionDecisionMovesTheActionAndInsertsTheClaim() throws Exception
{
AssistantStore store = store();
Instant now = Instant.parse("2026-02-03T04:05:06Z");
AssistantPendingAction action = store.insert(new AssistantPendingAction().withUserId(USER).withToolName("update_record").withStatus(AssistantPendingAction.STATUS_PENDING));
assertThat(action.getCreateDate()).isNotNull();
assertThat(action.getModifyDate()).isNotNull();
pause();
assertThat(store.decidePendingAction(USER, action.getId(), AssistantPendingAction.STATUS_RUNNING, null, now)).isTrue();
AssistantPendingAction decided = store.getPendingAction(USER, action.getId());
assertThat(decided.getStatus()).isEqualTo(AssistantPendingAction.STATUS_RUNNING);
assertThat(decided.getCreateDate()).isEqualTo(action.getCreateDate());
assertThat(decided.getModifyDate()).isAfter(action.getModifyDate());
List<AssistantPendingActionClaim> claims = store.query(AssistantPendingActionClaim.TABLE_NAME,
new QQueryFilter().withCriteria("pendingActionId", QCriteriaOperator.EQUALS, action.getId()), AssistantPendingActionClaim.class);
assertThat(claims).hasSize(1);
assertEqualDates(claims.get(0).getCreateDate(), claims.get(0).getModifyDate());
pause();
store.finishPendingAction(action.getId(), AssistantPendingAction.STATUS_EXECUTED, 200, "done");
AssistantPendingAction finished = store.getPendingAction(USER, action.getId());
assertThat(finished.getCreateDate()).isEqualTo(action.getCreateDate());
assertThat(finished.getModifyDate()).isAfter(decided.getModifyDate());
}
/*******************************************************************************
** Clearing a tool call's result (the retention step) is an update.
*******************************************************************************/
@Test
void clearingAToolResultMovesModifyDate() throws Exception
{
AssistantStore store = store();
AssistantToolCall call = store.insert(new AssistantToolCall().withUserId(USER).withToolName("query_records").withResultJson("{}"));
assertThat(call.getCreateDate()).isNotNull();
assertThat(call.getModifyDate()).isNotNull();
pause();
assertThat(store.clearToolResultsBefore(Instant.now().plusSeconds(60), 10)).isEqualTo(1);
AssistantToolCall cleared = store.query(AssistantToolCall.TABLE_NAME, new QQueryFilter().withCriteria("id", QCriteriaOperator.EQUALS, call.getId()), AssistantToolCall.class).get(0);
assertThat(cleared.getResultJson()).isNull();
assertThat(cleared.getCreateDate()).isEqualTo(call.getCreateDate());
assertThat(cleared.getModifyDate()).isAfter(call.getModifyDate());
}
/*******************************************************************************
** A scheduled run is inserted when it is claimed and updated, as a whole
** entity, when it ends.
*******************************************************************************/
@Test
void scheduledRunUpdateMovesModifyDate() throws Exception
{
AssistantStore store = store();
AssistantScheduledRun run = store.insert(new AssistantScheduledRun().withTaskId(1).withUserId(USER).withScheduledFor(Instant.parse("2026-02-03T04:05:06Z")).withStatus("RUNNING"));
assertThat(run.getCreateDate()).isNotNull();
assertThat(run.getModifyDate()).isNotNull();
pause();
store.update(run.withStatus("SUCCEEDED"));
AssistantScheduledRun ended = store.query(AssistantScheduledRun.TABLE_NAME, new QQueryFilter().withCriteria("id", QCriteriaOperator.EQUALS, run.getId()), AssistantScheduledRun.class).get(0);
assertThat(ended.getStatus()).isEqualTo("SUCCEEDED");
assertThat(ended.getCreateDate()).isEqualTo(run.getCreateDate());
assertThat(ended.getModifyDate()).isAfter(run.getModifyDate());
}
/*******************************************************************************
** The budget accounting row is a projection of a usage row: written with
** the usage, and moved when the usage is updated.
*******************************************************************************/
@Test
void budgetProjectionFollowsItsUsageRow() throws Exception
{
AssistantStore store = store();
AssistantUsage usage = store.insert(new AssistantUsage().withUserId(USER).withModel("m").withCostMicros(5L));
assertThat(usage.getCreateDate()).isNotNull();
assertThat(usage.getModifyDate()).isNotNull();
AssistantBudgetUsage projection = projectionOf(store, usage);
assertThat(projection.getCreateDate()).isNotNull();
assertThat(projection.getModifyDate()).isNotNull();
pause();
store.updateFields(AssistantUsage.TABLE_NAME, new QRecord().withValue("id", usage.getId()).withValue("costMicros", 9L));
AssistantUsage updated = store.query(AssistantUsage.TABLE_NAME, new QQueryFilter().withCriteria("id", QCriteriaOperator.EQUALS, usage.getId()), AssistantUsage.class).get(0);
assertThat(updated.getCreateDate()).isEqualTo(usage.getCreateDate());
assertThat(updated.getModifyDate()).isAfter(usage.getModifyDate());
AssistantBudgetUsage moved = projectionOf(store, usage);
assertThat(moved.getCostMicros()).isEqualTo(9L);
assertThat(moved.getCreateDate()).isEqualTo(projection.getCreateDate());
assertThat(moved.getModifyDate()).isAfter(projection.getModifyDate());
}
/*******************************************************************************
**
*******************************************************************************/
private static AssistantBudgetUsage projectionOf(AssistantStore store, AssistantUsage usage) throws Exception
{
List<AssistantBudgetUsage> rows = store.query(AssistantBudgetUsage.TABLE_NAME, new QQueryFilter().withCriteria("usageId", QCriteriaOperator.EQUALS, usage.getId()), AssistantBudgetUsage.class);
assertThat(rows).hasSize(1);
return (rows.get(0));
}
/*******************************************************************************
**
*******************************************************************************/
private static void assertEqualDates(Instant createDate, Instant modifyDate)
{
assertThat(createDate).isNotNull();
assertThat(modifyDate).isNotNull().isEqualTo(createDate);
}
/*******************************************************************************
** The store, under an owner's session.
*******************************************************************************/
private AssistantStore store()
{
QContext.init(qInstance, AssistantTestFixture.ownerSession(USER, config));
return (config.getRuntime().getStore());
}
/*******************************************************************************
** Let the clock move, so a later write is told from an earlier one.
*******************************************************************************/
private static void pause() throws InterruptedException
{
Thread.sleep(25);
}
}