feat(schema): every table has a create date and a modify date #118

Merged
james.maes merged 2 commits from feat/table-standard-dates into develop 2026-11-20 15:58:16 +00:00
16 changed files with 889 additions and 0 deletions

No files matched your search

+6
View File
@@ -1,5 +1,11 @@
# Changelog
## Unreleased - Every table has a create date and a modify date
- Nine tables lacked a standard date: `assistant_turn_event`, `assistant_tool_call`, `assistant_feedback`, `assistant_usage`, `assistant_data_audit`, `assistant_budget_event`, `assistant_client_error` and `assistant_pending_action_claim` had `createDate` but no `modifyDate`, and `assistant_budget_usage` had neither. All nine now declare both, `DATE_TIME`, not editable, set by QQQ (`CREATE_DATE` and `MODIFY_DATE` behaviors); every other table already had them.
- Every write the QBit makes goes through QQQ's insert and update actions, so an insert sets both dates and an update (an incremented daily usage counter, a decided pending action, a cleared tool result, a scheduled run that ends, the budget accounting row that follows a usage row) moves `modifyDate` and leaves `createDate`. A row of a table whose rows never change (`assistant_pending_action_claim`, `assistant_budget_event`) has `modifyDate` equal to `createDate` exactly: QQQ's two date behaviors read the clock separately, so a new insert customizer (`AssistantInsertOnlyDatesCustomizer`, registered by the QBit on those two tables) makes the two one instant. The other new `modifyDate` values are QQQ's own (turn events, feedback, client errors and the data audit are inserted once, and rewritten only when a transcript is redacted).
- `docs/schema-postgres.sql` has a new block at its end (additive; see UPGRADE.md): the columns are nullable and default to the time of the write, so the previous release keeps working against the same database while a host migrates.
## Unreleased - The assistant can read the files of records, where a host turns that on; a link in an answer is one the application gave
Design: `docs/file-contents-and-links.md`. **Off by default: an upgrade changes nothing until a host turns file reading on in code (see `UPGRADE.md`).**
+21
View File
@@ -1,5 +1,26 @@
# Upgrading qbit-assistant
## Every table has create and modify dates (Unreleased)
Nine tables gain columns: `modify_date` on `assistant_turn_event`, `assistant_tool_call`,
`assistant_feedback`, `assistant_usage`, `assistant_data_audit`, `assistant_budget_event`,
`assistant_client_error` and `assistant_pending_action_claim`, and `create_date` and `modify_date` on
`assistant_budget_usage` (the names an application gets when it derives snake_case backend names from
field names; otherwise `modifyDate` and `createDate`). A host that writes its own migrations runs
the block at the end of `docs/schema-postgres.sql` ("Table standard") **before** deploying this release:
- It is safe while the previous release is still running against the same database: the columns are
nullable and default to `CURRENT_TIMESTAMP`, so rows the previous release inserts are given a value.
On PostgreSQL 11 and later adding a column with that (stable) default does not rewrite the table.
- The `UPDATE` statements give existing rows the table's own creation time (`occurred_at` for
`assistant_budget_usage`). They touch every row: on a large table (`assistant_usage`,
`assistant_tool_call`, `assistant_turn_event`) run each in id ranges, and run them before the new
release starts writing, or they overwrite real `modify_date` values.
- `assistant_pending_action_claim` and `assistant_budget_event` rows are written once: their `modifyDate` equals their `createDate` and never moves.
Nothing else changes: no configuration, no code to change. A host whose database is made from
the reference DDL, or whose own migrations run the block, needs no customizer.
## The assistant can read the files of records (Unreleased)
**Nothing changes on upgrade: reading the content of files is off until a host turns it on in code.** Turned on, the assistant reads the content of a file a record holds, for a person who may open it. An answer keeps no address of the application that the application did not give. No schema change, no permission, no API change. Design: `docs/file-contents-and-links.md`.
+34
View File
@@ -777,3 +777,37 @@ CREATE TABLE IF NOT EXISTS assistant_pending_action_claim
CONSTRAINT assistant_pending_action_claim_action UNIQUE (pending_action_id)
);
CREATE INDEX IF NOT EXISTS assistant_pending_action_claim_user_id ON assistant_pending_action_claim (user_id);
-- Table standard: every table has create_date and modify_date. Eight tables had only
-- create_date and assistant_budget_usage had neither. Additive and safe while the
-- previous release is still running against the same database:
-- * the columns are nullable and default to the time of the write, so rows the previous
-- release inserts (it does not know the columns) are still given a value;
-- * on PostgreSQL 11 and later adding a column whose default is CURRENT_TIMESTAMP
-- (STABLE, evaluated once at ALTER time) does not rewrite the table and takes only a
-- short lock;
-- * the UPDATE statements give existing rows the table's own creation time (for
-- assistant_budget_usage, occurred_at, the usage row's creation time). They touch every
-- row, so on a large table (assistant_usage, assistant_tool_call, assistant_turn_event)
-- run each one in id ranges (AND id BETWEEN a AND b). Run them BEFORE the new release
-- starts writing: afterwards they would overwrite real modify_date values.
-- Safe to re-run.
ALTER TABLE assistant_turn_event ADD COLUMN IF NOT EXISTS modify_date TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP;
UPDATE assistant_turn_event SET modify_date = create_date WHERE create_date IS NOT NULL AND modify_date IS DISTINCT FROM create_date;
ALTER TABLE assistant_tool_call ADD COLUMN IF NOT EXISTS modify_date TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP;
UPDATE assistant_tool_call SET modify_date = create_date WHERE create_date IS NOT NULL AND modify_date IS DISTINCT FROM create_date;
ALTER TABLE assistant_feedback ADD COLUMN IF NOT EXISTS modify_date TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP;
UPDATE assistant_feedback SET modify_date = create_date WHERE create_date IS NOT NULL AND modify_date IS DISTINCT FROM create_date;
ALTER TABLE assistant_usage ADD COLUMN IF NOT EXISTS modify_date TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP;
UPDATE assistant_usage SET modify_date = create_date WHERE create_date IS NOT NULL AND modify_date IS DISTINCT FROM create_date;
ALTER TABLE assistant_data_audit ADD COLUMN IF NOT EXISTS modify_date TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP;
UPDATE assistant_data_audit SET modify_date = create_date WHERE create_date IS NOT NULL AND modify_date IS DISTINCT FROM create_date;
ALTER TABLE assistant_budget_event ADD COLUMN IF NOT EXISTS modify_date TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP;
UPDATE assistant_budget_event SET modify_date = create_date WHERE create_date IS NOT NULL AND modify_date IS DISTINCT FROM create_date;
ALTER TABLE assistant_client_error ADD COLUMN IF NOT EXISTS modify_date TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP;
UPDATE assistant_client_error SET modify_date = create_date WHERE create_date IS NOT NULL AND modify_date IS DISTINCT FROM create_date;
ALTER TABLE assistant_pending_action_claim ADD COLUMN IF NOT EXISTS modify_date TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP;
UPDATE assistant_pending_action_claim SET modify_date = create_date WHERE create_date IS NOT NULL AND modify_date IS DISTINCT FROM create_date;
ALTER TABLE assistant_budget_usage ADD COLUMN IF NOT EXISTS create_date TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP;
ALTER TABLE assistant_budget_usage ADD COLUMN IF NOT EXISTS modify_date TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP;
UPDATE assistant_budget_usage SET create_date = occurred_at, modify_date = occurred_at WHERE create_date IS DISTINCT FROM occurred_at OR modify_date IS DISTINCT FROM occurred_at;
@@ -0,0 +1,63 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant.metadata;
import java.util.List;
import com.kingsrook.qqq.backend.core.actions.customizers.AbstractPreInsertCustomizer;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** For a table whose rows are written once and never change (a decision claim,
** a budget event): modifyDate equals createDate, exactly. QQQ's two date
** behaviors each read the clock on their own, so left alone the two values of
** one insert differ by a few microseconds. This runs after the behaviors and
** before the row is stored, and makes the second the same instant as the first.
*******************************************************************************/
public class AssistantInsertOnlyDatesCustomizer extends AbstractPreInsertCustomizer
{
/*******************************************************************************
** After the date behaviors ran (they run before the unique-key checks).
*******************************************************************************/
@Override
public WhenToRun getWhenToRun()
{
return (WhenToRun.BEFORE_UNIQUE_KEY_CHECKS);
}
/*******************************************************************************
** modifyDate = createDate on every record.
*******************************************************************************/
@Override
public List<QRecord> apply(List<QRecord> records) throws QException
{
for(QRecord record : records)
{
if(record.getValue("createDate") != null)
{
record.setValue("modifyDate", record.getValue("createDate"));
}
}
return (records);
}
}
@@ -165,6 +165,11 @@ public class AssistantMetaDataBuilder
{
List<QTableMetaData> tables = new ArrayList<>();
//////////////////////////////////////////////////////////////////////////////////
// tables whose rows are written once and never change: modifyDate = createDate //
//////////////////////////////////////////////////////////////////////////////////
QCodeReference insertOnlyDates = new QCodeReference(AssistantInsertOnlyDatesCustomizer.class);
QCodeReference conversationCustomizer = new QCodeReference(AssistantConversationCustomizer.class);
tables.add(owned(base(AssistantConversation.TABLE_NAME, "Assistant Conversation", "forum", AssistantConversation.class), false)
.withRecordLabelFormat("%s")
@@ -261,6 +266,7 @@ public class AssistantMetaDataBuilder
///////////////////////////////////////////////////////////////////////
tables.add(readOnly(owned(base(AssistantPendingActionClaim.TABLE_NAME, "Assistant Pending Action Claim", "task_alt", AssistantPendingActionClaim.class), false))
.withUniqueKey(new UniqueKey("pendingActionId"))
.withCustomizer(TableCustomizers.PRE_INSERT_RECORD, insertOnlyDates)
.withRecordLabelFormat("%s #%s").withRecordLabelFields("decision", "pendingActionId"));
tables.add(readOnly(owned(base(AssistantDataAudit.TABLE_NAME, "Assistant Data Audit", "shield", AssistantDataAudit.class), false))
@@ -301,6 +307,7 @@ public class AssistantMetaDataBuilder
.withCustomizer(TableCustomizers.PRE_UPDATE_RECORD, budgetCustomizer));
tables.add(readOnly(base(AssistantBudgetEvent.TABLE_NAME, "Budget Event", "notifications", AssistantBudgetEvent.class))
.withoutCapabilities(Capability.TABLE_DELETE)
.withCustomizer(TableCustomizers.PRE_INSERT_RECORD, insertOnlyDates)
.withUniqueKey(new UniqueKey("budgetId", "scopeKey", "periodStart", "state")));
tables.forEach(AssistantRecordScreens::apply);
return (tables);
@@ -59,6 +59,9 @@ public class AssistantBudgetEvent extends QRecordEntity
@QField(isEditable = false)
private Instant createDate;
@QField(isEditable = false)
private Instant modifyDate;
/*******************************************************************************
** Default constructor.
@@ -356,4 +359,35 @@ public class AssistantBudgetEvent extends QRecordEntity
return (this);
}
/*******************************************************************************
** Getter for modifyDate
*******************************************************************************/
public Instant getModifyDate()
{
return (this.modifyDate);
}
/*******************************************************************************
** Setter for modifyDate
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Fluent setter for modifyDate
*******************************************************************************/
public AssistantBudgetEvent withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
}
@@ -66,6 +66,12 @@ public class AssistantBudgetUsage extends QRecordEntity
@QField()
private Long cacheWriteTokens;
@QField(isEditable = false)
private Instant createDate;
@QField(isEditable = false)
private Instant modifyDate;
/*******************************************************************************
** Default constructor.
@@ -425,4 +431,66 @@ public class AssistantBudgetUsage extends QRecordEntity
return (this);
}
/*******************************************************************************
** Getter for createDate
*******************************************************************************/
public Instant getCreateDate()
{
return (this.createDate);
}
/*******************************************************************************
** Setter for createDate
*******************************************************************************/
public void setCreateDate(Instant createDate)
{
this.createDate = createDate;
}
/*******************************************************************************
** Fluent setter for createDate
*******************************************************************************/
public AssistantBudgetUsage withCreateDate(Instant createDate)
{
this.createDate = createDate;
return (this);
}
/*******************************************************************************
** Getter for modifyDate
*******************************************************************************/
public Instant getModifyDate()
{
return (this.modifyDate);
}
/*******************************************************************************
** Setter for modifyDate
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Fluent setter for modifyDate
*******************************************************************************/
public AssistantBudgetUsage withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
}
@@ -85,6 +85,9 @@ public class AssistantClientError extends QRecordEntity
@QField(isEditable = false)
private Instant createDate;
@QField(isEditable = false)
private Instant modifyDate;
/*******************************************************************************
@@ -631,4 +634,35 @@ public class AssistantClientError extends QRecordEntity
return (this);
}
/*******************************************************************************
** Getter for modifyDate
*******************************************************************************/
public Instant getModifyDate()
{
return (this.modifyDate);
}
/*******************************************************************************
** Setter for modifyDate
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Fluent setter for modifyDate
*******************************************************************************/
public AssistantClientError withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
}
@@ -72,6 +72,9 @@ public class AssistantDataAudit extends QRecordEntity
@QField(isEditable = false)
private Instant createDate;
@QField(isEditable = false)
private Instant modifyDate;
/*******************************************************************************
@@ -477,4 +480,33 @@ public class AssistantDataAudit extends QRecordEntity
/*******************************************************************************
** Getter for modifyDate
*******************************************************************************/
public Instant getModifyDate()
{
return (this.modifyDate);
}
/*******************************************************************************
** Setter for modifyDate
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Fluent setter for modifyDate
*******************************************************************************/
public AssistantDataAudit withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
}
@@ -61,6 +61,9 @@ public class AssistantFeedback extends QRecordEntity
@QField(isEditable = false)
private Instant createDate;
@QField(isEditable = false)
private Instant modifyDate;
/*******************************************************************************
@@ -328,4 +331,35 @@ public class AssistantFeedback extends QRecordEntity
return (this);
}
/*******************************************************************************
** Getter for modifyDate
*******************************************************************************/
public Instant getModifyDate()
{
return (this.modifyDate);
}
/*******************************************************************************
** Setter for modifyDate
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Fluent setter for modifyDate
*******************************************************************************/
public AssistantFeedback withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
}
@@ -50,6 +50,9 @@ public class AssistantPendingActionClaim extends QRecordEntity
@QField(isEditable = false)
private Instant createDate;
@QField(isEditable = false)
private Instant modifyDate;
/*******************************************************************************
@@ -224,4 +227,35 @@ public class AssistantPendingActionClaim extends QRecordEntity
return (this);
}
/*******************************************************************************
** Getter for modifyDate
*******************************************************************************/
public Instant getModifyDate()
{
return (this.modifyDate);
}
/*******************************************************************************
** Setter for modifyDate
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Fluent setter for modifyDate
*******************************************************************************/
public AssistantPendingActionClaim withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
}
@@ -109,6 +109,9 @@ public class AssistantToolCall extends QRecordEntity
@QField(isEditable = false)
private Instant createDate;
@QField(isEditable = false)
private Instant modifyDate;
/*******************************************************************************
@@ -837,4 +840,36 @@ public class AssistantToolCall extends QRecordEntity
this.clientChannel = clientChannel;
return (this);
}
/*******************************************************************************
** Getter for modifyDate
*******************************************************************************/
public Instant getModifyDate()
{
return (this.modifyDate);
}
/*******************************************************************************
** Setter for modifyDate
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Fluent setter for modifyDate
*******************************************************************************/
public AssistantToolCall withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
}
@@ -59,6 +59,9 @@ public class AssistantTurnEvent extends QRecordEntity
@QField(isEditable = false)
private Instant createDate;
@QField(isEditable = false)
private Instant modifyDate;
/*******************************************************************************
@@ -295,4 +298,35 @@ public class AssistantTurnEvent extends QRecordEntity
return (this);
}
/*******************************************************************************
** Getter for modifyDate
*******************************************************************************/
public Instant getModifyDate()
{
return (this.modifyDate);
}
/*******************************************************************************
** Setter for modifyDate
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Fluent setter for modifyDate
*******************************************************************************/
public AssistantTurnEvent withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
}
@@ -90,6 +90,9 @@ public class AssistantUsage extends QRecordEntity
@QField(isEditable = false)
private Instant createDate;
@QField(isEditable = false)
private Instant modifyDate;
/*******************************************************************************
@@ -698,4 +701,35 @@ public class AssistantUsage extends QRecordEntity
return (this);
}
/*******************************************************************************
** Getter for modifyDate
*******************************************************************************/
public Instant getModifyDate()
{
return (this.modifyDate);
}
/*******************************************************************************
** Setter for modifyDate
*******************************************************************************/
public void setModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
}
/*******************************************************************************
** Fluent setter for modifyDate
*******************************************************************************/
public AssistantUsage withModifyDate(Instant modifyDate)
{
this.modifyDate = modifyDate;
return (this);
}
}
@@ -0,0 +1,90 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.Statement;
import java.util.HashSet;
import java.util.Set;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import com.kingsrook.qbits.assistant.metadata.AssistantMetaDataBuilder;
import com.kingsrook.qbits.assistant.testing.BaseAssistantTest;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** The reference DDL (docs/schema-postgres.sql) gives every table the QBit
** declares the columns create_date and modify_date, for a database made from
** it (the creates, then the later additions, as the file says). The
** PostgreSQL-only parts are run by SchemaForeignKeyPostgresTest.
*******************************************************************************/
class TableStandardSchemaTest extends BaseAssistantTest
{
/*******************************************************************************
** Every declared table has both columns after the reference DDL ran.
*******************************************************************************/
@Test
void everyDeclaredTableHasBothDateColumns() throws Exception
{
Set<String> declared = new HashSet<>();
for(QTableMetaData table : new AssistantMetaDataBuilder(config).buildTables())
{
declared.add(table.getName());
}
try(Connection connection = DriverManager.getConnection("jdbc:h2:mem:tablestandard" + System.nanoTime() + ";MODE=PostgreSQL;DEFAULT_NULL_ORDERING=HIGH"))
{
String ddl = Files.readString(Path.of("..", "docs", "schema-postgres.sql"), StandardCharsets.UTF_8);
Pattern pattern = Pattern.compile("^(CREATE TABLE|ALTER TABLE)( IF NOT EXISTS)? (assistant_[a-z_]+)\\b(.*)", Pattern.DOTALL);
try(Statement statement = connection.createStatement())
{
for(String sql : AssistantDefaultsChangelogTest.statements(ddl))
{
Matcher m = pattern.matcher(sql);
if(m.find() && (m.group(1).startsWith("CREATE") || m.group(4).strip().startsWith("ADD COLUMN")))
{
statement.execute(sql.replace("TIMESTAMPTZ", "TIMESTAMP WITH TIME ZONE"));
}
}
}
for(String table : declared)
{
Set<String> columns = new HashSet<>();
try(Statement statement = connection.createStatement(); ResultSet rs = statement.executeQuery("SELECT LOWER(column_name) FROM information_schema.columns WHERE LOWER(table_name) = '" + table + "'"))
{
while(rs.next())
{
columns.add(rs.getString(1));
}
}
assertThat(columns).as(table + " columns").contains("create_date", "modify_date");
}
}
}
}
@@ -0,0 +1,329 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.assistant.persistence;
import java.math.BigDecimal;
import java.time.Instant;
import java.time.LocalDate;
import java.util.List;
import com.kingsrook.qbits.assistant.metadata.AssistantMetaDataBuilder;
import com.kingsrook.qbits.assistant.model.AssistantBudgetEvent;
import com.kingsrook.qbits.assistant.model.AssistantBudgetUsage;
import com.kingsrook.qbits.assistant.model.AssistantClientError;
import com.kingsrook.qbits.assistant.model.AssistantDataAudit;
import com.kingsrook.qbits.assistant.model.AssistantFeedback;
import com.kingsrook.qbits.assistant.model.AssistantPendingAction;
import com.kingsrook.qbits.assistant.model.AssistantPendingActionClaim;
import com.kingsrook.qbits.assistant.model.AssistantScheduledRun;
import com.kingsrook.qbits.assistant.model.AssistantToolCall;
import com.kingsrook.qbits.assistant.model.AssistantTurnEvent;
import com.kingsrook.qbits.assistant.model.AssistantUsage;
import com.kingsrook.qbits.assistant.model.AssistantUsageDaily;
import com.kingsrook.qbits.assistant.spi.ModelUsage;
import com.kingsrook.qbits.assistant.testing.AssistantTestFixture;
import com.kingsrook.qbits.assistant.testing.BaseAssistantTest;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.fields.DynamicDefaultValueBehavior;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** The table standard: every table the QBit declares has a createDate and a
** modifyDate (date-time, not editable, set by QQQ), and every write path of
** the store sets them: an insert sets both, an update moves modifyDate and
** leaves createDate. A row of a table that is only ever inserted (a log, an
** audit row, a claim) has a modifyDate equal to its createDate, always.
*******************************************************************************/
class TableStandardDatesTest extends BaseAssistantTest
{
private static final String USER = AssistantTestFixture.ADMIN;
/*******************************************************************************
** Every table the producer declares, found from the metadata and not listed
** by hand, has both dates as the standard says.
*******************************************************************************/
@Test
void everyDeclaredTableHasBothDates() throws Exception
{
List<QTableMetaData> declared = new AssistantMetaDataBuilder(config).buildTables();
assertThat(declared).hasSizeGreaterThan(20);
for(QTableMetaData declaredTable : declared)
{
QTableMetaData table = qInstance.getTable(declaredTable.getName());
assertThat(table).as(declaredTable.getName() + " is in the instance").isNotNull();
assertDate(table, "createDate", DynamicDefaultValueBehavior.CREATE_DATE);
assertDate(table, "modifyDate", DynamicDefaultValueBehavior.MODIFY_DATE);
}
}
/*******************************************************************************
**
*******************************************************************************/
private static void assertDate(QTableMetaData table, String name, DynamicDefaultValueBehavior behavior)
{
QFieldMetaData field = table.getFields().get(name);
assertThat(field).as(table.getName() + "." + name).isNotNull();
assertThat(field.getType()).as(table.getName() + "." + name + " type").isEqualTo(QFieldType.DATE_TIME);
assertThat(field.getIsEditable()).as(table.getName() + "." + name + " editable").isFalse();
assertThat(field.getBehaviors()).as(table.getName() + "." + name + " behaviors").contains(behavior);
}
/*******************************************************************************
** A usage counter row that is incremented moves its modifyDate each time.
*******************************************************************************/
@Test
void incrementedUsageCounterMovesModifyDate() throws Exception
{
AssistantStore store = store();
LocalDate day = LocalDate.of(2026, 1, 15);
ModelUsage usage = new ModelUsage();
store.addUsage(USER, day, usage, 10L, true);
AssistantUsageDaily first = store.getUserUsage(USER, day);
assertThat(first.getCreateDate()).isNotNull();
assertThat(first.getModifyDate()).isNotNull();
pause();
store.addUsage(USER, day, usage, 10L, true);
AssistantUsageDaily second = store.getUserUsage(USER, day);
assertThat(second.getTurnCount()).isEqualTo(2);
assertThat(second.getCreateDate()).isEqualTo(first.getCreateDate());
assertThat(second.getModifyDate()).isAfter(first.getModifyDate());
}
/*******************************************************************************
** Rows written once (a decision claim, a budget event) have modifyDate equal
** to createDate, exactly: QQQ's two behaviors read the clock separately, so
** the table's insert customizer makes them one instant. Rows of the tables
** that are inserted and, only when a transcript is redacted, rewritten, have
** both dates set, modifyDate not before createDate.
*******************************************************************************/
@Test
void insertOnlyRowsHaveEqualDates() throws Exception
{
AssistantStore store = store();
Instant now = Instant.parse("2026-02-03T04:05:06Z");
for(int i = 0; i < 20; i++)
{
AssistantBudgetEvent budgetEvent = store.insert(new AssistantBudgetEvent().withBudgetId(1).withScopeKey("k" + i).withPeriodStart(now).withState("WARN")
.withUsedValue(BigDecimal.ONE).withLimitValue(BigDecimal.TEN));
assertEqualDates(budgetEvent.getCreateDate(), budgetEvent.getModifyDate());
AssistantPendingActionClaim claim = store.insert(new AssistantPendingActionClaim().withPendingActionId(100 + i).withUserId(USER).withDecision("EXECUTED"));
assertEqualDates(claim.getCreateDate(), claim.getModifyDate());
}
AssistantTurnEvent event = store.insert(new AssistantTurnEvent().withTurnId(1).withUserId(USER).withSeq(1).withEventType("text.delta").withDataJson("{}"));
assertBothSet(event.getCreateDate(), event.getModifyDate());
AssistantFeedback feedback = store.insert(new AssistantFeedback().withUserId(USER).withRating(1));
assertBothSet(feedback.getCreateDate(), feedback.getModifyDate());
AssistantClientError error = store.insert(new AssistantClientError().withUserId(USER).withScope("test").withMessage("m"));
assertBothSet(error.getCreateDate(), error.getModifyDate());
AssistantDataAudit audit = store.insert(new AssistantDataAudit().withUserId(USER).withDecision("ALLOWED"));
assertBothSet(audit.getCreateDate(), audit.getModifyDate());
}
/*******************************************************************************
**
*******************************************************************************/
private static void assertBothSet(Instant createDate, Instant modifyDate)
{
assertThat(createDate).isNotNull();
assertThat(modifyDate).isNotNull().isAfterOrEqualTo(createDate);
}
/*******************************************************************************
** A decision of a pending action: the action moves its modifyDate, and the
** claim row is written once with equal dates.
*******************************************************************************/
@Test
void pendingActionDecisionMovesTheActionAndInsertsTheClaim() throws Exception
{
AssistantStore store = store();
Instant now = Instant.parse("2026-02-03T04:05:06Z");
AssistantPendingAction action = store.insert(new AssistantPendingAction().withUserId(USER).withToolName("update_record").withStatus(AssistantPendingAction.STATUS_PENDING));
assertThat(action.getCreateDate()).isNotNull();
assertThat(action.getModifyDate()).isNotNull();
pause();
assertThat(store.decidePendingAction(USER, action.getId(), AssistantPendingAction.STATUS_RUNNING, null, now)).isTrue();
AssistantPendingAction decided = store.getPendingAction(USER, action.getId());
assertThat(decided.getStatus()).isEqualTo(AssistantPendingAction.STATUS_RUNNING);
assertThat(decided.getCreateDate()).isEqualTo(action.getCreateDate());
assertThat(decided.getModifyDate()).isAfter(action.getModifyDate());
List<AssistantPendingActionClaim> claims = store.query(AssistantPendingActionClaim.TABLE_NAME,
new QQueryFilter().withCriteria("pendingActionId", QCriteriaOperator.EQUALS, action.getId()), AssistantPendingActionClaim.class);
assertThat(claims).hasSize(1);
assertEqualDates(claims.get(0).getCreateDate(), claims.get(0).getModifyDate());
pause();
store.finishPendingAction(action.getId(), AssistantPendingAction.STATUS_EXECUTED, 200, "done");
AssistantPendingAction finished = store.getPendingAction(USER, action.getId());
assertThat(finished.getCreateDate()).isEqualTo(action.getCreateDate());
assertThat(finished.getModifyDate()).isAfter(decided.getModifyDate());
}
/*******************************************************************************
** Clearing a tool call's result (the retention step) is an update.
*******************************************************************************/
@Test
void clearingAToolResultMovesModifyDate() throws Exception
{
AssistantStore store = store();
AssistantToolCall call = store.insert(new AssistantToolCall().withUserId(USER).withToolName("query_records").withResultJson("{}"));
assertThat(call.getCreateDate()).isNotNull();
assertThat(call.getModifyDate()).isNotNull();
pause();
assertThat(store.clearToolResultsBefore(Instant.now().plusSeconds(60), 10)).isEqualTo(1);
AssistantToolCall cleared = store.query(AssistantToolCall.TABLE_NAME, new QQueryFilter().withCriteria("id", QCriteriaOperator.EQUALS, call.getId()), AssistantToolCall.class).get(0);
assertThat(cleared.getResultJson()).isNull();
assertThat(cleared.getCreateDate()).isEqualTo(call.getCreateDate());
assertThat(cleared.getModifyDate()).isAfter(call.getModifyDate());
}
/*******************************************************************************
** A scheduled run is inserted when it is claimed and updated, as a whole
** entity, when it ends.
*******************************************************************************/
@Test
void scheduledRunUpdateMovesModifyDate() throws Exception
{
AssistantStore store = store();
AssistantScheduledRun run = store.insert(new AssistantScheduledRun().withTaskId(1).withUserId(USER).withScheduledFor(Instant.parse("2026-02-03T04:05:06Z")).withStatus("RUNNING"));
assertThat(run.getCreateDate()).isNotNull();
assertThat(run.getModifyDate()).isNotNull();
pause();
store.update(run.withStatus("SUCCEEDED"));
AssistantScheduledRun ended = store.query(AssistantScheduledRun.TABLE_NAME, new QQueryFilter().withCriteria("id", QCriteriaOperator.EQUALS, run.getId()), AssistantScheduledRun.class).get(0);
assertThat(ended.getStatus()).isEqualTo("SUCCEEDED");
assertThat(ended.getCreateDate()).isEqualTo(run.getCreateDate());
assertThat(ended.getModifyDate()).isAfter(run.getModifyDate());
}
/*******************************************************************************
** The budget accounting row is a projection of a usage row: written with
** the usage, and moved when the usage is updated.
*******************************************************************************/
@Test
void budgetProjectionFollowsItsUsageRow() throws Exception
{
AssistantStore store = store();
AssistantUsage usage = store.insert(new AssistantUsage().withUserId(USER).withModel("m").withCostMicros(5L));
assertThat(usage.getCreateDate()).isNotNull();
assertThat(usage.getModifyDate()).isNotNull();
AssistantBudgetUsage projection = projectionOf(store, usage);
assertThat(projection.getCreateDate()).isNotNull();
assertThat(projection.getModifyDate()).isNotNull();
pause();
store.updateFields(AssistantUsage.TABLE_NAME, new QRecord().withValue("id", usage.getId()).withValue("costMicros", 9L));
AssistantUsage updated = store.query(AssistantUsage.TABLE_NAME, new QQueryFilter().withCriteria("id", QCriteriaOperator.EQUALS, usage.getId()), AssistantUsage.class).get(0);
assertThat(updated.getCreateDate()).isEqualTo(usage.getCreateDate());
assertThat(updated.getModifyDate()).isAfter(usage.getModifyDate());
AssistantBudgetUsage moved = projectionOf(store, usage);
assertThat(moved.getCostMicros()).isEqualTo(9L);
assertThat(moved.getCreateDate()).isEqualTo(projection.getCreateDate());
assertThat(moved.getModifyDate()).isAfter(projection.getModifyDate());
}
/*******************************************************************************
**
*******************************************************************************/
private static AssistantBudgetUsage projectionOf(AssistantStore store, AssistantUsage usage) throws Exception
{
List<AssistantBudgetUsage> rows = store.query(AssistantBudgetUsage.TABLE_NAME, new QQueryFilter().withCriteria("usageId", QCriteriaOperator.EQUALS, usage.getId()), AssistantBudgetUsage.class);
assertThat(rows).hasSize(1);
return (rows.get(0));
}
/*******************************************************************************
**
*******************************************************************************/
private static void assertEqualDates(Instant createDate, Instant modifyDate)
{
assertThat(createDate).isNotNull();
assertThat(modifyDate).isNotNull().isEqualTo(createDate);
}
/*******************************************************************************
** The store, under an owner's session.
*******************************************************************************/
private AssistantStore store()
{
QContext.init(qInstance, AssistantTestFixture.ownerSession(USER, config));
return (config.getRuntime().getStore());
}
/*******************************************************************************
** Let the clock move, so a later write is told from an earlier one.
*******************************************************************************/
private static void pause() throws InterruptedException
{
Thread.sleep(25);
}
}