diff --git a/CHANGELOG.md b/CHANGELOG.md index 9f7f1a4..e483aa8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,18 @@ # Changelog +## Unreleased - Fixed: a setting saved from its record screen is stored, and is held to its rules as it will be stored + +**Nothing for a host to change.** + +- **A budget could not be edited on its record screen.** The screen saves through the UI API's table update route (`PATCH /qqq/v1/table/{tableName}/{primaryKey}`): only the fields that were changed, and the version that was read. The save was answered with 400 ("Invalid scopeType., Invalid period., Invalid metric. ...") and nothing was stored. The record's primary key comes from the request's path, so it reaches a table customizer as text; `AssistantBudgetCustomizer` looked the stored row up by comparing that text with the stored integer key, did not find it, and judged the one changed field as if it were a whole new budget. +- **The same mistake was in every settings customizer, and is fixed in all of them.** A budget override, a data access row, a provider, a knowledge source and a scheduled task were refused the same way (400, nothing stored). A profile and a skill were stored, with the defaults of a new row written over what the save had not sent: a profile switched off was switched on and its routing mode set to OFF, a skill switched off was switched on. +- **The fix.** The customizers no longer pair a record with its stored row themselves (`ArchitectureTest` keeps it so). One class does it (`StoredRecords`): the key of the record being saved and the key of each stored row are both read as the type of the table's key field before they are compared, so the pairing holds whether the key arrives as text or as a number. The values being saved are laid over the stored row the same way, each read as the type of its field. A value that cannot be read as that type (text in a number field) is refused with 400 and a message that names the field and does not repeat the value. +- **The rules are judged on the record as it will be stored, for a save of one field exactly as for a save of all of them:** the host's ceiling on a cost limit (`withBudgetCostLimitCeilingUsd`; a save that only changes a token limit into a cost limit is over the ceiling with the limit already stored), required fields (a save cannot blank the name, the limit, the period, the metric or the scope), the allowed periods, scopes, metrics and actions, scope values, the downgrade model, the thresholds, and for an override the reason, the expiry and "exactly one of extra value or unlimited until". A refusal has the same words for a partial save as for a whole one. +- **A change is found by comparing values of the same type.** A scheduled task keeps the approval of its writes and its next run when a save sends a value it already has as text (`enabled=true`), and loses the approval only when its prompt, its user, its credential, its profile or its write mode really changed. A knowledge source keeps its vectors when a save sends the chunk size it already has. +- **Nothing a save could not set before can be set now.** The key in the path is the record that is saved, whatever the body says; who granted an override is the session's on insert and is not taken from an update; the approval of a scheduled task's writes is not taken from a save. +- **Field-by-field audit** (where a host turns it on for budgets) records the one field that was saved, with its old and its new value. The stored row a rule is judged on is never what is saved. +- Tests, all through the real route against the fixture application (`RecordScreen` sends what the screen sends: a multipart `PATCH` of the changed fields with `expectedVersion`): `BudgetEditOverHttpTest` (the limit, the enabled flag, the period, the name, the notes and a threshold, each saved alone with every other field kept; a second save from the stale version is refused with 409 and stores nothing; the ceiling for a partial save, a whole save and a new budget; twenty refusals of a partial save, each storing nothing; a blanked setting gets its default; what the server decides; an override; the audit; a person without `assistant.admin`), `RecordEditsOverHttpTest` (profile, default profile, skill, data access), and one test each in `AssistantSettingsTest` (provider and model), `RagTest` (knowledge source) and `ScheduledTasksTest` (scheduled task). Fourteen of the sixteen fail without the fix. `StoredRecordsTest` pairs a record with its stored row for an integer, a long and a text key, with the key as text and as a number. + ## Unreleased - Every table has a create date and a modify date - Nine tables lacked a standard date: `assistant_turn_event`, `assistant_tool_call`, `assistant_feedback`, `assistant_usage`, `assistant_data_audit`, `assistant_budget_event`, `assistant_client_error` and `assistant_pending_action_claim` had `createDate` but no `modifyDate`, and `assistant_budget_usage` had neither. All nine now declare both, `DATE_TIME`, not editable, set by QQQ (`CREATE_DATE` and `MODIFY_DATE` behaviors); every other table already had them. diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantBudgetCustomizer.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantBudgetCustomizer.java index fb51c9a..c8885da 100644 --- a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantBudgetCustomizer.java +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantBudgetCustomizer.java @@ -32,9 +32,7 @@ import com.kingsrook.qqq.backend.core.model.data.QRecord; import com.kingsrook.qqq.backend.core.model.statusmessages.BadInputStatusMessage; import java.io.Serializable; import java.math.BigDecimal; -import java.util.LinkedHashMap; import java.util.List; -import java.util.Map; import java.util.Optional; @@ -52,12 +50,10 @@ public class AssistantBudgetCustomizer implements TableCustomizerInterface @Override public List preInsertOrUpdate(AbstractActionInput input, List records, boolean isPreview, Optional> oldRecordList) throws QException { - Optional> old = oldRecordListToMap("id", oldRecordList); + StoredRecords stored = StoredRecords.of(input, oldRecordList); for(QRecord record : records) { - QRecord merged = new QRecord(); - old.map(m -> m.get(record.getValue("id"))).ifPresent(o -> merged.setValues(new LinkedHashMap<>(o.getValues()))); - record.getValues().forEach(merged::setValue); + QRecord merged = stored.merged(record); if(merged.getValue("budgetId") != null) { checkOverride(record, merged, input instanceof InsertInput); diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantDataAccessCustomizer.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantDataAccessCustomizer.java index 0623cce..16c6a27 100644 --- a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantDataAccessCustomizer.java +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantDataAccessCustomizer.java @@ -17,10 +17,7 @@ package com.kingsrook.qbits.assistant.metadata; -import java.io.Serializable; -import java.util.LinkedHashMap; import java.util.List; -import java.util.Map; import java.util.Optional; import com.kingsrook.qbits.assistant.AssistantRuntime; import com.kingsrook.qbits.assistant.access.DataAccessMode; @@ -53,13 +50,11 @@ public class AssistantDataAccessCustomizer implements TableCustomizerInterface @Override public List preInsertOrUpdate(AbstractActionInput input, List records, boolean isPreview, Optional> oldRecordList) throws QException { - QInstance qInstance = QContext.getQInstance(); - Optional> old = oldRecordListToMap("id", oldRecordList); + QInstance qInstance = QContext.getQInstance(); + StoredRecords stored = StoredRecords.of(input, oldRecordList); for(QRecord record : records) { - QRecord merged = new QRecord(); - old.map(m -> m.get(record.getValue("id"))).ifPresent(o -> merged.setValues(new LinkedHashMap<>(o.getValues()))); - record.getValues().forEach(merged::setValue); + QRecord merged = stored.merged(record); String type = merged.getValueString("objectType"); String name = merged.getValueString("objectName"); diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantProfileCustomizer.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantProfileCustomizer.java index f9cd464..24ad0cd 100644 --- a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantProfileCustomizer.java +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantProfileCustomizer.java @@ -17,10 +17,7 @@ package com.kingsrook.qbits.assistant.metadata; -import java.io.Serializable; -import java.util.LinkedHashMap; import java.util.List; -import java.util.Map; import java.util.Optional; import com.kingsrook.qbits.assistant.AssistantRuntime; import com.kingsrook.qbits.assistant.engine.ProfileService; @@ -57,13 +54,11 @@ public class AssistantProfileCustomizer implements TableCustomizerInterface @Override public List preInsertOrUpdate(AbstractActionInput input, List records, boolean isPreview, Optional> oldRecordList) throws QException { - String table = tableName(input); - Optional> old = oldRecordListToMap("id", oldRecordList); + String table = tableName(input); + StoredRecords stored = StoredRecords.of(input, oldRecordList); for(QRecord record : records) { - QRecord merged = new QRecord(); - old.map(m -> m.get(record.getValue("id"))).ifPresent(o -> merged.setValues(new LinkedHashMap<>(o.getValues()))); - record.getValues().forEach(merged::setValue); + QRecord merged = stored.merged(record); if(AssistantProfile.TABLE_NAME.equals(table)) { @@ -139,7 +134,7 @@ public class AssistantProfileCustomizer implements TableCustomizerInterface if(Boolean.TRUE.equals(record.getValueBoolean("isDefault")) && record.getValue("id") != null && record.getErrors().isEmpty()) { for(AssistantProfile other : runtime.getStore().query(AssistantProfile.TABLE_NAME, - new QQueryFilter().withCriteria("isDefault", QCriteriaOperator.EQUALS, true).withCriteria("id", QCriteriaOperator.NOT_EQUALS, record.getValue("id")), AssistantProfile.class)) + new QQueryFilter().withCriteria("isDefault", QCriteriaOperator.EQUALS, true).withCriteria("id", QCriteriaOperator.NOT_EQUALS, record.getValueInteger("id")), AssistantProfile.class)) { runtime.getStore().updateFields(AssistantProfile.TABLE_NAME, new QRecord().withValue("id", other.getId()).withValue("isDefault", false)); } diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantRagSourceCustomizer.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantRagSourceCustomizer.java index ccf9caf..acf4b6b 100644 --- a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantRagSourceCustomizer.java +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantRagSourceCustomizer.java @@ -18,10 +18,8 @@ package com.kingsrook.qbits.assistant.metadata; import java.io.Serializable; -import java.util.LinkedHashMap; import java.util.List; import java.util.Map; -import java.util.Objects; import java.util.Optional; import com.kingsrook.qbits.assistant.AssistantRuntime; import com.kingsrook.qbits.assistant.model.AssistantRagSource; @@ -50,7 +48,7 @@ public class AssistantRagSourceCustomizer implements TableCustomizerInterface @Override public List preInsertOrUpdate(AbstractActionInput input, List records, boolean isPreview, Optional> oldRecordList) throws QException { - Optional> old = oldRecordListToMap("id", oldRecordList); + StoredRecords stored = StoredRecords.of(input, oldRecordList); AssistantRuntime runtime = AssistantRuntime.of(QContext.getQInstance()); for(QRecord record : records) { @@ -58,9 +56,7 @@ public class AssistantRagSourceCustomizer implements TableCustomizerInterface { continue; } - QRecord merged = new QRecord(); - old.map(m -> m.get(record.getValue("id"))).ifPresent(o -> merged.setValues(new LinkedHashMap<>(o.getValues()))); - record.getValues().forEach(merged::setValue); + QRecord merged = stored.merged(record); for(var entry : Map.of("enabled", true, "chunkChars", 1500, "chunkOverlapChars", 200).entrySet()) { if(merged.getValue(entry.getKey()) == null) @@ -88,14 +84,14 @@ public class AssistantRagSourceCustomizer implements TableCustomizerInterface @Override public List postInsertOrUpdate(AbstractActionInput input, List records, Optional> oldRecordList) throws QException { - Optional> old = oldRecordListToMap("id", oldRecordList); + StoredRecords stored = StoredRecords.of(input, oldRecordList); AssistantRuntime runtime = AssistantRuntime.of(QContext.getQInstance()); - if(runtime != null && old.isPresent()) + if(runtime != null && oldRecordList.isPresent()) { for(QRecord record : records) { - QRecord previous = old.get().get(record.getValue("id")); - if(previous != null && SEMANTIC_FIELDS.stream().anyMatch(name -> record.getValues().containsKey(name) && !Objects.equals(previous.getValue(name), record.getValue(name)))) + QRecord previous = stored.of(record); + if(previous != null && SEMANTIC_FIELDS.stream().anyMatch(name -> record.getValues().containsKey(name) && stored.differs(previous, record, name))) { runtime.getRagService().clear(new AssistantRagSource(previous)); } diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantScheduledTaskCustomizer.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantScheduledTaskCustomizer.java index 95fce21..303892a 100644 --- a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantScheduledTaskCustomizer.java +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantScheduledTaskCustomizer.java @@ -22,10 +22,8 @@ package com.kingsrook.qbits.assistant.metadata; import java.io.Serializable; -import java.util.LinkedHashMap; import java.util.List; import java.util.Map; -import java.util.Objects; import java.util.Optional; import com.kingsrook.qbits.assistant.AssistantRuntime; import com.kingsrook.qbits.assistant.model.AssistantScheduledTask; @@ -52,7 +50,7 @@ public class AssistantScheduledTaskCustomizer implements TableCustomizerInterfac @Override public List preInsertOrUpdate(AbstractActionInput input, List records, boolean isPreview, Optional> oldRecordList) throws QException { - Optional> old = oldRecordListToMap("id", oldRecordList); + StoredRecords stored = StoredRecords.of(input, oldRecordList); AssistantRuntime runtime = AssistantRuntime.of(QContext.getQInstance()); for(QRecord record : records) { @@ -62,14 +60,12 @@ public class AssistantScheduledTaskCustomizer implements TableCustomizerInterfac { continue; } - QRecord previous = old.map(rows -> rows.get(record.getValue("id"))).orElse(null); + QRecord previous = stored.of(record); for(String field : List.of("writeModeApprovedBy", "writeModeApprovedAt", "writeModeApprovalHash", "nextRunAt", "lastRunAt", "lastRunStatus", "lastRunError")) { record.getValues().remove(field); } - QRecord merged = new QRecord(); - merged.setValues(previous == null ? new LinkedHashMap<>() : new LinkedHashMap<>(previous.getValues())); - record.getValues().forEach(merged::setValue); + QRecord merged = stored.merged(record); for(var entry : Map.of("enabled", false, "timezone", "UTC", "outputMode", "RECORD", "writeModeRequested", false).entrySet()) { if(merged.getValue(entry.getKey()) == null) @@ -78,7 +74,7 @@ public class AssistantScheduledTaskCustomizer implements TableCustomizerInterfac record.setValue(entry.getKey(), entry.getValue()); } } - if(previous == null || ScheduledTasks.APPROVAL_FIELDS.stream().anyMatch(field -> !Objects.equals(previous.getValue(field), merged.getValue(field)))) + if(previous == null || ScheduledTasks.APPROVAL_FIELDS.stream().anyMatch(field -> stored.differs(previous, merged, field))) { record.setValue("writeModeApprovedBy", null); record.setValue("writeModeApprovedAt", null); @@ -101,7 +97,7 @@ public class AssistantScheduledTaskCustomizer implements TableCustomizerInterfac throw (new QException("The per-run cost cap cannot be negative.")); } var next = ScheduledTasks.next(task, runtime.getClock().instant()); - if(previous == null || List.of("enabled", "cronExpression", "timezone").stream().anyMatch(field -> !Objects.equals(previous.getValue(field), merged.getValue(field)))) + if(previous == null || List.of("enabled", "cronExpression", "timezone").stream().anyMatch(field -> stored.differs(previous, merged, field))) { record.setValue("nextRunAt", Boolean.TRUE.equals(task.getEnabled()) ? next : null); } diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantSettingsCustomizer.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantSettingsCustomizer.java index b69cd0d..e2d10fa 100644 --- a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantSettingsCustomizer.java +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/AssistantSettingsCustomizer.java @@ -107,7 +107,7 @@ public class AssistantSettingsCustomizer implements TableCustomizerInterface } } } - validateFallbacks(records, oldRecordList); + validateFallbacks(input, records, oldRecordList); return (records); } if(service == null || !AssistantProvider.TABLE_NAME.equals(tableName(input))) @@ -115,7 +115,7 @@ public class AssistantSettingsCustomizer implements TableCustomizerInterface return (records); } - Optional> old = oldRecordListToMap("id", oldRecordList); + StoredRecords stored = StoredRecords.of(input, oldRecordList); for(QRecord record : records) { record.removeValue(ProviderService.KEY_STATUS_FIELD); @@ -124,9 +124,7 @@ public class AssistantSettingsCustomizer implements TableCustomizerInterface continue; } - QRecord merged = new QRecord(); - old.map(m -> m.get(record.getValue("id"))).ifPresent(o -> merged.setValues(new java.util.LinkedHashMap<>(o.getValues()))); - record.getValues().forEach(merged::setValue); + QRecord merged = stored.merged(record); AssistantProvider provider = new AssistantProvider(merged); if(merged.getValue("kind") == null) @@ -193,12 +191,12 @@ public class AssistantSettingsCustomizer implements TableCustomizerInterface } if(AssistantModel.TABLE_NAME.equals(tableName(input)) && oldRecordList.isPresent()) { - Map previous = oldRecordListToMap("id", oldRecordList).orElse(Map.of()); + StoredRecords stored = StoredRecords.of(input, oldRecordList); for(QRecord record : records) { - QRecord before = previous.get(record.getValue("id")); + QRecord before = stored.of(record); if(before != null && List.of("providerId", "modelId", "modelKind", "embeddingDimensions").stream() - .anyMatch(name -> record.getValues().containsKey(name) && !java.util.Objects.equals(before.getValue(name), record.getValue(name)))) + .anyMatch(name -> record.getValues().containsKey(name) && stored.differs(before, record, name))) { for(var source : runtime.getRagService().sources()) { @@ -290,7 +288,7 @@ public class AssistantSettingsCustomizer implements TableCustomizerInterface /******************************************************************************* ** Validate the resulting graph, including all changes in this save batch. *******************************************************************************/ - private void validateFallbacks(List records, Optional> oldRecords) throws QException + private void validateFallbacks(AbstractActionInput input, List records, Optional> oldRecords) throws QException { AssistantRuntime runtime = AssistantRuntime.of(QContext.getQInstance()); if(runtime == null) @@ -302,13 +300,11 @@ public class AssistantSettingsCustomizer implements TableCustomizerInterface { models.put(model.getId(), model); } - Optional> old = oldRecordListToMap("id", oldRecords); + StoredRecords stored = StoredRecords.of(input, oldRecords); List proposed = new java.util.ArrayList<>(); for(QRecord record : records) { - QRecord merged = new QRecord(); - old.map(m -> m.get(record.getValue("id"))).ifPresent(o -> merged.setValues(new java.util.LinkedHashMap<>(o.getValues()))); - record.getValues().forEach(merged::setValue); + QRecord merged = stored.merged(record); AssistantModel model = new AssistantModel(merged); proposed.add(model); if(model.getId() != null) diff --git a/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/StoredRecords.java b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/StoredRecords.java new file mode 100644 index 0000000..3ce9ed6 --- /dev/null +++ b/qbit-assistant/src/main/java/com/kingsrook/qbits/assistant/metadata/StoredRecords.java @@ -0,0 +1,221 @@ +/* + * Copyright 2026 Kingsrook, LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.assistant.metadata; + + +import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput; +import com.kingsrook.qqq.backend.core.model.actions.AbstractTableActionInput; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData; +import com.kingsrook.qqq.backend.core.model.statusmessages.BadInputStatusMessage; +import com.kingsrook.qqq.backend.core.utils.ValueUtils; +import java.io.Serializable; +import java.math.BigDecimal; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.Objects; +import java.util.Optional; + + +/******************************************************************************* + ** The stored rows QQQ hands a table customizer with an update, matched to + ** the records being saved. + ** + ** A record being saved carries what its caller put in it. A save from a + ** screen carries text: the primary key comes from the request's path, and + ** every value of a multipart body is a string. The stored row carries the + ** types of the table's fields. So a record is never matched to its stored row + ** by comparing the two keys as they come, and a value being saved is never + ** compared with a stored one as it comes: both are first read as the type of + ** their field. This is the one place the customizers do either. + *******************************************************************************/ +final class StoredRecords +{ + private static final String DEFAULT_KEY_FIELD = "id"; + + private final QTableMetaData table; + private final String keyField; + private final Map byKey = new HashMap<>(); + + + + /******************************************************************************* + ** + *******************************************************************************/ + private StoredRecords(QTableMetaData table, List stored) + { + this.table = table; + this.keyField = table == null || table.getPrimaryKeyField() == null ? DEFAULT_KEY_FIELD : table.getPrimaryKeyField(); + for(QRecord row : stored) + { + Serializable key = key(row); + if(key != null) + { + byKey.put(key, row); + } + } + } + + + + /******************************************************************************* + ** The stored rows of this insert or update (none for an insert). + *******************************************************************************/ + static StoredRecords of(AbstractActionInput input, Optional> oldRecordList) + { + QTableMetaData table = input instanceof AbstractTableActionInput tableInput ? tableInput.getTable() : null; + return (new StoredRecords(table, oldRecordList.orElse(List.of()))); + } + + + + /******************************************************************************* + ** The stored row of a record being saved, or null: for an insert, and for + ** a key that is no row QQQ read. + *******************************************************************************/ + QRecord of(QRecord record) + { + Serializable key = key(record); + return (key == null ? null : byKey.get(key)); + } + + + + /******************************************************************************* + ** The record as it will be stored: the stored row, with the values being + ** saved laid over it, each as the type of its field. This is what a rule + ** is judged on, so that a save of one field is held to the same rules as a + ** save of the whole record. A value that cannot be read as the type of its + ** field is refused on the record being saved, and the stored value is kept + ** here. + *******************************************************************************/ + QRecord merged(QRecord record) + { + QRecord merged = new QRecord(); + QRecord stored = of(record); + if(stored != null) + { + merged.setValues(new LinkedHashMap<>(stored.getValues())); + } + for(Map.Entry entry : new ArrayList<>(record.getValues().entrySet())) + { + try + { + merged.setValue(entry.getKey(), typed(entry.getKey(), entry.getValue())); + } + catch(RuntimeException e) + { + /////////////////////////////////////////////////////////////////////// + // the value itself is not repeated: it may be something to keep safe // + /////////////////////////////////////////////////////////////////////// + record.addError(new BadInputStatusMessage("The value given for " + label(entry.getKey()) + " is not valid.")); + } + } + return (merged); + } + + + + /******************************************************************************* + ** Whether two records differ in a field, each value read as the type of + ** the field (so text and the number it spells are the same, and so are two + ** decimals of different scale). + *******************************************************************************/ + boolean differs(QRecord one, QRecord other, String fieldName) + { + return (!same(readable(fieldName, one.getValue(fieldName)), readable(fieldName, other.getValue(fieldName)))); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + private static boolean same(Serializable one, Serializable other) + { + if(one instanceof BigDecimal oneDecimal && other instanceof BigDecimal otherDecimal) + { + return (oneDecimal.compareTo(otherDecimal) == 0); + } + return (Objects.equals(one, other)); + } + + + + /******************************************************************************* + ** A key as the type of the key field; null when it is no key of this table. + *******************************************************************************/ + private Serializable key(QRecord record) + { + Serializable key = readable(keyField, record.getValue(keyField)); + return (key instanceof Unreadable ? null : key); + } + + + + /******************************************************************************* + ** A value as the type of its field, or a marker that equals nothing when + ** it cannot be read as that. + *******************************************************************************/ + private Serializable readable(String fieldName, Serializable value) + { + try + { + return (typed(fieldName, value)); + } + catch(RuntimeException e) + { + return (new Unreadable()); + } + } + + + + /******************************************************************************* + ** A value as the type of its field. A name that is no field of the table + ** is left as it is. + *******************************************************************************/ + private Serializable typed(String fieldName, Serializable value) + { + QFieldMetaData field = table == null ? null : table.getFields().get(fieldName); + return (field == null || value == null ? value : ValueUtils.getValueAsFieldType(field.getType(), value)); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + private String label(String fieldName) + { + QFieldMetaData field = table == null ? null : table.getFields().get(fieldName); + return (field == null || field.getLabel() == null ? fieldName : field.getLabel()); + } + + + + /******************************************************************************* + ** A value that could not be read as the type of its field. + *******************************************************************************/ + private static final class Unreadable implements Serializable + { + } +} diff --git a/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/ArchitectureTest.java b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/ArchitectureTest.java index 295f788..3053de3 100644 --- a/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/ArchitectureTest.java +++ b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/ArchitectureTest.java @@ -115,6 +115,24 @@ class ArchitectureTest + /******************************************************************************* + ** A customizer never pairs a record with its stored row itself. The map + ** QQQ offers for that is keyed by the stored key, and the key of a record + ** saved from a screen is text: a lookup by the record's own key misses + ** the row. StoredRecords does the pairing, for every customizer. + *******************************************************************************/ + @Test + void customizersPairARecordWithItsStoredRowOnlyThroughStoredRecords() + { + ArchRule rule = noClasses().that().resideInAPackage(BASE) + .should().callMethodWhere(com.tngtech.archunit.core.domain.JavaCall.Predicates.target( + com.tngtech.archunit.core.domain.properties.HasName.Predicates.name("oldRecordListToMap"))) + .because("a record saved from a screen carries its key as text: metadata.StoredRecords pairs it with its stored row"); + rule.check(classes); + } + + + /******************************************************************************* ** *******************************************************************************/ diff --git a/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/metadata/BudgetEditOverHttpTest.java b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/metadata/BudgetEditOverHttpTest.java new file mode 100644 index 0000000..fc274a3 --- /dev/null +++ b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/metadata/BudgetEditOverHttpTest.java @@ -0,0 +1,454 @@ +/* + * Copyright 2026 Kingsrook, LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.assistant.metadata; + + +import java.math.BigDecimal; +import java.time.Instant; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import com.kingsrook.qbits.assistant.AssistantQBitConfig; +import com.kingsrook.qbits.assistant.model.AssistantBudget; +import com.kingsrook.qbits.assistant.model.AssistantBudgetOverride; +import com.kingsrook.qbits.assistant.persistence.AssistantStore; +import com.kingsrook.qbits.assistant.testing.AssistantTestFixture; +import com.kingsrook.qbits.assistant.testing.BaseAssistantTest; +import com.kingsrook.qbits.assistant.testing.UiApiTestServer; +import com.kingsrook.qbits.assistant.uiapi.UiApiResult; +import com.kingsrook.qqq.backend.core.actions.tables.QueryAction; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.audits.AuditLevel; +import com.kingsrook.qqq.backend.core.model.metadata.audits.QAuditRules; +import org.junit.jupiter.api.Test; +import static org.assertj.core.api.Assertions.assertThat; + + +/******************************************************************************* + ** An administrator edits a spending limit on its record screen. The screen + ** saves through the UI API's table update route (see RecordScreen): only the + ** fields that were changed, with the version that was read. The record's id + ** comes from the request's path, so it reaches the table customizer as text, + ** and so does every value. + ** + ** The rules a budget is held to are the same for such a partial save as for a + ** whole one, and they are judged on the stored row with the change applied. + *******************************************************************************/ +class BudgetEditOverHttpTest extends BaseAssistantTest +{ + private static final String ADMIN = AssistantTestFixture.ADMIN; + + private static final String BUDGET = AssistantBudget.TABLE_NAME; + private static final String OVERRIDE = AssistantBudgetOverride.TABLE_NAME; + + private static final String OVER_THE_CEILING = "A cost limit must not be more than 1000 US dollars for one period."; + + private RecordScreen screen; + + + + /******************************************************************************* + ** The host's ceiling on a cost limit, and field-by-field audit of budgets: + ** both are a host's to turn on. + *******************************************************************************/ + @Override + protected void customizeConfig(AssistantQBitConfig config) + { + config.setBudgetCostLimitCeilingUsd(new BigDecimal("1000")); + config.setTableMetaDataCustomizer((qInstance, table) -> + { + if(BUDGET.equals(table.getName())) + { + table.withAuditRules(new QAuditRules().withAuditLevel(AuditLevel.FIELD)); + } + return (table); + }); + } + + + + /******************************************************************************* + ** The limit, the enabled flag, the period, the name, the notes and a + ** threshold, each saved alone from the version that was read: each is + ** stored, and nothing else of the budget changes. + *******************************************************************************/ + @Test + void oneChangedFieldIsSavedAndEveryOtherFieldIsKept() throws Exception + { + AssistantBudget budget = weekly(); + Map expected = screen.read(BUDGET, budget.getId()); + + Map changes = new LinkedHashMap<>(); + changes.put("limitValue", "750"); + changes.put("enabled", "false"); + changes.put("period", "MONTH"); + changes.put("name", "Everyone, a month"); + changes.put("notes", "Raised for the quarter end."); + changes.put("warnPercent", "90"); + + for(Map.Entry change : changes.entrySet()) + { + UiApiResult saved = screen.change(BUDGET, budget.getId(), Map.of(change.getKey(), change.getValue())); + assertThat(saved.status()).as("saving only " + change.getKey() + ": " + saved.body()).isEqualTo(200); + + Map after = screen.read(BUDGET, budget.getId()); + assertThat(after.get(change.getKey())).as(change.getKey() + " was stored").isNotEqualTo(expected.get(change.getKey())); + expected.put(change.getKey(), after.get(change.getKey())); + expected.put("modifyDate", after.get("modifyDate")); + assertThat(after).as("only " + change.getKey() + " changed").isEqualTo(expected); + } + + AssistantBudget stored = store().budget(budget.getId()); + assertThat(stored.getLimitValue()).isEqualByComparingTo("750"); + assertThat(stored.getEnabled()).isFalse(); + assertThat(stored.getPeriod()).isEqualTo("MONTH"); + assertThat(stored.getName()).isEqualTo("Everyone, a month"); + assertThat(stored.getNotes()).isEqualTo("Raised for the quarter end."); + assertThat(stored.getWarnPercent()).isEqualTo(90); + assertThat(stored.getScopeType()).isEqualTo("INSTANCE"); + assertThat(stored.getScopeValue()).isNull(); + assertThat(stored.getMetric()).isEqualTo("COST_USD"); + assertThat(stored.getHardCeilingPercent()).isEqualTo(120); + assertThat(stored.getAtLimitAction()).isEqualTo("STOP"); + assertThat(stored.getCreateDate()).isEqualTo(budget.getCreateDate()); + } + + + + /******************************************************************************* + ** Two people read the same budget; the second save is from a version that + ** is no longer the stored one. It is refused, and stores nothing. + *******************************************************************************/ + @Test + void aSecondSaveFromTheStaleVersionIsRefusedAndStoresNothing() throws Exception + { + AssistantBudget budget = weekly(); + String version = RecordScreen.version(screen.read(BUDGET, budget.getId())); + + assertThat(screen.save(BUDGET, budget.getId(), Map.of("limitValue", "750"), version).status()).isEqualTo(200); + Map afterTheFirst = screen.read(BUDGET, budget.getId()); + + UiApiResult stale = screen.save(BUDGET, budget.getId(), Map.of("enabled", "false"), version); + assertThat(stale.status()).as(stale.body()).isEqualTo(409); + assertThat(screen.read(BUDGET, budget.getId())).as("nothing of the refused save is stored").isEqualTo(afterTheFirst); + assertThat(store().budget(budget.getId()).getEnabled()).isTrue(); + + assertThat(screen.save(BUDGET, budget.getId(), Map.of("enabled", "false"), RecordScreen.version(afterTheFirst)).status()).as("from the version now stored").isEqualTo(200); + assertThat(store().budget(budget.getId()).getEnabled()).isFalse(); + assertThat(store().budget(budget.getId()).getLimitValue()).isEqualByComparingTo("750"); + } + + + + /******************************************************************************* + ** The host's ceiling holds for a partial save exactly as for a whole one, + ** with the same message, and it is judged on the budget as it would be + ** stored: a save that only turns a token limit into a cost limit is over + ** the ceiling with the limit that is already stored. + *******************************************************************************/ + @Test + void aPartialSaveIsHeldToTheHostsCeilingLikeAWholeOne() throws Exception + { + AssistantBudget budget = weekly(); + Map before = screen.read(BUDGET, budget.getId()); + + UiApiResult partial = screen.change(BUDGET, budget.getId(), Map.of("limitValue", "1000.01")); + assertThat(partial.status()).isEqualTo(400); + assertThat(partial.body()).contains(OVER_THE_CEILING); + + Map whole = new LinkedHashMap<>(); + for(String field : List.of("name", "enabled", "scopeType", "period", "metric", "warnPercent", "atLimitAction", "hardCeilingPercent")) + { + whole.put(field, String.valueOf(before.get(field))); + } + whole.put("limitValue", "1000.01"); + UiApiResult wholeSave = screen.change(BUDGET, budget.getId(), whole); + assertThat(wholeSave.status()).isEqualTo(400); + assertThat(message(wholeSave)).as("the same refusal, in the same words").isEqualTo(message(partial)); + assertThat(screen.read(BUDGET, budget.getId())).as("nothing stored").isEqualTo(before); + + assertThat(screen.change(BUDGET, budget.getId(), Map.of("limitValue", "1000")).status()).as("the ceiling itself is allowed").isEqualTo(200); + + AssistantBudget tokens = store().insert(new AssistantBudget().withName("Tokens, a week").withScopeType("INSTANCE").withPeriod("WEEK") + .withMetric("TOKENS").withLimitValue(new BigDecimal("5000000")).withAtLimitAction("STOP")); + UiApiResult metricOnly = screen.change(BUDGET, tokens.getId(), Map.of("metric", "COST_USD")); + assertThat(metricOnly.status()).as("5,000,000 tokens would become 5,000,000 dollars").isEqualTo(400); + assertThat(metricOnly.body()).contains(OVER_THE_CEILING); + assertThat(store().budget(tokens.getId()).getMetric()).isEqualTo("TOKENS"); + + UiApiResult created = screen.create(BUDGET, Map.of("name", "New", "scopeType", "USER", "scopeValue", "someone", "period", "DAY", "metric", "COST_USD", "limitValue", "2000")); + assertThat(created.status()).as("and for a new budget").isEqualTo(400); + assertThat(created.body()).contains(OVER_THE_CEILING); + } + + + + /******************************************************************************* + ** A partial save cannot blank what a budget must have, and cannot leave it + ** with a value no whole save could give it. Each is refused, and nothing + ** is stored. + *******************************************************************************/ + @Test + void aPartialSaveIsHeldToEveryRuleOfAWholeOne() throws Exception + { + AssistantBudget budget = weekly(); + Map before = screen.read(BUDGET, budget.getId()); + + Map, String> refused = new LinkedHashMap<>(); + refused.put(Map.of("name", ""), "Name"); + refused.put(Map.of("limitValue", ""), "Limit"); + refused.put(Map.of("period", ""), "Period"); + refused.put(Map.of("metric", ""), "Metric"); + refused.put(Map.of("scopeType", ""), "Scope Type"); + refused.put(Map.of("period", "FORTNIGHT"), "Invalid period."); + refused.put(Map.of("scopeType", "TEAM"), "Invalid scopeType."); + refused.put(Map.of("metric", "REQUESTS"), "Invalid metric."); + refused.put(Map.of("atLimitAction", "IGNORE"), "Invalid atLimitAction."); + refused.put(Map.of("scopeType", "USER"), "Scope value must be empty for INSTANCE and present for every other scope."); + refused.put(Map.of("scopeValue", "someone"), "Scope value must be empty for INSTANCE and present for every other scope."); + refused.put(Map.of("scopeType", "PROVIDER", "scopeValue", "987654"), "Scope value must reference an existing provider or model registry row."); + refused.put(Map.of("scopeType", "MODEL", "scopeValue", "main"), "Scope value must be a numeric record id."); + refused.put(Map.of("downgradeModelId", "987654"), "Downgrade model must reference an existing model registry row."); + refused.put(Map.of("limitValue", "0"), "Limit must be positive; token limits must be whole numbers."); + refused.put(Map.of("limitValue", "-5"), "Limit must be positive; token limits must be whole numbers."); + refused.put(Map.of("limitValue", "a lot"), "Limit Value"); + refused.put(Map.of("metric", "TOKENS", "limitValue", "10.5"), "Limit must be positive; token limits must be whole numbers."); + refused.put(Map.of("warnPercent", "100"), "Warning percent must be 0 through 99 and hard ceiling must be at least 100."); + refused.put(Map.of("hardCeilingPercent", "99"), "Warning percent must be 0 through 99 and hard ceiling must be at least 100."); + + for(Map.Entry, String> entry : refused.entrySet()) + { + UiApiResult result = screen.change(BUDGET, budget.getId(), entry.getKey()); + assertThat(result.status()).as("saving only " + entry.getKey() + ": " + result.body()).isEqualTo(400); + assertThat(message(result)).as("saving only " + entry.getKey()).contains(entry.getValue()).doesNotContain("a lot"); + assertThat(screen.read(BUDGET, budget.getId())).as("nothing stored by " + entry.getKey()).isEqualTo(before); + } + + UiApiResult scope = screen.change(BUDGET, budget.getId(), Map.of("scopeType", "USER", "scopeValue", "someone")); + assertThat(scope.status()).as("a scope and its value together: " + scope.body()).isEqualTo(200); + assertThat(store().budget(budget.getId()).getScopeValue()).isEqualTo("someone"); + } + + + + /******************************************************************************* + ** A setting a budget may be without gets its default back when a save + ** blanks it, as it does on a whole save that leaves it out: a budget is + ** never stored without a threshold or an action. + *******************************************************************************/ + @Test + void aBlankedSettingGetsItsDefault() throws Exception + { + AssistantBudget budget = weekly(); + assertThat(screen.change(BUDGET, budget.getId(), Map.of("warnPercent", "50", "hardCeilingPercent", "150", "enabled", "false")).status()).isEqualTo(200); + + UiApiResult blanked = screen.change(BUDGET, budget.getId(), Map.of("warnPercent", "", "hardCeilingPercent", "", "atLimitAction", "", "enabled", "")); + assertThat(blanked.status()).as(blanked.body()).isEqualTo(200); + AssistantBudget stored = store().budget(budget.getId()); + assertThat(stored.getWarnPercent()).isEqualTo(80); + assertThat(stored.getHardCeilingPercent()).isEqualTo(120); + assertThat(stored.getAtLimitAction()).isEqualTo("DOWNGRADE"); + assertThat(stored.getEnabled()).as("a blanked switch is a limit in force, never no limit").isTrue(); + assertThat(stored.getLimitValue()).isEqualByComparingTo("500"); + } + + + + /******************************************************************************* + ** What the server decides stays the server's: a save cannot name another + ** record in its body, set the version it is saved as, or say who granted + ** an override. + *******************************************************************************/ + @Test + void aSaveSetsNothingTheServerDecides() throws Exception + { + AssistantBudget budget = weekly(); + AssistantBudget other = store().insert(new AssistantBudget().withName("Another").withScopeType("USER").withScopeValue("someone").withPeriod("DAY") + .withMetric("COST_USD").withLimitValue(BigDecimal.TEN)); + Map before = screen.read(BUDGET, budget.getId()); + + Map changed = new LinkedHashMap<>(); + changed.put("id", String.valueOf(other.getId())); + changed.put("modifyDate", "2001-01-01T00:00:00Z"); + changed.put("notes", "Mine."); + UiApiResult saved = screen.change(BUDGET, budget.getId(), changed); + assertThat(saved.status()).as(saved.body()).isEqualTo(200); + + Map after = screen.read(BUDGET, budget.getId()); + assertThat(after.get("notes")).isEqualTo("Mine."); + assertThat(after.get("createDate")).isEqualTo(before.get("createDate")); + assertThat(Instant.parse(String.valueOf(after.get("modifyDate")))).as("the modify date is the server's").isAfterOrEqualTo(Instant.parse(String.valueOf(before.get("modifyDate")))); + assertThat(store().budget(other.getId()).getNotes()).as("the record of the path is the one saved").isNull(); + + Instant expiry = Instant.now().plusSeconds(3600); + UiApiResult granted = screen.create(OVERRIDE, Map.of("budgetId", String.valueOf(budget.getId()), "reason", "Quarter end", "extraValue", "50", + "expiresAt", expiry.toString(), "grantedBy", "somebody-else")); + assertThat(granted.status()).as(granted.body()).isEqualTo(200); + AssistantBudgetOverride override = store().query(OVERRIDE, new QQueryFilter(), AssistantBudgetOverride.class).get(0); + assertThat(override.getGrantedBy()).as("who granted it is the session's").isEqualTo(ADMIN); + + UiApiResult edited = screen.change(OVERRIDE, override.getId(), Map.of("reason", "Quarter end, extended", "grantedBy", "somebody-else")); + assertThat(edited.status()).as(edited.body()).isEqualTo(200); + AssistantBudgetOverride stored = store().query(OVERRIDE, new QQueryFilter(), AssistantBudgetOverride.class).get(0); + assertThat(stored.getGrantedBy()).isEqualTo(ADMIN); + assertThat(stored.getReason()).isEqualTo("Quarter end, extended"); + } + + + + /******************************************************************************* + ** An override of a budget is edited the same way, and held to its rules on + ** the stored row with the change applied. + *******************************************************************************/ + @Test + void anOverrideIsSavedOneFieldAtATime() throws Exception + { + AssistantBudget budget = weekly(); + Instant expiry = Instant.parse("2099-01-01T00:00:00Z"); + AssistantBudgetOverride override = store().insert(new AssistantBudgetOverride().withBudgetId(budget.getId()).withReason("Quarter end") + .withExtraValue(new BigDecimal("50")).withExpiresAt(expiry)); + Map expected = screen.read(OVERRIDE, override.getId()); + + UiApiResult saved = screen.change(OVERRIDE, override.getId(), Map.of("extraValue", "75")); + assertThat(saved.status()).as(saved.body()).isEqualTo(200); + Map after = screen.read(OVERRIDE, override.getId()); + expected.put("extraValue", after.get("extraValue")); + expected.put("modifyDate", after.get("modifyDate")); + assertThat(after).as("only the extra value changed").isEqualTo(expected); + AssistantBudgetOverride stored = store().query(OVERRIDE, new QQueryFilter(), AssistantBudgetOverride.class).get(0); + assertThat(stored.getExtraValue()).isEqualByComparingTo("75"); + assertThat(stored.getExpiresAt()).isEqualTo(expiry); + assertThat(stored.getBudgetId()).isEqualTo(budget.getId()); + + Map, String> refused = new LinkedHashMap<>(); + refused.put(Map.of("reason", ""), "Reason"); + refused.put(Map.of("expiresAt", ""), "Expires At"); + refused.put(Map.of("extraValue", ""), "Set exactly one of extra value or unlimited until."); + refused.put(Map.of("unlimitedUntil", "2098-01-01T00:00:00Z"), "Set exactly one of extra value or unlimited until."); + refused.put(Map.of("extraValue", "-1"), "Extra value must be positive; token values must be whole numbers."); + refused.put(Map.of("budgetId", "987654"), "Budget must reference an existing budget."); + for(Map.Entry, String> entry : refused.entrySet()) + { + UiApiResult result = screen.change(OVERRIDE, override.getId(), entry.getKey()); + assertThat(result.status()).as("saving only " + entry.getKey() + ": " + result.body()).isEqualTo(400); + assertThat(message(result)).as("saving only " + entry.getKey()).contains(entry.getValue()); + assertThat(screen.read(OVERRIDE, override.getId())).as("nothing stored by " + entry.getKey()).isEqualTo(after); + } + + UiApiResult swapped = screen.change(OVERRIDE, override.getId(), Map.of("extraValue", "", "unlimitedUntil", "2098-01-01T00:00:00Z")); + assertThat(swapped.status()).as(swapped.body()).isEqualTo(200); + UiApiResult late = screen.change(OVERRIDE, override.getId(), Map.of("unlimitedUntil", "2099-06-01T00:00:00Z")); + assertThat(late.status()).isEqualTo(400); + assertThat(message(late)).contains("Unlimited until cannot be after expiry."); + } + + + + /******************************************************************************* + ** Where a host audits budgets field by field, a save of one field records + ** that field, with its old and its new value, and no other: the stored row + ** the rules are judged on is not what is saved. + *******************************************************************************/ + @Test + void theAuditOfAPartialSaveNamesOnlyTheChangedField() throws Exception + { + AssistantBudget budget = weekly(); + assertThat(screen.change(BUDGET, budget.getId(), Map.of("limitValue", "750")).status()).isEqualTo(200); + + List details = editDetails(budget.getId()); + assertThat(details).as("one field was saved, one field is recorded").hasSize(1); + assertThat(details.get(0).getValueString("fieldName")).isEqualTo("limitValue"); + assertThat(new BigDecimal(details.get(0).getValueString("oldValue"))).isEqualByComparingTo("500"); + assertThat(new BigDecimal(details.get(0).getValueString("newValue"))).isEqualByComparingTo("750"); + + assertThat(screen.change(BUDGET, budget.getId(), Map.of("enabled", "false", "notes", "Paused.")).status()).isEqualTo(200); + assertThat(editDetails(budget.getId())).extracting(detail -> detail.getValueString("fieldName")).containsExactlyInAnyOrder("limitValue", "enabled", "notes"); + } + + + + /******************************************************************************* + ** A person without assistant.admin is refused at the door, and nothing is + ** stored. + *******************************************************************************/ + @Test + void onlyAnAdministratorSavesABudget() throws Exception + { + AssistantBudget budget = weekly(); + String version = RecordScreen.version(screen.read(BUDGET, budget.getId())); + UiApiResult refused = new RecordScreen(UiApiTestServer.serve(qInstance, config), AssistantTestFixture.EDITOR).save(BUDGET, budget.getId(), Map.of("limitValue", "1"), version); + assertThat(refused.status()).as(refused.body()).isEqualTo(403); + assertThat(store().budget(budget.getId()).getLimitValue()).isEqualByComparingTo("500"); + } + + + + /******************************************************************************* + ** A budget of 500 dollars a week for everyone, and the administrator's + ** screen. + *******************************************************************************/ + private AssistantBudget weekly() throws Exception + { + screen = new RecordScreen(UiApiTestServer.serve(qInstance, config), ADMIN); + QContext.init(qInstance, AssistantTestFixture.ownerSession(ADMIN, config)); + AssistantBudget budget = store().insert(new AssistantBudget().withName("Everyone, a week").withScopeType("INSTANCE").withPeriod("WEEK").withMetric("COST_USD") + .withLimitValue(new BigDecimal("500")).withAtLimitAction("STOP")); + return (store().budget(budget.getId())); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + private AssistantStore store() + { + return (config.getRuntime().getStore()); + } + + + + /******************************************************************************* + ** The refusal a screen shows. + *******************************************************************************/ + private static String message(UiApiResult result) + { + return (String.valueOf(result.json().get("error"))); + } + + + + /******************************************************************************* + ** The field lines of the audits of one budget's edits (every audit after + ** the first, which is the audit of its creation). + *******************************************************************************/ + private List editDetails(Integer budgetId) throws Exception + { + List audits = new QueryAction().execute(new QueryInput("audit").withFilter(new QQueryFilter() + .withCriteria("auditTable.name", QCriteriaOperator.EQUALS, BUDGET).withCriteria("recordId", QCriteriaOperator.EQUALS, budgetId) + .withOrderBy(new QFilterOrderBy("id", true)))).getRecords(); + assertThat(audits).as("the creation and at least one edit are audited").hasSizeGreaterThan(1); + return (new QueryAction().execute(new QueryInput("auditDetail").withFilter(new QQueryFilter() + .withCriteria("auditId", QCriteriaOperator.IN, audits.stream().skip(1).map(audit -> audit.getValue("id")).toList()) + .withCriteria("fieldName", QCriteriaOperator.IS_NOT_BLANK))).getRecords()); + } +} diff --git a/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/metadata/RecordEditsOverHttpTest.java b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/metadata/RecordEditsOverHttpTest.java new file mode 100644 index 0000000..6223712 --- /dev/null +++ b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/metadata/RecordEditsOverHttpTest.java @@ -0,0 +1,173 @@ +/* + * Copyright 2026 Kingsrook, LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.assistant.metadata; + + +import java.util.Map; +import com.kingsrook.qbits.assistant.access.DataAccessMode; +import com.kingsrook.qbits.assistant.model.AssistantDataAccess; +import com.kingsrook.qbits.assistant.model.AssistantProfile; +import com.kingsrook.qbits.assistant.model.AssistantSkill; +import com.kingsrook.qbits.assistant.persistence.AssistantStore; +import com.kingsrook.qbits.assistant.testing.AssistantTestFixture; +import com.kingsrook.qbits.assistant.testing.BaseAssistantTest; +import com.kingsrook.qbits.assistant.testing.UiApiTestServer; +import com.kingsrook.qbits.assistant.uiapi.UiApiResult; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import static org.assertj.core.api.Assertions.assertThat; + + +/******************************************************************************* + ** The assistant's settings are records an administrator edits on record + ** screens, which save only the fields that were changed (see RecordScreen). + ** A save of one field is held to the rules on the stored row with the change + ** applied, and leaves every other field as it is: a rule's default is for a + ** value the row does not have, never for one the save did not send. + ** + ** Budgets are in BudgetEditOverHttpTest; providers and models, knowledge + ** sources and scheduled tasks are with their own tests. + *******************************************************************************/ +class RecordEditsOverHttpTest extends BaseAssistantTest +{ + private RecordScreen screen; + + + + /******************************************************************************* + ** The administrator's screen. + *******************************************************************************/ + @BeforeEach + void administrator() + { + screen = new RecordScreen(UiApiTestServer.serve(qInstance, config), AssistantTestFixture.ADMIN); + QContext.init(qInstance, AssistantTestFixture.ownerSession(AssistantTestFixture.ADMIN, config)); + } + + + + /******************************************************************************* + ** A profile that is switched off and routes by heuristic stays so when + ** only its display name is saved. + *******************************************************************************/ + @Test + void aProfileKeepsWhatASaveDidNotSend() throws Exception + { + AssistantProfile profile = store().insert(new AssistantProfile().withName("analyst").withDisplayName("Analyst").withRoutingMode("HEURISTIC") + .withEnabled(false).withIsDefault(false).withPersonality("Brief.")); + Map expected = screen.read(AssistantProfile.TABLE_NAME, profile.getId()); + + UiApiResult saved = screen.change(AssistantProfile.TABLE_NAME, profile.getId(), Map.of("displayName", "Data analyst")); + assertThat(saved.status()).as(saved.body()).isEqualTo(200); + Map after = screen.read(AssistantProfile.TABLE_NAME, profile.getId()); + expected.put("displayName", "Data analyst"); + expected.put("modifyDate", after.get("modifyDate")); + assertThat(after).as("only the display name changed").isEqualTo(expected); + assertThat(after.get("routingMode")).isEqualTo("HEURISTIC"); + assertThat(after.get("enabled")).isEqualTo(false); + + UiApiResult routing = screen.change(AssistantProfile.TABLE_NAME, profile.getId(), Map.of("routingMode", "SOMETIMES")); + assertThat(routing.status()).isEqualTo(400); + assertThat(routing.body()).contains("Routing mode must be OFF, HEURISTIC or CLASSIFIER."); + UiApiResult app = screen.change(AssistantProfile.TABLE_NAME, profile.getId(), Map.of("appName", "no-such-app")); + assertThat(app.status()).isEqualTo(400); + assertThat(screen.read(AssistantProfile.TABLE_NAME, profile.getId())).as("nothing stored").isEqualTo(after); + } + + + + /******************************************************************************* + ** Marking a profile the default on its screen leaves it the default, and + ** unmarks the one that was. + *******************************************************************************/ + @Test + void theProfileMarkedDefaultOnItsScreenIsTheOneDefault() throws Exception + { + AssistantProfile first = store().insert(new AssistantProfile().withName("first").withDisplayName("First").withEnabled(true).withIsDefault(true)); + AssistantProfile second = store().insert(new AssistantProfile().withName("second").withDisplayName("Second").withEnabled(true).withIsDefault(false)); + + UiApiResult saved = screen.change(AssistantProfile.TABLE_NAME, second.getId(), Map.of("isDefault", "true")); + assertThat(saved.status()).as(saved.body()).isEqualTo(200); + assertThat(store().query(AssistantProfile.TABLE_NAME, new QQueryFilter().withCriteria("isDefault", QCriteriaOperator.EQUALS, true), AssistantProfile.class)) + .extracting(AssistantProfile::getId).containsExactly(second.getId()); + assertThat(screen.read(AssistantProfile.TABLE_NAME, first.getId()).get("isDefault")).isEqualTo(false); + } + + + + /******************************************************************************* + ** A skill that is switched off stays off when only its description is + ** saved, and its instructions stay bounded. + *******************************************************************************/ + @Test + void aSkillKeepsWhatASaveDidNotSend() throws Exception + { + AssistantSkill skill = store().insert(new AssistantSkill().withName("summaries").withInstructions("Summarize in three lines.").withEnabled(false)); + Map expected = screen.read(AssistantSkill.TABLE_NAME, skill.getId()); + + UiApiResult saved = screen.change(AssistantSkill.TABLE_NAME, skill.getId(), Map.of("description", "Short summaries")); + assertThat(saved.status()).as(saved.body()).isEqualTo(200); + Map after = screen.read(AssistantSkill.TABLE_NAME, skill.getId()); + expected.put("description", "Short summaries"); + expected.put("modifyDate", after.get("modifyDate")); + assertThat(after).as("only the description changed").isEqualTo(expected); + assertThat(after.get("enabled")).isEqualTo(false); + } + + + + /******************************************************************************* + ** A data access row of a process stays a row of that process when only its + ** mode is saved, and the mode is one of the three. + *******************************************************************************/ + @Test + void aDataAccessRowKeepsWhatASaveDidNotSend() throws Exception + { + String process = qInstance.getProcesses().keySet().stream().sorted().findFirst().orElseThrow(); + AssistantDataAccess row = store().insert(new AssistantDataAccess().withObjectType(AssistantDataAccess.TYPE_PROCESS).withObjectName(process).withMode(DataAccessMode.READ_WRITE.name())); + Map expected = screen.read(AssistantDataAccess.TABLE_NAME, row.getId()); + + UiApiResult saved = screen.change(AssistantDataAccess.TABLE_NAME, row.getId(), Map.of("mode", "NONE")); + assertThat(saved.status()).as(saved.body()).isEqualTo(200); + Map after = screen.read(AssistantDataAccess.TABLE_NAME, row.getId()); + expected.put("mode", "NONE"); + expected.put("modifyDate", after.get("modifyDate")); + assertThat(after).as("only the mode changed").isEqualTo(expected); + assertThat(after.get("objectType")).isEqualTo(AssistantDataAccess.TYPE_PROCESS); + assertThat(config.getRuntime().getDataAccessService().forProcess(process).mode()).as("and it is in force").isEqualTo(DataAccessMode.NONE); + + UiApiResult mode = screen.change(AssistantDataAccess.TABLE_NAME, row.getId(), Map.of("mode", "EVERYTHING")); + assertThat(mode.status()).isEqualTo(400); + assertThat(mode.body()).contains("Assistant access must be NONE, READ or READ_WRITE."); + UiApiResult name = screen.change(AssistantDataAccess.TABLE_NAME, row.getId(), Map.of("objectName", "page")); + assertThat(name.status()).as("a table's name on a row of a process").isEqualTo(400); + assertThat(screen.read(AssistantDataAccess.TABLE_NAME, row.getId())).as("nothing stored").isEqualTo(after); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + private AssistantStore store() + { + return (config.getRuntime().getStore()); + } +} diff --git a/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/metadata/RecordScreen.java b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/metadata/RecordScreen.java new file mode 100644 index 0000000..0c2e8e5 --- /dev/null +++ b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/metadata/RecordScreen.java @@ -0,0 +1,150 @@ +/* + * Copyright 2026 Kingsrook, LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.assistant.metadata; + + +import java.net.URI; +import java.net.URLEncoder; +import java.net.http.HttpClient; +import java.net.http.HttpRequest; +import java.net.http.HttpResponse; +import java.nio.charset.StandardCharsets; +import java.util.LinkedHashMap; +import java.util.Map; +import com.kingsrook.qbits.assistant.uiapi.UiApiResult; +import static org.assertj.core.api.Assertions.assertThat; + + +/******************************************************************************* + ** What a record screen of the application sends when a person reads and + ** saves a record, over the real UI API routes of the test server: + ** + ** - a read is `GET /qqq/v1/table/{tableName}/{primaryKey}`; + ** - a save is `PATCH /qqq/v1/table/{tableName}/{primaryKey}` with a + ** multipart body of ONLY the fields that were changed (an empty field + ** clears a value) and the version that was read as `expectedVersion`; + ** - a new record is `POST /qqq/v1/table/{tableName}` with a multipart body. + ** + ** So the record's key reaches the application as text (it is part of the + ** path), and so does every value (a multipart field is text). + *******************************************************************************/ +public final class RecordScreen +{ + private final String base; + private final String userId; + + + + /******************************************************************************* + ** The screen of one signed-in person, against a served application. + *******************************************************************************/ + public RecordScreen(String base, String userId) + { + this.base = base; + this.userId = userId; + } + + + + /******************************************************************************* + ** The values of a record as the screen reads them. + *******************************************************************************/ + @SuppressWarnings("unchecked") + public Map read(String tableName, Object id) throws Exception + { + UiApiResult result = send("GET", "/table/" + tableName + "/" + id, null, null); + assertThat(result.status()).as(result.body()).isEqualTo(200); + return (new LinkedHashMap<>((Map) ((Map) result.json().get("record")).get("values"))); + } + + + + /******************************************************************************* + ** The version the screen sends back with a save: the modify date as read. + *******************************************************************************/ + public static String version(Map values) + { + return (String.valueOf(values.get("modifyDate"))); + } + + + + /******************************************************************************* + ** Read the record, then save only these fields from the version read. + *******************************************************************************/ + public UiApiResult change(String tableName, Object id, Map changed) throws Exception + { + return (save(tableName, id, changed, version(read(tableName, id)))); + } + + + + /******************************************************************************* + ** Save only these fields, from this version. + *******************************************************************************/ + public UiApiResult save(String tableName, Object id, Map changed, String expectedVersion) throws Exception + { + String query = expectedVersion == null ? "" : "?expectedVersion=" + URLEncoder.encode(expectedVersion, StandardCharsets.UTF_8); + return (multipart("PATCH", "/table/" + tableName + "/" + id + query, changed)); + } + + + + /******************************************************************************* + ** Create a record from these fields. + *******************************************************************************/ + public UiApiResult create(String tableName, Map values) throws Exception + { + return (multipart("POST", "/table/" + tableName, values)); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + private UiApiResult multipart(String method, String path, Map fields) throws Exception + { + String boundary = "----record-screen-" + System.nanoTime(); + StringBuilder body = new StringBuilder(); + for(Map.Entry field : fields.entrySet()) + { + body.append("--").append(boundary).append("\r\n") + .append("Content-Disposition: form-data; name=\"").append(field.getKey()).append("\"\r\n\r\n") + .append(field.getValue()).append("\r\n"); + } + body.append("--").append(boundary).append("--\r\n"); + return (send(method, path, "multipart/form-data; boundary=" + boundary, body.toString())); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + private UiApiResult send(String method, String path, String contentType, String body) throws Exception + { + HttpRequest.Builder request = HttpRequest.newBuilder(URI.create(base + "/qqq/v1" + path)).header("Cookie", "sessionId=" + userId).header("X-QQQ-Client", "next-ui"); + if(contentType != null) + { + request.header("Content-Type", contentType); + } + request.method(method, body == null ? HttpRequest.BodyPublishers.noBody() : HttpRequest.BodyPublishers.ofString(body, StandardCharsets.UTF_8)); + HttpResponse response = HttpClient.newHttpClient().send(request.build(), HttpResponse.BodyHandlers.ofString()); + return (new UiApiResult(response.statusCode(), response.body())); + } +} diff --git a/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/metadata/StoredRecordsTest.java b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/metadata/StoredRecordsTest.java new file mode 100644 index 0000000..2ea1a82 --- /dev/null +++ b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/metadata/StoredRecordsTest.java @@ -0,0 +1,216 @@ +/* + * Copyright 2026 Kingsrook, LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.assistant.metadata; + + +import java.io.Serializable; +import java.math.BigDecimal; +import java.time.Instant; +import java.util.List; +import java.util.Optional; +import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput; +import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType; +import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData; +import org.junit.jupiter.api.Test; +import static org.assertj.core.api.Assertions.assertThat; + + +/******************************************************************************* + ** A record being saved is paired with its stored row whatever type its key + ** arrives as: text from a request's path, or the type of the key field from + ** a caller (or a QQQ version) that already made it so. + *******************************************************************************/ +class StoredRecordsTest +{ + private static final String INTEGER_KEY = "integerKey"; + private static final String LONG_KEY = "longKey"; + private static final String TEXT_KEY = "textKey"; + + + + /******************************************************************************* + ** An update of one of three tables that differ only in the type of their + ** key. + *******************************************************************************/ + private static UpdateInput update(String tableName) + { + UpdateInput input = new UpdateInput(); + input.setTableMetaData(switch(tableName) + { + case INTEGER_KEY -> table(tableName, "id", QFieldType.INTEGER); + case LONG_KEY -> table(tableName, "id", QFieldType.LONG); + default -> table(tableName, "code", QFieldType.STRING); + }); + return (input); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + private static QTableMetaData table(String name, String keyField, QFieldType keyType) + { + return (new QTableMetaData().withName(name).withPrimaryKeyField(keyField) + .withField(new QFieldMetaData(keyField, keyType)) + .withField(new QFieldMetaData("name", QFieldType.STRING).withLabel("Name")) + .withField(new QFieldMetaData("quantity", QFieldType.INTEGER).withLabel("Quantity")) + .withField(new QFieldMetaData("price", QFieldType.DECIMAL).withLabel("Price")) + .withField(new QFieldMetaData("enabled", QFieldType.BOOLEAN)) + .withField(new QFieldMetaData("seenAt", QFieldType.DATE_TIME).withLabel("Seen At"))); + } + + + + /******************************************************************************* + ** An integer key and a long key: the stored row is found for the key as + ** text, as an Integer and as a Long; a key of another row, no key, and + ** text that is no key find none. + *******************************************************************************/ + @Test + void aRecordIsPairedWithItsStoredRowWhateverTypeItsKeyArrivesAs() + { + for(String tableName : List.of(INTEGER_KEY, LONG_KEY)) + { + Serializable storedKey = tableName.equals(INTEGER_KEY) ? (Serializable) 41 : (Serializable) 41L; + QRecord row = new QRecord().withValue("id", storedKey).withValue("name", "Widget"); + StoredRecords stored = StoredRecords.of(update(tableName), Optional.of(List.of(row, new QRecord().withValue("name", "A row without a key")))); + + for(Serializable key : List.of("41", 41, 41L)) + { + assertThat(stored.of(new QRecord().withValue("id", key))).as(tableName + ", key " + key.getClass().getSimpleName()).isSameAs(row); + } + assertThat(stored.of(new QRecord().withValue("id", "42"))).isNull(); + assertThat(stored.of(new QRecord().withValue("id", "forty-one"))).as("text that is no key").isNull(); + assertThat(stored.of(new QRecord().withValue("name", "No key"))).isNull(); + } + } + + + + /******************************************************************************* + ** A text key is compared as the text it is: "007" is not 7. + *******************************************************************************/ + @Test + void aTextKeyIsNeverReadAsANumber() + { + QRecord row = new QRecord().withValue("code", "007"); + StoredRecords stored = StoredRecords.of(update(TEXT_KEY), Optional.of(List.of(row))); + assertThat(stored.of(new QRecord().withValue("code", "007"))).isSameAs(row); + assertThat(stored.of(new QRecord().withValue("code", "7"))).isNull(); + } + + + + /******************************************************************************* + ** The merged record is the stored row with the values being saved laid + ** over it, each as the type of its field; the record being saved is not + ** changed, a value sent as nothing blanks the field, and a name that is no + ** field is passed on as it is. + *******************************************************************************/ + @Test + void theMergedRecordIsTheStoredRowWithTheSavedValuesAsTheirFieldsTypes() + { + QRecord row = new QRecord().withValue("id", 41).withValue("name", "Widget").withValue("quantity", 5).withValue("price", new BigDecimal("19.99")).withValue("enabled", true); + StoredRecords stored = StoredRecords.of(update(INTEGER_KEY), Optional.of(List.of(row))); + + QRecord saved = new QRecord().withValue("id", "41").withValue("quantity", "7").withValue("enabled", "false").withValue("seenAt", "2026-11-20T10:15:30Z") + .withValue("name", null).withValue("notAField", "12"); + QRecord merged = stored.merged(saved); + + assertThat(merged.getValue("id")).isEqualTo(41); + assertThat(merged.getValue("quantity")).isEqualTo(7); + assertThat(merged.getValue("enabled")).isEqualTo(false); + assertThat(merged.getValue("seenAt")).isEqualTo(Instant.parse("2026-11-20T10:15:30Z")); + assertThat(merged.getValue("price")).as("kept from the stored row").isEqualTo(new BigDecimal("19.99")); + assertThat(merged.getValues()).containsEntry("name", null); + assertThat(merged.getValue("notAField")).isEqualTo("12"); + assertThat(saved.getValue("quantity")).as("the record being saved is as it came").isEqualTo("7"); + assertThat(saved.getErrors()).isEmpty(); + assertThat(row.getValue("quantity")).as("and so is the stored row").isEqualTo(5); + + QRecord inserted = stored.merged(new QRecord().withValue("quantity", "3")); + assertThat(inserted.getValues()).as("an insert has no stored row").containsOnlyKeys("quantity"); + assertThat(inserted.getValue("quantity")).isEqualTo(3); + } + + + + /******************************************************************************* + ** A value that cannot be read as the type of its field is refused on the + ** record being saved, by the field's name and without the value, and the + ** merged record keeps the stored value. + *******************************************************************************/ + @Test + void aValueThatIsNotOfItsFieldsTypeIsRefused() + { + QRecord row = new QRecord().withValue("id", 41).withValue("quantity", 5).withValue("enabled", true); + StoredRecords stored = StoredRecords.of(update(INTEGER_KEY), Optional.of(List.of(row))); + + QRecord saved = new QRecord().withValue("id", 41).withValue("quantity", "a-great-many").withValue("price", "a-great-many"); + QRecord merged = stored.merged(saved); + + assertThat(saved.getErrors()).extracting(error -> error.getMessage()).containsExactly("The value given for Quantity is not valid.", "The value given for Price is not valid."); + assertThat(merged.getValue("quantity")).isEqualTo(5); + assertThat(merged.getValue("price")).isNull(); + } + + + + /******************************************************************************* + ** Two values differ only when they differ as the type of their field: text + ** and the number it spells do not, nor do two decimals of different scale. + *******************************************************************************/ + @Test + void valuesAreComparedAsTheTypeOfTheirField() + { + StoredRecords stored = StoredRecords.of(update(INTEGER_KEY), Optional.empty()); + QRecord row = new QRecord().withValue("quantity", 5).withValue("price", new BigDecimal("19.90")).withValue("enabled", true).withValue("name", "Widget"); + + assertThat(stored.differs(row, new QRecord().withValue("quantity", "5"), "quantity")).isFalse(); + assertThat(stored.differs(row, new QRecord().withValue("quantity", 5L), "quantity")).isFalse(); + assertThat(stored.differs(row, new QRecord().withValue("quantity", "6"), "quantity")).isTrue(); + assertThat(stored.differs(row, new QRecord().withValue("price", "19.9"), "price")).isFalse(); + assertThat(stored.differs(row, new QRecord().withValue("price", "19.91"), "price")).isTrue(); + assertThat(stored.differs(row, new QRecord().withValue("enabled", "true"), "enabled")).isFalse(); + assertThat(stored.differs(row, new QRecord().withValue("enabled", "false"), "enabled")).isTrue(); + assertThat(stored.differs(row, new QRecord(), "name")).as("a value against none").isTrue(); + assertThat(stored.differs(new QRecord(), new QRecord(), "name")).isFalse(); + assertThat(stored.differs(row, new QRecord().withValue("quantity", "many"), "quantity")).as("text that is no number is not the number").isTrue(); + assertThat(stored.differs(new QRecord().withValue("quantity", "many"), new QRecord().withValue("quantity", "many"), "quantity")).as("and is not known to be the same as itself").isTrue(); + } + + + + /******************************************************************************* + ** With no table to read the types from, keys and values are taken as + ** they come. + *******************************************************************************/ + @Test + void withoutATableValuesAreTakenAsTheyCome() + { + QRecord row = new QRecord().withValue("id", 41).withValue("name", "Widget"); + StoredRecords stored = StoredRecords.of(new RunProcessInput(), Optional.of(List.of(row))); + + assertThat(stored.of(new QRecord().withValue("id", 41))).isSameAs(row); + assertThat(stored.of(new QRecord().withValue("id", "41"))).isNull(); + assertThat(stored.merged(new QRecord().withValue("id", 41).withValue("quantity", "7")).getValue("quantity")).isEqualTo("7"); + } +} diff --git a/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/rag/RagTest.java b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/rag/RagTest.java index 191d5f8..cf640bb 100644 --- a/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/rag/RagTest.java +++ b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/rag/RagTest.java @@ -42,6 +42,7 @@ import com.kingsrook.qbits.assistant.spi.EmbeddingResponse; import com.kingsrook.qbits.assistant.spi.ModelPart; import com.kingsrook.qbits.assistant.spi.ModelResponse; import com.kingsrook.qbits.assistant.spi.ModelUsage; +import com.kingsrook.qbits.assistant.metadata.RecordScreen; import com.kingsrook.qbits.assistant.testing.AssistantTestFixture; import com.kingsrook.qbits.assistant.testing.BaseAssistantTest; import com.kingsrook.qbits.assistant.testing.UiApiTestServer; @@ -49,6 +50,7 @@ import com.kingsrook.qbits.assistant.tools.KnowledgeTool; import com.kingsrook.qbits.assistant.tools.Toolboxes; import com.kingsrook.qbits.assistant.tools.UiApiToolbox; import com.kingsrook.qbits.assistant.uiapi.UiApiCall; +import com.kingsrook.qbits.assistant.uiapi.UiApiResult; import com.kingsrook.qqq.backend.core.actions.ai.AiExposure; import com.kingsrook.qqq.backend.core.actions.ai.AiFieldMetaData; import com.kingsrook.qqq.backend.core.actions.ai.AiToolOutcome; @@ -230,6 +232,43 @@ class RagTest extends BaseAssistantTest + /******************************************************************************* + ** A knowledge source is edited on its record screen one field at a time: a + ** save of its name is held to the rules on the stored source, and keeps + ** its vectors; a save that changes what the vectors mean drops them. + *******************************************************************************/ + @Test + void aSourceIsSavedOneFieldAtATimeFromItsScreen() throws Exception + { + RecordScreen screen = new RecordScreen(UiApiTestServer.serve(qInstance, config), AssistantTestFixture.ADMIN); + assertThat(config.getRuntime().getRagService().index(source).errors()).isZero(); + var indexed = vectors.count(source.getId()); + assertThat(indexed).isPositive(); + + Map expected = screen.read(AssistantRagSource.TABLE_NAME, source.getId()); + UiApiResult renamed = screen.change(AssistantRagSource.TABLE_NAME, source.getId(), Map.of("name", "Blog posts")); + assertThat(renamed.status()).as(renamed.body()).isEqualTo(200); + Map after = screen.read(AssistantRagSource.TABLE_NAME, source.getId()); + expected.put("name", "Blog posts"); + expected.put("modifyDate", after.get("modifyDate")); + assertThat(after).as("only the name changed").isEqualTo(expected); + assertThat(vectors.count(source.getId())).as("a new name leaves the vectors").isEqualTo(indexed); + + UiApiResult same = screen.change(AssistantRagSource.TABLE_NAME, source.getId(), Map.of("chunkChars", "1500")); + assertThat(same.status()).as(same.body()).isEqualTo(200); + assertThat(vectors.count(source.getId())).as("the chunk size it already has, sent as text, is no change").isEqualTo(indexed); + + UiApiResult tooSmall = screen.change(AssistantRagSource.TABLE_NAME, source.getId(), Map.of("chunkChars", "50")); + assertThat(tooSmall.status()).isEqualTo(400); + assertThat(tooSmall.body()).contains("Chunk size must be 100 to 16000"); + + UiApiResult chunks = screen.change(AssistantRagSource.TABLE_NAME, source.getId(), Map.of("chunkChars", "800")); + assertThat(chunks.status()).as(chunks.body()).isEqualTo(200); + assertThat(vectors.count(source.getId())).as("another chunk size drops the vectors").isZero(); + } + + + /******************************************************************************* ** The session used both for configuration reads and UI API calls. *******************************************************************************/ diff --git a/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/scheduled/ScheduledTasksTest.java b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/scheduled/ScheduledTasksTest.java index 64239ea..166d7ab 100644 --- a/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/scheduled/ScheduledTasksTest.java +++ b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/scheduled/ScheduledTasksTest.java @@ -41,6 +41,7 @@ import com.kingsrook.qbits.assistant.RuntimeInternals; import com.kingsrook.qbits.assistant.AssistantQBitConfig; import com.kingsrook.qbits.assistant.AssistantRuntime; import com.kingsrook.qbits.assistant.AssistantSecurity; +import com.kingsrook.qbits.assistant.metadata.RecordScreen; import com.kingsrook.qbits.assistant.model.AssistantDataAccess; import com.kingsrook.qbits.assistant.model.AssistantPendingAction; import com.kingsrook.qbits.assistant.model.AssistantPolicy; @@ -57,6 +58,7 @@ import com.kingsrook.qbits.assistant.testing.AssistantTestFixture; import com.kingsrook.qbits.assistant.testing.BaseAssistantTest; import com.kingsrook.qbits.assistant.testing.UiApiTestServer; import com.kingsrook.qbits.assistant.uiapi.UiApiCall; +import com.kingsrook.qbits.assistant.uiapi.UiApiResult; import com.kingsrook.qqq.backend.core.actions.ai.DataClassification; import com.kingsrook.qqq.backend.core.actions.ai.AiFieldMetaData; import com.kingsrook.qqq.backend.core.actions.tables.QueryAction; @@ -606,6 +608,55 @@ class ScheduledTasksTest extends BaseAssistantTest + /******************************************************************************* + ** A scheduled task is edited on its record screen one field at a time: a + ** save of its name is held to the rules on the stored task, and leaves the + ** approval of its writes and its next run as they are; a save of its + ** prompt takes the approval away. + *******************************************************************************/ + @Test + void aTaskIsSavedOneFieldAtATimeFromItsScreen() throws Exception + { + AssistantScheduledTask task = task(); + secrets.put("SCHEDULED_TOKEN", AssistantTestFixture.ADMIN); + runtime.getStore().updateFields(AssistantScheduledTask.TABLE_NAME, new QRecord().withValue("id", task.getId()).withValue("runAsUserId", AssistantTestFixture.ADMIN).withValue("writeModeRequested", true)); + runtime.getStore().scheduledUpdate(new QRecord().withValue("id", task.getId()).withValue("nextRunAt", NOW.minusSeconds(60))); + action(runtime.getStore().scheduledTask(task.getId()), "Approve"); + AssistantScheduledTask approved = runtime.getStore().scheduledTask(task.getId()); + assertThat(ScheduledTasks.writesApproved(approved)).isTrue(); + + RecordScreen screen = new RecordScreen(UiApiTestServer.serve(qInstance, config), AssistantTestFixture.ADMIN); + UiApiResult renamed = screen.change(AssistantScheduledTask.TABLE_NAME, task.getId(), Map.of("name", "Morning report, weekdays")); + assertThat(renamed.status()).as(renamed.body()).isEqualTo(200); + AssistantScheduledTask stored = runtime.getStore().scheduledTask(task.getId()); + assertThat(stored.getName()).isEqualTo("Morning report, weekdays"); + assertThat(stored.getPrompt()).isEqualTo(approved.getPrompt()); + assertThat(stored.getCronExpression()).isEqualTo(approved.getCronExpression()); + assertThat(stored.getTimezone()).isEqualTo(approved.getTimezone()); + assertThat(stored.getEnabled()).isTrue(); + assertThat(ScheduledTasks.writesApproved(stored)).as("a new name does not take the approval away").isTrue(); + assertThat(stored.getWriteModeApprovedBy()).isEqualTo(approved.getWriteModeApprovedBy()); + assertThat(stored.getNextRunAt()).as("a new name does not move the next run").isEqualTo(NOW.minusSeconds(60)); + + UiApiResult same = screen.change(AssistantScheduledTask.TABLE_NAME, task.getId(), Map.of("writeModeRequested", "true", "enabled", "true")); + assertThat(same.status()).as(same.body()).isEqualTo(200); + assertThat(ScheduledTasks.writesApproved(runtime.getStore().scheduledTask(task.getId()))).as("the values it already has, sent as text, are no change").isTrue(); + assertThat(runtime.getStore().scheduledTask(task.getId()).getNextRunAt()).isEqualTo(NOW.minusSeconds(60)); + + UiApiResult blank = screen.change(AssistantScheduledTask.TABLE_NAME, task.getId(), Map.of("prompt", "")); + assertThat(blank.status()).as("a task is not left without a prompt").isEqualTo(400); + assertThat(ScheduledTasks.writesApproved(runtime.getStore().scheduledTask(task.getId()))).isTrue(); + + UiApiResult prompt = screen.change(AssistantScheduledTask.TABLE_NAME, task.getId(), Map.of("prompt", "Summarize and publish the posts of today")); + assertThat(prompt.status()).as(prompt.body()).isEqualTo(200); + assertThat(runtime.getStore().scheduledTask(task.getId()).getWriteModeApprovedAt()).as("a changed prompt was not approved").isNull(); + + UiApiResult approval = screen.change(AssistantScheduledTask.TABLE_NAME, task.getId(), Map.of("writeModeApprovedBy", "somebody", "writeModeApprovedAt", NOW.toString(), "name", "Mine")); + assertThat(runtime.getStore().scheduledTask(task.getId()).getWriteModeApprovedBy()).as("an approval is not something a save sets: " + approval.body()).isNull(); + } + + + /******************************************************************************* ** Application record locks stay effective with a designated-user login. *******************************************************************************/ diff --git a/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/settings/AssistantSettingsTest.java b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/settings/AssistantSettingsTest.java index f723eeb..45861ce 100644 --- a/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/settings/AssistantSettingsTest.java +++ b/qbit-assistant/src/test/java/com/kingsrook/qbits/assistant/settings/AssistantSettingsTest.java @@ -35,6 +35,7 @@ import com.kingsrook.qbits.assistant.home.AssistantHome; import com.kingsrook.qbits.assistant.home.AssistantHomeData; import com.kingsrook.qbits.assistant.home.AssistantHomeView; import com.kingsrook.qbits.assistant.metadata.AssistantRecordScreens; +import com.kingsrook.qbits.assistant.metadata.RecordScreen; import com.kingsrook.qbits.assistant.model.AssistantModel; import com.kingsrook.qbits.assistant.model.AssistantModelPrice; import com.kingsrook.qbits.assistant.model.AssistantProvider; @@ -51,6 +52,7 @@ import com.kingsrook.qbits.assistant.testing.AssistantTestFixture; import com.kingsrook.qbits.assistant.testing.UiApiTestServer; import com.kingsrook.qbits.assistant.testing.BaseAssistantTest; import com.kingsrook.qbits.assistant.testing.TurnRunner; +import com.kingsrook.qbits.assistant.uiapi.UiApiResult; import com.kingsrook.qqq.backend.core.actions.ai.AiExposure; import com.kingsrook.qqq.backend.core.actions.ai.AiTableMetaData; import com.kingsrook.qqq.backend.core.actions.customizers.QCodeLoader; @@ -657,6 +659,45 @@ class AssistantSettingsTest extends BaseAssistantTest + /******************************************************************************* + ** A provider and a registered model are edited on their record screens one + ** field at a time: a save of one field is held to the rules on the stored + ** row, and leaves every other field as it is. + *******************************************************************************/ + @Test + void aProviderAndAModelAreSavedOneFieldAtATimeFromTheirScreens() throws Exception + { + asAdmin(); + Integer id = providerId(); + AssistantModel model = new AssistantStore().insert(new AssistantModel().withProviderId(id).withModelId("registered-chat").withEnabled(true).withModelKind("CHAT")); + RecordScreen screen = new RecordScreen(UiApiTestServer.serve(qInstance, config), AssistantTestFixture.ADMIN); + + Map expected = screen.read(AssistantProvider.TABLE_NAME, id); + UiApiResult renamed = screen.change(AssistantProvider.TABLE_NAME, id, Map.of("name", "Renamed")); + assertThat(renamed.status()).as(renamed.body()).isEqualTo(200); + Map after = screen.read(AssistantProvider.TABLE_NAME, id); + expected.put("name", "Renamed"); + expected.put("modifyDate", after.get("modifyDate")); + assertThat(after).as("only the name changed").isEqualTo(expected); + + UiApiResult unknownType = screen.change(AssistantProvider.TABLE_NAME, id, Map.of("providerType", "nope")); + assertThat(unknownType.status()).isEqualTo(400); + assertThat(unknownType.body()).contains("not installed"); + UiApiResult cap = screen.change(AssistantProvider.TABLE_NAME, id, Map.of("monthlyCostCapUsd", "-1")); + assertThat(cap.status()).as("a rule of the whole row holds for a save of one field").isEqualTo(400); + assertThat(screen.read(AssistantProvider.TABLE_NAME, id)).as("nothing stored").isEqualTo(after); + + Map expectedModel = screen.read(AssistantModel.TABLE_NAME, model.getId()); + UiApiResult disabled = screen.change(AssistantModel.TABLE_NAME, model.getId(), Map.of("enabled", "false")); + assertThat(disabled.status()).as(disabled.body()).isEqualTo(200); + Map afterModel = screen.read(AssistantModel.TABLE_NAME, model.getId()); + expectedModel.put("enabled", false); + expectedModel.put("modifyDate", afterModel.get("modifyDate")); + assertThat(afterModel).as("only the switch changed").isEqualTo(expectedModel); + } + + + /******************************************************************************* ** Kinds use the selected type and keep their existing labels. *******************************************************************************/