From bdb1934c6abab684dcbb3e559972c186bd665bec Mon Sep 17 00:00:00 2001 From: James Maes Date: Thu, 22 Oct 2026 18:25:20 -0500 Subject: [PATCH 1/3] feat: expose tag labels and EAV values through core reads Use batched providers without replacing host customizers. Preserve transaction visibility and parent authorization, retain tag picker behavior, and verify query, GET and export values. Requires coordinated core PR QRun-IO/qqq#1322; published dependency pin follows after its snapshot is available. Refs QRun-IO/qqq#1196 Co-Authored-By: GPT-6 (Codex) --- README.md | 7 +- docs/VIRTUAL-FIELD-UPGRADE.md | 41 ++++ pom.xml | 5 + .../collaboration/CollaborationRecords.java | 11 +- .../customfields/CustomFieldStorage.java | 4 +- .../CustomFieldValueProvider.java | 101 ++++++++++ .../customfields/CustomFieldsProvider.java | 3 +- .../collaboration/tags/TagValueProvider.java | 90 +++++++++ .../tags/TagsMetaDataProvider.java | 3 +- .../CollaborationApiHttpTest.java | 80 ++++++++ .../collaboration/CoreContractsTest.java | 3 +- .../CustomFieldValuesPostgresTest.java | 188 ++++++++++++++++++ .../qbits/collaboration/CustomFieldsTest.java | 92 ++++++++- .../qbits/collaboration/TagsServiceTest.java | 120 +++++++++++ 14 files changed, 737 insertions(+), 11 deletions(-) create mode 100644 docs/VIRTUAL-FIELD-UPGRADE.md create mode 100644 src/main/java/com/kingsrook/qbits/collaboration/customfields/CustomFieldValueProvider.java create mode 100644 src/main/java/com/kingsrook/qbits/collaboration/tags/TagValueProvider.java create mode 100644 src/test/java/com/kingsrook/qbits/collaboration/CustomFieldValuesPostgresTest.java diff --git a/README.md b/README.md index 1cdb8d3..69af815 100644 --- a/README.md +++ b/README.md @@ -408,7 +408,7 @@ size, pending suggestions, and the five most-used approved tags. `collaborationTags` is a virtual STRING field labeled Tags with possible-value source `collaboration_tag`. `TagCriteriaExpander` supports IN, NOT_IN, EQUALS, NOT_EQUALS, IS_BLANK, IS_NOT_BLANK and CONTAINS_ALL using core `QSubQuery` membership filters. Ordinary quick search matches labels; `#name` matches normalized names. Numeric criteria are tag IDs; string criteria are names. The old ID materializer is removed. Core owns SQL execution or bounded cross-backend materialization; its bound applies to inner rows, not final parent matches. -The runtime provider adds this field and its search entry where TAGS is enabled. It is queryable but not selectable: the overlay has no virtual-field post-query hook and existing host customizers must be preserved. +The runtime provider adds this field and its search entry where TAGS is enabled. With the coordinated core value-provider API, it is selectable in queries, record reads and exports. Labels are sorted, deduplicated and joined with a comma and space; the picker still uses tag IDs. The provider runs after the host post-query customizer without replacing it. This is an unreleased source change: see [virtual-field upgrade status](docs/VIRTUAL-FIELD-UPGRADE.md) before building or adopting this branch. ## UI API @@ -683,8 +683,9 @@ metadata. Memory uses core `JsonPathStorage`. PostgreSQL uses it after a cached `SELECT qqq_custom_fields FROM LIMIT 0` probe succeeds. Missing columns and unsupported backends get read-only EAV virtual filters over `collaboration_field_value`, with -typed `QSubQuery` criteria. These fields are not query-selectable; EAV value writes and projection -are outside this version. `statusMessage` records the fallback reason. After adding a missing +typed `QSubQuery` criteria. The coordinated value-provider API also makes existing EAV values +selectable in queries, record reads and exports, with their declared types and possible-value +labels. EAV writes remain unsupported. See [virtual-field upgrade status](docs/VIRTUAL-FIELD-UPGRADE.md). `statusMessage` records the fallback reason. After adding a missing column, call `CustomFieldStorage.invalidate(instance, tableName)` and refresh the overlay. ### PostgreSQL setup and indexes diff --git a/docs/VIRTUAL-FIELD-UPGRADE.md b/docs/VIRTUAL-FIELD-UPGRADE.md new file mode 100644 index 0000000..fad4b7f --- /dev/null +++ b/docs/VIRTUAL-FIELD-UPGRADE.md @@ -0,0 +1,41 @@ +# Selectable collaboration fields: unreleased upgrade + +Tracks [QQQ #1196](https://github.com/QRun-IO/qqq/issues/1196). + +This source branch requires the new core virtual-field value-provider API. The +published versions in CANDIDATE-RELEASE.md do not include that API. The checked-in +core pin still identifies the earlier published candidate and must be updated +before this branch can be built or released with its default configuration. + +Local verification uses core artifacts installed as +`4.1.0-values1196-SNAPSHOT`, with +`./mvnw.cmd -Dqqq.snapshot.version=4.1.0-values1196-SNAPSHOT verify`. +That version exists only in the local Maven repository; it is not an adoption pin. + +## Behavior + +- `collaborationTags` returns sorted, distinct approved labels from global or + matching-table vocabulary. Merged tags and deleted assignments are excluded. + The picker and filters retain their tag IDs; exports contain one Tags column. +- EAV custom fields return typed existing values and ordinary possible-value + display labels. Missing values are null internally and omitted from HTTP value + maps. This change does not enable EAV writes. +- Providers process batches and use the originating transaction. Parent record + authorization and field visibility remain enforced by core. Existing host + post-query customizers remain installed. +- SQL DISTINCT on provider values is explicitly unsupported. Backend primary-key + identity is required; joined heavy primary keys must be fetched explicitly. + +## Release sequence + +1. Publish and verify the coordinated core artifact containing the provider API. +2. Replace this branch's core BOM pin with that published version. +3. Run full collaboration verification against the published prerequisite, + including coverage, HTTP, PostgreSQL transaction and Redis tests. +4. Publish the collaboration snapshot and verify an independent consumer using + published artifacts. Update CANDIDATE-RELEASE.md with the exact pair. +5. Update Kof22 Website-Backend adoption work and qualify it in the selected + isolated staging environment before requesting production deployment. + +Local verification is evidence for this implementation, not proof that the +published candidate or Kof22 deployment has these changes. diff --git a/pom.xml b/pom.xml index 32a601d..47305d1 100644 --- a/pom.xml +++ b/pom.xml @@ -72,6 +72,11 @@ + + com.kingsrook.qqq + qqq-backend-module-postgres + test + com.kingsrook.qbits qbit-redis diff --git a/src/main/java/com/kingsrook/qbits/collaboration/CollaborationRecords.java b/src/main/java/com/kingsrook/qbits/collaboration/CollaborationRecords.java index a396690..f40b112 100644 --- a/src/main/java/com/kingsrook/qbits/collaboration/CollaborationRecords.java +++ b/src/main/java/com/kingsrook/qbits/collaboration/CollaborationRecords.java @@ -23,6 +23,7 @@ package com.kingsrook.qbits.collaboration; import java.io.Serializable; import java.util.List; +import com.kingsrook.qqq.backend.core.actions.QBackendTransaction; import com.kingsrook.qqq.backend.core.actions.tables.DeleteAction; import com.kingsrook.qqq.backend.core.actions.tables.InsertAction; import com.kingsrook.qqq.backend.core.actions.tables.QueryAction; @@ -74,7 +75,15 @@ public final class CollaborationRecords *******************************************************************************/ public static List query(String table, QQueryFilter filter) throws QException { - QueryInput input = new QueryInput(table).withFilter(filter).withShouldFetchHeavyFields(true).withTransaction(QContext.getQBackendTransaction()); + return (query(table, filter, QContext.getQBackendTransaction())); + } + + /******************************************************************************* + ** Provider reads participate in the originating query's explicit transaction. + *******************************************************************************/ + public static List query(String table, QQueryFilter filter, QBackendTransaction transaction) throws QException + { + QueryInput input = new QueryInput(table).withFilter(filter).withShouldFetchHeavyFields(true).withTransaction(transaction); input.setTableMetaData(serviceMetadata(table)); input.withPermissionBypass("Trusted collaboration service performs parent/ownership authorization."); return (new QueryAction().execute(input).getRecords().stream().map(QRecord::new).toList()); diff --git a/src/main/java/com/kingsrook/qbits/collaboration/customfields/CustomFieldStorage.java b/src/main/java/com/kingsrook/qbits/collaboration/customfields/CustomFieldStorage.java index cfc5959..f70b8b7 100644 --- a/src/main/java/com/kingsrook/qbits/collaboration/customfields/CustomFieldStorage.java +++ b/src/main/java/com/kingsrook/qbits/collaboration/customfields/CustomFieldStorage.java @@ -83,7 +83,7 @@ public final class CustomFieldStorage } if(!postgres(table)) { - return (new Availability(false, "Backend does not support JsonPathStorage; EAV fields are read-only filters (values are not selected).")); + return (new Availability(false, "Backend does not support JsonPathStorage; EAV fields support read-only values and filters.")); } try(Connection connection = connection(table); var statement = connection.createStatement()) { @@ -92,7 +92,7 @@ public final class CustomFieldStorage } catch(SQLException | IllegalArgumentException e) { - return (new Availability(false, "JSON column unavailable; EAV fields are read-only filters. " + e.getMessage())); + return (new Availability(false, "JSON column unavailable; EAV fields support read-only values and filters. " + e.getMessage())); } } diff --git a/src/main/java/com/kingsrook/qbits/collaboration/customfields/CustomFieldValueProvider.java b/src/main/java/com/kingsrook/qbits/collaboration/customfields/CustomFieldValueProvider.java new file mode 100644 index 0000000..b974680 --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/collaboration/customfields/CustomFieldValueProvider.java @@ -0,0 +1,101 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + + +package com.kingsrook.qbits.collaboration.customfields; + +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import com.kingsrook.qbits.collaboration.CollaborationRecords; +import com.kingsrook.qbits.collaboration.model.CollaborationTables; +import com.kingsrook.qqq.backend.core.actions.tables.QueryAction; +import com.kingsrook.qqq.backend.core.actions.tables.query.VirtualFieldValueProviderInterface; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordReferenceTableMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData; +import com.kingsrook.qqq.backend.core.utils.CollectionUtils; +import com.kingsrook.qqq.backend.core.utils.ValueUtils; + +/******************************************************************************* + ** Supply EAV values for the parent records selected by the core query pipeline. + ** Do not re-read parents here: that would recursively execute this provider. + *******************************************************************************/ +public class CustomFieldValueProvider implements VirtualFieldValueProviderInterface +{ + /******************************************************************************* + ** Bound each lookup and retain the caller's transaction through service storage. + *******************************************************************************/ + @Override + public void fill(QTableMetaData table, String fieldName, List records, QueryInput input) throws QException + { + String name = fieldName.substring(3); + var definitions = CollaborationRecords.query(CollaborationTables.FIELD_DEFINITION, new QQueryFilter() + .withCriteria("tableName", QCriteriaOperator.EQUALS, table.getName()).withCriteria("name", QCriteriaOperator.EQUALS, name) + .withCriteria("status", QCriteriaOperator.EQUALS, "ACTIVE"), input.getTransaction()); + if(definitions.size() != 1) + { + throw (new QException("Custom field is unavailable")); + } + String column = FieldCriteriaExpander.valueColumn(definitions.getFirst().getValueString("type")); + for(List page : CollectionUtils.getPages(records, 500)) + { + List ids = page.stream().map(record -> String.valueOf(record.getValue(table.getPrimaryKeyField()))).distinct().toList(); + Map values = new HashMap<>(); + for(QRecord value : valuesForSelectedParents(new QQueryFilter() + .withCriteria("tableName", QCriteriaOperator.EQUALS, table.getName()).withCriteria("fieldName", QCriteriaOperator.EQUALS, name) + .withCriteria("parentDeleted", QCriteriaOperator.EQUALS, false).withCriteria("recordId", QCriteriaOperator.IN, ids), input)) + { + if(values.put(value.getValueString("recordId"), value) != null) + { + throw (new QException("Duplicate custom field values")); + } + } + for(QRecord record : page) + { + QRecord value = values.get(String.valueOf(record.getValue(table.getPrimaryKeyField()))); + record.setValue(fieldName, value == null ? null : ValueUtils.getValueAsFieldType(table.getVirtualField(fieldName).getType(), value.getValue(column))); + } + } + } + + /******************************************************************************* + ** The enclosing query already selected these parents under its access policy. + ** Remove only recursive parent verification on a private child metadata clone; + ** preserve child locks and never install this metadata on the instance. + *******************************************************************************/ + private List valuesForSelectedParents(QQueryFilter filter, QueryInput origin) throws QException + { + QTableMetaData metadata = QContext.getQInstance().getTable(CollaborationTables.FIELD_VALUE).clone(); + metadata.withCustomizers(null); + metadata.getSupplementalMetaData().remove(RecordReferenceTableMetaData.NAME); + QueryInput input = new QueryInput(CollaborationTables.FIELD_VALUE).withFilter(filter) + .withShouldFetchHeavyFields(true).withTransaction(origin.getTransaction()); + input.setTableMetaData(metadata); + input.withPermissionBypass("EAV provider reads only parents selected by the enclosing query."); + return (new QueryAction().execute(input).getRecords()); + } + +} diff --git a/src/main/java/com/kingsrook/qbits/collaboration/customfields/CustomFieldsProvider.java b/src/main/java/com/kingsrook/qbits/collaboration/customfields/CustomFieldsProvider.java index 96df5f6..91ea2a7 100644 --- a/src/main/java/com/kingsrook/qbits/collaboration/customfields/CustomFieldsProvider.java +++ b/src/main/java/com/kingsrook/qbits/collaboration/customfields/CustomFieldsProvider.java @@ -122,7 +122,8 @@ public class CustomFieldsProvider implements RuntimeMetaDataProviderInterface String name = "cf_" + definition.getValueString("name"); QFieldType type = type(definition.getValueString("type")); QFieldMetaData field = json ? new QFieldMetaData(name, type).withBackendStorage(new JsonPathStorage().withColumnName("qqq_custom_fields").withKey(definition.getValueString("name"))) - : new QVirtualFieldMetaData(name, type).withIsEditable(false).withIsQueryCriteria(true).withIsQuerySelectable(false) + : new QVirtualFieldMetaData(name, type).withIsEditable(false).withIsQueryCriteria(true).withIsQuerySelectable(true) + .withValueProvider(new QCodeReference(CustomFieldValueProvider.class)) .withCriteriaExpander(new QCodeReference(FieldCriteriaExpander.class)); field.withLabel(definition.getValueString("label")).withIsRequired(json && Boolean.TRUE.equals(definition.getValueBoolean("required"))) .withMaxLength(definition.getValueInteger("maxLength")); diff --git a/src/main/java/com/kingsrook/qbits/collaboration/tags/TagValueProvider.java b/src/main/java/com/kingsrook/qbits/collaboration/tags/TagValueProvider.java new file mode 100644 index 0000000..4bccd84 --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/collaboration/tags/TagValueProvider.java @@ -0,0 +1,90 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + + +package com.kingsrook.qbits.collaboration.tags; + +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import com.kingsrook.qbits.collaboration.CollaborationRecords; +import com.kingsrook.qbits.collaboration.model.CollaborationTables; +import com.kingsrook.qqq.backend.core.actions.tables.QueryAction; +import com.kingsrook.qqq.backend.core.actions.tables.query.VirtualFieldValueProviderInterface; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordReferenceTableMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData; +import com.kingsrook.qqq.backend.core.utils.CollectionUtils; + +/******************************************************************************* + ** Return approved tag labels for parents selected by the enclosing query. + *******************************************************************************/ +public class TagValueProvider implements VirtualFieldValueProviderInterface +{ + /******************************************************************************* + ** Batch both assignments and vocabulary; never recursively load parent records. + *******************************************************************************/ + @Override + public void fill(QTableMetaData table, String fieldName, List records, QueryInput origin) throws QException + { + for(List page : CollectionUtils.getPages(records, 500)) + { + List ids = page.stream().map(record -> String.valueOf(record.getValue(table.getPrimaryKeyField()))).distinct().toList(); + QTableMetaData metadata = QContext.getQInstance().getTable(CollaborationTables.RECORD_TAG).clone(); + metadata.withCustomizers(null); + metadata.getSupplementalMetaData().remove(RecordReferenceTableMetaData.NAME); + QueryInput input = new QueryInput(CollaborationTables.RECORD_TAG).withTransaction(origin.getTransaction()).withFilter(new QQueryFilter() + .withCriteria("tableName", QCriteriaOperator.EQUALS, table.getName()).withCriteria("recordId", QCriteriaOperator.IN, ids) + .withCriteria("parentDeleted", QCriteriaOperator.EQUALS, false)); + input.setTableMetaData(metadata); + input.withPermissionBypass("Tag provider reads only parents selected by the enclosing query."); + List assignments = new QueryAction().execute(input).getRecords(); + Map labels = new HashMap<>(); + List tagIds = assignments.stream().map(row -> row.getValueInteger("tagId")).distinct().toList(); + for(List tags : CollectionUtils.getPages(tagIds, 500)) + { + for(QRecord tag : CollaborationRecords.query(CollaborationTables.TAG, TagsService.vocabularyFilter(table.getName()) + .withCriteria("approved", QCriteriaOperator.EQUALS, true).withCriteria("id", QCriteriaOperator.IN, tags), origin.getTransaction())) + { + labels.put(tag.getValueInteger("id"), tag.getValueString("label")); + } + } + Map> byParent = new HashMap<>(); + for(QRecord assignment : assignments) + { + String label = labels.get(assignment.getValueInteger("tagId")); + if(label != null) + { + byParent.computeIfAbsent(assignment.getValueString("recordId"), key -> new java.util.TreeSet<>()).add(label); + } + } + for(QRecord record : page) + { + var values = byParent.get(String.valueOf(record.getValue(table.getPrimaryKeyField()))); + record.setValue(fieldName, values == null ? null : String.join(", ", values)); + } + } + } +} diff --git a/src/main/java/com/kingsrook/qbits/collaboration/tags/TagsMetaDataProvider.java b/src/main/java/com/kingsrook/qbits/collaboration/tags/TagsMetaDataProvider.java index f4ac096..51cdfa8 100644 --- a/src/main/java/com/kingsrook/qbits/collaboration/tags/TagsMetaDataProvider.java +++ b/src/main/java/com/kingsrook/qbits/collaboration/tags/TagsMetaDataProvider.java @@ -56,7 +56,8 @@ public class TagsMetaDataProvider implements RuntimeMetaDataProviderInterface { contribution.getTables().put(table.getName(), new RuntimeTableMetaDataContribution() .withVirtualFields(List.of(new QVirtualFieldMetaData("collaborationTags", QFieldType.STRING).withLabel("Tags") - .withPossibleValueSourceName(CollaborationTables.TAG).withIsQueryCriteria(true).withIsQuerySelectable(false) + .withPossibleValueSourceName(CollaborationTables.TAG).withIsQueryCriteria(true).withIsQuerySelectable(true) + .withValueProvider(new QCodeReference(TagValueProvider.class)).withShouldTranslatePossibleValues(false) .withCriteriaExpander(new QCodeReference(TagCriteriaExpander.class)))) .withSearchFields(List.of("collaborationTags"))); } diff --git a/src/test/java/com/kingsrook/qbits/collaboration/CollaborationApiHttpTest.java b/src/test/java/com/kingsrook/qbits/collaboration/CollaborationApiHttpTest.java index 6d86d75..c6d1c60 100644 --- a/src/test/java/com/kingsrook/qbits/collaboration/CollaborationApiHttpTest.java +++ b/src/test/java/com/kingsrook/qbits/collaboration/CollaborationApiHttpTest.java @@ -39,6 +39,12 @@ import com.fasterxml.jackson.databind.JsonNode; import com.fasterxml.jackson.databind.ObjectMapper; import com.fasterxml.jackson.databind.node.ObjectNode; import com.kingsrook.qbits.collaboration.attachments.AttachmentStorage; +import com.kingsrook.qbits.collaboration.customfields.FieldDefinitions; +import com.kingsrook.qbits.collaboration.model.CollaborationTables; +import com.kingsrook.qqq.backend.core.actions.tables.InsertAction; +import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData; import com.kingsrook.qbits.objectstorage.api.PresignGet; import com.kingsrook.qbits.objectstorage.api.PutRequest; import com.kingsrook.qqq.backend.core.actions.collaboration.RuntimeMetaDataOverlay; @@ -384,6 +390,80 @@ class CollaborationApiHttpTest extends PersonalServiceFixture assertThat(tagQuery("IN", "[\"alpha\"]")).containsExactlyInAnyOrder(1, 2); } + /******************************************************************************* + ** EAV choices retain their stored ids and translated labels on the HTTP wire. + *******************************************************************************/ + @Test + void customFieldValuesThroughTableReads() throws Exception + { + admin(); + instance.addBackend(new RDBMSBackendMetaData().withName("fallback").withVendor("h2")); + instance.getTable("document").setBackendName("fallback"); + QRecord definition = new InsertAction().execute(new InsertInput(CollaborationTables.FIELD_DEFINITION).withRecords(List.of( + new QRecord().withValue("tableName", "document").withValue("name", "choice").withValue("label", "Choice").withValue("type", "SELECT") + .withValue("options", "[{\"id\":\"x\",\"label\":\"Choice X\"}]")))).getRecords().getFirst(); + assertThat(definition.getErrors()).isNullOrEmpty(); + FieldDefinitions.apply(definition.getValue("id"), "Publish"); + instance.getTable("document").setBackendName("memory"); + CollaborationRecords.insert(CollaborationTables.FIELD_VALUE, new QRecord().withValue("tableName", "document").withValue("recordId", "1") + .withValue("fieldName", "choice").withValue("valueText", "x").withValue("parentDeleted", false)); + login("alice"); + String serverUrl = baseUrl.substring(0, baseUrl.indexOf("/qbit/")); + HttpResponse response = client.send(HttpRequest.newBuilder(URI.create(serverUrl + "/table/document/query")).timeout(Duration.ofSeconds(20)) + .header("Cookie", "sessionUUID=alice" + sessionSuffix).header("Content-Type", "application/json") + .POST(HttpRequest.BodyPublishers.ofString("{\"filter\":{\"orderBys\":[{\"fieldName\":\"id\"}]}}")).build(), HttpResponse.BodyHandlers.ofString()); + JsonNode records = json(response, 200).get("records"); + assertThat(records).hasSize(2); + assertThat(records.get(0).at("/values/id").asInt()).isEqualTo(1); + assertThat(records.get(0).at("/values/cf_choice").asText()).isEqualTo("x"); + assertThat(records.get(0).at("/displayValues/cf_choice").asText()).isEqualTo("Choice X"); + assertThat(records.get(1).at("/values/cf_choice").isMissingNode()).isTrue(); + + response = client.send(HttpRequest.newBuilder(URI.create(serverUrl + "/table/document/1")).timeout(Duration.ofSeconds(20)) + .header("Cookie", "sessionUUID=alice" + sessionSuffix).GET().build(), HttpResponse.BodyHandlers.ofString()); + JsonNode record = json(response, 200).get("record"); + assertThat(record.at("/values/cf_choice").asText()).isEqualTo("x"); + assertThat(record.at("/displayValues/cf_choice").asText()).isEqualTo("Choice X"); + + response = client.send(HttpRequest.newBuilder(URI.create(serverUrl + "/table/document/1")).timeout(Duration.ofSeconds(20)) + .header("Cookie", "sessionUUID=outsider" + sessionSuffix).GET().build(), HttpResponse.BodyHandlers.ofString()); + assertThat(response.statusCode()).isEqualTo(404); + assertThat(response.body()).doesNotContain("Choice X", "First document"); + } + + /******************************************************************************* + ** Table reads expose tag labels while retaining parent-record authorization. + *******************************************************************************/ + @Test + void tagValuesThroughTableReads() throws Exception + { + RuntimeMetaDataOverlay.refresh(instance); + int tag = ok("POST", "/tags", "{\"table\":\"document\",\"label\":\"Review, urgent\"}").get("id").asInt(); + ok("PUT", "/records/document/1/tags", "{\"add\":[" + tag + "]}"); + String serverUrl = baseUrl.substring(0, baseUrl.indexOf("/qbit/")); + HttpResponse response = client.send(HttpRequest.newBuilder(URI.create(serverUrl + "/table/document/query")).timeout(Duration.ofSeconds(20)) + .header("Cookie", "sessionUUID=alice" + sessionSuffix).header("Content-Type", "application/json") + .POST(HttpRequest.BodyPublishers.ofString("{\"filter\":{\"orderBys\":[{\"fieldName\":\"id\"}]}}")).build(), HttpResponse.BodyHandlers.ofString()); + JsonNode records = json(response, 200).get("records"); + assertThat(records).hasSize(2); + assertThat(records.get(0).at("/values/id").asInt()).isEqualTo(1); + assertThat(records.get(0).at("/values/collaborationTags").asText()).isEqualTo("Review, urgent"); + assertThat(records.get(0).at("/displayValues/collaborationTags").asText()).isEqualTo("Review, urgent"); + assertThat(records.get(1).at("/values/id").asInt()).isEqualTo(2); + assertThat(records.get(1).at("/values/collaborationTags").isMissingNode()).as(records.toString()).isTrue(); + + response = client.send(HttpRequest.newBuilder(URI.create(serverUrl + "/table/document/1")).timeout(Duration.ofSeconds(20)) + .header("Cookie", "sessionUUID=alice" + sessionSuffix).GET().build(), HttpResponse.BodyHandlers.ofString()); + JsonNode record = json(response, 200).get("record"); + assertThat(record.at("/values/collaborationTags").asText()).isEqualTo("Review, urgent"); + assertThat(record.at("/displayValues/collaborationTags").asText()).isEqualTo("Review, urgent"); + + response = client.send(HttpRequest.newBuilder(URI.create(serverUrl + "/table/document/1")).timeout(Duration.ofSeconds(20)) + .header("Cookie", "sessionUUID=outsider" + sessionSuffix).GET().build(), HttpResponse.BodyHandlers.ofString()); + assertThat(response.body()).doesNotContain("Review, urgent", "First document"); + assertThat(response.statusCode()).isEqualTo(404); + } + /******************************************************************************* ** Run the v1 table query filtered on the virtual tags field; return matching ids. *******************************************************************************/ diff --git a/src/test/java/com/kingsrook/qbits/collaboration/CoreContractsTest.java b/src/test/java/com/kingsrook/qbits/collaboration/CoreContractsTest.java index d4573a0..9778328 100644 --- a/src/test/java/com/kingsrook/qbits/collaboration/CoreContractsTest.java +++ b/src/test/java/com/kingsrook/qbits/collaboration/CoreContractsTest.java @@ -84,7 +84,8 @@ class CoreContractsTest extends PersonalServiceFixture RuntimeMetaDataOverlay.refresh(instance); long generation = RuntimeMetaDataOverlay.getGeneration(instance); assertThat(instance.getTable("document").getVirtualFields()).containsKey("collaborationTags"); - assertThat(instance.getTable("document").getVirtualField("collaborationTags").getIsQuerySelectable()).isFalse(); + assertThat(instance.getTable("document").getVirtualField("collaborationTags").getIsQuerySelectable()).isTrue(); + assertThat(instance.getTable("document").getVirtualField("collaborationTags").getValueProvider()).isNotNull(); QBackendTransaction tx = new QBackendTransaction(); var row = new InsertAction().execute(new InsertInput(CollaborationTables.FEATURE_SETTING).withTransaction(tx).withRecords(List.of( new QRecord().withValue("scopeType", "TABLE").withValue("scopeName", "document").withValue("feature", "TAGS") diff --git a/src/test/java/com/kingsrook/qbits/collaboration/CustomFieldValuesPostgresTest.java b/src/test/java/com/kingsrook/qbits/collaboration/CustomFieldValuesPostgresTest.java new file mode 100644 index 0000000..34c07e7 --- /dev/null +++ b/src/test/java/com/kingsrook/qbits/collaboration/CustomFieldValuesPostgresTest.java @@ -0,0 +1,188 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.collaboration; + +import java.sql.Connection; +import java.sql.DriverManager; +import java.util.Set; +import com.kingsrook.qbits.collaboration.customfields.CustomFieldValueProvider; +import com.kingsrook.qbits.collaboration.model.CollaborationTables; +import com.kingsrook.qqq.backend.core.actions.tables.QueryAction; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference; +import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType; +import com.kingsrook.qqq.backend.core.model.metadata.fields.QVirtualFieldMetaData; +import com.kingsrook.qqq.backend.module.postgres.model.metadata.PostgreSQLBackendMetaData; +import com.kingsrook.qqq.backend.module.rdbms.actions.RDBMSTransaction; +import com.kingsrook.qqq.backend.module.rdbms.jdbc.ConnectionManager; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.ValueSource; +import org.testcontainers.containers.GenericContainer; +import org.testcontainers.containers.wait.strategy.Wait; +import static org.assertj.core.api.Assertions.assertThat; + +/******************************************************************************* + ** Verify provider reads using a real database and explicit caller transactions. + *******************************************************************************/ +class CustomFieldValuesPostgresTest extends PersonalServiceFixture +{ + private static GenericContainer postgres; + + /******************************************************************************* + ** Own an isolated database; never connect to an application database. + *******************************************************************************/ + @BeforeAll + static void startPostgres() + { + postgres = new GenericContainer<>("postgres:17-alpine").withEnv("POSTGRES_PASSWORD", "eav-fixture-only") + .withExposedPorts(5432).waitingFor(Wait.forLogMessage(".*database system is ready to accept connections.*", 2)); + postgres.start(); + } + + /******************************************************************************* + ** Stop only the fixture's database. + *******************************************************************************/ + @AfterAll + static void stopPostgres() + { + if(postgres != null) + { + postgres.stop(); + } + } + + /******************************************************************************* + ** Reuse the secured host metadata with real PostgreSQL storage. + *******************************************************************************/ + @Override + @BeforeEach + public void setUp() throws Exception + { + super.setUp(); + instance.getBackends().remove("memory"); + instance.addBackend(new PostgreSQLBackendMetaData().withName("memory").withHostName(postgres.getHost()) + .withPort(postgres.getMappedPort(5432)).withDatabaseName("postgres").withUsername("postgres").withPassword("eav-fixture-only")); + try(Connection connection = connect(); var sql = connection.createStatement()) + { + sql.execute("DROP SCHEMA public CASCADE; CREATE SCHEMA public"); + sql.execute(CollaborationSchema.postgresScript(config)); + sql.execute("CREATE TABLE document (id INTEGER PRIMARY KEY, title TEXT, tenant TEXT)"); + sql.execute("INSERT INTO document VALUES (1, 'Visible', 'a'), (3, 'Private', 'b')"); + } + CollaborationRecords.insert(CollaborationTables.FIELD_DEFINITION, new QRecord().withValue("tableName", "document") + .withValue("name", "rating").withValue("label", "Rating").withValue("type", "INTEGER").withValue("status", "ACTIVE")); + instance.getTable("document").withVirtualField(new QVirtualFieldMetaData("cf_rating", QFieldType.INTEGER).withLabel("Rating") + .withIsQuerySelectable(true).withValueProvider(new QCodeReference(CustomFieldValueProvider.class))); + } + + /******************************************************************************* + ** Release transaction state and pooled connections before parent fixture cleanup. + *******************************************************************************/ + @Override + public void tearDown() + { + QContext.clearTransaction(); + ConnectionManager.closeConnectionProviders(instance); + super.tearDown(); + } + + /******************************************************************************* + ** A separate connection establishes committed visibility independently. + *******************************************************************************/ + private Connection connect() throws Exception + { + return (DriverManager.getConnection("jdbc:postgresql://" + postgres.getHost() + ":" + postgres.getMappedPort(5432) + + "/postgres", "postgres", "eav-fixture-only")); + } + + /******************************************************************************* + ** Both vocabulary and assignments use the explicit transaction's visibility. + *******************************************************************************/ + @ParameterizedTest + @ValueSource(booleans = {false, true}) + void tagLabelsSeePendingVocabularyAndCompletion(boolean commit) throws Exception + { + instance.getTable("document").withVirtualField(new QVirtualFieldMetaData("collaborationTags", QFieldType.STRING).withLabel("Tags") + .withIsQuerySelectable(true).withPossibleValueSourceName(CollaborationTables.TAG).withShouldTranslatePossibleValues(false) + .withValueProvider(new QCodeReference(com.kingsrook.qbits.collaboration.tags.TagValueProvider.class))); + var tags = new com.kingsrook.qbits.collaboration.tags.TagsService(); + try(var transaction = new RDBMSTransaction(connect())) + { + QContext.setTransaction(transaction); + var tag = tags.create("document", "Pending transaction label", null); + tags.apply("document", 1, java.util.List.of(tag.id()), java.util.List.of()); + QContext.clearTransaction(); + assertThat(QContext.getQBackendTransaction()).isNull(); + var pending = new QueryAction().execute(new QueryInput("document").withFieldNamesToInclude(Set.of("collaborationTags")) + .withShouldTranslatePossibleValues(true).withShouldGenerateDisplayValues(true).withTransaction(transaction)).getRecords(); + assertThat(pending).hasSize(1); + assertThat(pending.getFirst().getValues()).containsOnlyKeys("collaborationTags").containsEntry("collaborationTags", "Pending transaction label"); + assertThat(pending.getFirst().getDisplayValue("collaborationTags")).isEqualTo("Pending transaction label"); + assertThat(new QueryAction().execute(new QueryInput("document")).getRecords().getFirst().getValue("collaborationTags")).isNull(); + if(commit) + { + transaction.commit(); + } + else + { + transaction.rollback(); + } + } + assertThat(new QueryAction().execute(new QueryInput("document")).getRecords().getFirst().getValue("collaborationTags")) + .isEqualTo(commit ? "Pending transaction label" : null); + } + + /******************************************************************************* + ** Explicit query transactions see pending values even without thread context. + *******************************************************************************/ + @ParameterizedTest + @ValueSource(booleans = {false, true}) + void explicitTransactionSeesPendingValueAndCompletion(boolean commit) throws Exception + { + try(var transaction = new RDBMSTransaction(connect())) + { + QContext.setTransaction(transaction); + CollaborationRecords.insert(CollaborationTables.FIELD_VALUE, new QRecord().withValue("tableName", "document").withValue("recordId", "1") + .withValue("fieldName", "rating").withValue("valueNumber", 42).withValue("parentDeleted", false)); + QContext.clearTransaction(); + assertThat(QContext.getQBackendTransaction()).isNull(); + var pending = new QueryAction().execute(new QueryInput("document").withFieldNamesToInclude(Set.of("cf_rating")) + .withTransaction(transaction)).getRecords(); + assertThat(pending).hasSize(1); + assertThat(pending.getFirst().getValues()).containsOnlyKeys("cf_rating").containsEntry("cf_rating", 42); + assertThat(new QueryAction().execute(new QueryInput("document")).getRecords().getFirst().getValue("cf_rating")).isNull(); + if(commit) + { + transaction.commit(); + } + else + { + transaction.rollback(); + } + } + assertThat(new QueryAction().execute(new QueryInput("document")).getRecords().getFirst().getValue("cf_rating")).isEqualTo(commit ? 42 : null); + } +} diff --git a/src/test/java/com/kingsrook/qbits/collaboration/CustomFieldsTest.java b/src/test/java/com/kingsrook/qbits/collaboration/CustomFieldsTest.java index 2fde99c..d986eb4 100644 --- a/src/test/java/com/kingsrook/qbits/collaboration/CustomFieldsTest.java +++ b/src/test/java/com/kingsrook/qbits/collaboration/CustomFieldsTest.java @@ -51,6 +51,9 @@ import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaDat import com.kingsrook.qbits.collaboration.app.CollaborationHome; import com.kingsrook.qqq.backend.core.model.dashboard.widgets.TableData; import org.junit.jupiter.api.Test; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.Arguments; +import org.junit.jupiter.params.provider.MethodSource; import static org.assertj.core.api.Assertions.assertThat; import static org.assertj.core.api.Assertions.assertThatThrownBy; @@ -242,6 +245,59 @@ public class CustomFieldsTest extends PersonalServiceFixture assertThat(FieldProcesses.metadata()).extracting(process -> process.getLabel()).containsExactly("Publish field", "Archive field", "Rebuild index"); } + /******************************************************************************* + ** Typed EAV storage returns the declared Java value through an actual query. + *******************************************************************************/ + @ParameterizedTest + @MethodSource("eavValueTypes") + void eavValueTypes(String type, String column, java.io.Serializable stored, java.io.Serializable expected) throws Exception + { + admin(); + instance.addBackend(new RDBMSBackendMetaData().withName("fallback").withVendor("h2")); + instance.getTable("document").setBackendName("fallback"); + QRecord draft = new QRecord().withValue("tableName", "document").withValue("name", "typed").withValue("label", "Typed").withValue("type", type); + if(List.of("SELECT", "MULTI_SELECT").contains(type)) + { + draft.withValue("options", "[{\"id\":\"x\",\"label\":\"Choice X\"}]"); + } + if("RECORD_LINK".equals(type)) + { + draft.withValue("linkTableName", CollaborationTables.USER); + } + QRecord definition = new InsertAction().execute(new InsertInput(CollaborationTables.FIELD_DEFINITION).withRecords(List.of(draft))).getRecords().getFirst(); + assertThat(definition.getErrors()).isNullOrEmpty(); + FieldDefinitions.apply(definition.getValue("id"), "Publish"); + instance.getTable("document").setBackendName("memory"); + CollaborationRecords.insert(CollaborationTables.FIELD_VALUE, new QRecord().withValue("tableName", "document").withValue("recordId", "1") + .withValue("fieldName", "typed").withValue(column, stored).withValue("parentDeleted", false)); + login("alice"); + var records = new QueryAction().execute(new QueryInput("document").withFieldNamesToInclude(java.util.Set.of("cf_typed"))).getRecords(); + assertThat(records).hasSize(2); + assertThat(records).allSatisfy(record -> assertThat(record.getValues()).containsOnlyKeys("cf_typed")); + assertThat(records).extracting(record -> record.getValue("cf_typed")).containsExactlyInAnyOrder(expected, null); + Object actual = records.stream().map(record -> record.getValue("cf_typed")).filter(java.util.Objects::nonNull).findFirst().orElseThrow(); + assertThat(actual).isExactlyInstanceOf(expected.getClass()); + } + + /******************************************************************************* + ** Keep expectations independent of the provider's column-mapping helper. + *******************************************************************************/ + static java.util.stream.Stream eavValueTypes() + { + return (java.util.stream.Stream.of( + Arguments.of("INTEGER", "valueNumber", new BigDecimal("7.0"), 7), + Arguments.of("DECIMAL", "valueNumber", new BigDecimal("12.345"), new BigDecimal("12.345")), + Arguments.of("BOOLEAN", "valueBoolean", false, false), + Arguments.of("DATE", "valueDate", java.time.LocalDate.of(2026, 10, 22), java.time.LocalDate.of(2026, 10, 22)), + Arguments.of("DATE_TIME", "valueDateTime", java.time.Instant.parse("2026-10-22T12:34:56Z"), java.time.Instant.parse("2026-10-22T12:34:56Z")), + Arguments.of("SELECT", "valueText", "x", "x"), + Arguments.of("MULTI_SELECT", "valueText", "[\"x\"]", "[\"x\"]"), + Arguments.of("USER", "valueText", "alice", "alice"), + Arguments.of("RECORD_LINK", "valueText", "alice", "alice"), + Arguments.of("STRING", "valueText", "A, B", "A, B"), + Arguments.of("TEXT", "valueText", "First\nSecond", "First\nSecond"))); + } + /******************************************************************************* ** An unsupported JSON backend gets typed, read-only subquery filters. *******************************************************************************/ @@ -256,18 +312,50 @@ public class CustomFieldsTest extends PersonalServiceFixture instance.getTable("document").setBackendName("memory"); var virtual = instance.getTable("document").getVirtualField("cf_rating"); assertThat(virtual).isNotNull(); - assertThat(virtual.getIsQuerySelectable()).isFalse(); + assertThat(virtual.getIsQuerySelectable()).isTrue(); assertThat(virtual.getIsEditable()).isFalse(); assertThat(FieldDefinitions.get(definition.getValue("id")).getValueString("storage")).isEqualTo("EAV"); CollaborationRecords.insert(CollaborationTables.FIELD_VALUE, new QRecord().withValue("tableName", "document").withValue("recordId", "1") .withValue("fieldName", "rating").withValue("valueNumber", new BigDecimal("7")).withValue("parentDeleted", false)); + QContext.getQSession().withSecurityKeyValue("allTenants", true); + CollaborationRecords.insert(CollaborationTables.FIELD_VALUE, new QRecord().withValue("tableName", "document").withValue("recordId", "3") + .withValue("fieldName", "rating").withValue("valueNumber", new BigDecimal("99")).withValue("parentDeleted", false)); login("alice"); + var selected = new QueryAction().execute(new QueryInput("document").withFieldNamesToInclude(java.util.Set.of("cf_rating"))).getRecords(); + assertThat(selected).hasSize(2); + assertThat(selected).allSatisfy(row -> assertThat(row.getValues()).containsOnlyKeys("cf_rating")); + assertThat(selected).extracting(row -> row.getValue("cf_rating")).containsExactlyInAnyOrder(7, null); assertThat(query("cf_rating", QCriteriaOperator.GREATER_THAN, 5)).extracting(row -> row.getValueInteger("id")).containsExactly(1); assertThat(query("cf_rating", QCriteriaOperator.NOT_EQUALS, 7)).extracting(row -> row.getValueInteger("id")).containsExactly(2); assertThat(new QueryAction().execute(new QueryInput("document").withFilter(new QQueryFilter().withCriteria("cf_rating", QCriteriaOperator.IS_BLANK))).getRecords()) .extracting(row -> row.getValueInteger("id")).containsExactly(2); + // Reuse the same session and query to prove results are not cached across access changes. + QueryInput projected = new QueryInput("document").withFieldNamesToInclude(java.util.Set.of("cf_rating")); + QContext.getQSession().getSecurityKeyValues().put("tenant", List.of("b")); + assertThat(new QueryAction().execute(projected).getRecords()).extracting(row -> row.getValue("cf_rating")).containsExactly(99); + QContext.getQSession().getSecurityKeyValues().put("tenant", List.of("a")); + assertThat(new QueryAction().execute(projected).getRecords()).extracting(row -> row.getValue("cf_rating")).containsExactlyInAnyOrder(7, null); + QContext.getQSession().setPermissions(java.util.Set.of()); + assertThatThrownBy(() -> new QueryAction().execute(projected)).isInstanceOf(com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException.class); + login("alice"); + assertThat(com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordReferenceTableMetaData.of(instance.getTable(CollaborationTables.FIELD_VALUE))).isNotNull(); + assertThat(new QueryAction().execute(new QueryInput(CollaborationTables.FIELD_VALUE) + .withFilter(new QQueryFilter().withCriteria("tableName", QCriteriaOperator.EQUALS, "document") + .withCriteria("recordId", QCriteriaOperator.IN, List.of("1", "3")))).getRecords()) + .extracting(row -> row.getValueString("recordId")).containsExactly("1"); + virtual.setIsHidden(true); + assertThat(new QueryAction().execute(new QueryInput("document")).getRecords()).allSatisfy(row -> assertThat(row.getValues()).doesNotContainKey("cf_rating")); + virtual.setIsHidden(false); + admin(); + QContext.getQSession().withSecurityKeyValue("allTenants", true); + assertThat(FieldDefinitions.valueCount(FieldDefinitions.get(definition.getValue("id")))).isEqualTo(2); + QRecord deleted = CollaborationRecords.insert(CollaborationTables.FIELD_VALUE, new QRecord().withValue("tableName", "document").withValue("recordId", "2") + .withValue("fieldName", "rating").withValue("valueNumber", new BigDecimal("88")).withValue("parentDeleted", true)); + assertThat(deleted.getValueBoolean("parentDeleted")).isTrue(); + login("alice"); + assertThat(new QueryAction().execute(new QueryInput("document").withFieldNamesToInclude(java.util.Set.of("cf_rating"))).getRecords()) + .extracting(row -> row.getValue("cf_rating")).containsExactlyInAnyOrder(7, null); admin(); - assertThat(FieldDefinitions.valueCount(FieldDefinitions.get(definition.getValue("id")))).isEqualTo(1); var expander = new com.kingsrook.qbits.collaboration.customfields.FieldCriteriaExpander(); assertThat(expander.expandSearchTerm(instance.getTable("document"), "text", com.kingsrook.qqq.backend.core.actions.tables.query.SearchTermKind.TEXT, null)).isNotNull(); assertThat(com.kingsrook.qbits.collaboration.customfields.FieldCriteriaExpander.valueColumn("DATE")).isEqualTo("valueDate"); diff --git a/src/test/java/com/kingsrook/qbits/collaboration/TagsServiceTest.java b/src/test/java/com/kingsrook/qbits/collaboration/TagsServiceTest.java index 85ca704..63348c5 100644 --- a/src/test/java/com/kingsrook/qbits/collaboration/TagsServiceTest.java +++ b/src/test/java/com/kingsrook/qbits/collaboration/TagsServiceTest.java @@ -73,6 +73,126 @@ public class TagsServiceTest extends PersonalServiceFixture private final TagsService tags = new TagsService(); + /******************************************************************************* + ** Tag labels are selectable while the ID-based filter picker remains intact. + *******************************************************************************/ + @Test + void selectsTagLabelsWithoutTranslatingThemAsIds() throws Exception + { + TagEntry beta = tags.create("document", "Beta", null); + TagEntry alpha = tags.create("document", "Alpha", null); + tags.apply("document", 1, List.of(beta.id(), alpha.id()), List.of()); + com.kingsrook.qqq.backend.core.actions.collaboration.RuntimeMetaDataOverlay.refresh(instance); + var field = instance.getTable("document").getVirtualField("collaborationTags"); + assertThat(field.getIsQuerySelectable()).isTrue(); + assertThat(field.getPossibleValueSourceName()).isEqualTo(CollaborationTables.TAG); + var rows = new QueryAction().execute(new QueryInput("document").withFieldNamesToInclude(java.util.Set.of("collaborationTags")) + .withShouldTranslatePossibleValues(true).withShouldGenerateDisplayValues(true)).getRecords(); + assertThat(rows).hasSize(2); + assertThat(rows).allSatisfy(row -> assertThat(row.getValues()).containsOnlyKeys("collaborationTags")); + assertThat(rows).extracting(row -> row.getValueString("collaborationTags")).containsExactlyInAnyOrder("Alpha, Beta", null); + assertThat(rows).filteredOn(row -> row.getValue("collaborationTags") != null) + .allSatisfy(row -> assertThat(row.getDisplayValue("collaborationTags")).isEqualTo("Alpha, Beta")); + assertThat(new QueryAction().execute(new QueryInput("document").withFilter(new QQueryFilter() + .withCriteria("collaborationTags", QCriteriaOperator.EQUALS, alpha.id()))).getRecords()) + .extracting(row -> row.getValueInteger("id")).containsExactly(1); + } + + /******************************************************************************* + ** Exported labels obey parent visibility and vocabulary approval, with CSV quoting. + *******************************************************************************/ + @Test + void tagLabelsRespectPrivacyAndExportQuoting() throws Exception + { + TagEntry visible = tags.create("document", "Needs, review", null); + TagEntry secret = tags.create("document", "Secret label", null); + tags.apply("document", 1, List.of(visible.id()), List.of()); + QContext.getQSession().withSecurityKeyValue("allTenants", true); + tags.apply("document", 3, List.of(secret.id()), List.of()); + config.withDefaultTagMode("APPROVED_ONLY"); + TagEntry pending = tags.create("document", "Pending label", null); + assertThat(pending.approved()).isFalse(); + // A stale assignment must not turn a pending suggestion into a visible label. + CollaborationRecords.insert(CollaborationTables.RECORD_TAG, new QRecord().withValue("tableName", "document") + .withValue("recordId", "1").withValue("tagId", pending.id()).withValue("createdByUserId", "alice").withValue("parentDeleted", false)); + login("alice"); + com.kingsrook.qqq.backend.core.actions.collaboration.RuntimeMetaDataOverlay.refresh(instance); + var input = new QueryInput("document").withFieldNamesToInclude(java.util.Set.of("collaborationTags")); + assertThat(new QueryAction().execute(input).getRecords()).extracting(row -> row.getValueString("collaborationTags")) + .containsExactlyInAnyOrder("Needs, review", null); + + java.io.ByteArrayOutputStream output = new java.io.ByteArrayOutputStream(); + var export = new com.kingsrook.qqq.backend.core.model.actions.reporting.ExportInput(); + export.setTableName("document"); + export.setFieldNames(List.of("collaborationTags")); + export.setReportDestination(new com.kingsrook.qqq.backend.core.model.actions.reporting.ReportDestination() + .withReportFormat(com.kingsrook.qqq.backend.core.model.actions.reporting.ReportFormat.CSV).withReportOutputStream(output)); + new com.kingsrook.qqq.backend.core.actions.reporting.ExportAction().execute(export); + String csv = output.toString(java.nio.charset.StandardCharsets.UTF_8); + assertThat(csv).doesNotContain("Secret label", "Pending label"); + try(var parser = org.apache.commons.csv.CSVParser.parse(csv, org.apache.commons.csv.CSVFormat.DEFAULT.withFirstRecordAsHeader())) + { + assertThat(parser.getHeaderNames()).containsExactly("Tags"); + assertThat(parser.getRecords()).extracting(row -> row.get("Tags")).containsExactlyInAnyOrder("Needs, review", ""); + } + QContext.getQSession().getSecurityKeyValues().put("tenant", List.of("b")); + assertThat(new QueryAction().execute(input).getRecords()).extracting(row -> row.getValueString("collaborationTags")).containsExactly("Secret label"); + login("alice"); + assertThat(new QueryAction().execute(input).getRecords()).extracting(row -> row.getValueString("collaborationTags")) + .containsExactlyInAnyOrder("Needs, review", null); + } + + /******************************************************************************* + ** A page spanning the provider's batch boundary retains its last parent values. + *******************************************************************************/ + @Test + void tagLabelsSpanMultipleParentBatches() throws Exception + { + java.util.List parents = new java.util.ArrayList<>(); + for(int id = 4; id <= 504; id++) + { + parents.add(new QRecord().withValue("id", id).withValue("title", "Document " + id).withValue("tenant", "a")); + } + assertThat(new InsertAction().execute(new InsertInput("document").withRecords(parents)).getRecords()) + .allSatisfy(row -> assertThat(row.getErrors()).isNullOrEmpty()); + TagEntry tag = tags.create("document", "Boundary", null); + tags.apply("document", 1, List.of(tag.id()), List.of()); + tags.apply("document", 504, List.of(tag.id()), List.of()); + com.kingsrook.qqq.backend.core.actions.collaboration.RuntimeMetaDataOverlay.refresh(instance); + var rows = new QueryAction().execute(new QueryInput("document").withFieldNamesToInclude(java.util.Set.of("id", "collaborationTags"))).getRecords(); + assertThat(rows).hasSize(503); + assertThat(rows).filteredOn(row -> row.getValue("collaborationTags") != null).extracting(row -> row.getValueInteger("id")) + .containsExactlyInAnyOrder(1, 504); + assertThat(rows).filteredOn(row -> row.getValue("collaborationTags") != null) + .allSatisfy(row -> assertThat(row.getValueString("collaborationTags")).isEqualTo("Boundary")); + } + + /******************************************************************************* + ** Stale links cannot expose merged vocabulary, other-table tags, or deleted links. + *******************************************************************************/ + @Test + void tagLabelsExcludeStaleAndOutOfScopeAssignments() throws Exception + { + login("admin", TagsService.MANAGE); + TagEntry global = tags.create(null, "Global", null); + TagEntry current = tags.create("document", "Current", null); + TagEntry merged = tags.create("document", "Merged old label", null); + TagEntry deleted = tags.create("document", "Deleted link label", null); + TagEntry other = tags.create("document", "Other table label", null); + CollaborationRecords.update(CollaborationTables.TAG, new QRecord().withValue("id", merged.id()).withValue("mergedIntoTagId", current.id())); + CollaborationRecords.update(CollaborationTables.TAG, new QRecord().withValue("id", other.id()).withValue("tableName", "different_table")); + for(TagEntry tag : List.of(global, current, merged, deleted, other)) + { + CollaborationRecords.insert(CollaborationTables.RECORD_TAG, new QRecord().withValue("tableName", "document").withValue("recordId", "1") + .withValue("tagId", tag.id()).withValue("createdByUserId", "admin").withValue("parentDeleted", tag.id().equals(deleted.id()))); + } + login("alice"); + com.kingsrook.qqq.backend.core.actions.collaboration.RuntimeMetaDataOverlay.refresh(instance); + var rows = new QueryAction().execute(new QueryInput("document").withFieldNamesToInclude(java.util.Set.of("collaborationTags"))).getRecords(); + assertThat(rows).extracting(row -> row.getValueString("collaborationTags")).containsExactlyInAnyOrder("Current, Global", null); + assertThat(com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordReferenceTableMetaData.of(instance.getTable(CollaborationTables.RECORD_TAG))).isNotNull(); + } + /******************************************************************************* ** Natural keys are locale-independent and validate after collapsing whitespace. *******************************************************************************/ -- 2.54.0 From ecff347efe2bb84949b8452a3383e3fdeb4bcede Mon Sep 17 00:00:00 2001 From: James Maes Date: Mon, 26 Oct 2026 08:21:29 -0500 Subject: [PATCH 2/3] build: adopt published virtual field provider core Co-Authored-By: GPT-6 (Codex) --- docs/VIRTUAL-FIELD-UPGRADE.md | 33 ++++++++++++++++++++++++--------- pom.xml | 2 +- 2 files changed, 25 insertions(+), 10 deletions(-) diff --git a/docs/VIRTUAL-FIELD-UPGRADE.md b/docs/VIRTUAL-FIELD-UPGRADE.md index fad4b7f..8271d98 100644 --- a/docs/VIRTUAL-FIELD-UPGRADE.md +++ b/docs/VIRTUAL-FIELD-UPGRADE.md @@ -2,12 +2,26 @@ Tracks [QQQ #1196](https://github.com/QRun-IO/qqq/issues/1196). -This source branch requires the new core virtual-field value-provider API. The -published versions in CANDIDATE-RELEASE.md do not include that API. The checked-in -core pin still identifies the earlier published candidate and must be updated -before this branch can be built or released with its default configuration. +This source branch requires the new core virtual-field value-provider API. Its +default BOM pin is now the published +`4.1.0-virtual-field-values-1196-cc3981bb3-SNAPSHOT`. Core publisher job 9716 +passed from commit `cc3981bb3c95e8f2dd7ef2a02e2233db7d0e2a79`; an independent +consumer with an initially empty Maven repository downloaded its BOM and core +and compiled the provider interface on October 26, 2026. Both artifacts resolve +to timestamp `20261026.130631-1` from Sonatype's snapshot repository. -Local verification uses core artifacts installed as +Published artifact SHA-256 receipts: + +| Artifact | SHA-256 | +| --- | --- | +| BOM POM | `d68cf837d7e07513c4e70977829f99e6236cb3e89f8af85fc75239d5b52564a5` | +| Core POM | `d89d491aaa9312051ec462e1cf165b948ee3e88c9aec2721ff25a00e956c431b` | +| Core JAR | `ae6dbeba8d0e4bc55f2714303ee3482dbf6a0cd34c03663c6009d8fe2c1da54d` | + +This collaboration follow-up is still unpublished. The coordinated versions in +CANDIDATE-RELEASE.md describe the earlier candidate, not this source branch. + +Earlier local verification used core artifacts installed as `4.1.0-values1196-SNAPSHOT`, with `./mvnw.cmd -Dqqq.snapshot.version=4.1.0-values1196-SNAPSHOT verify`. That version exists only in the local Maven repository; it is not an adoption pin. @@ -28,10 +42,11 @@ That version exists only in the local Maven repository; it is not an adoption pi ## Release sequence -1. Publish and verify the coordinated core artifact containing the provider API. -2. Replace this branch's core BOM pin with that published version. -3. Run full collaboration verification against the published prerequisite, - including coverage, HTTP, PostgreSQL transaction and Redis tests. +1. Publish and verify the coordinated core artifact containing the provider API (done). +2. Replace this branch's core BOM pin with that published version (done). +3. Run full collaboration verification against the published prerequisite + (done October 26: 198 tests, no failures/errors/skips; packaging and coverage + passed, including HTTP, PostgreSQL transaction and Redis tests). 4. Publish the collaboration snapshot and verify an independent consumer using published artifacts. Update CANDIDATE-RELEASE.md with the exact pair. 5. Update Kof22 Website-Backend adoption work and qualify it in the selected diff --git a/pom.xml b/pom.xml index 47305d1..2d39aa9 100644 --- a/pom.xml +++ b/pom.xml @@ -252,7 +252,7 @@ true - 4.1.0-collab-33464d37e-SNAPSHOT + 4.1.0-virtual-field-values-1196-cc3981bb3-SNAPSHOT -- 2.54.0 From 1da538cf43ba1ed80e3f8fd599729b5691563cbf Mon Sep 17 00:00:00 2001 From: James Maes Date: Mon, 26 Oct 2026 08:36:16 -0500 Subject: [PATCH 3/3] docs: record published selectable-field snapshot evidence Co-Authored-By: GPT-6 (Codex) --- docs/CANDIDATE-RELEASE.md | 27 ++++++++++++++++++++++++++- docs/VIRTUAL-FIELD-UPGRADE.md | 32 ++++++++++++++++++++++++++------ 2 files changed, 52 insertions(+), 7 deletions(-) diff --git a/docs/CANDIDATE-RELEASE.md b/docs/CANDIDATE-RELEASE.md index 0eee83b..cd76105 100644 --- a/docs/CANDIDATE-RELEASE.md +++ b/docs/CANDIDATE-RELEASE.md @@ -1,6 +1,31 @@ # Coordinated collaboration snapshot -The main-line candidate is published. Use these versions together: +## Selectable-field follow-up snapshot + +The following focused upgrade is published for QQQ #1196: + +| Component | Version | +| --- | --- | +| QQQ BOM and modules | `4.1.0-virtual-field-values-1196-cc3981bb3-SNAPSHOT` | +| qbit-collaboration | `0.1.0-field-values-1196-ecff347-SNAPSHOT` | +| qbit-redis (unchanged prerequisite) | `0.1.0-redis-7fe5492-SNAPSHOT` | + +Import this QQQ BOM in the consuming host. Collaboration publisher 168 passed +from `ecff347efe2bb84949b8452a3383e3fdeb4bcede`; its POM and JAR resolve to +timestamp `20261026.133203-1`. Full collaboration verification against the +published core passed 198 tests with no failures, errors or skips, plus packaging +and coverage gates. An independent empty-cache Java 21 consumer downloaded the +pair and compiled both collaboration value providers against the core API. +See [the upgrade evidence](VIRTUAL-FIELD-UPGRADE.md) for checksums and scope. + +This is not a combined release of the separate attachment, My Work and presence +recovery follow-ups, nor a Kof22 deployment. The earlier candidate's browser +acceptance below must not be treated as browser qualification of this new pair. +A compatible packaged frontend and host staging qualification remain required. + +## Earlier main-line candidate + +The earlier main-line candidate is published. Its coordinated versions are: | Component | Version | | --- | --- | diff --git a/docs/VIRTUAL-FIELD-UPGRADE.md b/docs/VIRTUAL-FIELD-UPGRADE.md index 8271d98..fda061e 100644 --- a/docs/VIRTUAL-FIELD-UPGRADE.md +++ b/docs/VIRTUAL-FIELD-UPGRADE.md @@ -1,4 +1,4 @@ -# Selectable collaboration fields: unreleased upgrade +# Selectable collaboration fields: published feature snapshot Tracks [QQQ #1196](https://github.com/QRun-IO/qqq/issues/1196). @@ -18,8 +18,28 @@ Published artifact SHA-256 receipts: | Core POM | `d89d491aaa9312051ec462e1cf165b948ee3e88c9aec2721ff25a00e956c431b` | | Core JAR | `ae6dbeba8d0e4bc55f2714303ee3482dbf6a0cd34c03663c6009d8fe2c1da54d` | -This collaboration follow-up is still unpublished. The coordinated versions in -CANDIDATE-RELEASE.md describe the earlier candidate, not this source branch. +The collaboration follow-up is published as +`0.1.0-field-values-1196-ecff347-SNAPSHOT`, timestamp +`20261026.133203-1`. Publisher job 168 passed from commit +`ecff347efe2bb84949b8452a3383e3fdeb4bcede`, using signed tag +`publish-values1196-20261026-ecff347`. + +| Collaboration artifact | SHA-256 | +| --- | --- | +| POM | `cc12f59cbdfd4bc53ba545a9f2b6d7c14a316034b107ef5541ddc48df45fe6af` | +| JAR | `284fa370504dd82cee8c87d2b8f5c0cecba8c4b15b32a4f9d3d62a0a062e549d` | + +On October 26, an independent Java 21 consumer downloaded this pair into an +initially empty Maven repository and compiled both `TagValueProvider` and +`CustomFieldValueProvider` as `VirtualFieldValueProviderInterface` instances. +Maven recorded Sonatype `central-portal-snapshots` as the artifact source. The +downloaded collaboration POM pins the core version above; the independently +downloaded core JAR matches the receipt above. This checks published dependency +resolution and API compatibility; runtime behavior is covered by the 198-test +verification below, not by this compilation probe. + +This snapshot does not include the separate attachment, My Work or presence +recovery follow-up PRs. It has not been deployed to Kof22. Earlier local verification used core artifacts installed as `4.1.0-values1196-SNAPSHOT`, with @@ -48,9 +68,9 @@ That version exists only in the local Maven repository; it is not an adoption pi (done October 26: 198 tests, no failures/errors/skips; packaging and coverage passed, including HTTP, PostgreSQL transaction and Redis tests). 4. Publish the collaboration snapshot and verify an independent consumer using - published artifacts. Update CANDIDATE-RELEASE.md with the exact pair. + published artifacts (done; exact pair recorded in CANDIDATE-RELEASE.md). 5. Update Kof22 Website-Backend adoption work and qualify it in the selected isolated staging environment before requesting production deployment. -Local verification is evidence for this implementation, not proof that the -published candidate or Kof22 deployment has these changes. +The published pair contains this implementation. Kof22 adoption, a compatible +packaged frontend and staging qualification remain separate release steps. -- 2.54.0