diff --git a/README.md b/README.md index 1cdb8d3..69af815 100644 --- a/README.md +++ b/README.md @@ -408,7 +408,7 @@ size, pending suggestions, and the five most-used approved tags. `collaborationTags` is a virtual STRING field labeled Tags with possible-value source `collaboration_tag`. `TagCriteriaExpander` supports IN, NOT_IN, EQUALS, NOT_EQUALS, IS_BLANK, IS_NOT_BLANK and CONTAINS_ALL using core `QSubQuery` membership filters. Ordinary quick search matches labels; `#name` matches normalized names. Numeric criteria are tag IDs; string criteria are names. The old ID materializer is removed. Core owns SQL execution or bounded cross-backend materialization; its bound applies to inner rows, not final parent matches. -The runtime provider adds this field and its search entry where TAGS is enabled. It is queryable but not selectable: the overlay has no virtual-field post-query hook and existing host customizers must be preserved. +The runtime provider adds this field and its search entry where TAGS is enabled. With the coordinated core value-provider API, it is selectable in queries, record reads and exports. Labels are sorted, deduplicated and joined with a comma and space; the picker still uses tag IDs. The provider runs after the host post-query customizer without replacing it. This is an unreleased source change: see [virtual-field upgrade status](docs/VIRTUAL-FIELD-UPGRADE.md) before building or adopting this branch. ## UI API @@ -683,8 +683,9 @@ metadata. Memory uses core `JsonPathStorage`. PostgreSQL uses it after a cached `SELECT qqq_custom_fields FROM LIMIT 0` probe succeeds. Missing columns and unsupported backends get read-only EAV virtual filters over `collaboration_field_value`, with -typed `QSubQuery` criteria. These fields are not query-selectable; EAV value writes and projection -are outside this version. `statusMessage` records the fallback reason. After adding a missing +typed `QSubQuery` criteria. The coordinated value-provider API also makes existing EAV values +selectable in queries, record reads and exports, with their declared types and possible-value +labels. EAV writes remain unsupported. See [virtual-field upgrade status](docs/VIRTUAL-FIELD-UPGRADE.md). `statusMessage` records the fallback reason. After adding a missing column, call `CustomFieldStorage.invalidate(instance, tableName)` and refresh the overlay. ### PostgreSQL setup and indexes diff --git a/docs/CANDIDATE-RELEASE.md b/docs/CANDIDATE-RELEASE.md index 0eee83b..cd76105 100644 --- a/docs/CANDIDATE-RELEASE.md +++ b/docs/CANDIDATE-RELEASE.md @@ -1,6 +1,31 @@ # Coordinated collaboration snapshot -The main-line candidate is published. Use these versions together: +## Selectable-field follow-up snapshot + +The following focused upgrade is published for QQQ #1196: + +| Component | Version | +| --- | --- | +| QQQ BOM and modules | `4.1.0-virtual-field-values-1196-cc3981bb3-SNAPSHOT` | +| qbit-collaboration | `0.1.0-field-values-1196-ecff347-SNAPSHOT` | +| qbit-redis (unchanged prerequisite) | `0.1.0-redis-7fe5492-SNAPSHOT` | + +Import this QQQ BOM in the consuming host. Collaboration publisher 168 passed +from `ecff347efe2bb84949b8452a3383e3fdeb4bcede`; its POM and JAR resolve to +timestamp `20261026.133203-1`. Full collaboration verification against the +published core passed 198 tests with no failures, errors or skips, plus packaging +and coverage gates. An independent empty-cache Java 21 consumer downloaded the +pair and compiled both collaboration value providers against the core API. +See [the upgrade evidence](VIRTUAL-FIELD-UPGRADE.md) for checksums and scope. + +This is not a combined release of the separate attachment, My Work and presence +recovery follow-ups, nor a Kof22 deployment. The earlier candidate's browser +acceptance below must not be treated as browser qualification of this new pair. +A compatible packaged frontend and host staging qualification remain required. + +## Earlier main-line candidate + +The earlier main-line candidate is published. Its coordinated versions are: | Component | Version | | --- | --- | diff --git a/docs/VIRTUAL-FIELD-UPGRADE.md b/docs/VIRTUAL-FIELD-UPGRADE.md new file mode 100644 index 0000000..fda061e --- /dev/null +++ b/docs/VIRTUAL-FIELD-UPGRADE.md @@ -0,0 +1,76 @@ +# Selectable collaboration fields: published feature snapshot + +Tracks [QQQ #1196](https://github.com/QRun-IO/qqq/issues/1196). + +This source branch requires the new core virtual-field value-provider API. Its +default BOM pin is now the published +`4.1.0-virtual-field-values-1196-cc3981bb3-SNAPSHOT`. Core publisher job 9716 +passed from commit `cc3981bb3c95e8f2dd7ef2a02e2233db7d0e2a79`; an independent +consumer with an initially empty Maven repository downloaded its BOM and core +and compiled the provider interface on October 26, 2026. Both artifacts resolve +to timestamp `20261026.130631-1` from Sonatype's snapshot repository. + +Published artifact SHA-256 receipts: + +| Artifact | SHA-256 | +| --- | --- | +| BOM POM | `d68cf837d7e07513c4e70977829f99e6236cb3e89f8af85fc75239d5b52564a5` | +| Core POM | `d89d491aaa9312051ec462e1cf165b948ee3e88c9aec2721ff25a00e956c431b` | +| Core JAR | `ae6dbeba8d0e4bc55f2714303ee3482dbf6a0cd34c03663c6009d8fe2c1da54d` | + +The collaboration follow-up is published as +`0.1.0-field-values-1196-ecff347-SNAPSHOT`, timestamp +`20261026.133203-1`. Publisher job 168 passed from commit +`ecff347efe2bb84949b8452a3383e3fdeb4bcede`, using signed tag +`publish-values1196-20261026-ecff347`. + +| Collaboration artifact | SHA-256 | +| --- | --- | +| POM | `cc12f59cbdfd4bc53ba545a9f2b6d7c14a316034b107ef5541ddc48df45fe6af` | +| JAR | `284fa370504dd82cee8c87d2b8f5c0cecba8c4b15b32a4f9d3d62a0a062e549d` | + +On October 26, an independent Java 21 consumer downloaded this pair into an +initially empty Maven repository and compiled both `TagValueProvider` and +`CustomFieldValueProvider` as `VirtualFieldValueProviderInterface` instances. +Maven recorded Sonatype `central-portal-snapshots` as the artifact source. The +downloaded collaboration POM pins the core version above; the independently +downloaded core JAR matches the receipt above. This checks published dependency +resolution and API compatibility; runtime behavior is covered by the 198-test +verification below, not by this compilation probe. + +This snapshot does not include the separate attachment, My Work or presence +recovery follow-up PRs. It has not been deployed to Kof22. + +Earlier local verification used core artifacts installed as +`4.1.0-values1196-SNAPSHOT`, with +`./mvnw.cmd -Dqqq.snapshot.version=4.1.0-values1196-SNAPSHOT verify`. +That version exists only in the local Maven repository; it is not an adoption pin. + +## Behavior + +- `collaborationTags` returns sorted, distinct approved labels from global or + matching-table vocabulary. Merged tags and deleted assignments are excluded. + The picker and filters retain their tag IDs; exports contain one Tags column. +- EAV custom fields return typed existing values and ordinary possible-value + display labels. Missing values are null internally and omitted from HTTP value + maps. This change does not enable EAV writes. +- Providers process batches and use the originating transaction. Parent record + authorization and field visibility remain enforced by core. Existing host + post-query customizers remain installed. +- SQL DISTINCT on provider values is explicitly unsupported. Backend primary-key + identity is required; joined heavy primary keys must be fetched explicitly. + +## Release sequence + +1. Publish and verify the coordinated core artifact containing the provider API (done). +2. Replace this branch's core BOM pin with that published version (done). +3. Run full collaboration verification against the published prerequisite + (done October 26: 198 tests, no failures/errors/skips; packaging and coverage + passed, including HTTP, PostgreSQL transaction and Redis tests). +4. Publish the collaboration snapshot and verify an independent consumer using + published artifacts (done; exact pair recorded in CANDIDATE-RELEASE.md). +5. Update Kof22 Website-Backend adoption work and qualify it in the selected + isolated staging environment before requesting production deployment. + +The published pair contains this implementation. Kof22 adoption, a compatible +packaged frontend and staging qualification remain separate release steps. diff --git a/pom.xml b/pom.xml index 32a601d..2d39aa9 100644 --- a/pom.xml +++ b/pom.xml @@ -72,6 +72,11 @@ + + com.kingsrook.qqq + qqq-backend-module-postgres + test + com.kingsrook.qbits qbit-redis @@ -247,7 +252,7 @@ true - 4.1.0-collab-33464d37e-SNAPSHOT + 4.1.0-virtual-field-values-1196-cc3981bb3-SNAPSHOT diff --git a/src/main/java/com/kingsrook/qbits/collaboration/CollaborationRecords.java b/src/main/java/com/kingsrook/qbits/collaboration/CollaborationRecords.java index a396690..f40b112 100644 --- a/src/main/java/com/kingsrook/qbits/collaboration/CollaborationRecords.java +++ b/src/main/java/com/kingsrook/qbits/collaboration/CollaborationRecords.java @@ -23,6 +23,7 @@ package com.kingsrook.qbits.collaboration; import java.io.Serializable; import java.util.List; +import com.kingsrook.qqq.backend.core.actions.QBackendTransaction; import com.kingsrook.qqq.backend.core.actions.tables.DeleteAction; import com.kingsrook.qqq.backend.core.actions.tables.InsertAction; import com.kingsrook.qqq.backend.core.actions.tables.QueryAction; @@ -74,7 +75,15 @@ public final class CollaborationRecords *******************************************************************************/ public static List query(String table, QQueryFilter filter) throws QException { - QueryInput input = new QueryInput(table).withFilter(filter).withShouldFetchHeavyFields(true).withTransaction(QContext.getQBackendTransaction()); + return (query(table, filter, QContext.getQBackendTransaction())); + } + + /******************************************************************************* + ** Provider reads participate in the originating query's explicit transaction. + *******************************************************************************/ + public static List query(String table, QQueryFilter filter, QBackendTransaction transaction) throws QException + { + QueryInput input = new QueryInput(table).withFilter(filter).withShouldFetchHeavyFields(true).withTransaction(transaction); input.setTableMetaData(serviceMetadata(table)); input.withPermissionBypass("Trusted collaboration service performs parent/ownership authorization."); return (new QueryAction().execute(input).getRecords().stream().map(QRecord::new).toList()); diff --git a/src/main/java/com/kingsrook/qbits/collaboration/customfields/CustomFieldStorage.java b/src/main/java/com/kingsrook/qbits/collaboration/customfields/CustomFieldStorage.java index cfc5959..f70b8b7 100644 --- a/src/main/java/com/kingsrook/qbits/collaboration/customfields/CustomFieldStorage.java +++ b/src/main/java/com/kingsrook/qbits/collaboration/customfields/CustomFieldStorage.java @@ -83,7 +83,7 @@ public final class CustomFieldStorage } if(!postgres(table)) { - return (new Availability(false, "Backend does not support JsonPathStorage; EAV fields are read-only filters (values are not selected).")); + return (new Availability(false, "Backend does not support JsonPathStorage; EAV fields support read-only values and filters.")); } try(Connection connection = connection(table); var statement = connection.createStatement()) { @@ -92,7 +92,7 @@ public final class CustomFieldStorage } catch(SQLException | IllegalArgumentException e) { - return (new Availability(false, "JSON column unavailable; EAV fields are read-only filters. " + e.getMessage())); + return (new Availability(false, "JSON column unavailable; EAV fields support read-only values and filters. " + e.getMessage())); } } diff --git a/src/main/java/com/kingsrook/qbits/collaboration/customfields/CustomFieldValueProvider.java b/src/main/java/com/kingsrook/qbits/collaboration/customfields/CustomFieldValueProvider.java new file mode 100644 index 0000000..b974680 --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/collaboration/customfields/CustomFieldValueProvider.java @@ -0,0 +1,101 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + + +package com.kingsrook.qbits.collaboration.customfields; + +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import com.kingsrook.qbits.collaboration.CollaborationRecords; +import com.kingsrook.qbits.collaboration.model.CollaborationTables; +import com.kingsrook.qqq.backend.core.actions.tables.QueryAction; +import com.kingsrook.qqq.backend.core.actions.tables.query.VirtualFieldValueProviderInterface; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordReferenceTableMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData; +import com.kingsrook.qqq.backend.core.utils.CollectionUtils; +import com.kingsrook.qqq.backend.core.utils.ValueUtils; + +/******************************************************************************* + ** Supply EAV values for the parent records selected by the core query pipeline. + ** Do not re-read parents here: that would recursively execute this provider. + *******************************************************************************/ +public class CustomFieldValueProvider implements VirtualFieldValueProviderInterface +{ + /******************************************************************************* + ** Bound each lookup and retain the caller's transaction through service storage. + *******************************************************************************/ + @Override + public void fill(QTableMetaData table, String fieldName, List records, QueryInput input) throws QException + { + String name = fieldName.substring(3); + var definitions = CollaborationRecords.query(CollaborationTables.FIELD_DEFINITION, new QQueryFilter() + .withCriteria("tableName", QCriteriaOperator.EQUALS, table.getName()).withCriteria("name", QCriteriaOperator.EQUALS, name) + .withCriteria("status", QCriteriaOperator.EQUALS, "ACTIVE"), input.getTransaction()); + if(definitions.size() != 1) + { + throw (new QException("Custom field is unavailable")); + } + String column = FieldCriteriaExpander.valueColumn(definitions.getFirst().getValueString("type")); + for(List page : CollectionUtils.getPages(records, 500)) + { + List ids = page.stream().map(record -> String.valueOf(record.getValue(table.getPrimaryKeyField()))).distinct().toList(); + Map values = new HashMap<>(); + for(QRecord value : valuesForSelectedParents(new QQueryFilter() + .withCriteria("tableName", QCriteriaOperator.EQUALS, table.getName()).withCriteria("fieldName", QCriteriaOperator.EQUALS, name) + .withCriteria("parentDeleted", QCriteriaOperator.EQUALS, false).withCriteria("recordId", QCriteriaOperator.IN, ids), input)) + { + if(values.put(value.getValueString("recordId"), value) != null) + { + throw (new QException("Duplicate custom field values")); + } + } + for(QRecord record : page) + { + QRecord value = values.get(String.valueOf(record.getValue(table.getPrimaryKeyField()))); + record.setValue(fieldName, value == null ? null : ValueUtils.getValueAsFieldType(table.getVirtualField(fieldName).getType(), value.getValue(column))); + } + } + } + + /******************************************************************************* + ** The enclosing query already selected these parents under its access policy. + ** Remove only recursive parent verification on a private child metadata clone; + ** preserve child locks and never install this metadata on the instance. + *******************************************************************************/ + private List valuesForSelectedParents(QQueryFilter filter, QueryInput origin) throws QException + { + QTableMetaData metadata = QContext.getQInstance().getTable(CollaborationTables.FIELD_VALUE).clone(); + metadata.withCustomizers(null); + metadata.getSupplementalMetaData().remove(RecordReferenceTableMetaData.NAME); + QueryInput input = new QueryInput(CollaborationTables.FIELD_VALUE).withFilter(filter) + .withShouldFetchHeavyFields(true).withTransaction(origin.getTransaction()); + input.setTableMetaData(metadata); + input.withPermissionBypass("EAV provider reads only parents selected by the enclosing query."); + return (new QueryAction().execute(input).getRecords()); + } + +} diff --git a/src/main/java/com/kingsrook/qbits/collaboration/customfields/CustomFieldsProvider.java b/src/main/java/com/kingsrook/qbits/collaboration/customfields/CustomFieldsProvider.java index 96df5f6..91ea2a7 100644 --- a/src/main/java/com/kingsrook/qbits/collaboration/customfields/CustomFieldsProvider.java +++ b/src/main/java/com/kingsrook/qbits/collaboration/customfields/CustomFieldsProvider.java @@ -122,7 +122,8 @@ public class CustomFieldsProvider implements RuntimeMetaDataProviderInterface String name = "cf_" + definition.getValueString("name"); QFieldType type = type(definition.getValueString("type")); QFieldMetaData field = json ? new QFieldMetaData(name, type).withBackendStorage(new JsonPathStorage().withColumnName("qqq_custom_fields").withKey(definition.getValueString("name"))) - : new QVirtualFieldMetaData(name, type).withIsEditable(false).withIsQueryCriteria(true).withIsQuerySelectable(false) + : new QVirtualFieldMetaData(name, type).withIsEditable(false).withIsQueryCriteria(true).withIsQuerySelectable(true) + .withValueProvider(new QCodeReference(CustomFieldValueProvider.class)) .withCriteriaExpander(new QCodeReference(FieldCriteriaExpander.class)); field.withLabel(definition.getValueString("label")).withIsRequired(json && Boolean.TRUE.equals(definition.getValueBoolean("required"))) .withMaxLength(definition.getValueInteger("maxLength")); diff --git a/src/main/java/com/kingsrook/qbits/collaboration/tags/TagValueProvider.java b/src/main/java/com/kingsrook/qbits/collaboration/tags/TagValueProvider.java new file mode 100644 index 0000000..4bccd84 --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/collaboration/tags/TagValueProvider.java @@ -0,0 +1,90 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + + +package com.kingsrook.qbits.collaboration.tags; + +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import com.kingsrook.qbits.collaboration.CollaborationRecords; +import com.kingsrook.qbits.collaboration.model.CollaborationTables; +import com.kingsrook.qqq.backend.core.actions.tables.QueryAction; +import com.kingsrook.qqq.backend.core.actions.tables.query.VirtualFieldValueProviderInterface; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordReferenceTableMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData; +import com.kingsrook.qqq.backend.core.utils.CollectionUtils; + +/******************************************************************************* + ** Return approved tag labels for parents selected by the enclosing query. + *******************************************************************************/ +public class TagValueProvider implements VirtualFieldValueProviderInterface +{ + /******************************************************************************* + ** Batch both assignments and vocabulary; never recursively load parent records. + *******************************************************************************/ + @Override + public void fill(QTableMetaData table, String fieldName, List records, QueryInput origin) throws QException + { + for(List page : CollectionUtils.getPages(records, 500)) + { + List ids = page.stream().map(record -> String.valueOf(record.getValue(table.getPrimaryKeyField()))).distinct().toList(); + QTableMetaData metadata = QContext.getQInstance().getTable(CollaborationTables.RECORD_TAG).clone(); + metadata.withCustomizers(null); + metadata.getSupplementalMetaData().remove(RecordReferenceTableMetaData.NAME); + QueryInput input = new QueryInput(CollaborationTables.RECORD_TAG).withTransaction(origin.getTransaction()).withFilter(new QQueryFilter() + .withCriteria("tableName", QCriteriaOperator.EQUALS, table.getName()).withCriteria("recordId", QCriteriaOperator.IN, ids) + .withCriteria("parentDeleted", QCriteriaOperator.EQUALS, false)); + input.setTableMetaData(metadata); + input.withPermissionBypass("Tag provider reads only parents selected by the enclosing query."); + List assignments = new QueryAction().execute(input).getRecords(); + Map labels = new HashMap<>(); + List tagIds = assignments.stream().map(row -> row.getValueInteger("tagId")).distinct().toList(); + for(List tags : CollectionUtils.getPages(tagIds, 500)) + { + for(QRecord tag : CollaborationRecords.query(CollaborationTables.TAG, TagsService.vocabularyFilter(table.getName()) + .withCriteria("approved", QCriteriaOperator.EQUALS, true).withCriteria("id", QCriteriaOperator.IN, tags), origin.getTransaction())) + { + labels.put(tag.getValueInteger("id"), tag.getValueString("label")); + } + } + Map> byParent = new HashMap<>(); + for(QRecord assignment : assignments) + { + String label = labels.get(assignment.getValueInteger("tagId")); + if(label != null) + { + byParent.computeIfAbsent(assignment.getValueString("recordId"), key -> new java.util.TreeSet<>()).add(label); + } + } + for(QRecord record : page) + { + var values = byParent.get(String.valueOf(record.getValue(table.getPrimaryKeyField()))); + record.setValue(fieldName, values == null ? null : String.join(", ", values)); + } + } + } +} diff --git a/src/main/java/com/kingsrook/qbits/collaboration/tags/TagsMetaDataProvider.java b/src/main/java/com/kingsrook/qbits/collaboration/tags/TagsMetaDataProvider.java index f4ac096..51cdfa8 100644 --- a/src/main/java/com/kingsrook/qbits/collaboration/tags/TagsMetaDataProvider.java +++ b/src/main/java/com/kingsrook/qbits/collaboration/tags/TagsMetaDataProvider.java @@ -56,7 +56,8 @@ public class TagsMetaDataProvider implements RuntimeMetaDataProviderInterface { contribution.getTables().put(table.getName(), new RuntimeTableMetaDataContribution() .withVirtualFields(List.of(new QVirtualFieldMetaData("collaborationTags", QFieldType.STRING).withLabel("Tags") - .withPossibleValueSourceName(CollaborationTables.TAG).withIsQueryCriteria(true).withIsQuerySelectable(false) + .withPossibleValueSourceName(CollaborationTables.TAG).withIsQueryCriteria(true).withIsQuerySelectable(true) + .withValueProvider(new QCodeReference(TagValueProvider.class)).withShouldTranslatePossibleValues(false) .withCriteriaExpander(new QCodeReference(TagCriteriaExpander.class)))) .withSearchFields(List.of("collaborationTags"))); } diff --git a/src/test/java/com/kingsrook/qbits/collaboration/CollaborationApiHttpTest.java b/src/test/java/com/kingsrook/qbits/collaboration/CollaborationApiHttpTest.java index 6d86d75..c6d1c60 100644 --- a/src/test/java/com/kingsrook/qbits/collaboration/CollaborationApiHttpTest.java +++ b/src/test/java/com/kingsrook/qbits/collaboration/CollaborationApiHttpTest.java @@ -39,6 +39,12 @@ import com.fasterxml.jackson.databind.JsonNode; import com.fasterxml.jackson.databind.ObjectMapper; import com.fasterxml.jackson.databind.node.ObjectNode; import com.kingsrook.qbits.collaboration.attachments.AttachmentStorage; +import com.kingsrook.qbits.collaboration.customfields.FieldDefinitions; +import com.kingsrook.qbits.collaboration.model.CollaborationTables; +import com.kingsrook.qqq.backend.core.actions.tables.InsertAction; +import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData; import com.kingsrook.qbits.objectstorage.api.PresignGet; import com.kingsrook.qbits.objectstorage.api.PutRequest; import com.kingsrook.qqq.backend.core.actions.collaboration.RuntimeMetaDataOverlay; @@ -384,6 +390,80 @@ class CollaborationApiHttpTest extends PersonalServiceFixture assertThat(tagQuery("IN", "[\"alpha\"]")).containsExactlyInAnyOrder(1, 2); } + /******************************************************************************* + ** EAV choices retain their stored ids and translated labels on the HTTP wire. + *******************************************************************************/ + @Test + void customFieldValuesThroughTableReads() throws Exception + { + admin(); + instance.addBackend(new RDBMSBackendMetaData().withName("fallback").withVendor("h2")); + instance.getTable("document").setBackendName("fallback"); + QRecord definition = new InsertAction().execute(new InsertInput(CollaborationTables.FIELD_DEFINITION).withRecords(List.of( + new QRecord().withValue("tableName", "document").withValue("name", "choice").withValue("label", "Choice").withValue("type", "SELECT") + .withValue("options", "[{\"id\":\"x\",\"label\":\"Choice X\"}]")))).getRecords().getFirst(); + assertThat(definition.getErrors()).isNullOrEmpty(); + FieldDefinitions.apply(definition.getValue("id"), "Publish"); + instance.getTable("document").setBackendName("memory"); + CollaborationRecords.insert(CollaborationTables.FIELD_VALUE, new QRecord().withValue("tableName", "document").withValue("recordId", "1") + .withValue("fieldName", "choice").withValue("valueText", "x").withValue("parentDeleted", false)); + login("alice"); + String serverUrl = baseUrl.substring(0, baseUrl.indexOf("/qbit/")); + HttpResponse response = client.send(HttpRequest.newBuilder(URI.create(serverUrl + "/table/document/query")).timeout(Duration.ofSeconds(20)) + .header("Cookie", "sessionUUID=alice" + sessionSuffix).header("Content-Type", "application/json") + .POST(HttpRequest.BodyPublishers.ofString("{\"filter\":{\"orderBys\":[{\"fieldName\":\"id\"}]}}")).build(), HttpResponse.BodyHandlers.ofString()); + JsonNode records = json(response, 200).get("records"); + assertThat(records).hasSize(2); + assertThat(records.get(0).at("/values/id").asInt()).isEqualTo(1); + assertThat(records.get(0).at("/values/cf_choice").asText()).isEqualTo("x"); + assertThat(records.get(0).at("/displayValues/cf_choice").asText()).isEqualTo("Choice X"); + assertThat(records.get(1).at("/values/cf_choice").isMissingNode()).isTrue(); + + response = client.send(HttpRequest.newBuilder(URI.create(serverUrl + "/table/document/1")).timeout(Duration.ofSeconds(20)) + .header("Cookie", "sessionUUID=alice" + sessionSuffix).GET().build(), HttpResponse.BodyHandlers.ofString()); + JsonNode record = json(response, 200).get("record"); + assertThat(record.at("/values/cf_choice").asText()).isEqualTo("x"); + assertThat(record.at("/displayValues/cf_choice").asText()).isEqualTo("Choice X"); + + response = client.send(HttpRequest.newBuilder(URI.create(serverUrl + "/table/document/1")).timeout(Duration.ofSeconds(20)) + .header("Cookie", "sessionUUID=outsider" + sessionSuffix).GET().build(), HttpResponse.BodyHandlers.ofString()); + assertThat(response.statusCode()).isEqualTo(404); + assertThat(response.body()).doesNotContain("Choice X", "First document"); + } + + /******************************************************************************* + ** Table reads expose tag labels while retaining parent-record authorization. + *******************************************************************************/ + @Test + void tagValuesThroughTableReads() throws Exception + { + RuntimeMetaDataOverlay.refresh(instance); + int tag = ok("POST", "/tags", "{\"table\":\"document\",\"label\":\"Review, urgent\"}").get("id").asInt(); + ok("PUT", "/records/document/1/tags", "{\"add\":[" + tag + "]}"); + String serverUrl = baseUrl.substring(0, baseUrl.indexOf("/qbit/")); + HttpResponse response = client.send(HttpRequest.newBuilder(URI.create(serverUrl + "/table/document/query")).timeout(Duration.ofSeconds(20)) + .header("Cookie", "sessionUUID=alice" + sessionSuffix).header("Content-Type", "application/json") + .POST(HttpRequest.BodyPublishers.ofString("{\"filter\":{\"orderBys\":[{\"fieldName\":\"id\"}]}}")).build(), HttpResponse.BodyHandlers.ofString()); + JsonNode records = json(response, 200).get("records"); + assertThat(records).hasSize(2); + assertThat(records.get(0).at("/values/id").asInt()).isEqualTo(1); + assertThat(records.get(0).at("/values/collaborationTags").asText()).isEqualTo("Review, urgent"); + assertThat(records.get(0).at("/displayValues/collaborationTags").asText()).isEqualTo("Review, urgent"); + assertThat(records.get(1).at("/values/id").asInt()).isEqualTo(2); + assertThat(records.get(1).at("/values/collaborationTags").isMissingNode()).as(records.toString()).isTrue(); + + response = client.send(HttpRequest.newBuilder(URI.create(serverUrl + "/table/document/1")).timeout(Duration.ofSeconds(20)) + .header("Cookie", "sessionUUID=alice" + sessionSuffix).GET().build(), HttpResponse.BodyHandlers.ofString()); + JsonNode record = json(response, 200).get("record"); + assertThat(record.at("/values/collaborationTags").asText()).isEqualTo("Review, urgent"); + assertThat(record.at("/displayValues/collaborationTags").asText()).isEqualTo("Review, urgent"); + + response = client.send(HttpRequest.newBuilder(URI.create(serverUrl + "/table/document/1")).timeout(Duration.ofSeconds(20)) + .header("Cookie", "sessionUUID=outsider" + sessionSuffix).GET().build(), HttpResponse.BodyHandlers.ofString()); + assertThat(response.body()).doesNotContain("Review, urgent", "First document"); + assertThat(response.statusCode()).isEqualTo(404); + } + /******************************************************************************* ** Run the v1 table query filtered on the virtual tags field; return matching ids. *******************************************************************************/ diff --git a/src/test/java/com/kingsrook/qbits/collaboration/CoreContractsTest.java b/src/test/java/com/kingsrook/qbits/collaboration/CoreContractsTest.java index d4573a0..9778328 100644 --- a/src/test/java/com/kingsrook/qbits/collaboration/CoreContractsTest.java +++ b/src/test/java/com/kingsrook/qbits/collaboration/CoreContractsTest.java @@ -84,7 +84,8 @@ class CoreContractsTest extends PersonalServiceFixture RuntimeMetaDataOverlay.refresh(instance); long generation = RuntimeMetaDataOverlay.getGeneration(instance); assertThat(instance.getTable("document").getVirtualFields()).containsKey("collaborationTags"); - assertThat(instance.getTable("document").getVirtualField("collaborationTags").getIsQuerySelectable()).isFalse(); + assertThat(instance.getTable("document").getVirtualField("collaborationTags").getIsQuerySelectable()).isTrue(); + assertThat(instance.getTable("document").getVirtualField("collaborationTags").getValueProvider()).isNotNull(); QBackendTransaction tx = new QBackendTransaction(); var row = new InsertAction().execute(new InsertInput(CollaborationTables.FEATURE_SETTING).withTransaction(tx).withRecords(List.of( new QRecord().withValue("scopeType", "TABLE").withValue("scopeName", "document").withValue("feature", "TAGS") diff --git a/src/test/java/com/kingsrook/qbits/collaboration/CustomFieldValuesPostgresTest.java b/src/test/java/com/kingsrook/qbits/collaboration/CustomFieldValuesPostgresTest.java new file mode 100644 index 0000000..34c07e7 --- /dev/null +++ b/src/test/java/com/kingsrook/qbits/collaboration/CustomFieldValuesPostgresTest.java @@ -0,0 +1,188 @@ +/* + * QQQ - Low-code Application Framework for Engineers. + * Copyright (C) 2021-2025. Kingsrook, LLC + * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States + * contact@kingsrook.com + * https://github.com/Kingsrook/ + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.collaboration; + +import java.sql.Connection; +import java.sql.DriverManager; +import java.util.Set; +import com.kingsrook.qbits.collaboration.customfields.CustomFieldValueProvider; +import com.kingsrook.qbits.collaboration.model.CollaborationTables; +import com.kingsrook.qqq.backend.core.actions.tables.QueryAction; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference; +import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType; +import com.kingsrook.qqq.backend.core.model.metadata.fields.QVirtualFieldMetaData; +import com.kingsrook.qqq.backend.module.postgres.model.metadata.PostgreSQLBackendMetaData; +import com.kingsrook.qqq.backend.module.rdbms.actions.RDBMSTransaction; +import com.kingsrook.qqq.backend.module.rdbms.jdbc.ConnectionManager; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.ValueSource; +import org.testcontainers.containers.GenericContainer; +import org.testcontainers.containers.wait.strategy.Wait; +import static org.assertj.core.api.Assertions.assertThat; + +/******************************************************************************* + ** Verify provider reads using a real database and explicit caller transactions. + *******************************************************************************/ +class CustomFieldValuesPostgresTest extends PersonalServiceFixture +{ + private static GenericContainer postgres; + + /******************************************************************************* + ** Own an isolated database; never connect to an application database. + *******************************************************************************/ + @BeforeAll + static void startPostgres() + { + postgres = new GenericContainer<>("postgres:17-alpine").withEnv("POSTGRES_PASSWORD", "eav-fixture-only") + .withExposedPorts(5432).waitingFor(Wait.forLogMessage(".*database system is ready to accept connections.*", 2)); + postgres.start(); + } + + /******************************************************************************* + ** Stop only the fixture's database. + *******************************************************************************/ + @AfterAll + static void stopPostgres() + { + if(postgres != null) + { + postgres.stop(); + } + } + + /******************************************************************************* + ** Reuse the secured host metadata with real PostgreSQL storage. + *******************************************************************************/ + @Override + @BeforeEach + public void setUp() throws Exception + { + super.setUp(); + instance.getBackends().remove("memory"); + instance.addBackend(new PostgreSQLBackendMetaData().withName("memory").withHostName(postgres.getHost()) + .withPort(postgres.getMappedPort(5432)).withDatabaseName("postgres").withUsername("postgres").withPassword("eav-fixture-only")); + try(Connection connection = connect(); var sql = connection.createStatement()) + { + sql.execute("DROP SCHEMA public CASCADE; CREATE SCHEMA public"); + sql.execute(CollaborationSchema.postgresScript(config)); + sql.execute("CREATE TABLE document (id INTEGER PRIMARY KEY, title TEXT, tenant TEXT)"); + sql.execute("INSERT INTO document VALUES (1, 'Visible', 'a'), (3, 'Private', 'b')"); + } + CollaborationRecords.insert(CollaborationTables.FIELD_DEFINITION, new QRecord().withValue("tableName", "document") + .withValue("name", "rating").withValue("label", "Rating").withValue("type", "INTEGER").withValue("status", "ACTIVE")); + instance.getTable("document").withVirtualField(new QVirtualFieldMetaData("cf_rating", QFieldType.INTEGER).withLabel("Rating") + .withIsQuerySelectable(true).withValueProvider(new QCodeReference(CustomFieldValueProvider.class))); + } + + /******************************************************************************* + ** Release transaction state and pooled connections before parent fixture cleanup. + *******************************************************************************/ + @Override + public void tearDown() + { + QContext.clearTransaction(); + ConnectionManager.closeConnectionProviders(instance); + super.tearDown(); + } + + /******************************************************************************* + ** A separate connection establishes committed visibility independently. + *******************************************************************************/ + private Connection connect() throws Exception + { + return (DriverManager.getConnection("jdbc:postgresql://" + postgres.getHost() + ":" + postgres.getMappedPort(5432) + + "/postgres", "postgres", "eav-fixture-only")); + } + + /******************************************************************************* + ** Both vocabulary and assignments use the explicit transaction's visibility. + *******************************************************************************/ + @ParameterizedTest + @ValueSource(booleans = {false, true}) + void tagLabelsSeePendingVocabularyAndCompletion(boolean commit) throws Exception + { + instance.getTable("document").withVirtualField(new QVirtualFieldMetaData("collaborationTags", QFieldType.STRING).withLabel("Tags") + .withIsQuerySelectable(true).withPossibleValueSourceName(CollaborationTables.TAG).withShouldTranslatePossibleValues(false) + .withValueProvider(new QCodeReference(com.kingsrook.qbits.collaboration.tags.TagValueProvider.class))); + var tags = new com.kingsrook.qbits.collaboration.tags.TagsService(); + try(var transaction = new RDBMSTransaction(connect())) + { + QContext.setTransaction(transaction); + var tag = tags.create("document", "Pending transaction label", null); + tags.apply("document", 1, java.util.List.of(tag.id()), java.util.List.of()); + QContext.clearTransaction(); + assertThat(QContext.getQBackendTransaction()).isNull(); + var pending = new QueryAction().execute(new QueryInput("document").withFieldNamesToInclude(Set.of("collaborationTags")) + .withShouldTranslatePossibleValues(true).withShouldGenerateDisplayValues(true).withTransaction(transaction)).getRecords(); + assertThat(pending).hasSize(1); + assertThat(pending.getFirst().getValues()).containsOnlyKeys("collaborationTags").containsEntry("collaborationTags", "Pending transaction label"); + assertThat(pending.getFirst().getDisplayValue("collaborationTags")).isEqualTo("Pending transaction label"); + assertThat(new QueryAction().execute(new QueryInput("document")).getRecords().getFirst().getValue("collaborationTags")).isNull(); + if(commit) + { + transaction.commit(); + } + else + { + transaction.rollback(); + } + } + assertThat(new QueryAction().execute(new QueryInput("document")).getRecords().getFirst().getValue("collaborationTags")) + .isEqualTo(commit ? "Pending transaction label" : null); + } + + /******************************************************************************* + ** Explicit query transactions see pending values even without thread context. + *******************************************************************************/ + @ParameterizedTest + @ValueSource(booleans = {false, true}) + void explicitTransactionSeesPendingValueAndCompletion(boolean commit) throws Exception + { + try(var transaction = new RDBMSTransaction(connect())) + { + QContext.setTransaction(transaction); + CollaborationRecords.insert(CollaborationTables.FIELD_VALUE, new QRecord().withValue("tableName", "document").withValue("recordId", "1") + .withValue("fieldName", "rating").withValue("valueNumber", 42).withValue("parentDeleted", false)); + QContext.clearTransaction(); + assertThat(QContext.getQBackendTransaction()).isNull(); + var pending = new QueryAction().execute(new QueryInput("document").withFieldNamesToInclude(Set.of("cf_rating")) + .withTransaction(transaction)).getRecords(); + assertThat(pending).hasSize(1); + assertThat(pending.getFirst().getValues()).containsOnlyKeys("cf_rating").containsEntry("cf_rating", 42); + assertThat(new QueryAction().execute(new QueryInput("document")).getRecords().getFirst().getValue("cf_rating")).isNull(); + if(commit) + { + transaction.commit(); + } + else + { + transaction.rollback(); + } + } + assertThat(new QueryAction().execute(new QueryInput("document")).getRecords().getFirst().getValue("cf_rating")).isEqualTo(commit ? 42 : null); + } +} diff --git a/src/test/java/com/kingsrook/qbits/collaboration/CustomFieldsTest.java b/src/test/java/com/kingsrook/qbits/collaboration/CustomFieldsTest.java index 2fde99c..d986eb4 100644 --- a/src/test/java/com/kingsrook/qbits/collaboration/CustomFieldsTest.java +++ b/src/test/java/com/kingsrook/qbits/collaboration/CustomFieldsTest.java @@ -51,6 +51,9 @@ import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaDat import com.kingsrook.qbits.collaboration.app.CollaborationHome; import com.kingsrook.qqq.backend.core.model.dashboard.widgets.TableData; import org.junit.jupiter.api.Test; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.Arguments; +import org.junit.jupiter.params.provider.MethodSource; import static org.assertj.core.api.Assertions.assertThat; import static org.assertj.core.api.Assertions.assertThatThrownBy; @@ -242,6 +245,59 @@ public class CustomFieldsTest extends PersonalServiceFixture assertThat(FieldProcesses.metadata()).extracting(process -> process.getLabel()).containsExactly("Publish field", "Archive field", "Rebuild index"); } + /******************************************************************************* + ** Typed EAV storage returns the declared Java value through an actual query. + *******************************************************************************/ + @ParameterizedTest + @MethodSource("eavValueTypes") + void eavValueTypes(String type, String column, java.io.Serializable stored, java.io.Serializable expected) throws Exception + { + admin(); + instance.addBackend(new RDBMSBackendMetaData().withName("fallback").withVendor("h2")); + instance.getTable("document").setBackendName("fallback"); + QRecord draft = new QRecord().withValue("tableName", "document").withValue("name", "typed").withValue("label", "Typed").withValue("type", type); + if(List.of("SELECT", "MULTI_SELECT").contains(type)) + { + draft.withValue("options", "[{\"id\":\"x\",\"label\":\"Choice X\"}]"); + } + if("RECORD_LINK".equals(type)) + { + draft.withValue("linkTableName", CollaborationTables.USER); + } + QRecord definition = new InsertAction().execute(new InsertInput(CollaborationTables.FIELD_DEFINITION).withRecords(List.of(draft))).getRecords().getFirst(); + assertThat(definition.getErrors()).isNullOrEmpty(); + FieldDefinitions.apply(definition.getValue("id"), "Publish"); + instance.getTable("document").setBackendName("memory"); + CollaborationRecords.insert(CollaborationTables.FIELD_VALUE, new QRecord().withValue("tableName", "document").withValue("recordId", "1") + .withValue("fieldName", "typed").withValue(column, stored).withValue("parentDeleted", false)); + login("alice"); + var records = new QueryAction().execute(new QueryInput("document").withFieldNamesToInclude(java.util.Set.of("cf_typed"))).getRecords(); + assertThat(records).hasSize(2); + assertThat(records).allSatisfy(record -> assertThat(record.getValues()).containsOnlyKeys("cf_typed")); + assertThat(records).extracting(record -> record.getValue("cf_typed")).containsExactlyInAnyOrder(expected, null); + Object actual = records.stream().map(record -> record.getValue("cf_typed")).filter(java.util.Objects::nonNull).findFirst().orElseThrow(); + assertThat(actual).isExactlyInstanceOf(expected.getClass()); + } + + /******************************************************************************* + ** Keep expectations independent of the provider's column-mapping helper. + *******************************************************************************/ + static java.util.stream.Stream eavValueTypes() + { + return (java.util.stream.Stream.of( + Arguments.of("INTEGER", "valueNumber", new BigDecimal("7.0"), 7), + Arguments.of("DECIMAL", "valueNumber", new BigDecimal("12.345"), new BigDecimal("12.345")), + Arguments.of("BOOLEAN", "valueBoolean", false, false), + Arguments.of("DATE", "valueDate", java.time.LocalDate.of(2026, 10, 22), java.time.LocalDate.of(2026, 10, 22)), + Arguments.of("DATE_TIME", "valueDateTime", java.time.Instant.parse("2026-10-22T12:34:56Z"), java.time.Instant.parse("2026-10-22T12:34:56Z")), + Arguments.of("SELECT", "valueText", "x", "x"), + Arguments.of("MULTI_SELECT", "valueText", "[\"x\"]", "[\"x\"]"), + Arguments.of("USER", "valueText", "alice", "alice"), + Arguments.of("RECORD_LINK", "valueText", "alice", "alice"), + Arguments.of("STRING", "valueText", "A, B", "A, B"), + Arguments.of("TEXT", "valueText", "First\nSecond", "First\nSecond"))); + } + /******************************************************************************* ** An unsupported JSON backend gets typed, read-only subquery filters. *******************************************************************************/ @@ -256,18 +312,50 @@ public class CustomFieldsTest extends PersonalServiceFixture instance.getTable("document").setBackendName("memory"); var virtual = instance.getTable("document").getVirtualField("cf_rating"); assertThat(virtual).isNotNull(); - assertThat(virtual.getIsQuerySelectable()).isFalse(); + assertThat(virtual.getIsQuerySelectable()).isTrue(); assertThat(virtual.getIsEditable()).isFalse(); assertThat(FieldDefinitions.get(definition.getValue("id")).getValueString("storage")).isEqualTo("EAV"); CollaborationRecords.insert(CollaborationTables.FIELD_VALUE, new QRecord().withValue("tableName", "document").withValue("recordId", "1") .withValue("fieldName", "rating").withValue("valueNumber", new BigDecimal("7")).withValue("parentDeleted", false)); + QContext.getQSession().withSecurityKeyValue("allTenants", true); + CollaborationRecords.insert(CollaborationTables.FIELD_VALUE, new QRecord().withValue("tableName", "document").withValue("recordId", "3") + .withValue("fieldName", "rating").withValue("valueNumber", new BigDecimal("99")).withValue("parentDeleted", false)); login("alice"); + var selected = new QueryAction().execute(new QueryInput("document").withFieldNamesToInclude(java.util.Set.of("cf_rating"))).getRecords(); + assertThat(selected).hasSize(2); + assertThat(selected).allSatisfy(row -> assertThat(row.getValues()).containsOnlyKeys("cf_rating")); + assertThat(selected).extracting(row -> row.getValue("cf_rating")).containsExactlyInAnyOrder(7, null); assertThat(query("cf_rating", QCriteriaOperator.GREATER_THAN, 5)).extracting(row -> row.getValueInteger("id")).containsExactly(1); assertThat(query("cf_rating", QCriteriaOperator.NOT_EQUALS, 7)).extracting(row -> row.getValueInteger("id")).containsExactly(2); assertThat(new QueryAction().execute(new QueryInput("document").withFilter(new QQueryFilter().withCriteria("cf_rating", QCriteriaOperator.IS_BLANK))).getRecords()) .extracting(row -> row.getValueInteger("id")).containsExactly(2); + // Reuse the same session and query to prove results are not cached across access changes. + QueryInput projected = new QueryInput("document").withFieldNamesToInclude(java.util.Set.of("cf_rating")); + QContext.getQSession().getSecurityKeyValues().put("tenant", List.of("b")); + assertThat(new QueryAction().execute(projected).getRecords()).extracting(row -> row.getValue("cf_rating")).containsExactly(99); + QContext.getQSession().getSecurityKeyValues().put("tenant", List.of("a")); + assertThat(new QueryAction().execute(projected).getRecords()).extracting(row -> row.getValue("cf_rating")).containsExactlyInAnyOrder(7, null); + QContext.getQSession().setPermissions(java.util.Set.of()); + assertThatThrownBy(() -> new QueryAction().execute(projected)).isInstanceOf(com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException.class); + login("alice"); + assertThat(com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordReferenceTableMetaData.of(instance.getTable(CollaborationTables.FIELD_VALUE))).isNotNull(); + assertThat(new QueryAction().execute(new QueryInput(CollaborationTables.FIELD_VALUE) + .withFilter(new QQueryFilter().withCriteria("tableName", QCriteriaOperator.EQUALS, "document") + .withCriteria("recordId", QCriteriaOperator.IN, List.of("1", "3")))).getRecords()) + .extracting(row -> row.getValueString("recordId")).containsExactly("1"); + virtual.setIsHidden(true); + assertThat(new QueryAction().execute(new QueryInput("document")).getRecords()).allSatisfy(row -> assertThat(row.getValues()).doesNotContainKey("cf_rating")); + virtual.setIsHidden(false); + admin(); + QContext.getQSession().withSecurityKeyValue("allTenants", true); + assertThat(FieldDefinitions.valueCount(FieldDefinitions.get(definition.getValue("id")))).isEqualTo(2); + QRecord deleted = CollaborationRecords.insert(CollaborationTables.FIELD_VALUE, new QRecord().withValue("tableName", "document").withValue("recordId", "2") + .withValue("fieldName", "rating").withValue("valueNumber", new BigDecimal("88")).withValue("parentDeleted", true)); + assertThat(deleted.getValueBoolean("parentDeleted")).isTrue(); + login("alice"); + assertThat(new QueryAction().execute(new QueryInput("document").withFieldNamesToInclude(java.util.Set.of("cf_rating"))).getRecords()) + .extracting(row -> row.getValue("cf_rating")).containsExactlyInAnyOrder(7, null); admin(); - assertThat(FieldDefinitions.valueCount(FieldDefinitions.get(definition.getValue("id")))).isEqualTo(1); var expander = new com.kingsrook.qbits.collaboration.customfields.FieldCriteriaExpander(); assertThat(expander.expandSearchTerm(instance.getTable("document"), "text", com.kingsrook.qqq.backend.core.actions.tables.query.SearchTermKind.TEXT, null)).isNotNull(); assertThat(com.kingsrook.qbits.collaboration.customfields.FieldCriteriaExpander.valueColumn("DATE")).isEqualTo("valueDate"); diff --git a/src/test/java/com/kingsrook/qbits/collaboration/TagsServiceTest.java b/src/test/java/com/kingsrook/qbits/collaboration/TagsServiceTest.java index 85ca704..63348c5 100644 --- a/src/test/java/com/kingsrook/qbits/collaboration/TagsServiceTest.java +++ b/src/test/java/com/kingsrook/qbits/collaboration/TagsServiceTest.java @@ -73,6 +73,126 @@ public class TagsServiceTest extends PersonalServiceFixture private final TagsService tags = new TagsService(); + /******************************************************************************* + ** Tag labels are selectable while the ID-based filter picker remains intact. + *******************************************************************************/ + @Test + void selectsTagLabelsWithoutTranslatingThemAsIds() throws Exception + { + TagEntry beta = tags.create("document", "Beta", null); + TagEntry alpha = tags.create("document", "Alpha", null); + tags.apply("document", 1, List.of(beta.id(), alpha.id()), List.of()); + com.kingsrook.qqq.backend.core.actions.collaboration.RuntimeMetaDataOverlay.refresh(instance); + var field = instance.getTable("document").getVirtualField("collaborationTags"); + assertThat(field.getIsQuerySelectable()).isTrue(); + assertThat(field.getPossibleValueSourceName()).isEqualTo(CollaborationTables.TAG); + var rows = new QueryAction().execute(new QueryInput("document").withFieldNamesToInclude(java.util.Set.of("collaborationTags")) + .withShouldTranslatePossibleValues(true).withShouldGenerateDisplayValues(true)).getRecords(); + assertThat(rows).hasSize(2); + assertThat(rows).allSatisfy(row -> assertThat(row.getValues()).containsOnlyKeys("collaborationTags")); + assertThat(rows).extracting(row -> row.getValueString("collaborationTags")).containsExactlyInAnyOrder("Alpha, Beta", null); + assertThat(rows).filteredOn(row -> row.getValue("collaborationTags") != null) + .allSatisfy(row -> assertThat(row.getDisplayValue("collaborationTags")).isEqualTo("Alpha, Beta")); + assertThat(new QueryAction().execute(new QueryInput("document").withFilter(new QQueryFilter() + .withCriteria("collaborationTags", QCriteriaOperator.EQUALS, alpha.id()))).getRecords()) + .extracting(row -> row.getValueInteger("id")).containsExactly(1); + } + + /******************************************************************************* + ** Exported labels obey parent visibility and vocabulary approval, with CSV quoting. + *******************************************************************************/ + @Test + void tagLabelsRespectPrivacyAndExportQuoting() throws Exception + { + TagEntry visible = tags.create("document", "Needs, review", null); + TagEntry secret = tags.create("document", "Secret label", null); + tags.apply("document", 1, List.of(visible.id()), List.of()); + QContext.getQSession().withSecurityKeyValue("allTenants", true); + tags.apply("document", 3, List.of(secret.id()), List.of()); + config.withDefaultTagMode("APPROVED_ONLY"); + TagEntry pending = tags.create("document", "Pending label", null); + assertThat(pending.approved()).isFalse(); + // A stale assignment must not turn a pending suggestion into a visible label. + CollaborationRecords.insert(CollaborationTables.RECORD_TAG, new QRecord().withValue("tableName", "document") + .withValue("recordId", "1").withValue("tagId", pending.id()).withValue("createdByUserId", "alice").withValue("parentDeleted", false)); + login("alice"); + com.kingsrook.qqq.backend.core.actions.collaboration.RuntimeMetaDataOverlay.refresh(instance); + var input = new QueryInput("document").withFieldNamesToInclude(java.util.Set.of("collaborationTags")); + assertThat(new QueryAction().execute(input).getRecords()).extracting(row -> row.getValueString("collaborationTags")) + .containsExactlyInAnyOrder("Needs, review", null); + + java.io.ByteArrayOutputStream output = new java.io.ByteArrayOutputStream(); + var export = new com.kingsrook.qqq.backend.core.model.actions.reporting.ExportInput(); + export.setTableName("document"); + export.setFieldNames(List.of("collaborationTags")); + export.setReportDestination(new com.kingsrook.qqq.backend.core.model.actions.reporting.ReportDestination() + .withReportFormat(com.kingsrook.qqq.backend.core.model.actions.reporting.ReportFormat.CSV).withReportOutputStream(output)); + new com.kingsrook.qqq.backend.core.actions.reporting.ExportAction().execute(export); + String csv = output.toString(java.nio.charset.StandardCharsets.UTF_8); + assertThat(csv).doesNotContain("Secret label", "Pending label"); + try(var parser = org.apache.commons.csv.CSVParser.parse(csv, org.apache.commons.csv.CSVFormat.DEFAULT.withFirstRecordAsHeader())) + { + assertThat(parser.getHeaderNames()).containsExactly("Tags"); + assertThat(parser.getRecords()).extracting(row -> row.get("Tags")).containsExactlyInAnyOrder("Needs, review", ""); + } + QContext.getQSession().getSecurityKeyValues().put("tenant", List.of("b")); + assertThat(new QueryAction().execute(input).getRecords()).extracting(row -> row.getValueString("collaborationTags")).containsExactly("Secret label"); + login("alice"); + assertThat(new QueryAction().execute(input).getRecords()).extracting(row -> row.getValueString("collaborationTags")) + .containsExactlyInAnyOrder("Needs, review", null); + } + + /******************************************************************************* + ** A page spanning the provider's batch boundary retains its last parent values. + *******************************************************************************/ + @Test + void tagLabelsSpanMultipleParentBatches() throws Exception + { + java.util.List parents = new java.util.ArrayList<>(); + for(int id = 4; id <= 504; id++) + { + parents.add(new QRecord().withValue("id", id).withValue("title", "Document " + id).withValue("tenant", "a")); + } + assertThat(new InsertAction().execute(new InsertInput("document").withRecords(parents)).getRecords()) + .allSatisfy(row -> assertThat(row.getErrors()).isNullOrEmpty()); + TagEntry tag = tags.create("document", "Boundary", null); + tags.apply("document", 1, List.of(tag.id()), List.of()); + tags.apply("document", 504, List.of(tag.id()), List.of()); + com.kingsrook.qqq.backend.core.actions.collaboration.RuntimeMetaDataOverlay.refresh(instance); + var rows = new QueryAction().execute(new QueryInput("document").withFieldNamesToInclude(java.util.Set.of("id", "collaborationTags"))).getRecords(); + assertThat(rows).hasSize(503); + assertThat(rows).filteredOn(row -> row.getValue("collaborationTags") != null).extracting(row -> row.getValueInteger("id")) + .containsExactlyInAnyOrder(1, 504); + assertThat(rows).filteredOn(row -> row.getValue("collaborationTags") != null) + .allSatisfy(row -> assertThat(row.getValueString("collaborationTags")).isEqualTo("Boundary")); + } + + /******************************************************************************* + ** Stale links cannot expose merged vocabulary, other-table tags, or deleted links. + *******************************************************************************/ + @Test + void tagLabelsExcludeStaleAndOutOfScopeAssignments() throws Exception + { + login("admin", TagsService.MANAGE); + TagEntry global = tags.create(null, "Global", null); + TagEntry current = tags.create("document", "Current", null); + TagEntry merged = tags.create("document", "Merged old label", null); + TagEntry deleted = tags.create("document", "Deleted link label", null); + TagEntry other = tags.create("document", "Other table label", null); + CollaborationRecords.update(CollaborationTables.TAG, new QRecord().withValue("id", merged.id()).withValue("mergedIntoTagId", current.id())); + CollaborationRecords.update(CollaborationTables.TAG, new QRecord().withValue("id", other.id()).withValue("tableName", "different_table")); + for(TagEntry tag : List.of(global, current, merged, deleted, other)) + { + CollaborationRecords.insert(CollaborationTables.RECORD_TAG, new QRecord().withValue("tableName", "document").withValue("recordId", "1") + .withValue("tagId", tag.id()).withValue("createdByUserId", "admin").withValue("parentDeleted", tag.id().equals(deleted.id()))); + } + login("alice"); + com.kingsrook.qqq.backend.core.actions.collaboration.RuntimeMetaDataOverlay.refresh(instance); + var rows = new QueryAction().execute(new QueryInput("document").withFieldNamesToInclude(java.util.Set.of("collaborationTags"))).getRecords(); + assertThat(rows).extracting(row -> row.getValueString("collaborationTags")).containsExactlyInAnyOrder("Current, Global", null); + assertThat(com.kingsrook.qqq.backend.core.model.metadata.collaboration.RecordReferenceTableMetaData.of(instance.getTable(CollaborationTables.RECORD_TAG))).isNotNull(); + } + /******************************************************************************* ** Natural keys are locale-independent and validate after collapsing whitespace. *******************************************************************************/