feat: tags with approval modes and bulk actions #5

Merged
KofTwentyTwo merged 1 commits from feature/tags into develop 2026-10-13 22:40:51 +00:00
20 changed files with 1731 additions and 10 deletions

No files matched your search

+5
View File
@@ -6,6 +6,11 @@ All notable changes are documented here, following [Keep a Changelog](https://ke
### Added
- Phase-d tag vocabulary and record assignments on QQQ 4.1.0-RC.5; normalized names, FREE/APPROVED_ONLY policy resolved from system-read scoped feature settings, explicit global/manage permissions, parent-checked idempotent assignment, usage counts and TAGGED/UNTAGGED activity without follower notifications.
- Tags navigation, secured Record Tags child list, FIELD audit, label search, approved/scoped table PVS, streamed Approve/Merge/Delete and configured host Add/Remove actions, parent cleanup and real Tags home metrics. Bulk multi-select intent is exposed in picker metadata for the separate Next UI implementation.
- Standalone `TagFilterTranslator` for IN/CONTAINS_ALL/NOT_IN/IS_BLANK/IS_NOT_BLANK with a 10,000 matching-ID cap. Filter/quick-search integration lands after QRun-IO/qqq#1140 is released.
- Tag service, security, vocabulary, settings, schema, cap and streamed Review/Result tests; existing JaCoCo gates retained.
- Phase-e comments/mentions, one-level threads, edit revisions, author/admin soft deletion, resolve/reopen, ranked mention candidates and keyset pagination on QQQ 4.1.0-RC.5.
- Comments, Comment History, Mentions, Followers, Activity and durable Follow Digests tables, child-list widgets, service-only writes, shared parent-table query restrictions, schema indexes and real home counts.
- Lazy people-directory upserts with ten-minute throttling from comments/follow/favorites/recents/attachments entry points; bounded active-name/email search.
+64 -1
View File
@@ -1,6 +1,6 @@
# QBit Collaboration
Record collaboration for QQQ: attachments, tags, custom fields, comments with mentions, presence, sharing, favorites, follow, timeline and tasks. **Implemented now:** feature settings and people tables, favorites and recents services (phase b), per-user security, daily recents retention, phase-c attachments (upload/list/download/rename/delete, previews and parent cleanup), and the Collaboration app home with real session-scoped counts. Phase e adds comments/mentions, follow actions and digests, the audit/activity timeline, and automatic people-directory updates. Tags (d) and phases f to i remain planned; stored settings are not yet applied as runtime overrides.
Record collaboration for QQQ: attachments, tags, custom fields, comments with mentions, presence, sharing, favorites, follow, timeline and tasks. **Implemented now:** feature settings and people tables, favorites and recents services (phase b), per-user security, daily recents retention, phase-c attachments (upload/list/download/rename/delete, previews and parent cleanup), and the Collaboration app home with real session-scoped counts. Phase e adds comments/mentions, follow actions and digests, the audit/activity timeline, and automatic people-directory updates. Phase d adds tags, vocabulary policies, management and bulk actions, and standalone filter translation. Phases f to i remain planned; general runtime overrides are still deferred. TAGS mode settings are read directly by the service.
Design: [QRun-IO/qqq PR #1136](https://github.com/QRun-IO/qqq/pull/1136), `docs/COLLABORATION.md`. See [the phase table](docs/DESIGN.md).
@@ -218,3 +218,66 @@ Offline verification: `./mvnw -o -q verify` (Windows: `./mvnw.cmd -o -q verify`)
## License
[Apache License 2.0](LICENSE). Copyright 2026 Kingsrook, LLC; see [NOTICE](NOTICE).
## Tags
Phase d builds against **QQQ 4.1.0-RC.5**. Register host actions with
`config.withTagActionTables(List.of("order"))`; `withDefaultTagMode("FREE")` is the
default. Set `collaboration_feature_setting.feature = TAGS` and JSON
`{"mode":"APPROVED_ONLY"}` to require approval. The service reads settings as the
system, in TABLE → preferred APP → INSTANCE order, falling back to config. The
first scope containing `mode` wins; `enabled` remains part of future general
feature resolution. Invalid policy JSON or mode fails closed.
`TagsService` provides:
- `vocabulary(table, query, limit)`: global and table vocabulary, approved tags plus
the caller's pending suggestions, excluding merged aliases; usage descending,
then label. Default limit 20, maximum 100.
- `create(tableOrNull, label, color)`: trimmed, lowercase, collapsed name of 1–60
characters; the label retains its case. Duplicate names reuse the existing tag.
Per-table creation requires `<table>.tag` or table EDIT and READ. FREE creates
approved tags; APPROVED_ONLY creates suggestions unless the caller holds
`collaboration.tags.manage`. Global creation always requires that manage grant.
- `tagsFor(table, recordId)` and `apply(table, recordId, add, remove)`: parent READ
and record locks are checked first. Writes also require the tag feature grant or
EDIT. Numeric values are tag IDs; strings are labels. Unknown add labels are
created in FREE; in APPROVED_ONLY they fail with
“Pick an approved tag (or suggest one)”. Pending tags cannot be applied by anyone.
Repeated changes are idempotent; actual changes write TAGGED/UNTAGGED activity,
recount usage, and do not notify followers.
Tags is a main navigation table with FIELD auditing and native label search.
Record Tags is a secured child list. Both use service-managed writes. Standard
streamed **Approve tags**, **Merge tags**, and **Delete tags** actions require
`collaboration.tags.manage`; Delete Review counts distinct affected parents.
Merge deduplicates references and retains the source as an alias. Targets must be
approved and in the source scope or global; a global source cannot merge into a
single table's vocabulary.
Host **Add tags** and **Remove tags** actions use the table-type
`collaboration_tag` PVS, filtered to approved, unmerged global or same-table tags.
Each record is rechecked during Review and execution. Input `tags` is a list of
integer IDs (comma-separated IDs are also accepted); `targetTagId` is a scalar.
The picker step exposes `EDIT_FORM` with `multiSelectFields = ["tags"]`. RC.5 has
no native multi-valued field type: rendering this component-value contract as a
multi-select belongs to the separate Next UI phase. The qbit tests exercise the
submitted arrays and real streamed Review/Result lifecycle.
`AttachmentCleanup.onParentsDeleted` now also removes tag assignments and
recounts vocabulary usage. Hosts keep using this trusted callback until the core
cleanup registration contract ships. The home shows session-visible vocabulary
size, pending suggestions, and the five most-used approved tags.
`TagFilterTranslator.recordIds(table, operatorName, tags)` supports `IN` (any),
`CONTAINS_ALL`, `NOT_IN` (none), `IS_BLANK`, and `IS_NOT_BLANK`. Every operator
returns matching visible parent IDs as strings, including complements. Exactly
10,000 results are allowed; larger results throw. Unknown tags match nothing;
empty IN matches nothing and empty CONTAINS_ALL/NOT_IN match all visible parents.
Numeric inputs are IDs, strings are normalized names; if a global and local tag
share a name, the global tag wins (use IDs to disambiguate).
**Filter/quick-search integration lands after QRun-IO/qqq#1140 is released.** This
phase does not register a virtual filter field, core expander, `QSubQuery`, tag
column filler, or `#tag`/host-record quick-search provider. The standalone
translator deliberately imports none of those unreleased types.
+41
View File
@@ -41,3 +41,44 @@ See [README](../README.md#attachments) for the service contract, permissions and
- **User-facing metadata.** Comments and admin-only Activity are main nav tables. Revisions, mentions and Followers are child lists; digest/follow tables are admin-browsable with HIDDEN deny behavior. Follow actions use generic streamed ETL Review/Result against any host key. The home reads real session-scoped comments/open-thread/follower counts. Standard QQQ writes remain available to host ESB listeners.
The service API and configuration are documented in [README](../README.md#comments-follow-and-timeline). Endpoints remain deferred to QRun-IO/qqq#1128; no unreleased core dependency, runtime overlay, mirrored-lock implementation or realtime/UI delivery is introduced.
## Phase-d implementation choices
- **RC.5 boundary.** `tags/TagFilterTranslator` uses operator names, QQQ queries and
string parent IDs. It returns matching rows for all operators, so NOT_IN and
IS_BLANK enumerate the visible host-table complement. The 10,000 limit applies
to final matching IDs, not assignment count. It pages parent reads under the
caller's permissions and locks. Filter/quick-search integration lands after
QRun-IO/qqq#1140 is released; no unreleased core types or virtual fields are used.
- **Vocabulary versus assignments.** Tag names are unique per nullable scope;
PostgreSQL DDL emits NULLS NOT DISTINCT for nullable unique-key components.
Record Tags inherits `ParentTableRestriction` and the parent row customizer,
including the existing approximation for aggregate counts and short pages.
Vocabulary pickers need no collaboration.view grant: ordinary users see
approved tags and their own suggestions in readable scopes, while manage sees
all vocabulary for moderation. Direct writes are denied; services/actions
preserve normalization, approval and counts.
- **Policy lookup.** Settings are read in a temporary system context, retaining
the transaction and restoring the caller. TABLE overrides preferred APP (QQQ's
app-affinity path), then INSTANCE, then `defaultTagMode`. Missing mode inherits;
malformed JSON/invalid mode throws. General enabled/permission-mode runtime
overrides remain outside this phase. Name collision resolution prefers global
vocabulary; IDs provide an unambiguous selection.
- **Mutation boundaries.** Like existing services, writes participate in the
caller's QQQ transaction and serialize within the installed config's monitor.
Unique keys enforce assignment identity. Counts are recomputed from stored
links, including merge deduplication, deletion and trusted parent cleanup.
This is not a cross-pod lock: concurrent writers in separate instances need
host transaction/serialization coordination for exact denormalized counts.
Standard ETL runs with its process transaction. Activity is generated only for
actual apply/remove changes and never calls follower notification services.
- **Review and Result.** Actions derive operation and table from installed process
metadata. Management requires collaboration.tags.manage at both stages. Bulk
actions resolve approved PVS selections and recheck every parent and feature
permission at both stages. Review is read-only, counts already-satisfied rows,
and Delete reports distinct affected parents. Result reports successful and
failed records separately. The picker exposes multiSelectFields on EDIT_FORM
for the separate Next client; RC.5 itself has no native multi-value form field.
- **Presentation.** Tags is in the admin app nav; Record Tags is a child widget.
The home queries under the caller session and HTML-escapes labels. Native tag
label search metadata is present; host-record tag search awaits #1140.
@@ -40,6 +40,8 @@ public class CollaborationQBitConfig implements QBitConfig
private QCodeReference notifier = new QCodeReference(NoOpNotifier.class);
private QCodeReference userSessionSupplier = null;
private String defaultTagMode = "FREE";
private List<String> tagActionTables = List.of();
private List<String> followActionTables = List.of();
private List<String> followChangeTables = List.of();
private String backendName;
@@ -250,8 +252,9 @@ public class CollaborationQBitConfig implements QBitConfig
{
assertCondition(instance.getBackend(backendName) != null, "Unknown Collaboration backend: " + backendName, errors);
}
assertCondition(List.of("FREE", "APPROVED_ONLY").contains(defaultTagMode == null ? "" : defaultTagMode), "defaultTagMode must be FREE or APPROVED_ONLY", errors);
assertCondition(notifier != null, "notifier is required", errors);
for(List<String> tables : List.of(followActionTables, followChangeTables))
for(List<String> tables : List.of(followActionTables, followChangeTables, tagActionTables))
{
for(String table : tables)
{
@@ -439,4 +442,38 @@ public class CollaborationQBitConfig implements QBitConfig
this.followChangeTables = followChangeTables;
return (this);
}
/*******************************************************************************
** Default vocabulary policy when no persisted override supplies a mode.
*******************************************************************************/
public String getDefaultTagMode()
{
return (defaultTagMode);
}
/*******************************************************************************
** Configure the default tag policy.
*******************************************************************************/
public CollaborationQBitConfig withDefaultTagMode(String mode)
{
this.defaultTagMode = mode;
return (this);
}
/*******************************************************************************
** Host tables receiving Add tags and Remove tags actions.
*******************************************************************************/
public List<String> getTagActionTables()
{
return (tagActionTables);
}
/*******************************************************************************
** Configure host tag actions.
*******************************************************************************/
public CollaborationQBitConfig withTagActionTables(List<String> tables)
{
this.tagActionTables = tables;
return (this);
}
}
@@ -137,6 +137,11 @@ public class CollaborationQBitProducer implements QBitMetaDataProducer<Collabora
{
instance.withRecordChangeListener(new com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference(com.kingsrook.qbits.collaboration.follow.FollowChangeBatcher.class));
}
com.kingsrook.qbits.collaboration.tags.TagProcesses.management().forEach(output::add);
for(String table : qBitConfig.getTagActionTables())
{
com.kingsrook.qbits.collaboration.tags.TagProcesses.processesFor(table).forEach(output::add);
}
output.add(DeleteAttachmentsProcess.metadata());
for(QPossibleValueSource source : CollaborationTables.possibleValueSources())
{
@@ -66,9 +66,10 @@ public final class CollaborationSchema
index++;
List<String> names = key.getFieldNames().stream().map(name -> table.getField(name).getBackendName()).toList();
statements.add("CREATE UNIQUE INDEX IF NOT EXISTS uq_" + table.getName() + "_" + index + " ON " + table.getName() + " (" + String.join(", ", names) + ")"
+ (key.getFieldNames().contains("recordId") ? " NULLS NOT DISTINCT" : ""));
+ (key.getFieldNames().stream().anyMatch(name -> !Boolean.TRUE.equals(table.getField(name).getIsRequired())) ? " NULLS NOT DISTINCT" : ""));
}
}
statements.add("CREATE INDEX IF NOT EXISTS idx_collaboration_record_tag_tag ON collaboration_record_tag (tag_id)");
statements.add("CREATE INDEX IF NOT EXISTS idx_collaboration_attachment_parent ON collaboration_attachment (table_name, record_id)");
statements.add("CREATE INDEX IF NOT EXISTS idx_collaboration_comment_parent ON collaboration_comment (table_name, record_id, last_activity_at, id)");
statements.add("CREATE INDEX IF NOT EXISTS idx_collaboration_comment_created ON collaboration_comment (table_name, record_id, create_date)");
@@ -42,6 +42,7 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateRe
import com.kingsrook.qqq.backend.core.model.actions.tables.count.CountInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
@@ -82,7 +83,7 @@ public final class CollaborationHome
.withPermissionRules(CollaborationPermissions.permissionRules())
.withWidgets(List.of(HOME_WIDGET))
.withSection(new QAppSection().withName("collaboration").withLabel("Collaboration").withIcon(new QIcon("forum"))
.withTables(List.of(CollaborationTables.COMMENT, CollaborationTables.ATTACHMENT, CollaborationTables.ACTIVITY, CollaborationTables.FEATURE_SETTING, CollaborationTables.USER)));
.withTables(List.of(CollaborationTables.COMMENT, CollaborationTables.ATTACHMENT, CollaborationTables.TAG, CollaborationTables.ACTIVITY, CollaborationTables.FEATURE_SETTING, CollaborationTables.USER)));
app.setParentAppName(config.getParentAppName());
return (app);
}
@@ -140,6 +141,7 @@ public final class CollaborationHome
personalSummary(html, CollaborationTables.FAVORITE, "Favorites");
personalSummary(html, CollaborationTables.RECENT, "Recently viewed");
attachmentSummary(html);
tagSummary(html);
html.append("<section><h3>Comments this week</h3><p>").append(count(CollaborationTables.COMMENT, new QQueryFilter()
.withCriteria("createDate", QCriteriaOperator.GREATER_THAN_OR_EQUALS, now.minus(7, ChronoUnit.DAYS))
.withCriteria("createDate", QCriteriaOperator.LESS_THAN_OR_EQUALS, now).withCriteria("deletedAt", QCriteriaOperator.IS_BLANK))).append("</p></section>");
@@ -156,6 +158,27 @@ public final class CollaborationHome
/*******************************************************************************
** Caller-filtered vocabulary totals and five most used active tags.
*******************************************************************************/
private static void tagSummary(StringBuilder html) throws QException
{
QQueryFilter active = new QQueryFilter().withCriteria("mergedIntoTagId", QCriteriaOperator.IS_BLANK);
html.append("<section><h3>Tags</h3><p>Vocabulary size: ").append(count(CollaborationTables.TAG, active))
.append("</p><p>Pending suggestions: ").append(count(CollaborationTables.TAG, new QQueryFilter()
.withCriteria("mergedIntoTagId", QCriteriaOperator.IS_BLANK).withCriteria("approved", QCriteriaOperator.EQUALS, false)))
.append("</p><ul>");
com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput input = new com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput(CollaborationTables.TAG)
.withFilter(active.withCriteria("approved", QCriteriaOperator.EQUALS, true).withOrderBy(new QFilterOrderBy("usageCount", false)).withOrderBy(new QFilterOrderBy("label")).withLimit(5));
input.setPermissionEnforcement(PermissionEnforcement.ENFORCE);
for(com.kingsrook.qqq.backend.core.model.data.QRecord tag : new com.kingsrook.qqq.backend.core.actions.tables.QueryAction().execute(input).getRecords())
{
html.append("<li>").append(tag.getValueString("label").replace("&", "&amp;").replace("<", "&lt;").replace(">", "&gt;"))
.append(": ").append(tag.getValueInteger("usageCount")).append("</li>");
}
html.append("</ul></section>");
}
/*******************************************************************************
** Real table aggregates, restricted to readable parent tables. Parent locks
** are post-filtered on rows, so these aggregate totals are approximate.
@@ -61,6 +61,7 @@ public class AttachmentCleanup
{
return;
}
new com.kingsrook.qbits.collaboration.tags.TagsService().onParentsDeleted(table, ids);
while(true)
{
List<QRecord> rows = AttachmentRecords.query(new QQueryFilter().withCriteria("tableName", QCriteriaOperator.EQUALS, table)
@@ -58,6 +58,8 @@ import com.kingsrook.qqq.backend.core.model.metadata.tables.UniqueKey;
*******************************************************************************/
public final class CollaborationTables
{
public static final String TAG = "collaboration_tag";
public static final String RECORD_TAG = "collaboration_record_tag";
public static final String COMMENT = "collaboration_comment";
public static final String COMMENT_REVISION = "collaboration_comment_revision";
public static final String MENTION = "collaboration_mention";
@@ -91,6 +93,7 @@ public final class CollaborationTables
public static List<QPossibleValueSource> possibleValueSources()
{
return (List.of(
QPossibleValueSource.newForTable(TAG), QPossibleValueSource.newForTable(RECORD_TAG),
QPossibleValueSource.newForTable(COMMENT), QPossibleValueSource.newForTable(COMMENT_REVISION), QPossibleValueSource.newForTable(MENTION), QPossibleValueSource.newForTable(FOLLOW), QPossibleValueSource.newForTable(ACTIVITY), QPossibleValueSource.newForTable(FOLLOW_DIGEST),
enumSource(TARGET_TYPE, "TABLE", "APP", "RECORD", "PROCESS", "REPORT", "SAVED_VIEW"),
QPossibleValueSource.newForTable(ATTACHMENT), QPossibleValueSource.newForTable(FAVORITE), QPossibleValueSource.newForTable(RECENT),
@@ -161,6 +164,7 @@ public final class CollaborationTables
.withSection(new QFieldSection("details", new QIcon("history"), Tier.T2, List.of("label", "lastViewedAt", "viewCount")).withLabel("Details"));
List<QTableMetaData> tables = new ArrayList<>(List.of(settings, people, favorites, recents, attachment(config)));
tables.addAll(conversationTables(config));
tables.addAll(tagTables(config));
for(QTableMetaData table : tables)
{
table.withSection(new QFieldSection("dates", new QIcon("calendar_today"), Tier.T3, List.of("createDate", "modifyDate")).withLabel("Dates"));
@@ -343,4 +347,39 @@ public final class CollaborationTables
.withRecordLabelFormatAndFields("%s", "id")
.withSection(new QFieldSection("parent", new QIcon("link"), Tier.T2, List.of("tableName", "recordId", "parentDeleted")).withLabel("Parent")));
}
/*******************************************************************************
** Vocabulary is visible; assignments inherit parent restrictions and locks.
*******************************************************************************/
private static List<QTableMetaData> tagTables(CollaborationQBitConfig config)
{
QTableMetaData tags = base(config, TAG, "Tags", "label")
.withPermissionRules(new QPermissionRules().withLevel(PermissionLevel.NOT_PROTECTED)
.withCustomPermissionChecker(new QCodeReference(com.kingsrook.qbits.collaboration.tags.TagSecurity.class)))
.withAuditRules(new QAuditRules().withAuditLevel(AuditLevel.FIELD))
.withSearchFields(List.of("label"))
.withRecordLabelFormatAndFields("%s", "label")
.withField(string("name", "Name", 60).withIsRequired(true))
.withField(string("label", "Label", 255).withIsRequired(true))
.withField(string("color", "Color", 64))
.withField(string("tableName", "Table", 128))
.withField(new QFieldMetaData("approved", QFieldType.BOOLEAN).withIsRequired(true))
.withField(new QFieldMetaData("description", QFieldType.TEXT))
.withField(string("createdByUserId", "Created By", 255).withIsRequired(true))
.withField(new QFieldMetaData("usageCount", QFieldType.INTEGER).withIsRequired(true).withDefaultValue(0))
.withField(new QFieldMetaData("mergedIntoTagId", QFieldType.INTEGER).withPossibleValueSourceName(TAG))
.withUniqueKey(new UniqueKey("tableName", "name"))
.withSection(new QFieldSection("details", new QIcon("label"), Tier.T1,
List.of("id", "name", "label", "color", "tableName", "approved", "description", "createdByUserId", "usageCount", "mergedIntoTagId")).withLabel("Details"));
QTableMetaData links = referenced(config, RECORD_TAG, "Record Tags", "label")
.withField(new QFieldMetaData("tagId", QFieldType.INTEGER).withIsRequired(true).withPossibleValueSourceName(TAG))
.withField(string("createdByUserId", "Created By", 255).withIsRequired(true))
.withUniqueKey(new UniqueKey("tableName", "recordId", "tagId"))
.withSection(new QFieldSection("details", new QIcon("label"), Tier.T1, List.of("id", "tagId", "createdByUserId")).withLabel("Details"));
for(QTableMetaData table : List.of(tags, links))
{
table.getFields().values().forEach(field -> field.setIsEditable(false));
}
return (List.of(tags, links));
}
}
@@ -48,6 +48,12 @@ public final class ConversationChildren
*******************************************************************************/
public static void install(QInstance instance)
{
QJoinMetaData tagJoin = new QJoinMetaData().withLeftTable(CollaborationTables.TAG).withRightTable(CollaborationTables.RECORD_TAG)
.withInferredName().withType(JoinType.ONE_TO_MANY).withJoinOns(List.of(new JoinOn("id", "tagId")));
instance.addJoin(tagJoin);
instance.addWidget(ChildRecordListRenderer.widgetMetaDataBuilder(tagJoin).withLabel("Record Tags").withCanAddChildRecord(false).getWidgetMetaData());
instance.getTable(CollaborationTables.TAG).withSection(new QFieldSection("recordTags", new QIcon("list"), Tier.T2)
.withLabel("Record Tags").withWidgetName(tagJoin.getName()));
child(instance, CollaborationTables.COMMENT_REVISION, "Comment History", List.of(new JoinOn("id", "commentId")));
child(instance, CollaborationTables.MENTION, "Mentions", List.of(new JoinOn("id", "commentId")));
child(instance, CollaborationTables.FOLLOW, "Followers", List.of(new JoinOn("tableName", "tableName"), new JoinOn("recordId", "recordId")));
@@ -29,7 +29,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRule
/*******************************************************************************
** Full permission names; fields and tags management are reserved for later phases.
** Full permission names; fields management is reserved for a later phase.
*******************************************************************************/
public final class CollaborationPermissions
{
@@ -94,10 +94,22 @@ public final class ParentRecordAccess
public static void requireFeature(String tableName, Serializable recordId, String feature) throws QException
{
requireVisibleParent(tableName, recordId);
requireTableFeature(tableName, feature);
}
/*******************************************************************************
** Authorize vocabulary creation without requiring an arbitrary parent record.
*******************************************************************************/
public static void requireTableFeature(String tableName, String feature) throws QException
{
if(!canReadTable(tableName))
{
throw (new QNotFoundException("Not found"));
}
QTableMetaData table = QContext.getQInstance().getTable(tableName);
String base = table.getPermissionRules() == null ? null : table.getPermissionRules().getPermissionBaseName();
base = base == null || base.isBlank() ? tableName : base;
if(!QContext.getQSession().hasPermission(base + "." + feature)
if(!QContext.getQSession().hasPermission(tableName + "." + feature) && !QContext.getQSession().hasPermission(base + "." + feature)
&& !PermissionsHelper.hasTablePermission(new AbstractActionInput(), tableName, TablePermissionSubType.EDIT))
{
throw (new QPermissionDeniedException("This needs the " + base + "." + feature + " permission or table EDIT."));
@@ -0,0 +1,29 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.tags;
/*******************************************************************************
** A vocabulary entry or record chip, without exposing assignment rows.
*******************************************************************************/
public record TagEntry(Integer id, String name, String label, String color, String tableName, Boolean approved, Integer usageCount)
{
}
@@ -0,0 +1,126 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.tags;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.Collection;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import com.kingsrook.qbits.collaboration.CollaborationRecords;
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
import com.kingsrook.qqq.backend.core.actions.permissions.PermissionEnforcement;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
/*******************************************************************************
** Materialized tag filter logic independent of the unreleased core hooks in #1140.
** Every operator returns matching caller-visible parent ids, including complements.
*******************************************************************************/
public class TagFilterTranslator
{
public static final int MAX_RECORD_IDS = 10000;
/*******************************************************************************
** IN means any tag, CONTAINS_ALL means every requested tag, NOT_IN means none.
** Empty IN matches none; empty CONTAINS_ALL and NOT_IN match all visible rows.
*******************************************************************************/
public List<String> recordIds(String table, String op, Collection<? extends Serializable> tags) throws QException
{
TagsService.requireTable(table);
if(!List.of("IN", "CONTAINS_ALL", "NOT_IN", "IS_BLANK", "IS_NOT_BLANK").contains(op))
{
throw (new QException("Unsupported tag operator: " + op));
}
Set<Integer> requested = new HashSet<>();
boolean missing = false;
if(!op.equals("IS_BLANK") && !op.equals("IS_NOT_BLANK"))
{
for(Serializable tag : tags == null ? List.<Serializable>of() : tags)
{
QQueryFilter filter = TagsService.vocabularyFilter(table).withCriteria("approved", QCriteriaOperator.EQUALS, true);
filter.withCriteria(tag instanceof Number ? "id" : "name", QCriteriaOperator.EQUALS,
tag instanceof Number ? tag : TagsService.normalize(tag == null ? null : tag.toString()));
List<QRecord> matches = CollaborationRecords.query(CollaborationTables.TAG, filter.withOrderBy(new QFilterOrderBy("id")));
if(matches.isEmpty())
{
missing = true;
}
else
{
QRecord match = matches.stream().filter(row -> row.getValue("tableName") == null).findFirst().orElse(matches.getFirst());
requested.add(match.getValueInteger("id"));
}
}
}
Map<String, Set<Integer>> assignments = new HashMap<>();
for(QRecord row : CollaborationRecords.query(CollaborationTables.RECORD_TAG, new QQueryFilter()
.withCriteria("tableName", QCriteriaOperator.EQUALS, table).withCriteria("parentDeleted", QCriteriaOperator.EQUALS, false)))
{
assignments.computeIfAbsent(row.getValueString("recordId"), ignored -> new HashSet<>()).add(row.getValueInteger("tagId"));
}
List<String> result = new ArrayList<>();
String key = QContext.getQInstance().getTable(table).getPrimaryKeyField();
for(int skip = 0; ; skip += 500)
{
QueryInput input = new QueryInput(table).withFilter(new QQueryFilter().withOrderBy(new QFilterOrderBy(key)).withSkip(skip).withLimit(500))
.withTransaction(QContext.getQBackendTransaction());
input.setPermissionEnforcement(PermissionEnforcement.ENFORCE);
List<QRecord> parents = new QueryAction().execute(input).getRecords();
for(QRecord parent : parents)
{
String id = parent.getValueString(key);
Set<Integer> applied = assignments.getOrDefault(id, Set.of());
boolean any = requested.stream().anyMatch(applied::contains);
boolean match = switch(op)
{
case "IN" -> any;
case "NOT_IN" -> !any;
case "CONTAINS_ALL" -> !missing && applied.containsAll(requested);
case "IS_BLANK" -> applied.isEmpty();
default -> !applied.isEmpty();
};
if(match)
{
result.add(id);
if(result.size() > MAX_RECORD_IDS)
{
throw (new QException("Tag filter exceeds 10,000 matching record ids."));
}
}
}
if(parents.size() < 500)
{
return (result);
}
}
}
}
@@ -0,0 +1,316 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.tags;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.Collection;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
import com.kingsrook.qbits.collaboration.security.ParentRecordAccess;
import com.kingsrook.qqq.backend.core.actions.permissions.TablePermissionSubType;
import com.kingsrook.qqq.backend.core.actions.permissions.UseTablePermissionCustomPermissionChecker;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QIcon;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.PermissionLevel;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRules;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QComponentType;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendComponentMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendStepMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractLoadStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractTransformStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ExtractViaQueryStep;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ProcessSummaryProviderInterface;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
/*******************************************************************************
** Managed vocabulary and host bulk actions using the standard streamed ETL flow.
*******************************************************************************/
public final class TagProcesses
{
public static final String TAGS = "tags";
public static final String TARGET = "targetTagId";
/*******************************************************************************
** Metadata factory.
*******************************************************************************/
private TagProcesses()
{
}
/*******************************************************************************
** Vocabulary record actions, never navigation entries.
*******************************************************************************/
public static List<QProcessMetaData> management()
{
return (List.of(process(CollaborationTables.TAG, "Approve"), process(CollaborationTables.TAG, "Merge"), process(CollaborationTables.TAG, "Delete")));
}
/*******************************************************************************
** Host record actions configured by tagActionTables.
*******************************************************************************/
public static List<QProcessMetaData> processesFor(String table)
{
return (List.of(process(table, "Add"), process(table, "Remove")));
}
/*******************************************************************************
** Approved scoped PVS inputs and explicit selection cardinality for clients.
*******************************************************************************/
private static QProcessMetaData process(String table, String operation)
{
boolean bulk = !table.equals(CollaborationTables.TAG);
Map<String, Serializable> values = new HashMap<>();
values.put(StreamedETLWithFrontendProcess.FIELD_SOURCE_TABLE, table);
values.put(StreamedETLWithFrontendProcess.FIELD_DESTINATION_TABLE, table);
values.put(StreamedETLWithFrontendProcess.FIELD_SUPPORTS_FULL_VALIDATION, true);
values.put(StreamedETLWithFrontendProcess.FIELD_DO_FULL_VALIDATION, true);
values.put(StreamedETLWithFrontendProcess.FIELD_PREVIEW_MESSAGE, operation + " tags on the selected records.");
QProcessMetaData process = StreamedETLWithFrontendProcess.defineProcessMetaData(ExtractViaQueryStep.class, Review.class, Apply.class, values)
.withName("collaboration" + operation + "Tags" + (bulk ? "_" + table : "")).withLabel(operation + " tags").withTableName(table).withIcon(new QIcon("label"))
.withPermissionRules(new QPermissionRules().withLevel(PermissionLevel.NOT_PROTECTED).withCustomPermissionChecker(bulk
? UseTablePermissionCustomPermissionChecker.build(table, TablePermissionSubType.READ) : new QCodeReference(TagSecurity.class)));
if(bulk || operation.equals("Merge"))
{
QQueryFilter filter = bulk ? TagsService.vocabularyFilter(table) : new QQueryFilter().withCriteria("mergedIntoTagId", QCriteriaOperator.IS_BLANK);
filter.withCriteria("approved", QCriteriaOperator.EQUALS, true).withOrderBy(new QFilterOrderBy("usageCount", false)).withOrderBy(new QFilterOrderBy("label"));
QFieldMetaData field = new QFieldMetaData(bulk ? TAGS : TARGET, QFieldType.INTEGER).withLabel(bulk ? "Tags" : "Target tag")
.withIsRequired(true).withPossibleValueSourceName(CollaborationTables.TAG).withPossibleValueSourceFilter(filter);
process.withStep(0, new QFrontendStepMetaData().withName("tags").withLabel(operation + " tags").withFormField(field)
.withComponent(new QFrontendComponentMetaData().withType(QComponentType.EDIT_FORM)
.withValue("multiSelectFields", new ArrayList<>(bulk ? List.of(TAGS) : List.of()))));
}
return (process);
}
/*******************************************************************************
** Operation comes from installed metadata, never caller-supplied values.
*******************************************************************************/
private static String operation(RunBackendStepInput input)
{
return (input.getProcess().getName().substring("collaboration".length()).split("Tags")[0]);
}
/*******************************************************************************
** PVS submissions are ids; accept serialized comma lists as well as arrays.
*******************************************************************************/
private static List<Integer> tagIds(RunBackendStepInput input) throws QException
{
Serializable value = input.getValue(TAGS);
Collection<?> values = value instanceof Collection<?> collection ? collection : java.util.Arrays.asList(value == null ? new String[0] : value.toString().split(","));
List<Integer> ids = new ArrayList<>();
try
{
for(Object id : values)
{
ids.add(Integer.valueOf(id.toString().strip()));
}
}
catch(NumberFormatException expected)
{
throw (new QException("Pick at least one approved tag."));
}
if(ids.isEmpty())
{
throw (new QException("Pick at least one approved tag."));
}
return (ids.stream().distinct().toList());
}
/*******************************************************************************
** Shared renderer for Review and Result language.
*******************************************************************************/
private static ProcessSummaryLine line(Status status, String future, String past)
{
ProcessSummaryLine line = new ProcessSummaryLine(status, 0, null);
line.setSingularFutureMessage(future);
line.setPluralFutureMessage(future);
line.setSingularPastMessage(past);
line.setPluralPastMessage(past);
return (line);
}
/*******************************************************************************
** Validate each record without mutation, reporting affected and unchanged rows.
*******************************************************************************/
public static class Review extends AbstractTransformStep implements ProcessSummaryProviderInterface
{
private ProcessSummaryLine selected;
private final ProcessSummaryLine skipped = line(Status.WARNING, "records cannot be processed (skipped)", "records skipped");
private final ProcessSummaryLine unchanged = line(Status.INFO, "records already in the requested tag state", "records already in the requested tag state");
private final ProcessSummaryLine affected = line(Status.INFO, "records will lose tag assignments", "records affected by tag deletion");
private final Set<String> affectedParents = new HashSet<>();
/*******************************************************************************
** Parent and feature checks are made independently for every selected row.
*******************************************************************************/
@Override
public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
String table = input.getProcess().getTableName();
String op = operation(input);
boolean bulk = !table.equals(CollaborationTables.TAG);
List<Integer> ids = bulk ? tagIds(input) : List.of();
if(selected == null)
{
selected = line(Status.OK, "records: " + op + " " + (bulk ? ids.size() + " " : "") + "tags", "records: " + op + " tags completed");
}
String key = QContext.getQInstance().getTable(table).getPrimaryKeyField();
TagsService service = new TagsService();
for(QRecord row : input.getRecords())
{
Serializable id = row.getValue(key);
try
{
if(bulk)
{
ParentRecordAccess.requireFeature(table, id, "tag");
service.resolve(table, ids, false);
Set<Integer> present = new HashSet<>(service.tagsFor(table, id).stream().map(TagEntry::id).toList());
if(op.equals("Add") ? present.containsAll(ids) : ids.stream().noneMatch(present::contains))
{
unchanged.incrementCountAndAddPrimaryKey(id);
}
}
else
{
TagsService.requireManage();
TagsService.requireTag(id);
if(op.equals("Merge"))
{
service.mergeTarget(id, input.getValue(TARGET));
}
if(op.equals("Delete"))
{
for(QRecord link : TagsService.links(id))
{
String parent = link.getValueString("tableName") + ":" + link.getValueString("recordId");
if(affectedParents.add(parent))
{
affected.incrementCountAndAddPrimaryKey(parent);
}
}
}
}
output.getRecords().add(row);
selected.incrementCountAndAddPrimaryKey(id);
}
catch(QException expected)
{
skipped.incrementCountAndAddPrimaryKey(id);
}
}
}
/*******************************************************************************
** Review totals are cumulative over streamed pages.
*******************************************************************************/
@Override
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean isForResultScreen)
{
ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>();
if(selected != null)
{
selected.addSelfToListIfAnyCount(lines);
}
skipped.addSelfToListIfAnyCount(lines);
unchanged.addSelfToListIfAnyCount(lines);
affected.addSelfToListIfAnyCount(lines);
return (lines);
}
}
/*******************************************************************************
** Execute with fresh authorization; report actual successes and failures.
*******************************************************************************/
public static class Apply extends AbstractLoadStep implements ProcessSummaryProviderInterface
{
private final ProcessSummaryLine done = line(Status.OK, "records will be processed", "records processed");
private final ProcessSummaryLine failed = line(Status.ERROR, "records cannot be processed", "records failed (retry required)");
/*******************************************************************************
** The management service and parent service both recheck access at load time.
*******************************************************************************/
@Override
public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
{
String table = input.getProcess().getTableName();
String key = QContext.getQInstance().getTable(table).getPrimaryKeyField();
TagsService service = new TagsService();
for(QRecord row : input.getRecords())
{
Serializable id = row.getValue(key);
try
{
switch(operation(input))
{
case "Approve" -> service.approve(id);
case "Merge" -> service.merge(id, input.getValue(TARGET));
case "Delete" -> service.delete(id);
case "Add" -> service.apply(table, id, tagIds(input), List.of());
case "Remove" -> service.apply(table, id, List.of(), tagIds(input));
default -> throw new QException("Unknown tag action.");
}
done.incrementCountAndAddPrimaryKey(id);
}
catch(QException expected)
{
failed.incrementCountAndAddPrimaryKey(id);
}
}
}
/*******************************************************************************
** Result retains review skips and reports load failures separately.
*******************************************************************************/
@Override
public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean isForResultScreen)
{
ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>();
done.addSelfToListIfAnyCount(lines);
failed.addSelfToListIfAnyCount(lines);
if(getTransformStep() instanceof Review review)
{
review.skipped.addSelfToListIfAnyCount(lines);
}
return (lines);
}
}
}
@@ -0,0 +1,104 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.tags;
import com.kingsrook.qbits.collaboration.CollaborationSecurity;
import com.kingsrook.qbits.collaboration.security.ParentRecordAccess;
import com.kingsrook.qqq.backend.core.actions.permissions.CustomPermissionChecker;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
import com.kingsrook.qqq.backend.core.model.actions.AbstractActionInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.QueryOrCountInputInterface;
import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.permissions.MetaDataWithPermissionRules;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
/*******************************************************************************
** Vocabulary reads support pickers without collaboration.view. Management actions
** require the explicit manage permission; direct writes must use the services.
*******************************************************************************/
public class TagSecurity implements CustomPermissionChecker
{
/*******************************************************************************
** Restrict before query/count/aggregate and protect direct gets as well.
*******************************************************************************/
@Override
public void checkPermissionsThrowing(AbstractActionInput input, MetaDataWithPermissionRules metadata) throws QPermissionDeniedException
{
if(metadata instanceof QProcessMetaData)
{
TagsService.requireManage();
return;
}
if(input instanceof InsertInput || input instanceof UpdateInput || input instanceof DeleteInput)
{
throw (new QPermissionDeniedException("Use the tag service actions."));
}
if(TagsService.canManage())
{
return;
}
String user = CollaborationSecurity.userId(QContext.getQSession());
if(user == null)
{
throw (new QPermissionDeniedException("Sign in to view tags."));
}
if(input instanceof QueryOrCountInputInterface query)
{
QQueryFilter original = query.getFilter();
QQueryFilter filter = new QQueryFilter().withCriteria("mergedIntoTagId", QCriteriaOperator.IS_BLANK)
.withSubFilter(new QQueryFilter().withBooleanOperator(QQueryFilter.BooleanOperator.OR).withCriteria("tableName", QCriteriaOperator.IS_BLANK)
.withCriteria("tableName", QCriteriaOperator.IN, QContext.getQInstance().getTables().keySet().stream().filter(ParentRecordAccess::canReadTable).toList()))
.withSubFilter(new QQueryFilter().withBooleanOperator(QQueryFilter.BooleanOperator.OR).withCriteria("approved", QCriteriaOperator.EQUALS, true)
.withCriteria("createdByUserId", QCriteriaOperator.EQUALS, user));
if(original != null)
{
filter.withSubFilter(original).withLimit(original.getLimit()).withSkip(original.getSkip());
filter.setOrderBys(original.getOrderBys());
}
query.setFilter(filter);
}
if(input instanceof GetInput get)
{
try
{
QRecord row = TagsService.requireTag(get.getPrimaryKey());
if((row.getValue("tableName") != null && !ParentRecordAccess.canReadTable(row.getValueString("tableName")))
|| (!Boolean.TRUE.equals(row.getValueBoolean("approved")) && !user.equals(row.getValueString("createdByUserId"))))
{
throw (new QPermissionDeniedException("Tag not available."));
}
}
catch(QException expected)
{
throw (new QPermissionDeniedException("Tag not available."));
}
}
}
}
@@ -0,0 +1,463 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration.tags;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.Collection;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Objects;
import com.kingsrook.qbits.collaboration.CollaborationRecords;
import com.kingsrook.qbits.collaboration.PersonalRecords;
import com.kingsrook.qbits.collaboration.activity.ActivityWriter;
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
import com.kingsrook.qbits.collaboration.security.ParentRecordAccess;
import com.kingsrook.qqq.backend.core.context.CapturedContext;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.exceptions.QNotFoundException;
import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.session.QSystemUserSession;
import com.kingsrook.qqq.backend.core.utils.JsonUtils;
/*******************************************************************************
** Tag vocabulary and assignments. Parent authorization precedes trusted storage.
** Mutations join the caller's backend transaction; the config lock serializes
** service calls within this instance, as in the existing collaboration services.
*******************************************************************************/
public class TagsService
{
public static final String MANAGE = com.kingsrook.qbits.collaboration.permissions.CollaborationPermissions.TAGS_MANAGE;
/*******************************************************************************
** Case-independent natural key; labels retain the user's capitalization.
*******************************************************************************/
public static String normalize(String label) throws QException
{
String name = label == null ? "" : label.strip().toLowerCase(Locale.ROOT).replaceAll("(?U)\\s+", " ").strip();
if(name.isEmpty() || name.length() > 60)
{
throw (new QException("Tag names must contain 1 to 60 characters."));
}
return (name);
}
/*******************************************************************************
** Global and table vocabulary; pending suggestions are private to their author.
*******************************************************************************/
public List<TagEntry> vocabulary(String table, String q, Integer limit) throws QException
{
requireTable(table);
QQueryFilter filter = vocabularyFilter(table).withSubFilter(new QQueryFilter().withBooleanOperator(QQueryFilter.BooleanOperator.OR)
.withCriteria("approved", QCriteriaOperator.EQUALS, true)
.withCriteria("createdByUserId", QCriteriaOperator.EQUALS, PersonalRecords.currentUserId()));
if(q != null && !q.isBlank())
{
filter.withCriteria("name", QCriteriaOperator.CONTAINS, q.strip().toLowerCase(Locale.ROOT));
}
filter.withOrderBy(new QFilterOrderBy("usageCount", false)).withOrderBy(new QFilterOrderBy("label")).withLimit(limit == null ? 20 : Math.max(1, Math.min(100, limit)));
return (CollaborationRecords.query(CollaborationTables.TAG, filter).stream().map(TagsService::entry).toList());
}
/*******************************************************************************
** Shared picker filter, also enforced when submitted values reach the service.
*******************************************************************************/
public static QQueryFilter vocabularyFilter(String table)
{
QQueryFilter scope = new QQueryFilter().withBooleanOperator(QQueryFilter.BooleanOperator.OR).withCriteria("tableName", QCriteriaOperator.IS_BLANK);
if(table != null)
{
scope.withCriteria("tableName", QCriteriaOperator.EQUALS, table);
}
return (new QQueryFilter().withCriteria("mergedIntoTagId", QCriteriaOperator.IS_BLANK).withSubFilter(scope));
}
/*******************************************************************************
** Resolve persisted policy as system, preserving the caller and transaction.
** The preferred app path supplies the APP scope when a table has several apps.
*******************************************************************************/
public String mode(String table) throws QException
{
List<String> resolved = new ArrayList<>();
QContext.withTemporaryContext(new CapturedContext(QContext.getQInstance(), new QSystemUserSession(), QContext.getQBackendTransaction(), null), () ->
{
String path = QContext.getQInstance().getTablePath(table);
String[] parts = path == null ? new String[0] : path.split("/");
String app = parts.length > 2 ? parts[parts.length - 2] : null;
for(String scope : List.of("TABLE", "APP", "INSTANCE"))
{
if(scope.equals("APP") && app == null)
{
continue;
}
QQueryFilter filter = new QQueryFilter().withCriteria("feature", QCriteriaOperator.EQUALS, "TAGS")
.withCriteria("scopeType", QCriteriaOperator.EQUALS, scope);
if(!scope.equals("INSTANCE"))
{
filter.withCriteria("scopeName", QCriteriaOperator.EQUALS, scope.equals("TABLE") ? table : app);
}
for(QRecord row : CollaborationRecords.query(CollaborationTables.FEATURE_SETTING, filter))
{
String json = row.getValueString("settings");
if(json != null && !json.isBlank())
{
Object value = settingMode(json);
if(value != null)
{
String mode = value.toString();
if(!List.of("FREE", "APPROVED_ONLY").contains(mode))
{
throw (new QException("Invalid TAGS mode: " + mode));
}
resolved.add(mode);
return;
}
}
}
}
});
return (resolved.isEmpty() ? CollaborationRecords.config().getDefaultTagMode() : resolved.getFirst());
}
/*******************************************************************************
** Invalid JSON fails closed instead of silently relaxing an approval policy.
*******************************************************************************/
private Object settingMode(String json) throws QException
{
try
{
return (JsonUtils.toObject(json, Map.class).get("mode"));
}
catch(Exception e)
{
throw (new QException("Invalid TAGS settings JSON", e));
}
}
/*******************************************************************************
** Create an approved free tag or a pending suggestion. Globals require manage.
*******************************************************************************/
public TagEntry create(String table, String label, String color) throws QException
{
if(table == null)
{
requireManage();
}
else
{
ParentRecordAccess.requireTableFeature(table, "tag");
}
synchronized(CollaborationRecords.config())
{
String name = normalize(label);
QQueryFilter filter = new QQueryFilter().withCriteria("name", QCriteriaOperator.EQUALS, name);
filter.withCriteria("tableName", table == null ? QCriteriaOperator.IS_BLANK : QCriteriaOperator.EQUALS, table);
List<QRecord> existing = CollaborationRecords.query(CollaborationTables.TAG, filter);
if(!existing.isEmpty())
{
QRecord row = existing.getFirst();
if(row.getValue("mergedIntoTagId") != null || (!Boolean.TRUE.equals(row.getValueBoolean("approved"))
&& !PersonalRecords.currentUserId().equals(row.getValueString("createdByUserId")) && !canManage()))
{
throw (new QException("Tag name is already reserved."));
}
return (entry(row));
}
boolean approved = table == null || canManage() || mode(table).equals("FREE");
return (entry(CollaborationRecords.insert(CollaborationTables.TAG, new QRecord().withValue("name", name)
.withValue("label", label.strip()).withValue("color", color).withValue("tableName", table).withValue("approved", approved)
.withValue("createdByUserId", PersonalRecords.currentUserId()).withValue("usageCount", 0))));
}
}
/*******************************************************************************
** Read chips only after resolving the canonical, visible parent key.
*******************************************************************************/
public List<TagEntry> tagsFor(String table, Serializable recordId) throws QException
{
String id = parentId(table, recordId);
List<Integer> ids = assignments(table, id).stream().map(row -> row.getValueInteger("tagId")).toList();
if(ids.isEmpty())
{
return (List.of());
}
return (CollaborationRecords.query(CollaborationTables.TAG, vocabularyFilter(table).withCriteria("id", QCriteriaOperator.IN, ids)
.withCriteria("approved", QCriteriaOperator.EQUALS, true).withOrderBy(new QFilterOrderBy("label"))).stream().map(TagsService::entry).toList());
}
/*******************************************************************************
** Resolve and validate all inputs before changing assignments. Numbers are ids;
** strings are labels (including numeric labels), with global matches preferred.
*******************************************************************************/
List<QRecord> resolve(String table, Collection<? extends Serializable> values, boolean createMissing) throws QException
{
Map<Integer, QRecord> result = new LinkedHashMap<>();
for(Serializable value : values == null ? List.<Serializable>of() : values)
{
QQueryFilter filter = vocabularyFilter(table);
if(value instanceof Number)
{
filter.withCriteria("id", QCriteriaOperator.EQUALS, value);
}
else
{
filter.withCriteria("name", QCriteriaOperator.EQUALS, normalize(Objects.toString(value, null)));
}
List<QRecord> matches = CollaborationRecords.query(CollaborationTables.TAG, filter.withOrderBy(new QFilterOrderBy("id")));
QRecord row;
if(matches.isEmpty())
{
if(!createMissing || value instanceof Number || !mode(table).equals("FREE"))
{
throw (new QException("Pick an approved tag (or suggest one)"));
}
row = requireTag(create(table, value.toString(), null).id());
}
else
{
row = matches.stream().filter(tag -> tag.getValue("tableName") == null).findFirst().orElse(matches.getFirst());
}
if(!Boolean.TRUE.equals(row.getValueBoolean("approved")))
{
throw (new QException("Pick an approved tag (or suggest one)"));
}
result.put(row.getValueInteger("id"), row);
}
return (new ArrayList<>(result.values()));
}
/*******************************************************************************
** Idempotent assignment changes, with activity only for actual changes.
** Tags intentionally do not notify followers.
*******************************************************************************/
public void apply(String table, Serializable recordId, Collection<? extends Serializable> add, Collection<? extends Serializable> remove) throws QException
{
ParentRecordAccess.requireFeature(table, recordId, "tag");
String id = parentId(table, recordId);
synchronized(CollaborationRecords.config())
{
List<QRecord> removals = resolve(table, remove, false);
List<QRecord> additions = resolve(table, add, true);
for(QRecord tag : removals)
{
List<QRecord> links = assignments(table, id).stream().filter(row -> Objects.equals(row.getValue("tagId"), tag.getValue("id"))).toList();
for(QRecord link : links)
{
CollaborationRecords.delete(CollaborationTables.RECORD_TAG, link.getValue("id"));
ActivityWriter.write(table, id, "UNTAGGED", "Tag removed", CollaborationTables.TAG, tag.getValueString("id"));
}
recompute(tag.getValueInteger("id"));
}
for(QRecord tag : additions)
{
if(assignments(table, id).stream().noneMatch(row -> Objects.equals(row.getValue("tagId"), tag.getValue("id"))))
{
CollaborationRecords.insert(CollaborationTables.RECORD_TAG, new QRecord().withValue("tableName", table).withValue("recordId", id)
.withValue("tagId", tag.getValue("id")).withValue("createdByUserId", PersonalRecords.currentUserId()).withValue("parentDeleted", false));
ActivityWriter.write(table, id, "TAGGED", "Tag added", CollaborationTables.TAG, tag.getValueString("id"));
}
recompute(tag.getValueInteger("id"));
}
}
}
/*******************************************************************************
** Approval never changes an already merged alias.
*******************************************************************************/
public void approve(Serializable id) throws QException
{
requireManage();
synchronized(CollaborationRecords.config())
{
requireTag(id);
CollaborationRecords.update(CollaborationTables.TAG, new QRecord().withValue("id", id).withValue("approved", true));
}
}
/*******************************************************************************
** Validate a merge before review and again before load; prevent scope loss.
*******************************************************************************/
public QRecord mergeTarget(Serializable sourceId, Serializable targetId) throws QException
{
requireManage();
QRecord source = requireTag(sourceId);
QRecord target = requireTag(targetId);
if(Objects.equals(source.getValue("id"), target.getValue("id")) || !Boolean.TRUE.equals(target.getValueBoolean("approved"))
|| (target.getValue("tableName") != null && !Objects.equals(source.getValue("tableName"), target.getValue("tableName"))))
{
throw (new QException("Choose a different approved tag in the same scope or global vocabulary."));
}
return (target);
}
/*******************************************************************************
** Move references with deduplication and retain the source as an audited alias.
*******************************************************************************/
public void merge(Serializable sourceId, Serializable targetId) throws QException
{
requireManage();
synchronized(CollaborationRecords.config())
{
QRecord target = mergeTarget(sourceId, targetId);
for(QRecord link : links(sourceId))
{
boolean duplicate = assignments(link.getValueString("tableName"), link.getValueString("recordId")).stream()
.anyMatch(row -> Objects.equals(row.getValue("tagId"), target.getValue("id")));
if(duplicate)
{
CollaborationRecords.delete(CollaborationTables.RECORD_TAG, link.getValue("id"));
}
else
{
CollaborationRecords.update(CollaborationTables.RECORD_TAG, new QRecord().withValue("id", link.getValue("id")).withValue("tagId", target.getValue("id")));
}
}
CollaborationRecords.update(CollaborationTables.TAG, new QRecord().withValue("id", sourceId).withValue("mergedIntoTagId", target.getValue("id")));
recompute(sourceId);
recompute(targetId);
}
}
/*******************************************************************************
** Delete the vocabulary entry and its assignments through the managed action.
*******************************************************************************/
public void delete(Serializable id) throws QException
{
requireManage();
synchronized(CollaborationRecords.config())
{
for(QRecord link : links(id))
{
CollaborationRecords.delete(CollaborationTables.RECORD_TAG, link.getValue("id"));
}
CollaborationRecords.delete(CollaborationTables.TAG, id);
}
}
/*******************************************************************************
** Trusted parent cleanup callback; the parent no longer exists to authorize.
*******************************************************************************/
public void onParentsDeleted(String table, List<? extends Serializable> ids) throws QException
{
synchronized(CollaborationRecords.config())
{
for(QRecord link : CollaborationRecords.query(CollaborationTables.RECORD_TAG, new QQueryFilter().withCriteria("tableName", QCriteriaOperator.EQUALS, table)
.withCriteria("recordId", QCriteriaOperator.IN, ids.stream().map(Object::toString).toList())))
{
CollaborationRecords.delete(CollaborationTables.RECORD_TAG, link.getValue("id"));
recompute(link.getValue("tagId"));
}
}
}
/*******************************************************************************
** Recount instead of incrementing a potentially stale denormalized value.
*******************************************************************************/
private void recompute(Serializable id) throws QException
{
CollaborationRecords.update(CollaborationTables.TAG, new QRecord().withValue("id", id).withValue("usageCount", links(id).size()));
}
/*******************************************************************************
** Internal references used by management actions after checking manage.
*******************************************************************************/
static List<QRecord> links(Serializable id) throws QException
{
return (CollaborationRecords.query(CollaborationTables.RECORD_TAG, new QQueryFilter().withCriteria("tagId", QCriteriaOperator.EQUALS, id)));
}
/*******************************************************************************
** Canonical assignment list for an authorized parent.
*******************************************************************************/
private List<QRecord> assignments(String table, String id) throws QException
{
return (CollaborationRecords.query(CollaborationTables.RECORD_TAG, CollaborationRecords.parent(table, id)));
}
/*******************************************************************************
** Reject missing and merged vocabulary ids.
*******************************************************************************/
static QRecord requireTag(Serializable id) throws QException
{
List<QRecord> rows = CollaborationRecords.query(CollaborationTables.TAG, new QQueryFilter().withCriteria("id", QCriteriaOperator.EQUALS, id)
.withCriteria("mergedIntoTagId", QCriteriaOperator.IS_BLANK));
if(rows.isEmpty())
{
throw (new QNotFoundException("Tag not found"));
}
return (rows.getFirst());
}
/*******************************************************************************
** Table reads guard vocabulary calls without needing a particular record.
*******************************************************************************/
static void requireTable(String table) throws QException
{
if(!ParentRecordAccess.canReadTable(table))
{
throw (new QNotFoundException("Not found"));
}
}
/*******************************************************************************
** Canonical keys prevent aliases such as 01 from creating duplicate assignments.
*******************************************************************************/
private String parentId(String table, Serializable id) throws QException
{
return (ParentRecordAccess.requireVisibleParent(table, id).getValueString(QContext.getQInstance().getTable(table).getPrimaryKeyField()));
}
/*******************************************************************************
** Management is an explicit permission independent of collaboration.admin.
*******************************************************************************/
static boolean canManage()
{
return (QContext.getQSession().hasPermission(MANAGE));
}
/*******************************************************************************
** Shared action and service permission check.
*******************************************************************************/
public static void requireManage() throws QPermissionDeniedException
{
if(!canManage())
{
throw (new QPermissionDeniedException("This needs " + MANAGE + "."));
}
}
/*******************************************************************************
** Public vocabulary projection.
*******************************************************************************/
private static TagEntry entry(QRecord row)
{
return (new TagEntry(row.getValueInteger("id"), row.getValueString("name"), row.getValueString("label"), row.getValueString("color"),
row.getValueString("tableName"), row.getValueBoolean("approved"), row.getValueInteger("usageCount")));
}
}
@@ -114,7 +114,7 @@ class CollaborationTest
CollaborationTables.SCOPE_TYPE, CollaborationTables.FEATURE, CollaborationTables.PERMISSION_MODE);
assertThat(instance.getPossibleValueSource(CollaborationTables.FEATURE).getEnumValues()).hasSize(12);
assertThat(instance.getApp("collaboration").getChildren()).extracting(child -> child.getName())
.containsExactlyInAnyOrder(CollaborationTables.COMMENT, CollaborationTables.ATTACHMENT, CollaborationTables.ACTIVITY, CollaborationTables.FEATURE_SETTING, CollaborationTables.USER);
.containsExactlyInAnyOrder(CollaborationTables.COMMENT, CollaborationTables.ATTACHMENT, CollaborationTables.TAG, CollaborationTables.ACTIVITY, CollaborationTables.FEATURE_SETTING, CollaborationTables.USER);
assertThat(instance.getApp("collaboration").getWidgets()).containsExactly(CollaborationHome.HOME_WIDGET);
assertThat(instance.getWidget(CollaborationHome.HOME_WIDGET)).isNotNull();
assertThat(instance.getTable(CollaborationTables.FEATURE_SETTING).getAuditRules().getAuditLevel()).isEqualTo(AuditLevel.FIELD);
@@ -177,9 +177,9 @@ class CollaborationTest
QInstance headless = new QInstance();
headless.addBackend(new QBackendMetaData().withName("memory").withBackendType(MemoryBackendModule.class));
new CollaborationQBitProducer().withQBitConfig(config.withAdminApp(false)).produce(headless);
assertThat(headless.getTables()).hasSize(11);
assertThat(headless.getTables()).hasSize(13);
assertThat(headless.getApps()).isEmpty();
assertThat(headless.getWidgets()).hasSize(3);
assertThat(headless.getWidgets()).hasSize(4);
}
@@ -489,11 +489,13 @@ public class ConversationServicesTest extends PersonalServiceFixture
installed.addSecurityKeyType(instance.getSecurityKeyType("tenant"));
installed.addScheduler(new com.kingsrook.qqq.backend.core.model.metadata.scheduleing.simple.SimpleSchedulerMetaData().withName("minute"));
CollaborationQBitConfig configured = new CollaborationQBitConfig().withBackendName("memory").withFollowActionTables(List.of("document"))
.withFollowChangeTables(List.of("document")).withSchedulerName("minute");
.withFollowChangeTables(List.of("document")).withTagActionTables(List.of("document")).withSchedulerName("minute");
new CollaborationQBitProducer().withQBitConfig(configured).produce(installed);
assertThat(installed.getProcess("collaborationFollow_document").getLabel()).isEqualTo("Follow");
assertThat(installed.getProcess("collaborationUnfollow_document").getLabel()).isEqualTo("Unfollow");
assertThat(installed.getRecordChangeListeners()).hasSize(1);
assertThat(installed.getProcess("collaborationAddTags_document").getLabel()).isEqualTo("Add tags");
assertThat(installed.getProcess("collaborationRemoveTags_document").getLabel()).isEqualTo("Remove tags");
assertThat(installed.getProcess(SendFollowDigestsProcess.NAME).getSchedule().getRepeatSeconds()).isEqualTo(60);
assertThat(installed.getProcess(SendFollowDigestsProcess.NAME).getIsHidden()).isTrue();
String sql = CollaborationSchema.postgresScript(config);
@@ -0,0 +1,448 @@
/*
* QQQ - Low-code Application Framework for Engineers.
* Copyright (C) 2021-2025. Kingsrook, LLC
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
* contact@kingsrook.com
* https://github.com/Kingsrook/
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.collaboration;
import java.io.Serializable;
import java.time.Instant;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import com.kingsrook.qbits.collaboration.app.CollaborationHome;
import com.kingsrook.qbits.collaboration.attachments.AttachmentCleanup;
import com.kingsrook.qbits.collaboration.model.CollaborationTables;
import com.kingsrook.qbits.collaboration.tags.TagEntry;
import com.kingsrook.qbits.collaboration.tags.TagFilterTranslator;
import com.kingsrook.qbits.collaboration.tags.TagProcesses;
import com.kingsrook.qbits.collaboration.tags.TagsService;
import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
import com.kingsrook.qqq.backend.core.actions.tables.CountAction;
import com.kingsrook.qqq.backend.core.actions.tables.DeleteAction;
import com.kingsrook.qqq.backend.core.actions.tables.GetAction;
import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QNotFoundException;
import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
import com.kingsrook.qqq.backend.core.instances.QInstanceValidator;
import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessOutput;
import com.kingsrook.qqq.backend.core.model.actions.tables.count.CountInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppSection;
import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendStepMetaData;
import com.kingsrook.qqq.backend.core.model.session.QSession;
import com.kingsrook.qqq.backend.core.model.session.QUser;
import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
/*******************************************************************************
** Real QQQ actions exercise tag permissions, persistence, processes and metadata.
*******************************************************************************/
public class TagsServiceTest extends PersonalServiceFixture
{
private final TagsService tags = new TagsService();
private final TagFilterTranslator filters = new TagFilterTranslator();
/*******************************************************************************
** Natural keys are locale-independent and validate after collapsing whitespace.
*******************************************************************************/
@Test
void normalizationAndFreeCreate() throws Exception
{
assertThat(TagsService.normalize(" Priority:HIGH today ")).isEqualTo("priority:high today");
assertThat(TagsService.normalize("a".repeat(60))).hasSize(60);
assertThatThrownBy(() -> TagsService.normalize("a".repeat(61))).hasMessageContaining("60");
assertThatThrownBy(() -> TagsService.normalize(null)).hasMessageContaining("60");
assertThatThrownBy(() -> TagsService.normalize(" ")).hasMessageContaining("60");
TagEntry tag = tags.create("document", "Urgent", "red");
assertThat(tag.name()).isEqualTo("urgent");
assertThat(tag.label()).isEqualTo("Urgent");
assertThat(tag.color()).isEqualTo("red");
assertThat(tag.tableName()).isEqualTo("document");
assertThat(tag.approved()).isTrue();
assertThat(tag.usageCount()).isZero();
assertThat(tags.create("document", " urgent ", null).id()).isEqualTo(tag.id());
}
/*******************************************************************************
** Approval-only never permits applying a pending tag, including its author.
*******************************************************************************/
@Test
void suggestionsAndUnknownLabels() throws Exception
{
config.withDefaultTagMode("APPROVED_ONLY");
TagEntry pending = tags.create("document", "Suggested", null);
assertThat(pending.approved()).isFalse();
assertThatThrownBy(() -> tags.apply("document", 1, List.of(pending.id()), List.of())).hasMessageContaining("Pick an approved tag");
assertThatThrownBy(() -> tags.apply("document", 1, List.of("Unknown"), List.of())).hasMessageContaining("Pick an approved tag");
assertThat(rows(CollaborationTables.TAG)).hasSize(1);
login("bob");
assertThat(tags.vocabulary("document", null, 20)).isEmpty();
assertThatThrownBy(() -> tags.create("document", "Suggested", null)).hasMessageContaining("reserved");
login("alice", TagsService.MANAGE);
assertThat(tags.create("document", "Managed", null).approved()).isTrue();
tags.approve(pending.id());
tags.apply("document", 1, List.of(pending.id()), List.of());
assertThat(tags.tagsFor("document", 1)).extracting(TagEntry::id).containsExactly(pending.id());
config.withDefaultTagMode("FREE");
tags.apply("document", 2, List.of("New on the fly"), List.of());
assertThat(tags.tagsFor("document", 2)).extracting(TagEntry::name).containsExactly("new on the fly");
}
/*******************************************************************************
** Global vocabulary and management demand the explicit permission.
*******************************************************************************/
@Test
void globalsAndManagementPermissions() throws Exception
{
assertThatThrownBy(() -> tags.create(null, "Global", null)).isInstanceOf(QPermissionDeniedException.class);
TagEntry own = tags.create("document", "Local", null);
assertThatThrownBy(() -> tags.approve(own.id())).isInstanceOf(QPermissionDeniedException.class);
assertThatThrownBy(() -> tags.delete(own.id())).isInstanceOf(QPermissionDeniedException.class);
assertThatThrownBy(() -> tags.merge(own.id(), own.id())).isInstanceOf(QPermissionDeniedException.class);
login("manager", TagsService.MANAGE);
TagEntry global = tags.create(null, "Global", null);
assertThat(global.tableName()).isNull();
assertThat(global.approved()).isTrue();
assertThat(tags.vocabulary("document", "glo", null)).extracting(TagEntry::id).containsExactly(global.id());
}
/*******************************************************************************
** Invisible parents return Not found; read-only users need an explicit grant.
*******************************************************************************/
@Test
void parentAndFeatureSecurity() throws Exception
{
TagEntry tag = tags.create("document", "A", null);
assertThatThrownBy(() -> tags.tagsFor("document", 3)).isInstanceOf(QNotFoundException.class).hasMessage("Not found");
assertThatThrownBy(() -> tags.apply("document", 3, List.of(tag.id()), List.of())).isInstanceOf(QNotFoundException.class);
QContext.init(instance, new QSession().withUser(new QUser().withIdReference("reader")).withPermissions("document.read").withSecurityKeyValue("tenant", "a"));
assertThatThrownBy(() -> tags.create("document", "No", null)).isInstanceOf(QPermissionDeniedException.class);
assertThatThrownBy(() -> tags.apply("document", 1, List.of(tag.id()), List.of())).isInstanceOf(QPermissionDeniedException.class);
QContext.getQSession().withPermissions("document.tag");
tags.create("document", "Granted", null);
tags.apply("document", 1, List.of(tag.id()), List.of());
assertThat(tags.tagsFor("document", 1)).hasSize(1);
assertThatThrownBy(() -> tags.vocabulary("missing", null, 1)).isInstanceOf(QNotFoundException.class);
assertThatThrownBy(() -> tags.create("missing", "No", null)).isInstanceOf(QNotFoundException.class);
}
/*******************************************************************************
** Repeated add/remove does not inflate counts or produce duplicate activity.
*******************************************************************************/
@Test
void idempotentAssignmentsCountsAndActivity() throws Exception
{
TagEntry tag = tags.create("document", "A", null);
tags.apply("document", "01", List.of(tag.id(), tag.id()), List.of());
tags.apply("document", 1, List.of("A"), null);
tags.apply("document", 2, List.of(tag.id()), List.of());
assertThat(rows(CollaborationTables.RECORD_TAG)).hasSize(2);
assertThat(tags.vocabulary("document", null, 20).getFirst().usageCount()).isEqualTo(2);
assertThat(rows(CollaborationTables.ACTIVITY)).extracting(row -> row.getValueString("type")).containsExactly("TAGGED", "TAGGED");
assertThat(rows(CollaborationTables.FOLLOW_DIGEST)).isEmpty();
tags.apply("document", 1, null, List.of(tag.id()));
tags.apply("document", 1, List.of(), List.of("A"));
assertThat(tags.tagsFor("document", 1)).isEmpty();
assertThat(tags.vocabulary("document", null, 20).getFirst().usageCount()).isEqualTo(1);
assertThat(rows(CollaborationTables.ACTIVITY)).hasSize(3);
}
/*******************************************************************************
** TABLE wins over APP then INSTANCE then the code default, with caller restored.
*******************************************************************************/
@Test
void settingsHierarchyReadAsSystem() throws Exception
{
instance.addApp(new QAppMetaData().withName("work").withLabel("Work").withChild(instance.getTable("document")).withSection(new QAppSection().withName("main").withTables(List.of("document"))));
assertThat(tags.mode("document")).isEqualTo("FREE");
setting("INSTANCE", "*", "APPROVED_ONLY");
assertThat(tags.mode("document")).isEqualTo("APPROVED_ONLY");
setting("APP", "work", "FREE");
assertThat(tags.mode("document")).isEqualTo("FREE");
setting("TABLE", "document", "APPROVED_ONLY");
assertThat(tags.create("document", "Pending", null).approved()).isFalse();
assertThat(QContext.getQSession().getUser().getIdReference()).isEqualTo("alice");
assertThat(QContext.getQSession().hasPermission("collaboration.admin")).isFalse();
}
/*******************************************************************************
** Vocabulary ordering, scope, limits and private suggestions are stable.
*******************************************************************************/
@Test
void vocabularyOrderingScopeAndSuggestions() throws Exception
{
TagEntry z = tags.create("document", "Zulu", null);
TagEntry b = tags.create("document", "Beta", null);
TagEntry a = tags.create("document", "Alpha", null);
tags.apply("document", 1, List.of(z.id()), List.of());
config.withDefaultTagMode("APPROVED_ONLY");
TagEntry pending = tags.create("document", "Mine", null);
login("bob");
tags.create("document", "Other", null);
login("alice");
assertThat(tags.vocabulary("document", null, 1000)).extracting(TagEntry::id).containsExactly(z.id(), a.id(), b.id(), pending.id());
assertThat(tags.vocabulary("document", " ALP ", 10)).extracting(TagEntry::id).containsExactly(a.id());
assertThat(tags.vocabulary("document", null, 0)).hasSize(1);
assertThat(tags.vocabulary("document", null, 2)).hasSize(2);
}
/*******************************************************************************
** Merge deduplicates assignments, preserves aliases and rejects invalid scope.
*******************************************************************************/
@Test
void mergeDeleteAndParentCleanup() throws Exception
{
TagEntry source = tags.create("document", "Source", null);
TagEntry target = tags.create("document", "Target", null);
tags.apply("document", 1, List.of(source.id(), target.id()), List.of());
tags.apply("document", 2, List.of(source.id()), List.of());
login("manager", TagsService.MANAGE);
assertThatThrownBy(() -> tags.merge(source.id(), source.id())).hasMessageContaining("different");
TagEntry global = tags.create(null, "Global", null);
assertThatThrownBy(() -> tags.merge(global.id(), target.id())).hasMessageContaining("scope");
tags.merge(source.id(), target.id());
assertThat(tags.tagsFor("document", 1)).extracting(TagEntry::id).containsExactly(target.id());
assertThat(tags.tagsFor("document", 2)).extracting(TagEntry::id).containsExactly(target.id());
assertThat(tags.vocabulary("document", null, 20)).extracting(TagEntry::id).doesNotContain(source.id());
assertThat(rows(CollaborationTables.TAG)).filteredOn(row -> row.getValueInteger("id").equals(source.id()))
.singleElement().satisfies(row -> { assertThat(row.getValueInteger("usageCount")).isZero(); assertThat(row.getValueInteger("mergedIntoTagId")).isEqualTo(target.id()); });
assertThatThrownBy(() -> tags.create("document", "Source", null)).hasMessageContaining("reserved");
new DeleteAction().execute(new DeleteInput("document").withPrimaryKeys(List.of(1)));
new AttachmentCleanup().onParentsDeleted("document", List.of(1));
new AttachmentCleanup().onParentsDeleted("document", List.of(1));
assertThat(tags.vocabulary("document", null, 20).getFirst().usageCount()).isEqualTo(1);
tags.delete(target.id());
assertThat(rows(CollaborationTables.RECORD_TAG)).isEmpty();
}
/*******************************************************************************
** All translator operators use visible parent ids, with set semantics.
*******************************************************************************/
@Test
void translatorOperators() throws Exception
{
TagEntry a = tags.create("document", "A", null);
TagEntry b = tags.create("document", "B", null);
tags.apply("document", 1, List.of(a.id(), b.id()), List.of());
tags.apply("document", 2, List.of(a.id()), List.of());
assertThat(filters.recordIds("document", "IN", List.of("A"))).containsExactly("1", "2");
assertThat(filters.recordIds("document", "IN", List.of(b.id()))).containsExactly("1");
assertThat(filters.recordIds("document", "CONTAINS_ALL", List.of(a.id(), b.id(), a.id()))).containsExactly("1");
assertThat(filters.recordIds("document", "CONTAINS_ALL", List.of(a.id(), "unknown"))).isEmpty();
assertThat(filters.recordIds("document", "NOT_IN", List.of(b.id()))).containsExactly("2");
assertThat(filters.recordIds("document", "IS_NOT_BLANK", null)).containsExactly("1", "2");
assertThat(filters.recordIds("document", "IS_BLANK", null)).isEmpty();
tags.apply("document", 2, List.of(), List.of(a.id()));
assertThat(filters.recordIds("document", "IS_BLANK", List.of())).containsExactly("2");
assertThat(filters.recordIds("document", "IN", List.of())).isEmpty();
assertThat(filters.recordIds("document", "NOT_IN", List.of("unknown"))).containsExactly("1", "2");
assertThat(filters.recordIds("document", "CONTAINS_ALL", List.of())).containsExactly("1", "2");
assertThatThrownBy(() -> filters.recordIds("document", "LIKE", List.of())).hasMessageContaining("Unsupported");
}
/*******************************************************************************
** Exactly 10,000 is allowed; the next matching id fails instead of truncating.
*******************************************************************************/
@Test
void translatorCap() throws Exception
{
List<QRecord> parents = new ArrayList<>();
for(int id = 4; id <= 10001; id++)
{
parents.add(new QRecord().withValue("id", id).withValue("tenant", "a"));
}
new InsertAction().execute(new InsertInput("document").withRecords(parents));
assertThat(filters.recordIds("document", "IS_BLANK", null)).hasSize(10000);
new InsertAction().execute(new InsertInput("document").withRecords(List.of(new QRecord().withValue("id", 10002).withValue("tenant", "a"))));
assertThatThrownBy(() -> filters.recordIds("document", "IS_BLANK", null)).hasMessageContaining("10,000");
}
/*******************************************************************************
** Direct access cannot bypass vocabulary approval or parent lock restrictions.
*******************************************************************************/
@Test
void tableSecurityAndHome() throws Exception
{
TagEntry tag = tags.create("document", "<Urgent>", null);
tags.apply("document", 1, List.of(tag.id()), List.of());
config.withDefaultTagMode("APPROVED_ONLY");
TagEntry pending = tags.create("document", "Pending", null);
login("bob", "collaboration.view");
assertThat(new QueryAction().execute(new QueryInput(CollaborationTables.TAG)).getRecords()).hasSize(1);
assertThat(new CountAction().execute(new CountInput(CollaborationTables.TAG)).getCount()).isEqualTo(1);
assertThatThrownBy(() -> new GetAction().execute(new GetInput(CollaborationTables.TAG).withPrimaryKey(pending.id()))).isInstanceOf(QPermissionDeniedException.class);
assertThat(new GetAction().execute(new GetInput(CollaborationTables.TAG).withPrimaryKey(tag.id())).getRecord()).isNotNull();
assertThatThrownBy(() -> new InsertAction().execute(new InsertInput(CollaborationTables.TAG).withRecords(List.of(new QRecord())))).isInstanceOf(QPermissionDeniedException.class);
assertThatThrownBy(() -> new UpdateAction().execute(new UpdateInput(CollaborationTables.TAG).withRecords(List.of(new QRecord().withValue("id", tag.id()))))).isInstanceOf(QPermissionDeniedException.class);
assertThatThrownBy(() -> new DeleteAction().execute(new DeleteInput(CollaborationTables.TAG).withPrimaryKeys(List.of(tag.id())))).isInstanceOf(QPermissionDeniedException.class);
assertThat(new QueryAction().execute(new QueryInput(CollaborationTables.RECORD_TAG)).getRecords()).hasSize(1);
QContext.getQSession().getSecurityKeyValues().put("tenant", List.of("b"));
assertThat(new QueryAction().execute(new QueryInput(CollaborationTables.RECORD_TAG)).getRecords()).isEmpty();
login("manager", "collaboration.admin", TagsService.MANAGE);
String html = CollaborationHome.render(Instant.now());
assertThat(html).contains("Vocabulary size: 2", "Pending suggestions: 1", "&lt;Urgent&gt;: 1");
assertThat(html).doesNotContain("<Urgent>");
}
/*******************************************************************************
** Metadata, nullable uniqueness and scoped PVS are validated together.
*******************************************************************************/
@Test
void schemaAndMetadata() throws Exception
{
TagProcesses.processesFor("document").forEach(instance::addProcess);
new QInstanceValidator().validate(instance);
assertThat(instance.getTable(CollaborationTables.TAG).getSearchFields()).containsExactly("label");
assertThat(instance.getTable(CollaborationTables.TAG).getSections()).anySatisfy(section -> assertThat(section.getLabel()).isEqualTo("Record Tags"));
assertThat(instance.getPossibleValueSource(CollaborationTables.TAG).getTableName()).isEqualTo(CollaborationTables.TAG);
QFrontendStepMetaData pick = (QFrontendStepMetaData) instance.getProcess("collaborationAddTags_document").getStep("tags");
assertThat(pick.getFormFields().getFirst().getPossibleValueSourceName()).isEqualTo(CollaborationTables.TAG);
assertThat(pick.getComponents().getFirst().getValues().get("multiSelectFields")).isEqualTo(List.of("tags"));
assertThat(CollaborationSchema.postgresScript(config)).contains("ON collaboration_tag (table_name, name) NULLS NOT DISTINCT", "ON collaboration_record_tag (table_name, record_id, tag_id)", "usage_count INTEGER NOT NULL", "approved BOOLEAN NOT NULL");
assertThatThrownBy(() -> config.withDefaultTagMode("INVALID").validate(instance)).hasMessageContaining("defaultTagMode");
config.withDefaultTagMode("FREE").withTagActionTables(List.of("missing"));
assertThatThrownBy(() -> config.validate(instance)).hasMessageContaining("host table");
}
/*******************************************************************************
** Bulk actions execute genuine Review and Result steps with idempotent counts.
*******************************************************************************/
@Test
void bulkActionsReviewAndResult() throws Exception
{
TagProcesses.processesFor("document").forEach(instance::addProcess);
TagEntry a = tags.create("document", "A", null);
TagEntry b = tags.create("document", "B", null);
tags.apply("document", 1, List.of(a.id(), b.id()), List.of());
RunProcessOutput review = review("collaborationAddTags_document", "1,2", Map.of(TagProcesses.TAGS, new ArrayList<>(List.of(a.id(), b.id()))));
assertThat(lines(review, false)).anySatisfy(line -> { assertThat(line.getCount()).isEqualTo(2); assertThat(line.getPluralFutureMessage()).contains("Add 2 tags"); });
assertThat(lines(review, false)).anySatisfy(line -> { assertThat(line.getCount()).isEqualTo(1); assertThat(line.getPluralFutureMessage()).contains("already"); });
assertThat(tags.tagsFor("document", 2)).isEmpty();
assertThat(lines(finish(review), true)).anySatisfy(line -> assertThat(line.getCount()).isEqualTo(2));
assertThat(tags.tagsFor("document", 2)).hasSize(2);
finish(review("collaborationRemoveTags_document", "1,2", Map.of(TagProcesses.TAGS, a.id() + "," + b.id())));
assertThat(rows(CollaborationTables.RECORD_TAG)).isEmpty();
}
/*******************************************************************************
** Management Review counts affected records; Result performs each operation.
*******************************************************************************/
@Test
void managementActionsReviewAndResult() throws Exception
{
config.withDefaultTagMode("APPROVED_ONLY");
TagEntry pending = tags.create("document", "Pending", null);
login("manager", TagsService.MANAGE);
RunProcessOutput approve = review("collaborationApproveTags", pending.id().toString(), Map.of());
assertThat(lines(approve, false)).anySatisfy(line -> assertThat(line.getPluralFutureMessage()).contains("Approve"));
finish(approve);
tags.apply("document", 1, List.of(pending.id()), List.of());
TagEntry target = tags.create("document", "Target", null);
RunProcessOutput merge = review("collaborationMergeTags", pending.id().toString(), Map.of(TagProcesses.TARGET, target.id()));
finish(merge);
assertThat(tags.tagsFor("document", 1)).extracting(TagEntry::id).containsExactly(target.id());
RunProcessOutput delete = review("collaborationDeleteTags", target.id().toString(), Map.of());
assertThat(lines(delete, false)).anySatisfy(line -> { assertThat(line.getCount()).isEqualTo(1); assertThat(line.getPluralFutureMessage()).contains("lose tag assignments"); });
assertThat(lines(finish(delete), true)).anySatisfy(line -> assertThat(line.getPluralPastMessage()).contains("processed"));
assertThat(tags.tagsFor("document", 1)).isEmpty();
}
/*******************************************************************************
** Persist one scoped feature policy as a trusted fixture setup.
*******************************************************************************/
private void setting(String scope, String name, String mode) throws Exception
{
CollaborationRecords.insert(CollaborationTables.FEATURE_SETTING, new QRecord().withValue("scopeType", scope).withValue("scopeName", name)
.withValue("feature", "TAGS").withValue("enabled", true).withValue("settings", "{\"mode\":\"" + mode + "\"}"));
}
/*******************************************************************************
** Inspect persistence to assert no hidden writes or duplicate activity.
*******************************************************************************/
private List<QRecord> rows(String table) throws Exception
{
return (CollaborationRecords.query(table, new QQueryFilter()));
}
/*******************************************************************************
** Supply picker values and stop at the genuine standard Review screen.
*******************************************************************************/
private RunProcessOutput review(String name, String records, Map<String, Serializable> values) throws Exception
{
new QInstanceValidator().validate(instance);
RunProcessInput input = new RunProcessInput();
input.setProcessName(name);
input.addValue("testProcessName", name);
input.setFrontendStepBehavior(RunProcessInput.FrontendStepBehavior.BREAK);
input.addValue(StreamedETLWithFrontendProcess.FIELD_DO_FULL_VALIDATION, true);
input.addValue("recordsParam", "recordIds");
input.addValue("recordIds", records);
values.forEach(input::addValue);
if(instance.getProcess(name).getStep("tags") != null)
{
RunProcessOutput picker = new RunProcessAction().execute(input);
assertThat(picker.getException()).isEmpty();
input.setProcessUUID(picker.getProcessUUID());
input.setStartAfterStep("tags");
}
RunProcessOutput output = new RunProcessAction().execute(input);
assertThat(output.getException()).isEmpty();
return (output);
}
/*******************************************************************************
** Resume the stored process state after Review.
*******************************************************************************/
private RunProcessOutput finish(RunProcessOutput review) throws Exception
{
RunProcessInput go = new RunProcessInput();
go.setProcessName((String) review.getValues().get("testProcessName"));
go.setProcessUUID(review.getProcessUUID());
go.setStartAfterStep(StreamedETLWithFrontendProcess.STEP_NAME_REVIEW);
go.setFrontendStepBehavior(RunProcessInput.FrontendStepBehavior.SKIP);
RunProcessOutput result = new RunProcessAction().execute(go);
assertThat(result.getException()).isEmpty();
return (result);
}
/*******************************************************************************
** Summary values are supplied by the streamed process engine.
*******************************************************************************/
@SuppressWarnings("unchecked")
private List<ProcessSummaryLine> lines(RunProcessOutput output, boolean result)
{
return ((List<ProcessSummaryLine>) output.getValues().get(result ? StreamedETLWithFrontendProcess.FIELD_PROCESS_SUMMARY : StreamedETLWithFrontendProcess.FIELD_VALIDATION_SUMMARY));
}
}