Compare commits

...
Author SHA1 Message Date
james.maes a1ca3121e3 Bump version to 1.1.0-RC.12 [skip ci] 2026-10-15 14:33:56 -05:00
james.maes 6028a66ece Bump version to 1.1.0-RC.11 [skip ci] 2026-10-15 14:21:14 -05:00
james.maesandClaude Opus 5.5 ca8d776845 Merge develop into release/1.1 for 1.1.0-RC.11 (cluster caches, QQQ RC.14)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 14:17:06 -05:00
james.maes 832c628298 Merge pull request #62 from QRun-IO/feature/cluster-caches
Clear the Quick Search config cache on every node (ClusterCaches)
2026-10-15 14:11:18 -05:00
james.maesandClaude Opus 5.5 b4c7ba4697 build: QQQ 4.1.0-RC.14 (ships ClusterCaches)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 14:05:04 -05:00
james.maesandClaude Opus 5.5 af9eefe311 feat(cluster): clear the Quick Search config cache on every node
Register the runtime config cache (config store snapshot and settings,
per-table enabled flags) with QQQ ClusterCaches as
qbit-quick-search.config. invalidateConfig() now clears locally and
publishes a cluster invalidation, so a config edit on one node reaches
every node at once; the existing TTLs stay as the safety net. The
warn-once table set in QuickSearchFieldRules is registered node-local.

Builds against QQQ 4.1.0-SNAPSHOT (qqq.snapshot.version) until the QQQ
RC with ClusterCaches is published; a release must pin that RC.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 12:10:35 -05:00
CircleCI 2e2501df31 Bump version to 1.1.0-RC.11 [skip ci] 2026-10-15 12:57:27 +00:00
github-actions[bot] eccffec4b9 Merge develop into release/1.1 for the next 1.1.0 RC 2026-10-15 12:52:38 +00:00
james.maes 3f7e67df4b Merge pull request #61 from QRun-IO/feature/quiet-index-bookkeeping-audits
fix(audit): do not record-audit indexing bookkeeping
2026-10-15 07:52:26 -05:00
james.maesandClaude Opus 5.5 a6887f7ea2 fix(audit): do not record-audit indexing bookkeeping
Basepull, full reindex and real-time indexing update runtime state on the
quick_search_index row (status, lastBasepullTime, documentCount, last run,
realTimeErrorCount), and every startup re-records the priming outcome on the
settings row. Each of those writes produced a FIELD audit ("Record was Edited
during process: Update search index"), about 2,800 audit rows in 6 hours on
kof22 production. These writes now set omitDmlAudit; admin edits to the
index, field and settings tables keep their audits. Tests prove both sides.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 07:47:23 -05:00
CircleCI ff8dba8cbb Bump version to 1.1.0-RC.10 [skip ci] 2026-10-15 03:22:02 +00:00
james.maesandClaude Opus 5.5 6e13c1505d Merge develop into release/1.1 for 1.1.0-RC.10 (#59 help bundle, #57 health, #60 bundle version)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 22:13:04 -05:00
james.maes bce0f7ff53 Merge pull request #60 from QRun-IO/fix/help-bundle-version
fix(help): help bundle version comes from the build, not a hardcoded SNAPSHOT
2026-10-14 22:07:01 -05:00
CircleCI 035697f2cf Bump version to 1.1.0-RC.9 [skip ci] 2026-10-14 22:35:38 +00:00
james.maesandClaude Opus 5.5 4fd4a9423f Merge develop into release/1.1 for 1.1.0-RC.9 (#58 setup card, seeds, help)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:27:14 -05:00
CircleCI d8efbee58d Bump version to 1.1.0-RC.8 [skip ci] 2026-10-14 19:03:33 +00:00
james.maesandClaude Opus 5.5 12d85c9923 Merge develop into release/1.1 for 1.1.0-RC.8 (#40 scores/highlights, #56 suggest)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 13:54:50 -05:00
CircleCI 5265d203a9 Bump version to 1.1.0-RC.7 [skip ci] 2026-10-14 16:37:04 +00:00
james.maesandClaude Opus 5.5 3f5dfa6d67 Merge develop into release/1.1 for 1.1.0-RC.7 (#55 never index PII/PHI/secure fields)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 11:26:19 -05:00
CircleCI 20aab6aae5 Bump version to 1.1.0-RC.6 [skip ci] 2026-10-14 15:58:42 +00:00
james.maesandClaude Opus 5.5 1cd32841fd Merge develop into release/1.1 for 1.1.0-RC.6 (#54 realTimeErrorCount)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 10:51:21 -05:00
CircleCI 532a17c889 Bump version to 1.1.0-RC.5 [skip ci] 2026-10-14 12:31:17 +00:00
james.maesandClaude Opus 5.5 6ed75ffd6d Merge develop into release/1.1 for 1.1.0-RC.5 (#51 QuickSearchStatus API on QQQ RC.6)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 07:23:17 -05:00
CircleCI ad20b350e6 Bump version to 1.1.0-RC.4 [skip ci] 2026-10-12 21:18:41 +00:00
james.maesandClaude Opus 5.5 1298af3c57 Merge develop into release/1.1 for 1.1.0-RC.4
Data-driven Quick Search configuration (#47).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 16:14:17 -05:00
james.maesandClaude Opus 5.5 a8c0210643 ci: fix the GA tag filter YAML (no brackets in the flow sequence) [skip ci]
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 16:01:58 -05:00
james.maesandClaude Opus 5.5 9343143b33 ci: publish only GA tags on this release branch [skip ci]
RC tags no longer trigger a publish (/^v[0-9]+\.[0-9]+\.[0-9]+$/ only).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 15:57:03 -05:00
CircleCI 6d656369cf Bump version to 1.1.0-RC.3 [skip ci] 2026-10-12 20:34:00 +00:00
james.maesandClaude Opus 5.5 5c39fbb69e Merge develop into release/1.1 for 1.1.0-RC.3
Brings the record developer panel provider (#41) and the snake_case
table, PVS and permission names (#42).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 15:29:29 -05:00
CircleCI bd110b0814 Bump version to 1.1.0-RC.2 [skip ci] 2026-10-12 12:46:46 +00:00
james.maesandClaude Opus 5.5 16ee516447 Merge develop into release/1.1 for 1.1.0-RC.2
Brings search admin tooling (#35/#39), stuck REBUILDING recovery (#24/#25),
cron zone validation (#29/#31) and docs fixes to the 1.1 release line.
Owner approval 2026-10-12: "all approved and releases approved".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 07:42:40 -05:00
CircleCI 60e977cf95 Bump version to 1.1.0-RC.1 [skip ci] 2026-10-11 18:55:55 +00:00
11 changed files with 360 additions and 9 deletions

No files matched your search

+3 -1
View File
@@ -4,9 +4,11 @@ All notable changes to this project are documented here. The format follows Keep
## [Unreleased]
Requires QQQ 4.1.0-RC.8 or later (central health checks, help bundles and the spec-managed UI API extension contract).
Requires QQQ 4.1.0-RC.8 or later (central health checks, help bundles and the spec-managed UI API extension contract), and a QQQ 4.1 build with `ClusterCaches` for multi-node deployments (the pom builds against 4.1.0-SNAPSHOT until the next QQQ RC; a release must pin that RC).
### Fixed
- Multi-node deployments: a Quick Search config change (an index row, its fields or the settings row, from the admin UI or a config process) now clears the cached config on every node, not only the node that made the change. The runtime's config cache (the config store snapshot and settings, and the per-table enabled flags) is registered with QQQ `ClusterCaches` as `qbit-quick-search.config`, and `QuickSearchRuntime.invalidateConfig()` publishes a cluster invalidation. The 30 s and 60 s TTLs remain as the safety net. `QuickSearchFieldRules`' warn-once table set is registered as node-local. `QuickSearchClusterCachesTest` audits the package for unregistered static caches and covers a remote invalidation.
- Indexing bookkeeping is no longer record-audited: the `quick_search_index` runtime fields written by basepull, full reindex and real-time indexing (status, last basepull time, document count, last run, real-time error count) and the settings row's priming outcome (`lastPrimedAt`, summary) are written with `omitDmlAudit`. On a production host these wrote about 470 "Record was Edited during process: Update search index" audit rows an hour. Admin edits to an index, its fields or the settings are still audited.
- Real-time indexing failures now increase the table's error count by the number of failed events. Successful basepulls and full reindexes reset the count. Counter update failures are logged without failing the source write.
### Added
+2 -2
View File
@@ -20,7 +20,7 @@
<properties>
<qbit.guardrails.enforce>true</qbit.guardrails.enforce>
<qbit.guardrails.strictUiApiRoutes>true</qbit.guardrails.strictUiApiRoutes>
<revision>1.1.0-SNAPSHOT</revision>
<revision>1.1.0-RC.12</revision>
<maven.compiler.source>21</maven.compiler.source>
<maven.compiler.target>21</maven.compiler.target>
<opensearch.version>3.10.0</opensearch.version>
@@ -283,7 +283,7 @@
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<qqq.snapshot.version>4.1.0-RC.8</qqq.snapshot.version>
<qqq.snapshot.version>4.1.0-RC.14</qqq.snapshot.version>
</properties>
<dependencyManagement>
<dependencies>
@@ -23,6 +23,8 @@ import java.util.Collections;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.WeakHashMap;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicLong;
@@ -34,6 +36,7 @@ import com.kingsrook.qbits.quicksearch.opensearch.QuickSearchOpenSearchClient;
import com.kingsrook.qbits.quicksearch.publisher.IndexEventPublisher;
import com.kingsrook.qbits.quicksearch.publisher.SynchronousIndexEventPublisher;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.cluster.ClusterCaches;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
@@ -70,6 +73,17 @@ public class QuickSearchRuntime
private static final long ENABLED_CACHE_TTL_MILLIS = 60_000L;
/*******************************************************************************
** Cluster cache name for every runtime's config cache (the config store's
** snapshot and settings, and the enabled flags). A config row edited on
** one node clears it on every node; the TTLs remain the safety net.
*******************************************************************************/
public static final String CONFIG_CACHE_NAME = "qbit-quick-search.config";
private static final Set<QuickSearchRuntime> RUNTIMES = ClusterCaches.shared(CONFIG_CACHE_NAME,
Collections.synchronizedSet(Collections.newSetFromMap(new WeakHashMap<>())),
QuickSearchRuntime::invalidateAllLocal, null);
public static final long RECORD_SEARCH_BACKOFF_MILLIS = 30_000L;
private final StatusCache statusCache = new StatusCache();
@@ -128,6 +142,7 @@ public class QuickSearchRuntime
{
this.config = config;
this.configStore = new QuickSearchConfigStore(this);
RUNTIMES.add(this);
}
@@ -261,9 +276,21 @@ public class QuickSearchRuntime
/*******************************************************************************
** Drop every cached config (a config row changed).
** Drop every cached config (a config row changed): on this node now, and
** on the other nodes through the cluster bus.
*******************************************************************************/
public void invalidateConfig()
{
invalidateConfigLocal();
ClusterCaches.invalidate(CONFIG_CACHE_NAME);
}
/*******************************************************************************
** Drop this runtime's cached config on this node only.
*******************************************************************************/
void invalidateConfigLocal()
{
configStore.invalidate();
enabledCache.clear();
@@ -271,6 +298,25 @@ public class QuickSearchRuntime
/*******************************************************************************
** Drop the cached config of every live runtime on this node (a cluster
** invalidation arrived).
*******************************************************************************/
private static void invalidateAllLocal()
{
List<QuickSearchRuntime> runtimes;
synchronized(RUNTIMES)
{
runtimes = new ArrayList<>(RUNTIMES);
}
for(QuickSearchRuntime runtime : runtimes)
{
runtime.invalidateConfigLocal();
}
}
/*******************************************************************************
** Return the OpenSearch client, building it on first use. Building the
** client does not contact the cluster; {@link #ensureIndexReady()} does.
@@ -422,6 +468,16 @@ public class QuickSearchRuntime
/*******************************************************************************
** How many enabled flags are cached (for tests).
*******************************************************************************/
int getCachedEnabledFlagCount()
{
return (enabledCache.size());
}
/*******************************************************************************
** Forget cached enabled flags (after an operator toggles a table).
*******************************************************************************/
@@ -741,6 +797,7 @@ public class QuickSearchRuntime
return;
}
closed = true;
RUNTIMES.remove(this);
statusCache.close();
if(publisher != null)
@@ -412,8 +412,12 @@ public class QuickSearchConfigPrimer
}
else
{
///////////////////////////////////////////////////////////////////////
// Only the priming outcome (lastPrimedAt, summary) changes here, on //
// every startup: bookkeeping, not a settings change, so not audited. //
///////////////////////////////////////////////////////////////////////
values.withValue("id", rows.get(0).getValue("id"));
updateAll(settingsTable, List.of(values));
updateAll(settingsTable, List.of(values), true);
}
}
@@ -477,6 +481,17 @@ public class QuickSearchConfigPrimer
** Update rows, failing on any record error.
*******************************************************************************/
public static void updateAll(String tableName, List<QRecord> records) throws QException
{
updateAll(tableName, records, false);
}
/*******************************************************************************
** Update rows, failing on any record error; omitAudit skips the DML audit
** for runtime bookkeeping writes.
*******************************************************************************/
public static void updateAll(String tableName, List<QRecord> records, boolean omitAudit) throws QException
{
if(records.isEmpty())
{
@@ -485,6 +500,7 @@ public class QuickSearchConfigPrimer
UpdateInput updateInput = new UpdateInput();
updateInput.setTableName(tableName);
updateInput.setRecords(records);
updateInput.setOmitDmlAudit(omitAudit);
for(QRecord record : CollectionUtils.nonNullList(new UpdateAction().execute(updateInput).getRecords()))
{
if(CollectionUtils.nullSafeHasContents(record.getErrors()))
@@ -60,8 +60,9 @@ import static com.kingsrook.qqq.backend.core.logging.LogUtils.logPair;
** before 1.1.0-RC.3.
**
** Reads are cached for CACHE_TTL_MILLIS and dropped at once by the config
** tables' customizers when a row changes on this server; other servers pick
** the change up when their cache expires.
** tables' customizers when a row changes, on every server: the runtime
** publishes a cluster cache invalidation (QuickSearchRuntime.CONFIG_CACHE_NAME).
** The TTL remains the safety net when the cluster bus is down.
*******************************************************************************/
public class QuickSearchConfigStore
{
@@ -29,6 +29,7 @@ import com.kingsrook.qqq.backend.core.actions.ai.AiDataClassifications;
import com.kingsrook.qqq.backend.core.actions.ai.DataClassification;
import com.kingsrook.qqq.backend.core.actions.permissions.PermissionsHelper;
import com.kingsrook.qqq.backend.core.actions.permissions.TablePermissionSubType;
import com.kingsrook.qqq.backend.core.cluster.ClusterCaches;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.logging.QLogger;
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
@@ -59,7 +60,8 @@ import com.kingsrook.qbits.quicksearch.model.QuickSearchFieldRole;
public final class QuickSearchFieldRules
{
private static final QLogger LOG = QLogger.getLogger(QuickSearchFieldRules.class);
private static final WeakHashMap<QTableMetaData, Boolean> WARNED_TABLES = new WeakHashMap<>();
private static final WeakHashMap<QTableMetaData, Boolean> WARNED_TABLES = ClusterCaches.nodeLocal(new WeakHashMap<>(),
"warn-once log de-duplication for this node's table metadata objects");
public static final int MAX_DEFAULT_FIELDS = 20;
public static final int TITLE_WEIGHT = 3;
@@ -334,6 +334,8 @@ public class QuickSearchRecordChangeListener implements RecordChangeListenerInte
updateInput.setTableName(indexTable);
updateInput.setRecords(List.of(new QRecord().withValue("id", row.getValue("id"))
.withValue("realTimeErrorCount", Math.addExact(previous == null ? 0 : previous, entry.getValue()))));
// A runtime counter, not a configuration change: not record-audited.
updateInput.setOmitDmlAudit(true);
List<QRecord> updated = QuickSearchConfigWrites.asSystem(() -> new UpdateAction().execute(updateInput)).getRecords();
if(CollectionUtils.nullSafeIsEmpty(updated) || CollectionUtils.nullSafeHasContents(updated.get(0).getErrors()))
{
@@ -493,7 +493,10 @@ public abstract class AbstractIndexingStep implements BackendStep
/*******************************************************************************
** Update fields on a quick_search_index row.
** Update runtime state on a quick_search_index row (status, last run, last
** basepull time, document count). Indexing bookkeeping, not a configuration
** change: it is not record-audited, or every basepull tick would write an
** audit row. Admin edits to the row go through the table and stay audited.
*******************************************************************************/
protected void updateIndexRow(Integer indexId, Map<String, ? extends Serializable> values) throws QException
{
@@ -506,6 +509,7 @@ public abstract class AbstractIndexingStep implements BackendStep
UpdateInput updateInput = new UpdateInput();
updateInput.setTableName(getConfig().getQuickSearchIndexTableName());
updateInput.setRecords(List.of(record));
updateInput.setOmitDmlAudit(true);
QuickSearchConfigWrites.asSystem(() -> new UpdateAction().execute(updateInput));
}
@@ -0,0 +1,193 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.quicksearch;
import java.util.EnumSet;
import java.util.List;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.function.Consumer;
import com.kingsrook.qqq.backend.core.cluster.ClusterBus;
import com.kingsrook.qqq.backend.core.cluster.ClusterBusInterface;
import com.kingsrook.qqq.backend.core.cluster.ClusterBusMetaData;
import com.kingsrook.qqq.backend.core.cluster.ClusterCacheAudit;
import com.kingsrook.qqq.backend.core.cluster.ClusterEvent;
import com.kingsrook.qqq.backend.core.cluster.QNodeRoles;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
import com.kingsrook.qbits.quicksearch.model.QuickSearchIndex;
import com.kingsrook.qbits.quicksearch.model.QuickSearchIndexField;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** The Quick Search caches are correct on several nodes: every static cache is
** registered with ClusterCaches, and a config change on one node clears the
** config cache on the others.
*******************************************************************************/
class QuickSearchClusterCachesTest extends BaseQuickSearchTest
{
/** the "wire": events every node's transport sees */
static final List<Consumer<ClusterEvent>> WIRE = new CopyOnWriteArrayList<>();
/*******************************************************************************
** A transport over the static WIRE, so a "remote" publish reaches us.
*******************************************************************************/
public static class WireBus implements ClusterBusInterface
{
/*******************************************************************************
**
*******************************************************************************/
@Override
public void publish(ClusterEvent event)
{
WIRE.forEach(listener -> listener.accept(event));
}
/*******************************************************************************
**
*******************************************************************************/
@Override
public AutoCloseable subscribe(Consumer<ClusterEvent> listener)
{
WIRE.add(listener);
return (() -> WIRE.remove(listener));
}
}
/*******************************************************************************
**
*******************************************************************************/
@AfterEach
void afterEach()
{
ClusterBus.close(QContext.getQInstance());
WIRE.clear();
QNodeRoles.set(null);
}
/*******************************************************************************
** Connect the current instance to the wire as node-a.
*******************************************************************************/
private static void connectAsNodeA()
{
QInstance instance = QContext.getQInstance();
instance.withSupplementalMetaData(new ClusterBusMetaData().withBus(new QCodeReference(WireBus.class)));
QNodeRoles.set(new QNodeRoles.Settings(EnumSet.allOf(QNodeRoles.Role.class), true, true, true, null, "node-a"));
ClusterBus.connect(instance);
}
/*******************************************************************************
** A runtime with one code-declared table.
*******************************************************************************/
private static QuickSearchRuntime newRuntime()
{
///////////////////////////////////////////////////////////////////////
// without the field table the config comes from code, so the enabled //
// flag is read from the index row and cached //
///////////////////////////////////////////////////////////////////////
QContext.getQInstance().getTables().remove(QuickSearchIndexField.TABLE_NAME);
QContext.getQInstance().getTable(QuickSearchIndex.TABLE_NAME).getAssociations().clear();
QuickSearchRuntime runtime = new QuickSearchRuntime(new QuickSearchQBitConfig().withBackendName(TEST_BACKEND_NAME).withOpensearchHost("localhost").withOpensearchPort(9200).withOpensearchIndexName("t"));
runtime.setDiscoveredTables(List.of(new QuickSearchableTableConfig().withTableName(TEST_ENTITY_TABLE).withPrimaryKeyField("id").withSearchableFields(List.of("name"))));
return (runtime);
}
/*******************************************************************************
** Every static Memoization in the QBit is registered (shared or node-local).
*******************************************************************************/
@Test
void testNoUnregisteredStaticCaches() throws Exception
{
assertThat(ClusterCacheAudit.unregisteredStaticCaches("com.kingsrook.qbits.quicksearch")).isEmpty();
}
/*******************************************************************************
** Another node's config invalidation clears this node's cached config.
*******************************************************************************/
@Test
void testRemoteInvalidationClearsConfigCache()
{
connectAsNodeA();
QuickSearchRuntime runtime = newRuntime();
try
{
runtime.isTableEnabled(TEST_ENTITY_TABLE);
assertThat(runtime.getCachedEnabledFlagCount()).isEqualTo(1);
ClusterEvent remote = new ClusterEvent(ClusterEvent.TYPE_MEMO_INVALIDATE, QuickSearchRuntime.CONFIG_CACHE_NAME, null);
remote.setOriginNodeId("node-b");
WIRE.forEach(listener -> listener.accept(remote));
assertThat(runtime.getCachedEnabledFlagCount()).isZero();
}
finally
{
runtime.close();
}
}
/*******************************************************************************
** A config change here clears the cache here and tells the other nodes.
*******************************************************************************/
@Test
void testInvalidateConfigPublishesToOtherNodes()
{
connectAsNodeA();
List<ClusterEvent> sent = new CopyOnWriteArrayList<>();
WIRE.add(sent::add);
QuickSearchRuntime runtime = newRuntime();
try
{
runtime.isTableEnabled(TEST_ENTITY_TABLE);
runtime.invalidateConfig();
assertThat(runtime.getCachedEnabledFlagCount()).isZero();
assertThat(sent).anySatisfy(event ->
{
assertThat(event.getType()).isEqualTo(ClusterEvent.TYPE_MEMO_INVALIDATE);
assertThat(event.getName()).isEqualTo(QuickSearchRuntime.CONFIG_CACHE_NAME);
assertThat(event.getOriginNodeId()).isEqualTo("node-a");
});
}
finally
{
runtime.close();
}
}
}
@@ -21,7 +21,9 @@ import java.util.List;
import java.util.Map;
import com.kingsrook.qqq.backend.core.actions.tables.DeleteAction;
import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.audits.AuditsMetaDataProvider;
import com.kingsrook.qqq.backend.core.model.actions.tables.delete.DeleteInput;
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
@@ -77,6 +79,38 @@ class QuickSearchConfigPrimerTest extends BaseDataDrivenTest
/*******************************************************************************
** Re-priming only records the priming outcome on the settings row: that is
** not audited. An admin settings edit still is.
*******************************************************************************/
@Test
void testRepriming_settingsBookkeepingIsNotAudited() throws QException
{
new AuditsMetaDataProvider().defineAll(QContext.getQInstance(), TEST_BACKEND_NAME, null);
new QuickSearchConfigPrimer(runtime).prime();
long afterFirst = settingsEditAudits();
new QuickSearchConfigPrimer(runtime).prime();
assertThat(settingsEditAudits()).as("re-priming is not audited").isEqualTo(afterFirst);
new UpdateAction().execute(new UpdateInput().withTableName(QuickSearchSettings.TABLE_NAME)
.withRecord(new QRecord().withValue("id", QuickSearchSettings.SETTINGS_ID).withValue("indexingPaused", true)));
assertThat(settingsEditAudits()).as("an admin settings edit is audited").isEqualTo(afterFirst + 1);
}
/*******************************************************************************
** Audit rows recording an edit of the settings row.
*******************************************************************************/
private long settingsEditAudits() throws QException
{
return (rows(AuditsMetaDataProvider.TABLE_NAME_AUDIT).stream()
.filter(audit -> audit.getValueString("message") != null && audit.getValueString("message").contains("Edited"))
.count());
}
/*******************************************************************************
** Code defaults update rows nobody edited, and never rows an owner edited.
*******************************************************************************/
@@ -29,6 +29,7 @@ import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.model.audits.AuditsMetaDataProvider;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QCollectingLogger;
import com.kingsrook.qqq.backend.core.logging.QLogger;
@@ -324,6 +325,45 @@ class BasepullIndexStepTest extends BaseQuickSearchTest
/*******************************************************************************
** Basepull bookkeeping on the index row (status, lastBasepullTime,
** documentCount, last run) writes no audit; an admin edit of the same row
** is still audited.
*******************************************************************************/
@Test
void testBasepullBookkeeping_isNotAudited_adminEditIs() throws QException
{
if(QContext.getQInstance().getTable(AuditsMetaDataProvider.TABLE_NAME_AUDIT) == null)
{
new AuditsMetaDataProvider().defineAll(QContext.getQInstance(), TEST_BACKEND_NAME, null);
}
insertTestEntities(3);
QRecord row = insertIndexRow(TEST_ENTITY_TABLE, true, null, 5);
when(mockClient.countDocumentsForTable(TEST_ENTITY_TABLE)).thenReturn(3L);
new BasepullIndexStep().run(new RunBackendStepInput(), new RunBackendStepOutput());
assertThat(queryAll(QuickSearchIndex.TABLE_NAME).get(0).getValueInteger("documentCount")).isEqualTo(3);
assertThat(indexRowEditAudits()).as("basepull bookkeeping is not audited").isZero();
new UpdateAction().execute(new UpdateInput().withTableName(QuickSearchIndex.TABLE_NAME)
.withRecord(new QRecord().withValue("id", row.getValue("id")).withValue("basepullIntervalMinutes", 15)));
assertThat(indexRowEditAudits()).as("an admin configuration edit is audited").isEqualTo(1);
}
/*******************************************************************************
** Audit rows recording an edit of a quick_search_index row.
*******************************************************************************/
private long indexRowEditAudits() throws QException
{
return (queryAll(AuditsMetaDataProvider.TABLE_NAME_AUDIT).stream()
.filter(audit -> audit.getValueString("message") != null && audit.getValueString("message").contains("Edited"))
.count());
}
/*******************************************************************************
** Test: no discovered tables, no quickSearchIndex rows - step completes
** without calling the OpenSearch client.