From d8cab84387a4f92fbba6b9d338b5e31d30f6569d Mon Sep 17 00:00:00 2001 From: James Maes Date: Wed, 14 Oct 2026 16:52:39 -0500 Subject: [PATCH] feat(help): empty-state help for every quick_search table, guardrail and primer tests Adds help/quick-search.yaml (QQQ help bundle) applied by QuickSearchHelp, a tables guardrail test, and a seeded-defaults test proving the primer fills the config tables on a fresh backend. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 1 + README.md | 4 + docs/MIGRATION-1.1.md | 4 + .../quicksearch/QuickSearchQBitProducer.java | 6 + .../quicksearch/metadata/QuickSearchHelp.java | 134 ++++++++ src/main/resources/help/quick-search.yaml | 54 ++++ .../QuickSearchSeededDefaultsTest.java | 207 +++++++++++++ .../QuickSearchTablesGuardrailTest.java | 292 ++++++++++++++++++ 8 files changed, 702 insertions(+) create mode 100644 src/main/java/com/kingsrook/qbits/quicksearch/metadata/QuickSearchHelp.java create mode 100644 src/main/resources/help/quick-search.yaml create mode 100644 src/test/java/com/kingsrook/qbits/quicksearch/QuickSearchSeededDefaultsTest.java create mode 100644 src/test/java/com/kingsrook/qbits/quicksearch/QuickSearchTablesGuardrailTest.java diff --git a/CHANGELOG.md b/CHANGELOG.md index f4b779c..44b2b82 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,7 @@ Requires QQQ 4.1.0-RC.6 or later (the spec-managed UI API extension contract). - Real-time indexing failures now increase the table's error count by the number of failed events. Successful basepulls and full reindexes reset the count. Counter update failures are logged without failing the source write. ### Added +- Every `quick_search_*` table and the Quick Search Admin app carry help that says what the table is for and how its rows get created (`help/quick-search.yaml`, QQQ help-bundle format, applied by `QuickSearchHelp` as a stand-in for core help bundles; host help is never overridden). `QuickSearchTablesGuardrailTest` fails when a table is unused by runtime code, neither primed nor listed as starting empty, or has no such help; `QuickSearchSeededDefaultsTest` proves priming populates the config tables on a fresh backend, is idempotent and keeps admin edits. - Session-only typeahead: `GET /qqq/v1/qbit/quick-search/suggest` (`getQuickSearchSuggestions`) and `QuickSearchSuggestService.suggest(...)`. Reuses the permission-aware search action with one bounded candidate batch per table; groups expose only record IDs, filtered labels/subtitles, `hasMore` and `truncated`. Defaults to 5 items across 8 tables (maximum 10/20); short terms and unavailable search return empty groups. The existing UI extension and generated contract now carry both status and suggestions. - Supported host status API (`api.QuickSearchStatusService.get()`), immutable status values, and `QuickSearchApi` home navigation constants (issue #50/F10). Permission-checked, bounded diagnostics refreshes share a short per-runtime cache and reuse saved validation counts. New spec-managed `GET /qqq/v1/qbit/quick-search/status` (`getQuickSearchStatus`) with committed OpenAPI fragment and strict HTTP/contract tests. RC.6 requires the default `/qbit/quick-search` prefix; `/quick-search` is not permitted by core. Existing routes and wire shapes are unchanged. - Record developer panel: `QuickSearchDeveloperPanelProvider` (id `quickSearch`) adds a "Quick Search" panel to the developer view of every record of an indexed table (`/app/{table}/{id}/dev` in the Next UI), for users who may read Quick Search Index. It shows the table's index row (enabled, status, record count, last basepull and full reindex, searchable fields), the latest index run, the record's indexed document compared field by field with the database (Inspect Search Document's comparison; needs `debugPermissionRules`), the record's failed index events, the Inspect Search Document and Reindex in Search actions the user may run, and links to the admin app and the index row. An unreachable OpenSearch shows a warning in the panel. Registered by `QuickSearchDeveloperPanelsMetaDataProducer`. diff --git a/README.md b/README.md index b5eafc5..8f1cdb1 100644 --- a/README.md +++ b/README.md @@ -257,6 +257,10 @@ The panel's status is Indexed, Stale, Not indexed, Disabled (the table is turned - `quick_search_validation_run` (with `enableDiagnosticTools`): `status` (`PASS`, `WARN`, `FAIL`), `triggerType` (`MANUAL`, `SCHEDULED`), `userName`, `tableNames`, `sampleSize`, `tablesChecked`, `failCount`, `warnCount`, `summary`, `reportHtml` (HTML), `reportJson` (text), `startTime`, `endTime`, `createDate`. Purged after `runHistoryRetentionDays`. - `quick_search_failed_event`: real-time events that could not be applied; `status` `PENDING` or `EXHAUSTED` after 10 attempts. Deletes made while a full reindex of all tables runs are also recorded here with status `AWAITING_REINDEX`: the reindex applies them to the new index after the alias swap (basepull does not replay them). While a table is `REBUILDING`, drift detection is suspended for it. Only one full reindex runs at a time: starting one while another is running fails. A full reindex shows it is running through its `FULL_REINDEX` run records, which stay `RUNNING` with `modifyDate` refreshed after every page. If it is killed mid-run (pod restart, OOM, deploy), recovery is automatic once its run records have shown no sign of life for `fullReindexStaleMinutes` (30 by default): the next basepull, or the next full reindex, marks those run records `FAILED`, returns the tables to `ACTIVE` with `lastBasepullTime` moved back to the run's start, and turns the `AWAITING_REINDEX` rows `PENDING` so basepull replays the deletes. No data is lost. A run that was only slow and is judged stopped notices before its alias swap and fails without swapping, or, after its swap, stops applying its captured deletes; basepull skips a replayed delete whose record exists again. +### Starter defaults and empty-state help + +This qbit ships no DDL and no seed changelog: startup self-priming (`QuickSearchConfigPrimer`) is its seed mechanism. On a fresh database it writes a `quick_search_index` row and `quick_search_index_field` rows for every table the code declares searchable, and the single `quick_search_settings` row (id 1). `quick_search_config_tombstone` and the five log and history tables start empty by design (rows appear when an admin removes a table or field, or when the qbit runs). Every table and the Quick Search Admin app carry help (`help/quick-search.yaml`, applied by `QuickSearchHelp` without overriding help the host set) that says what the table is for and how its rows get created, so an empty table explains itself. No host change is needed. + Real-time error counts on Quick Search Index show failures since the last successful full reindex or basepull. Each failed event increases the count; deferred changes while indexing is paused do not. Counter updates are best effort and serialized across listeners in one JVM; concurrent application nodes can still race because QQQ has no arithmetic update API. ## Extending diff --git a/docs/MIGRATION-1.1.md b/docs/MIGRATION-1.1.md index 93c6614..143547f 100644 --- a/docs/MIGRATION-1.1.md +++ b/docs/MIGRATION-1.1.md @@ -228,6 +228,10 @@ MySQL: use `INT AUTO_INCREMENT` for the ids, `DATETIME` for the timestamps and ` - `quick_search_searchable_table` is now a custom possible-value source over the effective tables (it was an enum of the code-declared ones). - Tables added at runtime get core `searchFields` (for the global search) when they have none, but the per-table Inspect Search Document and Reindex in Search record actions appear only after the next restart. +### Table help (no action needed) + +The qbit now sets bundle help (`help/quick-search.yaml`) on the Quick Search Admin app and each `quick_search_*` table, so an empty table says what it is for and how its rows get created. It fills only empty `help` slots; help a host already set on these tables is kept. There is no DDL change and no host include. + ### Checklist for data-driven configuration 1. Add the DDL above (it is additive). diff --git a/src/main/java/com/kingsrook/qbits/quicksearch/QuickSearchQBitProducer.java b/src/main/java/com/kingsrook/qbits/quicksearch/QuickSearchQBitProducer.java index 9152f7d..939a6bd 100644 --- a/src/main/java/com/kingsrook/qbits/quicksearch/QuickSearchQBitProducer.java +++ b/src/main/java/com/kingsrook/qbits/quicksearch/QuickSearchQBitProducer.java @@ -33,6 +33,7 @@ import com.kingsrook.qbits.quicksearch.config.QuickSearchFieldRules; import com.kingsrook.qbits.quicksearch.listeners.QuickSearchRecordChangeListener; import com.kingsrook.qbits.quicksearch.metadata.QuickSearchAdminAppMetaDataProducer; import com.kingsrook.qbits.quicksearch.metadata.QuickSearchDiagnosticsProcessMetaDataHelper; +import com.kingsrook.qbits.quicksearch.metadata.QuickSearchHelp; import com.kingsrook.qbits.quicksearch.metadata.QuickSearchProcessMetaDataHelper; import com.kingsrook.qqq.backend.core.exceptions.QException; import com.kingsrook.qqq.backend.core.logging.QLogger; @@ -218,6 +219,11 @@ public class QuickSearchQBitProducer implements QBitMetaDataProducer entries() throws QException + { + try(InputStream stream = QuickSearchHelp.class.getResourceAsStream(RESOURCE)) + { + if(stream == null) + { + throw (new IOException("missing resource")); + } + JsonNode bundle = new ObjectMapper(new YAMLFactory()).readTree(stream); + if(!"quick-search".equals(bundle.path("bundle").asText()) || !"QBIT".equals(bundle.path("layer").asText())) + { + throw (new IOException("invalid bundle header")); + } + Map entries = new LinkedHashMap<>(); + for(JsonNode entry : bundle.path("entries")) + { + String key = entry.path("key").asText(); + String summary = entry.path("summary").asText(); + if(key.isBlank() || summary.isBlank() || summary.length() > 120 || entry.path("content").asText().isBlank() || entries.put(key, entry) != null) + { + throw (new IOException("invalid or duplicate entry: " + key)); + } + } + return (entries); + } + catch(IOException | RuntimeException e) + { + throw (new QException("Could not load " + RESOURCE, e)); + } + } + + + + /******************************************************************************* + ** Apply the help to the app and every quick_search_* table in the instance + ** (names carry the config's prefix, if any). A host's own help on a table + ** is kept (the bundle only fills the slot). + *******************************************************************************/ + public static void apply(QInstance qInstance, QuickSearchQBitConfig config) throws QException + { + for(Map.Entry entry : entries().entrySet()) + { + String key = entry.getKey(); + String name = key.substring(key.indexOf(':') + 1, key.indexOf(';')); + if(key.startsWith("table:")) + { + QTableMetaData table = qInstance.getTable(config.applyTablePrefix(name)); + if(table != null && (table.getHelpContent() == null || !table.getHelpContent().containsKey(SLOT))) + { + table.withHelpContent(SLOT, content(entry.getValue())); + } + } + else if(key.startsWith("app:")) + { + QAppMetaData app = qInstance.getApp(config.applyPrefix(name)); + if(app != null && (app.getHelpContent() == null || !app.getHelpContent().containsKey(SLOT))) + { + app.withHelpContent(SLOT, content(entry.getValue())); + } + } + } + } + + + + /******************************************************************************* + ** Summary as the first paragraph, then the content, as markdown. + *******************************************************************************/ + static QHelpContent content(JsonNode entry) + { + return (new QHelpContent().withContentAsMarkdown(entry.path("summary").asText().strip() + "\n\n" + entry.path("content").asText().strip())); + } + +} diff --git a/src/main/resources/help/quick-search.yaml b/src/main/resources/help/quick-search.yaml new file mode 100644 index 0000000..c28f8da --- /dev/null +++ b/src/main/resources/help/quick-search.yaml @@ -0,0 +1,54 @@ +bundle: quick-search +version: 1 +layer: QBIT +# Help for the Quick Search Admin app and every quick_search_* table, in the QQQ +# help-bundle format (qqq#1198). Each table entry says what the table is for and +# how its rows get created, so an empty table explains itself. QuickSearchHelp +# applies it until the core HelpBundles registry ships in a QQQ release +# (a stand-in for core#1198). Summaries stay under 120 characters +# (QuickSearchTablesGuardrailTest). +entries: + - key: app:quickSearchAdmin;slot:help + summary: "Watch and tune full-text search: which tables are indexed, how they build, and whether search is healthy." + content: | + Nothing needs to be set up by hand. On startup the qbit reads the tables your code marks searchable and writes the index and field rows itself (see Search Indexes). Add a table with Add Search Index, change weights on Searchable Fields, and use Pause Indexing as the emergency stop. The home shows whether OpenSearch is reachable and what the last startup priming changed. + + - key: table:quick_search_index;slot:help + summary: "One row per table that is searchable: whether it is indexed, how often it refreshes, and how its last build went." + content: | + **How rows get created:** automatically. On startup the qbit adds a row for every table declared searchable in code (@QuickSearchable or the host's searchable tables) and queues its first build. Add a table yourself with the Add Search Index action; Disable or Remove Search Index stops indexing it. A table you removed is remembered (Removed Config) and not added back. Change the interval or the fields, then run Apply Search Index Changes to rebuild. + + - key: table:quick_search_index_field;slot:help + summary: "The fields of each indexed table that search looks at, with their weight and analyzer." + content: | + **How rows get created:** automatically, with the index row, from the field annotations or the host's field list. Edit a weight to make matches in that field rank higher, or add a field by hand; your edits are kept and never overwritten by startup. Changes take effect after Apply Search Index Changes rebuilds the table. + + - key: table:quick_search_settings;slot:help + summary: "The single settings row: the indexing pause switch and what startup priming last did." + content: | + **How rows get created:** the qbit creates the one row (id 1) on first startup; do not add others. Use the Pause Indexing and Resume Indexing actions to stop and restart all indexing while search keeps answering from the existing index. The priming fields are written by the qbit and are read-only. + + - key: table:quick_search_config_tombstone;slot:help + summary: "Tables and fields you removed from search, so startup does not add them back." + content: | + **How rows get created:** this table starts empty on purpose. A row appears when you Remove Search Index or remove a field that code still declares as searchable. Delete a row to let the code defaults add that table or field again on the next startup. + + - key: table:quick_search_index_run;slot:help + summary: "History of every index build, refresh and reconcile: when it ran, how many records, and how it ended." + content: | + **How rows get created:** by the qbit itself. Each Full Reindex, Basepull (the scheduled refresh) and Reconcile writes a row, and the first build of a new index writes the first one. Nothing to set up. Cancel Index Run stops one that is still running. Old rows are purged after the run history retention period. + + - key: table:quick_search_failed_event;slot:help + summary: "Real-time index updates that failed and are waiting to be replayed." + content: | + **How rows get created:** by the qbit. When saving a record cannot update OpenSearch, the change is recorded here and the next refresh replays it. This table is normally empty, which is the healthy state. Rows that stay here mean OpenSearch is unreachable or rejecting a document; check the home status. + + - key: table:quick_search_diagnostic_event;slot:help + summary: "Failed searches and a trail of diagnostic tool runs, kept for troubleshooting." + content: | + **How rows get created:** by the qbit. A search that hits an OpenSearch error or timeout writes a row (core search then falls back to its own), and so does each run of a diagnostic tool such as Search Playground. Rows are read-only. An empty table means no search has failed. + + - key: table:quick_search_validation_run;slot:help + summary: "Results of each Validate Search Index check: pass, warnings and the full report." + content: | + **How rows get created:** run the Validate Search Index action from the Search Indexes table (it can also run on a schedule); each run writes one row with its status and report. It starts empty until someone validates. Rows are kept as history and purged after the run history retention period. diff --git a/src/test/java/com/kingsrook/qbits/quicksearch/QuickSearchSeededDefaultsTest.java b/src/test/java/com/kingsrook/qbits/quicksearch/QuickSearchSeededDefaultsTest.java new file mode 100644 index 0000000..fa7bbdf --- /dev/null +++ b/src/test/java/com/kingsrook/qbits/quicksearch/QuickSearchSeededDefaultsTest.java @@ -0,0 +1,207 @@ +/* + * Copyright 2026 Kingsrook, LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.quicksearch; + + +import java.util.List; +import com.kingsrook.qbits.quicksearch.annotations.QuickSearchField; +import com.kingsrook.qbits.quicksearch.annotations.QuickSearchable; +import com.kingsrook.qbits.quicksearch.config.QuickSearchConfigPrimer; +import com.kingsrook.qbits.quicksearch.model.QuickSearchConfigTombstone; +import com.kingsrook.qbits.quicksearch.model.QuickSearchIndex; +import com.kingsrook.qbits.quicksearch.model.QuickSearchIndexField; +import com.kingsrook.qbits.quicksearch.model.QuickSearchSettings; +import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType; +import com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.QInstance; +import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType; +import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData; +import com.kingsrook.qqq.backend.core.model.session.QSession; +import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryBackendModule; +import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryRecordStore; +import org.junit.jupiter.api.AfterEach; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import static org.assertj.core.api.Assertions.assertThat; + + +/******************************************************************************* + ** This qbit ships no DDL and no changelog: QuickSearchConfigPrimer seeds its + ** config tables at runtime. This proves, on a fresh (memory) backend produced + ** by the real producer, that priming populates quick_search_index, + ** quick_search_index_field and the quick_search_settings row (id 1), leaves + ** quick_search_config_tombstone empty, is idempotent, and never overwrites an + ** admin's edit. + *******************************************************************************/ +class QuickSearchSeededDefaultsTest +{ + /******************************************************************************* + ** A searchable entity. + *******************************************************************************/ + @QuickSearchable(tableName = "person", basepullIntervalMinutes = 10) + static class PersonEntity + { + @QuickSearchField(weight = 3) + private String name; + + @QuickSearchField + private String email; + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + @BeforeEach + void setUp() + { + MemoryRecordStore.getInstance().reset(); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + @AfterEach + void tearDown() + { + QContext.clear(); + } + + + + /******************************************************************************* + ** A fresh install: priming fills the config tables, twice changes nothing, + ** and an edited row survives. + *******************************************************************************/ + @Test + void primerPopulatesConfigTablesOnFreshBackend_idempotent_andKeepsEdits() throws QException + { + QuickSearchQBitConfig config = produce(List.of(PersonEntity.class)); + QuickSearchRuntime runtime = config.getRuntime(); + + assertThat(rows(QuickSearchIndex.TABLE_NAME)).isEmpty(); + assertThat(rows(QuickSearchSettings.TABLE_NAME)).isEmpty(); + + QuickSearchConfigPrimer.PrimingReport first = new QuickSearchConfigPrimer(runtime).prime(); + + assertThat(first.newTables()).hasSize(1); + assertThat(rows(QuickSearchIndex.TABLE_NAME)).singleElement().satisfies(row -> + { + assertThat(row.getValueString("tableName")).isEqualTo("person"); + assertThat(row.getValueString("source")).isEqualTo(QuickSearchIndex.SOURCE_CODE); + assertThat(row.getValueInteger("basepullIntervalMinutes")).isEqualTo(10); + }); + assertThat(rows(QuickSearchIndexField.TABLE_NAME)).extracting(r -> r.getValueString("fieldName")).containsExactlyInAnyOrder("name", "email"); + assertThat(rows(QuickSearchSettings.TABLE_NAME)).singleElement().satisfies(row -> + { + assertThat(row.getValueInteger("id")).isEqualTo(QuickSearchSettings.SETTINGS_ID); + assertThat(row.getValueBoolean("indexingPaused")).isFalse(); + assertThat(row.getValueString("lastPrimingSummary")).isNotBlank(); + }); + assertThat(rows(QuickSearchConfigTombstone.TABLE_NAME)).as("tombstones legitimately start empty").isEmpty(); + + ///////////////////////////////////////////// + // an admin edits the interval (user write) // + ///////////////////////////////////////////// + QRecord indexRow = rows(QuickSearchIndex.TABLE_NAME).get(0); + new UpdateAction().execute(new UpdateInput(QuickSearchIndex.TABLE_NAME).withRecord(new QRecord().withValue("id", indexRow.getValue("id")).withValue("basepullIntervalMinutes", 45))); + + QuickSearchConfigPrimer.PrimingReport second = new QuickSearchConfigPrimer(runtime).prime(); + + assertThat(second.newTables()).isEmpty(); + assertThat(second.newFields()).isZero(); + assertThat(rows(QuickSearchIndex.TABLE_NAME)).hasSize(1); + assertThat(rows(QuickSearchIndexField.TABLE_NAME)).hasSize(2); + assertThat(rows(QuickSearchSettings.TABLE_NAME)).hasSize(1); + assertThat(rows(QuickSearchIndex.TABLE_NAME).get(0).getValueInteger("basepullIntervalMinutes")).as("the admin's edit survives priming").isEqualTo(45); + } + + + + /******************************************************************************* + ** Even with nothing searchable declared in code, priming creates the settings + ** row (id 1), so the pause switch and priming summary always exist. + *******************************************************************************/ + @Test + void primerCreatesSettingsRowEvenWithNoCodeTables() throws QException + { + QuickSearchQBitConfig config = produce(List.of(PersonEntity.class)); + config.getRuntime().setDiscoveredTables(List.of()); + + new QuickSearchConfigPrimer(config.getRuntime()).prime(); + + assertThat(rows(QuickSearchIndex.TABLE_NAME)).isEmpty(); + assertThat(rows(QuickSearchSettings.TABLE_NAME)).singleElement().satisfies(row -> + { + assertThat(row.getValueInteger("id")).isEqualTo(QuickSearchSettings.SETTINGS_ID); + assertThat(row.getValueBoolean("indexingPaused")).isFalse(); + assertThat(row.getValueString("lastPrimingSummary")).isNotBlank(); + }); + + new QuickSearchConfigPrimer(config.getRuntime()).prime(); + assertThat(rows(QuickSearchSettings.TABLE_NAME)).hasSize(1); + } + + + + /******************************************************************************* + ** Produce the qbit on a fresh memory-backed instance. + *******************************************************************************/ + private QuickSearchQBitConfig produce(List> entityClasses) throws QException + { + QInstance qInstance = new QInstance(); + qInstance.addBackend(new QBackendMetaData().withName("memory").withBackendType(MemoryBackendModule.class)); + qInstance.withInstanceDefaultAuthentication(new QAuthenticationMetaData().withName("anonymous").withType(QAuthenticationType.FULLY_ANONYMOUS)); + qInstance.addTable(new QTableMetaData().withName("person").withBackendName("memory").withPrimaryKeyField("id") + .withField(new QFieldMetaData("id", QFieldType.INTEGER).withIsEditable(false)) + .withField(new QFieldMetaData("name", QFieldType.STRING)) + .withField(new QFieldMetaData("email", QFieldType.STRING)) + .withField(new QFieldMetaData("modifyDate", QFieldType.DATE_TIME))); + QContext.init(qInstance, new QSession()); + + QuickSearchQBitConfig config = new QuickSearchQBitConfig() + .withStartupMode(QuickSearchStartupMode.DEGRADED) + .withBackendName("memory") + .withOpensearchHost("localhost") + .withOpensearchPort(9200) + .withOpensearchIndexName("seeded-defaults") + .withSearchableEntityClasses(entityClasses); + new QuickSearchQBitProducer().withConfig(config).produce(qInstance); + return (config); + } + + + + /******************************************************************************* + ** Every row of a table. + *******************************************************************************/ + private List rows(String tableName) throws QException + { + return (QuickSearchConfigPrimer.queryAll(tableName)); + } + +} diff --git a/src/test/java/com/kingsrook/qbits/quicksearch/QuickSearchTablesGuardrailTest.java b/src/test/java/com/kingsrook/qbits/quicksearch/QuickSearchTablesGuardrailTest.java new file mode 100644 index 0000000..aee767b --- /dev/null +++ b/src/test/java/com/kingsrook/qbits/quicksearch/QuickSearchTablesGuardrailTest.java @@ -0,0 +1,292 @@ +/* + * Copyright 2026 Kingsrook, LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.quicksearch; + + +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.regex.Matcher; +import java.util.regex.Pattern; +import java.util.stream.Stream; +import com.kingsrook.qbits.quicksearch.annotations.QuickSearchField; +import com.kingsrook.qbits.quicksearch.annotations.QuickSearchable; +import com.kingsrook.qbits.quicksearch.metadata.QuickSearchHelp; +import com.kingsrook.qqq.backend.core.context.QContext; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType; +import com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.QInstance; +import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData; +import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType; +import com.kingsrook.qqq.backend.core.model.metadata.help.QHelpContent; +import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData; +import com.kingsrook.qqq.backend.core.model.session.QSession; +import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryBackendModule; +import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryRecordStore; +import org.junit.jupiter.api.AfterEach; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import static org.assertj.core.api.Assertions.assertThat; + + +/******************************************************************************* + ** Guardrail (owner requirement 2026-10-14: "all tables should be populated and + ** the qbit should be fully using them"). For every quick_search_* table: + ** - runtime code (not just its model or metadata) reads or writes it, so no + ** table exists only on paper; + ** - it is either populated by the startup primer (QuickSearchConfigPrimer; this + ** qbit ships no DDL or changelog, so the primer is its seed mechanism) or + ** listed in STARTS_EMPTY with why it starts empty; + ** - it has bundle help that says what it is for and how its rows get created. + ** A new table fails this test until it is wired, seeded or explained. + *******************************************************************************/ +class QuickSearchTablesGuardrailTest +{ + private static final Path MAIN = Path.of("src", "main", "java", "com", "kingsrook", "qbits", "quicksearch"); + + /******************************************************************************* + ** Tables the primer fills from code on startup (QuickSearchSeededDefaultsTest + ** proves it). + *******************************************************************************/ + static final Map PRIMED = Map.of( + "quick_search_index", "one row per table code declares searchable", + "quick_search_index_field", "one row per searchable field code declares", + "quick_search_settings", "the single settings row (id 1), with the priming summary"); + + /******************************************************************************* + ** Tables that legitimately start empty, and where their rows come from. + *******************************************************************************/ + static final Map STARTS_EMPTY = Map.of( + "quick_search_config_tombstone", "an admin removes a table or field that code still declares", + "quick_search_index_run", "runtime: every build, basepull and reconcile", + "quick_search_failed_event", "runtime: a real-time index update fails (empty is healthy)", + "quick_search_diagnostic_event", "runtime: a search fails or a diagnostic tool runs", + "quick_search_validation_run", "runtime: someone runs Validate Search Index"); + + + + /******************************************************************************* + ** A searchable entity for the produced instance. + *******************************************************************************/ + @QuickSearchable(tableName = "person") + static class PersonEntity + { + @QuickSearchField(weight = 3) + private String name; + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + @BeforeEach + void setUp() + { + MemoryRecordStore.getInstance().reset(); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + @AfterEach + void tearDown() + { + QContext.clear(); + } + + + + /******************************************************************************* + ** Every table is used by runtime code. + *******************************************************************************/ + @Test + void everyTableIsUsedAtRuntime() throws Exception + { + Map entityByTable = entityByTable(); + List runtimeSource; + try(Stream files = Files.walk(MAIN)) + { + runtimeSource = files.filter(p -> p.toString().endsWith(".java")) + .filter(p -> !p.startsWith(MAIN.resolve("model")) && !p.startsWith(MAIN.resolve("metadata"))) + .map(QuickSearchTablesGuardrailTest::read).toList(); + } + for(String table : quickSearchTables()) + { + String entity = entityByTable.get(table); + assertThat(entity).as("entity class for " + table).isNotNull(); + + ////////////////////////////////////////////////////////////////////////// + // the entity, or the config's name accessor for its table (index and // + // index run are reached by their prefixed names) // + ////////////////////////////////////////////////////////////////////////// + Pattern use = Pattern.compile("\\b" + entity + "(\\.TABLE_NAME|\\.class|\\(|::)|\\bget" + entity + "TableName\\("); + assertThat(runtimeSource.stream().anyMatch(source -> use.matcher(source).find())) + .as(table + " (" + entity + ") must be read or written by runtime code, not only declared").isTrue(); + } + } + + + + /******************************************************************************* + ** Every table is primed or explained. + *******************************************************************************/ + @Test + void everyTableIsPrimedOrExplained() + { + QInstance qInstance = producedInstance(); + for(String table : quickSearchTables()) + { + assertThat(PRIMED.containsKey(table) || STARTS_EMPTY.containsKey(table)).as(table + " must be primed or listed in STARTS_EMPTY").isTrue(); + assertThat(PRIMED.containsKey(table) && STARTS_EMPTY.containsKey(table)).as(table + " cannot be both primed and listed as starting empty").isFalse(); + } + assertThat(PRIMED.keySet()).allMatch(table -> qInstance.getTable(table) != null, "PRIMED names only real tables"); + assertThat(STARTS_EMPTY.keySet()).allMatch(table -> qInstance.getTable(table) != null, "STARTS_EMPTY names only real tables"); + } + + + + /******************************************************************************* + ** Every table and the app have bundle help that says how rows get created. + *******************************************************************************/ + @Test + void everyTableHasEmptyStateHelp() throws Exception + { + QInstance qInstance = producedInstance(); + assertThat(QuickSearchHelp.entries()).allSatisfy((key, entry) -> assertThat(entry.path("summary").asText().length()).as(key).isLessThanOrEqualTo(120)); + for(String table : quickSearchTables()) + { + QTableMetaData meta = qInstance.getTable(table); + List help = meta.getHelpContent() == null ? null : meta.getHelpContent().get(QuickSearchHelp.SLOT); + assertThat(help).as(table + " table help").isNotEmpty(); + assertThat(help.get(0).getContent()).as(table + " help says how rows get created").contains("How rows get created"); + } + assertThat(qInstance.getApp("quickSearchAdmin").getHelpContent()).containsKey(QuickSearchHelp.SLOT); + assertThat(QuickSearchHelp.entries().keySet()).allMatch(key -> !key.startsWith("table:") || qInstance.getTable(key.substring(6, key.indexOf(';'))) != null, + "help names only real tables"); + } + + + + /******************************************************************************* + ** Help a host already wrote is kept; the bundle only fills empty slots. + *******************************************************************************/ + @Test + void hostHelpIsNotOverridden() throws Exception + { + QInstance qInstance = producedInstance(); + QTableMetaData table = qInstance.getTable("quick_search_index"); + table.getHelpContent().remove(QuickSearchHelp.SLOT); + table.withHelpContent(QuickSearchHelp.SLOT, new QHelpContent("host words")); + + QuickSearchHelp.apply(qInstance, new QuickSearchQBitConfig()); + + assertThat(table.getHelpContent().get(QuickSearchHelp.SLOT).get(0).getContent()).isEqualTo("host words"); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + private List quickSearchTables() + { + List tables = producedInstance().getTables().values().stream().map(QTableMetaData::getName).filter(n -> n.startsWith("quick_search_")).sorted().toList(); + assertThat(tables).hasSize(8); + return (tables); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + private QInstance producedInstance() + { + try + { + QInstance qInstance = new QInstance(); + qInstance.addBackend(new QBackendMetaData().withName("memory").withBackendType(MemoryBackendModule.class)); + qInstance.withInstanceDefaultAuthentication(new QAuthenticationMetaData().withName("anonymous").withType(QAuthenticationType.FULLY_ANONYMOUS)); + qInstance.addTable(new QTableMetaData().withName("person").withBackendName("memory").withPrimaryKeyField("id") + .withField(new QFieldMetaData("id", QFieldType.INTEGER).withIsEditable(false)) + .withField(new QFieldMetaData("name", QFieldType.STRING)) + .withField(new QFieldMetaData("modifyDate", QFieldType.DATE_TIME))); + QContext.init(qInstance, new QSession()); + new QuickSearchQBitProducer().withConfig(new QuickSearchQBitConfig() + .withStartupMode(QuickSearchStartupMode.DEGRADED) + .withBackendName("memory") + .withOpensearchHost("localhost") + .withOpensearchPort(9200) + .withOpensearchIndexName("guardrail") + .withSearchableEntityClasses(List.of(PersonEntity.class))).produce(qInstance); + return (qInstance); + } + catch(QException e) + { + throw (new IllegalStateException(e)); + } + } + + + + /******************************************************************************* + ** model/X.java declares TABLE_NAME = "x". + *******************************************************************************/ + private static Map entityByTable() throws Exception + { + Map rs = new HashMap<>(); + Pattern p = Pattern.compile("TABLE_NAME\\s*=\\s*\"([a-z_]+)\""); + try(Stream files = Files.list(MAIN.resolve("model"))) + { + for(Path file : files.toList()) + { + Matcher m = p.matcher(read(file)); + if(m.find()) + { + rs.put(m.group(1), file.getFileName().toString().replace(".java", "")); + } + } + } + return (rs); + } + + + + /******************************************************************************* + ** + *******************************************************************************/ + private static String read(Path path) + { + try + { + return (Files.readString(path, StandardCharsets.UTF_8)); + } + catch(Exception e) + { + throw (new IllegalStateException(e)); + } + } + +} -- 2.54.0