Every quick_search table explains itself; guardrail and primer tests #58

Merged
KofTwentyTwo merged 1 commits from feature/tables-populated-defaults into develop 2026-10-14 22:19:52 +00:00
8 changed files with 702 additions and 0 deletions

No files matched your search

+1
View File
@@ -10,6 +10,7 @@ Requires QQQ 4.1.0-RC.6 or later (the spec-managed UI API extension contract).
- Real-time indexing failures now increase the table's error count by the number of failed events. Successful basepulls and full reindexes reset the count. Counter update failures are logged without failing the source write.
### Added
- Every `quick_search_*` table and the Quick Search Admin app carry help that says what the table is for and how its rows get created (`help/quick-search.yaml`, QQQ help-bundle format, applied by `QuickSearchHelp` as a stand-in for core help bundles; host help is never overridden). `QuickSearchTablesGuardrailTest` fails when a table is unused by runtime code, neither primed nor listed as starting empty, or has no such help; `QuickSearchSeededDefaultsTest` proves priming populates the config tables on a fresh backend, is idempotent and keeps admin edits.
- Session-only typeahead: `GET /qqq/v1/qbit/quick-search/suggest` (`getQuickSearchSuggestions`) and `QuickSearchSuggestService.suggest(...)`. Reuses the permission-aware search action with one bounded candidate batch per table; groups expose only record IDs, filtered labels/subtitles, `hasMore` and `truncated`. Defaults to 5 items across 8 tables (maximum 10/20); short terms and unavailable search return empty groups. The existing UI extension and generated contract now carry both status and suggestions.
- Supported host status API (`api.QuickSearchStatusService.get()`), immutable status values, and `QuickSearchApi` home navigation constants (issue #50/F10). Permission-checked, bounded diagnostics refreshes share a short per-runtime cache and reuse saved validation counts. New spec-managed `GET /qqq/v1/qbit/quick-search/status` (`getQuickSearchStatus`) with committed OpenAPI fragment and strict HTTP/contract tests. RC.6 requires the default `/qbit/quick-search` prefix; `/quick-search` is not permitted by core. Existing routes and wire shapes are unchanged.
- Record developer panel: `QuickSearchDeveloperPanelProvider` (id `quickSearch`) adds a "Quick Search" panel to the developer view of every record of an indexed table (`/app/{table}/{id}/dev` in the Next UI), for users who may read Quick Search Index. It shows the table's index row (enabled, status, record count, last basepull and full reindex, searchable fields), the latest index run, the record's indexed document compared field by field with the database (Inspect Search Document's comparison; needs `debugPermissionRules`), the record's failed index events, the Inspect Search Document and Reindex in Search actions the user may run, and links to the admin app and the index row. An unreachable OpenSearch shows a warning in the panel. Registered by `QuickSearchDeveloperPanelsMetaDataProducer`.
+4
View File
@@ -257,6 +257,10 @@ The panel's status is Indexed, Stale, Not indexed, Disabled (the table is turned
- `quick_search_validation_run` (with `enableDiagnosticTools`): `status` (`PASS`, `WARN`, `FAIL`), `triggerType` (`MANUAL`, `SCHEDULED`), `userName`, `tableNames`, `sampleSize`, `tablesChecked`, `failCount`, `warnCount`, `summary`, `reportHtml` (HTML), `reportJson` (text), `startTime`, `endTime`, `createDate`. Purged after `runHistoryRetentionDays`.
- `quick_search_failed_event`: real-time events that could not be applied; `status` `PENDING` or `EXHAUSTED` after 10 attempts. Deletes made while a full reindex of all tables runs are also recorded here with status `AWAITING_REINDEX`: the reindex applies them to the new index after the alias swap (basepull does not replay them). While a table is `REBUILDING`, drift detection is suspended for it. Only one full reindex runs at a time: starting one while another is running fails. A full reindex shows it is running through its `FULL_REINDEX` run records, which stay `RUNNING` with `modifyDate` refreshed after every page. If it is killed mid-run (pod restart, OOM, deploy), recovery is automatic once its run records have shown no sign of life for `fullReindexStaleMinutes` (30 by default): the next basepull, or the next full reindex, marks those run records `FAILED`, returns the tables to `ACTIVE` with `lastBasepullTime` moved back to the run's start, and turns the `AWAITING_REINDEX` rows `PENDING` so basepull replays the deletes. No data is lost. A run that was only slow and is judged stopped notices before its alias swap and fails without swapping, or, after its swap, stops applying its captured deletes; basepull skips a replayed delete whose record exists again.
### Starter defaults and empty-state help
This qbit ships no DDL and no seed changelog: startup self-priming (`QuickSearchConfigPrimer`) is its seed mechanism. On a fresh database it writes a `quick_search_index` row and `quick_search_index_field` rows for every table the code declares searchable, and the single `quick_search_settings` row (id 1). `quick_search_config_tombstone` and the five log and history tables start empty by design (rows appear when an admin removes a table or field, or when the qbit runs). Every table and the Quick Search Admin app carry help (`help/quick-search.yaml`, applied by `QuickSearchHelp` without overriding help the host set) that says what the table is for and how its rows get created, so an empty table explains itself. No host change is needed.
Real-time error counts on Quick Search Index show failures since the last successful full reindex or basepull. Each failed event increases the count; deferred changes while indexing is paused do not. Counter updates are best effort and serialized across listeners in one JVM; concurrent application nodes can still race because QQQ has no arithmetic update API.
## Extending
+4
View File
@@ -228,6 +228,10 @@ MySQL: use `INT AUTO_INCREMENT` for the ids, `DATETIME` for the timestamps and `
- `quick_search_searchable_table` is now a custom possible-value source over the effective tables (it was an enum of the code-declared ones).
- Tables added at runtime get core `searchFields` (for the global search) when they have none, but the per-table Inspect Search Document and Reindex in Search record actions appear only after the next restart.
### Table help (no action needed)
The qbit now sets bundle help (`help/quick-search.yaml`) on the Quick Search Admin app and each `quick_search_*` table, so an empty table says what it is for and how its rows get created. It fills only empty `help` slots; help a host already set on these tables is kept. There is no DDL change and no host include.
### Checklist for data-driven configuration
1. Add the DDL above (it is additive).
@@ -33,6 +33,7 @@ import com.kingsrook.qbits.quicksearch.config.QuickSearchFieldRules;
import com.kingsrook.qbits.quicksearch.listeners.QuickSearchRecordChangeListener;
import com.kingsrook.qbits.quicksearch.metadata.QuickSearchAdminAppMetaDataProducer;
import com.kingsrook.qbits.quicksearch.metadata.QuickSearchDiagnosticsProcessMetaDataHelper;
import com.kingsrook.qbits.quicksearch.metadata.QuickSearchHelp;
import com.kingsrook.qbits.quicksearch.metadata.QuickSearchProcessMetaDataHelper;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.logging.QLogger;
@@ -218,6 +219,11 @@ public class QuickSearchQBitProducer implements QBitMetaDataProducer<QuickSearch
MetaDataProducerMultiOutput output = QBitMetaDataProducer.super.produce(qInstance);
output.addSelfToInstance(qInstance);
//////////////////////////////////////////////////////////////////////////
// after the tables and app are in the instance: fill their empty help //
//////////////////////////////////////////////////////////////////////////
QuickSearchHelp.apply(qInstance, qBitConfig);
return (new MetaDataProducerMultiOutput());
}
@@ -0,0 +1,134 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.quicksearch.metadata;
import java.io.IOException;
import java.io.InputStream;
import java.util.LinkedHashMap;
import java.util.Map;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.dataformat.yaml.YAMLFactory;
import com.kingsrook.qbits.quicksearch.QuickSearchQBitConfig;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.help.QHelpContent;
import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
/*******************************************************************************
** Applies help/quick-search.yaml (the QQQ help-bundle format) to the Quick
** Search Admin app and its quick_search_* tables: what each table is for and
** how its rows get created, shown on the table screen and when it is empty.
** STAND-IN(core=QRun-IO/qqq#1198, removeBy=4.1.0-RC.9): replace with HelpBundles.register once the pinned QQQ
** release includes it; the bundle file stays the same.
*******************************************************************************/
public final class QuickSearchHelp
{
public static final String RESOURCE = "/help/quick-search.yaml";
public static final String SLOT = "help";
/*******************************************************************************
** Utility class.
*******************************************************************************/
private QuickSearchHelp()
{
}
/*******************************************************************************
** The bundle's entries by key, validated (fails startup if malformed).
*******************************************************************************/
public static Map<String, JsonNode> entries() throws QException
{
try(InputStream stream = QuickSearchHelp.class.getResourceAsStream(RESOURCE))
{
if(stream == null)
{
throw (new IOException("missing resource"));
}
JsonNode bundle = new ObjectMapper(new YAMLFactory()).readTree(stream);
if(!"quick-search".equals(bundle.path("bundle").asText()) || !"QBIT".equals(bundle.path("layer").asText()))
{
throw (new IOException("invalid bundle header"));
}
Map<String, JsonNode> entries = new LinkedHashMap<>();
for(JsonNode entry : bundle.path("entries"))
{
String key = entry.path("key").asText();
String summary = entry.path("summary").asText();
if(key.isBlank() || summary.isBlank() || summary.length() > 120 || entry.path("content").asText().isBlank() || entries.put(key, entry) != null)
{
throw (new IOException("invalid or duplicate entry: " + key));
}
}
return (entries);
}
catch(IOException | RuntimeException e)
{
throw (new QException("Could not load " + RESOURCE, e));
}
}
/*******************************************************************************
** Apply the help to the app and every quick_search_* table in the instance
** (names carry the config's prefix, if any). A host's own help on a table
** is kept (the bundle only fills the slot).
*******************************************************************************/
public static void apply(QInstance qInstance, QuickSearchQBitConfig config) throws QException
{
for(Map.Entry<String, JsonNode> entry : entries().entrySet())
{
String key = entry.getKey();
String name = key.substring(key.indexOf(':') + 1, key.indexOf(';'));
if(key.startsWith("table:"))
{
QTableMetaData table = qInstance.getTable(config.applyTablePrefix(name));
if(table != null && (table.getHelpContent() == null || !table.getHelpContent().containsKey(SLOT)))
{
table.withHelpContent(SLOT, content(entry.getValue()));
}
}
else if(key.startsWith("app:"))
{
QAppMetaData app = qInstance.getApp(config.applyPrefix(name));
if(app != null && (app.getHelpContent() == null || !app.getHelpContent().containsKey(SLOT)))
{
app.withHelpContent(SLOT, content(entry.getValue()));
}
}
}
}
/*******************************************************************************
** Summary as the first paragraph, then the content, as markdown.
*******************************************************************************/
static QHelpContent content(JsonNode entry)
{
return (new QHelpContent().withContentAsMarkdown(entry.path("summary").asText().strip() + "\n\n" + entry.path("content").asText().strip()));
}
}
+54
View File
@@ -0,0 +1,54 @@
bundle: quick-search
version: 1
layer: QBIT
# Help for the Quick Search Admin app and every quick_search_* table, in the QQQ
# help-bundle format (qqq#1198). Each table entry says what the table is for and
# how its rows get created, so an empty table explains itself. QuickSearchHelp
# applies it until the core HelpBundles registry ships in a QQQ release
# (a stand-in for core#1198). Summaries stay under 120 characters
# (QuickSearchTablesGuardrailTest).
entries:
- key: app:quickSearchAdmin;slot:help
summary: "Watch and tune full-text search: which tables are indexed, how they build, and whether search is healthy."
content: |
Nothing needs to be set up by hand. On startup the qbit reads the tables your code marks searchable and writes the index and field rows itself (see Search Indexes). Add a table with Add Search Index, change weights on Searchable Fields, and use Pause Indexing as the emergency stop. The home shows whether OpenSearch is reachable and what the last startup priming changed.
- key: table:quick_search_index;slot:help
summary: "One row per table that is searchable: whether it is indexed, how often it refreshes, and how its last build went."
content: |
**How rows get created:** automatically. On startup the qbit adds a row for every table declared searchable in code (@QuickSearchable or the host's searchable tables) and queues its first build. Add a table yourself with the Add Search Index action; Disable or Remove Search Index stops indexing it. A table you removed is remembered (Removed Config) and not added back. Change the interval or the fields, then run Apply Search Index Changes to rebuild.
- key: table:quick_search_index_field;slot:help
summary: "The fields of each indexed table that search looks at, with their weight and analyzer."
content: |
**How rows get created:** automatically, with the index row, from the field annotations or the host's field list. Edit a weight to make matches in that field rank higher, or add a field by hand; your edits are kept and never overwritten by startup. Changes take effect after Apply Search Index Changes rebuilds the table.
- key: table:quick_search_settings;slot:help
summary: "The single settings row: the indexing pause switch and what startup priming last did."
content: |
**How rows get created:** the qbit creates the one row (id 1) on first startup; do not add others. Use the Pause Indexing and Resume Indexing actions to stop and restart all indexing while search keeps answering from the existing index. The priming fields are written by the qbit and are read-only.
- key: table:quick_search_config_tombstone;slot:help
summary: "Tables and fields you removed from search, so startup does not add them back."
content: |
**How rows get created:** this table starts empty on purpose. A row appears when you Remove Search Index or remove a field that code still declares as searchable. Delete a row to let the code defaults add that table or field again on the next startup.
- key: table:quick_search_index_run;slot:help
summary: "History of every index build, refresh and reconcile: when it ran, how many records, and how it ended."
content: |
**How rows get created:** by the qbit itself. Each Full Reindex, Basepull (the scheduled refresh) and Reconcile writes a row, and the first build of a new index writes the first one. Nothing to set up. Cancel Index Run stops one that is still running. Old rows are purged after the run history retention period.
- key: table:quick_search_failed_event;slot:help
summary: "Real-time index updates that failed and are waiting to be replayed."
content: |
**How rows get created:** by the qbit. When saving a record cannot update OpenSearch, the change is recorded here and the next refresh replays it. This table is normally empty, which is the healthy state. Rows that stay here mean OpenSearch is unreachable or rejecting a document; check the home status.
- key: table:quick_search_diagnostic_event;slot:help
summary: "Failed searches and a trail of diagnostic tool runs, kept for troubleshooting."
content: |
**How rows get created:** by the qbit. A search that hits an OpenSearch error or timeout writes a row (core search then falls back to its own), and so does each run of a diagnostic tool such as Search Playground. Rows are read-only. An empty table means no search has failed.
- key: table:quick_search_validation_run;slot:help
summary: "Results of each Validate Search Index check: pass, warnings and the full report."
content: |
**How rows get created:** run the Validate Search Index action from the Search Indexes table (it can also run on a schedule); each run writes one row with its status and report. It starts empty until someone validates. Rows are kept as history and purged after the run history retention period.
@@ -0,0 +1,207 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.quicksearch;
import java.util.List;
import com.kingsrook.qbits.quicksearch.annotations.QuickSearchField;
import com.kingsrook.qbits.quicksearch.annotations.QuickSearchable;
import com.kingsrook.qbits.quicksearch.config.QuickSearchConfigPrimer;
import com.kingsrook.qbits.quicksearch.model.QuickSearchConfigTombstone;
import com.kingsrook.qbits.quicksearch.model.QuickSearchIndex;
import com.kingsrook.qbits.quicksearch.model.QuickSearchIndexField;
import com.kingsrook.qbits.quicksearch.model.QuickSearchSettings;
import com.kingsrook.qqq.backend.core.actions.tables.UpdateAction;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.actions.tables.update.UpdateInput;
import com.kingsrook.qqq.backend.core.model.data.QRecord;
import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType;
import com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import com.kingsrook.qqq.backend.core.model.session.QSession;
import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryBackendModule;
import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryRecordStore;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** This qbit ships no DDL and no changelog: QuickSearchConfigPrimer seeds its
** config tables at runtime. This proves, on a fresh (memory) backend produced
** by the real producer, that priming populates quick_search_index,
** quick_search_index_field and the quick_search_settings row (id 1), leaves
** quick_search_config_tombstone empty, is idempotent, and never overwrites an
** admin's edit.
*******************************************************************************/
class QuickSearchSeededDefaultsTest
{
/*******************************************************************************
** A searchable entity.
*******************************************************************************/
@QuickSearchable(tableName = "person", basepullIntervalMinutes = 10)
static class PersonEntity
{
@QuickSearchField(weight = 3)
private String name;
@QuickSearchField
private String email;
}
/*******************************************************************************
**
*******************************************************************************/
@BeforeEach
void setUp()
{
MemoryRecordStore.getInstance().reset();
}
/*******************************************************************************
**
*******************************************************************************/
@AfterEach
void tearDown()
{
QContext.clear();
}
/*******************************************************************************
** A fresh install: priming fills the config tables, twice changes nothing,
** and an edited row survives.
*******************************************************************************/
@Test
void primerPopulatesConfigTablesOnFreshBackend_idempotent_andKeepsEdits() throws QException
{
QuickSearchQBitConfig config = produce(List.of(PersonEntity.class));
QuickSearchRuntime runtime = config.getRuntime();
assertThat(rows(QuickSearchIndex.TABLE_NAME)).isEmpty();
assertThat(rows(QuickSearchSettings.TABLE_NAME)).isEmpty();
QuickSearchConfigPrimer.PrimingReport first = new QuickSearchConfigPrimer(runtime).prime();
assertThat(first.newTables()).hasSize(1);
assertThat(rows(QuickSearchIndex.TABLE_NAME)).singleElement().satisfies(row ->
{
assertThat(row.getValueString("tableName")).isEqualTo("person");
assertThat(row.getValueString("source")).isEqualTo(QuickSearchIndex.SOURCE_CODE);
assertThat(row.getValueInteger("basepullIntervalMinutes")).isEqualTo(10);
});
assertThat(rows(QuickSearchIndexField.TABLE_NAME)).extracting(r -> r.getValueString("fieldName")).containsExactlyInAnyOrder("name", "email");
assertThat(rows(QuickSearchSettings.TABLE_NAME)).singleElement().satisfies(row ->
{
assertThat(row.getValueInteger("id")).isEqualTo(QuickSearchSettings.SETTINGS_ID);
assertThat(row.getValueBoolean("indexingPaused")).isFalse();
assertThat(row.getValueString("lastPrimingSummary")).isNotBlank();
});
assertThat(rows(QuickSearchConfigTombstone.TABLE_NAME)).as("tombstones legitimately start empty").isEmpty();
/////////////////////////////////////////////
// an admin edits the interval (user write) //
/////////////////////////////////////////////
QRecord indexRow = rows(QuickSearchIndex.TABLE_NAME).get(0);
new UpdateAction().execute(new UpdateInput(QuickSearchIndex.TABLE_NAME).withRecord(new QRecord().withValue("id", indexRow.getValue("id")).withValue("basepullIntervalMinutes", 45)));
QuickSearchConfigPrimer.PrimingReport second = new QuickSearchConfigPrimer(runtime).prime();
assertThat(second.newTables()).isEmpty();
assertThat(second.newFields()).isZero();
assertThat(rows(QuickSearchIndex.TABLE_NAME)).hasSize(1);
assertThat(rows(QuickSearchIndexField.TABLE_NAME)).hasSize(2);
assertThat(rows(QuickSearchSettings.TABLE_NAME)).hasSize(1);
assertThat(rows(QuickSearchIndex.TABLE_NAME).get(0).getValueInteger("basepullIntervalMinutes")).as("the admin's edit survives priming").isEqualTo(45);
}
/*******************************************************************************
** Even with nothing searchable declared in code, priming creates the settings
** row (id 1), so the pause switch and priming summary always exist.
*******************************************************************************/
@Test
void primerCreatesSettingsRowEvenWithNoCodeTables() throws QException
{
QuickSearchQBitConfig config = produce(List.of(PersonEntity.class));
config.getRuntime().setDiscoveredTables(List.of());
new QuickSearchConfigPrimer(config.getRuntime()).prime();
assertThat(rows(QuickSearchIndex.TABLE_NAME)).isEmpty();
assertThat(rows(QuickSearchSettings.TABLE_NAME)).singleElement().satisfies(row ->
{
assertThat(row.getValueInteger("id")).isEqualTo(QuickSearchSettings.SETTINGS_ID);
assertThat(row.getValueBoolean("indexingPaused")).isFalse();
assertThat(row.getValueString("lastPrimingSummary")).isNotBlank();
});
new QuickSearchConfigPrimer(config.getRuntime()).prime();
assertThat(rows(QuickSearchSettings.TABLE_NAME)).hasSize(1);
}
/*******************************************************************************
** Produce the qbit on a fresh memory-backed instance.
*******************************************************************************/
private QuickSearchQBitConfig produce(List<Class<?>> entityClasses) throws QException
{
QInstance qInstance = new QInstance();
qInstance.addBackend(new QBackendMetaData().withName("memory").withBackendType(MemoryBackendModule.class));
qInstance.withInstanceDefaultAuthentication(new QAuthenticationMetaData().withName("anonymous").withType(QAuthenticationType.FULLY_ANONYMOUS));
qInstance.addTable(new QTableMetaData().withName("person").withBackendName("memory").withPrimaryKeyField("id")
.withField(new QFieldMetaData("id", QFieldType.INTEGER).withIsEditable(false))
.withField(new QFieldMetaData("name", QFieldType.STRING))
.withField(new QFieldMetaData("email", QFieldType.STRING))
.withField(new QFieldMetaData("modifyDate", QFieldType.DATE_TIME)));
QContext.init(qInstance, new QSession());
QuickSearchQBitConfig config = new QuickSearchQBitConfig()
.withStartupMode(QuickSearchStartupMode.DEGRADED)
.withBackendName("memory")
.withOpensearchHost("localhost")
.withOpensearchPort(9200)
.withOpensearchIndexName("seeded-defaults")
.withSearchableEntityClasses(entityClasses);
new QuickSearchQBitProducer().withConfig(config).produce(qInstance);
return (config);
}
/*******************************************************************************
** Every row of a table.
*******************************************************************************/
private List<QRecord> rows(String tableName) throws QException
{
return (QuickSearchConfigPrimer.queryAll(tableName));
}
}
@@ -0,0 +1,292 @@
/*
* Copyright 2026 Kingsrook, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.kingsrook.qbits.quicksearch;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.stream.Stream;
import com.kingsrook.qbits.quicksearch.annotations.QuickSearchField;
import com.kingsrook.qbits.quicksearch.annotations.QuickSearchable;
import com.kingsrook.qbits.quicksearch.metadata.QuickSearchHelp;
import com.kingsrook.qqq.backend.core.context.QContext;
import com.kingsrook.qqq.backend.core.exceptions.QException;
import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType;
import com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
import com.kingsrook.qqq.backend.core.model.metadata.help.QHelpContent;
import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
import com.kingsrook.qqq.backend.core.model.session.QSession;
import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryBackendModule;
import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryRecordStore;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
/*******************************************************************************
** Guardrail (owner requirement 2026-10-14: "all tables should be populated and
** the qbit should be fully using them"). For every quick_search_* table:
** - runtime code (not just its model or metadata) reads or writes it, so no
** table exists only on paper;
** - it is either populated by the startup primer (QuickSearchConfigPrimer; this
** qbit ships no DDL or changelog, so the primer is its seed mechanism) or
** listed in STARTS_EMPTY with why it starts empty;
** - it has bundle help that says what it is for and how its rows get created.
** A new table fails this test until it is wired, seeded or explained.
*******************************************************************************/
class QuickSearchTablesGuardrailTest
{
private static final Path MAIN = Path.of("src", "main", "java", "com", "kingsrook", "qbits", "quicksearch");
/*******************************************************************************
** Tables the primer fills from code on startup (QuickSearchSeededDefaultsTest
** proves it).
*******************************************************************************/
static final Map<String, String> PRIMED = Map.of(
"quick_search_index", "one row per table code declares searchable",
"quick_search_index_field", "one row per searchable field code declares",
"quick_search_settings", "the single settings row (id 1), with the priming summary");
/*******************************************************************************
** Tables that legitimately start empty, and where their rows come from.
*******************************************************************************/
static final Map<String, String> STARTS_EMPTY = Map.of(
"quick_search_config_tombstone", "an admin removes a table or field that code still declares",
"quick_search_index_run", "runtime: every build, basepull and reconcile",
"quick_search_failed_event", "runtime: a real-time index update fails (empty is healthy)",
"quick_search_diagnostic_event", "runtime: a search fails or a diagnostic tool runs",
"quick_search_validation_run", "runtime: someone runs Validate Search Index");
/*******************************************************************************
** A searchable entity for the produced instance.
*******************************************************************************/
@QuickSearchable(tableName = "person")
static class PersonEntity
{
@QuickSearchField(weight = 3)
private String name;
}
/*******************************************************************************
**
*******************************************************************************/
@BeforeEach
void setUp()
{
MemoryRecordStore.getInstance().reset();
}
/*******************************************************************************
**
*******************************************************************************/
@AfterEach
void tearDown()
{
QContext.clear();
}
/*******************************************************************************
** Every table is used by runtime code.
*******************************************************************************/
@Test
void everyTableIsUsedAtRuntime() throws Exception
{
Map<String, String> entityByTable = entityByTable();
List<String> runtimeSource;
try(Stream<Path> files = Files.walk(MAIN))
{
runtimeSource = files.filter(p -> p.toString().endsWith(".java"))
.filter(p -> !p.startsWith(MAIN.resolve("model")) && !p.startsWith(MAIN.resolve("metadata")))
.map(QuickSearchTablesGuardrailTest::read).toList();
}
for(String table : quickSearchTables())
{
String entity = entityByTable.get(table);
assertThat(entity).as("entity class for " + table).isNotNull();
//////////////////////////////////////////////////////////////////////////
// the entity, or the config's name accessor for its table (index and //
// index run are reached by their prefixed names) //
//////////////////////////////////////////////////////////////////////////
Pattern use = Pattern.compile("\\b" + entity + "(\\.TABLE_NAME|\\.class|\\(|::)|\\bget" + entity + "TableName\\(");
assertThat(runtimeSource.stream().anyMatch(source -> use.matcher(source).find()))
.as(table + " (" + entity + ") must be read or written by runtime code, not only declared").isTrue();
}
}
/*******************************************************************************
** Every table is primed or explained.
*******************************************************************************/
@Test
void everyTableIsPrimedOrExplained()
{
QInstance qInstance = producedInstance();
for(String table : quickSearchTables())
{
assertThat(PRIMED.containsKey(table) || STARTS_EMPTY.containsKey(table)).as(table + " must be primed or listed in STARTS_EMPTY").isTrue();
assertThat(PRIMED.containsKey(table) && STARTS_EMPTY.containsKey(table)).as(table + " cannot be both primed and listed as starting empty").isFalse();
}
assertThat(PRIMED.keySet()).allMatch(table -> qInstance.getTable(table) != null, "PRIMED names only real tables");
assertThat(STARTS_EMPTY.keySet()).allMatch(table -> qInstance.getTable(table) != null, "STARTS_EMPTY names only real tables");
}
/*******************************************************************************
** Every table and the app have bundle help that says how rows get created.
*******************************************************************************/
@Test
void everyTableHasEmptyStateHelp() throws Exception
{
QInstance qInstance = producedInstance();
assertThat(QuickSearchHelp.entries()).allSatisfy((key, entry) -> assertThat(entry.path("summary").asText().length()).as(key).isLessThanOrEqualTo(120));
for(String table : quickSearchTables())
{
QTableMetaData meta = qInstance.getTable(table);
List<QHelpContent> help = meta.getHelpContent() == null ? null : meta.getHelpContent().get(QuickSearchHelp.SLOT);
assertThat(help).as(table + " table help").isNotEmpty();
assertThat(help.get(0).getContent()).as(table + " help says how rows get created").contains("How rows get created");
}
assertThat(qInstance.getApp("quickSearchAdmin").getHelpContent()).containsKey(QuickSearchHelp.SLOT);
assertThat(QuickSearchHelp.entries().keySet()).allMatch(key -> !key.startsWith("table:") || qInstance.getTable(key.substring(6, key.indexOf(';'))) != null,
"help names only real tables");
}
/*******************************************************************************
** Help a host already wrote is kept; the bundle only fills empty slots.
*******************************************************************************/
@Test
void hostHelpIsNotOverridden() throws Exception
{
QInstance qInstance = producedInstance();
QTableMetaData table = qInstance.getTable("quick_search_index");
table.getHelpContent().remove(QuickSearchHelp.SLOT);
table.withHelpContent(QuickSearchHelp.SLOT, new QHelpContent("host words"));
QuickSearchHelp.apply(qInstance, new QuickSearchQBitConfig());
assertThat(table.getHelpContent().get(QuickSearchHelp.SLOT).get(0).getContent()).isEqualTo("host words");
}
/*******************************************************************************
**
*******************************************************************************/
private List<String> quickSearchTables()
{
List<String> tables = producedInstance().getTables().values().stream().map(QTableMetaData::getName).filter(n -> n.startsWith("quick_search_")).sorted().toList();
assertThat(tables).hasSize(8);
return (tables);
}
/*******************************************************************************
**
*******************************************************************************/
private QInstance producedInstance()
{
try
{
QInstance qInstance = new QInstance();
qInstance.addBackend(new QBackendMetaData().withName("memory").withBackendType(MemoryBackendModule.class));
qInstance.withInstanceDefaultAuthentication(new QAuthenticationMetaData().withName("anonymous").withType(QAuthenticationType.FULLY_ANONYMOUS));
qInstance.addTable(new QTableMetaData().withName("person").withBackendName("memory").withPrimaryKeyField("id")
.withField(new QFieldMetaData("id", QFieldType.INTEGER).withIsEditable(false))
.withField(new QFieldMetaData("name", QFieldType.STRING))
.withField(new QFieldMetaData("modifyDate", QFieldType.DATE_TIME)));
QContext.init(qInstance, new QSession());
new QuickSearchQBitProducer().withConfig(new QuickSearchQBitConfig()
.withStartupMode(QuickSearchStartupMode.DEGRADED)
.withBackendName("memory")
.withOpensearchHost("localhost")
.withOpensearchPort(9200)
.withOpensearchIndexName("guardrail")
.withSearchableEntityClasses(List.of(PersonEntity.class))).produce(qInstance);
return (qInstance);
}
catch(QException e)
{
throw (new IllegalStateException(e));
}
}
/*******************************************************************************
** model/X.java declares TABLE_NAME = "x".
*******************************************************************************/
private static Map<String, String> entityByTable() throws Exception
{
Map<String, String> rs = new HashMap<>();
Pattern p = Pattern.compile("TABLE_NAME\\s*=\\s*\"([a-z_]+)\"");
try(Stream<Path> files = Files.list(MAIN.resolve("model")))
{
for(Path file : files.toList())
{
Matcher m = p.matcher(read(file));
if(m.find())
{
rs.put(m.group(1), file.getFileName().toString().replace(".java", ""));
}
}
}
return (rs);
}
/*******************************************************************************
**
*******************************************************************************/
private static String read(Path path)
{
try
{
return (Files.readString(path, StandardCharsets.UTF_8));
}
catch(Exception e)
{
throw (new IllegalStateException(e));
}
}
}