feat: serve QQQ /qqq/v1/search from OpenSearch (record-search provider) #32

2026-10-11T00:18:27Z

KofTwentyTwo commented

2026-10-11 00:18:27 +00:00

(Migrated from github.com)

Summary

Adds QuickSearchRecordSearchProvider, an implementation of QQQ core's RecordSearchProviderInterface. With it, RecordSearchAction / POST /qqq/v1/search (the Next UI global search) is served from the Quick Search index for the tables this QBit indexes. This closes audit finding F-11 / decision D1.

Depends on QRun-IO/qqq#1042 (not yet merged or released). The qqq-snapshot profile now imports qqq-bom-pom:4.1.0-SNAPSHOT. CI on this PR fails to compile until #1042 is merged and a 4.1.0-SNAPSHOT that contains the SPI is published. Before releasing 1.1.0, re-point the profile to the QQQ release that carries the SPI (tracked in docs/TODO.md).

What it does

Claim: the provider claims a table when Quick Search indexes it, it is enabled in quickSearchIndex, and record search is not in backoff. The runtime caches the enabled flag.
Search: sends one _msearch request with one top-N search per table. Each search uses the same query, AND semantics and field weights as QuickSearchAction. It fetches only recordId, up to min(100, 4 x limitPerTable) ids per table, in relevance order. Core then re-reads the ids by primary key, so permissions, record locks and labels stay with core, and stale ids drop out.
Timeouts and fallback: recordSearchTimeoutMillis (default 2500) caps the wall-clock time. On timeout the request is cancelled and an OpenSearch server-side timeout also applies. Any failure throws, and core then falls back to its LIKE search. A failure also starts a 30 s backoff (QuickSearchRuntime.markRecordSearchFailed) during which no tables are claimed. This way a down cluster does not add a timeout to every search.
Registration: QuickSearchQBitConfig.withServeCoreRecordSearch(Boolean) defaults to true. postProduceActions registers the provider unless the host already registered a different one; in that case it keeps the host's provider and logs a warning.
Default searchFields: every indexed table that has none gets its searchable fields that core accepts (visible STRING/TEXT/HTML/INTEGER/LONG, never hidden or PASSWORD). It skips tables on variant backends and tables that are not in the instance yet. App-provided searchFields are never replaced.
Security fix found while verifying: PASSWORD-type fields were indexed in 1.0 (only hidden fields were excluded). They are now excluded too. Reconcile or reindex once to purge them from existing documents.
Version and docs: version bumped to 1.1.0-SNAPSHOT. Updated README (new "Core search" section and config rows), docs/MIGRATION-1.1.md, CHANGELOG [1.1.0], CLAUDE/AGENTS and TODO.

Tests

QuickSearchRecordSearchProviderTest (10 tests):
- claim logic: indexed, unindexed, disabled, off, backoff, no runtime
- id mapping, order and per-table cap
- short term and unknown tables
- failure, both checked and unchecked, leads to backoff
- a real client against a hung TCP server times out with QException after 300 ms
- core RecordSearchAction end to end with a mock client: re-read, order, stale id, limitPerTable, unclaimed table
- fallback plus backoff
QuickSearchQBitProducerTest (+6 tests):
- registration, opt-out, host provider and searchFields kept, idempotent registration
- hidden, password and non-text fields excluded
- missing tables and variant backends
QuickSearchQBitConfigValidationTest: covers recordSearchTimeoutMillis.
RecordSearchProviderIntegrationTest (Testcontainers OpenSearch, 5 tests):
- the producer wires the provider and searchFields
- "blue widget" returns both matching products in Quick Search's order (LIKE would find one) with core labels
- the unindexed table keeps LIKE
- limitPerTable
- a stale index hit is dropped
- after the container is stopped, core falls back to LIKE results and the provider backs off
Local run, against 4.1.0-SNAPSHOT built from the #1042 branch, on Windows with Docker: CI=true ./mvnw verify gave BUILD SUCCESS. 369 unit tests and 24 integration tests passed, and the JaCoCo gate was met.

For consuming apps (Website-Backend, qqq-all)

Depend on quick-search 1.1.0 and on the QQQ release that includes #1042.
Nothing else is needed for the default: the provider is registered and indexed tables get searchFields.
Add tables to the QInstance before producing the QBit. Otherwise those tables get no searchFields, and core search will not offer them.
If the app sets searchFields itself, they are kept.
To opt out, use withServeCoreRecordSearch(false).
Optionally tune withRecordSearchTimeoutMillis(...).

🤖 Generated with Claude Code

## Summary Adds `QuickSearchRecordSearchProvider`, an implementation of QQQ core's `RecordSearchProviderInterface`. With it, `RecordSearchAction` / `POST /qqq/v1/search` (the Next UI global search) is served from the Quick Search index for the tables this QBit indexes. This closes audit finding F-11 / decision D1. > **Depends on QRun-IO/qqq#1042 (not yet merged or released).** The `qqq-snapshot` profile now imports `qqq-bom-pom:4.1.0-SNAPSHOT`. CI on this PR **fails to compile** until #1042 is merged and a 4.1.0-SNAPSHOT that contains the SPI is published. Before releasing 1.1.0, re-point the profile to the QQQ release that carries the SPI (tracked in `docs/TODO.md`). ## What it does - **Claim:** the provider claims a table when Quick Search indexes it, it is enabled in `quickSearchIndex`, and record search is not in backoff. The runtime caches the enabled flag. - **Search:** sends one `_msearch` request with one top-N search per table. Each search uses the same query, AND semantics and field weights as `QuickSearchAction`. It fetches only `recordId`, up to `min(100, 4 x limitPerTable)` ids per table, in relevance order. Core then re-reads the ids by primary key, so permissions, record locks and labels stay with core, and stale ids drop out. - **Timeouts and fallback:** `recordSearchTimeoutMillis` (default 2500) caps the wall-clock time. On timeout the request is cancelled and an OpenSearch server-side `timeout` also applies. Any failure throws, and core then falls back to its LIKE search. A failure also starts a 30 s backoff (`QuickSearchRuntime.markRecordSearchFailed`) during which no tables are claimed. This way a down cluster does not add a timeout to every search. - **Registration:** `QuickSearchQBitConfig.withServeCoreRecordSearch(Boolean)` defaults to true. `postProduceActions` registers the provider unless the host already registered a different one; in that case it keeps the host's provider and logs a warning. - **Default `searchFields`:** every indexed table that has none gets its searchable fields that core accepts (visible STRING/TEXT/HTML/INTEGER/LONG, never hidden or PASSWORD). It skips tables on variant backends and tables that are not in the instance yet. App-provided `searchFields` are never replaced. - **Security fix found while verifying:** PASSWORD-type fields were indexed in 1.0 (only hidden fields were excluded). They are now excluded too. Reconcile or reindex once to purge them from existing documents. - **Version and docs:** version bumped to `1.1.0-SNAPSHOT`. Updated README (new "Core search" section and config rows), `docs/MIGRATION-1.1.md`, CHANGELOG `[1.1.0]`, CLAUDE/AGENTS and TODO. ## Tests - `QuickSearchRecordSearchProviderTest` (10 tests): - claim logic: indexed, unindexed, disabled, off, backoff, no runtime - id mapping, order and per-table cap - short term and unknown tables - failure, both checked and unchecked, leads to backoff - a real client against a hung TCP server times out with `QException` after 300 ms - core `RecordSearchAction` end to end with a mock client: re-read, order, stale id, `limitPerTable`, unclaimed table - fallback plus backoff - `QuickSearchQBitProducerTest` (+6 tests): - registration, opt-out, host provider and `searchFields` kept, idempotent registration - hidden, password and non-text fields excluded - missing tables and variant backends - `QuickSearchQBitConfigValidationTest`: covers `recordSearchTimeoutMillis`. - `RecordSearchProviderIntegrationTest` (Testcontainers OpenSearch, 5 tests): - the producer wires the provider and `searchFields` - "blue widget" returns both matching products in Quick Search's order (LIKE would find one) with core labels - the unindexed table keeps LIKE - `limitPerTable` - a stale index hit is dropped - after the container is stopped, core falls back to LIKE results and the provider backs off - Local run, against 4.1.0-SNAPSHOT built from the #1042 branch, on Windows with Docker: `CI=true ./mvnw verify` gave BUILD SUCCESS. 369 unit tests and 24 integration tests passed, and the JaCoCo gate was met. ## For consuming apps (Website-Backend, qqq-all) - Depend on quick-search 1.1.0 and on the QQQ release that includes #1042. - Nothing else is needed for the default: the provider is registered and indexed tables get `searchFields`. - Add tables to the `QInstance` before producing the QBit. Otherwise those tables get no `searchFields`, and core search will not offer them. - If the app sets `searchFields` itself, they are kept. - To opt out, use `withServeCoreRecordSearch(false)`. - Optionally tune `withRecordSearchTimeoutMillis(...)`. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

KofTwentyTwo commented

2026-10-11 18:48:17 +00:00

(Migrated from github.com)

Superseded by a re-signed PR pinned to the released QQQ 4.1.0-RC.2 (see the PR referencing #32). Closing.

🤖 Generated with Claude Code

Superseded by a re-signed PR pinned to the released QQQ 4.1.0-RC.2 (see the PR referencing #32). Closing. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

Pull request closed

Please reopen this pull request to perform a merge.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: QRun-IO/qbit-quick-search#32