Regenerated against UI API 1.6.0: every operation has an operationId and
the standard error responses (BasicErrorResponseV1: error, code, message,
fieldErrors, traceId); the extension's own error schema is kept but
deprecated; hand-written error bodies go through QJavalinUtils so they carry
code/message/traceId. Extension MINOR bumped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Host-neutral help (deployment-specific facts removed), registered with
HelpBundles, gated by HelpBundleAssertions. QQQ 4.1.0-RC.7.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds help/quick-search.yaml (QQQ help bundle) applied by QuickSearchHelp, a
tables guardrail test, and a seeded-defaults test proving the primer fills
the config tables on a fresh backend.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Secured-data spec section 15: Quick Search must never put classified
values in the shared search index, snippets or results.
QuickSearchFieldRules decides it in one place on QQQ 4.1.0-RC.6:
isProtected is true for fields classified PII or PHI (field, else the
table's classification), fields with a field security lock, sensitive
fields and masked/password types. Such fields are never indexable;
explicit configs drop them with a warning (never the value); a table
whose primary key is protected (for example a PHI table) is not indexed
at all. Record labels built from protected fields fall back, the
version field is skipped when protected, and results from tables with
protected fields are rebuilt from allowed values only, so documents
indexed before this change cannot leak through labels or snippets.
Tables with only harmless non-indexable fields (booleans, blobs) keep
their labels and are not result-filtered.
A canary test proves a PHI and a PII marker never reach a captured
index request, result or snippet. RC.7 follow-up: also honour core
ClassifiedValues.isClassified (qqq#1182) and FieldAccessHelper lock
tiers (qqq#1172).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
realTimeErrorCount was only ever written as 0, so the status screen,
diagnostics and the status API always said there were no real-time
errors. When the change listener records failed events it now adds
their number to the table's count (after commit, serialized in the JVM,
never failing the user's write). A successful basepull or full reindex
resets it; the field's help text says so.
Fixes#53
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hosts no longer need QuickSearchOpenSearchClient or a copy of this qbit's
diagnostics. The new public api package holds:
- QuickSearchStatus, an immutable snapshot: overall state, cluster
reachability, per-table index counts, backlog, last sync, last error
and checkedAt;
- QuickSearchStatusService.get(), bounded and cached for five seconds,
built on the existing diagnostics;
- QuickSearchApi, with the app name and home href, so hosts link to the
Quick Search home instead of duplicating it.
The same data is served as UI API extension operation
GET /qqq/v1/qbit/quick-search/status (getQuickSearchStatus), gated by the
admin permission. The fragment is committed and checked by a contract
test. The core baseline becomes 4.1.0-RC.6.
Closes#50 (architecture review QRun-IO/qqq#1150, finding F10).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- quick_search_index_run and quick_search_failed_event are system
history: Create and Edit are off (Delete stays), with named sections,
record labels and pickers (run type and status, event status and
action, and the table picker for failed events' tableName).
- The index record lists its Recent Runs (read-only child list).
- quick_search_validation_run.reportJson uses a JSON code editor.
- Basepull, Full Reindex and Reconcile are table actions on Quick Search
Index, not app (nav) children. Their table input is searchTableName
(tableName still accepted; a Quick Search table name is ignored), so
running them from the table no longer limits them to quick_search_index.
- Process input steps carry HELP_TEXT and EDIT_FORM components, so the
Next UI renders their fields; Add Search Index's table input is
searchTableName for the same reason.
- The Search Diagnostics widget shows the pause, priming and pending
changes as banners; screenshots under docs/images/data-driven-config.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The searchable tables and fields are now standard QQQ records:
quick_search_index (the registry; new source and ownerEdited columns) and
the new quick_search_index_field (weight, role, analyzer, snippet and
label flags). Indexing, search, the record-search provider and the
diagnostics read the effective config from them through
QuickSearchConfigStore (30 s cache, dropped by the tables' customizers),
so changes need no redeploy.
Code (@QuickSearchable, searchableTables) only seeds the records:
QuickSearchConfigPrimer adds missing tables and fields at startup and
queues their builds, updates rows nobody edited, never overwrites rows an
admin edited (ownerEdited) and never re-adds what an admin removed
(quick_search_config_tombstone). Basepull builds queued tables
(PENDING_BUILD, BUILDING) and recreates a missing index; an empty index
is rebuilt. Without the config tables the code config is used as before.
The generic Create and Delete of quick_search_index are replaced by
streamed-ETL processes with review steps: Add Search Index (eligible
readable tables only, estimated record count, default fields, sensitive
and hidden fields never indexed), Apply Search Index Changes (previews
field changes, then rebuilds), Disable / Remove Search Index (delete or
keep documents), Cancel Index Run (cooperative flag between batches; runs
end CANCELLED with counts) and Pause / Resume Indexing (global pause in
quick_search_settings; search keeps serving; changes are deferred and
replayed). Field roles shape documents (TITLE label, SUBTITLE, DISPLAY
values, KEYWORD and EXACT outside the snippet) and WORD, ENGLISH and EXACT
analyzers get their own document objects and mapping templates.
Config tables are audited at field level and permission-gated; the
processes need quick_search_admin. Upgrade notes and DDL:
docs/MIGRATION-1.1.md, "Data-driven configuration".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner rule (2026-10-12): "table names should always be lowercase and use
_, and qbits should always be prefixed with their name". For 1.1.0-RC.3,
metadata only (the QBit ships no DDL):
- tables: quick_search_index, quick_search_index_run,
quick_search_failed_event, quick_search_diagnostic_event,
quick_search_validation_run (were camelCase)
- possible-value sources: quick_search_index, quick_search_searchable_table
- permission bases: quick_search_admin, quick_search_debug; table
permissions follow the table names
- applyTablePrefix: a host prefix is snake_cased and joined with "_"
(myApp -> my_app_quick_search_index); processes, widgets and apps keep
applyPrefix
- @Deprecated LEGACY_* constants keep the old names for upgrade scripts
- docs/MIGRATION-1.1.md: upgrade steps, ALTER TABLE for camelCase
databases, UPDATEs for audit_table, saved views and reports, permission
grants; no reindex needed
- QuickSearchTableNamesTest covers new names, legacy constants and prefixing
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keeps develop's scheduling section and applies its cron rules (cron-capable
scheduler, valid Quartz expression, time zone ID) to validationCronExpression
through one shared check.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds diagnostic tools so every QQQ app using the QBit can validate its
search and debug it from the admin UI. All are table or record actions
(processes with a tableName, never added to the app nav), gated by
permissions, rate-limited per user and tool, recorded as diagnostic events,
and read-only except Reindex in Search.
- Search Playground (Quick Search Index): runs a term as the current user
the way the global search does; per-table path and reason (OpenSearch
provider, core database search, not searched), the provider's request
timed, the global search results, the production query's scores,
highlights and matched clauses, readable / shown flags, query DSL and raw
response (unreadable hits redacted), term tokens, explain for a hit.
The form loops back so a user can search again.
- Validate Search Index (Quick Search Index; schedulable with
validationCronExpression): cluster and index health, alias, leftover
physical indexes, live mapping and analysis settings, per-table counts,
sampled missing / stale documents and orphaned documents, index row status
and failed events. Reports kept in the new quickSearchValidationRun table.
- Test Search Analyzer: _analyze with the index's analyzers, and whether a
term's tokens are among a text's indexed tokens.
- Inspect Search Document and Reindex in Search on every indexed table and
on quickSearchFailedEvent rows: database vs index diff, and why a record
does or does not match a term (token-level reasons, explain, rank).
- Search failure diagnostics: provider and QuickSearchAction failures are
recorded as SEARCH_FAILED events (never the term, only its length), with
runtime counters; the Search Diagnostics widget shows provider state,
recent failures from every server and the latest validation.
- New settings: enableDiagnosticTools, enableRecordDiagnosticActions,
validationCronExpression, validationCronTimeZoneId, validationSampleSize,
diagnosticRunsPerMinute, debugPermissionRules (quickSearchDebug).
- History purge covers the two new tables.
Tests: unit (metadata incl. QInstanceValidator, offline tool behaviour
against a closed port, the playground loop through RunProcessAction) and a
Testcontainers OpenSearch integration test for every tool.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(#29): validate cron zone, warn when basepull is off, document schedulers
A reconcileCronExpression without reconcileCronTimeZoneId passed the
QBit's validation, then QQQ's QInstanceValidator rejected the reconcile
process schedule at startup ("a cron schedule must specify a
cronTimeZoneId"); QQQ has no default zone. validate() now applies the
same checks QQQ does (Quartz CronExpression.validateExpression, a zone
from TimeZone.getAvailableIDs) and requires the zone, naming both
fields. It also rejects a cron on a scheduler in the instance that does
not support cron: QQQ's SimpleScheduler runs fixed intervals only and
fails when the schedule manager starts.
Config-driven tables without their basepull timestamp field now log the
same startup warning as annotated tables, once per table, through one
shared helper. A field set to null is an opt-out and stays quiet.
README and MIGRATION-1.0 document that schedules run only when the host
starts through QApplicationLauncher (qqq-middleware-javalin), and that a
reconcile cron needs a Quartz scheduler and a time zone ID.
Closes#29
* docs(producer): keep develop's hidden-and-password Javadoc after the merge
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* docs(#26): fix migration guide and README gaps
The migration guide now names the permissions a host must grant and has
a schema section for the 0.x to 1.0 table changes, including the new
quickSearchFailedEvent table and the status value lengths. The
plaintext credentials setter is spelled withAllowPlaintextCredentials,
and the README and PRD point at 1.0.0-RC.2 as the current release.
Closes#26
* docs(#26): name each table in the migration schema step
The schema step now says which table each STRING to TEXT change belongs
to and includes the required, unique quickSearchIndex.tableName, matching
the schema section.
Refs #26
* docs(readme): name 1.1.0-RC.1 as the current release candidate
Maven Central's newest is 1.1.0-RC.1 (develop is the 1.1 line). Keep the
1.0 line's 1.0.0-RC.2 for hosts that stay on 1.0.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Implements QQQ core's RecordSearchProviderInterface so the Next UI global
search (POST /qqq/v1/search) is served from the Quick Search index for the
tables this QBit indexes. The provider returns record ids only; core re-reads
them through QueryAction, so table permissions, record security locks and
labels stay core's job. One bounded _msearch per call, with a short backoff
after a failure during which core falls back to its own LIKE search.
Builds against the released QQQ 4.1.0-RC.2, the first release that carries
the SPI (replaces the 4.1.0-SNAPSHOT pin from PR #32).
Re-signed from PR #32 (originally 75dc2b3).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Real-time INSERT events were indexed from the event's own records, which
carry raw values only, so a foreign key was indexed as its id and the
record label as "2 - email" until the next basepull. Inserts now take the
same path as updates: the listener re-reads the written records inside
the write's transaction with display values and possible-value labels.
A failed re-read is handled as for updates (#13): logged at once, then
recorded in quickSearchFailedEvent after commit for basepull replay,
never failing the host's insert, and nothing recorded on rollback.
Tests: listener unit tests for the PVS label and record label of a
just-inserted record, and for a failed insert re-read (commit and
rollback); an OpenSearch Testcontainers test that a just-inserted
record is searchable by its possible-value label and carries it in
fieldValues and the record label.
Closes#28
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs: add 1.0 PRD, deep-dive audit, plan, and session state
* build: target qqq 4.1.0-RC.1, opensearch-java 3.10.0, wrapper 3.9.11, real JaCoCo gate
* feat: add 1.0 config surface, instance-scoped runtime, and deprecated static facade
* feat: implement the 4.1 QBit contract with a record-change listener and failed-event capture
* feat: opensearch client with auth modes, TLS, timeouts, mapping v2, aliases, versioned bulk writes
* feat: trustworthy indexing steps and permission-aware bounded search
* docs: rewrite README for 1.0, add CHANGELOG and migration guide
* feat: table picker and result screens for the maintenance processes; close out 1.0 planning docs
* docs: add AGENTS.md so Codex shares the CLAUDE.md guidance
* fix: keep the live index after an alias swap, atomic legacy upgrade, exact paging under record locks
From the PR #10 review: never delete the new physical index once the alias points at it, upgrade a legacy concrete index in one aliases request, base hasMore on raw hits so filtered pages stay reachable, and deduplicate requested table names.
* docs: explain the hasMore trade-off under record locks
* docs: session state for the RC.1 hand-off
- Add FullReindexStepTest with 7 tests
- Add BasepullIndexStepTest with 8 tests
- Add QuickSearchQBitProducerTest with 12 tests
- Update CLAUDE.md with project status
- Add docs/TODO.md and docs/SESSION-STATE.md
james.maesandClaude Opus 5.5