feat(#35): search admin tooling (playground, validation, record tools, analyzer, failure diagnostics) #39

Merged
KofTwentyTwo merged 5 commits from feature/GH-35-search-admin-tooling into develop 2026-10-12 12:37:58 +00:00
5 Commits
Author SHA1 Message Date
james.maes e6d181c1f8 Merge remote-tracking branch 'origin/develop' into feature/GH-35-search-admin-tooling 2026-10-12 06:14:01 -05:00
james.maesandClaude Opus 5.5 be5df8e00a fix(#35): review findings: never store the term, hide unreadable hits, server-side targets
- Search failure events redact the user term (and its words) from the
  error message and detail: the client wrap message names the term.
- Playground: hits the user cannot read are counted, never described (no
  id, score or matched fields); they are removed from the raw response with
  the total and max score; provider ids are filtered to readable records.
- Inspect / Reindex run steps take their targets from the records the
  backend loaded into the process state, never from client values; failed
  event targets must be indexed tables; reindex skips ids that are not
  valid keys instead of deleting documents for them.
- History tables disable insert, update and delete for users and the API,
  so validation report HTML is always the QBit output.
- A validation counts as scheduled only without a user session.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 06:13:48 -05:00
james.maesandClaude Opus 5.5 7287093afa Merge origin/develop (#25 stuck REBUILDING recovery) into feature/GH-35-search-admin-tooling
The validator's REBUILDING finding now points at the automatic recovery.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 06:07:25 -05:00
james.maesandClaude Opus 5.5 e9ec814762 Merge origin/develop into feature/GH-35-search-admin-tooling
Keeps develop's scheduling section and applies its cron rules (cron-capable
scheduler, valid Quartz expression, time zone ID) to validationCronExpression
through one shared check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 06:04:44 -05:00
james.maesandClaude Opus 5.5 b421b4002a feat(#35): search admin tooling (playground, validation, record tools, analyzer, failure diagnostics)
Adds diagnostic tools so every QQQ app using the QBit can validate its
search and debug it from the admin UI. All are table or record actions
(processes with a tableName, never added to the app nav), gated by
permissions, rate-limited per user and tool, recorded as diagnostic events,
and read-only except Reindex in Search.

- Search Playground (Quick Search Index): runs a term as the current user
  the way the global search does; per-table path and reason (OpenSearch
  provider, core database search, not searched), the provider's request
  timed, the global search results, the production query's scores,
  highlights and matched clauses, readable / shown flags, query DSL and raw
  response (unreadable hits redacted), term tokens, explain for a hit.
  The form loops back so a user can search again.
- Validate Search Index (Quick Search Index; schedulable with
  validationCronExpression): cluster and index health, alias, leftover
  physical indexes, live mapping and analysis settings, per-table counts,
  sampled missing / stale documents and orphaned documents, index row status
  and failed events. Reports kept in the new quickSearchValidationRun table.
- Test Search Analyzer: _analyze with the index's analyzers, and whether a
  term's tokens are among a text's indexed tokens.
- Inspect Search Document and Reindex in Search on every indexed table and
  on quickSearchFailedEvent rows: database vs index diff, and why a record
  does or does not match a term (token-level reasons, explain, rank).
- Search failure diagnostics: provider and QuickSearchAction failures are
  recorded as SEARCH_FAILED events (never the term, only its length), with
  runtime counters; the Search Diagnostics widget shows provider state,
  recent failures from every server and the latest validation.
- New settings: enableDiagnosticTools, enableRecordDiagnosticActions,
  validationCronExpression, validationCronTimeZoneId, validationSampleSize,
  diagnosticRunsPerMinute, debugPermissionRules (quickSearchDebug).
- History purge covers the two new tables.

Tests: unit (metadata incl. QInstanceValidator, offline tool behaviour
against a closed port, the playground loop through RunProcessAction) and a
Testcontainers OpenSearch integration test for every tool.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 06:02:29 -05:00