From d74bd56b100c5154e79c2edf093d978efb05eb8a Mon Sep 17 00:00:00 2001 From: James Maes Date: Tue, 6 Oct 2026 21:03:18 -0500 Subject: [PATCH] docs: record the 1.0.0-RC.2 release (#12) CHANGELOG gains the RC.2 entry and the #13 to #17 fixes; TODO and SESSION-STATE reflect RC.2 and the remaining GA items. --- CHANGELOG.md | 13 +++++++++++++ docs/SESSION-STATE.md | 24 ++++++++++++++++++------ docs/TODO.md | 15 +++++++++------ 3 files changed, 40 insertions(+), 12 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4905fd9..b78b611 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,10 @@ All notable changes to this project are documented here. The format follows Keep a Changelog; versions follow semantic versioning. +## [1.0.0-RC.2] - 2026-10-06 + +Second release candidate, published to Maven Central from `release/1.0`. Fixes the five pre-GA findings from the PR #10 review (#13 to #17); the changes are listed under 1.0.0 below. Requires QQQ 4.1. No schema change, but two new status values: `REBUILDING` on `quickSearchIndex.status` and `AWAITING_REINDEX` on `quickSearchFailedEvent.status`; a host that sized its own `quickSearchFailedEvent.status` column needs at least 16 characters. Known gaps before GA are tracked in #12. + ## [1.0.0-RC.1] - 2026-10-05 First release candidate of 1.0, published to Maven Central from `release/1.0`. It contains every change listed under 1.0.0 below. Requires QQQ 4.1 (built against 4.1.0-RC.1). Known gaps before GA are tracked in #12. @@ -35,13 +39,22 @@ Requires QQQ 4.1. Closes every blocker of the 2026-10-04 production-readiness au - `IndexingUtils.normalizeSearchText` no longer lower-cases (analyzers do). - `enableScheduledProcesses` is deprecated in favour of `enableBasepullProcess` and `enableMaintenanceProcesses`. - `QuickSearchQBitContext` is deprecated; state lives on `QuickSearchRuntime`, resolved from the `QInstance`. +- Search `offset` counts results the session can read. With record security locks on, hits are scanned from the start in batches of at most 1,000, filtered for access and deduplicated; one call reads at most 10,000 hits over 32 requests, and in per-table mode each of T tables gets max(10,000 / T, 2 x `limitPerTable`) hits. When that share runs out `hasMore` is false and `totalHitsIsLowerBound` is true. With locks on, `totalHits` counts hits confirmed readable. Per-table mode now honours `offset` (#16). +- A full reindex of all tables marks each table `REBUILDING` while it runs; drift detection is suspended for those tables. After a killed run, run the full reindex again (#14). ### Removed - The three table customizers (`QuickSearchPostInsertCustomizer`, `QuickSearchPostUpdateCustomizer`, `QuickSearchPostDeleteCustomizer`), replaced by `QuickSearchRecordChangeListener`. - Support for QQQ 4.0. +### Security +- `opensearchUrl` with userinfo (`https://user:pass@host`) fails validation; an `${env.X}` or `${prop.X}` credential or TLS path that resolves to an empty or blank value fails transport creation and names the field, never the value; `tls.hostnameVerification=false` logs a warning (#15). + ### Fixed - Rolled-back writes could be indexed; real-time failures were silent; basepull lost rows modified during a run and rows at the boundary second; unordered paging skipped or duplicated rows; full reindex blanked search; search leaked labels and snippets across tables regardless of permissions; mixed field types across tables broke indexing; non-text searchable fields failed every search; record labels were null for annotation tables; duplicate `quickSearchIndex` rows on concurrent first runs; unbounded `limit`/`offset`; unchecked `OpenSearchException`; unreachable cluster at startup left the host healthy with search broken. +- Deletes made during a full reindex of all tables were lost from the new index; they are now captured and applied after the alias swap (#14). +- An update was dropped without a `quickSearchFailedEvent` row when the listener's re-read failed or the optional AWS SDK was missing (`LinkageError`); both are now recorded for basepull, and a `DEGRADED` start without the AWS SDK no longer aborts host startup (#13). +- Search pages overlapped under record locks, per-table mode ignored `offset`, and a large `maxSearchLimit` could exceed the 10,000 result window (#16). +- A failing document count was swallowed and cleared `documentCount`; it is now logged at warn and the previous count is kept (#17). ## [0.2.0] - 2026-03-30 - Config-driven table indexing (`SearchableTableConfig`). diff --git a/docs/SESSION-STATE.md b/docs/SESSION-STATE.md index 36a7b35..6c59a29 100644 --- a/docs/SESSION-STATE.md +++ b/docs/SESSION-STATE.md @@ -1,19 +1,31 @@ # Session State -Updated: 2026-10-05 +Updated: 2026-10-06 ## Context -`1.0.0-RC.1` is released (2026-10-05). PR #10 merged as `0cd892d` and PR #8 as `36bc0fb`, both on `develop`. `release/1.0` was cut from `36bc0fb`; the orb's `publish_release_candidate` workflow succeeded, pushed `0b7bc98` ("Bump version to 1.0.0-RC.1 [skip ci]") to `release/1.0`, and published to Maven Central (jar, pom, sources, javadoc, signatures). A clean consumer resolves it with qqq-backend-core 4.1.0-RC.1 and opensearch-java 3.10.0. +`1.0.0-RC.2` is released (2026-10-06). It fixes the five pre-GA findings from the PR #10 review, each on its own branch and PR into `develop`: + +| Issue | PR | Merge commit | Fix | +|---|---|---|---| +| #15 | #18 | `2398e62` | Transport: warn on `hostnameVerification=false`, blank `${env.X}` treated as missing, userinfo in `opensearchUrl` rejected | +| #17 | #21 | `e9ec54a` | `safeCount` logs at warn and keeps the previous `documentCount` | +| #16 | #19 | `9941a70` | Search `offset` counts readable results under record locks; bounded scan; 10,000 window cap | +| #14 | #20 | `dce0f26` | Deletes during a full reindex captured (`REBUILDING`, `AWAITING_REINDEX`) and applied after the alias swap | +| #13 | #22 | `c5c41d4` | Listener re-read failure and missing AWS SDK (`LinkageError`) recorded as failed events | + +`develop` (`c5c41d4`) was merged into `release/1.0` as `b7d366c` (only difference from develop: `1.0.0-RC.1`). The orb's `publish_release_candidate` workflow succeeded on `b7d366c` (CircleCI build 82) and pushed `c11b2c2` ("Bump version to 1.0.0-RC.2 [skip ci]"). Maven Central serves the jar, pom, sources, javadoc and signatures for `com.kingsrook.qbits:qbit-quick-search:1.0.0-RC.2`. ## Release mechanics (qqq-orb 0.6.8, confirmed in its source) - RCs publish artifacts only: no git tag and no GitHub release. The GitHub release comes when the release branch merges to `main` and a `vX.Y.Z` tag is pushed. - Never push a `vX.Y.Z-RC.n` tag: it matches the GA job's `v*` filter, maps back to `release/X.Y.Z`, and would publish a stray `RC.n+1`. -- The next push to `release/1.0` publishes `1.0.0-RC.2`. -- The agent cannot run `gh pr merge` (auto-mode classifier); James merges. GitHub SSH times out from this machine; push over HTTPS with `git -c credential.helper='!gh auth git-credential' push https://github.com/QRun-IO/qbit-quick-search.git `. +- The next push to `release/1.0` publishes `1.0.0-RC.3`. Keep the release branch's `` on merge. +- `develop` requires one approving review and the PR author cannot approve their own PR; James merges with `gh pr merge --squash --admin`. The agent cannot run `gh pr merge`, and the auto-mode classifier also refused a background loop that polled PR merge state. +- GitHub SSH times out from this machine; push over HTTPS with `git -c credential.helper='!gh auth git-credential' push https://github.com/QRun-IO/qbit-quick-search.git `. ## Next -1. Pin `1.0.0-RC.1` in qqq-all and Voyage (see `docs/TODO.md`); their feedback decides RC.2 or GA. -2. Work the "Before 1.0.0 GA" list in `docs/TODO.md` on feature branches into `develop`, then merge `develop` into `release/1.0` for RC.2. +1. qqq-all and Voyage pin `1.0.0-RC.2` and report back; their feedback decides RC.3 or GA. +2. Remaining "Before 1.0.0 GA" items in `docs/TODO.md`: stuck `REBUILDING` after a killed or overlapping full reindex (from the PR #20 review), and the `qbit-build-parent` 2.1.0 re-pin. +3. GA: merge `release/1.0` to `main` and tag `v1.0.0`. diff --git a/docs/TODO.md b/docs/TODO.md index b0afdee..d6bc9a3 100644 --- a/docs/TODO.md +++ b/docs/TODO.md @@ -2,7 +2,7 @@ ## Before tagging 1.0.0 -- [ ] qqq-all and Voyage adopt `1.0.0-RC.1` (items below) and report back. +- [ ] qqq-all and Voyage adopt `1.0.0-RC.2` (items below) and report back. - [x] PR #10 (`feature/GH-1-0-release-readiness`) opened and reviewed 2026-10-05; `AGENTS.md` kept. - [ ] Re-pin to `qbit-build-parent` 2.1.0 when qbit-bom publishes it, and remove the `activeByDefault` on the `qqq-snapshot` profile (ADR-0007). @@ -17,11 +17,14 @@ ## Before 1.0.0 GA (from the PR #10 review, 2026-10-05; filed as #13 to #17 under #12) -- [ ] Listener: if the in-transaction re-read in `fetchCurrentRecords` throws, record a `quickSearchFailedEvent` row from the event's primary keys instead of dropping the update (`QuickSearchRecordChangeListener`). -- [ ] Full reindex: deletes that arrive during a rebuild are not replayed into the new index; capture them (failed-event or a delete log) and apply after the swap. -- [ ] Transport: warn when `tls.hostnameVerification=false`; treat an empty `${env.X}` value as missing and name the field; reject userinfo in `opensearchUrl`. -- [ ] Search: with record locks on, `offset` counts raw hits, so pages can overlap; per-table mode ignores `offset`; cap `fetchSize` at the 10,000 window. -- [ ] `safeCount` in `BasepullIndexStep` swallows exceptions silently; log at warn. +- [x] #13 Listener: a failed in-transaction re-read or a missing AWS SDK (`LinkageError`) records `quickSearchFailedEvent` rows (PR #22). +- [x] #14 Full reindex: deletes made during a rebuild are captured (`REBUILDING`, `AWAITING_REINDEX`) and applied after the alias swap (PR #20). +- [x] #15 Transport: warn on `tls.hostnameVerification=false`, empty `${env.X}` treated as missing, userinfo in `opensearchUrl` rejected (PR #18). +- [x] #16 Search: `offset` counts readable results under record locks, per-table `offset`, bounded scan and 10,000 window cap (PR #19). +- [x] #17 `safeCount` logs at warn and keeps the previous `documentCount` (PR #21). +- [x] `1.0.0-RC.2` published to Maven Central from `release/1.0` (2026-10-06). +- [ ] Stuck `REBUILDING` after a killed or overlapping full reindex (from the PR #20 review): only another all-tables reindex clears it, `AWAITING_REINDEX` rows accumulate and drift detection stays off for those tables. No data is lost. Fix needs a liveness check on the `FULL_REINDEX` run record before reconcile may reset the status, plus refusing a second concurrent full reindex. +- [ ] Drift detection still writes `NEEDS_REINDEX` into `status`, so a few-millisecond window can overwrite `REBUILDING`; the durable fix is a separate drift field (schema change, so 1.1 at the earliest). ## Later (1.x) -- 2.54.0