Host-neutral help (deployment-specific facts removed), registered with
HelpBundles, gated by HelpBundleAssertions. QQQ 4.1.0-RC.7.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@@ -6,6 +6,8 @@ Following [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and [Semantic
### Added
- Complete host-neutral in-app help bundle with live metadata references and a core help coverage gate.
- Analysis Settings table action **Create rows for analyzable tables**: writes the effective settings as editable `data_analysis_table_config` rows (insert-only, no change in behavior).
- Setup card on the Analyses home (settings, result cache, saved analyses, watches, schedules).
- Help bundle `help/qbit-data-analysis.yaml`: every table says what it is for and how its rows get created.
@@ -40,12 +42,13 @@ Following [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and [Semantic
### Changed
- Pin QQQ 4.1.0-RC.7 and replace the temporary help loader with core bundle registration and startup sync.
- Boolean pivot/crosstab headers now say Yes / No / (Blank). Pivot measures use human labels such as Count, Average Amount, Sum of Amount, Distinct Email, Median Amount and 90th percentile of Amount.
- Watch DDL includes owner/date fields and last value/message; existing databases need an owner/date backfill before enabling watches.
- Scalable estimates sample each native aggregate without primary-key IN lists, scale counts using observed sample sizes, retain exact population totals and unscaled ratios, and drill over the full population. Portable estimates use one primary-key boundary and report unscaled prefix counts.
- Target QQQ 4.1.0-RC.6; local verification uses `-Dqqq.snapshot.version=4.1.0-DATAANALYSIS-SNAPSHOT`.
- Empty-string and temporal drills round-trip through core HTTP filter interpretation.
@@ -10,7 +10,7 @@ Twenty-one HTTP operations are available under `/qqq/v1/qbit/data-analysis` (ext
## Requirements
- Java 21; QQQ **4.1.0-RC.6+**.
- Java 21; QQQ **4.1.0-RC.7+**.
- A configured QQQ backend for the qbit tables and authenticated sessions with permissions and security keys.
- Core aggregate and table-feature permission APIs ([qqq#1139](https://github.com/QRun-IO/qqq/pull/1139)) and the UI API extension point ([qqq#1128](https://github.com/QRun-IO/qqq/pull/1128)). Middleware Javalin remains optional; the producer registers endpoints only when it is present.
@@ -187,7 +187,7 @@ Requests and drills reuse core's `QueryFilter` JSON shape and schema reference.
Regenerate deliberately after an API change:
```bash
./mvnw -o -q test -Dtest=DataAnalysisApiTest#contract -Dopenapi.write=true -Dqqq.snapshot.version=4.1.0-DATAANALYSIS-SNAPSHOT
./mvnw -q test -Dtest=DataAnalysisApiTest#contract -Dopenapi.write=true
```
## Pivot backend and field access
@@ -202,7 +202,7 @@ The planner first requests count-ordered top-N groups for every dimension (defau
Hosts can supply `withFieldAnalysisAccessResolver(() -> resolver)`. FULL fields alone appear in capabilities and may be selected. EXACT_MATCH_ONLY and NONE get the same 400 as an unknown field; user-authored filters are passed through for core enforcement. Generated drills using a restricted field (including representative primary keys) are disabled. Related PVS display labels also require FULL access. Exact-run cache identities hash the sorted FULL-field list, the optional core access fingerprint, and related label access.
The default resolver discovers the optional core `fieldaccess.FieldAccessHelper` class and its static methods once. FULL maps to FULL, REVEALABLE to EXACT_MATCH_ONLY, MASKED/HIDDEN to NONE. Without that class, legacy field locks supply FULL/NONE. Replace the reflective calls when the core package ships. Reflection tests use a local stand-in helper rather than shadowing a future real core class at its FQN; the installed snapshot exercises the actual fallback selection. Exact-run persistence continues to support profiles only; pivot results are synchronous.
The default resolver discovers the optional core `fieldaccess.FieldAccessHelper` class and its static methods once. FULL maps to FULL, REVEALABLE to EXACT_MATCH_ONLY, MASKED/HIDDEN to NONE. Without that class, legacy field locks supply FULL/NONE. Replace the reflective calls when the core package ships. Reflection tests use a local stand-in helper rather than shadowing a future real core class at its FQN; the pinned RC.7 exercises the actual core tier resolver selection. Exact-run persistence continues to support profiles only; pivot results are synchronous.
## Backend selection
@@ -304,13 +304,26 @@ The Analyses home uses standard `multiStatistics` and `table` widgets: mine/shar
Backend phases **b–f** are implemented, including saved/shared/starred analyses, schedules, standard widgets, home, export and SQL handoff. The separate profile, pivot and relationship frontend remains outside this repository.
## In-app help
The qbit registers `src/main/resources/help/qbit-data-analysis.yaml` as a QBIT bundle.
Hosts with the core `helpContent` table receive it through QQQ's startup sync; registration
is harmless without that table. The app help follows `appName` when you configure a custom name.
To override text, register your own `layer: HOST` bundle with the same entry key and role
(using your configured app name for app help). HOST entries take precedence over QBIT entries;
QQQ preserves manually edited help during sync.
`DataAnalysisTablesGuardrailTest` uses `HelpBundleAssertions` to check owned metadata,
summary length and live references, and checks widget and possible-value-source coverage.
Update the bundle's literal version alongside the POM version when releasing.
The wrapper pins Maven 3.9.11. On Windows use `mvnw.cmd`. Verification runs Checkstyle, a shared JUnit 5/AssertJ contract suite on memory and H2, PostgreSQL via Testcontainers when Docker is available, and JaCoCo gates of 70% instructions and 90% classes. The suite checks numerical references, every exact count/drill pair, full-population estimate drills, security, guarded cache JSON, native sampling, asynchronous jobs and cache invalidation. The POM targets RC.6; until its release, always supply the locally installed snapshot override shown above. RDBMS, H2, Testcontainers PostgreSQL and the PostgreSQL driver are test-scoped; middleware Javalin is optional.
The wrapper pins Maven 3.9.11. On Windows use `mvnw.cmd`. Verification runs Checkstyle, a shared JUnit 5/AssertJ contract suite on memory and H2, PostgreSQL via Testcontainers when Docker is available, and JaCoCo gates of 70% instructions and 90% classes. The suite checks numerical references, every exact count/drill pair, full-population estimate drills, security, guarded cache JSON, native sampling, asynchronous jobs and cache invalidation. The POM pins QQQ 4.1.0-RC.7 from Maven Central. RDBMS, H2, Testcontainers PostgreSQL and the PostgreSQL driver are test-scoped; middleware Javalin is optional.
# Help for the Analyses app and every data_analysis_* table, in the QQQ help-bundle
# format (qqq#1198). Each table entry says what the table is for and how its rows get
# created, so an empty table explains itself. DataAnalysisHelp applies it until the core
# HelpBundles registry ships in a QQQ release (STAND-IN(core=QRun-IO/qqq#1198, removeBy=4.1.0-RC.8)).
# Summaries stay under 120 characters (DataAnalysisTablesGuardrailTest).
entries:
- key:app:dataAnalysis;slot:help
summary:"Profile, pivot, compare and watch any table, and tune how analysis behaves per table."
content:|
The Setup card on the home lists each part, its state and the next step. Analysis works on every analyzable table from the start with built-in limits, so nothing is required. Open Analysis Settings and run Create rows for analyzable tables to see those limits as rows you can edit per table.
summary:"Review watches associated with this saved analysis."
- key:widget:dataAnalysisHome;slot:label
summary:"See an overview of saved analyses, watches and runs you can access."
- key:widget:dataAnalysisStarred;slot:label
summary:"Open saved analyses you have starred."
- key:widget:dataAnalysisFiring;slot:label
summary:"Review watches that are currently firing."
- key:widget:dataAnalysisRecent;slot:label
summary:"Review your recent analysis runs."
- key:widget:DATA_ANALYSIS_SAVED;slot:label
summary:"View the chart produced by a saved analysis."
- key:app:dataAnalysis;slot:help
summary:"Explore your data, revisit saved analyses and monitor changes with watches."
content:|
Use Analyses to return to saved work, view starred analyses and review watches that are firing. Analyze a table to explore its records, then save the choices you want to revisit. Analysis Settings controls availability and limits for each table. Analysis Runs helps you review exact results and investigate failures.
- key:table:data_analysis_table_config;slot:help
summary:"Control analysis availability, row limits, sampling and result reuse for each table."
content:|
**Howrows get created:** Run **Create rows for analyzable tables** to add the effective settings for tables that do not have a row, or choose New for one table. Existing rows are left unchanged. Without a row, your configured defaults apply.
- key:table:data_analysis_table_config;slot:help
summary:"Per-table analysis limits: on or off, row limits, sampling, smallest group, cache minutes and field lists."
content:|
**Howrows get created:** run the table action Create rows for analyzable tables once. It adds one row per table holding the settings that already apply, so nothing changes until you edit a row. Or add a row with New for just one table. A table without a row uses the defaults from the configuration, so an empty table means every table is on with the defaults. Cache minutes controls how long an exact result is reused (0 turns caching off). Smallest group hides groups with fewer records.
Use Analysis Settings to adjust how a table can be analyzed. You can enable analysis, set row and sample limits, choose allowed or excluded fields, and hide groups smaller than a chosen size. Cache Ttl Minutes controls how long completed exact results can be reused. Tables without an entry use the configured defaults.
summary:"The smallest group of records that may appear in results."
content:|
Min Group Size prevents very small groups from appearing in analysis results. Choose the minimum number of records a group must contain before it can be shown. If expected categories are missing, review this limit along with your filters before assuming that the underlying records are absent.
summary:"How many minutes completed exact results can be reused; zero turns reuse off."
- key:table:data_analysis_saved;slot:help
summary:"Return to analyses you saved with their table, filters, view and chart choices."
content:|
**Howrows get created:** Save your analysis choices to create a saved analysis. An empty list means you have no saved analyses available to you.
- key:table:data_analysis_saved;slot:help
summary:"Analyses people saved: the table, filter and view they want to come back to."
content:|
**Howrows get created:** a person saves an analysis from the Analyze screen on any table. Each saved analysis belongs to its owner; stars, schedules and watches hang off it. Empty means nobody has saved one yet.
Save an analysis from Analyze to keep the table, filters and analysis choices you want to revisit. Each saved analysis has an owner and can be starred for quick access. Open its Schedules and Watches lists to review related deliveries and checks. Sharing the definition does not remove access restrictions on the underlying data.
- key:table:data_analysis_saved;field:name
summary:"The name you use to find this saved analysis."
- key:table:data_analysis_saved;field:description
summary:"A description that helps you understand this entry."
- key:table:data_analysis_saved;field:table_name
summary:"The table whose records you want to analyze."
- key:table:data_analysis_saved;field:filter_json
summary:"The saved record filters, stored as JSON."
- key:table:data_analysis_saved;field:kind
summary:"The type of analysis, such as a profile, pivot or comparison."
summary:"The person who owns and manages this analysis or watch."
- key:table:data_analysis_saved;field:shared
summary:"Whether other permitted people can read the saved analysis definition."
- key:table:data_analysis_saved_star;slot:help
summary:"Keep track of the saved analyses you have starred."
content:|
**Howrows get created:** Star a saved analysis to create your personal star. An empty list means you have no stars available here.
- key:table:data_analysis_saved_star;slot:help
summary:"Which saved analyses each person starred."
content:|
**Howrows get created:** a person stars a saved analysis, and the row is theirs. Starred analyses show on the Analyses home. Empty means nothing is starred yet.
Star a saved analysis to make it easier to find on the Analyses home. Each star links one person to one saved analysis, so your starred list is personal. These entries are created through the star control. An empty list means you have not starred an analysis available here.
summary:"Run saved analyses at chosen times and deliver their results to selected people."
content:|
**Howrows get created:** Add a schedule to a saved analysis to create a delivery definition. An empty list means you have no schedules available here.
- key:table:data_analysis_schedule;slot:help
summary:"Schedules that run a saved analysis daily, weekly or monthly and send the result."
content:|
**Howrows get created:** a person adds a schedule on a saved analysis (the Schedules list on the saved analysis). Schedules run only when the host configures a scheduler for the qbit. Empty means no analysis is scheduled.
Add a schedule to a saved analysis to send results daily, weekly or monthly. Choose the local time, time zone, recipients and output format, then enable it when ready. Automatic runs require scheduling to be configured. Use Send now for an immediate delivery and review Last Status for the outcome.
- key:table:data_analysis_schedule;field:saved_id
summary:"The saved analysis this entry belongs to."
- key:table:data_analysis_schedule;field:cron
summary:"An advanced schedule expression that takes precedence over Frequency, Day and Time."
content:|
Leave Cron blank when you want to schedule deliveries with Frequency, Day and Time. If you supply an advanced expression, it uses the schedule's Time Zone and replaces those calendar choices. You must use a valid Quartz schedule expression. Check Next Run At after saving to confirm the resulting delivery time.
summary:"Choose daily, weekly or monthly delivery."
- key:table:data_analysis_schedule;field:day
summary:"The weekday number or day of the month used by the schedule."
content:|
For a weekly schedule, enter a weekday number from 1 for Monday through 7 for Sunday. For a monthly schedule, enter a day from 1 through 31. If that day does not exist in a month, delivery uses the last day of that month. Daily schedules do not need a day.
- key:table:data_analysis_schedule;field:time
summary:"The local time of day when the analysis should run."
summary:"The people receiving the result, listed by user ID in a JSON array."
content:|
Recipients accepts a list of user IDs, not email addresses. Supply a JSON array containing between one and one hundred nonblank IDs. Duplicate IDs are treated as one recipient. Review the people in the list before you enable the schedule or choose Send now, since those actions deliver the analysis result.
- key:table:data_analysis_schedule;field:enabled
summary:"Whether the schedule is allowed to run automatically."
summary:"Check analysis results for spikes, drops, new categories or other selected conditions."
content:|
**Howrows get created:** Add a watch for a saved analysis or a table and metric. An empty list means you have no watches available here.
- key:table:data_analysis_watch;slot:help
summary:"Watches that alert when a saved analysis spikes, drops, goes out of range or gets a new category."
content:|
**Howrows get created:** a person adds a watch on a saved analysis (the Watches list on the saved analysis). Enabled watches are evaluated every 15 minutes when a scheduler is configured; Evaluate now checks one by hand. A firing watch raises an alert. Empty means nothing is being watched.
Create a watch to check a saved analysis or a table and metric you specify. Choose the detector, its parameters, the date field and time grouping. Enabled watches are checked automatically when scheduling is configured, and you can use Evaluate now to check immediately. Review State and Last Message to understand the result.
- key:table:data_analysis_watch;field:saved_id
summary:"The saved analysis this entry belongs to."
- key:table:data_analysis_watch;field:table_name
summary:"The table whose records you want to analyze."
- key:table:data_analysis_watch;field:filter_json
summary:"The saved record filters, stored as JSON."
- key:table:data_analysis_watch;field:metric_json
summary:"The measurement checked by the watch, stored as JSON."
- key:table:data_analysis_watch;field:detector
summary:"The condition to detect, such as a spike, drop or new category."
summary:"The latest explanation of the watch result."
- key:table:data_analysis_watch;field:state
summary:"Whether the watch is currently OK or firing."
- key:table:data_analysis_watch;field:alert_key
summary:"The identifier used to associate the watch with its alert."
- key:table:data_analysis_run;slot:help
summary:"Review exact analysis runs, their results, timing and any errors."
content:|
**Howrows get created:** Start an exact analysis to record its execution and reusable result. An empty list means you have no recorded runs available here.
- key:table:data_analysis_run;slot:help
summary:"Each exact analysisrun: who, what, when and the stored result that is reused as a cache."
content:|
**Howrows get created:** automatically when someone starts an exact analysis. A finished run is reused for the table's cache minutes (Analysis Settings). Empty means nobody has run an exact analysis yet.
Analysis Runs records exact analyses as they run. You can review who requested an analysis, its table and operation, the number of rows scanned and its status. Completed results can be reused for the time allowed in Analysis Settings. If a run fails, read Error before trying the analysis again.
- key:table:data_analysis_run;field:table_name
summary:"The table whose records you want to analyze."
- key:table:data_analysis_run;field:operation
summary:"The analysis operation performed by the run."
- key:table:data_analysis_run;field:request_hash
summary:"A fingerprint of the analysis request used to match reusable results."
- key:table:data_analysis_run;field:filter_hash
summary:"A fingerprint of the filters used to match reusable results."
- key:table:data_analysis_run;field:data_version
summary:"The data version associated with the stored result."
- key:table:data_analysis_run;field:status
summary:"Whether the analysis is running, complete or failed."
- key:table:data_analysis_run;field:started_at
summary:"When the analysis run started."
- key:table:data_analysis_run;field:finished_at
summary:"When the analysis run finished."
- key:table:data_analysis_run;field:rows_scanned
summary:"The number of rows examined by the analysis."
- key:table:data_analysis_run;field:sampled
summary:"Whether the result was calculated from a sample of rows."
- key:table:data_analysis_run;field:result_json
summary:"The stored analysis result in JSON form."
summary:"Run selected analysis schedules and send their results now."
content:|
Use Send now on selected Schedules when you need a delivery without waiting for the next scheduled time. Review the schedules before continuing. The action uses each schedule’s saved analysis, recipients and format. Check the result and Last Status afterward to confirm which deliveries completed or failed.
summary:"Run analysis schedules whose next delivery time is due."
content:|
Run analysis schedules is the background action that checks for scheduled deliveries. Configure and enable schedules on a saved analysis to use automatic delivery. You can inspect Next Run At, Last Run At and Last Status on Schedules to understand timing and outcomes. Automatic execution requires a configured scheduler.
summary:"Check selected watches now and update their results."
content:|
Use Evaluate now to check selected Watches without waiting for the next automatic check. Review the selection before continuing. After the action, inspect State, Last Value and Last Message to understand each result. A watch that detects its configured condition can raise an alert with the selected severity.
summary:"Include selected watches in automatic checks."
content:|
Use Enable when selected Watches are ready for automatic checking. Review the detector, parameters, date field and time grouping before enabling them. The action checks your access to each watch and reports the outcome. Automatic checks require a configured scheduler; use Evaluate now when you need an immediate result.
summary:"Exclude selected watches from automatic checks."
content:|
Use Disable when you want selected Watches to stop participating in automatic checks. Review the selection before continuing and read the result for each watch. The watch definition remains available for inspection and later use. Use Enable when you are ready to include it in automatic checks again.
summary:"Check enabled watches and update their alert conditions."
content:|
Evaluate watches is the background action that checks enabled Watches. With scheduling configured, it runs every fifteen minutes. You can review Last Evaluated At, State and Last Message to see what happened. Use Evaluate now on a watch when you need to check its condition before the next scheduled run.
Use **{{label:data_analysis_table_config.bulkInsert}}** to add records to {{label:data_analysis_table_config}} from a file. Match your file columns to the available fields and review the input before continuing. Check the result for validation errors and records you need to correct.
summary:"Apply changes to multiple selected records."
content:|
Use **{{label:data_analysis_table_config.bulkEdit}}** to change fields on your selected records in {{label:data_analysis_table_config}}. Review your selection and the values you want to change before continuing. Check the result for records that could not be updated.
summary:"Update existing records using values from a file."
content:|
Use **{{label:data_analysis_table_config.bulkEditWithFile}}** to update records in {{label:data_analysis_table_config}}. Match your file columns to fields and identify the existing records to update. Review the input before continuing and check the result for errors.
Use **{{label:data_analysis_table_config.bulkDelete}}** to remove selected records from {{label:data_analysis_table_config}}. Review your selection carefully before confirming deletion. Check the result for records that could not be deleted.
Use **{{label:data_analysis_saved.bulkInsert}}** to add records to {{label:data_analysis_saved}} from a file. Match your file columns to the available fields and review the input before continuing. Check the result for validation errors and records you need to correct.
summary:"Apply changes to multiple selected records."
content:|
Use **{{label:data_analysis_saved.bulkEdit}}** to change fields on your selected records in {{label:data_analysis_saved}}. Review your selection and the values you want to change before continuing. Check the result for records that could not be updated.
summary:"Update existing records using values from a file."
content:|
Use **{{label:data_analysis_saved.bulkEditWithFile}}** to update records in {{label:data_analysis_saved}}. Match your file columns to fields and identify the existing records to update. Review the input before continuing and check the result for errors.
Use **{{label:data_analysis_saved.bulkDelete}}** to remove selected records from {{label:data_analysis_saved}}. Review your selection carefully before confirming deletion. Check the result for records that could not be deleted.
Use **{{label:data_analysis_saved_star.bulkInsert}}** to add records to {{label:data_analysis_saved_star}} from a file. Match your file columns to the available fields and review the input before continuing. Check the result for validation errors and records you need to correct.
summary:"Apply changes to multiple selected records."
content:|
Use **{{label:data_analysis_saved_star.bulkEdit}}** to change fields on your selected records in {{label:data_analysis_saved_star}}. Review your selection and the values you want to change before continuing. Check the result for records that could not be updated.
summary:"Update existing records using values from a file."
content:|
Use **{{label:data_analysis_saved_star.bulkEditWithFile}}** to update records in {{label:data_analysis_saved_star}}. Match your file columns to fields and identify the existing records to update. Review the input before continuing and check the result for errors.
Use **{{label:data_analysis_saved_star.bulkDelete}}** to remove selected records from {{label:data_analysis_saved_star}}. Review your selection carefully before confirming deletion. Check the result for records that could not be deleted.
Use **{{label:data_analysis_schedule.bulkInsert}}** to add records to {{label:data_analysis_schedule}} from a file. Match your file columns to the available fields and review the input before continuing. Check the result for validation errors and records you need to correct.
summary:"Apply changes to multiple selected records."
content:|
Use **{{label:data_analysis_schedule.bulkEdit}}** to change fields on your selected records in {{label:data_analysis_schedule}}. Review your selection and the values you want to change before continuing. Check the result for records that could not be updated.
summary:"Update existing records using values from a file."
content:|
Use **{{label:data_analysis_schedule.bulkEditWithFile}}** to update records in {{label:data_analysis_schedule}}. Match your file columns to fields and identify the existing records to update. Review the input before continuing and check the result for errors.
Use **{{label:data_analysis_schedule.bulkDelete}}** to remove selected records from {{label:data_analysis_schedule}}. Review your selection carefully before confirming deletion. Check the result for records that could not be deleted.
Use **{{label:data_analysis_watch.bulkInsert}}** to add records to {{label:data_analysis_watch}} from a file. Match your file columns to the available fields and review the input before continuing. Check the result for validation errors and records you need to correct.
summary:"Apply changes to multiple selected records."
content:|
Use **{{label:data_analysis_watch.bulkEdit}}** to change fields on your selected records in {{label:data_analysis_watch}}. Review your selection and the values you want to change before continuing. Check the result for records that could not be updated.
summary:"Update existing records using values from a file."
content:|
Use **{{label:data_analysis_watch.bulkEditWithFile}}** to update records in {{label:data_analysis_watch}}. Match your file columns to fields and identify the existing records to update. Review the input before continuing and check the result for errors.
Use **{{label:data_analysis_watch.bulkDelete}}** to remove selected records from {{label:data_analysis_watch}}. Review your selection carefully before confirming deletion. Check the result for records that could not be deleted.
Use **{{label:data_analysis_run.bulkInsert}}** to add records to {{label:data_analysis_run}} from a file. Match your file columns to the available fields and review the input before continuing. Check the result for validation errors and records you need to correct.
summary:"Apply changes to multiple selected records."
content:|
Use **{{label:data_analysis_run.bulkEdit}}** to change fields on your selected records in {{label:data_analysis_run}}. Review your selection and the values you want to change before continuing. Check the result for records that could not be updated.
summary:"Update existing records using values from a file."
content:|
Use **{{label:data_analysis_run.bulkEditWithFile}}** to update records in {{label:data_analysis_run}}. Match your file columns to fields and identify the existing records to update. Review the input before continuing and check the result for errors.
Use **{{label:data_analysis_run.bulkDelete}}** to remove selected records from {{label:data_analysis_run}}. Review your selection carefully before confirming deletion. Check the result for records that could not be deleted.
content:"Use your host's saved analysis workflow."
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
james.maesandClaude Opus 5.5