chore(api): regenerate the OpenAPI fragment against QQQ 4.1.0-RC.6

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

CHANGELOG.md

@@ -6,6 +6,19 @@ Following [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and [Semantic
 
 ### Added
 
+- `data-analysis` UI API extension 1.0.0: seven session-bound profile, capability, distinct-value, duplicate and run endpoints, typed schemas and a checked OpenAPI fragment.
+- Core table-feature permission registration and capability-directed aggregate pushdown, with native sampling and portable streamed fallbacks.
+- HTTP authorization, drill-to-core-count round trips, async polling, served OpenAPI and assistant discovery tests; H2 streamed and native-compatibility contract variants.
+
+### Changed
+
+- Scalable estimates sample each native aggregate without primary-key IN lists, scale counts using observed sample sizes, retain exact population totals and unscaled ratios, and drill over the full population. Portable estimates use one primary-key boundary and report unscaled prefix counts.
+
+- Target QQQ 4.1.0-RC.6; local verification uses `-Dqqq.snapshot.version=4.1.0-DATAANALYSIS-SNAPSHOT`.
+- Empty-string and temporal drills round-trip through core HTTP filter interpretation.
+
+### Previously added
+
 - Phase b host engine: typed column/table profiles, distinct pages, duplicates, count-equivalent drills, chart data, candidate keys and completeness heatmaps.
 - Internal `AnalysisBackend` port and RC.5 stand-in with aggregate pushdown, bounded query pages, deterministic prefix estimates and honest truncation metadata.
 - Personalized field/table authorization, enforced FK label lookup, runtime configuration overrides and minimum-group-size privacy before caching.

README.md

@@ -6,13 +6,13 @@ Statistical analysis and data dives on any QQQ table, filter or saved view: prof
 
 The host API provides column and table profiles, distinct-value pages, duplicate groups, drill-through filters, effective per-table configuration, privacy suppression, and asynchronous exact runs with a persistent per-user cache. Metadata, six visible tables, owner locks, PostgreSQL DDL generation and the Analyses home are also available.
 
-HTTP endpoints require the QQQ RC.6 extension point and are next, along with the Analyze UI. Pivots, comparisons, anomalies, saved-analysis execution, scheduling and exports remain later phases.
+Seven HTTP operations are available under `/qqq/v1/qbit/data-analysis`; the Analyze UI is a later phase. Pivots, comparisons, anomalies, saved-analysis execution, scheduling and exports remain later phases.
 
 ## Requirements
 
-- Java 21; QQQ **4.1.0-RC.5+**.
+- Java 21; QQQ **4.1.0-RC.6+**.
 - A configured QQQ backend for the qbit tables and authenticated sessions with permissions and security keys.
-- The engine works on RC.5 through an internal stand-in. A future `CoreAnalysisBackend` can replace it when the core aggregate extensions ship. HTTP endpoints require the UI API extension point ([qqq#1128](https://github.com/QRun-IO/qqq/pull/1128), RC.6).
+- Core aggregate and table-feature permission APIs ([qqq#1139](https://github.com/QRun-IO/qqq/pull/1139)) and the UI API extension point ([qqq#1128](https://github.com/QRun-IO/qqq/pull/1128)). Middleware Javalin remains optional; the producer registers endpoints only when it is present.
 
 ## Install
 
@@ -58,7 +58,7 @@ Only Saved Analyses, Watches, Analysis Settings and Analysis Runs appear in navi
 | `data_analysis.admin` | The all-access key of the `data_analysis_user` record locks (see Security) |
 | `<effective table permission base>.analyze` | Analyze a source table, also requiring its READ permission |
 
-`DataAnalysisPermissions.analyzePermissionName(table)` honors a custom permission base name and instance defaults. `canAnalyze(table)` is the legacy helper; its READ check follows the instance setting. The engine adds an explicit READ check and always uses `PermissionEnforcement.ENFORCE`, even when the instance default is off. The feature check is a stand-in for core `TableFeaturePermissionsMetaData` (§3.6); hosts must grant these analyze names explicitly until that hook registers them in the catalog. All qbit tables use standard permission rules (no custom checker), so their CRUD permissions appear in the core permission catalog and the qbit-authentik policy editor. Hosts explicitly grant source-table analyze permissions until the core feature hook ships.
+`DataAnalysisPermissions.analyzePermissionName(table)` honors the effective table permission base. `canAnalyze(table)` delegates to core's table-feature permission check. The producer registers `analyze`, labelled **Analyze**, on configured source tables, excluding this qbit's own tables. No CRUD permission implies analyze. Core checks READ as well as the feature grant on protected tables. All analysis queries enforce permissions even when the instance default is off. Qbit CRUD permissions also appear in the core catalog.
 
 ## Security
 
@@ -102,7 +102,7 @@ Cached results are scoped to the authenticated user, normalized request, effecti
 
 All properties have fluent `withX` setters. Global numeric defaults must be non-null and non-negative. Nullable per-table numeric overrides inherit global defaults; configured values must be non-negative. `TableAnalysisDefaults` also supports enabled, analyzable fields and excluded fields. Runtime `data_analysis_table_config` values override per-table code defaults, which override global defaults. Null row values inherit; blank field lists mean all readable fields or no exclusions. Include/exclude table lists remain an outer restriction, even if a runtime row enables the table. Only the config-row lookup uses a temporary system session, restored in `finally`. It selects configuration fields only and is memoized for 60 seconds. Settings editors can call `service.getConfigResolver().invalidate()` for immediate refresh.
 
-`max_rows` bounds the stand-in's selected population and retained records. An exact request disables sampling but still respects this bound and reports `truncated: true` when necessary; increase the limit to obtain a complete larger profile. Zero selects no rows. Above `sample_threshold_rows`, estimates use the first `min(sample_rows, max_rows)` rows ordered by primary key. Counts describe that prefix, are not extrapolated, and carry `estimate`, `sampleRows`, `rowsScanned`, warnings and an `asOf` timestamp. This is deterministic prefix sampling, not a random sample. `cache_ttl_minutes: 0` disables cache reuse.
+`max_rows` bounds the stand-in's selected population and retained records. An exact request disables sampling but still respects this bound and reports `truncated: true` when necessary; increase the limit to obtain a complete larger profile. Zero selects no rows in the stand-in. Above `sample_threshold_rows`, capable backends run each aggregate with `samplePercent = min(100, sample_rows * 100 / total)` on the user's filter. Counts, top values and histograms are scaled by the full population divided by the observed sample count for that call; ratios and descriptive statistics remain in sample units. The `rows` metric always reports the unsampled total for estimates. Other metrics carry `estimate: true` and an `≈ ` formatted prefix. Sampling never materializes a primary-key IN filter. Backends without sampling use one `pk <= boundary` criterion to compute an unscaled first-N prefix estimate. Estimate drills combine the original filter with the metric condition over the full population, so count/drill equivalence applies only to exact results. Warnings distinguish random samples from prefixes; `sampleRows`, `rowsScanned` and `asOf` describe coverage. `cache_ttl_minutes: 0` disables cache reuse.
 
 ## Host API
 
@@ -136,9 +136,37 @@ The `result` package contains Serializable, Jackson-friendly request/result POJO
 
 Count drills combine an untouched copy of the request filter with their producing condition. Cardinality drills select one representative record per distinct non-null value; the privacy guard removes those representative drills. Truncated profiles retain their primary-key boundary in drills, so their counts remain reproducible. Serialized drill getters restore temporal operand types from the current instance metadata before core evaluates them.
 
-## RC.5 stand-in
+## HTTP endpoints
 
-`engine.AnalysisBackend` isolates profilers from missing core operators. `StandInAnalysisBackend` pushes down COUNT, COUNT_DISTINCT, MIN, MAX, AVG, SUM, grouped counts, aggregate ordering and limits through core actions. RC.5 has no aggregate OFFSET, so pages over-fetch through the requested end and slice locally. Memory grouping pagination and scale-sensitive decimal equality have isolated compatibility handling.
+Extension `data-analysis` version **1.0.0**, prefix `/qbit/data-analysis` inside `/qqq/v1`:
+
+| Method | Path after `/qqq/v1` | Body or purpose |
+| --- | --- | --- |
+| GET | `/qbit/data-analysis/tables/{table}/capabilities` | Availability, fields and charts, limits, backend capabilities, SQL Workbench presence |
+| POST | `/qbit/data-analysis/tables/{table}/column-profile` | `{fieldName, filter, exact}` |
+| POST | `/qbit/data-analysis/tables/{table}/distinct-values` | `{fieldName, filter, page, pageSize, search}` |
+| POST | `/qbit/data-analysis/tables/{table}/table-profile` | `{filter, fields}` |
+| POST | `/qbit/data-analysis/tables/{table}/duplicates` | `{fields, filter, limit}`; default limit 50, maximum 500 |
+| POST | `/qbit/data-analysis/runs` | `{operation: COLUMN_PROFILE or TABLE_PROFILE, tableName, request}` returns `{runId, status}` |
+| GET | `/qbit/data-analysis/runs/{runId}` | Status, timestamps, error, typed `columnProfile` or `tableProfile` |
+
+Every operation uses the request session through `DataAnalysisService`. Capabilities returns 200 with `canAnalyze: false` and no fields when denied. Other permission denials return 403, bad bodies or unavailable fields return 400, and unknown tables return 404. Core middleware supplies plain messages in its standard `{error: message}` JSON envelope. Run rows remain owner-locked.
+
+Requests and drills reuse core's `QueryFilter` JSON shape and schema reference. Versioned `DataAnalysis...V1` schemas cover profiles, metrics, drills, charts, series, points, field summaries, distinct and duplicate results, capabilities and runs. The [OpenAPI fragment](src/main/resources/openapi/qqq-v1/data-analysis.yaml) is checked against regeneration. The deployed `/qqq/v1/openapi.json` and `UiApiInvoker.listOperations()` include all seven operations.
+
+Regenerate deliberately after an API change:
+
+```bash
+./mvnw -o -q test -Dtest=DataAnalysisApiTest#contract -Dopenapi.write=true -Dqqq.snapshot.version=4.1.0-DATAANALYSIS-SNAPSHOT
+```
+
+## Backend selection
+
+`DataAnalysisService` selects `CoreAnalysisBackend` when the source backend advertises aggregate capabilities, including memory. Supported standard deviation, percentiles, null/blank counts, numeric/date buckets and length statistics execute through core aggregates as the current session. Unsupported primitives delegate to `StandInAnalysisBackend`. Patterns and derived length histograms remain streamed; representative distinct drills and cyclic-date membership use bounded scans.
+
+The installed snapshot requires H2 PostgreSQL compatibility mode for native bucket SQL (`GROUP BY` positions).
+
+`engine.AnalysisBackend` isolates profilers from backend-specific operators. `StandInAnalysisBackend` pushes down COUNT, COUNT_DISTINCT, MIN, MAX, AVG, SUM, grouped counts, aggregate ordering and limits through core actions. RC.5 has no aggregate OFFSET, so pages over-fetch through the requested end and slice locally. Memory grouping pagination and scale-sensitive decimal equality have isolated compatibility handling.
 
 Variance, interpolated percentiles, string lengths/shapes, date classification, whitespace detection and representative-row drills use bounded primary-key query pages. Histogram/date membership counts use enforced range/count queries after their boundaries have been calculated. Numeric histograms have 20 interior bins between p1 and p99 plus below/above bins, and the last interior bin is closed. Profiles include Tukey outliers, date gaps, date-time weekday/hour patterns, chart alternatives, a seven-bucket moving average, table completeness and candidate keys (single fields and pairs among the eight most unique).
 
@@ -152,15 +180,15 @@ Data versions use filtered COUNT plus MAX of a readable modification-date-like f
 4. Review per-table include/exclude sets, limits and minimum group sizes before enabling analysis. Integrate Saved Analyses and Watches with host ESB/Quick Search when those optional host features are installed; this scaffold does not add those dependencies.
 5. Use the Analyses home as the owner of analysis data. Other dashboards should link there until saved-analysis widgets are implemented.
 
-Phase **b** profiles, drill-through and runs/cache are implemented through the host facade. Remaining work: **a** core aggregate extensions; **b** HTTP endpoints after QQQ RC.6 and analysis UI; **c** pivots; **d** comparisons and relationships; **e** watch evaluation and alerts; **f** saved-analysis execution, sharing, stars integration, schedules, chart widgets, full home and exports. The current home summarizes stored rows; the host facade executes profiles. The home does not send deliveries.
+Phase **b** profiles, drill-through, runs/cache and HTTP endpoints are implemented. Remaining work: **b** analysis UI; **c** pivots; **d** comparisons and relationships; **e** watch evaluation and alerts; **f** saved-analysis execution, sharing, stars integration, schedules, chart widgets, full home and exports. The current home summarizes stored rows; the host facade executes profiles. The home does not send deliveries.
 
 ## Build
 
 ```bash
-./mvnw verify
+./mvnw -o -q verify -Dqqq.snapshot.version=4.1.0-DATAANALYSIS-SNAPSHOT
 ```
 
-The wrapper pins Maven 3.9.11. On Windows use `mvnw.cmd`. Verification runs Checkstyle, a shared JUnit 5/AssertJ contract suite on memory and H2, PostgreSQL via Testcontainers when Docker is available, and JaCoCo gates of 70% instructions and 90% classes. The suite checks numerical references, every emitted count/drill pair, security, guarded cache JSON, prefix sampling, asynchronous jobs and cache invalidation. For a populated local dependency cache use `./mvnw -o -q verify`. RDBMS, H2, Testcontainers PostgreSQL and the PostgreSQL driver are test-scoped; middleware Javalin is optional.
+The wrapper pins Maven 3.9.11. On Windows use `mvnw.cmd`. Verification runs Checkstyle, a shared JUnit 5/AssertJ contract suite on memory and H2, PostgreSQL via Testcontainers when Docker is available, and JaCoCo gates of 70% instructions and 90% classes. The suite checks numerical references, every exact count/drill pair, full-population estimate drills, security, guarded cache JSON, native sampling, asynchronous jobs and cache invalidation. The POM targets RC.6; until its release, always supply the locally installed snapshot override shown above. RDBMS, H2, Testcontainers PostgreSQL and the PostgreSQL driver are test-scoped; middleware Javalin is optional.
 
 ## License
 

pom.xml

@@ -215,7 +215,7 @@
             <activeByDefault>true</activeByDefault>
          </activation>
          <properties>
-            <qqq.snapshot.version>4.1.0-RC.5</qqq.snapshot.version>
+            <qqq.snapshot.version>4.1.0-RC.6</qqq.snapshot.version>
          </properties>
          <dependencyManagement>
             <dependencies>

src/main/java/com/kingsrook/qbits/dataanalysis/DataAnalysisQBitProducer.java

@@ -21,9 +21,12 @@
 package com.kingsrook.qbits.dataanalysis;
 
 
+import com.kingsrook.qbits.dataanalysis.api.DataAnalysisApi;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerMultiOutput;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.permissions.TableFeaturePermission;
+import com.kingsrook.qqq.backend.core.model.metadata.permissions.TableFeaturePermissionsMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.qbits.QBitMetaDataProducer;
 
@@ -129,6 +132,22 @@ public class DataAnalysisQBitProducer implements QBitMetaDataProducer<DataAnalys
       MetaDataProducerMultiOutput output = QBitMetaDataProducer.super.produce(instance);
       output.addSelfToInstance(instance);
       new DataAnalysisRuntime(qBitConfig).addSelfToInstance(instance);
+      var included = java.util.Set.copyOf(qBitConfig.getIncludeTables());
+      var excluded = java.util.Set.copyOf(qBitConfig.getExcludeTables());
+      TableFeaturePermissionsMetaData.ofOrWithNew(instance).withPermission(new TableFeaturePermission()
+         .withSuffix("analyze").withLabel("Analyze").withImpliedBy(null)
+         .withAppliesTo((qInstance, table) -> !table.getName().startsWith(DataAnalysisQBitConfig.TABLE_NAME_PREFIX)
+            && (included.isEmpty() || included.contains(table.getName())) && !excluded.contains(table.getName())));
+      try
+      {
+         Class.forName("io.javalin.Javalin");
+         Class.forName("com.kingsrook.qqq.middleware.javalin.QJavalinImplementation");
+         DataAnalysisApi.register(instance);
+      }
+      catch(ClassNotFoundException expected)
+      {
+         // The engine and metadata also work without the optional middleware.
+      }
       if(qBitConfig.getParentAppName() != null && instance.getApp(qBitConfig.getParentAppName()) != null)
       {
          instance.getApp(qBitConfig.getParentAppName()).withChild(instance.getApp(qBitConfig.getAppName()));

src/main/java/com/kingsrook/qbits/dataanalysis/DataAnalysisRuntime.java

@@ -25,7 +25,7 @@ import com.kingsrook.qqq.backend.core.context.QContext;
 import com.kingsrook.qqq.backend.core.model.metadata.QSupplementalInstanceMetaData;
 
 /*******************************************************************************
- ** Instance-scoped service holder for future endpoints and host integrations.
+ ** Instance-scoped service holder for UI API endpoints and host integrations.
  *******************************************************************************/
 public class DataAnalysisRuntime implements QSupplementalInstanceMetaData
 {

src/main/java/com/kingsrook/qbits/dataanalysis/DataAnalysisService.java

@@ -25,6 +25,7 @@ import java.util.concurrent.Executor;
 import com.kingsrook.qbits.dataanalysis.engine.AnalysisBackend;
 import com.kingsrook.qbits.dataanalysis.engine.AnalysisSecurity;
 import com.kingsrook.qbits.dataanalysis.engine.ColumnProfiler;
+import com.kingsrook.qbits.dataanalysis.engine.CoreAnalysisBackend;
 import com.kingsrook.qbits.dataanalysis.engine.DistinctValues;
 import com.kingsrook.qbits.dataanalysis.engine.DuplicateFinder;
 import com.kingsrook.qbits.dataanalysis.engine.EffectiveConfigResolver;
@@ -82,7 +83,7 @@ public class DataAnalysisService
    public Capabilities capabilities(String table) throws QException
    {
       return (new Capabilities().withTableName(table).withFields(security.analyzableFields(table).stream().map(QFieldMetaData::getName).toList())
-         .withConfig(configs.resolve(table)).withBackend("RC.5 stand-in; deterministic primary-key prefix sampling"));
+         .withConfig(configs.resolve(table)).withBackend(CoreAnalysisBackend.capabilities(table).isEmpty() ? "Streamed analysis" : "Core aggregate pushdown"));
    }
 
 
@@ -163,11 +164,11 @@ public class DataAnalysisService
 
 
    /*******************************************************************************
-    ** Fresh per-request backend, ready for a future core-backed implementation.
+    ** Select core pushdown when the backend advertises analysis capabilities.
     *******************************************************************************/
    public AnalysisBackend backend(String table) throws QException
    {
-      return (new StandInAnalysisBackend(configs.resolve(table)));
+      return (CoreAnalysisBackend.capabilities(table).isEmpty() ? new StandInAnalysisBackend(configs.resolve(table)) : new CoreAnalysisBackend(configs.resolve(table)));
    }
 
 

src/main/java/com/kingsrook/qbits/dataanalysis/api/DataAnalysisApi.java

@@ -0,0 +1,507 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.dataanalysis.api;
+
+import java.util.List;
+import java.util.Map;
+import com.kingsrook.qbits.dataanalysis.DataAnalysisRuntime;
+import com.kingsrook.qbits.dataanalysis.DataAnalysisService;
+import com.kingsrook.qbits.dataanalysis.engine.CoreAnalysisBackend;
+import com.kingsrook.qbits.dataanalysis.result.AnalysisRequest;
+import com.kingsrook.qbits.dataanalysis.result.ChartKind;
+import com.kingsrook.qbits.dataanalysis.result.ColumnProfileRequest;
+import com.kingsrook.qbits.dataanalysis.result.DistinctValuesRequest;
+import com.kingsrook.qbits.dataanalysis.result.DuplicatesRequest;
+import com.kingsrook.qbits.dataanalysis.result.TableProfileRequest;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.exceptions.QBadRequestException;
+import com.kingsrook.qqq.backend.core.exceptions.QNotFoundException;
+import com.kingsrook.qqq.backend.core.exceptions.QPermissionDeniedException;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+import com.kingsrook.qqq.backend.core.utils.JsonUtils;
+import com.kingsrook.qqq.middleware.javalin.specs.BasicOperation;
+import com.kingsrook.qqq.middleware.javalin.specs.extensions.AbstractJsonEndpointSpec;
+import com.kingsrook.qqq.middleware.javalin.specs.extensions.MiddlewareSpecExtension;
+import com.kingsrook.qqq.middleware.javalin.specs.extensions.MiddlewareSpecExtensionsMetaData;
+import com.kingsrook.qqq.openapi.model.HttpMethod;
+import com.kingsrook.qqq.openapi.model.In;
+import com.kingsrook.qqq.openapi.model.Parameter;
+import com.kingsrook.qqq.openapi.model.Schema;
+import com.kingsrook.qqq.openapi.model.Type;
+
+/*******************************************************************************
+ ** Session-bound profile operations registered only with optional Javalin present.
+ *******************************************************************************/
+public class DataAnalysisApi
+{
+   public static final String PREFIX = "/qbit/data-analysis";
+
+   /*******************************************************************************
+    ** Publish the extension through the core spec registry.
+    *******************************************************************************/
+   public static void register(QInstance instance)
+   {
+      MiddlewareSpecExtensionsMetaData.ofOrWithNew(instance).withExtension(extension());
+   }
+
+   /*******************************************************************************
+    ** The same descriptor drives routes, OpenAPI and assistant tool discovery.
+    *******************************************************************************/
+   public static MiddlewareSpecExtension extension()
+   {
+      return (new MiddlewareSpecExtension().withName("data-analysis").withSemanticVersion("1.0.0")
+         .withDescription("Profiles, distinct values, duplicate groups and exact analysis runs as the signed-in user.")
+         .withSpec(CapabilitiesSpec.class).withSpec(ColumnProfileSpec.class).withSpec(DistinctValuesSpec.class)
+         .withSpec(TableProfileSpec.class).withSpec(DuplicatesSpec.class).withSpec(StartRunSpec.class).withSpec(GetRunSpec.class));
+   }
+
+   /*******************************************************************************
+    ** Common schema catalog and path parameters for the seven JSON specs.
+    *******************************************************************************/
+   public abstract static class ProfileSpec extends AbstractJsonEndpointSpec
+   {
+      private final String path;
+      private final HttpMethod method;
+      private final String summary;
+      private final String description;
+      private final String request;
+      private final String response;
+
+      /*******************************************************************************
+       ** Bind immutable operation metadata.
+       *******************************************************************************/
+      protected ProfileSpec(String path, HttpMethod method, String summary, String description, String request, String response)
+      {
+         this.path = path;
+         this.method = method;
+         this.summary = summary;
+         this.description = description;
+         this.request = request;
+         this.response = response;
+      }
+
+      /*******************************************************************************
+       ** Assistant-facing operation purpose.
+       *******************************************************************************/
+      @Override
+      public BasicOperation defineBasicOperation()
+      {
+         return (new BasicOperation().withPath(PREFIX + path).withHttpMethod(method).withShortSummary(summary).withLongDescription(description));
+      }
+
+      /*******************************************************************************
+       ** Required path parameter, when present.
+       *******************************************************************************/
+      @Override
+      public List<Parameter> defineRequestParameters()
+      {
+         String name = path.contains("{table}") ? "table" : path.contains("{runId}") ? "runId" : null;
+         return (name == null ? List.of() : List.of(new Parameter().withName(name).withIn(In.PATH).withRequired(true)
+            .withDescription(name.equals("table") ? "QQQ table name to analyze." : "Owner-scoped analysis run id.").withSchema(new Schema().withType(Type.STRING))));
+      }
+
+      /*******************************************************************************
+       ** Versioned request schema.
+       *******************************************************************************/
+      @Override
+      protected Schema defineRequestBodySchema()
+      {
+         return (request == null ? null : new Schema().withRef("#/components/schemas/DataAnalysis" + request + "V1"));
+      }
+
+      /*******************************************************************************
+       ** Versioned response schema.
+       *******************************************************************************/
+      @Override
+      protected String getResponseSchemaName()
+      {
+         return ("DataAnalysis" + response + "V1");
+      }
+
+      /*******************************************************************************
+       ** Include all referenced transport models and the shared core filter.
+       *******************************************************************************/
+      @Override
+      public Map<String, Schema> defineComponentSchemas()
+      {
+         return (this instanceof CapabilitiesSpec ? DataAnalysisSchemas.components() : Map.of());
+      }
+   }
+
+   /*******************************************************************************
+    ** Existing table lookup before invoking permission-sensitive operations.
+    *******************************************************************************/
+   private static QTableMetaData table(String name) throws QNotFoundException
+   {
+      QTableMetaData table = QContext.getQInstance().getTable(name);
+      if(table == null)
+      {
+         throw new QNotFoundException("Unknown analysis table.");
+      }
+      return (table);
+   }
+
+   /*******************************************************************************
+    ** Parse one JSON object with strict scalar/list types and core filter JSON.
+    *******************************************************************************/
+   private static AnalysisRequest request(String operation, String table, Map<String, Object> body) throws Exception
+   {
+      table(table);
+      DataAnalysisService service = DataAnalysisRuntime.service();
+      List<String> readable = service.capabilities(table).getFields();
+      try
+      {
+         for(String name : List.of("fieldName", "search"))
+         {
+            if(body.get(name) != null && !(body.get(name) instanceof String))
+            {
+               throw new IllegalArgumentException(name + " must be a string.");
+            }
+         }
+         if(body.get("exact") != null && !(body.get("exact") instanceof Boolean))
+         {
+            throw new IllegalArgumentException("exact must be a boolean.");
+         }
+         QQueryFilter filter = body.get("filter") == null ? null : JsonUtils.toObject(JsonUtils.toJson(body.get("filter")), QQueryFilter.class);
+         validateFilter(filter, readable);
+         List<String> fields = null;
+         if(body.get("fields") != null)
+         {
+            if(!(body.get("fields") instanceof List<?> values) || values.stream().anyMatch(v -> !(v instanceof String)))
+            {
+               throw new IllegalArgumentException("fields must be an array of field names.");
+            }
+            fields = ((List<?>) body.get("fields")).stream().map(String.class::cast).toList();
+            if(fields.isEmpty() || !readable.containsAll(fields))
+            {
+               throw new IllegalArgumentException("Choose readable analysis fields.");
+            }
+         }
+         String field = (String) body.get("fieldName");
+         if((operation.equals("COLUMN_PROFILE") || operation.equals("DISTINCT_VALUES")) && !readable.contains(field))
+         {
+            throw new IllegalArgumentException("Choose a readable analysis field.");
+         }
+         boolean exact = Boolean.TRUE.equals(body.get("exact"));
+         return (switch(operation)
+         {
+            case "COLUMN_PROFILE" -> new ColumnProfileRequest().withTableName(table).withFilter(filter).withFieldName(field).withExact(exact);
+            case "TABLE_PROFILE" -> new TableProfileRequest().withTableName(table).withFilter(filter).withFields(fields);
+            case "DISTINCT_VALUES" -> new DistinctValuesRequest().withTableName(table).withFilter(filter).withFieldName(field)
+               .withPage(integer(body, "page", 0, 0, 1_000_000)).withPageSize(integer(body, "pageSize", 100, 1, 500)).withSearch((String) body.get("search"));
+            case "DUPLICATES" -> new DuplicatesRequest().withTableName(table).withFilter(filter)
+               .withFields(java.util.Objects.requireNonNull(fields, "Choose duplicate fields.")).withLimit(integer(body, "limit", 50, 1, 500));
+            default -> throw new IllegalArgumentException("operation must be COLUMN_PROFILE or TABLE_PROFILE.");
+         });
+      }
+      catch(Exception e)
+      {
+         throw new QBadRequestException("Invalid analysis request: " + e.getMessage());
+      }
+   }
+
+   /*******************************************************************************
+    ** Bad filter fields fail before any aggregate or cached result is consulted.
+    *******************************************************************************/
+   private static void validateFilter(QQueryFilter filter, List<String> readable)
+   {
+      if(filter == null)
+      {
+         return;
+      }
+      for(var criterion : filter.getCriteria())
+      {
+         if(criterion.getOperator() == null || !readable.contains(criterion.getFieldName())
+            || criterion.getOtherFieldName() != null && !readable.contains(criterion.getOtherFieldName()))
+         {
+            throw new IllegalArgumentException("Choose readable filter fields and a valid operator.");
+         }
+      }
+      filter.getSubFilters().forEach(sub -> validateFilter(sub, readable));
+   }
+
+   /*******************************************************************************
+    ** Reject fractional, null, overflowing and out-of-range numeric parameters.
+    *******************************************************************************/
+   private static int integer(Map<String, Object> body, String name, int fallback, int min, int max)
+   {
+      if(!body.containsKey(name))
+      {
+         return (fallback);
+      }
+      Object value = body.get(name);
+      if(!(value instanceof Number number) || number.doubleValue() != number.intValue() || number.intValue() < min || number.intValue() > max)
+      {
+         throw new IllegalArgumentException(name + " must be an integer from " + min + " to " + max + ".");
+      }
+      return (number.intValue());
+   }
+
+   /*******************************************************************************
+    ** Transport conversion avoids middleware dependencies in engine results.
+    *******************************************************************************/
+   private static <T> T transport(Object value, Class<T> type) throws Exception
+   {
+      return (JsonUtils.toObject(JsonUtils.toJson(value), type));
+   }
+
+   /*******************************************************************************
+    ** Check analysis availability.
+    *******************************************************************************/
+   public static class CapabilitiesSpec extends ProfileSpec
+   {
+      /*******************************************************************************
+       ** Public constructor used by the core extension registry.
+       *******************************************************************************/
+      public CapabilitiesSpec()
+      {
+         super("/tables/{table}/capabilities", HttpMethod.GET, "Check analysis availability",
+            "Discover whether the current user can analyze this table, readable fields, chart defaults, limits and backend capabilities.", null, "Capabilities");
+      }
+
+      /*******************************************************************************
+       ** Execute in the request session established by core middleware.
+       *******************************************************************************/
+      @Override
+      protected Object handle(JsonInput input) throws Exception
+      {
+         return (capabilities(input.getPathParam("table")));
+      }
+   }
+
+   /*******************************************************************************
+    ** Profile one column.
+    *******************************************************************************/
+   public static class ColumnProfileSpec extends ProfileSpec
+   {
+      /*******************************************************************************
+       ** Public constructor used by the core extension registry.
+       *******************************************************************************/
+      public ColumnProfileSpec()
+      {
+         super("/tables/{table}/column-profile", HttpMethod.POST, "Profile one column",
+            "Profile one column of a table under a filter: counts, nulls, distinct values, top values, numeric statistics and histogram, date patterns; every number has a drill-through filter unless privacy suppresses it.", "ColumnProfileRequest", "ColumnProfile");
+      }
+
+      /*******************************************************************************
+       ** Execute in the request session established by core middleware.
+       *******************************************************************************/
+      @Override
+      protected Object handle(JsonInput input) throws Exception
+      {
+         return (transport(DataAnalysisRuntime.service().columnProfile((ColumnProfileRequest) request("COLUMN_PROFILE", input.getPathParam("table"), input.getBody())), DataAnalysisSchemas.DataAnalysisColumnProfileV1.class));
+      }
+   }
+
+   /*******************************************************************************
+    ** List distinct column values.
+    *******************************************************************************/
+   public static class DistinctValuesSpec extends ProfileSpec
+   {
+      /*******************************************************************************
+       ** Public constructor used by the core extension registry.
+       *******************************************************************************/
+      public DistinctValuesSpec()
+      {
+         super("/tables/{table}/distinct-values", HttpMethod.POST, "List distinct column values",
+            "Page through distinct stored values with display labels, counts and drill-through filters; search narrows stored values.", "DistinctValuesRequest", "DistinctValuesPage");
+      }
+
+      /*******************************************************************************
+       ** Execute in the request session established by core middleware.
+       *******************************************************************************/
+      @Override
+      protected Object handle(JsonInput input) throws Exception
+      {
+         return (transport(DataAnalysisRuntime.service().distinctValues((DistinctValuesRequest) request("DISTINCT_VALUES", input.getPathParam("table"), input.getBody())), DataAnalysisSchemas.DataAnalysisDistinctValuesPageV1.class));
+      }
+   }
+
+   /*******************************************************************************
+    ** Profile table completeness and keys.
+    *******************************************************************************/
+   public static class TableProfileSpec extends ProfileSpec
+   {
+      /*******************************************************************************
+       ** Public constructor used by the core extension registry.
+       *******************************************************************************/
+      public TableProfileSpec()
+      {
+         super("/tables/{table}/table-profile", HttpMethod.POST, "Profile table completeness and keys",
+            "Profile readable fields under a filter for completeness, uniqueness, constant or empty columns and candidate keys.", "TableProfileRequest", "TableProfile");
+      }
+
+      /*******************************************************************************
+       ** Execute in the request session established by core middleware.
+       *******************************************************************************/
+      @Override
+      protected Object handle(JsonInput input) throws Exception
+      {
+         return (transport(DataAnalysisRuntime.service().tableProfile((TableProfileRequest) request("TABLE_PROFILE", input.getPathParam("table"), input.getBody())), DataAnalysisSchemas.DataAnalysisTableProfileV1.class));
+      }
+   }
+
+   /*******************************************************************************
+    ** Find duplicate groups.
+    *******************************************************************************/
+   public static class DuplicatesSpec extends ProfileSpec
+   {
+      /*******************************************************************************
+       ** Public constructor used by the core extension registry.
+       *******************************************************************************/
+      public DuplicatesSpec()
+      {
+         super("/tables/{table}/duplicates", HttpMethod.POST, "Find duplicate groups",
+            "Group matching records by the selected readable fields and return repeated tuples with counts and drill-through filters.", "DuplicatesRequest", "DuplicatesResult");
+      }
+
+      /*******************************************************************************
+       ** Execute in the request session established by core middleware.
+       *******************************************************************************/
+      @Override
+      protected Object handle(JsonInput input) throws Exception
+      {
+         return (transport(DataAnalysisRuntime.service().duplicates((DuplicatesRequest) request("DUPLICATES", input.getPathParam("table"), input.getBody())), DataAnalysisSchemas.DataAnalysisDuplicatesResultV1.class));
+      }
+   }
+
+   /*******************************************************************************
+    ** Start an exact profile run.
+    *******************************************************************************/
+   public static class StartRunSpec extends ProfileSpec
+   {
+      /*******************************************************************************
+       ** Public constructor used by the core extension registry.
+       *******************************************************************************/
+      public StartRunSpec()
+      {
+         super("/runs", HttpMethod.POST, "Start an exact profile run",
+            "Start an owner-scoped exact column or table profile. Poll the returned runId until COMPLETE or FAILED; results honor the current session permissions.", "StartRunRequest", "RunStarted");
+      }
+
+      /*******************************************************************************
+       ** Execute in the request session established by core middleware.
+       *******************************************************************************/
+      @Override
+      protected Object handle(JsonInput input) throws Exception
+      {
+         Map<String, Object> body = input.getBody();
+         if(body == null || !(body.get("tableName") instanceof String name) || !(body.get("operation") instanceof String operation)
+            || !List.of("COLUMN_PROFILE", "TABLE_PROFILE").contains(operation) || !(body.get("request") instanceof Map<?, ?>))
+         {
+            throw new QBadRequestException("Supply operation, tableName and a profile request object.");
+         }
+         Map<String, Object> parameters = JsonUtils.toObject(JsonUtils.toJson(body.get("request")), java.util.LinkedHashMap.class);
+         String id = DataAnalysisRuntime.service().startExactRun(request(operation, name, parameters));
+         DataAnalysisSchemas.DataAnalysisRunStartedV1 result = new DataAnalysisSchemas.DataAnalysisRunStartedV1();
+         result.runId = id;
+         result.status = DataAnalysisRuntime.service().getRun(id).getStatus();
+         return (result);
+      }
+   }
+
+   /*******************************************************************************
+    ** Get an exact profile run.
+    *******************************************************************************/
+   public static class GetRunSpec extends ProfileSpec
+   {
+      /*******************************************************************************
+       ** Public constructor used by the core extension registry.
+       *******************************************************************************/
+      public GetRunSpec()
+      {
+         super("/runs/{runId}", HttpMethod.GET, "Get an exact profile run",
+            "Read status, timestamps and the typed columnProfile or tableProfile result of an owner-scoped run. Completed results are rechecked against current permissions.", null, "RunStatus");
+      }
+
+      /*******************************************************************************
+       ** Execute in the request session established by core middleware.
+       *******************************************************************************/
+      @Override
+      protected Object handle(JsonInput input) throws Exception
+      {
+         return (transport(DataAnalysisRuntime.service().getRun(input.getPathParam("runId")), DataAnalysisSchemas.DataAnalysisRunStatusV1.class));
+      }
+   }
+
+   /*******************************************************************************
+    ** Permission probing is a successful response with no disclosed field list.
+    *******************************************************************************/
+   private static DataAnalysisSchemas.DataAnalysisCapabilitiesV1 capabilities(String name) throws Exception
+   {
+      QTableMetaData table = table(name);
+      DataAnalysisSchemas.DataAnalysisCapabilitiesV1 result = new DataAnalysisSchemas.DataAnalysisCapabilitiesV1();
+      result.tableName = name;
+      result.tableLabel = table.getLabel();
+      result.canAnalyze = false;
+      result.fields = List.of();
+      result.backendCapabilities = List.of();
+      result.sqlWorkbenchAvailable = QContext.getQInstance().getApp("sqlWorkbench") != null
+         || QContext.getQInstance().getTables().keySet().stream().anyMatch(t -> t.startsWith("sql_workbench_"));
+      try
+      {
+         var capabilities = DataAnalysisRuntime.service().capabilities(name);
+         result.canAnalyze = true;
+         result.backendCapabilities = CoreAnalysisBackend.capabilities(name).stream().map(Enum::name).sorted().toList();
+         result.fields = capabilities.getFields().stream().map(f -> capabilityField(table.getField(f))).toList();
+         var config = capabilities.getConfig();
+         result.limits = new DataAnalysisSchemas.DataAnalysisLimitsV1();
+         result.limits.maxRows = config.getMaxRows();
+         result.limits.sampleThresholdRows = config.getSampleThresholdRows();
+         result.limits.sampleRows = config.getSampleRows();
+         result.limits.minGroupSize = config.getMinGroupSize();
+      }
+      catch(QPermissionDeniedException expected)
+      {
+         // A UI capability probe deliberately reports denial without an error.
+      }
+      return (result);
+   }
+
+   /*******************************************************************************
+    ** Semantic type and the default visualization, with PVS/FK taking priority.
+    *******************************************************************************/
+   private static DataAnalysisSchemas.DataAnalysisCapabilityFieldV1 capabilityField(QFieldMetaData field)
+   {
+      DataAnalysisSchemas.DataAnalysisCapabilityFieldV1 result = new DataAnalysisSchemas.DataAnalysisCapabilityFieldV1();
+      result.name = field.getName();
+      result.label = field.getLabel();
+      result.type = field.getType().name();
+      result.kind = field.getPossibleValueSourceName() != null ? "CATEGORY" : switch(field.getType())
+      {
+         case INTEGER, LONG, DECIMAL -> "NUMERIC";
+         case DATE -> "DATE";
+         case DATE_TIME -> "DATE_TIME";
+         case BOOLEAN -> "BOOLEAN";
+         default -> "TEXT";
+      };
+      result.defaultChartKind = switch(result.kind)
+      {
+         case "NUMERIC" -> ChartKind.HISTOGRAM;
+         case "DATE", "DATE_TIME" -> ChartKind.TIME_SERIES;
+         default -> ChartKind.BAR;
+      };
+      return (result);
+   }
+}

src/main/java/com/kingsrook/qbits/dataanalysis/app/DataAnalysisHome.java

@@ -22,26 +22,33 @@ package com.kingsrook.qbits.dataanalysis.app;
 
 
 import java.time.Instant;
-import java.time.temporal.ChronoUnit;
+import java.util.Arrays;
 import java.util.List;
+import java.util.Locale;
+import java.util.stream.Collectors;
 import com.kingsrook.qbits.dataanalysis.DataAnalysisQBitConfig;
+import com.kingsrook.qbits.dataanalysis.engine.AnalysisFormatting;
 import com.kingsrook.qbits.dataanalysis.model.DataAnalysisTables;
 import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
 import com.kingsrook.qqq.backend.core.actions.permissions.PermissionEnforcement;
 import com.kingsrook.qqq.backend.core.actions.permissions.PermissionsHelper;
 import com.kingsrook.qqq.backend.core.actions.permissions.TablePermissionSubType;
 import com.kingsrook.qqq.backend.core.actions.tables.CountAction;
+import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
 import com.kingsrook.qqq.backend.core.context.QContext;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
 import com.kingsrook.qqq.backend.core.model.actions.tables.count.CountInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryJoin;
 import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetInput;
 import com.kingsrook.qqq.backend.core.model.actions.widgets.RenderWidgetOutput;
 import com.kingsrook.qqq.backend.core.model.dashboard.widgets.RawHTML;
 import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
 import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
 import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
@@ -93,7 +100,8 @@ public class DataAnalysisHome extends AbstractWidgetRenderer
 
 
    /*******************************************************************************
-    ** Standard QQQ widget renderer entry point.
+    ** RawHTML is carried as a JSON string by the HTML widget, not a text/html response.
+    ** Entities keep static separators independent of any host charset handling.
     *******************************************************************************/
    @Override
    public RenderWidgetOutput render(RenderWidgetInput input) throws QException
@@ -110,7 +118,7 @@ public class DataAnalysisHome extends AbstractWidgetRenderer
    {
       StringBuilder html = new StringBuilder("<div class=\"data-analysis-home\" style=\"display:grid;gap:1rem;min-width:0;overflow-wrap:anywhere\">");
       String userId = QContext.getQSession() == null || QContext.getQSession().getUser() == null ? null : QContext.getQSession().getUser().getIdReference();
-      html.append("<section><h2>Saved analyses</h2>");
+      html.append("<section><h3>Saved analyses</h3>");
       if(userId == null)
       {
          html.append("<p>Sign in to see your saved analyses and stars.</p>");
@@ -119,7 +127,7 @@ public class DataAnalysisHome extends AbstractWidgetRenderer
       {
          int mine = count(DataAnalysisTables.SAVED, equal("owner_user_id", userId));
          int shared = count(DataAnalysisTables.SAVED, equal("shared", true));
-         html.append("<p>Mine: ").append(mine).append(" · Shared (visible to you): ").append(shared).append("</p>");
+         html.append("<p>Mine: ").append(AnalysisFormatting.number(mine, 0)).append(" &middot; Shared (visible to you): ").append(AnalysisFormatting.number(shared, 0)).append("</p>");
          if(canRead(DataAnalysisTables.SAVED_STAR))
          {
             CountInput stars = new CountInput(DataAnalysisTables.SAVED).withIncludeDistinctCount(true)
@@ -127,28 +135,28 @@ public class DataAnalysisHome extends AbstractWidgetRenderer
                .withFilter(equal(DataAnalysisTables.SAVED_STAR + ".user_id", userId));
             stars.setPermissionEnforcement(PermissionEnforcement.ENFORCE);
             int starred = new CountAction().execute(stars).getDistinctCount();
-            html.append("<p>Starred by me: ").append(starred).append("</p>");
+            html.append("<p>Starred by me: ").append(AnalysisFormatting.number(starred, 0)).append("</p>");
          }
          if(mine == 0 && shared == 0)
          {
-            html.append("<p>No saved analyses yet. Saving analyses from the Analyze action is planned for a later release.</p>");
+            html.append("<p>No saved analyses yet. Open any table and choose Analyze on a column header to explore its data.</p>");
          }
-         html.append("<p>The Analyze action will be available in a later UI release.</p>");
          link(html, DataAnalysisTables.SAVED);
       }
       else
       {
          html.append("<p>Saved analyses are unavailable: you need data_analysis.saved.read.</p>");
       }
-      html.append("</section><section><h2>Watches</h2>");
+      html.append("</section><section><h3>Watches</h3>");
       if(canRead(DataAnalysisTables.WATCH))
       {
+         int total = count(DataAnalysisTables.WATCH, new QQueryFilter());
          int enabled = count(DataAnalysisTables.WATCH, equal("enabled", true));
          int firing = count(DataAnalysisTables.WATCH, equal("state", "FIRING"));
-         html.append("<p>Enabled: ").append(enabled).append(" · Firing: ").append(firing).append("</p>");
-         if(enabled == 0 && firing == 0)
+         html.append("<p>Total: ").append(AnalysisFormatting.number(total, 0)).append(" &middot; Enabled: ").append(AnalysisFormatting.number(enabled, 0)).append(" &middot; Firing: ").append(AnalysisFormatting.number(firing, 0)).append("</p>");
+         if(total == 0)
          {
-            html.append("<p>No active watches: watch evaluation is planned for phase e.</p>");
+            html.append("<p>No watches yet.</p>");
          }
          link(html, DataAnalysisTables.WATCH);
       }
@@ -156,15 +164,37 @@ public class DataAnalysisHome extends AbstractWidgetRenderer
       {
          html.append("<p>Watches are unavailable: you need data_analysis.watch.read.</p>");
       }
-      html.append("</section><section><h2>Runs in the last 7 days</h2>");
-      if(canRead(DataAnalysisTables.RUN))
+      html.append("</section><section><h3>Recent analysis runs</h3>");
+      if(userId == null)
       {
-         int complete = count(DataAnalysisTables.RUN, recentRuns("COMPLETE", now));
-         int failed = count(DataAnalysisTables.RUN, recentRuns("FAILED", now));
-         html.append("<p>Complete: ").append(complete).append(" · Failed: ").append(failed).append("</p>");
-         if(complete == 0 && failed == 0)
+         html.append("<p>Sign in to see your recent analysis runs.</p>");
+      }
+      else if(canRead(DataAnalysisTables.RUN))
+      {
+         QueryInput query = new QueryInput(DataAnalysisTables.RUN).withFilter(equal("user_id", userId)
+            .withCriteria(new QFilterCriteria("started_at", QCriteriaOperator.LESS_THAN_OR_EQUALS, now))
+            .withOrderBy(new QFilterOrderBy("started_at", false)).withOrderBy(new QFilterOrderBy("id", false)).withLimit(5));
+         query.setPermissionEnforcement(PermissionEnforcement.ENFORCE);
+         List<QRecord> runs = new QueryAction().execute(query).getRecords();
+         if(runs.isEmpty())
          {
-            html.append("<p>No completed or failed runs in this period.</p>");
+            html.append("<p>No analysis runs yet.</p>");
+         }
+         else
+         {
+            html.append("<ul class=\"data-analysis-runs\">");
+            for(QRecord run : runs)
+            {
+               QTableMetaData table = QContext.getQInstance().getTable(run.getValueString("table_name"));
+               String tableLabel = table == null ? run.getValueString("table_name") : table.getLabel();
+               String path = QContext.getQInstance().getTablePath(DataAnalysisTables.RUN) + "/" + run.getValueInteger("id");
+               html.append("<li><a href=\"").append(escape(path)).append("\">").append(escape(tableLabel)).append(" &middot; ")
+                  .append(escape(words(run.getValueString("operation")))).append("</a><dl class=\"data-analysis-run-details\"><dt>Status</dt><dd>")
+                  .append(escape(words(run.getValueString("status")))).append("</dd><dt>Finished</dt><dd>")
+                  .append(run.getValue("finished_at") == null ? "Not finished" : escape(AnalysisFormatting.instant(run.getValueInstant("finished_at"))))
+                  .append("</dd></dl></li>");
+            }
+            html.append("</ul>");
          }
          link(html, DataAnalysisTables.RUN);
       }
@@ -198,13 +228,22 @@ public class DataAnalysisHome extends AbstractWidgetRenderer
 
 
    /*******************************************************************************
-    ** Runs started within the inclusive seven-day interval, excluding future rows.
+    ** Stored operation and status keys are displayed as ordinary words.
     *******************************************************************************/
-   private static QQueryFilter recentRuns(String status, Instant now)
+   private static String words(String value)
    {
-      return (equal("status", status)
-         .withCriteria(new QFilterCriteria("started_at", QCriteriaOperator.GREATER_THAN_OR_EQUALS, now.minus(7, ChronoUnit.DAYS)))
-         .withCriteria(new QFilterCriteria("started_at", QCriteriaOperator.LESS_THAN_OR_EQUALS, now)));
+      return (value == null ? "Unknown" : Arrays.stream(value.toLowerCase(Locale.ROOT).split("_"))
+         .filter(part -> !part.isEmpty()).map(part -> Character.toUpperCase(part.charAt(0)) + part.substring(1)).collect(Collectors.joining(" ")));
+   }
+
+
+
+   /*******************************************************************************
+    ** Metadata and persisted values must never become executable widget markup.
+    *******************************************************************************/
+   private static String escape(String value)
+   {
+      return (value == null ? "Unknown" : value.replace("&", "&amp;").replace("\"", "&quot;").replace("<", "&lt;").replace(">", "&gt;").replace("'", "&#39;"));
    }
 
 
@@ -227,7 +266,7 @@ public class DataAnalysisHome extends AbstractWidgetRenderer
    private static void link(StringBuilder html, String table) throws QException
    {
       String path = QContext.getQInstance().getTablePath(table);
-      html.append("<a href=\"").append(path.replace("&", "&amp;").replace("\"", "&quot;").replace("<", "&lt;").replace(">", "&gt;"))
+      html.append("<a href=\"").append(escape(path))
          .append("\">View all</a>");
    }
 }

src/main/java/com/kingsrook/qbits/dataanalysis/engine/AnalysisBackend.java

@@ -38,16 +38,33 @@ public interface AnalysisBackend
     *******************************************************************************/
    enum DateUnit
    {
-      DAY, WEEK, MONTH, YEAR, DAY_OF_WEEK, HOUR_OF_DAY
+      DAY, WEEK, MONTH, QUARTER, YEAR, HOUR, DAY_OF_WEEK, HOUR_OF_DAY
    }
 
 
 
    /*******************************************************************************
-    ** Actual row coverage and deterministic selected population.
+    ** Computation scope, observed coverage and optional native sampling percentage.
     *******************************************************************************/
-   record Scope(QQueryFilter filter, long total, long rowsScanned, boolean truncated, boolean estimate)
+   record Scope(QQueryFilter filter, long total, long rowsScanned, boolean truncated, boolean estimate, BigDecimal samplePercent)
    {
+      /*******************************************************************************
+       ** Exact and prefix scopes have no random sampling percentage.
+       *******************************************************************************/
+      public Scope(QQueryFilter filter, long total, long rowsScanned, boolean truncated, boolean estimate)
+      {
+         this(filter, total, rowsScanned, truncated, estimate, null);
+      }
+
+      /*******************************************************************************
+       ** Explain whether counts extrapolate a random sample or describe a prefix.
+       *******************************************************************************/
+      public String warning()
+      {
+         return (samplePercent == null ? "Estimate from the first " + rowsScanned + " rows by id"
+            : "Estimate from a random sample of about " + rowsScanned + " rows (" + samplePercent.stripTrailingZeros().toPlainString()
+               + "%). Counts are scaled to the full " + total + " rows; run exact for exact numbers.");
+      }
    }
 
 
@@ -55,8 +72,16 @@ public interface AnalysisBackend
    /*******************************************************************************
     ** Blank includes null and whitespace; distinct excludes null.
     *******************************************************************************/
-   record FieldCounts(long nonNull, long nulls, long blanks, long distinct, QQueryFilter blankFilter, QQueryFilter distinctFilter)
+   record FieldCounts(long nonNull, long nulls, long blanks, long distinct, QQueryFilter blankFilter, QQueryFilter distinctFilter, BigDecimal uniqueness, BigDecimal completeness, boolean constant)
    {
+      /*******************************************************************************
+       ** Exact and prefix counters use their own population for ratios.
+       *******************************************************************************/
+      public FieldCounts(long nonNull, long nulls, long blanks, long distinct, QQueryFilter blankFilter, QQueryFilter distinctFilter)
+      {
+         this(nonNull, nulls, blanks, distinct, blankFilter, distinctFilter, ProfileSupport.share(distinct, nonNull),
+            ProfileSupport.share(nonNull + nulls - blanks, nonNull + nulls), distinct == 1);
+      }
    }
 
 
@@ -64,8 +89,15 @@ public interface AnalysisBackend
    /*******************************************************************************
     ** Group and exact predicate producing its members.
     *******************************************************************************/
-   record GroupCount(Serializable value, String label, long count, QQueryFilter condition)
+   record GroupCount(Serializable value, String label, long count, QQueryFilter condition, BigDecimal share)
    {
+      /*******************************************************************************
+       ** Ordinary groups derive their share from the profiler population.
+       *******************************************************************************/
+      public GroupCount(Serializable value, String label, long count, QQueryFilter condition)
+      {
+         this(value, label, count, condition, null);
+      }
    }
 
 
@@ -89,12 +121,21 @@ public interface AnalysisBackend
 
 
    /*******************************************************************************
-    ** Exact filtered count.
+    ** Actual inspected non-null values, independent of estimated population counts.
+    *******************************************************************************/
+   record PatternSummary(List<GroupCount> groups, long values, boolean truncated)
+   {
+   }
+
+
+
+   /*******************************************************************************
+    ** Filtered count; a native estimate scales conditional counts to its population.
     *******************************************************************************/
    long count(String table, QQueryFilter filter) throws QException;
 
    /*******************************************************************************
-    ** Select the bounded population used consistently by all operations.
+    ** Configure a request-local exact, prefix or native sampling scope.
     *******************************************************************************/
    Scope scope(String table, QQueryFilter filter, boolean exact) throws QException;
 
@@ -136,7 +177,7 @@ public interface AnalysisBackend
    /*******************************************************************************
     ** Top shapes over a bounded primary-key prefix.
     *******************************************************************************/
-   List<GroupCount> patterns(String table, QQueryFilter filter, String field, int sampleRows) throws QException;
+   PatternSummary patterns(String table, QQueryFilter filter, String field, int sampleRows) throws QException;
 
    /*******************************************************************************
     ** Top groups with count greater than one.

src/main/java/com/kingsrook/qbits/dataanalysis/engine/AnalysisFormatting.java

@@ -0,0 +1,225 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.dataanalysis.engine;
+
+import java.io.Serializable;
+import java.math.BigDecimal;
+import java.math.MathContext;
+import java.math.RoundingMode;
+import java.time.DayOfWeek;
+import java.time.Instant;
+import java.time.LocalDate;
+import java.time.Period;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.time.format.TextStyle;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Locale;
+import com.kingsrook.qbits.dataanalysis.result.Metric;
+import com.kingsrook.qbits.dataanalysis.result.ValueType;
+import com.kingsrook.qqq.backend.core.actions.values.QValueFormatter;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.DisplayFormat;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
+import com.kingsrook.qqq.backend.core.utils.ValueUtils;
+
+/*******************************************************************************
+ ** Display-ready values shared by metrics, chart labels and the home widget.
+ *******************************************************************************/
+public final class AnalysisFormatting
+{
+   /*******************************************************************************
+    ** Utility class.
+    *******************************************************************************/
+   private AnalysisFormatting()
+   {
+   }
+
+
+
+   /*******************************************************************************
+    ** Ratios remain numeric and bounded, with stable JSON precision.
+    *******************************************************************************/
+   public static BigDecimal ratio(BigDecimal value)
+   {
+      return (value.max(BigDecimal.ZERO).min(BigDecimal.ONE).setScale(6, RoundingMode.HALF_UP).stripTrailingZeros());
+   }
+
+
+
+   /*******************************************************************************
+    ** Plain numeric defaults are independent of the server locale.
+    *******************************************************************************/
+   public static String number(Number value, int decimals)
+   {
+      return (String.format(Locale.US, "%,." + decimals + "f", new BigDecimal(value.toString())));
+   }
+
+
+
+   /*******************************************************************************
+    ** Types and length keys determine display without changing numeric raw values.
+    *******************************************************************************/
+   public static String scalar(String key, Serializable value, ValueType type)
+   {
+      if(value == null)
+      {
+         return (null);
+      }
+      if(type == ValueType.PERCENT)
+      {
+         BigDecimal percent = new BigDecimal(value.toString()).multiply(BigDecimal.valueOf(100)).setScale(1, RoundingMode.HALF_UP);
+         return (percent.signum() == 0 || percent.compareTo(BigDecimal.valueOf(100)) == 0 ? percent.toBigInteger() + "%" : percent.toPlainString() + "%");
+      }
+      if(type == ValueType.DATE_TIME)
+      {
+         return (instant((Instant) value));
+      }
+      if(value instanceof Number n)
+      {
+         return (number(n, "meanLength".equals(key) ? 1 : type == ValueType.COUNT || key.endsWith("Length") ? 0 : 2));
+      }
+      return (value.toString());
+   }
+
+
+
+   /*******************************************************************************
+    ** Display formats belong to the host field; fallback defaults preserve fractions.
+    *******************************************************************************/
+   public static String numeric(QFieldMetaData field, String key, BigDecimal value)
+   {
+      if(value == null)
+      {
+         return (null);
+      }
+      String format = field.getDisplayFormat();
+      if(format != null && !format.isBlank() && !DisplayFormat.DEFAULT.equals(format))
+      {
+         Serializable displayValue = format.endsWith("d") ? value.toBigInteger() : value;
+         return (QValueFormatter.formatValue(format, displayValue));
+      }
+      return (number(value, field.getType() == QFieldType.DECIMAL || "mean".equals(key) || "stddev".equals(key) ? 2 : 0));
+   }
+
+
+
+   /*******************************************************************************
+    ** Decimal statistics retain ten significant digits; formatting uses the original.
+    *******************************************************************************/
+   public static Metric numericMetric(QFieldMetaData field, String key, BigDecimal value)
+   {
+      BigDecimal raw = value == null || field.getType() != QFieldType.DECIMAL ? value : value.round(new MathContext(10));
+      return (ProfileSupport.metric(key, raw, ValueType.DECIMAL).withFormatted(numeric(field, key, value)));
+   }
+
+
+
+   /*******************************************************************************
+    ** Use the same field formatter on both ends of every histogram interval.
+    *******************************************************************************/
+   public static String interval(QFieldMetaData field, BigDecimal low, BigDecimal high)
+   {
+      return (low == null ? "Below " + edge(field, high)
+         : high == null ? edge(field, low) + " and above"
+         : edge(field, low) + " – " + edge(field, high));
+   }
+
+
+
+   /*******************************************************************************
+    ** Whole edges omit redundant cents; unformatted fractional edges stay distinct.
+    *******************************************************************************/
+   private static String edge(QFieldMetaData field, BigDecimal value)
+   {
+      String format = field.getDisplayFormat();
+      int scale = Math.max(0, value.stripTrailingZeros().scale());
+      if(format == null || format.isBlank() || DisplayFormat.DEFAULT.equals(format))
+      {
+         return (number(value, scale));
+      }
+      String formatted = numeric(field, "min", value);
+      return (scale == 0 && DisplayFormat.CURRENCY.equals(format) && formatted.endsWith(".00")
+         ? formatted.substring(0, formatted.length() - 3) : formatted);
+   }
+
+
+
+   /*******************************************************************************
+    ** Instants remain UTC in raw values and display in the current session zone.
+    *******************************************************************************/
+   public static String instant(Instant value)
+   {
+      return (DateTimeFormatter.ofPattern("uuuu-MM-dd HH:mm", Locale.US).format(value.atZone(ValueUtils.getSessionOrInstanceZoneId())));
+   }
+
+
+
+   /*******************************************************************************
+    ** Calendar duration avoids misleading fixed-length month and year arithmetic.
+    *******************************************************************************/
+   public static String duration(LocalDate first, LocalDate last)
+   {
+      Period period = Period.between(first, last);
+      List<String> parts = new ArrayList<>();
+      if(period.getYears() != 0)
+      {
+         parts.add(period.getYears() + (period.getYears() == 1 ? " year" : " years"));
+      }
+      if(period.getMonths() != 0)
+      {
+         parts.add(period.getMonths() + (period.getMonths() == 1 ? " month" : " months"));
+      }
+      if(period.getDays() != 0 || parts.isEmpty())
+      {
+         parts.add(period.getDays() + (period.getDays() == 1 ? " day" : " days"));
+      }
+      return (String.join(" ", parts));
+   }
+
+
+
+   /*******************************************************************************
+    ** Calendar labels describe one bucket; the raw value remains its ISO start.
+    *******************************************************************************/
+   public static String dateBucket(Serializable start, AnalysisBackend.DateUnit unit, ZoneId zone)
+   {
+      if(unit == AnalysisBackend.DateUnit.DAY_OF_WEEK)
+      {
+         return (DayOfWeek.of(((Number) start).intValue()).getDisplayName(TextStyle.FULL, Locale.US));
+      }
+      if(unit == AnalysisBackend.DateUnit.HOUR_OF_DAY)
+      {
+         return (String.format(Locale.US, "%02d:00", ((Number) start).intValue()));
+      }
+      LocalDate date = ProfileSupport.day(start, zone);
+      return (switch(unit)
+      {
+         case YEAR -> Integer.toString(date.getYear());
+         case QUARTER -> date.getYear() + " Q" + ((date.getMonthValue() - 1) / 3 + 1);
+         case MONTH -> DateTimeFormatter.ofPattern("MMM uuuu", Locale.US).format(date);
+         case WEEK -> "Week of " + date;
+         case HOUR -> DateTimeFormatter.ofPattern("uuuu-MM-dd HH:00", Locale.US).format(((Instant) start).atZone(zone));
+         default -> date.toString();
+      });
+   }
+}

src/main/java/com/kingsrook/qbits/dataanalysis/engine/ColumnProfiler.java

@@ -68,38 +68,45 @@ public class ColumnProfiler
       String name = field.getName();
       QQueryFilter filter = scope.filter();
       long rows = backend.count(table, filter);
+      QQueryFilter drillFilter = scope.estimate() ? request.getFilter() : filter;
       AnalysisBackend.FieldCounts counts = backend.fieldCounts(table, filter, List.of(name)).get(name);
       ColumnProfile result = new ColumnProfile().withTableName(table).withFieldName(name).withFieldLabel(field.getLabel()).withFieldType(field.getType().name())
-         .withFilter(DrillFilters.restoreTypes(request.getTableName(), request.getFilter())).withEstimate(scope.estimate()).withSampleRows(scope.estimate() ? rows : 0L)
+         .withFilter(DrillFilters.restoreTypes(request.getTableName(), request.getFilter())).withEstimate(scope.estimate()).withSampleRows(scope.estimate() ? scope.rowsScanned() : 0L)
          .withRowsScanned(scope.rowsScanned()).withTruncated(scope.truncated()).withAsOf(Instant.now());
-      result.getCounts().add(ProfileSupport.count("rows", rows, table, filter, null));
-      result.getCounts().add(ProfileSupport.count("nonNull", counts.nonNull(), table, filter, DrillFilters.nonNull(name, field.getType().isStringLike())));
-      result.getCounts().add(ProfileSupport.count("nulls", counts.nulls(), table, filter, DrillFilters.nulls(name, field.getType().isStringLike())));
+      result.getCounts().add(ProfileSupport.count("rows", scope.estimate() ? scope.total() : rows, table, drillFilter, null));
+      result.getCounts().add(ProfileSupport.count("nonNull", counts.nonNull(), table, drillFilter, DrillFilters.nonNull(name, field.getType().isStringLike())));
+      result.getCounts().add(ProfileSupport.count("nulls", counts.nulls(), table, drillFilter, DrillFilters.nulls(name, field.getType().isStringLike())));
       if(field.getType().isStringLike())
       {
-         result.getCounts().add(ProfileSupport.count("blanks", counts.blanks(), table, filter, counts.blankFilter()));
+         result.getCounts().add(ProfileSupport.count("blanks", counts.blanks(), table, drillFilter, counts.blankFilter()));
       }
-      result.getCounts().add(ProfileSupport.count("distinct", counts.distinct(), table, filter, counts.distinctFilter()));
-      result.getCounts().add(ProfileSupport.metric("uniqueness", ProfileSupport.share(counts.distinct(), counts.nonNull()), ValueType.PERCENT));
-      result.withTopValues(ProfileSupport.values(table, filter, field, backend.topValues(table, filter, name, 20, 0, null), rows));
+      result.getCounts().add(ProfileSupport.count("distinct", counts.distinct(), table, drillFilter, counts.distinctFilter()));
+      result.getCounts().add(ProfileSupport.metric("uniqueness", counts.uniqueness(), ValueType.PERCENT));
+      result.withTopValues(ProfileSupport.values(table, drillFilter, field, backend.topValues(table, filter, name, 20, 0, null), rows));
       QFieldType type = field.getType();
       if(type == QFieldType.INTEGER || type == QFieldType.LONG || type == QFieldType.DECIMAL)
       {
-         numeric(result, filter, rows);
+         numeric(result, filter, drillFilter, rows, field);
       }
       else if(type == QFieldType.DATE || type == QFieldType.DATE_TIME)
       {
-         dates(result, filter, rows, type);
+         dates(result, filter, drillFilter, rows, type);
       }
       else if(type.isStringLike())
       {
          AnalysisBackend.LengthSummary summary = backend.lengthSummary(table, filter, name);
-         TextProfile text = new TextProfile().withHistogram(ProfileSupport.values(table, filter, null, summary.histogram(), rows))
-            .withPatterns(ProfileSupport.values(table, filter, null, backend.patterns(table, filter, name, 10_000), Math.min(rows, 10_000)))
-            .withSampleRows(Math.min(rows, 10_000));
+         AnalysisBackend.PatternSummary patterns = backend.patterns(table, filter, name, 10_000);
+         long patternValues = patterns.values();
+         TextProfile text = new TextProfile().withHistogram(ProfileSupport.values(table, drillFilter, null, summary.histogram(), rows))
+            .withPatterns(ProfileSupport.values(table, drillFilter, null, patterns.groups(), patternValues))
+            .withSampleRows(patternValues);
          summary.metrics().forEach((key, value) -> text.getMetrics().add(ProfileSupport.metric(key, value, ValueType.DECIMAL)));
          result.withText(text);
-         result.getWarnings().add("Patterns use the first " + Math.min(rows, 10_000) + " rows by primary key.");
+         if(patterns.truncated())
+         {
+            result.getWarnings().add(scope.samplePercent() == null ? "Patterns are based on the first " + AnalysisFormatting.number(patternValues, 0) + " values."
+               : "Patterns are based on " + AnalysisFormatting.number(patternValues, 0) + " sampled values.");
+         }
       }
       else if(type == QFieldType.BOOLEAN)
       {
@@ -107,11 +114,28 @@ public class ColumnProfiler
          for(boolean value : List.of(true, false))
          {
             QQueryFilter condition = DrillFilters.value(name, value, false);
-            result.getBooleans().add(ProfileSupport.count(Boolean.toString(value), backend.count(table, DrillFilters.and(filter, condition)), table, filter, condition));
+            result.getBooleans().add(ProfileSupport.count(Boolean.toString(value), backend.count(table, DrillFilters.and(filter, condition)), table, drillFilter, condition));
          }
-         result.getBooleans().add(ProfileSupport.count("nulls", counts.nulls(), table, filter, DrillFilters.nulls(name, false)));
+         result.getBooleans().add(ProfileSupport.count("nulls", counts.nulls(), table, drillFilter, DrillFilters.nulls(name, false)));
       }
       new MinGroupSizeGuard(config.getMinGroupSize()).column(result, rows);
+      if(scope.estimate())
+      {
+         ProfileSupport.markEstimates(result.getCounts());
+         ProfileSupport.markEstimates(result.getBooleans());
+         if(result.getNumeric() != null)
+         {
+            ProfileSupport.markEstimates(result.getNumeric().getMetrics());
+         }
+         if(result.getDates() != null)
+         {
+            ProfileSupport.markEstimates(result.getDates().getMetrics());
+         }
+         if(result.getText() != null)
+         {
+            ProfileSupport.markEstimates(result.getText().getMetrics());
+         }
+      }
       if(field.getPossibleValueSourceName() != null || result.getNumeric() == null && result.getDates() == null)
       {
          result.withChart(ProfileSupport.chart(ChartKind.BAR, ChartKind.PARETO, name, result.getTopValues()));
@@ -123,22 +147,28 @@ public class ColumnProfiler
       else
       {
          result.withChart(ProfileSupport.chart(ChartKind.TIME_SERIES, ChartKind.BAR, name, result.getDates().getBuckets()));
+         result.getChart().getX().withLabel(field.getLabel() + (result.getDates().getUnit() == null ? "" : " (" + result.getDates().getUnit().toLowerCase(java.util.Locale.ROOT) + ")"));
       }
-      if(rows == 0)
+      if(scope.total() == 0)
       {
          result.getWarnings().add("No readable records match this analysis scope.");
       }
-      else if(counts.nonNull() == 0)
+      else if(counts.nonNull() == 0 && rows > 0 && (!scope.estimate() || scope.rowsScanned() > 0))
       {
-         result.getWarnings().add("All values in this analysis scope are null.");
+         result.getWarnings().add(scope.estimate() ? "All observed values are null." : "All values in this analysis scope are null.");
       }
       if(scope.estimate())
       {
-         result.getWarnings().add("Estimate: first " + rows + " rows by primary key; not a random sample.");
+         result.getWarnings().add(scope.warning());
+         if(scope.rowsScanned() == 0)
+         {
+            result.getWarnings().add("No sample rows were observed; run exact for exact numbers.");
+         }
       }
       if(scope.truncated())
       {
-         result.getWarnings().add("Partial result: max_rows reached; counts and drills cover only the selected prefix.");
+         result.getWarnings().add(scope.estimate() ? "Partial result: max_rows reached; counts cover the selected prefix and drills cover the full population."
+            : "Partial result: max_rows reached; counts and drills cover only the selected prefix.");
       }
       return (result);
    }
@@ -146,24 +176,24 @@ public class ColumnProfiler
 
 
    /*******************************************************************************
-    ** Numeric moments, twenty interior bins, exterior bins and Tukey outliers.
+    ** Numeric moments, nice interior bins, exterior bins and Tukey outliers.
     *******************************************************************************/
-   private void numeric(ColumnProfile result, QQueryFilter filter, long rows) throws QException
+   private void numeric(ColumnProfile result, QQueryFilter filter, QQueryFilter drillFilter, long rows, QFieldMetaData metadata) throws QException
    {
       String table = result.getTableName();
       String field = result.getFieldName();
       AnalysisBackend.NumericSummary summary = backend.numericSummary(table, filter, field,
          List.of("0.01", "0.05", "0.25", "0.50", "0.75", "0.95", "0.99").stream().map(BigDecimal::new).toList());
       NumericProfile numeric = new NumericProfile();
-      summary.metrics().forEach((key, value) -> numeric.getMetrics().add(ProfileSupport.metric(key, value, ValueType.DECIMAL)));
+      summary.metrics().forEach((key, value) -> numeric.getMetrics().add(AnalysisFormatting.numericMetric(metadata, key, value)));
       for(AnalysisBackend.GroupCount count : summary.counts())
       {
-         numeric.getMetrics().add(ProfileSupport.count(count.label(), count.count(), table, filter, count.condition()));
+         numeric.getMetrics().add(ProfileSupport.count(count.label(), count.count(), table, drillFilter, count.condition()));
       }
-      numeric.getMetrics().add(ProfileSupport.count("outliers", summary.outlierCount(), table, filter, summary.outlierFilter()));
-      numeric.withHistogram(ProfileSupport.values(table, filter, null,
+      numeric.getMetrics().add(ProfileSupport.count("outliers", summary.outlierCount(), table, drillFilter, summary.outlierFilter()));
+      numeric.withHistogram(ProfileSupport.values(table, drillFilter, null,
          backend.numericHistogram(table, filter, field, summary.metrics().get("p1"), summary.metrics().get("p99"), 20), rows));
-      numeric.withOutliers(ProfileSupport.values(table, filter, null, summary.outliers(), rows));
+      numeric.withOutliers(ProfileSupport.values(table, drillFilter, null, summary.outliers(), rows));
       result.withNumeric(numeric);
    }
 
@@ -172,7 +202,7 @@ public class ColumnProfiler
    /*******************************************************************************
     ** Date extrema, range, calendar volume, gaps and cyclic time patterns.
     *******************************************************************************/
-   private void dates(ColumnProfile result, QQueryFilter filter, long rows, QFieldType type) throws QException
+   private void dates(ColumnProfile result, QQueryFilter filter, QQueryFilter drillFilter, long rows, QFieldType type) throws QException
    {
       String table = result.getTableName();
       String field = result.getFieldName();
@@ -187,13 +217,13 @@ public class ColumnProfiler
          LocalDate last = ProfileSupport.day(extrema.get("max"), zone);
          AnalysisBackend.DateUnit unit = autoBucket(first, last);
          dates.withUnit(unit.name());
-         dates.getMetrics().add(ProfileSupport.metric("range", ChronoUnit.DAYS.between(first, last), ValueType.DURATION));
-         List<ValueCount> buckets = ProfileSupport.values(table, filter, null, backend.dateBuckets(table, filter, field, unit, zone), rows);
+         dates.getMetrics().add(ProfileSupport.metric("range", ChronoUnit.DAYS.between(first, last), ValueType.DURATION).withFormatted(AnalysisFormatting.duration(first, last)));
+         List<ValueCount> buckets = ProfileSupport.values(table, drillFilter, null, backend.dateBuckets(table, filter, field, unit, zone), rows);
          dates.withBuckets(buckets).withGaps(buckets.stream().filter(b -> b.getCount() == 0).toList());
          if(type == QFieldType.DATE_TIME)
          {
-            dates.withDayOfWeek(ProfileSupport.values(table, filter, null, backend.dateBuckets(table, filter, field, AnalysisBackend.DateUnit.DAY_OF_WEEK, zone), rows))
-               .withHourOfDay(ProfileSupport.values(table, filter, null, backend.dateBuckets(table, filter, field, AnalysisBackend.DateUnit.HOUR_OF_DAY, zone), rows));
+            dates.withDayOfWeek(ProfileSupport.values(table, drillFilter, null, backend.dateBuckets(table, filter, field, AnalysisBackend.DateUnit.DAY_OF_WEEK, zone), rows))
+               .withHourOfDay(ProfileSupport.values(table, drillFilter, null, backend.dateBuckets(table, filter, field, AnalysisBackend.DateUnit.HOUR_OF_DAY, zone), rows));
          }
       }
       result.withDates(dates);

src/main/java/com/kingsrook/qbits/dataanalysis/engine/CoreAnalysisBackend.java

@@ -0,0 +1,483 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.dataanalysis.engine;
+
+import java.io.Serializable;
+import java.math.BigDecimal;
+import java.math.MathContext;
+import java.math.RoundingMode;
+import java.time.LocalDate;
+import java.time.ZoneId;
+import java.util.ArrayList;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.Aggregate;
+import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateCapability;
+import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateOperator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateResult;
+import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.DateBucket;
+import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.DateBucketUnit;
+import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.GroupBy;
+import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.NumericBucket;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
+import com.kingsrook.qqq.backend.core.modules.backend.QBackendModuleDispatcher;
+import com.kingsrook.qqq.backend.core.utils.JsonUtils;
+
+/*******************************************************************************
+ ** Capability-directed aggregates; streamed primitives retain portable drills.
+ *******************************************************************************/
+public class CoreAnalysisBackend extends StandInAnalysisBackend
+{
+   private BigDecimal samplePercent;
+   private QQueryFilter populationFilter;
+   private long populationRows;
+   private long aggregateRows;
+   private BigDecimal lastCountShare;
+
+   /*******************************************************************************
+    ** Bind the same bounded configuration as the streamed implementation.
+    *******************************************************************************/
+   public CoreAnalysisBackend(EffectiveConfig config)
+   {
+      super(config);
+   }
+
+   /*******************************************************************************
+    ** Ask the actual backend implementation, including dialect-specific support.
+    *******************************************************************************/
+   public static Set<AggregateCapability> capabilities(String table) throws QException
+   {
+      AggregateInput input = new AggregateInput(table);
+      return (new QBackendModuleDispatcher().getQBackendModule(input.getBackend()).getAggregateInterface().getSupportedCapabilities(input));
+   }
+
+   /*******************************************************************************
+    ** Sampling belongs to each aggregate, never to a materialized key filter.
+    *******************************************************************************/
+   @Override
+   public Scope scope(String table, QQueryFilter filter, boolean exact) throws QException
+   {
+      samplePercent = null;
+      long total = super.count(table, filter);
+      if(exact || total <= config.getSampleThresholdRows() || total <= config.getSampleRows() || !capabilities(table).contains(AggregateCapability.SAMPLING))
+      {
+         return (super.scope(table, filter, exact));
+      }
+      estimating = true;
+      populationRows = total;
+      populationFilter = DrillFilters.copy(filter);
+      samplePercent = BigDecimal.valueOf(config.getSampleRows()).multiply(BigDecimal.valueOf(100))
+         .divide(BigDecimal.valueOf(total), MathContext.DECIMAL128).min(BigDecimal.valueOf(100));
+      long seen = sampledCount(table, populationFilter);
+      return (new Scope(DrillFilters.copy(filter), total, seen, false, true, samplePercent));
+   }
+
+   /*******************************************************************************
+    ** Count the sampled call's actual rows rather than assuming the requested size.
+    *******************************************************************************/
+   private long sampledCount(String table, QQueryFilter filter) throws QException
+   {
+      if(samplePercent.signum() == 0)
+      {
+         return (0);
+      }
+      List<AggregateResult> result = super.aggregate(new AggregateInput(table).withFilter(DrillFilters.copy(filter))
+         .withAggregate(countOf(primaryKey(table), AggregateOperator.COUNT)).withSamplePercent(samplePercent));
+      return (result.isEmpty() ? 0 : number(result.getFirst(), primaryKey(table), AggregateOperator.COUNT).longValue());
+   }
+
+   /*******************************************************************************
+    ** Unpaginated groups supply their own denominator, including null groups.
+    ** Limited groups use a separate sampled population count for this call.
+    *******************************************************************************/
+   @Override
+   protected List<AggregateResult> aggregate(AggregateInput input) throws QException
+   {
+      if(samplePercent == null)
+      {
+         return (super.aggregate(input));
+      }
+      if(samplePercent.signum() == 0)
+      {
+         aggregateRows = 0;
+         return (List.of());
+      }
+      Aggregate count = countOf(primaryKey(input.getTableName()), AggregateOperator.COUNT);
+      if(input.getAggregates() == null || !input.getAggregates().contains(count))
+      {
+         input.withAggregate(count);
+      }
+      input.withSamplePercent(samplePercent);
+      boolean fullPopulation = JsonUtils.toJson(DrillFilters.copy(input.getFilter())).equals(JsonUtils.toJson(populationFilter));
+      List<AggregateResult> result = super.aggregate(input);
+      aggregateRows = fullPopulation && input.getLimit() == null
+         ? result.stream().mapToLong(r -> number(r, primaryKey(input.getTableName()), AggregateOperator.COUNT).longValue()).sum()
+         : sampledCount(input.getTableName(), populationFilter);
+      return (result);
+   }
+
+   /*******************************************************************************
+    ** Population rows remain exact; conditional counters estimate their membership.
+    *******************************************************************************/
+   @Override
+   public long count(String table, QQueryFilter filter) throws QException
+   {
+      if(samplePercent == null)
+      {
+         return (super.count(table, filter));
+      }
+      if(JsonUtils.toJson(DrillFilters.copy(filter)).equals(JsonUtils.toJson(populationFilter)))
+      {
+         lastCountShare = ProfileSupport.share(populationRows, populationRows);
+         return (populationRows);
+      }
+      long value = sampledCount(table, filter);
+      aggregateRows = sampledCount(table, populationFilter);
+      lastCountShare = ProfileSupport.share(value, aggregateRows);
+      return (aggregateCount(value));
+   }
+
+   /*******************************************************************************
+    ** Round count estimates only after normalizing by observed sample rows.
+    *******************************************************************************/
+   @Override
+   protected long aggregateCount(long value)
+   {
+      return (samplePercent == null ? value : aggregateRows == 0 ? 0 : BigDecimal.valueOf(value).multiply(BigDecimal.valueOf(populationRows))
+         .divide(BigDecimal.valueOf(aggregateRows), 0, RoundingMode.HALF_UP).longValueExact());
+   }
+
+   /*******************************************************************************
+    ** Keep ratios in sample units, independent of count rounding.
+    *******************************************************************************/
+   @Override
+   protected BigDecimal aggregateShare(long value)
+   {
+      return (samplePercent == null ? null : ProfileSupport.share(value, aggregateRows));
+   }
+
+   /*******************************************************************************
+    ** Conditional groups retain the ratio from their sampled count call.
+    *******************************************************************************/
+   @Override
+   protected BigDecimal countShare()
+   {
+      return (samplePercent == null ? null : lastCountShare);
+   }
+
+   /*******************************************************************************
+    ** Streamed fallbacks also observe native sampling, without binding sampled ids.
+    *******************************************************************************/
+   @Override
+   protected List<QRecord> rows(String table, QQueryFilter filter, Set<String> fields, int limit) throws QException
+   {
+      if(samplePercent == null)
+      {
+         return (super.rows(table, filter, fields, limit));
+      }
+      if(limit == 0 || config.getMaxRows() == 0)
+      {
+         return (List.of());
+      }
+      AggregateInput input = new AggregateInput(table).withFilter(DrillFilters.copy(filter)).withGroupBy(new GroupBy(field(table, primaryKey(table))))
+         .withLimit(Math.min(limit, config.getMaxRows()));
+      fields.stream().filter(f -> !f.equals(primaryKey(table))).forEach(f -> input.withGroupBy(new GroupBy(field(table, f))));
+      List<QRecord> result = new ArrayList<>();
+      for(AggregateResult row : aggregate(input))
+      {
+         QRecord record = new QRecord();
+         row.getGroupByValues().forEach((key, value) -> record.setValue(key.getFieldName(), value));
+         result.add(record);
+      }
+      return (result);
+   }
+
+   /*******************************************************************************
+    ** Every field batch shares a COUNT denominator from the same sampled call.
+    *******************************************************************************/
+   @Override
+   public Map<String, FieldCounts> fieldCounts(String table, QQueryFilter filter, List<String> fields) throws QException
+   {
+      if(samplePercent == null)
+      {
+         return (super.fieldCounts(table, filter, fields));
+      }
+      Map<String, FieldCounts> result = new LinkedHashMap<>();
+      for(int start = 0; start < fields.size(); start += 20)
+      {
+         List<String> batch = fields.subList(start, Math.min(start + 20, fields.size()));
+         AggregateInput input = new AggregateInput(table).withFilter(DrillFilters.copy(filter));
+         for(String field : batch)
+         {
+            input.withAggregate(countOf(field, AggregateOperator.COUNT)).withAggregate(countOf(field, AggregateOperator.COUNT_DISTINCT));
+         }
+         fieldCountAggregates(input, batch);
+         List<String> stringFields = batch.stream().filter(f -> field(table, f).getType().isStringLike()).toList();
+         List<QRecord> records = stringFields.isEmpty() ? List.of() : rows(table, filter, Set.copyOf(stringFields), config.getMaxRows());
+         List<AggregateResult> results = aggregate(input);
+         long seen = aggregateRows;
+         for(String field : batch)
+         {
+            boolean string = field(table, field).getType().isStringLike();
+            long nonNull = results.isEmpty() ? 0 : number(results.getFirst(), field, AggregateOperator.COUNT).longValue();
+            long distinct = results.isEmpty() ? 0 : number(results.getFirst(), field, AggregateOperator.COUNT_DISTINCT).longValue();
+            long blanks = string && capabilities(table).contains(AggregateCapability.NULL_BLANK_COUNTS) && !results.isEmpty()
+               ? number(results.getFirst(), field, AggregateOperator.COUNT_BLANK).longValue() : seen - nonNull;
+            QQueryFilter blankFilter = DrillFilters.nulls(field, string);
+            if(string)
+            {
+               blankFilter = new QQueryFilter().withBooleanOperator(QQueryFilter.BooleanOperator.OR).withSubFilter(blankFilter);
+               for(String blank : records.stream().map(r -> r.getValueString(field)).filter(Objects::nonNull).filter(String::isBlank).distinct().toList())
+               {
+                  blankFilter.withSubFilter(DrillFilters.condition(field, QCriteriaOperator.EQUALS, blank));
+               }
+            }
+            long blankCount = aggregateCount(blanks);
+            BigDecimal completeness = ProfileSupport.share(seen - blanks, seen);
+            if(string && !capabilities(table).contains(AggregateCapability.NULL_BLANK_COUNTS))
+            {
+               blankCount = count(table, DrillFilters.and(filter, blankFilter));
+               completeness = BigDecimal.ONE.subtract(countShare());
+               aggregateRows = seen;
+            }
+            result.put(field, new FieldCounts(aggregateCount(nonNull), aggregateCount(seen - nonNull), blankCount, aggregateCount(distinct),
+               blankFilter, DrillFilters.nonNull(field, string), ProfileSupport.share(distinct, nonNull), completeness, distinct == 1));
+         }
+      }
+      return (result);
+   }
+
+   /*******************************************************************************
+    ** Add supported counters to the existing twenty-field aggregate batch.
+    *******************************************************************************/
+   @Override
+   protected void fieldCountAggregates(AggregateInput input, List<String> fields) throws QException
+   {
+      if(capabilities(input.getTableName()).contains(AggregateCapability.NULL_BLANK_COUNTS))
+      {
+         for(String name : fields)
+         {
+            input.withAggregate(new Aggregate(name, AggregateOperator.COUNT_NULL));
+            if(field(input.getTableName(), name).getType().isStringLike())
+            {
+               input.withAggregate(new Aggregate(name, AggregateOperator.COUNT_BLANK));
+            }
+         }
+      }
+   }
+
+   /*******************************************************************************
+    ** Read native counters, falling back to the portable predicate counter.
+    *******************************************************************************/
+   @Override
+   protected long fieldCount(String table, QQueryFilter filter, String field, AggregateOperator operator,
+      List<AggregateResult> results, QQueryFilter condition) throws QException
+   {
+      if(capabilities(table).contains(AggregateCapability.NULL_BLANK_COUNTS))
+      {
+         return (results.isEmpty() ? 0 : number(results.getFirst(), field, operator).longValue());
+      }
+      return (super.fieldCount(table, filter, field, operator, results, condition));
+   }
+
+   /*******************************************************************************
+    ** Push each supported statistic, streaming only unavailable operators.
+    *******************************************************************************/
+   @Override
+   protected Map<String, BigDecimal> numericMetrics(String table, QQueryFilter filter, String field, List<BigDecimal> percentiles, List<BigDecimal> values) throws QException
+   {
+      Set<AggregateCapability> supported = capabilities(table);
+      if(!supported.containsAll(Set.of(AggregateCapability.STDDEV, AggregateCapability.PERCENTILE)))
+      {
+         return (partialNumericMetrics(table, filter, field, percentiles, values, supported));
+      }
+      return (nativeNumericMetrics(table, filter, field, percentiles, supported));
+   }
+
+   /*******************************************************************************
+    ** Existing streamed arithmetic supplies only unavailable metrics.
+    *******************************************************************************/
+   private Map<String, BigDecimal> partialNumericMetrics(String table, QQueryFilter filter, String field, List<BigDecimal> percentiles,
+      List<BigDecimal> values, Set<AggregateCapability> supported) throws QException
+   {
+      Map<String, BigDecimal> metrics = super.numericMetrics(table, filter, field, percentiles, values);
+      metrics.putAll(nativeNumericMetrics(table, filter, field, percentiles, supported));
+      return (metrics);
+   }
+
+   /*******************************************************************************
+    ** One aggregate call computes basic moments and all supported extensions.
+    *******************************************************************************/
+   private Map<String, BigDecimal> nativeNumericMetrics(String table, QQueryFilter filter, String field, List<BigDecimal> percentiles,
+      Set<AggregateCapability> supported) throws QException
+   {
+      Map<String, BigDecimal> metrics = new LinkedHashMap<>();
+      AggregateInput input = new AggregateInput(table).withFilter(DrillFilters.copy(filter));
+      Map<AggregateOperator, String> names = Map.of(AggregateOperator.MIN, "min", AggregateOperator.MAX, "max", AggregateOperator.AVG, "mean", AggregateOperator.SUM, "sum", AggregateOperator.STDDEV_SAMP, "stddev");
+      for(AggregateOperator op : names.keySet())
+      {
+         if(op != AggregateOperator.STDDEV_SAMP || supported.contains(AggregateCapability.STDDEV))
+         {
+            input.withAggregate(new Aggregate(field, op));
+         }
+      }
+      if(supported.contains(AggregateCapability.PERCENTILE))
+      {
+         percentiles.forEach(p -> input.withAggregate(new Aggregate(field, AggregateOperator.PERCENTILE_CONT).withPercentile(p)));
+      }
+      for(AggregateResult row : aggregate(input))
+      {
+         row.getAggregateValues().entrySet().stream().filter(e -> e.getKey().getOperator() != AggregateOperator.COUNT).forEach(entry ->
+         {
+            Aggregate key = entry.getKey();
+            Serializable value = entry.getValue();
+            metrics.put(key.getOperator() == AggregateOperator.PERCENTILE_CONT
+               ? "p" + key.getPercentile().multiply(BigDecimal.valueOf(100)).stripTrailingZeros().toPlainString() : names.get(key.getOperator()),
+               value == null ? null : new BigDecimal(value.toString()));
+         });
+      }
+      return (metrics);
+   }
+
+   /*******************************************************************************
+    ** Convert native bin numbers into identical half-open drill predicates.
+    *******************************************************************************/
+   @Override
+   public List<GroupCount> numericHistogram(String table, QQueryFilter filter, String field, BigDecimal low, BigDecimal high, int bins) throws QException
+   {
+      if(low == null || high == null || low.compareTo(high) >= 0 || !capabilities(table).contains(AggregateCapability.NUMERIC_BUCKET))
+      {
+         return (super.numericHistogram(table, filter, field, low, high, bins));
+      }
+      NiceBins nice = NiceBins.covering(low, high);
+      low = nice.low();
+      high = nice.high();
+      bins = nice.count();
+      GroupBy group = new GroupBy(field(table, field)).withBucket(new NumericBucket(low, high, bins));
+      Map<Serializable, Long> counts = bucketCounts(table, filter, group);
+      List<BigDecimal> values = !estimating && isMemory(table) ? rows(table, filter, Set.of(field), config.getMaxRows()).stream().map(r -> r.getValue(field))
+         .filter(Objects::nonNull).map(v -> new BigDecimal(v.toString())).toList() : List.of();
+      List<GroupCount> result = new ArrayList<>();
+      result.add(new GroupCount(AnalysisFormatting.interval(field(table, field), null, low), AnalysisFormatting.interval(field(table, field), null, low), aggregateCount(counts.getOrDefault(0, 0L)), numericRange(table, field, values, null, low, false), aggregateShare(counts.getOrDefault(0, 0L))));
+      BigDecimal width = high.subtract(low).divide(BigDecimal.valueOf(bins), MathContext.DECIMAL128);
+      for(int i = 0; i < bins; i++)
+      {
+         BigDecimal start = low.add(width.multiply(BigDecimal.valueOf(i)));
+         BigDecimal end = i == bins - 1 ? high : low.add(width.multiply(BigDecimal.valueOf(i + 1)));
+         String label = AnalysisFormatting.interval(field(table, field), start, end);
+         result.add(new GroupCount(label, label, aggregateCount(counts.getOrDefault(i + 1, 0L)), numericRange(table, field, values, start, end, i == bins - 1), aggregateShare(counts.getOrDefault(i + 1, 0L))));
+      }
+      result.add(new GroupCount(AnalysisFormatting.interval(field(table, field), high, null), AnalysisFormatting.interval(field(table, field), high, null), aggregateCount(counts.getOrDefault(bins + 1, 0L)),
+         DrillFilters.and(DrillFilters.condition(field, QCriteriaOperator.IS_NOT_BLANK), DrillFilters.condition(field, QCriteriaOperator.GREATER_THAN, high)), aggregateShare(counts.getOrDefault(bins + 1, 0L))));
+      return (result);
+   }
+
+   /*******************************************************************************
+    ** Native dates use the session zone; fill calendar gaps with zero counts.
+    *******************************************************************************/
+   @Override
+   public List<GroupCount> dateBuckets(String table, QQueryFilter filter, String field, DateUnit unit, ZoneId zone) throws QException
+   {
+      if(unit == DateUnit.HOUR || unit == DateUnit.QUARTER)
+      {
+         return (super.dateBuckets(table, filter, field, unit, zone));
+      }
+      GroupBy group = new GroupBy(field(table, field)).withBucket(new DateBucket(DateBucketUnit.valueOf(unit.name())));
+      AggregateInput probe = new AggregateInput(table).withGroupBy(group);
+      if(!new QBackendModuleDispatcher().getQBackendModule(probe.getBackend()).getAggregateInterface().getSupportedCapabilities(probe).contains(AggregateCapability.DATE_BUCKET))
+      {
+         return (super.dateBuckets(table, filter, field, unit, zone));
+      }
+      Map<Serializable, Long> counts = bucketCounts(table, filter, group);
+      if(unit == DateUnit.DAY_OF_WEEK || unit == DateUnit.HOUR_OF_DAY)
+      {
+         long seen = aggregateRows;
+         List<GroupCount> conditions = super.dateBuckets(table, filter, field, unit, zone);
+         aggregateRows = seen;
+         return (conditions.stream()
+            .map(g -> new GroupCount(g.value(), g.label(), aggregateCount(counts.getOrDefault(g.value(), 0L)), g.condition(), aggregateShare(counts.getOrDefault(g.value(), 0L)))).toList());
+      }
+      List<LocalDate> dates = counts.keySet().stream().filter(Objects::nonNull).map(v -> (LocalDate) v).sorted().toList();
+      List<GroupCount> result = new ArrayList<>();
+      if(!dates.isEmpty())
+      {
+         LocalDate start = dates.getFirst();
+         while(!start.isAfter(dates.getLast()))
+         {
+            LocalDate end = next(start, unit);
+            String label = AnalysisFormatting.dateBucket(start, unit, zone);
+            result.add(new GroupCount(start, label, aggregateCount(counts.getOrDefault(start, 0L)),
+               DrillFilters.dateRange(field, start, end, field(table, field).getType() == QFieldType.DATE_TIME, zone), aggregateShare(counts.getOrDefault(start, 0L))));
+            start = end;
+         }
+      }
+      return (result);
+   }
+
+   /*******************************************************************************
+    ** Length moments are native; the derived length histogram remains streamed.
+    *******************************************************************************/
+   @Override
+   public LengthSummary lengthSummary(String table, QQueryFilter filter, String field) throws QException
+   {
+      LengthSummary streamed = super.lengthSummary(table, filter, field);
+      if(capabilities(table).contains(AggregateCapability.STRING_LENGTH))
+      {
+         AggregateInput input = new AggregateInput(table).withFilter(DrillFilters.copy(filter));
+         Map<AggregateOperator, String> names = Map.of(AggregateOperator.MIN_LENGTH, "minLength", AggregateOperator.MAX_LENGTH, "maxLength", AggregateOperator.AVG_LENGTH, "meanLength");
+         names.keySet().forEach(op -> input.withAggregate(new Aggregate(field, op)));
+         for(AggregateResult row : aggregate(input))
+         {
+            row.getAggregateValues().forEach((key, value) ->
+            {
+               if(value != null && names.containsKey(key.getOperator()))
+               {
+                  streamed.metrics().put(names.get(key.getOperator()), new BigDecimal(value.toString()));
+               }
+            });
+         }
+      }
+      return (streamed);
+   }
+
+   /*******************************************************************************
+    ** Bucket result keys retain core's typed date or integer values.
+    *******************************************************************************/
+   private Map<Serializable, Long> bucketCounts(String table, QQueryFilter filter, GroupBy group) throws QException
+   {
+      Map<Serializable, Long> result = new LinkedHashMap<>();
+      AggregateInput input = new AggregateInput(table).withFilter(DrillFilters.copy(filter)).withGroupBy(group)
+         .withAggregate(countOf(primaryKey(table), AggregateOperator.COUNT));
+      for(AggregateResult row : aggregate(input))
+      {
+         result.put(row.getGroupByValues().get(group), number(row, primaryKey(table), AggregateOperator.COUNT).longValue());
+      }
+      return (result);
+   }
+}

src/main/java/com/kingsrook/qbits/dataanalysis/engine/DrillFilters.java

@@ -75,11 +75,63 @@ public final class DrillFilters
     *******************************************************************************/
    public static Drill drill(String table, QQueryFilter original, QQueryFilter condition)
    {
-      return (new Drill().withTableName(table).withFilter(and(original, condition)));
+      QQueryFilter filter = and(original, condition);
+      var instance = com.kingsrook.qqq.backend.core.context.QContext.getQInstance();
+      if(instance != null && "memory".equals(instance.getBackend(instance.getTable(table).getBackendName()).getBackendType()))
+      {
+         filter = memoryTemporalBounds(restoreTypes(table, filter));
+      }
+      return (new Drill().withTableName(table).withFilter(filter));
    }
 
 
 
+   /*******************************************************************************
+    ** Memory's inclusive lower bound uses Java equality, which differs after
+    ** HTTP turns dates into strings. Strict lower bounds avoid that equality.
+    ** Only memory uses this adjustment: SQL timestamp precision may be coarser.
+    *******************************************************************************/
+   private static QQueryFilter memoryTemporalBounds(QQueryFilter filter)
+   {
+      filter.setSubFilters(new java.util.ArrayList<>(filter.getSubFilters()));
+      for(var criterion : new java.util.ArrayList<>(filter.getCriteria()))
+      {
+         if(criterion.getValues() != null && !criterion.getValues().isEmpty()
+            && criterion.getValues().stream().allMatch(v -> v instanceof java.time.Instant || v instanceof LocalDate)
+            && (criterion.getOperator() == QCriteriaOperator.EQUALS || criterion.getOperator() == QCriteriaOperator.IN))
+         {
+            QQueryFilter members = new QQueryFilter().withBooleanOperator(QQueryFilter.BooleanOperator.OR);
+            for(Serializable value : criterion.getValues())
+            {
+               members.withSubFilter(range(criterion.getFieldName(), value, value, true));
+            }
+            filter.getCriteria().remove(criterion);
+            filter.withSubFilter(members);
+            continue;
+         }
+         if(criterion.getValues() != null && criterion.getValues().size() == 1)
+         {
+            Serializable value = criterion.getValues().getFirst();
+            if(value instanceof java.time.Instant || value instanceof LocalDate)
+            {
+               Serializable previous = value instanceof java.time.Instant instant ? instant.minusNanos(1) : ((LocalDate) value).minusDays(1);
+               if(criterion.getOperator() == QCriteriaOperator.GREATER_THAN_OR_EQUALS)
+               {
+                  criterion.setOperator(QCriteriaOperator.GREATER_THAN);
+                  criterion.setValues(List.of(previous));
+               }
+               else if(criterion.getOperator() == QCriteriaOperator.LESS_THAN)
+               {
+                  criterion.setOperator(QCriteriaOperator.LESS_THAN_OR_EQUALS);
+                  criterion.setValues(List.of(previous));
+               }
+            }
+         }
+      }
+      filter.getSubFilters().forEach(DrillFilters::memoryTemporalBounds);
+      return (filter);
+   }
+
    /*******************************************************************************
     ** One criterion, including predicates without operands.
     *******************************************************************************/
@@ -116,7 +168,7 @@ public final class DrillFilters
       QQueryFilter result = condition(field, QCriteriaOperator.IS_NOT_BLANK);
       if(string)
       {
-         result.withBooleanOperator(QQueryFilter.BooleanOperator.OR).withCriteria(field, QCriteriaOperator.EQUALS, "");
+         result.withBooleanOperator(QQueryFilter.BooleanOperator.OR).withSubFilter(emptyString(field));
       }
       return (result);
    }
@@ -128,7 +180,8 @@ public final class DrillFilters
     *******************************************************************************/
    public static QQueryFilter value(String field, Serializable value, boolean string)
    {
-      return (value == null ? nulls(field, string) : condition(field, QCriteriaOperator.EQUALS, value));
+      return (value == null ? nulls(field, string) : "".equals(value) ? emptyString(field)
+         : value instanceof java.time.Instant || value instanceof LocalDate ? range(field, value, value, true) : condition(field, QCriteriaOperator.EQUALS, value));
    }
 
 
@@ -181,11 +234,33 @@ public final class DrillFilters
     *******************************************************************************/
    public static QQueryFilter in(String field, List<? extends Serializable> values)
    {
+      if(values.stream().anyMatch(v -> v instanceof java.time.Instant || v instanceof LocalDate))
+      {
+         QQueryFilter result = new QQueryFilter().withBooleanOperator(QQueryFilter.BooleanOperator.OR);
+         values.forEach(value -> result.withSubFilter(range(field, value, value, true)));
+         return (result);
+      }
+      if(values.contains(""))
+      {
+         List<? extends Serializable> nonEmpty = values.stream().filter(v -> !"".equals(v)).toList();
+         return (new QQueryFilter().withBooleanOperator(QQueryFilter.BooleanOperator.OR).withSubFilter(emptyString(field)).withSubFilter(in(field, nonEmpty)));
+      }
       return (values.isEmpty() ? condition(field, QCriteriaOperator.FALSE) : new QQueryFilter().withCriteria(field, QCriteriaOperator.IN, values));
    }
 
 
 
+   /*******************************************************************************
+    ** Core HTTP interpretation treats empty string operands as missing inputs.
+    ** A zero-length predicate expresses the same set without an empty operand.
+    *******************************************************************************/
+   private static QQueryFilter emptyString(String field)
+   {
+      return (new QQueryFilter().withCriteria(new com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria(field, QCriteriaOperator.EQUALS, 0)
+         .withFieldFunction(new com.kingsrook.qqq.backend.core.model.metadata.fields.functions.FieldFunction().withFieldName(field)
+            .withFunctionTypeIdentifier(com.kingsrook.qqq.backend.core.model.metadata.fields.functions.implementations.StringLengthFunction.IDENTIFIER))));
+   }
+
    /*******************************************************************************
     ** JSON has no temporal scalar types. Restore them from current metadata before
     ** a cached drill is handed back to core, without mutating the stored filter.

src/main/java/com/kingsrook/qbits/dataanalysis/engine/DuplicateFinder.java

@@ -51,6 +51,6 @@ public class DuplicateFinder
       return (new DuplicatesResult().withTableName(request.getTableName()).withFields(request.getFields()).withFilter(DrillFilters.copy(request.getFilter()))
          .withRowsScanned(scope.rowsScanned()).withTruncated(scope.truncated()).withAsOf(Instant.now())
          .withGroups(new MinGroupSizeGuard(config.getMinGroupSize()).groups(ProfileSupport.values(request.getTableName(), scope.filter(), null,
-            backend.duplicates(request.getTableName(), scope.filter(), request.getFields(), 50), scope.rowsScanned()))));
+            backend.duplicates(request.getTableName(), scope.filter(), request.getFields(), request.getLimit()), scope.rowsScanned()))));
    }
 }

src/main/java/com/kingsrook/qbits/dataanalysis/engine/MetricLabels.java

@@ -0,0 +1,65 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.dataanalysis.engine;
+
+import java.util.Map;
+import java.util.Objects;
+import com.kingsrook.qbits.dataanalysis.result.ValueType;
+
+/*******************************************************************************
+ ** One vocabulary for every emitted metric, including date-specific extrema.
+ *******************************************************************************/
+public final class MetricLabels
+{
+   private static final Map<String, String> LABELS = Map.ofEntries(
+      Map.entry("rows", "Rows"), Map.entry("nonNull", "Non-empty"), Map.entry("nulls", "Nulls"),
+      Map.entry("blanks", "Blanks"), Map.entry("distinct", "Distinct values"), Map.entry("uniqueness", "Uniqueness"),
+      Map.entry("completeness", "Completeness"), Map.entry("share", "Share"),
+      Map.entry("min", "Minimum"), Map.entry("max", "Maximum"), Map.entry("mean", "Mean"),
+      Map.entry("median", "Median"), Map.entry("stddev", "Standard deviation"), Map.entry("sum", "Sum"),
+      Map.entry("zeros", "Zeros"), Map.entry("negatives", "Negatives"), Map.entry("outliers", "Outliers"),
+      Map.entry("p1", "1st percentile"), Map.entry("p5", "5th percentile"), Map.entry("p25", "25th percentile (Q1)"),
+      Map.entry("p50", "Median"), Map.entry("p75", "75th percentile (Q3)"), Map.entry("p95", "95th percentile"), Map.entry("p99", "99th percentile"),
+      Map.entry("minLength", "Shortest"), Map.entry("maxLength", "Longest"), Map.entry("meanLength", "Average length"),
+      Map.entry("earliest", "Earliest"), Map.entry("latest", "Latest"), Map.entry("range", "Range"),
+      Map.entry("true", "True"), Map.entry("false", "False"), Map.entry("other", "Other"));
+
+   /*******************************************************************************
+    ** Utility class.
+    *******************************************************************************/
+   private MetricLabels()
+   {
+   }
+
+
+
+   /*******************************************************************************
+    ** Fail at construction when a new metric lacks a human label.
+    *******************************************************************************/
+   public static String label(String key, ValueType type)
+   {
+      if(type == ValueType.DATE || type == ValueType.DATE_TIME)
+      {
+         key = "min".equals(key) ? "earliest" : "max".equals(key) ? "latest" : key;
+      }
+      return (Objects.requireNonNull(LABELS.get(key), "Missing metric label: " + key));
+   }
+}

src/main/java/com/kingsrook/qbits/dataanalysis/engine/MinGroupSizeGuard.java

@@ -71,7 +71,7 @@ public class MinGroupSizeGuard
       }
       if(hidden > 0)
       {
-         result.add(new ValueCount().withLabel("Other (" + hidden + " groups below " + minimum + ")").withCount(count).withShare(share));
+         result.add(new ValueCount().withLabel("Other (" + hidden + " groups below " + minimum + ")").withCount(count).withShare(AnalysisFormatting.ratio(share)));
       }
       return (result);
    }
@@ -114,7 +114,7 @@ public class MinGroupSizeGuard
             List<ValueCount> booleanGroups = profile.getBooleans().stream().map(m -> new ValueCount().withValue(m.getKey()).withLabel(m.getLabel())
                .withCount(((Number) m.getValue()).longValue()).withShare(ProfileSupport.share(((Number) m.getValue()).longValue(), rows)).withDrill(m.getDrill())).toList();
             profile.withBooleans(groups(booleanGroups).stream().map(v -> ProfileSupport.metric(v.getValue() == null ? "other" : v.getValue().toString(), v.getCount(),
-               com.kingsrook.qbits.dataanalysis.result.ValueType.COUNT).withLabel(v.getLabel()).withDrill(v.getDrill())).toList());
+               com.kingsrook.qbits.dataanalysis.result.ValueType.COUNT).withDrill(v.getDrill())).toList());
          }
       }
       if(profile.getNumeric() != null)

src/main/java/com/kingsrook/qbits/dataanalysis/engine/NiceBins.java

@@ -0,0 +1,62 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.dataanalysis.engine;
+
+import java.math.BigDecimal;
+import java.math.RoundingMode;
+import java.util.List;
+
+/*******************************************************************************
+ ** Exact decimal boundaries with a readable step and approximately twenty bins.
+ *******************************************************************************/
+public record NiceBins(BigDecimal low, BigDecimal high, int count)
+{
+   /*******************************************************************************
+    ** Select the closest 10..25-bin enclosure from 1, 2, 2.5 and 5 times powers of ten.
+    *******************************************************************************/
+   public static NiceBins covering(BigDecimal low, BigDecimal high)
+   {
+      BigDecimal span = high.subtract(low);
+      if(span.signum() == 0)
+      {
+         span = low.abs().max(BigDecimal.ONE);
+         low = low.subtract(span.divide(BigDecimal.valueOf(2)));
+         high = high.add(span.divide(BigDecimal.valueOf(2)));
+      }
+      int exponent = span.precision() - span.scale() - 2;
+      NiceBins best = null;
+      for(int power = exponent - 2; power <= exponent + 2; power++)
+      {
+         for(String factor : List.of("1", "2", "2.5", "5"))
+         {
+            BigDecimal step = new BigDecimal(factor).scaleByPowerOfTen(power);
+            BigDecimal first = low.divide(step, 0, RoundingMode.FLOOR).multiply(step);
+            BigDecimal last = high.divide(step, 0, RoundingMode.CEILING).multiply(step);
+            int count = last.subtract(first).divide(step).intValueExact();
+            if(count >= 10 && count <= 25 && (best == null || Math.abs(count - 20) < Math.abs(best.count() - 20)))
+            {
+               best = new NiceBins(first, last, count);
+            }
+         }
+      }
+      return (java.util.Objects.requireNonNull(best));
+   }
+}

src/main/java/com/kingsrook/qbits/dataanalysis/engine/ProfileSupport.java

@@ -56,7 +56,7 @@ public final class ProfileSupport
     *******************************************************************************/
    public static BigDecimal share(long count, long rows)
    {
-      return (rows == 0 ? BigDecimal.ZERO : BigDecimal.valueOf(count).divide(BigDecimal.valueOf(rows), MathContext.DECIMAL128));
+      return (rows == 0 ? BigDecimal.ZERO : BigDecimal.valueOf(count).divide(BigDecimal.valueOf(rows), 6, java.math.RoundingMode.HALF_UP).stripTrailingZeros());
    }
 
 
@@ -76,11 +76,34 @@ public final class ProfileSupport
     *******************************************************************************/
    public static Metric metric(String key, Serializable value, ValueType type)
    {
-      return (new Metric().withKey(key).withLabel(key).withValue(value).withValueType(type).withFormatted(value == null ? null : value.toString()));
+      if(type == ValueType.PERCENT && value != null)
+      {
+         value = AnalysisFormatting.ratio(new BigDecimal(value.toString()));
+      }
+      return (new Metric().withKey(key).withLabel(MetricLabels.label(key, type)).withValue(value).withValueType(type)
+         .withFormatted(AnalysisFormatting.scalar(key, value, type)));
    }
 
 
 
+   /*******************************************************************************
+    ** The population row total is exact even when other metrics are estimates.
+    *******************************************************************************/
+   public static void markEstimates(List<Metric> metrics)
+   {
+      if(metrics != null)
+      {
+         metrics.stream().filter(m -> !"rows".equals(m.getKey())).forEach(m ->
+         {
+            m.withEstimate(true);
+            if(m.getFormatted() != null)
+            {
+               m.withFormatted("≈ " + m.getFormatted());
+            }
+         });
+      }
+   }
+
    /*******************************************************************************
     ** PVS translations run as the current user; drill values always remain ids.
     *******************************************************************************/
@@ -98,7 +121,7 @@ public final class ProfileSupport
                label = translated;
             }
          }
-         result.add(new ValueCount().withValue(group.value()).withLabel(label).withCount(group.count()).withShare(share(group.count(), total))
+         result.add(new ValueCount().withValue(group.value()).withLabel(label).withCount(group.count()).withShare(group.share() == null ? share(group.count(), total) : AnalysisFormatting.ratio(group.share()))
             .withDrill(DrillFilters.drill(table, filter, group.condition())));
       }
       return (result);

src/main/java/com/kingsrook/qbits/dataanalysis/engine/StandInAnalysisBackend.java

@@ -68,7 +68,8 @@ import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
 public class StandInAnalysisBackend implements AnalysisBackend
 {
    private static final MathContext MATH = MathContext.DECIMAL128;
-   private final EffectiveConfig config;
+   protected final EffectiveConfig config;
+   protected boolean estimating;
 
    /*******************************************************************************
     ** Bind one request's detached configuration.
@@ -94,13 +95,14 @@ public class StandInAnalysisBackend implements AnalysisBackend
 
 
    /*******************************************************************************
-    ** A boundary query selects the prefix retained in every drill.
+    ** A single boundary parameter bounds the prefix used for computation.
     *******************************************************************************/
    @Override
    public Scope scope(String table, QQueryFilter filter, boolean exact) throws QException
    {
       long total = count(table, filter);
-      boolean estimate = !exact && total > config.getSampleThresholdRows();
+      boolean estimate = !exact && total > config.getSampleThresholdRows() && total > Math.min(config.getSampleRows(), config.getMaxRows());
+      estimating = estimate;
       int limit = estimate ? Math.min(config.getSampleRows(), config.getMaxRows()) : config.getMaxRows();
       QQueryFilter scoped = DrillFilters.copy(filter);
       if(total > limit)
@@ -108,7 +110,7 @@ public class StandInAnalysisBackend implements AnalysisBackend
          String pk = primaryKey(table);
          if(limit == 0)
          {
-            scoped = DrillFilters.and(scoped, DrillFilters.in(pk, List.of()));
+            scoped = DrillFilters.and(scoped, DrillFilters.condition(pk, QCriteriaOperator.IS_BLANK));
          }
          else
          {
@@ -141,7 +143,7 @@ public class StandInAnalysisBackend implements AnalysisBackend
    /*******************************************************************************
     ** Bound both page size and retained records.
     *******************************************************************************/
-   private List<QRecord> rows(String table, QQueryFilter filter, Set<String> fields, int limit) throws QException
+   protected List<QRecord> rows(String table, QQueryFilter filter, Set<String> fields, int limit) throws QException
    {
       List<QRecord> records = new ArrayList<>();
       int bound = Math.min(limit, config.getMaxRows());
@@ -163,7 +165,7 @@ public class StandInAnalysisBackend implements AnalysisBackend
    /*******************************************************************************
     ** Core applies table permissions, field locks and record locks.
     *******************************************************************************/
-   private List<AggregateResult> aggregate(AggregateInput input) throws QException
+   protected List<AggregateResult> aggregate(AggregateInput input) throws QException
    {
       QQueryFilter restored = DrillFilters.restoreTypes(input.getTableName(), input.getFilter());
       restored.setOrderBys(input.getFilter().getOrderBys());
@@ -191,6 +193,7 @@ public class StandInAnalysisBackend implements AnalysisBackend
          {
             input.withAggregate(countOf(field, AggregateOperator.COUNT)).withAggregate(countOf(field, AggregateOperator.COUNT_DISTINCT));
          }
+         fieldCountAggregates(input, batch);
          List<AggregateResult> aggregates = aggregate(input);
          Set<String> selection = new LinkedHashSet<>(batch);
          selection.add(primaryKey(table));
@@ -198,7 +201,7 @@ public class StandInAnalysisBackend implements AnalysisBackend
          for(String field : batch)
          {
             boolean string = field(table, field).getType().isStringLike();
-            long nulls = count(table, DrillFilters.and(filter, DrillFilters.nulls(field, string)));
+            long nulls = fieldCount(table, filter, field, AggregateOperator.COUNT_NULL, aggregates, DrillFilters.nulls(field, string));
             Map<Serializable, Serializable> representatives = new LinkedHashMap<>();
             Set<Serializable> blanks = new LinkedHashSet<>();
             for(QRecord record : records)
@@ -220,12 +223,12 @@ public class StandInAnalysisBackend implements AnalysisBackend
                   .withSubFilter(DrillFilters.in(field, new ArrayList<>(blanks)));
             }
             long distinct = aggregates.isEmpty() ? 0 : number(aggregates.getFirst(), field, AggregateOperator.COUNT_DISTINCT).longValue();
-            if(isMemory(table) && field(table, field).getType() == QFieldType.DECIMAL)
+            if(!estimating && isMemory(table) && field(table, field).getType() == QFieldType.DECIMAL)
             {
                distinct = representatives.size();
             }
-            result.put(field, new FieldCounts(total - nulls, nulls, string ? count(table, DrillFilters.and(filter, blankFilter)) : nulls,
-               distinct, blankFilter, DrillFilters.in(primaryKey(table), new ArrayList<>(representatives.values()))));
+            result.put(field, new FieldCounts(total - nulls, nulls, string ? fieldCount(table, filter, field, AggregateOperator.COUNT_BLANK, aggregates, blankFilter) : nulls,
+               distinct, blankFilter, estimating ? DrillFilters.nonNull(field, string) : DrillFilters.in(primaryKey(table), new ArrayList<>(representatives.values()))));
          }
       }
       return (result);
@@ -233,6 +236,23 @@ public class StandInAnalysisBackend implements AnalysisBackend
 
 
 
+   /*******************************************************************************
+    ** Subclasses may extend the existing batch with native null/blank counts.
+    *******************************************************************************/
+   protected void fieldCountAggregates(AggregateInput input, List<String> fields) throws QException
+   {
+      // The portable backend counts null and blank predicates separately.
+   }
+
+   /*******************************************************************************
+    ** Count the exact drill predicate when native counters are unavailable.
+    *******************************************************************************/
+   protected long fieldCount(String table, QQueryFilter filter, String field, AggregateOperator operator,
+      List<AggregateResult> results, QQueryFilter condition) throws QException
+   {
+      return (count(table, DrillFilters.and(filter, condition)));
+   }
+
    /*******************************************************************************
     ** Push down group, count, ordering and pagination.
     *******************************************************************************/
@@ -254,6 +274,30 @@ public class StandInAnalysisBackend implements AnalysisBackend
 
 
 
+   /*******************************************************************************
+    ** Random backends scale a count using the current aggregate's sample size.
+    *******************************************************************************/
+   protected long aggregateCount(long value)
+   {
+      return (value);
+   }
+
+   /*******************************************************************************
+    ** Random backends retain the raw sample ratio before rounding counts.
+    *******************************************************************************/
+   protected BigDecimal aggregateShare(long value)
+   {
+      return (null);
+   }
+
+   /*******************************************************************************
+    ** Exact and prefix groups derive shares from the profiler's row count.
+    *******************************************************************************/
+   protected BigDecimal countShare()
+   {
+      return (null);
+   }
+
    /*******************************************************************************
     ** Tuple drills use IN for non-null values and null-only predicates otherwise.
     *******************************************************************************/
@@ -270,7 +314,8 @@ public class StandInAnalysisBackend implements AnalysisBackend
       Map<Serializable, GroupCount> merged = new LinkedHashMap<>();
       for(AggregateResult row : aggregate(input))
       {
-         long size = number(row, primaryKey(table), AggregateOperator.COUNT).longValue();
+         long rawSize = number(row, primaryKey(table), AggregateOperator.COUNT).longValue();
+         long size = aggregateCount(rawSize);
          LinkedHashMap<String, Serializable> tuple = new LinkedHashMap<>();
          QQueryFilter condition = new QQueryFilter();
          for(String field : fields)
@@ -278,7 +323,7 @@ public class StandInAnalysisBackend implements AnalysisBackend
             Serializable value = row.getGroupByValues().entrySet().stream().filter(e -> field.equals(e.getKey().getFieldName())).findFirst().orElseThrow().getValue();
             tuple.put(field, value instanceof BigDecimal d ? new BigDecimal(d.stripTrailingZeros().toPlainString()) : value);
             condition.withSubFilter(value == null ? DrillFilters.nulls(field, field(table, field).getType().isStringLike())
-               : value instanceof BigDecimal ? DrillFilters.in(field, List.of(value.toString()))
+               : value instanceof BigDecimal && !estimating ? DrillFilters.in(field, List.of(value.toString()))
                : duplicates ? DrillFilters.in(field, List.of(value)) : DrillFilters.value(field, value, field(table, field).getType().isStringLike()));
          }
          Serializable value = duplicates ? tuple : tuple.get(fields.getFirst());
@@ -288,7 +333,7 @@ public class StandInAnalysisBackend implements AnalysisBackend
             size += previous.count();
             condition = new QQueryFilter().withBooleanOperator(QQueryFilter.BooleanOperator.OR).withSubFilter(previous.condition()).withSubFilter(condition);
          }
-         merged.put(value, new GroupCount(value, value == null ? "(null)" : value.toString(), size, condition));
+         merged.put(value, new GroupCount(value, value == null ? "(null)" : value.toString(), size, condition, aggregateShare(rawSize)));
       }
       List<GroupCount> groups = new ArrayList<>(merged.values());
       if(isMemory(table))
@@ -308,14 +353,8 @@ public class StandInAnalysisBackend implements AnalysisBackend
    {
       List<BigDecimal> values = rows(table, filter, Set.of(field), config.getMaxRows()).stream().map(r -> r.getValue(field))
          .filter(Objects::nonNull).map(v -> new BigDecimal(v.toString())).sorted().toList();
-      Map<String, BigDecimal> metrics = new LinkedHashMap<>();
-      AggregateInput input = new AggregateInput(table).withFilter(DrillFilters.copy(filter));
-      for(AggregateOperator op : List.of(AggregateOperator.MIN, AggregateOperator.MAX, AggregateOperator.AVG, AggregateOperator.SUM))
-      {
-         input.withAggregate(new Aggregate(field, op));
-      }
-      List<AggregateResult> results = aggregate(input);
-      if(values.isEmpty())
+      Map<String, BigDecimal> metrics = numericMetrics(table, filter, field, percentiles, values);
+      if(metrics.isEmpty() || metrics.get("min") == null)
       {
          for(String key : List.of("min", "max", "mean", "stddev", "sum"))
          {
@@ -325,23 +364,12 @@ public class StandInAnalysisBackend implements AnalysisBackend
          {
             metrics.put("p" + p.multiply(BigDecimal.valueOf(100)).stripTrailingZeros().toPlainString(), null);
          }
-         QQueryFilter empty = DrillFilters.in(primaryKey(table), List.of());
-         return (new NumericSummary(metrics, List.of(), 0, empty, List.of(new GroupCount("zeros", "zeros", 0, empty), new GroupCount("negatives", "negatives", 0, empty))));
+         QQueryFilter empty = DrillFilters.condition(primaryKey(table), QCriteriaOperator.IS_BLANK);
+         return (new NumericSummary(metrics, List.of(), 0, empty, List.of(new GroupCount("zeros", "zeros", 0, estimating ? DrillFilters.condition(field, QCriteriaOperator.EQUALS, BigDecimal.ZERO) : empty),
+            new GroupCount("negatives", "negatives", 0, estimating ? DrillFilters.range(field, null, BigDecimal.ZERO, false) : empty))));
       }
-      AggregateResult result = results.getFirst();
-      metrics.put("min", number(result, field, AggregateOperator.MIN));
-      metrics.put("max", number(result, field, AggregateOperator.MAX));
-      metrics.put("mean", number(result, field, AggregateOperator.AVG));
-      metrics.put("sum", number(result, field, AggregateOperator.SUM));
-      BigDecimal mean = values.stream().reduce(BigDecimal.ZERO, BigDecimal::add).divide(BigDecimal.valueOf(values.size()), MATH);
-      BigDecimal variance = values.stream().map(v -> v.subtract(mean).pow(2)).reduce(BigDecimal.ZERO, BigDecimal::add);
-      metrics.put("stddev", values.size() < 2 ? null : variance.divide(BigDecimal.valueOf(values.size() - 1), MATH).sqrt(MATH));
-      for(BigDecimal p : percentiles)
-      {
-         metrics.put("p" + p.multiply(BigDecimal.valueOf(100)).stripTrailingZeros().toPlainString(), percentile(values, p));
-      }
-      BigDecimal q1 = percentile(values, new BigDecimal("0.25"));
-      BigDecimal q3 = percentile(values, new BigDecimal("0.75"));
+      BigDecimal q1 = metrics.containsKey("p25") ? metrics.get("p25") : percentile(values, new BigDecimal("0.25"));
+      BigDecimal q3 = metrics.containsKey("p75") ? metrics.get("p75") : percentile(values, new BigDecimal("0.75"));
       BigDecimal span = q3.subtract(q1).multiply(new BigDecimal("1.5"));
       BigDecimal low = q1.subtract(span);
       BigDecimal high = q3.add(span);
@@ -355,10 +383,10 @@ public class StandInAnalysisBackend implements AnalysisBackend
       List<GroupCount> extremeGroups = new ArrayList<>();
       for(BigDecimal value : extremes)
       {
-         QQueryFilter condition = DrillFilters.in(field, equalNumbers(values, value));
-         extremeGroups.add(new GroupCount(value, value.toPlainString(), count(table, DrillFilters.and(filter, condition)), condition));
+         QQueryFilter condition = estimating ? DrillFilters.condition(field, QCriteriaOperator.EQUALS, value) : DrillFilters.in(field, equalNumbers(values, value));
+         extremeGroups.add(new GroupCount(value, value.toPlainString(), count(table, DrillFilters.and(filter, condition)), condition, countShare()));
       }
-      QQueryFilter zeros = DrillFilters.in(field, equalNumbers(values, BigDecimal.ZERO));
+      QQueryFilter zeros = estimating ? DrillFilters.condition(field, QCriteriaOperator.EQUALS, BigDecimal.ZERO) : DrillFilters.in(field, equalNumbers(values, BigDecimal.ZERO));
       QQueryFilter negatives = numericRange(table, field, values, null, BigDecimal.ZERO, false);
       return (new NumericSummary(metrics, extremeGroups, count(table, DrillFilters.and(filter, outliers)), outliers,
          List.of(bucket(table, filter, "zeros", zeros), bucket(table, filter, "negatives", negatives))));
@@ -366,6 +394,39 @@ public class StandInAnalysisBackend implements AnalysisBackend
 
 
 
+   /*******************************************************************************
+    ** Portable numeric moments; core-backed subclasses replace supported work.
+    *******************************************************************************/
+   protected Map<String, BigDecimal> numericMetrics(String table, QQueryFilter filter, String field, List<BigDecimal> percentiles, List<BigDecimal> values) throws QException
+   {
+      Map<String, BigDecimal> metrics = new LinkedHashMap<>();
+      if(values.isEmpty())
+      {
+         return (metrics);
+      }
+      AggregateInput input = new AggregateInput(table).withFilter(DrillFilters.copy(filter));
+      for(AggregateOperator op : List.of(AggregateOperator.MIN, AggregateOperator.MAX, AggregateOperator.AVG, AggregateOperator.SUM))
+      {
+         input.withAggregate(new Aggregate(field, op));
+      }
+      List<AggregateResult> results = aggregate(input);
+      AggregateResult result = results.getFirst();
+      metrics.put("min", number(result, field, AggregateOperator.MIN));
+      metrics.put("max", number(result, field, AggregateOperator.MAX));
+      metrics.put("mean", number(result, field, AggregateOperator.AVG));
+      metrics.put("sum", number(result, field, AggregateOperator.SUM));
+      BigDecimal mean = values.stream().reduce(BigDecimal.ZERO, BigDecimal::add).divide(BigDecimal.valueOf(values.size()), MATH);
+      BigDecimal variance = values.stream().map(v -> v.subtract(mean).pow(2)).reduce(BigDecimal.ZERO, BigDecimal::add);
+      metrics.put("stddev", values.size() < 2 ? null : variance.divide(BigDecimal.valueOf(values.size() - 1), MATH).sqrt(MATH));
+      for(BigDecimal p : percentiles)
+      {
+         metrics.put("p" + p.multiply(BigDecimal.valueOf(100)).stripTrailingZeros().toPlainString(), percentile(values, p));
+      }
+      return (metrics);
+   }
+
+
+
    /*******************************************************************************
     ** PostgreSQL percentile_cont interpolation at p * (n - 1).
     *******************************************************************************/
@@ -394,17 +455,21 @@ public class StandInAnalysisBackend implements AnalysisBackend
       {
          return (result);
       }
+      NiceBins nice = NiceBins.covering(low, high);
+      low = nice.low();
+      high = nice.high();
+      bins = nice.count();
       List<BigDecimal> values = rows(table, filter, Set.of(field), config.getMaxRows()).stream().map(r -> r.getValue(field))
          .filter(Objects::nonNull).map(v -> new BigDecimal(v.toString())).toList();
-      result.add(bucket(table, filter, "Below " + low, numericRange(table, field, values, null, low, false)));
+      result.add(bucket(table, filter, AnalysisFormatting.interval(field(table, field), null, low), numericRange(table, field, values, null, low, false)));
       BigDecimal width = high.subtract(low).divide(BigDecimal.valueOf(bins), MATH);
       for(int i = 0; i < bins; i++)
       {
          BigDecimal start = low.add(width.multiply(BigDecimal.valueOf(i)));
          BigDecimal end = i == bins - 1 ? high : low.add(width.multiply(BigDecimal.valueOf(i + 1)));
-         result.add(bucket(table, filter, "[" + start + ", " + end + (i == bins - 1 ? "]" : ")"), numericRange(table, field, values, start, end, i == bins - 1)));
+         result.add(bucket(table, filter, AnalysisFormatting.interval(field(table, field), start, end), numericRange(table, field, values, start, end, i == bins - 1)));
       }
-      result.add(bucket(table, filter, "Above " + high, DrillFilters.and(DrillFilters.condition(field, QCriteriaOperator.IS_NOT_BLANK), DrillFilters.condition(field, QCriteriaOperator.GREATER_THAN, high))));
+      result.add(bucket(table, filter, AnalysisFormatting.interval(field(table, field), high, null), DrillFilters.and(DrillFilters.condition(field, QCriteriaOperator.IS_NOT_BLANK), DrillFilters.condition(field, QCriteriaOperator.GREATER_THAN, high))));
       return (result);
    }
 
@@ -415,7 +480,7 @@ public class StandInAnalysisBackend implements AnalysisBackend
     *******************************************************************************/
    private GroupCount bucket(String table, QQueryFilter filter, String label, QQueryFilter condition) throws QException
    {
-      return (new GroupCount(label, label, count(table, DrillFilters.and(filter, condition)), condition));
+      return (new GroupCount(label, label, count(table, DrillFilters.and(filter, condition)), condition, countShare()));
    }
 
 
@@ -430,7 +495,26 @@ public class StandInAnalysisBackend implements AnalysisBackend
       if(unit == DateUnit.DAY_OF_WEEK || unit == DateUnit.HOUR_OF_DAY)
       {
          return (mappedGroups(table, filter, field, records, v -> unit == DateUnit.DAY_OF_WEEK ? day(v, zone).getDayOfWeek().getValue()
-            : ((Instant) v).atZone(zone).getHour(), 24));
+            : ((Instant) v).atZone(zone).getHour(), 24).stream()
+            .map(g -> new GroupCount(g.value(), AnalysisFormatting.dateBucket(g.value(), unit, zone), g.count(), g.condition(), g.share())).toList());
+      }
+      if(unit == DateUnit.HOUR)
+      {
+         List<Instant> instants = records.stream().map(r -> (Instant) r.getValue(field)).filter(Objects::nonNull).sorted().toList();
+         List<GroupCount> hours = new ArrayList<>();
+         if(!instants.isEmpty())
+         {
+            Instant start = instants.getFirst().atZone(zone).truncatedTo(java.time.temporal.ChronoUnit.HOURS).toInstant();
+            while(!start.isAfter(instants.getLast()))
+            {
+               Instant end = start.plus(1, java.time.temporal.ChronoUnit.HOURS);
+               QQueryFilter condition = DrillFilters.and(DrillFilters.condition(field, QCriteriaOperator.GREATER_THAN_OR_EQUALS, start),
+                  DrillFilters.condition(field, QCriteriaOperator.LESS_THAN, end));
+               hours.add(new GroupCount(start, AnalysisFormatting.dateBucket(start, unit, zone), count(table, DrillFilters.and(filter, condition)), condition, countShare()));
+               start = end;
+            }
+         }
+         return (hours);
       }
       List<LocalDate> dates = records.stream().map(r -> r.getValue(field)).filter(Objects::nonNull).map(v -> day(v, zone)).sorted().toList();
       List<GroupCount> result = new ArrayList<>();
@@ -443,7 +527,8 @@ public class StandInAnalysisBackend implements AnalysisBackend
       while(!start.isAfter(last))
       {
          LocalDate end = next(start, unit);
-         result.add(bucket(table, filter, start + " / " + end, DrillFilters.dateRange(field, start, end, field(table, field).getType() == QFieldType.DATE_TIME, zone)));
+         QQueryFilter condition = DrillFilters.dateRange(field, start, end, field(table, field).getType() == QFieldType.DATE_TIME, zone);
+         result.add(new GroupCount(start, AnalysisFormatting.dateBucket(start, unit, zone), count(table, DrillFilters.and(filter, condition)), condition, countShare()));
          start = end;
       }
       return (result);
@@ -464,12 +549,13 @@ public class StandInAnalysisBackend implements AnalysisBackend
    /*******************************************************************************
     ** ISO weeks begin Monday.
     *******************************************************************************/
-   private static LocalDate floor(LocalDate date, DateUnit unit)
+   protected static LocalDate floor(LocalDate date, DateUnit unit)
    {
       return (switch(unit)
       {
          case WEEK -> date.with(TemporalAdjusters.previousOrSame(DayOfWeek.MONDAY));
          case MONTH -> date.withDayOfMonth(1);
+         case QUARTER -> date.withMonth((date.getMonthValue() - 1) / 3 * 3 + 1).withDayOfMonth(1);
          case YEAR -> date.withDayOfYear(1);
          default -> date;
       });
@@ -480,12 +566,13 @@ public class StandInAnalysisBackend implements AnalysisBackend
    /*******************************************************************************
     ** Calendar arithmetic handles leap years and daylight savings.
     *******************************************************************************/
-   private static LocalDate next(LocalDate date, DateUnit unit)
+   protected static LocalDate next(LocalDate date, DateUnit unit)
    {
       return (switch(unit)
       {
          case WEEK -> date.plusWeeks(1);
          case MONTH -> date.plusMonths(1);
+         case QUARTER -> date.plusMonths(3);
          case YEAR -> date.plusYears(1);
          default -> date.plusDays(1);
       });
@@ -506,7 +593,13 @@ public class StandInAnalysisBackend implements AnalysisBackend
       result.put("max", null);
       for(AggregateResult row : aggregate(input))
       {
-         row.getAggregateValues().forEach((key, value) -> result.put(key.getOperator().name().toLowerCase(java.util.Locale.ROOT), value));
+         row.getAggregateValues().forEach((key, value) ->
+         {
+            if(key.getOperator() == AggregateOperator.MIN || key.getOperator() == AggregateOperator.MAX)
+            {
+               result.put(key.getOperator().name().toLowerCase(java.util.Locale.ROOT), value);
+            }
+         });
       }
       return (result);
    }
@@ -538,12 +631,14 @@ public class StandInAnalysisBackend implements AnalysisBackend
     ** Shapes use a primary-key prefix of at most the requested sample size.
     *******************************************************************************/
    @Override
-   public List<GroupCount> patterns(String table, QQueryFilter filter, String field, int sampleRows) throws QException
+   public PatternSummary patterns(String table, QQueryFilter filter, String field, int sampleRows) throws QException
    {
-      List<QRecord> records = rows(table, filter, Set.of(field, primaryKey(table)), sampleRows);
+      List<QRecord> records = rows(table, DrillFilters.and(filter, DrillFilters.nonNull(field, true)), Set.of(field, primaryKey(table)), sampleRows + 1);
+      boolean truncated = records.size() > sampleRows;
+      records = records.subList(0, Math.min(records.size(), sampleRows));
       QQueryFilter sampled = records.isEmpty() ? DrillFilters.in(primaryKey(table), List.of())
          : DrillFilters.condition(primaryKey(table), QCriteriaOperator.LESS_THAN_OR_EQUALS, records.getLast().getValue(primaryKey(table)));
-      return (mappedGroups(table, DrillFilters.and(filter, sampled), field, records, v -> shape(v.toString()), 10));
+      return (new PatternSummary(mappedGroups(table, estimating ? filter : DrillFilters.and(filter, sampled), field, records, v -> shape(v.toString()), 10), records.size(), truncated));
    }
 
 
@@ -575,8 +670,8 @@ public class StandInAnalysisBackend implements AnalysisBackend
       List<GroupCount> groups = new ArrayList<>();
       for(Map.Entry<Serializable, Set<Serializable>> entry : members.entrySet())
       {
-         QQueryFilter condition = DrillFilters.and(filter, DrillFilters.in(field, new ArrayList<>(entry.getValue())));
-         groups.add(new GroupCount(entry.getKey(), entry.getKey().toString(), count(table, condition), condition));
+         QQueryFilter condition = DrillFilters.in(field, new ArrayList<>(entry.getValue()));
+         groups.add(new GroupCount(entry.getKey(), entry.getKey().toString(), count(table, DrillFilters.and(filter, condition)), estimating ? condition : DrillFilters.and(filter, condition), countShare()));
       }
       return (groups.stream().sorted(Comparator.comparingLong(GroupCount::count).reversed().thenComparing(GroupCount::label)).limit(limit).toList());
    }
@@ -597,7 +692,7 @@ public class StandInAnalysisBackend implements AnalysisBackend
    /*******************************************************************************
     ** Metadata lookup follows facade security checks.
     *******************************************************************************/
-   private static QFieldMetaData field(String table, String field)
+   protected static QFieldMetaData field(String table, String field)
    {
       return (QContext.getQInstance().getTable(table).getField(field));
    }
@@ -607,7 +702,7 @@ public class StandInAnalysisBackend implements AnalysisBackend
    /*******************************************************************************
     ** Deterministic paging requires an addressable primary key.
     *******************************************************************************/
-   private static String primaryKey(String table)
+   protected static String primaryKey(String table)
    {
       return (QContext.getQInstance().getTable(table).getPrimaryKeyField());
    }
@@ -618,7 +713,7 @@ public class StandInAnalysisBackend implements AnalysisBackend
     ** COUNT and COUNT_DISTINCT always read as integers: RC.5's RDBMS reader
     ** otherwise reads them as the field's type (a boolean count fails).
     *******************************************************************************/
-   private static Aggregate countOf(String field, AggregateOperator op)
+   protected static Aggregate countOf(String field, AggregateOperator op)
    {
       return (new Aggregate(field, op).withFieldType(QFieldType.INTEGER));
    }
@@ -628,7 +723,7 @@ public class StandInAnalysisBackend implements AnalysisBackend
    /*******************************************************************************
     ** The value of one aggregate of a result row (0 when absent).
     *******************************************************************************/
-   private static BigDecimal number(AggregateResult row, String field, AggregateOperator op)
+   protected static BigDecimal number(AggregateResult row, String field, AggregateOperator op)
    {
       Serializable value = row.getAggregateValues().entrySet().stream().filter(e -> field.equals(e.getKey().getFieldName()) && op == e.getKey().getOperator())
          .map(Map.Entry::getValue).filter(Objects::nonNull).findFirst().orElse(0);
@@ -642,9 +737,9 @@ public class StandInAnalysisBackend implements AnalysisBackend
     ** Use strict comparison plus actual boundary spellings to preserve half-open
     ** membership. SQL backends use ordinary range predicates.
     *******************************************************************************/
-   private static QQueryFilter numericRange(String table, String field, List<BigDecimal> values, BigDecimal low, BigDecimal high, boolean closed)
+   protected QQueryFilter numericRange(String table, String field, List<BigDecimal> values, BigDecimal low, BigDecimal high, boolean closed)
    {
-      if(!isMemory(table))
+      if(estimating || !isMemory(table))
       {
          return (DrillFilters.range(field, low, high, closed));
       }
@@ -682,7 +777,7 @@ public class StandInAnalysisBackend implements AnalysisBackend
    /*******************************************************************************
     ** Isolate compatibility fixes to the RC.5 memory implementation.
     *******************************************************************************/
-   private static boolean isMemory(String table)
+   protected static boolean isMemory(String table)
    {
       return ("memory".equals(QContext.getQInstance().getBackend(QContext.getQInstance().getTable(table).getBackendName()).getBackendType()));
    }

src/main/java/com/kingsrook/qbits/dataanalysis/engine/TableProfiler.java

@@ -60,22 +60,23 @@ public class TableProfiler
       String table = request.getTableName();
       QQueryFilter filter = scope.filter();
       long rows = backend.count(table, filter);
+      QQueryFilter drillFilter = scope.estimate() ? request.getFilter() : filter;
       TableProfile result = new TableProfile().withTableName(table).withFilter(DrillFilters.restoreTypes(request.getTableName(), request.getFilter())).withEstimate(scope.estimate())
-         .withSampleRows(scope.estimate() ? rows : 0L).withRowsScanned(scope.rowsScanned()).withTruncated(scope.truncated()).withAsOf(Instant.now())
-         .withRowCount(ProfileSupport.count("rows", rows, table, filter, null));
+         .withSampleRows(scope.estimate() ? scope.rowsScanned() : 0L).withRowsScanned(scope.rowsScanned()).withTruncated(scope.truncated()).withAsOf(Instant.now())
+         .withRowCount(ProfileSupport.count("rows", scope.estimate() ? scope.total() : rows, table, drillFilter, null));
       Map<String, AnalysisBackend.FieldCounts> counts = backend.fieldCounts(table, filter, fields.stream().map(QFieldMetaData::getName).toList());
       for(QFieldMetaData field : fields)
       {
          String name = field.getName();
          AnalysisBackend.FieldCounts c = counts.get(name);
          result.getFields().add(new FieldSummary().withField(name).withLabel(field.getLabel()).withType(field.getType().name())
-            .withCompleteness(ProfileSupport.share(rows - c.blanks(), rows)).withDistinct(c.distinct()).withUniqueness(ProfileSupport.share(c.distinct(), c.nonNull()))
-            .withConstant(c.distinct() == 1).withEmpty(c.blanks() == rows)
-            .withNulls(ProfileSupport.count("nulls", c.nulls(), table, filter, DrillFilters.nulls(name, field.getType().isStringLike())))
-            .withBlanks(ProfileSupport.count("blanks", c.blanks(), table, filter, c.blankFilter())));
+            .withCompleteness(AnalysisFormatting.ratio(c.completeness())).withDistinct(c.distinct()).withUniqueness(AnalysisFormatting.ratio(c.uniqueness()))
+            .withConstant(c.constant()).withEmpty(c.blanks() == rows)
+            .withNulls(ProfileSupport.count("nulls", c.nulls(), table, drillFilter, DrillFilters.nulls(name, field.getType().isStringLike())))
+            .withBlanks(ProfileSupport.count("blanks", c.blanks(), table, drillFilter, c.blankFilter())));
          if(rows > 0 && c.distinct() == rows && c.nulls() == 0)
          {
-            result.getCandidateKeys().add(key(table, filter, List.of(name), rows));
+            result.getCandidateKeys().add(key(table, drillFilter, List.of(name), rows));
          }
       }
       List<FieldSummary> ranked = result.getFields().stream().sorted(Comparator.comparing(FieldSummary::getUniqueness).reversed().thenComparing(FieldSummary::getField)).limit(8).toList();
@@ -86,7 +87,7 @@ public class TableProfiler
             List<String> pair = List.of(ranked.get(i).getField(), ranked.get(j).getField());
             if(rows > 0 && pair.stream().allMatch(f -> counts.get(f).nulls() == 0) && backend.duplicates(table, filter, pair, 1).isEmpty())
             {
-               result.getCandidateKeys().add(key(table, filter, pair, rows));
+               result.getCandidateKeys().add(key(table, drillFilter, pair, rows));
             }
          }
       }
@@ -97,7 +98,12 @@ public class TableProfiler
       {
          result.getHeatmap().getSeries().getFirst().getPoints().add(new Point().withX(field.getField()).withLabel(field.getLabel()).withY(field.getCompleteness()));
       }
-      if(scope.estimate() || scope.truncated())
+      if(scope.estimate())
+      {
+         result.getFields().forEach(f -> ProfileSupport.markEstimates(List.of(f.getNulls(), f.getBlanks())));
+         result.getWarnings().add(scope.warning());
+      }
+      else if(scope.truncated())
       {
          result.getWarnings().add("Profiles and candidate keys describe only the first " + rows + " rows by primary key.");
       }

src/main/java/com/kingsrook/qbits/dataanalysis/permissions/DataAnalysisPermissions.java

@@ -27,12 +27,11 @@ import com.kingsrook.qqq.backend.core.context.QContext;
 import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
 import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRules;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
-import com.kingsrook.qqq.backend.core.model.session.QSession;
 import com.kingsrook.qqq.backend.core.utils.StringUtils;
 
 
 /*******************************************************************************
- ** Data Analysis permissions and the temporary per-table feature check.
+ ** Data Analysis permissions through the core table feature registry.
  *******************************************************************************/
 public final class DataAnalysisPermissions
 {
@@ -68,9 +67,8 @@ public final class DataAnalysisPermissions
     *******************************************************************************/
    public static boolean canAnalyze(QTableMetaData table)
    {
-      QSession session = QContext.getQSession();
       boolean read = !Boolean.TRUE.equals(QContext.getQInstance().getEnforcePermissionsInCoreActions())
          || PermissionsHelper.hasTablePermission(new QueryInput(table.getName()), table.getName(), TablePermissionSubType.READ);
-      return (read && session != null && session.hasPermission(analyzePermissionName(table)));
+      return (read && PermissionsHelper.hasTableFeaturePermission(QContext.getQInstance(), QContext.getQSession(), table, "analyze"));
    }
 }

src/main/java/com/kingsrook/qbits/dataanalysis/result/DuplicatesRequest.java

@@ -36,6 +36,7 @@ public class DuplicatesRequest implements AnalysisRequest
    private QQueryFilter filter;
    private Boolean exact = false;
    private List<String> fields;
+   private Integer limit = 50;
 
 
 
@@ -120,4 +121,21 @@ public class DuplicatesRequest implements AnalysisRequest
       this.fields = fields;
       return (this);
    }
+
+   /*******************************************************************************
+    ** Maximum duplicate groups requested.
+    *******************************************************************************/
+   public Integer getLimit()
+   {
+      return (limit);
+   }
+
+   /*******************************************************************************
+    ** Fluent duplicate group limit.
+    *******************************************************************************/
+   public DuplicatesRequest withLimit(Integer limit)
+   {
+      this.limit = limit;
+      return (this);
+   }
 }

src/main/java/com/kingsrook/qbits/dataanalysis/result/Metric.java

@@ -31,6 +31,7 @@ import com.fasterxml.jackson.annotation.JsonAutoDetect;
 @JsonAutoDetect(fieldVisibility = JsonAutoDetect.Visibility.ANY)
 public class Metric implements Serializable
 {
+   private Boolean estimate = false;
    private String key;
    private String label;
    private Serializable value;
@@ -42,6 +43,23 @@ public class Metric implements Serializable
 
 
 
+   /*******************************************************************************
+    ** Whether the metric describes a sampled estimate.
+    *******************************************************************************/
+   public Boolean getEstimate()
+   {
+      return (estimate);
+   }
+
+   /*******************************************************************************
+    ** Mark an estimated metric independently of the exact population row total.
+    *******************************************************************************/
+   public Metric withEstimate(Boolean estimate)
+   {
+      this.estimate = estimate;
+      return (this);
+   }
+
    /*******************************************************************************
     ** Getter for key.
     *******************************************************************************/

src/main/java/com/kingsrook/qbits/dataanalysis/run/AnalysisRunService.java

@@ -96,7 +96,7 @@ public class AnalysisRunService
       AnalysisBackend.Scope scope = backend.scope(request.getTableName(), request.getFilter(), Boolean.TRUE.equals(request.getExact()));
       if(scope.estimate())
       {
-         return (service.compute(request, backend, scope).withDataVersion(DataVersion.of(backend, request.getTableName(), request.getFilter())));
+         return (service.compute(request, backend, scope).withDataVersion(DataVersion.of(service.backend(request.getTableName()), request.getTableName(), request.getFilter())));
       }
       return (exact(request).getColumnProfile());
    }
@@ -113,7 +113,7 @@ public class AnalysisRunService
       AnalysisBackend.Scope scope = backend.scope(request.getTableName(), request.getFilter(), Boolean.TRUE.equals(request.getExact()));
       if(scope.estimate())
       {
-         return (service.compute(request, backend, scope).withDataVersion(DataVersion.of(backend, request.getTableName(), request.getFilter())));
+         return (service.compute(request, backend, scope).withDataVersion(DataVersion.of(service.backend(request.getTableName()), request.getTableName(), request.getFilter())));
       }
       return (exact(request).getTableProfile());
    }

src/test/java/com/kingsrook/qbits/dataanalysis/AnalysisFormattingTest.java

@@ -0,0 +1,158 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.dataanalysis;
+
+import java.math.BigDecimal;
+import java.time.Instant;
+import java.time.LocalDate;
+import java.time.ZoneId;
+import java.util.List;
+import java.util.Map;
+import com.kingsrook.qbits.dataanalysis.engine.AnalysisBackend;
+import com.kingsrook.qbits.dataanalysis.engine.AnalysisFormatting;
+import com.kingsrook.qbits.dataanalysis.engine.MetricLabels;
+import com.kingsrook.qbits.dataanalysis.engine.NiceBins;
+import com.kingsrook.qbits.dataanalysis.engine.ProfileSupport;
+import com.kingsrook.qbits.dataanalysis.result.ValueType;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.DisplayFormat;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
+import org.junit.jupiter.api.Test;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+/*******************************************************************************
+ ** Public presentation contracts, including precision and calendar edge cases.
+ *******************************************************************************/
+class AnalysisFormattingTest
+{
+   /*******************************************************************************
+    ** Raw ratios remain numeric while display strings are ready for a CLI.
+    *******************************************************************************/
+   @Test
+   void numbersPercentagesAndLengths()
+   {
+      assertThat(ProfileSupport.metric("rows", 12498L, ValueType.COUNT).getFormatted()).isEqualTo("12,498");
+      for(String key : List.of("uniqueness", "completeness", "share"))
+      {
+         var metric = ProfileSupport.metric(key, new BigDecimal("0.987654321"), ValueType.PERCENT);
+         assertThat(metric.getValue()).isEqualTo(new BigDecimal("0.987654"));
+         assertThat(metric.getFormatted()).isEqualTo("98.8%");
+         assertThat(metric.getValueType()).isEqualTo(ValueType.PERCENT);
+      }
+      assertThat(ProfileSupport.metric("share", BigDecimal.ONE, ValueType.PERCENT).getFormatted()).isEqualTo("100%");
+      assertThat(ProfileSupport.metric("share", BigDecimal.ZERO, ValueType.PERCENT).getFormatted()).isEqualTo("0%");
+      assertThat(ProfileSupport.share(1, 3)).isEqualByComparingTo("0.333333");
+      assertThat(ProfileSupport.share(0, 0)).isZero();
+      assertThat(ProfileSupport.metric("meanLength", new BigDecimal("12.345"), ValueType.DECIMAL).getFormatted()).isEqualTo("12.3");
+      assertThat(ProfileSupport.metric("minLength", BigDecimal.TEN, ValueType.DECIMAL).getFormatted()).isEqualTo("10");
+      assertThat(ProfileSupport.metric("max", null, ValueType.DECIMAL).getFormatted()).isNull();
+      assertThatThrownBy(() -> MetricLabels.label("unknown", ValueType.COUNT)).isInstanceOf(NullPointerException.class).hasMessageContaining("Missing metric label");
+   }
+
+
+
+   /*******************************************************************************
+    ** Host currency and custom display formats take precedence over numeric defaults.
+    *******************************************************************************/
+   @Test
+   void fieldFormatsAndRawPrecision()
+   {
+      QFieldMetaData decimal = new QFieldMetaData("amount", QFieldType.DECIMAL);
+      assertThat(AnalysisFormatting.numeric(decimal, "min", new BigDecimal("81753.32"))).isEqualTo("81,753.32");
+      decimal.withDisplayFormat(DisplayFormat.CURRENCY);
+      assertThat(AnalysisFormatting.numeric(decimal, "min", new BigDecimal("81753.32"))).isEqualTo("$81,753.32");
+      var metric = AnalysisFormatting.numericMetric(decimal, "mean", new BigDecimal("125916.45167830173"));
+      assertThat(metric.getValue()).isEqualTo(new BigDecimal("125916.4517"));
+      assertThat(metric.getFormatted()).isEqualTo("$125,916.45");
+      assertThat(AnalysisFormatting.numericMetric(decimal, "max", null).getValue()).isNull();
+      QFieldMetaData integer = new QFieldMetaData("quantity", QFieldType.INTEGER);
+      assertThat(AnalysisFormatting.numeric(integer, "max", new BigDecimal("12498"))).isEqualTo("12,498");
+      assertThat(AnalysisFormatting.numeric(integer, "mean", new BigDecimal("12.345"))).isEqualTo("12.35");
+      assertThat(AnalysisFormatting.numeric(integer, "stddev", new BigDecimal("12.345"))).isEqualTo("12.35");
+      assertThat(AnalysisFormatting.numeric(integer.withDisplayFormat(DisplayFormat.DECIMAL1_COMMAS), "mean", new BigDecimal("12.345"))).isEqualTo("12.3");
+      QFieldMetaData large = new QFieldMetaData("large", QFieldType.LONG).withDisplayFormat(DisplayFormat.COMMAS);
+      assertThat(AnalysisFormatting.numeric(large, "sum", new BigDecimal("9000000000"))).isEqualTo("9,000,000,000");
+   }
+
+
+
+   /*******************************************************************************
+    ** Nice boundaries remain exact at small scales, negative values and constants.
+    *******************************************************************************/
+   @Test
+   void niceEdgesAndLabels()
+   {
+      for(List<String> range : List.of(List.of("25317.13", "65781.79"), List.of("-0.0093", "0.0357"), List.of("42", "42"), List.of("0", "0"), List.of("-9", "905")))
+      {
+         BigDecimal low = new BigDecimal(range.getFirst());
+         BigDecimal high = new BigDecimal(range.getLast());
+         NiceBins bins = NiceBins.covering(low, high);
+         assertThat(bins.count()).isBetween(10, 25);
+         assertThat(bins.low()).isLessThanOrEqualTo(low);
+         assertThat(bins.high()).isGreaterThanOrEqualTo(high);
+         BigDecimal step = bins.high().subtract(bins.low()).divide(BigDecimal.valueOf(bins.count())).stripTrailingZeros();
+         assertThat(step.scaleByPowerOfTen(step.scale() - step.precision() + 1).toPlainString()).isIn("1", "2", "2.5", "5");
+         assertThat(bins.low().remainder(step)).isZero();
+         assertThat(bins.high().remainder(step)).isZero();
+      }
+      QFieldMetaData field = new QFieldMetaData("amount", QFieldType.INTEGER);
+      assertThat(AnalysisFormatting.interval(field, new BigDecimal("60000"), new BigDecimal("65000"))).isEqualTo("60,000 – 65,000");
+      assertThat(AnalysisFormatting.interval(field, null, new BigDecimal("25000"))).isEqualTo("Below 25,000");
+      assertThat(AnalysisFormatting.interval(field, new BigDecimal("65000"), null)).isEqualTo("65,000 and above");
+      assertThat(AnalysisFormatting.interval(field, new BigDecimal("1.25"), new BigDecimal("1.5"))).isEqualTo("1.25 – 1.5");
+      field.withDisplayFormat(DisplayFormat.CURRENCY);
+      assertThat(AnalysisFormatting.interval(field, new BigDecimal("60000"), new BigDecimal("65000"))).isEqualTo("$60,000 – $65,000");
+   }
+
+
+
+   /*******************************************************************************
+    ** Every supported unit has a human label without replacing its ISO coordinate.
+    *******************************************************************************/
+   @Test
+   void dateUnitsAndDurations()
+   {
+      ZoneId zone = ZoneId.of("America/Chicago");
+      LocalDate date = LocalDate.of(2024, 3, 4);
+      Map<AnalysisBackend.DateUnit, String> labels = Map.of(
+         AnalysisBackend.DateUnit.YEAR, "2024", AnalysisBackend.DateUnit.QUARTER, "2024 Q1", AnalysisBackend.DateUnit.MONTH, "Mar 2024",
+         AnalysisBackend.DateUnit.WEEK, "Week of 2024-03-04", AnalysisBackend.DateUnit.DAY, "2024-03-04");
+      labels.forEach((unit, label) -> assertThat(AnalysisFormatting.dateBucket(date, unit, zone)).isEqualTo(label));
+      assertThat(AnalysisFormatting.dateBucket(Instant.parse("2024-03-04T19:00:00Z"), AnalysisBackend.DateUnit.HOUR, zone)).isEqualTo("2024-03-04 13:00");
+      for(int i = 1; i <= 7; i++)
+      {
+         assertThat(AnalysisFormatting.dateBucket(i, AnalysisBackend.DateUnit.DAY_OF_WEEK, zone))
+            .isEqualTo(List.of("Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday").get(i - 1));
+      }
+      for(int i = 0; i < 24; i++)
+      {
+         assertThat(AnalysisFormatting.dateBucket(i, AnalysisBackend.DateUnit.HOUR_OF_DAY, zone)).isEqualTo(String.format(java.util.Locale.US, "%02d:00", i));
+      }
+      assertThat(AnalysisFormatting.duration(date, date.plusYears(48).plusMonths(2))).isEqualTo("48 years 2 months");
+      assertThat(AnalysisFormatting.duration(date, date.plusDays(12))).isEqualTo("12 days");
+      assertThat(AnalysisFormatting.duration(date, date.plusYears(1).plusMonths(1).plusDays(1))).isEqualTo("1 year 1 month 1 day");
+      assertThat(AnalysisFormatting.duration(date, date)).isEqualTo("0 days");
+      assertThat(ProfileSupport.metric("min", date, ValueType.DATE).getLabel()).isEqualTo("Earliest");
+      assertThat(ProfileSupport.metric("max", date, ValueType.DATE).getLabel()).isEqualTo("Latest");
+      assertThat(ProfileSupport.metric("min", date, ValueType.DATE).getFormatted()).isEqualTo("2024-03-04");
+   }
+}

src/test/java/com/kingsrook/qbits/dataanalysis/CoreAnalysisBackendTest.java

@@ -0,0 +1,205 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.dataanalysis;
+
+import java.util.Set;
+import com.kingsrook.qbits.dataanalysis.engine.CoreAnalysisBackend;
+import com.kingsrook.qbits.dataanalysis.engine.StandInAnalysisBackend;
+import com.kingsrook.qbits.dataanalysis.result.ColumnProfileRequest;
+import com.kingsrook.qqq.backend.core.actions.interfaces.AggregateInterface;
+import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateCapability;
+import com.kingsrook.qqq.backend.core.model.actions.tables.aggregate.AggregateInput;
+import com.kingsrook.qqq.backend.core.modules.backend.QBackendModuleDispatcher;
+import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryBackendModule;
+import com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryAggregateAction;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Test;
+import static org.assertj.core.api.Assertions.assertThat;
+
+/*******************************************************************************
+ ** Native sample membership and capability-directed fallback behavior.
+ *******************************************************************************/
+class CoreAnalysisBackendTest extends EngineFixture
+{
+   /*******************************************************************************
+    ** Restore the global dispatcher after a capability-specific fixture.
+    *******************************************************************************/
+   @AfterEach
+   void restoreModule()
+   {
+      QBackendModuleDispatcher.registerBackendModule(new MemoryBackendModule());
+   }
+
+   /*******************************************************************************
+    ** Deterministic memory sampling scales every-k-th counts to the population.
+    *******************************************************************************/
+   @Test
+   void sampledPopulation() throws Exception
+   {
+      config.withDefaultSampleThresholdRows(5).withDefaultSampleRows(4);
+      assertThat(service.backend("orders")).isInstanceOf(CoreAnalysisBackend.class);
+      var profile = service.columnProfile(new ColumnProfileRequest().withTableName("orders").withFieldName("amount"));
+      assertThat(profile.getEstimate()).isTrue();
+      assertThat(profile.getSampleRows()).isEqualTo(4);
+      assertThat(profile.getNumeric().getMetrics().stream().filter(m -> m.getKey().equals("sum")).findFirst().orElseThrow().getValue().toString()).isEqualTo("1030");
+      var counts = profile.getCounts().stream().collect(java.util.stream.Collectors.toMap(m -> m.getKey(), m -> m));
+      assertThat(counts.get("rows").getValue()).isEqualTo(12L);
+      assertThat(counts.get("rows").getEstimate()).isFalse();
+      assertThat(counts.get("nonNull").getValue()).isEqualTo(Math.round(4.0 * 12 / 4));
+      assertThat(profile.getTopValues()).allSatisfy(v ->
+      {
+         assertThat(v.getCount()).isEqualTo(Math.round(1.0 * 12 / 4));
+         assertThat(v.getShare()).isEqualByComparingTo("0.25");
+      });
+      assertThat(com.kingsrook.qqq.backend.core.utils.JsonUtils.toJson(profile)).doesNotContain("\"operator\":\"IN\"");
+      new MemoryEngineTest().assertDrills(profile);
+      config.withDefaultMaxRows(0);
+      service.getConfigResolver().invalidate();
+      var empty = service.columnProfile(new ColumnProfileRequest().withTableName("orders").withFieldName("amount"));
+      assertThat(empty.getEstimate()).isTrue();
+      assertThat(empty.getSampleRows()).isEqualTo(4);
+      assertThat(empty.getTruncated()).isFalse();
+   }
+
+   /*******************************************************************************
+    ** Rounding counts must not alter raw shares, and drills retain the user filter.
+    *******************************************************************************/
+   @Test
+   void filteredSampleRatios() throws Exception
+   {
+      config.withDefaultSampleThresholdRows(2).withDefaultSampleRows(4);
+      var filter = com.kingsrook.qbits.dataanalysis.engine.DrillFilters.condition("id",
+         com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator.GREATER_THAN, 1);
+      var profile = service.columnProfile(new ColumnProfileRequest().withTableName("orders").withFieldName("amount").withFilter(filter));
+      // The filtered ids 2..12, sampled every third row, yield 2, 5, 8, 11.
+      var sampledAmounts = java.util.Arrays.asList(0, 20, 40, null);
+      long nonNull = sampledAmounts.stream().filter(java.util.Objects::nonNull).count();
+      var counts = profile.getCounts().stream().collect(java.util.stream.Collectors.toMap(m -> m.getKey(), m -> m));
+      assertThat(profile.getSampleRows()).isEqualTo(sampledAmounts.size());
+      assertThat(counts.get("rows").getValue()).isEqualTo(11L);
+      assertThat(counts.get("nonNull").getValue()).isEqualTo(Math.round(nonNull * 11.0 / sampledAmounts.size()));
+      assertThat(counts.get("nulls").getValue()).isEqualTo(Math.round((sampledAmounts.size() - nonNull) * 11.0 / sampledAmounts.size()));
+      assertThat(profile.getTopValues()).hasSize(4).allSatisfy(v ->
+      {
+         assertThat(v.getCount()).isEqualTo(Math.round(11.0 / sampledAmounts.size()));
+         assertThat(v.getShare()).isEqualByComparingTo("0.25");
+      });
+      var exact = new StandInAnalysisBackend(service.getConfigResolver().resolve("orders"));
+      assertThat(exact.count("orders", counts.get("rows").getDrill().getFilter())).isEqualTo(11);
+      assertThat(exact.count("orders", counts.get("nonNull").getDrill().getFilter())).isEqualTo(10);
+      var twenty = profile.getTopValues().stream().filter(v -> v.getValue() != null && v.getValue().toString().equals("20")).findFirst().orElseThrow();
+      assertThat(exact.count("orders", twenty.getDrill().getFilter())).isEqualTo(2);
+   }
+
+   /*******************************************************************************
+    ** Empty samples keep the population drill; inspecting every row is exact.
+    *******************************************************************************/
+   @Test
+   void samplingBoundaries() throws Exception
+   {
+      config.withDefaultSampleThresholdRows(2).withDefaultSampleRows(0);
+      var empty = service.columnProfile(new ColumnProfileRequest().withTableName("orders").withFieldName("amount"));
+      assertThat(empty.getSampleRows()).isZero();
+      assertThat(empty.getCounts().getFirst().getValue()).isEqualTo(12L);
+      assertThat(empty.getCounts().get(1).getValue()).isEqualTo(0L);
+      config.withDefaultSampleRows(100);
+      service.getConfigResolver().invalidate();
+      var scope = service.backend("orders").scope("orders", null, false);
+      assertThat(scope.samplePercent()).isNull();
+      assertThat(scope.estimate()).isFalse();
+      assertThat(scope.rowsScanned()).isEqualTo(12);
+   }
+
+   /*******************************************************************************
+    ** A non-random estimate has one boundary bind and population-wide drills.
+    *******************************************************************************/
+   @Test
+   void prefixBoundary() throws Exception
+   {
+      capabilities(Set.of());
+      config.withDefaultSampleThresholdRows(2).withDefaultSampleRows(4);
+      var filter = com.kingsrook.qbits.dataanalysis.engine.DrillFilters.condition("id",
+         com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator.GREATER_THAN, 2);
+      var backend = service.backend("orders");
+      var scope = backend.scope("orders", filter, false);
+      assertThat(scope.filter().getSubFilters().get(1).getCriteria()).singleElement().satisfies(c ->
+      {
+         assertThat(c.getFieldName()).isEqualTo("id");
+         assertThat(c.getOperator()).isEqualTo(com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator.LESS_THAN_OR_EQUALS);
+         assertThat(c.getValues()).containsExactly(6);
+      });
+      var profile = service.columnProfile(new ColumnProfileRequest().withTableName("orders").withFieldName("quantity").withFilter(filter));
+      assertThat(profile.getCounts().getFirst().getValue()).isEqualTo(10L);
+      assertThat(profile.getCounts().stream().filter(m -> m.getKey().equals("nonNull")).findFirst().orElseThrow().getValue()).isEqualTo(4L);
+      assertThat(profile.getWarnings()).contains("Estimate from the first 4 rows by id");
+      var drill = profile.getCounts().getFirst().getDrill();
+      assertThat(new StandInAnalysisBackend(service.getConfigResolver().resolve("orders")).count("orders", drill.getFilter())).isEqualTo(10);
+      assertThat(com.kingsrook.qqq.backend.core.utils.JsonUtils.toJson(profile)).doesNotContain("\"operator\":\"IN\"");
+   }
+
+   /*******************************************************************************
+    ** Empty capabilities select the stand-in; partial support uses core plus fallback.
+    *******************************************************************************/
+   @Test
+   void fallbackSelection() throws Exception
+   {
+      capabilities(Set.of());
+      assertThat(service.backend("orders")).isExactlyInstanceOf(StandInAnalysisBackend.class);
+      capabilities(Set.of(AggregateCapability.STDDEV));
+      assertThat(service.backend("orders")).isInstanceOf(CoreAnalysisBackend.class);
+      var profile = service.columnProfile(new ColumnProfileRequest().withTableName("orders").withFieldName("amount"));
+      assertThat(profile.getNumeric().getMetrics().stream().filter(m -> m.getKey().equals("p50")).findFirst().orElseThrow().getValue().toString()).isEqualTo("20.0");
+      new MemoryEngineTest().assertDrills(profile);
+   }
+
+   /*******************************************************************************
+    ** Advertise selected capabilities while retaining the fixture's actual actions.
+    *******************************************************************************/
+   private void capabilities(Set<AggregateCapability> supported)
+   {
+      LimitedMemoryBackend.supported = supported;
+      QBackendModuleDispatcher.registerBackendModule(new LimitedMemoryBackend());
+   }
+
+   /*******************************************************************************
+    ** Public no-argument backend class required by the module dispatcher.
+    *******************************************************************************/
+   public static class LimitedMemoryBackend extends MemoryBackendModule
+   {
+      private static Set<AggregateCapability> supported = Set.of();
+
+      /*******************************************************************************
+       ** Retain real memory execution with a deliberately reduced feature set.
+       *******************************************************************************/
+      @Override
+      public AggregateInterface getAggregateInterface()
+      {
+         return (new MemoryAggregateAction()
+         {
+            @Override
+            public Set<AggregateCapability> getSupportedCapabilities(AggregateInput input)
+            {
+               return (supported);
+            }
+         });
+      }
+   }
+}

src/test/java/com/kingsrook/qbits/dataanalysis/DataAnalysisApiTest.java

@@ -0,0 +1,290 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.dataanalysis;
+
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.List;
+import java.util.Map;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.kingsrook.qbits.dataanalysis.api.DataAnalysisApi;
+import com.kingsrook.qqq.backend.core.instances.AbstractQQQApplication;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType;
+import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
+import com.kingsrook.qqq.backend.core.model.session.QSession;
+import com.kingsrook.qqq.backend.core.modules.authentication.QAuthenticationModuleCustomizerInterface;
+import com.kingsrook.qqq.middleware.javalin.QApplicationJavalinServer;
+import com.kingsrook.qqq.middleware.javalin.QJavalinImplementation;
+import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiInvoker;
+import com.kingsrook.qqq.middleware.javalin.specs.invoke.UiApiRequest;
+import com.kingsrook.qqq.middleware.javalin.specs.v1.MiddlewareVersionV1;
+import com.kingsrook.qqq.middleware.javalin.tools.UiApiContract;
+import com.kingsrook.qqq.openapi.model.HttpMethod;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Test;
+import static org.assertj.core.api.Assertions.assertThat;
+
+/*******************************************************************************
+ ** Real HTTP session, schema and drill round-trip acceptance tests.
+ *******************************************************************************/
+class DataAnalysisApiTest extends EngineFixture
+{
+   private QApplicationJavalinServer server;
+   private UiApiInvoker invoker;
+   private String caller = "analysis-user";
+
+   /*******************************************************************************
+    ** Give MOCK sessions only the grants represented by their caller cookie.
+    *******************************************************************************/
+   public static class Sessions implements QAuthenticationModuleCustomizerInterface
+   {
+      /*******************************************************************************
+       ** Build the host's ordinary user permission and record-lock policy.
+       *******************************************************************************/
+      @Override
+      public void customizeSession(QInstance instance, QSession session, Map<String, Object> context)
+      {
+         QSession grants = TestSupport.admin().withPermissions("orders.read", "customer.read").withSecurityKeyValue("orders.all", true);
+         session.setPermissions(grants.getPermissions());
+         session.setSecurityKeyValues(grants.getSecurityKeyValues());
+         if(!"denied".equals(session.getUuid()))
+         {
+            session.withPermission("orders.analyze");
+         }
+         session.withSecurityKeyValue("data_analysis_user", session.getUser().getIdReference());
+      }
+   }
+
+   /*******************************************************************************
+    ** Start on an OS-selected port using the actual spec router and authentication.
+    *******************************************************************************/
+   private void start() throws Exception
+   {
+      instance.withInstanceDefaultAuthentication(new QAuthenticationMetaData().withName("http-test").withType(QAuthenticationType.MOCK)
+         .withCustomizer(new QCodeReference(Sessions.class)));
+      QInstance fixture = instance;
+      server = new QApplicationJavalinServer(new AbstractQQQApplication()
+      {
+         @Override
+         public QInstance defineQInstance()
+         {
+            return (fixture);
+         }
+      }).withPort(0).withServeFrontendNext(false).withServeFrontendMaterialDashboard(false)
+         .withServeLegacyUnversionedMiddlewareAPI(false).withStrictUiApiRoutes(true);
+      server.start();
+      invoker = UiApiInvoker.forServer(server);
+   }
+
+   /*******************************************************************************
+    ** Stop the server before the fixture releases context.
+    *******************************************************************************/
+   @AfterEach
+   void stop()
+   {
+      if(server != null)
+      {
+         server.stop();
+      }
+      QJavalinImplementation.setSessionCookieHttpOnly(false);
+   }
+
+   /*******************************************************************************
+    ** Invoke through loopback, forwarding the actual session cookie.
+    *******************************************************************************/
+   private JsonNode call(HttpMethod method, String path, Object body, int status) throws Exception
+   {
+      var request = new UiApiRequest().withMethod(method).withPath(DataAnalysisApi.PREFIX + path)
+         .withPathParam("table", "orders").withSessionUUID(caller).withUserTimezone("America/Chicago");
+      if(body != null)
+      {
+         request.withJsonBody(body);
+      }
+      var response = invoker.invoke(request);
+      assertThat(response.getStatus()).as(response.getBody()).isEqualTo(status);
+      return (response.getJson());
+   }
+
+   /*******************************************************************************
+    ** All operations, assistant discovery, OpenAPI and count-drill equivalence.
+    *******************************************************************************/
+   @Test
+   void happyPathsAndDrills() throws Exception
+   {
+      start();
+      JsonNode capabilities = call(HttpMethod.GET, "/tables/{table}/capabilities", null, 200);
+      assertThat(capabilities.path("canAnalyze").asBoolean()).isTrue();
+      assertThat(capabilities.path("backendCapabilities").toString()).contains("NUMERIC_BUCKET");
+      assertThat(capabilities.path("fields").toString()).doesNotContain("secret").contains("CATEGORY", "HISTOGRAM");
+      for(String field : List.of("amount", "note", "created", "order_date", "paid", "status"))
+      {
+         checkDrills(call(HttpMethod.POST, "/tables/{table}/column-profile", Map.of("fieldName", field), 200));
+      }
+      checkDrills(call(HttpMethod.POST, "/tables/{table}/column-profile", Map.of("fieldName", "amount", "filter",
+         Map.of("criteria", List.of(Map.of("fieldName", "created", "operator", "EQUALS", "values", List.of("2026-01-01T12:00:00Z"))))), 200));
+      checkDrills(call(HttpMethod.POST, "/tables/{table}/distinct-values", Map.of("fieldName", "status", "pageSize", 1), 200));
+      checkDrills(call(HttpMethod.POST, "/tables/{table}/table-profile", Map.of("fields", List.of("amount", "note")), 200));
+      checkDrills(call(HttpMethod.POST, "/tables/{table}/duplicates", Map.of("fields", List.of("quantity"), "limit", 2), 200));
+      for(String operation : List.of("COLUMN_PROFILE", "TABLE_PROFILE"))
+      {
+         JsonNode started = call(HttpMethod.POST, "/runs", Map.of("operation", operation, "tableName", "orders", "request",
+            operation.equals("COLUMN_PROFILE") ? Map.of("fieldName", "quantity") : Map.of("fields", List.of("paid"))), 200);
+         JsonNode run = null;
+         for(int i = 0; i < 100; i++)
+         {
+            var response = invoker.invoke(new UiApiRequest().withMethod(HttpMethod.GET).withPath(DataAnalysisApi.PREFIX + "/runs/{runId}")
+               .withPathParam("runId", started.path("runId").asText()).withSessionUUID(caller).withUserTimezone("America/Chicago"));
+            assertThat(response.getStatus()).as(response.getBody()).isEqualTo(200);
+            run = response.getJson();
+            if(!run.path("status").asText().equals("RUNNING"))
+            {
+               break;
+            }
+            Thread.sleep(20);
+         }
+         assertThat(run.path("status").asText()).isEqualTo("COMPLETE");
+         var denied = invoker.invoke(new UiApiRequest().withMethod(HttpMethod.GET).withPath(DataAnalysisApi.PREFIX + "/runs/{runId}")
+            .withPathParam("runId", started.path("runId").asText()).withSessionUUID("another-owner"));
+         assertThat(denied.getStatus()).isEqualTo(403);
+         assertThat(run.path(operation.equals("COLUMN_PROFILE") ? "columnProfile" : "tableProfile").isObject()).isTrue();
+      }
+      assertThat(invoker.listOperations().stream().filter(o -> o.pathTemplate().startsWith(DataAnalysisApi.PREFIX)).toList()).hasSize(7);
+      var client = java.net.http.HttpClient.newHttpClient();
+      var response = client.send(java.net.http.HttpRequest.newBuilder(java.net.URI.create("http://127.0.0.1:" + server.getBoundPort() + "/qqq/v1/openapi.json")).build(), java.net.http.HttpResponse.BodyHandlers.ofString());
+      assertThat(response.statusCode()).isEqualTo(200);
+      JsonNode api = new ObjectMapper().readTree(response.body());
+      assertThat(api.path("paths").has("/qqq/v1/qbit/data-analysis/tables/{table}/column-profile")).isTrue();
+      assertThat(api.path("components").path("schemas").has("DataAnalysisMetricV1")).isTrue();
+   }
+
+   /*******************************************************************************
+    ** Estimate payloads preserve exact totals and send drills to all matching rows.
+    *******************************************************************************/
+   @Test
+   void estimateDrills() throws Exception
+   {
+      config.withDefaultSampleThresholdRows(2).withDefaultSampleRows(4);
+      instance.withSupplementalMetaData(new DataAnalysisRuntime(config));
+      start();
+      JsonNode profile = call(HttpMethod.POST, "/tables/{table}/column-profile", Map.of("fieldName", "amount"), 200);
+      assertThat(profile.path("estimate").asBoolean()).isTrue();
+      assertThat(profile.path("sampleRows").asLong()).isEqualTo(4);
+      JsonNode rows = profile.path("counts").get(0);
+      assertThat(rows.path("value").asLong()).isEqualTo(12);
+      assertThat(rows.path("estimate").asBoolean()).isFalse();
+      assertThat(profile.path("counts").get(1).path("formatted").asText()).startsWith("≈ ");
+      assertThat(profile.toString()).doesNotContain("\"operator\":\"IN\"");
+      var response = invoker.invoke(new UiApiRequest().withMethod(HttpMethod.POST).withPath("/table/{tableName}/count")
+         .withPathParam("tableName", "orders").withJsonBody(Map.of("filter", rows.path("drill").path("filter")))
+         .withSessionUUID(caller).withUserTimezone("America/Chicago"));
+      assertThat(response.getStatus()).isEqualTo(200);
+      assertThat(response.getJson().path("count").asLong()).isEqualTo(12);
+      checkDrills(profile);
+      JsonNode table = call(HttpMethod.POST, "/tables/{table}/table-profile", Map.of("fields", List.of("amount", "quantity")), 200);
+      assertThat(table.path("estimate").asBoolean()).isTrue();
+      assertThat(table.path("rowCount").path("value").asLong()).isEqualTo(12);
+      checkDrills(table);
+   }
+
+   /*******************************************************************************
+    ** Recursively post returned count drills to the core count endpoint.
+    *******************************************************************************/
+   private void checkDrills(JsonNode node) throws Exception
+   {
+      if(node.path("estimate").asBoolean(false))
+      {
+         return;
+      }
+      if(node.isObject() && node.hasNonNull("drill"))
+      {
+         JsonNode expected = node.has("count") ? node.get("count") : node.has("y") ? node.get("y") : "COUNT".equals(node.path("valueType").asText()) ? node.get("value") : null;
+         if(expected != null && expected.isNumber())
+         {
+            JsonNode drill = node.get("drill");
+            var response = invoker.invoke(new UiApiRequest().withMethod(HttpMethod.POST).withPath("/table/{tableName}/count")
+               .withPathParam("tableName", drill.path("tableName").asText()).withJsonBody(Map.of("filter", drill.get("filter"))).withSessionUUID(caller).withUserTimezone("America/Chicago"));
+            assertThat(response.getStatus()).as(response.getBody()).isEqualTo(200);
+            assertThat(response.getJson().path("count").asLong()).as(drill.toString()).isEqualTo(expected.asLong());
+         }
+      }
+      if(node.isContainerNode())
+      {
+         for(JsonNode child : node)
+         {
+            checkDrills(child);
+         }
+      }
+   }
+
+   /*******************************************************************************
+    ** Denials, locked fields, missing tables and malformed request bodies.
+    *******************************************************************************/
+   @Test
+   void errors() throws Exception
+   {
+      start();
+      call(HttpMethod.POST, "/tables/{table}/column-profile", Map.of("fieldName", "secret"), 400);
+      call(HttpMethod.POST, "/tables/{table}/column-profile", Map.of("fieldName", "missing"), 400);
+      call(HttpMethod.POST, "/tables/{table}/column-profile", Map.of(), 400);
+      call(HttpMethod.POST, "/tables/{table}/column-profile", Map.of("fieldName", 1), 400);
+      call(HttpMethod.POST, "/tables/{table}/column-profile", Map.of("fieldName", "amount", "exact", "yes"), 400);
+      call(HttpMethod.POST, "/tables/{table}/distinct-values", Map.of("fieldName", "note", "pageSize", 0), 400);
+      call(HttpMethod.POST, "/tables/{table}/duplicates", Map.of("fields", List.of()), 400);
+      call(HttpMethod.POST, "/tables/{table}/table-profile", Map.of("fields", "amount"), 400);
+      call(HttpMethod.POST, "/runs", Map.of("operation", "PIVOT"), 400);
+      call(HttpMethod.POST, "/runs", com.fasterxml.jackson.databind.node.NullNode.getInstance(), 400);
+      call(HttpMethod.POST, "/tables/{table}/column-profile", Map.of("fieldName", "amount", "filter",
+         Map.of("criteria", List.of(Map.of("fieldName", "secret", "operator", "EQUALS", "values", List.of("hidden"))))), 400);
+      var absent = invoker.invoke(new UiApiRequest().withMethod(HttpMethod.GET).withPath(DataAnalysisApi.PREFIX + "/tables/{table}/capabilities")
+         .withPathParam("table", "missing").withSessionUUID(caller).withUserTimezone("America/Chicago"));
+      assertThat(absent.getStatus()).isEqualTo(404);
+      caller = "denied";
+      assertThat(call(HttpMethod.GET, "/tables/{table}/capabilities", null, 200).path("canAnalyze").asBoolean()).isFalse();
+      call(HttpMethod.POST, "/tables/{table}/column-profile", Map.of("fieldName", "amount"), 403);
+      call(HttpMethod.POST, "/tables/{table}/distinct-values", Map.of("fieldName", "amount"), 403);
+      call(HttpMethod.POST, "/tables/{table}/table-profile", Map.of(), 403);
+      call(HttpMethod.POST, "/tables/{table}/duplicates", Map.of("fields", List.of("amount")), 403);
+      call(HttpMethod.POST, "/runs", Map.of("operation", "COLUMN_PROFILE", "tableName", "orders", "request", Map.of("fieldName", "amount")), 403);
+   }
+
+   /*******************************************************************************
+    ** Generated fragment is committed verbatim and has only the seven operations.
+    *******************************************************************************/
+   @Test
+   void contract() throws Exception
+   {
+      MiddlewareVersionV1 version = new MiddlewareVersionV1();
+      version.setQInstance(instance);
+      String yaml = UiApiContract.generateExtensionYaml(version, DataAnalysisApi.extension());
+      Path path = Path.of("src/main/resources/openapi/qqq-v1/data-analysis.yaml");
+      if(Boolean.getBoolean("openapi.write"))
+      {
+         Files.createDirectories(path.getParent());
+         Files.writeString(path, yaml);
+      }
+      assertThat(Files.exists(path)).as("Regenerate with ./mvnw -o test -Dtest=DataAnalysisApiTest#contract -Dopenapi.write=true -Dqqq.snapshot.version=4.1.0-DATAANALYSIS-SNAPSHOT").isTrue();
+      assertThat(Files.readString(path).replace("\r\n", "\n")).as("OpenAPI changed: regenerate with -Dopenapi.write=true").isEqualTo(yaml.replace("\r\n", "\n"));
+   }
+}
+

src/test/java/com/kingsrook/qbits/dataanalysis/DataAnalysisHomeTest.java

@@ -33,7 +33,6 @@ import com.kingsrook.qqq.backend.core.model.session.QSession;
 import org.junit.jupiter.api.AfterEach;
 import org.junit.jupiter.api.Test;
 import static org.assertj.core.api.Assertions.assertThat;
-import static org.assertj.core.api.Assertions.assertThatThrownBy;
 
 
 /*******************************************************************************
@@ -62,8 +61,12 @@ class DataAnalysisHomeTest
       QContext.init(instance, TestSupport.user("alice").withPermissions("data_analysis.saved.read", "data_analysis.watch.read", "data_analysis.run.read"));
       String html = DataAnalysisHome.render(Instant.now());
       assertThat(html).contains("Mine: 0", "Starred by me: 0", "No saved analyses yet",
-         "No active watches", "No completed or failed runs", "View all");
-      assertThat(new DataAnalysisHome().render(new RenderWidgetInput())).isNotNull();
+         "No watches yet.", "No analysis runs yet.", "View all", "Open any table and choose Analyze on a column header to explore its data.", "<h3>");
+      var widget = new DataAnalysisHome().render(new RenderWidgetInput()).getWidgetData();
+      assertThat(widget).isInstanceOf(com.kingsrook.qqq.backend.core.model.dashboard.widgets.RawHTML.class);
+      assertThat(((com.kingsrook.qqq.backend.core.model.dashboard.widgets.RawHTML) widget).getType()).isEqualTo("html");
+      assertThat(html).contains("&middot;").doesNotContain("Â", "later", "phase", "<table", "color:", "width:360", "width: 360");
+      assertThat(html.chars().allMatch(c -> c < 128)).isTrue();
       QContext.init(instance, TestSupport.user("alice"));
       html = DataAnalysisHome.render(Instant.now());
       assertThat(html).contains("you need data_analysis.saved.read", "you need data_analysis.watch.read").doesNotContain("Mine: 0", "Complete: 0");
@@ -99,11 +102,39 @@ class DataAnalysisHomeTest
       run("bob", "FAILED", now);
       QContext.init(instance, TestSupport.user("alice").withPermissions("data_analysis.saved.read", "data_analysis.watch.read", "data_analysis.run.read"));
       String html = DataAnalysisHome.render(now);
-      assertThat(html).contains("Mine: 2", "Shared (visible to you): 1", "Starred by me: 1", "Enabled: 2", "Firing: 1", "Complete: 2", "Failed: 1")
-         .doesNotContain("No saved analyses", "No active watches", "No completed or failed runs");
+      assertThat(html).contains("Mine: 2", "Shared (visible to you): 1", "Starred by me: 1", "Enabled: 2", "Firing: 1", "Recent analysis runs", "Complete", "Failed", "Running", "Not finished")
+         .doesNotContain("No saved analyses", "No watches yet", "No analysis runs yet");
       QContext.init(instance, TestSupport.user("bob").withPermissions("data_analysis.saved.read", "data_analysis.run.read"));
       html = DataAnalysisHome.render(now);
-      assertThat(html).contains("Mine: 1", "Shared (visible to you): 1", "Starred by me: 0", "Complete: 0", "Failed: 1", "you need data_analysis.watch.read");
+      assertThat(html).contains("Mine: 1", "Shared (visible to you): 1", "Starred by me: 0", "Failed", "you need data_analysis.watch.read");
+   }
+
+
+
+   /*******************************************************************************
+    ** Latest five remain owner-scoped even with administrator bypass and escape HTML.
+    *******************************************************************************/
+   @Test
+   void recentRunDetailsAndLimit() throws Exception
+   {
+      QInstance instance = TestSupport.instance();
+      instance.getTable(DataAnalysisTables.SAVED).withLabel("Saved <analysis> & review");
+      Instant now = Instant.parse("2026-10-13T23:10:00Z");
+      for(int i = 0; i < 7; i++)
+      {
+         TestSupport.insert(DataAnalysisTables.RUN, new QRecord().withValue("table_name", DataAnalysisTables.SAVED)
+            .withValue("operation", "COLUMN_PROFILE").withValue("user_id", "admin").withValue("status", "COMPLETE")
+            .withValue("started_at", now.minusSeconds(i * 60L)).withValue("finished_at", now.minusSeconds(i * 60L)));
+      }
+      TestSupport.insert(DataAnalysisTables.RUN, new QRecord().withValue("table_name", "private_bob_table")
+         .withValue("operation", "TABLE_PROFILE").withValue("user_id", "bob").withValue("status", "FAILED").withValue("started_at", now));
+      QContext.init(instance, TestSupport.admin().withValue(QSession.VALUE_KEY_USER_TIMEZONE, "America/Chicago"));
+      String html = DataAnalysisHome.render(now);
+      assertThat(html).contains("Saved &lt;analysis&gt; &amp; review", "Column Profile", "2026-10-13 18:10", "2026-10-13 18:06", "<ul", "<dl")
+         .doesNotContain("private_bob_table", "2026-10-13 18:05", "<analysis>");
+      assertThat(html.split("<li>", -1)).hasSize(6);
+      assertThat(html).contains("href=\"" + instance.getTablePath(DataAnalysisTables.RUN) + "/1\"");
+      assertThat(html).contains("href=\"" + instance.getTablePath(DataAnalysisTables.RUN) + "\">View all</a>");
    }
 
 
@@ -113,7 +144,7 @@ class DataAnalysisHomeTest
     *******************************************************************************/
    private static void run(String user, String status, Instant started) throws Exception
    {
-      TestSupport.insert(DataAnalysisTables.RUN, new QRecord().withValue("table_name", "orders").withValue("operation", "profile")
+      TestSupport.insert(DataAnalysisTables.RUN, new QRecord().withValue("table_name", "orders").withValue("operation", "COLUMN_PROFILE")
          .withValue("user_id", user).withValue("status", status).withValue("started_at", started));
    }
 }

src/test/java/com/kingsrook/qbits/dataanalysis/DataAnalysisSecurityTest.java

@@ -91,7 +91,7 @@ class DataAnalysisSecurityTest
       instance.withEnforcePermissionsInCoreActions(false);
       assertThat(DataAnalysisPermissions.canAnalyze(table)).isFalse();
       QContext.init(instance, TestSupport.user("alice").withPermission("sales.orders.analyze"));
-      assertThat(DataAnalysisPermissions.canAnalyze(table)).isTrue();
+      assertThat(DataAnalysisPermissions.canAnalyze(table)).isFalse();
       table.withPermissionRules(null);
       instance.withDefaultPermissionRules(new QPermissionRules().withLevel(PermissionLevel.READ_INSERT_EDIT_DELETE_PERMISSIONS).withPermissionBaseName("defaultBase"));
       assertThat(DataAnalysisPermissions.analyzePermissionName(table)).isEqualTo("defaultBase.analyze");
@@ -101,6 +101,29 @@ class DataAnalysisSecurityTest
 
 
 
+   /*******************************************************************************
+    ** The core permission catalog honors configured inclusion and exclusion.
+    *******************************************************************************/
+   @Test
+   void analyzeCatalog() throws Exception
+   {
+      QInstance instance = new QInstance().withEnforcePermissionsInCoreActions(true);
+      instance.addBackend(new com.kingsrook.qqq.backend.core.model.metadata.QBackendMetaData().withName("memory").withBackendType(com.kingsrook.qqq.backend.core.modules.backend.implementations.memory.MemoryBackendModule.class));
+      for(String name : List.of("orders", "excluded", "outside"))
+      {
+         instance.addTable(new QTableMetaData().withName(name).withLabel(name).withPermissionRules(
+            new QPermissionRules().withLevel(PermissionLevel.READ_INSERT_EDIT_DELETE_PERMISSIONS)));
+      }
+      new DataAnalysisQBitProducer().withQBitConfig(new DataAnalysisQBitConfig().withBackendName("memory")
+         .withIncludeTables(java.util.Set.of("orders", "excluded")).withExcludeTables(java.util.Set.of("excluded"))).produce(instance);
+      assertThat(com.kingsrook.qqq.backend.core.actions.permissions.PermissionsHelper.getAllAvailablePermissions(instance)).anySatisfy(permission ->
+      {
+         assertThat(permission.getName()).isEqualTo("orders.analyze");
+         assertThat(permission.getPermissionType()).isEqualTo("Analyze");
+      }).noneMatch(permission -> permission.getName().equals("excluded.analyze") || permission.getName().equals("outside.analyze")
+         || permission.getName().startsWith("data_analysis") && permission.getName().endsWith(".analyze"));
+   }
+
    /*******************************************************************************
     ** Saved, star and run locks hide others even when the shared flag is true.
     *******************************************************************************/

src/test/java/com/kingsrook/qbits/dataanalysis/EngineContractTest.java

@@ -64,7 +64,60 @@ import static org.assertj.core.api.Assertions.within;
 abstract class EngineContractTest extends EngineFixture
 {
    /*******************************************************************************
-    ** All numeric moments and percentiles agree to 1e-9.
+    ** Every emitted key across all profile kinds has a label, on every backend.
+    *******************************************************************************/
+   @Test
+   void presentationAcrossProfileKinds() throws Exception
+   {
+      java.util.Set<String> keys = new java.util.HashSet<>();
+      for(String field : List.of("amount", "quantity", "note", "paid", "order_date", "created"))
+      {
+         ColumnProfile profile = service.columnProfile(column(field));
+         java.util.List<Metric> metrics = new java.util.ArrayList<>(profile.getCounts());
+         if(profile.getNumeric() != null)
+         {
+            metrics.addAll(profile.getNumeric().getMetrics());
+         }
+         if(profile.getText() != null)
+         {
+            metrics.addAll(profile.getText().getMetrics());
+         }
+         if(profile.getDates() != null)
+         {
+            metrics.addAll(profile.getDates().getMetrics());
+            assertThat(profile.getChart().getX().getLabel()).endsWith(" (day)");
+            assertThat(profile.getChart().getSeries().getFirst().getPoints().getFirst().getX().toString()).isEqualTo("2026-01-01");
+         }
+         if(profile.getBooleans() != null)
+         {
+            metrics.addAll(profile.getBooleans());
+         }
+         for(Metric metric : metrics)
+         {
+            keys.add(metric.getKey());
+            assertThat(metric.getLabel()).isEqualTo(com.kingsrook.qbits.dataanalysis.engine.MetricLabels.label(metric.getKey(), metric.getValueType()));
+            assertThat(metric.getFormatted()).isNotBlank();
+         }
+      }
+      assertThat(keys).containsExactlyInAnyOrder("rows", "nonNull", "nulls", "blanks", "distinct", "uniqueness", "min", "max", "mean", "sum", "stddev",
+         "p1", "p5", "p25", "p50", "p75", "p95", "p99", "zeros", "negatives", "outliers", "minLength", "maxLength", "meanLength", "true", "false", "range");
+      ColumnProfile amount = service.columnProfile(column("amount"));
+      assertThat(metric(amount.getNumeric().getMetrics(), "sum").getFormatted()).isEqualTo("$1,170.00");
+      assertThat(amount.getNumeric().getHistogram()).allSatisfy(v -> assertThat(v.getLabel()).contains("$"));
+      Metric uniqueness = metric(amount.getCounts(), "uniqueness");
+      assertThat(uniqueness.getValueType().name()).isEqualTo("PERCENT");
+      assertThat(uniqueness.getValue().toString()).isEqualTo("0.727273");
+      assertThat(uniqueness.getFormatted()).isEqualTo("72.7%");
+      assertThat(metric(service.columnProfile(column("created")).getDates().getMetrics(), "min").getFormatted()).isEqualTo("2026-01-01 06:00");
+      assertThat(service.columnProfile(column("note")).getWarnings()).isEmpty();
+      var table = service.tableProfile(new TableProfileRequest().withTableName("orders").withFields(List.of("note")));
+      assertThat(table.getRowCount().getLabel()).isEqualTo("Rows");
+      assertThat(table.getFields().getFirst().getNulls().getLabel()).isEqualTo("Nulls");
+      assertThat(table.getFields().getFirst().getBlanks().getLabel()).isEqualTo("Blanks");
+   }
+
+   /*******************************************************************************
+    ** Decimal moments retain ten significant digits; drills retain exact membership.
     *******************************************************************************/
    @Test
    void numericProfile() throws Exception
@@ -80,12 +133,12 @@ abstract class EngineContractTest extends EngineFixture
          Map.entry("zeros", "2"), Map.entry("negatives", "1"), Map.entry("outliers", "1"));
       for(var entry : expected.entrySet())
       {
-         assertMetric(result.getNumeric().getMetrics(), entry.getKey(), entry.getValue());
+         assertMetric(result.getNumeric().getMetrics(), entry.getKey(), new BigDecimal(entry.getValue()).round(new java.math.MathContext(10)).toPlainString());
       }
       double[] values = {-10, 0, 0, 10, 10, 20, 20, 30, 40, 50, 1000};
       double mean = 1170.0 / 11;
       double variance = java.util.Arrays.stream(values).map(v -> Math.pow(v - mean, 2)).sum() / 10;
-      assertMetric(result.getNumeric().getMetrics(), "stddev", Double.toString(Math.sqrt(variance)));
+      assertMetric(result.getNumeric().getMetrics(), "stddev", BigDecimal.valueOf(Math.sqrt(variance)).round(new java.math.MathContext(10)).toPlainString());
       assertThat(result.getNumeric().getHistogram()).hasSize(22);
       assertThat(result.getNumeric().getHistogram().stream().mapToLong(v -> v.getCount()).sum()).isEqualTo(11);
       assertThat(result.getNumeric().getOutliers()).singleElement().satisfies(v -> assertThat(new BigDecimal(v.getValue().toString())).isEqualByComparingTo("1000"));
@@ -156,7 +209,7 @@ abstract class EngineContractTest extends EngineFixture
       assertThat(table.getFields()).anySatisfy(f ->
       {
          assertThat(f.getField()).isEqualTo("note");
-         assertThat(f.getCompleteness().doubleValue()).isCloseTo(8.0 / 12, within(1e-9));
+         assertThat(f.getCompleteness().doubleValue()).isEqualTo(0.666667);
       });
       assertThat(table.getCandidateKeys()).anyMatch(k -> k.getFields().equals(List.of("id")));
       var pairs = service.tableProfile(new TableProfileRequest().withTableName("orders").withFields(List.of("status", "quantity")));
@@ -279,12 +332,26 @@ abstract class EngineContractTest extends EngineFixture
    void estimatesRunsAndCache() throws Exception
    {
       config.withDefaultSampleThresholdRows(5).withDefaultSampleRows(4);
-      service = new DataAnalysisService(config);
+      service = newService(config, java.util.concurrent.ForkJoinPool.commonPool());
       ColumnProfile estimate = service.columnProfile(column("amount"));
       assertThat(estimate.getEstimate()).isTrue();
-      assertThat(estimate.getSampleRows()).isEqualTo(4);
-      assertMetric(estimate.getCounts(), "rows", "4");
-      assertMetric(estimate.getNumeric().getMetrics(), "sum", "10");
+      assertMetric(estimate.getCounts(), "rows", "12");
+      assertThat(metric(estimate.getCounts(), "rows").getEstimate()).isFalse();
+      assertThat(metric(estimate.getCounts(), "nonNull").getEstimate()).isTrue();
+      assertThat(metric(estimate.getCounts(), "nonNull").getFormatted()).startsWith("≈ ");
+      assertDrills(estimate);
+      assertThat(count(metric(estimate.getCounts(), "rows").getDrill())).isEqualTo(12);
+      if(service.backend("orders") instanceof com.kingsrook.qbits.dataanalysis.engine.CoreAnalysisBackend)
+      {
+         assertThat(estimate.getWarnings()).anyMatch(w -> w.contains("random sample") && w.contains("scaled to the full 12 rows"));
+      }
+      else
+      {
+         assertThat(estimate.getSampleRows()).isEqualTo(4);
+         assertMetric(estimate.getCounts(), "nonNull", "4");
+         assertMetric(estimate.getNumeric().getMetrics(), "sum", "10");
+         assertThat(estimate.getWarnings()).contains("Estimate from the first 4 rows by id");
+      }
       String id = service.startExactRun(column("amount"));
       RunStatus run = awaitRun(id);
       assertThat(run.getStatus()).isEqualTo("COMPLETE");
@@ -395,7 +462,7 @@ abstract class EngineContractTest extends EngineFixture
       assertThat(ColumnProfiler.autoBucket(first, first.plusYears(2).plusDays(1))).isEqualTo(AnalysisBackend.DateUnit.MONTH);
       assertThat(ColumnProfiler.autoBucket(first, first.plusYears(10))).isEqualTo(AnalysisBackend.DateUnit.MONTH);
       assertThat(ColumnProfiler.autoBucket(first, first.plusYears(10).plusDays(1))).isEqualTo(AnalysisBackend.DateUnit.YEAR);
-      for(AnalysisBackend.DateUnit unit : List.of(AnalysisBackend.DateUnit.WEEK, AnalysisBackend.DateUnit.MONTH, AnalysisBackend.DateUnit.YEAR))
+      for(AnalysisBackend.DateUnit unit : List.of(AnalysisBackend.DateUnit.WEEK, AnalysisBackend.DateUnit.MONTH, AnalysisBackend.DateUnit.QUARTER, AnalysisBackend.DateUnit.YEAR))
       {
          var buckets = service.backend("orders").dateBuckets("orders", null, "order_date", unit, java.time.ZoneId.of("America/Chicago"));
          assertThat(buckets.stream().mapToLong(AnalysisBackend.GroupCount::count).sum()).isEqualTo(12);
@@ -444,6 +511,10 @@ abstract class EngineContractTest extends EngineFixture
     *******************************************************************************/
    private void walk(JsonNode node) throws Exception
    {
+      if(node.path("estimate").asBoolean(false))
+      {
+         return;
+      }
       if(node.isObject())
       {
          if(node.hasNonNull("drill"))
@@ -551,7 +622,7 @@ abstract class EngineContractTest extends EngineFixture
       int previous = runs().size();
       service.columnProfile(column("amount"));
       assertThat(runs()).hasSize(previous + 1);
-      service = new DataAnalysisService(config, task ->
+      service = newService(config, task ->
       {
          throw new java.util.concurrent.RejectedExecutionException("test");
       });
@@ -632,7 +703,7 @@ abstract class EngineContractTest extends EngineFixture
    void executionFailureAndGuardedBooleans() throws Exception
    {
       List<Runnable> jobs = new java.util.ArrayList<>();
-      service = new DataAnalysisService(config, jobs::add);
+      service = newService(config, jobs::add);
       String id = service.startExactRun(column("amount"));
       var table = instance.getTables().remove("orders");
       jobs.getFirst().run();
@@ -641,7 +712,7 @@ abstract class EngineContractTest extends EngineFixture
       assertThat(service.getRun(id).getError()).contains("could not be completed");
       assertThat(runs().getFirst().getValue("result_json")).isNull();
       config.withDefaultMinGroupSize(3);
-      service = new DataAnalysisService(config, Runnable::run);
+      service = newService(config, Runnable::run);
       var paid = service.columnProfile(column("paid"));
       assertThat(paid.getBooleans()).anySatisfy(m ->
       {

src/test/java/com/kingsrook/qbits/dataanalysis/EngineFixture.java

@@ -31,6 +31,7 @@ import com.kingsrook.qqq.backend.core.context.QContext;
 import com.kingsrook.qqq.backend.core.instances.QInstanceEnricher;
 import com.kingsrook.qqq.backend.core.model.data.QRecord;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.DisplayFormat;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
 import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValue;
@@ -96,7 +97,7 @@ abstract class EngineFixture
          .withPermissionRules(DataAnalysisTables.permissionRules("orders"))
          .withField(new QFieldMetaData("id", QFieldType.INTEGER)).withField(new QFieldMetaData("customer", QFieldType.INTEGER).withPossibleValueSourceName("customer"))
          .withField(new QFieldMetaData("status", QFieldType.STRING).withPossibleValueSourceName("statusValues"))
-         .withField(new QFieldMetaData("amount", QFieldType.DECIMAL)).withField(new QFieldMetaData("quantity", QFieldType.INTEGER))
+         .withField(new QFieldMetaData("amount", QFieldType.DECIMAL).withDisplayFormat(DisplayFormat.CURRENCY)).withField(new QFieldMetaData("quantity", QFieldType.INTEGER))
          .withField(new QFieldMetaData("order_date", QFieldType.DATE)).withField(new QFieldMetaData("created", QFieldType.DATE_TIME))
          .withField(new QFieldMetaData("note", QFieldType.STRING)).withField(new QFieldMetaData("paid", QFieldType.BOOLEAN))
          .withField(new QFieldMetaData("modifyDate", QFieldType.DATE_TIME).withBackendName("modify_date"))
@@ -118,7 +119,15 @@ abstract class EngineFixture
             .withValue("note", notes[i]).withValue("paid", i == 11 ? null : i % 2 == 0).withValue("modifyDate", Instant.parse("2026-10-13T12:00:00Z")));
       }
       config = new DataAnalysisQBitConfig().withBackendName("memory");
-      service = new DataAnalysisService(config, Runnable::run);
+      service = newService(config, Runnable::run);
+   }
+
+   /*******************************************************************************
+    ** Allow the identical contract to exercise both primitive implementations.
+    *******************************************************************************/
+   protected DataAnalysisService newService(DataAnalysisQBitConfig configuration, java.util.concurrent.Executor executor)
+   {
+      return (new DataAnalysisService(configuration, executor));
    }
 
    /*******************************************************************************

src/test/java/com/kingsrook/qbits/dataanalysis/H2CompatibilityEngineTest.java

@@ -0,0 +1,39 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.dataanalysis;
+
+import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData;
+
+/*******************************************************************************
+ ** Verify native bucket SQL in H2's ordinal-grouping compatibility mode.
+ *******************************************************************************/
+class H2CompatibilityEngineTest extends H2EngineTest
+{
+   /*******************************************************************************
+    ** H2 supports core's GROUP BY positions in PostgreSQL compatibility mode.
+    *******************************************************************************/
+   @Override
+   protected RDBMSBackendMetaData database()
+   {
+      RDBMSBackendMetaData database = super.database();
+      return (database.withJdbcUrl(database.getJdbcUrl() + ";MODE=PostgreSQL"));
+   }
+}

src/test/java/com/kingsrook/qbits/dataanalysis/H2SamplingTest.java

@@ -0,0 +1,82 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.dataanalysis;
+
+import java.util.List;
+import com.kingsrook.qbits.dataanalysis.engine.DrillFilters;
+import com.kingsrook.qbits.dataanalysis.engine.StandInAnalysisBackend;
+import com.kingsrook.qbits.dataanalysis.result.ColumnProfileRequest;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
+import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData;
+import org.junit.jupiter.api.Test;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.within;
+
+/*******************************************************************************
+ ** RAND samples fluctuate around population counts without changing drill scope.
+ *******************************************************************************/
+class H2SamplingTest extends EngineFixture
+{
+   /*******************************************************************************
+    ** Ordinal grouping is supported by H2's PostgreSQL compatibility mode.
+    *******************************************************************************/
+   @Override
+   protected RDBMSBackendMetaData database()
+   {
+      return (new RDBMSBackendMetaData().withVendor("h2").withUsername("sa").withPassword("")
+         .withJdbcUrl("jdbc:h2:mem:sampling_" + java.util.UUID.randomUUID() + ";DB_CLOSE_DELAY=-1;DATABASE_TO_UPPER=FALSE;MODE=PostgreSQL"));
+   }
+
+   /*******************************************************************************
+    ** A large sample gives stable tolerances and includes filtered-out records.
+    *******************************************************************************/
+   @Test
+   void randomCountsAndHistograms() throws Exception
+   {
+      try(var statement = connection.createStatement())
+      {
+         statement.execute("INSERT INTO orders (id, customer, amount) SELECT x, 1, CASE WHEN MOD(x, 4) = 0 THEN NULL ELSE MOD(x, 10) END FROM SYSTEM_RANGE(1001, 11000) AS source(x)");
+      }
+      config.withDefaultSampleThresholdRows(100).withDefaultSampleRows(2000);
+      service.getConfigResolver().invalidate();
+      var filter = DrillFilters.condition("id", QCriteriaOperator.GREATER_THAN, 1000);
+      var profile = service.columnProfile(new ColumnProfileRequest().withTableName("orders").withFieldName("amount").withFilter(filter));
+      assertThat(profile.getEstimate()).isTrue();
+      var metrics = profile.getCounts().stream().collect(java.util.stream.Collectors.toMap(m -> m.getKey(), m -> m));
+      assertThat(metrics.get("rows").getValue()).isEqualTo(10000L);
+      assertThat(metrics.get("rows").getEstimate()).isFalse();
+      assertThat(((Number) metrics.get("nulls").getValue()).longValue()).isCloseTo(2500L, within(500L));
+      assertThat(((Number) metrics.get("nonNull").getValue()).longValue()).isCloseTo(7500L, within(500L));
+      assertThat(metrics.get("nonNull").getFormatted()).startsWith("≈ ");
+      assertThat(profile.getNumeric().getHistogram().stream().mapToLong(v -> v.getCount()).sum()).isCloseTo(7500L, within(700L));
+      var exact = new StandInAnalysisBackend(service.getConfigResolver().resolve("orders"));
+      assertThat(exact.count("orders", metrics.get("rows").getDrill().getFilter())).isEqualTo(10000);
+      for(var value : profile.getTopValues())
+      {
+         long population = exact.count("orders", value.getDrill().getFilter());
+         assertThat(value.getCount()).isCloseTo(population, within(500L));
+      }
+      assertThat(com.kingsrook.qqq.backend.core.utils.JsonUtils.toJson(profile)).doesNotContain("\"operator\":\"IN\"");
+      var table = service.tableProfile(new com.kingsrook.qbits.dataanalysis.result.TableProfileRequest().withTableName("orders").withFilter(filter).withFields(List.of("amount")));
+      assertThat(table.getRowCount().getValue()).isEqualTo(10000L);
+      assertThat(table.getEstimate()).isTrue();
+   }
+}

src/test/java/com/kingsrook/qbits/dataanalysis/H2StreamedEngineTest.java

@@ -0,0 +1,48 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.dataanalysis;
+
+import java.util.concurrent.Executor;
+import com.kingsrook.qbits.dataanalysis.engine.AnalysisBackend;
+import com.kingsrook.qbits.dataanalysis.engine.StandInAnalysisBackend;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+
+/*******************************************************************************
+ ** Run every engine contract assertion with the portable streamed implementation.
+ *******************************************************************************/
+class H2StreamedEngineTest extends H2EngineTest
+{
+   /*******************************************************************************
+    ** Change only the backend under test, preserving the entire shared contract.
+    *******************************************************************************/
+   @Override
+   protected DataAnalysisService newService(DataAnalysisQBitConfig configuration, Executor executor)
+   {
+      return (new DataAnalysisService(configuration, executor)
+      {
+         @Override
+         public AnalysisBackend backend(String table) throws QException
+         {
+            return (new StandInAnalysisBackend(getConfigResolver().resolve(table)));
+         }
+      });
+   }
+}

src/test/java/com/kingsrook/qbits/dataanalysis/OptionalMiddlewareTest.java

@@ -0,0 +1,77 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.dataanalysis;
+
+import java.nio.file.Path;
+import java.util.Arrays;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+import org.junit.jupiter.api.Test;
+import static org.assertj.core.api.Assertions.assertThat;
+
+/*******************************************************************************
+ ** A separate JVM proves the optional middleware is not needed by the engine.
+ *******************************************************************************/
+class OptionalMiddlewareTest
+{
+   /*******************************************************************************
+    ** Remove Javalin and middleware artifacts from the actual test classpath.
+    *******************************************************************************/
+   @Test
+   void engineWithoutMiddleware() throws Exception
+   {
+      String classpath = Arrays.stream(System.getProperty("surefire.test.class.path", System.getProperty("java.class.path")).split(java.io.File.pathSeparator))
+         .filter(entry -> !entry.contains("qqq-middleware-javalin") && !Path.of(entry).getFileName().toString().startsWith("javalin-"))
+         .collect(Collectors.joining(java.io.File.pathSeparator));
+      Path log = Path.of("target", "core-only-smoke.log");
+      Process process = new ProcessBuilder(Path.of(System.getProperty("java.home"), "bin", "java").toString(), "-cp", classpath, CoreOnlySmoke.class.getName())
+         .redirectErrorStream(true).redirectOutput(log.toFile()).start();
+      try
+      {
+         assertThat(process.waitFor(30, TimeUnit.SECONDS)).as("Core-only JVM timed out").isTrue();
+         assertThat(process.exitValue()).as(java.nio.file.Files.readString(log)).isZero();
+      }
+      finally
+      {
+         if(process.isAlive())
+         {
+            process.destroyForcibly();
+         }
+      }
+   }
+
+   /*******************************************************************************
+    ** Entry point in the classpath with middleware removed.
+    *******************************************************************************/
+   public static class CoreOnlySmoke
+   {
+      /*******************************************************************************
+       ** Metadata and runtime must initialize without loading any API classes.
+       *******************************************************************************/
+      public static void main(String[] args) throws Exception
+      {
+         var instance = TestSupport.instance();
+         assertThat(instance.getSupplementalMetaData("dataAnalysisRuntime")).isNotNull();
+         assertThat(instance.getSupplementalMetaData("middlewareSpecExtensions")).isNull();
+         assertThat(instance.getTable("data_analysis_run")).isNotNull();
+      }
+   }
+}

src/test/java/com/kingsrook/qbits/dataanalysis/PostgresEngineTest.java

@@ -60,6 +60,29 @@ class PostgresEngineTest extends EngineContractTest
       }
    }
 
+   /*******************************************************************************
+    ** Large populations never become one bind per sampled primary key.
+    *******************************************************************************/
+   @org.junit.jupiter.api.Test
+   void largeEstimate() throws Exception
+   {
+      try(var statement = connection.createStatement())
+      {
+         statement.execute("TRUNCATE TABLE orders");
+         statement.execute("INSERT INTO orders (id, customer, paid) SELECT id, 1, id % 2 = 0 FROM generate_series(1, 50000) AS id");
+      }
+      config.withDefaultSampleThresholdRows(100).withDefaultSampleRows(1000);
+      service.getConfigResolver().invalidate();
+      var profile = service.columnProfile(new com.kingsrook.qbits.dataanalysis.result.ColumnProfileRequest().withTableName("orders").withFieldName("paid"));
+      org.assertj.core.api.Assertions.assertThat(profile.getEstimate()).isTrue();
+      assertMetric(profile.getCounts(), "rows", "50000");
+      org.assertj.core.api.Assertions.assertThat(com.kingsrook.qqq.backend.core.utils.JsonUtils.toJson(profile)).doesNotContain("\"operator\":\"IN\"");
+      // Also exceed the old parameter ceiling with the requested sample itself.
+      config.withDefaultSampleRows(40000);
+      service.getConfigResolver().invalidate();
+      assertMetric(service.columnProfile(new com.kingsrook.qbits.dataanalysis.result.ColumnProfileRequest().withTableName("orders").withFieldName("paid")).getCounts(), "rows", "50000");
+   }
+
    /*******************************************************************************
     ** Credentials belong only to the ephemeral test container.
     *******************************************************************************/

src/test/java/com/kingsrook/qbits/dataanalysis/PresentationSamplingTest.java

@@ -0,0 +1,115 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.dataanalysis;
+
+import java.time.Instant;
+import java.time.ZoneId;
+import java.util.ArrayList;
+import java.util.List;
+import com.kingsrook.qbits.dataanalysis.engine.AnalysisBackend;
+import com.kingsrook.qbits.dataanalysis.engine.DrillFilters;
+import com.kingsrook.qbits.dataanalysis.result.ColumnProfileRequest;
+import com.kingsrook.qqq.backend.core.actions.tables.InsertAction;
+import com.kingsrook.qqq.backend.core.model.actions.tables.insert.InsertInput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import org.junit.jupiter.api.Test;
+import static org.assertj.core.api.Assertions.assertThat;
+
+/*******************************************************************************
+ ** Real rows prove warning boundaries and date coordinates without Docker.
+ *******************************************************************************/
+class PresentationSamplingTest extends EngineFixture
+{
+   /*******************************************************************************
+    ** Null prefixes do not consume the budget; warn only after 10,000 values.
+    *******************************************************************************/
+   @Test
+   void patternWarningOnlyWhenTruncated() throws Exception
+   {
+      List<QRecord> records = new ArrayList<>();
+      for(int i = 100; i <= 10101; i++)
+      {
+         records.add(new QRecord().withValue("id", i).withValue("customer", 1).withValue("note", i == 100 ? null : "AA-11"));
+      }
+      assertThat(new InsertAction().execute(new InsertInput("orders").withRecords(records)).getRecords()).allSatisfy(r -> assertThat(r.getErrors()).isNullOrEmpty());
+      var filter = DrillFilters.condition("id", QCriteriaOperator.GREATER_THAN_OR_EQUALS, 100);
+      var request = new ColumnProfileRequest().withTableName("orders").withFieldName("note").withFilter(filter).withExact(true);
+      var truncated = service.columnProfile(request);
+      assertThat(truncated.getText().getSampleRows()).isEqualTo(10000);
+      assertThat(truncated.getWarnings()).containsExactly("Patterns are based on the first 10,000 values.");
+      request.withFilter(DrillFilters.and(filter, DrillFilters.condition("id", QCriteriaOperator.LESS_THAN_OR_EQUALS, 10100)));
+      var all = service.columnProfile(request);
+      assertThat(all.getText().getSampleRows()).isEqualTo(10000);
+      assertThat(all.getWarnings()).isEmpty();
+   }
+
+
+
+   /*******************************************************************************
+    ** Empty samples are not empty populations; full scans are not estimates.
+    *******************************************************************************/
+   @Test
+   void warningsDescribeObservedScope() throws Exception
+   {
+      config.withDefaultSampleThresholdRows(1).withDefaultSampleRows(100);
+      var request = new ColumnProfileRequest().withTableName("orders").withFieldName("note");
+      var complete = service.columnProfile(request);
+      assertThat(complete.getEstimate()).isFalse();
+      assertThat(complete.getWarnings()).isEmpty();
+      config.withDefaultSampleRows(0);
+      service.getConfigResolver().invalidate();
+      var emptySample = service.columnProfile(request);
+      assertThat(emptySample.getWarnings()).contains("No sample rows were observed; run exact for exact numbers.")
+         .doesNotContain("No readable records match this analysis scope.", "All values in this analysis scope are null.");
+      var empty = service.columnProfile(request.withExact(true).withFilter(DrillFilters.condition("id", QCriteriaOperator.EQUALS, -1)));
+      assertThat(empty.getWarnings()).containsExactly("No readable records match this analysis scope.");
+      var nulls = service.columnProfile(request.withFilter(DrillFilters.condition("id", QCriteriaOperator.EQUALS, 1)));
+      assertThat(nulls.getWarnings()).containsExactly("All values in this analysis scope are null.");
+   }
+
+
+
+   /*******************************************************************************
+    ** Hour buckets cross DST in real elapsed hours and preserve ISO instants.
+    *******************************************************************************/
+   @Test
+   void hourlyBucketCoordinatesAndLabels() throws Exception
+   {
+      var backend = service.backend("orders");
+      ZoneId zone = ZoneId.of("America/Chicago");
+      var buckets = backend.dateBuckets("orders", null, "created", AnalysisBackend.DateUnit.HOUR, zone);
+      assertThat(buckets.getFirst().value()).isEqualTo(Instant.parse("2026-01-01T12:00:00Z"));
+      assertThat(buckets.getFirst().label()).isEqualTo("2026-01-01 06:00");
+      assertThat(buckets.stream().mapToLong(AnalysisBackend.GroupCount::count).sum()).isEqualTo(12);
+      for(var bucket : buckets)
+      {
+         assertThat(backend.count("orders", bucket.condition())).isEqualTo(bucket.count());
+      }
+      TestSupport.insert("orders", new QRecord().withValue("id", 20).withValue("customer", 1).withValue("created", Instant.parse("2026-11-01T06:30:00Z")));
+      TestSupport.insert("orders", new QRecord().withValue("id", 21).withValue("customer", 1).withValue("created", Instant.parse("2026-11-01T07:30:00Z")));
+      var filter = DrillFilters.condition("id", QCriteriaOperator.GREATER_THAN_OR_EQUALS, 20);
+      var dst = backend.dateBuckets("orders", filter, "created", AnalysisBackend.DateUnit.HOUR, zone);
+      assertThat(dst).hasSize(2).allSatisfy(b -> assertThat(b.label()).isEqualTo("2026-11-01 01:00"));
+      assertThat(dst).extracting(AnalysisBackend.GroupCount::value).doesNotHaveDuplicates();
+      assertThat(dst).allSatisfy(b -> assertThat(b.count()).isEqualTo(1));
+   }
+}