test(reports): give the old retention report a real run

The foreign key on lab_report.run_id rejects develop's placeholder run 999999,
so the 40-day-old report now belongs to an inserted FAILED run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

.circleci/config.yml

@@ -1,7 +1,7 @@
 version: 2.1
 
 orbs:
-   qqq-orb: kingsrook/qqq-orb@0.6.8
+   qqq-orb: kingsrook/qqq-orb@0.6.9
 
 commands:
    configure-package-install:

CHANGELOG.md

@@ -9,4 +9,19 @@ All notable changes to this project are documented here. The format follows Keep
 - Allow manual Tested Pages inserts with a URL. Validate absolute HTTPS URLs and fill in the site, path and Manual source automatically; validate URL edits too.
 
 ### Added
+- Foreign keys for every reference between the QBit's tables (changesets 12-26): findings and reports cascade with their run, comparisons with their site change; optional pointers (likely cause, first and confirmation run, a run's site change) are set to NULL; tested pages and budgets with history cannot be deleted (disable them instead). Rows whose parent is already missing are recorded in `qqq_fk_quarantine`, then nulled or (NOT NULL columns, PostgreSQL) deleted, so the upgrade never fails. Foreign key columns are indexed. A PostgreSQL guardrail test fails when a new reference lacks its constraint.
+- Every `web_performance_*` table and the Performance app carry help saying what the table is for and how its rows get created (`help/web-performance.yaml`, applied by `WebPerfHelp`; host help is never overridden). A Setup card at the top of the home lists tested pages, the lab runner, budgets (reference defaults or your own) and Google field data, each with its state and next step, using the runtime's own resolution. `WebPerfTablesGuardrailTest` fails when a table is unused, unexplained or lacks that help; `WebPerfDefaultsChangelogTest` runs the shipped changelog and its seeded budgets through real Liquibase.
+- `reportRetentionDays` is now enforced: with a value above 0, Lighthouse Reports older than that many days are deleted after a result is stored, in batches, at most once an hour per process (runs, scores and findings stay). It was documented but nothing read it; the default 0 still keeps everything.
+- Record deploy and external site changes once, queue settled retests, and attribute lab runs to the observed build and CDN stage.
+- Evaluate enabled lab and URL CrUX budgets; retain breaches, confirmation evidence, automatic recovery and accurate open counts in the standard regression records.
+- Confirm statistical lab regressions with a median of three, reset baselines on Lighthouse major changes, and detect deterministic category/audit failures.
+- Add permission-gated Dismiss, Mark resolved and Compare before and after record actions with standard Review/Result summaries.
+- Store before/after lab medians, sample counts and windows, including measured TTFB; schedule CDN samples before launch and at one and 24 hours after launch.
+- Raise and resolve optional alerts through QQQ core, with logging fallback when no Alerts QBit is installed.
+
+### Changed
+- Use QQQ 4.1.0-RC.6 for the core alerts API; add migration fields for durable evaluation and source-event identity.
+- Store complete run results atomically and retain confirmation/comparison identities when retrying samples.
+
+### Previously added
 - First implementation; see README.md for the tables, permissions and configuration.

README.md

@@ -13,15 +13,19 @@ Design and owner decisions: QRun-IO/qqq `docs/specs/2026-10-12-web-performance-d
 
 ## Status
 
-This is a **skeleton (0.1.0-SNAPSHOT)**. It is local only, and the GitHub repository does not exist yet. It contains:
+The lab and Google field-data runners, queue, schedules, record actions and performance widgets are implemented. This change adds durable site changes, budget evaluation, confirmed lab regressions and before/after lab comparisons.
 
-- the producer and config, with validation and secret redaction
-- the nine `web_performance_` tables, their lookups and the Performance app
-- the host-included changelog, including the seeded budgets (D12)
-- the SPIs: `UrlSource`, `ContentResolver`, `LabRunner`, `DeployProbe`, `RumSource`, `ChangeEventSource`
-- two implementations that need no Google API key: `SitemapUrlSource` and `HttpBuildIdProbe`, which reads Website-Frontend's `/api/build`
+- Deploy probes run at startup through the scheduler and every five minutes. New build ids are recorded once per component, and KEY pages are queued after `deploySettleSeconds`. External `ChangeEventSource` events are deduplicated by source registration and their complete event tuple. Keep source registration order stable across restarts. Content events queue enabled pages after `publishSettleSeconds`, because the current SPI does not identify affected target ids.
+- Each run records the latest observed frontend build, hostname CDN stage and preceding site change when it starts. A rollback to an already recorded build updates its last observation without duplicating its change row.
+- Enabled LAB budgets match site, tier, target, content table or literal path prefix, plus strategy and metric. Comparators describe acceptable values. Breaches are recorded in Regressions with the budget id; there is no separate budget-check table in the design. The strongest matching severity wins. CRUX budgets evaluate URL p75 snapshots; NO_DATA, errors and origin fallback are never treated as page measurements.
+- Statistical lab baselines require five successful SCHEDULE, CONFIRM or SITE_AUDIT runs in the preceding 30 days on the same Lighthouse major version. Performance drops of 10 points, LCP increases of both 25% and 500 ms, CLS increases of 0.05, TBT increases of 200 ms, and budget breaches create suspected regressions. Two independent confirmation runs provide a median of three. Category drops of 5 points and newly failing Accessibility, Best Practices or SEO audits confirm immediately. Recovery requires a median of three later runs. Lighthouse major changes record a marker and reset incompatible baselines.
+- The page's open count includes suspected and confirmed regressions. **Dismiss** and **Mark resolved** require a note and `web_performance.manage.hasAccess`, use standard Review/Result summary lines, and preserve history.
+- **Compare before and after** uses three successful same-version lab samples on each side of a change, with page, sample counts and actual time windows. It stores Performance, LCP, FCP and measured TTFB when available. Results are also updated after attributed runs complete. CDN HEADERS_ONLY queues three samples per strategy; LIVE queues three at +1 hour and three at +24 hours. Insufficient history produces no invented comparison.
+- Confirmed regressions raise keyed `webperf.regression` alerts through QQQ core `QAlerts`; recovery, dismissal and manual resolution clear them. `ALERTS` can be disabled. Without an Alerts QBit, core uses its logging sink. No Alerts QBit dependency is required.
 
-Not built yet: the PSI and CrUX runners (they need the key), the queue worker, the schedules, the actions, the widgets, and the Alerts qbit producer integration.
+Evaluation progress and related writes commit together. Retrying an evaluation does not duplicate incidents, and retrying a failed confirmation preserves its sample reference. The additive changelog must be applied before running the new worker. This version requires QQQ **4.1.0-RC.6** for the core alerts API.
+
+RUM drift and RUM/country before-and-after comparisons remain separate work: the existing `RumSource` exposes fixed daily/28-day aggregates, not arbitrary before/after windows. Enum field metadata and Tested Pages insertion are also outside this change.
 
 ## Install
 
@@ -64,6 +68,10 @@ Tier, Scope, Source, Strategy, Metric, Comparator and Severity use dropdowns wit
 | `web_performance_change_event` (Site Changes) | probes, other QBits | read-only |
 | `web_performance_comparison` (Before and After) | the QBit | read-only |
 
+### Starter defaults and empty-state help
+
+The changelog's changeset 10 seeds 13 reference budgets (Google's "good" thresholds and the Lighthouse category bands); nothing else is seeded. The target table is filled by Sync pages from sitemap, and the lab, finding, report, field, regression, change and comparison tables are filled by the qbit as it runs (empty is a normal start). Every table's help (`help/web-performance.yaml`, shown on the table screen) says how its rows get created, and the Setup card on the home shows what still needs doing. Budgets with source RUM are not evaluated yet (no real-user source is wired); LAB and CRUX budgets are.
+
 ## Permissions
 
 Tables and the app use `READ_WRITE_PERMISSIONS` with the base name `web_performance` (configurable), which gives `web_performance.read` and `web_performance.write`. Processes will add `web_performance.test`, `web_performance.manage` and `web_performance.admin` (design §8.1). No table stores a secret.

UPGRADE.md

@@ -0,0 +1,5 @@
+# Upgrading
+
+## Table help, Setup card and `reportRetentionDays` (unreleased)
+
+No DDL change, no new changeset and no host include change. The qbit now fills empty `help` slots on its app and tables (help a host set is kept) and adds a Setup widget to the Performance home. `reportRetentionDays` (default 0, keep forever) is now enforced: set it above 0 only if you want Lighthouse Reports older than that many days deleted.

pom.xml

@@ -22,6 +22,8 @@
       <maven.compiler.source>21</maven.compiler.source>
       <maven.compiler.target>21</maven.compiler.target>
       <jacoco.version>0.8.15</jacoco.version>
+      <liquibase.version>4.33.0</liquibase.version>
+      <testcontainers.version>1.21.4</testcontainers.version>
    </properties>
 
    <parent>
@@ -82,6 +84,13 @@
          <version>2.2.220</version>
          <scope>test</scope>
       </dependency>
+      <!-- WebPerfDefaultsChangelogTest runs the shipped changelog (with its seeded budgets) through real Liquibase. -->
+      <dependency>
+         <groupId>org.liquibase</groupId>
+         <artifactId>liquibase-core</artifactId>
+         <version>4.33.0</version>
+         <scope>test</scope>
+      </dependency>
       <dependency>
          <groupId>org.junit.jupiter</groupId>
          <artifactId>junit-jupiter</artifactId>
@@ -93,6 +102,31 @@
          <version>3.27.7</version>
          <scope>test</scope>
       </dependency>
+      <!-- The foreign key guardrail runs the changelog with Liquibase on PostgreSQL (Docker). -->
+      <dependency>
+         <groupId>org.liquibase</groupId>
+         <artifactId>liquibase-core</artifactId>
+         <version>${liquibase.version}</version>
+         <scope>test</scope>
+      </dependency>
+      <dependency>
+         <groupId>org.postgresql</groupId>
+         <artifactId>postgresql</artifactId>
+         <version>42.7.14</version>
+         <scope>test</scope>
+      </dependency>
+      <dependency>
+         <groupId>org.testcontainers</groupId>
+         <artifactId>postgresql</artifactId>
+         <version>${testcontainers.version}</version>
+         <scope>test</scope>
+      </dependency>
+      <dependency>
+         <groupId>org.testcontainers</groupId>
+         <artifactId>junit-jupiter</artifactId>
+         <version>${testcontainers.version}</version>
+         <scope>test</scope>
+      </dependency>
    </dependencies>
 
    <build>
@@ -143,14 +177,14 @@
    <profiles>
       <!-- Built against qqq 4.1 (QBitMetaDataProducer). Until qbit-build-parent  -->
       <!-- imports the released 4.1 BOM, this profile is active by default and     -->
-      <!-- imports qqq-bom-pom 4.1.0-RC.4 ahead of the parent's 4.0.0 BOM.         -->
+      <!-- imports qqq-bom-pom 4.1.0-RC.6 ahead of the parent's 4.0.0 BOM.         -->
       <profile>
          <id>qqq-snapshot</id>
          <activation>
             <activeByDefault>true</activeByDefault>
          </activation>
          <properties>
-            <qqq.snapshot.version>4.1.0-RC.4</qqq.snapshot.version>
+            <qqq.snapshot.version>4.1.0-RC.6</qqq.snapshot.version>
          </properties>
          <dependencyManagement>
             <dependencies>

src/main/java/com/kingsrook/qbits/webperf/WebPerfQBitProducer.java

@@ -191,6 +191,7 @@ public class WebPerfQBitProducer implements QBitMetaDataProducer<WebPerfQBitConf
       qInstance.addQBit(getQBitMetaData());
       new WebPerfRuntime(qBitConfig).addSelfToInstance(qInstance);
       new WebPerfMetaData(qBitConfig).addTo(qInstance);
+      com.kingsrook.qbits.webperf.metadata.WebPerfHelp.apply(qInstance, qBitConfig);
 
       LOG.info("Web performance registered", logPair("tablePrefix", qBitConfig.getTableNamePrefix()), logPair("app", qBitConfig.getAppName()),
          logPair("siteBaseUrl", qBitConfig.getSiteBaseUrl()), logPair("features", qBitConfig.getFeatures()),

src/main/java/com/kingsrook/qbits/webperf/WebPerfRuntime.java

@@ -46,7 +46,13 @@ public class WebPerfRuntime implements QSupplementalInstanceMetaData
    public static final String NAME_PREFIX    = "webPerformance.";
    public static final String INSTANCE_FIELD = "webPerformanceInstance";
 
+   /** At most one report-retention pass an hour per process; the first one waits so restarts and deploys do not each trigger a pass. */
+   public static final long REPORT_PRUNE_INTERVAL_MILLIS = 60L * 60 * 1000;
+   public static final long REPORT_PRUNE_FIRST_DELAY     = 5L * 60 * 1000;
+
    private final WebPerfQBitConfig config;
+   private final java.util.concurrent.atomic.AtomicLong lastReportPruneMillis = new java.util.concurrent.atomic.AtomicLong(
+      System.currentTimeMillis() - REPORT_PRUNE_INTERVAL_MILLIS + REPORT_PRUNE_FIRST_DELAY);
 
 
 
@@ -133,6 +139,29 @@ public class WebPerfRuntime implements QSupplementalInstanceMetaData
 
 
 
+   /*******************************************************************************
+    ** Whether a report-retention pass is due; true for exactly one caller per
+    ** hour (shared by every worker of this process).
+    *******************************************************************************/
+   public boolean reportPruneDue()
+   {
+      long now  = System.currentTimeMillis();
+      long last = lastReportPruneMillis.get();
+      return (now - last >= REPORT_PRUNE_INTERVAL_MILLIS && lastReportPruneMillis.compareAndSet(last, now));
+   }
+
+
+
+   /*******************************************************************************
+    ** For tests: make the next reportPruneDue() true.
+    *******************************************************************************/
+   public void allowReportPruneNow()
+   {
+      lastReportPruneMillis.set(0);
+   }
+
+
+
    /*******************************************************************************
     ** The lab runner: the configured one, else PageSpeed Insights with the
     ** Google API key, else null (no key: lab tests are off).

src/main/java/com/kingsrook/qbits/webperf/lab/LabResultParser.java

@@ -139,6 +139,53 @@ public final class LabResultParser
 
 
 
+   /*******************************************************************************
+    ** Read a reported TTFB subpart from Lighthouse's LCP insight, never savings.
+    *******************************************************************************/
+   public static Integer ttfb(String json)
+   {
+      JSONObject root = new JSONObject(json);
+      JSONObject lhr = root.optJSONObject("lighthouseResult");
+      JSONObject audits = (lhr == null ? root : lhr).optJSONObject("audits");
+      if(audits == null)
+      {
+         return (null);
+      }
+      for(String name : List.of("lcp-phases-insight", "lcp-breakdown-insight"))
+      {
+         JSONObject audit = audits.optJSONObject(name);
+         JSONObject details = audit == null ? null : audit.optJSONObject("details");
+         JSONArray items = details == null ? null : details.optJSONArray("items");
+         if(items != null)
+         {
+            for(int i = 0; i < items.length(); i++)
+            {
+               JSONObject item = items.optJSONObject(i);
+               if(item == null)
+               {
+                  continue;
+               }
+               JSONArray subparts = item.optJSONArray("items");
+               if(subparts == null)
+               {
+                  subparts = new JSONArray().put(item);
+               }
+               for(int j = 0; j < subparts.length(); j++)
+               {
+                  JSONObject part = subparts.optJSONObject(j);
+                  if(part != null && "timeToFirstByte".equals(part.optString("subpart")) && part.opt("duration") instanceof Number duration)
+                  {
+                     return ((int) Math.round(duration.doubleValue()));
+                  }
+               }
+            }
+         }
+      }
+      return (null);
+   }
+
+
+
    /*******************************************************************************
     ** Parser only.
     *******************************************************************************/

src/main/java/com/kingsrook/qbits/webperf/metadata/WebPerfHelp.java

@@ -0,0 +1,134 @@
+/*
+ * Copyright 2026 Kingsrook, LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.webperf.metadata;
+
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.LinkedHashMap;
+import java.util.Map;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.dataformat.yaml.YAMLFactory;
+import com.kingsrook.qbits.webperf.WebPerfQBitConfig;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.help.QHelpContent;
+import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+
+
+/*******************************************************************************
+ ** Applies help/web-performance.yaml (the QQQ help-bundle format) to the
+ ** Performance app and its web_performance_* tables: what each table is for and
+ ** how its rows get created, shown on the table screen and when it is empty.
+ ** STAND-IN(core=QRun-IO/qqq#1198, removeBy=4.1.0-RC.9): replace with HelpBundles.register once the pinned QQQ
+ ** release includes it; the bundle file stays the same.
+ *******************************************************************************/
+public final class WebPerfHelp
+{
+   public static final String RESOURCE = "/help/web-performance.yaml";
+   public static final String SLOT     = "help";
+
+
+
+   /*******************************************************************************
+    ** Utility class.
+    *******************************************************************************/
+   private WebPerfHelp()
+   {
+   }
+
+
+
+   /*******************************************************************************
+    ** The bundle's entries by key, validated (fails startup if malformed).
+    *******************************************************************************/
+   public static Map<String, JsonNode> entries() throws QException
+   {
+      try(InputStream stream = WebPerfHelp.class.getResourceAsStream(RESOURCE))
+      {
+         if(stream == null)
+         {
+            throw (new IOException("missing resource"));
+         }
+         JsonNode bundle = new ObjectMapper(new YAMLFactory()).readTree(stream);
+         if(!"web-performance".equals(bundle.path("bundle").asText()) || !"QBIT".equals(bundle.path("layer").asText()))
+         {
+            throw (new IOException("invalid bundle header"));
+         }
+         Map<String, JsonNode> entries = new LinkedHashMap<>();
+         for(JsonNode entry : bundle.path("entries"))
+         {
+            String key     = entry.path("key").asText();
+            String summary = entry.path("summary").asText();
+            if(key.isBlank() || summary.isBlank() || summary.length() > 120 || entry.path("content").asText().isBlank() || entries.put(key, entry) != null)
+            {
+               throw (new IOException("invalid or duplicate entry: " + key));
+            }
+         }
+         return (entries);
+      }
+      catch(IOException | RuntimeException e)
+      {
+         throw (new QException("Could not load " + RESOURCE, e));
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Apply the help to the app and every web_performance_* table in the instance
+    ** (names carry the config's prefix, if any). A host's own help on a table
+    ** is kept (the bundle only fills the slot).
+    *******************************************************************************/
+   public static void apply(QInstance qInstance, WebPerfQBitConfig config) throws QException
+   {
+      for(Map.Entry<String, JsonNode> entry : entries().entrySet())
+      {
+         String key  = entry.getKey();
+         String name = key.substring(key.indexOf(':') + 1, key.indexOf(';'));
+         if(key.startsWith("table:"))
+         {
+            QTableMetaData table = qInstance.getTable(config.applyPrefix(name.startsWith(WebPerfQBitConfig.DEFAULT_TABLE_PREFIX) ? name.substring(WebPerfQBitConfig.DEFAULT_TABLE_PREFIX.length()) : name));
+            if(table != null && (table.getHelpContent() == null || !table.getHelpContent().containsKey(SLOT)))
+            {
+               table.withHelpContent(SLOT, content(entry.getValue()));
+            }
+         }
+         else if(key.startsWith("app:"))
+         {
+            QAppMetaData app = qInstance.getApp(config.getAppName());
+            if(app != null && (app.getHelpContent() == null || !app.getHelpContent().containsKey(SLOT)))
+            {
+               app.withHelpContent(SLOT, content(entry.getValue()));
+            }
+         }
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Summary as the first paragraph, then the content, as markdown.
+    *******************************************************************************/
+   static QHelpContent content(JsonNode entry)
+   {
+      return (new QHelpContent().withContentAsMarkdown(entry.path("summary").asText().strip() + "\n\n" + entry.path("content").asText().strip()));
+   }
+
+}

src/main/java/com/kingsrook/qbits/webperf/metadata/WebPerfMetaData.java

@@ -26,21 +26,25 @@ import java.util.LinkedHashMap;
 import java.util.List;
 import java.util.Map;
 import java.util.Set;
+import com.kingsrook.qbits.webperf.WebPerfFeature;
 import com.kingsrook.qbits.webperf.WebPerfQBitConfig;
 import com.kingsrook.qbits.webperf.WebPerfRuntime;
 import com.kingsrook.qbits.webperf.WebPerfSchema;
 import com.kingsrook.qbits.webperf.processes.WebPerfProcesses;
+import com.kingsrook.qbits.webperf.service.WebPerfEvaluationService;
 import com.kingsrook.qbits.webperf.widgets.WebPerfWidgets;
 import com.kingsrook.qqq.backend.core.actions.customizers.TableCustomizers;
 import com.kingsrook.qqq.backend.core.actions.dashboard.widgets.AbstractWidgetRenderer;
-import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
-import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
-import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
-import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
+import com.kingsrook.qqq.backend.core.alerts.AlertSeverity;
+import com.kingsrook.qqq.backend.core.alerts.AlertsInstanceMetaData;
+import com.kingsrook.qqq.backend.core.alerts.QAlertTypeMetaData;
 import com.kingsrook.qqq.backend.core.instances.QInstanceEnricher;
+import com.kingsrook.qqq.backend.core.model.dashboard.widgets.WidgetType;
 import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
 import com.kingsrook.qqq.backend.core.model.metadata.audits.AuditLevel;
 import com.kingsrook.qqq.backend.core.model.metadata.audits.QAuditRules;
+import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
+import com.kingsrook.qqq.backend.core.model.metadata.dashboard.QWidgetMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldType;
 import com.kingsrook.qqq.backend.core.model.metadata.layout.QAppMetaData;
@@ -50,6 +54,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.permissions.PermissionLevel
 import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRules;
 import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValue;
 import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSource;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.possiblevalues.QPossibleValueSourceType;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.Capability;
 import com.kingsrook.qqq.backend.core.model.metadata.tables.QFieldSection;
@@ -101,6 +106,16 @@ public final class WebPerfMetaData
     *******************************************************************************/
    public void addTo(QInstance qInstance)
    {
+      if(config.isOn(WebPerfFeature.ALERTS))
+      {
+         AlertsInstanceMetaData.ofOrWithNew(qInstance).withAlertType(
+            new QAlertTypeMetaData()
+               .withName(WebPerfEvaluationService.ALERT_TYPE)
+               .withLabel("Web performance regression").withSourceQBitName("web-performance")
+               .withDefaultSeverity(AlertSeverity.WARNING)
+               .withSubjectTableName(config.applyPrefix(WebPerfSchema.REGRESSION))
+               .withVisibilityPermission(config.getPermissionBaseName() + ".read"));
+      }
       for(QTableMetaData table : tables().values())
       {
          qInstance.addTable(table);
@@ -114,6 +129,7 @@ public final class WebPerfMetaData
       {
          qInstance.addProcess(process);
       }
+      addWidget(qInstance, WebPerfWidgets.SETUP, "Setup", WidgetType.HTML, WebPerfWidgets.Setup.class);
       addWidget(qInstance, WebPerfWidgets.SITE_SCORES, "Site Scores", WidgetType.HTML, WebPerfWidgets.SiteScores.class);
       addWidget(qInstance, WebPerfWidgets.TREND, "Performance Trend", WidgetType.LINE_CHART, WebPerfWidgets.Trend.class);
       addWidget(qInstance, WebPerfWidgets.SLOWEST_PAGES, "Slowest Pages", WidgetType.HTML, WebPerfWidgets.SlowestPages.class);
@@ -365,7 +381,10 @@ public final class WebPerfMetaData
          f("lcpElement", QFieldType.STRING, "LCP Element").withMaxLength(300),
          f("runWarnings", QFieldType.STRING, "Warnings").withMaxLength(2000),
          f("deploymentId", QFieldType.STRING, "Frontend Build").withMaxLength(64),
-         f("cdnStage", QFieldType.STRING, "CDN Stage").withMaxLength(20)));
+         f("cdnStage", QFieldType.STRING, "CDN Stage").withMaxLength(20),
+         f("ttfbMs", QFieldType.INTEGER, "TTFB (ms)"),
+         f("evaluatedAt", QFieldType.DATE_TIME, "Evaluated"),
+         f("changeEventId", QFieldType.LONG, "Site Change").withPossibleValueSourceName(config.applyPrefix(WebPerfSchema.CHANGE_EVENT))));
 
       return (table(WebPerfSchema.LAB_RUN, "Lab Runs", "speed", fields, false, Set.of())
          .withRecordLabelFormatAndFields("%s %s", "strategy", "url")
@@ -374,7 +393,7 @@ public final class WebPerfMetaData
          .withSection(section("metrics", "Lab Metrics", "timer", Tier.T2, "fcpMs", "lcpMs", "tbtMs", "cls", "speedIndexMs", "lcpElement"))
          .withSection(section("details", "Details", "info", Tier.T2, "targetId", "finalUrl", "runner", "triggerRef", "batchId", "notBefore",
             "queuedAt", "startedAt", "attempts", "httpStatus", "errorMessage", "lighthouseVersion", "chromeUserAgent", "benchmarkIndex",
-            "runWarnings", "deploymentId", "cdnStage"))
+            "runWarnings", "deploymentId", "cdnStage", "ttfbMs", "evaluatedAt", "changeEventId"))
          .withSection(dates()));
    }
 
@@ -451,11 +470,12 @@ public final class WebPerfMetaData
          f("p75FcpMs", QFieldType.INTEGER, "FCP p75 (ms)"),
          f("p75TtfbMs", QFieldType.INTEGER, "TTFB p75 (ms)"),
          f("distributions", QFieldType.TEXT, "Distributions"),
-         f("lastCheckedAt", QFieldType.DATE_TIME, "Last Checked")));
+         f("lastCheckedAt", QFieldType.DATE_TIME, "Last Checked"),
+         f("evaluatedAt", QFieldType.DATE_TIME, "Evaluated")));
 
       return (table(WebPerfSchema.FIELD_SNAPSHOT, "Google Field Data", "public", fields, false, Set.of())
          .withRecordLabelFormatAndFields("%s %s", "scopeKey", "formFactor")
-         .withSection(section("snapshot", "Snapshot", "public", Tier.T1, "id", "scopeKey", "scope", "formFactor", "status", "lastCheckedAt"))
+         .withSection(section("snapshot", "Snapshot", "public", Tier.T1, "id", "scopeKey", "scope", "formFactor", "status", "lastCheckedAt", "evaluatedAt"))
          .withSection(section("p75", "p75 (28 days)", "insights", Tier.T2, "p75LcpMs", "p75InpMs", "p75Cls", "p75FcpMs", "p75TtfbMs",
             "periodStart", "periodEnd", "source", "distributions"))
          .withSection(dates()));
@@ -515,14 +535,18 @@ public final class WebPerfMetaData
          f("detectedAt", QFieldType.DATE_TIME, "Detected"),
          f("confirmedAt", QFieldType.DATE_TIME, "Confirmed"),
          f("resolvedAt", QFieldType.DATE_TIME, "Resolved"),
-         f("resolutionNote", QFieldType.STRING, "Resolution Note").withMaxLength(2000)));
+         f("resolutionNote", QFieldType.STRING, "Resolution Note").withMaxLength(2000),
+         f("firstRunId", QFieldType.LONG, "First Run").withPossibleValueSourceName(config.applyPrefix(WebPerfSchema.LAB_RUN)),
+         f("checkpointRunId", QFieldType.LONG, "Confirmation Run").withPossibleValueSourceName(config.applyPrefix(WebPerfSchema.LAB_RUN)),
+         f("lighthouseMajor", QFieldType.STRING, "Lighthouse Major").withMaxLength(20),
+         f("failedAudits", QFieldType.STRING, "Newly Failing Audits").withMaxLength(2000)));
 
       return (table(WebPerfSchema.REGRESSION, "Regressions", "trending_down", fields, true, Set.of())
          .withRecordLabelFormatAndFields("%s %s", "metric", "status")
          .withSection(section("regression", "Regression", "trending_down", Tier.T1, "id", "targetId", "metric", "strategy", "baselineValue",
             "currentValue", "delta", "severity", "status"))
          .withSection(section("details", "Details", "info", Tier.T2, "source", "contentTable", "contentId", "baselineRef", "currentRef",
-            "budgetId", "causeChangeEventId", "detectedAt", "confirmedAt", "resolvedAt", "resolutionNote"))
+            "budgetId", "causeChangeEventId", "detectedAt", "confirmedAt", "resolvedAt", "resolutionNote", "firstRunId", "checkpointRunId", "lighthouseMajor", "failedAudits"))
          .withSection(dates()));
    }
 
@@ -540,12 +564,15 @@ public final class WebPerfMetaData
          f("toValue", QFieldType.STRING, "To").withMaxLength(200),
          f("observedAt", QFieldType.DATE_TIME, "Observed"),
          f("source", QFieldType.STRING, "Source").withPossibleValueSourceName(config.applyPrefix("change_source")).withMaxLength(20),
-         f("note", QFieldType.STRING, "Note").withMaxLength(2000)));
+         f("note", QFieldType.STRING, "Note").withMaxLength(2000),
+         f("lastSeenAt", QFieldType.DATE_TIME, "Last Observed"),
+         f("eventKey", QFieldType.STRING, "Source Event Key").withMaxLength(64),
+         f("siteKey", QFieldType.STRING, "Site").withMaxLength(200)));
 
       return (table(WebPerfSchema.CHANGE_EVENT, "Site Changes", "change_circle", fields, false, Set.of())
          .withRecordLabelFormatAndFields("%s %s", "kind", "toValue")
          .withSection(section("change", "Change", "change_circle", Tier.T1, "id", "kind", "component", "fromValue", "toValue", "observedAt",
-            "source", "note"))
+            "source", "note", "lastSeenAt", "eventKey", "siteKey"))
          .withSection(dates()));
    }
 
@@ -558,6 +585,7 @@ public final class WebPerfMetaData
    {
       List<QFieldMetaData> fields = withCommon(List.of(
          f("changeEventId", QFieldType.LONG, "Change").withPossibleValueSourceName(config.applyPrefix(WebPerfSchema.CHANGE_EVENT)),
+         f("targetId", QFieldType.LONG, "Page").withPossibleValueSourceName(config.applyPrefix(WebPerfSchema.TARGET)),
          f("scope", QFieldType.STRING, "Scope").withPossibleValueSourceName(config.applyPrefix("comparison_scope")).withMaxLength(20),
          f("source", QFieldType.STRING, "Source").withPossibleValueSourceName(config.applyPrefix("comparison_source")).withMaxLength(10),
          f("strategy", QFieldType.STRING, "Strategy or Form Factor").withPossibleValueSourceName(config.applyPrefix("strategy_or_form_factor")).withMaxLength(10),
@@ -574,7 +602,7 @@ public final class WebPerfMetaData
 
       return (table(WebPerfSchema.COMPARISON, "Before and After", "compare_arrows", fields, false, Set.of())
          .withRecordLabelFormatAndFields("%s %s", "metric", "segment")
-         .withSection(section("comparison", "Comparison", "compare_arrows", Tier.T1, "id", "changeEventId", "metric", "segment", "beforeValue",
+         .withSection(section("comparison", "Comparison", "compare_arrows", Tier.T1, "id", "changeEventId", "targetId", "metric", "segment", "beforeValue",
             "afterValue", "delta", "deltaPct"))
          .withSection(section("details", "Details", "info", Tier.T2, "scope", "source", "strategy", "beforeN", "beforeWindow", "afterN",
             "afterWindow"))

src/main/java/com/kingsrook/qbits/webperf/processes/RegressionSteps.java

@@ -0,0 +1,276 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.webperf.processes;
+
+
+import java.sql.SQLException;
+import java.time.Instant;
+import java.util.ArrayList;
+import java.util.List;
+import com.kingsrook.qbits.webperf.WebPerfRuntime;
+import com.kingsrook.qbits.webperf.service.WebPerfComparisonService;
+import com.kingsrook.qbits.webperf.service.WebPerfEvaluationService;
+import com.kingsrook.qqq.backend.core.exceptions.QException;
+import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLine;
+import com.kingsrook.qqq.backend.core.model.actions.processes.ProcessSummaryLineInterface;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepInput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunBackendStepOutput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.Status;
+import com.kingsrook.qqq.backend.core.model.data.QRecord;
+import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractLoadStep;
+import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractTransformStep;
+import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.ProcessSummaryProviderInterface;
+
+
+/*******************************************************************************
+ ** Standard streamed Review/Result actions for regression status and comparisons.
+ *******************************************************************************/
+public final class RegressionSteps
+{
+   public static final String NOTE = "resolutionNote";
+
+
+
+   /*******************************************************************************
+    ** Steps only.
+    *******************************************************************************/
+   private RegressionSteps()
+   {
+   }
+
+
+
+   /*******************************************************************************
+    ** Review open records; closed records are explicitly skipped.
+    *******************************************************************************/
+   public static class Transform extends AbstractTransformStep implements ProcessSummaryProviderInterface
+   {
+      private ProcessSummaryLine change;
+      private final ProcessSummaryLine skip = Lines.line(Status.INFO, "closed regression will be skipped", "closed regressions will be skipped",
+         "closed regression was skipped", "closed regressions were skipped");
+
+
+
+      /*******************************************************************************
+       ** Validate the note and classify this page without writing anything.
+       *******************************************************************************/
+      @Override
+      public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
+      {
+         String status = input.getValueString(WebPerfProcesses.TRIGGER_FIELD);
+         String note = input.getValueString(NOTE);
+         if(note == null || note.isBlank() || note.length() > 2000)
+         {
+            throw (new QException("Enter a resolution note of 1 to 2000 characters"));
+         }
+         if(change == null)
+         {
+            String verb = "DISMISSED".equals(status) ? "dismissed" : "marked resolved";
+            change = Lines.line(Status.OK, "regression will be " + verb, "regressions will be " + verb,
+               "regression was " + verb, "regressions were " + verb);
+         }
+         for(QRecord record : input.getRecords())
+         {
+            if(List.of("SUSPECTED", "CONFIRMED").contains(record.getValueString("status")))
+            {
+               change.incrementCountAndAddPrimaryKey(record.getValue("id"));
+               output.addRecord(record);
+            }
+            else
+            {
+               skip.incrementCountAndAddPrimaryKey(record.getValue("id"));
+            }
+         }
+      }
+
+
+
+      /*******************************************************************************
+       ** Review summary.
+       *******************************************************************************/
+      @Override
+      public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean result)
+      {
+         ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>();
+         if(change != null)
+         {
+            Lines.addAll(lines, change);
+         }
+         Lines.addAll(lines, skip);
+         return (lines);
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Apply the requested state under a database lock, rechecking eligibility.
+    *******************************************************************************/
+   public static class Load extends AbstractLoadStep implements ProcessSummaryProviderInterface
+   {
+      private ProcessSummaryLine changed;
+      private final ProcessSummaryLine skipped = Lines.line(Status.INFO, "regression will be skipped", "regressions will be skipped",
+         "regression was already closed", "regressions were already closed");
+
+
+
+      /*******************************************************************************
+       ** Apply one page and report only the rows actually changed.
+       *******************************************************************************/
+      @Override
+      public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
+      {
+         String status = input.getValueString(WebPerfProcesses.TRIGGER_FIELD);
+         if(changed == null)
+         {
+            String verb = "DISMISSED".equals(status) ? "dismissed" : "marked resolved";
+            changed = Lines.line(Status.OK, "regression will be " + verb, "regressions will be " + verb,
+               "regression was " + verb, "regressions were " + verb);
+         }
+         WebPerfEvaluationService service = new WebPerfEvaluationService(WebPerfRuntime.forInput(input).store());
+         try
+         {
+            for(QRecord record : input.getRecords())
+            {
+               boolean updated = service.finishRegression(record.getValueLong("id"), status, input.getValueString(NOTE), Instant.now());
+               (updated ? changed : skipped).incrementCountAndAddPrimaryKey(record.getValue("id"));
+            }
+         }
+         catch(SQLException | IllegalArgumentException e)
+         {
+            throw (new QException("Could not update the regressions", e));
+         }
+      }
+
+
+
+      /*******************************************************************************
+       ** Result summary.
+       *******************************************************************************/
+      @Override
+      public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean result)
+      {
+         ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>();
+         if(changed != null)
+         {
+            Lines.addAll(lines, changed);
+         }
+         Lines.addAll(lines, skipped);
+         if(getTransformStep() instanceof Transform transform)
+         {
+            Lines.addAll(lines, transform.skip);
+         }
+         return (lines);
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Review comparisons using recorded lab history.
+    *******************************************************************************/
+   public static class CompareTransform extends AbstractTransformStep implements ProcessSummaryProviderInterface
+   {
+      private final ProcessSummaryLine changes = Lines.line(Status.OK, "site change will be compared using recorded lab samples",
+         "site changes will be compared using recorded lab samples", "site change was compared", "site changes were compared");
+
+
+
+      /*******************************************************************************
+       ** Pass through a bounded page of selected changes.
+       *******************************************************************************/
+      @Override
+      public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
+      {
+         for(QRecord record : input.getRecords())
+         {
+            changes.incrementCountAndAddPrimaryKey(record.getValue("id"));
+            output.addRecord(record);
+         }
+      }
+
+
+
+      /*******************************************************************************
+       ** Review summary.
+       *******************************************************************************/
+      @Override
+      public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean result)
+      {
+         ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>();
+         Lines.addAll(lines, changes);
+         return (lines);
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Write comparisons when both sides have three comparable measurements.
+    *******************************************************************************/
+   public static class CompareLoad extends AbstractLoadStep implements ProcessSummaryProviderInterface
+   {
+      private final ProcessSummaryLine compared = Lines.line(Status.OK, "comparison will be recorded", "comparisons will be recorded",
+         "comparison was recorded", "comparisons were recorded");
+      private final ProcessSummaryLine insufficient = Lines.line(Status.INFO, "change needs more samples", "changes need more samples",
+         "change had fewer than three comparable samples on each side", "changes had fewer than three comparable samples on each side");
+
+
+
+      /*******************************************************************************
+       ** Compute and persist this page of selected changes.
+       *******************************************************************************/
+      @Override
+      public void runOnePage(RunBackendStepInput input, RunBackendStepOutput output) throws QException
+      {
+         try
+         {
+            WebPerfComparisonService service = new WebPerfComparisonService(WebPerfRuntime.forInput(input).store());
+            for(QRecord record : input.getRecords())
+            {
+               int count = service.compare(record.getValueLong("id"));
+               compared.incrementCount(count);
+               if(count == 0)
+               {
+                  insufficient.incrementCountAndAddPrimaryKey(record.getValue("id"));
+               }
+            }
+         }
+         catch(SQLException e)
+         {
+            throw (new QException("Could not compare the site changes", e));
+         }
+      }
+
+
+
+      /*******************************************************************************
+       ** Result summary, including an honest insufficient-history line.
+       *******************************************************************************/
+      @Override
+      public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput output, boolean result)
+      {
+         ArrayList<ProcessSummaryLineInterface> lines = new ArrayList<>();
+         Lines.addAll(lines, compared, insufficient);
+         return (lines);
+      }
+   }
+}

src/main/java/com/kingsrook/qbits/webperf/processes/RetryRunsSteps.java

@@ -144,8 +144,23 @@ public final class RetryRunsSteps
          String                             batch    = java.util.UUID.randomUUID().toString();
          for(QRecord record : input.getRecords())
          {
-            requests.add(new WebPerfService.RunRequest(record.getValueLong("targetId"), record.getValueString("url"), record.getValueString("strategy"),
-               "MANUAL", "retry of run " + record.getValue("id"), batch, Instant.now()));
+            if(List.of("CONFIRM", "CHANGE_COMPARE").contains(record.getValueString("triggerType")))
+            {
+               try
+               {
+                  String status = service.retrySample(record.getValueLong("id"), Instant.now());
+                  ("QUEUED".equals(status) ? queued : "MERGED".equals(status) ? merged : capped).incrementCount(1);
+               }
+               catch(SQLException e)
+               {
+                  throw (new QException("Could not retry the measurement sample", e));
+               }
+            }
+            else
+            {
+               requests.add(new WebPerfService.RunRequest(record.getValueLong("targetId"), record.getValueString("url"), record.getValueString("strategy"),
+                  "MANUAL", "retry of run " + record.getValue("id"), batch, Instant.now()));
+            }
          }
          try
          {

src/main/java/com/kingsrook/qbits/webperf/processes/ScheduledSteps.java

@@ -25,6 +25,7 @@ import java.time.Instant;
 import java.util.List;
 import java.util.Map;
 import com.kingsrook.qbits.webperf.WebPerfRuntime;
+import com.kingsrook.qbits.webperf.service.WebPerfChangeService;
 import com.kingsrook.qbits.webperf.service.WebPerfService;
 import com.kingsrook.qqq.backend.core.actions.processes.BackendStep;
 import com.kingsrook.qqq.backend.core.exceptions.QException;
@@ -54,6 +55,30 @@ public final class ScheduledSteps
 
 
 
+   /*******************************************************************************
+    ** Poll deploy probes and external change sources on the existing scheduler.
+    *******************************************************************************/
+   public static class ProbeChanges implements BackendStep
+   {
+      /*******************************************************************************
+       ** Store changes and queue the affected pages.
+       *******************************************************************************/
+      @Override
+      public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
+      {
+         try
+         {
+            new WebPerfChangeService(WebPerfRuntime.forInput(input).store()).poll(Instant.now());
+         }
+         catch(Exception e)
+         {
+            throw (new QException("Checking site changes failed", e));
+         }
+      }
+   }
+
+
+
    /*******************************************************************************
     ** Every minute: run due queued tests.
     *******************************************************************************/

src/main/java/com/kingsrook/qbits/webperf/processes/WebPerfProcesses.java

@@ -37,6 +37,7 @@ import com.kingsrook.qqq.backend.core.model.metadata.permissions.PermissionLevel
 import com.kingsrook.qqq.backend.core.model.metadata.permissions.QPermissionRules;
 import com.kingsrook.qqq.backend.core.model.metadata.processes.QBackendStepMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.processes.QFunctionInputMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.processes.QFrontendStepMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.processes.QProcessMetaData;
 import com.kingsrook.qqq.backend.core.model.metadata.scheduleing.QScheduleMetaData;
 import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractExtractStep;
@@ -63,6 +64,10 @@ import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwith
  *******************************************************************************/
 public final class WebPerfProcesses
 {
+   public static final String DISMISS_REGRESSIONS = "DismissRegressions";
+   public static final String RESOLVE_REGRESSIONS = "ResolveRegressions";
+   public static final String COMPARE_CHANGE = "CompareChange";
+   public static final String PROBE_DEPLOYS = "ProbeDeploys";
    public static final String TEST_PAGE_SPEED     = "TestPageSpeed";
    public static final String RUN_SITE_AUDIT      = "RunSiteAudit";
    public static final String SYNC_TARGETS        = "SyncPages";
@@ -146,6 +151,29 @@ public final class WebPerfProcesses
          processes.add(scheduled(config, SCHEDULED_FIELD, "Refresh Google field data (scheduled)", ScheduledSteps.FieldData.class,
             new QScheduleMetaData().withCronExpression(FIELD_CRON).withCronTimeZoneId("UTC")));
       }
+      String regressions = config.applyPrefix(WebPerfSchema.REGRESSION);
+      for(String status : List.of("DISMISSED", "RESOLVED"))
+      {
+         boolean dismiss = "DISMISSED".equals(status);
+         QProcessMetaData process = etl(config, dismiss ? DISMISS_REGRESSIONS : RESOLVE_REGRESSIONS, dismiss ? "Dismiss" : "Mark resolved", "done",
+            regressions, regressions, ExtractViaQueryStep.class, RegressionSteps.Transform.class, RegressionSteps.Load.class, status, "manage", 1,
+            "Open regressions will be updated. Closed regressions will be skipped.");
+         process.withStep(0, new QFrontendStepMetaData().withName("note").withLabel("Resolution note")
+            .withFormField(new QFieldMetaData(RegressionSteps.NOTE, QFieldType.STRING).withLabel("Resolution note").withIsRequired(true).withMaxLength(2000)));
+         processes.add(process);
+      }
+      if(config.isOn(WebPerfFeature.CDN_COMPARE))
+      {
+         String changes = config.applyPrefix(WebPerfSchema.CHANGE_EVENT);
+         processes.add(etl(config, COMPARE_CHANGE, "Compare before and after", "compare", changes, config.applyPrefix(WebPerfSchema.COMPARISON),
+            ExtractViaQueryStep.class, RegressionSteps.CompareTransform.class, RegressionSteps.CompareLoad.class, null, "view", 1,
+            "Lab comparisons require three successful samples on each side, on the same Lighthouse major version."));
+      }
+      if(config.isOn(WebPerfFeature.DEPLOY_PROBES) || !config.getChangeEventSources().isEmpty())
+      {
+         processes.add(scheduled(config, PROBE_DEPLOYS, "Check site changes", ScheduledSteps.ProbeChanges.class,
+            new QScheduleMetaData().withRepeatSeconds(300).withInitialDelaySeconds(1)));
+      }
       return (processes);
    }
 

src/main/java/com/kingsrook/qbits/webperf/service/WebPerfChangeService.java

@@ -0,0 +1,246 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.webperf.service;
+
+
+import java.net.URI;
+import java.nio.charset.StandardCharsets;
+import java.sql.SQLException;
+import java.time.Instant;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import com.kingsrook.qbits.webperf.WebPerfFeature;
+import com.kingsrook.qbits.webperf.WebPerfQBitConfig;
+import com.kingsrook.qbits.webperf.WebPerfRuntime;
+import com.kingsrook.qbits.webperf.WebPerfSchema;
+import com.kingsrook.qbits.webperf.spi.ChangeEventSource;
+import com.kingsrook.qbits.webperf.spi.DeployProbe;
+import com.kingsrook.qbits.webperf.store.WebPerfStore;
+import org.json.JSONArray;
+
+
+/*******************************************************************************
+ ** Records source changes and attributes runs to the state observed at start.
+ ** All identities and queue writes are durable; polling can safely repeat.
+ *******************************************************************************/
+public class WebPerfChangeService
+{
+   private final WebPerfStore store;
+   private final WebPerfQBitConfig config;
+
+
+
+   /*******************************************************************************
+    ** Changes for one installation.
+    *******************************************************************************/
+   public WebPerfChangeService(WebPerfStore store)
+   {
+      this.store = store;
+      this.config = store.getConfig();
+   }
+
+
+
+   /*******************************************************************************
+    ** Poll configured sources. Failed probes propagate so the schedule retries.
+    *******************************************************************************/
+   public void poll(Instant now) throws Exception
+   {
+      if(config.isOn(WebPerfFeature.DEPLOY_PROBES))
+      {
+         for(DeployProbe probe : config.getDeployProbes())
+         {
+            String build = probe.currentBuildId();
+            if(build != null && !build.isBlank())
+            {
+               String kind = "BACKEND".equalsIgnoreCase(probe.component()) ? "BACKEND_DEPLOY" : "FRONTEND_DEPLOY";
+               Map<String, Object> previous = latest(kind, probe.component(), now);
+               record(new ChangeEventSource.Change(kind, probe.component(), previous == null ? null : (String) previous.get("to_value"), build, now),
+                  "PROBE", probe.component() + ":" + build, true);
+            }
+         }
+      }
+      for(int i = 0; i < config.getChangeEventSources().size(); i++)
+      {
+         String sourceKey = "source:" + i;
+         List<Map<String, Object>> last = store.query("SELECT MAX(observed_at) AS observed_at FROM {change_event} WHERE site_key = ? AND note = ?",
+            siteKey(), sourceKey);
+         Instant since = (Instant) last.get(0).get("observed_at");
+         for(ChangeEventSource.Change change : config.getChangeEventSources().get(i).changesSince(since == null ? Instant.EPOCH : since))
+         {
+            if(!"CDN_STAGE".equals(change.kind()) || config.isOn(WebPerfFeature.CDN_COMPARE))
+            {
+               record(change, "CDN_STAGE".equals(change.kind()) ? "QBIT_CDN" : "WEBHOOK", sourceKey, true);
+            }
+         }
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Insert once per build or source event and queue affected pages atomically.
+    ** The source SPI has no event id, so its complete tuple is its identity.
+    *******************************************************************************/
+   public long record(ChangeEventSource.Change change, String source, String sourceKey, boolean queue) throws SQLException
+   {
+      String identity = new JSONArray(List.of(siteKey(), source, sourceKey, change.kind(), change.component(), change.toValue(),
+         "PROBE".equals(source) ? "" : change.observedAt().toString(), "PROBE".equals(source) ? "" : String.valueOf(change.fromValue()))).toString();
+      String key = WebPerfService.sha256(identity.getBytes(StandardCharsets.UTF_8));
+      try
+      {
+         return (store.transaction(tx ->
+         {
+            List<Map<String, Object>> existing = tx.query("SELECT id FROM {change_event} WHERE event_key = ?", key);
+            if(!existing.isEmpty())
+            {
+               if("PROBE".equals(source))
+               {
+                  tx.update("UPDATE {change_event} SET last_seen_at = ? WHERE id = ? AND (last_seen_at IS NULL OR last_seen_at < ?)",
+                     change.observedAt(), existing.get(0).get("id"), change.observedAt());
+               }
+               return (WebPerfStore.asLong(existing.get(0).get("id")));
+            }
+            Map<String, Object> values = new LinkedHashMap<>();
+            values.put("kind", change.kind());
+            values.put("component", change.component());
+            values.put("from_value", change.fromValue());
+            values.put("to_value", change.toValue());
+            values.put("observed_at", change.observedAt());
+            values.put("last_seen_at", change.observedAt());
+            values.put("source", source);
+            values.put("note", sourceKey);
+            values.put("event_key", key);
+            values.put("site_key", siteKey());
+            long id = tx.insert(WebPerfSchema.CHANGE_EVENT, values);
+            if(queue && config.isOn(WebPerfFeature.LAB))
+            {
+               queueChange(tx, id, change);
+            }
+            return (id);
+         }));
+      }
+      catch(SQLException e)
+      {
+         /////////////////////////////////////////////////////////////////////
+         // A competing poll may have committed this exact identity first. //
+         /////////////////////////////////////////////////////////////////////
+         if("23505".equals(e.getSQLState()))
+         {
+            List<Map<String, Object>> existing = store.query("SELECT id FROM {change_event} WHERE event_key = ?", key);
+            if(!existing.isEmpty())
+            {
+               return (WebPerfStore.asLong(existing.get(0).get("id")));
+            }
+         }
+         throw (e);
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Page through targets; CDN sampling bypasses the ordinary queue merge.
+    *******************************************************************************/
+   private void queueChange(WebPerfStore tx, long eventId, ChangeEventSource.Change change) throws SQLException
+   {
+      WebPerfService service = new WebPerfService(new WebPerfRuntime(config), tx);
+      boolean cdn = "CDN_STAGE".equals(change.kind());
+      boolean deploy = change.kind().endsWith("DEPLOY");
+      long after = 0;
+      while(true)
+      {
+         List<Map<String, Object>> pages = tx.query("SELECT id, url FROM {target} WHERE site_key = ? AND enabled = TRUE AND id > ?"
+            + (cdn || deploy ? " AND tier = 'KEY'" : "") + " ORDER BY id FETCH FIRST 100 ROWS ONLY", siteKey(), after);
+         if(pages.isEmpty())
+         {
+            return;
+         }
+         for(Map<String, Object> page : pages)
+         {
+            after = WebPerfStore.asLong(page.get("id"));
+            for(String strategy : config.getStrategies())
+            {
+               if(cdn && ("HEADERS_ONLY".equals(change.toValue()) || "LIVE".equals(change.toValue())))
+               {
+                  for(int sample = 0; sample < 3; sample++)
+                  {
+                     service.enqueueSample(page, strategy, "CHANGE_COMPARE", "change:" + eventId + ":1:" + sample,
+                        change.observedAt().plusSeconds("LIVE".equals(change.toValue()) ? 3600 : 0));
+                     if("LIVE".equals(change.toValue()))
+                     {
+                        service.enqueueSample(page, strategy, "CHANGE_COMPARE", "change:" + eventId + ":24:" + sample, change.observedAt().plusSeconds(86400));
+                     }
+                  }
+               }
+               else
+               {
+                  Instant due = change.observedAt().plusSeconds(deploy ? config.getDeploySettleSeconds() : config.getPublishSettleSeconds());
+                  service.enqueue(List.of(new WebPerfService.RunRequest(after, (String) page.get("url"), strategy,
+                     deploy ? "DEPLOY" : "PUBLISH", "change:" + eventId, null, due)), change.observedAt());
+               }
+            }
+         }
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Latest observed state at a time, optionally for a specific component.
+    *******************************************************************************/
+   public Map<String, Object> latest(String kind, String component, Instant at) throws SQLException
+   {
+      List<Map<String, Object>> rows = component == null
+         ? store.query("SELECT * FROM {change_event} WHERE site_key = ? AND kind = ? AND observed_at <= ? ORDER BY last_seen_at DESC, id DESC FETCH FIRST 1 ROWS ONLY",
+            siteKey(), kind, at)
+         : store.query("SELECT * FROM {change_event} WHERE site_key = ? AND kind = ? AND component = ? AND observed_at <= ? ORDER BY last_seen_at DESC, id DESC FETCH FIRST 1 ROWS ONLY",
+            siteKey(), kind, component, at);
+      return (rows.isEmpty() ? null : rows.get(0));
+   }
+
+
+
+   /*******************************************************************************
+    ** Attribute at execution, not queue time (a deployment may happen meanwhile).
+    *******************************************************************************/
+   public void attribute(long runId, Instant now) throws SQLException
+   {
+      Map<String, Object> build = latest("FRONTEND_DEPLOY", null, now);
+      Map<String, Object> cdn = latest("CDN_STAGE", URI.create(config.getSiteBaseUrl()).getHost(), now);
+      List<Map<String, Object>> cause = store.query("SELECT id FROM {change_event} WHERE site_key = ? AND observed_at <= ? ORDER BY observed_at DESC, id DESC FETCH FIRST 1 ROWS ONLY",
+         siteKey(), now);
+      store.update("UPDATE {lab_run} SET deployment_id = ?, cdn_stage = ?, change_event_id = ? WHERE id = ?",
+         build == null ? null : build.get("to_value"), cdn == null ? null : cdn.get("to_value"), cause.isEmpty() ? null : cause.get(0).get("id"), runId);
+   }
+
+
+
+   /*******************************************************************************
+    ** Same site identity as the tested pages.
+    *******************************************************************************/
+   String siteKey()
+   {
+      return ("host:" + URI.create(config.getSiteBaseUrl()).getHost().toLowerCase(Locale.ROOT));
+   }
+}

src/main/java/com/kingsrook/qbits/webperf/service/WebPerfComparisonService.java

@@ -0,0 +1,188 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.webperf.service;
+
+
+import java.sql.SQLException;
+import java.time.Duration;
+import java.time.Instant;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import com.kingsrook.qbits.webperf.WebPerfSchema;
+import com.kingsrook.qbits.webperf.store.WebPerfStore;
+
+
+/*******************************************************************************
+ ** Reproducible before/after lab medians, with counts and actual sample windows.
+ ** Three successful samples on each side and one Lighthouse major are required.
+ *******************************************************************************/
+public class WebPerfComparisonService
+{
+   private final WebPerfStore store;
+
+
+
+   /*******************************************************************************
+    ** Comparisons for one installation.
+    *******************************************************************************/
+   public WebPerfComparisonService(WebPerfStore store)
+   {
+      this.store = store;
+   }
+
+
+
+   /*******************************************************************************
+    ** Recompute the run's attributed change after its results have been stored.
+    *******************************************************************************/
+   public void compareForRun(Map<String, Object> run) throws SQLException
+   {
+      if(run.get("change_event_id") != null)
+      {
+         List<Map<String, Object>> events = store.query("SELECT * FROM {change_event} WHERE id = ?", run.get("change_event_id"));
+         if(!events.isEmpty())
+         {
+            compareTarget(events.get(0), WebPerfStore.asLong(run.get("target_id")), (String) run.get("strategy"));
+         }
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Record action: page through enabled targets and compare existing evidence.
+    ** Sparse history stays absent instead of being filled with estimated values.
+    *******************************************************************************/
+   public int compare(long eventId) throws SQLException
+   {
+      return (store.transaction(tx ->
+      {
+         Map<String, Object> event = tx.query("SELECT * FROM {change_event} WHERE id = ? FOR UPDATE", eventId).get(0);
+         WebPerfComparisonService service = new WebPerfComparisonService(tx);
+         int count = 0;
+         long after = 0;
+         while(true)
+         {
+            List<Map<String, Object>> pages = tx.query("SELECT id FROM {target} WHERE enabled = TRUE AND site_key = ? AND id > ? ORDER BY id FETCH FIRST 100 ROWS ONLY",
+               event.get("site_key"), after);
+            if(pages.isEmpty())
+            {
+               return (count);
+            }
+            for(Map<String, Object> page : pages)
+            {
+               after = WebPerfStore.asLong(page.get("id"));
+               for(String strategy : store.getConfig().getStrategies())
+               {
+                  count += service.compareTarget(event, after, strategy);
+               }
+            }
+         }
+      }));
+   }
+
+
+
+   /*******************************************************************************
+    ** Serialize comparison upserts on the change row, shared by all callers.
+    *******************************************************************************/
+   private int compareTarget(Map<String, Object> event, long targetId, String strategy) throws SQLException
+   {
+      store.query("SELECT id FROM {change_event} WHERE id = ? FOR UPDATE", event.get("id"));
+      Instant at = (Instant) event.get("observed_at");
+      List<Map<String, Object>> after = store.query("SELECT * FROM {lab_run} WHERE target_id = ? AND strategy = ? AND status = 'SUCCEEDED'"
+         + " AND started_at >= ? AND change_event_id = ? ORDER BY completed_at DESC, id DESC FETCH FIRST 3 ROWS ONLY", targetId, strategy, at, event.get("id"));
+      if(after.size() != 3)
+      {
+         return (0);
+      }
+      String major = WebPerfEvaluationService.major(after.get(0));
+      if(after.stream().anyMatch(row -> !major.equals(WebPerfEvaluationService.major(row))))
+      {
+         return (0);
+      }
+      List<Map<String, Object>> before = store.query("SELECT * FROM {lab_run} WHERE target_id = ? AND strategy = ? AND status = 'SUCCEEDED'"
+         + " AND completed_at < ? AND completed_at >= ? AND (lighthouse_version = ? OR lighthouse_version LIKE ?)"
+         + " ORDER BY completed_at DESC, id DESC FETCH FIRST 3 ROWS ONLY", targetId, strategy, at, at.minus(Duration.ofDays(30)), major, major + ".%");
+      if(before.size() != 3)
+      {
+         return (0);
+      }
+      int count = 0;
+      for(String metric : List.of("PERFORMANCE_SCORE", "LCP", "FCP", "TTFB"))
+      {
+         String column = WebPerfEvaluationService.LAB_METRICS.get(metric);
+         if(before.stream().anyMatch(row -> row.get(column) == null) || after.stream().anyMatch(row -> row.get(column) == null))
+         {
+            continue;
+         }
+         double beforeValue = WebPerfEvaluationService.median(before, column);
+         double afterValue = WebPerfEvaluationService.median(after, column);
+         double delta = afterValue - beforeValue;
+         Double percentage = beforeValue == 0 ? null : delta * 100 / beforeValue;
+         if(percentage != null && Math.abs(percentage) >= 1000000)
+         {
+            percentage = null;
+         }
+         List<Map<String, Object>> existing = store.query("SELECT id FROM {comparison} WHERE change_event_id = ? AND target_id = ? AND source = 'LAB'"
+            + " AND strategy = ? AND metric = ? AND segment = 'ALL'", event.get("id"), targetId, strategy, metric);
+         if(existing.isEmpty())
+         {
+            Map<String, Object> row = new LinkedHashMap<>();
+            row.put("change_event_id", event.get("id"));
+            row.put("target_id", targetId);
+            row.put("scope", "TARGET");
+            row.put("source", "LAB");
+            row.put("strategy", strategy);
+            row.put("metric", metric);
+            row.put("segment", "ALL");
+            row.put("before_value", beforeValue);
+            row.put("before_n", 3);
+            row.put("before_window", window(before));
+            row.put("after_value", afterValue);
+            row.put("after_n", 3);
+            row.put("after_window", window(after));
+            row.put("delta", delta);
+            row.put("delta_pct", percentage);
+            store.insert(WebPerfSchema.COMPARISON, row);
+         }
+         else
+         {
+            store.update("UPDATE {comparison} SET before_value = ?, before_n = 3, before_window = ?, after_value = ?, after_n = 3, after_window = ?,"
+               + " delta = ?, delta_pct = ?, modify_date = ? WHERE id = ?", beforeValue, window(before), afterValue, window(after), delta, percentage,
+               Instant.now(), existing.get(0).get("id"));
+         }
+         count++;
+      }
+      return (count);
+   }
+
+
+
+   /*******************************************************************************
+    ** The actual sample times, not an implied continuous measurement window.
+    *******************************************************************************/
+   private static String window(List<Map<String, Object>> samples)
+   {
+      return (samples.get(samples.size() - 1).get("completed_at") + " / " + samples.get(0).get("completed_at"));
+   }
+}

src/main/java/com/kingsrook/qbits/webperf/service/WebPerfEvaluationService.java

@@ -0,0 +1,611 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.webperf.service;
+
+
+import java.sql.SQLException;
+import java.time.Duration;
+import java.time.Instant;
+import java.util.ArrayList;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Objects;
+import com.kingsrook.qbits.webperf.WebPerfFeature;
+import com.kingsrook.qbits.webperf.WebPerfRuntime;
+import com.kingsrook.qbits.webperf.WebPerfSchema;
+import com.kingsrook.qbits.webperf.spi.ChangeEventSource;
+import com.kingsrook.qbits.webperf.store.WebPerfStore;
+import com.kingsrook.qqq.backend.core.alerts.AlertRecordRef;
+import com.kingsrook.qqq.backend.core.alerts.AlertSeverity;
+import com.kingsrook.qqq.backend.core.alerts.AlertSignal;
+import com.kingsrook.qqq.backend.core.alerts.QAlerts;
+
+
+/*******************************************************************************
+ ** Budget breaches and confirmed lab regressions, backed by ordinary records.
+ ** Target row locks serialize evaluation across workers. Baselines and samples
+ ** are bounded, and evaluation progress commits with the resulting records.
+ *******************************************************************************/
+public class WebPerfEvaluationService
+{
+   public static final String ALERT_TYPE = "webperf.regression";
+   public static final Map<String, String> LAB_METRICS = Map.of(
+      "PERFORMANCE_SCORE", "score_performance", "ACCESSIBILITY_SCORE", "score_accessibility",
+      "BEST_PRACTICES_SCORE", "score_best_practices", "SEO_SCORE", "score_seo",
+      "LCP", "lcp_ms", "FCP", "fcp_ms", "CLS", "cls", "TBT", "tbt_ms", "TTFB", "ttfb_ms");
+   private final WebPerfStore store;
+
+
+
+   /*******************************************************************************
+    ** Evaluation for one installation.
+    *******************************************************************************/
+   public WebPerfEvaluationService(WebPerfStore store)
+   {
+      this.store = store;
+   }
+
+
+
+   /*******************************************************************************
+    ** Bounded work per worker tick, including results left by an interrupted tick.
+    *******************************************************************************/
+   public void evaluatePending() throws SQLException
+   {
+      for(Map<String, Object> run : store.query("SELECT id FROM {lab_run} WHERE status = 'SUCCEEDED' AND evaluated_at IS NULL ORDER BY completed_at, id FETCH FIRST 100 ROWS ONLY"))
+      {
+         evaluateLab(WebPerfStore.asLong(run.get("id")));
+      }
+      for(Map<String, Object> field : store.query("SELECT id FROM {field_snapshot} WHERE evaluated_at IS NULL ORDER BY id FETCH FIRST 100 ROWS ONLY"))
+      {
+         evaluateField(WebPerfStore.asLong(field.get("id")));
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Evaluate a stored successful result once; publish alerts after commit.
+    *******************************************************************************/
+   public void evaluateLab(long runId) throws SQLException
+   {
+      List<Long> changed = store.transaction(tx ->
+      {
+         List<Map<String, Object>> runs = tx.query("SELECT * FROM {lab_run} WHERE id = ? AND status = 'SUCCEEDED'", runId);
+         if(runs.isEmpty())
+         {
+            return (List.of());
+         }
+         Map<String, Object> run = runs.get(0);
+         List<Map<String, Object>> targets = tx.query("SELECT * FROM {target} WHERE id = ? FOR UPDATE", run.get("target_id"));
+         if(targets.isEmpty() || !tx.query("SELECT id FROM {lab_run} WHERE id = ? AND evaluated_at IS NOT NULL", runId).isEmpty())
+         {
+            return (List.of());
+         }
+         Map<String, Object> target = targets.get(0);
+         WebPerfEvaluationService evaluator = new WebPerfEvaluationService(tx);
+         List<Long> transitions = new ArrayList<>();
+         evaluator.versionChange(run, transitions);
+         List<Map<String, Object>> baseline = evaluator.baseline(run);
+         for(Map.Entry<String, String> metric : LAB_METRICS.entrySet())
+         {
+            if(run.get(metric.getValue()) != null)
+            {
+               evaluator.evaluateMetric(target, run, metric.getKey(), metric.getValue(), baseline, transitions);
+            }
+         }
+         evaluator.updateCount(target.get("id"));
+         if(store.getConfig().isOn(WebPerfFeature.CDN_COMPARE))
+         {
+            new WebPerfComparisonService(tx).compareForRun(run);
+         }
+         tx.update("UPDATE {lab_run} SET evaluated_at = ? WHERE id = ?", Instant.now(), runId);
+         return (transitions);
+      });
+      changed.forEach(this::sendAlert);
+   }
+
+
+
+   /*******************************************************************************
+    ** CrUX URL budgets use real p75 values, never NO_DATA, errors or origin
+    ** fallback as page measurements. A new collection period can resolve a breach.
+    *******************************************************************************/
+   public void evaluateField(long snapshotId) throws SQLException
+   {
+      List<Long> changed = store.transaction(tx ->
+      {
+         Map<String, Object> field = tx.query("SELECT * FROM {field_snapshot} WHERE id = ? FOR UPDATE", snapshotId).get(0);
+         List<Long> transitions = new ArrayList<>();
+         if(field.get("evaluated_at") != null)
+         {
+            return (transitions);
+         }
+         if("OK".equals(field.get("status")) && "URL".equals(field.get("scope")))
+         {
+            List<Map<String, Object>> targets = tx.query("SELECT * FROM {target} WHERE site_key = ? AND url = ? FOR UPDATE",
+               new WebPerfChangeService(tx).siteKey(), field.get("scope_key"));
+            WebPerfEvaluationService evaluator = new WebPerfEvaluationService(tx);
+            for(Map<String, Object> target : targets)
+            {
+               for(String metric : List.of("LCP", "INP", "CLS", "FCP", "TTFB"))
+               {
+                  String column = "p75_" + metric.toLowerCase(Locale.ROOT) + ("CLS".equals(metric) ? "" : "_ms");
+                  Double value = WebPerfStore.asDouble(field.get(column));
+                  if(value == null)
+                  {
+                     continue;
+                  }
+                  String strategy = "PHONE".equals(field.get("form_factor")) ? "MOBILE" : (String) field.get("form_factor");
+                  Map<String, Object> budget = evaluator.breachedBudget(target, "CRUX", strategy, metric, value);
+                  List<Map<String, Object>> open = tx.query("SELECT * FROM {regression} WHERE target_id = ? AND source = 'CRUX' AND strategy = ? AND metric = ?"
+                     + " AND status = 'CONFIRMED' FETCH FIRST 1 ROWS ONLY", target.get("id"), field.get("form_factor"), metric);
+                  Instant at = (Instant) field.get("last_checked_at");
+                  if(budget != null && open.isEmpty())
+                  {
+                     Map<String, Object> row = new LinkedHashMap<>();
+                     row.put("target_id", target.get("id"));
+                     row.put("content_table", target.get("content_table"));
+                     row.put("content_id", target.get("content_id"));
+                     row.put("source", "CRUX");
+                     row.put("strategy", field.get("form_factor"));
+                     row.put("metric", metric);
+                     row.put("current_value", value);
+                     row.put("current_ref", "field:" + snapshotId);
+                     row.put("budget_id", budget.get("id"));
+                     row.put("severity", budget.get("severity"));
+                     row.put("status", "CONFIRMED");
+                     row.put("detected_at", at);
+                     row.put("confirmed_at", at);
+                     transitions.add(tx.insert(WebPerfSchema.REGRESSION, row));
+                  }
+                  else if(!open.isEmpty())
+                  {
+                     Map<String, Object> regression = open.get(0);
+                     tx.update("UPDATE {regression} SET current_value = ?, current_ref = ?, modify_date = ? WHERE id = ?",
+                        value, "field:" + snapshotId, at, regression.get("id"));
+                     if(budget == null)
+                     {
+                        evaluator.close(regression, "RESOLVED", "Google field data is within its budget", at);
+                        transitions.add(WebPerfStore.asLong(regression.get("id")));
+                     }
+                  }
+               }
+               evaluator.updateCount(target.get("id"));
+            }
+         }
+         tx.update("UPDATE {field_snapshot} SET evaluated_at = ? WHERE id = ?", Instant.now(), snapshotId);
+         return (transitions);
+      });
+      changed.forEach(this::sendAlert);
+   }
+
+
+
+   /*******************************************************************************
+    ** Five eligible results in the preceding 30 days, excluding the new result.
+    *******************************************************************************/
+   private List<Map<String, Object>> baseline(Map<String, Object> run) throws SQLException
+   {
+      return (store.query("SELECT * FROM {lab_run} WHERE target_id = ? AND strategy = ? AND status = 'SUCCEEDED'"
+         + " AND trigger_type IN ('SCHEDULE', 'CONFIRM', 'SITE_AUDIT') AND completed_at >= ?"
+         + " AND (completed_at < ? OR (completed_at = ? AND id < ?)) AND (lighthouse_version = ? OR lighthouse_version LIKE ?)"
+         + " ORDER BY completed_at DESC, id DESC FETCH FIRST 5 ROWS ONLY", run.get("target_id"), run.get("strategy"),
+         ((Instant) run.get("completed_at")).minus(Duration.ofDays(30)), run.get("completed_at"), run.get("completed_at"), run.get("id"),
+         major(run), major(run) + ".%"));
+   }
+
+
+
+   /*******************************************************************************
+    ** Major-version changes are markers and retire incompatible open baselines.
+    *******************************************************************************/
+   private void versionChange(Map<String, Object> run, List<Long> changed) throws SQLException
+   {
+      List<Map<String, Object>> previous = store.query("SELECT lighthouse_version FROM {lab_run} WHERE target_id = ? AND strategy = ?"
+         + " AND status = 'SUCCEEDED' AND id < ? ORDER BY id DESC FETCH FIRST 1 ROWS ONLY", run.get("target_id"), run.get("strategy"), run.get("id"));
+      if(!previous.isEmpty() && !major(previous.get(0)).equals(major(run)))
+      {
+         new WebPerfChangeService(store).record(new ChangeEventSource.Change("LIGHTHOUSE_VERSION", "LIGHTHOUSE", major(previous.get(0)), major(run),
+            (Instant) run.get("completed_at")), "PROBE", "lighthouse:" + major(run), false);
+      }
+      for(Map<String, Object> open : store.query("SELECT * FROM {regression} WHERE target_id = ? AND strategy = ? AND source = 'LAB'"
+         + " AND status IN ('SUSPECTED', 'CONFIRMED') AND lighthouse_major <> ?", run.get("target_id"), run.get("strategy"), major(run)))
+      {
+         close(open, "CONFIRMED".equals(open.get("status")) ? "RESOLVED" : "NOT_CONFIRMED", "Lighthouse major version changed; baseline reset",
+            (Instant) run.get("completed_at"));
+         changed.add(WebPerfStore.asLong(open.get("id")));
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Keep a single open row per page, source, strategy and metric. Its baseline
+    ** stays fixed through confirmation and recovery rather than drifting.
+    *******************************************************************************/
+   private void evaluateMetric(Map<String, Object> target, Map<String, Object> run, String metric, String column,
+                               List<Map<String, Object>> baseline, List<Long> changed) throws SQLException
+   {
+      List<Map<String, Object>> existing = store.query("SELECT * FROM {regression} WHERE target_id = ? AND strategy = ? AND source = 'LAB' AND metric = ?"
+         + " AND status IN ('SUSPECTED', 'CONFIRMED') FETCH FIRST 1 ROWS ONLY", target.get("id"), run.get("strategy"), metric);
+      if(!existing.isEmpty())
+      {
+         advance(target, run, metric, column, existing.get(0), changed);
+         return;
+      }
+      Double value = WebPerfStore.asDouble(run.get(column));
+      Double reference = baseline.size() == 5 && baseline.stream().allMatch(row -> row.get(column) != null) ? median(baseline, column) : null;
+      Map<String, Object> budget = breachedBudget(target, "LAB", (String) run.get("strategy"), metric, value);
+      List<String> failedAudits = newFailures(run, metric);
+      if(!drift(metric, reference, value) && budget == null && failedAudits.isEmpty())
+      {
+         return;
+      }
+      boolean deterministic = metric.endsWith("_SCORE") && !"PERFORMANCE_SCORE".equals(metric);
+      Instant at = (Instant) run.get("completed_at");
+      Map<String, Object> row = new LinkedHashMap<>();
+      row.put("target_id", target.get("id"));
+      row.put("content_table", target.get("content_table"));
+      row.put("content_id", target.get("content_id"));
+      row.put("source", "LAB");
+      row.put("strategy", run.get("strategy"));
+      row.put("metric", metric);
+      row.put("baseline_value", reference);
+      row.put("baseline_ref", reference == null ? null : refs(baseline));
+      row.put("current_value", value);
+      row.put("current_ref", run.get("id").toString());
+      row.put("delta", reference == null ? null : value - reference);
+      row.put("budget_id", budget == null ? null : budget.get("id"));
+      row.put("severity", budget == null ? "WARNING" : budget.get("severity"));
+      row.put("status", deterministic ? "CONFIRMED" : "SUSPECTED");
+      row.put("detected_at", at);
+      row.put("confirmed_at", deterministic ? at : null);
+      row.put("cause_change_event_id", run.get("change_event_id"));
+      row.put("first_run_id", run.get("id"));
+      row.put("checkpoint_run_id", run.get("id"));
+      row.put("lighthouse_major", major(run));
+      row.put("failed_audits", String.join(",", failedAudits));
+      long id = store.insert(WebPerfSchema.REGRESSION, row);
+      if(deterministic)
+      {
+         changed.add(id);
+      }
+      else
+      {
+         WebPerfService service = new WebPerfService(new WebPerfRuntime(store.getConfig()), store);
+         for(int sample = 0; sample < 2; sample++)
+         {
+            service.enqueueSample(target, (String) run.get("strategy"), "CONFIRM", "confirm:" + run.get("id") + ":" + sample, at);
+         }
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Confirm with the original plus two explicit samples. Recover only after
+    ** three later successful runs on the same major version.
+    *******************************************************************************/
+   private void advance(Map<String, Object> target, Map<String, Object> run, String metric, String column,
+                        Map<String, Object> open, List<Long> changed) throws SQLException
+   {
+      boolean suspected = "SUSPECTED".equals(open.get("status"));
+      List<Map<String, Object>> samples = suspected
+         ? store.query("SELECT * FROM {lab_run} WHERE target_id = ? AND strategy = ? AND status = 'SUCCEEDED' AND id <= ?"
+            + " AND (id = ? OR trigger_ref IN (?, ?)) AND (lighthouse_version = ? OR lighthouse_version LIKE ?) ORDER BY id FETCH FIRST 3 ROWS ONLY",
+            target.get("id"), run.get("strategy"), run.get("id"), open.get("first_run_id"), "confirm:" + open.get("first_run_id") + ":0",
+            "confirm:" + open.get("first_run_id") + ":1", major(run), major(run) + ".%")
+         : store.query("SELECT * FROM {lab_run} WHERE target_id = ? AND strategy = ? AND status = 'SUCCEEDED' AND id > ? AND id <= ?"
+            + " AND (lighthouse_version = ? OR lighthouse_version LIKE ?) ORDER BY id DESC FETCH FIRST 3 ROWS ONLY",
+            target.get("id"), run.get("strategy"), open.get("checkpoint_run_id"), run.get("id"), major(run), major(run) + ".%");
+      if(samples.size() < 3 || samples.stream().anyMatch(row -> row.get(column) == null))
+      {
+         return;
+      }
+      double value = median(samples, column);
+      Double reference = WebPerfStore.asDouble(open.get("baseline_value"));
+      Map<String, Object> budget = breachedBudget(target, "LAB", (String) run.get("strategy"), metric, value);
+      boolean breached = drift(metric, reference, value) || budget != null || auditsStillFail(open, samples);
+      Instant at = (Instant) run.get("completed_at");
+      store.update("UPDATE {regression} SET current_value = ?, current_ref = ?, delta = ?, budget_id = ?, severity = ?, modify_date = ? WHERE id = ?",
+         value, refs(samples), reference == null ? null : value - reference, budget == null ? null : budget.get("id"),
+         budget == null ? "WARNING" : budget.get("severity"), at, open.get("id"));
+      if(suspected && breached)
+      {
+         store.update("UPDATE {regression} SET status = 'CONFIRMED', confirmed_at = ?, checkpoint_run_id = ? WHERE id = ?", at, run.get("id"), open.get("id"));
+         changed.add(WebPerfStore.asLong(open.get("id")));
+      }
+      else if(!breached)
+      {
+         close(open, suspected ? "NOT_CONFIRMED" : "RESOLVED", suspected ? "Median of three did not confirm the regression" : "Recovered: median of three later runs", at);
+         changed.add(WebPerfStore.asLong(open.get("id")));
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Match enabled budgets by source, strategy and every supported scope.
+    ** The comparator describes acceptable values; a breach is its inverse.
+    *******************************************************************************/
+   public Map<String, Object> breachedBudget(Map<String, Object> target, String source, String strategy, String metric, double value) throws SQLException
+   {
+      Map<String, Object> winner = null;
+      long after = 0;
+      while(true)
+      {
+         List<Map<String, Object>> budgets = store.query("SELECT * FROM {budget} WHERE enabled = TRUE AND source = ? AND metric = ? AND id > ?"
+            + " AND (strategy IS NULL OR strategy = '' OR strategy = ?) ORDER BY id FETCH FIRST 100 ROWS ONLY", source, metric, after, strategy);
+         if(budgets.isEmpty())
+         {
+            return (winner);
+         }
+         for(Map<String, Object> budget : budgets)
+         {
+            after = WebPerfStore.asLong(budget.get("id"));
+            if(matches(target, budget) && budget.get("threshold") != null && !acceptable(value, (String) budget.get("comparator"), WebPerfStore.asDouble(budget.get("threshold"))))
+            {
+               if(winner == null || "CRITICAL".equals(budget.get("severity")))
+               {
+                  winner = budget;
+               }
+            }
+         }
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Scope values are literal values, never SQL or wildcard expressions.
+    *******************************************************************************/
+   private static boolean matches(Map<String, Object> target, Map<String, Object> budget)
+   {
+      String value = (String) budget.get("scope_value");
+      return (switch(String.valueOf(budget.get("scope")))
+      {
+         case "SITE" -> value == null || value.isBlank() || value.equals(target.get("site_key"));
+         case "TIER" -> Objects.equals(value, target.get("tier"));
+         case "TARGET" -> Objects.equals(value, String.valueOf(target.get("id")));
+         case "CONTENT_TABLE" -> Objects.equals(value, target.get("content_table"));
+         case "PATH_PREFIX" -> value != null && target.get("path") instanceof String path && path.startsWith(value);
+         default -> false;
+      });
+   }
+
+
+
+   /*******************************************************************************
+    ** Supported budget comparators. Invalid records fail visibly on evaluation.
+    *******************************************************************************/
+   static boolean acceptable(double value, String comparator, double threshold)
+   {
+      return (switch(String.valueOf(comparator))
+      {
+         case "<=", "LE" -> value <= threshold;
+         case "<", "LT" -> value < threshold;
+         case ">=", "GE" -> value >= threshold;
+         case ">", "GT" -> value > threshold;
+         case "=", "==", "EQ" -> value == threshold;
+         case "!=", "<>", "NE" -> value != threshold;
+         default -> throw new IllegalArgumentException("Unsupported performance budget comparator: " + comparator);
+      });
+   }
+
+
+
+   /*******************************************************************************
+    ** Design section 6.4: relative and absolute LCP limits must both be met.
+    *******************************************************************************/
+   static boolean drift(String metric, Double baseline, double value)
+   {
+      if(baseline == null)
+      {
+         return (false);
+      }
+      double delta = value - baseline;
+      return (switch(metric)
+      {
+         case "PERFORMANCE_SCORE" -> delta <= -10;
+         case "ACCESSIBILITY_SCORE", "BEST_PRACTICES_SCORE", "SEO_SCORE" -> delta <= -5;
+         case "LCP" -> delta >= 500 && value >= baseline * 1.25;
+         case "CLS" -> delta >= 0.05 - 0.0000001;
+         case "TBT" -> delta >= 200;
+         default -> false;
+      });
+   }
+
+
+
+   /*******************************************************************************
+    ** Newly failing deterministic audits, compared only within the same major.
+    *******************************************************************************/
+   private List<String> newFailures(Map<String, Object> run, String metric) throws SQLException
+   {
+      if(!metric.endsWith("_SCORE") || "PERFORMANCE_SCORE".equals(metric))
+      {
+         return (List.of());
+      }
+      String category = metric.replace("_SCORE", "");
+      List<Map<String, Object>> previous = store.query("SELECT id FROM {lab_run} WHERE target_id = ? AND strategy = ? AND id < ?"
+         + " AND status = 'SUCCEEDED' AND (lighthouse_version = ? OR lighthouse_version LIKE ?) ORDER BY id DESC FETCH FIRST 1 ROWS ONLY",
+         run.get("target_id"), run.get("strategy"), run.get("id"), major(run), major(run) + ".%");
+      if(previous.isEmpty())
+      {
+         return (List.of());
+      }
+      return (store.query("SELECT audit_id FROM {lab_finding} f WHERE run_id = ? AND category = ? AND score < 1"
+         + " AND NOT EXISTS (SELECT 1 FROM {lab_finding} old WHERE old.run_id = ? AND old.audit_id = f.audit_id AND old.score < 1)"
+         + " ORDER BY audit_id FETCH FIRST 19 ROWS ONLY", run.get("id"), category, previous.get(0).get("id"))
+         .stream().map(row -> (String) row.get("audit_id")).toList());
+   }
+
+
+
+   /*******************************************************************************
+    ** An audit-caused regression recovers when a majority no longer fail it.
+    *******************************************************************************/
+   private boolean auditsStillFail(Map<String, Object> open, List<Map<String, Object>> samples) throws SQLException
+   {
+      String audits = (String) open.get("failed_audits");
+      if(audits == null || audits.isBlank())
+      {
+         return (false);
+      }
+      for(String audit : audits.split(","))
+      {
+         int failed = 0;
+         for(Map<String, Object> sample : samples)
+         {
+            if(!store.query("SELECT id FROM {lab_finding} WHERE run_id = ? AND audit_id = ? AND score < 1 FETCH FIRST 1 ROWS ONLY", sample.get("id"), audit).isEmpty())
+            {
+               failed++;
+            }
+         }
+         if(failed >= 2)
+         {
+            return (true);
+         }
+      }
+      return (false);
+   }
+
+
+
+   /*******************************************************************************
+    ** Record the terminal state without discarding evidence.
+    *******************************************************************************/
+   private void close(Map<String, Object> regression, String status, String note, Instant at) throws SQLException
+   {
+      store.update("UPDATE {regression} SET status = ?, resolved_at = ?, resolution_note = ?, modify_date = ? WHERE id = ?",
+         status, at, note, at, regression.get("id"));
+   }
+
+
+
+   /*******************************************************************************
+    ** Record action; rechecks state under the same target lock as evaluation.
+    *******************************************************************************/
+   public boolean finishRegression(long id, String status, String note, Instant at) throws SQLException
+   {
+      if(!List.of("DISMISSED", "RESOLVED").contains(status) || note == null || note.isBlank() || note.length() > 2000)
+      {
+         throw (new IllegalArgumentException("A resolution note of 1 to 2000 characters is required"));
+      }
+      boolean changed = store.transaction(tx ->
+      {
+         List<Map<String, Object>> rows = tx.query("SELECT * FROM {regression} WHERE id = ?", id);
+         if(rows.isEmpty())
+         {
+            return (false);
+         }
+         Map<String, Object> row = rows.get(0);
+         tx.query("SELECT id FROM {target} WHERE id = ? FOR UPDATE", row.get("target_id"));
+         int updated = tx.update("UPDATE {regression} SET status = ?, resolved_at = ?, resolution_note = ?, modify_date = ? WHERE id = ? AND status IN ('SUSPECTED', 'CONFIRMED')",
+            status, at, note, at, id);
+         new WebPerfEvaluationService(tx).updateCount(row.get("target_id"));
+         return (updated == 1);
+      });
+      if(changed)
+      {
+         sendAlert(id);
+      }
+      return (changed);
+   }
+
+
+
+   /*******************************************************************************
+    ** The list count includes suspected and confirmed open regressions.
+    *******************************************************************************/
+   private void updateCount(Object targetId) throws SQLException
+   {
+      store.update("UPDATE {target} SET open_regression_count = (SELECT COUNT(*) FROM {regression} WHERE target_id = ? AND status IN ('SUSPECTED', 'CONFIRMED')) WHERE id = ?",
+         targetId, targetId);
+   }
+
+
+
+   /*******************************************************************************
+    ** Optional core producer API; without an Alerts QBit core uses its log sink.
+    *******************************************************************************/
+   private void sendAlert(long id)
+   {
+      if(!store.getConfig().isOn(WebPerfFeature.ALERTS))
+      {
+         return;
+      }
+      try
+      {
+         Map<String, Object> row = store.query("SELECT * FROM {regression} WHERE id = ?", id).get(0);
+         String key = store.getConfig().getTableNamePrefix() + ":" + row.get("target_id") + ":" + row.get("source") + ":" + row.get("strategy") + ":" + row.get("metric");
+         if("CONFIRMED".equals(row.get("status")))
+         {
+            QAlerts.raise(new AlertSignal(ALERT_TYPE).withDedupKey(key).withSeverity(AlertSeverity.valueOf((String) row.get("severity")))
+               .withTitle("Confirmed web performance regression").withSummary(row.get("strategy") + " " + row.get("metric"))
+               .withRecordRef(AlertRecordRef.of(store.getConfig().applyPrefix(WebPerfSchema.REGRESSION), id)));
+         }
+         else if(List.of("RESOLVED", "DISMISSED").contains(row.get("status")))
+         {
+            QAlerts.resolve(ALERT_TYPE, key, "Performance regression " + row.get("status").toString().toLowerCase(Locale.ROOT));
+         }
+      }
+      catch(SQLException e)
+      {
+         throw (new IllegalStateException("Reading a committed performance regression failed", e));
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Median of bounded, non-null measurement values.
+    *******************************************************************************/
+   static Double median(List<Map<String, Object>> rows, String column)
+   {
+      List<Double> values = rows.stream().map(row -> WebPerfStore.asDouble(row.get(column))).filter(Objects::nonNull).sorted().toList();
+      int size = values.size();
+      return (size == 0 ? null : size % 2 == 1 ? values.get(size / 2) : (values.get(size / 2 - 1) + values.get(size / 2)) / 2);
+   }
+
+
+
+   /*******************************************************************************
+    ** Stored ids make a statistical result reproducible.
+    *******************************************************************************/
+   static String refs(List<Map<String, Object>> rows)
+   {
+      return (String.join(",", rows.stream().map(row -> row.get("id").toString()).toList()));
+   }
+
+
+
+   /*******************************************************************************
+    ** Unknown versions never mix with a known Lighthouse baseline.
+    *******************************************************************************/
+   static String major(Map<String, Object> run)
+   {
+      return (String.valueOf(run.get("lighthouse_version")).split("\\.")[0]);
+   }
+}

src/main/java/com/kingsrook/qbits/webperf/service/WebPerfService.java

@@ -76,6 +76,9 @@ public class WebPerfService
    /** Statuses that still hold a place in the queue (a new request merges into them). */
    public static final List<String> OPEN_STATUSES = List.of("QUEUED", "RUNNING");
 
+   static final int REPORT_BATCH       = 500;
+   static final int REPORT_MAX_BATCHES = 20;
+
    private final WebPerfRuntime    runtime;
    private final WebPerfStore      store;
    private final WebPerfQBitConfig config;
@@ -205,7 +208,6 @@ public class WebPerfService
          values.put("enabled", true);
          values.put("in_sitemap", true);
          values.put("last_seen_in_sitemap_at", now);
-         values.put("open_regression_count", 0);
          store.insert(WebPerfSchema.TARGET, values);
       }
       for(Long id : plan.seenIds())
@@ -303,10 +305,6 @@ public class WebPerfService
    public QueuePlan planQueue(List<RunRequest> requests, Instant now) throws SQLException
    {
       Set<String> open = new HashSet<>();
-      for(Map<String, Object> row : store.query("SELECT target_id, strategy FROM {lab_run} WHERE status IN ('QUEUED', 'RUNNING')"))
-      {
-         open.add(row.get("target_id") + "|" + row.get("strategy"));
-      }
       int runsToday = runsSince(startOfToday(now));
 
       List<RunRequest> queue     = new ArrayList<>();
@@ -315,7 +313,8 @@ public class WebPerfService
       for(RunRequest request : requests)
       {
          String key = request.targetId() + "|" + request.strategy();
-         if(!open.add(key))
+         if(!open.add(key) || !store.query("SELECT id FROM {lab_run} WHERE target_id = ? AND strategy = ? AND status IN ('QUEUED', 'RUNNING') FETCH FIRST 1 ROWS ONLY",
+            request.targetId(), request.strategy()).isEmpty())
          {
             merged.add(request);
          }
@@ -359,6 +358,48 @@ public class WebPerfService
 
 
 
+   /*******************************************************************************
+    ** Queue an independent confirmation or comparison sample. Its reference is
+    ** stable across retries; deliberate samples must not merge with each other.
+    *******************************************************************************/
+   public String enqueueSample(Map<String, Object> target, String strategy, String trigger, String reference, Instant due) throws SQLException
+   {
+      if(!store.query("SELECT id FROM {lab_run} WHERE target_id = ? AND strategy = ? AND trigger_ref = ? AND status NOT IN ('FAILED', 'RUNTIME_ERROR', 'SKIPPED_QUOTA') FETCH FIRST 1 ROWS ONLY",
+         target.get("id"), strategy, reference).isEmpty())
+      {
+         return ("MERGED");
+      }
+      Instant now = Instant.now();
+      boolean capped = runsSince(startOfToday(now)) >= config.getMaxRunsPerDay();
+      RunRequest request = new RunRequest(WebPerfStore.asLong(target.get("id")), (String) target.get("url"), strategy, trigger, reference, null, due);
+      store.insert(WebPerfSchema.LAB_RUN, runValues(request, capped ? "SKIPPED_QUOTA" : "QUEUED", now,
+         capped ? "Daily run cap reached; use Retry to collect this sample" : null));
+      return (capped ? "SKIPPED_QUOTA" : "QUEUED");
+   }
+
+
+
+   /*******************************************************************************
+    ** Retry a deliberate sample without losing its confirmation/comparison slot.
+    ** Keep the failed row as history; simultaneous retries share the target lock.
+    *******************************************************************************/
+   public String retrySample(long runId, Instant now) throws SQLException
+   {
+      return (store.transaction(tx ->
+      {
+         Map<String, Object> run = tx.query("SELECT * FROM {lab_run} WHERE id = ?", runId).get(0);
+         Map<String, Object> target = tx.query("SELECT id, url FROM {target} WHERE id = ? FOR UPDATE", run.get("target_id")).get(0);
+         if(!List.of("FAILED", "RUNTIME_ERROR", "SKIPPED_QUOTA").contains(run.get("status")))
+         {
+            return ("MERGED");
+         }
+         return (new WebPerfService(runtime, tx).enqueueSample(target, (String) run.get("strategy"), (String) run.get("trigger_type"),
+            (String) run.get("trigger_ref"), now));
+      }));
+   }
+
+
+
    /*******************************************************************************
     ** The column values of a new run.
     *******************************************************************************/
@@ -460,6 +501,7 @@ public class WebPerfService
     *******************************************************************************/
    public int runDue(Instant now) throws Exception
    {
+      new WebPerfEvaluationService(store).evaluatePending();
       LabRunner runner = runtime.labRunner();
       if(runner == null)
       {
@@ -467,7 +509,7 @@ public class WebPerfService
          return (0);
       }
 
-      List<Map<String, Object>> due = store.query("SELECT id FROM {lab_run} WHERE status = 'QUEUED' AND not_before <= ? ORDER BY not_before, id", now);
+      List<Map<String, Object>> due = store.query("SELECT id FROM {lab_run} WHERE status = 'QUEUED' AND not_before <= ? ORDER BY not_before, id FETCH FIRST " + config.getConcurrency() + " ROWS ONLY", now);
       List<Long>                claimed = new ArrayList<>();
       for(Map<String, Object> row : due)
       {
@@ -513,6 +555,7 @@ public class WebPerfService
       {
          executor.shutdownNow();
       }
+      new WebPerfEvaluationService(store).evaluatePending();
       return (claimed.size());
    }
 
@@ -529,6 +572,7 @@ public class WebPerfService
    {
       Map<String, Object> run      = store.query("SELECT id, target_id, url, strategy, attempts FROM {lab_run} WHERE id = ?", runId).get(0);
       int                 attempts = WebPerfStore.asInt(run.get("attempts")) + 1;
+      new WebPerfChangeService(store).attribute(runId, Instant.now());
       LabRunner.LabResponse response;
       try
       {
@@ -567,11 +611,66 @@ public class WebPerfService
 
 
 
+   /*******************************************************************************
+    ** Apply reportRetentionDays: delete Lighthouse Reports older than that many
+    ** days (0 keeps them forever). The run, its scores and findings stay; only
+    ** the full compressed response goes. Runs after a result is committed (a
+    ** failure here never loses a run's evidence), at most once an hour per
+    ** process (WebPerfRuntime.reportPruneDue, shared by the workers), in batches
+    ** of REPORT_BATCH rows, at most REPORT_MAX_BATCHES per pass. There is no
+    ** table for state shared across replicas, so each replica keeps its own
+    ** hourly limit; passes are bounded and idempotent, so overlapping replicas
+    ** only repeat one cheap query.
+    *******************************************************************************/
+   void pruneReports()
+   {
+      int days = runtime.getConfig().getReportRetentionDays();
+      if(days <= 0 || !runtime.reportPruneDue())
+      {
+         return;
+      }
+
+      try
+      {
+         java.sql.Timestamp cutoff = WebPerfStore.utc(Instant.now().minus(java.time.Duration.ofDays(days)));
+         for(int batch = 0; batch < REPORT_MAX_BATCHES; batch++)
+         {
+            int deleted = store.update("DELETE FROM {lab_report} WHERE id IN (SELECT id FROM {lab_report} WHERE create_date < ? ORDER BY id FETCH FIRST " + REPORT_BATCH + " ROWS ONLY)", cutoff);
+            if(deleted < REPORT_BATCH)
+            {
+               return;
+            }
+         }
+      }
+      catch(SQLException | RuntimeException e)
+      {
+         LOG.warn("Could not prune old Lighthouse Reports", e);
+      }
+   }
+
+
+
    /*******************************************************************************
     ** Store a successful response: the run's scores and metrics, its findings,
     ** the compressed report, and the page's latest results.
     *******************************************************************************/
    private void store(long runId, Map<String, Object> run, int attempts, LabResultParser.ParsedLab parsed, LabRunner.LabResponse response) throws SQLException
+   {
+      store.transaction(tx ->
+      {
+         new WebPerfService(runtime, tx).storeResult(runId, run, attempts, parsed, response);
+         return (null);
+      });
+      pruneReports();
+   }
+
+
+
+   /*******************************************************************************
+    ** Commit the result, findings, report and latest values as one unit, so an
+    ** evaluator never sees a successful run with only some of its evidence.
+    *******************************************************************************/
+   private void storeResult(long runId, Map<String, Object> run, int attempts, LabResultParser.ParsedLab parsed, LabRunner.LabResponse response) throws SQLException
    {
       Instant completed = response.completedAt();
       String  status    = parsed.runtimeError() == null ? "SUCCEEDED" : "RUNTIME_ERROR";
@@ -585,6 +684,7 @@ public class WebPerfService
          parsed.score("SEO"), parsed.fcpMs(), parsed.lcpMs(), parsed.tbtMs(), parsed.cls(), parsed.speedIndexMs(), parsed.lcpElement(), parsed.runWarnings(),
          completed, runId);
 
+      store.update("UPDATE {lab_run} SET ttfb_ms = ? WHERE id = ?", LabResultParser.ttfb(response.rawJson()), runId);
       Long targetId = WebPerfStore.asLong(run.get("target_id"));
       for(LabResultParser.Finding finding : parsed.findings())
       {
@@ -718,6 +818,7 @@ public class WebPerfService
             }
          }
       }
+      new WebPerfEvaluationService(store).evaluatePending();
       return (counts);
    }
 

src/main/java/com/kingsrook/qbits/webperf/store/WebPerfStore.java

@@ -74,6 +74,86 @@ public class WebPerfStore
 
    private final WebPerfQBitConfig  config;
    private final ConnectionSupplier connections;
+   private Connection transactionConnection;
+
+
+
+   /*******************************************************************************
+    ** Work sharing one database transaction.
+    *******************************************************************************/
+   @FunctionalInterface
+   public interface TransactionWork<T>
+   {
+      /*******************************************************************************
+       ** Execute against the transaction's store.
+       *******************************************************************************/
+      T run(WebPerfStore store) throws SQLException;
+   }
+
+
+
+   /*******************************************************************************
+    ** Commit related writes together; roll back every write on failure.
+    *******************************************************************************/
+   public <T> T transaction(TransactionWork<T> work) throws SQLException
+   {
+      if(transactionConnection != null)
+      {
+         return (work.run(this));
+      }
+      try(Connection connection = connections.get())
+      {
+         boolean autoCommit = connection.getAutoCommit();
+         connection.setAutoCommit(false);
+         WebPerfStore transactional = new WebPerfStore(config, connections);
+         transactional.transactionConnection = connection;
+         try
+         {
+            T result = work.run(transactional);
+            connection.commit();
+            return (result);
+         }
+         catch(SQLException | RuntimeException e)
+         {
+            connection.rollback();
+            throw (e);
+         }
+         finally
+         {
+            connection.setAutoCommit(autoCommit);
+         }
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Own a connection only outside an enclosing transaction.
+    *******************************************************************************/
+   private ConnectionLease lease() throws SQLException
+   {
+      return (new ConnectionLease(transactionConnection == null ? connections.get() : transactionConnection, transactionConnection == null));
+   }
+
+
+
+   /*******************************************************************************
+    ** A statement's connection lifetime.
+    *******************************************************************************/
+   private record ConnectionLease(Connection connection, boolean owned) implements AutoCloseable
+   {
+      /*******************************************************************************
+       ** Leave an enclosing transaction's connection open.
+       *******************************************************************************/
+      @Override
+      public void close() throws SQLException
+      {
+         if(owned)
+         {
+            connection.close();
+         }
+      }
+   }
 
 
 
@@ -119,7 +199,7 @@ public class WebPerfStore
     *******************************************************************************/
    public int update(String template, Object... params) throws SQLException
    {
-      try(Connection connection = connections.get(); PreparedStatement statement = connection.prepareStatement(sql(template)))
+      try(ConnectionLease lease = lease(); PreparedStatement statement = lease.connection().prepareStatement(sql(template)))
       {
          bind(statement, params);
          return (statement.executeUpdate());
@@ -145,8 +225,8 @@ public class WebPerfStore
 
       String columns      = String.join(", ", row.keySet());
       String placeholders = String.join(", ", java.util.Collections.nCopies(row.size(), "?"));
-      try(Connection connection = connections.get();
-         PreparedStatement statement = connection.prepareStatement("INSERT INTO " + config.applyPrefix(table) + " (" + columns + ") VALUES (" + placeholders + ")",
+      try(ConnectionLease lease = lease();
+         PreparedStatement statement = lease.connection().prepareStatement("INSERT INTO " + config.applyPrefix(table) + " (" + columns + ") VALUES (" + placeholders + ")",
             Statement.RETURN_GENERATED_KEYS))
       {
          bind(statement, row.values().toArray());
@@ -171,7 +251,7 @@ public class WebPerfStore
     *******************************************************************************/
    public List<Map<String, Object>> query(String template, Object... params) throws SQLException
    {
-      try(Connection connection = connections.get(); PreparedStatement statement = connection.prepareStatement(sql(template)))
+      try(ConnectionLease lease = lease(); PreparedStatement statement = lease.connection().prepareStatement(sql(template)))
       {
          bind(statement, params);
          List<Map<String, Object>> rows = new ArrayList<>();

src/main/java/com/kingsrook/qbits/webperf/widgets/WebPerfSetupStatus.java

@@ -0,0 +1,183 @@
+/*
+ * Copyright 2026 Kingsrook, LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.webperf.widgets;
+
+
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.List;
+import com.kingsrook.qbits.webperf.WebPerfFeature;
+import com.kingsrook.qbits.webperf.WebPerfQBitConfig;
+import com.kingsrook.qbits.webperf.WebPerfRuntime;
+import com.kingsrook.qbits.webperf.WebPerfSchema;
+import com.kingsrook.qbits.webperf.store.WebPerfStore;
+
+
+/*******************************************************************************
+ ** What is set up on the Performance home and what is next: whether pages have
+ ** been synced, whether budgets exist (and whether they are only the seeded
+ ** defaults), and whether the lab runner and Google field data are configured.
+ ** The checks use the runtime's own resolution (WebPerfRuntime.labRunner() and
+ ** cruxClient(), the url source), so the card never says "needs setup" for
+ ** something the runtime would in fact run.
+ *******************************************************************************/
+public final class WebPerfSetupStatus
+{
+   public static final String SET_UP      = "Set up";
+   public static final String NEEDS_SETUP = "Needs setup";
+   public static final String DEFAULTS    = "Using defaults";
+   public static final String OPTIONAL    = "Optional";
+   public static final String OFF         = "Off";
+
+
+
+   /*******************************************************************************
+    ** One line of the card.
+    **
+    ** @param name     what is being set up
+    ** @param state    one of the state constants
+    ** @param detail   what the state is (counts, and the next step)
+    ** @param tableName the table to open (prefixed name), or null
+    *******************************************************************************/
+   public record Item(String name, String state, String detail, String tableName)
+   {
+   }
+
+
+
+   /*******************************************************************************
+    ** Utility class.
+    *******************************************************************************/
+   private WebPerfSetupStatus()
+   {
+   }
+
+
+
+   /*******************************************************************************
+    ** The card's lines, in setup order.
+    *******************************************************************************/
+   public static List<Item> items(WebPerfRuntime runtime) throws SQLException
+   {
+      WebPerfQBitConfig config = runtime.getConfig();
+      WebPerfStore      store  = runtime.store();
+      List<Item>        items  = new ArrayList<>();
+
+      //////////////////////////////////////////////////////////////////////////
+      // tested pages: synced from the url source (sitemap) or added by hand  //
+      //////////////////////////////////////////////////////////////////////////
+      long pages = count(store, "SELECT COUNT(*) AS n FROM {target}");
+      String target = config.applyPrefix(WebPerfSchema.TARGET);
+      if(pages > 0)
+      {
+         items.add(new Item("Tested pages", SET_UP, pages + " page" + (pages == 1 ? "" : "s") + " are tracked.", target));
+      }
+      else if(config.getUrlSource() == null)
+      {
+         items.add(new Item("Tested pages", NEEDS_SETUP, "No pages yet and no page source is configured. Add a page by hand, or have the host set a URL source (the sitemap) in the qbit config.", target));
+      }
+      else
+      {
+         items.add(new Item("Tested pages", NEEDS_SETUP, "No pages yet. Run “Sync pages from sitemap” on Tested Pages.", target));
+      }
+
+      //////////////////////////////////////////////////////////////////////////
+      // lab runner: the configured one, else PageSpeed Insights with a key   //
+      //////////////////////////////////////////////////////////////////////////
+      if(!config.isOn(WebPerfFeature.LAB))
+      {
+         items.add(new Item("Lab tests (Lighthouse)", OFF, "The LAB feature is turned off in the qbit config.", null));
+      }
+      else if(runtime.labRunner() != null)
+      {
+         items.add(new Item("Lab tests (Lighthouse)", SET_UP, "A lab runner is configured. Use “Run site audit” on Tested Pages, or wait for the schedule.", config.applyPrefix(WebPerfSchema.LAB_RUN)));
+      }
+      else
+      {
+         items.add(new Item("Lab tests (Lighthouse)", NEEDS_SETUP, "No Google API key is configured, so lab tests cannot run. The host sets googleApiKey in the qbit config (read from an environment variable).", null));
+      }
+
+      //////////////////////////////////////////////////////////////////////////
+      // budgets: the seeded reference budgets, or the admin's own            //
+      //////////////////////////////////////////////////////////////////////////
+      long budgets  = count(store, "SELECT COUNT(*) AS n FROM {budget} WHERE enabled = TRUE");
+      long custom   = count(store, "SELECT COUNT(*) AS n FROM {budget} WHERE enabled = TRUE AND (seeded IS NULL OR seeded = FALSE)");
+      String budget = config.applyPrefix(WebPerfSchema.BUDGET);
+      if(budgets == 0)
+      {
+         items.add(new Item("Budgets", NEEDS_SETUP, "No budget is enabled, so nothing is flagged as too slow. Add one, or enable the reference budgets.", budget));
+      }
+      else if(custom == 0)
+      {
+         items.add(new Item("Budgets", DEFAULTS, budgets + " reference budgets (Google's “good” thresholds) are enabled. Edit them or add your own.", budget));
+      }
+      else
+      {
+         items.add(new Item("Budgets", SET_UP, budgets + " budgets are enabled, " + custom + " of them your own.", budget));
+      }
+
+      //////////////////////////////////////////////////////////////////////////
+      // Google field data (CrUX)                                             //
+      //////////////////////////////////////////////////////////////////////////
+      if(!config.isOn(WebPerfFeature.FIELD_CRUX))
+      {
+         items.add(new Item("Google field data", OFF, "The FIELD_CRUX feature is turned off in the qbit config.", null));
+      }
+      else if(runtime.cruxClient() != null)
+      {
+         items.add(new Item("Google field data", SET_UP, "Checked daily; use “Refresh Google field data” to check now.", config.applyPrefix(WebPerfSchema.FIELD_SNAPSHOT)));
+      }
+      else
+      {
+         items.add(new Item("Google field data", NEEDS_SETUP, "No Google API key is configured, so Google field data is not checked.", null));
+      }
+
+      items.add(new Item("Regressions, changes, comparisons", OPTIONAL, "Filled in by the qbit as tests run and deploys are detected. Nothing to set up.", config.applyPrefix(WebPerfSchema.REGRESSION)));
+      return (items);
+   }
+
+
+
+   /*******************************************************************************
+    ** The card's HTML.
+    *******************************************************************************/
+   public static String html(WebPerfRuntime runtime) throws SQLException
+   {
+      List<List<String>> rows = new ArrayList<>();
+      for(Item item : items(runtime))
+      {
+         String color = NEEDS_SETUP.equals(item.state()) ? Html.POOR : SET_UP.equals(item.state()) ? Html.GOOD : Html.MEDIUM;
+         rows.add(List.of(
+            item.tableName() == null ? Html.escape(item.name()) : Html.link("/app/" + item.tableName(), item.name()),
+            "<b style=\"color:" + color + "\">" + Html.escape(item.state()) + "</b>",
+            Html.escape(item.detail())));
+      }
+      return (Html.table(List.of("Part", "State", "What is next"), rows));
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private static long count(WebPerfStore store, String sql) throws SQLException
+   {
+      Long n = WebPerfStore.asLong(store.query(sql).get(0).get("n"));
+      return (n == null ? 0 : n);
+   }
+
+}

src/main/java/com/kingsrook/qbits/webperf/widgets/WebPerfWidgets.java

@@ -58,6 +58,7 @@ import com.kingsrook.qqq.backend.core.model.dashboard.widgets.RawHTML;
  *******************************************************************************/
 public final class WebPerfWidgets
 {
+   public static final String SETUP           = "Setup";
    public static final String SITE_SCORES     = "SiteScores";
    public static final String TREND           = "Trend";
    public static final String SLOWEST_PAGES   = "SlowestPages";
@@ -67,7 +68,7 @@ public final class WebPerfWidgets
    public static final String RECORD_PANEL    = "RecordPanel";
 
    /** The home widgets, in order. */
-   public static final List<String> HOME = List.of(SITE_SCORES, TREND, SLOWEST_PAGES, OPPORTUNITIES, FIELD_STATUS, RECENT_RUNS);
+   public static final List<String> HOME = List.of(SETUP, SITE_SCORES, TREND, SLOWEST_PAGES, OPPORTUNITIES, FIELD_STATUS, RECENT_RUNS);
 
    static final int TREND_WEEKS = 26;
 
@@ -415,6 +416,31 @@ public final class WebPerfWidgets
 
 
 
+   /*******************************************************************************
+    ** Setup: what is configured (pages, lab runner, budgets, field data) and
+    ** the next step for each (WebPerfSetupStatus).
+    *******************************************************************************/
+   public static class Setup extends AbstractWidgetRenderer
+   {
+      /*******************************************************************************
+       ** Render.
+       *******************************************************************************/
+      @Override
+      public RenderWidgetOutput render(RenderWidgetInput input) throws QException
+      {
+         try
+         {
+            return (html(WebPerfSetupStatus.html(runtime(input))));
+         }
+         catch(SQLException e)
+         {
+            throw (new QException("Error loading setup status", e));
+         }
+      }
+   }
+
+
+
    /*******************************************************************************
     ** Field Data Status: what Google's field data (CrUX) says about the site,
     ** honestly NO_DATA when Google has none.

src/main/resources/db/liquibase/qbit-web-performance/web_performance.changelog.sql

@@ -261,3 +261,173 @@ SELECT name, scope, scope_value, source, strategy, metric, comparator, threshold
    UNION ALL SELECT 'Performance at least 50 (KEY pages)', 'TIER', 'KEY', 'LAB', NULL, 'PERFORMANCE_SCORE', '>=', 50, 'CRITICAL'
 ) seeds
 WHERE NOT EXISTS (SELECT 1 FROM ${web_performance.tablePrefix}budget WHERE seeded = TRUE);
+
+
+-- changeset qbit-web-performance:11
+-- comment: Durable evaluation progress, event identity and comparison page identity. Existing history is preserved.
+ALTER TABLE ${web_performance.tablePrefix}lab_run ADD COLUMN evaluated_at TIMESTAMP WITHOUT TIME ZONE;
+ALTER TABLE ${web_performance.tablePrefix}lab_run ADD COLUMN change_event_id BIGINT;
+ALTER TABLE ${web_performance.tablePrefix}regression ADD COLUMN first_run_id BIGINT;
+ALTER TABLE ${web_performance.tablePrefix}regression ADD COLUMN checkpoint_run_id BIGINT;
+ALTER TABLE ${web_performance.tablePrefix}regression ADD COLUMN lighthouse_major VARCHAR(20);
+ALTER TABLE ${web_performance.tablePrefix}change_event ADD COLUMN event_key VARCHAR(64);
+ALTER TABLE ${web_performance.tablePrefix}change_event ADD COLUMN site_key VARCHAR(200);
+ALTER TABLE ${web_performance.tablePrefix}comparison ADD COLUMN target_id BIGINT;
+CREATE UNIQUE INDEX uq_${web_performance.tablePrefix}change_event_key ON ${web_performance.tablePrefix}change_event (event_key);
+CREATE UNIQUE INDEX uq_${web_performance.tablePrefix}comparison_page ON ${web_performance.tablePrefix}comparison (change_event_id, target_id, source, strategy, metric, segment);
+CREATE INDEX idx_${web_performance.tablePrefix}lab_evaluation ON ${web_performance.tablePrefix}lab_run (evaluated_at, status, id);
+ALTER TABLE ${web_performance.tablePrefix}regression ADD COLUMN failed_audits VARCHAR(2000);
+ALTER TABLE ${web_performance.tablePrefix}lab_run ADD COLUMN ttfb_ms INTEGER;
+ALTER TABLE ${web_performance.tablePrefix}field_snapshot ADD COLUMN evaluated_at TIMESTAMP WITHOUT TIME ZONE;
+ALTER TABLE ${web_performance.tablePrefix}change_event ADD COLUMN last_seen_at TIMESTAMP WITHOUT TIME ZONE;
+
+-- changeset qbit-web-performance:12
+-- comment: Referential integrity (foreign keys), part 1: every foreign key column leads an index, so a parent delete checks its children by index. The other foreign key columns already lead one (target_id, run_id, change_event_id on comparison).
+CREATE INDEX IF NOT EXISTS idx_${web_performance.tablePrefix}lab_run_change_event ON ${web_performance.tablePrefix}lab_run (change_event_id);
+CREATE INDEX IF NOT EXISTS idx_${web_performance.tablePrefix}regression_budget ON ${web_performance.tablePrefix}regression (budget_id);
+CREATE INDEX IF NOT EXISTS idx_${web_performance.tablePrefix}regression_cause ON ${web_performance.tablePrefix}regression (cause_change_event_id);
+CREATE INDEX IF NOT EXISTS idx_${web_performance.tablePrefix}regression_first_run ON ${web_performance.tablePrefix}regression (first_run_id);
+CREATE INDEX IF NOT EXISTS idx_${web_performance.tablePrefix}regression_checkpoint_run ON ${web_performance.tablePrefix}regression (checkpoint_run_id);
+CREATE INDEX IF NOT EXISTS idx_${web_performance.tablePrefix}comparison_target ON ${web_performance.tablePrefix}comparison (target_id);
+
+-- Referential integrity (foreign keys), part 2: one changeset per foreign key, named
+-- fk_<child table>_<column>. Each first records and repairs rows whose parent is
+-- missing (a nullable column is set to NULL; a NOT NULL row is copied in full to
+-- qqq_fk_quarantine and deleted, PostgreSQL only), then adds the constraint. Each
+-- is marked ran when its constraint already exists (a host that added it itself).
+-- ON DELETE: owned details CASCADE (findings and the report of a run,
+-- comparisons of a site change); optional pointers SET NULL; tested pages and
+-- budgets RESTRICT, because history is kept (D8): disable a page or budget instead
+-- of deleting it.
+
+-- changeset qbit-web-performance:13
+-- comment: Lab runs keep their tested page (D8, history is kept): a page with runs cannot be deleted, disable it instead (enabled = false). Orphan target_id values are recorded in qqq_fk_quarantine and set to NULL.
+-- preconditions onFail:MARK_RAN onError:HALT
+-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM information_schema.table_constraints WHERE LOWER(constraint_name) = 'fk_${web_performance.tablePrefix}lab_run_target_id'
+CREATE TABLE IF NOT EXISTS qqq_fk_quarantine (id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, table_name VARCHAR(100) NOT NULL, column_name VARCHAR(100) NOT NULL, row_id VARCHAR(100), missing_parent_id VARCHAR(100), action VARCHAR(20) NOT NULL, row_data TEXT, changeset VARCHAR(200), quarantined_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP);
+INSERT INTO qqq_fk_quarantine (table_name, column_name, row_id, missing_parent_id, action, changeset) SELECT '${web_performance.tablePrefix}lab_run', 'target_id', CAST(c.id AS VARCHAR(100)), CAST(c.target_id AS VARCHAR(100)), 'NULLED', 'qbit-web-performance:13' FROM ${web_performance.tablePrefix}lab_run c WHERE c.target_id IS NOT NULL AND NOT EXISTS (SELECT 1 FROM ${web_performance.tablePrefix}target p WHERE p.id = c.target_id);
+UPDATE ${web_performance.tablePrefix}lab_run c SET target_id = NULL WHERE c.target_id IS NOT NULL AND NOT EXISTS (SELECT 1 FROM ${web_performance.tablePrefix}target p WHERE p.id = c.target_id);
+ALTER TABLE ${web_performance.tablePrefix}lab_run ADD CONSTRAINT fk_${web_performance.tablePrefix}lab_run_target_id FOREIGN KEY (target_id) REFERENCES ${web_performance.tablePrefix}target (id) ON DELETE RESTRICT;
+-- rollback ALTER TABLE ${web_performance.tablePrefix}lab_run DROP CONSTRAINT IF EXISTS fk_${web_performance.tablePrefix}lab_run_target_id;
+
+-- changeset qbit-web-performance:14 dbms:postgresql
+-- comment: Orphan cleanup before fk_${web_performance.tablePrefix}lab_finding_run_id: ${web_performance.tablePrefix}lab_finding rows whose run_id names a missing ${web_performance.tablePrefix}lab_run row are copied in full to qqq_fk_quarantine, then deleted (run_id is NOT NULL).
+-- preconditions onFail:MARK_RAN onError:HALT
+-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM information_schema.table_constraints WHERE LOWER(constraint_name) = 'fk_${web_performance.tablePrefix}lab_finding_run_id'
+CREATE TABLE IF NOT EXISTS qqq_fk_quarantine (id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, table_name VARCHAR(100) NOT NULL, column_name VARCHAR(100) NOT NULL, row_id VARCHAR(100), missing_parent_id VARCHAR(100), action VARCHAR(20) NOT NULL, row_data TEXT, changeset VARCHAR(200), quarantined_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP);
+INSERT INTO qqq_fk_quarantine (table_name, column_name, row_id, missing_parent_id, action, row_data, changeset) SELECT '${web_performance.tablePrefix}lab_finding', 'run_id', CAST(c.id AS VARCHAR(100)), CAST(c.run_id AS VARCHAR(100)), 'DELETED', row_to_json(c)::text, 'qbit-web-performance:14' FROM ${web_performance.tablePrefix}lab_finding c WHERE c.run_id IS NOT NULL AND NOT EXISTS (SELECT 1 FROM ${web_performance.tablePrefix}lab_run p WHERE p.id = c.run_id);
+DELETE FROM ${web_performance.tablePrefix}lab_finding c WHERE c.run_id IS NOT NULL AND NOT EXISTS (SELECT 1 FROM ${web_performance.tablePrefix}lab_run p WHERE p.id = c.run_id);
+-- rollback SELECT 1;
+
+-- changeset qbit-web-performance:15
+-- comment: Findings belong to their run: deleting a run deletes its findings.
+-- preconditions onFail:MARK_RAN onError:HALT
+-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM information_schema.table_constraints WHERE LOWER(constraint_name) = 'fk_${web_performance.tablePrefix}lab_finding_run_id'
+ALTER TABLE ${web_performance.tablePrefix}lab_finding ADD CONSTRAINT fk_${web_performance.tablePrefix}lab_finding_run_id FOREIGN KEY (run_id) REFERENCES ${web_performance.tablePrefix}lab_run (id) ON DELETE CASCADE;
+-- rollback ALTER TABLE ${web_performance.tablePrefix}lab_finding DROP CONSTRAINT IF EXISTS fk_${web_performance.tablePrefix}lab_finding_run_id;
+
+-- changeset qbit-web-performance:16
+-- comment: A finding's page (copied from its run) is kept like the run's. Orphan target_id values are recorded in qqq_fk_quarantine and set to NULL.
+-- preconditions onFail:MARK_RAN onError:HALT
+-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM information_schema.table_constraints WHERE LOWER(constraint_name) = 'fk_${web_performance.tablePrefix}lab_finding_target_id'
+CREATE TABLE IF NOT EXISTS qqq_fk_quarantine (id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, table_name VARCHAR(100) NOT NULL, column_name VARCHAR(100) NOT NULL, row_id VARCHAR(100), missing_parent_id VARCHAR(100), action VARCHAR(20) NOT NULL, row_data TEXT, changeset VARCHAR(200), quarantined_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP);
+INSERT INTO qqq_fk_quarantine (table_name, column_name, row_id, missing_parent_id, action, changeset) SELECT '${web_performance.tablePrefix}lab_finding', 'target_id', CAST(c.id AS VARCHAR(100)), CAST(c.target_id AS VARCHAR(100)), 'NULLED', 'qbit-web-performance:16' FROM ${web_performance.tablePrefix}lab_finding c WHERE c.target_id IS NOT NULL AND NOT EXISTS (SELECT 1 FROM ${web_performance.tablePrefix}target p WHERE p.id = c.target_id);
+UPDATE ${web_performance.tablePrefix}lab_finding c SET target_id = NULL WHERE c.target_id IS NOT NULL AND NOT EXISTS (SELECT 1 FROM ${web_performance.tablePrefix}target p WHERE p.id = c.target_id);
+ALTER TABLE ${web_performance.tablePrefix}lab_finding ADD CONSTRAINT fk_${web_performance.tablePrefix}lab_finding_target_id FOREIGN KEY (target_id) REFERENCES ${web_performance.tablePrefix}target (id) ON DELETE RESTRICT;
+-- rollback ALTER TABLE ${web_performance.tablePrefix}lab_finding DROP CONSTRAINT IF EXISTS fk_${web_performance.tablePrefix}lab_finding_target_id;
+
+-- changeset qbit-web-performance:17 dbms:postgresql
+-- comment: Orphan cleanup before fk_${web_performance.tablePrefix}lab_report_run_id: ${web_performance.tablePrefix}lab_report rows whose run_id names a missing ${web_performance.tablePrefix}lab_run row are copied in full to qqq_fk_quarantine, then deleted (run_id is NOT NULL).
+-- preconditions onFail:MARK_RAN onError:HALT
+-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM information_schema.table_constraints WHERE LOWER(constraint_name) = 'fk_${web_performance.tablePrefix}lab_report_run_id'
+CREATE TABLE IF NOT EXISTS qqq_fk_quarantine (id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, table_name VARCHAR(100) NOT NULL, column_name VARCHAR(100) NOT NULL, row_id VARCHAR(100), missing_parent_id VARCHAR(100), action VARCHAR(20) NOT NULL, row_data TEXT, changeset VARCHAR(200), quarantined_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP);
+INSERT INTO qqq_fk_quarantine (table_name, column_name, row_id, missing_parent_id, action, row_data, changeset) SELECT '${web_performance.tablePrefix}lab_report', 'run_id', CAST(c.id AS VARCHAR(100)), CAST(c.run_id AS VARCHAR(100)), 'DELETED', row_to_json(c)::text, 'qbit-web-performance:17' FROM ${web_performance.tablePrefix}lab_report c WHERE c.run_id IS NOT NULL AND NOT EXISTS (SELECT 1 FROM ${web_performance.tablePrefix}lab_run p WHERE p.id = c.run_id);
+DELETE FROM ${web_performance.tablePrefix}lab_report c WHERE c.run_id IS NOT NULL AND NOT EXISTS (SELECT 1 FROM ${web_performance.tablePrefix}lab_run p WHERE p.id = c.run_id);
+-- rollback SELECT 1;
+
+-- changeset qbit-web-performance:18
+-- comment: The Lighthouse report belongs to its run: deleting a run deletes its report.
+-- preconditions onFail:MARK_RAN onError:HALT
+-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM information_schema.table_constraints WHERE LOWER(constraint_name) = 'fk_${web_performance.tablePrefix}lab_report_run_id'
+ALTER TABLE ${web_performance.tablePrefix}lab_report ADD CONSTRAINT fk_${web_performance.tablePrefix}lab_report_run_id FOREIGN KEY (run_id) REFERENCES ${web_performance.tablePrefix}lab_run (id) ON DELETE CASCADE;
+-- rollback ALTER TABLE ${web_performance.tablePrefix}lab_report DROP CONSTRAINT IF EXISTS fk_${web_performance.tablePrefix}lab_report_run_id;
+
+-- changeset qbit-web-performance:19
+-- comment: Regressions keep their page: a page with regressions cannot be deleted, disable it instead. Orphan target_id values are recorded in qqq_fk_quarantine and set to NULL.
+-- preconditions onFail:MARK_RAN onError:HALT
+-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM information_schema.table_constraints WHERE LOWER(constraint_name) = 'fk_${web_performance.tablePrefix}regression_target_id'
+CREATE TABLE IF NOT EXISTS qqq_fk_quarantine (id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, table_name VARCHAR(100) NOT NULL, column_name VARCHAR(100) NOT NULL, row_id VARCHAR(100), missing_parent_id VARCHAR(100), action VARCHAR(20) NOT NULL, row_data TEXT, changeset VARCHAR(200), quarantined_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP);
+INSERT INTO qqq_fk_quarantine (table_name, column_name, row_id, missing_parent_id, action, changeset) SELECT '${web_performance.tablePrefix}regression', 'target_id', CAST(c.id AS VARCHAR(100)), CAST(c.target_id AS VARCHAR(100)), 'NULLED', 'qbit-web-performance:19' FROM ${web_performance.tablePrefix}regression c WHERE c.target_id IS NOT NULL AND NOT EXISTS (SELECT 1 FROM ${web_performance.tablePrefix}target p WHERE p.id = c.target_id);
+UPDATE ${web_performance.tablePrefix}regression c SET target_id = NULL WHERE c.target_id IS NOT NULL AND NOT EXISTS (SELECT 1 FROM ${web_performance.tablePrefix}target p WHERE p.id = c.target_id);
+ALTER TABLE ${web_performance.tablePrefix}regression ADD CONSTRAINT fk_${web_performance.tablePrefix}regression_target_id FOREIGN KEY (target_id) REFERENCES ${web_performance.tablePrefix}target (id) ON DELETE RESTRICT;
+-- rollback ALTER TABLE ${web_performance.tablePrefix}regression DROP CONSTRAINT IF EXISTS fk_${web_performance.tablePrefix}regression_target_id;
+
+-- changeset qbit-web-performance:20
+-- comment: A regression keeps the budget it breached: a budget with regressions cannot be deleted, disable it instead (enabled = false). Orphan budget_id values are recorded in qqq_fk_quarantine and set to NULL.
+-- preconditions onFail:MARK_RAN onError:HALT
+-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM information_schema.table_constraints WHERE LOWER(constraint_name) = 'fk_${web_performance.tablePrefix}regression_budget_id'
+CREATE TABLE IF NOT EXISTS qqq_fk_quarantine (id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, table_name VARCHAR(100) NOT NULL, column_name VARCHAR(100) NOT NULL, row_id VARCHAR(100), missing_parent_id VARCHAR(100), action VARCHAR(20) NOT NULL, row_data TEXT, changeset VARCHAR(200), quarantined_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP);
+INSERT INTO qqq_fk_quarantine (table_name, column_name, row_id, missing_parent_id, action, changeset) SELECT '${web_performance.tablePrefix}regression', 'budget_id', CAST(c.id AS VARCHAR(100)), CAST(c.budget_id AS VARCHAR(100)), 'NULLED', 'qbit-web-performance:20' FROM ${web_performance.tablePrefix}regression c WHERE c.budget_id IS NOT NULL AND NOT EXISTS (SELECT 1 FROM ${web_performance.tablePrefix}budget p WHERE p.id = c.budget_id);
+UPDATE ${web_performance.tablePrefix}regression c SET budget_id = NULL WHERE c.budget_id IS NOT NULL AND NOT EXISTS (SELECT 1 FROM ${web_performance.tablePrefix}budget p WHERE p.id = c.budget_id);
+ALTER TABLE ${web_performance.tablePrefix}regression ADD CONSTRAINT fk_${web_performance.tablePrefix}regression_budget_id FOREIGN KEY (budget_id) REFERENCES ${web_performance.tablePrefix}budget (id) ON DELETE RESTRICT;
+-- rollback ALTER TABLE ${web_performance.tablePrefix}regression DROP CONSTRAINT IF EXISTS fk_${web_performance.tablePrefix}regression_budget_id;
+
+-- changeset qbit-web-performance:21
+-- comment: The likely cause is an optional pointer: deleting the site change clears it. Orphan values are recorded in qqq_fk_quarantine and set to NULL.
+-- preconditions onFail:MARK_RAN onError:HALT
+-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM information_schema.table_constraints WHERE LOWER(constraint_name) = 'fk_${web_performance.tablePrefix}regression_cause_change_event_id'
+CREATE TABLE IF NOT EXISTS qqq_fk_quarantine (id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, table_name VARCHAR(100) NOT NULL, column_name VARCHAR(100) NOT NULL, row_id VARCHAR(100), missing_parent_id VARCHAR(100), action VARCHAR(20) NOT NULL, row_data TEXT, changeset VARCHAR(200), quarantined_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP);
+INSERT INTO qqq_fk_quarantine (table_name, column_name, row_id, missing_parent_id, action, changeset) SELECT '${web_performance.tablePrefix}regression', 'cause_change_event_id', CAST(c.id AS VARCHAR(100)), CAST(c.cause_change_event_id AS VARCHAR(100)), 'NULLED', 'qbit-web-performance:21' FROM ${web_performance.tablePrefix}regression c WHERE c.cause_change_event_id IS NOT NULL AND NOT EXISTS (SELECT 1 FROM ${web_performance.tablePrefix}change_event p WHERE p.id = c.cause_change_event_id);
+UPDATE ${web_performance.tablePrefix}regression c SET cause_change_event_id = NULL WHERE c.cause_change_event_id IS NOT NULL AND NOT EXISTS (SELECT 1 FROM ${web_performance.tablePrefix}change_event p WHERE p.id = c.cause_change_event_id);
+ALTER TABLE ${web_performance.tablePrefix}regression ADD CONSTRAINT fk_${web_performance.tablePrefix}regression_cause_change_event_id FOREIGN KEY (cause_change_event_id) REFERENCES ${web_performance.tablePrefix}change_event (id) ON DELETE SET NULL;
+-- rollback ALTER TABLE ${web_performance.tablePrefix}regression DROP CONSTRAINT IF EXISTS fk_${web_performance.tablePrefix}regression_cause_change_event_id;
+
+-- changeset qbit-web-performance:22
+-- comment: A before-and-after comparison is computed for its site change: deleting the change deletes its comparisons. Orphan values are recorded in qqq_fk_quarantine and set to NULL.
+-- preconditions onFail:MARK_RAN onError:HALT
+-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM information_schema.table_constraints WHERE LOWER(constraint_name) = 'fk_${web_performance.tablePrefix}comparison_change_event_id'
+CREATE TABLE IF NOT EXISTS qqq_fk_quarantine (id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, table_name VARCHAR(100) NOT NULL, column_name VARCHAR(100) NOT NULL, row_id VARCHAR(100), missing_parent_id VARCHAR(100), action VARCHAR(20) NOT NULL, row_data TEXT, changeset VARCHAR(200), quarantined_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP);
+INSERT INTO qqq_fk_quarantine (table_name, column_name, row_id, missing_parent_id, action, changeset) SELECT '${web_performance.tablePrefix}comparison', 'change_event_id', CAST(c.id AS VARCHAR(100)), CAST(c.change_event_id AS VARCHAR(100)), 'NULLED', 'qbit-web-performance:22' FROM ${web_performance.tablePrefix}comparison c WHERE c.change_event_id IS NOT NULL AND NOT EXISTS (SELECT 1 FROM ${web_performance.tablePrefix}change_event p WHERE p.id = c.change_event_id);
+UPDATE ${web_performance.tablePrefix}comparison c SET change_event_id = NULL WHERE c.change_event_id IS NOT NULL AND NOT EXISTS (SELECT 1 FROM ${web_performance.tablePrefix}change_event p WHERE p.id = c.change_event_id);
+ALTER TABLE ${web_performance.tablePrefix}comparison ADD CONSTRAINT fk_${web_performance.tablePrefix}comparison_change_event_id FOREIGN KEY (change_event_id) REFERENCES ${web_performance.tablePrefix}change_event (id) ON DELETE CASCADE;
+-- rollback ALTER TABLE ${web_performance.tablePrefix}comparison DROP CONSTRAINT IF EXISTS fk_${web_performance.tablePrefix}comparison_change_event_id;
+
+-- changeset qbit-web-performance:23
+-- comment: The site change a run was attributed to is an optional pointer: deleting the change clears it. Orphan values are recorded in qqq_fk_quarantine and set to NULL.
+-- preconditions onFail:MARK_RAN onError:HALT
+-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM information_schema.table_constraints WHERE LOWER(constraint_name) = 'fk_${web_performance.tablePrefix}lab_run_change_event_id'
+CREATE TABLE IF NOT EXISTS qqq_fk_quarantine (id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, table_name VARCHAR(100) NOT NULL, column_name VARCHAR(100) NOT NULL, row_id VARCHAR(100), missing_parent_id VARCHAR(100), action VARCHAR(20) NOT NULL, row_data TEXT, changeset VARCHAR(200), quarantined_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP);
+INSERT INTO qqq_fk_quarantine (table_name, column_name, row_id, missing_parent_id, action, changeset) SELECT '${web_performance.tablePrefix}lab_run', 'change_event_id', CAST(c.id AS VARCHAR(100)), CAST(c.change_event_id AS VARCHAR(100)), 'NULLED', 'qbit-web-performance:23' FROM ${web_performance.tablePrefix}lab_run c WHERE c.change_event_id IS NOT NULL AND NOT EXISTS (SELECT 1 FROM ${web_performance.tablePrefix}change_event p WHERE p.id = c.change_event_id);
+UPDATE ${web_performance.tablePrefix}lab_run c SET change_event_id = NULL WHERE c.change_event_id IS NOT NULL AND NOT EXISTS (SELECT 1 FROM ${web_performance.tablePrefix}change_event p WHERE p.id = c.change_event_id);
+ALTER TABLE ${web_performance.tablePrefix}lab_run ADD CONSTRAINT fk_${web_performance.tablePrefix}lab_run_change_event_id FOREIGN KEY (change_event_id) REFERENCES ${web_performance.tablePrefix}change_event (id) ON DELETE SET NULL;
+-- rollback ALTER TABLE ${web_performance.tablePrefix}lab_run DROP CONSTRAINT IF EXISTS fk_${web_performance.tablePrefix}lab_run_change_event_id;
+
+-- changeset qbit-web-performance:24
+-- comment: Evidence pointer to the run that first showed the regression: deleting the run clears it. Orphan values are recorded in qqq_fk_quarantine and set to NULL.
+-- preconditions onFail:MARK_RAN onError:HALT
+-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM information_schema.table_constraints WHERE LOWER(constraint_name) = 'fk_${web_performance.tablePrefix}regression_first_run_id'
+CREATE TABLE IF NOT EXISTS qqq_fk_quarantine (id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, table_name VARCHAR(100) NOT NULL, column_name VARCHAR(100) NOT NULL, row_id VARCHAR(100), missing_parent_id VARCHAR(100), action VARCHAR(20) NOT NULL, row_data TEXT, changeset VARCHAR(200), quarantined_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP);
+INSERT INTO qqq_fk_quarantine (table_name, column_name, row_id, missing_parent_id, action, changeset) SELECT '${web_performance.tablePrefix}regression', 'first_run_id', CAST(c.id AS VARCHAR(100)), CAST(c.first_run_id AS VARCHAR(100)), 'NULLED', 'qbit-web-performance:24' FROM ${web_performance.tablePrefix}regression c WHERE c.first_run_id IS NOT NULL AND NOT EXISTS (SELECT 1 FROM ${web_performance.tablePrefix}lab_run p WHERE p.id = c.first_run_id);
+UPDATE ${web_performance.tablePrefix}regression c SET first_run_id = NULL WHERE c.first_run_id IS NOT NULL AND NOT EXISTS (SELECT 1 FROM ${web_performance.tablePrefix}lab_run p WHERE p.id = c.first_run_id);
+ALTER TABLE ${web_performance.tablePrefix}regression ADD CONSTRAINT fk_${web_performance.tablePrefix}regression_first_run_id FOREIGN KEY (first_run_id) REFERENCES ${web_performance.tablePrefix}lab_run (id) ON DELETE SET NULL;
+-- rollback ALTER TABLE ${web_performance.tablePrefix}regression DROP CONSTRAINT IF EXISTS fk_${web_performance.tablePrefix}regression_first_run_id;
+
+-- changeset qbit-web-performance:25
+-- comment: Evidence pointer to the confirmation run: deleting the run clears it. Orphan values are recorded in qqq_fk_quarantine and set to NULL.
+-- preconditions onFail:MARK_RAN onError:HALT
+-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM information_schema.table_constraints WHERE LOWER(constraint_name) = 'fk_${web_performance.tablePrefix}regression_checkpoint_run_id'
+CREATE TABLE IF NOT EXISTS qqq_fk_quarantine (id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, table_name VARCHAR(100) NOT NULL, column_name VARCHAR(100) NOT NULL, row_id VARCHAR(100), missing_parent_id VARCHAR(100), action VARCHAR(20) NOT NULL, row_data TEXT, changeset VARCHAR(200), quarantined_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP);
+INSERT INTO qqq_fk_quarantine (table_name, column_name, row_id, missing_parent_id, action, changeset) SELECT '${web_performance.tablePrefix}regression', 'checkpoint_run_id', CAST(c.id AS VARCHAR(100)), CAST(c.checkpoint_run_id AS VARCHAR(100)), 'NULLED', 'qbit-web-performance:25' FROM ${web_performance.tablePrefix}regression c WHERE c.checkpoint_run_id IS NOT NULL AND NOT EXISTS (SELECT 1 FROM ${web_performance.tablePrefix}lab_run p WHERE p.id = c.checkpoint_run_id);
+UPDATE ${web_performance.tablePrefix}regression c SET checkpoint_run_id = NULL WHERE c.checkpoint_run_id IS NOT NULL AND NOT EXISTS (SELECT 1 FROM ${web_performance.tablePrefix}lab_run p WHERE p.id = c.checkpoint_run_id);
+ALTER TABLE ${web_performance.tablePrefix}regression ADD CONSTRAINT fk_${web_performance.tablePrefix}regression_checkpoint_run_id FOREIGN KEY (checkpoint_run_id) REFERENCES ${web_performance.tablePrefix}lab_run (id) ON DELETE SET NULL;
+-- rollback ALTER TABLE ${web_performance.tablePrefix}regression DROP CONSTRAINT IF EXISTS fk_${web_performance.tablePrefix}regression_checkpoint_run_id;
+
+-- changeset qbit-web-performance:26
+-- comment: A per-page comparison keeps its page: a page with comparisons cannot be deleted, disable it instead. Orphan target_id values are recorded in qqq_fk_quarantine and set to NULL.
+-- preconditions onFail:MARK_RAN onError:HALT
+-- precondition-sql-check expectedResult:0 SELECT COUNT(*) FROM information_schema.table_constraints WHERE LOWER(constraint_name) = 'fk_${web_performance.tablePrefix}comparison_target_id'
+CREATE TABLE IF NOT EXISTS qqq_fk_quarantine (id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, table_name VARCHAR(100) NOT NULL, column_name VARCHAR(100) NOT NULL, row_id VARCHAR(100), missing_parent_id VARCHAR(100), action VARCHAR(20) NOT NULL, row_data TEXT, changeset VARCHAR(200), quarantined_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP);
+INSERT INTO qqq_fk_quarantine (table_name, column_name, row_id, missing_parent_id, action, changeset) SELECT '${web_performance.tablePrefix}comparison', 'target_id', CAST(c.id AS VARCHAR(100)), CAST(c.target_id AS VARCHAR(100)), 'NULLED', 'qbit-web-performance:26' FROM ${web_performance.tablePrefix}comparison c WHERE c.target_id IS NOT NULL AND NOT EXISTS (SELECT 1 FROM ${web_performance.tablePrefix}target p WHERE p.id = c.target_id);
+UPDATE ${web_performance.tablePrefix}comparison c SET target_id = NULL WHERE c.target_id IS NOT NULL AND NOT EXISTS (SELECT 1 FROM ${web_performance.tablePrefix}target p WHERE p.id = c.target_id);
+ALTER TABLE ${web_performance.tablePrefix}comparison ADD CONSTRAINT fk_${web_performance.tablePrefix}comparison_target_id FOREIGN KEY (target_id) REFERENCES ${web_performance.tablePrefix}target (id) ON DELETE RESTRICT;
+-- rollback ALTER TABLE ${web_performance.tablePrefix}comparison DROP CONSTRAINT IF EXISTS fk_${web_performance.tablePrefix}comparison_target_id;

src/main/resources/help/web-performance.yaml

@@ -0,0 +1,58 @@
+bundle: web-performance
+version: 1
+layer: QBIT
+# Help for the Performance app and every web_performance_* table, in the QQQ
+# help-bundle format (qqq#1198). Each table entry says what the table is for and
+# how its rows get created, so an empty table explains itself. WebPerfHelp
+# applies it until the core HelpBundles registry ships in a QQQ release.
+# Summaries stay under 120 characters (WebPerfTablesGuardrailTest).
+entries:
+  - key: app:webPerformance;slot:help
+    summary: "Page speed for your site: Lighthouse lab tests, Google field data, budgets and regressions."
+    content: |
+      The Setup card on the home lists what is configured and the next step: sync the pages, give the qbit a Google API key, and review the budgets. A new installation starts with reference budgets (Google's good thresholds) already in place.
+
+  - key: table:web_performance_target;slot:help
+    summary: "The pages that get tested: one row per URL, with its tier, latest scores and open regressions."
+    content: |
+      **How rows get created:** run Sync pages from sitemap (also scheduled), which adds every page the host's URL source returns and links it to its content record. You can also add a page by hand with New. Mark a page KEY to put it in the site scores and the frequent tests. Run site audit or Test page speed queues lab tests.
+
+  - key: table:web_performance_lab_run;slot:help
+    summary: "Every Lighthouse test of a page: queued, running, finished or failed, with its scores and timings."
+    content: |
+      **How rows get created:** by the qbit. Test page speed, Run site audit and the schedules queue runs here, and the worker runs them through the lab runner (PageSpeed Insights when a Google API key is set). Nothing is added by hand. Without a key, runs wait in the queue. Retry re-queues a failed run.
+
+  - key: table:web_performance_lab_finding;slot:help
+    summary: "What Lighthouse says to fix on each tested page: the audits that failed and how much they would save."
+    content: |
+      **How rows get created:** by the qbit, from each finished lab run. One row per failed or opportunity audit. It stays empty until a lab test succeeds.
+
+  - key: table:web_performance_lab_report;slot:help
+    summary: "The complete Lighthouse response of each run, compressed, for download and later comparison."
+    content: |
+      **How rows get created:** by the qbit, one per successful lab run. Nothing to set up. Reports are kept forever unless the host sets reportRetentionDays, which deletes older reports (the run and its scores stay).
+
+  - key: table:web_performance_field_snapshot;slot:help
+    summary: "What Google's field data (CrUX) says real Chrome users experience: p75 Core Web Vitals per URL or origin."
+    content: |
+      **How rows get created:** by the qbit. Refresh Google field data (and the daily schedule) asks Google for the site and each tested page. It needs the Google API key. A row says NO_DATA when Google has too little traffic to publish figures for that page.
+
+  - key: table:web_performance_budget;slot:help
+    summary: "Limits a metric must stay within, per site, tier, page, content table or path; a breach raises a regression."
+    content: |
+      **How rows get created:** the qbit seeds reference budgets (Google's good thresholds for each Core Web Vital, and Lighthouse category scores for KEY pages), so a new installation already flags slow pages. Edit them, turn them off, or add your own with New. Lab and Google field data (CRUX) budgets are evaluated; budgets with source RUM (real users) are not evaluated yet, because no real-user source is wired in.
+
+  - key: table:web_performance_regression;slot:help
+    summary: "A page that got slower or broke a budget: suspected, confirmed, then resolved or dismissed."
+    content: |
+      **How rows get created:** by the qbit. When a test breaks a budget or drifts from its baseline, it opens a suspected regression and confirms it after more runs. Mark resolved and Dismiss close one by hand. Empty means nothing is wrong.
+
+  - key: table:web_performance_change_event;slot:help
+    summary: "Site changes (a deploy, a Lighthouse version, a CDN stage change) that trigger re-tests."
+    content: |
+      **How rows get created:** by the qbit. Check site changes (scheduled when deploy probes or change sources are configured) records a change, and the qbit notes when Lighthouse itself changes version. It stays empty until the host configures a deploy probe or change event source.
+
+  - key: table:web_performance_comparison;slot:help
+    summary: "Before and after results around a site change, per page and metric."
+    content: |
+      **How rows get created:** run Compare before and after on a change event (it needs the CDN_COMPARE feature and a change event). Nothing to add by hand.

src/test/java/com/kingsrook/qbits/webperf/WebPerfDefaultsChangelogTest.java

@@ -0,0 +1,139 @@
+/*
+ * Copyright 2026 Kingsrook, LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.webperf;
+
+
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.Statement;
+import liquibase.Liquibase;
+import liquibase.database.Database;
+import liquibase.database.DatabaseFactory;
+import liquibase.database.jvm.JdbcConnection;
+import liquibase.resource.ClassLoaderResourceAccessor;
+import org.junit.jupiter.api.Test;
+import static org.assertj.core.api.Assertions.assertThat;
+
+
+/*******************************************************************************
+ ** The shipped changelog, run through real Liquibase on H2 (PostgreSQL mode):
+ ** it creates the nine tables, changeset 10 seeds the reference budgets, a
+ ** second update runs nothing, and the seed statement run again inserts nothing
+ ** and keeps an administrator's edit.
+ *******************************************************************************/
+class WebPerfDefaultsChangelogTest
+{
+   private static final String PREFIX = "web_performance_";
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @Test
+   void seededBudgetsThroughRealLiquibase_idempotent_andKeepEdits() throws Exception
+   {
+      try(Connection connection = connect())
+      {
+         update();
+         assertThat(count(connection, "SELECT COUNT(*) FROM " + PREFIX + "budget")).isEqualTo(13);
+         assertThat(count(connection, "SELECT COUNT(*) FROM " + PREFIX + "budget WHERE seeded = TRUE AND enabled = TRUE")).isEqualTo(13);
+         for(String base : WebPerfSchema.TABLES)
+         {
+            assertThat(count(connection, "SELECT COUNT(*) FROM " + PREFIX + base)).as(base + " is queryable").isGreaterThanOrEqualTo(0);
+         }
+         int changesets = count(connection, "SELECT COUNT(*) FROM DATABASECHANGELOG");
+
+         ////////////////////////////////////////////////////////////////
+         // a second update has nothing to run                          //
+         ////////////////////////////////////////////////////////////////
+         update();
+         assertThat(count(connection, "SELECT COUNT(*) FROM DATABASECHANGELOG")).isEqualTo(changesets);
+         assertThat(count(connection, "SELECT COUNT(*) FROM " + PREFIX + "budget")).isEqualTo(13);
+
+         ////////////////////////////////////////////////////////////////////////////
+         // an admin edits a seed; the seed statement (changeset 10) is run again: //
+         // nothing is inserted and the edit survives                              //
+         ////////////////////////////////////////////////////////////////////////////
+         try(Statement statement = connection.createStatement())
+         {
+            statement.execute("UPDATE " + PREFIX + "budget SET threshold = 3000 WHERE name = 'LCP good (lab, mobile)'");
+            statement.execute(seedStatement());
+         }
+         assertThat(count(connection, "SELECT COUNT(*) FROM " + PREFIX + "budget")).isEqualTo(13);
+         assertThat(count(connection, "SELECT COUNT(*) FROM " + PREFIX + "budget WHERE name = 'LCP good (lab, mobile)' AND threshold = 3000")).isEqualTo(1);
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** The INSERT of changeset 10, with the prefix filled in.
+    *******************************************************************************/
+   private static String seedStatement() throws Exception
+   {
+      String script;
+      try(java.io.InputStream in = WebPerfDefaultsChangelogTest.class.getClassLoader().getResourceAsStream(WebPerfSchema.CHANGELOG))
+      {
+         script = new String(in.readAllBytes(), java.nio.charset.StandardCharsets.UTF_8);
+      }
+      String seed = script.substring(script.indexOf("-- changeset qbit-web-performance:10"), script.indexOf("-- changeset qbit-web-performance:11"));
+      seed = seed.substring(seed.indexOf("INSERT INTO")).trim().replaceAll(";$", "");
+      return (seed.replace("${" + WebPerfSchema.CHANGELOG_PREFIX_PROPERTY + "}", PREFIX));
+   }
+
+
+
+   /*******************************************************************************
+    ** A connection to the shared in-memory database.
+    *******************************************************************************/
+   private static Connection connect() throws Exception
+   {
+      return (DriverManager.getConnection("jdbc:h2:mem:wpdefaults;MODE=PostgreSQL;DB_CLOSE_DELAY=-1", "sa", ""));
+   }
+
+
+
+   /*******************************************************************************
+    ** One real Liquibase update on its own connection (Liquibase closes it).
+    *******************************************************************************/
+   private static void update() throws Exception
+   {
+      Database database = DatabaseFactory.getInstance().findCorrectDatabaseImplementation(new JdbcConnection(connect()));
+      try(Liquibase liquibase = new Liquibase(WebPerfSchema.CHANGELOG, new ClassLoaderResourceAccessor(), database))
+      {
+         liquibase.setChangeLogParameter(WebPerfSchema.CHANGELOG_PREFIX_PROPERTY, PREFIX);
+         liquibase.update("");
+      }
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private static int count(Connection connection, String sql) throws Exception
+   {
+      try(Statement statement = connection.createStatement(); ResultSet resultSet = statement.executeQuery(sql))
+      {
+         resultSet.next();
+         return (resultSet.getInt(1));
+      }
+   }
+
+}

src/test/java/com/kingsrook/qbits/webperf/WebPerfForeignKeyTest.java

@@ -0,0 +1,251 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.webperf;
+
+
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.Statement;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.fields.QFieldMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.joins.QJoinMetaData;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+import liquibase.Contexts;
+import liquibase.LabelExpression;
+import liquibase.Liquibase;
+import liquibase.database.Database;
+import liquibase.database.DatabaseFactory;
+import liquibase.database.jvm.JdbcConnection;
+import liquibase.resource.ClassLoaderResourceAccessor;
+import org.junit.jupiter.api.Test;
+import org.testcontainers.containers.PostgreSQLContainer;
+import org.testcontainers.junit.jupiter.Container;
+import org.testcontainers.junit.jupiter.Testcontainers;
+import static org.assertj.core.api.Assertions.assertThat;
+
+
+/*******************************************************************************
+ ** Referential integrity on PostgreSQL (the production database), with the
+ ** changelog run by Liquibase as a host runs it:
+ **
+ ** - every table possible-value field that names one of the QBit's tables,
+ **   and every join, has a database foreign key (or an exemption with a
+ **   reason), so a new reference cannot be added without its constraint;
+ ** - rows whose parent is missing when the constraints are added are recorded
+ **   in qqq_fk_quarantine and repaired (nulled, or deleted with a full copy),
+ **   so the upgrade never fails on existing data;
+ ** - the foreign key changesets are no-ops on a database that already has the
+ **   constraints (a host that added them itself).
+ **
+ ** Skipped when Docker is not available.
+ *******************************************************************************/
+@Testcontainers(disabledWithoutDocker = true)
+class WebPerfForeignKeyTest
+{
+   private static final String PREFIX = "web_performance_";
+
+   /////////////////////////////////////////////////////////////////////////////
+   // references that are deliberately not foreign keys: "child.column" ->    //
+   // reason. None today (content_table + content_id is polymorphic, and has  //
+   // no possible-value source).                                              //
+   /////////////////////////////////////////////////////////////////////////////
+   private static final Map<String, String> EXEMPT = Map.of();
+
+   /////////////////////////////////////////////////////////////////////////////
+   // the changesets before the foreign keys (1-11)                           //
+   /////////////////////////////////////////////////////////////////////////////
+   private static final int CHANGESETS_BEFORE_FOREIGN_KEYS = 11;
+
+   @Container
+   private static final PostgreSQLContainer<?> POSTGRES = new PostgreSQLContainer<>("postgres:17-alpine");
+
+
+
+   /*******************************************************************************
+    ** Orphans are quarantined, every reference gets its constraint, and a
+    ** re-run of the foreign key changesets changes nothing.
+    *******************************************************************************/
+   @Test
+   void testEveryReferenceHasAForeignKey() throws Exception
+   {
+      try(Connection connection = DriverManager.getConnection(POSTGRES.getJdbcUrl(), POSTGRES.getUsername(), POSTGRES.getPassword());
+         Statement statement = connection.createStatement())
+      {
+         statement.execute("DROP SCHEMA public CASCADE");
+         statement.execute("CREATE SCHEMA public");
+
+         ////////////////////////////////////////////////////////////////////////
+         // an existing install with rows whose parents are gone               //
+         ////////////////////////////////////////////////////////////////////////
+         migrate(connection, CHANGESETS_BEFORE_FOREIGN_KEYS);
+         statement.execute("INSERT INTO web_performance_target (id, url) VALUES (1, 'https://www.example.com/')");
+         statement.execute("INSERT INTO web_performance_lab_run (id, target_id) VALUES (1, 1), (2, 404)");
+         statement.execute("INSERT INTO web_performance_lab_finding (id, run_id, target_id) VALUES (1, 1, 1), (2, 404, 1)");
+         statement.execute("INSERT INTO web_performance_lab_report (id, run_id) VALUES (1, 404)");
+         statement.execute("INSERT INTO web_performance_regression (id, target_id, budget_id, first_run_id) VALUES (1, 1, 404, 404)");
+
+         migrate(connection, null);
+
+         ////////////////////////////////////////////////////////////////////////
+         // every reference the meta-data declares has its constraint          //
+         ////////////////////////////////////////////////////////////////////////
+         Map<String, String> foreignKeys = foreignKeys(statement);
+         List<String>        missing     = new ArrayList<>();
+         for(String reference : references())
+         {
+            if(!foreignKeys.containsKey(reference) && !EXEMPT.containsKey(reference))
+            {
+               missing.add(reference);
+            }
+         }
+         assertThat(missing).as("references without a foreign key (add one to the changelog, or an exemption with a reason)").isEmpty();
+         assertThat(foreignKeys).containsEntry("web_performance_lab_finding.run_id", "web_performance_lab_run CASCADE")
+            .containsEntry("web_performance_lab_run.target_id", "web_performance_target RESTRICT")
+            .containsEntry("web_performance_regression.cause_change_event_id", "web_performance_change_event SET NULL");
+
+         ////////////////////////////////////////////////////////////////////////
+         // the orphans were recorded, then nulled or deleted                  //
+         ////////////////////////////////////////////////////////////////////////
+         assertThat(rows(statement, "SELECT table_name || '.' || column_name || ' ' || row_id || '->' || missing_parent_id || ' ' || action || ' ' || (row_data IS NOT NULL) FROM qqq_fk_quarantine"))
+            .containsExactlyInAnyOrder(
+               "web_performance_lab_run.target_id 2->404 NULLED false",
+               "web_performance_lab_finding.run_id 2->404 DELETED true",
+               "web_performance_lab_report.run_id 1->404 DELETED true",
+               "web_performance_regression.budget_id 1->404 NULLED false",
+               "web_performance_regression.first_run_id 1->404 NULLED false");
+         assertThat(rows(statement, "SELECT id || ' ' || COALESCE(CAST(target_id AS TEXT), '-') FROM web_performance_lab_run")).containsExactlyInAnyOrder("1 1", "2 -");
+         assertThat(rows(statement, "SELECT CAST(id AS TEXT) FROM web_performance_lab_finding")).containsExactly("1");
+         assertThat(rows(statement, "SELECT CAST(id AS TEXT) FROM web_performance_lab_report")).isEmpty();
+
+         ////////////////////////////////////////////////////////////////////////
+         // a host that already has the constraints: the changesets are marked //
+         // ran and change nothing                                             //
+         ////////////////////////////////////////////////////////////////////////
+         statement.execute("DELETE FROM databasechangelog WHERE CAST(SUBSTRING(id FROM '[0-9]+$') AS INTEGER) > " + CHANGESETS_BEFORE_FOREIGN_KEYS);
+         migrate(connection, null);
+         assertThat(foreignKeys(statement)).isEqualTo(foreignKeys);
+         assertThat(rows(statement, "SELECT CAST(COUNT(*) AS TEXT) FROM qqq_fk_quarantine")).containsExactly("5");
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Run the changelog (all of it, or the first n changesets).
+    *******************************************************************************/
+   private static void migrate(Connection connection, Integer count) throws Exception
+   {
+      Database database = DatabaseFactory.getInstance().findCorrectDatabaseImplementation(new JdbcConnection(connection));
+      Liquibase liquibase = new Liquibase(WebPerfSchema.CHANGELOG, new ClassLoaderResourceAccessor(), database);
+      liquibase.setChangeLogParameter(WebPerfSchema.CHANGELOG_PREFIX_PROPERTY, PREFIX);
+      if(count == null)
+      {
+         liquibase.update(new Contexts(), new LabelExpression());
+      }
+      else
+      {
+         liquibase.update(count, new Contexts(), new LabelExpression());
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** "child.column" for every table possible-value field and join of the QBit.
+    *******************************************************************************/
+   private static List<String> references() throws Exception
+   {
+      QInstance qInstance = WebPerfTestSupport.newInstance();
+      new WebPerfQBitProducer().withQBitConfig(WebPerfTestSupport.config().withTableNamePrefix(PREFIX)).produce(qInstance);
+
+      List<String> references = new ArrayList<>();
+      for(QTableMetaData table : qInstance.getTables().values())
+      {
+         for(QFieldMetaData field : table.getFields().values())
+         {
+            if(field.getPossibleValueSourceName() != null && qInstance.getTable(field.getPossibleValueSourceName()) != null)
+            {
+               references.add(table.getName() + "." + field.getBackendName());
+            }
+         }
+      }
+      for(QJoinMetaData join : qInstance.getJoins().values())
+      {
+         /////////////////////////////////////////////////////////////////////
+         // the referencing side is the one whose field is not its key      //
+         /////////////////////////////////////////////////////////////////////
+         QTableMetaData left  = qInstance.getTable(join.getLeftTable());
+         QTableMetaData right = qInstance.getTable(join.getRightTable());
+         join.getJoinOns().forEach(on -> references.add(on.getRightField().equals(right.getPrimaryKeyField())
+            ? left.getName() + "." + left.getField(on.getLeftField()).getBackendName()
+            : right.getName() + "." + right.getField(on.getRightField()).getBackendName()));
+      }
+      assertThat(references).as("the QBit's references").hasSizeGreaterThanOrEqualTo(12);
+      return (references);
+   }
+
+
+
+   /*******************************************************************************
+    ** "child.column" -> "parent RULE" for every foreign key in the database.
+    *******************************************************************************/
+   private static Map<String, String> foreignKeys(Statement statement) throws Exception
+   {
+      Map<String, String> foreignKeys = new HashMap<>();
+      try(ResultSet resultSet = statement.executeQuery("""
+         SELECT tc.table_name || '.' || kcu.column_name, ccu.table_name || ' ' || rc.delete_rule
+         FROM information_schema.table_constraints tc
+         JOIN information_schema.key_column_usage kcu ON kcu.constraint_name = tc.constraint_name
+         JOIN information_schema.referential_constraints rc ON rc.constraint_name = tc.constraint_name
+         JOIN information_schema.constraint_column_usage ccu ON ccu.constraint_name = tc.constraint_name
+         WHERE tc.constraint_type = 'FOREIGN KEY'"""))
+      {
+         while(resultSet.next())
+         {
+            foreignKeys.put(resultSet.getString(1), resultSet.getString(2));
+         }
+      }
+      return (foreignKeys);
+   }
+
+
+
+   /*******************************************************************************
+    ** The first column of each row of a query, as text.
+    *******************************************************************************/
+   private static List<String> rows(Statement statement, String sql) throws Exception
+   {
+      List<String> rows = new ArrayList<>();
+      try(ResultSet resultSet = statement.executeQuery(sql))
+      {
+         while(resultSet.next())
+         {
+            rows.add(resultSet.getString(1));
+         }
+      }
+      return (rows);
+   }
+}

src/test/java/com/kingsrook/qbits/webperf/WebPerfP1Test.java

@@ -206,6 +206,68 @@ class WebPerfP1Test
 
 
 
+   /*******************************************************************************
+    ** The Setup card reports pages, budgets, the lab runner and field data from
+    ** the runtime's own resolution, with the next step for each.
+    *******************************************************************************/
+   @Test
+   void testSetupCard() throws Exception
+   {
+      String card = html(WebPerfWidgets.SETUP, Map.of());
+      assertThat(card).contains("Needs setup").contains("Sync pages from sitemap").contains("Using defaults").contains("13 reference budgets");
+      assertThat(card).as("a configured runner and CrUX client count as set up").contains("A lab runner is configured").contains("Checked daily");
+
+      WebPerfService service = service();
+      service.applySync(service.planSync(List.of(SITE + "/")), Instant.now());
+      store().update("INSERT INTO {budget} (name, scope, source, metric, comparator, threshold, severity, enabled, seeded) VALUES ('mine', 'SITE', 'LAB', 'LCP', '<=', 2000, 'WARNING', TRUE, FALSE)");
+      card = html(WebPerfWidgets.SETUP, Map.of());
+      assertThat(card).contains("1 page").contains("14 budgets are enabled, 1 of them your own");
+
+      store().update("UPDATE {budget} SET enabled = FALSE");
+      assertThat(html(WebPerfWidgets.SETUP, Map.of())).contains("No budget is enabled");
+
+      config.withLabRunner(null).withCruxClient(null);
+      assertThat(html(WebPerfWidgets.SETUP, Map.of())).contains("No Google API key is configured, so lab tests cannot run").contains("Google field data is not checked");
+   }
+
+
+
+   /*******************************************************************************
+    ** reportRetentionDays deletes old Lighthouse Reports when a run stores its
+    ** own, and 0 (the default) keeps them forever.
+    *******************************************************************************/
+   @Test
+   void testReportRetention() throws Exception
+   {
+      for(int days : new int[] { 0, 30 })
+      {
+         store().update("DELETE FROM {lab_report}");
+         store().update("DELETE FROM {lab_finding}");
+         store().update("DELETE FROM {lab_run}");
+         store().update("DELETE FROM {budget}");
+         config.withReportRetentionDays(days);
+         google.respond(new FixtureServer.Response(200, FixtureServer.fixture("psi-kof22-home-mobile.json")));
+         long oldRunId = store().insert("lab_run", new java.util.LinkedHashMap<>(Map.of("status", "FAILED")));
+         store().insert("lab_report", new java.util.LinkedHashMap<>(Map.of("run_id", oldRunId, "bytes", 1,
+            "create_date", Instant.now().minus(java.time.Duration.ofDays(40)))));
+         WebPerfRuntime.of(qInstance, "web_performance_").allowReportPruneNow();
+         WebPerfService service = service();
+         service.applySync(service.planSync(List.of(SITE + "/")), Instant.now());
+         service.enqueue(service.requestsFor(service.enabledTargets(false), "MANUAL", "test", Instant.now()), Instant.now());
+         assertEquals(2, service.runDue(Instant.now()));
+
+         assertEquals(2, store().query("SELECT id FROM {lab_run} WHERE status = 'SUCCEEDED'").size(), "runs are untouched by pruning");
+         assertThat(store().query("SELECT id FROM {lab_finding}")).as("findings are untouched by pruning").hasSizeGreaterThan(5);
+         long old   = store().query("SELECT id FROM {lab_report} WHERE run_id = ?", oldRunId).size();
+         long fresh = store().query("SELECT id FROM {lab_report} WHERE run_id <> ?", oldRunId).size();
+         assertEquals(2, fresh, "the new reports are kept");
+         assertEquals(days == 0 ? 1 : 0, old, "retention " + days + " days");
+      }
+      assertThat(new WebPerfRuntime(config).reportPruneDue()).as("a new process does not prune right away").isFalse();
+   }
+
+
+
    /*******************************************************************************
     ** The worker runs a test, stores scores, findings, the report and the
     ** page's latest results.
@@ -213,6 +275,7 @@ class WebPerfP1Test
    @Test
    void testWorkerSuccess() throws Exception
    {
+      store().update("UPDATE {budget} SET enabled = FALSE");
       google.respond(new FixtureServer.Response(200, FixtureServer.fixture("psi-kof22-home-mobile.json")));
       WebPerfService service = service();
       service.applySync(service.planSync(List.of(SITE + "/")), Instant.now());
@@ -319,6 +382,7 @@ class WebPerfP1Test
    @Test
    void testActions() throws Exception
    {
+      store().update("UPDATE {budget} SET enabled = FALSE");
       RunProcessOutput sync = run(WebPerfProcesses.SYNC_TARGETS);
       assertThat(summary(sync)).anyMatch(l -> l.contains("3") && l.contains("added"));
       assertEquals(3, store().query("SELECT id FROM {target}").size());

src/test/java/com/kingsrook/qbits/webperf/WebPerfRegressionTest.java

@@ -0,0 +1,828 @@
+/*
+ * QQQ - Low-code Application Framework for Engineers.
+ * Copyright (C) 2021-2025.  Kingsrook, LLC
+ * 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
+ * contact@kingsrook.com
+ * https://github.com/Kingsrook/
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.webperf;
+
+
+import java.io.Serializable;
+import java.time.Instant;
+import java.time.LocalDate;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Executors;
+import com.kingsrook.qbits.webperf.lab.LabResultParser;
+import com.kingsrook.qbits.webperf.processes.RegressionSteps;
+import com.kingsrook.qbits.webperf.processes.WebPerfProcesses;
+import com.kingsrook.qbits.webperf.service.WebPerfChangeService;
+import com.kingsrook.qbits.webperf.service.WebPerfComparisonService;
+import com.kingsrook.qbits.webperf.service.WebPerfEvaluationService;
+import com.kingsrook.qbits.webperf.service.WebPerfService;
+import com.kingsrook.qbits.webperf.spi.ChangeEventSource;
+import com.kingsrook.qbits.webperf.spi.DeployProbe;
+import com.kingsrook.qbits.webperf.spi.LabRunner;
+import com.kingsrook.qbits.webperf.store.WebPerfStore;
+import com.kingsrook.qqq.backend.core.actions.processes.QProcessCallback;
+import com.kingsrook.qqq.backend.core.actions.processes.RunProcessAction;
+import com.kingsrook.qqq.backend.core.alerts.AlertResolution;
+import com.kingsrook.qqq.backend.core.alerts.AlertSignal;
+import com.kingsrook.qqq.backend.core.alerts.AlertSinkInterface;
+import com.kingsrook.qqq.backend.core.alerts.AlertsInstanceMetaData;
+import com.kingsrook.qqq.backend.core.alerts.QAlerts;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessInput;
+import com.kingsrook.qqq.backend.core.model.actions.processes.RunProcessOutput;
+import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
+import com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess;
+import org.json.JSONObject;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+
+
+/*******************************************************************************
+ ** P2 behavior on H2, with fake probes and runner and real QQQ record actions.
+ *******************************************************************************/
+public class WebPerfRegressionTest
+{
+   private static final Instant START = Instant.parse("2026-10-01T12:00:00Z");
+   private QInstance instance;
+   private WebPerfQBitConfig config;
+   private WebPerfStore store;
+   private WebPerfEvaluationService evaluator;
+   private long target;
+   private int sequence;
+
+
+
+   /*******************************************************************************
+    ** Fresh tables; disable reference budgets to isolate statistical rules.
+    *******************************************************************************/
+   @BeforeEach
+   void setUp() throws Exception
+   {
+      instance = WebPerfTestSupport.newInstance();
+      config = WebPerfTestSupport.config().withStrategies(List.of("MOBILE")).withMinSpacingMillis(0).withConcurrency(2);
+      new WebPerfQBitProducer().withQBitConfig(config).produce(instance);
+      WebPerfTestSupport.applyChangelog(instance, config.getTableNamePrefix());
+      WebPerfTestSupport.login(instance, "web_performance.read", "web_performance.write", "web_performance.manage.hasAccess",
+         "web_performance.view.hasAccess", "web_performance.admin.hasAccess", "web_performance.test.hasAccess");
+      store = new WebPerfRuntime(config).store(instance);
+      evaluator = new WebPerfEvaluationService(store);
+      store.update("UPDATE {budget} SET enabled = FALSE");
+      WebPerfService service = new WebPerfRuntime(config).service();
+      service.applySync(service.planSync(List.of("https://www.example.com/")), START);
+      target = WebPerfStore.asLong(store.query("SELECT id FROM {target}").get(0).get("id"));
+      store.update("UPDATE {target} SET tier = 'KEY', content_table = 'page', content_id = 7 WHERE id = ?", target);
+      RecordingSink.signals.clear();
+      RecordingSink.resolutions.clear();
+      AlertsInstanceMetaData.ofOrWithNew(instance).withSink(new QCodeReference(RecordingSink.class));
+   }
+
+
+
+   /*******************************************************************************
+    ** Release this test's context.
+    *******************************************************************************/
+   @AfterEach
+   void tearDown()
+   {
+      QContext.clear();
+   }
+
+
+
+   /*******************************************************************************
+    ** Suspect, two confirmations, one alert, then three recovery samples.
+    *******************************************************************************/
+   @Test
+   void confirmsAndRecoversWithoutDuplicates() throws Exception
+   {
+      baseline();
+      long change = change("FRONTEND_DEPLOY", "FRONTEND", "build-2", START.plusSeconds(6));
+      long first = measurement(70, "SCHEDULE", "13.1", Map.of("change_event_id", change));
+      evaluator.evaluateLab(first);
+      assertThat(regression().get("status")).isEqualTo("SUSPECTED");
+      assertThat(store.query("SELECT * FROM {lab_run} WHERE trigger_type = 'CONFIRM'")).hasSize(2);
+      assertThat(RecordingSink.signals).isEmpty();
+      assertThat(openCount()).isEqualTo(1);
+      completeConfirmations(first, 72, 71);
+      assertThat(regression().get("status")).isEqualTo("CONFIRMED");
+      assertThat(WebPerfStore.asDouble(regression().get("current_value"))).isEqualTo(71);
+      assertThat(regression().get("baseline_ref")).isEqualTo("5,4,3,2,1");
+      assertThat(WebPerfStore.asLong(regression().get("cause_change_event_id"))).isEqualTo(change);
+      evaluator.evaluateLab(first);
+      assertThat(store.query("SELECT * FROM {regression}")).hasSize(1);
+      assertThat(RecordingSink.signals).hasSize(1);
+      evaluate(96);
+      evaluate(95);
+      assertThat(regression().get("status")).isEqualTo("CONFIRMED");
+      evaluate(94);
+      assertThat(regression().get("status")).isEqualTo("RESOLVED");
+      assertThat(openCount()).isZero();
+      assertThat(RecordingSink.resolutions).hasSize(1);
+      assertThat(RecordingSink.signals.get(0).getDedupKey()).isEqualTo(RecordingSink.resolutions.get(0).getDedupKey());
+   }
+
+
+
+   /*******************************************************************************
+    ** Lab noise stays visible as not confirmed, with no alert.
+    *******************************************************************************/
+   @Test
+   void rejectsNoise() throws Exception
+   {
+      baseline();
+      long first = evaluate(70);
+      completeConfirmations(first, 96, 95);
+      assertThat(regression().get("status")).isEqualTo("NOT_CONFIRMED");
+      assertThat(openCount()).isZero();
+      assertThat(RecordingSink.signals).isEmpty();
+   }
+
+
+
+   /*******************************************************************************
+    ** Baselines require five eligible recent results on the same major version.
+    *******************************************************************************/
+   @Test
+   void boundsBaselineHistory() throws Exception
+   {
+      for(int i = 0; i < 4; i++)
+      {
+         measurement(96, "SCHEDULE", "13.0", Map.of());
+      }
+      measurement(96, "MANUAL", "13.0", Map.of());
+      measurement(96, "SCHEDULE", "12.9", Map.of());
+      long old = measurement(96, "SCHEDULE", "13.0", Map.of());
+      store.update("UPDATE {lab_run} SET completed_at = ? WHERE id = ?", START.minusSeconds(31 * 86400L), old);
+      evaluate(70);
+      assertThat(store.query("SELECT * FROM {regression}")).isEmpty();
+      measurement(96, "SITE_AUDIT", "13.1", Map.of());
+      evaluate(60);
+      assertThat(regression().get("status")).isEqualTo("SUSPECTED");
+   }
+
+
+
+   /*******************************************************************************
+    ** A major-version change records a marker and retires incompatible baselines.
+    *******************************************************************************/
+   @Test
+   void resetsAtMajorVersionChange() throws Exception
+   {
+      baseline();
+      long first = evaluate(70);
+      completeConfirmations(first, 71, 72);
+      evaluator.evaluateLab(measurement(60, "SCHEDULE", "14.0", Map.of()));
+      assertThat(regression().get("status")).isEqualTo("RESOLVED");
+      assertThat(regression().get("resolution_note")).isEqualTo("Lighthouse major version changed; baseline reset");
+      assertThat(store.query("SELECT * FROM {change_event} WHERE kind = 'LIGHTHOUSE_VERSION'")).hasSize(1);
+      assertThat(store.query("SELECT * FROM {regression}")).hasSize(1);
+   }
+
+
+
+   /*******************************************************************************
+    ** Relative and absolute thresholds; deterministic categories confirm now.
+    *******************************************************************************/
+   @Test
+   void appliesMetricThresholds() throws Exception
+   {
+      baseline();
+      evaluator.evaluateLab(measurement(95, "MANUAL", "13.0", Map.of("lcp_ms", 2499, "cls", 0.049, "tbt_ms", 199)));
+      assertThat(store.query("SELECT * FROM {regression}")).isEmpty();
+      evaluator.evaluateLab(measurement(95, "MANUAL", "13.0", Map.of("lcp_ms", 2500, "cls", 0.05, "tbt_ms", 200,
+         "score_accessibility", 95, "score_best_practices", 95, "score_seo", 95)));
+      assertThat(store.query("SELECT * FROM {regression} WHERE status = 'SUSPECTED'")).hasSize(3);
+      assertThat(store.query("SELECT * FROM {regression} WHERE status = 'CONFIRMED'")).hasSize(3);
+      assertThat(store.query("SELECT * FROM {lab_run} WHERE trigger_type = 'CONFIRM'")).hasSize(2);
+      assertThat(RecordingSink.signals).hasSize(3);
+   }
+
+
+
+   /*******************************************************************************
+    ** A newly failing audit is actionable even if the rounded score is unchanged.
+    *******************************************************************************/
+   @Test
+   void newlyFailingAuditConfirms() throws Exception
+   {
+      baseline();
+      long run = measurement(96, "MANUAL", "13.0", Map.of());
+      store.insert(WebPerfSchema.LAB_FINDING, Map.of("run_id", run, "target_id", target, "strategy", "MOBILE",
+         "category", "SEO", "audit_id", "robots-txt", "kind", "FAILED_AUDIT", "score", 0));
+      evaluator.evaluateLab(run);
+      assertThat(regression().get("status")).isEqualTo("CONFIRMED");
+      assertThat(regression().get("failed_audits")).isEqualTo("robots-txt");
+      evaluate(96);
+      evaluate(96);
+      evaluate(96);
+      assertThat(regression().get("status")).isEqualTo("RESOLVED");
+   }
+
+
+
+   /*******************************************************************************
+    ** Every scope and comparator, with source, strategy and severity matching.
+    *******************************************************************************/
+   @Test
+   void matchesBudgetScopesComparatorsAndSeverity() throws Exception
+   {
+      Map<String, Object> page = store.query("SELECT * FROM {target}").get(0);
+      for(String scope : List.of("SITE", "TIER", "TARGET", "CONTENT_TABLE", "PATH_PREFIX"))
+      {
+         String value = switch(scope)
+         {
+            case "SITE" -> "host:www.example.com";
+            case "TIER" -> "KEY";
+            case "TARGET" -> String.valueOf(target);
+            case "CONTENT_TABLE" -> "page";
+            default -> "/";
+         };
+         long id = budget(scope, value, "LAB", "MOBILE", "LCP", "<=", 2500, "WARNING");
+         assertThat(evaluator.breachedBudget(page, "LAB", "MOBILE", "LCP", 2500)).isNull();
+         assertThat(evaluator.breachedBudget(page, "LAB", "MOBILE", "LCP", 2501)).isNotNull();
+         assertThat(evaluator.breachedBudget(page, "LAB", "DESKTOP", "LCP", 2501)).isNull();
+         assertThat(evaluator.breachedBudget(page, "CRUX", "MOBILE", "LCP", 2501)).isNull();
+         store.update("UPDATE {budget} SET scope_value = 'unmatched' WHERE id = ?", id);
+         assertThat(evaluator.breachedBudget(page, "LAB", "MOBILE", "LCP", 2501)).isNull();
+         store.update("UPDATE {budget} SET enabled = FALSE WHERE id = ?", id);
+      }
+      for(String comparator : List.of("<", ">", ">=", "=", "!=", "<=", "==", "<>", "LE", "GE", "LT", "GT", "EQ", "NE"))
+      {
+         long id = budget("SITE", "", "LAB", null, "LCP", comparator, 10, "WARNING");
+         for(double value : List.of(9.0, 10.0, 11.0))
+         {
+            boolean acceptable = switch(comparator)
+            {
+               case "<", "LT" -> value < 10;
+               case ">", "GT" -> value > 10;
+               case ">=", "GE" -> value >= 10;
+               case "<=", "LE" -> value <= 10;
+               case "=", "==", "EQ" -> value == 10;
+               default -> value != 10;
+            };
+            assertThat(evaluator.breachedBudget(page, "LAB", "MOBILE", "LCP", value) == null).isEqualTo(acceptable);
+         }
+         store.update("UPDATE {budget} SET enabled = FALSE WHERE id = ?", id);
+      }
+      budget("TIER", "KEY", "LAB", null, "PERFORMANCE_SCORE", ">=", 90, "WARNING");
+      long critical = budget("TIER", "KEY", "LAB", null, "PERFORMANCE_SCORE", ">=", 50, "CRITICAL");
+      long first = evaluate(40);
+      assertThat(WebPerfStore.asLong(regression().get("budget_id"))).isEqualTo(critical);
+      completeConfirmations(first, 40, 41);
+      assertThat(regression().get("severity")).isEqualTo("CRITICAL");
+   }
+
+
+
+   /*******************************************************************************
+    ** CrUX URL budgets use real values, never absent data or origin fallbacks.
+    *******************************************************************************/
+   @Test
+   void evaluatesCruxUrlBudgets() throws Exception
+   {
+      budget("SITE", "", "CRUX", "MOBILE", "LCP", "<=", 2500, "WARNING");
+      long first = field("URL", "OK", 3000, 1);
+      evaluator.evaluateField(first);
+      evaluator.evaluateField(first);
+      assertThat(regression().get("source")).isEqualTo("CRUX");
+      assertThat(regression().get("status")).isEqualTo("CONFIRMED");
+      evaluator.evaluateField(field("URL", "NO_DATA", null, 2));
+      evaluator.evaluateField(field("ORIGIN", "OK", 1000, 3));
+      assertThat(regression().get("status")).isEqualTo("CONFIRMED");
+      evaluator.evaluateField(field("URL", "OK", 2000, 4));
+      assertThat(regression().get("status")).isEqualTo("RESOLVED");
+      assertThat(openCount()).isZero();
+      assertThat(RecordingSink.signals).hasSize(1);
+   }
+
+
+
+   /*******************************************************************************
+    ** Poll replay, settle delays and attribution through the real queue worker.
+    *******************************************************************************/
+   @Test
+   void pollsAndAttributesWorkerResults() throws Exception
+   {
+      MutableProbe probe = new MutableProbe();
+      config.withDeployProbe(probe).withLabRunner(new FakeRunner());
+      ChangeEventSource.Change cdn = new ChangeEventSource.Change("CDN_STAGE", "www.example.com", "OFF", "HEADERS_ONLY", START);
+      ChangeEventSource.Change content = new ChangeEventSource.Change("CONTENT", "page:7", "old", "new", START.plusSeconds(1));
+      config.withChangeEventSource(since -> List.of(cdn, content));
+      WebPerfChangeService changes = new WebPerfChangeService(store);
+      changes.poll(START.plusSeconds(2));
+      changes.poll(START.plusSeconds(3));
+      assertThat(store.query("SELECT * FROM {change_event}")).hasSize(3);
+      assertThat(store.query("SELECT * FROM {lab_run} WHERE trigger_type = 'CHANGE_COMPARE'")).hasSize(3);
+      assertThat(store.query("SELECT * FROM {lab_run} WHERE trigger_type = 'DEPLOY'").get(0).get("not_before")).isEqualTo(START.plusSeconds(302));
+      new WebPerfRuntime(config).service().runDue(Instant.now());
+      List<Map<String, Object>> done = store.query("SELECT * FROM {lab_run} WHERE status = 'SUCCEEDED'");
+      assertThat(done).hasSize(2);
+      assertThat(done).allSatisfy(row ->
+      {
+         assertThat(row.get("deployment_id")).isEqualTo("build-1");
+         assertThat(row.get("cdn_stage")).isEqualTo("HEADERS_ONLY");
+         assertThat(row.get("evaluated_at")).isNotNull();
+      });
+      probe.build = "build-2";
+      changes.poll(Instant.now());
+      assertThat(store.query("SELECT * FROM {change_event} WHERE kind = 'FRONTEND_DEPLOY'")).hasSize(2);
+      probe.build = null;
+      changes.poll(Instant.now());
+      assertThat(store.query("SELECT * FROM {change_event} WHERE kind = 'FRONTEND_DEPLOY'")).hasSize(2);
+   }
+
+
+
+   /*******************************************************************************
+    ** Three samples on each side create reproducible, idempotent comparisons.
+    *******************************************************************************/
+   @Test
+   void recordsComparisonsWithCountsAndWindows() throws Exception
+   {
+      for(int i = 0; i < 3; i++)
+      {
+         measurement(90, "CHANGE_COMPARE", "13.0", Map.of("lcp_ms", 3000, "fcp_ms", 2000, "ttfb_ms", 900));
+      }
+      long event = change("CDN_STAGE", "www.example.com", "LIVE", START.plusSeconds(4));
+      sequence = 4;
+      for(int i = 0; i < 3; i++)
+      {
+         long id = measurement(96, "CHANGE_COMPARE", "13.1", Map.of("change_event_id", event, "lcp_ms", 2000, "fcp_ms", 1500, "ttfb_ms", 600));
+         evaluator.evaluateLab(id);
+         assertThat(store.query("SELECT * FROM {comparison}")).hasSize(i == 2 ? 4 : 0);
+      }
+      assertThat(new WebPerfComparisonService(store).compare(event)).isEqualTo(4);
+      assertThat(store.query("SELECT * FROM {comparison}")).hasSize(4);
+      Map<String, Object> lcp = store.query("SELECT * FROM {comparison} WHERE metric = 'LCP'").get(0);
+      assertThat(WebPerfStore.asDouble(lcp.get("delta"))).isEqualTo(-1000);
+      assertThat(WebPerfStore.asInt(lcp.get("before_n"))).isEqualTo(3);
+      assertThat(WebPerfStore.asInt(lcp.get("after_n"))).isEqualTo(3);
+      assertThat(lcp.get("before_window").toString()).contains(START.plusSeconds(1).toString());
+      assertThat(runAction(WebPerfProcesses.COMPARE_CHANGE, Map.of()).getException()).isEmpty();
+   }
+
+
+
+   /*******************************************************************************
+    ** Real ETL actions require a note and manage permission, and retain history.
+    *******************************************************************************/
+   @Test
+   void actionsRequireNotesAndPermissions() throws Exception
+   {
+      budget("SITE", "", "LAB", null, "SEO_SCORE", ">=", 100, "WARNING");
+      evaluator.evaluateLab(measurement(96, "MANUAL", "13.0", Map.of("score_seo", 90)));
+      long id = WebPerfStore.asLong(regression().get("id"));
+      assertThrows(com.kingsrook.qqq.backend.core.exceptions.QException.class, () -> runAction(WebPerfProcesses.DISMISS_REGRESSIONS, Map.of()));
+      assertThat(regression().get("status")).isEqualTo("CONFIRMED");
+      WebPerfTestSupport.login(instance, "web_performance.read", "web_performance.write");
+      assertThrows(com.kingsrook.qqq.backend.core.exceptions.QException.class, () -> runAction(WebPerfProcesses.DISMISS_REGRESSIONS, Map.of(RegressionSteps.NOTE, "Expected change")));
+      assertThat(regression().get("status")).isEqualTo("CONFIRMED");
+      WebPerfTestSupport.login(instance, "web_performance.read", "web_performance.write", "web_performance.manage.hasAccess");
+      RunProcessOutput result = runAction(WebPerfProcesses.DISMISS_REGRESSIONS, Map.of(RegressionSteps.NOTE, "Expected change"));
+      assertThat(result.getException()).isEmpty();
+      assertThat(result.getValues().get(StreamedETLWithFrontendProcess.FIELD_PROCESS_SUMMARY)).isNotNull();
+      assertThat(regression().get("status")).isEqualTo("DISMISSED");
+      assertThat(regression().get("resolution_note")).isEqualTo("Expected change");
+      assertThat(openCount()).isZero();
+      assertThat(evaluator.finishRegression(id, "RESOLVED", "Already closed", START)).isFalse();
+      evaluator.evaluateLab(measurement(96, "MANUAL", "13.0", Map.of("score_seo", 90)));
+      assertThat(runAction(WebPerfProcesses.RESOLVE_REGRESSIONS, Map.of(RegressionSteps.NOTE, "Fixed externally")).getException()).isEmpty();
+      assertThat(store.query("SELECT * FROM {regression} WHERE status = 'RESOLVED'")).hasSize(1);
+   }
+
+
+
+   /*******************************************************************************
+    ** Target locking prevents concurrent duplicate incidents; failed writes roll back.
+    *******************************************************************************/
+   @Test
+   void serializesEvaluationAndRollsBack() throws Exception
+   {
+      baseline();
+      long id = measurement(70, "SCHEDULE", "13.0", Map.of());
+      try(var executor = Executors.newFixedThreadPool(2))
+      {
+         var first = executor.submit(() ->
+         {
+            evaluator.evaluateLab(id);
+            return (null);
+         });
+         var second = executor.submit(() ->
+         {
+            evaluator.evaluateLab(id);
+            return (null);
+         });
+         first.get();
+         second.get();
+      }
+      assertThat(store.query("SELECT * FROM {regression}")).hasSize(1);
+      assertThat(store.query("SELECT * FROM {lab_run} WHERE trigger_type = 'CONFIRM'")).hasSize(2);
+      assertThrows(java.sql.SQLException.class, () -> store.transaction(tx ->
+      {
+         tx.update("UPDATE {target} SET label = 'must roll back' WHERE id = ?", target);
+         tx.update("UPDATE no_such_table SET id = 1");
+         return (null);
+      }));
+      assertThat(store.query("SELECT label FROM {target}").get(0).get("label")).isNotEqualTo("must roll back");
+   }
+
+
+
+   /*******************************************************************************
+    ** Core falls back to logging without an Alerts QBit; feature gating works.
+    *******************************************************************************/
+   @Test
+   void alertsAreOptional() throws Exception
+   {
+      AlertsInstanceMetaData.ofOrWithNew(instance).withSink(null);
+      long failures = QAlerts.getProducerFailureCount();
+      budget("SITE", "", "LAB", null, "SEO_SCORE", ">=", 100, "WARNING");
+      evaluator.evaluateLab(measurement(96, "MANUAL", "13.0", Map.of("score_seo", 90)));
+      assertThat(QAlerts.getProducerFailureCount()).isEqualTo(failures);
+      config.getFeatures().remove(WebPerfFeature.ALERTS);
+      evaluator.finishRegression(WebPerfStore.asLong(regression().get("id")), "RESOLVED", "Done", START);
+      assertThat(RecordingSink.signals).isEmpty();
+   }
+
+
+
+   /*******************************************************************************
+    ** TTFB comes from a measured subpart and is null when not reported.
+    *******************************************************************************/
+   @Test
+   void parsesMeasuredTtfbOnly()
+   {
+      assertThat(LabResultParser.ttfb("{\"audits\":{}}")).isNull();
+      assertThat(LabResultParser.ttfb("{\"audits\":{\"lcp-phases-insight\":{\"details\":{\"items\":[{\"subpart\":\"timeToFirstByte\",\"duration\":321.4}]}}}}")).isEqualTo(321);
+   }
+
+
+
+   /*******************************************************************************
+    ** Quota skips remain visible, and retry preserves each deliberate sample slot.
+    *******************************************************************************/
+   @Test
+   void retriesCappedConfirmationSamples() throws Exception
+   {
+      baseline();
+      config.withMaxRunsPerDay(1);
+      long first = evaluate(70);
+      assertThat(store.query("SELECT * FROM {lab_run} WHERE status = 'SKIPPED_QUOTA'")).hasSize(1);
+      config.withMaxRunsPerDay(500);
+      assertThat(runAction(WebPerfProcesses.RETRY_RUNS, Map.of()).getException()).isEmpty();
+      assertThat(store.query("SELECT * FROM {lab_run} WHERE status = 'QUEUED' AND trigger_type = 'CONFIRM'")).hasSize(2);
+      assertThat(runAction(WebPerfProcesses.RETRY_RUNS, Map.of()).getException()).isEmpty();
+      assertThat(store.query("SELECT * FROM {lab_run} WHERE status = 'QUEUED' AND trigger_type = 'CONFIRM'")).hasSize(2);
+      store.update("UPDATE {lab_run} SET status = 'SUCCEEDED', completed_at = ?, lighthouse_version = '13.0', score_performance = 70 WHERE status = 'QUEUED'",
+         START.plusSeconds(10));
+      evaluator.evaluatePending();
+      assertThat(regression().get("status")).isEqualTo("CONFIRMED");
+      assertThat(WebPerfStore.asLong(regression().get("first_run_id"))).isEqualTo(first);
+   }
+
+
+
+   /*******************************************************************************
+    ** Existing seeded budgets are evaluated even before enough baseline exists.
+    *******************************************************************************/
+   @Test
+   void seededBudgetsWorkWithoutBaseline() throws Exception
+   {
+      store.update("UPDATE {budget} SET enabled = TRUE WHERE seeded = TRUE");
+      long first = evaluate(40);
+      assertThat(regression().get("status")).isEqualTo("SUSPECTED");
+      assertThat(regression().get("baseline_value")).isNull();
+      assertThat(regression().get("severity")).isEqualTo("CRITICAL");
+      completeConfirmations(first, 40, 40);
+      assertThat(regression().get("status")).isEqualTo("CONFIRMED");
+   }
+
+
+
+   /*******************************************************************************
+    ** Build replays are idempotent even after a rollback to an earlier build.
+    *******************************************************************************/
+   @Test
+   void attributesRollbackAndQueuesBothCdnWindows() throws Exception
+   {
+      MutableProbe probe = new MutableProbe();
+      config.withDeployProbe(probe);
+      WebPerfChangeService changes = new WebPerfChangeService(store);
+      changes.poll(START);
+      probe.build = "build-2";
+      changes.poll(START.plusSeconds(1));
+      probe.build = "build-1";
+      changes.poll(START.plusSeconds(2));
+      long run = measurement(96, "MANUAL", "13.0", Map.of());
+      changes.attribute(run, START.plusSeconds(3));
+      assertThat(store.query("SELECT deployment_id FROM {lab_run} WHERE id = ?", run).get(0).get("deployment_id")).isEqualTo("build-1");
+      assertThat(store.query("SELECT * FROM {change_event}")).hasSize(2);
+      ChangeEventSource.Change cdn = new ChangeEventSource.Change("CDN_STAGE", "www.example.com", "HEADERS_ONLY", "LIVE", START.plusSeconds(4));
+      config.withChangeEventSource(since -> List.of(cdn));
+      changes.poll(START.plusSeconds(4));
+      changes.poll(START.plusSeconds(5));
+      assertThat(store.query("SELECT * FROM {lab_run} WHERE trigger_type = 'CHANGE_COMPARE'")).hasSize(6);
+      assertThat(store.query("SELECT * FROM {lab_run} WHERE trigger_type = 'CHANGE_COMPARE' AND not_before = ?", START.plusSeconds(3604))).hasSize(3);
+      assertThat(store.query("SELECT * FROM {lab_run} WHERE trigger_type = 'CHANGE_COMPARE' AND not_before = ?", START.plusSeconds(86404))).hasSize(3);
+      assertThat(runAction(WebPerfProcesses.PROBE_DEPLOYS, Map.of()).getException()).isEmpty();
+   }
+
+
+
+   /*******************************************************************************
+    ** Five clean baseline measurements.
+    *******************************************************************************/
+   private void baseline() throws Exception
+   {
+      for(int i = 0; i < 5; i++)
+      {
+         measurement(96, "SCHEDULE", "13.0", Map.of());
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Store a deterministic successful measurement, with no network access.
+    *******************************************************************************/
+   private long measurement(int score, String trigger, String version, Map<String, Object> extra) throws Exception
+   {
+      Instant time = START.plusSeconds(++sequence);
+      Map<String, Object> row = new LinkedHashMap<>();
+      row.put("target_id", target);
+      row.put("url", "https://www.example.com/");
+      row.put("strategy", "MOBILE");
+      row.put("status", "SUCCEEDED");
+      row.put("trigger_type", trigger);
+      row.put("lighthouse_version", version);
+      row.put("started_at", time);
+      row.put("completed_at", time);
+      row.put("queued_at", time);
+      row.put("score_performance", score);
+      row.put("score_accessibility", 100);
+      row.put("score_best_practices", 100);
+      row.put("score_seo", 100);
+      row.put("lcp_ms", 2000);
+      row.put("fcp_ms", 1000);
+      row.put("tbt_ms", 0);
+      row.put("cls", 0);
+      row.putAll(extra);
+      return (store.insert(WebPerfSchema.LAB_RUN, row));
+   }
+
+
+
+   /*******************************************************************************
+    ** Store and evaluate the next performance measurement.
+    *******************************************************************************/
+   private long evaluate(int score) throws Exception
+   {
+      long id = measurement(score, "SCHEDULE", "13.0", Map.of());
+      evaluator.evaluateLab(id);
+      return (id);
+   }
+
+
+
+   /*******************************************************************************
+    ** Complete the independently queued confirmation samples.
+    *******************************************************************************/
+   private void completeConfirmations(long first, int score1, int score2) throws Exception
+   {
+      int index = 0;
+      for(Map<String, Object> row : store.query("SELECT id FROM {lab_run} WHERE trigger_ref IN (?, ?) ORDER BY id", "confirm:" + first + ":0", "confirm:" + first + ":1"))
+      {
+         store.update("UPDATE {lab_run} SET status = 'SUCCEEDED', completed_at = ?, lighthouse_version = '13.0', score_performance = ? WHERE id = ?",
+            START.plusSeconds(++sequence), index++ == 0 ? score1 : score2, row.get("id"));
+         evaluator.evaluateLab(WebPerfStore.asLong(row.get("id")));
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** One editable budget.
+    *******************************************************************************/
+   private long budget(String scope, String scopeValue, String source, String strategy, String metric, String comparator, double threshold, String severity) throws Exception
+   {
+      Map<String, Object> row = new LinkedHashMap<>();
+      row.put("name", "Test budget");
+      row.put("scope", scope);
+      row.put("scope_value", scopeValue);
+      row.put("source", source);
+      row.put("strategy", strategy);
+      row.put("metric", metric);
+      row.put("comparator", comparator);
+      row.put("threshold", threshold);
+      row.put("severity", severity);
+      row.put("enabled", true);
+      return (store.insert(WebPerfSchema.BUDGET, row));
+   }
+
+
+
+   /*******************************************************************************
+    ** A source event without automatic queueing for fixtures.
+    *******************************************************************************/
+   private long change(String kind, String component, String value, Instant time) throws Exception
+   {
+      return (new WebPerfChangeService(store).record(new ChangeEventSource.Change(kind, component, "before", value, time), "WEBHOOK", "test", false));
+   }
+
+
+
+   /*******************************************************************************
+    ** A real or unavailable CrUX collection period.
+    *******************************************************************************/
+   private long field(String scope, String status, Integer lcp, int day) throws Exception
+   {
+      Map<String, Object> row = new LinkedHashMap<>();
+      row.put("source", "CRUX_API");
+      row.put("scope", scope);
+      row.put("scope_key", "https://www.example.com/");
+      row.put("form_factor", "PHONE");
+      row.put("status", status);
+      row.put("p75_lcp_ms", lcp);
+      row.put("period_end", LocalDate.of(2026, 10, day));
+      row.put("last_checked_at", START.plusSeconds(day));
+      return (store.insert(WebPerfSchema.FIELD_SNAPSHOT, row));
+   }
+
+
+
+   /*******************************************************************************
+    ** The first incident, retained through status changes.
+    *******************************************************************************/
+   private Map<String, Object> regression() throws Exception
+   {
+      return (store.query("SELECT * FROM {regression} ORDER BY id").get(0));
+   }
+
+
+
+   /*******************************************************************************
+    ** The count shown in Tested Pages.
+    *******************************************************************************/
+   private int openCount() throws Exception
+   {
+      return (WebPerfStore.asInt(store.query("SELECT open_regression_count FROM {target} WHERE id = ?", target).get(0).get("open_regression_count")));
+   }
+
+
+
+   /*******************************************************************************
+    ** Run the actual ETL process, following the existing frontend-skip convention.
+    *******************************************************************************/
+   private RunProcessOutput runAction(String name, Map<String, Serializable> values) throws Exception
+   {
+      RunProcessInput input = new RunProcessInput();
+      input.setPermissionEnforcement(com.kingsrook.qqq.backend.core.actions.permissions.PermissionEnforcement.ENFORCE);
+      input.setProcessName(config.qualify(name));
+      input.setValues(new HashMap<>(values));
+      input.setFrontendStepBehavior(RunProcessInput.FrontendStepBehavior.SKIP);
+      input.setCallback(new QProcessCallback()
+      {
+         /*******************************************************************************
+          ** Select this test's records.
+          *******************************************************************************/
+         @Override
+         public QQueryFilter getQueryFilter()
+         {
+            return (new QQueryFilter());
+         }
+      });
+      return (new RunProcessAction().execute(input));
+   }
+
+
+
+   /*******************************************************************************
+    ** Core alert sink, without any Alerts QBit dependency.
+    *******************************************************************************/
+   public static class RecordingSink implements AlertSinkInterface
+   {
+      static final List<AlertSignal> signals = new ArrayList<>();
+      static final List<AlertResolution> resolutions = new ArrayList<>();
+
+
+
+      /*******************************************************************************
+       ** Capture a raise.
+       *******************************************************************************/
+      @Override
+      public void raise(AlertSignal signal)
+      {
+         signals.add(signal);
+      }
+
+
+
+      /*******************************************************************************
+       ** Capture a resolution.
+       *******************************************************************************/
+      @Override
+      public void resolve(AlertResolution resolution)
+      {
+         resolutions.add(resolution);
+      }
+
+
+
+      /*******************************************************************************
+       ** This producer does not emit heartbeats.
+       *******************************************************************************/
+      @Override
+      public void heartbeat(String key, Instant at)
+      {
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** A probe controlled by the test.
+    *******************************************************************************/
+   private static class MutableProbe implements DeployProbe
+   {
+      String build = "build-1";
+
+
+
+      /*******************************************************************************
+       ** Frontend component.
+       *******************************************************************************/
+      @Override
+      public String component()
+      {
+         return ("FRONTEND");
+      }
+
+
+
+      /*******************************************************************************
+       ** Build id without transport.
+       *******************************************************************************/
+      @Override
+      public String currentBuildId()
+      {
+         return (build);
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** A minimal Lighthouse runner without transport.
+    *******************************************************************************/
+   private static class FakeRunner implements LabRunner
+   {
+      /*******************************************************************************
+       ** Stored runner name.
+       *******************************************************************************/
+      @Override
+      public String name()
+      {
+         return ("TEST");
+      }
+
+
+
+      /*******************************************************************************
+       ** Return a successful response.
+       *******************************************************************************/
+      @Override
+      public LabResponse run(LabRequest request)
+      {
+         JSONObject body = new JSONObject().put("lighthouseVersion", "13.0").put("finalUrl", request.url())
+            .put("categories", new JSONObject().put("performance", new JSONObject().put("score", 0.96)))
+            .put("audits", new JSONObject());
+         return (new LabResponse(200, body.toString(), Instant.now()));
+      }
+   }
+}

src/test/java/com/kingsrook/qbits/webperf/WebPerfTablesGuardrailTest.java

@@ -0,0 +1,185 @@
+/*
+ * Copyright 2026 Kingsrook, LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.kingsrook.qbits.webperf;
+
+
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.regex.Pattern;
+import java.util.stream.Stream;
+import com.kingsrook.qbits.webperf.metadata.WebPerfHelp;
+import com.kingsrook.qqq.backend.core.context.QContext;
+import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
+import com.kingsrook.qqq.backend.core.model.metadata.help.QHelpContent;
+import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Test;
+import static org.assertj.core.api.Assertions.assertThat;
+
+
+/*******************************************************************************
+ ** Guardrail (owner requirement 2026-10-14: "all tables should be populated and
+ ** the qbit should be fully using them"). For every web_performance_* table:
+ ** - runtime code (not just its metadata or the home widgets) reads or writes
+ **   it, so no table exists only on paper;
+ ** - it is either seeded by the shipped changelog or listed in STARTS_EMPTY
+ **   with why it starts empty;
+ ** - it has bundle help that says what it is for and how its rows get created.
+ ** A new table fails this test until it is wired, seeded or explained.
+ *******************************************************************************/
+class WebPerfTablesGuardrailTest
+{
+   private static final Path MAIN = Path.of("src", "main", "java", "com", "kingsrook", "qbits", "webperf");
+
+   /*******************************************************************************
+    ** Tables the shipped changelog seeds (WebPerfDefaultsChangelogTest proves it).
+    *******************************************************************************/
+   static final Map<String, String> SEEDED = Map.of(
+      "budget", "changeset 10: Google's good thresholds and the Lighthouse category bands");
+
+   /*******************************************************************************
+    ** Tables that legitimately start empty, and where their rows come from.
+    *******************************************************************************/
+   static final Map<String, String> STARTS_EMPTY = Map.of(
+      "target", "synced from the sitemap (Sync pages from sitemap) or added by hand; the page list is host content",
+      "lab_run", "runtime: every queued or finished Lighthouse test",
+      "lab_finding", "runtime: findings of each successful lab run",
+      "lab_report", "runtime: the full response of each successful lab run",
+      "field_snapshot", "runtime: Google field data checks",
+      "regression", "runtime: a budget is broken or a score drifts (empty is healthy)",
+      "change_event", "runtime: a deploy or content change is detected (needs a probe or source)",
+      "comparison", "runtime: Compare before and after on a change event");
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   @AfterEach
+   void tearDown()
+   {
+      QContext.clear();
+   }
+
+
+
+   /*******************************************************************************
+    ** Every table is used by runtime code.
+    *******************************************************************************/
+   @Test
+   void everyTableIsUsedAtRuntime() throws Exception
+   {
+      List<String> runtimeSource;
+      try(Stream<Path> files = Files.walk(MAIN))
+      {
+         runtimeSource = files.filter(p -> p.toString().endsWith(".java"))
+            .filter(p -> !p.startsWith(MAIN.resolve("metadata")) && !p.startsWith(MAIN.resolve("widgets")))
+            .map(WebPerfTablesGuardrailTest::read).toList();
+      }
+      for(String base : WebPerfSchema.TABLES)
+      {
+         Pattern use = Pattern.compile("\\{" + base + "\\}|WebPerfSchema\\." + base.toUpperCase(Locale.ROOT) + "\\b");
+         assertThat(runtimeSource.stream().anyMatch(source -> use.matcher(source).find()))
+            .as(base + " must be read or written by runtime code, not only declared").isTrue();
+      }
+   }
+
+
+
+   /*******************************************************************************
+    ** Every table is seeded or explained.
+    *******************************************************************************/
+   @Test
+   void everyTableIsSeededOrExplained() throws Exception
+   {
+      String changelog = Files.readString(Path.of("src", "main", "resources", WebPerfSchema.CHANGELOG), StandardCharsets.UTF_8);
+      for(String base : WebPerfSchema.TABLES)
+      {
+         boolean seeded = SEEDED.containsKey(base);
+         assertThat(seeded || STARTS_EMPTY.containsKey(base)).as(base + " must be seeded or listed in STARTS_EMPTY").isTrue();
+         assertThat(seeded && STARTS_EMPTY.containsKey(base)).as(base + " cannot be both").isFalse();
+         assertThat(changelog.contains("INSERT INTO ${web_performance.tablePrefix}" + base + " ")).as(base + " seeded iff the changelog inserts it").isEqualTo(seeded);
+      }
+      assertThat(WebPerfSchema.TABLES).containsAll(SEEDED.keySet()).containsAll(STARTS_EMPTY.keySet());
+   }
+
+
+
+   /*******************************************************************************
+    ** Every table and the app have bundle help that says how rows get created.
+    *******************************************************************************/
+   @Test
+   void everyTableHasEmptyStateHelp() throws Exception
+   {
+      QInstance qInstance = WebPerfTestSupport.newInstance();
+      new WebPerfQBitProducer().withQBitConfig(WebPerfTestSupport.config()).produce(qInstance);
+      WebPerfQBitConfig config = WebPerfTestSupport.config();
+
+      assertThat(WebPerfHelp.entries()).allSatisfy((key, entry) -> assertThat(entry.path("summary").asText().length()).as(key).isLessThanOrEqualTo(120));
+      for(String base : WebPerfSchema.TABLES)
+      {
+         QTableMetaData     table = qInstance.getTable(config.applyPrefix(base));
+         List<QHelpContent> help  = table.getHelpContent() == null ? null : table.getHelpContent().get(WebPerfHelp.SLOT);
+         assertThat(help).as(base + " table help").isNotEmpty();
+         assertThat(help.get(0).getContent()).as(base + " help says how rows get created").contains("How rows get created");
+      }
+      assertThat(qInstance.getApp(config.getAppName()).getHelpContent()).containsKey(WebPerfHelp.SLOT);
+      assertThat(WebPerfHelp.entries()).hasSize(WebPerfSchema.TABLES.size() + 1);
+   }
+
+
+
+   /*******************************************************************************
+    ** Help a host already wrote is kept; the bundle only fills empty slots.
+    *******************************************************************************/
+   @Test
+   void hostHelpIsNotOverridden() throws Exception
+   {
+      QInstance qInstance = WebPerfTestSupport.newInstance();
+      WebPerfQBitConfig config = WebPerfTestSupport.config();
+      new WebPerfQBitProducer().withQBitConfig(config).produce(qInstance);
+      QTableMetaData table = qInstance.getTable(config.applyPrefix(WebPerfSchema.BUDGET));
+      table.getHelpContent().remove(WebPerfHelp.SLOT);
+      table.withHelpContent(WebPerfHelp.SLOT, new QHelpContent("host words"));
+
+      WebPerfHelp.apply(qInstance, config);
+
+      assertThat(table.getHelpContent().get(WebPerfHelp.SLOT).get(0).getContent()).isEqualTo("host words");
+   }
+
+
+
+   /*******************************************************************************
+    **
+    *******************************************************************************/
+   private static String read(Path path)
+   {
+      try
+      {
+         return (Files.readString(path, StandardCharsets.UTF_8));
+      }
+      catch(Exception e)
+      {
+         throw (new IllegalStateException(e));
+      }
+   }
+
+}

src/test/java/com/kingsrook/qbits/webperf/WebPerfTestSupport.java

@@ -127,9 +127,18 @@ public final class WebPerfTestSupport
 
       List<String>  statements = new ArrayList<>();
       StringBuilder current    = new StringBuilder();
+      boolean       skip       = false;
       for(String line : script.split("\n"))
       {
-         if(line.trim().startsWith("--"))
+         //////////////////////////////////////////////////////////////////////
+         // a changeset limited to other databases (the dbms:postgresql      //
+         // orphan cleanup) is skipped, as Liquibase skips it on H2          //
+         //////////////////////////////////////////////////////////////////////
+         if(line.startsWith("-- changeset "))
+         {
+            skip = line.contains(" dbms:") && !line.contains("h2");
+         }
+         if(skip || line.trim().startsWith("--"))
          {
             continue;
          }