diff --git a/CHANGELOG.md b/CHANGELOG.md index e46a115..cf7b4b3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,12 +5,17 @@ All notable changes to this project are documented here. The format follows Keep ## [0.1.1] - Unreleased ### Added +- Central `cdn.provider.` dependency health checks for enabled providers, every 900 seconds with a 20000 ms timeout, reusing the existing connection test with bounded, non-secret results. - Ship the host-neutral in-app help bundle with QQQ registration, live references, and coverage checks for all CDN metadata and generated purge actions. Hosts can override entries with a HOST bundle. - Origin header check: Check Origin Headers on Hostnames (and a nightly run over hostnames that are not Off) asks the origin for sample pages and writes `cdn_origin_check` rows. The table had no writer before. - Guided empty states: help bundle `help/qbit-cdn.yaml` (app and all nine tables, with "How rows get created") applied by `CdnHelp`, and a Setup card on the CDN home. - "Create default provider" (No CDN), "Create default policies" (two turned-off examples) and "Create default rules" (a tags-only rule per purgeable table) table actions; all insert-only. - `hostnameRole` is documented as a label that changes nothing; cache policies are documented as applying only where the application calls `CdnResponseHeaderPolicy`. +### Removed +- Connection testing from the daily drift step; drift checks and the manual Test Connection action remain. +- The `cdnHealthCard` HTML widget, its renderer and dedicated status summary. + ### Changed - Pin the released QQQ BOM to 4.1.0-RC.7 and replace the temporary help loader with the core bundle registry. - Require released QQQ BOM 4.1.0-RC.5; revalidation hooks default to core Standard Webhooks signing. Per-hook `signatureFormat=LEGACY` preserves the prior header and signature bytes. diff --git a/README.md b/README.md index 223af23..a54585f 100644 --- a/README.md +++ b/README.md @@ -25,12 +25,26 @@ Design: QRun-IO/qqq `docs/specs/2026-10-12-qbit-cdn-design.md`. Purge Everything, Test Connection, Import Edge Stats, Check Config Drift; scheduled: dispatcher, daily stats import, daily drift check. - **Admin -> CDN app:** sections Connection / Settings / Activity / Performance, - a home widget with honest empty states, and a Health status card. + a home widget with honest empty states. - **Providers:** CloudFront (invalidations, connection test, CloudWatch stats, drift against the OpenTofu `expected_config`), Souin (self-hosted, tags), NONE. See `docs/cloudfront-expected-config.md`. - **Permissions:** `cdn.view`, `cdn.purge`, `cdn.purgeAll`, `cdn.admin`. +## Health checks + +Central System Health registers `cdn.provider.` for each enabled +`cdn_provider` row: **DEPENDENCY**, component `cdn`, category **CDN**, every +**900 seconds**, with a **20000 ms** timeout. It reuses the read-only connection +test: failure is DOWN; success without the origin header is DEGRADED; otherwise +UP. Providers without connection-test support report UNKNOWN. Disable a check +centrally with `health_check.enabled`. + +Details contain only provider type, a known provider status (or UNKNOWN), and +whether the origin header is configured. The manual `cdnTestConnection` action +and its `lastHealth*` fields remain; the daily scheduled step checks drift only. +The host must use QQQ **4.1.0-RC.8** or later for the central health SPI. + ## Setup This qbit ships no DDL and seeds nothing from a changelog (a host chooses its own column names). Open the CDN app: the Setup card lists each part with the next step. On a fresh install run **Create default provider** (No CDN: purges are recorded, nothing is sent), **Create default rules** (tags for every purgeable table), and optionally **Create default policies** (two turned-off examples). Add your CDN and hostnames when you have them. Cache policies change a response only where the application calls `CdnResponseHeaderPolicy.decide`. **Check Origin Headers** on a hostname (and the nightly run) asks the origin, the provider's `originHost` when set, for the home page, prefix policy paths and literal tag-rule paths, and records how many have the right Cache-Control. @@ -78,7 +92,7 @@ variable named by the provider setting `expectedConfigEnv`). ## Revalidation hooks -Requires QQQ **4.1.0-RC.7** (the default `qqq-snapshot` profile imports the released BOM; +Requires QQQ **4.1.0-RC.8** (the default `qqq-snapshot` profile imports the released BOM; its historical property name does not select a core SNAPSHOT). New `cdn_revalidation_hook` rows default `signatureFormat` to `STANDARD_WEBHOOKS`; @@ -144,7 +158,7 @@ with purging enabled. Purge From CDN, Purge Everything, and Retry Purge show a Review warning and a recorded-only Result when sending is unavailable. Requests for enabled providers remain visible with `SKIPPED_NONE_PROVIDER` or `SKIPPED_STAGE_OFF`; with no enabled provider, no new request row is created. -The home Connection and Health cards explain the unconfigured state, and purge +The home Connection card explains the unconfigured state, and purge activity labels recorded-only requests when disconnected. Run **Check Origin Headers** on a hostname to populate origin-check results. diff --git a/pom.xml b/pom.xml index 1b446bd..b8d9032 100644 --- a/pom.xml +++ b/pom.xml @@ -194,7 +194,7 @@ true - 4.1.0-RC.7 + 4.1.0-RC.8 diff --git a/src/main/java/com/kingsrook/qbits/cdn/CdnQBitProducer.java b/src/main/java/com/kingsrook/qbits/cdn/CdnQBitProducer.java index edcf34f..9d8a32c 100644 --- a/src/main/java/com/kingsrook/qbits/cdn/CdnQBitProducer.java +++ b/src/main/java/com/kingsrook/qbits/cdn/CdnQBitProducer.java @@ -20,10 +20,13 @@ package com.kingsrook.qbits.cdn; import java.util.ArrayList; import java.util.List; import com.kingsrook.qbits.cdn.app.CdnWidgets; +import com.kingsrook.qbits.cdn.health.CdnProviderHealthChecks; import com.kingsrook.qbits.cdn.model.CdnTables; import com.kingsrook.qbits.cdn.processes.CdnProcesses; import com.kingsrook.qbits.cdn.purge.CdnRecordChangeListener; import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.health.HealthCheckSupplierMetaData; +import com.kingsrook.qqq.backend.core.health.HealthChecksMetaData; import com.kingsrook.qqq.backend.core.model.metadata.MetaDataProducerMultiOutput; import com.kingsrook.qqq.backend.core.model.metadata.QInstance; import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference; @@ -162,6 +165,8 @@ public class CdnQBitProducer implements QBitMetaDataProducer } output.add(CdnWidgets.app(qBitConfig, tables)); output.addSelfToInstance(qInstance); + HealthChecksMetaData.ofOrWithNew(qInstance).withSupplier(new HealthCheckSupplierMetaData() + .withNamePrefix("cdn.provider").withComponent("cdn").withSupplier(new QCodeReference(CdnProviderHealthChecks.class))); com.kingsrook.qbits.cdn.model.CdnHelp.register(qInstance, qBitConfig); if(!qBitConfig.getPurgeableTables().isEmpty()) diff --git a/src/main/java/com/kingsrook/qbits/cdn/app/CdnHomeData.java b/src/main/java/com/kingsrook/qbits/cdn/app/CdnHomeData.java index 8353365..377c472 100644 --- a/src/main/java/com/kingsrook/qbits/cdn/app/CdnHomeData.java +++ b/src/main/java/com/kingsrook/qbits/cdn/app/CdnHomeData.java @@ -252,24 +252,6 @@ public record CdnHomeData(List providers, List hostn - /******************************************************************************* - ** Overall state for the Health card: NOT_CONNECTED, ATTENTION or OK. - *******************************************************************************/ - public String overallState() - { - if(!isConnected()) - { - return ("NOT_CONNECTED"); - } - boolean attention = dead24h > 0 || !openDriftByArea.isEmpty() - || providers.stream().anyMatch(p -> p.isConnected() && ("FAILED".equals(p.lastHealthStatus()) - || (p.credentialAgeDays() != null && p.credentialAgeDays() >= ROTATION_WARNING_DAYS))) - || budgets.stream().anyMatch(b -> b.billed() > b.soft()); - return (attention ? "ATTENTION" : "OK"); - } - - - /******************************************************************************* ** Paths billed this month for a provider. *******************************************************************************/ diff --git a/src/main/java/com/kingsrook/qbits/cdn/app/CdnHomeView.java b/src/main/java/com/kingsrook/qbits/cdn/app/CdnHomeView.java index 6da31c2..d19e5c3 100644 --- a/src/main/java/com/kingsrook/qbits/cdn/app/CdnHomeView.java +++ b/src/main/java/com/kingsrook/qbits/cdn/app/CdnHomeView.java @@ -72,26 +72,6 @@ public final class CdnHomeView - /******************************************************************************* - ** The Health status card. - *******************************************************************************/ - public static String renderHealthCard(CdnHomeData data, String appPath) - { - String state = data.overallState(); - String text = "All CDN checks are OK."; - if("NOT_CONNECTED".equals(state)) - { - text = "No CDN provider configured: purges are recorded only."; - } - else if("ATTENTION".equals(state)) - { - text = "Needs attention: see the CDN home."; - } - return ("
CDN

" + esc(text) + "

Open the CDN home
"); - } - - - /******************************************************************************* ** Connection card. *******************************************************************************/ diff --git a/src/main/java/com/kingsrook/qbits/cdn/app/CdnWidgets.java b/src/main/java/com/kingsrook/qbits/cdn/app/CdnWidgets.java index cdd117a..ae75d8c 100644 --- a/src/main/java/com/kingsrook/qbits/cdn/app/CdnWidgets.java +++ b/src/main/java/com/kingsrook/qbits/cdn/app/CdnWidgets.java @@ -37,13 +37,12 @@ import com.kingsrook.qqq.backend.core.model.metadata.tables.QTableMetaData; /******************************************************************************* ** The Admin -> CDN app: a home widget (the single owner of CDN data), four - ** sections of standard tables, and a status card for the app's Health page. + ** sections of standard tables. ** Processes are never added to the nav. *******************************************************************************/ public final class CdnWidgets { - public static final String HOME_WIDGET = "cdnHome"; - public static final String HEALTH_CARD_WIDGET = "cdnHealthCard"; + public static final String HOME_WIDGET = "cdnHome"; @@ -63,9 +62,7 @@ public final class CdnWidgets { return (List.of( new QWidgetMetaData().withName(HOME_WIDGET).withLabel("CDN").withType(WidgetType.HTML.getType()) - .withCodeReference(new QCodeReference(HomeRenderer.class)).withGridColumns(12).withIsCard(false).withPermissionRules(CdnTables.permissionRules()), - new QWidgetMetaData().withName(HEALTH_CARD_WIDGET).withLabel("CDN").withType(WidgetType.HTML.getType()) - .withCodeReference(new QCodeReference(HealthCardRenderer.class)).withGridColumns(12).withIsCard(true).withPermissionRules(CdnTables.permissionRules()))); + .withCodeReference(new QCodeReference(HomeRenderer.class)).withGridColumns(12).withIsCard(false).withPermissionRules(CdnTables.permissionRules()))); } @@ -132,21 +129,4 @@ public final class CdnWidgets return (new RenderWidgetOutput(new RawHTML(null, CdnHomeView.render(CdnHomeData.load(), tablePathPrefix())))); } } - - - - /******************************************************************************* - ** Health status card renderer. - *******************************************************************************/ - public static class HealthCardRenderer extends AbstractWidgetRenderer - { - /******************************************************************************* - ** Render. - *******************************************************************************/ - @Override - public RenderWidgetOutput render(RenderWidgetInput input) throws QException - { - return (new RenderWidgetOutput(new RawHTML("CDN", CdnHomeView.renderHealthCard(CdnHomeData.load(), appPath())))); - } - } } diff --git a/src/main/java/com/kingsrook/qbits/cdn/health/CdnProviderHealthChecks.java b/src/main/java/com/kingsrook/qbits/cdn/health/CdnProviderHealthChecks.java new file mode 100644 index 0000000..afea743 --- /dev/null +++ b/src/main/java/com/kingsrook/qbits/cdn/health/CdnProviderHealthChecks.java @@ -0,0 +1,149 @@ +/* + * Copyright 2026 Kingsrook, LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.kingsrook.qbits.cdn.health; + + +import java.io.Serializable; +import java.time.Duration; +import java.time.Instant; +import java.util.ArrayList; +import java.util.List; +import java.util.Locale; +import java.util.concurrent.ExecutionException; +import java.util.concurrent.FutureTask; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.TimeoutException; +import com.kingsrook.qbits.cdn.CdnRuntime; +import com.kingsrook.qbits.cdn.model.CdnTables; +import com.kingsrook.qbits.cdn.purge.CdnPurgeDispatcher; +import com.kingsrook.qbits.cdn.spi.CdnOperations; +import com.kingsrook.qbits.cdn.spi.CdnProviderConfig; +import com.kingsrook.qbits.cdn.spi.CdnProviderInterface; +import com.kingsrook.qqq.backend.core.actions.tables.GetAction; +import com.kingsrook.qqq.backend.core.actions.tables.QueryAction; +import com.kingsrook.qqq.backend.core.exceptions.QException; +import com.kingsrook.qqq.backend.core.health.HealthCategory; +import com.kingsrook.qqq.backend.core.health.HealthCheckInput; +import com.kingsrook.qqq.backend.core.health.HealthCheckKind; +import com.kingsrook.qqq.backend.core.health.HealthCheckMetaData; +import com.kingsrook.qqq.backend.core.health.HealthCheckOutcome; +import com.kingsrook.qqq.backend.core.health.HealthCheckSupplier; +import com.kingsrook.qqq.backend.core.health.HealthDetails; +import com.kingsrook.qqq.backend.core.model.actions.tables.get.GetInput; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter; +import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput; +import com.kingsrook.qqq.backend.core.model.data.QRecord; +import com.kingsrook.qqq.backend.core.model.metadata.QInstance; + + +/******************************************************************************* + ** Central health checks reuse the read-only provider connection test. + *******************************************************************************/ +public class CdnProviderHealthChecks implements HealthCheckSupplier +{ + /******************************************************************************* + ** Resolve the enabled provider rows each time the registry refreshes. + *******************************************************************************/ + @Override + public List supply(QInstance qInstance) throws QException + { + List checks = new ArrayList<>(); + for(QRecord provider : new QueryAction().execute(new QueryInput(CdnTables.PROVIDER) + .withFilter(new QQueryFilter(new QFilterCriteria("enabled", QCriteriaOperator.EQUALS, true)))).getRecords()) + { + Serializable id = provider.getValue("id"); + String name = "cdn.provider." + id.toString().toLowerCase(Locale.ROOT); + checks.add(new HealthCheckMetaData().withName(name).withLabel(name).withComponent("cdn") + .withKind(HealthCheckKind.DEPENDENCY).withCategory(HealthCategory.CDN) + .withIntervalSeconds(900).withTimeoutMs(20000).withCheckInstance(input -> run(id, input))); + } + return (checks); + } + + + + /******************************************************************************* + ** Reload configuration so disabled or deleted providers are not probed. + ** Never copy provider messages, addresses or exception text into outcomes. + *******************************************************************************/ + private static HealthCheckOutcome run(Serializable id, HealthCheckInput input) throws QException + { + if(!Instant.now().isBefore(input.getDeadline())) + { + return (HealthCheckOutcome.down("CDN connection test timed out")); + } + QRecord provider = new GetAction().executeForRecord(new GetInput(CdnTables.PROVIDER).withPrimaryKey(id)); + if(provider == null || !Boolean.TRUE.equals(provider.getValueBoolean("enabled"))) + { + return (HealthCheckOutcome.unknown("CDN provider is disabled or removed")); + } + CdnProviderConfig config = CdnPurgeDispatcher.configOf(provider); + CdnProviderInterface adapter = CdnRuntime.provider(config.type()); + if(!(adapter instanceof CdnOperations operations)) + { + return (HealthCheckOutcome.unknown("CDN provider does not support connection tests")); + } + return (probe(operations, config, input)); + } + + + + /******************************************************************************* + ** The existing SPI has no timeout parameter. Bound its wait by the remaining + ** deadline and interrupt the probe on timeout or runner cancellation. + *******************************************************************************/ + private static HealthCheckOutcome probe(CdnOperations operations, CdnProviderConfig config, HealthCheckInput input) + { + long remaining = Duration.between(Instant.now(), input.getDeadline()).toNanos(); + if(remaining <= 0) + { + return (HealthCheckOutcome.down("CDN connection test timed out")); + } + FutureTask task = new FutureTask<>(() -> operations.connectionTest(config)); + Thread.startVirtualThread(task); + try + { + CdnOperations.ConnectionTest test = task.get(remaining, TimeUnit.NANOSECONDS); + HealthCheckOutcome outcome = !test.ok() ? HealthCheckOutcome.down("CDN connection test failed") + : !test.originHeaderConfigured() ? HealthCheckOutcome.degraded("CDN origin header is not configured") + : HealthCheckOutcome.up("CDN connection test passed"); + // Only known provider states are safe; arbitrary status text may contain secrets. + String status = "Deployed".equals(test.status()) ? "Deployed" : "InProgress".equals(test.status()) ? "InProgress" : "UNKNOWN"; + return (outcome.withDetails(new HealthDetails().withValue("providerType", config.type().name()) + .withValue("status", status).withValue("originHeaderConfigured", test.originHeaderConfigured()))); + } + catch(TimeoutException e) + { + return (HealthCheckOutcome.down("CDN connection test timed out")); + } + catch(InterruptedException e) + { + Thread.currentThread().interrupt(); + return (HealthCheckOutcome.down("CDN connection test interrupted")); + } + catch(ExecutionException e) + { + return (HealthCheckOutcome.down("CDN connection test failed")); + } + finally + { + task.cancel(true); + } + } +} diff --git a/src/main/java/com/kingsrook/qbits/cdn/processes/CdnProviderProcesses.java b/src/main/java/com/kingsrook/qbits/cdn/processes/CdnProviderProcesses.java index 6e4da39..f9389b8 100644 --- a/src/main/java/com/kingsrook/qbits/cdn/processes/CdnProviderProcesses.java +++ b/src/main/java/com/kingsrook/qbits/cdn/processes/CdnProviderProcesses.java @@ -525,7 +525,7 @@ public final class CdnProviderProcesses /******************************************************************************* - ** Scheduled: daily drift check (and connection test) for every operable + ** Scheduled: daily drift check for every operable ** provider. *******************************************************************************/ public static class ScheduledDriftStep implements BackendStep @@ -538,7 +538,6 @@ public final class CdnProviderProcesses { for(QRecord provider : operableProviders()) { - testAndRecord(provider); driftAndRecord(provider); } } diff --git a/src/main/resources/help/qbit-cdn.yaml b/src/main/resources/help/qbit-cdn.yaml index 912293b..5ef8d3f 100644 --- a/src/main/resources/help/qbit-cdn.yaml +++ b/src/main/resources/help/qbit-cdn.yaml @@ -324,7 +324,7 @@ entries: content: | **{{label:cdnCheckDrift}}** (and a daily job when a scheduler is configured) reads the live CDN settings and compares them with the expected setup defined by the infrastructure code. Each difference becomes a finding with what was expected and what was found. - A finding stays open until a later check no longer sees it, then it is marked resolved. Open findings turn the CDN status on the Health screen to needing attention. You cannot edit findings; fix the CDN or the expected setup instead. + A finding stays open until a later check no longer sees it, then it is marked resolved. You cannot edit findings; fix the CDN or the expected setup instead. **How rows get created:** Check Config Drift on a provider writes it, and, when a scheduler is configured, a nightly run does the same. It needs the expected configuration in the environment variable the provider names (CDN_EXPECTED_CONFIG by default); a finding is resolved when it no longer appears. - key: table:cdn_drift_finding;field:cdnProviderId @@ -386,18 +386,14 @@ entries: content: | When your application configures a scheduler, this job imports the last three complete UTC days for enabled providers that support statistics. Use **{{label:cdnImportEdgeStats}}** to refresh on demand. - key: process:cdnScheduledCheckDrift;slot:help - summary: Test enabled CDN providers and check their configuration on the configured schedule. + summary: Check enabled CDN providers for configuration drift on the configured schedule. content: | - When your application configures a scheduler, this job updates connection health and drift findings for supported providers. Use **{{label:cdnCheckDrift}}** to check drift on demand. + When your application configures a scheduler, this job updates drift findings for supported providers. Connection health is checked separately by central System Health. Use **{{label:cdnCheckDrift}}** to check drift on demand. - key: widget:cdnHome;slot:label summary: 'The CDN at a glance: provider, hostnames, traffic, purge activity, invalidation budget, origin checks and drift.' content: | Each card says plainly when there is nothing yet, for example when no CDN is connected or no statistics have been imported. Traffic figures cover the last 7 and 30 days; purge failures cover 24 hours and 7 days. -- key: widget:cdnHealthCard;slot:label - summary: 'A one-line CDN status for the Health screen: not connected, needs attention, or all checks OK.' - content: | - It needs attention when a purge went dead in the last 24 hours, a drift finding is open, the last connection test failed, the credentials are 75 or more days old, or this month's billed paths passed the soft limit. - key: possibleValueSource:cdn_provider_type;slot:help summary: The CDN services a provider can be. content: | diff --git a/src/test/java/com/kingsrook/qbits/cdn/CdnQBitProducerTest.java b/src/test/java/com/kingsrook/qbits/cdn/CdnQBitProducerTest.java index 0fb06c7..5dc3ff6 100644 --- a/src/test/java/com/kingsrook/qbits/cdn/CdnQBitProducerTest.java +++ b/src/test/java/com/kingsrook/qbits/cdn/CdnQBitProducerTest.java @@ -52,6 +52,7 @@ class CdnQBitProducerTest assertThat(instance.getProcess(CdnProcesses.PURGE_HOSTNAME)).isNotNull(); assertThat(instance.getProcess(CdnProcesses.DISPATCH_PURGES).getIsHidden()).isTrue(); assertThat(instance.getWidget(CdnWidgets.HOME_WIDGET)).isNotNull(); + assertThat(instance.getWidget("cdnHealthCard")).isNull(); assertThat(instance.getApp("cdnAdmin").getSections()).extracting(section -> section.getLabel()) .containsExactly("Connection", "Settings", "Activity", "Performance"); assertThat(instance.getRecordChangeListeners()).isNotEmpty(); diff --git a/src/test/java/com/kingsrook/qbits/cdn/app/CdnHomeTest.java b/src/test/java/com/kingsrook/qbits/cdn/app/CdnHomeTest.java index 929ecec..7aa2552 100644 --- a/src/test/java/com/kingsrook/qbits/cdn/app/CdnHomeTest.java +++ b/src/test/java/com/kingsrook/qbits/cdn/app/CdnHomeTest.java @@ -34,7 +34,7 @@ import static org.junit.jupiter.api.Assertions.assertEquals; /******************************************************************************* - ** The CDN home and Health card: honest empty states, then real data. + ** The CDN home: honest empty states, then real data. *******************************************************************************/ class CdnHomeTest { @@ -61,7 +61,6 @@ class CdnHomeTest assertThat(html).contains("No CDN provider configured: purges are recorded only.").contains("No edge statistics yet.").contains("No purges recorded yet.") .contains("No path-billed provider connected yet.").contains("No origin check yet.").contains("Nothing to compare yet.") .contains("minmax(min(100%,22rem),1fr)"); - assertThat(render(CdnWidgets.HEALTH_CARD_WIDGET)).contains("No CDN provider configured: purges are recorded only.").contains("href=\"/cdnAdmin\""); } @@ -91,7 +90,6 @@ class CdnHomeTest CdnTestSupport.insert(CdnTables.DRIFT_FINDING, new QRecord().withValue("cdnProviderId", provider).withValue("area", "BEHAVIOR").withValue("checkedDate", CdnTestSupport.NOW)); CdnHomeData data = CdnHomeData.load(); - assertEquals("ATTENTION", data.overallState()); assertEquals(1, data.dead24h()); assertEquals(4, data.budgets().get(0).billed()); assertEquals(age, data.providers().get(0).credentialAgeDays()); @@ -100,7 +98,6 @@ class CdnHomeTest assertThat(html).contains("d1.cloudfront.net").contains(age + " days (rotate every 90) - rotation due").contains("1,500").contains("91.5%") .contains(" adapter); + } + + + + /******************************************************************************* + ** Restore process-wide test state. + *******************************************************************************/ + @AfterEach + void tearDown() + { + CdnRuntime.setEnvironment(null); + CdnRuntime.setClock(null); + Thread.interrupted(); + QContext.clear(); + } + + + + /******************************************************************************* + ** The producer registers a dynamic dependency supplier with the spec defaults. + *******************************************************************************/ + @Test + void registrationAndRefresh() throws Exception + { + var registry = HealthChecksMetaData.ofOrWithNew(instance); + assertThat(registry.getSuppliers()).singleElement().satisfies(supplier -> + { + assertThat(supplier.getNamePrefix()).isEqualTo("cdn.provider"); + assertThat(supplier.getComponent()).isEqualTo("cdn"); + }); + CdnTestSupport.insert(CdnTables.PROVIDER, new QRecord().withValue("name", "disabled").withValue("providerType", "CLOUDFRONT").withValue("enabled", false)); + // Match the runner's automated context rather than relying on CDN UI permissions. + QContext.init(instance, new QSession().withUser(new QUser().withIdReference("health_runner").withFullName("health_runner")) + .withSecurityKeyValues(Map.of()).withPermissions("health.read", "health.write")); + assertThat(registry.resolveAll(instance)).singleElement().satisfies(check -> + { + assertThat(check.getName()).isEqualTo("cdn.provider." + providerId).matches("[a-z0-9]+([._-][a-z0-9]+)*"); + assertThat(check.getComponent()).isEqualTo("cdn"); + assertThat(check.getKind()).isEqualTo(HealthCheckKind.DEPENDENCY); + assertThat(check.getCategory()).isEqualTo(HealthCategory.CDN); + assertThat(check.getIntervalSeconds()).isEqualTo(900); + assertThat(check.getTimeoutMs()).isEqualTo(20000); + assertThat(check.getIncludeInReadiness()).isFalse(); + }); + assertThat(check().getCheckInstance().run(input()).getStatus()).isEqualTo(HealthStatus.UP); + update(new QRecord().withValue("enabled", false)); + assertThat(registry.resolveAll(instance)).isEmpty(); + update(new QRecord().withValue("enabled", true)); + assertThat(registry.resolveAll(instance)).hasSize(1); + } + + + + /******************************************************************************* + ** Map success, missing origin protection and failure without writing lastHealth. + *******************************************************************************/ + @Test + void connectionStates() throws Exception + { + var check = check().getCheckInstance(); + HealthCheckOutcome up = check.run(input()); + assertThat(up.getStatus()).isEqualTo(HealthStatus.UP); + assertThat(up.getDetails().getValues()).containsExactlyInAnyOrderEntriesOf(Map.of( + "providerType", "CLOUDFRONT", "status", "Deployed", "originHeaderConfigured", true)); + doReturn(new CdnOperations.ConnectionTest(true, null, null, null, false, "InProgress", "unused")).when(adapter).connectionTest(any()); + HealthCheckOutcome degraded = check.run(input()); + assertThat(degraded.getStatus()).isEqualTo(HealthStatus.DEGRADED); + assertThat(degraded.getDetails().getValues()).containsEntry("originHeaderConfigured", false).containsEntry("status", "InProgress"); + doReturn(new CdnOperations.ConnectionTest(false, null, null, null, false, null, "unused")).when(adapter).connectionTest(any()); + assertThat(check.run(input()).getStatus()).isEqualTo(HealthStatus.DOWN); + QRecord row = new GetAction().executeForRecord(new GetInput(CdnTables.PROVIDER).withPrimaryKey(providerId)); + assertThat(row.getValue("lastHealthCheckDate")).isNull(); + assertThat(row.getValue("lastHealthStatus")).isNull(); + } + + + + /******************************************************************************* + ** A previously resolved check reloads config and skips disabled/deleted rows. + *******************************************************************************/ + @Test + void reloadsProviderAndHandlesUnsupported() throws Exception + { + var check = check().getCheckInstance(); + update(new QRecord().withValue("providerType", "NONE")); + assertThat(check.run(input()).getStatus()).isEqualTo(HealthStatus.UNKNOWN); + update(new QRecord().withValue("enabled", false)); + assertThat(check.run(input()).getStatus()).isEqualTo(HealthStatus.UNKNOWN); + new DeleteAction().execute(new DeleteInput(CdnTables.PROVIDER).withPrimaryKeys(List.of(providerId))); + assertThat(check.run(input()).getStatus()).isEqualTo(HealthStatus.UNKNOWN); + verify(adapter, never()).connectionTest(any()); + } + + + + /******************************************************************************* + ** Poison credentials, every free-form probe field and thrown error messages. + ** HealthRunner.sanitize is package-private, so assert the raw boundary here. + *******************************************************************************/ + @Test + void noSecretsInOutcomes() throws Exception + { + update(new QRecord().withValue("secretRef", "TEST_CDN").withValue("name", SECRET) + .withValue("zoneOrDistributionId", SECRET).withValue("settingsJson", "{\"originVerifyHeader\":\"" + SECRET + "\"}")); + CdnRuntime.setEnvironment(name -> SECRET); + for(HealthStatus expected : List.of(HealthStatus.UP, HealthStatus.DEGRADED, HealthStatus.DOWN)) + { + doAnswer(invocation -> + { + CdnProviderConfig config = invocation.getArgument(0); + assertThat(CdnRuntime.environment(config.secretRef() + "_ACCESS_KEY_ID")).isEqualTo(SECRET); + assertThat(CdnRuntime.environment(config.secretRef() + "_SECRET_ACCESS_KEY")).isEqualTo(SECRET); + return (new CdnOperations.ConnectionTest(expected != HealthStatus.DOWN, SECRET, SECRET, 443, + expected == HealthStatus.UP, SECRET, SECRET)); + }).when(adapter).connectionTest(any()); + HealthCheckOutcome outcome = check().getCheckInstance().run(input()); + assertThat(outcome.getStatus()).isEqualTo(expected); + assertClean(outcome); + } + doThrow(new IllegalStateException(SECRET)).when(adapter).connectionTest(any()); + HealthCheckOutcome failed = check().getCheckInstance().run(input()); + assertThat(failed.getStatus()).isEqualTo(HealthStatus.DOWN); + assertClean(failed); + update(new QRecord().withValue("providerType", "NONE")); + assertClean(check().getCheckInstance().run(input())); + } + + + + /******************************************************************************* + ** An expired deadline never starts the existing probe. + *******************************************************************************/ + @Test + void expiredDeadline() throws Exception + { + HealthCheckOutcome outcome = check().getCheckInstance().run(input().withDeadline(Instant.now().minusSeconds(1))); + assertThat(outcome.getStatus()).isEqualTo(HealthStatus.DOWN); + assertThat(outcome.getMessage()).contains("timed out"); + verify(adapter, never()).connectionTest(any()); + } + + + + /******************************************************************************* + ** A slow adapter is cancelled at the deadline, without a blocking executor close. + *******************************************************************************/ + @Test + void deadlineCancelsProbe() throws Exception + { + CountDownLatch cancelled = new CountDownLatch(1); + doAnswer(invocation -> + { + try + { + new CountDownLatch(1).await(); + return (null); + } + finally + { + cancelled.countDown(); + } + }).when(adapter).connectionTest(any()); + HealthCheckOutcome outcome = assertTimeoutPreemptively(Duration.ofSeconds(3), () -> + { + QContext.init(instance, new QSession()); + try + { + return (check().getCheckInstance().run(input().withDeadline(Instant.now().plusMillis(200)))); + } + finally + { + QContext.clear(); + } + }); + assertThat(outcome.getStatus()).isEqualTo(HealthStatus.DOWN); + assertThat(outcome.getMessage()).contains("timed out"); + assertThat(cancelled.await(2, TimeUnit.SECONDS)).isTrue(); + } + + + + /******************************************************************************* + ** Runner cancellation preserves the caller's interrupt status. + *******************************************************************************/ + @Test + void interruptedProbe() throws Exception + { + doAnswer(invocation -> + { + new CountDownLatch(1).await(); + return (null); + }).when(adapter).connectionTest(any()); + var check = check().getCheckInstance(); + Thread.currentThread().interrupt(); + try + { + HealthCheckOutcome outcome = check.run(input()); + assertThat(outcome.getStatus()).isEqualTo(HealthStatus.DOWN); + assertThat(outcome.getMessage()).contains("interrupted"); + assertThat(Thread.currentThread().isInterrupted()).isTrue(); + } + finally + { + Thread.interrupted(); + } + } + + + + /******************************************************************************* + ** Resolve through the actual registration, including QCodeReference loading. + *******************************************************************************/ + private HealthCheckMetaData check() throws Exception + { + return (HealthChecksMetaData.ofOrWithNew(instance).resolveAll(instance).getFirst()); + } + + + + /******************************************************************************* + ** Fresh deadline for each call. + *******************************************************************************/ + private HealthCheckInput input() + { + return (new HealthCheckInput().withQInstance(instance).withDeadline(Instant.now().plusSeconds(5))); + } + + + + /******************************************************************************* + ** Modify the real memory row to exercise fresh reads. + *******************************************************************************/ + private void update(QRecord values) throws Exception + { + new UpdateAction().execute(new UpdateInput(CdnTables.PROVIDER).withRecord(values.withValue("id", providerId))); + } + + + + /******************************************************************************* + ** No provider-controlled text can reach the central health boundary. + *******************************************************************************/ + private static void assertClean(HealthCheckOutcome outcome) + { + assertThat(outcome.getMessage()).doesNotContain(SECRET, "\n", "\r"); + assertThat(outcome.getDetails().toJson()).doesNotContain(SECRET, "TEST_CDN", "origin.example.com", "d1.cloudfront.net"); + } +} diff --git a/src/test/java/com/kingsrook/qbits/cdn/processes/CdnProviderProcessesTest.java b/src/test/java/com/kingsrook/qbits/cdn/processes/CdnProviderProcessesTest.java index c7d7df4..837b389 100644 --- a/src/test/java/com/kingsrook/qbits/cdn/processes/CdnProviderProcessesTest.java +++ b/src/test/java/com/kingsrook/qbits/cdn/processes/CdnProviderProcessesTest.java @@ -199,9 +199,14 @@ class CdnProviderProcessesTest @Test void scheduledSteps() throws Exception { + FakePathProvider adapter = org.mockito.Mockito.spy(new FakePathProvider()); + CdnRuntime.registerProvider(CdnProviderType.CLOUDFRONT, () -> adapter); + FakePathProvider.drift = List.of(new CdnOperations.DriftItem("origin.header", "present", "missing")); new CdnProviderProcesses.ScheduledStatsStep().run(null, null); new CdnProviderProcesses.ScheduledDriftStep().run(null, null); assertThat(new QueryAction().execute(new QueryInput(CdnTables.EDGE_STAT_DAILY)).getRecords()).hasSize(2); - assertEquals("OK", row().getValueString("lastHealthStatus")); + assertThat(new QueryAction().execute(new QueryInput(CdnTables.DRIFT_FINDING)).getRecords()).hasSize(1); + assertNull(row().getValueString("lastHealthStatus")); + org.mockito.Mockito.verify(adapter, org.mockito.Mockito.never()).connectionTest(org.mockito.ArgumentMatchers.any()); } }