Compare commits
16
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
fe7a8b5089
|
||
|
|
51ca0bb920 | ||
|
|
d0f558dc60
|
||
|
|
945f01371a
|
||
|
|
021e86c202
|
||
|
|
a48933bedc
|
||
|
|
c9acd0921c
|
||
|
|
60f8ced9bb
|
||
|
|
746d2edd17
|
||
|
|
ee6ee59793
|
||
|
|
409e900f10
|
||
|
|
9094e9a760
|
||
|
|
72bc4267e6
|
||
|
|
d6f21f6594 | ||
|
|
00f798668a | ||
|
|
a7b42339b9 |
No files matched your search
@@ -1,12 +1,12 @@
|
||||
version: 2.1
|
||||
setup: true
|
||||
orbs:
|
||||
qqq-orb: kingsrook/qqq-orb@0.7.17
|
||||
qqq-orb: kingsrook/qqq-orb@0.8
|
||||
workflows:
|
||||
setup:
|
||||
jobs:
|
||||
- qqq-orb/generate_pipeline:
|
||||
orb_version: 0.7.17
|
||||
orb_version: '0.8'
|
||||
filters:
|
||||
branches:
|
||||
ignore: automation/qbit-rc-pending
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
name: build
|
||||
on:
|
||||
push:
|
||||
jobs:
|
||||
maven:
|
||||
uses: self:KofTwentyTwo/ci-workflows/.gitea/workflows/maven.yaml@v1
|
||||
secrets: inherit
|
||||
+1
-1
@@ -1,5 +1,5 @@
|
||||
profile: qbit
|
||||
orb_version: 0.7.17
|
||||
orb_version: '0.8'
|
||||
default_branch: develop
|
||||
java_version: '21'
|
||||
browsers: true
|
||||
|
||||
@@ -2,6 +2,10 @@
|
||||
|
||||
## Unreleased / 0.2.0
|
||||
|
||||
- Register CRM health checks (`crm.email`, `crm.sequence_lag`, `crm.lead_score`) with the QQQ 4.1 core health registry (`HealthChecksMetaData`, qqq#1199), replacing the `qqq-middleware-health` indicator stand-in and its optional dependency.
|
||||
|
||||
- Lead scoring keeps its incremental watermark at millisecond precision, matching `modifyDate`, so a contact changed in the same millisecond a pass starts is not skipped.
|
||||
|
||||
- Read the CRM QBit metadata version from the Maven-filtered artifact version so release metadata matches the build.
|
||||
|
||||
- Adopt QQQ 4.1.0-RC.7 native help bundles, replacing the RC.6 stand-in loader with host-neutral CRM help, live references and the core coverage gate.
|
||||
|
||||
@@ -62,9 +62,9 @@ qInstance.setEnforcePermissionsInCoreActions(true);
|
||||
- `withApiName(...)` and `withApiVersion(...)`: publish main and settings tables into an existing host API definition. The host supplies `ApiInstanceMetaData` and includes optional `qqq-middleware-api` at runtime. Main tables declare core search fields.
|
||||
- `withEsbDestinationName(...)`: reserved but rejected on RC.6. Its ESB event factory copies all values and has no classified-field exclusion; CRM does not publish those payloads.
|
||||
- `CrmBulkLoadProfiles.ensure()`: call once in host startup after registering core's `SavedBulkLoadProfileMetaDataProvider`. It seeds three CSV mappings by label without replacing edited profiles. CRM does not register the core profile tables.
|
||||
- `CrmHealthChecks.indicators(monitoringSession)`: register the returned indicators with the host health middleware. Include optional `qqq-middleware-health`; the supplied session determines tenant scope and is restored on worker threads. `run()` uses the caller's context. No check adds permissions or tenant keys.
|
||||
- Health: the producer registers `crm.email`, `crm.sequence_lag` and `crm.lead_score` with the core health registry (`HealthChecksMetaData`). Hosts that enable core health monitoring (`HealthMetaDataProvider`) get them scheduled by the core `HealthRunner`, which runs each check as the core system user. `CrmHealthChecks.run()` evaluates them once in the caller's context. No check adds permissions or tenant keys or returns personal values.
|
||||
|
||||
The shipped help bundle and health adapter are temporary core integration adapters; their removal target is RC.9. See [P4B implementation notes](docs/P4B-IMPLEMENTATION.md) for RC.6 limits.
|
||||
Help and health both use the core registries (`HelpBundles`, `HealthChecksMetaData`). See [P4B implementation notes](docs/P4B-IMPLEMENTATION.md) for RC.6 limits.
|
||||
|
||||
| Scope | Permissions |
|
||||
|---|---|
|
||||
|
||||
+11
-11
@@ -89,8 +89,8 @@ These parts are real and are kept, adapted to 0.2.0:
|
||||
| Imports | `importContacts` | core bulk load + saved bulk-load profiles |
|
||||
| PII | none | `DataClassification.PII` on every personal field; secured-data tiers apply |
|
||||
| Reporting | 14 bespoke widgets | home widgets for the daily view; trend reports via data analysis (`crm_*.analyze`) |
|
||||
| Health | none | `crm` health checks (stand-in until core #1199) |
|
||||
| Help | none | `help/qbit-crm.yaml` bundle (stand-in loader until core #1198) |
|
||||
| Health | none | `crm.*` checks in the core health registry (`HealthChecksMetaData`) |
|
||||
| Help | none | `help/qbit-crm.yaml` bundle registered with core `HelpBundles` |
|
||||
| Tenancy | clientId lock on 3 tables | configurable tenant key type on every table; the host maps Authentik claims to keys |
|
||||
|
||||
## Data model (33 tables: 6 main, 15 settings, 12 sub-tables)
|
||||
@@ -268,13 +268,14 @@ signals are logged.
|
||||
|
||||
### Health
|
||||
|
||||
`CrmHealthChecks` reports three checks:
|
||||
- email transport readiness
|
||||
- sequence lag (the oldest due enrollment more than 30 min late)
|
||||
- the scoring job's last success time
|
||||
`CrmHealthChecks` registers three checks with the core health registry
|
||||
(`HealthChecksMetaData`, component `crm`):
|
||||
- `crm.email`: email transport readiness
|
||||
- `crm.sequence_lag`: the oldest due enrollment more than 30 min late
|
||||
- `crm.lead_score`: the scoring job's last success time
|
||||
|
||||
It is a `STAND-IN(core=QRun-IO/qqq#1199, removeBy=4.1.0-RC.9)` adapter onto the
|
||||
`qqq-middleware-health` `HealthIndicator` until the central `HealthCheck` SPI ships.
|
||||
The core `HealthRunner` schedules them on hosts that enable health monitoring and runs each one
|
||||
as the core system user. A failing check reports UNKNOWN with a fixed message.
|
||||
|
||||
### Help
|
||||
|
||||
@@ -282,9 +283,8 @@ It is a `STAND-IN(core=QRun-IO/qqq#1199, removeBy=4.1.0-RC.9)` adapter onto the
|
||||
- the app
|
||||
- every table, field, action and possible-value source
|
||||
|
||||
`HelpBundleAssertions` checks that it is complete. Until `HelpBundles.register` is in a core RC,
|
||||
`CrmHelpBundleStandIn` (`STAND-IN(core=QRun-IO/qqq#1198, removeBy=4.1.0-RC.9)`) loads the same
|
||||
file into `QHelpContent`.
|
||||
The producer registers it with core `HelpBundles.register`; `HelpBundleAssertions` checks that it
|
||||
is complete.
|
||||
|
||||
### Contingent dropdowns
|
||||
|
||||
|
||||
@@ -22,6 +22,7 @@
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
|
||||
<properties>
|
||||
<qqq.version>4.1.0-RC.26</qqq.version> <!-- feeds qbit-build-parent's qqq-bom-pom import -->
|
||||
<revision>0.2.0-SNAPSHOT</revision>
|
||||
<maven.compiler.source>21</maven.compiler.source>
|
||||
<maven.compiler.target>21</maven.compiler.target>
|
||||
@@ -33,7 +34,7 @@
|
||||
<parent>
|
||||
<groupId>com.kingsrook</groupId>
|
||||
<artifactId>qbit-build-parent</artifactId>
|
||||
<version>2.1.2</version>
|
||||
<version>2.2.1</version>
|
||||
</parent>
|
||||
|
||||
<groupId>com.kingsrook.qbits</groupId>
|
||||
@@ -78,16 +79,11 @@
|
||||
<artifactId>qqq-middleware-api</artifactId>
|
||||
<optional>true</optional>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>com.kingsrook.qqq</groupId>
|
||||
<artifactId>qqq-middleware-health</artifactId>
|
||||
<optional>true</optional>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>com.kingsrook.qbits</groupId>
|
||||
<artifactId>qbit-email</artifactId>
|
||||
<version>0.1.0-RC.3</version>
|
||||
<version>0.1.0-RC.11</version>
|
||||
<optional>true</optional>
|
||||
</dependency>
|
||||
<dependency>
|
||||
@@ -239,7 +235,7 @@
|
||||
<activeByDefault>true</activeByDefault>
|
||||
</activation>
|
||||
<properties>
|
||||
<qqq.snapshot.version>4.1.0-RC.7</qqq.snapshot.version>
|
||||
<qqq.snapshot.version>4.1.0-RC.26</qqq.snapshot.version>
|
||||
</properties>
|
||||
<dependencyManagement>
|
||||
<dependencies>
|
||||
|
||||
@@ -46,6 +46,7 @@ import com.kingsrook.qbits.crm.deals.customizers.DealStageChangeCustomizer;
|
||||
import com.kingsrook.qbits.crm.deals.model.Deal;
|
||||
import com.kingsrook.qbits.crm.deals.model.DealProduct;
|
||||
import com.kingsrook.qbits.crm.email.CrmEmailProcessIdentity;
|
||||
import com.kingsrook.qbits.crm.health.CrmHealthChecks;
|
||||
import com.kingsrook.qbits.crm.scoring.AutoAssignCustomizer;
|
||||
import com.kingsrook.qbits.crm.security.CrmDataPolicy;
|
||||
import com.kingsrook.qbits.crm.security.CrmPermissions;
|
||||
@@ -151,6 +152,7 @@ public class CrmQBitProducer implements QBitMetaDataProducer<CrmQBitConfig>
|
||||
}
|
||||
CrmDataPolicy.apply(metaDataProducerMultiOutput, qInstance, qBitConfig);
|
||||
CrmIntegrations.apply(metaDataProducerMultiOutput, qBitConfig);
|
||||
CrmHealthChecks.register(qInstance, qBitConfig);
|
||||
for(QWidgetMetaData widget : metaDataProducerMultiOutput.getEach(QWidgetMetaData.class))
|
||||
{
|
||||
widget.setName(snakeCase(widget.getName()));
|
||||
|
||||
@@ -23,6 +23,7 @@ package com.kingsrook.qbits.crm.health;
|
||||
|
||||
import java.time.Instant;
|
||||
import java.time.temporal.ChronoUnit;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import com.kingsrook.qbits.crm.CrmQBitConfig;
|
||||
@@ -32,9 +33,15 @@ import com.kingsrook.qbits.crm.email.CrmEmailTransport;
|
||||
import com.kingsrook.qbits.crm.scoring.JobState;
|
||||
import com.kingsrook.qbits.crm.scoring.processes.CalculateLeadScoresProcess;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
|
||||
import com.kingsrook.qqq.backend.core.context.CapturedContext;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.alerts.AlertSeverity;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.core.health.HealthCategory;
|
||||
import com.kingsrook.qqq.backend.core.health.HealthCheck;
|
||||
import com.kingsrook.qqq.backend.core.health.HealthCheckInput;
|
||||
import com.kingsrook.qqq.backend.core.health.HealthCheckKind;
|
||||
import com.kingsrook.qqq.backend.core.health.HealthCheckMetaData;
|
||||
import com.kingsrook.qqq.backend.core.health.HealthCheckOutcome;
|
||||
import com.kingsrook.qqq.backend.core.health.HealthChecksMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QCriteriaOperator;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterCriteria;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QFilterOrderBy;
|
||||
@@ -42,24 +49,25 @@ import com.kingsrook.qqq.backend.core.model.actions.tables.query.QQueryFilter;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
|
||||
import com.kingsrook.qqq.backend.core.model.session.QSession;
|
||||
import com.kingsrook.qqq.backend.core.model.session.QSystemUserSession;
|
||||
import com.kingsrook.qqq.middleware.health.HealthIndicator;
|
||||
import com.kingsrook.qqq.middleware.health.model.metadata.HealthCheckResult;
|
||||
import com.kingsrook.qqq.middleware.health.model.metadata.HealthStatus;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** STAND-IN(core=QRun-IO/qqq#1199, removeBy=4.1.0-RC.9)
|
||||
** CRM checks for hosts to register with their health middleware. Run in the
|
||||
** caller's tenant context when one is supplied, else as the core system user;
|
||||
** no check returns personal values.
|
||||
** CRM checks registered with the core health registry (HealthChecksMetaData).
|
||||
** The core HealthRunner schedules them and runs each one in its own automated
|
||||
** system-user context; no check returns personal values.
|
||||
*******************************************************************************/
|
||||
public class CrmHealthChecks
|
||||
{
|
||||
public static final String EMAIL = "crm.email";
|
||||
public static final String SEQUENCE_LAG = "crm.sequence_lag";
|
||||
public static final String LEAD_SCORE = "crm.lead_score";
|
||||
|
||||
private static final String COMPONENT = "crm";
|
||||
|
||||
private final CrmEmailTransport transport;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Use the same readiness decision as actual email delivery.
|
||||
*******************************************************************************/
|
||||
@@ -69,6 +77,7 @@ public class CrmHealthChecks
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Allow a host transport adapter.
|
||||
*******************************************************************************/
|
||||
@@ -78,121 +87,141 @@ public class CrmHealthChecks
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Run all checks without registering tables or starting background workers.
|
||||
** Register the CRM checks with the instance's core health registry.
|
||||
** Production may run repeatedly; an already registered name is kept.
|
||||
*******************************************************************************/
|
||||
public Map<String, HealthCheckResult> run() throws QException
|
||||
public static void register(QInstance instance, CrmQBitConfig config)
|
||||
{
|
||||
return (Map.of("crm.email", email(), "crm.sequenceLag", sequenceLag(), "crm.leadScore", leadScore()));
|
||||
new CrmHealthChecks(config).registerWith(instance);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Expose individual checks through the RC.6 health SPI. With no caller
|
||||
** session (registration at startup), each check runs as the core system
|
||||
** user: token-based authentication modules (OAuth2, Auth0) reject any other
|
||||
** tokenless session, and the checks report only UP/DEGRADED, never values.
|
||||
** Register these checks (with this transport) with the core health registry.
|
||||
*******************************************************************************/
|
||||
public List<HealthIndicator> indicators()
|
||||
public void registerWith(QInstance instance)
|
||||
{
|
||||
return (indicators(QContext.getQSession()));
|
||||
HealthChecksMetaData registry = HealthChecksMetaData.ofOrWithNew(instance);
|
||||
add(registry, EMAIL, "CRM email transport", HealthCategory.EMAIL, 300);
|
||||
add(registry, SEQUENCE_LAG, "CRM sequence lag", HealthCategory.JOBS, 300);
|
||||
add(registry, LEAD_SCORE, "CRM lead scoring", HealthCategory.JOBS, 900);
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Supply the host's monitoring session explicitly for middleware worker threads.
|
||||
** Its permissions and tenant keys are preserved; this adapter never grants keys.
|
||||
** A null session runs each check as the core system user.
|
||||
*******************************************************************************/
|
||||
public List<HealthIndicator> indicators(QSession session)
|
||||
{
|
||||
return (List.of(new Indicator("crm.email", session), new Indicator("crm.sequenceLag", session), new Indicator("crm.leadScore", session)));
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** One independently evaluated middleware indicator.
|
||||
** Run all checks once in the caller's context, without registering anything.
|
||||
*******************************************************************************/
|
||||
private class Indicator implements HealthIndicator
|
||||
public Map<String, HealthCheckOutcome> run() throws QException
|
||||
{
|
||||
private final String name;
|
||||
private final QSession session;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Select a check.
|
||||
*******************************************************************************/
|
||||
private Indicator(String name, QSession session)
|
||||
Map<String, HealthCheckOutcome> outcomes = new LinkedHashMap<>();
|
||||
for(String name : List.of(EMAIL, SEQUENCE_LAG, LEAD_SCORE))
|
||||
{
|
||||
this.name = name;
|
||||
this.session = session;
|
||||
outcomes.put(name, check(name).run(new HealthCheckInput()));
|
||||
}
|
||||
return (outcomes);
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public String getName()
|
||||
|
||||
/*******************************************************************************
|
||||
** The core SPI body for one named check. Failures are reported as UNKNOWN
|
||||
** with a fixed message, so no exception text (or value) leaves the check.
|
||||
*******************************************************************************/
|
||||
public HealthCheck check(String name)
|
||||
{
|
||||
return (new HealthCheck()
|
||||
{
|
||||
return (name);
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public HealthCheckResult check(QInstance instance)
|
||||
{
|
||||
CapturedContext original = QContext.capture();
|
||||
try
|
||||
/*******************************************************************
|
||||
** Evaluate the named check in the current QContext.
|
||||
*******************************************************************/
|
||||
@Override
|
||||
public HealthCheckOutcome run(HealthCheckInput input)
|
||||
{
|
||||
QContext.init(instance, session != null ? session : new QSystemUserSession(instance));
|
||||
return (switch(name)
|
||||
try
|
||||
{
|
||||
case "crm.email" -> email();
|
||||
case "crm.sequenceLag" -> sequenceLag();
|
||||
default -> leadScore();
|
||||
});
|
||||
return (switch(name)
|
||||
{
|
||||
case EMAIL -> email();
|
||||
case SEQUENCE_LAG -> sequenceLag();
|
||||
case LEAD_SCORE -> leadScore();
|
||||
default -> throw (new IllegalArgumentException("Unknown CRM health check"));
|
||||
});
|
||||
}
|
||||
catch(QException | RuntimeException e)
|
||||
{
|
||||
return (HealthCheckOutcome.unknown("CRM check could not be completed"));
|
||||
}
|
||||
}
|
||||
catch(QException | RuntimeException e)
|
||||
{
|
||||
return (new HealthCheckResult().withStatus(HealthStatus.UNKNOWN).withDetail("reason", "CRM check could not be completed"));
|
||||
}
|
||||
finally
|
||||
{
|
||||
QContext.init(original);
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Add one check definition unless the name is already registered.
|
||||
*******************************************************************************/
|
||||
private void add(HealthChecksMetaData registry, String name, String label, HealthCategory category, int intervalSeconds)
|
||||
{
|
||||
if(registry.getChecks().stream().anyMatch(existing -> name.equals(existing.getName())))
|
||||
{
|
||||
return;
|
||||
}
|
||||
registry.withCheck(new HealthCheckMetaData()
|
||||
.withName(name)
|
||||
.withLabel(label)
|
||||
.withComponent(COMPONENT)
|
||||
.withCategory(category)
|
||||
.withKind(HealthCheckKind.DEPENDENCY)
|
||||
.withSeverity(AlertSeverity.WARNING)
|
||||
.withIntervalSeconds(intervalSeconds)
|
||||
.withTimeoutMs(5000)
|
||||
.withCheckInstance(check(name)));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** An unconfigured transport is degraded, not a claim that email can be sent.
|
||||
*******************************************************************************/
|
||||
private HealthCheckResult email() throws QException
|
||||
private HealthCheckOutcome email() throws QException
|
||||
{
|
||||
return (new HealthCheckResult().withStatus(transport.readiness() == CrmReadiness.NOT_CONFIGURED ? HealthStatus.DEGRADED : HealthStatus.UP));
|
||||
return (transport.readiness() == CrmReadiness.NOT_CONFIGURED
|
||||
? HealthCheckOutcome.degraded("Email transport not configured")
|
||||
: HealthCheckOutcome.up("Email transport ready"));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Query just the oldest active due enrollment.
|
||||
*******************************************************************************/
|
||||
private HealthCheckResult sequenceLag() throws QException
|
||||
private HealthCheckOutcome sequenceLag() throws QException
|
||||
{
|
||||
List<QRecord> rows = new QueryAction().execute(new QueryInput("crm_sequence_enrollment").withFilter(new QQueryFilter()
|
||||
.withCriteria(new QFilterCriteria("status", QCriteriaOperator.EQUALS, CrmEnrollmentStatus.ACTIVE.getId()))
|
||||
.withCriteria(new QFilterCriteria("nextStepDate", QCriteriaOperator.LESS_THAN, Instant.now().minus(30, ChronoUnit.MINUTES)))
|
||||
.withOrderBy(new QFilterOrderBy("nextStepDate")).withLimit(1))).getRecords();
|
||||
return (new HealthCheckResult().withStatus(rows.isEmpty() ? HealthStatus.UP : HealthStatus.DEGRADED));
|
||||
return (rows.isEmpty()
|
||||
? HealthCheckOutcome.up("No sequence step is more than 30 minutes late")
|
||||
: HealthCheckOutcome.degraded("A sequence step is more than 30 minutes late"));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** No successful score pass, or one more than three hours old, needs attention.
|
||||
*******************************************************************************/
|
||||
private HealthCheckResult leadScore() throws QException
|
||||
private HealthCheckOutcome leadScore() throws QException
|
||||
{
|
||||
List<QRecord> rows = new QueryAction().execute(new QueryInput(JobState.TABLE_NAME).withFilter(new QQueryFilter(
|
||||
new QFilterCriteria("name", QCriteriaOperator.EQUALS, CalculateLeadScoresProcess.NAME)).withLimit(1))).getRecords();
|
||||
Instant last = rows.isEmpty() ? null : rows.getFirst().getValueInstant("lastSuccess");
|
||||
return (new HealthCheckResult().withStatus(last == null || last.isBefore(Instant.now().minus(3, ChronoUnit.HOURS))
|
||||
? HealthStatus.DEGRADED : HealthStatus.UP));
|
||||
return (last == null || last.isBefore(Instant.now().minus(3, ChronoUnit.HOURS))
|
||||
? HealthCheckOutcome.degraded("No lead score pass succeeded in the last 3 hours")
|
||||
: HealthCheckOutcome.up("Lead scores are current"));
|
||||
}
|
||||
}
|
||||
+5
-1
@@ -105,7 +105,11 @@ public class CalculateLeadScoresProcess implements BackendStep, MetaDataProducer
|
||||
@Override
|
||||
public void run(RunBackendStepInput input, RunBackendStepOutput output) throws QException
|
||||
{
|
||||
Instant startedAt = Instant.now();
|
||||
////////////////////////////////////////////////////////////////////////
|
||||
// modifyDate is kept at millisecond precision; a microsecond watermark //
|
||||
// would skip a change made in the same millisecond the pass started. //
|
||||
////////////////////////////////////////////////////////////////////////
|
||||
Instant startedAt = Instant.now().truncatedTo(ChronoUnit.MILLIS);
|
||||
List<QRecord> states = new QueryAction().execute(new QueryInput(JobState.TABLE_NAME)
|
||||
.withFilter(new QQueryFilter(new QFilterCriteria("name", QCriteriaOperator.EQUALS, NAME)))).getRecords();
|
||||
QRecord state = states.isEmpty() ? new QRecord().withValue("name", NAME) : states.getFirst();
|
||||
|
||||
@@ -33,14 +33,19 @@ import com.kingsrook.qbits.crm.health.CrmHealthChecks;
|
||||
import com.kingsrook.qbits.crm.scoring.processes.CalculateLeadScoresProcess;
|
||||
import com.kingsrook.qqq.backend.core.actions.tables.QueryAction;
|
||||
import com.kingsrook.qqq.backend.core.context.QContext;
|
||||
import com.kingsrook.qqq.backend.core.health.HealthCheckInput;
|
||||
import com.kingsrook.qqq.backend.core.health.HealthCheckMetaData;
|
||||
import com.kingsrook.qqq.backend.core.health.HealthChecksMetaData;
|
||||
import com.kingsrook.qqq.backend.core.health.HealthStatus;
|
||||
import com.kingsrook.qqq.backend.core.model.actions.tables.query.QueryInput;
|
||||
import com.kingsrook.qqq.backend.core.model.data.QRecord;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QAuthenticationType;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.QInstance;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.authentication.QAuthenticationMetaData;
|
||||
import com.kingsrook.qqq.backend.core.model.metadata.code.QCodeReference;
|
||||
import com.kingsrook.qqq.backend.core.model.savedbulkloadprofiles.SavedBulkLoadProfileMetaDataProvider;
|
||||
import com.kingsrook.qqq.backend.core.model.session.QSystemUserSession;
|
||||
import com.kingsrook.qqq.backend.core.processes.implementations.bulk.insert.model.BulkLoadProfile;
|
||||
import com.kingsrook.qqq.middleware.health.model.metadata.HealthStatus;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
@@ -51,27 +56,41 @@ import static org.assertj.core.api.Assertions.assertThat;
|
||||
class CrmHealthBulkLoadTest extends BaseTest
|
||||
{
|
||||
/*******************************************************************************
|
||||
** No transport or score run needs attention.
|
||||
** No transport or score run needs attention; the producer registered the
|
||||
** same checks with the core health registry.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void noTransportOrScoreRunNeedsAttention() throws Exception
|
||||
{
|
||||
var checks = new CrmHealthChecks(new CrmQBitConfig());
|
||||
var results = checks.run();
|
||||
assertThat(results.get("crm.email").getStatus()).isEqualTo(HealthStatus.DEGRADED);
|
||||
assertThat(results.get("crm.leadScore").getStatus()).isEqualTo(HealthStatus.DEGRADED);
|
||||
assertThat(results.get("crm.sequenceLag").getStatus()).isEqualTo(HealthStatus.UP);
|
||||
checks.indicators().forEach(indicator ->
|
||||
assertThat(results.get(CrmHealthChecks.EMAIL).getStatus()).isEqualTo(HealthStatus.DEGRADED);
|
||||
assertThat(results.get(CrmHealthChecks.LEAD_SCORE).getStatus()).isEqualTo(HealthStatus.DEGRADED);
|
||||
assertThat(results.get(CrmHealthChecks.SEQUENCE_LAG).getStatus()).isEqualTo(HealthStatus.UP);
|
||||
for(HealthCheckMetaData registered : crmChecks(QContext.getQInstance()))
|
||||
{
|
||||
try
|
||||
{
|
||||
assertThat(indicator.check(QContext.getQInstance()).getStatus()).isEqualTo(results.get(indicator.getName()).getStatus());
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
throw (new AssertionError(e));
|
||||
}
|
||||
assertThat(registered.getCheckInstance().run(new HealthCheckInput()).getStatus()).isEqualTo(results.get(registered.getName()).getStatus());
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The producer registers exactly the three CRM checks as valid core
|
||||
** definitions, and re-registration does not duplicate them.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void producerRegistersCoreHealthChecks()
|
||||
{
|
||||
QInstance instance = QContext.getQInstance();
|
||||
assertThat(crmChecks(instance)).extracting(HealthCheckMetaData::getName)
|
||||
.containsExactlyInAnyOrder(CrmHealthChecks.EMAIL, CrmHealthChecks.SEQUENCE_LAG, CrmHealthChecks.LEAD_SCORE);
|
||||
assertThat(crmChecks(instance)).allSatisfy(check ->
|
||||
{
|
||||
assertThat(check.getCheckInstance()).isNotNull();
|
||||
assertThat(check.getTimeoutMs()).isLessThan(check.getIntervalSeconds() * 1000);
|
||||
});
|
||||
CrmHealthChecks.register(instance, new CrmQBitConfig());
|
||||
assertThat(crmChecks(instance)).hasSize(3);
|
||||
}
|
||||
|
||||
|
||||
@@ -101,8 +120,8 @@ class CrmHealthBulkLoadTest extends BaseTest
|
||||
.withValue("lastSuccess", Instant.now().minus(181, ChronoUnit.MINUTES)));
|
||||
com.kingsrook.qbits.crm.core.CrmRecords.update("crm_sequence_enrollment", new QRecord().withValue("id", 1)
|
||||
.withValue("nextStepDate", Instant.now().minus(31, ChronoUnit.MINUTES)));
|
||||
assertThat(checks.run().get("crm.sequenceLag").getStatus()).isEqualTo(HealthStatus.DEGRADED);
|
||||
assertThat(checks.run().get("crm.leadScore").getStatus()).isEqualTo(HealthStatus.DEGRADED);
|
||||
assertThat(checks.run().get(CrmHealthChecks.SEQUENCE_LAG).getStatus()).isEqualTo(HealthStatus.DEGRADED);
|
||||
assertThat(checks.run().get(CrmHealthChecks.LEAD_SCORE).getStatus()).isEqualTo(HealthStatus.DEGRADED);
|
||||
}
|
||||
|
||||
|
||||
@@ -120,7 +139,7 @@ class CrmHealthBulkLoadTest extends BaseTest
|
||||
throw (new IllegalStateException("unavailable"));
|
||||
}
|
||||
});
|
||||
assertThat(checks.indicators().getFirst().check(QContext.getQInstance()).getStatus()).isEqualTo(HealthStatus.UNKNOWN);
|
||||
assertThat(checks.check(CrmHealthChecks.EMAIL).run(new HealthCheckInput()).getStatus()).isEqualTo(HealthStatus.UNKNOWN);
|
||||
}
|
||||
|
||||
|
||||
@@ -163,27 +182,36 @@ class CrmHealthBulkLoadTest extends BaseTest
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Health middleware invokes checks in worker threads without a caller QContext.
|
||||
** The core HealthRunner runs each check on a worker thread with no caller
|
||||
** QContext, as a QSystemUserSession. A registered CRM check must succeed that
|
||||
** way on a host whose authentication rejects tokenless sessions (OAuth2, Auth0).
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void indicatorsCarryMonitoringSessionAndRestoreWorkerContext() throws Exception
|
||||
void registeredChecksRunAsCoreSystemUserOnTokenRequiredHost() throws Exception
|
||||
{
|
||||
var instance = QContext.getQInstance();
|
||||
var indicator = new CrmHealthChecks(new CrmQBitConfig()).indicators(QContext.getQSession()).get(1);
|
||||
instance.withInstanceDefaultAuthentication(new QAuthenticationMetaData()
|
||||
.withName("token-required")
|
||||
.withType(QAuthenticationType.MOCK)
|
||||
.withModuleCodeReference(new QCodeReference(TokenRequiredAuthenticationModule.class)));
|
||||
HealthCheckMetaData sequenceLag = crmChecks(instance).stream()
|
||||
.filter(check -> CrmHealthChecks.SEQUENCE_LAG.equals(check.getName())).findFirst().orElseThrow();
|
||||
var status = java.util.concurrent.CompletableFuture.supplyAsync(() ->
|
||||
{
|
||||
assertThat(QContext.getQInstance()).isNull();
|
||||
QContext.init(instance, new QSystemUserSession(instance));
|
||||
try
|
||||
{
|
||||
assertThat(QContext.getQInstance()).isNull();
|
||||
var result = indicator.check(instance);
|
||||
assertThat(QContext.getQInstance()).isNull();
|
||||
assertThat(QContext.getQSession()).isNull();
|
||||
return (result.getStatus());
|
||||
return (sequenceLag.getCheckInstance().run(new HealthCheckInput().withQInstance(instance)).getStatus());
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
throw (new AssertionError(e));
|
||||
}
|
||||
finally
|
||||
{
|
||||
QContext.clear();
|
||||
}
|
||||
}).get();
|
||||
assertThat(status).isEqualTo(HealthStatus.UP);
|
||||
assertThat(QContext.getQInstance()).isSameAs(instance);
|
||||
@@ -192,32 +220,12 @@ class CrmHealthBulkLoadTest extends BaseTest
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Regression: indicators registered with no caller session (at host startup)
|
||||
** must still run on a host whose authentication rejects tokenless sessions
|
||||
** (OAuth2, Auth0). They used a plain new QSession(), so every check failed
|
||||
** session validation and reported UNKNOWN.
|
||||
** The CRM definitions in the core health registry.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void contextlessIndicatorsRunOnTokenRequiredHost() throws Exception
|
||||
private static List<HealthCheckMetaData> crmChecks(QInstance instance)
|
||||
{
|
||||
var instance = QContext.getQInstance();
|
||||
instance.withInstanceDefaultAuthentication(new QAuthenticationMetaData()
|
||||
.withName("token-required")
|
||||
.withType(QAuthenticationType.MOCK)
|
||||
.withModuleCodeReference(new QCodeReference(TokenRequiredAuthenticationModule.class)));
|
||||
var status = java.util.concurrent.CompletableFuture.supplyAsync(() ->
|
||||
{
|
||||
try
|
||||
{
|
||||
var indicator = new CrmHealthChecks(new CrmQBitConfig()).indicators().get(1);
|
||||
return (indicator.check(instance).getStatus());
|
||||
}
|
||||
catch(Exception e)
|
||||
{
|
||||
throw (new AssertionError(e));
|
||||
}
|
||||
}).get();
|
||||
assertThat(status).isEqualTo(HealthStatus.UP);
|
||||
return (HealthChecksMetaData.ofOrWithNew(instance).getChecks().stream()
|
||||
.filter(check -> "crm".equals(check.getComponent())).toList());
|
||||
}
|
||||
|
||||
}
|
||||
Reference in new issue
Block a user