Compare commits
17
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
73341e59c5
|
||
|
|
0d989683eb
|
||
|
|
6d733cde82 | ||
|
|
72e8ffb12d
|
||
|
|
b3d2f26548
|
||
|
|
0aa2f5180e | ||
|
|
cc08447020
|
||
|
|
6c16d73f28 | ||
|
|
266d101c91
|
||
|
|
cef1476d85
|
||
|
|
5493b598c6
|
||
|
|
07f846975b
|
||
|
|
a368187b44 | ||
|
|
4e6a2ae365
|
||
|
|
adcafe33b2
|
||
|
|
01022e03c5
|
||
|
|
3f08c584d0
|
No files matched your search
@@ -0,0 +1,7 @@
|
||||
name: build
|
||||
on:
|
||||
push:
|
||||
jobs:
|
||||
maven:
|
||||
uses: self:KofTwentyTwo/ci-workflows/.gitea/workflows/maven.yaml@v1
|
||||
secrets: inherit
|
||||
@@ -4,6 +4,12 @@ All notable changes are documented here, following [Keep a Changelog](https://ke
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Changed
|
||||
|
||||
- PostgreSQL qualification (`RecordFilesPostgres`) accepts `full_page_writes = off` when, and only when, the server is Amazon Aurora PostgreSQL. Aurora reports `full_page_writes = off` (measured on engine 17.9) and the parameter is not in its cluster parameter group, so it cannot be changed; its storage layer does not write torn pages, so the protection that setting gives stock PostgreSQL is not needed there. Before this change Record Files refused every Aurora PostgreSQL database with its "exact writable PostgreSQL 17 database" message. Aurora is detected inside the existing settings query with `to_regprocedure('aurora_version()') IS NOT NULL`, which returns false and never raises on stock PostgreSQL. Nothing else is relaxed: stock PostgreSQL with `full_page_writes = off` is still refused, and on Aurora `fsync = on`, `synchronous_commit` of `on` or `remote_apply`, a writable primary, PostgreSQL 17, a UTC session, READ COMMITTED, a `lock_timeout` from 1 to 5000 ms and every owner-table check are required exactly as before. Covered by `PostgresDurabilityRuleTest` (every build) and `PostgresAuroraQualificationTest` (opt-in, real PostgreSQL 17, Aurora simulated by a test-only `aurora_version()` function).
|
||||
|
||||
## [0.1.0-RC.1] - 2026-11-06
|
||||
|
||||
Planned as **0.1.0-RC.1**, the first artifact published from this repository. It is published by pushing `develop` to `release/0.1`; the shared `KofTwentyTwo/ci-workflows` Maven workflow computes the `0.1.0-RC.N` version, deploys to the Gitea Maven registry (`https://git.kof22.com/api/packages/QRun-IO/maven`) and tags `v0.1.0-RC.N`. Nothing is published from `develop` except the `0.1.0-SNAPSHOT`.
|
||||
|
||||
### Added
|
||||
|
||||
@@ -68,9 +68,33 @@ applies to all trusted generated outputs, not only previews. Hosts must trust ev
|
||||
in-process producer allowed to supply that origin; origin is provenance, not a
|
||||
cryptographic attestation of safe content.
|
||||
|
||||
`quarantineCapturedFiles` is a second optional host setting with the same shape
|
||||
(`getQuarantineCapturedFiles()` / `withQuarantineCapturedFiles(Boolean)`; null, the
|
||||
default, inherits `quarantineNewFiles`). It applies only to intake with origin
|
||||
`CAPTURE` **and** a nonblank device identity, and `GENERATED` keeps its own policy.
|
||||
In the upload path that identity is the server-persisted capture identity that the
|
||||
host's `CaptureIdentityProvider` resolved and revalidated; the manifest cannot carry
|
||||
a device, actor, policy or `CAPTURE` label (strict JSON rejects them; `CAMERA` and
|
||||
`GALLERY` items map to `CAPTURE`, so a device that passes the provider can label an
|
||||
item either way). A desktop request with no provider identity, or any `UPLOAD`,
|
||||
`SCANNER_IMPORT`, `IMPORT` or attachment-panel file, keeps `quarantineNewFiles`.
|
||||
Direct Java `RetainedFileService.intake` callers are trusted host code, like
|
||||
`GENERATED`: they could pass `CAPTURE` plus a device id, so hosts must not expose
|
||||
that call to untrusted input.
|
||||
|
||||
Host prerequisites before setting it to `false`: a `CaptureIdentityProvider` that
|
||||
actually approves each device and actor (a device id alone proves nothing, and Record
|
||||
Files does not decide approval); an `UploadConfiguration` that bounds media types,
|
||||
sizes and quotas to what the host accepts (for example JPEG/PNG only); and
|
||||
parent-table visibility and permissions as before. Nothing scans content, so this
|
||||
is a host trust decision, not a malware or human-review result. Existing quarantined
|
||||
files are not released by configuration; release them through the normal release
|
||||
process. Compatibility: the setting is additive and defaults to null, so existing
|
||||
hosts and persisted data behave exactly as before.
|
||||
|
||||
Upload manifests cannot select `GENERATED` or change host policy. Uploaded,
|
||||
captured, imported and standard attachment-panel originals retain
|
||||
`quarantineNewFiles`. Initial admission records `FILE_REGISTERED` with the chosen
|
||||
imported and standard attachment-panel originals, and captures without a device
|
||||
identity, retain `quarantineNewFiles`. Initial admission records `FILE_REGISTERED` with the chosen
|
||||
policy and lifecycle, without a release, scan or human-review claim. Hash failures
|
||||
still produce `FAILED`; permissions, current parent authorization and mandatory
|
||||
audit remain enforced. Changing configuration never releases existing quarantined
|
||||
|
||||
@@ -49,6 +49,8 @@ Final receipt: `{uploadId,collectionId,state,completedAt,items:[{itemId,fileId,v
|
||||
|
||||
Hard ceilings:20MiB per original,200MiB total,100items,64KiB–4MiB fixed chunk size,60–86400second staging lifetime,256KiB manifest JSON. Deployment limits can be lower and are frozen per upload. The original20MiB bound reflects the conditional single-PUT owner path, not a throughput benchmark. Accepted media types are declarations subject to quarantine/scanning, not verified file contents.
|
||||
|
||||
Initial lifecycle: finalized items are QUARANTINED unless the host sets the Java-only `quarantineCapturedFiles=false`, which makes only `CAMERA`/`GALLERY` items (origin `CAPTURE`) with the provider-resolved, revalidated device identity AVAILABLE. The manifest carries no policy, device or actor field and strict parsing rejects them. This is a host trust decision (approved devices, bounded media types), not a scan or review result; it emits no release event and leaves review status unchanged. See README "Not a 1.0".
|
||||
|
||||
Durable quota defaults:10 active uploads,512MiB reserved staging bytes and100 new uploads/hour per actor. Cancel/expiry does not immediately release capacity: bounded exact-key cleanup must finish and commit its cursor/outcome/quota release. `UploadMaintenance.cleanupOwn` permits currently authorized terminal cleanup. A trusted host scheduler can invoke bounded `cleanupBatch` under `record-files.maintain`; this does not grant content access. Expired finalization fences commit before irreversible staging deletion. Failed deletion/outcome retains cursor and quota for safe retry. Canonical originals are outside cleanup.
|
||||
|
||||
## Remaining integration and release evidence
|
||||
|
||||
@@ -7,3 +7,4 @@ Updated: 2026-11-06. These are unresolved items carried into the first release c
|
||||
- `qbit-collaboration.version` is still `0.1.0-SNAPSHOT`, a placeholder for source integration. The release candidate must pin a published Collaboration RC before `release/0.1` is pushed (see RELEASE-STATUS.md). No clean consumer installation of the complete public cohort has been demonstrated.
|
||||
- Full Record Files verification, database integration, retained-file recovery and final standard-UI acceptance remain incomplete for this reconciled source.
|
||||
- Permanent access depends on the host's durable object storage, backups, retention and permission configuration. A source-level interface or unit test alone does not prove operational permanence.
|
||||
- The Amazon Aurora PostgreSQL rule (`full_page_writes = off` accepted on Aurora only) is tested on stock PostgreSQL 17 with Aurora simulated by a test-only `aurora_version()` function. No test in this repository runs against a real Aurora cluster, and the opt-in PostgreSQL suites do not run in the shared CI build. The detection resolves `aurora_version()` through the session `search_path`, so a role that can create functions in a schema on that path of a stock PostgreSQL database could make it report Aurora; the application role must not hold that privilege.
|
||||
@@ -18,22 +18,20 @@
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
|
||||
<properties>
|
||||
<revision>0.1.0-SNAPSHOT</revision>
|
||||
<revision>0.5.0-SNAPSHOT</revision>
|
||||
<maven.compiler.source>21</maven.compiler.source>
|
||||
<maven.compiler.target>21</maven.compiler.target>
|
||||
<jacoco.version>0.8.15</jacoco.version>
|
||||
<!-- Released qqq (Maven Central); one core for the whole module, imported through qqq-bom-pom (see the qqq-snapshot profile). -->
|
||||
<qqq.version>4.1.0-RC.21</qqq.version>
|
||||
<!-- NOTE: collaboration's lifecycle code also needs core after-rollback callbacks (CORE-DEPENDENCY.md); raise this to the core release that has them. -->
|
||||
<!-- Candidate collaboration carrying the attachment lifecycle SPI (feat/record-files-standalone, not yet released).
|
||||
Not in public 0.1.0-RC.6. Qualify with a locally installed 0.1.0-SNAPSHOT; see COLLABORATION-HANDOFF.md. -->
|
||||
<qbit-collaboration.version>0.1.0-SNAPSHOT</qbit-collaboration.version>
|
||||
<!-- QQQ Core 4.1.0-RC.22 from the Gitea registry, imported through qqq-bom-pom. It includes the after-rollback callbacks Collaboration needs. -->
|
||||
<qqq.version>4.1.0-RC.26</qqq.version>
|
||||
<!-- Released Collaboration carrying the attachment lifecycle SPI. -->
|
||||
<qbit-collaboration.version>1.2.0-RC.1</qbit-collaboration.version>
|
||||
</properties>
|
||||
|
||||
<parent>
|
||||
<groupId>com.kingsrook</groupId>
|
||||
<artifactId>qbit-build-parent</artifactId>
|
||||
<version>2.1.1</version>
|
||||
<version>2.2.1</version>
|
||||
</parent>
|
||||
|
||||
<groupId>com.kingsrook.qbits</groupId>
|
||||
@@ -238,7 +236,7 @@
|
||||
<activeByDefault>true</activeByDefault>
|
||||
</activation>
|
||||
<properties>
|
||||
<qqq.snapshot.version>${qqq.version}</qqq.snapshot.version>
|
||||
<qqq.snapshot.version>4.1.0-RC.26</qqq.snapshot.version>
|
||||
</properties>
|
||||
<dependencyManagement>
|
||||
<dependencies>
|
||||
|
||||
@@ -46,14 +46,25 @@ import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSTableBackendDe
|
||||
** Qualification of this QBit's actual PostgreSQL 17 target (the native QQQ
|
||||
** postgres module: vendor and backend type "postgres") and its owned tables.
|
||||
** Observes only; no migrations or session/global setting changes happen here.
|
||||
**
|
||||
** Amazon Aurora PostgreSQL is the one accepted variant of the durability rule:
|
||||
** it reports full_page_writes = off, and that alone is accepted there (see
|
||||
** durable). Every other check is identical on stock PostgreSQL and Aurora.
|
||||
*******************************************************************************/
|
||||
final class RecordFilesPostgres
|
||||
{
|
||||
static final String VENDOR = "postgres";
|
||||
|
||||
private static final String SETTINGS = "SELECT current_setting('server_version_num')::int, current_setting('transaction_read_only'), pg_is_in_recovery(),"
|
||||
/////////////////////////////////////////////////////////////////////////////////////////
|
||||
// Column 12 is true only on Amazon Aurora PostgreSQL, which defines aurora_version(). //
|
||||
// to_regprocedure returns NULL for a function that does not exist and never raises, //
|
||||
// so the probe is safe on stock PostgreSQL; calling aurora_version() directly there //
|
||||
// would abort the transaction, and pg_proc does not list the function on Aurora. //
|
||||
/////////////////////////////////////////////////////////////////////////////////////////
|
||||
static final String SETTINGS = "SELECT current_setting('server_version_num')::int, current_setting('transaction_read_only'), pg_is_in_recovery(),"
|
||||
+ " current_setting('synchronous_commit'), current_setting('fsync'), current_setting('full_page_writes'), current_setting('TimeZone'),"
|
||||
+ " (SELECT setting::bigint FROM pg_settings WHERE name='lock_timeout'), current_setting('transaction_isolation'), current_schema(), current_database()";
|
||||
+ " (SELECT setting::bigint FROM pg_settings WHERE name='lock_timeout'), current_setting('transaction_isolation'), current_schema(), current_database(),"
|
||||
+ " to_regprocedure('aurora_version()') IS NOT NULL";
|
||||
|
||||
private static final String TABLES = "SELECT c.relname, c.relkind, c.relpersistence FROM pg_class c JOIN pg_namespace n ON n.oid=c.relnamespace"
|
||||
+ " WHERE n.nspname=current_schema() AND c.relname = ANY(?)";
|
||||
@@ -131,7 +142,7 @@ final class RecordFilesPostgres
|
||||
try(ResultSet settings = statement.executeQuery(SETTINGS))
|
||||
{
|
||||
if(!settings.next() || settings.getInt(1) / 10000 != 17 || !"off".equals(settings.getString(2)) || settings.getBoolean(3)
|
||||
|| !Set.of("on", "remote_apply").contains(settings.getString(4)) || !"on".equals(settings.getString(5)) || !"on".equals(settings.getString(6))
|
||||
|| !durable(settings.getString(4), settings.getString(5), settings.getString(6), settings.getBoolean(12))
|
||||
|| !Set.of("UTC", "Etc/UTC").contains(settings.getString(7)) || settings.getLong(8) < 1 || settings.getLong(8) > 5000
|
||||
|| !"read committed".equals(settings.getString(9)) || settings.getString(10) == null || !catalog.equals(settings.getString(11)))
|
||||
{
|
||||
@@ -155,6 +166,30 @@ final class RecordFilesPostgres
|
||||
requireTables(connection, tables);
|
||||
}
|
||||
|
||||
/***************************************************************************
|
||||
** Committed durability from the observed server settings: synchronous_commit
|
||||
** on or remote_apply, fsync on, and torn-page protection.
|
||||
***************************************************************************/
|
||||
static boolean durable(String synchronousCommit, String fsync, String fullPageWrites, boolean aurora)
|
||||
{
|
||||
if(synchronousCommit == null || !Set.of("on", "remote_apply").contains(synchronousCommit) || !"on".equals(fsync))
|
||||
{
|
||||
return (false);
|
||||
}
|
||||
|
||||
////////////////////////////////////////////////////////////////////////////////////////
|
||||
// Stock PostgreSQL needs full_page_writes = on: after a crash, a partly written page //
|
||||
// is repaired from the full page image in the write-ahead log. //
|
||||
// Amazon Aurora PostgreSQL reports full_page_writes = off (measured on engine 17.9) //
|
||||
// and the parameter cannot be changed there: it is not in the cluster parameter //
|
||||
// group. Its storage layer does not write torn pages, so the protection is not //
|
||||
// needed. Off is therefore accepted on Aurora only; on stock PostgreSQL it is still //
|
||||
// refused. fsync and synchronous_commit are required on Aurora exactly as on stock //
|
||||
// PostgreSQL. //
|
||||
////////////////////////////////////////////////////////////////////////////////////////
|
||||
return ("on".equals(fullPageWrites) || (aurora && "off".equals(fullPageWrites)));
|
||||
}
|
||||
|
||||
/***************************************************************************
|
||||
** Permanent tables, deterministic comparison, owner column types and complete unique keys.
|
||||
***************************************************************************/
|
||||
|
||||
@@ -81,6 +81,7 @@ public class RecordFilesQBitConfig implements QBitConfig
|
||||
private List<String> parentTables = List.of();
|
||||
private Boolean quarantineNewFiles = true;
|
||||
private Boolean quarantineGeneratedFiles;
|
||||
private Boolean quarantineCapturedFiles;
|
||||
private Integer maxItemsPerCollection = 500;
|
||||
private QCodeReference auditPort = new QCodeReference(TableAuditPort.class);
|
||||
|
||||
@@ -227,6 +228,32 @@ public class RecordFilesQBitConfig implements QBitConfig
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Optional trusted captured-intake policy. Null inherits quarantineNewFiles.
|
||||
** Applies only to CAPTURE intake that carries a nonblank server-resolved device
|
||||
** identity; never to GENERATED, UPLOAD, IMPORT or attachment-panel files.
|
||||
*******************************************************************************/
|
||||
public Boolean getQuarantineCapturedFiles()
|
||||
{
|
||||
return (quarantineCapturedFiles);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Host decision for device capture intake, not a scan or human review result.
|
||||
** False makes valid captured files AVAILABLE on registration. The host must
|
||||
** wire a CaptureIdentityProvider that approves each device and bound the
|
||||
** accepted media types; a device id alone proves no approval.
|
||||
*******************************************************************************/
|
||||
public RecordFilesQBitConfig withQuarantineCapturedFiles(Boolean quarantineCapturedFiles)
|
||||
{
|
||||
this.quarantineCapturedFiles = quarantineCapturedFiles;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Upper bound on the items of one collection.
|
||||
*******************************************************************************/
|
||||
|
||||
@@ -279,7 +279,15 @@ public class RetainedFileService
|
||||
|
||||
boolean generatedPolicy = pending != null && RecordFilesValues.ORIGIN_GENERATED.equals(pending.request().origin())
|
||||
&& config.getQuarantineGeneratedFiles() != null;
|
||||
boolean quarantine = Boolean.TRUE.equals(generatedPolicy ? config.getQuarantineGeneratedFiles() : config.getQuarantineNewFiles());
|
||||
// Captured policy needs the service-owned intake scope (never a plain standard upload), the CAPTURE origin
|
||||
// and a nonblank device identity. UploadFinalizer supplies that identity only from the server-persisted,
|
||||
// revalidated capture identity; a client cannot choose this policy through the upload JSON.
|
||||
boolean capturedPolicy = !generatedPolicy && pending != null && RecordFilesValues.ORIGIN_CAPTURE.equals(pending.request().origin())
|
||||
&& pending.request().deviceId() != null && !pending.request().deviceId().isBlank()
|
||||
&& config.getQuarantineCapturedFiles() != null;
|
||||
boolean quarantine = Boolean.TRUE.equals(generatedPolicy ? config.getQuarantineGeneratedFiles()
|
||||
: capturedPolicy ? config.getQuarantineCapturedFiles() : config.getQuarantineNewFiles());
|
||||
String admissionPolicy = generatedPolicy ? "quarantineGeneratedFiles" : capturedPolicy ? "quarantineCapturedFiles" : "quarantineNewFiles";
|
||||
String lifecycle = mismatch ? RecordFilesValues.LIFECYCLE_FAILED
|
||||
: quarantine ? RecordFilesValues.LIFECYCLE_QUARANTINED : RecordFilesValues.LIFECYCLE_AVAILABLE;
|
||||
QRecord row = new QRecord()
|
||||
@@ -313,7 +321,7 @@ public class RetainedFileService
|
||||
audit.withChange(new RecordFilesEvent(RecordFilesEventType.FILE_REGISTERED, actor, correlationId).withRequestId(requestId)
|
||||
.withFile(fileId, attachmentId).withSource(tableName, recordId)
|
||||
.withDetail("lifecycle=" + lifecycle + ", origin=" + row.getValueString("origin") + ", version=" + row.getValueInteger("versionNumber")
|
||||
+ ", admissionPolicy=" + (generatedPolicy ? "quarantineGeneratedFiles" : "quarantineNewFiles") + ", quarantine=" + quarantine
|
||||
+ ", admissionPolicy=" + admissionPolicy + ", quarantine=" + quarantine
|
||||
+ (mismatch ? ", declared SHA-256 did not match the stored bytes" : "")));
|
||||
if(pending != null)
|
||||
{
|
||||
|
||||
@@ -94,6 +94,105 @@ public class IntakeAndRetentionTest extends RecordFilesFixture
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Without the host opt-in, device capture keeps the product quarantine default.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void capturedIntakeQuarantinesByDefault() throws Exception
|
||||
{
|
||||
assertThat(config.getQuarantineCapturedFiles()).isNull();
|
||||
RetainedFile captured = files.intake(new IntakeRequest("captured-default", "document", 1, "photo.jpg", sha("photo"))
|
||||
.withOrigin(RecordFilesValues.ORIGIN_CAPTURE, "CAMERA").withDevice("device-a", null), stream("photo"));
|
||||
assertThat(captured.lifecycle()).isEqualTo(RecordFilesValues.LIFECYCLE_QUARANTINED);
|
||||
assertThat(events()).anySatisfy(event -> assertThat(event.getValueString("detail"))
|
||||
.contains("lifecycle=QUARANTINED", "admissionPolicy=quarantineNewFiles", "quarantine=true"));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** An opted-in host makes a valid device capture AVAILABLE and audits the policy,
|
||||
** with no release, review or scan event and an unchanged initial review status.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void capturedPolicyMakesValidDeviceCaptureAvailableWithoutReviewClaim() throws Exception
|
||||
{
|
||||
RetainedFile quarantined = files.intake(new IntakeRequest("baseline-upload", "document", 1, "baseline.jpg", sha("baseline"))
|
||||
.withOrigin(RecordFilesValues.ORIGIN_UPLOAD, null), stream("baseline"));
|
||||
config.withQuarantineCapturedFiles(false);
|
||||
assertThat(config.getQuarantineNewFiles()).isTrue();
|
||||
RetainedFile captured = files.intake(new IntakeRequest("captured-open", "document", 1, "photo.jpg", sha("photo"))
|
||||
.withOrigin(RecordFilesValues.ORIGIN_CAPTURE, "CAMERA").withDevice("device-a", null), stream("photo"));
|
||||
assertThat(captured.lifecycle()).isEqualTo(RecordFilesValues.LIFECYCLE_AVAILABLE);
|
||||
assertThat(RecordFilesStore.first(RecordFilesTables.FILE, "id", captured.id()).getValueInteger("reviewStatusId"))
|
||||
.isEqualTo(RecordFilesStore.first(RecordFilesTables.FILE, "id", quarantined.id()).getValueInteger("reviewStatusId"));
|
||||
assertThat(events()).anySatisfy(event ->
|
||||
{
|
||||
assertThat(event.getValueString("eventType")).isEqualTo("FILE_REGISTERED");
|
||||
assertThat(event.getValueString("detail")).contains("lifecycle=AVAILABLE", "admissionPolicy=quarantineCapturedFiles", "quarantine=false");
|
||||
});
|
||||
assertThat(eventTypes()).doesNotContain("FILE_RELEASED", "REVIEW_CHANGE_ATTEMPT");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The captured opt-in needs CAPTURE origin plus a nonblank device identity.
|
||||
** Other origins, standard attachments and missing identity keep quarantine,
|
||||
** and the generated policy is neither widened nor narrowed by it.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void capturedPolicyDoesNotApplyToOtherOriginsOrMissingDeviceIdentity() throws Exception
|
||||
{
|
||||
config.withQuarantineCapturedFiles(false);
|
||||
for(String origin : List.of(RecordFilesValues.ORIGIN_UPLOAD, RecordFilesValues.ORIGIN_IMPORT, RecordFilesValues.ORIGIN_GENERATED))
|
||||
{
|
||||
assertThat(files.intake(new IntakeRequest("other-" + origin, "document", 1, "input.jpg", sha("input"))
|
||||
.withOrigin(origin, "CAMERA").withDevice("device-a", null), stream("input")).lifecycle())
|
||||
.as(origin).isEqualTo(RecordFilesValues.LIFECYCLE_QUARANTINED);
|
||||
}
|
||||
int index = 0;
|
||||
for(String device : new String[]{null, "", " "})
|
||||
{
|
||||
assertThat(files.intake(new IntakeRequest("no-device-" + index++, "document", 1, "nodevice.jpg", sha("nodevice"))
|
||||
.withOrigin(RecordFilesValues.ORIGIN_CAPTURE, "CAMERA").withDevice(device, null), stream("nodevice")).lifecycle())
|
||||
.as("device [" + device + "]").isEqualTo(RecordFilesValues.LIFECYCLE_QUARANTINED);
|
||||
}
|
||||
AttachmentEntry panel = attachments.upload("document", 1, "panel.pdf", "application/pdf", stream("pdf"), "CAMERA");
|
||||
assertThat(RecordFilesStore.first(RecordFilesTables.FILE, "attachmentId", panel.id()).getValueString("lifecycle"))
|
||||
.isEqualTo(RecordFilesValues.LIFECYCLE_QUARANTINED);
|
||||
|
||||
config.withQuarantineGeneratedFiles(true);
|
||||
assertThat(files.intake(new IntakeRequest("generated-still-quarantined", "document", 1, "gen.pdf", sha("gen"))
|
||||
.withOrigin(RecordFilesValues.ORIGIN_GENERATED, null).withDevice("device-a", null), stream("gen")).lifecycle())
|
||||
.isEqualTo(RecordFilesValues.LIFECYCLE_QUARANTINED);
|
||||
config.withQuarantineGeneratedFiles(false).withQuarantineCapturedFiles(true);
|
||||
assertThat(files.intake(new IntakeRequest("generated-open", "document", 1, "gen2.pdf", sha("gen2"))
|
||||
.withOrigin(RecordFilesValues.ORIGIN_GENERATED, null).withDevice("device-a", null), stream("gen2")).lifecycle())
|
||||
.isEqualTo(RecordFilesValues.LIFECYCLE_AVAILABLE);
|
||||
assertThat(events()).anySatisfy(event -> assertThat(event.getValueString("detail"))
|
||||
.contains("origin=GENERATED", "admissionPolicy=quarantineGeneratedFiles"));
|
||||
assertThat(eventTypes()).doesNotContain("FILE_RELEASED");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A hash mismatch is FAILED and inaccessible even when captured files are open.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void capturedPolicyStillFailsHashMismatch() throws Exception
|
||||
{
|
||||
config.withQuarantineCapturedFiles(false);
|
||||
IntakeRequest request = new IntakeRequest("captured-mismatch", "document", 1, "photo.jpg", sha("expected"))
|
||||
.withOrigin(RecordFilesValues.ORIGIN_CAPTURE, "CAMERA").withDevice("device-a", null);
|
||||
assertThatThrownBy(() -> files.intake(request, stream("different"))).isInstanceOf(QBadRequestException.class);
|
||||
assertThat(RecordFilesStore.first(RecordFilesTables.FILE, "fileName", "photo.jpg").getValueString("lifecycle"))
|
||||
.isEqualTo(RecordFilesValues.LIFECYCLE_FAILED);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Generated policy changes no permissions, integrity or mandatory audit checks.
|
||||
*******************************************************************************/
|
||||
|
||||
@@ -0,0 +1,312 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
|
||||
package com.kingsrook.qbits.recordfiles;
|
||||
|
||||
|
||||
import java.sql.Connection;
|
||||
import java.sql.SQLException;
|
||||
import java.time.Duration;
|
||||
import com.kingsrook.qqq.backend.core.exceptions.QException;
|
||||
import com.kingsrook.qqq.backend.module.rdbms.jdbc.ConnectionManager;
|
||||
import com.kingsrook.qqq.backend.module.rdbms.model.metadata.RDBMSBackendMetaData;
|
||||
import org.junit.jupiter.api.AfterAll;
|
||||
import org.junit.jupiter.api.AfterEach;
|
||||
import org.junit.jupiter.api.BeforeAll;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.api.condition.EnabledIfSystemProperty;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Opt-in qualification of the Amazon Aurora PostgreSQL rule on a real, stock
|
||||
** PostgreSQL 17 server (the disposable container of PostgresRecordFilesSupport).
|
||||
**
|
||||
** What is simulated: Aurora itself. The fixture superuser creates a function
|
||||
** public.aurora_version() in the test database, so the qualification's probe
|
||||
** to_regprocedure('aurora_version()') resolves as it does on Aurora, and sets
|
||||
** full_page_writes = off with ALTER SYSTEM and a configuration reload, the value
|
||||
** Aurora reports. Aurora's storage layer and its other settings are not
|
||||
** reproduced. Everything else is real: the server, the DML-only application
|
||||
** role, the shipped owner script, the settings query and the store calls.
|
||||
*******************************************************************************/
|
||||
@EnabledIfSystemProperty(named = "postgresRecordFilesTests", matches = "true")
|
||||
class PostgresAuroraQualificationTest extends RecordFilesFixture
|
||||
{
|
||||
private static final String REFUSAL = "Record Files requires its exact writable PostgreSQL 17 database";
|
||||
|
||||
private RDBMSBackendMetaData database;
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Start the provider with fsync and full_page_writes left off the server command
|
||||
** line (both default to on): a setting fixed there cannot be changed by ALTER
|
||||
** SYSTEM, and these cases change both.
|
||||
*******************************************************************************/
|
||||
@BeforeAll
|
||||
static void startPostgres() throws Exception
|
||||
{
|
||||
PostgresRecordFilesSupport.start("synchronous_commit=on");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Remove the provider created by this class.
|
||||
*******************************************************************************/
|
||||
@AfterAll
|
||||
static void stopPostgres()
|
||||
{
|
||||
PostgresRecordFilesSupport.stop();
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Every case ends on a stock server with durable settings.
|
||||
*******************************************************************************/
|
||||
@AfterEach
|
||||
void restoreStockDurableServer() throws Exception
|
||||
{
|
||||
execute("DROP FUNCTION IF EXISTS public.aurora_version()");
|
||||
execute("ALTER ROLE " + PostgresRecordFilesSupport.APP_ROLE + " RESET synchronous_commit");
|
||||
execute("ALTER SYSTEM RESET fsync");
|
||||
execute("ALTER SYSTEM RESET full_page_writes");
|
||||
execute("SELECT pg_reload_conf()");
|
||||
awaitSetting("fsync", "on");
|
||||
awaitSetting("full_page_writes", "on");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The native postgres backend with the DML-only application identity.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
protected RDBMSBackendMetaData createBackend() throws Exception
|
||||
{
|
||||
database = PostgresRecordFilesSupport.backend(backendName);
|
||||
return (database);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Generic tables, the Record Files owner script and the journal's PostgreSQL script.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
protected void createSchema(RDBMSBackendMetaData backend) throws Exception
|
||||
{
|
||||
PostgresRecordFilesSupport.createSchema(instance, config);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Owner-level connection for test-only schema fault injection.
|
||||
*******************************************************************************/
|
||||
@Override
|
||||
protected Connection adminConnection(RDBMSBackendMetaData backend) throws Exception
|
||||
{
|
||||
return (PostgresRecordFilesSupport.root());
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The settings query runs as the application role on a stock server without an
|
||||
** error and reports "not Aurora" in column 12; with the function present it
|
||||
** reports Aurora.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void settingsQueryReportsAurora() throws Exception
|
||||
{
|
||||
assertThat(auroraColumn()).isFalse();
|
||||
simulateAurora();
|
||||
assertThat(auroraColumn()).isTrue();
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Stock PostgreSQL with full_page_writes off is still refused, by the probe and
|
||||
** by the transaction the store opens.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void stockPostgresWithFullPageWritesOffIsRefused() throws Exception
|
||||
{
|
||||
assertThat(RecordFilesStore.isDurableBackend(database)).isTrue();
|
||||
server("full_page_writes", "off");
|
||||
assertThat(auroraColumn()).isFalse();
|
||||
assertThat(RecordFilesStore.isDurableBackend(database)).isFalse();
|
||||
assertThatThrownBy(RecordFilesStore::openTransaction).isInstanceOf(QException.class).hasMessageStartingWith(REFUSAL);
|
||||
assertThatThrownBy(() -> collections.create("Refused", "document", 1, 1, null, null)).isInstanceOf(QException.class);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Aurora with full_page_writes off qualifies: the probe, the owned transaction
|
||||
** and a committed owner write.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void auroraWithFullPageWritesOffIsAccepted() throws Exception
|
||||
{
|
||||
server("full_page_writes", "off");
|
||||
simulateAurora();
|
||||
assertThat(RecordFilesStore.isDurableBackend(database)).isTrue();
|
||||
var transaction = RecordFilesStore.openTransaction();
|
||||
try
|
||||
{
|
||||
RecordFilesStore.requireCompatibleTransaction(transaction);
|
||||
}
|
||||
finally
|
||||
{
|
||||
transaction.rollback();
|
||||
transaction.close();
|
||||
}
|
||||
var collection = collections.create("Aurora", "document", 1, 1, null, null);
|
||||
assertThat(RecordFilesStore.first(RecordFilesTables.COLLECTION, "id", collection.id()).getValueString("title")).isEqualTo("Aurora");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Aurora relaxes only full_page_writes: fsync off is refused.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void auroraWithFsyncOffIsRefused() throws Exception
|
||||
{
|
||||
server("full_page_writes", "off");
|
||||
simulateAurora();
|
||||
assertThat(RecordFilesStore.isDurableBackend(database)).isTrue();
|
||||
server("fsync", "off");
|
||||
assertThat(RecordFilesStore.isDurableBackend(database)).isFalse();
|
||||
assertThatThrownBy(RecordFilesStore::openTransaction).isInstanceOf(QException.class).hasMessageStartingWith(REFUSAL);
|
||||
server("fsync", "on");
|
||||
assertThat(RecordFilesStore.isDurableBackend(database)).isTrue();
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Aurora relaxes only full_page_writes: synchronous_commit off or local is refused.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void auroraWithSynchronousCommitOffIsRefused() throws Exception
|
||||
{
|
||||
server("full_page_writes", "off");
|
||||
simulateAurora();
|
||||
assertThat(RecordFilesStore.isDurableBackend(database)).isTrue();
|
||||
for(String weak : new String[] {"off", "local"})
|
||||
{
|
||||
execute("ALTER ROLE " + PostgresRecordFilesSupport.APP_ROLE + " SET synchronous_commit='" + weak + "'");
|
||||
assertThat(setting("synchronous_commit")).isEqualTo(weak);
|
||||
assertThat(RecordFilesStore.isDurableBackend(database)).isFalse();
|
||||
assertThatThrownBy(RecordFilesStore::openTransaction).isInstanceOf(QException.class).hasMessageStartingWith(REFUSAL);
|
||||
}
|
||||
execute("ALTER ROLE " + PostgresRecordFilesSupport.APP_ROLE + " RESET synchronous_commit");
|
||||
assertThat(RecordFilesStore.isDurableBackend(database)).isTrue();
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The stand-in for Aurora: a function with Aurora's name in the test database.
|
||||
*******************************************************************************/
|
||||
private void simulateAurora() throws SQLException
|
||||
{
|
||||
execute("CREATE FUNCTION public.aurora_version() RETURNS text LANGUAGE sql AS 'SELECT ''17.9.simulated''::text'");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Change one reload-time server setting and wait until new application sessions see it.
|
||||
*******************************************************************************/
|
||||
private void server(String name, String value) throws Exception
|
||||
{
|
||||
execute("ALTER SYSTEM SET " + name + " = " + value);
|
||||
execute("SELECT pg_reload_conf()");
|
||||
awaitSetting(name, value);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A reload is asynchronous; fail rather than assert against a stale value.
|
||||
*******************************************************************************/
|
||||
private void awaitSetting(String name, String value) throws Exception
|
||||
{
|
||||
long deadline = System.nanoTime() + Duration.ofSeconds(30).toNanos();
|
||||
while(!value.equals(setting(name)))
|
||||
{
|
||||
assertThat(System.nanoTime()).as("server setting " + name + " did not become " + value).isLessThan(deadline);
|
||||
Thread.sleep(50);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** One setting as a new application-role session observes it.
|
||||
*******************************************************************************/
|
||||
private String setting(String name) throws Exception
|
||||
{
|
||||
try(Connection connection = ConnectionManager.getConnection(database); var statement = connection.createStatement();
|
||||
var rows = statement.executeQuery("SELECT current_setting('" + name + "')"))
|
||||
{
|
||||
rows.next();
|
||||
return (rows.getString(1));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Column 12 of the shipped settings query, run as the application role.
|
||||
*******************************************************************************/
|
||||
private boolean auroraColumn() throws Exception
|
||||
{
|
||||
try(Connection connection = ConnectionManager.getConnection(database); var statement = connection.createStatement();
|
||||
var rows = statement.executeQuery(RecordFilesPostgres.SETTINGS))
|
||||
{
|
||||
assertThat(rows.next()).isTrue();
|
||||
assertThat(rows.getMetaData().getColumnCount()).isEqualTo(12);
|
||||
return (rows.getBoolean(12));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Fixture superuser statement, separate from native QQQ actions.
|
||||
*******************************************************************************/
|
||||
private void execute(String sql) throws SQLException
|
||||
{
|
||||
try(Connection connection = PostgresRecordFilesSupport.root(); var statement = connection.createStatement())
|
||||
{
|
||||
statement.execute(sql);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,90 @@
|
||||
/*
|
||||
* QQQ - Low-code Application Framework for Engineers.
|
||||
* Copyright (C) 2021-2025. Kingsrook, LLC
|
||||
* 651 N Broad St Ste 205 # 6917 | Middletown DE 19709 | United States
|
||||
* contact@kingsrook.com
|
||||
* https://github.com/Kingsrook/
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
|
||||
package com.kingsrook.qbits.recordfiles;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
/***************************************************************************
|
||||
** The PostgreSQL durability decision as a pure function of the observed
|
||||
** setting values. No server is used here, so this runs in every build; the
|
||||
** same rule on a real PostgreSQL 17 server is PostgresAuroraQualificationTest.
|
||||
***************************************************************************/
|
||||
class PostgresDurabilityRuleTest
|
||||
{
|
||||
/************************************************************************
|
||||
** Stock PostgreSQL keeps the strict rule: full_page_writes must be on.
|
||||
************************************************************************/
|
||||
@Test
|
||||
void stockPostgresRequiresFullPageWrites()
|
||||
{
|
||||
assertThat(RecordFilesPostgres.durable("on", "on", "on", false)).isTrue();
|
||||
assertThat(RecordFilesPostgres.durable("remote_apply", "on", "on", false)).isTrue();
|
||||
assertThat(RecordFilesPostgres.durable("on", "on", "off", false)).isFalse();
|
||||
assertThat(RecordFilesPostgres.durable("remote_apply", "on", "off", false)).isFalse();
|
||||
assertThat(RecordFilesPostgres.durable("on", "on", null, false)).isFalse();
|
||||
}
|
||||
|
||||
/************************************************************************
|
||||
** Amazon Aurora PostgreSQL is accepted with full_page_writes off (and on).
|
||||
************************************************************************/
|
||||
@Test
|
||||
void auroraAcceptsFullPageWritesOff()
|
||||
{
|
||||
assertThat(RecordFilesPostgres.durable("on", "on", "off", true)).isTrue();
|
||||
assertThat(RecordFilesPostgres.durable("remote_apply", "on", "off", true)).isTrue();
|
||||
assertThat(RecordFilesPostgres.durable("on", "on", "on", true)).isTrue();
|
||||
assertThat(RecordFilesPostgres.durable("on", "on", null, true)).isFalse();
|
||||
assertThat(RecordFilesPostgres.durable("on", "on", "unknown", true)).isFalse();
|
||||
}
|
||||
|
||||
/************************************************************************
|
||||
** Aurora relaxes nothing else: fsync and synchronous_commit stay required.
|
||||
************************************************************************/
|
||||
@Test
|
||||
void auroraStillRequiresFsyncAndSynchronousCommit()
|
||||
{
|
||||
for(String fullPageWrites : new String[] {"off", "on"})
|
||||
{
|
||||
assertThat(RecordFilesPostgres.durable("on", "off", fullPageWrites, true)).isFalse();
|
||||
assertThat(RecordFilesPostgres.durable("on", null, fullPageWrites, true)).isFalse();
|
||||
assertThat(RecordFilesPostgres.durable(null, "on", fullPageWrites, true)).isFalse();
|
||||
for(String synchronousCommit : new String[] {"off", "local", "remote_write"})
|
||||
{
|
||||
assertThat(RecordFilesPostgres.durable(synchronousCommit, "on", fullPageWrites, true)).isFalse();
|
||||
assertThat(RecordFilesPostgres.durable(synchronousCommit, "on", fullPageWrites, false)).isFalse();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/************************************************************************
|
||||
** The settings query carries the Aurora probe as its last column, in the
|
||||
** form that cannot raise on stock PostgreSQL. Its execution on a real
|
||||
** server is PostgresAuroraQualificationTest.settingsQueryReportsAurora.
|
||||
************************************************************************/
|
||||
@Test
|
||||
void settingsQueryProbesAuroraWithoutCallingTheFunction()
|
||||
{
|
||||
assertThat(RecordFilesPostgres.SETTINGS).endsWith(", to_regprocedure('aurora_version()') IS NOT NULL");
|
||||
assertThat(RecordFilesPostgres.SETTINGS).doesNotContain("SELECT aurora_version()").doesNotContain("pg_proc");
|
||||
}
|
||||
}
|
||||
@@ -51,7 +51,7 @@ public final class PostgresRecordFilesSupport
|
||||
{
|
||||
private static final String DATABASE = "record_files_test";
|
||||
private static final String OPTIONS = "?connectTimeout=5&socketTimeout=30&options=-c%20TimeZone%3DUTC";
|
||||
private static final String APP_ROLE = "record_files_app";
|
||||
public static final String APP_ROLE = "record_files_app";
|
||||
private static final String ROOT_PASSWORD = UUID.randomUUID().toString();
|
||||
private static final String APP_PASSWORD = UUID.randomUUID().toString();
|
||||
private static GenericContainer<?> postgres;
|
||||
@@ -75,11 +75,29 @@ public final class PostgresRecordFilesSupport
|
||||
*******************************************************************************/
|
||||
public static void start() throws Exception
|
||||
{
|
||||
start("fsync=on", "synchronous_commit=on", "full_page_writes=on");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Start the disposable server with exactly these name=value settings fixed on
|
||||
** its command line. A setting fixed there cannot be changed later by ALTER
|
||||
** SYSTEM, so a suite that changes a reload-time setting leaves it off this list.
|
||||
*******************************************************************************/
|
||||
public static void start(String... commandLineSettings) throws Exception
|
||||
{
|
||||
List<String> serverCommand = new ArrayList<>(List.of("postgres"));
|
||||
for(String setting : commandLineSettings)
|
||||
{
|
||||
serverCommand.add("-c");
|
||||
serverCommand.add(setting);
|
||||
}
|
||||
previousZone = TimeZone.getDefault();
|
||||
TimeZone.setDefault(TimeZone.getTimeZone("UTC"));
|
||||
postgres = new GenericContainer<>("postgres:17")
|
||||
.withEnv("POSTGRES_PASSWORD", ROOT_PASSWORD).withEnv("POSTGRES_DB", DATABASE)
|
||||
.withCommand("postgres", "-c", "fsync=on", "-c", "synchronous_commit=on", "-c", "full_page_writes=on")
|
||||
.withCommand(serverCommand.toArray(String[]::new))
|
||||
.withExposedPorts(5432)
|
||||
.withCreateContainerCmdModifier(command -> command.getHostConfig().withMemory(768L * 1024 * 1024)
|
||||
.withPortBindings(FixtureEndpoint.publishedBinding(5432)))
|
||||
|
||||
@@ -83,6 +83,10 @@ class UploadJsonTest
|
||||
VALID.replace("\"origin\":\"CAMERA\"", "\"origin\":0"),
|
||||
VALID.replace("\"origin\":\"CAMERA\"", "\"origin\":\"GENERATED\""),
|
||||
VALID.replace("\"title\":", "\"quarantineGeneratedFiles\":false,\"title\":"),
|
||||
VALID.replace("\"title\":", "\"quarantineCapturedFiles\":false,\"title\":"),
|
||||
VALID.replace("\"caption\":null", "\"quarantineCapturedFiles\":false,\"caption\":null"),
|
||||
VALID.replace("\"title\":", "\"admissionPolicy\":\"quarantineCapturedFiles\",\"title\":"),
|
||||
VALID.replace("\"origin\":\"CAMERA\"", "\"origin\":\"CAPTURE\""),
|
||||
VALID.replace("\"title\":", "\"title\":null,\"title\":"),
|
||||
VALID + "{}", "null", "[]", "{}");
|
||||
for(String json : invalid)
|
||||
|
||||
+69
-1
@@ -87,10 +87,78 @@ class UploadSessionServiceTest extends RecordFilesFixture
|
||||
** A synthetic manifest preserves its parent alias until owner authorization.
|
||||
*******************************************************************************/
|
||||
private UploadManifest manifest(String title)
|
||||
{
|
||||
return (manifest(title, UploadManifest.Origin.UPLOAD));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** The same synthetic manifest with the client-declared item origin chosen.
|
||||
*******************************************************************************/
|
||||
private UploadManifest manifest(String title, UploadManifest.Origin origin)
|
||||
{
|
||||
return (new UploadManifest(REQUEST, new UploadManifest.Parent("document", "01"), title,
|
||||
List.of(new UploadManifest.Item(ITEM, 0, "receipt.pdf", "application/pdf", (long) CONTENT.length, UploadManifest.hash(CONTENT),
|
||||
UploadManifest.Origin.UPLOAD, null, null, null))));
|
||||
origin, null, null, null))));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Create, upload and finalize one item; returns the committed lifecycle.
|
||||
*******************************************************************************/
|
||||
private String finalizeLifecycle(UploadManifest.Origin origin) throws Exception
|
||||
{
|
||||
String id = service.create(manifest("Policy", origin)).uploadId();
|
||||
service.putChunk(id, ITEM, 0, CONTENT.length, UploadManifest.hash(CONTENT), new ByteArrayInputStream(CONTENT));
|
||||
var completed = new UploadFinalizer(service, files, collections, clock).finalizeUpload(id, REQUEST);
|
||||
assertThat(completed.state()).isEqualTo("COMPLETE");
|
||||
return (completed.receipt().items().getFirst().lifecycleAtCommit());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Without the host opt-in, an approved-device camera item is still quarantined.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void deviceCameraUploadQuarantinesByDefault() throws Exception
|
||||
{
|
||||
identity.captured = new CaptureIdentity("test", "alice", "device-a", "public-a");
|
||||
assertThat(finalizeLifecycle(UploadManifest.Origin.CAMERA)).as("default").isEqualTo("QUARANTINED");
|
||||
assertThat(eventTypes()).doesNotContain("FILE_RELEASED");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Opted-in host: a camera item with the provider-resolved device identity
|
||||
** becomes AVAILABLE and audits its policy, with no release/review event.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void optedInHostMakesProviderVerifiedDeviceCameraItemAvailable() throws Exception
|
||||
{
|
||||
config.withQuarantineCapturedFiles(false);
|
||||
identity.captured = new CaptureIdentity("test", "alice", "device-a", "public-a");
|
||||
assertThat(finalizeLifecycle(UploadManifest.Origin.CAMERA)).isEqualTo("AVAILABLE");
|
||||
assertThat(events()).anySatisfy(event -> assertThat(event.getValueString("detail"))
|
||||
.contains("origin=CAPTURE", "admissionPolicy=quarantineCapturedFiles", "lifecycle=AVAILABLE"));
|
||||
assertThat(eventTypes()).doesNotContain("FILE_RELEASED", "REVIEW_CHANGE_ATTEMPT");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** A manifest origin label alone never opts in: no provider identity (desktop)
|
||||
** or a non-capture origin keeps quarantine even when the host opted in.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void manifestCaptureLabelWithoutProviderIdentityStaysQuarantined() throws Exception
|
||||
{
|
||||
config.withQuarantineCapturedFiles(false);
|
||||
assertThat(finalizeLifecycle(UploadManifest.Origin.CAMERA)).isEqualTo("QUARANTINED");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** An UPLOAD-origin item stays quarantined even with a device identity.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void nonCaptureItemsWithDeviceIdentityStayQuarantined() throws Exception
|
||||
{
|
||||
config.withQuarantineCapturedFiles(false);
|
||||
identity.captured = new CaptureIdentity("test", "alice", "device-a", "public-a");
|
||||
assertThat(finalizeLifecycle(UploadManifest.Origin.UPLOAD)).isEqualTo("QUARANTINED");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
|
||||
Reference in new issue
Block a user