feat: allow host-controlled admission for captured files #1
No files matched your search
@@ -68,9 +68,33 @@ applies to all trusted generated outputs, not only previews. Hosts must trust ev
|
||||
in-process producer allowed to supply that origin; origin is provenance, not a
|
||||
cryptographic attestation of safe content.
|
||||
|
||||
`quarantineCapturedFiles` is a second optional host setting with the same shape
|
||||
(`getQuarantineCapturedFiles()` / `withQuarantineCapturedFiles(Boolean)`; null, the
|
||||
default, inherits `quarantineNewFiles`). It applies only to intake with origin
|
||||
`CAPTURE` **and** a nonblank device identity, and `GENERATED` keeps its own policy.
|
||||
In the upload path that identity is the server-persisted capture identity that the
|
||||
host's `CaptureIdentityProvider` resolved and revalidated; the manifest cannot carry
|
||||
a device, actor, policy or `CAPTURE` label (strict JSON rejects them; `CAMERA` and
|
||||
`GALLERY` items map to `CAPTURE`, so a device that passes the provider can label an
|
||||
item either way). A desktop request with no provider identity, or any `UPLOAD`,
|
||||
`SCANNER_IMPORT`, `IMPORT` or attachment-panel file, keeps `quarantineNewFiles`.
|
||||
Direct Java `RetainedFileService.intake` callers are trusted host code, like
|
||||
`GENERATED`: they could pass `CAPTURE` plus a device id, so hosts must not expose
|
||||
that call to untrusted input.
|
||||
|
||||
Host prerequisites before setting it to `false`: a `CaptureIdentityProvider` that
|
||||
actually approves each device and actor (a device id alone proves nothing, and Record
|
||||
Files does not decide approval); an `UploadConfiguration` that bounds media types,
|
||||
sizes and quotas to what the host accepts (for example JPEG/PNG only); and
|
||||
parent-table visibility and permissions as before. Nothing scans content, so this
|
||||
is a host trust decision, not a malware or human-review result. Existing quarantined
|
||||
files are not released by configuration; release them through the normal release
|
||||
process. Compatibility: the setting is additive and defaults to null, so existing
|
||||
hosts and persisted data behave exactly as before.
|
||||
|
||||
Upload manifests cannot select `GENERATED` or change host policy. Uploaded,
|
||||
captured, imported and standard attachment-panel originals retain
|
||||
`quarantineNewFiles`. Initial admission records `FILE_REGISTERED` with the chosen
|
||||
imported and standard attachment-panel originals, and captures without a device
|
||||
identity, retain `quarantineNewFiles`. Initial admission records `FILE_REGISTERED` with the chosen
|
||||
policy and lifecycle, without a release, scan or human-review claim. Hash failures
|
||||
still produce `FAILED`; permissions, current parent authorization and mandatory
|
||||
audit remain enforced. Changing configuration never releases existing quarantined
|
||||
|
||||
@@ -49,6 +49,8 @@ Final receipt: `{uploadId,collectionId,state,completedAt,items:[{itemId,fileId,v
|
||||
|
||||
Hard ceilings:20MiB per original,200MiB total,100items,64KiB–4MiB fixed chunk size,60–86400second staging lifetime,256KiB manifest JSON. Deployment limits can be lower and are frozen per upload. The original20MiB bound reflects the conditional single-PUT owner path, not a throughput benchmark. Accepted media types are declarations subject to quarantine/scanning, not verified file contents.
|
||||
|
||||
Initial lifecycle: finalized items are QUARANTINED unless the host sets the Java-only `quarantineCapturedFiles=false`, which makes only `CAMERA`/`GALLERY` items (origin `CAPTURE`) with the provider-resolved, revalidated device identity AVAILABLE. The manifest carries no policy, device or actor field and strict parsing rejects them. This is a host trust decision (approved devices, bounded media types), not a scan or review result; it emits no release event and leaves review status unchanged. See README "Not a 1.0".
|
||||
|
||||
Durable quota defaults:10 active uploads,512MiB reserved staging bytes and100 new uploads/hour per actor. Cancel/expiry does not immediately release capacity: bounded exact-key cleanup must finish and commit its cursor/outcome/quota release. `UploadMaintenance.cleanupOwn` permits currently authorized terminal cleanup. A trusted host scheduler can invoke bounded `cleanupBatch` under `record-files.maintain`; this does not grant content access. Expired finalization fences commit before irreversible staging deletion. Failed deletion/outcome retains cursor and quota for safe retry. Canonical originals are outside cleanup.
|
||||
|
||||
## Remaining integration and release evidence
|
||||
|
||||
@@ -81,6 +81,7 @@ public class RecordFilesQBitConfig implements QBitConfig
|
||||
private List<String> parentTables = List.of();
|
||||
private Boolean quarantineNewFiles = true;
|
||||
private Boolean quarantineGeneratedFiles;
|
||||
private Boolean quarantineCapturedFiles;
|
||||
private Integer maxItemsPerCollection = 500;
|
||||
private QCodeReference auditPort = new QCodeReference(TableAuditPort.class);
|
||||
|
||||
@@ -227,6 +228,32 @@ public class RecordFilesQBitConfig implements QBitConfig
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Optional trusted captured-intake policy. Null inherits quarantineNewFiles.
|
||||
** Applies only to CAPTURE intake that carries a nonblank server-resolved device
|
||||
** identity; never to GENERATED, UPLOAD, IMPORT or attachment-panel files.
|
||||
*******************************************************************************/
|
||||
public Boolean getQuarantineCapturedFiles()
|
||||
{
|
||||
return (quarantineCapturedFiles);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Host decision for device capture intake, not a scan or human review result.
|
||||
** False makes valid captured files AVAILABLE on registration. The host must
|
||||
** wire a CaptureIdentityProvider that approves each device and bound the
|
||||
** accepted media types; a device id alone proves no approval.
|
||||
*******************************************************************************/
|
||||
public RecordFilesQBitConfig withQuarantineCapturedFiles(Boolean quarantineCapturedFiles)
|
||||
{
|
||||
this.quarantineCapturedFiles = quarantineCapturedFiles;
|
||||
return (this);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Upper bound on the items of one collection.
|
||||
*******************************************************************************/
|
||||
|
||||
@@ -279,7 +279,15 @@ public class RetainedFileService
|
||||
|
||||
boolean generatedPolicy = pending != null && RecordFilesValues.ORIGIN_GENERATED.equals(pending.request().origin())
|
||||
&& config.getQuarantineGeneratedFiles() != null;
|
||||
boolean quarantine = Boolean.TRUE.equals(generatedPolicy ? config.getQuarantineGeneratedFiles() : config.getQuarantineNewFiles());
|
||||
// Captured policy needs the service-owned intake scope (never a plain standard upload), the CAPTURE origin
|
||||
// and a nonblank device identity. UploadFinalizer supplies that identity only from the server-persisted,
|
||||
// revalidated capture identity; a client cannot choose this policy through the upload JSON.
|
||||
boolean capturedPolicy = !generatedPolicy && pending != null && RecordFilesValues.ORIGIN_CAPTURE.equals(pending.request().origin())
|
||||
&& pending.request().deviceId() != null && !pending.request().deviceId().isBlank()
|
||||
&& config.getQuarantineCapturedFiles() != null;
|
||||
boolean quarantine = Boolean.TRUE.equals(generatedPolicy ? config.getQuarantineGeneratedFiles()
|
||||
: capturedPolicy ? config.getQuarantineCapturedFiles() : config.getQuarantineNewFiles());
|
||||
String admissionPolicy = generatedPolicy ? "quarantineGeneratedFiles" : capturedPolicy ? "quarantineCapturedFiles" : "quarantineNewFiles";
|
||||
String lifecycle = mismatch ? RecordFilesValues.LIFECYCLE_FAILED
|
||||
: quarantine ? RecordFilesValues.LIFECYCLE_QUARANTINED : RecordFilesValues.LIFECYCLE_AVAILABLE;
|
||||
QRecord row = new QRecord()
|
||||
@@ -313,7 +321,7 @@ public class RetainedFileService
|
||||
audit.withChange(new RecordFilesEvent(RecordFilesEventType.FILE_REGISTERED, actor, correlationId).withRequestId(requestId)
|
||||
.withFile(fileId, attachmentId).withSource(tableName, recordId)
|
||||
.withDetail("lifecycle=" + lifecycle + ", origin=" + row.getValueString("origin") + ", version=" + row.getValueInteger("versionNumber")
|
||||
+ ", admissionPolicy=" + (generatedPolicy ? "quarantineGeneratedFiles" : "quarantineNewFiles") + ", quarantine=" + quarantine
|
||||
+ ", admissionPolicy=" + admissionPolicy + ", quarantine=" + quarantine
|
||||
+ (mismatch ? ", declared SHA-256 did not match the stored bytes" : "")));
|
||||
if(pending != null)
|
||||
{
|
||||
|
||||
@@ -94,6 +94,105 @@ public class IntakeAndRetentionTest extends RecordFilesFixture
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Without the host opt-in, device capture keeps the product quarantine default.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void capturedIntakeQuarantinesByDefault() throws Exception
|
||||
{
|
||||
assertThat(config.getQuarantineCapturedFiles()).isNull();
|
||||
RetainedFile captured = files.intake(new IntakeRequest("captured-default", "document", 1, "photo.jpg", sha("photo"))
|
||||
.withOrigin(RecordFilesValues.ORIGIN_CAPTURE, "CAMERA").withDevice("device-a", null), stream("photo"));
|
||||
assertThat(captured.lifecycle()).isEqualTo(RecordFilesValues.LIFECYCLE_QUARANTINED);
|
||||
assertThat(events()).anySatisfy(event -> assertThat(event.getValueString("detail"))
|
||||
.contains("lifecycle=QUARANTINED", "admissionPolicy=quarantineNewFiles", "quarantine=true"));
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** An opted-in host makes a valid device capture AVAILABLE and audits the policy,
|
||||
** with no release, review or scan event and an unchanged initial review status.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void capturedPolicyMakesValidDeviceCaptureAvailableWithoutReviewClaim() throws Exception
|
||||
{
|
||||
RetainedFile quarantined = files.intake(new IntakeRequest("baseline-upload", "document", 1, "baseline.jpg", sha("baseline"))
|
||||
.withOrigin(RecordFilesValues.ORIGIN_UPLOAD, null), stream("baseline"));
|
||||
config.withQuarantineCapturedFiles(false);
|
||||
assertThat(config.getQuarantineNewFiles()).isTrue();
|
||||
RetainedFile captured = files.intake(new IntakeRequest("captured-open", "document", 1, "photo.jpg", sha("photo"))
|
||||
.withOrigin(RecordFilesValues.ORIGIN_CAPTURE, "CAMERA").withDevice("device-a", null), stream("photo"));
|
||||
assertThat(captured.lifecycle()).isEqualTo(RecordFilesValues.LIFECYCLE_AVAILABLE);
|
||||
assertThat(RecordFilesStore.first(RecordFilesTables.FILE, "id", captured.id()).getValueInteger("reviewStatusId"))
|
||||
.isEqualTo(RecordFilesStore.first(RecordFilesTables.FILE, "id", quarantined.id()).getValueInteger("reviewStatusId"));
|
||||
assertThat(events()).anySatisfy(event ->
|
||||
{
|
||||
assertThat(event.getValueString("eventType")).isEqualTo("FILE_REGISTERED");
|
||||
assertThat(event.getValueString("detail")).contains("lifecycle=AVAILABLE", "admissionPolicy=quarantineCapturedFiles", "quarantine=false");
|
||||
});
|
||||
assertThat(eventTypes()).doesNotContain("FILE_RELEASED", "REVIEW_CHANGE_ATTEMPT");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** The captured opt-in needs CAPTURE origin plus a nonblank device identity.
|
||||
** Other origins, standard attachments and missing identity keep quarantine,
|
||||
** and the generated policy is neither widened nor narrowed by it.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void capturedPolicyDoesNotApplyToOtherOriginsOrMissingDeviceIdentity() throws Exception
|
||||
{
|
||||
config.withQuarantineCapturedFiles(false);
|
||||
for(String origin : List.of(RecordFilesValues.ORIGIN_UPLOAD, RecordFilesValues.ORIGIN_IMPORT, RecordFilesValues.ORIGIN_GENERATED))
|
||||
{
|
||||
assertThat(files.intake(new IntakeRequest("other-" + origin, "document", 1, "input.jpg", sha("input"))
|
||||
.withOrigin(origin, "CAMERA").withDevice("device-a", null), stream("input")).lifecycle())
|
||||
.as(origin).isEqualTo(RecordFilesValues.LIFECYCLE_QUARANTINED);
|
||||
}
|
||||
int index = 0;
|
||||
for(String device : new String[]{null, "", " "})
|
||||
{
|
||||
assertThat(files.intake(new IntakeRequest("no-device-" + index++, "document", 1, "nodevice.jpg", sha("nodevice"))
|
||||
.withOrigin(RecordFilesValues.ORIGIN_CAPTURE, "CAMERA").withDevice(device, null), stream("nodevice")).lifecycle())
|
||||
.as("device [" + device + "]").isEqualTo(RecordFilesValues.LIFECYCLE_QUARANTINED);
|
||||
}
|
||||
AttachmentEntry panel = attachments.upload("document", 1, "panel.pdf", "application/pdf", stream("pdf"), "CAMERA");
|
||||
assertThat(RecordFilesStore.first(RecordFilesTables.FILE, "attachmentId", panel.id()).getValueString("lifecycle"))
|
||||
.isEqualTo(RecordFilesValues.LIFECYCLE_QUARANTINED);
|
||||
|
||||
config.withQuarantineGeneratedFiles(true);
|
||||
assertThat(files.intake(new IntakeRequest("generated-still-quarantined", "document", 1, "gen.pdf", sha("gen"))
|
||||
.withOrigin(RecordFilesValues.ORIGIN_GENERATED, null).withDevice("device-a", null), stream("gen")).lifecycle())
|
||||
.isEqualTo(RecordFilesValues.LIFECYCLE_QUARANTINED);
|
||||
config.withQuarantineGeneratedFiles(false).withQuarantineCapturedFiles(true);
|
||||
assertThat(files.intake(new IntakeRequest("generated-open", "document", 1, "gen2.pdf", sha("gen2"))
|
||||
.withOrigin(RecordFilesValues.ORIGIN_GENERATED, null).withDevice("device-a", null), stream("gen2")).lifecycle())
|
||||
.isEqualTo(RecordFilesValues.LIFECYCLE_AVAILABLE);
|
||||
assertThat(events()).anySatisfy(event -> assertThat(event.getValueString("detail"))
|
||||
.contains("origin=GENERATED", "admissionPolicy=quarantineGeneratedFiles"));
|
||||
assertThat(eventTypes()).doesNotContain("FILE_RELEASED");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** A hash mismatch is FAILED and inaccessible even when captured files are open.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void capturedPolicyStillFailsHashMismatch() throws Exception
|
||||
{
|
||||
config.withQuarantineCapturedFiles(false);
|
||||
IntakeRequest request = new IntakeRequest("captured-mismatch", "document", 1, "photo.jpg", sha("expected"))
|
||||
.withOrigin(RecordFilesValues.ORIGIN_CAPTURE, "CAMERA").withDevice("device-a", null);
|
||||
assertThatThrownBy(() -> files.intake(request, stream("different"))).isInstanceOf(QBadRequestException.class);
|
||||
assertThat(RecordFilesStore.first(RecordFilesTables.FILE, "fileName", "photo.jpg").getValueString("lifecycle"))
|
||||
.isEqualTo(RecordFilesValues.LIFECYCLE_FAILED);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/*******************************************************************************
|
||||
** Generated policy changes no permissions, integrity or mandatory audit checks.
|
||||
*******************************************************************************/
|
||||
|
||||
@@ -83,6 +83,10 @@ class UploadJsonTest
|
||||
VALID.replace("\"origin\":\"CAMERA\"", "\"origin\":0"),
|
||||
VALID.replace("\"origin\":\"CAMERA\"", "\"origin\":\"GENERATED\""),
|
||||
VALID.replace("\"title\":", "\"quarantineGeneratedFiles\":false,\"title\":"),
|
||||
VALID.replace("\"title\":", "\"quarantineCapturedFiles\":false,\"title\":"),
|
||||
VALID.replace("\"caption\":null", "\"quarantineCapturedFiles\":false,\"caption\":null"),
|
||||
VALID.replace("\"title\":", "\"admissionPolicy\":\"quarantineCapturedFiles\",\"title\":"),
|
||||
VALID.replace("\"origin\":\"CAMERA\"", "\"origin\":\"CAPTURE\""),
|
||||
VALID.replace("\"title\":", "\"title\":null,\"title\":"),
|
||||
VALID + "{}", "null", "[]", "{}");
|
||||
for(String json : invalid)
|
||||
|
||||
+69
-1
@@ -87,10 +87,78 @@ class UploadSessionServiceTest extends RecordFilesFixture
|
||||
** A synthetic manifest preserves its parent alias until owner authorization.
|
||||
*******************************************************************************/
|
||||
private UploadManifest manifest(String title)
|
||||
{
|
||||
return (manifest(title, UploadManifest.Origin.UPLOAD));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** The same synthetic manifest with the client-declared item origin chosen.
|
||||
*******************************************************************************/
|
||||
private UploadManifest manifest(String title, UploadManifest.Origin origin)
|
||||
{
|
||||
return (new UploadManifest(REQUEST, new UploadManifest.Parent("document", "01"), title,
|
||||
List.of(new UploadManifest.Item(ITEM, 0, "receipt.pdf", "application/pdf", (long) CONTENT.length, UploadManifest.hash(CONTENT),
|
||||
UploadManifest.Origin.UPLOAD, null, null, null))));
|
||||
origin, null, null, null))));
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Create, upload and finalize one item; returns the committed lifecycle.
|
||||
*******************************************************************************/
|
||||
private String finalizeLifecycle(UploadManifest.Origin origin) throws Exception
|
||||
{
|
||||
String id = service.create(manifest("Policy", origin)).uploadId();
|
||||
service.putChunk(id, ITEM, 0, CONTENT.length, UploadManifest.hash(CONTENT), new ByteArrayInputStream(CONTENT));
|
||||
var completed = new UploadFinalizer(service, files, collections, clock).finalizeUpload(id, REQUEST);
|
||||
assertThat(completed.state()).isEqualTo("COMPLETE");
|
||||
return (completed.receipt().items().getFirst().lifecycleAtCommit());
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Without the host opt-in, an approved-device camera item is still quarantined.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void deviceCameraUploadQuarantinesByDefault() throws Exception
|
||||
{
|
||||
identity.captured = new CaptureIdentity("test", "alice", "device-a", "public-a");
|
||||
assertThat(finalizeLifecycle(UploadManifest.Origin.CAMERA)).as("default").isEqualTo("QUARANTINED");
|
||||
assertThat(eventTypes()).doesNotContain("FILE_RELEASED");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** Opted-in host: a camera item with the provider-resolved device identity
|
||||
** becomes AVAILABLE and audits its policy, with no release/review event.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void optedInHostMakesProviderVerifiedDeviceCameraItemAvailable() throws Exception
|
||||
{
|
||||
config.withQuarantineCapturedFiles(false);
|
||||
identity.captured = new CaptureIdentity("test", "alice", "device-a", "public-a");
|
||||
assertThat(finalizeLifecycle(UploadManifest.Origin.CAMERA)).isEqualTo("AVAILABLE");
|
||||
assertThat(events()).anySatisfy(event -> assertThat(event.getValueString("detail"))
|
||||
.contains("origin=CAPTURE", "admissionPolicy=quarantineCapturedFiles", "lifecycle=AVAILABLE"));
|
||||
assertThat(eventTypes()).doesNotContain("FILE_RELEASED", "REVIEW_CHANGE_ATTEMPT");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** A manifest origin label alone never opts in: no provider identity (desktop)
|
||||
** or a non-capture origin keeps quarantine even when the host opted in.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void manifestCaptureLabelWithoutProviderIdentityStaysQuarantined() throws Exception
|
||||
{
|
||||
config.withQuarantineCapturedFiles(false);
|
||||
assertThat(finalizeLifecycle(UploadManifest.Origin.CAMERA)).isEqualTo("QUARANTINED");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
** An UPLOAD-origin item stays quarantined even with a device identity.
|
||||
*******************************************************************************/
|
||||
@Test
|
||||
void nonCaptureItemsWithDeviceIdentityStayQuarantined() throws Exception
|
||||
{
|
||||
config.withQuarantineCapturedFiles(false);
|
||||
identity.captured = new CaptureIdentity("test", "alice", "device-a", "public-a");
|
||||
assertThat(finalizeLifecycle(UploadManifest.Origin.UPLOAD)).isEqualTo("QUARANTINED");
|
||||
}
|
||||
|
||||
/*******************************************************************************
|
||||
|
||||
Reference in new issue
Block a user