feat: allow host-controlled admission for captured files #1

Merged
james.maes merged 1 commits from feat/captured-photo-admission-policy into develop 2026-11-06 16:43:29 +00:00
7 changed files with 237 additions and 5 deletions

No files matched your search

+26 -2
View File
@@ -68,9 +68,33 @@ applies to all trusted generated outputs, not only previews. Hosts must trust ev
in-process producer allowed to supply that origin; origin is provenance, not a
cryptographic attestation of safe content.
`quarantineCapturedFiles` is a second optional host setting with the same shape
(`getQuarantineCapturedFiles()` / `withQuarantineCapturedFiles(Boolean)`; null, the
default, inherits `quarantineNewFiles`). It applies only to intake with origin
`CAPTURE` **and** a nonblank device identity, and `GENERATED` keeps its own policy.
In the upload path that identity is the server-persisted capture identity that the
host's `CaptureIdentityProvider` resolved and revalidated; the manifest cannot carry
a device, actor, policy or `CAPTURE` label (strict JSON rejects them; `CAMERA` and
`GALLERY` items map to `CAPTURE`, so a device that passes the provider can label an
item either way). A desktop request with no provider identity, or any `UPLOAD`,
`SCANNER_IMPORT`, `IMPORT` or attachment-panel file, keeps `quarantineNewFiles`.
Direct Java `RetainedFileService.intake` callers are trusted host code, like
`GENERATED`: they could pass `CAPTURE` plus a device id, so hosts must not expose
that call to untrusted input.
Host prerequisites before setting it to `false`: a `CaptureIdentityProvider` that
actually approves each device and actor (a device id alone proves nothing, and Record
Files does not decide approval); an `UploadConfiguration` that bounds media types,
sizes and quotas to what the host accepts (for example JPEG/PNG only); and
parent-table visibility and permissions as before. Nothing scans content, so this
is a host trust decision, not a malware or human-review result. Existing quarantined
files are not released by configuration; release them through the normal release
process. Compatibility: the setting is additive and defaults to null, so existing
hosts and persisted data behave exactly as before.
Upload manifests cannot select `GENERATED` or change host policy. Uploaded,
captured, imported and standard attachment-panel originals retain
`quarantineNewFiles`. Initial admission records `FILE_REGISTERED` with the chosen
imported and standard attachment-panel originals, and captures without a device
identity, retain `quarantineNewFiles`. Initial admission records `FILE_REGISTERED` with the chosen
policy and lifecycle, without a release, scan or human-review claim. Hash failures
still produce `FAILED`; permissions, current parent authorization and mandatory
audit remain enforced. Changing configuration never releases existing quarantined
+2
View File
@@ -49,6 +49,8 @@ Final receipt: `{uploadId,collectionId,state,completedAt,items:[{itemId,fileId,v
Hard ceilings:20MiB per original,200MiB total,100items,64KiB–4MiB fixed chunk size,60–86400second staging lifetime,256KiB manifest JSON. Deployment limits can be lower and are frozen per upload. The original20MiB bound reflects the conditional single-PUT owner path, not a throughput benchmark. Accepted media types are declarations subject to quarantine/scanning, not verified file contents.
Initial lifecycle: finalized items are QUARANTINED unless the host sets the Java-only `quarantineCapturedFiles=false`, which makes only `CAMERA`/`GALLERY` items (origin `CAPTURE`) with the provider-resolved, revalidated device identity AVAILABLE. The manifest carries no policy, device or actor field and strict parsing rejects them. This is a host trust decision (approved devices, bounded media types), not a scan or review result; it emits no release event and leaves review status unchanged. See README "Not a 1.0".
Durable quota defaults:10 active uploads,512MiB reserved staging bytes and100 new uploads/hour per actor. Cancel/expiry does not immediately release capacity: bounded exact-key cleanup must finish and commit its cursor/outcome/quota release. `UploadMaintenance.cleanupOwn` permits currently authorized terminal cleanup. A trusted host scheduler can invoke bounded `cleanupBatch` under `record-files.maintain`; this does not grant content access. Expired finalization fences commit before irreversible staging deletion. Failed deletion/outcome retains cursor and quota for safe retry. Canonical originals are outside cleanup.
## Remaining integration and release evidence
@@ -81,6 +81,7 @@ public class RecordFilesQBitConfig implements QBitConfig
private List<String> parentTables = List.of();
private Boolean quarantineNewFiles = true;
private Boolean quarantineGeneratedFiles;
private Boolean quarantineCapturedFiles;
private Integer maxItemsPerCollection = 500;
private QCodeReference auditPort = new QCodeReference(TableAuditPort.class);
@@ -227,6 +228,32 @@ public class RecordFilesQBitConfig implements QBitConfig
/*******************************************************************************
** Optional trusted captured-intake policy. Null inherits quarantineNewFiles.
** Applies only to CAPTURE intake that carries a nonblank server-resolved device
** identity; never to GENERATED, UPLOAD, IMPORT or attachment-panel files.
*******************************************************************************/
public Boolean getQuarantineCapturedFiles()
{
return (quarantineCapturedFiles);
}
/*******************************************************************************
** Host decision for device capture intake, not a scan or human review result.
** False makes valid captured files AVAILABLE on registration. The host must
** wire a CaptureIdentityProvider that approves each device and bound the
** accepted media types; a device id alone proves no approval.
*******************************************************************************/
public RecordFilesQBitConfig withQuarantineCapturedFiles(Boolean quarantineCapturedFiles)
{
this.quarantineCapturedFiles = quarantineCapturedFiles;
return (this);
}
/*******************************************************************************
** Upper bound on the items of one collection.
*******************************************************************************/
@@ -279,7 +279,15 @@ public class RetainedFileService
boolean generatedPolicy = pending != null && RecordFilesValues.ORIGIN_GENERATED.equals(pending.request().origin())
&& config.getQuarantineGeneratedFiles() != null;
boolean quarantine = Boolean.TRUE.equals(generatedPolicy ? config.getQuarantineGeneratedFiles() : config.getQuarantineNewFiles());
// Captured policy needs the service-owned intake scope (never a plain standard upload), the CAPTURE origin
// and a nonblank device identity. UploadFinalizer supplies that identity only from the server-persisted,
// revalidated capture identity; a client cannot choose this policy through the upload JSON.
boolean capturedPolicy = !generatedPolicy && pending != null && RecordFilesValues.ORIGIN_CAPTURE.equals(pending.request().origin())
&& pending.request().deviceId() != null && !pending.request().deviceId().isBlank()
&& config.getQuarantineCapturedFiles() != null;
boolean quarantine = Boolean.TRUE.equals(generatedPolicy ? config.getQuarantineGeneratedFiles()
: capturedPolicy ? config.getQuarantineCapturedFiles() : config.getQuarantineNewFiles());
String admissionPolicy = generatedPolicy ? "quarantineGeneratedFiles" : capturedPolicy ? "quarantineCapturedFiles" : "quarantineNewFiles";
String lifecycle = mismatch ? RecordFilesValues.LIFECYCLE_FAILED
: quarantine ? RecordFilesValues.LIFECYCLE_QUARANTINED : RecordFilesValues.LIFECYCLE_AVAILABLE;
QRecord row = new QRecord()
@@ -313,7 +321,7 @@ public class RetainedFileService
audit.withChange(new RecordFilesEvent(RecordFilesEventType.FILE_REGISTERED, actor, correlationId).withRequestId(requestId)
.withFile(fileId, attachmentId).withSource(tableName, recordId)
.withDetail("lifecycle=" + lifecycle + ", origin=" + row.getValueString("origin") + ", version=" + row.getValueInteger("versionNumber")
+ ", admissionPolicy=" + (generatedPolicy ? "quarantineGeneratedFiles" : "quarantineNewFiles") + ", quarantine=" + quarantine
+ ", admissionPolicy=" + admissionPolicy + ", quarantine=" + quarantine
+ (mismatch ? ", declared SHA-256 did not match the stored bytes" : "")));
if(pending != null)
{
@@ -94,6 +94,105 @@ public class IntakeAndRetentionTest extends RecordFilesFixture
/*******************************************************************************
** Without the host opt-in, device capture keeps the product quarantine default.
*******************************************************************************/
@Test
void capturedIntakeQuarantinesByDefault() throws Exception
{
assertThat(config.getQuarantineCapturedFiles()).isNull();
RetainedFile captured = files.intake(new IntakeRequest("captured-default", "document", 1, "photo.jpg", sha("photo"))
.withOrigin(RecordFilesValues.ORIGIN_CAPTURE, "CAMERA").withDevice("device-a", null), stream("photo"));
assertThat(captured.lifecycle()).isEqualTo(RecordFilesValues.LIFECYCLE_QUARANTINED);
assertThat(events()).anySatisfy(event -> assertThat(event.getValueString("detail"))
.contains("lifecycle=QUARANTINED", "admissionPolicy=quarantineNewFiles", "quarantine=true"));
}
/*******************************************************************************
** An opted-in host makes a valid device capture AVAILABLE and audits the policy,
** with no release, review or scan event and an unchanged initial review status.
*******************************************************************************/
@Test
void capturedPolicyMakesValidDeviceCaptureAvailableWithoutReviewClaim() throws Exception
{
RetainedFile quarantined = files.intake(new IntakeRequest("baseline-upload", "document", 1, "baseline.jpg", sha("baseline"))
.withOrigin(RecordFilesValues.ORIGIN_UPLOAD, null), stream("baseline"));
config.withQuarantineCapturedFiles(false);
assertThat(config.getQuarantineNewFiles()).isTrue();
RetainedFile captured = files.intake(new IntakeRequest("captured-open", "document", 1, "photo.jpg", sha("photo"))
.withOrigin(RecordFilesValues.ORIGIN_CAPTURE, "CAMERA").withDevice("device-a", null), stream("photo"));
assertThat(captured.lifecycle()).isEqualTo(RecordFilesValues.LIFECYCLE_AVAILABLE);
assertThat(RecordFilesStore.first(RecordFilesTables.FILE, "id", captured.id()).getValueInteger("reviewStatusId"))
.isEqualTo(RecordFilesStore.first(RecordFilesTables.FILE, "id", quarantined.id()).getValueInteger("reviewStatusId"));
assertThat(events()).anySatisfy(event ->
{
assertThat(event.getValueString("eventType")).isEqualTo("FILE_REGISTERED");
assertThat(event.getValueString("detail")).contains("lifecycle=AVAILABLE", "admissionPolicy=quarantineCapturedFiles", "quarantine=false");
});
assertThat(eventTypes()).doesNotContain("FILE_RELEASED", "REVIEW_CHANGE_ATTEMPT");
}
/*******************************************************************************
** The captured opt-in needs CAPTURE origin plus a nonblank device identity.
** Other origins, standard attachments and missing identity keep quarantine,
** and the generated policy is neither widened nor narrowed by it.
*******************************************************************************/
@Test
void capturedPolicyDoesNotApplyToOtherOriginsOrMissingDeviceIdentity() throws Exception
{
config.withQuarantineCapturedFiles(false);
for(String origin : List.of(RecordFilesValues.ORIGIN_UPLOAD, RecordFilesValues.ORIGIN_IMPORT, RecordFilesValues.ORIGIN_GENERATED))
{
assertThat(files.intake(new IntakeRequest("other-" + origin, "document", 1, "input.jpg", sha("input"))
.withOrigin(origin, "CAMERA").withDevice("device-a", null), stream("input")).lifecycle())
.as(origin).isEqualTo(RecordFilesValues.LIFECYCLE_QUARANTINED);
}
int index = 0;
for(String device : new String[]{null, "", " "})
{
assertThat(files.intake(new IntakeRequest("no-device-" + index++, "document", 1, "nodevice.jpg", sha("nodevice"))
.withOrigin(RecordFilesValues.ORIGIN_CAPTURE, "CAMERA").withDevice(device, null), stream("nodevice")).lifecycle())
.as("device [" + device + "]").isEqualTo(RecordFilesValues.LIFECYCLE_QUARANTINED);
}
AttachmentEntry panel = attachments.upload("document", 1, "panel.pdf", "application/pdf", stream("pdf"), "CAMERA");
assertThat(RecordFilesStore.first(RecordFilesTables.FILE, "attachmentId", panel.id()).getValueString("lifecycle"))
.isEqualTo(RecordFilesValues.LIFECYCLE_QUARANTINED);
config.withQuarantineGeneratedFiles(true);
assertThat(files.intake(new IntakeRequest("generated-still-quarantined", "document", 1, "gen.pdf", sha("gen"))
.withOrigin(RecordFilesValues.ORIGIN_GENERATED, null).withDevice("device-a", null), stream("gen")).lifecycle())
.isEqualTo(RecordFilesValues.LIFECYCLE_QUARANTINED);
config.withQuarantineGeneratedFiles(false).withQuarantineCapturedFiles(true);
assertThat(files.intake(new IntakeRequest("generated-open", "document", 1, "gen2.pdf", sha("gen2"))
.withOrigin(RecordFilesValues.ORIGIN_GENERATED, null).withDevice("device-a", null), stream("gen2")).lifecycle())
.isEqualTo(RecordFilesValues.LIFECYCLE_AVAILABLE);
assertThat(events()).anySatisfy(event -> assertThat(event.getValueString("detail"))
.contains("origin=GENERATED", "admissionPolicy=quarantineGeneratedFiles"));
assertThat(eventTypes()).doesNotContain("FILE_RELEASED");
}
/*******************************************************************************
** A hash mismatch is FAILED and inaccessible even when captured files are open.
*******************************************************************************/
@Test
void capturedPolicyStillFailsHashMismatch() throws Exception
{
config.withQuarantineCapturedFiles(false);
IntakeRequest request = new IntakeRequest("captured-mismatch", "document", 1, "photo.jpg", sha("expected"))
.withOrigin(RecordFilesValues.ORIGIN_CAPTURE, "CAMERA").withDevice("device-a", null);
assertThatThrownBy(() -> files.intake(request, stream("different"))).isInstanceOf(QBadRequestException.class);
assertThat(RecordFilesStore.first(RecordFilesTables.FILE, "fileName", "photo.jpg").getValueString("lifecycle"))
.isEqualTo(RecordFilesValues.LIFECYCLE_FAILED);
}
/*******************************************************************************
** Generated policy changes no permissions, integrity or mandatory audit checks.
*******************************************************************************/
@@ -83,6 +83,10 @@ class UploadJsonTest
VALID.replace("\"origin\":\"CAMERA\"", "\"origin\":0"),
VALID.replace("\"origin\":\"CAMERA\"", "\"origin\":\"GENERATED\""),
VALID.replace("\"title\":", "\"quarantineGeneratedFiles\":false,\"title\":"),
VALID.replace("\"title\":", "\"quarantineCapturedFiles\":false,\"title\":"),
VALID.replace("\"caption\":null", "\"quarantineCapturedFiles\":false,\"caption\":null"),
VALID.replace("\"title\":", "\"admissionPolicy\":\"quarantineCapturedFiles\",\"title\":"),
VALID.replace("\"origin\":\"CAMERA\"", "\"origin\":\"CAPTURE\""),
VALID.replace("\"title\":", "\"title\":null,\"title\":"),
VALID + "{}", "null", "[]", "{}");
for(String json : invalid)
@@ -87,10 +87,78 @@ class UploadSessionServiceTest extends RecordFilesFixture
** A synthetic manifest preserves its parent alias until owner authorization.
*******************************************************************************/
private UploadManifest manifest(String title)
{
return (manifest(title, UploadManifest.Origin.UPLOAD));
}
/*******************************************************************************
** The same synthetic manifest with the client-declared item origin chosen.
*******************************************************************************/
private UploadManifest manifest(String title, UploadManifest.Origin origin)
{
return (new UploadManifest(REQUEST, new UploadManifest.Parent("document", "01"), title,
List.of(new UploadManifest.Item(ITEM, 0, "receipt.pdf", "application/pdf", (long) CONTENT.length, UploadManifest.hash(CONTENT),
UploadManifest.Origin.UPLOAD, null, null, null))));
origin, null, null, null))));
}
/*******************************************************************************
** Create, upload and finalize one item; returns the committed lifecycle.
*******************************************************************************/
private String finalizeLifecycle(UploadManifest.Origin origin) throws Exception
{
String id = service.create(manifest("Policy", origin)).uploadId();
service.putChunk(id, ITEM, 0, CONTENT.length, UploadManifest.hash(CONTENT), new ByteArrayInputStream(CONTENT));
var completed = new UploadFinalizer(service, files, collections, clock).finalizeUpload(id, REQUEST);
assertThat(completed.state()).isEqualTo("COMPLETE");
return (completed.receipt().items().getFirst().lifecycleAtCommit());
}
/*******************************************************************************
** Without the host opt-in, an approved-device camera item is still quarantined.
*******************************************************************************/
@Test
void deviceCameraUploadQuarantinesByDefault() throws Exception
{
identity.captured = new CaptureIdentity("test", "alice", "device-a", "public-a");
assertThat(finalizeLifecycle(UploadManifest.Origin.CAMERA)).as("default").isEqualTo("QUARANTINED");
assertThat(eventTypes()).doesNotContain("FILE_RELEASED");
}
/*******************************************************************************
** Opted-in host: a camera item with the provider-resolved device identity
** becomes AVAILABLE and audits its policy, with no release/review event.
*******************************************************************************/
@Test
void optedInHostMakesProviderVerifiedDeviceCameraItemAvailable() throws Exception
{
config.withQuarantineCapturedFiles(false);
identity.captured = new CaptureIdentity("test", "alice", "device-a", "public-a");
assertThat(finalizeLifecycle(UploadManifest.Origin.CAMERA)).isEqualTo("AVAILABLE");
assertThat(events()).anySatisfy(event -> assertThat(event.getValueString("detail"))
.contains("origin=CAPTURE", "admissionPolicy=quarantineCapturedFiles", "lifecycle=AVAILABLE"));
assertThat(eventTypes()).doesNotContain("FILE_RELEASED", "REVIEW_CHANGE_ATTEMPT");
}
/*******************************************************************************
** A manifest origin label alone never opts in: no provider identity (desktop)
** or a non-capture origin keeps quarantine even when the host opted in.
*******************************************************************************/
@Test
void manifestCaptureLabelWithoutProviderIdentityStaysQuarantined() throws Exception
{
config.withQuarantineCapturedFiles(false);
assertThat(finalizeLifecycle(UploadManifest.Origin.CAMERA)).isEqualTo("QUARANTINED");
}
/*******************************************************************************
** An UPLOAD-origin item stays quarantined even with a device identity.
*******************************************************************************/
@Test
void nonCaptureItemsWithDeviceIdentityStayQuarantined() throws Exception
{
config.withQuarantineCapturedFiles(false);
identity.captured = new CaptureIdentity("test", "alice", "device-a", "public-a");
assertThat(finalizeLifecycle(UploadManifest.Origin.UPLOAD)).isEqualTo("QUARANTINED");
}
/*******************************************************************************