feat: sealed exports to S3 Object Lock and the export verifier #7

2026-10-14T01:58:21Z

KofTwentyTwo commented

2026-10-14 01:58:21 +00:00

(Migrated from github.com)

Summary

Phase (b) of docs/DESIGN.md §6: sealed exports. Stacked on #6. This PR contains #6's commits until #6 merges; review only the last commit.

Exporter: daily (exportCron, 02:00 UTC) and "Export now" (a streamed-ETL table action on worm_audit_export, permission worm_audit.export).
- Exports contiguous ranges up to the latest signed checkpoint, streamed in 1,000-link pages to temp files: links.jsonl (link plus the exact canonical content), checkpoints.jsonl and manifest.json (file hashes, head hash, prevManifestSha256).
- Each object goes through qbit-object-storage with LockMode.COMPLIANCE and retain-until = now + retentionYears, and is checked by head. Version ids are recorded in worm_audit_export.object_versions.
- It refuses and alerts when the bucket lacks Object Lock or versioning. It takes the chainer's writer lock.
Verify sealed export: a record action that is also the restore tool.
- Checks the manifest hashes, COMPLIANCE and retention, and the manifest chain; detects an overwrite against the pinned version and still verifies the sealed version; recomputes the links and checkpoint signatures; compares with the live chain and live canonical content, naming the exact link.
- Purged links are verified from the sealed copy only. Writes a SEALED_EXPORT verification row.
Stand-in: qbit-object-storage RC.2 has no versioned GET, so ObjectStorageSealedStore.getVersion uses object storage's configured S3 connection (rate limiter, key normalisation). Replace it when object storage adds a version-aware get.
README: "Sealed exports" (bucket requirements and IAM).

Tests (CI=true ./mvnw verify with Docker): 67 pass, including a MinIO Testcontainers run with an Object Lock BACKUP bucket. It covers the round trip; COMPLIANCE on every object with retain-until about 6 years out; a version delete refused; overwrites detected while the sealed version verifies; a live tamper reported as "link 1 (audit 1): live content differs from the sealed copy"; and a non-lock bucket refused with an alert.

🤖 Generated with Claude Code

## Summary Phase (b) of `docs/DESIGN.md` §6: sealed exports. **Stacked on #6.** This PR contains #6's commits until #6 merges; review only the last commit. - **Exporter:** daily (`exportCron`, 02:00 UTC) and "Export now" (a streamed-ETL table action on `worm_audit_export`, permission `worm_audit.export`). - Exports contiguous ranges up to the latest signed checkpoint, streamed in 1,000-link pages to temp files: `links.jsonl` (link plus the exact canonical content), `checkpoints.jsonl` and `manifest.json` (file hashes, head hash, `prevManifestSha256`). - Each object goes through qbit-object-storage with `LockMode.COMPLIANCE` and retain-until = now + `retentionYears`, and is checked by `head`. Version ids are recorded in `worm_audit_export.object_versions`. - It refuses and alerts when the bucket lacks Object Lock or versioning. It takes the chainer's writer lock. - **Verify sealed export:** a record action that is also the restore tool. - Checks the manifest hashes, COMPLIANCE and retention, and the manifest chain; detects an overwrite against the pinned version and still verifies the sealed version; recomputes the links and checkpoint signatures; compares with the live chain and live canonical content, naming the exact link. - Purged links are verified from the sealed copy only. Writes a `SEALED_EXPORT` verification row. - **Stand-in:** qbit-object-storage RC.2 has no versioned GET, so `ObjectStorageSealedStore.getVersion` uses object storage's configured S3 connection (rate limiter, key normalisation). Replace it when object storage adds a version-aware `get`. - README: "Sealed exports" (bucket requirements and IAM). **Tests** (`CI=true ./mvnw verify` with Docker): 67 pass, including a MinIO Testcontainers run with an Object Lock BACKUP bucket. It covers the round trip; COMPLIANCE on every object with retain-until about 6 years out; a version delete refused; overwrites detected while the sealed version verifies; a live tamper reported as "link 1 (audit 1): live content differs from the sealed copy"; and a non-lock bucket refused with an alert. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: QRun-IO/qbit-worm-audit#7