WIP: fix: retain attachment cleanup across transaction and provider failures #45

2026-10-22T19:24:58Z

KofTwentyTwo commented

2026-10-22 19:24:58 +00:00

(Migrated from github.com)

Attachment deletion previously removed provider bytes before database deletion succeeded, leaving retained metadata pointing to missing content on rollback. This change commits durable cleanup intent with metadata deletion, removes bytes after commit, and retries provider failures through the host scheduler.

On PostgreSQL and other backends supporting transactional FOR UPDATE SKIP LOCKED, uploads commit their locator before provider I/O and lock it until their transaction ends. Metadata insertion retires that intent in the same transaction. Cleanup workers skip active uploads and serialize deletion per intent. Backends without row-lock support retain callback-based compensation.

A successful DELETE alone cannot settle a timed-out PUT: the PUT may finish afterward. Generic storage retains uncertain upload intents with persisted 60-second backoff and bounded 100-row sweeps. The new Resolve settled uploads table action lets an identified collaboration administrator supply provider completion/cancellation evidence, review the selection, and resolve it. Apply rechecks permission, locks the row, refuses existing references, deletes current bytes, and preserves resolvedAt/resolvedBy/settlementEvidence on the service-managed row. Repeated requests preserve the original receipt; scheduled retries exclude resolved receipts. Provider or receipt-write failure leaves the intent recoverable. Evidence is an operator attestation, not automatic proof of external provider completion.

Validation at dc65a82: full Maven verify passes 206 tests, zero failures/errors/skips; packaging, help coverage, architecture guardrails and JaCoCo gates pass. Real PostgreSQL tests cover rollback/commit, intent creation failure before I/O, concurrent workers, active-upload protection, late PUT completion, retained receipts, permission denial, reference protection, database/provider failures, and actual form/Review/Result execution with access revoked after Review. Abrupt child-JVM termination is tested with file storage. Real MinIO plain/versioned bucket tests verify deletion replay preserves unrelated objects. Kof22 draft PR338 adds the matching additive schema migration; seven focused PostgreSQL/H2/convention tests pass there.

Related: QRun-IO/qqq#1289. Keep draft and issue open for remote CI, coordinated artifact adoption and actual staging/rollback acceptance. Late completion is a deterministic fixture, not an observed MinIO timeout; JVM termination does not prove power-loss durability. Apply schema before runtime adoption. Preserve pending intents and resolved receipts during rollback; do not run older cleanup workers that ignore resolvedAt. See docs/ATTACHMENT-RECOVERY.md.

Attachment deletion previously removed provider bytes before database deletion succeeded, leaving retained metadata pointing to missing content on rollback. This change commits durable cleanup intent with metadata deletion, removes bytes after commit, and retries provider failures through the host scheduler. On PostgreSQL and other backends supporting transactional FOR UPDATE SKIP LOCKED, uploads commit their locator before provider I/O and lock it until their transaction ends. Metadata insertion retires that intent in the same transaction. Cleanup workers skip active uploads and serialize deletion per intent. Backends without row-lock support retain callback-based compensation. A successful DELETE alone cannot settle a timed-out PUT: the PUT may finish afterward. Generic storage retains uncertain upload intents with persisted 60-second backoff and bounded 100-row sweeps. The new Resolve settled uploads table action lets an identified collaboration administrator supply provider completion/cancellation evidence, review the selection, and resolve it. Apply rechecks permission, locks the row, refuses existing references, deletes current bytes, and preserves resolvedAt/resolvedBy/settlementEvidence on the service-managed row. Repeated requests preserve the original receipt; scheduled retries exclude resolved receipts. Provider or receipt-write failure leaves the intent recoverable. Evidence is an operator attestation, not automatic proof of external provider completion. Validation at dc65a82: full Maven verify passes 206 tests, zero failures/errors/skips; packaging, help coverage, architecture guardrails and JaCoCo gates pass. Real PostgreSQL tests cover rollback/commit, intent creation failure before I/O, concurrent workers, active-upload protection, late PUT completion, retained receipts, permission denial, reference protection, database/provider failures, and actual form/Review/Result execution with access revoked after Review. Abrupt child-JVM termination is tested with file storage. Real MinIO plain/versioned bucket tests verify deletion replay preserves unrelated objects. Kof22 draft PR338 adds the matching additive schema migration; seven focused PostgreSQL/H2/convention tests pass there. Related: QRun-IO/qqq#1289. Keep draft and issue open for remote CI, coordinated artifact adoption and actual staging/rollback acceptance. Late completion is a deterministic fixture, not an observed MinIO timeout; JVM termination does not prove power-loss durability. Apply schema before runtime adoption. Preserve pending intents and resolved receipts during rollback; do not run older cleanup workers that ignore resolvedAt. See docs/ATTACHMENT-RECOVERY.md.

You are not authorized to merge this pull request.

This pull request can be merged automatically.

This pull request is marked as a work in progress.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin fix/attachment-transactions-1289:fix/attachment-transactions-1289

git checkout fix/attachment-transactions-1289

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: QRun-IO/qbit-collaboration#45