qbit-object-storage/CHANGELOG.md

Changelog

All notable changes to this project are documented here. The format follows Keep a Changelog; versions follow semantic versioning.

[0.1.0] - Unreleased

Added

• Central health supplier object_storage.connection.<id> (DEPENDENCY/STORAGE, 300s interval, 20000ms timeout), reusing bounded read-only capability probes with safe problem kinds and clock-skew details. Turn a connection's check off in the central health_check table.
• In-memory health tests for registration, refresh, UP/DOWN/DEGRADED/UNKNOWN outcomes, deadlines, read-only execution and sentinel-secret exclusion.
• Expanded the host-neutral in-app help bundle to cover storage fields, actions, widgets and possible values, with live references and a HelpBundleAssertions coverage gate. Registered it for QQQ startup sync and HOST overrides; retained inline table/app help for hosts without help-content tables.
• Every table explains itself: a help bundle (help/qbit-object-storage.yaml, QQQ help-bundle format) says what each object_storage_* table is for and how its rows get created; it shows on the table screen and when the table is empty. Applied inline by ObjectStorageHelp for hosts without help-content tables; a host's own help is kept.
• A Setup card on the Storage home (connections, buckets, header rules, protection rules) with the state and next step of each, resolved the way the runtime resolves them.
• A starter changelog db/liquibase/qbit-object-storage/object-storage-defaults.changelog.sql: a disabled default-mutable header rule. Insert-only, run through real Liquibase in DefaultsChangelogTest.
• ObjectStorageTablesGuardrailTest (each table is used at runtime, seeded or explained, and has help).
• The presign limits on the bucket row (maxPresignGetSeconds, maxPresignPutSeconds) are now read at runtime (ConfigSync.effectiveMaxPresignGetSeconds / Put) as a lower cap: the effective limit is the smaller of the code config and the row. It applies to presigned GET, PUT, POST, previews and the Create download link action. Previously the row value was never read.
• object_storage_object_version is documented as reserved and not used yet (the indexer lists current versions only).
• StorageBucket.maxUploadBytes() in the api package: the bucket's upload limit in bytes (null when none), so other qbits need not call config(), which returns an internal type.
• The service (P1). ObjectStorage.bucket(alias) and ObjectStorage.asSystem(caller, runId): put (multipart above the part size), get with ranges, head, copy (in parts for very large objects), header and metadata changes, delete (delete marker, private trash, or permanent), restore, list, presigned GET, PUT and POST, public URLs, tags. Key building and URL encoding, the header policy (object_storage_header_rule), PublicUrlResolver, StorageChangeListener, StorageReferenceProvider, StorageAlertSink.
• Configuration in code, upserted into visible object_storage_* tables; secret references with an allow-list; per-connection rate limits; per-bucket and per-connection record security locks.
• The capability model and the Test connection probe (read-only by default, write tests on request, clock skew, presign host), with seeded provider defaults.
• The object_storage_operation audit log (presigned URLs are logged without their signature) and protection rules (READ_ONLY, NO_DELETE, WARN, with allowed system callers).
• Backups follow-ups SF1 to SF5: BucketRole.BACKUP, object lock on put, system-caller permanent delete, probes that skip locked backup buckets, and the system-caller context.
• The index (P2). object_storage_object and object_storage_folder kept in step with each indexed bucket: resumable full scan with a mass-removal safety stop, HEAD backlog, write-through on every service write, soft-deleted rows, folder rollups. Orphaned and missing objects from FieldReferenceProvider, daily usage snapshots, quota and freshness alerts, the nightly job.
• The Storage app (P3, P4). Admin -> Storage with a responsive home; record screens with sections, child lists and a thumbnail preview; actions Rescan bucket, Check configuration, Refresh from bucket, Apply header policy (replaces app-specific backfills), Change headers and metadata, Edit tags, Copy, Move or rename, Delete, Restore, Create download link, Create upload link, folder rename and delete, Upload files, Test connection; every one a standard Review and Result with a bulk cap.
• ObjectStorageSchema: PostgreSQL DDL generated from the table metadata.
• Foreign keys on every relation between the qbit tables (fk_<table>_<column>), generated by ObjectStorageSchema: owned observed state (capabilities, folders, objects, versions, usage) cascades from its bucket or object, object.folder_id and object_reference.object_id are set null, and a bucket's connection, header and protection rules and operation log restrict. Each key is added by an idempotent block that first records rows whose parent is gone in qqq_fk_quarantine and clears (nullable column) or deletes (required column) them, so it also upgrades an existing schema. New indexes ix_object_storage_bucket_1 (connection_id), ix_object_storage_header_rule_1 and ix_object_storage_protection_rule_1 (bucket_id) back the keys. Upgrade: run the generated statements again (or the host's copy of them).
• Conformance suite against MinIO and Garage (Testcontainers), service, index and action tests, and the index and actions on PostgreSQL.

Fixed

• The service's own writes are no longer refused in a host that enforces permissions inside the core actions (QInstance.enforcePermissionsInCoreActions). Db.asSystem starts a new context, so each of its writes is an outermost action there and core asks ObjectStoragePermissionChecker, which refuses every write of a table nobody may write directly, to every session. The object_storage_operation row of every stored object was lost (logged as "Could not write a storage operation log row", the object itself was stored), and so was each other system write of those tables. Db.insert, Db.update and Db.delete now mark their input with core's per-call permission bypass (withPermissionBypass, reason Db.OWN_WRITE_REASON), so core does not ask about the service's own writes. The checker is unchanged in what it refuses: a write of those tables that does not carry a bypass is refused to a person, to object_storage.admin and to the core system user alike, so server code running as the system user cannot edit or delete operation log rows through an enforced action.
• An operation log row that could not be written is no longer only a warning. It is logged at ERROR, raises the storage.operation_log.failed alert (StorageAlertSink, severity error, dedupe key bucket:<alias>, resolved by the next row written for that bucket) and is counted in OperationLog.failedWriteCount() (per node, since start). A row the insert action hands back with a record error (a refused value, a record lock) is now detected as well; before, it was dropped without any log line from the qbit. The operation the row describes still succeeds.

Removed

• The QRun-IO/qqq#1165 stand-ins: SessionCredentialsClient (session tokens now go through core S3BackendMetaData.sessionToken / S3ClientFactory, including for the presigner) and the file reading in SecretRefs (core QMetaDataVariableInterpreter resolves ${file.PATH}). Behavior change: core reads ${file.PATH} only inside its allowed roots, which are /var/run/secrets and /run/secrets by default. A host that mounts storage secrets elsewhere must set qqq.fileReference.roots or QQQ_FILE_REFERENCE_ROOTS. The qbit's default allow-list (file./var/run/secrets/storage/*) is already inside the default roots.
• None for the health migration; both Test connection record actions, including on-demand write tests, are retained.

Changed

• Multi-node deployments: the runtime caches are registered with QQQ ClusterCaches. Shared (cleared on every node): the bucket rows' presign limits (qbit-object-storage.bucket-row-limits, on a bucket row insert or update), the enabled header rules (qbit-object-storage.header-rules, on a header rule insert, update or delete), both through the new ConfigRowChangeCustomizer, and the capability statuses (qbit-object-storage.capabilities: a probe result recorded on one node clears that capability on the others, which reload it from the capability row; before, other nodes kept the old status until restart). The 15 s TTLs remain the safety net. Node-local, with reasons: the alias and connection row-id maps, the S3 client map, the process-permission map, the provider defaults, the running-scan guard and the per-pod presign rate limiter (StorageBucket.PRESIGN_COUNTS; per-node rate limits are accepted for now). ObjectStorageClusterCachesTest audits the package for unregistered static caches and covers remote and local invalidation.
• Builds against QQQ 4.1.0-SNAPSHOT (qqq.snapshot.version) for ClusterCaches until the next QQQ RC ships; a release must pin that RC.
• Read the qbit metadata version from the Maven-filtered artifact version so release metadata matches the build and help bundle.
• Rechecked the released RC.8 JARs: core still has no S3 session-token/provider hook or file-reference interpreter. Retained both compatibility shims and moved their review deadline to RC.9 under issue #10.
• Raised the QQQ BOM baseline to released 4.1.0-RC.8 for the central health API.
• Raised the QQQ BOM baseline to released 4.1.0-RC.7 for the core help-bundle registry and coverage assertions.
• Raised the QQQ BOM baseline to released 4.1.0-RC.5 and delegated S3 client construction to core S3ClientFactory.buildClient. Removed local HTTP client and trust-manager implementations; existing endpoint, region, addressing, checksum, TLS/CA, credentials and timeout settings remain supported.
• Clients remain owned by each connection runtime and are rebuilt on reset/reinstall. Retry behavior now follows core/SDK defaults instead of forcing ADAPTIVE_V2; hosts can select the SDK retry mode through AWS_RETRY_MODE or aws.retryMode.
• Retained local presigning (including a separate endpoint/key). A request-credentials bridge preserves static session tokens because RC.5 core metadata/factory has no session-token field or credentials-provider hook. This bridge and the existing ${file.PATH} resolver remain necessary in released RC.7 (no session-token/provider hook or file-reference interpreter); their review deadline is now RC.8 under issue #10.
• Added the filesystem module with its unused SFTP/SSH stack excluded. Resolved dependencies otherwise only change QQQ RC.4 to RC.5 and jsoup 1.23.1 to 1.23.2; see dependency details.

Notes

• Deliberate differences from the design text: Test connection is two actions (read-only, and with write tests) so the write variant can require object_storage.admin; a folder record has no subfolder child list (core does not allow a self join as an exposed join); there is no withTableNamePrefix (the object_storage_ prefix is fixed by the naming rule).