fix(postgres): accept full_page_writes = off on Amazon Aurora PostgreSQL only (release/0.4) #10

Merged

james.maes merged 1 commits from fix/aurora-full-page-writes-0.4 into release/0.4 2026-11-10 15:31:18 +00:00

Conversation 0 Commits 1 Files Changed 6

+467 -5

james.maes commented 2026-11-10 15:13:11 +00:00

Owner

Copy link

Copy source

Defect

RecordFilesPostgres.require refuses every Amazon Aurora PostgreSQL database. Its settings query requires full_page_writes = on, and Aurora reports off.

Measured on the dev Aurora PostgreSQL cluster (engine 17.9; staging and production run the same engine): synchronous_commit = on, fsync = on, full_page_writes = off (context sighup, source default; the parameter is not in the Aurora cluster parameter group, so it cannot be changed), TimeZone = UTC, transaction_isolation = read committed, pg_is_in_recovery() = false. Aurora keeps full_page_writes off because its storage layer does not write torn pages, so the protection that setting gives stock PostgreSQL is not needed there.

The refusal is the single message Record Files requires its exact writable PostgreSQL 17 database, permanent owner tables, deterministic comparison, READ COMMITTED isolation, UTC session, bounded lock waits and synchronous durable commits.

Change

• The settings query gets a twelfth column: to_regprocedure('aurora_version()') IS NOT NULL. It is true on Aurora and false on stock PostgreSQL 17, and it never raises (a direct SELECT aurora_version() would abort the transaction on stock PostgreSQL; pg_proc does not list the function on Aurora).
• The durability decision moves into one pure function, RecordFilesPostgres.durable(synchronousCommit, fsync, fullPageWrites, aurora):
  • synchronous_commit must be on or remote_apply (unchanged);
  • fsync must be on (unchanged);
  • full_page_writes must be on, or off when the server is Aurora.
• Nothing else changes. Stock PostgreSQL with full_page_writes = off is still refused. On Aurora, fsync, synchronous_commit, the writable primary, PostgreSQL 17, the UTC session, READ COMMITTED, lock_timeout from 1 to 5000 ms and every owner-table check are required exactly as before. The refusal message is unchanged.
• Comments in RecordFilesPostgres state the Aurora rule and why. CHANGELOG.md and docs/KNOWN-ISSUES.md record it.

RecordFilesMySql has the analogous torn-page rule (innodb_doublewrite must be ON or DETECT_AND_RECOVER). It is not changed here: no Aurora MySQL server was measured, and the Record Files MySQL qualification also requires product version 8.4..

Tests

• PostgresDurabilityRuleTest (new, runs in every build, no server): the pure rule. Stock with full_page_writes off is refused; Aurora with it off is accepted; Aurora with fsync off or synchronous_commit off, local or remote_write is refused; the settings query ends with the detection column and does not call the function or read pg_proc.
• PostgresAuroraQualificationTest (new, opt-in with -DpostgresRecordFilesTests=true, real postgres:17 container through PostgresRecordFilesSupport):
  • the shipped settings query, run as the DML-only application role, returns 12 columns and reports false on the stock server and true once the function exists;
  • stock server with full_page_writes = off: isDurableBackend is false, openTransaction throws the refusal, a collection write is refused;
  • simulated Aurora with full_page_writes = off: isDurableBackend is true, the owned transaction qualifies, a collection write commits and is read back;
  • simulated Aurora with fsync = off: refused;
  • simulated Aurora with synchronous_commit off or local: refused.
• PostgresRecordFilesSupport.start(String...) (new overload) lets a suite leave fsync and full_page_writes off the server command line, because a setting fixed there cannot be changed by ALTER SYSTEM. start() is unchanged in effect.

What is simulated: Aurora itself. The fixture superuser creates public.aurora_version() in the test database so to_regprocedure resolves, and sets full_page_writes = off with ALTER SYSTEM and pg_reload_conf(). Aurora's storage layer and its other settings are not reproduced. No test here runs on a real Aurora cluster.

The opt-in PostgreSQL suites do not run in the shared CI build (mvn verify without the property), so CI exercises PostgresDurabilityRuleTest only.

Local results

This is commit af5728a (pull request 9, released as 0.1.0-RC.2) cherry-picked onto release/0.4 as 3eb120a. The tree of this branch differs from v0.1.0-RC.2 only in pom.xml.

Run on Windows 11, Temurin 21.0.12, Docker 29.9.2, with the repositories of the shared CI workflow and a dedicated Maven cache, on 3eb120a:

• ./mvnw -B clean verify (nothing skipped by flag): 305 tests, 0 failures, 0 errors, 67 skipped (the opt-in MySQL and PostgreSQL suites), 0 Checkstyle violations, BUILD SUCCESS.
• ./mvnw -B -DpostgresRecordFilesTests=true -Dtest='Postgres*Test' clean verify (every PostgreSQL class on a real postgres:17 container): 56 tests, 0 failures, 0 errors, 0 skipped, BUILD SUCCESS. Per class: PostgresRecordFilesTest 13, PostgresIntakeAndRetentionTest 26, PostgresRetainedUploadRecoveryTest 5, PostgresTrackedDownloadTest 3, PostgresAuroraQualificationTest 5, PostgresDurabilityRuleTest 4.
• The jar built from 3eb120a compared with the published 0.1.0-RC.2 jar: every .class file is byte-identical; only qbit-record-files.properties, pom.properties and the embedded pom.xml differ (version and dependency versions).
• The opt-in MySQL suite was not run.
• CI run 7348 on this branch: success, 305 tests, 0 failures, 0 errors, 67 skipped, 0 Checkstyle violations.

Release

Target release/0.4, the line that published 0.4.0-RC.1 (v0.4.0-RC.1, commit f3e814d, the head of release/0.4, <revision>0.4.0-RC.1</revision>). Merging this pull request pushes to release/0.4; the shared Maven workflow (KofTwentyTwo/ci-workflows maven.yaml@v1) then reads the committed revision, publishes 0.4.0-RC.2, commits Bump version to 0.4.0-RC.2 [skip ci] and tags v0.4.0-RC.2. This commit is the only difference from 0.4.0-RC.1. The same change is already released on release/0.1 as 0.1.0-RC.2 (pull request 9) and proposed for develop in pull request 8.

Do not merge without the owner: a merge publishes.

🤖 Generated with Claude Code

## Defect `RecordFilesPostgres.require` refuses every Amazon Aurora PostgreSQL database. Its settings query requires `full_page_writes = on`, and Aurora reports `off`. Measured on the dev Aurora PostgreSQL cluster (engine 17.9; staging and production run the same engine): `synchronous_commit = on`, `fsync = on`, `full_page_writes = off` (context sighup, source default; the parameter is not in the Aurora cluster parameter group, so it cannot be changed), `TimeZone = UTC`, `transaction_isolation = read committed`, `pg_is_in_recovery() = false`. Aurora keeps `full_page_writes` off because its storage layer does not write torn pages, so the protection that setting gives stock PostgreSQL is not needed there. The refusal is the single message `Record Files requires its exact writable PostgreSQL 17 database, permanent owner tables, deterministic comparison, READ COMMITTED isolation, UTC session, bounded lock waits and synchronous durable commits.` ## Change - The settings query gets a twelfth column: `to_regprocedure('aurora_version()') IS NOT NULL`. It is true on Aurora and false on stock PostgreSQL 17, and it never raises (a direct `SELECT aurora_version()` would abort the transaction on stock PostgreSQL; `pg_proc` does not list the function on Aurora). - The durability decision moves into one pure function, `RecordFilesPostgres.durable(synchronousCommit, fsync, fullPageWrites, aurora)`: - `synchronous_commit` must be `on` or `remote_apply` (unchanged); - `fsync` must be `on` (unchanged); - `full_page_writes` must be `on`, or `off` when the server is Aurora. - Nothing else changes. Stock PostgreSQL with `full_page_writes = off` is still refused. On Aurora, `fsync`, `synchronous_commit`, the writable primary, PostgreSQL 17, the UTC session, READ COMMITTED, `lock_timeout` from 1 to 5000 ms and every owner-table check are required exactly as before. The refusal message is unchanged. - Comments in `RecordFilesPostgres` state the Aurora rule and why. `CHANGELOG.md` and `docs/KNOWN-ISSUES.md` record it. `RecordFilesMySql` has the analogous torn-page rule (`innodb_doublewrite` must be `ON` or `DETECT_AND_RECOVER`). It is not changed here: no Aurora MySQL server was measured, and the Record Files MySQL qualification also requires product version `8.4.`. ## Tests - `PostgresDurabilityRuleTest` (new, runs in every build, no server): the pure rule. Stock with `full_page_writes` off is refused; Aurora with it off is accepted; Aurora with `fsync` off or `synchronous_commit` `off`, `local` or `remote_write` is refused; the settings query ends with the detection column and does not call the function or read `pg_proc`. - `PostgresAuroraQualificationTest` (new, opt-in with `-DpostgresRecordFilesTests=true`, real `postgres:17` container through `PostgresRecordFilesSupport`): - the shipped settings query, run as the DML-only application role, returns 12 columns and reports `false` on the stock server and `true` once the function exists; - stock server with `full_page_writes = off`: `isDurableBackend` is false, `openTransaction` throws the refusal, a collection write is refused; - simulated Aurora with `full_page_writes = off`: `isDurableBackend` is true, the owned transaction qualifies, a collection write commits and is read back; - simulated Aurora with `fsync = off`: refused; - simulated Aurora with `synchronous_commit` `off` or `local`: refused. - `PostgresRecordFilesSupport.start(String...)` (new overload) lets a suite leave `fsync` and `full_page_writes` off the server command line, because a setting fixed there cannot be changed by `ALTER SYSTEM`. `start()` is unchanged in effect. What is simulated: Aurora itself. The fixture superuser creates `public.aurora_version()` in the test database so `to_regprocedure` resolves, and sets `full_page_writes = off` with `ALTER SYSTEM` and `pg_reload_conf()`. Aurora's storage layer and its other settings are not reproduced. No test here runs on a real Aurora cluster. The opt-in PostgreSQL suites do not run in the shared CI build (`mvn verify` without the property), so CI exercises `PostgresDurabilityRuleTest` only. ## Local results This is commit `af5728a` (pull request 9, released as `0.1.0-RC.2`) cherry-picked onto `release/0.4` as `3eb120a`. The tree of this branch differs from `v0.1.0-RC.2` only in `pom.xml`. Run on Windows 11, Temurin 21.0.12, Docker 29.9.2, with the repositories of the shared CI workflow and a dedicated Maven cache, on `3eb120a`: - `./mvnw -B clean verify` (nothing skipped by flag): 305 tests, 0 failures, 0 errors, 67 skipped (the opt-in MySQL and PostgreSQL suites), 0 Checkstyle violations, BUILD SUCCESS. - `./mvnw -B -DpostgresRecordFilesTests=true -Dtest='Postgres*Test' clean verify` (every PostgreSQL class on a real `postgres:17` container): 56 tests, 0 failures, 0 errors, 0 skipped, BUILD SUCCESS. Per class: `PostgresRecordFilesTest` 13, `PostgresIntakeAndRetentionTest` 26, `PostgresRetainedUploadRecoveryTest` 5, `PostgresTrackedDownloadTest` 3, `PostgresAuroraQualificationTest` 5, `PostgresDurabilityRuleTest` 4. - The jar built from `3eb120a` compared with the published `0.1.0-RC.2` jar: every `.class` file is byte-identical; only `qbit-record-files.properties`, `pom.properties` and the embedded `pom.xml` differ (version and dependency versions). - The opt-in MySQL suite was not run. - CI run 7348 on this branch: success, 305 tests, 0 failures, 0 errors, 67 skipped, 0 Checkstyle violations. ## Release Target `release/0.4`, the line that published `0.4.0-RC.1` (`v0.4.0-RC.1`, commit `f3e814d`, the head of `release/0.4`, `<revision>0.4.0-RC.1</revision>`). Merging this pull request pushes to `release/0.4`; the shared Maven workflow (`KofTwentyTwo/ci-workflows` `maven.yaml@v1`) then reads the committed revision, publishes `0.4.0-RC.2`, commits `Bump version to 0.4.0-RC.2 [skip ci]` and tags `v0.4.0-RC.2`. This commit is the only difference from `0.4.0-RC.1`. The same change is already released on `release/0.1` as `0.1.0-RC.2` (pull request 9) and proposed for `develop` in pull request 8. Do not merge without the owner: a merge publishes. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

james.maes added 1 commit 2026-11-10 15:13:11 +00:00

fix(postgres): accept full_page_writes = off on Amazon Aurora PostgreSQL only ... 3eb120a823

RecordFilesPostgres refused every Amazon Aurora PostgreSQL database: Aurora
reports full_page_writes = off (measured on engine 17.9) and the parameter is
not in the cluster parameter group, so it cannot be changed. Its storage layer
does not write torn pages, so the protection that setting gives stock
PostgreSQL is not needed there.

The settings query now carries a twelfth column,
to_regprocedure('aurora_version()') IS NOT NULL, which is true on Aurora and
false on stock PostgreSQL without raising. full_page_writes = off is accepted
only when that column is true. Stock PostgreSQL with full_page_writes = off is
still refused, and fsync, synchronous_commit, the writable primary, version,
UTC, READ COMMITTED, the lock_timeout bound and every owner-table check are
unchanged on both.

Tests: PostgresDurabilityRuleTest (pure rule, every build) and
PostgresAuroraQualificationTest (opt-in, real PostgreSQL 17, Aurora simulated
by a test-only aurora_version() function and ALTER SYSTEM full_page_writes).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

james.maes merged commit 388b72bd40 into release/0.4 2026-11-10 15:31:18 +00:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

forge-recovery (Forge Recovery) james.maes (James Maes)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: QRun-IO/qbit-record-files#10