fix(postgres): accept full_page_writes = off on Amazon Aurora PostgreSQL only #8

Merged

james.maes merged 1 commits from fix/aurora-full-page-writes-develop into develop 2026-11-10 15:31:15 +00:00

Conversation 0 Commits 1 Files Changed 6

+467 -5

james.maes commented 2026-11-10 02:38:07 +00:00

Owner

Copy link

Copy source

Defect

RecordFilesPostgres.require refuses every Amazon Aurora PostgreSQL database. Its settings query requires full_page_writes = on, and Aurora reports off.

Measured on the dev Aurora PostgreSQL cluster (engine 17.9; staging and production run the same engine): synchronous_commit = on, fsync = on, full_page_writes = off (context sighup, source default; the parameter is not in the Aurora cluster parameter group, so it cannot be changed), TimeZone = UTC, transaction_isolation = read committed, pg_is_in_recovery() = false. Aurora keeps full_page_writes off because its storage layer does not write torn pages, so the protection that setting gives stock PostgreSQL is not needed there.

The refusal is the single message Record Files requires its exact writable PostgreSQL 17 database, permanent owner tables, deterministic comparison, READ COMMITTED isolation, UTC session, bounded lock waits and synchronous durable commits.

Change

• The settings query gets a twelfth column: to_regprocedure('aurora_version()') IS NOT NULL. It is true on Aurora and false on stock PostgreSQL 17, and it never raises (a direct SELECT aurora_version() would abort the transaction on stock PostgreSQL; pg_proc does not list the function on Aurora).
• The durability decision moves into one pure function, RecordFilesPostgres.durable(synchronousCommit, fsync, fullPageWrites, aurora):
  • synchronous_commit must be on or remote_apply (unchanged);
  • fsync must be on (unchanged);
  • full_page_writes must be on, or off when the server is Aurora.
• Nothing else changes. Stock PostgreSQL with full_page_writes = off is still refused. On Aurora, fsync, synchronous_commit, the writable primary, PostgreSQL 17, the UTC session, READ COMMITTED, lock_timeout from 1 to 5000 ms and every owner-table check are required exactly as before. The refusal message is unchanged.
• Comments in RecordFilesPostgres state the Aurora rule and why. CHANGELOG.md and docs/KNOWN-ISSUES.md record it.

RecordFilesMySql has the analogous torn-page rule (innodb_doublewrite must be ON or DETECT_AND_RECOVER). It is not changed here: no Aurora MySQL server was measured, and the Record Files MySQL qualification also requires product version 8.4..

Tests

• PostgresDurabilityRuleTest (new, runs in every build, no server): the pure rule. Stock with full_page_writes off is refused; Aurora with it off is accepted; Aurora with fsync off or synchronous_commit off, local or remote_write is refused; the settings query ends with the detection column and does not call the function or read pg_proc.
• PostgresAuroraQualificationTest (new, opt-in with -DpostgresRecordFilesTests=true, real postgres:17 container through PostgresRecordFilesSupport):
  • the shipped settings query, run as the DML-only application role, returns 12 columns and reports false on the stock server and true once the function exists;
  • stock server with full_page_writes = off: isDurableBackend is false, openTransaction throws the refusal, a collection write is refused;
  • simulated Aurora with full_page_writes = off: isDurableBackend is true, the owned transaction qualifies, a collection write commits and is read back;
  • simulated Aurora with fsync = off: refused;
  • simulated Aurora with synchronous_commit off or local: refused.
• PostgresRecordFilesSupport.start(String...) (new overload) lets a suite leave fsync and full_page_writes off the server command line, because a setting fixed there cannot be changed by ALTER SYSTEM. start() is unchanged in effect.

What is simulated: Aurora itself. The fixture superuser creates public.aurora_version() in the test database so to_regprocedure resolves, and sets full_page_writes = off with ALTER SYSTEM and pg_reload_conf(). Aurora's storage layer and its other settings are not reproduced. No test here runs on a real Aurora cluster.

The opt-in PostgreSQL suites do not run in the shared CI build (mvn verify without the property), so CI exercises PostgresDurabilityRuleTest only.

Local results

Run on Windows 11, Temurin 21.0.12, Docker 29.9.2, with the repositories of the shared CI workflow and a dedicated empty Maven cache.

• On this commit (0d98968): ./mvnw -B -DpostgresRecordFilesTests=true -Dtest='Postgres*Test' clean verify (every PostgreSQL class on a real postgres:17 container): 56 tests, 0 failures, 0 errors, 0 skipped, 0 Checkstyle violations, BUILD SUCCESS.
• The full local ./mvnw -B clean verify was run on the release/0.1 copy of this commit (af5728a, pull request 9): 305 tests, 0 failures, 0 errors, 67 skipped, 0 Checkstyle violations, BUILD SUCCESS. It was not completed locally on this branch.
• CI run 7341 on this branch ran the full mvn verify: success, 305 tests, 0 failures, 0 errors, 67 skipped (the opt-in MySQL and PostgreSQL suites), 0 Checkstyle violations.
• Sensitivity check of the new tests (on af5728a): with the rule changed to never accept off, 4 of 9 failed; with the rule changed to accept off everywhere, 2 of 9 failed. The rule was restored afterwards.
• The opt-in MySQL suite was not run.

Branch

Target develop: this is the same commit as pull request 9 for release/0.1 (cherry-picked; develop and release/0.1 differ only in pom.xml), so that the next release line cut from develop keeps the Aurora rule. Merging it pushes to develop; the shared Maven workflow then deploys a snapshot (the committed revision is 0.5.0-SNAPSHOT). It does not publish a release candidate. GG Forge does not consume this branch; the artifact for Forge comes from pull request 9.

Do not merge without the owner: a merge deploys a snapshot.

🤖 Generated with Claude Code

## Defect `RecordFilesPostgres.require` refuses every Amazon Aurora PostgreSQL database. Its settings query requires `full_page_writes = on`, and Aurora reports `off`. Measured on the dev Aurora PostgreSQL cluster (engine 17.9; staging and production run the same engine): `synchronous_commit = on`, `fsync = on`, `full_page_writes = off` (context sighup, source default; the parameter is not in the Aurora cluster parameter group, so it cannot be changed), `TimeZone = UTC`, `transaction_isolation = read committed`, `pg_is_in_recovery() = false`. Aurora keeps `full_page_writes` off because its storage layer does not write torn pages, so the protection that setting gives stock PostgreSQL is not needed there. The refusal is the single message `Record Files requires its exact writable PostgreSQL 17 database, permanent owner tables, deterministic comparison, READ COMMITTED isolation, UTC session, bounded lock waits and synchronous durable commits.` ## Change - The settings query gets a twelfth column: `to_regprocedure('aurora_version()') IS NOT NULL`. It is true on Aurora and false on stock PostgreSQL 17, and it never raises (a direct `SELECT aurora_version()` would abort the transaction on stock PostgreSQL; `pg_proc` does not list the function on Aurora). - The durability decision moves into one pure function, `RecordFilesPostgres.durable(synchronousCommit, fsync, fullPageWrites, aurora)`: - `synchronous_commit` must be `on` or `remote_apply` (unchanged); - `fsync` must be `on` (unchanged); - `full_page_writes` must be `on`, or `off` when the server is Aurora. - Nothing else changes. Stock PostgreSQL with `full_page_writes = off` is still refused. On Aurora, `fsync`, `synchronous_commit`, the writable primary, PostgreSQL 17, the UTC session, READ COMMITTED, `lock_timeout` from 1 to 5000 ms and every owner-table check are required exactly as before. The refusal message is unchanged. - Comments in `RecordFilesPostgres` state the Aurora rule and why. `CHANGELOG.md` and `docs/KNOWN-ISSUES.md` record it. `RecordFilesMySql` has the analogous torn-page rule (`innodb_doublewrite` must be `ON` or `DETECT_AND_RECOVER`). It is not changed here: no Aurora MySQL server was measured, and the Record Files MySQL qualification also requires product version `8.4.`. ## Tests - `PostgresDurabilityRuleTest` (new, runs in every build, no server): the pure rule. Stock with `full_page_writes` off is refused; Aurora with it off is accepted; Aurora with `fsync` off or `synchronous_commit` `off`, `local` or `remote_write` is refused; the settings query ends with the detection column and does not call the function or read `pg_proc`. - `PostgresAuroraQualificationTest` (new, opt-in with `-DpostgresRecordFilesTests=true`, real `postgres:17` container through `PostgresRecordFilesSupport`): - the shipped settings query, run as the DML-only application role, returns 12 columns and reports `false` on the stock server and `true` once the function exists; - stock server with `full_page_writes = off`: `isDurableBackend` is false, `openTransaction` throws the refusal, a collection write is refused; - simulated Aurora with `full_page_writes = off`: `isDurableBackend` is true, the owned transaction qualifies, a collection write commits and is read back; - simulated Aurora with `fsync = off`: refused; - simulated Aurora with `synchronous_commit` `off` or `local`: refused. - `PostgresRecordFilesSupport.start(String...)` (new overload) lets a suite leave `fsync` and `full_page_writes` off the server command line, because a setting fixed there cannot be changed by `ALTER SYSTEM`. `start()` is unchanged in effect. What is simulated: Aurora itself. The fixture superuser creates `public.aurora_version()` in the test database so `to_regprocedure` resolves, and sets `full_page_writes = off` with `ALTER SYSTEM` and `pg_reload_conf()`. Aurora's storage layer and its other settings are not reproduced. No test here runs on a real Aurora cluster. The opt-in PostgreSQL suites do not run in the shared CI build (`mvn verify` without the property), so CI exercises `PostgresDurabilityRuleTest` only. ## Local results Run on Windows 11, Temurin 21.0.12, Docker 29.9.2, with the repositories of the shared CI workflow and a dedicated empty Maven cache. - On this commit (`0d98968`): `./mvnw -B -DpostgresRecordFilesTests=true -Dtest='Postgres*Test' clean verify` (every PostgreSQL class on a real `postgres:17` container): 56 tests, 0 failures, 0 errors, 0 skipped, 0 Checkstyle violations, BUILD SUCCESS. - The full local `./mvnw -B clean verify` was run on the `release/0.1` copy of this commit (`af5728a`, pull request 9): 305 tests, 0 failures, 0 errors, 67 skipped, 0 Checkstyle violations, BUILD SUCCESS. It was not completed locally on this branch. - CI run 7341 on this branch ran the full `mvn verify`: success, 305 tests, 0 failures, 0 errors, 67 skipped (the opt-in MySQL and PostgreSQL suites), 0 Checkstyle violations. - Sensitivity check of the new tests (on `af5728a`): with the rule changed to never accept `off`, 4 of 9 failed; with the rule changed to accept `off` everywhere, 2 of 9 failed. The rule was restored afterwards. - The opt-in MySQL suite was not run. ## Branch Target `develop`: this is the same commit as pull request 9 for `release/0.1` (cherry-picked; `develop` and `release/0.1` differ only in `pom.xml`), so that the next release line cut from `develop` keeps the Aurora rule. Merging it pushes to `develop`; the shared Maven workflow then deploys a snapshot (the committed revision is `0.5.0-SNAPSHOT`). It does not publish a release candidate. GG Forge does not consume this branch; the artifact for Forge comes from pull request 9. Do not merge without the owner: a merge deploys a snapshot. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

james.maes added 1 commit 2026-11-10 02:38:08 +00:00

fix(postgres): accept full_page_writes = off on Amazon Aurora PostgreSQL only ... 0d989683eb

RecordFilesPostgres refused every Amazon Aurora PostgreSQL database: Aurora
reports full_page_writes = off (measured on engine 17.9) and the parameter is
not in the cluster parameter group, so it cannot be changed. Its storage layer
does not write torn pages, so the protection that setting gives stock
PostgreSQL is not needed there.

The settings query now carries a twelfth column,
to_regprocedure('aurora_version()') IS NOT NULL, which is true on Aurora and
false on stock PostgreSQL without raising. full_page_writes = off is accepted
only when that column is true. Stock PostgreSQL with full_page_writes = off is
still refused, and fsync, synchronous_commit, the writable primary, version,
UTC, READ COMMITTED, the lock_timeout bound and every owner-table check are
unchanged on both.

Tests: PostgresDurabilityRuleTest (pure rule, every build) and
PostgresAuroraQualificationTest (opt-in, real PostgreSQL 17, Aurora simulated
by a test-only aurora_version() function and ALTER SYSTEM full_page_writes).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

james.maes merged commit 73341e59c5 into develop 2026-11-10 15:31:15 +00:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

forge-recovery (Forge Recovery) james.maes (James Maes)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: QRun-IO/qbit-record-files#8