Automatic verified database backup and Liquibase migration on startup #47

New Issue

2026-10-07T12:16:12Z

KofTwentyTwo commented

2026-10-07 12:16:12 +00:00

(Migrated from github.com)

Carl must back up its own database automatically and manage schema and versioned data with Liquibase as part of startup. Related: #5 (CI/CD and Liquibase), KofTwentyTwo/kof22-agent-foundation#83 (reusable lifecycle command).

Owner decisions (2026-10-07)

Backups are stored in a local directory beside the running stack, for now.
A backup runs before every startup and again daily while Carl runs.
Every backup gets a full restore test into a throwaway PostgreSQL before any migration proceeds.
Once the combined build (PRs #35-#46) passes CI, the first automated startup may run against real data, including the Flyway-to-Liquibase handover and migrations 63-65, after its backup verifies.

Design

Startup order in the local Docker Compose stack (and later the Argo rollout): backup → verify-restore → Liquibase update as the migrator → Carl starts with restricted runtime credentials and only validates. Carl waits on the lifecycle step completing successfully; any failure stops startup with the data untouched. A daily backup job reuses backup + verify without migrating.

Deliverables

Carl domain changes managed by Liquibase, with the retained Flyway history (V1-V65) handed over and checksums validated.
Compose services: db-lifecycle (one-shot, gates carl) and db-backup-daily.
Encrypted backups plus manifests in the configured local directory, with retention.
README covering restore steps.

Acceptance

Real PostgreSQL: fresh install, retained-history handover, repeated startups (no duplicate changes), failed backup or failed restore blocks startup, failed migration leaves data at its pre-run state.
Restore drill from a produced backup succeeds and matches row counts.
Negative access: the runtime role cannot migrate or read backups.
Synthetic qualification first; real-data first run only after the combined build passes CI and its backup verifies.

Carl must back up its own database automatically and manage schema and versioned data with Liquibase as part of startup. Related: #5 (CI/CD and Liquibase), KofTwentyTwo/kof22-agent-foundation#83 (reusable lifecycle command). ## Owner decisions (2026-10-07) - Backups are stored in a local directory beside the running stack, for now. - A backup runs before every startup and again daily while Carl runs. - Every backup gets a full restore test into a throwaway PostgreSQL before any migration proceeds. - Once the combined build (PRs #35-#46) passes CI, the first automated startup may run against real data, including the Flyway-to-Liquibase handover and migrations 63-65, after its backup verifies. ## Design Startup order in the local Docker Compose stack (and later the Argo rollout): `backup` → `verify-restore` → Liquibase `update` as the migrator → Carl starts with restricted runtime credentials and only validates. Carl waits on the lifecycle step completing successfully; any failure stops startup with the data untouched. A daily backup job reuses backup + verify without migrating. ## Deliverables - Carl domain changes managed by Liquibase, with the retained Flyway history (V1-V65) handed over and checksums validated. - Compose services: `db-lifecycle` (one-shot, gates `carl`) and `db-backup-daily`. - Encrypted backups plus manifests in the configured local directory, with retention. - README covering restore steps. ## Acceptance - Real PostgreSQL: fresh install, retained-history handover, repeated startups (no duplicate changes), failed backup or failed restore blocks startup, failed migration leaves data at its pre-run state. - Restore drill from a produced backup succeeds and matches row counts. - Negative access: the runtime role cannot migrate or read backups. - Synthetic qualification first; real-data first run only after the combined build passes CI and its backup verifies.

This repo is archived. You cannot comment on issues.

1 Participants

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: KofTwentyTwo/carl#47