test(authentik): fix slow contract stack startup in CI (Postgres JIT) #65

Merged

james.maes merged 3 commits from ci/authentik-ci-stability into develop 2026-11-07 00:08:28 +00:00

Conversation 0 Commits 3 Files Changed 5

+155 -40

3 Commits

Author	SHA1	Message	Date
James MaesandClaude Opus 5.5	d3f334cd42	fix(test): turn off Postgres JIT in the contract stack ... Root cause of the slow/failed Authentik contract stack on busy CI runners: Authentik validates every policy binding in a blueprint by loading PolicyBindingModel.select_subclasses(), a ~20-way LEFT JOIN whose planner estimate on a fresh database (cost ~1e8) is far above Postgres' JIT thresholds. Postgres JIT-compiled it on every call: 7.6 s instead of 77 ms at 0.5 CPU, with the Postgres container pegged and the worker idle. The fixtures' nested metaapplyblueprint chain (tokens -> bootstrap -> oauth2-apps) runs oauth2-apps 8 times per tokens apply, so one apply outran the worker's 10-minute task limit, was killed and retried, and the stack never became ready (runs 5754, 6277, 6363). With jit=off the full contract suite passes locally with every container capped at 0.5 CPU; blueprint startup drops from >14 minutes to ~1.5 minutes and BlueprintReapplyIT from 228 s to 34 s. Also give waits on worker-delivered results (emails, notifications, events) a shared 3-minute budget and blueprint re-applies 10 minutes; the waits still return as soon as the condition holds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 17:36:47 -06:00
James MaesandClaude Opus 5.5	7adc25bf7e	fix(test): drain Authentik's first-boot blueprint work before re-applying ours ... Root cause of the flaky contract stack (diagnostics from run 6277): on first boot the worker (2 threads) applies ~40 built-in blueprints plus the three kof22 fixtures concurrently. On a slow shared runner ours can validate before the built-in objects they reference exist (managed OAuth2 scope mappings, default flows) and end in "error", which Authentik never retries for an unchanged file; concurrent applies also deadlocked in Postgres. The earlier throttled re-apply made it worse: it enqueued three applies every 30s regardless of pending work (69 queued apply_blueprint tasks, applies running for 6+ minutes), and the backlog then starved the tests' own worker tasks (blueprint applies, recovery/invitation emails, notification rules) and held locks that timed out plain API writes. Now startup waits until no blueprint discovery/apply task is queued or in progress, then re-applies the first non-successful kof22 blueprint in dependency order, one at a time, never enqueueing while an apply is pending. The startup budget is 15 minutes (with a comment) because a cold shared runner can need well over 8 minutes for the built-in storm. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 12:00:01 -06:00
James MaesandClaude Opus 5.5	2f6a83db27	test(authentik): print the worker task queue on contract-test timeouts; restore seed state in BlueprintReapplyIT ... Most contract-test waits depend on Authentik worker tasks (blueprint applies, recovery/invitation emails, notification rules). When a wait times out, print the global task status and the running, queued, warning and error tasks with their logs, so CI shows whether the worker was stuck, backlogged or failing. BlueprintReapplyIT now restores the seed groups and the service-account role's view_user permission in a finally block. A failed re-apply used to leave those edits behind and cascade into unrelated failures (QbitReadContractIT.groups saw "authentik.*, cms.publish"; TokenContractIT got 403 on /core/users/). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 10:38:25 -06:00