Make the UI end-to-end suite a required check for main and releases once proven stable #54

New Issue

2026-10-09T11:21:36Z

KofTwentyTwo commented

2026-10-09 11:21:36 +00:00

(Migrated from github.com)

Context

The FlaUI UI end-to-end job (UI end-to-end tests (x64)) is deliberately not a required status check on main: hosted-runner UI automation was new and had to prove itself stable first (see the comment in ci.yml). With #44, releases call the CI workflow, and to avoid holding an immutable tag to a stricter bar than a merge, the release passes ui-tests-advisory: true so a red UI job is visible but does not block publishing.

Task

Once the job has run green for a sustained period on hosted runners (suggest: no non-code-related failures over ~4 weeks of dev pushes and PRs):

Add UI end-to-end tests (x64) to the protect-main ruleset's required status checks.
Set ui-tests-advisory: false in .github/workflows/release.yml (or remove the input and the continue-on-error plumbing from ci.yml).
Update docs/RELEASING.md and CONTRIBUTING.md accordingly.

## Context The FlaUI UI end-to-end job (`UI end-to-end tests (x64)`) is deliberately **not** a required status check on `main`: hosted-runner UI automation was new and had to prove itself stable first (see the comment in `ci.yml`). With #44, releases call the CI workflow, and to avoid holding an immutable tag to a stricter bar than a merge, the release passes `ui-tests-advisory: true` so a red UI job is visible but does not block publishing. ## Task Once the job has run green for a sustained period on hosted runners (suggest: no non-code-related failures over ~4 weeks of dev pushes and PRs): - [ ] Add `UI end-to-end tests (x64)` to the `protect-main` ruleset's required status checks. - [ ] Set `ui-tests-advisory: false` in `.github/workflows/release.yml` (or remove the input and the `continue-on-error` plumbing from `ci.yml`). - [ ] Update `docs/RELEASING.md` and `CONTRIBUTING.md` accordingly.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: KofTwentyTwo/gclo#54