Compare commits

..
Author SHA1 Message Date
james.maesandCodex 09046d45b0 feat(ci): centralize orb branch-check enforcement
Co-Authored-By: Codex <noreply@openai.com>
2026-10-21 18:55:40 -05:00
james.maesandCodex 9e9e6c1fa5 fix(ci): verify frontend maintenance without publishing (#92)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 17:30:16 -05:00
james.maesandCodex 45aaccd2a9 fix(ci): gate core snapshots on all acceptance checks (#90)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 17:14:44 -05:00
james.maesandCodex ab77962b6a fix(audit): discover current organization repositories (#89)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 16:59:04 -05:00
james.maesandCodex d0792d60d6 fix(ci): verify maintenance on nonpublishing branches (#87)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 16:46:05 -05:00
james.maesandCodex 72f5f4d4e7 fix: allow managed orb tracks during QBit onboarding (#85)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 16:26:12 -05:00
james.maesandCodex 00af9aae31 perf: reuse React build before frontend tests (#84)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 15:55:47 -05:00
james.maesandCodex 7e60372948 fix: isolate browser package repositories (#82)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 15:29:28 -05:00
james.maesandCodex 45bab9d960 feat: support managed patch tracks [minor] (#80)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 15:06:44 -05:00
james.maesandCodex f304afb0be fix: isolate Ubuntu bootstrap from image PPAs (#78)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 14:27:11 -05:00
james.maesandCodex f76b9f7a7c fix: refresh only the newly added Temurin repository (#76)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 13:51:50 -05:00
james.maesandCodex fdf62a2f5a fix: align effective Maven versions before publication (#75)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 13:15:09 -05:00
james.maesandCodex 94341ef481 fix(ci): retain verification for schema caller migrations (#73)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 12:57:36 -05:00
james.maesandCodex 2debeada5d feat(ci): centralize opt-in QBit feature tag publication (#72)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 12:52:18 -05:00
james.maesandCodex 6febeff590 fix(audit): verify release and hotfix branch enforcement (#71)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 12:30:59 -05:00
james.maesandCodex 8b58a45f35 ci: include shared RC caller updates in maintenance verification (#70)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 11:59:07 -05:00
james.maesandCodex fbe36d683d ci: verify maintenance changes without publishing applications (#68)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 11:53:20 -05:00
james.maesandCodex bd5ee3bec5 ci: centralize policy-only branches and shared scanner enforcement (#65)
Add the repository-policy profile, validate actual legacy policy trees, normalize packed resource line endings, and scope maintenance requirements to branch source. Preserve canonical scanner enforcement for qqq-all and policy-only main branches.

Validation: 32 audit tests, 7 policy tests, all four target source trees, and Linux packed-generator check pass. Replaces this PR's unrecognized-email signatures using the established verified identity; signature enforcement remains enabled.

Co-authored-by: Codex <noreply@openai.com>
2026-10-21 11:21:15 -05:00
james.maesandCodex d4d6e97df0 fix: retry transient package downloads during CI bootstrap (#66)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 10:56:34 -05:00
james.maesandCodex c288a8864f ci: centralize branch-aware QBit source scanning (#63)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 09:36:07 -05:00
james.maesandCodex 9906c8f120 fix: audit explicitly selected repository branches (#62)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 09:15:33 -05:00
james.maesandCodex 4dcf2c1891 fix: preserve legacy qbit hotfix publication policies (#61)
* fix: preserve legacy qbit hotfix publication policies

Co-Authored-By: Codex <noreply@openai.com>

* fix: regenerate packed profiles with Linux zlib

Co-Authored-By: Codex <noreply@openai.com>

---------

Co-authored-by: Codex <noreply@openai.com>
2026-10-21 09:00:43 -05:00
james.maesandCodex cc30f599e6 fix(ci): provide a stable verification gate across branches (#59)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 03:12:12 -05:00
james.maesandCodex 05a2a5217a fix: audit rc worker pins independently of native workers (#58)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 02:23:51 -05:00
james.maesandCodex 7d73ac90b6 fix: distinguish missing RC refs from GitHub error bodies (#57)
* fix: distinguish missing rc refs from github error bodies

Co-Authored-By: Codex <noreply@openai.com>

* fix: reject empty ref responses across jq versions

Co-Authored-By: Codex <noreply@openai.com>

---------

Co-authored-by: Codex <noreply@openai.com>
2026-10-21 02:15:32 -05:00
james.maesandCodex edc1365ac9 fix: recover verified rc uploads after publisher timeout (#56)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 02:02:14 -05:00
james.maesandCodex 0f33ddadf8 ci: run business E2E on changes and preserve main validation (#55)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 01:19:08 -05:00
james.maesandCodex 5e9bbb54ec ci: add batch version rollout for canonical consumers (#54)
Validate every checkout before updating its orb pin. Preserve publication policy, filters, credentials contexts and forwarded parameters; reject custom callers and mismatched declarations. Cover all 12 profiles, dry runs, no-op repeats, invalid batches and duplicate input with six new tests in the existing contract job. Existing migration and standard tests pass; Linux packed generator check passes.

Co-authored-by: Codex <noreply@openai.com>
2026-10-21 00:45:50 -05:00
james.maesandCodex 5bcec90e33 ci: report the shared database gate on every pull request (#53)
Preserve migration verification, avoid duplicate feature push builds, and cancel superseded pull request runs so the gate can be required safely.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 22:25:38 -05:00
james.maesandCodex a50b07d927 fix: escape Android heredoc in continued Circle configuration (#51)
Circle accepts the continuation request before reporting an unescaped heredoc as a pipeline configuration error. Escape the template marker and emulate Circle unescaping in executable publisher tests. Validate the actual continued config with CircleCI.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 22:00:56 -05:00
james.maesandCodex 6318d90e96 feat: centralize lightweight GitOps rendering validation (#50)
Add a read-only native worker and canonical consumer bridge. Verify the pinned Kustomize binary, render every configuration without logging resources, and audit adoption. Behavioral and audit tests pass; all five marketing GitOps configurations render successfully.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:54:25 -05:00
james.maesandCodex a6fea661ff fix: authenticate Android publishing with GitHub package token (#47)
* fix: authenticate Android publishing with GitHub package token

Use the shared GitHub Packages token and verify its identity and registry repository access before Gradle starts. Preserve unit and emulator gates; package write success still requires hosted publication.

Co-Authored-By: Codex <noreply@openai.com>

* fix: keep embedded JSON resources consistent across checkouts

Co-Authored-By: Codex <noreply@openai.com>

---------

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:40:01 -05:00
james.maesandCodex 2894bae29a test: retain All release shell behavior in shared owner (#46)
Move the consumer publication regression to the reusable worker; existing test discovery executes it. Verify exact RC1 prerelease/non-latest flags, GA behavior and rejection of other RC versions.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:19:09 -05:00
james.maesandCodex a14e8f9b24 fix: default All releases to shared Circle smoke evidence (#35)
Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:12:58 -05:00
james.maesandCodex 83611872a3 fix: verify public All smoke evidence without secret export (#44)
Allow unauthenticated Circle API reads for public projects, preserve optional authentication and all fail-closed provenance checks.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:12:00 -05:00
james.maesandCodex 30c4e0e85c fix: use shared Maven credentials for Android publication (#42)
Use the existing organization context and Gradle environment properties. Fail before publishing if either credential is absent and avoid writing secrets to gradle.properties.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:07:21 -05:00
james.maesandCodex f429d9f3c4 fix: build marketing site before browser checks (#40)
Run lint and one production build in every shared marketing workflow before Playwright and Lighthouse. Preserve the delegated delivery contracts.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 20:56:32 -05:00
38 changed files with 1228 additions and 52 deletions

No files matched your search

+7
View File
@@ -31,6 +31,9 @@ jobs:
- run:
name: Verify resolved multi-module publication POMs
command: python3 tests/test_maven_publication_poms.py
- run:
name: Verify calculated versions match installed Maven coordinates
command: python3 tests/test_effective_publication_version.py
pipeline-contract-test:
resource_class: small
docker:
@@ -50,8 +53,10 @@ jobs:
python3 tests/test_maven_ci_contract.py
python3 tests/test_standardize_qbit.py
python3 tests/test_qbit_hotfix_profile.py
python3 tests/test_qbit_feature_profile.py
python3 tests/test_ci_maintenance.py
python3 tests/test_rollout_version.py
python3 tests/test_enforce_orb_checks.py
python3 tests/test_validate_standard.py
python3 tests/test_voyage_profile.py
python3 tests/test_next_native.py
@@ -81,6 +86,8 @@ jobs:
bash tests/test_maven_ci_cache.sh
bash tests/test_maven_test_results.sh
bash tests/test_maven_ci_install.sh
bash tests/test_browser_install.sh
bash tests/test_frontend_build_reuse.sh
bash tests/test_snapshot_refresh.sh
bash tests/test_maven_settings_output.sh
# Create jobs to test the commands of your orbs.
+2
View File
@@ -61,6 +61,7 @@ test-pipelines: pack
python3 tests/test_qbit_hotfix_profile.py
python3 tests/test_ci_maintenance.py
python3 tests/test_rollout_version.py
python3 tests/test_enforce_orb_checks.py
python3 tests/test_validate_standard.py
python3 tests/test_voyage_profile.py
python3 tests/test_next_native.py
@@ -220,3 +221,4 @@ publish-release: check-clean check-branch lint clean dev test-all
# an isolated temporary repository. Never publishes or uses signing credentials.
test-publication-poms:
python3 tests/test_maven_publication_poms.py
python3 tests/test_effective_publication_version.py
+7
View File
@@ -4,6 +4,8 @@ CircleCI orb for QQQ framework projects.
For centralized pipeline ownership, see [standardized manifest-driven pipelines](docs/GENERATED-PIPELINES.md).
The initial QBit profile uses a small `.qqq-ci.yml` and a setup entry point; existing direct job callers remain supported.
For centrally managed patch adoption and exact-version rollback, see
[managed patch updates](docs/managed-patch-updates.md).
**For:** Teams using CircleCI to build, test, and publish QQQ applications
**Status:** Stable (v0.6.x)
@@ -87,6 +89,11 @@ real publication, not a dry run, verifies the trust relationship.
| `hotfix/1.5.1` | `1.5.1` |
| `feature/*` | `1.5.0-feature-abc1234-SNAPSHOT` |
The Maven version step verifies the effective `project.version`, not only the
`revision` property. For older POMs with a literal project version, it uses
Maven's version plugin to align project and reactor coordinates with the
calculated version. A mismatch after updating fails before publication.
### Java Version
Maven jobs run on Ubuntu machine executors with Temurin JDK. Default is Java 21. To use Java 17:
+33
View File
@@ -80,6 +80,14 @@ required-check unknown states, and the lint image/tool/command/caller contract.
## Batch orb-version rollout
For a standard QBit without a manifest, start directly on the managed patch
track using `python tools/standardize_qbit.py /checkout --orb-version 0.8
--default-branch main`. Inspect the preview, then repeat with `--write`.
This preserves the inferred publication and branch-filter policy while writing
the same track to the manifest and setup caller. Use `rollout_version.py` below
for repositories that already have a manifest. Neither command configures
repository protection or automatically enrolls newly created repositories.
Upgrade existing canonical CircleCI consumers with one command:
```sh
@@ -103,3 +111,28 @@ verification and artifact evidence. The requested version must already be publis
```sh
python3 -m unittest discover -s tests -p test_rollout_version.py
```
## Live organization discovery
Use `--organization QRun-IO --live` instead of `--inventory inventory.json` to
include newly created repositories on each audit. Discovery uses the existing
`gh` authentication and all API pages, retaining archived entries for explicit
skip reporting. Unknown repositories receive no automatic exemption. Duplicate,
empty, malformed or failed discovery aborts instead of reusing a stale inventory.
```sh
python tools/audit_estate.py --organization QRun-IO --live --orb-version 0.8 \
--worker-sha <40SHA> --policy observed-checks.json --output audit.json
```
The organization mode has no fixed repository-count ceiling. Supply
`--expected-active N` for an independent count check when needed; saved-inventory
mode preserves its default of 68. Reports identify the inventory source and
credential visibility limit: inaccessible repositories cannot be certified.
This is discovery and audit, not automatic enrollment or branch-rule mutation.
For a qualified batch of canonical orb branches, use the shared
[branch-check enforcement command](branch-enforcement.md). It adds the observed
CircleCI setup/verify gates while preserving other protections. Source enrollment,
scanner transitions and scheduled reconciliation remain separate requirements.
Scheduling and authenticated onboarding still need to be configured centrally.
+4 -2
View File
@@ -111,8 +111,10 @@ The independent baseline fixture pins job sets, dependency edges, contexts,
coverage requirements and actual verification commands across all six workflows.
Core additionally supports `publish.feature` and `publish.hotfix` (both true by
default), preserving `publish*` tag and hotfix branch publication. QBit does not
accept these keys. Snapshot, RC and GA behavior remains distinct; RC/GA/hotfix
default), preserving `publish*` tag and hotfix branch publication. QBit supports
both keys as explicit opt-ins (false by default). QBit feature publishing requires
a `publish*` tag resolving to the checked-out commit; feature branch pushes remain
test-only. Snapshot, RC and GA behavior remains distinct; RC/GA/hotfix
publication still requires the complete source coverage gate. Cold bootstrap and
fresh consumer Maven repositories remain uncached. Shared installer policy replaces
local package preparation; preliminary compilation is disabled before clean
+11 -1
View File
@@ -37,9 +37,17 @@ fail migration rather than silently changing release behavior.
Use `tools/standardize_qbit.py --write` for the two-file migration. API callers
must pass the inferred policy to
`canonical_caller(version, manifest["release_tag_policy"])`; the legacy one-argument
`canonical_caller(version, manifest["release_tag_policy"], manifest["publish"].get("feature", False))`; the legacy one-argument
form is only for manifests without an explicit policy.
`publish.feature` is false by default. Opting in enables the audited `publish*`
tag contract, with the tag required to resolve to the checked-out commit. The
canonical setup admits those tags alongside the declared release tag policy.
Feature branch pushes remain test-only. Migration accepts Workflows' exact legacy
literal `feature/.*` branch selector (not a regular expression); Git forbids `*`
in branch names, so that selector cannot publish a branch. Unknown feature
filters require a central profile change rather than silently broadening triggers.
Focused tests cover the immutable inventory, unknown-filter rejection, canonical
caller agreement, main opt-in, broad/stable tag selection, HEAD provenance,
publication opt-out and test exclusions. The packed generator suite also exercises
@@ -51,3 +59,5 @@ CMS's original YAML is retained with its immutable source SHA: its unquoted
stable-tag regex is invalid YAML. The fixture repairs only that quoting. Both
audited spellings of the stable regex map to the same ASCII-digit tag policy;
arbitrary regexes remain rejected.
CI-only commits on publishing branches retain Maven verification without publishing when their changes are limited to the manifest, CircleCI caller, shared scanner caller, RC caller, or schema-drift caller. The commit must have one parent and a conventional ci title. Tags, application POMs, schema resources, tests, and other application inputs retain normal publication behavior. Canonical caller enforcement separately verifies the shared workflow definition and immutable worker pin.
+49
View File
@@ -0,0 +1,49 @@
# Shared orb check enforcement
`tools/enforce_orb_checks.py` replaces repository-specific scripts for the shared
CircleCI `setup` and `verify` branch gates. One invocation processes an explicit
batch of current `main` and `develop` branches using existing `gh` authentication.
Create a targets file:
```json
[
{"repository": "QRun-IO/qbit-example", "branch": "main"},
{"repository": "QRun-IO/qbit-example", "branch": "develop"}
]
```
Preview, then apply the same batch:
```sh
python tools/enforce_orb_checks.py --targets targets.json --orb-version 0.8 --output enforcement-plan.json
python tools/enforce_orb_checks.py --targets targets.json --orb-version 0.8 --output enforcement-result.json --apply
```
Every branch must already contain the exact canonical caller for its manifest and
have successful `setup` and `verify` checks from CircleCI app 18001 at its current
commit. Supported profiles are QBit, Core, frontend-core and frontend-material.
The complete batch is inspected before any mutation. Source/head/protection drift,
unavailable APIs, unknown profiles, and ruleset-managed branches fail closed.
The tool appends the app-bound checks, enables strict checks and administrator
enforcement, and reads the result back. Existing required checks remain required.
For protected branches it uses the narrow status-check/admin endpoints, preserving
reviews, signatures, linear history, restrictions and other settings. For a
previously unprotected branch it creates protection without adding human reviews.
It never changes scanner configuration, repository visibility, billing, or source.
Output records the planned head and original protection, then verification as each
branch completes. A batch is not transactional: if an API call fails, inspect the
output and rerun; successful protections are retained rather than rolled back.
An already-correct branch needs no mutation.
This command covers the shared orb gate pair only. Continue to run the estate
audit for scanner/native-worker requirements and other policy. Automatic repository
discovery, source enrollment and scheduled reconciliation remain separate work.
GitHub API reference: [protected branches](https://docs.github.com/en/rest/branches/branch-protection).
```sh
python -m unittest discover -s tests -p test_enforce_orb_checks.py -v
```
+37
View File
@@ -0,0 +1,37 @@
# Managed patch updates
The canonical caller and manifest accept either an exact stable version such as
`0.8.0` or a quoted minor track such as `"0.8"`. CircleCI resolves a minor track
to its latest published patch. A major-only version, `latest`, `volatile`, and
wildcards are rejected. The caller import, setup parameter, and manifest must
still agree exactly.
Use a minor track after qualifying that release line. Future patch fixes then
arrive on the repository's next pipeline without a version-bump PR or an extra
fleet-wide build. Existing exact pins remain supported and are not changed by
publishing this capability. YAML minor versions must be quoted to remain strings.
The central batch tool applies the same update to all supplied checkouts:
```sh
python3 tools/rollout_version.py --orb-version 0.8 --write /path/to/repo1 /path/to/repo2
```
This changes only the two canonical CI declarations. Publication policies,
contexts, branch/tag filters, and profile options stay unchanged. Normal source
and publication checks still run when a pipeline is triggered; this setting does
not skip tests. GitHub reusable-worker commit pins are separate and unchanged.
Automatic patch adoption trades exact-version reproducibility for centrally
managed fixes. Keep patch releases backward compatible, qualify central changes
before production publication, and use an exact stable pin for a canary or
rollback. For example, the same tool can move a consumer from `0.8` back to
`0.8.0`. Historical pipeline reruns can resolve a newer patch; choose an exact
pin when reproducing an earlier run. Setup and continuation each resolve their
orb imports, so patch compatibility also matters when publication overlaps a run.
Changing minor tracks is an explicit migration. Do not use this feature to push
breaking changes into a patch release or to bypass repository checks.
Reference: [CircleCI orb semantic versioning](https://circleci.com/docs/orbs/use/orb-concepts/).
+8 -1
View File
@@ -10,6 +10,11 @@ description: >
- Collects JUnit XML test results
- Stores test results and screenshots
- Saves Maven cache for future builds
parameters:
build_before_test:
type: boolean
default: true
description: "Build the React app first; combined jobs already build it with frontend_build"
steps:
# Ensure no stale Chrome/Driver is shadowing the PATH (your base image had Chrome 109)
- run:
@@ -17,7 +22,7 @@ steps:
command: <<include(scripts/frontend_test_cleanup.sh)>>
# Install current Chrome; DO NOT install chromedriver (let Selenium 4 auto-manage it)
- browser-tools/install-chrome
- install_chrome
# System libs Chrome needs in headless CI
- run:
@@ -52,6 +57,8 @@ steps:
# GPG signing will happen later in mvn_jar_deploy when deploying to Maven Central.
- run:
name: Build app, start server, run tests
environment:
PARAM_BUILD_BEFORE_TEST: << parameters.build_before_test >>
command: <<include(scripts/frontend_test_run.sh)>>
# Always collect and publish test results & screenshots
+5
View File
@@ -0,0 +1,5 @@
description: Install stable Chrome using only its signed repository and existing Ubuntu dependency indexes.
steps:
- run:
name: Install Google Chrome
command: <<include(scripts/install_chrome.sh)>>
+1 -1
View File
@@ -21,7 +21,7 @@ steps:
- when:
condition: << parameters.install_browsers >>
steps:
- browser-tools/install-chrome
- install_chrome
- browser-tools/install-chromedriver
- when:
condition: << parameters.compile_before_verify >>
+2 -1
View File
@@ -27,7 +27,8 @@ steps:
- git_full_checkout
- mvn_manage_version
- frontend_build
- frontend_test
- frontend_test:
build_before_test: false
- frontend_package
- mvn_jar_deploy
- git_operations:
+2 -1
View File
@@ -23,4 +23,5 @@ steps:
node_version: << parameters.node_version >>
- git_full_checkout
- frontend_build
- frontend_test
- frontend_test:
build_before_test: false
+24 -1
View File
@@ -161,7 +161,7 @@ set_version() {
##############################################
## Use Maven versions plugin to set version ##
##############################################
mvn versions:set-property -Dproperty=revision -DnewVersion="$new_version" -DgenerateBackupPoms=false
mvn -f "$POM_FILE" versions:set-property -Dproperty=revision -DnewVersion="$new_version" -DgenerateBackupPoms=false
#######################
## Verify the change ##
@@ -175,6 +175,26 @@ set_version() {
fi
}
ensure_project_version() {
local expected=$1 actual
# A revision property can exist without being used by project.version.
# Ask Maven for the effective coordinate, including inherited configuration.
actual=$(mvn -f "$POM_FILE" -q -Dstyle.color=never \
org.apache.maven.plugins:maven-help-plugin:3.5.1:evaluate \
-Dexpression=project.version -DforceStdout)
if [[ "$actual" != "$expected" ]]; then
mvn -f "$POM_FILE" -B versions:set -DnewVersion="$expected" -DgenerateBackupPoms=false
actual=$(mvn -f "$POM_FILE" -q -Dstyle.color=never \
org.apache.maven.plugins:maven-help-plugin:3.5.1:evaluate \
-Dexpression=project.version -DforceStdout)
fi
if [[ "$actual" != "$expected" ]]; then
echo "ERROR: Effective project version is $actual; expected $expected" >&2
exit 1
fi
echo "Verified effective project version: $actual"
}
#############################################
## Branch Detection and Routing Functions ##
#############################################
@@ -434,6 +454,9 @@ main() {
fi
echo ""
if [[ "$DRY_RUN" != "true" ]]; then
ensure_project_version "$NEW_VERSION"
fi
echo "=== Version calculation complete ==="
}
+12 -2
View File
@@ -11,10 +11,20 @@
## Output: Installs required system libraries
############################################################################
set -e
set -euo pipefail
echo "Installing browser dependencies"
export DEBIAN_FRONTEND=noninteractive
sudo apt-get update
ubuntu_sources=/etc/apt/sources.list
if [[ ! -s "$ubuntu_sources" ]]; then
ubuntu_sources=/etc/apt/sources.list.d/ubuntu.sources
fi
if [[ ! -s "$ubuntu_sources" ]]; then
echo 'Cannot locate Ubuntu package sources' >&2
exit 1
fi
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get update \
-o "Dir::Etc::sourcelist=$ubuntu_sources" \
-o Dir::Etc::sourceparts=- -o APT::Get::List-Cleanup=0
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y \
libnss3-dev libgdk-pixbuf2.0-dev libgtk-3-dev libxss-dev
+3 -1
View File
@@ -15,7 +15,9 @@ set -e
echo "Building app, starting server, running tests"
echo "HTTPS=true" >> ./.env
npm run build
if [[ "${PARAM_BUILD_BEFORE_TEST:-true}" == true ]]; then
npm run build
fi
export REACT_APP_PROXY_LOCALHOST_PORT=8001
export PORT=3001
npm run start &
+16 -7
View File
@@ -89,8 +89,8 @@ def load_manifest(path, setup_version):
allowed += ("java_version",)
mapping(data, allowed, "manifest")
version = data.get("orb_version")
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+\.\d+|dev:[A-Za-z0-9_-]+)", version):
raise ManifestError("orb_version must be an explicit stable or development version")
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+(?:\.\d+)?|dev:[A-Za-z0-9_-]+)", version):
raise ManifestError("orb_version must be a quoted minor track, exact stable pin, or development version")
if version != setup_version:
raise ManifestError("Manifest orb_version must match the setup job orb_version")
default_branch = data.get("default_branch")
@@ -112,7 +112,7 @@ def load_manifest(path, setup_version):
if profile in ("qbit", "core"):
publish_keys += ("asciidoc",)
if profile == "qbit":
publish_keys += ("hotfix",)
publish_keys += ("hotfix", "feature")
if profile not in ("qbit", "cli", "all", "app-starter", "voyage", "frontend-next"):
publish_keys += ("feature", "hotfix")
publishing = mapping(data.get("publish", {}), publish_keys, "publish")
@@ -141,7 +141,7 @@ def load_manifest(path, setup_version):
"docs_only": boolean(data, "docs_only", False),
"publish_context": publish_context,
"publish": {key: boolean(publishing, key, key != "asciidoc" and profile not in ("cli", "all", "repository-policy")
and not (profile == "qbit" and key == "hotfix"))
and not (profile == "qbit" and key in ("hotfix", "feature")))
for key in publish_keys}}
@@ -191,7 +191,7 @@ def proven_ci_maintenance(branch, tag):
if not changed:
return False
allowed = {b".qqq-ci.yml", b".circleci/config.yml", b".github/workflows/codeql.yml",
b".github/workflows/release-rc.yml"}
b".github/workflows/release-rc.yml", b".github/workflows/schema-drift.yml"}
return all(path in allowed for path in changed.rstrip(b"\0").split(b"\0"))
@@ -215,11 +215,17 @@ def event_kind(branch, tag, release_on_main=False, release_tag_policy="stable"):
def qbit_template(config, branch, tag):
"""Profile-owned template; add future profiles as reviewed renderers, not raw YAML."""
kind = event_kind(branch, tag, config.get("release_on_main", False), config.get("release_tag_policy", "stable"))
if tag.startswith("publish") and config["publish"].get("feature", False):
tagged = git("rev-parse", "--verify", f"refs/tags/{tag}^{{commit}}")
if not tagged and git("fetch", "--no-tags", "origin", f"refs/tags/{tag}:refs/tags/{tag}") is not None:
tagged = git("rev-parse", "--verify", f"refs/tags/{tag}^{{commit}}")
if tagged and tagged == git("rev-parse", "HEAD"):
kind = "feature"
if not tag and re.fullmatch(r"hotfix/.+", branch) and config["publish"].get("hotfix", False):
kind = "hotfix"
if kind != "test" and not config["publish"][kind]:
kind = "test"
maintenance = kind != "test" and proven_ci_maintenance(branch, tag)
maintenance = proven_ci_maintenance(branch, tag)
if maintenance:
kind = "test"
workflow = "test_only" if kind == "test" else "publish_" + kind
@@ -336,7 +342,10 @@ def frontend_template(config, branch, tag):
kind = "release"
if kind != "test" and not config["publish"][kind]:
kind = "test"
if kind == "test" and proven_docs_only(config, branch, tag):
maintenance = proven_ci_maintenance(branch, tag)
if maintenance:
kind = "test"
if kind == "test" and not maintenance and proven_docs_only(config, branch, tag):
result = qbit_template(config, branch, tag)
result["workflows"]["test_only"]["jobs"][0]["docs_only"]["name"] = (
"qqq-orb/mvn_frontend_test_only" if material else "qqq-orb/node_test_only")
+25
View File
@@ -0,0 +1,25 @@
#!/bin/bash
set -euo pipefail
# Machine executors are Ubuntu amd64. Preserve an existing browser for Maven
# consumers; frontend_test deliberately removes its stale browser beforehand.
if command -v google-chrome >/dev/null 2>&1; then
google-chrome --version
exit 0
fi
architecture=$(dpkg --print-architecture)
if [[ "$architecture" != amd64 ]]; then
echo "Google Chrome requires an amd64 machine executor" >&2
exit 1
fi
curl --fail --silent --show-error --location https://dl.google.com/linux/linux_signing_key.pub \
| sudo gpg --batch --yes --dearmor -o /usr/share/keyrings/qqq-google-chrome.gpg
printf '%s\n' 'deb [arch=amd64 signed-by=/usr/share/keyrings/qqq-google-chrome.gpg] https://dl.google.com/linux/chrome/deb/ stable main' \
| sudo tee /etc/apt/sources.list.d/qqq-google-chrome.list >/dev/null
# Ubuntu dependency indexes were refreshed by install_java. Only the newly added
# signed Google repository needs refreshing; image PPAs are unrelated to Chrome.
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get update \
-o Dir::Etc::sourcelist=/etc/apt/sources.list.d/qqq-google-chrome.list \
-o Dir::Etc::sourceparts=- -o APT::Get::List-Cleanup=0
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get install -y google-chrome-stable
google-chrome --version
+22 -4
View File
@@ -29,8 +29,21 @@ if [[ -f /etc/apt/sources.list ]]; then
sudo sed -i 's|http://security.ubuntu.com/ubuntu|https://archive.ubuntu.com/ubuntu|g' /etc/apt/sources.list
fi
# Add Eclipse Temurin repository
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get update -qq
# Base tools come from Ubuntu, not the unrelated PPAs bundled in machine images.
# Support both the 22.04 list and the newer deb822 Ubuntu source file. Preserve
# other indexes for later jobs without contacting their repositories here.
ubuntu_sources=/etc/apt/sources.list
if [[ ! -s "$ubuntu_sources" ]]; then
ubuntu_sources=/etc/apt/sources.list.d/ubuntu.sources
fi
if [[ ! -s "$ubuntu_sources" ]]; then
echo 'Cannot locate Ubuntu package sources' >&2
exit 1
fi
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get update -qq \
-o "Dir::Etc::sourcelist=$ubuntu_sources" \
-o Dir::Etc::sourceparts=- \
-o APT::Get::List-Cleanup=0
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get install -y -qq wget apt-transport-https gpg maven curl
# Add Temurin GPG key
@@ -39,8 +52,13 @@ wget -qO - https://packages.adoptium.net/artifactory/api/gpg/key/public | sudo g
# Add repository
echo "deb [signed-by=/usr/share/keyrings/adoptium.gpg] https://packages.adoptium.net/artifactory/deb $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/adoptium.list
# Install Java
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get update -qq
# Refresh only the repository just added. Re-fetching unrelated image PPAs here
# caused otherwise successful setup to fail on Launchpad 503 responses. Keep
# Ubuntu indexes from the first refresh so Java dependencies remain resolvable.
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get update -qq \
-o Dir::Etc::sourcelist=/etc/apt/sources.list.d/adoptium.list \
-o Dir::Etc::sourceparts=- \
-o APT::Get::List-Cleanup=0
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get install -y -qq "temurin-${JAVA_VERSION}-jdk"
# Set JAVA_HOME
+17 -8
View File
@@ -16,7 +16,7 @@ import yaml
# Generated by scripts/pack_pipeline.py; edit source/resources, then regenerate.
import base64
import zlib
CORE_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNrtPdl22ziy7/0VuOrM2M6IpPclifvcxHbfeCar48z0nChXByJBiW1uJig7Oonn26cKACmKIiVSluUlTr/IJFAo1F6FAvv7L4Q0LljEncBvPCPr+loTn4Q0oh6L4Tk8/A5P4JkX+N2g3XF8q205tOsHPHbM4XsYEQ9CBn83OkHgMuo3mskLi9m078bwzqYuZ+LxlXzb8Nm3uE1Nk4Ux9U3W7vR9y2XtfuQWgeZx5PjdIsiNRhWwvEfXt7bnCjliJnPC+Fqgf1HgG0HUyZD8/Pxcgyc48Awg8CgIzgz18H9X9W19r5HONAPPo76Vmc1jGsWaYJuGbNPiiJosiyCPWYgTvqgHCKbHzLOgH6fYknS4eB/1/QwE9dAHYUEkP0QMJIeR4NJnFnmLKxMemGcsJlmRaY7OVpgjAMcmvG8FhIax1oVZ/dCiMSPa+Tn5619HXzk+7M91iTYQr7nYHIxq+QBTgSTaBfE580GQCecpiHQqDPDIztYWMWIvFITNUCuDcQJWzFYraYyIH/s4mGjBNBhGGDHbdbq9WHeDLszts+ck7jEfIYcgILFNlti3MIhi8vHjx/bb9+/+73379OTlwVH75PO7/b+ct1r+Emk1nhwcnxy8OWr/6/3JP35/8/5f7b+/f9U+Pmw1yG+/4etXLz+9bh+9+2er0fIZaBvCZ2YvIEuSIYg1CBNJ8SF9n15Qx6UdF1AyAz92/D4wMXK6jk9d8pZeMJ8IweD6Usu3nZbfyLDwKv19NYvQvO9wFl3Aer47AII4nPwZdJY4AXyY66ZyZDvf4n7ExqSnQ82zbhT0hQAhVculK2Y8JpqPRPpeQOJn2hUQ8ccPwr45MVlt+YLdmgaEEhhqzL/Yl8Q/OG4Wc6FZAJeEg7gX+BtEN53IdJnpGEJyMoKih4NSmqpfX0eMEI+DsKZqV+fI747v8F5NLb4EUcbJ1L2kg0k6PhMX4qBv9qaqGFJFTBXifhD0XYv4QUy40xWCLLZiBiBVZhxEQpaDiNA4Zl6IFoU8WebsnKyR9dWV58QKUHUEuv9DNHvq6iAmYNX1P3ngIxKdiNEzhMBdxkKy1vKtwGdqP2bPC8BqnBD6t+iPqeanYEsePWMEjdhwHAFz79jUjDmBpS1UZ9wi2FRJ8lo7GNomk8Z1JhYbnQyWyl8Sz+Ec7NDzRLZQ3CLec0IY4VGw0WCXwigA2zOb0QFhiFg7pcm4uIc07qFATtlcXpQtICbYxVhGTI2SSTV0GSyu7XTBtmkh2DLaZZpyUPPS53+g/CnYifMT+KPlB+H3LWZNUNhfyWkPLCDp9B0QPo+aPcdnnKCjtxxu0gimE2qjkwWZQeutt/xfyWeY8hnCrri/hINhFhh5z4miIGqSy54D+gzBIKgCjSIH4OFcVFQAxpnZj5x4AA7XiZmudIbDG80hS/xHL47DZ4aRDNP7YhkdcDbkTzGCwxC1bsGI7hIxWGwaEE8YPOhHJuO66/BYLZaJEcQwnzELpBxDKgMZplstP/Haf+HCNS898SN89V2NuyL7YOTcVuP5EvkhY4eYsTJwxt6ehkJoOjo+IL8ZFrsw/L7r1vYMIEmceqHLNKmXfF6CdCBNJ/n7Zx9ss7AqEPFBxDK7N/DOLCciWkhw7wpr0Po/YRkDqAORnvFnH1ZLtiJNtlwVLXb5NA4aZTsREkGgaJwefTrVnurfPHfCLBsCIU7tolmJRwCD+gWtKTgwOQj81dehvQSLGY68q7gzAyejmZNuYhZjhxxpj/M8Z+8q4nMbCAiFYJFGw9B1TGGjNIHUDZn+ckw6UXDJEZM00Vs4CmBcQ5cOtEvHgj/5wteX62qYEESBu/j1bUYx4gcELliEbjETZSwcGcz6M8KggaR22SRUCo1ziJUWHidrsAvHYv5o4F5QepF76IGjzWGerzCMrZ4SZC5mX2JNEqwJWmL02gmDSBeS9et7AhGP5ShkvHhBhpTRBTEg35XeAOaJgH6ADiFv9kneopNSI0PGlZ7klJDklGLUJ1hg98tF6EmKZ4HDgFSg8twaBLqWU1HS2o6D9mUQnfFwNMdMRCcI4jSMzgt2c1zPRqVQvSjGvjEy7GsVRUvrYRCGZmphjHfawN0zFrUxugoij+b0TsW1o0rneCDWuDkZNGrr66ubzyDojJifuqShjoF9EoFk23Qpx8UbLnKv0ZxUbysN/oekS+qAhnrV/pNe0Oz7gqJdOse78Nsicp+povdPFjn2gBxiihYRBwL5pFIktF+SlGRJWq7glgICY+dYPjrp++Ql6LOHyPkWOaGdjhO//VgPN6AS0V4RjUvlSgs18FzjLI6x+ioiRy2U0SMIFNGoR7RD+KVnFtH5mRPuizo3MbEMTi4EDee5ZXbeB3OQ7FDlEdmdEsjt3jh+/9uELSdVKY20Wi9eLH34N2bZUeARVFLX6RDHE0H2B/hTvQEC6CyOIIlR745c5oEqnMKjFtiyxMLui0nL0qABfcqC61ZjRVpwtRMw3zBJcbPVaIJJTNgJQ59JY4nowQrJWgaxWw0RpMPe9LRMrsPSep413+U6VwfDZ8enyFaslRJR4IV8E4Bn9qWDXeJsGVdd0WETaO6WV+Rwj8agerjfLyblUiXED+GHMHW1Hd8CjYVdYWiM72Aj6C3wJ4KDN8hWfLqP6SJY+cD3wdTzl5hRvxIIn0gOtxpf5bpgXRhQ32X+skJBzF5rIi2STT7Ly4fI11TVJdmwgqSgfFn9KjAGpFCKQ2YJbDl5B96jAvBLCt41mdjyP/z7ptOYCbJVK2cWXqvdAdZidT9sSzuedQ/sGzP7gF2yMprWETO8SONeI3TDySTdGIHBPnANDaXLIJJBBnaB1Hj+xysYi6IIBbk1JN0wdhJF7WkTVZDfTmLISZXw2Y6kADCR/MzQQZgzsM1Y+B0kJxwQAbDZaFCyfaLJPCFZX9MwjsIwN605pjM1JXPzdxMqXhknQlmUNs/izXB/adpQudKTFFpKiWVkqC3K7pmo2gxrzKuCf27OjayRRvOy4I3+BA8LwZ2Ur/MUzxMLalPwdJhnqJqUfFYLk+eZ9OEul95mrbpl9zdBw2UQp+VrEoVKHknFSwarvFlPiXuzeVjpxisnY6UyXykfu9sVyusVBcfU4z4UxSaFPJny5iKCnRFqFAY+YyRJ3vqBxdQJ/zSa32A+/EqcgdlYqUBVJBYL8fjMN/EEK6mHSdrOJds8jEDSMDjbX9Xhv0QoMgUq7RAj7lPQK67yzYTs84skZBQJBBXbCx1fNAZQyE8c6hKL8l4nwOpk+Y67DuSnLhhZkpzMwZNevyNO5D5CDq8dvxfuDQIzH08kNU+B11LwOgLJVpnGh7R8HKMdTBlGEgEBY22xGOI+srazu8k22a61vmdumXvM3jU3V5m5aa9vm3TLtOyt3c7a3s567eCUSaWXweUYhc/YoNAEpzj7oaddrGlV0dO+f1edOn0P3OFkMhiJorqBeSaDmga5urqOla+gPBNFiNAuHv3HshEo1bMJkmVaU2XiVwIJPxAE3KjojJDAlZoAGh/evyWXII6JgAvzsMSTLjLNdUBuwQ6DH45Bz/VhoKCZZGLVJIQXaTEE/pC1hpUWiPi3GN7gI8SojX9jVSEAAmEZ4AWWL1Rf5m+bqPovjOwjTK9V7o5TQY36frwMs5NCQKvxQdJ4SNqPHz8SNR9khPpdmaUjDpeREzOJhAAHUZQLQQYCbOaxKTFEefxWYDdAG5BfYjqgZy7rUnOghYxFoHMhl41VNvwV2LagsKYJJSH/MXSY1fIPjvfR2hMEITtYNMygsFTx0nVViQ2irb5/7L8CPiGj3+JbwUSJGRAAGGf1TdHuIBj7HDuLsIWjqyI2kJmQXNLIR7Ormk/Sv3WBh1wLEYHVJJiWn2Y/PDINnGQkBWBuFImhrPKLyYZebZKBezqV0grvsIlJUCJ1OwMyIsmihw42KgR8pF8D9lHD62RYOcHxVPBN3bArC6KCk0vaIaK//xTd1dKou6pdKoKcfYJVFW72No1o1WBbSvuN2Vwsj6tzO4vGtElen55+EPUfddhFMiFghWLxK2yZxSBcyqHDRSCKQjUes6JuLImI/ynkSUupLH98BU4ay/WJAgqxVQ2ezYw4g9AHIy2hgQkaIJheT55rRVGQX2b2glTTzjtObCj7nRfb+aJRTfNwVBIp6QWR0lQIdlGOm2xQ+5A56Eb+8WZKhSyU7PHGXIt2KLRvqNexEgmpJqRAbaKxfgDhRciwmjBn5qijH8+xLJeBgwCdF0g28Wk/dlwnHqhHPEecCpWyfNFEOIzC9SaXWUaPQeRaNAz1T+KXJOuhw0PhR9PdoehPqc+MLZdimNv7dfD7dM4/S2jXxW2qPBa3ys2pfFGZaTd7QFKRMzPVoyZ0seR6VlS1AVgfsRmKFB7s08H8f9Hlils8eL8jhYaU9AupM4hgBXCChcWa6qKUPNgXqMjny69frlRzCfMkQE2fSVotLFmrkFdZN7G5dwDuUG3tVCQ2h4lC6ui2ju13wacQ0LQdZgmKJ8nOoRPrGXBvETmEdnwK79KadI+68Xv/d4AEUAtaEebrq5NmhCDuoZS6wQCPzokHaHEC8VsPAnVMmxbbiiAN3WgnQqHrSHsQpClsEs/xHQ8Cf2xGwIPwWZxYIZtl8t8k2ytNefhe9A9WLD4HmbxiRhKSZdbWVpKWifK2BkUmQ+1+rMMhAqHZxzh8WfYzyI4FId+Y4gPH8u+GjQMrIy0GCOm3/YS6splALnoFsgVvr4hUG/jZhBQd5B+z9+9qwlWuYyG/rC2lXWK1v09Whc3Ij2J4+2A4pqm2XbFVoThgmOS5TFV3enRco44roQt/QEXyxBCqnLbLfLDGKMKq31QIZKbnNOPARPWRuHhB5u3g08c3ear4QRt4Ffbjdux4DNkG622sevVyoVQYUcH3nyx7Z3jvTdxwSQ6X09bYoS/7Q/zDkiLine truncated
CORE_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNrtPWt3m0iy3+dX9NVk13ZWgN+PJJ5zE9tz4908HWd39kS5Oi1oJMYIMI3s6CTe375V3Q1CCBDIsvyIM19k6K6urndVVzPffyGkccFC7vhe4xlZ19ea+CSgIe2zCJ7Dw+/wBJ71fa/rtzuOZ7Uth3Y9n0eOOXoPI6JhwODvRsf3XUa9RjN+YTGbDtwI3tnU5Uw8vpJvGx77FrWpabIgop7J2p2BZ7msPQjdPNA8Ch2vmwe50agClvfo+tb2XCGHzGROEF0L9C8KfMMPOymSn5+fa/AEB54BBB76/pmhHv7vqr6t7zWSmabf71PPSs3mEQ0jTbBNQ7ZpUUhNlkaQRyzACV/UAwTTY+aZP4gSbEkyXLwPB14KgnrogbAgkh9CBpLDiH/pMYu8xZUJ980zFpG0yDTHZyvMEYBjEz6wfEKDSOvCrEFg0YgR7fyc/PWv468cD/bnukQbitdcbA5GtTyAqUAS7YJ4nHkgyITzBEQyFQb0yc7WFjGifiAIm6JWCuMYrJitVtIYET/2cTDR/GkwjCBktut0e5Hu+l2YO2DPSdRjHkIOQEAimyyxb4EfRuTjx4/tt+/f/d/79unJy4Oj9snnd/t/OW+1vCXSajw5OD45eHPU/tf7k3/8/ub9v9p/f/+qfXzYapDffsPXr15+et0+evfPVqPlMdA2hM/Mnk+WJEMQaxAmkuBDBh69oI5LOy6gZPpe5HgDYGLodB2PuuQtvWAeEYLB9aWWZzstr5Fi4VXy+2oWoXnf4Sy8gPU8dwgEcTj50+8scQL4MNdN5Mh2vkWDkE1IT4eaZ93QHwgBQqoWS1fEeEQ0D4n0PYfEz7QrIOKPH4R9cyKy2vIEuzUNCCUw1Jh3sS+Jf3DczOdCMwcuCYZRz/c2iG46oeky0zGE5KQERQ+GhTRVv76OGSEe+UFN1a7Okd8dz+G9mlp8CaKMk6l7SYdlOj4TFyJ/YPamqhhSRUwV4n7gD1yLeH5EuNMVgiy2YvogVWbkh0KW/ZDQKGL9AC0KebLM2TlZI+urK8+J5aPqCHT/h2j21NVBTMCq639y30MkOiGjZwiBu4wFZK3lWb7H1H7MXt8Hq3FC6N/CP6aan5wt9ekZI2jERuMImHvHpmbECSxtoTrjFsGmSpLX2sHINpk0qjMx3+iksFT+kvQdzsEOPY9lC8Ut5D0ngBF9CjYa7FIQ+mB7ZjM6IAwhayc0mRT3gEY9FMgpm8uKsgXEBLsYyYipUTCphi6DxbWdLtg2LQBbRrtMUw5qXvr8D5Q/BTt2fgJ/tPwg/J7FrBKF/ZWc9sACks7AAeHrU7PneIwTdPSWw00awnRCbXSyIDNovfWW9yv5DFM+Q9gVDZZwMMwCI993wtAPm+Sy54A+QzAIqkDD0AF4OBcVFYBxZg5CJxqCw3Uipiud4fBGc8gS/9GLouCZYcTD9IFYRgecDflTjOAwRK2bM6K7RAwWmQbEEwb3B6HJuO46PFKLpWIEMcxjzAIpx5DKQIbpVsuLvfZfuHDNS0+8EF99V+OuyD4YObfVeL5EfsjYIWKsCJyxt6ehEJqOjg/Ib4bFLgxv4Lq1PQNIEqf9wGWa1Es+L0E6kKaT/P2zB7ZZWBWI+CBimd0b9M8sJyRaQHDvCmvQ+j9hGQOoA5Ge8ecAVou3Ik22XBUtdvE0DhplOyESQaBonB59OtWe6t/6bsksGwIhTu28WbFHAIP6Ba0pODA5CPzV15G9BIsZjL2ruDMDJ6OZk25iFmOHHGlP8jxj7yricxsICIVgoUaDwHVMYaM0gdQNmf5iTDqhf8kRkyTRWzgKYFwDlw61S8eCP/nC15frapgQhL67+PVtRjHiBwQuWIhuMRVlLBwZzPpTwqCBpHZZGSq5xjnASguP4jXYhWMxbzxwzym9yD30wNFmMM9WGCZWTwgyF7MvsSYx1gQtMXrtmEGkC8n69T2BiMcyFDJevCAjyuiCGJDvSm8A80RAP0SHkDX7JGvRSaGRIZNKTzJKSDJKMe4TLLD7xSL0JMEzx2FAKlB5bg0CXcupKGltR3770g/PeDCeY8ai4/tREkZnBbs5qWfjUqhe5GPfGBv2tYqiJfUwCENTtTDGO23g7hkL2xhd+WGfZvROxbXjSuf0QaxxczJo1NbXVzefQdAZMi9xSSMdA/skAsm26VKOizdc5F6jWVZvKwz+R6SL64CGetX+k17Q9Pucol0yp3/htUXkPlNF758sdOwhOcQULSQOBPJxpUhovyQpSZO0WMEtBQTGzrF8dDLwyEvQ5z4i51nkhHY6TvT2Yz3cgEpEe0U0LpUrKdTAc42zKMLqq4gctUBGjyBQRKN9oh3CLz21iM7PnGBf1LmJiWVwciFoOM8ts/MBmIN4hyqPSO+UQG73xvEG30q2HFelNNJqvXix9OHfmGWHfp+gkrpOhzh9EWR/gD/VGyCAzqIQkhj17shlfVCFU3jUAlsWW9h9MWlZGjSgT1Fw3WqsSAuudgLmGyYpbrYaTTCJMTth6DNpLBE9WCFeyyB2qyGCdNibnpTJdVhaz7Lmu1zn6mD07PgU2Yq1UiIKvJBvAvDUvnSwS5wt46orOmwCzd3yihzepxGoHu73i0m5VAnxQ/ghTF1tx7NAY2FXGBrjO9gIegv8ieDgDbIVn+5jughW3vc8MPX8JWbUrwTCJ5LDrcZXuS5YFwbUd5m3rFAQs9eaSIt4k8+y8iHyNVV1iTesICkoX1a/CowBKZTigFkCW07egfeoAPySgneNJ7a8D/++6TSmRLZq5czCa7U7wFqs7gdtacfT7oF9Y+YAsItXRtM6ZoYXadxrhG44mSQbIzDYA66hoXQZRDLIwC6QGs//eAVjkRehILdGpBvFTqKoPW2iCvLbcQxZVgmf7UgKABPJzxQdhDkD24yF32F8wgERAJuNBgXbJ5rME+L1NQ3jKAxzk5pjMlNTMjd/N6HilUkiFEVp8yzejPaXpA2VKz1xoaWQWEaK2qLsnoqqzaDGvCr4Z+bcyBpJNC8L3uhP8LAQ3EnxOk/xPDGnNgVPR3mGqknJZ7UweZ5KH+5y6W3Wqlt6fyUaLoM4LVuTyFXyUCpePFjlzXpC3JvNwwo3XjkZK5T5SvnY3a5QXq8oOKEe96EoVhbypMqbiwh2xqiRG/hMkCR+6/kWUyf802h+g/nwK3EGZmOlAlWRWCzA4zPPxBOsuB4maTuXbPMwBEnD4Gx/VYf/YqFIFai0Q4y4T0GvuMo3Y7LPL5KQUSQQVGwvcDzRGEAhP3GoSyzKex0fq5PFO+46kJ+6YGRJfDIHT3qDjjiR+wg5vHb8Xrg3CMw8PJHU+gq8loDXEUi6yjQ5pOXhGO1gyjASCwgYa4tFEPeRtZ3dTbbJdq31PXPL3GP2rrm5ysxNe33bpFumZW/tdtb2dtZrB6dMKr0MLicofMaGuSY4wdkL+trFmlYVPe37d9WpM+iDOywngxErquubZzKoaZCrq+tY+QrKUypChHbx6D+SjUCJnpVIlmlNlYlfCST8QBBwo6IzQgJXagJofHj/llyCOMYCLszDEo+7yDTXAbkFOwx+OAI910eBgmaS0qpJAC+SYgj8IWsNKy0Q8W8RvMFHiFEb/8aqgg8EwjLACyxfqL7M3zZR9V8Y6UeYXqvcHaeCGg28aBlmx4WAVuODpPGItB8/fiRqPsgI9boyS0ccLkMnYhIJAQ6iKBeCDATYzGJTYIiy+K3AboA2IL/EdEDPXNal5lALGAtB5wIuG6ts+Mu3bUFhTRNKQv5j6DCr5R0c76O1JwhCdrBomEFhqeKl66oSG0RbA+/YewV8Qka/xbeCiRIzIAAwzhqYot1BMPY5dhZhC0dXRWwgMwG5pKGHZlc1nyR/6wIPuRYiAqtJMC0vyX54aBo4yYgLwNzIE0NZ5ReTDb3aJAP3dCqlFd5hE5OgROJ2hmRMkkUPHWxUCPhYvwbso4bXSbGyxPFU8E3doCsLooKTS9ohor//FN3V0ri7ql0qgpy9xKoKN3ubRrRqsC2l/cZsLpbH1bmdRSPaJK9PTz+I+o867CKpELBCsfgVtsxiEC7l0OEiEEWhmoxZUTeWRMT/FPKkpUSWP74CJ43l+lgBhdiqBs9mSpxB6P2xllDfBA0QTK8nz7WiKMgvU3tBqmnnHScylP3Oiu180aimeTgqjpT0nEhpKgQ7L8eNN6h9SB10I/94M6FCGkr6eGOuRTsU2je037FiCakmpEBtorGBD+FFwLCaMGfmqKOfvmNZLgMHATovkGzi00HkuE40VI94hjgVKmXZoolwGLnrlZdZxo9B5Fo0CPRP4pck66HDA+FHk92h6E+pz0wsl2CY2ft18Pt0zj9LaNfFbao85rfKzal8UZlpN3tAUpEzM9WjSrpYMj0rqtoArA/ZDEWKPuzTwfx/0eWKWzx4vyOFhoT0C6kziGAFcIKFxZrqopQ82BeoyOfLr1+uVHMJ8yRATZ9JWi0sWauQV1k3sbl3AO5Qbe1UJDaHsULq6LaO7Xf+pwDQtB1mCYrHyc6hE+kpcG8ROYR2fArvkpp0j7rRe+93gARQc1oR5uur42YEP+qhlLr+EI/OSR/Q4gTitx4E6pg2LbYVQRq68U6EXNeR9CBIU9gkfcdz+hD4YzMCHoTP4sRy2SyT/ybZXmnKw/e8f7Bi/jlI+YopSYiXWVtbiVsmitsaFJkMtfuJDocQhGYf4/Bl2c8gOxaEfGOKDxzLvhs1DqyMtRggpN/2Y+rKZgK56BXIFry9IlJt4GcTUnSQf8zev6sJV5mOheyytpR2idX+PlkVNiM7iuHtg9GYptp2xVaF/IChzHOZqu706LjGHVdMF/6AiuSxIVQ5bZd5YI1RhFW/qRDIVM9pyoGJ6iNx8YLM2+Gnj2+yVPH8NvAqGETtyOkzZBust7Har5cLJcKICr7/ZLl/hvfexA2X+HA5aY0d+bI/xDLine truncated
MATERIAL_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNq1lMtugzAQRfd8heV1TNRKlar8QXdVt02EXDMhDthGfqRBiH/vQAmPvKRU6g7PjH3PXI+pI0IoHEEEb6yjK1JjAENlwatvK7OdH2IYTY3IwWLks4+QIdflpeIZYJoqYWMlhTXObH0sjFqOB64OT/HLa/zM9lypig77m/5rE51WzaKl25uvq2CJB+fdFO/UR0swaWBxyjsPpZvRU7EDkZswFp11ZFHDWEgEx8qJVp/OoZof2Md1qViKYqyuSSfhgiJrWnKRo0OsQB/jvTN6TUnTTLTPt9NZZjNZNaNvt9CDvgTWXHUX9Kad50VBUAV0ClpIcGccFK9NcZ225QhEhCSMFZBxUbESwHaE9CEkxw/3rOyV/uwcVvnd9fswKSTKpKEA92+efgRN3oexIwfpAi+IhQyHyEmjye/A3nP5SMa57cof9LebVW693HIxexsTg+avg1kojfXLc6wUxaXmHrmvbvgLWNtQgmaE4h5bW8X6quUNmctfRdREPykCLqM=")))
VOYAGE_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNrtW+tz2zYS/56/AqPJXJKeSUqqHzndqBNbdm1NGzsnObneVDkNREISLBKgAVCWYuv+9lvwpQepV2K1TpoPsUhggV38drEPgLl/hlBhSISknBUqqGyW9nTLDe9IeL2HZ3jrBNR1DMwcQxGp0nbo8bDdp4zMNEEj9XBPNxWCTsBUYJTLxf2KHQhBmCrEZJO9ZApBJA+ETdq2i6VmWnCxgPEpAWFDKjjz9Og5Pm+PP5xdtq/eXTf1KOM3bwSMXntZFlIRX8/8+3Ss3Sf2gAcqZYNmZob+29tbg4uORZlU2HXbN3iI57iHVLq1PUWvUC4VZggm6fNkyiWd2Ruytg3wkTYgoLggy0QRActyZtgLIf5ABO2OEfZ9l9pYgRx7iDJFeiJ8QVphcg/dBtilCuiYg2wOEuPeLL8IEu550K9nBdmQ7RLM0DCa3zhBxvU+MozAd7AihmTYl32uJDQxbiiBmewSYfiC92A58nNgkHi4FIMQoLZejEYrcJXMQuJj1dfCK20+ypKBIF0qiCGIz4VaL9KXMuxi6krc/SKGWCjaxfYG3IZ8DDo0b7BY1KMDQlMWql/Tg2mEVDuVRlJF1sgRm2DoRlIT3EooX280UIji7TsuBtLHNslKJjjXfqJgLsqjRZ73AnFHvIbQ/xC5MGzar43BWEeUsbplhEv1h1aBOu28wTa3uUlGxM7SnHJ7QESXujnjfe6ZI8/NdkiRM5Npc9alvRxqznC432+IrUz4Be0oSuasHqGP+Qp+Ntsbq7sgiS2Ikoa08azDKzjhYuYUN2cWabixqdezOliSxWAzw3FF2JEeOPrClwSNlZ66HkUSdE4VeNaBXOF+70DByLhFfaV8WbGsHlX9oAOq8PRjOHr6IAj8gklaDr9jLseONXxtlktmOSVpR+9tl7Jg1B4d7ptgQ2bvU4vBLzJGn7poPand97iD/j5KSVtsu927EpxGwDYBxkzXhByiwPAg/ERqRCY8RraKzITIVNxzoR2cTYfLZd4m3xgxbPbPtkSw40AwBRtU+JYkHsREv+LiMHvayiY94tDA251RNiPZUPO4eY2aernLwY/XgTQqqNVqMTSFHAeKL7b5lk6Psq38DkvfUBz+EZbtBjcQiDBQBA5VaT8Rgov07QbcD/zAsv1AoVgyI47UZtgbkZro4QEpEZAW26X8Jir/9LcSAlaEzIijQ4CpRmoqxI7CcB4Ca+Jx7pBdS5cAsrFs6YDtAklkQYv7dyfVyvotOgcaVgqkWJXA4Dh92c67RjVElEpn5xyQcW7qE2UhkL/T4dgYFo8Oi8ZGAfyzfI2d1BOUOLMVC/hynzCHMJtmMqsC4+1ol7cV9Yj2czBV6cBb6aggLpCAI5/6RGflEOgAHmRIlEm+WswbOBSioI9ahecXV2/PLDPE0AoxaRWSIalRxYldi0XRAMGawTsIL3EUQz3Ru3+fWhleFUsb5GyD4MBgybhFhhUr4Tw/JCMzEEIKnDSl1BgScJgzpimaR4dm8Y3s4/LBYaW8Xzp0St2j/eL+QefgqHy073RL+Mf9IiHksHNQKsJcztGP9uEhPrTtzmsHF/9R7pRKR6Rz5JTLZaQ5dmXqK/U6GWTraBi4Gh1JtN6h/ryon1/s1Rr163rt+Fft1UcUcmruEFRKB7uQ4xuQKRn+oKfryy4XHlZo3uUnYFizthM5/hDlrT0tFJ8rNo82q5y9cn+PQAy7jyaTjauN/80p60t229ZF2oJFrXHCKbLgSOORW3nhsEB4Ii54bZa0I7f8tRwjNbWyai4PHHTMsDuGOnvN8ZBxgkINV8K/yeY9Dd/MuCz8hYyrt8DXoLwd7Z4FOi56mNFPockllAskfS6VGQi3mhRDYbOtRTUzxIoPCKs+v29eXR432tdXv5xdThZIkqMHMy6goRRuhNvhnd6w1ZnSOy6xrSnhwlTS9kyHStxxiVNdmd7lbpAofhjh4epfeIustMsTDQ6KjjNQ3Vt9ahkH5BBQZCgUmVzl+X2t3qj9etZuXhyXKsXK0QSZafRWuLeCLu6JKrdHLXivtfNHNQ4lOOharKu9rhv1D/9pfzhrNOtXl9VWoWgeQKXeKqzIQqwhFhY0WxGFKeGnkteYmx88v59jOUGhAU4DfBLQk1i+pyP7tFjLCeu0x8A7GQFUUSPI/6LmpdA/KtjvAtlHUOCdU3URdNA7bA+A6yoPR+w+18nV/flFLXEkkEk9oBhul/coQ72+LbQTMoIp7fvmWePy+O2ZJjcMH7YfbBjHkMqhbEOri6e1YodorQApO19kq8vmSCw5HudrYLZnt2pYwoF20e8hKvEsJ43jy9qFRqUKzQ7Yj8t9ePuoa+MVlB7sD032T6T6hGmbeRQMEybT+TbCYmZYbCLatsCcNx0IO7ZLW2y7wVMNFLYLMbLDvc8/SuWQ/904g0q5ZBb/gKPUR05pzgn4VTBG1Dy5eote1saQMzBy+turNYkN5CSmHRE7o0r6ZHhQITDDdwPY+RUPD8hxrydIDzic8NTtnkblyc9hyVKdOYdKei5BuqpWS9J+srMzoSSVAWYbnQeldFsYWHwdaDthTfEkT+3nwSMq8MGsPK5IOxVxAcCZTNwhXRy46hFj0fsQMVQ7RRoxdEdVHzFyF8dXh/b0GfEfH5harP72+PysfVo/P2teV5+/jOcDvrSrAwpUML4+cN/YV03P3VH5JyjThxYLXBckvblFhkAvzFMibUF9MGczWjayLEQ8X41fJOelr7RgYSQxPoUrmRVykhsWfM1ki5CGUP7Ck/Um5w/VF/f3FMrhETJ1vXAaSixRcTJ5sQUmD8gOYE7nDTK65VdRLEijQX3GBCoos9gtsIgnPGs0rhoVyDIDSIgZV+GlifAg4ZwzNz01jIGcDZUSkXrwEno+AKAjMLP7SEfi9EoKjKp+ffH+JDbBNzNXVP9qAAr1K+tDdF5SOzXeUZ+4wFVfzLSY7aBsl+YpIRQaFOSWDwkI8fmU+cPDElgeYOG6onPxWFpQdjqBrR2aNQjAT3pxaWmOsefG8OmFObTbhXXdBpSoDUdnsL3kyO5jBkLpzFLvUJjYiBFF2BUEO2MU+LpX7/gZkItzIEfXCoGEbFz7CJi6RqG+JbW6HrJIQkALUIeC13yjrQ2sLqHCjrMxEtGskcienq0PMeXldNCrCoo8e5xhxeYCS8nPSfR8Yf7EBdX+J0zapobdDGybSNmFrTmOJ3ZWe8BtE534EhTCs/75/DjEoGaplItmqfSk7o719ejCEpeHCOaPFohRtDuNuDUtyZLjkrmtnPZq7Rj65CXZ0PNHOMlshhr7pBpmR2mXH1VYhpa/Gg1rsRy50lfxJ8m0kY09i+2soI8+ui6/m/k0Tocb4HobLHwYF39At8Toln5QF/d3qQuOOpvb6aGhMyZ5fdqawyI759g5PlmiLPMlR5wLhvVYfqf133m1GYb5g7WGtNWyNFGG5uNCy+TZsrcNMuH8D0a+Y7jVvUf2O4fvAG5nhLmXzWmogCxDkNyboAUvMH8TtPddGSuUkX+gHov857thztxxVD5GEj19p7dLiXfgYnYL8JPe0Ltcet6VbZpgMkVGKm/Z0xu51Uv+pqGbP2ldD1qUkv91AVty+7kBcH1bPBps2c+aZ91rtjd1ZXkDZ9zG01XHqvL56UTOLZOVb6Hs+BpWvrtI/sQV/jVkBE8awr9GZvGkVfAtZyhPGfjvmc43qdalV98b76y9x1X4nJ3tNgENVfCIWC69vtm9l/oaUFtyHcHF7H/Sj7+e0YwGlPWk4HxgxY1v2u2rxknyMV+7raWYPJs8+z8FFeia")))
NEXT_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNrNVE1v2zAMvedXCFqAtlscb70tQLGtXTEUGFYgHXqJM0OxmUSJLCmS7DYr8t9HubaXOOnHYYfdLPLp6fGR9EOHEFqAsVxJOiCn/Q89H1FmYvH4gN94Wq1WAUYwQJdczqxRahlWwc9xfD08j28vhzdX1z/imOKVTcmxUNscOp8Ibudxyux8ophJ4wUzTRoBcA9J7pSP1Q+GC1Yw2qsR1oH2jKMqgKFkDslS5a4BkYZwW3nIpXVMiLgkHOxgEFVnpUoBs87k0NtF+Ez81yZ6ekq3AJvme/OEDpPL/Wcly/xz9BYMn66JmwPhWZY7NhFADAhgFkj9aEtQorKMydRfr5C1vLOIdh8uroYX3y/jn1++vSk2EY3k1CjpQKYNqnvsayKBJhEyrHJu4PioH2qWLNkM+gur5NFJv0JH9CSSGStAbt3PCkmCcxKsyByEHkDBRM4cUn6Fe23AljBt1AISVxNhcqpMAjcuxa4hqQPrUEG3rS+i5MzHW8X5UuorO3qewf/DTp3nXKRlo3BgHE8IVqqMe6Y5iTLgPSUgfVsjqaXOSDVwJAiw7t8gA6GS5ZQ3ebfWUI52dRZcujqF1VefE69m8CjhFWU2y4CN88sXp6CFWtf3xp365uP63imznAp1d2CHK5O3t7da9tHza1iuEdM6LqXv+4wWOGzcXgJTE8MkWnIo5zd4JtFob3jYfxvSFmDTa7M5NnuCSUmx3ilkKxcWB8hr5w653z69PH4v/SabCcM1uHcHhJZel6sRGJhx68w6SAykIB1nwu6qH/f+V/9p+KsYvQ8+jt9FUb/1cRwML5rDyafu/nuvacD+2Hc2nT/snNGl")))
@@ -89,8 +89,8 @@ def load_manifest(path, setup_version):
allowed += ("java_version",)
mapping(data, allowed, "manifest")
version = data.get("orb_version")
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+\.\d+|dev:[A-Za-z0-9_-]+)", version):
raise ManifestError("orb_version must be an explicit stable or development version")
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+(?:\.\d+)?|dev:[A-Za-z0-9_-]+)", version):
raise ManifestError("orb_version must be a quoted minor track, exact stable pin, or development version")
if version != setup_version:
raise ManifestError("Manifest orb_version must match the setup job orb_version")
default_branch = data.get("default_branch")
@@ -112,7 +112,7 @@ def load_manifest(path, setup_version):
if profile in ("qbit", "core"):
publish_keys += ("asciidoc",)
if profile == "qbit":
publish_keys += ("hotfix",)
publish_keys += ("hotfix", "feature")
if profile not in ("qbit", "cli", "all", "app-starter", "voyage", "frontend-next"):
publish_keys += ("feature", "hotfix")
publishing = mapping(data.get("publish", {}), publish_keys, "publish")
@@ -141,7 +141,7 @@ def load_manifest(path, setup_version):
"docs_only": boolean(data, "docs_only", False),
"publish_context": publish_context,
"publish": {key: boolean(publishing, key, key != "asciidoc" and profile not in ("cli", "all", "repository-policy")
and not (profile == "qbit" and key == "hotfix"))
and not (profile == "qbit" and key in ("hotfix", "feature")))
for key in publish_keys}}
@@ -191,7 +191,7 @@ def proven_ci_maintenance(branch, tag):
if not changed:
return False
allowed = {b".qqq-ci.yml", b".circleci/config.yml", b".github/workflows/codeql.yml",
b".github/workflows/release-rc.yml"}
b".github/workflows/release-rc.yml", b".github/workflows/schema-drift.yml"}
return all(path in allowed for path in changed.rstrip(b"\0").split(b"\0"))
@@ -215,11 +215,17 @@ def event_kind(branch, tag, release_on_main=False, release_tag_policy="stable"):
def qbit_template(config, branch, tag):
"""Profile-owned template; add future profiles as reviewed renderers, not raw YAML."""
kind = event_kind(branch, tag, config.get("release_on_main", False), config.get("release_tag_policy", "stable"))
if tag.startswith("publish") and config["publish"].get("feature", False):
tagged = git("rev-parse", "--verify", f"refs/tags/{tag}^{{commit}}")
if not tagged and git("fetch", "--no-tags", "origin", f"refs/tags/{tag}:refs/tags/{tag}") is not None:
tagged = git("rev-parse", "--verify", f"refs/tags/{tag}^{{commit}}")
if tagged and tagged == git("rev-parse", "HEAD"):
kind = "feature"
if not tag and re.fullmatch(r"hotfix/.+", branch) and config["publish"].get("hotfix", False):
kind = "hotfix"
if kind != "test" and not config["publish"][kind]:
kind = "test"
maintenance = kind != "test" and proven_ci_maintenance(branch, tag)
maintenance = proven_ci_maintenance(branch, tag)
if maintenance:
kind = "test"
workflow = "test_only" if kind == "test" else "publish_" + kind
@@ -336,7 +342,10 @@ def frontend_template(config, branch, tag):
kind = "release"
if kind != "test" and not config["publish"][kind]:
kind = "test"
if kind == "test" and proven_docs_only(config, branch, tag):
maintenance = proven_ci_maintenance(branch, tag)
if maintenance:
kind = "test"
if kind == "test" and not maintenance and proven_docs_only(config, branch, tag):
result = qbit_template(config, branch, tag)
result["workflows"]["test_only"]["jobs"][0]["docs_only"]["name"] = (
"qqq-orb/mvn_frontend_test_only" if material else "qqq-orb/node_test_only")
+3 -1
View File
@@ -649,7 +649,9 @@
"qqq-orb/mvn_publish": {
"requires": [
"sample_bootstrap_source",
"snapshot_esb_broker_conformance"
"snapshot_esb_broker_conformance",
"snapshot_sample_coverage_gate",
"qqq-orb/static_analysis"
],
"pre-steps": [
"configure-package-install"
+11 -5
View File
@@ -59,7 +59,7 @@ def load_yaml(raw):
raise StandardError("Invalid YAML declaration") from error
def canonical_caller(profile, version, explicit_manifest=False, release_tag_policy=None):
def canonical_caller(profile, version, explicit_manifest=False, release_tag_policy=None, publish_feature=False):
if not isinstance(profile, str) or profile not in PROFILE_PARAMETERS:
raise StandardError("Unsupported CI profile")
options = {
@@ -73,6 +73,9 @@ def canonical_caller(profile, version, explicit_manifest=False, release_tag_poli
if release_tag_policy not in tag_filters:
raise StandardError("Unsupported QBit release tag policy")
options["filters"]["tags"] = tag_filters[release_tag_policy]
if publish_feature:
options["filters"]["tags"] = {"only": ([tag_filters[release_tag_policy]["only"]]
if release_tag_policy != "none" else []) + ["/publish.*/"]}
if explicit_manifest:
options["manifest"] = ".qqq-ci.yml"
parameters = PROFILE_PARAMETERS[profile]
@@ -106,12 +109,15 @@ def validate(config_raw, manifest_raw, setup_version):
if not isinstance(manifest, dict):
raise StandardError("Manifest must be a mapping")
version = manifest.get("orb_version")
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+\.\d+|dev:[A-Za-z0-9_-]+)", version):
raise StandardError("Manifest must pin an explicit orb version")
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+(?:\.\d+)?|dev:[A-Za-z0-9_-]+)", version):
raise StandardError("Manifest must select a quoted minor track, exact stable pin, or development version")
if version != setup_version:
raise StandardError("Manifest and running setup orb versions differ")
expected = canonical_caller(manifest.get("profile"), version, release_tag_policy=manifest.get("release_tag_policy"))
explicit = canonical_caller(manifest.get("profile"), version, explicit_manifest=True, release_tag_policy=manifest.get("release_tag_policy"))
publishing = manifest.get("publish", {})
if not isinstance(publishing, dict) or type(publishing.get("feature", False)) is not bool:
raise StandardError("Feature publication must be an explicit boolean")
expected = canonical_caller(manifest.get("profile"), version, release_tag_policy=manifest.get("release_tag_policy"), publish_feature=publishing.get("feature", False))
explicit = canonical_caller(manifest.get("profile"), version, explicit_manifest=True, release_tag_policy=manifest.get("release_tag_policy"), publish_feature=publishing.get("feature", False))
if not exact(config, expected) and not exact(config, explicit):
raise StandardError("Consumer config must be the canonical setup caller with only its profile's declared and forwarded parameters")
+90
View File
@@ -0,0 +1,90 @@
version: 2.1
orbs:
qqq-orb: kingsrook/qqq-orb@0.6.8
commands:
configure-package-install:
steps:
- run:
name: Keep package installation unattended
command: |
# These build machines are discarded after the job.
sudo install -d /etc/needrestart/conf.d
printf '%s\n' '$nrconf{restart} = "l";' | sudo tee /etc/needrestart/conf.d/99-qqq-ci.conf >/dev/null
workflows:
test_only:
jobs:
- qqq-orb/mvn_test_only:
pre-steps:
- configure-package-install
context: [qqq-maven-registry-credentials]
filters:
branches:
ignore: /(develop|main|release/.*|hotfix/.*|integration.*)/
tags:
only: []
publish_snapshot:
jobs:
- qqq-orb/mvn_publish:
pre-steps:
- configure-package-install
context: [qqq-maven-registry-credentials]
branch_type: snapshot
filters:
branches:
only:
- develop
publish_feature:
jobs:
- qqq-orb/mvn_publish:
pre-steps:
- configure-package-install
context: [qqq-maven-registry-credentials]
branch_type: feature
filters:
branches:
only:
- feature/.*
tags:
only: [/publish.*/]
publish_release_candidate:
jobs:
- qqq-orb/mvn_publish:
pre-steps:
- configure-package-install
context: [qqq-maven-registry-credentials]
branch_type: release_candidate
filters:
branches:
only:
- /release\/.*/
publish_release:
jobs:
- qqq-orb/mvn_publish:
pre-steps:
- configure-package-install
context: [qqq-maven-registry-credentials]
branch_type: release
filters:
branches:
ignore: /.*/
tags:
only: [/^v\d+\.\d+\.\d+$/]
publish_hotfix_release:
jobs:
- qqq-orb/mvn_publish:
pre-steps:
- configure-package-install
context: [qqq-maven-registry-credentials]
branch_type: hotfix
filters:
branches:
only:
- /hotfix\/.*/
+72 -1
View File
@@ -1,4 +1,6 @@
import importlib.util
import json
import io
import pathlib
import unittest
from unittest.mock import patch
@@ -25,6 +27,54 @@ class Snapshot:
return self.required
class InventoryTests(unittest.TestCase):
def repo(self, name, archived=False):
return {'name': name, 'full_name': 'QRun-IO/' + name,
'archived': archived, 'default_branch': 'main'}
def test_all_pages_include_new_private_and_archived_repositories(self):
new = dict(self.repo('new-product'), private=True)
pages = [[self.repo('qbit-existing')], [new, self.repo('retired', True)]]
with patch.object(audit, 'command', return_value=json.dumps(pages)) as command:
entries = audit.discover_inventory('QRun-IO')
self.assertEqual([e['name'] for e in entries], ['new-product', 'qbit-existing', 'retired'])
self.assertTrue(entries[-1]['isArchived'])
self.assertEqual(command.call_args.args[0], ['gh', 'api', '--method', 'GET',
'--paginate', '--slurp', 'orgs/QRun-IO/repos?type=all&per_page=100'])
def test_empty_duplicate_foreign_and_malformed_inventories_fail_closed(self):
repo = self.repo('qbit-example')
for pages in ([], [[]], {}, [repo], [[repo], [repo]],
[[dict(repo, full_name='Other/qbit-example')]],
[[dict(repo, archived='false')]], [[dict(repo, default_branch=None)]]):
with self.subTest(pages=pages), patch.object(audit, 'command', return_value=json.dumps(pages)):
with self.assertRaises(audit.ReadError):
audit.discover_inventory('QRun-IO')
with patch.object(audit, 'command') as command:
with self.assertRaises(audit.ReadError):
audit.discover_inventory('QRun-IO/../../other')
command.assert_not_called()
def test_live_cli_count_is_dynamic_but_explicit_expectation_is_enforced(self):
for expected, status in ((None, 0), ('1', 0), ('68', 1)):
args = ['audit', '--organization', 'QRun-IO', '--live', '--orb-version', '0.8']
if expected:
args += ['--expected-active', expected]
report = {'active_count': 1, 'unique_repository_count': 1,
'inventory_count': 1, 'results': [{'status': 'pass'}]}
output = io.StringIO()
with patch.object(audit.sys, 'argv', args), patch.object(audit.sys, 'stdout', output), \
patch.object(audit, 'discover_inventory', return_value=[ENTRY]), \
patch.object(audit, 'run', return_value=report):
self.assertEqual(audit.main(), status)
self.assertEqual(json.loads(output.getvalue())['inventory_source'], 'organization-api')
def test_discovery_api_failure_never_falls_back_to_old_inventory(self):
with patch.object(audit, 'command', side_effect=audit.ReadError('unavailable')):
with self.assertRaises(audit.ReadError):
audit.discover_inventory('QRun-IO')
class AuditTests(unittest.TestCase):
def test_all_scanner_and_release_requirements_follow_branch_profile(self):
entry = dict(ENTRY, name='qqq-all', nameWithOwner='QRun-IO/qqq-all')
@@ -387,10 +437,31 @@ class AuditTests(unittest.TestCase):
audit.audit(other, Snapshot({}, checks=[]), VERSION, SHA, policy)
def test_invalid_managed_branches_cannot_bypass_policy(self):
for managed in [[], ['main', 'main'], ['feature'], 'main', None, [True]]:
for managed in [[], ['main', 'main'], ['feature'], 'main', None, [True],
['release/*'], ['release/1.2/extra'], ['hotfix/1..2'],
['release/1.2', 'release/1.2']]:
with self.subTest(managed=managed), self.assertRaises(audit.ReadError):
self.native(policy={'managed_branches': managed, 'required_checks': {}})
def test_release_and_hotfix_enforcement_requires_actual_checks(self):
files = {'.github/workflows/build.yml': audit.qctl.generate(SHA)}
for branch in ['release/0.1', 'release/4.1.0', 'hotfix/4.0.1']:
with self.subTest(branch=branch):
snapshot = Snapshot(files)
snapshot.branch = branch
policy = {'managed_branches': [branch],
'required_checks': {branch: ['native / test']}}
with patch.object(snapshot, 'checks', wraps=snapshot.checks) as read:
report = audit.audit(ENTRY, snapshot, VERSION, SHA, policy)
self.assertEqual('pass', report['status'])
read.assert_called_once_with(branch)
snapshot.required = []
report = audit.audit(ENTRY, snapshot, VERSION, SHA, policy)
self.assertEqual('missing', report['required_checks'][branch]['state'])
self.assertEqual('findings', report['status'])
policy['required_checks'] = {}
self.assertEqual('unknown', audit.audit(ENTRY, snapshot, VERSION, SHA, policy)['status'])
def test_gh_is_explicit_get_and_never_changes_settings(self):
with patch.object(audit, 'command', return_value='{}') as command:
audit.gh('repos/QRun-IO/example')
+69
View File
@@ -0,0 +1,69 @@
#!/bin/bash
# Exercise real browser scripts without sudo, package installation or network.
set -euo pipefail
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
scratch=$(mktemp -d)
trap 'rm -rf "$scratch"' EXIT
mkdir "$scratch/bin"
export INSTALL_LOG="$scratch/install.log"
cat > "$scratch/bin/sudo" <<'STUB'
#!/bin/bash
printf '%s\n' "$*" >> "$INSTALL_LOG"
if [[ "$*" == *'apt-get update'* ]]; then
[[ "$*" == *'Dir::Etc::sourceparts=-'* ]] || { echo 'unrelated image PPA: 503' >&2; exit 100; }
[[ "$*" == *'APT::Get::List-Cleanup=0'* ]] || exit 91
[[ ${REQUIRED_REPO_FAIL:-0} == 0 ]] || exit 100
fi
if [[ "$1" == tee ]]; then cat >> "$INSTALL_LOG"; fi
if [[ "$1" == gpg ]]; then cat >/dev/null; fi
STUB
chmod +x "$scratch/bin/sudo"
export PATH="$scratch/bin:$PATH"
bash "$root/src/scripts/frontend_test_dependencies.sh"
echo 'PASS: browser dependencies ignore unavailable image PPAs'
# Model Chrome presence without depending on the host's installed browsers.
export CHROME_MARKER="$scratch/chrome-installed"
export BASH_ENV="$scratch/shell-env"
cat > "$BASH_ENV" <<'STUB'
command() {
if [[ "$*" == '-v google-chrome' ]]; then [[ -f "$CHROME_MARKER" ]]; else builtin command "$@"; fi
}
google-chrome() { [[ -f "$CHROME_MARKER" ]] && echo 'Google Chrome test'; }
STUB
cat > "$scratch/bin/curl" <<'STUB'
#!/bin/bash
printf '%s\n' "$*" >> "$INSTALL_LOG"
[[ ${KEY_FAIL:-0} == 0 ]] || exit 22
printf 'signing-key\n'
STUB
cat > "$scratch/bin/dpkg" <<'STUB'
#!/bin/bash
echo amd64
STUB
# Successful installation makes Chrome available to the final version check.
# shellcheck disable=SC2016
sed -i '/if \[\[ "$1" == tee/i if [[ "$*" == *"apt-get install -y google-chrome-stable"* ]]; then touch "$CHROME_MARKER"; fi' "$scratch/bin/sudo"
chmod +x "$scratch/bin/"*
: > "$INSTALL_LOG"
bash "$root/src/scripts/install_chrome.sh" >/dev/null
grep -q 'Dir::Etc::sourcelist=/etc/apt/sources.list.d/qqq-google-chrome.list' "$INSTALL_LOG"
grep -q 'gpg --batch --yes --dearmor -o /usr/share/keyrings/qqq-google-chrome.gpg' "$INSTALL_LOG"
grep -q 'signed-by=/usr/share/keyrings/qqq-google-chrome.gpg] https://dl.google.com/linux/chrome/deb/' "$INSTALL_LOG"
grep -q 'apt-get install -y google-chrome-stable' "$INSTALL_LOG"
: > "$INSTALL_LOG"
bash "$root/src/scripts/install_chrome.sh" >/dev/null
[[ ! -s "$INSTALL_LOG" ]] # Existing Chrome needs no network/package work.
rm "$CHROME_MARKER"
for script in install_chrome frontend_test_dependencies; do
: > "$INSTALL_LOG"
if REQUIRED_REPO_FAIL=1 bash "$root/src/scripts/$script.sh" >/dev/null 2>&1; then
echo "Required repository outage accepted: $script" >&2; exit 1
fi
if grep -q 'apt-get install' "$INSTALL_LOG"; then exit 1; fi
done
: > "$INSTALL_LOG"
if KEY_FAIL=1 bash "$root/src/scripts/install_chrome.sh" >/dev/null 2>&1; then
echo 'Signing key failure accepted' >&2; exit 1
fi
if grep -q 'apt-get' "$INSTALL_LOG"; then exit 1; fi
echo 'PASS: Chrome ignores unrelated PPAs; required repositories and signing key fail closed; existing browser reused'
+51
View File
@@ -34,12 +34,63 @@ class MaintenanceTests(unittest.TestCase):
self.assertEqual(job['filters']['branches'], {'ignore': 'automation/qbit-rc-pending'})
self.assertNotIn('branch_type', job)
def test_nonpublishing_branches_verify_maintenance_but_keep_application_policy(self):
manifest = {'profile': 'qbit', 'orb_version': '0.8', 'default_branch': 'main',
'release_on_main': False, 'test_branch_policy': 'feature-no-hotfix',
'publish': {'snapshot': False, 'release_candidate': False, 'hotfix': False}}
with mock.patch.object(Path, 'read_text', return_value=yaml.safe_dump(manifest)):
config = pipeline.load_manifest('unused', '0.8')
maintenance_paths = self.paths
with mock.patch.object(pipeline, 'git', side_effect=self.git_result):
for branch in ('main', 'develop', 'release/0.1', 'hotfix/1.2.3'):
for maintenance in (True, False):
with self.subTest(branch=branch, maintenance=maintenance):
self.paths = maintenance_paths if maintenance else b'src/Main.java\0'
result = pipeline.qbit_template(config, branch, '')
job = result['workflows']['test_only']['jobs'][0]['qqq-orb/mvn_test_only']
ignored = job['filters']['branches']['ignore']
if maintenance:
self.assertEqual(ignored, 'automation/qbit-rc-pending')
else:
self.assertEqual(ignored, ['automation/qbit-rc-pending',
'/(develop|main|release/.*|hotfix/.*|integration.*)/'])
self.assertNotIn('branch_type', job)
def test_frontend_maintenance_verifies_without_publishing(self):
for profile, prefix in (('frontend-core', 'node_'), ('frontend-material', 'mvn_frontend_')):
manifest = {'profile': profile, 'orb_version': '0.8', 'default_branch': 'develop'}
with mock.patch.object(Path, 'read_text', return_value=yaml.safe_dump(manifest)):
config = pipeline.load_manifest('unused', '0.8')
for branch in ('main', 'develop', 'release/1.0', 'hotfix/1.0.1'):
with self.subTest(profile=profile, branch=branch), mock.patch.object(pipeline, 'git', side_effect=self.git_result):
result = pipeline.frontend_template(config, branch, '')
job = result['workflows']['test_only']['jobs'][0]['qqq-orb/' + prefix + 'test_only']
self.assertNotIn('branch_type', job)
self.assertEqual(job['filters']['branches'], {'ignore': 'automation/qbit-rc-pending'})
self.paths = b'package.json\0'
with mock.patch.object(pipeline, 'git', side_effect=self.git_result):
result = pipeline.frontend_template(config, 'develop', '')
self.assertIn('qqq-orb/' + prefix + 'publish', result['workflows']['publish_snapshot']['jobs'][0])
self.paths = b'.qqq-ci.yml\0'
with mock.patch.object(pipeline, 'git', return_value=b'verified-head\n'):
result = pipeline.frontend_template(config, 'main', 'v1.2.3')
job = result['workflows']['publish_release']['jobs'][0]['qqq-orb/' + prefix + 'publish']
self.assertEqual(job['branch_type'], 'release')
def test_application_changes_and_unknown_files_publish_normally(self):
for paths in (b'pom.xml\0', b'.qqq-ci.yml\0src/Main.java\0', b'.github/workflows/custom-release.yml\0', b'', None):
self.paths = paths
with mock.patch.object(pipeline, 'git', side_effect=self.git_result):
self.assertFalse(pipeline.proven_ci_maintenance('release/0.1', ''))
def test_schema_caller_migration_is_maintenance_but_schema_inputs_are_not(self):
self.paths = b'.github/workflows/schema-drift.yml\0'
with mock.patch.object(pipeline, 'git', side_effect=self.git_result):
self.assertTrue(pipeline.proven_ci_maintenance('release/0.1', ''))
for path in (b'pom.xml\0', b'src/test/resources/schema.yml\0', b'src/test/java/SchemaDriftTest.java\0'):
self.paths = b'.github/workflows/schema-drift.yml\0' + path
self.assertFalse(pipeline.proven_ci_maintenance('release/0.1', ''))
def test_merge_root_and_unavailable_history_do_not_suppress_release(self):
for parents in (None, b'a'*40+b'\n', self.parents.rstrip()+b' '+b'c'*40+b'\n'):
self.parents = parents
@@ -0,0 +1,69 @@
#!/usr/bin/env python3
"""Exercise the real version script and Maven install; never publish remotely."""
import os
from pathlib import Path
import shutil
import subprocess
import tempfile
import xml.etree.ElementTree as ET
ROOT = Path(__file__).resolve().parents[1]
NS = {'m': 'http://maven.apache.org/POM/4.0.0'}
BASH = os.environ.get('TEST_BASH') or shutil.which('bash')
MAVEN = shutil.which('mvn') or shutil.which('mvn.cmd')
def run(args, directory, env=None):
subprocess.run(args, cwd=directory, env=env, check=True, timeout=300)
def project(body):
return '<project xmlns="http://maven.apache.org/POM/4.0.0">\n<modelVersion>4.0.0</modelVersion>\n' + body + '\n</project>\n'
def main():
assert BASH and MAVEN, 'Real Bash, Java and Maven are required'
with tempfile.TemporaryDirectory(prefix='qqq-version-') as temporary:
base = Path(temporary)
for name, version, revision, branch, expected, reactor in (
('literal', '0.4.0-SNAPSHOT', '0.4.0-RC.1', 'release/0.4', '0.4.0-RC.2', False),
('friendly', '${revision}', '0.4.0-RC.1', 'release/0.4', '0.4.0-RC.2', False),
('reactor', '0.4.0-SNAPSHOT', '0.4.0-RC.1', 'release/0.4', '0.4.0-RC.2', True),
('unchanged', '0.3.0-SNAPSHOT', '0.4.0-SNAPSHOT', 'develop', '0.4.0-SNAPSHOT', False),
):
root = base / name
root.mkdir()
modules = '<modules><module>child</module></modules>' if reactor else ''
(root / 'pom.xml').write_text(project(f'''<groupId>example.version</groupId>
<artifactId>{name}</artifactId><version>{version}</version><packaging>pom</packaging>
<properties>
<revision>{revision}</revision>
</properties>
{modules}'''), encoding='utf-8')
if reactor:
(root / 'child').mkdir()
(root / 'child/pom.xml').write_text(project(f'''<parent><groupId>example.version</groupId>
<artifactId>{name}</artifactId><version>{version}</version></parent>
<artifactId>child</artifactId><packaging>pom</packaging>'''), encoding='utf-8')
run(['git', 'init', '-q'], root)
env = dict(os.environ, CIRCLE_BRANCH=branch, CIRCLE_TAG='')
run([BASH, str(ROOT / 'src/scripts/calculate_version.sh')], root, env)
if name == 'friendly':
assert ET.parse(root / 'pom.xml').findtext('m:version', namespaces=NS) == '${revision}'
# Install to a temporary repository so the actual artifact coordinate
# proves the fix, rather than merely checking the revision property.
repository = base / 'repository'
run([MAVEN, '-B', '-ntp', '-Dmaven.repo.local=' + str(repository),
'-Dflatten.mode=resolveCiFriendliesOnly', '-DupdatePomFile=true',
'org.codehaus.mojo:flatten-maven-plugin:1.7.3:flatten', 'install'], root)
for artifact in ([name, 'child'] if reactor else [name]):
installed = repository / 'example/version' / artifact / expected / f'{artifact}-{expected}.pom'
assert installed.exists(), installed
pom = ET.parse(installed)
actual = pom.findtext('m:version', namespaces=NS) or pom.findtext('m:parent/m:version', namespaces=NS)
assert actual == expected, (artifact, actual, expected)
print(f'PASS: {name} installed at {expected}', flush=True)
if __name__ == '__main__':
main()
+92
View File
@@ -0,0 +1,92 @@
"""Shared enforcement only adds proven gates and preserves unrelated policy."""
import copy
import importlib.util
from pathlib import Path
import unittest
spec = importlib.util.spec_from_file_location('enforcement', Path(__file__).resolve().parents[1] / 'tools/enforce_orb_checks.py')
enforcement = importlib.util.module_from_spec(spec)
spec.loader.exec_module(enforcement)
def runs():
return [{'id': index, 'name': name, 'app': {'id': 18001}, 'conclusion': 'success'}
for index, name in enumerate(('setup', 'verify'), 1)]
class EnforcementTests(unittest.TestCase):
def test_preserves_security_gate_and_binds_orb_checks(self):
before = {'required_status_checks': {'strict': False, 'contexts': ['security', 'setup'],
'checks': [{'context': 'security', 'app_id': 15368}, {'context': 'setup', 'app_id': -1}]}}
original = copy.deepcopy(before)
result = enforcement.required_checks(before, runs())
self.assertEqual(original, before)
self.assertEqual(result, {'strict': True, 'checks': [{'context': 'security', 'app_id': 15368}, *enforcement.DESIRED]})
def test_wrong_app_failed_latest_and_missing_checks_reject(self):
cases = [[], [runs()[0]], [dict(run, app={'id': 999}) for run in runs()],
runs() + [dict(runs()[1], id=3, conclusion='failure')]]
for candidate in cases:
with self.subTest(candidate=candidate), self.assertRaises(ValueError):
enforcement.required_checks(None, candidate)
with self.assertRaises(ValueError):
enforcement.required_checks({'required_status_checks': {'checks': [{'context': 'verify', 'app_id': 999}]}}, runs())
def test_existing_unrestricted_contexts_are_retained(self):
result = enforcement.required_checks({'required_status_checks': {'contexts': ['legacy-build']}}, runs())
self.assertIn({'context': 'legacy-build', 'app_id': -1}, result['checks'])
def fixture(self, before):
state = {'before': copy.deepcopy(before), 'head': 'a' * 40, 'writes': []}
def request(path, method='GET', payload=None):
if method != 'GET':
state['writes'].append((method, path, payload))
if method == 'PUT':
state['before'] = {**payload, 'enforce_admins': {'enabled': True}}
elif method == 'PATCH':
state['before']['required_status_checks'] = copy.deepcopy(payload)
elif method == 'POST':
state['before']['enforce_admins'] = {'enabled': True}
return None
if '/rules/' in path:
return []
if path.endswith('/protection'):
return copy.deepcopy(state['before'])
return {'commit': {'sha': state['head']}}
plan = {'repository': 'QRun-IO/example', 'branch': 'develop', 'head': 'a' * 40,
'before': copy.deepcopy(before), 'required_status_checks': enforcement.required_checks(before, runs()), 'changed': True}
return plan, request, state
def test_narrow_updates_preserve_reviews_signatures_and_restrictions(self):
before = {'required_status_checks': {'strict': False, 'checks': []}, 'enforce_admins': {'enabled': False},
'required_signatures': {'enabled': True}, 'required_linear_history': {'enabled': True},
'required_pull_request_reviews': {'required_approving_review_count': 2},
'restrictions': {'users': [{'login': 'owner'}]}}
plan, request, state = self.fixture(before)
enforcement.enforce(plan, request)
self.assertEqual(['PATCH', 'POST'], [write[0] for write in state['writes']])
for key in ('required_signatures', 'required_linear_history', 'required_pull_request_reviews', 'restrictions'):
self.assertEqual(before[key], state['before'][key])
def test_new_protection_and_idempotent_noop(self):
plan, request, state = self.fixture(None)
enforcement.enforce(plan, request)
self.assertEqual('PUT', state['writes'][0][0])
self.assertIsNone(state['before']['required_pull_request_reviews'])
plan['before'] = copy.deepcopy(state['before'])
plan['changed'] = False
state['writes'] = []
enforcement.enforce(plan, request)
self.assertEqual([], state['writes'])
def test_head_or_protection_drift_never_mutates(self):
for field, value in [('head', 'b' * 40), ('before', {'unexpected': True})]:
plan, request, state = self.fixture(None)
state[field] = value
with self.assertRaises(ValueError):
enforcement.enforce(plan, request)
self.assertEqual([], state['writes'])
if __name__ == '__main__':
unittest.main()
+37
View File
@@ -0,0 +1,37 @@
#!/bin/bash
# Exercise the actual test runner with no server, browser, network or Maven work.
set -euo pipefail
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
scratch=$(mktemp -d)
trap 'rm -rf "$scratch"' EXIT
mkdir "$scratch/bin"
export RUN_LOG="$scratch/calls"
cat > "$scratch/bin/npm" <<'STUB'
#!/bin/bash
printf 'npm %s\n' "$*" >> "$RUN_LOG"
if [[ "$*" == 'run build' && ${BUILD_FAIL:-0} == 1 ]]; then exit 12; fi
STUB
for tool in mvn dockerize; do
# shellcheck disable=SC2016
printf '#!/bin/bash\nprintf "%s %%s\\n" "$*" >> "$RUN_LOG"\n' "$tool" > "$scratch/bin/$tool"
done
chmod +x "$scratch/bin/"*
export PATH="$scratch/bin:$PATH"
cd "$scratch"
PARAM_BUILD_BEFORE_TEST=false bash "$root/src/scripts/frontend_test_run.sh" >/dev/null
if grep -qx 'npm run build' "$RUN_LOG"; then
echo 'FAIL: combined job rebuilt the already-built React app' >&2; exit 1
fi
grep -q '^mvn .* -B test$' "$RUN_LOG"
echo 'PASS: combined job retains tests without a duplicate build'
: > "$RUN_LOG"
bash "$root/src/scripts/frontend_test_run.sh" >/dev/null
grep -qx 'npm run build' "$RUN_LOG"
grep -q '^mvn .* -B test$' "$RUN_LOG"
: > "$RUN_LOG"
if BUILD_FAIL=1 bash "$root/src/scripts/frontend_test_run.sh" >/dev/null 2>&1; then
echo 'FAIL: required standalone build failure ignored' >&2; exit 1
fi
if grep -q '^mvn ' "$RUN_LOG"; then exit 1; fi
echo 'PASS: standalone command still builds and fails before tests when its build fails'
+75
View File
@@ -215,6 +215,14 @@ class PipelineTests(unittest.TestCase):
self.assertEqual(options["filters"]["branches"], {"ignore": "automation/qbit-rc-pending"})
if PACKED:
self.assertEqual(result, self.render(packed=True, CIRCLE_BRANCH=branch))
schema = self.repo / ".github/workflows/schema-drift.yml"
schema.write_text("# central schema drift caller migration\n", encoding="utf-8")
self.git("add", ".github/workflows/schema-drift.yml")
self.git("-c", "commit.gpgsign=false", "commit", "-qm", "ci: centralize schema drift")
result = self.render(CIRCLE_BRANCH="release/0.1")
self.assert_full(result)
if PACKED:
self.assertEqual(result, self.render(packed=True, CIRCLE_BRANCH="release/0.1"))
(self.repo / "App.java").write_text("application change\n", encoding="utf-8")
self.git("add", "App.java")
self.git("-c", "commit.gpgsign=false", "commit", "-qm", "ci: misleading title")
@@ -233,6 +241,15 @@ class PipelineTests(unittest.TestCase):
self.assertTrue(options["check_middleware_api_versions"])
self.assertEqual(options["java_version"], "17")
def test_minor_track_reaches_continuation_without_changing_jobs(self):
expected = self.render()
expected['orbs']['qqq-orb'] = 'kingsrook/qqq-orb@0.7'
self.write_manifest(orb_version='0.7')
self.assertEqual(expected, self.render(QQQ_CI_ORB_VERSION='0.7'))
if PACKED:
self.assertEqual(expected, self.render(packed=True, QQQ_CI_ORB_VERSION='0.7'))
self.render(ok=False, QQQ_CI_ORB_VERSION='0.8')
def test_branch_matrix(self):
for branch, workflow in [("develop", "publish_snapshot"), ("release/1.0", "publish_release_candidate"),
("main", "test_only"), ("hotfix/fix", "test_only"),
@@ -260,6 +277,19 @@ class PipelineTests(unittest.TestCase):
self.write_manifest(publish={"hotfix": "true"})
self.render(packed=packed, CIRCLE_BRANCH="hotfix/1.2.3", ok=False)
def test_qbit_feature_tags_source_and_packed_require_opt_in_and_head(self):
self.git("tag", "publish-test")
for packed in ([False, True] if PACKED else [False]):
self.write_manifest()
self.assert_full(self.render(packed=packed, CIRCLE_BRANCH="", CIRCLE_TAG="publish-test"))
self.write_manifest(publish={"feature": True})
self.assert_full(self.render(packed=packed, CIRCLE_BRANCH="", CIRCLE_TAG="publish-test"), "publish_feature")
self.assert_full(self.render(packed=packed, CIRCLE_BRANCH="feature/example"))
self.assert_full(self.render(packed=packed, CIRCLE_BRANCH="", CIRCLE_TAG="publish-missing"))
self.commit()
for packed in ([False, True] if PACKED else [False]):
self.assert_full(self.render(packed=packed, CIRCLE_BRANCH="", CIRCLE_TAG="publish-test"))
def test_repository_policy_source_and_packed_never_select_publication(self):
self.write_manifest(profile="repository-policy")
for packed in ([False, True] if PACKED else [False]):
@@ -399,9 +429,27 @@ class PipelineTests(unittest.TestCase):
self.change_docs()
self.assertIn("docs_only", self.render()["jobs"])
def test_core_snapshot_publisher_waits_for_all_acceptance_and_analysis(self):
self.write_manifest(profile="core")
result = self.render(CIRCLE_BRANCH="develop")
jobs = {options.get("name", kind): options
for entry in result["workflows"]["verify"]["jobs"]
for kind, options in entry.items()}
def ancestors(name, path=()):
self.assertNotIn(name, path, "Workflow dependencies must be acyclic")
required = jobs[name].get("requires", [])
return set(required).union(*(ancestors(parent, path + (name,)) for parent in required))
self.assertEqual(ancestors("qqq-orb/mvn_publish"), set(jobs) - {"qqq-orb/mvn_publish"})
if PACKED:
self.assertEqual(result, self.render(packed=True, CIRCLE_BRANCH="develop"))
def test_core_workflow_graphs_match_baseline(self):
self.write_manifest(profile="core")
baseline = yaml.safe_load((ROOT / "tests/fixtures/core_pipeline_baseline.yml").read_text(encoding="utf-8"))
# Snapshot publication must now wait for the existing sample and analysis gates.
for entry in baseline["workflows"]["publish_snapshot"]["jobs"]:
if "qqq-orb/mvn_publish" in entry:
entry["qqq-orb/mvn_publish"]["requires"] += ["snapshot_sample_coverage_gate", "qqq-orb/static_analysis"]
self.git("tag", "v1.0.0")
self.git("tag", "publish-fixture")
events = [("feature/test", "", "test_only"), ("develop", "", "publish_snapshot"),
@@ -575,6 +623,33 @@ class PipelineTests(unittest.TestCase):
if PACKED:
self.assertEqual(result, self.render(packed=True, CIRCLE_BRANCH="main"))
def test_frontend_ci_only_commits_verify_with_packed_parity(self):
for profile, job_name in (("frontend-core", "qqq-orb/node_test_only"),
("frontend-material", "qqq-orb/mvn_frontend_test_only")):
self.write_manifest(profile=profile)
self.git("add", ".qqq-ci.yml")
self.git("-c", "commit.gpgsign=false", "commit", "-qm", "ci: adopt managed orb")
for branch in ("main", "develop"):
with self.subTest(profile=profile, branch=branch):
result = self.render(CIRCLE_BRANCH=branch)
job = result["workflows"]["verify"]["jobs"][0][job_name]
self.assertNotIn("branch_type", job)
self.assertEqual(job["filters"]["branches"], {"ignore": "automation/qbit-rc-pending"})
if PACKED:
self.assertEqual(result, self.render(packed=True, CIRCLE_BRANCH=branch))
def test_nonpublishing_main_ci_change_runs_verification_with_packed_parity(self):
self.write_manifest(default_branch="main", release_on_main=False,
test_branch_policy="feature")
self.git("add", ".qqq-ci.yml")
self.git("-c", "commit.gpgsign=false", "commit", "-qm", "ci: use central pipeline")
result = self.render(CIRCLE_BRANCH="main")
job = result["workflows"]["verify"]["jobs"][0]["qqq-orb/mvn_test_only"]
self.assertEqual(job["filters"]["branches"], {"ignore": "automation/qbit-rc-pending"})
self.assertNotIn("branch_type", job)
if PACKED:
self.assertEqual(result, self.render(packed=True, CIRCLE_BRANCH="main"))
def test_qbit_publication_policy_packed_parity(self):
self.write_manifest(release_on_main=True, release_tag_policy="v-prefix",
test_branch_policy="feature")
+37
View File
@@ -17,6 +17,24 @@ if [[ "$*" == *apt-get* ]]; then
[[ "$1" == env && "$2" == DEBIAN_FRONTEND=noninteractive && "$3" == NEEDRESTART_MODE=l ]] || exit 90
[[ -f "$APT_CONFIG_CAPTURE" ]] && grep -qx 'Acquire::Retries "3";' "$APT_CONFIG_CAPTURE" || exit 91
[[ ${APT_FAIL:-0} == 0 ]] || exit "$APT_FAIL"
if [[ "$*" == *'apt-get update'* ]]; then
updates=$(grep -c 'apt-get update' "$INSTALL_LOG")
if [[ $updates -eq 1 ]]; then
# An unavailable image PPA must not participate in base-tool setup.
[[ "$*" == *'Dir::Etc::sourceparts=-'* ]] || exit 100
[[ "$*" == *'Dir::Etc::sourcelist=/etc/apt/sources.list'* || "$*" == *'Dir::Etc::sourcelist=/etc/apt/sources.list.d/ubuntu.sources'* ]] || exit 93
[[ "$*" == *'APT::Get::List-Cleanup=0'* ]] || exit 92
[[ ${UBUNTU_UPDATE_FAIL:-0} == 0 ]] || exit "$UBUNTU_UPDATE_FAIL"
fi
if [[ $updates -eq 2 ]]; then
# Model a PPA outage after the initial base-package installation. Only
# the new signed repository may be contacted; existing indexes survive.
[[ "$*" == *'Dir::Etc::sourcelist=/etc/apt/sources.list.d/adoptium.list'* ]] || exit 100
[[ "$*" == *'Dir::Etc::sourceparts=-'* ]] || exit 100
[[ "$*" == *'APT::Get::List-Cleanup=0'* ]] || exit 92
[[ ${TEMURIN_UPDATE_FAIL:-0} == 0 ]] || exit "$TEMURIN_UPDATE_FAIL"
fi
fi
fi
if [[ "$1" == tee && "$2" == /etc/apt/apt.conf.d/80-qqq-download-retries ]]; then
cat > "$APT_CONFIG_CAPTURE"; exit 0
@@ -42,6 +60,25 @@ rm "$APT_CONFIG_CAPTURE"
PARAM_INSTALL_NODE=true bash "$script_dir/install_java.sh" >/dev/null 2>&1
[[ $(grep -c 'apt-get' "$INSTALL_LOG") -eq 5 ]]
grep -q 'apt-get install -y -qq nodejs' "$INSTALL_LOG"
# An outage of the required repository must still stop before Java installation.
: > "$INSTALL_LOG"
if UBUNTU_UPDATE_FAIL=100 PARAM_INSTALL_NODE=false bash "$script_dir/install_java.sh" >/dev/null 2>&1; then
echo 'required Ubuntu repository failure accepted' >&2; exit 1
else
[[ $? -eq 100 ]]
fi
if grep -q 'apt-get install' "$INSTALL_LOG"; then
echo 'package installation continued after Ubuntu repository failure' >&2; exit 1
fi
: > "$INSTALL_LOG"
if TEMURIN_UPDATE_FAIL=100 PARAM_INSTALL_NODE=false bash "$script_dir/install_java.sh" >/dev/null 2>&1; then
echo 'required Temurin repository failure accepted' >&2; exit 1
else
[[ $? -eq 100 ]]
fi
if grep -q 'apt-get install -y -qq temurin-' "$INSTALL_LOG"; then
echo 'Java installation continued after repository failure' >&2; exit 1
fi
# Execute the actual generated selector. Arguments containing spaces stay intact.
export SELECTOR_LOG="$scratch/selector.log"
cat > "$scratch/bin/mvn" <<'NATIVE'
+73
View File
@@ -0,0 +1,73 @@
"""Preserve Workflows' audited publish-tag contract without broadening branches."""
import unittest
from unittest import mock
from pathlib import Path
import yaml
from test_qbit_hotfix_profile import load, migration, pipeline, selects, ROOT
standard = load('standard', 'src/scripts/validate_standard.py')
class FeaturePublicationTests(unittest.TestCase):
def config(self, manifest):
with mock.patch.object(Path, 'read_text', return_value=yaml.safe_dump(manifest)):
return pipeline.load_manifest('unused', '0.7.16')
def test_audited_event_selection_matches_legacy(self):
legacy = yaml.safe_load((ROOT / 'tests/fixtures/qbit-feature-publication.yml').read_text())
manifest = migration.infer_manifest(legacy, '0.7.16', 'main')
self.assertTrue(manifest['publish']['feature'])
caller = migration.canonical_caller('0.7.16', manifest['release_tag_policy'], True)
standard.validate(yaml.safe_dump(caller), yaml.safe_dump(manifest), '0.7.16')
setup = caller['workflows']['setup']['jobs'][0]['qqq-orb/generate_pipeline']
cases = [(branch, '') for branch in ['main', 'develop', 'feature/example', 'release/0.3.0', 'hotfix/0.3.1', 'integration-test']]
cases += [('', tag) for tag in ['publish', 'publish-example', 'v1.2.3', 'v1.2.3-RC.1', 'other']]
for branch, tag in cases:
with self.subTest(branch=branch, tag=tag):
expected = sorted((job, args.get('branch_type')) for workflow in legacy['workflows'].values()
for entry in workflow['jobs'] for job, args in entry.items()
if selects(args.get('filters', {}), branch, tag))
with mock.patch.object(pipeline, 'git', return_value=b'verified-head\n'):
rendered = pipeline.qbit_template(self.config(manifest), branch, tag)
actual = sorted((job, args.get('branch_type')) for workflow in rendered['workflows'].values()
for entry in workflow['jobs'] for job, args in entry.items()
if selects(args.get('filters', {}), branch, tag))
if not selects(setup['filters'], branch, tag):
actual = []
self.assertEqual(expected, actual)
def test_feature_is_opt_in_and_requires_exact_tag_head(self):
manifest = {'profile': 'qbit', 'orb_version': '0.7.16', 'default_branch': 'main'}
self.assertFalse(self.config(manifest)['publish']['feature'])
manifest['publish'] = {'feature': True}
for tagged in [None, b'wrong-head\n']:
def git(*args):
return b'head\n' if args == ('rev-parse', 'HEAD') else tagged
with self.subTest(tagged=tagged), mock.patch.object(pipeline, 'git', side_effect=git):
rendered = pipeline.qbit_template(self.config(manifest), '', 'publish-test')
self.assertIn('qqq-orb/mvn_test_only', rendered['workflows']['test_only']['jobs'][0])
def test_feature_tag_recovery_checks_fetched_commit(self):
manifest = {'profile': 'qbit', 'orb_version': '0.7.16', 'default_branch': 'main', 'publish': {'feature': True}}
with mock.patch.object(pipeline, 'git', side_effect=[None, b'', b'head\n', b'head\n']):
rendered = pipeline.qbit_template(self.config(manifest), '', 'publish-test')
self.assertEqual('feature', rendered['workflows']['publish_feature']['jobs'][0]['qqq-orb/mvn_publish']['branch_type'])
def test_setup_preserves_release_policy_and_rejects_drift(self):
for policy in ['stable', 'v-prefix', 'none']:
manifest = {'profile': 'qbit', 'orb_version': '0.7.16', 'default_branch': 'main',
'release_tag_policy': policy, 'publish': {'feature': True}}
caller = migration.canonical_caller('0.7.16', policy, True)
standard.validate(yaml.safe_dump(caller), yaml.safe_dump(manifest), '0.7.16')
self.assertEqual(caller, standard.canonical_caller('qbit', '0.7.16', release_tag_policy=policy, publish_feature=True))
with self.assertRaises(standard.StandardError):
standard.validate(yaml.safe_dump(migration.canonical_caller('0.7.16', policy)), yaml.safe_dump(manifest), '0.7.16')
def test_unrecognized_feature_branch_filters_are_rejected(self):
for branches in [{'only': ['/feature/.*/']}, {'only': ['main']}, {'ignore': '/.*/'}]:
with self.subTest(branches=branches), self.assertRaises(ValueError):
migration.publication_filters('feature', {'branches': branches, 'tags': {'only': ['/publish.*/']}})
if __name__ == '__main__':
unittest.main()
+25 -1
View File
@@ -68,6 +68,30 @@ class RolloutTests(unittest.TestCase):
self.assertFalse(rollout.rollout([root], '0.7.11', write=True)[0]['changed'])
self.assertEqual(applied, self.contents(root))
def test_managed_patch_track_and_exact_pin_rollback_preserve_every_profile(self):
roots = [self.checkout(profile, profile=profile, default_branch='main')
for profile in rollout.standard.PROFILE_PARAMETERS]
originals = [yaml.safe_load((root / '.qqq-ci.yml').read_text()) for root in roots]
rollout.rollout(roots, '0.7', write=True)
for root, original in zip(roots, originals):
self.assertEqual({**original, 'orb_version': '0.7'},
yaml.safe_load((root / '.qqq-ci.yml').read_text()))
rollout.standard.validate(*(data.decode() for data in self.contents(root)), '0.7')
rollout.pipeline.load_manifest(root / '.qqq-ci.yml', '0.7')
with self.assertRaises(ValueError):
rollout.pipeline.load_manifest(root / '.qqq-ci.yml', '0.8')
self.assertTrue(all(not r['changed'] for r in rollout.rollout(roots, '0.7', write=True)))
rollout.rollout(roots, '0.7.4', write=True)
for root, original in zip(roots, originals):
self.assertEqual(original, yaml.safe_load((root / '.qqq-ci.yml').read_text()))
def test_unquoted_track_is_rejected_as_numeric_yaml(self):
root = self.checkout('numeric')
manifest = root / '.qqq-ci.yml'
manifest.write_text(manifest.read_text().replace('0.7.4', '0.7'))
with self.assertRaises(ValueError):
rollout.pipeline.load_manifest(manifest, '0.7')
def test_one_custom_caller_rejects_entire_batch_before_any_write(self):
good = self.checkout('good')
bad = self.checkout('bad')
@@ -97,7 +121,7 @@ class RolloutTests(unittest.TestCase):
def test_invalid_target_and_duplicate_checkout_are_rejected(self):
root = self.checkout('qbit')
before = self.contents(root)
for version in ['latest', 'dev:test', '0.7', '0.7.11\n']:
for version in ['latest', 'volatile', 'dev:test', '0', '0.7.*', '0.7.11\n']:
with self.assertRaises(ValueError):
rollout.rollout([root], version, write=True)
with self.assertRaises(ValueError):
+18
View File
@@ -76,6 +76,24 @@ class MigrationTests(unittest.TestCase):
manifest = migration.infer_manifest(self.config(), "0.7.0", "main")
self.assertFalse(any(manifest["publish"].values()))
def test_managed_track_preserves_policy_and_caller_pins(self):
fixtures = json.loads((Path(__file__).parent / "fixtures/qbit-publication-policies.json").read_text(encoding="utf-8"))
for fixture in fixtures:
with self.subTest(repository=fixture["name"]):
expected = migration.infer_manifest(fixture["config"], "0.8.1", fixture["default_branch"])
expected["orb_version"] = "0.8"
actual = migration.infer_manifest(fixture["config"], "0.8", fixture["default_branch"])
self.assertEqual(actual, expected)
caller = migration.canonical_caller("0.8", actual["release_tag_policy"], actual["publish"].get("feature", False))
self.assertEqual(caller["orbs"]["qqq-orb"], "kingsrook/qqq-orb@0.8")
self.assertEqual(caller["workflows"]["setup"]["jobs"][0]["qqq-orb/generate_pipeline"]["orb_version"], "0.8")
self.assertEqual(yaml.safe_load(yaml.safe_dump(actual))["orb_version"], "0.8")
def test_reject_ambiguous_or_invalid_versions(self):
for version in (0.8, None, "0", "latest", "volatile", "0.8.*", "0.8.1.2", ""):
with self.subTest(version=version), self.assertRaises(ValueError):
migration.infer_manifest(self.config(), version, "main")
def test_snapshot_does_not_enable_rc_or_release(self):
manifest = migration.infer_manifest(
self.config("qqq-orb/mvn_publish", branch_type="snapshot",
+51 -7
View File
@@ -66,6 +66,38 @@ def gh(path):
return json.loads(command(['gh', 'api', '--method', 'GET', path]))
def discover_inventory(organization):
"""Discover every repository visible to existing gh authentication, across all pages."""
if not re.fullmatch(r'[A-Za-z0-9][A-Za-z0-9-]*', organization):
raise ReadError('invalid organization name')
pages = json.loads(command(['gh', 'api', '--method', 'GET', '--paginate', '--slurp',
f'orgs/{organization}/repos?type=all&per_page=100']))
if not isinstance(pages, list) or any(not isinstance(page, list) for page in pages):
raise ReadError('invalid organization inventory response')
entries, seen = [], set()
for page in pages:
for repo in page:
if not isinstance(repo, dict):
raise ReadError('invalid organization repository')
name = repo.get('name')
full_name = repo.get('full_name')
archived = repo.get('archived')
branch = repo.get('default_branch')
if (not isinstance(name, str) or not re.fullmatch(r'[A-Za-z0-9_.-]+', name)
or not isinstance(full_name, str)
or full_name.lower() != f'{organization}/{name}'.lower()
or type(archived) is not bool
or not isinstance(branch, str) or not branch
or full_name.lower() in seen):
raise ReadError('invalid or duplicate organization repository')
seen.add(full_name.lower())
entries.append({'name': name, 'nameWithOwner': full_name,
'isArchived': archived, 'defaultBranchRef': {'name': branch}})
if not entries:
raise ReadError('empty organization inventory; coverage is unproven')
return sorted(entries, key=lambda entry: entry['nameWithOwner'].lower())
class Snapshot:
def __init__(self, entry, live=False):
self.repo = entry['nameWithOwner']
@@ -318,9 +350,11 @@ def audit(entry, snapshot, version, worker_sha, policy):
branches = policy.get('required_checks', {})
managed = policy.get('managed_branches', ['main', 'develop'])
if (not isinstance(managed, list) or not managed
or any(branch not in ('main', 'develop') for branch in managed)
or any(not isinstance(branch, str) or not re.fullmatch(
r'(?:main|develop|(?:release|hotfix)/[0-9]+\.[0-9]+(?:\.[0-9]+)?)', branch)
for branch in managed)
or len(set(managed)) != len(managed)):
raise ReadError('managed_branches must be a nonempty unique main/develop list')
raise ReadError('managed_branches must be a nonempty unique main/develop or numeric release/hotfix list')
result['managed_branches'] = managed
for branch in managed:
if branch not in branches:
@@ -372,22 +406,32 @@ def run(entries, version, worker_sha, policies, live=False, snapshot_factory=Sna
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--inventory', type=Path, required=True)
inventory = parser.add_mutually_exclusive_group(required=True)
inventory.add_argument('--inventory', type=Path)
inventory.add_argument('--organization', help='Discover current repositories using existing gh authentication; requires --live')
parser.add_argument('--orb-version', required=True)
parser.add_argument('--worker-sha')
parser.add_argument('--policy', type=Path)
parser.add_argument('--live', action='store_true', help='Read current source and branch rules using existing gh authentication')
parser.add_argument('--source-branch', help='Inspect this branch instead of the repository default; requires --live')
parser.add_argument('--expected-active', type=int, default=68)
parser.add_argument('--expected-active', type=int, help='Optional independent coverage check; saved inventories default to 68')
parser.add_argument('--output', type=Path)
args = parser.parse_args()
if args.source_branch is not None and not args.live:
parser.error('--source-branch requires --live')
entries = json.loads(args.inventory.read_text(encoding='utf-8'))
if args.organization and not args.live:
parser.error('--organization requires --live')
entries = (discover_inventory(args.organization) if args.organization else
json.loads(args.inventory.read_text(encoding='utf-8')))
expected_active = args.expected_active
if expected_active is None and not args.organization:
expected_active = 68
policies = json.loads(args.policy.read_text(encoding='utf-8')) if args.policy else {}
report = run(entries, args.orb_version, args.worker_sha, policies, args.live, source_branch=args.source_branch)
report['expected_active_count'] = args.expected_active
report['coverage_matches_expected'] = (report['active_count'] == args.expected_active
report['inventory_source'] = ('organization-api' if args.organization else 'saved-inventory')
report['inventory_visibility'] = 'authenticated repositories; inaccessible repositories cannot be certified'
report['expected_active_count'] = expected_active
report['coverage_matches_expected'] = ((expected_active is None or report['active_count'] == expected_active)
and report['unique_repository_count'] == report['inventory_count'])
text = json.dumps(report, indent=2) + '\n'
if args.output:
+151
View File
@@ -0,0 +1,151 @@
#!/usr/bin/env python3
"""Batch-enforce observed orb checks; dry-run unless --apply. Never changes source."""
import argparse
import base64
import importlib.util
import json
from pathlib import Path
import re
import subprocess
from urllib.parse import quote
ROOT = Path(__file__).resolve().parents[1]
spec = importlib.util.spec_from_file_location('standard', ROOT / 'src/scripts/validate_standard.py')
standard = importlib.util.module_from_spec(spec)
spec.loader.exec_module(standard)
DESIRED = [{'context': name, 'app_id': 18001} for name in ('setup', 'verify')]
PROFILES = {'qbit', 'core', 'frontend-core', 'frontend-material'}
def api(path, method='GET', payload=None):
"""Use existing gh authentication; never expose response bodies on failure."""
args = ['gh', 'api', '--method', method, path]
if payload is not None:
args += ['--input', '-']
result = subprocess.run(args, input=None if payload is None else json.dumps(payload),
text=True, encoding='utf-8', capture_output=True, timeout=60)
if result.returncode:
try:
error = json.loads(result.stdout)
except ValueError:
error = {}
if method == 'GET' and path.endswith('/protection') and error.get('status') == '404' and error.get('message') == 'Branch not protected':
return None
raise ValueError('GitHub operation unavailable: ' + method + ' ' + path.split('?')[0])
return json.loads(result.stdout) if result.stdout.strip() else None
def required_checks(before, runs):
"""Append app-bound gates only after both succeeded; preserve all other gates."""
for desired in DESIRED:
matches = [run for run in runs if run['name'] == desired['context'] and run['app']['id'] == desired['app_id']]
if not matches or max(matches, key=lambda run: run['id'])['conclusion'] != 'success':
raise ValueError('Current branch head has no successful CircleCI ' + desired['context'])
prior = (before or {}).get('required_status_checks') or {}
checks = [dict(check) for check in prior.get('checks', [])]
represented = {check['context'] for check in checks}
checks += [{'context': name, 'app_id': -1} for name in prior.get('contexts', []) if name not in represented]
for desired in DESIRED:
matches = [check for check in checks if check['context'] == desired['context']]
if any(check.get('app_id') not in (None, -1, desired['app_id']) for check in matches):
raise ValueError('Existing gate uses a different app: ' + desired['context'])
checks = [check for check in checks if check['context'] != desired['context']]
checks.append(dict(desired))
return {'strict': True, 'checks': checks}
def inspect(repo, branch, version, request=api):
"""Read immutable canonical source, latest check runs, and current protections."""
if not re.fullmatch(r'[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+', repo) or branch not in ('main', 'develop'):
raise ValueError('Targets must name an exact repository and main/develop branch')
root = 'repos/' + repo
info = request(root)
if info['archived']:
raise ValueError('Archived repository')
prefix = root + '/branches/' + branch
head = request(prefix)['commit']['sha']
source = []
for path in ('.circleci/config.yml', '.qqq-ci.yml'):
item = request(root + '/contents/' + quote(path, safe='/') + '?ref=' + head)
if item.get('encoding') != 'base64' or item.get('type') != 'file':
raise ValueError('Canonical declarations must be regular files')
source.append(base64.b64decode(item['content'], validate=False).decode('utf-8'))
standard.validate(*source, version)
if standard.load_yaml(source[1])['profile'] not in PROFILES:
raise ValueError('Profile does not use the shared setup/verify gate pair')
rules = request(root + '/rules/branches/' + branch)
if rules:
raise ValueError('Ruleset-managed branch requires ruleset reconciliation')
before = request(prefix + '/protection')
runs = []
page = 1
while True:
batch = request(root + '/commits/' + head + '/check-runs?filter=latest&per_page=100&page=' + str(page))['check_runs']
runs += batch
if len(batch) < 100:
break
page += 1
desired = required_checks(before, runs)
current = (before or {}).get('required_status_checks') or {}
ready = (current.get('strict') is True and current.get('checks') == desired['checks']
and (before or {}).get('enforce_admins', {}).get('enabled') is True)
return {'repository': repo, 'branch': branch, 'head': head, 'before': before,
'required_status_checks': desired, 'changed': not ready}
def enforce(plan, request=api):
"""Recheck drift, use narrow updates, and verify preservation of other settings."""
root = 'repos/' + plan['repository']
prefix = root + '/branches/' + plan['branch']
if request(prefix)['commit']['sha'] != plan['head'] or request(prefix + '/protection') != plan['before']:
raise ValueError('Branch or protection changed after preflight')
if request(root + '/rules/branches/' + plan['branch']):
raise ValueError('Ruleset changed after preflight')
if not plan['changed']:
return
before = plan['before']
if before is None:
request(prefix + '/protection', 'PUT', {
'required_status_checks': plan['required_status_checks'], 'enforce_admins': True,
'required_pull_request_reviews': None, 'restrictions': None})
else:
request(prefix + '/protection/required_status_checks', 'PATCH', plan['required_status_checks'])
if not before['enforce_admins']['enabled']:
request(prefix + '/protection/enforce_admins', 'POST')
after = request(prefix + '/protection')
actual = after['required_status_checks']
if not actual['strict'] or actual['checks'] != plan['required_status_checks']['checks'] or not after['enforce_admins']['enabled']:
raise ValueError('Protection readback did not match requested gates')
if before:
for key, value in before.items():
if key not in ('required_status_checks', 'enforce_admins') and after.get(key) != value:
raise ValueError('Unrelated protection changed: ' + key)
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--targets', type=Path, required=True, help='JSON list of {repository, branch}')
parser.add_argument('--orb-version', required=True)
parser.add_argument('--apply', action='store_true')
parser.add_argument('--output', type=Path, required=True)
args = parser.parse_args()
targets = json.loads(args.targets.read_text(encoding='utf-8'))
if not isinstance(targets, list) or not targets:
raise ValueError('A nonempty explicit target list is required')
if len({(row['repository'], row['branch']) for row in targets}) != len(targets):
raise ValueError('Duplicate branch target')
# Resolve the entire batch before any mutations. Failures never trigger rollback
# that would remove successful protections; rerunning converges idempotently.
plans = [inspect(row['repository'], row['branch'], args.orb_version) for row in targets]
args.output.write_text(json.dumps({'state': 'planned', 'plans': plans}, indent=2), encoding='utf-8')
if args.apply:
for plan in plans:
enforce(plan)
plan['verified'] = True
args.output.write_text(json.dumps({'state': 'applying', 'plans': plans}, indent=2), encoding='utf-8')
args.output.write_text(json.dumps({'state': 'verified', 'plans': plans}, indent=2), encoding='utf-8')
print(json.dumps({'targets': len(plans), 'changed': sum(plan['changed'] for plan in plans), 'applied': args.apply}))
if __name__ == '__main__':
main()
+2 -2
View File
@@ -56,8 +56,8 @@ def plan(checkout, version):
def rollout(checkouts, version, write=False):
if not re.fullmatch(r'\d+\.\d+\.\d+', version):
raise ValueError('Fleet upgrades require an explicit stable orb version')
if not re.fullmatch(r'\d+\.\d+(?:\.\d+)?', version):
raise ValueError('Fleet upgrades require a stable minor track or exact patch version')
# Validate the entire batch before writing any repository.
plans = [plan(path, version) for path in checkouts]
if len({item['checkout'] for item in plans}) != len(plans):
+15 -5
View File
@@ -11,7 +11,7 @@ import re
import yaml
def canonical_caller(version, release_tag_policy=None):
def canonical_caller(version, release_tag_policy=None, publish_feature=False):
caller = {
"version": 2.1,
"setup": True,
@@ -28,6 +28,9 @@ def canonical_caller(version, release_tag_policy=None):
if release_tag_policy not in filters:
raise ValueError("Unknown tag policy")
caller["workflows"]["setup"]["jobs"][0]["qqq-orb/generate_pipeline"]["filters"]["tags"] = filters[release_tag_policy]
if publish_feature:
caller["workflows"]["setup"]["jobs"][0]["qqq-orb/generate_pipeline"]["filters"]["tags"] = {
"only": ([filters[release_tag_policy]["only"]] if release_tag_policy != "none" else []) + ["/publish.*/"]}
return caller
@@ -43,6 +46,12 @@ def publication_filters(kind, filters):
raise ValueError("Unknown publication filters")
branches = filters.get("branches")
tags = filters.get("tags")
if kind == "feature":
# Preserve the audited literal branch selector; it is not a regex and
# contains a character Git forbids, so only publish* tags can select it.
if branches != {"only": ["feature/.*"]} or tags != {"only": ["/publish.*/"]}:
raise ValueError("Unknown feature publication filters")
return False, "none"
if kind in ("snapshot", "release_candidate", "hotfix"):
expected = {"snapshot": "develop", "release_candidate": r"/release\/.*/", "hotfix": r"/hotfix\/.*/"}[kind]
if not isinstance(branches, dict) or set(branches) != {"only"} or single_filter(branches["only"]) != expected or tags not in (None, {"only": []}):
@@ -91,8 +100,8 @@ def test_filters(filters):
def infer_manifest(config, version, default_branch):
if not re.fullmatch(r"(?:\d+\.\d+\.\d+|dev:[A-Za-z0-9_-]+)", version):
raise ValueError("An explicit published orb version is required")
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+(?:\.\d+)?|dev:[A-Za-z0-9_-]+)", version):
raise ValueError("A minor track, exact patch or explicit development orb version is required")
if default_branch not in ("main", "develop"):
raise ValueError("Unsupported default branch")
if set(config) - {"version", "orbs", "commands", "workflows"}:
@@ -123,7 +132,7 @@ def infer_manifest(config, version, default_branch):
raise ValueError("Custom pre-steps require review")
if job.endswith("/mvn_publish"):
kind = options.get("branch_type")
if kind not in ("snapshot", "release_candidate", "release", "hotfix"):
if kind not in ("snapshot", "release_candidate", "release", "hotfix", "feature"):
raise ValueError("Custom publication contract requires review")
if kind in publications:
raise ValueError("Duplicate publication jobs require review")
@@ -148,6 +157,7 @@ def infer_manifest(config, version, default_branch):
"publish": {**{kind: kind in publications
for kind in ("snapshot", "release_candidate", "release")},
**({"hotfix": True} if "hotfix" in publications else {}),
**({"feature": True} if "feature" in publications else {}),
"asciidoc": False},
}
@@ -165,7 +175,7 @@ def main():
if manifest_path.exists():
raise ValueError("Already has a manifest; use tools/rollout_version.py")
manifest = infer_manifest(config, args.orb_version, args.default_branch)
caller = canonical_caller(args.orb_version, manifest["release_tag_policy"])
caller = canonical_caller(args.orb_version, manifest["release_tag_policy"], manifest["publish"].get("feature", False))
if args.write:
for path, data in ((config_path, caller), (manifest_path, manifest)):
path.write_bytes(yaml.safe_dump(data, sort_keys=False).encode("utf-8"))