Compare commits
21
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
fc39293b9d
|
||
|
|
e83b8481b9 | ||
|
|
e4ecae6294
|
||
|
|
37b91db07b | ||
|
|
c30d821f7e
|
||
|
|
6ae688dcf4 | ||
|
|
78207c16be
|
||
|
|
efd50aee2f | ||
|
|
6dd473be25
|
||
|
|
1c8a7c57e5 | ||
|
|
abaebf50ff
|
||
|
|
c56e52d4cb | ||
|
|
09046d45b0
|
||
|
|
9e9e6c1fa5 | ||
|
|
45aaccd2a9 | ||
|
|
ab77962b6a | ||
|
|
d0792d60d6 | ||
|
|
72f5f4d4e7 | ||
|
|
00af9aae31 | ||
|
|
7e60372948 | ||
|
|
45bab9d960 |
No files matched your search
@@ -56,6 +56,12 @@ jobs:
|
||||
python3 tests/test_qbit_feature_profile.py
|
||||
python3 tests/test_ci_maintenance.py
|
||||
python3 tests/test_rollout_version.py
|
||||
python3 tests/test_enforce_orb_checks.py
|
||||
python3 tests/test_enroll_consumers.py
|
||||
python3 tests/test_reconcile_enrollment.py
|
||||
python3 tests/test_discover_enrollment.py
|
||||
python3 tests/test_finish_enrollment.py
|
||||
python3 tests/test_static_analysis_exit.py
|
||||
python3 tests/test_validate_standard.py
|
||||
python3 tests/test_voyage_profile.py
|
||||
python3 tests/test_next_native.py
|
||||
@@ -85,6 +91,8 @@ jobs:
|
||||
bash tests/test_maven_ci_cache.sh
|
||||
bash tests/test_maven_test_results.sh
|
||||
bash tests/test_maven_ci_install.sh
|
||||
bash tests/test_browser_install.sh
|
||||
bash tests/test_frontend_build_reuse.sh
|
||||
bash tests/test_snapshot_refresh.sh
|
||||
bash tests/test_maven_settings_output.sh
|
||||
# Create jobs to test the commands of your orbs.
|
||||
|
||||
@@ -61,6 +61,12 @@ test-pipelines: pack
|
||||
python3 tests/test_qbit_hotfix_profile.py
|
||||
python3 tests/test_ci_maintenance.py
|
||||
python3 tests/test_rollout_version.py
|
||||
python3 tests/test_enforce_orb_checks.py
|
||||
python3 tests/test_enroll_consumers.py
|
||||
python3 tests/test_reconcile_enrollment.py
|
||||
python3 tests/test_discover_enrollment.py
|
||||
python3 tests/test_finish_enrollment.py
|
||||
python3 tests/test_static_analysis_exit.py
|
||||
python3 tests/test_validate_standard.py
|
||||
python3 tests/test_voyage_profile.py
|
||||
python3 tests/test_next_native.py
|
||||
|
||||
@@ -4,6 +4,8 @@ CircleCI orb for QQQ framework projects.
|
||||
|
||||
For centralized pipeline ownership, see [standardized manifest-driven pipelines](docs/GENERATED-PIPELINES.md).
|
||||
The initial QBit profile uses a small `.qqq-ci.yml` and a setup entry point; existing direct job callers remain supported.
|
||||
For centrally managed patch adoption and exact-version rollback, see
|
||||
[managed patch updates](docs/managed-patch-updates.md).
|
||||
|
||||
**For:** Teams using CircleCI to build, test, and publish QQQ applications
|
||||
**Status:** Stable (v0.6.x)
|
||||
|
||||
@@ -80,6 +80,14 @@ required-check unknown states, and the lint image/tool/command/caller contract.
|
||||
|
||||
## Batch orb-version rollout
|
||||
|
||||
For a standard QBit without a manifest, start directly on the managed patch
|
||||
track using `python tools/standardize_qbit.py /checkout --orb-version 0.8
|
||||
--default-branch main`. Inspect the preview, then repeat with `--write`.
|
||||
This preserves the inferred publication and branch-filter policy while writing
|
||||
the same track to the manifest and setup caller. Use `rollout_version.py` below
|
||||
for repositories that already have a manifest. Neither command configures
|
||||
repository protection or automatically enrolls newly created repositories.
|
||||
|
||||
Upgrade existing canonical CircleCI consumers with one command:
|
||||
|
||||
```sh
|
||||
@@ -103,3 +111,33 @@ verification and artifact evidence. The requested version must already be publis
|
||||
```sh
|
||||
python3 -m unittest discover -s tests -p test_rollout_version.py
|
||||
```
|
||||
|
||||
## Live organization discovery
|
||||
|
||||
Use `--organization QRun-IO --live` instead of `--inventory inventory.json` to
|
||||
include newly created repositories on each audit. Discovery uses the existing
|
||||
`gh` authentication and all API pages, retaining archived entries for explicit
|
||||
skip reporting. Unknown repositories receive no automatic exemption. Duplicate,
|
||||
empty, malformed or failed discovery aborts instead of reusing a stale inventory.
|
||||
|
||||
```sh
|
||||
python tools/audit_estate.py --organization QRun-IO --live --orb-version 0.8 \
|
||||
--worker-sha <40SHA> --policy observed-checks.json --output audit.json
|
||||
```
|
||||
|
||||
The organization mode has no fixed repository-count ceiling. Supply
|
||||
`--expected-active N` for an independent count check when needed; saved-inventory
|
||||
mode preserves its default of 68. Reports identify the inventory source and
|
||||
credential visibility limit: inaccessible repositories cannot be certified.
|
||||
This is discovery and audit, not automatic enrollment or branch-rule mutation.
|
||||
|
||||
For a qualified batch of canonical orb branches, use the shared
|
||||
[branch-check enforcement command](branch-enforcement.md). It adds the observed
|
||||
CircleCI setup/verify gates while preserving other protections. Source enrollment,
|
||||
scanner transitions and scheduled reconciliation remain separate requirements.
|
||||
Scheduling and authenticated onboarding still need to be configured centrally.
|
||||
|
||||
## Batch enrollment
|
||||
|
||||
Use [the shared enrollment command](consumer-enrollment.md) to convert recognized
|
||||
legacy QBits and update existing canonical consumers in one validated batch.
|
||||
@@ -0,0 +1,68 @@
|
||||
# Shared orb check enforcement
|
||||
|
||||
`tools/enforce_orb_checks.py` replaces repository-specific scripts for the shared
|
||||
CircleCI `setup` and `verify` branch gates. One invocation processes an explicit
|
||||
batch of current `main` and `develop` branches using existing `gh` authentication.
|
||||
|
||||
Create a targets file:
|
||||
|
||||
```json
|
||||
[
|
||||
{"repository": "QRun-IO/qbit-example", "branch": "main"},
|
||||
{"repository": "QRun-IO/qbit-example", "branch": "develop"}
|
||||
]
|
||||
```
|
||||
|
||||
Preview, then apply the same batch:
|
||||
|
||||
```sh
|
||||
python tools/enforce_orb_checks.py --targets targets.json --orb-version 0.8 --output enforcement-plan.json
|
||||
python tools/enforce_orb_checks.py --targets targets.json --orb-version 0.8 --output enforcement-result.json --apply
|
||||
```
|
||||
|
||||
Every branch must already contain the exact canonical caller for its manifest and
|
||||
have successful `setup` and `verify` checks from CircleCI app 18001 at its current
|
||||
commit. Supported profiles are QBit, Core, frontend-core, frontend-material and repository-policy.
|
||||
The complete batch is inspected before any mutation. Source/head/protection drift,
|
||||
unavailable APIs and unknown profiles fail closed. Existing repository rulesets are verified as described below.
|
||||
|
||||
The tool appends the app-bound checks, enables strict checks and administrator
|
||||
enforcement, and reads the result back. Existing required checks remain required.
|
||||
For protected branches it uses the narrow status-check/admin endpoints, preserving
|
||||
reviews, signatures, linear history, restrictions and other settings. For a
|
||||
previously unprotected branch it creates protection without adding human reviews.
|
||||
It never changes scanner configuration, repository visibility, billing, or source.
|
||||
|
||||
Output records the planned head and original protection, then verification as each
|
||||
branch completes. A batch is not transactional: if an API call fails, inspect the
|
||||
output and rerun; successful protections are retained rather than rolled back.
|
||||
An already-correct branch needs no mutation.
|
||||
|
||||
This command covers the shared orb gate pair only. Continue to run the estate
|
||||
audit for scanner/native-worker requirements and other policy. Automatic repository
|
||||
discovery, source enrollment and scheduled reconciliation remain separate work.
|
||||
|
||||
GitHub API reference: [protected branches](https://docs.github.com/en/rest/branches/branch-protection).
|
||||
|
||||
```sh
|
||||
python -m unittest discover -s tests -p test_enforce_orb_checks.py -v
|
||||
```
|
||||
|
||||
## Existing rulesets
|
||||
|
||||
When effective branch rules come from repository rulesets, the tool can verify
|
||||
existing enforcement without adding classic protection. Each contributing ruleset
|
||||
must be active for branches, have no bypass actors and require both setup and
|
||||
verify from CircleCI app18001. The current branch must also have successful gate
|
||||
runs. The tool rechecks the branch, effective rules and full ruleset records before
|
||||
reporting verified. Rule drift or an unbound/missing gate stops verification.
|
||||
|
||||
This path writes nothing. It preserves scanner requirements, conditions and the
|
||||
existing up-to-date-branch policy, including a false strict setting that avoids
|
||||
unnecessary rebuilds. Reports identify protection_mode=rulesets and the observed
|
||||
strict value. Unsupported organization-level rulesets, bypass actors and missing
|
||||
gates remain unresolved rather than being rewritten or ignored.
|
||||
|
||||
For classic protection, an empty push allowlist with admin enforcement disabled
|
||||
requires an explicit access-policy resolution before enabling admin enforcement.
|
||||
The tool rejects that case before any writes to avoid locking every actor out.
|
||||
@@ -0,0 +1,119 @@
|
||||
# Batch consumer enrollment
|
||||
|
||||
Use one target file for canonical consumers of every supported Circle profile and
|
||||
legacy QBits whose jobs and publication filters the shared converter recognizes:
|
||||
|
||||
```json
|
||||
[
|
||||
{"checkout": "/work/qbit-example", "default_branch": "main"},
|
||||
{"checkout": "/work/qqq", "default_branch": "develop"}
|
||||
]
|
||||
```
|
||||
|
||||
The branch values must come from repository inventory. Run on clean isolated
|
||||
checkouts of the branches being migrated.
|
||||
|
||||
```sh
|
||||
python tools/enroll_consumers.py targets.json --orb-version 0.8
|
||||
python tools/enroll_consumers.py targets.json --orb-version 0.8 --write
|
||||
```
|
||||
|
||||
The command validates the entire batch before writing. Existing manifests keep
|
||||
all publication and profile options; only the managed orb version changes.
|
||||
Recognized legacy QBits receive the canonical caller and inferred publication
|
||||
policy. An unknown job, filter, profile option, duplicate target, symlink, or
|
||||
concurrent declaration edit rejects the batch. Repeating a completed enrollment
|
||||
makes no changes. Filesystem errors during the write phase can leave a partial
|
||||
batch; rerun after correcting the error. This is not a filesystem transaction.
|
||||
|
||||
Only `.circleci/config.yml` and `.qqq-ci.yml` are changed. The tool does not delete
|
||||
other workflows, infer publication from repository names, or enroll unknown
|
||||
custom pipelines. Their behavior must first be represented by a central profile.
|
||||
After committing and qualifying the exact source, apply the shared
|
||||
[branch enforcement tool](branch-enforcement.md) to supported gate profiles.
|
||||
|
||||
This command provides batch source conversion. Use the coordinator below for remote signed submissions. Local conversion alone does not prove fleet deployment or release success.
|
||||
|
||||
## Remote signed submissions
|
||||
|
||||
Supply observed current branches, without cloning each repository yourself:
|
||||
|
||||
```json
|
||||
[
|
||||
{"repository": "QRun-IO/qbit-example", "branch": "main"},
|
||||
{"repository": "QRun-IO/qqq", "branch": "develop"}
|
||||
]
|
||||
```
|
||||
|
||||
```sh
|
||||
python tools/reconcile_enrollment.py --targets remote-targets.json --orb-version 0.8 --output plan.json
|
||||
python tools/reconcile_enrollment.py --targets remote-targets.json --orb-version 0.8 --output result.json --apply
|
||||
```
|
||||
|
||||
Use existing `gh` authentication, GitHub SSH access, Git author identity and a
|
||||
working Git signing configuration. The coordinator never installs credentials
|
||||
or changes authentication. Every target is inspected at an immutable head and
|
||||
validated before any remote mutation. Only changed targets are cloned. A moved
|
||||
target stops submission; refresh the plan instead of overwriting it.
|
||||
|
||||
An automation branch derives from the source commit and proposed declarations.
|
||||
The coordinator signs its commit, pushes without force, verifies the remote
|
||||
signature, parent, exact changed paths and declaration bytes, then opens a PR.
|
||||
An interrupted push or PR creation can be resumed with the same invocation:
|
||||
matching signed branches and PRs are reused. A mismatched branch is rejected.
|
||||
Closed existing PRs are reported as existing; they are not silently reopened.
|
||||
Reports include repository, branch, immutable head and submission URL, without
|
||||
embedding declaration contents or credentials. Keep the report if a later target
|
||||
fails; earlier valid submissions remain intact.
|
||||
|
||||
This handles remote target lists and signed submissions. Organization discovery is described below. Qualification-aware merging and classic branch enforcement are described below; scheduling and ruleset reconciliation remain separate work. A submitted PR is not a completed migration. Unknown profiles and
|
||||
native workflows require their own central conversion, not deletion by this tool.
|
||||
|
||||
## Organization discovery
|
||||
|
||||
The coordinator can select observed current branches directly from GitHub:
|
||||
|
||||
```sh
|
||||
python tools/reconcile_enrollment.py --organization QRun-IO --orb-version 0.8 --output organization-plan.json
|
||||
python tools/reconcile_enrollment.py --organization QRun-IO --orb-version 0.8 --output organization-result.json --apply
|
||||
```
|
||||
|
||||
Repository and branch inventories are paginated. Only existing main/develop
|
||||
branches are considered; historical release branches are excluded. Archived
|
||||
repositories and the orb's own bootstrap repository are reported separately.
|
||||
Each supported branch receives the same immutable preflight as explicit targets.
|
||||
Unknown workflows, missing entrypoints, unsupported branch layouts and unavailable
|
||||
APIs stay `unresolved` in the output. They are never counted as conforming or
|
||||
silently converted. No-CI and native-only repositories may therefore appear as
|
||||
unresolved here; use the estate auditor for their separate policy classification.
|
||||
|
||||
Discovery finishes before any submission. `--apply` processes qualified plans
|
||||
and retains unresolved entries for follow-up; one unsupported repository does
|
||||
not prevent independent valid submissions. An explicit `--targets` batch keeps
|
||||
its stricter behavior: any preflight failure stops the whole batch. By default neither mode merges PRs or changes branch protections. Use the explicit completion flags below.
|
||||
|
||||
## Qualification and enforcement
|
||||
|
||||
```sh
|
||||
python tools/reconcile_enrollment.py --targets remote-targets.json --orb-version 0.8 --output result.json --apply --merge-ready --enforce
|
||||
```
|
||||
|
||||
Both flags require `--apply` and are optional. `--merge-ready` revalidates the
|
||||
signed submission, exact parent/files/content and current base, requires successful
|
||||
CircleCI-app `setup` and `verify` checks and every advertised latest check/status,
|
||||
then requests a normal exact-head squash merge. It never uses admin override or
|
||||
force pushes. The resulting signed merge must have the qualified source tree.
|
||||
Pending/failed checks and changed bases are reported without merging. Repeated
|
||||
runs can advance the same PR once checks complete; there is no polling loop that
|
||||
holds a CI executor open.
|
||||
|
||||
`--enforce` independently checks the actual current target branch, which must
|
||||
pass its own orb gates before shared classic protection enforcement. Existing
|
||||
protections and other required checks are preserved. Supported gate profiles are
|
||||
QBit, Core, frontend-core, frontend-material and repository-policy. Other profiles
|
||||
remain explicitly unsupported for completion even when source enrollment works.
|
||||
Existing active repository rulesets with app-bound gates and no bypass actors are verified without writes. Unsupported or incomplete rulesets remain unresolved; none are rewritten or layered with classic protection. API/plan restrictions remain visible.
|
||||
|
||||
A merged source update and an enforced branch are separate reported outcomes.
|
||||
Neither proves artifact publication or deployment. Scheduling and ruleset
|
||||
reconciliation remain outstanding.
|
||||
@@ -0,0 +1,37 @@
|
||||
# Managed patch updates
|
||||
|
||||
The canonical caller and manifest accept either an exact stable version such as
|
||||
`0.8.0` or a quoted minor track such as `"0.8"`. CircleCI resolves a minor track
|
||||
to its latest published patch. A major-only version, `latest`, `volatile`, and
|
||||
wildcards are rejected. The caller import, setup parameter, and manifest must
|
||||
still agree exactly.
|
||||
|
||||
Use a minor track after qualifying that release line. Future patch fixes then
|
||||
arrive on the repository's next pipeline without a version-bump PR or an extra
|
||||
fleet-wide build. Existing exact pins remain supported and are not changed by
|
||||
publishing this capability. YAML minor versions must be quoted to remain strings.
|
||||
|
||||
The central batch tool applies the same update to all supplied checkouts:
|
||||
|
||||
```sh
|
||||
python3 tools/rollout_version.py --orb-version 0.8 --write /path/to/repo1 /path/to/repo2
|
||||
```
|
||||
|
||||
This changes only the two canonical CI declarations. Publication policies,
|
||||
contexts, branch/tag filters, and profile options stay unchanged. Normal source
|
||||
and publication checks still run when a pipeline is triggered; this setting does
|
||||
not skip tests. GitHub reusable-worker commit pins are separate and unchanged.
|
||||
|
||||
Automatic patch adoption trades exact-version reproducibility for centrally
|
||||
managed fixes. Keep patch releases backward compatible, qualify central changes
|
||||
before production publication, and use an exact stable pin for a canary or
|
||||
rollback. For example, the same tool can move a consumer from `0.8` back to
|
||||
`0.8.0`. Historical pipeline reruns can resolve a newer patch; choose an exact
|
||||
pin when reproducing an earlier run. Setup and continuation each resolve their
|
||||
orb imports, so patch compatibility also matters when publication overlaps a run.
|
||||
|
||||
Changing minor tracks is an explicit migration. Do not use this feature to push
|
||||
breaking changes into a patch release or to bypass repository checks.
|
||||
|
||||
Reference: [CircleCI orb semantic versioning](https://circleci.com/docs/orbs/use/orb-concepts/).
|
||||
|
||||
@@ -10,6 +10,11 @@ description: >
|
||||
- Collects JUnit XML test results
|
||||
- Stores test results and screenshots
|
||||
- Saves Maven cache for future builds
|
||||
parameters:
|
||||
build_before_test:
|
||||
type: boolean
|
||||
default: true
|
||||
description: "Build the React app first; combined jobs already build it with frontend_build"
|
||||
steps:
|
||||
# Ensure no stale Chrome/Driver is shadowing the PATH (your base image had Chrome 109)
|
||||
- run:
|
||||
@@ -17,7 +22,7 @@ steps:
|
||||
command: <<include(scripts/frontend_test_cleanup.sh)>>
|
||||
|
||||
# Install current Chrome; DO NOT install chromedriver (let Selenium 4 auto-manage it)
|
||||
- browser-tools/install-chrome
|
||||
- install_chrome
|
||||
|
||||
# System libs Chrome needs in headless CI
|
||||
- run:
|
||||
@@ -52,6 +57,8 @@ steps:
|
||||
# GPG signing will happen later in mvn_jar_deploy when deploying to Maven Central.
|
||||
- run:
|
||||
name: Build app, start server, run tests
|
||||
environment:
|
||||
PARAM_BUILD_BEFORE_TEST: << parameters.build_before_test >>
|
||||
command: <<include(scripts/frontend_test_run.sh)>>
|
||||
|
||||
# Always collect and publish test results & screenshots
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
description: Install stable Chrome using only its signed repository and existing Ubuntu dependency indexes.
|
||||
steps:
|
||||
- run:
|
||||
name: Install Google Chrome
|
||||
command: <<include(scripts/install_chrome.sh)>>
|
||||
@@ -21,7 +21,7 @@ steps:
|
||||
- when:
|
||||
condition: << parameters.install_browsers >>
|
||||
steps:
|
||||
- browser-tools/install-chrome
|
||||
- install_chrome
|
||||
- browser-tools/install-chromedriver
|
||||
- when:
|
||||
condition: << parameters.compile_before_verify >>
|
||||
|
||||
@@ -27,7 +27,8 @@ steps:
|
||||
- git_full_checkout
|
||||
- mvn_manage_version
|
||||
- frontend_build
|
||||
- frontend_test
|
||||
- frontend_test:
|
||||
build_before_test: false
|
||||
- frontend_package
|
||||
- mvn_jar_deploy
|
||||
- git_operations:
|
||||
|
||||
@@ -23,4 +23,5 @@ steps:
|
||||
node_version: << parameters.node_version >>
|
||||
- git_full_checkout
|
||||
- frontend_build
|
||||
- frontend_test
|
||||
- frontend_test:
|
||||
build_before_test: false
|
||||
@@ -11,10 +11,20 @@
|
||||
## Output: Installs required system libraries
|
||||
############################################################################
|
||||
|
||||
set -e
|
||||
set -euo pipefail
|
||||
|
||||
echo "Installing browser dependencies"
|
||||
export DEBIAN_FRONTEND=noninteractive
|
||||
sudo apt-get update
|
||||
ubuntu_sources=/etc/apt/sources.list
|
||||
if [[ ! -s "$ubuntu_sources" ]]; then
|
||||
ubuntu_sources=/etc/apt/sources.list.d/ubuntu.sources
|
||||
fi
|
||||
if [[ ! -s "$ubuntu_sources" ]]; then
|
||||
echo 'Cannot locate Ubuntu package sources' >&2
|
||||
exit 1
|
||||
fi
|
||||
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get update \
|
||||
-o "Dir::Etc::sourcelist=$ubuntu_sources" \
|
||||
-o Dir::Etc::sourceparts=- -o APT::Get::List-Cleanup=0
|
||||
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y \
|
||||
libnss3-dev libgdk-pixbuf2.0-dev libgtk-3-dev libxss-dev
|
||||
@@ -15,7 +15,9 @@ set -e
|
||||
|
||||
echo "Building app, starting server, running tests"
|
||||
echo "HTTPS=true" >> ./.env
|
||||
npm run build
|
||||
if [[ "${PARAM_BUILD_BEFORE_TEST:-true}" == true ]]; then
|
||||
npm run build
|
||||
fi
|
||||
export REACT_APP_PROXY_LOCALHOST_PORT=8001
|
||||
export PORT=3001
|
||||
npm run start &
|
||||
|
||||
@@ -89,8 +89,8 @@ def load_manifest(path, setup_version):
|
||||
allowed += ("java_version",)
|
||||
mapping(data, allowed, "manifest")
|
||||
version = data.get("orb_version")
|
||||
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+\.\d+|dev:[A-Za-z0-9_-]+)", version):
|
||||
raise ManifestError("orb_version must be an explicit stable or development version")
|
||||
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+(?:\.\d+)?|dev:[A-Za-z0-9_-]+)", version):
|
||||
raise ManifestError("orb_version must be a quoted minor track, exact stable pin, or development version")
|
||||
if version != setup_version:
|
||||
raise ManifestError("Manifest orb_version must match the setup job orb_version")
|
||||
default_branch = data.get("default_branch")
|
||||
@@ -225,7 +225,7 @@ def qbit_template(config, branch, tag):
|
||||
kind = "hotfix"
|
||||
if kind != "test" and not config["publish"][kind]:
|
||||
kind = "test"
|
||||
maintenance = kind != "test" and proven_ci_maintenance(branch, tag)
|
||||
maintenance = proven_ci_maintenance(branch, tag)
|
||||
if maintenance:
|
||||
kind = "test"
|
||||
workflow = "test_only" if kind == "test" else "publish_" + kind
|
||||
@@ -342,7 +342,10 @@ def frontend_template(config, branch, tag):
|
||||
kind = "release"
|
||||
if kind != "test" and not config["publish"][kind]:
|
||||
kind = "test"
|
||||
if kind == "test" and proven_docs_only(config, branch, tag):
|
||||
maintenance = proven_ci_maintenance(branch, tag)
|
||||
if maintenance:
|
||||
kind = "test"
|
||||
if kind == "test" and not maintenance and proven_docs_only(config, branch, tag):
|
||||
result = qbit_template(config, branch, tag)
|
||||
result["workflows"]["test_only"]["jobs"][0]["docs_only"]["name"] = (
|
||||
"qqq-orb/mvn_frontend_test_only" if material else "qqq-orb/node_test_only")
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
|
||||
# Machine executors are Ubuntu amd64. Preserve an existing browser for Maven
|
||||
# consumers; frontend_test deliberately removes its stale browser beforehand.
|
||||
if command -v google-chrome >/dev/null 2>&1; then
|
||||
google-chrome --version
|
||||
exit 0
|
||||
fi
|
||||
architecture=$(dpkg --print-architecture)
|
||||
if [[ "$architecture" != amd64 ]]; then
|
||||
echo "Google Chrome requires an amd64 machine executor" >&2
|
||||
exit 1
|
||||
fi
|
||||
curl --fail --silent --show-error --location https://dl.google.com/linux/linux_signing_key.pub \
|
||||
| sudo gpg --batch --yes --dearmor -o /usr/share/keyrings/qqq-google-chrome.gpg
|
||||
printf '%s\n' 'deb [arch=amd64 signed-by=/usr/share/keyrings/qqq-google-chrome.gpg] https://dl.google.com/linux/chrome/deb/ stable main' \
|
||||
| sudo tee /etc/apt/sources.list.d/qqq-google-chrome.list >/dev/null
|
||||
# Ubuntu dependency indexes were refreshed by install_java. Only the newly added
|
||||
# signed Google repository needs refreshing; image PPAs are unrelated to Chrome.
|
||||
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get update \
|
||||
-o Dir::Etc::sourcelist=/etc/apt/sources.list.d/qqq-google-chrome.list \
|
||||
-o Dir::Etc::sourceparts=- -o APT::Get::List-Cleanup=0
|
||||
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get install -y google-chrome-stable
|
||||
google-chrome --version
|
||||
@@ -1,5 +1,5 @@
|
||||
#!/bin/bash
|
||||
set -e
|
||||
set -eo pipefail
|
||||
|
||||
echo "Running PMD static analysis..."
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
#!/bin/bash
|
||||
set -e
|
||||
set -eo pipefail
|
||||
|
||||
echo "Running SpotBugs static analysis..."
|
||||
|
||||
|
||||
@@ -16,7 +16,7 @@ import yaml
|
||||
# Generated by scripts/pack_pipeline.py; edit source/resources, then regenerate.
|
||||
import base64
|
||||
import zlib
|
||||
CORE_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNrtPdl22ziy7/0VuOrM2M6IpPclifvcxHbfeCar48z0nChXByJBiW1uJig7Oonn26cKACmKIiVSluUlTr/IJFAo1F6FAvv7L4Q0LljEncBvPCPr+loTn4Q0oh6L4Tk8/A5P4JkX+N2g3XF8q205tOsHPHbM4XsYEQ9CBn83OkHgMuo3mskLi9m078bwzqYuZ+LxlXzb8Nm3uE1Nk4Ux9U3W7vR9y2XtfuQWgeZx5PjdIsiNRhWwvEfXt7bnCjliJnPC+Fqgf1HgG0HUyZD8/Pxcgyc48Awg8CgIzgz18H9X9W19r5HONAPPo76Vmc1jGsWaYJuGbNPiiJosiyCPWYgTvqgHCKbHzLOgH6fYknS4eB/1/QwE9dAHYUEkP0QMJIeR4NJnFnmLKxMemGcsJlmRaY7OVpgjAMcmvG8FhIax1oVZ/dCiMSPa+Tn5619HXzk+7M91iTYQr7nYHIxq+QBTgSTaBfE580GQCecpiHQqDPDIztYWMWIvFITNUCuDcQJWzFYraYyIH/s4mGjBNBhGGDHbdbq9WHeDLszts+ck7jEfIYcgILFNlti3MIhi8vHjx/bb9+/+73379OTlwVH75PO7/b+ct1r+Emk1nhwcnxy8OWr/6/3JP35/8/5f7b+/f9U+Pmw1yG+/4etXLz+9bh+9+2er0fIZaBvCZ2YvIEuSIYg1CBNJ8SF9n15Qx6UdF1AyAz92/D4wMXK6jk9d8pZeMJ8IweD6Usu3nZbfyLDwKv19NYvQvO9wFl3Aer47AII4nPwZdJY4AXyY66ZyZDvf4n7ExqSnQ82zbhT0hQAhVculK2Y8JpqPRPpeQOJn2hUQ8ccPwr45MVlt+YLdmgaEEhhqzL/Yl8Q/OG4Wc6FZAJeEg7gX+BtEN53IdJnpGEJyMoKih4NSmqpfX0eMEI+DsKZqV+fI747v8F5NLb4EUcbJ1L2kg0k6PhMX4qBv9qaqGFJFTBXifhD0XYv4QUy40xWCLLZiBiBVZhxEQpaDiNA4Zl6IFoU8WebsnKyR9dWV58QKUHUEuv9DNHvq6iAmYNX1P3ngIxKdiNEzhMBdxkKy1vKtwGdqP2bPC8BqnBD6t+iPqeanYEsePWMEjdhwHAFz79jUjDmBpS1UZ9wi2FRJ8lo7GNomk8Z1JhYbnQyWyl8Sz+Ec7NDzRLZQ3CLec0IY4VGw0WCXwigA2zOb0QFhiFg7pcm4uIc07qFATtlcXpQtICbYxVhGTI2SSTV0GSyu7XTBtmkh2DLaZZpyUPPS53+g/CnYifMT+KPlB+H3LWZNUNhfyWkPLCDp9B0QPo+aPcdnnKCjtxxu0gimE2qjkwWZQeutt/xfyWeY8hnCrri/hINhFhh5z4miIGqSy54D+gzBIKgCjSIH4OFcVFQAxpnZj5x4AA7XiZmudIbDG80hS/xHL47DZ4aRDNP7YhkdcDbkTzGCwxC1bsGI7hIxWGwaEE8YPOhHJuO66/BYLZaJEcQwnzELpBxDKgMZplstP/Haf+HCNS898SN89V2NuyL7YOTcVuP5EvkhY4eYsTJwxt6ehkJoOjo+IL8ZFrsw/L7r1vYMIEmceqHLNKmXfF6CdCBNJ/n7Zx9ss7AqEPFBxDK7N/DOLCciWkhw7wpr0Po/YRkDqAORnvFnH1ZLtiJNtlwVLXb5NA4aZTsREkGgaJwefTrVnurfPHfCLBsCIU7tolmJRwCD+gWtKTgwOQj81dehvQSLGY68q7gzAyejmZNuYhZjhxxpj/M8Z+8q4nMbCAiFYJFGw9B1TGGjNIHUDZn+ckw6UXDJEZM00Vs4CmBcQ5cOtEvHgj/5wteX62qYEESBu/j1bUYx4gcELliEbjETZSwcGcz6M8KggaR22SRUCo1ziJUWHidrsAvHYv5o4F5QepF76IGjzWGerzCMrZ4SZC5mX2JNEqwJWmL02gmDSBeS9et7AhGP5ShkvHhBhpTRBTEg35XeAOaJgH6ADiFv9kneopNSI0PGlZ7klJDklGLUJ1hg98tF6EmKZ4HDgFSg8twaBLqWU1HS2o6D9mUQnfFwNMdMRCcI4jSMzgt2c1zPRqVQvSjGvjEy7GsVRUvrYRCGZmphjHfawN0zFrUxugoij+b0TsW1o0rneCDWuDkZNGrr66ubzyDojJifuqShjoF9EoFk23Qpx8UbLnKv0ZxUbysN/oekS+qAhnrV/pNe0Oz7gqJdOse78Nsicp+povdPFjn2gBxiihYRBwL5pFIktF+SlGRJWq7glgICY+dYPjrp++Ql6LOHyPkWOaGdjhO//VgPN6AS0V4RjUvlSgs18FzjLI6x+ioiRy2U0SMIFNGoR7RD+KVnFtH5mRPuizo3MbEMTi4EDee5ZXbeB3OQ7FDlEdmdEsjt3jh+/9uELSdVKY20Wi9eLH34N2bZUeARVFLX6RDHE0H2B/hTvQEC6CyOIIlR745c5oEqnMKjFtiyxMLui0nL0qABfcqC61ZjRVpwtRMw3zBJcbPVaIJJTNgJQ59JY4nowQrJWgaxWw0RpMPe9LRMrsPSep413+U6VwfDZ8enyFaslRJR4IV8E4Bn9qWDXeJsGVdd0WETaO6WV+Rwj8agerjfLyblUiXED+GHMHW1Hd8CjYVdYWiM72Aj6C3wJ4KDN8hWfLqP6SJY+cD3wdTzl5hRvxIIn0gOtxpf5bpgXRhQ32X+skJBzF5rIi2STT7Ly4fI11TVJdmwgqSgfFn9KjAGpFCKQ2YJbDl5B96jAvBLCt41mdjyP/z7ptOYCbJVK2cWXqvdAdZidT9sSzuedQ/sGzP7gF2yMprWETO8SONeI3TDySTdGIHBPnANDaXLIJJBBnaB1Hj+xysYi6IIBbk1JN0wdhJF7WkTVZDfTmLISZXw2Y6kADCR/MzQQZgzsM1Y+B0kJxwQAbDZaFCyfaLJPCFZX9MwjsIwN605pjM1JXPzdxMqXhknQlmUNs/izXB/adpQudKTFFpKiWVkqC3K7pmo2gxrzKuCf27OjayRRvOy4I3+BA8LwZ2Ur/MUzxMLalPwdJhnqJqUfFYLk+eZ9OEul95mrbpl9zdBw2UQp+VrEoVKHknFSwarvFlPiXuzeVjpxisnY6UyXykfu9sVyusVBcfU4z4UxSaFPJny5iKCnRFqFAY+YyRJ3vqBxdQJ/zSa32A+/EqcgdlYqUBVJBYL8fjMN/EEK6mHSdrOJds8jEDSMDjbX9Xhv0QoMgUq7RAj7lPQK67yzYTs84skZBQJBBXbCx1fNAZQyE8c6hKL8l4nwOpk+Y67DuSnLhhZkpzMwZNevyNO5D5CDq8dvxfuDQIzH08kNU+B11LwOgLJVpnGh7R8HKMdTBlGEgEBY22xGOI+srazu8k22a61vmdumXvM3jU3V5m5aa9vm3TLtOyt3c7a3s567eCUSaWXweUYhc/YoNAEpzj7oaddrGlV0dO+f1edOn0P3OFkMhiJorqBeSaDmga5urqOla+gPBNFiNAuHv3HshEo1bMJkmVaU2XiVwIJPxAE3KjojJDAlZoAGh/evyWXII6JgAvzsMSTLjLNdUBuwQ6DH45Bz/VhoKCZZGLVJIQXaTEE/pC1hpUWiPi3GN7gI8SojX9jVSEAAmEZ4AWWL1Rf5m+bqPovjOwjTK9V7o5TQY36frwMs5NCQKvxQdJ4SNqPHz8SNR9khPpdmaUjDpeREzOJhAAHUZQLQQYCbOaxKTFEefxWYDdAG5BfYjqgZy7rUnOghYxFoHMhl41VNvwV2LagsKYJJSH/MXSY1fIPjvfR2hMEITtYNMygsFTx0nVViQ2irb5/7L8CPiGj3+JbwUSJGRAAGGf1TdHuIBj7HDuLsIWjqyI2kJmQXNLIR7Ormk/Sv3WBh1wLEYHVJJiWn2Y/PDINnGQkBWBuFImhrPKLyYZebZKBezqV0grvsIlJUCJ1OwMyIsmihw42KgR8pF8D9lHD62RYOcHxVPBN3bArC6KCk0vaIaK//xTd1dKou6pdKoKcfYJVFW72No1o1WBbSvuN2Vwsj6tzO4vGtElen55+EPUfddhFMiFghWLxK2yZxSBcyqHDRSCKQjUes6JuLImI/ynkSUupLH98BU4ay/WJAgqxVQ2ezYw4g9AHIy2hgQkaIJheT55rRVGQX2b2glTTzjtObCj7nRfb+aJRTfNwVBIp6QWR0lQIdlGOm2xQ+5A56Eb+8WZKhSyU7PHGXIt2KLRvqNexEgmpJqRAbaKxfgDhRciwmjBn5qijH8+xLJeBgwCdF0g28Wk/dlwnHqhHPEecCpWyfNFEOIzC9SaXWUaPQeRaNAz1T+KXJOuhw0PhR9PdoehPqc+MLZdimNv7dfD7dM4/S2jXxW2qPBa3ys2pfFGZaTd7QFKRMzPVoyZ0seR6VlS1AVgfsRmKFB7s08H8f9Hlils8eL8jhYaU9AupM4hgBXCChcWa6qKUPNgXqMjny69frlRzCfMkQE2fSVotLFmrkFdZN7G5dwDuUG3tVCQ2h4lC6ui2ju13wacQ0LQdZgmKJ8nOoRPrGXBvETmEdnwK79KadI+68Xv/d4AEUAtaEebrq5NmhCDuoZS6wQCPzokHaHEC8VsPAnVMmxbbiiAN3WgnQqHrSHsQpClsEs/xHQ8Cf2xGwIPwWZxYIZtl8t8k2ytNefhe9A9WLD4HmbxiRhKSZdbWVpKWifK2BkUmQ+1+rMMhAqHZxzh8WfYzyI4FId+Y4gPH8u+GjQMrIy0GCOm3/YS6splALnoFsgVvr4hUG/jZhBQd5B+z9+9qwlWuYyG/rC2lXWK1v09Whc3Ij2J4+2A4pqm2XbFVoThgmOS5TFV3enRco44roQt/QEXyxBCqnLbLfLDGKMKq31QIZKbnNOPARPWRuHhB5u3g08c3ear4QRt4Ffbjdux4DNkG622sevVyoVQYUcH3nyx7Z3jvTdxwSQ6X09bYoS/7Q/zDkiLine truncated
|
||||
CORE_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNrtPWt3m0iy3+dX9NVk13ZWgN+PJJ5zE9tz4908HWd39kS5Oi1oJMYIMI3s6CTe375V3Q1CCBDIsvyIM19k6K6urndVVzPffyGkccFC7vhe4xlZ19ea+CSgIe2zCJ7Dw+/wBJ71fa/rtzuOZ7Uth3Y9n0eOOXoPI6JhwODvRsf3XUa9RjN+YTGbDtwI3tnU5Uw8vpJvGx77FrWpabIgop7J2p2BZ7msPQjdPNA8Ch2vmwe50agClvfo+tb2XCGHzGROEF0L9C8KfMMPOymSn5+fa/AEB54BBB76/pmhHv7vqr6t7zWSmabf71PPSs3mEQ0jTbBNQ7ZpUUhNlkaQRyzACV/UAwTTY+aZP4gSbEkyXLwPB14KgnrogbAgkh9CBpLDiH/pMYu8xZUJ980zFpG0yDTHZyvMEYBjEz6wfEKDSOvCrEFg0YgR7fyc/PWv468cD/bnukQbitdcbA5GtTyAqUAS7YJ4nHkgyITzBEQyFQb0yc7WFjGifiAIm6JWCuMYrJitVtIYET/2cTDR/GkwjCBktut0e5Hu+l2YO2DPSdRjHkIOQEAimyyxb4EfRuTjx4/tt+/f/d/79unJy4Oj9snnd/t/OW+1vCXSajw5OD45eHPU/tf7k3/8/ub9v9p/f/+qfXzYapDffsPXr15+et0+evfPVqPlMdA2hM/Mnk+WJEMQaxAmkuBDBh69oI5LOy6gZPpe5HgDYGLodB2PuuQtvWAeEYLB9aWWZzstr5Fi4VXy+2oWoXnf4Sy8gPU8dwgEcTj50+8scQL4MNdN5Mh2vkWDkE1IT4eaZ93QHwgBQqoWS1fEeEQ0D4n0PYfEz7QrIOKPH4R9cyKy2vIEuzUNCCUw1Jh3sS+Jf3DczOdCMwcuCYZRz/c2iG46oeky0zGE5KQERQ+GhTRVv76OGSEe+UFN1a7Okd8dz+G9mlp8CaKMk6l7SYdlOj4TFyJ/YPamqhhSRUwV4n7gD1yLeH5EuNMVgiy2YvogVWbkh0KW/ZDQKGL9AC0KebLM2TlZI+urK8+J5aPqCHT/h2j21NVBTMCq639y30MkOiGjZwiBu4wFZK3lWb7H1H7MXt8Hq3FC6N/CP6aan5wt9ekZI2jERuMImHvHpmbECSxtoTrjFsGmSpLX2sHINpk0qjMx3+iksFT+kvQdzsEOPY9lC8Ut5D0ngBF9CjYa7FIQ+mB7ZjM6IAwhayc0mRT3gEY9FMgpm8uKsgXEBLsYyYipUTCphi6DxbWdLtg2LQBbRrtMUw5qXvr8D5Q/BTt2fgJ/tPwg/J7FrBKF/ZWc9sACks7AAeHrU7PneIwTdPSWw00awnRCbXSyIDNovfWW9yv5DFM+Q9gVDZZwMMwCI993wtAPm+Sy54A+QzAIqkDD0AF4OBcVFYBxZg5CJxqCw3Uipiud4fBGc8gS/9GLouCZYcTD9IFYRgecDflTjOAwRK2bM6K7RAwWmQbEEwb3B6HJuO46PFKLpWIEMcxjzAIpx5DKQIbpVsuLvfZfuHDNS0+8EF99V+OuyD4YObfVeL5EfsjYIWKsCJyxt6ehEJqOjg/Ib4bFLgxv4Lq1PQNIEqf9wGWa1Es+L0E6kKaT/P2zB7ZZWBWI+CBimd0b9M8sJyRaQHDvCmvQ+j9hGQOoA5Ge8ecAVou3Ik22XBUtdvE0DhplOyESQaBonB59OtWe6t/6bsksGwIhTu28WbFHAIP6Ba0pODA5CPzV15G9BIsZjL2ruDMDJ6OZk25iFmOHHGlP8jxj7yricxsICIVgoUaDwHVMYaM0gdQNmf5iTDqhf8kRkyTRWzgKYFwDlw61S8eCP/nC15frapgQhL67+PVtRjHiBwQuWIhuMRVlLBwZzPpTwqCBpHZZGSq5xjnASguP4jXYhWMxbzxwzym9yD30wNFmMM9WGCZWTwgyF7MvsSYx1gQtMXrtmEGkC8n69T2BiMcyFDJevCAjyuiCGJDvSm8A80RAP0SHkDX7JGvRSaGRIZNKTzJKSDJKMe4TLLD7xSL0JMEzx2FAKlB5bg0CXcupKGltR3770g/PeDCeY8ai4/tREkZnBbs5qWfjUqhe5GPfGBv2tYqiJfUwCENTtTDGO23g7hkL2xhd+WGfZvROxbXjSuf0QaxxczJo1NbXVzefQdAZMi9xSSMdA/skAsm26VKOizdc5F6jWVZvKwz+R6SL64CGetX+k17Q9Pucol0yp3/htUXkPlNF758sdOwhOcQULSQOBPJxpUhovyQpSZO0WMEtBQTGzrF8dDLwyEvQ5z4i51nkhHY6TvT2Yz3cgEpEe0U0LpUrKdTAc42zKMLqq4gctUBGjyBQRKN9oh3CLz21iM7PnGBf1LmJiWVwciFoOM8ts/MBmIN4hyqPSO+UQG73xvEG30q2HFelNNJqvXix9OHfmGWHfp+gkrpOhzh9EWR/gD/VGyCAzqIQkhj17shlfVCFU3jUAlsWW9h9MWlZGjSgT1Fw3WqsSAuudgLmGyYpbrYaTTCJMTth6DNpLBE9WCFeyyB2qyGCdNibnpTJdVhaz7Lmu1zn6mD07PgU2Yq1UiIKvJBvAvDUvnSwS5wt46orOmwCzd3yihzepxGoHu73i0m5VAnxQ/ghTF1tx7NAY2FXGBrjO9gIegv8ieDgDbIVn+5jughW3vc8MPX8JWbUrwTCJ5LDrcZXuS5YFwbUd5m3rFAQs9eaSIt4k8+y8iHyNVV1iTesICkoX1a/CowBKZTigFkCW07egfeoAPySgneNJ7a8D/++6TSmRLZq5czCa7U7wFqs7gdtacfT7oF9Y+YAsItXRtM6ZoYXadxrhG44mSQbIzDYA66hoXQZRDLIwC6QGs//eAVjkRehILdGpBvFTqKoPW2iCvLbcQxZVgmf7UgKABPJzxQdhDkD24yF32F8wgERAJuNBgXbJ5rME+L1NQ3jKAxzk5pjMlNTMjd/N6HilUkiFEVp8yzejPaXpA2VKz1xoaWQWEaK2qLsnoqqzaDGvCr4Z+bcyBpJNC8L3uhP8LAQ3EnxOk/xPDGnNgVPR3mGqknJZ7UweZ5KH+5y6W3Wqlt6fyUaLoM4LVuTyFXyUCpePFjlzXpC3JvNwwo3XjkZK5T5SvnY3a5QXq8oOKEe96EoVhbypMqbiwh2xqiRG/hMkCR+6/kWUyf802h+g/nwK3EGZmOlAlWRWCzA4zPPxBOsuB4maTuXbPMwBEnD4Gx/VYf/YqFIFai0Q4y4T0GvuMo3Y7LPL5KQUSQQVGwvcDzRGEAhP3GoSyzKex0fq5PFO+46kJ+6YGRJfDIHT3qDjjiR+wg5vHb8Xrg3CMw8PJHU+gq8loDXEUi6yjQ5pOXhGO1gyjASCwgYa4tFEPeRtZ3dTbbJdq31PXPL3GP2rrm5ysxNe33bpFumZW/tdtb2dtZrB6dMKr0MLicofMaGuSY4wdkL+trFmlYVPe37d9WpM+iDOywngxErquubZzKoaZCrq+tY+QrKUypChHbx6D+SjUCJnpVIlmlNlYlfCST8QBBwo6IzQgJXagJofHj/llyCOMYCLszDEo+7yDTXAbkFOwx+OAI910eBgmaS0qpJAC+SYgj8IWsNKy0Q8W8RvMFHiFEb/8aqgg8EwjLACyxfqL7M3zZR9V8Y6UeYXqvcHaeCGg28aBlmx4WAVuODpPGItB8/fiRqPsgI9boyS0ccLkMnYhIJAQ6iKBeCDATYzGJTYIiy+K3AboA2IL/EdEDPXNal5lALGAtB5wIuG6ts+Mu3bUFhTRNKQv5j6DCr5R0c76O1JwhCdrBomEFhqeKl66oSG0RbA+/YewV8Qka/xbeCiRIzIAAwzhqYot1BMPY5dhZhC0dXRWwgMwG5pKGHZlc1nyR/6wIPuRYiAqtJMC0vyX54aBo4yYgLwNzIE0NZ5ReTDb3aJAP3dCqlFd5hE5OgROJ2hmRMkkUPHWxUCPhYvwbso4bXSbGyxPFU8E3doCsLooKTS9ohor//FN3V0ri7ql0qgpy9xKoKN3ubRrRqsC2l/cZsLpbH1bmdRSPaJK9PTz+I+o867CKpELBCsfgVtsxiEC7l0OEiEEWhmoxZUTeWRMT/FPKkpUSWP74CJ43l+lgBhdiqBs9mSpxB6P2xllDfBA0QTK8nz7WiKMgvU3tBqmnnHScylP3Oiu180aimeTgqjpT0nEhpKgQ7L8eNN6h9SB10I/94M6FCGkr6eGOuRTsU2je037FiCakmpEBtorGBD+FFwLCaMGfmqKOfvmNZLgMHATovkGzi00HkuE40VI94hjgVKmXZoolwGLnrlZdZxo9B5Fo0CPRP4pck66HDA+FHk92h6E+pz0wsl2CY2ft18Pt0zj9LaNfFbao85rfKzal8UZlpN3tAUpEzM9WjSrpYMj0rqtoArA/ZDEWKPuzTwfx/0eWKWzx4vyOFhoT0C6kziGAFcIKFxZrqopQ82BeoyOfLr1+uVHMJ8yRATZ9JWi0sWauQV1k3sbl3AO5Qbe1UJDaHsULq6LaO7Xf+pwDQtB1mCYrHyc6hE+kpcG8ROYR2fArvkpp0j7rRe+93gARQc1oR5uur42YEP+qhlLr+EI/OSR/Q4gTitx4E6pg2LbYVQRq68U6EXNeR9CBIU9gkfcdz+hD4YzMCHoTP4sRy2SyT/ybZXmnKw/e8f7Bi/jlI+YopSYiXWVtbiVsmitsaFJkMtfuJDocQhGYf4/Bl2c8gOxaEfGOKDxzLvhs1DqyMtRggpN/2Y+rKZgK56BXIFry9IlJt4GcTUnSQf8zev6sJV5mOheyytpR2idX+PlkVNiM7iuHtg9GYptp2xVaF/IChzHOZqu706LjGHVdMF/6AiuSxIVQ5bZd5YI1RhFW/qRDIVM9pyoGJ6iNx8YLM2+Gnj2+yVPH8NvAqGETtyOkzZBust7Har5cLJcKICr7/ZLl/hvfexA2X+HA5aY0d+bI/xDLine truncated
|
||||
MATERIAL_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNq1lMtugzAQRfd8heV1TNRKlar8QXdVt02EXDMhDthGfqRBiH/vQAmPvKRU6g7PjH3PXI+pI0IoHEEEb6yjK1JjAENlwatvK7OdH2IYTY3IwWLks4+QIdflpeIZYJoqYWMlhTXObH0sjFqOB64OT/HLa/zM9lypig77m/5rE51WzaKl25uvq2CJB+fdFO/UR0swaWBxyjsPpZvRU7EDkZswFp11ZFHDWEgEx8qJVp/OoZof2Md1qViKYqyuSSfhgiJrWnKRo0OsQB/jvTN6TUnTTLTPt9NZZjNZNaNvt9CDvgTWXHUX9Kad50VBUAV0ClpIcGccFK9NcZ225QhEhCSMFZBxUbESwHaE9CEkxw/3rOyV/uwcVvnd9fswKSTKpKEA92+efgRN3oexIwfpAi+IhQyHyEmjye/A3nP5SMa57cof9LebVW693HIxexsTg+avg1kojfXLc6wUxaXmHrmvbvgLWNtQgmaE4h5bW8X6quUNmctfRdREPykCLqM=")))
|
||||
VOYAGE_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNrtW+tz2zYS/56/AqPJXJKeSUqqHzndqBNbdm1NGzsnObneVDkNREISLBKgAVCWYuv+9lvwpQepV2K1TpoPsUhggV38drEPgLl/hlBhSISknBUqqGyW9nTLDe9IeL2HZ3jrBNR1DMwcQxGp0nbo8bDdp4zMNEEj9XBPNxWCTsBUYJTLxf2KHQhBmCrEZJO9ZApBJA+ETdq2i6VmWnCxgPEpAWFDKjjz9Og5Pm+PP5xdtq/eXTf1KOM3bwSMXntZFlIRX8/8+3Ss3Sf2gAcqZYNmZob+29tbg4uORZlU2HXbN3iI57iHVLq1PUWvUC4VZggm6fNkyiWd2Ruytg3wkTYgoLggy0QRActyZtgLIf5ABO2OEfZ9l9pYgRx7iDJFeiJ8QVphcg/dBtilCuiYg2wOEuPeLL8IEu550K9nBdmQ7RLM0DCa3zhBxvU+MozAd7AihmTYl32uJDQxbiiBmewSYfiC92A58nNgkHi4FIMQoLZejEYrcJXMQuJj1dfCK20+ypKBIF0qiCGIz4VaL9KXMuxi6krc/SKGWCjaxfYG3IZ8DDo0b7BY1KMDQlMWql/Tg2mEVDuVRlJF1sgRm2DoRlIT3EooX280UIji7TsuBtLHNslKJjjXfqJgLsqjRZ73AnFHvIbQ/xC5MGzar43BWEeUsbplhEv1h1aBOu28wTa3uUlGxM7SnHJ7QESXujnjfe6ZI8/NdkiRM5Npc9alvRxqznC432+IrUz4Be0oSuasHqGP+Qp+Ntsbq7sgiS2Ikoa08azDKzjhYuYUN2cWabixqdezOliSxWAzw3FF2JEeOPrClwSNlZ66HkUSdE4VeNaBXOF+70DByLhFfaV8WbGsHlX9oAOq8PRjOHr6IAj8gklaDr9jLseONXxtlktmOSVpR+9tl7Jg1B4d7ptgQ2bvU4vBLzJGn7poPand97iD/j5KSVtsu927EpxGwDYBxkzXhByiwPAg/ERqRCY8RraKzITIVNxzoR2cTYfLZd4m3xgxbPbPtkSw40AwBRtU+JYkHsREv+LiMHvayiY94tDA251RNiPZUPO4eY2aernLwY/XgTQqqNVqMTSFHAeKL7b5lk6Psq38DkvfUBz+EZbtBjcQiDBQBA5VaT8Rgov07QbcD/zAsv1AoVgyI47UZtgbkZro4QEpEZAW26X8Jir/9LcSAlaEzIijQ4CpRmoqxI7CcB4Ca+Jx7pBdS5cAsrFs6YDtAklkQYv7dyfVyvotOgcaVgqkWJXA4Dh92c67RjVElEpn5xyQcW7qE2UhkL/T4dgYFo8Oi8ZGAfyzfI2d1BOUOLMVC/hynzCHMJtmMqsC4+1ol7cV9Yj2czBV6cBb6aggLpCAI5/6RGflEOgAHmRIlEm+WswbOBSioI9ahecXV2/PLDPE0AoxaRWSIalRxYldi0XRAMGawTsIL3EUQz3Ru3+fWhleFUsb5GyD4MBgybhFhhUr4Tw/JCMzEEIKnDSl1BgScJgzpimaR4dm8Y3s4/LBYaW8Xzp0St2j/eL+QefgqHy073RL+Mf9IiHksHNQKsJcztGP9uEhPrTtzmsHF/9R7pRKR6Rz5JTLZaQ5dmXqK/U6GWTraBi4Gh1JtN6h/ryon1/s1Rr163rt+Fft1UcUcmruEFRKB7uQ4xuQKRn+oKfryy4XHlZo3uUnYFizthM5/hDlrT0tFJ8rNo82q5y9cn+PQAy7jyaTjauN/80p60t229ZF2oJFrXHCKbLgSOORW3nhsEB4Ii54bZa0I7f8tRwjNbWyai4PHHTMsDuGOnvN8ZBxgkINV8K/yeY9Dd/MuCz8hYyrt8DXoLwd7Z4FOi56mNFPockllAskfS6VGQi3mhRDYbOtRTUzxIoPCKs+v29eXR432tdXv5xdThZIkqMHMy6goRRuhNvhnd6w1ZnSOy6xrSnhwlTS9kyHStxxiVNdmd7lbpAofhjh4epfeIustMsTDQ6KjjNQ3Vt9ahkH5BBQZCgUmVzl+X2t3qj9etZuXhyXKsXK0QSZafRWuLeCLu6JKrdHLXivtfNHNQ4lOOharKu9rhv1D/9pfzhrNOtXl9VWoWgeQKXeKqzIQqwhFhY0WxGFKeGnkteYmx88v59jOUGhAU4DfBLQk1i+pyP7tFjLCeu0x8A7GQFUUSPI/6LmpdA/KtjvAtlHUOCdU3URdNA7bA+A6yoPR+w+18nV/flFLXEkkEk9oBhul/coQ72+LbQTMoIp7fvmWePy+O2ZJjcMH7YfbBjHkMqhbEOri6e1YodorQApO19kq8vmSCw5HudrYLZnt2pYwoF20e8hKvEsJ43jy9qFRqUKzQ7Yj8t9ePuoa+MVlB7sD032T6T6hGmbeRQMEybT+TbCYmZYbCLatsCcNx0IO7ZLW2y7wVMNFLYLMbLDvc8/SuWQ/904g0q5ZBb/gKPUR05pzgn4VTBG1Dy5eote1saQMzBy+turNYkN5CSmHRE7o0r6ZHhQITDDdwPY+RUPD8hxrydIDzic8NTtnkblyc9hyVKdOYdKei5BuqpWS9J+srMzoSSVAWYbnQeldFsYWHwdaDthTfEkT+3nwSMq8MGsPK5IOxVxAcCZTNwhXRy46hFj0fsQMVQ7RRoxdEdVHzFyF8dXh/b0GfEfH5harP72+PysfVo/P2teV5+/jOcDvrSrAwpUML4+cN/YV03P3VH5JyjThxYLXBckvblFhkAvzFMibUF9MGczWjayLEQ8X41fJOelr7RgYSQxPoUrmRVykhsWfM1ki5CGUP7Ck/Um5w/VF/f3FMrhETJ1vXAaSixRcTJ5sQUmD8gOYE7nDTK65VdRLEijQX3GBCoos9gtsIgnPGs0rhoVyDIDSIgZV+GlifAg4ZwzNz01jIGcDZUSkXrwEno+AKAjMLP7SEfi9EoKjKp+ffH+JDbBNzNXVP9qAAr1K+tDdF5SOzXeUZ+4wFVfzLSY7aBsl+YpIRQaFOSWDwkI8fmU+cPDElgeYOG6onPxWFpQdjqBrR2aNQjAT3pxaWmOsefG8OmFObTbhXXdBpSoDUdnsL3kyO5jBkLpzFLvUJjYiBFF2BUEO2MU+LpX7/gZkItzIEfXCoGEbFz7CJi6RqG+JbW6HrJIQkALUIeC13yjrQ2sLqHCjrMxEtGskcienq0PMeXldNCrCoo8e5xhxeYCS8nPSfR8Yf7EBdX+J0zapobdDGybSNmFrTmOJ3ZWe8BtE534EhTCs/75/DjEoGaplItmqfSk7o719ejCEpeHCOaPFohRtDuNuDUtyZLjkrmtnPZq7Rj65CXZ0PNHOMlshhr7pBpmR2mXH1VYhpa/Gg1rsRy50lfxJ8m0kY09i+2soI8+ui6/m/k0Tocb4HobLHwYF39At8Toln5QF/d3qQuOOpvb6aGhMyZ5fdqawyI759g5PlmiLPMlR5wLhvVYfqf133m1GYb5g7WGtNWyNFGG5uNCy+TZsrcNMuH8D0a+Y7jVvUf2O4fvAG5nhLmXzWmogCxDkNyboAUvMH8TtPddGSuUkX+gHov857thztxxVD5GEj19p7dLiXfgYnYL8JPe0Ltcet6VbZpgMkVGKm/Z0xu51Uv+pqGbP2ldD1qUkv91AVty+7kBcH1bPBps2c+aZ91rtjd1ZXkDZ9zG01XHqvL56UTOLZOVb6Hs+BpWvrtI/sQV/jVkBE8awr9GZvGkVfAtZyhPGfjvmc43qdalV98b76y9x1X4nJ3tNgENVfCIWC69vtm9l/oaUFtyHcHF7H/Sj7+e0YwGlPWk4HxgxY1v2u2rxknyMV+7raWYPJs8+z8FFeia")))
|
||||
NEXT_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNrNVE1v2zAMvedXCFqAtlscb70tQLGtXTEUGFYgHXqJM0OxmUSJLCmS7DYr8t9HubaXOOnHYYfdLPLp6fGR9EOHEFqAsVxJOiCn/Q89H1FmYvH4gN94Wq1WAUYwQJdczqxRahlWwc9xfD08j28vhzdX1z/imOKVTcmxUNscOp8Ibudxyux8ophJ4wUzTRoBcA9J7pSP1Q+GC1Yw2qsR1oH2jKMqgKFkDslS5a4BkYZwW3nIpXVMiLgkHOxgEFVnpUoBs87k0NtF+Ez81yZ6ekq3AJvme/OEDpPL/Wcly/xz9BYMn66JmwPhWZY7NhFADAhgFkj9aEtQorKMydRfr5C1vLOIdh8uroYX3y/jn1++vSk2EY3k1CjpQKYNqnvsayKBJhEyrHJu4PioH2qWLNkM+gur5NFJv0JH9CSSGStAbt3PCkmCcxKsyByEHkDBRM4cUn6Fe23AljBt1AISVxNhcqpMAjcuxa4hqQPrUEG3rS+i5MzHW8X5UuorO3qewf/DTp3nXKRlo3BgHE8IVqqMe6Y5iTLgPSUgfVsjqaXOSDVwJAiw7t8gA6GS5ZQ3ebfWUI52dRZcujqF1VefE69m8CjhFWU2y4CN88sXp6CFWtf3xp365uP63imznAp1d2CHK5O3t7da9tHza1iuEdM6LqXv+4wWOGzcXgJTE8MkWnIo5zd4JtFob3jYfxvSFmDTa7M5NnuCSUmx3ilkKxcWB8hr5w653z69PH4v/SabCcM1uHcHhJZel6sRGJhx68w6SAykIB1nwu6qH/f+V/9p+KsYvQ8+jt9FUb/1cRwML5rDyafu/nuvacD+2Hc2nT/snNGl")))
|
||||
@@ -89,8 +89,8 @@ def load_manifest(path, setup_version):
|
||||
allowed += ("java_version",)
|
||||
mapping(data, allowed, "manifest")
|
||||
version = data.get("orb_version")
|
||||
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+\.\d+|dev:[A-Za-z0-9_-]+)", version):
|
||||
raise ManifestError("orb_version must be an explicit stable or development version")
|
||||
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+(?:\.\d+)?|dev:[A-Za-z0-9_-]+)", version):
|
||||
raise ManifestError("orb_version must be a quoted minor track, exact stable pin, or development version")
|
||||
if version != setup_version:
|
||||
raise ManifestError("Manifest orb_version must match the setup job orb_version")
|
||||
default_branch = data.get("default_branch")
|
||||
@@ -225,7 +225,7 @@ def qbit_template(config, branch, tag):
|
||||
kind = "hotfix"
|
||||
if kind != "test" and not config["publish"][kind]:
|
||||
kind = "test"
|
||||
maintenance = kind != "test" and proven_ci_maintenance(branch, tag)
|
||||
maintenance = proven_ci_maintenance(branch, tag)
|
||||
if maintenance:
|
||||
kind = "test"
|
||||
workflow = "test_only" if kind == "test" else "publish_" + kind
|
||||
@@ -342,7 +342,10 @@ def frontend_template(config, branch, tag):
|
||||
kind = "release"
|
||||
if kind != "test" and not config["publish"][kind]:
|
||||
kind = "test"
|
||||
if kind == "test" and proven_docs_only(config, branch, tag):
|
||||
maintenance = proven_ci_maintenance(branch, tag)
|
||||
if maintenance:
|
||||
kind = "test"
|
||||
if kind == "test" and not maintenance and proven_docs_only(config, branch, tag):
|
||||
result = qbit_template(config, branch, tag)
|
||||
result["workflows"]["test_only"]["jobs"][0]["docs_only"]["name"] = (
|
||||
"qqq-orb/mvn_frontend_test_only" if material else "qqq-orb/node_test_only")
|
||||
|
||||
@@ -649,7 +649,9 @@
|
||||
"qqq-orb/mvn_publish": {
|
||||
"requires": [
|
||||
"sample_bootstrap_source",
|
||||
"snapshot_esb_broker_conformance"
|
||||
"snapshot_esb_broker_conformance",
|
||||
"snapshot_sample_coverage_gate",
|
||||
"qqq-orb/static_analysis"
|
||||
],
|
||||
"pre-steps": [
|
||||
"configure-package-install"
|
||||
|
||||
@@ -109,8 +109,8 @@ def validate(config_raw, manifest_raw, setup_version):
|
||||
if not isinstance(manifest, dict):
|
||||
raise StandardError("Manifest must be a mapping")
|
||||
version = manifest.get("orb_version")
|
||||
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+\.\d+|dev:[A-Za-z0-9_-]+)", version):
|
||||
raise StandardError("Manifest must pin an explicit orb version")
|
||||
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+(?:\.\d+)?|dev:[A-Za-z0-9_-]+)", version):
|
||||
raise StandardError("Manifest must select a quoted minor track, exact stable pin, or development version")
|
||||
if version != setup_version:
|
||||
raise StandardError("Manifest and running setup orb versions differ")
|
||||
publishing = manifest.get("publish", {})
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
import importlib.util
|
||||
import json
|
||||
import io
|
||||
import pathlib
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
@@ -25,6 +27,54 @@ class Snapshot:
|
||||
return self.required
|
||||
|
||||
|
||||
class InventoryTests(unittest.TestCase):
|
||||
def repo(self, name, archived=False):
|
||||
return {'name': name, 'full_name': 'QRun-IO/' + name,
|
||||
'archived': archived, 'default_branch': 'main'}
|
||||
|
||||
def test_all_pages_include_new_private_and_archived_repositories(self):
|
||||
new = dict(self.repo('new-product'), private=True)
|
||||
pages = [[self.repo('qbit-existing')], [new, self.repo('retired', True)]]
|
||||
with patch.object(audit, 'command', return_value=json.dumps(pages)) as command:
|
||||
entries = audit.discover_inventory('QRun-IO')
|
||||
self.assertEqual([e['name'] for e in entries], ['new-product', 'qbit-existing', 'retired'])
|
||||
self.assertTrue(entries[-1]['isArchived'])
|
||||
self.assertEqual(command.call_args.args[0], ['gh', 'api', '--method', 'GET',
|
||||
'--paginate', '--slurp', 'orgs/QRun-IO/repos?type=all&per_page=100'])
|
||||
|
||||
def test_empty_duplicate_foreign_and_malformed_inventories_fail_closed(self):
|
||||
repo = self.repo('qbit-example')
|
||||
for pages in ([], [[]], {}, [repo], [[repo], [repo]],
|
||||
[[dict(repo, full_name='Other/qbit-example')]],
|
||||
[[dict(repo, archived='false')]], [[dict(repo, default_branch=None)]]):
|
||||
with self.subTest(pages=pages), patch.object(audit, 'command', return_value=json.dumps(pages)):
|
||||
with self.assertRaises(audit.ReadError):
|
||||
audit.discover_inventory('QRun-IO')
|
||||
with patch.object(audit, 'command') as command:
|
||||
with self.assertRaises(audit.ReadError):
|
||||
audit.discover_inventory('QRun-IO/../../other')
|
||||
command.assert_not_called()
|
||||
|
||||
def test_live_cli_count_is_dynamic_but_explicit_expectation_is_enforced(self):
|
||||
for expected, status in ((None, 0), ('1', 0), ('68', 1)):
|
||||
args = ['audit', '--organization', 'QRun-IO', '--live', '--orb-version', '0.8']
|
||||
if expected:
|
||||
args += ['--expected-active', expected]
|
||||
report = {'active_count': 1, 'unique_repository_count': 1,
|
||||
'inventory_count': 1, 'results': [{'status': 'pass'}]}
|
||||
output = io.StringIO()
|
||||
with patch.object(audit.sys, 'argv', args), patch.object(audit.sys, 'stdout', output), \
|
||||
patch.object(audit, 'discover_inventory', return_value=[ENTRY]), \
|
||||
patch.object(audit, 'run', return_value=report):
|
||||
self.assertEqual(audit.main(), status)
|
||||
self.assertEqual(json.loads(output.getvalue())['inventory_source'], 'organization-api')
|
||||
|
||||
def test_discovery_api_failure_never_falls_back_to_old_inventory(self):
|
||||
with patch.object(audit, 'command', side_effect=audit.ReadError('unavailable')):
|
||||
with self.assertRaises(audit.ReadError):
|
||||
audit.discover_inventory('QRun-IO')
|
||||
|
||||
|
||||
class AuditTests(unittest.TestCase):
|
||||
def test_all_scanner_and_release_requirements_follow_branch_profile(self):
|
||||
entry = dict(ENTRY, name='qqq-all', nameWithOwner='QRun-IO/qqq-all')
|
||||
|
||||
@@ -0,0 +1,69 @@
|
||||
#!/bin/bash
|
||||
# Exercise real browser scripts without sudo, package installation or network.
|
||||
set -euo pipefail
|
||||
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
|
||||
scratch=$(mktemp -d)
|
||||
trap 'rm -rf "$scratch"' EXIT
|
||||
mkdir "$scratch/bin"
|
||||
export INSTALL_LOG="$scratch/install.log"
|
||||
cat > "$scratch/bin/sudo" <<'STUB'
|
||||
#!/bin/bash
|
||||
printf '%s\n' "$*" >> "$INSTALL_LOG"
|
||||
if [[ "$*" == *'apt-get update'* ]]; then
|
||||
[[ "$*" == *'Dir::Etc::sourceparts=-'* ]] || { echo 'unrelated image PPA: 503' >&2; exit 100; }
|
||||
[[ "$*" == *'APT::Get::List-Cleanup=0'* ]] || exit 91
|
||||
[[ ${REQUIRED_REPO_FAIL:-0} == 0 ]] || exit 100
|
||||
fi
|
||||
if [[ "$1" == tee ]]; then cat >> "$INSTALL_LOG"; fi
|
||||
if [[ "$1" == gpg ]]; then cat >/dev/null; fi
|
||||
STUB
|
||||
chmod +x "$scratch/bin/sudo"
|
||||
export PATH="$scratch/bin:$PATH"
|
||||
bash "$root/src/scripts/frontend_test_dependencies.sh"
|
||||
echo 'PASS: browser dependencies ignore unavailable image PPAs'
|
||||
# Model Chrome presence without depending on the host's installed browsers.
|
||||
export CHROME_MARKER="$scratch/chrome-installed"
|
||||
export BASH_ENV="$scratch/shell-env"
|
||||
cat > "$BASH_ENV" <<'STUB'
|
||||
command() {
|
||||
if [[ "$*" == '-v google-chrome' ]]; then [[ -f "$CHROME_MARKER" ]]; else builtin command "$@"; fi
|
||||
}
|
||||
google-chrome() { [[ -f "$CHROME_MARKER" ]] && echo 'Google Chrome test'; }
|
||||
STUB
|
||||
cat > "$scratch/bin/curl" <<'STUB'
|
||||
#!/bin/bash
|
||||
printf '%s\n' "$*" >> "$INSTALL_LOG"
|
||||
[[ ${KEY_FAIL:-0} == 0 ]] || exit 22
|
||||
printf 'signing-key\n'
|
||||
STUB
|
||||
cat > "$scratch/bin/dpkg" <<'STUB'
|
||||
#!/bin/bash
|
||||
echo amd64
|
||||
STUB
|
||||
# Successful installation makes Chrome available to the final version check.
|
||||
# shellcheck disable=SC2016
|
||||
sed -i '/if \[\[ "$1" == tee/i if [[ "$*" == *"apt-get install -y google-chrome-stable"* ]]; then touch "$CHROME_MARKER"; fi' "$scratch/bin/sudo"
|
||||
chmod +x "$scratch/bin/"*
|
||||
: > "$INSTALL_LOG"
|
||||
bash "$root/src/scripts/install_chrome.sh" >/dev/null
|
||||
grep -q 'Dir::Etc::sourcelist=/etc/apt/sources.list.d/qqq-google-chrome.list' "$INSTALL_LOG"
|
||||
grep -q 'gpg --batch --yes --dearmor -o /usr/share/keyrings/qqq-google-chrome.gpg' "$INSTALL_LOG"
|
||||
grep -q 'signed-by=/usr/share/keyrings/qqq-google-chrome.gpg] https://dl.google.com/linux/chrome/deb/' "$INSTALL_LOG"
|
||||
grep -q 'apt-get install -y google-chrome-stable' "$INSTALL_LOG"
|
||||
: > "$INSTALL_LOG"
|
||||
bash "$root/src/scripts/install_chrome.sh" >/dev/null
|
||||
[[ ! -s "$INSTALL_LOG" ]] # Existing Chrome needs no network/package work.
|
||||
rm "$CHROME_MARKER"
|
||||
for script in install_chrome frontend_test_dependencies; do
|
||||
: > "$INSTALL_LOG"
|
||||
if REQUIRED_REPO_FAIL=1 bash "$root/src/scripts/$script.sh" >/dev/null 2>&1; then
|
||||
echo "Required repository outage accepted: $script" >&2; exit 1
|
||||
fi
|
||||
if grep -q 'apt-get install' "$INSTALL_LOG"; then exit 1; fi
|
||||
done
|
||||
: > "$INSTALL_LOG"
|
||||
if KEY_FAIL=1 bash "$root/src/scripts/install_chrome.sh" >/dev/null 2>&1; then
|
||||
echo 'Signing key failure accepted' >&2; exit 1
|
||||
fi
|
||||
if grep -q 'apt-get' "$INSTALL_LOG"; then exit 1; fi
|
||||
echo 'PASS: Chrome ignores unrelated PPAs; required repositories and signing key fail closed; existing browser reused'
|
||||
@@ -34,6 +34,49 @@ class MaintenanceTests(unittest.TestCase):
|
||||
self.assertEqual(job['filters']['branches'], {'ignore': 'automation/qbit-rc-pending'})
|
||||
self.assertNotIn('branch_type', job)
|
||||
|
||||
def test_nonpublishing_branches_verify_maintenance_but_keep_application_policy(self):
|
||||
manifest = {'profile': 'qbit', 'orb_version': '0.8', 'default_branch': 'main',
|
||||
'release_on_main': False, 'test_branch_policy': 'feature-no-hotfix',
|
||||
'publish': {'snapshot': False, 'release_candidate': False, 'hotfix': False}}
|
||||
with mock.patch.object(Path, 'read_text', return_value=yaml.safe_dump(manifest)):
|
||||
config = pipeline.load_manifest('unused', '0.8')
|
||||
maintenance_paths = self.paths
|
||||
with mock.patch.object(pipeline, 'git', side_effect=self.git_result):
|
||||
for branch in ('main', 'develop', 'release/0.1', 'hotfix/1.2.3'):
|
||||
for maintenance in (True, False):
|
||||
with self.subTest(branch=branch, maintenance=maintenance):
|
||||
self.paths = maintenance_paths if maintenance else b'src/Main.java\0'
|
||||
result = pipeline.qbit_template(config, branch, '')
|
||||
job = result['workflows']['test_only']['jobs'][0]['qqq-orb/mvn_test_only']
|
||||
ignored = job['filters']['branches']['ignore']
|
||||
if maintenance:
|
||||
self.assertEqual(ignored, 'automation/qbit-rc-pending')
|
||||
else:
|
||||
self.assertEqual(ignored, ['automation/qbit-rc-pending',
|
||||
'/(develop|main|release/.*|hotfix/.*|integration.*)/'])
|
||||
self.assertNotIn('branch_type', job)
|
||||
|
||||
def test_frontend_maintenance_verifies_without_publishing(self):
|
||||
for profile, prefix in (('frontend-core', 'node_'), ('frontend-material', 'mvn_frontend_')):
|
||||
manifest = {'profile': profile, 'orb_version': '0.8', 'default_branch': 'develop'}
|
||||
with mock.patch.object(Path, 'read_text', return_value=yaml.safe_dump(manifest)):
|
||||
config = pipeline.load_manifest('unused', '0.8')
|
||||
for branch in ('main', 'develop', 'release/1.0', 'hotfix/1.0.1'):
|
||||
with self.subTest(profile=profile, branch=branch), mock.patch.object(pipeline, 'git', side_effect=self.git_result):
|
||||
result = pipeline.frontend_template(config, branch, '')
|
||||
job = result['workflows']['test_only']['jobs'][0]['qqq-orb/' + prefix + 'test_only']
|
||||
self.assertNotIn('branch_type', job)
|
||||
self.assertEqual(job['filters']['branches'], {'ignore': 'automation/qbit-rc-pending'})
|
||||
self.paths = b'package.json\0'
|
||||
with mock.patch.object(pipeline, 'git', side_effect=self.git_result):
|
||||
result = pipeline.frontend_template(config, 'develop', '')
|
||||
self.assertIn('qqq-orb/' + prefix + 'publish', result['workflows']['publish_snapshot']['jobs'][0])
|
||||
self.paths = b'.qqq-ci.yml\0'
|
||||
with mock.patch.object(pipeline, 'git', return_value=b'verified-head\n'):
|
||||
result = pipeline.frontend_template(config, 'main', 'v1.2.3')
|
||||
job = result['workflows']['publish_release']['jobs'][0]['qqq-orb/' + prefix + 'publish']
|
||||
self.assertEqual(job['branch_type'], 'release')
|
||||
|
||||
def test_application_changes_and_unknown_files_publish_normally(self):
|
||||
for paths in (b'pom.xml\0', b'.qqq-ci.yml\0src/Main.java\0', b'.github/workflows/custom-release.yml\0', b'', None):
|
||||
self.paths = paths
|
||||
|
||||
@@ -0,0 +1,66 @@
|
||||
"""Discovery covers all current branches and never turns an unknown into a target."""
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'tools'))
|
||||
import discover_enrollment as discovery
|
||||
|
||||
|
||||
def entry(name, archived=False):
|
||||
return {'nameWithOwner': 'QRun-IO/' + name, 'isArchived': archived}
|
||||
|
||||
|
||||
class DiscoveryTests(unittest.TestCase):
|
||||
def test_paginates_branches_and_ignores_historical_releases(self):
|
||||
requests = []
|
||||
|
||||
def request(path):
|
||||
requests.append(path)
|
||||
if path.endswith('page=1'):
|
||||
return [{'name': 'release/' + str(i)} for i in range(99)] + [{'name': 'main'}]
|
||||
return [{'name': 'develop'}]
|
||||
|
||||
inspect = mock.Mock(side_effect=lambda target, version: {**target, 'version': version})
|
||||
plans, findings = discovery.discover('QRun-IO', '0.8', inspect, request, [entry('example')])
|
||||
self.assertEqual(['develop', 'main'], [p['branch'] for p in plans])
|
||||
self.assertEqual(2, len(requests))
|
||||
self.assertEqual([], findings)
|
||||
|
||||
def test_unknown_branch_and_profile_stay_unresolved_while_other_targets_progress(self):
|
||||
def inspect(target, version):
|
||||
if target['repository'].endswith('/custom'):
|
||||
raise ValueError('Unknown orchestration')
|
||||
return target
|
||||
|
||||
def request(path):
|
||||
return [{'name': 'feature/new'}] if '/new/' in path else [{'name': 'main'}]
|
||||
|
||||
plans, findings = discovery.discover('QRun-IO', '0.8', inspect, request,
|
||||
[entry('custom'), entry('new'), entry('good')])
|
||||
self.assertEqual([{'repository': 'QRun-IO/good', 'branch': 'main'}], plans)
|
||||
self.assertEqual(2, len(findings))
|
||||
self.assertTrue(all(f['state'] == 'unresolved' for f in findings))
|
||||
|
||||
def test_archived_and_owner_repositories_are_never_modified(self):
|
||||
inspect, request = mock.Mock(), mock.Mock()
|
||||
plans, findings = discovery.discover('QRun-IO', '0.8', inspect, request,
|
||||
[entry('old', True), entry('qqq-orb')])
|
||||
self.assertEqual([], plans)
|
||||
self.assertEqual(['archived', 'owner-bootstrap'], [f['state'] for f in findings])
|
||||
inspect.assert_not_called()
|
||||
request.assert_not_called()
|
||||
|
||||
def test_api_errors_and_duplicate_branches_never_become_empty_success(self):
|
||||
for result in (ValueError('GitHub read unavailable'), [{'name': 'main'}, {'name': 'main'}]):
|
||||
request = mock.Mock(side_effect=result) if isinstance(result, Exception) else mock.Mock(return_value=result)
|
||||
inspect = mock.Mock()
|
||||
plans, findings = discovery.discover('QRun-IO', '0.8', inspect, request, [entry('example')])
|
||||
self.assertEqual([], plans)
|
||||
self.assertEqual('unresolved', findings[0]['state'])
|
||||
inspect.assert_not_called()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,173 @@
|
||||
"""Shared enforcement only adds proven gates and preserves unrelated policy."""
|
||||
import copy
|
||||
import base64
|
||||
import importlib.util
|
||||
from pathlib import Path
|
||||
import unittest
|
||||
|
||||
import yaml
|
||||
|
||||
spec = importlib.util.spec_from_file_location('enforcement', Path(__file__).resolve().parents[1] / 'tools/enforce_orb_checks.py')
|
||||
enforcement = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(enforcement)
|
||||
|
||||
|
||||
def runs():
|
||||
return [{'id': index, 'name': name, 'app': {'id': 18001}, 'conclusion': 'success'}
|
||||
for index, name in enumerate(('setup', 'verify'), 1)]
|
||||
|
||||
|
||||
class EnforcementTests(unittest.TestCase):
|
||||
def test_preserves_security_gate_and_binds_orb_checks(self):
|
||||
before = {'required_status_checks': {'strict': False, 'contexts': ['security', 'setup'],
|
||||
'checks': [{'context': 'security', 'app_id': 15368}, {'context': 'setup', 'app_id': -1}]}}
|
||||
original = copy.deepcopy(before)
|
||||
result = enforcement.required_checks(before, runs())
|
||||
self.assertEqual(original, before)
|
||||
self.assertEqual(result, {'strict': True, 'checks': [{'context': 'security', 'app_id': 15368}, *enforcement.DESIRED]})
|
||||
|
||||
def test_wrong_app_failed_latest_and_missing_checks_reject(self):
|
||||
cases = [[], [runs()[0]], [dict(run, app={'id': 999}) for run in runs()],
|
||||
runs() + [dict(runs()[1], id=3, conclusion='failure')]]
|
||||
for candidate in cases:
|
||||
with self.subTest(candidate=candidate), self.assertRaises(ValueError):
|
||||
enforcement.required_checks(None, candidate)
|
||||
with self.assertRaises(ValueError):
|
||||
enforcement.required_checks({'required_status_checks': {'checks': [{'context': 'verify', 'app_id': 999}]}}, runs())
|
||||
|
||||
def test_existing_unrestricted_contexts_are_retained(self):
|
||||
result = enforcement.required_checks({'required_status_checks': {'contexts': ['legacy-build']}}, runs())
|
||||
self.assertIn({'context': 'legacy-build', 'app_id': -1}, result['checks'])
|
||||
|
||||
def fixture(self, before):
|
||||
state = {'before': copy.deepcopy(before), 'head': 'a' * 40, 'writes': []}
|
||||
def request(path, method='GET', payload=None):
|
||||
if method != 'GET':
|
||||
state['writes'].append((method, path, payload))
|
||||
if method == 'PUT':
|
||||
state['before'] = {**payload, 'enforce_admins': {'enabled': True}}
|
||||
elif method == 'PATCH':
|
||||
state['before']['required_status_checks'] = copy.deepcopy(payload)
|
||||
elif method == 'POST':
|
||||
state['before']['enforce_admins'] = {'enabled': True}
|
||||
return None
|
||||
if '/rules/' in path:
|
||||
return []
|
||||
if path.endswith('/protection'):
|
||||
return copy.deepcopy(state['before'])
|
||||
return {'commit': {'sha': state['head']}}
|
||||
plan = {'repository': 'QRun-IO/example', 'branch': 'develop', 'head': 'a' * 40,
|
||||
'before': copy.deepcopy(before), 'required_status_checks': enforcement.required_checks(before, runs()), 'changed': True}
|
||||
return plan, request, state
|
||||
|
||||
def test_narrow_updates_preserve_reviews_signatures_and_restrictions(self):
|
||||
before = {'required_status_checks': {'strict': False, 'checks': []}, 'enforce_admins': {'enabled': False},
|
||||
'required_signatures': {'enabled': True}, 'required_linear_history': {'enabled': True},
|
||||
'required_pull_request_reviews': {'required_approving_review_count': 2},
|
||||
'restrictions': {'users': [{'login': 'owner'}]}}
|
||||
plan, request, state = self.fixture(before)
|
||||
enforcement.enforce(plan, request)
|
||||
self.assertEqual(['PATCH', 'POST'], [write[0] for write in state['writes']])
|
||||
for key in ('required_signatures', 'required_linear_history', 'required_pull_request_reviews', 'restrictions'):
|
||||
self.assertEqual(before[key], state['before'][key])
|
||||
|
||||
def test_new_protection_and_idempotent_noop(self):
|
||||
plan, request, state = self.fixture(None)
|
||||
enforcement.enforce(plan, request)
|
||||
self.assertEqual('PUT', state['writes'][0][0])
|
||||
self.assertIsNone(state['before']['required_pull_request_reviews'])
|
||||
plan['before'] = copy.deepcopy(state['before'])
|
||||
plan['changed'] = False
|
||||
state['writes'] = []
|
||||
enforcement.enforce(plan, request)
|
||||
self.assertEqual([], state['writes'])
|
||||
|
||||
def test_head_or_protection_drift_never_mutates(self):
|
||||
for field, value in [('head', 'b' * 40), ('before', {'unexpected': True})]:
|
||||
plan, request, state = self.fixture(None)
|
||||
state[field] = value
|
||||
with self.assertRaises(ValueError):
|
||||
enforcement.enforce(plan, request)
|
||||
self.assertEqual([], state['writes'])
|
||||
|
||||
def ruleset_fixture(self):
|
||||
parameters = {'strict_required_status_checks_policy': False,
|
||||
'required_status_checks': [{'context': c['context'], 'integration_id': c['app_id']}
|
||||
for c in enforcement.DESIRED] + [{'context': 'scanner', 'integration_id': 15368}]}
|
||||
record = {'id': 123, 'enforcement': 'active', 'target': 'branch', 'source_type': 'Repository',
|
||||
'source': 'QRun-IO/example', 'bypass_actors': [],
|
||||
'rules': [{'type': 'required_status_checks', 'parameters': parameters}]}
|
||||
effective = [{'type': 'required_status_checks', 'ruleset_source_type': 'Repository',
|
||||
'ruleset_source': 'QRun-IO/example', 'ruleset_id': 123, 'parameters': copy.deepcopy(parameters)}]
|
||||
return record, effective
|
||||
|
||||
def test_ruleset_verification_preserves_freshness_scanner_and_never_writes(self):
|
||||
record, effective = self.ruleset_fixture()
|
||||
original = copy.deepcopy(record)
|
||||
|
||||
def request(path, method='GET', payload=None):
|
||||
self.assertEqual('GET', method)
|
||||
if path.endswith('/rulesets/123'):
|
||||
return copy.deepcopy(record)
|
||||
if '/rules/branches/' in path:
|
||||
return copy.deepcopy(effective)
|
||||
if path.endswith('/protection'):
|
||||
return None
|
||||
if '/check-runs?' in path:
|
||||
return {'check_runs': runs()}
|
||||
if '/contents/' in path:
|
||||
data = ({'profile': 'qbit', 'orb_version': '0.8', 'default_branch': 'develop'}
|
||||
if '/.qqq-ci.yml?' in path else enforcement.standard.canonical_caller('qbit', '0.8'))
|
||||
return {'encoding': 'base64', 'type': 'file', 'content': base64.b64encode(yaml.safe_dump(data).encode()).decode()}
|
||||
if '/branches/' in path:
|
||||
return {'commit': {'sha': 'a' * 40}}
|
||||
return {'archived': False}
|
||||
|
||||
plan = enforcement.inspect('QRun-IO/example', 'develop', '0.8', request)
|
||||
self.assertFalse(plan['changed'])
|
||||
self.assertFalse(plan['required_status_checks']['strict'])
|
||||
self.assertEqual('rulesets', plan['protection_mode'])
|
||||
enforcement.enforce(plan, request)
|
||||
self.assertEqual(original, record)
|
||||
record['bypass_actors'] = [{'actor_type': 'RepositoryRole', 'actor_id': 5}]
|
||||
with self.assertRaises(ValueError):
|
||||
enforcement.enforce(plan, request)
|
||||
|
||||
def test_inactive_bypass_unbound_and_missing_ruleset_gates_reject(self):
|
||||
record, effective = self.ruleset_fixture()
|
||||
cases = [dict(record, enforcement='evaluate'), dict(record, bypass_actors=[{'actor_id': 1}]),
|
||||
dict(record, rules=[])]
|
||||
unbound = copy.deepcopy(record)
|
||||
unbound['rules'][0]['parameters']['required_status_checks'][1]['integration_id'] = None
|
||||
cases.append(unbound)
|
||||
for candidate in cases:
|
||||
with self.subTest(candidate=candidate), self.assertRaises(ValueError):
|
||||
enforcement.verified_rulesets('QRun-IO/example', effective, lambda path: candidate)
|
||||
|
||||
def test_unsupported_ruleset_sources_are_not_inferred_as_protected(self):
|
||||
record, effective = self.ruleset_fixture()
|
||||
effective[0]['ruleset_source_type'] = 'Organization'
|
||||
with self.assertRaises(ValueError):
|
||||
enforcement.verified_rulesets('QRun-IO/example', effective, lambda path: record)
|
||||
|
||||
def test_empty_push_allowlist_does_not_gain_admin_enforcement(self):
|
||||
def request(path, method='GET', payload=None):
|
||||
self.assertEqual('GET', method)
|
||||
if '/rules/branches/' in path:
|
||||
return []
|
||||
if path.endswith('/protection'):
|
||||
return {'restrictions': {'users': [], 'teams': [], 'apps': []}, 'enforce_admins': {'enabled': False}}
|
||||
if '/contents/' in path:
|
||||
data = ({'profile': 'core', 'orb_version': '0.8', 'default_branch': 'develop'}
|
||||
if '/.qqq-ci.yml?' in path else enforcement.standard.canonical_caller('core', '0.8'))
|
||||
return {'encoding': 'base64', 'type': 'file', 'content': base64.b64encode(yaml.safe_dump(data).encode()).decode()}
|
||||
if '/branches/' in path:
|
||||
return {'commit': {'sha': 'a' * 40}}
|
||||
return {'archived': False}
|
||||
|
||||
with self.assertRaisesRegex(ValueError, 'Empty push allowlist'):
|
||||
enforcement.inspect('QRun-IO/example', 'main', '0.8', request)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,105 @@
|
||||
"""Batch enrollment preserves policy and rejects unsafe or unknown declarations."""
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
import yaml
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'tools'))
|
||||
import enroll_consumers as enrollment
|
||||
|
||||
|
||||
class EnrollmentTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.temp.cleanup)
|
||||
self.root = Path(self.temp.name)
|
||||
|
||||
def legacy(self, name):
|
||||
root = self.root / name
|
||||
(root / '.circleci').mkdir(parents=True)
|
||||
config = {'version': 2.1, 'orbs': {'qqq-orb': 'kingsrook/qqq-orb@0.5.0'},
|
||||
'workflows': {'verify': {'jobs': [{'qqq-orb/mvn_test_only': {}}]}}}
|
||||
(root / '.circleci/config.yml').write_text(yaml.safe_dump(config), encoding='utf-8')
|
||||
return {'checkout': str(root), 'default_branch': 'main'}
|
||||
|
||||
def canonical(self, profile):
|
||||
target = self.legacy(profile)
|
||||
root = Path(target['checkout'])
|
||||
data = {'profile': profile, 'orb_version': '0.7.4', 'default_branch': 'main'}
|
||||
(root / '.qqq-ci.yml').write_text(yaml.safe_dump(data), encoding='utf-8')
|
||||
(root / '.circleci/config.yml').write_text(yaml.safe_dump(
|
||||
enrollment.rollout.standard.canonical_caller(profile, '0.7.4')), encoding='utf-8')
|
||||
return target
|
||||
|
||||
def test_mixed_fleet_preview_write_and_idempotence(self):
|
||||
legacy = self.legacy('legacy')
|
||||
targets = [legacy] + [self.canonical(p) for p in enrollment.rollout.standard.PROFILE_PARAMETERS]
|
||||
preview = enrollment.enroll(targets, '0.8')
|
||||
self.assertTrue(all(item['changed'] for item in preview))
|
||||
self.assertFalse((Path(legacy['checkout']) / '.qqq-ci.yml').exists())
|
||||
result = enrollment.enroll(targets, '0.8', write=True)
|
||||
self.assertEqual(1, sum(item['enrollment'] for item in result))
|
||||
data = yaml.safe_load((Path(legacy['checkout']) / '.qqq-ci.yml').read_text())
|
||||
self.assertFalse(any(data['publish'].values()))
|
||||
self.assertEqual('all', data['test_branch_policy'])
|
||||
self.assertTrue(all(not item['changed'] for item in enrollment.enroll(targets, '0.8', write=True)))
|
||||
|
||||
def test_unknown_orchestration_rejects_entire_batch(self):
|
||||
good, bad = self.legacy('good'), self.legacy('bad')
|
||||
path = Path(bad['checkout']) / '.circleci/config.yml'
|
||||
path.write_text(path.read_text() + '\njobs: {custom: {steps: [checkout]}}\n')
|
||||
original = (Path(good['checkout']) / '.circleci/config.yml').read_bytes()
|
||||
with self.assertRaises(ValueError):
|
||||
enrollment.enroll([good, bad], '0.8', write=True)
|
||||
self.assertEqual(original, (Path(good['checkout']) / '.circleci/config.yml').read_bytes())
|
||||
self.assertFalse((Path(good['checkout']) / '.qqq-ci.yml').exists())
|
||||
|
||||
def test_duplicate_targets_and_wrong_default_reject_before_write(self):
|
||||
target = self.canonical('core')
|
||||
for targets in ([target, target], [{**target, 'default_branch': 'develop'}]):
|
||||
with self.assertRaises(ValueError):
|
||||
enrollment.enroll(targets, '0.8', write=True)
|
||||
self.assertEqual('0.7.4', yaml.safe_load((Path(target['checkout']) / '.qqq-ci.yml').read_text())['orb_version'])
|
||||
|
||||
def test_invalid_manifest_and_unstable_versions_rejected(self):
|
||||
target = self.canonical('qbit')
|
||||
path = Path(target['checkout']) / '.qqq-ci.yml'
|
||||
path.write_text(path.read_text() + '\nunknown_policy: true\n')
|
||||
for version in ('0.8', 'dev:snapshot', 'latest', 0.8):
|
||||
with self.assertRaises(ValueError):
|
||||
enrollment.enroll([target], version, write=True)
|
||||
|
||||
def test_drift_after_preflight_is_not_overwritten(self):
|
||||
target = self.legacy('drift')
|
||||
path = Path(target['checkout']) / '.circleci/config.yml'
|
||||
original_plan = enrollment.plan
|
||||
|
||||
def changed_plan(*args):
|
||||
result = original_plan(*args)
|
||||
path.write_bytes(b'concurrent edit\n')
|
||||
return result
|
||||
|
||||
with mock.patch.object(enrollment, 'plan', side_effect=changed_plan):
|
||||
with self.assertRaises(ValueError):
|
||||
enrollment.enroll([target], '0.8', write=True)
|
||||
self.assertEqual(b'concurrent edit\n', path.read_bytes())
|
||||
self.assertFalse((Path(target['checkout']) / '.qqq-ci.yml').exists())
|
||||
|
||||
def test_manifest_symlink_is_rejected(self):
|
||||
target = self.legacy('redirect')
|
||||
outside = self.root / 'outside.yml'
|
||||
outside.write_text('untouched\n')
|
||||
try:
|
||||
(Path(target['checkout']) / '.qqq-ci.yml').symlink_to(outside)
|
||||
except OSError as error:
|
||||
self.skipTest('Symlink creation unavailable: ' + str(error))
|
||||
with self.assertRaises(ValueError):
|
||||
enrollment.enroll([target], '0.8', write=True)
|
||||
self.assertEqual('untouched\n', outside.read_text())
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,92 @@
|
||||
"""Automatic completion requires qualified source and preserves normal merge policy."""
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'tools'))
|
||||
import finish_enrollment as finish
|
||||
|
||||
|
||||
class FinishTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.plan = {'repository': 'QRun-IO/example', 'branch': 'main', 'head': 'a' * 40,
|
||||
'profile': 'qbit', 'version': '0.8'}
|
||||
self.result = {'state': 'existing-pr', 'head': 'b' * 40, 'url': 'https://github.com/QRun-IO/example/pull/1'}
|
||||
self.pr = {'number': 1, 'state': 'OPEN', 'headRefOid': 'b' * 40,
|
||||
'baseRefOid': 'a' * 40, 'baseRefName': 'main', 'mergeCommit': None}
|
||||
|
||||
def checks(self):
|
||||
return [{'id': i, 'name': name, 'app': {'id': 18001}, 'status': 'completed', 'conclusion': 'success'}
|
||||
for i, name in enumerate(('setup', 'verify'))]
|
||||
|
||||
def test_all_checks_and_statuses_must_pass_and_gates_must_use_circle_app(self):
|
||||
checks = self.checks()
|
||||
checks[1]['app']['id'] = 123
|
||||
checks.append({'id': 3, 'name': 'scanner', 'app': {'id': 15368}, 'status': 'completed', 'conclusion': 'failure'})
|
||||
with mock.patch.object(finish.remote, 'api', side_effect=[{'check_runs': checks},
|
||||
{'statuses': [{'context': 'legacy-test', 'state': 'pending'}]}]):
|
||||
self.assertEqual(['legacy-test', 'scanner', 'verify'], finish.qualification('QRun-IO/example', 'b' * 40))
|
||||
|
||||
def test_failed_or_missing_checks_never_merge(self):
|
||||
with mock.patch.object(finish.remote, 'verify_submission'), mock.patch.object(finish.remote, 'command', return_value=json.dumps(self.pr)) as command, mock.patch.object(finish.remote, 'api', return_value={'commit': {'sha': self.plan['head']}}), mock.patch.object(finish, 'qualification', return_value=['verify']):
|
||||
self.assertEqual('waiting-checks', finish.merge_ready(self.plan, self.result)['state'])
|
||||
self.assertEqual(1, command.call_count)
|
||||
self.assertEqual(['gh', 'pr', 'view'], command.call_args.args[0][:3])
|
||||
|
||||
def test_stale_base_never_merges_even_when_checks_were_green(self):
|
||||
self.pr['baseRefOid'] = 'c' * 40
|
||||
with mock.patch.object(finish.remote, 'verify_submission'), mock.patch.object(finish.remote, 'command', return_value=json.dumps(self.pr)) as command:
|
||||
self.assertEqual('stale-base', finish.merge_ready(self.plan, self.result)['state'])
|
||||
self.assertEqual(1, command.call_count)
|
||||
|
||||
def test_success_uses_exact_head_normal_merge_and_verifies_result(self):
|
||||
calls = []
|
||||
|
||||
def command(args):
|
||||
calls.append(args)
|
||||
if args[2] == 'merge':
|
||||
return ''
|
||||
if len(calls) == 1:
|
||||
return json.dumps(self.pr)
|
||||
return json.dumps({'state': 'MERGED', 'mergeCommit': {'oid': 'd' * 40}})
|
||||
|
||||
def api(path):
|
||||
if '/branches/' in path:
|
||||
return {'commit': {'sha': self.plan['head']}}
|
||||
return {'commit': {'tree': {'sha': 'e' * 40}, 'verification': {'verified': True}}}
|
||||
|
||||
with mock.patch.object(finish.remote, 'verify_submission'), mock.patch.object(finish.remote, 'command', side_effect=command), mock.patch.object(finish.remote, 'api', side_effect=api), mock.patch.object(finish, 'qualification', return_value=[]):
|
||||
result = finish.merge_ready(self.plan, self.result)
|
||||
self.assertEqual('merged', result['state'])
|
||||
self.assertEqual(['gh', 'pr', 'merge', '1', '--repo', 'QRun-IO/example', '--squash', '--match-head-commit', 'b' * 40], calls[1])
|
||||
|
||||
def test_enforcement_waits_for_current_head_and_uses_shared_preserving_writer(self):
|
||||
with mock.patch.object(finish.enforcement, 'inspect', side_effect=ValueError('Current branch has no successful verify')), mock.patch.object(finish.enforcement, 'enforce') as enforce:
|
||||
self.assertEqual('waiting-enforcement', finish.enforce_current(self.plan)['state'])
|
||||
enforce.assert_not_called()
|
||||
current = {'head': 'd' * 40, 'changed': True}
|
||||
with mock.patch.object(finish.enforcement, 'inspect', return_value=current), mock.patch.object(finish.enforcement, 'enforce') as enforce:
|
||||
self.assertEqual('enforced', finish.enforce_current(self.plan)['state'])
|
||||
enforce.assert_called_once_with(current)
|
||||
|
||||
def test_unknown_gate_profile_never_merges_or_enforces(self):
|
||||
self.plan['profile'] = 'all'
|
||||
with mock.patch.object(finish.remote, 'command') as command, mock.patch.object(finish.enforcement, 'inspect') as inspect:
|
||||
self.assertEqual('unsupported-gates', finish.merge_ready(self.plan, self.result)['state'])
|
||||
self.assertEqual('unsupported-gates', finish.enforce_current(self.plan)['state'])
|
||||
command.assert_not_called()
|
||||
inspect.assert_not_called()
|
||||
|
||||
def test_merged_tree_mismatch_is_not_reported_as_success(self):
|
||||
self.pr.update(state='MERGED', mergeCommit={'oid': 'd' * 40})
|
||||
with mock.patch.object(finish.remote, 'verify_submission'), mock.patch.object(finish.remote, 'command', return_value=json.dumps(self.pr)), mock.patch.object(finish.remote, 'api', side_effect=[
|
||||
{'commit': {'tree': {'sha': 'e' * 40}}},
|
||||
{'commit': {'tree': {'sha': 'f' * 40}, 'verification': {'verified': True}}}]):
|
||||
with self.assertRaises(ValueError):
|
||||
finish.merge_ready(self.plan, self.result)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,37 @@
|
||||
#!/bin/bash
|
||||
# Exercise the actual test runner with no server, browser, network or Maven work.
|
||||
set -euo pipefail
|
||||
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
|
||||
scratch=$(mktemp -d)
|
||||
trap 'rm -rf "$scratch"' EXIT
|
||||
mkdir "$scratch/bin"
|
||||
export RUN_LOG="$scratch/calls"
|
||||
cat > "$scratch/bin/npm" <<'STUB'
|
||||
#!/bin/bash
|
||||
printf 'npm %s\n' "$*" >> "$RUN_LOG"
|
||||
if [[ "$*" == 'run build' && ${BUILD_FAIL:-0} == 1 ]]; then exit 12; fi
|
||||
STUB
|
||||
for tool in mvn dockerize; do
|
||||
# shellcheck disable=SC2016
|
||||
printf '#!/bin/bash\nprintf "%s %%s\\n" "$*" >> "$RUN_LOG"\n' "$tool" > "$scratch/bin/$tool"
|
||||
done
|
||||
chmod +x "$scratch/bin/"*
|
||||
export PATH="$scratch/bin:$PATH"
|
||||
cd "$scratch"
|
||||
PARAM_BUILD_BEFORE_TEST=false bash "$root/src/scripts/frontend_test_run.sh" >/dev/null
|
||||
if grep -qx 'npm run build' "$RUN_LOG"; then
|
||||
echo 'FAIL: combined job rebuilt the already-built React app' >&2; exit 1
|
||||
fi
|
||||
grep -q '^mvn .* -B test$' "$RUN_LOG"
|
||||
echo 'PASS: combined job retains tests without a duplicate build'
|
||||
|
||||
: > "$RUN_LOG"
|
||||
bash "$root/src/scripts/frontend_test_run.sh" >/dev/null
|
||||
grep -qx 'npm run build' "$RUN_LOG"
|
||||
grep -q '^mvn .* -B test$' "$RUN_LOG"
|
||||
: > "$RUN_LOG"
|
||||
if BUILD_FAIL=1 bash "$root/src/scripts/frontend_test_run.sh" >/dev/null 2>&1; then
|
||||
echo 'FAIL: required standalone build failure ignored' >&2; exit 1
|
||||
fi
|
||||
if grep -q '^mvn ' "$RUN_LOG"; then exit 1; fi
|
||||
echo 'PASS: standalone command still builds and fails before tests when its build fails'
|
||||
@@ -241,6 +241,15 @@ class PipelineTests(unittest.TestCase):
|
||||
self.assertTrue(options["check_middleware_api_versions"])
|
||||
self.assertEqual(options["java_version"], "17")
|
||||
|
||||
def test_minor_track_reaches_continuation_without_changing_jobs(self):
|
||||
expected = self.render()
|
||||
expected['orbs']['qqq-orb'] = 'kingsrook/qqq-orb@0.7'
|
||||
self.write_manifest(orb_version='0.7')
|
||||
self.assertEqual(expected, self.render(QQQ_CI_ORB_VERSION='0.7'))
|
||||
if PACKED:
|
||||
self.assertEqual(expected, self.render(packed=True, QQQ_CI_ORB_VERSION='0.7'))
|
||||
self.render(ok=False, QQQ_CI_ORB_VERSION='0.8')
|
||||
|
||||
def test_branch_matrix(self):
|
||||
for branch, workflow in [("develop", "publish_snapshot"), ("release/1.0", "publish_release_candidate"),
|
||||
("main", "test_only"), ("hotfix/fix", "test_only"),
|
||||
@@ -420,9 +429,27 @@ class PipelineTests(unittest.TestCase):
|
||||
self.change_docs()
|
||||
self.assertIn("docs_only", self.render()["jobs"])
|
||||
|
||||
def test_core_snapshot_publisher_waits_for_all_acceptance_and_analysis(self):
|
||||
self.write_manifest(profile="core")
|
||||
result = self.render(CIRCLE_BRANCH="develop")
|
||||
jobs = {options.get("name", kind): options
|
||||
for entry in result["workflows"]["verify"]["jobs"]
|
||||
for kind, options in entry.items()}
|
||||
def ancestors(name, path=()):
|
||||
self.assertNotIn(name, path, "Workflow dependencies must be acyclic")
|
||||
required = jobs[name].get("requires", [])
|
||||
return set(required).union(*(ancestors(parent, path + (name,)) for parent in required))
|
||||
self.assertEqual(ancestors("qqq-orb/mvn_publish"), set(jobs) - {"qqq-orb/mvn_publish"})
|
||||
if PACKED:
|
||||
self.assertEqual(result, self.render(packed=True, CIRCLE_BRANCH="develop"))
|
||||
|
||||
def test_core_workflow_graphs_match_baseline(self):
|
||||
self.write_manifest(profile="core")
|
||||
baseline = yaml.safe_load((ROOT / "tests/fixtures/core_pipeline_baseline.yml").read_text(encoding="utf-8"))
|
||||
# Snapshot publication must now wait for the existing sample and analysis gates.
|
||||
for entry in baseline["workflows"]["publish_snapshot"]["jobs"]:
|
||||
if "qqq-orb/mvn_publish" in entry:
|
||||
entry["qqq-orb/mvn_publish"]["requires"] += ["snapshot_sample_coverage_gate", "qqq-orb/static_analysis"]
|
||||
self.git("tag", "v1.0.0")
|
||||
self.git("tag", "publish-fixture")
|
||||
events = [("feature/test", "", "test_only"), ("develop", "", "publish_snapshot"),
|
||||
@@ -596,6 +623,33 @@ class PipelineTests(unittest.TestCase):
|
||||
if PACKED:
|
||||
self.assertEqual(result, self.render(packed=True, CIRCLE_BRANCH="main"))
|
||||
|
||||
def test_frontend_ci_only_commits_verify_with_packed_parity(self):
|
||||
for profile, job_name in (("frontend-core", "qqq-orb/node_test_only"),
|
||||
("frontend-material", "qqq-orb/mvn_frontend_test_only")):
|
||||
self.write_manifest(profile=profile)
|
||||
self.git("add", ".qqq-ci.yml")
|
||||
self.git("-c", "commit.gpgsign=false", "commit", "-qm", "ci: adopt managed orb")
|
||||
for branch in ("main", "develop"):
|
||||
with self.subTest(profile=profile, branch=branch):
|
||||
result = self.render(CIRCLE_BRANCH=branch)
|
||||
job = result["workflows"]["verify"]["jobs"][0][job_name]
|
||||
self.assertNotIn("branch_type", job)
|
||||
self.assertEqual(job["filters"]["branches"], {"ignore": "automation/qbit-rc-pending"})
|
||||
if PACKED:
|
||||
self.assertEqual(result, self.render(packed=True, CIRCLE_BRANCH=branch))
|
||||
|
||||
def test_nonpublishing_main_ci_change_runs_verification_with_packed_parity(self):
|
||||
self.write_manifest(default_branch="main", release_on_main=False,
|
||||
test_branch_policy="feature")
|
||||
self.git("add", ".qqq-ci.yml")
|
||||
self.git("-c", "commit.gpgsign=false", "commit", "-qm", "ci: use central pipeline")
|
||||
result = self.render(CIRCLE_BRANCH="main")
|
||||
job = result["workflows"]["verify"]["jobs"][0]["qqq-orb/mvn_test_only"]
|
||||
self.assertEqual(job["filters"]["branches"], {"ignore": "automation/qbit-rc-pending"})
|
||||
self.assertNotIn("branch_type", job)
|
||||
if PACKED:
|
||||
self.assertEqual(result, self.render(packed=True, CIRCLE_BRANCH="main"))
|
||||
|
||||
def test_qbit_publication_policy_packed_parity(self):
|
||||
self.write_manifest(release_on_main=True, release_tag_policy="v-prefix",
|
||||
test_branch_policy="feature")
|
||||
|
||||
@@ -0,0 +1,159 @@
|
||||
"""Remote enrollment keeps immutable source checks and resumes signed submissions."""
|
||||
import base64
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
import yaml
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'tools'))
|
||||
import reconcile_enrollment as remote
|
||||
|
||||
|
||||
class ReconcileTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.head = 'a' * 40
|
||||
self.new = 'b' * 40
|
||||
self.repo = 'QRun-IO/example'
|
||||
self.target = {'repository': self.repo, 'branch': 'main'}
|
||||
self.manifest = {'profile': 'qbit', 'orb_version': '0.7.4', 'default_branch': 'main',
|
||||
'publish': {'snapshot': False, 'release_candidate': False, 'release': False}}
|
||||
self.sources = {
|
||||
'.qqq-ci.yml': yaml.safe_dump(self.manifest).encode(),
|
||||
'.circleci/config.yml': yaml.safe_dump(remote.enroll_consumers.rollout.standard.canonical_caller('qbit', '0.7.4')).encode()}
|
||||
|
||||
def request(self, path, missing=False):
|
||||
if path == 'repos/' + self.repo:
|
||||
return {'archived': False, 'default_branch': 'main'}
|
||||
if path.endswith('/branches/main'):
|
||||
return {'commit': {'sha': self.head}}
|
||||
if '/contents/' in path:
|
||||
self.assertTrue(path.endswith('?ref=' + self.head))
|
||||
name = path.split('/contents/')[1].split('?')[0]
|
||||
raw = self.sources.get(name)
|
||||
return None if raw is None else {'type': 'file', 'encoding': 'base64', 'content': base64.b64encode(raw).decode()}
|
||||
raise AssertionError(path)
|
||||
|
||||
def plan(self):
|
||||
with mock.patch.object(remote, 'api', side_effect=self.request):
|
||||
return remote.inspect(self.target, '0.8')
|
||||
|
||||
def test_immutable_plan_is_deterministic_and_preserves_policy(self):
|
||||
first, second = self.plan(), self.plan()
|
||||
self.assertEqual(first['automation_branch'], second['automation_branch'])
|
||||
self.assertEqual(self.sources, first['originals'])
|
||||
self.assertEqual({**self.manifest, 'orb_version': '0.8'},
|
||||
yaml.safe_load(first['replacements']['.qqq-ci.yml']))
|
||||
self.assertTrue(first['changed'])
|
||||
|
||||
def test_missing_manifest_uses_legacy_converter(self):
|
||||
self.sources.pop('.qqq-ci.yml')
|
||||
self.sources['.circleci/config.yml'] = yaml.safe_dump({'version': 2.1,
|
||||
'orbs': {'qqq-orb': 'kingsrook/qqq-orb@0.5.0'},
|
||||
'workflows': {'verify': {'jobs': [{'qqq-orb/mvn_test_only': {}}]}}}).encode()
|
||||
plan = self.plan()
|
||||
self.assertIsNone(plan['originals']['.qqq-ci.yml'])
|
||||
self.assertFalse(any(yaml.safe_load(plan['replacements']['.qqq-ci.yml'])['publish'].values()))
|
||||
|
||||
def submission_api(self, plan, path, missing=False):
|
||||
if '/branches/main' in path:
|
||||
return {'commit': {'sha': self.head}}
|
||||
if '/git/ref/' in path:
|
||||
return {'object': {'sha': self.new}}
|
||||
if '/commits/' in path:
|
||||
return {'commit': {'verification': {'verified': True}},
|
||||
'parents': [{'sha': self.head}], 'files': [{'filename': p} for p in remote.PATHS]}
|
||||
if '/contents/' in path:
|
||||
self.assertTrue(path.endswith('?ref=' + self.new))
|
||||
name = path.split('/contents/')[1].split('?')[0]
|
||||
return {'type': 'file', 'encoding': 'base64', 'content': base64.b64encode(plan['replacements'][name]).decode()}
|
||||
if '/pulls?' in path:
|
||||
return [{'head': {'sha': self.new}, 'html_url': 'https://github.com/QRun-IO/example/pull/1'}]
|
||||
raise AssertionError(path)
|
||||
|
||||
def test_matching_existing_submission_uses_no_git_or_pr_mutations(self):
|
||||
plan = self.plan()
|
||||
with mock.patch.object(remote, 'api', side_effect=lambda *a, **k: self.submission_api(plan, *a, **k)), mock.patch.object(remote, 'command') as command:
|
||||
result = remote.apply(plan)
|
||||
self.assertEqual('existing-pr', result['state'])
|
||||
command.assert_not_called()
|
||||
|
||||
def test_unsigned_existing_branch_is_rejected_without_mutation(self):
|
||||
plan = self.plan()
|
||||
|
||||
def request(path, missing=False):
|
||||
value = self.submission_api(plan, path, missing)
|
||||
if '/commits/' in path:
|
||||
value['commit']['verification']['verified'] = False
|
||||
return value
|
||||
|
||||
with mock.patch.object(remote, 'api', side_effect=request), mock.patch.object(remote, 'command') as command:
|
||||
with self.assertRaises(ValueError):
|
||||
remote.apply(plan)
|
||||
command.assert_not_called()
|
||||
|
||||
def test_source_drift_prevents_submission(self):
|
||||
plan = self.plan()
|
||||
with mock.patch.object(remote, 'api', return_value={'commit': {'sha': 'c' * 40}}), mock.patch.object(remote, 'command') as command:
|
||||
with self.assertRaises(ValueError):
|
||||
remote.apply(plan)
|
||||
command.assert_not_called()
|
||||
|
||||
def test_interrupted_push_creates_only_missing_pr(self):
|
||||
plan = self.plan()
|
||||
seen = []
|
||||
|
||||
def request(path, missing=False):
|
||||
return [] if '/pulls?' in path else self.submission_api(plan, path, missing)
|
||||
|
||||
def command(args, cwd=None):
|
||||
seen.append(args[:3])
|
||||
self.assertEqual(['gh', 'pr', 'create'], args[:3])
|
||||
self.assertIn(self.head, Path(args[args.index('--body-file') + 1]).read_text())
|
||||
return 'https://github.com/QRun-IO/example/pull/2'
|
||||
|
||||
with mock.patch.object(remote, 'api', side_effect=request), mock.patch.object(remote, 'command', side_effect=command):
|
||||
result = remote.apply(plan)
|
||||
self.assertEqual('submitted', result['state'])
|
||||
self.assertEqual([['gh', 'pr', 'create']], seen)
|
||||
|
||||
def test_new_submission_requires_signed_commit_and_never_force_pushes(self):
|
||||
plan = self.plan()
|
||||
committed = False
|
||||
seen = []
|
||||
|
||||
def request(path, missing=False):
|
||||
if '/git/ref/' in path:
|
||||
return None
|
||||
if '/pulls?' in path:
|
||||
return []
|
||||
return self.submission_api(plan, path, missing)
|
||||
|
||||
def command(args, cwd=None):
|
||||
nonlocal committed
|
||||
seen.append(args)
|
||||
if args[:2] == ['git', 'clone']:
|
||||
root = Path(args[-1])
|
||||
(root / '.circleci').mkdir(parents=True)
|
||||
for name, data in self.sources.items():
|
||||
(root / name).write_bytes(data)
|
||||
elif args[:2] == ['git', 'rev-parse']:
|
||||
return self.new if committed else self.head
|
||||
elif args[:2] == ['git', 'commit']:
|
||||
self.assertIn('-S', args)
|
||||
for name in remote.PATHS:
|
||||
self.assertEqual(plan['replacements'][name], (cwd / name).read_bytes())
|
||||
committed = True
|
||||
elif args[:3] == ['gh', 'pr', 'create']:
|
||||
return 'https://github.com/QRun-IO/example/pull/3'
|
||||
return ''
|
||||
|
||||
with mock.patch.object(remote, 'api', side_effect=request), mock.patch.object(remote, 'command', side_effect=command):
|
||||
self.assertEqual('submitted', remote.apply(plan)['state'])
|
||||
pushes = [args for args in seen if args[:2] == ['git', 'push']]
|
||||
self.assertEqual([['git', 'push', 'origin', 'HEAD:refs/heads/' + plan['automation_branch']]], pushes)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -68,6 +68,30 @@ class RolloutTests(unittest.TestCase):
|
||||
self.assertFalse(rollout.rollout([root], '0.7.11', write=True)[0]['changed'])
|
||||
self.assertEqual(applied, self.contents(root))
|
||||
|
||||
def test_managed_patch_track_and_exact_pin_rollback_preserve_every_profile(self):
|
||||
roots = [self.checkout(profile, profile=profile, default_branch='main')
|
||||
for profile in rollout.standard.PROFILE_PARAMETERS]
|
||||
originals = [yaml.safe_load((root / '.qqq-ci.yml').read_text()) for root in roots]
|
||||
rollout.rollout(roots, '0.7', write=True)
|
||||
for root, original in zip(roots, originals):
|
||||
self.assertEqual({**original, 'orb_version': '0.7'},
|
||||
yaml.safe_load((root / '.qqq-ci.yml').read_text()))
|
||||
rollout.standard.validate(*(data.decode() for data in self.contents(root)), '0.7')
|
||||
rollout.pipeline.load_manifest(root / '.qqq-ci.yml', '0.7')
|
||||
with self.assertRaises(ValueError):
|
||||
rollout.pipeline.load_manifest(root / '.qqq-ci.yml', '0.8')
|
||||
self.assertTrue(all(not r['changed'] for r in rollout.rollout(roots, '0.7', write=True)))
|
||||
rollout.rollout(roots, '0.7.4', write=True)
|
||||
for root, original in zip(roots, originals):
|
||||
self.assertEqual(original, yaml.safe_load((root / '.qqq-ci.yml').read_text()))
|
||||
|
||||
def test_unquoted_track_is_rejected_as_numeric_yaml(self):
|
||||
root = self.checkout('numeric')
|
||||
manifest = root / '.qqq-ci.yml'
|
||||
manifest.write_text(manifest.read_text().replace('0.7.4', '0.7'))
|
||||
with self.assertRaises(ValueError):
|
||||
rollout.pipeline.load_manifest(manifest, '0.7')
|
||||
|
||||
def test_one_custom_caller_rejects_entire_batch_before_any_write(self):
|
||||
good = self.checkout('good')
|
||||
bad = self.checkout('bad')
|
||||
@@ -97,7 +121,7 @@ class RolloutTests(unittest.TestCase):
|
||||
def test_invalid_target_and_duplicate_checkout_are_rejected(self):
|
||||
root = self.checkout('qbit')
|
||||
before = self.contents(root)
|
||||
for version in ['latest', 'dev:test', '0.7', '0.7.11\n']:
|
||||
for version in ['latest', 'volatile', 'dev:test', '0', '0.7.*', '0.7.11\n']:
|
||||
with self.assertRaises(ValueError):
|
||||
rollout.rollout([root], version, write=True)
|
||||
with self.assertRaises(ValueError):
|
||||
|
||||
@@ -76,6 +76,24 @@ class MigrationTests(unittest.TestCase):
|
||||
manifest = migration.infer_manifest(self.config(), "0.7.0", "main")
|
||||
self.assertFalse(any(manifest["publish"].values()))
|
||||
|
||||
def test_managed_track_preserves_policy_and_caller_pins(self):
|
||||
fixtures = json.loads((Path(__file__).parent / "fixtures/qbit-publication-policies.json").read_text(encoding="utf-8"))
|
||||
for fixture in fixtures:
|
||||
with self.subTest(repository=fixture["name"]):
|
||||
expected = migration.infer_manifest(fixture["config"], "0.8.1", fixture["default_branch"])
|
||||
expected["orb_version"] = "0.8"
|
||||
actual = migration.infer_manifest(fixture["config"], "0.8", fixture["default_branch"])
|
||||
self.assertEqual(actual, expected)
|
||||
caller = migration.canonical_caller("0.8", actual["release_tag_policy"], actual["publish"].get("feature", False))
|
||||
self.assertEqual(caller["orbs"]["qqq-orb"], "kingsrook/qqq-orb@0.8")
|
||||
self.assertEqual(caller["workflows"]["setup"]["jobs"][0]["qqq-orb/generate_pipeline"]["orb_version"], "0.8")
|
||||
self.assertEqual(yaml.safe_load(yaml.safe_dump(actual))["orb_version"], "0.8")
|
||||
|
||||
def test_reject_ambiguous_or_invalid_versions(self):
|
||||
for version in (0.8, None, "0", "latest", "volatile", "0.8.*", "0.8.1.2", ""):
|
||||
with self.subTest(version=version), self.assertRaises(ValueError):
|
||||
migration.infer_manifest(self.config(), version, "main")
|
||||
|
||||
def test_snapshot_does_not_enable_rc_or_release(self):
|
||||
manifest = migration.infer_manifest(
|
||||
self.config("qqq-orb/mvn_publish", branch_type="snapshot",
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
"""Maven failures must survive tee without overriding project analysis policy."""
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
SCRIPTS = [('mvn_spotbugs.sh', 'SPOTBUGS_FAIL_ON_ERROR', 'spotbugs.failOnError'),
|
||||
('mvn_pmd.sh', 'PMD_FAIL_ON_VIOLATION', 'pmd.failOnViolation')]
|
||||
|
||||
|
||||
class StaticAnalysisExitTests(unittest.TestCase):
|
||||
def invoke(self, script, variable, setting, status):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
binary = root / 'mvn'
|
||||
binary.write_text('#!/bin/sh\nprintf "%s\\n" "$@" > "$ARGUMENT_FILE"\necho analysis-output\nexit "$FAKE_MAVEN_EXIT"\n')
|
||||
binary.chmod(0o755)
|
||||
env = dict(os.environ, PATH=str(root) + os.pathsep + os.environ['PATH'],
|
||||
ARGUMENT_FILE=str(root / 'arguments'), FAKE_MAVEN_EXIT=str(status))
|
||||
env.pop(variable, None)
|
||||
if setting is not None:
|
||||
env[variable] = setting
|
||||
result = subprocess.run(['bash', str(ROOT / 'src/scripts' / script)], cwd=root,
|
||||
env=env, text=True, capture_output=True, timeout=15)
|
||||
return result, (root / 'arguments').read_text().splitlines()
|
||||
|
||||
def test_maven_failure_is_never_hidden_by_successful_tee(self):
|
||||
for script, variable, _ in SCRIPTS:
|
||||
for setting in ('true', 'false'):
|
||||
with self.subTest(script=script, setting=setting):
|
||||
result, _ = self.invoke(script, variable, setting, 17)
|
||||
self.assertEqual(17, result.returncode)
|
||||
self.assertIn('analysis-output', result.stdout)
|
||||
|
||||
def test_success_preserves_explicit_flag_and_project_defaults(self):
|
||||
for script, variable, property_name in SCRIPTS:
|
||||
for setting in ('true', 'false', None):
|
||||
with self.subTest(script=script, setting=setting):
|
||||
result, arguments = self.invoke(script, variable, setting, 0)
|
||||
self.assertEqual(0, result.returncode, result.stderr)
|
||||
if setting == 'true':
|
||||
self.assertIn('-D' + property_name + '=true', arguments)
|
||||
else:
|
||||
self.assertFalse(any(arg.startswith('-D' + property_name + '=') for arg in arguments))
|
||||
self.assertIn('-DskipTests', arguments)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
+47
-5
@@ -66,6 +66,38 @@ def gh(path):
|
||||
return json.loads(command(['gh', 'api', '--method', 'GET', path]))
|
||||
|
||||
|
||||
def discover_inventory(organization):
|
||||
"""Discover every repository visible to existing gh authentication, across all pages."""
|
||||
if not re.fullmatch(r'[A-Za-z0-9][A-Za-z0-9-]*', organization):
|
||||
raise ReadError('invalid organization name')
|
||||
pages = json.loads(command(['gh', 'api', '--method', 'GET', '--paginate', '--slurp',
|
||||
f'orgs/{organization}/repos?type=all&per_page=100']))
|
||||
if not isinstance(pages, list) or any(not isinstance(page, list) for page in pages):
|
||||
raise ReadError('invalid organization inventory response')
|
||||
entries, seen = [], set()
|
||||
for page in pages:
|
||||
for repo in page:
|
||||
if not isinstance(repo, dict):
|
||||
raise ReadError('invalid organization repository')
|
||||
name = repo.get('name')
|
||||
full_name = repo.get('full_name')
|
||||
archived = repo.get('archived')
|
||||
branch = repo.get('default_branch')
|
||||
if (not isinstance(name, str) or not re.fullmatch(r'[A-Za-z0-9_.-]+', name)
|
||||
or not isinstance(full_name, str)
|
||||
or full_name.lower() != f'{organization}/{name}'.lower()
|
||||
or type(archived) is not bool
|
||||
or not isinstance(branch, str) or not branch
|
||||
or full_name.lower() in seen):
|
||||
raise ReadError('invalid or duplicate organization repository')
|
||||
seen.add(full_name.lower())
|
||||
entries.append({'name': name, 'nameWithOwner': full_name,
|
||||
'isArchived': archived, 'defaultBranchRef': {'name': branch}})
|
||||
if not entries:
|
||||
raise ReadError('empty organization inventory; coverage is unproven')
|
||||
return sorted(entries, key=lambda entry: entry['nameWithOwner'].lower())
|
||||
|
||||
|
||||
class Snapshot:
|
||||
def __init__(self, entry, live=False):
|
||||
self.repo = entry['nameWithOwner']
|
||||
@@ -374,22 +406,32 @@ def run(entries, version, worker_sha, policies, live=False, snapshot_factory=Sna
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--inventory', type=Path, required=True)
|
||||
inventory = parser.add_mutually_exclusive_group(required=True)
|
||||
inventory.add_argument('--inventory', type=Path)
|
||||
inventory.add_argument('--organization', help='Discover current repositories using existing gh authentication; requires --live')
|
||||
parser.add_argument('--orb-version', required=True)
|
||||
parser.add_argument('--worker-sha')
|
||||
parser.add_argument('--policy', type=Path)
|
||||
parser.add_argument('--live', action='store_true', help='Read current source and branch rules using existing gh authentication')
|
||||
parser.add_argument('--source-branch', help='Inspect this branch instead of the repository default; requires --live')
|
||||
parser.add_argument('--expected-active', type=int, default=68)
|
||||
parser.add_argument('--expected-active', type=int, help='Optional independent coverage check; saved inventories default to 68')
|
||||
parser.add_argument('--output', type=Path)
|
||||
args = parser.parse_args()
|
||||
if args.source_branch is not None and not args.live:
|
||||
parser.error('--source-branch requires --live')
|
||||
entries = json.loads(args.inventory.read_text(encoding='utf-8'))
|
||||
if args.organization and not args.live:
|
||||
parser.error('--organization requires --live')
|
||||
entries = (discover_inventory(args.organization) if args.organization else
|
||||
json.loads(args.inventory.read_text(encoding='utf-8')))
|
||||
expected_active = args.expected_active
|
||||
if expected_active is None and not args.organization:
|
||||
expected_active = 68
|
||||
policies = json.loads(args.policy.read_text(encoding='utf-8')) if args.policy else {}
|
||||
report = run(entries, args.orb_version, args.worker_sha, policies, args.live, source_branch=args.source_branch)
|
||||
report['expected_active_count'] = args.expected_active
|
||||
report['coverage_matches_expected'] = (report['active_count'] == args.expected_active
|
||||
report['inventory_source'] = ('organization-api' if args.organization else 'saved-inventory')
|
||||
report['inventory_visibility'] = 'authenticated repositories; inaccessible repositories cannot be certified'
|
||||
report['expected_active_count'] = expected_active
|
||||
report['coverage_matches_expected'] = ((expected_active is None or report['active_count'] == expected_active)
|
||||
and report['unique_repository_count'] == report['inventory_count'])
|
||||
text = json.dumps(report, indent=2) + '\n'
|
||||
if args.output:
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
"""Discover current enrollment targets without assuming unknown repositories conform."""
|
||||
import audit_estate
|
||||
|
||||
|
||||
def discover(organization, version, inspect, request, inventory=None):
|
||||
"""Inspect every active main/develop branch; retain unsupported results explicitly."""
|
||||
entries = audit_estate.discover_inventory(organization) if inventory is None else inventory
|
||||
plans, findings = [], []
|
||||
for entry in entries:
|
||||
repo = entry['nameWithOwner']
|
||||
if entry['isArchived']:
|
||||
findings.append({'repository': repo, 'state': 'archived'})
|
||||
continue
|
||||
if repo.lower() == (organization + '/qqq-orb').lower():
|
||||
findings.append({'repository': repo, 'state': 'owner-bootstrap',
|
||||
'reason': 'Orb publisher is not its own consumer'})
|
||||
continue
|
||||
try:
|
||||
branches, page = [], 1
|
||||
while True:
|
||||
batch = request(f'repos/{repo}/branches?per_page=100&page={page}')
|
||||
if not isinstance(batch, list) or any(not isinstance(b, dict) or not isinstance(b.get('name'), str) for b in batch):
|
||||
raise ValueError('Invalid branch inventory')
|
||||
branches.extend(b['name'] for b in batch)
|
||||
if len(batch) < 100:
|
||||
break
|
||||
page += 1
|
||||
if len(set(branches)) != len(branches):
|
||||
raise ValueError('Duplicate branch inventory')
|
||||
selected = sorted(set(branches) & {'main', 'develop'})
|
||||
if not selected:
|
||||
raise ValueError('No current main/develop branch; explicit profile decision required')
|
||||
except (ValueError, OSError) as error:
|
||||
findings.append({'repository': repo, 'state': 'unresolved', 'reason': str(error)})
|
||||
continue
|
||||
for branch in selected:
|
||||
target = {'repository': repo, 'branch': branch}
|
||||
try:
|
||||
plans.append(inspect(target, version))
|
||||
except (ValueError, OSError) as error:
|
||||
findings.append({**target, 'state': 'unresolved', 'reason': str(error)})
|
||||
return plans, findings
|
||||
@@ -0,0 +1,190 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Batch-enforce observed orb checks; dry-run unless --apply. Never changes source."""
|
||||
import argparse
|
||||
import base64
|
||||
import importlib.util
|
||||
import json
|
||||
from pathlib import Path
|
||||
import re
|
||||
import subprocess
|
||||
from urllib.parse import quote
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
spec = importlib.util.spec_from_file_location('standard', ROOT / 'src/scripts/validate_standard.py')
|
||||
standard = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(standard)
|
||||
DESIRED = [{'context': name, 'app_id': 18001} for name in ('setup', 'verify')]
|
||||
PROFILES = {'qbit', 'core', 'frontend-core', 'frontend-material', 'repository-policy'}
|
||||
|
||||
|
||||
def api(path, method='GET', payload=None):
|
||||
"""Use existing gh authentication; never expose response bodies on failure."""
|
||||
args = ['gh', 'api', '--method', method, path]
|
||||
if payload is not None:
|
||||
args += ['--input', '-']
|
||||
result = subprocess.run(args, input=None if payload is None else json.dumps(payload),
|
||||
text=True, encoding='utf-8', capture_output=True, timeout=60)
|
||||
if result.returncode:
|
||||
try:
|
||||
error = json.loads(result.stdout)
|
||||
except ValueError:
|
||||
error = {}
|
||||
if method == 'GET' and path.endswith('/protection') and error.get('status') == '404' and error.get('message') == 'Branch not protected':
|
||||
return None
|
||||
raise ValueError('GitHub operation unavailable: ' + method + ' ' + path.split('?')[0])
|
||||
return json.loads(result.stdout) if result.stdout.strip() else None
|
||||
|
||||
|
||||
def required_checks(before, runs):
|
||||
"""Append app-bound gates only after both succeeded; preserve all other gates."""
|
||||
for desired in DESIRED:
|
||||
matches = [run for run in runs if run['name'] == desired['context'] and run['app']['id'] == desired['app_id']]
|
||||
if not matches or max(matches, key=lambda run: run['id'])['conclusion'] != 'success':
|
||||
raise ValueError('Current branch head has no successful CircleCI ' + desired['context'])
|
||||
prior = (before or {}).get('required_status_checks') or {}
|
||||
checks = [dict(check) for check in prior.get('checks', [])]
|
||||
represented = {check['context'] for check in checks}
|
||||
checks += [{'context': name, 'app_id': -1} for name in prior.get('contexts', []) if name not in represented]
|
||||
for desired in DESIRED:
|
||||
matches = [check for check in checks if check['context'] == desired['context']]
|
||||
if any(check.get('app_id') not in (None, -1, desired['app_id']) for check in matches):
|
||||
raise ValueError('Existing gate uses a different app: ' + desired['context'])
|
||||
checks = [check for check in checks if check['context'] != desired['context']]
|
||||
checks.append(dict(desired))
|
||||
return {'strict': True, 'checks': checks}
|
||||
|
||||
|
||||
def verified_rulesets(repo, effective, request=api):
|
||||
"""Recognize active repository rulesets with bound gates and no bypass actors."""
|
||||
selected = [rule for rule in effective if rule['type'] == 'required_status_checks']
|
||||
if not selected or any(rule.get('ruleset_source_type') != 'Repository'
|
||||
or rule.get('ruleset_source') != repo for rule in selected):
|
||||
raise ValueError('Required rulesets need unsupported source reconciliation')
|
||||
records = []
|
||||
for identifier in sorted({rule['ruleset_id'] for rule in selected}):
|
||||
record = request(f'repos/{repo}/rulesets/{identifier}')
|
||||
if (record.get('enforcement') != 'active' or record.get('target') != 'branch'
|
||||
or record.get('source_type') != 'Repository' or record.get('source') != repo
|
||||
or record.get('bypass_actors') != []):
|
||||
raise ValueError('Ruleset must be active with no bypass actors')
|
||||
records.append(record)
|
||||
requirements = [rule['parameters'] for record in records for rule in record['rules']
|
||||
if rule['type'] == 'required_status_checks']
|
||||
for desired in DESIRED:
|
||||
if not any(check['context'] == desired['context'] and check.get('integration_id') == desired['app_id']
|
||||
for parameters in requirements for check in parameters['required_status_checks']):
|
||||
raise ValueError('Ruleset is missing app-bound gate: ' + desired['context'])
|
||||
# Retain the existing freshness policy: recognition does not add rebuilds or
|
||||
# weaken any existing scanner, merge, or bypass requirement.
|
||||
return records, any(parameters.get('strict_required_status_checks_policy', False) for parameters in requirements)
|
||||
|
||||
|
||||
def inspect(repo, branch, version, request=api):
|
||||
"""Read immutable canonical source, latest check runs, and current protections."""
|
||||
if not re.fullmatch(r'[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+', repo) or branch not in ('main', 'develop'):
|
||||
raise ValueError('Targets must name an exact repository and main/develop branch')
|
||||
root = 'repos/' + repo
|
||||
info = request(root)
|
||||
if info['archived']:
|
||||
raise ValueError('Archived repository')
|
||||
prefix = root + '/branches/' + branch
|
||||
head = request(prefix)['commit']['sha']
|
||||
source = []
|
||||
for path in ('.circleci/config.yml', '.qqq-ci.yml'):
|
||||
item = request(root + '/contents/' + quote(path, safe='/') + '?ref=' + head)
|
||||
if item.get('encoding') != 'base64' or item.get('type') != 'file':
|
||||
raise ValueError('Canonical declarations must be regular files')
|
||||
source.append(base64.b64decode(item['content'], validate=False).decode('utf-8'))
|
||||
standard.validate(*source, version)
|
||||
if standard.load_yaml(source[1])['profile'] not in PROFILES:
|
||||
raise ValueError('Profile does not use the shared setup/verify gate pair')
|
||||
rules = request(root + '/rules/branches/' + branch)
|
||||
rulesets, ruleset_strict = verified_rulesets(repo, rules, request) if rules else ([], False)
|
||||
before = request(prefix + '/protection')
|
||||
restrictions = (before or {}).get('restrictions')
|
||||
if (not rulesets and restrictions is not None
|
||||
and not any(restrictions.get(kind) for kind in ('users', 'teams', 'apps'))
|
||||
and not (before or {}).get('enforce_admins', {}).get('enabled')):
|
||||
raise ValueError('Empty push allowlist needs explicit access policy before enforcing admins')
|
||||
runs = []
|
||||
page = 1
|
||||
while True:
|
||||
batch = request(root + '/commits/' + head + '/check-runs?filter=latest&per_page=100&page=' + str(page))['check_runs']
|
||||
runs += batch
|
||||
if len(batch) < 100:
|
||||
break
|
||||
page += 1
|
||||
desired = required_checks(before, runs)
|
||||
if rulesets:
|
||||
desired['strict'] = ruleset_strict or bool(((before or {}).get('required_status_checks') or {}).get('strict'))
|
||||
return {'repository': repo, 'branch': branch, 'head': head, 'before': before,
|
||||
'required_status_checks': desired, 'changed': False, 'protection_mode': 'rulesets',
|
||||
'effective_rules': rules, 'rulesets': rulesets}
|
||||
current = (before or {}).get('required_status_checks') or {}
|
||||
ready = (current.get('strict') is True and current.get('checks') == desired['checks']
|
||||
and (before or {}).get('enforce_admins', {}).get('enabled') is True)
|
||||
return {'repository': repo, 'branch': branch, 'head': head, 'before': before,
|
||||
'required_status_checks': desired, 'changed': not ready}
|
||||
|
||||
|
||||
def enforce(plan, request=api):
|
||||
"""Recheck drift, use narrow updates, and verify preservation of other settings."""
|
||||
root = 'repos/' + plan['repository']
|
||||
prefix = root + '/branches/' + plan['branch']
|
||||
if request(prefix)['commit']['sha'] != plan['head'] or request(prefix + '/protection') != plan['before']:
|
||||
raise ValueError('Branch or protection changed after preflight')
|
||||
rules = request(root + '/rules/branches/' + plan['branch'])
|
||||
if plan.get('protection_mode') == 'rulesets':
|
||||
if rules != plan['effective_rules'] or verified_rulesets(plan['repository'], rules, request)[0] != plan['rulesets']:
|
||||
raise ValueError('Ruleset changed after preflight')
|
||||
return
|
||||
if rules:
|
||||
raise ValueError('Ruleset changed after preflight')
|
||||
if not plan['changed']:
|
||||
return
|
||||
before = plan['before']
|
||||
if before is None:
|
||||
request(prefix + '/protection', 'PUT', {
|
||||
'required_status_checks': plan['required_status_checks'], 'enforce_admins': True,
|
||||
'required_pull_request_reviews': None, 'restrictions': None})
|
||||
else:
|
||||
request(prefix + '/protection/required_status_checks', 'PATCH', plan['required_status_checks'])
|
||||
if not before['enforce_admins']['enabled']:
|
||||
request(prefix + '/protection/enforce_admins', 'POST')
|
||||
after = request(prefix + '/protection')
|
||||
actual = after['required_status_checks']
|
||||
if not actual['strict'] or actual['checks'] != plan['required_status_checks']['checks'] or not after['enforce_admins']['enabled']:
|
||||
raise ValueError('Protection readback did not match requested gates')
|
||||
if before:
|
||||
for key, value in before.items():
|
||||
if key not in ('required_status_checks', 'enforce_admins') and after.get(key) != value:
|
||||
raise ValueError('Unrelated protection changed: ' + key)
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--targets', type=Path, required=True, help='JSON list of {repository, branch}')
|
||||
parser.add_argument('--orb-version', required=True)
|
||||
parser.add_argument('--apply', action='store_true')
|
||||
parser.add_argument('--output', type=Path, required=True)
|
||||
args = parser.parse_args()
|
||||
targets = json.loads(args.targets.read_text(encoding='utf-8'))
|
||||
if not isinstance(targets, list) or not targets:
|
||||
raise ValueError('A nonempty explicit target list is required')
|
||||
if len({(row['repository'], row['branch']) for row in targets}) != len(targets):
|
||||
raise ValueError('Duplicate branch target')
|
||||
# Resolve the entire batch before any mutations. Failures never trigger rollback
|
||||
# that would remove successful protections; rerunning converges idempotently.
|
||||
plans = [inspect(row['repository'], row['branch'], args.orb_version) for row in targets]
|
||||
args.output.write_text(json.dumps({'state': 'planned', 'plans': plans}, indent=2), encoding='utf-8')
|
||||
if args.apply:
|
||||
for plan in plans:
|
||||
enforce(plan)
|
||||
plan['verified'] = True
|
||||
args.output.write_text(json.dumps({'state': 'applying', 'plans': plans}, indent=2), encoding='utf-8')
|
||||
args.output.write_text(json.dumps({'state': 'verified', 'plans': plans}, indent=2), encoding='utf-8')
|
||||
print(json.dumps({'targets': len(plans), 'changed': sum(plan['changed'] for plan in plans), 'applied': args.apply}))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,101 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Enroll a batch of audited consumers; preview by default, write with --write.
|
||||
|
||||
Existing canonical profiles retain their manifests. Legacy QBits use the shared
|
||||
policy inference. Unknown orchestration is rejected before any checkout changes.
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
from pathlib import Path
|
||||
import re
|
||||
import tempfile
|
||||
|
||||
import yaml
|
||||
|
||||
import rollout_version as rollout
|
||||
import standardize_qbit as qbit
|
||||
|
||||
|
||||
def declaration(root, relative):
|
||||
"""Reject redirected declarations, including symlinked parent directories."""
|
||||
path = root / relative
|
||||
current = path
|
||||
while current != root:
|
||||
if current.is_symlink():
|
||||
raise ValueError('CI declarations must not use symlinks')
|
||||
current = current.parent
|
||||
if not path.resolve().is_relative_to(root):
|
||||
raise ValueError('CI declarations must remain inside the checkout')
|
||||
return path
|
||||
|
||||
|
||||
def plan(target, version):
|
||||
"""Preserve explicit publication policy and validate generated declarations."""
|
||||
if not isinstance(target, dict) or set(target) != {'checkout', 'default_branch'}:
|
||||
raise ValueError('Each target requires checkout and observed default_branch')
|
||||
if target['default_branch'] not in ('main', 'develop'):
|
||||
raise ValueError('Unsupported default branch')
|
||||
root = Path(target['checkout']).resolve(strict=True)
|
||||
config = declaration(root, '.circleci/config.yml')
|
||||
manifest = declaration(root, '.qqq-ci.yml')
|
||||
if manifest.exists():
|
||||
data = rollout.standard.load_yaml(manifest.read_text(encoding='utf-8'))
|
||||
if not isinstance(data, dict) or data.get('default_branch') != target['default_branch']:
|
||||
raise ValueError('Observed default branch differs from manifest')
|
||||
return {**rollout.plan(root, version), 'enrollment': False}
|
||||
original = config.read_bytes()
|
||||
data = qbit.infer_manifest(rollout.standard.load_yaml(original.decode('utf-8')),
|
||||
version, target['default_branch'])
|
||||
caller = rollout.standard.canonical_caller(data['profile'], version,
|
||||
release_tag_policy=data['release_tag_policy'], publish_feature=data['publish'].get('feature', False))
|
||||
config_raw = yaml.safe_dump(caller, sort_keys=False)
|
||||
manifest_raw = yaml.safe_dump(data, sort_keys=False)
|
||||
rollout.standard.validate(config_raw, manifest_raw, version)
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
temporary_manifest = Path(directory) / 'manifest.yml'
|
||||
temporary_manifest.write_text(manifest_raw, encoding='utf-8')
|
||||
rollout.pipeline.load_manifest(temporary_manifest, version)
|
||||
return {'checkout': str(root), 'profile': data['profile'], 'from': None,
|
||||
'to': version, 'enrollment': True,
|
||||
'files': [(config, original, config_raw.encode('utf-8')),
|
||||
(manifest, None, manifest_raw.encode('utf-8'))]}
|
||||
|
||||
|
||||
def enroll(targets, version, write=False):
|
||||
"""Validate the whole batch before writing; repeat applications are no-ops."""
|
||||
if not isinstance(version, str) or not re.fullmatch(r'\d+\.\d+(?:\.\d+)?', version):
|
||||
raise ValueError('Enrollment requires a stable minor track or exact patch')
|
||||
if not isinstance(targets, list) or not targets:
|
||||
raise ValueError('A nonempty target list is required')
|
||||
plans = [plan(target, version) for target in targets]
|
||||
if len({item['checkout'] for item in plans}) != len(plans):
|
||||
raise ValueError('Each checkout may appear only once')
|
||||
if write:
|
||||
for item in plans:
|
||||
for path, original, replacement in item['files']:
|
||||
declaration(Path(item['checkout']), path.relative_to(item['checkout']))
|
||||
current = path.read_bytes() if path.exists() else None
|
||||
if current != original:
|
||||
raise ValueError('Declaration changed after preflight')
|
||||
for item in plans:
|
||||
for path, original, replacement in item['files']:
|
||||
if replacement != original:
|
||||
path.write_bytes(replacement)
|
||||
return [{**{key: value for key, value in item.items() if key != 'files'},
|
||||
'changed': any(old != new for _, old, new in item['files']),
|
||||
'written': write} for item in plans]
|
||||
|
||||
|
||||
def main():
|
||||
"""Read one fleet target file and emit an auditable result."""
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('targets', type=Path)
|
||||
parser.add_argument('--orb-version', required=True)
|
||||
parser.add_argument('--write', action='store_true')
|
||||
args = parser.parse_args()
|
||||
print(json.dumps(enroll(json.loads(args.targets.read_text(encoding='utf-8')),
|
||||
args.orb_version, args.write), indent=2))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,78 @@
|
||||
"""Qualify signed enrollment PRs and enforce only successful current branch heads."""
|
||||
import json
|
||||
|
||||
import enforce_orb_checks as enforcement
|
||||
import reconcile_enrollment as remote
|
||||
|
||||
|
||||
def qualification(repo, head):
|
||||
"""Require both app-bound orb gates and every advertised latest check/status."""
|
||||
checks, page = [], 1
|
||||
while True:
|
||||
batch = remote.api(f'repos/{repo}/commits/{head}/check-runs?filter=latest&per_page=100&page={page}')['check_runs']
|
||||
checks.extend(batch)
|
||||
if len(batch) < 100:
|
||||
break
|
||||
page += 1
|
||||
reasons = []
|
||||
for name in ('setup', 'verify'):
|
||||
found = [check for check in checks if check['name'] == name and check['app']['id'] == 18001]
|
||||
if not found or max(found, key=lambda check: check['id'])['conclusion'] != 'success':
|
||||
reasons.append(name)
|
||||
for check in checks:
|
||||
if check['status'] != 'completed' or check['conclusion'] not in ('success', 'neutral', 'skipped'):
|
||||
reasons.append(check['name'])
|
||||
page = 1
|
||||
while True:
|
||||
batch = remote.api(f'repos/{repo}/commits/{head}/status?per_page=100&page={page}')['statuses']
|
||||
reasons.extend(status['context'] for status in batch if status['state'] != 'success')
|
||||
if len(batch) < 100:
|
||||
break
|
||||
page += 1
|
||||
return sorted(set(reasons))
|
||||
|
||||
|
||||
def merge_ready(plan, result):
|
||||
"""Normal exact-head squash merge only; no administrative or force bypass."""
|
||||
if plan['profile'] not in enforcement.PROFILES:
|
||||
return {'state': 'unsupported-gates', 'profile': plan['profile']}
|
||||
if result['state'] == 'unchanged':
|
||||
return {'state': 'unchanged'}
|
||||
repo, head = plan['repository'], result['head']
|
||||
remote.verify_submission(plan, head)
|
||||
pr = json.loads(remote.command(['gh', 'pr', 'view', result['url'], '--repo', repo,
|
||||
'--json', 'number,state,headRefOid,baseRefOid,baseRefName,mergeCommit']))
|
||||
if pr['headRefOid'] != head or pr['baseRefName'] != plan['branch']:
|
||||
raise ValueError('PR differs from enrollment target')
|
||||
if pr['state'] == 'CLOSED':
|
||||
return {'state': 'closed', 'url': result['url']}
|
||||
if pr['state'] != 'MERGED':
|
||||
if pr['baseRefOid'] != plan['head'] or remote.api(f'repos/{repo}/branches/{plan["branch"]}')['commit']['sha'] != plan['head']:
|
||||
return {'state': 'stale-base', 'url': result['url']}
|
||||
pending = qualification(repo, head)
|
||||
if pending:
|
||||
return {'state': 'waiting-checks', 'checks': pending, 'url': result['url']}
|
||||
remote.command(['gh', 'pr', 'merge', str(pr['number']), '--repo', repo, '--squash', '--match-head-commit', head])
|
||||
pr = json.loads(remote.command(['gh', 'pr', 'view', result['url'], '--repo', repo,
|
||||
'--json', 'state,mergeCommit']))
|
||||
if pr['state'] != 'MERGED':
|
||||
raise ValueError('PR merge was not confirmed')
|
||||
merged = pr['mergeCommit']['oid']
|
||||
expected = remote.api(f'repos/{repo}/commits/{head}')['commit']['tree']['sha']
|
||||
actual = remote.api(f'repos/{repo}/commits/{merged}')['commit']
|
||||
if not actual['verification']['verified'] or actual['tree']['sha'] != expected:
|
||||
raise ValueError('Merged signature or source tree differs from qualified source')
|
||||
return {'state': 'merged', 'url': result['url'], 'head': head, 'merge': merged}
|
||||
|
||||
|
||||
def enforce_current(plan):
|
||||
"""Use shared narrow enforcement after the actual branch head passes its gates."""
|
||||
if plan['profile'] not in enforcement.PROFILES:
|
||||
return {'state': 'unsupported-gates', 'profile': plan['profile']}
|
||||
try:
|
||||
current = enforcement.inspect(plan['repository'], plan['branch'], plan['version'])
|
||||
except ValueError as error:
|
||||
return {'state': 'waiting-enforcement', 'reason': str(error)}
|
||||
enforcement.enforce(current)
|
||||
return {'state': 'enforced', 'head': current['head'], 'changed': current['changed'],
|
||||
'protection_mode': current.get('protection_mode', 'classic')}
|
||||
@@ -0,0 +1,195 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Plan remote enrollment and submit signed, repeatable PRs with --apply."""
|
||||
import argparse
|
||||
import base64
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import subprocess
|
||||
import tempfile
|
||||
|
||||
import enroll_consumers
|
||||
|
||||
PATHS = ('.circleci/config.yml', '.qqq-ci.yml')
|
||||
|
||||
|
||||
def command(args, cwd=None):
|
||||
"""Use existing credentials and signing configuration without interactive prompts."""
|
||||
env = dict(os.environ, GIT_TERMINAL_PROMPT='0')
|
||||
result = subprocess.run(args, cwd=cwd, env=env, capture_output=True,
|
||||
text=True, encoding='utf-8', timeout=120)
|
||||
if result.returncode:
|
||||
raise ValueError('Command failed: ' + args[0])
|
||||
return result.stdout.strip()
|
||||
|
||||
|
||||
def api(path, missing=False):
|
||||
"""Only an explicit 404 is treated as absence; other API failures stop work."""
|
||||
result = subprocess.run(['gh', 'api', path], capture_output=True,
|
||||
text=True, encoding='utf-8', timeout=60)
|
||||
try:
|
||||
data = json.loads(result.stdout)
|
||||
except ValueError:
|
||||
raise ValueError('Invalid GitHub response') from None
|
||||
if result.returncode:
|
||||
if missing and str(data.get('status')) == '404':
|
||||
return None
|
||||
raise ValueError('GitHub read unavailable: ' + path.split('?')[0])
|
||||
return data
|
||||
|
||||
|
||||
def content(repo, path, head):
|
||||
"""Read a regular declaration at an immutable commit, never at a moving branch."""
|
||||
item = api(f'repos/{repo}/contents/{path}?ref={head}', missing=True)
|
||||
if item is None:
|
||||
return None
|
||||
if item.get('type') != 'file' or item.get('encoding') != 'base64':
|
||||
raise ValueError('Declaration is not a regular file')
|
||||
return base64.b64decode(item['content'])
|
||||
|
||||
|
||||
def inspect(target, version):
|
||||
"""Resolve observed repository identity and reuse the shared enrollment planner."""
|
||||
if not isinstance(target, dict) or set(target) != {'repository', 'branch'}:
|
||||
raise ValueError('Each target requires repository and branch')
|
||||
repo, branch = target['repository'], target['branch']
|
||||
if not isinstance(repo, str) or not re.fullmatch(r'[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+', repo):
|
||||
raise ValueError('Invalid repository')
|
||||
if branch not in ('main', 'develop'):
|
||||
raise ValueError('Enrollment targets current main/develop only')
|
||||
info = api('repos/' + repo)
|
||||
if info['archived']:
|
||||
raise ValueError('Archived repository')
|
||||
head = api(f'repos/{repo}/branches/{branch}')['commit']['sha']
|
||||
originals = {path: content(repo, path, head) for path in PATHS}
|
||||
if originals[PATHS[0]] is None:
|
||||
raise ValueError('No recognized CircleCI entrypoint')
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
(root / '.circleci').mkdir()
|
||||
for path, data in originals.items():
|
||||
if data is not None:
|
||||
(root / path).write_bytes(data)
|
||||
plan = enroll_consumers.plan({'checkout': str(root), 'default_branch': info['default_branch']}, version)
|
||||
replacements = {str(path.relative_to(root)).replace('\\', '/'): new for path, _, new in plan['files']}
|
||||
digest = hashlib.sha256(head.encode() + b''.join(replacements[path] for path in PATHS)).hexdigest()[:20]
|
||||
return {**target, 'head': head, 'originals': originals, 'replacements': replacements,
|
||||
'profile': plan['profile'], 'version': version,
|
||||
'changed': any(originals[path] != replacements[path] for path in PATHS),
|
||||
'automation_branch': f'automation/qqq-enroll-{branch}-{digest}'}
|
||||
|
||||
|
||||
def verify_submission(plan, head):
|
||||
"""Accept only a signed, one-parent commit with exactly the planned changes."""
|
||||
repo = plan['repository']
|
||||
commit = api(f'repos/{repo}/commits/{head}')
|
||||
expected = {path for path in PATHS if plan['originals'][path] != plan['replacements'][path]}
|
||||
if (not commit['commit']['verification']['verified']
|
||||
or [parent['sha'] for parent in commit['parents']] != [plan['head']]
|
||||
or {item['filename'] for item in commit['files']} != expected):
|
||||
raise ValueError('Existing automation branch differs from the signed plan')
|
||||
for path in PATHS:
|
||||
if content(repo, path, head) != plan['replacements'][path]:
|
||||
raise ValueError('Submitted declaration differs from plan')
|
||||
|
||||
|
||||
def apply(plan):
|
||||
"""Never force push; reuse matching submissions after interrupted executions."""
|
||||
if not plan['changed']:
|
||||
return {'state': 'unchanged'}
|
||||
repo, branch = plan['repository'], plan['automation_branch']
|
||||
if api(f'repos/{repo}/branches/{plan["branch"]}')['commit']['sha'] != plan['head']:
|
||||
raise ValueError('Target branch changed after preflight')
|
||||
ref = api(f'repos/{repo}/git/ref/heads/{branch}', missing=True)
|
||||
if ref is None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory) / 'checkout'
|
||||
command(['git', 'clone', '-c', 'core.autocrlf=false', '--depth', '1', '--single-branch', '--branch', plan['branch'],
|
||||
f'git@github.com:{repo}.git', str(root)])
|
||||
if command(['git', 'rev-parse', 'HEAD'], root) != plan['head']:
|
||||
raise ValueError('Target branch moved before checkout')
|
||||
for path in PATHS:
|
||||
target = enroll_consumers.declaration(root, path)
|
||||
current = target.read_bytes() if target.exists() else None
|
||||
if current != plan['originals'][path]:
|
||||
raise ValueError('Checkout differs from immutable preflight')
|
||||
target.write_bytes(plan['replacements'][path])
|
||||
command(['git', 'add', '--', *PATHS], root)
|
||||
command(['git', 'diff', '--cached', '--check'], root)
|
||||
command(['git', 'commit', '-S', '-m', 'ci: enroll in the shared QQQ orb', '-m',
|
||||
'Co-Authored-By: Codex <noreply@openai.com>'], root)
|
||||
head = command(['git', 'rev-parse', 'HEAD'], root)
|
||||
if api(f'repos/{repo}/branches/{plan["branch"]}')['commit']['sha'] != plan['head']:
|
||||
raise ValueError('Target branch moved before submission')
|
||||
command(['git', 'push', 'origin', f'HEAD:refs/heads/{branch}'], root)
|
||||
else:
|
||||
head = ref['object']['sha']
|
||||
verify_submission(plan, head)
|
||||
pulls = api(f'repos/{repo}/pulls?state=all&head={repo.split("/")[0]}:{branch}&base={plan["branch"]}')
|
||||
if pulls:
|
||||
if len(pulls) != 1 or pulls[0]['head']['sha'] != head:
|
||||
raise ValueError('Ambiguous existing enrollment PR')
|
||||
return {'state': 'existing-pr', 'head': head, 'url': pulls[0]['html_url']}
|
||||
body = ('Uses the shared orb enrollment converter and preserves existing publication policy. '
|
||||
'Only the canonical caller and manifest change. Exact-head hosted checks must pass '
|
||||
'before merge; branch enforcement follows successful qualification.\n\n'
|
||||
'Source base: ' + plan['head'])
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory) / 'body.md'
|
||||
path.write_text(body, encoding='utf-8')
|
||||
url = command(['gh', 'pr', 'create', '--repo', repo, '--base', plan['branch'], '--head', branch,
|
||||
'--title', 'ci: enroll in the shared QQQ orb', '--body-file', str(path)])
|
||||
return {'state': 'submitted', 'head': head, 'url': url}
|
||||
|
||||
|
||||
def main():
|
||||
"""Preflight every explicit target before mutations and save resumable results."""
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
source = parser.add_mutually_exclusive_group(required=True)
|
||||
source.add_argument('--targets', type=Path)
|
||||
source.add_argument('--organization')
|
||||
parser.add_argument('--orb-version', required=True)
|
||||
parser.add_argument('--apply', action='store_true')
|
||||
parser.add_argument('--merge-ready', action='store_true')
|
||||
parser.add_argument('--enforce', action='store_true')
|
||||
parser.add_argument('--output', type=Path, required=True)
|
||||
args = parser.parse_args()
|
||||
if (args.merge_ready or args.enforce) and not args.apply:
|
||||
parser.error('--merge-ready and --enforce require --apply')
|
||||
if not re.fullmatch(r'\d+\.\d+(?:\.\d+)?', args.orb_version):
|
||||
raise ValueError('A stable minor track or exact patch is required')
|
||||
findings = []
|
||||
if args.organization:
|
||||
from discover_enrollment import discover
|
||||
plans, findings = discover(args.organization, args.orb_version, inspect, api)
|
||||
else:
|
||||
targets = json.loads(args.targets.read_text(encoding='utf-8'))
|
||||
if not isinstance(targets, list) or not targets:
|
||||
raise ValueError('A nonempty target list is required')
|
||||
plans = [inspect(target, args.orb_version) for target in targets]
|
||||
if len({(p['repository'].lower(), p['branch']) for p in plans}) != len(plans):
|
||||
raise ValueError('Duplicate target')
|
||||
rows = [{key: value for key, value in p.items() if key not in ('originals', 'replacements')} for p in plans]
|
||||
report = findings + rows
|
||||
args.output.write_text(json.dumps(report, indent=2), encoding='utf-8')
|
||||
if args.apply:
|
||||
for plan, row in zip(plans, rows):
|
||||
row['result'] = apply(plan)
|
||||
args.output.write_text(json.dumps(report, indent=2), encoding='utf-8')
|
||||
if args.merge_ready or args.enforce:
|
||||
import finish_enrollment
|
||||
if args.merge_ready:
|
||||
row['qualification'] = finish_enrollment.merge_ready(plan, row['result'])
|
||||
args.output.write_text(json.dumps(report, indent=2), encoding='utf-8')
|
||||
if args.enforce:
|
||||
row['enforcement'] = finish_enrollment.enforce_current(plan)
|
||||
args.output.write_text(json.dumps(report, indent=2), encoding='utf-8')
|
||||
print(json.dumps({'targets': len(plans), 'changed': sum(p['changed'] for p in plans),
|
||||
'unresolved': sum(row['state'] == 'unresolved' for row in findings),
|
||||
'applied': args.apply}))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -56,8 +56,8 @@ def plan(checkout, version):
|
||||
|
||||
|
||||
def rollout(checkouts, version, write=False):
|
||||
if not re.fullmatch(r'\d+\.\d+\.\d+', version):
|
||||
raise ValueError('Fleet upgrades require an explicit stable orb version')
|
||||
if not re.fullmatch(r'\d+\.\d+(?:\.\d+)?', version):
|
||||
raise ValueError('Fleet upgrades require a stable minor track or exact patch version')
|
||||
# Validate the entire batch before writing any repository.
|
||||
plans = [plan(path, version) for path in checkouts]
|
||||
if len({item['checkout'] for item in plans}) != len(plans):
|
||||
|
||||
@@ -100,8 +100,8 @@ def test_filters(filters):
|
||||
|
||||
|
||||
def infer_manifest(config, version, default_branch):
|
||||
if not re.fullmatch(r"(?:\d+\.\d+\.\d+|dev:[A-Za-z0-9_-]+)", version):
|
||||
raise ValueError("An explicit published orb version is required")
|
||||
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+(?:\.\d+)?|dev:[A-Za-z0-9_-]+)", version):
|
||||
raise ValueError("A minor track, exact patch or explicit development orb version is required")
|
||||
if default_branch not in ("main", "develop"):
|
||||
raise ValueError("Unsupported default branch")
|
||||
if set(config) - {"version", "orbs", "commands", "workflows"}:
|
||||
|
||||
Reference in new issue
Block a user