Compare commits

..
Author SHA1 Message Date
james.maesandCodex 8c0530ee74 fix: reject empty ref responses across jq versions
Co-Authored-By: Codex <noreply@openai.com>
2026-10-21 02:13:02 -05:00
james.maesandCodex 225186356a fix: distinguish missing rc refs from github error bodies
Co-Authored-By: Codex <noreply@openai.com>
2026-10-21 02:09:46 -05:00
james.maesandCodex edc1365ac9 fix: recover verified rc uploads after publisher timeout (#56)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 02:02:14 -05:00
james.maesandCodex 0f33ddadf8 ci: run business E2E on changes and preserve main validation (#55)
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 01:19:08 -05:00
james.maesandCodex 5e9bbb54ec ci: add batch version rollout for canonical consumers (#54)
Validate every checkout before updating its orb pin. Preserve publication policy, filters, credentials contexts and forwarded parameters; reject custom callers and mismatched declarations. Cover all 12 profiles, dry runs, no-op repeats, invalid batches and duplicate input with six new tests in the existing contract job. Existing migration and standard tests pass; Linux packed generator check passes.

Co-authored-by: Codex <noreply@openai.com>
2026-10-21 00:45:50 -05:00
james.maesandCodex 5bcec90e33 ci: report the shared database gate on every pull request (#53)
Preserve migration verification, avoid duplicate feature push builds, and cancel superseded pull request runs so the gate can be required safely.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 22:25:38 -05:00
james.maesandCodex a50b07d927 fix: escape Android heredoc in continued Circle configuration (#51)
Circle accepts the continuation request before reporting an unescaped heredoc as a pipeline configuration error. Escape the template marker and emulate Circle unescaping in executable publisher tests. Validate the actual continued config with CircleCI.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 22:00:56 -05:00
james.maesandCodex 6318d90e96 feat: centralize lightweight GitOps rendering validation (#50)
Add a read-only native worker and canonical consumer bridge. Verify the pinned Kustomize binary, render every configuration without logging resources, and audit adoption. Behavioral and audit tests pass; all five marketing GitOps configurations render successfully.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:54:25 -05:00
james.maesandCodex a6fea661ff fix: authenticate Android publishing with GitHub package token (#47)
* fix: authenticate Android publishing with GitHub package token

Use the shared GitHub Packages token and verify its identity and registry repository access before Gradle starts. Preserve unit and emulator gates; package write success still requires hosted publication.

Co-Authored-By: Codex <noreply@openai.com>

* fix: keep embedded JSON resources consistent across checkouts

Co-Authored-By: Codex <noreply@openai.com>

---------

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:40:01 -05:00
james.maesandCodex 2894bae29a test: retain All release shell behavior in shared owner (#46)
Move the consumer publication regression to the reusable worker; existing test discovery executes it. Verify exact RC1 prerelease/non-latest flags, GA behavior and rejection of other RC versions.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:19:09 -05:00
james.maesandCodex a14e8f9b24 fix: default All releases to shared Circle smoke evidence (#35)
Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:12:58 -05:00
james.maesandCodex 83611872a3 fix: verify public All smoke evidence without secret export (#44)
Allow unauthenticated Circle API reads for public projects, preserve optional authentication and all fail-closed provenance checks.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:12:00 -05:00
james.maesandCodex 30c4e0e85c fix: use shared Maven credentials for Android publication (#42)
Use the existing organization context and Gradle environment properties. Fail before publishing if either credential is absent and avoid writing secrets to gradle.properties.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 21:07:21 -05:00
james.maesandCodex f429d9f3c4 fix: build marketing site before browser checks (#40)
Run lint and one production build in every shared marketing workflow before Playwright and Lighthouse. Preserve the delegated delivery contracts.

Co-authored-by: Codex <noreply@openai.com>
2026-10-20 20:56:32 -05:00
102 changed files with 1803 additions and 9022 deletions

No files matched your search

+1 -34
View File
@@ -4,9 +4,6 @@ parameters:
run_rc_finalize_sweep:
type: boolean
default: false
run_enrollment_reconcile:
type: boolean
default: false
orbs:
orb-tools: circleci/orb-tools@12.3
shellcheck: circleci/shellcheck@3.2
@@ -16,23 +13,6 @@ filters: &filters
only: /.*/
jobs:
enrollment-reconcile:
docker:
- image: cimg/python:3.12
resource_class: small
steps:
- checkout
- run:
name: Install reconciliation tools
command: |
bash tools/install_reconcile_gh.sh
python3 -m pip install --disable-pip-version-check PyYAML==6.0.3
- run:
name: Reconcile current QQQ branches
no_output_timeout: 21m
command: python3 tools/run_enrollment.py
- store_artifacts:
path: artifacts/enrollment
rc-finalize-sweep:
docker:
- image: cimg/python:3.12
@@ -48,10 +28,7 @@ jobs:
workflows:
lint-pack:
when:
not:
or:
- << pipeline.parameters.run_rc_finalize_sweep >>
- << pipeline.parameters.run_enrollment_reconcile >>
not: << pipeline.parameters.run_rc_finalize_sweep >>
jobs:
- orb-tools/lint:
filters: *filters
@@ -79,13 +56,3 @@ workflows:
only: main
tags:
ignore: /.*/
reconcile-enrollment:
when: << pipeline.parameters.run_enrollment_reconcile >>
jobs:
- enrollment-reconcile:
context: [qqq-maven-registry-credentials, orb-publishing]
filters:
branches:
only: main
tags:
ignore: /.*/
-24
View File
@@ -31,9 +31,6 @@ jobs:
- run:
name: Verify resolved multi-module publication POMs
command: python3 tests/test_maven_publication_poms.py
- run:
name: Verify calculated versions match installed Maven coordinates
command: python3 tests/test_effective_publication_version.py
pipeline-contract-test:
resource_class: small
docker:
@@ -47,35 +44,16 @@ jobs:
name: Verify pipeline selection and Maven compatibility contracts
command: |
python3 tests/test_generate_pipeline.py
python3 tests/test_marketing_link_check.py
python3 scripts/pack_pipeline.py --check
bash tests/test_cli_ci_behavior.sh
python3 tests/test_cli_ci_contract.py
python3 tests/test_maven_ci_contract.py
python3 tests/test_standardize_qbit.py
python3 tests/test_qbit_hotfix_profile.py
python3 tests/test_qbit_feature_profile.py
python3 tests/test_ci_maintenance.py
python3 tests/test_rollout_version.py
python3 tests/test_enforce_orb_checks.py
python3 tests/test_native_next_gates.py
python3 tests/test_native_gitops_gates.py
python3 tests/test_native_specialist_gates.py
python3 tests/test_enroll_consumers.py
python3 tests/test_reconcile_enrollment.py
python3 tests/test_discover_enrollment.py
python3 tests/test_finish_enrollment.py
python3 tests/test_run_enrollment.py
python3 tests/test_install_reconcile_gh.py
python3 tests/test_circle_project_settings.py
python3 tests/test_static_analysis_exit.py
python3 tests/test_validate_standard.py
python3 tests/test_voyage_profile.py
python3 tests/test_next_native.py
python3 tests/test_auxiliary_workflows.py
python3 tests/test_qbit_codeql.py
sudo apt-get update -qq && sudo apt-get install -y --no-install-recommends shellcheck
python3 tests/test_repository_policy.py
python3 tests/test_intellij_plugin.py
python3 tests/test_workflow_lint.py
python3 tests/test_business_e2e.py
@@ -99,8 +77,6 @@ jobs:
bash tests/test_maven_ci_cache.sh
bash tests/test_maven_test_results.sh
bash tests/test_maven_ci_install.sh
bash tests/test_browser_install.sh
bash tests/test_frontend_build_reuse.sh
bash tests/test_snapshot_refresh.sh
bash tests/test_maven_settings_output.sh
# Create jobs to test the commands of your orbs.
-58
View File
@@ -1,58 +0,0 @@
name: QBit CodeQL
'on':
workflow_call:
permissions:
contents: read
actions: read
security-events: write
packages: read
jobs:
analyze:
name: Analyze branch source
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- name: Select languages present in this checkout
id: languages
run: |
python3 <<'PY'
import os
from pathlib import PurePosixPath
import subprocess
import sys
files = subprocess.check_output(['git', 'ls-files', '-z']).decode('utf-8').split('\0')
extensions = {PurePosixPath(name).suffix.lower() for name in files if name}
languages = []
if extensions & {'.java', '.kt', '.kts'}:
languages.append('java-kotlin')
if extensions & {'.js', '.jsx', '.mjs', '.cjs', '.ts', '.tsx', '.mts', '.cts'}:
languages.append('javascript-typescript')
if '.py' in extensions:
languages.append('python')
if any((name.startswith('.github/workflows/') and PurePosixPath(name).suffix in {'.yml', '.yaml'})
or PurePosixPath(name).name in {'action.yml', 'action.yaml'} for name in files if name):
languages.append('actions')
if not languages:
sys.exit('No supported source in this branch; classify placeholders before enabling QBit scanning')
build_mode = 'autobuild' if extensions & {'.kt', '.kts'} else 'none'
with open(os.environ['GITHUB_OUTPUT'], 'a', encoding='utf-8') as output:
output.write('languages=' + ','.join(languages) + '\n')
output.write('build-mode=' + build_mode + '\n')
PY
- uses: actions/setup-java@v4
if: steps.languages.outputs.build-mode == 'autobuild'
with:
distribution: temurin
java-version: '21'
- name: Initialize CodeQL
uses: github/codeql-action/init@v4
with:
languages: ${{ steps.languages.outputs.languages }}
build-mode: ${{ steps.languages.outputs.build-mode }}
- name: Build Kotlin sources
if: steps.languages.outputs.build-mode == 'autobuild'
uses: github/codeql-action/autobuild@v4
- name: Analyze source
uses: github/codeql-action/analyze@v4
+5 -5
View File
@@ -135,7 +135,7 @@ jobs:
echo "merge_sha=$merge_sha"
echo "merged_at=$merged_at"
} >> "$GITHUB_OUTPUT"
echo "Pushed $branch; the CircleCI publisher cuts the RC." >> "$GITHUB_STEP_SUMMARY"
echo "Pushed $branch; CircleCI publish_release_candidate cuts the RC." >> "$GITHUB_STEP_SUMMARY"
echo "Release line: $branch. Waiting without a runner for qbit-rc-published." >> "$GITHUB_STEP_SUMMARY"
- name: Persist trusted release intent
if: steps.merge.outputs.merge_sha != ''
@@ -203,7 +203,7 @@ jobs:
raise ReleaseFailure("Status response does not identify the exact release merge")
# Circle's workflow Check Run is stable even when job statuses receive -1/-2 suffixes.
publishers = [x for x in status.get("check_runs", [])
if x.get("name") in ("publish_release_candidate", "verify") and x.get("app", {}).get("id") == 18001]
if x.get("name") == "publish_release_candidate" and x.get("app", {}).get("id") == 18001]
if not publishers:
return ""
publisher = max(publishers, key=lambda x: x["id"])
@@ -265,10 +265,10 @@ jobs:
job.get('branch'), str(job.get('build_num'))) != (repo, merge, branch, build):
raise ReleaseFailure('Timeout job does not identify the exact cut')
workflow = job.get('workflows', {})
if workflow.get('workflow_name') not in ('publish_release_candidate', 'verify') or workflow.get('job_name') != 'qqq-orb/mvn_publish':
if workflow.get('workflow_name') != 'publish_release_candidate' or workflow.get('job_name') != 'qqq-orb/mvn_publish':
raise ReleaseFailure('Unexpected Circle publication workflow/job')
if (publisher.get('head_sha') != merge or publisher.get('app', {}).get('id') != 18001
or publisher.get('name') != workflow.get('workflow_name')
or publisher.get('name') != 'publish_release_candidate'
or publisher.get('status') != 'completed' or publisher.get('conclusion') != 'failure'):
raise ReleaseFailure('Recovery requires the failed exact Circle publisher')
from urllib.parse import urlsplit
@@ -335,7 +335,7 @@ jobs:
directory = pathlib.Path(directory)
status = json.loads((directory / 'status.json').read_text())
intent = json.loads((directory / 'intent.json').read_text())
publishers = [p for p in status['check_runs'] if p.get('name') in ('publish_release_candidate', 'verify') and p.get('app', {}).get('id') == 18001]
publishers = [p for p in status['check_runs'] if p.get('name') == 'publish_release_candidate' and p.get('app', {}).get('id') == 18001]
if status.get('sha') != merge or not publishers:
raise ReleaseFailure('Exact publisher evidence missing')
class NoRedirect(urllib.request.HTTPRedirectHandler):
-11
View File
@@ -58,21 +58,11 @@ test-pipelines: pack
bash tests/test_cli_ci_behavior.sh
python3 tests/test_cli_ci_contract.py
python3 tests/test_standardize_qbit.py
python3 tests/test_qbit_hotfix_profile.py
python3 tests/test_ci_maintenance.py
python3 tests/test_rollout_version.py
python3 tests/test_enforce_orb_checks.py
python3 tests/test_enroll_consumers.py
python3 tests/test_reconcile_enrollment.py
python3 tests/test_discover_enrollment.py
python3 tests/test_finish_enrollment.py
python3 tests/test_static_analysis_exit.py
python3 tests/test_validate_standard.py
python3 tests/test_voyage_profile.py
python3 tests/test_next_native.py
python3 tests/test_auxiliary_workflows.py
python3 tests/test_qbit_codeql.py
python3 tests/test_repository_policy.py
python3 tests/test_intellij_plugin.py
python3 tests/test_workflow_lint.py
python3 tests/test_business_e2e.py
@@ -226,4 +216,3 @@ publish-release: check-clean check-branch lint clean dev test-all
# an isolated temporary repository. Never publishes or uses signing credentials.
test-publication-poms:
python3 tests/test_maven_publication_poms.py
python3 tests/test_effective_publication_version.py
-22
View File
@@ -4,8 +4,6 @@ CircleCI orb for QQQ framework projects.
For centralized pipeline ownership, see [standardized manifest-driven pipelines](docs/GENERATED-PIPELINES.md).
The initial QBit profile uses a small `.qqq-ci.yml` and a setup entry point; existing direct job callers remain supported.
For centrally managed patch adoption and exact-version rollback, see
[managed patch updates](docs/managed-patch-updates.md).
**For:** Teams using CircleCI to build, test, and publish QQQ applications
**Status:** Stable (v0.6.x)
@@ -89,11 +87,6 @@ real publication, not a dry run, verifies the trust relationship.
| `hotfix/1.5.1` | `1.5.1` |
| `feature/*` | `1.5.0-feature-abc1234-SNAPSHOT` |
The Maven version step verifies the effective `project.version`, not only the
`revision` property. For older POMs with a literal project version, it uses
Maven's version plugin to align project and reactor coordinates with the
calculated version. A mismatch after updating fails before publication.
### Java Version
Maven jobs run on Ubuntu machine executors with Temurin JDK. Default is Java 21. To use Java 17:
@@ -121,21 +114,6 @@ Maven jobs use machine executors with Docker pre-installed. Tests using Testcont
## Orb Publishing
### QBit CI maintenance
The QBit profile runs Maven verification without publishing when a single-parent
`ci: ...` or `ci(scope): ...` commit changes only `.qqq-ci.yml`,
`.circleci/config.yml`, `.github/workflows/codeql.yml`, or
`.github/workflows/release-rc.yml`. This applies on
main, develop, release, and hotfix branches. The normal test job still runs,
including on branches normally excluded from feature tests, and reports through
the required `verify` workflow.
Tags, merge commits (including release automation), application changes, other
files, empty diffs, and unavailable history retain normal publishing behavior.
This lets central configuration updates avoid unnecessary application versions
without introducing per-repository maintenance workflows.
The orb auto-publishes via CircleCI:
| Branch | Action |
-50
View File
@@ -3,18 +3,6 @@
`tools/audit_estate.py` classifies repositories from explicit mappings and observed source.
It never changes consumers, branch rules or publication settings.
Source auditing reads the repository's current default branch unless
`--live --source-branch main` (or an explicit `release/...` or `hotfix/...` branch)
is supplied. Run separate audits for each maintained source branch: a clean
default-branch report does not establish conformity on main or release branches.
The selected branch is resolved to an immutable commit before any files are read.
Reports identify both `source_branch` and the repository's `default_branch`;
the manifest's `default_branch` must still match the repository default.
Unavailable selected branches remain unknown without falling back to default.
Explicit branch selection requires live lookup and cannot relabel an offline SHA.
Required-check policy still controls which branch settings are inspected;
`--source-branch` changes source selection, not that enforcement policy.
- Circle profiles include android, mcp-policy and marketing. Their manifest and canonical
setup caller remain mandatory; a missing configuration is a finding.
- MCP, business-platform and the organization `.github` repository use approved native
@@ -80,14 +68,6 @@ required-check unknown states, and the lint image/tool/command/caller contract.
## Batch orb-version rollout
For a standard QBit without a manifest, start directly on the managed patch
track using `python tools/standardize_qbit.py /checkout --orb-version 0.8
--default-branch main`. Inspect the preview, then repeat with `--write`.
This preserves the inferred publication and branch-filter policy while writing
the same track to the manifest and setup caller. Use `rollout_version.py` below
for repositories that already have a manifest. Neither command configures
repository protection or automatically enrolls newly created repositories.
Upgrade existing canonical CircleCI consumers with one command:
```sh
@@ -111,33 +91,3 @@ verification and artifact evidence. The requested version must already be publis
```sh
python3 -m unittest discover -s tests -p test_rollout_version.py
```
## Live organization discovery
Use `--organization QRun-IO --live` instead of `--inventory inventory.json` to
include newly created repositories on each audit. Discovery uses the existing
`gh` authentication and all API pages, retaining archived entries for explicit
skip reporting. Unknown repositories receive no automatic exemption. Duplicate,
empty, malformed or failed discovery aborts instead of reusing a stale inventory.
```sh
python tools/audit_estate.py --organization QRun-IO --live --orb-version 0.8 \
--worker-sha <40SHA> --policy observed-checks.json --output audit.json
```
The organization mode has no fixed repository-count ceiling. Supply
`--expected-active N` for an independent count check when needed; saved-inventory
mode preserves its default of 68. Reports identify the inventory source and
credential visibility limit: inaccessible repositories cannot be certified.
This is discovery and audit, not automatic enrollment or branch-rule mutation.
For a qualified batch of canonical orb branches, use the shared
[branch-check enforcement command](branch-enforcement.md). It adds the observed
CircleCI setup/verify gates while preserving other protections. Source enrollment,
scanner transitions and scheduled reconciliation remain separate requirements.
Scheduling and authenticated onboarding still need to be configured centrally.
## Batch enrollment
Use [the shared enrollment command](consumer-enrollment.md) to convert recognized
legacy QBits and update existing canonical consumers in one validated batch.
+224 -248
View File
@@ -1,248 +1,224 @@
# Standardized QQQ pipelines
The orb owns workflow selection and job configuration. A consumer declares a
small manifest; it does not copy pipeline jobs. The initial supported profile is
`qbit`; `core` owns the framework's complete acceptance/release graph. Frontend
and CLI profiles described below retain their own publication contracts. Additional
profiles belong in reviewed `PROFILE_TEMPLATES` renderers in
`src/scripts/generate_pipeline.py`, with profile-specific schema and tests. There
is no arbitrary script or job-YAML passthrough.
Use the same **published** orb version in all three positions below. `0.7.0` is
an illustrative version, not a claim that this change is published. A development
orb version can be used for a controlled canary.
`.circleci/config.yml`:
```yaml
version: 2.1
setup: true
orbs:
qqq-orb: kingsrook/qqq-orb@0.7.0
workflows:
setup:
jobs:
- qqq-orb/generate_pipeline:
orb_version: "0.7.0"
filters:
branches:
ignore: automation/qbit-rc-pending
tags:
only: /.*/
```
`.qqq-ci.yml`:
```yaml
profile: qbit
orb_version: "0.7.0"
default_branch: develop
java_version: "21"
browsers: false
api_checks: false
docs_only: false
publish:
snapshot: true
release_candidate: true
release: true
asciidoc: false
```
Only `profile`, `orb_version` and `default_branch` are required. `default_branch`
must be `develop` or `main`; Java must be the string `17` or `21`. Browsers default
to **true** for compatibility: opt out only after verifying that the consumer has
no browser tests. API checks and docs-only default to false. Publication defaults
to true for snapshots, RCs and tagged releases; AsciiDoc publication defaults to
false. Unknown/duplicate keys, unexpected types, unsupported profiles and a setup
parameter/manifest version mismatch fail before continuation. The canonical-caller
validator also verifies the setup import, filters and profile-specific parameter
declarations; additional jobs, commands, workflows or orbs fail validation. The
parser is pinned to PyYAML 6.0.3 and uses its safe loader.
| Event | Workflow | Job contract |
| --- | --- | --- |
| develop | verify | existing mvn_publish, snapshot |
| release/* | verify | existing mvn_publish, release_candidate |
| vX.Y.Z tag resolving to the checkout | verify | existing mvn_publish, release |
| main, feature/*, hotfix/*, other/unknown branch | verify | existing mvn_test_only |
| Disabled publication or unrecognized/unproven tag | verify | full verification, no publication |
Generated `qbit`, `core`, `frontend-core`, and `frontend-material` pipelines use
the stable workflow name `verify`. All selected jobs, dependencies and filters
remain intact; this adds no executor or duplicate build. Other profiles retain
their existing workflow names and native checks.
When upgrading from legacy workflow names, upgrade the immutable RC worker pin
alongside the orb so finalization recognizes both names. Confirm the CircleCI
app's successful `verify` check on the exact migration commit before replacing
legacy required contexts. Require `setup` and `verify` on managed branches,
preserving other required checks and controls. Human approval is not required;
automated checks authorize the migration.
The pipeline SHA must match the checkout when Circle supplies it. A tagged release
must resolve to that checkout; missing local tags are fetched individually. A main
push does **not** publish a release. Migrating a legacy main-publishing consumer
therefore requires explicit review of its release contract. Existing direct orb
callers remain supported.
The setup filter and generated jobs exclude `automation/qbit-rc-pending`, the
shared RC orchestrator's persistent release lock. Lock updates must not trigger
Maven test or publication work.
The shared jobs keep the `qqq-maven-registry-credentials` context, complete verify,
coverage and configured API checks. Publishing retains version management, signed
artifact deployment, Git operations and release handling. Generated calls disable
the redundant preliminary compile; they do not skip tests or verification. Existing
callers retain the compatibility defaults in the jobs.
## Core profile
QBit alone accepts `release_completion: true` (default false). It enables the
shared Maven RC job's `notify_rc_published` only for RC publication, including
fail-closed verification that the bumped version was pushed before dispatching
the completion event. Enable this on a consumer only after its default branch
contains the shared completion caller. Snapshot and GA jobs never receive this
opt-in; other profiles reject the key.
Use `profile: core` for the QQQ framework. The orb owns the complete baseline
acceptance, broker conformance, cold bootstrap, coverage, diagnostics, static
analysis and publication graph. Core scripts remain in the core source repository.
The independent baseline fixture pins job sets, dependency edges, contexts,
coverage requirements and actual verification commands across all six workflows.
Core additionally supports `publish.feature` and `publish.hotfix` (both true by
default), preserving `publish*` tag and hotfix branch publication. QBit supports
both keys as explicit opt-ins (false by default). QBit feature publishing requires
a `publish*` tag resolving to the checked-out commit; feature branch pushes remain
test-only. Snapshot, RC and GA behavior remains distinct; RC/GA/hotfix
publication still requires the complete source coverage gate. Cold bootstrap and
fresh consumer Maven repositories remain uncached. Shared installer policy replaces
local package preparation; preliminary compilation is disabled before clean
lifecycles and the dependency cache is saved after successful verification.
Lambda XML copies remain in sample coverage evidence. The sample JUnit collector
excludes those two copied reports because their original module directories are
already uploaded, avoiding duplicate test identities without dropping evidence.
External core API triggers require these declarations in the tiny **setup** config:
```yaml
parameters:
mongo_bind_diagnostics:
type: boolean
default: false
next_acceptance_bundle_url:
type: string
default: ""
next_acceptance_bundle_sha256:
type: string
default: ""
next_acceptance_receipt_sha256:
type: string
default: ""
```
Forward each value on `qqq-orb/generate_pipeline`, for example
`mongo_bind_diagnostics: << pipeline.parameters.mongo_bind_diagnostics >>`, and
likewise for the three bundle parameters. Requested values become defaults in the
continued configuration; the continuation POST deliberately omits `parameters` to
avoid CircleCI's setup/continuation/API parameter conflict. Verify both default and
explicit API-triggered parameter values in the hosted canary before migrating
triggering consumers. Other profiles reject non-default core parameters.
## Frontend profiles
`frontend-material` preserves the Maven frontend build/test/package/publication
jobs, Java21/Node20 defaults, registry context, feature-tag, RC, GA and hotfix
contracts. Declare setup pipeline parameter `run_playwright` (boolean, default
false) and forward `run_playwright: << pipeline.parameters.run_playwright >>` on
the generator job. Test/snapshot events select exactly one base or Playwright graph,
so enabling visual tests cannot publish twice. Visual testing retains the pinned
Playwright image, npm ci with legacy peer dependencies and report uploads; only
npm downloads (`~/.npm`) are cached, never node_modules.
`frontend-core` preserves Node24.21, npm trusted publishing and both existing
registry/build-sample contexts. Unlike QBit, its existing main-push **and v-tag**
release contract remains intentional. Both frontend profiles support explicit
numeric-string `node_version` and publication toggles including feature/hotfix.
Material also supports `java_version`. Browser/API/AsciiDoc flags are rejected
rather than silently ignored on these profiles. Existing direct jobs remain usable.
## CLI profile
`cli` owns npm install, generated-client drift, build, unit tests and the real
Prism smoke contract through `cli_verify_publish`. Publication is **disabled by
default**. To enable the existing v-tag publication, declare `publish.release:
true` plus `publish_context: VERIFIED_CONTEXT_NAME`, after verifying that an
existing trusted CircleCI context grants the required GitHub Packages permission.
No context name or credential availability is inferred from the former Actions
GITHUB_TOKEN. A tag must resolve to the current checkout; the job additionally
checks the client version before publishing its three fixed workspaces with the
existing rc dist-tag. Main pushes never publish. CLI docs-only is currently
rejected because generated-client verification remains mandatory.
## Conservative documentation routing
With `docs_only: true`, only feature/hotfix and main changes can take the light
route. Feature/hotfix branches fetch the declared default branch without tags and
compare its merge base with HEAD, covering the whole branch. Main compares its first
parent. Develop, release branches and all tags remain full. Shallow repositories,
missing ancestry, fetch/diff errors, empty diffs and unknown paths remain full.
Only root README.md, CHANGELOG.md, CONTRIBUTING.md and Markdown files below docs/
are eligible. Renames are treated as deletion plus addition, so moving code into
docs cannot hide it. The lightweight job checks whitespace over the proven diff;
manifest validation already ran in setup. It adds no new security scanner and
makes no security-scanning claim. Keep docs-only disabled if those paths are build
inputs or packaged application content.
The light route keeps workflow `verify` and names its job
`qqq-orb/mvn_test_only`. Verify actual CircleCI/GitHub status contexts on the canary
before changing any protection rule; workflow/job identity alone is not proof of
the hosted context name. Current protections must not be removed speculatively.
## Implementation and verification
The setup job checks out source, installs the pinned parser, runs the embedded
Python resource and invokes CircleCI's native continuation API once. JSON output
is valid CircleCI configuration YAML; it is saved at
`/tmp/qqq-generated-config.json`. No external framework or per-consumer generator
installation is required. Readable templates live in src/scripts/templates. scripts/pack_pipeline.py embeds
them as base64 into packed_generate_pipeline.py before orb packing, preventing
CircleCI from interpolating template expressions during setup. The orb includes
the complete generated resource; no runtime source-tree template access is needed.
make pack regenerates it and CI rejects a stale generated resource.
Continuation credentials and response bodies are never printed.
Run `make test-pipelines` with CircleCI CLI, Python3/PyYAML and Bash available.
Tests use offline temporary Git repositories and a bare origin to cover selection,
whole-branch changes, tags, errors, defaults and source/packed parity. The hosted
pipeline-contract-test runs the source suite and shared Maven behavioral tests;
the full local target additionally extracts and executes the actual packed command.
## Node snapshot versions
The shared Node publisher calculates develop snapshots as
`MAJOR.MINOR.NEXT_PATCH-SNAPSHOT.CIRCLE_BUILD_NUM`. A positive CircleCI job number
is required, so independent publication jobs receive distinct npm coordinates.
The package version is changed only in the build workspace and published under
the `snapshot` dist-tag. No snapshot version commit is pushed: the branch remains
at the source revision whose required checks ran, avoiding unchecked `[skip ci]`
heads and recursive publication. Stable, RC and hotfix version policies are unchanged.
## Android and MCP policy profiles
`android` and `mcp-policy` accept only profile, orb_version and default_branch.
Android retains the audited Android/GCP orb versions and every active unit,
emulator and Maven-publication step. The emulator now runs on main as well as
feature branches so main publication cannot omit its required instrumentation
gate. Dormant Firebase benchmark and release-build definitions referencing absent
app/macrobenchmark modules are not scheduled or copied into the profile.
MCP policy validation remains a small Python container validating all visible YAML
and required pattern fields. Its existing PyYAML dependency is pinned to 6.0.3.
Neither profile introduces Maven/JUnit stand-ins for its actual tests.
`tests/test_mobile_policy_profiles.py` protects Android steps and dependency edges
and executes the policy validator with valid, incomplete and malformed documents.
The generator suite verifies source/packed parity for both profiles.
# Standardized QQQ pipelines
The orb owns workflow selection and job configuration. A consumer declares a
small manifest; it does not copy pipeline jobs. The initial supported profile is
`qbit`; `core` owns the framework's complete acceptance/release graph. Frontend
and CLI profiles described below retain their own publication contracts. Additional
profiles belong in reviewed `PROFILE_TEMPLATES` renderers in
`src/scripts/generate_pipeline.py`, with profile-specific schema and tests. There
is no arbitrary script or job-YAML passthrough.
Use the same **published** orb version in all three positions below. `0.7.0` is
an illustrative version, not a claim that this change is published. A development
orb version can be used for a controlled canary.
`.circleci/config.yml`:
```yaml
version: 2.1
setup: true
orbs:
qqq-orb: kingsrook/qqq-orb@0.7.0
workflows:
setup:
jobs:
- qqq-orb/generate_pipeline:
orb_version: "0.7.0"
filters:
branches:
ignore: automation/qbit-rc-pending
tags:
only: /.*/
```
`.qqq-ci.yml`:
```yaml
profile: qbit
orb_version: "0.7.0"
default_branch: develop
java_version: "21"
browsers: false
api_checks: false
docs_only: false
publish:
snapshot: true
release_candidate: true
release: true
asciidoc: false
```
Only `profile`, `orb_version` and `default_branch` are required. `default_branch`
must be `develop` or `main`; Java must be the string `17` or `21`. Browsers default
to **true** for compatibility: opt out only after verifying that the consumer has
no browser tests. API checks and docs-only default to false. Publication defaults
to true for snapshots, RCs and tagged releases; AsciiDoc publication defaults to
false. Unknown/duplicate keys, unexpected types, unsupported profiles and a setup
parameter/manifest version mismatch fail before continuation. The canonical-caller
validator also verifies the setup import, filters and profile-specific parameter
declarations; additional jobs, commands, workflows or orbs fail validation. The
parser is pinned to PyYAML 6.0.3 and uses its safe loader.
| Event | Workflow | Job contract |
| --- | --- | --- |
| develop | publish_snapshot | existing mvn_publish, snapshot |
| release/* | publish_release_candidate | existing mvn_publish, release_candidate |
| vX.Y.Z tag resolving to the checkout | publish_release | existing mvn_publish, release |
| main, feature/*, hotfix/*, other/unknown branch | test_only | existing mvn_test_only |
| Disabled publication or unrecognized/unproven tag | test_only | full verification, no publication |
The pipeline SHA must match the checkout when Circle supplies it. A tagged release
must resolve to that checkout; missing local tags are fetched individually. A main
push does **not** publish a release. Migrating a legacy main-publishing consumer
therefore requires explicit review of its release contract. Existing direct orb
callers remain supported.
The setup filter and generated jobs exclude `automation/qbit-rc-pending`, the
shared RC orchestrator's persistent release lock. Lock updates must not trigger
Maven test or publication work.
The shared jobs keep the `qqq-maven-registry-credentials` context, complete verify,
coverage and configured API checks. Publishing retains version management, signed
artifact deployment, Git operations and release handling. Generated calls disable
the redundant preliminary compile; they do not skip tests or verification. Existing
callers retain the compatibility defaults in the jobs.
## Core profile
QBit alone accepts `release_completion: true` (default false). It enables the
shared Maven RC job's `notify_rc_published` only for RC publication, including
fail-closed verification that the bumped version was pushed before dispatching
the completion event. Enable this on a consumer only after its default branch
contains the shared completion caller. Snapshot and GA jobs never receive this
opt-in; other profiles reject the key.
Use `profile: core` for the QQQ framework. The orb owns the complete baseline
acceptance, broker conformance, cold bootstrap, coverage, diagnostics, static
analysis and publication graph. Core scripts remain in the core source repository.
The independent baseline fixture pins job sets, dependency edges, contexts,
coverage requirements and actual verification commands across all six workflows.
Core additionally supports `publish.feature` and `publish.hotfix` (both true by
default), preserving `publish*` tag and hotfix branch publication. QBit does not
accept these keys. Snapshot, RC and GA behavior remains distinct; RC/GA/hotfix
publication still requires the complete source coverage gate. Cold bootstrap and
fresh consumer Maven repositories remain uncached. Shared installer policy replaces
local package preparation; preliminary compilation is disabled before clean
lifecycles and the dependency cache is saved after successful verification.
Lambda XML copies remain in sample coverage evidence. The sample JUnit collector
excludes those two copied reports because their original module directories are
already uploaded, avoiding duplicate test identities without dropping evidence.
External core API triggers require these declarations in the tiny **setup** config:
```yaml
parameters:
mongo_bind_diagnostics:
type: boolean
default: false
next_acceptance_bundle_url:
type: string
default: ""
next_acceptance_bundle_sha256:
type: string
default: ""
next_acceptance_receipt_sha256:
type: string
default: ""
```
Forward each value on `qqq-orb/generate_pipeline`, for example
`mongo_bind_diagnostics: << pipeline.parameters.mongo_bind_diagnostics >>`, and
likewise for the three bundle parameters. Requested values become defaults in the
continued configuration; the continuation POST deliberately omits `parameters` to
avoid CircleCI's setup/continuation/API parameter conflict. Verify both default and
explicit API-triggered parameter values in the hosted canary before migrating
triggering consumers. Other profiles reject non-default core parameters.
## Frontend profiles
`frontend-material` preserves the Maven frontend build/test/package/publication
jobs, Java21/Node20 defaults, registry context, feature-tag, RC, GA and hotfix
contracts. Declare setup pipeline parameter `run_playwright` (boolean, default
false) and forward `run_playwright: << pipeline.parameters.run_playwright >>` on
the generator job. Test/snapshot events select exactly one base or Playwright graph,
so enabling visual tests cannot publish twice. Visual testing retains the pinned
Playwright image, npm ci with legacy peer dependencies and report uploads; only
npm downloads (`~/.npm`) are cached, never node_modules.
`frontend-core` preserves Node24.21, npm trusted publishing and both existing
registry/build-sample contexts. Unlike QBit, its existing main-push **and v-tag**
release contract remains intentional. Both frontend profiles support explicit
numeric-string `node_version` and publication toggles including feature/hotfix.
Material also supports `java_version`. Browser/API/AsciiDoc flags are rejected
rather than silently ignored on these profiles. Existing direct jobs remain usable.
## CLI profile
`cli` owns npm install, generated-client drift, build, unit tests and the real
Prism smoke contract through `cli_verify_publish`. Publication is **disabled by
default**. To enable the existing v-tag publication, declare `publish.release:
true` plus `publish_context: VERIFIED_CONTEXT_NAME`, after verifying that an
existing trusted CircleCI context grants the required GitHub Packages permission.
No context name or credential availability is inferred from the former Actions
GITHUB_TOKEN. A tag must resolve to the current checkout; the job additionally
checks the client version before publishing its three fixed workspaces with the
existing rc dist-tag. Main pushes never publish. CLI docs-only is currently
rejected because generated-client verification remains mandatory.
## Conservative documentation routing
With `docs_only: true`, only feature/hotfix and main changes can take the light
route. Feature/hotfix branches fetch the declared default branch without tags and
compare its merge base with HEAD, covering the whole branch. Main compares its first
parent. Develop, release branches and all tags remain full. Shallow repositories,
missing ancestry, fetch/diff errors, empty diffs and unknown paths remain full.
Only root README.md, CHANGELOG.md, CONTRIBUTING.md and Markdown files below docs/
are eligible. Renames are treated as deletion plus addition, so moving code into
docs cannot hide it. The lightweight job checks whitespace over the proven diff;
manifest validation already ran in setup. It adds no new security scanner and
makes no security-scanning claim. Keep docs-only disabled if those paths are build
inputs or packaged application content.
The light route keeps workflow `test_only` and names its job
`qqq-orb/mvn_test_only`. Verify actual CircleCI/GitHub status contexts on the canary
before changing any protection rule; workflow/job identity alone is not proof of
the hosted context name. Current protections must not be removed speculatively.
## Implementation and verification
The setup job checks out source, installs the pinned parser, runs the embedded
Python resource and invokes CircleCI's native continuation API once. JSON output
is valid CircleCI configuration YAML; it is saved at
`/tmp/qqq-generated-config.json`. No external framework or per-consumer generator
installation is required. Readable templates live in src/scripts/templates. scripts/pack_pipeline.py embeds
them as base64 into packed_generate_pipeline.py before orb packing, preventing
CircleCI from interpolating template expressions during setup. The orb includes
the complete generated resource; no runtime source-tree template access is needed.
make pack regenerates it and CI rejects a stale generated resource.
Continuation credentials and response bodies are never printed.
Run `make test-pipelines` with CircleCI CLI, Python3/PyYAML and Bash available.
Tests use offline temporary Git repositories and a bare origin to cover selection,
whole-branch changes, tags, errors, defaults and source/packed parity. The hosted
pipeline-contract-test runs the source suite and shared Maven behavioral tests;
the full local target additionally extracts and executes the actual packed command.
## Android and MCP policy profiles
`android` and `mcp-policy` accept only profile, orb_version and default_branch.
Android retains the audited Android/GCP orb versions and every active unit,
emulator and Maven-publication step. The emulator now runs on main as well as
feature branches so main publication cannot omit its required instrumentation
gate. Dormant Firebase benchmark and release-build definitions referencing absent
app/macrobenchmark modules are not scheduled or copied into the profile.
MCP policy validation remains a small Python container validating all visible YAML
and required pattern fields. Its existing PyYAML dependency is pinned to 6.0.3.
Neither profile introduces Maven/JUnit stand-ins for its actual tests.
`tests/test_mobile_policy_profiles.py` protects Android steps and dependency edges
and executes the policy validator with valid, incomplete and malformed documents.
The generator suite verifies source/packed parity for both profiles.
+17 -22
View File
@@ -6,18 +6,17 @@ orb version. The canonical QQQ setup produces one final continuation containing
jobs from the pinned `kof22/munitor@0.3.11` orb. It never invokes Munitor's setup
generator or creates another continuation.
The baseline fixture records the original Munitor 0.3.11 render. The shared
profile selects one workflow for the current branch and exposes it as `verify`:
PR checks for feature/hotfix/automation branches, develop, staging, release
candidate, or production for main. Tags and the RC bookkeeping branch do not
run these jobs. Existing deployment dependencies and contexts are preserved.
The profile is the exact semantic graph rendered by Munitor 0.3.11 from
marketing-website commit `4eb5f63b9fbfac8e5529fcd638324870cea08ae2`, with its
manifest's floating `0.3` pin changed to `0.3.11` for the baseline render.
The checked-in fixture records that real render; the JSON resource contains the
same jobs, parameters, contexts, filters and dependency edges.
The build job installs dependencies once, lints, builds, runs Chromium
Playwright tests, crawls local links and runs Lighthouse. The link crawl waits
at most 60 attempts for the app on port 3005, propagates failures, and stops its
server on exit. Playwright and Lighthouse reports are retained as artifacts,
including on failure. The old synthetic coverage job was removed because it
fabricated 100% rather than measuring coverage; actual tests and scans remain.
There are five workflows and 33 job invocations: PR checks (7), develop (7),
staging (7), release candidate (8), and production (4). They preserve Node 22,
Chromium Playwright and Lighthouse, the existing scan and quality jobs, advisory
SAST, and the existing synthetic zero-threshold coverage command. That command
does not measure actual coverage.
Production follows build → image build/Trivy/push/health → independent CD update
and GitHub release. Both final jobs depend on the image job; neither depends on
@@ -27,19 +26,15 @@ existing `github` context; image publication uses `ghcr`. Production-only means
no non-main deployment update, not no non-main artifacts: develop/staging image
publishing and release-candidate GitHub releases remain unchanged.
Consumers need only the QQQ caller and `.qqq-ci.yml`. Before tests, the shared
profile writes minimal `.munitor.yml` worker input into the build workspace:
`pipeline: node-webapp`, Node 22, and `docker.use_repo_dockerfile: true`.
Munitor's Docker worker still requires this file; the orb owns its contents.
This preserves the repository Dockerfile rather than generating a replacement.
Container health remains `/` on port 3000, distinct from the local test server
on port 3005. The repository Dockerfile remains responsible for the standalone
Keep `.munitor.yml` as declarative worker input when migrating the consumer.
Munitor's `generate_dockerfile` reads it from the attached workspace and requires
`pipeline: node-webapp` and `docker.use_repo_dockerfile: true` to preserve the
repository Dockerfile. The profile does not replace that file or validate all
of its values. Changes to worker input require review alongside the centralized
profile; arbitrary manifest job overrides are not supported. Health remains `/`
on port 3000; the repository Dockerfile remains responsible for the standalone
runtime without npm or npx.
The duplicate consumer GitHub test workflow can be removed only after these
central checks qualify. Application test settings and assertions remain in the
application repository; orchestration belongs here.
`MarketingContractTests` checks whole-graph equality and explicit release/test
contracts. The normal generator suite additionally checks source/packed parity.
These are configuration tests, not a claim of hosted delivery, credentials,
-12
View File
@@ -1,12 +0,0 @@
# Shared QBit source scanning
Generate the thin caller with `python3 src/scripts/generate_auxiliary_workflow.py qbit-codeql --ref <immutable-worker-sha>`.
The reusable worker selects supported languages from tracked files in the actual checked-out branch. Historical Java branches therefore do not request an absent Actions database. Java, JavaScript/TypeScript, Python and Actions share one job; Kotlin selects an actual build instead of silently omitting Kotlin from no-build Java analysis.
The caller scans pull requests, main/develop/release/hotfix pushes and weekly, cancels superseded PR scans, and contains no scan implementation. Set `codeql_worker_sha` in the estate enforcement policy to require the exact canonical caller independently of other native worker pins. Existing default-setup repositories remain unchanged until migrated.
Roll out to both active default and maintained branches. Verify the worker on representative branch contents and preserve existing query coverage before replacing GitHub default setup; do not run duplicate default and advanced scanners permanently. Default setup cannot select languages independently for historical branches. A branch with no supported application or workflow source is a placeholder to classify, not a passing application scan.
Local tests execute the embedded language-selection step against real git indexes, including absent Actions, untracked generated files, mixed source, Kotlin and placeholders. Hosted CodeQL database creation and SARIF upload must also pass before claiming rollout completion.
References: [GitHub setup types](https://docs.github.com/en/code-security/concepts/code-scanning/setup-types) and [CodeQL build modes](https://docs.github.com/en/code-security/reference/code-scanning/codeql/build-options-for-compiled-languages).
+2 -24
View File
@@ -8,16 +8,11 @@ recognized contracts:
|---|---|---|
| `release_on_main` | boolean, default false | Main pushes select the existing release job when release publishing is enabled. |
| `release_tag_policy` | `stable`, `v-prefix`, `none` | Stable `v1.2.3`, the existing broad `/v.*/` contract, or no tags. |
| `test_branch_policy` | `all`, `non-release`, `non-release-no-integration`, `feature`, `non-release-no-main`, `non-release-no-hotfix`, `non-release-no-main-hotfix`, `feature-no-hotfix` | Selects an audited test branch exclusion. |
| `publish.hotfix` | boolean, default false | Runs the existing Maven hotfix publisher for nonempty `hotfix/*` branch events only when explicitly enabled. |
| `test_branch_policy` | `all`, `non-release`, `non-release-no-integration`, `feature`, `non-release-no-main` | Selects one of the five audited test branch exclusions. |
`feature` preserves the original exclusion of develop, main, release branches,
and integration-prefixed branches; it is not a new feature-prefix-only rule.
All policies continue to exclude the RC coordination lock branch.
The three hotfix exclusions preserve the corresponding legacy policies while
the single shared publisher handles hotfix events. Existing manifests do not
enable hotfix publication implicitly; migration enables it only when the source
configuration contains the recognized hotfix publisher and branch filter.
No consumer-supplied regex, command, job, or arbitrary branch-publication rule
is accepted.
@@ -37,27 +32,10 @@ fail migration rather than silently changing release behavior.
Use `tools/standardize_qbit.py --write` for the two-file migration. API callers
must pass the inferred policy to
`canonical_caller(version, manifest["release_tag_policy"], manifest["publish"].get("feature", False))`; the legacy one-argument
`canonical_caller(version, manifest["release_tag_policy"])`; the legacy one-argument
form is only for manifests without an explicit policy.
`publish.feature` is false by default. Opting in enables the audited `publish*`
tag contract, with the tag required to resolve to the checked-out commit. The
canonical setup admits those tags alongside the declared release tag policy.
Feature branch pushes remain test-only. Migration accepts Workflows' exact legacy
literal `feature/.*` branch selector (not a regular expression); Git forbids `*`
in branch names, so that selector cannot publish a branch. Unknown feature
filters require a central profile change rather than silently broadening triggers.
Focused tests cover the immutable inventory, unknown-filter rejection, canonical
caller agreement, main opt-in, broad/stable tag selection, HEAD provenance,
publication opt-out and test exclusions. The packed generator suite also exercises
main, broad RC-shaped tags, and excluded integration branches.
Fifteen additional main-branch fixtures exercise 195 branch/tag cases,
checking both test and publication selection. Source and packed generator tests
also verify hotfix opt-in, its boolean validation, and unchanged main behavior.
CMS's original YAML is retained with its immutable source SHA: its unquoted
stable-tag regex is invalid YAML. The fixture repairs only that quoting. Both
audited spellings of the stable regex map to the same ASCII-digit tag policy;
arbitrary regexes remain rejected.
CI-only commits on publishing branches retain Maven verification without publishing when their changes are limited to the manifest, CircleCI caller, shared scanner caller, RC caller, or schema-drift caller. The commit must have one parent and a conventional ci title. Tags, application POMs, schema resources, tests, and other application inputs retain normal publication behavior. Canonical caller enforcement separately verifies the shared workflow definition and immutable worker pin.
-16
View File
@@ -1,16 +0,0 @@
# Application-free branch verification
Use `profile: repository-policy` only on a branch containing repository policies,
documentation, workflow configuration and optional unused Maven wrappers. It uses
the same canonical setup caller as other profiles and one small Python verification
job. It checks tracked YAML/JSON syntax and rejects application manifests, source
files, unknown file types and symlinks. It never builds or publishes an application.
The manifest accepts only `profile`, `orb_version` and `default_branch`. The latter
remains the repository's actual default branch. A future application promotion must
select its real build profile in the same change; adding source without doing so fails.
Estate policy can set `branch_profiles: {main: repository-policy}` without changing
the default branch's application profile. Require the CircleCI `setup` and `verify`
checks on the policy branch. This profile does not establish application release
readiness or replace the build/release checks on source-bearing branches.
-66
View File
@@ -1,66 +0,0 @@
# Shared orb check enforcement
`tools/enforce_orb_checks.py` replaces repository-specific scripts for the shared
CircleCI `setup` and `verify` branch gates. One invocation processes an explicit
batch of current `main` and `develop` branches using existing `gh` authentication.
Create a targets file:
```json
[
{"repository": "QRun-IO/qbit-example", "branch": "main"},
{"repository": "QRun-IO/qbit-example", "branch": "develop"}
]
```
Preview, then apply the same batch:
```sh
python tools/enforce_orb_checks.py --targets targets.json --orb-version 0.8 --output enforcement-plan.json
python tools/enforce_orb_checks.py --targets targets.json --orb-version 0.8 --output enforcement-result.json --apply
```
Every branch must already contain the exact canonical caller for its manifest and
have successful `setup` and `verify` checks from CircleCI app 18001 at its current
commit. Supported profiles are QBit, Core, frontend-core, frontend-material and repository-policy.
The complete batch is inspected before any mutation. Source/head/protection drift,
unavailable APIs and unknown profiles fail closed. Existing repository rulesets are verified as described below.
The tool appends the app-bound checks, enables strict checks and administrator
enforcement, and reads the result back. Existing required checks remain required.
For protected branches it uses the narrow status-check/admin endpoints, preserving
reviews, signatures, linear history, restrictions and other settings. For a
previously unprotected branch it creates protection without adding human reviews.
It never changes scanner configuration, repository visibility, billing, or source.
Output records the planned head and original protection, then verification as each
branch completes. A batch is not transactional: if an API call fails, inspect the
output and rerun; successful protections are retained rather than rolled back.
An already-correct branch needs no mutation.
This command covers the shared orb gate pair only. Continue to run the estate
audit for scanner/native-worker requirements and other policy. Automatic repository
discovery, source enrollment and scheduled reconciliation remain separate work.
GitHub API reference: [protected branches](https://docs.github.com/en/rest/branches/branch-protection).
```sh
python -m unittest discover -s tests -p test_enforce_orb_checks.py -v
```
## Existing rulesets
When effective branch rules come from repository rulesets, the tool can verify
existing enforcement without adding classic protection. Each contributing ruleset
must be active for branches, have no bypass actors and require both setup and
verify from CircleCI app18001. The current branch must also have successful gate
runs. The tool rechecks the branch, effective rules and full ruleset records before
reporting verified. Rule drift or an unbound/missing gate stops verification.
This path writes nothing. It preserves scanner requirements, conditions and the
existing up-to-date-branch policy, including a false strict setting that avoids
unnecessary rebuilds. Reports identify protection_mode=rulesets and the observed
strict value. Unsupported organization-level rulesets, bypass actors and missing
gates remain unresolved rather than being rewritten or ignored.
For classic protection with an empty push allowlist and admin enforcement disabled, the tool leaves the entire classic policy unchanged. After current-head gates pass, it creates a narrowly scoped active repository ruleset requiring app-bound setup and verify with no bypass actors. This enforces CI even for administrators without changing their existing branch access or locking all actors out. The ruleset retains the fleet freshness policy (strict=false); it does not weaken any existing classic policy. A same-named ruleset causes rejection rather than overwrite or duplication. Readback verifies the effective gates and unchanged classic protection; subsequent runs recognize the existing ruleset without writes.
-152
View File
@@ -1,152 +0,0 @@
# Batch consumer enrollment
Use one target file for canonical consumers of every supported Circle profile and
legacy QBits whose jobs and publication filters the shared converter recognizes:
```json
[
{"checkout": "/work/qbit-example", "default_branch": "main"},
{"checkout": "/work/qqq", "default_branch": "develop"}
]
```
The branch values must come from repository inventory. Run on clean isolated
checkouts of the branches being migrated.
```sh
python tools/enroll_consumers.py targets.json --orb-version 0.8
python tools/enroll_consumers.py targets.json --orb-version 0.8 --write
```
The command validates the entire batch before writing. Existing manifests keep
all publication and profile options; only the managed orb version changes.
Recognized legacy QBits receive the canonical caller and inferred publication
policy. An unknown job, filter, profile option, duplicate target, symlink, or
concurrent declaration edit rejects the batch. Repeating a completed enrollment
makes no changes. Filesystem errors during the write phase can leave a partial
batch; rerun after correcting the error. This is not a filesystem transaction.
Only `.circleci/config.yml` and `.qqq-ci.yml` are changed. The tool does not delete
other workflows, infer publication from repository names, or enroll unknown
custom pipelines. Their behavior must first be represented by a central profile.
After committing and qualifying the exact source, apply the shared
[branch enforcement tool](branch-enforcement.md) to supported gate profiles.
This command provides batch source conversion. Use the coordinator below for remote signed submissions. Local conversion alone does not prove fleet deployment or release success.
## Remote signed submissions
Supply observed current branches, without cloning each repository yourself:
```json
[
{"repository": "QRun-IO/qbit-example", "branch": "main"},
{"repository": "QRun-IO/qqq", "branch": "develop"}
]
```
```sh
python tools/reconcile_enrollment.py --targets remote-targets.json --orb-version 0.8 --output plan.json
python tools/reconcile_enrollment.py --targets remote-targets.json --orb-version 0.8 --output result.json --apply
```
Use existing `gh` authentication with repository write access. The coordinator
never installs credentials or changes authentication. Every target is inspected
at an immutable head and validated before any remote mutation. No repository
clone, SSH key or local signing key is required. A moved target stops submission;
refresh the plan instead of overwriting it.
An automation branch derives from the source commit and proposed declarations.
The coordinator creates the branch at the observed base and uses GitHub's
`createCommitOnBranch` API with `expectedHeadOid` for an atomic append. GitHub
signs the commit when supported by the authenticated identity; the coordinator
requires remote signature verification, exact parent, changed paths and
declaration bytes before opening a PR. It never force-updates a branch.
An interrupted branch creation, commit or PR creation can be resumed:
an existing branch still at the planned base receives the missing commit;
matching signed branches and PRs are reused. A mismatched branch is rejected.
Closed existing PRs are reported as existing; they are not silently reopened.
Reports include repository, branch, immutable head and submission URL, without
embedding declaration contents or credentials. Keep the report if a later target
fails; earlier valid submissions remain intact.
API permission, signature or commit-conflict failures stop the run. A timeout
can occur after GitHub accepted a mutation, so inspect remote state or repeat
the same invocation; do not create a different automation branch to retry.
See [GitHub commit API documentation](https://docs.github.com/en/graphql/reference/commits#createcommitonbranch).
This handles remote target lists and signed submissions. Organization discovery is described below. Qualification-aware merging and classic branch enforcement are described below; scheduling and ruleset reconciliation remain separate work. A submitted PR is not a completed migration. Unknown profiles and
native workflows require their own central conversion, not deletion by this tool.
## Organization discovery
The coordinator can select observed current branches directly from GitHub:
```sh
python tools/reconcile_enrollment.py --organization QRun-IO --orb-version 0.8 --output organization-plan.json
python tools/reconcile_enrollment.py --organization QRun-IO --orb-version 0.8 --output organization-result.json --apply
```
Repository and branch inventories are paginated. Only existing main/develop
branches are considered; historical release branches are excluded. Archived
repositories and the orb's own bootstrap repository are reported separately.
Each supported branch receives the same immutable preflight as explicit targets.
Unknown workflows, missing entrypoints, unsupported branch layouts and unavailable
APIs stay `unresolved` in the output. They are never counted as conforming or
silently converted. No-CI and native-only repositories may therefore appear as
unresolved here; use the estate auditor for their separate policy classification.
Discovery finishes before any submission. `--apply` processes qualified plans
and retains unresolved entries for follow-up; one unsupported repository does
not prevent independent valid submissions. An explicit `--targets` batch keeps
its stricter behavior: any preflight failure stops the whole batch. By default neither mode merges PRs or changes branch protections. Use the explicit completion flags below.
## Qualification and enforcement
```sh
python tools/reconcile_enrollment.py --targets remote-targets.json --orb-version 0.8 --output result.json --apply --merge-ready --enforce
```
Both flags require `--apply` and are optional. `--merge-ready` revalidates the
signed submission, exact parent/files/content and current base, requires successful
CircleCI-app `setup` and `verify` checks and every advertised latest check/status,
then requests a normal exact-head squash merge. It never uses admin override or
force pushes. The resulting signed merge must have the qualified source tree.
Pending/failed checks and changed bases are reported without merging. Repeated
runs can advance the same PR once checks complete; there is no polling loop that
holds a CI executor open.
`--enforce` independently checks the actual current target branch, which must
pass its own orb gates before shared classic protection enforcement. Existing
protections and other required checks are preserved. Supported gate profiles are
QBit, Core, frontend-core, frontend-material and repository-policy. Other profiles
remain explicitly unsupported for completion even when source enrollment works.
Existing active repository rulesets with app-bound gates and no bypass actors are verified without writes. Unsupported or incomplete rulesets remain unresolved; none are rewritten or layered with classic protection. API/plan restrictions remain visible.
A merged source update and an enforced branch are separate reported outcomes.
Neither proves artifact publication or deployment. Scheduling and ruleset
reconciliation remain outstanding.
## Central hosted pass
The orb owner's CircleCI config accepts `run_enrollment_reconcile: true` on
`main`. This selects one small Python container instead of the normal orb
build/publication workflow. It uses the existing
`qqq-maven-registry-credentials` context's GitHub token through `gh`.
No consumer-specific runner configuration or local signing key is needed.
`tools/run_enrollment.py` verifies the owner repository and main branch, runs
organization discovery, signed enrollment, qualified merging and enforcement,
then exits. The pass has a 20-minute execution limit. Pending checks are reported
for the next pass rather than polled while a paid executor waits. Incremental
results and execution status are retained under `artifacts/enrollment`, including
when execution fails or times out. A completed pass does not mean every repository
conforms: unresolved and waiting entries remain explicit follow-up work.
Create a single daily CircleCI schedule for the owner project only after a manual
main-branch pass verifies its hosted credentials and results. Set `branch: main`
and `run_enrollment_reconcile: true` in the schedule parameters. Scheduling is
an external project setting; merging this config does not itself enable a schedule.
Keep the existing RC-finalization parameter false for these runs. A timed-out
GitHub request may already have succeeded remotely; the next pass resumes from
the deterministic branch and verified source rather than forcing updates.
-61
View File
@@ -7,64 +7,3 @@ One native job downloads Kustomize 5.7.1 with its pinned official SHA-256 and re
The caller runs on main pushes and pull requests without path filters, so a required check always reports. Superseded PR runs are canceled. Feature pushes do not create duplicate jobs. Remove the old Circle pipeline only after the native job passes on the consumer, then disable obsolete Circle build requests and require the observed native check with no human review.
For marketing-website-cd, the prior Circle job only printed Hello World. Its former production role moved to KofTwentyTwo/cluster-gitops; validation does not re-enable that retired deployment path.
The shared registry includes marketing-website-cd and voyage-cd. Daily enrollment recognizes their canonical main-branch caller without creating Circle configuration. Recognition requires a signed immutable worker pin matching the supported worker, a complete source tree, and no additional Circle or GitHub workflows. Enforcement requires the GitHub Actions app-bound gitops / validate gate and preserves existing branch rules. Unavailable private-repository protections remain an explicit enforcement failure; recognition alone is not enforcement.
### Documentation repositories with shell automation
Use the canonical `repository-policy` manifest with `shell_checks: true` for
repositories containing documentation, YAML/JSON, Mermaid source, and shell tools.
The shared job remains one small executor with no publication or deployment.
It installs ShellCheck only for this opt-in and checks every tracked `.sh` file
with `bash -n` and ShellCheck at warning severity; scripts are never executed.
Mermaid files are treated as UTF-8 documentation, not rendered or syntax-validated.
Application manifests and other executable languages still require a build profile.
### Homebrew distribution taps
The `homebrew-tap` profile takes only `tap` (lowercase owner/name) and `formula`.
One small job in a digest-pinned Homebrew container validates the exact checkout,
reads all OS/architecture variants, fetches their binaries with checksum verification,
then installs and tests the Linux formula. It does not publish or update formulae.
macOS downloads are verified, but this Linux job does not execute macOS binaries.
### Application template acceptance
The `application-template` profile renders the checked-out template through pinned
qctl code and a checksum-verified catalog. It runs transforms but never post-generation
hooks, then performs Maven verify on the generated application with all its tests.
The renderer and Maven dependencies are cached; test reports and the generated JAR
are retained. This is acceptance of a generated consumer, not Maven publication.
Only the standard profile, orb version, and default branch are consumer settings.
The coordinator also recognizes qctl on main/develop and the approved .github,
business-platform, MCP-RAG and IntelliJ callers on main. It verifies each complete
workflow inventory, canonical caller, signed immutable orb pin and matching shared
worker source before recognizing the repository. Native verification gates bind to
the GitHub Actions app; release-only jobs are not merge requirements. No CircleCI
caller is added to these native consumers. qctl requires its shared tests; the native binary job is push-only, and
full platform release builds retain their existing shared-worker policy.
Existing classic branch protections without a required-status-check section gain
an additive shared ruleset. Signing, reviews, history and access restrictions remain
unchanged, and a subsequent reconciliation verifies the effective ruleset.
The same daily pass manages CircleCI's native `autocancel-builds` setting for
canonical `repository-policy`, `homebrew-tap`, `application-template`, and
`marketing` consumers. Marketing must retain `main` as its default branch;
environment deployment remains main-only. CircleCI excludes default-branch,
scheduled and rerun workflows from automatic cancellation. This avoids a custom
cancellation job and does not add executions to consumer pipelines.
The pass uses the existing orb-publishing context on the trusted orb main branch.
It changes only the cancellation flag, checks for source movement before writing,
and verifies preservation of all other flags. Its separate
`result.circle-settings.json` artifact records missing credentials/API access,
unconverted sources and profiles needing cancellation-policy review explicitly.
Publication-heavy profiles are not changed until their non-default release
semantics are reviewed; a successful reconciliation pass does not certify them.
Because cancellation is project-wide, eligibility requires every discovered current main/develop branch to be canonical and use a reviewed profile. An unresolved branch or an unreviewed publishing profile prevents enabling the flag, regardless of branch order. The report lists all observed profiles.
-37
View File
@@ -1,37 +0,0 @@
# Managed patch updates
The canonical caller and manifest accept either an exact stable version such as
`0.8.0` or a quoted minor track such as `"0.8"`. CircleCI resolves a minor track
to its latest published patch. A major-only version, `latest`, `volatile`, and
wildcards are rejected. The caller import, setup parameter, and manifest must
still agree exactly.
Use a minor track after qualifying that release line. Future patch fixes then
arrive on the repository's next pipeline without a version-bump PR or an extra
fleet-wide build. Existing exact pins remain supported and are not changed by
publishing this capability. YAML minor versions must be quoted to remain strings.
The central batch tool applies the same update to all supplied checkouts:
```sh
python3 tools/rollout_version.py --orb-version 0.8 --write /path/to/repo1 /path/to/repo2
```
This changes only the two canonical CI declarations. Publication policies,
contexts, branch/tag filters, and profile options stay unchanged. Normal source
and publication checks still run when a pipeline is triggered; this setting does
not skip tests. GitHub reusable-worker commit pins are separate and unchanged.
Automatic patch adoption trades exact-version reproducibility for centrally
managed fixes. Keep patch releases backward compatible, qualify central changes
before production publication, and use an exact stable pin for a canary or
rollback. For example, the same tool can move a consumer from `0.8` back to
`0.8.0`. Historical pipeline reruns can resolve a newer patch; choose an exact
pin when reproducing an earlier run. Setup and continuation each resolve their
orb imports, so patch compatibility also matters when publication overlaps a run.
Changing minor tracks is an explicit migration. Do not use this feature to push
breaking changes into a patch release or to bypass repository checks.
Reference: [CircleCI orb semantic versioning](https://circleci.com/docs/orbs/use/orb-concepts/).
+1 -12
View File
@@ -26,8 +26,7 @@ https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-t
- Finalization retrieves the unique unexpired intent artifact and verifies its
originating successful `release-rc.yml` run, repository, trigger and attempt.
Callback fields must match that immutable intent.
- The exact merge SHA must have a successful `verify` (or legacy
`publish_release_candidate`) Check Run
- The exact merge SHA must have a successful `publish_release_candidate` Check Run
from CircleCI app ID `18001`; unrelated job statuses cannot authorize release.
Terminal publisher failure stops finalization immediately. Pending/absent publication cannot authorize a tag.
The RC bump must have exactly that one parent and the release branch must still
@@ -108,13 +107,3 @@ is gone, the completed release needs no recovery dispatch. Ordinary callbacks an
scheduled reconciliation retain the requirement for a successful Circle publisher.
Central status API: https://central.sonatype.org/publish/publish-portal-api/
### Auditing independent worker revisions
The estate audit accepts `rc_worker_sha` in a profile or repository policy. For
example, set `profiles.qbit.rc_worker_sha` to the approved immutable RC worker
commit and keep `--worker-sha` at the approved revision for unchanged native
workers. The audit still compares every caller to its exact canonical template
and rejects mutable or mismatched pins. This avoids a fleet-wide native caller
update when only RC release behavior changes. Required-check policy is separate;
omitting it continues to report enforcement as unknown.
+2 -3
View File
@@ -12,9 +12,8 @@ text = source.read_text(encoding="utf-8")
start = text.index("# BEGIN PROFILE RESOURCES")
end = text.index("# END PROFILE RESOURCES") + len("# END PROFILE RESOURCES")
embedded = ["# Generated by scripts/pack_pipeline.py; edit source/resources, then regenerate.", "import base64", "import zlib"]
for variable, name in (("CORE_TEMPLATE", "core"), ("MATERIAL_TEMPLATE", "frontend-material"), ("VOYAGE_TEMPLATE", "voyage"), ("NEXT_TEMPLATE", "frontend-next"), ("ALL_TEMPLATE", "all"), ("STARTER_TEMPLATE", "app-starter"), ("ANDROID_TEMPLATE", "android"), ("POLICY_TEMPLATE", "mcp-policy"), ("REPOSITORY_POLICY_TEMPLATE", "repository-policy"), ("MARKETING_TEMPLATE", "marketing"), ("HOMEBREW_TEMPLATE", "homebrew-tap"), ("APPLICATION_TEMPLATE", "application-template")):
resource = (source.parent / "templates" / (name + ".json")).read_text(encoding="utf-8").encode("utf-8")
payload = base64.b64encode(zlib.compress(resource, level=9)).decode("ascii")
for variable, name in (("CORE_TEMPLATE", "core"), ("MATERIAL_TEMPLATE", "frontend-material"), ("VOYAGE_TEMPLATE", "voyage"), ("NEXT_TEMPLATE", "frontend-next"), ("ALL_TEMPLATE", "all"), ("STARTER_TEMPLATE", "app-starter"), ("ANDROID_TEMPLATE", "android"), ("POLICY_TEMPLATE", "mcp-policy"), ("MARKETING_TEMPLATE", "marketing")):
payload = base64.b64encode(zlib.compress((source.parent / "templates" / (name + ".json")).read_bytes(), level=9)).decode("ascii")
embedded.append(f'{variable} = json.loads(zlib.decompress(base64.b64decode("{payload}")))')
packed = text[:start] + "\n".join(embedded) + text[end:]
if len(packed.encode("utf-8")) > 100_000:
+1 -8
View File
@@ -10,11 +10,6 @@ description: >
- Collects JUnit XML test results
- Stores test results and screenshots
- Saves Maven cache for future builds
parameters:
build_before_test:
type: boolean
default: true
description: "Build the React app first; combined jobs already build it with frontend_build"
steps:
# Ensure no stale Chrome/Driver is shadowing the PATH (your base image had Chrome 109)
- run:
@@ -22,7 +17,7 @@ steps:
command: <<include(scripts/frontend_test_cleanup.sh)>>
# Install current Chrome; DO NOT install chromedriver (let Selenium 4 auto-manage it)
- install_chrome
- browser-tools/install-chrome
# System libs Chrome needs in headless CI
- run:
@@ -57,8 +52,6 @@ steps:
# GPG signing will happen later in mvn_jar_deploy when deploying to Maven Central.
- run:
name: Build app, start server, run tests
environment:
PARAM_BUILD_BEFORE_TEST: << parameters.build_before_test >>
command: <<include(scripts/frontend_test_run.sh)>>
# Always collect and publish test results & screenshots
-5
View File
@@ -1,5 +0,0 @@
description: Install stable Chrome using only its signed repository and existing Ubuntu dependency indexes.
steps:
- run:
name: Install Google Chrome
command: <<include(scripts/install_chrome.sh)>>
+1 -1
View File
@@ -21,7 +21,7 @@ steps:
- when:
condition: << parameters.install_browsers >>
steps:
- install_chrome
- browser-tools/install-chrome
- browser-tools/install-chromedriver
- when:
condition: << parameters.compile_before_verify >>
-8
View File
@@ -5,12 +5,6 @@ description: >
- Deploys signed artifacts and waits for confirmed Central release publication
- Saves Maven cache for future builds
parameters:
skip_verified_analysis:
type: boolean
default: false
description: "Skip repeated SpotBugs and PMD only after this job has completed mvn_verify"
steps:
- run:
name: Setup Maven Settings
@@ -23,8 +17,6 @@ steps:
# Central's native publication wait can be silent for its 30-minute limit.
# Let Maven return its real result before CircleCI's inactivity watchdog.
no_output_timeout: 35m
environment:
QQQ_SKIP_VERIFIED_ANALYSIS: << parameters.skip_verified_analysis >>
command: <<include(scripts/mvn_jar_deploy.sh)>>
- save_cache:
paths:
+1 -1
View File
@@ -3,7 +3,7 @@ description: >
- Adds SSH keys for Git operations
- Sets up Git identity for version commits
- Synchronizes NPM version
- Commits version changes if modified (skipped for feature and snapshot builds)
- Commits version changes if modified (skipped for feature branches)
parameters:
branch_type:
+1 -2
View File
@@ -27,8 +27,7 @@ steps:
- git_full_checkout
- mvn_manage_version
- frontend_build
- frontend_test:
build_before_test: false
- frontend_test
- frontend_package
- mvn_jar_deploy
- git_operations:
+1 -2
View File
@@ -23,5 +23,4 @@ steps:
node_version: << parameters.node_version >>
- git_full_checkout
- frontend_build
- frontend_test:
build_before_test: false
- frontend_test
+1 -2
View File
@@ -57,8 +57,7 @@ steps:
steps:
- check_middleware_api_versions
- mvn_jar_deploy:
skip_verified_analysis: true
- mvn_jar_deploy
- when:
condition:
+1 -24
View File
@@ -161,7 +161,7 @@ set_version() {
##############################################
## Use Maven versions plugin to set version ##
##############################################
mvn -f "$POM_FILE" versions:set-property -Dproperty=revision -DnewVersion="$new_version" -DgenerateBackupPoms=false
mvn versions:set-property -Dproperty=revision -DnewVersion="$new_version" -DgenerateBackupPoms=false
#######################
## Verify the change ##
@@ -175,26 +175,6 @@ set_version() {
fi
}
ensure_project_version() {
local expected=$1 actual
# A revision property can exist without being used by project.version.
# Ask Maven for the effective coordinate, including inherited configuration.
actual=$(mvn -f "$POM_FILE" -q -Dstyle.color=never \
org.apache.maven.plugins:maven-help-plugin:3.5.1:evaluate \
-Dexpression=project.version -DforceStdout)
if [[ "$actual" != "$expected" ]]; then
mvn -f "$POM_FILE" -B versions:set -DnewVersion="$expected" -DgenerateBackupPoms=false
actual=$(mvn -f "$POM_FILE" -q -Dstyle.color=never \
org.apache.maven.plugins:maven-help-plugin:3.5.1:evaluate \
-Dexpression=project.version -DforceStdout)
fi
if [[ "$actual" != "$expected" ]]; then
echo "ERROR: Effective project version is $actual; expected $expected" >&2
exit 1
fi
echo "Verified effective project version: $actual"
}
#############################################
## Branch Detection and Routing Functions ##
#############################################
@@ -454,9 +434,6 @@ main() {
fi
echo ""
if [[ "$DRY_RUN" != "true" ]]; then
ensure_project_version "$NEW_VERSION"
fi
echo "=== Version calculation complete ==="
}
+2 -12
View File
@@ -11,20 +11,10 @@
## Output: Installs required system libraries
############################################################################
set -euo pipefail
set -e
echo "Installing browser dependencies"
export DEBIAN_FRONTEND=noninteractive
ubuntu_sources=/etc/apt/sources.list
if [[ ! -s "$ubuntu_sources" ]]; then
ubuntu_sources=/etc/apt/sources.list.d/ubuntu.sources
fi
if [[ ! -s "$ubuntu_sources" ]]; then
echo 'Cannot locate Ubuntu package sources' >&2
exit 1
fi
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get update \
-o "Dir::Etc::sourcelist=$ubuntu_sources" \
-o Dir::Etc::sourceparts=- -o APT::Get::List-Cleanup=0
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y \
libnss3-dev libgdk-pixbuf2.0-dev libgtk-3-dev libxss-dev
+1 -3
View File
@@ -15,9 +15,7 @@ set -e
echo "Building app, starting server, running tests"
echo "HTTPS=true" >> ./.env
if [[ "${PARAM_BUILD_BEFORE_TEST:-true}" == true ]]; then
npm run build
fi
npm run build
export REACT_APP_PROXY_LOCALHOST_PORT=8001
export PORT=3001
npm run start &
+18 -164
View File
@@ -23,10 +23,7 @@ ALL_TEMPLATE = json.loads((_TEMPLATE_DIR / "all.json").read_text(encoding="utf-8
STARTER_TEMPLATE = json.loads((_TEMPLATE_DIR / "app-starter.json").read_text(encoding="utf-8"))
ANDROID_TEMPLATE = json.loads((_TEMPLATE_DIR / "android.json").read_text(encoding="utf-8"))
POLICY_TEMPLATE = json.loads((_TEMPLATE_DIR / "mcp-policy.json").read_text(encoding="utf-8"))
REPOSITORY_POLICY_TEMPLATE = json.loads((_TEMPLATE_DIR / "repository-policy.json").read_text(encoding="utf-8"))
MARKETING_TEMPLATE = json.loads((_TEMPLATE_DIR / "marketing.json").read_text(encoding="utf-8"))
HOMEBREW_TEMPLATE = json.loads((_TEMPLATE_DIR / "homebrew-tap.json").read_text(encoding="utf-8"))
APPLICATION_TEMPLATE = json.loads((_TEMPLATE_DIR / "application-template.json").read_text(encoding="utf-8"))
# END PROFILE RESOURCES
@@ -72,20 +69,16 @@ def load_manifest(path, setup_version):
if not isinstance(data, dict):
raise ManifestError("Manifest must be a mapping")
profile = data.get("profile")
if profile not in ("qbit", "core", "frontend-material", "frontend-core", "cli", "voyage", "frontend-next", "all", "app-starter", "android", "mcp-policy", "repository-policy", "marketing", "node-artifact", "homebrew-tap", "application-template"):
if profile not in ("qbit", "core", "frontend-material", "frontend-core", "cli", "voyage", "frontend-next", "all", "app-starter", "android", "mcp-policy", "marketing"):
raise ManifestError("Unsupported profile")
allowed = ("profile", "orb_version", "default_branch", "docs_only", "publish")
if profile == "homebrew-tap":
allowed = ("profile", "orb_version", "default_branch", "tap", "formula")
elif profile == "node-artifact":
allowed = ("profile", "orb_version", "default_branch", "node_version", "context", "npm_scopes", "artifact_path", "build_branches")
elif profile in ("qbit", "core"):
if profile in ("qbit", "core"):
allowed += ("java_version", "browsers", "api_checks")
if profile == "qbit":
allowed += ("release_completion", "release_on_main", "release_tag_policy", "test_branch_policy")
elif profile == "cli":
allowed += ("publish_context",)
elif profile in ("voyage", "frontend-next", "app-starter", "android", "mcp-policy", "repository-policy", "marketing", "application-template"):
elif profile in ("voyage", "frontend-next", "app-starter", "android", "mcp-policy", "marketing"):
allowed = ("profile", "orb_version", "default_branch")
elif profile == "all":
allowed = ("profile", "orb_version", "default_branch", "publish", "publish_context")
@@ -93,12 +86,10 @@ def load_manifest(path, setup_version):
allowed += ("node_version",)
if profile == "frontend-material":
allowed += ("java_version",)
if profile == "repository-policy":
allowed += ("shell_checks",)
mapping(data, allowed, "manifest")
version = data.get("orb_version")
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+(?:\.\d+)?|dev:[A-Za-z0-9_-]+)", version):
raise ManifestError("orb_version must be a quoted minor track, exact stable pin, or development version")
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+\.\d+|dev:[A-Za-z0-9_-]+)", version):
raise ManifestError("orb_version must be an explicit stable or development version")
if version != setup_version:
raise ManifestError("Manifest orb_version must match the setup job orb_version")
default_branch = data.get("default_branch")
@@ -106,38 +97,12 @@ def load_manifest(path, setup_version):
raise ManifestError("default_branch must be main or develop")
if profile == "marketing" and default_branch != "main":
raise ManifestError("Marketing requires the audited main production branch")
if profile == "homebrew-tap":
tap, formula = data.get("tap"), data.get("formula")
if not isinstance(tap, str) or not re.fullmatch(r"[a-z0-9][a-z0-9-]*/[a-z0-9][a-z0-9-]*", tap):
raise ManifestError("Homebrew requires an explicit lowercase owner/tap")
if not isinstance(formula, str) or not re.fullmatch(r"[a-z0-9][a-z0-9+@._-]*", formula):
raise ManifestError("Homebrew requires a literal formula name")
return {"profile": profile, "orb_version": version, "default_branch": default_branch,
"tap": tap, "formula": formula}
java = data.get("java_version", "21")
if java not in ("17", "21"):
raise ManifestError("java_version must be the string 17 or 21")
node = data.get("node_version", "24.21" if profile == "frontend-core" else "20")
if not isinstance(node, str) or not re.fullmatch(r"\d+(?:\.\d+){0,2}", node):
raise ManifestError("node_version must be an explicit numeric version string")
if profile == "node-artifact":
if "node_version" not in data or not re.fullmatch(r"\d+\.\d+(?:\.\d+)?", node):
raise ManifestError("Node artifact requires an explicit minor or patch image version")
context = data.get("context", "")
scopes = data.get("npm_scopes", [])
artifact = data.get("artifact_path")
branches = data.get("build_branches")
if not isinstance(context, str) or not re.fullmatch(r"[A-Za-z0-9_-]+", context):
raise ManifestError("Node artifact requires an explicit context")
if not isinstance(scopes, list) or any(not isinstance(x, str) or not re.fullmatch(r"@[a-z0-9][a-z0-9-]*", x) for x in scopes) or len(scopes) != len(set(scopes)):
raise ManifestError("Invalid npm registry scopes")
if not isinstance(artifact, str) or not re.fullmatch(r"[A-Za-z0-9_-]+(?:/[A-Za-z0-9_.-]+)+", artifact) or any(x in (".", "..") for x in artifact.split("/")):
raise ManifestError("Artifact must be a relative file path")
if not isinstance(branches, list) or not branches or any(not isinstance(x, str) or not re.fullmatch(r"[A-Za-z0-9_-]+(?:/[A-Za-z0-9_.-]+)*", x) for x in branches) or len(branches) != len(set(branches)):
raise ManifestError("Build branches must be distinct literal branch names")
return {"profile": profile, "orb_version": version, "default_branch": default_branch,
"node_version": node, "context": context, "npm_scopes": scopes,
"artifact_path": artifact, "build_branches": branches}
publish_keys = ("snapshot", "release_candidate", "release")
if profile == "all":
publish_keys = ("snapshot",)
@@ -145,8 +110,6 @@ def load_manifest(path, setup_version):
publish_keys = ("release",)
if profile in ("qbit", "core"):
publish_keys += ("asciidoc",)
if profile == "qbit":
publish_keys += ("hotfix", "feature")
if profile not in ("qbit", "cli", "all", "app-starter", "voyage", "frontend-next"):
publish_keys += ("feature", "hotfix")
publishing = mapping(data.get("publish", {}), publish_keys, "publish")
@@ -163,20 +126,17 @@ def load_manifest(path, setup_version):
if tag_policy not in ("stable", "v-prefix", "none"):
raise ManifestError("Unsupported QBit release tag policy")
test_policy = data.get("test_branch_policy", "all")
if test_policy not in ("all", "non-release", "non-release-no-integration", "feature", "non-release-no-main",
"non-release-no-hotfix", "non-release-no-hotfix-integration", "non-release-no-main-hotfix", "feature-no-hotfix"):
if test_policy not in ("all", "non-release", "non-release-no-integration", "feature", "non-release-no-main"):
raise ManifestError("Unsupported QBit test branch policy")
return {"profile": profile, "orb_version": version, "default_branch": default_branch,
"java_version": java, "node_version": node, "browsers": boolean(data, "browsers", True),
"api_checks": boolean(data, "api_checks", False),
"shell_checks": boolean(data, "shell_checks", False),
"release_completion": boolean(data, "release_completion", False),
"release_on_main": boolean(data, "release_on_main", False),
"release_tag_policy": tag_policy, "test_branch_policy": test_policy,
"docs_only": boolean(data, "docs_only", False),
"publish_context": publish_context,
"publish": {key: boolean(publishing, key, key != "asciidoc" and profile not in ("cli", "all", "repository-policy", "application-template")
and not (profile == "qbit" and key in ("hotfix", "feature")))
"publish": {key: boolean(publishing, key, key != "asciidoc" and profile not in ("cli", "all"))
for key in publish_keys}}
@@ -211,28 +171,9 @@ def proven_docs_only(config, branch, tag):
return base.decode().strip() if eligible else False
def proven_ci_maintenance(branch, tag):
"""Keep application tests, but do not publish for isolated CI declarations."""
if tag or not (branch == "main" or branch == "develop" or branch.startswith(("release/", "hotfix/"))):
return False
subject = git("log", "-1", "--format=%s")
if not subject or not re.match(rb"ci(?:\([^\r\n()]+\))?: ", subject):
return False
parents = git("rev-list", "--parents", "-n", "1", "HEAD")
if not parents or not re.fullmatch(rb"[0-9a-f]{40} [0-9a-f]{40}\n", parents):
return False
base = parents.split()[1].decode("ascii")
changed = git("diff", "--no-ext-diff", "--no-renames", "--name-only", "-z", base, "HEAD")
if not changed:
return False
allowed = {b".qqq-ci.yml", b".circleci/config.yml", b".github/workflows/codeql.yml",
b".github/workflows/release-rc.yml", b".github/workflows/schema-drift.yml"}
return all(path in allowed for path in changed.rstrip(b"\0").split(b"\0"))
def event_kind(branch, tag, release_on_main=False, release_tag_policy="stable"):
if tag:
if release_tag_policy == "none" or not re.fullmatch(r"v.*" if release_tag_policy == "v-prefix" else r"v\d+\.\d+\.\d+", tag, flags=re.ASCII):
if release_tag_policy == "none" or not re.fullmatch(r"v.*" if release_tag_policy == "v-prefix" else r"v\d+\.\d+\.\d+", tag):
return "test"
tagged = git("rev-parse", "--verify", f"refs/tags/{tag}^{{commit}}")
if not tagged and git("fetch", "--no-tags", "origin", f"refs/tags/{tag}:refs/tags/{tag}") is not None:
@@ -250,36 +191,21 @@ def event_kind(branch, tag, release_on_main=False, release_tag_policy="stable"):
def qbit_template(config, branch, tag):
"""Profile-owned template; add future profiles as reviewed renderers, not raw YAML."""
kind = event_kind(branch, tag, config.get("release_on_main", False), config.get("release_tag_policy", "stable"))
if tag.startswith("publish") and config["publish"].get("feature", False):
tagged = git("rev-parse", "--verify", f"refs/tags/{tag}^{{commit}}")
if not tagged and git("fetch", "--no-tags", "origin", f"refs/tags/{tag}:refs/tags/{tag}") is not None:
tagged = git("rev-parse", "--verify", f"refs/tags/{tag}^{{commit}}")
if tagged and tagged == git("rev-parse", "HEAD"):
kind = "feature"
if not tag and re.fullmatch(r"hotfix/.+", branch) and config["publish"].get("hotfix", False):
kind = "hotfix"
if kind != "test" and not config["publish"][kind]:
kind = "test"
maintenance = proven_ci_maintenance(branch, tag)
if maintenance:
kind = "test"
workflow = "test_only" if kind == "test" else "publish_" + kind
result = {"version": 2.1, "orbs": {"qqq-orb": "kingsrook/qqq-orb@" + config["orb_version"]}}
filters = ({"branches": {"ignore": "/.*/"}, "tags": {"only": "/.*/"}} if tag else
{"branches": {"ignore": "automation/qbit-rc-pending"}, "tags": {"ignore": "/.*/"}})
if kind == "test" and not tag and not maintenance:
if kind == "test" and not tag:
ignores = {"all": None, "non-release": "/(develop|release/.*)/",
"non-release-no-integration": "/(develop|release/.*|integration.*)/",
"feature": "/(develop|main|release/.*|integration.*)/",
"non-release-no-main": "/(develop|main|release/.*)/",
"non-release-no-hotfix": "/(develop|release/.*|hotfix/.*)/",
"non-release-no-hotfix-integration": "/(develop|release/.*|hotfix/.*|integration.*)/",
"non-release-no-main-hotfix": "/(develop|main|release/.*|hotfix/.*)/",
"feature-no-hotfix": "/(develop|main|release/.*|hotfix/.*|integration.*)/"}
"non-release-no-main": "/(develop|main|release/.*)/"}
ignored = ignores[config.get("test_branch_policy", "all")]
if ignored:
filters["branches"]["ignore"] = ["automation/qbit-rc-pending", ignored]
docs_base = proven_docs_only(config, branch, tag) if kind == "test" and not maintenance else False
docs_base = proven_docs_only(config, branch, tag) if kind == "test" else False
if docs_base:
# Keep the existing workflow/result identity even when Maven is not needed.
result["jobs"] = {"docs_only": {"resource_class": "small", "docker": [{"image": "cimg/base:2026.03"}],
@@ -378,10 +304,7 @@ def frontend_template(config, branch, tag):
kind = "release"
if kind != "test" and not config["publish"][kind]:
kind = "test"
maintenance = proven_ci_maintenance(branch, tag)
if maintenance:
kind = "test"
if kind == "test" and not maintenance and proven_docs_only(config, branch, tag):
if kind == "test" and proven_docs_only(config, branch, tag):
result = qbit_template(config, branch, tag)
result["workflows"]["test_only"]["jobs"][0]["docs_only"]["name"] = (
"qqq-orb/mvn_frontend_test_only" if material else "qqq-orb/node_test_only")
@@ -442,18 +365,9 @@ def cli_template(config, branch, tag):
def marketing_template(config, branch, tag):
del config
# Select the existing branch policy once; automation PRs use normal PR checks.
selected = {"main": "production", "develop": "develop", "staging": "staging"}.get(
branch, "release-candidate" if branch.startswith("release/") else "pr-checks")
result = copy.deepcopy(MARKETING_TEMPLATE)
workflow = result["workflows"][selected]
for job in workflow["jobs"]:
next(iter(job.values()))["filters"] = {
"branches": {"ignore": "/.*/" if tag else "automation/qbit-rc-pending"},
"tags": {"ignore": "/.*/"}}
result["workflows"] = {"verify": workflow}
return result
del config, branch, tag
# One final continuation; Munitor supplies jobs, never another setup pipeline.
return copy.deepcopy(MARKETING_TEMPLATE)
def android_template(config, branch, tag):
@@ -489,69 +403,15 @@ def next_template(config, branch, tag):
def voyage_template(config, branch, tag):
del tag # Existing job filters retain tag exclusion and main-only release actions.
selected = "develop" if branch == "develop" else (
"release" if branch == "main" or branch.startswith("release/") else "pull-request")
result = json.loads(json.dumps(VOYAGE_TEMPLATE).replace("__ORB_VERSION__", config["orb_version"]))
result["workflows"] = {"verify": result["workflows"][selected]}
return result
del branch, tag # This profile's reviewed static filters own its three workflows.
return json.loads(json.dumps(VOYAGE_TEMPLATE).replace("__ORB_VERSION__", config["orb_version"]))
def repository_policy_template(config, branch, tag):
del branch, tag
result = copy.deepcopy(REPOSITORY_POLICY_TEMPLATE)
if config.get("shell_checks", False):
job = result["jobs"]["validate-policy"]
job["environment"] = {"QQQ_POLICY_SHELL": "true"}
job["steps"].insert(1, {"run": {
"name": "Install static shell validation",
"command": "sudo apt-get update -qq && sudo apt-get install -y --no-install-recommends shellcheck"}})
return result
def homebrew_template(config, branch, tag):
del branch, tag
result = copy.deepcopy(HOMEBREW_TEMPLATE)
result["jobs"]["validate-tap"]["environment"].update({
"QQQ_HOMEBREW_TAP": config["tap"], "QQQ_HOMEBREW_FORMULA": config["formula"]})
return result
def application_template(config, branch, tag):
del config, branch, tag
return copy.deepcopy(APPLICATION_TEMPLATE)
PROFILE_TEMPLATES = {"application-template": application_template, "homebrew-tap": homebrew_template, "qbit": qbit_template, "core": core_template,
"repository-policy": repository_policy_template,
PROFILE_TEMPLATES = {"qbit": qbit_template, "core": core_template,
"frontend-material": frontend_template, "frontend-core": frontend_template,
"cli": cli_template, "voyage": voyage_template, "frontend-next": next_template, "all": all_template, "app-starter": starter_template, "android": android_template, "mcp-policy": policy_template, "marketing": marketing_template}
def node_artifact_template(config, branch, tag):
# One checkout/install is shared by tests and an optional artifact build.
lines = ["registry=https://registry.npmjs.org/"]
lines += [scope + ":registry=https://npm.pkg.github.com/" for scope in config["npm_scopes"]]
if config["npm_scopes"]:
lines.append("//npm.pkg.github.com/:_authToken=${NPM_TOKEN}")
npmrc = "printf '%s\\n' " + " ".join("'" + line + "'" for line in lines) + " >> .npmrc"
steps = ["checkout", {"run": {"name": "Configure npm registries", "command": npmrc}},
{"restore_cache": {"keys": ['node-artifact-npm-v1-{{ checksum "package-lock.json" }}', 'node-artifact-npm-v1-']}},
{"run": {"name": "Install locked dependencies", "command": "npm ci --prefer-offline --cache ~/.npm"}},
{"save_cache": {"key": 'node-artifact-npm-v1-{{ checksum "package-lock.json" }}', "paths": ["~/.npm"]}},
{"run": {"name": "Test project", "command": "npm test"}}]
if not tag and branch in config["build_branches"]:
steps += [{"run": {"name": "Build artifact", "command": "npm run build"}},
{"store_artifacts": {"path": config["artifact_path"], "destination": config["artifact_path"].split("/")[-1]}}]
return {"version": 2.1,
"jobs": {"node_artifact": {"docker": [{"image": "cimg/node:" + config["node_version"]}], "steps": steps}},
"workflows": {"verify": {"jobs": [{"node_artifact": {"context": [config["context"]],
"filters": {"branches": {"ignore": "automation/qbit-rc-pending"}, "tags": {"ignore": "/.*/"}}}}]}}}
PROFILE_TEMPLATES["node-artifact"] = node_artifact_template
def runtime_boolean(name):
value = os.environ.get(name, "false").lower()
if value in ("true", "1"):
@@ -581,12 +441,6 @@ def main():
if expected and (not re.fullmatch(r"[0-9a-fA-F]{40}", expected) or git("rev-parse", "HEAD") != (expected + "\n").encode()):
raise ManifestError("Checkout does not match the pipeline commit")
output = PROFILE_TEMPLATES[config["profile"]](config, os.environ.get("CIRCLE_BRANCH", ""), os.environ.get("CIRCLE_TAG", ""))
if config["profile"] in ("qbit", "core", "frontend-core", "frontend-material"):
# One whole-workflow check follows the same commit from feature to release.
# Keep every selected job, dependency and filter; add no runner or duplicate build.
if len(output["workflows"]) != 1:
raise ManifestError("Stable verification requires exactly one selected workflow")
output["workflows"] = {"verify": next(iter(output["workflows"].values()))}
parameters = continuation_parameters(config["profile"])
if config["profile"] == "core":
output["parameters"] = copy.deepcopy(CORE_TEMPLATE["parameters"])
-25
View File
@@ -1,25 +0,0 @@
#!/bin/bash
set -euo pipefail
# Machine executors are Ubuntu amd64. Preserve an existing browser for Maven
# consumers; frontend_test deliberately removes its stale browser beforehand.
if command -v google-chrome >/dev/null 2>&1; then
google-chrome --version
exit 0
fi
architecture=$(dpkg --print-architecture)
if [[ "$architecture" != amd64 ]]; then
echo "Google Chrome requires an amd64 machine executor" >&2
exit 1
fi
curl --fail --silent --show-error --location https://dl.google.com/linux/linux_signing_key.pub \
| sudo gpg --batch --yes --dearmor -o /usr/share/keyrings/qqq-google-chrome.gpg
printf '%s\n' 'deb [arch=amd64 signed-by=/usr/share/keyrings/qqq-google-chrome.gpg] https://dl.google.com/linux/chrome/deb/ stable main' \
| sudo tee /etc/apt/sources.list.d/qqq-google-chrome.list >/dev/null
# Ubuntu dependency indexes were refreshed by install_java. Only the newly added
# signed Google repository needs refreshing; image PPAs are unrelated to Chrome.
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get update \
-o Dir::Etc::sourcelist=/etc/apt/sources.list.d/qqq-google-chrome.list \
-o Dir::Etc::sourceparts=- -o APT::Get::List-Cleanup=0
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get install -y google-chrome-stable
google-chrome --version
+4 -26
View File
@@ -20,30 +20,13 @@ sudo install -d /etc/needrestart/conf.d
# shellcheck disable=SC2016
printf '%s\n' '$nrconf{restart} = "l";' | sudo tee /etc/needrestart/conf.d/99-qqq-ci.conf >/dev/null
# Retry transient repository downloads within APT, including NodeSource's child process.
# Exhausted downloads retain APT's nonzero exit status and fail this build step.
printf '%s\n' 'Acquire::Retries "3";' | sudo tee /etc/apt/apt.conf.d/80-qqq-download-retries >/dev/null
# Ubuntu archive carries the same signed security suite; avoid the flaky HTTP mirror.
if [[ -f /etc/apt/sources.list ]]; then
sudo sed -i 's|http://security.ubuntu.com/ubuntu|https://archive.ubuntu.com/ubuntu|g' /etc/apt/sources.list
fi
# Base tools come from Ubuntu, not the unrelated PPAs bundled in machine images.
# Support both the 22.04 list and the newer deb822 Ubuntu source file. Preserve
# other indexes for later jobs without contacting their repositories here.
ubuntu_sources=/etc/apt/sources.list
if [[ ! -s "$ubuntu_sources" ]]; then
ubuntu_sources=/etc/apt/sources.list.d/ubuntu.sources
fi
if [[ ! -s "$ubuntu_sources" ]]; then
echo 'Cannot locate Ubuntu package sources' >&2
exit 1
fi
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get update -qq \
-o "Dir::Etc::sourcelist=$ubuntu_sources" \
-o Dir::Etc::sourceparts=- \
-o APT::Get::List-Cleanup=0
# Add Eclipse Temurin repository
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get update -qq
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get install -y -qq wget apt-transport-https gpg maven curl
# Add Temurin GPG key
@@ -52,13 +35,8 @@ wget -qO - https://packages.adoptium.net/artifactory/api/gpg/key/public | sudo g
# Add repository
echo "deb [signed-by=/usr/share/keyrings/adoptium.gpg] https://packages.adoptium.net/artifactory/deb $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/adoptium.list
# Refresh only the repository just added. Re-fetching unrelated image PPAs here
# caused otherwise successful setup to fail on Launchpad 503 responses. Keep
# Ubuntu indexes from the first refresh so Java dependencies remain resolvable.
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get update -qq \
-o Dir::Etc::sourcelist=/etc/apt/sources.list.d/adoptium.list \
-o Dir::Etc::sourceparts=- \
-o APT::Get::List-Cleanup=0
# Install Java
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get update -qq
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get install -y -qq "temurin-${JAVA_VERSION}-jdk"
# Set JAVA_HOME
-8
View File
@@ -22,13 +22,6 @@
set -e
# The shared publish job already ran these checks in its verify lifecycle.
# Standalone command callers retain analysis unless explicitly opted in.
analysis_args=()
if [[ "${QQQ_SKIP_VERIFIED_ANALYSIS:-false}" == "true" || "${QQQ_SKIP_VERIFIED_ANALYSIS:-false}" == "1" ]]; then
analysis_args=(-Dspotbugs.skip=true -Dpmd.skip=true)
fi
##############################################
## Deploy signed artifacts to Maven Central ##
##############################################
@@ -37,7 +30,6 @@ fi
# Central 0.8.x: this completion flag overrides inherited literal waitUntil=uploaded.
# Do not report a release as successful until Central confirms publication.
mvn -s /tmp/circleci/mvn-settings.xml -P release -B -DskipTests \
"${analysis_args[@]}" \
-Dgpg.keyname="$GPG_KEYNAME" -Dgpg.passphrase="$GPG_PASSPHRASE" \
-DwaitForPublishCompletion=true \
-Dflatten.mode=resolveCiFriendliesOnly -DupdatePomFile=true \
+1 -1
View File
@@ -1,5 +1,5 @@
#!/bin/bash
set -eo pipefail
set -e
echo "Running PMD static analysis..."
+1 -1
View File
@@ -1,5 +1,5 @@
#!/bin/bash
set -eo pipefail
set -e
echo "Running SpotBugs static analysis..."
+1 -1
View File
@@ -64,7 +64,7 @@ echo "Publishing version: $VERSION"
if [[ "$VERSION" =~ -RC\.[0-9]+$ ]]; then
NPM_TAG="rc"
echo "Detected RC version - using tag: $NPM_TAG"
elif [[ "$VERSION" =~ -SNAPSHOT(\.[1-9][0-9]*)?$ ]]; then
elif [[ "$VERSION" =~ -SNAPSHOT$ ]]; then
NPM_TAG="snapshot"
echo "Detected SNAPSHOT version - using tag: $NPM_TAG"
elif [[ "$VERSION" =~ -(alpha|beta) ]]; then
+5 -5
View File
@@ -9,7 +9,7 @@
## It also pushes the changes back to the current branch.
##
## Process:
## 1. Skips for feature and snapshot builds (ephemeral versions)
## 1. Skips for feature branches (ephemeral versions)
## 2. Checks if package.json has been modified
## 3. Extracts the new version from package.json
## 4. Commits the changes with skip ci flag
@@ -22,12 +22,12 @@
set -e
################################################################
## Skip version commit for feature and snapshot builds ##
## These versions are ephemeral and don't need to be ##
## Skip version commit for feature branches ##
## Feature versions are ephemeral and don't need to be ##
## persisted back to the repository ##
################################################################
if [[ "$BRANCH_TYPE" == "feature" || "$BRANCH_TYPE" == "snapshot" ]]; then
echo "Skipping version commit for $BRANCH_TYPE build (ephemeral version)"
if [[ "$BRANCH_TYPE" == "feature" ]]; then
echo "Skipping version commit for feature branch (ephemeral versions)"
exit 0
fi
+21 -167
View File
@@ -16,18 +16,15 @@ import yaml
# Generated by scripts/pack_pipeline.py; edit source/resources, then regenerate.
import base64
import zlib
CORE_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNrtPdl22ziy7/0VuOrM2M6YpPclifvcxHbfeCar48z0nChXByJBiW1uJig7Ponn26cKACmKIilSluUlTr/IJFAo1F6FAvv7L4S0zlnEncBvPSNr+uoyPglpRD0Ww3N4+B2ewDMv8HtBp+v4VsdyaM8PeOyYw/cwIr4MGfzd6gaBy6jfWk5eWMymAzeGdzZ1OROPr+Tbls++xR1qmiyMqW+yTnfgWy7rDCK3CDSPI8fvFUFuteqA5X26trk1U8gRM5kTxtcC/YsC3wqibobkZ2dnGjzBgacAgUdBcGqoh/+7om/pu610phl4HvWtzGwe0yjWBNs0ZJsWR9RkWQR5zEKc8EU9QDB9Zp4GgzjFlqTDxfto4GcgqIc+CAsi+SFiIDmMBBc+s8hbXJnwwDxlMcmKzPLobIU5AnBswgdWQGgYaz2YNQgtGjOinZ2Rv/519JXjw/5cl2iX4jUXm4NRbR9gKpBEOyc+Zz4IMuE8BZFOhQEe2d7cJEbshYKwGWplME7AitlqJY0R8WMPBxMtmATDCCNmu06vH+tu0IO5A/acxH3mI+QQBCS2yQL7FgZRTD5+/Nh5+/7d/73vnBy/3D/sHH9+t/eXs3bbXyDt1pP9o+P9N4edf70//sfvb97/q/P39686RwftFvntN3z96uWn153Dd/9st9o+A21D+MzsB2RBMgSxBmEiKT5k4NNz6ri06wJKZuDHjj8AJkZOz/GpS97Sc+YTIRhcX2j7ttP2WxkWXqW/r6YRmvddzqJzWM93L4EgDid/Bt0FTgAf5rqpHNnOt3gQsTHp6VLztBcFAyFASNVy6YoZj4nmI5G+F5D4mXYFRPzxg7BvTkxW2r5gt6YBoQSGGvPP9yTx94+Wi7mwXACXhJdxP/DXiW46keky0zGE5GQERQ8vS2mqfn0dMUI8DsKGql2fI787vsP7DbX4AkQZJ1P3gl5W6fhUXIiDgdmfqGJIFTFViPt+MHAt4gcx4U5PCLLYihmAVJlxEAlZDiJC45h5IVoU8mSRszOyStZWlp4TK0DVEej+D9HsiauDmIBV1//kgY9IdCNGTxECdxkLyWrbtwKfqf2YfS8Aq3FM6N+iPyaan4ItefSUETRiw3EEzL1jUzPmBJa2UJ1xi2BTJckb7WBom0waN5lYbHQyWCp/STyHc7BDzxPZQnGLeN8JYYRHwUaDXQqjAGzPdEYHhCFinZQm4+Ie0riPAjlhc3lRtoCYYBdjGTG1SiY10GWwuLbTA9umhWDLaI9pykHNSp//gfKnYCfOT+CPlh+E37eYVaGwv5KTPlhA0h04IHweNfuOzzhBR2853KQRTCfURicLMoPWW2/7v5LPMOUzhF3xYAEHwyww8p4TRUG0TC76DugzBIOgCjSKHICHc1FRARhn5iBy4ktwuE7MdKUzHN5oDlngP/pxHD4zjGSYPhDL6ICzIX+KERyGqHULRvQWiMFi04B4wuDBIDIZ112Hx2qxTIwghvmMWSDlGFIZyDDdavuJ1/4LF6554Ykf4avvatwV2QMj57ZbzxfIDxk7xIyVgTN2dzUUQtPR8QH5zbDYueEPXLexZwBJ4tQLXaZJveSzEqR9aTrJ3z/7YJuFVYGIDyKW6b2Bd2o5EdFCgntXWIPW/wnLGEAdiPSMPwewWrIVabLlqmixy6dx0CjbiZAIAkXj5PDTifZU/+a5FbNsCIQ4tYtmJR4BDOoXtKbgwOQg8Fdfh/YSLGY48q7mzgycjGZOuolpjB1ypDPO85y9q4nPbSAgFIJFGg1D1zGFjdIEUjdk+ssx6UbBBUdM0kRv7iiAcQ1deqldOBb8yee+vlxXw4QgCtz5r28zihE/IHDOInSLmShj7shg1p8RBg0ktceqUCk0ziFWWnicrMHOHYv5o4F7QelF7qEPjjaHeb7CMLZ6SpCZmH2JNUmwJmiJ0WsnDCI9SNav7wlEPJajkPHiBRlSRhfEgHxXegOYJwL6S3QIebNP8hadlBoZMq70JKeEJKcUoz7BArtfLkJPUjwLHAakArXnNiDQtZyKktZOHHQuguiUh6M5ZiI6QRCnYXResJfH9WxUCtWLYuxbI8O+1lG0tB4GYWimFsZ4twPcPWVRB6OrIPJoTu9UXDuqdI4HYo2bk0Gjtra2svEMgs6I+alLGuoY2CcRSHZMl3JcvOUi91rLVfW20uB/SLqkDmioV50/6TnNvi8o2qVzvHO/IyL3qSp6/2SRY1+SA0zRIuJAIJ9UioT2S5KSLEnLFdxSQGDsDMtHxwOfvAR99hA53yLHtNt14rcfm+EGVCLaK6JxqVxpoQaea5zFMVZfReSohTJ6BIEiGvWIdgC/9MwiOj91wj1R5yYmlsHJuaDhLLfMzgZgDpIdqjwiu1MCud0bxx98q9hyUpXSSLv94sXCh39jlh0FHkEldZ0ucTwRZH+AP9UbIIDO4giSGPXu0GUeqMIJPGqDLUss7J6YtCgNGtCnLLhut5akBVc7AfMNkxQ3261lMIkJO2HoM2ksET1YIVnLIHa7JYJ02Juelsl1WFrPs+a7XOdqf/js6ATZirVSIgq8kG8C8My+dLBLnC3iqks6bALN3eKSHO7RGFQP9/vFpFyqhPgh/BCmrrbjW6CxsCsMjfEdbAS9Bf5EcPAG2YpP9zBdBCsf+D6Yev4SM+pXAuFjyeF266tcF6wLA+q7zF9UKIjZq8tIi2STz/LyIfI1VXVJNqwgKShfVr4KjAEplOKQWQJbTt6B96gB/IKCd00mtv0P/77pNKZCthrlzMJrdbrAWqzuhx1px7PugX1j5gCwS1ZG0zpihudp3BuEbjiZpBsjMNgHrqGhdBlEMsjAHpAaz/94DWNRFKEgt4akG8ZOoqg9aaIK8jtJDFlVCZ/uSAoAE8nPDB2EOQPbjIXfy+SEAyIANh0NSrZPNJknJOtrGsZRGOamNcd0pqZkbvZuQsUr40Qoi9JmWbwZ7i9NG2pXepJCSymxjAy1Rdk9E1WbYYN5dfDPzbmRNdJoXha80Z/gYSG4k/J1nuJ5YkFtCp4O8wxVk5LPGmHyPJM+3OXS27RVt+z+KjRcBnFaviZRqOSRVLxksMqb9ZS4N5uHlW68djJWKvO18rG7XaG8XlFwTD3uQ1GsKuTJlDfnEeyMUKMw8BkjSfLWDyymTvgn0fwG8+FX4gzMxkoFqiKxWIjHZ76JJ1hJPUzSdibZ5kEEkobB2d6KDv8lQpEpUGkHGHGfgF5x/B0GcXcAs0UWisSCh6FnZf6WSWnCm9mFGzLUBKoLGoSOL7oHKCQxDnWJRXm/G2AJs5wsPQeSWBcsMUmO7+BJf9AVx3YfIdHXjt4LHwjRm4/HlpqnwGspeB2BZEtR40PaPo7R9icMI4kUgUW3WAzBIVnd3tlgG2zHWts1N81dZu+YGyvM3LDXtky6aVr25k53dXd7rXEEy6RlkBHoGIVP2WWhnU5x9kNPO1/V6qKnff+u2nkGHvjMajIYiTa7gXkqI58Wubq6jiuooWGVIkRoD/sDYtktlCpjhWSZ1kSZ+JXsQwo08MDXivYJCVypCaDx4f1bcgHimAi4sCELPGk101wH5BaMNTjrGIyBPowmNJNUllZCeJFWTOAPWZBYaoOIf4vhDT5CjDr4N5YeAiAQ1gpeYI1DNW/+toH24YWRfYQ5uErwcSqo0cCPF2F2Ui1otz5IGg9J+/HjR6Lmg4xQvydTecThInJiJpEQ4CDUciESQYDLeWxKrFUevyXYDdAG5JeYDuiZy3rUvNRCxiLQuZDL7isb/gpsW1BY04SSkP8YOsxq+/tH0qohCNnmomGahfWMl66r6nAQkg38I/8V8AkZ/RbfCiZKzIAAwDhrYIqeCMHY59h+hH0ePRXWgcyE5IJGPtpm1aGS/q0LPORaiAisJsG0/TRF4pFp4CQjqRJzo0gM5VGAmGzo9SYZuKcTKa3wDjudBCVS33RJRiRZNNrBRoWAjzR1wD4auKYMKyu8Uw0H1gt7Gf+0oB0g+ntP0actjLqrxvUkSOwrrKrwxbdpROtG5FLab8zmYg1dHe5ZNKbL5PXJyQdRJFInYiQTJ9aoKL/CvlqM1KUcOlxEqyhU44Et6saCSAueQjK1kMryx1fgpLGmnyigEFvVBbqcEWcQ+mCkbzQwQQME05vJc7NQy87uBammnXWd2FD2Oy+2s0WjnubhqCRS0gsipYkQ7KJEONmg9iFzGo7848spFbJQsmcgM63sodC+oV7XSiSknpACtYnGBgGEFyHDksOMmaPOhzzHslwGDgJ0XiC5jE8HseM68aV6xHPEqVFOy1dWhMMoXK+6FjN6ViLXomGofxK/JFkPHB4KP5ruDkV/QhFnbLkUw9zer4PfpzP+WUK7Lm4T5bG4n25GNY7aTLvZU5SanJmqaFXR6pJrbFElCWB9xKaoZHiwTweLBPOuadzi6fwdqUakpL87xQgR0QDigJ1ATF25ki0CAl/5fPH1y6V6fmOWVGroWEm7jcVvFRcrEyg29w7AHaitnYjs5yDRWh1925H9LvgUApq2wyzBliQjOnBiPQPuLSKH0I5O4F1a3e5TN37v/w6QAGpBU8NsHXrS1hDEfRRlN7jEQ3jiAVqcQJDXh2gec6v5NjVIazja01DoX9JuBmkvl4nn+I4H2QG2NeCR+jSerpDNskKwTLaWluUxftE/WLH4RKV6xYwkJMusri4lzRflDRKKTIba/VivRARCs4fB+qLsjJC9D0K+sQ4AHMu/G7YgLI00KyCk3/YS6sq2BLnoFcgWvL0iUm3g5zLk8SD/mOJ/VxOucr0P+WVtKe0Sq709siJsRn4Uw3sMwzHLats1mx6Ko4oq92aq4tSjdxv1bgld+M9Wbk+spcqOe8wHk41yrtpbhdRmWlwzXk7UMYmL93HeXn76+CZPOj/oAEPDQdyJHY8hb2G99RWvWVaVSixagb0ni94pXrLine truncated
CORE_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNrtPdl22ziy7/0VuOrM2M6IpPclifvcxHbfeCar48z0nChXByJBiW1uJig7Oonn26cKACmKIiVSluUlTr/IJFAo1F6FAvv7L4Q0LljEncBvPCPr+loTn4Q0oh6L4Tk8/A5P4JkX+N2g3XF8q205tOsHPHbM4XsYEQ9CBn83OkHgMuo3mskLi9m078bwzqYuZ+LxlXzb8Nm3uE1Nk4Ux9U3W7vR9y2XtfuQWgeZx5PjdIsiNRhWwvEfXt7bnCjliJnPC+Fqgf1HgG0HUyZD8/Pxcgyc48Awg8CgIzgz18H9X9W19r5HONAPPo76Vmc1jGsWaYJuGbNPiiJosiyCPWYgTvqgHCKbHzLOgH6fYknS4eB/1/QwE9dAHYUEkP0QMJIeR4NJnFnmLKxMemGcsJlmRaY7OVpgjAMcmvG8FhIax1oVZ/dCiMSPa+Tn5619HXzk+7M91iTYQr7nYHIxq+QBTgSTaBfE580GQCecpiHQqDPDIztYWMWIvFITNUCuDcQJWzFYraYyIH/s4mGjBNBhGGDHbdbq9WHeDLszts+ck7jEfIYcgILFNlti3MIhi8vHjx/bb9+/+73379OTlwVH75PO7/b+ct1r+Emk1nhwcnxy8OWr/6/3JP35/8/5f7b+/f9U+Pmw1yG+/4etXLz+9bh+9+2er0fIZaBvCZ2YvIEuSIYg1CBNJ8SF9n15Qx6UdF1AyAz92/D4wMXK6jk9d8pZeMJ8IweD6Usu3nZbfyLDwKv19NYvQvO9wFl3Aer47AII4nPwZdJY4AXyY66ZyZDvf4n7ExqSnQ82zbhT0hQAhVculK2Y8JpqPRPpeQOJn2hUQ8ccPwr45MVlt+YLdmgaEEhhqzL/Yl8Q/OG4Wc6FZAJeEg7gX+BtEN53IdJnpGEJyMoKih4NSmqpfX0eMEI+DsKZqV+fI747v8F5NLb4EUcbJ1L2kg0k6PhMX4qBv9qaqGFJFTBXifhD0XYv4QUy40xWCLLZiBiBVZhxEQpaDiNA4Zl6IFoU8WebsnKyR9dWV58QKUHUEuv9DNHvq6iAmYNX1P3ngIxKdiNEzhMBdxkKy1vKtwGdqP2bPC8BqnBD6t+iPqeanYEsePWMEjdhwHAFz79jUjDmBpS1UZ9wi2FRJ8lo7GNomk8Z1JhYbnQyWyl8Sz+Ec7NDzRLZQ3CLec0IY4VGw0WCXwigA2zOb0QFhiFg7pcm4uIc07qFATtlcXpQtICbYxVhGTI2SSTV0GSyu7XTBtmkh2DLaZZpyUPPS53+g/CnYifMT+KPlB+H3LWZNUNhfyWkPLCDp9B0QPo+aPcdnnKCjtxxu0gimE2qjkwWZQeutt/xfyWeY8hnCrri/hINhFhh5z4miIGqSy54D+gzBIKgCjSIH4OFcVFQAxpnZj5x4AA7XiZmudIbDG80hS/xHL47DZ4aRDNP7YhkdcDbkTzGCwxC1bsGI7hIxWGwaEE8YPOhHJuO66/BYLZaJEcQwnzELpBxDKgMZplstP/Haf+HCNS898SN89V2NuyL7YOTcVuP5EvkhY4eYsTJwxt6ehkJoOjo+IL8ZFrsw/L7r1vYMIEmceqHLNKmXfF6CdCBNJ/n7Zx9ss7AqEPFBxDK7N/DOLCciWkhw7wpr0Po/YRkDqAORnvFnH1ZLtiJNtlwVLXb5NA4aZTsREkGgaJwefTrVnurfPHfCLBsCIU7tolmJRwCD+gWtKTgwOQj81dehvQSLGY68q7gzAyejmZNuYhZjhxxpj/M8Z+8q4nMbCAiFYJFGw9B1TGGjNIHUDZn+ckw6UXDJEZM00Vs4CmBcQ5cOtEvHgj/5wteX62qYEESBu/j1bUYx4gcELliEbjETZSwcGcz6M8KggaR22SRUCo1ziJUWHidrsAvHYv5o4F5QepF76IGjzWGerzCMrZ4SZC5mX2JNEqwJWmL02gmDSBeS9et7AhGP5ShkvHhBhpTRBTEg35XeAOaJgH6ADiFv9kneopNSI0PGlZ7klJDklGLUJ1hg98tF6EmKZ4HDgFSg8twaBLqWU1HS2o6D9mUQnfFwNMdMRCcI4jSMzgt2c1zPRqVQvSjGvjEy7GsVRUvrYRCGZmphjHfawN0zFrUxugoij+b0TsW1o0rneCDWuDkZNGrr66ubzyDojJifuqShjoF9EoFk23Qpx8UbLnKv0ZxUbysN/oekS+qAhnrV/pNe0Oz7gqJdOse78Nsicp+povdPFjn2gBxiihYRBwL5pFIktF+SlGRJWq7glgICY+dYPjrp++Ql6LOHyPkWOaGdjhO//VgPN6AS0V4RjUvlSgs18FzjLI6x+ioiRy2U0SMIFNGoR7RD+KVnFtH5mRPuizo3MbEMTi4EDee5ZXbeB3OQ7FDlEdmdEsjt3jh+/9uELSdVKY20Wi9eLH34N2bZUeARVFLX6RDHE0H2B/hTvQEC6CyOIIlR745c5oEqnMKjFtiyxMLui0nL0qABfcqC61ZjRVpwtRMw3zBJcbPVaIJJTNgJQ59JY4nowQrJWgaxWw0RpMPe9LRMrsPSep413+U6VwfDZ8enyFaslRJR4IV8E4Bn9qWDXeJsGVdd0WETaO6WV+Rwj8agerjfLyblUiXED+GHMHW1Hd8CjYVdYWiM72Aj6C3wJ4KDN8hWfLqP6SJY+cD3wdTzl5hRvxIIn0gOtxpf5bpgXRhQ32X+skJBzF5rIi2STT7Ly4fI11TVJdmwgqSgfFn9KjAGpFCKQ2YJbDl5B96jAvBLCt41mdjyP/z7ptOYCbJVK2cWXqvdAdZidT9sSzuedQ/sGzP7gF2yMprWETO8SONeI3TDySTdGIHBPnANDaXLIJJBBnaB1Hj+xysYi6IIBbk1JN0wdhJF7WkTVZDfTmLISZXw2Y6kADCR/MzQQZgzsM1Y+B0kJxwQAbDZaFCyfaLJPCFZX9MwjsIwN605pjM1JXPzdxMqXhknQlmUNs/izXB/adpQudKTFFpKiWVkqC3K7pmo2gxrzKuCf27OjayRRvOy4I3+BA8LwZ2Ur/MUzxMLalPwdJhnqJqUfFYLk+eZ9OEul95mrbpl9zdBw2UQp+VrEoVKHknFSwarvFlPiXuzeVjpxisnY6UyXykfu9sVyusVBcfU4z4UxSaFPJny5iKCnRFqFAY+YyRJ3vqBxdQJ/zSa32A+/EqcgdlYqUBVJBYL8fjMN/EEK6mHSdrOJds8jEDSMDjbX9Xhv0QoMgUq7RAj7lPQK67yzYTs84skZBQJBBXbCx1fNAZQyE8c6hKL8l4nwOpk+Y67DuSnLhhZkpzMwZNevyNO5D5CDq8dvxfuDQIzH08kNU+B11LwOgLJVpnGh7R8HKMdTBlGEgEBY22xGOI+srazu8k22a61vmdumXvM3jU3V5m5aa9vm3TLtOyt3c7a3s567eCUSaWXweUYhc/YoNAEpzj7oaddrGlV0dO+f1edOn0P3OFkMhiJorqBeSaDmga5urqOla+gPBNFiNAuHv3HshEo1bMJkmVaU2XiVwIJPxAE3KjojJDAlZoAGh/evyWXII6JgAvzsMSTLjLNdUBuwQ6DH45Bz/VhoKCZZGLVJIQXaTEE/pC1hpUWiPi3GN7gI8SojX9jVSEAAmEZ4AWWL1Rf5m+bqPovjOwjTK9V7o5TQY36frwMs5NCQKvxQdJ4SNqPHz8SNR9khPpdmaUjDpeREzOJhAAHUZQLQQYCbOaxKTFEefxWYDdAG5BfYjqgZy7rUnOghYxFoHMhl41VNvwV2LagsKYJJSH/MXSY1fIPjvfR2hMEITtYNMygsFTx0nVViQ2irb5/7L8CPiGj3+JbwUSJGRAAGGf1TdHuIBj7HDuLsIWjqyI2kJmQXNLIR7Ormk/Sv3WBh1wLEYHVJJiWn2Y/PDINnGQkBWBuFImhrPKLyYZebZKBezqV0grvsIlJUCJ1OwMyIsmihw42KgR8pF8D9lHD62RYOcHxVPBN3bArC6KCk0vaIaK//xTd1dKou6pdKoKcfYJVFW72No1o1WBbSvuN2Vwsj6tzO4vGtElen55+EPUfddhFMiFghWLxK2yZxSBcyqHDRSCKQjUes6JuLImI/ynkSUupLH98BU4ay/WJAgqxVQ2ezYw4g9AHIy2hgQkaIJheT55rRVGQX2b2glTTzjtObCj7nRfb+aJRTfNwVBIp6QWR0lQIdlGOm2xQ+5A56Eb+8WZKhSyU7PHGXIt2KLRvqNexEgmpJqRAbaKxfgDhRciwmjBn5qijH8+xLJeBgwCdF0g28Wk/dlwnHqhHPEecCpWyfNFEOIzC9SaXWUaPQeRaNAz1T+KXJOuhw0PhR9PdoehPqc+MLZdimNv7dfD7dM4/S2jXxW2qPBa3ys2pfFGZaTd7QFKRMzPVoyZ0seR6VlS1AVgfsRmKFB7s08H8f9Hlils8eL8jhYaU9AupM4hgBXCChcWa6qKUPNgXqMjny69frlRzCfMkQE2fSVotLFmrkFdZN7G5dwDuUG3tVCQ2h4lC6ui2ju13wacQ0LQdZgmKJ8nOoRPrGXBvETmEdnwK79KadI+68Xv/d4AEUAtaEebrq5NmhCDuoZS6wQCPzokHaHEC8VsPAnVMmxbbiiAN3WgnQqHrSHsQpClsEs/xHQ8Cf2xGwIPwWZxYIZtl8t8k2ytNefhe9A9WLD4HmbxiRhKSZdbWVpKWifK2BkUmQ+1+rMMhAqHZxzh8WfYzyI4FId+Y4gPH8u+GjQMrIy0GCOm3/YS6splALnoFsgVvr4hUG/jZhBQd5B+z9+9qwlWuYyG/rC2lXWK1v09Whc3Ij2J4+2A4pqm2XbFVoThgmOS5TFV3enRco44roQt/QEXyxBCqnLbLfLDGKMKq31QIZKbnNOPARPWRuHhB5u3g08c3ear4QRt4Ffbjdux4DNkG622sevVyoVQYUcH3nyx7Z3jvTdxwSQ6X09bYoS/7Q/zDkiLine truncated
MATERIAL_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNq1lMtugzAQRfd8heV1TNRKlar8QXdVt02EXDMhDthGfqRBiH/vQAmPvKRU6g7PjH3PXI+pI0IoHEEEb6yjK1JjAENlwatvK7OdH2IYTY3IwWLks4+QIdflpeIZYJoqYWMlhTXObH0sjFqOB64OT/HLa/zM9lypig77m/5rE51WzaKl25uvq2CJB+fdFO/UR0swaWBxyjsPpZvRU7EDkZswFp11ZFHDWEgEx8qJVp/OoZof2Md1qViKYqyuSSfhgiJrWnKRo0OsQB/jvTN6TUnTTLTPt9NZZjNZNaNvt9CDvgTWXHUX9Kad50VBUAV0ClpIcGccFK9NcZ225QhEhCSMFZBxUbESwHaE9CEkxw/3rOyV/uwcVvnd9fswKSTKpKEA92+efgRN3oexIwfpAi+IhQyHyEmjye/A3nP5SMa57cof9LebVW693HIxexsTg+avg1kojfXLc6wUxaXmHrmvbvgLWNtQgmaE4h5bW8X6quUNmctfRdREPykCLqM=")))
VOYAGE_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNrtW+lz27gV/+6/AqNJuz1MUlJ8ZNVRJr5qu9nYruWk24m8HIiEJFgkQAOgLMXR/u194KWD1JVEW+f4YEsEHvAefngnQD1uIVTqEyEpZ6UaqpqVbd1yx1sSHh/hOzy1Quq5BmauoYhUWTv0+NjpUkYmmqCR+rijm0phK2QqNKrV8k7NCYUgTJUSstF2OoUgkofCIbbjYamZljwsYHxGQFifCs58PXqKz5uDdycX9uXVTUOPMn71B8DohZ9nIRUJ9Mzvx2OdLnF6PFQZGzQxM/Tf398bXLQsyqTCnmff4T6e4h5R6VZ7jF6pWilNEIyy76Mxl2xmv89sB+AjNiCguCDzRBEhy3Nm2I8gfkcEbQ8RDgKPOliBHNuIMkU6InpAesPkNroPsUcV0DEXORwkxp1JfjEk3PehX88KsiHHI5ihfjy/cYiMmx1kGGHgYkUMyXAgu1xJaGLcUAIz2SbCCATvwHLkp8AgcX8uBhFAtl6MRiv0lMxDEmDV1cIrrT7KkqEgbSqIIUjAhVou0ucybGPqSdz+LIZYKNrGzgrc+nwIe2jeYTG7jy4ITVm0/ZoeVCOi2qg0kiqyRI5EBSM3kqngWkIF2tBgQxS3H7joyQA7JC+Z4Fz7iZI5K48WedoLJB3JGiL/Q+TMsHG/VgZjGVFO6+YRzt0/tAjUcecddrjDTTIgTp7mmDs9ItrUKxgfcN8c+F6+Q4qCmUyHszbtFFBzhiN7vyOOMuETdkdRMqX1CN0Wb/DWZG+y3SVJHEGUNKSDJx1eyY0WM7VxU2qRhRuH+h2rhSWZDTYTHBeEHemDoy99TtBY6KnP40iCTqkCz9qTC9zvA2wwMu5RV6lA1iyrQ1U3bMFW+PprNHr8RRD4BJW0XP7API5dq//CrFbMakZix8+2R1k4sAd7OybokNn50GTwiYzBhzZaTup0fe6ivw8y0iZbz3oXgnMdslWAMbM1IZcoUDwIP/E2IhO+xrqKzJTIVNz3oB2cTYvLed6mWBkxGPsnayLocSiYAgMVgSWJDzExqHk4yp7W0kmfuDT0N6eUjVg21Dho3KCGXu588JN1II0KajabDI0hx6His22BpdOjfCt/wDIwFIc/wvLd4AZCEQWK0KUq6ydCcJE93YH7gQ9YdhAqlEhmJJHajHpjUhN9/IiUCEmTbVJ+E1Vf/rmCgBUhE+LoEGCqgRoLsaEwXITAknhcOGTT0qWArCxbNmC9QBJr0Kz9bqRaWW6iU6BhpUCKRQkMTtKX9bxrXEPEqXR+zh4ZFqY+cRYC+TvtD41+eX+vbKwUwD/J1zhpPUGJO1mxgC8PCHMJc2gusyoxbsdWbivqE+3nYKrKrr/QUUFcICFHAQ2Izsoh0AE8yJAol3w1md9zKUTBADVLz84u35xYZoShFWHSLKVDMqVKErsmi6MBgjWDdxB+6ij6eqKr/xxbOV41SyvkZIPgwGDOuFmGNSvlPD0kJzMQQgqcNmXUGBJwmDOhKZv7e2b5lezi6u5erbpT2XMr7f2d8s5ua3e/ur/jtiv4+U6ZELLX2q2UYS53/7mzt4f3HKf1wsXln6utSmWftPbdarWKNMe2zHylXieDbB31Q0+jI4ned6g/z85Pz7aPrs9vzo8OftFefUAhp+YuQZVssAc5vgGZkhH0Orq+bHPhY4WmXX4KhjWpO7Hjj1Be29NC8bnAeLRaFdjK4yMCMZwuGo1WrjZ+n9qsz7G2tYu0GY1a4oQzZMGRJiPX8sJRgfBEXPDSLGlDbvlrOUZq6M068njoogOGvSHU2UuOh4xDFO1wLfqfGu9x9GQmZeFrMqzfA1+Dcju2nhk6LjqY0Q+RyqWUMyRdLpUZCq+eFkNRs6NFNXPEivcIqz97bFxeHFzbN5evTy5GMyTp0YOZFNBQCl9H5nClDbY+UXonJbY1JpyZSjq+6VKJWx5x6wvTu0IDieOHER2ufscmslAvDzU4KD7OQOf+4lPLJCBHgCJDoVjlas8ej86vj345sRtnB5VaubY/QmYWvRXuLKBLeuLK7YsWvDfa+aMjDiU47LVYVnvdXJ+/+6/97uS6cX55UW+WyuYuVOrN0oIsxOpjYUGzFVOYEj5qRY2F+cGzxymWIxQp4DjApwE9jeXbOrKPi7WCsE47DLyTEUIVNYD8L26eC/0XBfsqlF0EBd4pVWdhC11hpwdcF3k44nS5Tq4eT8+OUkcCmdRHlMDt8Q5lqNN1hHZCRjimfds4ub44eHOiyQ0jAPMDg3ENqVzKVtS6ZForcYjWApDy88W6Om+OVJOTcYEGZn12i4alHGgbvY9QSWY5vD64ODrTqNSh2QX98XgAT7e6Nl5A6YN9aLJ/INUlTOvMF8EwZTKebyUsJoYlKqJ1C9R51YFgsW3aZOsNHu9Aab0QI1vc//SjVA75353bq1UrZvkPOEr9winNKQG/CsqIGoeXb9BfjoaQMzBy/OtflyQ2kJOYTkzsDmrZN8OHCoEZgReC5dd83CMHnY4gHeBwyDO3exyXJ/+MSpb6xDlU2nMB0tX1tqTthxs7E0pTGWC20nlQRreGgiWHzwCL/vh0VWMQK2rVslmpPKkze30sPbPEtY4dWDCYGY/is3wjac2iY5q5np7fnL09nEldjXgrjZbAzOnWtUvMunQtZ+j8+N8grHF+aU0n2ikjQw0DUo90OOsK4jho6NXW42GFImeP4umJu5KybiUKW9K5a9vjDxPvNgSh5wHX+3DmzYbkDYg52jv3jYikv009BfVcrkMPjVAhRX3aLKIsqeDcICkNKMtdxSXGHAXU4k7rtwTWZtMy/2aVcjS3My2jrXlPK/in4mu8H8AUXSn9QGXeYX3m8u9DKkjhSdqMEU6fpG1/fwgXnzIkcvz/XRtn3jA+V4wlevo+Z5MSb8AZbBbgJ22lm1x60Tl2lv0xRQaqaNnjY8rFS/6moZsuP5eDFifH3y9gc46EVwCu64gvBlv+Xa9J95rvzVxZ0cAJt/F0t2NRbft0IueaGci3kPV/DSvfXCR/4hv+NWQETxrC7yOzeNJb8C1nKE8Z+B+Zzje5rX7IqOLCin8mZDuurQ98/1ALm9KszaacEeiL0dsuvlhIfkbluNGBeO5SIYOt9Jq3bx4IU8ObB245XihBYgMg4kHu+nr6d3KlQHC3YOL4bTpN0IPJuE8/kAKq6HbGTqUtvp8sGJZdPRGGmZIJoZVyiu6azCHO/foEhCVG/vImr2ALLm1mEH4rib4xTt5bCMKWR6O7Vlg/tFOJ9M0NVcU2s/hW5310XT1xN6uvqn9Hv70vGz9jo337uFMePUO3t00WCMpUG6jJQL9ihK6uL/91cnSTvcjx05/kT82mvt+euP+euO1FL1/qnsODxpl9cvFu9vq3SONmn9d8gXfefd73ZrdrXStxMflr2eQaOzIxyjpScN6zksZXtn15fZjuv20n6pf6zWgMb1erVtLwqmw+Ty5FR1ujrf8BDqZ8bQ==")))
VOYAGE_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNrtW+tz2zYS/56/AqPJXJKeSUqqHzndqBNbdm1NGzsnObneVDkNREISLBKgAVCWYuv+9lvwpQepV2K1TpoPsUhggV38drEPgLl/hlBhSISknBUqqGyW9nTLDe9IeL2HZ3jrBNR1DMwcQxGp0nbo8bDdp4zMNEEj9XBPNxWCTsBUYJTLxf2KHQhBmCrEZJO9ZApBJA+ETdq2i6VmWnCxgPEpAWFDKjjz9Og5Pm+PP5xdtq/eXTf1KOM3bwSMXntZFlIRX8/8+3Ss3Sf2gAcqZYNmZob+29tbg4uORZlU2HXbN3iI57iHVLq1PUWvUC4VZggm6fNkyiWd2Ruytg3wkTYgoLggy0QRActyZtgLIf5ABO2OEfZ9l9pYgRx7iDJFeiJ8QVphcg/dBtilCuiYg2wOEuPeLL8IEu550K9nBdmQ7RLM0DCa3zhBxvU+MozAd7AihmTYl32uJDQxbiiBmewSYfiC92A58nNgkHi4FIMQoLZejEYrcJXMQuJj1dfCK20+ypKBIF0qiCGIz4VaL9KXMuxi6krc/SKGWCjaxfYG3IZ8DDo0b7BY1KMDQlMWql/Tg2mEVDuVRlJF1sgRm2DoRlIT3EooX280UIji7TsuBtLHNslKJjjXfqJgLsqjRZ73AnFHvIbQ/xC5MGzar43BWEeUsbplhEv1h1aBOu28wTa3uUlGxM7SnHJ7QESXujnjfe6ZI8/NdkiRM5Npc9alvRxqznC432+IrUz4Be0oSuasHqGP+Qp+Ntsbq7sgiS2Ikoa08azDKzjhYuYUN2cWabixqdezOliSxWAzw3FF2JEeOPrClwSNlZ66HkUSdE4VeNaBXOF+70DByLhFfaV8WbGsHlX9oAOq8PRjOHr6IAj8gklaDr9jLseONXxtlktmOSVpR+9tl7Jg1B4d7ptgQ2bvU4vBLzJGn7poPand97iD/j5KSVtsu927EpxGwDYBxkzXhByiwPAg/ERqRCY8RraKzITIVNxzoR2cTYfLZd4m3xgxbPbPtkSw40AwBRtU+JYkHsREv+LiMHvayiY94tDA251RNiPZUPO4eY2aernLwY/XgTQqqNVqMTSFHAeKL7b5lk6Psq38DkvfUBz+EZbtBjcQiDBQBA5VaT8Rgov07QbcD/zAsv1AoVgyI47UZtgbkZro4QEpEZAW26X8Jir/9LcSAlaEzIijQ4CpRmoqxI7CcB4Ca+Jx7pBdS5cAsrFs6YDtAklkQYv7dyfVyvotOgcaVgqkWJXA4Dh92c67RjVElEpn5xyQcW7qE2UhkL/T4dgYFo8Oi8ZGAfyzfI2d1BOUOLMVC/hynzCHMJtmMqsC4+1ol7cV9Yj2czBV6cBb6aggLpCAI5/6RGflEOgAHmRIlEm+WswbOBSioI9ahecXV2/PLDPE0AoxaRWSIalRxYldi0XRAMGawTsIL3EUQz3Ru3+fWhleFUsb5GyD4MBgybhFhhUr4Tw/JCMzEEIKnDSl1BgScJgzpimaR4dm8Y3s4/LBYaW8Xzp0St2j/eL+QefgqHy073RL+Mf9IiHksHNQKsJcztGP9uEhPrTtzmsHF/9R7pRKR6Rz5JTLZaQ5dmXqK/U6GWTraBi4Gh1JtN6h/ryon1/s1Rr163rt+Fft1UcUcmruEFRKB7uQ4xuQKRn+oKfryy4XHlZo3uUnYFizthM5/hDlrT0tFJ8rNo82q5y9cn+PQAy7jyaTjauN/80p60t229ZF2oJFrXHCKbLgSOORW3nhsEB4Ii54bZa0I7f8tRwjNbWyai4PHHTMsDuGOnvN8ZBxgkINV8K/yeY9Dd/MuCz8hYyrt8DXoLwd7Z4FOi56mNFPockllAskfS6VGQi3mhRDYbOtRTUzxIoPCKs+v29eXR432tdXv5xdThZIkqMHMy6goRRuhNvhnd6w1ZnSOy6xrSnhwlTS9kyHStxxiVNdmd7lbpAofhjh4epfeIustMsTDQ6KjjNQ3Vt9ahkH5BBQZCgUmVzl+X2t3qj9etZuXhyXKsXK0QSZafRWuLeCLu6JKrdHLXivtfNHNQ4lOOharKu9rhv1D/9pfzhrNOtXl9VWoWgeQKXeKqzIQqwhFhY0WxGFKeGnkteYmx88v59jOUGhAU4DfBLQk1i+pyP7tFjLCeu0x8A7GQFUUSPI/6LmpdA/KtjvAtlHUOCdU3URdNA7bA+A6yoPR+w+18nV/flFLXEkkEk9oBhul/coQ72+LbQTMoIp7fvmWePy+O2ZJjcMH7YfbBjHkMqhbEOri6e1YodorQApO19kq8vmSCw5HudrYLZnt2pYwoF20e8hKvEsJ43jy9qFRqUKzQ7Yj8t9ePuoa+MVlB7sD032T6T6hGmbeRQMEybT+TbCYmZYbCLatsCcNx0IO7ZLW2y7wVMNFLYLMbLDvc8/SuWQ/904g0q5ZBb/gKPUR05pzgn4VTBG1Dy5eote1saQMzBy+turNYkN5CSmHRE7o0r6ZHhQITDDdwPY+RUPD8hxrydIDzic8NTtnkblyc9hyVKdOYdKei5BuqpWS9J+srMzoSSVAWYbnQeldFsYWHwdaDthTfEkT+3nwSMq8MGsPK5IOxVxAcCZTNwhXRy46hFj0fsQMVQ7RRoxdEdVHzFyF8dXh/b0GfEfH5harP72+PysfVo/P2teV5+/jOcDvrSrAwpUML4+cN/YV03P3VH5JyjThxYLXBckvblFhkAvzFMibUF9MGczWjayLEQ8X41fJOelr7RgYSQxPoUrmRVykhsWfM1ki5CGUP7Ck/Um5w/VF/f3FMrhETJ1vXAaSixRcTJ5sQUmD8gOYE7nDTK65VdRLEijQX3GBCoos9gtsIgnPGs0rhoVyDIDSIgZV+GlifAg4ZwzNz01jIGcDZUSkXrwEno+AKAjMLP7SEfi9EoKjKp+ffH+JDbBNzNXVP9qAAr1K+tDdF5SOzXeUZ+4wFVfzLSY7aBsl+YpIRQaFOSWDwkI8fmU+cPDElgeYOG6onPxWFpQdjqBrR2aNQjAT3pxaWmOsefG8OmFObTbhXXdBpSoDUdnsL3kyO5jBkLpzFLvUJjYiBFF2BUEO2MU+LpX7/gZkItzIEfXCoGEbFz7CJi6RqG+JbW6HrJIQkALUIeC13yjrQ2sLqHCjrMxEtGskcienq0PMeXldNCrCoo8e5xhxeYCS8nPSfR8Yf7EBdX+J0zapobdDGybSNmFrTmOJ3ZWe8BtE534EhTCs/75/DjEoGaplItmqfSk7o719ejCEpeHCOaPFohRtDuNuDUtyZLjkrmtnPZq7Rj65CXZ0PNHOMlshhr7pBpmR2mXH1VYhpa/Gg1rsRy50lfxJ8m0kY09i+2soI8+ui6/m/k0Tocb4HobLHwYF39At8Toln5QF/d3qQuOOpvb6aGhMyZ5fdqawyI759g5PlmiLPMlR5wLhvVYfqf133m1GYb5g7WGtNWyNFGG5uNCy+TZsrcNMuH8D0a+Y7jVvUf2O4fvAG5nhLmXzWmogCxDkNyboAUvMH8TtPddGSuUkX+gHov857thztxxVD5GEj19p7dLiXfgYnYL8JPe0Ltcet6VbZpgMkVGKm/Z0xu51Uv+pqGbP2ldD1qUkv91AVty+7kBcH1bPBps2c+aZ91rtjd1ZXkDZ9zG01XHqvL56UTOLZOVb6Hs+BpWvrtI/sQV/jVkBE8awr9GZvGkVfAtZyhPGfjvmc43qdalV98b76y9x1X4nJ3tNgENVfCIWC69vtm9l/oaUFtyHcHF7H/Sj7+e0YwGlPWk4HxgxY1v2u2rxknyMV+7raWYPJs8+z8FFeia")))
NEXT_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNrNVE1v2zAMvedXCFqAtlscb70tQLGtXTEUGFYgHXqJM0OxmUSJLCmS7DYr8t9HubaXOOnHYYfdLPLp6fGR9EOHEFqAsVxJOiCn/Q89H1FmYvH4gN94Wq1WAUYwQJdczqxRahlWwc9xfD08j28vhzdX1z/imOKVTcmxUNscOp8Ibudxyux8ophJ4wUzTRoBcA9J7pSP1Q+GC1Yw2qsR1oH2jKMqgKFkDslS5a4BkYZwW3nIpXVMiLgkHOxgEFVnpUoBs87k0NtF+Ez81yZ6ekq3AJvme/OEDpPL/Wcly/xz9BYMn66JmwPhWZY7NhFADAhgFkj9aEtQorKMydRfr5C1vLOIdh8uroYX3y/jn1++vSk2EY3k1CjpQKYNqnvsayKBJhEyrHJu4PioH2qWLNkM+gur5NFJv0JH9CSSGStAbt3PCkmCcxKsyByEHkDBRM4cUn6Fe23AljBt1AISVxNhcqpMAjcuxa4hqQPrUEG3rS+i5MzHW8X5UuorO3qewf/DTp3nXKRlo3BgHE8IVqqMe6Y5iTLgPSUgfVsjqaXOSDVwJAiw7t8gA6GS5ZQ3ebfWUI52dRZcujqF1VefE69m8CjhFWU2y4CN88sXp6CFWtf3xp365uP63imznAp1d2CHK5O3t7da9tHza1iuEdM6LqXv+4wWOGzcXgJTE8MkWnIo5zd4JtFob3jYfxvSFmDTa7M5NnuCSUmx3ilkKxcWB8hr5w653z69PH4v/SabCcM1uHcHhJZel6sRGJhx68w6SAykIB1nwu6qH/f+V/9p+KsYvQ8+jt9FUb/1cRwML5rDyafu/nuvacD+2Hc2nT/snNGl")))
ALL_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNq1T7EOgjAU3PmKpjNIdHQyJg4ukmDiYkwDUkxt4UkLEkP4dwsSUojootu7u/fu7lUWQvhOpWKQ4iVazOZ2w4AMlYaVnjXKsszRjCYwZ+lFSQDuduSKEM9fk8PG32+9HSFYn9StRwmSxwJKw0gHsfjhBGnkqAQ47RWtXaGNPHYY9YpZwQ2EIC8X47bbSYOkccRhwUTUhuRU5dhYqvu5rfglZ1xxFHMGSbE91G4SYiYmZUmzgkk6/LPTplsjdPrXD3EhxIcf3sk//sEy1QbVVm09Ad15mhY=")))
STARTER_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNp1UMGKwkAMvc9XlDmKtejR0yJ42MsKCl5EQnUzZbbTiZ0ZqiL9d1OpWqt7y3t5eS/JRUSRrNB5TVZOo8loPGwYcjvP8MI1o7IsY2aYkLm2mXdEedKSXwCL5QzW8+Xqe/EDIHmkvnkcyeXK0LFjFNAHIGvOD4rJP7plbVocPTrd7KSoLHwab2V7Kg7aIOxQkUPgi7RqZCo1Hod9rQ14Ci+ZnbQirdDGDjPtgzvHe4e/aINmI/ki3/ZslTaBH/m2W3N3mn3iuaMzy/s2n01Gg0T2BLX4Dz3re7UVd1SLWlwByepyJQ==")))
ANDROID_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNrtWG1v2zYQ/p5fQWgF6rSW3CXYMDhJ13YNkm5rkrrugKEKDFqiZCYUyZBUbC/wfvuO1LvtpmmBAcUwIYAp3vHuuXcqdzsIebdEaSq4N0R7wfd9uyPUVMPrHazhDfNYCRrDhhdRFTES0UG592Iv+CF45vULxjSSfsRoh7Hce7Ef7AV7HvCtnIYr0daQc2p8Q7Spt2CTLEiUG6Fae7DLcUasghJABcTPcDSjnJRQHKvBqeWMSYJzZrySsKo4PG2ItCA+NkeiGYmuRW7aYipNCvAJRfxU4ZgRPwKFbXUNxvaZnDu7dMeIAh1s+5HIMuC1MINBIXiOLOU1meap1zqxqterLdg0vn0YMAC0CaXy6XuQ4rQjsBVcpltSCvc0cLPrmCrkS/T3wFlSnhhc2VgOQp5QHqMA+WYpCUoQ0FOyQKEXPBlMc8ri7qngySJjoYd8G3QUSXS32i4YheFByNdxzWeEu6Rgc7zUn3VbxyMuqBOralJZveEgic3Myu9C+ho9WBma4OjhSgq7P6GqXF3utBRXObFRTxIriLOBYu9WlF5CIWQ+zXBK1lB5NnoWkzaK8rTj9rquhl0R+qACsP/sIBUiZWC1pPpg8dOP3gb0phq/kXLfWsbaQNx8kuUMA0Iftv2vquxIcE4iQ2JX3i8L6WMbp7WEXouJd3iImvAFbSp6/vzL8vDb6QDgeJnDAdz44f+G8C80hAyCyv2YSCaW/5UBe28WX+RTRvUMzamZIT3DisTorXUCimBJuKGY3ZfZmhhIvFwgSSXYRlnIh5C4j+5O3oxPP7yajM9/Oz4b/nxCzWk+RRc4urZ9D80VNaSlAlENZXSTU9hZhV7Iv0MvczOzxAgD55QkYC40hUxSBu31ALil0BScsEQ4iojWVgR0DFhhtewjLgySSogEwZ8s9JZqC/4g5FBmFNr1cpJroo4e9eTSzATfhzo5PHx88ef49PzscchpJoUy6EoLXr8IXS9zxRidBhY9BCUEnohhgHMmRgSqHnpYr8sSnI7HFxXtFPzIiNodhty6FvIDLCtIk5K/pwlL+tY9fZTIPjghJn2U6bSPZgTH0OXAWjIHJZUU+yhMNUGjHDyYkWOlhOqFXhkG3LgWLrO1Qlgk4Ik49HatGeCYUp6QhBOFjtZsDVxvmhTUXmPvbmNLSkzP1mYHWXF6U9yotDf0ZsZIPRwMYBgGKWRmPg0g9ANock+RFddvpNmndMPRXejZrBGK/uUsCz2bi68IZLVC7rDQAeG3VAn+EbzRStHQu1wTWj8gFBJGmkIalpKVjhvc8riE99Rlh7fabWS4iip8E9ifXmlmH9mIQF0f7T/bRdjmvZaCazLs6lfE5Iq7tAuYwHGv4it12JwFH1oPh559gbAFxRsTKeUuipaRJq4YqKYchjOPSM9y9xFk/i4SyhGdsHJdyHD1NXgHCeS/OR/c3Nz4RW+saqbRJonKqLafRTr0+jB9akKuZ8C2kZR/YJbXKfnWHuUpquSW1Vmjl3CbMg4xbJCFDQQ6dj+gsBRdiH3vRv3xgpom0+uG02o1EebWzLJrQCWUDa9lGUqUyKCCOQl50QdC7tS7ij8fnUxORi9f/348uRid/3r8y3hSZMG7mxsnalRK+gCobaM9gobY6TW2w32psAtoKnOhYiusm7khb65NPtymuBaMHEmGKUey6O9Okuv1ReqOxclWJaO6q8JF4SFzs/5KBWjXCRPz1qeqG8X2Auj6RHuclp+0rWHXfNbef7+sr+qfvzR8YpZXU7GYNl0Y94DZBqNDvFwbkgllGx8QJWmqoAxnZBvN/keBs6W7NEL8vDXyqvO+2pjL3JCF2WbSZgH77QHfteShgd9Z7fwDUg7law==")))
POLICY_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNqdVU1v2zAMvedXCL442Rq3aYEegubQQwYMWLGeBgxVESg2nWqVJU9S1gZB/vtIWXbsdCuw+WKJH4+PlEjtR4wlv8A6aXQyZ5fZ7IwkP8za4XaPa9ILJQvhYVoL78HqowqVhcmfwaLkIUpYpwt6WYkNoDrJZbU5r3f+yej5VTa7TDqrQ1w9nrWgFpzZ2hxWuRKOwiWuEkolnYHzULtB0CR/gvzZbH1ndELEbnWPdxRqUQVyn7XzGIDd777f3n3pQTTQpqqELsiwljWT0Xg6LaQTa4WFkfU0VnEaeESgxeI6u8iukh7c4Zj2/xD9Fs+CxbNgpVTg3iMcKn7FOL+5Yeny66eUa1nVxnq2E5XqNm7nuC6tqQj5Sck1i4p73HLNNVhrrGML9vDIdYy+ys1We5RdkEVpbMBcESesUnAdp1k6yexGmfU4/ZCRPp3MuSaesmRC78a1sD7DklrvXmT0YARGCsLpQDOSuNadvtxoL/UWKD7tvd31tATHTA163EFMmHCs7NnQhxUVmAUZZU6UsFJGFONy0qJGsmnbAOcp0XLe9nBPIC383EoLBRUslUV6xlIvvQJa5HiAG2N3tC7A5VbWHq9OUGE+oH36OESrpHNSbwisDKUpiUAXA7mVTJtQLMrlxBvVEeCEJH3NuWaixjJhzjzZdzkd5uwuBi4lqMLhtYxAB55MhljDK/FxwWZt9eA1h7q5bhl1xZIi0jFAj877NMiNroODxhCJwJFCDLAMPyzkv2Evh4B4ua3UnszaZivYfpDdYdh90Q2r3ASKkRsYnnCul03vlOhczDvadI6hTwZuR1dkwNh0kCl2aQav0o9nKADl4CTWLY6l9pKyMLeDK7Y913+ZQu3wHbW7MJeSF2OfS2Ve/vAO9Md/fCl6c/jta/E2wugw+g3AlOQ8")))
REPOSITORY_POLICY_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNqlVttu4zYQfc9XEH6RDETM7gYoiuz6YbE1ugFSJ2ncFkEcGLQ0splQpEpSid3A/94Z6hLZi1zQ5iEmh6MzZ4aHQz4dMDZ4AOuk0YMT9ol/PCTLnVk4nD7hmNaFkpnwkJRGyXTTLeBSZtJ7sGi5aSysWwvrshBLwOVBKovlUbnxK6NPjvnHT4POa9uMbg9bUAvOVDaFeaqEIx4DVwilBp2D81C6naCDdAXpval857RHxFa6x7sxalEEcqfaeQzA/ArY9dffzlgprMO0DnfdU1MUQmf0RZ3JMUsKVsqSyQYgSTLpxEJhqWSZNHVNAjd2sSHo0egn/oEfD3rI2+dS/Bfyfza7E9iLssQtEp7C5haALazQ6eodmcxmX75EF9fT7+eTaKZlURrr2Z0zupsYN9O5NZix8CslF6yxX+C0c3LVorQmBec607pQHDyS4WMFBWg/JWLCsfG089mIQs0wFEE7NurB8FC9OW5tWfn4JlpKHx2ySLkklwocjZN/otshzyA1GcRR5fPk52jIHVbCx9Fs9iEaPgPfENuYajdkubGMRrh9rF6XeTDcIi8cGV+bT2aaqmaFdMCuNqi+Yrwm7Ilh3go8ARmrjwarOWE8gqF4T9HZ6bfx5GpMRCfnU5zQ6Nv5xfXp5Nd6+Mv4/K/J+PcrmnHMTy61sdDOhPdWLipPudZEwl/EIZPe2NToXC7JuXjQj+0vT4ss2s60W4FS81BCYmMcB/0grdF8CZjA5eXl/OIcGV7Pr76Pz86iIRuNWORtBSgCV+W5XDdpcAQkRtaF+nO/rn8FtoAw2BSq/hXNgMQTBriTJVgvMQOkhMKTOThfw5am4Ov6gxIrid2i+y4Vdmm4N83qBmHuIPWdBRcLk+0UJRRmUUmV8aUVmQpF7M/5vQ+SceC91EvXc9szBU+kSyIhFXQiadSAAqEpl27uMHWp7+Mha3w57T5X5hEsGvHDLueTZ7I/yCmPvj6fXoJqcJmFvyuJTZEJFnJhWAkSGrYEirYlvfUpCTcvjZPruN5NjoI4erTYGsAeFeIBdNLM+J2wUY8SaglLUMGrcLWY/EbBUa29sH89lPGUU5twKFu9jAOEBZHNPax9DBpPKdpH7TkdvhJ+R73Yr2o+tSy78oYU3arPoNc9sHVi01gIdKBOEYSVJPgf6QVuw9tDFiKMpqj64SsggU1wrUEcYIOXfjN6FFZjSu+H/h+JFngMe5m+Vd239ja0P2p0KNK6Zb0UvXFqe8LrQv4De0dJjX23M35mDhQe4beETLlgd3hfcm0qe3yRa7zXlIY90ko6H5OVO5HDXBmMgjd4TJFaTYJ6ATvsRGhTPUSac8JxNQjeOXgAqBrtDZ2xJwU6SMMNt3tXxxH20Ypux/r4h4vkM1a9f6U3VaMuUy1aI9Whvrdn+oVnRfvCOmhn4aExeDT2Psec+k89VHS+88JrnoK9p9b+c/BH9IPtwb8HDzW4")))
MARKETING_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNrtXE1v2zYYvudXvNG6GhhCKXGxHRKkKDDsUAy95VCgDgRaoizOEqlQVByvzX/fS0qKbVmO3SzfYQ6JRT4k32/yoYV83wPwLpkquRTeMQz9owPTItW4xMfv+Bmf8kpwLRU2eFOZDIdB0/Dp0P/gHx15iLq2w2ZSTZNMzpbGFopEKYumiyZs/Efa6b81z3DTs7xcIGTMwnHFszikIg41K/XSJA1Y0JwZwSyOII5Y3EEHZaZaqOkNh11EQaMpnbAwpwL/WF1FkXdRZu4wkjmiYqOB923k/bIfVKUKxlwETFzCmJbpaCQKxYVOYPBrOcIfMYBBwQuWccGOwUhDZmxMiwLbl2VDDwyxKZbRlKlj/ARQlSxUrJBh3ZjwDGfQqmID+Ah+Yyt/nmcj7wBGRmhQlQBcSa+2WAu1TVdQZHQ+U3ySauCi1DTLIEqVzHmVAyEzrlMSs6JcncPof9zC6q5+9UumgbBKgtE6oTzDtnYSXExpFD7QeRFcXFwQlHVKSqbQCH4mJzD8+P4I3ttZTFtY8Pj03T4+a0ULGEw5yopG9d4t+s0jjgtidhmICvt//LBGOoEZ5XpX9AD++vr5DBdSjMbz04RmJcOnRCoIqdYsL4yx4PuR7/9xeH0CscReAJ7APlipyOG2pU5Ap0zYYQAR1Rut0EDYFYp/ZB8S3q4WVQrXIsau+KfEkBBobiIrXVQ4441ahOT0imieMxhCqnVxHASZjGiWSvTjh8PD34MVeWq1jSWahjG2TJcXLzPGCitPLIVBoTTfap3tYKvu/qm1JpwvTb5VVRalEgZfqJoyzcUE6m6IeYz5omHMMO1YLSFIAYXEGDIaYBq8H9YztKaysrbRZquPXbHsZgjGfioxv0beeTfRIyk0u9IrRarpmmBqVGNvpfm8MxyzVGNGr5Ur7BorKlCmvj5TeEU271nU9gUJo7pSaPXA/y3oSNxiUqkTftVA1hDnnZbrlefrbk3EKCGlxirQI1KP8CVWIhZicvOERrpfQ1tqdWpq56ICEVPglO5XKcaawwXVTeleH7U26Lqr5cG9y+4voifiO8m9GLC73HubfLfoWdKtdxOtd42wjKjYvHnyHPc8YjGdELA9YYvL2+w0u1fJNevCFZvwUisTwt4kjZTPZXCByUa4XIdeVFyxvtDqbuavMdXu5EwMNsV0ucWbDarXn2/KWhQPa1tMRU2F67MTbq2hFGHCRYzxbs969jjwlg1qT6qR+XVR0Yzr+WbDGhRpUQ90GHcV5I4uxAJRKXTM9jpiYb0J4pz4gE7cWx7buBTPE5csk4Xj0Y5HOx7teLTj0TtU9Zui6XjvG+O9T0+Vdoy910ttHtIAjoo8jckddXhcozfXd/XpvajKdLPNayipNbfQR7vLSxnNdBq2m0Swqd9Uf+zH48bhBkg83liObnP3LTkOt1Z52FwooRvGtwbRbQcdNN4zjsBesoncYoKx4cimI5uObDqy6cjmLlW4LZqObDqy+dhkc7fYe8Vk8wEN4Mjm05jckc3HNbojm45sPnAE9pJNxTJGcctGvfHkSDVztNPRTkc7He10tPPOb640NdW9COw46vN+d/T/BepbezH0Cazl2O8L8I+jys/YQ45XO179gsK1PrSGzcqbY7XGkRb3E55ej/I7u+WFHrB/4nakUDKuoub06K5F3LWIuxZx1yLuWmTrNXVOuXBfxb+1aw531r53Hva03zbtkMZ3CpSqMN80hVFsjwabo6TGkSi2YbxW5G6Ge3/L5GyGu8j8bCaDKKuwkCiClU5iOemMwl2eKylyRNsUxxNez8S4Z+bUAqY4GZ4X/mU9qNUg7cRdsDVocYY2CjUTVOjSjg0aeNAubfPan9M8W5uBh6hlyNBRmb0E4p/MDP561DfIVtYvtSfgz8+vlzs8WPg6knZfhl4lXeZ/We1d7/0Hg+30Mg==")))
HOMEBREW_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNqVVNtu2zAMfc9XCMYeNqBurk7SAgXmpRlaIM2lTVYMy2AoMh2rtSVDknNBkX+fpCReLkWxvdjU4eERRVJ6KyHkLEBIyplzjWqX1QuDvPCZ1Ms3bRs/TmiIFbgKZwWq8ZCTVxAa+bVDUOGzfpriOWi3M4+JuKS8HPMUZgKWZfP5KmNc85rXFdDfegheu0a8eoVc6Z/XqFVIhJst8GbNyozMvEa11a7WrloA9QZpepU2wU1oNZq47hRbbnbW74t9hgIkzwWBgCRYmhM5MsVJ4hSEJRevlM2DkAogiou14dg8ywll+cpmmwn+or1/o4AtqOAsBaYO6qEdd4OH7rfH7nPQHwT+ZDwIJsNbf9w1otUi/JTX93s/x/edpw9Z9/2nsd/rBZ1e1+9PhlvujropEpMKMnnUEIfEQF55rg6Ej5okcnZ0BgsynNrG/QBBozXSJUNZglXERYpCvmQJx6FEmIWIMqmMW8WAYIWJQnpIkIAFtSN1caLLA51KlqtA0RRMVnoTLz2lEZ6mWtz2CxRyIecooxlEmCZTZnpid5k6n0ajUVCUaewPp45Bh8+3U2fKFEhllp/nVIt0rGmDXVdAxiU1DX9f5YsW0odwMywkoLuuf2uQm0LtzLdPS+TbPc8ldwwBODQFc125ZgqvtMHlzRbBgsTW/EggAkViQ04Sd98UqdeznCahGwmeutuhf0+lfIx8Hzw+THp+ob3v5v9H7mv972HOQc8353e4tF/Z6bYXNUr48vBZssN5+CDtnq2D6T96us6lS5vSH3rbZEU=")))
APPLICATION_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNrFWH1v2zYT/7+fgvUzYDZm0W+JnaR1gaxx0W5N81Kvw1AHBi1SNmOJVCjKcdBmn/05kpYs2U6XdBsWBJZE3h3v9ccjvzxDqLJgKuFSVI5QG7fqZuRaThL4/ALvbp4Hd55mURwSzfIJmKLSnzMFI59XIyifs/M8IlPDUPF5NG3ImIlrOj9qt3CzkpPdr96u6plUxRKZKp+N/ZAkRpFKxChPo0pOkWgWJ6VlK/6M+XOZ6pxoQxUQqqUCmQQoCzaspufsrixwNX5zc5Nb7i1a3jVZkHbLowd7ZL930JkEzUN//4DttSek1243O+29XtBuTuh+r3fYabYrJXlXha/7tQMe0jgV23oKElmHnisWE8VQzIVgFN34OkSKCcoUxKNe5vFlFBFBDVvCNPJYKoEvZgHh4UhoKcMx5ao/qvzw5eLiYjwcnJ6/Px4OxsOzs/fjk3eXR94Pb89OBw1sfdcouSRb835UGYmILJgYg2ZyW9jp8afBh/Hl4PwsFxe1G4Y24RCYOyuAB+jzZ/QceRSBgEyzBp5yPaqgq6sXSM+YGAmEYARxAT8FOiPBzXivy+PgmkhqhgilSCo+5QLNtI6To0YDyGfpBIOPGheXqfDenTWMM+2SD0kLmPZnyPMoi/Ws38pEPjYpHpSb5bAVrcHZ6M1g+Prt+O3g+GQkAg7Bgiw2TNUHrFx4kBUJQ4ajBiP9Jyjl001n/g/9nPKQGqcjHkWpJpOQ1REJFSP0zrtJScgDDumXZUEdCakRgZ8ZU8iX8R2SgWX3pUjSiClcCHJQCrLxuge51dBETZnOv70mBlzC18RY+PVrzrhOtgaXjRuoFsuylmP5NsXEMtrIpGghkPcz8oSON1UCYryMQmDwTuxy2CyHQ+mTsF9SwZDEIcoWQx6JID0TTcIQeJM5j4cQucQF8V9ZEXLRBEH4d0cTEzTPgmdM9MyIglkMiRWn+g0PWb+4ZE6H9VKbqG/Ch5BjxzrWPGIGYgFIWs2o8s+h2aXNH5sobEl8jTJ8WZfELdSpec6knCf/GcI5R2wLOvtteP7bMBNT4ie+DzBBhM+shFup5tv8v59d/nrkNXQUl5mt+Q5b52ABhNyki5FhxvxUQXp5xkp4JBBZYbAjmclbjyklgd4zqaNhe4dXxbS6Q214i8jSM8FE3WYOhIrcYgeGacIU1KsGaWVcLGiWeEDY2KPtduB395uH3R5jrNs8bDV7e70u3ad+h/j7NAjYpNlrNXI2fEciUFZmZjRAOxLKqR03NjF/ZiYPO3ttFvRa3aDVAiGkvU+6B3vBBH67+2yPkqDT7NDD7sFkPyBd0qLU97uU7XVIdzJpHXYQ2iUefUXJDGR1AYvADda7xmGa6BSKE4jRq1yzi5ub4Urt14YQm70fZIxGL1/++Mvxp+MfAcuiWCqNzAwWXOIAYoBNiSUvHpg8h0rbmEs1D/F7LuaMviXJ7JTEuwhKwxAWHBBogxSgBSjmzxMpMAVjJ1xQ/At8fZCUPZrhbHLNfA1LxEw9himQKiLaehX/cXz6/g0ULWzia1Yw10AytnspJA7OMgBnPh0I2DPZUzhOCewdAKOP4lFEJEbLwZL5KegGTCMRp5OQ+8hCHtqM70h8MfsBWhGZpIDHQnKKIsJF9aNWXEw/XyHYn5IagJWStwkaLE15Q4FZXicB/nhQNWQYSnIKEPy8jzq1bC4ngj8rBQl2i96FIZuS8FhNYZ8UOpdbHVUGyxiiA/vsKpvrDhYZ9QwiZjbXUQBN7gw5jEIAF8wGZVSpvchWvM9eTBpmlH37haVT+XP7ak0PZthsxmzJYf+qOo7a95tytqEcilLoaEzXYFfYqWsxOSEU9tG3CxVnqmagkIvV2lpUVg+wJ1O2BGYnBZs+ZqgYq5Yc0LyqYS2N3UUZm1kIMlYvoEsahjkhZF01XxDMByOP3MIY+hrwQQ6GYO1OT4LXRxUwx/YtoCmko8XwUQUz03QlVSsWm10bKDkFQZgkQ7bUoHGtIKko1QnOtX7u1K49Lm4nqdOCrbdmq0MxYO6v4JaVlzV4eCg/kTBlTvH6ljOxLcmSqPutPChq33+S9tlyKOJJAkUMlSKjrJqKJpTBCUy0D5ds5bnqmgny76UDhzpyz1doQRQ33XKyYi7B+8tXVfg1+Vawd1SJlTTp/AGaolGlDgM+z/dbNzBVMo3fUfeRgZ/P4bskiCjNA6iCjHJLTgxQDifz9UpFWYWcL1rhfIHZAuJoIFNGUMiM5iExrYIbqdXXjGtZDkd8KDnNTlblz1kOKWvIEWbjSYVer+lasHKRtq5gGcdbWK6ep18Jw3IddbYtJNVaXgG7StAGfHMPqdYAB80rq2Yr75S80/5iGq/s66OmPdE41/Dkkk3TkCiLPG4BbO5CwgWDHM6PBrVHwu9HbZK1UAWXq04W0JdayHXBsG13CWV2KP3xDtqAyJwhcAwZrsOCQHNuRD+tgvaT+TCdTuKKbN3T20NMDh8vXCe/gn7rVJqvDMvCv+mxRsJ0QMjzbeNbNR3aNw4vtaOHz5OW1GDuXzd4O1tTM3z++4l7cdGxp6V/7hT0yV60oSkTTBGz3ReCAodqilZl+7TTz/Zp83uOLn95Mn3ybQ9y94r/1nnT3faZ+xI4LidpqJNtx5vEMcL/fMjsLHeSVLEAMMszFigQ9R2qZKD8d/QoALm5ENn0HAVrubD5YsTliVTsISzf09SHaH/j2tSs9HdvSevb/th5G/vnRhptMDqKb5zhH3Ud+6w4a76sYyoGEoIQWv7Nm/Hihfjq5rxwL715e74t/dn9s/8DqAZZ8g==")))
MARKETING_TEMPLATE = json.loads(zlib.decompress(base64.b64decode("eNrtW89v4ygUvuevQD7MSKsSp5lbD6OR9rRa7W1uk8oiGMc0GFzATbNV/vd9+EeaOHaqVpt6knBpbfh4wHufee9z3ZcRQsET04YrGdyh6fj2xrUoPTdw+wLXcJcVkluloSFYqmQ6DeuGH5Pxt/HtbQCoTTlspfQyEWq1MzbXmKaMLl+boPFBleZ/1fdo27M7XShVzKJ5wUUcERlHlhm7Y6QGS5Ixt7AShwGHS9xNC+VMvW4zmE7biJzQJVmwKCMSfpV7lXnWRjnbEVUZoGK3g+DXzMGQLiQSXNpZcINeW8o1NU3PKBdkvdJ8kVrEpbFECERTrTJeZAjjFbcpjllu9m24Ge8a2H6XcKZSVRg2C+7bK6VKWvZs97xcdy1gpmIe7DXft4YnXFhw14G/oWuuiYSQdvU55kix7pi07AsTRmyh2WwWjv8IWytuMKmyCX+uIQeI+1bLZtR393q9uTnOs1jRJdORoUT284tnQAtcYlquKnuiBpcRvWSWywVesbnhlrXhmi24sdo5KVikVI+5Ch8hnJirQ+hjwTUzXUFs8d0Hsw6mYVQza96IZo3qjOdVeYvAefaGqwDS7SfCRaRklHAZA9/L4zAhwrBrdmiZaKj78VgQwe2637EOhRvUifKVP0E+HELwPjj2WPhqxIlCl3G5u4pgcpjhq87IKiUc4qGj8NmCTAFFi15HObHp7vLD5gLXiPGDUbLXTF37lJluGXONcI6aTvTlC2I0VejryyywCgqcWXAHl1AYMVNd5hRKpLvbyWSz2XxF39H7VuHZ/EE2Q7orNBwzb2fFEtZ53Psj6YRBHO2OrUMaxOyJCZV74XRNwmkb9YusjU+6u3OoZU/pAF97DuVyXyv6WnEo9vna7nOdXr8trMqrvDBpv88rKK52XkI/7dVhyoiw6fYJDvv6lXaVTvBtMpn0QOJ5bzI8Fu4jGQb1HojoaPWB+hM4ahP8KL2OlXjg1t+Ym506ASriBbDG64Sr0glN1C9TJ5xyd2ehE07oAK8ThnK51wleJwzFPq8TPtfpXid4nTAYNzt1gmaCEcMw7DvmMbHMKwb/SVZNCv+JzjCuON/vbwbwlhcuZxEfr3K8yjkLqnpJ9BtHyOsnr58ugshVbR/VM/ezuMLhBvcODhzy/8NhuSgd0qmCc63igtrquPby93r+YJYRLv2LvsES1XuKmGFfvJ2KKEXuXrpFNIZckKt+llQ4TGNc4tq+2Q4P/lbJzxWTdv1zpUIqCgP7xvAsqdy0RzH5xLWSGaDdSHcIdhhOlM5ICViCMXjA/2UdqH2StngXvklasNCw0DJJpDXl2LCGh83UxJ3R4zXJxIEFHsEuIwaBKuUb5T+chfEh62tks9Z/qkigP/+63PR6Mvr6Oub/cvR+XeL+e3q0Gf0Hgnpc+w==")))
class ManifestError(ValueError):
@@ -72,20 +69,16 @@ def load_manifest(path, setup_version):
if not isinstance(data, dict):
raise ManifestError("Manifest must be a mapping")
profile = data.get("profile")
if profile not in ("qbit", "core", "frontend-material", "frontend-core", "cli", "voyage", "frontend-next", "all", "app-starter", "android", "mcp-policy", "repository-policy", "marketing", "node-artifact", "homebrew-tap", "application-template"):
if profile not in ("qbit", "core", "frontend-material", "frontend-core", "cli", "voyage", "frontend-next", "all", "app-starter", "android", "mcp-policy", "marketing"):
raise ManifestError("Unsupported profile")
allowed = ("profile", "orb_version", "default_branch", "docs_only", "publish")
if profile == "homebrew-tap":
allowed = ("profile", "orb_version", "default_branch", "tap", "formula")
elif profile == "node-artifact":
allowed = ("profile", "orb_version", "default_branch", "node_version", "context", "npm_scopes", "artifact_path", "build_branches")
elif profile in ("qbit", "core"):
if profile in ("qbit", "core"):
allowed += ("java_version", "browsers", "api_checks")
if profile == "qbit":
allowed += ("release_completion", "release_on_main", "release_tag_policy", "test_branch_policy")
elif profile == "cli":
allowed += ("publish_context",)
elif profile in ("voyage", "frontend-next", "app-starter", "android", "mcp-policy", "repository-policy", "marketing", "application-template"):
elif profile in ("voyage", "frontend-next", "app-starter", "android", "mcp-policy", "marketing"):
allowed = ("profile", "orb_version", "default_branch")
elif profile == "all":
allowed = ("profile", "orb_version", "default_branch", "publish", "publish_context")
@@ -93,12 +86,10 @@ def load_manifest(path, setup_version):
allowed += ("node_version",)
if profile == "frontend-material":
allowed += ("java_version",)
if profile == "repository-policy":
allowed += ("shell_checks",)
mapping(data, allowed, "manifest")
version = data.get("orb_version")
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+(?:\.\d+)?|dev:[A-Za-z0-9_-]+)", version):
raise ManifestError("orb_version must be a quoted minor track, exact stable pin, or development version")
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+\.\d+|dev:[A-Za-z0-9_-]+)", version):
raise ManifestError("orb_version must be an explicit stable or development version")
if version != setup_version:
raise ManifestError("Manifest orb_version must match the setup job orb_version")
default_branch = data.get("default_branch")
@@ -106,38 +97,12 @@ def load_manifest(path, setup_version):
raise ManifestError("default_branch must be main or develop")
if profile == "marketing" and default_branch != "main":
raise ManifestError("Marketing requires the audited main production branch")
if profile == "homebrew-tap":
tap, formula = data.get("tap"), data.get("formula")
if not isinstance(tap, str) or not re.fullmatch(r"[a-z0-9][a-z0-9-]*/[a-z0-9][a-z0-9-]*", tap):
raise ManifestError("Homebrew requires an explicit lowercase owner/tap")
if not isinstance(formula, str) or not re.fullmatch(r"[a-z0-9][a-z0-9+@._-]*", formula):
raise ManifestError("Homebrew requires a literal formula name")
return {"profile": profile, "orb_version": version, "default_branch": default_branch,
"tap": tap, "formula": formula}
java = data.get("java_version", "21")
if java not in ("17", "21"):
raise ManifestError("java_version must be the string 17 or 21")
node = data.get("node_version", "24.21" if profile == "frontend-core" else "20")
if not isinstance(node, str) or not re.fullmatch(r"\d+(?:\.\d+){0,2}", node):
raise ManifestError("node_version must be an explicit numeric version string")
if profile == "node-artifact":
if "node_version" not in data or not re.fullmatch(r"\d+\.\d+(?:\.\d+)?", node):
raise ManifestError("Node artifact requires an explicit minor or patch image version")
context = data.get("context", "")
scopes = data.get("npm_scopes", [])
artifact = data.get("artifact_path")
branches = data.get("build_branches")
if not isinstance(context, str) or not re.fullmatch(r"[A-Za-z0-9_-]+", context):
raise ManifestError("Node artifact requires an explicit context")
if not isinstance(scopes, list) or any(not isinstance(x, str) or not re.fullmatch(r"@[a-z0-9][a-z0-9-]*", x) for x in scopes) or len(scopes) != len(set(scopes)):
raise ManifestError("Invalid npm registry scopes")
if not isinstance(artifact, str) or not re.fullmatch(r"[A-Za-z0-9_-]+(?:/[A-Za-z0-9_.-]+)+", artifact) or any(x in (".", "..") for x in artifact.split("/")):
raise ManifestError("Artifact must be a relative file path")
if not isinstance(branches, list) or not branches or any(not isinstance(x, str) or not re.fullmatch(r"[A-Za-z0-9_-]+(?:/[A-Za-z0-9_.-]+)*", x) for x in branches) or len(branches) != len(set(branches)):
raise ManifestError("Build branches must be distinct literal branch names")
return {"profile": profile, "orb_version": version, "default_branch": default_branch,
"node_version": node, "context": context, "npm_scopes": scopes,
"artifact_path": artifact, "build_branches": branches}
publish_keys = ("snapshot", "release_candidate", "release")
if profile == "all":
publish_keys = ("snapshot",)
@@ -145,8 +110,6 @@ def load_manifest(path, setup_version):
publish_keys = ("release",)
if profile in ("qbit", "core"):
publish_keys += ("asciidoc",)
if profile == "qbit":
publish_keys += ("hotfix", "feature")
if profile not in ("qbit", "cli", "all", "app-starter", "voyage", "frontend-next"):
publish_keys += ("feature", "hotfix")
publishing = mapping(data.get("publish", {}), publish_keys, "publish")
@@ -163,20 +126,17 @@ def load_manifest(path, setup_version):
if tag_policy not in ("stable", "v-prefix", "none"):
raise ManifestError("Unsupported QBit release tag policy")
test_policy = data.get("test_branch_policy", "all")
if test_policy not in ("all", "non-release", "non-release-no-integration", "feature", "non-release-no-main",
"non-release-no-hotfix", "non-release-no-hotfix-integration", "non-release-no-main-hotfix", "feature-no-hotfix"):
if test_policy not in ("all", "non-release", "non-release-no-integration", "feature", "non-release-no-main"):
raise ManifestError("Unsupported QBit test branch policy")
return {"profile": profile, "orb_version": version, "default_branch": default_branch,
"java_version": java, "node_version": node, "browsers": boolean(data, "browsers", True),
"api_checks": boolean(data, "api_checks", False),
"shell_checks": boolean(data, "shell_checks", False),
"release_completion": boolean(data, "release_completion", False),
"release_on_main": boolean(data, "release_on_main", False),
"release_tag_policy": tag_policy, "test_branch_policy": test_policy,
"docs_only": boolean(data, "docs_only", False),
"publish_context": publish_context,
"publish": {key: boolean(publishing, key, key != "asciidoc" and profile not in ("cli", "all", "repository-policy", "application-template")
and not (profile == "qbit" and key in ("hotfix", "feature")))
"publish": {key: boolean(publishing, key, key != "asciidoc" and profile not in ("cli", "all"))
for key in publish_keys}}
@@ -211,28 +171,9 @@ def proven_docs_only(config, branch, tag):
return base.decode().strip() if eligible else False
def proven_ci_maintenance(branch, tag):
"""Keep application tests, but do not publish for isolated CI declarations."""
if tag or not (branch == "main" or branch == "develop" or branch.startswith(("release/", "hotfix/"))):
return False
subject = git("log", "-1", "--format=%s")
if not subject or not re.match(rb"ci(?:\([^\r\n()]+\))?: ", subject):
return False
parents = git("rev-list", "--parents", "-n", "1", "HEAD")
if not parents or not re.fullmatch(rb"[0-9a-f]{40} [0-9a-f]{40}\n", parents):
return False
base = parents.split()[1].decode("ascii")
changed = git("diff", "--no-ext-diff", "--no-renames", "--name-only", "-z", base, "HEAD")
if not changed:
return False
allowed = {b".qqq-ci.yml", b".circleci/config.yml", b".github/workflows/codeql.yml",
b".github/workflows/release-rc.yml", b".github/workflows/schema-drift.yml"}
return all(path in allowed for path in changed.rstrip(b"\0").split(b"\0"))
def event_kind(branch, tag, release_on_main=False, release_tag_policy="stable"):
if tag:
if release_tag_policy == "none" or not re.fullmatch(r"v.*" if release_tag_policy == "v-prefix" else r"v\d+\.\d+\.\d+", tag, flags=re.ASCII):
if release_tag_policy == "none" or not re.fullmatch(r"v.*" if release_tag_policy == "v-prefix" else r"v\d+\.\d+\.\d+", tag):
return "test"
tagged = git("rev-parse", "--verify", f"refs/tags/{tag}^{{commit}}")
if not tagged and git("fetch", "--no-tags", "origin", f"refs/tags/{tag}:refs/tags/{tag}") is not None:
@@ -250,36 +191,21 @@ def event_kind(branch, tag, release_on_main=False, release_tag_policy="stable"):
def qbit_template(config, branch, tag):
"""Profile-owned template; add future profiles as reviewed renderers, not raw YAML."""
kind = event_kind(branch, tag, config.get("release_on_main", False), config.get("release_tag_policy", "stable"))
if tag.startswith("publish") and config["publish"].get("feature", False):
tagged = git("rev-parse", "--verify", f"refs/tags/{tag}^{{commit}}")
if not tagged and git("fetch", "--no-tags", "origin", f"refs/tags/{tag}:refs/tags/{tag}") is not None:
tagged = git("rev-parse", "--verify", f"refs/tags/{tag}^{{commit}}")
if tagged and tagged == git("rev-parse", "HEAD"):
kind = "feature"
if not tag and re.fullmatch(r"hotfix/.+", branch) and config["publish"].get("hotfix", False):
kind = "hotfix"
if kind != "test" and not config["publish"][kind]:
kind = "test"
maintenance = proven_ci_maintenance(branch, tag)
if maintenance:
kind = "test"
workflow = "test_only" if kind == "test" else "publish_" + kind
result = {"version": 2.1, "orbs": {"qqq-orb": "kingsrook/qqq-orb@" + config["orb_version"]}}
filters = ({"branches": {"ignore": "/.*/"}, "tags": {"only": "/.*/"}} if tag else
{"branches": {"ignore": "automation/qbit-rc-pending"}, "tags": {"ignore": "/.*/"}})
if kind == "test" and not tag and not maintenance:
if kind == "test" and not tag:
ignores = {"all": None, "non-release": "/(develop|release/.*)/",
"non-release-no-integration": "/(develop|release/.*|integration.*)/",
"feature": "/(develop|main|release/.*|integration.*)/",
"non-release-no-main": "/(develop|main|release/.*)/",
"non-release-no-hotfix": "/(develop|release/.*|hotfix/.*)/",
"non-release-no-hotfix-integration": "/(develop|release/.*|hotfix/.*|integration.*)/",
"non-release-no-main-hotfix": "/(develop|main|release/.*|hotfix/.*)/",
"feature-no-hotfix": "/(develop|main|release/.*|hotfix/.*|integration.*)/"}
"non-release-no-main": "/(develop|main|release/.*)/"}
ignored = ignores[config.get("test_branch_policy", "all")]
if ignored:
filters["branches"]["ignore"] = ["automation/qbit-rc-pending", ignored]
docs_base = proven_docs_only(config, branch, tag) if kind == "test" and not maintenance else False
docs_base = proven_docs_only(config, branch, tag) if kind == "test" else False
if docs_base:
# Keep the existing workflow/result identity even when Maven is not needed.
result["jobs"] = {"docs_only": {"resource_class": "small", "docker": [{"image": "cimg/base:2026.03"}],
@@ -378,10 +304,7 @@ def frontend_template(config, branch, tag):
kind = "release"
if kind != "test" and not config["publish"][kind]:
kind = "test"
maintenance = proven_ci_maintenance(branch, tag)
if maintenance:
kind = "test"
if kind == "test" and not maintenance and proven_docs_only(config, branch, tag):
if kind == "test" and proven_docs_only(config, branch, tag):
result = qbit_template(config, branch, tag)
result["workflows"]["test_only"]["jobs"][0]["docs_only"]["name"] = (
"qqq-orb/mvn_frontend_test_only" if material else "qqq-orb/node_test_only")
@@ -442,18 +365,9 @@ def cli_template(config, branch, tag):
def marketing_template(config, branch, tag):
del config
# Select the existing branch policy once; automation PRs use normal PR checks.
selected = {"main": "production", "develop": "develop", "staging": "staging"}.get(
branch, "release-candidate" if branch.startswith("release/") else "pr-checks")
result = copy.deepcopy(MARKETING_TEMPLATE)
workflow = result["workflows"][selected]
for job in workflow["jobs"]:
next(iter(job.values()))["filters"] = {
"branches": {"ignore": "/.*/" if tag else "automation/qbit-rc-pending"},
"tags": {"ignore": "/.*/"}}
result["workflows"] = {"verify": workflow}
return result
del config, branch, tag
# One final continuation; Munitor supplies jobs, never another setup pipeline.
return copy.deepcopy(MARKETING_TEMPLATE)
def android_template(config, branch, tag):
@@ -489,69 +403,15 @@ def next_template(config, branch, tag):
def voyage_template(config, branch, tag):
del tag # Existing job filters retain tag exclusion and main-only release actions.
selected = "develop" if branch == "develop" else (
"release" if branch == "main" or branch.startswith("release/") else "pull-request")
result = json.loads(json.dumps(VOYAGE_TEMPLATE).replace("__ORB_VERSION__", config["orb_version"]))
result["workflows"] = {"verify": result["workflows"][selected]}
return result
del branch, tag # This profile's reviewed static filters own its three workflows.
return json.loads(json.dumps(VOYAGE_TEMPLATE).replace("__ORB_VERSION__", config["orb_version"]))
def repository_policy_template(config, branch, tag):
del branch, tag
result = copy.deepcopy(REPOSITORY_POLICY_TEMPLATE)
if config.get("shell_checks", False):
job = result["jobs"]["validate-policy"]
job["environment"] = {"QQQ_POLICY_SHELL": "true"}
job["steps"].insert(1, {"run": {
"name": "Install static shell validation",
"command": "sudo apt-get update -qq && sudo apt-get install -y --no-install-recommends shellcheck"}})
return result
def homebrew_template(config, branch, tag):
del branch, tag
result = copy.deepcopy(HOMEBREW_TEMPLATE)
result["jobs"]["validate-tap"]["environment"].update({
"QQQ_HOMEBREW_TAP": config["tap"], "QQQ_HOMEBREW_FORMULA": config["formula"]})
return result
def application_template(config, branch, tag):
del config, branch, tag
return copy.deepcopy(APPLICATION_TEMPLATE)
PROFILE_TEMPLATES = {"application-template": application_template, "homebrew-tap": homebrew_template, "qbit": qbit_template, "core": core_template,
"repository-policy": repository_policy_template,
PROFILE_TEMPLATES = {"qbit": qbit_template, "core": core_template,
"frontend-material": frontend_template, "frontend-core": frontend_template,
"cli": cli_template, "voyage": voyage_template, "frontend-next": next_template, "all": all_template, "app-starter": starter_template, "android": android_template, "mcp-policy": policy_template, "marketing": marketing_template}
def node_artifact_template(config, branch, tag):
# One checkout/install is shared by tests and an optional artifact build.
lines = ["registry=https://registry.npmjs.org/"]
lines += [scope + ":registry=https://npm.pkg.github.com/" for scope in config["npm_scopes"]]
if config["npm_scopes"]:
lines.append("//npm.pkg.github.com/:_authToken=${NPM_TOKEN}")
npmrc = "printf '%s\\n' " + " ".join("'" + line + "'" for line in lines) + " >> .npmrc"
steps = ["checkout", {"run": {"name": "Configure npm registries", "command": npmrc}},
{"restore_cache": {"keys": ['node-artifact-npm-v1-{{ checksum "package-lock.json" }}', 'node-artifact-npm-v1-']}},
{"run": {"name": "Install locked dependencies", "command": "npm ci --prefer-offline --cache ~/.npm"}},
{"save_cache": {"key": 'node-artifact-npm-v1-{{ checksum "package-lock.json" }}', "paths": ["~/.npm"]}},
{"run": {"name": "Test project", "command": "npm test"}}]
if not tag and branch in config["build_branches"]:
steps += [{"run": {"name": "Build artifact", "command": "npm run build"}},
{"store_artifacts": {"path": config["artifact_path"], "destination": config["artifact_path"].split("/")[-1]}}]
return {"version": 2.1,
"jobs": {"node_artifact": {"docker": [{"image": "cimg/node:" + config["node_version"]}], "steps": steps}},
"workflows": {"verify": {"jobs": [{"node_artifact": {"context": [config["context"]],
"filters": {"branches": {"ignore": "automation/qbit-rc-pending"}, "tags": {"ignore": "/.*/"}}}}]}}}
PROFILE_TEMPLATES["node-artifact"] = node_artifact_template
def runtime_boolean(name):
value = os.environ.get(name, "false").lower()
if value in ("true", "1"):
@@ -581,12 +441,6 @@ def main():
if expected and (not re.fullmatch(r"[0-9a-fA-F]{40}", expected) or git("rev-parse", "HEAD") != (expected + "\n").encode()):
raise ManifestError("Checkout does not match the pipeline commit")
output = PROFILE_TEMPLATES[config["profile"]](config, os.environ.get("CIRCLE_BRANCH", ""), os.environ.get("CIRCLE_TAG", ""))
if config["profile"] in ("qbit", "core", "frontend-core", "frontend-material"):
# One whole-workflow check follows the same commit from feature to release.
# Keep every selected job, dependency and filter; add no runner or duplicate build.
if len(output["workflows"]) != 1:
raise ManifestError("Stable verification requires exactly one selected workflow")
output["workflows"] = {"verify": next(iter(output["workflows"].values()))}
parameters = continuation_parameters(config["profile"])
if config["profile"] == "core":
output["parameters"] = copy.deepcopy(CORE_TEMPLATE["parameters"])
+1 -1
View File
@@ -116,7 +116,7 @@ def reconcile(api, repo):
validate_run(run, intent, repo, merge, run_id)
checks = api.request(prefix + 'commits/' + merge + '/check-runs?filter=latest&per_page=100')
publishers = [c for c in checks.get('check_runs', [])
if c.get('name') in ('publish_release_candidate', 'verify') and c.get('app', {}).get('id') == 18001]
if c.get('name') == 'publish_release_candidate' and c.get('app', {}).get('id') == 18001]
if not publishers:
return 'publisher-running'
publisher = max(publishers, key=lambda c: c['id'])
+3 -7
View File
@@ -28,17 +28,13 @@ if [[ "$CURRENT_BRANCH" == "main" ]]; then
TARGET_VERSION="$MAJOR_MINOR.0"
echo "Main branch detected - targeting release version: $TARGET_VERSION"
elif [[ "$CURRENT_BRANCH" == "develop" ]]; then
# Snapshot coordinates are unique per CI job without pushing a version commit.
# The source head stays the head whose required checks are being reported.
if [[ ! "${CIRCLE_BUILD_NUM:-}" =~ ^[1-9][0-9]*$ ]]; then
echo "ERROR: Snapshot publication requires a positive CIRCLE_BUILD_NUM" >&2
exit 1
fi
# Develop branch - should be a snapshot version (e.g., 1.0.127-SNAPSHOT)
# Increment patch version for develop
MAJOR_MINOR_PATCH="${NPM_VERSION%-*}"
MAJOR_MINOR="${MAJOR_MINOR_PATCH%.*}"
PATCH="${MAJOR_MINOR_PATCH##*.}"
NEW_PATCH=$((PATCH + 1))
TARGET_VERSION="$MAJOR_MINOR.$NEW_PATCH-SNAPSHOT.$CIRCLE_BUILD_NUM"
TARGET_VERSION="$MAJOR_MINOR.$NEW_PATCH-SNAPSHOT"
echo "Develop branch detected - targeting snapshot version: $TARGET_VERSION"
elif [[ "$CURRENT_BRANCH" == release/* ]]; then
# Release branch - should be a release candidate (e.g., 1.0.0-RC.1, 1.0.0-RC.2, etc.)
@@ -1,70 +0,0 @@
{
"version": 2.1,
"jobs": {
"verify-template": {
"docker": [
{
"image": "cimg/openjdk:21.0"
}
],
"resource_class": "medium",
"steps": [
"checkout",
{
"restore_cache": {
"keys": [
"qqq-template-v1-java21-d84a5783bf09c58e42ba72203247f20bd5779302"
]
}
},
{
"run": {
"name": "Prepare pinned qctl renderer",
"command": "set -euo pipefail\ntool_dir=\"${QQQ_TEMPLATE_TOOL_DIR:-$HOME/.cache/qqq-template-renderer}\"\nmaven_repo=\"${QQQ_TEMPLATE_MAVEN_REPO:-$HOME/.m2/repository}\"\nif [[ ! -d \"$tool_dir/.git\" ]]; then\n git init \"$tool_dir\"\n git -C \"$tool_dir\" remote add origin https://github.com/QRun-IO/qctl.git\n git -C \"$tool_dir\" fetch --depth=1 origin d84a5783bf09c58e42ba72203247f20bd5779302\n git -C \"$tool_dir\" checkout --detach FETCH_HEAD\nfi\ntest \"$(git -C \"$tool_dir\" rev-parse HEAD)\" = d84a5783bf09c58e42ba72203247f20bd5779302\ncd \"$tool_dir\"\n# Build the immutable, already-qualified renderer, not another copy of the consumer.\nif [[ ! -f \"$tool_dir/qctl-qqq/target/qctl-qqq-0.2.1.jar\" || ! -f \"$maven_repo/io/qrun/qctl/qctl-qqq/0.2.1/qctl-qqq-0.2.1.pom\" ]]; then\n mvn -B -ntp -f \"$tool_dir/pom.xml\" -Dmaven.repo.local=\"$maven_repo\" -pl qctl-qqq -am install -DskipTests\nfi\nmvn -B -ntp -f \"$tool_dir/pom.xml\" -Dmaven.repo.local=\"$maven_repo\" -pl qctl-qqq dependency:build-classpath -Dmdep.outputFile=\"$tool_dir/classpath.txt\"\n",
"no_output_timeout": "10m"
}
},
{
"run": {
"name": "Render the exact template checkout without hooks",
"command": "set -euo pipefail\ntool_dir=\"${QQQ_TEMPLATE_TOOL_DIR:-$HOME/.cache/qqq-template-renderer}\"\noutput=\"${QQQ_TEMPLATE_OUTPUT:-$HOME/qqq-template-acceptance}\"\nwork=\"${QQQ_TEMPLATE_WORK:-/tmp/qqq-template-check}\"\nmkdir -p \"$work\"\ncurl --fail --silent --show-error --location --retry 2 --max-time 60 https://raw.githubusercontent.com/QRun-IO/qqq-templates-hub/4d22fc650967eee609107476d5dc3ac5dffeb071/templates.yaml -o \"$work/catalog.yaml\"\necho \"9342ef716f11ffea25a684fba6865e4daf303d968b5fa6a1ddcc6de43a6bb193 $work/catalog.yaml\" | sha256sum --check --status\ncat > \"$work/QqqTemplateCheck.java\" \\<<'JAVA'\nimport java.nio.file.Files;\nimport java.nio.file.Path;\nimport java.util.LinkedHashMap;\nimport java.util.Map;\nimport com.fasterxml.jackson.databind.JsonNode;\nimport com.fasterxml.jackson.databind.ObjectMapper;\nimport com.fasterxml.jackson.dataformat.yaml.YAMLFactory;\nimport io.qrun.qctl.qqq.template.TemplateEngine;\nimport io.qrun.qctl.qqq.template.TemplateManifest;\nimport io.qrun.qctl.qqq.template.TransformExecutor;\n\npublic class QqqTemplateCheck\n{\n public static void main(String[] args) throws Exception\n {\n if(args.length != 3)\n {\n throw new IllegalArgumentException(\"Expected catalog, checked-out template, fresh output directory\");\n }\n Path output = Path.of(args[2]);\n if(Files.exists(output))\n {\n throw new IllegalArgumentException(\"Output directory must not exist\");\n }\n ObjectMapper mapper = new ObjectMapper(new YAMLFactory());\n JsonNode index = mapper.readTree(Path.of(args[0]).toFile());\n TemplateManifest manifest = null;\n for(JsonNode entry : index.get(\"templates\"))\n {\n if(\"new-qqq-application\".equals(entry.path(\"id\").asText()))\n {\n if(manifest != null) throw new IllegalArgumentException(\"Duplicate template entry\");\n manifest = mapper.treeToValue(entry, TemplateManifest.class);\n }\n }\n if(manifest == null) throw new IllegalArgumentException(\"Template missing from catalog\");\n TemplateEngine engine = new TemplateEngine();\n Map<String, String> variables = new LinkedHashMap<>(Map.of(\n \"projectName\", \"ci-template\", \"groupId\", \"io.qrun.ci\",\n \"artifactId\", \"ci-template\", \"packageName\", \"io.qrun.ci\"));\n variables = engine.evaluateComputed(manifest.computed(), variables);\n Files.createDirectories(output);\n int count = engine.render(Path.of(args[1]), output, variables, manifest);\n if(manifest.transforms() != null)\n {\n new TransformExecutor().execute(output, manifest.transforms(), variables);\n }\n if(count == 0 || !Files.isRegularFile(output.resolve(\"pom.xml\")))\n {\n throw new IllegalStateException(\"Renderer did not create the application\");\n }\n System.out.println(\"Rendered \" + count + \" files from the exact local template; hooks not executed\");\n }\n}\nJAVA\njava -cp \"$(cat \"$tool_dir/classpath.txt\"):$tool_dir/qctl-qqq/target/classes\" \"$work/QqqTemplateCheck.java\" \"$work/catalog.yaml\" \"$PWD\" \"$output\"\n"
}
},
{
"run": {
"name": "Verify generated application and package",
"command": "set -euo pipefail\nmvn -B -ntp -f \"${QQQ_TEMPLATE_OUTPUT:-$HOME/qqq-template-acceptance}/pom.xml\" -Dmaven.repo.local=\"${QQQ_TEMPLATE_MAVEN_REPO:-$HOME/.m2/repository}\" verify\n",
"no_output_timeout": "10m"
}
},
{
"store_test_results": {
"path": "~/qqq-template-acceptance/target/surefire-reports"
}
},
{
"store_artifacts": {
"path": "~/qqq-template-acceptance/target/ci-template.jar",
"destination": "generated-application.jar"
}
},
{
"save_cache": {
"key": "qqq-template-v1-java21-d84a5783bf09c58e42ba72203247f20bd5779302",
"paths": [
"~/.m2/repository",
"~/.cache/qqq-template-renderer"
]
}
}
]
}
},
"workflows": {
"verify": {
"jobs": [
"verify-template"
]
}
}
}
+1 -44
View File
@@ -263,7 +263,7 @@
},
"concurrency": {
"group": "e2e-${{ github.ref }}",
"cancel-in-progress": "${{ github.event_name == 'pull_request' }}"
"cancel-in-progress": "${{ github.event_name == 'pull_request' }}"
},
"jobs": {
"validate": {
@@ -426,49 +426,6 @@
"uses": "QRun-IO/qqq-orb/.github/workflows/gitops.yml@WORKER_SHA"
}
}
},
"additional_repositories": [
"QRun-IO/voyage-cd"
]
},
"qbit-codeql": {
"repository": null,
"profile": "qbit",
"path": ".github/workflows/codeql.yml",
"caller": {
"name": "CodeQL",
"on": {
"push": {
"branches": [
"main",
"develop",
"release/**",
"hotfix/**"
]
},
"pull_request": {},
"schedule": [
{
"cron": "41 16 * * 0"
}
],
"workflow_dispatch": {}
},
"permissions": {
"contents": "read",
"actions": "read",
"security-events": "write",
"packages": "read"
},
"concurrency": {
"group": "codeql-${{ github.event.pull_request.number || github.ref }}",
"cancel-in-progress": "${{ github.event_name == 'pull_request' }}"
},
"jobs": {
"analyze": {
"uses": "QRun-IO/qqq-orb/.github/workflows/qbit-codeql.yml@WORKER_SHA"
}
}
}
}
}
+4 -6
View File
@@ -238,7 +238,7 @@
{
"run": {
"name": "Build framework dependencies for the sample",
"command": "mvn -B -s /tmp/circleci/mvn-settings.xml -Drevision=0.0.0-sample-acceptance -DskipTests -Dspotbugs.skip=true -Dpmd.skip=true clean install"
"command": "mvn -B -s /tmp/circleci/mvn-settings.xml -Drevision=0.0.0-sample-acceptance -DskipTests clean install"
}
},
{
@@ -308,7 +308,7 @@
{
"run": {
"name": "Build framework dependencies for the sample",
"command": "mvn -B -s /tmp/circleci/mvn-settings.xml -Drevision=0.0.0-sample-multinode -DskipTests -Dspotbugs.skip=true -Dpmd.skip=true clean install"
"command": "mvn -B -s /tmp/circleci/mvn-settings.xml -Drevision=0.0.0-sample-multinode -DskipTests clean install"
}
},
{
@@ -336,7 +336,7 @@
{
"run": {
"name": "Build framework dependencies for the consumers",
"command": "mvn -B -s /tmp/circleci/mvn-settings.xml -Drevision=0.0.0-sample-acceptance -DskipTests -Dspotbugs.skip=true -Dpmd.skip=true clean install"
"command": "mvn -B -s /tmp/circleci/mvn-settings.xml -Drevision=0.0.0-sample-acceptance -DskipTests clean install"
}
},
{
@@ -649,9 +649,7 @@
"qqq-orb/mvn_publish": {
"requires": [
"sample_bootstrap_source",
"snapshot_esb_broker_conformance",
"snapshot_sample_coverage_gate",
"qqq-orb/static_analysis"
"snapshot_esb_broker_conformance"
],
"pre-steps": [
"configure-package-install"
-36
View File
@@ -1,36 +0,0 @@
{
"version": 2.1,
"jobs": {
"validate-tap": {
"docker": [
{
"image": "ghcr.io/homebrew/brew@sha256:0e2563de582c530c92c55420cfa67e5b60bcb541781297ee34c6508ca6e746a3"
}
],
"resource_class": "small",
"working_directory": "/home/linuxbrew/project",
"environment": {
"HOMEBREW_NO_AUTO_UPDATE": "1",
"HOMEBREW_NO_ANALYTICS": "1",
"HOMEBREW_NO_INSTALL_CLEANUP": "1"
},
"steps": [
"checkout",
{
"run": {
"name": "Verify all platform downloads and install the exact tap revision",
"no_output_timeout": "5m",
"command": "set -euo pipefail\nbrew tap \"$QQQ_HOMEBREW_TAP\" \"$PWD\"\ntest \"$(git -C \"$(brew --repository \"$QQQ_HOMEBREW_TAP\")\" rev-parse HEAD)\" = \"$(git rev-parse HEAD)\"\nbrew trust \"$QQQ_HOMEBREW_TAP\"\nbrew readall --syntax --os=all --arch=all \"$QQQ_HOMEBREW_TAP\"\nbrew fetch --all-platforms --build-from-source \"$QQQ_HOMEBREW_TAP/$QQQ_HOMEBREW_FORMULA\"\nbrew install \"$QQQ_HOMEBREW_TAP/$QQQ_HOMEBREW_FORMULA\"\nbrew test \"$QQQ_HOMEBREW_TAP/$QQQ_HOMEBREW_FORMULA\"\n"
}
}
]
}
},
"workflows": {
"verify": {
"jobs": [
"validate-tap"
]
}
}
}
+94 -80
View File
@@ -11,7 +11,7 @@
"name": "build-and-test",
"node_version": "22",
"package_manager": "npm",
"test_commands": "[\"#!/usr/bin/env bash\\nprintf '%s\\\\n' 'pipeline: node-webapp' 'node_version: 22' 'docker:' ' use_repo_dockerfile: true' > .munitor.yml\", \"npm run lint\", \"npm run build\", \"npx playwright install chromium --with-deps\", \"npm run test:chromium\", \"#!/usr/bin/env bash\\nset -euo pipefail\\nnpm run start > /tmp/qqq-link-server.log 2>&1 &\\nserver_pid=$!\\ntrap 'kill \\\"$server_pid\\\" 2>/dev/null || true; wait \\\"$server_pid\\\" 2>/dev/null || true' EXIT\\nready=false\\nfor _attempt in {1..60}; do\\n if ! kill -0 \\\"$server_pid\\\" 2>/dev/null; then\\n cat /tmp/qqq-link-server.log\\n exit 1\\n fi\\n if curl --fail --silent --output /dev/null --max-time 2 http://localhost:3005/; then\\n ready=true\\n break\\n fi\\n sleep 1\\ndone\\nif [ \\\"$ready\\\" != true ]; then\\n cat /tmp/qqq-link-server.log\\n echo 'Marketing server did not become ready on port 3005' >&2\\n exit 1\\nfi\\nnpm run check-links\", \"npm run lighthouse\"]",
"test_commands": "[\"npm run lint\", \"npm run build\", \"npx playwright install chromium --with-deps\", \"npm run test:chromium\", \"npm run lighthouse\"]",
"context": [
"github"
],
@@ -22,21 +22,7 @@
"/hotfix\\/.*/"
]
}
},
"post-steps": [
{
"store_artifacts": {
"path": "playwright-report",
"destination": "playwright-report"
}
},
{
"store_artifacts": {
"path": ".lighthouseci",
"destination": "lighthouse-report"
}
}
]
}
}
},
{
@@ -102,6 +88,28 @@
}
}
},
{
"munitor/node_coverage": {
"name": "coverage",
"node_version": "22",
"package_manager": "npm",
"min_coverage": "0",
"coverage_tool": "jest",
"coverage_summary_path": "coverage/coverage-summary.json",
"coverage_command": "mkdir -p coverage && echo '{\"total\":{\"lines\":{\"pct\":100}}}' > coverage/coverage-summary.json",
"requires": [
"build-and-test"
],
"filters": {
"branches": {
"only": [
"/feature\\/.*/",
"/hotfix\\/.*/"
]
}
}
}
},
{
"munitor/node_security_scan": {
"name": "security-scan",
@@ -129,7 +137,7 @@
"name": "build-and-test",
"node_version": "22",
"package_manager": "npm",
"test_commands": "[\"#!/usr/bin/env bash\\nprintf '%s\\\\n' 'pipeline: node-webapp' 'node_version: 22' 'docker:' ' use_repo_dockerfile: true' > .munitor.yml\", \"npm run lint\", \"npm run build\", \"npx playwright install chromium --with-deps\", \"npm run test:chromium\", \"#!/usr/bin/env bash\\nset -euo pipefail\\nnpm run start > /tmp/qqq-link-server.log 2>&1 &\\nserver_pid=$!\\ntrap 'kill \\\"$server_pid\\\" 2>/dev/null || true; wait \\\"$server_pid\\\" 2>/dev/null || true' EXIT\\nready=false\\nfor _attempt in {1..60}; do\\n if ! kill -0 \\\"$server_pid\\\" 2>/dev/null; then\\n cat /tmp/qqq-link-server.log\\n exit 1\\n fi\\n if curl --fail --silent --output /dev/null --max-time 2 http://localhost:3005/; then\\n ready=true\\n break\\n fi\\n sleep 1\\ndone\\nif [ \\\"$ready\\\" != true ]; then\\n cat /tmp/qqq-link-server.log\\n echo 'Marketing server did not become ready on port 3005' >&2\\n exit 1\\nfi\\nnpm run check-links\", \"npm run lighthouse\"]",
"test_commands": "[\"npm run lint\", \"npm run build\", \"npx playwright install chromium --with-deps\", \"npm run test:chromium\", \"npm run lighthouse\"]",
"context": [
"github"
],
@@ -137,21 +145,7 @@
"branches": {
"only": "develop"
}
},
"post-steps": [
{
"store_artifacts": {
"path": "playwright-report",
"destination": "playwright-report"
}
},
{
"store_artifacts": {
"path": ".lighthouseci",
"destination": "lighthouse-report"
}
}
]
}
}
},
{
@@ -190,6 +184,25 @@
}
}
},
{
"munitor/node_coverage": {
"name": "coverage",
"node_version": "22",
"package_manager": "npm",
"min_coverage": "0",
"coverage_tool": "jest",
"coverage_summary_path": "coverage/coverage-summary.json",
"coverage_command": "mkdir -p coverage && echo '{\"total\":{\"lines\":{\"pct\":100}}}' > coverage/coverage-summary.json",
"requires": [
"build-and-test"
],
"filters": {
"branches": {
"only": "develop"
}
}
}
},
{
"munitor/node_security_scan": {
"name": "security-scan",
@@ -215,6 +228,7 @@
"health_db": "false",
"requires": [
"code-quality",
"coverage",
"secrets-scan",
"sast-scan",
"security-scan"
@@ -238,7 +252,7 @@
"name": "build-and-test",
"node_version": "22",
"package_manager": "npm",
"test_commands": "[\"#!/usr/bin/env bash\\nprintf '%s\\\\n' 'pipeline: node-webapp' 'node_version: 22' 'docker:' ' use_repo_dockerfile: true' > .munitor.yml\", \"npm run lint\", \"npm run build\", \"npx playwright install chromium --with-deps\", \"npm run test:chromium\", \"#!/usr/bin/env bash\\nset -euo pipefail\\nnpm run start > /tmp/qqq-link-server.log 2>&1 &\\nserver_pid=$!\\ntrap 'kill \\\"$server_pid\\\" 2>/dev/null || true; wait \\\"$server_pid\\\" 2>/dev/null || true' EXIT\\nready=false\\nfor _attempt in {1..60}; do\\n if ! kill -0 \\\"$server_pid\\\" 2>/dev/null; then\\n cat /tmp/qqq-link-server.log\\n exit 1\\n fi\\n if curl --fail --silent --output /dev/null --max-time 2 http://localhost:3005/; then\\n ready=true\\n break\\n fi\\n sleep 1\\ndone\\nif [ \\\"$ready\\\" != true ]; then\\n cat /tmp/qqq-link-server.log\\n echo 'Marketing server did not become ready on port 3005' >&2\\n exit 1\\nfi\\nnpm run check-links\", \"npm run lighthouse\"]",
"test_commands": "[\"npm run lint\", \"npm run build\", \"npx playwright install chromium --with-deps\", \"npm run test:chromium\", \"npm run lighthouse\"]",
"context": [
"github"
],
@@ -246,21 +260,7 @@
"branches": {
"only": "staging"
}
},
"post-steps": [
{
"store_artifacts": {
"path": "playwright-report",
"destination": "playwright-report"
}
},
{
"store_artifacts": {
"path": ".lighthouseci",
"destination": "lighthouse-report"
}
}
]
}
}
},
{
@@ -299,6 +299,25 @@
}
}
},
{
"munitor/node_coverage": {
"name": "coverage",
"node_version": "22",
"package_manager": "npm",
"min_coverage": "0",
"coverage_tool": "jest",
"coverage_summary_path": "coverage/coverage-summary.json",
"coverage_command": "mkdir -p coverage && echo '{\"total\":{\"lines\":{\"pct\":100}}}' > coverage/coverage-summary.json",
"requires": [
"build-and-test"
],
"filters": {
"branches": {
"only": "staging"
}
}
}
},
{
"munitor/node_security_scan": {
"name": "security-scan",
@@ -324,6 +343,7 @@
"health_db": "false",
"requires": [
"code-quality",
"coverage",
"secrets-scan",
"sast-scan",
"security-scan"
@@ -347,7 +367,7 @@
"name": "build-and-test",
"node_version": "22",
"package_manager": "npm",
"test_commands": "[\"#!/usr/bin/env bash\\nprintf '%s\\\\n' 'pipeline: node-webapp' 'node_version: 22' 'docker:' ' use_repo_dockerfile: true' > .munitor.yml\", \"npm run lint\", \"npm run build\", \"npx playwright install chromium --with-deps\", \"npm run test:chromium\", \"#!/usr/bin/env bash\\nset -euo pipefail\\nnpm run start > /tmp/qqq-link-server.log 2>&1 &\\nserver_pid=$!\\ntrap 'kill \\\"$server_pid\\\" 2>/dev/null || true; wait \\\"$server_pid\\\" 2>/dev/null || true' EXIT\\nready=false\\nfor _attempt in {1..60}; do\\n if ! kill -0 \\\"$server_pid\\\" 2>/dev/null; then\\n cat /tmp/qqq-link-server.log\\n exit 1\\n fi\\n if curl --fail --silent --output /dev/null --max-time 2 http://localhost:3005/; then\\n ready=true\\n break\\n fi\\n sleep 1\\ndone\\nif [ \\\"$ready\\\" != true ]; then\\n cat /tmp/qqq-link-server.log\\n echo 'Marketing server did not become ready on port 3005' >&2\\n exit 1\\nfi\\nnpm run check-links\", \"npm run lighthouse\"]",
"test_commands": "[\"npm run lint\", \"npm run build\", \"npx playwright install chromium --with-deps\", \"npm run test:chromium\", \"npm run lighthouse\"]",
"context": [
"github"
],
@@ -357,21 +377,7 @@
"/release\\/.*/"
]
}
},
"post-steps": [
{
"store_artifacts": {
"path": "playwright-report",
"destination": "playwright-report"
}
},
{
"store_artifacts": {
"path": ".lighthouseci",
"destination": "lighthouse-report"
}
}
]
}
}
},
{
@@ -416,6 +422,27 @@
}
}
},
{
"munitor/node_coverage": {
"name": "coverage",
"node_version": "22",
"package_manager": "npm",
"min_coverage": "0",
"coverage_tool": "jest",
"coverage_summary_path": "coverage/coverage-summary.json",
"coverage_command": "mkdir -p coverage && echo '{\"total\":{\"lines\":{\"pct\":100}}}' > coverage/coverage-summary.json",
"requires": [
"build-and-test"
],
"filters": {
"branches": {
"only": [
"/release\\/.*/"
]
}
}
}
},
{
"munitor/node_security_scan": {
"name": "security-scan",
@@ -443,6 +470,7 @@
"health_db": "false",
"requires": [
"code-quality",
"coverage",
"secrets-scan",
"sast-scan",
"security-scan"
@@ -486,7 +514,7 @@
"name": "build-and-test",
"node_version": "22",
"package_manager": "npm",
"test_commands": "[\"#!/usr/bin/env bash\\nprintf '%s\\\\n' 'pipeline: node-webapp' 'node_version: 22' 'docker:' ' use_repo_dockerfile: true' > .munitor.yml\", \"npm run lint\", \"npm run build\", \"npx playwright install chromium --with-deps\", \"npm run test:chromium\", \"#!/usr/bin/env bash\\nset -euo pipefail\\nnpm run start > /tmp/qqq-link-server.log 2>&1 &\\nserver_pid=$!\\ntrap 'kill \\\"$server_pid\\\" 2>/dev/null || true; wait \\\"$server_pid\\\" 2>/dev/null || true' EXIT\\nready=false\\nfor _attempt in {1..60}; do\\n if ! kill -0 \\\"$server_pid\\\" 2>/dev/null; then\\n cat /tmp/qqq-link-server.log\\n exit 1\\n fi\\n if curl --fail --silent --output /dev/null --max-time 2 http://localhost:3005/; then\\n ready=true\\n break\\n fi\\n sleep 1\\ndone\\nif [ \\\"$ready\\\" != true ]; then\\n cat /tmp/qqq-link-server.log\\n echo 'Marketing server did not become ready on port 3005' >&2\\n exit 1\\nfi\\nnpm run check-links\", \"npm run lighthouse\"]",
"test_commands": "[\"npm run lint\", \"npm run build\", \"npx playwright install chromium --with-deps\", \"npm run test:chromium\", \"npm run lighthouse\"]",
"context": [
"github"
],
@@ -494,21 +522,7 @@
"branches": {
"only": "main"
}
},
"post-steps": [
{
"store_artifacts": {
"path": "playwright-report",
"destination": "playwright-report"
}
},
{
"store_artifacts": {
"path": ".lighthouseci",
"destination": "lighthouse-report"
}
}
]
}
}
},
{
@@ -1,35 +0,0 @@
{
"version": 2.1,
"jobs": {
"validate-policy": {
"docker": [
{
"image": "cimg/python:3.12"
}
],
"resource_class": "small",
"steps": [
"checkout",
{
"run": {
"name": "Install the YAML parser",
"command": "python3 -m pip install --disable-pip-version-check PyYAML==6.0.3"
}
},
{
"run": {
"name": "Validate the application-free branch",
"command": "python3 \\<<'PYTHON'\nimport json\nimport os\nfrom pathlib import Path\nimport subprocess\nimport xml.etree.ElementTree as ET\nimport yaml\n\npaths = subprocess.check_output(['git', 'ls-files', '-z']).decode('utf-8').split('\\0')\npaths = [Path(name) for name in paths if name]\nif not paths:\n raise SystemExit('No tracked policy files')\nnames = {'LICENSE', 'NOTICE', 'COPYING', 'CODEOWNERS', '.gitignore', '.gitattributes',\n '.editorconfig', 'mvnw', 'mvnw.cmd'}\nshell_checks = os.environ.get('QQQ_POLICY_SHELL') == 'true'\nsuffixes = {'.md', '.rst', '.txt', '.adoc', '.yml', '.yaml', '.json', '.properties'}\nmanifests = {'pom.xml', 'package.json', 'cargo.toml', 'pyproject.toml', 'go.mod',\n 'build.gradle', 'build.gradle.kts', 'settings.gradle', 'settings.gradle.kts'}\nfor path in paths:\n if path.is_symlink() or path.name.lower() in manifests:\n raise SystemExit(f'Application or symlink requires a build profile: {path}')\n if path.as_posix() == '.mvn/wrapper/maven-wrapper.jar':\n continue\n if path.as_posix() == 'checkstyle/config.xml':\n ET.fromstring(path.read_text(encoding='utf-8'))\n continue\n if shell_checks and path.suffix.lower() == '.sh':\n subprocess.run(['bash', '-n', '--', str(path)], check=True)\n subprocess.run(['shellcheck', '--severity=warning', '--', str(path)], check=True)\n continue\n if shell_checks and path.suffix.lower() == '.mmd':\n path.read_text(encoding='utf-8')\n continue\n if path.name not in names and path.suffix.lower() not in suffixes:\n raise SystemExit(f'Unsupported policy file; select a build profile: {path}')\n text = path.read_text(encoding='utf-8')\n if path.suffix.lower() in ('.yml', '.yaml'):\n list(yaml.safe_load_all(text))\n elif path.suffix.lower() == '.json':\n json.loads(text)\nprint(f'Validated {len(paths)} tracked policy/documentation files; no application build or publication')\nPYTHON\n"
}
}
]
}
},
"workflows": {
"verify": {
"jobs": [
"validate-policy"
]
}
}
}
File diff suppressed because it is too large. Load diff
+5 -17
View File
@@ -45,11 +45,7 @@ PROFILE_PARAMETERS = {
"app-starter": {},
"android": {},
"mcp-policy": {},
"repository-policy": {},
"marketing": {},
"node-artifact": {},
"homebrew-tap": {},
"application-template": {},
}
@@ -62,15 +58,13 @@ def load_yaml(raw):
raise StandardError("Invalid YAML declaration") from error
def canonical_caller(profile, version, explicit_manifest=False, release_tag_policy=None, publish_feature=False):
def canonical_caller(profile, version, explicit_manifest=False, release_tag_policy=None):
if not isinstance(profile, str) or profile not in PROFILE_PARAMETERS:
raise StandardError("Unsupported CI profile")
options = {
"orb_version": version,
"filters": {"branches": {"ignore": "automation/qbit-rc-pending"}, "tags": {"only": "/.*/"}},
}
if profile in ("node-artifact", "homebrew-tap", "application-template"):
options["filters"]["tags"] = {"ignore": "/.*/"}
if profile == "frontend-next":
options["filters"]["branches"] = {"ignore": "/.*/"}
if profile == "qbit" and release_tag_policy is not None:
@@ -78,9 +72,6 @@ def canonical_caller(profile, version, explicit_manifest=False, release_tag_poli
if release_tag_policy not in tag_filters:
raise StandardError("Unsupported QBit release tag policy")
options["filters"]["tags"] = tag_filters[release_tag_policy]
if publish_feature:
options["filters"]["tags"] = {"only": ([tag_filters[release_tag_policy]["only"]]
if release_tag_policy != "none" else []) + ["/publish.*/"]}
if explicit_manifest:
options["manifest"] = ".qqq-ci.yml"
parameters = PROFILE_PARAMETERS[profile]
@@ -114,15 +105,12 @@ def validate(config_raw, manifest_raw, setup_version):
if not isinstance(manifest, dict):
raise StandardError("Manifest must be a mapping")
version = manifest.get("orb_version")
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+(?:\.\d+)?|dev:[A-Za-z0-9_-]+)", version):
raise StandardError("Manifest must select a quoted minor track, exact stable pin, or development version")
if not isinstance(version, str) or not re.fullmatch(r"(?:\d+\.\d+\.\d+|dev:[A-Za-z0-9_-]+)", version):
raise StandardError("Manifest must pin an explicit orb version")
if version != setup_version:
raise StandardError("Manifest and running setup orb versions differ")
publishing = manifest.get("publish", {})
if not isinstance(publishing, dict) or type(publishing.get("feature", False)) is not bool:
raise StandardError("Feature publication must be an explicit boolean")
expected = canonical_caller(manifest.get("profile"), version, release_tag_policy=manifest.get("release_tag_policy"), publish_feature=publishing.get("feature", False))
explicit = canonical_caller(manifest.get("profile"), version, explicit_manifest=True, release_tag_policy=manifest.get("release_tag_policy"), publish_feature=publishing.get("feature", False))
expected = canonical_caller(manifest.get("profile"), version, release_tag_policy=manifest.get("release_tag_policy"))
explicit = canonical_caller(manifest.get("profile"), version, explicit_manifest=True, release_tag_policy=manifest.get("release_tag_policy"))
if not exact(config, expected) and not exact(config, explicit):
raise StandardError("Consumer config must be the canonical setup caller with only its profile's declared and forwarded parameters")
-23
View File
@@ -1,23 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
npm run start > /tmp/qqq-link-server.log 2>&1 &
server_pid=$!
trap 'kill "$server_pid" 2>/dev/null || true; wait "$server_pid" 2>/dev/null || true' EXIT
ready=false
for _attempt in {1..60}; do
if ! kill -0 "$server_pid" 2>/dev/null; then
cat /tmp/qqq-link-server.log
exit 1
fi
if curl --fail --silent --output /dev/null --max-time 2 http://localhost:3005/; then
ready=true
break
fi
sleep 1
done
if [ "$ready" != true ]; then
cat /tmp/qqq-link-server.log
echo 'Marketing server did not become ready on port 3005' >&2
exit 1
fi
npm run check-links
-2
View File
@@ -1,2 +0,0 @@
#!/usr/bin/env bash
printf '%s\n' 'pipeline: node-webapp' 'node_version: 22' 'docker:' ' use_repo_dockerfile: true' > .munitor.yml
-90
View File
@@ -1,90 +0,0 @@
version: 2.1
orbs:
qqq-orb: kingsrook/qqq-orb@0.6.8
commands:
configure-package-install:
steps:
- run:
name: Keep package installation unattended
command: |
# These build machines are discarded after the job.
sudo install -d /etc/needrestart/conf.d
printf '%s\n' '$nrconf{restart} = "l";' | sudo tee /etc/needrestart/conf.d/99-qqq-ci.conf >/dev/null
workflows:
test_only:
jobs:
- qqq-orb/mvn_test_only:
pre-steps:
- configure-package-install
context: [qqq-maven-registry-credentials]
filters:
branches:
ignore: /(develop|main|release/.*|hotfix/.*|integration.*)/
tags:
only: []
publish_snapshot:
jobs:
- qqq-orb/mvn_publish:
pre-steps:
- configure-package-install
context: [qqq-maven-registry-credentials]
branch_type: snapshot
filters:
branches:
only:
- develop
publish_feature:
jobs:
- qqq-orb/mvn_publish:
pre-steps:
- configure-package-install
context: [qqq-maven-registry-credentials]
branch_type: feature
filters:
branches:
only:
- feature/.*
tags:
only: [/publish.*/]
publish_release_candidate:
jobs:
- qqq-orb/mvn_publish:
pre-steps:
- configure-package-install
context: [qqq-maven-registry-credentials]
branch_type: release_candidate
filters:
branches:
only:
- /release\/.*/
publish_release:
jobs:
- qqq-orb/mvn_publish:
pre-steps:
- configure-package-install
context: [qqq-maven-registry-credentials]
branch_type: release
filters:
branches:
ignore: /.*/
tags:
only: [/^v\d+\.\d+\.\d+$/]
publish_hotfix_release:
jobs:
- qqq-orb/mvn_publish:
pre-steps:
- configure-package-install
context: [qqq-maven-registry-credentials]
branch_type: hotfix
filters:
branches:
only:
- /hotfix\/.*/
File diff suppressed because it is too large. Load diff
+5 -257
View File
@@ -1,6 +1,4 @@
import importlib.util
import json
import io
import pathlib
import unittest
from unittest.mock import patch
@@ -27,197 +25,7 @@ class Snapshot:
return self.required
class InventoryTests(unittest.TestCase):
def repo(self, name, archived=False):
return {'name': name, 'full_name': 'QRun-IO/' + name,
'archived': archived, 'default_branch': 'main'}
def test_all_pages_include_new_private_and_archived_repositories(self):
new = dict(self.repo('new-product'), private=True)
pages = [[self.repo('qbit-existing')], [new, self.repo('retired', True)]]
with patch.object(audit, 'command', return_value=json.dumps(pages)) as command:
entries = audit.discover_inventory('QRun-IO')
self.assertEqual([e['name'] for e in entries], ['new-product', 'qbit-existing', 'retired'])
self.assertTrue(entries[-1]['isArchived'])
self.assertEqual(command.call_args.args[0], ['gh', 'api', '--method', 'GET',
'--paginate', '--slurp', 'orgs/QRun-IO/repos?type=all&per_page=100'])
def test_empty_duplicate_foreign_and_malformed_inventories_fail_closed(self):
repo = self.repo('qbit-example')
for pages in ([], [[]], {}, [repo], [[repo], [repo]],
[[dict(repo, full_name='Other/qbit-example')]],
[[dict(repo, archived='false')]], [[dict(repo, default_branch=None)]]):
with self.subTest(pages=pages), patch.object(audit, 'command', return_value=json.dumps(pages)):
with self.assertRaises(audit.ReadError):
audit.discover_inventory('QRun-IO')
with patch.object(audit, 'command') as command:
with self.assertRaises(audit.ReadError):
audit.discover_inventory('QRun-IO/../../other')
command.assert_not_called()
def test_live_cli_count_is_dynamic_but_explicit_expectation_is_enforced(self):
for expected, status in ((None, 0), ('1', 0), ('68', 1)):
args = ['audit', '--organization', 'QRun-IO', '--live', '--orb-version', '0.8']
if expected:
args += ['--expected-active', expected]
report = {'active_count': 1, 'unique_repository_count': 1,
'inventory_count': 1, 'results': [{'status': 'pass'}]}
output = io.StringIO()
with patch.object(audit.sys, 'argv', args), patch.object(audit.sys, 'stdout', output), \
patch.object(audit, 'discover_inventory', return_value=[ENTRY]), \
patch.object(audit, 'run', return_value=report):
self.assertEqual(audit.main(), status)
self.assertEqual(json.loads(output.getvalue())['inventory_source'], 'organization-api')
def test_discovery_api_failure_never_falls_back_to_old_inventory(self):
with patch.object(audit, 'command', side_effect=audit.ReadError('unavailable')):
with self.assertRaises(audit.ReadError):
audit.discover_inventory('QRun-IO')
class AuditTests(unittest.TestCase):
def test_all_scanner_and_release_requirements_follow_branch_profile(self):
entry = dict(ENTRY, name='qqq-all', nameWithOwner='QRun-IO/qqq-all')
scanner = '.github/workflows/codeql.yml'
release = audit.auxiliary.BRIDGES['all-release']['path']
for profile in ('all', 'repository-policy'):
files = {'.qqq-ci.yml': audit.generator.yaml.safe_dump({'profile': profile, 'orb_version': VERSION, 'default_branch': 'develop'}),
'.circleci/config.yml': audit.generator.yaml.safe_dump(audit.standard.canonical_caller(profile, VERSION))}
policy = dict(POLICY, profile=profile, codeql_worker_sha=SHA)
report = audit.audit(entry, Snapshot(files), VERSION, SHA, policy)
missing = {f.get('path') for f in report['findings'] if f['code'] == 'missing_native_maintenance_bridge'}
self.assertIn(scanner, missing)
self.assertEqual(profile == 'all', release in missing)
files[scanner] = audit.generator.yaml.safe_dump(audit.auxiliary.canonical('qbit-codeql', SHA), sort_keys=False)
if profile == 'all':
files[release] = audit.generator.yaml.safe_dump(audit.auxiliary.canonical('all-release', SHA), sort_keys=False)
self.assertEqual([], audit.audit(entry, Snapshot(files), VERSION, SHA, policy)['findings'])
files[scanner] = files[scanner].replace(SHA, 'b' * 40)
self.assertIn('native_maintenance_bridge_drift', [f['code'] for f in audit.audit(entry, Snapshot(files), VERSION, SHA, policy)['findings']])
files[release] = 'jobs: {}'
self.assertTrue(any(f['code'] == 'native_maintenance_bridge_drift' and f.get('path') == release for f in audit.audit(entry, Snapshot(files), VERSION, SHA, policy)['findings']))
def test_schema_drift_bridge_tracks_branch_source_without_exempting_default(self):
entry = dict(ENTRY, name='qbit-authentik', nameWithOwner='QRun-IO/qbit-authentik')
path = '.github/workflows/schema-drift.yml'
cases = [
('main', {}, False),
('main', {'src/test/java/example/SchemaDriftTest.java': 'class SchemaDriftTest {}'}, True),
('develop', {}, True),
]
for branch, files, required in cases:
with self.subTest(branch=branch, files=files):
snapshot = Snapshot(files)
snapshot.branch, snapshot.default_branch = branch, 'develop'
report = audit.audit(entry, snapshot, VERSION, SHA, POLICY)
missing = any(f['code'] == 'missing_native_maintenance_bridge' and f.get('path') == path for f in report['findings'])
self.assertEqual(required, missing)
snapshot = Snapshot({path: 'jobs: {}'})
snapshot.branch, snapshot.default_branch = 'main', 'develop'
report = audit.audit(entry, snapshot, VERSION, SHA, POLICY)
self.assertIn('native_maintenance_bridge_drift', [f['code'] for f in report['findings']])
def test_placeholder_can_retain_canonical_release_and_scanner_bridges(self):
entry = dict(ENTRY, name='qbit-help', nameWithOwner='QRun-IO/qbit-help')
files = {'.qqq-ci.yml': audit.generator.yaml.safe_dump({'profile': 'repository-policy', 'orb_version': VERSION, 'default_branch': 'develop'}),
'.circleci/config.yml': audit.generator.yaml.safe_dump(audit.standard.canonical_caller('repository-policy', VERSION)),
audit.rc_worker.CALLER_PATH: audit.rc_worker.generate(SHA),
'.github/workflows/codeql.yml': audit.generator.yaml.safe_dump(audit.auxiliary.canonical('qbit-codeql', SHA), sort_keys=False)}
snapshot = Snapshot(files)
snapshot.branch = 'main'
snapshot.default_branch = 'develop'
policy = dict(POLICY, branch_profiles={'main': 'repository-policy'}, rc_worker_sha=SHA, codeql_worker_sha=SHA)
self.assertEqual([], audit.audit(entry, snapshot, VERSION, SHA, policy)['findings'])
files['.github/workflows/codeql.yml'] = files['.github/workflows/codeql.yml'].replace(SHA, 'b' * 40)
self.assertTrue(any(f['code'] == 'native_maintenance_bridge_drift' for f in audit.audit(entry, snapshot, VERSION, SHA, policy)['findings']))
def test_placeholder_profile_is_scoped_to_selected_branch(self):
entry = dict(ENTRY, name='qbit-help', nameWithOwner='QRun-IO/qbit-help')
files = {'.qqq-ci.yml': audit.generator.yaml.safe_dump({'profile': 'repository-policy', 'orb_version': VERSION, 'default_branch': 'develop'}),
'.circleci/config.yml': audit.generator.yaml.safe_dump(audit.standard.canonical_caller('repository-policy', VERSION))}
snapshot = Snapshot(files)
snapshot.branch = 'main'
snapshot.default_branch = 'develop'
policy = dict(POLICY, branch_profiles={'main': 'repository-policy'})
self.assertEqual([], audit.audit(entry, snapshot, VERSION, SHA, policy)['findings'])
snapshot.branch = 'develop'
report = audit.audit(entry, snapshot, VERSION, SHA, policy)
self.assertEqual('qbit', report['profile'])
self.assertTrue(any(f['code'] == 'profile_mismatch' for f in report['findings']))
def test_qbit_scanner_policy_requires_exact_canonical_worker(self):
entry = dict(ENTRY, name='qbit-help', nameWithOwner='QRun-IO/qbit-help')
manifest = {'profile': 'qbit', 'orb_version': VERSION, 'default_branch': 'develop',
'publish': {'snapshot': False, 'release_candidate': False, 'release': False}}
files = {'.qqq-ci.yml': audit.generator.yaml.safe_dump(manifest),
'.circleci/config.yml': audit.generator.yaml.safe_dump(audit.standard.canonical_caller('qbit', VERSION))}
policy = dict(POLICY, codeql_worker_sha='b' * 40)
report = audit.audit(entry, Snapshot(files), VERSION, SHA, policy)
self.assertTrue(any(f['code'] == 'missing_native_maintenance_bridge' for f in report['findings']))
files['.github/workflows/codeql.yml'] = audit.generator.yaml.safe_dump(audit.auxiliary.canonical('qbit-codeql', 'b' * 40), sort_keys=False)
self.assertEqual([], audit.audit(entry, Snapshot(files), VERSION, SHA, policy)['findings'])
files['.github/workflows/codeql.yml'] = files['.github/workflows/codeql.yml'].replace('b' * 40, 'a' * 40)
report = audit.audit(entry, Snapshot(files), VERSION, SHA, policy)
self.assertTrue(any(f['code'] == 'native_maintenance_bridge_drift' for f in report['findings']))
def test_live_snapshot_reads_requested_branch_and_keeps_default_identity(self):
entry = dict(ENTRY, sourceBranch='release/1.2')
replies = [{'default_branch': 'develop', 'archived': False}, {'sha': SHA},
{'truncated': False, 'tree': []}]
with patch.object(audit, 'gh', side_effect=replies) as gh:
snapshot = audit.Snapshot(entry, live=True)
self.assertEqual('develop', snapshot.default_branch)
self.assertEqual('release/1.2', snapshot.branch)
self.assertEqual('repos/QRun-IO/qctl/commits/release%2F1.2', gh.call_args_list[1].args[0])
def test_other_source_branch_does_not_change_manifest_default_branch(self):
entry = dict(ENTRY, name='qbit-help', nameWithOwner='QRun-IO/qbit-help')
manifest = {'profile': 'qbit', 'orb_version': VERSION, 'default_branch': 'develop',
'publish': {'snapshot': False, 'release_candidate': False, 'release': False}}
files = {'.qqq-ci.yml': audit.generator.yaml.safe_dump(manifest),
'.circleci/config.yml': audit.generator.yaml.safe_dump(audit.standard.canonical_caller('qbit', VERSION))}
snapshot = Snapshot(files)
snapshot.branch = 'main'
snapshot.default_branch = 'develop'
report = audit.audit(entry, snapshot, VERSION, SHA, POLICY)
self.assertEqual([], report['findings'])
self.assertEqual('main', report['source_branch'])
self.assertEqual('develop', report['default_branch'])
def test_selected_branch_read_failure_is_unknown_not_default_fallback(self):
captured = []
def factory(entry, live):
captured.append(entry)
raise audit.ReadError('branch unavailable')
report = audit.run([ENTRY], VERSION, SHA, {}, live=True,
snapshot_factory=factory, source_branch='main')
self.assertEqual('main', captured[0]['sourceBranch'])
self.assertEqual('unknown', report['results'][0]['status'])
self.assertEqual('main', report['results'][0]['source_branch'])
self.assertNotIn('sourceBranch', ENTRY)
def test_omitted_source_branch_still_reads_repository_default(self):
replies = [{'default_branch': 'develop', 'archived': False}, {'sha': SHA},
{'truncated': False, 'tree': []}]
with patch.object(audit, 'gh', side_effect=replies) as gh:
snapshot = audit.Snapshot(ENTRY, live=True)
self.assertEqual('develop', snapshot.branch)
self.assertEqual('develop', snapshot.default_branch)
self.assertEqual('repos/QRun-IO/qctl/commits/develop', gh.call_args_list[1].args[0])
def test_explicit_source_branch_cannot_relabel_offline_snapshot(self):
with self.assertRaises(ValueError):
audit.run([ENTRY], VERSION, SHA, {}, source_branch='main')
with self.assertRaises(audit.ReadError):
audit.Snapshot(dict(ENTRY, sourceBranch='main'), live=False)
def test_invalid_source_branch_is_rejected_before_api_calls(self):
for branch in ('', 'main?ref=develop', 'main\n', 123):
with self.subTest(branch=branch), patch.object(audit, 'gh') as gh:
with self.assertRaises(audit.ReadError):
audit.Snapshot(dict(ENTRY, sourceBranch=branch), live=True)
gh.assert_not_called()
def native(self, files=None, checks=None, policy=POLICY):
files = files if files is not None else {'.github/workflows/build.yml': audit.qctl.generate(SHA)}
return audit.audit(ENTRY, Snapshot(files, checks), VERSION, SHA, policy)
@@ -266,25 +74,6 @@ class AuditTests(unittest.TestCase):
report = audit.audit(entry, Snapshot({'.qqq-ci.yml': audit.generator.yaml.safe_dump(manifest)}), VERSION, SHA, POLICY)
self.assertIn('missing_rc_bridge', [f['code'] for f in report['findings']])
def test_rc_pin_policy_does_not_force_unchanged_auxiliary_worker_updates(self):
entry = dict(ENTRY, name='qbit-authentik', nameWithOwner='QRun-IO/qbit-authentik')
rc_sha = 'b' * 40
definition = audit.auxiliary.BRIDGES['qbit-schema-drift']
manifest = {'profile': 'qbit', 'orb_version': VERSION, 'default_branch': 'develop', 'release_completion': True}
files = {audit.rc_worker.CALLER_PATH: audit.rc_worker.generate(rc_sha),
definition['path']: audit.generator.yaml.safe_dump(audit.auxiliary.canonical('qbit-schema-drift', SHA), sort_keys=False),
'.qqq-ci.yml': audit.generator.yaml.safe_dump(manifest),
'.circleci/config.yml': audit.generator.yaml.safe_dump(audit.standard.canonical_caller('qbit', VERSION))}
policy = dict(POLICY, rc_worker_sha=rc_sha)
self.assertEqual('pass', audit.audit(entry, Snapshot(files), VERSION, SHA, policy)['status'])
for pin in (SHA, 'main', ''):
with self.subTest(pin=pin):
report = audit.audit(entry, Snapshot(files), VERSION, SHA, dict(POLICY, rc_worker_sha=pin))
self.assertTrue(report['findings'])
files[definition['path']] = files[definition['path']].replace(SHA, rc_sha)
report = audit.audit(entry, Snapshot(files), VERSION, SHA, policy)
self.assertIn('native_maintenance_bridge_drift', [f['code'] for f in report['findings']])
def test_qbit_rc_bridge_missing_drift_pin_and_duplicate(self):
entry = dict(ENTRY, name='qbit-test', nameWithOwner='QRun-IO/qbit-test')
path = audit.rc_worker.CALLER_PATH
@@ -324,32 +113,13 @@ class AuditTests(unittest.TestCase):
self.assertIn('native_maintenance_bridge_drift', [f['code'] for f in report['findings']])
def test_new_circle_profile_mappings_require_actual_configuration(self):
for name, profile in [('qqq-android', 'android'), ('qqq-mcp-rag-standard-policies', 'mcp-policy'), ('marketing-website', 'marketing'), ('qqq-templates-hub', 'repository-policy'), ('qqq-meta-manager', 'repository-policy'), ('homebrew-qctl', 'homebrew-tap'), ('new-qqq-application-template', 'application-template')]:
for name, profile in [('qqq-android', 'android'), ('qqq-mcp-rag-standard-policies', 'mcp-policy'), ('marketing-website', 'marketing')]:
entry = dict(ENTRY, name=name, nameWithOwner='QRun-IO/' + name)
report = audit.audit(entry, Snapshot({}), VERSION, SHA, {})
self.assertEqual(profile, report['profile'])
self.assertIn('missing_manifest', [f['code'] for f in report['findings']])
self.assertIn('missing_canonical_caller', [f['code'] for f in report['findings']])
def test_catalog_requires_canonical_policy_pipeline(self):
self.assertNotIn('QRun-IO/qqq-templates-hub', audit.NO_CI)
entry = dict(ENTRY, name='qqq-templates-hub', nameWithOwner='QRun-IO/qqq-templates-hub')
files = {'.qqq-ci.yml': audit.generator.yaml.safe_dump({
'profile': 'repository-policy', 'orb_version': VERSION, 'default_branch': 'main'}),
'.circleci/config.yml': audit.generator.yaml.safe_dump(
audit.standard.canonical_caller('repository-policy', VERSION)),
'templates.yaml': 'version: 1.0'}
policy = {'managed_branches': ['main'], 'required_checks': {'main': ['setup', 'verify']}}
snapshot = Snapshot(files, ['setup', 'verify'])
snapshot.branch = 'main'
report = audit.audit(entry, snapshot, VERSION, SHA, policy)
self.assertEqual('repository-policy', report['profile'])
self.assertEqual([], report['findings'])
self.assertEqual('pass', report['status'])
files.pop('.circleci/config.yml')
report = audit.audit(entry, snapshot, VERSION, SHA, policy)
self.assertIn('missing_canonical_caller', [f['code'] for f in report['findings']])
def test_intellij_is_native_product_not_no_ci_exception(self):
repository = 'QRun-IO/qqq-app-developer-intellij-plugin'
self.assertNotIn(repository, audit.NO_CI)
@@ -359,8 +129,7 @@ class AuditTests(unittest.TestCase):
self.assertIn('missing_native_maintenance_bridge', [f['code'] for f in report['findings']])
def test_approved_no_ci_is_exact_allowlist_with_evidence_not_check_pass(self):
self.assertEqual(3, len(audit.NO_CI))
self.assertNotIn("QRun-IO/voyage-cd", audit.NO_CI)
self.assertEqual(8, len(audit.NO_CI))
for repository, policy in audit.NO_CI.items():
entry = dict(ENTRY, name=repository.split('/')[-1], nameWithOwner=repository)
files = {path: 'identity fixture' for path in policy['required_paths']}
@@ -382,13 +151,13 @@ class AuditTests(unittest.TestCase):
self.assertIn('profile_not_assigned', [f['code'] for f in report['findings']])
def test_native_specialists_require_bridges_and_report_remaining_workflows(self):
for name, kind in [('qqq-mcp-rag', 'mcp-rag'), ('qqq-business-platform', 'business-db'), ('.github', 'github-workflow-lint'), ('marketing-website-cd', 'gitops'), ('voyage-cd', 'gitops')]:
for name, kind in [('qqq-mcp-rag', 'mcp-rag'), ('qqq-business-platform', 'business-db'), ('.github', 'github-workflow-lint'), ('marketing-website-cd', 'gitops')]:
entry = dict(ENTRY, name=name, nameWithOwner='QRun-IO/' + name)
definition = audit.auxiliary.BRIDGES[kind]
missing = audit.audit(entry, Snapshot({}), VERSION, SHA, {})
self.assertIn('missing_native_maintenance_bridge', [f['code'] for f in missing['findings']])
files = {item['path']: audit.generator.yaml.safe_dump(audit.auxiliary.canonical(key, SHA), sort_keys=False)
for key, item in audit.auxiliary.BRIDGES.items() if item['repository'] == entry['nameWithOwner'] or entry['nameWithOwner'] in item.get('additional_repositories', [])}
for key, item in audit.auxiliary.BRIDGES.items() if item['repository'] == entry['nameWithOwner']}
report = audit.audit(entry, Snapshot(files), VERSION, SHA, {})
self.assertEqual([], report['findings'])
self.assertEqual('unknown', report['status'])
@@ -457,31 +226,10 @@ class AuditTests(unittest.TestCase):
audit.audit(other, Snapshot({}, checks=[]), VERSION, SHA, policy)
def test_invalid_managed_branches_cannot_bypass_policy(self):
for managed in [[], ['main', 'main'], ['feature'], 'main', None, [True],
['release/*'], ['release/1.2/extra'], ['hotfix/1..2'],
['release/1.2', 'release/1.2']]:
for managed in [[], ['main', 'main'], ['feature'], 'main', None, [True]]:
with self.subTest(managed=managed), self.assertRaises(audit.ReadError):
self.native(policy={'managed_branches': managed, 'required_checks': {}})
def test_release_and_hotfix_enforcement_requires_actual_checks(self):
files = {'.github/workflows/build.yml': audit.qctl.generate(SHA)}
for branch in ['release/0.1', 'release/4.1.0', 'hotfix/4.0.1']:
with self.subTest(branch=branch):
snapshot = Snapshot(files)
snapshot.branch = branch
policy = {'managed_branches': [branch],
'required_checks': {branch: ['native / test']}}
with patch.object(snapshot, 'checks', wraps=snapshot.checks) as read:
report = audit.audit(ENTRY, snapshot, VERSION, SHA, policy)
self.assertEqual('pass', report['status'])
read.assert_called_once_with(branch)
snapshot.required = []
report = audit.audit(ENTRY, snapshot, VERSION, SHA, policy)
self.assertEqual('missing', report['required_checks'][branch]['state'])
self.assertEqual('findings', report['status'])
policy['required_checks'] = {}
self.assertEqual('unknown', audit.audit(ENTRY, snapshot, VERSION, SHA, policy)['status'])
def test_gh_is_explicit_get_and_never_changes_settings(self):
with patch.object(audit, 'command', return_value='{}') as command:
audit.gh('repos/QRun-IO/example')
-69
View File
@@ -1,69 +0,0 @@
#!/bin/bash
# Exercise real browser scripts without sudo, package installation or network.
set -euo pipefail
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
scratch=$(mktemp -d)
trap 'rm -rf "$scratch"' EXIT
mkdir "$scratch/bin"
export INSTALL_LOG="$scratch/install.log"
cat > "$scratch/bin/sudo" <<'STUB'
#!/bin/bash
printf '%s\n' "$*" >> "$INSTALL_LOG"
if [[ "$*" == *'apt-get update'* ]]; then
[[ "$*" == *'Dir::Etc::sourceparts=-'* ]] || { echo 'unrelated image PPA: 503' >&2; exit 100; }
[[ "$*" == *'APT::Get::List-Cleanup=0'* ]] || exit 91
[[ ${REQUIRED_REPO_FAIL:-0} == 0 ]] || exit 100
fi
if [[ "$1" == tee ]]; then cat >> "$INSTALL_LOG"; fi
if [[ "$1" == gpg ]]; then cat >/dev/null; fi
STUB
chmod +x "$scratch/bin/sudo"
export PATH="$scratch/bin:$PATH"
bash "$root/src/scripts/frontend_test_dependencies.sh"
echo 'PASS: browser dependencies ignore unavailable image PPAs'
# Model Chrome presence without depending on the host's installed browsers.
export CHROME_MARKER="$scratch/chrome-installed"
export BASH_ENV="$scratch/shell-env"
cat > "$BASH_ENV" <<'STUB'
command() {
if [[ "$*" == '-v google-chrome' ]]; then [[ -f "$CHROME_MARKER" ]]; else builtin command "$@"; fi
}
google-chrome() { [[ -f "$CHROME_MARKER" ]] && echo 'Google Chrome test'; }
STUB
cat > "$scratch/bin/curl" <<'STUB'
#!/bin/bash
printf '%s\n' "$*" >> "$INSTALL_LOG"
[[ ${KEY_FAIL:-0} == 0 ]] || exit 22
printf 'signing-key\n'
STUB
cat > "$scratch/bin/dpkg" <<'STUB'
#!/bin/bash
echo amd64
STUB
# Successful installation makes Chrome available to the final version check.
# shellcheck disable=SC2016
sed -i '/if \[\[ "$1" == tee/i if [[ "$*" == *"apt-get install -y google-chrome-stable"* ]]; then touch "$CHROME_MARKER"; fi' "$scratch/bin/sudo"
chmod +x "$scratch/bin/"*
: > "$INSTALL_LOG"
bash "$root/src/scripts/install_chrome.sh" >/dev/null
grep -q 'Dir::Etc::sourcelist=/etc/apt/sources.list.d/qqq-google-chrome.list' "$INSTALL_LOG"
grep -q 'gpg --batch --yes --dearmor -o /usr/share/keyrings/qqq-google-chrome.gpg' "$INSTALL_LOG"
grep -q 'signed-by=/usr/share/keyrings/qqq-google-chrome.gpg] https://dl.google.com/linux/chrome/deb/' "$INSTALL_LOG"
grep -q 'apt-get install -y google-chrome-stable' "$INSTALL_LOG"
: > "$INSTALL_LOG"
bash "$root/src/scripts/install_chrome.sh" >/dev/null
[[ ! -s "$INSTALL_LOG" ]] # Existing Chrome needs no network/package work.
rm "$CHROME_MARKER"
for script in install_chrome frontend_test_dependencies; do
: > "$INSTALL_LOG"
if REQUIRED_REPO_FAIL=1 bash "$root/src/scripts/$script.sh" >/dev/null 2>&1; then
echo "Required repository outage accepted: $script" >&2; exit 1
fi
if grep -q 'apt-get install' "$INSTALL_LOG"; then exit 1; fi
done
: > "$INSTALL_LOG"
if KEY_FAIL=1 bash "$root/src/scripts/install_chrome.sh" >/dev/null 2>&1; then
echo 'Signing key failure accepted' >&2; exit 1
fi
if grep -q 'apt-get' "$INSTALL_LOG"; then exit 1; fi
echo 'PASS: Chrome ignores unrelated PPAs; required repositories and signing key fail closed; existing browser reused'
-109
View File
@@ -1,109 +0,0 @@
"""CI maintenance retains Maven verification and never becomes a release request."""
import importlib.util
from pathlib import Path
import unittest
from unittest import mock
import yaml
ROOT = Path(__file__).resolve().parents[1]
spec = importlib.util.spec_from_file_location('pipeline', ROOT/'src/scripts/generate_pipeline.py')
pipeline = importlib.util.module_from_spec(spec)
spec.loader.exec_module(pipeline)
class MaintenanceTests(unittest.TestCase):
def git_result(self, *args):
return {'log': self.subject, 'rev-list': self.parents, 'diff': self.paths}.get(args[0])
def setUp(self):
self.subject = b'ci: use central pipeline\n'
self.parents = b'a'*40 + b' ' + b'b'*40 + b'\n'
self.paths = b'.qqq-ci.yml\0.circleci/config.yml\0.github/workflows/codeql.yml\0.github/workflows/release-rc.yml\0'
def test_publishing_branches_keep_unfiltered_maven_tests(self):
manifest = {'profile': 'qbit', 'orb_version': '0.7.14', 'default_branch': 'develop',
'release_on_main': True, 'test_branch_policy': 'feature-no-hotfix',
'publish': {'hotfix': True}}
with mock.patch.object(Path, 'read_text', return_value=yaml.safe_dump(manifest)):
config = pipeline.load_manifest('unused', '0.7.14')
with mock.patch.object(pipeline, 'git', side_effect=self.git_result):
for branch in ('main', 'develop', 'release/0.1', 'hotfix/1.2.3'):
with self.subTest(branch=branch):
result = pipeline.qbit_template(config, branch, '')
job = result['workflows']['test_only']['jobs'][0]['qqq-orb/mvn_test_only']
self.assertEqual(job['filters']['branches'], {'ignore': 'automation/qbit-rc-pending'})
self.assertNotIn('branch_type', job)
def test_nonpublishing_branches_verify_maintenance_but_keep_application_policy(self):
manifest = {'profile': 'qbit', 'orb_version': '0.8', 'default_branch': 'main',
'release_on_main': False, 'test_branch_policy': 'feature-no-hotfix',
'publish': {'snapshot': False, 'release_candidate': False, 'hotfix': False}}
with mock.patch.object(Path, 'read_text', return_value=yaml.safe_dump(manifest)):
config = pipeline.load_manifest('unused', '0.8')
maintenance_paths = self.paths
with mock.patch.object(pipeline, 'git', side_effect=self.git_result):
for branch in ('main', 'develop', 'release/0.1', 'hotfix/1.2.3'):
for maintenance in (True, False):
with self.subTest(branch=branch, maintenance=maintenance):
self.paths = maintenance_paths if maintenance else b'src/Main.java\0'
result = pipeline.qbit_template(config, branch, '')
job = result['workflows']['test_only']['jobs'][0]['qqq-orb/mvn_test_only']
ignored = job['filters']['branches']['ignore']
if maintenance:
self.assertEqual(ignored, 'automation/qbit-rc-pending')
else:
self.assertEqual(ignored, ['automation/qbit-rc-pending',
'/(develop|main|release/.*|hotfix/.*|integration.*)/'])
self.assertNotIn('branch_type', job)
def test_frontend_maintenance_verifies_without_publishing(self):
for profile, prefix in (('frontend-core', 'node_'), ('frontend-material', 'mvn_frontend_')):
manifest = {'profile': profile, 'orb_version': '0.8', 'default_branch': 'develop'}
with mock.patch.object(Path, 'read_text', return_value=yaml.safe_dump(manifest)):
config = pipeline.load_manifest('unused', '0.8')
for branch in ('main', 'develop', 'release/1.0', 'hotfix/1.0.1'):
with self.subTest(profile=profile, branch=branch), mock.patch.object(pipeline, 'git', side_effect=self.git_result):
result = pipeline.frontend_template(config, branch, '')
job = result['workflows']['test_only']['jobs'][0]['qqq-orb/' + prefix + 'test_only']
self.assertNotIn('branch_type', job)
self.assertEqual(job['filters']['branches'], {'ignore': 'automation/qbit-rc-pending'})
self.paths = b'package.json\0'
with mock.patch.object(pipeline, 'git', side_effect=self.git_result):
result = pipeline.frontend_template(config, 'develop', '')
self.assertIn('qqq-orb/' + prefix + 'publish', result['workflows']['publish_snapshot']['jobs'][0])
self.paths = b'.qqq-ci.yml\0'
with mock.patch.object(pipeline, 'git', return_value=b'verified-head\n'):
result = pipeline.frontend_template(config, 'main', 'v1.2.3')
job = result['workflows']['publish_release']['jobs'][0]['qqq-orb/' + prefix + 'publish']
self.assertEqual(job['branch_type'], 'release')
def test_application_changes_and_unknown_files_publish_normally(self):
for paths in (b'pom.xml\0', b'.qqq-ci.yml\0src/Main.java\0', b'.github/workflows/custom-release.yml\0', b'', None):
self.paths = paths
with mock.patch.object(pipeline, 'git', side_effect=self.git_result):
self.assertFalse(pipeline.proven_ci_maintenance('release/0.1', ''))
def test_schema_caller_migration_is_maintenance_but_schema_inputs_are_not(self):
self.paths = b'.github/workflows/schema-drift.yml\0'
with mock.patch.object(pipeline, 'git', side_effect=self.git_result):
self.assertTrue(pipeline.proven_ci_maintenance('release/0.1', ''))
for path in (b'pom.xml\0', b'src/test/resources/schema.yml\0', b'src/test/java/SchemaDriftTest.java\0'):
self.paths = b'.github/workflows/schema-drift.yml\0' + path
self.assertFalse(pipeline.proven_ci_maintenance('release/0.1', ''))
def test_merge_root_and_unavailable_history_do_not_suppress_release(self):
for parents in (None, b'a'*40+b'\n', self.parents.rstrip()+b' '+b'c'*40+b'\n'):
self.parents = parents
with mock.patch.object(pipeline, 'git', side_effect=self.git_result):
self.assertFalse(pipeline.proven_ci_maintenance('release/0.1', ''))
def test_tags_and_release_requests_do_not_suppress_release(self):
with mock.patch.object(pipeline, 'git', side_effect=self.git_result):
self.assertFalse(pipeline.proven_ci_maintenance('main', 'v1.2.3'))
for subject in (b'Merge develop for next RC\n', b'fix: application\n', None):
self.subject = subject
self.assertFalse(pipeline.proven_ci_maintenance('release/0.1', ''))
if __name__ == '__main__':
unittest.main()
-105
View File
@@ -1,105 +0,0 @@
"""Cost settings preserve source qualification and unrelated project controls."""
from pathlib import Path
import sys
import unittest
from unittest.mock import Mock
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'tools'))
import circle_project_settings as settings
class SettingsTests(unittest.TestCase):
def setUp(self):
self.plan = {'repository': 'QRun-IO/marketing-website', 'profile': 'marketing',
'branch': 'main', 'head': 'a' * 40, 'changed': False}
self.before = {'autocancel-builds': False, 'setup-workflows': True,
'build-fork-prs': False, 'drop-all-build-requests': False}
self.source = Mock(side_effect=[{'archived': False, 'default_branch': 'main'},
{'commit': {'sha': self.plan['head']}}])
def test_narrow_update_and_idempotent_readback(self):
request = Mock(side_effect=[{'feature_flags': self.before}, {},
{'feature_flags': {**self.before, 'autocancel-builds': True}}])
result = settings.reconcile(self.plan, self.source, True, request)
self.assertEqual({'state': 'enforced', 'changed': True}, result)
self.assertEqual({'feature_flags': {'autocancel-builds': True}}, request.call_args_list[1].args[1])
request = Mock(return_value={'feature_flags': {**self.before, 'autocancel-builds': True}})
source = Mock(return_value={'archived': False, 'default_branch': 'main'})
self.assertEqual({'state': 'enforced', 'changed': False},
settings.reconcile(self.plan, source, True, request))
self.assertEqual(1, request.call_count)
def test_dry_run_never_writes(self):
request = Mock(return_value={'feature_flags': self.before})
self.assertEqual({'state': 'inspected', 'changed': True},
settings.reconcile(self.plan, self.source, False, request))
request.assert_called_once_with(self.plan['repository'])
def test_unreviewed_profiles_and_unconverted_sources_never_call_api(self):
for change, expected in [({'profile': 'qbit'}, 'not-managed'),
({'profile': 'native-gitops'}, 'not-managed'),
({'changed': True}, 'waiting-source')]:
source, request = Mock(), Mock()
result = settings.reconcile({**self.plan, **change}, source, True, request)
self.assertEqual(expected, result['state'])
source.assert_not_called()
request.assert_not_called()
def test_changed_head_or_unexpected_default_branch_prevents_write(self):
for responses in [
[{'archived': False, 'default_branch': 'main'}, {'commit': {'sha': 'b' * 40}}],
[{'archived': False, 'default_branch': 'develop'}],
[{'archived': True, 'default_branch': 'main'}],
]:
request = Mock(return_value={'feature_flags': self.before})
result = settings.reconcile(self.plan, Mock(side_effect=responses), True, request)
self.assertEqual('unresolved', result['state'])
self.assertTrue(all(len(call.args) == 1 for call in request.call_args_list))
def test_unrelated_setting_drift_and_unavailable_api_are_not_success(self):
request = Mock(side_effect=[{'feature_flags': self.before}, {},
{'feature_flags': {**self.before, 'autocancel-builds': True,
'build-fork-prs': True}}])
self.assertEqual('unresolved', settings.reconcile(self.plan, self.source, True, request)['state'])
source = Mock(return_value={'archived': False, 'default_branch': 'main'})
for response in [Mock(side_effect=ValueError('Central CircleCI credential unavailable')),
Mock(return_value={'feature_flags': {}})]:
self.assertEqual('unresolved', settings.reconcile(self.plan, source, True, response)['state'])
def test_repository_policy_requires_every_current_branch(self):
main = {**self.plan, 'profile': 'repository-policy'}
develop = {**main, 'branch': 'develop', 'profile': 'qbit'}
for plans in ([main, develop], [develop, main]):
source, request = Mock(), Mock()
result = settings.reconcile_repository(plans, [], source, True, request)
self.assertEqual('not-managed', result['state'])
source.assert_not_called()
request.assert_not_called()
for plans, findings in [
([main, {**main, 'branch': 'develop', 'changed': True}], []),
([main], [{'repository': main['repository'], 'branch': 'develop', 'state': 'unresolved'}]),
]:
request = Mock()
self.assertEqual('waiting-source', settings.reconcile_repository(
plans, findings, Mock(), True, request)['state'])
request.assert_not_called()
def test_repository_policy_checks_all_heads_and_writes_once(self):
plans = [self.plan, {**self.plan, 'branch': 'develop', 'head': 'b' * 40}]
def source(path):
if '/branches/' in path:
return {'commit': {'sha': next(p['head'] for p in plans if path.endswith('/' + p['branch']))}}
return {'archived': False, 'default_branch': 'main'}
request = Mock(side_effect=[{'feature_flags': self.before}, {},
{'feature_flags': {**self.before, 'autocancel-builds': True}}])
self.assertEqual({'state': 'enforced', 'changed': True},
settings.reconcile_repository(plans, [], source, True, request))
self.assertEqual(1, sum(len(call.args) == 2 for call in request.call_args_list))
request = Mock()
stale = lambda path: {'commit': {'sha': self.plan['head']}}
self.assertEqual('unresolved', settings.reconcile_repository(plans, [], stale, True, request)['state'])
request.assert_not_called()
if __name__ == '__main__':
unittest.main()
-131
View File
@@ -1,131 +0,0 @@
"""Discovery covers all current branches and never turns an unknown into a target."""
from pathlib import Path
import sys
import unittest
from unittest import mock
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'tools'))
import discover_enrollment as discovery
def entry(name, archived=False):
return {'nameWithOwner': 'QRun-IO/' + name, 'isArchived': archived}
class DiscoveryTests(unittest.TestCase):
def policy(self):
return {'QRun-IO/example': {'baseline_sha': 'a' * 40, 'required_paths': ['README.md'],
'forbidden_paths': ['.qqq-ci.yml'], 'forbidden_prefixes': ['.circleci/', '.github/workflows/'],
'rationale': 'Approved documentation-only snapshot'}}
def test_approved_snapshot_is_reported_without_enrollment(self):
def request(path):
if '/branches?' in path:
return [{'name': 'main'}, {'name': 'develop'}]
if '/branches/' in path:
return {'commit': {'sha': ('a' if path.endswith('main') else 'b') * 40}}
return {'truncated': False, 'tree': [{'path': 'README.md', 'type': 'blob'}]}
inspect = mock.Mock(side_effect=lambda target, version: target)
with mock.patch.dict(discovery.audit_estate.NO_CI, self.policy(), clear=True):
plans, findings = discovery.discover('QRun-IO', '0.8', inspect, request, [entry('example')])
self.assertEqual([{'repository': 'QRun-IO/example', 'branch': 'develop'}], plans)
self.assertEqual('approved-no-ci', findings[0]['state'])
self.assertEqual('a' * 40, findings[0]['head'])
inspect.assert_called_once_with(plans[0], '0.8')
def test_no_ci_tree_must_be_complete_and_match_approved_identity(self):
for tree in ({'truncated': True, 'tree': [{'path': 'README.md'}]},
{'truncated': False, 'tree': []},
{'truncated': False, 'tree': [{'path': 'README.md'}, {'path': '.circleci/config.yml'}]}):
def request(path):
if '/branches?' in path:
return [{'name': 'main'}]
if '/branches/' in path:
return {'commit': {'sha': 'a' * 40}}
return tree
inspect = mock.Mock()
with mock.patch.dict(discovery.audit_estate.NO_CI, self.policy(), clear=True):
plans, findings = discovery.discover('QRun-IO', '0.8', inspect, request, [entry('example')])
self.assertEqual([], plans)
self.assertEqual('unresolved', findings[0]['state'])
inspect.assert_not_called()
def test_old_no_ci_label_never_exempts_executable_or_deployment_source(self):
for source in ('base/deployment.yaml', 'template/pom.xml', 'scripts/customize.py', 'Formula/qctl.rb'):
def request(path):
if '/branches?' in path:
return [{'name': 'main'}]
if '/branches/' in path:
return {'commit': {'sha': 'a' * 40}}
return {'truncated': False, 'tree': [{'path': 'README.md', 'type': 'blob'},
{'path': source, 'type': 'blob'}]}
inspect = mock.Mock(side_effect=ValueError('Source needs centralized validation'))
with mock.patch.dict(discovery.audit_estate.NO_CI, self.policy(), clear=True):
plans, findings = discovery.discover('QRun-IO', '0.8', inspect, request, [entry('example')])
self.assertEqual([], plans)
self.assertEqual('unresolved', findings[0]['state'])
inspect.assert_called_once()
def test_changed_no_ci_source_is_never_exempted(self):
def request(path):
if '/branches?' in path:
return [{'name': 'main'}]
return {'commit': {'sha': 'b' * 40}}
inspect = mock.Mock(side_effect=ValueError('Unrecognized executable source'))
with mock.patch.dict(discovery.audit_estate.NO_CI, self.policy(), clear=True):
plans, findings = discovery.discover('QRun-IO', '0.8', inspect, request, [entry('example')])
self.assertEqual([], plans)
self.assertEqual('unresolved', findings[0]['state'])
inspect.assert_called_once()
def test_paginates_branches_and_ignores_historical_releases(self):
requests = []
def request(path):
requests.append(path)
if path.endswith('page=1'):
return [{'name': 'release/' + str(i)} for i in range(99)] + [{'name': 'main'}]
return [{'name': 'develop'}]
inspect = mock.Mock(side_effect=lambda target, version: {**target, 'version': version})
plans, findings = discovery.discover('QRun-IO', '0.8', inspect, request, [entry('example')])
self.assertEqual(['develop', 'main'], [p['branch'] for p in plans])
self.assertEqual(2, len(requests))
self.assertEqual([], findings)
def test_unknown_branch_and_profile_stay_unresolved_while_other_targets_progress(self):
def inspect(target, version):
if target['repository'].endswith('/custom'):
raise ValueError('Unknown orchestration')
return target
def request(path):
return [{'name': 'feature/new'}] if '/new/' in path else [{'name': 'main'}]
plans, findings = discovery.discover('QRun-IO', '0.8', inspect, request,
[entry('custom'), entry('new'), entry('good')])
self.assertEqual([{'repository': 'QRun-IO/good', 'branch': 'main'}], plans)
self.assertEqual(2, len(findings))
self.assertTrue(all(f['state'] == 'unresolved' for f in findings))
def test_archived_and_owner_repositories_are_never_modified(self):
inspect, request = mock.Mock(), mock.Mock()
plans, findings = discovery.discover('QRun-IO', '0.8', inspect, request,
[entry('old', True), entry('qqq-orb')])
self.assertEqual([], plans)
self.assertEqual(['archived', 'owner-bootstrap'], [f['state'] for f in findings])
inspect.assert_not_called()
request.assert_not_called()
def test_api_errors_and_duplicate_branches_never_become_empty_success(self):
for result in (ValueError('GitHub read unavailable'), [{'name': 'main'}, {'name': 'main'}]):
request = mock.Mock(side_effect=result) if isinstance(result, Exception) else mock.Mock(return_value=result)
inspect = mock.Mock()
plans, findings = discovery.discover('QRun-IO', '0.8', inspect, request, [entry('example')])
self.assertEqual([], plans)
self.assertEqual('unresolved', findings[0]['state'])
inspect.assert_not_called()
if __name__ == '__main__':
unittest.main()
@@ -1,69 +0,0 @@
#!/usr/bin/env python3
"""Exercise the real version script and Maven install; never publish remotely."""
import os
from pathlib import Path
import shutil
import subprocess
import tempfile
import xml.etree.ElementTree as ET
ROOT = Path(__file__).resolve().parents[1]
NS = {'m': 'http://maven.apache.org/POM/4.0.0'}
BASH = os.environ.get('TEST_BASH') or shutil.which('bash')
MAVEN = shutil.which('mvn') or shutil.which('mvn.cmd')
def run(args, directory, env=None):
subprocess.run(args, cwd=directory, env=env, check=True, timeout=300)
def project(body):
return '<project xmlns="http://maven.apache.org/POM/4.0.0">\n<modelVersion>4.0.0</modelVersion>\n' + body + '\n</project>\n'
def main():
assert BASH and MAVEN, 'Real Bash, Java and Maven are required'
with tempfile.TemporaryDirectory(prefix='qqq-version-') as temporary:
base = Path(temporary)
for name, version, revision, branch, expected, reactor in (
('literal', '0.4.0-SNAPSHOT', '0.4.0-RC.1', 'release/0.4', '0.4.0-RC.2', False),
('friendly', '${revision}', '0.4.0-RC.1', 'release/0.4', '0.4.0-RC.2', False),
('reactor', '0.4.0-SNAPSHOT', '0.4.0-RC.1', 'release/0.4', '0.4.0-RC.2', True),
('unchanged', '0.3.0-SNAPSHOT', '0.4.0-SNAPSHOT', 'develop', '0.4.0-SNAPSHOT', False),
):
root = base / name
root.mkdir()
modules = '<modules><module>child</module></modules>' if reactor else ''
(root / 'pom.xml').write_text(project(f'''<groupId>example.version</groupId>
<artifactId>{name}</artifactId><version>{version}</version><packaging>pom</packaging>
<properties>
<revision>{revision}</revision>
</properties>
{modules}'''), encoding='utf-8')
if reactor:
(root / 'child').mkdir()
(root / 'child/pom.xml').write_text(project(f'''<parent><groupId>example.version</groupId>
<artifactId>{name}</artifactId><version>{version}</version></parent>
<artifactId>child</artifactId><packaging>pom</packaging>'''), encoding='utf-8')
run(['git', 'init', '-q'], root)
env = dict(os.environ, CIRCLE_BRANCH=branch, CIRCLE_TAG='')
run([BASH, str(ROOT / 'src/scripts/calculate_version.sh')], root, env)
if name == 'friendly':
assert ET.parse(root / 'pom.xml').findtext('m:version', namespaces=NS) == '${revision}'
# Install to a temporary repository so the actual artifact coordinate
# proves the fix, rather than merely checking the revision property.
repository = base / 'repository'
run([MAVEN, '-B', '-ntp', '-Dmaven.repo.local=' + str(repository),
'-Dflatten.mode=resolveCiFriendliesOnly', '-DupdatePomFile=true',
'org.codehaus.mojo:flatten-maven-plugin:1.7.3:flatten', 'install'], root)
for artifact in ([name, 'child'] if reactor else [name]):
installed = repository / 'example/version' / artifact / expected / f'{artifact}-{expected}.pom'
assert installed.exists(), installed
pom = ET.parse(installed)
actual = pom.findtext('m:version', namespaces=NS) or pom.findtext('m:parent/m:version', namespaces=NS)
assert actual == expected, (artifact, actual, expected)
print(f'PASS: {name} installed at {expected}', flush=True)
if __name__ == '__main__':
main()
-279
View File
@@ -1,279 +0,0 @@
"""Shared enforcement only adds proven gates and preserves unrelated policy."""
import copy
import base64
import importlib.util
from pathlib import Path
import unittest
import yaml
spec = importlib.util.spec_from_file_location('enforcement', Path(__file__).resolve().parents[1] / 'tools/enforce_orb_checks.py')
enforcement = importlib.util.module_from_spec(spec)
spec.loader.exec_module(enforcement)
def runs():
return [{'id': index, 'name': name, 'app': {'id': 18001}, 'conclusion': 'success'}
for index, name in enumerate(('setup', 'verify'), 1)]
class EnforcementTests(unittest.TestCase):
def test_profile_gates_match_shared_branch_workflows(self):
spec = importlib.util.spec_from_file_location('pipeline',
Path(__file__).resolve().parents[1] / 'src/scripts/generate_pipeline.py')
pipeline = importlib.util.module_from_spec(spec)
spec.loader.exec_module(pipeline)
for profile in ('all', 'cli', 'android', 'app-starter', 'mcp-policy'):
desired = enforcement.gates(profile)
for branch in ('main', 'develop', 'feature/change'):
config = {'orb_version': '0.8', 'publish': {'snapshot': False, 'release': False}}
output = pipeline.PROFILE_TEMPLATES[profile](config, branch, '')
self.assertEqual([desired[1]['context']], list(output['workflows']))
successful = [{'id': i, 'name': gate['context'], 'app': {'id': 18001},
'conclusion': 'success'} for i, gate in enumerate(desired)]
before = {'required_status_checks': {'checks': [
*desired, {'context': 'scanner', 'app_id': 15368}]}}
checks = enforcement.required_checks(before, successful, desired)['checks']
self.assertTrue(enforcement.same_checks(before['required_status_checks']['checks'], checks))
self.assertNotIn({'context': 'verify', 'app_id': 18001}, checks)
for bad in (successful[:1], [*successful, dict(successful[1], id=99, conclusion='failure')],
[dict(run, app={'id': 999}) for run in successful]):
with self.subTest(profile=profile), self.assertRaises(ValueError):
enforcement.required_checks(before, bad, desired)
record, effective = self.ruleset_fixture()
record['rules'][0]['parameters']['required_status_checks'] = [
{'context': gate['context'], 'integration_id': gate['app_id']} for gate in desired]
enforcement.verified_rulesets('QRun-IO/example', effective, lambda path: record, desired)
with self.assertRaises(ValueError):
enforcement.verified_rulesets('QRun-IO/example', effective, lambda path: record)
def test_check_comparison_ignores_only_order(self):
checks = [*enforcement.DESIRED, {'context': 'scanner', 'app_id': 15368}]
self.assertTrue(enforcement.same_checks(checks, list(reversed(checks))))
self.assertFalse(enforcement.same_checks(checks, checks[:-1]))
self.assertFalse(enforcement.same_checks(checks, checks + checks[:1]))
self.assertFalse(enforcement.same_checks(checks, [*enforcement.DESIRED, {'context': 'scanner', 'app_id': 999}]))
def test_preserves_security_gate_and_binds_orb_checks(self):
before = {'required_status_checks': {'strict': False, 'contexts': ['security', 'setup'],
'checks': [{'context': 'security', 'app_id': 15368}, {'context': 'setup', 'app_id': -1}]}}
original = copy.deepcopy(before)
result = enforcement.required_checks(before, runs())
self.assertEqual(original, before)
self.assertEqual(result, {'strict': True, 'checks': [{'context': 'security', 'app_id': 15368}, *enforcement.DESIRED]})
def test_wrong_app_failed_latest_and_missing_checks_reject(self):
cases = [[], [runs()[0]], [dict(run, app={'id': 999}) for run in runs()],
runs() + [dict(runs()[1], id=3, conclusion='failure')]]
for candidate in cases:
with self.subTest(candidate=candidate), self.assertRaises(ValueError):
enforcement.required_checks(None, candidate)
with self.assertRaises(ValueError):
enforcement.required_checks({'required_status_checks': {'checks': [{'context': 'verify', 'app_id': 999}]}}, runs())
def test_existing_unrestricted_contexts_are_retained(self):
result = enforcement.required_checks({'required_status_checks': {'contexts': ['legacy-build']}}, runs())
self.assertIn({'context': 'legacy-build', 'app_id': -1}, result['checks'])
def fixture(self, before):
state = {'before': copy.deepcopy(before), 'head': 'a' * 40, 'writes': []}
def request(path, method='GET', payload=None):
if method != 'GET':
state['writes'].append((method, path, payload))
if method == 'PUT':
state['before'] = {**payload, 'enforce_admins': {'enabled': True}}
elif method == 'PATCH':
state['before']['required_status_checks'] = copy.deepcopy(payload)
elif method == 'POST':
state['before']['enforce_admins'] = {'enabled': True}
return None
if '/rules/' in path:
return []
if path.endswith('/protection'):
return copy.deepcopy(state['before'])
return {'commit': {'sha': state['head']}}
plan = {'repository': 'QRun-IO/example', 'branch': 'develop', 'head': 'a' * 40,
'before': copy.deepcopy(before), 'required_status_checks': enforcement.required_checks(before, runs()), 'changed': True}
return plan, request, state
def test_narrow_updates_preserve_reviews_signatures_and_restrictions(self):
before = {'required_status_checks': {'strict': False, 'checks': []}, 'enforce_admins': {'enabled': False},
'required_signatures': {'enabled': True}, 'required_linear_history': {'enabled': True},
'required_pull_request_reviews': {'required_approving_review_count': 2},
'restrictions': {'users': [{'login': 'owner'}]}}
plan, request, state = self.fixture(before)
enforcement.enforce(plan, request)
self.assertEqual(['PATCH', 'POST'], [write[0] for write in state['writes']])
for key in ('required_signatures', 'required_linear_history', 'required_pull_request_reviews', 'restrictions'):
self.assertEqual(before[key], state['before'][key])
def test_new_protection_and_idempotent_noop(self):
plan, request, state = self.fixture(None)
enforcement.enforce(plan, request)
self.assertEqual('PUT', state['writes'][0][0])
self.assertIsNone(state['before']['required_pull_request_reviews'])
plan['before'] = copy.deepcopy(state['before'])
plan['changed'] = False
state['writes'] = []
enforcement.enforce(plan, request)
self.assertEqual([], state['writes'])
def test_github_may_reorder_required_checks_on_readback(self):
before = {'required_status_checks': {'strict': True, 'checks': [
*enforcement.DESIRED, {'context': 'scanner', 'app_id': 15368}]},
'enforce_admins': {'enabled': False}, 'required_signatures': {'enabled': True}}
plan, request, state = self.fixture(before)
def reordered(path, method='GET', payload=None):
value = request(path, method, payload)
if method == 'PATCH':
state['before']['required_status_checks']['checks'].reverse()
return value
enforcement.enforce(plan, reordered)
self.assertTrue(state['before']['enforce_admins']['enabled'])
self.assertEqual(before['required_signatures'], state['before']['required_signatures'])
def test_head_or_protection_drift_never_mutates(self):
for field, value in [('head', 'b' * 40), ('before', {'unexpected': True})]:
plan, request, state = self.fixture(None)
state[field] = value
with self.assertRaises(ValueError):
enforcement.enforce(plan, request)
self.assertEqual([], state['writes'])
def ruleset_fixture(self):
parameters = {'strict_required_status_checks_policy': False,
'required_status_checks': [{'context': c['context'], 'integration_id': c['app_id']}
for c in enforcement.DESIRED] + [{'context': 'scanner', 'integration_id': 15368}]}
record = {'id': 123, 'enforcement': 'active', 'target': 'branch', 'source_type': 'Repository',
'source': 'QRun-IO/example', 'bypass_actors': [],
'rules': [{'type': 'required_status_checks', 'parameters': parameters}]}
effective = [{'type': 'required_status_checks', 'ruleset_source_type': 'Repository',
'ruleset_source': 'QRun-IO/example', 'ruleset_id': 123, 'parameters': copy.deepcopy(parameters)}]
return record, effective
def test_ruleset_verification_preserves_freshness_scanner_and_never_writes(self):
record, effective = self.ruleset_fixture()
original = copy.deepcopy(record)
def request(path, method='GET', payload=None):
self.assertEqual('GET', method)
if path.endswith('/rulesets/123'):
return copy.deepcopy(record)
if '/rules/branches/' in path:
return copy.deepcopy(effective)
if path.endswith('/protection'):
return None
if '/check-runs?' in path:
return {'check_runs': runs()}
if '/contents/' in path:
data = ({'profile': 'qbit', 'orb_version': '0.8', 'default_branch': 'develop'}
if '/.qqq-ci.yml?' in path else enforcement.standard.canonical_caller('qbit', '0.8'))
return {'encoding': 'base64', 'type': 'file', 'content': base64.b64encode(yaml.safe_dump(data).encode()).decode()}
if '/branches/' in path:
return {'commit': {'sha': 'a' * 40}}
return {'archived': False}
plan = enforcement.inspect('QRun-IO/example', 'develop', '0.8', request)
self.assertFalse(plan['changed'])
self.assertFalse(plan['required_status_checks']['strict'])
self.assertEqual('rulesets', plan['protection_mode'])
enforcement.enforce(plan, request)
self.assertEqual(original, record)
record['bypass_actors'] = [{'actor_type': 'RepositoryRole', 'actor_id': 5}]
with self.assertRaises(ValueError):
enforcement.enforce(plan, request)
def test_inactive_bypass_unbound_and_missing_ruleset_gates_reject(self):
record, effective = self.ruleset_fixture()
cases = [dict(record, enforcement='evaluate'), dict(record, bypass_actors=[{'actor_id': 1}]),
dict(record, rules=[])]
unbound = copy.deepcopy(record)
unbound['rules'][0]['parameters']['required_status_checks'][1]['integration_id'] = None
cases.append(unbound)
for candidate in cases:
with self.subTest(candidate=candidate), self.assertRaises(ValueError):
enforcement.verified_rulesets('QRun-IO/example', effective, lambda path: candidate)
def test_unsupported_ruleset_sources_are_not_inferred_as_protected(self):
record, effective = self.ruleset_fixture()
effective[0]['ruleset_source_type'] = 'Organization'
with self.assertRaises(ValueError):
enforcement.verified_rulesets('QRun-IO/example', effective, lambda path: record)
def test_empty_push_allowlist_gains_ruleset_without_changing_access(self):
self.assert_ruleset_preserves_classic({
'restrictions': {'users': [], 'teams': [], 'apps': []},
'enforce_admins': {'enabled': False}, 'required_signatures': {'enabled': True}})
def test_existing_protection_without_status_section_gains_ruleset(self):
for admin in (False, True):
with self.subTest(enforce_admins=admin):
self.assert_ruleset_preserves_classic({
'enforce_admins': {'enabled': admin},
'required_signatures': {'enabled': True},
'required_linear_history': {'enabled': True},
'required_pull_request_reviews': {'required_approving_review_count': 2},
'required_conversation_resolution': {'enabled': True}})
def assert_ruleset_preserves_classic(self, classic):
state = {'ruleset': None, 'writes': []}
def request(path, method='GET', payload=None):
if method != 'GET':
self.assertEqual(('POST', 'repos/QRun-IO/example/rulesets'), (method, path))
state['writes'].append(copy.deepcopy(payload))
state['ruleset'] = {**copy.deepcopy(payload), 'id': 123,
'source_type': 'Repository', 'source': 'QRun-IO/example'}
return state['ruleset']
if '/rulesets?' in path:
return []
if path.endswith('/rulesets/123'):
return copy.deepcopy(state['ruleset'])
if '/rules/branches/' in path:
if state['ruleset'] is None:
return []
return [{**state['ruleset']['rules'][0], 'ruleset_source_type': 'Repository',
'ruleset_source': 'QRun-IO/example', 'ruleset_id': 123}]
if path.endswith('/protection'):
return copy.deepcopy(classic)
if '/check-runs?' in path:
return {'check_runs': runs()}
if '/contents/' in path:
data = ({'profile': 'core', 'orb_version': '0.8', 'default_branch': 'develop'}
if '/.qqq-ci.yml?' in path else enforcement.standard.canonical_caller('core', '0.8'))
return {'encoding': 'base64', 'type': 'file', 'content': base64.b64encode(yaml.safe_dump(data).encode()).decode()}
if '/branches/' in path:
return {'commit': {'sha': 'a' * 40}}
return {'archived': False}
plan = enforcement.inspect('QRun-IO/example', 'main', '0.8', request)
self.assertEqual('create-ruleset', plan['protection_mode'])
enforcement.enforce(plan, request)
self.assertEqual(1, len(state['writes']))
self.assertEqual([], state['writes'][0]['bypass_actors'])
self.assertEqual({'include': ['refs/heads/main'], 'exclude': []}, state['writes'][0]['conditions']['ref_name'])
self.assertEqual(classic, plan['before'])
repeat = enforcement.inspect('QRun-IO/example', 'main', '0.8', request)
self.assertFalse(repeat['changed'])
enforcement.enforce(repeat, request)
self.assertEqual(1, len(state['writes']))
def test_existing_named_ruleset_is_not_overwritten_or_duplicated(self):
before = {'restrictions': {'users': [], 'teams': [], 'apps': []}, 'enforce_admins': {'enabled': False}}
plan, base_request, state = self.fixture(before)
plan['protection_mode'] = 'create-ruleset'
def request(path, method='GET', payload=None):
if '/rulesets?' in path:
return [{'name': 'QQQ automated verification: develop'}]
return base_request(path, method, payload)
with self.assertRaisesRegex(ValueError, 'already exists'):
enforcement.enforce(plan, request)
self.assertEqual([], state['writes'])
if __name__ == '__main__':
unittest.main()
-110
View File
@@ -1,110 +0,0 @@
"""Batch enrollment preserves policy and rejects unsafe or unknown declarations."""
from pathlib import Path
import sys
import tempfile
import unittest
from unittest import mock
import yaml
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'tools'))
import enroll_consumers as enrollment
class EnrollmentTests(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory()
self.addCleanup(self.temp.cleanup)
self.root = Path(self.temp.name)
def legacy(self, name):
root = self.root / name
(root / '.circleci').mkdir(parents=True)
config = {'version': 2.1, 'orbs': {'qqq-orb': 'kingsrook/qqq-orb@0.5.0'},
'workflows': {'verify': {'jobs': [{'qqq-orb/mvn_test_only': {}}]}}}
(root / '.circleci/config.yml').write_text(yaml.safe_dump(config), encoding='utf-8')
return {'checkout': str(root), 'default_branch': 'main'}
def canonical(self, profile):
target = self.legacy(profile)
root = Path(target['checkout'])
data = {'profile': profile, 'orb_version': '0.7.4', 'default_branch': 'main'}
if profile == 'node-artifact':
data.update(context='registry', npm_scopes=['@example'], node_version='20.19',
artifact_path='dist/bundle.js', build_branches=['main'])
if profile == 'homebrew-tap':
data.update(tap='qrun-io/qctl', formula='qctl')
(root / '.qqq-ci.yml').write_text(yaml.safe_dump(data), encoding='utf-8')
(root / '.circleci/config.yml').write_text(yaml.safe_dump(
enrollment.rollout.standard.canonical_caller(profile, '0.7.4')), encoding='utf-8')
return target
def test_mixed_fleet_preview_write_and_idempotence(self):
legacy = self.legacy('legacy')
targets = [legacy] + [self.canonical(p) for p in enrollment.rollout.standard.PROFILE_PARAMETERS]
preview = enrollment.enroll(targets, '0.8')
self.assertTrue(all(item['changed'] for item in preview))
self.assertFalse((Path(legacy['checkout']) / '.qqq-ci.yml').exists())
result = enrollment.enroll(targets, '0.8', write=True)
self.assertEqual(1, sum(item['enrollment'] for item in result))
data = yaml.safe_load((Path(legacy['checkout']) / '.qqq-ci.yml').read_text())
self.assertFalse(any(data['publish'].values()))
self.assertEqual('all', data['test_branch_policy'])
self.assertTrue(all(not item['changed'] for item in enrollment.enroll(targets, '0.8', write=True)))
def test_unknown_orchestration_rejects_entire_batch(self):
good, bad = self.legacy('good'), self.legacy('bad')
path = Path(bad['checkout']) / '.circleci/config.yml'
path.write_text(path.read_text() + '\njobs: {custom: {steps: [checkout]}}\n')
original = (Path(good['checkout']) / '.circleci/config.yml').read_bytes()
with self.assertRaises(ValueError):
enrollment.enroll([good, bad], '0.8', write=True)
self.assertEqual(original, (Path(good['checkout']) / '.circleci/config.yml').read_bytes())
self.assertFalse((Path(good['checkout']) / '.qqq-ci.yml').exists())
def test_duplicate_targets_and_wrong_default_reject_before_write(self):
target = self.canonical('core')
for targets in ([target, target], [{**target, 'default_branch': 'develop'}]):
with self.assertRaises(ValueError):
enrollment.enroll(targets, '0.8', write=True)
self.assertEqual('0.7.4', yaml.safe_load((Path(target['checkout']) / '.qqq-ci.yml').read_text())['orb_version'])
def test_invalid_manifest_and_unstable_versions_rejected(self):
target = self.canonical('qbit')
path = Path(target['checkout']) / '.qqq-ci.yml'
path.write_text(path.read_text() + '\nunknown_policy: true\n')
for version in ('0.8', 'dev:snapshot', 'latest', 0.8):
with self.assertRaises(ValueError):
enrollment.enroll([target], version, write=True)
def test_drift_after_preflight_is_not_overwritten(self):
target = self.legacy('drift')
path = Path(target['checkout']) / '.circleci/config.yml'
original_plan = enrollment.plan
def changed_plan(*args):
result = original_plan(*args)
path.write_bytes(b'concurrent edit\n')
return result
with mock.patch.object(enrollment, 'plan', side_effect=changed_plan):
with self.assertRaises(ValueError):
enrollment.enroll([target], '0.8', write=True)
self.assertEqual(b'concurrent edit\n', path.read_bytes())
self.assertFalse((Path(target['checkout']) / '.qqq-ci.yml').exists())
def test_manifest_symlink_is_rejected(self):
target = self.legacy('redirect')
outside = self.root / 'outside.yml'
outside.write_text('untouched\n')
try:
(Path(target['checkout']) / '.qqq-ci.yml').symlink_to(outside)
except OSError as error:
self.skipTest('Symlink creation unavailable: ' + str(error))
with self.assertRaises(ValueError):
enrollment.enroll([target], '0.8', write=True)
self.assertEqual('untouched\n', outside.read_text())
if __name__ == '__main__':
unittest.main()
-103
View File
@@ -1,103 +0,0 @@
"""Automatic completion requires qualified source and preserves normal merge policy."""
import json
from pathlib import Path
import sys
import unittest
from unittest import mock
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'tools'))
import finish_enrollment as finish
class FinishTests(unittest.TestCase):
def setUp(self):
self.plan = {'repository': 'QRun-IO/example', 'branch': 'main', 'head': 'a' * 40,
'profile': 'qbit', 'version': '0.8'}
self.result = {'state': 'existing-pr', 'head': 'b' * 40, 'url': 'https://github.com/QRun-IO/example/pull/1'}
self.pr = {'number': 1, 'state': 'OPEN', 'headRefOid': 'b' * 40,
'baseRefOid': 'a' * 40, 'baseRefName': 'main', 'mergeCommit': None}
def checks(self):
return [{'id': i, 'name': name, 'app': {'id': 18001}, 'status': 'completed', 'conclusion': 'success'}
for i, name in enumerate(('setup', 'verify'))]
def test_all_checks_and_statuses_must_pass_and_gates_must_use_circle_app(self):
checks = self.checks()
checks[1]['app']['id'] = 123
checks.append({'id': 3, 'name': 'scanner', 'app': {'id': 15368}, 'status': 'completed', 'conclusion': 'failure'})
with mock.patch.object(finish.remote, 'api', side_effect=[{'check_runs': checks},
{'statuses': [{'context': 'legacy-test', 'state': 'pending'}]}]):
self.assertEqual(['legacy-test', 'scanner', 'verify'], finish.qualification('QRun-IO/example', 'b' * 40))
def test_profile_qualification_requires_its_actual_workflow(self):
for profile in ('all', 'cli', 'android', 'app-starter', 'mcp-policy'):
checks = self.checks()
checks[1]['name'] = finish.enforcement.PROFILE_GATES[profile]
with mock.patch.object(finish.remote, 'api', side_effect=[
{'check_runs': checks}, {'statuses': []}]):
self.assertEqual([], finish.qualification('QRun-IO/example', 'b' * 40, profile))
with mock.patch.object(finish.remote, 'api', side_effect=[
{'check_runs': self.checks()}, {'statuses': []}]):
self.assertEqual([checks[1]['name']], finish.qualification('QRun-IO/example', 'b' * 40, profile))
def test_failed_or_missing_checks_never_merge(self):
with mock.patch.object(finish.remote, 'verify_submission'), mock.patch.object(finish.remote, 'command', return_value=json.dumps(self.pr)) as command, mock.patch.object(finish.remote, 'api', return_value={'commit': {'sha': self.plan['head']}}), mock.patch.object(finish, 'qualification', return_value=['verify']):
self.assertEqual('waiting-checks', finish.merge_ready(self.plan, self.result)['state'])
self.assertEqual(1, command.call_count)
self.assertEqual(['gh', 'pr', 'view'], command.call_args.args[0][:3])
def test_stale_base_never_merges_even_when_checks_were_green(self):
self.pr['baseRefOid'] = 'c' * 40
with mock.patch.object(finish.remote, 'verify_submission'), mock.patch.object(finish.remote, 'command', return_value=json.dumps(self.pr)) as command:
self.assertEqual('stale-base', finish.merge_ready(self.plan, self.result)['state'])
self.assertEqual(1, command.call_count)
def test_success_uses_exact_head_normal_merge_and_verifies_result(self):
calls = []
def command(args):
calls.append(args)
if args[2] == 'merge':
return ''
if len(calls) == 1:
return json.dumps(self.pr)
return json.dumps({'state': 'MERGED', 'mergeCommit': {'oid': 'd' * 40}})
def api(path):
if '/branches/' in path:
return {'commit': {'sha': self.plan['head']}}
return {'commit': {'tree': {'sha': 'e' * 40}, 'verification': {'verified': True}}}
with mock.patch.object(finish.remote, 'verify_submission'), mock.patch.object(finish.remote, 'command', side_effect=command), mock.patch.object(finish.remote, 'api', side_effect=api), mock.patch.object(finish, 'qualification', return_value=[]):
result = finish.merge_ready(self.plan, self.result)
self.assertEqual('merged', result['state'])
self.assertEqual(['gh', 'pr', 'merge', '1', '--repo', 'QRun-IO/example', '--squash', '--match-head-commit', 'b' * 40], calls[1])
def test_enforcement_waits_for_current_head_and_uses_shared_preserving_writer(self):
with mock.patch.object(finish.enforcement, 'inspect', side_effect=ValueError('Current branch has no successful verify')), mock.patch.object(finish.enforcement, 'enforce') as enforce:
self.assertEqual('waiting-enforcement', finish.enforce_current(self.plan)['state'])
enforce.assert_not_called()
current = {'head': 'd' * 40, 'changed': True}
with mock.patch.object(finish.enforcement, 'inspect', return_value=current), mock.patch.object(finish.enforcement, 'enforce') as enforce:
self.assertEqual('enforced', finish.enforce_current(self.plan)['state'])
enforce.assert_called_once_with(current)
def test_unknown_gate_profile_never_merges_or_enforces(self):
self.plan['profile'] = 'unknown-profile'
with mock.patch.object(finish.remote, 'command') as command, mock.patch.object(finish.enforcement, 'inspect') as inspect:
self.assertEqual('unsupported-gates', finish.merge_ready(self.plan, self.result)['state'])
self.assertEqual('unsupported-gates', finish.enforce_current(self.plan)['state'])
command.assert_not_called()
inspect.assert_not_called()
def test_merged_tree_mismatch_is_not_reported_as_success(self):
self.pr.update(state='MERGED', mergeCommit={'oid': 'd' * 40})
with mock.patch.object(finish.remote, 'verify_submission'), mock.patch.object(finish.remote, 'command', return_value=json.dumps(self.pr)), mock.patch.object(finish.remote, 'api', side_effect=[
{'commit': {'tree': {'sha': 'e' * 40}}},
{'commit': {'tree': {'sha': 'f' * 40}, 'verification': {'verified': True}}}]):
with self.assertRaises(ValueError):
finish.merge_ready(self.plan, self.result)
if __name__ == '__main__':
unittest.main()
-37
View File
@@ -1,37 +0,0 @@
#!/bin/bash
# Exercise the actual test runner with no server, browser, network or Maven work.
set -euo pipefail
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
scratch=$(mktemp -d)
trap 'rm -rf "$scratch"' EXIT
mkdir "$scratch/bin"
export RUN_LOG="$scratch/calls"
cat > "$scratch/bin/npm" <<'STUB'
#!/bin/bash
printf 'npm %s\n' "$*" >> "$RUN_LOG"
if [[ "$*" == 'run build' && ${BUILD_FAIL:-0} == 1 ]]; then exit 12; fi
STUB
for tool in mvn dockerize; do
# shellcheck disable=SC2016
printf '#!/bin/bash\nprintf "%s %%s\\n" "$*" >> "$RUN_LOG"\n' "$tool" > "$scratch/bin/$tool"
done
chmod +x "$scratch/bin/"*
export PATH="$scratch/bin:$PATH"
cd "$scratch"
PARAM_BUILD_BEFORE_TEST=false bash "$root/src/scripts/frontend_test_run.sh" >/dev/null
if grep -qx 'npm run build' "$RUN_LOG"; then
echo 'FAIL: combined job rebuilt the already-built React app' >&2; exit 1
fi
grep -q '^mvn .* -B test$' "$RUN_LOG"
echo 'PASS: combined job retains tests without a duplicate build'
: > "$RUN_LOG"
bash "$root/src/scripts/frontend_test_run.sh" >/dev/null
grep -qx 'npm run build' "$RUN_LOG"
grep -q '^mvn .* -B test$' "$RUN_LOG"
: > "$RUN_LOG"
if BUILD_FAIL=1 bash "$root/src/scripts/frontend_test_run.sh" >/dev/null 2>&1; then
echo 'FAIL: required standalone build failure ignored' >&2; exit 1
fi
if grep -q '^mvn ' "$RUN_LOG"; then exit 1; fi
echo 'PASS: standalone command still builds and fails before tests when its build fails'
File diff suppressed because it is too large. Load diff
-28
View File
@@ -1,28 +0,0 @@
"""Capability probes must drain help output while still rejecting missing options."""
from pathlib import Path
import shlex
import subprocess
import unittest
class InstallerTests(unittest.TestCase):
def probe(self, flags):
installer = Path(__file__).resolve().parents[1] / 'tools/install_reconcile_gh.sh'
checks = '\n'.join(line for line in installer.read_text().splitlines()
if line.startswith(('gh api --help', 'gh pr merge --help')))
self.assertEqual(2, len(checks.splitlines()))
producer = ("import signal,sys; signal.signal(signal.SIGPIPE,signal.SIG_DFL); "
f"print({flags!r},flush=True); sys.stdout.write('x'*1048576)")
script = 'set -euo pipefail\ngh() { python3 -c ' + shlex.quote(producer) + '; }\n' + checks
return subprocess.run(['bash', '-c', script], capture_output=True, timeout=10).returncode
def test_large_help_output_does_not_kill_producer(self):
self.assertEqual(0, self.probe('--slurp --match-head-commit'))
def test_missing_required_option_fails(self):
self.assertNotEqual(0, self.probe('--slurp'))
self.assertNotEqual(0, self.probe('--match-head-commit'))
if __name__ == '__main__':
unittest.main()
-62
View File
@@ -1,62 +0,0 @@
#!/usr/bin/env python3
"""Run the generated link-check shell with controlled server/readiness commands."""
import json
import os
from pathlib import Path
import subprocess
import tempfile
import unittest
ROOT = Path(__file__).resolve().parents[1]
class MarketingLinkCheckTests(unittest.TestCase):
def run_check(self, ready, link_status):
template = json.loads((ROOT / 'src/scripts/templates/marketing.json').read_text())
build = template['workflows']['pr-checks']['jobs'][0]['munitor/node_build_and_test']
command = json.loads(build['test_commands'])[5]
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
scripts = {
'npm': '#!/bin/bash\nif [ "$2" = start ]; then echo $$ > "$TEST_ROOT/server.pid"; exec /bin/sleep 30; fi\necho checked > "$TEST_ROOT/checked"\nexit "$LINK_STATUS"\n',
'curl': '#!/bin/bash\nexit "$READY_STATUS"\n',
'sleep': '#!/bin/bash\n/bin/sleep 0.01\n',
}
for name, text in scripts.items():
file = root / name
file.write_text(text)
file.chmod(0o755)
env = {**os.environ, 'PATH': str(root) + ':' + os.environ['PATH'],
'TEST_ROOT': str(root), 'READY_STATUS': str(ready), 'LINK_STATUS': str(link_status)}
result = subprocess.run(['bash', '-c', command], env=env, capture_output=True, timeout=10)
self.assertEqual((root / 'checked').exists(), ready == 0)
pid_file = root / 'server.pid'
if pid_file.exists():
with self.assertRaises(ProcessLookupError):
os.kill(int(pid_file.read_text()), 0)
return result
def test_worker_input_is_generated_before_tests(self):
import yaml
template = json.loads((ROOT / 'src/scripts/templates/marketing.json').read_text())
for workflow in template['workflows'].values():
build = workflow['jobs'][0]['munitor/node_build_and_test']
command = json.loads(build['test_commands'])[0]
with tempfile.TemporaryDirectory() as directory:
subprocess.run(['bash', '-c', command], cwd=directory, check=True)
actual = yaml.safe_load((Path(directory) / '.munitor.yml').read_text())
self.assertEqual(actual, {'pipeline': 'node-webapp', 'node_version': 22,
'docker': {'use_repo_dockerfile': True}})
def test_ready_server_runs_links_and_is_cleaned_up(self):
self.assertEqual(self.run_check(0, 0).returncode, 0)
def test_broken_links_fail_and_server_is_cleaned_up(self):
self.assertEqual(self.run_check(0, 7).returncode, 7)
def test_unready_server_fails_without_running_links(self):
result = self.run_check(1, 0)
self.assertEqual(result.returncode, 1)
self.assertIn(b'did not become ready', result.stderr)
if __name__ == '__main__':
unittest.main()
+1 -3
View File
@@ -43,7 +43,7 @@ class MavenCiContract(unittest.TestCase):
self.assertIs(job["parameters"]["notify_rc_published"]["default"], False)
self.assertIs(read("src/commands/git_operations.yml")["parameters"]["require_pushed_version"]["default"], False)
steps = job["steps"]
deploy = next(i for i, s in enumerate(steps) if isinstance(s, dict) and "mvn_jar_deploy" in s)
deploy = steps.index("mvn_jar_deploy")
pushed = next(i for i, s in enumerate(steps) if "git_operations" in str(s))
dispatch = next(i for i, s in enumerate(steps) if "notify_rc_published.sh" in str(s))
self.assertLess(deploy, pushed)
@@ -57,8 +57,6 @@ class MavenCiContract(unittest.TestCase):
return next(i for i, s in enumerate(steps) if s == name or isinstance(s, dict) and name in s)
self.assertLess(position("mvn_manage_version"), position("mvn_verify"))
self.assertLess(position("mvn_verify"), position("mvn_jar_deploy"))
self.assertEqual(steps[position("mvn_jar_deploy")], {"mvn_jar_deploy": {"skip_verified_analysis": True}})
self.assertIs(read("src/commands/mvn_jar_deploy.yml")["parameters"]["skip_verified_analysis"]["default"], False)
self.assertTrue(any("check_middleware_api_versions" in str(s) for s in steps[position("mvn_verify") + 1:position("mvn_jar_deploy")]))
deploy = (ROOT / "src/scripts/mvn_jar_deploy.sh").read_text()
self.assertIn("-P release", deploy)
-49
View File
@@ -6,7 +6,6 @@ scratch=$(mktemp -d)
trap 'rm -rf "$scratch"' EXIT
mkdir "$scratch/bin"
export INSTALL_LOG="$scratch/install.log"
export APT_CONFIG_CAPTURE="$scratch/apt.conf"
export BASH_ENV="$scratch/bash-env"
export QQQ_MAVEN_BIN_DIR="$scratch/maven-bin"
: > "$BASH_ENV"
@@ -15,29 +14,6 @@ cat > "$scratch/bin/sudo" <<'STUB'
printf '%s\n' "$*" >> "$INSTALL_LOG"
if [[ "$*" == *apt-get* ]]; then
[[ "$1" == env && "$2" == DEBIAN_FRONTEND=noninteractive && "$3" == NEEDRESTART_MODE=l ]] || exit 90
[[ -f "$APT_CONFIG_CAPTURE" ]] && grep -qx 'Acquire::Retries "3";' "$APT_CONFIG_CAPTURE" || exit 91
[[ ${APT_FAIL:-0} == 0 ]] || exit "$APT_FAIL"
if [[ "$*" == *'apt-get update'* ]]; then
updates=$(grep -c 'apt-get update' "$INSTALL_LOG")
if [[ $updates -eq 1 ]]; then
# An unavailable image PPA must not participate in base-tool setup.
[[ "$*" == *'Dir::Etc::sourceparts=-'* ]] || exit 100
[[ "$*" == *'Dir::Etc::sourcelist=/etc/apt/sources.list'* || "$*" == *'Dir::Etc::sourcelist=/etc/apt/sources.list.d/ubuntu.sources'* ]] || exit 93
[[ "$*" == *'APT::Get::List-Cleanup=0'* ]] || exit 92
[[ ${UBUNTU_UPDATE_FAIL:-0} == 0 ]] || exit "$UBUNTU_UPDATE_FAIL"
fi
if [[ $updates -eq 2 ]]; then
# Model a PPA outage after the initial base-package installation. Only
# the new signed repository may be contacted; existing indexes survive.
[[ "$*" == *'Dir::Etc::sourcelist=/etc/apt/sources.list.d/adoptium.list'* ]] || exit 100
[[ "$*" == *'Dir::Etc::sourceparts=-'* ]] || exit 100
[[ "$*" == *'APT::Get::List-Cleanup=0'* ]] || exit 92
[[ ${TEMURIN_UPDATE_FAIL:-0} == 0 ]] || exit "$TEMURIN_UPDATE_FAIL"
fi
fi
fi
if [[ "$1" == tee && "$2" == /etc/apt/apt.conf.d/80-qqq-download-retries ]]; then
cat > "$APT_CONFIG_CAPTURE"; exit 0
fi
if [[ "$1" == tee || "$1" == gpg || "$1" == -E ]]; then cat >/dev/null; fi
STUB
@@ -56,29 +32,9 @@ grep -q 'tee /etc/needrestart/conf.d/99-qqq-ci.conf' "$INSTALL_LOG"
grep -q 'export JAVA_HOME=' "$BASH_ENV"
: > "$INSTALL_LOG"
: > "$BASH_ENV"
rm "$APT_CONFIG_CAPTURE"
PARAM_INSTALL_NODE=true bash "$script_dir/install_java.sh" >/dev/null 2>&1
[[ $(grep -c 'apt-get' "$INSTALL_LOG") -eq 5 ]]
grep -q 'apt-get install -y -qq nodejs' "$INSTALL_LOG"
# An outage of the required repository must still stop before Java installation.
: > "$INSTALL_LOG"
if UBUNTU_UPDATE_FAIL=100 PARAM_INSTALL_NODE=false bash "$script_dir/install_java.sh" >/dev/null 2>&1; then
echo 'required Ubuntu repository failure accepted' >&2; exit 1
else
[[ $? -eq 100 ]]
fi
if grep -q 'apt-get install' "$INSTALL_LOG"; then
echo 'package installation continued after Ubuntu repository failure' >&2; exit 1
fi
: > "$INSTALL_LOG"
if TEMURIN_UPDATE_FAIL=100 PARAM_INSTALL_NODE=false bash "$script_dir/install_java.sh" >/dev/null 2>&1; then
echo 'required Temurin repository failure accepted' >&2; exit 1
else
[[ $? -eq 100 ]]
fi
if grep -q 'apt-get install -y -qq temurin-' "$INSTALL_LOG"; then
echo 'Java installation continued after repository failure' >&2; exit 1
fi
# Execute the actual generated selector. Arguments containing spaces stay intact.
export SELECTOR_LOG="$scratch/selector.log"
cat > "$scratch/bin/mvn" <<'NATIVE'
@@ -126,9 +82,4 @@ if PARAM_INSTALL_NODE=false bash "$script_dir/install_java.sh" >/dev/null 2>&1;
echo 'invalid native Maven accepted' >&2; exit 1
fi
unset -f mvn
if APT_FAIL=100 PARAM_INSTALL_NODE=false bash "$script_dir/install_java.sh" >/dev/null 2>&1; then
echo 'persistent apt failure accepted' >&2; exit 1
else
[[ $? -eq 100 ]]
fi
printf 'PASS: unattended installation and native/executable/nonexecutable wrapper selection\n'
+1 -16
View File
@@ -56,28 +56,13 @@ org.codehaus.mojo:flatten-maven-plugin:1.7.3:flatten
deploy
EOF
diff -u "$test_dir/expected-args" "$test_dir/mvn-args"
# The verified publishing job skips only duplicate analysis. Every signing,
# flattening and publication argument must remain identical to the default.
for mode in true 1 false 0; do
PATH="$test_dir/bin:$PATH" TEST_MVN_ARGS="$test_dir/verified-args" \
GPG_KEYNAME=test-key GPG_PASSPHRASE=test-passphrase QQQ_SKIP_VERIFIED_ANALYSIS="$mode" \
bash "$test_dir/deploy.sh" > "$test_dir/verified-output" 2>&1
if [[ "$mode" == true || "$mode" == 1 ]]; then
grep -Fxq -- '-Dspotbugs.skip=true' "$test_dir/verified-args"
grep -Fxq -- '-Dpmd.skip=true' "$test_dir/verified-args"
grep -Fxv -e '-Dspotbugs.skip=true' -e '-Dpmd.skip=true' "$test_dir/verified-args" > "$test_dir/verified-common-args"
diff -u "$test_dir/expected-args" "$test_dir/verified-common-args"
else
diff -u "$test_dir/expected-args" "$test_dir/verified-args"
fi
done
# A Central validation failure or publication timeout must fail this wrapper.
cat > "$test_dir/bin/mvn" <<'EOF'
#!/bin/bash
exit 42
EOF
set +e
PATH="$test_dir/bin:$PATH" GPG_KEYNAME=test-key GPG_PASSPHRASE=test-passphrase QQQ_SKIP_VERIFIED_ANALYSIS=true \
PATH="$test_dir/bin:$PATH" GPG_KEYNAME=test-key GPG_PASSPHRASE=test-passphrase \
bash "$test_dir/deploy.sh" > "$test_dir/failure-output" 2>&1
status=$?
set -e
-112
View File
@@ -1,112 +0,0 @@
"""Native GitOps discovery must prove shared ownership without adding Circle jobs."""
import base64
import copy
from pathlib import Path
import sys
import unittest
from unittest import mock
import yaml
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / 'tools'))
import native_gitops_gates as gitops
import reconcile_enrollment as remote
import enforce_orb_checks as enforcement
import finish_enrollment as finish
class GitOpsTests(unittest.TestCase):
def setUp(self):
self.pin, self.head = 'a' * 40, 'b' * 40
self.repo = 'QRun-IO/voyage-cd'
self.caller = gitops.bridge.canonical('gitops', self.pin)
self.signed, self.worker_drift, self.truncated = True, False, False
self.paths = ['.github/workflows/validate.yml', 'base/kustomization.yaml']
self.runs = [{'id': 1, 'name': 'gitops / validate', 'app': {'id': 15368},
'status': 'completed', 'conclusion': 'success'}]
self.before = {'required_status_checks': {'strict': True, 'checks': gitops.gates()},
'enforce_admins': {'enabled': True}}
def request(self, path, method='GET', payload=None):
self.assertEqual(method, 'GET')
if '/contents/' in path:
self.assertNotIn('.circleci/', path)
if '/QRun-IO/qqq-orb/' in path:
raw = (ROOT / '.github/workflows/gitops.yml').read_text() + ('# drift' if self.worker_drift else '')
else:
raw = yaml.safe_dump(self.caller, sort_keys=False)
return {'type': 'file', 'encoding': 'base64', 'content': base64.b64encode(raw.encode()).decode()}
if '/git/trees/' in path:
return {'truncated': self.truncated, 'tree': [{'path': p} for p in self.paths]}
if path.endswith('/commits/' + self.pin):
return {'commit': {'verification': {'verified': self.signed}}}
if '/check-runs?' in path:
return {'check_runs': copy.deepcopy(self.runs)}
if '/status?' in path:
return {'statuses': []}
if path.endswith('/protection'):
return copy.deepcopy(self.before)
if '/rules/branches/' in path:
return []
if '/branches/' in path:
return {'commit': {'sha': self.head}}
return {'archived': False, 'default_branch': 'main'}
def test_discovery_returns_unchanged_native_plan_without_circle_reads_or_writes(self):
for repo in sorted(gitops.REPOSITORIES):
with mock.patch.object(remote, 'api', side_effect=self.request):
plan = remote.inspect({'repository': repo, 'branch': 'main'}, '0.8')
self.assertEqual(plan['profile'], 'native-gitops')
self.assertFalse(plan['changed'])
self.assertEqual(plan['worker_pin'], self.pin)
self.assertEqual(remote.apply(plan), {'state': 'unchanged'})
self.assertEqual(finish.merge_ready(plan, {'state': 'unchanged'}), {'state': 'unchanged'})
def test_enforcement_retains_bound_native_gate(self):
plan = enforcement.inspect(self.repo, 'main', '0.8', self.request)
self.assertFalse(plan['changed'])
self.assertEqual(plan['gates'], gitops.gates())
enforcement.enforce(plan, self.request)
def test_wrong_app_and_failed_gate_reject(self):
for field, value in [('app', {'id': 18001}), ('conclusion', 'failure')]:
original = copy.deepcopy(self.runs)
self.runs[0][field] = value
with self.assertRaises(ValueError):
enforcement.inspect(self.repo, 'main', '0.8', self.request)
self.runs = original
def test_noncanonical_or_unsigned_or_changed_worker_rejects(self):
self.caller['jobs']['gitops']['uses'] = 'other/repo/workflow.yml@' + self.pin
with self.assertRaisesRegex(ValueError, 'canonical'):
gitops.validate(self.repo, self.head, 'main', self.request, yaml.safe_load)
self.caller = gitops.bridge.canonical('gitops', self.pin)
self.signed = False
with self.assertRaisesRegex(ValueError, 'not verified'):
gitops.validate(self.repo, self.head, 'main', self.request, yaml.safe_load)
self.signed, self.worker_drift = True, True
with self.assertRaisesRegex(ValueError, 'supported contract'):
gitops.validate(self.repo, self.head, 'main', self.request, yaml.safe_load)
def test_additional_orchestration_and_incomplete_inventory_reject(self):
for path in ['.circleci/config.yml', '.github/workflows/custom.yaml']:
self.paths.append(path)
with self.assertRaisesRegex(ValueError, 'additional orchestration'):
gitops.validate(self.repo, self.head, 'main', self.request, yaml.safe_load)
self.paths.pop()
self.truncated = True
with self.assertRaisesRegex(ValueError, 'complete source tree'):
gitops.validate(self.repo, self.head, 'main', self.request, yaml.safe_load)
def test_unknown_repository_and_develop_are_not_exempted(self):
for repo, branch in [('QRun-IO/unknown', 'main'), (self.repo, 'develop')]:
with self.assertRaises(ValueError):
gitops.validate(repo, self.head, branch, self.request, yaml.safe_load)
def test_native_pr_qualification(self):
with mock.patch.object(finish.remote, 'api', side_effect=self.request):
self.assertEqual(finish.qualification(self.repo, self.head, 'native-gitops'), [])
self.runs = []
self.assertEqual(finish.qualification(self.repo, self.head, 'native-gitops'), ['gitops / validate'])
if __name__ == '__main__':
unittest.main()
-108
View File
@@ -1,108 +0,0 @@
"""Recognize native ownership without duplicate CI or weaker PR protections."""
import base64
import copy
from pathlib import Path
import sys
import unittest
from unittest import mock
import yaml
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / 'tools'))
import enforce_orb_checks as enforcement
import finish_enrollment as finish
class NativeTests(unittest.TestCase):
def setUp(self):
self.pin = 'a' * 40
self.head = 'b' * 40
self.caller = enforcement.native.bridge.canonical_next(self.pin)
self.signed = True
self.worker_drift = False
self.runs = [{'id': i, 'name': gate['context'], 'app': {'id': 15368},
'status': 'completed', 'conclusion': 'success'}
for i, gate in enumerate(enforcement.native.gates())]
self.record = {'id': 1, 'enforcement': 'active', 'target': 'branch',
'source_type': 'Repository', 'source': 'QRun-IO/example', 'bypass_actors': [],
'rules': [{'type': 'required_status_checks', 'parameters': {
'strict_required_status_checks_policy': False,
'required_status_checks': [{'context': g['context'], 'integration_id': g['app_id']}
for g in enforcement.gates('frontend-next')]}}]}
def request(self, path, method='GET', payload=None):
self.assertEqual('GET', method, 'Recognition must not mutate correct protections')
if '/contents/' in path:
if '.github/workflows/ci.yml?' in path:
raw = yaml.safe_dump(self.caller)
elif 'repos/QRun-IO/qqq-orb/' in path:
name = path.split('/contents/')[1].split('?')[0]
raw = (ROOT / name).read_text(encoding='utf-8') + ('# drift' if self.worker_drift else '')
else:
data = ({'profile': 'frontend-next', 'orb_version': '0.8', 'default_branch': 'main'}
if '/.qqq-ci.yml?' in path else enforcement.standard.canonical_caller('frontend-next', '0.8'))
raw = yaml.safe_dump(data)
return {'type': 'file', 'encoding': 'base64', 'content': base64.b64encode(raw.encode()).decode()}
if path.endswith('/commits/' + self.pin):
return {'commit': {'verification': {'verified': self.signed}}}
if '/check-runs?' in path:
return {'check_runs': copy.deepcopy(self.runs)}
if '/status?' in path:
return {'statuses': []}
if '/rulesets/1' in path:
return copy.deepcopy(self.record)
if '/rules/branches/' in path:
return [{'type': 'required_status_checks', 'ruleset_source_type': 'Repository',
'ruleset_source': 'QRun-IO/example', 'ruleset_id': 1}]
if path.endswith('/protection'):
return None
if '/branches/' in path:
return {'commit': {'sha': self.head}}
return {'archived': False}
def test_main_recognition_retains_all_eight_pr_gates_without_image_run(self):
plan = enforcement.inspect('QRun-IO/example', 'main', '0.8', self.request)
self.assertFalse(plan['changed'])
self.assertEqual('rulesets', plan['protection_mode'])
self.assertEqual(enforcement.gates('frontend-next'), plan['required_status_checks']['checks'])
self.assertEqual(8, len(plan['gates']))
enforcement.enforce(plan, self.request)
def test_caller_drift_unsigned_and_changed_workers_fail_closed(self):
self.caller['jobs']['ci']['uses'] = 'untrusted/repo/ci.yml@' + self.pin
with self.assertRaisesRegex(ValueError, 'canonical'):
enforcement.inspect('QRun-IO/example', 'main', '0.8', self.request)
self.caller = enforcement.native.bridge.canonical_next(self.pin)
self.signed = False
with self.assertRaisesRegex(ValueError, 'not verified'):
enforcement.inspect('QRun-IO/example', 'main', '0.8', self.request)
self.signed = True
self.worker_drift = True
with self.assertRaisesRegex(ValueError, 'supported contract'):
enforcement.inspect('QRun-IO/example', 'main', '0.8', self.request)
def test_missing_main_gate_wrong_app_and_missing_pr_protection_reject(self):
self.runs[0]['app']['id'] = 18001
with self.assertRaises(ValueError):
enforcement.inspect('QRun-IO/example', 'main', '0.8', self.request)
self.runs[0]['app']['id'] = 15368
self.record['rules'][0]['parameters']['required_status_checks'].pop()
with self.assertRaisesRegex(ValueError, 'missing app-bound gate'):
enforcement.inspect('QRun-IO/example', 'main', '0.8', self.request)
def test_pr_qualification_requires_both_image_architectures(self):
with mock.patch.object(finish.remote, 'api', side_effect=self.request):
missing = finish.qualification('QRun-IO/example', self.head, 'frontend-next')
self.assertEqual(sorted(enforcement.native.PR_GATES[-2:]), missing)
for i, name in enumerate(enforcement.native.PR_GATES[-2:], 20):
self.runs.append({'id': i, 'name': name, 'app': {'id': 15368},
'status': 'completed', 'conclusion': 'success'})
self.assertEqual([], finish.qualification('QRun-IO/example', self.head, 'frontend-next'))
def test_develop_has_no_native_push_contract(self):
with self.assertRaisesRegex(ValueError, 'main only'):
enforcement.native.validate('QRun-IO/example', self.head, 'develop', self.request, enforcement.standard.load_yaml)
if __name__ == '__main__':
unittest.main()
-149
View File
@@ -1,149 +0,0 @@
"""Native recognition proves central ownership and app-bound verification gates."""
import base64
import copy
from pathlib import Path
import sys
import unittest
from unittest import mock
import yaml
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / 'tools'))
import native_specialist_gates as native
import reconcile_enrollment as remote
import enforce_orb_checks as enforcement
import finish_enrollment as finish
class NativeTests(unittest.TestCase):
def fixture(self, repo):
self.repo, self.pin, self.head = repo, 'a' * 40, 'b' * 40
self.profile, self.branches, kinds, _ = native.DEFINITIONS[repo]
self.callers = dict(native.caller(kind, self.pin) for kind in kinds)
self.paths = list(self.callers)
self.signed, self.drift, self.truncated = True, False, False
self.mode = '100644'
self.runs = [{'id': i, 'name': gate['context'], 'app': {'id': gate['app_id']},
'status': 'completed', 'conclusion': 'success'}
for i, gate in enumerate(native.gates(self.profile))]
self.before = {'required_status_checks': {'strict': True, 'checks': native.gates(self.profile)},
'enforce_admins': {'enabled': True}}
def request(self, path, method='GET', payload=None):
self.assertEqual('GET', method)
if '/contents/' in path:
self.assertNotIn('.circleci/', path)
file = path.split('/contents/')[1].split('?')[0]
if path.startswith('repos/QRun-IO/qqq-orb/'):
raw = (ROOT / file).read_text() + ('# changed' if self.drift else '')
else:
raw = yaml.safe_dump(self.callers[file], sort_keys=False)
return {'type': 'file', 'encoding': 'base64', 'content': base64.b64encode(raw.encode()).decode()}
if '/git/trees/' in path:
return {'truncated': self.truncated, 'tree': [{'path': p, 'type': 'blob', 'mode': self.mode} for p in self.paths]}
if path.endswith('/commits/' + self.pin):
return {'commit': {'verification': {'verified': self.signed}}}
if '/check-runs?' in path:
return {'check_runs': copy.deepcopy(self.runs)}
if '/status?' in path:
return {'statuses': []}
if path.endswith('/protection'):
return copy.deepcopy(self.before)
if '/rules/branches/' in path:
return []
if '/branches/' in path:
return {'commit': {'sha': self.head}}
return {'archived': False, 'default_branch': 'main'}
def test_all_approved_callers_discover_enforce_and_qualify_without_mutation(self):
for repo in native.DEFINITIONS:
self.fixture(repo)
for branch in self.branches:
with self.subTest(repo=repo, branch=branch), mock.patch.object(remote, 'api', side_effect=self.request):
plan = remote.inspect({'repository': repo, 'branch': branch}, '0.8')
self.assertEqual(self.profile, plan['profile'])
self.assertFalse(plan['changed'])
self.assertEqual(set(self.callers), set(plan['worker_pins']))
self.assertEqual({'state': 'unchanged'}, remote.apply(plan))
self.assertEqual({'state': 'unchanged'}, finish.merge_ready(plan, {'state': 'unchanged'}))
self.assertEqual([], finish.qualification(repo, self.head, self.profile))
protected = enforcement.inspect(repo, branch, '0.8', self.request)
self.assertFalse(protected['changed'])
enforcement.enforce(protected, self.request)
def test_failed_missing_wrong_app_and_newer_failed_gate_reject(self):
for repo in native.DEFINITIONS:
self.fixture(repo)
good = copy.deepcopy(self.runs)
for bad in [[], good[1:], [dict(x, app={'id': 18001}) for x in good],
good + [dict(good[0], id=99, conclusion='failure')]]:
self.runs = bad
with self.subTest(repo=repo), self.assertRaises(ValueError):
enforcement.inspect(repo, 'main', '0.8', self.request)
def test_custom_caller_unsigned_or_changed_worker_rejects(self):
for repo in native.DEFINITIONS:
for defect in ('caller', 'unsigned', 'worker', 'mutable'):
self.fixture(repo)
if defect == 'caller':
next(iter(self.callers.values()))['extra'] = True
elif defect == 'unsigned':
self.signed = False
elif defect == 'worker':
self.drift = True
else:
jobs = next(iter(self.callers.values()))['jobs']
job = next(iter(jobs.values()))
job['uses'] = job['uses'].replace(self.pin, 'main')
with self.subTest(repo=repo, defect=defect), self.assertRaises(ValueError):
native.validate(repo, self.head, 'main', self.request)
def test_complete_workflow_inventory_required(self):
self.fixture('QRun-IO/qqq-business-platform')
for path in ('.circleci/config.yml', '.github/workflows/custom.yml', '.github/workflows/extra.yaml'):
self.paths.append(path)
with self.assertRaisesRegex(ValueError, 'additional orchestration'):
native.validate(self.repo, self.head, 'main', self.request)
self.paths.pop()
self.truncated = True
with self.assertRaisesRegex(ValueError, 'complete source tree'):
native.validate(self.repo, self.head, 'main', self.request)
def test_symlinked_or_missing_caller_rejects(self):
self.fixture('QRun-IO/qctl')
self.mode = '120000'
with self.assertRaisesRegex(ValueError, 'regular source file'):
native.validate(self.repo, self.head, 'main', self.request)
self.paths = []
with self.assertRaisesRegex(ValueError, 'regular source file'):
native.validate(self.repo, self.head, 'main', self.request)
def test_unknown_repo_and_unapproved_branch_reject(self):
self.fixture('QRun-IO/qqq-mcp-rag')
for repo, branch in [('QRun-IO/unknown', 'main'), (self.repo, 'develop'), ('QRun-IO/qctl', 'release/old')]:
with self.assertRaises(ValueError):
native.validate(repo, self.head, branch, self.request)
def test_qctl_pr_gate_excludes_push_only_native_binary_job(self):
worker = native.load_workflow((ROOT / '.github/workflows/qctl-native.yml').read_text())
self.assertNotIn('if', worker['jobs']['test'])
self.assertEqual("github.event_name == 'push' || (github.event_name == 'workflow_dispatch' && inputs.full_native)",
worker['jobs']['build-native']['if'])
self.assertEqual([{'context': 'native / test', 'app_id': 15368}], native.gates('qctl'))
def test_github_yaml_events_and_duplicate_keys(self):
self.assertEqual({'on': {'push': None}, 'enabled': True},
native.load_workflow('on:\n push:\nenabled: true\n'))
for raw in ('on: {}\non: {}\n', 'jobs: {build: {}, build: {}}\n', '1: value\n'):
with self.subTest(raw=raw), self.assertRaises(ValueError):
native.load_workflow(raw)
def test_native_enforcement_preserves_other_required_checks(self):
self.fixture('QRun-IO/qctl')
self.before['required_status_checks']['checks'].append({'context': 'security', 'app_id': 15368})
plan = enforcement.inspect(self.repo, 'main', '0.8', self.request)
self.assertIn({'context': 'security', 'app_id': 15368}, plan['required_status_checks']['checks'])
if __name__ == '__main__':
unittest.main()
-34
View File
@@ -76,7 +76,6 @@ sys.exit(subprocess.run([os.environ['TEST_REAL_NODE'], *sys.argv[1:]], input=sou
def test_tags_and_legacy_authentication(self):
for trusted in [True, False]:
for version, tag in [('1.0.0-RC.1', 'rc'), ('1.0.0-SNAPSHOT', 'snapshot'),
('1.0.0-SNAPSHOT.101', 'snapshot'),
('1.0.0-alpha.1', 'next'), ('1.0.0-beta.1', 'next'), ('1.0.0', 'latest')]:
with self.subTest(trusted=trusted, version=version):
result, args, oidc = self.run_publish(version, trusted)
@@ -134,38 +133,5 @@ sys.exit(subprocess.run([os.environ['TEST_REAL_NODE'], *sys.argv[1:]], input=sou
self.assertNotIn('Successfully published', result.stdout)
class SnapshotVersionTests(unittest.TestCase):
def test_snapshot_jobs_keep_source_head_and_publish_distinct_versions(self):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
def git(*args):
return subprocess.check_output(['git', *args], cwd=root, text=True).strip()
git('init', '-q', '-b', 'develop')
source = '{"name": "snapshot-test", "version": "0.40.27-SNAPSHOT"}\n'
package = root / 'package.json'
package.write_text(source)
git('add', 'package.json')
git('-c', 'user.name=Test', '-c', 'user.email=test@example.invalid',
'-c', 'commit.gpgsign=false', 'commit', '-qm', 'initial source')
head = git('rev-parse', 'HEAD')
for number in ('101', '102'):
package.write_text(source)
env = dict(os.environ, CIRCLE_BUILD_NUM=number, BRANCH_TYPE='snapshot')
subprocess.run(['bash', str(SCRIPT.parent / 'sync-npm-version.sh')],
cwd=root, env=env, check=True, capture_output=True)
self.assertEqual(json.loads(package.read_text())['version'], '0.40.28-SNAPSHOT.' + number)
subprocess.run(['bash', str(SCRIPT.parent / 'node_version_commit.sh')],
cwd=root, env=env, check=True, capture_output=True)
self.assertEqual(git('rev-parse', 'HEAD'), head)
self.assertEqual(git('diff', '--cached', '--name-only'), '')
for number in ('', '0', '01', '-1', 'abc'):
package.write_text(source)
result = subprocess.run(['bash', str(SCRIPT.parent / 'sync-npm-version.sh')],
cwd=root, env=dict(os.environ, CIRCLE_BUILD_NUM=number),
capture_output=True)
self.assertNotEqual(result.returncode, 0)
self.assertEqual(package.read_text(), source)
if __name__ == '__main__':
unittest.main()
-58
View File
@@ -1,58 +0,0 @@
import json
import os
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
import yaml
ROOT = Path(__file__).parents[1]
class BranchLanguages(unittest.TestCase):
def select(self, tracked, untracked=()):
worker = yaml.safe_load((ROOT / '.github/workflows/qbit-codeql.yml').read_text())
script = next(s['run'] for s in worker['jobs']['analyze']['steps'] if s.get('id') == 'languages')
script = script.split("python3 <<'PY'\n", 1)[1].rsplit('\nPY', 1)[0]
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
subprocess.run(['git', 'init', '-q', directory], check=True)
for name in (*tracked, *untracked):
path = root / name
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text('fixture')
if tracked:
subprocess.run(['git', 'add', '--', *tracked], cwd=root, check=True)
output = root / 'outputs'
result = subprocess.run([sys.executable, '-c', script], cwd=root,
env={**os.environ, 'GITHUB_OUTPUT': str(output)}, capture_output=True, text=True)
return result, output.read_text() if output.exists() else ''
def test_historical_java_branch_does_not_scan_absent_actions(self):
result, output = self.select(['src/Example.java'])
self.assertEqual(0, result.returncode, result.stderr)
self.assertIn('languages=java-kotlin\n', output)
self.assertIn('build-mode=none\n', output)
def test_current_branch_preserves_all_present_supported_languages(self):
result, output = self.select(['src/Example.java', '.github/workflows/ci.yaml', 'web/app.tsx', 'tools/check.py'])
self.assertEqual(0, result.returncode, result.stderr)
self.assertIn('languages=java-kotlin,javascript-typescript,python,actions\n', output)
def test_kotlin_requires_a_build(self):
result, output = self.select(['src/Example.kt', '.github/workflows/ci.yml'])
self.assertEqual(0, result.returncode, result.stderr)
self.assertIn('build-mode=autobuild\n', output)
def test_untracked_files_do_not_select_languages(self):
result, output = self.select(['src/Example.java'], ['web/generated.js', '.github/workflows/untracked.yml'])
self.assertEqual(0, result.returncode, result.stderr)
self.assertIn('languages=java-kotlin\n', output)
def test_placeholder_fails_explicitly(self):
result, output = self.select(['README.md'])
self.assertNotEqual(0, result.returncode)
self.assertIn('No supported source', result.stderr)
if __name__ == '__main__':
unittest.main()
-73
View File
@@ -1,73 +0,0 @@
"""Preserve Workflows' audited publish-tag contract without broadening branches."""
import unittest
from unittest import mock
from pathlib import Path
import yaml
from test_qbit_hotfix_profile import load, migration, pipeline, selects, ROOT
standard = load('standard', 'src/scripts/validate_standard.py')
class FeaturePublicationTests(unittest.TestCase):
def config(self, manifest):
with mock.patch.object(Path, 'read_text', return_value=yaml.safe_dump(manifest)):
return pipeline.load_manifest('unused', '0.7.16')
def test_audited_event_selection_matches_legacy(self):
legacy = yaml.safe_load((ROOT / 'tests/fixtures/qbit-feature-publication.yml').read_text())
manifest = migration.infer_manifest(legacy, '0.7.16', 'main')
self.assertTrue(manifest['publish']['feature'])
caller = migration.canonical_caller('0.7.16', manifest['release_tag_policy'], True)
standard.validate(yaml.safe_dump(caller), yaml.safe_dump(manifest), '0.7.16')
setup = caller['workflows']['setup']['jobs'][0]['qqq-orb/generate_pipeline']
cases = [(branch, '') for branch in ['main', 'develop', 'feature/example', 'release/0.3.0', 'hotfix/0.3.1', 'integration-test']]
cases += [('', tag) for tag in ['publish', 'publish-example', 'v1.2.3', 'v1.2.3-RC.1', 'other']]
for branch, tag in cases:
with self.subTest(branch=branch, tag=tag):
expected = sorted((job, args.get('branch_type')) for workflow in legacy['workflows'].values()
for entry in workflow['jobs'] for job, args in entry.items()
if selects(args.get('filters', {}), branch, tag))
with mock.patch.object(pipeline, 'git', return_value=b'verified-head\n'):
rendered = pipeline.qbit_template(self.config(manifest), branch, tag)
actual = sorted((job, args.get('branch_type')) for workflow in rendered['workflows'].values()
for entry in workflow['jobs'] for job, args in entry.items()
if selects(args.get('filters', {}), branch, tag))
if not selects(setup['filters'], branch, tag):
actual = []
self.assertEqual(expected, actual)
def test_feature_is_opt_in_and_requires_exact_tag_head(self):
manifest = {'profile': 'qbit', 'orb_version': '0.7.16', 'default_branch': 'main'}
self.assertFalse(self.config(manifest)['publish']['feature'])
manifest['publish'] = {'feature': True}
for tagged in [None, b'wrong-head\n']:
def git(*args):
return b'head\n' if args == ('rev-parse', 'HEAD') else tagged
with self.subTest(tagged=tagged), mock.patch.object(pipeline, 'git', side_effect=git):
rendered = pipeline.qbit_template(self.config(manifest), '', 'publish-test')
self.assertIn('qqq-orb/mvn_test_only', rendered['workflows']['test_only']['jobs'][0])
def test_feature_tag_recovery_checks_fetched_commit(self):
manifest = {'profile': 'qbit', 'orb_version': '0.7.16', 'default_branch': 'main', 'publish': {'feature': True}}
with mock.patch.object(pipeline, 'git', side_effect=[None, b'', b'head\n', b'head\n']):
rendered = pipeline.qbit_template(self.config(manifest), '', 'publish-test')
self.assertEqual('feature', rendered['workflows']['publish_feature']['jobs'][0]['qqq-orb/mvn_publish']['branch_type'])
def test_setup_preserves_release_policy_and_rejects_drift(self):
for policy in ['stable', 'v-prefix', 'none']:
manifest = {'profile': 'qbit', 'orb_version': '0.7.16', 'default_branch': 'main',
'release_tag_policy': policy, 'publish': {'feature': True}}
caller = migration.canonical_caller('0.7.16', policy, True)
standard.validate(yaml.safe_dump(caller), yaml.safe_dump(manifest), '0.7.16')
self.assertEqual(caller, standard.canonical_caller('qbit', '0.7.16', release_tag_policy=policy, publish_feature=True))
with self.assertRaises(standard.StandardError):
standard.validate(yaml.safe_dump(migration.canonical_caller('0.7.16', policy)), yaml.safe_dump(manifest), '0.7.16')
def test_unrecognized_feature_branch_filters_are_rejected(self):
for branches in [{'only': ['/feature/.*/']}, {'only': ['main']}, {'ignore': '/.*/'}]:
with self.subTest(branches=branches), self.assertRaises(ValueError):
migration.publication_filters('feature', {'branches': branches, 'tags': {'only': ['/publish.*/']}})
if __name__ == '__main__':
unittest.main()
-73
View File
@@ -1,73 +0,0 @@
"""Preserve the audited main-branch hotfix contracts in the shared profile."""
import importlib.util
import json
from pathlib import Path
import re
import unittest
from unittest import mock
import yaml
ROOT = Path(__file__).resolve().parents[1]
def load(name, path):
spec = importlib.util.spec_from_file_location(name, ROOT / path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
migration = load('migration', 'tools/standardize_qbit.py')
pipeline = load('pipeline', 'src/scripts/generate_pipeline.py')
def selects(filters, branch, tag):
selector = filters.get('tags' if tag else 'branches', {})
if tag and not selector:
return False
def matches(patterns):
patterns = patterns if isinstance(patterns, list) else [patterns]
return any(re.fullmatch(p[1:-1], tag or branch, flags=re.ASCII) is not None if p.startswith('/') and p.endswith('/') else p == (tag or branch) for p in patterns)
return ('only' not in selector or matches(selector['only'])) and ('ignore' not in selector or not matches(selector['ignore']))
class HotfixMigrationTests(unittest.TestCase):
def test_audited_main_job_selection_is_preserved(self):
fixtures = json.loads((ROOT/'tests/fixtures/qbit-main-hotfix-policies.json').read_text())
for fixture in fixtures:
manifest = migration.infer_manifest(fixture['config'], '0.7.13', 'develop')
with mock.patch.object(Path, 'read_text', return_value=yaml.safe_dump(manifest)):
config = pipeline.load_manifest('unused', '0.7.13')
cases = [(branch, '') for branch in ('main','develop','feature/a','release/1','hotfix/1.2.3','integration-test','staging')]
cases += [('', tag) for tag in ('v1.2.3','v1.2.3-RC.1','vanything','publish1','invalid','v\u0661.\u0662.\u0663')]
for branch, tag in cases:
with self.subTest(repository=fixture['repository'], branch=branch, tag=tag):
expected = sorted((job, options.get('branch_type')) for workflow in fixture['config']['workflows'].values() for entry in workflow['jobs'] for job, options in entry.items() if selects(options.get('filters', {}), branch, tag))
with mock.patch.object(pipeline, 'git', return_value=b'verified-head\n'):
generated = pipeline.qbit_template(config, branch, tag)
actual = sorted((job, options.get('branch_type')) for workflow in generated['workflows'].values() for entry in workflow['jobs'] for job, options in entry.items() if selects(options.get('filters', {}), branch, tag))
# The canonical setup's tag filter prevents non-release tag runs.
caller = migration.canonical_caller('0.7.13', manifest['release_tag_policy'])
setup = caller['workflows']['setup']['jobs'][0]['qqq-orb/generate_pipeline']
if not selects(setup['filters'], branch, tag): actual = []
self.assertEqual(actual, expected)
def test_cms_repair_only_quotes_the_existing_tag_pattern(self):
fixtures = json.loads((ROOT/'tests/fixtures/qbit-main-hotfix-policies.json').read_text())
fixture = next(row for row in fixtures if row['repository'] == 'QRun-IO/qbit-cms')
with self.assertRaises(yaml.YAMLError):
yaml.safe_load(fixture['original_yaml'])
repaired = fixture['original_yaml'].replace(r'only: [/^v[0-9]+\.[0-9]+\.[0-9]+$/]',
r"only: ['/^v[0-9]+\.[0-9]+\.[0-9]+$/']")
self.assertEqual(yaml.safe_load(repaired), fixture['config'])
def test_stable_tag_selection_uses_ascii_digits(self):
with mock.patch.object(pipeline, 'git', return_value=b'verified-head\n'):
self.assertEqual(pipeline.event_kind('', 'v\u0661.\u0662.\u0663'), 'test')
self.assertEqual(pipeline.event_kind('', 'v1.2.3'), 'release')
def test_existing_manifest_does_not_enable_hotfix_publication(self):
raw = 'profile: qbit\norb_version: 0.7.13\ndefault_branch: develop\n'
with mock.patch.object(Path, 'read_text', return_value=raw):
config = pipeline.load_manifest('unused','0.7.13')
self.assertFalse(config['publish'].get('hotfix', False))
result = pipeline.qbit_template(config, 'hotfix/1.2.3', '')
self.assertIn('qqq-orb/mvn_test_only', result['workflows']['test_only']['jobs'][0])
if __name__ == '__main__': unittest.main()
-11
View File
@@ -58,17 +58,6 @@ class FixtureAPI:
class SweepTests(unittest.TestCase):
def test_stable_gate_and_mixed_name_latest_failure(self):
api = FixtureAPI()
api.status['check_runs'][0]['name'] = 'verify'
self.assertEqual('dispatched', sweep.reconcile(api, REPO))
api.sent.clear()
api.status['check_runs'].append(dict(api.status['check_runs'][0],
id=10, name='publish_release_candidate', conclusion='failure'))
with self.assertRaises(sweep.InvalidIntent):
sweep.reconcile(api, REPO)
self.assertEqual([], api.sent)
def test_idle_never_dispatches_or_reads_artifacts(self):
api = FixtureAPI(); api.lock = None
self.assertEqual('idle', sweep.reconcile(api, REPO))
-11
View File
@@ -23,17 +23,6 @@ def bump(parent=SHA, version='1.2.3-RC.4'):
class ReleaseGuardTests(unittest.TestCase):
def test_stable_workflow_preserves_exact_source_and_latest_attempt_checks(self):
result = status()
result['check_runs'][0]['name'] = 'verify'
self.assertEqual('1.2.3-RC.4', evaluate(SHA, result, bump()))
result['check_runs'].append(dict(result['check_runs'][0], id=10, conclusion='failure'))
with self.assertRaisesRegex(Failure, 'CircleCI failed'):
evaluate(SHA, result, bump())
result['check_runs'] = [dict(result['check_runs'][0], head_sha=OTHER)]
with self.assertRaises(Failure):
evaluate(SHA, result, bump())
def test_success_exact_built_source(self):
self.assertEqual('1.2.3-RC.4', evaluate(SHA, status(), bump()))
-9
View File
@@ -30,15 +30,6 @@ def validate(job, publisher):
class TimeoutRecoveryTests(unittest.TestCase):
def test_stable_gate_must_match_the_actual_publishing_workflow(self):
job, publisher = evidence()
job['workflows']['workflow_name'] = 'verify'
publisher['name'] = 'verify'
validate(job, publisher)
publisher['name'] = 'publish_release_candidate'
with self.assertRaises(Failure):
validate(job, publisher)
def test_exact_upload_timeout_after_successful_verify(self):
self.assertEqual((124, 0), validate(*evidence()))
-185
View File
@@ -1,185 +0,0 @@
"""Remote enrollment keeps immutable source checks and resumes signed submissions."""
import base64
from pathlib import Path
import sys
import unittest
from unittest import mock
import yaml
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'tools'))
import reconcile_enrollment as remote
class ReconcileTests(unittest.TestCase):
def setUp(self):
self.head = 'a' * 40
self.new = 'b' * 40
self.repo = 'QRun-IO/example'
self.target = {'repository': self.repo, 'branch': 'main'}
self.manifest = {'profile': 'qbit', 'orb_version': '0.7.4', 'default_branch': 'main',
'publish': {'snapshot': False, 'release_candidate': False, 'release': False}}
self.sources = {
'.qqq-ci.yml': yaml.safe_dump(self.manifest).encode(),
'.circleci/config.yml': yaml.safe_dump(remote.enroll_consumers.rollout.standard.canonical_caller('qbit', '0.7.4')).encode()}
def request(self, path, missing=False):
if path == 'repos/' + self.repo:
return {'archived': False, 'default_branch': 'main'}
if path.endswith('/branches/main'):
return {'commit': {'sha': self.head}}
if '/contents/' in path:
self.assertTrue(path.endswith('?ref=' + self.head))
name = path.split('/contents/')[1].split('?')[0]
raw = self.sources.get(name)
return None if raw is None else {'type': 'file', 'encoding': 'base64', 'content': base64.b64encode(raw).decode()}
raise AssertionError(path)
def plan(self):
with mock.patch.object(remote, 'api', side_effect=self.request):
return remote.inspect(self.target, '0.8')
def test_immutable_plan_is_deterministic_and_preserves_policy(self):
first, second = self.plan(), self.plan()
self.assertEqual(first['automation_branch'], second['automation_branch'])
self.assertEqual(self.sources, first['originals'])
self.assertEqual({**self.manifest, 'orb_version': '0.8'},
yaml.safe_load(first['replacements']['.qqq-ci.yml']))
self.assertTrue(first['changed'])
def test_missing_manifest_uses_legacy_converter(self):
self.sources.pop('.qqq-ci.yml')
self.sources['.circleci/config.yml'] = yaml.safe_dump({'version': 2.1,
'orbs': {'qqq-orb': 'kingsrook/qqq-orb@0.5.0'},
'workflows': {'verify': {'jobs': [{'qqq-orb/mvn_test_only': {}}]}}}).encode()
plan = self.plan()
self.assertIsNone(plan['originals']['.qqq-ci.yml'])
self.assertFalse(any(yaml.safe_load(plan['replacements']['.qqq-ci.yml'])['publish'].values()))
def submission_api(self, plan, path, missing=False):
if '/branches/main' in path:
return {'commit': {'sha': self.head}}
if '/git/ref/' in path:
return {'object': {'sha': self.new}}
if '/commits/' in path:
return {'commit': {'verification': {'verified': True}},
'parents': [{'sha': self.head}], 'files': [{'filename': p} for p in remote.PATHS]}
if '/contents/' in path:
self.assertTrue(path.endswith('?ref=' + self.new))
name = path.split('/contents/')[1].split('?')[0]
return {'type': 'file', 'encoding': 'base64', 'content': base64.b64encode(plan['replacements'][name]).decode()}
if '/pulls?' in path:
return [{'head': {'sha': self.new}, 'html_url': 'https://github.com/QRun-IO/example/pull/1'}]
raise AssertionError(path)
def test_matching_existing_submission_uses_no_git_or_pr_mutations(self):
plan = self.plan()
with mock.patch.object(remote, 'api', side_effect=lambda *a, **k: self.submission_api(plan, *a, **k)), mock.patch.object(remote, 'command') as command:
result = remote.apply(plan)
self.assertEqual('existing-pr', result['state'])
command.assert_not_called()
def test_unsigned_existing_branch_is_rejected_without_mutation(self):
plan = self.plan()
def request(path, missing=False):
value = self.submission_api(plan, path, missing)
if '/commits/' in path:
value['commit']['verification']['verified'] = False
return value
with mock.patch.object(remote, 'api', side_effect=request), mock.patch.object(remote, 'command') as command:
with self.assertRaises(ValueError):
remote.apply(plan)
command.assert_not_called()
def test_source_drift_prevents_submission(self):
plan = self.plan()
with mock.patch.object(remote, 'api', return_value={'commit': {'sha': 'c' * 40}}), mock.patch.object(remote, 'command') as command:
with self.assertRaises(ValueError):
remote.apply(plan)
command.assert_not_called()
def test_interrupted_push_creates_only_missing_pr(self):
plan = self.plan()
seen = []
def request(path, missing=False):
return [] if '/pulls?' in path else self.submission_api(plan, path, missing)
def command(args, cwd=None):
seen.append(args[:3])
self.assertEqual(['gh', 'pr', 'create'], args[:3])
self.assertIn(self.head, Path(args[args.index('--body-file') + 1]).read_text())
return 'https://github.com/QRun-IO/example/pull/2'
with mock.patch.object(remote, 'api', side_effect=request), mock.patch.object(remote, 'command', side_effect=command):
result = remote.apply(plan)
self.assertEqual('submitted', result['state'])
self.assertEqual([['gh', 'pr', 'create']], seen)
def test_new_api_submission_is_atomic_and_exact(self):
plan = self.plan()
mutations = []
def request(path, missing=False):
if '/git/ref/' in path:
return None
return self.submission_api(plan, path, missing)
def mutate(path, payload):
mutations.append((path, payload))
return {'data': {'createCommitOnBranch': {'commit': {'oid': self.new}}}}
with mock.patch.object(remote, 'api', side_effect=request), mock.patch.object(remote, 'mutation', side_effect=mutate), mock.patch.object(remote, 'command') as command:
self.assertEqual('existing-pr', remote.apply(plan)['state'])
command.assert_not_called()
self.assertEqual(('repos/' + self.repo + '/git/refs', {'ref': 'refs/heads/' + plan['automation_branch'], 'sha': self.head}), mutations[0])
commit = mutations[1][1]['variables']['input']
self.assertEqual(self.head, commit['expectedHeadOid'])
self.assertEqual({'repositoryNameWithOwner': self.repo, 'branchName': plan['automation_branch']}, commit['branch'])
self.assertEqual(plan['replacements'], {item['path']: base64.b64decode(item['contents']) for item in commit['fileChanges']['additions']})
def test_base_only_branch_resumes_without_creating_another_ref(self):
plan = self.plan()
result = {'data': {'createCommitOnBranch': {'commit': {'oid': self.new}}}}
with mock.patch.object(remote, 'api', return_value={'commit': {'sha': self.head}}), mock.patch.object(remote, 'mutation', return_value=result) as mutation:
self.assertEqual(self.new, remote.signed_api_commit(plan, {'object': {'sha': self.head}}))
self.assertEqual(1, mutation.call_count)
self.assertEqual('graphql', mutation.call_args.args[0])
def test_existing_completed_branch_is_not_mutated(self):
with mock.patch.object(remote, 'mutation') as mutation:
self.assertEqual(self.new, remote.signed_api_commit(self.plan(), {'object': {'sha': self.new}}))
mutation.assert_not_called()
def test_target_drift_prevents_commit_on_base_only_branch(self):
plan = self.plan()
with mock.patch.object(remote, 'api', return_value={'commit': {'sha': 'c' * 40}}), mock.patch.object(remote, 'mutation') as mutation:
with self.assertRaises(ValueError):
remote.signed_api_commit(plan, {'object': {'sha': self.head}})
mutation.assert_not_called()
def test_graphql_errors_fail_closed(self):
response = mock.Mock(returncode=0, stdout='{"errors": [{"message": "conflict"}]}')
with mock.patch.object(remote.subprocess, 'run', return_value=response):
with self.assertRaises(ValueError):
remote.mutation('graphql', {})
def test_unsigned_api_commit_cannot_create_pr(self):
plan = self.plan()
def request(path, missing=False):
if '/git/ref/' in path:
return {'object': {'sha': self.head}}
value = self.submission_api(plan, path, missing)
if '/commits/' in path:
value['commit']['verification']['verified'] = False
return value
with mock.patch.object(remote, 'api', side_effect=request), mock.patch.object(remote, 'mutation', return_value={'data': {'createCommitOnBranch': {'commit': {'oid': self.new}}}}), mock.patch.object(remote, 'command') as command:
with self.assertRaises(ValueError):
remote.apply(plan)
command.assert_not_called()
if __name__ == '__main__':
unittest.main()
-119
View File
@@ -1,119 +0,0 @@
"""Execute the policy job against real git indexes, including application promotion."""
import importlib.util
import json
import os
import shutil
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
from unittest.mock import patch
import yaml
ROOT = Path(__file__).resolve().parents[1]
spec = importlib.util.spec_from_file_location('pipeline', ROOT / 'src/scripts/generate_pipeline.py')
pipeline = importlib.util.module_from_spec(spec)
spec.loader.exec_module(pipeline)
TEMPLATE = json.loads((ROOT / 'src/scripts/templates/repository-policy.json').read_text())
SCRIPT = TEMPLATE['jobs']['validate-policy']['steps'][-1]['run']['command'].split("<<'PYTHON'\n", 1)[1].rsplit('PYTHON', 1)[0]
class RepositoryPolicyTests(unittest.TestCase):
def test_packing_is_independent_of_checkout_line_endings(self):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
shutil.copytree(ROOT / 'src/scripts', root / 'src/scripts')
(root / 'scripts').mkdir()
shutil.copy(ROOT / 'scripts/pack_pipeline.py', root / 'scripts/pack_pipeline.py')
resources = list((root / 'src/scripts/templates').glob('*.json'))
for resource in resources:
resource.write_bytes(resource.read_bytes().replace(b'\r\n', b'\n'))
subprocess.run([sys.executable, str(root / 'scripts/pack_pipeline.py')], check=True)
packed = (root / 'src/scripts/packed_generate_pipeline.py').read_bytes()
for resource in resources:
resource.write_bytes(resource.read_bytes().replace(b'\n', b'\r\n'))
subprocess.run([sys.executable, str(root / 'scripts/pack_pipeline.py'), '--check'], check=True)
self.assertEqual(packed, (root / 'src/scripts/packed_generate_pipeline.py').read_bytes())
def execute(self, files, untracked=None, shell_checks=False):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
subprocess.run(['git', 'init', '-q'], cwd=root, check=True)
for name, content in files.items():
file = root / name
file.parent.mkdir(parents=True, exist_ok=True)
file.write_text(content, encoding='utf-8')
subprocess.run(['git', 'add', '.'], cwd=root, check=True)
for name, content in (untracked or {}).items():
(root / name).write_text(content, encoding='utf-8')
env = dict(os.environ, QQQ_POLICY_SHELL='true' if shell_checks else 'false')
return subprocess.run([sys.executable, '-c', SCRIPT], cwd=root, env=env, capture_output=True, text=True)
def test_placeholder_with_wrappers_and_workflows(self):
result = self.execute({'LICENSE': 'Apache-2.0', 'docs/DESIGN.md': '# Planned application',
'.github/workflows/release-rc.yml': 'name: Release\non: workflow_dispatch\n',
'.mvn/wrapper/maven-wrapper.properties': 'distributionUrl=https://example.invalid/maven.zip',
'mvnw': '#!/bin/sh\n', 'mvnw.cmd': '@echo off\n'})
self.assertEqual(0, result.returncode, result.stderr)
def test_new_application_requires_real_build_profile(self):
for name in ('pom.xml', 'package.json', 'src/App.java', 'main.py', 'main.go', 'Dockerfile'):
with self.subTest(name=name):
self.assertNotEqual(0, self.execute({'LICENSE': 'license', name: '{}'}).returncode)
def test_malformed_configuration_fails(self):
for name in ('.github/dependabot.yml', 'settings.json'):
with self.subTest(name=name):
self.assertNotEqual(0, self.execute({name: '[unterminated'}).returncode)
def test_unused_checkstyle_policy_is_validated_without_allowing_application_xml(self):
self.assertEqual(0, self.execute({'checkstyle/config.xml': '<module name="Checker"/>'}).returncode)
self.assertNotEqual(0, self.execute({'checkstyle/config.xml': '<module>'}).returncode)
self.assertNotEqual(0, self.execute({'application.xml': '<application/>'}).returncode)
self.assertNotEqual(0, self.execute({'pom.xml': '<project/>'}).returncode)
def test_only_tracked_files_are_policy_inputs(self):
self.assertEqual(0, self.execute({'README.md': '# Docs'}, {'main.py': 'temporary tool'}).returncode)
def test_shell_validation_is_opt_in_and_never_executes_scripts(self):
files = {'automation/safe.sh': '#!/bin/bash\nexit 42\n', 'diagram.mmd': 'flowchart LR\n A-->B\n'}
self.assertNotEqual(0, self.execute(files).returncode)
result = self.execute(files, shell_checks=True)
self.assertEqual(0, result.returncode, result.stderr)
def test_shell_validation_rejects_syntax_warnings_and_application_sources(self):
for name, content in [('broken.sh', '#!/bin/bash\nif then\n'),
('unsafe.sh', '#!/bin/bash\necho $unquoted\n'),
('main.py', 'print(42)'), ('package.json', '{}')]:
with self.subTest(name=name):
self.assertNotEqual(0, self.execute({name: content}, shell_checks=True).returncode)
def test_shell_option_is_strict_and_keeps_single_small_nonpublishing_job(self):
manifest = {'profile': 'repository-policy', 'orb_version': '0.8',
'default_branch': 'main', 'shell_checks': True}
with patch.object(Path, 'read_text', return_value=yaml.safe_dump(manifest)):
config = pipeline.load_manifest('unused', '0.8')
result = pipeline.repository_policy_template(config, 'main', '')
self.assertFalse(any(config['publish'].values()))
self.assertEqual({'validate-policy'}, set(result['jobs']))
self.assertEqual('small', result['jobs']['validate-policy']['resource_class'])
self.assertEqual({'QQQ_POLICY_SHELL': 'true'}, result['jobs']['validate-policy']['environment'])
for overrides in ({'shell_checks': 'true'}, {'profile': 'qbit'}):
with patch.object(Path, 'read_text', return_value=yaml.safe_dump(dict(manifest, **overrides))):
with self.assertRaises(pipeline.ManifestError):
pipeline.load_manifest('unused', '0.8')
def test_manifest_cannot_enable_publication(self):
manifest = {'profile': 'repository-policy', 'orb_version': '0.7.14', 'default_branch': 'develop'}
with patch.object(Path, 'read_text', return_value=yaml.safe_dump(manifest)):
config = pipeline.load_manifest('unused', '0.7.14')
self.assertFalse(any(config['publish'].values()))
self.assertEqual({'verify': {'jobs': ['validate-policy']}}, pipeline.repository_policy_template(config, 'main', '')['workflows'])
manifest['publish'] = {'release': True}
with patch.object(Path, 'read_text', return_value=yaml.safe_dump(manifest)), self.assertRaises(pipeline.ManifestError):
pipeline.load_manifest('unused', '0.7.14')
if __name__ == '__main__':
unittest.main()
+1 -30
View File
@@ -20,11 +20,6 @@ class RolloutTests(unittest.TestCase):
def checkout(self, name, profile='qbit', explicit=False, **policy):
root = self.root / name
(root / '.circleci').mkdir(parents=True)
if profile == 'node-artifact':
policy = {'context': 'registry', 'npm_scopes': ['@example'], 'node_version': '20.19',
'artifact_path': 'dist/bundle.js', 'build_branches': ['main'], **policy}
if profile == 'homebrew-tap':
policy = {'tap': 'qrun-io/qctl', 'formula': 'qctl', **policy}
manifest = {'profile': profile, 'orb_version': '0.7.4', 'default_branch': 'develop', **policy}
caller = rollout.standard.canonical_caller(profile, '0.7.4', explicit_manifest=explicit,
release_tag_policy=manifest.get('release_tag_policy'))
@@ -73,30 +68,6 @@ class RolloutTests(unittest.TestCase):
self.assertFalse(rollout.rollout([root], '0.7.11', write=True)[0]['changed'])
self.assertEqual(applied, self.contents(root))
def test_managed_patch_track_and_exact_pin_rollback_preserve_every_profile(self):
roots = [self.checkout(profile, profile=profile, default_branch='main')
for profile in rollout.standard.PROFILE_PARAMETERS]
originals = [yaml.safe_load((root / '.qqq-ci.yml').read_text()) for root in roots]
rollout.rollout(roots, '0.7', write=True)
for root, original in zip(roots, originals):
self.assertEqual({**original, 'orb_version': '0.7'},
yaml.safe_load((root / '.qqq-ci.yml').read_text()))
rollout.standard.validate(*(data.decode() for data in self.contents(root)), '0.7')
rollout.pipeline.load_manifest(root / '.qqq-ci.yml', '0.7')
with self.assertRaises(ValueError):
rollout.pipeline.load_manifest(root / '.qqq-ci.yml', '0.8')
self.assertTrue(all(not r['changed'] for r in rollout.rollout(roots, '0.7', write=True)))
rollout.rollout(roots, '0.7.4', write=True)
for root, original in zip(roots, originals):
self.assertEqual(original, yaml.safe_load((root / '.qqq-ci.yml').read_text()))
def test_unquoted_track_is_rejected_as_numeric_yaml(self):
root = self.checkout('numeric')
manifest = root / '.qqq-ci.yml'
manifest.write_text(manifest.read_text().replace('0.7.4', '0.7'))
with self.assertRaises(ValueError):
rollout.pipeline.load_manifest(manifest, '0.7')
def test_one_custom_caller_rejects_entire_batch_before_any_write(self):
good = self.checkout('good')
bad = self.checkout('bad')
@@ -126,7 +97,7 @@ class RolloutTests(unittest.TestCase):
def test_invalid_target_and_duplicate_checkout_are_rejected(self):
root = self.checkout('qbit')
before = self.contents(root)
for version in ['latest', 'volatile', 'dev:test', '0', '0.7.*', '0.7.11\n']:
for version in ['latest', 'dev:test', '0.7', '0.7.11\n']:
with self.assertRaises(ValueError):
rollout.rollout([root], version, write=True)
with self.assertRaises(ValueError):
-49
View File
@@ -1,49 +0,0 @@
"""Central execution rejects untrusted branches and preserves partial evidence."""
import json
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
from unittest.mock import Mock
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'tools'))
import run_enrollment
class RunnerTests(unittest.TestCase):
def environment(self):
return {'CIRCLE_PROJECT_USERNAME': 'QRun-IO', 'CIRCLE_PROJECT_REPONAME': 'qqq-orb',
'CIRCLE_BRANCH': 'main', 'GITHUB_TOKEN': 'test-only-secret'}
def test_wrong_owner_branch_repo_tag_or_missing_auth_cannot_execute(self):
for change in ({'CIRCLE_PROJECT_USERNAME': 'other'}, {'CIRCLE_PROJECT_REPONAME': 'other'},
{'CIRCLE_BRANCH': 'develop'}, {'CIRCLE_TAG': 'v1.0.0'}, {'GITHUB_TOKEN': ''}):
execute = Mock()
with self.subTest(change=change), self.assertRaises(ValueError):
run_enrollment.run({**self.environment(), **change}, execute)
execute.assert_not_called()
def test_timeout_retains_partial_report_without_secrets(self):
with tempfile.TemporaryDirectory() as directory:
def execute(args, **kwargs):
output = Path(args[args.index('--output') + 1])
output.write_text('[{"state":"unresolved"}]')
self.assertEqual(1200, kwargs['timeout'])
self.assertIn('--circle-settings', args)
self.assertEqual('test-only-secret', kwargs['env']['GH_TOKEN'])
raise subprocess.TimeoutExpired(args, 1200)
self.assertEqual(124, run_enrollment.run(self.environment(), execute, directory))
output = Path(directory) / 'artifacts/enrollment'
self.assertEqual([{'state': 'unresolved'}], json.loads((output / 'result.json').read_text()))
self.assertEqual('timed-out', json.loads((output / 'run.json').read_text())['state'])
self.assertNotIn('test-only-secret', (output / 'run.json').read_text())
def test_failed_pass_propagates_nonzero_exit(self):
with tempfile.TemporaryDirectory() as directory:
self.assertEqual(2, run_enrollment.run(self.environment(), Mock(return_value=Mock(returncode=2)), directory))
self.assertEqual('failed', json.loads((Path(directory) / 'artifacts/enrollment/run.json').read_text())['state'])
if __name__ == '__main__':
unittest.main()
-51
View File
@@ -3,7 +3,6 @@ import importlib.util
import json
from pathlib import Path
import re
import tempfile
import unittest
from unittest import mock
@@ -16,38 +15,6 @@ spec.loader.exec_module(migration)
class MigrationTests(unittest.TestCase):
def test_hotfix_and_integration_exclusion_preserves_main_tests_and_publications(self):
pattern = "/(develop|release/.*|hotfix/.*|integration.*)/"
config = self.config(filters={"branches": {"ignore": pattern}, "tags": {"only": []}})
for kind, branch in (("snapshot", "develop"), ("release_candidate", r"/release\/.*/"), ("hotfix", r"/hotfix\/.*/")):
config["workflows"][kind] = {"jobs": [{"qqq-orb/mvn_publish": {
"branch_type": kind, "filters": {"branches": {"only": [branch]}}}}]}
config["workflows"]["release"] = {"jobs": [{"qqq-orb/mvn_publish": {
"branch_type": "release", "filters": {"branches": {"ignore": "/.*/"},
"tags": {"only": [r"/^v\d+\.\d+\.\d+$/"]}}}}]}
manifest = migration.infer_manifest(config, "0.8", "develop")
self.assertEqual("non-release-no-hotfix-integration", manifest["test_branch_policy"])
self.assertFalse(manifest["release_on_main"])
self.assertEqual("stable", manifest["release_tag_policy"])
self.assertTrue(all(manifest["publish"][kind] for kind in ("snapshot", "release_candidate", "release", "hotfix")))
spec = importlib.util.spec_from_file_location("filter_pipeline", Path(__file__).resolve().parents[1] / "src/scripts/generate_pipeline.py")
pipeline = importlib.util.module_from_spec(spec)
spec.loader.exec_module(pipeline)
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / "manifest.yml"
path.write_text(yaml.safe_dump(manifest), encoding="utf-8")
loaded = pipeline.load_manifest(path, "0.8")
with mock.patch.object(pipeline, "proven_ci_maintenance", return_value=False), mock.patch.object(pipeline, "proven_docs_only", return_value=False), mock.patch.object(pipeline, "git", return_value=b"qualified-head\n"):
for branch, tag, kind in (("main", "", None), ("feature/test", "", None),
("integration-test", "", None), ("develop", "", "snapshot"),
("release/1.0", "", "release_candidate"), ("hotfix/1.0.1", "", "hotfix"),
("", "v1.0.0", "release")):
result = pipeline.qbit_template(loaded, branch, tag)
options = next(iter(next(iter(result["workflows"].values()))["jobs"][0].values()))
self.assertEqual(kind, options.get("branch_type"))
if kind is None:
self.assertEqual(["automation/qbit-rc-pending", pattern], options["filters"]["branches"]["ignore"])
def test_630_original_filter_intersections_equal_generated_publications(self):
spec = importlib.util.spec_from_file_location(
"publication_pipeline", Path(__file__).resolve().parents[1] / "src/scripts/generate_pipeline.py")
@@ -109,24 +76,6 @@ class MigrationTests(unittest.TestCase):
manifest = migration.infer_manifest(self.config(), "0.7.0", "main")
self.assertFalse(any(manifest["publish"].values()))
def test_managed_track_preserves_policy_and_caller_pins(self):
fixtures = json.loads((Path(__file__).parent / "fixtures/qbit-publication-policies.json").read_text(encoding="utf-8"))
for fixture in fixtures:
with self.subTest(repository=fixture["name"]):
expected = migration.infer_manifest(fixture["config"], "0.8.1", fixture["default_branch"])
expected["orb_version"] = "0.8"
actual = migration.infer_manifest(fixture["config"], "0.8", fixture["default_branch"])
self.assertEqual(actual, expected)
caller = migration.canonical_caller("0.8", actual["release_tag_policy"], actual["publish"].get("feature", False))
self.assertEqual(caller["orbs"]["qqq-orb"], "kingsrook/qqq-orb@0.8")
self.assertEqual(caller["workflows"]["setup"]["jobs"][0]["qqq-orb/generate_pipeline"]["orb_version"], "0.8")
self.assertEqual(yaml.safe_load(yaml.safe_dump(actual))["orb_version"], "0.8")
def test_reject_ambiguous_or_invalid_versions(self):
for version in (0.8, None, "0", "latest", "volatile", "0.8.*", "0.8.1.2", ""):
with self.subTest(version=version), self.assertRaises(ValueError):
migration.infer_manifest(self.config(), version, "main")
def test_snapshot_does_not_enable_rc_or_release(self):
manifest = migration.infer_manifest(
self.config("qqq-orb/mvn_publish", branch_type="snapshot",
-51
View File
@@ -1,51 +0,0 @@
"""Maven failures must survive tee without overriding project analysis policy."""
import os
from pathlib import Path
import subprocess
import tempfile
import unittest
ROOT = Path(__file__).resolve().parents[1]
SCRIPTS = [('mvn_spotbugs.sh', 'SPOTBUGS_FAIL_ON_ERROR', 'spotbugs.failOnError'),
('mvn_pmd.sh', 'PMD_FAIL_ON_VIOLATION', 'pmd.failOnViolation')]
class StaticAnalysisExitTests(unittest.TestCase):
def invoke(self, script, variable, setting, status):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
binary = root / 'mvn'
binary.write_text('#!/bin/sh\nprintf "%s\\n" "$@" > "$ARGUMENT_FILE"\necho analysis-output\nexit "$FAKE_MAVEN_EXIT"\n')
binary.chmod(0o755)
env = dict(os.environ, PATH=str(root) + os.pathsep + os.environ['PATH'],
ARGUMENT_FILE=str(root / 'arguments'), FAKE_MAVEN_EXIT=str(status))
env.pop(variable, None)
if setting is not None:
env[variable] = setting
result = subprocess.run(['bash', str(ROOT / 'src/scripts' / script)], cwd=root,
env=env, text=True, capture_output=True, timeout=15)
return result, (root / 'arguments').read_text().splitlines()
def test_maven_failure_is_never_hidden_by_successful_tee(self):
for script, variable, _ in SCRIPTS:
for setting in ('true', 'false'):
with self.subTest(script=script, setting=setting):
result, _ = self.invoke(script, variable, setting, 17)
self.assertEqual(17, result.returncode)
self.assertIn('analysis-output', result.stdout)
def test_success_preserves_explicit_flag_and_project_defaults(self):
for script, variable, property_name in SCRIPTS:
for setting in ('true', 'false', None):
with self.subTest(script=script, setting=setting):
result, arguments = self.invoke(script, variable, setting, 0)
self.assertEqual(0, result.returncode, result.stderr)
if setting == 'true':
self.assertIn('-D' + property_name + '=true', arguments)
else:
self.assertFalse(any(arg.startswith('-D' + property_name + '=') for arg in arguments))
self.assertIn('-DskipTests', arguments)
if __name__ == '__main__':
unittest.main()
-45
View File
@@ -1,45 +0,0 @@
"""Voyage delegates live GitOps updates to the existing shared updater."""
import json
import os
from pathlib import Path
import subprocess
import tempfile
import unittest
TEMPLATE = json.loads((Path(__file__).resolve().parents[1] / "src/scripts/templates/voyage.json").read_text())
class VoyageGitOps(unittest.TestCase):
def test_live_target_uses_shared_updater_only_after_main_image_publication(self):
self.assertNotIn('update-cd-repo', TEMPLATE['jobs'])
self.assertEqual(TEMPLATE['orbs']['munitor'], 'kof22/munitor@0.3.11')
invocations = [(workflow, entry['munitor/update_cd_repo'])
for workflow, body in TEMPLATE['workflows'].items() for entry in body['jobs']
if 'munitor/update_cd_repo' in entry]
self.assertEqual(len(invocations), 1)
workflow, options = invocations[0]
self.assertEqual(workflow, 'release')
self.assertEqual(options['cd_repo'], 'KofTwentyTwo/cluster-gitops')
self.assertEqual(options['cd_path'], 'tenants/voyage/kustomization.yaml')
self.assertEqual(options['cd_image_name'], 'ghcr.io/qrun-io/voyage')
self.assertEqual(options['cd_format'], 'kustomize')
self.assertEqual(options['requires'], ['docker-build'])
self.assertEqual(options['context'], ['github'])
self.assertEqual(options['filters'], {'branches': {'only': 'main'}})
def test_updater_receives_published_commit_tag_and_rejects_invalid_revision(self):
options = next(entry['munitor/update_cd_repo'] for entry in TEMPLATE['workflows']['release']['jobs']
if 'munitor/update_cd_repo' in entry)
command = options['pre-steps'][0]['run']['command']
with tempfile.TemporaryDirectory() as directory:
env_file = Path(directory) / 'bash-env'
for revision in ('a' * 40, '', 'main', 'a' * 39):
env_file.write_text('')
env = {**os.environ, 'BASH_ENV': str(env_file), 'CIRCLE_SHA1': revision}
result = subprocess.run(['bash', '-c', command], env=env, capture_output=True)
if revision == 'a' * 40:
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(env_file.read_text(), "export PROJECT_VERSION='aaaaaaa'\n")
else:
self.assertNotEqual(result.returncode, 0)
self.assertEqual(env_file.read_text(), '')
+130 -131
View File
@@ -1,131 +1,130 @@
"""Voyage's authoritative CI contracts, extracted from source 9f598cc105ef."""
import json
import os
import subprocess
import tempfile
from pathlib import Path
import re
import unittest
TEMPLATE = json.loads((Path(__file__).resolve().parents[1] / "src/scripts/templates/voyage.json").read_text())
def matches(pattern, branch):
if isinstance(pattern, list):
return any(matches(p, branch) for p in pattern)
if pattern.startswith("/") and pattern.endswith("/"):
return re.fullmatch(pattern[1:-1], branch) is not None
return pattern == branch
def selected(branch):
found = []
for workflow, body in TEMPLATE["workflows"].items():
for invocation in body["jobs"]:
name, options = next(iter(invocation.items()))
filters = options.get("filters", {}).get("branches", {})
if "only" in filters and not matches(filters["only"], branch):
continue
if "ignore" in filters and matches(filters["ignore"], branch):
continue
found.append((workflow, options.get("name", name), options))
return found
class VoyageContract(unittest.TestCase):
def test_each_supported_branch_has_one_validation_owner(self):
for branch, workflow in (("feature/example", "pull-request"), ("hotfix/example", "pull-request"), ("develop", "develop"), ("main", "release"), ("release/1.0", "release")):
with self.subTest(branch=branch):
jobs = selected(branch)
self.assertEqual({w for w, _, _ in jobs}, {workflow})
self.assertEqual([n for _, n, _ in jobs].count("build-and-test"), 1)
self.assertEqual({n for _, n, _ in selected("release-please--branches--main")},
{"build-and-test", "secrets-scan", "sast-scan", "security-scan"})
def test_verify_owns_tests_quality_and_coverage_without_skips(self):
jobs = TEMPLATE["jobs"]
self.assertNotIn("coverage", jobs)
self.assertNotIn("code-quality", jobs)
steps = jobs["build-and-test"]["steps"]
commands = [s["run"]["command"] for s in steps if isinstance(s, dict) and "run" in s]
self.assertEqual(commands, ["mvn clean verify -B -T4 --update-snapshots --no-transfer-progress"])
full = json.dumps(jobs)
self.assertNotIn("mvn test", full)
self.assertNotIn("jacoco.skip", full)
self.assertNotIn("skipTests", full)
self.assertLess(steps.index("qqq-orb/mvn_cache_restore"), steps.index("qqq-orb/mvn_cache_save"))
def test_unit_integration_and_coverage_evidence_retained(self):
steps = TEMPLATE["jobs"]["build-and-test"]["steps"]
results = [s["store_test_results"]["path"] for s in steps if isinstance(s, dict) and "store_test_results" in s]
self.assertEqual(results, ["target/surefire-reports", "target/failsafe-reports"])
paths = next(s["persist_to_workspace"]["paths"] for s in steps if isinstance(s, dict) and "persist_to_workspace" in s)
for path in ("target/site", "target/jacoco.exec", "target/voyage.jar", "target/classes", "target/test-classes"):
self.assertIn(path, paths)
sonar = json.dumps(TEMPLATE["jobs"]["sonar-scan"])
self.assertIn("target/site/jacoco/jacoco.xml", sonar)
self.assertIn("attach_workspace", sonar)
def test_all_workflow_dependencies_exist_and_preserve_release_boundaries(self):
for workflow, body in TEMPLATE["workflows"].items():
names = {next(iter(entry.values())).get("name", next(iter(entry))) for entry in body["jobs"]}
for entry in body["jobs"]:
name, options = next(iter(entry.items()))
self.assertTrue(name in TEMPLATE["jobs"] or name == "munitor/update_cd_repo")
self.assertTrue(set(options.get("requires", [])).issubset(names))
for branch in ("feature/example", "release/1.0", "develop"):
self.assertNotIn("update-cd-repo", [n for _, n, _ in selected(branch)])
self.assertNotIn("release-please", [n for _, n, _ in selected(branch)])
main = {n: o for _, n, o in selected("main")}
self.assertEqual(main["update-cd-repo"]["requires"], ["docker-build"])
self.assertEqual(main["release-please"]["requires"], ["docker-build"])
self.assertEqual(main["docker-build"]["requires"], ["build-and-test", "secrets-scan", "sast-scan", "security-scan"])
def test_dependency_scan_propagates_findings_and_operational_errors(self):
command = next(s["run"]["command"] for s in TEMPLATE["jobs"]["security-scan"]["steps"]
if isinstance(s, dict) and "run" in s)
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
(root / "bin").mkdir()
docker = root / "bin/docker"
docker.write_text('#!/bin/sh\nexit "$SCANNER_RESULT"\n')
docker.chmod(0o755)
(root / "target").mkdir()
(root / "target/voyage.jar").write_bytes(b"artifact")
env = {**os.environ, "HOME": directory, "PATH": str(root / "bin") + os.pathsep + os.environ["PATH"],
"CIRCLE_BRANCH": "develop"}
for code in (0, 1, 2):
result = subprocess.run(["bash", "-c", command], cwd=root,
env={**env, "SCANNER_RESULT": str(code)}, capture_output=True)
self.assertEqual(result.returncode, code)
(root / "target/voyage.jar").unlink()
result = subprocess.run(["bash", "-c", command], cwd=root,
env={**env, "SCANNER_RESULT": "0"}, capture_output=True)
self.assertNotEqual(result.returncode, 0)
def test_security_policies_and_destinations_preserved(self):
jobs = TEMPLATE["jobs"]
self.assertIn('--exit-code 1', json.dumps(jobs["docker-build"]))
self.assertIn('--severity CRITICAL,HIGH', json.dumps(jobs["docker-build"]))
self.assertIn('ghcr.io/qrun-io/voyage', json.dumps(jobs["docker-build"]))
self.assertNotIn('update-cd-repo', jobs)
owasp = '\n'.join(s["run"]["command"] for s in jobs["security-scan"]["steps"] if isinstance(s, dict) and "run" in s)
self.assertIn('--severity HIGH,CRITICAL --exit-code 1', owasp)
self.assertIn('target/voyage.jar:/scan/voyage.jar:ro', owasp)
self.assertIn('aquasec/trivy:0.76.0@sha256:', owasp)
self.assertNotIn('--ignore-unfixed', owasp)
self.assertNotIn('NVD_API_KEY', owasp)
self.assertNotIn('mvn ', owasp)
self.assertIn('set -euo pipefail', owasp)
self.assertEqual(jobs["security-scan"]["resource_class"], "medium")
for workflow in TEMPLATE["workflows"].values():
for invocation in workflow["jobs"]:
if "security-scan" in invocation:
self.assertNotIn("context", invocation["security-scan"])
self.assertIn('./gitleaks detect', json.dumps(jobs["secrets-scan"]))
self.assertIn('|| true', json.dumps(jobs["sast-scan"]))
self.assertIn('makeAggregateBom', json.dumps(jobs["sbom"]))
if __name__ == "__main__":
unittest.main()
"""Voyage's authoritative CI contracts, extracted from source 9f598cc105ef."""
import json
import os
import subprocess
import tempfile
from pathlib import Path
import re
import unittest
TEMPLATE = json.loads((Path(__file__).resolve().parents[1] / "src/scripts/templates/voyage.json").read_text())
def matches(pattern, branch):
if isinstance(pattern, list):
return any(matches(p, branch) for p in pattern)
if pattern.startswith("/") and pattern.endswith("/"):
return re.fullmatch(pattern[1:-1], branch) is not None
return pattern == branch
def selected(branch):
found = []
for workflow, body in TEMPLATE["workflows"].items():
for invocation in body["jobs"]:
name, options = next(iter(invocation.items()))
filters = options.get("filters", {}).get("branches", {})
if "only" in filters and not matches(filters["only"], branch):
continue
if "ignore" in filters and matches(filters["ignore"], branch):
continue
found.append((workflow, name, options))
return found
class VoyageContract(unittest.TestCase):
def test_each_supported_branch_has_one_validation_owner(self):
for branch, workflow in (("feature/example", "pull-request"), ("hotfix/example", "pull-request"), ("develop", "develop"), ("main", "release"), ("release/1.0", "release")):
with self.subTest(branch=branch):
jobs = selected(branch)
self.assertEqual({w for w, _, _ in jobs}, {workflow})
self.assertEqual([n for _, n, _ in jobs].count("build-and-test"), 1)
self.assertEqual(selected("release-please--branches--main"), [])
def test_verify_owns_tests_quality_and_coverage_without_skips(self):
jobs = TEMPLATE["jobs"]
self.assertNotIn("coverage", jobs)
self.assertNotIn("code-quality", jobs)
steps = jobs["build-and-test"]["steps"]
commands = [s["run"]["command"] for s in steps if isinstance(s, dict) and "run" in s]
self.assertEqual(commands, ["mvn clean verify -B -T4 --update-snapshots --no-transfer-progress"])
full = json.dumps(jobs)
self.assertNotIn("mvn test", full)
self.assertNotIn("jacoco.skip", full)
self.assertNotIn("skipTests", full)
self.assertLess(steps.index("qqq-orb/mvn_cache_restore"), steps.index("qqq-orb/mvn_cache_save"))
def test_unit_integration_and_coverage_evidence_retained(self):
steps = TEMPLATE["jobs"]["build-and-test"]["steps"]
results = [s["store_test_results"]["path"] for s in steps if isinstance(s, dict) and "store_test_results" in s]
self.assertEqual(results, ["target/surefire-reports", "target/failsafe-reports"])
paths = next(s["persist_to_workspace"]["paths"] for s in steps if isinstance(s, dict) and "persist_to_workspace" in s)
for path in ("target/site", "target/jacoco.exec", "target/voyage.jar", "target/classes", "target/test-classes"):
self.assertIn(path, paths)
sonar = json.dumps(TEMPLATE["jobs"]["sonar-scan"])
self.assertIn("target/site/jacoco/jacoco.xml", sonar)
self.assertIn("attach_workspace", sonar)
def test_all_workflow_dependencies_exist_and_preserve_release_boundaries(self):
for workflow, body in TEMPLATE["workflows"].items():
names = {next(iter(entry)) for entry in body["jobs"]}
for entry in body["jobs"]:
name, options = next(iter(entry.items()))
self.assertIn(name, TEMPLATE["jobs"])
self.assertTrue(set(options.get("requires", [])).issubset(names))
for branch in ("feature/example", "release/1.0", "develop"):
self.assertNotIn("update-cd-repo", [n for _, n, _ in selected(branch)])
self.assertNotIn("release-please", [n for _, n, _ in selected(branch)])
main = {n: o for _, n, o in selected("main")}
self.assertEqual(main["update-cd-repo"]["requires"], ["docker-build"])
self.assertEqual(main["release-please"]["requires"], ["docker-build"])
self.assertEqual(main["docker-build"]["requires"], ["build-and-test", "secrets-scan", "sast-scan", "security-scan"])
def test_dependency_scan_propagates_findings_and_operational_errors(self):
command = next(s["run"]["command"] for s in TEMPLATE["jobs"]["security-scan"]["steps"]
if isinstance(s, dict) and "run" in s)
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
(root / "bin").mkdir()
docker = root / "bin/docker"
docker.write_text('#!/bin/sh\nexit "$SCANNER_RESULT"\n')
docker.chmod(0o755)
(root / "target").mkdir()
(root / "target/voyage.jar").write_bytes(b"artifact")
env = {**os.environ, "HOME": directory, "PATH": str(root / "bin") + os.pathsep + os.environ["PATH"],
"CIRCLE_BRANCH": "develop"}
for code in (0, 1, 2):
result = subprocess.run(["bash", "-c", command], cwd=root,
env={**env, "SCANNER_RESULT": str(code)}, capture_output=True)
self.assertEqual(result.returncode, code)
(root / "target/voyage.jar").unlink()
result = subprocess.run(["bash", "-c", command], cwd=root,
env={**env, "SCANNER_RESULT": "0"}, capture_output=True)
self.assertNotEqual(result.returncode, 0)
def test_security_policies_and_destinations_preserved(self):
jobs = TEMPLATE["jobs"]
self.assertIn('--exit-code 1', json.dumps(jobs["docker-build"]))
self.assertIn('--severity CRITICAL,HIGH', json.dumps(jobs["docker-build"]))
self.assertIn('ghcr.io/qrun-io/voyage', json.dumps(jobs["docker-build"]))
self.assertIn('overlays/production/kustomization.yaml', json.dumps(jobs["update-cd-repo"]))
owasp = '\n'.join(s["run"]["command"] for s in jobs["security-scan"]["steps"] if isinstance(s, dict) and "run" in s)
self.assertIn('--severity HIGH,CRITICAL --exit-code 1', owasp)
self.assertIn('target/voyage.jar:/scan/voyage.jar:ro', owasp)
self.assertIn('aquasec/trivy:0.76.0@sha256:', owasp)
self.assertNotIn('--ignore-unfixed', owasp)
self.assertNotIn('NVD_API_KEY', owasp)
self.assertNotIn('mvn ', owasp)
self.assertIn('set -euo pipefail', owasp)
self.assertEqual(jobs["security-scan"]["resource_class"], "medium")
for workflow in TEMPLATE["workflows"].values():
for invocation in workflow["jobs"]:
if "security-scan" in invocation:
self.assertNotIn("context", invocation["security-scan"])
self.assertIn('./gitleaks detect', json.dumps(jobs["secrets-scan"]))
self.assertIn('|| true', json.dumps(jobs["sast-scan"]))
self.assertIn('makeAggregateBom', json.dumps(jobs["sbom"]))
if __name__ == "__main__":
unittest.main()
-42
View File
@@ -1,42 +0,0 @@
"""Execute Voyage release commands; CD implementation belongs to Munitor."""
import json
import os
from pathlib import Path
import subprocess
import tempfile
import unittest
ROOT = Path(__file__).resolve().parents[1]
TEMPLATE = json.loads((ROOT / "src/scripts/templates/voyage.json").read_text())
COMMAND = next(step["run"]["command"] for step in TEMPLATE["jobs"]["release-please"]["steps"]
if isinstance(step, dict) and "run" in step)
class VoyageRelease(unittest.TestCase):
def run_release(self, fail=False):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
npx = root / "npx"
npx.write_text('''#!/usr/bin/env python3
import json, os, sys
with open(os.environ['EVENTS'], 'a') as output:
output.write(json.dumps(sys.argv[1:])+'\\n')
assert '--target-branch=main' in sys.argv
if os.environ['FAILURE']=='true' and 'github-release' in sys.argv: sys.exit(25)
''')
npx.chmod(0o755)
events = root / "events.jsonl"
env = {**os.environ, "PATH": str(root) + os.pathsep + os.environ["PATH"],
"EVENTS": str(events), "FAILURE": str(fail).lower(), "GITHUB_TOKEN": "fixture-token"}
result = subprocess.run(["bash", "-c", COMMAND], cwd=root, env=env, capture_output=True, text=True)
return result, [json.loads(line) for line in events.read_text().splitlines()]
def test_release_targets_main_for_both_commands(self):
result, rows = self.run_release()
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual([args[1] for args in rows], ['github-release', 'release-pr'])
def test_release_failure_is_not_hidden_by_second_command(self):
result, rows = self.run_release(fail=True)
self.assertEqual(result.returncode, 25, result.stderr)
self.assertEqual(len(rows), 1)
+23 -101
View File
@@ -6,10 +6,6 @@ Example: python tools/audit_estate.py --inventory inventory.json --orb-version 0
Policy: {"profiles":{"qbit":{"required_checks":{"main":["observed context"],
"develop":["observed context"]}}},"repositories":{"QRun-IO/example":{...}}}.
Only use contexts verified on hosted canaries. Absent policy is unknown, not pass.
Use --live --source-branch main (or release/...) to inspect that branch's source;
omitting it retains default-branch auditing. Unavailable branches remain unknown.
A profile/repository policy may set rc_worker_sha independently of --worker-sha
so an RC-only worker change does not require updating unchanged native workers.
"""
import argparse
import base64
@@ -46,7 +42,7 @@ PROFILES = {'qqq': 'core', 'qqq-frontend-material-dashboard': 'frontend-material
'qqq-android': 'android', 'qqq-mcp-rag-standard-policies': 'mcp-policy',
'marketing-website': 'marketing', 'qqq-mcp-rag': 'native-mcp',
'qqq-business-platform': 'native-business', '.github': 'native-workflow-lint',
'new-qqq-application-template': 'application-template', 'homebrew-qctl': 'homebrew-tap', 'qqq-meta-manager': 'repository-policy', 'qqq-templates-hub': 'repository-policy', 'qqq-orb': 'owner-bootstrap', 'marketing-website-cd': 'native-gitops', 'voyage-cd': 'native-gitops'}
'qqq-orb': 'owner-bootstrap', 'marketing-website-cd': 'native-gitops'}
NO_CI = json.loads((ROOT / 'tools/no-ci-policy.json').read_text(encoding='utf-8'))
@@ -66,38 +62,6 @@ def gh(path):
return json.loads(command(['gh', 'api', '--method', 'GET', path]))
def discover_inventory(organization):
"""Discover every repository visible to existing gh authentication, across all pages."""
if not re.fullmatch(r'[A-Za-z0-9][A-Za-z0-9-]*', organization):
raise ReadError('invalid organization name')
pages = json.loads(command(['gh', 'api', '--method', 'GET', '--paginate', '--slurp',
f'orgs/{organization}/repos?type=all&per_page=100']))
if not isinstance(pages, list) or any(not isinstance(page, list) for page in pages):
raise ReadError('invalid organization inventory response')
entries, seen = [], set()
for page in pages:
for repo in page:
if not isinstance(repo, dict):
raise ReadError('invalid organization repository')
name = repo.get('name')
full_name = repo.get('full_name')
archived = repo.get('archived')
branch = repo.get('default_branch')
if (not isinstance(name, str) or not re.fullmatch(r'[A-Za-z0-9_.-]+', name)
or not isinstance(full_name, str)
or full_name.lower() != f'{organization}/{name}'.lower()
or type(archived) is not bool
or not isinstance(branch, str) or not branch
or full_name.lower() in seen):
raise ReadError('invalid or duplicate organization repository')
seen.add(full_name.lower())
entries.append({'name': name, 'nameWithOwner': full_name,
'isArchived': archived, 'defaultBranchRef': {'name': branch}})
if not entries:
raise ReadError('empty organization inventory; coverage is unproven')
return sorted(entries, key=lambda entry: entry['nameWithOwner'].lower())
class Snapshot:
def __init__(self, entry, live=False):
self.repo = entry['nameWithOwner']
@@ -105,17 +69,11 @@ class Snapshot:
raise ReadError('invalid repository name')
self.live = live
self.path = entry.get('localPath', '')
requested = entry.get('sourceBranch')
if requested is not None and (not live or not isinstance(requested, str)
or not re.fullmatch(r'[A-Za-z0-9][A-Za-z0-9._/-]*', requested)):
raise ReadError('explicit source branch requires a valid branch name and live lookup')
self.default_branch = (entry.get('defaultBranchRef') or {}).get('name')
self.branch = requested or self.default_branch
self.branch = (entry.get('defaultBranchRef') or {}).get('name')
self.sha = entry.get('sha', '')
if live:
info = gh('repos/' + self.repo)
self.default_branch = info['default_branch']
self.branch = requested or self.default_branch
self.branch = info['default_branch']
if info['archived']:
raise ReadError('repository is now archived; refresh inventory')
self.sha = gh('repos/' + self.repo + '/commits/' + quote(self.branch, safe=''))['sha']
@@ -207,10 +165,9 @@ def owner_gate_errors(snapshot):
def audit(entry, snapshot, version, worker_sha, policy):
name = entry['name']
profile = policy.get('branch_profiles', {}).get(snapshot.branch) or policy.get('profile') or ('qbit' if name.startswith('qbit-') else PROFILES.get(name))
profile = policy.get('profile') or ('qbit' if name.startswith('qbit-') else PROFILES.get(name))
result = {'repository': entry['nameWithOwner'], 'sha': snapshot.sha,
'default_branch': getattr(snapshot, 'default_branch', snapshot.branch),
'source_branch': snapshot.branch, 'profile': profile, 'findings': [], 'required_checks': {}}
'default_branch': snapshot.branch, 'profile': profile, 'findings': [], 'required_checks': {}}
findings = result['findings']
def finding(code, path=None, detail=None):
findings.append({k: v for k, v in {'code': code, 'path': path, 'detail': detail}.items() if v is not None})
@@ -239,7 +196,7 @@ def audit(entry, snapshot, version, worker_sha, policy):
manifest = Path(directory) / 'manifest.yml'
manifest.write_text(raw, encoding='utf-8')
generator.load_manifest(manifest, pin)
if data.get('default_branch') != result['default_branch']:
if data.get('default_branch') != snapshot.branch:
finding('default_branch_drift', '.qqq-ci.yml')
if caller is not None:
standard.validate(caller, raw, pin)
@@ -249,15 +206,14 @@ def audit(entry, snapshot, version, worker_sha, policy):
native_paths = set()
rc_source = snapshot.read(rc_worker.CALLER_PATH)
existing_rc = rc_source is not None and 'qbit-release-rc.yml@' in rc_source
if profile in ('qbit', 'repository-policy') and (rc_enabled or existing_rc):
if profile == 'qbit' and (rc_enabled or existing_rc):
path = rc_worker.CALLER_PATH
raw = rc_source
rc_sha = policy.get("rc_worker_sha", worker_sha)
if raw is None:
finding('missing_rc_bridge', path)
elif not re.fullmatch(r'[0-9a-f]{40}', rc_sha or ''):
elif not re.fullmatch(r'[0-9a-f]{40}', worker_sha or ''):
finding('native_pin_policy_missing', path)
elif raw != rc_worker.generate(rc_sha):
elif raw != rc_worker.generate(worker_sha):
finding('rc_bridge_drift', path)
else:
native_paths.add(path)
@@ -265,31 +221,17 @@ def audit(entry, snapshot, version, worker_sha, policy):
if other != path and 'qbit-release-rc.yml@' in snapshot.read(other):
finding('duplicate_rc_bridge', other)
for kind, definition in auxiliary.BRIDGES.items():
bridge_sha = worker_sha
if kind == 'qbit-codeql':
if profile not in ('qbit', 'repository-policy', 'all') or 'codeql_worker_sha' not in policy:
continue
bridge_sha = policy['codeql_worker_sha']
elif (definition['repository'] != entry['nameWithOwner']
and entry['nameWithOwner'] not in definition.get('additional_repositories', [])):
if definition['repository'] != entry['nameWithOwner']:
continue
path = definition['path']
raw = snapshot.read(path)
if kind == 'all-release' and profile == 'repository-policy' and raw is None:
continue
# Older release branches can predate Authentik's schema contract test.
# Keep the default branch required, and validate every existing bridge.
if (kind == 'qbit-schema-drift' and raw is None
and snapshot.branch != result['default_branch']
and not any(path.endswith('/SchemaDriftTest.java') for path in snapshot.paths)):
continue
if raw is None:
finding('missing_native_maintenance_bridge', path)
continue
if not re.fullmatch(r'[0-9a-f]{40}', bridge_sha or ''):
if not re.fullmatch(r'[0-9a-f]{40}', worker_sha or ''):
finding('native_pin_policy_missing', path)
continue
expected = generator.yaml.safe_dump(auxiliary.canonical(kind, bridge_sha), sort_keys=False)
expected = generator.yaml.safe_dump(auxiliary.canonical(kind, worker_sha), sort_keys=False)
if raw != expected:
finding('native_maintenance_bridge_drift', path)
else:
@@ -351,11 +293,9 @@ def audit(entry, snapshot, version, worker_sha, policy):
branches = policy.get('required_checks', {})
managed = policy.get('managed_branches', ['main', 'develop'])
if (not isinstance(managed, list) or not managed
or any(not isinstance(branch, str) or not re.fullmatch(
r'(?:main|develop|(?:release|hotfix)/[0-9]+\.[0-9]+(?:\.[0-9]+)?)', branch)
for branch in managed)
or any(branch not in ('main', 'develop') for branch in managed)
or len(set(managed)) != len(managed)):
raise ReadError('managed_branches must be a nonempty unique main/develop or numeric release/hotfix list')
raise ReadError('managed_branches must be a nonempty unique main/develop list')
result['managed_branches'] = managed
for branch in managed:
if branch not in branches:
@@ -381,13 +321,9 @@ def audit(entry, snapshot, version, worker_sha, policy):
return result
def run(entries, version, worker_sha, policies, live=False, snapshot_factory=Snapshot, source_branch=None):
if source_branch is not None and not live:
raise ValueError('Explicit source branch audits require --live')
def run(entries, version, worker_sha, policies, live=False, snapshot_factory=Snapshot):
results = []
for entry in entries:
if source_branch is not None:
entry = dict(entry, sourceBranch=source_branch)
if entry.get('isArchived'):
results.append({'repository': entry['nameWithOwner'], 'status': 'archived_skip'})
continue
@@ -398,41 +334,27 @@ def run(entries, version, worker_sha, policies, live=False, snapshot_factory=Sna
try:
results.append(audit(entry, snapshot_factory(entry, live), version, worker_sha, policy))
except (ReadError, OSError, ValueError, KeyError, subprocess.TimeoutExpired):
results.append({'repository': entry['nameWithOwner'], 'source_branch': source_branch,
'status': 'unknown', 'reason': 'source/API/policy read failed'})
results.append({'repository': entry['nameWithOwner'], 'status': 'unknown', 'reason': 'source/API/policy read failed'})
return {'inventory_count': len(entries), 'active_count': sum(not e.get('isArchived', False) for e in entries),
'covered_count': len(results), 'unique_repository_count': len({e['nameWithOwner'] for e in entries}),
'live': live, 'source_branch_selector': source_branch or 'default', 'results': results}
'live': live, 'results': results}
def main():
parser = argparse.ArgumentParser(description=__doc__)
inventory = parser.add_mutually_exclusive_group(required=True)
inventory.add_argument('--inventory', type=Path)
inventory.add_argument('--organization', help='Discover current repositories using existing gh authentication; requires --live')
parser.add_argument('--inventory', type=Path, required=True)
parser.add_argument('--orb-version', required=True)
parser.add_argument('--worker-sha')
parser.add_argument('--policy', type=Path)
parser.add_argument('--live', action='store_true', help='Read current source and branch rules using existing gh authentication')
parser.add_argument('--source-branch', help='Inspect this branch instead of the repository default; requires --live')
parser.add_argument('--expected-active', type=int, help='Optional independent coverage check; saved inventories default to 68')
parser.add_argument('--expected-active', type=int, default=68)
parser.add_argument('--output', type=Path)
args = parser.parse_args()
if args.source_branch is not None and not args.live:
parser.error('--source-branch requires --live')
if args.organization and not args.live:
parser.error('--organization requires --live')
entries = (discover_inventory(args.organization) if args.organization else
json.loads(args.inventory.read_text(encoding='utf-8')))
expected_active = args.expected_active
if expected_active is None and not args.organization:
expected_active = 68
entries = json.loads(args.inventory.read_text(encoding='utf-8'))
policies = json.loads(args.policy.read_text(encoding='utf-8')) if args.policy else {}
report = run(entries, args.orb_version, args.worker_sha, policies, args.live, source_branch=args.source_branch)
report['inventory_source'] = ('organization-api' if args.organization else 'saved-inventory')
report['inventory_visibility'] = 'authenticated repositories; inaccessible repositories cannot be certified'
report['expected_active_count'] = expected_active
report['coverage_matches_expected'] = ((expected_active is None or report['active_count'] == expected_active)
report = run(entries, args.orb_version, args.worker_sha, policies, args.live)
report['expected_active_count'] = args.expected_active
report['coverage_matches_expected'] = (report['active_count'] == args.expected_active
and report['unique_repository_count'] == report['inventory_count'])
text = json.dumps(report, indent=2) + '\n'
if args.output:
-78
View File
@@ -1,78 +0,0 @@
"""Manage native cost controls only for profiles with reviewed cancellation semantics."""
import json
import os
import re
from urllib.error import URLError
from urllib.request import Request, urlopen
AUTOCANCEL_PROFILES = {'repository-policy', 'homebrew-tap', 'application-template', 'marketing'}
def request(repository, payload=None):
"""Use existing central credentials; never expose authentication or error bodies."""
if not re.fullmatch(r'QRun-IO/[A-Za-z0-9_.-]+', repository):
raise ValueError('Only QRun-IO projects can be managed')
token = os.environ.get('CIRCLE_TOKEN')
if not token:
raise ValueError('Central CircleCI credential unavailable')
headers = {'Circle-Token': token, 'Accept': 'application/json'}
if payload is not None:
headers['Content-Type'] = 'application/json'
operation = Request('https://circleci.com/api/v1.1/project/github/' + repository + '/settings',
headers=headers, method='GET' if payload is None else 'PUT',
data=None if payload is None else json.dumps(payload).encode())
try:
with urlopen(operation, timeout=30) as response:
return json.load(response)
except (URLError, ValueError):
raise ValueError('CircleCI project settings unavailable') from None
def reconcile(plan, source_request, apply=False, settings_request=request):
"""Require a current canonical source and preserve every unrelated project flag."""
if plan['profile'] not in AUTOCANCEL_PROFILES:
return {'state': 'not-managed', 'reason': 'Profile cancellation policy not reviewed'}
if plan['changed']:
return {'state': 'waiting-source', 'reason': 'Canonical enrollment must land first'}
repo = plan['repository']
try:
info = source_request(f'repos/{repo}')
if info['archived'] or (plan['profile'] == 'marketing' and info['default_branch'] != 'main'):
raise ValueError('Repository default branch or archive state differs from policy')
before = settings_request(repo)['feature_flags']
if not isinstance(before, dict) or type(before.get('autocancel-builds')) is not bool:
raise ValueError('CircleCI cancellation flag is missing or invalid')
changed = not before['autocancel-builds']
if apply and changed:
if source_request(f'repos/{repo}/branches/{plan["branch"]}')['commit']['sha'] != plan['head']:
raise ValueError('Source changed after enrollment inspection')
settings_request(repo, {'feature_flags': {'autocancel-builds': True}})
after = settings_request(repo)['feature_flags']
if after != {**before, 'autocancel-builds': True}:
raise ValueError('CircleCI settings readback differs from the narrow update')
return {'state': 'enforced' if apply else 'inspected', 'changed': changed}
except (ValueError, KeyError, TypeError) as error:
return {'state': 'unresolved', 'reason': str(error)}
def reconcile_repository(plans, findings, source_request, apply=False, settings_request=request):
"""A project-wide flag requires agreement across all discovered current branches."""
repo = plans[0]['repository']
if any(plan['repository'] != repo for plan in plans):
raise ValueError('Expected one repository')
if any(row['repository'] == repo and row.get('state') == 'unresolved' for row in findings):
return {'state': 'waiting-source', 'reason': 'A current branch is unresolved'}
if any(plan['profile'] not in AUTOCANCEL_PROFILES for plan in plans):
return {'state': 'not-managed', 'reason': 'A current branch has an unreviewed cancellation policy'}
if any(plan['changed'] for plan in plans):
return {'state': 'waiting-source', 'reason': 'All current branches must have canonical enrollment'}
try:
for plan in plans:
if source_request(f'repos/{repo}/branches/{plan["branch"]}')['commit']['sha'] != plan['head']:
raise ValueError('Source changed after enrollment inspection')
except (ValueError, KeyError, TypeError) as error:
return {'state': 'unresolved', 'reason': str(error)}
# Preserve the marketing default-branch guard even in mixed eligible profiles.
plan = next((p for p in plans if p['profile'] == 'marketing'), plans[0])
return reconcile(plan, source_request, apply, settings_request)
-74
View File
@@ -1,74 +0,0 @@
"""Discover current enrollment targets without assuming unknown repositories conform."""
import audit_estate
def approved_no_ci(target, request):
"""Recognize only unchanged documentation; executable/data sources still need inspection."""
repo = target['repository']
policy = audit_estate.NO_CI.get(repo)
if policy is None:
return None
head = request(f'repos/{repo}/branches/{target["branch"]}')['commit']['sha']
if head != policy['baseline_sha']:
return None
tree = request(f'repos/{repo}/git/trees/{head}?recursive=1')
if tree.get('truncated') is not False or not isinstance(tree.get('tree'), list):
raise ValueError('Approved no-CI snapshot requires a complete source tree')
paths = {item['path'] for item in tree['tree']}
if not set(policy['required_paths']) <= paths:
raise ValueError('Approved no-CI identity paths are missing')
if any(path in policy['forbidden_paths'] or any(path.startswith(prefix)
for prefix in policy['forbidden_prefixes']) for path in paths):
raise ValueError('Approved no-CI snapshot contains unexpected CI configuration')
documents = {'LICENSE', 'NOTICE', '.gitignore', '.gitattributes', '.gitkeep'}
for item in tree['tree']:
if item.get('type') == 'tree':
continue
if item.get('type') != 'blob' or not (item['path'].endswith(('.md', '.txt')) or item['path'] in documents):
return None
return {**target, 'head': head, 'state': 'approved-no-ci',
'source_kind': 'documentation-only', 'reason': policy['rationale']}
def discover(organization, version, inspect, request, inventory=None):
"""Inspect every active main/develop branch; retain unsupported results explicitly."""
entries = audit_estate.discover_inventory(organization) if inventory is None else inventory
plans, findings = [], []
for entry in entries:
repo = entry['nameWithOwner']
if entry['isArchived']:
findings.append({'repository': repo, 'state': 'archived'})
continue
if repo.lower() == (organization + '/qqq-orb').lower():
findings.append({'repository': repo, 'state': 'owner-bootstrap',
'reason': 'Orb publisher is not its own consumer'})
continue
try:
branches, page = [], 1
while True:
batch = request(f'repos/{repo}/branches?per_page=100&page={page}')
if not isinstance(batch, list) or any(not isinstance(b, dict) or not isinstance(b.get('name'), str) for b in batch):
raise ValueError('Invalid branch inventory')
branches.extend(b['name'] for b in batch)
if len(batch) < 100:
break
page += 1
if len(set(branches)) != len(branches):
raise ValueError('Duplicate branch inventory')
selected = sorted(set(branches) & {'main', 'develop'})
if not selected:
raise ValueError('No current main/develop branch; explicit profile decision required')
except (ValueError, OSError) as error:
findings.append({'repository': repo, 'state': 'unresolved', 'reason': str(error)})
continue
for branch in selected:
target = {'repository': repo, 'branch': branch}
try:
no_ci = approved_no_ci(target, request)
if no_ci is not None:
findings.append(no_ci)
else:
plans.append(inspect(target, version))
except (ValueError, OSError) as error:
findings.append({**target, 'state': 'unresolved', 'reason': str(error)})
return plans, findings
-265
View File
@@ -1,265 +0,0 @@
#!/usr/bin/env python3
"""Batch-enforce observed orb checks; dry-run unless --apply. Never changes source."""
import argparse
import base64
import importlib.util
import json
from pathlib import Path
import re
import subprocess
from urllib.parse import quote
ROOT = Path(__file__).resolve().parents[1]
spec = importlib.util.spec_from_file_location('standard', ROOT / 'src/scripts/validate_standard.py')
standard = importlib.util.module_from_spec(spec)
spec.loader.exec_module(standard)
DESIRED = [{'context': name, 'app_id': 18001} for name in ('setup', 'verify')]
PROFILE_GATES = {profile: 'verify' for profile in (
'qbit', 'core', 'frontend-core', 'frontend-material', 'repository-policy', 'node-artifact', 'homebrew-tap', 'application-template', 'marketing', 'voyage')}
PROFILE_GATES.update({'all': 'verify-and-smoke', 'cli': 'test_only',
'app-starter': 'test_only', 'android': 'test-and-build',
'mcp-policy': 'validate'})
PROFILES = set(PROFILE_GATES) | {'frontend-next', 'native-gitops'}
_native_spec = importlib.util.spec_from_file_location('native_next_gates', ROOT / 'tools/native_next_gates.py')
native = importlib.util.module_from_spec(_native_spec)
_native_spec.loader.exec_module(native)
_gitops_spec = importlib.util.spec_from_file_location('native_gitops_gates', ROOT / 'tools/native_gitops_gates.py')
gitops = importlib.util.module_from_spec(_gitops_spec)
_gitops_spec.loader.exec_module(gitops)
_specialist_spec = importlib.util.spec_from_file_location('native_specialist_gates', ROOT / 'tools/native_specialist_gates.py')
specialists = importlib.util.module_from_spec(_specialist_spec)
_specialist_spec.loader.exec_module(specialists)
PROFILES |= specialists.PROFILES
def gates(profile):
if profile in specialists.PROFILES:
return specialists.gates(profile)
if profile == 'native-gitops':
return gitops.gates()
if profile == 'frontend-next':
return native.gates(pull_request=True)
return [{'context': name, 'app_id': 18001}
for name in ('setup', PROFILE_GATES[profile])]
def same_checks(left, right):
"""GitHub may reorder checks; retain exact bindings and duplicate counts."""
return sorted(json.dumps(check, sort_keys=True) for check in left) == sorted(
json.dumps(check, sort_keys=True) for check in right)
def api(path, method='GET', payload=None):
"""Use existing gh authentication; never expose response bodies on failure."""
args = ['gh', 'api', '--method', method, path]
if payload is not None:
args += ['--input', '-']
result = subprocess.run(args, input=None if payload is None else json.dumps(payload),
text=True, encoding='utf-8', capture_output=True, timeout=60)
if result.returncode:
try:
error = json.loads(result.stdout)
except ValueError:
error = {}
if method == 'GET' and path.endswith('/protection') and error.get('status') == '404' and error.get('message') == 'Branch not protected':
return None
raise ValueError('GitHub operation unavailable: ' + method + ' ' + path.split('?')[0])
return json.loads(result.stdout) if result.stdout.strip() else None
def required_checks(before, runs, desired_gates=DESIRED, observed_gates=None):
"""Require successful event-appropriate gates and preserve every PR requirement."""
for desired in (desired_gates if observed_gates is None else observed_gates):
matches = [run for run in runs if run['name'] == desired['context'] and run['app']['id'] == desired['app_id']]
if not matches or max(matches, key=lambda run: run['id'])['conclusion'] != 'success':
raise ValueError('Current branch head has no successful app-bound gate: ' + desired['context'])
prior = (before or {}).get('required_status_checks') or {}
checks = [dict(check) for check in prior.get('checks', [])]
represented = {check['context'] for check in checks}
checks += [{'context': name, 'app_id': -1} for name in prior.get('contexts', []) if name not in represented]
for desired in desired_gates:
matches = [check for check in checks if check['context'] == desired['context']]
if any(check.get('app_id') not in (None, -1, desired['app_id']) for check in matches):
raise ValueError('Existing gate uses a different app: ' + desired['context'])
checks = [check for check in checks if check['context'] != desired['context']]
checks.append(dict(desired))
return {'strict': True, 'checks': checks}
def verified_rulesets(repo, effective, request=api, desired_gates=DESIRED):
"""Recognize active repository rulesets with bound gates and no bypass actors."""
selected = [rule for rule in effective if rule['type'] == 'required_status_checks']
if not selected or any(rule.get('ruleset_source_type') != 'Repository'
or rule.get('ruleset_source') != repo for rule in selected):
raise ValueError('Required rulesets need unsupported source reconciliation')
records = []
for identifier in sorted({rule['ruleset_id'] for rule in selected}):
record = request(f'repos/{repo}/rulesets/{identifier}')
if (record.get('enforcement') != 'active' or record.get('target') != 'branch'
or record.get('source_type') != 'Repository' or record.get('source') != repo
or record.get('bypass_actors') != []):
raise ValueError('Ruleset must be active with no bypass actors')
records.append(record)
requirements = [rule['parameters'] for record in records for rule in record['rules']
if rule['type'] == 'required_status_checks']
for desired in desired_gates:
if not any(check['context'] == desired['context'] and check.get('integration_id') == desired['app_id']
for parameters in requirements for check in parameters['required_status_checks']):
raise ValueError('Ruleset is missing app-bound gate: ' + desired['context'])
# Retain the existing freshness policy: recognition does not add rebuilds or
# weaken any existing scanner, merge, or bypass requirement.
return records, any(parameters.get('strict_required_status_checks_policy', False) for parameters in requirements)
def inspect(repo, branch, version, request=api):
"""Read immutable canonical source, latest check runs, and current protections."""
if not re.fullmatch(r'[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+', repo) or branch not in ('main', 'develop'):
raise ValueError('Targets must name an exact repository and main/develop branch')
root = 'repos/' + repo
info = request(root)
if info['archived']:
raise ValueError('Archived repository')
prefix = root + '/branches/' + branch
head = request(prefix)['commit']['sha']
if repo in specialists.DEFINITIONS:
profile, _ = specialists.validate(repo, head, branch, request)
elif repo in gitops.REPOSITORIES:
gitops.validate(repo, head, branch, request, standard.load_yaml)
profile = 'native-gitops'
else:
source = []
for path in ('.circleci/config.yml', '.qqq-ci.yml'):
item = request(root + '/contents/' + quote(path, safe='/') + '?ref=' + head)
if item.get('encoding') != 'base64' or item.get('type') != 'file':
raise ValueError('Canonical declarations must be regular files')
source.append(base64.b64decode(item['content'], validate=False).decode('utf-8'))
standard.validate(*source, version)
profile = standard.load_yaml(source[1])['profile']
if profile not in PROFILES:
raise ValueError('Profile has no supported shared branch gates')
desired_gates = gates(profile)
observed_gates = desired_gates
if profile == 'frontend-next':
native.validate(repo, head, branch, request, standard.load_yaml)
observed_gates = native.gates()
rules = request(root + '/rules/branches/' + branch)
rulesets, ruleset_strict = verified_rulesets(repo, rules, request, desired_gates) if rules else ([], False)
before = request(prefix + '/protection')
restrictions = (before or {}).get('restrictions')
missing_status_section = before is not None and before.get('required_status_checks') is None
empty_push_allowlist = (restrictions is not None
and not any(restrictions.get(kind) for kind in ('users', 'teams', 'apps'))
and not (before or {}).get('enforce_admins', {}).get('enabled'))
# GitHub cannot PATCH a missing status-check subresource. An additive ruleset
# preserves the existing signing, review, history and access policies.
use_ruleset = not rulesets and (missing_status_section or empty_push_allowlist)
runs = []
page = 1
while True:
batch = request(root + '/commits/' + head + '/check-runs?filter=latest&per_page=100&page=' + str(page))['check_runs']
runs += batch
if len(batch) < 100:
break
page += 1
desired = required_checks(before, runs, desired_gates, observed_gates)
if use_ruleset:
return {'repository': repo, 'branch': branch, 'head': head, 'before': before, 'gates': desired_gates,
'required_status_checks': {'strict': False, 'checks': desired_gates},
'changed': True, 'protection_mode': 'create-ruleset'}
if rulesets:
desired['strict'] = ruleset_strict or bool(((before or {}).get('required_status_checks') or {}).get('strict'))
return {'repository': repo, 'branch': branch, 'head': head, 'before': before, 'gates': desired_gates,
'required_status_checks': desired, 'changed': False, 'protection_mode': 'rulesets',
'effective_rules': rules, 'rulesets': rulesets}
current = (before or {}).get('required_status_checks') or {}
ready = (current.get('strict') is True and same_checks(current.get('checks', []), desired['checks'])
and (before or {}).get('enforce_admins', {}).get('enabled') is True)
return {'repository': repo, 'branch': branch, 'head': head, 'before': before, 'gates': desired_gates,
'required_status_checks': desired, 'changed': not ready}
def enforce(plan, request=api):
"""Recheck drift, use narrow updates, and verify preservation of other settings."""
desired_gates = plan.get('gates', DESIRED)
root = 'repos/' + plan['repository']
prefix = root + '/branches/' + plan['branch']
if request(prefix)['commit']['sha'] != plan['head'] or request(prefix + '/protection') != plan['before']:
raise ValueError('Branch or protection changed after preflight')
rules = request(root + '/rules/branches/' + plan['branch'])
if plan.get('protection_mode') == 'rulesets':
if rules != plan['effective_rules'] or verified_rulesets(plan['repository'], rules, request, desired_gates)[0] != plan['rulesets']:
raise ValueError('Ruleset changed after preflight')
return
if rules:
raise ValueError('Ruleset changed after preflight')
if plan.get('protection_mode') == 'create-ruleset':
name = 'QQQ automated verification: ' + plan['branch']
page = 1
while True:
existing = request(root + '/rulesets?per_page=100&page=' + str(page))
if any(record['name'] == name for record in existing):
raise ValueError('Named ruleset already exists; reconcile its scope explicitly')
if len(existing) < 100:
break
page += 1
request(root + '/rulesets', 'POST', {
'name': name, 'target': 'branch', 'enforcement': 'active', 'bypass_actors': [],
'conditions': {'ref_name': {'include': ['refs/heads/' + plan['branch']], 'exclude': []}},
'rules': [{'type': 'required_status_checks', 'parameters': {
'strict_required_status_checks_policy': False, 'do_not_enforce_on_create': False,
'required_status_checks': [{'context': gate['context'], 'integration_id': gate['app_id']}
for gate in desired_gates]}}]})
effective = request(root + '/rules/branches/' + plan['branch'])
verified_rulesets(plan['repository'], effective, request, desired_gates)
if request(prefix + '/protection') != plan['before']:
raise ValueError('Classic access policy changed during ruleset creation')
return
if not plan['changed']:
return
before = plan['before']
if before is None:
request(prefix + '/protection', 'PUT', {
'required_status_checks': plan['required_status_checks'], 'enforce_admins': True,
'required_pull_request_reviews': None, 'restrictions': None})
else:
request(prefix + '/protection/required_status_checks', 'PATCH', plan['required_status_checks'])
if not before['enforce_admins']['enabled']:
request(prefix + '/protection/enforce_admins', 'POST')
after = request(prefix + '/protection')
actual = after['required_status_checks']
if not actual['strict'] or not same_checks(actual['checks'], plan['required_status_checks']['checks']) or not after['enforce_admins']['enabled']:
raise ValueError('Protection readback did not match requested gates')
if before:
for key, value in before.items():
if key not in ('required_status_checks', 'enforce_admins') and after.get(key) != value:
raise ValueError('Unrelated protection changed: ' + key)
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--targets', type=Path, required=True, help='JSON list of {repository, branch}')
parser.add_argument('--orb-version', required=True)
parser.add_argument('--apply', action='store_true')
parser.add_argument('--output', type=Path, required=True)
args = parser.parse_args()
targets = json.loads(args.targets.read_text(encoding='utf-8'))
if not isinstance(targets, list) or not targets:
raise ValueError('A nonempty explicit target list is required')
if len({(row['repository'], row['branch']) for row in targets}) != len(targets):
raise ValueError('Duplicate branch target')
# Resolve the entire batch before any mutations. Failures never trigger rollback
# that would remove successful protections; rerunning converges idempotently.
plans = [inspect(row['repository'], row['branch'], args.orb_version) for row in targets]
args.output.write_text(json.dumps({'state': 'planned', 'plans': plans}, indent=2), encoding='utf-8')
if args.apply:
for plan in plans:
enforce(plan)
plan['verified'] = True
args.output.write_text(json.dumps({'state': 'applying', 'plans': plans}, indent=2), encoding='utf-8')
args.output.write_text(json.dumps({'state': 'verified', 'plans': plans}, indent=2), encoding='utf-8')
print(json.dumps({'targets': len(plans), 'changed': sum(plan['changed'] for plan in plans), 'applied': args.apply}))
if __name__ == '__main__':
main()
-101
View File
@@ -1,101 +0,0 @@
#!/usr/bin/env python3
"""Enroll a batch of audited consumers; preview by default, write with --write.
Existing canonical profiles retain their manifests. Legacy QBits use the shared
policy inference. Unknown orchestration is rejected before any checkout changes.
"""
import argparse
import json
from pathlib import Path
import re
import tempfile
import yaml
import rollout_version as rollout
import standardize_qbit as qbit
def declaration(root, relative):
"""Reject redirected declarations, including symlinked parent directories."""
path = root / relative
current = path
while current != root:
if current.is_symlink():
raise ValueError('CI declarations must not use symlinks')
current = current.parent
if not path.resolve().is_relative_to(root):
raise ValueError('CI declarations must remain inside the checkout')
return path
def plan(target, version):
"""Preserve explicit publication policy and validate generated declarations."""
if not isinstance(target, dict) or set(target) != {'checkout', 'default_branch'}:
raise ValueError('Each target requires checkout and observed default_branch')
if target['default_branch'] not in ('main', 'develop'):
raise ValueError('Unsupported default branch')
root = Path(target['checkout']).resolve(strict=True)
config = declaration(root, '.circleci/config.yml')
manifest = declaration(root, '.qqq-ci.yml')
if manifest.exists():
data = rollout.standard.load_yaml(manifest.read_text(encoding='utf-8'))
if not isinstance(data, dict) or data.get('default_branch') != target['default_branch']:
raise ValueError('Observed default branch differs from manifest')
return {**rollout.plan(root, version), 'enrollment': False}
original = config.read_bytes()
data = qbit.infer_manifest(rollout.standard.load_yaml(original.decode('utf-8')),
version, target['default_branch'])
caller = rollout.standard.canonical_caller(data['profile'], version,
release_tag_policy=data['release_tag_policy'], publish_feature=data['publish'].get('feature', False))
config_raw = yaml.safe_dump(caller, sort_keys=False)
manifest_raw = yaml.safe_dump(data, sort_keys=False)
rollout.standard.validate(config_raw, manifest_raw, version)
with tempfile.TemporaryDirectory() as directory:
temporary_manifest = Path(directory) / 'manifest.yml'
temporary_manifest.write_text(manifest_raw, encoding='utf-8')
rollout.pipeline.load_manifest(temporary_manifest, version)
return {'checkout': str(root), 'profile': data['profile'], 'from': None,
'to': version, 'enrollment': True,
'files': [(config, original, config_raw.encode('utf-8')),
(manifest, None, manifest_raw.encode('utf-8'))]}
def enroll(targets, version, write=False):
"""Validate the whole batch before writing; repeat applications are no-ops."""
if not isinstance(version, str) or not re.fullmatch(r'\d+\.\d+(?:\.\d+)?', version):
raise ValueError('Enrollment requires a stable minor track or exact patch')
if not isinstance(targets, list) or not targets:
raise ValueError('A nonempty target list is required')
plans = [plan(target, version) for target in targets]
if len({item['checkout'] for item in plans}) != len(plans):
raise ValueError('Each checkout may appear only once')
if write:
for item in plans:
for path, original, replacement in item['files']:
declaration(Path(item['checkout']), path.relative_to(item['checkout']))
current = path.read_bytes() if path.exists() else None
if current != original:
raise ValueError('Declaration changed after preflight')
for item in plans:
for path, original, replacement in item['files']:
if replacement != original:
path.write_bytes(replacement)
return [{**{key: value for key, value in item.items() if key != 'files'},
'changed': any(old != new for _, old, new in item['files']),
'written': write} for item in plans]
def main():
"""Read one fleet target file and emit an auditable result."""
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('targets', type=Path)
parser.add_argument('--orb-version', required=True)
parser.add_argument('--write', action='store_true')
args = parser.parse_args()
print(json.dumps(enroll(json.loads(args.targets.read_text(encoding='utf-8')),
args.orb_version, args.write), indent=2))
if __name__ == '__main__':
main()
-85
View File
@@ -1,85 +0,0 @@
"""Qualify signed enrollment PRs and enforce only successful current branch heads."""
import json
import enforce_orb_checks as enforcement
import reconcile_enrollment as remote
def qualification(repo, head, profile='qbit'):
"""Require profile-specific app-bound gates and every advertised latest check/status."""
checks, page = [], 1
while True:
batch = remote.api(f'repos/{repo}/commits/{head}/check-runs?filter=latest&per_page=100&page={page}')['check_runs']
checks.extend(batch)
if len(batch) < 100:
break
page += 1
reasons = []
for gate in enforcement.gates(profile):
name = gate['context']
found = [check for check in checks if check['name'] == name and check['app']['id'] == gate['app_id']]
if not found or max(found, key=lambda check: check['id'])['conclusion'] != 'success':
reasons.append(name)
for check in checks:
if check['status'] != 'completed' or check['conclusion'] not in ('success', 'neutral', 'skipped'):
reasons.append(check['name'])
page = 1
while True:
batch = remote.api(f'repos/{repo}/commits/{head}/status?per_page=100&page={page}')['statuses']
reasons.extend(status['context'] for status in batch if status['state'] != 'success')
if len(batch) < 100:
break
page += 1
return sorted(set(reasons))
def merge_ready(plan, result):
"""Normal exact-head squash merge only; no administrative or force bypass."""
if plan['profile'] not in enforcement.PROFILES:
return {'state': 'unsupported-gates', 'profile': plan['profile']}
if result['state'] == 'unchanged':
return {'state': 'unchanged'}
repo, head = plan['repository'], result['head']
remote.verify_submission(plan, head)
pr = json.loads(remote.command(['gh', 'pr', 'view', result['url'], '--repo', repo,
'--json', 'number,state,headRefOid,baseRefOid,baseRefName,mergeCommit']))
if pr['headRefOid'] != head or pr['baseRefName'] != plan['branch']:
raise ValueError('PR differs from enrollment target')
if pr['state'] == 'CLOSED':
return {'state': 'closed', 'url': result['url']}
if pr['state'] != 'MERGED':
if pr['baseRefOid'] != plan['head'] or remote.api(f'repos/{repo}/branches/{plan["branch"]}')['commit']['sha'] != plan['head']:
return {'state': 'stale-base', 'url': result['url']}
if plan['profile'] in enforcement.specialists.PROFILES:
enforcement.specialists.validate(repo, head, plan['branch'], remote.api)
if plan['profile'] == 'native-gitops':
enforcement.gitops.validate(repo, head, plan['branch'], remote.api, enforcement.standard.load_yaml)
if plan['profile'] == 'frontend-next':
enforcement.native.validate(repo, head, plan['branch'], remote.api, enforcement.standard.load_yaml)
pending = qualification(repo, head, plan['profile'])
if pending:
return {'state': 'waiting-checks', 'checks': pending, 'url': result['url']}
remote.command(['gh', 'pr', 'merge', str(pr['number']), '--repo', repo, '--squash', '--match-head-commit', head])
pr = json.loads(remote.command(['gh', 'pr', 'view', result['url'], '--repo', repo,
'--json', 'state,mergeCommit']))
if pr['state'] != 'MERGED':
raise ValueError('PR merge was not confirmed')
merged = pr['mergeCommit']['oid']
expected = remote.api(f'repos/{repo}/commits/{head}')['commit']['tree']['sha']
actual = remote.api(f'repos/{repo}/commits/{merged}')['commit']
if not actual['verification']['verified'] or actual['tree']['sha'] != expected:
raise ValueError('Merged signature or source tree differs from qualified source')
return {'state': 'merged', 'url': result['url'], 'head': head, 'merge': merged}
def enforce_current(plan):
"""Use shared narrow enforcement after the actual branch head passes its gates."""
if plan['profile'] not in enforcement.PROFILES:
return {'state': 'unsupported-gates', 'profile': plan['profile']}
try:
current = enforcement.inspect(plan['repository'], plan['branch'], plan['version'])
except ValueError as error:
return {'state': 'waiting-enforcement', 'reason': str(error)}
enforcement.enforce(current)
return {'state': 'enforced', 'head': current['head'], 'changed': current['changed'],
'protection_mode': current.get('protection_mode', 'classic')}
-19
View File
@@ -1,19 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
# Ubuntu Jammy ships gh 2.4, which lacks API --slurp and modern merge guards.
# Pin the official amd64 release and its GitHub release asset SHA-256.
version=2.103.0
checksum=fdd768eb70f6409f80fcfa9f3a1c0adf0fc98d827ab3ce0bed2ac22762c5d786
archive=$(mktemp)
binary=$(mktemp)
trap 'rm -f "$archive" "$binary"' EXIT
curl --fail --silent --show-error --location --retry 3 \
"https://github.com/cli/cli/releases/download/v${version}/gh_${version}_linux_amd64.tar.gz" -o "$archive"
printf '%s %s\n' "$checksum" "$archive" | sha256sum --check --status
tar -xOzf "$archive" "gh_${version}_linux_amd64/bin/gh" > "$binary"
sudo install -m 0755 "$binary" /usr/local/bin/gh
gh --version
# Drain help output: grep -q can SIGPIPE gh under pipefail.
gh api --help | grep -- --slurp >/dev/null
gh pr merge --help | grep -- --match-head-commit >/dev/null
-51
View File
@@ -1,51 +0,0 @@
"""Recognize approved GitOps repositories through their shared immutable worker."""
import base64
import importlib.util
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
spec = importlib.util.spec_from_file_location('gitops_bridge', ROOT / 'src/scripts/generate_auxiliary_workflow.py')
bridge = importlib.util.module_from_spec(spec)
spec.loader.exec_module(bridge)
DEFINITION = bridge.BRIDGES['gitops']
REPOSITORIES = {DEFINITION['repository'], *DEFINITION.get('additional_repositories', [])}
def gates():
return [{'context': 'gitops / validate', 'app_id': 15368}]
def validate(repo, head, branch, request, load_yaml):
if repo not in REPOSITORIES or branch != 'main':
raise ValueError('GitOps recognition requires an approved repository on main only')
def content(owner, path, ref):
item = request(f'repos/{owner}/contents/{path}?ref={ref}')
if item.get('type') != 'file' or item.get('encoding') != 'base64':
raise ValueError('GitOps workflow must be a regular file')
return base64.b64decode(item['content']).decode('utf-8').replace('\r\n', '\n')
path = DEFINITION['path']
caller = load_yaml(content(repo, path, head))
try:
pin = caller['jobs']['gitops']['uses'].rsplit('@', 1)[1]
expected = bridge.canonical('gitops', pin)
except (KeyError, TypeError, IndexError, ValueError, AttributeError):
raise ValueError('GitOps caller has no immutable shared worker pin') from None
if caller != expected:
raise ValueError('GitOps caller differs from the canonical shared bridge')
tree = request(f'repos/{repo}/git/trees/{head}?recursive=1')
if tree.get('truncated') is not False or not isinstance(tree.get('tree'), list):
raise ValueError('GitOps recognition requires the complete source tree')
for entry in tree['tree']:
name = entry['path']
if name == '.circleci/config.yml' or (name.startswith('.github/workflows/')
and name.endswith(('.yml', '.yaml')) and name != path):
raise ValueError('GitOps repository has additional orchestration: ' + name)
commit = request('repos/QRun-IO/qqq-orb/commits/' + pin)
if not commit['commit']['verification']['verified']:
raise ValueError('GitOps worker commit is not verified')
worker = '.github/workflows/gitops.yml'
if content('QRun-IO/qqq-orb', worker, pin) != (ROOT / worker).read_text(encoding='utf-8'):
raise ValueError('Pinned GitOps worker differs from the supported contract')
return pin
-51
View File
@@ -1,51 +0,0 @@
"""Validate the canonical native bridge and its immutable shared worker contract."""
import base64
import importlib.util
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
spec = importlib.util.spec_from_file_location('next_bridge', ROOT / 'src/scripts/generate_next_workflow.py')
bridge = importlib.util.module_from_spec(spec)
spec.loader.exec_module(bridge)
MAIN_GATES = [
'ci / Typecheck, lint, unit tests and coverage',
'ci / Mocked e2e (Playwright, chromium)',
'ci / Collaboration e2e (MSW; chromium, firefox, webkit, mobile, tablet)',
'ci / Bundle budget (static export)', 'ci / Storybook build',
'acceptance / acceptance gate',
]
PR_GATES = MAIN_GATES + ['dashboard-image / build (amd64, ubuntu-24.04)',
'dashboard-image / build (arm64, ubuntu-24.04-arm)']
def gates(pull_request=False):
return [{'context': name, 'app_id': 15368} for name in (PR_GATES if pull_request else MAIN_GATES)]
def validate(repo, head, branch, request, load_yaml):
"""Only main has a push trigger; require exact caller and reviewed worker bytes."""
if branch != 'main':
raise ValueError('Next native branch verification supports main only')
def content(owner, path, ref):
item = request(f'repos/{owner}/contents/{path}?ref={ref}')
if item.get('type') != 'file' or item.get('encoding') != 'base64':
raise ValueError('Native workflow must be a regular file')
return base64.b64decode(item['content']).decode('utf-8').replace('\r\n', '\n')
caller = load_yaml(content(repo, '.github/workflows/ci.yml', head))
try:
pin = caller['jobs']['ci']['uses'].rsplit('@', 1)[1]
expected = bridge.canonical_next(pin)
except (KeyError, TypeError, IndexError, ValueError):
raise ValueError('Native caller has no valid immutable worker pin') from None
if caller != expected:
raise ValueError('Native caller differs from the canonical shared bridge')
commit = request('repos/QRun-IO/qqq-orb/commits/' + pin)
if not commit['commit']['verification']['verified']:
raise ValueError('Shared native worker commit is not verified')
for name in ('next-ci.yml', 'next-acceptance.yml', 'next-dashboard-image.yml'):
path = '.github/workflows/' + name
if content('QRun-IO/qqq-orb', path, pin) != (ROOT / path).read_text(encoding='utf-8'):
raise ValueError('Pinned native worker differs from the supported contract: ' + name)
return pin
-115
View File
@@ -1,115 +0,0 @@
"""Recognize reviewed native consumers and their shared immutable orb workers."""
import base64
import importlib.util
import re
from pathlib import Path
import yaml
ROOT = Path(__file__).resolve().parents[1]
def module(name):
spec = importlib.util.spec_from_file_location(name, ROOT / 'src/scripts' / (name + '.py'))
result = importlib.util.module_from_spec(spec)
spec.loader.exec_module(result)
return result
auxiliary = module('generate_auxiliary_workflow')
qctl = module('generate_qctl_workflow')
standard = module('validate_standard')
class WorkflowLoader(standard.UniqueLoader):
# GitHub uses YAML 1.2: unquoted "on" is an event key, not a boolean.
yaml_implicit_resolvers = {
key: [(tag, pattern) for tag, pattern in values if tag != 'tag:yaml.org,2002:bool']
for key, values in standard.UniqueLoader.yaml_implicit_resolvers.items()
}
WorkflowLoader.add_implicit_resolver('tag:yaml.org,2002:bool', re.compile(r'^(true|false)$', re.I), list('tTfF'))
def load_workflow(raw):
try:
return yaml.load(raw, Loader=WorkflowLoader)
except yaml.YAMLError as error:
raise ValueError('Invalid native workflow YAML') from error
# Only observed, mandatory verification gates; release-only jobs are not PR gates.
DEFINITIONS = {
'QRun-IO/qctl': ('qctl', ('main', 'develop'), ('qctl',),
('native / test',)),
'QRun-IO/.github': ('native-workflow-lint', ('main',), ('github-workflow-lint',),
('actionlint / actionlint',)),
'QRun-IO/qqq-business-platform': ('native-business', ('main',), ('business-db', 'business-e2e'),
('validate / empty-database-migration', 'validate / Deployment gate / E2E passed')),
'QRun-IO/qqq-mcp-rag': ('native-mcp', ('main',), ('mcp-rag',),
('validate / Build and Test',)),
'QRun-IO/qqq-app-developer-intellij-plugin': ('intellij-plugin', ('main',), ('intellij-plugin',),
('plugin / build',)),
}
PROFILES = {definition[0] for definition in DEFINITIONS.values()}
def gates(profile):
names = next(definition[3] for definition in DEFINITIONS.values() if definition[0] == profile)
return [{'context': name, 'app_id': 15368} for name in names]
def caller(kind, pin):
if kind == 'qctl':
return '.github/workflows/build.yml', load_workflow(qctl.generate(pin))
return auxiliary.BRIDGES[kind]['path'], auxiliary.canonical(kind, pin)
def validate(repo, head, branch, request):
if repo not in DEFINITIONS or branch not in DEFINITIONS[repo][1]:
raise ValueError('Native recognition requires an approved repository and branch')
profile, _, kinds, _ = DEFINITIONS[repo]
def content(owner, path, ref):
item = request(f'repos/{owner}/contents/{path}?ref={ref}')
if item.get('type') != 'file' or item.get('encoding') != 'base64':
raise ValueError('Native workflow must be a regular file')
return base64.b64decode(item['content']).decode('utf-8').replace('\r\n', '\n')
paths, pins, verified = set(), {}, set()
for kind in kinds:
path, _ = caller(kind, '0' * 40)
actual = load_workflow(content(repo, path, head))
try:
pin = next(iter(actual['jobs'].values()))['uses'].rsplit('@', 1)[1]
_, expected = caller(kind, pin)
except (KeyError, TypeError, IndexError, ValueError, AttributeError, StopIteration):
raise ValueError('Native caller has no immutable orb pin') from None
if actual != expected:
raise ValueError('Native caller differs from the canonical shared bridge')
if pin not in verified:
commit = request('repos/QRun-IO/qqq-orb/commits/' + pin)
if not commit['commit']['verification']['verified']:
raise ValueError('Native worker commit is not verified')
verified.add(pin)
for job in expected['jobs'].values():
worker = job['uses'].split('@')[0].removeprefix('QRun-IO/qqq-orb/')
if content('QRun-IO/qqq-orb', worker, pin) != (ROOT / worker).read_text(encoding='utf-8'):
raise ValueError('Pinned native worker differs from the supported contract')
paths.add(path)
pins[path] = pin
tree = request(f'repos/{repo}/git/trees/{head}?recursive=1')
if tree.get('truncated') is not False or not isinstance(tree.get('tree'), list):
raise ValueError('Native recognition requires the complete source tree')
entries = {entry['path']: entry for entry in tree['tree']}
for path in paths:
entry = entries.get(path, {})
if entry.get('type') != 'blob' or entry.get('mode') not in ('100644', '100755'):
raise ValueError('Native caller is missing or is not a regular source file')
for entry in tree['tree']:
path = entry['path']
if path == '.circleci/config.yml' or (path.startswith('.github/workflows/')
and path.endswith(('.yml', '.yaml')) and path not in paths):
raise ValueError('Native repository has additional orchestration: ' + path)
return profile, pins
+80
View File
@@ -1,4 +1,68 @@
{
"QRun-IO/new-qqq-application-template": {
"baseline_sha": "730a836a666535a361927d83f15720e91eb71541",
"required_paths": [
"README.md"
],
"forbidden_prefixes": [
".circleci/",
".github/workflows/"
],
"forbidden_paths": [
".qqq-ci.yml",
"Jenkinsfile",
".gitlab-ci.yml"
],
"rationale": "Template source; core generated-consumer acceptance owns validation."
},
"QRun-IO/voyage-cd": {
"baseline_sha": "f8734e2d30b59af4909bd0a4c39fa6c9d6981d61",
"required_paths": [
"README.md"
],
"forbidden_prefixes": [
".circleci/",
".github/workflows/"
],
"forbidden_paths": [
".qqq-ci.yml",
"Jenkinsfile",
".gitlab-ci.yml"
],
"rationale": "GitOps deployment declarations; audited repository has no CI."
},
"QRun-IO/qqq-templates-hub": {
"baseline_sha": "bd591168c7200ce40a3535f2052b8327056e82e0",
"required_paths": [
"templates.yaml"
],
"forbidden_prefixes": [
".circleci/",
".github/workflows/"
],
"forbidden_paths": [
".qqq-ci.yml",
"Jenkinsfile",
".gitlab-ci.yml"
],
"rationale": "Template catalog, no standalone CI in approved snapshot."
},
"QRun-IO/qqq-meta-manager": {
"baseline_sha": "da17eb306bf1587cbf98a1555499e9b27f4caa50",
"required_paths": [
"README.md"
],
"forbidden_prefixes": [
".circleci/",
".github/workflows/"
],
"forbidden_paths": [
".qqq-ci.yml",
"Jenkinsfile",
".gitlab-ci.yml"
],
"rationale": "Metadata and automation knowledge repository; no registered CI entrypoint."
},
"QRun-IO/qqq-maven-registry": {
"baseline_sha": "134576e4e701984635105f97c8f52aa1e27205b2",
"required_paths": [
@@ -31,6 +95,22 @@
],
"rationale": "Component placeholder documentation with no executable CI."
},
"QRun-IO/homebrew-qctl": {
"baseline_sha": "8dc185d2172c9cbf14dd5b7f83caf350430a632b",
"required_paths": [
"README.md"
],
"forbidden_prefixes": [
".circleci/",
".github/workflows/"
],
"forbidden_paths": [
".qqq-ci.yml",
"Jenkinsfile",
".gitlab-ci.yml"
],
"rationale": "Distribution tap; qctl publication owns formula updates, no local CI."
},
"QRun-IO/.github-private": {
"baseline_sha": "3eb51cfc419398b2d4f1f6e5ebdeb17f3469ef8a",
"required_paths": [
-240
View File
@@ -1,240 +0,0 @@
#!/usr/bin/env python3
"""Plan remote enrollment and submit signed, repeatable PRs with --apply."""
import argparse
import base64
import hashlib
import json
import os
from pathlib import Path
import re
import subprocess
import tempfile
import enroll_consumers
import native_gitops_gates as gitops
import native_specialist_gates as specialists
import yaml
PATHS = ('.circleci/config.yml', '.qqq-ci.yml')
def command(args, cwd=None):
"""Use existing credentials and signing configuration without interactive prompts."""
env = dict(os.environ, GIT_TERMINAL_PROMPT='0')
result = subprocess.run(args, cwd=cwd, env=env, capture_output=True,
text=True, encoding='utf-8', timeout=120)
if result.returncode:
raise ValueError('Command failed: ' + args[0])
return result.stdout.strip()
def api(path, missing=False):
"""Only an explicit 404 is treated as absence; other API failures stop work."""
result = subprocess.run(['gh', 'api', path], capture_output=True,
text=True, encoding='utf-8', timeout=60)
try:
data = json.loads(result.stdout)
except ValueError:
raise ValueError('Invalid GitHub response') from None
if result.returncode:
if missing and str(data.get('status')) == '404':
return None
raise ValueError('GitHub read unavailable: ' + path.split('?')[0])
return data
def content(repo, path, head):
"""Read a regular declaration at an immutable commit, never at a moving branch."""
item = api(f'repos/{repo}/contents/{path}?ref={head}', missing=True)
if item is None:
return None
if item.get('type') != 'file' or item.get('encoding') != 'base64':
raise ValueError('Declaration is not a regular file')
return base64.b64decode(item['content'])
def mutation(path, payload):
"""Use the existing gh identity; never print credentials or response bodies."""
result = subprocess.run(['gh', 'api', path, '--method', 'POST', '--input', '-'],
input=json.dumps(payload), capture_output=True,
text=True, encoding='utf-8', timeout=60)
if result.returncode:
raise ValueError('GitHub mutation failed; inspect remote state before retrying')
try:
data = json.loads(result.stdout)
except ValueError:
raise ValueError('Invalid GitHub mutation response') from None
if not isinstance(data, dict) or data.get('errors'):
raise ValueError('GitHub mutation did not succeed')
return data
def signed_api_commit(plan, ref):
"""Resume a base-only branch; append atomically without local Git/SSH keys."""
repo, branch, base = plan['repository'], plan['automation_branch'], plan['head']
if ref is None:
mutation(f'repos/{repo}/git/refs', {'ref': 'refs/heads/' + branch, 'sha': base})
elif ref['object']['sha'] != base:
return ref['object']['sha']
if api(f'repos/{repo}/branches/{plan["branch"]}')['commit']['sha'] != base:
raise ValueError('Target branch moved before submission')
additions = [{'path': path, 'contents': base64.b64encode(plan['replacements'][path]).decode('ascii')}
for path in PATHS if plan['originals'][path] != plan['replacements'][path]]
result = mutation('graphql', {
'query': 'mutation($input: CreateCommitOnBranchInput!) { createCommitOnBranch(input: $input) { commit { oid } } }',
'variables': {'input': {
'branch': {'repositoryNameWithOwner': repo, 'branchName': branch},
'expectedHeadOid': base,
'message': {'headline': 'ci: enroll in the shared QQQ orb',
'body': 'Co-Authored-By: Codex <noreply@openai.com>'},
'fileChanges': {'additions': additions}}}})
try:
head = result['data']['createCommitOnBranch']['commit']['oid']
except (KeyError, TypeError):
raise ValueError('GitHub commit response is incomplete') from None
if not isinstance(head, str) or not re.fullmatch(r'[0-9a-f]{40}', head):
raise ValueError('GitHub commit response has no valid SHA')
return head
def inspect(target, version):
"""Resolve observed repository identity and reuse the shared enrollment planner."""
if not isinstance(target, dict) or set(target) != {'repository', 'branch'}:
raise ValueError('Each target requires repository and branch')
repo, branch = target['repository'], target['branch']
if not isinstance(repo, str) or not re.fullmatch(r'[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+', repo):
raise ValueError('Invalid repository')
if branch not in ('main', 'develop'):
raise ValueError('Enrollment targets current main/develop only')
info = api('repos/' + repo)
if info['archived']:
raise ValueError('Archived repository')
head = api(f'repos/{repo}/branches/{branch}')['commit']['sha']
if repo in specialists.DEFINITIONS:
profile, pins = specialists.validate(repo, head, branch, api)
return {**target, 'head': head, 'profile': profile, 'version': version,
'changed': False, 'worker_pins': pins}
if repo in gitops.REPOSITORIES:
pin = gitops.validate(repo, head, branch, api, yaml.safe_load)
return {**target, 'head': head, 'profile': 'native-gitops', 'version': version,
'changed': False, 'worker_pin': pin}
originals = {path: content(repo, path, head) for path in PATHS}
if originals[PATHS[0]] is None:
raise ValueError('No recognized CircleCI entrypoint')
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
(root / '.circleci').mkdir()
for path, data in originals.items():
if data is not None:
(root / path).write_bytes(data)
plan = enroll_consumers.plan({'checkout': str(root), 'default_branch': info['default_branch']}, version)
replacements = {str(path.relative_to(root)).replace('\\', '/'): new for path, _, new in plan['files']}
digest = hashlib.sha256(head.encode() + b''.join(replacements[path] for path in PATHS)).hexdigest()[:20]
return {**target, 'head': head, 'originals': originals, 'replacements': replacements,
'profile': plan['profile'], 'version': version,
'changed': any(originals[path] != replacements[path] for path in PATHS),
'automation_branch': f'automation/qqq-enroll-{branch}-{digest}'}
def verify_submission(plan, head):
"""Accept only a signed, one-parent commit with exactly the planned changes."""
repo = plan['repository']
commit = api(f'repos/{repo}/commits/{head}')
expected = {path for path in PATHS if plan['originals'][path] != plan['replacements'][path]}
if (not commit['commit']['verification']['verified']
or [parent['sha'] for parent in commit['parents']] != [plan['head']]
or {item['filename'] for item in commit['files']} != expected):
raise ValueError('Existing automation branch differs from the signed plan')
for path in PATHS:
if content(repo, path, head) != plan['replacements'][path]:
raise ValueError('Submitted declaration differs from plan')
def apply(plan):
"""Never force push; reuse matching submissions after interrupted executions."""
if not plan['changed']:
return {'state': 'unchanged'}
repo, branch = plan['repository'], plan['automation_branch']
if api(f'repos/{repo}/branches/{plan["branch"]}')['commit']['sha'] != plan['head']:
raise ValueError('Target branch changed after preflight')
ref = api(f'repos/{repo}/git/ref/heads/{branch}', missing=True)
head = signed_api_commit(plan, ref)
verify_submission(plan, head)
pulls = api(f'repos/{repo}/pulls?state=all&head={repo.split("/")[0]}:{branch}&base={plan["branch"]}')
if pulls:
if len(pulls) != 1 or pulls[0]['head']['sha'] != head:
raise ValueError('Ambiguous existing enrollment PR')
return {'state': 'existing-pr', 'head': head, 'url': pulls[0]['html_url']}
body = ('Uses the shared orb enrollment converter and preserves existing publication policy. '
'Only the canonical caller and manifest change. Exact-head hosted checks must pass '
'before merge; branch enforcement follows successful qualification.\n\n'
'Source base: ' + plan['head'])
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / 'body.md'
path.write_text(body, encoding='utf-8')
url = command(['gh', 'pr', 'create', '--repo', repo, '--base', plan['branch'], '--head', branch,
'--title', 'ci: enroll in the shared QQQ orb', '--body-file', str(path)])
return {'state': 'submitted', 'head': head, 'url': url}
def main():
"""Preflight every explicit target before mutations and save resumable results."""
parser = argparse.ArgumentParser(description=__doc__)
source = parser.add_mutually_exclusive_group(required=True)
source.add_argument('--targets', type=Path)
source.add_argument('--organization')
parser.add_argument('--orb-version', required=True)
parser.add_argument('--apply', action='store_true')
parser.add_argument('--merge-ready', action='store_true')
parser.add_argument('--enforce', action='store_true')
parser.add_argument('--circle-settings', action='store_true')
parser.add_argument('--output', type=Path, required=True)
args = parser.parse_args()
if (args.merge_ready or args.enforce) and not args.apply:
parser.error('--merge-ready and --enforce require --apply')
if not re.fullmatch(r'\d+\.\d+(?:\.\d+)?', args.orb_version):
raise ValueError('A stable minor track or exact patch is required')
findings = []
if args.organization:
from discover_enrollment import discover
plans, findings = discover(args.organization, args.orb_version, inspect, api)
else:
targets = json.loads(args.targets.read_text(encoding='utf-8'))
if not isinstance(targets, list) or not targets:
raise ValueError('A nonempty target list is required')
plans = [inspect(target, args.orb_version) for target in targets]
if len({(p['repository'].lower(), p['branch']) for p in plans}) != len(plans):
raise ValueError('Duplicate target')
rows = [{key: value for key, value in p.items() if key not in ('originals', 'replacements')} for p in plans]
report = findings + rows
args.output.write_text(json.dumps(report, indent=2), encoding='utf-8')
if args.apply:
for plan, row in zip(plans, rows):
row['result'] = apply(plan)
args.output.write_text(json.dumps(report, indent=2), encoding='utf-8')
if args.merge_ready or args.enforce:
import finish_enrollment
if args.merge_ready:
row['qualification'] = finish_enrollment.merge_ready(plan, row['result'])
args.output.write_text(json.dumps(report, indent=2), encoding='utf-8')
if args.enforce:
row['enforcement'] = finish_enrollment.enforce_current(plan)
args.output.write_text(json.dumps(report, indent=2), encoding='utf-8')
if args.circle_settings:
import circle_project_settings
settings = []
for repo in sorted({plan['repository'] for plan in plans}):
branches = [plan for plan in plans if plan['repository'] == repo]
settings.append({'repository': repo,
'profiles': sorted({plan['profile'] for plan in branches}),
**circle_project_settings.reconcile_repository(
branches, findings, api, apply=args.apply)})
args.output.with_suffix('.circle-settings.json').write_text(
json.dumps(settings, indent=2), encoding='utf-8')
print(json.dumps({'targets': len(plans), 'changed': sum(p['changed'] for p in plans),
'unresolved': sum(row['state'] == 'unresolved' for row in findings),
'applied': args.apply}))
if __name__ == '__main__':
main()
+2 -2
View File
@@ -56,8 +56,8 @@ def plan(checkout, version):
def rollout(checkouts, version, write=False):
if not re.fullmatch(r'\d+\.\d+(?:\.\d+)?', version):
raise ValueError('Fleet upgrades require a stable minor track or exact patch version')
if not re.fullmatch(r'\d+\.\d+\.\d+', version):
raise ValueError('Fleet upgrades require an explicit stable orb version')
# Validate the entire batch before writing any repository.
plans = [plan(path, version) for path in checkouts]
if len({item['checkout'] for item in plans}) != len(plans):
Loaded 100 of 102 files, more files were not shown because too many files have changed in this diff. Show more